Novel Air Temperature Measurement Using Midwave Hyperspectral Fourier Transform Infrared Imaging in the Carbon Dioxide Absorption Band

: Accurate visualization of air temperature distribution can be useful for various thermal analyses in ﬁelds such as human health and heat transfer of local area. This paper presents a novel approach to measuring air temperature from midwave hyperspectral Fourier transform infrared (FTIR) imaging in the carbon dioxide absorption band (between 4.25–4.35 µ m). In this study, the proposed visual air temperature (VisualAT) measurement is based on the observation that the carbon dioxide band shows zero transmissivity at short distances. Based on analysis of the radiative transfer equation in this band, only the path radiance by air temperature survives. Brightness temperature of the received radiance can provide the raw air temperature and spectral average, followed by a spatial median-mean ﬁlter that can produce ﬁnal air temperature images. Experiment results tested on a database obtained by a midwave extended FTIR system (Telops, Quebec City, QC, Canada) from February to July 2018 show a mean absolute error of 1.25 ◦ K for temperature range of 2.6 − 26.4 ◦ C.


Introduction
How accurately can we measure and visualize air temperature remotely using thermal sensing? Air temperature is an important meteorological factor, which has a wide range of applications in fields like human health [1], virus propagation [2], growth and reproduction of plants [3], climate change [4], and hydrology [5].
Air temperature can be measured in numerous ways, including contact sensors and remote sensors. Contact sensor-based methods include thermistors, thermocouples, and mercury thermometers [6]. Thermistors are metallic devices that undergo predictable changes in resistance in response to changes in temperature. This resistance is measured and converted to a temperature reading in Celsius, Fahrenheit, or Kelvin. A thermocouple consists of two dissimilar electrical conductors forming an electrical junction, which produces a temperature-dependent voltage, and this voltage can be converted to a temperature [7]. A mercury thermometer consists of liquid in a glass rod with a very thin tube in it. Mercury or red-colored alcohol inside the tube expands when the temperature rises. These sensors should be located in the shade to measure air temperature. If the sun shines on the thermometer directly, it heats the liquid and produces an incorrect, higher temperature than the true air temperature. In addition, it needs enough time (at least several minutes for the liquid to expand) to measure outdoor air temperature. Furthermore, it requires hundreds of thousands contact sensors to measure spatial distribution of air temperature.
The thermal remote sensing approaches will use the relationships between land surface temperature (LST) and near surface air temperature. Land surface radiance is measured by thermal infrared sensors mounted on a satellite or an airborne platform, and LST is retrieved via temperature-emissivity separation (TES) [8,9]. Surface air temperature can be estimated by the temperature-vegetation index (TVX), thermodynamics, and the regression method. The TVX is based on the assumption that a thick vegetation canopy can approximate air temperature [10,11]. It can be useful only if there is high vegetation cover. The thermodynamics-based method uses the energy balance between LST and the surface environment, such as air and water [12,13]. This method provides good air temperature measurement, but it requires many parameters as input. The last type is data-based regression between air temperature and LST. There is a linear regression model [14] and a nonlinear model, especially as a deep learning method [15]. Such machine learning models have reported successful air temperature estimation. One recent deep learning-based method, a five-layer deep belief network (DBN), showed promising air temperature estimation by establishing the relationship between ground station air temperature and multi-source data (remote sensing radiance, socioeconomic data, and assimilation data) [15]. Although the deep learning method showed quite accurate air temperature estimation, it requires huge amounts of data to train the multi-layered deep neural architecture.
The above mentioned approaches are not suitable for instantaneously measuring and visualizing air temperature of a viewing area. Using contact sensors requires several minutes and a huge number of sensors to measure spatial air temperature [16]. Non-contact (remote) sensors, such as thermal infrared (TIR) imagers, require LST and regression with huge amounts of training data. Despite being trained correctly, the regression-based approach is sensitive to many spatio-temporal conditions [17]. In addition, this approach is only suitable for aerial-based sensing in satellites or airborne platforms [18].
In this paper, our research focuses on how to measure spatial air temperature of the viewing area and visualize it instantaneously for environment monitoring of the surveillance area. The key idea is to use up-welling information in the carbon dioxide absorption band (4.25−4.35 µm) with a midwave Fourier transform infrared (FTIR) imager. Most temperature estimation research in TES and LST generates up-welling information by using the moderate-resolution atmospheric radiance and transmittance model (MODTRAN) for atmospheric correction (compensation) purposes [19][20][21]. Harig proposed a passive sensing of pollutant clouds by longwave FTIR to find optimal SNR [22]. His work focused on detecting cloud target in ground background not air temperature. In this paper, it is possible to measure and visualize air temperature accurately through careful analysis of upwelling (path) spectral radiance in the proposed visual air temperature (VisualAT).
The contributions from this paper can be summarized as follows. First, VisualAT can measure spatial air temperature accurately (mean absolute error [MAE]: 1.25 K). Second, VisualAT can visualize the distribution of air temperature with a high spatial resolution. Third, it can measure and visualize air temperature instantaneously. Finally, the proposed VisualAT can be used for various outdoor air temperature measurement applications, such as health monitoring, weather monitoring, and thermal surveillance of local area.
The remainder of this paper is organized as follows. Section 2 introduces the marials for FTIR analysis and Section 3 explains the proposed VisualAT method, including the radiative transfer equation. Section 4 analyzes VisualAT for temperature measurement applications, considering a range of environmental changes. The paper concludes in Section 5. Figure 1 presents a measurement scenario in an outdoor environment consisting of a scene and a sensor system. A painted target in front of a sky-and-sea background is 78 m away from the observation laboratory. MWIR hyperspectral images were acquired with the Telops Hyper-Cam MWE model [23]. It can provide calibrated spectral radiance images with a high spatial and spectral resolution from a Michelson interferometer in the short-wave to midwave band (1.5-5.6 µm). The spatial image resolution is 320 × 240, and the spectral resolution is up to 0.25 cm −1 . The noise equivalent spectral radiance (NESR) is 7 [nW/(cm 2 ·sr· cm −1 )], and the radiometric accuracy is approximately 2 K. The field of view is 6.5 × 5.1 deg. The objective of this research is to estimate air temperature spatially and to visualize the temperature distribution.

Outdoor Hyperspectral Data Acquisition System
A cropped hypercube image provides 128 × 200 × 374 (width × height × bands) data. A hyperspectral spectral imager (HSI) database was recorded daily from February to July 2018. Recording was done three times a day (10:30, 13:30, and 15:30 h) and four times an hour for a total of 12 times per day. In addition, an automatic weather station (AWS) recorded environmental information, such as the date, time, air temperature, humidity, air pressure, and visibility. The measured air temperatures were used to evaluate the temperature estimation accuracy by the proposed VisuaAT method.

FTIR Data Acquisition Interface
The selection of the CO 2 absorption band is critical in the proposed VisualAT method. An initial baseline band range was selected by the developed GUI shown in Figure 2. The midwave FTIR spectral image software platform was developed for hypercube image display and spectral profile analysis by varying the wavelength parameter and units. The top left image in Figure 2 represents a spectral image in a specific band, and the lower graph shows a spectral profile at a selected point (indicated by +). The top right image in Figure 2 is a broad-band image with a selected spectral range.

Radiometric Calibration of FTIR Imaging
The wavenumber and radiometric calibration of midwave hyperspectral data should be done accurately to measure air temperature. The technical details are described in [24] and one full set of data including the raw IR spectrum, internal calibration, and temperature extraction is described in the following paragraphs. A Michelson interferometer produces interferomgrams by moving a mirror. Figure 3a presents an interferogram image at optical path difference (OPD) ID 500 (total 1186). Figure 3b gives an example of the whole interferogram at pixel (60, 60). Figure 3c shows the results of spectrum extraction by applying the fast Fourier transform (FFT) to the interferogram (Figure 3b). The unit of the y-axis in Figure 3c is just spectral intensity in arbitrary units. The wavelength calibration is performed using the HeNe laser (wavelength λ = 632.8 nm). Figure 3d shows the wavenumber calibrated spectrum.
Spectral View @ ( | ) Spectral Profile @ (+) Figure 2. Midwave FTIR image analysis software platform. A spectral image at a selected wavelength can be visualized interactively. The next step is to calibrate radiometrically using two blackbodies. The HYPER-CAM MWE can provide the spectral radiance data using two built-in BBs (hot, cold) [25]. Figure 4 shows the measured interferogram image, spectra (arbitrary units), and calculated spectral radiances for the hot (95 • C) and cold (25 • C) blackbodies. Figures 5a,b show the estimated gain and offset magnitude at pixel (60, 60), respectively. Figure 5c,d presents the calculated spectral radiance with the unit of wavenumber and wavelength, respectively.
The amount of spectral radiance energy can be converted into equivalent brightness temperatures [26]. By inverting Equation (7), the temperature T (K) can be obtained as . (1)

MODTRAN Simulator
The moderate-resolution atmospheric radiance and transmittance model (MODTRAN, http://modtran.spectral.com/) is used in this paper to simulate atmospheric transmittance and path thermal calculation. Figure 6 shows a GUI interface to simulate spectral path radiance by setting geometric and atmospheric parameters.     Figure 7 shows the air temperature measurement scenario. Air temperature from VisualAT measurement can be derived from radiative transfer Equation (2). We adopt the radiative transfer equation used in MODTRAN [27]. In general, at-sensor received radiance in the midwave infrared (MWIR) region consists of opaque object-emitted radiance, reflected downwelling radiance, and total atmospheric path radiance (thermal+solar components). L obs (λ) is the at-sensor radiance; λ is wavelenth; ε(λ) is spectral object surface emissivity; L obj (λ, T obj ) is the spectral radiance of the object, assuming a blackbody in the Planck function with object surface temperature (T obj ). L ↓ s (λ) and L ↓ t (λ) represent spectral downwelling solar radiance and thermal irradiance, respectively; τ(λ) is the spectral atmospheric transmittance, and L ↑ s (λ) and L ↑ t (λ) are the spectral upwelling solar and thermal path radiance, respectively, reaching the sensor.

Derivation of Radiative Ttransfer Equation
According to the MODTRAN simulation in the MWIR band, the spectral transmittance of the carbon dioxide (CO 2 ) band (4.25-4.35 µm) decreases abruptly with distance, as shown in Figure 8. The average transmittance in the CO 2 band is 0.5, 0.13, 0.03, 0.005, 0.0001, and 0 at 1 m, 5 m, 10 m, 20 m, 50 m, and 100 m, respectively. If we consider only the CO 2 band (λ CO 2 ) with a minimum 20 m object distance, the transmittance (τ(λ CO 2 )) can be regarded as 0, which leads to Equation (3). An MWIR FTIR camera receives only the upwelling of path solar and thermal radiances in the λ CO 2 band where the range is normally 4.25-4.35 µm.
According to the MWIR radiometric characteristics [28], the contribution of solar radiance (L ↑ s (λ CO 2 )) from air scattering is very small, even for very dry conditions (less than 2% at 5 µm) [28]. Ignoring the first term, we can simplify Equation (3) into Equation (4): The definition of thermal upwelling is defined with Equation (5): Since the spectral transmittance in the CO 2 band is 0 (τ(λ CO 2 ) = 0), the final form is approximated in Equation (6): where B(λ CO 2 , T air ) denotes the spectral radiance ([W/(m 2 ·sr·µm)]) of a blackbody (Planck's law [29]), and T air is the air temperature in degrees Kelvin [K] of the atmosphere between the object and the camera sensor. The spectral radiation of atmosphere is modeled as blackbody [30][31][32]. Atmospheric path radiance can be described in difference methods, but the simplest way is to model the particles as blackbodies [32]. B(λ CO 2 , T air ) is defined with Equation (7): where h denotes Planck's constant, c is the speed of light, and k is the Boltzmann constant. The amount of spectral radiance energy can be converted into equivalent brightness temperatures ( [33]). By inverting Equation (7), temperature T air [K] can be obtained as follows: 3.2. VisualAT: Proposed Visual Air Temperature Measurement Figure 9 summarizes the overall processing flow of the proposed VisualAT method. The first row represents the three steps in spectral brightness air temperature extraction using Equation (8), described as follows.
The second row of Figure 9 represents the image processing for visual air temperature image generation, explained in the following steps. (4) A raw temperature image is extracted via pixel-wise temperature mean filter along the spectral axis. It still shows a noise-like image consisting of salt-and-pepper noise and thermal noise. This can be removed by consecutive spatial 2D median filtering and Gaussian filtering [34]. The Gaussian filtering is adopted to reduce spatial thermal noise. The empirically tuned kernel size of the median filter is 10 × 15, and sigma of the Gaussian filter is set to 2.
(1) MWIR HSI (spectral radiance) (2) CO 2 Band Selector  Figure 10a shows the spectral brightness temperature by applying Equation (1) to the calibrated spectral radiance at pixel (60, 60). Figure 10b shows the enlarged brightness temperature around CO 2 band (4.29-4.34 µm). Figure 10c represents the spatial raw temperature distribution of the CO 2 band image. Final visual air temperature image (Figure 10e) is acquired through the median and Gaussian filtering processes.

Analysis of Air Temperature Measurement
The proof of radiometric air temperature measurement and visualization is possible by mathematical derivations as Equations (2)- (8). However, it is a challenging problem to validate the air temperature measurement and visualization experimentally because we need a huge dark room with controllable air temperature. In this subsection, we analyze the properties of the VisualAT method by demonstrating extreme air temperature measurement and by thermal air flow visualization. Figure 11 demonstrates the air temperature measurement and visualization using the hot summer and cold winter data set. Upper row images represent visual temperature extraction process and camera's internal temperature information for hot summer data. The ground truth of air temperature is 26.4 • C and the estimated temperature is 25.6 • C. Likewise, lower row images represent the same process for cold winter data. Note that the ground truth of air temperature is 2.6 • C and the estimated temperature is 2.52 • C. The internal camera temperatures of IR lens, front wall, beam splitter, and etc are approximately 29 • C and they do not affect to the air temperature estimation because hot/cold blackbody-based radiometric calibration can remove the effect of stray light before each measurement. The proposed VisualAT can visualize thermal air flow using consecutive FTIR hyper-cubes acquired in very short period (1 s). Figure 12 shows the air flow directions indicated by the curves and arrows. Because the sea wind is so strong, the thermal air flow changes dynamically in very short time. Therefore, this result can be another indirect proof of imaging variations in air temperature.

Signal Analysis of Air Temperature Monitoring
It is important to analyze how the IR radiation from different parts of the probed air contributes to the received thermal signal. Figure 13 represents the geometric relationship between atmosphere plane and a pixel. If an atmospheric plane at distance R is considered, the probing area (A) is a × b using the instantaneous field of view (IFOV). The received thermal flux (Φ λ ) at the pixel detector where Ω is solid angle and τ o is lens transmittance [35]. Because air particles are regarded as blackbodies [32], the emissivity is regarded as 1. If we use basic geometrical relationships, the final form is changed to Φ λ = (1 − τ(λ CO 2 )) · L λ (T air ) · A IFOV · Ω IFOV · τ o where A IFOV denotes the pixel area and Ω IFOV represents the solid angle of IFOV. The received thermal flux is strongly related to (1 − τ(λ CO 2 )). The simulation of atmospheric transmittance is conducted by MODTRAN 4.0 and Figure 14a represents the result. As the distance increases up to 20 m, the thermal contribution of air particles increases. This analysis is confirmed through the MODTRAN-based path thermal simulation as shown in Figure 14b and Figure 6. Therefore, we can conclude that the received air flux is contributed dominantly by the air temperature at 20 m distance.

Front Optics
Atmosphere Plane

Performance Metric
The mean absolute error (MAE) metric was used to compute the performance of the VisualAT method in predicting air temperature. The MAE metric is defined in Equation (9): where T k air denotes the k − th predicted air temperature by the VisualAT method, and T k GT is the corresponding air temperature measured by the AWS, as shown in Figure 15.  Figure 16 shows the experimental environment. Figure 16a is the outdoor environment acquired via visible band camera, and Figure 16b shows a recorded broad-band image from summing all the spectral band images (1.5-5.6 µm). Figure 16c represents MODTRAN-based spectral transmittance at the object distance of 78 m, and where the average transmittance of the CO 2 band is 0.0001.

Parameter Analysis
A midwave hyperspectral image database was prepared for various evaluations. Hypercube images from 49 days were valid during the acquisition period (February to July 2018). In the first evaluation, the effect of the CO 2 band range was important in order to estimate air temperature accurately. As summarized in Figure 17, baseline bands were selected from a visual inspection using the GUI shown in Figure 2. Top row images in Figure 17 show the spectral images corresponding to specific wavelengths. The visual selection criterion was whether the spectral image looks like it has noise in the whole image area. If atmospheric transmittance is 0, the received radiance consists of thermal path (air) radiance. Therefore, the initially selected bands were 4.22, 4.27, 4.29, 4.31, and 4.34 µm. The MAE of the baseline band showed 1.32 K. If a lower band decreased to 4.20 µm, the MAE decreased to 1.29 K. If the lower band increased to 4.27, 4.29, and 4.31 µm individually, the corresponding MAEs were 1.26, 1.25, and 1.27 K, respectively. From these experiments, the lower band limit can be set at 4.29µm. On the other hand, if we increase the upper band limit to 4.36 µm with the selected lower band limit at 4.29 µm, the MAE increased to 1.29 K. Therefore, we can conclude that the optimal CO 2 absorption bands are 4.29, 4.31, and 4.34 µm. These bands were used in the following experiments.  Figure 17. CO 2 band range selection results: Baseline band is selected by visual inspection and the optimal band range is selected by maximizing mean absolute error (MAE) metric.
SNR can be improved additionally through the 2D median filter and 2D Gaussian filter. The 2D median filter is necessary to remove dead pixels as shown in the top-left of Figure 18 where the effects of median filter sizes from [1 1] to [15 15] were visualized. Even [3 3] median filter can remove the salt and pepper noise effectively and larger filter size can extract larger structure of air temperature distribution. The histogram distribution after the 2D median filter is Gaussian distribution, which is consensus to thermal noise distribution. True temperature signal with Gaussian noise can be estimated by Gaussian smoothing filter (unbiased, consistent linear estimator) [36]. The effects of sigma (σ) were displayed in Figure 19 where [3 3] median filter was used initially. Note that larger σ can extract larger structure of air temperature distribution. The selection of median filter size and Gaussian filter parameter depends on application images. If salt and pepper noise is strong, larger median filter size with smaller Gaussian smoothing is suitable. If the salt and pepper noise is weak, smaller median filter size with stronger Gaussian smoothing is recommended. In this paper, we used a kernel size of 10 × 15 for the median filter and σ = 2 for the Gaussian filter because salt and pepper noise is spread horizontally and thermal noise is high.
The VisualAT method depends on the spectral radiance of atmosphere. The sensitivity analysis and error propagation can be useful to understand the properties of VisualAT. The simulation is conducted by adding spectral radiance noise to Equation (7) and estimating air temperature from Equation (8). Figure 20 shows the sensitivity analysis results. The ground truth air temperature is 20 • C and it increases linearly according to the spectral noise level. In the case of FTIR camera noise (NESR, 7 × 10 −5 [W/m 2 ,sr,cm −1 ]), the estimated temperature is 21.08 • C.

Experiment Results
Radiometric accuracy should be evaluated to validate the usefulness of the proposed VisualAT method. Although VisualAT can present measured temperature spatially, a global mean temperature per image was extracted to compare with the ground truth air temperature recorded by the AWS. Figure 21 shows a quantitative evaluation graph between the air temperature estimated by the proposed VisualAT method and the ground truth air temperature for the 49-day dataset (acquired time: 15:30 h). The MAE was 1.25 K, which is accurate considering the remote sensing method and dynamic weather changes. The correlation coefficient between the VisualAT-based air temperature and ground truth air temperature was 0.97. Figure 22 shows representative visual air temperature images measured using the VisualAT method. To the best of our knowledge, there is no method by which to compare these results to others, and this is the first trial of instantaneous ground air temperature measurement and visualization in a ground-based approach.  The dependency of temperature estimation accuracy from the proposed VisualAT is important in order to check robustness to various environmental changes. Figure 23 summarizes  . The correlation coefficient (R) is annotated on the results. According to the results, the estimation error is negative relationship with air temperature, air pressure, and visibility. In addition, the error has positive relationships with humidity. This means that a more accurate temperature can be estimated if the humidity is lower. On the other hand, the long-wave thermal radiation had no specific relationship. The dependency analysis is for reference only because the data points are very scattered.
The best advantage of the proposed VisualAT method is that it can measure visual air temperature instantly. It takes approximately one second to scan a hypercube. Figure 24 shows consecutive air temperature images acquired at 20-min intervals on 6 March 2018. The color axis was set by the caxis ([7.83 9.83]) function in MATLAB for a fair temperature reading. We can see the flow of the heat flux over one hour.   Figure 24. VisualAT-based dynamic temperature images measured at different times (20-min. intervals): Temporal variation of air temperature distribution can be found. Figure 25 shows the visual air temperature images measured at night (21:29 h) and the next early morning (09:23 h). There was strong sea fog, and VisualAT can still produce air temperature images.  Figure 26 shows VisualAT-based remote air temperature measurement and visualization in a sea environment. Top left shows a broad-band image integrating a 1.5-5.6 µm hypercube; Top right is the corresponding visible band image. The bottom image is the measured air temperature distribution from the VisualAT method. The average temperature was 16.73 • C, and relatively hot and cold regions can be found. Figure 27 presents various air temperature visualization results in maritime environment. Different air temperature distributions can be found.
Through various analysis and evaluation, the proposed VisualAT can be a novel method to measure spatial air temperature and visualize its distribution instantly. The CO 2 concentration affects to the atmospheric transmittance [37], which is related with measurement air volume. It can be a useful approach if CO 2 band image is available but there are limitations such as expensive sensor system and relatively small measurement air volume (20 m distance times FOV). If there is an object in the measurement volume, the accuracy of air temperature measurement can be degraded. Partial artifact can appear when the assumption of zero atmospheric transmittance is broken. If radiometric calibration is not perfect, then internal camera device temperature can distort air temperature distribution.

Discussions and Conclusions
It is very challenging to measure air temperature contactlessly and instantly. This paper proposed a novel air temperature measurement and visualization method called VisualAT, which uses a midwave FTIR in the carbon dioxide absorption band. We found that spatial air temperature can be measured radiometrically by deriving the radiative transfer equation (RTE). The first physical atmospheric property is that atmospheric transmittance in the CO 2 band (4.25-4.35 µm) is 0.005 (at 20 m away), which can remove the effect of object and downwelling radiation. The second physical atmospheric property is that the portion of solar upwelling is very small in the CO 2 band. These two properties lead to a simpler received radiance at the sensor; it is just blackbody radiance of the air temperature.