Temperature Estimation Adaptive to Variables over Distance Using Infrared–LiDAR

: Measuring accurate surface temperature using a long-wave infrared camera and a non-contact thermometer, is very difﬁcult due to variables such as atmospheric transmittance, emissivity, and inﬂuences from the environment such as atmosphere, sun, and dust. Conventional approaches use geometric correction or atmospheric transmittance modeling for temperature correction. How-ever, these approaches have limitations in ﬁnding an accurate temperature because it is difﬁcult to fully model a physical phenomenon. In this paper, a new temperature estimation method using distance information of LiDAR and digital count of long-wave infrared camera is proposed. The proposed method estimates the temperature by redeﬁning the mapping function between radiation and digital count by distance. Using the proposed method, if the digital count is measured at a speciﬁc distance, accurate temperature can be estimated through the redeﬁned Radiation-Digital count mapping function at a speciﬁc distance. The most important property of proposed method is that complex physical modeling is complemented by mapping function of speciﬁc distances. In addition, digital counts that change according to the distance at the same temperature required for the mapping function are obtained through linear interpolation using digital count of speciﬁc distances. Experimental results using a blackbody, long-wave infrared camera and LiDAR verify that the proposed method estimates the precise temperature. In addition, through experiments on humans, it shows the possibility of accurate body temperature measurement through fusion of long-wave infrared cameras and LiDAR in the future. However, as a limitation, a new calibration is required when the temperature and humidity of the atmosphere change.


Introduction
Temperature measurement and heat sensing are widely used in various fields, such as military and industrial applications and medical health [1][2][3][4][5][6][7][8]. For temperature measurement, there are two main methods. One is a non-contact thermometer that measures temperature without contact, so it is mainly used for industrial high temperature measurement. The other is a contact thermometer that is primarily used when accurate temperature measurements are required, such as body temperature measurements. However, due to the risk of infection, it is increasingly preferred to use a non-contact thermometers to measure body temperature [9]. In particular, due to the recent spread of the COVID-19 epidemic [10][11][12][13], body temperature measurements using infrared cameras, a non-contact thermometer, is essential in crowded places such as airports and event venues. Therefore, accurate temperature measurement using infrared cameras has become an important challenge.
In long wave infrared cameras, the digital count is obtained through radiation. Therefore, the temperature is estimated through the obtained digital count. In fact, the model for measuring the radiation of an object with a long-wave infrared camera is shown in Equation (1). L obs (λ) = τ(λ) ε(λ)L obj (λ, T obj ) + (1 − ε(λ))(L ↓ t + L ↓ s ) +L s (λ) +L t (λ) (1) L obs (λ) is the observed at-sensor object radiation; λ is wavelenth; ε(λ) is spectral emissivity of object; L obj (λ, T obj ) is the spectral radiation of the object, assuming a blackbody in the Planck function with the object temperature (T obj ); τ(λ) is the spectral atmospheric transmittance, andL s (λ) andL t (λ) are the diffuse solar and thermal path radiation, respectively, reaching the sensor. Figure 1 intuitively shows Equation (1). Radiation actually measured is affected by atmospheric and solar noise and atmospheric transmittance according to distance. In fact, according to [14], visible radiation from the sun can be neglected from the mid-wave infrared band (4.2-5.6 µm). Likewise, the 8-14 µm band range in which the long wave infrared camera works, thermal radiation is important and visible radiation from the sun is negligible. Furthermore, emissivity is a very important factor in temperature estimation. Emissivity is the ratio of radiation emitted from a blackbody at the same temperature as the radiation emitted from the surface of an object. The emissivity of an ideal blackbody is 1, and human skin is mainly calculated as 0.98. In previous studies, temperature measurements attempted to estimate the temperature by calibrating for various variables. In [15], a method for estimating temperature by compensating for the size-of-source effect is proposed. In [16], the effect of the angle of view on the temperature measurement was analyzed using an infrared camera and an infrared radiator. In [17], it attempts to estimate precise temperature based on the nonlinear response of individual pixels. Ref. [18] proposes a method to reduce the influence of atmospheric dust in temperature measurement. In [19], analyzes the effect on atmospheric transmittance, and based on this, [20] proposes a method to reduce the influence of atmospheric transmittance in infrared cameras. In [21], the temperature is estimated as compensation for each effect by analyzing the object temperature, environmental temperature, and atmospheric transmittance according to the distance.
As mentioned above, conventional approaches to temperature estimation have attempted to estimate accurate temperature by means of geometric compensation, compensation for atmospheric transmittance, and compensation for the surrounding environment such as atmosphere, sun, and dust. However, there is a limit to estimating accurate temperature by considering various variables with only the measured digital count.
In this paper, we propose a new method for accurate temperature estimation using long-wave infrared cameras and LiDAR. Unlike previous studies, distance information using LiDAR is additionally required to use the proposed method. The proposed method first obtains a digital count for a specific distance and builds a Radiation-Digital count mapping function for each distance through this. The mapping function constructed by distance reflects the effects of various variables (atmospheric and solar Radiation, emissivity, atmospheric transmittance, etc.) by distance. It is used to estimate the exact temperature by obtaining the distance to the object and the digital count for the object. If an object exists at a specific distance where the mapping function is not established, it is solved through linear interpolation. However, the proposed method also has limitations. In fact, the effects of CO 2 absorption and water vapor are large in measurements made by infrared cameras. Therefore, the digital count is affected a lot by the air temperature and water vapor between the object and the camera. Therefore, if these conditions change, there is a limitation that recalibration is required.
The contribution of this paper can be summarized as follows: (1) We analyze Radiation-Digital count mapping and blackbody radiation in the longwave infrared band. (2) We propose a method to estimate accurate temperature through Temperature-Radiation and Radiation-Digital mapping by distance. (3) We verify that the proposed method accurately estimates the temperature of a blackbody over distance, and through body temperature experiments, we establish the possibility of accurate body temperature measurement with future LiDAR-Infrared camera fusion.
The remainder of this paper is organized as follows. Section 2 explains the Planck's law and radiation-digital counts mapping function as background knowledge. Section 3 explains the proposed method, a mapping function for each distance, and a mapping function using interpolation. Section 4 evaluates the accuracy of the proposed method for each distance through the temperature of the blackbody, and shows that it is possible to accurately estimate the temperature by fusion of LiDAR-Infrared cameras in the future through the experiment of measuring body temperature according to the distance. The paper concludes in Section 5.

Background
In this section, before describing the proposed method, we will examine the Planck's law [22][23][24] and Radiation-Digital count mapping, which are the background.

Planck's Law
Before Planck's Law in 1900, blackbody radiation consisted of two theories [25]. First, in 1879, with Stefan's experimental [26] discovery of the relationship between radiant energy and temperature, then in 1884, Boltzmann theoretically summarized Stefan's experiment [27]. The Stefan-Boltzmann law states that the sum of the energy of all wavelengths emitted per unit time in a unit area of a blackbody at temperature T is proportional to the absolute temperature to the fourth power. Equation (2) is the Stefan-Boltzmann law. σ is a Stefan-Boltzmann constant, σ = 5.6696 × 10 −8 W/m 2 ·K 4 . L(T) = σT 4 (2) Next, in 1893, Wilhelm Wien summarized that absolute temperature is inversely proportional to the wavelength at which the energy density is at its maximum [28]. Wien's displacement law is Equation (3). From the Planck curve in Figure 2a, it can be seen that the lower the temperature, the longer the wavelength at the maximum energy. b is a Wien's constant, b = 2.898 × 10 −3 m·K.
In 1900, Planck organized the two previous theories [22][23][24] as Planck's law. The density of radiant energy emitted at a specific temperature and specific wavelength of an ideal blackbody can be obtained through Planck's law Equation (4).
T is the absolute temperature of the blackbody, k is the Boltzmann constant, k = 1.38 × 10 −16 erg/K; h is the Planck constant, h = 6.625 × 10 −27 erg-sec; c is the speed of light, c = 3 × 10 8 m/s. Radiation measured at a specific temperature is obtained by integrating the wavelength band that can be measured with an infrared camera. If the infrared camera can measure from λ 1 to λ 2 , you can obtain the radiation corresponding to the temperature by Equation (5).
Furthermore, from Figure 2b, it can be seen that the wavelength at the maximum energy at room temperature is actually located in the long-wave infrared band (8-14 µm). Therefore, it can be seen that the long-wave infrared camera is suitable for room temperature measurement.

Radiation-Digital Count Mapping
The method of estimating temperature by mapping radiation and digital count obtained from infrared camera was used in [29][30][31][32]. This method of estimating the temperature first calculates the radiation of the temperature range (T min − T max ) to be measured through Equation (5). In this case, the wavelength range (λ 1 − λ 2 ) is applied to the range that the infrared camera used can measure. Thereafter, as shown in Figure 3, the digital count (G min − G max ) for T min and T max obtained through the blackbody and infrared camera and the previously calculated radiation (L min − L max ) mapping function is obtained. After that, when a specific digital count is obtained through measurement, the radiation is obtained through the mapping function, and the corresponding temperature is estimated by the radiation. For more information, see Figure 4 and the Equations (5)- (8). The Radiation-Digital Count mapping function is expressed in Equation (6) using the relationship curve between the radiation and digital count according to the temperature in Figure 4.
where a is the slope of the mapping function, obtained by Equation (7): Therefore, you can get Equation (6) to obtain a specific L(G obj ) through Equation (8): When the digital count is measured, the radiation is obtained through Equation (8), and the temperature of the object (T obj ) is determined through the value of T that makes L(G obj ) = L(T) from L(T) previously calculated by Planck's law.

Proposed Method
The proposed temperature estimation process consists largely of three steps. In simple terms, the STEP 1,2 offline process is the process of obtaining distance-specific data through infrared camera and LiDAR in advance. The STEP 3 online process is the process of estimating the temperature through the distance-specific mapping function of the radiationdigital count through the acquired data and the digital count and distance of the measured object. Figure 5 is the flow of the whole process of the proposed method. To estimate the temperature, you can first perform the offline process of steps 1 and 2 at least once, and then continue to estimate the temperature through the online process of 3 steps. This section details the three steps.

Offline: Data Acquisition and Interpolation
An offline process is required before the process of estimating the temperature. In this process, digital counts for each distance obtained through an infrared camera-LiDAR and radiation according to Planck's law are used to make radiation-digital count mapping function adaptable to a variable according to distance. Offline process consists of "STEP 1: Data Acquisition" and "STEP 2: Digital Count Interpolation" processes.
The "STEP 1: Data Acquisition" process involves obtaining a digital count for each distance and calculating the radiation through Planck's law. As shown in Figure 6, the digital count obtains the highest temperature digital count (G max ) and the lowest temperature digital count (G min ) for a specific distance in LiDAR over a range of temperatures to be measured with a blackbody and infrared camera. The reason why it is important to obtain a digital count for each distance is that it is possible to obtain a digital count value that is adapted to the influence of variables such as atmospheric transmittance according to the distance. Figure 7 shows the digital count value that decreases due to the influence of the atmosphere at the same temperature over distance. G max and G min measured by this process are used in the interpolation of STEP 2. Furthermore, the radiation calculation process obtains the radiation of each temperature through Planck's law in the temperature range to be measured. The calculated radiation (L(T)) is then used to obtain the Radiation-Digital Count mapping function for temperature estimation in STEP 3. However, it is not possible to measure the digital count for all distances in advance. In order to solve this problem, the "STEP 2: Digital Count Interpolation" process obtains G min (x) and G max (x) for the distance by linear interpolation of G min and G max acquired by distance, as shown in Figure 8. As such, the linear interpolation of the digital count according to the distance is equivalent to the following Equations (9)- (11).  The sets of Equation (9) mean the distance obtained in STEP 1 and the corresponding G min and G max . For example, the minimum and maximum digital counts measured at distance d 0 are G 0 min and G 0 max . Equations (10) and (11) are interpolation equations used to obtain the digital count at an unmeasured distance. Through this, when measuring an object in STEP 3, as shown in Figure 9, the mapping function according to the distance can be newly defined by newly setting the G min and G max values according to the distance obtained from LiDAR. The Radiation-Digital count mapping function according to the distance is shown in Figure 10. Therefore, precise temperature estimation is possible through a mapping function that adapts to various variables according to distance.

Online: Temperature Estimation Using Infrared-LiDAR
After going through the above offline process at least once, the temperature can still be estimated using the online process. In this process, distance and digital count are obtained through LiDAR and Infrared camera for an object, and temperature is estimated by configuring a mapping function.
The "STEP 3: Object Measurement" process uses LiDAR and infrared cameras to measure objects as shown in Figure 11. The distance to the object can be obtained using LiDAR, and the digital count (G obj ) by the radiation of the object can be obtained through the infrared camera. The distance(d) obtained through LiDAR is given as input to the linear interpolation functions G min (x) and G max (x) previously defined in STEP 2 to obtain the minimum and maximum digital counts (G min , G max ) according to a specific distance. Afterwards, the radiation (L(T max ), L(T min )) calculated in STEP1 and the digital count (G max , G min ) of a specific distance obtained through the interpolation function allows the Radiation-Digital count mapping function to be newly defined. L(G obj ) can be obtained by applying the digital count (G obj ) measured by the infrared camera to a newly defined mapping function, and the value of T that satisfies the pre-calculated L(T) = L(G obj ) is estimated as the object temperature T obj .

Experimental Results
This section describes equipment for experimentation and data set acquisition scenarios. It also shows the performance of the proposed method and confirms the possibility of LiDAR-Infrared camera fusion in the future through a body temperature measurement experiment.

Equipment
Equipment used in the experiment includes infrared camera, LiDAR, and blackbody. Infrared cameras are used to get a 16-bit digital count according to radiation, and LiDAR is used to get the distance to an object. Furthermore, the blackbody is used to emit the exact radiant energy according to the temperature of T min and T max .
Infrared camera is a (Figure 12a) FLIR T620, and its main specifications are as Table 1. In particular, the spectral range is 7.8-14 µm. Therefore, in STEP 1 of Figure 5, it is set as λ min = 7.8 µm, λ max = 14 µm. Therefore, T max at T min radiation can be calculated by integrating Planck's law over the [λ min , λ max ] interval. LiDAR is a (Figure 12b) Velodyne VLP-16, and its main specifications are shown in Table 2. LIDAR is a sensor that measures the distance through the reflection of a laser on an object. The distance to an object can be obtained as a distance (Equation (12)) through x, y, z coordinates obtained through LiDAR. distance = x 2 + y 2 + z 2 (12)  The blackbody is shown in Figure 12c, and the main specifications are shown in Table 3. The blackbody emits accurate radiation at a specific temperature, and in STEP 3: Measurements of the Object, the temperature set for the blackbody is used as the ground truth to measure its performance. In practice, the emissivity of an ideal blackbody is 1, but due to hardware limitations, the emissivity of the blackbody used is about 0.96. Therefore, when calculating the radiation, it must be multiplied by 0.96. The emissivity of the skin of a person is about 0.98. Therefore, in the experiment for measuring body temperature, the radiation is multiplied by 0.98. As for the data acquisition environment, as shown in Figure 13a, the digital count and distance of the blackbody were measured using the infrared camera and LiDAR together. Figure 13b shows the LiDAR map. The part marked by the red dot in the center is the location of LiDAR. The distance to the actual object is obtained by applying x, y, z points to Equation (12).
Datasets set the temperature of the blackbody at T min = 25 • C and T max = 45 • C, and measured T min and T max in 1 m increments, excluding 0.3 m from about 0.3 m-10 m distance. Since the proposed method uses interpolation, the proposed method can be used if there are data about T min and T max at appropriate distances. Figure 14 is the thermal image of the blackbody by distance obtained through the infrared camera. Furthermore, to measure the performance of the proposed method, a digital count by distance was obtained for the 5°C interval temperature between T min and T max . In addition, additional data on the blackbody at distances of about 3.5 m and 7.5 m were collected. In addition, digital counts for each distance were obtained for humans to ensure that the same temperature estimates were made at each distance for body temperature. Thus, the total acquisition data set was acquired at 5 temperatures at 5°C intervals from 25°C to 45°C and 11 points at 1 m intervals of approximately 0.3 m to 10 m. It was also acquired at two points, 3.5 m and 7.5 m. Thus, 65 data for black bodies and 11 data for humans were obtained.

Experiment: Performance of the Proposed Method
In this section, temperature estimation for a specific distance is compared through three methods. The first method is the estimation result before temperature correction with a single mapping function at a close distance, the second method is the result of temperature correction through various variables provided by FLIR, and the third is the result through the proposed method. Temperature correction via variables provided by FLIR can be done in FLIR Tools [33] with temperature correction via emissivity, distance, atmospheric temprature, relative humidity, etc. Figure 15 can be obtained through the Tables 4-6, which are the results of temperature estimation through each method. It can be seen that the temperature estimation result before correction in Table 4 decreases as the distance increases due to the variable according to the distance. Furthermore, with the FLIR correction in Table 5, it is difficult to estimate the exact temperature because values of various variables are required. It can be seen from Table 6 that the proposed method of obtaining the distance through LiDAR and redefining the mapping function for each distance can accurately estimate the temperature at most distances.  Table 7 shows the estimated results for each temperature at a distance (3.5 m, 7.5 m) not previously measured. In the proposed method, for distances not measured in advance, the temperature is estimated by obtaining a digital count using interpolation and defining a new mapping function. In particular, in this result, it can be seen that the temperature estimation error of the proposed method is low through Mean Square Error (MSE).

Experiment: Temperature Estimation for a Person
In body temperature measurement, since the exact value of the surface temperature of a person cannot be known, body temperature is estimated similarly at each distance, showing that the proposed method is also suitable for body temperature measurement. It also shows the possibility of estimating body temperature through fusion of infrared cameras and LiDAR in the future. Figure 16 is an image of a person obtained from an infrared camera by distance. Digital count is acquired from the person face area and used for temperature estimation. Furthermore, Figure 17 shows the LiDAR map at 6 m. Distance to a person can be obtained obtained through LiDAR (x, y, z) by distance. Unlike a blackbody, for person measurements, since the exact temperature value cannot be known, the estimated temperature for each distance is similar, confirming the validity of the proposed method. In addition, when estimating the temperature for an object other than a blackbody, the emissivity is an important factor and should be reflected in the radiation. Therefore, in the experiment, the human emissivity is set to 0.98, and the radiation is obtained through L(G obj ) * emissivity o f blackbody emissivity o f object and T is estimated through this. As a result of estimating the temperature by defining a new mapping function by obtaining the distance to a person through LiDAR and obtaining a digital count through it, it can be seen that similar temperature for each distance is estimated as shown in Table 8.   Through the proposed method and the fusion of LiDAR and IR cameras, it will be possible to quickly and accurately screen people with fever from the general public by checking the body temperature of many people. Through this, it can be usefully used in airports or buildings with frequent access by people.

Conclusions
Temperature estimation is used in a variety of medical, industrial, and military applications. In particular, body temperature measurement has emerged as a very important issue due to a fever-producing disease such as COVID-19. However, it is difficult to accurately estimate temperature due to variables such as atmospheric transmittance. In this study, we propose a method of estimating the temperature for each distance using LiDAR, which can additionally acquire distance information in an infrared camera. The proposed method constructs a digital count-radiation mapping function that reflects the effect of variables by distance, and then, when a digital count value comes in at a specific distance, the accurate temperature is estimated through the mapping function at a specific distance. In addition, the problem of not being able to build a mapping function for all distances in advance is solved through digital count interpolation. The proposed method shows that it is possible to accurately estimate temperature by distance at various temperatures, and also shows the possibility of automating body temperature measurement through Infrared-LiDAR fusion in the future as an experiment on humans. However, changes in atmospheric temperature and humidity conditions are mentioned as limitations. Therefore, it is necessary to study a robust temperature estimation method that reflects air temperature and humidity in addition to distance information.