Temperature Measurement Method of Flame Image Fusion with Different Exposures

Abstract

Fixed exposure will lead to underexposure or overexposure of collected flame radiation images using CCD, which has a great influence on the temperature measuring accuracy. A temperature measurement method was proposed by image fusion with multi-exposure, which can eliminate the influence of insufficient underexposure and overexposure. The approach was first to acquire a group of flame radiation images during different exposures. Then a partial region with good exposure effect in each radiation image was obtained by segmentation, with which the complete flame image can be spliced together. An experimental system was built to calibrate the temperature measurement parameters by two-color pyrometry through a blackbody furnace. The relation between exposure time and monochromatic gray level, as well as the relation between the temperature and temperature measurement coefficient were obtained. A candle flame was selected as the measuring object and the complete and accurate flame temperature distribution was acquired following our proposed method. The experimental results show that, compared with the temperature measurement using a single exposure time, our method can effectively avoid the measurement error caused by underexposure and overexposure, and improve the measurement accuracy.

1. Introduction

Flame temperature detection has always been a research hotspot in the combustion field as the temperature can reflect the combustion state and judge, predict and diagnose it. Accurate measurement of flame temperature is very important in order to adjust the combustion model, optimize the combustion process and control the pollutant emissions. With the development of digital image technology in recent years, radiation thermometry based on color CCD has become a hot topic in the research field of temperature measurement [1,2,3,4,5]; it is widely used in scientific research, industrial production and national defense research [6,7,8,9,10,11].

For decades, many researchers have investigated image-based non-contact thermometry intensively. Zhou et al. [12,13,14,15] have carried out dozens of experiments on flame temperature detection and proposed a monochromatic method for reference temperature measurement. After the CCD camera could automatically output RGB images, a calibration method was proposed for the conversion between image intensity and absolute radiation intensity, and subsequently put into industry applications. Levendis et al. [16] proposed to use a three-color near-infrared optical pyrometer to monitor the combustion of polymer particles, combined with a soot radiation model that can increase the upper limit of temperature detection. Compared with the two-color pyrometry, it has a higher temperature measurement accuracy but a narrower dynamic range. Yamashita et al. [17] developed a temperature measurement technology in the welding process, which used a multi-sensor camera based on the two-color thermometry method. The camera can use three filters to set different spectral wavelengths, to more accurately and quickly respond to the welding temperature in a wide temperature range. But the camera has a complex structure and high price, and it is also inconvenient to select the best measurement wavelength combination. Huang et al. [18] used the method of adding optical filters in front of a CCD camera to alternately capture the images at two wavelengths, and calculated the average temperature of the flame using two-color pyrometry based on the average gray ratio of the two images. Xu et al. [19] developed a new spectral-emittance method to modify RGB pyrometry, which was used to measure the surface temperature of dispersed chars in a Hencken flat-flame burner. In this method, 420/440 two-color pyrometry and RGB pyrometry were compared to build a calibrated normalized spectral emissivity model of chars, and then modified the lookup table of the Nikon camera to complete the RGB correction. Although the results obviously reduced the temperature measurement error, the calibration process was more complicated.

In recent years, many research works focused on improving the measurement accuracy and range of image temperature measurement technology. Sun et al. [20] built a normalized spatial distance weighted directional-distance filter to remove the interference of light noise, based on the spatial distance weighted function and the directional-distance filter. In conventional ratio pyrometry applications, a calibration process is tedious, and the range of the effective temperature is narrow. Shao et al. [21] improved the traditional two-color pyrometry, which can set different exposure times for different channels to simplify the calibration of the blackbody furnace and widen the temperature measurement range. However, the research used a 3CCD camera, which is more complicated and costly even if the camera can set the exposure time of the R, G and B channels, respectively. Yan et al. [22] combined spectral thermometry and image thermometry to measure the flame temperature and emissivity simultaneously. A portable optical fiber spectrometer and an industrial color camera were used to obtain measurement of the combustion flame during biomass volatile combustion, and a non-gray emissivity model with a third-order polynomial function was established to correct the image pyrometer data, which reduces the limitation that the image temperature measurement can only obtain radiation at finite wavelength. In the same year, Yan et al. [23] proposed another method to measure the temperature of objects with simultaneous use of a spectrometer and a high-speed camera, which improved the accuracy of transient flame temperature measurement. Li et al. [24] provided a novel flame temperature estimation method based on a flame light field sectioned imaging model of complex temperature distribution in different media. The method relies on multi-pixel reconstruction, the wavelet transformation and Lucy–Richardson algorithm to calculate the flame temperature, which improves the accuracy of the temperature estimation and can be used in over a large estimation range. Despite a great deal of effort to improve the temperature measurement accuracy, the error caused by overexposure and underexposure of image pixels has not been well solved so far.

Image fusion is an enhancement technique that aims to integrate a large amount of complementary information from multi-source images of the same scene into a new image [25]. This technology has been studied for decades and is widely used in industry, medical treatment, machine learning and other fields. In order to optimize the fusion technology, various fusion algorithms have been developed in recent years [26,27,28,29,30]. As early as in 1979, Daily et al. [31] fused radar and Landsat-MSS images for the first time to extract geological information. Zhang et al. [32] designed a multispectral and panchromatic image fusion model employing adaptive spectral–spatial gradient sparse regularization for vegetation phenotypes and generated accurate vegetation indices. Yin et al. [33] proposed a new multimodal medical image fusion method in the non-subsampled shearlet transform (NSST) domain. The reconstructed fusion image has led to state-of-the-art results on both visual quality and objective assessment. Liu et al. [34] adopted a new multi-focus image fusion method to obtain state-of-the-art-fusion performance, using deep learning to solve the two crucial factors of activity level measurement and fusion rules in image fusion.

This paper proposes a new image temperature measurement method based on image fusion technology, which is to collect a group of flame radiation images with a color CCD camera during different exposure times, and then splice a complete flame image by good exposure of parts of each image, so as to obtain a more accurate temperature distribution.

2. Principle and Methods

2.1. Two-Color Pyrometry

Two-color pyrometry is also called the colorimetry temperature measurement, which is a method to measure temperature by the ratio of radiation intensities in two adjacent narrow wave bands [35,36]. Through the R and G channels of CCD, the flame radiation information of the red and green narrow bands filtered by monochromatic filter is obtained. The flame temperature is calculated by Equation (1)

$$T=-C_2(\frac{1}{{\lambda }_r}-\frac{1}{{\lambda }_g}){\left(\ln \left(\frac{1}{C_g}\frac{R}{G}\frac{{\lambda }_r^5}{{\lambda }_g^5}\right)\right)}^{-1}$$

(1)

where $T$ is the flame temperature calculated by the two-color pyrometry, with the unit of K; $C_2$ is the constant of Planck with the value of $1.4388\times {10}^{-2}$; $\mathrm{W}·{\mathrm{m}}^2$, ${\lambda }_r$ and ${\lambda }_g$ are the fixed wavelengths of flame R and G light components, valued 700 nm and 546.1 nm, respectively; and $C_g$ denotes the corrected coefficient of component G.

2.2. Fusion Method

According to Jiang's [37] experimental research in 2004, the CCD works in the state of linear change with the RGB gray values from 50 to 220, which can ensure the accuracy of flame temperature measurement by two-color pyrometry. In a flame, the temperatures of different positions can vary from several hundred K to several thousand K, which will lead to an exponential growth of radiation intensity. In this case, if the radiation image is collected under a fixed exposure time, serious overexposure and underexposure will appear and reduce the measurement accuracy.

The temperature measurement method proposed in this paper is based on a pixel-level fusion processing for images with different exposure times. In order to extract the effective temperature measurement areas from different flame radiation images, it is necessary to define the overexposure and underexposure state by the gray values of R and G [38]. From the previously mentioned literature [37], when the gray values of R and G are less than 50, the pixels are defined in the state of underexposure. When the values are greater than 220, the pixels are defined overexposure. Thus, in our experiment, the pixels with the gray values from 50 to 220 in the flame radiation image are considered as the effective points to be extracted.

The process of our method is shown in Figure 1. Firstly, a group of flame radiation images during different exposures are obtained, named $I{O}_{\mathrm{n}}(x,y)$, see Figure 1A. Then, according to Equation (2), images in group (a) are segmented to group (b), named $I{S}_n(x,y)$. $R_n(x,y)$ and $G_n(x,y)$ in Equation (2) denote the gray values of R and G in $I{O}_{\mathrm{n}}(x,y)$.

$$I{S}_n(x,y)=\{\begin{array}{l}I{O}_n(x,y),\begin{array}{cc}& \end{array}50\le R_n(x,y)\le 220\begin{array}{c}\end{array}\mathrm{and}\begin{array}{c}\end{array}50\le G_n(x,y)\le 220\\ 0,\begin{array}{cc}& \end{array}else\end{array}$$

(2)

The two-color pyrometry formula is used to calculate the temperatures of the effective areas in the images of group B, and the image sequence of two-dimensional flame temperature distribution with effective measurement areas $I{E}_n(x,y)$ were obtained, as shown in group (c). The effective pixels may overlap in different pictures of the image sequence, which are recorded under different exposures. Therefore, the final complete temperature distribution requires numerical fusion rather than direct superposition. In the fusion process, the temperature at the same position in the image sequence needs to be the weighted average and the gross error points should be removed. The weight value of the weighted average fusion was determined by measured values R and G. The closer the temperature calculation result is to the median value, the greater the weight is. See Equation (3) for details, where m represents the number of images in image sequence that have an effective temperature at the same position. After the weighted fusion value is obtained, it is compared with the measured value of the images. When the relative error is more than 5%, it is eliminated as a gross error point to be removed. Finally, a complete flame temperature distribution is obtained, represented by $IT(x,y)$, shown in Figure 1d.

$$IT(x,y)=\frac{{\displaystyle \sum w_n\times I{E}_n(x,y)}}{m}$$

(3)

3. Experimental Setup

The experimental system is shown in Figure 2, composed of a candle, a CCD camera and a computer. The candle flame is the measuring object. After the candle was ignited, the evaporated paraffin diffused into the air and formed an orange flame, which produced the bright combustion radiation of carbon particles. In the three-dimensional temperature distribution of the candle flame, from previous literature [39], the temperature of the outermost flame is greater than 1473.15 K, and the highest value can be up to 1673.15 K. The inner flame is greater than 1273.15 K, and the flame core is less than 1273.15 K.

The CCD camera is the sensing device. In this research, the candle flame was considered to be stable in a short time, so a group of images collected in different exposure times were considered to record the same flame. The Daheng Image Mercury MER-132-30GC color CCD camera was adopted, with a frame rate of 30 fps, shutter time of 20 μs to 1 s, target size of 1/3 inch and the pixel format of Bayer rg8. From experience, the white balance RGB gain parameter was set as 2:4:6.

The computer is the control and the processing unit, which set the white balance gain parameters of the color CCD camera and controlled the CCD to collect the flame radiation images at different exposures in a short time. The two-dimensional flame temperature field was obtained after the image sequence was segmented. Before image thermometry, it is necessary to calibrate the temperature measurement coefficient. A blackbody furnace was selected to do the calibration, with an ISOTech Cyclops model 878, produced by the British company Essent. The temperature range of the furnace was from 373.15 K to 1573.15 K, with an accuracy of 0.1 K. The radiation rate was 0.999. In our calibration tests, the temperature was set from 1073.15 K to 1473.15 K, with an interval of 25 K.

4. Results and Discussion

4.1. Temperature Measurement Coefficient Calibration

In order to obtain the correct flame temperature, the parameters of the CCD need to be calibrated through a blackbody furnace before flame temperature measurement. The experimental parameter settings of the furnace and the camera are described in Section 3. The R and G gray values of the blackbody radiation images were calculated, then the effective temperature measurement areas with the gray value from 50 to 220 were extracted. The exposure time ranges corresponding to different temperatures were obtained, and the relation model between the monochromatic base value and the exposure time was established.

Figure 3 shows the fitting curves of the monochromatic gray values R and G with exposure time at five furnace temperatures: 1073.15 K, 1173.15 K, 1273.15 K, 1373.15 K and 1473.15 K. The abscissa x is the logarithm form. It can be seen from the figure that in order to obtain the effective gray values in different furnace temperatures, the exposure time varies greatly. Higher furnace temperature needs shorter exposure while lower temperature needs longer exposure. The fitting relationships between the R and G gray values and the exposure time at five temperatures were derived, as shown in Equations (4) and (5):

$$\left[\begin{array}{l}{R}_{1073.15}\\ R_{1173.15}\\ R_{1273.15}\\ R_{1373.15}\\ R_{1473.15}\end{array}\right]=\left[\begin{array}{ccc}\begin{array}{l}-8.23\times {10}^{-8}\\ -2.35\times {10}^{-6}\\ -3.56\times {10}^{-5}\\ -5.23\times {10}^{-4}\\ -4.34\times {10}^{-3}\end{array}& \begin{array}{l}9.10\times {10}^{-3}\\ 3.82\times {10}^{-2}\\ 1.60\times {10}^{-1}\\ 6.41\times {10}^{-1}\\ 1.72\times {10}^0\end{array}& \begin{array}{l}2.43\times {10}^0\\ 5.01\times {10}^0\\ 1.17\times {10}^1\\ 3.14\times {10}^1\\ 7.01\times {10}^1\end{array}\end{array}\right]\left[\begin{array}{l}{t}^2\\ t\\ 1\end{array}\right]$$

(4)

$$\left[\begin{array}{l}{G}_{1073.15}\\ G_{1173.15}\\ G_{1273.15}\\ G_{1373.15}\\ G_{1473.15}\end{array}\right]=\left[\begin{array}{ccc}\begin{array}{l}-2.01\times {10}^{-7}\\ -3.13\times {10}^{-6}\\ -6.47\times {10}^{-5}\\ -1.11\times {10}^{-3}\\ -3.30\times {10}^{-3}\end{array}& \begin{array}{l}1.26\times {10}^{-2}\\ 5.21\times {10}^{-2}\\ 2.27\times {10}^{-1}\\ 9.22\times {10}^{-1}\\ 1.13\times {10}^0\end{array}& \begin{array}{l}3.95\times {10}^0\\ 1.06\times {10}^1\\ 2.32\times {10}^1\\ 6.05\times {10}^1\\ 1.68\times {10}^2\end{array}\end{array}\right]\left[\begin{array}{l}{t}^2\\ t\\ 1\end{array}\right]$$

(5)

where R and G represent the R and G gray values of the blackbody radiation image, the subscript represents the furnace temperature and t is the exposure time.

From Figure 3 and the two fitting equations, the approximate range of exposure time at different temperatures can be obtained, as shown in

Table 1. Reasonable exposure ranges for different furnace temperatures.

Temperature (K)	Exposure Range (μs)
1073.15	(6234, 27421)
1173.15	(1260, 6317)
1273.15	(251, 1354)
1373.15	(29, 245)
1473.15	(20, 55)

. The exposure time setting range of the camera is from 20 μs to 1 s, hence the lower limit of the exposure is 20 μs in this experiment.

In the literature [37], the coefficient C_g almost has no relation with the exposure time but the temperature. So, the coefficient was evaluated as the mean value of the coefficients from the blackbody images, and it was found that the value changes linearly with the increasing temperature. The fitting relation is shown in Equation (6), and the fitting curve is shown in Figure 4.

$$C_g=7.51\times {10}^{-5}T-0.0681$$

(6)

From the literature as well [37], when the flame temperature ranges from 1073.15 K to 2073.15 K, the gray value of the blackbody radiation intensity has a mathematic relationship with the temperature. Therefore, the initial temperature can be estimated by the relation. Then the temperature measurement coefficient can be obtained by Equation (6). Finally, the flame temperature can be calculated by the two-color pyrometry formula.

From previous analysis, a different temperature needs a different exposure time, which will affect the relation between the gray value and the initial temperature. So, it is necessary to determine the relationships between the gray value of G and the temperature at different exposures. Eight exposure times were set to collect the radiation images of the same flame, which were 20 μs, 50 μs, 80 μs, 100 μs, 120 μs, 150 μs, 180 μs and 200 μs. The fitting relationship between the G value and the temperature can be obtained, as shown in Equation (7). The subscript of T represents the exposure time.

$$\left[\begin{array}{l}{T}_{20}\\ T_{50}\\ T_{80}\\ T_{100}\\ T_{120}\\ T_{150}\\ T_{180}\\ T_{200}\end{array}\right]=\left[\begin{array}{ccc}\begin{array}{l}-3.31\times {10}^{-3}\\ -3.10\times {10}^{-4}\\ -8.00\times {10}^{-4}\\ -6.48\times {10}^{-4}\\ 9.23\times {10}^{-4}\\ 7.06\times {10}^{-4}\\ 2.00\times {10}^{-4}\\ 9.00\times {10}^{-5}\end{array}& \begin{array}{l}1.91\times {10}^0\\ 1.86\times {10}^0\\ 1.13\times {10}^0\\ 1.00\times {10}^0\\ 5.01\times {10}^{-1}\\ 4.39\times {10}^{-1}\\ 6.32\times {10}^{-1}\\ 6.50\times {10}^{-1}\end{array}& \begin{array}{l}1.24\times {10}^3\\ 1.21\times {10}^3\\ 1.24\times {10}^3\\ 1.25\times {10}^3\\ 1.26\times {10}^3\\ 1.28\times {10}^3\\ 1.25\times {10}^3\\ 1.25\times {10}^3\end{array}\end{array}\right]\left[\begin{array}{l}{G}^2\\ G\\ 1\end{array}\right]$$

(7)

To obtain the flame temperature, the first step is to use the gray value of G to calculate the temperatures at different exposure times by Equation (7). Then calculate the temperature measurement coefficients using Equation (6). Finally, utilize the two-color pyrometry to obtain the flame temperature by Equation (1).

4.2. Image Fusion of Multi-Exposure Times

Figure 5 depicts the collected flame images at the selected eight exposures. Since the white balance ratio of R, G and B is set as 2:4:6, the flame images appear green. Comparing the eight images, it is evident that at a short exposure time, the flame contour is small. With the increase in exposure time, the flame gets brighter and the flame contour becomes bigger, tending to be close to the actual flame size observed by one’s eyes.

Figure 6 is the temperature distribution of the original flame image in Figure 5, calculated by the method described in Section 4.1. It is obvious that the contour of the temperature distribution has the same variation tendency with the original flame image, and the temperature range of the flame is between 1300 K and 1600 K, approximately. From the eight pictures, only the first one, which used the exposure time of 20 μs, has the highest temperature in the flame center, while in the other seven pictures the central temperatures are all lower than the surrounding part. The reason is that with the increment of the exposure in those seven cases, the flame became brighter in the center and the gray values of R and G in the central part became saturated gradually, which led to an inaccurate temperature calculation.

Thus, the multi-exposure time image fusion method presented in Section 2.2 will be applied here. The temperature distributions with effective areas are presented in Figure 7. It can be seen from the eight images that with the increase of exposure, the flame contour increases gradually, and the range of center removal is also increasing. This is due to the overexposure central gray value. Underexposure will also lead to partial flame loss, reflected in the smaller contour compared with the actual flame.

With the effective areas in Figure 7, a modified flame distribution can be acquired, with no underexposure and overexposure, shown in Figure 8. It should be noted that in Figure 8, the temperature of the inner flame is higher than the outer flame, which seems contradictory to the conclusion mentioned in the literature [39] in Section 3. That is because of the overlap of different parts of the flame. In detail, the flame itself is a three-dimensional object, which has an outer flame, inner flame and flame core from the outside to the inside. However, the flame temperature distribution obtained by image radiation thermometry is a two-dimensional field, which has some discrepancies with actual three-dimensional distribution. The CCD image is a projection of three-dimensional space, which is called “a projection temperature field”. It is an integral cumulative effect of three-dimensional radiation on the two-dimensional plane, not a temperature distribution in a strict sense. It reflects the integral average value of the space temperature distribution along the measurement optical path [40]. Our result is consistent with the research of Zheng et al. [41] on the candle flame distribution of temperature and emissivity, as well as the measurement results of candle temperature by Arpit et al. [42] using a CCD camera.

5. Conclusions

In this paper, on the issue of overexposure and underexposure in CCD collecting flame images, which are caused by the great differences in flame radiation intensity, a novel two-dimensional flame temperature distribution measuring method based on image fusion of multi-exposure times is proposed, which effectively solves the common problem in CCD flame temperature measurement with a wide temperature range. It mainly includes four aspects:

• According to our previous experimental researches and other literature, the optimal pixel gray range is determined to be 50 to 220, which can ensure the accuracy of the two-color pyrometry.
• Through the relationship between the exposure time and the radiation intensity (image pixel gray value) under different temperatures, the appropriate exposure intervals in different flame temperatures are determined, which ensures that overexposure and underexposure will not appear in the selected areas of the flame images.
• Flame images with different exposures were collected and effective measurement areas were extracted by means of threshold segmentation. Image fusion was performed to obtain the complete two-dimensional flame temperature distribution.
• This method has been proved to be effective in two-dimensional flame temperature distribution, which provides a foundation for applying the method in accurate three-dimensional flame temperature field reconstruction.