Thermal-Imaging-Based PCA Method for Monitoring Process Temperature

Abstract

To overcome the shortage of traditional temperature sensors, this paper adopts infrared thermal imaging technology for temperature measurement. To avoid the spatial information loss issue during the image data vectorization process, this paper adopted the spatial relationship between pixels in principal component analysis (PCA) model training, which is called spatial information-based PCA (SIPCA). Then, spatial information is also used in the fault localization method to enhance the fault location performance. Tested by an experimental tank system, the proposed method achieves better performance than the traditional PCA approach, and it can detect heat leakage faults on the surface of the equipment.

1. Introduction

Safety is an eternal topic for industrial process, and hence the fault detection and diagnosis (FDD) [1,2,3] become a research hotspot in recently years. Principal components analysis (PCA) [4,5,6,7], which transforms high-dimensional process data into a smaller set of uncorrelated principal components and monitors them by several statistics indices, is a popular FDD method for large-scale industrial processes that have complex structures and operate in extreme conditions. Up to now, various improved PCA versions have been proposed and these algorithms have achieved great success in real industry applications. Harkat et al. combined machine learning with Kernel PCA (KPCA) for reducing the computational load and applied this method in chemical process monitoring [8,9]; Lennox et al. proposed a new multiblock approach and applied it in condensate fractionation process [10]; and Bakdi et al. adopted the EWMA-based adaptive threshold monitoring scheme for PCA and applied it in a cement rotary kiln [11]. To address the fault localization problem, contribution plot method [12,13] was proposed for MSPM, which calculates the contribution of each variable of the original data set and picks the variables with high contributions as fault sources.

Temperature anomaly is a common safety issue for chemical, petroleum and building materials industries. However, traditional PCA-based approaches are unsuitable for some temperature anomaly faults. The reason is that PCA is based on temperature sensor data; however, (a) first, most traditional temperature sensors are contact-type, which cannot be installed in equipment/process with extreme high temperature, so the temperature values in some positions are unmeasurable; (b) second, for monitor equipment/process comprehensively, one needs to install temperature sensors over all positions of equipment/process in high density, which is expensive and difficult to implement; (c) third, heat leakage on the surface of the equipment/process cannot be detected by traditional PCA because usually sensors are only installed in the internal of devices and processes. Therefore, finding a new temperature detection technology and developing the corresponding monitoring method is very meaningful work.

With the rapid development of soft sensing technology, many engineers and scholars have begun to seek indirect temperature detection methods based on other sensor technologies. For example, the method proposed in reference [14] to measure the industrial high temperature that cannot be detected by traditional sensors by processing the CCD image data with a support vector machine (SVM) algorithm; Yan et al. adopted spectral thermometry and image thermometry to estimate the flame temperature [15]; and Wu et al. described an integrated soft-sensing method for estimating the coke-oven temperature, which is based on the linear-regression models [16].

Infrared thermal imaging technology [17,18] uses photoelectric technology to detect the infrared specific band signal of object thermal radiation, convert the signal into images and graphics that can be distinguished by human vision, and further calculate the temperature value of the corresponding position. As shown in Figure 1 below, compared with the point temperature measurement method of the traditional temperature sensor, the infrared thermal imager has characteristics such as a wide monitoring range, large monitoring density, and non-contact detection. In addition, an infrared thermal imager can effectively reflect the surface temperature distribution characteristics of the monitoring object so as to quickly and effectively detect the temperature anomaly and thermal leakage points. Therefore, in recent years, infrared thermal imaging technology, as a new soft temperature measurement method, has gradually attracted the attention of industry and academia. Many engineers and scholars have tried to apply infrared thermal imaging technology in real industrial problems, such as electrical equipment temperature monitoring [19], battery temperature monitoring [20], and mechanical equipment temperature monitoring [21].

Infrared thermal imaging technology and PCA are complementary: on the one hand, each infrared thermal image can be used as sampling data for PCA training and monitoring; on the other hand, PCA is a dimensionality reduction algorithm that can effectively extract the key information in thermal imaging images and reduce the complexity of image processing problems. As shown in Figure 1, once a leakage fault occurs in the process device, some pixel values on the infrared thermal imaging image will change significantly, and the PCA method can accurately grasp this feature, detect the fault earlier and accurately locate the heat leakage position. For the above reasons, this project plans to propose a temperature monitoring algorithm based on infrared thermal imaging technology and PCA.

The contribution of this paper is as follows: on the one hand, to compensate for the lost spatial information during the image data vectorization process, this paper proposed a spatial information-based PCA (SIPCA), which trains the PCA model based on the spatial relationship between pixels; on the other hand, a new spatial information-based fault localization method is also proposed in this paper to enhance the fault location performance.

The remainder of this paper is organized as follows. The idea of classic PCA and contribution plot are reviewed briefly in Section 2. Then, a new thermal monitoring method based on infrared thermal imaging technology and PCA is proposed in Section 3, and some details are discussed. To fully analyze the characteristics of the proposed method and compare it with the traditional PCA monitoring approach, tests are carried out on an experimental tank system in Section 4. Finally, the contributions of this paper are summarized in Section 5.

2. Methods

2.1. Principal Component Analysis (PCA)

In the current chemical industrial process, safety monitoring relies on data collected from massive sensors. It is hard to deal with such a large data set in a cost-controllable system; thus, PCA is introduced into process diagnosis for it can effectively reduce dimensions of data and save computing resources. Its idea is to project data from a higher dimension into a lower dimension, and then it will be much easier to compare the abnormal conditions with the normal conditions.

Let $X=[x_1,x_2,\cdots x_s]\in R^{n\times s}$ denotes process data with $n$ samples and $s$ variables, then the covariance matrix $S$ is used to derive a PCA model, defined as follows:

$$S=\frac{1}{n-1}{X}^{T}X.$$

(1)

By using singular value decomposition (SVD) [22] to $S$, one gets

$$X=T{P}^T+E=\widehat{X}+E$$

(2)

where $T\in {\mathbf{R}}^{n\times k}$ refers to the score matrix, $P\in {\mathbf{R}}^{s\times k}$ refers to the loading matrix, $E\in {\mathbf{R}}^{n\times s}$ is the residual matrix, and $k$ is the number of retained principal components (PCs) [23]. Number $k$ is usually calculated by the cumulative percent variance ($CPV$) method [23], which is defined as follows:

$$CPV={\displaystyle \sum _{i=1}^k{\lambda }_i/}{\displaystyle \sum _{i=1}^s{\lambda }_i\times 100\%\ge \epsilon }.$$

(3)

where ${\lambda }_i$ (${\lambda }_1\ge {\lambda }_2\ge \cdots {\lambda }_s\ge 0$) is the variance of the score vector and $\epsilon$ is a parameter usually set to 85%.

Give a new test data sample $x\left(t\right)\in {\mathbf{R}}^{1\times k}$, then $T^2$ and $SPE$ statistics [24] are constructed to monitor $\widehat{X}$ and $E$ as below:

$$T^2\left(t\right)=x{\left(t\right)}^{T}P{({\Lambda }_k)}^{-1}{P}^{T}x\left(t\right).$$

(4)

$$SPE\left(t\right)=(x\left(t\right)-\widehat{x}\left(t\right)){(x\left(t\right)-\widehat{x}\left(t\right))}^T,$$

(5)

where $t$ is the sample time, $\widehat{x}=T{P}^T=xP{P}^T$, and

$${\Lambda }_k=\left[\begin{array}{cccc}{\lambda }_1& 0& 0& 0\\ 0& {\lambda }_2& 0& 0\\ 0& 0& \ddots & 0\\ 0& 0& 0& {\lambda }_k\end{array}\right]({\lambda }_1\ge {\lambda }_2\ge \dots \ge {\lambda }_k\ge 0).$$

Statistic $T^2$ represents the distance between the location of the new data projected onto the subspace and the origin of subspace; statistic $SPE$ is a measure of the approximation error of the new data within the PCA subspace.

2.2. Fault Localization through Contribution Plot

Based on PCA, the operator would be alarmed when the working condition is abnormal, but one cannot locate the fault source further. To solve this problem, a contribution plot method based on multivariate statistics is proposed.

The contributions to $SPE$ are calculated as follows:

$$Con SP{E}_j=(x_j-{\widehat{x}}_j){(x_j-{\widehat{x}}_j)}^T,$$

(6)

where $x_j$ and ${\widehat{x}}_j$ are the $j^{th}$ columns of $x$ and $\widehat{x}$, respectively.

The contributions to $T^2$ are calculated as follows:

$$Con{T}_j^2={\displaystyle \sum _{i=1}^k(}{x}_j-{\widehat{x}}_j)P_{j,i}{\lambda }_i^{-1}{P}_{\mathbf{i}}^T{x}^T,$$

(7)

where $P_i$ is the $i^{th}$ column of $P$, and $P_{j,i}$ is the element in the $j^{th}$ column and $i^{th}$ row. The thresholds of $Con SP{E}_j$ and $Con T_j^2$ can be obtained by kernel density estimation [25].

3. Thermal Monitoring

Figure 2 summarizes the procedure presented above.

3.1. Image Data Preprocessing

For the thermal image data set $\left\{G_t\right\}$ where $G_t$ is the thermal image sample at time $t=1,2,\cdots ,n$, the first step is to convert the 3-channel RGB image to a 1-channel gray image, using the following equation:

$$Gray=0.3\times R+0.59\times G+0.11\times B.$$

(8)

where $Gray$, R, $G$, and $B$ are the values in the gray image and the RGB image, correspondingly. As such, the data set $\left\{G_t\right\}\in R^{n\times m\times l}$, where $m$ and $l$ are the number of rows and columns in each gray image.

Remark.

Thermal image contains a massive amount of high-dimension data, which introduces a large amount of computation. As such, methods such as Gaussian pyramid [26] can be adopted for image compression.

3.2. Spatial Information-Based PCA (SIPCA)

Before applying PCA, the data set $\left\{G_t\right\}$ should be converted into a 2-D matrix first, which means each gray image data $G_t$ should be converted into a vector. It should be noted that the thermal distribution is closely related to the spatial distribution, i.e., the pixels with closer spatial distance have a closer thermal relationship. However, spatial information is usually missing when a 2-D matrix gray image is converted into a 1-D vector during the traditional column- or row-expansion operation.

As such, this paper proposes a spatial information-based PCA (SIPCA) method for monitoring thermal image data. First, we construct the following function for measuring the distance between two pixels $G_t(i_1,j_1)$ and $G_t(i_2,j_2)$ as

$$Dis(G_t(i_1,j_1),G_t(i_2,j_2))=e^{\frac{\sqrt{{\left(i_1-i_2\right)}^2+{\left(j_1-j_2\right)}^2}}{\sigma }},$$

(9)

where $\sigma$ is a weight factor with a positive value (e.g., 1). Then convert $G_t$ into a 1-D vector, e.g., row-expansion as follows:

$$X\left(t\right)=\left[G_t(1,1),\cdots ,G_t(1,l),\cdots ,G_t(m,1),\cdots ,G_t(m,l)\right].$$

(10)

Synchronously, the distance matrix is calculated as:

$$D=\left[\begin{array}{cccc}Dis(G_t(1,1),G_t(1,1))& Dis(G_t(1,1),G_t(1,2))& \cdots & Dis(G_t(1,1),G_t(m,l))\\ Dis(G_t(1,2),G_t(1,1))& Dis(G_t(1,2),G_t(1,2))& \cdots & Dis(G_t(1,2),G_t(m,l))\\ ⋮& ⋮& \ddots & ⋮\\ Dis(G_t(m,l),G_t(1,1))& Dis(G_t(m,l),G_t(1,2))& \cdots & Dis(G_t(m,l),G_t(m,l))\end{array}\right]$$

(11)

Therefore, the new covariance matrix $S^{\prime }$ is calculated as follows:

$$S^{\prime }=\frac{1}{n-1}{X}^T{D}^{-1}X.$$

(12)

Then, the new PCA model can be obtained by SVD to $S^{\prime }$. As such, the new PCA method pays more attention to the thermal relationships of adjacent pixels.

3.3. Spatial Information-Based Fault Localization Strategy

However, the contribution plot method can pick out pixels with a high contribution to the fault. However, the high contribution is not equivalent to the high faulty probability, so some normal pixels may be misdiagnosed as fault sources and some faulty pixels may be diagnosed as normal.

When a thermal fault occurs, the pixels in a specific area are abnormal. As such, the following fault localization strategy is proposed.

Step 1. Transfer the contribution score vector to a matrix according to the spatial position of each pixel:

$$SCon SP{E}_{i,j}=Con SP{E}_{i\ast l+j}.$$

(13)

$$SCon T_{i,j}^2=Con T_{i\ast l+j}^2.$$

(14)

Step 2. Update the contribution scores with the convolution window function as

$$\frac{1}{{\beta }^2}{\displaystyle \sum _{a=-\beta }^{\beta }{\displaystyle \sum _{b=-\beta }^{\beta }SCon SP{E}_{i+a,j+b}}}\Rightarrow SCon SP{E}_{i,j}.$$

(15)

$$\frac{1}{{\beta }^2}{\displaystyle \sum _{a=-\beta }^{\beta }{\displaystyle \sum _{b=-\beta }^{\beta }SCon SP{E}_{i+a,j+b}}}\Rightarrow SCon SP{E}_{i,j}.$$

(16)

where $\beta$ is the size of the convolution window function. Therefore, fault localization is based on the pixels in a specific area rather than a single pixel, avoiding the wrong fault localization problem.

4. Experiment Test

An experimental tank system, shown in Figure 3, is used to test the proposed method, which consists of 2 main tanks. The water inlet of the system is located in the right tank and the outlet is located in the left tank. Two tanks are linked by a tube that is fully insulated, and water flows from the right tank to the left tank. Both tanks have a bulkhead to divide the water tank into two chambers and two chambers connected through the bottom channel. In this experiment, the following equipment are used: thermal camera for obtaining the thermal image (Guide sensmartMobIR air produced by Guide sensmart in Wuhan, China, 25 Hz, and 0.06 °C accuracy); a heating module (CHB401 produced by Hualongdianre in China, <0.5% full scale); and a temperature detection module, which includes 6 temperature sensors (SXPT100 produced by SENSOR in China, 0.15 °C accuracy).

The normal working conditions of the tank system are as follows:

(a)

The heating module is installed near the water inlet position: (a) once the temperature reaches the set upper value (46 °C), the heater will be turned off immediately; (b) until the detected value drops to the lower limit (44 °C), the heater will be turned on again. Thus, the temperature of the whole system is limited to a certain range. The temperature decreases from the right part to the left part.

(b)

All chambers’ water levels are kept at a certain height, e.g., 80%.

And 2000 samples of normal data are collected for model training. The sampling period is 1 s.

In addition, another 2000 samples of data are generated for testing. Each data set contains 2000 samples, and the fault occurs at the 501st sample point. The faults that occurred are of the following three types:

(a)

Peeling of insulation layer to simulate the heat leakage fault;

(b)

Adding another code water inlet in the right chamber of the right tank to simulate an abnormal stable working condition;

(c)

Adding another additional code water inlet into the left chamber of the left tank to simulate the situation in which one chamber’s condition is changed and the other three chambers’ conditions remain unchanged.

For comparison, 6 temperature sensors are installed in the experimental tank system, whose positions are as marked in Figure 3. The data obtained by temperature sensors are monitored by traditional PCA. For both temperature sensors and infrared thermal imaging technology.

Figure 4, Figure 5 and Figure 6 show the fault detection results of the SIPCA and the traditional PCA approach. As shown in Figure 4, PCA cannot detect Fault 1 because all temperature sensors are only installed in the internal of the experimental tank system and hence PCA cannot detect the temperature anomaly on the surface of the equipment. Different from traditional temperature sensors, a thermal image can successfully catch the temperature change on the object surface, and hence it detects the fault earlier. For both Faults 2–3 in Figure 5 and Figure 6, one gets that SIPCA detects these faults earlier than PCA: SIPCA detects Fault 2 in the 517th sample and PCA detects this fault on the 528th sample; SIPCA detects Fault 3 in the 510th sample and PCA detects this fault on the 558th sample. The reason for this phenomenon is that PCA is based on the data of the temperature sensors, so the detection delay is large when the sensor positions are far from the fault sauce; different from the traditional temperature sensors, the infrared thermal imager monitors the temperature of the process in pixel-level granularity, and hence it can detect this fault earlier. In addition, SIPCA adopts spatial information for model description, so it is sensitive to the temperature variation between adjacent spaces and hence can detect the fault earlier.

Figure 7 shows SIPCA’s fault localization result for fault 1. Based on the spatial information-based fault localization strategy, SIPCA can successfully locate the fault source position in the image (the red part) and filter out false alarming pixels.

5. Conclusions

In this paper, infrared thermal imaging technology was adopted for measuring the temperature of the equipment/process, and a new PCA called SIPCA was proposed for handling the information in thermal images. SIPCA can introduce spatial information into traditional PCA approaches; hence, it is much more sensitive to temperature variations between adjacent spaces. In addition, spatial information was also adopted in fault localization, and the wrong fault localization problem of the traditional contribution plot method was avoided. Then, the proposed method was tested on an experimental tank system, and the results confirmed that it performs better than the traditional PCA methods.

One drawback of SIPCA is that it is based on a 2D image, and hence it cannot detect the faults that occur on the back of the monitored object. To handle this issue, our future work will focus on the combination of SIPCA with multi-view 3D reconstruction algorithms [27].