Fault Identification for a Closed-Loop Control System Based on an Improved Deep Neural Network

Abstract

Fault identification for closed-loop control systems is a future trend in the field of fault diagnosis. Due to the inherent feedback adjustment mechanism, a closed-loop control system is generally very robust to external disturbances and internal noises. Closed-loop control systems often encourage faults to propagate inside the systems, which may lead to the consequence that faults amplitude becomes smaller and fault characteristics difference becomes more inapparent. Hence, it has been challenging to achieve fault identification for such systems. Traditional fault identification methods are not particularly designed for closed-loop control systems and thus cannot be applied directly. In this work, a new fault identification method is proposed, which is based on the deep neural network for closed-loop control systems. Firstly, the fault propagation mechanism in closed-loop control systems is theoretically derived, and the influence of fault propagation on system variables is analyzed. Then deep neural network is applied to find fault characteristics difference between different data modes, and a sliding window is used to amplify the fault-to-noise ratio and characteristics difference, with an aim to increase the identification performance. To verify this method, the simulations that are based on a numerical simulation model, the Tennessee industrial system and the satellite attitude control system are conducted. The results show that the proposed method is more feasible and more effective in fault identification for closed-loop control systems compared with traditional data-driven identification methods, including distance-based and angle-based identification methods.

1. Introduction

With the development of science and technology, the complexity of the industrial systems has been increasing rapidly. The fault of these complex systems can lead to the decline in product quality and may cause significant property damage or casualties. Therefore, it has become a hot research topic to improve safety and reliability of system operations and to detect or even to identify faults accurately in a timely manner [1,2,3,4].

In order to achieve the predetermined production goals and meet the stability and the robustness for the industrial systems, the closed-loop control is generally applied [5,6]. Through the closed-loop control, the influence of external disturbances and internal noises on the operation of the system is reduced, which makes the system much more robust. At present, a large number of closed control loops have been widely used in the industrial production processes, ranging from mechanical machine control to spacecraft control [7,8].

Due to the nature of closed control loops, the performance of fault diagnosis is degraded. The main reasons are as follows [9]:

(1) The closed control loop usually makes the system more robust to external disturbances and internal noises. When the fault happens in the early stage or the fault amplitude is small, the fault signal will be covered by external control signals, which is difficult for the fault to be detected, resulting in lower fault detection rates;

(2) Due to the inherent feedback adjustment mechanism, a closed-loop control system encourages faults to propagate inside the system, leading to the faults existing in many variables/signals and the faults amplitude becoming smaller, or resulting in the faults characteristics difference being much inapparent and the faults coupling with each other. Such feedback adjustment ability adds to the difficulty for fault identification.

This paper focuses on the second issue in closed-loop control systems. The main aim is to achieve fault identification in closed-loop control systems and improve fault identification performance.

For the structural health monitoring in opto-mechanical systems, the system scanning technology is based on a mechanical rotary mirror and the computational processing of an optoelectronic signal is applied [10]. The 3D measurement is improved with the multivariate outlier mining and regression feedback. New research performed in the signal and data processing of the 3D measurement system demonstrated the effectiveness of signal processing strategies as a tool for a feedback loop in any measurement system.

For position detection in real life application for structural health monitoring (SHM) by a novel method, support vector machine (SVM) regression was applied to predict measurements errors for accuracy enhancement in optical scanning systems [11]. The accuracy can be enhanced based on the power spectrum centroid calculation. The SVM regression method is proved that it can be used to increase measurement accuracy for optical scanning systems.

There are a lot of research on fault identification, and the core problem of fault identification lies in the extraction of fault characteristics.

Literature [12] proposed a fault identification method based on kernel-independent component analysis (KICA). This method extracts the direction information of the faults as the identification characteristics, and then uses the training data to build a model for fault identification. However, as the complexity of closed-loop control systems increases, it becomes more and more difficult to build an accurate model only based on sensor data.

With the development of machine learning research, more and more scholars apply it to the field of fault diagnosis. Literature [13] was inspired by contribution analysis (CA) method. This method decomposed the distance of k-nearest neighbor (kNN) and used it as the fault characteristics for fault identification. It addressed the shortcomings of traditional CA method in fault identification. However, the method based on the distance judgment could not properly identify the coupled fault modes. A closed-loop control system has high complexity and multiple coupled modes. Hence the above method is not suitable for fault identification in closed-loop control systems.

In non-homogeneous discrete systems, Literature [14] detected and identified faults in the case of a large number of sensors disturbances. The residual of the observed variable was calculated by estimating the residual characteristics of the state variable, and an appropriate threshold [7] was selected to compare the difference between the residual and the threshold in each dimension.

Literature [15] approached the fault detection and identification (FDI) problem for nonlinear systems by using neural networks. The proposed FDI approach employed a recurrent neural network-based observer for detecting, isolating and identifying actuator faults in presence of disturbances and uncertainties in model and sensor measurements. The method used a modified dynamic backpropagation scheme to update the weights of the neural network, and identified the fault characteristics by detecting the updated parameter characteristics of the neural network. However, for closed-loop control systems, different modes may have strong coupling, resulting in parameter changes not meeting the detection and identification requirements.

According to the statement above, most of existing fault identification methods for open-loop systems cannot be directly applied to closed-loop control systems. For closed-loop control systems, the dynamicity of the system poses great challenges in constructing an accurate physical model and determining the model parameters, and the mathematical model established by data fitting may not be malleable or expandable for test data, which leads to the consequence that state vectors or system parameters cannot be estimated accurately. In addition, a closed-loop control system is generally robust to the external disturbances [16], so that the fault amplitude gradually becomes smaller during fault propagating inside the system; moreover, the fault characteristics difference between different modes can be reduced, and the fault modes could be coupled with each other. Those all add to the difficulty of fault identification for closed-loop control systems.

Based on these observations, in this paper, a fault identification method based on a deep neural network is proposed. The deep neural network [17] can learn independently, and can obtain the characteristics of each data or fault mode through repeated training. It is expected to obtain a better fault identification performance.

The traditional deep neural network method only trains each set of data. If it is directly applied to a dynamic closed-loop system, the dynamics between each group of data will be neglected [18,19]. In addition, the method is proposed to amplify the fault-to-noise ratio and the characteristics difference through applying a sliding window. This paper proposes techniques to improve the deep neural network when the fault magnitude is small and the fault characteristics difference is inapparent in close-loop control systems. The method proposed in this paper has a better identification effect on tagged data. The tagged training data is used for studying with the supervision for each mode, and then the test data is fed into the trained deep neural network for fault mode classification, and then fault identification for closed-loop control system can be realized.

The main contributions of this paper are as follows: Firstly, the propagation mechanism of sensor failure and process fault in closed-loop control systems, and their influence on system output variables are analyzed theoretically. Secondly, a deep neural network fault identification method based on sliding window is proposed, and the advantage of this method for identifying strong coupled modes is demonstrated. Thirdly, the improved deep neural network method is applied to the numerical simulation system, the Tennessee industrial system and the satellite attitude control system to verify fault identification in closed-loop control systems. Finally, the paper also makes contributions for the problem of how to process the data from sensors.

The remainder of this paper is organized as follows: in Section 2, the propagation mechanism of faults and their influence on output variables in close-loop control systems are analyzed theoretically. In Section 3, traditional distance-based and angle-based fault identification methods are introduced. In Section 4, by introducing the deep neural network structure, a deep neural network fault identification method based on sliding window is proposed. Section 5 presents experiments on the numerical simulation model systems, the Tennessee industrial systems and the satellite attitude control systems, and then results are reported and discussed. Finally, the conclusion is drawn in Section 6.

2. Closed-Loop Control System and Fault Propagation

2.1. Closed-Loop Control System Model

Firstly, a discrete linear model for a control system is established as follows [20,21]:

$$\left\{\begin{array}{c}{\mathit{x}}_{k+1}=\mathit{A}{\mathit{x}}_k+\mathit{B}{\mathit{u}}_k+{\mathit{f}}_k^x+{\mathit{w}}_k\hfill \\ {\mathit{y}}_k=\mathit{C}{\mathit{x}}_k+{\mathit{f}}_k^y+{\mathit{v}}_k\hfill \end{array}\right.,$$

(1)

where ${\mathit{x}}_k\in {\mathbb{R}}^{n_x}$ is the state vector, ${\mathit{u}}_k\in {\mathbb{R}}^{n_u}$ is the control input vector, ${\mathit{y}}_k\in {\mathbb{R}}^{n_y}$ is the measurement output vector, ${\mathit{w}}_k\in {\mathbb{R}}^{n_x}$ and ${\mathit{v}}_k\in {\mathbb{R}}^{n_y}$ are the independent process noise and the measurement noise with zero mean, and the covariance matrices are the normal distribution of $\mathit{Q}\in {\mathbb{R}}^{n_x\times n_x}$ and $\mathit{R}\in {\mathbb{R}}^{n_y\times n_y}$ respectively; ${\mathit{f}}_k^x\in {\mathbb{R}}^{n_x}$ and ${\mathit{f}}_k^y\in {\mathbb{R}}^{n_y}$ are the process fault and the sensor fault respectively (When the system operates normally, i.e., ${\mathit{f}}_k^x=\mathbf{0}$ and ${\mathit{f}}_k^y=\mathbf{0}$), $\mathit{A},\mathit{B},\mathit{C}$ are the parameter matrices with appropriate dimensions, which are all unknown.

The basic principle of a closed-loop control system is that the system constructs a deviation vector between the measurement output vector and its expected output vector, uses the deviation feedback on the control input vector, and then introduces a closed-loop control rate to reduce the influence from external interference.

When adding the closed control loop into model (1), the closed-loop control system equation can be written as follows:

$${\mathit{x}}_{k+1}=\mathit{A}{\mathit{x}}_k+\mathit{B}{\mathit{u}}_{k+1}+{\mathit{f}}_k^x+{\mathit{w}}_{k+1}$$

(2a)

$${\mathit{y}}_k=\mathit{C}{\mathit{x}}_k+{\mathit{f}}_k^y+{\mathit{v}}_k$$

(2b)

$${\mathit{x}}_{k+1}^c={\mathit{A}}^c{\mathit{x}}_k^c+{\mathit{B}}^c{\mathit{y}}_k$$

(2c)

$${\mathit{u}}_{k+1}={\tilde{\mathit{u}}}_{k+1}+{\mathit{C}}^c\left({\tilde{\mathit{x}}}_{k+1}^c-{\mathit{x}}_{k+1}^c\right)+{\mathit{D}}^c\left({\tilde{\mathit{y}}}_k-{\mathit{y}}_k\right),$$

(2d)

where ${\mathit{x}}_k^c\in {\mathbb{R}}^{n_{x^c}}$ is the state control vector, ${\tilde{\mathit{u}}}_{k+1}\in {\mathbb{R}}^{n_u}$ is the expected control input vector, ${\tilde{\mathit{x}}}_k^c\in {\mathbb{R}}^{n_{x^c}}$ is the expected state control vector, ${\tilde{\mathit{y}}}_k\in {\mathbb{R}}^{n_y}$ is the expected measurement output vector, ${\mathit{A}}^c,{\mathit{B}}^c,{\mathit{C}}^c,{\mathit{D}}^c$ are the parameter matrices with appropriate dimensions. As can be seen from Model (2), the closed-loop control system feeds back the deviation vector (i.e., ${\tilde{\mathit{y}}}_k-{\mathit{y}}_k$) to the control input vector ${\mathit{u}}_{k+1}$ (i.e., the control input vector at the next $k+1$-th moment), thereby adjusting the operation situation of the closed-loop control system.

2.2. Propagation Mechanism of Sensor Fault

Suppose that the symbol $\widehat{·}$ represents the corresponding vector when the closed-loop control system operates normally. Consider that a sensor fault occurs at the k-th moment (i.e., ${\mathit{f}}_k^y\ne \mathbf{0}$), then the measurement output vector ${\mathit{y}}_k$ can be represented as

$${\mathit{y}}_k=\mathit{C}{\mathit{x}}_k+{\mathit{f}}_k^y+{\mathit{v}}_k={\widehat{\mathit{y}}}_k+{\mathit{f}}_k^y.$$

(3)

According to Equation (2c), the state control vector ${\mathit{x}}_{k+1}^c$ is affected by the measurement output vector as follows:

$${\mathit{x}}_{k+1}^c={\mathit{A}}^c{\mathit{x}}_k^c+{\mathit{B}}^c{\widehat{\mathit{y}}}_k+{\mathit{B}}^c{\mathit{f}}_k^y={\widehat{\mathit{x}}}_{k+1}^c+{\mathit{B}}^c{\mathit{f}}_k^y.$$

(4)

According to Equation (2d), the control input vector ${\mathit{u}}_{k+1}$ is affected by both the measurement output vector and the state control vector as follows:

$${\mathit{u}}_{k+1}={\tilde{\mathit{u}}}_{k+1}+{\mathit{C}}^c\left({\tilde{\mathit{x}}}_{k+1}^c-{\mathit{x}}_{k+1}^c\right)+{\mathit{D}}^c\left({\tilde{\mathit{y}}}_k-{\mathit{y}}_k\right)={\widehat{\mathit{u}}}_{k+1}-\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right){\mathit{f}}_k^y.$$

(5)

Then, the state vector at the $k+1$-th moment (i.e., ${\mathit{x}}_{k+1}$) can be obtained as follows:

$${\mathit{x}}_{k+1}=\mathit{A}{\mathit{x}}_k+\mathit{B}{\mathit{u}}_{k+1}+{\mathit{w}}_{k+1}={\widehat{\mathit{x}}}_{k+1}-\mathit{B}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right){\mathit{f}}_k^y.$$

(6)

According to the influence relationship above, the sensor fault at k moment (i.e., ${\mathit{f}}_k^y$) will ultimately affect the measurement output vector at the $k+1$-th moment (i.e., ${\mathit{y}}_{k+1}$) as follows:

$${\mathit{y}}_{k+1}=\mathit{C}{\mathit{x}}_{k+1}+{\mathit{v}}_{k+1}={\widehat{\mathit{y}}}_{k+1}-\mathit{CB}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right){\mathit{f}}_k^y.$$

(7)

Let

$$\mathit{H}=\mathit{CB}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right).$$

(8)

Make the eigenvalue decomposition (ED) on the matrix $\mathit{H}\in {\mathbb{R}}^{n_y\times n_y}$, s.t.

$$\mathit{H}={\mathit{Q}}_H{\mathsf{\Lambda }}_H{\mathit{Q}}_H^{-1},$$

(9)

where ${\mathit{Q}}_H$ is an eigenvector matrix, ${\mathsf{\Lambda }}_H=\left[\begin{array}{cc}\mathbf{\Sigma }& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right]\in {\mathbb{R}}^{n_y\times n_y}$, $\mathbf{\Sigma }=diag({\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_l)$, ${\lambda }_i(i=1,2,\cdots ,l)$ is the eigenvalues of $\mathit{H}$ and $l=rank\left(\mathit{H}\right)$.

Remark 1.

From the propagation process of sensor fault in a closed-loop control system above, if the sensor fault occurs at the k-th moment (i.e., ${\mathit{f}}_k^y\ne \mathbf{0}$), the influence of this sensor fault on the measurement output vector at the $k+1$-th moment (i.e., ${\mathit{y}}_{k+1}$) is $-{\mathit{Q}}_H{\mathsf{\Lambda }}_H{\mathit{Q}}_H^{-1}{\mathit{f}}_k^y$. If the closed-loop control system is selected properly, the eigenvalues ${\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_l$ of $\mathit{H}$ are all less than 1, and then the fault amplitude will be reduced at the $k+1$-th moment.

In addition, if this sensor fault ${\mathit{f}}_k^y$ propagates until the $k+p$th moment, the sensor fault amplitude occurring at the k-th moment will reduced to ${\left(-1\right)}^p{\mathit{Q}}_H{\mathsf{\Lambda }}_H^p{\mathit{Q}}_H^{-1}{\mathit{f}}_k^y$. Since

$$\underset{p\to \infty }{lim}{\mathsf{\Lambda }}_H^p=\mathbf{0},$$

(10)

the closed-loop control system can gradually reduce the impact of sensor fault on the system due to the closed control loop.

Remark 2.

Consider that the sensor fault occurs both at the k-th moment and the $k+1$-th moment, and the fault amplitudes are ${\mathit{f}}_k^y$ and ${\mathit{f}}_{k+1}^y$, respectively. From the propagation process of the sensor fault in the closed-loop control system above, the influence of the sensor fault with these two moments on the measurement output vector at the $k+1$-th moment is ${\mathit{f}}_{k+1}^y-\mathit{CB}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right){\mathit{f}}_k^y={\mathit{f}}_{k+1}^y-{\mathit{Hf}}_k^y$. If the closed-loop control system is selected properly to make ${\mathit{f}}_{k+1}^y-{\mathit{Hf}}_k^y<{\mathit{f}}_{k+1}^y$, and then the influence of the sensor fault on the closed-loop control system will also be reduced at the $k+1$-th moment.

2.3. Propagation Mechanism of Process Fault

Similar to Section 2.2, the process fault amplitude will decrease because of the closed control loop. The detail process is as Appendix A.

Remark 3.

From the propagation process of the process fault in the closed-loop control system above, if the process fault occurs at the kth moment (i.e., ${\mathit{f}}_k^x\ne \mathbf{0}$), the influence of this process fault on the measurement output vector at the $k+1$-th moment (i.e., ${\mathit{y}}_{k+1}$) is ${\mathit{Cf}}_k^x$ and $\mathit{C}\left(\mathit{A}-\mathit{B}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right)\mathit{C}\right){\mathit{f}}_k^x$ in the $k+2$-th moment. Based on Equations (A2) and (A6), if the closed-loop control system is selected properly, the eigenvalues ${\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_t$ of $\mathit{G}$ are all less than 1, and then the impact of the process fault on the measurement output vector will decrease gradually.

In addition, if this process fault ${\mathit{f}}_k^x$ propagates until to the $k+p$-th moment, the process fault amplitude occurring at the k-th moment will be reduced to ${\mathit{Q}}_G{\mathsf{\Lambda }}_G^{p-1}{\mathit{Q}}_G^{-1}{\mathit{f}}_k^x$. Since

$$\underset{p\to \infty }{lim}{\mathsf{\Lambda }}_G^{p-1}=\mathbf{0},$$

(11)

the closed-loop control system can gradually reduce the impact of the process fault on the system.

Remark 4.

Consider that the sensor fault occurs both at the k-th moment and the $k+1$-th moment, and the fault amplitudes are ${\mathit{f}}_k^y$ and ${\mathit{f}}_{k+1}^y$, respectively. The influence of the process fault with both the k-th moment and the $k+1$-th moment on the measurement output vector at the $k+2$-th moment is ${\mathit{Cf}}_{k+1}^x+\mathit{C}\left(\mathit{A}-\mathit{B}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right)\mathit{C}\right){\mathit{f}}_k^x$. If the closed-loop control system is selected properly to make ${\mathit{Cf}}_{k+1}^x+\mathit{C}\left(\mathit{A}-\mathit{B}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right)\mathit{C}\right){\mathit{f}}_k^x<{\mathit{Cf}}_{k+1}^x$, and then the influence of the process fault on the closed-loop control system will also be reduced.

Remark 5.

The influence of a sensor fault or a process fault on the system is the change of the measurement output vector. In a closed-loop control system, the influence of the two fault modes (sensor fault and process fault) on the system are similar. For example, the sensor fault propagates inside the system with the fault amplitude ${\left(-\mathit{CB}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right)\right)}^p{\mathit{f}}_k^y$, and the process fault propagates inside the system with the fault amplitude $\mathit{C}{\left(\mathit{A}-\mathit{B}\left({\mathit{C}}^c{\mathit{B}}^c+{\mathit{D}}^c\right)\mathit{C}\right)}^{p-1}{\mathit{f}}_k^x$. The fault characteristics difference of these two fault modes is inapparent; therefore, the two types of faults are difficult to distinguish, which increases the difficulty of fault identification.

3. Fault Identification Based on Distance and Angle

3.1. Distance Identification Method

Data from different modes tend to differ from each other, and fault identification can be performed by distance characteristics. By calculating the distance between the test data and the data from another mode, the test data can be identified to the closest mode.

Data standardization is necessary to make data nondimensionalization both for the test data and the training data. Firstly, calculate the mean and the covariance for the data with the kth mode respectively as follows:

$$\left\{\begin{array}{c}{\overline{\mathit{A}}}_k=\frac{1}{n_k}{\mathit{A}}_k\hfill \\ {\mathit{S}}_k=\frac{1}{n_k-1}{\left({\mathit{A}}_k-{\overline{\mathit{A}}}_k\right)}^{\mathrm{T}}\left({\mathit{A}}_k-{\overline{\mathit{A}}}_k\right)\hfill \end{array}\right.,$$

(12)

where the subscript indicates the data from the kth mode, and the sample number is $n_k$ in this mode. ${\mathit{A}}_k$ indicates the training data of this mode, ${\overline{\mathit{A}}}_k$ and ${\mathit{S}}_k$ are the mean and covariance of ${\mathit{A}}_k$, respectively. Since ${\mathit{S}}_k$ is a real symmetric matrix, its eigenvalue decomposition can be represented as follows:

$${\mathit{S}}_k={\mathit{V}}_k{\mathsf{\Lambda }}_k{\mathit{V}}_k^{\mathrm{T}},$$

(13)

where ${\mathsf{\Lambda }}_k$ is a diagonal matrix, and ${\mathit{V}}_k$ is an orthogonal matrix. Assume that the test data is $\mathit{x}$, and let

$${\mathit{z}}_k={\mathsf{\Lambda }}_k^{-1/2}{\mathit{V}}_k^{\mathrm{T}}\left(\mathit{x}-{\overline{\mathit{A}}}_k\right).$$

(14)

Then the Hotelling’s $T^2$ statistic can be obtained as follows:

$$T_k^2={\mathit{z}}_k^{\mathrm{T}}{\mathit{z}}_k.$$

(15)

This $T^2$ statistic obeys the F-distribution, and then its threshold can be represented as follows

$$T_{\alpha ,k}^2=\frac{m\left(n_k-1\right)\left(n_k+1\right)}{n_k\left(n_k-m\right)}{F}_{\alpha }\left(m,n_k-m\right)$$

(16)

where m is the dimension of the training data, $\alpha$ is the confidence level, and $F_{\alpha }\left(m,n_k-m\right)$ is the corresponding quantile for F-distribution. Thus, the following criterion can be proposed to realize fault identification.

Criterion 1: if $T_k^2\le T_{\alpha ,k}^2$, then this test data belongs to the kth mode; Otherwise, this test data does not belong to the kth mode.

3.2. Angle Identification Method

The distance identification mentioned in Section 3.1 usually works in the occasions that all modes separate from each other. It is not applicable for the data with some trend characteristics [12]. In this case, it can be considered to use angle characteristics for fault identification [22].

As it is shown in Figure 1, ${\mathit{r}}_i$ and ${\mathit{r}}_j$ represent the direction of the ith mode and the jth mode, respectively, and $\mathit{r}$ is the direction of the test data, $\tilde{\mathit{r}}=\mathit{r}-{\mathit{r}}_i$.

Let $\theta \left({\mathit{r}}_i,{\mathit{r}}_j\right)$ be the angle between ${\mathit{r}}_i$ and ${\mathit{r}}_j$, and calculate as follows:

$$\theta \left({\mathit{r}}_i,{\mathit{r}}_j\right)=arccos\left({\mathit{r}}_i^{\mathrm{T}}{\mathit{r}}_j/\left(∥{\mathit{r}}_i∥∥{\mathit{r}}_j∥\right)\right)$$

(17)

It is obvious that the larger $\theta \left({\mathit{r}}_i,{\mathit{r}}_j\right)$ is, the fault is easier to be identified. For any direction of the test data $\mathit{r}$, the angle between the test data and the kth mode is calculated as follows:

$$\theta \left(\mathit{r},{\mathit{r}}_k\right)=arccos\left({\mathit{r}}^{\mathrm{T}}{\mathit{r}}_k/\left(∥\mathit{r}∥∥{\mathit{r}}_k∥\right)\right).$$

(18)

Thus, the following criterion can be proposed to realize fault identification.

Criterion 2: Let

$$k_0=\underset{k=1,\cdots ,n_k}{argmin}\left\{\theta \left(\mathit{r},{\mathit{r}}_k\right)\right\}.$$

(19)

and this test data belongs to the $k_0$-th mode.

Remark 6.

From the distance identification and angle identification mentioned in this section, it can be seen that if some modes are coupled with each other, one test data may belong to multiple modes simultaneously. Therefore, the methods based on the distance or angle judgment could not properly identify the coupled fault modes from closed-loop control systems.

4. Deep Neural Network-Based Fault Identification for Closed-Loop Control System

4.1. Deep Neural Network Principle

Neural network is a branch of machine learning, and it has been widely used in various fields. However, the traditional single-layer neural network has a limitation of not meeting the classification requirements of complex systems. To solve this problem, deep neural network was proposed in literature [23].

Deep neural network is a multi-layer neural network with two or more hidden layers. Compared with the traditional single-layer neural network, it has the ability of enhancing the fitting effect to make classification and to identify complex data. Based on this, it is possible to train and capture data characteristics of each mode better, and to increase the identification rate of different modes.

A deep neural network is determined by data training with certain learning rules, and then the test data is input through the entire neural network to obtain the output vector [24]. The learning rules are the key in the network establishment process. Here the learning rules are introduced briefly firstly. Deep neural network is a multi-layer perceptron with multiple hidden layers. The network structure is shown in Figure 2.

Where $\mathit{a}={\left[a_1,a_2,\cdots ,a_n\right]}^{\mathrm{T}}$ is the input layer of the deep neural network, n is the number of nodes in the input layer, $\mathit{b}={\left[b_1,b_2,\cdots ,b_l\right]}^{\mathrm{T}}$ is the output layer of the deep neural network, l is the number of nodes in the output layer, in which the input layer and the output layer are different from that in Equation (1). Here, the input layer is the control input vector and the measurement output vector, and the output layer is the preset tag. In addition, ${\mathit{W}}_i,i=1,2,\cdots ,q$ is the weight matrix, ${\mathit{\theta }}_i,i=1,2,\cdots ,q$ is the bias terms vector, and ${\phi }_i,i=1,2,\cdots ,q$ is the activation function and q is the number of the hidden layers. Each hidden layer computes the activations on condition that the previous layer has been computed. Denote the input vector from the jth hidden layer (which is also the output vector of the $j-1$th hidden layer) as ${\mathit{v}}_{j-1}$, and then the output vector of the jth hidden layer can be computed as follows:

$${\mathit{v}}_j={\phi }_j\left({\mathit{W}}_j{\mathit{v}}_{j-1}+{\mathit{\theta }}_j\right),$$

(20)

where ${\mathit{W}}_j$ and ${\mathit{\theta }}_j$ are the weight matrix and bias vector of the jth layer, and ${\phi }_j\left(·\right)$ is the predefined nonlinear activation function. Some functions used often in the literature include the sigmoid function $\phi \left(\mathit{v}\right)=1/\left(1+\exp \left(-\mathit{v}\right)\right)$, the rectifier function $\phi \left(\mathit{v}\right)=max\left(\mathbf{0},\mathit{v}\right)$, and the tanh function $\phi \left(\mathit{v}\right)=\left(exp\left(2\mathit{v}\right)-1\right)/\left(exp\left(2\mathit{v}\right)+1\right)$, etc.

In the mode identification region, the softmax activation function is usually used in the last layer of a deep neural network, and its function is as follows:

$$\phi \left(v_i\right)=\frac{e^{v_i}}{{\displaystyle \sum _{k=1}^M}{e}^{v_k}},$$

(21)

where $v_i$ is the weighted sum of the ith output node, M is the number of output nodes. Equation (21) shows a perfect property that the sum of the outputs is 1, i.e.,

$$\phi \left(v_1\right)+\phi \left(v_2\right)+\phi \left(v_3\right)+\cdots +\phi \left(v_M\right)=1,$$

(22)

where $0\le \phi \left({\mathit{v}}_k\right)\le 1,k=1,2,\cdots ,M$, and $\phi \left(v_i\right)$ can be regarded as a probability for identifying the mode with the maximum probability.

In the training process, the weight matrix $\mathit{W}$ and the bias vector $\mathit{\theta }$ can be updated by using of the gradient descent method, as shown in Equation (23):

$$\left\{\begin{array}{c}E=\frac{1}{m}{\displaystyle \sum _{t=1}^m}E\left(t\right)\hfill \\ {\mathit{W}}_{ij}^{\left(k+1\right)}={\mathit{W}}_{ij}^{\left(k\right)}-\alpha \frac{\partial E}{\partial {\mathit{W}}_{ij}^{\left(k\right)}}\hfill \\ {\mathit{\theta }}_i^{\left(k+1\right)}={\mathit{\theta }}_i^{\left(k\right)}-\alpha \frac{\partial E}{\partial {\mathit{\theta }}_i^{\left(k\right)}}\hfill \end{array}\right.,$$

(23)

where $\alpha$ is the leaning efficiency, k is the iteration number, m is the number of training samples, $E\left(t\right)$ is the training error of the tth training sample, and $\mathit{E}$ is the mean value of $E\left(t\right)$. The training error is determined by the distance between the theoretical tag $\mathit{\beta }$ and the actual output $\mathit{b}$ of the deep neural network as follows:

$$E\left(t\right)={∥\mathit{b}\left(t\right)-\mathit{\beta }\left(t\right)∥}_2$$

(24)

If the training error is less than the preset value or the total iteration number reaches the preset number, the training process ends and the network is established.

4.2. Improved Deep Neural Network

At present, deep neural network is widely used in many areas, such as image identification, face identification, etc. Sometimes it can be more accurate than that of artificial identification. However, most of the successful cases are aimed at static data, while data from a closed-loop control system is usually dynamic, that is, the current data is affected by previous data or by a period of time. It is difficult to extract the system characteristics only by a single sample data accurately, which is one of the limitations of the traditional deep neural network.

In this paper, for dynamic data from a closed-loop control system, a sliding window can be set to train data with several moments together. The size of the sliding window can be determined empirically or be judged by the correlation of the training data.

Due to the influence of the closed control loop, external disturbance is weakened. Therefore, it is hard to determine whether the fault has occurred from a single data, and even harder for fault identification. An advantage for setting the sliding window is that the characteristics difference between different modes can be accumulated and increased, and then the fault identification rate can be improved.

Take two fault modes as an example, and assume that ${\mathit{y}}_k^i={\stackrel{⌢}{\mathit{y}}}_k^i+e_k^i$ is the measurement output vector of the ith mode and ${\mathit{y}}_k^j={\stackrel{⌢}{\mathit{y}}}_k^j+e_k^j$ is the measurement output vector of the jth mode in the kth moment, where ${\stackrel{⌢}{\mathit{y}}}_k^i$ and ${\stackrel{⌢}{\mathit{y}}}_k^j$ are the output vectors without noise of the ith mode and the jth mode, respectively. Then let ${\mathit{r}}_k={\mathit{y}}_k^i-{\mathit{y}}_k^j$, ${\mathit{z}}_k={\stackrel{⌢}{\mathit{y}}}_k^i-{\stackrel{⌢}{\mathit{y}}}_k^j$, and ${\mathit{e}}_k={\mathit{r}}_k-{\mathit{z}}_k$. Assume that the ${\mathit{e}}_k$ obeys normal distribution

$${\mathit{e}}_k\sim N\left(0,\mathbf{\Sigma }\right),$$

(25)

where ${\mathbf{\Sigma }}_k$ is noise covariance. Then $r_k$ obeys the distribution as follows:

$${\mathit{r}}_k\sim N\left({\mathit{z}}_k,\mathbf{\Sigma }\right).$$

(26)

Since the ratio (i.e., $\left|{\mathit{z}}_k/\sqrt{\mathbf{\Sigma }}\right|$) between the characteristics difference and the noise is small, it is easily overwhelmed by the noise. This means it is difficult to identify the fault mode accurately. Accumulate the characteristics difference with the sliding window, and then:

$${\tilde{\mathit{r}}}_k=\sum _{j=k-N_0+1}^k{\mathit{r}}_j,$$

(27)

where N₀ is the window length. The characteristics difference ${\tilde{\mathit{r}}}_k$ obeys the distribution as follows:

$${\tilde{\mathit{r}}}_k\sim N\left(\sum _{j=k-N_0+1}^k{\mathit{z}}_j,N_0\mathbf{\Sigma }\right).$$

(28)

The ratio $\left|{\displaystyle \sum _{j=k-N_0+1}^k}{\mathit{z}}_j/\sqrt{N_0\mathbf{\Sigma }}\right|$ between the characteristics difference and the noise increases significantly. Therefore, the characteristic difference between two modes can be increased with a sliding window, and the fault identification rate may be increased.

4.3. Fault Identification Step Based on Improved Neural Network

According to the discussions in Section 2, due to the adjustment function of a closed-loop control system, the coupling effect among different modes becomes stronger, which brings greater difficulty for fault identification. Thus, in Section 4.2, a fault identification method based on improved deep neural network is proposed. The steps for the proposed fault identification method are given as follows:

Step 1. Preprocess the tagged training data: set an appropriate sliding window size, and then rearrange the data according to the sliding window;

Step 2. Select an appropriate network structure [25]: preset the number of hidden layers, the node number of the input layer, the output layer and each hidden layer to establish a deep neural network structure;

Step 3. Feed the training data into the deep neural network and select an appropriate activation function for the supervised training process;

Step 4. Feed the test data into the trained deep neural network, and record the output result to identify the test data;

Step 5. Calculate the identification rate of the whole test data and each data mode, respectively, and obtain the relationship between the identification rate and the window size.

5. Simulations

In this section, three simulations that verify the feasibility of the proposed fault identification approach in closed-loop control systems based on the improved deep neural network are included. The simulation cases include the numerical simulation model, the Tennessee Eastman Process (TEP) based on the industrial process simulation model and the satellite attitude control system (SACS).

5.1. Case 1: Numerical Simulation Model

The total amount of simulation data is 30,000 sets according to Equation (2). There are three modes, i.e., a normal mode and two fault modes. The data sample number of each mode is 10,000, and the data dimension is 1.

The first fault mode is process fault and the second is sensor fault, of which the fault amplitude are 0.3 and −0.3, respectively. The parameter matrices are $\mathit{A}=0.5{\mathit{I}}_{1\times 1}$, $\mathit{B}=0.5{\mathit{I}}_{1\times 1}$, $\mathit{C}={\mathit{I}}_{1\times 1}$, ${\mathit{A}}^c=0.7{\mathit{I}}_{1\times 1}$, ${\mathit{B}}^c=0.3{\mathit{I}}_{1\times 1}$, ${\mathit{C}}^c=0.5{\mathit{I}}_{1\times 1}$, ${\mathit{D}}^c=0.5{\mathit{I}}_{1\times 1}$. The state noise and the measurement noise are independent of each other, and both obey the normal distribution with zero mean and the standard deviation is 0.1. The initial state ${\mathit{x}}_1=10$, and the constant variables $\tilde{\mathit{u}}=10$, ${\tilde{\mathit{x}}}^c=10$, $\tilde{\mathit{y}}=10$. Then the numerical simulation data is obtained, as shown in Figure 3.

Firstly, examine the case that there is no closed control loop, i.e., the control input vector of the system is given artificially. To simulate an open-loop control system, remove the closed control loop, and set the expected control input vector $\tilde{\mathit{u}}$, then the simulation data set is obtained, as shown in Figure 4.

As can be seen from Figure 3 and Figure 4, the difference between the measurement output vectors of the closed-loop system of any two modes is much smaller than that of the open-loop system. The measurement mean of the normal data, the first fault data and the second fault data were 10.001, 10.300, 9.850 in the closed-loop control system, respectively. Meanwhile, they were 9.998, 10.602, 9.702 in the open-loop control system, respectively. When taking the control input vector of the closed-loop control system as the horizontal axis, the measurement output vector as the vertical axis, the simulation data of three modes can be presented in Figure 5.

From Figure 5, the normal data and the second mode data are coupled with each other. If the control input and the measurement output of the current time are directly used as the input nodes of the deep neural network without a sliding window, it will be difficult to identify the fault mode, especially in the coupling parts. Therefore, the sliding window is necessary and used as follows. Three hidden layers are adopted to establish the deep neural network. The total iteration number is 2000. Both the first hidden layer with 10 nodes and the second with five nodes apply Sigmoid function, while the third with three nodes applies the softmax function [25]. We used 70% of data with each mode to train the deep neural network, and the remaining 30% of the data was used as a test dataset.

Table 1. The relationship between the sliding window size and fault identification indicators.

Window Size	Identification Rate	The Normal Data	The First Fault Data	The Second Fault Data	The Accuracy
No window	94.622%	91.467%	99.600%	92.800%	0.0250
2	96.066%	93.462%	99.733%	95.000%	0.0186
3	97.044%	94.963%	99.867%	96.300%	0.0141
4	97.766%	95.896%	99.933%	97.467%	0.0106
5	98.366%	96.796%	99.933%	98.367%	0.00788
6	98.733%	97.095%	99.933%	99.167%	0.00589

lists the relationship between the sliding window size and the identification rate, as well as the training accuracy.

From Table 1, the identification rate of various modes increases and the training accuracy is more accurate with the increase of the window size. It shows that the method using sliding windows to improve deep neural networks is effective.

The identification rates with the traditional methods introduced in Section 3 are also shown in

Table 2. The identification rates of various methods.

Fault Identification Method	Identification Rate	The Normal Data	The First Fault Data	The Second Fault Data
The improved DNN	98.733%	97.095%	99.933%	99.167%
The distance-based method	77.80%	66.03%	91.50%	75.87%
The angle-based method	77.01%	66.47%	90.27%	74.30%

. Obviously, compared with traditional distance-based or angle-based identification methods, the improved deep neural network fault identification method improves the fault identification rate effectively.

5.2. Case 2: TEP Simulation Model

The Tennessee-Eastman Process (TEP) system is based on an industrial process simulation model, which was created by an American company named Eastman in 1993. TEP can provide a realistic and usable industrial process for evaluating the process monitoring and control methods [19,26]. A large amount of literature refers to it as a data source for research on control, optimization, process monitoring, and fault diagnosis and so on. Figure 6 shows the flow chart of TEP industrial equipment.

The TEP system consists of five parts: reactor, condenser, compressor, steam/liquid separator and stripper. The equipment is operated under a closed-loop controller, and a total of 41 variables are collected. In this experiment, three modes were simulated, namely normal data, A/C feed ratio fault, and B-component feed fault. The simulation time was set to 50 h, and the sampling period was 0.01 h. Hence, 5000 sets of data were sampled in total. The variables in the three modes for the reactor are plotted in Figure 7, including reactor feed rate, reactor pressure, reactor grade and reactor temperature, respectively. The data can be gotten from the webpage: http://depts.washington.edu/control/LARRY/LE/download.html.

The first 3500 sets of data in each mode were trained in the deep neural network, and the remaining 1500 sets of data were used for test. The total number of iterations was 100. There were three hidden layers in the deep neural network. The first is with 100 nodes and the second is with 50 nodes. Both layers apply Sigmoid function, while the third with three nodes applies the softmax function.

Table 3. The relationship between the sliding window length and fault identification indicators.

Window Size	Identification Rate	The Normal Data	The First Fault Data	The Second Fault Data	The Accuracy
No window	99.756%	99.533%	99.867%	99.867%	0.00562
2	99.822%	99.733%	99.933%	99.800%	0.00196
3	99.911%	99.933%	99.867%	99.933%	0.00167

shows the relationship between the window size and the identification rate, as well as the training accuracy.

Similarly, from Table 3, the identification rate of various modes increases and the training accuracy is more accurate with the increase of the window size. The identification rates with the traditional methods are shown in

Table 4. The identification rates of various methods.

Fault Identification Method	Identification Rate	The Normal Data	The First Fault Data	The Second Fault Data
The improved DNN	99.911%	99.933%	99.867%	99.933%
The distance-based method	94.76%	95.47%	93.07%	95.73%
The angle-based method	62.98%	48.33%	47.67%	92.93%

. Obviously, compared with the traditional distance-based or angle-based identification methods, the improved deep neural network fault identification method improves the fault identification rate effectively. It indicates that the proposed method can be applied for fault identification in TEP systems and other industrial systems.

5.3. Case 3: SACS Simulation Model

As shown in

Table 5. The identification rates of various methods.

Variable	Sensor	Variable	Sensor
$e_r$	earth sensor in the roll direction	$g_r$	gyroscope in the roll direction
$e_p$	earth sensor in the pitch direction	$g_p$	gyroscope in the pitch direction
$s_r$	sun sensor in the roll direction	$g_y$	gyroscope in the yaw direction
$s_p$	sun sensor in the pitch direction

, there are seven variables in the satellite attitude control system (SACS). They are earth sensors in the roll ($e_r$) and pitch ($e_p$) directions, sun sensors in the roll ($s_r$) and pitch ($s_p$) directions, gyroscopes in the roll ($g_r$), pitch ($g_p$) and yaw ($g_y$) directions. Three modes of data are collected, which are the normal, $e_p$ fault and $s_p$ fault. The data set is shown as Figure 8.

There were 501 sets of the normal data and 143 sets of both fault data. Half of the each mode data is used to train the deepneural network, and the structure and the activation functions are the same as the Section 5.2’s. It is shown that the identification rate of various modes increases with the increase of the window size in the

Table 6. The relationship between the sliding window length and fault identification indicators.

Window Size	Identification Rate	The Normal Data	${\mathit{e}}_{\mathit{p}}$ Fault Data	${\mathit{s}}_{\mathit{p}}$ Fault Data
No window	96.610%	99.401%	82.707%	100.00%
2	99.608%	100.00%	98.496%	99.248%
3	99.739%	100.00%	98.496%	100.00%

. The identification rates with the traditional methods are shown in

Table 7. The identification rates of various methods.

Fault Identification Method	Identification Rate	The Normal Data	${\mathit{e}}_{\mathit{p}}$ Fault Data	${\mathit{s}}_{\mathit{p}}$ Fault Data
The improved DNN	99.739%	100.00%	98.496%	100.00%
The distance-based method	83.31%	77.25%	89.47%	100.00%
The angle-based method	76.79%	64.47%	100.00%	100.00%

. Obviously, compared with the traditional distance-based or angle-based identification methods, the improved deep neural network fault identification method improves the fault identification rate effectively in the SACS.

6. Conclusions

In this paper, fault identification for closed-loop control systems has been studied. The characteristics of closed-loop control systems were analyzed, the propagation mechanism of faults inside of the system was researched and their influence on measurement output vectors theoretically. The difficulty of fault identification for closed-loop control systems was revealed. The deep neural network method has been proposed. Then the sliding window technique has been used to improve the deep neural network method, and its feasibility for fault identification for closed-loop control systems has been proved. Finally, numerical simulations, the Tennessee-Eastman system and the satellite attitude control system have been used to verify the feasibility of the fault identification of closed-loop control systems.