Towards Enhanced Performance of Neural-Network-Based Fault Detection Using an Sequential D-Optimum Experimental Design

Abstract

Increasing expectations of industrial system reliability require development of more effective and robust fault diagnosis methods. The paper presents a framework for quality improvement on the neural model applied for fault detection purposes. In particular, the proposed approach starts with an adaptation of the modified quasi-outer-bounding algorithm towards non-linear neural network models. Subsequently, its convergence is proven using quadratic boundedness paradigm. The obtained algorithm is then equipped with the sequential D-optimum experimental design mechanism allowing gradual reduction of the neural model uncertainty. Finally, an emerging robust fault detection framework on the basis of the neural network uncertainty description as the adaptive thresholds is proposed.

1. Introduction

The Artificial Neural Networks (ANNs) [1,2] have an established position among the data-driven techniques which are applied to identify non-linear dynamic systems [3]. It follows from the fact that such computational approach can be used when a knowledge about the system in the form of physical description is not available. Thanks to learning capabilities, in a relatively easy way without large effort of work a neural model of quite complicated system may be obtained. Moreover, generalisation properties of the ANNs allow to use such a model in several practical applications for different data sets not used during system identification. The above mentioned properties and especially the ability of modeling of complex non-linear dynamic systems cause that the ANNs stand out among the Fault Diagnosis (FD) methods within the analytical/model-based approaches, signal-based approaches, knowledge-based approaches can be distinguished [4,5,6,7,8,9,10]. To the task of the FD methods belong the fault detection, isolation and identification [9,11] but the fault detection seems to be essentially crucial because it determines appropriate performing of the fault isolation and identification. Commonly the fault detection relies on the comparison of the diagnosed system with its nominal model [11,12]. Unfortunately, the efficacy of such method strongly depends on the quality of neural model. Therefore, all kinds of efforts to its improvement should be made. In practice the researchers usually only focus on the problem of the proper neural model architecture selection or/and training algorithms improvements [2,3,13]. However, it is worth noting that very rarely the problem of appropriate data selection in the context of the ANN quality improvement is taken into consideration. Moreover, assuming that the model is rarely an ideal copy of the diagnosed system in the nominal fault-free state what often results from the measurement noise included in the data used during system identification, the mathematical description of its uncertainty [8] should be delivered. The omission of such factors as the model uncertainty, measurements noise and other disturbances during FD may lead to undetected faults or false alarms what is not accepted in the modern industrial systems. In other words, the robust FD system against such factors should be designed. To calculate the uncertainty of the ANN the class of the Outer Bounding Ellipsoid (OBE) algorithms [14,15,16,17] can be used. There is of course a wide spectrum of approaches which can be used to asses model quality in various contexts [18,19,20,21]. However, the aim of this paper is to obtain a model with possibly small output variance, which is associated with the so-called D-optimality criteria [22,23]. As a result the aim is to minimize the volume of the model parameters uncertainty region.

In this paper the proposed modification of one of such algorithms can be applied to training the Multi-Layer Perceptron (MLP) [1,2,3] and alongside calculating its uncertainty as the feasible parameter set. Moreover, a new approach for the MLP quality improvement with the application of the Sequential D-optimum Experimental Design (SDED) is proposed. All that efforts are going to propose a new robust and efficient adaptive threshold-based Robust Fault Detection (RFD) system. In particular, it is composed of three main stages:

• It is based on the classical approach to training neural networks and can be realized with state-of-the-art approaches available in the literature;
• Having, an initial structure and parameters of a neural network a novel Modified Nonlinear Quasi-OBE Algorithm (MNQOA) with optimal input sequence is used to gradually decrease its uncertainty. After this process, an adaptive threshold is obtained for the resulting neural network.
• The neural network obtained in Stage 2 along with an associated adaptive threshold are used for the on-line RFD.

It should be emphasized that there is a large number of possible machine learning approaches which can be used for the purpose of fault diagnosis and even structural health monitoring [7]. Indeed, recently Deep Neural Networks (DNN) have received a considerable research attention. Apart from their incontestable modeling appeal they usually contain a large number of parameters which increase the overall uncertainty of the resulting neural model. Thus, taking into account the well known rule from statistical and experimental design literature [22], among a spectrum of models with comparable quality the simplest one should be selected as the number of uncertain parameters is related to the overall uncertainty of the model. Taking into account the above discussion, a simple MLP is selected. Its wide popularity in numerous engineering applications results from its possibility of modeling any non-linear dynamic system [3]. Another advantage is that the several mature and effective training algorithms for such networks were developed. However, such a choice does not exclude more advanced structures which can be used instead.

The paper is organized as follows. In Section 2 the active and passive approaches to fault detection related with the proposed method are described. Section 3 presents the overall scheme of the model-based RFD and the structure with the description of the ANNs used for the RFD purpose. In Section 4 a guaranteed convergence modified non-linear quasi OBE algorithm used for the gradual improvements of the MLP parameters estimation quality is developed. In Section 5, the quality of the MLP is additionally enhanced by the application of the developed SDED algorithm. The description of the MLP uncertainty calculated with the application of the developed approach is employed in Section 6 to design the adaptive thresholds. Section 7 contains examples of the adaptive thresholds application in the RFD of the three-screw spindle oil pump and two-tank system. Finally, the last section of this paper is devoted to conclusions and future research direction.

2. Related Works: Active and Passive Approaches to Fault Detection

As it was already mentioned, the main objective of this paper is to develop a technique which makes it possible to enhance performance of a neural network-based fault detection scheme. Before proceeding to the subsequent deliberations, let us start with precisely defining fault diagnosis process which is composed of three phases [9]:

fault detection:

it makes it possible to undertake a diagnostic decision concerning the fault. In other words, it provides a binary decision concerning the fault, e.g., a pump is faulty;

fault isolation:

it enables to determine the location of the fault, e.g., an induction motor driving the pump is faulty;

fault estimation:

it allows to determine the size of the fault as well as its time varying nature, e.g., the induction motor driving the pump is faulty and it losts 20% of its performance.

The objective of the remaining part of this section is to review fault detection detection strategies which can be divided into two classes [24,25,26,27,28]:

Active fault detection:

aims at eliminating modeling uncertainty;

Passive fault detection:

focuses on providing a adaptive threshold forming an enclosure associated with modeling uncertainty.

Let us start with the active approaches, which can be generally divided into observer-based [18,29,30] and parity relation ones [28,31,32]. In both cases, modeling uncertainty is represented by the so-called unknown input which is suitably decoupled during the fault detection procedure. Apart from their incontestable appeal, active approaches have one common disadvantage, which is associated with the fact that along with decoupling unknown inputs, there is a chance that the faults can be decoupled as well. Thus, their design requires suitable caution preventing such an unappealing effect. Even if the unappealing full fault decoupling is avoided, there is still a chance of reduced fault sensitivity. Alternative approaches are named as passive ones. Instead of decoupling model uncertainty, they aim at providing its enclosure in which all responses of fault-free system should be contained. The first attempt at developing such an approach was made by Emami-Naeini et al. in 1988 [33]. The authors clearly indicated that performance of all fault detection systems is influenced by modeling uncertainty. They presented a unique framework incorporating a knowledge of modeling error in the analysis and design of fault detection systems. The developed framework was called threshold selector and it evolved in the subsequent literature to adaptive threshold [34,35]. Irrespective of the name being used, the optimal threshold is shown to be a function of the bound on modeling errors as well as the noise properties. Such a bound is frequently called uncertainty interval [32,36,37] and it is a natural fact that it should be as small as possible to provide reliable diagnostic decisions. The concept of passive fault detection was developed for various classes of systems and applications, e.g., power converters, motor systems, intelligent autonomous vehicle, wind turbines, oil and gas industry.

In the light of the above discussion, the objective of this paper is to propose a neural network-based fault detection scheme and then to enhance its performance by minimizing the size of the uncertainty interval. For that purpose a quality measure called Mean Uncertainty Interval (MUI) [38,39,40] is employed. Finally, it should be noted that the proposed approach does not estimate the fault but it provides a binary evaluation if there is the fault or not. Thus, fault convergence analysis can be investigated during the last phase of fault diagnosis which is fault estimation. This is, however, beyond the scope of this paper and it is indicated as a future research direction.

3. Model-Based RFD

Let us assume that the following relation is used to describe the diagnosed system:

$$y_k=f({\mathit{p}}_{}^{},{\mathit{u}}_k^{})+{\epsilon }_k^{},$$

(1)

where variables ${\mathit{u}}_k^{}={[u_{k,1},u_{k,2},\dots ,u_{k,n_u}]}^T$ and $y_k$ denote the diagnosed system inputs and the corresponding output. The expression $f(·,·)$ denotes the general relation describing the diagnosed non-linear system behavior, ${\epsilon }_k^{}$ is a noise/disturbances and vector $\mathit{p}$ represents model parameters. In practice such values represent the real parameters of systems which are manufactured during the process of production. Unfortunately, as the result of the influence of the disturbances during the manufacturing process the obtained parameters can be biased and in consequence, the faulty system can be obtained. Similarly, the changes of the parameters during the system operation may denote that the system is faulty.

One of the most popular FD method depends on the application of a diagnosed system model. Such model usually is obtained involving the known physical laws specifying the diagnosed system or it can be created in the process of system identification [3]. During the system identification, an analytical model of the real system is achieved. It imitates the nominal system behavior despite that its structure does not reflect the system structure. Therefore, the quality of the model is a significant factor which determines how efficiently the model-based FD system operates.

The model-based FD relies on the calculation of the spread between the system and model outputs which is called residuum. Such a signal may contain information about faults and may be used for the fault detection, isolation and identification [9,11]. Usually, the fault detection stage of the FD comes down to check out whether the absolute value of the residuum is smaller than the assumed constant threshold $\gamma$.

$$|y_k-y_{m,k}|\le \gamma .$$

(2)

If such a condition is not fulfilled for any value of k the diagnosed system is faulty. In the literature several examples of using constant threshold obtained with the use of the MLP can be found, e.g., in the power systems [41], nuclear plant [42] and valve actuators [43].

It should be noticed that such relatively simple FD method is usually safer in real applications. It may arise from the fact that such technique is not robust against uncertainty. Among the sources of uncertainty the variance of the system parameters, time-varying properties of the system, disturbances and measurements noise, model uncertainty resulting from its inappropriate structure and the parameters estimation errors can be distinguished [8]. In such a case the variation of the residuum caused by the noise and/or uncertainty does not enable to perform the reliable fault detection process and leads to undetected faults or signalization of the false alarms. To avoid such situations wider thresholds for the fault detection are assumed. Unfortunately, such solution reduces the sensitivity of the system for fault detection.

To overcome this problem a new RFD scheme is proposed where the MLP [3,44] with its uncertainty description during the nominal/non-faulty system identification process is obtained. It is assumed that the parameters of the MLP (c.f. Figure 1) are approximated by the feasible parameter set ${\mathit{P}}_{}^{}$. Its size mainly depends on the inaccuracy of parameters estimates, neural model structure errors and represents neural model uncertainty [8].

During the research the following structure of the neural model is applied:

$$y_{m,k}={\mathit{P}}_{}^{\left(l\right)}\mathcal{F}({\mathit{P}}_{}^{\left(n\right)},{\mathit{u}}_k^{})=f({\mathit{p}}_{}^{},{\mathit{u}}_k^{}),$$

(3)

where vector of neural model inputs is defined as ${\mathit{u}}_k^{}\in {\mathbb{R}}^{n_u+1}$, i.e., ${\mathit{u}}_k^{}=[u_1,\dots ,u_{n_u-1},1]$, where 1 stands for the so-called bias [3] and the values $y_{m,k}$ represents a neural model output. The vector $f(·)$ consists of $n_h$ non-linear activation functions ${[f_1(·),\dots ,f_{n_h}(·)]}^T$ of neurons in the hidden layer, where $n_h$ represents its number. It should be underlined that the non-linear activation functions is depicted as $f(·)=tanh(·)$. This assumption is not restrictive and other types of non-linear activation functions can be chosen [3]. Moreover, $f_l(·)$ denotes the linear function of the output neuron of the MLP. Furthermore, both linear and non-linear parameters describing the properties of the identified system may be described as ${\mathit{p}}_{}^{}={[{\mathit{P}}_{}^{\left(l\right)},{\mathit{P}}_{}^{\left(n\right)}]}^T={[{p_1^{\left(l\right)},\dots ,p_{n_h}^{\left(l\right)}}^T,{p_{1,1}^{\left(n\right)},\dots ,p_{n_u,n_h}^{\left(n\right)}}^T]}^T\in {\mathbb{R}}^{n_p}$ whereas $n_p=n_u{n}_h+n_h$.

The performance of the fault detection scheme shall be conditional upon the quality of the MLP which is obtained at the system identification stage. Despite the efforts such model can be uncertain. One of the main sources of the MLP uncertainty follows from the inappropriate structure of the MLP. The structure of the network is usually chosen in an arbitrary way which relies on the subjective indication of the number of neurons in each layer what may result in an inadequate network architecture. Such a problem can be reduced by various well-known selection methods of the neural models structure [45,46,47,48,49]. The description of the MLP uncertainty, defined as ${\mathit{P}}_{}^{}$, is required to design so called adaptive threshold [50]. On the contrary to the constant threshold, the adaptive one over-bounds the diagnosed system output. Its width depends on neural model uncertainty (c.f. Figure 2) described by the feasible parameter set ${\mathit{P}}_{}^{}$ and hence, it provides a much more reliable FD.

The adaptive threshold, which includes the response of the diagnosed system $y_k$ in the fault-free case, is defined by the following general form:

$${\widehat{y}}_k^m\le y_k\le {\widehat{y}}_k^M,$$

(4)

where ${\widehat{y}}_k^m$ and ${\widehat{y}}_k^M$ denote the bounds of the adaptive threshold. Such values can be calculated as follows:

$${\widehat{y}}_k^m=\underset{p\in {\mathit{P}}_{m,k}^{}}{min}{y}_{m,k}-\gamma ,$$

(5)

and

$${\widehat{y}}_k^M=\underset{p\in {\mathit{P}}_{m,k}^{}}{max}{y}_{m,k}+\gamma ,$$

(6)

whereas $y_{m,k}$ is the estimate of the diagnosed system, ${\mathit{P}}_{m,k}^{}$ represents the neural network uncertainty and $\gamma$ is the noise. Thus, the spread of the threshold (4) depends on the values of the noise and the neural model uncertainty expressed as the feasible parameter set which should be minimized in order to enhance the RFD sensitivity. To achieve such goal a new approach of MLP model improving with the SDED approach is developed. Such approach allows for appropriate training data selection what results in smaller neural model uncertainty and more narrow adaptive threshold.

An exemplary application of the proposed threshold is depicted in Figure 3. The values of the intervals (4) vary with the changes of the values of the system input ${\mathit{u}}_k^{}$. The application of the proposed SDED method enables to obtain the narrow internal. The fault may be detected when the diagnosed system response $y_k$ crosses the upper bound of the designed adaptive threshold. Furthermore, it can be seen that such small faults can not be detected by the wider adaptive threshold obtained for classical MLP without the application of the proposed approach.

4. Modified Non-Linear Quasi OBE Algorithm

Having a neural network structure it is possible to obtain its parameters ${\mathit{p}}_{}^{}$ in a way that it is possible to minimize the mismatch between its response and the one of the system being modeled. Such a process can be realized with a wide spectrum of approaches among which the gradient-based, statistical, evolutionary and genetic algorithms can be distinguished [3]. Such algorithms enable the effective training of the MLP, however, they do not let to obtain the mathematical description of its uncertainty which is required in the RFD. The objective of this section is to propose a novel algorithm which may be used for training of MLP with description of its uncertainty. Subsequently, it allows for gradual decrease of modeling uncertainty of MLP by the application of the SDED approach developed in the Section 5.

First, let us start with the development of a suitable parameter estimation algorithm. For that purpose, Modified Quasi OBE Algorithm (MQOA) [17] is utilized and suitably extended to cope with the nonlinearities presented in (3). It goes without saying that the parameter estimate ${\widehat{\mathit{p}}}_{}^{}$ should be selected as follows:

$${\epsilon }_{k+1}^{}=y_{k+1}-f({\widehat{\mathit{p}}}_k^{},{\mathit{u}}_{k+1}^{}),$$

(7)

and should be as small as possible for the entire dataset samples enumerated by $k=0,\dots ,n_t$. One way to settle the nonlinearity issue is to use the Taylor expansion-based linearization which leads to:

$$f({\widehat{\mathit{p}}}_k^{},{\mathit{u}}_{k+1}^{})\approx {\mathit{r}}_{k+1}^T{\widehat{\mathit{p}}}_k^{}.$$

(8)

where the so-called regressor is given by:

$${\mathit{r}}_{k+1}={\left.\frac{\partial f({\mathit{p}}_{}^{},{\mathit{u}}_{k+1}^{})}{\partial {\mathit{p}}_{}^{}}\right|}_{{\mathit{p}}_{}^{}={\widehat{\mathit{p}}}_k^{}}.$$

(9)

To avoid the above approximation the following exact equality is introduced:

$$f({\widehat{\mathit{p}}}_k^{},{\mathit{u}}_{k+1}^{})={\mathit{r}}_{k+1}^T{\widehat{\mathit{p}}}_k^{}+{\mathit{\omega }}_k,$$

(10)

where ${\mathit{\omega }}_k$ stands for the linearization error which is over-bounded as:

$$-q\le {\mathit{\omega }}_k\le q,$$

(11)

$$\frac{1}{q^2}{\mathit{\omega }}_k^2\le 1,q>0.$$

(12)

Since the linearization error ${\mathit{\omega }}_k$ is unknown, the output error cannot theoretically attain its possible low value. Thus, following the general guidance of the MQOA, it is reasonable to set up a constraint on the magnitude of the estimation error (17), which can be expressed as:

$$|{\epsilon }_{k+1}^{}|\le \gamma ,\gamma >0.$$

(13)

Such a constraint leads to the feasible parameter set:

$${\mathit{P}}_{m,k}^{}=\bigcap _{k=1}^{n_t}\{{\mathit{p}}_{}^{}\in {\mathbb{R}}^{n_p}:|y_k-f({\widehat{\mathit{p}}}_{k-1}^{},{\mathit{u}}_k^{})|\le \gamma \},$$

(14)

rather then to a single parameter estimate ${\widehat{\mathit{p}}}_k^{}$.

After the above-detailed deliberations, it is possible to propose Modified Nonlinear Quasi OBE Algorithm (MNQOA):

Initialization: 

$$\begin{array}{cc}\hfill {\mathit{P}}_0^{}& =\delta {\mathit{I}}_{n_p},\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\widehat{\mathit{p}}}_0^{}& ={\mathit{p}}_0.\hfill \end{array}$$

(16)

where $\delta >0$ is a large constant and ${\mathit{p}}_0$ is initial parameter estimate.

Sequential iteration:

Calculate:

$${\epsilon }_{k+1}=y_{k+1}-f({\widehat{\mathit{p}}}_k^{},{\mathit{u}}_{k+1}),$$

(17)

if $|{\epsilon }_{k+1}^{}|>\gamma$ then

$$\begin{array}{cc}\hfill g_{k+1}& ={\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill l_k& =\frac{{\mathit{g}}_{k+1}^{}}{\frac{|{\epsilon }_{k+1}^{}|}{\gamma }-1},\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill {\mathit{P}}_{k+1}^{}& =\frac{1}{l_k}\left[\mathit{I}-\frac{{\mathit{P}}_k^{}{\mathit{r}}_{k+1}{\mathit{r}}_{k+1}^T}{l_k+{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}}\right]{\mathit{P}}_k^{},\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\widehat{\mathit{p}}}_{k+1}^{}& ={\widehat{\mathit{p}}}_k^{}+{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\epsilon }_{k+1}^{},\hfill \end{array}$$

(21)

else:

$$\begin{array}{cc}\hfill {\mathit{P}}_{k+1}^{}& ={\mathit{P}}_k^{},\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill {\widehat{\mathit{p}}}_{k+1}^{}& ={\widehat{\mathit{p}}}_k^{}.\hfill \end{array}$$

(23)

The main goal of the next part of this section is to deliver convergence conditions of MNQOA. Thus, let us begin with defining the parameter estimation error which by using (21) is given by:

$$\begin{array}{cc}\hfill {\mathit{e}}_{k+1}^{}=& {\mathit{p}}_{}^{}-{\widehat{\mathit{p}}}_{k+1}^{}\hfill \\ \hfill =& {\mathit{p}}_{}^{}-{\widehat{\mathit{p}}}_k^{}-{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\epsilon }_{k+1}^{}\hfill \\ \hfill =& {\mathit{e}}_k^{}-{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\epsilon }_{k+1}^{}.\hfill \end{array}$$

(24)

By applying (10) into (24) it can be shown that:

$$\begin{array}{c}\hfill {\mathit{e}}_{k+1}^{}=({\mathit{I}}_{n_p}-{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\mathit{r}}_{k+1}^T){\mathit{e}}_k^{}+{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\mathit{\omega }}_k.\end{array}$$

(25)

Note that for ${\mathit{\omega }}_k=0$, the system (25) becomes an autonomous system, and hence, its convergence can be validated with the Lyapunov method. In this paper, it is assumed that, in general, ${\mathit{\omega }}_k\ne 0$. This means that the tools for stability analysis of forces systems have to be employed. Taking into account (12), a natural solution is to use the Quadratic Boundedness (QB) technique [51]. Thus, by adapting the notion of QB to (25) the following definition can be formulated:

Definition 1.

The considered system (25) is absolutely QB iff for all admissible ${\mathit{\omega }}_k$ obeying (12) there exists ${\alpha }_k\in [0,1]$ for which the succeeding inequality is met:

$$V_{k+1}-(1-{\alpha }_k)V_k-{\alpha }_k\frac{1}{q^2}{w}_k^2<0,{\forall }_k,$$

(26)

where $V_k$ is the Lyapunov candidate function. Thus, QB notation can be equivalently named robust convergence, and hence the objective of the next part of this section is to prove such convergence for MNQOA.

Theorem 1.

The MNQOA is robustly convergent for all admissible ${\mathit{\omega }}_k$ obeying (12) if there exists ${\alpha }_k\in [0,1]$ such that the following condition is satisfied:

$$\begin{array}{cc}\hfill |{\epsilon }_{k+1}^{}|& >\frac{q^2-{\alpha }_k}{\gamma },\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill l_k& <1-{\alpha }_k.\hfill \end{array}$$

(28)

Proof. 

Having the parameter estimation error, it is possible to define the Lyapunov function:

$$\begin{array}{c}\hfill V_k={\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{},\end{array}$$

(29)

which due to the positive definiteness of ${\mathit{P}}_k^{}$ satisfies $V_k>0$ for all ${\mathit{e}}_k^{}\ne 0$. Subsequently, using (24) it can be shown that:

$$\begin{array}{cc}\hfill V_{k+1}=& {\mathit{e}}_{k+1}^T{\mathit{P}}_{k+1}^{-1}{\mathit{e}}_{k+1}^{}\hfill \\ \hfill =& \left[{\mathit{e}}_k^T-{\mathit{r}}_{k+1}^T{\mathit{P}}_{k+1}^{}{\epsilon }_k^{}\right]{\mathit{P}}_{k+1}^{-1}\left[{\mathit{e}}_k^{}-{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}{\epsilon }_{k+1}^{}\right]\hfill \\ \hfill =& {\mathit{e}}_k^T{\mathit{P}}_{k+1}^{-1}{\mathit{e}}_k^{}-{\epsilon }_{k+1}^{}{\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}-{\mathit{e}}_k^T{\mathit{r}}_{k+1}^{}{\epsilon }_{k+1}^{}+{\epsilon }_{k+1}^2{\mathit{r}}_{k+1}^T{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}^{},\hfill \end{array}$$

(30)

 ☐

By applying the matrix inversion lemma [52] to (20) it can be shown that:

$${\mathit{P}}_{k+1}^{-1}=l_k{\mathit{P}}_k^{-1}+{\mathit{r}}_{k+1}^{}{\mathit{r}}_{k+1}^T,$$

(31)

and consequently:

$$V_{k+1}=l_k{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}+{\mathit{e}}_k^T{\mathit{r}}_{k+1}^{}{\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}-2{\epsilon }_{k+1}^{}{\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}+{\epsilon }_{k+1}^2{\mathit{r}}_{k+1}^T{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}^{},$$

(32)

Multiplying (20) from left and right by ${\mathit{r}}_{k+1}^T$ and ${\mathit{r}}_{k+1}^{}$, respectively, gives:

$$\begin{array}{cc}\hfill {\mathit{r}}_{k+1}^T{\mathit{P}}_{k+1}^{}{\mathit{r}}_{k+1}^{}=& \frac{1}{l_k}\left[{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}-\frac{{\left({\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}\right)}^2}{l_k+{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}}\right]\hfill \\ \hfill =& \frac{1}{l_k}\left[\frac{l_k{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}+{\left({\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}\right)}^2-{\left({\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}\right)}^2}{l_k+{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}}\right]\hfill \\ \hfill =& \frac{{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}}{l_k+{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}},\hfill \end{array}$$

(33)

which allows transforming (32) into:

$$\begin{array}{c}\hfill V_{k+1}=l_k{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}+{\left({\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}\right)}^2-2{\epsilon }_{k+1}^{}{\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}+{\epsilon }_{k+1}^2\frac{{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}}{l_k+{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}^{}},\end{array}$$

(34)

By applying the output error relation:

$$\begin{array}{c}\hfill {\mathit{r}}_{k+1}^T{\mathit{e}}_k^{}={\epsilon }_{k+1}^{}-{\mathit{\omega }}_k,\end{array}$$

(35)

into (34) yields:

$$\begin{array}{c}\hfill V_{k+1}=l_k{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}+{({\epsilon }_{k+1}^{}-{\mathit{\omega }}_k)}^2-2{\epsilon }_{k+1}^{}\left({\epsilon }_{k+1}^{}-{\mathit{\omega }}_k\right)+{\epsilon }_{k+1}^2\frac{{\mathit{g}}_{k+1}^{}}{l_k+{\mathit{g}}_{k+1}^{}},\end{array}$$

(36)

which can be written in an equivalent form:

$$\begin{array}{c}\hfill V_{k+1}=l_k{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}+{\epsilon }_{k+1}^2\left(\frac{{\mathit{g}}_{k+1}^{}}{l_k+{\mathit{g}}_{k+1}^{}}-1\right)+{\mathit{\omega }}_k^2,\end{array}$$

(37)

Thus, substituting (37) into (26) leads to:

$$\begin{array}{c}\hfill \left[l_k-(1-{\alpha }_k)\right]{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}+{\epsilon }_{k+1}^2\left[-\frac{l_k}{l_k+{\mathit{g}}_{k+1}^{}}\right]+\left[1-{\alpha }_k\frac{1}{q^2}\right]{\mathit{\omega }}_k^2<0,\end{array}$$

(38)

Subsequently, substituting (19) yields:

$$\begin{array}{c}\hfill \left[l_k-(1-{\alpha }_k)\right]{\mathit{e}}_k^T{\mathit{P}}_k^{-1}{\mathit{e}}_k^{}-\gamma \left|{\epsilon }_{k+1}^{}\right|+\left[1-{\alpha }_k\frac{1}{{\mathit{g}}_{}^2}\right]{\mathit{\omega }}_k^2<0.\end{array}$$

(39)

Note that satisfying the above inequality can be equivalently written as:

$$\begin{array}{c}{l}_k-(1-{\alpha }_k)<0,\hfill \end{array}$$

(40)

$$\begin{array}{c}-\gamma |{\epsilon }_{k+1}^{}|+\left(1-{\alpha }_k\frac{1}{q^2}\right)w_k^2<0,\hfill \end{array}$$

(41)

Finally, it can be observed that (40) is equivalent to (28). While taking the worst case of ${\mathit{\omega }}_k$ according to (12), condition (41) leads to (27), what finishes the proof.

The main remark, which can be formulated at the of this section, is that the convergent condition of the proposed approach depends definitely on the relevant input selection (cf. (18)). Moreover, by noticing that MNQOA iterates while $|{\epsilon }_{k+1}^{}|>\gamma$, condition (27) is equivalent to:

$$\begin{array}{c}\hfill {\gamma }^2>q^2-{\alpha }_k\end{array}$$

(42)

which, by taking into account the fact ${\alpha }_k\in [0,1]$, provides an elegant tool for selecting $\gamma >0$. It should be also noted that the algorithm developed in this section avoids unappealing effects of the similar algorithms, which pertains neglecting linearization error.

5. Enhancing Performance of Neural Networks with SDED

The algorithm presented in the preceding section aims at guaranteed convergence iterative improvements of the MLP parameter estimates based on the sequentially provided input and output measurements. It is well known fact that the quality of such an iterative process can be measured using ${\mathit{P}}_{k+1}^{}$ [22,23]. Having a parameter updating algorithm along with the description of parameter uncertainty it is desirable to develop an algorithm capable of determining additional input values leading to uncertainty minimization. Indeed, while analyzing (20) it can be observed that parameters uncertainty is strongly correlated with input selection policy. In the MQOA developed in Section 4, the ellipsoidal shape of the parameter uncertainty region is employed. Thus, this section aims at developing an algorithm capable of determining a sequence of D-optimal input values [22] minimizing the determinant of the ellipsoidal uncertainty region which is proportional to the volume of such an ellipsoid. The developed algorithm will be called Sequential D-optimum Experimental Design. It should be underlined that the selection of determinant criterion is not accidental as it is a common practice to do it while assessing the parameter estimation quality [53,54]. The developed iterative improvement algorithm guarantees that the resulting neural network will make it possible to design more reliable fault diagnosis scheme capable of making appropriate decisions about the current state of the system.

To proceed with further deliberations, let us start with identifying the determinant of ${\mathit{P}}_k^{}$ by exploiting (20):

$$\begin{array}{cc}\hfill det\left({\mathit{P}}_{k+1}^{}\right)=& det\left(\frac{1}{l_k}{\mathit{P}}_k^{}\right)det\left({\mathit{I}}_{n_p}-\frac{{\mathit{P}}_k^{}{\mathit{r}}_{k+1}{\mathit{r}}_{k+1}^T}{l_k+g_{k+1}}\right),\hfill \end{array}$$

(43)

Taking into account the following identity $det(\mathit{I}+\mathit{a}{\mathit{b}}^T)=1+{\mathit{b}}^T\mathit{a}$ yields:

$$det\left({\mathit{I}}_{n_p}-\frac{{\mathit{P}}_k^{}{\mathit{r}}_{k+1}{\mathit{r}}_{k+1}^T}{l_k+g_{k+1}}\right)=\frac{1}{1+l_k^{-1}{g}_{k+1}},$$

(44)

which results in more condensed form of (43):

$$det\left({\mathit{P}}_{k+1}^{}\right)=\frac{l_k^{-n_p}}{1+l_k^{-1}{g}_{k+1}}det\left({\mathit{P}}_k^{}\right),$$

(45)

While analyzing (45) and noticing that the regressor ${\mathit{r}}_k^{}$ depends on ${\mathit{u}}_k^{}$, it is clear that an appropriate input selection may lead to a gradual decrease of ${\mathit{P}}_k^{}$ resulting in an enhanced model quality. Therefore, bearing in mind the fact that $g_{k+1}={\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1}$, ${\mathit{u}}_{k+1}^{}$ minimizing $det\left({\mathit{P}}_{k+1}^{}\right)$ is:

$${\mathit{u}}_{k+1}^{*}=arg\underset{{\mathit{u}}_{k+1}^{}\in \mathcal{U}}{max}{\mathit{r}}_{k+1}^T{\mathit{P}}_k^{}{\mathit{r}}_{k+1},$$

(46)

where $\mathcal{U}$ denotes an admissible input set, which can be expressed by various constraints, e.g., commonly used linear ones. The above optimization problem can be solved by the application of various global optimization algorithms, e.g., an evolutionary or genetic algorithms or an adaptive random search [55], which is employed within the framework of this paper. To summarize, the structure of the proposed iterative algorithm (cf. Figure 4) is:

Step 1: 

Set $k=0$, determine an initial parameter ${\widehat{\mathit{p}}}_0^{}$ by exploiting the collected input-output data measurement set of the diagnosed system in the nominal state, set ${\mathit{P}}_0^{}=\delta \mathit{I}$ where $\delta$ denotes sufficiently large constant. Select $n_t$, the number of gradual improvement iterations.

Step 2: 

Obtain ${\mathit{u}}_k^{}$ by D-optimal input value, and then fed it into the modeled system to get $y_k$.

Step 3: 

Update ${\widehat{\mathit{p}}}_{k+1}^{}$ the parameter estimate. If $k=n_t$ then STOP the algorithm, else set $k=k+1$ and go to Step 2.

It should be noted that the above algorithm starts with an initial parameter vector which should be obtained in such a way as to achieve a rough consistency between measured data and the neural network response (Step 1). However, as it was already mentioned, this step can be realized with the available state-of-the-art solutions such as the Levenberg-Marquard (LM) algorithm [2] which is a celebrated routine in engineering computation packages. Subsequently, the optimization problem of Step 2 can be solved by the application of various global optimization algorithms, e.g., an evolutionary or genetic algorithms or an adaptive random search [55], which is employed within the framework of this paper. In conclusion, the proposed framework alternates two phases: Step 2—optimal input determination feeding it into the system to get $y_k$, Step 3—update of neural network parameters along with determinant minimization of ${\mathit{P}}_k^{}$.

6. Adaptive Thresholds for the RFD

In Section 4, a training method for the MLP on the basis of the MNQOA is proposed. During the application of such a method the parameters estimate ${\widehat{\mathit{p}}}_{}^{}$ and its uncertainty $\mathit{P}$ for the MLP, are obtained. Such knowledge is necessary to develop a RFD approach. Moreover, the method developed in Section 5 enables to improve the MLP quality. It follows from the appropriate selection of training data which allows to minimize the volume of the feasible parameter set and in the consequence reduce the MLP uncertainty. The obtained matrix $\mathit{P}$ describing MLP uncertainty allows to calculate so-called the MLP output uncertainty interval. Such interval includes the model output $y_{m,k}$. It should be underlined that the range of such interval depends on the orientation and size of the ellipsoid which describe the feasible parameter set ${\mathit{P}}_{m,k}^{}$.

Taking into account the facts that the ${\mathit{P}}_k^{}$ is an ellipsoid and the output error is bounded by $\gamma$, the adaptive threshold is given by:

$${\widehat{y}}_k^m\le y_k\le {\widehat{y}}_k^M,$$

(47)

where $y_k$ is the diagnosed system output, whereas:

$${\widehat{y}}_k^m=y_{m,k}-{\left({\mathit{r}}_k^T\mathit{P}{\mathit{r}}_k\right)}^{1/2}-\gamma ,$$

(48)

and

$${\widehat{y}}_k^M=y_{m,k}+{\left({\mathit{r}}_k^T\mathit{P}{\mathit{r}}_k\right)}^{1/2}+\gamma ,$$

(49)

are the minimum and maximum admissible output estimate of the diagnosed system extended by the values of $\gamma$. The abnormal work of the diagnosed system is detected when the monitored system output $y_k$ extends over the adaptive thresholds calculated by (47) (cf. Figure 5). In other words, the system is fault-free when (47) contains real system response.

For the purpose of further analysis the following model quality index is introduced and called Mean Uncertainty Interval (MUI) is introduced:

$$\begin{array}{c}\hfill \mathrm{MUI}=\frac{1}{N}\sum _{k=1}^N({\widehat{y}}_k^M-{\widehat{y}}_k^m).\end{array}$$

(50)

It can be perceived as a mean value of model uncertainty and it will be used to evaluate its evolution.

7. An Illustrative Example of the RFD

7.1. Experimental Study with the Three-Screw Spindle Oil Pump

The primary goal of the illustrative part of the paper is to show the details of designing of the MLP and its application in the RFD of the oil pump [56]. The scheme of such system is presented in Figure 6.

Such class systems are widely applied in many industries e.g., the chemical, petrochemical and automotive [57,58,59]. It is worth to emphasize that the appropriate monitoring of such systems being part of several industrial plants allows for an early RFD and avoidance of the economic losses. The relation describing the diagnosed pump is the following: $y_k=f({\mathit{u}}_k^{},{\mathit{p}}_{}^{})$ where:

• $u_{k,1}$—the differential pressure between inlet and outlet of the pump,
• $u_{k,2}$—the motor speed,
• $y_k$—the torque of the pump.

To simulate the faults in the diagnosed system a simulator of the pump was implemented in the Simulink environment. On the basis of such simulator the training, validation and faults containing data sets required to the research were generated. All data were generated with sampling time $t=0.01$ s. The training data were used to estimate the MLP parameters in the fault-free mode. The validation data were applied to check the generalization ability of the MLP. Moreover, it allows to confirm the increase of the modeling quality resulting from the application of the developed SDED approach. The faults containing data were used to show the effectiveness of the MLP-based RFD method.

On the beginning of the system identification procedure the MLP consisting of neurons with the hyperbolic tangent transfer function in the hidden layer and one neuron with the linear transfer function in the output layer were assumed. Subsequently, the structure of MLP was optimized with the application of the Cascade Correlation Method (CCM) [45] to minimize the MLP uncertainty following from model over-parametrization. Finally, the neural model consisting of 8 neurons in hidden layer was obtained.

After obtaining the MLP architecture, the initial training process with the LM algorithm was made. Next, the SDED algorithm developed in Section 5 was used in order to choose the training data samples which bring the most information in the training process. Figure 7 and Figure 8 show the chosen data samples for inputs $u_{k,1}$ and $u_{k,2}$, where circles represent the measurements which taking into account during the training results in decreasing of the MLP uncertainty.

Subsequently, in the Section 4 the developed modified non-linear quasi OBE algorithm was used to retrain the network and in the consequence to decrease MLP uncertainty. Figure 9 shows the comparison of the modeling quality with and without the application of the developed SDED approach.

Simultaneously with the retraining of the MLP, the feasible parameter set was estimated with the application of the proposed MNQOA and adaptive threshold containing the outlet of the pump in the fault-free mode was calculated (c.f. Figure 10).

As it can be observed, the application of the developed MNQOA join with the SDED radically decrease the neural model uncertainty. Therefore, the narrower adaptive threshold is obtained and more sensitive the RFD system is achieved. Such designed FD system allows to detect even small or incipient faults.

To validate such scenario the data containing the $20\%$ performance degradation of an induction motor feeding the pump were simulated. Thus, in the case of such a fault, the system (1):

$$y_k=f\left({\mathit{p}}_{}^{},{\mathit{u}}_k^{}+\left[\begin{array}{c}0\\ 1\end{array}\right]f_k\right)+{\epsilon }_k^{},$$

(51)

where

$$\begin{array}{c}\hfill f_k=\left\{\begin{array}{cc}-0.2u_{2,k}\hfill & k=200,\dots ,400,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(52)

Figure 11 presents the outlet of the failed pump with the corresponding adaptive threshold determined with the MLP. From the above results it can be noticed that the fault is correctly and immediately detected what proofs the effectiveness of the developed approach.

7.2. Experimental Study with the Two-Tank System

The objective of the preceding section was to validate the proposed approach using a realistic three-screw spindle pump benchmark. The achieved results clearly indicate the usefulness of the proposed approach and recommend its application to real systems such as the two-tank system presented in Figure 12.

The two-tank system is equipped with two pumps feeding the first and second tank, respectively. The detailed description of the two-tank system with its parameters in the [60] can be found. The modeling task pertains development of a neural model describing the relation between the inputs $u_1$ and $u_2$ being liquid flows (provided by pump 1 and pump 2) and the liquid level in the second tank. The initial phase was realized by feeding the pump with randomly selected pumping levels and collecting the resulting liquid level in the second tank. All data sets were obtained with sampling time $t=1$ s. Having training dataset, the structure of the MLP was optimized with the application of the CCM resulting in 7 neurons in the hidden layer. The selected neural model was trained with the application of the LM algorithm. Subsequently, the proposed SDED strategy was applied resulting in the D-optimum input training data set portrayed in Figure 13 and Figure 14 which was iteratively used by the proposed MNQOA to decrease the uncertainty of the trained neural network.

Finally, the initial and suitably updated neural network uncertainties were compared within the closed loop system, i.e., a controller was applied to keep a desired constant level in the second tank. The achieved results are presented in Figure 15.

As it can be observed, the neural model trained with the proposed approach has a significantly lower uncertainty which is exhibited by the size of the uncertainty interval.

Finally, the proposed approach was used for checking the loss of effectiveness of the second pump. For that purpose a 20% loss of its effectiveness was introduced. Thus, in the case of such a fault, the system (1):

$$y_k=f\left({\mathit{p}}_{}^{},{\mathit{u}}_k^{}+\left[\begin{array}{c}0\\ 1\end{array}\right]f_k\right)+{\epsilon }_k^{},$$

(53)

where:

$$\begin{array}{c}\hfill f_k=\left\{\begin{array}{cc}-0.2u_{2,k}\hfill & k=200,\dots ,400,\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(54)

The obtained results are portrayed in Figure 16.

From the above results it can be noticed that the fault is correctly and immediately detected what proofs the effectiveness of the developed approach.

In order to show the effectiveness of the proposed method a comparison with the Extended Kalman Filter (EKF) [61] was performed. For this aim the adaptive thresholds for the three-screw spindle oil pump and the two-tank system with the application of the EKF were calculated. Then, the values of the model quality index MUI defined according Equation (50) were compared to show the quality of each method. The obtained results are presented in

Table 1. Mean Uncertainty Interval (MUI) obtained with the application of the Extended Kalman Filter (EKF), classical Multi-Layer Perceptron (MLP) and MLP with Sequential D-optimum Experimental Design (SDED).

	Three-Screw Spindle Oil Pump	Two-Tank System
EKF	1.9880	0.4191
Classical MLP	1.7025	0.3256
MLP with SDED	0.5108	0.0788

.

8. Conclusions

The paper is aimed at developing a new model-network based RFD scheme with enhanced performance neural networks. To achieve this goal, a guaranteed convergence algorithm for an iterative gradual improvements of neural network parameter estimation quality was proposed. It should be noted that the developed conditions result in two inequalities which can be easily checked while applying the proposed algorithm. Having an estimation algorithm, a way of further increase of parameter estimation quality was proposed. It is based on iteratively solving an optimization problem guarantying size decrease of the feasible parameter set. The next step was oriented towards deriving uncertainty intervals of the neural network response based on the attained feasible parameter set. As a result of the application of the proposed approach, an enhanced quality neural network-based RFD schemed was developed. The final part of the paper focuses on the application of the proposed framework to the three-screw spindle oil pump and two-tank system RFD. The obtained results clearly indicate that it can effectively detect even small performance degradations of these systems. These results recommend its application for a wide range of industrial components. Finally, it should be pointed out that the future research direction will concern the subsequent fault diagnosis stage, which is fault estimation.