Intelligent Prognostics of Degradation Trajectories for Rotating Machinery Based on Asymmetric Penalty Sparse Decomposition Model

Abstract

The ability to accurately track the degradation trajectories of rotating machinery components is arguably one of the challenging problems in prognostics and health management (PHM). In this paper, an intelligent prediction approach based on asymmetric penalty sparse decomposition (APSD) algorithm combined with wavelet neural network (WNN) and autoregressive moving average-recursive least squares algorithm (ARMA-RLS) is proposed for degradation prognostics of rotating machinery, taking the accelerated life test of rolling bearings as an example. Specifically, the health indicators time series (e.g., peak-to-peak value and Kurtosis) is firstly decomposed into low frequency component (LFC) and high frequency component (HFC) using the APSD algorithm; meanwhile, the resulting non-convex regularization problem can be efficiently solved using the majorization-minimization (MM) method. In particular, the HFC part corresponds to the stable change around the zero line of health indicators which most extensively occurs; in contrast, the LFC part is essentially related to the evolutionary trend of health indicators. Furthermore, the nonparametric-based method, i.e., WNN, and parametric-based method, i.e., ARMA-RLS, are respectively introduced to predict the LFC and HFC that focus on abrupt degradation regions (e.g., last 100 points). Lastly, the final predicted data could be correspondingly obtained by integrating the predicted LFC and predicted HFC. The proposed methodology is tested using degradation health indicator time series from four rolling bearings. The proposed approach performed favorably when compared to some state-of-the-art benchmarks such as WNN and largest Lyapunov (LLyap) methods.

1. Introduction

Rotating machines are critical elements of almost all forms of mechanical assemblies, which play an important role in today’s industrial applications. Unexpected failures and unscheduled system interruptions caused by harsh working environments may result in costly lapse in production or even catastrophic incidents. Accurate/timely prediction and health assessment of rotating machines are of great significance in the functionality and performance of these equipment, especially in the early stages of failure [1,2,3].

Accordingly, in recent years, a plethora of research works have proposed tracking the degradation trend of rotating machines and predicting their remaining useful life (RUL), wherein condition-based prognostics maintenance (CBPM) has become an efficient strategy for the PHM of rotating machines. The CBPM is based on the data information collected via embedded sensors to assess and judge the real-time state of the mechanical assemblies, and then predict their degradation trend to develop appropriate decisions on maintenance activities before failure occurs. More advanced prognostics techniques are focused on degradation assessment of the systems, but it is also a relatively complex problem due to the non-stationary and nonlinear characteristic of data information [4,5,6]. Generally, to fulfill the goal of prognostics, three crucial steps are needed:

(1)

The historical data with long-terms, including normal and abnormal time series data, should be collected using sensors and other portable testing devices. Typically, good degradation data must capture the physical transitions that the rotating machinery such as rolling bearing undergo during different stages of its running life; fortunately, most of the historical data are easy to collect since the rotating machinery is frequently updated in real time.

(2)

The health indicators (e.g., peak-to-peak value and Kurtosis) should be extracted to assess and continuously track the health condition of rotating machines or system. The health indicators are used to design a suitable prediction model that captures the evolution of the degradation trend of rotating machines.

(3)

The reliable prediction models and possible failure models should be established and the remaining useful life (RUL) or future trajectory of rotating machines could be effectively predicted.

Among these three steps, model establishment and trajectory estimation are the most important and challenging steps because of the stochastic nature and nonlinear characteristics of the health indicators. Therefore, many researchers have laid their emphasis on model foundation in recent years. From the view of application, roughly, the existing prediction approaches can be divided into two categories: (a) parametric-based methods and (b) nonparametric-based methods. In the literature, parametric-based methods mainly include time-series methods such as autoregressive moving average (ARMA) [7,8], fractional autoregressive integrated moving average (FARIMA) [9,10], fractional Brownian motion (FBM) [11,12], hidden Markov model (HMM) [13,14] and grey theoretical model (GTM) [15,16], etc. Generally, the parametric-based methods overcome the hurdle of predictive availability during long-term prediction (according to needs) which assumes the model’s parameters to be constants in the predicted region. However, due to the stochastic, complex and nonlinear properties of health indicators time series (HITS), the trade-off of parametric-based method is that only accurate prediction with real-time can help adjust the system model. Otherwise, the prediction performance may not be ensured. In addition, the parametric-based methods are often developed case by case, thus model parameters identification also requires extensive experiments, and therefore, it is usually not desirable in practice.

To address this issue, much attention has recently been focused on the nonparametric-based methods in the degradation prognostic of rotating machinery. For instance, artificial neural network (ANN) [17,18,19,20], fuzzy logic [21], and deep learning network (DPN) [22,23], etc., could be considered as successful nonparametric-based approaches in the PHM field. The benefit of using nonparametric-based methods lies in their ability to model the evolution of complex multi-dimensional degradation data, which can effectively extract latent features such as the spatio-temporal correlations (STC) among historical data. In addition, for the nonparametric-based methods, it is not necessary to establish an accurate linkage between the reliability health indicators and physical degradation such as crack propagation in an individual element. However, the common drawbacks associated with these nonparametric-based techniques are time-consuming and a large number of training samples are required in advance.

Due to time-variant and non-stationary characteristics of the health indicators, and the disadvantages of the model, which is sometimes case-specific, it is difficult to determine the best prognostic model. Therefore, it is intuitive to develop a fusion idea via combining parametric-based methods and nonparametric-based methods to blend their merits and enhance the prediction performance of degradation trajectories. However, it is not clear how to fuse parametric-based and nonparametric-based models to improve the prediction accuracy, especially when health indicators become abundant and redundant.

It is generally thought that the health indicators time series (HITS) is commonly composed of different sub-components which correspond to different time-variant characteristics of the systems. In this work, we assume that the HITS is mainly composed of high frequency component (HFC) and low frequency component (LFC). The HFC part corresponds to the stable change around the zero line of the HITS which frequently occurs; in contrast, the LFC part is essentially related to the evolutionary trend of the HITS which rarely occurs in practice, because the stable running trend occupies most of the life time, only the drastic trend can be clearly emerged when the failures are serious. This can be explained using degradation trajectories of several rotating machines (e.g., rolling bearings) as shown in Figure 1, where the degradation process of the bearings generally consists of two phases, i.e., phase I: the normal operation phase and phase II: the failure phase. The degradation health indicator in phase I is stable, which means that the bearing is normal. The degradation process changes from phase I to phase II once a fault occurs, where the HITS generally increases as the fault gets worse, the curve that rises steadily could be assumed as the LFC; in contrast, the volatile components with large and small amplitude could be treated as the HFC. To ensure better prediction accuracy, it is necessary to isolate the LFC associated with bearing defects and HRC associated with structural vibration from the degradation trajectories, and then develop a degradation algorithm based on both components.

To overcome this problem, a sparse low-rank matrix decomposition (SLMD) method is introduced in this research. In recent years, many researchers have laid their emphasis on fault diagnosis and sparse filtering using SLMD algorithm. For instance, in ref. [24], Selesnick et al. developed a convex low-rank matrix decomposition approach based on non-convex bivariate penalty function for sparse one-dimensional deconvolution. In ref. [25], He et al. proposed convex sparsity-based regularization scheme to extract multiple faults of motor bearing and locomotive bearing. In refs. [26,27], Li et al. proposed a bicomponent sparse low-rank matrix separation and group sparsity total variation de-noising approach to extract transient fault impulses from the noisy vibration signals, using the rolling bearing and gearbox as example. In ref. [28], Wang et al. utilized a sparse low-rank matrix decomposition method based on generalized minimax-concave (GMC) penalty to explore and diagnosis the localized faults in rolling bearings. In refs. [29,30], Parekh et al. proposed an enhanced low-rank matrix approximation and its effectiveness was verified by the synthetic data and nonlocal self-similarity based image. In ref. [31], Ding et al. proposed the sparse low-rank matrix decomposition (SLMD) algorithm based on split augmented Lagrangian shrinkage (SALS) algorithm with majorization-minimization (MM) to detect and extract the periodical oscillatory features of gearbox of an oil refinery.

Unfortunately, despite the effectiveness of the above approaches, the SLMD approaches still suffer from the following several challenges and their application development has been restricted:

(1)

Generally, the penalty functions that are established in a low-rank matrix approximation model are symmetric function, e.g., absolute value function; the common drawback is that this penalty function is non-differentiable at zero point, which can lead to some numerical issues such as local optimum and early termination of algorithm.

(2)

In conventional SLMD methods, the convex regularizer (e.g., L1-norm) which usually underestimates the sparse signal when the absolute value function is used as a sparsity regularizer. Additionally, both the convex and nonconvex methods shrink all the coefficients equally and remove too much energy of useful information, resulting in the separation of the LFC and HFC becoming more challenging.

To overcome these limitations and to robustly separate the LFC and HFC from degradation trajectories of health indicators, a novel asymmetric penalty sparse decomposition (APSD) algorithm with non-convex sparsity constraint is proposed in this work. The resulting non-convex regularization problem will be efficiently handled using the majorization-minimization (MM) method. Having the LFC and HFC separated, the LFC and HFC components (e.g., last 100 points) will be predicted using the wavelet neural network (WNN) and ARMA combined with recursive least squares algorithm (ARMA-RLS) methods, respectively. In an effort to establish a comprehensive assessment of degradation processes, the proposed fusion prognostics framework combines the WNN (a nonparametric-based approach) and the ARMA-RLS (a parametric-based approach) in an attempt to improve the prediction performance, in which the physical information of the degenerative process and massive sample data are not considered and accommodated. The final predicted data could be generated by combining the predicted LFC and HFC components. Finally, we demonstrate the proposed approach by four case studies of accelerated aging tests of rolling bearing.

The main contributions of this paper are summarized as follows:

(1)

To address the issue of models merging and improve the prediction accuracy, the health indicators time series (HITS) is decomposed into different sub-components, i.e., low frequency component (LFC) and high frequency component (HFC), using the APSD algorithm.

(2)

To address the drawback that penalty function is non-differentiable at zero point, a new asymmetric penalty function is proposed.

(3)

To solve the proposed non-convex regularization problem based on asymmetric penalty function, the majorization-minimization (MM) algorithm is introduced.

(4)

The decomposed LFC and HFC components can be predicted using the wavelet neural network (WNN) and ARMA combined with recursive least squares algorithm (ARMA-RLS) methods, respectively.

(5)

The prediction accuracy is greatly improved compared with some state-of-the-art models, and the presented approach has powerful application potentials.

The frame of this paper is structured as follows. Section 2 contains the description of the asymmetric nonconvex sparse decomposition (APSD) algorithm. Section 3 provides the algorithms of degradation modeling, i.e., wavelet neural network and ARMA combined with recursive least squares algorithm (ARMA-RLS) methods. Experimental results and discussion are given in Section 4. Finally, Section 5 concludes with a discussion of future research.

2. Asymmetric Penalty Sparse Decomposition (APSD) Algorithm

2.1. Sparse Representation and Filter Banks

Generally, the fault vibration signal (observation signal) y of rotating machinery such as rolling bearing, which contains three parts below: the fault transient impulses x, the systematic natural signal f and the additive noise w, i.e.,

$$y=x+f+w$$

(1)

The core work of the fault diagnosis is to extract the fault transient impulses x from the noisy observation y. Assume that fault transient impulses and the systematic natural signal are estimated, we have,

$$\stackrel{\wedge }{x}\approx x,\stackrel{\wedge }{f}\approx f$$

(2)

Given an estimation $\stackrel{\wedge }{x}$ of x, we can estimate f as follows,

$$\stackrel{\wedge }{f}=LPF(y-\stackrel{\wedge }{x})$$

(3)

where LPF is a specified low-pass filter. Substituting $\stackrel{\wedge }{f}=LPF(y-\stackrel{\wedge }{x})$ into $\stackrel{\wedge }{f}\approx f$, we have,

$$LPF(y-\stackrel{\wedge }{x})\approx f$$

(4)

Substituting $y=x+f+w$ into $LPF(y-\stackrel{\wedge }{x})\approx f$, we have,

$$LPF(y-\stackrel{\wedge }{x})\approx y-x-w$$

(5)

Substituting $\stackrel{\wedge }{x}\approx x$ into $LPF(y-\stackrel{\wedge }{x})\approx y-x-w$, we have,

$$LPF(y-\stackrel{\wedge }{x})\approx y-\stackrel{\wedge }{x}-w\iff (I-LPF)(y-\stackrel{\wedge }{x})\approx w$$

(6)

Defining HPF = I − LPF = H, which is a high-pass filter, thus we have,

$$HPF(y-\stackrel{\wedge }{x})\approx w$$

(7)

On the other hand, it should be noted that Equation (1) is a highly underdetermined equation, i.e., ill-posed or N − P hard problem [32,33], and has an infinite set of solutions because the number of unknowns is greater than the number of equations. Usually, convex optimization techniques are commonly used to estimate transient components from the observation signal, based on the aforementioned work [24,25,26,27,28,29,30,31], the estimation of x can be formulated as the optimization problem, i.e.,

$$\stackrel{\wedge }{x}=\arg \underset{x}{\min }\left\{\frac{1}{2}{\Vert H(y-x)\Vert }_2^2+\lambda {\Vert \mathit{D}x\Vert }_1\right\}$$

(8)

where H is a specified high-pass filter, i.e., the HPF in Equation (7), λ represents regularization term parameter, D is a matrix defined as $\mathit{D}=\left[\begin{array}{cccccc}-1,& 1& & & & \\ & & \dots & & & \\ & & & \dots & & \\ & & & & -1,& 1\end{array}\right]$, which determines the sparsity degree of the approximating value of x. Commonly, if x is a sparse component, i.e., most of the values in x tend to zero, the correspondingly optimization problem in Equation (8) can be estimated by L1-norm fused lasso optimization (LFLO) model [24,25,26,27,28,29,30,31], i.e.,

$$\stackrel{\wedge }{x}=\arg \underset{x}{\min }\left\{\frac{1}{2}{\Vert H(y-x)\Vert }_2^2+{\lambda }_0{\Vert x\Vert }_1+{\lambda }_1{\Vert \mathit{D}x\Vert }_1\right\}$$

(9)

where λ₀ and λ₁ are regularization parameters. The solution of LFLO model can be obtained by the soft-threshold method [34] and total variation de-noising (TVD) algorithm [35,36,37], we have,

$$x=\mathrm{soft}\left(\mathrm{tvd}\left(y,{\lambda }_2\right),{\lambda }_1\right)$$

(10)

where function $\mathrm{tvd}\left(\cdot ,\cdot \right)$ is the TVD algorithm and the soft-threshold model is given as follows,

$$\mathrm{soft}(x,\lambda )=\{\begin{array}{l}x-\lambda \frac{x}{\left|x\right|},\left|x\right|>\lambda \\ 0,\left|x\right|\le \lambda \end{array}$$

(11)

In addition, the high-pass filter H described above could be formulated as follows [36,37,38],

$$H={\mathit{A}}^{-1}\mathit{B}$$

(12)

where A and B are Toeplitz matrices.

2.2. Asymmetric Penalty Regularization Model

Compare with optimization algorithm Equations (8) and (9), to estimate the fault transient impulses x precisely, this work introduces a novel penalty regularization method, i.e., asymmetric and symmetric nonconvex penalty regularization model,

$$\stackrel{\wedge }{x}=\arg \underset{x}{\min }\left\{F\left(x\right)=\frac{1}{2}{\Vert H\left(y-x\right)\Vert }_2^2+{\lambda }_0{\displaystyle \sum _{n=0}^{N-1}{\theta }_{\epsilon }\left(x_n;r\right)}+{\displaystyle \sum _{i=1}^M{\lambda }_i}{\displaystyle \sum _{n=0}^{N-1}\varphi \left({\left[{\mathit{D}}_{i}x\right]}_n\right)}\right\}$$

(13)

where F(x) is the proposed objective cost function (OCF), penalty function ${\theta }_{\epsilon }\left(x_n;r\right)$ is a asymmetric and differentiable function, and $\varphi \left({\left[{\mathit{D}}_{i}x\right]}_n\right)$ is a symmetric and differentiable function, if term M = 2, matrix D₁ defined as ${\mathit{D}}_1=\left[\begin{array}{cccccc}-1,& 1& & & & \\ & & \dots & & & \\ & & & ...& & \\ & & & & -1,& 1\end{array}\right]$, and matrix D₂ defined as ${\mathit{D}}_2=\left[\begin{array}{cccccc}-1,& 2,& -1& & & \\ & -1,& 2,& -1,& & \\ & & \dots & & & \\ & & & -1,& 2,& -1\end{array}\right]$. The innovations of the novel compound regularizer model are as follows:

(I)

The M-term compound regularizers to estimate the fault transient impulses;

(II)

The compound regularizer model consists of symmetric and asymmetric penalty functions, wherein the symmetric penalty function is a differentiable function compared with the nondifferentiable function $\Vert x_i\Vert$ at point i = 0.

(III)

The MM algorithm is introduced for the solution of proposed compound regularization method, i.e., the OCF.

Based on this, the core issues of the proposed algorithm are (1) how to construct a symmetric and differentiable penalty function; (2) how to construct an asymmetric and differentiable penalty function and; (3) how to solve the proposed method based on the MM algorithm and make diagnosis results are more accurate than the traditional LFLO and nonconvex penalty regularization approaches.

For the first issue, traditional LFLO regularization approach uses the absolute value function ${\varphi }_A(x)=\Vert x\Vert$ as the penalty function; however, the common drawback of function ${\varphi }_A(x)=\Vert x\Vert$ is that this function is non-differentiable at zero point, which can cause some numerical problems. To address this problem, a non-linear approximation function of ${\varphi }_B(x)$ or ${\varphi }_c(x)$ is proposed, i.e.,

$${\varphi }_B(x)=\sqrt{{\left|x\right|}^2+\epsilon }$$

(14)

$${\varphi }_c(x)=\left|x\right|-\epsilon \log \left(\left|x\right|+\epsilon \right)$$

(15)

Note that when $\epsilon =0$, then the ${\varphi }_B(x)$ and ${\varphi }_c(x)$ degrade into the absolute value function ${\varphi }_A(x)$, while the $\epsilon >0$, the ${\varphi }_B(x)$ and ${\varphi }_c(x)$ are differentiable at zero point. The function ${\varphi }_A(x)$, ${\varphi }_B(x)$, ${\varphi }_c(x)$ and their first-order derivatives are listed in

Table 1. Symmetric penalty functions and their derivatives.

Functions	$\mathit{\varphi }\left(\mathit{x}\right)$	${\mathit{\varphi }}^{\prime }\left(\mathit{x}\right)$
${\varphi }_A\left(x\right)$	$\Vert x\Vert$	$\mathrm{Signal}\left(x\right)$
${\varphi }_B\left(x\right)$	$\sqrt{{\left|x\right|}^2+\epsilon }$	$x/\sqrt{{\left|x\right|}^2+\epsilon }$
${\varphi }_c\left(x\right)$	$\left|x\right|-\epsilon \log \left(\left|x\right|+\epsilon \right)$	$x/\left(\left|x\right|+\epsilon \right)$

.

In order for the non-linear approximation functions to maintain the reliable sparsity-inducting behavior of the original LFLO algorithm, the parameter $\epsilon$ should be presented to an adequately small positive value. Fortunately, for example, the parameter $\epsilon ={10}^{-5}$ and $\epsilon ={10}^{-6}$ which are small enough so that the numerical issues can be avoided.

For the second issue, inspired by the absolute value function ${\varphi }_A(x)=\Vert x\Vert$, and also in contrast to the symmetric and differentiable penalty function ${\varphi }_B(x)$ and ${\varphi }_c(x)$, here, a segmented function is proposed as follows,

$${\theta }_{\epsilon }\left(x\right)=\{\begin{array}{l}x,x>\epsilon \\ f\left(x\right),\Vert x\Vert \le \epsilon \\ -rx,x<\epsilon \end{array}$$

(16)

where $r>0$ is a positive constant. It should be noted that if we exclude the intermediate function f(x), the ${\theta }_{\epsilon }\left(x\right)$ is also degrade into the absolute value function ${\varphi }_A\left(x\right)$ when r = 1. Therefore, the main problem of Equation (16) will transform into a task of how to construct the intermediate function $f\left(x\right),-\epsilon \le x\le \epsilon$. To address this issue, we seek a majorizer (here the Majorization-minimization algorithm is utilized) as the approximation function of $f\left(x\right),-\epsilon \le x\le \epsilon$. In order to eliminate the issue that penalty function is non-differentiable at zero point, a simple quadratic equation (QE) is introduced accordingly,

$$g\left(x,v\right)=a{x}^2+bx+c$$

(17)

According to the theory of majorization-minimization [39,40], we have,

$$\begin{array}{l}g\left(v,v\right)=\theta \left(v,r\right),g^{\prime }\left(v,v\right)={\theta }^{\prime }\left(v,r\right)\\ g\left(s,v\right)=\theta \left(s,r\right),g^{\prime }\left(s,v\right)={\theta }^{\prime }\left(s,r\right)\end{array}$$

(18)

The parameter a, b, c and s are all functions of v, we have,

$$a=\frac{1+r}{4\left|v\right|},b=\frac{1-r}{2},c=\frac{\left(1+r\right)\left|v\right|}{4},s=-v$$

(19)

Substituting Equation (19) into $g\left(x,v\right)=a{x}^2+bx+c$, we have,

$$g\left(x,v\right)=\frac{1+r}{4\left|v\right|}{x}^2+\frac{1-r}{2}x+\frac{\left(1+r\right)\left|v\right|}{4}$$

(20)

Similarly, the numerical issue of Equation (20) will appear if the parameter v approaches zero. To address this problem, the sufficiently small positive value $\epsilon$ is used instead of $\left|v\right|$, thus, the segmented function Equation (16) can be rewritten as,

$${\theta }_{\epsilon }\left(x\right)=\{\begin{array}{ll}x,& x>\epsilon \\ \frac{1+r}{4\epsilon }{x}^2+\frac{1-r}{2}x+\frac{\left(1+r\right)\epsilon }{4},& \Vert x\Vert \le \epsilon \\ -rx,& x<\epsilon \end{array}$$

(21)

Hence, the new function ${\theta }_{\epsilon }\left(x\right)$ is a continuously differentiable function. The plot of the continuously differentiable asymmetric penalty function ${\theta }_{\epsilon }\left(x;r\right)$ is shown in Figure 2, and the function ${\theta }_{\epsilon }\left(x;r\right)$ is a second-order polynomial on $\left[-\epsilon ,\epsilon \right]$.

The third issue, will be solved and derived in Section 2.3 using the majorization-minimization algorithm.

2.3. The Solution of Proposed Model Based on Majorization-Minimization Algorithm

In this paper, the majorization-minimization (MM) algorithm is implemented to derive an iterative solution procedure for the proposed approach [38]. The G(x,v) is chosen as the majorizer of F(x). Specifically, the iterative solution procedure could be divided into three phases:

(a)

Majorizer of symmetric and differentiable function $\varphi \left({\left[{\mathit{D}}_{i}x\right]}_n\right)$ based on MM algorithm.

(b)

Majorizer of asymmetric and differentiable function ${\theta }_{\epsilon }\left(x_n;r\right)$ based on MM algorithm.

(c)

Majorizer of objective cost F(x) based on MM algorithm.

For problem (a), we first seek a majorizer $g\left(x,v\right)$ for $\varphi \left(x\right)$, i.e.,

$$\begin{array}{l}g\left(v,v\right)=\varphi \left(v\right),\mathrm{for}\mathrm{all}x,v\\ g\left(x,v\right)\ge \varphi \left(x\right)\end{array}$$

(22)

Since $\varphi \left(x\right)$ is symmetric function, we set $g\left(x,v\right)$ to be an even second-order polynomial, i.e.,

$$g\left(x,v\right)=m{x}^2+b$$

(23)

Thus, according to Equation (22) and $g\left(v,v\right)=\varphi \left(v\right)$ and $g^{\prime }\left(v,v\right)={\varphi }^{\prime }\left(v\right)$, we have,

$$m{x}^2+b=\varphi \left(v\right)\mathrm{and}2mv={\varphi }^{\prime }\left(v\right)$$

(24)

The parameter m and b can be computed as,

$$m=\frac{{\varphi }^{\prime }\left(v\right)}{2v}\mathrm{and}b=\varphi \left(v\right)-\frac{v}{2}{\varphi }^{\prime }\left(v\right)$$

(25)

Substituting Equation (25) in Equation (23), we have

$$g\left(x,v\right)=\frac{{\varphi }^{\prime }\left(v\right)}{2v}{x}^2+\varphi \left(v\right)-\frac{v}{2}{\varphi }^{\prime }\left(v\right)$$

(26)

Summing, we obtain,

$$\begin{array}{l}{\displaystyle \sum _{n}g\left(x_n,v_n\right)}={\displaystyle \sum _n\left[\frac{{\varphi }^{\prime }\left(v_n\right)}{2v_n}{x}_n{}^2+\varphi \left(v_n\right)-\frac{v_n}{2}{\varphi }^{\prime }\left(v_n\right)\right]}\\ =\frac{1}{2}{x}^T\frac{{\varphi }^{\prime }\left(v_n\right)}{v_n}x+{\displaystyle \sum _n\left[\varphi \left(v_n\right)-\frac{v_n}{2}{\varphi }^{\prime }\left(v_n\right)\right]}\\ =\frac{1}{2}{x}^T{\left[\mathsf{\Lambda }\left(v\right)\right]}_{n}x+c\left(v\right)\\ \ge {\displaystyle \sum _{n=0}^{N-1}\varphi \left(x_n\right)}\end{array}$$

(27)

where ${\left[\mathsf{\Lambda }\left(v\right)\right]}_n=\frac{{\varphi }^{\prime }\left(v_n\right)}{v_n}$ is diagonal matrix and $c\left(v\right)={\displaystyle \sum _n\left[\varphi \left(v_n\right)-\frac{v_n}{2}{\varphi }^{\prime }\left(v_n\right)\right]}$.

Therefore, based on Equation (27), we obtain,

$$\begin{array}{l}{\displaystyle \sum _{i=0}^M{\lambda }_i}{\displaystyle \sum _{n=0}^{N-1}g\left({\left[{\mathit{D}}_{i}x\right]}_n,{\left[{\mathit{D}}_{i}v\right]}_n\right)}\\ ={\displaystyle \sum _{i=0}^M\left[\frac{{\lambda }_i}{2}{\left({\mathit{D}}_{i}x\right)}^T\left[\frac{{\varphi }^{\prime }\left({\mathit{D}}_{i}v\right)}{{\mathit{D}}_{i}v}\right]\left({\mathit{D}}_{i}x\right)+{\displaystyle \sum _n\left[\varphi \left({\left[{\mathit{D}}_{i}v\right]}_n\right)-\frac{{\left[{\mathit{D}}_{i}v\right]}_n}{2}{\varphi }^{\prime }\left({\left[{\mathit{D}}_{i}v\right]}_n\right)\right]}\right]}\\ ={\displaystyle \sum _{i=0}^M\left[\frac{{\lambda }_i}{2}{\left({\mathit{D}}_{i}x\right)}^T\left[\mathsf{\Lambda }\left(D_{i}v\right)\right]\left({\mathit{D}}_{i}x\right)+c_i\left({\mathit{D}}_{i}v\right)\right]}\\ \ge {\displaystyle \sum _{i=0}^M{\lambda }_i}{\displaystyle \sum _{n=0}^{N-1}\varphi \left({\left[{\mathit{D}}_{i}x\right]}_n\right)}\end{array}$$

(28)

where ${\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right)\right]}_n=\frac{{\varphi }^{\prime }\left({\left[{\mathit{D}}_{i}v\right]}_n\right)}{{\left[{\mathit{D}}_{i}v\right]}_n}$ is diagonal matrix and $c_i\left({\mathit{D}}_{i}v\right)={\displaystyle \sum _n\left[\varphi \left({\left[{\mathit{D}}_{i}v\right]}_n\right)-\frac{{\left[{\mathit{D}}_{i}v\right]}_n}{2}{\varphi }^{\prime }\left({\left[{\mathit{D}}_{i}v\right]}_n\right)\right]}$.

For problem (b), assume that $g_0\left(x,v\right)$ is the majorizer of the asymmetric and differentiable function ${\theta }_{\epsilon }\left(x_n;r\right)$, since the $f(x)=\frac{1+r}{4\epsilon }{x}^2+\frac{1-r}{2}x+\frac{(1+r)\epsilon }{4},\Vert x\Vert \le \epsilon$, we have,

$$\{\begin{array}{l}{g}_0(x,v)=\frac{1+r}{4v}{x}^2+\frac{1-r}{2}x+\frac{(1+r)v}{4}\ge f(x),v>\epsilon \\ g_0(x,v)=-\frac{1+r}{4v}{x}^2+\frac{1-r}{2}x-\frac{(1+r)v}{4}\ge f(x),v<-\epsilon \end{array}$$

(29)

when $v>\epsilon$, then,

$$\begin{array}{l}{g}_0(x,v)-f(x)=\left(\frac{1+r}{4v}-\frac{1+r}{4\epsilon }\right)x^2+(v-\epsilon )\frac{1+r}{4}\\ =\frac{(1+r)(v-\epsilon )(v\epsilon -x^2)}{4v\epsilon }>0\end{array}$$

(30)

when $v<-\epsilon$, then,

$$\begin{array}{l}{g}_0(x,v)-f(x)=\left(-\frac{1+r}{4v}-\frac{1+r}{4\epsilon }\right)x^2-(v+\epsilon )\frac{1+r}{4}\\ =-\frac{(1+r)(v+\epsilon )(v\epsilon +x^2)}{4v\epsilon }>0\end{array}$$

(31)

Therefore, the majorizer of the asymmetric and differentiable function ${\theta }_{\epsilon }\left(x_n;r\right)$ is obtained,

$$\{\begin{array}{ll}{g}_0(x,v)=\frac{1+r}{4\left|v\right|}{x}^2+\frac{1+r}{2}x+\left|v\right|\frac{1+r}{4},& \left|v\right|>\epsilon \\ g_0(x,v)=\frac{1+r}{4\epsilon }{x}^2+\frac{1+r}{2}x+\epsilon \frac{1+r}{4},& \left|v\right|\le \epsilon \end{array}$$

(32)

Summing, we obtain,

$$\begin{array}{l}{\displaystyle \sum _{n=0}^{N-1}{g}_0\left(x_n,v_n\right)}=x^T\left[\mathsf{\Gamma }\left(v\right)\right]x+b^{T}x+c\left(v\right)\\ \ge {\displaystyle \sum _{n=0}^{N-1}{\theta }_{\epsilon }\left(x_n,r\right)}\end{array}$$

(33)

where $\mathsf{\Gamma }\left(v\right)$ is a diagonal matrix, i.e., ${\left[\mathsf{\Gamma }\left(v\right)\right]}_n=\left(1+r\right)/4\left|v_n\right|,\left|v_n\right|\ge \epsilon$ and ${\left[\mathsf{\Gamma }\left(v\right)\right]}_n=\left(1+r\right)/4\epsilon ,\left|v_n\right|\le \epsilon$, and ${\left[b\right]}_n=\left(1-r\right)/2$.

For problem (c), based on Equations (28) and (33), the majorizer of F(x) based on MM algorithm is given by,

$$\begin{array}{l}G\left(x,v\right)=\frac{1}{2}{\Vert H\left(y-x\right)\Vert }_2^2+{\lambda }_0{x}^T\left[\mathsf{\Gamma }\left(v\right)\right]x+{\lambda }_0{b}^{T}x\\ +{\displaystyle \sum _{i=1}^M\left[\frac{{\lambda }_i}{2}{\left({\mathit{D}}_{i}x\right)}^T\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right)\right]\left({\mathit{D}}_{i}x\right)\right]}+c\left(v\right)\end{array}$$

(34)

Minimizing G(x,v) with respect to x yields,

$$x=H^{T}H+2{\lambda }_0\mathsf{\Gamma }\left(v\right)+{\displaystyle \sum _{i=1}^M{\lambda }_i{\mathit{D}}_i^T{\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right){\mathit{D}}_i\right]}^{-1}\left(H^{T}Hy-{\lambda }_{0}b\right)}$$

(35)

Substituting H = BA⁻¹ in Equation (35), we have,

$$\begin{array}{l}x=\mathit{A}{\left\{{\mathit{B}}^T\mathit{B}+{\mathit{A}}^T\left(2{\lambda }_0\mathsf{\Gamma }\left(v\right)+{\displaystyle \sum _{i=1}^M{\lambda }_i{\mathit{D}}_i^T\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right)\right]{\mathit{D}}_i}\right)\mathit{A}\right\}}^{-1}\times \left({\mathit{B}}^T\mathit{B}{\mathit{A}}^{-1}y-{\lambda }_0{\mathit{A}}^{T}b\right)\\ =\mathit{A}{\left({\mathit{B}}^T\mathit{B}+{\mathit{A}}^T\mathit{M}\mathit{A}\right)}^{-1}\left({\mathit{B}}^T\mathit{B}{\mathit{A}}^{-1}y-{\lambda }_0{\mathit{A}}^{T}b\right)\\ =\mathit{A}{\mathit{Q}}^{-1}\left({\mathit{B}}^T\mathit{B}{\mathit{A}}^{-1}y-{\lambda }_0{\mathit{A}}^{T}b\right)\end{array}$$

(36)

where matrix $\mathit{M}=2{\lambda }_0\mathsf{\Gamma }\left(v\right)+{\displaystyle \sum _{i=1}^M{\lambda }_i{\mathit{D}}_i^T\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right)\right]{\mathit{D}}_i}$ and matrix $\mathit{Q}={\mathit{B}}^T\mathit{B}+{\mathit{A}}^T\mathit{M}\mathit{A}$.

Finally, by using the above formulas, the fault transient impulses x can be obtained by the following iterations,

$${\mathit{M}}^{(k)}=2{\lambda }_0\mathsf{\Gamma }\left(x^{(k)}\right)+{\displaystyle \sum _{i=1}^M{\lambda }_i{\mathit{D}}_i^T\left[\mathsf{\Lambda }\left({\mathit{D}}_i{x}^{(k)}\right)\right]{\mathit{D}}_i}$$

(37)

$${\mathit{Q}}^{(k)}={\mathit{B}}^T\mathit{B}+{\mathit{A}}^T{\mathit{M}}^{(k)}\mathit{A}$$

(38)

$$x^{(k+1)}=\mathit{A}{\left[{\mathit{Q}}^{(k)}\right]}^{-1}\left({\mathit{B}}^T\mathit{B}{\mathit{A}}^{-1}y-{\lambda }_0{\mathit{A}}^{T}b\right)$$

(39)

In conclusion, the complete steps of the proposed algorithm are summarized as follows,

(1)

Inputs: signal y, r ≥ 1, matrix A, matrix B, ${\lambda }_i,i=0,1,\dots ,M$, k = 0;

(2)

$\mathit{E}={\mathit{B}}^T\mathit{B}{\mathit{A}}^{-1}y-{\lambda }_0{\mathit{A}}^{T}b$

(3)

Initialize x = y;

(4)

Repeat the following iterations:

$${\left[\mathsf{\Gamma }\left(v\right)\right]}_n=\left(1+r\right)/4\left|v_n\right|,\left|v_n\right|\ge \epsilon ;$$

$${\left[\mathsf{\Gamma }\left(v\right)\right]}_n=\left(1+r\right)/4\epsilon ,\left|v_n\right|\le \epsilon ;$$

$${\left[\mathsf{\Lambda }\left({\mathit{D}}_{i}v\right)\right]}_n=\frac{{\varphi }^{\prime }\left({\left[{\mathit{D}}_{i}v\right]}_n\right)}{{\left[{\mathit{D}}_{i}v\right]}_n},i=0,1,2,\dots ,M;$$

$${\mathit{M}}^{(k)}=2{\lambda }_0\mathsf{\Gamma }\left(x^{(k)}\right)+{\displaystyle \sum _{i=1}^M{\lambda }_i{\mathit{D}}_i^T\left[\mathsf{\Lambda }\left({\mathit{D}}_i{x}^{(k)}\right)\right]{\mathit{D}}_i};$$

$${\mathit{Q}}^{(k)}={\mathit{B}}^T\mathit{B}+{\mathit{A}}^T{\mathit{M}}^{(k)}\mathit{A};$$

$$x^{(k+1)}=\mathit{A}{\left[{\mathit{Q}}^{(k)}\right]}^{-1}\mathit{E};$$

(5)

If the stopping criterion is satisfied, then output signal x, otherwise, k = k + 1, and go to step (4).

(6)

Output: signal x.

2.4. Parameters Selection

In this section, the regularization parameters λ_i are set as [26]:

$$\begin{array}{l}{\lambda }_0={\beta }_0\sigma \\ {\lambda }_1={\beta }_1\sigma =\gamma \left(1-{\beta }_0\right)\sigma \\ {\lambda }_2={\beta }_0\sigma \end{array}$$

(40)

where β₀ and β₁ are the constants so as to maximize the signal-to-noise-rate (SNR), β₀ and γ are typically set up to be constants, i.e., β₀ = [0.5, 1], γ = [7.5, 8], and σ is the standard deviation (SD) of the external noise. In practical engineering applications, the SD of the external noise in Equation (40) could be computed using the healthy data and fault data under the same operating environment. Moreover, when the healthy data is not available or unknown, the SD of the external noise can still be estimated by the following formula,

$$\stackrel{\wedge }{\sigma }=MAD\left(y\right)/0.6745$$

(41)

where the Equation (41) is a noise level estimator that used for traditional wavelet denoising in ref. [41], and MAD(y) represents the median absolute deviation (MAD) of observation signal y, i.e.,

$$MAD\left(y\right)=\mathrm{median}\left(\left|y_i-\mathrm{median}\left(y\right)\right|\right),i=1,2,\dots ,N$$

(42)

To better understand the asymmetric penalty sparse decomposition algorithm, a chromatogram signal is analyzed by the proposed APSD algorithm [38]. For illustration, Figure 3a exhibits the raw chromatogram signal. Figure 3b,c respectively shows the LFC and HFC that generated by the proposed APSD. Note that the main evolutionary trend, i.e., low frequency component, is well estimated as illustrated in Figure 3b, meanwhile, the estimated peaks, i.e., high frequency component, illustrated in Figure 3c, are well delineated. Therefore, the LFC and HFC components could be captured by the asymmetric penalty sparse decomposition algorithm.

3. Wavelet Neural Network and ARMA-RLS Algorithm

3.1. Wavelet Neural Network Algorithm

Wavelet neural network (WNN) is a recently developed neural network with a topology and structure that are similar to that of the back-propagation neural network (BPNN) [42,43,44,45]. The network structure is composed of the input layer, hidden layer and output layer; the topology structure of the WNN is shown in Figure 4, wherein the customary sigmoid function (CSF) is replaced by the wavelet basis functions h(j) as activation function for the neurons in hidden layer. Hence, the combination of wavelet transform and BPNN has the advantage of both wavelet analysis and neural network; the data is transmitted forward and the prediction error is propagated backward, so as to achieve a more accurate predictive data.

Generally, prediction accuracy and generalization ability will be affected by the choice of wavelet base function; compared to orthogonal wavelet, Gauss spline wavelet and Mexico hat wavelet, the Morlet wavelet has the smallest error and the reliable computational stability [46,47], thus this study employed Morlet wavelet as the activation function of hidden layer nodes, the formula is given below,

$$y=\cos \left(1.75x\right)e^{-x^2/2}$$

(43)

In Figure 4, the input data are represented by X₁, X₂, …, X_k, and the predicted outputs are denoted by Y₁, Y₂, …, Y_m, w_ij is the link-weight between the input layer and hidden layer. When the input data is x_i (i = 1, 2, …, k), the output of hidden layer can be calculated as,

$$h\left(j\right)=h_j\left(\frac{{\displaystyle \sum _{i=1}^k{w}_{ij}{x}_i-b_j}}{a_j}\right),j=1,2,\dots ,l$$

(44)

where h(j) is the output of the j-th hidden layer and also represents the wavelet basis function, a_j is the scaling factor of function h(j), b_j is the translation factor of function h(j), The output of the WNN can be expressed as,

$$y\left(k\right)={\displaystyle \sum _{i=1}^l{w}_{ik}h\left(i\right)},k=1,2,\dots ,m$$

(45)

where w_ik denotes the link-weight between the hidden layer and output layer, h(i) is the output of the i-th hidden layer, l represents the number of hidden layer and m represents the number of output layer. According to the fundamental principle of BPNN and gradient descent learning algorithm, the corresponding adjustment process of network weights and wavelet basis function is as follows,

(1) Calculate network prediction error, i.e.,

$$e={\displaystyle \sum _{k=1}^m{y}_n(k)}-y(k)$$

(46)

where y_n(k) is desired outputs and y(k) predicted outputs generated by the WNN method.

(2) network weights and wavelet basis function can be adjusted by network prediction error, i.e.,

$$w_{n,k}^{(i+1)}=w_{n,k}^i+\Delta w_{n,k}^{(i+1)}$$

(47)

$$a_k^{(i+1)}=a_k^i+\Delta a_k^{(i+1)}$$

(48)

$$b_k^{(i+1)}=b_k^i+\Delta b_k^{(i+1)}$$

(49)

where the parameters $\Delta w_{n,k}^{(i+1)}$, $\Delta a_k^{(i+1)}$ and $\Delta b_k^{(i+1)}$ could be calculated as follows,

$$\Delta w_{n,k}^{(i+1)}=-\eta \frac{\partial e}{\partial w_{n,k}^{(i)}}$$

(50)

$$\Delta a_k^{(i+1)}=-\eta \frac{\partial e}{\partial a_k^{(i)}}$$

(51)

$$\Delta b_k^{(i+1)}=-\eta \frac{\partial e}{\partial b_k^{(i)}}$$

(52)

where η is learning rate. The training and correcting algorithm steps of the network parameters are as follows:

Step 1: Network initialization. The expansion parameters a_k and translation parameters b_k of the wavelet basis function as well as the network learning rate η, the link-weights are w_ij and w_jk, respectively, the error threshold ε, and maximum iterations T.

Step 2: Samples classification. Samples can be divided into training samples and test samples, wherein the training samples are applied for network training and the test samples are applied for testing prediction accuracy.

Step 3: Predicted output. The training samples are feed into the WNN network, and the predicted outputs are calculated. Then the error and gradient vectors of the output of WNN and the desired output are obtained.

Step 4: Weights adjustment. The parameters of the wavelet basis function and the back propagation of error corrects the weights of the WNN.

Step 5: End conditions. The training algorithm will judge whether the targeted error is less than the predetermined threshold ε (ε > 0) or exceeds the maximum iterations. If yes, the network training is stopped, otherwise, the algorithm returns to step 3. The training and correcting steps of the WNN are shown in Figure 5.

3.2. ARMA Combined With Recursive Least Squares (RLS) Algorithm

3.2.1. ARMA Review of ARMA Model

A stochastic process X_t is called an autoregressive moving-average process (ARMA) with order p and q, namely ARMA (p, q) [48,49], if the process is stationary and satisfies a linear stochastic difference equation of the form,

$$X_t={\varphi }_1{X}_{t-1}+\dots +{\varphi }_p{X}_{t-p}+e_t+{\theta }_1{e}_{t-1}+\dots +{\theta }_q{e}_{t-q}$$

(53)

where e_t is white gaussian noise (WGN), i.e., $e_t\sim WN(0,{\sigma }^2)$, parameters ${\varphi }_1,{\varphi }_2,\dots ,{\varphi }_p$ and ${\theta }_1,{\theta }_2,\dots ,{\theta }_q$ are coefficients of AR(p) and MA(q) models, and the polynomials are as follows,

$$\varphi (z)=1-{\varphi }_{1}z-{\varphi }_{2}z-\dots -{\varphi }_p{z}^p$$

(54)

$$\theta (z)=1+{\theta }_{1}z+{\theta }_{2}z+\dots +{\theta }_p{z}^p$$

(55)

3.2.2. Recursive Least Squares (RLS) Algorithm

Given an input samples set $\left\{u(1),u(2),\dots ,u(N)\right\}$ and a desired response set $\left\{d(1),d(2),\dots ,d(N)\right\}$, the output y(n) could be computed by linear filter method as follows,

$$y\left(n\right)={\displaystyle \sum _{k=0}^M{w}_{k}u\left(n-k\right)},n=0,1,2,\dots$$

(56)

Minimizing the sum of the error squares, we have,

$$\begin{array}{l}\epsilon \left(n\right)=\epsilon \left({\omega }_1\left(n\right),{\omega }_2\left(n\right),\dots ,{\omega }_{M-1}\left(n\right)\right)\\ ={\displaystyle \sum _{i=i_1}^n\beta \left(n,i\right)\times \left[e{\left(i\right)}^2\right]}\\ ={\displaystyle \sum _{i=i_1}^n\beta \left(n,i\right)\times {\left[d\left(i\right)-{\displaystyle \sum _{k=0}^{M-1}{\omega }_k\left(n\right)u\left(i-k\right)}\right]}^2}\end{array}$$

(57)

where the error e(i) is $e(i)=d(i)-y(i)=d(i)-{\displaystyle \sum _{k=0}^{M-1}{\omega }_k(n)u(i-k)}$, and the $\beta (n,i)$ is the forgetting factor and $0<\beta (n,i)\le 1,i=1,2,\dots ,n$. To simplify the writing format, taking the form as $\beta (n,i)={\lambda }^{n-i},i=1,2,\dots ,n$. Thus, the sum of error squares can be rewritten as,

$$\begin{array}{l}\epsilon \left(n\right)=\epsilon \left({\omega }_1\left(n\right),{\omega }_2\left(n\right),\dots ,{\omega }_{M-1}\left(n\right)\right)\\ ={\displaystyle \sum _{i=i_1}^n{\lambda }^{n-i}\times {\left[d\left(i\right)-{\displaystyle \sum _{k=0}^{M-1}{\omega }_k\left(n\right)u\left(i-k\right)}\right]}^2}\end{array}$$

(58)

Defining the following formulas,

$$u^{\prime }(i)=\sqrt{{\lambda }^{n-i}}u(i);d^{\prime }(i)=\sqrt{{\lambda }^{n-i}}d(i)$$

(59)

Then the Equation (58) can be rewritten as,

$$\begin{array}{l}\epsilon \left(n\right)={\displaystyle \sum _{i=i_1}^n{\lambda }^{n-i}\times {\left[d\left(i\right)-{\displaystyle \sum _{k=0}^{M-1}{\omega }_k\left(n\right)u\left(i-k\right)}\right]}^2}\\ ={\displaystyle \sum _{i=i_1}^n{\left[d^{\prime }\left(i\right)-{\displaystyle \sum _{k=0}^{M-1}{\omega }_k\left(n\right)u^{\prime }\left(i-k\right)}\right]}^2}\end{array}$$

(60)

which is the standard least squares (LS) criterion, the solution of the LS can be obtained as,

$$\begin{array}{l}\omega \left(n\right)={\left({\displaystyle \sum _{i=i_1}^n{u}^{\prime }\left(i\right)u^{\prime }{\left(i\right)}^T}\right)}^{-1}{\displaystyle \sum _{i=i_1}^n{u}^{\prime }\left(i\right)d^{\prime }\left(i\right)}\\ ={\left({\displaystyle \sum _{i=i_1}^n{\lambda }^{n-i}u\left(i\right)u{\left(i\right)}^T}\right)}^{-1}{\displaystyle \sum _{i=i_1}^n{\lambda }^{n-i}u\left(i\right)d\left(i\right)}\\ ={\left[\mathsf{\Phi }\left(n\right)\right]}^{-1}\psi \left(n\right)\end{array}$$

(61)

where parameters $\mathsf{\Phi }(n)={\displaystyle \sum _{i=1}^n{\lambda }^{n-i}u(i)u{(i)}^T}$ and $\psi (n)={\displaystyle \sum _{i=1}^n{\lambda }^{n-i}u(i)d(i)}$. Based on $\omega \left(n\right)={\left[\mathsf{\Phi }\left(n\right)\right]}^{-1}\psi \left(n\right)$, thus the $\omega (n)$ at time n − 1 could be obtained, i.e.,

$$\omega \left(n-1\right)={\left[\mathsf{\Phi }\left(n-1\right)\right]}^{-1}\psi \left(n-1\right)$$

(62)

Therefore, the variables $\mathsf{\Phi }(n)$ and $\psi (n)$ can be rewritten using $\mathsf{\Phi }(n-1)$ and $\psi (n-1)$, i.e.,

$$\begin{array}{l}\mathsf{\Phi }\left(n\right)={\displaystyle \sum _{i=1}^n{\lambda }^{n-i}u\left(i\right)u{\left(i\right)}^T}=\lambda {\displaystyle \sum _{i=1}^{n-1}{\lambda }^{n-1-i}u\left(i\right)u{\left(i\right)}^T}+u\left(n\right)u{\left(n\right)}^T\\ =\lambda \mathsf{\Phi }\left(n-1\right)+u\left(n\right)u{\left(n\right)}^T\end{array}$$

(63)

and

$$\begin{array}{l}\psi (n)={\displaystyle \sum _{i=1}^n{\lambda }^{n-i}u(i)d(i)}=\lambda {\displaystyle \sum _{i=1}^{n-1}{\lambda }^{n-1-i}u(i)d(i)}+u(n)d(n)\\ =\lambda \psi (n-1)+u(n)d(n)\end{array}$$

(64)

The matrix inversion formula of ${\mathsf{\Phi }}^{-1}(n)$ is as follows,

$${\mathsf{\Phi }}^{-1}(n)={\lambda }^{-1}{\mathsf{\Phi }}^{-1}(n-1)-\frac{{\lambda }^{-2}{\mathsf{\Phi }}^{-1}(n-1)u(n)u^T(n){\mathsf{\Phi }}^T(n-1)}{1+{\lambda }^{-1}{u}^T(n){\mathsf{\Phi }}^{-1}(n-1)u(n)}$$

(65)

Denoting

$$P(n)={\mathsf{\Phi }}^{-1}(n)$$

(66)

and

$$k(n)=\frac{{\lambda }^{-1}P(n-1)u(n)}{1+{\lambda }^{-1}{u}^T(n)P(n-1)u(n)}=P(n)u(n)$$

(67)

Therefore,

$$P(n)={\lambda }^{-1}P(n-1)-{\lambda }^{-1}k(n)u^T(n)P(n-1)$$

(68)

The main time-update equation $\omega (n)$ can be derived as,

$$\begin{array}{l}\omega (n)={\left[\mathsf{\Phi }(n)\right]}^{-1}\psi (n)=P(n)\psi (n)\\ =P(n)\left(\lambda \psi (n-1)+u(n)d(n)\right)\\ =P(n)\left(\lambda \mathsf{\Phi }(n-1)\omega (n-1)+u(n)d(n)\right)\\ =P(n)\left\{\left[\mathsf{\Phi }(n)-u(n)u^T(n)\right]\omega (n-1)+u(n)d(n)\right\}\\ =\omega (n-1)-P(n)u(n)u^T(n)\omega (n-1)+P(n)u(n)d(n)\\ =\omega (n-1)+P(n)u(n)\left[d(n)-u^T(n)\omega (n-1)\right]\\ =\omega (n-1)+P(n)u(n)\alpha (n)\\ =\omega (n-1)+k(n)\alpha (n)\end{array}$$

(69)

where $\alpha (n)=d(n)-u^T(n)\omega (n-1)$ is the innovation process. In conclusion, the prediction processes of the ARMA combined with RLS algorithm are summarized as follows,

Step 1: Algorithm initialization. The forgetting factor $0<\lambda \le 1$, the predicted steps n.

Step 2: Initial parameter calculation. Calculate the initial order p and q of ARMA (p, q) model using Akaike information criterion (AIC) [50] based on historical data.

Step 3: Coefficients extraction. Extract the coefficients ${\varphi }_1,{\varphi }_2,\dots ,{\varphi }_p$ and ${\theta }_1,{\theta }_2,\dots ,{\theta }_q$ as the input u(i) of RLS algorithm, i.e., $u(i)=\left[1,{\varphi }_1,{\varphi }_2,\dots ,{\varphi }_p,{\theta }_1,{\theta }_2,\dots ,{\theta }_q\right],i=1,2,\dots ,p+q$.

Step 4: Prediction iteration based on RLS algorithm. Repeat the following iterations,

$$k(i)=\frac{{\lambda }^{-1}P(i-1)u(i)}{1+{\lambda }^{-1}{u}^T(i)P(i-1)u(i)};$$

$$\alpha (i)=d(i)-u^T(i)\omega (i-1);$$

$$\omega (i)=\omega (i-1)+k(i)\alpha (i);$$

$$P(i)={\lambda }^{-1}P(i-1)-{\lambda }^{-1}k(i)u^T(i)P(i-1);$$

$$y(i)=y(i-1)+k\times \alpha (i);$$

Step 5: Judgment of the end conditions. If the iterative step is satisfied, then output predicted signal y(n), otherwise, i = i + 1, and go to step (4).

4. Experimental Validations

The vibration data collected from accelerated life tests (ALT) of rolling bearings were employed to validate the effectiveness of the proposed approach, the experimental setup is shown in Figure 6a. The experimental data were acquired and published by the IEEE-PHM Association [51,52]. The experimental platform included National Instruments (NI) data acquisition card, pressure regulator, cylinder pressure, force sensors, motor, speed sensor, torque-meter, accelerometers and tested bearing, etc. During the process of the experiment, the rotating speeds of the bearing were set to 1650 r/min and 1800 r/min, and 17 bearings were chosen during all those experiments. The sampling frequency was 25.6 kHz, and 2560 sample points (i.e., 0.1 s) were recorded each 10 s. The experimental tests were stopped if the amplitude of time-domain data exceeded 20 g. As shown in Figure 6b, the severe wear in bearing elements and the severe spalling failure in inner race were observed after dismantling.

Figure 7a shows the whole lifetime data of a tested bearing in the time-domain, and the time-domain waveform of the tested bearing in the normal stage and the time-domain waveform of the tested bearing in the catastrophic failure stage are shown in Figure 7b,c, respectively. In the normal stage, the failure impulses cannot be observed in the time-domain, the range of the amplitude is [−2 g, 2 g]. However, in the failure stage, the range of the amplitude is [−50 g, 50 g], and there exist obvious impulses with time interval of $\Delta T=0.0059s$, which is corresponding to the fault frequency of 169 Hz.

In this experiment, four bearings (bearing 1, bearing 2, bearing 3 and bearing 4) are randomly selected and employed as testing targets to evaluate the prediction performance. Each bearing is degraded during the accelerated life tests without implanting any artificial fault in advance.

(1)

bearing 1: operating conditions: speed 1800 rpm and load 4000 N; whole test life: 28,030 s;

(1)

bearing 2: operating conditions: speed 1800 rpm and load 4000 N; whole test life: 18,020 s;

(1)

bearing 3: operating conditions: speed 1650 rpm and load 4200 N; whole test life: 7970 s;

(1)

bearing 4: operating conditions: speed 1800 rpm and load 4000 N; whole test life: 14,280 s.

Depending on the diversity of operating conditions and the manufacturing accuracy of tested bearings, the effective running time may be different for different bearings. Therefore, the degradation trajectories of the health indicators are different for each tested bearing. Figure 8 shows the peak-to-peak values of the whole lifetime of bearing 1. Accordingly, the health indicators, i.e., equivalent vibration intensity (EVI) [9,11], Kurtosis and EVI of bearing 2, bearing 3 and bearing 4 are illustrated in Figure 9a–c, respectively. It is seen that the amplitudes of bearings 1, 3 and 4 have gradual increasing trends, in addition, the whole test life of bearing 3 is the shortest due to harsh operating conditions, which indicates that the extremely failures are occurred before the experiment stops, thus, representing abrupt degradation processes, whereas the EVI amplitudes of bearings 2 show gradual increases; it might be concluded that the design/manufacturing quality and fatigue resistance strength are much higher than others under the same operating conditions.

Taking bearing 1 as an example, as shown in Figure 8b, because of the abrupt degradation time series is the most interesting and difficult part; thus, datasets 2001 to 2803 are selected as survey regions, where the dataset from 2001 to 2703 are selected as a historical curve, and the remaining 100 datasets, i.e., dataset 2704 to dataset 2803, are a predicted region. Specifically, the proposed APSD method is adopted to process the peak-to-peak curve of bearing 1, since the standard deviation (SD) of the peak-to-peak is unknown, it can be calculated by the following equation, i.e., $\stackrel{\wedge }{\sigma }=MAD\left(y\right)/0.6745=2.1750$. Therefore, the related parameter specification of the APSD approach are summarized in

Table 2. The parameters setting of APSD method for bearing 1.

Parameters β0	Parameters γ	Regularization Parameter λ0	Regularization Parameter λ1	Regularization Parameter λ2	M-Term	Iteration Times
β0 = 0.8	γ = 7.5	λ0 = 1.7400	λ1 = 3.2625	λ2 = 1.7400	2	50

. The decomposition results are presented in Figure 10, Figure 10a is the low frequency component and Figure 10b is the high frequency component, respectively. It should be noted that the low frequency component almost coincides with the actual peak-to-peak trends, especially in the abrupt degradation regions, which means the degradation processes and trend could be reflected by the low frequency component.

Furthermore, the WNN algorithm and ARMA-RLS model are introduced for prediction. For the prediction of a low frequency component, the number of input neurons, hidden neurons, output neurons, training iterations, learning rate for WNN are set at 7, 10, 1, 300 and 0.01, respectively. For the prediction of high frequency components, the modelling orders, i.e., p and q, are calculated by Akaike information criterion (AIC) criterion, and then all the parameters of ARMA are fed into the RLS learning algorithm; to recursively estimate and update the related parameters, the forgetting factor is set to 0.99. The prediction curves of the LFC and HFC for rolling bearing are illustrated in Figure 11a,b, respectively. As can be seen in Figure 11, the prediction data generated by the ANN and ARMA-RLS models are able to reasonably trace the variation trend of original peak-to-peak time series. The final predicted result can be correspondingly obtained by integrating the predicted LFC and HFC components, as shown in Figure 12.

As benchmarking approaches for the prognosis of bearing degradation trajectories, the ARMA, FARIMA and WNN, largest Lyapunov (LLyap) exponent models are employed for comparison. All the predicted results with the actual values are illustrated in Figure 12. As shown in Figure 12, all the benchmarking approaches could generally track the peak-to-peak fluctuation trend well except FARIMA because the memory function might be eliminated due to difference and inverse difference operating. Importantly, compared with the predicted results generated by benchmark approaches, the predictions created by the proposed method are closer to the actual values, and the specific quantitative comparison of the prediction performance are presented in following section.

Five assessment criteria of prediction results are introduced and calculated including mean absolute error (MAE), average relative error (ARE), root-mean-square error (RMSE), normalized mean square error (NMSE) and maximum of absolute error (Max-AE), which are denoted by,

$$MAE=\frac{1}{N}{\displaystyle \sum _{k=1}^N\left|x(k)-\stackrel{\wedge }{x(k)}\right|}$$

(70)

$$ARE=\frac{1}{N}{\displaystyle \sum _{k=1}^N\frac{\left|x(k)-\stackrel{\wedge }{x(k)}\right|}{x(k)}}$$

(71)

$$RMSE=\sqrt{{\displaystyle \sum _{k=1}^N\frac{1}{N-1}{\left[x(k)-\stackrel{\wedge }{x(k)}\right]}^2}}$$

(72)

$$NMSE=\frac{{\displaystyle \sum _{k=1}^N{\left[x(k)-\stackrel{\wedge }{x)(k)}\right]}^2}}{{\displaystyle \sum _{k=1}^N{\left[x(k)-\overline{x(k)}\right]}^2}}$$

(73)

$$Max-AE=\max \left(\left|x(k)-\stackrel{\wedge }{x(k)}\right|\right)$$

(74)

where N is the number of time points, $x(k)$ and $\stackrel{\wedge }{x(k)}$ are actual data and predicted data, respectively. Generally speaking, obviously, the smaller MAE, ARE, RMSE, NMSE and Max-AE values, which means lower prediction errors and higher prediction accuracies. The performance of the proposed methodology is evaluated by computed prediction errors. The computed prediction errors are summarized in

Table 3. The computed prediction errors of proposed method and benchmark methods.

Index	ARMA	FARIMA	WNN	L-Lyap	Proposed
Mean Absolute Error-MAE	0.5880	17.2700	8.2364	3.2453	0.9770
Average relative error-ARE	0.1872	0.4666	0.1583	0.2691	0.2599
Root-Mean-Square Error-RMSE	5.9094	173.5702	82.7793	32.6164	9.8190
Normalized Mean Square Error-NMSE	0.1657	2.4144	0.7173	0.3008	0.2244
Maximum of Absolute Error-MaxAE	21.9865	75.0981	36.4112	28.3548	32.9561

. From the results in Table 3, it is noticed that the proposed method has relatively higher accuracy than FARIMA, WNN and L-Lyap methods. In addition, it is observed that the ARMA model has smaller errors among these models, but the tracking trend of LFC and fluctuation trend of HFC cannot be reflected at all; see black line in Figure 12. This indicates that the presented prediction approach has certain application potentials.

Meanwhile, to better evaluate the performances of all the methods from a statistical perspective, box plots were introduced. The box plots of all the errors based on benchmark approaches and proposed method are shown in Figure 13. The results show that the median values of the prediction errors converge to 0 based on proposed method, which demonstrate that the predicted data generated by the proposed approach are close to their true values.

Furthermore, the prediction performances of the remaining bearings are illustrated in this section. The model parameter settings of the APSD, WNN, ARMA-RLS are similar to those in the aforementioned steps, the specific parameters are omitted here for the sake of simplification. The decomposition results, i.e., the LFC and HFC, of the bearing 2, bearing 3 and bearing 4 are respectively presented in Figure 14a–f; overall, it can be found that the low frequency component almost coincides with the actual health indicators trends, especially in the abrupt degradation regions.

The final predicted results of bearing 2, bearing 3 and bearing 4 are shown in Figure 15a–c, respectively. It is clear that all the predicted data converge to the actual data as time goes on. Interestingly, one can find that, the abrupt degradation points cannot be followed starting point, for example, the point 89 and point 90 in Figure 15b and point 72 in Figure 15c, nevertheless, subsequent points after those abrupt points could be restored, the reason is that the modeling parameters are updated by the artificial neural network and recursive least squares learning algorithm step by step, and the error is eliminated to the minimal range. The proposed two-step prediction method is more robust to the data collection errors. In conclusion, the proposed method performs best in the degradation trend prediction of the rolling bearings.

Additionally, Figure 16 summarizes the box plots of the absolute error (AE) of the remaining three bearings. Analogously, the box plots show that the median values of the absolute error of those three bearings converge to 0, which further reflects the effectiveness of the proposed method. Therefore, the proposed decomposition algorithm and prediction models have accurate prediction results.

5. Conclusions

Accurate prognosis of the degradation trend of rotating machinery plays an important role in industrial applications. Currently, most developments in the mechanical fault prognostics area have been targeted towards directly utilizing degradation-based data to trace the degradation trajectories, very few studies have used the idea of sparse decomposition. In this work, a novel intelligent prediction approach based on asymmetric penalty sparse decomposition (APSD) combined with WNN and ARMA-RLS models for health indicator degradation trajectories of four rolling bearings is proposed. The original health indicators od degradation trajectory are rearranged as two components, i.e., LFC and HFC. In particular, the HFC corresponds to the stable change around the zero line of health indicators, whereas the LFC is essentially related to the evolutionary trend of health indicators which rarely occurs in practice. The LFC and HFC are, respectively, predicted using the WNN and ARMA-RLS models. The final degradation regions (e.g., last 100 points) is correspondingly obtained by combining the predicted LFC and predicted HFC. Experimental results of four rolling bearings have demonstrated the superiority of the proposed method in terms of quantitative and qualitative evaluation compared to three commonly-used parametric-based and nonparametric-based methods, i.e., ARMA, FARIMA, WNN, L-Lyap methods.

This observation motivates the study of an integrated method for bearing prediction, which combines the strength of both parametric-based and nonparametric-based techniques. This paper focuses on the development of an intelligent degradation prognosis model that the error reverse transmission is employed to optimize the initial parameters of WNN, and the initial modelling parameters of ARMA are also optimized by the RLS algorithm step by step. The proposed method is more robust to different operating conditions and outperforms other one-step prediction methods by taking the LFC and HFC properties of health indicators into account. Therefore, the proposed method has huge potential in the field of PHM of mechanical equipment.

For future research, it would be interesting to investigate more advanced a sparse low-rank matrix decomposition (SLMD) algorithm to robustly separate the health indicator time series (HITS). Both LFC and HFC could be predicted using more sophisticated adaptive algorithms under more complex and harsh environments.