Automatic and Generic Prognosis Method Based on Data Trend Analysis and Neural Network

Abstract

This paper presents a generic and unsupervised failure prognosis method which can be applied to wide scope of applications. The main contribution of the presented method is automatic relevant data identification based on signal smoothing and trendability analysis and automatic degradation model identification for health indices construction, built using a trained neural network, thus allowing for the automatic adaptation of the degradation trend model to changes in the degradation dynamic. Regarding the failure prognosis, the end of life is first predicted using a fitting model; then, the remaining useful life is predicted using a similarity algorithm. The proposed approach is validated using the turbofan engine data sets provided by NASA. The prediction results have been evaluated using accuracy metrics such as root mean square error and prognostic metrics such $\alpha -\lambda$ and relative accuracy. The obtained results show the effectiveness of the proposed method, both for the end of life and remaining useful life predictions.

1. Introduction

Predicting system failure is nowadays a challenge for industries and scientists, since this information is essential for maintenance management. Indeed, the implementation of conditional and predictive maintenance strategies requires knowledge of the temporal evolution of the health-state of the system, the main objective being a reduction in the system’s downtime and an increase in its availability. The standard ISO 13381-1:2015 provides guidance and a general guideline for the development and application of prognosis processes. It specifies failure prognostics and presents the road maps of the prognostics steps and details the monitoring system setup and how to estimate the confidence interval related to the predicted RUL. It also proposes some useful tools for modeling degradation. The presented approach is developed with respect to ISO standards and proposes a generic and automatic prognosis algorithm that can be applied on a wide scale in industrial applications considering different steps to perform failure prognosis: data filtering, trend analysis and health indices (HIs) construction, the end of life (EOL) and the remaining useful lifetime (RUL) prediction.

The main contributions of the proposed method are:

• Automatic relevant data identification based on signal smoothing and trendability analysis.
• Automatic degradation model identification for HIs construction, built using an offline trained neural network, thus allowing for the automatic adaptation of the degradation trend model to changes in the degradation dynamic.

The similarity method is used in this paper for RUL prediction, since it is an efficient geometric method which does not require prior knowledge of the complete profile of the degradation process. This gives this method an advantage on the application level, knowing that the data available on the degradation profile are often incomplete in real applications.

The remainder of this paper is organized as follows. In Section 2, related works are presented. The proposed algorithm is described in Section 3. Section 4 presents experiment methods used to validate the approach. Section 5 is devoted to the results. Finally, in Section 6, the conclusions and some remarks are presented.

2. Related Works

In the literature, three main approaches are distinguished for failure prognosis [1,2]: physics-based approaches, data-driven approaches and hybrid approaches.

Physics-based approaches are suitable when physical knowledge about system dynamics is deeply known and enough data are available on the normal operation of the system for parameter identification and model validation [3,4,5]. They give accurate results when the physical knowledge of the system is adequately expressed to take the right modeling assumptions about the degradation process; therefore, a detailed knowledge of these complex processes is required [6,7]. Some models have been developed, such as the Arrhenius model for semiconductor electronic components [8] and the Paris model [9] for crack propagation in mechanical systems. In practice, it is the lack of physical knowledge about the degradation process that prevents large-scale development of these methods. To overcome this issue, recent research works proposed methods that do not require prior knowledge of the degradation process [10,11].

Data-driven approaches are based on data analysis using artificial intelligence (AI) tools, such as neural networks [12,13], support vector classification [14,15] and support vector regression [16,17]; continuous statistical tools, such as the Wiener process [18] and the Gamma process [19,20]; discrete statistical tools, such as the hidden Markov model and the hidden semi-Markov model [21], and probabilistic tools, such as Bayesian networks [22] and Gaussian mixture models (GMM) [23]. Data-driven approaches have evolved rapidly in recent years by the development of fast computing systems and the evolution of data storage, which have expanded their application fields. However, data presenting the degradation profile are required. The latter is rarely available, and when available, it does not represent all the possible operating conditions of the considered system.

Hybrid approaches combine two methods or more, taking advantage of their benefits [24]. For example, by combining physics knowledge about system dynamics and historical data to build a degradation model, a failure prognosis task is performed [10]. Hybrid approaches can be also obtained by combining expert knowledge either with signal processing [25] or artificial intelligence tools [26,27,28].

3. Overview of the Proposed Approach

As illustrated in Figure 1, the proposed approach is composed of two stages: an offline stage and an online one. The first four steps of the offline stage deal with relevant data identification (that will be used in the training model) as well as in the characterisation of the degradation model (used for HI generation). The selected variables are used as training data to build an HI for degradation estimation. The online part uses the data collected in real time to construct the HI for predicting the EOL and the RUL of the system.

3.1. Offline Stage

The offline stage consists of four steps and is devoted to building a model which will be used online for degradation estimation and RUL prediction. In this stage, relevant data that bring degradation information are selected automatically using a smoothing filter and trend analysis. In step 1, signal smoothing is used to extract useful information and to reduce the noise ratio. In step 2, trend analysis is used to identify only data (or measurement) that represent tendency. These data are relevant for failure prognosis. The data relating the transient dynamic are deleted in an automatic way. In step 3, a degradation trend profile is constructed using the fit technique and injected into a neural network (step 4) for model training.

3.1.1. Signal Smoothing

In most applications, data are contaminated with noise, particularly in a harsh environment (high pressure, temperature, vibration, etc.), so smoothing data is an essential step in any failure diagnosis and prognosis schema. Porotsky [29] used a fitting technique to reduce noise based on non-linear regression methods, considering two types of smoothed functions: polynomial functions and exponential functions. Since the smoothing function did not guarantee permanent stability, Porotsky proposed to select the right function for each considered sensor. Bektas et al. [30] used ninth- and fourth-degree polynomial regression to filter data, and each polynomial degree was used either for training or testing. Coble et al. [31] proposed for reducing noise a quadratic fitting technique. In data analysis, these methods can sometimes lead to an undershoot and a loss of critical information (peaks). Additionally, the performance depends on the appropriate selection of polynomial order, which is difficult.

In this work, a modified Savitzky–Golay (SG) filter [32,33,34] is used. The original algorithm fits a low-degree polynomial in a least-squares sense on the samples within a sliding window m. The smoothing result depends on the appropriate choice of polynomial order and the sliding window length. An arbitrary selection of these parameters is difficult for any user. The SG filter is distinguished from other filters since it gives a good compromise between the possible loss of information caused by filtering and the improvement of the quality of the signals by eliminating noise.

The contribution made in this work tends to automatically choose the sliding window value (m) that keeps only the appearance of the signal by removing noise and outlier components. Choosing the smoothing parameters is a common problem in smoothing methods. One of the most effective optimal choices for smoothing windows is in kernel smoothing. In [35], the authors discussed the most common optimal choice of the parameter h. In the case of a normal data vector y, the kernel estimator is written as in Equation (1):

$$\tilde{f}\left(y\right)=\frac{1}{n}\sum _i^{n}w(y-y_i;h)$$

(1)

where $\tilde{f}$ is the kernel estimator, n is the amount of data and w is the kernel function whose variance is controlled by the parameter h, representing the smoothing function or the bandwidth.

From the general optimal h, which minimizes the mean integrated squared error (MISE) in order to smooth $\tilde{f}$, the optimum formula to calculate h is given by Equation (2):

$$h={\left(\frac{4}{3n}\right)}^{\frac{1}{5}}\sigma$$

(2)

The choice of m is driven by a close similarity between kernel smoothing regression and the SG filter. First, both methods are used for smoothing. Second, in kernel smoothing regression, a kernel function and a bandwidth parameter h are chosen by users, which is similar to the SG filter, where the polynomial degree and the sliding window m must be chosen. Third, for kernel smoothing regression or the SG filter, increasing h or m makes the data smoother, while decreasing them makes the estimation wiggly. This broad analogy between kernel smoothing regression and the SG filter, in the sense that the parameters h and m in both methods have the same effects, encourages us to use h as m. In this work, the polynomial order is fixed to two (2) to avoid any intense wiggling in the smoothed signal.

The assumption of normality can cause problems of overshooting when dealing with non-normal data. To reduce overshooting, $\sigma$ must be adjusted by Equation (3).

$$\tilde{\sigma }=\frac{median\left\{abs\left(y_i-\tilde{\mu }\right)\right\}}{0.6745}$$

(3)

where $\tilde{\mu }$ notes the median of the sample.

3.1.2. Trend Analyses

Trendability and monotonicity are important parameters for failure prognosis. A measurement that monotonically decreases or increases along the system life can represent fault or be related to it. In this paper, the relevant measurement (or data) variables are identified based on trendability analysis. The most popular statistical tests for the detection of the monotonic trend in time series are: the t-test for regression slope, the Mann–Kendall (MK) test, the test based on the Theil–Sen slope, the Brillinger test, the Abelson–Tukey test, the Spearman tests and the Cox–Stuart (CS) test. All mentioned tests, except for the t-statistic, do not assume the shape of the trend: linear, exponential, logarithmic or more complex. The distribution type of the data is also not specified.

In this paper, the CS and MK tests [36,37] are used. In most applications, the MK test has the dominant position if the trend does not exist. In cases when the trend is very weak and the time series is short, the CS test has a better chance of detecting the trend than the MK test. As these conditions cannot always be fulfilled, it is better to use another test as a complement [36].

The CS and MK tests are statistical methods used to check an upward or a downward monotonic trend in data over time, which means that the data magnitude is increasing or decreasing or not through time. These tests are non-parametric or distribution-free tests.

The MK and CS tests determine whether to reject the null hypothesis H and accept the alternative hypothesis HA, where:

• H: no monotonic trend;
• HA: a monotonic trend is present.

Consider a time series $Y={[y_1,y_2,\dots ,y_N]}^T$. For the CS test, the $CS$ is computed by Equations (4) and (5).

$$CS=\sum _{i=1}^{N/2}C{S}_i$$

(4)

$$C{S}_i=\left\{\begin{array}{c}1\mathrm{if}{y}_i-y_{\left(\frac{N}{2+i}\right)}>0\hfill \\ 0\mathrm{if}else\hfill \end{array}\right.$$

(5)

3.1.3. Degradation Fit Model

There exist in the literature different models describing component degradation over time. These models are mainly based on a prior understanding of the physics process of component degradation or on data that can be modeled by a stochastic process, such as the Wiener or Gamma processes. In this paper, analytical models of the degradation process of incipient faults have been considered. The time/cycle evolution of the these models are illustrated by Figure 2. These curves represent the trajectory of the HI, considering the life evolution of the system from a healthy state to a failure one, and normalize from 1 (health state) to 0 (failure state). They are obtained by simulating the corresponding models from the health-operating mode to failure during the life cycle of the system.

The fit model (step 3) is selected for the training data within multiple fitting functions once the relevant data are identified (from step 1 and 2). The most effective fit model is declared as the degradation function based on root mean square error (RMSE) and R-squared ($R^2$) statistical measures.

3.1.4. Model Training

There are several neural network techniques to build a data-driven model, such as: recurrent neural networks [38,39], convolutional neural networks [40] and deep bidirectional (LSTM) neural networks [41]. In this work, a neural network is used as a data-driven model and trained to estimate the HI trajectory. The training inputs are the relevant data identified in step 2 with the corresponding time vectors or life cycles and as targets the HI constructed in step 3. In this step, the training inputs are not smoothed. Signal filtering is used to smooth measurement in order to identify the degradation model, since with noisy measurement it is hard to distinguish with high accuracy the degradation as well as to construct the HI. However, in the training step, it is necessary to consider the raw signal to take into account the different operation of all engines.

3.2. Online Stage

3.2.1. Health Indices Construction

The trained data-driven model in step 4 of the offline stage is used to estimate HI in the online stage. The trajectory of HI is generated every time sample/cycle by using the same relevant data used for the model training. The obtained HI is used for failure prognosis in step 6.

3.2.2. Failure Prognosis

The RUL prediction in the present work is conducted in two ways depending on the HI value, either by fitting the HI to reach 0 using the degradation model identified in the offline stage or by finding the most similar HI in the training model step [42]. The threshold depends on the degradation model, for example, in the case of exponential trend, the probability of false fit is at a minimum when the HI reaches 0.6. The RUL is calculated based on the estimated EOL as given by Equation (6). If the threshold is reached, the EOL is estimated by fit and the RUL by Equation (7).

$$RUL=\mathrm{E}\mathrm{O}{\mathrm{L}}_{est}-t_p$$

(6)

$\mathrm{E}\mathrm{O}{\mathrm{L}}_{est}$ is estimated by fit and $t_p$ is the present time or cycle of the system.

If the EOL is estimated by a similarity approach, the RUL is calculated [42] by Equation (7).

$$RUL=\sum _{i=1}^k{w}_{i}RU{L}_i$$

(7)

where $w_i$ is the weight assigned to the estimated RUL and depends on the similarity between the test and trained HI instances and k is the number of selected neighbors.

4. Application to Turbofan Engine: A Case Study

The proposed algorithm is applied on the Commercial Modular Aero-Propulsion System Simulation data, or C-MAPSS data, from the Prognostics Data Repository [43]. The available data set is mostly time series data from some nominal state to a failed one. Four different sets are simulated under different combinations of operational conditions and fault modes. The simulation is carried with MATLAB software 2019 with an i3 Intel processor and 4 Gigabytes of RAM. The MATLAB environment allows for fast development of application, especially for data engineering and machine learning simulation.

The system settings and the available sensors are described in Appendix A—

Table A1. C-MAPSS data set parameters for PHM08 challenge. LPC/HPC = low/high-pressure compressor, LPT/HPT = low/high-pressure turbine.

	Symbol	Description	Unit
Unit	/	/	/
Time	/	/	t
Setting 1	/	Altitude	ft
Setting 2	/	Mach number	M
Setting 3	/	Sea-level temperature	°F
Sensor 1	T2	Total temperature at fan inlet	°R
Sensor 2	T24	Total temperature at LPC outlet	°R
Sensor 3	T30	Total temperature at HPC outlet	°R
Sensor 4	T50	Total temperature at LPT outlet	°R
Sensor 5	P2	Pressure at fan inlet	Psia
Sensor 6	P15	Total pressure in bypass duct	Psia
Sensor 7	P30	Total pressure at HPC outlet	Psia
Sensor 8	Nf	Physical fan speed	rpm
Sensor 9	Nc	Phsical core speed	rpm
Sensor 10	epr	Engine pressure ratio	/
Sensor 11	Ps30	Static pressure at HPC outlet	Psia
Sensor 12	phi	Ratio of fuel flow to PS30	pps
Sensor 13	NRf	Corrected fan speed	rpm
Sensor 14	NRc	Corrected core speed	rpm
Sensor 15	BPR	Bypass ratio	/
Sensor 16	farB	Burner fuel-air ratio	/
Sensor 17	htBleed	Bleed enthalpy	/
Sensor 18	Nf-dmd	Demanded fan speed	rpm
Sensor 19	PCNfR-dmd	Demanded corrected fan speed	rpm
Sensor 20	W31	HPT coolant bleed	lbm/s
Sensor 21	W32	LPT coolant bleed	lbm/s

. In total, there are twenty-one (21) sensors placed at different parts of the engine. Appendix A—

Table A2. C-MAPSS data characteristics.

Characteristics	Data Set Number
	1	2	3	4
Number of faults	1	1	2	2
Operation conditions	1	6	1	6
N° of training data engines	100	260	100	249
N° of test data engines	100	259	100	248

shows all the information about the four data sets. The easiest data set is the first one with one fault mode (HPC degradation) and one operation condition. The 4th data set represents the most complex, with two fault modes and six operation conditions [43].

The first step is to identify the relevant data from the data sets. For this, the data are smoothed using the SG filter with automatic window calculation and then the trend analysis is triggered to identify the relevant data to be considered as the training input variables.

Figure 3 presents an example of the application of the SG filter on random measurements of engine 1 from the four (04) data sets. Smoothing using an SG filter is very good even in high noise contamination. Thus, the trendability and the monotonicity can be easily analyzed from the resulting signal. Regarding the trend analysis step, all smoothed measurements of all engines are analysed using CS and MK algorithms (step 2).

Table 1. The results of trend analyses and the trendability rate of each measurement.

Sensors	Symbol	Data Set Number
		1	2	3	4
Sensor 1	T2	100	6	95	10
Sensor 2	T24	96	12	100	18
Sensor 3	T30	100	13	94	23
Sensor 4	T50	0	4	0	4
Sensor 5	P2	0	3	15	5
Sensor 6	P15	100	22	100	22
Sensor 7	P30	89	16	97	17
Sensor 8	Nf	81	35	94	39
Sensor 9	Nc	0	6	4	10
Sensor 10	epr	100	4	96	13
Sensor 11	Ps30	100	19	100	22
Sensor 12	phi	91	6	94	8
Sensor 13	NRf	87	15	93	28
Sensor 14	NRc	99	7	100	11
Sensor 15	BPR	0	10	0	9
Sensor 16	farB	99	4	99	11
Sensor 17	htBleed	0	16	0	17
Sensor 18	Nf-dmd	0	0	0	0
Sensor 19	PCNfR-dmd	100	5	98	6
Sensor 20	W31	99	4	100	5
Sensor 21	W32	0	0	0	0

shows the results of this step. All variables presenting trendability are selected as relevant data; however, only those with the highest rates of trendability are used to select a degradation trend model. The corresponding sensors are selected automatically to be input variables for model training. The selected variables are given in bold in Table 1. Regarding data set 1, 14 measurements are considered relevant data (T30, T24, T2, farB, NRc, NRf, Phi, ps30, epr, Nf, P30, P15, W31 and PCNfR-dmd). Then, for degradation model fitting, only 6 relevant data values (T2, T30, P15, Ps30, epr and PCNfR-dmd) with the highest trend percentages are considered. Concerning data set 2, fewer inputs are selected. In this case, 8 sensors’ data values (T24, T30, P15, P30, Nf, Ps30, NRf and htBleed) are selected as training input and one sensor (Nf) is used to determine damage function.

The third step concerns degradation trend identification based on the highest rate of trendability signal and by using metrics such as the RMSE and the adjusted $R^2$. Figure 4 shows the fitting results in the sense of RMSE of the different data sets. From this figure, one can notice that the degradation follows an exponential trend, as it was considered in damage propagation model built for C-MAPSS data [43].

In the last step of the offline stage, the data-driven model for HI construction is trained using relevant variables (14 variables in the case of data set 1) and targeted to HIs constructed from the degradation fit model. The considered neural network is a two-layer feed-forward network with 9 sigmoid hidden neurons and one linear output neuron. The neural network is trained with the Levenberg–Marquardt backpropagation algorithm. Figure 5 shows an example of HI estimation of a randomly selected engine from training data set 1 in the online stage. From 155 cycles, 108 cycles are used for HI estimation. Using the estimated HI, a fit is performed to reach the estimated EOL of the engine, which is in this case equal to 163, i.e., 8 cycles over the real EOL of 155 cycles. The RUL is calculated as the difference between the 108th cycle and the estimated EOL cycle.

5. Results and Discussion

The results are related to the online parts of the algorithm. First, the online stage of the algorithm is tested on training data to validate it in all engine life. After that, the algorithm is applied on test data. Figure 6 presents the results of the RUL prediction of the first 20 engines of data set 1 compared to their real RUL values. The results show a strong capability of RUL estimation, especially at the EOL cycles.

The aim of the presented paper is to show that the proposed algorithm performs RUL prediction in an unsupervised manner by making good prediction results. For this, the obtained results are evaluated by:

• Accuracy metrics such as RMSE;
• Prognostics metrics such as $\alpha -\lambda$ and relative accuracy.

The RMSE indicates how data fit the target prediction model, and it is calculated as follows:

$$RMSE=\sqrt{\frac{1}{N}\sum _{n=1}^N(RU{L}_{est}-RU{L}_{real}{)}_n^2}$$

(8)

Table 2. Performance of algorithms on the full training/testing turbofan first data sets.

Ref.	RMSE
RULCLIPPER [44]	13.26
Multi-layer perceptron [45]	14.39
Proposed app.	21.68
ESN trained by Kalman Filter [46]	63

presents a comparison of the proposed algorithm results obtained by an application on the test data set 1 which contains 100 test samples. The obtained RMSE of $21.68$ is acceptable compared to that of the other approaches since the target RUL model is not fixed and depends on the fitted degradation trajectory. The other approaches suppose that the RUL follows a linear piecewise function with a maximum RUL limit fixed a priori.

To better assess the performance of RUL prediction, $\alpha -\lambda$ and relative accuracy are used.

$\alpha -\lambda$ is used to evaluate the performance of the prediction accuracy within $\alpha \ast 100$ of the real RUL at specific time instance $t_{\lambda }$, and it is calculated by:

$$\left(1-\alpha \right)ru{l}^{*}\left(t\right)\le rul\left(t_{\lambda }\right)\le \left(1+\alpha \right)ru{l}^{*}\left(t\right)$$

(9)

where $\alpha$ is the accuracy bound and $t_{\lambda }=t_p+\lambda \left(EOL-t_p\right)$ such that $t_p$ is the time prediction.

However, the relative accuracy provides an easily interpretable measurement of the confidence that can be given to the prediction accuracy (from 0 to 1), as it is expressed according to the real RUL by:

$$RA\left(t\right)=1-\frac{\left|RU{L}^{*}\left(t\right)-RUL\left(t\right)\right|}{RU{L}^{*}\left(t\right)}$$

(10)

where $\mathrm{R}\mathrm{U}{\mathrm{L}}^{*}$ is the real RUL.

From Figure 7, the prediction performance of engine 63 is within the $\alpha =20\%$ accuracy interval with relative accuracy close to 1 throughout the prediction process. At the end of prediction, the relative accuracy tends to infinity because of its mathematical formulation. These performance results are acceptable.

The proposed approach, being unsupervised, has a wider scope of application, and it presents good results in the sense that:

• There is no expert supervision in the choice of training variables: by using the proposed approach, the most trended signal in data set 1 is sensor 3: T30. This sensor is related to the HPC and to the degradation in data set 1 [43]. The trend analyses can find without any prior knowledge the sensors which are related to the degradation and which can be used as HIs.
• The degradation model is determined using RMSE and R-squared metrics, among multiple predefined fitting functions (sum of sin, power, exponential, Gaussian, etc.). Considering C-MAPSS data, the degradation model was identified to follow an exponential trend as it was considered in the damage propagation model [43].
• The failure prognosis results depend on whether the run-to-failure data are complete or not, i.e, in the case of run-to-failure data, the estimated HI is near the EOL, so the degradation state is clear and a fit of HI will estimate a good EOL. However, in the case of non-complete data, the degradation trajectory and the EOL are hard to drive.
• The presented approach is a generic one, and it can be applied on any time series data with neither knowledge of system dynamics nor expert intervention.

6. Conclusions

In this paper, a generic method of fault diagnosis and failure prognosis is presented. The main contribution of the proposed approach compared to other diagnostic and prognostic methods is its unsupervised aspect, through automatic degradation model identification and the automatic adaptation of the trend model to changes in the degradation dynamic, which makes the proposed method more generic with a wider scope of applications. To show its effectiveness, the proposed method has been applied to well-known C-MAPSS data proposed by NASA. With an RMSE of 21.68 and a prediction performance of engine 63 within the $\alpha =20\%$ accuracy interval with relative accuracy close to 1 throughout the prediction process, the proposed method has demonstrated its effectiveness.