^{1}

^{2}

^{*}

^{1}

^{3}

^{1}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

In a health management system, prognostics, which is an engineering discipline that predicts a system's future health, is an important aspect yet there is currently limited research in this field. In this paper, a hybrid approach for prognostics is proposed. The approach combines the least squares support vector regression (LSSVR) with the hidden Markov model (HMM). Features extracted from sensor signals are used to train HMMs, which represent different health levels. A LSSVR algorithm is used to predict the feature trends. The LSSVR training and prediction algorithms are modified by adding new data and deleting old data and the probabilities of the predicted features for each HMM are calculated based on forward or backward algorithms. Based on these probabilities, one can determine a system's future health state and estimate the remaining useful life (RUL). To evaluate the proposed approach, a test was carried out using bearing vibration signals. Simulation results show that the LSSVR/HMM approach can forecast faults long before they occur and can predict the RUL. Therefore, the LSSVR/HMM approach is very promising in the field of prognostics.

Condition-based maintenance (CBM), which can increase system maintenance efficiency and reduce life cycle cost, is gaining popularity. Developing the capability of prognostics and remaining useful life (RUL) prediction can save great costs and improve the logistics support. Therefore, prognostics and RUL prediction can improve the functions of CBM [

At present, various health management systems are being gradually proposed and applied, such as health and usage monitoring system (HUMS) [

Prognostic approaches can be placed into three categories: physical model-based prognostics, evolutionary or trending model-based prognostics, and experience-based prognostics [

In this paper, a hybrid fault prognosis method based on LSSVR and HMM is proposed. In this method, LSSVR is used to predict the fault features, while HMMs are used to describe health states. Fault processes can be divided into several stages and HMMs are built and trained for each stage. Suppose given an observation sequence, the probabilities of the observation for HMMs are used to determine which health state of the given observations the sequence belongs to. Combining these HMMs with the future features predicted by LSSVR can provide an estimate of the remaining useful life (RUL) of a system.

The remainder of the paper is organized as follows: Section 2 outlines the framework of the LSSVR/HMM prognostics approach. Section 3 presents the training and estimation algorithms of HMM. Section 4 introduces the prediction algorithm based on LSSVR. Section 5 provides an application and simulation results. Section 6 presents our conclusions.

This aim of this section is to introduce how to use the HMM-based method to describe a fault process. The fault process of a machine has a certain time-span. To analyze the fault process, it can be divided into several health states. However, fault states are unobservable but hidden in some observable signals, such as vibration signals. The HMM-based method provides a way to detect the unobservable fault states, as HMM is a stochastic technique for classifying signals and modeling [_{1}, h_{2},…, h_{N}_{N}

As shown in _{ij}_{i}_{j}_{jj}_{N × N} is the probability transition matrix which describes the transition relations between the states, _{jk}_{N × M} describes the observation probability distribution and _{jk}

As the observation sequence from a health state can train an HMM, N HMMs can be trained, which are written as _{1}, _{2},…, _{N}_{i}_{1}, _{2},…, _{T}_{i}_{j}_{i}

As mentioned above, using HMMs can describe machine fault processes. According to the likelihood probabilities of present observations, HMMs can detect and diagnose faults. However, the HMM method cannot predict future health states and the RUL, because the future observation sequence cannot be obtained. To solve this problem, a hybrid prognostics method based on LSSVR and HMM is proposed in this paper, where LSSVR provides a way to predict the future fault features.

The framework of the hybrid fault prognosis scheme is described in

As shown in

Feature extraction

As most observation signals have much random noise and uncertain interferences, features should be extracted from the signals before fault diagnosis and prognosis. In this paper, the linear predictive (LP) method, as mentioned in article [_{n}_{1}, _{2}, …, _{ρ}

The residual errors are written as:

The value of the linear prediction coefficients _{1}, _{2}, …, _{ρ}_{n}_{n}

Vibration signals are always non-stationary. If signals are segmented into several short-time signals, they can be considered stationary. Therefore, we can obtain short-time signals by segmenting vibration signals into small windows; and each window as a frame. To segment the vibration signals into small windows, the length of a small window should be determined. Each window has equal length. Based on the given length, the vibration signals can be divided into several small windows. And then the linear prediction coefficients can be extracted from these windows. The feature extraction is described in

Each window is coded into a feature vector which is written as:
_{t}_{t}_{1}, _{t}_{2}, …, _{tρ}_{1}, _{2}, …, _{T}

HMMs training

HMMs training can now be carried out based on the fault features history. In this paper, the historic fault features are experimental data. The training data used for a specific state come from the corresponding experimental conditions. The essence of this step is to estimate the HMM parameters for each state. Based on observation sequences from _{1}, _{2}, …, _{N}

Prediction of features based on LSSVR

Suppose the present time is _{t+k} which can be predicted based on the data before time _{t+k}, one can predict the system health state at the future time

Log-likelihood calculation

Calculating likelihood probabilities _{t+k} |_{i}_{1}, _{2}, …, _{N}

The corresponding state h_{i}

RUL prediction

As shown in _{t+k}|_{N}

HMM training is to obtain the optimal model and the training process needs to estimate the model parameters. An HMM is described in _{j}_{t}_{t}_{j}_{t}_{jl}_{j}_{jl}_{jl}

Then an HMM is re-written as:

Given an observation sequence _{1}, _{2},… , _{T}

In a forward algorithm, define the forward variables as _{i}_{1}, _{2}, …, _{t}_{t}_{i}_{1}, _{2}, …, _{t}_{i}_{t}

The recursive estimation of the forward is as follows [

Initialization:

Forward recursion (For

End:

In a backward algorithm, we define the backward variables as _{i}_{t}_{+1}, _{t}_{+2},…, _{T}_{t}_{i}_{t}_{+1}, _{t}_{+2},…, _{T}_{i}

The recursive estimation of the backward is as follows [

Initialization (1 ≤

Backward recursion (For

End:

The goal of HMM training is to find the model

Given an observation sequence _{t}_{t}_{i}_{i}_{t}

Define _{t}_{i}_{t}

_{t}_{t}_{i}_{t+1} = h_{j}_{i}_{j}_{t}

According to _{i}_{t}

The idea of SVR is to map non-linear problems to linear problems in a high dimension space. LSSVR is an improved algorithm based on SVR, and it uses a least square loss function instead of the

Considering the sample data as {(_{1},_{1}), …, (_{i}_{i}_{D}_{D}_{i}_{i}_{i}

The Lagrangian is written as:
_{i}

According to Karush-Kuhn-Tucker (KKT) optimization conditions, the optimality conditions are given as [

By solving _{i}_{i}

Frequently used kernel functions are the polynomial, sigmoid and radial basis kernel (RBF) functions. This paper applies RBF because it can classify multi-dimensional data. The RBF kernel function is given as:

In fault prognosis, time series are the extracted feature data. Suppose the front _{i}_{i}_{r+k-υ,} …., _{r}_{+1},…, _{r}_{+k-1}}.

As mentioned above, LSSVR can predict future fault features. However, it is difficult to describe and assess health states. Combining LSSVR with HMM, it can realize health state prediction and RUL prediction. The feature vectors predicted based on the LSSVR are used as observations in each HMM to calculate the log-likelihood probabilities as shown in

As analyzed above, two parameters

The particle positions and velocities are written as _{i}_{i1}, x_{i2}_{iD}^{T} and _{i}_{i1}, v_{i2}_{iD}^{T} (_{i}_{g}. Then the velocity and position updating can be written as:
_{id}(_{id}(_{1} and _{2} are study factor, _{id}_{gd}_{1} and _{2} are random during [0,1]. The optimization process is described as

In _{best} is the partial best fitness and _{best} is the global best fitness.

The end condition is meeting the accuracy requirements or arriving at the maximum iterations. When meeting the end condition, the best position is considered as the optimal parameters used in LSSVR.

As time goes by, new observations are added which may include fault information and can improve prediction accuracy. However, the training data will become large with time growth. If the number of sample data is too large, it will affect training efficiency. To solve this problem, some unimportant old data, such as non-support vector samples or far history samples, are selected for deletion in this paper. These data will not affect the training accuracy seriously. By adding new data and deleting some old data, the training samples sequence will be renewed online with time growth. The new training sample can be written as:
_{new}(_{del}(

The RUL prediction process based on the improved LSSVR algorithm is summarized in

As shown in

To evaluate the performance of the LSSVR/HMM prognostic approach, it was tested using experimental bearing vibration data from the Case Western Reserve University Bearing Data Center [

In our simulation, the inner race vibrations signals are used. Its fault diameter includes the following levels: 0.007 inches, 0.014 inches, and 0.021 inches. The data was sampled at 12 kHz. Added to the normal states, fault process can be divided into four health states. The bearing data center provides vibration data from different health states. Data from different conditions are discrete, which can be used to train HMMs for the corresponding health status.

Vibration data are segmented into several windows and extracted feature vectors as mentioned in Section 2.2 and _{t}_{t}_{1}, _{t}_{2}, …, _{t}_{8}]^{T} and the number of the feature dimensions is 8. Take the first dimension as an example; the features of the four health states are shown in

Suppose three Gaussian distributions are used to describe observation probabilities for each state. Give the initial HMM parameters as follows:

Generally speaking, the influence of the initial values of _{jl}_{jl}_{jl}

To improve the training accuracy, several observation sequences are used to train HMM for a health state. The log-likelihood probabilities of HMMs training are shown in

The simulation results shown in

As shown in

Collecting lifetime data is a difficult work and the bearing data center does not provide this lifetime data. Therefore, we construct a lif time data sequence by taking some vibration data separately from each health states. Then the constructed lifetime data sequence is used to test for features prediction and fault prognostics. After frame segmentation and feature extraction, the length of the feature sequence is 71. We take the front 50 feature vectors as LSSVR training data, and the back 21 as unknown test data. With these parameters, the present time is

In

Based on the trained four HMMs, the log-likelihood probabilities of the life time data for each state are shown in

The state with the highest log-likelihood is the health state of the corresponding time. As shown in

In

Fault prognosis is an essential and difficult technology in health management and CBM. In this paper, a hybrid prognostics approach which integrates the HMM method and LSSVR is presented. In the proposed LSSVR/HMM approach, the fault process is divided into several health states modeled by HMMs. These HMMs are trained based on history sample data. The lot-likelihood probabilities of the HMMs are used to classify data and determine the health state. The state with the highest log-likelihood is the corresponding health state of given data. LSSVR is used to predict the feature trend. Based on the predicted features and trained HMMs, the future health state can be predicted. LSSVR trains and predicts RULs online until the highest probability of the predicted feature vector is at the last health state. Then the RUL is equal to the prediction time.

Testing of the proposed LSSVR/HMM prognostics approach was carried out using bearing vibration data. The simulation results showed that the novel approach is efficient in fault diagnosis and prognostics and can also predict the RUL. Furthermore, the results showed that the RUL prediction accuracy increases with the passing of time. Future work will focus on how to improve the prediction accuracy and expand the applications.

The authors declare no conflict of interest.

N-state left-to-right hidden Markov model.

Framework of the LSSVR/HMM fault prognostics scheme which contains two processes. One is HMMs training (off line) and another is fault feature and RUL prediction process (on line).

Feature extraction by segmenting the vibration signals into small windows.

The parameters optimization process of the LSSVR based the PSO algorithm.

RUL prediction process based on the improved LSSVR with data renewed online.

The first dimension of the feature vectors for each health states.

Log-likelihoods of HMMs during training for the four states.

Log-likelihoods for the four state HMMs of the fault-free test data.

Log-likelihoods for the four state HMMs of the test data.

The prediction result of the first dimension of the feature vectors based on the LSSVR.

Log-likelihoods for the four state HMMs of the test life time data.

Remaining useful life prediction results.

50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | |

16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | |

9 | 10 | 10 | 11 | 9 | 10 | 9 | 8 | 7 | 6 | 5 | |

7 | 5 | 4 | 2 | 3 | 1 | 1 | 1 | 1 | 1 | 1 |