RUL Prediction of Rolling Bearings Based on a DCAE and CNN

Abstract

Predicting the remaining useful life (RUL) of mechanical equipment can improve production efficiency while effectively reducing the life cycle cost and failure rate. This paper proposes a method for predicting the remaining service life of equipment through a combination of a deep convolutional autoencoder (DCAE) and a convolutional neural network (CNN). For rolling bearings, a health indicator (HI) could be built by combining DCAE and self-organizing map (SOM) networks, performing more advanced characterization against the original vibration data and modeling the degradation state of the rolling bearings. The HI serves as the label of the original vibration data, and the original data with such label is input into the prediction model of the RUL based on a one-dimensional convolutional neural network (1D-CNN). The model was trained for predicting the RUL of a rolling bearing. The bearing degradation dataset was evaluated to verify the method’s effectiveness. The results demonstrate that the constructed HI can characterize the bearing degradation state effectively and that the method of predicting the RUL can accurately predict the bearing degradation trend.

1. Introduction

The deep learning (DL)-based data-driven method has become a hotspot in the field of prognostic and health management (PHM) as a result of the promotion of enormous industrial status monitoring data and computing power enhancement [1,2,3,4]. Because rolling bearings are both a basic and fundamental component of mechanical equipment, their quality has a direct impact on production efficiency. Predicting the RUL of mechanical equipment can improve production efficiency while effectively reducing the life cycle cost and failure rate. The main methods currently available for predicting the RUL of rolling bearings are roughly divided into three categories: physical knowledge-based prediction methods, the data-driven prediction methods, and the method combining both of the above. According to the physical model-based prediction method, a physical model would be constructed utilizing the advanced knowledge learned and the systematic working status after completely understanding the underlying system and failure mechanism so as to predict the RUL of the equipment. Lundberg et al. created a bearing service life prediction theory using the Weibull metal fatigue probability distribution method, which was the earliest theory containing the calculation of the service life of a rolling bearing (L-P life model). This theory coincided with the experimental results at that time and was widely recognized. Some researchers improved the prediction method from different angles and successfully applied the physical model to the RUL prediction of rolling bearings [5,6,7]. Although the physical model of a bearing’s useful life prediction features strong interpretability and superior prediction accuracy, it requires a considerable amount of artificial prior knowledge that serves as the very basis. For the purpose of obtaining a highly precise physical model, experiments and calculations must be conducted substantially, resulting in high implementation costs. In addition, in consideration of certain difficulties in establishing a universal physical failure model, its reusability is limited. As a result, data-driven remaining useful life prediction methods have gradually begun to show advantages. Data-driven prediction methods are generally utilized to extract features that are sensitive to the current degradation state and to predict future damage by taking advantage of the existing historical data (observation data) upon obtaining a model that complies with the laws of nature. To predict the RUL of bearings, Peng et al. proposed a method that combines the deep belief network with the particle filter [8]. Tian et al. predicted the RUL of bearings based on the reliable degradation indicator and temporal convolution network with quantile regression [9]. Kaji et al. combined the convolutional autoencoder and continuous wavelet transform to construct a health indicator for bearings to predict the RUL [10]. Ren et al. applied a stacked autoencoder (SAE) to the extraction of the vibration signal characteristics of a bearing and predicted the RUL based on the deep neural network [11]. Elsheikh et al. used a long- and short-term memory (LSTM) network to monitor how a machine ran and predicted the remaining service life of the machine components [12]. Y. Qian et al. applied the multi-time scale method to the prediction of the RUL of rolling bearings, established a multi-dimensional dynamic system based on phase space warping (PSW) technology and finally predicted the RUL of bearings successfully and quickly by combining the system with the improved Paris model [13]. Some researchers combined the physical model with the data-driven method based on deep learning and have achieved good results in RUL prediction for rolling bearings [14,15,16,17,18]. Through the current research results of scholars, it can be found that the main processes of the life analysis and prediction methods of mechanical equipment are data acquisition, degradation state modeling and RUL prediction. In the degradation state modeling stage, the main research content includes HI construction and health stage (HS) division. In the past, the construction of HI required more expert experience and more human intervention. At present, the magnitude of the data is getting larger, the signal state is becoming more and more complex, and the poor feature extraction ability of traditional methods and the insufficient nonlinear function mapping ability are gradually exposed. As a result, there is an urgent need to find new intelligent methods for more accurate and efficient degradation state modeling of mechanical equipment. The prediction method of the bearing RUL based on deep learning has gradually become the mainstream [19,20,21,22,23,24]. The research results of many scholars show that the feature information of bearings can be adaptively extracted from the original data by using the deep learning method, and the deep learning method has a strong capability of nonlinear function relation mapping [25,26,27,28,29]. Scholars have achieved good results in processing massive data by using the features of local weight sharing and the sparse connection of a CNN, and the improved algorithm based on a recurrent neural network (RNN) can process time series data well. At present, most of these prediction methods belong to supervised learning methods. One of the necessary requirements for training a good supervised learning method model is that data labeling information is sufficient and accurate; that is, data can be accurately labeled. At present, most research methods add labels to data by linear or segmented marking, which is inaccurate in the description of data information. Therefore, it needs to be improved in the data processing stage.

As the high-precision mechanical equipment develops and the monitoring equipment rapidly upgrades, higher requirements have been placed on the prediction of the RUL of rolling bearings in the context of massive monitoring data, such as how to extract useful information from massive data and how to accurately model the degradation state of rolling bearings [30,31]. The purpose of this paper is to propose an intelligent feature self-extracting bearing degradation state modeling method independent of expert experience, propose a more accurate life prediction model and establish degradation state modeling (i.e., health factors) and a related RUL method. Therefore, a rolling bearing RUL prediction method based on a DCAE and a CNN is proposed. The overall life prediction process is divided into two parts as shown in Figure 1. The first part is the modeling of rolling bearing degradation based on one-dimensional vibration monitoring data. In the stage of modeling the bearing degradation state, the improved one-dimensional convolution autoencoder is mainly selected to extract the characteristics of the vibration data. The CAE has good denoising ability. The training of the CAE adopts an unsupervised method and does not rely on expert experience or complex signal-processing technology. The CNN has the characteristics of weight sharing and sparse connections. This feature is very suitable for processing large data volumes and full life cycle data. A one-dimensional convolution kernel is selected in the convolution operation, and the one-dimensional convolution kernel is more suitable for processing vibration data based on a time series. The original data are characterized by a one-dimensional convolutional autoencoder, and then the HI value of the bearing is obtained by the SOM feature reduction technique. In the RUL prediction stage, a 1D-CNN model is proposed to predict the RUL of the rolling bearing. On the basis of obtaining the HI in the degradation state modeling stage, the original vibration signal is processed in segments, and the constructed HI is used as the label of the segmented data and input into the 1D-CNN model for RUL prediction.

This paper is organized as follows. Section 2 briefly introduces the basic theory of the rolling bearing degradation state and the required methods. Section 3 describes the DCAE-based bearing degradation state modeling scheme (i.e., HI acquisition). The content of Section 4 is the proposal of the RUL prediction method based on 1D-CNN, the experimental results and algorithm evaluation. The conclusions of the full paper are given in Section 5.

2. Theoretical Background

2.1. The Degraded State of a Rolling Bearing

A rolling bearing has a complex vibration system. When the bearing rotates, it generates complex vibration signals under the interaction of its parts (e.g., the inner ring, outer ring, rolling element and cage). Common forms of bearing failure include wear, plastic deformation, gluing, rust, peeling, fractures and cage damage. The object of life prediction of rolling bearings is generally its gradual failure. The formation process of gradual failure generally represents the process of mechanical failure due to changes in the output parameters caused by damage or wear. The gradual failure of rolling bearings is a loss-type failure, and the degree of loss is related to time. The occurrence and development of bearing failure is an evolving process, and the evolution law of its degraded state is shown in Figure 2. Most of the current research mainly uses vibration data to determine the health of the bearings, and vibration analysis is currently one of the most effective methods. This paper takes the vibration signal as the time series data for the research object to model the degradation state and predict the RUL of the rolling bearing.

2.2. Autoencoder (AE)

An autoencoder (AE) is a type of neural network which consists of an input layer, a hidden layer and an output layer. Neurons in different layers are connected in a fully connected manner. Such a structure is essentially a special multilayer perceptron, which can be regarded as a special case of a feedforward NN and is trained using gradient descent. The quantity of neurons in the input layer and output layer of the autoencoder are totally the same, but the quantity of the neurons in the hidden layer is generally lower than that in the input layer. The structure above is called an under-complete autoencoder. Learning an under-complete representation will force the autoencoder to capture the salient features of the data in the hope of feature reduction [32]. The structure of the autoencoder is shown in Figure 3.

The autoencoder is usually composed of an encoding network and a decoding network, which can be represented by functions $\mathit{h}=f(\mathit{x})$ and $\mathit{r}=g(\mathit{h})$, respectively, where $\mathit{x}$ and $\mathit{r}$ are the input and output of the autoencoder, respectively. Its optimization objective can be defined as

$$\min L(\mathit{x},\mathit{r})$$

(1)

where $L$ is the cost function which penalizes the difference between $\mathit{r}$ and $\mathit{x}$, which is generally the mean square error.

2.3. Convolutional Neural Network (CNN)

A CNN is a typical feedforward neural network. A typical CNN usually includes a convolutional layer, pooling layer and fully connected layer.

2.3.1. Convolutional Layer

Convolution operation improves traditional neural networks through three key ideas: parameter sharing, equivariant representations and sparse interactions. The convolution kernel performs a convolution operation on the feature vector output by the previous layer and uses a nonlinear activation function to construct the output feature vector. The output of each layer is the convolution result of multiple input features. Its mathematical model can be described as

$$x_j^l=f\left({\displaystyle \sum _{i\in M_j}{x}_i^{l-1}}\times k_{ij}^l+b_j^l\right)$$

(2)

where $M_j$ is the input eigenvector, $l$ is the $l$th layer in the network, $k$ is the convolution kernel, $b$ is the network bias, $x_j^l$ is the output of the $l$-th layer and $x_i^{l-1}$ is the input of the $l$th layer.

2.3.2. Pooling Layer

Pooling is a form of nonlinear downsampling, which reduces the amount of calculation by reducing the network parameters and can control overfitting to a certain extent. Currently, a pooling layer is added after the convolutional layer. Maximum pooling is performed to divide the input layer into different regions with non-overlapping rectangular boxes. The maximum number of rectangular boxes is taken as the output layer. The transformation function of maximum pooling is expressed as

$$P_i^{l+1}(j)=\underset{(j-1)V+1\le n\le jV}{\max }\left\{q_i^l(n)\right\}$$

(3)

where $q_i^l(n)$ represents the value of the $n$th neuron in the $i$th eigenvector of the $l$th layer, $n\in [(j-1)V+1,jV]$, $V$ is the width of the pooling area and $P_i^{l+1}(j)$ represents the corresponding value of the neurons in the $(l+1)$th layer.

2.4. Convolutional Autoencoder (CAE)

As a special type of the traditional autoencoder, the convolutional autoencoder replaces the original fully connected layer using a convolutional layer and a pooling layer. The convolutional autoencoder has been fitted with the operations of convolution, activation and pooling in the CNN to enhance the nonlinear mapping ability of the autoencoder and its ability to extract spatial related information. The deep neural network of the convolutional autoencoder is formed by stacking several convolutional autoencoders. The hidden layer features extracted by the previous autoencoder also serve as the input of the next autoencoder. Through layer-by-layer feature extraction, the deep neural network of the trained convolutional autoencoder can complete various feature recognition tasks efficiently [33].

2.5. Self-Organizing Map (SOM) Network

As the unsupervised learning network, the SOM network changes the network parameters and structure in a self-organizing and adaptive manner after finding the inherent laws and essential attributes in the samples. The similarity of the input pattern vectors of the neural network can be measured by the distance between the vectors. Methods that are generally used include the Euclidean distance method and cosine method. The structure of the SOM network is shown in Figure 4. The typical SOM network has a competition layer and is typically designed with an input layer and competition layer. The input layer receives external information and transmits the input mode to the competition layer for the purpose of observation. The competition layer is generally used for “analyzing and comparing” the input patterns, observing the laws and then categorizing them.

3. Degradation Modeling of Rolling Bearings

Based on the full life cycle data of the rolling bearings, this article constructs a health indicator (HI) that can indicate the degradation state of the bearings. In the process of constructing the HI, performing effective feature extraction on the original data to mine its deeper characterization, followed by performing feature selection to remove redundant features, is crucial. On this basis, feature fusion is also required, which involves combining multiple effective features to establish a degradation curve that reflects the health of the bearing. Among them, feature extraction, feature selection and dimensionality reduction methods are the most essential parts.

Since the original vibration data are one-dimensional time series data, this paper uses one-dimensional convolution for improving the autoencoder; that is, the encoding and decoding part makes use of the one-dimensional convolution operation so as to construct a feature extraction model based on DCAE. In time series data, there is a strong correlation among the data. One-dimensional convolution is employed to directly process the original data without destroying the correlation between the data. The deep structure in DCAE can obtain a higher-level representation of the input data layer by layer through feature extraction, with the output of the coding network being chosen as the extracted feature. The output of the hidden layer can be regarded as a better feature expression of the input data and has stronger robustness. Because of the high dimension of the original bearing vibration signal, the single feature, when extracted directly, cannot represent its degradation status well. An SOM is an effective unsupervised feature reduction method, and the HI value can be obtained by computing the distance among the input data and the vector representation of the low-dimensional network nodes. Thus, this paper makes use of the SOM network for feature reduction based on multiple high-level features.

The process of the rolling bearing degradation modeling method is shown in Figure 5. After the original vibration data are processed in segments, they are directly input into the DCAE model. DCAE is composed of two parts: an encoding network and a decoding network. In DCAE, the coding part consists of convolutional layers and pooling layers. The decoding part includes convolutional layers and upsampling layers. The data dimension in the upsampling layers is expanded by

$$U_{n,m}^i=upsampling\left(I_{n,m}^i\right)$$

(4)

where $I_{n,m}^i$ denotes the input of the $i$th point at the $n$th block data of the $m$th layer and $U_{n,m}^i$ denotes the output of the upsampling layer of the $i$th point at the $n$th block data of the $m$th layer.

Suppose that $X={\left[\begin{array}{llll}{X}_1\hfill & X_2\hfill & \cdots \hfill & X_n\hfill \end{array}\right]}^T$ and $X^{\prime }={\left[\begin{array}{llll}{X}_1^{\prime }\hfill & X_2^{\prime }\hfill & \cdots \hfill & X_n^{\prime }\hfill \end{array}\right]}^T$ denote the input and output of DCAE, respectively. The loss function of DCAE is defined as

$$J=\frac{1}{n}{\displaystyle \sum _{j=1}^n{\Vert X_j-X_j^{\prime }\Vert }^2}$$

(5)

The processed low-dimensional features are selected as the input of the SOM network. The output value of the SOM network is the bearing degradation HI.

Experimental Data

In order to verify the effectiveness of the method, the PHM2012 bearing dataset was selected as the experimental data. The PHM2012 dataset was provided by FEMTO-ST Research Institute. The experiment was carried out on the PRONOSTIA test bench, which realized the accelerated degradation experiment of the bearing under different working conditions. The test bench is shown in Figure 6. The dataset was designed for three different working conditions according to the load force and speed, as shown in

Table 1. Three working conditions of the PHM2012 dataset.

Working Conditions	Load Force	Speed
1	4000 N	1800 r/min
2	4200 N	1650 r/min
3	5000 N	1500 r/min

.

This article used 7 bearings under working condition 1 to conduct experiments and re-divide the dataset. The signal sampling frequency was 25.6 kHz, and sampling was performed every 10 s. Each sampling contained 2560 sampling points, and the sampling time was 0.1 s. Figure 7 shows the original vibration data of each bearing in the horizontal direction under working condition 1. The degradation of some bearings was complicated.

The specific steps of modeling the bearing degradation state are explained with Bearing1_3 described as an example:

Step 1: The sampling point of Bearing1_3 is 2375, and each sampling point corresponds to 2560 vibration values. Taking the 2560 vibration values as one sample, there are 2375 samples in total. After segmented processing, the original data are directly input into the DCAE model.

Step 2: DCAE model training. The detailed structure diagram of the DCAE is shown in Figure 8, where C1–C12 represents the convolution layer, P1–P4 represents the pooling layer, and U1–U4 represents the upsampling layer. The coding part contains 6 convolutional layers, and the number of convolution kernels in each layer is 16, 32, 64, 128, 32 and 1. The decoding part contains 6 convolutional layers, and the number of convolution kernels in each layer is 32, 128, 64, 32, 16 and 1. The size of all convolution kernels is $3\times 1$. The model learning rate is set to 0.001, the ReLU function is selected as the activation function after each convolutional layer, and the optimizer selects RMSProp.

Step 3: SOM network training. The output after layer C6 is selected as the input of the SOM network, 80 features extracted from the DCAE are input into the SOM network as a sample, and the number of nodes in the input layer of the SOM network is consistent with the number of features. Each neuron in the topological layer is represented by a vector composed of the weights between the neurons in the input layer connected to it; that is, the dimension is consistent with the input data.

The training steps of the SOM network are as follows:

Step 1: Set the number of neurons at the topological layer as $d$ and the maximum training times as $N$. Usually, the number of neurons at the topological layer is $d=5\sqrt{M}$, and $M$ is the input sample number.

Step 2: Select the 80-dimension sample $x_t=\left\{x_{1t},x_{2t},\dots ,x_{80t}\right\}$ at the $t$th time point of the training data, where $x_t$ is the 80-dimension feature extracted from the training set data at each time point. $t=1,2,\dots ,p$, $p$ is the number of time points, $x_{jt}$ is the $j$th neuron at the $t$th time point of the input layer and $j=1,2,\dots ,80$. Input the samples of all $p$ time points into the SOM network, and the number of nodes in the input layer of the network is the same as the number of features.

Step 3: Calculate the distance $d_t$ between the vector representation of all $d$ neurons in the topology layer and the input layer sample $x_t$. Choose the neuron with the smallest distance from $x_t$ as the best matching neuron $l$, namely $\Vert x_t-w_l\Vert =\mathit{min}\left\{d_t\right\}$, where $w_l$ is the vector representation of the best matching neuron. The vector of the connection weights between each neuron in the topological layer and the input layer neuron connected to it is represented by $w_{i^{\prime }}=\left\{w_{i^{\prime }1},w_{i^{\prime }2},\dots ,w_{i^{\prime }80}\right\}$, where $i^{\prime }$ represents the $i^{\prime }$th neuron of the topological layer and $i^{\prime }=1,2,\dots ,d$.

Step 4: Update the best matching neuron and the connection weights of the neighborhood neuron and the input layer neuron:

$$w_{i^{″}j}(n+1)=w_{i^{″}j}(n)+\eta (n)\cdot T_{i^{″},l}\cdot \left(x_{jt}-w_{i^{\prime \prime }}(n)\right)$$

(6)

where $i^{″}$ is the $i^{″}$th neighborhood neuron in the topological layer around the best matching neuron $l$, $n$ is the training step, $w_{i^{″}j}(n+1)$ is the connection weight between the input layer neuron $x_{jt}$ and the $i^{″}$th neighborhood neuron in the topology layer during $n+1$ training, $\eta (n)$ is the gain function, $0<\eta (n)<1$, $\eta (n)$ gradually decreases as the number of training increases and $T_{i^{″},l}$ is the weight:

$$T_{i^{″},l}=\mathit{exp}\left(-S_{i^{″},l}^2/2{\sigma }^2\right)$$

(7)

where $S_{i^{″},l}^{}$ is the Euclidean distance between the $i^{″}$th neighborhood neuron of the best matching neuron $l$ of the topological layer and the best matching neuron $l$. Here, $\sigma$ is the standard deviation of the distance between each neighborhood neuron of the best matching neuron $l$ of the topological layer and the best matching neuron $l$.

Step 5: Choose an 80-dimension sample at another point in time to provide to the input layer of the SOM network and return to step 3 until all the training set samples are provided to the SOM network.

Step 6: Perform training step $n=n+1$ and return to step 2 until the training step reaches the maximum number of training times $N$.

Step 7: After training, calculate the bearing health factor at each time point:

$$HI=MQE=\Vert x_t-w_l\Vert$$

(8)

where $MQE$ is the minimum quantization error. Taking the features extracted by Bearing1_3 as an example, the number of nodes in the input layer of the SOM network is 80, and the corresponding HI can be calculated for each sampling point so that the HI curve of the training set can be constructed.

Step 4: Algorithm evaluation and verification.

In order to make the HI curve smoother, the obtained HI curve is subjected to sliding filter processing with a window size of 8. The normalized results of the seven bearings are shown in Figure 9. The degraded state modeling method of rolling bearings proposed in this paper can better extract the degraded curve of the original vibration signal. The whole method is carried out without supervision, does not rely on expert experience and has good practicability and versatility.

In order to verify the advantages of the method in this paper, the HI curves constructed by four other methods are selected for comparison. The first method, the traditional time–frequency domain feature extraction of the signal, is a method to extract the following features: time domain characteristics (root mean square, peak value, peak-to-peak value, variance, kurtosis, skew, waveform index, peak index and margin index) and frequency domain characteristics (root mean square frequency, center of gravity frequency, frequency standard deviation and frequency variance). For the energy coefficient after discrete wavelet packet decomposition, perform three-layer wavelet packet decomposition on the original signal to obtain eight sub-bands and calculate the energy value of each sub-band as a feature. The 21 features obtained are reduced by PCA to obtain the state degradation curve of the bearing. The method was named PCA. The second method, ELM_AE, was used in [34]. The third method, RMS, was used in [35]. Finally, the fourth method, SAE, was used in [36]. The HI curves constructed by bearing1_3 under the five methods were compared, and the normalized results of each method are shown in Figure 10.

In order to analyze the pros and cons of the above methods in more detail, this article selected the monotonicity and correlation indicators from the commonly used health factor evaluation indicators to quantitatively analyze each method. Monotonicity measures the monotonic change trend of the HI curve, and correlation reflects the correlation between the degradation trend of the HI curve and the working time. The expressions of the two are defined as follows:

$$V_{mon}=\left|\frac{Num of dF>0}{T-1}-\frac{Num of dF<0}{T-1}\right|$$

(9)

$$V_{corr}=\frac{\left|{\displaystyle \sum _{t=1}^T(H_t-\widehat{H})}(l_t-\widehat{l})\right|}{\sqrt{{\displaystyle \sum _{t=1}^T{(H_t-\widehat{H})}^2}{\displaystyle \sum _{t-1}^T{(l_t-\widehat{l})}^2}}}$$

(10)

where dF is the differential between two adjacent points in the HI curve, $H_t$ and $l_t$ represent the HI curve value and sample number corresponding to the $t$th sample, respectively, and $T$ is the number of samples for the full life of the bearing.

Table 2. HI curve evaluation results.

Method		Bearing1_1	Bearing1_2	Bearing1_3	Bearing1_4	Bearing1_5	Bearing1_6	Bearing1_7
DCAE + SOM	Mon	0.21	0.27	0.35	0.41	0.19	0.21	0.21
	Corr	0.97	0.94	0.97	0.91	0.92	0.97	0.96
PCA	Mon	0.13	0.07	0.13	0.09	0.21	0.04	0.06
	Corr	0.86	0.19	0.73	0.32	0.19	0.19	0.56
RMS	Mon	0.17	0.13	0.14	0.16	0.12	0.10	0.11
	Corr	0.83	0.29	0.77	0.32	0.16	0.10	0.23
ELM_AE	Mon	0.15	0.14	0.14	0.15	0.12	0.11	0.10
	Corr	0.86	0.33	0.77	0.31	0.28	0.18	0.34
SAE	Mon	0.21	0.16	0.36	0.33	0.21	0.17	0.21
	Corr	0.96	0.91	0.91	0.92	0.91	0.93	0.94

shows the evaluation index values of the HI curve constructed by the method in this article and the comparison method mentioned above. The overall performance of the HI curve constructed by the method in this paper was better than that of the other methods on seven bearings.

This part describes the selection of some parameters in the convolutional autoencoder combined with the evaluation index. After experimental analysis, it was found that in the CNN, the number of convolutional layers and convolution kernel size were the key factors that determined the performance of the network. Combined with the two evaluation indexes (monotonicity and correlation), the influence of the size of the convolution kernel and the number of the convolution kernel layers on the model effect was analyzed. The five structures constructed in the DCAE encoder and decoder are shown in

Table 3. The five structures of the DCAE.

Number	Number of Convolution Kernels and Network Layers	Convolution Kernel Size
Structure 1	16-32-64-128-32-1-32-128-64-32-16-1	$3\times 1$
Structure 2	16-32-64-128-32-1-32-128-64-32-16-1	$7\times 1$
Structure 3	128-256-512-1024-512-1-512-1024-512-256-128-1	$3\times 1$
Structure 4	16-32-64-128-256-512-1024-512-1-512-1024-512-256-128-64-32-16-1	$3\times 1$
Structure 5	16-32-64-1-64-32-16-1	$3\times 1$

.

We took the seven bearings under working condition 1 as the experimental data. The evaluation results of the HI curves obtained by the five structures are shown in

Table 4. Evaluation results of HI curves trained by five structures.

Number	Structure 1		Structure 2		Structure 3		Structure 4		Structure 5
	Mon	Corr	Mon	Corr	Mon	Corr	Mon	Corr	Mon	Corr
Bearing1_1	0.21	0.97	0.14	0.85	0.20	0.85	0.10	0.35	0.16	0.85
Bearing1_2	0.27	0.94	0.15	0.94	0.25	0.87	0.16	0.32	0.14	0.87
Bearing1_3	0.35	0.97	0.19	0.95	0.21	0.94	0.23	0.11	0.19	0.86
Bearing1_4	0.41	0.91	0.37	0.90	0.17	0.91	0.15	0.04	0.15	0.84
Bearing1_5	0.19	0.92	0.15	0.90	0.19	0.85	0.14	0.32	0.11	0.81
Bearing1_6	0.21	0.97	0.18	0.85	0.15	0.87	0.10	0.31	0.11	0.87
Bearing1_7	0.21	0.96	0.19	0.85	0.11	0.55	0.11	0.11	0.13	0.75

. The effect of structure 1 was superior to that of the other methods. Comparing structure 1 with structure 2, the model effect of the larger convolution kernel was not better than that of the smaller convolution kernel. By comparing structure 1 with structure 3 and structure 4, the reasons were analyzed. As the original data were a one-dimensional time series with high dimensions but no complex structure, a larger number of convolution kernels and convolution layers did not provide better results and brought about a longer training time. After analyzing the computational loss function and data volume of structure 4, it was found that when a model could achieve better training results, the more the network layers and the number of convolution kernels increased, and the more likely it was that overfitting would occur. Structure 5 was obviously underfitting and did not capture the implied features in the data well.

4. Prediction of the RUL Based on 1D-CNN

The prediction process of the RUL of the rolling bearing based on the 1D-CNN is shown in Figure 11. After the segmented processing of the original data, labels were added to the original data using the constructed HI value. The labeled data were directly input into the 1D-CNN model, and the output of the model was the predicted value of the remaining life of the bearing reflected by the vibration signal at the input time.

There are two key factors in training a good intelligent diagnosis model or RUL prediction model based on deep learning methods: one is whether the selection of the algorithm is suitable for the data sample, and the second consideration is the data quality. This is because in an actual industrial environment, the data volume of the whole life cycle of the mechanical equipment will be enormous. A CNN is characterized by weight sharing and local connections between neurons, making it ideal for processing copious amounts of data. Simultaneously, the original data are the vibration data of a one-dimensional time series. The model uses a one-dimensional convolution operation to directly input the original data into the model, which will not destroy the correlation between data and allows it to acquire the feature information in the data to the maximum extent. Empirically, a model trained on inferior quality data will lead to problems such as low model accuracy, poor generalization ability and overfitting.

Currently, in the model training of supervised learning, the data are input into the model with labels, which means that when the data are input into the model, the data are artificially judged, and labels are added [37]. The ability to accurately add labels to the data influences the data’s quality and thus affects the effect of the training model.

In the problem of predicting the remaining life of mechanical equipment based on a supervised learning model, adding labels to the data is equivalent to modeling the degradation state of the equipment. At present, there are two main ways to add labels [38,39,40,41,42]. One is to use a linear function, as shown in Figure 12a. The linear function is employed to simulate the linear degradation trend of the bearing, which is defined as

$$f\left(t_i\right)=-\left(\frac{1}{t_n}*t_i\right)+1$$

(11)

where $t_n$ is the full lifetime of the bearing and $t_i$ is the current time in the bearing life cycle. In Figure 12a, the ordinate is the label value, and the actual meaning is the RUL. The trend of the degradation rate is linear, which illustrates that the degradation rate is steady. However, the degradation rate of the bearing will not remain unchanged during the operation of the bearing.

The second way is to design a piecewise function to describe the degradation process of the bearing, as shown in Figure 12b. Its definition is written as

$$f\left(t_i\right)=\{\begin{array}{cc}1& t_i\le t_j\\ \left(\frac{1}{t_j-t_n}\right)*t_i+\left(\frac{t_n}{t_n-t_j}\right)& t_i>t_j\end{array}$$

(12)

where $t_j$ is the initial degradation time of the bearing (i.e., the initial failure point). From Figure 12b, the health degree always remained the same before the time $t_j$ and then linearly descended until the time of $t_n$ (i.e., the total failure of the bearing). This method is also inaccurate. First, it is difficult to select critical points. Secondly, this method ignores the weak information hidden in the early data, and the degradation rate in the accelerated degradation stage is not linear. The model trained by these two ways of adding labels can fit the actual labels in some working conditions, but there are problems in the expression of meaning and practical application. The predicted value of the model only predicts the tag well but may not accurately predict the actual remaining life, and the generalization ability of the model is weak. Therefore, this paper selected the extracted HI as labels to add to the original data and input them into the RUL model for training.

The structure of the convolutional neural network established in this paper is shown in Figure 13. The convolution layer was expressed as Conv_layer (number of convolution kernels, kernel size, padding and stride), and the maximum pooling layer was expressed as MaxPool1d (kernel size and stride). During training, the optimizer chose RMSProp, where the learning rate was 0.001, the batch size was 32, and the maximum iterations was 100. In order to prevent overfitting, the early-stopping mechanism was introduced in the subsequent training process. The original vibration data in the dataset were selected again for life prediction. The HI value was constructed with a single sample length of 2560. In order to maintain the consistency of the data in the RUL prediction model with the added labels, the sample length was also set at 2560. After segmented processing, the original samples were added with labels and directly input into the model for training. The training set and testing set were divided by an alternate method; that is, one numbered bearing of the seven bearings was selected as the test set, and the other six bearings were selected as the training set, again, taking Bearing1_3 as an example. Therefore, the data of Bearing1_3 were used as the testing set, and the life prediction effect of the model is shown in Figure 14. The model could accurately predict the residual life of Bearing1_3.

In order to verify the advantages of this method, two common ways of adding labels were selected for comparison. In [41], the method of adding labels in Figure 12a was selected, named Method2. The method of adding labels in [39] is shown in Figure 12b, named Method3. The method in this paper is named HI–Method. After the same data were segmented, labels were added in three diverse ways. All three kinds of data were input into the 1D-CNN model, and the three models were trained. The recognition effect of the seven bearings of the three models under working condition 1 is shown in Figure 15.

As can be seen from Figure 15, the second half of the training results of Method3 had a large gap with its own label but was closer to the label value of HI–Method. The features extracted from the original vibration signals by the model were more in line with the trend of adding labels with HI–Method. On the whole, the predicted value of Method3 fluctuated more violently in comparison with HI–Method, and in most cases, the predicted value was more unstable after the selected obvious degradation point. In the life prediction of several bearings, the predicted value of Method2 produced a large oscillation near the real value, demonstrating that the way of adding labels to linear functions was not consistent with the actual situation, resulting in the data with the same status being added with greatly different label values.

In order to quantitatively analyze and evaluate the prediction results to ensure the validity of the prediction results, this paper selected the mean absolute error (MAE), root mean squared error (RMSE) and the score given in the PHM2012 dataset, and these three evaluation indicators are defined as follows.

The MAE refers to the average of the absolute value of the deviation between the predicted value and the true value at all times, so it can accurately reflect the magnitude of the prediction error:

$$MAE=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|{\widehat{r}}_i-r_i\right|}$$

(13)

The RMSE refers to the square root of the ratio of the square deviation between the predicted value and the real value and the total number of predictions at all times, so it can reflect the prediction accuracy well:

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{\left({\widehat{r}}_i-r_i\right)}^2}}$$

(14)

This paper refers to the evaluation index score of the PHM2012 Data Challenge. Assume that the definition of the percentage error $\%E_{r_i}$ is shown in Equation (15), where ${\mathrm{ActRUL} }_i$ represents the actual RUL of the $i$th predicted object and ${\mathrm{PredRUL} }_i$ represents the predicted value of the RUL of the $i$th predicted object. Considering that the risk of advanced prediction of equipment in actual situations is lower than that of lagging prediction, the scoring value of the $i$th prediction object is defined as shown in Equation (16). The score value of the final RUL prediction result is shown in Equation (17), where $N$ is the number of test objects. The higher the score, the better the prediction of the remaining life:

$$\%E_{r_i}=\frac{{\mathrm{ActRUL} }_i -{ \mathrm{PredRUL} }_i}{{\mathrm{ActRUL} }_i}\times 100\%$$

(15)

$$A_i=\{\begin{array}{ll}{\mathit{exp}}^{-\mathit{ln}(0.5)\cdot \left(\frac{E_{r_i}}{5}\right)}\hfill & \mathrm{if} E_{r_i}\le 0\hfill \\ {\mathit{exp}}^{+\mathit{ln}(0.5)\cdot \left(\frac{E_{r_i}}{20}\right)}\hfill & \mathrm{if} E_{r_i}>0\hfill \end{array}$$

(16)

$$\mathit{Scor}e=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left(A_i\right)}$$

(17)

In the above indicators, $r$ and $\widehat{r}$ are the real value and the predicted value at a certain moment, respectively. The lower the values of the MAE and RMSE, the better the prediction effect.

The prediction results of Bearing1_3, Bearing1_5 and Bearing1_6 were selected for display. The prediction results of the three methods are shown in

Table 5. Performance comparison of bearing RUL prediction models.

Method	Evaluation Index	Bearing1_3	Bearing1_5	Bearing1_6
HI–Method	MAE	0.037	0.103	0.063
	RMSE	0.054	0.068	0.083
	Score	0.686	0.586	0.487
Method 2	MAE	0.083	0.184	0.153
	RMSE	0.073	0.099	0.121
	Score	0.374	0.311	0.366
Method 3	MAE	0.091	0.213	0.237
	RMSE	0.087	0.121	0.131
	Score	0.469	0.441	0.377

. As shown in Table 5, HI–Method was lower than the other two methods in terms of the MAE and RMSE, and the score was higher than that for the other two methods. For bearing 3 specifically, the RMSE reached 0.054, and the score reached 0.686. The average score of the seven bearings was 0.677. In general, the method proposed in this paper had the best effect and high accuracy.

Generalization Performance Test

A good intelligent diagnosis model must have a good generalization ability. For the data other than the learning set with the same law, the trained network can also give a suitable output. Two conditions were constructed for the purpose of assessing the generalization ability of the proposed life prediction scheme: (1) selecting a small sample of data to test whether the model could learn valuable features from a relatively small amount of data and (2) variable working conditions to asses the performance of the model in the new data without changing the model’s structural parameters.

Four bearings (Bearing2_1, Bearing2_2, Bearing2_3 and Bearing2_4) in the PHM2012 dataset under working condition 2 were selected for training the model in accordance with the scheme proposed in this paper. Figure 16 depicts the final prediction effect of the four bearings. The original vibration data, HI curve and life prediction effect diagram are all included in each figure. As can be observed, the model could likewise accurately predict the remaining life of the bearing and deduce the bearing degradation trend under variable working conditions. This shows that the model had good generalization performance.

5. Conclusions

This paper primarily studied the prediction method of the RUL of a rolling bearing based on deep learning. First, as a modeling method based on the vibration signal equipment performance degradation condition, a DCAE was utilized to analyze the data feature extraction along with an SOM network for high-dimensional feature dimension reduction. The method accurately modeled the degradation state of the roller bearings. The entire modeling process was conducted in an unsupervised manner, which reduced manual dependence. Subsequently, a 1D-CNN-based RUL prediction model was proposed. The original data were directly input into the model training, and the output was the predicted value of the RUL, which accurately predicted the RUL of the bearing and realized end-to-end prediction. The best evaluation index RMSE reached 0.054. The average score of the seven bearings was 0.677.

In this paper, two main parts of the RUL prediction scheme were improved by one-dimensional convolution operation. Experimental verification was carried out through the PHM2012 dataset. The results show that one-dimensional convolution can effectively extract the features of time series vibration data, and the two models proposed in this paper have good robustness and generalization abilities.

The method of adding labels put forward in this paper greatly enhances the prediction effect of the model, no longer relies on artificial labeling and reduces the labeling error rate. Using the HI value as the label of the data greatly improves the quality of the training data. The overall scheme proposed in this paper reduces the dependence on manual and complex signal processing algorithms, and the method has good stability and versatility.