A Review on Data-Driven Condition Monitoring of Industrial Equipment

Abstract

This paper presents an up-to-date review of data-driven condition monitoring of industrial equipment with the focus on three commonly used equipment: motors, pumps, and bearings. Firstly, the general framework of data-driven condition monitoring is discussed and the utilized mathematical and statistical approaches are introduced. The utilized techniques in recent literature are discussed. Then, fault detection, diagnosis, and prognosis on the three types of equipment are highlighted using a variety of popular shallow and deep learning models. Applications of these techniques in recent literature are summarized. Finally, some potential future challenges and research directions are presented.

1. Introduction

With the continuous improvement of the industrialization level, the design and manufacturing technologies of the equipment have also advanced swiftly, making the equipment’s safety and reliability increasingly significant [1]. Equipment in operation is constantly exposed to various environmental forces (such as cutting forces, friction, ambient temperature, and vibration), and is susceptible to wear and tear, rusting of parts, deterioration of components, and other problems, leading to frequent abnormalities in the equipment, resulting in a gradual decline in its efficiency and life, and even catastrophic failures. To address these issues, condition monitoring of equipment has attracted tremendous research attention to ensure the safety and reliability of production [2,3,4,5,6,7]. Based on online monitoring of time-series data, the failure process can be separated into two stages for most of the equipment, as depicted in Figure 1. The first phase is the normal operating phase, which begins with the installation and commissioning of the system and ends with the appearance of abnormalities. During this phase, the monitored data is relatively steady and has no discernible pattern of change. The second stage is the progression from the appearance of anomalies to the degradation and eventual failure of the equipment. If the abnormalities discovered in this phase lead directly to equipment failure, a fault detection and diagnostic model is constructed based on the monitored data to determine the fault type, severity, and location. If the data observed during this phase reveals a tendency of degradation that is not severe enough to impede the normal operation of the equipment, a model can be constructed to predict the failure and estimate the remaining useful life of the equipment based on the degradation trend. There are three steps in equipment condition monitoring: (1). data collection; (2). data processing; and (3). Fault detection, diagnosis, and prediction. Data collection refers to the real-time monitoring of parameters or signals relevant to the performance state of the equipment using various types of sensors, and the aggregation of the collected data. Data processing refers to the pre-processing of signals or data obtained by sensors, including noise reduction, reconstruction, and feature extraction. Features include time-domain features, frequency-domain features, time-frequency-domain features, and features autonomously learnt by deep learning models. Lastly, for the detected deviations from normal operation, intelligent fault diagnosis algorithms are used to determine the specific fault type of the equipment, and for equipment exhibiting signs of degradation or failure, prediction algorithms are used to predict the future trend of the equipment’s performance state and its remaining useful life.

In general, the method to condition monitoring has undergone three stages of development, with the first relying on manual monitoring and diagnosis by specialized engineers. The second development stage produces the data-driven condition monitoring based on the continuous growth of sensor technology, more and more novel sensors are installed or integrated in the equipment to gather real-time monitoring information of the equipment, and a large number of signal analysis techniques are suggested and implemented in the monitoring sector, realizing the contemporary fault diagnosis technology with fault mechanism, sensor monitoring, and signal analysis as the co-main technologies. The most renowned of these are mathematics and statistics-based techniques, including Fourier Transform (FT), Wavelet Transform (WT), and Empirical Modal Decomposition (EMD) [8,9,10]. The third stage of development is artificial intelligence-based condition monitoring, where artificial intelligence techniques, mainly represented by machine learning, have been quickly developed. In the field of intelligent condition monitoring of equipment, machine learning algorithms are used in conjunction with techniques such as fault mechanism and signal analysis. The most common approaches are shallow learning algorithms represented by artificial neural networks (ANN), support vector machines (SVM) and extreme learning machines (ELM) [11,12,13], and deep learning algorithms represented by deep belief networks (DBN), convolutional neural networks (CNN) and recurrent neural networks (RNN) [14,15,16]. Motors, pumps, and bearings, as key components of industrial systems and applications, are essential equipment in the contemporary industrial sector. These essential pieces of equipment must work with reliability and safety [17,18,19,20]. Therefore, the main objective of this paper is to review the condition monitoring of equipment, particularly for motors, pumps, and bearings, based on the data-driven methods.

The remaining parts of the paper are organized as follows. Data-driven condition monitoring approaches for motors, pumps and bearings are presented in Section 2. Different data processing methods and their applications in the monitoring of industrial equipment are discussed. Then, data-driven fault detection techniques and their applications to industrial equipment are presented. Finally, Section 3 discusses challenges and future trends.

2. Data-Driven Condition Monitoring of Industrial Equipment

Data-driven condition monitoring technology relies on various sources and types of monitoring data collected during the operation of the equipment and employs various data mining techniques to extract the useful information implied therein, thereby enabling condition monitoring and fault diagnosis of the equipment [21,22,23]. As mentioned previously, the deployment of this technology consists of three steps: data collection; data processing; and fault detection, diagnosis and prediction. The methods and applications of data processing and condition monitoring and prediction are reviewed next in this section.

2.1. Data Processing Methods

Feature extraction and feature selection are the most critical parts of data processing techniques. Features extraction is the process of obtaining sensitive features from the original signal that effectively highlight variations between the various operation conditions of the equipment. Time-domain feature extraction, frequency-domain feature extraction, and time-frequency feature extraction are the primary feature extraction methods, with the kind and physical relevance of the derived characteristics dictating which approach is utilized. The method for selecting a sensitive, highly differentiated, and moderately dimensional collection of features from the initial defect feature set is known as feature selection [24,25].

Fourier Transform, Wavelet Transform, and Empirical Model Decomposition are the primary approaches. Figure 2 shows a simple flow chart for applying these three techniques.

2.1.1. The Fourier Transform Approach

The FT approach can be used to extract time-varying signal information in the frequency domain and compare it to the characteristics of the normal state for condition monitoring. Although the Fourier transform links the time and frequency domain properties of a signal, allowing for analysis and diagnosis of the signal state in the frequency domain, it is impossible to establish the time of occurrence of any frequency in the signal spectrum in the time domain. Frequently, the test signal at the time of failure coincides to an abrupt shift in frequency. Based on the Fourier transform, the Short-time Fourier transform (STFT) method is presented to analyze the frequency information of a time-varying signal at a certain time interval or instant.

The STFT for any signal $\mathrm{x}\left(\mathrm{t}\right)$ is defined as:

$${\mathrm{S}}_{\mathrm{x}}\left(\mathrm{t}, \mathsf{\omega }\right)=\underset{-\infty }{\overset{+\infty }{{\displaystyle \int }}}\left[\mathrm{x}\left(\mathsf{\tau }\right)\mathrm{g}\left(\mathrm{t}-\mathsf{\tau }\right)\right]{\mathrm{e}}^{-\mathrm{j}\mathsf{\omega }\mathsf{\tau }}\mathrm{d}\mathsf{\tau }$$

(1)

where $\mathsf{\tau }$ is the time shift parameter, $\mathrm{x}\left(\mathsf{\tau }\right)$ is the time-domain signal, and $\mathrm{g}\left(\mathrm{t} - \mathsf{\tau }\right)$ represent the time window, which represents a function that is constantly equal to 0 or quickly converges to 0 outside a finite interval, in which the centre is located at time $\mathsf{\tau }$. Although STFT is capable of handling non-stationary signals, it is better suited for the analysis of quasi-stationary signals, particularly long-period quasi-stationary signals, but not for the analysis of local time-frequency characteristics of non-stationary signals, as it is a Fourier transform on local intervals. Currently, STFT is mostly utilized for equipment condition monitoring.

STFT has been successfully applied to equipment signal analysis and feature extraction in previous research because its benefits in dealing with non-stationary random signals [26,27,28,29,30]. Therefore, extensive study has been conducted on condition monitoring strategies that combine this method with other methodologies. For example, in [31], the discrete Fourier transform (DFT) is used to analyse the voltage signal waveform of the Brushless DC motor to detect if a fault has occurred and to determine the number of turns shorted. Then, frequency domain analysis of the integrated voltage characteristic waveform is performed by STFT to determine the index of the faulty phase. Finally, an independent adaptive neuro-fuzzy inference system (ANFIS) is used to implement detection and diagnosis of stator interturn short circuit faults. Aimer et al. [32] analyzed the current signal of an induction motor with a broken rotor bar fault using STFT. On the produced time-frequency diagram of the current signal, several harmonics were noted, and the sideband frequencies were computed, allowing for the detection and diagnosis of the fault. In [33], STFT is used to analyse the vibration signals of rolling element bearing under different states and obtained the time-frequency distribution (TFD) characterizing the local faults. Then, the TFDs of nine different bearing faults (e.g., inner race fault, ball fault and outer race fault) were extracted, clustered, and identified by non-negative matrix decomposition (NMF). The results show that the fault identification accuracy of the method can reach 99.3%, which is much higher than the accuracy of 75.8% of ANN. He et al. [34] converted the raw bearing time-domain acoustic emission sensor signals into frequency domain signals by STFT, which resulted in a spectrum matrix. The sub-modes generated from the spectrum matrix are then used to train the LSMSTAR network model for bearing fault diagnosis, which achieves accurate classification of various bearing faults under different operating conditions. Similarly, Zhao et al. [35] used the STFT to decompose multiple eigenmode functions (IMFs) of the plunger pump’s vibration signal obtained by complete empirical mode decomposition (CEMD). After that, the time-frequency matrices of the signals in different operating states of the plunger pump were obtained, and the time-frequency entropy of each state was calculated once to extract the features. The obtained feature matrices are then dimensionally reduced by using principal component analysis (PCA). Finally, the processed signal features are used to train a support vector machine (SVM) classifier for the diagnosis of swash plate and rotor wear faults. In addition, STFT has also been applied to signal dimensional conversion. For example, Chao et al. [36] converted the one-dimensional vibration signal of an axial piston pump into a two-dimensional spectrogram by STFT, and effectively improved the accuracy of the STFT spectrogram feature map by a denoising method, which was used as the input to a LeNet-5 convolutional neural network to effectively improve the ability to identify cavitation conditions in noisy environments. Tao et al. [37] also used STFT to transform the 1D time-domain vibration signals of bearings into 2D time-frequency maps, and the multiple time-frequency maps obtained by adjusting the length of the STFT window were used as input to a classification generative adversarial network (CatGAN), which was trained to obtain a CatGAN model for diagnosing inner race and outer race faults.

The corresponding references for the FT methods reviewed in this section are summarised in

Table 1. Motor condition monitoring using Fourier transform.

References	Application	Type of Equipment	Signal	Fault Type
[26]	Fault diagnosis	Servo motor	Current signal	Axis misalignment (right and left axis with different amplitude)
[30]	Fault diagnosis	Induction motor	Current signal	Rotor and bearing fault
[31]	Fault diagnosis	Brushless DC motor	Voltage signal	Stator interturn short circuits
[32]	Fault diagnosis	Induction motor	Current signal	Broken rotor bar fault (One or several bars broken)
[38]	Fault diagnosis	Asynchronous motor	Vibration signal	Built-in rotor imbalance, stator winding faults, built-in faulted bearing, built-in bowed rotor, built-in broken rotor bars, voltage imbalance and single phasing

,

Table 2. Pump condition monitoring using Fourier transform.

References	Application	Type of Equipment	Signal	Fault Type
[27]	Feature extraction	Centrifugal pump	Vibration signal	Cavitation
[35]	Fault diagnosis	Plunger pump	Vibration signal	Swash plate wear and rotor wear
[36]	Fault diagnosis	Axial piston pumps	Vibration signal	Cavitation (different severity)
[39]	Fault detection	Centrifugal pump	Vibration signal	Cavitation
[40]	Fault detection	Centrifugal pump	Vibration signal	Cracks and imbalances in impellers of varying degrees are simulated manually by making port dramas and hammer blows. (Impeller damage due to corrosion in the fluid and external solids materials)
[41]	Fault detection	Gear pump	Vibration signal	Abrupt changes in the behaviour caused by cavitation
[42]	Fault classification	Motor pump	Vibration signal	Misalignment unbalance rubbing accelerometer fault

and

Table 3. Bearing condition monitoring using Fourier transform.

References	Application	Type of Equipment	Signal	Fault Type
[28]	Feature extraction	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
[29]	Fault diagnosis	Rolling bearing	Vibration signal	Roller, inner race, and outer race faults
[33]	Feature extraction	Rolling bearing	Vibration signal	9 kinds of bearings with various faults, i.e., inner race fault, ball fault and outer race fault with 3 diameters status
[34]	Feature extraction	Rolling bearing	Vibration signal	Inner race, outer race, cage, and ball faults
[37]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults

.

2.1.2. The Wavelet Transform Approach

The WT approach can be used to decompose the signal at multiple scales and obtain local features in the time-frequency domain through a series of wavelet basis functions and is one of the most important tools in the time-frequency analysis of non-stationary signals.

Let $\mathsf{\Psi }\left(\mathrm{t}\right)\in {\mathrm{L}}^2\left(\mathrm{R}\right)$ (where ${\mathrm{L}}^2\left(\mathrm{R}\right)$ is the square integrable real number space), then the Fourier transform of $\mathsf{\Psi }\left(\mathrm{t}\right)$ is when the following admissibility condition is met:

$${\mathrm{C}}_{\mathsf{\Psi }}=\underset{-\mathrm{\infty }}{\overset{+\mathrm{\infty }}{{\displaystyle \int }}}\frac{|\hat{\mathsf{\Psi }}(\mathsf{\omega }){|}^2}{|\mathsf{\omega }|}\mathrm{d}\mathsf{\omega }<+\mathrm{\infty }$$

(2)

where $\hat{\mathsf{\Psi }}(\mathsf{\omega }=0)=0$.

$\mathsf{\Psi }\left(\mathrm{t}\right)$ represents the mother function, which can be shifted, stretched, or compressed to give a wavelet sequence (sub wavelet) after the transformation as follow.

$${\mathsf{\Psi }}_{\mathrm{a},\mathrm{b}}\left(\mathrm{t}\right)=\frac{1}{\sqrt{\mathrm{a}}}\mathsf{\Psi }\left(\frac{\mathrm{t}-\mathrm{b}}{\mathrm{a}}\right) \mathrm{a},\mathrm{b}\in \mathrm{R}, \mathrm{a}\ne 0$$

(3)

where the role of $\mathrm{a}$ is to scale the mother function $\mathsf{\Psi }\left(\mathrm{t}\right)$, known as the scale factor, and the role of $\mathrm{b}$ is to temporally localize the mother wavelet, known as the shift factor.

The WT inherits and expands upon the concept of STFT localization, resolving the issue that the window size in STFT does not change with frequency by providing a time-frequency window that fluctuates with frequency. Therefore, the WT can be utilized for condition monitoring by analyzing changes in the signal’s mutation value and frequency structure, as well as by utilizing the WT to denoise and extract waveform information from the signal [30,43,44]. In [45], Siddiqui and Giri used WT to decompose the stator current signal of an induction motor and extract low frequency features to represent the fault information. All the useful information of the fault is obtained by superimposing the waveforms, distinguishing the healthy and faulty states of the motor, and achieving the detection of broken rotor bar faults. In addition, many variants of the WT have also been successfully applied to equipment condition monitoring. These include continuous wavelet transform (CWT) [46,47], discrete wavelet transforms (DWT) [48,49,50,51], and empirical wavelet transform (EWT) [52]. CWT is often used to extract and analyse the wavelet features of a signal. Muralidharan and Sugumaran [53] used CWT to analyze the Monoblock centrifugal pump vibration signals and obtained feature sets consisting of different wavelets and used them as inputs to a classifier to obtain the maximum fault identification capability for the system. Finally, the diagnosis of single and hybrid faults of the pump bearing was achieved by combining the decision tree approach. In addition, CWT is also used to convert time series signals into time-frequency images. For example, Xu et al. [54] converted the bearing vibration signal into a time-frequency image by a series of CWT and used it as the input to a convolutional neural network. After training the network with multilayer pooling, unique features corresponding to faults can be captured from the original CWT images, which are then used as inputs to the gcForest model for fault classification. The results show that the hybrid model achieves high accuracy in detecting and diagnosing multiple bearing faults for datasets of different sizes. Tang et al. [55] used CWT to convert time series of vibration, pressure, and sound signals of hydraulic axial piston pumps under different operating conditions into two-dimensional time-frequency images and developed a convolutional neural network fault diagnosis model based on an adaptive learning rate strategy for each signal. The results showed that the average accuracy of fault diagnosis using different signals was 97.33%, 99.48% and 98.77%. In [56], Kamiel et al. used the DWT with multi-resolution analysis (MRA) to perform a five-level wavelet decomposition of the original pump vibration signal to obtain the low and high frequency features of the signal. Subsequently, a principal component analysis (PCA) model was constructed from the low frequency features obtained under normal operating conditions to accurately detect single and multiple faults (cavitation, impeller faults and combinations of cavitation and impeller faults) in centrifugal pumps, and analysis of the principal component scores and load maps enabled fault identification. Al Tobi et al. [57] also used DWT to perform a six-level decomposition of the vibration signal of a centrifugal pump, extracting 60 approximate and detailed features for different mother wavelet functions, and applied them to train a multilayer perceptron-based backpropagation neural network and support vector machine model. It was found that with the use of rbio1.5 mother wavelet decomposition, not only fewer feature parameters could be obtained, but also the accuracy of the fault detection and diagnosis model could be achieved at 100%. In [58], the EWT was used to extract the modes of the bearing vibration signals of the motor and their corresponding envelope spectra, and the corresponding fault eigenfrequencies obtained were used to implement fault detection. By comparing and analyzing the effectiveness of EWT with empirical mode decomposition (EMD) using simulated and real data, it was found that EWT has better performance in weak feature detection and composite fault detection, while EMD can only extract one temporal frequency of the fault feature frequencies and cannot detect faults under strong noise interference. Eren et al. [59] applied EWT to analyze the Fourier spectrum segments of motor vibration signals extracted by Fast Fourier Transform (FFT). The time domain contribution of the corresponding spectral bands was also obtained by inverse Fourier transform and finally the root mean square value of the time domain signal was calculated to achieve motor bearing fault detection.

The corresponding references for the WT methods reviewed in this section are summarized in

Table 4. Motor condition monitoring using wavelet transform.

References	Application	Type of Equipment	Signal	Fault Type
[30]	Fault diagnosis	Induction motor	Current signal	Rotor and bearing fault
[43]	Fault detection	Induction motor	Current signal	Air gap eccentricity fault
[45]	Fault detection	Induction motor	Current signal	Broken rotor bar fault
[48]	Fault diagnosis	Induction motor	Vibration signal	Rotor and bearing faults
[49]	Fault diagnosis	Permanent magnet synchronous motor	Current signal	Broken magnet and eccentricity faults
[59]	Fault detection	Induction motor	Vibration signal	Bearing fault
[60]	Fault diagnosis	Permanent magnet synchronous motor	Current, voltage and speed signal	Static and dynamic eccentricity fault

,

Table 5. Pump condition monitoring using wavelet transform.

References	Application	Type of Equipment	Signal	Fault Type
[51]	Fault feature identification	Reactor coolant pump	Vibration signal	Rotor crack faults
[52]	Feature extraction	Hydraulic pump	Vibration signal	Loose slipper fault
[53]	Feature extraction and fault diagnosis	Monoblock centrifugal pump	Vibration signal	Bearing fault, impeller defect, bearing, and impeller defect together and cavitation
[55]	Fault diagnosis	Hydraulic pump	Vibration, pressure, and sound signal	Swash plate wear, loose slipper, slipper wear, and central spring wear
[56]	Feature extraction	Hydraulic pump	Vibration signal	Slipper fault
[57]	Fault diagnosis	Centrifugal pump	Vibration signal	Five mechanical faults (bearing, misalignment, unbalance, impeller, and looseness), and a hydraulic fault (cavitation)
[61]	Fault diagnosis	Centrifugal pump	Vibration signal	Suction flow blockages and casing cavitation
[62]	Fault diagnosis	Hydraulic pump	Vibration signal	Slipper loosing and Valve plate wear fault

and

Table 6. Bearing condition monitoring using wavelet transform.

References	Application	Type of Equipment	Signal	Fault Type
[44]	Fault diagnosis	Rolling bearings	Vibration signal	Roller, inner race, and outer race faults
[46]	Fault detection	Rolling bearings	Vibration signal	Bearing faults
[47]	Fault diagnosis	Rolling bearings	Vibration signal	Inner race and outer race faults
[50]	Fault detection and diagnosis	Rolling bearings	Voltage and current signals	Partially and heavily damaged bearing fault
[54]	Fault diagnosis	Rolling bearings	Vibration signal	Ball, inner race, and outer race faults
[58]	Fault detection	Rolling bearings	Vibration signal	Motor bearing with outer race weak defect (spalling fault in the outer race of generator bearing in this wind turbine)
[63]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and roller faults
[64]	Fault diagnosis	Spindle bearing	Vibration signal	Inner race, outer race, and ball faults

below.

2.1.3. The Empirical Model Decomposition Approach

The EMD approach is an adaptive decomposition method for non-stationary signals proposed by Huang et al. [10]. in 1998. Using an iterative screening process, the approach decomposes the original non-stationary signal with poor performance and contaminants into a sequence of new, improved sequence of signals representing the oscillatory part of the original signal, which is also called the intrinsic mode functions (IMF). By layer-by-layer deconstructing the components of different feature scales in the signal, it is easier to extract the distinctive information parameters of the condition monitoring signal, which has gained significant attention in equipment fault diagnosis. The steps of EMD are as follows:

(a)

After calculating all local extremes of the original signal $\mathrm{x}\left(\mathrm{t}\right)$, a cubic spline function is used to link all local maxima as the upper envelope ${\mathrm{e}}_{+}\left(\mathrm{t}\right)$, followed by a cubic spline function to connect all local minima as the lower envelope ${\mathrm{e}}_{-}\left(\mathrm{t}\right)$. The mean envelope $\mathrm{e}\left(\mathrm{t}\right)$ is then determined between the upper and lower envelopes. Next, subtract $\mathrm{e}\left(\mathrm{t}\right)$ from the original signal $\mathrm{x}\left(\mathrm{t}\right)$ to obtain a new signal ${\mathrm{m}}_1\left(\mathrm{t}\right)$.

$${\mathrm{m}}_1\left(\mathrm{t}\right)=\mathrm{x}\left(\mathrm{t}\right) - \mathrm{e}\left(\mathrm{t}\right)$$

(4)

(b)

If ${\mathrm{m}}_1\left(\mathrm{t}\right)$ satisfies the IMF criteria [65], it is recorded as ${\mathrm{c}}_1\left(\mathrm{t}\right)$ as the first order IMF. If not, continue step a using ${\mathrm{m}}_1\left(\mathrm{t}\right)$ as the original signal $\mathrm{x}\left(\mathrm{t}\right)$ until, after $\mathrm{k}$ times computations, ${\mathrm{m}}_{\mathrm{k}}\left(\mathrm{t}\right)$ meets the IMF criteria then ${\mathrm{m}}_{\mathrm{k}}\left(\mathrm{t}\right)$ is the desired first order IMF.

$${\mathrm{c}}_1\left(\mathrm{t}\right){=\mathrm{m}}_{\mathrm{k}}\left(\mathrm{t}\right)$$

(5)

(c)

Subtract ${\mathrm{c}}_1\left(\mathrm{t}\right)$ from the original signal $\mathrm{x}\left(\mathrm{t}\right)$ to obtain the new signal ${\mathrm{r}}_1\left(\mathrm{t}\right)$ as:

$${\mathrm{r}}_1\left(\mathrm{t}\right)=\mathrm{x}\left(\mathrm{t}\right){ - \mathrm{c}}_1\left(\mathrm{t}\right)$$

(6)

As ${\mathrm{r}}_1\left(\mathrm{t}\right)$ still contains some IMF, it is treated as a new original input signal and steps a–c are repeated to find the $\mathrm{n}$ components until the iterative process stops when ${\mathrm{r}}_{\mathrm{n}}$ is a monotonic function or a constant.

$$\{\begin{array}{c}{\mathrm{r}}_1\left(\mathrm{t}\right)=\mathrm{x}\left(\mathrm{t}\right){-\mathrm{c}}_1\left(\mathrm{t}\right)\\ {\mathrm{r}}_2\left(\mathrm{t}\right){=\mathrm{r}}_1\left(\mathrm{t}\right){-\mathrm{c}}_2\left(\mathrm{t}\right)\\ ⋮\\ {\mathrm{r}}_{\mathrm{n}}\left(\mathrm{t}\right){=\mathrm{r}}_{\mathrm{n}-1}\left(\mathrm{t}\right){-\mathrm{c}}_{\mathrm{n}}\left(\mathrm{t}\right)\end{array}$$

(7)

where ${\mathrm{c}}_1\left(\mathrm{t}\right){, \mathrm{c}}_2\left(\mathrm{t}\right), \cdots {, \mathrm{c}}_{\mathrm{n}}\left(\mathrm{t}\right)$ represent the information of the signal from high to low frequencies. This completes the entire EMD process, where the original signal $\mathrm{x}$ is split into several components and a sum of residual functions.

$$\mathrm{x}\left(\mathrm{t}\right)={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{c}}_{\mathrm{i}}\left(\mathrm{t}\right){+\mathrm{r}}_{\mathrm{n}}\left(\mathrm{t}\right)$$

(8)

The various order components characterize the different frequency components of the signal and the residual function represents the average trend of the signal.

EMD is also a popular method for time-frequency domain signal analysis. Compared to wavelet transform techniques, it overcomes the problem of non-adaptive basis functions and eliminates the requirement for pre-determined basis functions, making it more adaptive to non-linear and non-stationary signals. This has won the approach a great deal of attention in the field of equipment condition monitoring [66,67,68,69,70,71,72,73]. In [74], the envelope analysis of the IMF obtained by EMD decomposition of the vibration signal of the hydraulic pump casing was carried out to derive the envelope spectrum of the first three IMFs containing the main information about the fault and used to implement fault diagnosis. The experiments show that the method can effectively diagnose the slipper loosing, swashplate wearing and valve plate wearing faults of hydraulic pump. In [75], EMD is used to extract the IMFs of the different frequency bands of the induction motor current. The dominating IMF is obtained for the analysis of motor faults by decomposing the current signals in different motor operating modes. The results show that the diagnosis of broken rotor bars faults in induction motors can be accurately done by evaluating the variation of the amplitude of the IMF oscillation, under varied loads. Sadeghi et al. [76], used EMD to decompose the stator current of an induction motor into multiple IMFs with specific frequency bands, and the IMF in the first row with the fastest response to the fault was used for the detection of motor stator faults. Subsequently, to distinguish between motor current fluctuations caused by normal operation and those caused by fault conditions, the instantaneous frequency (IF) variation of the IMFs is used as a criterion for fault detection. Through different simulations and experiments, this method outperforms the WT and FT methods overall in terms of efficiency and accuracy of fault detection. However, as the acquired equipment signals are typically combined with noise or unknown intermittent signals, this may lead to model confusion when utilizing EMD for signal decomposition due to repeated jumps in the local extremes in a short time interval. To tackle this difficulty, improved EMD algorithms have been brought into the field of equipment condition monitoring, such as ensemble EMD (EEMD) [77,78,79,80,81] and complete ensemble EMD (CEEMD) [82,83]. In [84], the collected aviation hydraulic pump pressure signals were decomposed into several IMFs using EEMD to reflect the fault characteristics in the time, frequency, and time-frequency domains to define the severity of the faults. Using PCA to scale down the defect characteristics, a support vector regression (SVR) model was then developed to predict the remaining life of the pump. It is demonstrated that the multi-domain fault characteristics retrieved by EEMD can more precisely reflect the severity of the loose piston defect and the remaining useful life of pump can be estimated by the SVR model. Zhao et al. [85] propose a new method for extracting fault features. Initially, the fault vibration signals of the inner ring, outer ring, and rolling element of the bearing are decomposed into IMFs with different physical significance by EEMD. Then, the IMFs with the highest correlation coefficients are chosen to characterize the original fault signals using the correlation coefficient analysis method. The obtained IMF multi-scale fuzzy entropy values are then used to generate a feature vector for the training and construction of the SVM classifier. Several experiments demonstrate that the approach can accurately detect and diagnose the type and severity of bearing faults under varying motor loads and bearing fault severity levels. CEEMD was suggested in [86] to increase algorithm efficiency and reduce computing costs, and it has been successfully used to condition monitoring. In [87], Delgado-Arredondo et al. used CEEMD to decompose microphone-recorded induction motor acoustic sound signals. The CEEMD was utilized to temporally decompose motor sound signals acquired under normal and fault operating conditions, and the corresponding spectrum of IMFs obtained by time-frequency distribution of Gabor (TFDG) was used to select specific IMFs containing fault information for fault diagnosis. It is shown that the diagnosis of unbalance condition, bearing faults, and broken rotor bar defects can be accomplished by comparing the spectra of IMFs of sound signals under normal and faulty operating conditions, and that the validity of the results can be verified by analyzing vibration signals. A defect feature extraction optimization approach is proposed in [88] for CEEMD. A genetic algorithm is used to optimize the white noise amplitude of the CEEMD so that the decomposition produces the least mean value of mutual information between the IMFs, thereby suppressing the model mixing phenomena. The method is used to the diagnosis of bearing faults, and the results indicate that the optimized method can adaptively analyze diverse signals and provide more sensitive diagnostic results than the empirically chosen white noise approach.

The corresponding references for the EMD methods reviewed in this section are summarized in

Table 7. Motor condition monitoring using empirical model decomposition.

References	Application	Type of Equipment	Signal	Fault Type
[66]	Fault diagnosis	Synchronous motor	Current signal	Broken damper bars with different asymmetry
[71]	Fault detection	Induction motor	Current signal	Rotor bar fault
[75]	Fault diagnosis	Induction motor	Current signal	Broken rotor bars (one or several bars)
[76]	Fault detection	Induction motor	Current signal	Stator short circuit faults (stator winding fault)
[87]	Fault diagnosis	Induction motor	Sound and vibration signal	Unbalance condition, bearing faults and broken rotor bars
[89]	Fault detection and diagnosis	Induction motor	Current signal	Bearing fault (Outer race and inner race)
[90]	Fault diagnosis	Permanent magnet Brushless DC motor	Current and vibration signal	Stator and rotor faults

,

Table 8. Pump condition monitoring using empirical model decomposition.

References	Application	Type of Equipment	Signal	Fault Type
[35]	Fault diagnosis	Plunger pump	Vibration signal	Swash plate wear and rotor wear
[67]	Fault diagnosis	Hydraulic piston pump	Discharge pressure signal	Swashplate wear fault, piston shoe loose fault, and piston shoe wear fault
[68]	Fault diagnosis	Airborne fuel pump	Vibration and pressure signal	Blade damage, diffusion tube damage, leakage, diffusion tube impeller rub, and bearing wear
[72]	Feature extraction	Nuclear main pump	Vibration signal	Rolling bearing fault
[74]	Fault diagnosis	Hydraulic pump	Vibration signal	Slipper losing fault, swashplate wearing faut and valve plate wearing fault
[77]	Fault diagnosis	Gear pump	Vibration signal	Tooth face wear, cavitation, oil pollution, and wear of internal surface sleeve
[81]	Fault prognosis	Reactor coolant pump	Shaft seal leakage flow	Seal leakage fault
[82]	Fault diagnosis	Vacuum pump	Acoustic emission signal	Overload fault (an overload fault was realized by changing the suction load conditions and extracting the atmosphere at full power is considered an overload fault and extracting the pressure vessel through the aperture is considered normal.)
[84]	Fault detection	Aviation piston pump	Discharge pressure signal	loose piston defect
[91]	Fault diagnosis	Hydraulic pump	Vibration signal	Single slipper wear, single slipper loose, and center spring wear faults
[92]	Fault diagnosis	Reciprocating pump	Vibration signal	Piston wear, bearing wear, and valve disc wear faults

and

Table 9. Bearing condition monitoring using empirical model decomposition.

References	Application	Type of Equipment	Signal	Fault Type
[69]	Fault diagnosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[70]	Fault diagnosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[73]	Fault detection and diagnosis	Bearing in main coolant pump and feed water pump	Vibration signal	Inner race, outer race, and ball faults
[78]	Fault diagnosis	Locomotive roller bearing	Vibration signal	Slight rub fault in the outer race Serious flaking fault in the outer race, slight rub fault in the inner race, roller rub fault, compound faults in the outer and inner races, compound faults in the outer race and rollers, compound faults in the inner race and rollers, compound faults in the outer and inner races and rollers
[79]	Fault diagnosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[80]	Fault diagnosis	Rolling bearing	Vibration signal	Outer race fault and ball faults
[83]	Fault diagnosis and prognosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[85]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race and rolling element faults
[88]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race faults

below.

2.2. Fault Diagnosis and Prediction

2.2.1. Method Based on Shallow Machine Learning

Most machine learning approaches have a shallow structure, often including one hidden layer, with strong self-learning capabilities, nonlinear mapping abilities, and high resilience for feature extraction and pattern analysis. Artificial neural networks (ANN), support vector machines (SVM), and extreme learning machines (ELM) are common shallow machine learning techniques. A typical neural network structure is shown in Figure 3.

The Artificial Neural Networks Approach

ANN is a data processing model like the human nervous system, which includes neuron nodes and weights. The nodes are the basic processing units and are arranged into layers. Neurons in the input layer receive inputs from the outside world and neurons in the output layer give out the neural network processing results. The layers between the input and output layers are called hidden layers and the neurons in these layers are known as hidden neurons. The neurons in the adjacent layers are connected and the weights indicate the strength of the connections between the nodes. It abstracts the human nervous system from the perspective of data processing by building individual neuron models and forming various types of networks according to different connections and weights. ANNs mimic the human nervous system in that they can learn from example data by changing their structure and connection weights so that they can learn the relationship between the input and output variables. Common ANNs include back propagation (BP) neural networks, radial basis function (RBF) neural networks, self-organizing competitive (SC) neural networks, etc. This paper only introduces BP neural networks in detail as they are the most widely used.

A BP neural network is a multi-layer feed-forward neural network consisting of non-linear units with an input layer, implicit (hidden) layers, and an output layer, with full connectivity between the adjacent layers and no interconnection between units in the same layer. A typical 3-layer BP neural network structure is given in Figure 3. The BP algorithm is the fundamental approach for training ANNs, which is simply a problem of finding the minimum of the error function and allows for the weight modification of a multi-layer feed-forward neural network. The BP network training process contains two stages: the forward propagation of information and the backward propagation of error. Forward propagation means that the neurons in the input layer receive incoming stimuli and transfer them, after interacting with the weights and biases, to the neurons in the hidden layer, which in turn process the received information and then pass to the output layer neurons through the connection weights. The calculated network outputs are compared with the corresponding target values to work out the neural network prediction errors. The neural network prediction errors are then processed backwards layer by layer to retrieve the deviation of each layer’s weights and biases, which are subsequently modified. The whole procedure consists of alternating forward propagation of information and backward propagation of error until the output errors are sufficiently small.

Take the BP neural network layout with a single hidden layer as an example and suppose that $\mathrm{n}$ is the number of neurons in the input layer ($\mathrm{i}$ is the index number of neurons), $\mathrm{l}$ is the number of neurons in the hidden layer ($\mathrm{j}$ is the index number of neurons), and $\mathrm{m}$ is the number of neurons in the output layer ($\mathrm{k}$ is the index number of neurons). ${\mathrm{W}}_{\mathrm{ij}}$ represents the weight between the input layer and the hidden layer neurons, whereas ${\mathrm{W}}_{\mathrm{jk}}$ represents the weight between the hidden layer and the output layer neurons. ${\mathrm{b}}_{\mathrm{j}}$ represents the bias of neurons in the hidden layer, whereas ${\mathrm{b}}_{\mathrm{k}}$ represents the bias of neurons in the output layer. The forward propagation of information is represented as follow:

$${\mathrm{y}}_{\mathrm{j}}=\mathrm{f}\left({\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{W}}_{\mathrm{ij}}{\mathrm{x}}_{\mathrm{i}}{+\mathrm{b}}_{\mathrm{j}}\right)$$

(9)

$${\mathrm{z}}_{\mathrm{k}}=\mathrm{f}\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{l}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{y}}_{\mathrm{j}}{+\mathrm{b}}_{\mathrm{k}}\right)$$

(10)

where ${\mathrm{x}=(\mathrm{x}}_1{,\mathrm{x}}_2,\cdots {,\mathrm{x}}_{\mathrm{n}})$ represents the input vector of the input layer, ${\mathrm{y}=(\mathrm{y}}_1{,\mathrm{y}}_2,\cdots {,\mathrm{y}}_{\mathrm{l}})$ represents the output vector of the hidden layer, and ${\mathrm{z}=(\mathrm{z}}_1{,\mathrm{z}}_2,\cdots {,\mathrm{z}}_{\mathrm{m}})$ represents the output vector of the output layer. Let ${\mathrm{z}}^{*}{=(\mathrm{z}}_1^{*}{, \mathrm{z}}_2^{*},\cdots {, \mathrm{z}}_{\mathrm{m}}^{*})$ be the desired output vector, $\mathrm{E}$ represents the error function which is typically half of the sum of squared errors, and $\mathsf{\eta }$ represents the learning rate in BP neural networks. The standard BP neural network modifies the weights and bias according to the negative gradient of the error function, and the backward propagation process of the errors is represented as follow:

$$\mathrm{E}=\frac{1}{2}{\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{m}}{\left({\mathrm{z}}_{\mathrm{k}}^{*}{-\mathrm{z}}_{\mathrm{k}}\right)}^2$$

(11)

$$\Delta {\mathrm{W}}_{\mathrm{jk}}=-\mathsf{\eta }\frac{\mathrm{\partial }\mathrm{E}}{\partial {\mathrm{W}}_{\mathrm{jk}}}=\mathsf{\eta }\left({\mathrm{z}}_{\mathrm{k}}^{*}{-\mathrm{z}}_{\mathrm{k}}\right){\mathrm{y}}_{\mathrm{j}}{\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{l}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{y}}_{\mathrm{j}}{-\mathrm{b}}_{\mathrm{k}}\right)$$

(12)

$$\Delta {\mathrm{b}}_{\mathrm{k}}=-\mathsf{\eta }\frac{\partial \mathrm{E}}{\partial {\mathrm{b}}_{\mathrm{k}}}=\mathsf{\eta }\left({\mathrm{z}}_{\mathrm{k}}{-\mathrm{z}}_{\mathrm{k}}^{*}\right){\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{l}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{y}}_{\mathrm{j}}{-\mathrm{b}}_{\mathrm{k}}\right)$$

(13)

$$\Delta {\mathrm{W}}_{\mathrm{ij}}=-\mathsf{\eta }\frac{\partial \mathrm{E}}{\partial {\mathrm{W}}_{\mathrm{ij}}}{=\mathsf{\eta }\mathrm{x}}_{\mathrm{i}}{\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{W}}_{\mathrm{ij}}{\mathrm{x}}_{\mathrm{i}}{-\mathrm{b}}_{\mathrm{j}}\right){\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{m}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{l}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{y}}_{\mathrm{j}}{-\mathrm{b}}_{\mathrm{k}}\right)\left({\mathrm{z}}_{\mathrm{k}}^{*}{-\mathrm{z}}_{\mathrm{k}}\right)$$

(14)

$$\Delta {\mathrm{b}}_{\mathrm{j}}=-\mathsf{\eta }\frac{\partial \mathrm{E}}{\partial {\mathrm{b}}_{\mathrm{j}}}{=\mathsf{\eta }\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{W}}_{\mathrm{ij}}{\mathrm{x}}_{\mathrm{i}}{-\mathrm{b}}_{\mathrm{j}}\right){\displaystyle \sum }_{\mathrm{k}=1}^{\mathrm{m}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{f}}^{\prime }\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{l}}{\mathrm{W}}_{\mathrm{jk}}{\mathrm{y}}_{\mathrm{j}}{-\mathrm{b}}_{\mathrm{k}}\right)\left({\mathrm{z}}_{\mathrm{k}}{-\mathrm{z}}_{\mathrm{k}}^{*}\right)$$

(15)

Once the adjustments for the weights and bias are found, the network weights and bias can be updated, and then the process of forward propagation of information, backward propagation of errors, and adjustments of weights and bias calculated again until the iteration termination condition is met.

Neural networks have received increasing interest from academics and have been used in the field of equipment condition monitoring as artificial intelligence algorithms have advanced. The condition information of a device is the carrier of condition diagnosis, and different types of condition information respond to different degrees of the device’s condition, so to achieve condition monitoring, scholars have tried to train ANN models with different condition information, which can be vibration, noise, temperature, pressure, acoustic emission, and so on [93,94,95,96,97,98,99,100,101,102]. Vibration signals are used more frequently. In [103], Patel et al. used the wavelet packet transform to extract statistical features from the original bearing vibration signal and developed a bearing damage index (BID) to select the dominant wavelet packet transform to further extract statistical features from the vibration signal features and used the extracted features as inputs to an ANN model. The results show that ANN is more effective in fault detection and prediction than features extracted from the original signal when data from higher BDI vibration signals are selected. Jami et al. [40] decomposed the vibration signal of a centrifugal pump impeller using time analysis (TA), frequency analysis (FA) and wavelet packet transform (WPT) respectively to obtain the time domain statistical parameters, frequency domain peaks and time frequency domain wavelet packet energy presented as feature datasets. ANN fault diagnosis models were established for each of these three data sets. After adjusting the network nodes and transfer functions, the results show that the feature datasets obtained after WPT analysis can better represent the fault information and have higher diagnostic performance for pump impellers under different operating conditions. In [41], a non-linear auto-regression (NLAR) model was developed based on the time series of pump casing vibration signals under non-fault conditions, with a multilayer neural network as the non-linear estimator. Compared to the Fast Fourier Transform (FFT) based diagnostic method, the ANN based NLAR model can effectively detect faults at the early stage and is successfully applied to pump cavitation fault detection. Sharma et al. [104] obtained the frequency domain characteristics of the vibration signals of three-phase asynchronous induction motor ball bearings under different conditions by frequency domain analysis. These features were used to train an ANN classifier model, which ultimately enabled fault diagnosis of defects in the inner and outer rings of the bearings. In addition to the use of vibration signals, current signals and temperature signals are often used as state information for the training of ANN models. In [89], Refaat et al. decomposed the stator current signal of a three-phase motor into an intrinsic mode function (IMF) containing the main amplitude and frequency information by means of the empirical mode decomposition (EMD) method. The IMFs were then applied to the Wigner-Ville distribution (WVD) to obtain WVD contour patterns to characterize the characteristic frequencies of bearing defects. The contour pattern is then pattern recognized using an ANN, which can effectively detect outer ring defects in bearings. Sheikh et al. [105] transformed the induction motor current signal into a two-dimensional park vector (PV) pattern image by the park vector analysis (PVA) method. The different features of the pattern image (Area of PV pattern; Circumference of the pattern; width of the pattern; resultant of the centroid) were then used as inputs to the ANN and the motor bearing state as output from the ANN, enabling the detection and diagnosis of faults caused by mis-mounted bearings. This enables the detection and diagnosis of defects caused by incorrectly mounted bearings. Bangalore and Tjernberg [106] constructed the detected average temperature signals of the wind turbine gearbox bearings into bearing behaviour patterns via the SCADA system. The alarm and warning feature data from the SCADA system were then used to train the ANN model. The results show that the method can efficiently and accurately detect failures due to spalled or damaged bearings. Additionally, in addition to employing a single signal as state information, multiple signals are often used in combination to train ANN-based state detection models. For instance, in [107], Verma et al. transformed the sample entropy (SampEn) into the multiscale entropy of the current and vibration signals of an induction motor into a time series and used it as input to an ANN for misalignment fault detection of induction motors. This approach not only reduces the number of sensors, but also avoids the need for extensive calculations in the frequency domain analysis. Sharma et al. [108] used the data set of electric submersible pump (ESP) system variables obtained from real-time sensor monitoring and the corresponding combination of variables calculated (e.g., temperature, pressure, etc.) as input to an ANN, with the input weights of the variables determined by the severity of their impact on the system. Afterwards, the inputs are compared with the operating ranges corresponding to the ESP components in the hidden layer, and finally the output layer determines whether the input conditions will lead to an ESP failure or predict the trend leading to an ESP failure. In [90], the current signal is analyzed by performing a third harmonic analysis to extract the amplitude and frequency features of the fault, and the vibration signal is decomposed into intrinsic mode functions (IMFs) using complete ensemble empirical mode decomposition (CEEMD) to extract the signal features with the best IMFs in the time and frequency domains. Afterwards, the features of the current and vibration signals that are most relevant to the health indicators are combined and reduced to two-dimensional primary metric space data using a PCA model. This dataset was used to train the ANN model and achieved a diagnostic accuracy of 99% on the test data obtained under stator fault of a brushless DC motor.

Figure 4 shows the basic processes for condition monitoring using the ANN model, and the methods described next in this paper have the similar process to this method. The relevant references for the ANN methods discussed in this section are summarized in

Table 10. Motor condition monitoring using artificial neural networks.

References	Application	Type of Equipment	Signal	Fault Type
[89]	Fault detection and diagnosis	Induction motor	Current signal	Bearing fault (inner and outer race)
[90]	Fault diagnosis	Permanent magnet Brushless DC motor	Current and vibration signal	Stator and rotor faults
[93]	Fault diagnosis	Induction motor	Current and vibration signal	Various types of motor faults such as bearing, stator, rotor, and eccentricity
[94]	Condition diagnosing and remaining useful life predicting	Induction motor	Current and voltage signal	Turn-to-turn short circuit in one phase, turn-to-turn short circuit in two phases, missing phase, and two phases
[95]	Fault detection	Induction motor	Vibration signal	Unbalance
[101]	Fault detection	Induction motor	Current signal	Stator inter turn short circuit fault and unbalance supply voltage fault
[105]	Fault diagnosis	Induction motor	Current signal	Defective due to misalignment of bearing installation (misalignment, shaft deflect, outer race damage, and inner race damage)
[107]	Fault detection	Induction motor	Current and vibration signal	Misalignment faults

,

Table 11. Pump condition monitoring using artificial neural networks.

References	Application	Type of Equipment	Signal	Fault Type
[39]	Fault detection	Centrifugal pump	Vibration signals	Cavitation Vans tip fault Impeller crack fault
[40]	Fault detection	Centrifugal pump	Impeller vibration signal	Cracks and imbalances in impellers of varying degrees are simulated manually by making port dramas and hammer blows. (Impeller damage due to corrosion in the fluid and external solids materials)
[41]	Fault detection	Gear pump	Pump casing vibration signal	Abrupt changes in the behavior caused by cavitation
[68]	Fault diagnosis	Airborne fuel pump	Vibration and pressure signal	Blade damage, diffusion tube damage, leakage, diffusion tube impeller rub, bearing wear
[96]	Malfunction detection	Shimizu PS-128BT water pump	Vibration signal (bearing, impeller, and capacitor)	Broken capacitor, broken impeller, broken bearing, broken capacitor & impeller, and broken capacitor & bearing
[100]	Fault diagnosis	Reactor coolant pump	Vibration signal	Bearing wear; rotor mass eccentricity; impeller mass eccentricity; wear ring abrasion
[102]	Fault detection	Circulating water pump	Bearing temperature signal	Broken bearings, damaged bearings, high cooling water temperatures, noisy equipment, etc.
[108]	Predict failure	Real-time data collected over a period of operation of electric submersible pumps (containing the information from surface and downhole data)	Pump discharge temperature, pump intake pressure, pump discharge pressure and so on.	Higher Flow rates, low pump intake pressures. Gas production, gas to oil ratio, leading to decrease in pump throughput. High Fluid Viscosity leading to pump failures. Pump being used outside its operating range. Corrosion and depositions leading to blockages in pump, debris in pump, shaft failures due to broken shafts, change in downhole pressure, blockage at perforations and pump intake.

and

Table 12. Bearing condition monitoring using artificial neural networks.

References	Application	Type of Equipment	Signal	Fault Type
[47]	Fault diagnosis	Rolling bearings	Vibration signal	Inner race and outer race faults
[78]	Fault diagnosis	Locomotive roller bearing	Vibration signal	Slight rub fault in the outer race, Serious flaking fault in the outer race, Slight rub fault in the inner race, Roller rubs fault, Compound faults in the outer and inner races, compound faults in the outer race and rollers, compound faults in the inner race and rollers, compound faults in the outer and inner races and rollers
[97]	Remaining useful life prediction	Rolling bearing	Vibration signals	Inner race, ball, and outer race fault
[98]	Fault diagnosis	Rolling bearing	Vibration signal	Local spalls fault and Pits or distributed surface wear fault
[99]	Fault prognosis	Rolling bearing	Vibration signal	Inner race, ball, and outer race fault
[103]	Fault detection and prognosis	Rolling bearing	Bearing vibration signal	Inner race, ball, and outer race fault
[104]	Fault diagnosis	Rolling bearing	Bearing vibration signal	Inner race and outer race faults
[106]	Fault detection	Rolling bearing	Temperature measurement for five bearings	Damaged due to spalling in the bearing.

.

The Support Vector Machine (SVM) Approach

The initial objective of SVM theory was to devise a method for dealing with classification issues by locating an optimum classification hyperplane that meets the classification requirements, and by maximizing the blank area on both sides of the hyperplane. Theoretically, SVM is capable of obtaining optimum classification for two-dimensional linearly divisible data.

Consider the classification of two data types as an example, given a training sample set ${\mathrm{D}}_{\mathrm{i}}=\left({\mathrm{x}}_{\mathrm{i}}{,\mathrm{y}}_{\mathrm{i}}\right), \mathrm{i}=1,2,\cdots ,\mathrm{l}, \mathrm{x}\in {\mathrm{R}}^{\mathrm{n}}, \mathrm{y}\in \left\{\pm 1\right\}$, where $\mathrm{x}$ is the input sample, $\mathrm{y}$ is the category value, and $\left(\mathsf{\omega }·\mathrm{x}\right)+\mathrm{b}=0 (\mathsf{\omega }\in {\mathrm{R}}^{\mathrm{n}}, \mathrm{b}\in \mathrm{R})$ is the hyperplane, in order for the classification plane to accurately classify all samples and have a classification interval, it is important to verify that the hyperplane meets the constraint ${\mathrm{y}}_{\mathrm{i}}\left[{(\mathsf{\omega }\mathrm{x}}_{\mathrm{i}}+\mathrm{b})\right]\ge 1, \mathrm{i}=1,2,\cdots \mathrm{l}$. Therefore, the task of generating the optimal hyperplane becomes the problem of obtaining the minimal value within the restrictions.

$$\min \frac{1}{2}{\Vert \mathsf{\omega }\Vert }^2$$

(16)

The Lagrange function is introduced to resolve this restricted optimization issue.

$$\left(\mathsf{\omega },\mathrm{b},\mathsf{\alpha }\right)=\frac{1}{2}{\Vert \mathsf{\omega }\Vert }^2-{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{l}}{\mathsf{\alpha }}_{\mathrm{i}}\left({\mathrm{y}}_{\mathrm{i}}\left({\mathsf{\omega }\mathrm{x}}_{\mathrm{i}}+\mathrm{b}\right)-1\right)$$

(17)

where, ${\mathsf{\alpha }}_{\mathrm{i}} > 0$ is the Lagrange multiplier for each sample. Letting the partial derivatives of Equation (18) be zero for $\mathrm{b}$ and $\mathsf{\omega }$ respectively, it can be derived that:

$$\mathsf{\omega }={\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{l}}{\mathsf{\alpha }}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{i}}{\mathrm{x}}_{\mathrm{i}}$$

(18)

$${\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{l}}{\mathsf{\alpha }}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{i}}=0$$

(19)

The solution vector’s expansion consists of a subset of training sample vectors, none of which have zero Lagrange multipliers, i.e., the support vector. A sample vector with zero Lagrange multipliers contributes zero and is irrelevant for selecting a classification hyperplane. Therefore, a decision function describing the optimal classification hyperplane, also known as a support vector machine, is obtained from the training set, and its classification function is determined by the support vector.

$$\mathrm{f}\left(\mathrm{x}\right)=\mathrm{sgn}\left({\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{m}}{\mathsf{\alpha }}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{i}}\left(\mathrm{x}·{\mathrm{x}}_{\mathrm{i}}\right)+\mathrm{b}\right)$$

(20)

In the field of condition monitoring SVM is mainly used for classification problems [109,110,111,112,113,114,115,116,117]. Compared to ANN, SVM is more suitable for small sample classification and is more efficient and accurate. In [118], when the current and vibration signals of the induction motor have been gathered, statistical characteristics are derived from their respective time domain data. Afterwards, the most effective features and the optimal SVM model parameters for fault diagnosis were chosen using a wraparound model and a grid search technique. Through fault diagnosis and prediction for diverse operating conditions of induction motors at different speeds and loads, a high level of precision can be achieved. The performance of diagnosis at an intermediate speed and load (which reflect the scenario of limited data) is also encouraging. Panda et al. [119] classified statistical characteristics taken from a centrifugal pump’s time-domain vibration data using SVM. Experiments were undertaken to identify two kinds of faults: flow obstructions and pump cavitation, each with a different fault severity and conditions. The results indicate that the SVM model gives more accuracy in predicting the beginning of cavitation events than the multi-class fault classification for different degrees of blockages. In [120], the time and frequency domain features of the bearing vibration signal are used to train an SVM model. The trained classifier can be used for real-time detection of inner race, outer race and rolling element faults in bearings. The Root mean square of vibration signal (RMS) features in the time domain are used for fault detection, while the energy content and energy deviation features in the frequency domain are used for fault classification. The experimental results show an accuracy of 86% and 96% for the detection of inner and outer race faults respectively. There are also modifications of the SVM, and optimized algorithms applied to the condition detection of equipment. For instance, in [60], Ebrahimi et al. used a fuzzy support vector machine (FSVM) to increase sensitivity to data and model generalization for the assessment of static and dynamic eccentricity fault severity in permanent magnet synchronous motors. The wavelet transform analysis of stator current features was used as training and testing data for the FSVM classifier, and it was determined that the model could accurately identify the type and degree of eccentricity and had a classification accuracy of 100% even in the presence of high noise. Since SVM can only perform binary classification, Gangsar et al. [121] have employed multiclass support vector machines (MSVM) to effectively diagnose and predict a broad variety of mechanical and electrical faults in induction motors. The time domain statistical features extracted from the current and vibration signals of the induction motor were then used to train the MSVM classifier. Several experiments demonstrate that the model can accurately predict all mechanical and electrical problems under varying loads. Some researchers also optimize the SVM parameters to achieve improved classification results. In [61], the frequency domain and wavelet analysis features of the vibration signals were extracted from various fault conditions of the pump under various operating conditions. These features serve as inputs to the SVM classifier to classify pump pipe blockage and cavitation faults. In addition, three optimization algorithms were used to determine the SVM classifier’s parameters for fault classification accuracy at higher pump speeds. The relevant references for the SVM methods discussed in this section are summarized in

Table 13. Motor condition monitoring using support vector machine.

References	Application	Type of Equipment	Signal	Fault Type
[60]	Fault diagnosis	Permanent magnet synchronous motor	Current, voltage signal and speed signal	Static and dynamic eccentricity fault
[109]	Fault diagnosis	Induction motor	Current and voltage signal	Inter-turn short-circuits, rotor, and bearing faults
[118]	Fault diagnosis	Induction motor	Current and vibration signal	Stator winding faults
[121]	Fault prognosis	Induction motor	Vibration and current signal	Bearing fault, unbalanced rotor fault, bowed rotor fault, rotor misalignment fault, broken-rotor bar fault, phase unbalance and single phasing fault with high resistance, phase unbalance and single phasing fault with low resistance, stator winding fault with high resistance and stator winding fault with low resistance

,

Table 14. Pump condition monitoring using support vector machine.

References	Application	Type of Equipment	Signal	Fault Type
[35]	Fault diagnosis	Plunger pump	Vibration signal	Swash plate wear and rotor wear
[57]	Fault diagnosis	Centrifugal pump	Vibration signal	Five mechanical faults (bearing, misalignment, unbalance, impeller, and looseness), and a hydraulic fault (cavitation)
[61]	Fault diagnosis	Centrifugal pump	Vibration signal	Suction flow blockages cavitations
[84]	Fault diagnosis	Aviation piston pump	Discharge pressure signal	loose piston defect
[110]	Fault diagnosis	Oil rig motor pump	Vibration signal	Misalignment, structural looseness, unbalance, hydrodynamic, mechanical looseness, rolling bearing
[111]	Fault diagnosis	Centrifugal pump	Vibration signal	Cavitation and impeller unbalance, cavitation and shaft misalignment, impeller unbalance and shaft misalignment
[112]	Fault diagnosis	Centrifugal pump	Vibration signal	Mechanical seal and Impeller faults
[115]	Condition evaluation	Canned motor pump	Performance parameters and the structure parameters of pump (flow, power consumption, stator temperature, winding insulation)	Severe or moderate degradation and normal or good condition
[116]	Condition prediction	Reactor coolant pump	Measurement variables	Variables out of control after a fault occurred
[117]	Fault diagnosis	Feed water pump	Vibration signal	Initial imbalance, rotor misalignment, rotor axial rubbing, thrust bearing damage, bearing looseness, bearing stiffness vary, foundation resonance, coupling damage
[119]	Fault prognosis	Centrifugal pump	Vibration signal	Flow blockages and cavitation

and

Table 15. Bearing condition monitoring using support vector machine.

References	Application	Type of Equipment	Signal	Fault Type
[46]	Fault detection	Rolling bearing	Vibration signal	Bearing faults
[73]	Fault detection and diagnosis	Bearing in main coolant pump and feed water pump	Vibration signal	Inner race; outer race and ball faults
[79]	Fault diagnosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[83]	Fault diagnosis and prognosis	Rolling bearing	Vibration signal	Outer race, inner race, and ball faults
[113]	Fault diagnosis	Bearing form induction motor	Vibration signal	Ball, Inner race, and Outer race faults
[114]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, Inner race, and Outer race faults
[120]	Early fault detection	A run-to-failure test conducted by Intelligent Maintenance Systems, University of Cincinnati, USA	Vibration signal	Roller, Inner race, and Outer race faults

.

The Extreme Learning Machine Approach

In traditional neural network training algorithms, many iterations of training are involved. This parameter adjustment process is not only time-consuming, but also computationally intensive, resulting in low network training efficiency. In the implementation of the algorithm, it is easy to generate local minima and has poor applicability. To address the above problems, Huang et al. [13]. proposed the ELM neural network. In contrast to previous single-hidden-layer feedforward neural networks, the ELM randomly selects weights and biases in the hidden layer, and then calculates the weights in the output layer by a regularized linear regression method.

The number of neurons in the input, hidden and output layers are $\mathrm{n}, \mathrm{l}, \mathrm{m}$, respectively. Let the connection weights between the input layer to the hidden layer and the hidden layer to the output layer be w and v respectively:

$$\mathrm{w}=\left[\begin{array}{c}{\mathrm{w}}_1\\ {\mathrm{w}}_2\\ \cdots \\ {\mathrm{w}}_{\mathrm{l}}\end{array}\right]={\left[\begin{array}{cccc}{\mathrm{w}}_{11}& {\mathrm{w}}_{12}& \cdots & {\mathrm{w}}_{1\mathrm{n}}\\ {\mathrm{w}}_{21}& {\mathrm{w}}_{22}& \cdots & {\mathrm{w}}_{2\mathrm{n}}\\ \cdots & \cdots & \ddots & ⋮\\ {\mathrm{w}}_{\mathrm{l}1}& {\mathrm{w}}_{\mathrm{l}2}& \cdots & {\mathrm{w}}_{\ln }\end{array}\right]}_{\mathrm{l}\times \mathrm{n}}$$

(21)

$$\mathrm{v}=\left[\begin{array}{c}{\mathrm{v}}_1\\ {\mathrm{v}}_2\\ \cdots \\ {\mathrm{v}}_{\mathrm{l}}\end{array}\right]= {\left[\begin{array}{cccc}\mathrm{v}& {\mathrm{v}}_{12}& \cdots & {\mathrm{v}}_{1\mathrm{n}}\\ {\mathrm{v}}_{21}& {\mathrm{v}}_{22}& \cdots & {\mathrm{v}}_{2\mathrm{n}}\\ \cdots & \cdots & \ddots & ⋮\\ {\mathrm{v}}_{\mathrm{l}1}& {\mathrm{v}}_{\mathrm{l}2}& \cdots & {\mathrm{v}}_{\mathrm{lm}}\end{array}\right]}_{\mathrm{l}\times \mathrm{m}}$$

(22)

The hidden layer bias is given as:

$$\mathrm{b}=\left[\begin{array}{c}{\mathrm{b}}_1\\ {\mathrm{b}}_2\\ \cdots \\ {\mathrm{b}}_{\mathrm{l}}\end{array}\right]$$

(23)

The activation function is $\mathrm{g}\left(\mathrm{x}\right)$, then for an output $\mathrm{Y}$ with $\mathrm{N}$ samples can be expressed as:

$$\mathrm{Y}=\mathrm{Hv}$$

(24)

where, $\mathrm{H}$ is the hidden layer output matrix:

$$\mathrm{H}={\left[\begin{array}{cccc}\mathrm{g}\left({\mathrm{w}}_1·{\mathrm{x}}_1{+\mathrm{b}}_1\right)& \mathrm{g}\left({\mathrm{w}}_2·{\mathrm{x}}_2{+\mathrm{b}}_2\right)& \cdots & \mathrm{g}\left({\mathrm{w}}_{\mathrm{l}}·{\mathrm{x}}_1{+\mathrm{b}}_{\mathrm{l}}\right)\\ \mathrm{g}\left({\mathrm{w}}_1·{\mathrm{x}}_2{+\mathrm{b}}_1\right)& \mathrm{g}\left({\mathrm{w}}_2·{\mathrm{x}}_2{+\mathrm{b}}_2\right)& \cdots & \mathrm{g}\left({\mathrm{w}}_1·{\mathrm{x}}_2{+\mathrm{b}}_{\mathrm{l}}\right)\\ \cdots & \cdots & \ddots & ⋮\\ \mathrm{g}\left({\mathrm{w}}_1·{\mathrm{x}}_{\mathrm{N}}{+\mathrm{b}}_1\right)& \mathrm{g}\left({\mathrm{w}}_2·{\mathrm{x}}_{\mathrm{N}}{+\mathrm{b}}_2\right)& \cdots & \mathrm{g}\left({\mathrm{w}}_1·{\mathrm{x}}_{\mathrm{N}}{+\mathrm{b}}_{\mathrm{l}}\right)\end{array}\right]}_{\mathrm{N}\times \mathrm{l}}$$

(25)

The following theorem was proposed by Huang et al.:

Given a single hidden layer forward neural network (SLFN) with $\mathrm{N}$ hidden layer neurons, $\mathrm{N}$ distinct samples ${(\mathrm{x}}_{\mathrm{i}}{, \mathrm{y}}_{\mathrm{i}})$, where ${\mathrm{x}}_{\mathrm{i}}{=[\mathrm{x}}_{\mathrm{i}1}{, \mathrm{x}}_{\mathrm{i}2}, \cdots {,\mathrm{x}}_{\mathrm{in}}{]}^{\mathrm{T}}\in {\mathrm{R}}^{\mathrm{n}}$ and ${\mathrm{y}}_{\mathrm{i}}{=[\mathrm{y}}_{\mathrm{i}1}{, \mathrm{y}}_{\mathrm{i}2}, \cdots {,\mathrm{y}}_{\mathrm{im}}{]}^{\mathrm{T}}\in {\mathrm{R}}^{\mathrm{m}}$, and an infinitely differentiable function $\mathrm{g}\left(\mathrm{x}\right)$, then for any assignment ${\mathrm{w}}_{\mathrm{i}}\in {\mathrm{R}}^{\mathrm{n}}$ and ${\mathrm{b}}_{\mathrm{i}}\in \mathrm{R}$, there is a hidden layer output matrix $\mathrm{H}$ that is invertible and satisfies $\Vert \mathrm{Hv}-\mathrm{Y}\Vert =0$.

Given $\mathrm{N}$ distinct samples ${(\mathrm{x}}_{\mathrm{i}}{, \mathrm{y}}_{\mathrm{i}})$, where ${\mathrm{x}}_{\mathrm{i}}{=[\mathrm{x}}_{\mathrm{i}1}{, \mathrm{x}}_{\mathrm{i}2}, \cdots {,\mathrm{x}}_{\mathrm{in}}{]}^{\mathrm{T}}\in {\mathrm{R}}^{\mathrm{n}}$ and ${\mathrm{y}}_{\mathrm{i}}{=[\mathrm{y}}_{\mathrm{i}1}{, \mathrm{y}}_{\mathrm{i}2}, \cdots {,\mathrm{y}}_{\mathrm{im}}{]}^{\mathrm{T}}\in {\mathrm{R}}^{\mathrm{m}}$, and an infinitely differentiable function $\mathrm{g}\left(\mathrm{x}\right)$. Given any small error $\mathsf{\epsilon }(\mathsf{\epsilon } > 0)$, there exists a SLFN with $\mathrm{M} (\mathrm{M} \le \mathrm{N})$ hidden layer neurons that has a hidden layer output matrix $\mathrm{H}$ that is invertible and satisfies $\Vert \mathrm{Hv}-\mathrm{Y} \Vert < \mathsf{\epsilon }$ for any assignment ${\mathrm{w}}_{\mathrm{i}}\in {\mathrm{R}}^{\mathrm{n}}$ and ${\mathrm{b}}_{\mathrm{i}}\in \mathrm{R}$.

From the above two theorems, if the number of hidden layer neurons $\mathrm{l}$ and the number of training samples $\mathrm{N}$ are equal, then the network can approximate the samples with zero error for any $\mathrm{w}, \mathrm{b}$. Thus, training for the SLFN model is also equivalent to solving the Equation (25) for the least-and-multiply solution $\hat{\mathrm{v}}$.

$$\Vert \mathrm{H}({\hat{\mathrm{w}}}_1,\cdots ,{\hat{\mathrm{w}}}_{\mathrm{M}},{\hat{\mathrm{b}}}_1,\cdots ,{\hat{\mathrm{b}}}_{\mathrm{M}})\hat{\mathrm{v}}-\mathrm{Y}\Vert =\underset{\mathrm{v}}{min}\Vert \mathrm{H}({\mathrm{w}}_1,\cdots ,{\mathrm{w}}_{\mathrm{M}},{\mathrm{b}}_1,\cdots ,{\mathrm{b}}_{\mathrm{M}})\mathrm{v}-\mathrm{Y}\Vert$$

(26)

However, when $\mathrm{N}$ is large, an excessive number of hidden layers implies a larger computational effort, so the size of the hidden layers can be reduced to $\mathrm{M}$ so that the training error can be approximated by an arbitrarily small $\mathsf{\epsilon }$. In this case, the Moore-Penrose generalized inverse matrix ${\mathrm{H}}^{+}$ of $\mathrm{H}$ can be used to solve for Equation (26).

$$\hat{\mathrm{v}}={\mathrm{H}}^{+}\mathrm{Y}$$

(27)

The most common method currently used to solve the generalized inverse matrix ${\mathrm{H}}^{+}$ is the singular value decomposition method, regardless of whether ${\mathrm{H}}^{\mathrm{T}}\mathrm{H}$ is a singular or non-singular matrix, which is solved by the following equation.

$$\hat{\mathrm{v}}={\left({\mathrm{H}}^{\mathrm{T}}\mathrm{H}\right)}^{-1}{\mathrm{H}}^{\mathrm{T}}\mathrm{Y}$$

(28)

In contrast to traditional single hidden layer feedforward neural networks, ELM randomly selects weights and biases in the hidden layer and then calculates the weights of the output layer by a regularized linear regression method. Even though the weights of the hidden layer are randomly generated, ELM maintains the universal approximation capability of SLFN. Therefore, from the perspective of learning efficiency, ELM networks are not only simple to operate, but also learn faster and have better global search capability, which can overcome the problem of traditional neural networks falling into local optima, without generating overfitting or inappropriate learning rates, and have better generalization capability.

Compared to standard ANN and SVM algorithms, the rapid training speed and excellent learning efficiency of ELM have brought it into focus for equipment condition monitoring [62,91,122,123,124,125]. In [126], the RMS values of the voltage and current characteristics of the three-phase induction motor are used as ELM model inputs. Six external faults of the three-phase induction motor were discovered, with ELM providing faster results and reducing the computational load more effectively than SVM and MLP. In [42], the Fourier spectrum features of eight manually collected motor pump vibration signals are utilized as input to an ELM model to categorize motor pump faults such as misalignment, unbalance, and rubbing. When compared to other statistical classifiers (K-Nearest-Neighbor, SVM, etc.), the generalization and classification accuracy of the ELM were shown to be better to those of the statistical classifiers. In [127], Lan et al. extracted information from the vibration signal of a faulty hydraulic pump using three different signal processing techniques. The retrieved signal features that are most sensitive to the fault are utilized to train the classifier model. Comparing ELM, BP, and SVM classifiers revealed that ELM has benefits in terms of training speed and generalization and is easier in terms of model construction and parameter selection than the other two approaches. In the sphere of condition monitoring, the combination of ELM and signal pre-processing techniques has also garnered considerable interest. By integrating ELM with a depth wavelet autoencoder in [128], ELM was able to classify bearing faults with better accuracy. Multiple wavelet auto-encoders were utilized to create a depth wavelet autoencoder that efficiently captures the vibration signal’s features. The recorded features are then fed into an ELM model to identify various bearing faults. The experimental results demonstrated that the suggested technique achieves over 95% accuracy, which is much greater than classic ELM, wavelet neural network, BP neural network, and SVM.

The relevant references for the ELM methods discussed in this section are summarized in

Table 16. Equipment condition monitoring using extreme learning machine.

Equipment	References	Application	Type of Equipment	Signal	Fault Type
Motor	[126]	Fault classification	Induction motor	Voltage and current signal	External faults (Mechanical, environmental, and electrical faults)
Pump	[62]	Fault diagnosis	Hydraulic pump	Vibration signal	Slipper loosing and Valve plate wear fault
	[122]	Fault diagnosis	Hydraulic pump	Sound signal	Single slipper wear, single slipper loose, swash plate wear, and combined faults
	[91]	Fault diagnosis	Hydraulic pump	Vibration signal	Single slipper wear, single slipper loose, and center spring wear faults
	[42]	Fault classification	Motor pump	Vibration signal	Misalignment unbalance rubbing accelerometer fault
	[127]	Fault detection	Hydraulic pump	Vibration signal	Slipper abrasion
Bearing	[123]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[124]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[125]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[128]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults

.

2.2.2. Method Based on Deep Learning

As an extension and evolution of machine learning, deep learning is based on neural networks but is distinct from them. It solves the inherent problems of traditional neural networks by learning the data representation layer by layer through a multi-hidden layer network structure. It has been widely implemented in the fields of image processing and text recognition, but industrial equipment condition monitoring is still in the research phase. Deep belief networks (DBN), convolutional neural networks (CNN), and recurrent neural networks (RNN) are the most common deep learning models at present.

The Deep Belief Network Approach

As shown in Figure 5, a DBN is multi-hidden layer probabilistic generative model consisting of multiple restricted Boltzmann machines (RBM) and one output layer (usually a classification layer) combined to build a joint distribution between observed data and labels by training layer by layer. Unlike the directed/unidirectional connectivity of the hidden layers in DBNs, RBMs are multi-hidden Boltzmann networks in which the hidden layers both convey information and allow for top-down feedback adjustment.

The RBM can derive potential feature representations from the training data distribution. It is assumed that the activation conditions of each hidden unit are independent for a given input data again, and conversely, the activation conditions of the visible units are independent for a given hidden unit state. The model structure of the RBM consists of the weight matrix $\mathrm{W}$ of size $\mathrm{m} \times \mathrm{n}$, each element ${\mathrm{w}}_{\mathrm{ij}}$ in this matrix is associated with the corresponding visible layer neuron state vector ${\mathrm{v}}_{\mathrm{i}}$ and hidden layer neuron state vector ${\mathrm{h}}_{\mathrm{j}}$. Also, each layer has a corresponding offset coefficient ${\mathrm{a}}_{\mathrm{i}}$ (for visible units) and ${\mathrm{b}}_{\mathrm{j}}$ (for hidden units). When given the model parameters $\mathsf{\theta }=[\mathsf{\omega }, \mathrm{a}, \mathrm{b}]$, the energy function of the RBM can be written as:

$$\mathrm{E}\left(\mathrm{v}, \mathrm{h}; \mathsf{\theta }\right)=-{\displaystyle \sum }_{\mathrm{i}}^{\mathrm{m}}{\displaystyle \sum }_{\mathrm{j}}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{ij}}{\mathrm{v}}_{\mathrm{i}}{\mathrm{h}}_{\mathrm{j}}-{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{a}}_{\mathrm{i}}{\mathrm{v}}_{\mathrm{i}}-{\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{n}}{\mathrm{b}}_{\mathrm{j}}{\mathrm{h}}_{\mathrm{j}}$$

(29)

where $\mathrm{m}$ and $\mathrm{n}$ represent the number of visible units and hidden units respectively.

The joint distribution of all units can be obtained from the calculation of the energy function $\mathrm{E}$:

$$\mathrm{p}\left(\mathrm{v},\mathrm{h};\mathsf{\theta }\right)=\frac{\exp \left(-\mathrm{E}\left(\mathrm{v},\mathrm{h};\mathsf{\theta }\right)\right)}{\mathrm{Z}}$$

(30)

where, $\mathrm{Z}={{\displaystyle \sum }}_{\mathrm{h},\mathrm{v}}\exp (-\mathrm{E}(\mathrm{v},\mathrm{h};\mathsf{\theta }\left)\right)$ is the normalization factor.

From the above it follows that the conditional probability of a hidden layer neuron being activated is:

$$\mathrm{p}\left({\mathrm{h}}_{\mathrm{j}}|\mathrm{v};\mathsf{\theta }\right)=\mathsf{\delta }\left({\mathrm{b}}_{\mathrm{j}}+{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{m}}{\mathrm{w}}_{\mathrm{ij}}{\mathrm{v}}_{\mathrm{i}}\right)$$

(31)

Since it is a bidirectional connection, the explicit layer neuron can be equally activated by the implicit layer neuron with the conditional probability of:

$$\mathrm{p}\left({\mathrm{v}}_{\mathrm{i}}|\mathrm{h};\mathsf{\theta }\right)=\mathsf{\delta }\left({\mathrm{a}}_{\mathrm{i}}+{\displaystyle \sum }_{\begin{array}{c}\mathrm{i}\\ \mathrm{j}=1\end{array}}^{\mathrm{n}}{\mathrm{w}}_{\mathrm{ij}}{\mathrm{h}}_{\mathrm{j}}\right)$$

(32)

A deep belief network can be formed by stacking multiple RBMs, as shown in Figure 5b, where the outputs of layer $\mathrm{i}$ (the hidden layer) are used as inputs to layer $\mathrm{i}+1$ (the visible layer). The joint distribution between the observed data and the labels is established by training layer by layer, the DBN can learn a deep representation of the training data.

Tamilselvan et al. [129] introduced DBN to the field of defect diagnostics after the deep learning theory was developed. First, a pre-set health status is created, followed by the pre-processing of the acquired sensor data. The health states were then categorized using a DBN classifier, and the method’s validity was validated using two datasets, a power transformer and an aviation engine. In various fields, such as motors, pumps, and bearings, study and application of DBN-based condition monitoring techniques are expanding [130,131]. In [132], a DBN-based fault diagnostic approach for induction motors is proposed. The motor vibration signal is used as input for feature learning by the DBN classifier. The acquired features are then utilized to train a BP neural network for fault identification. The results were verified using time-domain analysis and wavelet transform, which revealed that the DBN-learned features attained the maximum classification accuracy of 95.8%. Wang et al. [133] implemented a DBN for axial piston pump fault diagnosis. First, the time-domain, frequency-domain, and time-frequency-domain data indicators of the original vibration signals of the faults were computed. The data metrics are then used as training samples and input to the DBN, which is trained layer by layer to achieve classification and detection of multiple faults. By comparison with SVM and ANN, it was determined that the fault features automatically learnt by DBN could reach a classification accuracy of 97.4%. In [134], DBN is used to improve rolling bearing fault diagnosis. The DBN classifier was trained and evaluated using time-domain characteristics of vibration signals from multiple kinds of faults. The results indicate that a diagnostic accuracy of 97.5% is achieved using the multi-signal fusion technique, as opposed to employing a single fault signal. Moreover, compared to SVM, KNN, and BP neural networks, DBN has the highest average classification accuracy at 93.17%. In the present day, DBNs have been successfully used to fault classification and diagnostic problems [135,136], and in recent years, DBNs have been utilized by combining them with other approaches. For example, in [137], Shao et al. proposed a combination of adaptive DBNs and dual-tree complex wavelet packets (DTCWPT). DTCWPT was used to decompose the original bearing vibration signal into eight distinct frequency band components, from which nine statistical features were recovered to generate a feature set. A 5-layer adaptive DBN is then trained these features to classify faults. Comparing the method to other traditional machine learning methods using the same data revealed that the method had the highest average accuracy, 94.38%. Chen and Li [138] also propose a method for fault diagnosis of bearings using a combination of sparse autoencoder (SAE) network and DBN. First, statistical features are derived from the time and frequency domains of various vibration signals. Multiple two-level sparse self-encoders (SAEs) are then fed these features to learn higher-level features. Based on the fused feature vectors, DBN is then utilized to identify bearing faults. Recent research has begun to improve the diagnostic and predictive performance of DBN models by optimizing parameters and adjusting internal structure of the model [139,140,141,142]. Li et al. [143] proposed a Bispectrum entropy and DBN-based method for enhancing the performance of fault prognosis for hydraulic pumps. First, a Bispectrum analysis of the pump vibration signal was carried out to extract 15 Bispectrum entropies in distinct frequency bands as predictive features. The normalized features are then utilized as training data for a 3-layer DBN network to learn features. Using the quantum particle swarm optimization (QPSO) approach, the DBN model parameters are optimized to increase prediction accuracy. In conclusion, a comparison with conventional approaches (SVM and ANN models) demonstrates that the method can reliably anticipate trends and random fluctuations in hydraulic pump performance deterioration with good generality and prediction accuracy. To increase the accuracy of continuous data modelling, Yu et al. [144] proposed a DBN model consisting of improved condition restrict Boltzmann Machines (ICRBMs) to predict the remaining useful life of a hydraulic pump. Adding timing linkage factors and constraint variables to nodes on the same layer enhanced the original RBM. Following Bispectral analysis, the normalized data of the vibration signal’s features were utilized for the training and testing of the DBN model. Comparative experiments show that the improved DBN model can achieves more prediction accuracy than the original DBN, BP neural network, and SVM models.

The relevant references for the DBN methods discussed in this section are summarized in

Table 17. Equipment condition monitoring using deep belief network.

Equipment	References	Application	Type of Equipment	Signal	Fault Type
Motor	[130]	Fault diagnosis	Induction motor	Vibration signal	Stator winding defect, unbalanced rotor, defective bearing, broken bar, and bowed rotor
	[132]	Feature extraction	Induction motor	Vibration signal	Broken bar, broken rotor, defective bearing, stator winding defect and unbalanced rotor
	[135]	Fault diagnosis	Traction motor	Vibration signal	Bearing fault
Pump	[133]	Fault diagnosis	Axial piston pump	Vibration signal	Bearing fault, wear in three pistons, blocked support hole in static pressure slippers, wear in shaft shoulder, and cylinder block with a pitting defect
	[143]	Fault prognosis	Hydraulic pump	Vibration signal	Loose slipper
	[144]	Remaining useful life	Hydraulic pump	Vibration signal	Loose slipper
Bearing	[134]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[136]	Fault detection	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[137]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[138]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race and outer race
	[139]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
	[140]	Fault diagnosis	Electric locomotive bearing	Vibration signal	Outer race, inner race, roller, and compound faults
	[141]	Fault diagnosis	Rolling bearing	Vibration signal	Inner ring faults, outer ring faults, rolling element faults, rotor imbalance faults, and the coupling of these faults
	[142]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults

.

The Convolutional Neural Network Approach

A convolutional neural network is a typical supervised feed-forward neural network whose training goal is to learn abstract features by alternating and superimposing convolutional kernels and pooling operations. It is structured with an input layer, a convolutional layer, a pooling layer kernel, a fully connected layer, and an output layer.

The CNN model is a supervised learning model that requires learning with sample labels. Therefore, the input consists of sample X and the corresponding label Y. For example, for a classification problem, the inputs to the model are as follows:

$$\left\{\mathrm{X},\mathrm{Y}\right\}= {\left\{{\mathrm{x}}_{\mathrm{i}}{,\mathrm{y}}_{\mathrm{i}}\right\}}^{\mathrm{N}}$$

(33)

where $\mathrm{N}$ denotes the number of samples, ${\mathrm{x}}_{\mathrm{i}}$ denotes the ${\mathrm{i}}^{\mathrm{th}}$ input sample and ${\mathrm{y}}_{\mathrm{i}}$ denotes the category label corresponding to the ${\mathrm{i}}^{\mathrm{th}}$ input sample.

The convolutional layer is the core component of the CNN model. The idea of local connectivity and weight sharing is achieved by means of convolutional kernels, which slide vertically along the coordinate transverse kernels of the input feature map and perform convolutional operations with the data in the receptive field, thereby extracting the structural features hidden inside the data. The convolutional layers are organized according to three dimensions: depth, width, and height, with width and height referring to the width and height of the convolutional kernel respectively, i.e., the size of the local receptive field. Depth is the number of convolution kernels. To extract different features on the input feature map, the convolution layer performs a convolution operation by a certain number of convolution kernels, each with a different weight, corresponding to a feature extraction. The convolution operation extracts feature from the input feature map based on the convolution kernel size, and shift compensation. The process of feature extraction by convolution kernels is described as follows:

$$\begin{array}{c}{\mathrm{fea}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}}={{\displaystyle \sum }}_{\mathrm{q}=1}^{\mathrm{Q}}{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{J}}{\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}}{*\mathrm{fea}}_{\mathrm{M},\mathrm{q}}^{\mathrm{L}-1}{+\mathrm{b}}_{\mathrm{j}}^{\mathrm{L}}\\ \mathrm{i}=⎣\frac{\mathrm{M}+2\mathrm{p}-\mathrm{k}}{\mathrm{s}}⎦+1\end{array}$$

(34)

where ${\mathrm{w}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}}$ and ${\mathrm{b}}_{\mathrm{j}}^{\mathrm{L}}$ denote the weight and bias of the ${\mathrm{j}}^{\mathrm{th}}$ convolutional kernel in the ${\mathrm{L}}^{\mathrm{th}}$ convolutional layer, respectively, $\mathrm{J}$ denotes the number of convolutional kernels, $\mathrm{s}$ is the distance that convolutional kernel slides over the input feature map (step size), $\mathrm{p}$ denotes the fill size, ${\mathrm{fea}}_{\mathrm{M},\mathrm{q}}^{\mathrm{L}-1}$ represents the ${\mathrm{q}}^{\mathrm{th}}$ feature map of size $\mathrm{M}$ at the output of the ${(\mathrm{L}-1)}^{\mathrm{th} }$ layer, * denotes the discrete convolution operator, and ${\mathrm{fea}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}}$ is the ${\mathrm{j}}^{\mathrm{th}}$ convolutional kernel in the ${\mathrm{L}}^{\mathrm{th}}$ convolutional layer extracted over the output feature map to generate feature map of size $\mathrm{i}$. Each convolution kernel finds a specific feature at each position in the feature map, and the type of feature learned is determined dynamically by the algorithm.

The pooling layer Is the network layer that Implements the pooling operation. After the features are extracted from the convolutional layer, direct use for classification can lead to excessive computational effort and thus overfitting. Therefore, pooling of the feature map is required to reduce the data dimensionality. The pooling operation is a process of further abstraction of information, similar to the process of extracting features from a convolutional layer, by sliding a sliding window over the feature map, taking the statistical values of the local region corresponding to the sliding window as the sampled values of that region, and then concatenating the values extracted from these local regions to form a new feature map. The pooling operation preserves significant feature information while reducing dimensionality, and its operation can be expressed as:

$$\begin{array}{c}{\mathrm{fea}}_{{\mathrm{i}}^{\prime },\mathrm{j}}^{\mathrm{L}}=\mathrm{f}\left({\mathrm{fea}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}-1}\right)\\ {\mathrm{i}}^{\prime }=\left(0,1,2,\cdots ,⎣\frac{\mathrm{i}-\mathrm{d}}{\mathrm{s}}⎦\right)\end{array}$$

(35)

where ${\mathrm{fea}}_{\mathrm{i},\mathrm{j}}^{\mathrm{L}-1}$ denotes the ${\mathrm{j}}^{\mathrm{th}}$ feature map of size $\mathrm{i}$ input at layer $\mathrm{L}$, $\mathrm{f}$ represents the method of pooling processing. And $\mathrm{d}$ denotes the pooling function size, when $\mathrm{d} < \mathrm{i}$, it denotes local pooling of the feature map, and when $\mathrm{d}=\mathrm{i}$, it represents pooling of the entire feature map. $\mathrm{s}$ denotes the pooling function move step, and ${\mathrm{fea}}_{{\mathrm{i}}^{\prime },\mathrm{j}}^{\mathrm{L}}$ is its corresponding output feature map of size ${\mathrm{i}}^{\prime }$ after the pooling operation.

A fully connected layer is used to integrate local features extracted from a convolutional or pooling layer, and it connects each neuron in the previous layer to each neuron in the next layer, thus a fully connected structure. For a one-dimensional input $\mathrm{x}$ of length $\mathrm{M}$, the fully connected layer has $\mathrm{N}$ neurons, and the output of each neuron can be expressed as:

$${\mathrm{z}}_{\mathrm{j}}^{\mathrm{L}}=\mathsf{\phi }\left({\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{M}}{\mathrm{z}}_{\mathrm{i}}^{\mathrm{L}-1}{\mathrm{w}}_{\mathrm{j},\mathrm{i}}^{\mathrm{L}}{+\mathrm{b}}_{\mathrm{j}}^{\mathrm{L}}\right), \mathrm{j}=1,2,\cdots ,\mathrm{N}$$

(36)

where ${\mathrm{w}}_{\mathrm{j},\mathrm{i}}^{\mathrm{L}}$ is the connection weight of the ${\mathrm{i}}^{\mathrm{th}}$ neuron in layer $\mathrm{L}-1$ to the ${\mathrm{j}}^{\mathrm{th}}$ neuron in layer $\mathrm{L}$, ${\mathrm{b}}_{\mathrm{j}}^{\mathrm{L}}$ denotes the bias of the ${\mathrm{j}}^{\mathrm{th}}$ neuron in layer $\mathrm{L}$, ${\mathrm{z}}_{\mathrm{i}}^{\mathrm{L}-1}$ and ${\mathrm{z}}_{\mathrm{j}}^{\mathrm{L}}$ are the input and output of the ${\mathrm{j}}^{\mathrm{th}}$ neuron in layer $\mathrm{L}$, $\mathrm{M}$ and $\mathrm{N}$ denote the number of neurons in layers $\mathrm{L}-1$ and $\mathrm{L}$, respectively, and $\mathsf{\phi }$ is the non-linear activation function. The output layer outputs the recognition results of the model in the form of categories or probabilities through the classifier and calculates the difference between the actual output and the ideal output. The error back propagation algorithm then passes the error layer by layer and finally the parameters of each layer are updated using gradient descent.

Initially, CNNs were mostly employed for two-dimensional image processing. However, because to its powerful feature learning and pattern recognition capabilities, it has been recently used to equipment condition monitoring to enhance the analysis and diagnosis of non-linear, multi-source, high-dimensional data. Depending on the size of the convolutional kernel, its condition monitoring applications can be divided into one-dimensional convolutional models and two-dimensional convolutional models.

CNNs for condition monitoring were first primarily based on a two-dimensional convolutional strategy [63,145,146,147,148,149,150]. Wen et al. [151] constructed a unique LeNet-5 based CNN for fault diagnostics. The raw 1-dimensional device signal data is transformed into a 2-dimensional picture input, avoiding the need to manually choose features, and allowing the CNN to classify the 2-dimensional images directly. The approach was verified on rolling bearing, centrifugal pump, and hydraulic pump datasets, and the results indicated that the achieved diagnostic accuracy are 99.79%, 99.481%, and 100%, respectively. Ding et al. [64] combined wavelet packets with image-space reconstruction to reconstruct wavelet packet energy (WPE) information in the frequency domain. The feature map is then fed into a deep convolutional network (ConvNet) to learn distinguishable features, and ConvNet will directly link the final convolutional layer as input to the multiscale layer to keep global and local information. Experimentally, the strategy is shown to improve classification accuracy for metaclassifier main bearing fault diagnosis feature clustering. Wang et al. [38] utilized the short-time Fourier transform to transform a one-dimensional vibration signal into a time-frequency domain picture. A CNN model comprised of four convolutional layers, two pooling layers, and two fully connected layers was developed for image classification-based health monitoring of asynchronous motor operating conditions. In fact, CNNs have powerful signal processing and analysis capabilities. If a one-dimensional signal is utilized directly as its input, it is possible to merge classical signal processing and feature extraction techniques to accomplish the most direct fault diagnosis and optimize the condition monitoring process. To incorporate a one-dimensional signal as input in a CNN, the CNN network topology must be modified, and a one-dimensional convolution kernel must be used. Many one-dimensional CNN models have been developed and implemented in equipment condition monitoring at present [152,153,154,155,156,157,158,159]. Junior et al. [160] employed vibration data derived from multiple sensors as input to a multi-headed one-dimensional CNN model to diagnose faults in induction motors. The one-dimensional CNN in each head consists of batch normalization, two convolutional layers, two pooling layers, a fully connected layer, and a softmax layer, with each output corresponding to a motor operating condition. The approach produces a diagnosis accuracy of 99.92% through experimentation and parameter optimization, and the network is quick to train and test. Wang et al. [161] used raw vibration signals of each fault pattern to train a one-dimensional CNN model to detect and recognize data features. The acquired features were then utilized to train an HMM classifier for fault diagnosis. The classification results were compared with CNN, SVM and BP neural networks and showed that the model gave accurate classification for different bearing datasets. Liu et al. [162] suggested a multi-scale kernel residual CNN (MK-ResCNN) for diagnosing motor faults. The vibration signals from various motor running states were segmented into fragment samples to build the training and test sets of the CNN model, enabling the distinction and classification of fault features. Residual learning is included into the multi-scale kernel CNN to prevent degradation problems in deep networks. The approach eliminates the requirement for converted data, and the results demonstrate that reliable classification of noisy fault data can be performed under non-stationary operating conditions by evaluating a normal motor and five faulty motors.

The relevant references for the CNN methods discussed in this section are summarized in

Table 18. Motor condition monitoring using convolutional neural network.

References	Application	Type of Equipment	Signal	Fault Type
[38]	Fault diagnosis	Asynchronous motor	Vibration signal	Built-in rotor imbalance, stator winding faults, built-in faulted bearing, built-in bowed rotor, built-in broken rotor bars, and voltage imbalance and single phasing
[152]	Fault detection	Induction motor	Current signal	Bearing fault
[153]	Fauld diagnosis	Permanent magnet synchronous motor	Current signal	Demagnetization fault and bearing fault
[160]	Fault detection and diagnosis	Induction motor	Vibration signal	Bent shaft, broken bar, misalignment, mechanical looseness, bearing fault and unbalance
[162]	Fault diagnosis	Induction motor	Vibration signal	Bowed rotor, broken rotor bar, faulty bearing, high impedance and, unbalance rotor

,

Table 19. Pump condition monitoring using convolutional neural network.

References	Application	Type of Equipment	Signal	Fault Type
[36]	Fault diagnosis	Axial piston pump	Vibration signal	Cavitation (different severity)
[55]	Fault diagnosis	Hydraulic pump	Vibration, pressure, and sound signal	Swash plate wear, Loose slipper, Slipper wear, Central spring wear
[150]	Fault pattern recognition	Water pump	Vibration signal	Bearing wear, and rotor eccentricity faults
[156]	Fault diagnosis	Centrifugal pump	Vibration signal	Cavitation, impeller unbalance, and shaft misalignment
[157]	Condition monitoring	Hydraulic pump	Vibration signal	High temperature influence on the volumetric efficiency
[158]	Fault diagnosis	Hydraulic pump	Vibration signal	Slipper failure, loose slipper, swash plate wear, and central spring wear
[159]	Fault diagnosis	Hydraulic pump	Pressure signal	Swash plate wear, loose slipper failure, slipper wear, and central spring wear

and

Table 20. Bearing condition monitoring using convolutional neural network.

References	Application	Type of Equipment	Signal	Fault Type
[47]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race and outer race faults
[54]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
[63]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and roller faults
[64]	Fault diagnosis	Spindle bearing	Vibration signal	Inner race, outer race, and ball faults
[145]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[146]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[147]	Fault diagnosis	Plant bearing	Vibration signal	Inner race, and outer race faults
[148]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[149]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[154]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[155]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race; outer race, and roller faults
[161]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults

.

The Recurrent Neural Network Approach

Recurrent neural networks (RNN) are a framework for processing sequential data, remembering the previous information through the connection structure between each layer and using this information to influence the output of later nodes. RNNs can fully exploit the temporal and semantic information in sequential data, and this approach is more capable of deeper representation than full connected neural networks and CNNs in processing temporal data. A typical recurrent neural network structure is shown in Figure 6. For ease of understanding, the RNN structure in Figure 6 is expanded dutifully in a time series as shown in Figure 7. It is worth noting that the nodes in the recurrent layer of the RNN represent a hidden layer state rather than a neuron.

In Figure 7, $\mathrm{x}$ represents the inputs, $\mathrm{h}$ represents the hidden layer output, and $\mathrm{y}$ represents the output layer output. The right-hand side of Figure 7 is the form of an RNN expanded on the time axis, where the input is a time series $\left\{\cdots {,\mathrm{x}}_{\mathrm{t}-1}{,\mathrm{x}}_{\mathrm{t}}{,\mathrm{x}}_{\mathrm{t}+1},\cdots \right\}$, where ${\mathrm{x}}_{\mathrm{t}}\in {\mathrm{R}}^{\mathrm{n}}$ and $\mathrm{n}$ is the number of neurons in the input layer. Correspondingly, the hidden layer is $\left\{\cdots {,\mathrm{h}}_{\mathrm{t}-1}{,\mathrm{h},\mathrm{h}}_{\mathrm{t}+1},\cdots \right\}$, where ${\mathrm{h}}_{\mathrm{t}}\in {\mathrm{R}}^{\mathrm{m}}$ and $\mathrm{m}$ is the number of neurons in the hidden layer. The output value $\mathrm{y}$ at the moment $\mathrm{t}$ is not only influenced by the input $\mathrm{x}$ at the same moment, but also by the state of the hidden layer $\mathrm{s}$ at the previous moment. Such feedback is called ‘memory’ or ‘state’ $\mathrm{s}$ in RNN and is at the heart of it. This leads to the following computational process for the RNN model. The hidden layer calculation formula can be expressed as:

$${\mathrm{h}}_{\mathrm{t}}{=\mathrm{f}}_{\mathrm{h}}\left({\mathrm{s}}_{\mathrm{t}}\right)$$

(37)

where,

$${\mathrm{s}}_{\mathrm{t}}{=\mathrm{Wx}}_{\mathrm{t}}{+\mathrm{Uh}}_{\mathrm{t}-1}{+\mathrm{b}}_{\mathrm{h}}$$

(38)

${\mathrm{f}}_{\mathrm{h}}$ represents the activation function of the hidden layer, ${\mathrm{b}}_{\mathrm{h}}$ represents the bias of the hidden layer, $\mathrm{W}$ is the weight matrix from the input layer to the hidden layer, $\mathrm{U}$ is the weight matrix from the hidden layer outputs at time $\mathrm{t}-1$ to the hidden layer outputs at time $\mathrm{t}$, ${\mathrm{s}}_{\mathrm{t}}$ represent the state of the neuron at moments $\mathrm{t}$, and ${\mathrm{h}}_{\mathrm{t}-1}$ represents the output of the hidden layer at the previous moment.

The RNN outputs are $\left\{\cdots {,\mathrm{y}}_{\mathrm{t}-1}{,\mathrm{y},\mathrm{y}}_{\mathrm{t}+1},\cdots \right\}$, where ${\mathrm{y}}_{\mathrm{t}}\in {\mathrm{R}}^{\mathrm{p}}$ with $\mathrm{p}$ being the number of neurons in the output layer. The output layer is a fully connected layer, which means that each neuron in the hidden layer is connected to all the neurons in the output layer, and the output layer expression is:

$${\mathrm{y}}_{\mathrm{t}}{=\mathrm{f}}_{\mathrm{o}}\left({\mathrm{Vh}}_{\mathrm{t}}{+\mathrm{b}}_{\mathrm{o}}\right)$$

(39)

where $\mathrm{V}$ is the weight matrix from the hidden layer to the output layer, ${\mathrm{f}}_{\mathrm{o}}$ and ${\mathrm{b}}_{\mathrm{o}}$ represent the output layer activation function and bias respectively.

As with traditional neural networks, the training of the network pattern parameters is based on an error back-propagation algorithm. The difference is that the parameters of the recurrent neurons in the RNN are shared from moment to moment, and thus the calculation of the gradient relies on the gradient of all past moments. For this reason, the error back propagation algorithm in RNNs is also known as the back propagation through time (BPTT). In the following, the cross-entropy is considered as the loss function, the $\tan \mathrm{h}$ function is the hidden layer activation function and the final output of the RNN is $\mathrm{y}$, the state and output at time $\mathrm{t}$ are:

$${\mathrm{h}}_{\mathrm{t}}=\tan \mathrm{h}\left({\mathrm{s}}_{\mathrm{t}}\right)=\tan \mathrm{h}\left(\mathrm{W}\right)$$

(40)

$${\hat{\mathrm{y}}}_{\mathrm{t}}=\mathrm{softmax}\left(\mathrm{V}{\mathrm{s}}_{\mathrm{t}}\right)$$

(41)

For sequence training the loss function is:

$$\mathrm{E}(\mathrm{y},\hat{\mathrm{y}})=\sum _{\mathrm{t}=0}^{\mathrm{T}}{\mathrm{E}}_{\mathrm{t}}\left({\mathrm{y}}_{\mathrm{t}},{\hat{\mathrm{y}}}_{\mathrm{t}}\right)=-\sum _{\mathrm{t}=0}^{\mathrm{T}}{\mathrm{y}}_{\mathrm{t}}\log {\hat{\mathrm{y}}}_{\mathrm{t}}$$

(42)

where ${\mathrm{y}}_{\mathrm{t}}$ and ${\hat{\mathrm{y}}}_{\mathrm{t}}$ are the true output value and the model prediction at moment $\mathrm{t}$ respectively, is and ${\mathrm{E}}_{\mathrm{t}}$ is the cross-entropy at moment $\mathrm{t}$.

In recent years, there has been increasing interest in the use of RNNs for equipment condition monitoring [163,164,165,166,167]. Abed et al. [168] have implemented RNN successfully to the detection and classification of motor bearing faults. Using discrete wavelet transforms, fault features are extracted from the current and vibration signals of induction motors. Using orthogonal fuzzy neighborhood discriminant analysis (OFDNA), these characteristics are decreased further. OFDNA is then applied to acquire the most effective fault classification features. The RNN classifier uses these features as training and test sets to provide fault detection and diagnosis. The results shows that the RNN-based fault diagnosis approach can detect and classify induction motor bearing faults under various operating conditions. Li et al. predict the deterioration trend of rolling bearings by integrating RNN and reinforcement learning in [169]. The singular spectral entropy of the vibration signal is utilized to characterize the deterioration condition of the bearing, and the degradation trend is split into several stages by moving average noise reduction. The training and testing set for the RNN model is comprised of these phases. Then, reinforcement learning is employed at each stage to optimize the parameters of the RNN’s hidden layer. By comparing RNN with reinforcement learning to ELM, SVM, and the original RNN model, the results indicate that RNN with reinforcement learning surpasses the other approaches in terms of prediction accuracy and convergence speed.

As seen in the previous paragraph, RNNs have already made progress in the application of equipment condition monitoring. In practice, however, RNNs still have limits, most notably the disappearance or expansion of gradients when dealing with lengthy sequence issues. To solve this problem, a long short-term memory (LSTM) network structure containing memory units is proposed in [170]. A typical LSTM neural network cell is shown in Figure 8. Both LSTM and RNN are trained with similar parameters on the neural network structure, with the main difference being on the recurrent neuron nodes.

In Figure 8, ${\mathrm{h}}_{\mathrm{t}}$ is the hidden state, which represents short-term memory, and ${\mathrm{C}}_{\mathrm{t}}$ is the cell state, which represents long-term memory. Compared to the RNN basic recurrent neurons, the LSTM has an additional state or memory for long-term storage and another for short-term storage. The long-term state is updated slowly and mostly maintains long-term dependent information, whereas the short-term state is updated more often and varies more depending on the neuron’s current state. The LSTM additionally has three gated cyclic units to regulate the updating and forgetting of memories, ensuring that vital information is constantly remembered, and less significant information is discarded, hence permitting the storage and flow of memories in the hidden layer units.

In the recent decade, LSTM has received a great deal of interest in the field of equipment condition monitoring since it overcomes the limitations of RNNs in processing long sequences [81,171,172,173,174,175,176,177,178,179,180,181,182]. Regarding fault detection and diagnosis, Luo et al. [183] implemented fault detection for permanent magnet synchronous motor using LSTM. To train the LSTM model, the three-phase current values and rotor position data at the moment of continuous sampling are utilized as inputs, and the current value at the next sampling moment is used as the desired output. Detection of faults is then accomplished by monitoring the model’s prediction error. In [184], Xiao et al. propose a LSTM-based approach for fault diagnosis in three-phase asynchronous motors. Each type of vibration signal is first converted into a three-dimensional tensor sample matching to the current condition and utilized as input to the LSTM classifier to establish the link between the vibration signal and the fault condition. After testing the LSTM, linear regression (LR), SVM, multilayer neural network, and original RNN on six asynchronous motors with various failure circumstances, the LSTM demonstrated the highest classification accuracy at 98.28%. In [92], Bie et al. presented a model based on complete ensemble empirical model decomposition (CEEMD) and LSTM for fault diagnostics of reciprocating pumps. The vibration signal is processed using CEEMD and singular spectral entropy to generate feature vectors for fault diagnostics using an LSTM classifier. In comparison to traditional neural network approaches, the LSTM has the best classification accuracy. Regarding fault prognosis and remaining useful life (RUL) estimation, Lee et al. [185] utilized LSTM to estimate the RUL for gear pumps. Using the temporal and frequency domain features of the vibration signal obtained by signal analysis, the health index of the pump was characterized. The RUL estimate was then done using a Kalman filter to combine the health index and pressure data. Through training in a bi-directional LSTM model, it is able to predict the future of the RUL. Gate recurrent units (GRUs) were introduced by Cho et al. [186] due to the varying contributions of various gating to learning ability. By removing the gates with minor contributions and their related weights, the model simplifies the structure of the LSTM network and enhances the efficiency of learning. It consists of two gating units, the reset gate and the update gate. This method has also been used to the monitoring of equipment condition [187,188,189]. The reset gate acts similarly to the input gate of the LSTM unit, but the update gate implements the forgetting gate and output gate. Zhao et al. [190] proposed a hybrid strategy for machine condition monitoring that combines manual feature creation with automatic feature learning. First, time domain features are retrieved manually from the device signals and utilized as input to an improved bidirectional GRU network. This is followed by learning to represent faults using the GRU network’s produced local features. A supervised learning layer is then employed to predict the machine condition. The research demonstrates the efficacy and generalizability of the system through the prediction of tool wear, the diagnosis of gearbox faults, and the detection of early bearing faults.

The relevant references for the RNN methods discussed in this section are summarized in

Table 21. Motor condition monitoring using recurrent neural network.

References	Application	Type of Equipment	Signal	Fault Type
[163]	Fault detection and isolation	Induction motor	Current signal	Stator and bearing fault
[168]	Fault detection and diagnosis	Brushless DC motor	Current and vibration signal	Bearing fault (Ball, inner race, and outer race faults)
[171]	Fault diagnosis	Induction motor	Rotating sound signal	Bearing fault (Ball, inner race, and outer race faults)
[172]	Fault diagnosis	Induction motor	Vibration and current signal	Unbalance fault with different severity
[183]	Fault detection	Permanent Magnet Synchronous Motor	Three-phase current signal and rotor position information	open and shot circuit fault (instantaneous fault or gradual fault), and winding resistance increase or decrease (early faults)
[184]	Fault diagnosis	Three-phase asynchronous motor	Vibration signal	Voltage imbalance, rotor imbalance, faulty bearing, broken rotor bars, and bowed rotor
[186]	Remaining useful life prediction	Induction motor	Current and voltage signal	Bearing fault

,

Table 22. Pump condition monitoring using recurrent neural network.

References	Application	Type of Equipment	Signal	Fault Type
[164]	Modelling	Heat pump	Temperature	Clogging fault
[173]	Fault prognosis	Power pump	Multiple sensors monitoring data	N/A
[174]	Remaining useful life	Electromagnetic pump	Vibration and pressure signal	Cavitation
[175]	Future state prediction	Water injection pump	Multiple sensors monitoring data (vibration, temperature, flow, pressure, distance)	N/A
[92]	Fault diagnosis	Reciprocating pump	Vibration signal	Piston wear, bearing wear, and valve disc wear faults
[185]	Remaining useful life	Hydraulic gear pump	Vibration, flow, and pressure signal	Fuel contamination
[188]	State trend prediction	Aircraft pump	Accumulator pressure data	N/A
[81]	Fault prognosis	Reactor coolant pump	Shaft seal leakage flow	Seal leakage fault
[189]	Condition prediction	Main pump	Temperature and leakage flow	Bearing wear fault

and

Table 23. Bearing condition monitoring using recurrent neural network.

References	Application	Type of Equipment	Signal	Fault Type
[34]	Feature extraction	Rolling bearing	Vibration signal	Inner race, outer race, cage and ball faults
[165]	Fault prognosis	Rolling bearing	Vibration signal	Outer race fault
[169]	State trend prediction	Rolling bearing	Vibration signal	Inner race fault
[176]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
[177]	Fault diagnosis	Rolling bearing	Vibration signal	Ball, inner race, and outer race faults
[178]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race and roller fault and combination of outer race and roller faults
[179]	Condition monitoring	Rolling bearing	Vibration signal	roller, inner race, and outer race fault
[180]	Fault prediction	Aero engine bearing	Vibration signal	Inner race, outer race, and ball faults
[181]	Remaining useful life	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[187]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[190]	Prognosis and remaining useful life prediction	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[166]	Fault detection	Motor bearing	Current and vibration signal	Air gap eccentricity and ball faults
[167]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults
[182]	Fault diagnosis	Rolling bearing	Vibration signal	Inner race, outer race, and ball faults

.

3. Conclusions

In this paper, we review the progress of the last 10 years of research into data-driven condition monitoring applied to three types of commonly industrial equipment: motors, pumps, and bearings. The structure and applications of data-driven condition monitoring are investigated. The large number of reviewed works indicate that data-driven condition monitoring of industrial equipment is gaining popularity. The reported data-driven equipment condition monitoring approaches generally involve a feature extraction phase and fault classification/prediction phase. The most widely used feature extraction techniques include Fourier Transform (FT), Wavelet Transform (WT), and Empirical Modal Decomposition (EMD). The most widely used classification and prediction techniques include artificial neural networks (ANN), support vector machines (SVM) and extreme learning machines (ELM), deep belief networks (DBN), convolutional neural networks (CNN) and recurrent neural networks (RNN).

As a result of the literature assessment shown above, the following list of unsolved critical future challenges is presented.

• Literature has focused on various types of faults, including rotor, stator winding, bearing wear, unbalance faults, etc., in motors; cavitation, leakage, impeller faults, etc., in pumps; and inner race, outer race, ball, and roller faults in bearings. These faults are often assessed individually in condition monitoring, however, multiple faults can exist in a single component, so it is necessary to consider this situation carefully and to achieve differentiation and resolution of multiple faults.
• As different types of equipment may have faults of varying severity in their operating condition, it is essential to consider the state of development of faults to correctly diagnose and detect them at the earliest stages of their occurrence, which is very rare in research work.
• Another issue that cannot be disregarded is the imbalance of data categories, since equipment always operates under normal conditions to collect normal data, resulting in a small number of fault sample data. In addition, most AI-based monitoring systems utilize historical or current databases. However, it is impossible to have a database for all machines operating under all conditions. Therefore, it is necessary to research how to allow AI models to execute condition monitoring in the absence of training data or under particular operating conditions that have not been trained.