A Bearing Fault Diagnosis Method Using Multi-Branch Deep Neural Network

Abstract

Feature extraction from a signal is the most important step in signal-based fault diagnosis. Deep learning or deep neural network (DNN) is an effective method to extract features from signals. In this paper, a novel vibration signal-based bearing fault diagnosis method using DNN is proposed. First, the measured vibration signals are transformed into a new data form called multiple-domain image-representation. By this transformation, the task of signal-based fault diagnosis is transferred into the task of image classification. After that, a DNN with a multi-branch structure is proposed to handle the multiple-domain image representation data. The multi-branch structure of the proposed DNN helps to extract features in multiple domains simultaneously, and to lead to better feature extraction. Better feature extraction leads to a better performance of fault diagnosis. The effectiveness of the proposed method was verified via the experiments conducted with actual bearing fault signals and its comparisons with well-established published methods.

1. Introduction

Rolling element bearings are the most important components in rotary machines. The health condition of bearings has a profound effect on the performance of the machines. According to a literature review, about 40% of failure cases in large machinery systems and 90% in small rotary machines are caused by bearing defects [1]. In the industry, bearing faults can lead to massive losses of time and money; therefore, early detecting fault of bearings is a critical task in the industry.

Intelligent signal-based fault diagnosis is the most popular approach in machine health monitoring. The signal types used for diagnosing can be vibration signal [2,3], acoustic emission signal [4], or current signal [5,6]. Among those types of signals, vibration signal is exploited most extensively because vibration signal is easy to measure and can provide highly accurate information about the bearing health condition [7]. The fault diagnosis performance of the signal-based approach highly depends on the procedure of feature extraction in which discriminating features are extracted from vibration signals. After extracting the fault features from the fault signals, an intelligent decision-maker based on machine learning algorithms is exploited to determine the type of fault occurring.

Traditionally, feature extraction exploits signal processing techniques to extract information from the fault signal in the time domain, frequency domain, and time-frequency domain [8]. This traditional approach has some disadvantages, as follows. First, the diagnosing accuracy depends on the signal processing technique and requires expert knowledge [9]. Because of the expert knowledge requirement, it is difficult to propose a generalized framework for feature extraction. It means that features used to make predictions under certain circumstances cannot ensure accurate predictions under other circumstances; therefore, for each new fault diagnosis task, a new feature extractor must be redesigned manually.

Deep learning (DL) or deep neural network (DNN) is a branch of machine learning (ML) in which a neural network with many layers is exploited [10,11,12]. DL algorithms with deep structures have the ability to learn hierarchical representations directly from the input data. A huge number of DNN models had been introduced for many areas of research and application. DL has been extensively used in signal-based fault diagnosis as an effective feature learning tool [13,14,15]. DL-based fault diagnosis is an end-to-end fault diagnosis process in which the feature extractor is trainable; it can overcome the limitation of hand-crafted feature extractor in traditional signal-based fault diagnosis [16,17]. Currently, all DNN models can be considered as the extensions of five basic models: autoencoder (AE), restricted Boltzmann machine (RBM), recurrent neural network (RNN), generative adversarial network (GAN), and convolutional neural network (CNN). AE and RBM are classified as unsupervised learning methods. Multiple AE or RBM can be stacked together to construct deep models, named stacked autoencoder (SAE) and deep Boltzmann machine (DBM), respectively. SAE and DBM are often used as feature extractors, and they require additional classifiers in the classification tasks. RNN models are often applied to prognosis problems. GAN is a special case since it is categorized as a semi-supervised learning method [18]. CNN is a neural network that mimics the mammalian visual system [19]. Since the design concept of CNN possesses three key architectural ideas: local receptive fields, weight sharing, and pooling in spatial domain, CNN is suitable for the recognition of two-dimensional visual data [20,21]. As a result, CNN and its variants are extensively exploited in the image classification topic [22,23]. To apply CNN in the bearing fault diagnosis task, authors and researchers often try to represent or convert 1-D bearing signals into 2-D forms. In this way, the fault diagnosis task is transformed into an image classification task, which can exploit the advantage of CNN models. L. Wen et al. converted vibration signals into gray pixel images [24]. They simply rearranged each signal with a length of 4096 samples to form a 64 × 64 gray pixel image. J. Wang et al. used eight different time-frequency analysis methods to represent vibration signals under time-frequency images, including short-time Fourier transform, constant-Q Gabor transform, instantaneous frequency, fast kurtogram, Winger–Ville distribution, Hilbert–Huang transform, and Fourier synchrosqueezed transform [25]. Using a different approach, L. Eren proposed a 1-D CNN model to process bearing signals without converting them into the 2-D form [26].

In traditional signal-based fault diagnosis, extracting fault features from multiple domains of signals is a popular approach; however, it can be observed that one common point in current DL-based fault diagnosis methods is that their DNN models only attempt to extract features from a single domain of vibration signals, such as time domain [24], frequency domain [27], or time-frequency domain [25,28]. In literature, research that tries to extract features from multiple-domain image representation of vibration signals is rare. This point may be a drawback since the goal of feature extraction is to extract as much information as possible from the signal. Moreover, it is shown that multiple domain feature models can achieve better performance compared to single domain feature models [8]. Based on this observation, in this paper, a bearing fault diagnosis method based on the multiple domain feature model approach is proposed. First, a new method transforming vibration signals into high-dimensional data is proposed. In this method, vibration signals are simultaneously converted into time domain images and time-frequency images. This representation type of vibration signal is named here as multiple-domain image-representation (MDIR) data. A CNN-based deep model named multi-branch deep neural network (MB-DNN) is proposed to classify the MDIR data. The proposed MB-DNN has two main characteristics. Firstly, it inherits the deep learning ability of DL. Secondly, it has a multi-branch structure to extract features from multiple domains of signals simultaneously. More details of the MDIR data and the MB-DNN are explained in Section 2 and Section 3 of this paper. The task of signal-based fault diagnosis is now considered as the task of image classification. The effectiveness of the proposed method is verified through experiments with actual bearing fault data supplied by Case Western Reverse University [29]. The major contributions of this paper are summarized as follows:

• Proposing a method to represent vibration signals in high dimensional form;
• Proposing an MB-DNN with a multi-branch structure to handle the new representation of vibration signal in the high-dimensional domain;
• Transforming the task of signal-based fault diagnosis into the task of image classification;
• The proposed fault diagnosis method achieves significant classification accuracy.

The remainder of the paper is constructed as follows. Section 2 describes the way of constructing MDIR data from vibration signals. Section 3 explains MB-DNN and its application in bearing fault diagnosis. Experiments with actual bearing data were conducted and are presented in Section 4. Section 5 concludes the paper.

2. Multiple-Domain Image-Representation of Vibration Signal

Originally, vibration signal is time-series data, which is a 1-D data form. A new method to represent vibration signals in high dimensions is proposed. It is motivated by three following facts:

• It may be easier to understand and mine information in high-dimensional data [30].
• CNN and its variants are suitable for the task of recognizing two-dimensional visual patterns [31].
• By transforming the signal into visual data, the task of fault diagnosis can be converted into the task of image classification.

The vibration signal will be transformed into a time-domain image by a simple method proposed by D. Nguyen et al. in [32]. Consider a signal $x_l$ with the number of samples is $n^2$, all the samples in the signal are rearranged into a square matrix with a size of $n\times n$ as follows:

$$\left[\begin{array}{ccc}{x}_1& \dots & x_{n^2}\end{array}\right]\to \left[\begin{array}{cccc}{x}_1& x_2& \dots & x_n\\ x_{n+1}& x_{n+2}& \dots & x_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ x_{\left(n-1\right)n+1}& \dots & \dots & x_{n^2}\end{array}\right]$$

(1)

Then the obtained square matrix is normalized to range $\left[0.0-1.0\right]$ by the linear normalization:

$$I_{ij}=\frac{X_{ij}-min\left(X\right)}{max\left(X\right)-min\left(X\right)}$$

(2)

The normalized sample $I_{ij}, i, j\in \left[0, n\right]$ is placed at the row $i$, column $j$ of the matrix. The corresponding time domain image consists of pixels that are normalized samples of the corresponding signal, as shown in Figure 1.

The time-frequency image representation is based on the continuous wavelet transform (CWT). The CWT uses the same length of time data as in the previous transformation. A mother wavelet is a function $\psi (t)$ with zero average (i.e., ${\int }_{\mathrm{R}}\psi -0$), normalized (i.e., $\Vert \psi \Vert =1$), and centered in the neighborhood of $t=0$ [33]. Scaling $\psi (t)$ by a positive quantity $s$, and translating it by $y\in \mathbf{R}$, a wavelet family can be defined as:

$${\psi }_{u,s}(t):=\frac{1}{\sqrt{s}}\psi \left(\frac{t-u}{s}\right), u\in \mathbf{R}, s>0$$

(3)

Given $x(t)\in L^2(\mathbf{R})$, the continuous wavelet transform of $x(t)$ at time $u$ and scale $s$ (which inversely relates to frequency) is defined as:

$$W(s,u):=\langle x(t), {\psi }_{s,u}\rangle =\frac{1}{\sqrt{s}}\int x\left(t\right)\psi *\left(\frac{t-u}{s}dt\right)$$

(4)

where $\psi *$ denotes the complex conjugate of $\psi$. CWT decomposes the input signal $x(t)$ into a series of wavelet coefficients. The scalogram of $x(t)$ is defined by the function:

$$\mathbf{S}(s):=\Vert W(s,u)\Vert =\sqrt{{{\int }_{-\infty }^{+\infty }\left|W(s,u)\right|}^{2}du}$$

(5)

If a time interval $\left[t_0, t_1\right]$ needs to be considered, the corresponding windowed scalogram is defined by the function:

$${\mathbf{S}}_{\left[t_0, t_1\right]}(s):={\Vert W(s,u)\Vert }_{\left[t_0, t_1\right]}=\sqrt{{{\int }_{t_0}^{t_1}\left|W(s,u)\right|}^{2}du}$$

(6)

In other words, the scalogram is the absolute value of the CWT of a signal, plotted as a function of time and frequency, as shown in Figure 2.

By combining two transformations, each vibration signal is transformed simultaneously into image form: the time domain image and the time-frequency domain image as shown in Figure 3. This representation of vibration signal is named multiple-domain image-representation (MDIR). By using MDIR data, the problem of fault diagnosis based on vibration signal can now be considered as a task of image classification.

3. Proposed Multi-Branch Deep Neural Network

In the previous section, the multiple-domain image-representation (MDIR) of vibration signal has been described. To handle the MDIR data, a multi-branch deep neural network (MB-DNN) is proposed. The structure of MB-DNN is shown in Figure 4.

The proposed MB-DNN consists of several types of layers, including convolutional layer, batch normalization layer, pooling layer, feature fusing layer, dense layers, and SoftMax layer.

The convolutional layer based on the convolution operation is the most important layer type, which has been employed in many well-known DNN models such as LeNet [34], AlexNet [35], VGGNet [36], ResNet [37], and DenseNet [38]. The convolutional layer convolves the input with its kernels and feeds the obtained result into the activate function to generate the output. Consider a convolutional layer with $m$ kernels, the input has $M$ feature maps. The output of that layer can be calculated as follows:

$$y_k=\mathit{f}\left({\displaystyle \sum _{i=1}^M{\omega }_k\otimes x_i+b_k}\right), k=1:m$$

(7)

where $\mathit{f}$ denotes the activate function. In this paper, the Rectified Linear Unit (ReLU) is used as the activate function since it is simple and easy to compute.

The batch normalization layer exploited the batch normalization technique proposed in [39] to improve the training process of DNNs. The batch normalization ensures the transformation inserted in the network by representing the identity transform. The pooling layer reduces the dimension of the input feature maps. The dimension reduction operation can be conducted by max or average operation. The pooling layer computes the max value or average value of a group of neurons at the previous layer.

For convenience in calling and easer in drawing the neural network structure, three layers, including one convolutional layer, one batch normalization layer, and one pooling layer, are grouped successively to construct a convolutional–batch normalization–pooling (CBP) module. Assume that a CBP module with a convolutional layer with $m$ kernels. With the input feature map $x^{M\times n\times n}$, the output of the CBP module will be $y^{m\times h\times h}$ where $h=n/2$.

As shown in Figure 4, the proposed MB-DNN has two branches. While branch I handles the time domain images, branch II handles the time-frequency domain images of the vibration signals. Each branch consists of several CBP modules. Two branches simultaneously extract features from the input MDIR data, generate two types of feature maps. After that, the feature fusing layer will fuse these feature maps to generate a single one and forward it to the next part of the network. The operation of the feature fusing layer is described as follows:

$$x=\mathit{C}\left(x_t,x_f\right)$$

(8)

where $\mathit{C}$ denotes the fusing operation of the layer; $x_t,x_f$ are the feature maps extracted by branch I and II, respectively. Assume that $x_t$ and $x_f$ has sizes $a_1\times \left(b_1\times b_1\right)$ and $a_2\times \left(b_2\times b_2\right)$, respectively. First, two feature maps will be flattened to have size $1\times \left(a_1\times b_1{}^2\right)$ and $1\times \left(a_2\times b_2{}^2\right)$, respectively. The output of this layer $x$ will have size $1\times \left(a_1\times b_1{}^2+a_2\times b_2{}^2\right)$.

The dense layer is a traditional perception neural network. The purpose of using a dense layer is to collect all features from the previous feature map. The generated feature map of this layer will be used for the classification task, which will be performed by the SoftMax layer. With an input feature map $x$, the SoftMax layer computes the probabilities:

$$p_r=\frac{\exp \left({\omega }_k\times x\right)}{{\displaystyle {\sum }_{r=1}^N\exp \left({\omega }_k\times x\right)}}$$

(9)

where $N$ is the number of classes in the classification task. The SoftMax function calculates the probabilities of each target class over all target classes.

The loss function of MB-DNN is calculated by cross-entropy loss as follows:

$$L\left(q,p\right)=-{\displaystyle \sum _r^N{q}_r\log (p_r)}$$

(10)

where $q$ is the true label of the input data, $p$ is the output of the SoftMax function.

Using the MDIR data and MB-DNN, the proposed fault diagnosis method is illustrated in Figure 5. As shown in the diagram, the original fault signals are transformed into MDIR data. Then the obtained dataset is split into a training set and a testing set. The MB-DNN is trained with the training set by the back-propagation algorithm. Using the stochastic gradient descent with momentum, the weights of MB-DNN are updated by the equations

$${\upsilon }_t=\beta {\upsilon }_{t-1}+(1-\beta ){\nabla }_{\omega }L$$

(11)

$$\omega =\omega -\alpha {\upsilon }_t$$

(12)

where $\alpha$ denotes the learning rate, $\alpha$ is the momentum parameter.

4. Experiments and Results

4.1. Data Preparation

The actual bearing fault data are supplied by the Bearing Data Center of Case Western Reserve University (CWRU) [40]. The bearing fault testbed is shown in Figure 6. The testbed consisted of a 2-hp motor (left), a torque transducer (center), and a dynamo-meter (right). The test bearings support the motor shaft. The test bearings were seeded with faults using electro-discharge machining (EMD).

The vibration signals in this paper are measured by an accelerometer and digitized with a sampling frequency of 12 kHz. The accelerometer is installed at the driver end with magnetic bases. There are four types of bearing conditions are considered, including no-fault condition, bearing with inner race fault, bearing with outer race fault, and bearing with fault at rolling elements (ball fault). Each type of bearing fault is introduced to the test bearings with different defect sizes, including 7 mils (mili-inches), 14 mils, and 21 mils. The types of bearing conditions with different fault diameters are shown in

Table 1. Vibration labels.

Bearing Fault	Fault Size (mils)	Label
No fault		0
Inner race fault	7	1
Inner race fault	14	2
Inner race fault	21	3
Ball fault	7	4
Ball fault	14	5
Ball fault	21	6
Outer race	7	7
Outer race	14	8
Outer race	21	9

and Figure 7. There are 10 types of bearing conditions labeled from 0 to 9, respectively. The bearing testbed can be operated under different load conditions. In this paper, four load conditions were considered, including 0 hp, 1 hp, 2 hp, and 3 hp.

4.2. Signal Pre-Processing

In the data source supplied by CWRU, each bearing condition fault signal is measured and stored in a single file Matlab file; thus, there are 10 different signal files corresponding to 10 bearing conditions as in Table 1. In the intelligent fault diagnosis approach, classifiers require data samples to be trained; therefore, the original signal files are split into equal signal samples. Each signal sample must contain enough sampling points to convey the information of the bearing status; that is, if the length is too short, the signal sample cannot reflect the bearing health status. Normally, the sample length is selected to be equal to one revolution of the rotary shaft. In this work, the rotary speed of the shaft is $S=1796 \mathrm{rpm}$. Accordingly, the rotary frequency is $S_f=1796/60\approx 30 \mathrm{Hz}$. In the CWRU testbed, the sampling frequency $F=12,000 \mathrm{Hz}$. The minimum value of the sample length is $F/S_f=400$ (sampling point). As mentioned in Section 3, in the proposed method, there is a step where signal samples are transformed into gray images and apply CNN-like neural network to classify. So, we aim to make an MNIST-like data set where each image has a size of $1\times 28\times 28$; therefore, the length of signal samples is selected at the value of $28\times 28=784$. The time-domain image has the size of $1\times 28\times 28$.

The time-frequency domain image conversion exploits CWT using the Morse wavelet function. To deal with the time-frequency domain image, the obtained time-frequency images are scaled to a size of $3\times 224\times 224$. The MDIR data of the vibration signals with ten labels were obtained as shown in Figure 8.

From each original signal file, 300 data samples were obtained. Accordingly, the 10-class classification task is balanced since each class has 300 data samples. For each label, the image data were split randomly with the ratio 7:3 for the training set and the test set.

4.3. Design and Train the Proposed DNN

The structure of the proposed MB-DNN is designed as follows. First of all, the number of branches is two, corresponding to two types of input images. The input sizes of branches must be suitable for the input images; therefore, the first branch has the input size of $1\times 28\times 28$ and the second branch has the input size of $3\times 224\times 224$. The kernel size of convolutional layers and pooling layers are $3\times 3$ and $2\times 2$, respectively. The number of kernels in the convolutional layer is selected by a simple rule. Start with a small number of kernels in the first layer and double that number in the next layer. The first convolutional layer has 8 kernels, the second layer has 16 kernels, the third layer has 32 kernels, and so on. Each module CBP consists of one convolutional layer, one batch normalization layer, and one pooling layer. Since we use the zero-padding method in convolutional layers and the kernel size of $2\times 2$ in pooling layers, after each module CBP, the size of data will decrease by a factor of 2. The number of CBP modules is increased by one until the size of the output data is an odd number. In the first branch, two CBP modules are used; in the second branch, five CBP modules are used. The proposed MB-DNN has the configuration as shown in Figure 9. Branch I consists of two CPB modules that handle the time domain images. Branch II uses five CBP modules to handle the time-frequency domain images. The output of branch I has a size of $16\times 7\times 7$. The output of branch II has a size of $128\times 7\times 7$. These two outputs are fed into the feature fusing layer to generate the output data with size of $1\times 7056$. Then three successive dense layers are used to learn from the fused feature map. Finally, a SoftMax layer with ten outputs is used to classify the feature map generated by the third dense layer.

The MB-DNN is trained by mini-batch stochastic gradient descent with momentum algorithm, the learning rate $\alpha =0.001$, the momentum $\beta =0.9$, and the batch size $B=10$.

4.4. Fault Diagnosis Result

Four other bearing fault diagnosis methods are adopted to make comparisons with the proposed method as follows. The first method is published in [25]. In this method, the deep transfer learning technique is utilized to transfer the very deep neural network (Alexnet) pre-trained in the image classification domain into the domain of bearing fault diagnosis. The vibration signals are transformed into image form by using CWT. The second method is published in [24]. This method uses Lenet-5, which is a classical type of CNN in [41]. In this method, each sample of vibration signal is rearranged into a square matrix. The third method is published in [26]. This method utilized the 1-D form of CNN, which can process the vibration signal samples directly in the 1-D form without any transformation. The fourth method is published in [42]. The same as the method in [26], the vibration signals in [42] are used directly in 1-D form without any transformation. However, the number of convolution kernels of CNN is decreased with the reduction in the convolution kernel size.

The accuracy of all methods is shown in Figure 10. It can be observed that in all cases of load conditions, the proposed method and the Alexnet-based method in [25] achieve the best performance. The two methods have high mean accuracy and small standard deviation. The Lenet5-based method in [24] has a little lower accuracy. The CNN1D-based method in [26] has a lower performance with low accuracy. The CNN1D-based method in [42] has the poorest performance with low accuracy.

4.5. Evaluation under Noisy Conditions

The bearing data set supplied by CWRU has been extensively employed as a benchmark for evaluating bearing fault diagnosis methods. Recently, proposed fault diagnosis methods with advantage signal processing and feature learning techniques can achieve very high accuracy. It is not easy to highlight the performance of newly proposed fault diagnosis methods; therefore, noise signals are often added to the original signals to evaluate methods. This way can help to evaluate the robustness of fault diagnosis methods under more challenging conditions. In this scenario of evaluating diagnosis methods, Gaussian white noise (AGWN) is added into the original vibration signals as in Figure 11.

The signal-to-noise ratio (SNR) is defined to measure the level of the obtained noisy signal to the level of the additional Gaussian noise. The SNR of a noisy signal is computed as follows:

$$\mathrm{SNM}=10\log \left(\frac{{\mathrm{P}}_{\mathrm{signal}}}{{\mathrm{P}}_{\mathrm{noise}}}\right)$$

(13)

where ${\mathrm{P}}_{\mathrm{signal}}$ and ${\mathrm{P}}_{\mathrm{noise}}$ are the power of signal and noise, respectively. Nineteen noise levels (dB) in the range of [−8, −7, …, 10] are taken into account. The comparison is illustrated in Figure 12.

In the previous section, all compared methods achieved very high diagnostic accuracy when the inputs were the original signals. When the input signals were transformed by adding low-level noisy signals with the SNR in the range of [0, …, 10], the performance of the method in [26,42] was significantly reduced, but the performance of the proposed algorithm and the methods in [24,25] were reduced insignificantly; however, when the input signals are transformed by adding high-level noisy signals with the SNR in the range of [−8, …, −1], their performance decreases dramatically. Obviously, the noise makes it harder to extract fault signatures from the signals. It can be observed that in the worst case (−8 dB), the methods in [24,26,42] totally fail since their accuracy is under 50%. The trend of all methods’ performance is, the higher the noise level, the lower the accuracy of the diagnosis result. Among all methods, the proposed one achieves the best performance with good robustness against noise. Even under the worst noise case (−8 dB), it achieves an accuracy of 57%. Comparing the structures of all methods, we can see that the method in [25] only takes care of the time-frequency domain features; the methods in [24,26,42] only consider the time domain features. The difference of the proposed methods is that they can receive multiple feature domains. As a result, the proposed method can extract more robust features from signals in noise conditions. So, it can be concluded that the proposed network with a multi-branch deep structure can extract fault features more effectively even under severe noise conditions. That leads to better diagnostic performance compared to other DNNs.

5. Conclusions

This paper proposed a novel method of bearing fault diagnosis based on vibration signals. By using simple transformation methods, time-series vibration signals are transformed into high dimensional data form (MDIR). By using this transformation, the task of fault diagnosis becomes the task of image classification. A novel DNN with a multi-branch structure is proposed to handle the MDIR data of vibration signal, named MB-DNN. The proposed MB-DNN inherits the advantages of CNN in processing high-dimensional data. In addition, MB-DNN has a multi-branch structure with two branches that can simultaneously extract features from the time domain and time-frequency domain. The proposed method obtains high classification accuracy; especially, it shows the efficiency even under noise effects. The proposed algorithm can be applied as an automatic fault detection process for the early detection of bearing faults; therefore, it helps to reduce the failure rate of machinery and save repair costs. Besides the ability to process vibration signals, the proposed method can be applied to process the current signals and acoustic emission signals.