1. Introduction
Most of the fault diagnosis of rolling bearings is based on vibration signals and acoustic emission signals. However, the acquisition of both signals is susceptible to the influence of the external environment; i.e., noise is easily collected. Therefore, it is very difficult to extract useful information [
1,
2,
3,
4]. In recent years, temperature signals have gradually received the attention of researchers. Infrared thermal imaging cameras can acquire infrared images and thus display the thermal characteristics of objects. Compared with the two signals discussed above, the temperature signal has the characteristics of non-contact, high sensitivity and real-time. Due to its strong advantages for test targets running at high speeds, some scholars have introduced it into the troubleshooting of mechanical equipment [
5,
6]. Therefore, using infrared images as a research object for rolling bearing fault diagnosis has certain advantages.
Regardless of the signal base, the essential series of processes for fault diagnosis consists of feature extraction, feature selection and fault diagnosis. There is much related research on traditional vibration signals and acoustic emissions. Zhang et al. extracted features from bearing vibration signals in the time and frequency domains and applied Multidimensional Scaling (MDS) for feature selection and finally obtained highly accurate fault diagnosis results by fusing a support vector machine (SVM) [
7]. Sun et al. combined the 1 Dimension Convolutional Neural Network (1DCNN) and Long Short-Term Memory (LSTM) to avoid the errors caused by relying on expert experience and incomplete information in traditional feature extraction methods [
8]. Tian et al. proposed a Multi-domain Entropy-random Forest (MDERF) method that fuses multi-domain entropy and random forest to extract the multi-domain entropy of the acoustic emission signal as a feature extraction method to extract the four entropies of the AE signal and used the random forest to complete the fault diagnosis [
9]. Pham et al. two-dimensionally verified the diagnostic capability of the improved generative adversarial network (GAN) under unbalanced acoustic emission signals [
10]. Motor Current Signature Analysis (MCSA) is widely used for motor fault diagnosis. Granda et al. obtained the current signal of induction motor by MCSA and used the Continuous Wavelet Transform (CWT) model of the squirrel-cage induction motor for broken bar diagnosis [
11]. Juan et al. obtained the vibration signal and current signal of the motor, calculating the Power Spectral Density (PSD) and Fast Fourier Transform (FFT). The amplitudes at the frequencies of interest given by the theory were analyzed to provide reliable fault diagnosis results [
12].
Usually, feature extraction from IR images and feature selection are the top priority to ensure the completion of the subsequent work. Currently, most research scholars use the method of manually extracting the grayscale features, extracting texture features of the image or parsing the image using image analysis methods. Liu et al. extracted texture features, moment features and modified entropy features for server fault diagnosis [
13]. Thobiani et al. used a two-dimensional empirical mode decomposition (BEMD) which was used to enhance the infrared thermal image of the bearing, after which the second-order statistical texture features of the image were extracted, and feature selection was performed using Minimum Redundancy Maximum Relevance (MRMR) [
14]. In addition, another feature extraction method is to train and learn the data using deep neural network models to obtain high-level feature representation in the data. This feature extraction method has better performance and generalization capability than the traditional manual feature extraction method. Choudhary et al. directly extracted the features of the image using the 2 Dimension Convolutional Neural Network (2DCNN) and showed significantly better diagnostic results compared to others [
15]. He et al. addressed the situation of insufficient data samples by incorporating the convolutional auto-encoder (CAE) and enhanced convolutional neural network (ECNN) to extract deep features [
16]. Wei et al. constructed an extended model of faulty samples by migration learning and deep convolutional generative adversarial network (DCGAN) to solve the small sample infrared image fault diagnosis [
17]. Thus, it can be seen that the criteria for feature selection are either difficult to develop or difficult to understand. Even if the identified best features are selected, it is difficult to ensure the optimal and minimum number of features. So, the way of feature selection needs to be improved.
Whether the data features are extracted from the infrared images as a way to characterize the images or the deep features of the images are constructed directly using deep learning, the later articulated methods are basically like those used for vibration and acoustic emission signals. Choudhary et al. used decision tree (DT), linear discriminant analysis (LDA) and SVM for fault diagnosis of electric motors, respectively, and eventually SVM proved to be superior for diagnosis [
18]. Glowacz used a nearest neighbor classifier and back propagation neural network for fault diagnosis of infrared images of electric drills [
19]. Janssens et al. acquired infrared images to extract pixel points in the x- and y-axis directions of the pixel histogram, light moments and Gini coefficients as data features and finally classified them using a random forest (RF) model [
20]. Li et al. used SoftMax for classification after 2DCNN [
21]. In conclusion, machine learning and deep learning are heavily cited for infrared image troubleshooting of machinery or electricity. However, machine learning often possesses data dependency and poor generalization of the models. Deep models, on the other hand, are time-consuming, resemble a “black box” and do not have good explanatory power.
In 2018, Chen, C.L.P. proposed the Broad Learning System (BLS), which is a random vector function linked neural network. It contains only a single layer of hidden layers and is expanded incrementally in a transversal manner anytime when faced with a learning inaccuracy. It greatly solves the problem of long training time for deep learning [
22]. Wang et al. extracted features describing temporal and spatial distribution from infrared images of power equipment and finally diagnosed them using BLS [
23]. Zhang et al. used double-tree complex wavelets to decompose vibration signals of bearing faults, extracted sub-bands as feature vectors and finally completed fast fault classification using BLS [
24]. Wang et al. proposed TSK-BLS, which can be calculated quickly and accurately by pseudo-inverse and symmetric methods and has good fault diagnosis advantages [
25]. Therefore, the introduction of width learning into the field of fault diagnosis has some practical significance. However, the parameters of the network model can have an impact on the accuracy. The parameters provided by the original authors are used in the above application of BLS for fault diagnosis. Some BLS-based fault diagnoses are aimed at the internal algorithms, which all use manually set parameters, generally for test verification or manual experience, and do not guarantee the best accuracy. Yu et al. defined connection weight matrices for inter-class and intra-class graphs to improve the fault diagnosis accuracy considering the sameness between similar samples and the difference between different classes of samples [
26]. However, their parameters are set to fixed values based on manual experience, which may miss the optimal accuracy. Therefore, it is very necessary work to perform an optimization search for the parameters.
Infrared thermal images often contain more complex and stronger noise than ordinary optical images due to factors such as spatial and temporal environment and detection equipment. Likewise, the temperature of the surrounding environment can interfere with the focused information of the image. Therefore, noise removal and other pre-processing efforts are necessary.
In view of the above research background and existing problems, this article proposes an RFR-GA-BLS fault diagnosis model. Firstly, a series of pre-processing such as grayscale map conversion, region of interest (RoI) acquisition, denoising and image segmentation are performed on the acquired infrared images. Then, the second-order statistical features are extracted using the grayscale co-occurrence matrix (GLCM). After that, they are imported into RFR-GA-BLS. The extracted features are ranked by composing Robust Feature Ranking (RFR) with robust feature properties. The optimal features are added one at a time, and a feature search is performed to derive the optimal and minimum number of features. The optimal features and the highest accuracy of each round are filtered out after each round of data input to BLS using the optimization-seeking property of the Genetic Algorithm (GA).
The rest of the article is organized as follows:
Section 2 introduces the relevant technical theory.
Section 3 reviews the rolling bearing infrared image data acquisition experiments.
Section 4 describes the model building.
Section 5 gives the experimental results of fault diagnosis.
Section 6 draws conclusions.
The contributions of this article are as follows:
The infrared image replaces the previous vibration signal and acoustic emission signal, so that the fault performance is visualized.
RFR uses the robustness of features to rank features in the RFR-GA-BLS fault diagnosis model. Instead of the previous way of filtering or rejecting features, it solves the problem of difficult feature selection by finding the optimal number of features and corresponding features in an iterative way.
The GA-BLS in RFR-GA-BLS fault diagnosis model solves the problem of difficult setting of BLS parameters and can find the best parameters and the highest accuracy rate.
RFR-GA-BLS has superior accuracy than machine learning and faster training time than deep learning.
2. Materials and Methods
2.1. GLCM and Second-Order Statistical Features
The second-order statistical features have a strong interpretation and good robustness, which can reflect the characteristics of the pictures more comprehensively [
14]. It can be obtained by GLCM, which can be defined as:
where
denotes the probability of occurrence of pixels with grayscale values
i and
j at the positions of phase
and angle
.
denotes the number of pixels with grayscale values
i and
j at the positions of phase
and angle
.
G denotes the number of grayscale levels of the image, i.e., 256.
Pixel distances of 1, 2 and 3 were chosen, respectively. Some 0°, 45°, 90° and 135° pixel angles were selected. The extracted second-order statistical features contain energy, contrast, entropy, sum average, variance and correlation. A total of 72 features are noted as GLCM_0, GLCM_1, … and GLCM_71. The extracted second-order statistical features are shown in
Table 1.
2.2. RFR
Features with robustness can reduce the effect of data outliers and noise and improve the stability of the classifier. An RFR method is given which exploits robustness and ranks features according to size. Thus, the most robust features are utilized in preference. Here, the robustness calculation metric is calculated using Mean Absolute Deviation (MAD), a measure of how discrete the data are. Larger values of MAD indicate more robust data, i.e., more tolerant of noise or outliers.
Robustness is used to assess the stability of the feature column in the face of outliers. Unlike other studies, after calculating the robustness metric of the feature column, the features with poor robustness performance are not eliminated or retained with good robustness performance. Instead, they are ranked according to the magnitude of the robustness values. The features GLCM_0, GLCM_1, … and GLCM_71 are ranked according to the 72 evaluation metrics obtained.
When performing fault diagnosis, the first diagnosis uses the first feature, which is the feature with the best robustness. The second diagnosis uses the first two features and so on, adding one new feature at a time until all 72 features have been taken.
2.3. GA
GA is an optimization algorithm that follows the principles of survival of the fittest and meritocracy. It is developed by borrowing from the genetic process in evolutionary biology. A series of operations such as inheritance, mutation, crossover and natural selection are included in it.
The genetic algorithm generates an initial population for the parameters to be optimized and evaluates it according to the fitness function. Thus, the set of solutions with small fitness function is eliminated, and the population with high fitness is retained. After operations such as amplitude, crossover and variation, the individuals are continuously preferred to obtain the optimal solution.
2.4. BLS
BLS is a random vector function linked neural network. By calculating the weights of feature nodes and augmentation nodes through pseudo-inverse operation, the calculation speed and measurement accuracy of the model can be improved comprehensively. The structure diagram is shown in
Figure 1.
First, the model completes the mapping of the input data to the feature nodes, which can be defined as:
where
Zi is the
i-th feature node;
Wei is the random weight of the feature mapping layer;
βei represents the random bias of the feature mapping layer.
The generation of augmented nodes from mapped nodes after transformation can be defined as:
where
Hj is the
j-th enhanced node;
Whj is the random weight of the feature mapping layer;
βhj represents the random bias of the feature mapping layer.
Output all mapped and enhanced nodes, which can be defined as:
where
Y is the node set;
represents the mapping node set;
presents the enhancing node set;
Wm represents the weight set.
Finally, the pseudo-inverse is then calculated to give the weights of the output layer, which can be defined as:
Therefore, it is very essential to set the number of feature nodes, the number of feature node windows and the number of enhancement nodes in the whole model. Here, they are denoted as N1, N2 and N3, respectively. In general, these parameters need to be manually tuned to achieve a high standard of diagnostic results. However, such a model does not have the ability to generalize and does not necessarily guarantee that the optimal diagnostic rate is obtained. Therefore, parameter optimization is necessary for N1, N2 and N3.
3. Experiment
The infrared image signal applied in this article comes from the rolling bearing failure experimental bench in
Figure 2a. The intercepted IR video interface is shown in
Figure 2b. The experimental bench includes a servo motor, controller, coupling, shaft, faulty bearing and an infrared thermal image acquisition system. The bearing model is S6205-2RSR. The motor speed is stabilized at 2000 rmp. The IR thermal image acquisition system includes BM_RI Software V7.4 and the IR thermal imaging camera FLIRA35. The parameters of the IR thermal imaging camera and the acquisition system are shown in
Table 2.
In order to test the performance of the proposed model, nine states of the bearing are set. They are health condition (HE), holder failure (HO), inner ring 0.5 mm crack (IN05), inner ring 1.0 mm crack (IN10), inner ring 1.5 mm crack (IN15), outer ring 0.5 mm crack (OU05), outer ring 1.0 mm crack (OU10), outer ring 1.5 mm crack (OU15) and rolling ball pitting (RO).
During the acquisition process, the ambient temperature was maintained at about 24.5 °C. Each bearing state was waited for the temperature to remain stable while video acquisition was performed. Subsequently, 15 min of infrared video was acquired. The steps of infrared thermal image acquisition are shown below:
Step 1: A state bearing is mounted to the shaft.
Step 2: Turn on the motor controller and make sure the bearing is running at a constant speed of 2000 rmp.
Step 3: Observe the temperature change and start collecting infrared video for 15 min after it is roughly stable.
Step 4: Cool the rolling bearing failure test bench to ambient temperature of 24.5 °C.
Step 5: Repeat steps 1–4 to complete the acquisition of a total of nine 15 min infrared videos of different bearing failures.
Step 6: Extract infrared images from the videos according to the 9 s time interval, 100 images for each category. In total, 900 images were extracted.
Step 7: Select the bearing and shaft areas that produce temperature changes and crop them to obtain the RoI. The RoI of the extracted IR images for each category of bearings is shown in
Figure 3.
4. Fault Diagnosis Model
4.1. RFR-GA-BLS Fault Diagnosis Model
The flow chart of the proposed RFR-GA-BLS fault diagnosis model is shown in
Figure 4.
The outermost side of the model is the RFR method. It sorts the 72 features by calculating the robustness of each column of features. The first loop selects the first feature at the top of the ranking. The second loop selects the second feature at the top of the ranking, and one is added in each loop until all 72 features are selected. In addition, a way to jump out of the loop is designed. Once the accuracy of a certain time exceeds 85% and the next four features are not as high as this one, it is determined that this is the best result found. All results are output at this point.
The middle part is the GA optimized BLS model. The fault diagnosis accuracy of the BLS model is used as the fitness function. N1, N2 and N3 are used as the optimization parameters. After setting the parameter ranges of N1, N2 and N3 and generating the initial population, the initial population is first subjected to fitness calculation, and the dominant individuals are selected for retention. The left individuals are subjected to a series of genetic operations to generate offspring. In the continuous evolution, the remaining offspring have increasing fitness, and thus the optimal solution that meets the objective is selected.
The remaining parameters were chosen as follows: for RFR, i from 1 to 72. For GA, the initial population N = 30; the spatial dimension d = 3; the maximum number of iterations ger = 10; the crossover probability Pc = 0.8, and the variation probability Pm = 0.1. For BLS, the shrinkage factor of the augmented nodes was s = 0.8; the regularization parameter was C = 2–20, and the range of N1 was [10, 30]; the range of N2 is [10, 30]; the range of N3 is [500, 600], and the step length is 1.
4.2. Fault Diagnosis Process
In this article, rolling bearing fault diagnosis based on infrared images is studied. Firstly, the infrared videos of nine kinds of faulty bearings are acquired, so as to crop the infrared images. After that, a series of pre-processing work such as RoI acquisition, grayscale image conversion, image denoising and image segmentation are performed. After completing these tasks, GLCM is applied to extract features. Finally, the acquired feature set is fault diagnosed using RFR-GA-BLS. The steps are as follows:
Step 1: Acquisition of IR images. IR video of nine types of rolling bearings were acquired for 15 min after temperature stabilization: i.e., HE, HO, IN05, IN10, IN15, OU05, OU10, OU15 and RO. IR images were cropped at 9 s intervals. Some 900 images can be obtained.
Step 2: Cropping images. The temperature of the bearing and the axis affected by the bearing temperature can be acquired as RoI. The 280*150 pixel area is set.
Step 3: Grayscale image conversion.
Step 4: Median filtering to remove noise. The size of the filter is set to 3*3, and the median is taken as the value of a certain pixel point.
Step 5: Adaptive thresholding segmentation. The image is transformed into a binary image. Firstly, a Gaussian weighted average is applied to set the threshold value, and the filter size is set to 5*5. Each pixel point is processed according to the obtained threshold value; pixels larger than the threshold value are 255, and pixels smaller than the threshold value are 0.
Step 6: Extraction of features. Extract the image features using GLCM, which contains energy, contrast, entropy, sum average, variance and correlation, with designed pixel distances of 1, 2 and 3 and angles of 0°, 45°, 90° and 135°, respectively. There are 72 features in total.
Step 7: Fault diagnosis. The obtained 72 features are input into the RFR-GA-BLS fault diagnosis model. Finally, the optimal features, the best parameters and the highest accuracy are obtained.
5. Results and Discussion
The specific experimental environment for completing the data dimensionality reduction and fault diagnosis stages on the Windows 10 operating system is shown in
Table 3.
After completing steps 1–5 of the fault diagnosis steps, the results are shown in
Figure 5.
Subsequently, the 72 features, GLCM_0, GLCM_1, … and GLCM_71, were extracted using GLCM. Two fault diagnosis models, RFR-GA-BLS and CFR-GA-BLS, are applied to diagnose the faults, respectively. The RFR method and correlation feature ranking (CFR) are applied to rank these features, respectively, where the CFR method is the comparison scheme set up, based on the correlation of the features. It is obvious that these two feature ranking methods have a very different order for the ranked features. After sorting, the datasets were divided in a ratio of 9:1. Totals of 90 of each class were used as the training set and of 10 as the test set.
Additionally, then the GA-BLS model is set up in turn. For each round of feature increase, the training accuracy and test accuracy are saved, and the results are shown in
Figure 6.
Figure 6a indicates the running results of CFR-GA-BLS. The first 55 features satisfy the algorithm conditions, and the final fault diagnosis accuracy obtained is only 0.86667.
Figure 6b indicates the running results of RFR-GA-BLS. The final fault diagnosis accuracy obtained is 0.98889 for the first 20 features. In addition, the derived time variation is shown in
Figure 7. The specific optimization-seeking parameters and corresponding results are shown in
Table 4.
Figure 7 shows the time consumed for each feature addition. The CFR-GA-BLS satisfies the end condition only because of the first 55 features screened. Therefore, its running time is included in the next four diagnoses, which are used as a comparison diagnostic rate for skipping out of the loop. Its total run time is 2374.18 s. Similarly, the run time of RFR-GA-BLS is 918.28 s, which is about 15 min. Its speed is much smaller than that of CFR-GA-BLS, which is about 2/5.
Table 4 lists the model run time, accuracy, number of features and parameters. Compared with CFR-GA-BLS, RFR-GA-BLS can apply a shorter time to show the best results, and its final selected features are the top 20. After optimization by GA, N1, N2 and N3 are 24, 19 and 544, respectively.
Figure 8 illustrates the confusion matrix of RFR-GA-BLS. It clearly shows the distinction between the categories that generate errors. A sample that should belong to the OU10 category is incorrectly distinguished as OU05.
To determine whether the features, parameters and accuracy found by RFR-GA-BLS perform well, the model state was manually changed to remove the jump-out condition and run 72 times. The features sorted by CFR and RFR were sequentially imported into GA-BLS, and the change in accuracy was observed for each round, and the results are shown in
Figure 9. The running time of each round of GA-BLS was exported, and the results are shown in
Figure 10.
According to
Figure 9, it can be concluded that the CFR-GA-BLS model satisfies the end condition at the first 55 features, but its accuracy does not reach the optimal value. After the number of features is 65, the accuracy rate jumps. In the case of RFR-GA-BLS, the condition is satisfied at the number of features 20. Although 25 features can reach 1, 20 features make its operation time shorter, and the smaller accuracy gap can be ignored. Therefore, RFR-GA-BLS can achieve more efficient and accurate fault diagnosis.
Figure 10 shows the running time of each GA-BLS. It can be observed that adding a few features does not increase the GA-BLS operation time, and the running time is kept within 30–50 s. Therefore, width learning has great advantages as an optimization model.
In addition, this article verifies the superiority of GA-BLS. In order to reduce time consumption, only the first 20 features obtained in the previous experiments are selected for fault diagnosis. The comparison method was set up for comparison, as shown in
Table 5.
The BLS model was chosen with the parameters of the code shared by the original authors, i.e., N1 = 10, N2 = 10, N3 = 500. The optimal accuracy obtained was 0.93333 with a running time of 0.18 s. Particle Swarm Optimization (PSO) was applied to determine the three parameters of the BLS model, and the number of iterations was again chosen to be 10. The accuracy is 0.97778, and the running time is 39.20 s. The Grdy-BLS using greedy algorithm (Grdy) to optimize the BLS obtains an accuracy of 0.95556 and the running time is 5.83 s.
Although the BLS model has a short running time, the accuracy of the model is low due to the poor generalization ability of the model by using only the parameters provided in the original text. PSO-BLS consumes slightly less time than GA-BLS, but the time difference is not significant. Its accuracy is lower than that of GA-BLS. Grdy-BLS has a short computing time, but the greedy algorithm tends to fall into a local optimum. As seen, the three parameters chosen fluctuate only in the initial state.
Therefore, GA-BLS is a good choice. As part of RFR-GA-BLS, it shortens the operation time and does not easily fall into the local optimum. Not only the good parameters and the minimum number of features are found, but also the optimal accuracy is obtained.
Subsequently, the first 20 features were used for fault diagnosis using SVM and RF. As shown in
Table 6, SVM and RF are inferior to BLS in terms of speed and accuracy. The running time of SVM is 3.32 s, and that of RF is 2.19 s, which is faster but difficult to compare with BLS, and the algorithm will be exponentially higher if optimization parameters such as GA are added.
In addition, 1DCNN and 2DCNN were applied for fault diagnosis of infrared images. The top 20 features of the ranking were processed using 1DCNN, using two layers of convolution and two layers of pooling. The epoch was chosen to be 100. The speed was similar to that of GA-BLS, but only an accuracy of 0.9333 was obtained. The 2DCNN was used to process the image after adaptive thresholding, using two layers of convolution and two layers of pooling. The epoch was chosen to be 30. Still following the 9:1 dataset partitioning. Although it achieves an accuracy of 0.9889, it takes 10,147 times longer than BLS. The speed is also much slower than RFR-GA-BLS.
Thus, GA-BLS not only achieves the parameter search, but also has more advantages in terms of speed and accuracy. The problem of low accuracy of traditional machine learning fault diagnosis is overcome, and the problem of time-consuming deep learning of direct image processing is also overcome. The RFR-GA-BLS proposed in this article has strong advantages in the field of infrared image fault diagnosis of bearings.