# Multiscale Distribution Entropy and t-Distributed Stochastic Neighbor Embedding-Based Fault Diagnosis of Rolling Bearings

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Algorithms of DE and MDE

#### 2.1. Definition of DistEn

_{i,j}= max(|u(i + k)− u(j + k)|, 0 ≤ k ≤ m − 1) are computed for all $1\le i,j\le N-m$. The distance matrix is defined as D = {d

_{i,j}}.

_{t}(t = 1, 2,…, M) is frequency and the estimated value are not included in the distance matrix D while i = j.

#### 2.2. DistEn Parameter Selection

#### 2.3. Multiscale Distribution Entropy

## 3. Comparison Analysis of MSE and MDE

#### Simulation Tests

## 4. The Proposed Fault Diagnosis Approach and Its Applications

#### 4.1. t-SNE Algorithm

^{(T)}= {y

_{1}, y

_{2},…, y

_{n}} are obtained.

#### 4.2. KVPMCD

#### 4.2.1. Basic Concepts and Frameworks of KVPMCD

#### 4.2.2. Kriging Model-Based KVPMCD Method

- (1)
- For g class classification problem, n training samples are collected and each sample number is ${n}_{1},{n}_{2},\dots ,{n}_{g}$. The feature X = [${x}_{1},{x}_{2},\dots ,{x}_{p}$] is extracted from all training samples and the size of each feature is ${n}_{1}\times p,{n}_{2}\times p,\dots ,{n}_{g}\times p$ respectively.
- (2)
- The feature ${X}_{i}$ (i = 1, 2, …, p) of the kth (1 ≤ k ≤ g) training sample is selected as the predicted variable and the remaining p-1 feature ${X}_{j}$ (j ≠ i) is seen as predictive variables.
- (3)
- Let the regression model type z = 1 (1 ≤ z ≤ Z) (zero, one and two order polynomial. Three models are marked as 1, 2, and 3, respectively). The model category of the relevant models is h = 1 (1 ≤ h ≤ H) (exponential, generalized exponential, Gaussian, linear, spherical, cubic, spline, respectively, is marked as 1, 2, 3, 4, 5, 6 and 7), and then a mathematical model is established.
- (4)
- Set h = h + 1 and z = z + 1, respectively, until h = H, z = Z. The combination of predictive variables is common to H × Z species. Therefore, ${n}_{k}$ = H × Z mathematical equations can be established.
- (5)
- ${n}_{k}$ equations can be established for each feature set ${X}_{i}$. The feature of each training sample in the kth class can be obtained. The predicted value ${X}_{ipred}$ of the feature ${X}_{i}$ can be obtained by the Kriging model.
- (6)
- To calculate the prediction error square sum $SS{E}_{l}={\displaystyle \sum _{v=1}^{{n}_{k}}{({X}_{iv}-{X}_{ivpred})}^{2}}$ of the ${n}_{k}$ variable prediction model respectively, where v represents the vth training sample and $l=1,2,\dots ,{n}_{k}$. The variable prediction model corresponding to the minimum value of $SS{E}_{l}$ is selected as the variable prediction model $VP{M}_{\mathrm{i}}^{k}(i=1,2,\dots ,p)$ of the feature ${X}_{\mathrm{i}}(i=1,2,\dots ,p)$ in the kth class training sample. Then save the corresponding model parameters and predictive variables.
- (7)
- Let k = k + 1, repeat steps (3)~(6) until k = g. At this point, in the case the variable prediction model $VP{M}_{i}^{k}$ are established for all the feature of g categories respectively, where k (k = 1, 2, …, g) denotes category label and i (i = 1, 2, …, p) represents the feature. These variable predictive models form a VPM matrix with size of g × p.
- (8)
- All training samples are constructed as a test sample set to perform a return classification test for each VPM matrix. The regression model type and the relevant model type corresponding to the VPM matrix with the highest correct classification rate are selected as the type of the best variable prediction model.
- (9)
- The feature X = [${x}_{1},{x}_{2},\dots ,{x}_{p}$] are extracted for the selected test sample set. For all the feature values ${X}_{i}(i=1,2,\dots ,p)$ of the test sample, respectively, the variable prediction model $VP{M}_{i}^{k}$ is employed to predict it, and the predicted value $VP{M}_{ipred}^{k}$ is obtained.
- (10)
- The squared sum $SS{E}^{k}={\displaystyle \sum _{i=1}^{p}{({X}_{i}-{X}_{ipred}^{k})}^{2}}$ of the predicted errors is calculated for all features in the same category. And the minimum $SS{E}^{k}$ is used as the discriminant function to classify the test samples.

#### 4.3. The Proposed Fault Diagnosis Method

- (1)
- Assume that the states of rolling bearing contain K classes and each state is collected by N groups. MDE of each vibration signal is computed with parameters m = 2, δ = 1, M = 512 and the maximum scale factor ${\tau}_{\mathrm{max}}$. ${\tau}_{\mathrm{max}}$ eigenvalues were obtained to represent the fault information of the vibration signals of rolling bearing in each group and the feature vector matrix ${R}^{N\times {\tau}_{\mathrm{max}}}$ is constituted, which can adequately digs out the characteristics information of different complex time series.
- (2)
- t-SNE is used to reduce the dimension of feature vector matrix and a low dimensional sensitive feature set R
^{N×i}can be obtained, where N represents the number of samples and i represent the dimensions after dimensionality reduction. - (3)
- Training samples are composed of the selected 1/2 N group of each state randomly, the rest as test samples. The training samples are input to the KVPMCD based multi-classifier for training. The predictive model ${KVPM}_{i}^{k}$. is established, where k (k = 1, 2, …, g) represents different categories, i (i = 1, 2, …, p) represents different characteristic values.
- (4)
- The outputs of classifier are used to diagnose the fault types of rolling bearing.

#### 4.4. Experimental Data Analysis

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Henao, H.; Capolino, G.A.; Fernandez-Cabanas, M.; Filippetti, F.; Bruzzese, C.; Strangas, E.; Pusca, R.; Estima, J.; Riera-Guasp, M.; Hedayati-Kia, S. Trends in fault diagnosis for electrical machines: A review of diagnostic techniques. IEEE Ind. Electron. Mag.
**2014**, 8, 31–42. [Google Scholar] [CrossRef] - Frosini, L.; Harlişca, C.; Szabó, L. Induction machine bearing fault detection by means of statistical processing of the stray flux measurement. IEEE Trans. Ind. Electron.
**2015**, 62, 1846–1854. [Google Scholar] [CrossRef] - Zurek, S.; Guzik, P.; Pawlak, S.; Kosmider, M.; Piskorski, J. On the relation between correlation dimension, approximate entropy and sample entropy parameters, and a fast algorithm for their calculation. Physical A
**2012**, 391, 6601–6610. [Google Scholar] [CrossRef] - He, Y.; Huang, J.; Zhang, B. Approximate entropy as a nonlinear feature parameter for fault diagnosis in rotating machinery. Meas. Sci. Technol.
**2012**, 23, 045603. [Google Scholar] [CrossRef] - Kaffashi, F.; Foglyano, R.; Wilson, C.G.; Loparo, K.A. The effect of time delay on approximate & sample entropy calculations. Phys. D Nonlinear Phenom.
**2008**, 237, 3069–3074. [Google Scholar] - Shang, D.; Xu, M.; Shang, P. Generalized sample entropy analysis for traffic signals based on similarity measure. Physical A
**2017**, 474, 1–7. [Google Scholar] [CrossRef] - Miśkiewicz, J. Improving quality of sample entropy estimation for continuous distribution probability functions. Physical A
**2016**, 450, 473–485. [Google Scholar] [CrossRef] - Zhang, W.B.; Zhou, Y.J.; Zhu, J.X.; Pu, Y.S. A New Rotor Fault Diagnosis Method Based on EEMD Sample Entropy and Grey Relation Degree. In Applied Mechanics and Materials; Trans Tech Publications Ltd.: Zürich, Switzerland, 2013; Volume 347, pp. 426–429. [Google Scholar]
- Zheng, J.; Cheng, J.; Yang, Y. A rolling bearing fault diagnosis approach based on LCD and fuzzy entropy. Mech. Mach. Theory
**2013**, 70, 441–453. [Google Scholar] [CrossRef] - Lee, S.H.; Kim, S.; Kim, J.M.; Choi, C.; Kim, J.; Lee, S.; Oh, Y. Extraction of Induction Motor Fault Characteristics in Frequency Domain and Fuzzy Entropy. In Proceedings of the 2005 IEEE International Conference on Electric Machines and Drives, San Antonio, TX, USA, 15 May 2005; pp. 35–40. [Google Scholar]
- Shi, Z.; Song, W.; Taheri, S. Improved LMD, permutation entropy and optimized K-means to fault diagnosis for roller bearings. Entropy
**2016**, 18, 70. [Google Scholar] [CrossRef] - Jiang, J.; Shang, P.; Zhang, Z.; Li, X. Permutation entropy analysis based on Gini-Simpson index for financial time series. Physical A
**2017**, 486, 273–283. [Google Scholar] [CrossRef] - Rényi, A. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 20 June–30 July 1961; pp. 547–561. [Google Scholar]
- Coifman, R.R.; Wickerhauser, M.V. Entropy-based algorithms for best basis selection. IEEE. Trans. Inf. Theory
**1992**, 38, 713–718. [Google Scholar] [CrossRef] - Yan, R.Q.; Gao, R.X. Approximate entropy as a diagnostic tool for machine health monitoring. Mech. Syst. Signal Process.
**2007**, 21, 824–839. [Google Scholar] [CrossRef] - Liming, D.; Weixing, H.E.; Li, B. Fault diagnosis of drilling for oil bearing based on sample entropy and neural network. Microcomput. Inf.
**2007**, 23, 223–225. [Google Scholar] - Sheng, J.L.; Zhou, M.S.; Guo, Z.P.; Liu, Z. Fault diagnosis for transformer based on fuzzy entropy. In Proceedings of the Annual Report-Conference on Electrical Insulation and Dielectric Phenomena, Vancouver, BC, Canada, 4–17 October 2007; pp. 759–762. [Google Scholar]
- Feng, F.; Rao, G.; Jiang, P.; Si, A. Research on early fault diagnosis for rolling bearing based on permutation entropy algorithm. In Proceedings of the 2012 IEEE Conference on Prognostics and System Health Management (PHM), Beijing, China, 23–25 May 2012; pp. 1–5. [Google Scholar]
- Costa, M.; Goldberger, A.L.; Peng, C.K. Multiscale entropy analysis of complex physiologic time series. Phys. Rev. Lett.
**2002**, 89, 068102. [Google Scholar] [CrossRef] [PubMed] - Jinde, Z.; Zhanwei, J.; Ziwei, P.; Kang, Z. VMD based adaptive multiscale fuzzy entropy and its application to rolling bearing fault diagnosis. In Proceedings of the 2016 10th International Conference on Sensing Technology (ICST), Nanjing, China, 11–13 November 2016; pp. 1–4. [Google Scholar]
- Zhang, L.; Xiong, G.; Liu, H.; Zou, H.; Guo, W. Bearing fault diagnosis using multi-scale entropy and adaptive neuro-fuzzy inference. Expert Syst. Appl.
**2010**, 37, 6077–6085. [Google Scholar] [CrossRef] - Zheng, J.; Cheng, J.; Yang, Y. Multiscale permutation entropy based rolling bearing fault diagnosis. Shock Vib.
**2014**, 1, 1–8. [Google Scholar] [CrossRef] - Zheng, J.; Cheng, J.; Yang, Y.; Luo, S. A rolling bearing fault diagnosis method based on multi-scale fuzzy entropy and variable predictive model-based class discrimination. Mech. Mach. Theory
**2014**, 78, 187–200. [Google Scholar] [CrossRef] - Li, P.; Liu, C.; Li, K.; Zheng, D.; Liu, C.; Hou, Y. Assessing the complexity of short-term heartbeat interval series by distribution entropy. Med. Biol. Eng. Comput.
**2015**, 53, 77–87. [Google Scholar] [CrossRef] [PubMed] - Maaten, L.V.D.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res.
**2008**, 9, 2579–2605. [Google Scholar] - Maaten, L.V.D. Accelerating t-SNE using tree-based algorithms. J. Mach. Learn. Res.
**2014**, 15, 3221–3245. [Google Scholar] - Kabla, A.; Mokrani, K. Bearing fault diagnosis using Hilbert-Huang transform (HHT) and support vector machine (SVM). Mech. Ind.
**2016**, 17, 308. [Google Scholar] [CrossRef] - Hwang, D.H.; Youn, Y.W.; Sun, J.H.; Choi, K.H.; Lee, J.H.; Kim, Y.H. Support vector machine based bearing fault diagnosis for induction motors using vibration signals. J. Electr. Eng. Technol.
**2015**, 10, 1558–1565. [Google Scholar] [CrossRef] - Huang, G.; Zhu, Q.; Siew, C. Extreme learning machine: Theory and applications. Neurocomputing
**2006**, 70, 489–501. [Google Scholar] [CrossRef] - Milačić, L.; Jović, S.; Vujović, T.; Miljković, J. Application of artificial neural network with extreme learning machine for economic growth estimation. Physical A
**2017**, 465, 285–288. [Google Scholar] [CrossRef] - Lei, Y.; He, Z.; Zi, Y. EEMD method and WNN for fault diagnosis of locomotive roller bearings. Expert Syst. Appl.
**2011**, 38, 7334–7341. [Google Scholar] [CrossRef] - Lei, Y.; He, Z.; Zi, Y. A new approach to intelligent fault diagnosis of rotating machinery. Expert Syst. Appl.
**2008**, 35, 1593–1600. [Google Scholar] [CrossRef] - Raghuraj, R.; Lakshminarayanan, S. Variable predictive models—A new multivariate classification approach for pattern recognition applications. Pattern Recogn.
**2009**, 42, 7–16. [Google Scholar] [CrossRef] - Tao, R.; Xu, Y.C.; Li, X.S.; Guo, S.; Li, K.; Gou, M. The Research of Fault Diagnosis Method of Roller Bearing Based on EMD and VPMCD. In Advanced Materials Research; Trans Tech Publications: Zürich, Switzerland, 2014; Volume1014, pp. 505–509. [Google Scholar]
- Cheng, J.S.; Xing-Wei, M.A.; Yu, Y. Rolling bearing fault diagnosis method based on permutation entropy and VPMCD. J. Vib. Shock
**2014**, 34, 802–806. [Google Scholar] - Kui, L.I.; Fan, Y.; Jiande, W.U. Research on bearing fault intelligent diagnosis method based on MRSVD and VPMCD. Comput. Eng. Appl.
**2016**, 52, 153–157. [Google Scholar] - Pan, H.Y.; Yang, Y.; Li, Y.G.; Cheng, J. The rolling bearings fault diagnosis method based on manifold learning and improved VPMCD. J. Vib. Eng.
**2014**, 27, 934–941. [Google Scholar] - Yang, Y.; Pan, H.; Ma, L.; Cheng, J. Applications of KVPMCD based on Kriging function in rolling bearing fault diagnosis. China Mech. Eng.
**2014**, 25, 2131–2136. [Google Scholar] - Fraser, A.M.; Swinney, H.L. Independent coordinates for strange attractors from mutual information. Phys. Rev. A
**1986**, 33, 1134. [Google Scholar] [CrossRef] - Albano, A.M.; Mees, A.I.; De Guzman, G.C.; Rapp, P.E. Data requirements for reliable estimation of correlation dimensions. In Chaos in Biological Systems; Springer: Berlin, Germany, 1987; pp. 207–220. [Google Scholar]
- Richman, J.S.; Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Heart Circ. Physiol.
**2000**, 278, H2039–H2049. [Google Scholar] [CrossRef] [PubMed] - Wu, S.D.; Wu, C.W.; Lee, K.Y.; Lin, S.G. Modified multiscale entropy for short-term time series analysis. Physical A
**2013**, 392, 5865–5873. [Google Scholar] [CrossRef] - Li, P.; Ji, L.; Yan, C.; Li, K.; Liu, C.; Liu, C. Coupling between short-term heart rate and diastolic period is reduced in heart failure patients as indicated by multivariate entropy analysis. In Proceedings of the Computing in Cardiology Conference (CinC), Cambridge, MA, USA, 7–10 September 2014; pp. 97–100. [Google Scholar]
- Li, P.; Liu, C.Y.; Li, L.P.; Ji, L.Z.; Yu, S.Y.; Liu, C.C. Multiscale multivariate fuzzy entropy analysis. Acta Phys. Sin.
**2013**, 62, 120512. [Google Scholar] - Bearing Data Center, Case Western Reserve University. Available online: http://csegroups.case.edu/bearingdatacenter/pages/download-data-file (accessed on 11 February 2012).

**Figure 3.**Comparison of standard deviations between MSE and MDE of white noise with data lengths (

**a**) N = 2000 and (

**b**) N = 10,000.

**Figure 4.**Comparison of standard deviations between MSE and MDE of 1/f noise with data lengths (

**a**) N = 2000 and (

**b**) N = 10,000.

**Figure 7.**(

**a**) Time domain waveforms and (

**b**) frequency spectrum of rolling bearing vibration signal under four different conditions.

**Figure 8.**MDE over 20 scales of signals shown in Figure 7.

**Figure 15.**Outputs of KVPMCD classifier with features consisting of DistEns in scales: 1st, 8th and 15th.

Methods | Accuracy Rate (%) |
---|---|

DistEn + SVM | 69.64 |

MDE (the first 8 scales) + SVM | 82.14 |

MDE (all 20 scales) + KVPMCD | 94.64 |

MDE (three DEs in 1, 8 and 15 scales) + KVPMCD | 89.29 |

MDE + PCA + KVPMCD | 96.43 |

MDE + t-SNE + KVPMCD | 100 |

MDE + t-SNE + VPMCD | 98.21 |

MDE + t-SNE + SVM | 87.50 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tu, D.; Zheng, J.; Jiang, Z.; Pan, H.
Multiscale Distribution Entropy and t-Distributed Stochastic Neighbor Embedding-Based Fault Diagnosis of Rolling Bearings. *Entropy* **2018**, *20*, 360.
https://doi.org/10.3390/e20050360

**AMA Style**

Tu D, Zheng J, Jiang Z, Pan H.
Multiscale Distribution Entropy and t-Distributed Stochastic Neighbor Embedding-Based Fault Diagnosis of Rolling Bearings. *Entropy*. 2018; 20(5):360.
https://doi.org/10.3390/e20050360

**Chicago/Turabian Style**

Tu, Deyu, Jinde Zheng, Zhanwei Jiang, and Haiyang Pan.
2018. "Multiscale Distribution Entropy and t-Distributed Stochastic Neighbor Embedding-Based Fault Diagnosis of Rolling Bearings" *Entropy* 20, no. 5: 360.
https://doi.org/10.3390/e20050360