Fault Detection Based on Multi-Dimensional KDE and Jensen–Shannon Divergence

Abstract

Weak fault signals, high coupling data, and unknown faults commonly exist in fault diagnosis systems, causing low detection and identification performance of fault diagnosis methods based on $T^2$ statistics or cross entropy. This paper proposes a new fault diagnosis method based on optimal bandwidth kernel density estimation (KDE) and Jensen–Shannon (JS) divergence distribution for improved fault detection performance. KDE addresses weak signal and coupling fault detection, and JS divergence addresses unknown fault detection. Firstly, the formula and algorithm of the optimal bandwidth of multidimensional KDE are presented, and the convergence of the algorithm is proved. Secondly, the difference in JS divergence between the data is obtained based on the optimal KDE and used for fault detection. Finally, the fault diagnosis experiment based on the bearing data from Case Western Reserve University Bearing Data Center is conducted. The results show that for known faults, the proposed method has $10\%$ and $2\%$ higher detection rate than $T^2$ statistics and the cross entropy method, respectively. For unknown faults, $T^2$ statistics cannot effectively detect faults, and the proposed method has approximately $15\%$ higher detection rate than the cross entropy method. Thus, the proposed method can effectively improve the fault detection rate.

1. Introduction

The development of industrial informatization has given rise to a large amount of data in various fields. This has led to data processing becoming a difficult problem in the industry, especially for fault diagnosis. The explosive growth of data provides more information, and therefore, typical data analysis theories often fail in achieving the necessary results. The main reason for this failure can be attributed to the typical data analysis theory that often sets the data distribution type through prior information and performs analyses based on this assumption. Once the distribution type is set, the subsequent analysis can perform the estimation and parametric analysis based on only that distribution type; however, with the growth of data, more information is provided, and thus, the type of data distribution will need to be modified. As a nonparametric estimation method, kernel density estimation (KDE) is the most suitable method for the massive amount of the current data. KDE does not employ a priori assumption for the overall data distribution, and it directly starts from the sample data. When the sample size is sufficient, the KDE can approximate different distributions. Furthermore, Sheather and Jones [1] provides the optimal bandwidth estimation formula for a one-dimensional KDE and proves that the kernel function is asymptotically unbiased and consistent in the density estimation. However, with the growth of the dimension, the multidimensional KDE becomes more complex, and its optimal bandwidth formula is not provided. The distribution of multidimensional data has been described to a certain extent by estimating the kernel density of the reduced data in different dimensions Muir [2], Laurent [3]. In fact, the optimal KDE of multidimensional data is a problem that needs to be studied further.

In the field of fault diagnosis, an essential problem is measuring the difference between samples. A frequency histogram has been used to indicate the distribution difference between two samples Sugumaran and Ramachandran [4], Scott [5]; however, there are three shortcomings to this method: (1) the large number of discrete operations require a higher amount of time; (2) the process depends on the selection of the interval, which is more subjective; (3) there is no intuitive index to reflect this difference. In fact, based on KDE, the JS divergence can be used to measure the difference in data distribution, which can overcome the above shortcomings to a certain extent. For example, the failure of a rolling bearing, which is a key component of mechanical equipment, will have a serious effect on the safe and stable operation of the equipment, and the incipient fault detection of rolling bearings can help avoid equipment running with faults and avoid causing serious safety accidents and economic losses, which has important practical and engineering significance.

In Saruhan et al. [6], vibration analysis of rolling element bearings (REBs) defects is studied. The REBs are the most widely used mechanical parts in rotating machinery under high load and high rotational speeds. In addition, characteristics of bearing faults are analyzed in detail in references Razavi-Far et al. [7], Harmouche et al. [8]. Compared with traditional fault diagnosis, the fault diagnosis of rolling bearings is more complex:

• The fault signal is weak: Bearing data is a type of high-frequency data, and the fault signal is often covered by these high-frequency signals, thereby leading to the failure of traditional fault diagnosis methods. KDE is highly accurate in describing data distribution, so it can identify weak signals.
• Data is highly coupled: Bearing data is reflected in the form of a vibration signal, and there is strong coupling in different dimension signals, thereby making fault diagnosis difficult. Multi-dimensional KDE plays an important role in depicting the correlation of data, which can characterize the relationship between different dimensions of data.
• Incomplete data set: Most bearings work under normal conditions, and the fault data collected are often fewer, which makes the data incomplete, thereby resulting in the imperfection of the fault data set and increasing the difficulty of fault detection. The fault detection method constructed by JS divergence can deal with unknown faults and incomplete data sets without using additional data sets.

To overcome these problems, in-depth research has been conducted on this topic. Fault detection technology based on trend elimination and noise reduction has been proposed previously He et al. [9], Demetriou and Polycarpou [10]. The signal trend ratio is enhanced by eliminating the trend, and the signal–noise ratio is enhanced by noise reduction, and therefore, the fault detection effect is improved. However, this method uses the traditional detection method and cannot effectively solve the problem of data coupling. In reference Zhang et al. [11], Fu et al. [12], a fault detection method based on PCA dimension reduction and modal decomposition feature extraction is proposed. For multidimensional data, PCA dimension reduction is performed to reduce data dimensions and eliminate correlation between different dimensions. Then, the modal decomposition method is used to extract features among dimensions for fault detection. This method can effectively solve the strong coupling between data; however, it will lose some information in the process of PCA dimension reduction, and it leads to a reduction in the fault detection effect. In reference Itani et al. [13], Kong et al. [14], Jones and Sheather [15], Desforges et al. [16], a bearing fault detection method based on KDE is proposed. These studies analyzed the feasibility of KDE method in fault detection, and combined different classification methods for experiments. However, these methods only use one-dimensional KDE, and cannot directly describe high-dimensional data.

The data distribution is reconstructed by KDE and the cross-entropy function is constructed to measure the distribution difference for improving the fault detection results. However, this method cannot reflect the correlation between different dimensions, and the cross-entropy function is not precise in the description of density distribution, which leads to a reduction in the fault detection effect, especially for unknown fault detection, which is not included in the fault set.

In this study, the KDE method is extended to multidimensional data to avoid information loss caused by the KDE for each dimension, and to better describe the density probability distribution of the data. Meanwhile, this study improves the traditional method using the cross-entropy function as the measurement of density distribution difference, and it uses JS divergence as the measurement of density distribution difference, thereby avoiding the relativity caused by the cross-entropy function. Most fault identification methods are based only on distance measurement; however, only relying on distance measurement cannot effectively detect unknown faults. Based on JS divergence, distribution characteristics of JS divergence between the sample density distribution and population density distribution are derived using the sliding window principle. Thus, the detection threshold of fault identification is assigned to realize the identification of unknown faults.

This paper is based on the following structure. In Section 2, the trend elimination method and detection method are introduced, and the intrinsic and extrinsic signals in the observation data are separated. Then, the fault detection threshold is constructed via statistics. In Section 3, the KDE method is extended to multidimensional data, and the optimal bandwidth is derived. Then, JS divergence is employed to measure the difference between probability distributions of different densities. In Section 4, the sliding window principle is used to sample the training data to obtain the distribution characteristics of JS divergence between the sample density distribution and the overall density distribution, and the detection threshold of fault identification is obtained using the KDE method. In Section 5, the normal data, two known faults, and one unknown fault are identified using the bearing data of the Case Western Reserve University Bearing Data Center as the fault diagnosis data. The experimental results show that the method can identify all types of faults well.

2. ${\mathit{T}}^2$ Statistics Fault Detection

In the operation process of the complex equipment or systems, the common observation state can be divided into intrinsic and extrinsic parts. In general, the intrinsic part represents the main working state of the system, which has a certain trend, monotony, and periodicity. The extrinsic part represents system noise, which has a certain zero mean value, high frequency vibration, and statistical stability. For the intrinsic part, the state equation of the system can be used to describe the law. When a fault occurs in the intrinsic part, the symptoms are relatively significant, and the corresponding fault detection methods are relatively mature. However, for high-frequency vibration signals, the incipient fault is often hidden in the extrinsic part, which is easily covered by noise. Therefore, it is necessary to analyze the observed data in depth.

2.1. Signal Decomposition

In the initial operation stage of the equipment, the unstable operation of the system causes large data fluctuations, which will not only have a great effect on the system trend, but also affect the statistical characteristics of the data. Therefore, it is necessary to truncate the data to remove unstable signals [9]. The corresponding time of the time series after removing the nonstationary period data is $t_1,t_2,\dots ,t_m$, and the following m observation data are obtained:

$$\mathit{Y}=\left[\mathit{y}\left(t_1\right),\mathit{y}\left(t_2\right),\dots ,\mathit{y}\left(t_m\right)\right].$$

(1)

Each sampling $\mathit{y}\left(t_i\right)$ contains n features, which are expressed as components in the form of

$$\mathit{y}\left(t_i\right)={\left[y_1\left(t_i\right),y_2\left(t_i\right),\dots ,y_n\left(t_i\right)\right]}^{\mathrm{T}},i=1,2,\dots ,m.$$

(2)

Then, the data $\mathit{Y}$ can be decomposed into

$$\mathit{Y}=\widehat{\mathit{Y}}+\mathit{R},$$

(3)

where $\widehat{\mathit{Y}}$ denotes the intrinsic part, which is composed of trend, and $\mathit{R}$ denotes the extrinsic part, which is composed of observation noise and fault data.

The intrinsic part is composed of multiple signals. Selecting the appropriate basis function $\mathit{f}\left(\mathrm{t}\right)={\left[f_1\left(t\right),f_2\left(t\right),\dots ,f_s\left(t\right)\right]}^{\mathrm{T}}$ can help describe the intrinsic part. By traversing m data to model the nonlinear data $\mathit{Y}$,

$$\left[{\mathit{y}}_1,{\mathit{y}}_2,\dots ,{\mathit{y}}_m\right]=\left[\begin{array}{cccc}{\beta }_1^{\left(1\right)}\hfill & {\beta }_2^{\left(1\right)}& \dots & {\beta }_s^{\left(1\right)}\\ {\beta }_1^{\left(2\right)}\hfill & {\beta }_2^{\left(2\right)}& \dots & {\beta }_s^{\left(2\right)}\\ ⋮\hfill & ⋮& \ddots & ⋮\\ {\beta }_1^{\left(n\right)}\hfill & {\beta }_2^{\left(n\right)}& \dots & {\beta }_s^{\left(n\right)}\end{array}\right]\left[\begin{array}{cccc}{f}_0\left(t_1\right)\hfill & f_0\left(t_2\right)& \dots & f_0\left(t_m\right)\\ f_1\left(t_1\right)\hfill & f_1\left(t_2\right)& \dots & f_1\left(t_m\right)\\ ⋮\hfill & ⋮& \ddots & ⋮\\ f_s\left(t_1\right)\hfill & f_s\left(t_2\right)& \dots & f_s\left(t_m\right)\end{array}\right].$$

(4)

Note that

$$\mathit{F}\stackrel{\mathsf{\Delta }}{=}\left[\begin{array}{cccc}{f}_0\left(t_1\right)\hfill & f_0\left(t_2\right)& \dots & f_0\left(t_m\right)\\ f_1\left(t_1\right)\hfill & f_1\left(t_2\right)& \dots & f_1\left(t_m\right)\\ ⋮\hfill & ⋮& \ddots & ⋮\\ f_s\left(t_1\right)\hfill & f_s\left(t_2\right)& \dots & f_s\left(t_m\right)\end{array}\right],\mathit{\beta }\stackrel{\mathsf{\Delta }}{=}\left[\begin{array}{cccc}{\beta }_1^{\left(1\right)}\hfill & {\beta }_2^{\left(1\right)}& \dots & {\beta }_s^{\left(1\right)}\\ {\beta }_1^{\left(2\right)}\hfill & {\beta }_2^{\left(2\right)}& \dots & {\beta }_s^{\left(2\right)}\\ ⋮\hfill & ⋮& \ddots & ⋮\\ {\beta }_1^{\left(n\right)}\hfill & {\beta }_2^{\left(n\right)}& \dots & {\beta }_s^{\left(n\right)}\end{array}\right]$$

(5)

Then, Equation (4) can be expressed as

$$\mathit{Y}=\mathit{\beta }\mathit{F}.$$

(6)

Thus, the efficient estimator of $\mathit{\beta }$ is

$$\widehat{\mathit{\beta }}=\mathit{Y}{\mathit{F}}^{\mathrm{T}}{\left(\mathit{F}{\mathit{F}}^{\mathrm{T}}\right)}^{-1}.$$

(7)

Using Equations (3) and (7), the signal can be decomposed into

$$\left\{\begin{array}{cc}\hfill & \widehat{\mathit{Y}}=\widehat{\mathit{\beta }}\mathit{F}=\mathit{Y}{\mathit{F}}^{\mathrm{T}}{\left(\mathit{F}{\mathit{F}}^{\mathrm{T}}\right)}^{-1}\mathit{F}\hfill \\ \hfill & \mathit{R}=\mathit{Y}-\widehat{\mathit{Y}}=\mathit{Y}\left(\mathit{I}-{\mathit{F}}^{\mathrm{T}}{\left(\mathit{F}{\mathit{F}}^{\mathrm{T}}\right)}^{-1}\mathit{F}\right)\hfill \end{array}\right.$$

(8)

Usually, the choice of the basis function is a problem worthy of discussion, and it depends on prior knowledge of practical application scenarios; however, this is not the focus of this paper, and is therefore not covered here.

Remark 1.

For the bearing data, the data is generally stable and periodic. Therefore, Fourier transform is usually used to extract periodic features instead of more complex basis functions, such as a polynomial basis function and wavelet basis function.

2.2. $T^2$ Statistics Detection

For simplicity, remember ${\mathit{r}}_i=\mathit{r}\left(t_i\right),i=1,2,\dots ,m$. According to Equation (8), the training data after signal decomposition are $\mathit{R}=\left[{\mathit{r}}_1,{\mathit{r}}_2,\dots ,{\mathit{r}}_m\right]$, which is generally considered a normal random vector with expectation of $\mathbf{0}$, so that

$${\mathit{r}}_i\sim N\left(\mathbf{0},\mathsf{\Sigma }\right),$$

(9)

where $\mathsf{\Sigma }$ denotes the total covariance matrix. When the covariance matrix $\mathsf{\Sigma }$ is unknown, the unbiased estimation is given by

$$\widehat{\mathsf{\Sigma }}=\frac{\mathit{R}{\mathit{R}}^{\mathrm{T}}}{m-1}.$$

(10)

Let $\mathit{Z}=\left[{\mathit{z}}_1,{\mathit{z}}_2,\dots ,{\mathit{z}}_p\right]$ be the data in the test window to be tested; the sample mean value $\overline{\mathit{z}}$ is

$$\overline{\mathit{z}}=\frac{1}{p}\sum _{i=1}^p{\mathit{z}}_i.$$

(11)

Then, $\overline{\mathit{z}}$ is still normal distributed and

$$\overline{\mathit{z}}\sim N\left(\mathbf{0},\frac{1}{p}\mathsf{\Sigma }\right).$$

(12)

The $T^2$ statistics can be constructed as

$$T^2=p{\overline{\mathit{z}}}^{\mathrm{T}}{\widehat{\mathsf{\Sigma }}}^{-1}\overline{\mathit{z}}.$$

(13)

Reference Solomons and Hotelling [17] reports that the distribution of the $T^2$ statistic satisfies

$$\frac{m-n}{n\left(m-1\right)}{T}^2=\frac{p\left(m-n\right)}{n\left(m-1\right)}{\overline{\mathit{z}}}^{\mathrm{T}}{\widehat{\mathsf{\Sigma }}}^{-1}\overline{\mathit{z}}\sim F\left(n,m-n\right).$$

(14)

Therefore, if the significance level is $\alpha$, we can get that

$$\frac{m-n}{n\left(m-1\right)}{T}^2=\frac{l\left(m-n\right)}{n\left(m-1\right)}{\overline{\mathit{z}}}^{\mathrm{T}}{\widehat{\mathsf{\Sigma }}}^{-1}\overline{\mathit{z}}<F_{\alpha }\left(n,m-n\right).$$

(15)

The testing data $\mathit{Z}$ and the training data $\mathit{R}$ both come from the same mode; otherwise, they are considered different. The error rate of this criterion is $\alpha$.

3. Optimal Kernel Density Estimation

Section 2 introduces the fault detection method based on $T^2$ statistics, including the signal decomposition technology and fault detection method based on the $T^2$ statistics. However, the fault detection method based on the $T^2$ statistics assumes that data satisfies the normal distribution, while the actual observation data may not meet the hypothesis, which can lead the discriminant performance of the $T^2$ statistics to not satisfy the design requirements. In addition, the statistics test the data from the angle of the intrinsic part $\widehat{\mathit{Y}}$ and covariance matrix $\widehat{\mathsf{\Sigma }}$. These two attributes are not sufficient to describe all statistical characteristics of the system. When the incipient fault is submerged by data noise, it is easy to miss the detection. In this study, a KDE method for multidimensional data is constructed to describe the probability and statistical characteristics of the data more accurately.

3.1. Optimal Bandwidth Theorem

For the observed data, the frequency histogram can be used to show its statistical characteristics directly. However, in the actual application process, the frequency histogram is a discrete statistical method, the interval number of the histogram is difficult to divide, and more importantly, the discretization operation inconveniences the subsequent data processing. To overcome these limitations, the KDE method is proposed. This method is a nonparametric estimation method that estimates the population probability density distribution directly by sampling data.

For any point $\mathit{x}\in {\mathbb{R}}^n$, assuming that the probability density of a certain mode is $f\left(\mathit{x}\right)$, the kernel density of $f\left(\mathit{x}\right)$ is estimated based on the sampling data $\mathit{R}=\left[{\mathit{r}}_1,{\mathit{r}}_2,\dots ,{\mathit{r}}_m\right]$ in Section 2.1. As reported in reference Rao [18], the estimation formula is

$${\widehat{f}}_K\left(\mathit{x},h_m\right)=\frac{1}{m{h}_m^n}\sum _{i=1}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right),$$

(16)

where $m,$ $n,$ $K\left(·\right),$ and $h_m$ denote the number of sampling data, dimension of sampling data, kernel function, and bandwidth, respectively.

For the sake of convenience in the following discussions, in the case of no doubt,

$$\left\{\begin{array}{cc}\hfill & {\widehat{f}}_K\left(\mathit{x}\right)\triangleq {\widehat{f}}_K\left(\mathit{x},h_m\right)\hfill \\ \hfill & \int g\left(\mathit{x}\right)d\mathit{x}\triangleq {\int }_{\mathit{x}\in {\mathbb{R}}^n}g\left(\mathit{x}\right)d\mathit{x}\hfill \end{array}\right.$$

(17)

The kernel function $K\left(·\right)$ satisfies $\int K\left(\mathit{x}\right)d\mathit{x}=1$; therefore, $\int K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)d\mathit{x}=h_m^n$, that is, $\int {\widehat{f}}_K\left(\mathit{x}\right)d\mathit{x}=1$. Thus, ${\widehat{f}}_K\left(\mathit{x}\right)$ satisfies both positive definiteness, continuity, and normality. Therefore, it is reasonable to use it as the KDE. The Gaussian kernel function is a good choice as given by

$$K\left(\mathit{x}\right)={\left(2\pi \right)}^{-n/\phantom{n2}2}{e}^{-\left({\mathit{x}}^{\mathrm{T}}\mathit{x}\right)/\phantom{\left({\mathit{x}}^T\mathit{x}\right)2}2}$$

(18)

In this study, the performance of the kernel density estimator is characterized by the mean integral square error (MISE).

$$\mathrm{MISE}\left({\widehat{f}}_K\left(\mathit{x}\right)\right)=\int \mathrm{E}{\left[{\widehat{f}}_K\left(\mathit{x}\right)-f\left(\mathit{x}\right)\right]}^{2}d\mathit{x}$$

(19)

Reference Rao [18] shows that the estimation result ${\widehat{f}}_K\left(\mathit{x}\right)$ is not sensitive to the selection of the kernel function $K\left(·\right)$; that is, the MISE of the estimation results obtained using different kernel functions is almost the same, which is reflected in the subsequent derivation process. In addition, the MISE depends on the selection of the bandwidth $h_m$. If $h_m$ is too small, the density estimation ${\widehat{f}}_K\left(\mathit{x}\right)$ shows an irregular shape because of the increase in the randomness. While $h_m$ is too large, density estimation ${\widehat{f}}_K\left(\mathit{x}\right)$ is too averaged to show sufficient detail.

The optimal bandwidth formula is provided in the following theorem, and it is one of the key theoretical results of this study.

Theorem 1.

For any dimensional probability density function $f\left(·\right)$ and any kernel function $K\left(·\right)$ with a symmetric form, if ${\widehat{f}}_K\left(·\right)$ in Equation (16) is used to estimate $f\left(·\right)$, and if the function $\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)$ with respect to $\mathit{x}$ is integrable when the $\mathrm{MISE}\left({\widehat{f}}_K\left(·\right)\right)$ in Equation (19) is the minimum, the bandwidth $h_m$ satisfies

$$h_m={\left(\frac{m{d}_K^2}{n^3{c}_K}\int tr{\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)}^{2}d\mathit{x}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)},$$

(20)

where $c_K$ and $d_K$ are two constant values given by

$$\left\{\begin{array}{cc}\hfill & c_K=\int K^2\left(\mathit{x}\right)d\mathit{x}\hfill \\ \hfill & d_K=\int {\mathit{x}}^{\mathrm{T}}\mathit{x}{K}^2\left(\mathit{x}\right)d\mathit{x}\hfill \end{array}\right.$$

(21)

Equation (20) is called the optimal bandwidth formula and $h_m$ denotes the optimal bandwidth.

A detailed proof of this theorem is given below.

Proof. 

It can be proved that the following two equations hold

$$\left\{\begin{array}{cc}\hfill & \mathrm{E}\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]=\int K\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}\hfill \\ \hfill & \mathrm{E}\left[{\widehat{f}}_K^2\left(\mathit{x}\right)\right]=\frac{\int K^2\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}}{m{h}_m^n}+\frac{\left(m-1\right){\left(\int K\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}\right)}^2}{m}\hfill \end{array}\right.$$

(22)

In fact,

$$\begin{array}{cc}\hfill \mathrm{E}\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]& =\int \dots \int \prod _{i=1}^{m}f\left({\mathit{r}}_i\right)\frac{1}{m{h}_m^n}\sum _{i=1}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)d{\mathit{r}}_m\dots d{\mathit{r}}_1\hfill \\ \hfill & =\frac{1}{m{h}_m^n}\sum _{i=1}^m\int f\left(\mathit{r}\right)K\left(\frac{\mathit{r}-\mathit{x}}{h_m}\right)d\mathit{r}\hfill \\ \hfill & =\int f\left(\mathit{x}+h_m\mathit{u}\right)K\left(\mathit{u}\right)d\mathit{u}.\hfill \end{array}$$

(23)

In addition,

$$\begin{array}{c}\mathrm{E}\left[{\widehat{f}}_K^2\left(\mathit{x}\right)\right]=\int \dots \int \prod _{i=1}^{m}f\left({\mathit{r}}_i\right){\left({\left(m{h}_m^n\right)}^{-1}\sum _{i=1}^{m}f\left({\mathit{r}}_i\right)K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)\right)}^{2}d{\mathit{r}}_1\dots d{\mathit{r}}_m\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\int \dots \int \prod _{i=1}^{m}f\left({\mathit{r}}_i\right){\left(\sum _{i=1}^{m}f\left({\mathit{r}}_i\right)K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)\right)}^{2}d{\mathit{r}}_1\dots d{\mathit{r}}_m\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\int \dots \int \prod _{i=1}^{m}f\left({\mathit{r}}_i\right)\left(\sum _{i=1}^m{K}^2\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)+\sum _{i\ne j}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)K\left(\frac{{\mathit{r}}_j-\mathit{x}}{h_m}\right)\right)d{\mathit{r}}_1\dots d{\mathit{r}}_m\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\int \dots \int \left(\prod _{i=1}^{m}f\left({\mathit{r}}_i\right)\sum _{i=1}^m{K}^2\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)+\prod _{i=1}^{m}f\left({\mathit{r}}_i\right)\sum _{i\ne j}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)K\left(\frac{{\mathit{r}}_j-\mathit{x}}{h_m}\right)\right)d{\mathit{r}}_1\dots d{\mathit{r}}_m\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\left(\sum _{i=1}^m\int f\left({\mathit{r}}_i\right)K^2\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)d\mathit{r}+\sum _{i\ne j}^m\int \int \left(f\left({\mathit{r}}_i\right)K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)f\left({\mathit{r}}_j\right)K\left(\frac{{\mathit{r}}_j-\mathit{x}}{h_m}\right)\right)d{\mathit{r}}_{i}d{\mathit{r}}_j\right)\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\left(m\int f\left(\mathit{r}\right)K^2\left(\frac{\mathit{r}-\mathit{x}}{h_m}\right)d\mathit{r}+m\left(m-1\right){\left(\int f\left(\mathit{r}\right)K\left(\frac{\mathit{r}-\mathit{x}}{h_m}\right)d\mathit{r}\right)}^2\right)\hfill \\ ={\left(m{h}_m^n\right)}^{-2}\left(m{h}_m^n\int K^2\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}+m\left(m-1\right){\left(h_m^n\int f\left(\mathit{x}+h_m\mathit{u}\right)K\left(\mathit{u}\right)d\mathit{u}\right)}^2\right).\hfill \end{array}$$

(24)

From Equation (23),

$$E\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]-f\left(\mathit{x}\right)=\frac{h_m^2}{2}\int {\mathit{u}}^{\mathrm{T}}\left(\frac{{\partial }^{2}f\left(\mathit{x}+\theta h_m\mathit{u}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\mathit{u}K\left(\mathit{u}\right)d\mathit{u},$$

(25)

where $\theta$ represents a constant value between 0 and 1. According to Equations (23) and (24),

$$E\left[{\widehat{f}}_K^2\left(\mathit{x}\right)\right]-{\left(E\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]\right)}^2=\frac{\int K^2\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}}{m{h}_m^n}-\frac{{\left(\int K\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}\right)}^2}{m}.$$

(26)

According to the Equations (25) and (26), the following equation holds.

$$\begin{array}{cc}\hfill E{\left[{\widehat{f}}_K\left(\mathit{x}\right)-f\left(\mathit{x}\right)\right]}^2& =E\left[{\widehat{f}}_K^2\left(\mathit{x}\right)\right]-{\left(E\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]\right)}^2+{\left(E\left[{\widehat{f}}_K\left(\mathit{x}\right)\right]-f\left(\mathit{x}\right)\right)}^2\hfill \\ \hfill & =\frac{\int K^2\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}}{m{h}_m^n}-\frac{{\left(\int K\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}\right)}^2}{m}\hfill \\ \hfill & +{\left(\frac{1}{2}{h}_m^2\int {\mathit{u}}^{\mathrm{T}}\left(\frac{{\partial }^{2}f\left(\mathit{x}+\theta h_m\mathit{u}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\mathit{u}K\left(\mathit{u}\right)d\mathit{u}\right)}^2\hfill \end{array}$$

(27)

To facilitate the subsequent reasoning, the following theorem is given.

Theorem 2.

For any matrixΦ, $K\left(·\right)$ is a kernel density function with symmetric form; then,

$$\int {\mathit{x}}^{\mathrm{T}}\mathsf{\Phi }\mathit{x}K\left(\mathit{x}\right)d\mathit{x}=\frac{tr\left(\mathsf{\Phi }\right)}{n}\int {\mathit{x}}^{\mathrm{T}}\mathit{x}K\left(\mathit{x}\right)d\mathit{x}.$$

(28)

Proof. 

If the odd function $g\left(x\right)$ is integrable on $\mathbb{R}$, there must be ${\int }_{-\infty }^{\infty }g\left(x\right)dx=0$. Similarly, it can be verified that the kernel function $K\left(·\right)$ with a symmetric form satisfies

$$\int \dots \int \sum _{i\ne j}{\mathsf{\Phi }}_{ij}{x}_i{x}_{j}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n=0.$$

(29)

Then,

$$\begin{array}{cc}\hfill & \int {\mathit{x}}^{\mathrm{T}}\mathsf{\Phi }\mathit{x}K\left(\mathit{x}\right)d\mathit{x}=\int \dots \int {\mathit{x}}^{\mathrm{T}}\mathsf{\Phi }\mathit{x}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n\hfill \\ \hfill & =\int \dots \int \sum _i{\mathsf{\Phi }}_{ii}{x}_i^{2}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n+\int \dots \int \sum _{i\ne j}{\mathsf{\Phi }}_{ij}{x}_i{x}_{j}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n\hfill \\ \hfill & =\mathrm{tr}\left(\mathsf{\Phi }\right)\int \dots \int x_1^{2}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n\hfill \\ \hfill & =\frac{tr\left(\mathsf{\Phi }\right)}{n}\int \dots \int {\mathit{x}}^{\mathrm{T}}\mathit{x}K\left(\mathit{x}\right)d{x}_1\dots d{x}_n\hfill \\ \hfill & =\frac{tr\left(\mathsf{\Phi }\right)}{n}\int {\mathit{x}}^{\mathrm{T}}\mathit{x}K\left(\mathit{x}\right)d\mathit{x}.\hfill \end{array}$$

(30)

Thus, the Theorem 2 is proved. □

For any unit length vector $\mathit{u}\in {\mathbb{R}}^n$, the Taylor expansion can be used to obtain

$$\left\{\begin{array}{cc}\hfill & f\left(\mathit{x}+h_m\mathit{u}\right)=f\left(\mathit{x}\right)+h_m{\mathit{u}}^{\mathrm{T}}\nabla \left(f\left(\mathit{x}\right)\right)+o\left(h_m\right)\hfill \\ \hfill & \frac{{\partial }^{2}f\left(\mathit{x}+\theta h_m\mathit{u}\right)}{\partial x_i\partial x_j}=\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial x_i\partial x_j}+\theta h_m{\mathit{u}}^{\mathrm{T}}\nabla \left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial x_i\partial x_j}\right)+o\left(h_m\right)\hfill \end{array}\right.$$

(31)

If the bandwidth $h_m$ satisfies the condition

$$\left\{\begin{array}{cc}\hfill & \underset{m\to \infty }{lim}\left(h_m\right)=0,\hfill \\ \hfill & \underset{m\to \infty }{lim}\left(\frac{1}{m{h}_m^n}\right)=0,\hfill \end{array}\right.$$

(32)

Then, from Equations (22)–(32), we get that

$$\mathrm{E}{\left[{\widehat{f}}_K\left(\mathit{x}\right)-f\left(\mathit{x}\right)\right]}^2=\frac{c_{K}f\left(\mathit{x}\right)}{m{h}_m^n}+o\left(\frac{1}{m{h}_m^n}\right)+\frac{h_m^4{d}_K^2}{4n^2}{\left(\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\right)}^2+o\left(h_m^4\right).$$

(33)

In fact,

$$\begin{array}{cc}& \begin{array}{cc}\hfill \mathrm{E}{\left[{\widehat{f}}_K\left(\mathit{x}\right)-f\left(\mathit{x}\right)\right]}^2& =\frac{\int K^2\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}}{m{h}_m^n}-\frac{{\left(\int K\left(\mathit{u}\right)f\left(\mathit{x}+h_m\mathit{u}\right)d\mathit{u}\right)}^2}{m}\hfill \\ \hfill & +{\left(\frac{h_m^2}{2}\int {\mathit{u}}^{\mathrm{T}}\left(\frac{{\partial }^{2}f\left(\mathit{x}+\theta h_m\mathit{u}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\mathit{u}K\left(\mathit{u}\right)d\mathit{u}\right)}^2\hfill \end{array}\hfill \\ & =\frac{c_{K}f\left(\mathit{x}\right)}{m{h}_m^n}+o\left(\frac{1}{m{h}_m^n}\right)-\frac{f{\left(\mathit{x}\right)}^2}{m}+o\left(\frac{1}{m}\right)+{\left(\frac{h_m^2}{2n}\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}+\theta h_m\mathit{u}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\int {\mathit{u}}^{\mathrm{T}}\mathit{u}K\left(\mathit{u}\right)d\mathit{u}\right)}^2\hfill \\ & =\frac{c_{K}f\left(\mathit{x}\right)}{m{h}_m^n}+o\left(\frac{1}{m{h}_m^n}\right)+{\left(\frac{h_m^2}{2n}\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)d_K+o\left(h_m^2\right)\right)}^2\hfill \\ & =\frac{c_{K}f\left(\mathit{x}\right)}{m{h}_m^n}+o\left(\frac{1}{m{h}_m^n}\right)+\frac{h_m^4{d}_K^2}{4n^2}{\left(\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\right)}^2+o\left(h_m^4\right).\hfill \end{array}$$

(34)

Based on Equation (33), if $\mathrm{tr}\left(\frac{{\partial }^{2}f\left(x\right)}{\partial x\partial x^{\mathrm{T}}}\right)$ is integrable, there is

$$\begin{array}{cc}\hfill \mathrm{MISE}\left({\widehat{f}}_K\left(\mathit{x}\right)\right)& =\int \left(\frac{c_{K}f\left(\mathit{x}\right)}{m{h}_m^n}+\frac{h_m^4}{4n^2}{\left(d_K\mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)\right)}^2\right)d\mathit{x}+o\left(\frac{1}{m{h}_m^n}\right)+o\left(h_m\right)\hfill \\ \hfill & =\frac{c_K}{m{h}_m^n}+\frac{1}{4n^2}{h}_m^4{d}_K^2\int \mathrm{tr}{\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)}^{2}d\mathit{x}+o\left(\frac{1}{m{h}_m^n}\right)+o\left(h_m\right).\hfill \end{array}$$

(35)

When $\mathrm{MISE}\left({\widehat{f}}_K\left(·\right)\right)$ is the smallest, the derivative of Equation (35) with respect to $h_m$ is 0, which means

$$\frac{\partial MISE\left({\widehat{f}}_K\left(\mathit{x}\right)\right)}{\partial h_m}=0.$$

(36)

Thus, the optimal bandwidth $h_m$ in Theorem 1 is obtained as

$$h_m={\left(\frac{m{d}_K^2}{n^3{c}_K}\int tr{\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)}^{2}d\mathit{x}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}.$$

(37)

 □

Remark 2.

When the number of samples m is determined, the appropriate bandwidth $h_m$ can be selected using Equation (20) to construct the KDE, which can better fit the sample distribution. In Equation (20), the influence of the kernel function on bandwidth selection is on $c_K$ and $d_K$, which are almost the same under different kernel function selection, and they have a slight effect on the final bandwidth selection.

3.2. Optimal Bandwidth Algorithm

The optimal bandwidth formula is given by Equation (20). However, $f\left(\mathit{x}\right)$ is unknown in Equation (20), and therefore, $\int \mathrm{tr}\left(\frac{{\partial }^{2}f\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)d\mathit{x}$ is also unknown. An approximate value of the bandwidth parameter $h_m$ can be obtained by replacing $f\left(\mathit{x}\right)$ with ${\widehat{f}}_K\left(\mathit{x}\right)$ in Equation (16). Furthermore, an iterative algorithm can be used to calculate a more accurate bandwidth parameter. Theorem 3 shows that the algorithm is convergent.

Theorem 3.

For any n-dimensional probability density function $f\left(·\right)$ and Gaussian kernel function $K\left(·\right)$, if ${\widehat{f}}_K\left(·\right)$ in Equation (16) is used to estimate $f\left(·\right)$, then the iterative calculation formula of $h_m$ is obtained as

$$h_{m,k+1}={\left(\frac{m{d}_K^2}{n^3{c}_K}\int \mathrm{tr}{\left(\frac{{\partial }^2{\widehat{f}}_K\left(\mathit{x},h_{m,k}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)}^{2}d\mathit{x}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}$$

(38)

and it is convergent, where $h_{m,k}$ is the value of $h_m$ during the $k-\mathrm{th}$ iteration.

Proof. 

For a particular Gaussian kernel function

$$K\left(\mathit{u}\right)={\left(2\pi \right)}^{-n/\phantom{n2}2}{e}^{-\left({\mathit{u}}^{\mathrm{T}}\mathit{u}\right)/\phantom{\left({\mathit{u}}^{\mathrm{T}}\mathit{u}\right)2}2}$$

(39)

$d_K$ is a ${\chi }^2$ distribution with degree of freedom n, and the expectation is equal to the degree of freedom.

$$d_K=\int {\mathit{u}}^{\mathrm{T}}\mathit{u}K\left(\mathit{u}\right)d\mathit{u}=n$$

(40)

In addition,

$$c_K=\int K^2\left(\mathit{u}\right)d\mathit{u}=\int {\left(2\pi \right)}^{-n}{e}^{-{\mathit{u}}^{\mathrm{T}}\mathit{u}}d\mathit{u}={\left(2\sqrt{\pi }\right)}^{-n}.$$

(41)

Substituting Equations (39)–(40) into Equation (20) and substituting ${\widehat{f}}_K\left(x\right)$ in Equation (16) for $f\left(x\right)$, the iterative form of calculating $h_m$ is obtained as

$$\begin{array}{cc}\hfill & h_{m,k+1}={\left(\frac{n}{m}\right)}^{1/\phantom{1\left(n+4\right)}\left(n+4\right)}{\left(2\sqrt{\pi }\right)}^{-n/\phantom{n\left(n+4\right)}\left(n+4\right)}{\left(\int \mathrm{tr}{\left(\frac{{\partial }^2{\widehat{f}}_K\left(\mathit{x}\right)}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\right)}^{2}d\mathit{x}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}\hfill \\ \hfill & ={\left(mn{h}_{m,k}^{2n}\right)}^{1/\phantom{1\left(n+4\right)}\left(n+4\right)}{\left(2\sqrt{\pi }\right)}^{-n/\phantom{n\left(n+4\right)}\left(n+4\right)}{\left(\int \mathrm{tr}{\left(\frac{{\partial }^2}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\left(\sum _{i=1}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_{m,k}}\right)\right)\right)}^{2}d\mathit{x}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}\hfill \end{array}$$

(42)

To facilitate the subsequent reasoning, the following lemma is given as

Lemma 1.

For any function $f_1,f_2,\dots ,f_n$, inequality

$$\int {\left(f_1+f_2+\dots +f_n\right)}^{2}d\mathit{x}\le \int n\left(f_1^2+f_2^2+\dots +f_n^2\right)d\mathit{x}.$$

(43)

If and only if $f_1\left(x\right)=f_2\left(x\right)=\dots =f_n\left(x\right)$ holds almost everywhere.

Proof. 

In fact, for any function $f_1,f_2,\dots ,f_n$, there are

$$0\le {\left(f_1\left(\mathit{x}\right)+f_2\left(\mathit{x}\right)+\dots +f_n\left(\mathit{x}\right)\right)}^2\le n\left(f_1{\left(\mathit{x}\right)}^2+f_2{\left(\mathit{x}\right)}^2+\dots +f_n{\left(\mathit{x}\right)}^2\right).$$

(44)

Thus, the two sides of Equation (44) are integrated as

$$\int {\left(f_1+f_2+\dots +f_n\right)}^{2}d\mathit{x}\le \int n\left(f_1^2+f_2^2+\dots +f_n^2\right)d\mathit{x}.$$

(45)

It is obvious that the sign of Equation (43) holds the condition that $f_1\left(x\right)=f_2\left(x\right)=\dots =f_n\left(x\right)$ is almost everywhere. □

Because the second derivative of Equation (39) with respect to $x_i$ is

$$\frac{{\partial }^2}{\partial x_i\partial x_i}K\left(\mathit{x}\right)={\left(2\pi \right)}^{-n/\phantom{n2}2}{e}^{-\left({\mathit{x}}^{\mathrm{T}}\mathit{x}\right)/\phantom{\left({\mathit{x}}^{\mathrm{T}}\mathit{x}\right)2}2}\left(x_i^2-1\right).$$

(46)

In addition,

$$\begin{array}{cc}\hfill \int {\left(\frac{{\partial }^2}{\partial x_j\partial x_j}\left(K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_{m,k}}\right)\right)\right)}^{2}d\mathit{x}& =\int {\left(\frac{{\partial }^2}{\partial x_j\partial x_j}\left({\left(2\pi \right)}^{-n/\phantom{n2}2}{e}^{-{\left({\mathit{r}}_i-\mathit{x}\right)}^{\mathrm{T}}\left({\mathit{r}}_i-\mathit{x}\right)/\phantom{{\left({\mathit{r}}_i-\mathit{x}\right)}^T\left({\mathit{r}}_i-\mathit{x}\right)2h_{m,k}^2}2h_{m,k}^2}\right)\right)}^{2}d\mathit{x}\hfill \\ \hfill & =\frac{3}{4}{\left(2\sqrt{\pi }\right)}^{-n}{h}_{m,k}^{n-4}.\hfill \end{array}$$

(47)

From Lemma 1 and Equation (47)

$$\begin{array}{cc}\hfill \int tr{\left(\frac{{\partial }^2}{\partial \mathit{x}\partial {\mathit{x}}^{\mathrm{T}}}\left(\sum _{i=1}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_{m,k}}\right)\right)\right)}^{2}d\mathit{x}& \le \int nm\sum _{i,j}{\left(\frac{{\partial }^2}{\partial x_j\partial x_j}\left(K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_{m,k}}\right)\right)\right)}^{2}d\mathit{x}\hfill \\ \hfill & =\frac{3}{4}{\left(nm\right)}^2{\left(2\sqrt{\pi }\right)}^{-n}{h}_{m,k}^{n-4}.\hfill \end{array}$$

(48)

When $h_{m,k}$ is sufficiently large, we can assume that $K\left(\frac{r_i-x}{h_{m,k}}\right)$ is almost the same everywhere, i.e., the equal sign in Equation (48) is tenable.

$$\begin{array}{cc}\hfill h_{m,k+1}& ={\left(\frac{mn{h}_{m,k}^{2n}}{2\sqrt{\pi }}\right)}^{1/\phantom{1\left(n+4\right)}\left(n+4\right)}{\left(\frac{3}{4}{\left(nm\right)}^2{\left(2\sqrt{\pi }\right)}^{-n}{h}_{m,k}^{n-4}\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}\hfill \\ \hfill & =h_{m,k}{\left(\frac{3}{4}nm\right)}^{-1/\phantom{1\left(n+4\right)}\left(n+4\right)}<h_{m,k}\hfill \end{array}$$

(49)

When $h_{m,k}$ is large, the iterative process decreases. Because $h_{m,k}$ has a lower bound, the algorithm converges. □

In summary, the KDE method based on optimal bandwidth is given (See Algorithm 1), and the flowchart of the KDE method is shown in Figure 1.

Algorithm 1: Kernel density estimation (KDE) method based on optimal bandwidth.

4. Fault Detection Method Based on JS Divergence Distribution

In Section 3, we construct a multidimensional KDE method based on the optimal bandwidth; this method can accurately describe the density distribution of multidimensional data. JS divergence is used to measure the distribution difference, and thus, it can highlight the difference in the statistical characteristics of different mode data.

4.1. Mode Difference Index

In Section 3, the probability density estimation of multidimensional data is obtained using the kernel function method, and the optimal bandwidth formula is derived. When the system fails, the state of the system will inevitably change, and the statistical characteristics of the system output will also change, thereby leading to significant changes in the density distribution of the observed data. For two groups of the sample window data $\mathit{R}$ and $\mathit{Z}$, the cross entropy $H\left(\mathit{R},\mathit{Z}\right)$ can be used to measure the distribution difference of $\mathit{R}$ and $\mathit{Z}$.

$$H\left(\mathit{R},\mathit{Z}\right)=\int -{\widehat{f}}_{K,Z}\left(\mathit{x}\right)log\left({\widehat{f}}_{K,R}\left(\mathit{x}\right)\right)d\mathit{x},$$

(50)

where ${\widehat{f}}_{K,R},{\widehat{f}}_{K,Z}$ represents the optimal KDE of $\mathit{R}$ and $\mathit{Z}$ calculated using Equation (16).

$H\left(\mathit{R},\mathit{Z}\right)$ does not satisfy the definition of distance because $H\left(\mathit{R},\mathit{Z}\right)$ does not necessarily satisfy positive definiteness and symmetry; that is, $H\left(\mathit{R},\mathit{Z}\right)<0$ or $H\left(\mathit{R},\mathit{Z}\right)\ne H\left(\mathit{R},\mathit{Z}\right)$.

• The smaller the difference of distribution, the smaller is $H\left(\mathit{R},\mathit{Z}\right)$, which means that even $H\left(\mathit{R},\mathit{Z}\right)<0$, and therefore, it is reasonable to use $H\left(\mathit{R},\mathit{Z}\right)$ to measure the distribution difference of $\mathit{R}$ and $\mathit{Z}$.
• However, the quantitative description of distribution difference must satisfy symmetry; otherwise, the exchange position and distribution difference will be different, which is difficult to accept.

The JS divergence $JS\left(\mathit{R},\mathit{Z}\right)$ was used as a measure of the distribution difference between $\mathit{R}$ and $\mathit{Z}$ in reference Zhang et al. [19], Bruni et al. [20] as follows:

$$JS\left(\mathit{R},\mathit{Z}\right)=\int \begin{array}{cc}{\widehat{f}}_{K,R}log\left({\widehat{f}}_{K,R}\right)\hfill & +{\widehat{f}}_{K,Z}log\left({\widehat{f}}_{K,Z}\right)\hfill \\ \hfill & -\left({\widehat{f}}_{K,R}+{\widehat{f}}_{K,Z}\right)log\left(\left({\widehat{f}}_{K,R}+{\widehat{f}}_{K,Z}\right)/2\right)\hfill \end{array}d\mathit{x}.$$

(51)

It is easy to get that

$$\left\{\begin{array}{cc}\hfill & JS\left(\mathit{R},\mathit{Z}\right)\ge 0\hfill \\ \hfill & JS\left(\mathit{R},\mathit{Z}\right)=JS\left(\mathit{Z},\mathit{R}\right)\hfill \end{array}\right.$$

(52)

In this paper, Equation (52) is used to measure the distribution difference between testing data $\mathit{Z}$ and training data $\mathit{R}$ for realizing fault detection and isolation.

4.2. Mode Discrimination Method

If the training data has q patterns $\left\{{\mathit{R}}_1,{\mathit{R}}_2,\dots ,{\mathit{R}}_q\right\}$, the JS divergence set

$$\left\{JS\left(\mathit{Z},{\mathit{R}}_1\right),JS\left(\mathit{Z},{\mathit{R}}_2\right),\dots ,JS\left(\mathit{Z},{\mathit{R}}_q\right)\right\}$$

between the testing data $\mathit{Z}$ and different modes $\mathit{R}$ can be calculated using Equation (51).

If $i_0$ is the schema tag corresponding to the minimum JS divergence, it means that

$$i_0=argmin\left\{JS\left(\mathit{Z},{\mathit{R}}_1\right),JS\left(\mathit{Z},{\mathit{R}}_2\right),\dots ,JS\left(\mathit{Z},{\mathit{R}}_q\right)\right\}.$$

(53)

It is reasonable to assume that testing data $\mathit{Z}$ and training data ${\mathit{R}}_{i_0}$ belong to the same mode. However, for a new failure mode that may be unknown in the application, Equation (50) evaluates the testing data $\mathit{Z}$ as the known failure mode of type $i_0$, which is obviously unreasonable.

If $JS\left(\mathit{Z},{\mathit{R}}_{i_0}\right)$ is too large, we believe that testing data $\mathit{Z}$ comes from an unknown new failure mode; its label is $q+1$. However, the method to obtain the threshold $J{S}_{\mathrm{high}}$ of $JS\left(\mathit{Z},{\mathit{R}}_{i_0}\right)$ is a problem that should be investigated. A method to determine $J{S}_{\mathrm{high}}$ is provided below.

For the training data ${\mathit{R}}_{i_0}=\left[{\mathit{r}}_1,{\mathit{r}}_2,\dots ,{\mathit{r}}_m\right]$ of the $i_0$ mode, the density estimation of the data set can be obtained using Equation (16).

$${\widehat{f}}_{K,R}\left(\mathit{x}\right)=\frac{1}{m{\left(h_m\right)}^n}\sum _{i=1}^{m}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_m}\right)$$

(54)

In addition, if the length of the sampling window is fixed as $p\left(p<m\right)$, the new sampling data is ${\mathit{R}}^{\left(j\right)}=\left[{\mathit{r}}_j,{\mathit{r}}_{j+1},\dots ,{\mathit{r}}_{j+p}\right]\subset {\mathit{R}}_{i_0},j=1,2,\dots ,m-p$ by sliding the sampling window. For each ${\mathit{R}}^{\left(j\right)}$, the density of the dataset can be estimated as

$${\widehat{f}}_{K,R^{\left(j\right)}}\left(\mathit{x}\right)=\frac{1}{p{\left(h_p\right)}^n}\sum _{i=j}^{j+p}K\left(\frac{{\mathit{r}}_i-\mathit{x}}{h_p}\right).$$

(55)

Using Equation (52), the divergence between the training data $\mathit{R}$ and the sample data ${\mathit{R}}^{\left(j\right)}$ can be obtained as

$$\begin{array}{cc}\hfill J{S}_j& =JS\left(\mathit{R},{\mathit{R}}^{\left(j\right)}\right)\hfill \\ \hfill & =H\left(\left({\widehat{f}}_{K,R}+{\widehat{f}}_{K,R^{\left(j\right)}}\right),\left({\widehat{f}}_{K,R}+{\widehat{f}}_{K,R^{\left(j\right)}}\right)/2\right)-H\left({\widehat{f}}_{K,R}\right)-H\left({\widehat{f}}_{K,R^{\left(j\right)}}\right).\hfill \end{array}$$

(56)

Using Equation (55), we can obtain a series of JS divergence calculation value sets

$$\mathit{JS}=\left\{J{S}_1,J{S}_2,\dots ,J{S}_{m-p}\right\}.$$

We use this set to provide the estimation formula ${\widehat{f}}_{JS}\left(x\right)$ of the density function $f_{JS}\left(x\right)$ of the JS divergence as

$${\widehat{f}}_{JS}\left(x\right)=\frac{1}{\left(m-p\right){\left(h_{m-p}\right)}^n}\sum _{j=1}^{m-p}K\left(\frac{J{S}_j-x}{h_{m-p}}\right).$$

(57)

If the significance level is $\alpha$, the probability of ${\widehat{f}}_{JS}\left(x\right)$ that exceeds the threshold $J{S}_{\mathrm{high}}$ is

$$P\left\{{\int }_0^{J{S}_{\mathrm{high}}}{\widehat{f}}_{JS}\left(x\right)dx.\right\}<\alpha$$

(58)

Because the distribution type of JS divergence is not a common random distribution, the quantile cannot be obtained by looking up the table; instead, it can only be obtained by numerical integration. If h is the step size, and

$${\int }_{h*\left(i-1\right)}^{+\infty }{\widehat{f}}_{JS}\left(x\right)dx\le \alpha \le {\int }_{h*i}^{+\infty }{\widehat{f}}_{JS}\left(x\right)dx,$$

(59)

It is reasonable to deduce that

$$J{S}_{\mathrm{high}}=h*i.$$

(60)

The following fault detection and isolation criteria are constructed by Equation (58).

Criterion 1.

Suppose $i_0$ is the pattern label corresponding to the minimum JS divergence—see Equation (38)—the training data ${\mathit{R}}_{i_0}=\left[{\mathit{r}}_1,{\mathit{r}}_2,\dots ,{\mathit{r}}_m\right]$ corresponding to the $i_0$ mode and the upper bound of JS divergence is $J{S}_{\mathrm{high}}$—see Equation (56). If the testing data $\mathit{Z}=\left[{\mathit{z}}_1,{\mathit{z}}_2,\dots ,{\mathit{z}}_l\right]$ meet the requirements,

$$JS\left(\mathit{Z},{\mathit{R}}_{i_0}\right)\le J{S}_{\mathrm{high}}.$$

(61)

The testing data $\mathit{Z}$ and training data ${\mathit{R}}_{i_0}$ belong to the same failure mode; otherwise, the testing data $\mathit{Z}$ are considered to originate from the unknown new failure mode, and their label is marked as $q+1$.

In conclusion, the fault diagnosis method based on optimal bandwidth is provided (See Algorithm 2), and the corresponding fault diagnosis method flowchart is shown in Figure 2.

Algorithm 2: Fault Diagnosis Method Based on Optimal KDE.

Remark 3.

Equations (54) and (55) show that the calculation result of JS divergence is directly related to the length of sampling data. Indeed, with the increase in the sampling data length, the density estimation obtained by Equation (54) can describe the distribution characteristics of samples more effectively, thereby significantly improving the accuracy of fault detection.

5. Numerical Simulation

The bearing data from Case Western Reserve University Bearing Data Center were used as the diagnosis research object, and they have been considered as a case for many fault diagnosis, such as in references Smith and Randall [21], Lou and Loparo [22], Rai and Mohanty [23].

The sampling frequency of the motor data was 12 kHz, and 12 kHz is the default sampling frequency for Case Western Reserve University Bearing Data Center. The dataset contains four groups of sample data: normal data ($f_0$), 0.007 inch inner raceway fault data ($f_1$), 0.014 inch inner raceway fault data ($f_2$), and 0.014 inch outer raceway fault data($f_3$). Each group of data had two dimensions: the acceleration data of the drive end ($f_i-DE$) and the acceleration data of the fan end ($f_i-FE$). All the experiments were conducted on an Lenovo Ryzen 3700X CPU with 3.60 GHz processor, 16 GB RAM.

5.1. Data Preprocessing

The observed data in the process of the bearing operation show obvious periodicity, which needs to be eliminated. Taking normal data $f_0$ as an example, the main frequency in the observed signal can be obtained by fast Fourier transform (FFT), and the result of the FFT is shown in Figure 3.

Figure 3 indicates that the main frequency is approximately 1036 Hz, and thus, the basis function is constructed as

$$\mathit{f}\left(t\right)={\left[\begin{array}{ccc}1& sin\left(1036\times 2\pi t\right)& cos\left(1036\times 2\pi t\right)\end{array}\right]}^{\mathrm{T}}.$$

The estimation of $\mathit{\beta }$ calculated using Equation (7) is

$$\widehat{\mathit{\beta }}=\left[\begin{array}{ccc}\hfill 0.0116& -0.0158& 0.0548\\ \hfill 0.0280& 0.0326& -0.0396\end{array}\right].$$

Thus, the data after removing the intrinsic signal are shown in Figure 4, where Figure 4a represents the acceleration data of the drive end and Figure 4b represents the acceleration data of the fan end.

In the later fault detection process, the data of all modes are similar to the above operation, and the results are recorded as $f_i$.

5.2. Fault Detection Effect

5.2.1. Norm Data and Known Fault

For the norm data $f_0$ and the known fault $f_1,f_2$, the first 20,480 sample points are selected as the training set, which are recorded as $f_{i-\mathrm{train}}$. The last 81,920 sample points are taken as the testing set, which are recorded as $f_{i-\mathrm{test}}$. A total of 128 sample points are used as detection objects in each test. The training set data are shown in Figure 5, where Figure 5a,b represent data $f_{i-\mathrm{train}},i=1,2$ of the two dimensions, respectively.

Figure 5 shows that the bearing data have high frequency, and the fault does not change the observed mean value; however, it changes the dispersion characteristics or the correlation of data.

5.2.2. Unknown Fault

The training data does not necessarily contain all types of patterns, and the detection of unknown faults is always a difficult problem. $f_3$ is used as an unknown fault for fault detection; the training set sample does not contain any information about $f_3$. The unknown fault data are shown in Figure 6, where in Figure 6a represents the acceleration data at the driving end and Figure 6b represents the acceleration data at the fan end.

Figure 6 shows that the data of unknown faults is close to the other two types of fault data. If the fault detection method is not sensitive, the detection rate will be reduced significantly.

5.2.3. Detection Effect

The characteristics of bearing data make bearing fault detection extremely challenging. The input of the training set is $f_{0-\mathrm{train}}$, the estimation accuracy is $\epsilon ={10}^{-4}$, and the maximum number of iterations is $k_{max}=100$, according to Algorithm 1, the optimal bandwidth is

$$h_m=0.0445.$$

The KDE of the training set is obtained by Equation (15), and the results are shown in Figure 7, where Figure 7a,c,e represent the two-dimensional frequency histograms of the training data $f_{i-\mathrm{train}},i=0,1,2$, and Figure 7b,d,f represent the two-dimensional KDE of the training data $f_{i-\mathrm{train}},i=0,1,2$.

Figure 7 further shows that the bearing fault changes the dispersion characteristics and data correlation. Meanwhile, Figure 7 shows that the KDE of the training data obtained by Equation (15) is in good agreement with the data distribution of the training data, and therefore, this method can really describe the distribution of multidimensional data.

The JS divergence of the training data and KDE of the distribution are obtained by Equations (51) and (58); the results are shown in Figure 8.

When the significance level is $\alpha =95\%$, the detection thresholds of the training set, which are calculated using Equation (58), are

$$\left\{\begin{array}{c}{f}_0:J{S}_{\mathrm{high}}<0.1375\\ f_1:J{S}_{\mathrm{high}}<0.0995\\ f_2:J{S}_{\mathrm{high}}<0.1225\end{array}\right.$$

Thus, the detection results of using JS divergence methods on the testing data are shown in Figure 9. If the detection points fall within the threshold, the data set to be detected is in the same pattern; otherwise, the data have different patterns.

Furthermore, detection rates using different methods are shown in

Table 1. Detection rate of normal and different failure modes using different methods.

Method	${\mathit{T}}^2$ Statistics Detection	Cross Entropy	JS Divergence
Normal mode $f_0$	$95.80\%$	$96.95\%$	$97.03\%$
Known fault $f_1$	$83.47\%$	$94.41\%$	$95.81\%$
Known fault $f_2$	$78.11\%$	$94.19\%$	$95.36\%$
Unknown fault $f_3$	\	$53.16\%$	$69.49\%$

.

For the known fault, Table 1 indicates that the bearing fault identification based on multidimensional KDE and JS divergence achieves better results compared to those obtained using the $T^2$ statistics detection methods in the testing data. The detection rate of normal data $f_0$ increases from $95.08\%$ to $97.03\%$, the detection rate of fault data $f_1$ increases from $81.33\%$ to $95.81\%$, and the detection rate of fault data $f_2$ increases from $70.69\%$ to $95.36\%$. Meanwhile, compared with the cross-entropy methods, the detection rate of normal data $f_0$ increased from $96.95\%$ to $97.03\%$; of fault data $f_1$ increased from $94.41\%$ to $95.81\%$; and of fault data $f_2$ increased from $94.19\%$ to $95.36\%$.

For the unknown fault $f_3$, Table 1 shows that the $T^2$ statistics detection method cannot detect the unknown faults. The method using cross entropy as a measure can only detect unknown faults with a detection rate of $53.16\%$, which is not obvious. The JS divergence method constructed in this study can identify the unknown fault accurately, and the detection rate reaches $69.49\%$. This is because JS divergence is more accurate at measuring the differences between distributions.

5.3. Influence of Window Width on Fault Diagnosis

The fault diagnosis effect is related to the data window width; therefore, the fault diagnosis effect under different window widths is investigated. The results are shown in Figure 10.

Figure 10 indicates that, with the increase in the detection window, the detection performance of the proposed method for the known fault detection first rises, and then, it tends to be stable. This is because when the length of the detection window increases to a certain extent, the data to be detected already contains sufficient information. Meanwhile, if the detection window continues to increase, the contribution rate to the improvement of the fault detection rate is not large. Meanwhile, for unknown faults, the detection rate increases rapidly with the length of the detection window because the longer the detection window, the higher the amount of information contained in the data to be detected, and the better is the difference characterized between the fault and the known fault.

6. Conclusions

In this study, a method of bearing fault detection and identification was constructed using multidimensional KDE and JS divergence. The distribution characteristics of JS divergence between the sample density distribution and population density distribution were derived using the sliding sampling window method. Thus, the threshold of fault detection was provided, and therefore, different faults, especially unknown faults, could be identified. The theory showed that the multidimensional KDE method could reduce information loss caused by processing each dimension; the JS divergence is more accurate than the traditional cross entropy to measure the difference in density distribution. The experimental results verified the above conclusions.

For a known fault, the detection effect of this method was obviously better than that of the traditional method, and it also had a certain degree of improvement compared with the cross-entropy method. Second, for unknown faults, the traditional method could not detect the distribution difference accurately, while the detection effect of the proposed method was significantly improved.

Furthermore, the detection effect of this method depends on the window width. The detection effect improved with a growth in the detection window. In this paper, under the condition of a given window width, the estimation formula for the optimal bandwidth of a multidimensional KDE was provided. The experimental results showed that the formula was applicable to any mode of data, and therefore, it had a certain universality.

However, this study has certain limitations. Firstly, although the calculation formula of multidimensional KDE is given in this study, the computational complexity will increase when the dimension is large, which may restrict the further application of the method. Secondly, the calculation of JS divergence is time consuming, which is not conducive to rapid fault diagnosis.

In future research, we can try to use the PCA dimension reduction method to solve the computational complexity caused by very large dimension, and optimize the algorithm flow of JS divergence to expedite the calculation. In the latest study Ginzarly et al. [24], prognosis of the vehicle’s electrical machine is treated using a hidden Markov model after modeling the electrical machine using the finite element method. Therefore, we will try to combine this method in future work and apply it to the fault detection of other systems.