Dynamic Reweighted Domain Adaption for Cross-Domain Bearing Fault Diagnosis

Abstract

Certain progress has been made in fault diagnosis under cross-domain scenarios recently. Most researchers have paid almost all their attention to promoting domain adaptation in a common space. However, several challenges that will cause negative transfer have been ignored. In this paper, a reweighting method is proposed to overcome this difficulty from two aspects. First, extracted features differ greatly from one another in promoting positive transfer, and measuring the difference is important. Measured by conditional entropy, the weight of adversarial losses for those well aligned features are reduced. Second, the balance between domain adaptation and class discrimination greatly influences the transferring task. Here, a dynamic weight strategy is adopted to compute the balance factor. Consideration is made from the perspective of maximum mean discrepancy and multiclass linear discriminant analysis. The first item is supposed to measure the degree of the domain adaptation between source and the target domain, and the second is supposed to show the classification performance of the classifier on the learned features in the current training epoch. Finally, extensive experiments on several bearing fault diagnosis datasets are conducted. The performance shows that our model has an obvious advantage compared with other common transferring algorithms.

1. Introduction

Bearinga are one of the most important components of all mechanical systems, and their healthy working state is the basic guarantee for all kinds of machines to engage in production [1,2,3,4,5]. With the continuous progress of measurement technology and the development of Internet of Things technology [6], massive data measured through sensors are available, and bearing diagnosis methods based on signal analysis or machine learning have made great progress in recent work [7,8,9,10]. Among fault diagnosis methods based on signal analysis, common feature extraction methods include wavelet transform, spectral analysis, empirical mode decomposition, and fast Fourier transform [11,12,13,14]. Feng et al. investigated the correlation between tribological features of abrasive wear and fatigue pitting in gear meshing and constructed an indicator of vibration cyclostationarity (CS) to identify and track wear evolution [15]. Cédric Peeters et al. derived blind filters using constructed envelope spectrum sparsity indicators and proposed an effective method of fault detection in rotating machinery [16]. In addition, the methods based on machine learning and deep learning are more and more widely used because they do not rely on rich expert experience and can learn complex nonlinear relationships effectively [17]. The common models among them are deep neural networks, sparse coding, and Bayesian analysis, which optimize the parameters of classifiers by utilizing labeled historical fault data [18,19,20].

Although samples from unlabeled test datasets can often be effectively diagnosed by these methods, most of them are usually based on an ideal assumption that training datasets and test datasets are collected from the same part of the same machine and are quite similar in terms of data distribution [21]. Real world conditions are often very different from ideal ones because obtaining vibration data with labels from critical parts of some machines is hard or even impossible [22]. Furthermore, collecting data samples from each machine and labeling them manually cost a large amount of time and money [23]. Hence, several training datasets can only be collected from another part of the same machine or from other similar but different scenarios [24]. This kind of bearing fault diagnosis task with training data and test data from different datasets is usually called cross-domain fault diagnosis by researchers. Intrinsic discrepancies are observed in the data distribution between training datasets (source domain) with labels and test datasets (target domain) without labels. The discrepancies between source domain and target domain always have an adverse impact on the diagnostic accuracy, which is the main difficulty of cross-domain diagnosis tasks. Domain adaptation is the main strategy to solve this difficulty at present, which means making knowledge from the source domain useful for the target task by eliminating the distribution discrepancies between two domains.

To bridge the gap between the two domains in cross-domain diagnosis tasks and avoid the deterioration of performance of source data classifier on discrepant target data, several transfer learning methods based on domain adaptation have been developed in recent studies [25,26]. The common strategy of transfer learning methods based on domain adaptation is to train a domain-invariant classifier by eliminating the distribution mismatch between source and target domains [27,28]. To reduce the distribution mismatch between the two domains, distance matching methods and adversarial learning models are two main types of strategies. As for the former, maximum mean discrepancy (MMD) [3] is widely applied for distance measurement. For a better, faster computation, multikernel maximum mean discrepancy (MK-MMD) is proposed [29]. For instance, Xie et al. trained a domain-invariant classifier for cross-domain gearbox fault diagnosis by transfer component analysis (TCA) technology [30]. The latter kind of methods train a domain-invariant classifier through adversarial training between domains inspired from the strategy of generative adversarial network (GAN) [31]. In adversarial domain adaptation methods, the discriminator is responsible for distinguishing whether the feature produced by the feature extractor comes from the source domain or the target domain, whereas the feature extractor tries to obtain features that can not be distinguished by the discriminator. The competition between them drives them to promote their performance until the source domain features and target domain features are indistinguishable. A domain-invariant classifier that can be applied in source datasets and target datasets is then obtained.

In addition to marginal distribution alignment methods based on distance matching and adversarial training, conditional distribution alignment for unsupervised bearing health status diagnosis has been recently considered by many researchers. For example, Li et al. used the over fitting of various classifiers in a proposed adversarial multiclassifier model to improve class-level alignment [32]. Qian et al. introduced a novel soft label strategy to assist joint distribution alignment of fault source and target domain datasets [33]. Zhang et al. combined a novel subspace alignment model with JDA to achieve joint distribution alignment in fault diagnosis of rolling bearings under varying working conditions [34]. Wu et al. utilized joint distribution adaptation and a long-short term memory recurrent neural network model simultaneously to achieve domain adaptation under the condition of unbalanced bearing fault data [35]. Yu et al. proposed an effective simulation data-based domain adaptation strategy for intelligent fault diagnosis in which conditional and marginal distribution alignment is achieved between source data from the simulation model and target data from mechanical equipment in the actual field [36]. Moreover, Kuniaki Saito et al. proposed a novel maximum classifier discrepancy (MCD) to reduce the decision boundary of the classifier and achieved better classification performance [37]. Li et al. measured the completion degree of classification by employing a criterion based on linear discriminant analysis (LDA) to enhance the performance of the classifier on the target domain [38]. Yu et al. quantitatively measured the relative importance of marginal and conditional distributions of two domains and dynamically extracted domain-invariant features in a proposed novel dynamic adversarial adaptation network (DAAN) [39].

Although these transfer learning methods have made some progress in cross-domain bearing fault diagnosis, there are still some challenges that will cause negative transfer. Positive transfer means the knowledge learned from the source domain can effectively improve the performance of the model in the target domain. In contrast, the learning behaviour will be called negative transfer [40,41] when the knowledge from the source domain hinders the performance of learning in the target task. Obviously, it should be avoided in cross-domain diagnosis tasks. However, these difficulties that may lead to negative transfer have not received sufficient attention from previous researchers. The two main problems leading to negative transfer in cross-domain bearing fault diagnosis are as follows. First, some samples with poor transferability have not received as much attention as they deserve, which will lead to negative transfer. Almost all the existing domain adaptation models assume that all the data have the same influence. Certainly, this assumption does not hold in many real-world scenarios. Some sample data are more difficult to align because they are far from the distribution center when there is some noise in the measurement environment or under non-stationary conditions [42,43]. If each sample has the same weight in the domain adaptation training, achieving a good conditional distribution alignment will be difficult no matter how many training epochs are taken. Moreover, with the continuous updating of model parameters, the good adaptation performance of features from samples with strong transferability will be destroyed. At this stage, negative transfer occurs.

Second, the imbalance between the classifier training and domain adaptation may also cause negative transfer. Training of the fault classifier and the domain adaptation process are carried out simultaneously in most previous models, but the relationship between them is ignored and the two processes are considered independently. Indeed, excessive adaptation is likely to cause the failure of a diagnostic classifier, whereas over completion of classifier training could cause domain mismatch [44]. Both processes influence the diagnosis accuracy, that is, the over/under completion of any of the two processes leads to negative transfer. From the above analysis, we can see that there is a strong motivation to establish a more advanced method to solve the two challenges that may cause negative transfer in cross-domain fault diagnosis.

In order to solve the above challenges in the field of cross-domain fault diagnosis, a novel DRDA method is proposed in this paper. To address the negative transfer caused by some samples with poor transferability, a soft reweighting strategy inspired by curriculum learning and conditional information entropy is proposed. Such an indicator can well measure the adaption performance of each sample in real time to provide more attention to poorly aligned samples. After proper weight adjustment, the clustering of source and target domain datasets are strengthened, thus, the conditional distribution alignment can also be improved. To address the negative transfer caused by the imbalance between the classifier training and domain adaptation, a balance factor is introduced in our method to strike a balance between them and obtain a higher accuracy in the final diagnosis on the target domain dataset. Specifically, MMD is used as an estimator to observe the degree of domain adaptation. The factor $J\left(w\right)$ based on linear discriminant analysis (LDA) [38] was proposed to estimate the degree of classifier training. An effective balance factor $\lambda$ can then be constructed by combining these two items. Sufficient verification experiments demonstrate that our model outperforms state-of-the-art methods.

The main contributions of our work are as follows.

1

A novel dynamic reweighted domain adaptation method is proposed to address challenges that will cause negative transfer in cross-domain bearing fault diagnosis. A reweighted adversarial loss strategy is introduced in DRDA to eliminate negative transfer caused by samples with poor transferability in cross-domain bearing fault diagnosis.

2

A powerful balance factor $\lambda$ is constructed in the proposed method to eliminate negative transfer caused by the imbalance between the classifier training and domain adaptation in cross-domain bearing fault diagnosis.

This paper is organised into the following sections. The preliminary concept is introduced in Section 2, including the problem description and a brief introduction to the domain adversarial learning and maximum classifier discrepancy theories. In Section 3, the proposed DRDA model for cross-domain diagnosis is introduced in detail. Section 4 presents and analyzes the diagnosis performance on cross-location and cross-speed cases. Finally, this article is concluded in Section 5.

2. Principle Knowledge

2.1. Problem Description

Our work mainly focuses on bearing health status diagnosis in cross-domain scenarios and aims to develop a transferable diagnosis model with strong generalization that can effectively combat negative transfer and identify fault types corresponding to the target domain samples accurately utilizing the knowledge learned from samples of the source domain. Thus, our paper is based on the following several reasonable hypotheses:

1

Massive bearing source domain datasets with labels and target domain datasets without labels are available for domain adaptation and fault diagnosis.

2

Although most of the samples from the two domains are similar to one another in terms of data distribution, several are not and are more difficult to align.

To study bearing health status diagnosis for bearing datasets from two different domains, this paper utilizes two industrial transfer scenarios:

1

Bearing data from different locations of the same mechanical system are used as target datasets for cross-location domain adaptation.

2

Bearing data from the same mechanical system under different rotation speed conditions are used as target datasets for cross-speed domain adaptation.

In this paper, the source dataset is denoted as $D_s$, its label is denoted as $Y_s$, and $D_t$ represents the target domain dataset. Superscript s represents the source domain data, and t represents the target domain data. This work focuses on four kinds of bearing health status: ball fault (BF), healthy (H), inner race fault (IF), and outer race fault (OF). In this paper, the structures of modules with the same name in different methods are exactly the same, so the same modules in different methods are uniformly represented by unified symbols (feature extractor: $G_f$, classifier: $G_y$, domain discriminator: $G_d$). Details of the structure and parameters of these modules are shown in the Figure 1.

2.2. Domain Adversarial Learning

The main ideal of adversarial domain adaptation is to apply the min-max optimization strategy of GAN to extract domain-invariant features [45]. The basic structure of the adversarial domain adaptation network is shown in Figure 2.

This model has three main components: A feature extractor ($G_f$), a health status classifier ($G_y$), and a discriminator ($G_d$). The expressions ${\theta }_f$, ${\theta }_y$, and ${\theta }_d$ are the parameters of $G_f$, $G_y$, and $G_d$, respectively. The parameters of the feature extractor and the discriminator are trained by samples from the source dataset and the target dataset, whereas the classifier is trained only on the source domain examples with labels. Specifically, the goal of the discriminator is to discern whether the features obtained by the feature extractor come from the source domain or the target domain, whereas the feature extractor aims to gain domain-invariant features that cannot be distinguished by the discriminator. The updating of parameters will eventually converge to an optimal equilibrium state, where the loss of the discriminator reaches its maximum corresponding to ${\theta }_f$ and minimum corresponding to ${\theta }_d$. The loss can be expressed as follows:

$$\begin{array}{c}\hfill L({\theta }_f,{\theta }_y,{\theta }_d)=\frac{1}{n_s}\sum _{x_k\in D_s}{L}_y(G_y\left(G_f\left(x_k\right)\right),y_k)-\frac{\alpha }{n_s+n_t}\sum _{x_k\in D_s\cup D_t}{L}_{da}(G_d\left(G_f\left(x_k\right)\right),d_k)\end{array}$$

(1)

where $n_s$ and $n_t$ are the number of samples in the source domain and target domain, respectively; $L_y$ is the classifier cross-entropy loss, and $L_{da}$ is the adversarial loss of the domain discriminator; $d_k$ is the label generated by the discriminator to represent the domain of input data (1 represents the target domain and 0 represents the source domain); and $\alpha$ is a parameter with the function of trade-off. When the iterative process converges to the optimal state, the parameters satisfy Equation (2):

$$\begin{array}{c}\hfill ({\widehat{\theta }}_f,{\widehat{\theta }}_y)=arg\underset{{\theta }_f,{\theta }_y}{min}L({\theta }_f,{\theta }_y,{\theta }_d)\\ \hfill \left({\widehat{\theta }}_d\right)=arg\underset{{\theta }_d}{max}L({\theta }_f,{\theta }_y,{\theta }_d)\end{array}$$

(2)

Thus, the domain adaptation can be achieved by the counterbalance of adversarial training.

2.3. Maximum Classifier Discrepancy (MCD)

The main goal of MCD is to narrow the decision boundary of the classification and eliminate the misclassification of samples whose features are near to the boundary in the feature space [37]. As shown in Figure 3, this model mainly consists of three modules: a feature extractor ($G_f$) and two classifiers ($G_{y1}$, $G_{y2}$). The feature extractor is optimized to obtain features that could minimize the discrepancy of predictions given by the two classifiers. Discrepancy loss is defined as follows:

$$\begin{array}{cc}\hfill & d(G_{y1},G_{y2})=\frac{1}{c}\sum _{k=1}^c\parallel p_{1k}-p_{2k}\parallel \hfill \\ \hfill & {\mathcal{L}}_{cd}=\frac{1}{n_t}\sum _{x_t\in D_t}[d\left(p_1\left(y\right|x_t)\right),d\left(p_2\left(y\right|x_t)\right)]\hfill \end{array}$$

(3)

where c is the total number of all health status classes and equals 4 in this paper; $p_{1k}$, $p_{2k}$ denote probability output of $G_{y1}$, $G_{y2}$ for class k, respectively. The selection of the norm is crucial to the final results and $L_1$-norm is adopted, as in the original paper. In MCD, min-max optimizing class discrepancy is a core procedure. First, the feature generator ($G_f$) is fixed to maximize discrepancy loss. Next, the two classifiers $G_{y1}$ and $G_{y2}$ are fixed to minimize discrepancy loss. Such min-max optimization procedures are all towards target samples, and source samples are only trained by the fault classifier’s cross entropy loss L. The optimization details can be expressed as follows:

$$\begin{array}{cc}\hfill StepA:& \underset{G_f,G_{y1},G_{y2}}{min}L(D_s,Y_s)\hfill \\ \hfill StepB:& \underset{G_{y1},G_{y2}}{min}L(D_s,Y_s)-L_{cd}\left(D_t\right)\hfill \\ \hfill StepC:& \underset{G_f}{min}{L}_{cd}\left(D_t\right)\hfill \end{array}$$

(4)

3. Proposed Method

For a more robust and positive transfer, this paper introduces two reweighting strategies from two perspectives. First, adversarial loss is reweighted from the perspective of each sample. Second, a balance between domain adaptation and discrimination is made from the perspective of the whole dataset. In this section, the method, which mainly consists of four parts, will be explained in detail.

3.1. Expanding MCD

Original MCD and much derived research achieve this idea using two classifiers [46]. However, in the final step, several methods may be used for eventually classifying target samples. For example, several methods may use the best result between $G_{y1}$, $G_{y2}$. Several methods may combine two classifiers’ probability outputs and compare the sum to identify the final label. To remove such ambiguity, three classifiers are utilized, as shown in Figure 4, where $G_y$ is the main fault classifier; $G_{y1}$ and $G_{y2}$ are the two classifiers used only for computing discrepancy loss. Discrepancy loss contains three parts, and the equation below is our optimization objective:

$$\begin{array}{cc}\hfill & \underset{{\theta }_f,{\theta }_y}{min}\underset{{\theta }_{y_1},{\theta }_{y_2}}{max}{L}_{cd}({\theta }_f,{\theta }_y,{\theta }_{y_1},{\theta }_{y_2})\hfill \\ & =\frac{1}{n_t}\{\parallel G_{y_1}\left(G_f\left(x_j^t\right)\right)-G_{y_2}\left(G_f\left(x_j^t\right)\right){\parallel }_1\hfill \\ \hfill & +\parallel G_y\left(G_f\left(x_j^t\right)\right)-G_{y_1}\left(G_f\left(x_j^t\right)\right){\parallel }_1\hfill \\ & +\parallel G_y\left(G_f\left(x_j^t\right)\right)-G_{y_2}\left(G_f\left(x_j^t\right)\right){\parallel }_1\left)\right\}\hfill \end{array}$$

(5)

where $G_f$, $G_y$, $G_{y1}$, $G_{y2}$ are the feature generator and three fault classifiers that can be trained by the source domain data, respectively; ${\theta }_f$, ${\theta }_y$, ${\theta }_{y1}$, ${\theta }_{y2}$ are their parameters; $x_j^t$ is the target domain sample that is input into the model. In our model, $G_y$ is used as the main classifier. Fixing ${\theta }_f$ and ${\theta }_y$ maximize the discrepancy between $G_{y1}$ and $G_{y2}$ in the target domain, and the target data excluded by the support of the source can be detected. Next, ${\theta }_{y_1}$ and ${\theta }_{y_2}$ are fixed and the discrepancy is minimized by training $G_f$ and $G_y$ to learn strong discriminating features.

3.2. Reweighted Adversarial Loss

In realistic domain adaptation scenarios, domain distributions that embody multi-mode structures prone to negative transfer are always a great challenge. Previous studies were aimed at finding excellent domain adaptation but rarely considered the role of each sample. They assumed that all the samples have equal transferabilities and make equal transfer contributions.

To promote positive transfer, it should be noticed that the transferability of each sample in the source domain or the target domain is different [47]. Different samples do not align well at the same time, that is, several samples with strong transferability may generate easily good transferable feature and have excellent performance in earlier iterations, whereas others may be just the opposite. Therefore, introducing an effective indicator to estimate such differences is needed. Inspired by information theory, entropy can measure uncertainty [48], and a reweighting strategy is proposed in this work to measure the degree of adaptation for each sample. The well-aligned samples obtain a lower entropy, whereas the poorly-aligned samples always have a higher entropy.

In the domain adaptation process, poorly aligned or slowly aligned samples should be given more attention. Hence, probability of a classifier is utilized to compute conditional entropy and reweight the domain adversarial loss of each sample. The traditional domain adversarial loss $L_{da}$ can be extended as follows:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{da}({\theta }_f,{\theta }_d)=-\frac{1}{n_s}\sum _{x_i^s\in D_s}log\left(G_d\left(G_f\left(x_i^s\right)\right)\right)-\frac{1}{n_t}\sum _{x_j^t\in D_t}log[1-G_d\left(G_f\left(x_j^t\right)\right)]\hfill \end{array}$$

(6)

where $G_d$ is the domain discriminator, and ${\theta }_d$ is its parameters. Joint with self-adaptive weight, domain adversarial loss can be rewritten as follows:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{da}({\theta }_f,{\theta }_d)& =-\frac{1}{n_s}\sum _{x_i^s\in D_s}{w}_i^s·log\left(G_d\left(G_f\left(x_i^s\right)\right)\right)-\frac{1}{n_t}\sum _{x_j^t\in D_t}{w}_j^t·log[1-G_d\left(G_f\left(x_j^t\right)\right)]\hfill \\ \hfill where w_i^s& =1+{\mathcal{H}}_p\left(x_i^s\right)=1-\sum _{k=1}^c{p}_{x_i^s}^k·log\left(p_{x_i^s}^k\right)\hfill \\ \hfill w_j^t& =1+{\mathcal{H}}_p\left(x_j^t\right)=1-\sum _{k=1}^c{p}_{x_j^t}^k·log\left(p_{x_j^t}^k\right)\hfill \end{array}$$

(7)

where $w_i^s$ is the weight of source sample $x_i^s$ and $w_j^t$ is the weight of target sample $x_j^t$; $p_{x_i^s}^k$ and $p_{x_j^t}^k$ are the probabilities for each class k towards $x_i^s$ and $x_j^t$, respectively, computed by the main fault classifier $G_y$; and ${\mathcal{H}}_p$ is the conditional information entropy. The larger $H_p$ of a sample means its worse adaptation performance in the current epoch of training. Therefore, its corresponding weight $w_i^s$ or $w_j^t$ in domain adversarial training will be larger. Through this mechanism of paying more attention to poorly aligned samples, the negative transfer caused by samples with poor transferability could be effectively eliminated.

3.3. Balance Factor for Domain Adaptation and Class Discrimination

In the former section, two main approaches are introduced to solve the domain adaptation problem. First, represented by a deep adversarial neural network (DANN), such methods aim to pursue feature representations that make both domains align well, and these common features can have a good transferability between domains. Second, similar to MCD, such methods believe that features aligning well are not sufficient, and obtaining excellent performance in the target domain task is the ultimate goal. For this purpose, these methods try to find class-specific features, and this process is so-called class discrimination.

Xiao et al. [44] proposed that the importance of two said items are different during algorithm iteration. For example, in the beginning of algorithm training, domain adaptation is more important than class discrimination. With an increase of training iterations, domain-share features are learned better and the class discrimination should receive more consideration. Inadequate and excessive domain adaptation or discriminant learning are harmful to positive transfer. Thus, how to balance these two items dynamically matters. It is well known that MMD is a good choice to measure the difference between two distributions. Naturally, MMD is used in this work as an estimator to observe the degree of adaptation between the two domains. MMD is defined as follows:

$$\begin{array}{cc}\hfill \mathrm{MMD}(D_s,D_t)=& \parallel \frac{1}{n_s}\sum _{x_i\in D_s}{G}_f(x_i;{\theta }_f)-\frac{1}{n_t}\sum _{x_j\in D_t}{G}_f(x_j;{\theta }_f){\parallel }^2\hfill \end{array}$$

(8)

For class discrimination, $J\left(w\right)$ based on linear discriminant analysis (LDA) [38] was proposed to estimate this item. LDA’s optimization objective of conventional two-category classification is defined as follows:

$$\underset{w}{max}J\left(w\right)=\frac{w^T{S}_{b}w}{w^T{S}_{w}w}$$

(9)

where $S_w$ is the intra-class scatter matrix, and $S_b$ is the inter-class scatter matrix [49]; $S_b$ indicates the mutual distance of clusters having different labels; and $S_w$ shows the compactness of data having the same labels. From this point, the class discrimination can be well measured. Expanding this idea to multiclass learning problem, the corresponding indicator is defined as follows:

$$\underset{W}{max}J\left(W\right)=\frac{tr\left(W^T{S}_{b}W\right)}{tr\left(W^T{S}_{w}W\right)}$$

(10)

The above discussion shows that MMD can depict the overall degree of domain adaptation. It can be applied in transfer learning, and the $J\left(w\right)$ can be used to measure the feature’s degree of class discrimination. Combining these two items, a proper computation for balance factor $\lambda$ can be found. However, it should be noticed that J(W) and MMD($D_s,D_t$) may have different magnitudes, and normalization is necessary; $\widetilde{\mathrm{MMD}}$ and $\widetilde{\mathrm{J}\left(\mathrm{W}\right)}$ are defined as the corresponding normalized value:

$$\begin{array}{cc}\hfill \widetilde{\mathrm{MMD}}& =\frac{\mathrm{MMD}(D_s,D_t)-\mathrm{MMD}{(D_s,D_t)}_{min}}{\mathrm{MMD}{(D_s,D_t)}_{max}-\mathrm{MMD}{(D_s,D_t)}_{min}}\hfill \\ \hfill \widetilde{\mathrm{J}\left(\mathrm{W}\right)}& =\frac{\mathrm{J}\left(\mathrm{W}\right)-\mathrm{J}{\left(\mathrm{W}\right)}_{min}}{\mathrm{J}{\left(\mathrm{W}\right)}_{max}-\mathrm{J}{\left(\mathrm{W}\right)}_{min}}\hfill \end{array}$$

(11)

where $\widetilde{\mathrm{MMD}}$ and $\widetilde{\mathrm{J}\left(\mathrm{W}\right)}$ are all located in interval [0,1]. The balance factor can be computed as follows:

$$\lambda =\frac{\widetilde{\mathrm{MMD}}}{\widetilde{\mathrm{MMD}}+(1-\widetilde{\mathrm{J}\left(\mathrm{W}\right))}}$$

(12)

Notably, $\lambda$ is the weight of domain adaptation loss, and $1-\lambda$ is the weight of class discrimination loss. Thus, the loss after adding the balance factor for domain adaptation and class discrimination is written as follows:

$$\underset{{\theta }_f,{\theta }_y}{min}\underset{{\theta }_d,{\theta }_{y_1},{\theta }_{y_2}}{max}\lambda ·L_{da}({\theta }_f,{\theta }_d)+(1-\lambda )·L_{cd}({\theta }_f,{\theta }_y,{\theta }_{y_1},{\theta }_{y_2})$$

(13)

A smaller $\widetilde{\mathrm{MMD}}$ means better domain adaptation, and a larger $1-\widetilde{\mathrm{J}\left(\mathrm{W}\right)}$ indicates worse class discrimination. If domain adaptation is much better than class discrimination, the $\widetilde{\mathrm{MMD}}$ and $\widetilde{\mathrm{J}\left(\mathrm{W}\right)}$ can be very small and the corresponding $\lambda$ is close to 0. If domain adaptation is much worse than class discrimination, the $\widetilde{\mathrm{MMD}}$ and $\widetilde{\mathrm{J}\left(\mathrm{W}\right)}$ will be close to 1 and $\lambda$ approaches 1. If $\lambda$ equals to 0.5, these two loss items have the same weight. The factor $\lambda$ can dynamically adjust the weight of loss items and effectively control excessive or insufficient domain adaptation and class discrimination.

3.4. Dynamical Reweighted Domain Adaptation (DRDA)

The concrete architecture of the proposed DRDA model can be seen in Figure 4. According to the three parts, the overall optimization objective can be obtained:

$$\begin{array}{c}\hfill \underset{{\theta }_f,{\theta }_y}{min}\underset{{\theta }_d,{\theta }_{y_1},{\theta }_{y_2}}{max}{L}_y(G_y(G_f(x^s;{\theta }_f);{\theta }_y),y^s)+\lambda ·L_{da}({\theta }_f,{\theta }_d)+(1-\lambda )·L_{cd}({\theta }_f,{\theta }_y,{\theta }_{y_1},{\theta }_{y_2})\end{array}$$

(14)

The first item is classifier cross-entropy loss, the second item is reweighted adversarial loss (RAL), and the last item is class discrimination loss. The last two items multiply their own balance factor. Through the dynamic reweight for adversarial loss and balance factor to tune up the importance of domain adaptation and classifier training, our model effectively avoids negative transfer phenomenon and obtains a robust end-to-end cross domain bearing fault diagnosis system.

3.5. Training Steps

To obtain the optimal solution of the proposed model in the previous discussion, a feature generator, three fault classifiers, and one domain discriminator need to be trained. In the last subsection, the optimization objective was given, and how to solve this problem in four steps will be shown next. The expression $\eta$ is hyper parameters learning rate and in the experiments, 0.01 or 0.001 is selected; $\lambda$ is the balance factor. The concrete computing method was given before, and only the detailed procedure of parameters updating will be discussed here.

Step A: The main fault classifier $G_y$ and feature generator $G_f$ are trained to make the main fault classifier more discriminantive and to classify the source data correctly. The network is trained to minimize the cross entropy loss $L_y$ and update the parameters, ${\theta }_f$, ${\theta }_y$:

$$\begin{array}{c}\hfill {\theta }_f\leftarrow {\theta }_f-\eta ·\frac{\partial L_y}{\partial {\theta }_f}\\ \hfill {\theta }_y\leftarrow {\theta }_y-\eta ·\frac{\partial L_y}{\partial {\theta }_y}\end{array}$$

(15)

Step B: Re-weighted domain adversarial loss $L_{da}$ is minimized. The feature generator $G_f$ is fixed, and only the domain adversarial module $G_d$ is trained. The parameters ${\theta }_d$ are updated as follows:

$$\begin{array}{c}\hfill {\theta }_d\leftarrow {\theta }_d-\eta ·\frac{\partial L_{da}}{\partial {\theta }_d}\end{array}$$

(16)

Step C: The feature generator $G_f$ is fixed, and three fault classifiers are trained to increase the discrepancy, which means minimizing $-L_{cd}$. As MCD, the main fault classifier’s cross entropy loss $L_y$ is also added to $-L_{cd}$. Parameters ${\theta }_y$, ${\theta }_{y_1}$, and ${\theta }_{y_2}$ are updated as follows:

$$\begin{array}{cc}\hfill & {\theta }_y\leftarrow {\theta }_y-\eta ·(\frac{\partial L_y}{\partial {\theta }_y}-\frac{\partial L_{cd}}{\partial {\theta }_y})\hfill \\ \hfill & {\theta }_{y_1}\leftarrow {\theta }_{y_1}+\eta ·\frac{\partial L_{cd}}{\partial {\theta }_{y_1}}\hfill \\ \hfill & {\theta }_{y_2}\leftarrow {\theta }_{y_2}+\eta ·\frac{\partial L_{cd}}{\partial {\theta }_{y_2}}\hfill \end{array}$$

(17)

Step D: Finally, the balanced reweighted domain adversarial loss $L_{da}$ and discrepancy loss $L_{cd}$ are optimized. In this step, fault classifiers $G_y$, $G_{y_1}$, $G_{y_2}$ and domain adversarial module $G_d$ are fixed to update parameter ${\theta }_f$:

$${\theta }_f\leftarrow {\theta }_f-\eta ·(\lambda ·\frac{\partial L_{da}}{\partial {\theta }_f}+(1-\lambda )·\frac{\partial L_{cd}}{\partial {\theta }_f})$$

(18)

4. Experiments

The advantage of the proposed DRDA is demonstrated through evaluation experiments on two cases: a bearing fault dataset from Case West Reserve University (CWRU) and a rotor test dataset (RT). Various previous cross-domain diagnosis models are introduced for a thorough comparison. The performance of the proposed method is compared with the following cross-domain bearing fault diagnosis methods in previous studies: CNN without domain adaptation, transfer component analysis (TCA), joint distribution analysis (JDA), deep adversarial neural networks (DANN), fine-grained adversarial network-based domain adaptation (FANDA), and maximum classifier discrepancy (MCD).

4.1. Cross-Location Diagnosis on CWRU Case

The bearing fault datasets from CWRU are available from its official website [50]. Bearings with faults are placed at either the fan end (FE) or the drive end (DE) in each test. The bearing types at the FE and DE are SKF 6203-2RS and SKF 6205-2RS, respectively. The parameters of the bearings at both ends are listed in

Table 1. Parameters of bearings in CWRU case.

Location	Bearing Type	Ball Diameter	Outside Diameter	Inside Diameter	Number of Balls
Drive End (DE)	6205-2RS	7.94 mm	52.00 mm	25.00 mm	9
Fan End (FE)	6203-2RS	6.75 mm	40.00 mm	17.00 mm	8

.

The bearing data samples are divided into four classes according to health status: outer race fault (OF), inner race fault (IF), ball fault (BF), and health (H). These bearing faults are all obtained by artificial processing. The frequency of the sampling instrument is 12 kHz, and the depth of every fault is 0.007 inches. Specific working conditions corresponding to each dataset are shown in

Table 2. Description of experimental setting of CWRU case, which includes four categories and four domains.

Faulty Type	Ball	Health	Inner Race	Outer Race	Working Conditions
Damage Diameter (in.)	0.007	—–	0.007	0.007	—–
Class Label	0	1	2	3	—–
Domain $A_1$	1000	1000	1000	1000	0 HP (1797 r/min)
Domain $A_2$	1000	1000	1000	1000	3 HP (1730 r/min)
Domain $B_1$	1000	1000	1000	1000	0 HP (1797 r/min)
Domain $B_2$	1000	1000	1000	1000	3 HP (1730 r/min)

.

Data from $A_1$ and $A_2$ are collected from the bearing at the DE and their working loads are 0 HP (1797 r/min) and 3 HP (1730 r/min), respectively. Similarly, $B_1$ and $B_2$ are collected from the bearing at the FE and their working loads are also 0 HP (1797 r/min) and 3 HP (1730 r/min), respectively. Every dataset has 4000 examples, and the data shape of each time series example is $4096\times 1$. As shown in Figure 5 the CWRU case has four cross-location diagnosis tasks, i.e., $A_1$→$B_1$, $B_1$→$A_1,$$A_2$→$B_2$, $B_2$→$A_2$. In each task, the former is the source domain with labels and the latter is the target domain without labels. The performance of each model is measured by the accuracy of fault predictions, which is defined as follows:

$$Accuracy=\frac{Numberofcorrectpredictions}{Totalnumberofpredictions}$$

(19)

The diagnosis results of the four cross-location tasks are listed in

Table 3. Accuracy (%) of different models for CWRU case tasks.

	CNN	TCA	JDA	DANN	FANDA	MCD	DRDA
$A_1$→$B_1$	69.85	52.85	55.42	73.52	81.13	82.52	98.20
$B_1$→$A_1$	44.70	48.60	49.72	78.93	83.30	83.15	84.33
$A_2$→$B_2$	67.07	78.87	79.12	90.83	91.47	89.12	91.67
$B_2$→$A_2$	75.83	61.85	62.05	82.63	83.30	83.30	85.15
Average	64.36	60.54	61.58	81.41	84.80	84.52	89.38

. First, as expected, the performance of CNN is the worst among all the models because the distribution mismatch is not eliminated at all. The performance of CNN shows great difference in two reversed tasks, in which the source and target domains correspond to two identical datasets but the roles are reversed. For example, the diagnosis accuracy reached 69% in the task $A_1\to B_1$, but this number is reduced to approximately 44% in the corresponding reversed task $B_1\to A_1$, probably because of the large statistical distribution discrepancy between the two domains. This situation is clearly improved in other domain adaptation based models, and this confirms the necessity of domain adaptation in cross-domain bearing fault diagnosis.

Figure 5. Cross-location bearing diagnosis tasks in CWRU case. Data samples with inner race faults are used for illustration.

Figure 5. Cross-location bearing diagnosis tasks in CWRU case. Data samples with inner race faults are used for illustration.

Second, the mean accuracy of the proposed method in this paper is 89.38%, which outperforms that of all the comparison models. In addition, domain adaptation models that align both marginal and conditional data distribution of the two domains perform better than those that only align marginal distribution. The mean accuracy of FANDA and JDA exceeds 84%, whereas for the DANN aligning marginal distribution only, the mean accuracy is under 82%. The DANN utilizing adversarial domain adaptation performs better than TCA utilizing MMD criterion. Moreover, MCD performs better than TCA, JDA, and DANN due to the longer distance between the feature of each sample and the decision boundary, and it performs slightly worse than FANDA, probably because of an insufficient degree of domain adaptation.

The confusion matrices of task $A_1$→$B_1$ shown in Figure 6 are introduced to analyze the results in more detail. The diagnosis accuracies of both FANDA and DANN are low, and the classifiers learned from them are unable to identify ball fault (BF) and outer race fault (OF) effectively. Part of the classification error of BF is corrected by MCD, whereas the misclassification phenomenon of OF is still evident. As can be seen from the figure, the proposed DRDA can accurately distinguish four kinds of bearing health status.

4.2. Cross-Speed Diagnosis on RT Case

Another dataset is collected from a rotor test in a practical scenario [51], and the schematic diagram of the test rig is shown in Figure 7. The power source of this rotor rig is a three-phase induction motor. The motor is connected to a shaft that is supported by a few bearings through a coupling. The bearings to be monitored are located at the right end of the shaft, and radial loads are provided by a loaded device. The vibration sensor for data collection is installed at point A. The bearing type in this diagnosis case is HRB 6010-2RZ (Harbin Bearing Manufacturing Co., Ltd., Harbin, China), and the related details are shown in

Table 4. Parameters of tested bearings in RT case.

Bearing Type	Ball Diameter (mm)	Outside Diameter (mm)	Inside Diameter (mm)	Number of Balls
HRB 6010-2RZ	9	80	50	13

.

Similarly, data samples are divided into four classes according to health status: outer race fault (OF), inner race fault (IF), ball fault (BF), and health (H). Three kinds of faults are processed manually by wire electrical discharge machining. As shown in Figure 8, the rotational speed of the motor for domains C, D, and E is set to 3000, 5000, and 8000 revolutions per minute (rpm) during the measurement, respectively. Every domain has 4000 samples, and the data shape of each time series sample is $4096\times 1$. The frequency of the sampling instrument is 65,536 Hz, and the load exerted on the shaft and bearings by the radial loaded device is 2.0 kN. According to the said description, the details of RT case datasets are listed in the

Table 5. Description of experimental setting on RT case, which includes four categories and three domains.

Faulty Type	Ball	Health	Inner Race	Outer Race	Radical Load	Rotation Speed
Class Label	0	1	2	3	—–	—–
Domain C	1000	1000	1000	1000	2.0 kN	3000 r/min
Domain D	1000	1000	1000	1000	2.0 kN	5000 r/min
Domain E	1000	1000	1000	1000	2.0 kN	8000 r/min

.

The designed RT case has six cross-domain diagnosis tasks in d, namely C→D, D→C, C→E, E→C, D→E, E→D, and their results are listed in

Table 6. Accuracy (%) of different models for RT case tasks.

	CNN	TCA	JDA	DANN	FANDA	MCD	DRDA
C→D	51.82	26.55	27.05	99.05	99.77	99.95	100.00
D→C	49.80	24.52	26.42	63.05	58.42	69.67	75.00
C→E	49.58	22.08	24.45	81.95	97.90	99.72	100.00
E→C	25.00	22.90	22.90	51.52	48.52	49.83	50.00
D→E	52.00	40.15	43.62	97.75	99.85	100.00	100.00
E→D	56.83	40.63	46.30	74.78	81.70	72.75	78.20
Average	47.51	29.47	31.79	78.02	81.06	82.00	83.87

.

Previous work in signal processing showed that the knowledge for fault diagnosis at variable speeds becomes more difficult to extract [52]. Moreover, the excellent performance of the proposed DRDA confirms that our model can robustly extract domain-invariant features under cross-speed diagnosis scenarios. First, as listed in Table 6, the performance of TCA and JDA degenerates more seriously than that of the other cross-domain diagnosis models. Their accuracies are under 30% in most cross-speed scenarios. For example, TCA and JDA only achieve an accuracy of 22.90% in the task of E→C. In accordance with the CWRU case, JDA performs slightly better than TCA in almost all tasks.

Second, the proposed DRDA performs better than other domain adaptation methods in almost all tasks, with a mean accuracy of 83.87% and an accuracy of 100% in tasks C→D, C→E, and D→E. Similarly, FANDA based on marginal and conditional distribution adaptation outperforms DANN in most tasks. In addition, the performances of almost all methods in tasks with a high speed dataset as the source domain and a low speed dataset as the target domain are unexpectedly much worse than those in reversed tasks (with a low speed dataset as the source domain and a high speed dataset as the target domain). For instance, the accuracies of FANDA, MCD, and DRDA are all above 95% in task C→E, whereas the numbers greatly reduce to approximately 50% in task E→C, because samples obtained under the working condition of high rotating speed have more noise, and extracting effective domain-invariant information for diagnosis is much more difficult when high speed datasets are set as the source domain.

To demonstrate the effectiveness of the proposed DRDA method more intuitively, the t-distributed stochastic neighbor embedding (t-SNE) technology [53] is utilized to visualize features by mapping them into 2D space. The t-SNE visual renderings of all the models on task D→C are shown in Figure 9. TCA and JDA almost fail completely on this difficult task in the RT case. CNN can only align a small part of the features with IF and OF. The confusion between IF and OF is serious in both DANN and FANDA. The confusion between IF and OF is remarkably improved in MCD, but the discrimination performance between H and BF deteriorates sharply. It is shown in Figure 9(g) that the distribution boundary of features from each class is tighter in DRDA, and the domain adaptation of features of samples with H, IF, and OF is clearly improved. In addition, another $F_1$ score metric is also introduced to verify the effect of the proposed DRDA method in eliminating negative transfer in difficult diagnosis tasks and improving the final diagnosis performance. The $F_1$ score is defined as

$$F_1 score=\frac{2\times precision\times recall}{precision+recall}$$

(20)

where precision is the proportion of bearing samples given the correct label to all bearing samples in the predicted class. Recall is the proportion of bearing samples given the correct label to all bearing samples in the real class. The $F_1$ score of four models that perform well under the accuracy metric in $A_1$→$B_1$ and D→C tasks are shown in

Table 7. $F_1$ score (%) of DANN, FANDA, MCD, and DRDA in $A_1$→$B_1$ and D→C tasks.

	DANN	FANDA	MCD	DRDA
$A_1$→$B_1$	73.00	80.92	81.63	98.19
D→C	54.57	52.81	66.16	73.75

. As shown in the table, the diagnosis performance of the proposed DRDA method is still obviously improved compared with other three methods under the $F_1$ score metric. Moreover, compared with the traditional DANN and FANDA methods based on domain adversarial learning, the proposed DRDA has a greater improvement in diagnosis performance under the $F_1$ score metric than under the accuracy metric.

4.3. Ablation Study

The effects of the RAL and balance factor for domain adaptation and class discrimination on enhancing the diagnosis accuracy are analyzed through comparative experiments. The performance of DRDA with or without RAL and balance factor on task E→C in the RT case is shown in

Table 8. Accuracy (%) of DRDA with or without RAL and balance factor on task E → C in the RT case.

RAL	Balance Factor	Accuracy (%)
×	×	47.17
✓	×	48.33
×	✓	49.85
✓	✓	50.00

. The diagnosis performance in this task is enhanced when RAL strategy and balance factor or one of them is introduced. Furthermore, our RAL strategy is introduced to the traditional DANN model. The performance of DANN with or without RAL module on task $B_1$→$A_1$ in the CWRU case and task D→C in the RT case are shown in Figure 10, where great improvement can be noticed. This finding shows that the proposed RAL strategy can also promote positive transfer when used in other domain adaptation methods based on adversarial training.

5. Conclusions

A novel domain adaptation method named DRDA for cross-domain bearing fault is proposed in this paper. The negative transfer caused by samples with poor transferability in cross-domain bearing fault diagnosis is eliminated in DRDA by introducing a reweighted adversarial loss strategy. The negative transfer caused by the imbalance between the classifier training and the domain adaptation in cross-domain bearing fault diagnosis is eliminated in the proposed method by constructing a powerful balance factor $\lambda$. The performance of the DRDA measured by accuracy and $F_1$ score are better than other comparison methods in cross-location and cross-speed diagnosis tasks, as can be seen from Table 3, Table 6 and Table 7. After eliminating the negative transfer caused by the imbalance between the classifier training and domain adaptation, the percentage of samples correctly classified by the DRDA method in each class has been improved, as can be seen from the confusion matrix in Figure 6. The improvement of the DRDA in terms of marginal alignment and conditional distribution alignment after eliminating the negative transfer caused by samples with poor transferability is shown in the t-SNE features visualization of Figure 9. In addition, the ablation study shows that the two modules proposed in this article can still improve the diagnosis performance by eliminating negative transfer when they are introduced into the diagnosis model separately. To sum up, sufficient comparative experiments prove the feasibility and the superiority of the proposed novel domain adaptation method for cross-domain bearing fault diagnosis.