RUL Prediction Based on xLSTM–Transformer Neural Network for Rolling Element Bearings Under Different Working Conditions

Abstract

Remaining useful life (RUL) prediction of rolling bearings is a crucial issue in intelligent predictive maintenance, thereby ensuring equipment safety and reducing maintenance costs. To address the challenge that traditional deep learning models struggle to simultaneously capture local temporal features and global degradation trends when processing degradation health indicators (HI), this paper proposes a hybrid RUL prediction model based on extended Long Short-Term Memory (xLSTM) and Transformer. The model employs an encoder–decoder architecture, integrating the Multi-Head Attention mechanism with the xLSTM module. This design simultaneously enhances the modeling capability of short-term dynamic features and effectively captures long-term degradation patterns. Validation was conducted on the XJTU-SY and PHM2012 datasets. The proposed model outperformed the comparative models across evaluation metrics such as Root Mean Square Error (RMSE), Coefficient of Determination (R²) and the Score, achieving a significant improvement in prediction accuracy and multi-dataset generalization capability. The proposed network provides a more accurate and generalizable solution for bearing health assessment and remaining useful life prediction and demonstrates significant potential for intelligent health management of industrial equipment.

1. Introduction

As critical components in modern rotating machinery, rolling element bearings are widely utilized in various industrial fields. However, bearings are prone to failure due to fatigue and wear with the operating time. Although such failures occur at the component level, they often lead to more severe equipment breakdowns [1]. According to statistics from the Institute of Electrical and Electronics Engineers (IEEE), bearing failures are the most common type of fault in low-voltage permanent magnet synchronous motors, accounting for approximately 41% of all failures [2]. Considering the operating environment of rolling bearings, frequent shutdowns for maintenance are not an optimal maintenance strategy. Therefore, it is meaningful to perform remaining useful life (RUL) prediction for bearings to arrange scheduled maintenance before failure occurs and to reduce unnecessary losses caused by equipment damage or production downtime.

RUL prediction of bearings aims to estimate the remaining operational time of a bearing at a future moment based on its current condition and historical information. Traditional physics-based methods for RUL prediction have been widely applied to bearing life estimation [3,4]. However, traditional physics-based models struggle to capture the complex nonlinear relationships involved in equipment degradation. Accurate modeling under complex operating conditions and individual differences is tough work, resulting in difficulty in applying it in real industrial scenarios.

With the widespread adoption of sensor technologies and the advent of the industrial big data era, data-driven methods have become increasingly prominent. Among them, deep learning-based approaches can leverage the vast amounts of data collected by sensors to uncover hidden features and nonlinear relationships in the degradation process. These methods have demonstrated enhanced accuracy and applicability, which have attracted much more attention in recent years.

Deep learning methodologies have evolved significantly in RUL prediction. Early research utilized Convolutional Neural Networks [5,6] to extract spatial features. Recurrent Neural Networks (RNNs), such as LSTM and GRU [7,8,9,10,11], are used to model temporal dependencies. However, these single-architecture models struggle to balance local feature extraction with long-term dependency modeling.

To address this problem, Transformer network [12] is utilized through introducing the self-attention mechanism, which enables global dependency modeling. Yet, the lack of recurrent structure makes the Transformer less sensitive to local ordering in time-series data. Therefore, hybrid architectures have emerged as an effective way. Recent research has explored hybrid architectures to synergize the global receptive field of Transformers with the local temporal sensitivity of RNNs. Lei et al. integrated hierarchical clustering with a GRU–Transformer architecture to mitigate feature redundancy [13]. Ali and Kamal enhanced the robustness of such hybrid models by employing a hybrid HHO-WHO meta-heuristic algorithm for hyperparameter optimization [14]. Jin et al. combined EMD with a BiLSTM–Transformer network to process non-stationary vibration signals [15].

Although these studies demonstrate superior predictive accuracy compared to single models, their architectural designs are predominantly characterized by simple network stacking. The prevalent strategy of sequentially connecting heterogeneous module lacks deep internal integration or adaptive feature fusion mechanisms. Furthermore, the recurrent components in these hybrid architectures, such as standard LSTM and GRU, are fundamentally constrained by their sigmoid gating and scalar memory cells.

Overall, these constraints affect the performance of bearing RUL prediction in two ways. First, standard LSTM units rely on sigmoid gating, which compresses inputs into the range of [0, 1]. In the final stage of bearing life, the HI value typically exhibits an exponential rise with a wide dynamic range. The sigmoid function tends to saturate when processing these rapidly increasing values, losing sensitivity to the severity of degradation. Secondly, the degradation process of bearing is a long-term evolution. Standard LSTM utilizes scalar memory cells, which has limited capacity to encode the accumulation of damage over long historical windows. This restricts the ability of the RUL prediction model to capture the subtle trend occurring before the accelerated failure point.

To overcome these limitations, this study proposed an xLSTM–Transformer hybrid network. Unlike existing hybrid networks that simply stack standard modules, this study conceptually upgrades the recurrent unit by integrating xLSTM into the Transformer encoder. This integration introduces two key mechanisms absent in conventional hybrid networks. First, by replacing sigmoid gating with exponential gating, the xLSTM module prevents saturation. This is particularly crucial for tracking the accelerated degradation trend of the HI. The exponential mechanism allows the model to retain sensitivity to large values in the late failure stage. This ensures accurate RUL estimation when the degradation accelerates sharply. Secondly, the xLSTM upgrades the storage structure from scalar form, which is an inherent limitation of standard LSTM, to matrix form. This upgrade provides superior capacity to model the degradation trajectory. It enables the hybrid encoder to robustly memorize the long-term evolution of the HI sequence, which can effectively address the problem between early degradation signs and final failure.

The primary contributions of this work are in the following three points: (1) A novel hybrid architecture xLSTM–Transformer is proposed, which effectively fills the gap between global trend modeling and high-dynamic local feature extraction in HI sequences. (2) The saturation issue in the final degradation phase is addressed by the exponential gating of xLSTM, ensuring the prediction sensitivity in the late failure stage. (3) Extensive experiments on bearing datasets demonstrate that the proposed method significantly outperforms standard LSTM–Transformer and other baseline models in RUL prediction.

The rest of the paper is arranged as follows. Section 1 introduces the theoretical background and related work. Section 2 details the proposed xLSTM–Transformer hybrid RUL prediction model. Section 3 describes the experimental setup and datasets, while presenting the results and discussion. Finally, Section 4 provides the concluding remarks of this study.

2. Methods

2.1. ISOMAP

The Isometric Feature Mapping (ISOMAP) algorithm is a nonlinear dimensionality reduction method based on manifold learning [16]. This method replaces the Euclidean distance with the geodesic distance and applies it to the Multidimensional Scaling (MDS) framework to uncover the intrinsic low-dimensional structure within the data. The main computational steps of the ISOMAP algorithm are as follows:

(1)

Construct an adjacency graph G for all data points in the high-dimensional space, where the distance between nodes is defined by the Euclidean metric. Adjacency is defined as an ε-neighborhood or a k-nearest neighbor criterion.

(2)

The Dijkstra or Floyd–Warshall algorithm is employed to compute the shortest path between any pair of nodes, $x_i$ and $x_j$, on the graph G. The shortest path is represented as the geodesic distance $d_G(x_i,x_j)$ on the manifold. Consequently, the shortest path matrix of the graph is obtained as $D_G=\left\{d_G\right(x_i,x_j\left)\right\}$.

(3)

The matrix $D_G$ is used as the input to the classical MDS algorithm, as described below: Compute the squared distance matrix $S=\left(S_{ij}\right)=\left(D_{ij}^2\right)$. Next, double centering is applied: $\tau \left(D\right)=-HSH/2$, where $H=\left(H_{ij}\right)=({\delta }_{ij}-1/N)$, select the d largest eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_d$ of $\tau \left(D\right)$ and their corresponding eigenvectors $v_1,v_2,\dots ,v_d$ to form the matrix. The low-dimensional embedding coordinates are given by $Y=diag(\sqrt{{\lambda }_1},\dots \sqrt{{\lambda }_d})V^T$.

2.2. xLSTM

The xLSTM is an improved version of the LSTM [17], which overcomes certain limitations of the standard LSTM and provides optimized performance. The xLSTM consists of an input layer, an LSTM layer, and an output layer. The input layer receives sequential data, which is processed and then fed into the LSTM layer. The LSTM layer comprises two types of LSTM modules: the scalar LSTM (sLSTM) and the matrix LSTM (mLSTM). After being processed by the LSTM layer, the data is then transmitted to the output layer. The structure of the xLSTM is illustrated in Figure 1.

2.2.1. sLSTM

The sLSTM enhances the standard LSTM by introducing an exponential gating to prevent saturation and a stabilizer state to permit revisiting storage decisions. Given an input $x_t$ at time step t, the sLSTM computes the exponential input gate $i_t$, forget gate $f_t$, and output gate $o_t$, as follows:

$$i_t=\exp \left({\tilde{i}}_t\right), {\tilde{i}}_t={\omega }_i^T{x}_t+r_i{h}_{t-1}+b_i$$

(1)

$$f_t=\exp \left({\tilde{f}}_t\right), {\tilde{f}}_t={\omega }_f^T{x}_t+r_f{h}_{t-1}+b_f$$

(2)

$$o_t=\sigma ({\omega }_o^T{x}_t+r_o{h}_{t-1}+b_o)$$

(3)

Unlike standard LSTM, the cell state $c_t$ and normalizer state $n_t$ are updated via:

$$c_t=f_t⨀c_{t-1}+i_t⨀z_t$$

(4)

$$n_t=f_t⨀n_{t-1}+i_t$$

(5)

where $z_t=\tanh (W_z{x}_t+R_z{h}_{t-1}+b_z)$ represents the input transformation.

Finally, the hidden state $h_t$ is normalized to ensure stability:

$$h_t=o_t⨀(c_t/n_t)$$

(6)

2.2.2. mLSTM

The mLSTM replaces the scalar memory cell with a matrix memory cell $C_t\in {\mathbb{R}}^{d\times d}$, which enables the model to strictly store more information than scalar standard LSTMs. It also abandons the memory mixing to achieve parallelizability.

First, the input $x_t$ is projected into query ($q_t$), key ($k_t$) and value ($v_t$) vectors:

$$q_t=W_q{x}_t+b_q, k_t=1/\sqrt{d}{W}_k{x}_t+b_k, v_t=W_v{x}_t+b_v$$

(7)

The core innovation is the covariance update rule for the matrix memory cell $C_t$. Combined with exponential gating $(i_t,f_t)$, the update rule is formulated as:

$$C_t=f_t{C}_{t-1}+i_t{v}_t{K}_t^T$$

(8)

where $v_t{K}_t^T$ represents the outer product, resulting in a matrix update that encodes the key–value relationship.

To retrieve information, the model queries the memory matrix using $q_t$:

$${\tilde{h}}_t=C_t{q}_t$$

(9)

Finally, the output is obtained by normalizing the retrieved signal:

$$h_t=o_t⨀({\tilde{h}}_t/\mathrm{m}\mathrm{a}\mathrm{x}(n_t,1))$$

(10)

2.3. Transformer

The Transformer is a neural network model based on the attention mechanism. By employing Multi-Head Attention and positional encoding, it overcomes the limitation that RNN cannot perform parallel computations. It effectively addresses the poor performance and low efficiency of RNN when processing long sequential data [18].

Figure 2 illustrates the structure of the Transformer network. The Transformer mainly consists of three components: encoder, decoder, and positional encoding. Both the encoder and decoder consist of multiple identical layers. Each layer of the encoder consists of a Multi-Head Attention mechanism and two sub-layers of a feedforward neural network. Each layer of the decoder consists of a Masked Multi-Head Attention mechanism and three sub-layers of a feedforward neural network. The encoder processes the input sequence to generate an intermediate sequence containing the sequence information. The decoder then decodes this intermediate sequence to generate the output sequence. Positional encoding adds positional information to the input data, which addresses the issue that attention mechanisms cannot learn positional information in temporal sequences.

The Transformer network is built upon the attention mechanism, utilizing self-attention to capture the most relevant information within the input itself, thereby reducing the network’s reliance on external information. The introduction of the attention mechanism enables the dynamic generation of a unique context vector at each step of the decoding process. This mechanism allows the decoder to directly access all the output states of the encoder and automatically learn to assign attention to the parts of the input sequence most relevant to the current output through a differentiable weighted sum process. The Multi-Head Attention mechanism is illustrated in Figure 3.

The computation process of the Multi-Head Attention mechanism is as follows: Let the input sequence be denoted as $X\in {\mathbb{R}}^{B\times F\times E}$, where the maximum length of the output sequence is T. The number of attention heads is N, and the dimensionality of the output for each head is H. The batch size is B, and the embedding dimension satisfies $E=N·H$. The weight matrices in each attention head satisfy $W^Q\in {\mathbb{R}}^{E\times H}$, $W^K\in {\mathbb{R}}^{E\times H}$, and $W^V\in {\mathbb{R}}^{E\times H}$.

The linear transformations of the input data in the Multi-Head Attention mechanism are given by:

$$\begin{array}{c}{Q}^{\prime }=linear\left(Q\right)=Q·W^Q, (W^Q\in {\mathbb{R}}^{E\times H}, Q\in {\mathbb{R}}^{B\times F\times E})\\ K^{\prime }=linear\left(K\right)=K·W^K, (W^K\in {\mathbb{R}}^{E\times H}, K\in {\mathbb{R}}^{B\times F\times E})\\ V^{\prime }=linear\left(V\right)=V·W^V, (W^V\in {\mathbb{R}}^{E\times H}, V\in {\mathbb{R}}^{B\times F\times E})\end{array}$$

(11)

The scaled dot-product attention is defined as:

$$\begin{array}{c}{out}^n=Attention\left(Q^{\prime },K^{\prime },V^{\prime }\right)=softmax\left({\displaystyle \frac{Q^{\prime }{K^{\prime }}^T}{\sqrt{H}}}\right)·V^{\prime },n=1,2,\dots ,N\\ \left[y_1,y_2,\dots ,y_l\right]=softmax(v_1,v_2,\dots ,v_l)\\ y_i={\displaystyle \frac{e^{v_i}}{{\sum }_{i=1}^l{e}^{v_i}}}\\ {\displaystyle \sum _{i=1}^l}{y}_i=1\end{array}$$

(12)

After the attention layer computation, the resulting outputs are obtained as $\{{out}^1,{out}^2,\dots ,{out}^n\}$. These outputs are then concatenated as:

$$out\_Concat=Concat({out}^1,{out}^2,\dots ,{out}^n)$$

(13)

After a linear transformation, the final output of the Multi-Head Attention mechanism is given by:

$$MultiHead\left(Q,K,V\right)=out\_Concat·W^O, (W^O\in {\mathbb{R}}^{E\times E})$$

(14)

2.4. The Structure of the Proposed xLSTM-Transformer

From the above analysis, owing to the multi-branch gating mechanisms of sLSTM and mLSTM, xLSTM is capable of capturing complex nonlinear relationships between the input and the hidden state within a single time step. All these characteristics make it more suitable for bearing RUL prediction tasks. Moreover, its recurrent structure enables the xLSTM to achieve stable convergence even when the available training samples are limited. However, because of its recursive structure, xLSTM suffers from information decay in long sequences, resulting in difficulty capturing global degradation trends.

Through its Multi-Head Attention mechanism, Transformer can simultaneously attend to interactions across multiple time steps within a single forward pass, thereby effectively capturing the global degradation trends throughout the entire degradation process. However, unlike recursive models, Transformer does not inherently encode temporal order. It will easily lose sequential consistency in small-sample scenarios, which could result in oscillations or instability in the predicted curves. Furthermore, since the attention mechanism is a weighted mechanism, local short-term fluctuations may be averaged out, leading to the loss of fine-grained temporal features.

Therefore, this study proposes an xLSTM–Transformer hybrid network, which aims to achieve a synergistic integration of local temporal prediction and global trend modeling. In this hybrid network, the xLSTM dynamic gating structure is incorporated into the encoder layers of the standard Transformer to enable a synergistic prediction of local temporal features and global dependencies.

In the encoder of the network, Multi-Head Attention layer first captures global dependencies across long time steps. To further enhance short-term nonlinear dynamics and local predictive capabilities, an xLSTM module is embedded as the nonlinear transition block. It is positioned between the Multi-Head Attention layer and the subsequent residual connection and layer normalization stage. In this hybrid encoder architecture, an input sequence of length L with dimensions D denoted as $X_{in}\in {\mathbb{R}}^{L\times D}$, is first fed into the Multi-Head Attention layer. The global degradation features are extracted through the attention mechanism, which is formulated as:

$$X_{Att}=Attention\left(X_{in},X_{in},X_{in}\right), X_{Att}\in {\mathbb{R}}^{L\times D}$$

(15)

To ensure feature integration and dimension alignment, the xLSTM module employs an internal projection and expansion strategy:

• Up-projection: The attention output $X_{Att}$ is first mapped into a high-dimensional feature space via a linear projection layer, where p denotes the expansion factor.
• Nonlinear processing: The exponential gating mechanism of the xLSTM is utilized to perform nonlinear degradation modeling on these high-dimensional features.
• Down-projection: The processed features are then projected back to the original dimension D through a second linear layer.

This process yields the final output of the xLSTM layer, $X_{xLSTM}$, as shown in Equations (16)–(18):

$$X_{up}={Linear}_{up}\left(X_{Att}\right), X_{up}\in {\mathbb{R}}^{L\times (p·D)}$$

(16)

$$X_p=\mathrm{x}\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left(X_{up}\right), X_p\in {\mathbb{R}}^{L\times (p·D)}$$

(17)

$$X_{xLSTM}={Linear}_{down}\left(X_p\right), X_{xLSTM}\in {\mathbb{R}}^{L\times D}$$

(18)

This process is realized through the internal symmetric projection architecture of the xLSTM, where the expansion scale is determined by the projection factor p. Such a design not only enhances the model’s capacity to capture complex local dynamics but also ensures dimensional alignment between the xLSTM output and the residual paths within the Transformer encoder. Consequently, this architecture effectively balances the strengths of the Transformer in long-range prediction with the advantages of xLSTM in local temporal forecasting.

The decoder of the network shares the same input embedding as the encoder, with the purpose of reconstructing and reintegrating the encoded features rather than performing autoregressive prediction. Compared to the original Transformer network, the proposed network incorporates an additional normalization layer. With the combined effect of multi-layer residual connections and normalization, the model exhibits improved gradient stability and enhanced generalization capability. Finally, the decoder results are flattened and fed into a linear layer to produce the predicted RUL of the equipment.

In summary, compared to the standard Transformer, the proposed xLSTM–Transformer hybrid network integrates xLSTM submodules to achieve precise prediction of both local dynamics and global trends. This enhances the network’s capability to characterize the degradation process, thereby improving the accuracy and stability of RUL prediction. The structure of the proposed xLSTM–Transformer hybrid network is illustrated in Figure 4.

The workflow for bearing RUL prediction in this study is illustrated in Figure 5, and the detailed procedure is as follows:

Step 1 Feature extraction. Based on existing expert knowledge, the raw vibration signals of the rolling bearings are analyzed in the time, frequency, and time–frequency domains to compute commonly used statistical features for RUL prediction. Evaluation metrics are used to conduct quantitative analysis of the strengths and weaknesses of each feature. Features that meet predefined thresholds are selected, ultimately forming the feature set used to train the bearing RUL prediction neural network.

Step 2 Neural network construction. Based on the conventional Transformer network, xLSTM layers are embedded within the encoder layers of Transformer to construct the proposed xLSTM–Transformer hybrid neural network.

Step 3 Test set validation. After splitting the feature set into training and test subsets, the data are fed into the neural network. The bearing RUL prediction performance of the network is evaluated by comparing the prediction effectiveness of various networks on the test set.

3. Experimental Validation

3.1. Dataset Description

To verify the superiority of the proposed xLSTM–Transformer hybrid network, this study conducts experimental validation using the following two datasets.

3.1.1. XJTU-SY Dataset

The XJTU-SY dataset is an open-access accelerated life test dataset for rolling bearings provided by Xi’an Jiaotong University [19]. The experimental platform is shown in Figure 6. The dataset contains complete vibration data of 15 bearings under three operating conditions, spanning from healthy to failure. The data were sampled at 25.6 kHz, with one sample recorded per minute, and each sample lasting 1.26 s. The horizontal vibration signals from this dataset were selected for subsequent experiments. The usage details of the XJTU-SY dataset in this study are summarized in

Table 1. The settings of the train set and test set in XJTU-SY dataset.

Condition	Speed/rpm	Load (kN)	Train Set	Test Set
1	2100	12	Bearing1_1, Bearing1_2 Bearing1_4, Bearing1_5	Bearing1_3
2	2250	11	Bearing2_1, Bearing2_2 Bearing2_4, Bearing2_5	Bearing2_3
3	2400	10	Bearing3_1, Bearing3_2 Bearing3_4, Bearing3_5	Bearing3_3

.

3.1.2. PHM2012 Dataset

The PHM2012 dataset is a publicly available dataset released by the IEEE [20]. It was collected using the PRONOSTIA experimental platform, which is shown in Figure 7. The experimental platform conducted accelerated degradation tests on 17 rolling bearings under three different operating conditions. Accelerometers were used to collect the full-life raw vibration signals of all 17 bearings in both horizontal and vertical directions. The data were sampled at 25.6 kHz, with one sample recorded every 10 s, and each sample lasting 0.1 s. The horizontal vibration signals from this dataset were selected for subsequent experiments. The usage details of the PHM2012 dataset in this study are summarized in

Table 2. The settings of the train set and test set in PHM2012 dataset.

Condition	Speed/rpm	Load (kN)	Train Set	Test Set
1	2100	12	Bearing1_1, Bearing1_2	Bearing1_3
2	2250	11	Bearing2_1, Bearing2_2	Bearing2_3
3	2400	10	Bearing3_1, Bearing3_2	Bearing3_3

.

3.2. Health Index Construction

3.2.1. Feature Extraction

The raw vibration signals generated during the operation of rolling bearings can reflect the health condition of the bearings to some extent. However, the raw vibration signal data are large in volume and contain noise interference, making them unsuitable for direct use in neural network training. By exploring the latent information within the raw vibration signals, features that effectively characterize the health condition of rolling bearings can be identified. In particular, when a bearing develops a fault, the waveform, frequency components, and energy at each frequency of the vibration signals undergo changes. Therefore, it is crucial to extract time-domain, frequency-domain, and time–frequency features from the raw vibration signals using feature extraction techniques.

In this study, 17 time-domain features are selected, including the maximum, minimum, median, peak-to-peak value, mean of absolute values, variance, standard deviation, kurtosis, skewness, root mean square (RMS), mean square value, square root amplitude, waveform factor, peak factor, impulse factor, margin factor, and others. Let $p_i(i=1,2,\dots ,17)$ denote the 17 time-domain features. Five frequency-domain features are selected, including the spectral centroid, mean square frequency, root mean square frequency, and frequency variance. Let $p_i(i=18,19,20,21,22)$ denote these five frequency-domain features. Four spectral kurtosis-related features are selected, including the mean, standard deviation, skewness, and kurtosis of the spectra. Let $p_i(i=23,24,25,26)$ denote these four spectral kurtosis features. Eight time–frequency features are selected, including the energy proportions of the eight nodes at the third level of wavelet packet decomposition. Let $p_i(i=27,28,29,30,31,32,33,34)$ denote these eight time–frequency features.

3.2.2. Feature Selection and Fusion

Not all the features obtained from time-domain, frequency-domain, and time–frequency analyses can effectively reflect the degradation state of bearings. To prevent low-performance features from adversely affecting the subsequent prediction accuracy, it is necessary to establish performance metrics for the features and perform feature selection. The performance of a bearing is an irreversible process that continuously degrades over time. Therefore, degradation features are required to exhibit strong temporal correlation and monotonicity. In addition, features need to be resistant to random noise; they should also exhibit strong robustness.

To evaluate the candidate features, each feature is decomposed into a stationary trend and a random residual using the moving average method formulated as (19). Let the time length of the feature parameter be k. The time series can be represented as $T=[t_1,t_2,\dots ,t_k]$, and the feature parameter sequence can be represented as $X=\left[x\right(t_1),x(t_2),\dots ,x(t_k\left)\right]$. $X_T$ represents the stationary trend of the feature parameter sequence and $X_R$ represents the random component of the feature parameter sequence.

$$X=X_T+X_R$$

(19)

Let the correlation metric be denoted as Corr, the monotonicity metric as Mon, and the robustness metric as Rob. The calculation formulas for these three performance metrics are given as follows:

Correlation metric:

$$Corr(X,T)={\displaystyle \frac{\left|k{\sum }_{i=1}^{k}x\left(t_i\right)t_i-{\sum }_{i=1}^{k}x\left(t_i\right){\sum }_{i=1}^k{t}_i\right|}{\sqrt{\left[k{\sum }_{i=1}^k{\left(x\right(t_i\left)\right)}^2-{\left({\sum }_{i=1}^{k}x\left(t_i\right)\right)}^2\right]}\left[k{\sum }_{i=1}^k{\left(t_i\right)}^2-{\left({\sum }_{i=1}^k{t}_i\right)}^2\right]}}$$

(20)

Monotonicity metric:

$$Mon\left(X\right)={\displaystyle \frac{1}{k-1}}\left|\sum _{i=1}^k\left[x\left(t_{t-1}\right)-x\left(t_i\right)\right]\right|$$

(21)

Robustness metric:

$$Rob\left(X\right)={\displaystyle \frac{1}{k}}\sum _{i=1}^k\mathrm{e}\mathrm{x}\mathrm{p}(-\left|{\displaystyle \frac{X_R\left(t_i\right)}{X\left(t_i\right)}}\right|)$$

(22)

The correlation, monotonicity, and robustness metrics of each feature are calculated through Equations (20)–(22). These three evaluation metrics are then combined by a weighted approach, formulated as Equation (23), to obtain the degradation evaluation metric, $T_o$, for the bearing features.

$$T_o={\omega }_{1}Corr\left(X,T\right)+{\omega }_{2}Mon\left(X\right)+{\omega }_{3}Rob\left(X\right), \left\{\begin{array}{c}{\omega }_i>0\\ {\sum }_{i=1}^3{\omega }_i=1\end{array}\right.$$

(23)

In the equation, ${\omega }_1$, ${\omega }_2$, and ${\omega }_3$ are the weighting coefficients. According to Reference [21], the three weights are set to 0.2, 0.5, and 0.3. A higher value of the evaluation metric $T_o$ indicates better performance of the feature in representing the bearing degradation. The evaluation metrics $T_o$ are ranked in descending order, and features with $T_o>0.6$ are selected to form the feature set for training the RUL prediction model.

The selected feature set is subjected to dimensionality reduction through the ISOMAP algorithm. The number of neighbors for each data point is set to 10, and the target dimensionality is set to one.

The label column used as input to the network in this study adopts a piecewise labeling scheme. Using the one-dimensional HI obtained via ISOMAP dimensionality reduction as a reference, the first 20% of the HI values are considered as the normal operation stage of the bearing. The degradation starting point of the bearing is then determined based on the 3σ criterion, and a piecewise bearing degradation label is constructed accordingly.

3.3. Bearing RUL Prediction Results

In the proposed xLSTM–Transformer fusion network, the xLSTM layer is composed of a stacked structure consisting of one mLSTM layer and one sLSTM layer. The detailed hyperparameters configurations are summarized in

Table 3. Setting of hyperparameters.

Parameter	Value	Parameter	Value
Learning Rate	0.001	Sequences Length	10
Batch size	32 or 64	Hidden dimensions	16–32
Training Epochs	50	Dropout Rate	0.1
Optimizer	Adam	Loss Function	MSE

. Considering training efficiency and the length of the input data, the batch size is set to 32 for sequences shorter than 500 and increased to 64 for those exceeding 500. To ensure a fair comparison, the hidden layer dimensions were set within the range of 16 to 32, ensuring that the number of trainable parameters for all models remained at a comparable level.

Both the proposed method and the comparative methods are implemented based on the PyTorch 2.7.1 framework. The training and testing environments are equipped with an AMD Ryzen 9700X CPU, an NVIDIA GeForce RTX 5070 GPU, and 32 GB of RAM.

To evaluate the performance of the RUL prediction models, the RMSE and R² are employed as evaluation metrics. Lower values of RMSE indicate better model performance, while a higher value of R² indicates better performance. The calculation formulas for these three metrics are given in Equations (24) and (25).

$$RMSE=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^k{(y_i-p_r)}^2}{k}}}$$

(24)

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^k{(y_i-p_r)}^2}{{\sum }_{i=1}^k{(\overline{y}-p_r)}^2}}$$

(25)

where $y_i$ denotes the true value, $p_r$ denotes the predicted value, $\overline{y}$ denotes the mean of the true values, and k denotes the length of the bearing life sequence.

To further evaluate the reliability of the proposed method, a Score function, which is defined in the IEEE PHM 2012 Prognostic challenge, was employed [20]. Unlike symmetric metrics such as RMSE, this Score function imposes different penalty coefficients for early and late predictions, which aligns with industrial maintenance requirements where late predictions carry higher risks. A lower value of the Score indicates better model performance. The Score function is formulated as Equations (26)–(28).

$$Erro{r}_i={\widehat{y}}_i-y_i$$

(26)

$$s_i=\left\{\begin{array}{c}{e}^{-Erro{r}_i/13}-1, if Erro{r}_i<0\\ e^{Erro{r}_i/10}-1, if Erro{r}_i>0\end{array}\right.$$

(27)

$$Score=\sum _{i=1}^n{s}_i$$

(28)

where ${\widehat{y}}_i$ denotes the predicted value, $y_i$ denotes the true value, n denotes the total number of samples.

3.3.1. Case 1: Validation on the XJTU-SY Dataset

To validate the effectiveness and superiority of the proposed RUL prediction method, experiments are first conducted using the XJTU-SY dataset. The usage details of the dataset are presented in Table 1. To improve prediction accuracy and increase the dataset size while preventing overfitting due to the complexity of the proposed fusion network, the input feature set of the network includes not only the HI obtained via the ISOMAP method described above but also a time index column.

Six networks—LSTM, xLSTM, Transformer, LSTM–Transformer, BiLSTM–Transformer, and GRU–Transformer—are selected for comparative experiments. The prediction results are shown in Figure 8.

As shown in Figure 8, the RUL prediction curves of the seven neural networks can all reflect the degradation trend of the bearing to some extent, but noticeable differences exist in prediction accuracy, smoothness, and degradation responsiveness.

The LSTM prediction results are able to closely track the actual RUL changes during the early degradation stage, exhibiting smooth and stable fitting. However, during the mid-degradation stage, the prediction results decline too rapidly, and in the late degradation stage, the predictions exhibit a noticeable lag, failing to accurately reflect the actual RUL decrease. This indicates that LSTM has limitations in capturing nonlinear degradation features and long-term dependencies.

xLSTM demonstrates good smoothness and stability in capturing the overall trend, but it exhibits instances of both underestimation and overestimation of the RUL. Although xLSTM improves the gating mechanism and enhances feature memory capability, it tends to overestimate the RUL in the late degradation stage, and the predicted RUL still decreases more slowly than the actual decline.

The Transformer is capable of effectively learning global temporal dependency features, and its prediction curve aligns well with the overall trend of the actual RUL. However, due to the high sensitivity of the attention mechanism, the prediction curve exhibits an upward trend during the early degradation stage. The LSTM–Transformer combined network integrates the smoothness of LSTMs with the global feature extraction capability of the Transformer, but its performance improvement is limited due to the inherent constraints of LSTMs.

Compared with the aforementioned networks, the proposed xLSTM–Transformer network achieves the best prediction results. The network produces a smooth and stable prediction curve during the early degradation stage. It can accurately track the degradation onset and trend in the mid-degradation stage and sensitively capture the actual RUL decline rate in the late degradation stage. As a result, the prediction curve closely fits the actual RUL overall. This indicates that the synergistic effect of the xLSTM’s improved gating structure and the Transformer’s self-attention mechanism enables the combined network to achieve a good balance between smoothness and accuracy, thereby enhancing the reliability of RUL prediction.

To further evaluate the performance of the networks in RUL prediction, a quantitative comparison of the seven aforementioned networks is conducted. The evaluation metric results under Condition 1, Condition 2, and Condition 3 are presented in

Table 4. Prediction errors under different conditions in XJTU-SY.

Condition	1			2			3
Evaluation Metrics	RMSE	R2	Score	RMSE	R2	Score	RMSE	R2	Score
LSTM	0.0982	0.9121	0.8531	0.1284	0.8441	4.2450	0.0819	0.8053	1.1687
xLSTM	0.0778	0.9449	0.8707	0.1123	0.8806	3.6941	0.0611	0.8915	1.6017
Transformer	0.0885	0.9287	0.9666	0.1110	0.8833	4.5334	0.0742	0.8401	0.9482
LSTM–Transformer	0.0666	0.9596	0.6788	0.0942	0.9160	3.2863	0.0574	0.9045	0.8114
BiLSTM–Transformer	0.0651	0.9614	0.6835	0.0914	0.9205	2.7122	0.0615	0.8903	0.9686
GRU–Transformer	0.0686	0.9572	0.6978	0.0888	0.9255	3.0362	0.0593	0.8979	1.0050
Proposed method	0.0583	0.9691	0.5572	0.0784	0.9418	2.5777	0.0532	0.9179	0.8410

.

As the results demonstrated in Table 4, the performance of individual models remains inconsistent across different conditions. For instance, xLSTM outperforms the Transformer in Conditions 1 and 2, whereas the Transformer slightly surpasses xLSTM in Condition 3. This reflects the inherent limitations of single-architecture models. However, the introduction of the hybrid architecture leads to a significant improvement in prediction accuracy. In particular, the proposed method consistently outperforms all single models across various metrics. It demonstrates that the hybrid architecture approach achieves a synergistic integration of global trends and local dynamics, effectively compensating for the shortcomings of individual models.

More importantly, a comparison between the proposed method and the closely related hybrid architecture, LSTM–Transformer, reveals that the proposed method exhibits superior performance. This superiority is attributed to the exponential gating and matrix memory structures of xLSTM, which provided a more robust nonlinear fitting capability and a broader feature representation space compared to the scalar memory of standard LSTM. These architectural advantages allow for a more complete retention of the deep representations extracted by the Multi-Head Attention mechanism, thereby demonstrating exceptional performance in RUL prediction tasks. Furthermore, the proposed method demonstrates superior overall performance in terms of the Score. This indicates that the proposed method not only improves prediction accuracy but also effectively minimizes the risk of late predictions, thereby offering superior reliability for industrial maintenance safety.

3.3.2. Case 2: Validation on the PHM2012 Dataset

To evaluate the generalizability of the proposed method and further validate its effectiveness, experiments are also conducted on the PHM2012 dataset. The usage details of the dataset are shown in Table 2. The experimental environment and procedures are the same as those described in the XJTU-SY experiments. The prediction results are shown in Figure 9.

From Figure 9, the overall prediction trends of the networks are consistent with those observed in the XJTU-SY experiments. LSTM still exhibits significant lag in the late degradation stage, indicating its limited capability in predicting complex nonlinear degradation processes. xLSTM shows certain improvements over LSTM, but issues of both underestimation and overestimation of the RUL still exist. The prediction results of the Transformer closely follow the overall trend of the actual RUL; however, fluctuations are still observed during the early degradation stage. The incorporation of LSTMs improves the prediction accuracy of the Transformer; however, lag is observed during the mid-degradation stage.

In contrast, the proposed xLSTM–Transformer network demonstrates good consistency with the actual RUL in both the stable and degradation stages. Overestimation or lag observed in the previously mentioned networks is mitigated, resulting in a prediction curve that fits the actual RUL more closely.

The quantitative analysis results of the comparative experiments under Condition 1, Condition 2, and Condition 3 are presented in

Table 5. Prediction errors under different conditions in PHM2012.

Condition	1			2			3
Evaluation Metrics	RMSE	R2	Score	RMSE	R2	Score	RMSE	R2	Score
LSTM	0.1275	0.8502	21.9981	0.0725	0.5705	6.1899	0.1426	0.8154	4.4517
xLSTM	0.1202	0.8670	22.4332	0.0693	0.6082	9.3849	0.1408	0.8202	4.7340
Transformer	0.1138	0.8807	18.6070	0.0675	0.6281	4.5328	0.1376	0.8284	3.6868
LSTM–Transformer	0.1007	0.9067	17.9940	0.0856	0.4021	8.0399	0.1335	0.8384	3.6223
BiLSTM–Transformer	0.0961	0.9150	15.8698	0.0803	0.4738	8.2596	0.1247	0.8589	3.3644
GRU–Transformer	0.0980	0.9117	16.3259	0.0652	0.6531	4.1951	0.1259	0.8563	3.3398
Proposed method	0.0565	0.9706	10.1616	0.0651	0.6539	4.1367	0.1211	0.8671	3.4079

.

As observed from Table 5, the proposed method consistently achieves the best prediction performance across different working conditions. Compared with the other networks, it exhibits lower errors and higher fitting accuracy.

Overall, based on the analyses of the experiments presented in the above sections, the proposed method maintains high prediction accuracy and trend consistency across different datasets and operating conditions. This demonstrates the strong applicability and robust cross-scenario adaptability of the proposed method for bearing RUL prediction.

3.3.3. Discussion

This study achieves high-precision RUL prediction on two distinct datasets, XJTU-SY and PHM2012. This demonstrates the superior generalization capability of the proposed method. This success is mainly attributed to the synergistic ability of the hybrid xLSTM–Transformer architecture to capture both short-term local variations and long-term global trends.

Specifically, the bearing degradation process includes both irreversible long-term wear trends and local intense fluctuations. Compared to the scalar memory of standard LSTMs, the matrix memory introduced by xLSTM offers a larger feature storage capacity. This allows for the retention of richer fluctuation information. Combined with the exponential gating mechanism, the model effectively avoids the numerical compression issue in long-sequence inputs, thereby accurately capturing local degradation details even in highly dynamic datasets like PHM2012. Furthermore, the Transformer’s self-attention mechanism excels at modeling long-range dependencies, ensuring the prediction curve globally adheres to the bearing’s life decay trajectory, and correcting the potential local overfitting risks of the xLSTM. This complementary fusion of “local-global” features enables the model to exhibit strong representation capabilities across different operating conditions and degradation stages.

Despite the overall excellent performance, there remains room for improvement under specific complex conditions. Notably, in Condition 2 of the PHM2012 dataset, the goodness of fit (R²) for all models showed a marked decline. Although the proposed method achieved the best result in this group, a significant gap remains compared to Condition 1. This indicates that the robustness of current feature extraction and model structure still faces challenges when encountering extremely atypical degradation patterns. Additionally, while the proposed method improves accuracy, its complex stacked structure and Multi-Head Attention mechanism increase computational overhead.

To address these limitations, future work will focus on two main aspects: First, enhancing model robustness under extreme conditions. To overcome the performance degradation observed in Condition 2 of the PHM2012 dataset, future research aims to introduce data augmentation strategies, such as injecting random noise during training to simulate harsher operating environments. Additionally, transfer learning could be employed, utilizing models trained under standard conditions as a baseline and fine-tuning them with a small number of samples to adapt to extreme conditions. Second, research on model lightweighting. To meet the real-time requirements of industrial sites, the model needs to be optimized for efficiency. Future work will explore model pruning techniques to remove unimportant weights or neurons from the neural network. The goal is to appropriately reduce the parameter count and computational load without significantly compromising prediction accuracy.

4. Conclusions

This paper proposes a hybrid network combining the xLSTM and Transformer architectures to improve the prediction accuracy of bearing RUL. The network integrates the local temporal dependency modeling capability of the xLSTM with the global attention mechanism of the Transformer. By leveraging enhanced deep temporal modeling ability, it accurately characterizes the degradation process and enhances the network’s generalization capability.

To validate the effectiveness and stability of the proposed network, comparative experiments were conducted under different operating conditions on two distinct datasets. The LSTM, xLSTM, Transformer, and LSTM–Transformer networks were selected as benchmarks. The results indicate that the proposed method accurately fits the degradation trend and achieves the best performance in evaluation metrics such as RMSE, R², and the Score. Through validation on different datasets, the proposed method maintains high prediction accuracy, demonstrating superior generalization performance and robustness.

In summary, the proposed xLSTM–Transformer network demonstrates excellent prediction stability, generalization capability, and practical value across different datasets. It shows strong potential for engineering applications, providing a novel approach for time-series-based remaining useful life prediction in equipment health management.