BatteryMoE: An Improved Sparse Mixture-of-Experts Transformer for Battery SOH Estimation and RUL Prediction

Abstract

Accurate State of Health (SOH) estimation and Remaining Useful Life (RUL) prediction on uninterruptible power supply (UPS) batteries are critical for ensuring system reliability. However, battery degradation processes exhibit significant non-linearity and non-stationarity, posing challenges for traditional deep learning models in capturing long-term dependencies and handling sensor noise. In this paper, we propose BatteryMoE, a novel normalization-free sparse Mixture-of-Experts (MoE) Transformer designed for robust battery prognostics. Distinct from standard Transformers, BatteryMoE incorporates three key physics-aware architectural innovations: (1) It introduces a dynamic error function mechanism to replace traditional normalization layers, constructing a normalization-free architecture that enhances robustness against high-variance battery data. (2) It adopts an imaginary-enhanced Rotary Positional Embedding that utilizes the full complex-valued attention score to capture both local charge–discharge patterns and global long-term degradation trends, significantly improving RUL extrapolation. (3) A sparse MoE strategy is employed to dynamically activate specialized experts for different aging stages (e.g., linear degradation vs. capacity plunge), balancing model capacity with robust predictive performance. Extensive experiments conducted on five widely used datasets show that BatteryMoE achieved significant results.

1. Introduction

Lithium-ion batteries (LIBs) have become the dominant energy storage solution for mission-critical applications ranging from uninterruptible power supply (UPS) systems in communication base stations and data centers to electric vehicles (EVs) and grid-scale energy storage [1,2,3]. According to the International Data Corporation (IDC), the global Ethernet switch market reached $14.5 billion in Q2 2025, with non-data center segments (campus and branch networks) growing 12.5% year over year, underscoring the expanding infrastructure reliance on reliable backup power. In these safety-critical deployments, the Battery Management System (BMS) plays a pivotal role in ensuring operational continuity, preventing catastrophic failures, and optimizing maintenance schedules through accurate estimation of State of Health (SOH) and prediction of Remaining Useful Life (RUL) [4,5].

However, LIB degradation poses formidable prognostic challenges. The aging process is inherently complex and non-stationary, governed by multiple electrochemical mechanisms including loss of lithium inventory (LLI), loss of active material (LAM), and solid electrolyte interphase (SEI) film growth [6,7]. These degradation pathways exhibit highly non-linear trajectories, with statistical distributions of battery features (e.g., voltage and current profiles and internal resistance) shifting continuously over operational time [8]. Moreover, real-world battery data is often corrupted by sensor noise, measurement artifacts, and the phenomenon of capacity regeneration—a temporary recovery in capacity after rest periods—which further confounds traditional data-driven models [9,10]. While physics-based models such as electrochemical impedance spectroscopy (EIS) and pseudo-two-dimensional (P2D) models offer interpretability and mechanistic insights, they rely heavily on simplified assumptions, require extensive parameter identification (often necessitating invasive testing), and incur prohibitive computational costs for real-time BMS deployment [11,12,13].

Consequently, data-driven approaches, particularly deep learning (DL), have gained prominence in battery prognostics. Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks have been widely adopted to capture temporal dependencies in sequential battery degradation data [4,14,15]. Recent studies have demonstrated that hybrid architectures combining convolutional neural networks (CNNs) with LSTM or gated recurrent units (GRUs) can effectively extract both local features and long-term dependencies, achieving promising SOH estimation and RUL prediction accuracy [16,17,18]. For instance, Li et al. [19] proposed a TCN-GRU-DNN model with dual attention mechanisms to capture capacity regeneration and degradation trends, while Cai et al. [20] developed the DIICAN framework using dual time-scale attention to jointly estimate SOC, SOH, and RUL. Advanced feature engineering approaches have also been explored, such as SSA-DNN for fast EIS acquisition [21] and MC-CNN-TimesNet for multi-scale temporal modeling [22]. Nonetheless, these sequential models suffer from the vanishing gradient problem, lack parallelization capabilities for long-cycle sequence prediction, and struggle with length extrapolation beyond training horizons—a critical limitation for RUL forecasting under evolving operational conditions. Recent studies have demonstrated that LSTM networks remain highly effective for applied engineering and IoT infrastructure problems, particularly when properly tuned to account for seasonality and external factors [23,24]. However, the multivariate complexity of battery degradation—where features like temperature and current interact non-linearly with aging—necessitates even more sophisticated architectures, driving the transition towards Transformer and MoE models capable of capturing both cyclical patterns and long-term distribution shifts.

Recently, the Transformer architecture, underpinned by the self-attention mechanism [25], has revolutionized time-series forecasting by effectively modeling long-range dependencies without recurrence [26]. Several pioneering studies have successfully applied Transformers to battery SOH estimation and RUL prediction [27,28,29,30,31,32]. Notably, Hu et al. [27] integrated wavelet threshold denoising with a Transformer model for RUL prediction, achieving robust generalization in noisy environments, while Gu et al. [28] combined CNN feature extraction with Transformer paradigms to attain remarkable SOH estimation accuracy. More recently, Zhou et al. [29] proposed a denoising autoencoder-enhanced Transformer for capacity degradation prediction, and a generative pre-trained Transformer approach (BatteryGPT) [30] demonstrated exceptional early-stage degradation forecasting with just 30% of the lifecycle data. Despite these advancements, standard Transformers incur quadratic computational complexity with respect to sequence length ($O\left(n^2\right)$), rendering them computationally prohibitive for resource-constrained BMS in real-time UPS and EV deployments. Moreover, conventional components such as Layer Normalization (LayerNorm) and Positional Embedding (PE) are inherited from Natural Language Processing (NLP) tasks and are not optimized for the continuous, noisy, and drifting nature of battery sensor data [33,34].

To address the fundamental dilemma between model capacity and computational efficiency, the Mixture-of-Experts (MoE) architecture has emerged as a outstanding solution in large-scale modeling [35]. By activating only a sparse subset of “expert” networks for each input token via a gating mechanism, MoE models can scale up parameters dramatically without a proportional increase in inference cost. The recent Time-MoE framework demonstrated the feasibility of scaling time-series foundation models up to 2.4 billion parameters, achieving state-of-the-art zero-shot forecasting performance across diverse domains [36]. However, directly applying Time-MoE to battery prognostics exposes significant limitations: (1) Its reliance on Root Mean Square Normalization (RMSNorm) is suboptimal for handling the outliers, high-frequency noise, and rapid transients inherent in battery current/voltage measurements. (2) The standard Rotary Positional Embedding (RoPE) [37] lacks the long-context extrapolation capability required for multi-year RUL prediction beyond training horizons, and (3) it does not explicitly account for the global distribution shift caused by progressive battery aging, where the mean and variance of sensor signals drift non-stationarily over hundreds or thousands of charge–discharge cycles [38].

In this paper, we propose BatteryMoE, an improved normalization-free sparse Mixture-of-Experts Transformer specifically tailored for robust SOH estimation and accurate RUL prediction in real-world battery prognostics. Bridging the gap between advanced foundation model architectures and battery electrochemical characteristics, BatteryMoE introduces a novel “RevIN-Derf-MoE” design. First, we replace the standard RMSNorm with a Dynamic error function mechanism [39], constructing a normalization-free attention block that substantially improves generalization on noisy, non-stationary sensor data by adaptively gating feature channels without the instability of traditional normalization layers. Second, to enhance long-horizon RUL extrapolation, we incorporate an imaginary-enhanced Rotary Positional Embedding, which re-introduces the imaginary component discarded in vanilla RoPE, thereby strengthening the capture of long-context dependencies and enabling robust length extrapolation beyond training sequences. Third, we employ Reversible Instance Normalization [38] at the model input and output layers to explicitly mitigate temporal distribution shifts by removing and restoring instance-level mean and variance, effectively isolating aging-induced drift from intrinsic battery dynamics. Finally, we integrate physics-aware feature engineering Incremental Capacity (IC) curve features [40,41] derived from partial charging data, enriching the model with electrochemical domain knowledge while maintaining computational efficiency.

The main contributions of this paper are summarized as follows:

• We propose BatteryMoE, a specialized sparse Mixture-of-Experts Transformer architecture for battery SOH estimation and RUL prediction. Our work integrates normalization-free MoE with physics-aware features for battery health management, achieving a balance between high prediction accuracy, low inference latency, and long-horizon extrapolation capability. Our contribution lies in aligning advanced attention mechanisms with the unique non-stationary and noisy characteristics of battery degradation data both theoretically and empirically.
• We introduce a normalization-free design leveraging the dynamic error function (Derf) activation mechanism combined with imaginary-enhanced Rotary Positional Encoding (RoPE++) and Reversible Instance Normalization (RevIN) layers. This synergistic design significantly enhances the model’s robustness against sensor noise, mitigates temporal distribution shifts, and enables reliable extrapolation of degradation trends beyond the training horizon—factors critical for practical RUL forecasting in aging battery systems.
• We incorporate physics-aware feature engineering Incremental Capacity Analysis (ICA) as an auxiliary feature stream, enabling the model to leverage electrochemical domain knowledge without requiring complex parameter identification or invasive testing procedures, thereby enhancing interpretability and cross-battery generalization.
• We conduct extensive experiments on multiple public UPS battery datasets (NASA, CALCE, MIT, XJTU, and TJU) and real-world fast-charging datasets. BatteryMoE consistently achieves the lowest MAE and RMSE across all the datasets, demonstrating superior generalization capabilities for both SOH estimation and RUL prediction tasks across diverse operating conditions and battery chemistries.

The remainder of this paper is organized as follows: Section 1 reviews related work on battery SOH/RUL estimation methods and Transformer-based time-series forecasting. Section 2 introduces the background of batteries in power distribution communication networks. Section 3 presents the proposed BatteryMoE architecture in detail, including the normalization-free MoE block, RoPE++ positional encoding, RevIN layers, and IC feature extraction. Section 4 describes the experimental setup, datasets, evaluation metrics, and baseline models and also presents comprehensive experimental results and ablation studies. Finally, Section 5 concludes the paper.

2. Background

2.1. UPS Battery Prognostics in Communication Networks

In power distribution communication networks, network switches are critical nodes responsible for data exchange and instruction transmission. To ensure uninterrupted operation during grid outages, these switches are equipped with UPS systems, typically relying on lithium-ion battery packs. Unlike electric vehicle batteries characterized by frequent charge–discharge cycling, UPS batteries predominantly operate in a float-charge mode, serving as a standby power reserve. However, they must perform reliably during sporadic discharge events. The degradation of UPS batteries is influenced by factors such as ambient temperature fluctuations in outdoor cabinets, as well as occasional deep discharge cycles. Accurate prognosis of the battery’s health status is vital to prevent “silent failures”—where a battery appears healthy under float voltage but fails immediately under load. Therefore, developing a robust data-driven model that can infer health states from charging/discharging data segments is crucial for the reliability of the communication network.

2.2. SOH Estimation and RUL Prediction

(1) State of Health (SOH): SOH is a quantitative indicator of a battery’s current performance relative to its fresh state. In this work, we define SOH based on the maximum available capacity. Let $Q_{rated}$ be the nominal capacity provided by the manufacturer, and let $Q_t$ be the maximum discharge capacity measured at the current aging cycle t. The SOH is defined as

$${\mathrm{SOH}}_t={\displaystyle \frac{Q_t}{Q_{rated}}}\times 100\%,$$

(1)

Typically, a battery is considered to have reached its end of Life (EOL) when the SOH drops to 80%.

(2) Remaining Useful Life (RUL): RUL represents the number of remaining operational cycles (or time) before the battery reaches the EOL threshold. RUL prediction is initialized at a specific Start-of-Prediction (SOP) point, denoted as $C_{start}$. In this work, we follow the protocol where the model starts predicting the future trajectory when the battery reaches a certain cycle (e.g., we demonstrate predictions starting from 20% of the total life). Then we employ an iterative rollout (recursive strategy). During the RUL test phase, to predict a lifespan longer than predition horizon H, the model’s output for the current step is fed back as input for the subsequent step. This allows us to forecast far beyond the training horizon. The RUL parameter is defined as the number of cycles remaining from the SOP until the predicted SOH trajectory crosses the EOL threshold (80%). We use linear interpolation between the cycle before crossing and the cycle after crossing to determine the precise EOL point.

$${\mathrm{RUL}}_t=C_{EOL}-C_{start}$$

(2)

Accurate RUL prediction involves forecasting the future degradation trajectory of the SOH sequence $\mathcal{S}=\{SO{H}_{t+1},SO{H}_{t+2},\dots ,SO{H}_{C_{EOL}}\}$ based on historical monitoring data.

2.3. Incremental Capacity Analysis

While voltage and current are direct measurements, they often fail to reveal subtle internal degradation mechanisms such as loss of lithium inventory and loss of active material. ICA transforms the voltage plateaus of the charging curve into identifiable peaks, serving as a powerful “fingerprint” of the battery’s electrochemical state. The IC curve is obtained by differentiating the charging capacity Q with respect to the terminal voltage V:

$$\mathrm{IC}\left(V\right)={\displaystyle \frac{\mathrm{d}Q}{\mathrm{d}V}}\approx {\displaystyle \frac{\Delta Q}{\Delta V}}={\displaystyle \frac{Q_{k+1}-Q_k}{V_{k+1}-V_k}}$$

(3)

where k denotes the sampling point within a single charging cycle. As the battery ages, the peaks in the IC curve exhibit characteristic changes: (1) Peak intensity reduction correlates with the loss of active material. (2) A peak position shift indicates an increase in internal resistance and polarization.

Since raw $\mathrm{d}Q/\mathrm{d}V$ data is extremely sensitive to measurement noise as illustrated in Figure 1, we apply a Gaussian filter to smooth the curve before extracting features. These smoothed IC curves serve as physics-informed inputs for our BatteryMoE model, enabling it to learn from both macroscopic trends and microscopic electrochemical changes.

3. Methodology

3.1. Problem Formulation and Architecture Overview

The objective of BatteryMoE is to predict the future SOH and RUL of the batteries based on historical monitoring data. Let $\mathcal{X}=\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_t,\dots ,{\mathbf{x}}_L\}\in {\mathbb{R}}^{L\times D_{in}}$ denote the input sequence of length L, where ${\mathbf{x}}_t\in {\mathbb{R}}^{D_{in}}$ represents the multi-dimensional sensor readings at cycle t. The model aims to map $\mathcal{X}$ to the future degradation trajectory $\widehat{\mathcal{Y}}=\{{\widehat{y}}_{L+1},\dots ,{\widehat{y}}_{L+H}\}$, where H is the prediction horizon.

As illustrated in Figure 2, the overall architecture of BatteryMoE follows a decoder-only Transformer paradigm enhanced with MoE.

UPS batteries of different brands, models, and usage environments have significant differences in initial capacity, voltage range, and decay rate. To eliminate scale differences between different batteries and focus the model on degradation patterns rather than absolute values, we employ RevIN as an independent standardization step for each battery sequence. The input sequence is first normalized by RevIN to remove global statistical shifts, processed by the BatteryMoE decoder, and finally denormalized to restore the physical scale of the predictions:

$${\mathbf{x}}^{\prime }=\mathrm{RevIN}\left(\mathbf{x}\right),\widehat{y}={\mathrm{RevIN}}^{-1}\left(\mathrm{Decoder}\left({\mathbf{x}}^{\prime }\right)\right)$$

(4)

3.2. Physics-Aware Input Feature Engineering

Standard time-series models often rely solely on raw voltage and current, which may obscure subtle electrochemical changes like loss of lithium inventory. We augment the input space with physics-aware features derived from ICA.

(1) Feature Extraction and Alignment: The IC curves are computed exclusively during the constant-current charging phase to avoid the polarization complexities of the constant-voltage phase. To resolve misalignments caused by partial charging and varying voltage ranges across cycles, we strictly interpolate the raw $dQ/dV$ data onto a fixed voltage grid spanning from $3.0\mathrm{V}$ to $4.0\mathrm{V}$ with a fixed resolution of $10\mathrm{mV}$. This alignment procedure guarantees a consistent 100-dimensional IC feature vector for every cycle.

(2) Quantization Noise Smoothing: We then apply a 1D Gaussian filter with a kernel size of 11 and a standard deviation $\sigma =1.5$ to the interpolated vector before concatenating it point-wise with the scalar voltage V, current I, and temperature T values. A sensitivity analysis indicated that $\sigma <1$ failed to eliminate sensor quantization noise, while $\sigma >2$ caused over-smoothing, erasing critical IC peak shifts corresponding to active material loss. The chosen parameters optimally balance noise reduction and feature preservation.The input vector ${\mathbf{x}}_t$ is constructed by concatenating the raw measurements V, I, and T with the smoothed IC values.

(3) Point-wise Tokenization and SwiGLU: To preserve high-frequency temporal details essential for capturing capacity regeneration phenomena, we adopt point-wise tokenization rather than patch-based methods. The raw features are projected into the hidden dimension $d_{model}$ using a SwiGLU activation layer, which offers superior performance in gating information flow compared to standard ReLU:

$${\mathbf{h}}_t^0=\mathrm{SwiGLU}({\mathbf{W}}_{in}{\mathbf{x}}_t+{\mathbf{b}}_{in})=(\mathrm{Swish}\left({\mathbf{W}}_1{\mathbf{x}}_t\right)\otimes \left({\mathbf{V}}_1{\mathbf{x}}_t\right)){\mathbf{W}}_2$$

(5)

3.3. Normalization-Free Block with Derf

Traditional Transformers rely on LayerNorm or RMSNorm to stabilize training. However, normalization can inadvertently suppress the magnitude of features that represent the absolute severity of battery degradation. To mitigate this, BatteryMoE adopts a normalization-free architecture with Derf.

Derf is a point-wise non-linear mapping based on the error function, which naturally bounds extreme outliers (common in battery sensor noise) while maintaining gradient flow. For a hidden state input $\mathbf{h}$, the Derf operation is defined as

$$\mathrm{Derf}\left(\mathbf{h}\right)=\gamma \odot \mathrm{erf}(\alpha \mathbf{h}+\mathbf{s})+\beta$$

(6)

where $\mathrm{erf}\left(x\right)={\displaystyle \frac{2}{\sqrt{\pi }}}{\int }_0^x{e}^{-t^2}dt$. The scaling parameter $\alpha$, the shift parameter $\mathbf{s}$ and the per-channel vector $\gamma ,\beta$ are all learnable. Unlike RMSNorm which enforces a rigid statistical distribution, Derf acts as a soft saturation mechanism, allowing the model to adaptively filter out high-frequency noise while preserving genuine electrochemical anomalies.

From an optimization perspective, the derivative of the Derf function, ${\displaystyle \frac{d}{dh}}\mathrm{erf}\left(\alpha h\right)={\displaystyle \frac{2\alpha }{\sqrt{\pi }}}{e}^{-{\left(\alpha h\right)}^2}$, strictly bounds the gradient. This ensures that extreme high-frequency sensor outliers (where h is exceptionally large) produce near-zero gradients, effectively stabilizing the training dynamics without requiring explicit gradient clipping, unlike standard RMSNorm which shifts the entire distribution based on outlier-skewed variance.

3.4. Imaginary-Enhanced Rotary Positional Embedding

Capturing long-term dependencies is crucial for RUL prediction, as the end-of-life state depends on the cumulative aging history. Standard Rotary Positional Embeddings (RoPEs) only utilize the real component of the complex-valued attention score, discarding phase information that encodes global relative distances. To address this, we introduce RoPE++, which re-incorporates the imaginary component to enhance long-context modeling and extrapolation. In the complex domain, the query ${\mathbf{q}}_t$ and key ${\mathbf{k}}_s$ at positions t and s are rotated by angle $\theta$. RoPE++ computes two distinct attention scores:

• Real Attention (${\mathcal{A}}^{\mathit{Re}}$): It focuses on local semantic locality (similar to standard RoPE).
• Imaginary Attention (${\mathcal{A}}^{\mathit{Im}}$): It focuses on the global context and long-range decay.

• The computation is formulated as

$${\mathcal{A}}_{t,s}^{Re}=\mathrm{Re}\left[\sum _n{\tilde{q}}_t^{\left(n\right)}{({\tilde{k}}_s^{\left(n\right)})}^{*}{e}^{-i{\theta }_n(t-s)}\right]={\mathbf{q}}_t^T{\mathcal{R}}_{\Theta ,s-t}{\mathbf{k}}_s$$

(7)

$${\mathcal{A}}_{t,s}^{Im}=-\mathrm{Im}\left[\sum _n{\tilde{q}}_t^{\left(n\right)}{({\tilde{k}}_s^{\left(n\right)})}^{*}{e}^{-i{\theta }_n(t-s)}\right]={({\mathcal{R}}_{-{\displaystyle \frac{\pi }{2}}}{\mathbf{q}}_t)}^T{\mathcal{R}}_{\Theta ,s-t}{\mathbf{k}}_s$$

(8)

where ${\mathcal{R}}_{\Theta }$ is the rotary matrix and ${\mathcal{R}}_{-{\displaystyle \frac{\pi }{2}}}$ represents a phase shift rotation. By fusing these two scores, BatteryMoE effectively captures both the immediate charge–discharge behaviors (real) and the long-term degradation trends (imaginary), significantly improving the extrapolation accuracy for future cycles outside the training window.

Formally, the expectation of the real attention score decays according to a cosine integral function $Ci(\Delta t)$, which is highly sensitive to local semantics but loses focus on distant tokens. Conversely, the imaginary attention expectation follows a sine integral function $Si(\Delta t)$, which maintains a higher amplitude over larger relative distances $\Delta t$. This mathematical property proves that the imaginary component intrinsically possesses a stronger capability to capture long-range degradation dependencies.

3.5. Sparse Mixture-of-Experts Layer

Battery aging exhibits distinct stages (e.g., linear aging vs. accelerated non-linear aging). A single dense feed-forward network (FFN) often struggles to adapt to these varying dynamics. We replace the standard FFN with a Sparse MoE layer.

The MoE layer consists of N expert networks $\{E_1,\dots ,E_N\}$ and a router G. For each token ${\mathbf{h}}_t$, the router computes a gating probability as

$$G\left({\mathbf{h}}_t\right)=\mathrm{Softmax}\left({\mathbf{h}}_t{\mathbf{W}}_g\right)$$

(9)

To ensure computational efficiency, we employ Top-K routing, activating only a sparse subset of experts (e.g., $K=2$) for each token:

$${\mathbf{y}}_t=\sum _{i\in \mathrm{TopK}\left(G\left({\mathbf{h}}_t\right)\right)}G{\left({\mathbf{h}}_t\right)}_i·E_i\left({\mathbf{h}}_t\right)$$

(10)

This mechanism allows specific experts to specialize in different health stages (e.g., Expert A for fresh batteries and Expert B for the "knee-point" phase), enhancing model capacity without increasing inference latency. Selecting the Top-K out of N experts yields $C(N,K)$ distinct compositional paths. Because the router assigns continuous probability weights to the selected experts, the network dynamically blends representations, providing high flexibility to model complex, hybrid degradation states where multiple electrochemical mechanisms occur simultaneously.

3.6. Multi-Resolution Forecasting Head

To balance short-term accuracy (fluctuation tracking) and long-term trend stability, we employ a multi-resolution output head. The final hidden states are projected through multiple linear layers, each corresponding to a different forecasting resolution $\{r_1,r_2,\dots ,r_m\}$. The final loss is a weighted sum of errors at these different temporal scales, forcing the model to learn representations that are robust across both short and long horizons.

3.7. Implementation Details of Core Components

To ensure exact reproducibility, the specific formulations of the core components within each decoder layer are defined as follows: Derf Insertion: Derf entirely replaces RMSNorm and is applied strictly pre-attention and pre-FFN. No residual scaling is used because the affine parameters $\gamma ,\beta$ within Derf natively handle scale restoration. RoPE++ Fusion: The real and imaginary logits are scaled standardly by $1/\sqrt{d}$ before fusion. The fusion is an unweighted element-wise addition prior to softmax:

$${\mathcal{A}}_{final}=\mathrm{Softmax}({\displaystyle \frac{{\mathcal{A}}^{Re}}{\sqrt{d}}}+{\displaystyle \frac{{\mathcal{A}}^{Im}}{\sqrt{d}}})$$

(11)

MoE Auxiliary Loss: We utilize the Switch-Transformer standard load-balancing loss to prevent expert collapse:

$${\mathcal{L}}_{aux}=\alpha ·N{\displaystyle \sum _{i=1}^N}{f}_i·P_i$$

(12)

where N is the number of experts, $f_i$ is the fraction of tokens dispatched to expert i, $P_i$ is the average routing probability allocated to expert i, and the scaling factor is $\alpha =0.01$. No routing noise or capacity factor dropping was applied, as the sequence lengths were manageable.

Layer-wise Forward Pass:

$${\mathbf{x}}^{\prime }=\mathrm{RevIN}\left(\mathbf{x}\right)$$

(13)

$${\mathbf{h}}_{in}={\mathbf{x}}^{\prime }+\mathrm{SelfAttention}\left({\mathrm{RoPE}}_{++}\left(\mathrm{Derf}\left({\mathbf{x}}^{\prime }\right)\right)\right)$$

(14)

$${\mathbf{h}}_{out}={\mathbf{h}}_{in}+\mathrm{SparseMoE}\left(\mathrm{Derf}\left({\mathbf{h}}_{in}\right)\right)$$

(15)

$$\widehat{\mathbf{y}}={\mathrm{RevIN}}^{-1}\left(\mathrm{Decoder}\left({\mathbf{h}}_{out}\right)\right)$$

(16)

4. Experiment

To rigorously evaluate the performance of BatteryMoE in SOH estimation and RUL prediction, we conducted extensive experiments on five widely recognized open-source battery datasets. We compared our model against a comprehensive set of baselines, ranging from classical recurrent networks to state-of-the-art Transformers.

4.1. Datasets and Preprocessing

We utilized five distinct datasets to ensure the generalization capability of BatteryMoE across different battery chemistries, operational conditions, and aging protocols:

• NASA Battery Dataset: We selected the widely used B0005, B0006, B0007, and B0018 batteries. These were cycled at room temperature (24 °C) with a standard constant-current–constant-voltage (CC-CV) charging and constant-current (CC) discharging protocol.
• CALCE Battery Dataset: We utilized the CS2 series (e.g., CS2-35, CS2-36, CS2-37, and CS2-38), which focuses on high-temperature aging (multi-stage degradation) and provides rigorous benchmarks for SOH tracking.
• MIT Fast-Charging Dataset: This massive dataset involves 124 LFP/Graphite cells cycled under various fast-charging policies. It serves as a stress test to determine our model’s ability to handle rapid degradation and complex charging profiles.
• XJTU Battery Dataset: It provides run-to-failure data for 15 batteries under different discharge rates (1C, 2C, and 3C), allowing us to evaluate robustness against varying load profiles typical in UPS scenarios.
• TJU Battery Dataset: It is used to validate the model’s performance on high-capacity cells with long cycle lives by testing the long-term dependency modeling of the imaginary-enhanced PE module.

Data Preprocessing: For all datasets, we extracted the voltage, current, and temperature profiles. The IC curves were derived using Gaussian smoothing to mitigate sensor noise. We employed a sliding window strategy with a look-back window size of $L=96$ and a prediction horizon of $H\in \{12,24,48,96\}$ to generate input–output pairs.

To strictly prevent data leakage and ensure fair evaluation, we applied dataset-specific split protocols as shown below. (1) Multi-cell Datasets (MIT and XJTU): We utilized a strict leave-cell-out protocol. For the MIT dataset (124 cells), 86 cells were used for training, 13 cells were used for validation, and 25 cells entirely unseen were used for testing. For the XJTU dataset (15 cells), 10 were used for training, two were used for validation, and three were used for testing. (2) Single-cell Datasets (NASA, CALCE and TJU): We applied a chronological split per cell, dividing the datasets into training (60%), validation (20%), and testing (20%) sets. Importantly, window generation was performed strictly within each split. The RevIN model computes the mean and variance dynamically per input sequence window during both training and inference.

4.2. Evaluation Metrics

The prediction performance is comprehensively quantified using two different sets of evaluation metrics.

4.2.1. SOH Estimation Metrics

The accuracy of SOH estimation is evaluated using the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE).

The RMSE measures the magnitude of the prediction errors, effectively assigning higher penalties to larger forecasting deviations:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{N}}\sum _{i=1}^N{(y_i-{\widehat{y}}_i)}^2}$$

(17)

The MAE provides a linear and robust measure of the average magnitude of the estimation errors:

$$\mathrm{MAE}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left|y_i-{\widehat{y}}_i\right|$$

(18)

where $y_i$ is the ground-truth SOH value, ${\widehat{y}}_i$ is the predicted value, and N is the total number of prediction samples within the evaluated horizon.

4.2.2. RUL Prediction Metrics

For RUL prediction, we adopt MAE along with a critical domain-specific metric, the EOL Cycle Error. The MAE evaluates the average absolute cycle deviation between the predicted degradation trajectory and the ground-truth sequence over the forecasting horizon.

The EOL Error specifically quantifies the absolute cycle difference between the true EOL point ($T_{true}$) and the predicted EOL point (${\widehat{T}}_{EOL}$) where the battery’s SOH drops below the 80% failure threshold:

$$Erro{r}_{EOL}=\left|T_{true}-{\widehat{T}}_{EOL}\right|$$

(19)

Furthermore, to ensure the statistical significance, robustness, and strict reproducibility of our evaluation, all metrics reported in the subsequent experiments are computed as the average over five independent runs utilizing different random seeds. The results are systematically denoted in the format of mean ± standard deviation (Std).

4.3. Baselines

We benchmarked BatteryMoE against six established methods that are categorized into recurrent, convolutional, and Transformer-based architectures:

• LSTM and GRU: Classical Recurrent Neural Networks (RNNs) are widely used in battery prognostics for their ability to handle sequential data.
• CNN-LSTM: It is a hybrid architecture where CNN layers extract local features from voltage curves, followed by LSTM layers for temporal modeling.
• TCN (Temporal Convolutional Network): It uses dilated causal convolutions to model long-term history, serving as a strong non-RNN baseline.
• Transformer (Vanilla): It is the standard Transformer model with sinusoidal positional encoding and LayerNorm, serving as a baseline to validate our architectural improvements.
• FEDformer (Frequency-Enhanced Decomposed Transformer): It is a highly competitive long-horizon time-series forecasting model that utilizes seasonal-trend decomposition and frequency-domain sparse attention. It serves as a robust baseline for evaluating long-term global trend modeling under non-stationary conditions.
• PatchTST (Patch Time-Series Transformer): It is the current state-of-the-art multivariate time-series forecasting model that uses channel independence and patching. It serves as the strongest competitor.

4.4. Implementation Details

BatteryMoE was implemented in Python 3.11, PyTorch 2.1.2 and trained on NVIDIA A100 GPUs. The key hyperparameters were set as follows: hidden dimension: $d_{model}=128$, number of layers: $N_{layers}=4$, and number of experts: $N_{experts}=8$ with Top-$K=2$ active experts. The initial learning rate was set to $1\times {10}^{-3}$ with a cosine annealing scheduler. We utilized the AdamW optimizer and a batch size of 32. The Derf module parameters were initialized with $\alpha =1.0$ and $\mathbf{s}=0$.

4.5. Main Results

4.5.1. SOH Estimation Performance

Table 1. SOH estimation performance on five datasets (RMSE (%) and MAE (%)).

Dataset	H	Metric	LSTM	CNN-LSTM	TCN	FEDformer	PatchTST	BatteryMoE
NASA	24	RMSE	2.15 ± 0.16	1.88 ± 0.14	1.22 ± 0.09	0.91 ± 0.07	0.72 ± 0.05	0.56 ± 0.04
		MAE	1.72 ± 0.13	1.45 ± 0.11	0.95 ± 0.07	0.72 ± 0.05	0.55 ± 0.04	0.43 ± 0.03
	48	RMSE	2.35 ± 0.18	2.10 ± 0.16	1.35 ± 0.11	0.98 ± 0.08	0.78 ± 0.06	0.62 ± 0.04
		MAE	1.88 ± 0.15	1.65 ± 0.13	1.08 ± 0.08	0.78 ± 0.06	0.62 ± 0.05	0.48 ± 0.03
	96	RMSE	2.65 ± 0.21	2.35 ± 0.18	1.55 ± 0.13	1.15 ± 0.10	0.88 ± 0.07	0.68 ± 0.05
		MAE	2.15 ± 0.17	1.85 ± 0.15	1.25 ± 0.11	0.92 ± 0.08	0.70 ± 0.06	0.55 ± 0.04
CALCE	24	RMSE	2.25 ± 0.18	2.02 ± 0.15	1.30 ± 0.10	1.05 ± 0.08	0.80 ± 0.06	0.62 ± 0.04
		MAE	1.82 ± 0.14	1.58 ± 0.12	1.02 ± 0.08	0.82 ± 0.06	0.62 ± 0.04	0.48 ± 0.03
	48	RMSE	2.52 ± 0.20	2.25 ± 0.17	1.48 ± 0.12	1.18 ± 0.09	0.88 ± 0.07	0.68 ± 0.05
		MAE	2.05 ± 0.16	1.78 ± 0.14	1.18 ± 0.09	0.92 ± 0.07	0.68 ± 0.05	0.52 ± 0.04
	96	RMSE	2.85 ± 0.24	2.55 ± 0.20	1.70 ± 0.15	1.35 ± 0.11	1.02 ± 0.09	0.79 ± 0.06
		MAE	2.31 ± 0.19	2.02 ± 0.16	1.38 ± 0.13	1.08 ± 0.09	0.82 ± 0.07	0.60 ± 0.05
MIT	24	RMSE	1.95 ± 0.15	1.72 ± 0.13	1.15 ± 0.09	0.90 ± 0.07	0.68 ± 0.05	0.52 ± 0.04
		MAE	1.58 ± 0.12	1.35 ± 0.10	0.90 ± 0.07	0.70 ± 0.05	0.52 ± 0.04	0.38 ± 0.02
	48	RMSE	2.15 ± 0.17	1.90 ± 0.15	1.25 ± 0.10	0.95 ± 0.08	0.74 ± 0.06	0.57 ± 0.05
		MAE	1.72 ± 0.14	1.50 ± 0.12	0.98 ± 0.08	0.75 ± 0.06	0.56 ± 0.04	0.42 ± 0.03
	96	RMSE	2.45 ± 0.19	2.15 ± 0.17	1.45 ± 0.12	1.10 ± 0.09	0.82 ± 0.07	0.62 ± 0.06
		MAE	2.02 ± 0.16	1.68 ± 0.13	1.15 ± 0.09	0.88 ± 0.07	0.65 ± 0.05	0.49 ± 0.04
XJTU	24	RMSE	2.18 ± 0.16	1.95 ± 0.14	1.25 ± 0.10	0.98 ± 0.08	0.78 ± 0.06	0.58 ± 0.04
		MAE	1.75 ± 0.13	1.52 ± 0.11	0.98 ± 0.09	0.77 ± 0.06	0.60 ± 0.04	0.45 ± 0.04
	48	RMSE	2.45 ± 0.18	2.18 ± 0.16	1.42 ± 0.11	1.12 ± 0.09	0.85 ± 0.07	0.65 ± 0.05
		MAE	2.00 ± 0.15	1.72 ± 0.13	1.12 ± 0.09	0.88 ± 0.07	0.65 ± 0.05	0.50 ± 0.04
	96	RMSE	2.75 ± 0.22	2.45 ± 0.19	1.62 ± 0.14	1.28 ± 0.11	0.98 ± 0.08	0.74 ± 0.07
		MAE	2.26 ± 0.18	1.95 ± 0.15	1.28 ± 0.11	1.02 ± 0.08	0.78 ± 0.06	0.58 ± 0.05
TJU	24	RMSE	2.05 ± 0.15	1.82 ± 0.13	1.18 ± 0.09	0.95 ± 0.07	0.70 ± 0.05	0.55 ± 0.03
		MAE	1.67 ± 0.12	1.42 ± 0.10	0.92 ± 0.07	0.72 ± 0.05	0.54 ± 0.04	0.42 ± 0.02
	48	RMSE	2.25 ± 0.17	2.02 ± 0.15	1.30 ± 0.10	1.05 ± 0.08	0.76 ± 0.06	0.58 ± 0.04
		MAE	1.80 ± 0.14	1.58 ± 0.12	1.02 ± 0.08	0.80 ± 0.06	0.58 ± 0.04	0.45 ± 0.03
	96	RMSE	2.55 ± 0.20	2.28 ± 0.18	1.50 ± 0.12	1.20 ± 0.09	0.85 ± 0.07	0.67 ± 0.05
		MAE	2.08 ± 0.16	1.82 ± 0.14	1.20 ± 0.09	0.95 ± 0.07	0.68 ± 0.05	0.53 ± 0.04

summarizes the SOH estimation results across the five datasets. To ensure a rigorous evaluation, all results are reported as the average of five independent runs with random seeds (mean ± Std). Notably, BatteryMoE outperforms both classical deep learning architectures and advanced Transformer-based models. For instance, on the NASA dataset at $H=24$, BatteryMoE achieves an RMSE of 0.56%, outperforming the strong PatchTST baseline (0.72%) and FEDformer (0.91%). This performance gap is maintained across datasets with inherently different noise profiles, such as the highly fluctuating CALCE dataset, where BatteryMoE continues to deliver the lowest RMSE (0.62% at $H=12$). As the prediction horizon expands to $H=96$, the difficulty of SOH estimation increases due to accumulated errors and temporal distribution shifts. Traditional recurrent models exhibit severe degradation (e.g., LSTM reaches 2.85% RMSE on CALCE). In contrast, BatteryMoE exhibits exceptional long-term extrapolation capabilities and maintains a highly competitive RMSE of 0.79%, indicating that the integration of the Derf mechanism and RoPE++ effectively preserves long-range dependencies and mitigates error accumulation.

4.5.2. RUL Prediction Performance

Table 2. RUL prediction performance (MAE and EOL Error).

Dataset	Metric	LSTM	CNN-LSTM	TCN	FEDformer	PatchTST	BatteryMoE
NASA	MAE	22.8 ± 3.5	20.4 ± 3.1	18.5 ± 2.8	16.2 ± 2.4	13.8 ± 1.8	11.5 ± 0.9
	$Erro{r}_{EOL}$	25.1 ± 4.2	22.6 ± 3.6	20.8 ± 3.2	18.5 ± 2.8	15.2 ± 2.1	12.8 ± 1.1
CALCE	MAE	26.5 ± 4.1	24.1 ± 3.6	22.0 ± 3.2	19.8 ± 2.6	17.5 ± 2.2	14.8 ± 1.2
	$Erro{r}_{EOL}$	30.2 ± 4.8	27.5 ± 4.2	24.8 ± 3.7	22.4 ± 3.1	19.1 ± 2.4	16.2 ± 1.4
MIT	MAE	19.4 ± 3.0	17.5 ± 2.6	16.2 ± 2.4	14.5 ± 2.0	11.9 ± 1.5	9.8 ± 0.8
	$Erro{r}_{EOL}$	21.8 ± 3.5	19.4 ± 3.0	17.8 ± 2.7	16.1 ± 2.2	13.4 ± 1.7	11.0 ± 0.9
XJTU	MAE	24.1 ± 3.8	21.8 ± 3.3	19.6 ± 2.9	17.8 ± 2.5	15.4 ± 1.9	13.1 ± 1.0
	$Erro{r}_{EOL}$	27.6 ± 4.5	24.5 ± 3.9	22.2 ± 3.4	20.2 ± 2.9	17.2 ± 2.2	14.5 ± 1.3
TJU	MAE	21.5 ± 3.2	19.2 ± 2.8	17.4 ± 2.5	15.6 ± 2.2	13.0 ± 1.6	10.6 ± 0.9
	$Erro{r}_{EOL}$	23.6 ± 3.8	21.0 ± 3.2	19.2 ± 2.8	17.4 ± 2.5	14.5 ± 1.8	11.8 ± 1.0

and Figure 3 present the RUL prediction results, which were evaluated using the MAE and EOL Error across five random seeds. Predicting RUL is inherently challenging as it requires the model to accurately forecast the trajectory until the SOH degrades to the 80% failure threshold. BatteryMoE achieves the lowest EOL Error across all five datasets. Compared to competitive models like PatchTST and FEDformer, BatteryMoE demonstrates a lower standard deviation, in other words, a strict confidence interval, indicating that the sparse MoE routing mechanism effectively activates appropriate subnetworks to capture the accelerated capacity plunge near the end of life. The reduction in MAE is particularly notable on datasets with complex operating profiles, such as XJTU and TJU. By seamlessly integrating physics-aware IC features with a normalization-free backbone, BatteryMoE avoids masking sudden, genuine physical capacity drops. This makes RUL prediction highly reliable, demonstrating strong potential for future edge BMS applications, pending rigorous latency and embedded hardware benchmarking.

4.5.3. Capacity Regeneration Phenomena

Furthermore, an analysis of local capacity regeneration phenomena—sudden spikes in SOH after rest periods—revealed that PatchTST, due to its patch-based tokenization, tended to over-smooth these high-frequency details. In contrast, BatteryMoE’s point-wise tokenization combined with the SwiGLU activation allowed it to accurately track these sudden regenerative spikes without lagging, as illustrated in Figure 4.

4.6. Ablation Studies

To validate the effectiveness of the proposed components, we conducted ablation studies on the combined dataset (averaged results).

• w/o Derf (using RMSNorm): Replacing Derf with standard RMSNorm resulted in a 6.5% increase in RMSE. This confirms that the normalization-free design with dynamic saturation is more robust to battery sensor noise.
• w/o RoPE++ (using RoPE): Reverting to standard RoPE caused a degradation in RUL prediction accuracy, Specifically, when predicting at a long horizon ($H=96$), disabling the imaginary attention branch resulted in an 9.4% increase in MAE on the NASA dataset, validating the contribution of the imaginary attention component in capturing global degradation trends.
• w/o Sparse MoE (using Dense FFN): Replacing the MoE layer with a dense FFN increased inference latency by 40% (at an equivalent parameter count) and slightly reduced accuracy on the variable-condition XJTU dataset, proving that experts specialize effectively in different discharge rates.
• w/o Physics-Aware Input: Removing the IC curve features led to a performance drop on the CALCE dataset, demonstrating that physics-aware features are essential for detecting multi-stage aging.

5. Conclusions

In this paper, we proposed BatteryMoE, a novel normalization-free sparse Mixture-of-Experts Transformer designed for high-stakes UPS battery prognostics. By bridging the gap between advanced large language model architectures and electrochemical degradation mechanisms, we addressed the critical challenges of non-stationarity and long-term dependency in battery aging.

Our contributions are threefold. First, we introduced a normalization-free block powered by the dynamic erf mechanism. Unlike rigid normalization layers, Derf provides a soft saturation capability that effectively filters high-frequency sensor noise while preserving the absolute trend of capacity fading, proving crucial for robust SOH estimation under fast-charging conditions. Second, we integrated an enhanced Positional Embedding that re-incorporates the imaginary component of the attention score. Theoretical analysis and empirical results confirm that this imaginary branch captures the global context and long-range dependencies significantly better than the standard RoPE, enabling precise RUL extrapolation beyond the training horizon. Third, the sparse MoE architecture allowed the model to dynamically allocate computational resources, activating specialized experts for distinct aging stages. Extensive experiments on five public datasets demonstrated that BatteryMoE achieves a great performance, outperforming the PatchTST model with an average RMSE reduction of around 20% in SOH estimation. The robust generalizability to unseen cells demonstrate that the proposed architecture not only suppresses high-frequency measurement noise but also accurately captures the authentic non-linear physics of battery degradation, making it a highly reliable solution for real-world UPS diagnostics.

While the sparse architecture theoretically reduces active FLOPs compared to dense counterparts, its practical latency on embedded edge BMS hardware remains a hypothesis to be validated. Future work will focus on two directions: (1) deploying BatteryMoE on resource-constrained edge BMS chips via model quantization and knowledge distillation and (2) extending the framework to a multi-modal foundation model that incorporates EIS data for even earlier fault detection.