Research on the SOH of Lithium Batteries Based on the TCN–Transformer–BiLSTM Hybrid Model

Abstract

Lithium-ion batteries are widely used in energy storage and power systems due to their high energy density, long cycle life, and stability. Accurate prediction of the state of health (SOH) of batteries is critical to ensuring their safe and reliable operation. However, the prediction task remains challenging due to various complex factors. This paper proposes a hybrid TCN–Transformer–BiLSTM prediction model for battery SOH estimation. The model is first validated using the NASA public dataset, followed by further verification with dynamic operating condition simulation experimental data. Health features correlated with SOH are identified through Pearson analysis, and comparisons are conducted with existing LSTM, GRU, and BiLSTM methods. Experimental results demonstrate that the proposed model achieves outstanding performance across multiple datasets, with root mean square error (RMSE) values consistently below 2% and even below 1% in specific cases. Furthermore, the model maintains high prediction accuracy even when trained with only 50% of the data.

1. Introduction

Lithium-ion batteries have outstanding advantages such as high energy density, long cycle life, low self-discharge rate, no memory, and good stability [1,2], and have achieved a high status in the field of power batteries as well as various types of energy storage devices, and are widely used in major industries. As the battery use increases, the maximum capacity of the battery will decay accordingly, and when the current remaining maximum capacity of the battery is reduced to 80% of the initial value, the battery is considered to reach the end of life [3]. Due to the many influencing factors of power battery life degradation, it is a big challenge to give an accurate assessment of the battery’s health status. Battery management system (BMS) can detect the actual usage of the battery, provide timely feedback, and issue early warnings to ensure the long-term safe and stable operation of the battery [4]. The state of health (SOH) of the battery is an important indicator in the BMS to characterize the aging degree of the battery. Therefore, it is of great significance to accurately predict the state of health of batteries.

Currently, scholars at home and abroad have conducted a lot of research on SOH estimation methods for power batteries [5], and the existing estimation methods can be categorized into three types: model-based methods, data-driven methods [6], and fusion-based methods [7]. Model-based human methods have been widely used to predict battery performance [8]. Hosseininasab et al. proposed a SOH estimation method based on a reduced-order electrochemical model for joint estimation of capacity and resistance [9]. Nejad et al. applied a dual extended Kalman filtering algorithm to update the parameters of a second-order equivalent circuit battery model so that an accurate estimation of the state of charge can be obtained in real time [10]. However, their computational complexity limits their applicability in real-time scenarios. Instead of relying on physical and chemical models inside the battery, data-driven methods utilize a large amount of experimental or real-world usage data as well as machine learning techniques to build statistical models. Typical data-driven methods include correlation vector machine (RVM) [11], support vector regression (SVR) [12], support vector machine (SVM) [13], and neural network (NN) [14]. Traditional RNNs can effectively utilize a priori information to solve nonlinear problems, but the performance of RNNs is limited by gradient explosion or gradient vanishing during training. Therefore, researchers have created some variants of RNN to solve these problems, such as Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), GRU, etc. [15]. Traditional neural network models are not well suited to dealing with spatial patterns in data. Therefore, convolutional neural networks (CNNs) have become an effective solution to capture spatial patterns and short-term dependencies in time series data. For example, Chen et al. proposed a CV charging stage-based CNN for SOH estimation of lithium batteries, which extracts features from the early charging stage, avoids complex feature extraction engineering, and achieves SOH prediction by extracting features at the battery charging stage [16]. CNNs are good at recognizing spatial patterns and short-term temporal dependencies. Based on CNN, temporal convolutional networks (TCNs) are introduced to capture long-term dependencies while reducing computational complexity. TCNs lack the ability to model global features dynamically, and to further enhance the temporal modeling capability. Transformers have a unique advantage in capturing global dependencies through a self-attentive mechanism, and thus have been widely adopted [17]. Fusion–model-based approaches can overcome the limitations of a single model and better utilize the available information to improve the accuracy of the prediction [18].

Traditional single models (e.g., LSTM, CNN, etc.) perform well on specific tasks, but their capabilities may be limited by the characteristics of the model architecture. For example, LSTM is strong in capturing long-time dependencies but may underperform when dealing with short-time features [19]. Transformers excel in capturing global relationships but are deficient in dealing with sequence continuity (e.g., temporal dependencies). TCN excels in extracting short-term and localized features of a time series but lacks the ability to strongly capture global and nonlinear relationships of a sequence. Lithium battery SOH prediction involves multi-scale features (short-term changes and long-term trends), which are difficult to be adequately captured by a single model. In response to the above analysis, this paper proposes a prediction model based on TCN–Transformer–BiLSTM [20]. A neural network combining TCN, Transformer, and Bi-LSTM can integrate the advantages of these three models to handle complex time series data. This architecture aims to leverage the local time-dependent modeling of TCN, the global context modeling of Transformer, and the pre- and post-context-dependent processing capabilities of Bi-LSTM to improve the overall performance of the model [21].

2. Introduction to Algorithms

2.1. Temporal Convolutional Network

Temporal convolutional networks (TCNs) extract features from time series using causal convolution, dilated convolution, and residual connections. Causal convolution ensures temporal order, but its receptive field is limited by kernel size, requiring many stacked layers to capture long-term dependencies [22]. To address this, TCNs introduce dilated convolution, which inserts gaps between kernel elements and samples inputs at intervals. For example, at the bottom layer d = 1 every input point is sampled, while at higher layers d = 2, 4, … inputs are taken at wider intervals. This design allows the receptive field to grow exponentially with depth, enabling efficient modeling of both short- and long-range dependencies [23]. The results of dilated causal convolution in TCN are illustrated in Figure 1.

TCN consists of multiple hidden layers stacked on top of each other, each with a pair of hyperparameters (k, d), where k represents the size of the convolution kernel and d is the dilation factor. Each residual block consists of two branches. One branch contains two identical dilated causal convolutional layers, and the other one realizes the width alignment of the input and output by a 1 × 1 convolution and connects it directly to the output.

2.2. Sparrow’s Algorithm to Optimize BP Neural Networks

The Transformer was proposed to overcome the limitations of RNNs and LSTMs in handling long sequences, such as poor parallelization, difficulty in capturing long-range dependencies, and gradient vanishing or explosion. It addresses these issues through the self-attention mechanism, which allows the model to assign different weights to positions in the input sequence when generating outputs [24].

The Transformer architecture consists of an encoder and a decoder. The encoder transforms input data into an internal representation, while the decoder generates outputs based on this representation. Both components employ self-attention and feed-forward neural networks, as illustrated in Figure 2.

Since the Transformer is based on a parallel computation approach under the mechanism of multi-head self-attention, it does not have the ability to handle sequence ordering as classical recurrent neural networks such as RNN and LSTM inherently bring with them, and therefore introduces positional encoding to preserve positional information [25]. A common form of positional encoding is to use a combination of sine and cosine functions that can represent sequences of arbitrary length:

$$P{E}_{\left(pos,2i\right)}=\sin \left({\displaystyle \frac{pos}{{\mathrm{10,000}}^{2i/d}}}\right)$$

(1)

$$P{E}_{\left(pos,2i+1\right)}=\cos \left({\displaystyle \frac{pos}{{\mathrm{10,000}}^{2i/d}}}\right)$$

(2)

where pos is the position, i is the dimension index, and d is the model dimension. The position encoding will be added to the word vectors so that the representation of each word contains the position information.

The attention mechanism can be viewed as a means of resource allocation in the model learning process, and in deep neural networks, the weight parameter of each feature is a resource that the attention mechanism focuses on. The attention mechanism focuses the model on important information by adaptively assigning weights to input variables [26]. The structure of multi-head attention is shown in Figure 3.

The computation process of scaled dot-product attention is as follows: compute the product of query matrix, key matrix, and obtain the score matrix scores to represent the correlation between each query and all the keys; scale the score matrix to ensure the stability of the values; mask is equivalent to shielding the useless information; normalize the scaled scores by softmax function to normalize the scaled scores to obtain the attention weight distribution; use the obtained attention weights to weigh and sum the value vectors to obtain the weighted matrix. The formula is

$$Attention\left(Q,K,V\right)=softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(3)

where $d_k$ is the number of columns of the Q, K matrix, the vector dimension, and $\sqrt{d_k}$ is the scale factor contributing to the stabilization gradient.

The drawback of the self-attention mechanism is that the model will overly focus its attention on its own position when encoding the information of the current position, and the effective information grasping ability is a little worse. Therefore, it is proposed to solve this problem by a multi-head attention mechanism. Multi-head is performed by generating multiple sets of Q, K, V using attention to obtain multiple results, and then weighted averaging of the multiple results to obtain the final result [27]. The formula for multi-head self-attention is

$$MultiHead\left(Q,K,V\right)=Concat\left(hea{d}_1,\dots hea{d}_h\right)W^O$$

(4)

where Q, K, V denote the query vector, key vector, and value vector, respectively; h denotes the number of heads; $hea{d}_i$ denotes the output of the ith head; and $W^O$ is the output transformation matrix. The output of each head $hea{d}_i$ can be expressed as

$$where hea{d}_i=Attention\left(Q{W}_i^Q,K{W}_i^K,V{W}_i^V\right)$$

(5)

where $W_i^Q,W_i^K,W_i^V$ are the query, key, and value transformation matrices of the ith header, respectively. Attention is the attention calculation function.

2.3. Bidirectional Long and Short-Term Memory Neural Network

Long Short-Term Memory is a variant of recurrent neural networks designed to solve the gradient vanishing and gradient explosion problems of traditional RNNs when processing long sequences. Unlike the simple structure of standard RNNs, LSTMs have a more complex design containing a cellular shape and three main gating mechanisms (input gate, forgetting gate, and output gate):

Cell state: This represents the long-term memory of the network and runs through the entire chain of LSTM cells with only a small number of linear interactions. It can be thought of as a conveyor belt to carry important information [28].

Forget gate: Decides what information should be discarded from the cell state.

Input gate: Decides what new information should be added to the cell state.

Output gate: Decides the output of the next hidden state based on the cell state.

The workflow of LSTM is shown below:

(1)

Calculation of the forgetting gate value $f_t$:

$$f_t=\sigma \left(W_f\cdot \left|h_{t-1},x_t\right|+b_f\right)$$

(6)

where σ is the sigmoid activation function, $W_f$ is the weight matrix, ${\mathrm{b}}_{\mathrm{f}}$ is the bias term; ${\mathrm{h}}_{\mathrm{t}-1}$ the hidden state of the previous time step; ${\mathrm{x}}_{\mathrm{t}}$ is the input of the current time step.

(2)

Calculate the input gate value i_t and the candidate value ${\tilde{C}}_t$ :

$$i_t=\sigma \left(W_i\cdot \left|h_{t-1},x_t\right|+b_i\right)$$

(7)

$${\tilde{C}}_t=tan h\left(W_C\cdot \left|h_{t-1},x_t\right|+b_C\right)$$

(8)

where $W_i$ and $W_C$ are weight matrices; $b_i$ and $b_C$ are bias terms; and $tan h$ is the hyperbolic tangent activation function.

(3)

Renewal of cellular state $C_t$:

$$C_t=f_t\cdot C_{t-1}+i_t\cdot {\tilde{C}}_t$$

(9)

(4)

Compute the output gate value $o_t$ and the new hidden state $h_t$:

$$o_t=\sigma \left(W_o\cdot \left|h_{t-1},x_t\right|+b_o\right)$$

(10)

$$h_t=o_t\cdot tan h\left(C_t\right)$$

(11)

Its structure is shown in Figure 4.

LSTM processes sequence data in a unidirectional manner, i.e., it processes the input data sequentially from the beginning to the end of the sequence. This approach only captures forward contextual information. Bidirectional LSTM processes sequence data in a bidirectional manner; the single-layer Bi-LSTM is a combination of two independent LSTM layers, one going forward to process the input sequence, and the other processing the sequence in reverse, and the final output is a combination of the outputs of the forward and reverse LSTM layers, so the Bidirectional LSTM is able to capture both the forward and backward contextual information of the sequence [29]. Its working principle is shown in Figure 5.

2.4. Hybrid Modeling

A neural network combining TCN, Transformer, and Bi-LSTM can integrate the strengths of all three models to process complex temporal data. This architecture aims to leverage the local time-dependent modeling of TCN, the global context modeling of Transformer, and the pre- and post-context-dependent processing capabilities of Bi-LSTM to enhance the overall performance of the model. The model structure is shown in Figure 6, which is mainly composed of input layer, TCN layer, Transformer layer, BiLSTM layer, and output layer. The model structure is described as follows:

(1)

Input layer: Extract and filter features related to battery capacity, select highly correlated factors as indirect inputs, and preprocess the data. The training set is then fed into the model.

(2)

TCN layer: Processes the input sequence to extract local temporal features. By using dilated convolution, TCN captures short- and medium-term dependencies and provides refined temporal representations for the following layers.

(3)

Transformer layer: Takes TCN outputs as input and employs multi-head self-attention and feed-forward networks to capture global contextual dependencies, generating feature representations with long-range temporal information.

(4)

BiLSTM layer: Processes Transformer features in both forward and backward directions, capturing bidirectional dependencies in the sequence.

(5)

Output layer: The BiLSTM outputs are passed to a fully connected layer, which produces the final SOH prediction using the test data.

The model is based on MATLAB2023b implementation of the model and the model parameters are shown in

Table 1. Model parameters.

Parameters	Value
Optimizer	Adam
Loss function	RMSE
Learning rate	0.01
Maximum number of training rounds	100
Minimum number of iterations	10

.

TCN has two layers; the number of temporal convolutional layers in the first layer is 16, the size of the convolutional kernel is 3, and the expansion rate is 1. The number of convolutional kernels in the second layer is 32, the size of the convolutional kernel is 3, and the expansion rate is 2. There are two encoders for the Transformer, and the number of heads of each of the multi-head self-attention mechanisms is 4. The number of layers for the Bi-LSTM is 2, and the number of hidden units in the first layer is 16. The number of hidden units in the second layer is 32.

3. Dataset Validation

3.1. NASA Dataset Validation

The data is aligned by timestamp and denoised using median/SG filtering; the normalizer is fitted with the training set only for validation/testing; samples are constructed with a fixed sliding window. The feature pool contains cycle number, constant current capacity, constant voltage duration, temperature rise, pressure drop, dV/dt, dT/dt, etc. In this paper, we filtered under three dimensions: correlation (Pearson + Spearman), redundancy, and physical interpretability, and finally adopted HI1, HI3, and HI4 in order to obtain robust combinations that can be repeatedly collected in the field.

This study uses the battery dataset provided by NASA. The batteries used in the NASA dataset are 18,650 cylindrical Li-ion batteries with a capacity of 2 Ah and termination conditions of 70% of the rated capacity, i.e., 1.4 Ah. The batteries were divided into nine groups and were subjected to charging and discharging cyclic aging tests under random conditions as well as stationary conditions at 4 °C, 24 °C, and 43 °C. The batteries were tested at 4 °C, 24 °C, and 43 °C. The batteries were then subjected to the same conditions as those in the NASA dataset. In this paper, batteries 5, 6, and 7 were selected for prediction study, and the test conditions for this group of data were as follows:

Temperature: Room temperature 24 degrees.

Charging: Charge in constant current (CC) mode at 1.5 A until the battery voltage reaches 4.2 V, then charge in constant voltage (CV) mode until the charging current drops to 20 mA.

Discharge: Discharge in constant current (CC) mode at 2 A until the voltage of batteries 5, 6, and 7 drops to 2.7 V, 2.5 V, and 2.2 V.

Termination condition: Rated capacity drops to 1.4 Ah.

The cycle times of battery No. 5, 6, and 7 are all 168 times. Among them, batteries No. 5 and No. 6 reach below the failure threshold. The selected health features are six features: the number of cycles, the time of discharge to cut-off voltage, the time of iso-current drop, the time of iso-voltage rise, and the average temperature from the initial temperature to the highest point. For the convenience of description, these six features were replaced by HI1~HI6, respectively, and they were analyzed with SOH by Pearson correlation. The results of the analysis are shown in

Table 2. Features.

Health Characteristics	#5	#6	#7
HI1	−0.987	−0.981	−0.988
HI2	0.896	0.185	−0.041
HI3	0.985	0.987	0.986
HI4	0.993	0.992	0.994
HI5	0.932	0.950	0.874
HI6	−0.018	−0.149	0.293

.

The three features with the highest correlation are selected as inputs to the model, i.e., number of cycles, iso-current drop time, and iso-pressure rise time, and SOH is used as the output of the model. The data from the battery was divided into training and test sets, using 60% of the data as the training set and 40% of the data as the test set.

In order to verify the accuracy of the algorithm, the prediction is obtained by comparing the mainstream algorithms of LSTM, GRU, and Bi-LSTM, and the LSTM, GRU, and Bi-LSTM, and the proposed models named M1, M2, M3, and M4, respectively. The predicted results are shown in

Table 3. Projected results.

Battery Number	Methodologies	RMSE	R2	MAE
#5	M1	0.0516	0.2071	0.0477
	M2	0.0326	0.6840	0.0305
	M3	0.0382	0.5649	0.0355
	M4	0.0108	0.9650	0.0059
#6	M1	0.0503	0.6032	0.0444
	M2	0.0274	0.8821	0.0250
	M3	0.0444	0.6902	0.0390
	M4	0.0155	0.9623	0.0126
#7	M1	0.0363	0.4431	0.0330
	M2	0.0198	0.8346	0.0184
	M3	0.0266	0.7007	0.0236
	M4	0.0085	0.9692	0.0052

as well as in Figure 7.

As can be seen from the figure, the method proposed in this paper performs SOH prediction for different batteries, and the prediction results are compared with the test set and compared with the LSTM, GRU, and Bi-LSTM algorithms, all of which embody the most accurate prediction results. The root mean square error of the prediction results for three batteries is below 2%, the No. 7 batteries are controlled to be within 1%, the absolute coefficients are all greater than 0.96, the average absolute errors are all controlled to be within 1.5% or less, and battery No. 5 and No. 7 can be controlled to about 0.5%. By comparison, the method proposed in this paper has high accuracy and can accurately predict SOH for real vehicles.

3.2. Model Migration Capability Validation

In order to further verify the effectiveness of the algorithm, data are obtained through experiments for verification. The object of this experiment is a brand of 105 Ah lithium iron phosphate square shell battery; the experimental platform is mainly composed of test batteries, charge and discharge test equipment, constant temperature and humidity test chamber, connecting wires, temperature sensors, bolts, and so on. The experimental platform is shown in Figure 8.

The experimental conditions are extracted from the operating conditions of a real vehicle, which include driving with discharge, stopping and standing, braking feedback, and stopping and charging. The experimental conditions are in line with the daily vehicle driving status and driving behavior. The capacity change in one cycle of this condition is 10.5 Ah, and in the experiment, the operating condition is set to cycle eight times for one cycle, i.e., 84 Ah. After discharge, the standard charging method in the battery specification is adopted, i.e., constant current and constant voltage (CC-CV) charging method, whereby the battery is charged with a constant current of 0.5 C up to 3.65 V, and then charged with a constant voltage until the current is reduced to 0.05 C. At the end of each cycle, the battery is left to rest for a period of time in order to recover the internal capacity of the battery and to recover its internal resistance and reduce the temperature to avoid the cumulative thermal effect on the experimental results. Subsequently, the experiment continued with the next cycle.

The experimental capacity results are shown in Figure 9.

As can be seen from the figure, the battery capacity shows a capacity rebound, which is caused by the battery resting after cycling for a period of time. When the battery is resting for a period of time, the SEI membrane may self-repair or re-stabilize, resulting in a lower internal impedance and a recovery of the battery’s performance, which in turn leads to a capacity rebound.

Health features are similarly selected as inputs to the model before SOH prediction. Specifically selected features include the number of cycles, charging start voltage, charging maximum voltage, charging start current, charging median voltage, constant current charging capacity, constant voltage charging capacity, charging start average temperature, charging cut-off average temperature, charging process average maximum temperature, and charging time. These 11 features are named as HI1 to HI11, and the correlation analysis between the selected features and SOH is also carried out using the Pearson correlation coefficient method. The results of the analysis are shown in Figure 10.

Feature screening follows the three-dimensional criterion of ‘linear correlation + monotonicity + physical interpretability’: Pearson for linear initial screening, Spearman for monotonicity checking, and ultimately the combination of HI1/HI3/HI4 based on the electrochemical mechanism, with its limited capability of nonlinear inscription. Methods such as mutual information/HSIC will be used as an extension.

The calculation shows that the number of cycles, constant current charging capacity, and charging time have a higher correlation with SOH, so these three health features are selected as inputs to the model and SOH as outputs of the model for SOH prediction study. In order to verify whether the algorithm is able to make accurate predictions despite the small amount of data, we divided the dataset of the two batteries to different degrees and used 50%, 60%, and 70% of the training set for model training, respectively. The predicted results are shown in Figure 11.

The indicators for evaluating the predicted results are shown in

Table 4. Evaluation indicators.

Projected Starting Point	MAE	R2	RMSE
50%	0.00101	0.93529	0.00118
60%	0.00088	0.93479	0.00095
70%	0.00083	0.88689	0.00090

.

Observing the prediction results, it can be seen that with the increase in the dataset used for model training, the prediction results are gradually closer to the real values, and the MAE and RMSE are also gradually reduced. And when only 50% of the dataset is used for model training, the RMSE of the test set can be controlled at 0.00118, which indicates that the proposed model has strong prediction ability.

In this paper, RMSE is calculated as normalized SOH (0−1); its conversion to actual capacity units is RMSE_Ah = RMSE_norm × (SOH_max − SOH_min) × C_rated. The data division follows a chronological order: k cycles of isolation are set between the training/validation/testing segments; the normalizer is only fitted by the training set and used for validation/testing; all features and labels come from the same cycle and are not spliced across cycles to avoid information leakage.

3.3. Applicable Boundaries

The above three characterization schemes are valid in this data domain and do not guarantee the optimality for extreme temperature zones/anomalous working conditions; more complex scenarios can consider nonlinear indicators such as mutual information/HSIC and richer working condition characteristics.

4. Conclusions

This paper proposes a TCN–Transformer–BiLSTM based method for SOH estimation of lithium-ion batteries. Aiming at the limitations of existing single models in capturing complex temporal features, this paper proposes a hybrid model integrating Time Convolutional Network, Transformer, and Bidirectional Long and Short-Term Memory Network with the aim of improving the prediction accuracy through multi-scale feature extraction. The model architecture captures local temporal dependencies through TCN, models global contextual information using Transformer’s self-attention mechanism, and combines BiLSTM’s bidirectional sequence processing capability to achieve a comprehensive characterization of short-term fluctuations and long-term trends during battery degradation. Firstly, based on the NASA public dataset, the number of cycles, iso-current drop time, and isobaric rise time are selected as the key feature inputs, and the mainstream methods such as LSTM, GRU, and BiLSTM are compared. The results show that the root mean square error of the proposed models is lower than 2%, and the coefficient of determination (R²) is higher than 0.96, which performs better under the established baseline and the same evaluation protocols. Secondly, the migration capability and data adaptability of the model are further verified for the dynamic working condition data of a commercial lithium iron phosphate battery. The experiments show that even when only 50% of the training data is used, the model still maintains a prediction accuracy of RMSE below 0.12%, demonstrating strong potential for engineering. Under this data partitioning and model setup, the performance improves with increasing sample proportion, and the improvement tends to slow down at higher percentages; this observation is affected by both the data domain and the model capacity, and the conclusions are not generalized.