State-Partial Accurate Voltage Fault Prognosis for Lithium-Ion Batteries Based on Self-Attention Networks

Abstract

Multiple faults in new energy vehicle batteries can be diagnosed using voltage. To find voltage fault information in advance and reduce battery safety risk, a state-partitioned voltage fault prognosis method based on the self-attention network is proposed. The voltage data are divided into three parts with typical characteristics according to the charging voltage curve trends under different charge states. Subsequently, a voltage prediction model based on the self-attention network is trained separately with each part of the data. The voltage fault prognosis is realized using the threshold method. The effectiveness of the method is verified using real operating data of electric vehicles (EVs). The effects of different batch sizes and window sizes on model training are analyzed, and the optimized hyperparameters are used to train the voltage prediction model. The average error of predicted voltage is less than 2 mV. Finally, the superiority and robustness of the method are verified.

1. Introduction

Climate warming and the energy crisis are significant issues for the international community [1]. Instead of using fuel vehicles, the use of EVs can help reduce carbon emissions and fuel dependence, alleviating those ecological and energy issues [2,3]. Power batteries are the core components of EVs, most of which consist of lithium-ion batteries with high energy density [4]. However, the safety of lithium-ion batteries is the main problem limiting the promotion of EVs [5,6]. To improve the safety of power batteries, most EVs use a battery management system to diagnose battery faults. Many studies have revealed that voltage anomalies are an important cause of battery failure, for example, over-voltage, under-voltage, over-voltage change rate, and poor consistency of cell voltages [7,8]. Therefore, voltage is an essential aspect of battery fault diagnosis. Judging abnormal voltage and interrupting battery usage in a timely fashion is an important method to improve the safety of EVs.

1.1. Literature Review

The internal reactions of power batteries are extremely complex and variable, and are sensitive to temperature changes and battery aging [9,10]. As a result, the battery voltage may exhibit some abnormalities, such as overvoltage [11], under-voltage [12], or poor voltage consistency [13]. Thus, if the voltage of the battery is abnormal, there is a high probability that the battery has some internal failures or will have some failures [14,15]. Therefore, an abnormal battery voltage diagnosis is necessary to ensure the safety of power batteries. Researchers have explored voltage abnormality diagnosis methods [16,17]. Gan [18] et al. presented a machine learning-based two-layer over-discharge fault diagnosis strategy for Li-ion batteries in electric vehicles. Sun [19] et al. proposed a scheme for a three-layer fault detection method for lithium-ion batteries based on statistical analysis. The abnormal detection of battery voltage is realized. However, because these methods are based on a specific experimental environment, the universality of these methods is poor. On this basis, this paper will use the threshold method to realize the diagnosis of abnormal battery voltage.

Threshold method fault prognosis requires voltage prediction. Hsu [20] et al. developed novel-architecture deep neural networks with a special convolutional training strategy. The prediction of battery characteristics and voltage was realized. Han [21] et al. presented a state of energy estimation for lithium-ion battery packs in electric vehicle applications via prediction. The Markov chain model was used to accurately predict the future voltage in the battery pack. In addition, some scholars have used a random forest regression algorithm to predict the variation characteristics of battery voltage based on big data analysis. However, due to the limitations of the data used and the experimental environment, these methods have poor generality.

To solve the problem of poor generality, scholars have also proposed a lot of model-based voltage prediction methods. Wang [22] et al. presented a novel feedback correction-adaptive Kalman filtering method for the whole-life-cycle state of charge [23] (SOC) and closed-circuit voltage prediction of lithium-ion batteries based on the second-order electrical equivalent circuit model. Wu [24] et al. proposed a low-complexity SOC and anode potential prediction for lithium-ion batteries using a simplified electrochemical model-based observer under variable load conditions. The anode potential and SOC of the cell can be predicted with high accuracy. However, these model-based voltage prediction methods have limitations due to the experimental environment and the conditions of applicability of the models [25]. This is difficult to apply to the complex operating conditions of actual operating environments.

The generality of model-based voltage prediction methods is hardly able to satisfy the needs of the complex operating environment of batteries. Data-driven methods overcome this problem [26]. Hong [7] et al. used a hyperparametric optimized long short-term memory [27] (LSTM) model to predict voltage for fault warning. The feasibility and robustness of the model were also verified on actual vehicle data. Tian [28] et al. estimated the entire charging voltage curve by using a segment of the charging curve as input through a deep neural network. Yang [29] used convolutional neural networks to develop voltage prediction models for three kinds of modes of four-point consecutive voltage and time index. In addition, compared with experimental validation. Tobar [30] et al. used the kernel adaptive filter to develop a voltage prediction model that could analyze the relationship between voltage and altitude variations. However, the predictive effectiveness of data-driven methods is heavily influenced by the training data [31,32]. Data-driven methods place high demands on data quality [33,34].

There are three ways to address data quality issues. The first is to collect a large amount of data, which can make the prediction results more accurate and reliable. However, a major drawback of this approach is the high cost. On the one hand, collecting a large amount of data requires a significant investment. On the other hand, the large amount of data makes the cost of data storage and transmission much higher. Secondly, data augmentation is used to expand the sample [35,36]. Data augmentation can increase the amount of data for training and improve the generalization ability of the model. This can also increase the noise data and improve the robustness of the model. Compared with collecting a huge amount of data, data augmentation has a low cost. However, performing data augmentation reasonably without making the dataset defective is a major difficulty. Most data augmentation methods nowadays have low effectiveness in practical tasks and can only provide minimal and marginal gains. The third is to divide the data into parts according to the key states and train the voltage prognosis model, respectively. Partial training can reduce the range of features the model has to learn, thus avoiding the increased difficulty of model learning resulting from the differences in data in different states. Moreover, this facilitates a deeper exploration of features by the model. This method is beneficial for improving the accuracy of voltage fault prognosis.

1.2. Contributions of the Work

This paper makes notable contributions and improvements to current techniques, as follows:

• SOC-based data state-partitioning method: Depending on SOC, the data are divided into three parts with different trends, which is convenient for the model to learn features and improve the accuracy of voltage prediction;
• Voltage prediction model based on self-attention network: The self-attention network is used to predict voltage and improve the capacity for long-range data analysis;
• Voltage prognosis method applied to real vehicles: The voltage prognosis model was constructed using real vehicle data to verify the application ability of the model under complex operating conditions.

1.3. Organization of the Paper

The rest of this paper is organized as follows: Section 2 introduces the methodology used in this article. Section 3 describes the model set and hyperparameter optimization process. Section 4 presents the verification of the feasibility and superiority of the proposed method on the basis of experiments. Section 5 presents the conclusion.

2. Methodology

2.1. Overview

The early prognosis method for voltage fault presented in this paper is shown in Figure 1. First, depending on the SOC, the real voltage data are divided into three parts according to their trend. After that, a voltage prediction model based on the self-attention network is trained separately using the three parts of the data. Finally, voltage fault prognosis is realized based on the voltage predicted by the model and the threshold method.

2.2. Data Description

The data used in this study were obtained from a new government-level energy vehicle big data platform. The data were collected from an electric cab with a high-energy 18,650 lithium-ion battery pack. Several data items were collected, as shown in

Table 1. The data items collected from the EV.

Sort	Data Item	Sort	Data Item
Charging related	(1) Voltage of cells 1 to 100 (2) Temperature of probes 1 to16 (3) Pack voltage (4) Current (5) SOC	Environment related	(1) Humidity (2) Precipitation (3) Barometric pressure (4) Air temperature (5) Visibility
Constant when charging	(1) Brake pedal stroke value (2) Motor speed (3) Vehicle speed (4) Mileage

.

2.3. State-Partial by SOC

The proposed state-partial voltage fault prognosis model only applies to the charging phase of a lithium-ion battery, for the following reasons. First, safety accidents caused by lithium-ion batteries are more likely to occur during the charging phase [15]. Therefore, voltage fault prognosis in the charging phase can better avoid security risks. Second, the voltage change in the discharge phase is irregular and is more affected by the driver’s driving behavior. The drastic changes in voltage the difficulty of fault feature extraction using the prognosis model. When charging, the voltage change is mainly affected by the charging strategy, which is more regular. Predicting the voltage in the charging phase is more suitable for voltage fault prognosis using the proposed model.

The battery charging process is often split into multiple stages due to the battery charging strategy. For example, the trickle charging phase [37], the constant current phase [38], and the constant voltage phase [39]. Figure 2 shows the ten charging voltage curves in the dataset. The charging voltage curves in Figure 2b show this interphase difference. The charging voltage curves have a significant increase in voltage during the first 5% SOC. The increase then gradually diminishes, and the voltage slowly increases after the first 5% SOC. After 90% SOC, the voltage increases dramatically once again. This charging voltage change process corresponds to the constant current phase. The voltage or series resistance is controlled in the constant current phase to ensure a stable current. This control causes a variety of voltage variation trends.

Figure 2b shows that there are various trends in the charging voltage curves. After the SOC reaches 90%, the charging voltage curves largely converge. The trends are similar for the first 5% of each charging voltage curve. Resetting the SOC to start at 0 makes it easier to visualize and analyze the charging voltage trends for the first 5% of the SOC. The charging voltage curve after resetting the SOC is shown in Figure 2a. It can be seen that the charging voltage trends for the first 5% are roughly similar. In addition, the charging voltage in the middle rises smoothly. By observation, the charging voltage curve can be divided into three parts according to SOC: the first 5% SOC, the middle SOC, and the SOC after 90%.

2.4. Self-Attention Mechanism Network

Recurrent neural networks have achieved good results in regression problems such as temporal data prediction. However, due to the structure of the recurrent neural network, as shown in Figure 3a, the information of the first input segment will be covered by the subsequent input segment during the recurrent process. This causes insufficient long-range data processing capability and limits the model prediction accuracy. Subsequently, methods for improving recurrent neural networks have been proposed (e.g., LSTM [40]), and there have been improvements to their long-range data processing capability. However, the problems caused by the recurrent structure still exist. To better process temporal data, a neural network based on the self-attention mechanism has been proposed that uses different structures [41].

In 2017, on the basis of the seq2seq model, the Google Machine Translation team replaced the RNN layer and CNN layer with the self-attention layer and proposed a sequence converter network (i.e., Transformer [41]) model based on the attention mechanism. In the WMT2014 English-German translation task, the Transformer model increases the BLEU value to 28.4, 2 points higher than the optimal model at that time. Kitaev et al. [42] proposed the Reformer model, using the locality-sensitive hashing attention and reversible residual network to decrease the computational load of the model and accelerate the training.

The structure of the self-attention network is shown in Figure 3. First, input segments (x in Figure 3) are mapped into three segments (q, k, and v) by the embedding operation of the linear layer. The segments of q, k, and v are composed into three matrices Q, K, and V, respectively. Then, each q and k vector is multiplied to obtain the attention value of two data segments. During execution, Q and K matrices are dot multiplied. Relative to the fragmented segments, matrix multiplication is computationally fast on a computer. Then, the attention value is input into the softmax function. To avoid the input value being too great, which may cause the partial derivative to approach zero during backpropagation, first, $\sqrt{d_k}$ is divided by the attention value, where d_k represents the dimensionality of segments of k. Then, the softmax output matrix is dot multiplied with V to obtain Y. This is similar to the weighting of the vectors v to obtain the output y. Finally, the output Y is obtained by the self-attention mechanism using the equation:

$$\mathrm{Attention}\left(Q,K,V\right)=\mathrm{Softmax}\left(\frac{Q{K}^{\mathrm{T}}}{\sqrt{d_k}}\right)V$$

(1)

where T represents the transpose operation. The softmax operation is performed according to the following equation:

$$\mathrm{Softmax}\left(z_i\right)=\frac{e^{z_i-M}}{{\sum }_j{e}^{z_i-M}}$$

(2)

M = max(z)

(3)

where z_i represents the i-th data point within a data segment.

In computing the segment of y₁, the corresponding q₁ is subjected to a dot product operation with all k segments, respectively. The softmax operation is performed, and the result is used as weights to sum v segments to obtain the y₁ segment. The above process is repeated to obtain all y segments. Subsequently, all y segments are combined into the final output Y by a linear layer. This processing ignores the position relationship between data segments, thus avoiding the long-range memory problem caused by the recurrent structure. However, positional information is an essential feature of temporal data. A position encoding operation is performed before the data segments are input for the location information can be captured by the self-attention network. Position encoding is carried out using the following equation:

x_n = x_n^origin + p_n

(4)

$$p_n=\{\begin{array}{c}\sin \left(\frac{n}{{10000}^{\frac{2i}{d}}}\right),\mathrm{if} i=2t\\ \cos \left(\frac{n}{{10000}^{\frac{2i}{d}}}\right),\mathrm{if} i=2t+1\end{array}$$

(5)

where x_n^origin represents the n-th original data segment. p_n represents the n-th position encoding. d represents the dimension of the data segment. The input data segment with the added position encoding is obtained by summing p_n and x_n^origin.

3. Preparation for Validation Experiment

3.1. Input Data Items Selection

If all data items in Table 1 are used as the input data of the neural network, not only will the computation load significantly increase, but the learning efficiency of the neural network will also be affected. Therefore, selecting a few items from all the data items that can better characterize the charging voltage change is key to solving the problem.

First, no data items that are constant when charging will be selected, because they do not provide valid information related to charging voltage. Then, among charging-related data items, for the temperature of probes 1 to 16, it is challenging to determine the correspondence with the cell, and applying all probe temperature data would increase the computation load. Therefore, the temperature of probes 1 to 16 was not selected. Finally, among the environment-related data items, the effect of precipitation on the battery system is less direct than that of humidity, and the effect of visibility is so low as to be negligible. Therefore, precipitation and humidity were not selected. The input data items were selected by calculating the correlation coefficients of the remaining data items. The correlation coefficients of the data items are shown in Figure 4.

In the row of the voltage of cell 1, the data items for pack voltage, SOC, and humidity achieve a high correlation with the voltage of cell 1. Therefore, these three data items are selected as input data items. Additionally, in this case, the current curve of the constant current phase (as analyzed in Section 2.3) is jagged and horizontal, resulting in the current having low correlation coefficients with other curves that have an increasing or decreasing tendency. However, on the basis of electrical principles, battery current has a direct relation to voltage. Therefore, the current is still selected as an input data item. Finally, including the voltage of cell 1 itself, pack voltage, current, SOC, and humidity are selected as five input data items.

3.2. The Setting of Neural Network Training

Before the validation experiment, the structural hyperparameters of the model and the training set should be confirmed first. Moreover, when setting the structural hyperparameters of the model it is necessary to consider the input and output data sizes to ensure proper data transmission. At the same time, other structural hyperparameters should be set in conjunction with the volume of data to ensure that the model can be well fitted while avoiding overfitting due to excessive model settings. The structural hyperparameters of the model are shown in

Table 2. The structural and training hyperparameters of the model.

Hyperparameters	Values	Hyperparameters	Setting and Values
Position-wise hidden layer dimension	1024	Optimizer	Stochastic gradient descent
Output hidden layer dimension	128	Learn rate	0.1
Sliding window size	60	Momentum	0.9
Predicted window size	30	Batch size	256
Sliding size	15	Epoch number	50

.

The training settings can greatly affect the model training effect, in turn affecting the prediction effect of the model. A loss function is a necessary tool for the back-propagation of the model in order to adjust the parameters. This study chose mean square error (MSE) as the loss function. This is because MSE increases the feedback adjustment when the error is large and reduces the adjustment for fine-tuning when the error is small. The equation of MSE is as follows:

$$MSE=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2$$

(6)

where y is the true value, ŷ is the predicted value, and n is the sample number. The other hyperparameters for model training are shown in Table 2. The hyperparameter optimization and validation experiments of the model are based on the above settings.

3.3. Batch Size Optimization

To efficiently train the neural network model, the samples are integrated into a batch to train the model, and back-propagation is performed to adjust the parameters. The batch size is the number of samples in one batch. If the batch size is too large, the number of times back-propagation is performed in one epoch will be low, which could lead to insufficient model training. If the batch size is too small, back-propagation will be performed too many times in an epoch, thus increasing the training time. This can easily lead to serious overfitting of the model in one epoch. That is, batch size affects model training in many respects. Therefore, optimizing the batch size is crucial. The effect on training when increasing the batch size exponentially from 8 to 512 was tested, and the MSEs and time costs were recorded, as shown in Figure 5.

In Figure 5, the MSEs can be observed to reach a local optimum at 128 when batch size is changed from 32 to 512. However, the optimum in all tests was achieved with batch sizes of 8 and 16. In terms of time costs, the overall trend indicates that the larger the batch size, the smaller the time costs. Considering the MSEs and time costs, the batch size is set to 16.

3.4. Window Size Optimization

A variety of sample division methods have been proposed, and sliding windows are commonly used to perform temporal sample division. The use of a sliding window to divide the samples is shown in Figure 6. The key to the sliding window is the selection of the window size and the sliding size. The sliding size should not be set too high, so as to ensure the continuity of the predicted voltage data. The voltage data of each moment should be obtained by averaging at least two predicted values. Additionally, the sliding size should not be set too low, so as to avoid the averaging of too many predicted values, which would make the prediction curve too smooth and cause detailed information to be lost. Therefore, the sliding size is set to half of the predicated window size (PWS).

After determining the sliding size, the sliding window size (SWS) and PWS are determined by grid search. The selection of the SWS and PWS should be constrained. If the SWS is too small, it will not be able to provide enough information for the neural network model. In addition, if the SWS is too large, the model computation load will be increased, making it more difficult to extract processing information. If the PWS is too small, it will not be able to provide enough warning time. If the PWS is too large, the predictive ability of the neural network model will be reduced. Considering the prediction time and sampling frequency, the candidate window size is set as {6, 30, 60, 90, 120}. These values correspond to one minute, five minutes, ten minutes, fifteen minutes, and twenty minutes, respectively. Additionally, considering that sufficient input information for the neural network is necessary, the SWS should be larger than the PWS. The MREs with different window sizes are shown in Figure 7.

In Figure 7, the MSEs are smaller when the PWS is set to 6. However, the trend of SWS is not obvious. The lowest MSE is obtained when the SWS is 30, and the MSE increases when SWS increases to 60 and decreases when it reaches 90. This may be because the increase in SWS increases the difficulty of model learning, and the increase in input information can improve the prediction accuracy to some extent. Under the influence of these two factors, the trend of MREs with different SWS is not obvious. The PWS is set to 6 and the SWS is set to 30.

4. Results and Discussion

4.1. Prediction Results and Discussion

All data samples are divided into a training set, a validation set, and a test set, in proportions of 8:1:1, respectively. The model is trained according to the above settings and parameter optimization results, and the training set loss curve and validation set loss curve during the training process are shown in Figure 8.

It can be observed that the model converges rapidly in the first five epochs. After that, the MSE gradually decreases within a small range. Even in the latter epochs, the error difference between the training and validation sets is not significant, indicating that the model does not suffer from significant overfitting problems. The model training effect is good. The voltage prediction results before and after hyperparameter optimization are shown in Figure 9.

As can be seen from Figure 9, the optimized prediction results were much closer to the real voltage. The average error of the prediction results after optimization was less than 2 mV. The large voltage changes at around 100 points were also well predicted, avoiding the large absolute errors shown in Figure 9a. At the same time, the real voltage has a step-like curve due to the low accuracy of the data. The predicted voltage changes more gently, and the absolute error is around 0, which makes up for the low accuracy of the data, to some extent.

4.2. Verification of Superiority and Stability

To verify the superiority of the state partitioning method, the models were trained separately using three data partitions, and each partitioned model was used to predict the data of the different partitions. The prediction results are shown in Figure 10.

It can be seen that the model trained with the same partition data achieved a better prediction effect on the partition. To some extent, this verifies that the partitioning model is better able to learn the characteristics of the partitioning data, thus avoiding different partition data features reducing the fitting ability of the model. The accuracy of model analysis is improved.

5. Conclusions

Multiple faults of electric vehicle batteries can be diagnosed on the basis of voltage. To find voltage fault information in advance and reduce battery safety risk, a state-partitioned voltage fault prognosis method based on a self-attention network was proposed in this paper. First, the charging voltage curve trends under different states of charge are analyzed. The voltage data are divided into three parts with characteristics typical of each given state of charge. After that, a voltage prediction model based on a self-attentive network is trained with each part of the data separately. Finally, voltage fault prognosis is implemented using the threshold method.

The voltage prediction model trained using the state partition data is able to learn the typical features of each part of the data, improving prediction accuracy. Voltage prediction using a self-attention network improves the long-range data analysis capability and facilitates the learning of more data features.

The method was validated using real operating data of electric vehicles. The effects of different batch sizes and window sizes on model training were analyzed, and the optimized hyperparameters were used to train the voltage prediction model. Finally, the superiority and robustness of the voltage prediction method were verified. The average error of the predicted voltage is less than 2 mV. Moreover, the predicted voltage presents improved data accuracy to some extent. The voltage of each partition is predicted by the partition model. Each model achieves the best prediction results for the partitioning voltages to which it belongs. This verifies that the partitioning model is better able to learn the characteristics of the partitioning data.

The voltage fault prognosis method proposed in this paper is able to achieve better voltage prediction accuracy on real vehicle data under complex operating conditions. The model is able to be effectively trained using a small amount of data. It provides a new research concept for voltage fault prognosis.