Artificial Feature Extraction for Estimating State-of-Temperature in Lithium-Ion-Cells Using Various Long Short-Term Memory Architectures

Abstract

The temperature in each cell of a battery system should be monitored to correctly track aging behavior and ensure safety requirements. To eliminate the need for additional hardware components, a software based prediction model is needed to track the temperature behavior. This study looks at machine learning algorithms that learn physical behavior of non-linear systems based on sample data. Here, it is shown how to improve the prediction accuracy using a new method called “artificial feature extraction” compared to classical time series approaches. We show its effectiveness on tracking the temperature behavior of a Li-ion cell with limited training data at one defined ambient temperature. A custom measuring system was created capable of tracking the cell temperature, by installing a temperature sensor into the cell wrap instead of attaching it to the cell housing. Additionally, a custom early stopping algorithm was developed to eliminate the need for further hyperparameters. This study manifests that artificially training sub models that extract features with high accuracy aids models in predicting more complex physical behavior. On average, the prediction accuracy has been improved by $\Delta T_{\mathrm{cell}}=0.01 °\mathrm{C}$ for the training data and by $\Delta T_{\mathrm{cell}}=0.007 °\mathrm{C}$ for the validation data compared to the base model. In the field of electrical energy storage systems, this could reduce costs, increase safety and improve knowledge about the aging progress in an individual cell to sort out for second life applications.

1. Introduction

Motivation

Lithium-ion-battery technology is a promising energy storage technology for transitioning to electro-mobility due to its advantages, such as low self-discharge rate, long cycle life, scalability, high energy/power density, etc. To ensure safety requirements and longevity, cell parameters such as cell voltage U, cell current I and ideally cell temperature $T_{\mathrm{cell}}$ need to be monitored at all times [1,2,3]. From these parameters, other vital cell states, such as state-of-charge SoC and state-of-health SoH, can be derived. The performance, longevity, safety, and applicability for second-life-applications are strongly dependent on the accuracy of these parameters. Obtaining the temperature $T_{\mathrm{cell}}$ is especially challenging for two reasons. First, most battery systems work with temperature sensors that are attached to the cells’ housing. Thus, not the heat generation in the cell core is monitored, but the heat dissipation from the cell interior to the environment [3,4]. Furthermore, the cell temperature varies in a battery system depending on the location of the cell in a module [5,6]. Therefore, a sensor-based temperature tracking of each individual cell is recommended, however, not feasible in many applications. On the other hand, there are various software based prediction techniques such as mathematical–physical models and machine learning. However, physical models are highly computationally intensive [2,7], rely on information that is not available in real life applications [8,9,10] or is too specific to apply for every scenario that a battery is exposed to in real life applications. Some popular examples are finite element analysis, simulating the thermal field distribution by the Newman model [11], or the heat generation model by the Bernadi model [12]. In contrast, there are data-driven models, with their main advantages being transferability and universality [13]. Most data-driven models aim to make predictions about SoC [14] or SoH [15]. Temperature predictions using time series forecasts are rather rare and very specific. For example, Kleiner et al. [16] uses real life temperature data to make predictions for the future. The emphasis is thermal cause and thermal behavior over a cells lifetime in contrast to more complex real life situations that can have varying results [17,18,19].

This paper aims to compare artificial neural network (ANN) architectures to track thermal behavior of a single li-ion battery cell (LIBC) with a limited amount of training data and at a constant ambient temperature. This will show the impact of artificial feature extraction to improve the tracking capability of a physical non-linear systems.

2. Theoretical Foundations

This section will provide theoretical foundations regarding temperature behavior in LIBCs and the machine learning aspects used in this paper. First, it is shown how heat is generated and dissipated. This will give an insight of the complexity of dynamic non-linear systems such as LIBCs. Secondly, a short introduction to the machine learning aspects are given. The focus lays on recurrent neural networks to generate a time series forecast. It is important to notice, this section does not represent the scientific contributions gained by this paper, but shows the groundwork laid down by others.

2.1. Temperature Behavior in Lithium-Ion-Battery-Cells

The goal of this subsection is to gain an understanding of the complexity and investigate the necessary information to predict the temperature behavior in a LIBC. Therefore, we will look at the physical-thermal behavior of a single cylindrical LIBC. If we assume the internal temperature as well as the heat generation in a cell to be evenly distributed, we can derive from the law of energy conservation:

$$c_{\mathrm{cell}}{m}_{\mathrm{cell}}\frac{d{T}_{\mathrm{cell}}}{dt}=Q_{\mathrm{gen}}+Q_{\mathrm{dis}},$$

(1)

with $c_{\mathrm{cell}}$ representing the specific heat capacity, $m_{\mathrm{cell}}$ the mass of the LIBC and $\frac{d{T}_{\mathrm{cell}}}{dt}$ the temperature change over time t [20]. $Q_{\mathrm{gen}}$ is the heat generation rate and $Q_{\mathrm{dis}}$ the heat dissipation rate. Equation (1) assumes that every additional heat source is infinitely far from the LIBC. In other words, the ambient temperature $T_{\mathrm{amb}}$ is uniform.

2.1.1. Heat Generation

The heat generated inside a LIBC can be further subcategorized into reversible heat ${\dot{Q}}_{\mathrm{r}}$, ohmic heat ${\dot{Q}}_{\mathrm{o}}$, heat from material phase changes ${\dot{Q}}_{\mathrm{p}}$ and heat produced by the enthalpy absorbed or liberated from substances mixing ${\dot{Q}}_{\mathrm{m}}$ [3,20,21]. To simplify the model for heat generation, we will neglect voluminous expansion during charge and discharge cycles, assume that the active material is uniformly distributed and the diffusion coefficient of li-ion in solid particles is assumed as constant.

Reversible heat ${\dot{Q}}_{\mathrm{r}}$ is the energy needed to maintain the energy balance of the reaction caused by the gradient between the Gibbs free energy of product and reactant. Suppose the layer-by-layer cell structure is summarized as one block, the reversible heat ${\dot{Q}}_{\mathrm{r}}$ can be calculated by

$${\dot{Q}}_{\mathrm{r}}=I{T}_{\mathrm{cell}}\frac{\partial U_{\mathrm{OCV}}}{\partial T_{\mathrm{cell}}},$$

(2)

with $U_{\mathrm{OCV}}$ representing the open circuit voltage (OCV) potential [7,21,22]. $U_{\mathrm{OCV}}$ is a function of the State-of-Charge (SoC) without the effects of overvoltage taken into account. Ohmic heat ${\dot{Q}}_{\mathrm{o}}$ is generated by collisions of charged particles in the solid and electrolyte phase and can be simplified by

$${\dot{Q}}_{\mathrm{o}}=I(U-U_{\mathrm{OCV}}),$$

(3)

with (U-$U_{\mathrm{OCV}}$) filtering the overvoltage [21]. Thus, for ${\dot{Q}}_{\mathrm{o}}$, only the potential caused by the current load is analyzed. The heat from material phase changes ${\dot{Q}}_{\mathrm{p}}$ can be expressed by

$${\dot{Q}}_{\mathrm{p}}=-\sum _{\begin{array}{c}i\end{array}}\Delta H_{\mathrm{i}}^{avg}{r}_{\mathrm{i}},$$

(4)

and generally represents the heat either produced or consumed by the enthalpy change $\Delta H_{\mathrm{i}}^{avg}$ of a chemical reaction i multiplied by the rate $r_{\mathrm{i}}$ at which the reaction i takes place [21,23,24]. Heat produced from mixing is calculated by

$${\dot{Q}}_{\mathrm{m}}=-\int \sum _i(\overline{H_{\mathrm{i}}}-{\overline{H}}_{\mathrm{i}}^{avg})\frac{\partial c_{\mathrm{i}}}{\partial t}dv,$$

(5)

and is caused by the enthalpy $\overline{H_{\mathrm{i}}}$ absorbed or liberated by two or more substances mixing [24]. To isolate $\overline{H_{\mathrm{i}}}$, the average concentration of the enthalpy ${\overline{H}}_{\mathrm{i}}^{avg}$ of i has to be subtracted and then multiplied by the concentration change over time $\partial c_{\mathrm{i}}/\partial t$. In a LIBC, this represents the formation and relaxation of concentration gradients within the LIBC and has to be integrated over it’s volume v. To obtain the total heat generated by a LIBC ${\dot{Q}}_{\mathrm{gen}}$, all subcategories have to be summed up. However, it is worth knowing, that the impact of ${\dot{Q}}_{\mathrm{o}}$ and ${\dot{Q}}_{\mathrm{r}}$ far exceeds the impact of ${\dot{Q}}_{\mathrm{m}}$ and ${\dot{Q}}_{\mathrm{p}}$.

2.1.2. Heat Dissipation

With the aim of gaining a full understanding of the thermal behavior of a LIBC, not only heat generation but also, heat dissipation $Q_{\mathrm{dis}}$ needs to be taken into account. $Q_{\mathrm{dis}}$ describes the heat escaping from the LIBC and is caused by three main phenomena: Radiation heat ${\dot{Q}}_{\mathrm{ra}}$, convection heat ${\dot{Q}}_{\mathrm{cv}}$ and conduction heat ${\dot{Q}}_{\mathrm{cd}}$.

Radiation heat ${\dot{Q}}_{\mathrm{ra}}$ describes the energy transport from its surface to the surrounding atmosphere and can be calculated by Stefan–Boltzmann law with

$${\dot{Q}}_{\mathrm{ra}}=\epsilon \sigma A(T_{\mathrm{out}}^4-T_{\mathrm{amb}}^4),$$

(6)

where $\epsilon$ represents emissivity (dependent on the materials and colors used), $\sigma$ is the Stefan–Boltzmann constant, A is the surface area of the LIBC, $T_{\mathrm{out}}$ is the surface temperature and $T_{\mathrm{amb}}$ the ambient temperature [25]. Heat convection ${\dot{Q}}_{\mathrm{cv}}$ can be further subcategorized into forced and natural heat convection. In this study, we will only look at the natural ${\dot{Q}}_{\mathrm{cv}}$. A simplified approximation is given by Jossen [26] with

$${\dot{Q}}_{\mathrm{cv}}=A{c}_{\mathrm{K},\mathrm{n}}(T_{\mathrm{out}}-T_{\mathrm{amb}}),$$

(7)

with $c_{\mathrm{K},\mathrm{n}}$ representing the constant of natural convection [25]. Heat conduction ${\dot{Q}}_{\mathrm{cd}}$ is caused by mechanical contact from the LIBC surface area A to other materials as well as the electrical wiring of the terminals. With Fourier’s law, we can derive

$${\dot{Q}}_{\mathrm{cd}}=kA\frac{d{T}_{\mathrm{out}}}{dx},$$

(8)

as a one-dimensional representation x with the thermal conductivity between the two materials.

To summarize, aside from material constants and dimensional details, we would need information about the OCV, enthalpy behavior and concentration changes of several reactions and temperature data. In practical applications, this information is not accessible. This study aims to predict the temperature behavior with only the current profile I and its corresponding voltage response U without the need for solving differential equations.

2.2. Time Series Forecast

This section gives an overview of the machine learning aspects needed to follow along with this paper. ANNs are a subset of machine learning and at the heart of deep learning applications. A basic ANN is built with node layers, consisting of an input layer, a number of hidden layers, and an output layer [27]. Each layer has one or more neurons with individually associated weights and activation function. A training data set consists of labeled data points and is needed to train the weights [28]. During a supervised-learning–training process, the input layer provides the ANN with new information. This information is passed onto the first hidden layer, where it is forwarded to the following hidden layers. If all the information from one layer is passed onto each neuron from the next layer, it is called a fully connected layer [29]. The information passing through the neurons is altered by the weights. Once the information arrives at the output layer, it is compared with the label and an error value ${\tau }_{\mathrm{loss}}$ is calculated via the mean squared error method. After a number of data points pass through the ANN (batch) a backpropagation algorithm corrects the weights of each neuron so that ${\tau }_{\mathrm{loss}}$ is minimized [30]. This process is repeated for a number of iterations (epochs). For a time series forecast, it is not only important to know the information provided by the input layer, but it is crucial to know several time steps leading up to the information (window size $h_{\mathrm{win}}$) and the previously predicted value. This is made possible by a recurrent neural network (RNN). A special kind of RNN is an architecture made out of long short-term memory (LSTM) cells [31]. This is a four layered ANN compromised into one neuron that can learn to filter information based on its importance [32]. To ensure that the ANN is learning the nature behind the predicting values and not fitting to the training data, a validation data set is needed as well. With the validation data set, an additional validation error ${\tau }_{\mathrm{val},\mathrm{loss}}$ is calculated. If the behavior of ${\tau }_{\mathrm{loss}}$ and ${\tau }_{\mathrm{val},\mathrm{loss}}$ drift into different directions during the training process, overfitting occurs.

3. Experimental

3.1. Method of Measurement

The LIBC used in this study is a high power LIBC (INR18650-15L1) from Samsung. According to the manufacture, the LIBC can be cycled in a range of $2.5 \mathrm{V}\le U\le 4.5 \mathrm{V}$ and has a nominal discharge capacity of $C_{\mathrm{N}}=1.5 \mathrm{Ah}$. To avoid an offset from heat generation $Q_{\mathrm{gen}}$ to the actual measurement, a negative temperature coefficient (NTC)-sensor (NTC Thermistor 10k Perle SC30F103V by Amphenol) $R_{\mathrm{sens}}$ is placed in the center of the cell.

As seen in Figure 1 cylindrical cells come with a production-related cavity in the center of the cell wrap, where the sensor is located. To access the cavity, a through-hole was drilled in the center of the negative terminal. After positioning the sensor, the through-hole was sealed with a two-components-epoxide resin glue. This process took place in a glovebox filled with an argon atmosphere.

An electrochemical impedance spectroscopy (EIS) in the range of $0.01 \mathrm{Hz}\le f\le \mathrm{10,000} \mathrm{Hz}$ had been carried out before and after the procedure. Figure 2 shows the results in a Nyquist plot with a negative $\mathrm{Im}\left(\mathrm{Z}\right)$-axis. The graph marked as “Pre” represents the data measured before the LIBC was equipped with the sensor and the graph marked as “Post”, the data after the sensor was inserted. Two regions are highlighted by a zoom window. The zoom window on the right focuses on the zero crossing of $\mathrm{Im}\left(\mathrm{Z}\right)=0$. This is where the ohmic internal resistance $R_{\mathrm{i}}$ of the LIBC can be determined. The resistance after the procedure $R_{\mathrm{i}},\mathrm{post}=0.02240 \mathsf{\Omega }$ has only a minor increase compared to before the procedure $R_{\mathrm{i}},\mathrm{pre}=0.02237 \mathsf{\Omega }$. The zoom window on the right shows the other side of the semi-circle. The two high points (a) and (b) are (0.03078 Ω|-0.00080 Ω) and (0.03174 Ω|-0.00082 Ω), respectively. According to H. Ruan, B. Sun, and J. Jiang et al. [33], the accuracy in the low-frequency range can be drastically improved with the modified-electrochemical impedance spectroscopy-based multi-time-scale fractional-order model. However, here the focus lies on the two marked regions, rather than on the diffusion dynamics. In addition, Figure 2 is meant to compare relative changes instead of absolute values.

To read and interpret the sensor values, a custom measuring system was designed. The resistance value from the sensor was measured in a voltage divider against a SL1TTE1002F by KOA Speer Electronics inc, and digitalized by an ADS1115 by Adafruit. The data is then processed with an Arduino Mega 2560 and logged onto a Micro-SD-Card. The measuring system is able to log roughly 4.5 data points per second. In post production, the digitalized data was calculated to temperature values according to the sensors manual. In order to get a sense of the noise produced, every data point is logged and averaged out. To account for contact resistance and other influences by the circuit, the sensors were validated at various ambient temperatures $T_{\mathrm{amb},\mathrm{c}}=(283.15 \mathrm{K},298.15 \mathrm{K},313.26 \mathrm{K},333.15 \mathrm{K})$. The equation

$$R_{\mathrm{sens},\mathrm{x},\mathrm{new}}=1.027R_{\mathrm{sens},\mathrm{x}}-R_{\mathrm{offset}},$$

(9)

is capable to compensate the offset with $R_{\mathrm{offset}}=80.561 \mathsf{\Omega }$. The cell was placed in the climate chamber IPP260plus by Memmert and kept at $T_{\mathrm{amb}}=298.15 \mathrm{K}$ for the entirety of the experiment.

3.2. Current Profile

Figure 3 shows the data used to train the models. First $T_{\mathrm{cell}}$ as measured as in Section 3.1 is presented. Derived from $T_{\mathrm{cell}}$ is the linear approximation of $d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)=(T_{\mathrm{cell}}\left(t_{\mathrm{i}}\right)-T_{\mathrm{cell}}(t_{\mathrm{i}}-1))/(t_{\mathrm{i}}-t_{\mathrm{i}-1})$ followed by the current profile I and it’s corresponding voltage response U. The profile can be subdivided into four sections. Each section is presented with a sample of an enlarged view of the temperature $T_{\mathrm{cell}}$ as well as the current I in Figure 4. First, the LIBC is strained with 21 full constant current ($C=1 \mathrm{C}$) charge/discharge cycles with no rest in-between. An enlarged view is given in Figure 4a). This is followed by three full random constant current charge/discharge cycles in the range of $-4 \mathrm{A}\le I\le 4 \mathrm{A}$ and can be viewed in Figure 4b). Positive values represent charging- and negative values discharge-mode. To ensure that the temperature $T_{\mathrm{cell}}$ starts every cycle with $T_{\mathrm{cell}}=T_{\mathrm{amb}}$, there is an $I=0 \mathrm{A}$ phase for $t=1 \mathrm{h}$ in-between each charge/discharge cycle. After that nine different pulse cycles are implemented. The first four cycles are $t=10 \mathrm{s}$ pulses, with $t=10 \mathrm{s}$ rest in-between discharge cycles with random current values analogous to section (b) and the remaining cycles are charge-/discharge cycles for $t=30 \mathrm{s}$ with no rest in between. One example can be viewed in Figure 4c). It is important to use full cycles in order to ensure the models ability to learn the temperature $T_{\mathrm{cell}}$ to $SoC$ relationship. The last section is meant to increase the level of complexity. The LIBC starts with $SoC=50\%$. From there, a random value in the range of $-4 \mathrm{A}\le I\le 4 \mathrm{A}$ is applied to the LIBC for a random time segment in the range of $5 \mathrm{s}\le t\le 40 \mathrm{s}$. A zoomed view is extracted in Figure 4d).

Figure 5 shows the data used to validate the models. Every scenario described for the training data is implemented here as well. However, it was ensured that the current values I differ from the training data set. Analogous to Figure 4, there is an enlarged view of all scenarios in Figure 6 showing $T_{\mathrm{cell}}$ as well as the current I.

3.3. Data Preparation

The LIBC was cycled by the PEC ACT0550. In order to merge the temperature data with the cell data, the time stamps ($t_{\mathrm{pec}}$ for PEC and $t_{\mathrm{temp}}$ for the measuring system) of the two systems have to be synchronized. Analyzing the data, leads to the equation

$$t_{\mathrm{pec}}=1.0009t_{\mathrm{temp}}-t_{\mathrm{offset}},$$

(10)

with the intercept $t_{\mathrm{offset}}=2.5011 \mathrm{s}$ representing the offset in which the two systems measuring state were started. The cell is placed in a climate chamber as described in Section 3.1.

Finally, the data was normalized. Since I-data consists of positive as well as negative values, the scaling range is $-1\le I\le 1$. The scaling range for U and T are $0\le U\le 1$ and $0\le T\le 1$, respectively. This impacts the resolution by 50%; however, during the research conducted, it was revealed, that the ANN benefits more from learning the importance of the value zero.

4. ANN-Architectures

Every sub-model is constructed the same way as a function of:

• Input block (red) consists of the input nodes (U,I) as described in Section 2.1.
• Hidden block (green) is a function of the number of hidden layers $h_{\mathrm{lay}}$ and the number of neurons $h_{\mathrm{neu}}$. Each neuron is a LSTM neuron and each layer is fully connected as described in Section 2.2.
• dT block (purple) is a fully trained LSTM model built as in Figure 7—Model-dT. It is structured like Model A; however, it was trained to predict the linear approximation $d{T}_{\mathrm{cell}}/dt$ as calculated in Section 3.2.
• Output block (blue) is a single LSTM neuron to bundle all the information coming from the hidden block.

The minimal time span necessary to predict a temperature value is depended on the window size $h_{\mathrm{win}}$. This means if $h_{\mathrm{win}}=x$ the ANN will be able to make a prediction of $T\left(x\right)$ based on the input values of $I(x-h_{\mathrm{win}})$ to $I\left(x\right)$ and $U(x-h_{\mathrm{win}})$ to $U\left(x\right)$. In this study, three different model architectures are being investigated. The schematic of each model is represented in Figure 7. Model-A will function as the base model consisting of input-, hidden- and output blocks. Model-B has the same structure as Model-A except in-between the input block and hidden block is a fully trained dT block (Model-dT). Therefore, Model-B takes advantage of a form of artificial feature extraction, since the feature $d{T}_{\mathrm{cell}}/dt\left(t\right)$ is not a product of how a model was trained, but has been trained in a controlled environment to produce $d{T}_{\mathrm{cell}}/dt\left(t\right)$. Model-C includes the dT block as well. However, the hidden block is being supplied with the information from the input block as well as the dT block.

Figure 8 illustrates an example of a model structure like Model-A or Model-dT. White neurons represent input neurons and yellow neurons represent LSTM neurons. Each neuron is fully connected to the upcoming layer. The background colors are analogous to Figure 7. This example uses three neurons for two layers in the hidden layer. The input and output structure is the same across all models. Model-B and Model-C have a fully trained dT-Model in-between input and hidden layer as described.

Five hyperparameters were investigated:

• Window size $h_{\mathrm{win}}$: This is the number of data points the ANN will see at each time step. If the algorithm is supposed to estimate a temperature value for $T\left(100\right)$ and $h_{\mathrm{win}}=10$, then the ANN would be fed $T\left(89\right)-T\left(99\right)$. In order to make all architectures comparable, $h_{\mathrm{win}}$ will only be dynamic at the base model (Model-A). For Model-B, -C and –dT, $h_{\mathrm{win}}$ will be static. In case a value $T\left(x\right)$ needs to be estimated with a window size $h_{\mathrm{win}}=y$ with $y>x$, the data padding approach will be used [34]. This means, that the values will be automatically filled in to create an array with the appropriate dimensions.
• Number of neurons $h_{\mathrm{neu}}$: This value represents the total number of neurons in the hidden block.
• Number of hidden layers $h_{\mathrm{lay}}$: This is representative of the number of hidden layers in the hidden block. The number of neurons $h_{\mathrm{npl}}$ of each individual layer l can be calculated by

  $$h_{\mathrm{npl}}\left(l\right)=\left\{\begin{array}{cc}\lfloor \frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}\rfloor ,\hfill & \lfloor \frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}\rfloor -\frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}<\frac{l}{h_{\mathrm{lay}}}\hfill \\ \lfloor \frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}\rfloor +1,\hfill & \lfloor \frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}\rfloor -\frac{h_{\mathrm{neu}}}{h_{\mathrm{lay}}}\ge \frac{l}{h_{\mathrm{lay}}}\hfill \end{array}\right.$$

  with $h_{\mathrm{neu}}\ge h_{\mathrm{lay}}$ and $h_{\mathrm{lay}}\ge l$
• Learning rate: During the backpropagation process, the weights are being adjusted according to Section 2.2. Learning rate $h_{\mathrm{lear}}$ is the order of magnitude by which the weights are adjusted. A strong $h_{\mathrm{lear}}$ is unlikely to find the optimal solution while a small $h_{\mathrm{lear}}$ will make it challenging to reach a conclusion in a reasonable time frame. This is especially important when using an early stopping approach. In addition, this study uses an ADAM (derived from adaptive moment estimation) optimization algorithm to dynamically change the $h_{\mathrm{lear}}$ during the training process [35].
• Drop rate $h_{\mathrm{drop}}$: This parameter is meant to counteract overfitting by stochastically taking weights out of the equation. If we assume $h_{\mathrm{drop}}=0.1$, this would mean every neuron has a 10% chance of being bypassed.

To bypass the need for the number of training epochs $n_{\mathrm{ep}}$ being yet another hyperparamter, a custom early-stopping approach has been integrated. If the training process is running for $n_{\mathrm{ep}}>10$, the training will stop if either ${\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep}}\right)<{\tau }_{\mathrm{loss}}\left(1\right)$ or if the slope $a_{\mathrm{reg}}$ of

$$a_{\mathrm{reg}}=\overline{{\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep}}\right)}-\overline{n_{\mathrm{ep}}}\frac{{\sum }_{i=n_{\mathrm{ep}-10}}^{n_{\mathrm{ep}}}(n_{\mathrm{ep},\mathrm{i}}-\overline{n_{\mathrm{ep}}})({\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep},\mathrm{i}}\right)-\overline{{\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep}}\right)})}{{\sum }_{i=n_{\mathrm{ep}-10}}^{n_{\mathrm{ep}}}{(n_{\mathrm{ep},\mathrm{i}}-\overline{n_{\mathrm{ep}}})}^2},$$

(11)

with

$${\displaystyle \overline{n_{\mathrm{ep}}}=\frac{1}{10}\sum _{i=n_{\mathrm{ep}}-10}^{n_{\mathrm{ep}}}{n}_{\mathrm{ep},\mathrm{i}},}$$

(12)

and

$${\displaystyle \overline{{\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep}}\right)}=\frac{1}{10}\sum _{i=n_{\mathrm{ep}}-10}^{n_{\mathrm{ep}}}{\tau }_{\mathrm{loss}}\left(n_{\mathrm{ep},\mathrm{i}}\right),}$$

(13)

is positive.

To evaluate the models with individual hyperparameters, a constrained global optimization package built upon Bayesian inference and the Gaussian process were used. It attempts to find the maximum value of an unknown high cost function in as many few iterations as possible by adjusting the aforementioned hyperparameters [36]. It starts with a random grid search with $n_{\mathrm{grid}}=10$ epochs. From there, the algorithm picks the best result as a starting point for the Bayesian optimization that runs for a maximum of $n_{\mathrm{bays}}=100$ epochs. Because of the stochastic nature of ANN every iteration went through the training process twice and the arithmetic mean of the two error rates was calculated.

5. Results and Discussion

In this section, the results of all the models will be presented, compared and discussed.

Table 1. Hyperparameter results. The table shows the hyperparameter configuration for the hyperparameters $h_{\mathrm{win}}$, $h_{\mathrm{lay}}$, $h_{\mathrm{neu}}$, $h_{\mathrm{drop}}$ and $h_{\mathrm{learn}}$ for each model. The “Bayesian” area shows the arithmetic mean of two results with the same hyperparameter configuration and “Individual” area shows the results for each individual model.

	Model	${\mathit{\tau }}_{\mathbf{train}}$	${\mathit{h}}_{\mathbf{win}}$	${\mathit{h}}_{\mathbf{lay}}$	${\mathit{h}}_{\mathbf{neu}}$	${\mathit{h}}_{\mathbf{drop}}$	${\mathit{h}}_{\mathbf{learn}}$
	A	$9.02\times {10}^{-3}$	8	2	5	$0.0670$	$4.23\times {10}^{-6}$
Bayesian	B	$5.48\times {10}^{-2}$	8	1	5	$0.0818$	$1.72\times {10}^{-6}$
	C	$7.44\times {10}^{-3}$	8	1	5	$0.0876$	$8.95\times {10}^{-6}$
	A	$2.05\times {10}^{-3}$	10	1	28	$0.0419$	$6.88\times {10}^{-6}$
Individual	B	$5.73\times {10}^{-3}$	8	1	24	$0.0687$	$8.36\times {10}^{-6}$
	C	$1.04\times {10}^{-3}$	8	1	82	$0.0373$	$2.12\times {10}^{-5}$

shows the best results achieved in this experiment and is structured in the “Bayesian” and “Individual” areas. All models are trained to reduce ${\tau }_{\mathrm{train}}$, and the hyperparamter $h_{\mathrm{win}}$ is set static after the hyperparameter investigation of Model-A is completed as described in Section 3.

The area “Bayesian” is the arithmetic mean of the two results for the training loss ${\tau }_{\mathrm{train}}$ with the corresponding hyperparameters. Model-C was able to achieve the best results. To reduce the impact of the arithmetic mean, every individual model trained in “Bayesian” was investigated for “Individual”. Again, Model-C outperformed Model-A and Model-B.

The results shown in Table 1 are illustrated in Figure 9 for the training data set and Figure 10 for the validation data set.

Both figures show the predicted temperature values for the two models trained in the area “Bayesian” (“Bayesian #1” and “Bayesian #2”) and the overall best result in the area “Individual” for all models (Model-A, Model-B, Model-C and Model-dT) in contrast to the measured values (Ground Truth). In addition, each figure is subdivided into four areas (a)–(d).

These areas provide four enlarged views for a more detailed analysis on the training data in Figure 11 and validation data in Figure 12. The time span and position of all areas are identical to the areas in Figure 3 and Figure 5, respectively.

The success rate of the Model-dT is higher than predicting the absolute value. Calculating the linear approximation $dT/dt$ as shown in Section 3.2 will amplify the noise produced by $T_{\mathrm{cell}}$. Figure 9 and Figure 10 show, that the ANN successfully performs a noise reduction and therefore, compresses the upper and lower limits. However, it filters out the information when heat is generated $Q_{\mathrm{gen}}$$(d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)>0)$ and when heat is dissipated $Q_{\mathrm{dis}}$$(d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)<0)$. Thus, it aids Model-C to produce absolute values, which is to be taken as the key message of this research. Model-dT by itself is not sufficient to extrapolate the absolute values since every error is being summed up, resulting in a temperature drift which is not suitable for real life applications.

The predictions in Figure 9 show similar performance results to the predictions in Figure 10. Therefore, no signs of overfitting occurred.

The models do not aim to simply classify the sections in Section 3.2 and predict a mean value. Analyzing the enlarged sections (Figure 11a–d and Figure 12a–d) shows the models’ ability to learn the physical behavior of the LIBC. The model knows when $T_{\mathrm{cell}}$ increases/decreases depending on the previous time step, and the active current load and voltage response that had been given in $h_{\mathrm{win}}$. Therefore, the model understands when the effects of $Q_{\mathrm{gen}}$ or $Q_{\mathrm{dis}}$ dominate.

Due to the limited amount of training data, not every situation gets a sufficient amount of attention. This is most notably when the LIBC heats up from $T_{\mathrm{amb}}$ or cools down to $T_{\mathrm{amb}}$.

The “Individual” predictions perform visually the poorest of all graphs. This shows the importance of having more than one ANN trained with the same hyperparameter settings to reduce the effects of the stochastic nature.

The overall investigated temperature range is $24.53 °\mathrm{C}\pm 0.01 °\mathrm{C}\le T_{\mathrm{cell}}\le 35.76 °\mathrm{C}\pm 0.01 °\mathrm{C}$. In this spectrum, Model-A achieved an accuracy of $\Delta T_{\mathrm{cell}}=0.023 °\mathrm{C}$ during training and $\Delta T_{\mathrm{cell}}=0.015 °\mathrm{C}$ during validation, Model-B $\Delta T_{\mathrm{cell}}=0.064 °\mathrm{C}$ during training and $\Delta T_{\mathrm{cell}}=0.043 °\mathrm{C}$ during validation and Model-C $\Delta T_{\mathrm{cell}}=0.012 °\mathrm{C}$ during training and $\Delta T_{\mathrm{cell}}=0.008 °\mathrm{C}$ during validation.

Model-B shows the poorest performance. This is most likely due to the dimensionality reduction. Where the input layer of Model-A has a dimension of two (U,I) and Model-C a dimension of three (U,I,$d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)$), the hidden block in Model-B is only provided one dimension ($d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)$). Model-A and Model-C both show that the physical temperature behavior of a cell was learned. This is best illustrated in section (d) of Figure 11 and Figure 12. The LIBC can correctly predict the trend when to cool down and when to heat up based on the current inputs (U,I) and the previous cell states. Model-C outperformed Model-A and Model-B.

6. Conclusions and Future Work

The impact of artificial feature extraction using the first derivative in combination with ANN has been investigated. In this regard, the following steps have been taken to show its effectiveness:

• A LIBC has been prepared with an internal NTC-temperature sensor with the aim to prevent a time delay from heat generation to heat dissipation.
• A custom measuring system was designed to track the temperature and to synchronize the temperature data with the data of the battery system.
• Using this approach, a training- and validation dataset was created to investigate three LSTM-architectures. A hyperparameter analysis for each model has been carried out to find the optimal model structure for each sub model. Model-A architecture is the base model, Model-B architecture uses an additional dT-layer, that has been separately trained to forecast the linear approximation of $d{T}_{\mathrm{cell}}/dt\left(t_{\mathrm{i}}\right)$ and the third Model-C benefits from both approaches (Model-A and Model-B). Model-C was able to outperform Model-A and Model-B, which shows that artificial feature extraction is a useful method to improve model accuracy in the non-linear state of temperature prediction in LIBCs. This method made it possible to increase the accuracy by $\Delta T_{\mathrm{cell}}=0.01 °\mathrm{C}$ for the training data and by $\Delta T_{\mathrm{cell}}=0.007 °\mathrm{C}$ for the validation data compared to the base model with only the information of the current profile I and its corresponding voltage response U.

To show the full potential of data driven temperature prediction models for lithium-ion-battery systems, I would like to make the following suggestions for the data set used to train the model:

• Broader temperature range;
• Variable ambient temperature;
• Implementing real life drive cycles.

In addition, the transferability needs to be looked at. That would include transferability between cell technologies and extrapolation from cell to module level. To compensate for the high computational effort to train an ANNs and the need for training data, a cloud server can be implemented, trying to find patterns across battery technologies and different topologies.