Systematic Approach for the Test Data Generation and Validation of ISC/ESC Detection Methods

Abstract

Various methods published in recent years for reliable detection of battery faults (mainly internal short circuit (ISC)) raise the question of comparability and cross-method evaluation, which cannot yet be answered due to significant differences in training data and boundary conditions. This paper provides a Monte Carlo-like simulation approach to generate a reproducible, comprehensible and large dataset based on an extensive literature search on common assumptions and simulation parameters. In some cases, these assumptions are quite different from field data, as shown by comparison with experimentally determined values. Two relatively simple ISC detection methods are tested on the generated dataset and their performance is evaluated to illustrate the proposed approach. The evaluation of the detection performance by quantitative measures such as the Youden-index shows a high divergence with respect to internal and external parameters such as threshold level and cell-to-cell variations (CtCV), respectively. These results underline the importance of quantitative evaluations based on identical test data. The proposed approach is able to support this task by providing cost-effective test data generation with incorporation of known factors affecting detection quality.

1. Introduction

The transformation process towards electrical power systems such as from vehicles with combustion engines towards electrical vehicles (EV) has led to a significant increase in the demand for energy storage systems, which is mainly met by lithium-ion batteries (LIB) [1]. With increasing energy and power densities of such LIB, the thermal stability has captured great attention as potential failures might result in the explosive release of the stored chemical energy [2]. This destructive process called Thermal Runaway (TR) [3] has also come to public interest after the supra-regional media coverage of certain incidents and the consecutive recalls such as the grounding of Boeing 787 [4], the fire incidents of the Samsung Note 7 [5], burning electric buses [6] or problems with the Chevrolet Bolt [7].

The TR is characterized by a sequence of characteristic reactions which significantly influence the fault behaviour in certain temperature ranges. For more detailed information on the individual chemical reactions and mechanisms, please refer to the extensive work of Feng et al. in [2,8,9] as well as Li et al. [10] and Zheng et al. [11] in which the results of accelerating rate calorimetry tests is given. In [9], the authors also provide a detailed discussion of the various failure paths for initiating a TR.

Reported field failures as mentioned above often show a chain-reaction-like behaviour, since nearly every battery system in application consists of multiple cells forming battery modules and packs to fulfil the power and energy requirements. In case of a single-cell TR in such a dense-packed assembly, the released thermal energy can trigger a thermal failure of adjacent cells, propagating the TR through the whole battery system. Therefore, this failure is called Thermal Propagation (TP) and proposes significantly higher risks than a single TR due to the larger amounts of energy-release potential [12].

To address this problem—one of the greatest challenges in battery technology [13]—various solutions have been published and already integrated in battery systems. Despite the broad range of methods, in our previous work [14] three main approaches were identified:

• Reduction or delay of energy release by additives and alternative cell materials [15,16].
• Reduction in heat influx from adjacent TRs to slow down or rather prevent TP in accordance with the US Vehicle Battery Safety Roadmap Guidance, which states that TP must not be initiated [17] if a never fully excluded cell-level TR occurs [18]. Prevention by smart module design [19,20], active or passive cooling strategies [21,22] and/or thermal isolation [23].
• Detection of battery faults and abnormal conditions for countermeasures, warning and evacuation before a hazardous situation develops. Here, the Global Technical Regulation on Electrical Vehicle Safety (GTR-EVS) specifies at least 5 $\min$ or enough time for egress [24].

The first two methods require the implementation of additional material into the battery system or supplementation, reducing the power and energy density or the performance per cost in exchange for increased safety and thermal stability [15,25]. It was also found that reduction in the heat transfer capabilities causes further disadvantages such as limited cooling performance [26] and increased thermal differences within the battery system [27]. In addition, Grabow et al. [28] have proven in a recent study that battery failures such as particle-induced internal short circuits (ISC) cannot be safely ruled out. A passive safety concept might advert, and the affected cell will remain in an unknown—most likely more unstable—state.

Through implementation of a fault detection method, however, both disadvantages can be addressed. The knowledge of the fault appearance even provides the possibility of active counter-measures such as increasing the cooling power or just the warning of operators and the surrounding. Therefore, various methods for fault detection have been proposed in recent years, as extensively summarized by Hu et al. [29]. In accordance with Klink et al. [14], who prove the advantage of evaluating the cell voltage compared to external sensors, these methods are mostly focused on the electrical quantities voltage and current—sometimes extended by temperature. The algorithms and methods utilized to evaluate the battery data originate from various scientific disciplines such as outlier detection [30] from statistics, neural networks from machine learning/data science [31] or modelling [32]. These adoptions of common techniques to improve the detection capabilities underline the importance of the topic.

Despite these very promising studies, no systematic side-by-side comparison of different methods has been published yet—not even in the context of recent extensive review studies [29,33,34,35]. There are, however, studies evaluating advantages and disadvantages of certain methods, e.g., by Hu et al. [29], but the classification based on measures such as sensitivity for noise or high precision [29] is rather subjective and vague [36]. In addition, some researches have published a brief comparison with alternative methods, e.g., [37,38,39], but both implementation and evaluation criteria are limited.

This lack of the ability for comparison is, inter alia, caused by the large variance in validation data and the known or unknown boundary conditions and due to the sensitivity of gathered results to the experimental design [40]. In addition, the results are often based on assumptions [41], which further hampers comparability. Especially, simulation studies are repeatedly criticized for the missing consideration of measurement noise [42] as well as possible cell-to-cell-variations (CtCV) [43,44,45,46,47] when investigating modules.

In summary, the combination of non-standardized evaluation criteria and lack of similar boundary conditions and assumptions hinders the comparison of published results. Therefore, the selection of the optimal method for a certain application from all published approaches is currently not possible. To address these issues, three requirements were identified. First, all methods should be validated on the same dataset. For statistical evaluations, this dataset must contain several test cases with variable default conditions. For the analysis of the detection performance, comprehensible data are advantageous. Second, the dataset should contain sources of uncertainty to represent real application. Consequently, the implemented uncertainty should be based on real world data. Third, the evaluation of the detection methods should be focused on quantitative and statistical measures to guarantee a fair and rational comparison.

This study proposes a methodology for the first two requirements by a data generation method leaned on Monte Carlo simulation. The Monte Carlo approach allows full controllability of boundary conditions and parameters, guarantees the comprehensibility of the data and simplifies the creation of large datasets. Although this work focusses on ISC faults, most statements are also valid for external short-circuit faults (ESC) due to the similar electrical behaviour. Furthermore, the main approach is applicable for other faults by changing the fault simulation part. Besides the variable fault simulation, further variance is integrated into the data by simulation of measurement noise and CtCV. Based on this comprehensible and large dataset the third requirement—qualitative evaluation—can be achieved using established statistical indicators.

The main contributions of this paper are:

• Extensive literature review of disturbances on the measurement signal and their magnitudes;
• Summary of common qualitative and quantitative evaluation criteria;
• Generation of test data with stochastic disturbances and variations with consideration of both fault-free and fault-containing samples with the scope of ISC and ESC;
• Example comparison based on binary classifiers and identification of optimum parameter combinations.

The remainder of this paper is organized as follows. First, the literature review on common assumptions and previous evaluation aspects is given in Section 2 side-by-side with experimentally determined values. In Section 3, the proposed Monte Carlo simulation framework and the underlying assumptions are described in detail. Furthermore, the simulation boundaries are defined as well as two exemplary fault detection methods briefly introduced. The performance of both methods is presented and discussed in Section 4 before the main findings are summarized in Section 5.

2. State of the Art

As mentioned above, recent methods for battery fault detection have been evaluated or criticized—mostly qualitatively—with respect to various measures. Although a complete overview of aspects is not possible due to the broad range, recurring aspects are listed below:

• Complexity or difficulty of the application, e.g.,

  -

  Large battery model parameter sets [31,32,39,41,48,49,50];

  -

  Large fault model parameter sets [30];

  -

  Model limitations [51,52,53,54];

  -

  Processing time [34,37,39,41,42,43,46,47,48,52,55,56,57,58,59,60,61,62,63,64,65,66,67,68];

  -

  Dependency of training data [30,34,37,38,39,41,42,52,56,69,70,71,72];

  -

  General complexity [44,64,66,70,73,74,75,76].

• Simplifications and assumptions concerning:

  -

  Imperfect monitoring data [41,42,50,62,63,64,66,70,76,77,78];

  -

  Deviation from homogeneous cell parameter [43,44,45,46,47,62,72].

• Limitation to single cells [43,45,79,80].

Therefore, origin, experimentally estimated values and implementations in testing of fault detection methods are briefly described in the following.

2.1. Measurement Uncertainty

It is commonly known that every practical measurement is distorted, and the quantity estimated as such is always just an—often sufficient—approximation of the true value due to the existence of random and systematic errors. To standardize definitions, procedures and for extensive reference, the guide to the expression of uncertainty in measurement (GUM) was published. Here, the definitions for the above-mentioned errors can be found at ([81], [B.2.20–B.2.22]). Following this vocabulary, this expected deviation is given as uncertainty of the measurement. The uncertainty itself generally results from various sources, e.g., the measurement device, the conducting person, environmental conditions, the measurement strategy and the measured object itself ([82], [transl.]).

It should be noted that strict adherence to the GUM requires each source to be identified and its individual contribution to the measurement uncertainty to be assessed. The GUM differentiates the origin of the information of the uncertainty, which either is by statistical analysis or by knowledge and classified as Type A and Type B, respectively.

In the context of the commonly used voltage measurements, the resolution and accuracy, sample rate, temperature correction and signal-to-noise ratio can be identified as possible sources of uncertainty. With respect to the finite resolution d of both the sensor and the corresponding analogue-to-digital converter, the estimate $\widehat{X}$ of the true value X can be expressed as $X-\frac{d}{2}\le \widehat{X}\le X+\frac{d}{2}$. Here, the probability function is uniform and not (Gaussian) normally distributed. Strictly following GUM, a normal distribution must be used if the nature of the uncertainty and the probability function are unknown ([81], [4.3.7]).

It is obvious that this task becomes impractical with more complex systems outside well-controlled laboratory conditions. Here, the central limit theorem becomes handy when assuming the presence of multiple independent any-distributed uncertainties. It states that the sum of independently distributed variables will converge towards a normal distribution ([81], [G.2.1]). Thus, expressing measurement uncertainty with normally distributed behaviour, e.g., by Xia et al. and Zhao et al. [76,83], is feasible but still an approximation.

To model this normal distributed uncertainty, an additive component [73,79,84,85] with zero mean $\mu$ (Equation (1)) and given standard variation $\sigma$ (Equation (2)) is commonly used [66,76,77,86] as the given exemplary for a voltage measurement by Equation (3).

$$\mu =\frac{1}{N}\sum _{i=1}^N{x}_i$$

(1)

$$\sigma =\sqrt{\frac{1}{N}\sum _{i=1}^N{(x_i-\mu )}^2}$$

(2)

$$\begin{array}{cc}\hfill \widehat{U}& =U+\Delta U\mathrm{where}\Delta U\sim \mathcal{N}(\mu =0;{\sigma }_U)\hfill \end{array}$$

(3)

Please note that in this simple approach, the uncertainty $\Delta U$ is independent of the measured quantity U.

For application of Equation (3) in simulation, a realistic value for the standard deviation $\sigma$ has to be defined for each measurement quantity independently. Referencing recent approaches, this task is not trivial, as illustrated by the findings for voltage, current and temperature measurements presented in

Table 1. Assumptions for the level of measurement uncertainty for the common battery system quantities cell voltage (U), current (I) and temperature ($\vartheta$) if modelled by zero-mean Gaussian noise with standard deviation $\sigma$. Displayed values were derived from publication if standard deviation was not given. Please refer to the table footnotes for limitations due to the provided data.

Author et al.	${\mathit{\sigma }}_{\mathit{U}}/\mathbf{mV}$	${\mathit{\sigma }}_{\mathit{I}}/\mathbf{mA}$	${\mathit{\sigma }}_{\mathit{\vartheta }}/°\mathbf{C}$	Source
Alavi	0.316			[89] *
Dey	50	0.08	0.5	[55]
Dey	100	3.16	0.447	[56] *
Dey	5	10	0.3	[90]
Dey	5	10	0.3	[91]
Feng	2		0.1	[88]
Feng	1		0.01	[88] *,1
Kang	100			[53] *,2
Kang	100			[57] *,2
Kim		10		[59]
Pan		10		[92] *,2
Shang	10			[39] *
Son		450		[71]
Xia	1			[50]
Zhang	2	10		[87] *
Zhang	2	25	0.05	[93]
Zhao	6			[83]

* Standard deviation was calculated. ¹ Definition by accuracy. ² Definition by amplitude.

. On the one hand, investigation of measurement uncertainty in the context of fault detection is not often performed, despite the many mentions of advantages or disadvantages of certain detection methods. On the other hand, each study defines the uncertainty differently, e.g., in dB [39], as RMS [87], by variance [56], by standard deviation [50] or by accuracy [88]. Furthermore, in some studies the uncertainty seems to be meant Gaussian distributed, but only an amplitude is given [53,57], which is not a useful definition. For the representation in Table 1 a reference voltage of 3.7 V was assumed. The amplitudes and accuracy were treated as standard deviation.

For further illustration, an incomplete overview of exemplary values for measurement uncertainty from application is given in

Table 2. Reference values describing the measurement uncertainty from real application for common battery system quantities. For better comparability in case of percentages given, the absolute values were calculated based on 3.7 V and 44.4 V as nominal voltages for cell and module levels, respectively. The values derived as such are indicated by parenthesis.

Description	Value		Comment	Source
Accuracy from analysed SMC-EV 1 platform	<10 mV			[94]
Accuracy from investigated EV	±5 mV with resolution 1 mV		Cell voltage	[45]
	±1 °C		Cell temperature	[45]
	±0.1 A if $I<$30 A else ±1%		Pack current	[45]
	±1%	(±444 mV)	Pack voltage	[45]
BMS accuracy of EV	±0.1%	(±37 mV)	General assumption, no source	[57]
Standard deviation of investigated module	0.3806 mV		Data from previous study; not published	[14]
Accuracy from BMS-IC 2	±2.8 mV		Cell voltage, max. Value	[95]
	±2.5%	( ±1110 mV)	Pack voltage	[95]
	±5 °C		Temperature	[95]
Accuracy from BMS-IC 2	±1.4 mV		Cell voltage	[96]

¹ Service and Management Center for Electric Vehicles in Beijing. ² Integrated circuit.

. Here, given specifications for real monitoring systems from published studies are summarized as well as application notes, e.g., the guaranteed accuracy of battery management systems (BMS) integrated circuits (IC).

With focus on the voltage measurement uncertainty, a significant deviation between some model representations given in Table 1 with values > 50 mV and the values from application < 10 mV is visible.

Assuming that the exemplary chosen commercial BMS-ICs represent close-to-application values of the measurement uncertainty a selection of 1 mV to 10 mV for ${\sigma }_U$ seems feasible.

2.2. Cell-to-Cell Variations

For nearly every battery application, multiple cells have to be combined to achieve the electrical requirements. Since every cell in such a pack is subjected to small variations from production and material quality, for realistic simulation cell-to-cell variations (CtCV) have to be considered too. Since the CtCV are suspected for self-amplifying behaviour [18] the magnitude of variation is generally expected to increase over the module lifetime by individual ageing progresses [18]. Among other things, different operational conditions [97] such as temperature gradients cause uneven current distribution of parallel connected cells [98]. Similar to the measurement uncertainty, most approaches for describing the CtCV assume an underlying normal distribution. Thus, both mean $\mu$ and standard deviation $\sigma$ (see Equations (1) and (2)) are used for quantifying the variations. Since both measures will change significantly with cell types and sizes, scaling the standard deviation with the mean as described by Equation (4) simplifies the comparability. This ratio from standard deviation relative to the mean is called coefficient of variation ($\mathrm{CV}$) [99] and is often given in %.

$$\begin{array}{cc}\hfill \mathrm{CV}& =\frac{\sigma }{\mu }\hfill \end{array}$$

(4)

While CtCV should be incorporated into models for realistic results [100], there is no publicly available information on production quality of commercial cells. Thus, researchers have to assume proper variations based on educated guesses [101,102] or on the findings from extensive cell characterization studies. Within

Table 3. Overview of the literature on experimental determined CtCV of cell capacity and resistance, given as coefficient of variation ($\mathrm{CV}$); see Equation (4). Please refer to Table 2 for comparison with common approximations for CtCV simulation. Cell specifications were taken from source; please refer to Wildfeuer et al. [103] for an in depth analysis of recent studies.

Author et al.	Year	N	Cell	State	${\mathit{C}}_{\mathbf{nom}}$/Ah	${\mathbf{CV}}_{\mathit{R}}$/%	${\mathbf{CV}}_{\mathit{C}}$/%	Source
Dubarry	2009	100	-	-	0.30	-	1.86	[104]
	2010	100	-	-	0.30	30.12	1.86	[105]
	2011	10	-	-	1.90	5.66	0.16	[100]
Shin	2013	10,000	-	Model	-	4.40	0.00	[106]
Paul	2013	20,000	-	-	4.40	-	1.30	[107]
Zheng	2013	96	-	-	70.00	19.47	-	[45]
Baumhofer	2014	48	Sanyo/Panasonic UR18650E	-	1.85	-	0.50	[108]
Rothgang	2014	700	HP prismatic Cell	New	-	2.87	2.36	[109]
Schuster	2015	954	IHR18650A	Aged, from EV 2	1.95	3.19	1.57	[110]
	2015	954	IHR18650A	Aged, from EV 1	1.95	2.56	2.25	[110]
	2015	484	IHR18650A	New	1.95	1.94	0.80	[110]
Devie	2016	100	NCR 18650B	New	3.35	0.30	0.80	[111]
Campestrini	2016	250	Panasonic NCR18650PD	New	2.80	0.72	0.16	[112]
An	2016	198	-	-	5.30	2.85	1.34	[113]
	2016	7739	-	-	5.30	-	1.45	[114]
Rumpf	2017	600	Sony US26650FTC1	New, Batch 1	3.00	1.81	0.23	[102]
	2017	1100	Sony US26650FTC1	-	3.00	-	-	[102]
Barreras	2017	208	SLPB 120216216	New	53.00	5.63	0.35	[115]
Rumpf	2017	500	Sony US26650FTC1	New, Batch 2	3.00	0.73	0.33	[102]
Devie	2018	51	LG ICR18650 C2	New	2.80	3.55	2.00	[116]
	2018	15	LG ICR18650 C2	Aged, 1000 cycles	2.80	5.00	2.80	[116]
Oeser	2018	50	ICR 18650 26F	Aged, 1464 cycles, 77。8% SOH	2.60	-	1.10	[117]
Baumann	2018	185	BatteryPack, GS Yuasa (LEV50)	Aged, from EV	50.00	4.40	0.85	[118]
	2018	164	Panasonic NCR18650PF	Aged, 3 years	2.90	0.92	0.35	[118]
Zou	2018	248	-	New	3.00	0.95	0.37	[119]
Zilberman	2019	13	LG Chem INR18650-MJ1	New	3.50	1.08	0.22	[120]
	2019	48	LG MJ1	New	3.35	0.68	0.20	[121]
	2019	24	LG MJ1	Aged, 10 months	3.35	0.75	0.38	[121]
	2020	48	LG Chem INR18650-MJ1	New	3.35	0.79	0.20	[122]
Wildfeuer	2021	568	Sony US18650VTC5A	New	2.50	0.86	0.24	[103]
Schindler	2021	48	LG MJ1	New, Batch 1	3.35	0.65	0.20	[123]
	2021	200	LG MJ1	New, Batch 3	3.35	3.40	0.40	[123]
	2021	160	LG MJ1	New, Batch 2	3.35	1.04	0.36	[123]
Oeser	2022	137	ICR18650-26J	Aged, 2 years	2.60	2.00	0.26	[124]
	2022	480	ICR18650-26J	New	2.60	1.69	0.26	[124]
Reiter	2023	14	-	-	128.00	2.20	0.39	[125]
Hein	2023	200	ICR 18650-26J	-	2.60	1.59	0.23	[126]

, a broad overview over recent studies and the corresponding results is given.

Please refer to Wildfeuer et al. [103] for an in-depth analysis of previous studies and measurement procedures.

As indicated by the presented findings, these studies focus on quantities such as capacity, internal resistance and sometimes weight, since these characteristics can be determined by standard measurement procedures with acceptable complexity and time effort. The internal origin of these externally expressed variations is theoretically understood and suspected, e.g., in variations of electrolyte, electrode balancing, etc., as extensively summarized by Beck et al. [127] but no internal root-cause analysis is performed in general by the listed studies. Paul et al. [107] have investigated this aspect by simulating the influence of internal variations on the external parameters R and C; backtracking measured variations onto variations on material level, however, is not possible. Therefore, the only valid data basis for simulating CtCV is external parameters on the basis of a simplified equivalent circuit.

With respect to the given capacities of the investigated cells, with a few exceptions, a clear focus on small size—often cylindrical 18650—formats is recognizable. Thus, cells with capacities < 5 Ah predominate the presented findings. In addition, only very few studies have achieved sample sizes ≈1000 as the majority are ≤200, which is relatively close to a statistical significant sample size. Nevertheless, a very good accordance over all estimated CVs for both capacity and resistance is observable, where ${\mathrm{CV}}_C$ seems to be smaller than ${\mathrm{CV}}_R$ in general. It was assumed that this behaviour is caused by the aim of the manufactures for lower variations of the capacity due to its property as the main performance indicator [114,123]. Recent findings by Wildfeuer et al. after revising previous datasets, however, indicate that the observed differences between ${\mathrm{CV}}_R$ and ${\mathrm{CV}}_C$ may originate significantly by uncompensated measurement errors [103]. Thus, approximation of the CtCV by values in the magnitude of ${\mathrm{CV}}_C\approx 1\%$ and ${\mathrm{CV}}_C\approx 1$ to 5% seems feasible.

It has to be mentioned that the authors of the listed studies identified both a high dependency on the cell batch and transformation of the normal distribution towards a Weibull distribution with the lifetime [102].

When consulting non-academic sources for close-to-application CtCV-values, a broad range from capacity variations of <2.5% [128] up to expected resistance variations of 15% [129] can be found. Since this range differs significantly from the experimentally determined values as given above, a proper definition of CtCV for implementation in simulation remains unclear.

This discrepancy is continued when revisiting the implemented levels of CtCV to validate fault detection methods, as summarized in Table 4. Similar to the non-academic range, the variation is assumed to be »1%, which is not supported by the experimentally determined values. Therefore, these values have to be understood as the worst case approximation. Based on the gathered findings, two configurations of CtCV simulation seem feasible:

• Orientation at statistical founded experimentally determined variations;
• Assessment of the worst case boundaries.

Table 4. Assumptions of CtCV for both capacity (C) and resistance (R) utilized in recent studies in the context of battery fault detection evaluation. For three studies, no cell type was specified. Please refer to Table 3 for comparison with experimental determined CtCV values.

Author et al.	Year	Cell	${\mathit{C}}_{\mathbf{\text{nom.}}}$/Ah	$\mathbf{\Delta }\mathit{R}/\%$	$\mathbf{\Delta }\mathit{C}/\%$	Source
Dey	2016				5, 10 and 15	[77]
Chang	2019	18650 cell	2	10, 20 and 40	20	[130]
Chen	2019	A123 ANR26650-M1A	2300	±3		[30]
Dubarry	2019			0.0, 3.75, 7.5, 12.5 and 15	0.0, 1.25, 2.5, 3.75 and 5	[131]
Zhang	2019			−5, −3, 2 and 5	−5, −3, 2 and 5	[68]
Schmid	2021	Samsung INR18650-25R	2500	+10		[42]
Song	2021		60		0, 1.5 and 2.8	[132]

Table 4. Assumptions of CtCV for both capacity (C) and resistance (R) utilized in recent studies in the context of battery fault detection evaluation. For three studies, no cell type was specified. Please refer to Table 3 for comparison with experimental determined CtCV values.

Author et al.	Year	Cell	${\mathit{C}}_{\mathbf{\text{nom.}}}$/Ah	$\mathbf{\Delta }\mathit{R}/\%$	$\mathbf{\Delta }\mathit{C}/\%$	Source
Dey	2016				5, 10 and 15	[77]
Chang	2019	18650 cell	2	10, 20 and 40	20	[130]
Chen	2019	A123 ANR26650-M1A	2300	±3		[30]
Dubarry	2019			0.0, 3.75, 7.5, 12.5 and 15	0.0, 1.25, 2.5, 3.75 and 5	[131]
Zhang	2019			−5, −3, 2 and 5	−5, −3, 2 and 5	[68]
Schmid	2021	Samsung INR18650-25R	2500	+10		[42]
Song	2021		60		0, 1.5 and 2.8	[132]

Independent of the chosen configuration, the underlying design decisions and database should be disclosed.

Voltage Offset

Despite assumptions to the contrary [54], during the operation of battery packs, no perfect temperature homogeneity can be achieved [68], due to finite heat conductivity. Thus, the cells within a battery system are exposed to slightly different temperatures [21,133,134], which cause variations of the open circuit voltage (OCV) due to entropy effects. Since the entropy coefficient alters with respect to the state of charge (SOC), e.g., within −0.07 mV K⁻¹ to 0.2 mV K⁻¹ [135], no general statement of the effect can be made. With respect to published maximum temperature differences inside battery modules of <10 K [136,137,138,139,140] the voltage variation is expected to be <1 mV. In addition, the already mentioned CtCV causes further voltage variations since the differences in internal resistance will cause slight variations of the voltage-drop and overvoltage during charge and discharge, respectively.

Starting from an approximately identical state, the cells will drift as self-discharge [120], capacity and internal properties vary from cell-to-cell. To compensate for these influences and re-calibrate the cells towards a similar SOC, battery packs and systems are equipped with a monitoring unit (BMS) that will re-balance such deviations—usually by discharging cells with high voltage. Since this balancing causes losses and will never reach perfection due to the above-mentioned measurement uncertainty, a hysteresis is usually implemented. Due to this hysteresis, the open-circuit-voltage (OCV) of cells in battery packs will always slightly deviate. As the balancing is often performed at the end of the charge process, it can be assumed that the $\Delta \mathrm{OCV}$ is approximately constant in-between. Please refer to

Table 5. Published values for the balancing hysteresis $\Delta \mathrm{OCV}$ taken from sources close to field-application, such as application guidelines from BMS-manufacturers or accuracy values given for BMS in the academic literature.

Description	$\mathbf{\Delta }$OCV/mV	Comment	Source
Guideline	100	Trigger for balancing	[141]
Guideline	10	Recommendation for $U_{\text{max.}}-U_{\text{min.}}$	[142]
Guideline	50	Acceptable static voltage	[143]
	100	Acceptable dynamic voltage
Application	20	Optimized balancing	[144]
Application	100	Common hysteresis	[128]
Application	20	Measurement of EV	[145]
	7	Experimental balancing

for an overview of exemplary values for this OCV offset. Similar to the previous aspects, the published range is rather wide and identification of a proper realistic value not trivial. When these magnitudes are compared with the values given for measurement uncertainty (Table 2), CtCV of the measured voltage is significantly more influenced by the balancing hysteresis—thus a constant voltage offset—than by the imperfection of measurement accuracy and resolution. Nevertheless, to best of our knowledge, the performance of fault detection methods have not yet been evaluated under the influence of constant OCV-offset.

2.3. Evaluation Aspects

Irrespective of the chosen approximations of the influencing factors discussed before for the test data, after applying a fault detection method to this dataset, the result needs to be evaluated. First, the calculated defect feature or detection signal can be analysed qualitatively, e.g., by visual inspection as seen in [37,39,146]. However, this simple approach quickly reaches its limits when the properties of interest go beyond, e.g., consistency among few variations. In particular, when different detection parameters, methods or datasets are to be compared, it is necessary to transform the complex fault characteristics and corresponding fault features into a low-dimensional measure. Therefore, the detection time has been used in many studies [41,43,50,51,52,72,73,75,77,79,92,147]. Here, the detection time is defined as the time between the trigger of the fault $t_{\mathrm{ISC}}$ and the time of detection $t_{\mathrm{detection}}$, as given by Equation (5). Using $\Delta t_{\mathrm{detection}}$ also evaluates the requirement for fault detection in an early stage due to the unpredictable development of ISC faults from mild towards sudden TR [148]. This measure is also in line with the GTR requirements mentioned above, where a time between the trigger of the thermal failure and a dangerous situation for the passenger is defined. In addition to the simple evaluation of $\Delta t_{\mathrm{detection}}$, Liu et al. [79] have calculated the average (see Equation (1)), minimum and maximum value of $\Delta t_{\mathrm{detection}}$ for multiple repetitions of the same test.

$$\Delta t_{\mathrm{detection}}=t_{\mathrm{detection}}-t_{\mathrm{ISC}}$$

(5)

By varying the fault size, both Dey et al. and Marcicki et al. have further investigated the smallest fault that was still detectable by their methods [77,86]. This becomes interesting when the disturbances discussed above are included in the test, as these are likely to mask the fault signal of a low magnitude fault.

The process of applying a detection method to a dataset with and without faults is not a battery specific task, but known as binary classifier from many other disciplines, such as pharmacy [149]. As indicated by its name, with each investigated sample two possible states are considered—e.g., a present fault and normal operation. In addition, the applied test has two outputs, indicating either a fault situation (positive) or no fault (negative). Based on these prerequisites, four outcomes of the applied test are possible, as summarized below:

$t_{\mathrm{p}}$	True positive		$t_{\mathrm{n}}$	True negative
$f_{\mathrm{p}}$	False positive		$f_{\mathrm{n}}$	False negative

If evaluated and summed over all conducted tests, the total number of, for example, true positive states $T_{\mathrm{p}}$ is calculated. With these total counts, further measures are defined as listed in

Table 6. Summary of quality indicators for evaluation of a binary test, their definition and usage in recent battery fault detection studies. See also [149,150]. Please note that the reference figure is different in-between indicators and therefore the sum is not equal to 1.

Symbol	Name	Definition	Used in
TPR	True positive rate 1	$\frac{T_{\mathrm{p}}}{T_{\mathrm{p}}+F_{\mathrm{n}}}$	[40]
FNR	False negative rate 2	$\frac{F_{\mathrm{n}}}{T_{\mathrm{p}}+F_{\mathrm{n}}}$	[40,41,63,73,79]
TNR	True negative rate	$\frac{T_{\mathrm{n}}}{T_{\mathrm{n}}+F_{\mathrm{p}}}$
FPR	False positive rate	$\frac{F_{\mathrm{p}}}{T_{\mathrm{n}}+F_{\mathrm{p}}}$	[41,51,55,63,73,79]
PPV	Positive predictive value	$\frac{T_{\mathrm{p}}}{T_{\mathrm{p}}+F_{\mathrm{p}}}$
NPV	Negative predictive value	$\frac{T_{\mathrm{n}}}{T_{\mathrm{n}}+F_{\mathrm{n}}}$
Y	Youden-index	TPR + FNR − 1

¹ Alias: Sensitivity; ² Alias: Specificity.

as well as studies utilizing them.

One observation of the given table is that—to the best of our knowledge—there is no published TNR in the context of battery fault detection yet. This illustrates that usually the presented detection methods are not tested against fault-free data and therefore $T_{\mathrm{n}}=0$. If TPR (sensitivity) and FNR (specificity) have been calculated for different detection method parameters and test boundaries, they can be plotted as conducted by Meng et al. [40]. The resulting curves are called the receiver operating characteristic (ROC) curve and provide the opportunity to identify the parameters for an optimized classification result. A similar assessment is possible with the Youden-index, in which both sensitivity and specificity are considered. Please note that TPR and FNR have to be evaluated together, since a method which always outputs the presence of a fault will obviously catch all faults ($\mathrm{TPR}=1$) but is not useful at all ($\mathrm{FNR}=0$). Due to the severity of the TR, the response to a detection will be dramatic, such as immediate evacuation of an EV. Thus, $f_{\mathrm{p}}$ must not occur regularly, which is measured by the FNR. Nevertheless, due to the severity, TR must not occur without warning ($f_{\mathrm{n}}$), which is incorporated in the TPR.

In addition, some studies have analysed the functionality of the investigated methods, such as the correct identification of the type of fault [52]. Similarly, the convergence of the employed algorithms has been evaluated [90,91]. Methods that estimate the fault magnitude, e.g., the resistance of the ISC, have been accessed on the basis of the accordance between the estimated and correct magnitude [38,51,55].

With respect to the intended application of the various methods within a BMS and in real-time, computational effort becomes a critical factor [67]—especially when tools from data-science are applied that are usually used on computational clusters. Thus, the computational time has been included into the analysis of recent studies [39,59,67,71,151]. This measure, however, has a significant drawback as it is very sensitive to the implementation of the algorithm in detail. To illustrate this problem, a comparison of different moving average implementations written in Python™ is given in the appendix (see Appendix A.1), while the result of all functions is the same, the computational time differs significantly. Thus, deriving an advantage or disadvantage just from the computational time is problematic and most likely biased from the algorithm design. In addition, the importance of this aspect is expected to decrease as the cost of computing power continues to decrease.

3. Material and Methods

To demonstrate a method that incorporates the before-mentioned requirements for a sensible data-generation, an exemplary workflow of fault simulation under the influence of disturbances and the subsequent detection and evaluation of detection methods is presented in the following. After the introduction of the cell chosen as sample for simulation in Section 3.1 the descriptions of model (Section 3.2) and fault detection (Section 3.4) follow.

Within Figure 1 the overall workflow is given—detailed descriptions on certain aspects can be found in the following. First, a simulation case is initialized by the definition of the simulation boundaries for the underlying random influences on the model. Under consideration of both Monte Carlo parameters and fault representation parameters, the model defined as such is repeatedly simulated for no-fault and fault conditions. These two datasets are evaluated using a chosen detection method configuration (see Section 3.4) which gives the fault feature signal for each simulation run. Based on the defined threshold, the fault feature under no-fault condition is evaluated, and a proper threshold $\zeta$ is calculated. This value is then checked against the test dataset with mixed fault and no-fault conditions, and each simulation is classified with $t/f_{\text{p/n}}$. Besides evaluation of individual simulation runs, the summary performance of the individual investigated configurations is analysed in the end.

3.1. Reference Cell

For this study, a commercial off-the-shelf pouch cell by Kokam has been chosen to represent common cell properties. The model name is SLPB98106100 and the nominal capacity is 10 Ah, which is in the range of typical industrially used large-format-sized cells. Following the classification of the manufacturer, the cell is a high energy version. Please refer to

Table 7. Selected datasheet properties of the SLPB98106100 pouch cell from Kokam that was used as reference cell for the simulation.

Parameter	Symbol	Value
Nominal capacity	$C_{\text{nom.}}$	10 Ah
Nominal voltage	$U_{\text{nom.}}$	3.7 V
Upper voltage limit	$U_{\text{max.}}$	4.2 V
Lower voltage limit	$U_{\text{min.}}$	2.7 V
Charge current	$I_{\text{nom.}}$|$I_{\text{max.}}$	5 A|20 A
Discharge current	$I_{\text{nom.}}$|$I_{\text{max.}}$|$I_{<10s}$	5 A|20 A|30 A
Weight	m	0.210 kg

for an overview of cell properties.

3.2. Model

This simulation study is based on an equivalent circuit model (ECM) as the representation of the dynamic cell behaviour. The model of cell and fault was implemented within Matlab/Simulink^® [152] with pre- and post-processing was performed in native Matlab. As displayed in Figure 2, a second order ECM was chosen, which is in accordance with many other studies, where either a first or second order model was chosen as compromise between accuracy and complexity as investigated by Zhang et al. [153].

Using an ECM instead of elaborated models such as mathematical [154] or electrochemical models [88] comes with some advantages:

• Parameterization is doable by standard electrochemical tests;
• Implementation of parameter distribution is simplified;
• Fault representation (see below) is well-defined;
• Simulation time is fast.

To emulate a battery model in Ns1p configuration N cell models are simulated individually with the same load current. Within this study $N=12$ was chosen as common module configuration. In accordance with the ECM (see Figure 2) the individual cell voltage $U_k$ is calculated by Equation (6). The module voltage is then estimated by summation of all N cell voltages $U_k$.

$$U_k\left(t\right)=\mathrm{OCV}-\left(I_{\mathrm{load}}+I_{\mathrm{ISC}}\right)·\left[R_0+\sum _{i=1}^{i=2}{R}_i·\left(1-exp\frac{-t}{R_i·C_i}\right)\right]$$

(6)

In accordance with previous studies such as [60,64,155] the dependency of the model parameters and the OCV by the SOC is incorporated as look-up-table (LUT). Values between provided points are approximated by linear interpolation. The required SOC is calculated using integration of the load current $I_{\mathrm{load}}$ (coulomb-counting) as described by Equation (7).

$$\mathrm{SOC}\left(t\right)={\mathrm{SOC}}_0-\frac{1}{C_{\text{nom.}}}{\int }_{t=0}^t{I}_{\mathrm{load}}\left(t\right)dt$$

(7)

For this study, the simulated sample-rate was set to 10 Hz and the simulation output was stored in as double data type.

As indicated in Figure 2 the thermal dependency of parameters was neglected. With respect to the mild ISC-resistances and short fault duration, this simplification seems reasonable. However, the proposed method is also applicable to more advanced battery models without changes.

3.2.1. ISC/ESC-Fault Representation

Besides some electrochemical fault simulation [88], simplified P2D-models [156] or reduced network models [157] in most cases both ISC and ESC faults are represented by a parallel fault resistance as highlighted by red colour in Figure 2 as well as in Equation (6). Thus, the cell voltage is further reduced by the internal voltage drop caused by the short circuit current. When the fault resistance $R_{\mathrm{ISC}}$ is decreased, the deviation towards the normal cell behaviour increases.

The sudden fault appearance and clearance is realized by a time controlled switching behaviour.

3.2.2. Randomness and Variation

The influence of the previously discussed disturbance variables on a realistic voltage measurement signal should also be included in the generated test data. Thus, the ECM is extended to take into account both the imperfection of the measurement and the variation in the individual battery cells. The details of the implementation are described below.

Measurement Uncertainty

In accordance with most before-mentioned studies (see above, Section 2.1) additive zero-mean Gaussian noise ($\Delta U(t,k)\sim \mathcal{N}(\mu =0,{\sigma }_U)$) is used in this work to emulate a voltage signal with imperfect measurement. As indicated by the dependency of t and k, the noise value is generated randomly for each sample and cell.

Cell-to-Cell Variation

Both voltage offsets $\Delta {\mathrm{OCV}}_k$ and impedance parameter variations $\Delta Z_k$ are implemented into the simulation framework. Variances of cell capacity, however, are not considered separately. First, according to the literature review (see Section 2.2) the expected coefficient of variation is rather small (<1%), causing only small variations in the OCV-SOC behaviour. Second, this small variation is already implemented by the voltage offsets.

Unlike the measurement uncertainty, both variations are assumed to be approximately constant over the simulated time period. Therefore, the value is set for each cell during model initialization. In contrast to the $\Delta \mathrm{OCV}$, which is also implemented as an additive variation, the parameter variation causes a deviation relative to the reference cell parameter as exemplary shown in Equation (8) for $R_0$ (see Figure 2), where $\Delta Z$ represents the relative deviation.

$$R_0=R_{0,\text{ref.}}\frac{1}{100\%}(100\%+\Delta Z)\mathrm{with}[\Delta Z]=\%$$

(8)

The LUT of all parameters given in Figure 2 ($R_{0,1,2}$, $C_{1,2}$) are scaled analogously by the same value. Since the impedance has experimentally proven (see Section 2.2) to behave normally distributed, the scaling factor $\Delta Z_k$ for each cell k is generated from a normal distribution with given standard deviation ($\Delta Z_k\sim \mathcal{N}(0,{\sigma }_Z)$). In contrast, the voltage offset $\Delta {\mathrm{OCV}}_k$ has been found to be significantly influenced by the balancing hysteresis and resolution, which behave uniform distributed according to GUM. Thus, the $\Delta {\mathrm{OCV}}_k$ was generated for each cell from a uniform distribution following $\Delta {\mathrm{OCV}}_k\sim \mathcal{U}(-\frac{d}{2},\frac{d}{2})$ where d is the selected hysteresis width.

3.2.3. Parameterization

The parameters of the ECM shown above were measured beforehand at 20° using the SL1002 6 V/1000 A/0.6 kW battery test bench from Keysight/Scienlab. For all tests, the cell was clamped between two aluminium plates to emulate the clamping force within a battery module [158]. Using screws to tighten the setup, a pre-tension of approximate 0.1 MPa was established, which is close to realistic applications [158,159]. Using screwed connections, the pouch cell tabs were connected to the battery test bench.

The correlation between OCV and SOC was measured by charging and discharging the cell at very low (0.05 C) current, which is called pseudo-OCV (P-OCV) measurement. Averaging the both voltage curves and normalization of the charge with the nominal capacity (see Table 7) gives the $\mathrm{OCV}\left(\mathrm{SOC}\right)$ relationship. The passive parameters of the ECM pictured in Figure 2 were calculated based on current steps with 1 C and 2 C in charge and discharge direction. Both pulses were applied for 10 s and were followed by a 50 s relaxation. To incorporate the SOC-dependency of the parameters, this pulse procedure was conducted for every 10%-SOC increment. Due to the operational limits for 100% and 0% no charge, respectively, no discharge pulse was applied.

Using the SciPy implementation of the Powell-algorithm [160] the model parameters were fitted to the data. Here, both the pulse and the relaxation were considered as well as both currents directions and amplitudes and an overall fit was performed.

The parameterized model was evaluated by means of both standard and normalized root mean squared error (RMSE and NRMSE) compared with a reference dynamic drive cycle test. Please refer to Equations (9) and (10) for the calculation of both metrics. The dynamic load was emulated using the WLTP drive cycle [161] six time, which results in a validation time period of 10,800 s. The achieved simulation quality was 0.0253 V and 0.0286 for RMSE and NRMSE, respectively. These values are in range to similar published results [41,54].

$$\begin{array}{cc}\hfill \mathrm{RMSE}& =\sqrt{\frac{1}{N}\sum _i^N{(U_{\text{meas.}}-U_{\text{sim.}})}^2}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \mathrm{NRMSE}& =\frac{RMSE}{\max \left(U_{\text{meas.}}\right)-\min \left(U_{\text{meas.}}\right)}\hfill \end{array}$$

(10)

3.3. Simulation Cases

For proof of concept of the above-described simulation framework, the following test cases were defined: First, only the measurement uncertainty $\Delta U$ is incorporated to the model (Default), which is the source of uncertainty most often used in recent studies. Second, the two other disturbances $\Delta \mathrm{OCV}$ and $\Delta Z$ are implemented both separately and combined to create test datasets with more kinds of variation. The values for all variations were chosen in accordance to the literature review given in Section 2.1 and Section 2.2 as given in

Table 8. Parameters of the Monte Carlo data generation, including simulated uncertainty and ISC-fault replication. The individual parameter-set was generated randomly based on either a uniform ($\mathcal{U}$) or a Gaussian ($\mathcal{N}$) distribution. Left: Values for the implemented model disturbances dependent on the simulation case, where measurement uncertainty only is considered as Default. Please refer to Section 2.1 and Section 2.2 for further details on the implementation. Right: Intervals for generation of a fault-simulation parameter-set based on a uniform distribution.

	$\mathbf{\Delta }\mathit{U}$	$\mathbf{\Delta }\mathbf{OCV}$	$\mathbf{\Delta }\mathit{Z}$			Range
Distribution	$\sim \mathcal{N}(0,{\mathit{\sigma }}_{\mathit{U}})$	$\sim \mathcal{U}(-\frac{\mathit{d}}{2},\frac{\mathit{d}}{2})$	$\sim \mathcal{N}(0,{\mathit{\sigma }}_{\mathit{Z}})$	Distribution		$\sim \mathcal{U}\left(\mathbf{Range}\right)$
Case	${\mathit{\sigma }}_{\mathit{U}}$/mV	d/mV	${\mathit{\sigma }}_{\mathit{Z}}$/%	Parameter	Symbol
Default ($\Delta U$)	1.0	0.0	0.0	Cell index of fault	k	$\in [1;N]$ *
Modified Default	0.5, 1, 2 and 10	0.0	0.0	Time of fault	$t_{\mathrm{ISC}}$	$\in [1;T]\mathrm{s}$ **
$\Delta U+\Delta \mathrm{OCV}$ or $+\Delta Z$	1.0	10	1.0	Fault duration	$\Delta t_{\mathrm{ISC}}$	$\in [1;120]\mathrm{s}$
$\Delta U+\Delta \mathrm{OCV}$ and $+\Delta Z$	1.0	10	0.1	Fault resistance	$R_{\mathrm{ISC}}$	$\in [1;100]\Omega$

* In this study, $N=12$; ** Using the WLTP cycle $T=1800\mathrm{s}$.

. For investigation of the sensitivity of the detection results towards the magnitude of the disturbances, multiple modified (mod.) parameter sets were created as well. The considered ranges are given in Table 8.

The fault appearance, however, was kept constant for all simulation cases and was based on the following assumptions:

• The fault chance is 80%;
• Only one cell fault per time;
• Only one fault event per simulation run.

All four fault-defining parameters were selected randomly from a uniform distribution. For incorporation of fault-free cases, the fault was applied with a chance of 80%, while the cell index k was chosen within the cell count, 1 to 12 the time of fault $t_{\mathrm{ISC}}$ was chosen from the simulation duration T. Thus, for simulation of the WLTP 1 s to 1800 s were considered. In addition, 1 $\Omega$ to 100 $\Omega$ and $\Delta t_{\mathrm{ISC}}\in [1;T-t_{\mathrm{ISC}}]$ were chosen as boundaries for the fault resistance $R_{\mathrm{ISC}}$ and fault duration, respectively. The selected range is in accordance with various recent studies [40,49,61,88] and the range incorporates both resistances commonly considered as safety-critical (<4 $\Omega$ [162], <10 $\Omega$ [88] and mild criticality (1 $\Omega$ to 100 $\Omega$ [75,163], >10 $\Omega$ [164].

3.4. Fault Detection Methods

To illustrate the proposed approach, two rather simple fault detection algorithms were implemented. Both the implementation of the detection methods and the pre- and post-processing were performed in Python™ (V3.9.12) and are heavily based on the NumPy (V1.21.5) [165], SciPy (V1.7.3) [166] and pandas (V1.4.2) [167] packages.

First, the deviation between individual cell voltages and the mean of the module is considered. Normalization of this deviation with the standard deviation leads to the z-score that is investigated as well. Please find the algorithms defined below. In accordance with other methods, a rolling window filter can be applied to the calculated fault signal for further signal refinement.

To eliminate small deviations stemming from the machine precision the calculated fault signal is rounded to the nearest 8 digits.

Based on the fault signals estimated as such, the required thresholds have to be defined. Within this study, a deterministic approach was chosen to ensure comparability. Since the threshold is often defined by trial-and-error with given reference and fault data, a deterministic approach as performed by Ouyang et al. [75] is seldom documented. The process is described as follows:

• Generate many samples without presence of a fault.
• Calculate the fault signals for the detection method for each sample.
• Determine the maximal fault signal value for each sample.
• Calculate the mean $\mu$ and standard deviation $\sigma$ (see Equations (1) and (2)) of the determined maximal values.
• Define the threshold $\zeta$ as $\zeta =\mu +\lambda \sigma$.
• If the fault signal is greater than $\zeta$ a fault will be assumed.

Thus, by changing the threshold level $\lambda$ the quality of the results (see Table 6), e.g., false positive values (FPR) can be altered. By approximation of an underlying normal distribution, the relationship between $\lambda$ and the samples inside the so-defined boundaries is as given in

Table 9. Probability of samples within multiple standard deviations around the mean of a normal distribution. The two-sided values describe $P(\mu -\lambda \sigma \le x\le \mu +\lambda \sigma )$ and for the one-sided case $P(x\le \mu +\lambda \sigma )$. Here, the left side of the distribution is already fully incorporated.

$\mathit{\lambda }$	2-Side/%	1-Side/%
1	68.27	84.13
2	95.45	97.72
3	99.73	99.87

. Due to the definition of the fault occurrence as excess of the threshold, the one side-probability is given in addition to the more common two-sided one. Within this work $\lambda \in$ 1, 2 and 3 was investigated.

3.4.1. Deviation from Mean

The input of the detection method is the voltage measurement matrix of the module ${\mathbf{U}}^{T\times N}$ with elements $u_{t,k}$. Here, N represents the number of cells and T is the number of samples. For each sample step t, the vector $u_t^{1\times N}$ is evaluated and the mean as well the difference to each cell is calculated as defined by Equations (12) and (11). In addition, following Equation (13) this fault signal vector $F_t^{1\times N}$ can be smoothed by subsequent application of a rolling average filter with window length w using previous sample steps.

$$\begin{array}{cc}\hfill f_{t,k}& ={\overline{u}}_t-u_{t,k}\hfill \end{array}$$

(11)

where

$$\begin{array}{cc}\hfill {\overline{u}}_t& =\frac{1}{N}\sum _{j=1}^N{u}_{t,j}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill f_{t,k}^w& =\frac{1}{w}\sum _{i=t-w+1}^t{f}_{i,k}\hfill \end{array}$$

(13)

Assuming that $u_{t,k}$ of the cell under fault condition will be smaller than without an ISC due to the additional voltage drop (see Figure 2) a positive correlation between amplitude of the fault signal and fault magnitude is expected.

3.4.2. z-Score

The z-score as utilized inter alia in [59,168] is quite similar to the above-mentioned deviation from the mean. However, the deviation as calculated in Equation (11) is standardized by the standard deviation $\sigma$ (see Equation (2) as shown by Equation (14). Thus, the resulting fault signal indicates its deviation from the mean relative to $\sigma$. Similar to before, by application of a moving average filter (see Equation (13)) the z-score can be smoothed, too.

$$\begin{array}{cc}\hfill f_{t,k}& ={\sigma }^{-1}({\overline{u}}_t-u_{t,k})\hfill \end{array}$$

(14)

As the definition is similar to Equation (11) and the difference just normalized, a positive correlation between fault magnitude and fault signal is expected as well.

4. Results and Discussion

Using the described simulation workflow, first the simulation setup and the validity of the gathered results are investigated in Section 4.1 and Section 4.2. Based on these prerequisites, the generated data and implemented fault detection methods are used to evaluate the fault detection functionality and transform the individual result per simulation into an overall describing metric within Section 4.3. The analysis is complemented by further investigations in Section 4.4 where individual simulation and evaluation parameters are investigated in detail.

4.1. Number of Simulations

Since the threshold definition is based on the estimated mean and standard deviation of the simulations without fault, the minimum number of simulations required for a good estimation of these statistics has to be determined. Due to the asymptotic convergence of the sample mean to the population mean with $\sim n^{-\frac{1}{2}}$, increasing the estimation accuracy will significantly increase the number of simulations. Thus, a trade-off between the two aspects is necessary.

Assuming a normal distribution, the confidence interval of the estimated mean of a sample with size n is defined by the limits $\overline{x}\pm z\frac{\sigma }{\sqrt{n}}$ [81]. Here, $\overline{x}$ is the sample mean, $\sigma$ the corresponding standard deviation and z the quantile of the t-distribution associated with the sample size n and the desired confidence level, e.g., 95%. With $n>100$, the t-distribution can be approximated by the normal distribution, thus $z_{95\%}=1.96$ (see Table 9). Rearranging the equation above gives

$$n={\left(\frac{100z\sigma }{\overline{x}ϵ}\right)}^2,$$

(15)

where $ϵ$ is the acceptable deviation in %.

Evaluating both detection methods with different window sizes w for a sample of 300 simulations gives the statistics summarized in

Table 10. Summary statistic coefficient of variation ($\mathrm{CV}$) for the default simulation case with 300 simulation runs. Evaluated maximum fault signal for deviation from mean $\Delta \mu$ and z-score z dependent on the filter window size w. Required minimal simulation runs N to achieve 2% accuracy results with 95% confidence.

Evaluation	$\mathbf{CV}$/%		${\mathit{N}}_{\mathbf{\text{min.}}}^{95\%}$
$\mathit{w}$	$\mathbf{\Delta }\mathit{\mu }$	$\mathit{z}$	$\mathbf{\Delta }\mathit{\mu }$	$\mathit{z}$
1	5.84	1.56	33	3
2	5.45	3.28	29	11
5	6.24	5.24	38	27
10	5.70	5.37	32	28
20	6.48	6.29	41	38
100	8.76	8.02	74	62
200	8.31	7.85	67	60
1000	11.17	10.80	120	113

. The derived minimum number of simulations for a 2% deviation with 95% confidence is given as well. Thus, due to the small sample variation observed, even few simulations < 100 achieve high reliability.

In order to represent the additional variations due to the error simulation, at least 100 simulations for the loads Zero and CC and 1000 simulations for the WLTP are used arbitrarily in the following for no-fault simulations. With respect to the additional variations under fault simulation, here, the number of simulations were doubled. Please also refer to

Table A3. Achieved detection quality of z-score method with threshold level of $\lambda =3$ for repetitive simulation of the default simulation case with no-fault condition (left) and with 80% failure rate (right). Results were obtained on the basis of 1200 and 2400 repetitions for fault-free and fault datasets, respectively. The mean $\mu$ of the maximum fault signal per simulation run is also given. For detailed information on the given indicators FPR and TNR please refer to Table 6.

	$\mathit{\mu }$		FPR/%		TNR/%			FPR/%		TNR/%
No	I	II	I	II	I	II	No	I	II	I	II
$\mathit{w}$							$\mathit{w}$
1	3.108	3.110	0.167	0.167	99.833	99.833	1	0.600	0.832	99.400	99.168
10	1.369	1.364	1.000	0.833	99.000	99.167	10	2.183	3.854	97.817	96.146
100	0.408	0.408	1.083	1.500	98.917	98.500	100	3.523	2.474	96.477	97.526
1000	0.111	0.111	1.083	0.917	98.917	99.083	1000	1.394	2.053	98.606	97.947

for a comparison of the evaluation (see below) of two simulation studies with identical boundary conditions. The high agreement between the two datasets proves that the number of simulations is sufficient and that the gathered results are valid.

4.2. Distribution of Fault Feature

For both the definition of the threshold and the approximation of the required number of simulation runs, a normal distribution of the maximum values of the calculated fault signals was assumed. On the left side of

Table 11. Statistical properties average $\mu$, standard deviation $\sigma$ and skewness ${\mu }_3$ for the maximum fault signal distribution of the fault-free simulation setup with $N=1200$. Evaluated of the fault signals for the detection methods z and $\Delta \mu$ for different window sizes w. The corresponding FPR in % is calculated based on a threshold $\zeta$ associated with $3\sigma$ which should result in a FPR of 0,18% according to Table 9. Left margin: Exemplary barplot of the maximum fault signal distribution for the z-score of $w=10$ (green, highlighted values) and approximation by a normal distribution based on the statistical properties $\mu$, $\sigma$ (blue). Skewness is visualized by marked peak position (white).

		$\mathit{\mu }$		$\mathit{\sigma }$		${\mathit{\mu }}_3$		FPR/%
	Evaluation	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$	z
	$\mathit{w}$
	1	0.004340	3.110	0.000241	0.051	1.053	0.079	0.833	0.167
	10	0.001349	1.364	0.000082	0.076	0.937	0.763	0.583	0.833
	100	0.000392	0.408	0.000032	0.033	0.932	0.935	1.083	1.500
	1000	0.000106	0.111	0.000011	0.011	0.624	0.669	0.583	0.917

this maximum value distribution is exemplary visualized for the z-score -method of 1200 simulations (Zero load: 100, CC-load: 100, WLTP: 1000) and window $w=10$ as highlighted in Table 11. It is visible, that the average is shifted (skewed) towards the right relative to the peak of the distribution, which is quantified by positive values for the skewness ${\mu }_3$ (see Equation (17) from [169]). The calculated skewness for $w=10$ and further distinct windows sizes is given in Table 11. This behaviour is in accordance with the known properties of sample maximum distributions. These sample maximum distributions are either Weibull, Fréchet or Gumbel distributions dependent on the underlying population distribution. For an underlying normal distribution that can be assumed for this case due to the implementation of the disturbances, a Gumbel distribution is expected as sample maximum distribution [170].

$$\begin{array}{cc}\hfill {\mu }_3& =\frac{{\kappa }_3}{{\kappa }_2^{3/2}}\hfill \end{array}$$

(16)

where

$$\begin{array}{cc}\hfill {\kappa }_i& =\frac{1}{N}\sum ^N{(x_i-\mu )}^i\hfill \end{array}$$

(17)

Due to the positive skewness, more values are located on the right side of the mean compared to the normal distribution. Thus, the assumed FPR associated with a certain $\sigma$-based threshold is underestimated, as shown in Table 11 for a $3\sigma$-range. Based on this threshold, the FPR should result in the range of ≈0.18%, as given in Table 9. It is shown that with one exception the FPR is larger but ≤1.5%. Despite the rather small error due to the simplification of assuming a normal distribution, the comparison made should raise awareness that any assumptions regarding the basic distribution functions should be carefully examined in order to be able to estimate and explain the resulting behaviour.

4.3. Fault Detection

The voltage profile of a simulated ISC-fault (here cell 11) is presented exemplary within the top axis in Figure 3 in comparison to a fault-free cell (01) during dynamic WLTP load. For illustrational purposes, a severe ISC-fault of 1 $\Omega$ was chosen, causing a significant voltage drop along the internal resistances as visible in the magnification on the right side. The fault was initiated at $t_{\mathrm{ISC}}=1518$ s and lasts for $\Delta t_{\mathrm{ISC}}=85$ s, as marked within the right axis and indicated by the red background colour. Due to the additional discharge during the ISC-fault, a remaining voltage offset between the faulty cell and the unaffected cell is visible.

In addition, the corresponding fault signal $f_z^{10}$ of both cells is given in the bottom part of the Figure. Here, the z-score filtered by 10 sample periods, thereby 1 s, was chosen. Please note the detection threshold $\zeta$ based on a $3\sigma$ interval as indicated by the horizontal line.

At the start of simulation—just under the influence of measurement noise—the fault signal is noisy but with the presence of the fault the z-score of the faulty cell increases virtually immediately and surpasses the threshold. Thus, the fault is already detected after 0.3 s. After the fault is gone, however, the fault signal remains above the threshold due to the above-mentioned voltage offset, while this sensitivity of the z-score to offsets simplifies the detection of smaller faults with less initial voltage-drop, it causes problems when voltage offsets exist already in fault-free samples, as discussed within Section 3.2.2.

Following the observations, this simulation in combination with the method $z^{10}$ and ${\zeta }^3$ is classified as true positive ($t_{\mathrm{p}}$). Evaluating all 2400 simulations for this method and threshold gives the results presented on the left side of Figure 4. Here, each simulation is coloured based on the achieved classification, where $t_{\mathrm{p}}$ is green and $f_{\mathrm{n}}$ is red. Please also note the simulation discussed above marked by a star in the upper left part.

An approximate linear dependency between both fault resistance $R_{\mathrm{ISC}}$ and fault duration $\Delta t_{\mathrm{ISC}}$ and the achieved classification is observable. In contrast, no such dependency was observed for the $f_{\mathrm{p}}$ classification that occurred randomly with low frequency. To illustrate the dependency between detection and no detection under the presence of a fault, the decision boundary for $t_{\mathrm{p}}$ and $f_{\mathrm{n}}$ was calculated by using a support vector machine (SVM) algorithm. This boundary is marked by a dashed line in the figure. Although the change from $t_{\mathrm{p}}$ to $f_{\mathrm{n}}$ is not so much sharp and other contrary classifications can be found beyond the boundary line, the chosen representation represents a good summary of the individual simulations:

First, the right and left hand side area approximates the FNR and TPR, respectively, since the figure displays all fault-cases ($T_{\mathrm{p}}+F_{\mathrm{n}}$, see Table 6). Secondly, the intercept with the horizontal axis (bottom and top) indicates the smallest detectable fault (see Section 2.3). In combination with the slope of the boundary, the smallest detectable fault with respect to the fault duration can be approximated as well. Thus, the slope can be used to understand which parameter has more impact on the classification quality.

With these prerequisites, the results of both detection methods and with variable window sizes w can be compared as given on the right in Figure 4. Here, each line is the calculated decision boundary between $t_{\mathrm{p}}$ and $f_{\mathrm{n}}$.

It is visible that with the same filter size the z-score is always left to the $\Delta \mu$. Thus, the FNR is expected higher and the smallest detectable fault or rather the highest detectable fault resistance is smaller. In addition, both methods show similar behaviour when the filter size is increased $w\to 100$ as the slope decreases and the intercept with the horizontal axis increases, resulting in a significant better detection performance based on FNR and detectable resistances. For filter sizes $w>100$, however, this trend is reversed, and a decreasing performance is observed.

This behaviour is due to two effects that occur with increasing filter size: First, the influence of the measurement noise on the fault signal is reduced, which also results in significant smaller threshold levels. Therefore, smaller faults can be detected since the signal-to-noise ratio increases. Secondly, the sudden fault signal deviation at fault appearance (see Figure 3) is filtered as well, which increases the time to surpass the threshold. Thus, the fault duration becomes more important on the detection results with increasing filter size. In addition, the charge difference between the faulty cell and the remaining cells also increases with the fault duration. Since both methods are biased by offsets, this developing deviation provides a second possibility for fault detection besides the initial voltage drop.

Therefore, an optimum between filtering noise and removing fault information has to be found, which was observed in this study at approximate $w=100$.

Within

Table 12. Classification quality indicators for the fault detection with both z-score and $\Delta \mu$ for a fault simulation setup with $N=2400$ and ≈80% fault cases under default measurement uncertainty. The classification is evaluated under different filter sizes w and underlying threshold level $\lambda$. Please refer to Table 6 for the definition of the indicators. The graphical illustration visualizes the values for $\lambda =3$, where the corresponding window is marked by colour.

			TPR		FPR		FNR		Youden
		Evaluation	z	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$	z	$\mathbf{\Delta }\mathit{\mu }$
	$\mathit{w}$	$\mathit{\lambda }$
	1	1	0.208	0.336	0.491	0.391	0.792	0.664	−0.283	−0.055
		2	0.136	0.268	0.123	0.123	0.864	0.732	0.013	0.144
		3	0.079	0.227	0.006	0.033	0.921	0.773	0.073	0.194
	10	1	0.774	0.820	0.354	0.362	0.226	0.180	0.420	0.458
		2	0.686	0.731	0.091	0.121	0.314	0.269	0.596	0.610
		3	0.617	0.665	0.022	0.024	0.383	0.335	0.595	0.642
	100	1	0.966	0.966	0.353	0.337	0.034	0.034	0.613	0.630
		2	0.962	0.963	0.118	0.108	0.038	0.037	0.845	0.855
		3	0.955	0.958	0.035	0.029	0.045	0.042	0.920	0.929
	1000	1	0.941	0.946	0.379	0.391	0.059	0.054	0.562	0.555
		2	0.931	0.935	0.104	0.100	0.069	0.065	0.826	0.835
		3	0.919	0.924	0.014	0.014	0.081	0.076	0.905	0.910

the calculated quality indicators of the discussed study are given. The table is accompanied by a graphical illustration of the values for the z-score with $\lambda =3$ for each given w.

Please note the decreasing FNR values with increasing w, as was visually analysed before. In addition, the approximately opposing characteristic of TPR is found in the data as well.

Taking TPR of the z-score at $w=100$ (Table 12, grey backfill) it seems that $\lambda =1$ is the best option, since it has the highest value and nearly every fault was detected. The FPR, however, also gives a high rating, meaning that $\approx 1/3$ of fault-free cases were also classified as fault. Thus, the TPR alone is not a suitable measure, since identifying just every test case as fault would give $\mathrm{TPR}=1$. This problem can be solved by the Youden-index, since it combines both sensitivity and specificity into one indicator. Using this index, the visually determined best configuration of window and threshold level at $w=100$ and $\lambda =3$, respectively, is confirmed. While the former analysis is focused on the classification into fault and no-fault, other potential measures are feasible as well, e.g., the before-mentioned detection time $\Delta t_{\mathrm{detection}}$. Due to its definition given in Equation (5), only cases with correct fault-detection ($t_{\mathrm{p}}$) are considered by this indicator. Accordingly, the evaluation based on only $\Delta t_{\mathrm{detection}}$ can only provide a limited picture of the detection performance—similar to using just TPR. Especially with low TPR rates (see Table 12) using $\Delta t_{\mathrm{detection}}$ might remove significant chunks from the data.

However, since $\Delta t_{\mathrm{detection}}$ has been used in recent studies (see Section 2.3) the achieved times are given in Figure 5 with respect to the corresponding fault parameter value. In accordance with the fault parameter space described in Table 8 the variables are (a) the fault resistance $R_{\mathrm{ISC}}$, (b) the fault duration $\Delta t_{\mathrm{ISC}}$ and (c) the time of fault $t_{\mathrm{ISC}}$. For the sake of clarity, only the z-score method is presented for the configuration $\lambda =3$, $w=10$ but similar behaviour can be observed for other configurations as well.

Despite the time-dependent load amplitude of the WLTP used, there is no correlation between $t_{\mathrm{ISC}}$ and $\Delta t_{\mathrm{detection}}$ in (c). Thus, the commonly found validation with fixed $t_{\mathrm{ISC}}$ may be less prone to a biased result.

Neither with regard to $R_{\mathrm{ISC}}$ nor $\Delta t_{\mathrm{ISC}}$ is a classical correlation to be observed. Instead, however, the range of possible detection times seems to change as a function of the variables, as can be seen from the relatively sharp edges in (a) and (b). The dashed border in (b) represents the duration of the fault itself, so that no fault could be detected after its clearing. This behaviour is disadvantageous for the application, as short faults may remain undetected. With regard to the resistance, a minimum size can be recognised, which is necessary for a detection within a certain period of time. This consideration is of particular interest for designs according to the 5 min criterion of the GTR-EVS mentioned at the beginning.

Removing all fault-cases without detection from the analysis for evaluating a fault detection method seems counter-intuitive; therefore, an opposing approach is described in the following:

For comparison of two not-detected ISC faults, the assessment of the corresponding criticality seems feasible. However, due to limited understanding of the ISC processes and the highly stochastic behaviour [171] the fault characteristic often remains unclear and the dynamic state not feasible for proper assessment [9,33,172].

Therefore, in the context of this study, the released energy during the fault duration $\Delta t_{\mathrm{ISC}}$ starting at the ISC trigger $t_{\mathrm{ISC}}$ is utilized for comparison. Since the energy increases with the $\Delta t_{\mathrm{ISC}}$ and decreases with the fault resistance $R_{\mathrm{ISC}}$ the fraction of both is taken as an approximation of the associated criticality $\kappa$ of an unidentified fault as given in Equation (18).

$$\kappa =\frac{\Delta t_{\mathrm{ISC}}}{R_{\mathrm{ISC}}}$$

(18)

Thus, in addition to the smallest detectable fault (see above), this indicator provides information on the severity of potential misses.

In Figure 6, the highest criticality value that was not detected is presented for the before-mentioned variations of detection methods and parameter are given. Here, a higher value represents an undetected fault with either longer fault duration or smaller resistance. Thus, for most applications, a small value is desired.

It is clearly visible that with increasing filter size the most severe missed fault becomes less and less significant. Increasing the threshold limit, however, has a contrary effect. The former observation is most likely linked to the already identified improvement of the detection results with increasing filter size (see Figure 3). On the contrary, enlarging the threshold will cause longer detection times and misses of smaller faults, which leads to a higher not-detected criticality.

4.4. Further Investigations

For the previous analysis, the unlimited range of parameters had to be restricted to certain values in order to allow clear evaluation and comparison. The sensitivity of these restrictions is examined below.

4.4.1. Threshold Level

In the previous discussion, the dependence of the classification result on the chosen threshold $\lambda$ was repeatedly observed. However, the observed characteristic of increasing performance with increasing threshold could not be predicted, as two effects are to be expected: On the one hand, increasing the threshold reduces the probability of $f_{\mathrm{p}}$. On the other hand, the significance of the error signal required to detect an error increases. Accordingly, an a priori consideration is difficult to make. Therefore, and since the values of 1, 2 and 3 were chosen rather arbitrary, the deviation of the Youden-index due to $\lambda$ was evaluated.

The corresponding characteristics are given in Figure 7 for both methods and the known selection of w. First, the dependency between achieved detection performance—assessed by the Youden-index $\mathrm{Y}$—and threshold level $\lambda$ is clearly visible. This dependency is in high accordance to literature statements that the threshold definition has significant impact on the detection result [35,71]. The observation, however, calls into question the general validity of results obtained by means of the often described trial-and-error procedure based on experimental data, which was also used by the authors in previous work. Due to the limited amount of test data in the context of timely and expensive experimental abuse test and the large sample size needed [44], the trial-and-error procedure is advantageous.

The evaluation in Figure 7 additionally shows that the achievable improvement decreases asymptotically for both methods and for all filter sizes. Thus, the reduction in $f_{\mathrm{p}}$, which is associated with higher thresholds, is advantageous in terms of classification quality. However, due to the asymptotic behaviour, further increases such as the $6\sigma$-level as described by Ouyang et al. [75] do not lead to large improvements.

4.4.2. Noise Level

For the preceding analysis, the default simulation case with normal distributed noise with $\sigma =1$ mV was considered. Although this value was chosen based on the broad literature review in Section 2.1, this value is not physically derived. Thus, the influence of the underlying noise level on the achieved detection results has to be evaluated.

In Figure 8, the detection results for simulation studies with $\sigma =$ 0.5 mV, 1 mV, 2 mV and 5 mV as separated by colour are given. According to the previous discussion, the Youden-index is chosen to represent both sensitivity and specificity. For each filter size w, the Youden value of both z-score and $\Delta \mu$ is given side-by-side with different fill-patterns. Please note the different alpha levels corresponding to the threshold levels.

The decrease in classification quality with increasing noise level is clearly observable for each w, which even results in negative Youden values when only a small filter is utilized. In addition, the differences between certain noise levels diminish for higher filter sizes. Besides the unfiltered case ($w=1$), no significant difference between z-score and $\Delta \mu$ can be observed—$\Delta \mu$ seem to be slightly higher more often.

The figure also shows the significant improvement of classification with higher threshold levels for all cases in accordance with the discussion before. Especially the improvement from $\lambda =1$ to $\lambda =2$ is advantageous for the overall performance.

Based on the results, no linear dependency between noise level and detection quality is identified. The level of decrease seems to be dependent on both w and $\sigma$ in a non-linear fashion.

4.4.3. CtCV

In contrast to the investigated simplified simulation case with only consideration of the measurement uncertainty, the initial review has discussed further influences of disturbances. Thus, the preceding analysis was performed under the influence of additional CtCV in the form of parameter variation ${\sigma }_Z$ and voltage offset $\Delta \mathrm{OCV}$. The corresponding fault detection accuracy is given in Figure 9 based on the already utilized Youden-index. For reference, the simplified simulation case is also presented.

It is visually obvious that the performance of the investigated methods decreases significantly under the influence of additional disturbance that are either constant ($\Delta \mathrm{OCV}$) or load dependent (${\sigma }_Z$). Especially, adding $\Delta \mathrm{OCV}$ into the data generation prevents any reliable fault detection. Under consideration of the discussion of the fault feature characteristic in Figure 3 this behaviour has become apparent due to the sensitivity of the fault feature towards the remaining charge deviation or rather voltage offset.

Thus, both methods—as implemented in this study—are not suitable for proper fault detection under the influence of CtCV in addition to measurement uncertainty and optimization is required. Here, one solution could be to evaluate $d{U}_k/dt$ instead of $U_k$ to compensate for $\Delta \mathrm{OCV}$. As long as the load current is constant—which it is usually not—this will also work for the deviation between cells due to the slightly deviated cell impedance.

While the performance of the investigated methods is limited by these results, the importance of implementing CtCV into the test datasets of fault detection methods has been underlined. As presented in Section 2.2 this has not been conducted in general yet. Thus, the performance of the published detection methods has to be evaluated with respect to CtCV.

5. Conclusions

Within this publication, the well-known and much discussed factors influencing the measurement signal of battery systems, which can affect the possibilities of reliable detection of ISC faults, were presented first. In accordance with the literature, this overview was focused on the voltage signal. By comparing common assumptions in the literature of these influencing variables for the validation of fault detection methods and corresponding values from experimental investigations or from the application, significant differences could be identified, while the measurement inaccuracy and scatter of cell parameters tend to be overestimated, no considerations of voltage offsets due to BMS hysteresis could be found. With respect to the orders of magnitude to be expected here compared to, for example, measurement inaccuracy, this influence should be taken more into account in future work.

Based on this preliminary work, a simulation workflow was presented to generate test data for the validation of fault detection methods in a controllable manner, with different boundary conditions and in a statistically sufficient quantity.

The resulting possibilities were tested exemplarily on two simple detection methods and the obtained results were evaluated using various measures. Here, among other things, the greatest criticality of false-negative classifications was introduced as a modification of the smallest detectable fault. In addition, established indicators such as sensitivity, specificity and the Youden-index were used to test the methods under different boundary conditions. Based on the generated dataset, the limits of common evaluation indicators such as TPR-only or detection time were discussed, since they give no information on false-positive and false-negative detections. To obtain a complete picture of the detection performance, this study emphasizes both usage of non-fault validation data and consideration of FPR or similar indicators.

For both methods, the best performance was found for a filter width of $w=100$ using a sample rate of 10 Hz and a deterministic threshold definition of $\zeta =\mu +\lambda \sigma$ with $\lambda =3$. Here, $\mu$ and $\sigma$ represent the mean and standard deviation of the fault signal under fault-free conditions, respectively. It could additionally be shown that the gain in performance decreases asymptotically by an additional increase in the limit value. Increasing the threshold limit further results in a higher energy release, as expressed by the criticality $\kappa$. By simulating variants with higher measurement noise and with additional parameter and OCV deviations, it could be shown that the performance decreases significantly with additional disturbances.

These observed dependencies have already been partially investigated in the literature, but not regularly or under non-uniform boundary conditions. The results of this work emphasize the necessity of investigating these confounding variables, since the detection performance is significantly affected. The partly significant deviations of the results depending on the definition of threshold and filter width show that published results are only comparable to a very limited extent if the boundary conditions and test data are not guaranteed to be identical. This results in the necessity mentioned above to compare the numerous published methods under identical conditions and on identical data. The adaptation of a Monte Carlo simulation for data generation presented here can be used very well for this purpose. The main underlying concept as displayed in Figure 1 can be also adopted to more advanced battery models and fault representation if required. Furthermore, by using a simulation approach, the extension of the investigation on the basis of another reference cell, as well as the investigation of a generic cell, is possible.

The identified influences of the signal disturbances on the detection quality can be further used to optimize the requirements of the BMS, e.g., an acceptable noise level with respect to the required detection accuracy.

Based on the preliminary work and methodology presented, the next step will be to expand the evaluation to include other established detection methods. Furthermore, it is planned to supplement the simulated data with experimentally determined faults in order to take into account the dynamic unsteady behaviour of a more realistic ISC.