Comparative Battery State of Charge (SoC) Estimation Using Shallow and Deep Machine Learning Models

Abstract

This paper evaluates neural-network approaches for lithium-ion battery state-of-charge (SoC) estimation under a unified pipeline, fixed data partitions, and identical preprocessing. We study FNNs trained with Levenberg–Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG) across three hidden sizes (10, 20, 30) and three topologies: Fitting, Nonlinear Input–Output (Nonlinear I/O), and time-series NAR/NARX. Models are assessed using test MSE and RMSE, correlation (R), generalization gap, convergence indicators (final gradient, damping factor), wall time per epoch, and a relative compute-cost index. On the Fitting task, BR-Fitting-FNN with 20 neurons provides the best accuracy-efficiency balance, while LM-Fitting-FNN with 30 neurons reaches slightly lower error at a higher cost. For Nonlinear I/O, BR-Nonlinear I/O-FNN with 30 neurons achieves the lowest test MSE with clear evidence of effective weight shrinkage; LM-Nonlinear I/O-FNN with 20 neurons is a close alternative. In time-series settings, LM-NAR-FNN with 10 neurons attains the lowest test error and fastest epochs but shows a very negative gap that suggests test-split favorability; BR-NAR-FNN with 30 neurons is more costly yet consistently strong. For NARX, LM-NARX-FNN with 20 neurons yields the best test accuracy and robust convergence. Overall, BR delivers the most reliable accuracy–robustness trade-off as networks widen, LM often achieves the best raw accuracy with careful split validation, and SCG offers the lowest training cost when resources are limited. These results provide practical guidance for selecting SoC estimators to match accuracy targets, computing budgets, and deployment constraints in battery management systems.

1. Introduction

Lithium-ion batteries are widely used in devices ranging from smartphones to electric vehicles, making real-time state-of-charge (SoC) monitoring essential for safety, longer battery life, and efficient energy usage [1]. Traditional SoC estimation methods, such as open-circuit voltage look-up tables, equivalent circuit models with Kalman filtering, and full electrochemical observers, offer valuable physical insights. However, they often require extensive parameter tuning and tend to lose accuracy over time as the battery ages or operates under changing temperatures [2]. With the availability of large-scale cycling datasets and affordable computing power, data-driven methods have gained attention. ANNs are capable of directly learning the relationship between current, voltage, and temperature signals and SoC, without the need for explicit battery modeling [3]. Early approaches used FNNs trained with algorithms like Levenberg–Marquardt or Bayesian regularization. These models can handle nonlinear relationships but rely on sliding-window inputs to capture time-related behaviors [4]. Recurrent neural networks have been introduced to solve this limitation. LSTM networks can retain information over long sequences, while GRUs achieve similar performance with fewer parameters and faster training [5]. More recently, TCNs have replaced recurrence with one-dimensional dilated causal convolutions and residual blocks, ensuring that predictions at time step t depend only on inputs from time t and earlier [6]. Hybrid models that combine TCN and LSTM have shown further improvements by combining convolutional feature extraction with recurrent memory, leading to lower RMSE values [7].

1.1. Feedforward Neural Networks (FNNs)

Multilayer FNNs consist of an input layer, one or more hidden layers, and an output layer, where data flows strictly in a forward direction. Thanks to their universal approximation property, FNNs can model any Borel-measurable function on a compact domain in ℝⁿ, provided that enough hidden units and appropriate nonlinear activations are used [8]. Training typically relies on backpropagation with gradient-based optimization. For regression problems, squared-error loss and its gradient are computed efficiently using the chain rule [9]. In SoC estimation, shallow FNNs can capture nonlinear static relationships between voltage, current, and temperature. Still, they require manual time-windowing to handle temporal patterns and often struggle with long-term dependencies [5,7,10].

1.2. Training Algorithms for FNNs

The Levenberg–Marquardt (LM) algorithm is a damped least-squares method that interpolates between Gauss–Newton and gradient descent. It is well-suited for medium-sized networks and typically converges quickly when the Hessian is well behaved [9,11,12]. Bayesian Regularization (BR) introduces weight decay into a probabilistic framework, allowing the network to balance accuracy and complexity through data-driven hyperparameter adjustment automatically. This tends to improve generalization, especially in noisy battery datasets [5,13,14]. The Scaled Conjugate Gradient (SCG) algorithm avoids explicit line searches by using second-order information to scale conjugate gradient directions. It is more memory-efficient than LM and converges faster than standard gradient descent [15].

1.3. Neural-Network Time-Series Families (Shallow/Dynamic Nets)

NARX (Nonlinear Autoregressive with Exogenous Inputs) models use delayed inputs and past outputs to feed a static FNN. In contrast, NAR networks use only past outputs, and nonlinear input–output (NIO) networks process only current inputs without feedback [16,17,18]. NARX architectures are particularly effective for nonlinear dynamics and systems with memory [19]. In SoC estimation, where the battery evolves slowly over time, these architectures can outperform plain FNNs by embedding useful temporal information directly into the network structure [5,7].

1.4. Long Short-Term Memory (LSTM) Networks

LSTMs enhance standard recurrent neural networks with gated memory cells that control input, output, and forget mechanisms. These gates help retain long-term information and prevent vanishing gradient issues using a structure known as the constant error carousel [20]. LSTMs are widely used in SoC estimation because they can learn temporal dependencies in voltage, current, and temperature without the need to manually select time lags [7,21]. However, they typically require more training time and parameters compared to GRUs or convolutional models [6,20].

1.5. Gated Recurrent Units (GRUs)

GRUs simplify LSTM architecture by merging the input and forget gates into a single update gate, and using a reset gate to manage memory updates. This reduces the number of trainable parameters while retaining strong performance on sequential tasks [22]. For SoC estimation, GRUs often match LSTM accuracy on moderate-length sequences and benefit from faster convergence and reduced data requirements, making them suitable for real-time applications in battery management systems (BMS) [6].

1.6. Temporal Convolutional Networks (TCNs)

TCNs replace recurrence with one-dimensional dilated causal convolutions, ensuring that predictions at time t depend only on past inputs. Dilation enables the network to capture long-range dependencies with fewer layers, and residual connections with dropout improve training stability [4,23]. TCNs have shown strong performance on sequence modeling tasks and allow full parallelization during training [23]. In SoC applications, both TCNs and hybrid TCN–LSTM models have reported lower RMSE by combining convolutional feature extraction with recurrent memory mechanisms [7].

1.7. Comparative Analysis for SoC Estimation

A standard SoC estimation pipeline includes preprocessing (e.g., normalization, windowing), training with cross-validation, and evaluation using metrics such as RMSE, MAE, or MAPE [5,7]. FNNs trained with LM, BR, or SCG remain competitive for static mappings but rely on manual windowing to capture temporal trends [5,9,12,14,15]. NARX, NAR, and NIO networks explicitly embed time delays, which is useful when domain knowledge identifies key lags [5,16,17,18,19]. LSTMs are preferable when long-term temporal behavior is complex or unknown, though they may overfit smaller datasets without proper regularization [7,20,21]. GRUs offer a balance between performance and efficiency, often matching LSTM accuracy with fewer resources [6,22]. TCNs provide high accuracy with fast, parallel training but require careful design of receptive fields to handle both fast transient and long-term dependencies [7,23]. Hybrid models such as TCN–LSTM increasingly represent the best compromise between accuracy and robustness [7]. Final model selection should account for sequence length, sampling rate, computational cost, and deployment constraints within the target BMS platform [5,6,7,21].

Direct comparisons between FNN, LSTM, GRU, and TCN architectures are still limited. Many existing studies use different preprocessing methods or hyperparameter settings, which makes it difficult to judge which model performs best under the same conditions [24,25]. These variations create inconsistencies that complicate accurate cross-study benchmarking and obscure the evaluation of relative model effectiveness. Furthermore, the implementation of complex composite algorithms, utilizing serial or parallel architectures, substantially increases model complexity [26]. As the number of integrated modules expands, the overall controllability of the system decreases. This reduction in control increases sensitivity to disturbances, raising critical concerns regarding the model’s robustness in practical applications [27]. This study aims to address the gap by training all four architectures on the same dataset using identical preprocessing and a unified hyperparameter search. Their performance is then compared based on accuracy, computational cost, and generalization ability. This work has conducted a systematic and fair comparison of four neural network architectures, FNN, LSTM, GRU, and TCN, for estimating the SoC in lithium-ion batteries. In summary, the following task has been performed.

• It has evaluated and compared the predictive accuracy, generalization capability, and computational efficiency of FNN, LSTM, GRU, and TCN models using the same dataset and unified preprocessing methods.
• It has also investigated the impact of different network configurations and hyperparameters on model performance under varied test conditions, including multiple temperatures and dynamic drive cycles.
• It identified the optimal model architecture for SoC estimation based on test MSE, RMSE, convergence quality, runtime efficiency, and generalization gap.

The remainder of this work is organized as follows: Section 2 describes the data source, model architecture, and training configuration used for a comprehensive comparison of several neural network models (FNN, LSTM, GRU, and TCN) for SoC estimation. Section 3 presents detailed simulations, discussions, and the formulated findings summarized in tables. Finally, Section 4 concludes the paper and provides recommendations.

2. Methodology

This study compares the effectiveness of several neural network models: FNN, LSTM, GRU, and TCN, for estimating the SoC in lithium-ion batteries. All models were trained and evaluated using the same real-world dataset to ensure a fair and consistent comparison. The analysis is based exclusively on the McMaster University LG HG2 dataset, which contains high-resolution time-series data recorded under controlled thermal chamber conditions across a range of temperatures and operational modes.

2.1. Data Source and Preparation

The dataset was collected at McMaster University using a brand new 3Ah LG HG2 cell tested within an eight cubic feet. The thermal chamber is monitored using a Digatron Universal Battery Tester (Digatron Power Electronics, Aachen, Germany). Measurements were performed at six ambient temperatures: 40 °C, 25 °C, 10 °C, 0 °C, −10 °C, and −20 °C. Each test included multiple drive cycles (UDDS, HWFET, LA92, US06), hybrid pulse power characterizations (HPPC), and constant current discharges at various C-rates [28]. The dataset was provided in .mat format and directly imported into MATLAB (v2021s) without further cleaning or normalization, as the original research team had already preprocessed the data. For input features, combinations of voltage, current, power, amp-hours (Ah), watt-hours (Wh), and battery temperature were used.

2.2. Data Splitting Strategy

To evaluate generalization, the dataset was divided into training, validation, and testing sets. For models trained with LM and SCG, a 70/15/15 split was used. For BR, which does not require a validation set, an 85/0/15 split was applied. These splits were kept consistent across all architecture and neuron configurations after running multiple tests, and this setup provided the best overall model performance.

2.3. Model Architectures

Three model families were implemented: FNN, LSTM/GRU, and TCN. FNN models included Fitting, NAR, and NARX structures, each trained with LM, BR, and SCG algorithms using 10, 20, or 30 hidden neurons. All FNNs had a single tansig hidden layer and linear output. LSTM and GRU models used two stacked recurrent layers followed by a fully connected output, with windowed inputs (e.g., length 80, stride 20). They were trained using the Adam optimizer with tuned hyperparameters. The number of hidden units was aligned with the 20-neuron FNN for fairness. TCN replaced recurrence with 1D dilated causal convolutions using dilation rates from 1 to 16, residual blocks, ReLU activations, dropout, and causal padding. Input formatting for TCN matched that of LSTM and GRU.

2.4. Training Configuration

Each model was trained using one of three optimization algorithms, and the training was conducted in MATLAB. FNN models were developed using built-in functions such as fitnet, narnet, and narxnet. They were typically trained for up to 1000 epochs, with early stopping triggered by validation checks when applicable. BR-based FNNs trained to full completion without validation monitoring. In contrast, LSTM, GRU, and TCN models were implemented using custom code via the Deep Learning Toolbox and trained with manually selected epoch limits (e.g., 80 epochs), optimized for convergence and runtime. Learning rate, batch size, and other training parameters were tuned individually for each model family to achieve a balance between accuracy, stability, and efficiency.

2.5. Evaluation Metrics

Model performance was evaluated using several key metrics: MSE and RMSE were used to measure overall prediction accuracy. At the same time, the regression coefficient (R) assessed the correlation between predicted and actual SoC. The generalization gap, defined as the relative difference between test and training MSE, provided insight into overfitting or underfitting behavior. For FNN models, convergence quality was examined using the final gradient and damping factor (μ). Training time and compute cost were also recorded, normalized relative to the fastest configuration in each group. These metrics offered a comprehensive view of each model’s accuracy, efficiency, and generalization ability. All models were trained using identical input features, consistent preprocessing, and fixed data splits to ensure fair and unbiased comparisons.

3. Results and Discussion

3.1. FNN-Fitting

3.1.1. LM-Fitting-FNN

The 30-neuron LM-Fitting-FNN is the preferred model, as it delivers the lowest test error MSE of 3.6279 × 10⁻⁴ and the strongest generalization of −4.71%, despite requiring nearly three times the computational effort compared to the 10-neuron baseline. When computational resources are more constrained, the 20-neuron network offers a balanced alternative, achieving a 15.5% accuracy improvement over the 10-neuron model with only a moderate increase to 1.9× in runtime and resource usage. The 10-neuron configuration remains a practical choice when fast training and low computational cost are prioritized, exceeding the absolute minimum error as shown in Table 1 and Table 2.

Table 1. Performance metrics for the LM-Fitting-FNN.

Metric	10 Neurons	20 Neurons	30 Neurons
Epochs	1000	1000	1000
Elapsed Time (hh:mm:ss)	00:14:53	00:28:22	00:43:19
Training MSE	4.5405 × 10−4	3.9738 × 10−4	3.8074 × 10−4
Validation MSE	4.4420 × 10−4	3.8263 × 10−4	3.7359 × 10−4
Test MSE	4.5728 × 10−4	3.8637 × 10−4	3.6279 × 10−4
Training R	0.9980	0.9982	0.9983
Validation R	0.9980	0.9983	0.9983
Test R	0.9979	0.9983	0.9984
Final Gradient	7.52 × 10−6	0.00382	0.00195
Validation Checks	1	1	0
Damping factor (μ) at Stop	1.0 × 10−10	1.0 × 10−8	1.0 × 10−8

Table 2. Derived indicators for the LM-Fitting-FNN.

Metric	10 Neurons	20 Neurons	30 Neurons
Test MSE improvement vs. 10 N	–	+15.5%	+20.7%
Test MSE improvement vs. 20 N	−18.4%	–	+6.10%
Generalization gap (Test − Train)/Train	+0.71%	−2.77%	−4.71%
Convergence signal (low grad & μ)	Very strong	Moderate	Strong
Compute cost (relative)	1×	1.9×	2.9×
Wall time per epoch (s)	0.90	1.70	2.60
Validation/Training MSE ratio	0.98	0.96	0.98
Training RMSE	2.130 × 10−2	1.993 × 10−2	1.951 × 10−2
Validation RMSE	2.107 × 10−2	1.956 × 10−2	1.933 × 10−2
Test RMSE	2.139 × 10−2	1.964 × 10−2	1.905 × 10−2

3.1.2. BR-Fitting-FNN

The 30-neuron BR-Fitting-FNN model achieved the lowest MSE test error of 3.6074 × 10⁻⁴, among the three configurations, its improvement over the 20-neuron version is minimal at only +0.09%. It slightly outperforms by eliminating the positive generalization gap, but this comes at a computational cost that is 1.8 times higher. Therefore, the 20-neuron BR Fitting FNN offers the best balance between accuracy and efficiency. The 10-neuron model remains a practical choice for resource-constrained situations where a modest increase in error is acceptable in exchange for faster training and reduced computational requirements as shown in Table 3 and Table 4.

Table 3. Performance metrics for the BR-Fitting-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	1000	1000	1000
Elapsed time (hh:mm:ss)	00:15:16	00:28:42	00:50:18
Training MSE	4.2820 × 10−4	3.5065 × 10−4	3.6173 × 10−4
Validation MSE	NaN	NaN	NaN
Test MSE	4.3939 × 10−4	3.6105 × 10−4	3.6074 × 10−4
Training R	0.9981	0.9984	0.9984
Validation R	NaN	NaN	NaN
Test R	0.9980	0.9984	0.9984
Final gradient	8.95 × 10−7	6.92 × 10−5	1.24 × 10−3
Damping factor (μ)	50	50	50
Effective Param (Neff)	69.4	140	207
Sum squared Param	2.73 × 105	6.91 × 104	1.92 × 104

Table 4. Derived indicators for the BR-Fitting-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	+17.8%	+17.9%
Test MSE improvement vs. 20 N	−21.7%	–	+0.09%
generalization gap (Test − Train)/Train	+2.6%	+3.0%	−0.27%
Convergence signal (final grad & μ)	Very strong	Strong	Moderate
Compute cost (relative)	1×	1.9×	3.3×
Wall-time per epoch (s)	0.92	1.72	3.02
Validation/Training MSE ratio	N/A	N/A	N/A
Training RMSE	2.07 × 10−2	1.87 × 10−2	1.90 × 10−2
Validation RMSE	N/A	N/A	N/A
Test RMSE	2.10 × 10−2	1.90 × 10−2	1.90 × 10−2

3.1.3. SCG-Fitting-FNN

The 30 neuron SCG Fitting FNN delivers the best accuracy of 5.363 × 10⁻⁴ and acceptable generalization gap of −2.4%, but it also requires the most time and resources reported in Table 5 and Table 6. The 20 neuron model is a practical compromise, offering most of the accuracy gains of the 30 neuron network while keeping computational demands modest. The 10 neuron configuration remains suitable for quick training and minimal cost when a slightly higher error can be tolerated.

Table 5. Performance metrics for the SCG-Fitting-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs (stopped)	663	963	882
Elapsed time (hh:mm:ss)	00:06:52	00:08:27	00:10:50
Training MSE	5.8790 × 10−4	5.4720 × 10−4	5.4970 × 10−4
Validation MSE	5.7982 × 10−4	5.3895 × 10−4	5.2370 × 10−4
Test MSE	5.7883 × 10−4	5.4278 × 10−4	5.3632 × 10−4
Training R	0.9973	0.9975	0.9975
Validation R	0.9974	0.9976	0.9976
Test R	0.9974	0.9976	0.9976
Final gradient	4.66 × 10−5	5.43 × 10−5	1.15 × 10−4
Validation checks	6	6	6

Table 6. Derived indicators for the SCG-Fitting-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Indicator	10 neurons	20 neurons	30 neurons
Test MSE improvement vs. 10 N	–	+6.2%	+7.3%
Test MSE improvement vs. 20 N	−6.6%	–	+1.2%
Generalization gap (Test − Train)/Train	−1.5%	−0.8%	−2.4%
Convergence signal (final grad)	Strong	Strong	Moderate
Compute cost (relative)	1×	1.2×	1.6×
Wall-time per epoch (s)	0.62	0.53	0.74
Validation/Training MSE ratio	0.986	0.985	0.953
Training RMSE	2.42 × 10−2	2.34 × 10−2	2.35 × 10−2
Validation RMSE	2.41 × 10−2	2.32 × 10−2	2.29 × 10−2

3.1.4. Optimal Fitting-FNN Model

The 20-neuron BR-Fitting-FNN is the best overall performer, achieving the lowest test MSE of 3.6105 × 10⁻⁴ and a top R value of 0.9984, with a small +3.0% generalization gap and moderate computational cost. In addition, The MSE curves of the training and test drop rapidly within the first 50 epochs, decreasing by more than three orders of magnitude as shown in Figure 1. After around epoch 200, the curves start to flatten, indicating that further training leads to only minor improvements. By epoch 700, the training MSE reaches its minimum value of 3.5065 × 10⁻⁴ and stays stable through to epoch 1000. The test MSE shows a similar trend, and the “Best” marker appears at the point where both curves align. Since the gap between training and test errors remains small and does not increase, there is no indication of overfitting. Each subplot in Figure 2 shows how well the predictions match the actual values. The blue points closely follow the red 1:1 reference line. For both the training and test data, the correlation is R = 0.9983. The combined plot also shows a slope near one and an intercept around +2 × 10⁻³, which means there is almost no systematic bias across the full SoC range. The tight clustering of points around the line suggests that the remaining variation is likely due to noise in the data rather than model error.

Figure 1. Performance curve for the 20-neuron BR-Fitting-FNN.

Figure 2. Regression plots for the 20-neuron BR-Fitting-FNN.

3.2. FNN-Time Series

All runs use the NAR topology (only past y(t)), time delay = 2. Three algorithms were tested: LM, BR, and SCG with 10, 20, and 30 hidden neurons each. The goal is to select the optimal neuron size within each optimizer and then determine the best optimizer overall.

3.2.1. LM NAR-FNN

The 10-neuron LM-NAR-FNN is the most effective configuration based on test MSE, convergence behavior, and computational efficiency. It achieved the lowest test error of 7.9666 × 10⁻⁹ by a large margin, demonstrated clean convergence, and required the least training time per epoch. Although the strongly negative generalization gap of −99.7%, suggests the test fold may be easier than the training data, the results clearly favor this configuration under current partitions. The 30-neuron model offers reliable validation accuracy and moderate generalization at a higher computational cost. In contrast, the 20-neuron network shows signs of overfitting and is not recommended for deployment without further refinement as shown in Table 7 and Table 8.

Table 7. Performance metrics for the LM-NAR-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	864	1000	688
Elapsed Time (hh:mm:ss)	00:05:19	00:15:25	00:13:45
Training MSE	2.9584 × 10−6	1.2656 × 10−6	2.8833 × 10−6
Validation MSE	1.5498 × 10−7	1.5534 × 10−7	1.2870 × 10−7
Test MSE	7.9666 × 10−9	8.0013 × 10−6	4.1406 × 10−7
Training R	1.0000	1.0000	1.0000
Validation R	1.0000	1.0000	1.0000
Test R	1.0000	1.0000	1.0000
Final Gradient	9.97 × 10−8	7.07 × 10−7	9.95 × 10−8
Validation Checks	0	0	0
Damping factor (μ) at Stop	1 × 10−8	1 × 10−8	1 × 10−8

Table 8. Derived indicators for the LM-NAR-FNN.

Indicator	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	−100,335.6%	−5097.4%
Test MSE improvement vs. 20 N	–	–	+94.8%
Generalization gap (Test − Train)/Train	−99.7%	+532.2%	−85.6%
Convergence signal (low grad & μ)	Very strong	Very strong	Very strong
Compute cost (relative)	1×	2.90×	2.59×
Wall-time per epoch (s)	0.37	0.93	1.20
Validation/Training MSE ratio	0.052	0.123	0.045
Training RMSE	1.72 × 10−3	1.12 × 10−3	1.70 × 10−3
Validation RMSE	3.94 × 10−4	3.94 × 10−4	3.59 × 10−4
Test RMSE	8.93 × 10−5	2.83 × 10−3	6.43 × 10−4

3.2.2. BR NAR-FNN

The 30-neuron BR-NAR-FNN is the best-performing configuration when predictive accuracy is prioritized, considering test set accuracy, generalization behavior. It consistently achieves the lowest MSE error metrics of 1.3550 × 10⁻⁷ and demonstrates strong convergence of −94.4%, although this comes with the highest training time and a 4.8× compute cost relative to the baseline as presented in Table 9 and Table 10. The 10-neuron model provides a compelling alternative for scenarios where efficiency is critical, offering competitive accuracy at the lowest computational expense. In contrast, the 20-neuron configuration exhibits substantial overfitting and poor generalization to unseen data, making it unsuitable.

Table 9. Performance metrics for the BR-NAR-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	560	261	775
Elapsed Time (hh:mm:ss)	00:03:36	00:04:10	00:17:13
Training MSE	2.4315 × 10−6	1.5454 × 10−6	2.4408 × 10−6
Validation MSE	N/A	N/A	N/A
Test MSE	1.8353 × 10−7	5.4863 × 10−6	1.3550 × 10−7
Training R	1.0000	1.0000	1.0000
Validation R	N/A	N/A	N/A
Test R	1.0000	1.0000	1.0000
Final Gradient	9.94 × 10−8	9.94 × 10−8	9.93 × 10−8
Damping factor (μ) at Stop	5 × 10+3	5 × 10+3	5 × 10+3
Effective Param	16.9	14	18.7
Sum Squared Parameters	23.8	10.4	22.8

Table 10. Derived indicators for the SCG-Fitting-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	−2889%	+26.2%
Test MSE improvement vs. 20 N	+96.65%	–	+97.5%
Generalization gap (Test − Train)/Train	−92.5%	+255%	−94.4%
Convergence signal (low grad & μ)	Very strong	Very strong	Very strong
Compute cost (relative)	1×	1.2×	4.8×
Wall-time per epoch (s)	0.39	0.96	1.33
Validation/Training MSE ratio	N/A	N/A	N/A
Training RMSE	1.56 × 10−3	1.24 × 10−3	1.56 × 10−3
Test RMSE	4.28 × 10−4	2.34 × 10−3	3.68 × 10−4

3.2.3. SCG NAR-FNN

The 10-neuron SCG-NAR-FNN is the best-performing configuration, considering test MSE as the primary accuracy metric. It achieved the lowest test MSE of 6.4303 × 10⁻⁷ with the smallest compute cost and stable convergence. Although the strongly negative generalization gap of −82.9% suggests the test fold may be easier than the training set. The 30-neuron model is a reasonable alternative, yielding a test MSE of 9.4941 × 10⁻⁷, which is better than the 20-neuron model by 19.4% but still falling short of the 10-neuron result and requiring substantially more computation. The 20-neuron model shows the worst test MSE, indicating the weakest generalization and making it unsuitable without improved regularization or adjusted data splits.

3.2.4. Optimal NAR-FNN Model

Across all NAR-FNN configurations and optimizers summarized in Table 7, Table 8, Table 9, Table 10, Table 11 and Table 12, the 10-neuron LM-NAR-FNN stands out as the best-performing model. It achieved the lowest test MSE of 7.9666 × 10⁻⁹, outperforming all BR and SCG variants by at least an order of magnitude. Convergence was stable, with a final gradient of 9.97 × 10⁻⁸ and a fixed damping factor μ = 1 × 10⁻⁸. In terms of efficiency, this model also recorded the fastest training time per epoch at 0.37 s and the lowest compute cost at 1× relative to the baseline. While the generalization gap was strongly negative at 99.7%, suggesting the test fold was much easier than the training set, this pattern was observed in other high-performing models as well. In addition, the MSE curves for the training, validation, and test sets drop rapidly during the first 100 epochs, reducing the error by more than three orders of magnitude before flattening with only minor changes as shown in Figure 3. The best validation performance occurs at epoch 864 with MSE = 1.5498 × 10⁻⁷. The generalization gap, calculated as (Test − Train)/Train, is −72.6%, indicating the test error is lower than the training error.

Table 11. Performance metrics for the SCG-NAR-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	300	701	1000
Elapsed Time (hh:mm:ss)	00:01:22	00:04:49	00:09:14
Training MSE	3.7517 × 10−6	4.2263 × 10−6	3.4698 × 10−6
Validation MSE	5.8218 × 10−6	9.7878 × 10−7	1.1954 × 10−6
Test MSE	6.4303 × 10−7	1.1785 × 10−6	9.4941 × 10−7
Training R	1.0000	1.0000	1.0000
Validation R	1.0000	1.0000	1.0000
Test R	1.0000	1.0000	1.0000
Final Gradient	8.71 × 10−6	6.92 × 10−6	6.80 × 10−6
Validation Checks	6	6	1

Table 12. Derived indicators for the SCG-NAR-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	−83.3%	−47.6%
Test MSE improvement vs. 20 N	+45.4%	–	+19.4%
generalization gap (Test − Train)/Train	−82.9%	−72.1%	−72.6%
Convergence signal (final grad)	Strong	Strong	Strong
Compute cost (relative)	1×	3.5×	6.8×
Wall-time per epoch (s)	0.27	0.41	0.55
Validation/Training MSE ratio	1.55	0.23	0.34
Training RMSE	1.94 × 10−3	2.06 × 10−3	1.86 × 10−3
Validation RMSE	2.41 × 10−3	9.89 × 10−4	1.09 × 10−3
Test RMSE	8.02 × 10−4	1.09 × 10−3	9.74 × 10−4

Figure 3. Performance curve for the 10-neuron LM-NAR-FNN.

The time-series response plot in Figure 4 shows the network outputs closely following the targets over the whole horizon, with minor, bounded deviations. The error trace remains tight around zero with occasional brief spikes. This behavior indicates sound amplitude and phase tracking with limited drift or accumulation of error, in line with the overall accuracy and convergence characteristics.

Figure 4. Time-series response and errors for the 10-neuron LM-NAR-FNN.

3.3. Nonlinear I/O FNN

All runs use the Nonlinear I/O topology. Three algorithms were tested: LM, BR, and SCG, with 10, 20, and 30 hidden neurons, respectively. The goal is to select the optimal neuron size within each optimizer and then determine the best optimizer overall.

3.3.1. LM Nonlinear I/O FNN

The 20-neuron LM-Nonlinear I/O-FNN is the best-performing configuration as per summarized results reported in Table 13 and Table 14. It delivers the lowest test MSE of 3.8568 × 10⁻⁴, the strongest convergence with the smallest final gradient, and maintains balanced behavior across training and validation sets. Although the generalization gap is slightly positive at 6.10%, it remains within acceptable bounds. The 10-neuron model offers the most computationally efficient option, completing training quickly with minimal compute cost and showing a near-zero generalization gap. Still, it falls short in predictive accuracy by 26.9% compared to the 20-neuron result. The 30-neuron model exhibits significant overfitting, with a large generalization gap of 73.7% and the worst test MSE, despite achieving the lowest training and validation errors. For this dataset and partition, the 20-neuron configuration offers the best balance of accuracy and convergence stability. The 10-neuron model is suitable when efficiency is the priority, while the 30-neuron variant should be avoided unless stronger regularization or revised data splits are applied.

Table 13. Performance metrics for the LM-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	198	1000	742
Elapsed Time (hh:mm:ss)	00:02:52	00:23:24	00:21:25
Training MSE	5.2588 × 10−4	3.6352 × 10−4	3.4649 × 10−4
Validation MSE	5.5412 × 10−4	3.6613 × 10−4	3.3565 × 10−4
Test MSE	5.2730 × 10−4	3.8568 × 10−4	6.0172 × 10−4
Training R	0.9976	0.9984	0.9984
Validation R	0.9975	0.9983	0.9985
Test R	0.9976	0.9983	0.9973
Final Gradient	0.000322	3.09 × 10−5	0.000366
Damping factor (μ) at Stop	1 × 10−6	1 × 10−8	1 × 10−8
Validation Checks	6	1	6

Table 14. Derived indicators for the LM-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	+26.9%	−14.1%
Test MSE improvement vs. 20 N	−36.7%	–	−56.0%
Generalization gap (Test − Train)/Train	+0.27%	+6.10%	+73.7%
Convergence signal (final grad & μ)	Moderate	Strong	Moderate
Compute cost (relative)	1.00×	8.16×	7.47×
Wall-time per epoch (s)	0.87	1.40	1.73
Validation/Training MSE ratio	1.05	1.01	0.97
Training RMSE	2.29 × 10−2	1.91 × 10−2	1.86 × 10−2
Validation RMSE	2.35 × 10−2	1.91 × 10−2	1.83 × 10−2
Test RMSE	2.30 × 10−2	1.96 × 10−2	2.45 × 10−2

3.3.2. BR Nonlinear I/O FNN

The 30-neuron BR-Nonlinear I/O-FNN is the best-performing model, offering the lowest MSE test error 3.7017 × 10⁻⁴, a 19.9% improvement over 10 neurons and 0.6% over the 20-neuron setup, robust convergence, and effective regularization as detailed in Table 15 and Table 16. The 20-neuron model is a close second, providing nearly equal accuracy at lower cost, making it an appealing compromise when efficiency matters. The 10-neuron model remains the most economical and converges cleanly but shows higher test error, making it suitable only for scenarios where speed and low computational load are prioritized over accuracy.

Table 15. Performance metrics for the BR-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	743	1000	1000
Elapsed Time (hh:mm:ss)	00:10:08	00:21:54	00:33:09
Training MSE	4.6582 × 10−4	3.6303 × 10−4	3.6780 × 10−4
Validation MSE	NaN	NaN	NaN
Test MSE	4.6236 × 10−4	3.7233 × 10−4	3.7017 × 10−4
Training R	0.9979	0.9984	0.9983
Validation R	NaN	NaN	NaN
Test R	0.9979	0.9984	0.9983
Final Gradient	9.54 × 10−8	0.0011	0.000204
Damping factor (μ) at Stop	5	5	50
Effective Param	117	227	326
Sum Squared Param	1.61 × 105	1.59 × 104	8.22 × 103

Table 16. Derived indicators for the BR-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	+19.5%	+19.9%
Test MSE improvement vs. 20 N	−24.2%	–	+0.6%
Generalization gap (Test − Train)/Train	−0.7%	+2.6%	+0.6%
Convergence signal (final grad & μ)	Very strong	Moderate	Strong
Compute cost (relative)	1.00×	2.16×	3.27×
Wall time per epoch (s)	0.82	1.31	1.99
Validation/Training MSE ratio	N/A	N/A	N/A
Training RMSE	2.16 × 10−2	1.91 × 10−2	1.92 × 10−2
Validation RMSE	N/A	N/A	N/A
Test RMSE	2.15 × 10−2	1.93 × 10−2	1.92 × 10−2

3.3.3. SCG Nonlinear I/O FNN

The 30-neuron SCG Nonlinear I/O FNN is the best-performing configuration, achieving the lowest test MSE of 5.3874 × 10⁻⁴, with stable convergence and reliable generalization of +0.57% as seen in Table 17 and Table 18. The 20-neuron model offers no meaningful improvement over the 10-neuron variant while requiring over three times the compute, making it inefficient. The 10-neuron model remains attractive when low compute and fast training are the priority, but it trails the 30-neuron accuracy by a small yet consistent margin. Therefore, select 30 neurons for best accuracy in the SCG Nonlinear I/O FNN group, use 10 neurons when efficiency matters most, and avoid 20 neurons unless architecture constraints demand it.

Table 17. Performance metrics for the SCG-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	467	1000	760
Elapsed Time (hh:mm:ss)	00:02:12	00:07:20	00:07:36
Training MSE	6.5299 × 10−4	5.5455 × 10−4	5.3566 × 10−4
Validation MSE	6.4763 × 10−4	5.5426 × 10−4	5.3474 × 10−4
Test MSE	5.5687 × 10−4	5.5511 × 10−4	5.3874 × 10−4
Training R	0.9971	0.9975	0.9976
Validation R	0.9971	0.9975	0.9976
Test R	0.9970	0.9975	0.9976
Final Gradient	0.000123	0.000134	0.000159
Validation Checks	6	0	6

Table 18. Derived indicators for the SCG-Nonlinear I/O-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	+0.3%	+3.3%
Test MSE improvement vs. 20 N	−0.3%	–	+2.9%
Generalization gap (Test − Train)/Train	−14.7%	+0.10%	+0.57%
Convergence signal (final gradient)	Strong	Strong	Strong
Compute cost (relative)	1.00×	3.33×	3.45×
Wall-time per epoch (s)	0.28	0.44	0.60
Validation/Training MSE ratio	0.992	0.999	0.998
Training RMSE	2.56 × 10−2	2.35 × 10−2	2.31 × 10−2
Validation RMSE	2.54 × 10−2	2.35 × 10−2	2.31 × 10−2
Test RMSE	2.36 × 10−2	2.36 × 10−2	2.32 × 10−2

3.3.4. Optimal Nonlinear I/O FNN Model

Across all tested configurations and training algorithms, the 30-neuron BR–Nonlinear I/O FNN stands out as the optimal model. It achieves the lowest test MSE at 3.7017 × 10⁻⁴, demonstrates the highest regularization efficiency with a sum of squared parameters equal to 8.22 × 10³, and maintains high predictive consistency with a test R value of 0.9983 and a generalization gap of +0.6%. Convergence is strong, and although its compute cost is the highest at 3.27 times the baseline. The performance curve in Figure 5, which plots MSE against epochs, shows a sharp drop in error during the initial training phase, followed by a prolonged plateau where both training and test losses settle in the mid-10⁻⁴ range. The best training performance marker at 3.66 × 10⁻⁴ aligns with the final training MSE as shown in Table 15, and the close alignment of the test curve reflects the small generalization gap of +0.6% as shown in Table 16. The absence of a validation curve is expected for Bayesian Regularization. The continued overlap between the training and test lines, along with the flattening trend in later epochs, suggests that overfitting does not occur and that further training yields only minor improvements. The time-series response plot in Figure 6 shows that the predicted outputs closely follow the target values across the whole-time horizon, with only minor and bounded deviations. In the lower panel, the error stays tightly clustered around zero, with occasional short spikes that do not accumulate over time. This pattern aligns with the low test MSE of 3.7017 × 10⁻⁴ reported in Table 15, the small positive generalization gap of +0.6% shown in Table 16, and the consistently high R = 0.9983 values across all data splits in Table 15. Overall, the model effectively captures the main system dynamics with accurate amplitude and phase tracking. The remaining differences are more likely due to measurement noise or rare outliers rather than consistent model error.

Figure 5. Performance curve for the 30-neuron BR-Nonlinear I/O-FNN.

Figure 6. Time-series response and errors for the 30-neuron BR-Nonlinear I/O-FNN.

3.4. NARX FNN

All runs use the NARX topology. Three algorithms were tested: LM, BR, and SCG, with 10, 20, and 30 hidden neurons each. The goal is to select the optimal neuron size within each optimizer and then determine the best optimizer overall.

3.4.1. LM NARX FNN

The 20-neuron LM-NARX FNN is the optimal configuration, offering the lowest test MSE of 1.6210 × 10⁻⁷, minimal of just 0.6× relative to the baseline, and the shortest training time, as seen in Table 19 and Table 20. The 10-neuron model ranks second, providing good accuracy at baseline cost and solid convergence. The 30-neuron model, while showing excellent internal optimization, fails to generalize effectively and incurs the highest computational load.

Table 19. Performance metrics for the LM-NARX-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	251	85	893
Elapsed Time (hh:mm:ss)	00:03:36	00:02:15	00:38:00
Training MSE	2.9175 × 10−6	2.9148 × 10−6	2.9058 × 10−6
Validation MSE	1.7787 × 10−7	1.8765 × 10−7	4.5288 × 10−9
Test MSE	1.8645 × 10−7	1.6210 × 10−7	4.7355 × 10−7
Training R	1.0000	1.0000	1.0000
Validation R	1.0000	1.0000	1.0000
Test R	1.0000	1.0000	1.0000
Final Gradient	3.32 × 10−5	1.01 × 10−5	9.88 × 10−8
Damping factor (μ) at Stop	1 × 10−8	1 × 10−8	1 × 10−9

Table 20. Derived indicators for the LM-NARX-FNN.

Metric	10 Neurons	20 Neurons	30 Neurons
Test MSE improvement vs. 10 N	–	+13.1%	−154.0%
Test MSE improvement vs. 20 N	−15.0%	–	−192.1%
Generalization gap (Test − Train)/Train	−93.6%	−94.4%	−83.7%
Convergence signal (final gradient)	Strong	Strong	Very strong
Compute cost (relative)	1×	0.6×	10.6×
Wall-time per epoch (s)	0.86	1.59	2.55
Validation/Training MSE ratio	0.061	0.064	0.0016
Training RMSE	1.71 × 10−3	1.71 × 10−3	1.70 × 10−3
Validation RMSE	4.22 × 10−4	4.33 × 10−4	6.73 × 10−5
Test RMSE	–	+13.1%	−154.0%

3.4.2. BR NARX FNN

The 10-neuron BR-NARX-FNN is the top performer in terms of test MSE, as shown in Table 21 and Table 22. It achieves the best accuracy of 1.3028 × 10⁻⁷, clean convergence by a final gradient of 9.67 × 10⁻⁸, and effective shrinkage at the lowest computational cost. The 20-neuron variant is more expensive and performs worse, whereas the 30-neuron setting overfits drastically and is not viable for generalization. For current data splits, 10 neurons are preferred.

Table 21. Performance metrics for the BR-NARX-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	743	985	653
Elapsed Time (hh:mm:ss)	00:09:56	00:36:36	00:26:16
Training MSE	2.4371 × 10−6	2.4125 × 10−6	1.0775 × 10−6
Validation MSE	NaN	NaN	NaN
Test MSE	1.3028 × 10−7	3.1137 × 10−7	7.8732 × 10−6
Training R	1.0000	1.0000	1.0000
Validation R	NaN	NaN	NaN
Test R	1.0000	1.0000	1.0000
Final Gradient	9.67 × 10−8	9.7 × 10−8	9.49 × 10−8
Damping factor (μ) at Stop	500	5 × 104	5 × 104
Effective Param	103	100	102
Sum Squared Param	9.95	18.5	14.2

Table 22. Derived indicators for the BR-NARX-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	−139.0%	−5943.3%
Test MSE improvement vs. 20 N	+58.2%	–	−2428.6%
Generalization gap (Test − Train)/Train	−94.7%	−87.1%	+630.7%
Convergence signal (final gradient)	Strong	Strong	Strong
Compute cost (relative)	1.0×	3.7×	2.6×
Wall-time per epoch (s)	0.80	2.23	2.41
Validation/Training MSE ratio	N/A	N/A	N/A
Training RMSE	1.56 × 10−3	1.55 × 10−3	1.04 × 10−3
Validation RMSE	N/A	N/A	N/A
Test RMSE	3.61 × 10−4	5.58 × 10−4	2.81 × 10−3

3.4.3. SCG-NARX-FNN

The 10-neuron SCG-NARX-FNN remains the best configuration as per summarized results reported in Table 23 and Table 24, and considering test MSE as the primary performance metric. It delivers the lowest test error of 7.4190 × 10⁻⁷, fastest training time of 311 s, and lowest computing cost of 1.00× relative to baseline, while maintaining strong convergence with final gradient of 1.13 × 10⁻⁵. By contrast, both the 20- and 30-neuron models show inferior test accuracy and substantially higher computational demands. Under the current partition, the 10-neuron model is clearly optimal, balancing accuracy, efficiency, and convergence most effectively.

Table 23. Performance metrics for the SCG-NARX-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Epochs	1000	1000	926
Elapsed Time (hh:mm:ss)	00:05:11	00:07:58	00:13:42
Training MSE	3.2756 × 10−6	3.5739 × 10−6	4.5603 × 10−6
Validation MSE	5.1115 × 10−7	9.9007 × 10−7	1.7628 × 10−6
Test MSE	7.4190 × 10−7	9.8161 × 10−7	1.7140 × 10−6
Training R	1.0000	1.0000	1.0000
Validation R	1.0000	1.0000	1.0000
Test R	1.0000	1.0000	1.0000
Final Gradient	1.13 × 10−5	1.44 × 10−5	2.88 × 10−5
Validation Checks	0	0	6

Table 24. Derived indicators for the SCG-NARX-FNN.

Metric	10-Neuron	20-Neuron	30-Neuron
Test MSE improvement vs. 10 N	–	−32.3%	−131%
Test MSE improvement vs. 20 N	+24.4%	–	−74.6%
Generalization gap (Test − Train)/Train	−77.4%	−72.5%	−62.4%
Convergence signal (final gradient)	Strong	Strong	Moderate
Compute cost (relative)	1.00×	1.54×	2.64×
Wall-time per epoch (s)	0.311	0.478	0.888
Validation/Training MSE ratio	0.156	0.277	0.387
Training RMSE	1.81 × 10−3	1.89 × 10−3	2.14 × 10−3
Validation RMSE	7.15 × 10−4	9.95 × 10−4	1.33 × 10−3
Test RMSE	8.61 × 10−4	9.91 × 10−4	1.31 × 10−3

3.4.4. Optimal NARX FNN Model

Across all NARX-FNN configurations tested, the 10-neuron BR-NARX-FNN emerges as the optimal model, achieving the lowest test MSE of 1.3028 × 10⁻⁷ and a perfect test R of 1.0000, as shown in Table 21 and Table 22. It also delivers robust convergence, with a final gradient of 9.67 × 10⁻⁸ and μ value of 500, a strong weight shrinkage signal with sum squared parameters of 9.95, and the lowest compute cost of 1.0× relative to baseline, completing training in under 10 min. The performance plot in Figure 7 shows a sharp drop in error during the early phase of training, followed by a long stretch where the error remains nearly flat. The best training performance is reached at epoch 743 with an MSE of 2.4371 × 10⁻⁶, which matches the value reported in Table 21. The test curve stays consistently below the training curve. It remains steady throughout, which corresponds to the very low test MSE of 1.3028 × 10⁻⁷ and the intensely negative generalization gap of −94.7% shown in Table 22. This behavior typically suggests that the test set is either easier or contains less noise, rather than the model genuinely outperforming its training. The flattening of both curves in the later stages indicates that further training brings slight improvement, which is in line with the strong convergence behavior reported.

Figure 7. Performance curve for the 10-neuron BR-NARX-FNN.

The time-series response in Figure 8 shows that the model’s output follows the target values closely across the entire time range, with only minor and controlled deviations. In the lower panel, the residuals stay centered around zero and show just a few brief spikes, without any signs of drift or buildup over time. This matches the low test MSE of 1.3028 × 10⁻⁷ and the strongly negative generalization gap of −94.7%, as shown in Table 21 and Table 22. The model tracks both the size and timing of the target signal well. The small differences are likely due to measurement noise or occasional outliers, rather than any consistent model error.

Figure 8. Time-series response and errors for the 10-neuron BR-NARX-FNN.

3.5. GRU Model

As shown in Table 25, the dataset was split into three parts: training with about 1.88 million samples, validation with around 0.40 million, and test with approximately 0.40 million. The error metrics are very similar across all splits, with MSE values around 2.3 to 2.4 × 10⁻³ and RMSE close to 4.8 × 10⁻². The correlation values are also consistent, with R near 0.998 and R² around 0.995. This strong agreement across splits suggests that the GRU model generalizes well and does not overfit, maintaining reliable performance on unseen data.

Table 25. Data split and basic performance statistics for the GRU model.

Data Split	Samples	MSE	RMSE	R	R2
Training	1,875,700	2.3414 × 10−3	4.8388 × 10−2	0.99768	0.99536
Validation	401,920	2.2780 × 10−3	4.7729 × 10−2	0.99774	0.99546
Test	401,920	2.3555 × 10−3	4.8534 × 10−2	0.99768	0.99537

Table 26 presents the architectural and training settings used for the GRU model. The network includes two GRU layers with 256 and 128 units, followed by a fully connected layer. This setup allows the model to capture temporal patterns over a window of 80 samples, with a stride of 20, resulting in 75% overlap. Training uses the Adam optimizer, starting with a learning rate of 0.002, which is reduced by a factor of 0.2 after epoch 40. A global gradient clipping threshold of 5 is applied to improve training stability, and dropout at 10% helps prevent overfitting. Input features are standardized using Z-score normalization, while the targets are scaled to the range from −1 to 1. Early stopping with a patience of 10 epochs selected the final model at epoch 17, keeping the total training time around 59 min.

Table 26. Hyperparameters and training settings for the GRU model.

Parameter	Value
Model Name	GRU
Architecture	2 × GRU (OutputMode = sequence) + FC
Window Length	80
Stride	20
Max Epochs	80
Best Epoch	17
Batch Size	128
Optimizer	Adam
Initial Learning Rate	0.0020
LR Drop Epoch	40
LR Drop Factor	0.2000
Gradient Clip (Global Norm)	5
Validation Every (iters)	50
Dropout Rate	0.1000
Hidden Units (Layer 1)	256
Hidden Units (Layer 2)	128
Input Normalization	Z-score
Target Scaling	Linear to [−1, 1]
Early Stop Patience	10
Best Validation MSE	2.3 × 10−3
Training Time	3534.0 s (~58.9 min)

Figure 9 shows the MSE plotted against epochs. The error drops sharply during the first few epochs and then stabilizes, fluctuating within a narrow range of about 2.3 to 3.2 × 10⁻³. The lowest validation MSE is reached at epoch 17, which matches the point chosen by early stopping. The training, validation, and test curves remain very close to each other, indicating a minimal generalization gap and consistent performance across all data splits.

Figure 9. Performance curve for the GRU model.

Figure 10 shows that the gradient magnitudes decrease from 2.7 × 10⁻⁴ to 0.5 × 10⁻⁴ and then stabilize. This pattern confirms smooth convergence, as the updates become smaller over time without any late spikes. The steady decline suggests that the learning rate schedule and gradient clipping are working effectively to maintain stable and efficient training.

Figure 10. Gradient curve for GRU model.

In Figure 11, the predicted SoC values align closely with the actual values along the 1:1 reference line, showing strong agreement between the model outputs and targets. The high R² value of approximately 0.995 and low RMSE of around 0.0477 confirm that the predictions are accurate and largely unbiased. The few outliers that appear match the edge cases previously seen in the error histogram, further supporting the model’s consistent performance across most of the data.

Figure 11. Regression plot for the GRU model.

3.6. LSTM Model

Table 27 shows the number of samples in each subset, with about 1.88 million for training and around 0.40 million each for validation and test. The error values across the three splits are very close, with MSE ranging from approximately 2.44 to 2.56 × 10⁻³ and RMSE between 4.94 and 5.06 × 10⁻². Correlation values are also high and consistent, with R around 0.9975 and R² near 0.995. This close agreement shows that the LSTM model performs similarly on both seen and unseen data, indicating strong generalization and little to no overfitting.

Table 27. Data split and basic performance statistics for the LSTM model.

Data Split	Samples	MSE	RMSE	R	R2
Training	1,875,700	2.5589 × 10−3	5.0586 × 10−2	0.99746	0.99493
Validation	401,920	2.4395 × 10−3	4.9391 × 10−2	0.99757	0.99514
Test	401,920	2.5139 × 10−3	5.0139 × 10−2	0.99753	0.99507

Table 28 outlines the full training setup. The network architecture includes two LSTM layers with 256 and 128 hidden units, using “OutputMode = sequence”, followed by a fully connected layer. A window length of 80 and a stride of 20 (75% overlap) increases the temporal context and expands the number of training windows. The model is trained using the AdamW optimizer, which combines Adam with decoupled L2 weight decay set to 1.0 × 10⁻³. The learning rate starts at 0.002 and decreases using cosine decay to a minimum factor of 0.005. Gradient clipping with a global norm of 5 helps prevent exploding gradients, while a dropout rate of 10% reduces the risk of overfitting. Inputs are normalized using Z scores, and targets are scaled linearly to the range from −1 to 1. Early stopping with a patience of 10 epochs selected the best model at epoch 60, where the validation MSE reached approximately 2.4 × 10⁻⁴. Total training time was about 2.3 h. These settings together help ensure stable training and reproducibility.

Table 28. Hyperparameters and training settings for the LSTM model.

Parameter	Value
Model Name	LSTM
Architecture	2×LSTM (OutputMode = sequence) + FC
Window Length	80
Stride	20
Max Epochs	80
Best Epoch	60
Batch Size	128
Optimizer	AdamW (Adam + decoupled L2)
Initial Learning Rate	0.0020
LR Min Factor (Cosine Decay)	0.0500
Cosine LR?	Yes
Weight Decay	1.0 × 10−4
Gradient Clip (Global Norm)	5
Validation Every (iters)	50
Dropout Rate	0.1000
Hidden Units (Layer 1)	256
Hidden Units (Layer 2)	128
Input Normalization	Z-score
Target Scaling	Linear to [−1, 1]
Early Stop Patience	10
Best Validation MSE	2.4 × 10−3
Training Time	8262.7 s (~2.3 h)

In Figure 12, the error drops steeply during the first 10 epochs, then fluctuates within a narrow range between approximately 2.4 and 3.5 × 10⁻³. A small spike around epoch 37 reflects temporary mini-batch variability rather than a structural problem. The lowest validation MSE appears at epoch 60, which matches the point selected by early stopping. The training, validation, and test curves stay closely grouped throughout, indicating a minimal generalization gap and confirming that the model has learned temporal features consistently across all data splits.

Figure 12. Performance curve for the LSTM model.

Figure 13 presents the LSTM validation regression, where predicted SoC values are plotted against the actual SoC values. The points cluster closely around the 1:1 reference line, with an R² of 0.995 and an RMSE of 0.0494. There is a slight increase in spread near the boundaries of the range, which is common when those regions have fewer training samples or stronger nonlinear behavior. The fitted trend has a slope close to one and an intercept near zero, indicating minimal systematic bias. Overall, the residuals appear predominantly random rather than patterned, suggesting that the model is well calibrated and performs reliably across the validation set.

Figure 13. Regression plot for the LSTM model.

3.7. TCN Model

As shown in Table 29, the training set includes 1,875,700 samples, while the validation and test sets each contain 401,920 samples. The MSE ranges from 3.02 × 10⁻³ to 3.12 × 10⁻³, and the RMSE falls between approximately 5.50 and 5.58 × 10⁻². Correlation values remain high, with R around 0.997 and R² close to 0.994. The similarity of these values across all splits suggests that the TCN model performs consistently on both seen and unseen data, indicating strong generalization and minimal overfitting.

Table 29. Data split and basic performance statistics for the TCN model.

Data Split	Samples	MSE	RMSE	R	R2
Training	1,875,700	3.0223 × 10−3	5.4976 × 10−2	0.99702	0.99401
Validation	401,920	3.1169 × 10−3	5.5830 × 10−2	0.99691	0.99379
Test	401,920	3.0983 × 10−3	5.5662 × 10−2	0.99694	0.99387

As summarized in Table 30, the network consists of five one-dimensional convolutional blocks with a kernel size of 3 and dilation rates of [1, 2, 4, 8, 16], followed by a fully connected layer. A window length of 80 and a stride of 20 (75% overlap) help capture longer temporal patterns and increase the number of training windows. The model is trained using the Adam optimizer with an initial learning rate of 0.002, which drops by a factor of 0.2 at epoch 40. Gradient clipping with a norm of 5 and dropout at 10% are applied to ensure stability and reduce overfitting. Inputs are standardized using Z-score normalization, and targets are scaled to the range from −1 to 1. Early stopping with a patience of 10 epochs selects the model at epoch 63, where the best validation MSE reaches approximately 3.1 × 10⁻³. The total training time is about 1.17 h.

Table 30. Hyperparameters and training settings for the TCN model.

Parameter	Value
Model Name	TCN
Architecture	TCN: 5 Conv1D blocks (k = 3, dil = [1 2 4 8 16]) + FC
Window Length	80
Stride	20
Max Epochs	80
Best Epoch	63
Batch Size	128
Optimizer	Adam
Initial Learning Rate	0.0020
LR Drop Epoch	40
LR Drop Factor	0.2000
Gradient Clip (Global Norm)	5
Validation Every (iters)	50
Dropout Rate	0.1000
Number of Filters	64
Kernel Size	3
Dilations	[1 2 4 8 16]
Input Normalization	Z-score
Target Scaling	Linear to [−1, 1]
Early Stop Patience	10
Best Validation MSE	3.1 × 10−3
Training Time	4207.3 s (~1.17 h)

As shown in Figure 14, MSE decreases steeply in the first 10 epochs and then oscillates narrowly around 0.003–0.005. The minimum validation MSE occurs at epoch 63. The train, validation, and test curves almost overlap, indicating a negligible generalization gap. As reported in Figure 15, GradRMS falls from 4.5 × 10⁻³ to 2.8 × 10⁻⁴ and then shows small oscillations. This indicates progressively smaller parameter updates and well-controlled gradients after the initial phase.

Figure 14. Performance curve for the TCN model.

Figure 15. Gradient curve for the TCN model.

As illustrated in Figure 16, the predicted values align closely with the 1:1 reference line, resulting in an R² of 0.994 and an RMSE of 0.0559. The scatter becomes slightly wider at the extreme SoC values, which is expected in regions with fewer data points or more complex nonlinear behavior. The slope remains close to one, indicating minimal systematic bias in the model’s predictions.

Figure 16. Regression plot for the TCN model.

3.8. Test Results

All tests were conducted under ambient temperature, both charging and discharging conditions. Specifically, Test 1 was performed at −10 °C, Test 2 at 0 °C, Test 3 at 10 °C, and Test 4 at 25 °C.

3.8.1. Test 20-Neuron BR Fitting FNN

Table 31 presents the performance of the model across four distinct test sets. All four tests show very high linear correlation (R ≥ 0.9977) and low error values, confirming that the network generalizes well across different conditions. Test 4, conducted at 25 °C, clearly performs the best. It achieves the lowest MSE of 1.5883 × 10⁻⁴ while maintaining near-perfect correlation. Conversely, Test 3 recorded the worst metrics, with an MSE of 3.4559 × 10⁻⁴. Throughout all tests, the intercept values β₀ remained near zero, ranging from +0.0028 to +0.020, confirming that the model maintains minimal systematic bias.

Table 31. Test results for the 20-neuron BR-Fitting-FNN.

Test	Observation	MSE	RMSE	R	Intercept β0
Test 1	39,293	1.6532 × 10−4	1.29 × 10−2	0.9991	+0.020
Test 2	42,530	2.0767 × 10−4	1.44 × 10−2	0.9986	+0.0028
Test 3	44,284	3.4559 × 10−4	1.86 × 10−2	0.9977	+0.008
Test 4	47,517	1.5883 × 10−4	1.26 × 10−2	0.9991	+0.0073

3.8.2. 10-Neurons LM NAR-FNN

Table 32 details the model demonstrated exceptional precision, achieving its best performance in Test 4 with the lowest MSE of 1.0750 × 10⁻⁸. Test 3 recorded the marginally highest error with an MSE of 1.3398 × 10⁻⁸, indicating very consistent behavior across different data splits. Notably, the correlation coefficient (R) remained at a perfect 1.0000 for all tests, confirming the model’s superior predictive capability and stability in capturing the underlying non-linear patterns.

Table 32. Test results for the 10-neuron LM-NAR-FNN.

Test	Observation	MSE	RMSE	R
Test 1	39,293	1.2033 × 10−8	3.469 × 10−5	1.0000
Test 2	39,293	1.3121 × 10−8	3.623 × 10−5	1.0000
Test 3	39,293	1.3398 × 10−8	3.660 × 10−5	1.0000
Test 4	39,293	1.0750 × 10−8	3.278 × 10−5	1.0000

For 10-neurons LM NAR-FNN, the time-series plot as shown in Figure 17, demonstrates perfect alignment between the predicted and actual SoC curves across two full charge–discharge cycles, with no visible lag or offset. The output curve lies directly on top of the target, except for one brief spike near 1.25 × 10⁴ s, where the error reaches approximately ±1.4 × 10⁻².

Figure 17. Time-series response and errors for Test 1 using the 10-neurons LM-NAR-FNN.

3.8.3. 30-Neuron BR-Nonlinear I/O-FNN

The model’s error continues to decrease, and the correlation improves as the temperature rises. At the lowest temperature (−10 °C), the test performs worse than the other tests. As per Table 33, test 4 is the most accurate and dependable for estimating the SoC, with its best performance observed, as the model achieved a MSE of 3.81 × 10⁻⁴ and R of 0.9978, indicating high prediction accuracy and strong agreement between the estimated and actual values.

Table 33. Test results for the 30-neuron BR-Nonlinear I/O-FNN.

Test	Observation	MSE	RMSE	R
Test 1	39,293	11.000 × 10−4	3.3166 × 10−2	0.9916
Test 2	42,530	7.0662 × 10−4	2.6582 × 10−2	0.9951
Test 3	44,284	6.3907 × 10−4	2.5260 × 10−2	0.9960
Test 4	47,517	3.8147 × 10−4	1.9531 × 10−2	0.9978

Similarly, the time series plot as illustrated in Figure 18 for 30-neuron BR-Nonlinear I/O-FNN, the most noticeable error spikes appear at around 1.4 × 10⁴ s and 3.1 × 10⁴ s, with corresponding errors of approximately −3.9 and −2.5 units, respectively.

Figure 18. Time-series response and errors for Test 2 using the 30-neuron BR-Nonlinear I/O-FNN.

3.8.4. 10-Neuron BR-NARX-FNN

The 10-neuron BR-NARX-FNN achieves exceptionally high predictive accuracy across all four test sets as shown in Table 34. The model consistently produces extremely low MSE values, ranging from 1.3178 × 10⁻⁸ to as low as 6.5646 × 10⁻⁹, indicating near-perfect error minimization. The model maintains perfect correlation (R = 1.0000) in every test, reflecting flawless alignment between predicted and true outputs. These results confirm the model’s outstanding stability, precision, and generalization capability, making it highly reliable prediction. In Figure 19, the model output overlaps the target curve almost perfectly, with only two brief excursions approximately at 1.25 × 10⁴ s and 2.75 × 10⁴ s, pushing the error up to ±2 × 10⁻² and −0.5 × 10⁻², respectively.

Table 34. Test results for the 10-neuron BR-NARX-FNN.

Test	Observation	MSE	RMSE	R
Test 1	39,293	1.3178 × 10−8	3.6315 × 10−5	1.0000
Test 2	42,530	1.0554 × 10−8	3.2486 × 10−5	1.0000
Test 3	44,284	1.1786 × 10−8	3.4331 × 10−5	1.0000
Test 4	47,517	6.5646 × 10−9	2.5616 × 10−5	1.0000

Figure 19. Time-series response and errors for Test 1 using the 10-neuron BR-NARX-FNN.

3.8.5. Test Results for GRU Model

The performance of the GRU model across four distinct test sets with increasing sample sizes is shown in Table 35. The model exhibits a clear trend of improved accuracy as the sample size increases, with Test 4 achieving the optimal result and the lowest MSE of 1.7793 × 10⁻⁴. Conversely, Test 1 recorded the highest error, with an MSE of 2.5687 × 10⁻⁴. Across all partitions, the model maintained exceptional predictive strength, with Correlation (R) values consistently exceeding 0.998 and a stable Mean Overlap Count averaging approximately 3.99. In the time-series plot, as shown in Figure 20, the predicted SoC closely follows the true curve, although minor delays are observed during rapid SoC increases.

Table 35. Test results for the GRU model.

Test	Samples	MSE (×10−4)	RMSE (×10−2)	R	R2	Mean Overlap Count
Test 1	39,293	2.5687	1.6027	0.99842	0.99580	3.9926
Test 2	42,530	2.0714	1.4392	0.99858	0.99712	3.9934
Test 3	44,284	2.1146	1.4542	0.99862	0.99719	3.9942
Test 4	47,517	1.7793	1.3339	0.99909	0.99789	3.9935

Figure 20. SoC prediction versus true curve for Test 1 using the GRU model.

3.8.6. LSTM Test Results

The performance results of the LSTM model show a clear temperature-dependent trend. Model accuracy steadily improves with temperature demonstrating consistently strong predictive accuracy, with MSE values ranging from 1.7065 × 10⁻⁴ to 2.9246 × 10⁻⁴, indicating low prediction error. The correlation coefficient R is extremely high, approximately 0.99 for all tests. Similarly, the R² values ranging from 0.99612 to 0.99797 show that the model explains nearly all variance in the target output. As shown in Figure 21, performance improves and becomes more compact and symmetric, corresponding to a lower RMSE of 1.3277 × 10⁻² across 42,530 samples. The predicted SoC curve matches the true SoC almost exactly across all charging and discharging segments, with no visible phase lag.

Figure 21. SoC prediction versus true curve for Test 2 using the LSTM model.

3.8.7. TCN Test Results

Across all four temperature conditions, the TCN model shows a clear trend. It performs best at the temperature extremes (−10 °C and 25 °C), while error increases slightly at 0 °C and peaks at 10 °C. All MSE values remain high, and R values are consistently above 0.998, showing that the model is highly reliable across a wide range of conditions. Compared with the GRU and LSTM models analyzed earlier, the TCN displays slightly better MSE than the LSTM at 25 °C. Overall, the TCN provides the most balanced performance, combining low random error with stable behavior across all SoC levels. Its best results occur at 25 °C, making that the model’s ideal operating point under the tested settings. In Figure 22, the predicted SoC closely follows the true curve during both charging and discharging, with no visible lag or deviation, which confirms that both random and systematic errors are at their lowest under these conditions.

Figure 22. SoC prediction versus true curve for Test 4 using the TCN model.

3.9. Performance Comparison of All Models for SoC Estimation

Across the seven test tables (Table 31, Table 32, Table 33, Table 34, Table 35, Table 36 and Table 37), the effects of ambient temperature and network architecture on State of Charge (SoC) estimation can be evaluated in a structured way. In the feedforward group, the 20-neuron BR-Fitting-FNN in Table 1 shows apparent sensitivity to temperature. Its error values are highest at 10 °C and lowest at 25 °C. The positive intercept β₀ also decreases as temperature increases, indicating that the model’s tendency to underpredict in cold conditions becomes less severe at warmer temperatures. In contrast, the two autoregressive feedforward models, the 10-neuron LM-NAR-FNN in Table 2 and the 10-neuron BR-NARX-FNN in Table 4, perform consistently across all temperature levels. Both maintain RMSE values around 3 × 10⁻⁵ and achieve perfect correlation with R equal to 1.0000 in every test. This high level of accuracy is achieved by relying on recent ground truth values, which limits their use in fully predictive onboard systems where past targets are unavailable. The 30-neuron BR-Nonlinear-IO-FNN in Table 3 shows both the most significant overall error and the most substantial dependence on temperature, highlighting the limitations of static input-output mappings when modeling dynamic battery behavior. Among the sequence learning models, the GRU in Table 5 and the LSTM in Table 6 show nearly identical behavior. In both cases, RMSE improves from about 1.6 × 10⁻² at −10 °C to just above 1.3 × 10⁻² at 25 °C, while correlation remains consistently high with R greater than 0.998. Both models show a slight positive bias at the lowest temperature, likely caused by increased impedance and voltage drop, but this bias disappears at warmer conditions. The TCN in Table 7 follows a similar trend but performs slightly better than the recurrent models in every test. At low temperature, its RMSE is slightly lower than that of the GRU, and at 25 °C, it achieves the best result among all sequence-based models, with an RMSE of 1.287 × 10⁻². It also produces the most symmetric residual distribution. Overall, the results suggest a clear performance ranking. If the use of past target values is acceptable, the BR-NARX-FNN delivers the highest accuracy across all temperatures. However, in fully predictive applications, the TCN offers the best combination of low random error, minimal bias, and stable performance across the whole temperature range from −10 °C to 25 °C. It consistently outperforms both GRU and LSTM models in terms of absolute accuracy and consistency, making it the most reliable sequence learning option under the tested conditions.

Table 36. Test results for the LSTM model.

Test	Samples	MSE (×10−4)	RMSE (×10−2)	R	R2	Mean Overlap Count
Test 1	39,293	2.2111	1.4870	0.99850	0.99638	3.9926
Test 2	42,530	1.7628	1.3277	0.99880	0.99755	3.9934
Test 3	44,284	2.9246	1.7101	0.99813	0.99612	3.9942
Test 4	47,517	1.7065	1.3063	0.99907	0.99797	3.9935

Table 37. Test results for the TCN model.

Test	Samples	MSE (×10−4)	RMSE (×10−2)	R	R2	Mean Overlap Count
Test 1	39,293	1.8623	1.3647	0.99874	0.99695	3.9926
Test 2	42,530	2.1725	1.4739	0.99853	0.99697	3.9934
Test 3	4.4284	2.8741	1.6953	0.99815	0.99618	3.9942
Test 4	47,517	1.6566	1.2871	0.99903	0.99803	3.9935

4. Conclusions and Recommendations

This study compared four types of neural networks (FNN, LSTM, GRU, and TCN) for estimating the state of charge (SoC) in lithium-ion batteries using a unified process and fixed data splits. For the FNN models, three training algorithms were tested: Levenberg–Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG). Each was evaluated with three hidden-layer sizes (10, 20, and 30 neurons) and across different model types: Fitting, Nonlinear Input-Output (I/O), and time-series-based NAR and NARX. All models were trained with the same preprocessing steps and input features. Their performance was assessed using metrics such as test MSE and RMSE, generalization gap, training convergence, average training time per epoch, and compute cost.

Among the Fitting-FNN models, the 20-neuron BR version showed the best balance between accuracy and training cost. The 30-neuron LM version performed slightly better in terms of accuracy but required more training time. The SCG-Fitting-FNN was faster but less accurate. For the Nonlinear I/O topology, the 30-neuron BR model gave the best results with strong regularization effects, while the 20-neuron LM model also performed well but took longer to converge. In the NAR models, the 10-neuron LM version achieved the lowest test error and stable convergence, although it is very negative generalization gap may mean the test data was easier than expected. The 30-neuron BR-NAR model was also reliable, though more computationally expensive. In the NARX models, the 20-neuron LM version achieved the highest accuracy. The 10-neuron BR version showed clean convergence and low compute cost, while the SCG version trained quickly but did not match LM’s accuracy.

In practical terms, the BR-Nonlinear I/O-FNN with 30 neurons and the LM-NARX-FNN with 20 neurons are the top choices when accuracy is the primary goal. The LM-NAR-FNN with 10 neurons can also be considered, but only after confirming its reliability on different data splits. For a balanced option between accuracy and computation time, the BR-Fitting-FNN with 20 neurons or the BR-Nonlinear I/O-FNN with 20 neurons is recommended. When fast training and low resource use are required, SCG-based models with 10 or 20 neurons are suitable.

The trained machine learning models in this study can be readily deployed in a Battery Management System (BMS) for real-time state-of-charge (SoC) estimation. In practice, the model would receive measured inputs such as voltage, current, and temperature from embedded sensors within the battery pack. These inputs are preprocessed—using normalization and windowing consistent with the training phase—and fed into the trained neural network to predict SoC instantaneously. The model can operate on embedded platforms such as microcontrollers, digital signal processors (DSPs), or edge computing units, depending on the computational complexity of the selected architecture (e.g., FNN, LSTM, GRU, or TCN).

However, these findings come with some limitations. The results are based on a single-cell dataset and a fixed data split. Very negative generalization gaps in some runs may indicate that the test split was easier, so more testing is needed. Battery aging, variation between cells, and pack-level behavior were not included. Uncertainty estimation was not performed, except for the effect of BR’s regularization. Also, real-time hardware limitations were not considered during training or model selection.

Future studies should include better validation methods like k-fold and temperature-based splits. More focus should be placed on adapting models to different temperatures and battery aging. Combining neural networks with physical models or Kalman filters could also improve accuracy and interpretability. Adding uncertainty estimation methods like dropout or ensemble learning, and testing deployment with pruning and quantization on real embedded systems, will help make the models more practical. Applying the same evaluation process to LSTM, GRU, and TCN models will provide a complete picture of the accuracy and efficiency trade-offs for real-time SoC estimation. In addition, a detailed analysis of hardware-specific parameter counts, and real-time inference performance can be explored to further validate the machine learning models.