A Comprehensive Benchmarking of Evolutionary, Swarm-Intelligence, and Surrogate-Assisted Optimization for Residual Demand Forecasting in South African Microgrids ^†

Abstract

Accurate residual demand forecasting (RDF) is essential for stable peer-to-peer energy trading in developing economies. This study benchmarks three hyperparameter optimization paradigms, HEBO, PSO, and GP-BO, applied to XGBoost (2.1.4) forecasting using seven-fold TimeSeriesSplit validation on South African hourly grid data. Results demonstrate a fundamental trade-off between accuracy and efficiency: PSO achieves superior accuracy (0.47% MAPE) at the cost of substantial computation (23.4 h), while GP-BO offers revolutionary speed (19 min) with acceptable accuracy trade-offs. HEBO provides balanced performance with stable convergence. Crucially, we identify a “data–optimizer coupling” effect where optimal scaling methods are algorithm-dependent. These findings provide context-specific deployment strategies for microgrid operators addressing energy trilemma challenges.

1. Introduction

The global energy sector’s transformation through decentralization, digitalization, and decarbonization poses complex challenges for grid stability [1]. This “Energy Trilemma”—balancing energy security, equity, and sustainability—is particularly acute in developing economies with ageing infrastructure [2]. South Africa exemplifies these challenges, facing chronic load-shedding while pursuing ambitious renewable targets under its Just Energy Transition framework [3].

Peer-to-peer energy trading enables direct renewable exchange but requires accurate residual demand forecasting (total load minus renewable generation) for market stability, enabling dynamic pricing and grid balancing [4,5]. Renewable generation’s stochastic nature and complex consumption patterns make this particularly challenging in microgrids [6].

While machine learning approaches like XGBoost show promise for energy forecasting [7], their effectiveness depends critically on hyperparameter optimization. Traditional methods become computationally prohibitive in high-dimensional spaces [8], necessitating intelligent strategies. The current literature lacks comprehensive HPO benchmarking for microgrid forecasting in resource-constrained environments [9]. This study addresses these gaps through rigorous empirical benchmarking of three HPO paradigms: Heteroscedastic Evolutionary Bayesian Optimization (HEBO) [10], Particle Swarm Optimization (PSO) [11], and Surrogate-Assisted Gaussian Process Bayesian Optimization (GP-BO) [12].

Contributions include advanced feature engineering, rigorous temporal validation, data–optimizer coupling analysis, computational efficiency benchmarking, and context-specific deployment guidelines.

2. Literature Review

2.1. Machine Learning in Energy Forecasting

Energy forecasting has progressed from statistical to machine-learning approaches. Traditional methods like ARIMA assume stationarity and linearity [13], assumptions frequently violated by renewable integration [14]. Modern ML captures complex non-linear relationships with tree-based ensembles like Extreme Gradient Boosting (XGBoost) [7] and Light Gradient Boosting Machine (LightGBM) [15] architectures, proving particularly effective for energy data. Deep learning shows promise for sequence forecasting [16] but requires substantial resources, making it impractical for resource-constrained microgrids [17]. This positions gradient boosting as the preferred approach, though its performance depends heavily on hyperparameter configuration [18].

2.2. Hyperparameter Optimization Paradigms

Hyperparameter optimization critically determines automated ML performance. Population-based methods like PSO [11] navigate high-dimensional spaces but require substantial evaluations, making them computationally expensive [19]. Recent metaheuristic advances address these limitations. Chou and Nguyen [20] proposed the Age of Exploration-Inspired Optimizer (AEIO), integrating DBSCAN clustering to balance exploration and exploitation, outperforming 11 established optimizers on energy forecasting tasks, with lower MAPE and reduced computational burden.

Hybrid architectures combining multiple paradigms have also emerged. Chen et al. [21] introduced a Chaotic Quasi-Reverse Artificial Lemming Algorithm (CQALA) within a multi-scale convolutional Kolmogorov–Arnold network (MCKAN), achieving a 27.6–33.4% lower mean absolute error in wind and solar forecasting.

Bayesian Optimization (BO) offers sample efficiency through probabilistic surrogates [12], though standard GP-BO assumes homoscedastic noise and scales poorly [22]. PSO-optimized Bayesian LSTM frameworks address this, achieving 22.84% MAPE reduction for solar PV forecasting [23]. Despite these advances, hybrid and adaptive optimizers for microgrid residual demand forecasting—particularly in resource-constrained developing economies—remain critically underexplored [10,24].

2.3. Data-Centric Optimization Gaps

A persistent HPO limitation is treating data preprocessing as static [25]. The “No Free Lunch” theorems [26] imply that optimizer performance depends on alignment with objective function geometry—which is inherently shaped by data scaling. Recent work beyond energy corroborates this interdependence. Murtiningsih et al. [27] demonstrated that boosting algorithm efficacy for hypertension prediction was highly sensitive to feature selection, resampling, and tuning protocols. Similarly, the performance of the HPO method in heart failure prediction varied significantly across imputation techniques and validation strategies [28]. These findings reinforce treating preprocessing and optimization as a coupled system—a principle absent from most of the energy forecasting literature. This study addresses this gap by systematically investigating “data–optimizer coupling” for microgrid residual demand forecasting.

3. Methodology

This study uses hourly residual demand data from Eskom (2018–2025) to benchmark hyperparameter optimization methods for microgrid forecasting. Residual demand, calculated as total grid load minus renewable generation, supports P2P energy trading. An advanced feature engineering pipeline generates 108 high-value predictors capturing seasonality, trends, and system state signals. XGBoost is used with 12-dimensional hyperparameter space, optimized via HEBO, PSO, and GP-BO. Rigorous 7-fold TimeSeriesSplit cross-validation with RMSLE objective evaluates performance across four scaling methods (Raw, MinMax, Standard, Robust). GPU acceleration (NVIDIA RTX A4500 which is manufactured by NVIDIA Corporation, Santa Clara, CA, USA) and computational resource monitoring ensure efficient experimentation, enabling data–optimizer coupling effects to be investigated.

4. Assessment Metrics and Forecasting Results

4.1. Performance Evaluation Metrics

The models were evaluated using five evaluation metrics, notably, Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Logarithmic Error (RMSLE) and Coefficient of Determination ($R^2$) to validate performance against benchmarks. These metrics are defined as follows:

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{i=1}^N}\left|y_i-{\widehat{y}}_i\right|$$

(1)

$$MAPE={\displaystyle \frac{100\%}{N}}{\displaystyle {\sum }_{i=1}^N}{\left|{\displaystyle \frac{y_i-{\widehat{y}}_i}{y_i}}\right|}^2$$

(2)

$$RMSLE=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle {\sum }_{i=1}^N}{\left(\log \left(1+y_i\right)-\log \left(1+{\widehat{y}}_i\right)\right)}^2}$$

(3)

$$R^2=1-{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^N}{\left(y_i-{\widehat{y}}_i\right)}^2}{{\displaystyle {\sum }_{i=1}^N}{\left(y_i-\overline{y}\right)}^2}}$$

(4)

where $N$ is the number of samples, $y_i$ is the actual value, $y_i$ is the predicted value, and $y$ is the mean of actual values.

4.2. Comprehensive Performance Benchmarking

Table 1. Optimizer performance across scaling methods.

Optimizer	Best Scaling	MAPE (%)	RMSLE	Optimization Time
PSO	Standard	0.469	0.00227	23.38 h
HEBO	Standard	0.482	0.00231	20.63 h
GP-BO	Robust	4.770	0.01768	0.32 h

reveals a clear accuracy–efficiency trade-off. PSO achieves superior forecasting accuracy (0.47% MAPE), making it suitable for mission-critical applications where precision outweighs computational cost. HEBO provides comparable accuracy with slightly better computational characteristics. GP-BO demonstrates revolutionary optimization speed (19 min versus 20+ h for population-based methods), though with reduced accuracy (4.77% MAPE).

Figure 1 illustrates distinct convergence patterns: GP-BO (a) shows rapid initial improvement characteristic of sample-efficient surrogate methods, while PSO (b) and HEBO (c) exhibit a more gradual but ultimately superior convergence. HEBO’s stable trajectory (c) suggests robust handling of the complex, multi-modal search space.

4.3. Multi-Horizon Forecasting Analysis

Figure 2 demonstrates temporal consistency across forecasting horizons. PSO (b) and HEBO (c) maintain stable performance from 1-h to 7-day forecasts, indicating robust capture of temporal dependencies.

GP-BO (a) shows degradation at longer horizons, suggesting limitations in identifying hyperparameters that generalize across multiple timescales. This has significant implications for microgrid operations requiring both short-term dispatch and long-term planning.

4.4. Residual Error Characterization

Figure 3 reveals error distribution characteristics critical for P2P trading risk assessment. PSO (b) exhibits the tightest error concentration around zero (±0.02), minimizing extreme forecasting errors that could destabilize energy markets. HEBO (c) shows slightly wider but well-behaved distributions. GP-BO’s broader, multi-modal error distribution (a) explains its higher MAPE despite competitive R² values, indicating consistent but larger-magnitude errors.

5. Case Study: Input Data and Setting

5.1. Input Data

The dataset spans 8 April 2018 to 22 September 2025, capturing critical transitions in South Africa’s energy policy and infrastructure. This temporal scope enables analysis of forecasting performance during both stable and crisis periods, providing robust insights for operational deployment.

5.2. Experimental Configuration

The model is implemented using Python 3.12.7 and CUDA 12.8.93, with libraries including scikit-optimize 0.9 and HEBO 0.3.0. The computational environment features an AMD (Advanced Micro Devices, Santa Clara, CA, USA) Ryzen 9 5950X 16-core processor (3.40 GHz), 20 GB NVIDIA RTX A4500 GPU, 64 GB RAM, 64-bit OS, and x86_64 architecture.

5.3. Validation Protocol

The seven-fold TimeSeriesSplit validation ensures temporal integrity with the following: training periods—sequentially expanding windows; validation periods—fixed 3-month intervals; test periods—subsequent 1-month intervals. This approach mimics real-world deployment where models are periodically retrained on expanding historical data.

6. In-Depth Discussion of Findings

6.1. Engineering Significance and Operational Implications

These results extend beyond comparative algorithm performance to provide actionable intelligence for engineering design and operational management of AI-enabled microgrid systems.

The Accuracy–Efficiency Trade-Off as a Deployment Decision Variable

The findings (

Table 2. Optimal scaling method by optimizer.

Optimizer	Optimal Scaling	Performance Drop with Alternative
PSO	Standard	41% higher MAPE with Robust
HEBO	Standard	47% higher MAPE with Robust
GP-BO	Robust	24% higher MAPE with Standard

) show that optimizer choice is a strategic decision impacting system reliability, operational agility, and costs. PSO with standard scaling offers high accuracy (0.47% MAPE) but is computationally intensive (23.4 h), as shown in

Table 3. Computational resource utilization.

Metric	PSO	HEBO	GP-BO
Time	23.38 h	20.63 h	0.32 h
Evaluations	372	372	60
CPU Utilization	56.3%	56.1%	5.6%
Model Size	7.05 MB	3.52 MB	-

, making it suitable for offline, periodic retraining. GP-BO with robust scaling enables rapid retraining, is 64–73× faster, and 6.2× more sample efficient, making it ideal for adaptive control. HEBO with standard scaling balances performance and efficiency, making it suitable for edge deployment on resource-constrained controllers. Co-designing preprocessing and optimization strategies is crucial, as GP-BO enables sub-daily retraining and adaptation to evolving grid conditions. The accuracy–efficiency trade-off guides deployment decisions, and standardized preprocessing workflows are inadequate. GP-BO supports environmental sustainability mandates and regulatory trends, offering a pathway to reduce the model training carbon footprint without compromising forecasting integrity [29].

7. Conclusions and Future Work

This study benchmarks hyperparameter optimization methods for microgrid residual demand forecasting, revealing accuracy-efficiency trade-offs. PSO achieves 0.47% MAPE but is computationally intensive, making it suitable for accuracy-critical applications. GP-BO offers revolutionary efficiency with 19-min optimization and acceptable accuracy trade-offs, enabling rapid model iteration. HEBO provides balanced performance with stable convergence and efficient model architectures.

Data–optimizer coupling is critical, requiring algorithm-specific preprocessing strategies. Optimal scaling methods vary by algorithm, necessitating co-design of data preprocessing and optimization. This insight has broad applicability beyond energy forecasting to domains employing automated machine learning pipelines.

The results guide policymakers and microgrid operators in deploying ML solutions, enabling practical AI-driven forecasting in resource-constrained environments. This supports the transition toward decentralized, sustainable energy systems. Future research directions include hybrid optimization, federated learning [30], transfer learning [31], explainable AI [32], and real-time adaptive optimization.