A VMD–Bayesian-Optimized XGBoost–BiLSTM Hybrid Model for Short-Term Load Forecasting

Abstract

Accurate short-term load forecasting is essential for reliable power system operation under increasingly nonlinear, volatile, and multi-scale load patterns. This study proposes a VMD–BayesXGB–BiLSTM hybrid forecasting framework that integrates time-series-cross-validation-based variational mode decomposition (VMD), Bayesian-optimized XGBoost (BayesXGB), and BiLSTM residual correction. First, abnormal values in the raw load and explanatory variables are detected using the $3\sigma$ criterion and corrected by cubic spline interpolation. Then, VMD parameters are selected only within the training sequence, and leakage-free VMD features are generated from historical input windows, avoiding the use of future information. BayesXGB is employed as the primary forecasting model to capture nonlinear relationships between historical load, VMD-derived multi-scale features, and external variables. Finally, a stacked BiLSTM module learns temporal patterns from historical BayesXGB predictions and residuals, and the predicted residual correction is added to the preliminary forecast. Experiments on an Australian electricity load dataset show that the proposed model achieves an RMSE of 122.1003, an MAE of 90.7386, a MAPE of 1.0269%, and an R² of 0.9921, outperforming all compared baseline models while maintaining sub-millisecond inference per sample.

1. Introduction

Short-term load forecasting is an important task for power system dispatch, operational planning, electricity market decision-making, and grid reliability management [1]. Accurate load forecasts help system operators allocate generation resources more effectively, reduce operational uncertainty, and maintain the stability of power supply. With the development of new power systems and the increasing integration of renewable energy, electricity load profiles have become more nonlinear, volatile, and multi-scale. These changes pose significant challenges to the accuracy and robustness of short-term load forecasting models [2,3,4].

Existing studies on short-term load forecasting have mainly developed along three directions: statistical forecasting, machine learning, and deep learning. Statistical models, including ARIMA and Kalman filtering, are easy to implement and interpret, but their performance is often limited when load data contain strong nonlinear and non-stationary patterns [5,6,7,8]. Machine learning methods, such as support vector regression, random forests, and XGBoost, can model nonlinear relationships between meteorological variables, electricity price, historical load, and future demand [9,10,11]. Among them, XGBoost has shown strong performance in regression tasks because of its ensemble learning structure and regularization mechanism. However, tree-based models do not directly learn sequential dependencies from time-series inputs. In contrast, deep learning models, such as RNN, LSTM, GRU, BiLSTM, attention-based networks, and convolutional-recurrent hybrid models, are able to extract temporal patterns from historical sequences [12,13,14,15]. Recently, Cavus et al. [16] developed a residual-normalized GRU-based forecasting framework, showing that recurrent learning, residual modeling, and normalization mechanisms can improve the robustness of short-term energy forecasting under non-stationary conditions. Nevertheless, deep learning models may still be affected by high-frequency noise, complex fluctuations, and relatively high training costs.

To reduce the influence of high-frequency noise and non-stationary fluctuations, signal decomposition methods have often been combined with forecasting models. EMD and EEMD can decompose complex load sequences into several frequency-related components, but they may be affected by mode mixing, endpoint effects, and empirical settings [17,18,19]. Variational Mode Decomposition (VMD) provides a more stable decomposition framework and has been applied to short-term load forecasting tasks [20]. Through decomposition, the original load sequence can be transformed into several relatively stable sub-sequences, which provides a useful basis for subsequent nonlinear forecasting.

The forecasting stage after decomposition also affects the overall performance of hybrid load forecasting models. For example, Nabavi et al. [21] combined DWT with LSTM for electricity load forecasting and showed that the DWT-LSTM model achieved higher forecasting accuracy than several benchmark models. This study demonstrates the effectiveness of combining signal decomposition with deep learning for load forecasting. However, after a primary prediction is obtained, the remaining errors may still contain nonlinear and time-dependent patterns. Therefore, residual correction can be introduced as a supplementary step to calibrate preliminary forecasts and improve the final forecasting results [22].

Based on the above discussion, this paper proposes a VMD–BayesXGB–BiLSTM hybrid model for short-term load forecasting. Specifically, the proposed framework combines TSCV-based VMD parameter selection, Bayesian-optimized XGBoost forecasting, and BiLSTM-based residual correction into a unified forecasting pipeline. This design progressively reduces the complexity of the load sequence, improves the nonlinear prediction capability of the primary model, and further calibrates residual temporal errors.

The main contributions of this study are summarized as follows:

• TSCV-based parameter selection for VMD: To reduce the dependence on empirical parameter settings in decomposition-based forecasting, a time-series cross-validation strategy is introduced for VMD parameter selection. By selecting the number of modes and penalty factor through validation across temporal folds, the decomposition process becomes more adaptive to the multi-scale characteristics of the load sequence.
• Enhanced forecasting via BayesXGB: Bayesian optimization is introduced to tune the hyperparameters of the XGBoost regressor, with mean absolute error (MAE) used as the optimization objective. By incorporating VMD-derived multi-scale features, the BayesXGB model is used as the primary forecasting model to capture nonlinear relationships in the load data.
• BiLSTM-based residual correction mechanism: A BiLSTM correction module is introduced after the BayesXGB forecasting stage to model the temporal patterns contained in prediction residuals. Historical BayesXGB predictions and residual sequences are used as inputs, and the predicted residual correction is added to the BayesXGB output to obtain the final forecast.

2. Methods

2.1. VMD

VMD is an adaptive mode decomposition method for analyzing non-stationary, nonlinear signals. It transforms the signal decomposition process into solving a constrained variational optimization problem. Compared to EMD, VMD typically yields more stable results with superior noise resistance, effectively resolving the mode mixing issues inherent in EMD [23]. The constrained variational model is formulated as follows:

$$\left\{\begin{array}{l}\min \{u_k\}\{{\omega }_k\}{\displaystyle \sum _k||𝜕t}\left[(\delta (t)+{\displaystyle \frac{j}{\pi t}})u_k(t))\right]e^{-j{\omega }_{k}t}{||}_2^2\\ s.t.{\displaystyle \sum _k{u}_k(t)}=f(t)\end{array}\right.$$

(1)

where

• $t$ represents time; $f(t)$ denotes the input raw load sequence;
• $u_k\left(t\right)$ is the $k$-th mode component;
• $\delta \left(t\right)$ is the unit impulse function;
• ${\omega }_k$ represents the center frequency of the $k$-th mode component;

Since solving the constrained variational problem directly is relatively challenging, it is transformed into an unconstrained optimization problem via the introduction of the Lagrangian function.

$$L\left(\left\{u_k\right\},\left\{{\omega }_k\right\},\lambda \right)=\alpha {{\displaystyle \sum _{\mathrm{k}=1}^K‖\left[\left(\delta (t)+\frac{j}{\pi t}\right)u_k(t)\right]e^{-j{\omega }_{k}t}‖}}_2^2+{‖f(t)-{\displaystyle \sum _{k=1}^K{u}_k(t)}‖}_2^2+〈\lambda (t),f(t)-{\displaystyle \sum _{k=1}^K{u}_k(t)}〉$$

(2)

• $\lambda \left(t\right)$ represents the Lagrange multiplier;
• $\alpha$ is the quadratic penalty factor.

Subsequently, the Alternating Direction Method of Multipliers (ADMM) is employed to iteratively solve the model until convergence is achieved. The update formulas for the modes and center frequencies are as follows:

Intrinsic Mode Function (IMF) Update:

$${\widehat{u}}_k^{n+1}(\omega )={\displaystyle \frac{\left[\widehat{f}(\omega )-{\displaystyle \sum _{i\ne k}{\widehat{u}}_i(\omega )}+\frac{\widehat{\lambda }(\omega )}{2}\right]}{1+2\alpha {(\omega -{\omega }_k)}^2}}$$

(3)

Among these, $\widehat{f}(\omega )$, ${\widehat{u}}_k\left(\omega \right)$ and $\widehat{u}\left(\omega \right)$ represent the Fourier transforms of $f\left(t\right)$, $u_k\left(t\right)$ and $u\left(t\right)$, respectively.

Center Frequency Update:

$${\omega }_k^{n+1}={\displaystyle \frac{{\displaystyle {\int }_0^{\infty }\omega {\left|{\widehat{u}}_k^{n+1}(\omega )\right|}^2}d\omega }{{{\displaystyle {\int }_0^{\infty }\left|{\widehat{u}}_k^{n+1}(\omega )\right|}}^{2}d\omega }}$$

(4)

2.2. Principles of XGBoost

XGBoost is an improved implementation of the gradient boosting decision tree (GBDT) algorithm [24,25]. GBDT performs classification or regression by integrating multiple weak learners into a strong ensemble learner. The weak learners integrated into XGBoost are typically CART (Classification and Regression Tree) regression trees [26], and its ensemble model is expressed as follows:

$${\widehat{y}}_i={\displaystyle \sum _{k=1}^n{f}_k(x_i)},f_k\in F$$

(5)

where

• ${\widehat{y}}_i$ represents the predicted value for the $i$-th sample;
• $f_k\left(·\right)$ denotes the functional relationship between the $k$-th tree structure and leaf weights;
• $n$ represents the total number of CART regression trees;
• $x_i$ denotes the feature vector of $i$-th sample;
• $F$ represents the space of all possible CART regression trees.

After the model training is completed, the final prediction for a given sample is obtained by aggregating the outputs of all individual trees. The iterative accumulation process for the inference stage is detailed in Algorithm 1.

Algorithm 1. Prediction Iteration Process for XGBoost

$\mathrm{Input:} \mathrm{Input} \mathrm{features} x_i$, number of trees $n$$\mathrm{and} \mathrm{trained} \mathrm{learners} \{f_1,f_2,\dots ,f_t\}$

$\mathrm{Output:} \mathrm{Final} \mathrm{prediction} \mathrm{value} {\widehat{y}}_i^{(n)}$

$\mathrm{1:} \mathrm{Initialize} \mathrm{prediction} \mathrm{value:} {\widehat{y}}_i^{(0)}\leftarrow 0$

$\mathrm{2:} \mathrm{for} k=1$ to $n$ do

$\mathrm{3:} \mathrm{Compute} \mathrm{learner} \mathrm{output:} current\_output\leftarrow f_k(x_i)$

$\mathrm{4:} {\widehat{y}}_i^{(k)}\leftarrow {\widehat{y}}_i^{(k-1)}+current\_output$

5: end for

$\mathrm{6:} \mathrm{return} {\widehat{y}}_i^{(n)}$

Among these,

• ${\widehat{y}}_i^{(k)}$ represents the predicted value after $k$ training iterations;
• ${\widehat{y}}_i^{(k-1)}$ denotes the cumulative prediction retained from the previous iteration;
• $f_k(x_i)$ signifies the newly added base learner at the $k$-th iteration.

To obtain the optimal functional forms for each $f_k$ during the training phase, XGBoost reduces prediction bias by performing a second-order Taylor expansion of the loss function. The objective function is formulated as

$$\left\{\begin{array}{l}{\mathrm{L}}_{loss}={\displaystyle \sum _{i=1}^n\varphi \left(y_i,{\widehat{y}}_i^{(k)}\right)+{\displaystyle \sum _{k=1}^k\Omega \left(f_k\right)}}\\ \Omega \left(f\right)=\gamma T+{\displaystyle \frac{1}{2}}\lambda {‖\omega ‖}^2\end{array}\right.$$

(6)

In the equation: $\varphi$ is the loss function, representing the error between the actual value and the predicted value; $\Omega \left(f\right)$ is the regularization term, where $\gamma$ and $\lambda$ are penalty coefficients, $T$ is the number of leaf nodes, and $\omega$ is the leaf weight. This function primarily serves to prevent model overfitting.

Performing a second-order Taylor expansion at $f_k=0$ yields an objective function that can be approximated as

$${\tau }^{(k)}\simeq {\displaystyle \sum _{i=1}^n\left[\varphi \left(y_i,{\widehat{y}}_i^{(k-1)}\right)+g_i{f}_k(x_i)+\frac{1}{2}{h}_i{f}_k^2(x_i)\right]}+\Omega (f_k)$$

(7)

where

• $g_i={\displaystyle \frac{𝜕}{𝜕{\widehat{y}}_i^{(k-1)}}}\varphi (y_i,{\widehat{y}}_i^{(k-1)})$ denotes the first derivative of the loss function;
• $h_i={\displaystyle \frac{{𝜕}^2}{𝜕{({\widehat{y}}_i^{(k-1)})}^2}}\varphi (y_i,{\widehat{y}}_i^{(k-1)})$ represents the second derivative of the loss function.

Equation (7) aggregates the loss functions for each sample, reorganizing all samples belonging to the same leaf node through the following process:

$$\begin{array}{l}{\mathrm{L}}_{loss}\simeq {\displaystyle \sum _{i=1}^n\varphi \left(g_i{f}_t(x_i),\frac{1}{2}{h}_i{f_t}^2(x_i)\right)}+\Omega \left(f_t\right)\\ ={\displaystyle {\displaystyle \sum _{i=1}^n}\left[g_i{\omega }_q(x_i)+\frac{1}{2}{h}_i{{\omega }^2}_q(x_i)\right]}+\gamma T+{\displaystyle \frac{1}{2}}\lambda {\displaystyle {\displaystyle \sum _{j=1}^T}{{\omega }^2}_j}\\ ={\displaystyle \sum ^T\left[\left({\displaystyle \sum _{i\in I_{\mathrm{j}}}{g}_i}\right){\omega }_j+\frac{1}{2}\left({\displaystyle \sum _{i\in I_j}{h}_i+\lambda }\right){{\omega }_j}^2\right]}+\gamma T\end{array}$$

(8)

Define $G_j={\displaystyle {\sum }_{i\in I_j}{g}_i}$ and $H_j={\displaystyle {\sum }_{i\in I_j}{h}_i}$. After taking the derivative with respect to $\omega$, the minimum value of the regularized objective can be obtained as

$${\mathrm{L}}_{loss}\cong -{\displaystyle \frac{1}{2}}{\displaystyle \sum _{j=1}^T\frac{{G_j}^2}{H_j+\lambda }}+\gamma T$$

(9)

2.3. BiLSTM Module

2.3.1. LSTM

Long Short-Term Memory (LSTM) is an improved variant of Recurrent Neural Networks (RNNs), which mitigates the gradient explosion and vanishing gradient problems that can arise in traditional RNN architectures [27,28]. As shown in Figure 1, LSTM primarily consists of an input gate, an output gate, and a forget gate.

Its structural formula is as follows:

$$\left\{\begin{array}{l}{f}_t=\sigma \left(W_f{h}_{t-1}+W_f{x}_t+b_f\right)\\ i_t=\sigma \left[W_i{h}_{t-1}+W_i{x}_t+b_i\right]\\ {\stackrel{-}{C}}_t=\tanh \left[W_c{h}_{t-1}+W_c{x}_t+b_c\right]\\ C_t=f_t\odot C_{t-1}+i_t\odot {\stackrel{-}{C}}_t\\ o_{\mathrm{t}}=\sigma \left[W_o(h_{t-1},x_t)+b_o\right]\\ h_t=o_t\odot \tanh \left(C_t\right)\end{array}\right.$$

(10)

where

• $f$ denotes the forget gate, which determines the proportion of information from the previous state to be discarded;
• $\sigma$ represents the sigmoid activation function;
• $W_f$, $W_i$, $W_c$, $W_o$ are the weight matrices corresponding to the forget gate, input gate, candidate cell state, and output gate, respectively;
• $x_t$ is the input value at the current time step;
• $b_f$, $b_i$, $b_c$, $b_o$ signify the bias vectors for each respective component;
• $i_t$ is the input gate, regulating the flow of new information into the cell state;
• ${\overline{C}}_t$ is the candidate cell state value;
• $\tanh$ is the activation function;
• $C_t$ is the cell state value of the current hidden layer;
• $C_{t-1}$ is the cell state value of the hidden layer in the previous state;
• $o_t$ is the output gate, which controls the amount of information to be transferred from the cell state to the hidden state;
• $h_t$ is the hidden-layer output, representing the final information output of the LSTM cell at the current step.

2.3.2. BiLSTM

Combining a forward LSTM and a backward LSTM into a BiLSTM network partially addresses the limitation of LSTMs, considering only unidirectional temporal sequences. This enables feature extraction from both forward and backward temporal sequences, thereby enhancing the model’s prediction accuracy [29]. The BiLSTM architecture is illustrated in Figure 2.

3. Research Methods and Model Architecture

3.1. Model Architecture

The overall architecture of the proposed VMD–BayesXGB–BiLSTM forecasting model is shown in Figure 3. The framework consists of four main stages: data preprocessing, VMD-based feature construction, BayesXGB forecasting, and BiLSTM-based residual correction.

First, the raw electricity load data and explanatory variables are preprocessed. The preprocessing procedure includes abnormal data detection, missing-value treatment, interpolation, and feature standardization. After preprocessing, the VMD parameters are selected using a TSCV strategy. The selected VMD parameters are then used to generate multi-scale IMFs from the historical load sequence. To avoid the use of future information, VMD feature construction is performed based on historical input windows. In other words, for each prediction sample, only the historical observations before the target time step are used for decomposition, while the target point and future load values are excluded.

Window-based samples are constructed for the primary BayesXGB forecasting stage using historical observations before the target time step. The input feature vector of BayesXGB consists of historical load-window features, VMD-derived multi-scale load features, and external explanatory variables. Based on these features, the BayesXGB model generates the preliminary one-step-ahead load prediction.

After the preliminary prediction is obtained, a separate historical sequence of BayesXGB predictions and residuals is constructed for the BiLSTM residual correction module. Therefore, the forecasting window used for the primary BayesXGB model and the sequence length used for BiLSTM residual correction refer to two different stages of the proposed framework. Their specific values are reported in the experimental settings.

After the preliminary prediction is obtained, a BiLSTM residual correction module is introduced to further improve the forecasting result. Let $y_t$ denote the actual load value at time step $t$, and let ${\widehat{y}}_{BayesXGB,t}$ denote the preliminary prediction generated by the BayesXGB model. The residual between the actual load and the BayesXGB prediction is defined as

$$e_t=y_t-{\widehat{y}}_{BayesXGB,t}$$

(11)

The residual sequence may still contain nonlinear and time-dependent patterns that are not fully captured by the primary BayesXGB model. Therefore, the BiLSTM correction module is used to learn the temporal characteristics of the residuals. For each time step t, the historical BayesXGB prediction sequence and the corresponding historical residual sequence are constructed as

$$\left[{\widehat{y}}_{BayesXGB,t-L},\dots ,{\widehat{y}}_{BayesXGB,t-1}\right]$$

(12)

and

$$\left[e_{t-L},\dots ,e_{t-1}\right]$$

(13)

where $L$ denotes the residual correction sequence length. These two sequences are combined as the input features of the BiLSTM residual correction module. Therefore, the input tensor of the BiLSTM model is organized as $N\times L\times d$, where $N$ is the batch size, $L$ is the sequence length, and $d$ is the feature dimension.

No teacher forcing is used in the BiLSTM residual correction module. During both training and testing, the residual correction at time step $t$ is estimated only from historical BayesXGB predictions and historical residuals before $t$. Therefore, the target load value and future residual information are not used to estimate the residual correction for the current prediction step.

The BiLSTM correction model outputs the predicted residual correction term ${\widehat{e}}_t$. The final corrected forecast is obtained by adding the predicted residual correction to the preliminary BayesXGB prediction:

$${\widehat{y}}_{final,t}={\widehat{y}}_{BayesXGB,t}+{\widehat{e}}_t$$

(14)

In this way, the BiLSTM module provides an additive correction to the BayesXGB preliminary forecast. Only historical BayesXGB predictions and historical residuals from previous time steps are used to estimate the residual correction at time step $t$.

To further clarify the residual correction process, Figure 4 illustrates the data flow from the BayesXGB preliminary prediction to the final BiLSTM-corrected forecast.

3.2. Model Evaluation Metrics

This paper employs four commonly used evaluation metrics in load forecasting tasks to assess the model’s predictive performance. These metrics are Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and $R^2$.

The specific calculation formulas for each metric are as follows:

$$\left\{\begin{array}{l}RMSE=\sqrt{{\displaystyle \frac{1}{n}}{{\displaystyle \sum _{i=1}^n\left(y_{true,i}-y_{pred,i}\right)}}^2}\\ MAE={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n\left|y_{true,i}-y_{pred,i}\right|}\\ MAPE={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n\left|\frac{y_{true,i}-y_{pred,i}}{y_{true,i}}\right|}\ast 100\%\\ R^2=1-{\displaystyle \frac{{\displaystyle \sum _{i=1}^n{\left(y_{true,i}-y_{pred,i}\right)}^2}}{{\displaystyle \sum _{i=1}^n{\left(y_{true,i}-{\overline{y}}_{true}\right)}^2}}}\end{array}\right.$$

(15)

where

• $y_{true}$ denotes the actual load value;
• $y_{pred}$ denotes the forecasted load value;
• $n$ denotes the sample size;
• ${\overline{y}}_{true}$ denotes the mean of the true load values.

4. Experimental Setup

4.1. Dataset Description and Preprocessing

The experiments were conducted using Python 3.7. A publicly available Australian electricity load dataset was used for simulation analysis. The dataset was obtained from the Australian Electricity Load and Price Forecasting Dataset repository, covering the period from 00:30 on 1 January 2006 to 00:00 on 1 January 2011, with a 30 min sampling interval. In addition to the electricity load, five explanatory variables were included: dry-bulb temperature, dew-point temperature, wet-bulb temperature, humidity, and electricity price. Figure 5 presents the time series plot of the electricity load for the Australian dataset.

As shown in Figure 5, the electricity load exhibits clear periodic fluctuations and seasonal variation. This indicates that electricity demand is affected by temporal patterns and meteorological factors, which increases the difficulty of short-term load forecasting.

Before model training, the raw dataset was processed through four main steps: outlier detection, outlier handling, feature standardization, and VMD-based feature construction. Section 4.1.1, Section 4.1.2, Section 4.1.3 and Section 4.1.4 describe these preprocessing steps in detail. After preprocessing, supervised learning samples were constructed using a sliding window. The input window length for the primary forecasting model was set to 24, corresponding to 12 h of historical observations, and the forecasting horizon was set to one step, corresponding to a 30 min-ahead prediction.

After window-sample construction, all samples were divided chronologically into training and testing sets. The first 80% of the samples were used as the training set, and the remaining 20% were used as the testing set. In total, 87,624 supervised samples were constructed, including 70,099 training samples and 17,525 testing samples. The training set was used for model fitting and parameter selection, while the testing set was reserved only for final performance evaluation.

For model tuning, validation was performed on the training set rather than using the testing set. The detailed validation strategies and hyperparameter settings for VMD, BayesXGB, and BiLSTM are reported in Section 4.2.

4.1.1. Outlier Detection

To mitigate the impact of dirty data caused by sensor failures, this paper employs the $3\sigma$ rule to identify outliers. This outlier detection method is based on the statistical principles of normal distribution.

Assuming data follows a normal distribution $N(\mu ,{\sigma }^2)$, the data distribution exhibits strict probabilistic patterns:

• Approximately 68.27% of the data falls within the interval $|\mu -\sigma ,\mu +\sigma |$;
• Approximately 95.45% of the data falls within the interval $|\mu -2\sigma ,\mu +2\sigma |$;
• Approximately 99.73% of the data falls within the interval $|\mu -3\sigma ,\mu +3\sigma |$.

Since the $3\sigma$ range encompasses nearly all (99.73%) of the normal data, the $3\sigma$ criterion stipulates that data exceeding the range $|\mu -3\sigma ,\mu +3\sigma |$ is highly likely to be an outlier (with a probability of only 0.27%, a low-probability event).

4.1.2. Outlier Handling

After outlier detection, abnormal values were replaced using cubic spline interpolation. This method was adopted because it can preserve the local trend and smoothness of the time series better than simple mean imputation or linear interpolation.

• For cubic spline interpolation, the original data sequence is divided into several intervals, and each interval is represented by a cubic polynomial:

$$Si(x)=ai{x}^3+bi{x}^2+cix+di$$

(where $i$ denotes the interval index, and $a_i$, $b_i$, $c_i$, $d_i$ are the polynomial coefficients for the $i$-th interval).

• Continuity Constraints: Polynomials in adjacent intervals must satisfy the following at nodes:

Function value continuity:

$$Si(x_{i+1})=Si+1(x_{i+1})$$

First derivative continuity:

$${S_i}^{\prime }(x_{i+1})={S_{i+1}}^{\prime }(x_{i+1})$$

Second derivative continuity:

$${S_i}^{″}(x_{i+1})={S_{i+1}}^{″}(x_{i+1})$$

• Boundary conditions: To ensure a unique solution, boundary constraints must be specified.

After applying the $3\sigma$-based outlier detection procedure, abnormal values were detected and corrected by cubic spline interpolation. Specifically, 195 outliers were detected in dry-bulb temperature, 55 in dew-point temperature, 217 in humidity, 174 in electricity price, and 124 in electricity load, corresponding to outlier ratios of 0.2225%, 0.0628%, 0.2476%, 0.1985%, and 0.1415%, respectively. No outliers were detected in wet-bulb temperature. Since the outlier ratios for all variables were below 0.25%, the preprocessing step mainly corrected abnormal local observations while preserving the overall structure of the original dataset.

4.1.3. Feature Standardization

To eliminate the influence of different feature scales on model training, feature standardization was applied before model fitting. The mean and standard deviation were calculated from the training set and then applied to both the training and testing sets. This procedure ensures that the testing data are transformed using only information obtained from the training set.

The standardization formula (Z-score normalization) is expressed as

$$x_{scaled}={\displaystyle \frac{x-\mu }{\sigma }}$$

(16)

where $x$ represents the original feature value, $\mu$ denotes the mean of the feature across the training set; $\sigma$ indicates the standard deviation of the feature across the training set; and $x_{scaled}$ signifies the standardized feature value.

4.1.4. VMD Feature Construction

VMD was used to extract multi-scale load features from the historical input windows. The VMD parameters were determined using only the training set, with the detailed selection procedure described in Section 4.2.1. For each prediction sample, VMD was applied only to the historical load window before the target time step, so the target value and future observations were not involved in feature construction.

Figure 6 shows the VMD results of a representative load segment from the training set.

4.2. Hyperparameter Settings

The main experimental settings include VMD parameter selection, Bayesian optimization of XGBoost hyperparameters for constructing BayesXGB, and the configuration of the BiLSTM residual correction module. All parameter tuning procedures were performed using only the training set, while the testing set was used only for final performance evaluation.

4.2.1. VMD Parameter Selection

For VMD, the number of modes $K$ and the penalty factor $\alpha$ were selected only within the training sequence. To reduce computational cost while preserving temporal order, the last 3000 consecutive observations from the training sequence were used as the parameter selection sample. The number of modes $K$ was searched from 3 to 14, and the penalty factor $\alpha$ was selected from {1000, 2000, 3000, 4000, 5000, 6000, 7000}.

For each candidate pair of $K$ and $\alpha$, three-fold time-series cross-validation was applied. The validation window length of each fold was 750 observations. The reconstruction mean squared error was calculated in each validation fold, and the average reconstruction MSE across the three folds was used as the final selection criterion. The mathematical procedure of the TSCV-based VMD parameter selection is summarized in Algorithm 2.

Algorithm 2. Time-series cross-validation-based VMD parameter selection

$\mathrm{Input:} \mathrm{Training} \mathrm{load} \mathrm{sequence} X_{train}=\left\{x_1,x_2,\dots ,x_T\right\}$$, \mathrm{sampling} \mathrm{length} S=3000$, $\mathrm{and} \mathrm{candidate} \mathrm{mode} \mathrm{number} \mathrm{set} K_c=\left\{3,4,\dots ,14\right\}$$, \mathrm{candidate} \mathrm{penalty} \mathrm{factor} \mathrm{set} {\alpha }_c=\left\{1000,2000,\dots ,7000\right\}$$, \mathrm{and} \mathrm{number} \mathrm{of} \mathrm{time-series} \mathrm{cross-validation} \mathrm{folds} \left(F=3\right)$.

$\mathrm{Output:} \mathrm{Optimal} \mathrm{VMD} \mathrm{parameter} \mathrm{pair} \left(K^{\ast },{\alpha }^{\ast }\right)$.

$\mathrm{1:} \mathrm{Select} \mathrm{the} \mathrm{last}$$S$$\mathrm{consecutive} \mathrm{observations} \mathrm{from} \mathrm{the} \mathrm{training} \mathrm{load} \mathrm{sequence} \mathrm{as} \mathrm{the} \mathrm{VMD} \mathrm{parameter} \mathrm{selection} \mathrm{sample:} X_s=\left\{x_{T-S+1},x_{T-S+2},\dots ,x_T\right\}$

$\mathrm{2:} \mathrm{Split} X_s$$\mathrm{into} F=3$$\mathrm{time-series} \mathrm{cross-validation} \mathrm{folds} \mathrm{while} \mathrm{preserving} \mathrm{chronological} \mathrm{order} X_s=\left\{V_1,V_2,V_3\right\}$$, \mathrm{where} \mathrm{each} \mathrm{validation} \mathrm{fold} V_f$$\mathrm{contains} N_f=750$ observations.

$\mathrm{3:} \mathrm{For} \mathrm{each} \mathrm{candidate} \mathrm{parameter} \mathrm{pair} \left(K,\alpha \right)\in K_c\times {\alpha }_c$$, \mathrm{apply} \mathrm{VMD} \mathrm{to} V_f$$, \mathrm{reconstruct} \mathrm{the} \mathrm{validation} \mathrm{signal} \mathrm{by} \mathrm{summing} \mathrm{all} \mathrm{decomposed} \mathrm{modes} \mathrm{and} \mathrm{the} \mathrm{residual} \mathrm{component}, \mathrm{and} \mathrm{compute} \mathrm{the} \mathrm{fold-wise} \mathrm{reconstruction} \mathrm{mean} \mathrm{squared} \mathrm{error:} {\mathrm{MSE}}_{\mathrm{f}}\left(K,\alpha \right)={\displaystyle \frac{1}{{\mathrm{N}}_{\mathrm{f}}}}{{\displaystyle \sum _{i=1}^{N_f}\left(x_{i,f}-{\widehat{x}}_{i,f}\left(K,\alpha \right)\right)}}^2$

$\mathrm{4:} \mathrm{Compute} \mathrm{the} \mathrm{average} \mathrm{reconstruction} \mathrm{error} \mathrm{across} \mathrm{all} \mathrm{folds:} \overline{\mathrm{MSE}}\left(K,\alpha \right)={\displaystyle \frac{1}{F}}{\displaystyle \sum _{f=1}^F{\mathrm{MSE}}_{\mathrm{f}}}\left(K,\alpha \right)$

$\mathrm{5:} \mathrm{Select} \mathrm{the} \mathrm{parameter} \mathrm{pair} \mathrm{with} \mathrm{the} \mathrm{minimum} \mathrm{average} \mathrm{reconstruction} \mathrm{error:} \left(K^{\ast },{\alpha }^{\ast }\right)=\arg \underset{K\in K_c,\alpha \in {\alpha }_c}{\min }\overline{\mathrm{MSE}}\left(K,\alpha \right)$

$\mathrm{6:} \mathrm{Use} \mathrm{the} \mathrm{selected} K^{\ast }=14$$\mathrm{and} {\alpha }^{\ast }=1000$ for leakage-free VMD feature construction, where VMD is applied only to the historical input window of each prediction sample.

As shown in

Table 1. VMD parameter selection settings and fold-wise validation results.

Panel A. VMD parameter search settings
Item		Setting
Parameter selection data		Training sequence
Sampling strategy		Last 3000 consecutive observations from the training sequence
Number of TSCV folds		3
Validation window length		750 observations
$K$ candidate range		3–14
$\alpha$ candidate set		{1000, 2000, 3000, 4000, 5000, 6000, 7000}
Selection criterion		Average reconstruction MSE across TSCV folds
Optimization method		Grid search
Final selected $K$		14
Final selected $\alpha$		1000
Average reconstruction MSE		388.5164
VMD parameter search time		48.2403 s
Panel B. Fold-wise VMD validation results
Fold	Validation window length	Selected $K$	Selected $\alpha$	Validation reconstruction MSE
Fold 1	750	14	1000	443.9469
Fold 2	750	14	1000	325.0876
Fold 3	750	14	1000	396.5147
Average	-	-	-	388.5164

, ($K$ = 14) and ($\alpha$ = 1000) were selected as the optimal parameter pair in all three validation folds. The average reconstruction MSE was 388.5164, and the VMD parameter search time was 48.2403 s. These fold-wise results indicate that the selected VMD parameters were stable under the time-series cross-validation scheme.

The reconstruction MSE was used because VMD serves as an unsupervised feature construction step before forecasting. This criterion was adopted to select a decomposition setting that preserves the main information of the original load sequence while generating multi-scale components. The forecasting contribution of the selected VMD features was further evaluated through the ablation study in Section 4.3.

4.2.2. BayesXGB Hyperparameter Optimization

The hyperparameters of XGBoost were optimized using Bayesian optimization. During the optimization process, five-fold time-series cross-validation was performed, and MAE was used as the optimization objective. The optimization objective is defined as

$$MAE={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N|y_i-{\widehat{y}}_i|}$$

(17)

where $y_i$ represents the actual electricity load value; ${\widehat{y}}_i$ denotes the predicted load value generated by XGBoost; and $N$ denotes the number of validation samples in each time-series cross-validation fold. The Bayesian optimization process searches for the hyperparameter combination that minimizes the cross-validation MAE. The optimized XGBoost hyperparameters are shown in

Table 2. Optimized hyperparameters of BayesXGB.

Optimization Parameter	Search Range	Final Optimization Result
colsample_bytree	(0.7, 1.0)	0.7000
gamma	(0, 5)	3.5828
learning_rate	(0.01, 0.3)	0.0593
max_depth	(6, 12)	9
min_child_weight	(1, 5)	5
n_estimators	(100, 500)	500
reg_alpha	(0, 10)	10.0000
reg_lambda	(1, 20)	16.2920
subsample	(0.7, 1.0)	0.7618

.

4.2.3. BiLSTM Residual Correction Settings

The BiLSTM residual correction module was configured as a stacked four-layer bidirectional LSTM structure. Each BiLSTM layer contains both a forward LSTM and a backward LSTM, and the output sequence of each layer is passed to the next BiLSTM layer. No residual skip connection is used between BiLSTM layers.

The BiLSTM residual correction module used a sequence length of 48, corresponding to one day of half-hour historical information. This differs from the 24-step input window used for the primary BayesXGB forecasting stage. The 24-step window was used to construct the first-stage forecasting samples, whereas the 48-step sequence was used to construct historical BayesXGB prediction and residual sequences for residual correction.

The input features of the BiLSTM residual correction module include historical BayesXGB predictions and their corresponding historical residuals. Therefore, the input tensor can be represented as $N\times 48\times 2$, where $N$ is the batch size, 48 is the sequence length, and 2 denotes the feature dimension.

The detailed BiLSTM parameter settings are shown in

Table 3. BiLSTM parameter settings.

Parameter	Parameter Setting
BiLSTM structure	4-layer Bidirectional LSTM
Layer connection	Output sequence of each BiLSTM layer is passed to the next layer
Hidden Units	256 (Layer 1), 128 (Layer 2), 96 (Layer 3), 64 (Layer 4)
Sequence Length	48
Input features	Historical BayesXGB prediction and historical residual
Input tensor shape	$N\times 48\times 2$
Optimizer	Adam
Initial Learning Rate	0.001
Batch Size	64
Training Epochs	100
Dropout Rate	0.4 for BiLSTM layers; 0.3 after the fourth BiLSTM layer; 0.2 for fully connected layers
Fully Connected Layer	Two dense layers with 64 and 32 units, ReLU activation

.

4.3. Ablation Study

To evaluate the contribution of each component in the proposed framework, four ablation models were constructed: BayesXGB, VMD–BayesXGB, BayesXGB–BiLSTM, and VMD–BayesXGB–BiLSTM. Here, BayesXGB denotes the Bayesian-optimized XGBoost regressor. The same optimized BayesXGB hyperparameter setting was used for all BayesXGB-based ablation models to ensure a consistent comparison. BayesXGB was used as the baseline model; VMD–BayesXGB introduced the VMD-based feature construction module; BayesXGB–BiLSTM introduced the BiLSTM-based residual correction module; and VMD–BayesXGB–BiLSTM represented the complete proposed model.

As shown in

Table 4. Results of the ablation study.

Model	RMSE	MAE	MAPE(%)	${\mathit{R}}^2$
BayesXGB	308.3163	231.0695	2.6434	0.9495
VMD–BayesXGB	143.8891	108.9592	1.2331	0.9890
BayesXGB–BiLSTM	187.2390	127.6176	1.4363	0.9814
VMD–BayesXGB–BiLSTM	122.1003	90.7386	1.0269	0.9921

, introducing VMD-based feature construction substantially improved the baseline BayesXGB model. Compared with BayesXGB, VMD–BayesXGB reduced RMSE, MAE, and MAPE by 53.33%, 52.85%, and 53.35%, respectively, while increasing $R^2$ from 0.9495 to 0.9890. This indicates that VMD-derived multi-scale features can effectively reduce the complexity of the original load sequence and improve the forecasting capability of BayesXGB.

The BiLSTM residual correction module also contributed to forecasting improvement. Compared with BayesXGB, BayesXGB–BiLSTM reduced RMSE, MAE, and MAPE by 39.27%, 44.77%, and 45.66%, respectively. This suggests that the residual sequence after the primary BayesXGB forecasting stage still contains useful temporal information that can be learned by the BiLSTM correction module.

The complete VMD–BayesXGB–BiLSTM model achieved the best overall performance among the ablation models, with an RMSE of 122.1003, an MAE of 90.7386, a MAPE of 1.0269%, and an $R^2$ of 0.9921. Compared with VMD–BayesXGB, the complete model further reduced RMSE, MAE, and MAPE by 15.14%, 16.72%, and 16.72%, respectively. Compared with the baseline BayesXGB model, the corresponding reductions reached 60.40%, 60.73%, and 61.15%, respectively. These results demonstrate that VMD-based feature construction and BiLSTM-based residual correction provide complementary improvements to the proposed forecasting framework.

Figure 7 provides a visual comparison of RMSE, MAE, MAPE, and $R^2$ among the ablation models. The complete VMD–BayesXGB–BiLSTM model consistently achieved the lowest RMSE, MAE, and MAPE, as well as the highest $R^2$, further confirming the effectiveness of the proposed hybrid structure.

Figure 8 presents the prediction curves of the ablation models on a representative test segment. The predicted curves generally follow the overall trend of the actual load. Although local differences exist among the models, the quantitative results in Table 4 show that VMD–BayesXGB–BiLSTM achieves the lowest overall prediction error, indicating more stable forecasting performance on the testing set.

4.4. Comparative Evaluation

To further evaluate the predictive performance of the proposed model, it was compared with several representative baseline models, including LSTM, VMD-LSTM, BayesXGB, VMD–BayesXGB, BiLSTM, VMD-BiLSTM, Attention-BiLSTM, VMD-Attention-BiLSTM, and DWT-LSTM. Here, BayesXGB denotes the XGBoost regressor whose hyperparameters were tuned by Bayesian optimization. The proposed model is denoted as VMD–BayesXGB–BiLSTM.

To ensure consistency, all models were evaluated using the same chronological training-testing split, input window length, forecasting horizon, and evaluation metrics. For the BayesXGB-based models, the same optimized hyperparameter setting was used. The LSTM-, BiLSTM-, and attention-based baseline models were trained using the same optimizer, batch size, maximum number of epochs, and early-stopping strategy. Therefore, the comparative evaluation focuses on the forecasting performance of different model structures under a consistent experimental setting.

The experimental results are shown in

Table 5. Experimental comparison of different models.

Model	RMSE	MAE	MAPE(%)	${\mathit{R}}^2$
LSTM	661.0743	518.8267	5.9788	0.7682
VMD–LSTM [30]	137.3648	104.7501	1.2105	0.9899
BayesXGB	308.3163	231.0695	2.6434	0.9495
VMD–BayesXGB	143.8891	108.9592	1.2331	0.9890
BiLSTM	673.1252	525.9073	5.9155	0.7596
VMD–BiLSTM [31]	185.1763	154.9294	1.7540	0.9730
Attention–BiLSTM	619.7955	487.2397	5.5849	0.7962
VMD–Attention–BiLSTM	168.8346	122.8087	1.3821	0.9849
DWT–LSTM [21]	296.7398	244.1437	2.7794	0.9533
VMD–BayesXGB–BiLSTM (Proposed)	122.1003	90.7386	1.0269	0.9921

and Figure 9.

As shown in Table 5, the proposed VMD–BayesXGB–BiLSTM model achieved the best performance among all compared models, with an RMSE of 122.1003, an MAE of 90.7386, a MAPE of 1.0269%, and an $R^2$ of 0.9921. These results indicate that the proposed hybrid framework can more accurately capture the nonlinear, non-stationary, and temporal characteristics of electricity load data.

The standalone recurrent neural network models, including LSTM, BiLSTM, and Attention–BiLSTM, produced relatively large prediction errors. In contrast, their VMD-based counterparts achieved much better performance, indicating that VMD-based feature construction is effective for reducing the complexity of the original load sequence and extracting useful multi-scale information for downstream forecasting models.

Among all baseline models, VMD–LSTM achieved the strongest performance, with an RMSE of 137.3648, an MAE of 104.7501, a MAPE of 1.2105%, and an $R^2$ of 0.9899. Nevertheless, the proposed VMD–BayesXGB–BiLSTM model further reduced RMSE, MAE, and MAPE by 11.11%, 13.38%, and 15.17%, respectively, compared with VMD-LSTM. This demonstrates that combining BayesXGB forecasting with BiLSTM residual correction can provide additional improvement beyond VMD-based recurrent forecasting alone.

Compared with VMD–BayesXGB, the proposed model further reduced RMSE from 143.8891 to 122.1003, MAE from 108.9592 to 90.7386, and MAPE from 1.2331% to 1.0269%. The corresponding reductions were 15.14%, 16.72%, and 16.72%, respectively. This improvement demonstrates that the residual sequence after the primary BayesXGB forecasting stage still contains useful temporal information, which can be further modeled by the BiLSTM residual correction module.

The proposed model also outperformed other decomposition-based and hybrid baselines. Compared with VMD–Attention–BiLSTM, it reduced RMSE, MAE, and MAPE by 27.68%, 26.11%, and 25.70%, respectively. Compared with DWT–LSTM, it reduced RMSE, MAE, and MAPE by 58.85%, 62.83%, and 63.05%, respectively. These comparisons show that the proposed framework provides more accurate forecasting performance than both attention-enhanced recurrent models and DWT-based recurrent models on the tested dataset.

Overall, the comparative results demonstrate that the proposed VMD–BayesXGB–BiLSTM model achieves superior predictive performance on the tested Australian electricity load dataset. However, since the experiments were conducted on a single dataset, broader generalizability should be further verified using additional datasets from different regions or power systems in future work.

4.5. Computational Cost Analysis

In addition to forecasting accuracy, computational cost is also an important factor for practical short-term load forecasting applications. Therefore, the training time, total inference time, inference time per sample, and number of trainable neural parameters were compared among all models used in the comparative evaluation. The results are shown in

Table 6. Computational cost comparison of different models.

Model	Training Time (s)	Total Inference Time (s)	Inference Time per Sample (ms)	Trainable Neural Parameters
LSTM	1091.7144	4.4910	0.2562	132,481
VMD–LSTM	1397.0314	4.4148	0.2519	139,649
BayesXGB	30.0264	0.0367	0.0021	0
VMD–BayesXGB	379.6089	0.1380	0.0078	0
BiLSTM	1671.0437	8.7020	0.4965	346,305
VMD–BiLSTM	4820.4418	25.5631	1.4586	360,641
Attention–BiLSTM	5260.5649	24.5254	1.3994	350,018
VMD–Attention–BiLSTM	2899.6422	9.1550	0.5224	364,354
DWT–LSTM	709.6795	4.4305	0.2528	134,017
VMD–BayesXGB–BiLSTM (Proposed)	3137.0643	16.7080	0.9559	1,602,049

.

It should be noted that the values in Table 6 mainly report the model training and inference costs after feature construction. In this study, VMD and DWT feature construction were treated as offline preprocessing steps. For leakage-free VMD feature construction, decomposition was applied independently to each historical input window to avoid the use of future information. Therefore, the offline feature generation cost should be considered separately from the online inference cost.

As shown in Table 6, BayesXGB required the lowest computational cost, with a training time of 30.0264 s and an inference time of 0.0021 ms per sample. After introducing VMD-based feature construction, VMD–BayesXGB still maintained a very low inference time of 0.0078 ms per sample, although its training time increased to 379.6089 s. This indicates that VMD-derived features can greatly improve forecasting accuracy while preserving low online inference cost for the BayesXGB-based model.

The recurrent neural network models required higher computational costs because of their trainable neural parameters. In particular, VMD-BiLSTM and Attention-BiLSTM required 4820.4418 s and 5260.5649 s for training, respectively, with inference times above 1 ms per sample. In comparison, the proposed VMD–BayesXGB–BiLSTM model required 3137.0643 s for training and 0.9559 ms per sample for inference. Although the proposed model contains 1,602,049 trainable neural parameters and is more complex than the tree-based models, its online inference time remains below 1 ms per sample.

The additional computational cost of the proposed model mainly comes from the BiLSTM residual correction module. However, this cost is accompanied by clear accuracy improvements. Compared with VMD–BayesXGB, the proposed model reduced RMSE from 143.8891 to 122.1003, MAE from 108.9592 to 90.7386, and MAPE from 1.2331% to 1.0269%. Therefore, the proposed model provides a trade-off between higher forecasting accuracy and increased computational complexity.

Figure 10 further illustrates the accuracy–cost trade-off of different forecasting models by comparing RMSE with inference time per sample. In this figure, models located closer to the lower-left region are preferable because they achieve lower prediction error and lower online inference cost simultaneously. BayesXGB and VMD–BayesXGB have the lowest inference time, but their RMSE values are higher than that of the proposed model. Several recurrent neural network baselines require comparable or higher inference time while still producing larger prediction errors. The proposed VMD–BayesXGB–BiLSTM model achieves the lowest RMSE among all compared models while maintaining an inference time below 1 ms per sample, indicating a favorable balance between forecasting accuracy and online inference efficiency.

From a practical perspective, VMD–BayesXGB may be more suitable for scenarios with strict computational constraints because it provides a large improvement over BayesXGB while maintaining very low inference time. In contrast, the complete VMD–BayesXGB–BiLSTM model is more suitable for applications where higher forecasting accuracy is prioritized and sufficient computational resources are available. Future work may focus on reducing the computational burden of the residual correction module and the VMD feature construction stage, for example, by using lightweight recurrent structures, more efficient decomposition strategies, or model compression techniques.

5. Conclusions

To improve short-term electricity load forecasting under nonlinear, volatile, and multi-scale load conditions, this study proposed a VMD–BayesXGB–BiLSTM hybrid forecasting model. In the proposed framework, abnormal values were detected using the $3\sigma$ criterion and corrected by cubic spline interpolation, while feature standardization was applied to reduce the influence of different variable scales. VMD parameters were selected using time-series cross-validation within the training sequence, and leakage-free VMD features were generated from historical input windows. BayesXGB was then used as the primary forecasting model, and a BiLSTM residual correction module was introduced to further model temporal patterns in the prediction residuals.

Experimental results on an Australian electricity load dataset show that the proposed model achieved an RMSE of 122.1003, an MAE of 90.7386, a MAPE of 1.0269%, and an $R^2$ of 0.9921, outperforming all compared baseline models. The ablation study confirms that both VMD-based feature construction and BiLSTM-based residual correction contribute to the forecasting improvement. Compared with BayesXGB, the complete model reduced RMSE, MAE, and MAPE by 60.40%, 60.73%, and 61.15%, respectively. The comparative evaluation further shows that the proposed model performed better than VMD-LSTM, VMD-BiLSTM, VMD-Attention-BiLSTM, and DWT-LSTM. In addition, the online inference time of the proposed model remained below 1 ms per sample, indicating a practical balance between forecasting accuracy and inference efficiency.

Future work will focus on testing the model on additional load datasets, reducing the computational cost of VMD feature construction and BiLSTM residual correction, and further evaluating statistical robustness through repeated runs and significance testing.