Quarter-Hourly Power Load Forecasting Based on a Hybrid CNN-BiLSTM-Attention Model with CEEMDAN, K-Means, and VMD

Abstract

Accurate long-term power load forecasting in the grid is crucial for supply–demand balance analysis in new power systems. It helps to identify potential power market risks and uncertainties in advance, thereby enhancing the stability and efficiency of power systems. Given the temporal and nonlinear features of power load, this paper proposes a hybrid load-forecasting model using attention mechanisms, CNN, and BiLSTM. Historical load data are processed via CEEMDAN, K-means clustering, and VMD for significant regularity and uncertainty feature extraction. The CNN layer extracts features from climate and date inputs, while BiLSTM captures short- and long-term dependencies from both forward and backward directions. Attention mechanisms enhance key information. This approach is applied for seasonal load forecasting. Several comparative experiments show the proposed model’s high accuracy, with MAPE values of 1.41%, 1.25%, 1.08% and 1.67% for the four seasons. It outperforms other methods, with improvements of 0.25–2.53 GWh² in MSE, 0.15–0.1 GWh in RMSE, 0.1–0.74 GWh in MAE and 0.22–1.40% in MAPE. Furthermore, the effectiveness of the data processing method and the impact of training data volume on forecasting accuracy are analyzed. The results indicate that decomposing and clustering historical load data, along with large-scale data training, can both boost forecasting accuracy.

1. Introduction

With the transformation of the global energy structure and the advancement of smart grid construction, power load forecasting is encountering new opportunities and challenges. As a core component of the energy management system, accurate load forecasting is crucial for building a new power system. It plays an irreplaceable role in achieving efficient energy dispatching, reducing operational costs, and ensuring power grid stability [1]. Especially against the backdrop of the continuous increase in the proportion of renewable energy, power load forecasting has become crucial for anticipating grid supply–demand balance and alleviating the volatility from intermittent energy integration [2]. It also plays an indispensable role in energy sector development and policy formulation [3].

Focusing on temporal and nonlinear characteristics, power load forecasting models are generally categorized into two types. The first is time-series analysis, which includes linear regression [4], exponential smoothing [5], Kalman filtering [6], multiple linear regression [7,8], Fourier expansion models [9], and autoregressive integral moving average (ARIMA) models [10], etc. Time-series analysis uses past and current load values from stochastic time series to predict future loads, considering temporal relationships in the data but having limited predictive power for nonlinear relationships. The second type is machine learning methods, such as BP neural networks [11,12], grey projection [13], and random forests [14]. Time-series analysis uses past and current load values from stochastic time series to predict future loads, considering temporal relationships in the data but having limited predictive power for nonlinear relationships. The second type is machine learning methods, such as BP neural networks [15,16].

The long short-term memory (LSTM) network, effective in nonlinear tasks like language modeling [17], machine translation [18], and speech recognition [19], is increasingly used in load forecasting. It handles temporal and nonlinear data relationships and outperforms traditional methods by capturing long-term dependencies in power data [20,21]. This makes LSTM suitable for addressing the high volatility and uncertainty in power loads, which is crucial as renewable energy penetration increases [22]. However, it struggles to uncover useful information and relationships in non-continuous data [23].

Hybrid architectures combining convolutional neural networks (CNNs) and LSTM models are gaining attention. They integrate CNNs’ spatial feature extraction with LSTMs’ temporal dependency modeling, enabling the handling of both spatial and temporal features. Tae-Young Kim et al. [15] and He et al. [24] show CNN-LSTM networks can effectively extract complex energy consumption features. Bouktif Salah et al. improved LSTM model accuracy by 28% in French grid load forecasting by introducing spatial correlation indicators [21]. Song Jiancai et al. used MAPE as the main evaluation index in regional heating load forecasting, keeping MAPE within 3.1–4.1% with their CNN-LSTM model [25]. Sekhar Charan et al. developed the GWO-CNN-BiLSTM model for weekly load forecasting with multi-metric optimization [26].

Recent studies indicate that hybrid neural network models optimized by attention mechanisms can significantly enhance prediction accuracy and interpretability [27], thus providing novel technical approaches for addressing load mutations. Others developed the CNN-LSTM-A model, which refines LSTM output weights through attention mechanisms, leading to marked improvements in accuracy [16]. Shengdong Du et al. introduced a temporal attention encoder–decoder model. This model employs bidirectional LSTM (BiLSTM) and attention mechanisms to capture long-term dependencies in multivariate time series [28]. Zhou Ning et al. [29] verified that the CNN-LSTM-attention model is superior to LSTM, GRU, CNN-LSTM, CNN-LSTM with autoencoder and parallel CNN-LSTM attention models in predicting fluctuating data.

Despite these advancements, power load forecasting still faces several challenges. The efficiency of deep learning models remains highly dependent on the quality and quantity of historical data. Nivethitha Somu et al. observed that while deep learning frameworks like CNN-LSTM can effectively capture spatiotemporal dependencies in energy consumption, their accuracy is constrained by the sampling frequency and completeness of input data [30]. Despite these advancements, power load forecasting still faces several challenges. The efficiency of deep learning models remains highly dependent on the quality and quantity of historical data. Nivethitha Somu et al. observed that while deep learning frameworks like CNN-LSTM can effectively capture spatiotemporal dependencies in energy consumption, their accuracy is constrained by the sampling frequency and completeness of input data. Lingling Lv et al. utilized Variational Mode Decomposition (VMD) to decompose load data into intrinsic mode functions across different scales [31]. Aksan, F. et al. discovered that the combined model of VMD-CNN-GRU shows good predictive performance under seasonal conditions [32]. Bixuan Gao et al. proposed a CEEMDAN (Complete Ensemble Empirical Mode Decomposition with Adaptive Noise)-CNN-LSTM model designed for feature alignment in historical data [33].

From a temporal perspective, power load forecasting is divided into short-term and long-term categories [3,34]. As the proportion of renewable energy in power systems increases, long-term load forecasting becomes increasingly critical. It plays a key role in effective renewable energy integration, determining a reasonable access scale and distribution, and assessing potential risks in power markets. This paper introduces a data processing method that combines CEEMDAN, K-Means, and VMD. This method is applied to historical power load data from Guangzhou’s power grid, aiming to reduce noise and extract salient features. CEEMDAN decomposes historical data into multiple frequency components. Sample entropy and K-means are utilized successively to evaluate and cluster these components’ regularity and uncertainty. VDM is utilized to further extract the modal components clusters with low frequency and low sample entropy. In this way, the significant regularity of historical data in terms of seasonal, weekly and holiday characteristics and the uncertainty in terms of hourly randomness and diurnal variation characteristics are extracted, which is more conducive to the model’s capturing of data characteristics. This study also proposes a CNN-BiLSTM model based on attention mechanisms for long-term load forecasting. The CNN layers extract spatial features from training data, the BiLSTM layers capture short- and long-term dependencies in sequential data from both directions, and the attention mechanism enhances key information impacts. The proposed model’s performance is compared with those of other deep learning methods, showing its effectiveness in improving forecasting accuracy across different seasons.

The main contributions of this paper are the following: (1) proposing a combined data processing approach, (2) developing the CNN-BiLSTM-attention model, (3) comparing the model’s effectiveness with other methods, (4) validating the data processing method’s effectiveness in improving forecasting accuracy, and (5) by grouping the training and test sets in different proportions, the impact of data quantity on model feature learning is further studied.

The rest of this paper is organized as follows: Section 2 details the data processing methods and steps. Section 3 presents the CNN-BiLSTM-attention model. Section 4 discusses the case study results. Finally, Section 5 summarizes the research.

2. Data Processing

This study uses the hourly historical power load data of Guangzhou from 2020, with a total of 8784 data points. The task of this study is to improve the robustness and accuracy of power load forecasting. Due to the influence of day–night differences and season, the power load shows multi-frequency characteristics. At the same time, user behavior and environmental factors also cause the power load to present uncertainty. Using CEEMDAN or VMD to decompose the data into simple frequency features and predict the modal decomposition is an effective method to ensure the function fitting and subsequent convergence of the prediction model. By introducing adaptive noise to reduce the modal effect, CEEMDAN can improve the accuracy of nonlinear and non-stationary signal decomposition. VMD has a good anti-noise ability and can effectively capture the frequency characteristics of signals. The combination of CEEMDAN and VMD can decompose the load sequence into trend terms and multiple random fluctuation terms, which can further reduce the complexity of the sequence, enhance the robustness of the data, and better preserve the main characteristics of the signal.

2.1. Complete Ensemble Empirical Mode Decomposition Adaptive Noise (CEEMDAN)

CEEMDAN is a signal processing technique proposed by Torres et al. in 2011 [35]. By adding white noise step by step, averaging step by step and introducing adaptive noise, CEEMDAN can effectively solve the common mode aliasing problem of traditional empirical mode decomposition (EMD) when dealing with nonlinear and non-stationary signals. CEEMDAN processes signals into multiple Intrinsic Mode Functions (IMFs) as follows:

(1)

Add Gaussian white noise to the original signal to initialize the original signal,

$$Y_n(t)=Y(t)+{\epsilon }_0{\lambda }_n(t),n=1,2,\dots \dots ,N$$

(1)

where ${\epsilon }_0$ is the weight coefficient of Gaussian white noise; ${\lambda }_n(t)$ is the Gaussian white noise at the n-th processing; N represents the number of realizations.

(2)

Perform the EMD decomposition on all preprocessing sequences, and then the first IMF ${\overline{Imf}}_1(t)$ is calculated by averaging the decomposed components,

$${\overline{Imf}}_1(t)={\displaystyle \frac{1}{N}}{\displaystyle \sum _1^{N}IM{F}_1^i(t)}$$

(2)

$$r_1(t)=Y(t)-\overline{Im{f}_1}(t)$$

(3)

where $r_1(t)$ is the first residual.

(3)

Gaussian white noise is added to the residual sequence to construct a new sequence, $r_1(t)+{\epsilon }_{1}EM{D}_1({\lambda }_n(t))$. After the further EMD decomposition of this sequence, the second IMF ${\overline{Imf}}_2(t)$ component and residual $r_2(t)$ are obtained by calculating its mean value,

$${\overline{Imf}}_2(t)={\displaystyle \frac{1}{N}}{\displaystyle \sum _1^{N}EM{D}_1(r_1(t)+{\epsilon }_{1}EM{D}_1({\lambda }_n(t)))}$$

(4)

$$r_2(t)=r_1(t)-\overline{Im{f}_2}(t)$$

(5)

(4)

The residual of stage m-th is shown as

$$r_m(t)=r_{m-1}(t)-\overline{Im{f}_m}(t)$$

(6)

(5)

Performing the EMD decomposition on the m + 1-th sequence $r_m(t)+{\epsilon }_{m}EM{D}_m({\lambda }_n(t))$ to obtain the m + 1-th IMF ${\overline{Imf}}_{m+1}(t)$ yields

$${\overline{Imf}}_{m+1}(t)={\displaystyle \frac{1}{N}}{\displaystyle \sum _1^{N}EM{D}_1(r_m(t)+{\epsilon }_{m}EM{D}_m({\lambda }_n(t)))}$$

(7)

where $EM{D}_m(\cdot )$ is the m-th IMF model obtained by EMD decomposition.

(6)

Repeat the above steps until the decomposition satisfies the stop condition, and the final residual sequence is

$${\displaystyle \sum _{t=0}^T\frac{{\left|r_{m-1}(t)-r_m(t)\right|}^2}{r_{m-1}^2}\le S{D}_m}$$

(8)

where T represents the length of the sequence $Y_n(t)$; $r_m(t)$ is the sequence after m-th decomposition, and the experience value $S{D}_m$ is set to 0.2 [36].

Through the above steps, the Guangzhou historical power load sequence is decomposed into 14 IMFs, which includes high frequency and low frequency, as shown in the Figure 1. IMF1 to IMF3 belong to high-frequency modes, mirroring short-term fluctuations and noise in the power load sequence, likely derived from sudden load changes. IMF4 to IMF6 belong to medium-frequency modes, contain multiple relatively stable frequencies, and show regular fluctuations of power load on a medium timescale. The rest are low-frequency IMFs, reflecting long-term trends and slow variations of power load, like gradual fluctuations and overall upward or downward trends caused by factors such as seasonality. While low-frequency IMFs have relatively few frequency components, their energy share is higher.

2.2. K-Means Clustering

This study applies CEEMDAN to decompose the original complex signal into multiple modal components with different frequencies and characteristics, and effectively extract different levels of information in the signal. K-means is a clustering algorithm that can divide data points into K clusters. The data points within each cluster are as similar as possible, and the data points between different clusters are as different as possible. To classify the regular characteristics of the IMFs, the regularity and uncertainty of the 14 IMFs in Figure 1 are first evaluated using the sample entropy method [37]. Then, we perform K-means clustering on the sample entropy values to classify the IMFs and reduce dimensionality. Combining CEEMDAN and K-means to process the data can provide better input for subsequent analysis and modeling, and improve the robustness and generalization ability of the model. The core of K-means is to optimize clustering results by minimizing the Within-Cluster Sum of Squares (WCSS). The objective function of K-means clustering is defined as

$$J={\displaystyle \sum _{j=1}^K{\displaystyle \sum _{i=1}^n{1}_{\left\{x_i\in Cl{u}_j\right\}}\cdot d{(x_i,cl{u}_j)}^2}}$$

(9)

where K is the number of predefined clusters; $Cl{u}_j$ presents the k-th cluster; $x_i$ is the j-th data point; $cl{u}_j$ is the center of the k-th cluster; $1_{\left\{x_i\in Cl{u}_j\right\}}$ represents the indicator function, which is 1 when belonging to a cluster and 0 otherwise. $d(x_i,cl{u}_j)$ represents the distance between $x_i$ and $cl{u}_j$, which is used to measure the similarity between the data points, usually using the squared Euclidean distance,

$$d(x_i,cl{u}_j)={\displaystyle \sum _{k=1}^d{(x_{i,k}-cl{u}_{j,k})}^2}$$

(10)

The clustering steps of K-means are as follows:

(1)

Initialization. Randomly select K data points as the initial cluster centers $cl{u}_1$,$cl{u}_2$,…, $cl{u}_k$;

(2)

Assign data points to the nearest cluster. For each $x_i$, calculate its distance to all cluster centers and assign it to the nearest cluster;

(2)

Update cluster center. For each $Cl{u}_j$, update the cluster centroid by calculating the mean of all $x_i$ in the cluster,

$$cl{u}_j={\displaystyle \frac{1}{\left|Cl{u}_j\right|}}{\displaystyle \sum _{x_i\in Cl{u}_j}{x}_i}$$

(11)

where $\left|Cl{u}_j\right|$ is the number of $x_i$ in $Cl{u}_j$.

The sample entropy values of 14 IMFs are shown in

Table 1. The sample entropy values of 14 IMFs.

IMF	Sample Entropy	IMF	Sample Entropy
1	1.5933	8	0.1343
2	1.8791	9	0.0618
3	1.8332	10	0.0311
4	0.7258	11	0.0121
5	0.5189	12	0.0041
6	0.5795	13	0.0017
7	0.2996	14	0.0002

. A smaller sample entropy value indicates that the IMF has strong regularity or predictability. The Silhouette Coefficient and Elbow methods are combined to select the optimal K value. In Figure 2a, the Silhouette Coefficient exceeds 0.9 for K = 2 and K = 3, indicating well-separated clusters. Figure 2b shows that when K = 3, the Sum of Squared Errors (SSE) still shows a rapid decline. However, as the K value continues to increase, the decline in SSE sharply decreases and the curve becomes flat. Based on these findings, K = 3 is chosen as the optimal value. Figure 2c presents the clustering results for the sample entropy of the 14 IMFs, where the cluster centers are 0.0704, 0.6501 and 1.7713, respectively.

By adding up the IMFs contained in each cluster individually, the 14 IMFs in Figure 1 are divided into three clusters as shown in Figure 3. Clu₁ is the largest cluster with an aggregation of eight IMFs. Clu₂ and Clu₃ each contain three IMFs.

2.3. Variational Mode Decomposition (VMD)

Finally, VMD is used to further decompose the Clu₁ in Figure 2 to effectively extract the key features of the signal. VMD is a non-recursive signal decomposition method based on variational theory proposed by Konstantin Dragomiretskiy and Dominique Zosso in 2014 [38], which has excellent performance in the decomposition of low-frequency components. The VMD algorithm shows little influence on the initial conditions and has strong robustness. The central idea of VMD is to assume that the original signal $u(t)$ can be decomposed into the sum of k IMFs, with each $u_k(t)$ having a specific central frequency ${\omega }_t$ and bandwidth $△k$. The $u_k(t)$ can be found by minimizing the total $△k$,

$$\underset{\left\{u_k,{\omega }_k\right\}}{\min }{\displaystyle \sum _{k=1}^K{‖{\partial }_t\left[\left(\delta (t)+\frac{j}{\pi t}\right)\ast u_k(t)\right]e^{-j{\omega }_{k}t}‖}_2^2}$$

(12)

$$\mathrm{s}.\mathrm{t}.{\displaystyle \sum _{k=1}^K{u}_k(t)=u(t)}$$

(13)

where ${\partial }_t$ represents the differential operation; $\ast$ represents the convolution operation; $\delta (t)$ is the Dirac function; j is an imaginary unit; $e^{-j{\omega }_{k}t}$ is the modulation operation.

VMD transforms the constrained variational problem into an unconstrained variational problem by using the augmented Lagrange function. The extended Lagrange expression is as follows:

$$\mathcal{L}(u_k,{\omega }_k,\lambda )=\alpha {\displaystyle \sum _{k=1}^K{‖{\partial }_t\left[\left(\delta (t)+\frac{j}{\pi t}\right)\ast u_k(t)\right]e^{-j{w}_{k}t}‖}_2^2}+{‖u(t)-{\displaystyle \sum _{k=1}^K{u}_k(t)}‖}_2^2+〈\lambda ,u(t)-{\displaystyle \sum _{k=1}^K{u}_k(t)}〉$$

(14)

where $\alpha$ is the quadratic penalty factor; $\lambda$ is the Lagrange multiplication operator.

Iterative optimization is performed by alternating direction multiplier method (ADMM), updating ${\widehat{u}}_k^{n+1}(\omega )$ and ${\omega }_k^{n+1}$ for all $\omega \ge 0$:

$${\widehat{u}}_k^{n+1}(\omega )={\displaystyle \frac{\widehat{f}(\omega )-{\displaystyle {\sum }_{i\ne k}{\widehat{u}}_i^n(\omega )}+{\widehat{\lambda }}^n(\omega )/2}{1+2\alpha {({\omega }_k^n-\omega )}^2}}$$

(15)

$${\omega }_k^{n+1}={\displaystyle \frac{{\displaystyle \int \omega {\left|{\widehat{u}}_k^{n+1}(\omega )\right|}^{2}d\omega }}{{\displaystyle \int {\left|{\widehat{u}}_k^{n+1}(\omega )\right|}^{2}d\omega }}}$$

(16)

The ${\widehat{\lambda }}^{n+1}(\omega )$ can be updated as

$${\widehat{\lambda }}^{n+1}(\omega )={\widehat{\lambda }}^n(\omega )+\tau \left(\widehat{u}(\omega )-{\displaystyle \sum _{k=1}^K{\widehat{u}}_k^{n+1}(\omega )}\right)$$

(17)

where $\widehat{f}$ is the frequency domain representation of $u(t)$; ${\widehat{u}}_k$ is the frequency domain representation of the IMF; ${\omega }_k^n$ is the estimated value of the center frequency of the k-th mode at the n-th iteration; ${\widehat{\lambda }}^n(\omega )$ is the Fourier transform of the Lagrange multiplier at the nth iteration; $\tau$ represents the noise tolerance.

The detailed decomposition steps of VMD for $u(t)$ are as follows:

(1)

Initialize $u(t)$, ${\omega }_t$ and ${\lambda }^1$;

(2)

Set n = n + 1;

(3)

k = k + 1, and for all $\omega \ge 0$, update ${\widehat{u}}_k^{n+1}(\omega )$ and ${\omega }_k^{n+1}$;

(4)

Update ${\widehat{\lambda }}^{n+1}(\omega )$;

(5)

Repeat steps (2)–(4) until the following stop conditions are met—

$${\displaystyle \sum _{k=1}^K\left(\frac{{‖{\widehat{u}}_k^{n+1}-{\widehat{u}}_k^n‖}_2^2}{{‖{\widehat{u}}_k^n‖}_2^2}\right)}<\gamma$$

(18)

As shown in Figure 3, Clu₁ contains multiple low-frequency and low-sample-entropy IMFs with significant regularity. Considering that the long-term characteristics selected by the model include week and holidays, and the power load has strong seasonal characteristics, the initial decomposition layer number of VMD is set to 3 in order to capture week, holiday, and seasonal features. This operation is carried out by using the VMD function in Octave software 10.1.0. The penalty factor alpha is set to 2500 and the convergence threshold is set to 1 × 10⁻⁶.

Through the above steps, Clu₁ is decomposed into three IMFs by the VMD algorithm. Combined with Clu₂ and Clu₃, the power load sequence is finally decomposed into five components, as shown in Figure 4.

Figure 4 shows that after CEEMDAN, K-means and VMD processing, the week characteristics (Data 1), holiday characteristics (Data 2) and seasonal characteristics (Data 3) of the power load sequence are extracted. Load uncertainty is divided into hourly random uncertainty (Data 4) and diurnal uncertainty (Data 5).

2.4. Feature Selection and Normalization

In this work, we study the hourly power load sequence of Guangzhou. We select the most commonly used features in the load forecasting literature to obtain attributes from the data. Guangzhou is a subtropical monsoon climate, which has a high demand for air conditioning cooling in summer and heating in winter. Thus, temperature is chosen as a feature. The above power load decomposition results show that the power load has obvious characteristics related to week, holiday and season. One hour in a day (values from 1 to 24), one day in a week (values from 1 to 7), and holiday indicators (0, 1 and 2 represent non-holidays, common legal holidays, and Spring Festival, respectively) are also considered as features. Through the above assignment, the temperature, date and other features of 2020 are composed into a 4-dimensional feature data matrix.

As can be seen from Figure 3, the power load in Guangzhou has obvious seasonal characteristics. Therefore, this paper separately models the four seasons to improve the prediction accuracy. The feature data matrix and each load IMF in Figure 3 are divided into four seasons, respectively: spring, summer, autumn and winter. Then, the feature matrices of each season and each load IMF are combined to form the model input matrix. Thus, the forecast model for each season contains five 5-dimensional input matrices. For each season, each input matrix is modeled in turn, and then the prediction performance is evaluated after summing the forecast results. Max–min scaling is used to normalize input data to eliminate dimensional effects between data features,

$$X_{norm}={\displaystyle \frac{X-X_{\min }}{X_{\max }-X_{\min }}}$$

(19)

where $X_{norm}$ is the data after normalization; $X$ is the input raw data set; $X_{\min }$ and $X_{\max }$ are the minimum and maximum values of the original data, respectively.

3. Methods

CNN demonstrates a robust capability in spatial feature extraction. LSTM is particularly effective in extracting sequential patterns and capturing both short-term and long-term dependencies of sequence data. The integration of CNN and LSTM has been proven to be highly effective in capturing spatial correlations and temporal trends, making it a powerful approach for complex sequence data analysis. BiLSTM can simultaneously process sequence data in both forward and backward directions, significantly enhancing the accuracy of sequence modeling tasks. By incorporating an attention mechanism and dynamically optimizing the output weights of BiLSTM through weight allocation, the influence of key information can be enhanced, while addressing the limitations of BiLSTM in processing long sequence data. Therefore, a hybrid model leveraging the synergistic advantages of CNN, BiLSTM, and attention mechanisms is expected to improve the accuracy of load prediction. This section provides a detailed description of the components of the CNN-BiLSTM-attention architecture.

3.1. Convolutional Neural Network (CNN)

Introduced by [39], CNN is a DNN model primarily inspired by biological processes [40]. CNN serves as a powerful tool for automatic feature extraction and expression, capable of achieving arbitrary function mapping from input to output. While CNN is widely used in image vision tasks, its applications have also expanded to time-based serial data, as evidenced by recent studies [41,42]. CNNs typically use multiple building blocks such as convolutional, pooling, and fully connected layers to learn the spatial hierarchy of features (as shown in Figure 5). The convolutional layer extracts input features by performing multi-depth convolutions with kernels and applying nonlinear mapping via activation functions, allowing the network to focus on local regions. The pooling layer summarizes features from convolutions, reducing data dimensionality through max or average pooling to minimize computations. Finally, the fully connected layer, integrated as a BP neural network at the network’s base, merges pooled features to learn global input characteristics, and generates the output [43].

3.2. Bidirectional Long-Term Memory Network (BiLSTM)

Recurrent Neural Networks (RNNs) are widely used for analyzing and predicting sequential data. However, the vanishing gradient problem limits their ability to capture long-term dependencies [44], while the exploding gradient problem can lead to model instability and failure to converge. To address these issues, Sepp Hochreiter and Jürgen Schmidhuber proposed LSTM networks in 1997. Subsequently, Felix Gers and colleagues introduced the forget gate mechanism to further enhance LSTM’s capabilities [45]. The improved LSTM architecture leverages a gating mechanism—comprising the input gate, forget gate, and output gate—along with a cell state update and self-connected memory cells to capture complex temporal dependencies in both long and short time-series data. This design aims to overcome the limitations of traditional RNNs, particularly their susceptibility to vanishing and exploding gradients when processing long sequences, as well as their inability to effectively model long-term dependencies.

The input gate $i_t$, forget gate $f_t$, and output gate $o_t$ regulate the flow of information by determining what to add or remove from the cell state [46,47] (as shown in Figure 6). The memory cell retains temporal state information from both the current and previous time steps. For a given input $x_t$ at time step t, the forward computation process of LSTM can be expressed as:

$$i_t=\sigma (W_i\left[h_{t-1},x_t\right]+b_i)$$

(20)

$$f_t=\sigma (W_f\left[h_{t-1},x_t\right]+b_f)$$

(21)

$$o_t=\sigma (W_o\left[h_{t-1},x_t\right]+b_o)$$

(22)

$${\widehat{C}}_t=tanh(W_C\left[h_{t-1},x_t\right]+b_C)$$

(23)

$$C_t=f_t\ast C_{t-1}+i_t\ast {\widehat{C}}_t$$

(24)

$$h_t=o_t\ast tanh(C_t)$$

(25)

where $\sigma$ is the activation function; $tanh$ denotes hyperbolic tangent activation; $W_i$, $W_f$, $W_o$, $W_C$ and $b_i$, $b_f$, $b_o$, $b_C$ are the weight matrices and bias of $i_t$, $f_t$, $o_t$ and cell state update ${\widehat{C}}_t$, respectively; $C_{t-1}$ and $C_t$ represent the previous and current cell states, respectively; $h_t$ is the hidden state and is updated according to the $o_t$ and the updated memory unit state.

Since inputs in LSTM are processed in a precise temporal order, future information cannot be utilized. It is critical to understand contextual information in many sequence modeling tasks (such as natural language processing and speech recognition). In 2005, Graves proposed combining LSTM with Bidirectional Recurrent Neural Networks (BRNNs) to form BiLSTM [48]. A BiLSTM consists of two LSTM layers: one processes the forward sequence (from the first element to the last), and the other processes the backward sequence (from the last element to the first). The outputs of these two layers are concatenated at each time step or at the end of the sequence, enabling the model to leverage both past and future information and capture contextual relationships more comprehensively. The hidden layer output of the BiLSTM model $h_t$ at time step t is

$$h_t=\left[h_t^f;h_t^b\right]$$

(26)

where $h_t^f$ and $h_t^b$ are forward and reverse hidden sequences, respectively. Figure 7 depicts the design of BiLSTM prediction. The output is represented by $y_t$.

3.3. Attention Mechanism

In practical problems, some features are more important than others. For instance, in electricity load forecasting, certain information like weather and day types may be more significant. Traditional neural networks treat all inputs equally, making it hard to focus on key information in the input sequence. Attention mechanisms, inspired by human visual attention, enhance important information in deep neural networks. They assign probabilities to information instead of using random weights, addressing information loss in LSTM and BiLSTM caused by long sequences. The structure of attention models is shown in Figure 8.

The attention mechanism assigns different weights to various inputs to discard irrelevant information, amplify key information and enhance model accuracy. This paper focuses on soft attention mechanisms that emphasize regions or channels. In soft attention, inputs are projected into multiple subspaces, each with different weights that can be learned or preset based on the problem. During each computation, these weights are recalculated to adapt to the current task, suppressing irrelevant features. In Figure 7, $a_t$ is the attention weight assigned to the BiLSTM hidden layer output $h_t$; higher $a_t$ values indicate more relevant historical features. The optimized BiLSTM output after attention is denoted as $c_t$. The attention layer weights are computed as follows:

$$e_t={\nu }^{T}tanh(W_e{h}_t+b_e)$$

(27)

$$a_t={\displaystyle \frac{\exp (e_t)}{{\displaystyle {\sum }_{j=0}^t\exp (e_j)}}}$$

(28)

$$c_t={\displaystyle \sum _{t=0}^t{a}_t{h}_t}$$

(29)

where $e_t$ represents the attention scoring function, which quantifies the correlation between the power load at time t and the output vector of the BiLSTM layer; ${\nu }^T$ is a learnable vector; $b_e$ is the bias in the network training; $tanh$ is the activation function; $e_j$ is the weighted result of the implicit function’s output.

3.4. Proposed Model Architecture

The architecture of the CNN-BiLSTM deep learning framework with an attention mechanism as proposed in this study consists of the following layers: input, CNN, BiLSTM, attention and output. The input layer receives historical load sequences and feature data such as weather, along with preprocessing. A sequence folding layer is inserted between the input and CNN layers to perform convolution independently at each time step.

The CNN is used to extract spatial features from temporal data and consists of a 2D convolution layer, a Rectified Linear Unit (ReLU) layer, and a MaxPooling layer. The 2D convolution layer has a kernel size of [1, 3] and contains 16 filters. The ReLU activation function maintains the nonlinearity of the output and addresses gradient issues. The MaxPooling layer, with a kernel size of [1, 2], reduces the spatial dimensions of the feature maps by a factor of two, lowering feature dimensionality and preventing overfitting. A Batch Normalization (BN) layer is added between the convolutional and ReLU layers to accelerate training and avoid gradient vanishing or explosion. After the MaxPooling layer, a sequence Unfolding Layer is added to restore the output of the convolutional layer to a sequence structure. Finally, a Flatten Layer is used to flatten the sequence data into a one-dimensional vector for connection to the BiLSTM layer.

A BiLSTM layer models dependencies in temporal data, comprising 20 hidden layers and outputting hidden states at the final time step. Its output connects to a self-attention layer, which optimizes the BiLSTM output weights. This enhances important features and suppresses irrelevant ones. The self-attention layer has 40 channels, two heads, and two key and query channels. Before the fully connected layer, a dropout layer randomly deactivates neurons with a 0.1 probability of reducing overfitting. The fully connected layer transforms features into a 1D structure for the output layer to produce final predictions. Parameters of this CNN-BiLSTM-attention prediction model are summarized in

Table 2. CNN-BiLSTM-attention architecture parameters.

Layers	Layer Parameters		Activation Functions	Initializers
Sequence input layer	Number of features	5	-	-
CNN2D	Kernel size	[1, 3]	State activation function-Relu	Weights Initializer-Glorot Bias Initializer-zeros
	Filter	16	-	-
	Stride	1	-	-
	Padding	Same	-	-
Pooling Layer	Pool Size	[1, 2]	Pooling function-Max Pooling	-
	Stride	2	-	-
	Padding	Same	-	-
BiLSTM Layer	Number of hidden layers	20	State Activation Function-tanh	Input weight initializer-Glorot Recurrent weight Initializer-orthogonal Bias Initializer-unit forget gate
Attention Layer	Num of Heads	2	-	Weights Initializer-Glorot Bias Initializer-zeros
	Num of Key Channels	2	-	-
	Attention mechanism	Softmax	-	-
Dropout Layer	Probability	0.1	-	-
Fully Connected Layer	Output Size	1	-	-

.

The entire process of the proposed model is shown in Figure 9 and is described in detail as follows:

(1)

Before decomposing the original power load sequence, data organization and cleaning are required to unify the data format and remove abnormal items;

(2)

CEEMDAN is applied to decompose the power load sequence into multiple IMFs with different frequencies and characteristics, removing noise from the historical load data. We then calculate the sample entropy of these IMFs, and K-means clustering is used to classify these sample entropies, extracting more representative features for the dimensionality reduction of each IMF component. VMD is then applied to further decompose the largest cluster from the K-means clustering to more finely extract the main signal characteristics;

(3)

The processed power load IMFs and feature matrices form five 5-dimensional data matrices, which are normalized to eliminate the impacts of different magnitudes on the prediction model;

(4)

The five processed data matrices are divided into four seasons—spring, summer, autumn, and winter. For each season, the training and test datasets are split in a ratio of 4:1;

(5)

The training data are loaded, and the model parameters are initialized. The model is trained using the five sets of training data for each season, monitoring the loss and accuracy during training. If the validation loss decreases, the model is saved with the updated weights, and the epoch count is increased. If the validation loss does not decrease for 10 consecutive epochs, the learning rate is reduced. Training stops when the epoch count reaches 150;

(6)

Finally, the optimal hyperparameter model of the CNN-BiLSTM-attention network obtained from the grid search is saved. This model is then loaded onto the test data for prediction and evaluation to avoid overfitting.

4. Case Study

4.1. Evaluation Index

The prediction accuracy of the model is evaluated using four error metrics. Mean Square Error (MSE) and Mean Absolute Error (MAE) reflect the deviation between predicted and actual values; smaller values indicate higher prediction accuracy. Root Mean Square Error (RMSE) measures the differences between predicted and actual values, with smaller values indicating smaller differences. R² assesses the goodness of fit of regression models; higher values suggest stronger explanatory power. The formulas for calculating these metrics are

$$\mathrm{MSE}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^n{(y_{\mathrm{true}}-y_{\mathrm{predict}})}^2}$$

(30)

$$\mathrm{MAE}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^n\left|y_{\mathrm{true}}-y_{\mathrm{predict}}\right|}$$

(31)

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^n{(y_{\mathrm{true}}-y_{\mathrm{predict}})}^2}}$$

(32)

$$R^2=1-{\displaystyle \frac{{\displaystyle \sum _{i=1}^n{(y_{\mathrm{true}}-y_{\mathrm{predict}})}^2}}{{\displaystyle \sum _{i=1}^n{(y_{\mathrm{true}}-\overline{y_{\mathrm{true}}})}^2}}}$$

(33)

where $y_{\mathrm{predict}}$ is the predicted value of the load; $y_{\mathrm{true}}$ is the actual value of the load; $\overline{y_{\mathrm{true}}}$ is the average value of the actual load; N indicates the number of data samples.

4.2. Hyper-Parameter Setting

The input layer of the model receives raw data of an eight-step time series with a dimension of 5 (8 × 5). The hyperparameters of the proposed deep learning framework are set as shown in the

Table 3. Hyper-parameter setting of CNN-BiLSTM-attention.

Parameter	Setting
Optimizer	Adam
Execution Environment	CPU
Loss Function	Mean Squared Error (MSE)
Max Epoch	150
Gradient Threshold	1
L2 Regularization	0.0001
Learning rate	0.001

. The solver used is Adam, and the loss function is MSE. The initial learning rate is 0.001. The maximum number of epochs is set to 150.

4.3. Prediction Performance Analysis

4.3.1. Training Process Evaluation

In this study, an hourly prediction model is established to forecast power load using temperature, time features, day-of-week features, and holiday features. The hourly dataset throughout the year is divided into training and test data at an 80% to 20% ratio, respectively. Furthermore, due to the influence of seasons on the accuracy of load forecasting, the dataset is further divided into four seasons.

To validate the superiority and predictive ability of the proposed model, CNN-BiLSTM (C-B), CNN-LSTM (C-L), LSTM-A (L-A), and LSTM (L) are chosen as benchmark models. The proposed model uses the five IMFs data of Guangzhou decomposed and clustered by CEEMDAN, K-means, and VMD algorithms, while the other models uses the original power load data sequence.

The model was run using Python 3.13 software on a computer with an i-7-14700 (28 CPU) processor and 16,384 MB of memory.

Table 4. Error change process.

Season	Method	Epoch
		1	10	20	30	40	50
Spring	Proposed	0.69	0.11	0.06	0.06	0.05	0.05
	C-B	0.52	0.06	0.04	0.05	0.04	0.04
	C-L	0.51	0.08	0.05	0.06	0.05	0.05
	L-A	0.40	0.14	0.10	0.08	0.07	0.07
	L	0.83	0.16	0.12	0.11	0.10	0.08
Summer	Proposed	0.70	0.10	0.07	0.06	0.05	0.05
	C-B	0.77	0.07	0.05	0.06	0.04	0.04
	C-L	0.59	0.10	0.09	0.09	0.08	0.08
	L-A	0.63	0.14	0.11	0.09	0.09	0.07
	L	0.52	0.13	0.10	0.09	0.09	0.08
Autumn	Proposed	0.40	0.08	0.06	0.05	0.05	0.04
	C-B	0.65	0.07	0.06	0.05	0.05	0.05
	C-L	0.58	0.10	0.08	0.07	0.06	0.07
	L-A	0.69	0.16	0.14	0.10	0.08	0.08
	L	0.65	0.14	0.12	0.11	0.10	0.09
Winter	Proposed	0.60	0.09	0.06	0.06	0.05	0.04
	C-B	0.36	0.06	0.05	0.05	0.06	0.04
	C-L	0.65	0.10	0.08	0.07	0.07	0.06
	L-A	0.52	0.15	0.10	0.09	0.08	0.08
	L	0.61	0.17	0.13	0.12	0.11	0.09

and Figure 10, respectively, present the changes and processes of MSE of the five models after 50 iterations. The results show that models with CNN structure converge quickly during the early training stages of four seasons, and adding a BiLSTM structure could further speed up early convergence. L-A and traditional LSTM converge slowly, with no clear convergence after 50 iterations, leading to higher prediction errors than CNN-based models. Figure 10 indicates that after optimizing weight allocation with attention mechanism, L-A converges faster than traditional LSTM in spring and winter, with smaller error values. Overall, the proposed model has good convergence speed and the most stable error distribution.

Table 5. Model training time for four seasons (seconds).

Season	Proposed (5 Database)	C-B	C-L	L-A	L
Spring	82	15	15	10	7
Summer	153	16	16	11	16
Autumn	134	38	31	11	9
Winter	80	16	17	8	10
Total	449	85	79	40	42

lists the total training time needed by each model to complete four-season forecasting, so as to evaluate the models’ computational efficiency. The proposed model needs to train on five datasets per season, with a total training time of 449 s across the four-season data. On average, it takes 22.5 s for our model to train on each dataset, similar to C-B’s average of 21.25 s, which is a bit higher than C-L’s 19.75 s, and about 2.2 times the training time needed by the L-A Traditional LSTM.

4.3.2. Prediction Effect

As shown in Figure 11, the proposed model demonstrates good predictive performance in terms of both load levels and trends for the initial 48 h of the test set across the four seasons. Except for winter, the actual values can all fall within the 95% confidence interval of the prediction. This is because the winter testing period coincided with the post-pandemic reopening phase. There was a large-scale commissioning of industries, foreign trade, and infrastructure, leading to a surge in electricity consumption. In the winter test set, two load peaks on several days increased dramatically. Such load distribution trends were absent in the training set. Moreover, such changes in trade were not among the features considered in the proposed model. Therefore, the model did not capture the impact of trade on electricity load during training. Although only four days of peak load predictions deviated from actual values, this was a significant proportion of the 18-day training set. This is also the main reason why all models performed the worst in winter predictions.

4.3.3. Prediction Accuracy

Each model of each season has been evaluated 10 times, and the results with the best prediction effect have been selected for comparison. The prediction errors of each proposed model are shown in Figure 12 and

Table 6. MSE, RMSE, MAE and R² forecasted for different models in four seasons.

Season	Method	Evaluation Index					Improvement
		MSE	RMSE	MAE	MAPE	R2	MSE	RMSE	MAE	MAPE	R2
		GWh2	GWh	GWh	%	%	GWh2	GWh	GWh	%	%
Spring	Proposed	0.39	0.62	0.49	1.41	99.10	-	-	-	-	-
	C-B	0.71	0.84	0.61	1.82	98.35	0.32	0.22	0.12	0.42	0.74
	C-L	0.75	0.87	0.64	1.92	98.25	0.36	0.24	0.15	0.51	0.84
	L-A	1.05	1.02	0.75	2.29	97.56	0.66	0.40	0.26	0.88	1.54
	L	1.50	1.23	0.93	2.81	96.50	1.12	0.60	0.44	1.40	2.60
Summer	Proposed	0.57	0.75	0.60	1.25	99.13	-	-	-	-	-
	C-B	0.82	0.91	0.70	1.47	98.74	0.25	0.15	0.10	0.22	0.39
	C-L	1.03	1.01	0.80	1.66	98.43	0.46	0.26	0.20	0.40	0.71
	L-A	1.26	1.12	0.87	1.88	98.07	0.69	0.37	0.27	0.63	1.06
	L	1.55	1.24	0.94	2.04	97.63	0.98	0.49	0.34	0.79	1.50
Autumn	Proposed	0.62	0.79	0.61	1.08	99.28	-	-	-	-	-
	C-B	0.95	0.97	0.74	1.31	98.89	0.33	0.19	0.13	0.23	0.38
	C-L	0.99	0.99	0.76	1.35	98.84	0.37	0.21	0.15	0.27	0.43
	L-A	1.97	1.40	1.03	1.88	97.70	1.35	0.62	0.42	0.80	1.57
	L	3.15	1.77	1.35	2.42	96.32	2.53	0.99	0.74	1.34	2.96
Winter	Proposed	0.79	0.89	0.68	1.67	98.56	-	-	-	-	-
	C-B	2.37	1.54	1.11	2.14	95.76	1.57	0.65	0.43	1.19	2.80
	C-L	1.79	1.34	1.03	2.40	96.76	1.00	0.45	0.35	0.86	1.80
	L-A	1.63	1.28	0.99	2.63	97.05	0.83	0.38	0.31	0.78	1.51
	L	2.91	1.68	1.31	3.29	94.92	2.12	0.78	0.63	1.62	3.64

. The results show the following:

• Overall, the proposed model demonstrated the highest accuracy in power load forecasting across all four seasons. The quarterly MSE values for spring, summer, autumn, and winter are 0.39 GWh², 0.57 GWh², 0.62 GWh², and 0.79 GWh², respectively. The quarterly RMSE values are 0.62 GWh, 0.75 GWh, 0.79 GWh, and 0.89 GWh, respectively. The quarterly MAE values are 0.49 GWh, 0.60 GWh, 0.61 GWh, and 0.68 GWh, respectively. The quarterly MAPE values are 1.41%, 1.25%, 1.08% and 1.67%, respectively. Except for the quarterly R² in winter, which is 98.56%, the R² in the other three seasons is all above 99%;
• The second model with an advantage in prediction accuracy is C-B, whose prediction accuracy in spring, summer and autumn is second only to that of the proposed model, but is lower than the L-A in winter. Compared to the C-B, the proposed model yields a higher accuracy of 0.25–1.57 GWh² in MSE, 0.15–0.65 GWh in RMSE, 0.10–0.43 GWh in RMSE, 0.22–1.19% in MAPE, and 0.38–2.8% in R² across all four seasons, respectively;
• The extraction of spatial features by CNN is conducive to enhancing the prediction accuracy of the model. Models with CNN showed higher accuracy than those without. Although C-B and C-L have approximately the same priorities in different seasons, the prediction accuracy of C-L is inferior to that of C-B except in winter. Compared to C-L, the proposed model yields a higher accuracy of 0.36–1.00 GWh² in MSE, 0.21–0.45 GWh in RMSE, 0.15–0.35 GWh in RMSE, 0.27–0.86% in MAPE, and 0.43–1.8% in R² across all four seasons, respectively;
• The accuracy performance of C-L is superior to that of L-A in spring, summer and autumn. However, in winter, the prediction accuracy of L-A is second only to the proposed model. Compared to L-A, the proposed model yields a higher accuracy of 0.66–1.35 GWh² in MSE, 0.37–0.62 GWh in RMSE, 0.26–0.42 GWh in RMSE, 0.63–0.88% in MAPE, and 1.01–1.57% in R² across all four seasons, respectively;
• The prediction accuracy and stability of LSTM performed the worst, with significantly higher MSE, RMSE and MAE values and lower R² than other models across all seasons. Compared to LSTM, the proposed model yields a higher accuracy of 0.98–2.53 GWh in MSE, 0.49–0.99 GWh² in RMSE, 0.34–0.74 GWh in RMSE, 0.63–0.88% in MAPE, and 0.79–1.62% in R² across all four seasons, respectively;
• From a seasonal perspective, all models reach the lowest MSE, RMSE, MAE and MAPE and the highest R² in spring, followed by summer, and all models share the largest prediction errors in winter.

The forecast errors of the proposed model and other benchmark models across seasons throughout the year are shown in

Table 7. Comparison of the annual forecasting results of different models.

Method	ME	SD	MSE	RMSE	MAE	MAPE
	GWh	GWh	GWh2	GWh	GWh	%
Proposed	0.11	0.77	0.60	0.77	0.60	1.38
C-B	−0.02	1.07	1.14	1.07	0.81	1.86
C-L	−0.08	1.07	1.14	1.07	0.81	1.84
L-A	−0.21	1.20	1.48	1.22	0.91	2.00
L	−0.24	1.48	2.26	1.50	1.14	2.51

to evaluate the superiority of the model from an overall perspective. The bolded values are the optimal values of each indicator. It can be seen that, except for the mean error (ME) indicator, the proposed model is significantly lower-performing than the other benchmark models in all forecast errors throughout the year, with 0.77 GWh in standard deviation (SD), 0.60 GWh² in MSE, 0.77 GWh in RMSE, 0.60 GWh in MAE and 1.38 GWh in MPAE, respectively. Despite a non-optimal SE, the lower SD, RMSE, and MAPE show that the prediction of the proposed model has better stability. C-B has the smallest overall SE, but its SD and MAPE are 40% and 35% higher than those of the proposed model. LSTM is the most unstable in ME and prediction stability, followed by L-A.

4.3.4. Ablation Study

As shown in

Table 8. The results of Ablation Study.

Method	ME	SD	MSE	RMSE	MAE	MAPE
	GWh	GWh	GWh2	GWh	GWh	%
Full Model	0.11	0.77	0.60	0.77	0.60	1.38%
w/o VDM	−0.15	0.80	0.67	0.82	0.63	1.46%
w/o CEEMDAN-K-Means-VDM	0.01	1.02	1.05	1.02	0.74	1.66%
w/o CNN	−0.26	1.09	1.26	1.12	0.87	1.95%
w/o BiLSTM	−0.21	0.85	0.77	0.88	0.68	1.58%
w/o Attention	0.14	0.87	0.78	0.88	0.66	1.54%
Full Model	0.11	0.77	0.60	0.77	0.60	1.38%

, ablation studies were carried out to evaluate the effectiveness of different modules in the proposed model, with the following structures:

• w/o VDM—only CEEMDAN and K-means are used to process load sequence data;
• w/o CEEMDAN-K-means-VDM—no decomposition preprocessing of load sequence data;
• w/o CNN—only BiLSTM-attention, no feature extraction of the training data by the CNN layer;
• w/o BiLSTM—the BiLSTM layer is replaced with an LSTM layer, with the same number of hidden layers;
• w/o Attention—no attention-based weight allocation for BiLSTM.

The full model serves as the baseline. First, the performance drops when data decomposition is removed. This confirms that CEEMDAN-K-means or CEEMDAN-K-means-VDM processing could help to boost prediction accuracy. Secondly, the performance of the model without a CNN structure is unsatisfactory, which indicates that the model lacks the ability to capture spatial features, resulting in increases of 0.33 GWh, 0.35 GWh and 0.57%, respectively, in cross-seasonal SD, RMSE and MAPE throughout the year. This highlights the critical role of CNN in the feature extraction of load sequences. Also, the model without BiLSTM sees SD, RMSE, and MAPE increase by 0.09 GWh, 0.10 GWh, and 0.20%. This shows the importance of using past and future information for contextual relationship capturing. Moreover, removing the attention mechanism significantly degrades performance, underscoring the necessity of attention-based weight allocation. In summary, the ablation study shows that all components are essential to the proposed model.

4.4. Improvement Impact of CEEMDAN, K-Means and VMD Data Processing

To further investigate the influence of CEEMDAN, K-means and VMD data processing on the prediction performance, the prediction errors obtained by using the processed load data and the original load data for the C-B-A, C-B, C-L, L-A and LSTM models, respectively, are summarized and analyzed in this section.

The boxplots and error distributions in Figure 13 show hourly differences between actual and predicted loads across four seasons of C-B-A, C-B, C-L, L-A and LSTM models, where the dots represent prediction errors, the line in the box indicates the median error, the box’s upper and lower edges represent the 75th and 25th percentiles, and the whiskers show the maximum and minimum values, excluding outliers.

Table 9. Comparison of the forecasting results with and without CEEMDAN, K-means and VMD processing.

Season	Method	With Processing					Without Processing
		MSE	RMSE	MAE	MAPE	R2	MSE	RMSE	MAE	MAPE	R2
		GWh2	GWh	GWh	%	%	GWh2	GWh	GWh	%	%
Spring	Proposed	0.39	0.62	0.49	1.41	99.10	0.52	0.72	0.55	1.65	98.79
	C-B	0.56	0.75	0.58	1.67	98.69	0.71	0.84	0.61	1.82	98.35
	C-L	0.68	0.83	0.65	1.89	98.41	0.75	0.87	0.64	1.92	98.25
	L-A	0.79	0.89	0.72	2.16	98.16	1.05	1.02	0.75	2.29	97.56
	L	1.36	1.17	0.93	2.86	96.84	1.50	1.23	0.93	2.81	96.50
Summer	Proposed	0.57	0.75	0.60	1.25	99.13	0.80	0.89	0.69	1.45	98.78
	C-B	0.50	0.71	0.55	1.16	99.24	0.82	0.91	0.70	1.47	98.74
	C-L	0.63	0.79	0.62	1.31	99.04	1.03	1.01	0.80	1.66	98.43
	L-A	0.72	0.85	0.66	1.43	98.90	1.26	1.12	0.87	1.88	98.07
	L	1.60	1.27	0.95	2.05	97.54	1.55	1.24	0.94	2.04	97.63
Autumn	Proposed	0.62	0.79	0.61	1.08	99.28	0.97	0.98	0.76	1.35	98.87
	C-B	0.69	0.83	0.65	1.15	99.19	0.95	0.97	0.74	1.31	98.89
	C-L	0.86	0.93	0.73	1.28	99.00	0.99	0.99	0.76	1.35	98.84
	L-A	1.31	1.14	0.88	1.57	98.47	1.97	1.40	1.03	1.88	97.70
	L	2.13	1.46	1.17	2.08	97.52	3.15	1.77	1.35	2.42	96.32
Winter	Proposed	0.79	0.89	0.68	1.67	98.56	1.89	1.37	0.98	2.48	96.59
	C-B	1.34	1.59	0.86	2.14	97.58	2.37	1.54	1.11	2.85	95.76
	C-L	1.73	1.31	0.97	2.40	96.90	1.79	1.34	1.03	2.52	96.76
	L-A	1.67	1.29	1.04	2.63	96.99	1.63	1.28	0.99	2.45	97.05
	L	2.54	1.59	1.32	3.29	95.41	2.91	1.68	1.31	3.29	94.92

compares the prediction accuracies of each model using the original data with the CEEMDAN, K-means and VMD processing data. Figure 13a,b, respectively, show the prediction error results obtained by processing the data with CEEMDAN, K-means, VMD and the original data. The following can be found:

• Overall, except for the L model, the prediction accuracy of all models is improved by using CEEMDAN, K-means, and VMD for data processing. The MSE is improved by up to 1.09 GWh², and the MAPE is improved by up to 0.45%;
• Figure 13 shows that performing CEEMDAN, K-means and VMD processing on the original load sequence and using the processed data for prediction can reduce the difference between the actual load and the prediction. Only the median error of some models deviates from the “0” value in autumn and winter. Overall, the median error of each model in the four seasons shows an approach to the “0” value;
• Compared with Figure 13b, in Figure 13a, except for L-A in spring and winter, and L in spring, summer and winter, most models have a reduced box height across all four seasons. Meanwhile, the error distribution in Figure 13a shows a bell-shaped feature. All these indicate that the modal decomposition and clustering of the original data can further improve the prediction accuracy;
• Moreover, the processed data are more conducive to the extraction of features by CNN. The errors of C-B-A, C-B and C-L show a more obvious normal distribution;
• Although the normal characteristics of the error distributions of L-A and LSTM in Figure 13a are relatively less obvious, compared with the situation in Figure 13b, the errors have a significant tendency to concentrate towards the median line, and the outliers decrease.

Overall, processing the original load sequences with CEEMDAN, K-means and VMD and applying them for hourly load prediction can enhance the prediction accuracy of each model. Additionally, the processed sequence data are more conducive to helping CNN better extract historical load sequence information.

4.5. Influence of Training Data Quantity

In addition to the model’s internal structure and the hyper-parameter settings, the prediction accuracy of the model is also affected by the volume and diversity of the training dataset. Complex models, with more parameters to estimate, are hard to train accurately with limited data. When training data are scarce, deep neural networks may overfit by learning the data’s noise and specific patterns instead of general trends. This prevents the models from generalizing to new situations and leads to poor test set performance. In reality, sometimes it is impossible to accurately obtain sufficient raw data for model prediction, which adds to the complexity of making accurate predictions.

This section examines how training data quantity affects the predictive performance of each model. Winter, where all models show the lowest accuracy, is chosen for this test. The training and test sets are split in ratios of 1:4, 2:3, and 3:2 to explore each model’s training data requirements. Figure 14 presents the hourly comparisons of each model’s predictions with the original data, along with MSE and MAE error indicators, for training set proportions of 20%, 40%, 60% and 80%, respectively. It shows that having sufficient training data is crucial for improving predictive accuracy. A substantial reduction in the training set hinders the model’s ability to learn regular features:

• When the training set is 20%, the MSE and MAE of the proposed model are 5.04 GWh² and 1.59 GWh, respectively. The MSE and MAE of C-B, C-L and L-A are approximately 9 GWh² and 2 GWh, respectively. The MSE of LSTM reaches 16.95 GWh², and the MAE is 3.07 GWh. By comparing the prediction effects of each model with the actual load between the 8364–8436 period, where all models performed poorly, and the final 8720–8784 period, it is found that C-L and LSTM fail to capture the second daily load peak information—which is the highest power demand peak throughout the day, predicting it to be significantly lower than the first load peak. However, C-L achieves a relatively accurate prediction of the first load peak. While L-A improves the prediction of the highest peak, its estimate was here still lower than that of the first peak. By contrast, the proposed model and C-B show better sequential information extraction, predicting the second peak as the day’s highest. However, all models underpredict the peaks, with the second peak predictions deviating most from the actual value;
• When the training set is 40%, all models show improved prediction accuracy. The MSE and MAE of the proposed model are 2.58 GWh² and 1.16 GWh respectively. LSTM remains the least accurate, with an MSE of 6.42 GWh² and an MAE of 1.80 GWh. Most models, except C-L, have learned to predict two daily load peaks, with the second higher than the first. In the periods 8364–8436 and 8720–8784, the proposed model, followed by C-B, L-A, and LSTM (excluding C-L), shows the smallest deviations in trend and value from the actual load
• When the training set is 60%, the prediction accuracy of L-A and LSTM improves significantly, with MSE values around 3.50 GWh² and MAE around 1.40 GWh. However, the prediction accuracy of C-L drops substantially, with an MSE of 5.48 GWh² and an MAE of 1.66 GWh, and it still fails to capture the second load peak. Despite more training data, the proposed model, C-B, and L-A may have captured noise, leading to a third predicted peak between the two actual peaks. This is also the reason why the changes in the prediction accuracy of the proposed model and C-B after the increase in training data are extremely slight;
• When the training set is 80%, all models demonstrate the learning of general trends in the training data. The proposed model best fits the load data, showing the smallest differences from actual load levels in terms of both magnitude and duration. C-L and L-A also predict load peaks well, but fail to capture the durations of these peaks and valleys. C-B and LSTM capture some information about the durations of peaks and valleys, but generally underpredict load peaks.

4.6. Model Generalization Test

The proposed model is applied to predict the power load of a tourist-oriented island to test its generalization. This area’s load and temperature distribution differs greatly from in the study case. The 1121-samples database is more volatile and uncertain due to the local tourism expansion, and there are a few discontinuous data. The proposed model achieved an SD of 264.34 kWh, a MAPE of 5.6%, and an R² of 90% on this dataset. As shown in Figure 15, despite inadequate training data, the model performed well in predicting load levels and overall trends. Except for the mutation that occurred at the end, the actual loads all fell within the predicted 95% confidence band. In summary, the proposed model demonstrates a good generalization ability and robustness.

5. Conclusions

This study introduces a data-processing method that uses CEEMDAN, K-means, and VMD sequentially on raw data for feature decomposition and clustering, and to extract the significant regularity of historical data in terms of seasonal, weekly and holiday characteristics, as well as the uncertainty in terms of hourly randomness and diurnal variation characteristics, which is more conducive to the capturing of data characteristics by the model. A CNN-BiLSTM-attention model is proposed for the power load forecasting. The proposed method is compared with other deep learning methods to prove its practicability and superiority.

The CNN and LSTM models capture climate and time information in multivariate time series, forecasting power loads for four seasons. The proposed model outperforms CNN-BiLSTM, CNN-LSTM, LSTMM-attention and LSTM in load prediction across all four seasons. Furthermore, ablation experiments on the model verified that each component contributed to the entire model. Moreover, CEEMDAN, K-means and VMD-processed data here show an enhancing effect on model feature extraction and prediction accuracy. Also, the generalization ability of the model has been verified on a small-batch dataset with strong fluctuations. Finally, through different divisions of the scale of the training data set, the influence of the amount of training data on the prediction accuracy and prediction trend of the model is here analyzed. The results indicate that the proposed model is efficient and stable, outperforming existing benchmarks.

• The proposed CNN-BiLSTM-attention network, based on CEEMDAN, K-means and VMD data processing, has MAPE values of 1.41%, 1.25%, 1.08% and 1.67% in spring, summer, autumn, and winter, respectively. The annual cross-seasonal SD is 0.77 GWh;
• The proposed model and contrast models perform best in spring, followed by summer, with the largest prediction errors in winter;
• Predicting with mode functions from CEEMDAN, K-means and VMD decomposition, as well as clustering, improves model prediction accuracy compared to using raw data;
• Mode functions help CNN better extract historical load sequence information, with errors showing a more obvious normal distribution;
• Insufficient training data may hinder the model’s learning of daily peak load features, and cause BiLSTM and attention models to capture noise features;
• The proposed model best fits load peaks and valleys in terms of magnitude and duration.

As renewable energy’s proportion in power systems continuously increases, long-term load forecasting is crucial for its effective integration and planning. Long-term load forecasting helps anticipate power market risks, determine a reasonable scale and distribution of renewable energy access, and optimize system operations, thus boosting the renewable energy penetration rate and enhancing power system stability. The proposed model forecasts long-term power loads using temperature, one hour in a day, one day in a week, and holiday indicator features. Moreover, power usage patterns and load curve distributions vary across industries and user groups. Economic and climate features (e.g., cooling needs in cold vs. tropical areas) also significantly impact power load demands. The current dataset lacks information on these features. Future work will collect more diverse data and test the model’s generalization in different climate zones or years, so as to explore the computational efficiency and limitations of the model on different volumes datasets.