Forecasting Energy Consumption Using a Hybrid LSTM-XGBoost Model ^†

Abstract

Accurate short-term forecasting for energy consumption is crucial in modern energy network management, especially for cities such as Tetouan, where considerable climate variability and diverse usage patterns present significant challenges when it comes to making short-term forecasts. This paper proposes a hybrid residual learning framework that combines a long short-term memory (LSTM) network with eXtreme Gradient Boosting (XGBoost) to improve short-term load forecasting for the Tetouan electricity network. The novelty of the proposed approach lies in coupling temporal sequence modeling with residual error correction driven by exogenous meteorological and calendar-related information. The proposed model is validated using real electricity consumption data from Zone 2 of Tetouan City, with further validation across all three available zones confirming the model’s generalizability. The proposed model achieves a coefficient of determination (R²) of 0.984, an RMSE of 687.21 kWh, and a MAPE of 2.41%, representing a 121.3 kWh RMSE improvement over the standalone LSTM baseline. These results confirm that the hybrid model is better at tracking periods of high demand compared to conventional machine learning approaches and standalone deep learning models.

1. Introduction

In dynamic urban environments such as Tetouan City, featuring high climatic variability and intricate consumption patterns, accurate electricity load forecasting is critical to grid stability and energy management [1,2]. ARIMA and other commonly used classical statistical models, based on linear and stationary assumptions, fail to capture the rapid and non-linear consumption patterns of electricity in Tetouan City [3,4]. Also, time-series data has sequential dependencies across the continuous range; classic machine learning models such as Random Forest or Support Vector Regression show a similar lack of attention to this complexity and are not capable of capturing the long-term temporal dependencies [2]. The limitations of a single-model-based architecture for large energy datasets justify the implementation of hybrid forecasting frameworks [5].

Hybrid forecasting frameworks with deep learning and ensemble methods have been extensively investigated in order to address this complex problem [5]. Long short-term memory networks are successful in modeling long-term temporal dependencies [5] and eXtreme Gradient Boosting accounts for non-linear feature interactions, but systematic enhancement of hybrid LSTM–XGBoost accuracy over extremely volatile urban electricity load data via explicit residual learning protocols remains a research gap [5]. Hence, we attempt to fill in this gap by introducing a strong hybrid LSTM–XGBoost framework with a residual learning approach for short-term electricity consumption prediction in Tetouan City. Some important contributions are: (1) a hybrid LSTM–XGBoost model is optimized for temporal dependencies modeling and to address non-linear residual errors for urban electricity demand; (2) it clearly outperforms pure LSTM forecasting models for volatile data of high resolution; (3) a residual learning method can provide robust predictions in dynamic smart grid conditions.

2. Related Work

Electricity load forecasting is key; however, classical statistical and machine learning methods tend to struggle with non-linear relationships and temporal dependencies [2,4]. In general, deep learning, especially LSTMs and GRUs, has been applied to the forecasting field, as it is a powerful tool that effectively incorporates complex dynamics and sudden oscillations and outperforms statistical models [4,6,7,8].

The performance of a hybrid framework such as a deep learning-ensemble method consistently outperforms the standalone method [1,3,5,6]. These approaches utilize the temporal sequence of LSTM and the robust non-linear residual error correction of XGBoost to make better predictions for non-stationary datasets [1,9].

However, there is still a gap: the majority of the hybrid approaches integrate models in parallel rather than requiring XGBoost to be available specifically for the dedicated residual error correction [6]. This inadequacy causes LSTMs to underreport peak loads and high-frequency variations. In this study, we address this by implementing a systematic residual-based hybrid framework.

3. Methodology

3.1. Dataset Description and Preprocessing

For this work, we used the consumption dataset for the city of Tetouan [2]. Zone 2 was used as the main benchmark case, and the same experimental pipeline was also applied to Zones 1 and 3 to assess the generalizability of the proposed framework across the three service areas. The dataset offers 10 min intervals of electricity consumption measurements for 2017, together with meteorological variables such as temperature, humidity, wind speed, and diffuse flux. The models receive all the numerical variables scaled between 0 and 1 with Min–Max normalization before their training. In addition, cyclical time variables (hour, day, month) are encoded with sine and cosine transformations as defined in Equations (1) and (2) to preserve periodicity:

$$x_{sin}=\mathit{sin}\left({\displaystyle \frac{2\pi x}{P}}\right)$$

(1)

$$x_{cos}=\mathit{cos}\left({\displaystyle \frac{2\pi x}{P}}\right)$$

(2)

LSTM input sequences using a sliding window approach have a sequence length of 144 time steps (24 h). The dataset has been chronologically divided, with 80% used for training and 20% used for testing.

3.2. LSTM Network Architecture

The long short-term memory (LSTM) network is employed as the baseline model to capture long-term temporal dependencies. The architecture consists of two stacked LSTM layers with 64 and 32 units, respectively, each followed by a dropout layer (rate = 0.2), and a final dense output layer. The internal operations of the LSTM cell are described by Equations (3)–(8) which define the forget gate, input gate, candidate cell state, updated cell state, output gate, and hidden state, respectively.

$$f_t=\mathsf{\sigma }\left(W_f\left[h_{t-1},x_t\right]+b_f\right)$$

(3)

$$i_t=\mathsf{\sigma }\left(W_i\left[h_{t-1},x_t\right]+b_i\right)$$

(4)

$$\widetilde{C_t}=\tanh \left(W_C\left[h_{t-1},x_t\right]+b_C\right)$$

(5)

$$C_t=f_t\odot C_{t-1}+i_t\odot \widetilde{C_t}$$

(6)

$$o_t=\mathsf{\sigma }\left(W_o\left[h_{t-1},x_t\right]+b_o\right)$$

(7)

$$h_t=o_t\odot \tanh \left(C_t\right)$$

(8)

3.3. Hybrid LSTM-XGBoost Framework

A residual learning strategy is adopted to enhance forecasting accuracy. The LSTM first generates a baseline prediction ${\widehat{y}}_t^{LSTM}$, and the residual error used for the correction stage is defined in Equation (9).

$$e_t=y_t-{\widehat{y}}_t^{LSTM}$$

(9)

The overall architecture of the proposed Hybrid LSTM–XGBoost framework is illustrated in Figure 1.

The LSTM network produces a baseline prediction from historical load and exogenous features. During the training phase, residual errors are computed using the actual load and modeled by an XGBoost regressor. During inference, the predicted residual corrects the LSTM output to generate the final hybrid forecast.

As shown in Figure 1, the LSTM network first learns long-term temporal dependencies from historical load data and exogenous variables to generate a baseline prediction. During the training phase, residual errors are computed using the actual load values and modeled by an XGBoost regressor using only weather- and time-related features. This choice was made because the LSTM baseline is designed to capture the main temporal dynamics of the load series, whereas XGBoost is used to model the remaining residual variations through exogenous meteorological and calendar-related information. During inference, the predicted residual is added to the LSTM output to obtain the final hybrid forecast.

The final hybrid forecast, obtained by combining the LSTM baseline prediction with the XGBoost-predicted residual, is given in Equation (10).

$${\widehat{y}}_t^{Hybrid}={\widehat{y}}_t^{LSTM}+\widehat{e_t}$$

(10)

The hybrid LSTM-XGBoost architecture leverages the sequential modeling strengths of LSTM with the Gradient Boosting power of XGBoost for microgrid energy forecasting. The optimal hyperparameters in

Table 1. Hyperparameters of the hybrid LSTM-XGBoost model.

Component	Hyperparameter	Value
LSTM Layer 1	Units	64
	Return sequences	True
	Dropout	0.2
LSTM Layer 2	Units	32
	Dropout	0.2
Dense Layer	Units	1
Input Shape	Window × Features	window × n_features
XGBoost	n_estimators	400
	learning_rate	0.05
	max_depth	4
	subsample	0.8
	colsample_bytree	0.8
	objective	reg:squarederror
	random_state	42
	n_jobs	−1

were obtained through comprehensive grid search optimization, featuring a two-layer LSTM stack (64 to 32 units) with 20% dropout regularization to mitigate overfitting, followed by a single output neuron. The XGBoost component employs 400 estimators at a conservative 0.05 learning rate, with moderate tree depth (4) and 80% subsampling for both rows and features to balance bias–variance tradeoffs in renewable energy prediction tasks.

3.4. Evaluation Metrics

The forecasting performance is evaluated using three standard metrics, defined in Equations (11)–(13): the Root Mean Squared Error (RMSE), the Mean Absolute Percentage Error (MAPE), and the coefficient of determination (R²).

• Root Mean Squared Error: measures the average magnitude of the errors.

$$RMSE=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\left(y_i-\widehat{y_i}\right)}^2}$$

(11)

• Mean Absolute Percentage Error: provides a measure of prediction accuracy in percentage terms.

$$MAPE={\displaystyle \frac{100}{N}}{\sum }_{i=1}^N\left|{\displaystyle \frac{y_i-\widehat{y_i}}{y_i}}\right|$$

(12)

• Coefficient of Determination ($R^2$): indicates the proportion of the variance in the dependent variable that is predictable from the independent variables.

  $$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^N{\left(y_i-\widehat{y_i}\right)}^2}{{\sum }_{i=1}^N{\left(y_i-\overline{y}\right)}^2}}$$

  (13)

4. Results and Discussion

This section first presents the detailed benchmark results on Zone 2 of Tetouan City and then reports the cross-zone validation results on Zones 1 and 3.

4.1. Quantitative Performance

Table 2. Zone 2 model comparison: ARIMA, ML, and hybrid deep learning.

Model	RMSE (kWh)	MAPE (%)	R2
ARIMA	5999.3440	20.1901	−0.1965
SVR	1209.8550	3.4681	0.9447
Random Forest	1105.3000	4.8715	0.9512
LSTM	808.51	2.94	0.978
Hybrid CNN-LSTM	1025.67	3.65	0.9618
Hybrid LSTM-XGBoost	687.21	2.41	0.984

presents the comparative evaluation of the proposed Hybrid LSTM-XGBoost model against baseline and competitive approaches for microgrid energy forecasting in Zone 2, assessed via RMSE (kWh), MAPE (%), and R². The hybrid model achieves the lowest RMSE of 687.21 kWh and MAPE of 2.41%, alongside the highest R² of 0.984, demonstrating superior accuracy and explanatory power over traditional ARIMA (RMSE: 5999.34 kWh, R²: −0.1965) and standalone machine learning methods.

Among deep learning alternatives, Hybrid LSTM-XGBoost outperforms LSTM (RMSE: 808.51 kWh, R²: 0.978) and Hybrid CNN-LSTM (RMSE: 1025.67 kWh) by significant margins while surpassing tree-based Random Forest (RMSE: 1105.30 kWh) and SVR (RMSE: 1209.86 kWh). These Zone 2-specific results validate the architecture’s effectiveness in capturing both temporal dependencies via LSTM and non-linear interactions through XGBoost, achieving an approximately 15.0% RMSE reduction compared to the next-best performer. Overall, the proposed hybrid model demonstrates the best performance across all evaluated metrics.

Extending the Zone 2 superiority,

Table 3. Hybrid LSTM vs. baseline LSTM performance across three zones.

Zone	Model	RMSE (kWh)	MAPE (%)	R2
Zone 1	LSTM	921.2895	2.4782	0.9777
	Hybrid LSTM-XGBoost	651.5100	1.6780	0.9889
Zone 2	LSTM	808.51	2.94	0.978
	Hybrid LSTM-XGBoost	687.21	2.41	0.984
Zone 3	LSTM	1153.5117	9.3104	0.8817
	Hybrid LSTM-XGBoost	852.3203	6.6527	0.9340

compares the Hybrid LSTM-XGBoost model against baseline LSTM across three zones, consistently showing superior performance in RMSE, MAPE, and R² metrics. In Zone 1, the hybrid model achieves 29% RMSE reduction (651.51 vs. 921.29 kWh) and improves R² from 0.9777 to 0.9889; Zone 2 shows 15% RMSE improvement (687.21 vs. 808.51 kWh); and Zone 3 delivers 26% RMSE reduction (852.32 vs. 1153.51 kWh) with substantial MAPE gains (6.65% vs. 9.31%).

These multi-zone results confirm the hybrid model’s robustness and generalizability across the three zones. The hybrid model consistently reduces RMSE and MAPE relative to the standalone LSTM while improving R² in all cases. The consistent outperformance validates XGBoost integration as a critical enhancement for LSTM-based renewable energy forecasting.

4.2. Visual Analysis

Even so, visual inspection of the forecast results yields some interesting qualitative insights that support this claim and even bolster the hypothesis to support the hybrid model’s best performance beyond the above-mentioned quantitative indices in terms of forecasts. Comparison against a time series, as shown in Figure 2, shows that the discrete LSTM model captures a good overall trend that shows the average power consumption. It shows a smoothed effect, resulting in the underestimation of significant peak loads and the inability to measure fast fluctuations in demand. Conversely, the Hybrid LSTM–XGBoost model closely resembles the ground truth, displaying dramatically improved performance in capturing peak load events and dynamic demand shifts with respect to the reality in this field. This improved behavior shows the improved performance of the hybrid model in rapidly changing conditions and severe change effects on consumption of electricity.

Additional information on the accuracy of predicting is supplied by the scatter plot in Figure 3, which shows the consumption of the actual and the predicted electricity values. The predictions created from the hybrid model are significantly more strongly clustered around the identity line, indicating perfect prediction. This closer alignment reflects a decrease in prediction errors and further visually demonstrates the enhanced goodness-of-fit suggested by the quantitative evaluation.

An analysis of the daily Mean Absolute Percentage Error distribution, denoted as Figure 4, in turn, emphasizes the durability of the hybrid framework. Even though the LSTM baseline shows larger fluctuations in daily forecasting accuracy, the Hybrid LSTM–XGBoost model always shows a lower daily MAPE and less variability. This stability across different days demonstrates the hybrid model’s ability to tolerate time variability and different consumption patterns, thereby cementing its place in short-term reliable electricity load forecasting under dynamic operational conditions.

Figure 5 presents the SHAP summary plot illustrating the contribution of input features to the prediction correction of the Hybrid LSTM–XGBoost model in Zone 2. Irradiance and temperature exhibit the strongest influence, while humidity and wind speed have lower contributions.

4.3. Discussion and Comparative Analysis

The observed performance gains when the Hybrid LSTM is applied to electricity demand forecasting in Tetouan City have led to a striking improvement in performance relative to traditional machine learning models. But in actual application in an electric urban area like Tetouan City, the gains cannot be overlooked either. However, based on different kinds of data, in contrast to traditional models such as Random Forest, Decision Tree, and Support Vector Regression which have been studied by Salam and El Hibaoui [2] on this exact same data, our LSTM-based process can cope with urban energy demand’s inherent non-linear properties and time dimensionality. We argue that these findings confirm the existing wisdom expressed in [10] and show that recurrent architectures like LSTMs always outperform classical statistical and machine learning models in time-series forecasting, as they effectively capture the long-term temporal dependencies and complicated sequence patterns.

Classical machine learning approaches often struggle to capture long-term temporal dependencies and complex non-linear relationships. The proposed hybrid LSTM–XGBoost framework addresses these challenges by combining sequential modeling with residual correction. The superior forecast accuracy established by this work further provides a strong benchmark for the Tetouan dataset, and consequently, provides a good basis for the current energy forecasting literature, which is moving toward hybrid and ensemble models at an increasing rate [7]. To compare our results with recent works such as the “shifting-approach” machine learning approach presented in [11], Islam et al. reported R² and reached a value of about 0.825, while our Hybrid LSTM–XGBoost model achieved a much higher R² of 0.984. This considerable enhancement indicates empirically that explicit modeling of residual errors with XGBoost provides a more robust method for obtaining the predictive data extracted from the intricate grid phenomena of Tetouan City compared to feature shifting, which offers accurate and fine-grained perception in the power consumption process. In addition, the performance of our hybrid architecture is better than that of optimized single deep learning models (e.g., the study in [12]) on household load forecasting, highlighting the positive aspect of our integrated method.

The goodness-of-fit of our residual learning scheme is attributed to its synergism. The LSTM network establishes a good baseline through an efficient modeling of long-term temporal relations and the general tendencies observed over time (i.e., consumption patterns). Our XGBoost mechanism perfectly rectifies the LSTM for systematic errors to be present, specifically, during the challenging periods with the highest load and high short-term stochastic volatility between long-term storage value [3,4,5]. As previously noted, this “prediction-correction” approach where XGBoost targets the non-linear errors which are not catered to by LSTM is a robust approach to improve the accuracy of this prediction, and its success has been demonstrated in a number of studies [3,4,5]. This architecture directly drives a drastic decrease in RMSE and MAPE and it offers much better forecasts for significant operational decisions. This approach is also in line with the established scientific trend that hybrid feature-based regression algorithms with large error variance reduce the error variance of single-model architectures, thus increasing robustness of features to inherent stochasticity and volatility in energy time series data [6,7,8,9,10,11,12,13].

Overall, the developed Hybrid LSTM–XGBoost model based on its well-tuned residual learning scheme is well-suited to coping with the stochastic characteristics faced by the Tetouan power grid. Through the efficient transition between comprehensive and dynamic long-term trend prediction and accurate short-term volatile analysis, this paradigm enhances the accuracy and reliability of electricity load forecasting, which has direct implications for smart-grid and energy management in practice.

Although the proposed hybrid model achieves strong predictive performance on the 2017 Tetouan Zone 2 dataset, some limitations should be acknowledged. First, the evaluation is restricted to a single year of data, which may not fully capture atypical seasonal variations or interannual changes in consumption behavior. Second, the two-stage LSTM–XGBoost framework introduces a higher computational cost than a single standalone model, since it requires sequential training of both components. While this additional complexity is justified by the gain in forecasting accuracy, future work should further examine the trade-off between predictive performance and computational efficiency.

5. Conclusions

This work proposes a short-term electricity load forecasting framework for the Tetouan grid and evaluates it across Zones 1, 2, and 3, with Zone 2 used as the main benchmark case. Our LSTM-XGBoost model combines the LSTM’s ability to represent the overall temporal dynamics of consumption with XGBoost’s residual error correction feature. The results show that this improves the quality of forecasts compared to a single deep learning model and recent methods based on feature shifting. This framework is particularly useful for power grid managers in the operational sense, as it allows for more accurate prediction of times of high demand and better management of grid imbalances. The multi-zone evaluation confirms the generalizability of the framework across Zones 1, 2, and 3. Future work could further extend this analysis to additional datasets, integrate additional exogenous variables, or explore attention-based architectures.