An Integrated Model of Deep Learning and Heuristic Algorithm for Load Forecasting in Smart Grid
Abstract
:1. Introduction
- A FPP-MLP-GWDO has been developed, where the preprocessing FPP and postprocessing GWDO have been cascaded with the MLP for accuracy improvement.
- Based on existing feature selection techniques [37], FPP has been developed where the feature interaction concept has been introduced, in addition to filters (irrelevancy, redundancy) for the selection of key features.
- The GWDO has been applied to the returned predictions from the MLP to further improve the accuracy by optimizing the filter threshold (irrelevancy and redundancy) weights and biases.
- The developed hybrid model, FPP-MLP-GWDO, is evaluated using Dayton Ohio grid load data regarding aspects of accuracy (the mean absolute percentage error (MAPE), Theil’s inequality coefficient (TIC), and the correlation coefficient (CC)) and convergence speed (the CT and CR). The findings endorsed the validity and superiority of the developed model compared to the literature models such as the feature selection–support vector machine–modified enhanced differential evolution (FS-SVM-mEDE) [38], the feature selection–artificial neural network (FS-ANN) [26], the support vector machine–differential evolution algorithm (SVM-DEA) [39], and the autoregressive (AR) model regarding aspects of accuracy and convergence speed.
Refs. | Methods | Objectives | Performance Metrics | Advantages/Disadvantages |
---|---|---|---|---|
Time series methods [6,7,8,9,10,11,12,13,14] | Exponential smoothing, Kalman filters, regression methods, grey model, ARIMA, and ARMAX | Forecast accuracy improvement | MAE, MAPE, RMSE | These prediction methods are capable of forecasting electric load. However, the accuracy improvement is not up to the mark due to the method’s inherent shortcomings. For instance, linear regressors are suitable for solving linear problems, but they perform worst while addressing nonlinear problems. Methods such as the ARIMA take historical/current records for prediction while ignoring other influencing parameters. GM methods can only cater to exponential growth trend problems. |
Artificial intelligence methods [16,17,18,19,20] | Expert systems, radial basis fuzzy logic models, machine learning models, and neural networks | Accuracy enhancement and compilation time improvement | MAPE, correlation coefficient | AI models outperform traditional methods when it comes to accuracy. Nevertheless, these advanced techniques have their limitations. For example, expert systems necessitate extensive knowledge acquisition and can need help with handling uncertainty. Radial basis logic models, on the other hand, are computationally expensive and exhibit limited generalization capabilities. Neural network models are powerful. However, they easily get stuck in local optima. |
Deep neural networks and hybrid models [24,25,26,27,28] | MLP, LSTM, RBM, etc. | Accuracy improvement | RMSE, MSE, R, etc. | Deep learning models have been introduced to address the limitations of existing forecasting methods and to enhance prediction accuracy. Nevertheless, these models come with a notable drawback: their high computational complexity. While deep-layer models excel in terms of accuracy compared to intrinsic methods, they often overlook the significance of data preprocessing techniques, which are crucial for achieving improved accuracy. |
Integrated FS-FCRBM-GWDO model [36] | AFC-STLF, MI-mEDE-ANN, FS-ANN, Bi-level. | Accuracy improvement | MAPD, variance, correlation, etc. | A hybrid model is introduced to tackle the constraints associated with current forecasting techniques. The aim is to enhance the prediction accuracy and convergence speed. However, the developed model has comparatively high complexity and a slow convergence. |
This work | FPP-MLP-GWDO | Accuracy, convergence rate, and computational time improvement | TIC, CC, CR, CT, and MAPE | The FPP-MLP-GWDO hybrid model offers several significant advantages over existing models in load forecasting. Its notably improved forecast accuracy stands out, thereby making it a valuable model for precise predictions. Moreover, its ability to converge quickly is ideal for real-time applications, while its adaptability allows it to handle various scenarios and datasets effectively. The model’s prowess in capturing nonlinear load patterns ensures accuracy even in complex situations, and its robustness in the face of data variability instills confidence in its reliability. Furthermore, it enhances resource allocation, thus leading to cost savings, and it is designed for scalability to meet the evolving demands of expanding smart city infrastructures. Lastly, the integration of feature preprocessing simplifies data preparation, thereby streamlining the forecasting process. Overall, the FPP-MLP-GWDO model significantly advances load forecasting, thus offering improved accuracy, efficiency, and fast convergence speed. |
2. Developed Hybrid FPP-MLP-GWDO Model
2.1. Feature Preprocessing
2.1.1. Relevant Feature Selection: Relevancy Filter
2.1.2. Redundant Feature Elimination: Redundancy Filter
2.1.3. Feature Interaction
2.1.4. FPP Stepwise Procedure
- The enclosed blocks within the dashed box represent the prefiltering part, during which the relevancy/interaction are computed. The potential inputs are then ranked according to these computed estimates/measures.
- We assess the individual and the gained information to measure the information content. This is done using an adapted form of Equation (4), which is illustrated in the flowchart presented in Figure 2. The function used in the equation monotonically increases, while the weight factor balances the relevancy and interaction measures. Depending on the specific forecasting problem, this factor can be adjusted and finely tuned.
- The potential inputs identified in the prefiltering step () are organized in a descending sequence as per their information value.
- The prefiltering stage output serves as the input for the filtering stage, where the preselected features are divided into selected () and nonselected () features, as illustrated in Figure 2. Redundancy measure is computed using Equation (9), which is modeled below:Here, represents the measure of redundancy for every potential input belonging to the set .
- The assessment of the informational significance of the potential features comprises three metrics: redundancy, relevancy, and interaction. In mathematical terms, this evaluation can be expressed as follows:Here, , represents the information content, denotes a monotonically increasing linear function, and corresponds to a tuneable parameter.
- The determination of the information content is made using the following decision process:In this process, information content is compared to the redundancy threshold, which is denoted as . If the information value is equal to or greater than the relevancy threshold, it is added to the list of selected features (). Otherwise, it is included in the list of (), which includes nonselected features.
2.2. MLP Forecaster
2.3. GWDO Optimizer
3. Results and Discussions
- The convergence speed encompasses two aspects: the CT and CR. The CT refers to the time it takes for the forecaster to return the predicted load pattern. On the other hand, the CR represents the rate at which the model converges to an iteration returning an optimal result, where the error no longer decreases significantly with increasing iterations. Forecasts with lower CT and CR values (requiring fewer epochs to converge) are considered faster, while higher CT and CR values indicate slower convergence. In this study, the CT is expressed in seconds, while the CR is in aspects of iterations.
3.1. Accuracy Evaluation
3.1.1. Day-Ahead Load Prediction
3.1.2. Week-Ahead Load Prediction
3.2. Convergence Speed Evaluation
3.2.1. Convergence Speed in terms of the CT
3.2.2. Convergence Speed in Terms of the CR
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Hafeez, G.; Khan, I.; Jan, S.; Shah, I.A.; Khan, F.A.; Derhab, A. A novel hybrid load forecasting framework with intelligent feature engineering and optimization algorithm in smart grid. Appl. Energy 2021, 299, 117178. [Google Scholar] [CrossRef]
- Hashmi, M.H.; Ullah, Z.; Asghar, R.; Shaker, B.; Tariq, M.; Saleem, H. An Overview of the current challenges and Issues in Smart Grid Technologies. In Proceedings of the 2023 International Conference on Emerging Power Technologies (ICEPT), Topi, Pakistan, 6–7 May 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–6. [Google Scholar]
- Asghar, R.; Sulaiman, M.H.; Saeed, S.; Wadood, H.; Mehmand, T.K.; Ullah, Z. Application of linear and nonlinear control schemes for the stability of Smart Grid. In Proceedings of the 2022 International Conference on Emerging Technologies in Electronics, Computing and Communication (ICETECC), Jamshoro, Sindh, Pakistan, 7–9 December 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–6. [Google Scholar]
- Mohan, N.; Soman, K.P.; Kumar, S.S. A data-driven strategy for short-term electric load forecasting using dynamic mode decomposition model. Appl. Energy 2018, 232, 229–244. [Google Scholar] [CrossRef]
- Glavan, M.; Gradišar, D.; Moscariello, S.; Juričić, Đ.; Vrančić, D. Demand-side improvement of short-term load forecasting using a proactive load management—A supermarket use case. Energy Build. 2019, 186, 186–194. [Google Scholar] [CrossRef]
- Billah, B.; King, M.L.; Snyder, R.D.; Koehler, A.B. Exponential smoothing model selection for forecasting. Int. J. Forecast. 2006, 22, 239–247. [Google Scholar] [CrossRef]
- Rendon-Sanchez, J.F.; de Menezes, L.M. Structural combination of seasonal exponential smoothing forecasts applied to load forecasting. Eur. J. Oper. Res. 2019, 275, 916–924. [Google Scholar] [CrossRef]
- Ribeiro, M.I. Kalman and extended kalman filters: Concept, derivation and properties. Inst. Syst. Robot. 2004, 43, 3736–3741. [Google Scholar]
- Song, K.-B.; Baek, Y.; Hong, D.H.; Jang, G. Short-term load forecasting for the holidays using fuzzy linear regression method. IEEE Trans. Power Syst. 2005, 20, 96–101. [Google Scholar] [CrossRef]
- Wi, Y.-M.; Joo, S.-K.; Song, K.B. Holiday load forecasting using fuzzy polynomial regression with weather feature selection and adjustment. IEEE Trans. Power Syst. 2011, 27, 596–603. [Google Scholar] [CrossRef]
- Zhao, H.; Guo, S. An optimized grey model for annual power load forecasting. Energy 2016, 107, 272–286. [Google Scholar] [CrossRef]
- Huang, S.-J.; Shih, K.R. Short-term load forecasting via ARMA model identification including non-Gaussian process considerations. IEEE Trans. Power Syst. 2003, 18, 673–679. [Google Scholar] [CrossRef]
- de Andrade, L.C.M.; da Silva, I.N. Very short-term load forecasting based on ARIMA model and intelligent systems. In Proceedings of the 2009 15th International Conference on Intelligent System Applications to Power Systems, Curitiba, Brazil, 8–12 November 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 1–6. [Google Scholar]
- Bacha, S.A.; Ahmad, G.; Hafeez, G.; Albogamy, F.R.; Murawwat, S. Compensation of Data Loss Using ARMAX Model in State Estimation for Control and Communication Systems Applications. Energies 2021, 14, 7573. [Google Scholar] [CrossRef]
- Ma, T.; Wang, F.; Wang, J.; Yao, Y.; Chen, X. A combined model based on seasonal autoregressive integrated moving average and modified particle swarm optimization algorithm for electrical load forecasting. J. Intell. Fuzzy Syst. 2017, 32, 3447–3459. [Google Scholar] [CrossRef]
- Islam, B.U. Comparison of conventional and modern load forecasting techniques based on artificial intelligence and expert systems. Int. J. Comput. Sci. Issues (IJCSI) 2011, 8, 504. [Google Scholar]
- Gontar, Z.; Hatziargyriou, N. Short term load forecasting with radial basis function network. In Proceedings of the 2001 IEEE Porto Power Tech Proceedings (Cat. No. 01EX502), Porto, Portugal, 10–13 September 2001; IEEE: Piscataway, NJ, USA, 2001; Volume 3. [Google Scholar]
- Salkuti, S.R. Short-term electrical load forecasting using radial basis function neural networks considering weather factors. Electr. Eng. 2018, 100, 1985–1995. [Google Scholar] [CrossRef]
- Chung, W.H.; Gu, Y.H.; Yoo, S.J. District heater load forecasting based on machine learning and parallel CNN-LSTM attention. Energy 2022, 246, 123350. [Google Scholar] [CrossRef]
- Zambrano-Asanza, S.; Morales, R.E.; Montalvan, J.A.; Franco, J.F. Integrating artificial neural networks and cellular automata model for spatial-temporal load forecasting. Int. J. Electr. Power Energy Syst. 2023, 148, 108906. [Google Scholar] [CrossRef]
- Ullah, S.; Khan, Q.; Mehmood, A.; Kirmani, S.A.M.; Mechali, O. Neuro-adaptive fast integral terminal sliding mode control design with variable gain robust exact differentiator for under-actuated quadcopter UAV. ISA Trans. 2022, 120, 293–304. [Google Scholar] [CrossRef]
- Ullah, S.; Khan, Q.; Mehmood, A. Neuro-adaptive fixed-time non-singular fast terminal sliding mode control design for a class of under-actuated nonlinear systems. Int. J. Control 2023, 96, 1529–1542. [Google Scholar] [CrossRef]
- Yan, Z.; Zhu, X.; Wang, X.; Ye, Z.; Guo, F.; Xie, L.; Zhang, G. A multi-energy load prediction of a building using the multi-layer perceptron neural network method with different optimization algorithms. Energy Explor. Exploit. 2023, 41, 273–305. [Google Scholar] [CrossRef]
- Yazici, I.; Beyca, O.F.; Delen, D. Deep-learning-based short-term electricity load forecasting: A real case application. Eng. Appl. Artif. Intell. 2022, 109, 104645. [Google Scholar] [CrossRef]
- Sekhar, C.; Dahiya, R. Robust framework based on hybrid deep learning approach for short term load forecasting of building electricity demand. Energy 2023, 268, 126660. [Google Scholar] [CrossRef]
- Liang, Y.; Niu, D.; Hong, W.-C. Short term load forecasting based on feature extraction and improved general regression neural network model. Energy 2019, 166, 653–663. [Google Scholar] [CrossRef]
- Hu, R.; Wen, S.; Zeng, Z.; Huang, T. A short-term power load forecasting model based on the generalized regression neural network with decreasing step fruit fly optimization algorithm. Neurocomputing 2017, 221, 24–31. [Google Scholar] [CrossRef]
- Li, H.-Z.; Guo, S.; Li, C.-J.; Sun, J.-Q. A hybrid annual power load forecasting model based on generalized regression neural network with fruit fly optimization algorithm. Knowl.-Based Syst. 2013, 37, 378–387. [Google Scholar] [CrossRef]
- Awan, S.M.; Aslam, M.; Khan, Z.A.; Saeed, H. An efficient model based on artificial bee colony optimization algorithm with Neural Networks for electric load forecasting. Neural Comput. Appl. 2014, 25, 1967–1978. [Google Scholar] [CrossRef]
- Dai, Y.; Zhao, P. A hybrid load forecasting model based on support vector machine with intelligent methods for feature selection and parameter optimization. Appl. Energy 2020, 279, 115332. [Google Scholar] [CrossRef]
- Massana, J.; Pous, C.; Burgas, L.; Melendez, J.; Colomer, J. Identifying services for short-term load forecasting using data driven models in a Smart City platform. Sustain. Cities Soc. 2017, 28, 108–117. [Google Scholar] [CrossRef]
- Saoud, L.S.; Al-Marzouqi, H. Metacognitive sedenion-valued neural network and its learning algorithm. IEEE Access 2020, 8, 144823–144838. [Google Scholar] [CrossRef]
- Saoud, L.S.; Al-Marzouqi, H.; Deriche, M. Wind speed forecasting using the stationary wavelet transform and quaternion adaptive-gradient methods. IEEE Access 2021, 9, 127356–127367. [Google Scholar] [CrossRef]
- Nawaz, A.; Hafeez, G.; Khan, I.; Jan, K.U.; Li, H.; Khan, S.A.; Wadud, Z. An intelligent integrated approach for efficient demand side management with forecaster and advanced metering infrastructure frameworks in smart grid. IEEE Access 2020, 8, 132551–132581. [Google Scholar] [CrossRef]
- Bayraktar, Z.; Komurcu, M.; Bossard, J.A.; Werner, D.H. The wind driven optimization technique and its application in electromagnetics. IEEE Trans. Antennas Propag. 2013, 61, 2745–2757. [Google Scholar] [CrossRef]
- Hafeez, G.; Alimgeer, K.S.; Wadud, Z.; Shafiq, Z.; Khan, M.U.A.; Khan, I.; Khan, F.A.; Derhab, A. A novel accurate and fast converging deep learning-based model for electrical energy consumption forecasting in a smart grid. Energies 2020, 13, 2244. [Google Scholar] [CrossRef]
- Hafeez, G.; Alimgeer, K.S.; Khan, I. Electric load forecasting based on deep learning and optimized by heuristic algorithm in smart grid. Appl. Energy 2020, 269, 114915. [Google Scholar] [CrossRef]
- Hafeez, G.; Alimgeer, K.S.; Qazi, A.B.; Khan, I.; Usman, M.; Khan, F.A.; Wadud, Z. A hybrid approach for energy consumption forecasting with a new feature engineering and optimization framework in smart grid. IEEE Access 2020, 8, 96210–96226. [Google Scholar] [CrossRef]
- Wang, J.; Li, L.; Niu, D.; Tan, Z. An annual load forecasting model based on support vector regression with differential evolution algorithm. Appl. Energy 2012, 94, 65–70. [Google Scholar] [CrossRef]
- Chandrashekar, G.; Sahin, F. A survey on feature selection methods. Comput. Electr. Eng. 2014, 40, 16–28. [Google Scholar] [CrossRef]
- Amjady, N.; Keynia, F. Day-ahead price forecasting of electricity markets by mutual information technique and cascaded neuro-evolutionary algorithm. IEEE Trans. Power Syst. 2008, 24, 306–318. [Google Scholar] [CrossRef]
- Amjady, N.; Keynia, F.; Zareipour, H. Short-term load forecast of microgrids by a new bilevel prediction strategy. IEEE Trans. Smart Grid 2010, 1, 286–294. [Google Scholar] [CrossRef]
- Zhao, C.; Zheng, C.; Zhao, M.; Tu, Y.; Liu, J. Multivariate autoregressive models and kernel learning algorithms for classifying driving mental fatigue based on electroencephalographic. Expert Syst. Appl. 2011, 38, 1859–1865. [Google Scholar] [CrossRef]
- Rehman, M.Z.; Nawi, N.M. Improving the accuracy of gradient descent back propagation algorithm (GDAM) on classification problems. Int. J. New Comput. Archit. Their Appl. 2011, 1, 838–847. [Google Scholar]
- Pavlyuk, D. Short-term traffic forecasting using multivariate autoregressive models. Procedia Eng. 2017, 178, 57–66. [Google Scholar] [CrossRef]
- Schmidt, M.; Safarani, S.; Gastinger, J.; Jacobs, T.; Nicolas, S.; Schülke, A. On the performance of differential evolution for hyperparameter tuning. In Proceedings of the 2019 international joint conference on neural networks (IJCNN), Budapest, Hungary, 14–19 July 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1–8. [Google Scholar]
- Albogamy, F.R.; Khan, S.A.; Hafeez, G.; Murawwat, S.; Khan, S.; Haider, S.I.; Basit, A.; Thoben, K.-D. Real-time energy management and load scheduling with renewable energy integration in smart grid. Sustainability 1792, 14, 1792. [Google Scholar] [CrossRef]
- Bao, Z.; Zhou, Y.; Li, L.; Ma, M. A hybrid global optimization algorithm based on wind driven optimization and differential evolution. Math. Probl. Eng. 2015, 2015, 620–635. [Google Scholar] [CrossRef]
- Mirjalili, S.; Mirjalili, S. Genetic algorithm. Evol. Algorithms Neural Netw. Theory Appl. 2019, 780, 43–55. [Google Scholar]
- PJM Electricity Market. Available online: https://www.pjm.com/ (accessed on 23 April 2022).
Forecaster Parameters | Values |
---|---|
Number of epochs | 110 |
Output layer | 1 |
Number of output neurons | 1 |
Hidden layer | 2 |
Neurons in hidden layer | 10 |
Learning rate | |
Momentum | 0.6 |
Initial weight | 0.1 |
Initial bias | 0 |
Max | 0.9 |
Min | 0.1 |
Feature selection threshold | 0.5 |
Decision variables | 2 |
Number of objectives | 0 |
Population size | 24 |
Delay of weight | 0.002 |
Metrics | Load Forecasters | ||||
---|---|---|---|---|---|
AR | FS-ANN | SVM-DEA | FS-SVM-mEDE | FPP-MLP-GWDO | |
Complexity (level) | Low | Low | Moderate | High | Moderate |
CR (epochs) | 55th | 39th | 35th | 31th | 18th |
CT (s) | 132 | 159 | 240 | 350 | 299 |
Accuracy (%) | 95.7 | 96.5 | 97.5 | 97.9 | 98.999 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alghamdi, H.; Hafeez, G.; Ali, S.; Ullah, S.; Khan, M.I.; Murawwat, S.; Hua, L.-G. An Integrated Model of Deep Learning and Heuristic Algorithm for Load Forecasting in Smart Grid. Mathematics 2023, 11, 4561. https://doi.org/10.3390/math11214561
Alghamdi H, Hafeez G, Ali S, Ullah S, Khan MI, Murawwat S, Hua L-G. An Integrated Model of Deep Learning and Heuristic Algorithm for Load Forecasting in Smart Grid. Mathematics. 2023; 11(21):4561. https://doi.org/10.3390/math11214561
Chicago/Turabian StyleAlghamdi, Hisham, Ghulam Hafeez, Sajjad Ali, Safeer Ullah, Muhammad Iftikhar Khan, Sadia Murawwat, and Lyu-Guang Hua. 2023. "An Integrated Model of Deep Learning and Heuristic Algorithm for Load Forecasting in Smart Grid" Mathematics 11, no. 21: 4561. https://doi.org/10.3390/math11214561
APA StyleAlghamdi, H., Hafeez, G., Ali, S., Ullah, S., Khan, M. I., Murawwat, S., & Hua, L.-G. (2023). An Integrated Model of Deep Learning and Heuristic Algorithm for Load Forecasting in Smart Grid. Mathematics, 11(21), 4561. https://doi.org/10.3390/math11214561