ShortTerm Forecasting of Wind Energy: A Comparison of Deep Learning Frameworks
Abstract
:1. Introduction
2. Data, Models, and Methodology
2.1. Dataset Description
2.2. Deep Learning Strategies for Time Series Forecasting Using LSTM
2.2.1. LSTM RNNs Based Models Description
2.2.2. Vanilla LSTM
2.2.3. Bidirectional LSTM
2.2.4. Stacked LSTM
2.2.5. Convolutional LSTM
2.2.6. Autoencoder LSTM
2.3. Implementation Methodology
Algorithm 1 Vanilla LSTM 

Algorithm 2 Stacked LSTM 

Algorithm 3 Bidirectional LSTM 

Algorithm 4 CNN LSTM 

Algorithm 5 Autoencoder LSTM 

2.4. Evaluation Metrics
3. Wind Power Forecasting—Experimental Results
3.1. OneStep Forecasting
3.2. OnetoThreeSteps Forecasting
4. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
Abbreviations
AE  Autoencoder 
ANN  Artificial Neural Network 
ARIMA  Autoregressive Integrated Moving Average Model 
BPNN  Back Propagation Neural Networks 
ConvLSTM  Convolutional LSTM 
CNN  Convolutional Neural Network 
DE  Differential Evolution 
DL  Deep Learning 
DW  Discrete Wavelet Transform 
ESN  Echo State Network 
FFNN  FeedForward Neural Networks 
GA  Genetic Algorithm 
GHG  GreenHouse Gas 
GLSTM  Genetic Long ShortTerm Memory 
KF  Kalman filter 
LMBNN  Levenberg–Marquardt Backpropagation Neural Network 
LSTM  Long ShortTerm Memory 
MAPE  Mean Absolute Percentage Error 
MARS  Multivariate Adaptive Regression Splines 
ML  Machine Learning 
MSE  Mean Square Error 
NWP  Numerical Weather Prediction 
PCA  Principal Component Analysis 
Quantile–Quantile  
RMSE  Root Mean Square Error 
RNN  Recurrent Neural Network 
STSRLSTM  Sequencetosequence Long Shortterm Memory Regression 
SVM  Support Vector Machine 
VMD  Variational Mode Decomposition 
References
 Tong, W. Wind Power Generation and Wind Turbine Design, 1st ed.; WIT Press: Billerica, MA, USA, 2010. [Google Scholar]
 Renewables 2020 Global Status Report. 2020. Available online: https://ren21.net/gsr2020/ (accessed on 1 August 2021).
 Li, L.; Lin, J.; Wu, N.; Xie, S.; Meng, C.; Zheng, Y.; Wang, X.; Zhao, Y. Review and outlook on the international renewable energy development. Energy Built Environ. 2020, in press. [Google Scholar] [CrossRef]
 Giebel, G.; Kariniotakis, G.; Brownsword, R. The stateoftheart in short term prediction of wind power from a danish perspective. In Proceedings of the 4th International Workshop on Large Scale Integration of Wind Power and Transmission Networks for Offshore Wind Farms, Billund, Denmark, 20–21 October 2003. [Google Scholar]
 Du, P.; Wang, J.; Yang, W.; Niu, T. A novel hybrid model for shortterm wind power forecasting. Appl. Soft Comput. 2019, 80, 93–106. [Google Scholar] [CrossRef]
 Marulanda, G.; Bello, A.; Cifuentes Quintero, J.; Reneses, J. Wind Power LongTerm Scenario Generation Considering SpatialTemporal Dependencies in Coupled Electricity Markets. Energies 2020, 13, 3427. [Google Scholar] [CrossRef]
 Ren, Y.; Suganthan, P.; Srikanth, N. Ensemble methods for wind and solar power forecasting—A stateoftheart review. Renew. Sustain. Energy Rev. 2015, 50, 82–91. [Google Scholar] [CrossRef]
 Cao, Y.; Gui, L. MultiStep wind power forecasting model Using LSTM networks, Similar Time Series and LightGBM. In Proceedings of the 2018 5th International Conference on Systems and Informatics (ICSAI), Nanjing, China, 10–12 November 2018; pp. 192–197. [Google Scholar] [CrossRef]
 Abdelaziz, A.Y.; Rahman, M.A.; ElKhayat, M.; Hakim, M. Short term wind power forecasting using autoregressive integrated moving average modeling. In Proceedings of the 15th International Middle East Power Systems Conference, Alexandria, Egypt, 23–25 December 2012. [Google Scholar]
 Cifuentes, J.; Marulanda, G.; Bello, A.; Reneses, J. Air temperature forecasting using machine learning techniques: A review. Energies 2020, 13, 4215. [Google Scholar] [CrossRef]
 Rodriguez, M.A.; Sotomonte, J.F.; Cifuentes, J.; BuenoLópez, M. A Classification Method for PowerQuality Disturbances Using Hilbert–Huang Transform and LSTM Recurrent Neural Networks. J. Electr. Eng. Technol. 2020, 16, 249–266. [Google Scholar] [CrossRef]
 Eseye, A.; Zhang, J.; Zheng, D.; Shiferaw, D. ShortTerm Wind Power Forecasting Using Artificial Neural Networks for Resource Scheduling in Microgrids. Int. J. Sci. Eng. Appl. 2016, 5, 144–151. [Google Scholar] [CrossRef]
 Shi, J.; Liu, Y.; Yang, Y.; Lee, W. Shortterm wind power prediction based on wavelet transformsupport vector machine and statistic characteristics analysis. In Proceedings of the 2011 IEEE Industrial and Commercial Power Systems Technical Conference, Newport Beach, CA, USA, 1–5 May 2011; pp. 1–7. [Google Scholar] [CrossRef] [Green Version]
 Makhloufi, S.; Pillai, G.G. Wind speed and wind power forecasting using wavelet denoisingGMDH neural network. In Proceedings of the 2017 5th International Conference on Electrical EngineeringBoumerdes (ICEEB), Boumerdes, Algeria, 29–31 October 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 1–5. [Google Scholar]
 Elattar, E.E.; Goulermas, J.Y.; Wu, Q.H. Generalized locally weighted GMDH for short term load forecasting. IEEE Trans. Syst. Man Cybern. Part (Appl. Rev.) 2011, 42, 345–356. [Google Scholar] [CrossRef]
 Yuan, X.; Chen, C.; Yuan, Y.; Huang, Y.; Tan, Q. Shortterm wind power prediction based on LSSVM–GSA model. Energy Convers. Manag. 2015, 101, 393–401. [Google Scholar] [CrossRef]
 Tascikaraoglu, A.; Uzunoglu, M. A review of combined approaches for prediction of shortterm wind speed and power. Renew. Sustain. Energy Rev. 2014, 34, 243–254. [Google Scholar] [CrossRef]
 Yildiz, C.; Acikgoz, H.; Korkmaz, D.; Budak, U. An improved residualbased convolutional neural network for very shortterm wind power forecasting. Energy Convers. Manag. 2021, 228, 113731. [Google Scholar] [CrossRef]
 Shewalkar, A.; Nyavanandi, D.; Ludwig, S.A. Performance evaluation of deep neural networks applied to speech recognition: RNN, LSTM and GRU. J. Artif. Intell. Soft Comput. Res. 2019, 9, 235–245. [Google Scholar] [CrossRef] [Green Version]
 Li, P.; Zhou, K.; Lu, X.; Yang, S. A hybrid deep learning model for shortterm PV power forecasting. Appl. Energy 2020, 259, 114216. [Google Scholar] [CrossRef]
 Althelaya, K.A.; ElAlfy, E.S.M.; Mohammed, S. Stock market forecast using multivariate analysis with bidirectional and stacked (LSTM, GRU). In Proceedings of the 2018 21st Saudi Computer Society National Computer Conference (NCC), Riyadh, Saudi Arabia, 25–26 April 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1–7. [Google Scholar]
 Qu, X.; Kang, X.; Zhang, C.; Jiang, S.; Ma, X. Shortterm prediction of wind power based on deep Long ShortTerm Memory. In Proceedings of the 2016 IEEE PES AsiaPacific Power and Energy Engineering Conference (APPEEC), Xi’an, China, 25–28 October 2016; pp. 1148–1152. [Google Scholar] [CrossRef]
 Liu, Y.; Guan, L.; Hou, C.; Han, H.; Liu, Z.; Sun, Y.; Zheng, M. Wind Power ShortTerm Prediction Based on LSTM and Discrete Wavelet Transform. Appl. Sci. 2019, 9, 1108. [Google Scholar] [CrossRef] [Green Version]
 Sun, Z.; Zhao, M. ShortTerm Wind Power Forecasting Based on VMD Decomposition, ConvLSTM Networks and Error Analysis. IEEE Access 2020, 8, 134422–134434. [Google Scholar] [CrossRef]
 Ye, R.; Suganthan, P.N.; Srikanth, N.; Sarkar, S. A hybrid ARIMADENFIS method for wind speed forecasting. In Proceedings of the 2013 IEEE International Conference on Fuzzy Systems (FUZZIEEE), New Orleans, LA, USA, 23–26 June 2019; IEEE: Piscataway, NJ, USA, 2013; pp. 1–6. [Google Scholar]
 Lee, S.; Kim, J. Predicting Inflow Rate of the Soyang River Dam Using Deep Learning Techniques. Water 2021, 13, 2447. [Google Scholar] [CrossRef]
 SalcedoSanz, S.; PastorSánchez, A.; Prieto, L.; BlancoAguilera, A.; GarcíaHerrera, R. Feature selection in wind speed prediction systems based on a hybrid coral reefs optimization–Extreme learning machine approach. Energy Convers. Manag. 2014, 87, 10–18. [Google Scholar] [CrossRef]
 Xu, X.; Zhang, X.; Lu, L.; Deng, W.; Zuo, K. Fast nearinfrared palmprint recognition using nonnegative matrix factorization extreme learning machine. Opt. Appl. 2014, 44, 285–298. [Google Scholar]
 Liang, S.; Nguyen, L.; Jin, F. A Multivariable Stacked LongShort Term Memory Network for Wind Speed Forecasting. In Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA, 10–13 December 2018; pp. 4561–4564. [Google Scholar] [CrossRef] [Green Version]
 Vinothkumar, T.; Deeba, K. Hybrid wind speed prediction model based on recurrent long shortterm memory neural network and support vector machine models. Soft Comput. 2020, 24, 5345–5355. [Google Scholar] [CrossRef]
 Kilic, H.; Yuzgec, U.; Karakuzu, C. A novel improved antlion optimizer algorithm and its comparative performance. Neural Comput. Appl. 2020, 32, 3803–3824. [Google Scholar] [CrossRef]
 Chen, Y.; Wang, Y.; Dong, Z.; Su, J.; Han, Z.; Zhou, D.; Zhao, Y.; Bao, Y. 2D regional shortterm wind speed forecast based on CNNLSTM deep learning model. Energy Convers. Manag. 2021, 244, 114451. [Google Scholar] [CrossRef]
 Hu, H.; Wang, L.; Tao, R. Wind speed forecasting based on variational mode decomposition and improved echo state network. Renew. Energy 2021, 164, 729–751. [Google Scholar] [CrossRef]
 Shahid, F.; Zameer, A.; Muneeb, M. A novel genetic LSTM model for wind power forecast. Energy 2021, 223, 1200692. [Google Scholar] [CrossRef]
 Ahmad, T.; Zhang, D. A datadriven deep sequencetosequence longshort memory method along with a gated recurrent neural network for wind power forecasting. Energy 2022, 239, 122109. [Google Scholar] [CrossRef]
 Spanish Wind Energy Association—Wind Energy in Spain. 2021. Available online: https://aeeolica.org/en/aboutwindenergy/windenergyinspain/ (accessed on 1 August 2021).
 Spanish Peninsula—Electricity Demand Tracking in Real Time. 2021. Available online: https://demanda.ree.es/visiona/peninsula/demanda/ (accessed on 1 August 2021).
 Chen, C.; Liu, L.M. Joint estimation of model parameters and outlier effects in time series. J. Am. Stat. Assoc. 1993, 88, 284–297. [Google Scholar]
 Gupta, M.; Gao, J.; Aggarwal, C.C.; Han, J. Outlier detection for temporal data: A survey. IEEE Trans. Knowl. Data Eng. 2013, 26, 2250–2267. [Google Scholar] [CrossRef]
 Tavakoli, N. Modeling Genome Data Using Bidirectional LSTM. In Proceedings of the 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), Milwaukee, WI, USA, 15–19 July 2019; Volume 2, pp. 183–188. [Google Scholar] [CrossRef]
 Staudemeyer, R.C.; Morris, E.R. Understanding LSTM—A tutorial into Long ShortTerm Memory Recurrent Neural Networks. arXiv 2019, arXiv:1909.09586. [Google Scholar]
 Zhen, H.; Niu, D.; Yu, M.; Wang, K.; Liang, Y.; Xu, X. A Hybrid Deep Learning Model and Comparison for Wind Power Forecasting Considering TemporalSpatial Feature Extraction. Sustainability 2020, 12, 9490. [Google Scholar] [CrossRef]
 Yu, L.; Qu, J.; Gao, F.; Tian, Y. A novel hierarchical algorithm for bearing fault diagnosis based on stacked LSTM. Shock Vib. 2019, 2019, 2756284. [Google Scholar] [CrossRef]
 Essein, A.; Giannetti, C. A Deep Learning Model for Smart Manufacturing Using Convolutional LSTM Neural Network Autoencoders. IEEE Trans. Ind. Inform. 2020, 16, 6069–6078. [Google Scholar] [CrossRef] [Green Version]
 O’Shea, K.; Nash, R. An Introduction to Convolutional Neural Networks. arXiv 2015, arXiv:1511.08458. [Google Scholar]
 Liu, G.; Bao, H.; Han, B. A Stacked AutoencoderBased Deep Neural Network for Achieving Gearbox Fault Diagnosis. Math. Probl. Eng. 2018, 2018, 5105709. [Google Scholar] [CrossRef] [Green Version]
 Benhmad, F.; Percebois, J. Wind power feedin impact on electricity prices in Germany 2009–2013. Eur. J. Comp. Econ. 2016, 13, 81. [Google Scholar]
 Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
 Chang, Z.; Zhang, Y.; Chen, W. Effective AdamOptimized LSTM Neural Network for Electricity Price Forecasting. In Proceedings of the 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China, 23–25 November 2018; pp. 245–248. [Google Scholar] [CrossRef]
 Kumar Dubey, A.; Kumar, A.; GarcíaDíaz, V.; Kumar Sharma, A.; Kanhaiya, K. Study and analysis of SARIMA and LSTM in forecasting time series data. Sustain. Energy Technol. Assess. 2021, 47, 101474. [Google Scholar] [CrossRef]
 Nhu, V.H.; Hoang, N.D.; Nguyen, H.; Ngo, P.T.T.; Bui, T.T.; Hoa, P.V.; Samui, P.; Bui, D.T. Effectiveness assessment of Keras based deep learning with different robust optimization algorithms for shallow landslide susceptibility mapping at tropical area. Catena 2020, 188, 104458. [Google Scholar] [CrossRef]
 Chen, S.; Yang, J.; Liu, Y.; Tang, K.; Song, P. Evaluation and Improvement of Power Forecasting Indicators for Renewable Energy Stations. In Proceedings of the 2019 IEEE 3rd International Electrical and Energy Conference (CIEEC), Beijing, China, 7–9 September 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1164–1169. [Google Scholar]
 Elsaraiti, M.; Merabet, A.; AlDurra, A. Time Series Analysis and Forecasting of Wind Speed Data. In Proceedings of the 2019 IEEE Industry Applications Society Annual Meeting, Baltimore, MD, USA, 29 September–3 October 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1–5. [Google Scholar]
 Prema, V.; Sarkar, S.; Rao, K.U.; Umesh, A. LSTM based Deep Learning model for accurate wind speed prediction. Data Sci. Mach. Learn. 2019, 1, 6–11. [Google Scholar]
 Chhay, L.; Reyad, M.A.H.; Suy, R.; Islam, M.R.; Mian, M.M. Municipal solid waste generation in China: Influencing factor analysis and multimodel forecasting. J. Mater. Cycles Waste Manag. 2018, 20, 1761–1770. [Google Scholar] [CrossRef]
 Jalali, M.F.M.; Heidari, H. Predicting changes in Bitcoin price using grey system theory. Financ. Innov. 2020, 6, 1–12. [Google Scholar]
 Rodriguez, M.A.; Sotomonte, J.F.; Cifuentes, J.; BuenoLópez, M. Power Quality Disturbance Classification via Deep Convolutional AutoEncoders and Stacked LSTM Recurrent Neural Networks. In Proceedings of the 2020 International Conference on Smart Energy Systems and Technologies (SEST), Virtual, 7–9 September 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–6. [Google Scholar]
 Wang, P.; Rowe, J.P.; Min, W.; Mott, B.W.; Lester, J.C. Interactive Narrative Personalization with Deep Reinforcement Learning. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI, Melbourne, Australia, 19–25 August 2017; pp. 3852–3858. [Google Scholar]
Input Sequence Size  Epochs  Batch Size  Neurons 

Weekly (168 steps)  100  12  32 
Monthly (720 steps)  200  24  64 
Quarterly (2160 steps)  300  46, 48  128 
Input Sequence Size  Epochs  Batch Size  Neurons (Layer 1)  Neurons (Layer 2) 

Weekly (168 steps)  100  12  32  32 
Monthly (720 steps)  200  24  64  64 
Quarterly (2160 steps)  300  46, 48 
Model  Input Time Steps  Min MAPE (%)  Max MAPE (%)  Mean MAPE (%)  Implementation Time hh:mm:ss  Optimum Configuration by Lowest MAPE (%) 

Vanilla LSTM  Weekly (168)  4.30  5.38  4.75  05:50:03  Time steps: 168 
Monthly (720)  4.61  8.49  5.93  06:43:03  Neurons: 32  
Quarterly (2160)  4.84  12.21  6.56  11:09:53  Epochs: 200  
Batch Size: 46  
Bidirectional LSTM  Weekly (168)  4.44  5.53  4.85  08:04:21  Time steps: 168 
Monthly (720)  4.59  8.91  6.19  13:18:53  Neurons: 32  
Quarterly (2160)  4.79  7.04  5.62  18:50:08  Epochs: 200  
Batch Size: 46  
Stacked LSTM  Weekly (168)  4.37  5.13  4.71  06:25:27  Time steps: 168 
Monthly (720)  4.68  8.99  6.18  09:07:01  Neurons per layer: 32, 32  
Quarterly (2160)  4.73  14.13  7.22  16:23:17  Epochs: 300  
Batch Size: 12  
Convolutional LSTM  Weekly (168)  5.45  10.15  7.49  21:47:49  Time steps: 720 
Monthly (720)  5.31  7.31  6.60  27:19:05  Neurons: 128  
Quarterly (2160)  5.87  6.81  6.60  57:39:11  Epochs: 100  
Batch Size: 12 
Model  Forecasted Steps  MAPE (%)  RMSE (MWh)  MAE (MWh)  ${\mathit{R}}^{2}$ (%)  Implementation Time hh:mm:ss  Model Configuration 

Vanilla LSTM  1  4.47  269.64  200.49  99.41  02:42:46  Time steps: 168 
2  9.04  512.83  375.89  97.74  Neurons: 32  
3  13.61  745.22  551.67  95.25  Epochs: 200, Batch Size: 46  
Stacked LSTM  1  4.17  259.76  188.01  99.40  19:24:30  Time steps: 168 
2  8.98  514.61  382.79  97.68  Neurons—Layer 1 and 2: 32, 32  
3  13.86  749.57  566.29  95.13  Epochs: 300, Batch Size: 12  
Bidirectional LSTM  1  4.50  267.95  197.26  99.40  03:44:40  Time steps: 168 
2  8.76  510.28  378.83  97.68  Neurons: 32  
3  13.27  741.82  558.82  95.11  Epochs: 200, Batch Size: 46  
Autoencoder LSTM  1  4.52  289.27  211.41  99.35  41:54:38  Time steps: 2160 
2  8.91  554.97  412.58  97.47  Neurons: 32  
3  13.46  807.43  605.94  94.57  Epochs: 100, Batch Size: 12  
Convolutional LSTM  1  8.24  463.22  342.99  98.13  03:53:02  Time steps: 720 
2  12.72  718.48  555.27  95.24  Neurons: 128  
3  17.21  962.18  757.90  91.29  Epochs: 100, Batch Size: 12  
MARS  1  6.77  373.64  284.29  98.75  00:00:10  
2  12.98  685.52  526.94  95.81  
3  18.82  953.36  737.91  91.89  
M5TREE  1  6.71  373.94  284.04  98.76  00:00:07  
2  12.66  686.66  526.09  95.81  
3  18.04  956.06  736.28  91.89 
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. 
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mora, E.; Cifuentes, J.; Marulanda, G. ShortTerm Forecasting of Wind Energy: A Comparison of Deep Learning Frameworks. Energies 2021, 14, 7943. https://doi.org/10.3390/en14237943
Mora E, Cifuentes J, Marulanda G. ShortTerm Forecasting of Wind Energy: A Comparison of Deep Learning Frameworks. Energies. 2021; 14(23):7943. https://doi.org/10.3390/en14237943
Chicago/Turabian StyleMora, Elianne, Jenny Cifuentes, and Geovanny Marulanda. 2021. "ShortTerm Forecasting of Wind Energy: A Comparison of Deep Learning Frameworks" Energies 14, no. 23: 7943. https://doi.org/10.3390/en14237943