Untangling Energy Consumption Dynamics with Renewable Energy Using Recurrent Neural Network

Abstract

The environmental issues we are currently facing require long-term prospective efforts for sustainable growth. Renewable energy sources seem to be one of the most practical and efficient alternatives in this regard. Understanding a nation’s pattern of energy use and renewable energy production is crucial for developing strategic plans. No previous study has been performed to explore the dynamics of power consumption with the change in renewable energy production on a country-wide scale. In contrast, a number of deep learning algorithms have demonstrated acceptable performance while handling sequential data in the era of data-driven predictions. In this study, we developed a scheme to investigate and predict total power consumption and renewable energy production time series for eleven years of data using a recurrent neural network (RNN). The dynamics of the interaction between the total annual power consumption and renewable energy production were investigated through extensive exploratory data analysis (EDA) and a feature engineering framework. The performance of the model was found to be satisfactory through the comparison of the predicted data with the observed data, the visualization of the distribution of the errors and root mean squared error (RMSE), and the R² values of 0.084 and 0.82. Higher performance was achieved by increasing the number of epochs and hyperparameter tuning. The proposed framework has the potential to be used and transferred to investigate the trend of renewable energy production and power consumption and predict future scenarios for different communities. The incorporation of a cloud-based platform into the proposed pipeline to perform predictive studies from data acquisition to outcome generation may lead to real-time forecasting.

1. Introduction

In recent decades, interest in renewable energy has grown significantly [1,2,3,4]. These non-polluting, resource-unrestricted energies would provide the perfect electrical source for any activity, whether household or industrial, if not for their unpredictability [5,6,7]. It is challenging to predict how much power will be gained from renewable sources, because their throughput varies greatly depending on the circumstances and qualities of the location in which they are found. In many nations today, it is essential to promote the use of renewable energy sources, because they provide a wealth of benefits [8,9,10]. As a result, while the main energy resource imports are greatly decreased, the security of the energy supply and the preservation of traditional resources are both guaranteed. Additionally, the use of renewable energy spurs economic growth on a local, regional, and international scale and generates new job possibilities [11,12,13,14]. Utilizing renewable energy has the advantage of lessening environmental degradation [15,16,17,18].

Solar energy has emerged as one of the most important sources of energy in recent years [19,20]. In some countries, solar energy uses a significant percentage of the sun’s energy and has more predictable behavior than wind-based energy. As a result, it ranks among the most significant renewable energy sources for a variety of nations in south Europe, including Spain, as well as other places along the same latitude, such as Saudi Arabia or India [21,22,23]. Solar energy includes thermal solar energy, which transforms solar radiation into thermal energy used to heat buildings, desalination plants, homes, and water treatment facilities, among other things, and photovoltaic solar energy, which transforms solar radiation into electrical energy that can be transported for purposes other than heating [24,25]. Wind is a plentiful natural resource and sustainable energy source that is known to be both clean and pollution-free. In general, the characteristics of wind are its speed, direction, and time of occurrence. The force or speed of the wind determines how much energy can be extracted from its natural flow [26,27]. Generally speaking, the wind speed or force has a nonlinear and variable nature. Despite its natural origins, wind has the capacity to produce the necessary amount of energy for a nation’s ongoing needs. It is necessary to forecast wind speeds in order to increase the amount of energy produced [28,29,30]. Wind speed forecasting strikes a balance between the energy generated and the demand. An efficient technique for reducing operating costs and enhancing grid system functionality is a wind speed prediction model that is very accurate and dependable [31,32,33,34,35].

The use of deep learning (DL) has made it possible to anticipate various physical systems with greater accuracy. Several industries use DL [36,37,38,39,40]. In the modern world, virtually every power grid incorporates renewable-energy-based sources. For successful participation in the electricity market, accurate predictions of renewable energy sources are crucial. Considering how much these sources depend on consumption, it can be difficult to forecast the planet’s production. In recent years, there has been a rise in ML research and applications for forecasting plant output from renewable energy sources. Different models, including feedforward backpropagation (FFBP), feedforward neural networks (FFNNs), and multilayer feedforward with backpropagation neural networks (MFFNNBP), with various learning algorithms, including Bayesian regularization (BR) and the Levenberg–Marquardt (LM) algorithm, can be found in references under the category of variants of neural networks [41,42,43]. Examples of these techniques include support vector regression (SVR), random trees (RTs), M5P decision trees (M5PDTs), Gaussian process regression (GPR), and physical photovoltaic forecasting models (P-PVFMs) [44,45,46,47]. Although several approaches for the supervised training of RNNs have been investigated over the past decade, and there are many types of training algorithms, none stand out as the ideal model. Backpropagation revisited and through time are common training methods for RNNs, as they combine the following two qualities: (1) they have a distributed hidden state that enables them to store a significant amount of historical data effectively; and (2) they implement nonlinear dynamics, which enables them to develop sophisticated ways to update their hidden state. These are the main reasons that RNN can compute a large dataset with enough neurons and time.

The use of RNN models to investigate the dynamics of energy consumption in relation to renewable energy is a relatively recent development [48,49,50,51,52]. This study aimed to evaluate how well RNN models can predict energy consumption using renewable energy sources. In order to enable researchers, engineers, and decision makers to understand the temporal dynamics of power consumption and renewable energy production so that they can make informed engineering/managerial decisions, the goal of this study was to build an effective and practical RNN framework for forecasting future scenarios of annual power production and consumption [53,54,55,56,57,58,59,60,61,62,63,64]. The model was tested against observed data to assess whether it performed well using daily energy consumption and renewable energy production data for a country. With this model, engineers and managers would be able to evaluate the short- and long-term behavior and trends of energy, allowing them to eventually develop preventative measures based on earlier observational data for a variety of issues in a region. The RNN-based method used in this study only requires observed data; therefore, a substantial amount of computational effort is needed. To make the most of the RNN results in this study, comprehensive exploratory data analysis, feature engineering, and hyperparameter optimization were conducted. The remainder of this paper is structured as follows: Section 2 provides a full explanation of the fundamentals of RNN and describes the data engineering and experimental methodology. The outcomes of the experiment are thoroughly discussed and analyzed in Section 3. The conclusion portion and closing thoughts regarding this article are presented in Section 4.

2. Data and Methods

2.1. Data Source and Workflow

The time series of the total energy consumption and wind and solar power production was used in this study to forecast the future trends of the variables in Germany. The time-series dataset was retrieved from open power system data (OPSD) for Germany, which has been rapidly expanding its renewable energy production in recent years [65,66]. The temporal resolution of the variables used for the RNN-based prediction was daily. The dataset’s timeframe included data over a decade, from 2006 to 2017. Electricity usage and generation from wind and solar sources were reported in gigawatt hours (GWh). In

Table 1. Full description of the energy consumption variables used for EDA and predictive analysis with RNN.

Energy Consumption Variables	Unit	Descriptions
Total consumption	GWh (gigawatt hours)	Daily total energy consumption
Wind power production	GWh	Daily wind power production
Solar power production	GWh	Daily solar power production

, a full description of the variables is presented. In Figure 1, the full workflow of the predictive analysis of renewable energy production and total energy consumption is shown.

The shifts and fluctuations in electricity usage and generation over time in Germany were scrutinized in this paper. Time-series tools were used to examine both seasonal variations and long-term trends in the production of wind and solar power, as well as their consumption. Furthermore, these tools were used to compare the wind and solar power production with electricity usage. Using an RNN model, we anticipated each day’s consumption based on historical and observed data. Figure 2 shows the temporal variation of the variables.

2.2. Multivariate Exploratory Data Analysis

As indicated by activity 2 in Figure 1, a multivariate exploratory data analysis (EDA) was performed to understand the internal distribution of the attributes of the variables. The temporal distribution of all the variables was explored using several visual and numerical representations. The EDA included the important process of conducting an initial exploration of the variables to investigate the hidden patterns in the dataset. The EDA was grouped into multiple activities in this study. The descriptive statistics of the variables are presented alongside the probability distribution via histograms to determine the normality (skewness) of the variables. Descriptive statistics provided an effective way to demonstrate the basic distribution of the values of the variables according to the number of data points, mean, standard deviation, percentiles, interquartile range, and range (max/min). The full multivariate descriptive statistics of the all the variables are shown in

Table 2. Descriptive statistics of the variables.

Variable	Count	Mean	Std	Min	25%	50%	75%	Max
Consumption	4383	1338.67	165.77	842.39	1217.85	1367.12	1457.76	1709.56
Wind	2920	164.81	143.69	5.75	62.35	119.09	217.90	826.27
Solar	2188	89.25	58.55	1.96	35.17	86.40	135.07	241.58

. To show the normality, histograms with a line of probability distribution were used as a visual representation, and the Pearson coefficient of skewness (PCS) was used as an indicator of skewness to analyze the distribution.

A visual representation of the distribution and the normality of the variables is shown in Figure 3. The overall non-linearity of the wind production was high with left skewness. The values of the Pearson correlation of skewness (PCS) were calculated for the numeric measurement of the skewness. The PCS values for wind and solar power production and total energy consumption were 5.97, 0.49, and 0.65. Wind power production showed the highest non-normality among the variables. Normal distribution is a very crucial part of RNN model performance, as it is directly linked to error minimization through backpropagation. Normal distribution is the most crucial factor in the field of data-driven predictive analysis, e.g., deep neural network regression. As the distribution of the values of wind power production was highly skewed to the left, showing significant non-normality, the neural network regression algorithms without appropriate data transformation did not contribute to satisfactory outcomes with good optimization. As the distribution of the renewable energy production series was found to be highly skewed, data transformation was performed to decrease the non-normality of the series in the feature engineering stage. The linear linkage was found to be low among the variables. The values of the linear correlation coefficients are shown in the bivariate correlation plot in Figure 4. The direction of the linear relationship was found to be both positive and negative.

2.3. Feature Engineering

As indicated by activity 3 in Figure 1, feature engineering (FE) was performed after a successful EDA. FE is an important step before the training/testing phase of an RNN algorithm. Without successful FE, any data-driven method may not yield a satisfactory performance with minimum errors. Adequate optimization through iterative gradient descent cannot be reached without the successful scrutiny of the dataset. Therefore, comprehensive feature engineering was performed to transform the dataset so that it was more suitable for the learning algorithm of the RNN. FE was performed to prepare the dataset for the predictive analysis. The FE involved imputation, data transformation, data standardization, and splitting the dataset into training and testing sets. Imputation was performed to fill the null values so that the entire dataset became consistent. In this research, null values or null observations were found in every variable. These cells in the dataset were imputed with the median of the entire series. Three methods of data transformation were considered, i.e., logarithmic, power, and cubic transformation, to bring the distribution of the features closer to normal distribution. The Pearson coefficient was used as an indicator of normality.

Through the data standardization process, the values of a variable were rescaled so that the variable had a mean of 0 and variance of 1 (or Z-score normalization), which was identical to the bell-shaped normal distribution curve. As the variable considered in this study was the continuous independent variable, the standardization of the variable was crucial for training/testing the neural network algorithm. Standardization was an important step for the optimization problem. The RNN recurrent neural network model used the gradient descent technique, with the feature value (renewable energy production) affecting the step size of the technique. Smooth progress towards minima in gradient descent required the updating of the steps at the same rate for all the feature values. A standardized variable is a prerequisite of reaching the minima in the gradient descent process.

$$z=\frac{x-\mu }{\sigma }$$

(1)

All the values in the renewable energy production series were standardized to prepare the training dataset for the RNN model.

Equation (1) shows the standardization formula for the renewable energy production series. The difference between the renewable energy production value and the minimum of the entire renewable energy production series was divided by the range of the series, providing the standardized data, which were further used in the training/testing process of the RNN, indicated as activity 4 in Figure 1. The entire standardized renewable energy production series was split into two portions, i.e., a training set that was used to train the model and a testing set that was used to test/evaluate the model. Seventy percent of the dataset was used for training, and thirty percent was used for testing. In a nutshell, EDA and feature engineering were pivotal steps for the satisfactory performance of the predictive model.

2.4. Recurrent Neural Network (RNN)

Recurrent neural networks (RNNs) are a type of neural system that can reveal dynamic temporal behavior by enabling the use of hidden states and previous outputs as inputs. RNNs, which are derived from feedforward neural networks, process input sequences of various lengths using their internal state (memory) and connect the outputs of all neurons to their inputs. The main structural concept of an NN is the replication of connection weight configurations to zero to imitate the lack of connections between particular neurons.

For each timestep $t$, the activation $a^{〈t〉}$ and the output $y^{〈t〉}$ are expressed as follows:

$$a^{〈t〉}=g_1\left(W_{aa}{a}^{〈t-1〉}+W_{ax}{x}^{〈t〉}+b_a\right)$$

(2)

$$y^{〈t〉}=g_2\left(W_{ya}{a}^{〈t〉}+b_y\right)$$

(3)

where $W_{ax}$, $W_{aa}$, $W_{ya}$, $b_a$, and $b_y$ are coefficients that are shared temporally, and $g_1$ and $g_2$ are activation functions (Figure 4).

In the case of a recurrent neural network, the loss function Ը of all timesteps is defined based on the loss at every timestep, as follows:

$$Ը\left(\widehat{y},y\right)={\displaystyle \sum }_{t=1}^{T_y}Ը\left({\widehat{y}}^{〈t〉},y^{〈t〉}\right)$$

(4)

Backpropagation was carried out at each point in time. At timestep T, the derivative of the loss Ը with respect to the weight matric W is expressed as follows:

$$\frac{\partial {Ը}^{\left(T\right)}}{\partial W}={\displaystyle \sum }_{t=1}^T{\left.\frac{\partial {Ը}^{\left(T\right)}}{\partial W}\right|}_{\left(t\right)}$$

(5)

2.5. Model Evaluation

As indicated by activity 5 in Figure 1, the model performance was evaluated. The coefficient of determination ($R^2$) is a popular error metric for assessing the accuracy of a model by depicting the model fitness to datapoint values. The better the model fits the data, the higher the $R^2$. The square root of the coefficient of determination is represented as the correlation coefficient (R), which was the second error function implemented in this study.

$$R^2=\frac{{\left({{\displaystyle \sum }}_{t=1}^N\left(Q_{t\left(com\right)}-{\overline{Q}}_{\left(com\right)}\right)\left(Q_{t\left(obs\right)}-{\overline{Q}}_{\left(obs\right)}\right)\right)}^2}{\left({{\displaystyle \sum }}_{t=1}^N{\left(Q_{t\left(com\right)}-{\overline{Q}}_{\left(com\right)}\right)}^2\right)\left({{\displaystyle \sum }}_{t=1}^N{\left(Q_{t\left(obs\right)}-{\overline{Q}}_{\left(obs\right)}\right)}^2\right)}$$

(6)

3. Results and Discussion

After the successful training of the RNN model, it was deployed to predict the target variable (activity 6 in Figure 1). The output from the RNN algorithm was compared to the observed renewable energy production data from the database through visualization, as shown in Figure 5. Both the observed and predicted renewable energy production time series were plotted against the number of datapoints. The overall distribution of the predicted renewable energy production values was approximately identical to that of the observed data, demonstrating the satisfactory performance of the RNN algorithm. After the RNN model was trained with the training portion of the dataset, the entire observed dataset was fed in to predict the outcome. The entire dataset was divided into training and testing sets in the proportions 70% and 30%. The training dataset was used to train the model, and the testing dataset was used to evaluate the model performance. The observed data are shown in Figure 5a in green. In Figure 5b, the deep cyan portion of the plot indicates the training portion of the dataset, whereas the deep blue portion shows the testing portion. The RMSE values of the training and testing portion were 0.097 and 0.045, respectively. The lower RMSE values showed the satisfactory performance of the RNN algorithm.

The rolling mean of the predicted consumption series showed the temporal dynamics of consumption in relation to the original observed consumption throughout the study period. It demonstrated the impact of downscaling the data with comparatively coarser datapoints for the target variable (consumption). The downscaling of a temporal series is usually conducted to reduce the size of the dataset to obtain faster computation ability as well as to reduce the burden of a large amount of observed data with a finer temporal resolution. However, observing and storing a large amount of data is always expensive and has significant limitations. Therefore, the rolling mean could be a good tool for determining whether there is any significant difference between observed and transformed data, with the potential to assist decision makers in evaluating future scenarios.

Model Evaluation Metrics and Improvement

The performance of the RNN neural network was evaluated using three error metrics, i.e., the root mean square error (RMSE), the coefficient of determination (R²), and the Nash–Sutcliffe model efficiency coefficient (E) [64]. Further, the performance of the model was also evaluated and improved through increasing the number of iterations, i.e., epochs, in the neural network. The value of R² is plotted against an increasing number of epochs in Figure 6. The number of epochs was increased up to 100 to improve the performance. The RMSE value was found to decrease from 0.01 to 0.0025, which indicated satisfactory performance by the RNN algorithm. The model performance increased significantly from the very beginning of the iterations for both the training and testing scenarios. The trend of change in the decrease in the RMSE values reached a near-steady state after 20 epochs. A local decrease in the performance, i.e., increase in the RMSE value, could be seen after 20 epochs (Figure 6).

Observed and predicted energy values from the RNN model and the distribution of the RMSE values are illustrated in Figure 7 using a scatter plot. The scatterplot shows that the points followed an approximately 45° trend line originating at zero. Some points were located outside of the main cluster of points, which showed an error in the prediction process. The R² value of the best-fitted straight line was +0.862, which indicated good performance.

Time-series prediction for renewable energy production and consumption is a pivotal task in the field of power management. The application of data-driven prediction models is highly efficacious in predicting energy variables without taking complicated equations and assumptions into consideration. In this study, annual power consumption and renewable energy production were predicted using the most powerful neural network for predicting sequential data, i.e., a recurrent neural network (RNN).

The RNN algorithm was capable of recalling both the short- and long-term patterns of the time series for forecasting. The range of the energy time series considered in this research was quite large, containing multiple seasonal dynamics. Traditional physics-based numerical modeling tools require assumptions, other correlated variables, and the expensive calibration of the parameters. Compared to the other neural network regression models, RNNs have been proven to show good performance, especially in time-series prediction. As energy consumption and production provided sequential data with considerable temporal dynamics, an RNN was used to quantify future values based on past data. As the shape of the energy dataset was comparatively large, representing eleven years of daily data, the RNN algorithms showed a highly satisfactory performance.

4. Conclusions

This study contributes to the development of a reproducible template for analyzing large amounts of exploratory data in order to understand the distinctive temporal dynamics of energy consumption and renewable energy production. Various modern data exploration technologies were used to uncover a hidden pattern in the distribution of energy values based on more than eleven years of data, which was a necessary condition for the successful training of the RNN algorithm. Following a successful training phase, an explicit iterative performance record was used to tweak and optimize the RNN. This performance record could then be used to anticipate the energy values in a similar geographic area. The effectiveness of the RNN algorithm in predicting energy showed how well-suited the algorithm is to energy time series. Many error metrics indicated positive performance with small errors. In this study, an RNN algorithm was used to forecast energy consumption based on the dynamics of renewable energy production. The proposed model could be used as a tool for planning and designing energy distribution systems in communities to improve operational and maintenance decisions. The proposed model framework was found to be highly efficient in predicting time-series data for energy consumption, presenting minimum errors when compared with the observed data.

The RNN algorithm was trained using only the observed data without any intermediate transformation of the variables using physics-based equations. Therefore, the proposed framework could be a powerful tool for predicting energy consumption in a real-time manner without the burden of further computation with physic-based models. This is a powerful analytical advancement in practice.

Along with the benefits, there are a few disadvantages of adopting this RNN, including the following: (1) a large amount of time is necessary to train the model, with more time required for RNN analysis and for running the model over a big dataset compared to other conceptual models. The computing effort/time needed for the RNN algorithm in this investigation was discovered to be extremely high. (2) The slow process also made it necessary to use more of the system’s memory and storage capacity, which could make it difficult to train on a large dataset, as was the case in our study. (3) A challenging aspect of RNNs for time-series data processing is overfitting, which can lead to incorrect extremely low error measurements. (4) Despite the fact that the essential factor for implementing RNNs is the capacity to compare time lags throughout the whole time-series dataset, we were unable to obtain sufficient performance for this dataset for periods longer than three days. Any time lapses that occurred during a three-day period were flawless and produced low error measurements that were satisfactory. To achieve the best results, future research projects should be carried out incorporating high-performance computing and cloud-based processes. To test the model’s transferability, RNN models with various configurations should be used in various geographic and climatic regions.