BiLSTM Network-Based Approach for Solar Irradiance Forecasting in Continental Climate Zones

Abstract

Recent research on solar irradiance forecasting has attracted considerable attention, as governments worldwide are displaying a keenness to harness green energy. The goal of this study is to build forecasting methods using deep learning (DL) approach to estimate daily solar irradiance in three sites in Kuwait over 12 years (2008–2020). Solar irradiance data are used to extract and understand the symmetrical hidden data pattern and correlations, which are then used to predict future solar irradiance. A DL model based on the attention mechanism applied to bidirectional long short-term memory (BiLSTM) is developed for accurate solar irradiation forecasting. The proposed model is designed for two different conditions (sunny and cloudy days) to ensure greater accuracy in different weather scenarios. Simulation results are presented which depict that the attention based BiLSTM model outperforms the other deep learning networks in the prediction analysis of solar irradiance. The attention based BiLSTM model was able to predict variations in solar irradiance over short intervals in continental climate zones (Kuwait) more efficiently with an RMSE of 4.24 and 20.95 for sunny and cloudy days, respectively.

1. Introduction

The global increase in sustainable electricity demands to save the environment has improved the penetration of renewable energy sources into electrical grids. Apart from being plentiful and sustainable energy sources, solar energy also has low-to-nil environmental damage, making it suitable for extensive electrical production [1]. Photovoltaic (PV) modules are used to harness solar energy, though being environmentally beneficial alone does not make PV systems a viable alternative to conventional energy sources. PV output power is not dispatchable in terms of supply and demand. The absorbed solar irradiance is the key meteorological element impacting the electricity generated by PV plants. There is a linear relationship between the maximum power of PV modules and the sun’s irradiance [2]. The degree to which PV modules accumulate solar irradiance varies depending on the time, as well as the panel’s alignment to the sun [3]. Energy storage technologies such as batteries and ultracapacitors are essential in managing the energy and transient power demands by the electrical grid from PV plants [4]. Solar irradiance forecast is critical to accurately size a solar PV power plant and energy storage. This study aims to predict irradiance in an optimal and generalized manner, using deep learning. Solar irradiance prediction is carried out using past data from Kuwait. The primary goal is to increase the contribution of renewable or green energy to the total quantity of energy generated.

1.1. World PV Growth

Electricity tariffs vary widely worldwide; installing solar power generation systems in certain countries is much more economical for small consumers if the electricity tariff electricity is higher, compared to the rate of solar power per kWh. In several countries, the government provides incentives to encourage renewable energy systems, making them reasonably profitable through attractive schemes. Solar cell technology is currently being expanded by various commercial solar cells, including crystalline silicon cells, thin-film, amorphous silicon cells, and multi-joint cells. By the end of 2040, almost 60% of all electricity generated is projected to come from renewable sources, primarily wind and solar photovoltaics [5,6]. A total of 629 GW of solar power had been installed globally by the end of 2019 [5]. China was leading in solar power production, with a total installed capacity of 208 GW by the beginning of 2020, accounting for almost one-third of the world’s solar energy [7,8]. By 2020, it is expected that at least 37 countries will have a PV capacity of more than one gigawatt. From 2016 to 2019, China, the USA, and India were the leading installers of PV power production [9,10].

1.2. Related Work

Many artificial intelligence (AI) strategies have been developed to predict solar irradiance, consisting of three fundamental “forecasting techniques: numerical prediction, image-based prediction, and statistical and machine learning (ML) methods. Solar irradiance data are time-series data, i.e., data that sequentially range over time” [11] (p. 2). Linear forecasting methods were frequently employed in the past because they were well known, simple to compute, and generated a consistent forecast for solar irradiance. Traditional forecast models include autoregressive moving average (ARMA) [11], autoregressive with exogenous inputs (ARX) [12], autoregressive integrated moving average (ARIMA) [13], autoregressive moving average with exogenous inputs (ARMAX) [14], autoregressive combined moving average with exogenous inputs (ARIMAX) [15], seasonal autoregressive integrated moving average (SARIMA) [16], generalized autoregressive score (GAS) [17,18], autoregressive integrated moving average (ARIMAX), and seasonal autoregressive integrated moving average with exogenous inputs (SARIMAX) [19]. To estimate the global solar radiation parameters, Belmahdi et al. presented the ARIMA and ARMA models [20]. In these models, only the solar radiation parameter was considered. There were no geographical or meteorological parameters used for model training; the models presume linearity in the data, making them incapable of capturing complicated nonlinear patterns. Ferlito et al. conducted a comparative study of eleven online and offline data-driven models concerning grid-connected photovoltaic efficiency forecasting [21]. An automated encoder was used by Gensler et al. to reduce historical data dimensions and LSTM was used to predict solar irradiance [22]. Zhen et al. used multi-level wavelet decomposition to pre-process solar irradiance data to further enhance the prediction accuracy [23]. A new day-to-day model for predicting solar irradiance was created in another Zhen article based on a time-section fusion pattern and mutual iterative optimization [24].

Yagli et al. tested 68 machine learning models, utilizing satellite-derived irradiance data from several sites [25]. Multilayer perceptron (MLP) models have proved to be among the best performers in the study. The artificial neural network (ANN) models utilized in this study were optimized for day-ahead forecasting. ANNs use nonlinear transforming layers to process data and are also good at detecting complicated structures in data; they can reconstruct a noisy system driven by data, which makes them qualify for variable time-series and complex forecasting. All of these are ideal to design challenges that need to capture the dependencies and preserve information, as they advance through the data’s successive time steps. In [26], the authors proposed employing deep recurrent neural networks to estimate solar irradiance, reducing model complexity and facilitating feature extraction. The proposed method outperformed traditional feedforward ANN and SVM. The “recurrent neural network (RNN) design recognizes sequential characteristics of data node dependencies by maintaining sequential information in an inner state, allowing data accumulated over time to be preserved” [27] (p. 3). The RNN, on the other hand, is prone to exploding and vanishing gradients. Bidirectional LSTM networks [28], long short-term memory (LSTM) networks [29], and “gated recurrent unit (GRU) have been created as RNN extensions, substituting the traditional perceptron design with memory cells and gating algorithms that govern information flow throughout the network” [30] (p. 3). LSTM is an effective approach for predicting time-series and has been developed in [30] for day-ahead solar irradiance prediction. The LSTM model was more robust than the other forecasting methods used in the study. Using weather data, the authors of [31] suggested a mechanism for hourly day-ahead sun irradiance prediction. RNN may be classified in attention-based and classic memory-based models. GRU, LSTM, bidirectional RNNs, and other memory-based models exist, while self-attention generative adversarial networks, attention LSTM, and multi-headed LSTM are examples of attention-based models.

A set of mathematical equations that describe the physical condition and dynamic motion of the atmosphere is referred to as a physical technique [32]. They are typically used for applications with very short to very long-time horizons. These systems rely heavily on numerical weather prediction (NPW), sky imagery, and satellite imaging [33]. They are classified as global or mesoscale physical approaches based on the size of the simulated atmosphere, which can be global or confined [34]. Only mesoscale models should be used to forecast the electricity generated by PV plants; the main disadvantage of such models is that their resolution is only 16–50 km [35].

Comparing forecast techniques is difficult in general because the factors influencing performance are numerous and vary depending on the situation. They include historical data and weather forecast availability, temporal horizon and resolution, weather conditions, geographical location, and installation conditions, to name a few. Proper data preprocessing (for example, deleting the night sample when no power is produced) is also required in the case of statistical approaches to ensure acceptable performance and lower computing costs [36]. The literature reviews offer some insight into the efficacy of various strategies, though their findings are more qualitative than quantitative. Recent reviews [33,36] provide a comparative analysis based on the work of multiple authors, as well as statistical flaws. The comparison is not valid from a quantitative standpoint because the settings and measures employed in each experiment differ.

In the literature, memory-based RNNs are by far the most extensively employed model for solar irradiance forecasting; however, the research lacks attention-based RNN models. In this work, an attention based BiLSTM mechanism for forecasting solar irradiation is proposed. The training of the models was performed using actual meteorological data from three regions of Kuwait—Al-Wafer, Kia, and Abdaly. The accuracy of the proposed attention based BiLSTM is compared to and evaluated against that of other existing models, using credible statistical indicators, such as root mean square error (RMSE), mean squared error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE).

2. Attention-Based BILSTM

The networks use historical sun irradiance data from the target locations as input characteristics. Figure 1 depicts the design of a bidirectional LSTM network with an attention mechanism. The input is represented as vector X^T, vector Y^T is the corresponding solar irradiation, and vector Y^T+θ represents the predicted solar irradiations in prediction analysis.

$$X^T=\left(X_1, X_2, X_3\dots ..X_T\right),$$

(1)

$$Y^T=\left(Y_1, Y_2, Y_3\dots ..Y_T\right),$$

(2)

where

$T=\mathrm{total} \mathrm{length} \mathrm{of} \mathrm{the} \mathrm{time} \mathrm{steps}$

$\theta =\mathrm{future} \mathrm{time} \mathrm{steps}$

As there is no expressive information before the time window ($w$), and the input is fixed, $\left\{X\left(t-w\right),X\left(t-w+1\right),\dots ,X\left(t-1\right)\right\}$ are utilized to calculate $Y(t+\theta )$ for each task, where $\Delta$ is the time frame ahead of prediction. The problem is denoted by Equation (3), with Y indicating predictions of solar irradiance data using only a deep neural network model f on previously observed real-world data.

$$Y(t+\Delta )=f\left(X\left(t-w\right),X\left(t-w+1\right),\dots ,X\left(t-1\right)\right),$$

(3)

Historical data at time $\left(t-w\right)$ are represented by $I_{rr}\left(t-w\right)$ in Figure 2, and the input variable is represented in Equation (4), for the solar irradiance forecast modeling.

$$\left\{I_{rr}\left(t-w\right),I_{rr}\left(t-w+1\right),\dots ,I_{rr}\left(t-1\right)\right\},$$

(4)

The partial autocorrelation and autocorrelation features of the data were used to establish the size of the window for the lag time-series. The ith hidden layer L, in which the i values are set during model tuning, is represented by L_i. Overall, future sun irradiance values were forecast based on previous and present values for the given window size.

The architecture used for prediction in this study is the attention based BiLSTM neural network, made up of three layers: an encoder, an attention layer, and a softmax layer.

2.1. Encoder Layer

The BiLSTM serves as the encoder layer. The attention layer uses the hidden outputs from this layer before constructing a context vector, to create the scoring function first. The predicted values of solar irradiance are subsequently transmitted towards the dense layer or decoding the fully connected layer.

LSTM: Recurrent neural networks have been employed to model sequential data in many engineering problems. However, due to difficulties with gradient vanishing or exploding, RNNs are unable to learn long-term dependencies. To remedy these flaws, LSTM networks are suggested and built based on RNNs. Cell memory states and three gates comprise an LSTM’s fundamental structure. The following composite functions implement a single LSTM cell:

$$f_t=\sigma (W_f[h_{t-1},x_t]+b_f),$$

(5)

$$i_t=\sigma (W_i[h_{t-1},x_t]+b_i),$$

(6)

$$o_t=\sigma (W_o[h_{t-1},x_t]+b_o),$$

(7)

$$C_t=f_t*C_{t-1}+i_t*\tanh \left(W_c[h_{t-1},x_t]+b_c\right),$$

(8)

$$h_t=o_t*\tanh (C_t),$$

(9)

Weighted matrices (W_i, W_f, W_o) and the LSTM cell biases (b_i, b_f, b_o) are all parameters of the input gate, forget gate, and output gate, correspondingly. The operator ∗ is an element-wise multiplication and the sigmoid function. The word embedding of the LSTM cell’s input is represented by x_t, and the hidden state vector by h_t.

BiLSTM: The inputs are processed in strict chronological order by the LSTM, leading to an influence of the prior inputs only, and not the future ones. To make the model also be influenced by future values, the bidirectional LSTM model was developed [37]. The LSTM processing chain is duplicated, allowing the inputs to handle both reverse and forward time sequences, allowing the network to consider the network’s future context. The final output, h_t, of the BiLSTM model at the step t is shown as:

$$h_t=\left[f{h}_t+b{h}_t\right],$$

(10)

2.2. Layer of Attention

The availability of solar irradiation depends on many weather parameters. The attention mechanism is used to consider sensitive design variables. In practice, the LSTM or BiLSTM network will output a hidden h_t state at each time step, depending on the above. The h_t vector is designed into a one-layer MLP, which then learns hidden representation u_t. Then, given u_t and a solar irradiation parameter context vector uw, a scalar significance value for h_t is computed. Finally, the attention-based model uses a softmax function to calculate the weighted mean of the state h_t. The mechanism discussed is modeled as follows:

$$u_t=\tanh \left(W_w{h}_t+b_w\right),$$

(11)

$$a_t=\frac{{\mathrm{e}}^{\left(u_t^T{u}_w\right)}}{{\displaystyle \sum _t{\mathrm{e}}^{\left(u_t^T{u}_w\right)}}},$$

(12)

$$c={\displaystyle \sum _t{a}_t{h}_t},$$

(13)

2.3. Softmax Layer

A fully connected softmax layer is employed as a classifier in this paper. Vector c can be used as the feature for irradiation prediction:

$$\overrightarrow{y_i}=softmax\left(W_{c}C+b_c\right),$$

(14)

$\overrightarrow{y_i}$ is the model’s predicted value, $W_{c}C$ represents the weighted matrix, and $b_c$ is bias.

2.4. Training of Model

The loss function is made up of the cross-entropy error of irritation classification:

$$L=-{\displaystyle \sum y_i\log \overrightarrow{y_i}},$$

(15)

where $y_i$ is the observed irritation and $\overrightarrow{y_i}$ is the model’s predicted irradiation. The back-propagation approach [32] is used to derive the derivative of the loss function for the entire set of parameters, and stochastic gradient descent is used to update all the model’s parameters.

2.5. Metrics for Performance Evaluation

To measure the quality of fit of forecasting models, the mean square error (MSE), the coefficient of determination (R²), the root mean square error (RMSE), the normalized and root mean square error (NRMSE), and the standard metrics were calculated. The metrics used for the performance evaluation are statistically represented in (16). E stands for the actual value observed, and F for the prediction model’s output, given weight w and input X. MSE, MAPE, NRMSE, and RMSE give information about the error. Low MSE, NRMSE, and RMSE values indicate better performance. R² indicates how well the model fits the baseline. The link between the response variable and the predictors are considered strong when the R² score value approaches 1, whereas an R² score near 0 indicates the reverse.

$$\begin{array}{c}NRMSE=\frac{\sqrt{\frac{1}{N}{\displaystyle \sum _{n=1}^N{E}_r\left(D_n\left|X_n\right.\right)-F{\left(X_n,W\right)}^2}}}{{\left(E_r\left(D_n\left|X_n\right.\right)-F\left(X_n,W\right)\right)}^2}\\ MSE=\frac{1}{N}{\displaystyle \sum _{n=1}^N{E}_r\left(D_n\left|X_n\right.\right)-F{\left(X_n,W\right)}^2}\\ RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum _{n=1}^N{E}_r\left(D_n\left|X_n\right.\right)-F{\left(X_n,W\right)}^2}}\\ R^2=1-\frac{{\displaystyle \sum _{n=1}^N{E}_r\left(D_n\left|X_n\right.\right)-F\left(X_n,W\right)}}{{\displaystyle \sum _{n=1}^N{E}_r\left(D_n\left|X_n\right.\right)-F{\left(X_n,W\right)}^2}}\end{array}\}$$

(16)

3. Results and Discussion

3.1. Data Analysis

The climate of the PV station and the distribution and generation of solar irradiance are determined by its position, which varies substantially depending on the latitude. A cross-regional study is required to investigate the scalability of the BiLSTM models. Solar irradiance forecast models were constructed using historical time-series data by accessing Meteoblue. For Kuwait, Meteoblue historical weather simulation data in hourly resolution, aggregated in daily values, were acquired for the period 2008–2020. Meteoblue provides local weather data derived from worldwide statistical experimental datasets, using the non-hydrostatic meso-scale modeling (NMN) technology and the NOAA environmental modeling system (NEMS) framework. The data were collected for the location of Al-Abdali, a farm in northern Kuwait (latitude: 30°1″ E, longitude: 47°71″ E, altitude: 23 m). There was about 10 h of sunlight in winter and 14 h in summer. The radiation data were measured every five minutes and averaged for 1 h over the entire study period. The data were collected over 4500 days, from 2008. Figure 2 displays the regular hourly irradiance difference all day long on 1 October 2020. As shown in Figure 2, solar irradiation was observable at 6 a.m. The estimated irradiation increased by approximately 200 W/m² per hour, reaching the peak irradiation at noon. The irradiation decreased by about 210 kW/m² per hour after 12 noon. In total, around 6 h of irradiation exceeded 600 W/m². Figure 3 shows a sample of all the data used in this study.

Figure 4 shows irregular fluctuations in the total solar irradiation (24 h) from January 2019 to December 2019. In general, the volume of irradiation during the year varies from 2487 to 29,374 kWh/m². Significant variations in irradiation are seen from February to April. Figure 5 indicates the normal year-round variation in solar irradiation in 2019, without monthly segregation, to better explain the annual trend. In general, the data are widely distributed, especially during the spring season. The dispersion is minimal in autumn.

The irradiation difference between summer and winter is nearly threefold, which could influence the performance of the solar photovoltaic panels. Irradiation ranges between 6 and 9 h in January, April, July, and October 2019. Overall, hourly irradiation appears to be continuous, but can often change abruptly due to obstructions such as sandstorms and clouds. The findings are shown in Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6 and depict a significant fluctuation of solar radiation and a fluctuation in electricity from solar collectors. This would necessitate the procurement of additional power from other sources to resolve deficiencies, leading to higher operating costs.

The comparison between January, April, July, and October is the typical hourly difference in irradiation. The highest irradiation is seen in July (summer) and the lowest in January (winter). The amount of irradiation during April and October is identical. The amount of solar energy harvested during winter is smaller than in summer due to significant variations. As a result, a backup power system will be required in winter to compensate for solar production deficiencies.

The data are split into training and testing sets in the ratio of 75% and 25%. The data are standardized to [0, 1] to avoid neuronal saturation during the study process. The number of neurons in the first and second hidden layers for the attention-based BiLSTM is set to 64, and the activation function is chosen as ReLu. The dropout layer is used to encounter the overfitting problem in the network. To optimize the system, the Adam optimizer is employed in this study and in the MSE as a loss function.

3.2. Results

Four different statistical error indicators, viz., MSE, R-value, RMSE, NRMSE, and MAPE, are chosen to measure the accuracy of the developed model as mentioned in

Table 1. Statistical indicators of error for forecast models developed.

Model Error Indicators	Attention-Based BiLSTM		BiLSTM		LSTM
	Sunny	Cloudy	Sunny	Cloudy	Sunny	Cloudy
MSE (W/m2)	18.01	438.9861	496.4167	4797.4	1563.3	9566.6
R-value	0.9998	0.9957	0.9958	0.9532	0.9867	0.9068
RMSE (W/m2)	4.2443	20.9520	22.2804	69.2635	39.5385	97.8091
NRMSE (W/m2)	0.0058	0.0249	0.0306	0.0823	0.0543	0.1162
MAPE (%)	2.4869	20.2509	12.0526	64.60	19.5141	69.1077

. Figure 7 shows that the model’s predicted values are consistent with the observed values. The results show that all the models perform well in forecasting on sunny days, while the attention-based BILSTM model outperforms (NME of 18.01, R-value of 0.9998, RMSE of 4.2443, NRMSE of 0.0058, and MAPE of 2.48%). The statistical errors are presented in Section 2.5. All models’ prediction performance suffers significantly on cloudy days. The attention based BILSTM model is still the most accurate amongst all the models (MSE of 438.9861, R-value of 0.9957, RMSE of 20.9520, NRMSE of 0.0249, and MAPE of 20.2509%).

As shown in Figure 8, the difference between the predicted values and the measured values is significant on cloudy days and, in this case, the attention based BiLSTM model outperforms the LSTM and BiLSTM networks.

The results show that, for binary sentiment categorization, the LSTM and BiLSTM networks are shown to be effective. When bidirectional semantic information is considered, the BiLSTM achieves an improvement over the LSTM.

According to the results of the LSTM and BiLSTM models, the bidirectional LSTM model may be able to obtain more semantic information, which is beneficial for sentiment classification.

As the basic LSTM model cannot attend to any informative sections of a sentence, it is difficult for the LSTM model to enhance sentiment classification accuracy.

Compared to the LSTM, the AB-LSTM model shows that the attention mechanism can improve the LSTM model’s accuracy for sentiment classification by around 2% to 3%. The attention-based BiLSTM model obtains equivalent results on both corpora, compared to many of the external baseline approaches by including an attention component. Compared to the above baseline approaches, the experiment results show that the suggested model is more effective for sentiment classification.

4. Conclusions

Solar irradiance forecast has drawn the focus of contemporary research due to the influx for and awareness in green and renewable energy. To comprehend the solar energy perspective of a place, accurate forecasts of solar irradiance are essential, considering both the potential and the constraints associated with forecasting. To properly estimate solar irradiance, this study used a historical data collection of solar irradiance from the previous 12 years, concerning both testing and training. Due to its distinctive hidden layer cell structure design, attention-based BiLSTM, as the deep structure of RNN, provides a solution for vanishing gradient and exploding gradient, allowing RNN models with LSTM units to simulate both short- and long-term temporal relationships in time-series data. The simulation results validate the fact that the attention mechanism in BILSTM was able to effectively capture the variations in solar radiation under changing weather conditions. Authors have compared the proposed attention based BiLSTM with the existing LSTM and BiLSTM models. Comparing the actual data to the forecast data, it is clear that the attention-based BiLSTM model is both more effective and reliable, compared to the LSTM and BiLSTM models.