Enhancing Solar Energy Forecast Using Multi-Column Convolutional Neural Network and Multipoint Time Series Approach

Abstract

The rapid expansion of solar industries presents unknown technological challenges. A dedicated and suitable energy forecast is an effective solution for the daily dispatching and production of the electricity grid. The traditional forecast technique uses weather and plant parameters as the model information. Nevertheless, these are insufficient to consider problematic weather variability and the various plant characteristics in the actual field. Considering the above facts and inspired by the excellent implementation of the multi-column convolutional neural network (MCNN) in image processing, we developed a novel approach for forecasting solar energy by transforming multipoint time series (MT) into images for the MCNN to examine. We first processed the data to convert the time series solar energy into image matrices. We observed that the MCNN showed a preeminent response under a ground-based high-resolution spatial–temporal image matrix with a 0.2826% and 0.5826% RMSE for 15 min-ahead forecast under clear (CR) and cloudy (CD) conditions, respectively. Our process was performed on the MATLAB deep learning platform and tested on CR and CD solar energy conditions. The excellent execution of the suggested technique was compared with state-of-the-art deep neural network solar forecasting techniques.

1. Introduction

Over several years, the main objective of global sustainable development and mitigation policies for climate change has been to focus on renewable against coal energy production. The energy crisis and environmental issues are being raised all over the world. Photovoltaic (PV) production has grown fast due to wide applications in clean energy [1]. According to the International Renewable Energy Agency (IRENA) statistics, the worldwide deployed potential exceeded 580 GW in 2019 [2]. The output of solar PV generation and solar thermal power stations or photovoltaic/thermal (PV/T) reservoir systems [3] depends on various factors such as solar irradiance, temperature, climate, and other factors [4]. The prediction of solar irradiance plays a significant role in the security of the power grid [5]. Several models have been developed by experts for the prediction of global horizontal irradiance (GHI), which are classified under six categories as follows: persistence models, classical statistical models, machine learning methods, cloud motion tracking methods, numerical weather prediction models (NWP), and hybrid models [6]. In persistence models, the solar irradiance measurement at time T is equal to that at time T-1. During the last few years, persistence models have been used to compare the improvements in the performance of other models. The most popular method for predicting solar irradiance is climatology-persistence (Clim-Pers) [7]. The classical statistical models establish the relation between the parameters obtained through the dataset’s statistical analysis. Such models are as follows: Markov processes, regression, exponential smoothing models, autoregressive (AR) models, autoregressive moving average (ARMA) models, and autoregressive integrated moving average (ARIMA) models [8]. One of the demerits of statistical models is that they cannot tackle instantly varying non-static data. The prediction accuracy for the sudden change in the non-static data of solar irradiance is low. A machine-learning-based model can learn and design the complex non-linear relationship using the solar irradiance data and meteorological parameters. Artificial neural networks (ANNs), support vector machine (SVM), extreme learning machine (ELM), the genetic algorithm (GA), the hidden Markov model (HMM), naive Bayes, the Gaussian process (GP), the k-means clustering algorithm, and decision trees (DTs) were formulated for irradiance prediction [9,10,11]. Caldas and Alonso-Suarez, Peng et al., and Chu et al. reported that various algorithms are utilized for cloud motion tracking to predict the irradiances by the all-sky imagers or satellites. ANN-based GHI prediction is assessed using satellite images, where the cloud fraction is used as the input, obtained from satellite image processing [12]. In order to simulate the physical system in atmospheric transformation and forecast weather, the current weather mathematical model is utilized in the NWP method [13,14,15]. In [16,17,18], the authors utilized the NWP model and related technology to predict the solar irradiance. The hybrid models are widely used as they combine other methods’ advantages. Currently, many hybrid models have been developed to predict solar irradiance. In [19], Ji and Chee developed a hybrid model consisting of the ARMA model and the controversial time-delay neural network (TDNN) for hourly solar energy forecasting. ARMA and the TDNN show good accuracy, where ARMA is designed for the prediction of linear components of time series and the TDNN is also used to predict the solar irradiance. In [20], the HMM and generalized fuzzy model (GFM) were utilized for solar forecasting. Unsupervised clustering is used in the HMM method, and GFM has been applied to fine-tune the clustering for a desirable accuracy of solar radiation prediction. In [21], Voyant et al. developed a model that combines ANN and ARMA for an hourly forecast of global horizontal irradiance. The ANN is utilized for cloudy conditions, and ARMA is utilized on sunny days. In [22], Dong et al. formulated a new hybrid framework that combines the exponential smoothing state space (ESSS) and the ANN for predicting GHI from satellite images. The ESSS was used to predict the cloud cover index, and solar irradiance estimation depending on the cloud cover index was performed utilizing the ANN. In [23], a hybrid framework was developed, which was the combination of self-organizing maps (SOMs), SVR, and the particle swarm optimization (PSO) technique. Several disorganized regions were created corresponding to all the inputs utilizing the SOMs, then SVR and PSO were utilized to frame each disjointed region to predict the output. A hybrid model was developed, which combined fuzzy regression functions (FRFs) and SVM in order to predict horizontal global solar radiation [24].

Some other hybrid models have been developed for the prediction of solar radiation, which are based on different data preprocessing methods such as the wavelet transform (WT), wavelet decomposition (WD), wavelet packet decomposition (WPD), empirical mode decomposition (EMD), ensemble empirical mode decomposition (EEMD), complete ensemble empirical mode decomposition-adaptive noise (CEEM-DAN), principal component analysis, etc.

Mohammadi et al. [25] and Deo et al. [26] developed a hybrid model where SVMs and the WT algorithm were combined together for the prediction of GHI. Wavelet analysis was utilized to decompose time series into various sub-time series. These sub-time series were utilized as the input for the SVMs to complete the GHI prediction. In [27], Hussian and A.I. Alili proposed a hybrid model, which combines WD with the ANN for the decomposition of complex meteorological signals into simple sub-time series. Then, these sub-time series were modeled using the ANN, and the output was reconstructed to predict GHI. In [28], Lan et al. proposed a novel method consisting of the EEMD-SOM-BP method. EEMD was utilized to decompose the original data into sub-time series with different frequencies. These sub-time-series were used as the input to the SOM-BP networks. The output of each SOM-BP network is the predicted output of the solar radiance. This output is the algebraic sum of each SOM-BP network. In [29], Davo et al. introduced a hybrid model, which was the combination of principal component analysis (PCA) with the ANN and the analog ensemble (AnEn) for the prediction of solar radiance and wind power. The main objective of PCA was to reduce the dimension of the NWP data, whereas the ANN and AnEn were used to train and forecast solar radiances. In [30], the data were decomposed using WPD, and a new deep-learning-based hybrid model was developed in which WPD was combined with LSTM for PV power prediction. In [31], a CNN, LSTM, and CNN-LSTM comparison was given for 60 min-ahead PV power prediction for different seasons using the historical PV power. Rami Al-Hajj [32] leveraged the advantages of the multi-level stacking of LSTM and MLP for day-ahead solar radiation forecasting. The CNN, LSTM, and its derivatives were developed for solar PV power forecast using ground and satellite data. The innovations of this study are given as follows:

• We propose the Gramian cloud field (GCF) matrix (Section 2.1.1) to understand the clouds’ impact on solar power generation from ground-measured spatial and temporal solar power components.
• We evaluated the performance of the proposed models in different domains, namely time series, temporal, spatial, and spatial–temporal.
• For the first time, the CNN was used to forecast solar PV power using the multipoint and GCF approaches.
• We designed the MCNN model for use with multi-resolution input images from satellite and ground data. However, the target was the ground (in situ) data only.

The following Section 2.1 describes the data processing and the filling of the missing values involved in the model. Section 2.1.1 describes the time series-to-image matrix conversion approach. Section 2.2 gives the DNN framework of the two network structures for the present model. Section 2.3 describes the training of the DNN for the present model, which consists of the model setup in Section 2.3.1, the settings in Section 2.3.2, and the performance in Section 2.3.3. The results are discussed in Section 3, which include a comparison of the models in Section 3.1, the effect of the MCNN in Section 3.2, the effect of clear–cloudy conditions in Section 3.3, and the effect of the satellite inputs in Section 3.4 for the model validation. The conclusions of this research work are described in the last Section 4.

2. Methodology

2.1. Data Processing

Solar PV energy depends on multiple factors, primarily geographic location and atmospheric conditions. These factors were further classified into and analyzed as spatial and temporal distributions. The spatial analysis consisted of plant size, weather phenomena, panel distribution, panel orientation, facing direction, Sun tracing, etc. Furthermore, the temporal analysis depends on weather disturbance and the seasons, months, days, and hours of the years [33]. The careful investigation of these spatial and temporal data can enhance the accuracy of the prediction [34]. We included solar irradiance and power generation data to forecast 2 h solar energy production. The collected raw dataset contains solar irradiance and the power generation time series from 00:00H, 1 March 2020, to 24:00H, 31 March 2021, from Southern India. The solar power generation consists of spatial and temporal components at 5 min intervals and 22 random locations of the plants, as depicted in Figure 1. Figure 1 is the general layout of the studied solar farm to show the sub-distribution of the panels and data collection regions. Our plant capacity is 80 MW, consists of 2.5 million modules, and covers an area of almost 1.2 km². Our aim was to elaborate the plant for better accuracy. Of course, we can increase the number of observation points, as well as the plants, but we had limitations. The units for solar energy are presented in kWh and MWh. The Gramian cloud field (GCF) matrix [35] is proposed to convert solar temporal series to spatial image matrices, which comprised the input data framework of the DNN model.

2.1.1. GCF Matrix

The Gram matrix (GM) is commonly used to evaluate the linear dependence of a set of vectors. The Gram matrix is defined in Equation (1) for a given set of vectors [35]. The Gram matrix was formulated by the use of actual time series data to create Figure 2b, and the spatial data were used to create Figure 2c.

$$GM=\left[\begin{array}{ccc}<x_1,x_1>& \dots & <x_1,x_m>\\ \dots & \ddots & \dots \\ <x_n,x_1>& \dots & <x_n,x_m>\end{array}\right]$$

(1)

where $<>$ is the inner product of the two matrices.

Inspired by the Gram matrix, we suggest the Gramian cloud field (GCF) matrix [36]. Here, to understand the clouds’ impact on solar power generation from the ground-measured parameters given in time series $u_1,u_2,\dots u_n$ and spatial data $v_1,v_2,\dots v_n$, the solar components provide the movements of the clouds. The cloud movements can be divided into two components for the ease of calculation. The first determines the cloud direction $D{C}_i$ and, the second, an approximation of the cloud potential $C{P}_i$ from multiple coordinates at every 5 min by Equation (2):

$$D{C}_i=arctan\left(\frac{u_i}{v_i}\right);C{P}_i=·5\times \sqrt{u_i^2+v_i^2}$$

(2)

After rescaling cloud potential component CP to [0, 1] by min–max normalization, we built a vector, $C=c_1,c_2\dots c_n$, which contains cloud features $c_i$, in the polar coordinate system $\overbrace{C{P}_i}$, $\overbrace{D{C}_i}$. The elements of vector $c_i$ are cloud features that contain the normalized value of DC and CP. The min–max normalization was obtained using Equation (3) to improve the computational speed and accelerate the network’s training convergence speed [37].

$$\overbrace{C{P}_i}=\frac{CP-C{P}_{min}}{C{P}_{max}-C{P}_{min}}$$

(3)

where $\overbrace{C{P}_i}$ is the data after the min–max normalization of the original data (CP). $C{P}_{max}$ and $C{P}_{min}$ are the maximum and minimum value of the original data. The GCF matrix formulation is explained in Equation (4):

$$GCF=\left[\begin{array}{ccc}<w_1,w_1>& \dots & <w_1,w_m>\\ \dots & \ddots & \dots \\ <w_n,w_1>& \dots & <w_n,w_m>\end{array}\right]$$

(4)

where $<w_1,w_2>$ is inner product between $w_1$ and $w_2$:

$$\begin{array}{cc}<w_1,w_2>=\overbrace{C{P}_1}cos\overbrace{D{C}_1}\times \overbrace{C{P}_2}cos\overbrace{D{C}_2}\hfill & +\overbrace{C{P}_1}sin\overbrace{D{C}_1}\times \overbrace{C{P}_2}sin\overbrace{D{C}_2}\hfill \end{array}$$

(5)

Every timestamp (i) of the time series (TS) data is represented as vector $w_i$ in the polar coordinate system ($C{P}_i$, $D{C}_i$). Diagonal elements characterize the features of the cloud correlation vectors at each timestamp. In comparison, the other elements represent the cross-co-relationship between cloud vectors at different timestamps. In the raw dataset, there were twenty-two TSs from different locations, which were mapped to five-channel images, as shown in Figure 2b–f.

Regarding the GCF image matrix conversion, the matrix’s row and column are the 22 locations (every 5 min) of solar power in the space and time domains. Therefore, n and m in the GCF matrix represent the 22 locations. An example of the GCF image formation based on the available spatial–temporal solar data is shown in Figure 2a; the spatial distributions of the normalized solar power of the 22 locations are shown in the bottom x-axis (space label) versus solar power in the left y-axis, similar to the time domain distribution of the previous 22 values $(t_1,t_2,\dots t_{22})$ shown in the top x-axis (time label) versus solar power in the right y-axis. In Figure 2a, the spatial data (red line) illustrate the values of a certain point in time, and temporal data (black line) are for a certain location. The image in Figure 2b,c is derived from the GM in Equation (1). The GCF image matrix is derived from Equation (4), and the proposed combination of multiple space and time correlation matrices is shown in Figure 2d–f to understand the solar power distribution properly in the time and space domains. Figure 2b is an image of the temporal distribution of Figure 2a (black color) of the solar power of the 22 previous temporal values derived from the GM in Equation (1). The row (n) and column (m) of Figure 2b are the time vectors. Figure 2c is also derived from the GM in Equation (1), which is based on the spatial distribution of the solar power at the 22 locations, shown in Figure 2a in red color. The row and column of Figure 2c are the space vectors. The GCF images in Figure 2d,e were derived from Equation (4), keeping the time and space domain values, respectively. Figure 2f represents the spatiotemporal image from Equation (4) of the 22 points’ space and time distributions.

The satellite irradiance components diffuse horizontal irradiance (DHI), direct normal irradiance (DNI), global horizontal irradiance (GHI), and insolation (INS) were considered for the further analysis and verification of the solar power generation at the plant location, as shown in Figure 3.

The satellite data’s 4 km spatial and 15 min temporal resolution were different from the ground-measured data, requiring transforming the GCF images. The satellite images benefited the cloud movement tracking for the large-scale spatial analysis. Therefore, the spatial distribution of the image size was considered as $100\times 100$ km, which is 30-times larger than the actual plant size. The Sun’s movement was considered better with a larger satellite image, around the clock. The cloud shadow variation calculated over the plants was based on the clouds’ altitude (roughly 8 to 20 km) with respect to the horizontal and vertical movements in the sky. In the present study, we used the radiative transfer model (RTM) simulations produced by libRadtran [38,39] to estimate the solar irradiance. A fast version of RTM was developed by [40] based on pre-calculated look-up tables and a replicate for India by [41]. The major input parameters for the RTM simulations are the aerosol and cloud optical properties, the solar elevation and orientation, the total ozone column, and the columnar water vapor. The outputs of the RTM simulation are the DHI, DNI, GHI, and INS, which cover the wavelength range from 285 to 2700 nm and use the SBDART radiative transfer solver [42] with the pseudo-spherical approximation to generate the valid output for solar elevations between 0 and 90 degrees. The model simulation was performed using the band parameterization method based on the correlated-K approximation [43], along with aerosol and cloud determination, which were performed based on the default available aerosol models [44].

2.2. Deep Neural Network Structure

The TS-to-image matrix conversion using the GCF algorithm in Equation (4), the available twenty-two $1D$ images, five $2D$ single-channel sequential images’ spatiotemporal mapping, and one $3D$ four-channel satellite image formed the solar irradiance dataset used for the study. The comparison was performed using the popular convolutional neural network (CNN) [45], long short-term memory (LSTM) [46], a combination of both (CNN-LSTM) [47], and the MCNN [48] to retrieve the features from the above matrices and the output of a 2 h solar power forecast, as shown in Figure 4, showing the complete model network structure. From Figure 4, Seq, Cov, Batch Nor, Tran Cov, Max Pool, F.C., and Reg denote the sequence, convolution, batch normalization, transposed convolution, max pooling, fully connected, and regression layers, respectively. The sequence input layer features are defined as per the input data types $1\times 1$, $22\times 1$, $22\times 22\times 1$, $22\times 22\times 5$, and $26\times 26\times 4$ for single-input time series, multiple-input time series, 2D images with five channels, and 3D images with four channels for the input dimensions, respectively [49]. The filters and convolution kernel size started from 8@ $3\times 3$ for the first, 16@$3\times 3$ for the second, etc., as depicted in Figure 4. In the present study, in the CNN model, a five-times Cov layer was used, with a maximum filter size of 72@$3\times 3$. Each Cov layer consisted of a stride of one and zero padding.

We used the Tran Cov layer for upsampling [50]. The Tran Cov layer creates an output feature map with a greater spatial dimension than the input, making the regression more robust. To avoid the overfitting problem and speed up the training procedure, we used a dropout layer (the dropout rate was 0.25) [51], a batch normalization layer, max pooling, to help extract low-level features, and ReLU, to avoid the vanishing gradient problem, which has a much lower runtime [52]. We used the BiLSTM layer [53,54] following the convolutional layer. The BiLSTM layer can control the bidirectional dependencies passed on in the sequence data at every step. The fully connected layer starts from 484, 242, 121, 60, and 22 units to the final regression layer. In order to prevent potential interferences, we merged the two networks (CNN and LSTM) and connected the final forecast layer to another fully connected layer. We used two spatial image sources, one the ground and the other the satellite image, with different spatial resolutions, to overcome the different spatial resolution problems using multi-stream CNN, multi-input CNN, or multi-column CNN-LSTM (MCCN) [48], as shown in Figure 4b.

2.3. DNN Properties

Once having processed the data (Section 2.1 and Section 2.1.1) and created the model network structure (Section 2.2), the next step was the model properties’ definition. This is divided into the following sub-sections:

Section 2.3.1: DNN Setup;

Section 2.3.2: DNN Settings;

Section 2.3.3: DNN Performance Parameters.

2.3.1. DNN setup

The whole dataset consisted of 22 TS data points, and each TS contained 1,03,680 points for every 5 min of solar power generation from March 2020 to 2021. Each TS point was considered an independent case, and we divided the 1,03,680-sample dataset into training and testing sets with a 7:3 ratio. Finally, we trained and tested the whole dataset using the deep learning toolbox from MATLAB 2021b for data processing, network design, and training using a GPU.

2.3.2. DNN Settings

Three popular time series forecasting methods for all possible input datasets were tested to demonstrate and analyze the proposed hybrid method. The proposed well-known methods were CNN-LSTM [47], LSTM [46], and CNN [45], taken for the comparison and assessments. The input data types were kept the same: 1D, 2D, and 3D, to assess the three network structures uniformly. The basic model settings and parameters are listed in

Table 1. Model properties.

Settings	Parameters
Solar dataset number	22
Training set size	72,576
LSTM network layer number	6
LSTM neuron number in each layer	100
CNN network layer number	6
CNN neuron number in each layer	100
LSTM-CNN network layer number	12
LSTM-CNN neuron number in each layer	100
Hidden dense layer number	2
Hidden neuron number	200, 200
Dropout rate	0.2
Training method	Adam optimizer
Loss function	RMSE, MAE
Forecast horizon (min)	5–120 (2-h)

.

2.3.3. DNN Performance Parameters

The model performance was evaluated based on the root-mean-squared error (RMSE) and mean absolute error (MAE) metrics for the purposes of comparison, which are formulated as below:

RMSE method: This method evaluates the standard deviations in the predicted solar power, calculated by using the equation:

$$RMS{E}_{Y_T}=\sqrt{\frac{{\sum }_{i=1}^N{(y_{actual}-y_{predict})}^2}{N}}$$

(6)

where $RMS{E}_{Y_T}$ denotes the RMSE of solar power, $y_{actual}$ is the solar power from the field test, $y_{predict}$ is the predicted solar power from the machine learning algorithm, and N is the number of non-missing data points.

MAE method: This method calculates the mean absolute error involved in solar power measurement. The absolute error is the difference between the actual and predicted one. The formula to calculate the MAE for power is given below:

$$MA{E}_{Y_T}=\frac{{\sum }_{i=1}^N|y_{actual}-y_{predict}|}{N}$$

(7)

where $MA{E}_{Y_T}$ is the mean absolute error of the power and N is the number of errors.

The assessment consisted of four major domains, namely time series, temporal, spatial, and spatiotemporal, to understand the response of the predefined methods as mentioned in Table 2:

1

The validation of the proposed approach was performed by comparing with single and multiple inputs to match the single and multiple outputs on a typical time series dataset.

2

The validation of the proposed approach is illustrated to reveal the effectiveness of the 2D input image metrics in the spatial domain to match the single and multiple outputs on a typical time series dataset.

3

Similarly, the validation of the proposed approach is illustrated to reveal the effectiveness of the 2D input image metrics in the temporal domain to match the single and multiple outputs on a typical time series dataset.

4

Finally, the validation of the proposed approach is illustrated to reveal the effectiveness of the 2D input image metrics in the spatiotemporal domain to match the single and multiple outputs on a typical time series dataset.

Table 2. Experiment settings and parameters.

Name	DI	Unit	Type	Sources	Category	Layers	M-Type	Outputs
M1	1D	MW	Ts	GD	CR+CD	2	LSTM	1
M2	1D	kW	Ts	GD	CR+CD	2	LSTM	1
M3	1D	kW	Ts	GD	CR+CD	2	LSTM	22
M4	2D	kW	Sp	GD	CR+CD	2	CNN	1
M5	2D	kW	Sp	GD	CR+CD	2	CNN	22
M6	2D	kW	Tp	GD	CR+CD	2	CNN	1
M7	2D	kW	Tp	GD	CR+CD	2	CNN	22
M8	2D	kW	Sp-Tp	GD	CR+CD	2	CNN	1
M9	2D	kW	Sp-Tp	GD	CR+CD	2	CNN	22
M10	1D	kW	Sp-Tp V	GD	CR+CD	2	LSTM	1
M11	1D	kW	Sp-Tp V	GD	CR+CD	2	LSTM	22
M12	2D	kW	Sp	GD	CR+CD	4	CNN-LSTM	1
M13	2D	kW	Sp	GD	CR+CD	4	CNN-LSTM	22
M14	2D	MW	Tp	GD	CR+CD	4	CNN-LSTM	1
M15	2D	kW	Tp	GD	CR+CD	4	CNN-LSTM	22
M16	2D	kW	Sp-Tp	GD	CR+CD	4	CNN-LSTM	1
M17	2D	kW	Sp-Tp	GD	CR+CD	4	CNN-LSTM	22
M18	2D	kW	Sp-Tp	GD	CR	4	CNN-LSTM	22
M19	2D	kW	Sp-Tp	GD	CD	4	CNN-LSTM	22
M20	2D	kW	Sp-Tp	Sat	CR	4	CNN-LSTM	22
M21	2D	kW	Sp-Tp	Sat	CD	4	CNN-LSTM	22
M22	2D	kW	Sp-Tp	Sat+GD	CR	4	CNN-LSTM	22
M23	2D	kW	Sp-Tp	Sat+GD	CD	4	CNN-LSTM	22
M24	2D	kW	Sp-Tp	GD	CR	4	MCNN	22
M25	2D	kW	Sp-Tp	GD	CD	4	MCNN	22
M26	2D	kW	Sp-Tp	Sat	CR	4	MCNN	22
M27	2D	kW	Sp-Tp	Sat	CD	4	MCNN	22
M28	2D	kW	Sp-Tp	Sat+GD	CR	4	MCNN	22
M29	2D	kW	Sp-Tp	Sat+GD	CD	4	MCNN	22

Table 2. Experiment settings and parameters.

Name	DI	Unit	Type	Sources	Category	Layers	M-Type	Outputs
M1	1D	MW	Ts	GD	CR+CD	2	LSTM	1
M2	1D	kW	Ts	GD	CR+CD	2	LSTM	1
M3	1D	kW	Ts	GD	CR+CD	2	LSTM	22
M4	2D	kW	Sp	GD	CR+CD	2	CNN	1
M5	2D	kW	Sp	GD	CR+CD	2	CNN	22
M6	2D	kW	Tp	GD	CR+CD	2	CNN	1
M7	2D	kW	Tp	GD	CR+CD	2	CNN	22
M8	2D	kW	Sp-Tp	GD	CR+CD	2	CNN	1
M9	2D	kW	Sp-Tp	GD	CR+CD	2	CNN	22
M10	1D	kW	Sp-Tp V	GD	CR+CD	2	LSTM	1
M11	1D	kW	Sp-Tp V	GD	CR+CD	2	LSTM	22
M12	2D	kW	Sp	GD	CR+CD	4	CNN-LSTM	1
M13	2D	kW	Sp	GD	CR+CD	4	CNN-LSTM	22
M14	2D	MW	Tp	GD	CR+CD	4	CNN-LSTM	1
M15	2D	kW	Tp	GD	CR+CD	4	CNN-LSTM	22
M16	2D	kW	Sp-Tp	GD	CR+CD	4	CNN-LSTM	1
M17	2D	kW	Sp-Tp	GD	CR+CD	4	CNN-LSTM	22
M18	2D	kW	Sp-Tp	GD	CR	4	CNN-LSTM	22
M19	2D	kW	Sp-Tp	GD	CD	4	CNN-LSTM	22
M20	2D	kW	Sp-Tp	Sat	CR	4	CNN-LSTM	22
M21	2D	kW	Sp-Tp	Sat	CD	4	CNN-LSTM	22
M22	2D	kW	Sp-Tp	Sat+GD	CR	4	CNN-LSTM	22
M23	2D	kW	Sp-Tp	Sat+GD	CD	4	CNN-LSTM	22
M24	2D	kW	Sp-Tp	GD	CR	4	MCNN	22
M25	2D	kW	Sp-Tp	GD	CD	4	MCNN	22
M26	2D	kW	Sp-Tp	Sat	CR	4	MCNN	22
M27	2D	kW	Sp-Tp	Sat	CD	4	MCNN	22
M28	2D	kW	Sp-Tp	Sat+GD	CR	4	MCNN	22
M29	2D	kW	Sp-Tp	Sat+GD	CD	4	MCNN	22

3. Experimental Results and Evaluation

Once having processed the data (Section 2.1 and Section 2.1.1), created the model network structure (Section 2.2), and defined the model properties (Section 2.3), the next step was the evaluation process. This is divided into the following sub-sections:

Section 3.1: Comparison of Methods;

Section 3.2: Effects of Multi-Column CNN-LSTM Model;

Section 3.3: Effects of Cloudy Conditions;

Section 3.4: Effects of Satellite Inputs.

3.1. Comparisons of Methods

To evaluate the performance of the proposed method, we compared the performance of different methods in the following time steps in minutes: 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 100, 115, 120. The results of the twenty-two locations were combined and presented in Figure 5 and Figure 6, and the detailed MAE and RMSE results are given in

Table 3. Metrics of methods with time steps.

Model	MAE			RMSE
	15	30	90	15	30	90
M1	27.2203	27.5405	24.7239	9.7590	10.3284	8.4229
M2	9.5770	15.3224	25.2258	1.5453	3.1543	9.0886
M3	17.3940	12.3964	20.9350	4.7824	2.2567	6.2924
M4	26.2417	26.2595	26.3432	9.0746	9.0960	9.1694
M5	25.8696	25.8866	25.9707	8.8227	8.8378	8.9141
M6	26.2419	26.2598	26.3453	9.0777	9.0982	9.1745
M7	8.4704	10.7054	19.9611	1.4691	2.1297	5.4158
M8	25.9674	24.8605	26.3303	8.8689	8.0851	9.1643
M9	13.7934	12.0208	20.1083	2.8352	2.2700	5.4955
M10	20.6757	14.4866	20.8888	6.4844	3.1501	5.9975
M11	13.9041	15.9961	22.4180	3.1708	3.8144	6.8712
M12	8.9575	12.1475	20.9664	1.5873	2.4501	5.9187
M13	9.0794	11.9933	20.9083	1.5672	2.3315	5.9165
M14	9.2380	10.4488	15.7862	1.6760	2.0737	4.1134
M15	8.9872	10.6592	15.3156	1.5511	1.9917	3.6687
M16	8.9911	10.9749	15.3569	1.6573	2.2885	3.9627
M17	8.7971	10.6773	14.8766	1.5922	2.1281	3.5232
M18	3.0580	2.1712	5.0391	0.1135	0.1093	0.7463
M19	4.4561	4.4877	9.9952	0.3324	0.3796	1.2174
M20	4.9387	5.1428	10.9953	0.6260	0.8058	2.6585
M21	9.6306	9.9354	13.2887	1.7636	1.8470	3.3160
M22	5.2187	5.4727	11.4658	0.6640	0.8664	2.7936
M23	9.9740	10.6457	15.4560	1.9031	2.5889	4.8156
M24	4.8158	3.5540	5.5037	0.2826	0.2826	0.4237
M25	5.0246	4.5348	9.3128	0.5826	0.5752	1.7748
M26	5.8434	8.03844	19.6718	0.7357	1.8813	8.7471
M27	8.9803	8.9398	19.0666	1.7213	1.7662	6.0082
M28	5.5534	7.0940	18.9287	0.6613	1.1534	5.9412
M29	9.2267	9.5179	18.3195	1.4669	1.8996	5.8551

. Moreover, we provide the detailed MAE and RMSE comparison results for the twenty-two locations under the 90 and 15 min time steps in Figure 7 and Figure 8, respectively. In addition, we introduce the MCNN to combine the ground and satellite images in Figure 9. Further, we introduced the clear and cloudy day effect on the model’s performance, as shown in Figure 10 and Figure 11, respectively, at 15 min time steps. We finally introduce the satellite spatial information compared to the ground database, as shown in Figure 12.

Table 3 presents the detailed MAE and RMSE comparison of all 29 models at multiple time steps (15, 30, and 90 min). We compared the few models that performed well, such as M7, M15, and M17, based on the ground image inputs under clear–cloudy circumstances. For further analysis of the model performance, we tested the models [M18-M29] based on the ground (GD), satellite (Sat), and satellite–ground (Sat-GD) combined inputs under the consideration of clear and cloudy situations separately. Among all these models, M18, M19, M24, and M25 showed excellent performance for all time steps. Above all, [M15,M17], [M18,24], and [M19, M25] performed well for multiple time steps based on the GD approach under the CR-CD, CR, and CD conditions, respectively. Further, the multiple input and output model’s results were constantly improved in comparison to the single input–output model [55,56].

Figure 5 shows the performance of the single versus multiple points approach for 2 h-ahead forecast in time steps of 5 min. Shown in Figure 5a,b are single versus multiple inputs and in Figure 5c,d single versus multiple outputs. The multipoint approach outperformed the single-point approach for all time steps. The multiple point approach offered 29.2060% and 36.5758% reductions in the MAE and 41.6292%, and 56.4366% reductions in the RMSE compared to the single-point approach. Notably, the multipoint approach had better performance between 1–9 and 8–21 time steps for the multiple input (M2) and multiple output (M3) approach, respectively.

Figure 6 shows the CNN and combined model (CNN-LSTM) approaches with spatial, temporal, and spatiotemporal as the input conditions and witnesses the MAE and RMSE decline for the spatiotemporal approach. However, the CNN-LSTM multi-output approach (M13 and M15) in both the spatial (Figure 6a,b) and temporal (Figure 6c,d) domains maintained outstanding forecasting performance without significant fluctuations. The combined model performed better than the individual approaches in the spatiotemporal domain, as shown in Figure 6e,f. The final comparison of all combined models is shown in Figure 6g,h, where the multi-output (M17) approach showed outstanding responses. We can therefore conclude that the spatiotemporal input in a multi-output approach is effective.

In [57], the CNN-LSTM hybrid model was simulated with meteorological parameters for PV power forecast with an RMSE of 22.5677% and an MAE of 58.9563% for 30 min-ahead forecast. Our proposed M17 (CNN-LSTM) hybrid model tested on a spatial–temporal image matrix showed excellent performance with a 2.1281% RMSE and a 10.67738% MAE for 30 min-ahead forecast, as shown in Table 3, and the detailed forecasting accuracy for 5 min time steps is shown in Figure 6g,h, being in the permissible range for 2 h-ahead forecast.

In Figure 7, all the models with multiple outputs were tested with time series, with the spatiotemporal vector and spatiotemporal input on different models with a fixed time of 90 min. The improvements for different locations did not follow the same pattern. The enhancement in the MAE and RMSE among the different points was different for Models M3, M9, and M11. However, the MAE and RMSE were significantly improved for M17 compared to M3, M9, and M11. Hence, the spatiotemporal approach with the CNN-LSTM is practical. The deep CNN with multi-input factors had an RMSE under mixed seasonal conditions for 1 h-ahead PV power forecast of 3.7448% [58]. Our CNN framework M9 based on the spatial–temporal image matrix achieved an RMSE ≤ 4% for 19 locations out of the 22 for the 90 min-ahead forecast, as shown in Figure 7.

In Figure 8, all the models were tested with the time series, with the spatiotemporal vector and spatiotemporal input, and the environment is shown in the form of a box plot. The box plot shows the absolute forecast errors of different models. Forecasting error (FE) =$\left(y_{actual}-y_{predict}\right)$, which is the absolute difference between the actual and predicted value. It seems that the performance ranking of different models according to the median (mean) of the absolute errors was consistent with the previous conclusion. The average deviation for all 22 locations for Model M17 was uniform, which signifies the suitability of Model M17. A narrower range of the error distribution was found with M17 compared to any other model under the CR-CD conditions. Hence, the spatiotemporal input showed a promising result in comparison to the other input types. The red-colored crosses in Figure 8 are outliers. The outliers are those data points that are significantly different from the rest of the dataset. Furthermore, most of the outliers in M9 were smaller than in any other model. Besides, the dispersion of the absolute errors was higher for longer forecast horizons.

3.2. Effects of Multi-Column CNN-LSTM model

The important applications and framework of the MCNN are explained in Section 2.2 and Figure 4b. There are multiple works in the literature [48,49,59,60] in favor of multiple columns for image classification. Here, we tried to use the MCNN for regression on different inputs. The first is from satellite images at low spatial resolutions (1–4 km), but with more spatial coverage, roughly a 100 × 100 km² grid area. Furthermore, the second is from ground spatial images at high resolutions with a low coverage area within 1 × 1 km${}^2$. The MCNN performed better for the multi-resolution input images, and the target was the in situ data. Actually, we performed the comparison at the ground level only. The comparison of the CNN-LSTM and MCNN for different data types such as GD, Sat, and the combined Sat-GD as the inputs were considered under the CR and CD conditions for time steps of 15 min, as shown in Figure 9. In Figure 9, [M18-M23] (left: 6-color bar) represents the CNN-LSTM model and [M24-M29] (right: 6-color bar) represents the MCNN model, whose details are given in Table 2. All these inputs were tested, and from the test response, we found that the ground-based approach showed a better response under both the CR and CD conditions due to the high-resolution spatial images. The multilevel stacking-based design of LSTM-MLP was used in the solar radiation forecasting, where a 6.08% RMSE was obtained in [32], whereas the CNN-LSTM [M18-M23] and MCNN [M24-M29] framework shown in Figure 9 achieved a <2% RMSE with Sat, GD, and Sat-GD as the inputs.

3.3. Effects of Cloudy Conditions

In Figure 10, the comparison of the hybrid forecasting model CNN-LSTM [M18,M19] and MCNN [M24,M25] under clear and cloudy sky conditions is shown separately. The top-left plot (Figure 10a) shows the actual and forecast power on a clear day using a dataset with 15 min time steps. The middle-left plot (Figure 10b) shows a linear regression graph of measured (actual) and predicted values on a clear day. The bottom-left plot (Figure 10c) shows the frequency of the forecast error graph of different models on a clear day. Similarly, all right-side plots represent a cloudy day. It is clearly shown in both conditions that the model prediction differed from clear day performance due to the non-linear nature of irradiance. Therefore, the linear regression and frequency distribution of the errors are scattered more in cloudy conditions in all three forecast plots.

The Taylor diagram is an acceptable and suitable tool to reflect the comparison results and evaluation of different models in a graphical fashion. It provides a concise statistical summary of how well patterns match each other regarding their standard deviation, root-mean-squared deviations (RMSD), and the correlation coefficient. It was applied to compare different energy forecast frameworks such as [M18, M24] and [M19, M25] under a clear and cloudy day, respectively. For these Taylor diagrams, the lower-right corner (where the correlation coefficient I = 1 and RMSD = 0) was considered the ground truth or the measured values (observation point). Moving along the radial distance towards the origin indicates that the standard deviation decreases, while moving along the radial angle toward the bottom axis improves the correlation. With the enhancement in the standard deviation and correlation coefficient, the RMSD will decrease. The better model will be close to the observation point on the graph. As shown in Figure 11, for all forecast models, all correlation coefficients were in the range of [0.99–1.00] and [0.95–0.99] under clear and cloudy conditions, whereas the models’ RMSD lies in the range of [0–0.1]. The observation arc shows that the standard deviation of the models lies in the range of 0.3–0.4 for both conditions.

3.4. Effects of Satellite Inputs

We also compared the above methods with satellite datasets since satellite coverage has a greater spatial dimension with a low spatial and temporal resolution, but the cloud origin and movement are easily calculated in such a region. In the present simulation and forecasting models, the satellite coverage area was more than (10-times) the ground-based approach. As shown in Figure 12, for all forecast models [M20, M26] and [M21, M27], the correlation coefficients were in the range [0.95–0.99] under clear and [0.9–0.95] under cloudy days. In comparison, the RMSDs of the models were in the range of [0–0.1] for clear and [0.1–0.2] for cloudy days. The observation arc shows that all models’ standard deviations lie in the range of [0.3–0.4] for clear and [0.2–0.4] for cloudy days. The correlation coefficients, standard deviations, and RMSD parameters are desirable with the GD inputs compared to the Sat inputs because of the high spatial and temporal resolution, and hence, the GD-based approach is promising in comparison to the Sat-based on.

4. Conclusions

There are two basic problems when dealing with multiple data sources with the CNN. First, the application of the CNN to time series data is a bit difficult. Second, the handling of data with different spatial resolutions is challenging. For the first time, this paper assessed the potential of utilizing the convolutional neural network to forecast solar power using the multipoint and GCF approaches. The MCNN encodes the solar power from time series and spatial data to map the plants’ data into image matrices. The assessed results indicate that the ground-based approach with the multipoint input and output approach significantly improved the solar power generation forecast. In comparison with state-of-the-art time-series-based LSTM and image-based CNN, the multiple point approach offered 29.2060% and 36.5758% reductions in the MAE and 41.6292% and 56.4366% reductions in the RMSE. The spatial–temporal-based hybrid CNN-LSTM model outperformed all methods with the CR and CD conditions under the ground data.

Furthermore, the model was tested under the clear and cloudy conditions separately for the ground and satellite data to better analyze it. The findings for the proposed approach were best with the spatiotemporal ground inputs, as the correlation coefficients were in the range of [0.99–1.00] and [0.95–0.99] under clear and cloudy days, respectively, which are closer to the observational point, as shown in Figure 11. Furthermore, the CNN-LSTM and MCNN showed excellent performance with the ground-based spatiotemporal image in comparison with other approaches, with an RMSE of 0.3324% and 0.5826%, respectively, under cloudy conditions, for a 15 min-ahead forecast. The assessments of our approach could be further improved with more data points and more cloud information at multiple locations.