Advances in Short-Term Solar Forecasting: A Review and Benchmark of Machine Learning Methods and Relevant Data Sources

Abstract

Solar forecasting is becoming increasingly important due to the exponential growth in total global solar capacity each year. More photovoltaic (PV) penetration in the grid poses problems for grid stability due to the inherent intermittent and variable nature of PV power production. Therefore, forecasting of solar quantities becomes increasingly important to grid operators and market participants. This review presents the most recent relevant studies focusing on short-term forecasting of solar irradiance and PV power production. Recent research has increasingly turned to machine learning to address this challenge. The paper provides a discussion about building a solar forecasting model, including evaluation measures and machine learning method selection through analysed literature. Given that machine learning is data-driven, the focus of this review has been placed on data sources referenced in the literature. Open-access data sources have been compiled and explored. The main contribution of this paper is the establishment of a benchmark for assessing the performance of solar forecasting models. This benchmark utilizes the mentioned open-source datasets, offering a standardized platform for future research. It serves the crucial purpose of streamlining investigations and facilitating direct comparisons among different forecasting methodologies in the field of solar forecasting.

1. Introduction

Solar energy has emerged as a vital and sustainable source of energy, contributing significantly to the global transition toward renewable energy sources. According to the International Energy Agency, solar energy capacity share accounted for 12.8% of the world’s total energy capacity for 2022. Their projections indicate that by 2027, it is expected to exceed 20%, surpassing all other sources of energy [1]. However, the inherently intermittent and variable nature of solar radiation poses challenges to grid integration, and subsequently, grid energy management. Photovoltaic (PV) power production forecasting becomes increasingly vital for both short-term and long-term horizons. PV production depends on the amount of solar radiation reaching the PV cells on panels. As solar radiance is affected by current weather conditions such as cloudiness, temperature and windiness, it becomes difficult to plan and manage the energy within the system [2]. As the share of solar energy rapidly increases, so does the need for reliable and accurate PV production forecasting. It is crucial for several reasons, such as grid integration, system stability, energy management, market participation and research and development [3]. Recent advancements in computing power have led to a significant increase in the popularity of machine learning (ML) algorithms for PV forecasting. This trend has resulted in a surge in the publication of research papers on this topic, with the number of such papers rising exponentially, as highlighted in a 2023 study [2] by Alcaniz et al. Researchers have been increasingly leveraging the capabilities of ML to enhance the accuracy and efficiency of PV forecasting models, reflecting the growing significance of this field in the renewable energy sector. This paper serves as a comprehensive review of the field of solar irradiance and PV power forecasting with a focus on data sources. Research questions that we focus on are as follows:

• The available open-source data enabling the comparison and benchmarking of different forecasting methods. Today, there is a significant body of work describing different ML methods and explaining the benefits of applying them to specific case studies. However, in most of the papers, the dataset is unavailable and unknown in terms of the number of data and quality of the data, as well as availability to produce the results from the published paper. We find that such an approach hinders future development, as each researcher/developer needs to self-test all available methods to learn about their advantages and disadvantages. Our goal is to list the open available data and to assist in creating an open-source community where transparency of newly developed tools/solutions is key to quality research.
• The relevant metrics to benchmark the effectiveness of a certain ML method as well as what is the range of the values for those metrics in previously published papers. Our goal is to provide a framework for future researchers to use adequate metrics and to understand the quality of their proposed method.
• New sources of data, previously less or not utilized, that could improve the existing or new ML methods. Here again, we focus on open sets of data that are transparent and available to everyone, and as such can serve as a unified benchmark of the proposed method.

Review papers [2,3,4,5,6,7] helped shape the outline of this paper. These review papers focus on summarizing state-of-the-art (SOTA) papers regarding PV forecasting using ML, outlining their highlights and techniques used to challenge the problem. In this context, this research builds upon the aforementioned studies while simultaneously going into greater depth of exploring utilized data sources of solar irradiance, historical PV power production and satellite images. Given the emphasis on data-driven forecasting methods in this research, it is important to highlight and explore available open-source data.

Outline This review encompasses papers that employ machine learning techniques for PV power or solar irradiance forecasting. While numerous papers, such as [8,9,10,11,12], concentrate on long-term forecasts, a greater emphasis on the short-term horizon (up to 7 days in advance [2]) has been placed in this review. Out of the total number of identified papers relevant to this topic, we have selected papers published in the past 5 years (2018 onwards), focusing on the ones most cited in literature. This includes journal and conference papers, along with book sections.

Contributions This paper presents a contribution to the field through the creation of a comprehensive benchmark for open-source datasets, providing a valuable resource accessible to the research community for rigorous and standardized benchmarking. Furthermore, an exploration of the training and testing periods within the analysed literature is included, further enriching the understanding of temporal aspects in the context of benchmarking practices for solar forecasting. Additionally, we provide a comprehensive discussion with suggestions on building a solar forecasting model; including the selection of machine learning methods and evaluation measures. Together, these contributions aim to enhance the methodological robustness and comparability of research in the domain of solar forecasting.

Contents In Section 2, Research Area Overview, are highlighted key features of analysed papers. Section 3, Machine Learning Methods, provides brief explanations of ML methods used, while Section 4, Open-Source Data, offers a more comprehensive summary of data utilized in this field. Finally, Section 5, Conclusions, summarizes the paper and recommends future work.

2. Research Area Overview

The focus is placed on literature regarding ML forecasting of the short-term PV power output of a solar power plant. As the PV power output of a plant is entirely dependent on the amount of solar irradiance reaching the solar panels, the focus was extended to literature regarding solar irradiance forecasting as well. The true amount of solar radiation reaching panels depends on the local weather conditions. Consequently, many papers focused on forecasting local cloud coverage to aid irradiance or PV power forecasting (up to 6 h in advance) [13,14,15,16]. This is carried out by using either satellite images [13] or images collected using a sky imager on the location [17]. The images are utilized to predict cloud movement for ultra-short-term horizons [13], as local weather forecasts are inaccurate beyond that time span.

Information on future local solar irradiance is most often obtained through Numerical Weather Prediction (NWP) [18], e.g., the Weather Research and Forecasting (WRF) model [19,20]. It often includes forecasts of multiple weather variables such as temperature, humidity, wind speed, etc., 72 h in advance, extracted for a specific location.

The next key highlights have been extracted from analysed literature and are shown in

Table 1. Research area overview.

Reference	Goal	Horizon (Step)	Test Size	Error Term	Methods	Location
[21]	PV power forecasting	24 h (15 min)	5 months	MAE, MSE, R${}^2$, RMSE, Error	MLR, ANN	Hungary
[22]	PV power forecasting	24 h (1 h)	not specified	RMSE, MAE	CNN, SVM, DT, MLP, LSTM, RF	Taiwan
[23]	PV power forecasting	15, 30, 60 min (15 min)	4 months	nMAE, nRMSE, 1-$\rho$	Conv-LSTM, CNN	China
[24]	PV power forecasting	15 min, 1 h, 3 h, 6 h (15 min)	not specified	nRMSE, nMAE	ST-Lasso, AR	France
[25]	PV power forecasting	15 min	20 days	RMSE, FS	CNN, AR	USA
[26]	Irradiance forecasting	6 h (1 h)	1 year	rRMSE, MBE, FS	DNN (FNN), AR, GBT	Netherlands
[27]	Irradiance forecasting	1 h (10 min)	2 years	RMSE, MBE, FS	ANN, GBM, RF, Conv ReLU + Dense	USA
[28]	PV power forecasting	1, 2, 3 h (1 h)	23 days	RMSE, MSE, $\rho$	BiLSTM, ARMA, ARIMA, SARIMA	China
[29]	Irradiance forecasting	24 h (15 min)	3 years	MSE, FS	Clear sky models	USA
[15]	Irradiance forecasting	15 min	36 days	RMSE, MAE, $\rho$	CNN-LSTM, CNN-ANN	USA
[30]	Irradiance forecasting	10 min (1 min)	1 month	RMSE, MAE, nRMSE, nMAE, FS	ResNet	USA
[31]	PV power forecasting	15 min (1 min)	20 days	RMSE, MAE, FS	CNN + Dense	USA
[32]	PV power forecasting	1 h (1 min)	1 week	RMSE, MAE, MAPE	LSTM, ANN, GRU	Taiwan (*)
[33]	PV power forecasting	6, 12, 24 h (1 h)	100 days	RMSE, MAE, MAPE, MRE, MBE	LSTM, ANN, GRU	USA
[16]	Irradiance forecasting	15 min	70 days	RMSE, MAE, $\rho$, nPIAW, PICP, CWC	CNN-QR, LSTM-QR, ANN	USA
[17]	Irradiance forecasting	10 min (1 min)	30 days	MAPE, RMSE, MBE	SVM, BPNN, ARIMA	USA
[14]	Irradiance forecasting	5, 10 min	5 months	Accuracy	ResNet	Italy, Switzerland
[34]	PV power forecasting	24 h (15 min)	1 year	CRPS, MARE, PIAW	QR	Hungary
[35]	PV power forecasting	24 h	31 days	RMSE, MSE, MAE, MAPE, nRMSE, R${}^2$	GRU, RF	China
[36]	PV power forecasting	5 min (0.5 min)	1 day	SSIM, MSE	Conv-AutoEncoder	USA
[37]	PV power forecasting	24 h	5 weeks	MAE, RMSE, nMAE, nRMSE, IA, Error	MLP	Finland
[38]	PV power forecasting	24 h (15 min)	1 year	nRMSE, nMAE, nMBE	ANN + Physical Model	Hungary
[39]	PV power forecasting	24 h (15 min)	2 years	nRMSE, nMAE, nMBE, FS, $\rho$	LR, SVM, CatBoost, MLP, RF, LGBM, XGB	Hungary
[40]	PV power forecasting	5 h (1 h)	4 months	MAPE, MAE, RMSE, nMAE	LSTM, CNN, FNN	South Korea
[41]	PV power forecasting	24 h	10 months	RMSE, MSE, MAE, $\rho$	XGB, AdaBoost, RF, ETR	Kuala Lumpur
[42]	PV power forecasting	24 h (15 min)	1.5 months	RMSE, MAPE, MAE, $\rho$	LSTM	Saudi Arabia (*)
[43]	PV power forecasting	6 h (15 min)	4 months, 1 year	nMAE, nRMSE	GCLSTM, GCTrafo, TSM-GAT, STCNN	Switzerland, USA
[44]	PV power forecasting	2, 6 h (1 h)	2.5 months	nRMSE, MAPE, Error	AR, FNN, LSTM, STCNN	USA
[45]	PV power forecasting	7.5, 15 min, (0.05 min)	4 days	MAE, RMSE, adj R${}^2$, Accuracy	DT, RF, SVR, MLP, LSTM, BiLSTM, NLSTM	China
[46]	PV power forecasting	24 h (1 h)	3.5 months	MAPE, MRE	SVM-SSA, CNN-SSA, LSTM-SSA	Taiwan
[47]	PV power forecasting	6 h (15 min)	1 year	nMAE, nRMSE	STAR, GCLSTM, GCTrafo, STCNN, SVR, EDLSTM	Switzerland
[48]	PV power forecasting	16 h (15 min)	not specified	RMSE, MAD	LSTM, Wavelet NN, SVM, BPNN	China
[49]	PV power forecasting	72 h (1 h)	2 months	MAE, MSE	BPNN	China
[50]	PV power forecasting	10 h (1 h)	500 samples	MAPE, Error	MR-ESN	USA
[51]	Irradiance forecasting	not specified	not specified	MAE, RMSE, PICP, PINAW, Accuracy	ELM	China
[20]	PV power forecasting	1 h (1 h)	3.5 months	MSE	Ridge	Croatia
[52]	PV power forecasting	24 h (1 h)	22 days	MAPE, nMAE, wMAE, eMAE, nRMSE, $\rho$, Error	PHANN	Italy
[53]	PV power forecasting	24, 72 h (1 h)	1 month	IAE, RMSE, MSE, R${}^2$	NARX	Egypt
[54]	PV power forecasting	14 h (1 h)	4 months	MAE, RMSE	ANN, DNN, LSTM, ARIMA, SARIMA	South Korea
[55]	PV power forecasting	24 h	4 months	MAE, RMSE, MAPE, Error, $\rho$	MLP (DNN)	South Korea
[56]	PV power forecasting	1, 5, 30, 60 min (1 min)	1 month	MAE, RMSE, MAPE, $\rho$	LSTM, ENN, NAR	Italy
[57]	Irradiance forecasting	17 h (1 h)	1 year	MAE, MAPE, RMSE	DRL + CNN-BiLSTM	USA
[13]	PV power forecasting	4 h (15 min)	30 days	nMAE, nRMSE	SVM, GBDT, ARMA	USA
[58]	Irradiance forecasting	6 h (15 min)	6 months	MAE, rMAE, RMSE, rRMSE	ConvNet + Dense	France
[59]	PV power forecasting	1 min	200 samples	$\rho$	K-means, PSO	China
[60]	PV power forecasting	1 h	1 year	RMSE, R${}^2$	LSTM, SVR, ANN, ANFIS, GPR	Kuala Lumpur
[61]	Irradiance forecasting	1 min	not specified	RMSE, MAE, R${}^2$	SVR, BPNN, XGB, DXNN	France
[62]	PV power forecasting	36 h (15 min)	1 month	RMSE, MAE, MBE, CRPS, PIAW, PICP	NGBoost, GP	Germany

*—Location of author affiliation

:

• Goal—goal of the paper: PV power or solar irradiance forecasting;
• Horizon (Step)— forecast horizon with the granularity of the forecast (step);
• Test size—it is important to highlight this feature, as a larger test set provides a more statistically significant sample of the data, indicates robustness, reduces risk of overfitting and gives more credibility to solutions tested on larger datasets;
• Error term—measure of performance used to compare methods;
• Method—ML method employed in the paper;
• Location—geographical location of the PV power plant(s); all locations referred to in the analysed literature are shown on the world map in Figure 1.

Table 1 encompasses the key features of the analysed literature. The terms from the table are explained in the subsection Performance Measures and Section 3, Machine Learning Methods.

Performance Measures

In order to evaluate predictive models, a number of performance measures have been reported in literature. Mean Squared Error (MSE), Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) are the most common performance measures used in analysed papers. Mean Absolute Deviation (MAD) [48] is also referred to as Mean Relative Error (MRE) in [33,46]; both measures represent a percentage deviation from the total (maximum) capacity of the PV plant. In contrast, Mean Absolute Percentage Error (MAPE) represents the percentage error relative to the actual production at a given moment. Forecast skill (FS) as employed in [25,31], explained in [63], quantifies the percentage deviation from persistence models. A positive percentage denotes improvement over persistence models, while a negative percentage indicates the opposite. Envelope Weighted Mean Absolute Error (eMAE) measures the absolute error between predicted values and the nearest points on the envelope (upper and lower bounds of historical data) curves [52]. If a prediction falls within the envelope, the error is zero; otherwise, it quantifies the vertical distance from the prediction to the envelope. Index of Agreement (IA) [37] denotes the closeness (agreement) between the forecasted and the actual values. IA ranges from 0 to 1, with 1 indicating perfect alignment between forecasted and actual values. It considers the squared differences with variability of the observed values to provide a comprehensive measure of model performance.

Probabilistic models are evaluated with the following measures. Continuous Ranked Probability Score (CRPS) [34] quantifies the discrepancy between a forecast probability distribution and the observed outcomes. It assesses the accuracy of probabilistic forecasts, with a lower CRPS value indicating better forecast performance. Prediction Interval Average Width (PIAW) is also commonly used for probabilistic forecasts and indicates sharpness of the prediction interval. In other words, it represents the ability of a forecast to accurately represent the variability or uncertainty in the data (prediction intervals are neither too wide or too narrow). Prediction Interval Coverage Probability (PICP), utilized in [16,51], denotes how often the forecasted value falls within the prediction intervals.

The aforementioned performance measures of the analysed papers are shown in more detail in

Table 2. Performance measures employed in analysed research studies.

Measure of Performance	Equation
Root Mean Squared Error (RMSE) *	${\displaystyle \sqrt{\frac{1}{n}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}}$
Mean Squared Error (MSE)	${\displaystyle \frac{1}{n}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}$
Mean Absolute Error (MAE) *	${\displaystyle \frac{1}{n}\sum _{i=1}^n|y_i-{\widehat{y}}_i|}$
Mean Absolute Percentage Error (MAPE)	${\displaystyle \frac{1}{n}\sum _{i=1}^n\frac{|y_i-{\widehat{y}}_i|}{|y_i|}\times 100\%}$
Mean Absolute Deviation (MAD)	${\displaystyle \frac{1}{n}\sum _{i=1}^n\frac{|y_i-{\widehat{y}}_i|}{|P{V}_{capacity}|}\times 100\%}$
Mean Bias Error (MBE) *	${\displaystyle \frac{1}{n}\sum _{i=1}^n(y_i-{\widehat{y}}_i)}$
Index of Agreement (IA)	${\displaystyle 1-\frac{{\displaystyle \sum _{i=1}^n}{(y_i-{\widehat{y}}_i)}^2}{{\displaystyle \sum _{i=1}^n}\left(\right|{\widehat{y}}_i-\overline{y}|+|y_i-\overline{y}{\left|\right)}^2}}$
Correlation Coefficient ($\rho$)	${\displaystyle \frac{{\displaystyle \sum _{i=1}^n}(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{\displaystyle \sum _{i=1}^n}{(x_i-\overline{x})}^2}\sqrt{{\displaystyle \sum _{i=1}^n}{(y_i-\overline{y})}^2}}}$
Coefficient of Determination (R${}^2$)	${\displaystyle 1-\frac{{\displaystyle \sum _{i=1}^n}(y_i-{\widehat{y}}_i)}{{\displaystyle \sum _{i=1}^n}(y_i-\overline{y})}}$
Continuous Ranked Probability Score (CRPS)	${\displaystyle \frac{1}{n}\sum _{i=1}^n\underset{-\infty }{\overset{\infty }{\int }}{[F^{y_i}\left(x\right)-1(x-y_i)]}^{2}dx}$
Prediction Interval Average Width (PIAW) *	${\displaystyle \frac{1}{n}\sum _{i=1}^n(U_i-L_i)}$
Prediction Interval Coverage Probability (PICP)	${\displaystyle \frac{1}{n}\sum _{i=1}^n{k}_i}$
Coverage Width Calculation (CWC)	${\displaystyle \left\{\begin{array}{cc}PIAW,\hfill & PICP\ge u\hfill \\ PIAW(1+PICP)e^{-g(PICP-u)},\hfill & PICP<u\hfill \end{array}\right.}$
Forecast Skill (FS)	${\displaystyle 1-\frac{RMS{E}_{model}}{RMS{E}_{persistence}}}$
Envelope Weighted Mean Absolute Error (eMAE)	${\displaystyle \frac{{\displaystyle \sum _{i=1}^n}|y_i-{\widehat{y}}_i|}{{\displaystyle \sum _{i=1}^n}max(y_i,{\widehat{y}}_i)}\times 100\%}$

* Also employed in normalized (or relative) shape related to maximum plant capacity.

.

When selecting error measures to evaluate forecasting models, the authors provide the following remarks. It is important not to evaluate forecasting performance on a single measure as it can be misleading. To give a banal example, a forecaster always outputTing the average value of the training set would have a very low mean bias error. Consequently, evaluating only on bias error in this example would be decieving. Therefore, multiple error measures must be utilized to accurately evaluate a forecasting model, preferably measures which focus on different aspects of the forecast deviation from the target values. Chicco et al. [64] argue that R${}^2$ is more informative than MAE, MAPE, RMSE, MSE and SMAPE (Symmetric MAPE) for evaluating regression models. They report comparisons in several use cases and suggest the use of R${}^2$ as a standard metric for regression analyses. Hodson in [65] debates about the choice of RMSE or MAE for model evaluation. He presents that RMSE is optimal for normal (Gaussian) errors while MAE is optimal for Laplacian errors. In the case of solar forecasting, the Laplacian distribution is often more suitable for modelling solar forecasting errors than the normal distribution. This is especially true in scenarios with heavier-tailed errors or sharp peaks, likely due to outliers or sudden environmental changes impacting PV generation. Nevertheless, there is no drawback in evaluating models on both measures. Furthermore, Koutsandreas et al. in [66] provide a comprehensive discussion about the selection of accuracy measures for forecasting models. Optimizing a forecasting model based on business case requirements, such as minimizing MSE, might seem intuitive. However, Koutsandreas et al. [66] show that mismatching the models’ loss function and evaluating measure has a minor effect on accuracy. Robust and accurate models are supposed to perform well according to any evaluating measure. Additionally, the authors recommend the use of forecast skill (FS) measure when building a project concerning solar forecasting. It requires comparison of the preferred model and simple, baseline models. Low FS value indicates that the preferred model is not significantly outperforming baseline models and needs further improvements. To summarize, multiple error measures need to be used for solar model evaluation. Among these errors, we suggest using RMSE, MAE, R${}^2$ and forecast skill as standard metrics for deterministic forecasting, at the minimum. For probabilistic models we recommend using CRPS, PICP and PIAW.

3. Machine Learning Methods

In this section we provide brief descriptions of various machine learning methods used in analysed literature. The idea is not to go into great detail of all the machine learning architectures for clarity purposes as these architectures are discussed in detail in other literature. To conclude the section, the authors provide their suggestions regarding the selection of machine learning architectures for solar forecasting.

Autoregressive models (AR) are a group of statistical models used in time-series analysis and forecasting. AR models describe a variable’s behaviour over time by relating it to its past values. They assume that the current value of a variable is a (non)linear function of its previous values. AR models can be very successful in solar forecasting, as often we can model forecasts on prior measurements. For example, during a summer period we can infer that the irradiance value at noon for tomorrow is likely to resemble the irradiance value of noon for today. Upgraded versions of AR models use exogenous variables (X) as inputs, denoted ARX. Most widespread AR models also use a moving average (MA) for forecasting, such as autoregressive moving average (ARMA) and its improvements ARIMA (I-Integrated), SARIMA (S-Seasonal) and SARIMAX [28,54]. Autoregressive Neural Networks (NAR(X)) have also been used in [53,56].

3.1. Classical Machine Learning

Linear regression (LR) is a simple ML method which assumes linear dependency between input and output variables. Linear regression is denoted as $y=wx+b$ and is considered to be a simple and interpretable ML solution. This approach remains relevant in the field of solar forecasting due to the inherent linear dependence between PV power plant production (y from the notation) and solar irradiance (x from the notation). In other words, the relationship between the aforementioned variables can be explained by a linear dependence in majority, and thus, LR remains a powerful tool in solar forecasting. This approach is simple to optimize, and is also interpretable, as we can explain how the inputs constructed the output of the model via weights associated with each input. To prevent overfitting, we discern between different types of regularization to obtain L0, L1 (Lasso) and L2 (Ridge) linear regression. The authors recommend employing this method as it is extremely effective and interpretable, despite its simplicity, when building a solar forecaster.

A Random Forest (RF [22,35,45]) is an ensemble ML method that combines multiple decision trees (DT) to improve generalization by aggregating individual DT predictions. By aggregating multiple simpler models, RF intrinsically reduces overfitting and improves generalization. This architecture can easily handle multiple features for solar forecasting such as temperature, wind speed, etc. Similar to LR, RF can be considered a more interpretable solution than most as it provides feature importance scores. Moreover, RF can handle missing data effectively. In practical scenarios, meteorological data may have missing values due to sensor malfunctions or outages. This is where an RF-based solar forecaster is extremely valuable and can provide accurate forecasts, even with incomplete data.

A Support Vector Machine (SVM) [13,17] is a classification ML algorithm that finds the optimal hyperplane that maximizes the margin between nearest data points of different classes. The Support Vector Regressor (SVR) [60] uses the same principle of support vectors. Like SVM, SVR aims to find an optimal hyperplane, but in the context of regression, the goal is to find a hyperplane that best fits the data points while minimizing the margin violations (i.e., how much the predicted values deviate from the actual values). Similar to RF, SVR is robust to outliers and can handle any possible non-linearity and high-dimensionality in the data space of solar forecasting. Furthermore, SVR can be tuned to handle non-uniform data distributions, as the distribution of solar parameters is often non-uniform due to sudden changes in weather conditions. Moreover, SVR employs a hyperparameter that defines a margin of tolerance for errors. This can be particularly useful in scenarios where a certain degree of error is acceptable, i.e., it allows for flexibility in the model’s forecasts. The major drawback of SVRs in solar forecasting is their complexity and lack of explainability, which can be a deal-breaker for many operators and grid participants utilizing solar forecasts.

A Gradient Boosting (GB) regression model is created in a sequential fashion, starting from a simple, “weak” prediction model (e.g., a decision tree). The initial model is improved (boosted) in each iteration based on the loss function gradient, with the goal of minimizing said loss function. Similar to RF and SVR, GB can handle complex, high-dimensional data and is robust to outliers due to the ensemble nature of the method. This method can also handle missing data and provide accurate forecasts, even when data are incomplete. GB can adapt to varying data distributions, making it suitable for forecasting scenarios where the distribution of solar irradiance may suddenly change during the day which is likely. It also allows for the use of various loss functions, providing flexibility in optimizing the solar forecaster for specific business requirements. Its effectiveness on complex data, along with its robustness, has sparked new iterations of it, such as the Light Gradient Boosting Model (LGBM) [39], Adaptive Boosting model (AdaBoost) [41] and Categorical Boosting model (CatBoost) [39]. In addition to LR, the authors recognize the significance of GB methods for building a solar forecaster.

3.2. Neural Networks (Deep Learning)

Feedforward Neural Networks (FNN) comprise fully interconnected nodes, including an input layer, one or more hidden layers and an output layer. Information flows unidirectionally from one node (neuron) to the node in the next layer, i.e., the node “feeds” information forward to the next node. FNN, also called Multilayer Perceptron (MLP) [55], is sometimes synonymous with the term Deep Neural Network (DNN), as any model with at least one hidden layer is called deep, so FNN with a hidden layer is actually a DNN as well. A Backpropagation Neural Network (BPNN), used in [17,48,49], is also a type of FNN that is specifically optimized using a backpropagation algorithm. FNNs can effectively capture non-linear relationships in data. However, they posses a limited capability to capture temporal dependencies, which is important in solar forecasting. Furthermore, FNNs often lack explainability, and the authors suggest other appropriate methods to tackle solar forecasting.

A Convolutional Neural Network (CNN), also referred to as ConvNet [58] in text, is a specialized deep learning architecture for processing grid-like data, such as images and spatial or tabular data. Unlike traditional NNs, CNNs excel at capturing localized patterns and spatial hierarchies within the input data. They are composed of convolutional layers that apply trainable filters to the input, producing feature maps that highlight relevant features. They typically produce outputs using fully connected (dense) layers after relevant feature extraction has been performed by convolutional and pooling layers, shown in Figure 2. CNNs can be highly effective for solar forecasting purposes. For example, Simeunović et al. in [43] use a Graph CNN for forecasting the production of multiple PV power plants, where each plant represents a node of the graph. This type of CNN can effectively model spatial and temporal dependencies between different PV power plants, identifying common patterns across different locations through these dependencies. The authors suggest this method when forecasting production for multiple PV power plants that have spatial dependencies between them.

Recurrent Neural Networks (RNNs) are designed for processing sequential data with the ability to capture dependencies over time. Long Short-Term Memory (LSTM) networks are the most prominent RNN-type architectures in all analysed papers. LSTM excels at capturing and modelling dependencies in sequential data using specialized memory cells, which can capture long-range dependencies. Their suitability for time-series (sequential data) analysis has led to the widespread adoption of LSTM networks in recent research papers focused on PV forecasting.

Figure 3 displays an LSTM cell, which is made of three discernable gates. The forget gate controls what information from the previous cell state should be forgotten, while the input gate determines what new information should be added to the current cell state. The output gate regulates what information should be exposed to the next cell, filtering less important information out. Networks based on bidirectional LSTM cells are also used in papers, such as [28]. Bidirectional cells add upon the original LSTM cells with a functionality of passing information forward or backwards through the cell. The Gated Recurrent Unit (GRU) [35] is also closely related to—and can be seen as a precursor to—LSTM. One of the disadvantages is the vulnerability to vanishing/exploding gradients when training RNN models. They are also often computationally expensive to train. However, due to their suitability for sequential and time-series data, the authors recognize RNNs as powerful tools in solar forecasting when trained and constructed correctly.

Finally, it is worth mentioning Residual Networks, or ResNets for short [14,30]. ResNets first introduced skip (residual) connections between layers [69]. Their goal is to mitigate the vanishing gradient problem, which allows for the training of very deep NNs more effectively. First introduced for CNNs, the term ResNet is now used for any NN that uses skip connections. These skip connections ensure that exploding or vanishing gradients do not propagate further by replacing them with an identity function. The authors advise constructing a solar forecaster using RNN-type architecture with the addition of skip connections to mitigate the problem of vanishing/exploding gradients.

Most of the ML methods used are constructed using some of the aforementioned methods, either individually or in a hybrid mode, combining multiple methods such as Conv-LSTM [40] or NARX [53].

Given NNs’ capability to model complex dependencies, it might seem intuitive to assume that they are always the optimal choice for solar forecasting, and forecasting tasks in general. However, this assumption relies on the availability of substantial computational power and extensive, theoretically sound datasets. However, real-world data often are noisy and scarce, and here, NNs fail to outperform simpler methods. Grinsztajn et al. in [70] provide a comprehensive comparison of tree-based models such as Random Forest (RF) and Gradient Boosting (GB) against state-of-the-art deep networks on various medium-sized tabular datasets. They clearly state the superiority of simpler, tree-based models against more complex, deep networks. Munawar et al. in [71] confirm this hypothesis for short-term solar forecasting with XGB outperforming NNs. On the other hand, Markovics et al. in [39] show that a Multilayer Perceptron (MLP) performs at least as well as the top classical methods across various evaluation metrics. Another aspect of building a solar forecasting model and understanding business needs includes the interpretability of the forecaster output. One of the problems with complex NNs is their black-box nature. It is often needed to reduce the complexity of the model (which may affect performance) in order to increase the understanding of the forecaster results. We recommend reading [72,73,74] in order to explore methods of increasing the interpretability of a forecaster. Papers that emphasize the interpretability of solar forecasting models include [61,62,75]. Taking all of this into consideration, we recommend the following methodology when building a solar forecasting model: Start the process with fundamental baseline models and progressively advance from simpler models, such as linear regression (LR), to more intricate ones, including gradient boosting (GB) models, culminating with recurrent neural networks (RNNs), which use skip connections (ResNet). Evaluate models on the already-mentioned minimum set of evaluation metrics: RMSE, MAE, FS and R${}^2$. When building a probabilistic forecaster, evaluate on CRPS, PICP and PIAW, at the minimum. Model architecture selection is an important part of building an accurate and robust solar forecasting model. Nonetheless, we argue that it is not the most important; the quality and timely accessibility of data is the most important part of building any forecasting model. Having sound data and using any of the above-mentioned ML methods should provide similar results at least. The final minor improvements can be made with method selection and proper training and hyperparameter optimization. This is why we put focus on the sources of data for solar forecasting in Section 4.

4. Open-Source Data

ML methods rely heavily on the data they are trained on, making the quality and accessibility of data the pivotal factor in the success of the methods solving complex tasks. Many authors gathered data directly from the power plant owner or from the grid operator. These data are most often closed-source. To oppose this and to encourage further research and benchmarking, we provide a compilation of open-access data regarding PV forecasting.

Table 3. Data sources.

Source/Name	Data Type	Link	References
NREL	Irradiance, satellite images	https://midcdmz.nrel.gov/apps/sitehome.pl?site=BMS	[15,16,27,33,43,44,57]
HelioClim3	Irradiance	https://www.soda-pro.com/help/helioclim/helioclim-3-overview	[24,76,77]
EUMETSAT	Satellite images	https://www.eumetsat.int/eumetsat-data-centre	[20]
Girasol	Irradiance	https://datadryad.org/stash/dataset/doi:10.5061/dryad.zcrjdfn9m	[78,79]
ECMWF	Irradiance	https://www.ecmwf.int/en/forecasts/datasets/set-i	[11,26]
SURFRAD	Satellite images	https://gml.noaa.gov/grad/surfrad/	[29,80]
SKIPP’D	Satellite images	https://github.com/yuhao-nie/Stanford-solar-forecasting-dataset	[31]
AERONET	Aerosol optical depth (AOD)	https://aeronet.gsfc.nasa.gov/new_web/data.html	[49,81]
GFS	Irradiance	global_weather_forecast	[47]
SARAH-2	Irradiance	pvgis_sarah-2	[82]
C3S	Irradiance	https://cds.climate.copernicus.eu/cdsapp#!/search?type=dataset	[83]

summarizes the data explored in this review. It states the Source/Name of the database or provider of the data, Data type, Link to the data or the website containing it (all accessed on 18 September 2023) and the literature citing the use of the database in the References column.

4.1. Sources

The National Renewable Energy Laboratory (NREL, Denver West Parkway Golden, CO, USA), specifically the Solar Radiation Research Laboratory (SRRL, Tokyo, Japan), provides the largest publicly available dataset with both total sky images (TSI) and numerical measurements. Continuous solar measurements have been gathered by the SRRL at NREL’s South Table Campus in Golden, Colorado (USA), starting from 1981. Alongside numerical measurements of minutely resolution, the SRRL dataset includes two sets of TSIs taken by a Yankee Total Sky Imager (TSI-800, Fort Washington, PA, USA) and EKO All Sky Imager (Lily Dale, NY, USA)with a 10 min resolution [27].

The HelioClim-3 database was created using a Meteosat First Generation (MFG) satellite of European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT, Darmstadt, Germany) [24,76]. The satellite images obtained from MFG EUMETSAT are processed to yield values of incoming surface solar irradiance (SSI). The data collection for the database began in 2004, and is updated daily.

EUMETSAT, named in the Source/Name column of Table 3, contains various products (types) of satellite images. In [20], cloud mask satellite images (clm) were explored to investigate their usefulness for PV power forecasting. Another example of EUMETSAT products would be a microphysics (mphys) image product that contains information of distinction between ice, water clouds and cirrus clouds. Some products are available in 5 min and some in 15 min temporal resolution, with 1 to 3 km spatial resolution [84].

Girasol, a sky imaging and global solar irradiance dataset, contains solar cycle recordings of 244 days from 3 years (the size of 110+ GB). The cameras sample the images in 15 s intervals, with logs recorded in periods when the Sun’s elevation angle is higher than 15°. The pyranometer data are sampled 4 to 6 times per second, while the weather station data sample interval is 10 min [78]. Further explanation can be found following the provided link in the Link column.

The European Centre for Medium-Range Weather Forecasts (ECMWF) provides forecasts of multiple weather variables, including various solar radiation parameters such as direct solar radiation and top net solar radiation, with a resolution of 0.1° × 0.1° latitude/longitude [85].

The Aerosol Robotic Network (AERONET) provides aerosol optical depth data. It is a measure of the attenuation of solar radiation due to the scattering and absorption of sunlight by aerosol particles in the Earth’s atmosphere [49,81], which can also be utilized to improve PV forecasting.

The Copernicus Climate Change Service (C3S) contains multiple useful datasets regarding solar irradiance. For example, the global dataset ERA5-Land hourly data from 1950 to present contains hourly data from 1950 until 2021, such as surface net solar radiation and surface solar radiation downwards. Some other possible useful datasets include the Surface radiation budget from 1982 to present derived from satellite observations. C3S also provides subregion extraction through latitudinal and longitudinal values [83,86].

The Surface Radiation Budget Network (SURFRAD) was established in 1993 with the focus on supporting climate research with accurate, continuous and long-term measurements of the surface radiation budget across USA. SURFRAD consists of a seven-station radiation monitoring network that has collected minutely solar radiation data since 2009 [29,87].

The Global Forecasting Service (GFS) is a weather forecast model that generates data for dozens of atmospheric and land-soil variables. GFS is a global model with a base horizontal resolution of 28 km between grid points. The forecast horizon is up to 16 days in advance, with a temporal resolution of 3 h [47,88].

A Sky Images and Photovoltaic Power Generation Dataset (SKIPP’D) contains three years (2017–2019) of quality-controlled downsampled minutely sky images and PV power generation data [31].

Photovoltaic Geographical Information System-Solar Air Heating 2 (PVGIS-SARAH-2) is derived based on the second version of the SARAH solar radiation data provided by the EUMETSAT Climate Monitoring Satellite Application Facility (CM SAF). The solar radiation data come in GeoTIFF [89] format and consist of the average irradiance over a specific time period, measured in W/m${}^2$ [82].

Some other possibly useful references can be the Goddard Earth Sciences Data and Information Services Center (GES-DISC) https://disc.gsfc.nasa.gov/ (last accessed on 18 September 2023), Meteotest https://meteotest.ch/en/ (last accessed on 18 September 2023) [43,47], Moderate Resolution Imaging Spectroradiometer (MODIS) of NASA’s Terra and Aqua satellites https://modis.gsfc.nasa.gov/data/dataprod/mod01.php (last accessed on 18 September 2023) [11] and the Finnish Meteorological Institute’s open-source datasets https://en.ilmatieteenlaitos.fi/open-data-sets-available (last accessed on 18 September 2023) [90].

4.2. Benchmark

In

Table 4. Benchmark.

Source/Ref.	Period (Training)	Testing	Goal	Horizon (Step)	Best Method	Error
NREL/[57]	(1 January 2015–31 December 2018)	1 January 2020–31 December 2020	Irradiance forecasting	17 h (1 h)	DRL + CNN-BiLSTM	RMSE = 80.02 W/m${}^2$ MAE = 51.95 W/m${}^2$ MAPE = 7.64%
NREL/[27]	(1 January 2012–31 December 2014)	1 January 2016–31 December 2017	Irradiance forecasting	1 h (10 min)	SolarNet	FS = 34.02 RMSE = 81.03 W/m${}^2$ MBE = −0.44 W/m${}^2$
NREL/[15]	(1 July 2017–19 April 2018)	25 May 2018–30 June 2018	Irradiance forecasting	15 min	CNN-LSTM CNN-ANN	$\rho$ = 0.97 RMSE = 80.48 W/m${}^2$ MAE = 51.89 W/m${}^2$
NREL/[33]	(1 January 2016–22 September 2018)	22 September 2018–31 December 2018	PV power forecasting	24 h (1 h)	LSTM-NN	MAE = 0.36 MW RMSE = 0.71 MW MAPE = 22.31% MRE = 1.44% MBE = 0.01 MW
NREL/[16]	1 July 2017–30 June 2018	5-fold CV∼70 days	Irradiance forecasting	15 min	CNN-QR LSTM-QR	MAE = 68.84 W/m${}^2$ RMSE = 98.94 W/m${}^2$ nPIAW = 0.09% PICP = 0.92% CWC = 0.16 $\rho$ = 0.96
NREL/[43]	(1 January 2006–31 August 2006)	1 September 2006–31 December 2006	PV power forecasting	6 h (15 min)	TSM-GAT	nMAE = 14.78% nRMSE = 10.37%
SKIPP’D/[31]	1 March 2017–1 December 2019	∼20 days (4% of data)	PV power forecasting	15 min (1 min)	ConvNet	FS = 16.44 RMSE = 0.0024 MW MAE = 0.0015 MW
SURFRAD/[29]	1 January 2015–31 December 2018	not specified	Irradiance forecasting	24 h (15 min)	Ineichen-Perez clear sky model	FS = 14.3 RMSE = 120 W/m${}^2$

are shown benchmarked results of the existing literature on open-access data. The idea is to enable novel research to compare the results against the existing ones on the identical, readily available datasets. Other datasets mentioned in Table 3 are left out due to the closed-source nature of the PV plant production data.

The column Period (Training) shows the temporal scale of the dataset used in the referenced paper. The dates in parentheses refer to the training part of the dataset.

The error measures RMSE, MAE and MBE are denoted in W/m${}^2$ (irradiance forecasting) or MW (PV power forecasting). The error measures MAPE, FS, MRE, nRMSE and nMAE are denoted in % deviation. We identified three sources available for benchmarking: NREL, SKIPP’D and SURFRAD. NREL is the most referenced dataset in the analysed literature. Nevertheless, no papers can be directly compared to each other as they have no overlapping test periods and have different forecast horizons or forecast steps for the NREL dataset. Despite that, a similarity becomes evident when comparing their common error measures. To be more specific, the RMSEs for [57], [27], [15] and [16] are 80.02, 81.03, 80.48 and 98.94 W/m${}^2$ for irradiance forecasting, respectively. Out of these results, we would emphasize the results obtained by Dolatabadi et al. [57] and Feng et al. [27], as their solutions yield robust results over longer testing periods of one year and two years, respectively. When benchmarking against these papers, future authors must have in mind the forecast horizon and the granularity of the forecast, as they differ for each paper encompassed in Table 4. We encourage future authors to use identical error measures to accurately benchmark models against existing ones. Regarding forecast skill (FS), new models must be compared against the same baseline models as in the referenced papers.

In [31] by Nie et al., the model was evaluated on only 4% of data (∼20 days) with a dataset containing almost three years of data (SKIPP’D). A more statistically robust approach would involve extending the testing period to cover a longer duration.

Finally, Yang et al. in [29] evaluated and compared different clear sky models for the SURFRAD dataset without specifying the testing period. We anticipate improvements to be made over the clear sky models by using a machine learning approach on the SURFRAD dataset.

5. Conclusions

In this paper, an overview of the state of the art of short-term PV forecasting using machine learning is provided. This review focused on the most recent relevant studies in the last 5 years with the highest number of citations. Key features of each relevant paper have been compiled and summarized in Table 1, including the goal of the paper, forecast horizon, ML methods employed, evaluation techniques, location of the power plants and finally the test size, as we have not seen this included in other similar review literature. A larger test size signifies a more reliable assessment of the method tested. Furthermore, the methods employed and evaluation measures used in the reviewed literature have been discussed. The use of deep learning in this field is on the upswing, as evident from analysed literature. Many of the papers describe the use of a hybrid deep learning model combining feedforward, convolutional or recurrent layers. Nevertheless, classical machine learning architectures and autoregressive methods continue to be relevant alongside more complex methods. We provide a discussion and offer recommendations on the methodology of building a solar forecasting model including evaluation measure and machine learning method selection. Machine learning methods are data-driven, and many data sources have been reported in the literature. Consequently, open-access data sources have been compiled and explored. These open-access data sources and papers utilizing them serve as a benchmark against which new research can be evaluated. Table 4 benchmarks the existing analysed literature on readily available datasets; we also provide suggestions on the steps to ensure proper comparison of new research against the benchmark.

Future work For future research directions, we encourage new authors to challenge the newly established benchmark with their innovative solutions. This serves to broaden and deepen the knowledge within the field of solar forecasting. Moreover, probabilistic solar forecasting is less represented in literature. We believe that this is a promising research direction as it can provide new solutions to the grid, such as probabilistic reserve markets. Additionally, a less-explored direction is the aspect of explainability when it comes to solar forecasters built on ML. This is important, as grid and plant operators often require full transparency in their decision-making process, which makes it worth pursuing.