A Comparative Study of Machine Learning-Based Methods for Global Horizontal Irradiance Forecasting

Abstract

The proliferation of photovoltaic (PV) power generation in power distribution grids induces increasing safety and service quality concerns for grid operators. The inherent variability, essentially due to meteorological conditions, of PV power generation affects the power grid reliability. In order to develop efficient monitoring and control schemes for distribution grids, reliable forecasting of the solar resource at several time horizons that are related to regulation, scheduling, dispatching, and unit commitment, is necessary. PV power generation forecasting can result from forecasting global horizontal irradiance (GHI), which is the total amount of shortwave radiation received from above by a surface horizontal to the ground. A comparative study of machine learning methods is given in this paper, with a focus on the most widely used: Gaussian process regression (GPR), support vector regression (SVR), and artificial neural networks (ANN). Two years of GHI data with a time step of 10 $\min$ are used to train the models and forecast GHI at varying time horizons, ranging from 10 $\min$ to 4 $\mathrm{h}$. Persistence on the clear-sky index, also known as scaled persistence model, is included in this paper as a reference model. Three criteria are used for in-depth performance estimation: normalized root mean square error (nRMSE), dynamic mean absolute error (DMAE) and coverage width-based criterion (CWC). Results confirm that machine learning-based methods outperform the scaled persistence model. The best-performing machine learning-based methods included in this comparative study are the long short-term memory (LSTM) neural network and the GPR model using a rational quadratic kernel with automatic relevance determination.

1. Introduction

In the past few years, the higher penetration of renewable energy sources, in particular solar photovoltaics, into power grids, has brought new challenges for grid operators [1,2,3,4,5]. As electricity is not easy to store, supply and demand have to be balanced at all times by grid operators. Nonetheless, due to the intermittent nature of the solar resource, the deployment of photovoltaic (PV) power generation makes the power grid balance more complex to ensure using standard tools [6]. Indeed, the proliferation of PV power generation in power distribution grids brings out constraints, in particular voltage constraints, mainly observed on the medium-voltage power distribution grid, on the low-voltage one. Ergo, evolution of the power distribution grid and smart management tools are necessary to alleviate these constraints. It seems necessary to develop new tools that must help to improve the grid observability and management to go along with the grid’s evolution. As a result, the development of predictive management strategies is a stepping stone to more efficient real-time monitoring and optimization of grid operation [6,7,8]. Therefore, tools that allow accurate forecasting of PV power generation at several time horizons are needed to achieve the power grid stability and reliability. Towards this same objective, in the context of the Smart Occitania project, PROMES-CNRS laboratory and ENEDIS (the French distribution grid operator) have been developing tools to improve the observability and regulation of the low-voltage distribution grid in the presence of PV power generation in Occitania (southern France). Intrahour, intraday and day-ahead PV power generation forecasts allow grid operators to make decisions related to real-time grid regulation, unit commitment, and control of electricity demand [9]. To reach the goals of the Smart Occitania project, this work will focus on short-term forecasting horizons, ranging from 10 $\min$ to 4 $\mathrm{h}$. Indeed, in order to perform a predictive management of power grids, which usually have fast dynamics, very short forecast horizons are needed.

The generated PV power can be deduced from forecasts of global horizontal irradiance (GHI), which is the total amount of shortwave radiation received from above by a horizontal surface on Earth. Hence, accurate forecasts of GHI at various time horizons are required for efficient development of grid-connected PV power systems. Reviewing the scientific literature, various solar irradiance forecasting methods have been developed (see e.g., [9,10,11,12,13,14,15]). These models are used to forecast GHI depending on the input data and forecast horizon. For the intrahour range, GHI forecasts with higher spatial and temporal resolution derived from cloud information from ground-based sky imagers are more accurate than the satellite-based forecasts [10,16,17]. For short-term horizons ranging from few minutes to 6 $\mathrm{h}$, statistical models with on-site GHI measurements are appropriate [10,11,12,13]. Satellite images, which provide information about cloud motion that can be extrapolated to the upcoming few hours, allow to have good forecasts for time horizons up to 6 $\mathrm{h}$ ahead [10,18]. However, the spatial extension of the monitored cloud scenes and corresponding cloud velocities limit the forecast horizons. Numerical weather prediction (NWP) models deliver more precise forecasts for time horizons from about 6 $\mathrm{h}$ onwards [9,10,19]. There are also combined or hybrid methods that integrate different kinds of input data and/or approaches to elaborate a high-performance forecasting model [20,21,22].

One drawback of statistical models is that they cannot account for dynamic phenomena like motion and formation of clouds that create sudden changes in the GHI signal [10]. Based on these effects, one might desire to use models to describe cloud motion and to derive irradiance from images provided by satellites or sky imagers. However, these models are complex and exhibit an inherent uncertainty related to limits in spatial and temporal resolution, uncertainty in input parameters, and simplifying assumptions within the models [10]. As a result, for short-term horizons, although dynamic phenomena may not be anticipated, statistical models developed are nonetheless used to provide forecasts. A simple way of using statistical approaches to forecast solar irradiance is to develop models that deliver forecasts based only on endogenous data (only GHI measurements). Statistical models can also be fed with exogenous input data such as NWP forecasts [20] or other data (direct horizontal irradiance, direct normal irradiance, dew point, temperature, humidity, wind direction) [23]. However, the development of statistical models using endogenous data is prevalent in the solar irradiance forecasting literature in a real-time operational context because they can provide accurate forecasts with limited investment and computational effort.

As the current paper deals with short-term GHI forecasting based on historical GHI data, the focus is put on statistical models. These models include classical time-series approaches such as the persistence model and autoregressive models, and artificial intelligence-based techniques such as regression trees, k-nearest neighbours (kNN), artificial neural networks (ANNs), support vector regression (SVR) and Gaussian process regression (GPR) [12,13,24,25,26,27,28]. Note that as most of classical time-series models need stationarity, the solar irradiance data, which are non-stationary, can be preprocessed using the clearness index [11] or the clear-sky index [29]. However, machine learning-based techniques can model non-stationary signals and are capable of capturing both the periodic component and the stochastic part of the GHI time-series. Additionally, in [30], it is argued that artificial intelligence-based techniques without any specific preprocessing of data outperform classical approaches with preprocessed data. Finally, the development of models free from using clear-sky models or other preprocessing steps implies that all errors come solely from the forecasting method. As a result, in this paper, GHI data without any specific preprocessing step will be used.

Machine learning methods have been increasingly used in recent years. In [31], the authors applied deep recurrent neural networks for solar irradiance forecasting and showed that these networks outperform SVR models and feedforward neural networks. In [23], long short-term memory (LSTM) neural networks were used for multi-step ahead forecasting of GHI. In [32], the authors developed a hybrid model using an autoregressive model and an ANN model for forecasting hourly solar radiation in the Mediterranean area. One-hour ahead solar irradiance were predicted using support vector machines (SVMs) in [33]. In [34], a deep convolutional neural network (CNN) model has been developed for hourly GHI forecasting based only on sky images without numerical measurements and extra feature engineering. A hybrid CNN with a LSTM neural network for forecasting half-hourly solar radiation has been proposed in [35]. This hybrid model has been compared to other deep learning models and results show that the hybrid model outperformed its counterparts. The potential of GPR for GHI forecasting has been investigated in [24,25]. In [24], the authors have made a comparative study of online GPR and online sparse GPR models based on simple kernels or combined kernels defined as sums or products of simple kernels and the results have shown the superiority of quasiperiodic kernels-based GPR models over the classic persistence model as well as simple kernels-based GPR models.

Based on their proven good performance, popularity and potential in providing accurate forecasts, it yields that machine learning methods such as ANN, SVR and GPR are well-suited for GHI forecasting. The present paper focuses on the development and comparison of intrahour and intraday machine learning-based GHI forecasting models. Even though several such comparative analyses exist in the literature (

Table 1. Main recent comparative studies using machine learning methods. For the meaning of abbreviations, see at the end of the paper.

Authors	Forecast Horizons	Forecasting Methods	Data Time Step	Performance Criteria	Input Variables	Output Variables	Database
Sharifzadeh et al. [28]	$1 \mathrm{h}$ to $6 \mathrm{h}$	ANN, SVR, GPR	$1 \mathrm{h}$	MSE	PV power, temperature, DHI, DNI	PV power	From 1985 to 2014
Benali et al. [26]	$1 \mathrm{h}$ to $6 \mathrm{h}$	Scaled persistence, MLP, Random forest	$1 \mathrm{h}$	RMSE, nRMSE, MAE, nMAE	GHI, DNI, DHI	GHI, DNI, DHI	Data covering three years
Chandola et al. [23]	$3 \mathrm{h}$ to $24 \mathrm{h}$	LSTM	$3 \mathrm{h}$	RMSE, MAPE	DHI, DNI, dew point, temperature, pressure, relative humidity, wind direction, wind speed	GHI	From 2010 to 2014
Lauret et al. [13]	$1 \mathrm{h}$ to $6 \mathrm{h}$	Persistence, scaled persistence, AR, MLP, SVR, GPR	$1 \mathrm{h}$	nRMSE, nMAE, nMBE, s-skill score	Clear sky index	Clear sky index	Three sites; for each site, one-year data for training and one-year data for test
Tolba et al. [24]	$30 \min$ to $48 \mathrm{h}$	Persistence, GPR	$30 \min$	nRMSE	Time	GHI	Two datasets, each covering a period of 45 days; 30 days for training and 15 days for test
Gbémou et al. (present paper)	$10 \min$ to $4 \mathrm{h}$	Scaled persistence, MLP, LSTM, LS-SVR, GPR	$10 \min$	nRMSE, DMAE, CWC	GHI	GHI	One-year data for training and one-year data for test

offers a comparison between the work presented in this paper and recent comparative studies), several questions still remain to be answered.

• Most studies use databases having at least a 1-hour time step, which leads to the impossibility of intrahour forecasting and to significant simplification of GHI dynamics, as illustrated in Figure 1.
• The methods themselves are not always optimized and used to their fullest extent. For example, when using GPR, the default kernel (i.e., the squared exponential) is usually used [13,36]; however, a kernel tailored to the application at hand has a significant influence on results [24].
• Regarding input data: some studies use only endogenous data, while others use additional data, which prevents from making fair comparisons between methods.
• Some authors forecast GHI directly, others the clear-sky index (using clear-sky models as pre- and postprocessing steps) or even PV power generation.

As a result, despite the extent of research on GHI forecasting in the scientific literature, a thorough comparative study of machine learning-based methods using endogenous data only, without any preprocessing step, for intrahour GHI forecasting is, to the best of the authors’ knowledge, inexistent.

The main contributions of the present paper are threefold.

• The models are developed using a two-year GHI database with a 10 $\min$ time step. As can be seen in Table 1, in previous machine learning studies the time step is usually 1 $\mathrm{h}$. However, such a time step leads to significant simplification of GHI dynamics: as can be noticed in Figure 1, GHI data sampled with 10 $\min$ time step exhibit more fluctuations and are thus more difficult to forecast.
• Contrary to developing a specific model for each forecast horizon, as shown in some studies in the literature [13,28], we made the choice of multi-horizon forecasting models. Developing a specific model for each forecast horizon can be computationally demanding when many horizons are considered and it would be more practical to use a multi-horizon forecasting model when trying to run the algorithms in situ to produce real-time forecasts at various horizons, especially when intrahour forecasts are needed. Therefore, in this paper, the models are developed for multi-step ahead GHI forecasting and once the training phase is over, the models are used to forecast GHI for all horizons.
• Besides, many authors generally choose classical performance criteria (nRMSE, MAE, MBE, MAPE) for their models’ evaluation. In the present paper, two criteria are used in addition to the nRMSE: DMAE, that accounts for temporal distortion error and absolute magnitude error simultaneously; and CWC, that assesses the quality of prediction intervals. These criteria provide more detailed and comprehensive information about the models’ performance, and allow an in-depth analysis of their forecasts.

In this paper, the same data are used for each model training and tests are performed on the same dataset to make a fair comparison and a thorough analysis of the results. When using GPR, kernels with automatic relevance determination (ARD), such as the squared exponential kernel with ARD ($k_{\mathrm{SE}\text{-}\mathrm{ARD}}$) and the rational quadratic kernel with ARD ($k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$), are chosen to account for the relevance of each input dimension in the underlying function modelling. It is a good option, when using GPR for GHI forecasting, as the underlying function has a multi-dimensional input variable, to use ARD kernels that implicitly determine the relevance of each input dimension [37]. That is why we have decided for such kernels. Nonetheless, the flexibility of ARD kernels means that they can be relatively slow to learn and, as a result, authors generally choose simple kernels with isotropic correlation length parameter [13]. The ANN models developed in this paper are based on MLP (multilayer perceptron) and LSTM neural networks. These artificial neural networks are widely used for time series forecasting.

The rest of the paper is organized as follows: in Section 2, the data used to develop and validate the models included in the comparative study are described. Section 3 provides a description of the scaled persistence model and the machine learning methods (GPR, SVR and ANN) used to forecast GHI and presents the models’ structure. The forecasting results as well as the criteria used to assess the models’ performance (i.e., forecasting accuracy) are presented and discussed in Section 4. The paper ends with a conclusion.

2. Data Description

The dataset is derived from measurements taken at PROMES-CNRS laboratory, located in Font-Romeu-Odeillo-Via (southern France) in the Occitania region. Located at high altitude, the site is characterized by a relatively dry mountain climate, with hot summers and icy winters. Besides, there is also occasional severe thunderstorms that fall on the region. GHI data are collected using a pyranometer KIPP and ZONEN CM5.

The dataset used in this paper covered the years 2018 and 2019. The data were sampled with a time step $\Delta t=10\min$, resulting in around 26,600 points per year (after nights were removed). Data of year 2018 were used for training the models and 2019’s data were used for the test phase. The aim behind such a partition was to train the models with data providing exhaustive information (this enabled the models to learn all possible patterns in one-year GHI data) and to assess their ability to provide GHI forecasts in all atmospheric conditions, whatever the season. The min-max normalization was used for data preprocessing to facilitate the learning process of the models. Figure 2 depicts a map of data covering the two years. Additionally, it can be noted that there were some missing values in the database, amounting to 0.64% of the whole database. As the missing values represented an insignificant part of the data, there was no need for a sophisticated way to handle them. They were removed from the dataset. Moreover, the models trained with data after the removal of missing values were often robust.

3. Forecasting Methods Included in the Comparative Study

This section presents the scaled persistence model and the three different machine learning-based methods included in the comparative study: Gaussian process regression (GPR), support vector regression (SVR) and artificial neural networks (ANNs). For the machine learning-based methods’ description, we considered a training set $\mathcal{D}=\{({\mathit{x}}_i,y_i)$, $1⩽i⩽n\}$, where ${\mathit{x}}_i\in {\mathbb{R}}^D$ is a vector of past GHI values, $y_i\in \mathbb{R}$ is the corresponding one-step ahead GHI forecasting value and n is the number of training samples. Let us aggregate all input vectors ${\mathit{x}}_i$ in a matrix $\mathit{X}\in {\mathbb{R}}^{n\times D}$ and note $\mathit{y}\in {\mathbb{R}}^n$ the vector of corresponding outputs $y_i$. The models were implemented on a machine having Intel® Xeon® CPU E7-4890 [email protected] as the CPU. The GPR models were implemented using the Gaussian Processes for Machine Learning Matlab Toolbox [38], the LS-SVR model was implemented using the LS-SVM Matlab Toolbox [39] and the ANN models were implemented using the Deep Leaning Matlab Toolbox.

This paper focuses on the multi-step ahead forecasting of GHI, which was achieved by iterating one-step ahead forecasting: this strategy involved using the estimate of the output of the current forecasting as well as previous outputs (up to the lag D) as the input to forecast GHI at the next time step and repeating this procedure until the forecast at the desired forecast horizon was obtained. During the training phase, each sample was used. Throughout the testing phase, the models’ parameters were updated each time step to forecast the one-step ahead GHI and successive one-step ahead forecasts were performed to reach the desired forecast horizon.

3.1. Scaled Persistence Model

The scaled persistence model, used in this study as a reference model, is given by:

$$GHI(t+k\Delta t)=\frac{GHI\left(t\right)}{{GHI}_{\mathrm{clsk}}\left(t\right)}·{GHI}_{\mathrm{clsk}}(t+k\Delta t)$$

(1)

where GHI is the measured global horizontal irradiance, ${GHI}_{\mathrm{clsk}}$ is the clear-sky model’s output, $\Delta t$ refers to the time step (10 $\min$) and $k\Delta t$ is the forecast horizon.

Clear-sky models estimate ground level irradiance under a clear sky as a function of the solar elevation angle, site altitude, aerosol concentration, water vapour and various atmospheric parameters. The scientific literature exhibits a wide variety of clear-sky models that are reviewed in [40]. The approach proposed in [41] for clear-sky DNI models, which combines Ineichen and Perez’s clear-sky model [42] with a persistence of atmospheric turbidity, is modified to obtain a clear-sky GHI model. The clear-sky global horizontal irradiance is given by:

$$\begin{array}{cc}\hfill {GHI}_{\mathrm{clsk}}& =c_{g1}·I_0·cos\left(z\right)·exp\left(-c_{g2}·\mathrm{AM}·\left(f_{h1}+f_{h2}(T_L-1)\right)\right)·exp\left(0.01·{\mathrm{AM}}^{1.8}\right)\hfill \end{array}$$

(2)

where $I_0$ is the extraterrestrial irradiance, z is the solar zenith angle, $\mathrm{AM}$ is the optical air mass, $T_L$ is the Linke turbidity coefficient and $f_{h1}$, $f_{h2}$, $c_{g1}$, and $c_{g2}$ are parameters.

The Linke turbidity coefficient is defined as the number of dry atmospheres necessary to produce the attenuation of the extraterrestrial irradiance produced by the atmosphere.

The parameters $f_{h1}$, $f_{h2}$, $c_{g1}$ and $c_{g2}$ are respectively given by:

$$\begin{array}{cc}\hfill f_{h1}& =exp\left(-\frac{h}{8000}\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill f_{h2}& =exp\left(-\frac{h}{1250}\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill c_{g1}& =5.09\times {10}^{-5}h+0.868\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill c_{g2}& =3.92\times {10}^{-5}h+0.0387\hfill \end{array}$$

(6)

where h is the elevation.

The optical air mass (AM) is calculated by Kasten and Young’s formula [43]:

$$\begin{array}{c}\hfill \mathrm{AM}=\frac{1}{cos\left(z\right)+0.50572{(96.07995-z)}^{-1.6364}}\end{array}$$

(7)

3.2. Gaussian Processes

A Gaussian process (GP) is a collection of random variables, any finite number of which have a joint Gaussian distribution [38]. As a Gaussian distribution is determined by a mean vector and a covariance matrix, in a similar manner a GP is also fully defined by a mean function and a covariance function. A function f distributed as a Gaussian process is denoted as $f\left(\mathit{x}\right)\sim \mathcal{GP}(\mu \left(\mathit{x}\right),k(\mathit{x},{\mathit{x}}^{\prime }))$, where $\mathit{x}$ and ${\mathit{x}}^{\prime }$ are arbitrary input variables, $\mu \left(\mathit{x}\right)=\mathbb{E}\left[f\right(\mathit{x}\left)\right]$ is the mean function and $k(\mathit{x},{\mathit{x}}^{\prime })=\mathbb{E}\left[(f\left(\mathit{x}\right)-\mu \left(\mathit{x}\right)){(f\left({\mathit{x}}^{\prime }\right)-\mu \left({\mathit{x}}^{\prime }\right))}^{\mathsf{T}}\right]$ is the covariance function, also called kernel.

3.2.1. Kernels Used in This Study

Kernels contain the assumptions about the underlying function to be learned and define the similarity between the function values at different input variables $\mathit{x}$ and ${\mathit{x}}^{\prime }$ [38,44]. It yields that the choice of kernel has a crucial impact on the performance of a GPR model [24]. Any function can be a covariance function as long as the covariance matrix obtained is positive semi-definite.

The kernel is a crucial part of GPR models and should be carefully chosen. For GHI forecasting by GPR, many authors have chosen kernels with isotropic correlation length parameter [13,36]. However, as the underlying function has a multi-dimensional input variable and doesn’t have the same variation level along each input dimension, it would be better to use kernels that define a specific and suitable length-scale for each input dimension. That is why Automatic Relevance Determination (ARD) kernels, which implicitly determine the relevance of each input dimension, are used to model functions that have multi-dimensional input [37].

Furthermore, additive structures can be developed for ARD kernels to capture the correlations between the input dimensions in order to improve the generalization ability of models for long-term horizons [45]. ARD kernels used in this work are described below. It is worth noting that several ARD kernels have been tested to identify the best-performing ones.

• The SE-ARD kernel $k_{\mathrm{SE}\text{-}\mathrm{ARD}}$, expressed as a product of squared exponential (SE) kernels over input dimensions each having a different length-scale, is given by:

  $$\begin{array}{c}\hfill k_{\mathrm{SE}\text{-}\mathrm{ARD}}(\mathit{x},{\mathit{x}}^{\prime })={\sigma }^{2}exp\left(-\frac{1}{2}\sum _{d=1}^D\frac{{(x_d-x_d^{\prime })}^2}{{\ell }_d^2}\right)\end{array}$$

  (8)

  where $\sigma >0$ is the amplitude and ${\ell }_1,{\ell }_2,\dots ,{\ell }_D>0$ are the length-scales which control the function’s variation along each input dimension. This kernel is a covariance function commonly chosen as default in Gaussian processes applications, because it has relatively few and easy-interpretable parameters to estimate. Moreover, it can be seen as an universal kernel, capable of learning any continuous function given enough data, under some conditions that are investigated in [46].
• The RQ-ARD (rational quadratic) kernel $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ is given by:

  $$\begin{array}{c}\hfill k_{\mathrm{RQ}\text{-}\mathrm{ARD}}(\mathit{x},{\mathit{x}}^{\prime })={\sigma }^2{\left(1+\frac{1}{2\alpha }\sum _{d=1}^D\frac{{(x_d-x_d^{\prime })}^2}{{\ell }_d^2}\right)}^{-\alpha }\end{array}$$

  (9)

  where $\alpha >0$ characterizes the relative weighting of large-scale and small-scale variations. The RQ kernel can be seen as an infinite sum of SE kernels with different characteristic length-scales [38] and models functions that vary smoothly across many length-scales. Analogously, the RQ-ARD kernel, which allows the modelling of functions that exhibit multi-scale variations along each input dimension, can be seen as an infinite sum of SE-ARD kernels.

3.2.2. Gaussian Process Regression

Consider the standard regression model:

$$\begin{array}{c}\hfill \mathit{y}=f\left(\mathit{X}\right)+\mathit{\epsilon }\end{array}$$

(10)

where f is the regression function and $\mathit{\epsilon }\sim \mathcal{N}(0,{\sigma }_{\epsilon }^2)$ is an independent identically distributed Gaussian noise.

GPR is a Bayesian non-parametric approach [38] towards regression problems, which consists in approximating $f\left(\mathit{x}\right)\sim \mathcal{GP}(\mu \left(\mathit{x}\right),k(\mathit{x},{\mathit{x}}^{\prime }))$ using a training set of n observations $\mathcal{D}$. Consider ${\mathit{X}}_{*}$ a matrix of test input vectors. Assuming $f\left(\mathit{x}\right)\sim \mathcal{GP}(\mu \left(\mathit{x}\right),k(\mathit{x},{\mathit{x}}^{\prime }))$ and $\mathit{\epsilon }\sim \mathcal{N}(0,{\sigma }_{\epsilon }^2)$, the training observations $\mathit{y}$ and the function values ${\mathit{f}}_{*}$ for the test inputs ${\mathit{X}}_{*}$ follow a joint Gaussian distribution. Thus, the posterior predictive density is also Gaussian [38]:

$$\begin{array}{c}\hfill {\mathit{f}}_{*}|\mathit{X},\mathit{y},{\mathit{X}}_{*}\sim \mathcal{N}({\mathit{\mu }}_{*},{\mathit{\sigma }}_{*}^2)\end{array}$$

(11)

where the mean prediction ${\mathit{\mu }}_{*}$ and the predictive variance ${\mathit{\sigma }}_{*}^2$ are respectively given by:

$$\begin{array}{c}\hfill {\mathit{\mu }}_{*}=\mu \left({\mathit{X}}_{*}\right)+{\mathit{K}}_{*}^{\mathsf{T}}{(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I})}^{-1}(\mathit{y}-\mu \left(\mathit{X}\right))\end{array}$$

(12)

and:

$$\begin{array}{c}\hfill {\mathit{\sigma }}_{*}^2={\mathit{K}}_{**}-{\mathit{K}}_{*}^{\mathsf{T}}{(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I})}^{-1}{\mathit{K}}_{*}\end{array}$$

(13)

with ${\mathit{f}}_{*}=f\left({\mathit{X}}_{*}\right)$, ${\mathit{K}}_{*}=k(\mathit{X},{\mathit{X}}_{*})$, ${\mathit{K}}_{**}=k({\mathit{X}}_{*},{\mathit{X}}_{*})$, and $\mathit{K}=k(\mathit{X},\mathit{X})$.

In the event of a single test input vector ${\mathit{x}}_{*}$, the mean prediction (Equation (12)) becomes:

$$\begin{array}{cc}\hfill {\mathit{\mu }}_{*}& =\mu \left({\mathit{x}}_{*}\right)+{\mathit{k}}_{*}^{\mathsf{T}}{(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I})}^{-1}(\mathit{y}-\mu \left(\mathit{X}\right))\hfill \end{array}$$

(14)

where ${\mathit{k}}_{*}={\left[\begin{array}{cccc}k({\mathit{x}}_1,{\mathit{x}}_{*})& k({\mathit{x}}_2,{\mathit{x}}_{*})& \dots & k({\mathit{x}}_n,{\mathit{x}}_{*})\end{array}\right]}^{\mathsf{T}}$.

Thus the mean prediction ${\mathit{\mu }}_{*}$ (Equation (14)) can be written as a linear combination of kernel functions, each one centred on a training point:

$$\begin{array}{c}\hfill {\mathit{\mu }}_{*}=\mu \left({\mathit{x}}_{*}\right)+\sum _{i=1}^n{\alpha }_{i}k({\mathit{x}}_i,{\mathit{x}}_{*})\end{array}$$

(15)

where $\mathit{\alpha }={(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I})}^{-1}(\mathit{y}-\mu \left(\mathit{X}\right))$.

The coefficients ${\alpha }_i$, also called parameters, are updated each time a new observation is added to the observation set contrary to the parameters of the kernel, referred to as hyperparameters, which do not change once training is over. Graphical views of GHI forecasting by the $k_{\mathrm{SE}\text{-}\mathrm{ARD}}$-based and $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR models are depicted by Figure 3 and Figure 4.

3.2.3. Training a GPR Model

The hyperparameters $\mathit{\theta }$ of a GPR model, which group the parameters involved in the kernel and the noise variance, have to be inferred from the training data. In practice, they are commonly estimated by maximizing the log marginal likelihood given by:

$$\begin{array}{c}\hfill \mathcal{L}\left(\mathit{\theta }\right)=-\frac{1}{2}\left({\mathit{y}}^{\mathsf{T}}{\left(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I}\right)}^{-1}\mathit{y}+log\left(det\left(\mathit{K}+{\sigma }_{\epsilon }^2\mathit{I}\right)\right)+nlog\left(2\pi \right)\right)\end{array}$$

(16)

Unfortunately, the log marginal likelihood function $\mathcal{L}\left(\mathit{\theta }\right)$ is usually non-convex with respect to the hyperparameters. The problem space can have many local optima and the optimized hyperparameters, which depend on their initialization, may not be the global optimum [37,47]. A classical approach to overcome this drawback is to use various starting points randomly selected from a suitable prior distribution. A study on the impact of prior distributions of the initial hyperparameters in GPR models has been conducted in [48] and the authors have arrived at the conclusion that simple priors such as the uniform distribution in an appropriate range may be sufficient in GPR modelling in terms of predictability.

For the initialization of the hyperparameters $\mathit{\theta }$, the following choices have been made: the hyperparameter ${\sigma }^2$ was chosen to be equal to the variance of the training data and the remaining hyperparameters are randomly drawn from an uniform distribution ${\theta }_i\sim \mathrm{Uniform}(0,1)$.

3.3. Support Vector Machines

Support vector machines (SVM), which have been developed in the past few decades [49,50,51,52,53], are learning systems that map inputs into a high dimensional feature space where linear separation or regression becomes much easier. The SVM algorithm, which is a nonlinear generalization of the method developed in [54], is based on a learning algorithm from optimization theory grounded in the statistical learning theory framework [49,55]. Firstly, SVM were developed for pattern recognition [52], but they also exhibit good performance in time series modelling and have been successfully applied to regression problems and time series forecasting [51,53].

3.3.1. Support Vector Regression

Support vector regression (SVR) maps the input $\mathit{X}$ into a high dimensional feature space F through a nonlinear mapping function $\varphi$ in order to do a linear regression in the feature space. We construct a linear regression function f in the feature space F:

$$\begin{array}{c}\hfill f\left(\mathit{X}\right)={\mathit{w}}^{\mathsf{T}}\varphi \left(\mathit{X}\right)+b\end{array}$$

(17)

where $\mathit{w}$ and b are obtained by solving a convex optimization problem.

In [49], SVR is first described as $\epsilon$-SV regression, which aims to find function f that is as flat as possible and concomitantly has at most $\epsilon$ error from the targets $y_i$ for all training data. This problem can be formulated as a convex optimization problem [53]. Furthermore, slack variables ${\xi }_i$, ${\xi }_i^{*}$ are introduced to deal with unfeasible constraints of the optimization problem [49].

Equations (18) and (19) describe the convex optimization problem as follows:

$$min\frac{1}{2}{\mathit{w}}^{\mathsf{T}}\mathit{w}+C\sum _{i=1}^n({\xi }_i+{\xi }_i^{*})$$

(18)

subject to, $\forall i$:

$$\begin{array}{l}\hfill \left\{\begin{array}{cc}\hfill & y_i-({\mathit{w}}^{\mathsf{T}}\varphi \left({\mathit{x}}_i\right)+b)⩽\epsilon +{\xi }_i\hfill \\ \hfill & ({\mathit{w}}^{\mathsf{T}}\varphi \left({\mathit{x}}_i\right)+b)-y_i⩽\epsilon +{\xi }_i^{*}\hfill \\ \hfill & {\xi }_i,{\xi }_i^{*}⩾0\hfill \end{array}\right.\end{array}$$

(19)

where $C>0$ determines the trade-off between the flatness of f and the amount up to which deviations larger than $\epsilon$ are tolerated.

It is shown in [53] that the optimization problem can be solved more easily in its dual formulation using Lagrange multipliers. The dual form of this problem is:

$$\begin{array}{c}\hfill min\frac{1}{2}\sum _{i=1}^n\sum _{j=1}^n({\alpha }_i-{\alpha }_i^{*})({\alpha }_j-{\alpha }_j^{*})k({\mathit{x}}_i,{\mathit{x}}_j)+\epsilon \sum _{i=1}^n({\alpha }_i+{\alpha }_i^{*})-\sum _{i=1}^n{y}_i({\alpha }_i-{\alpha }_i^{*})\end{array}$$

(20)

subject to, $\forall i$:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}\hfill & 0⩽{\alpha }_i,{\alpha }_i^{*}⩽C\hfill \\ \hfill & \sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})=0\hfill \end{array}\right.\end{array}$$

(21)

where ${\alpha }_i$, ${\alpha }_i^{*}$ are Lagrange multipliers and k is a kernel function, defined as:

$$\begin{array}{c}\hfill k(\mathit{x},{\mathit{x}}^{\prime })={\varphi }^{\mathsf{T}}\left(\mathit{x}\right)\varphi \left({\mathit{x}}^{\prime }\right)\end{array}$$

(22)

The weight $\mathit{w}$ can be written as:

$$\begin{array}{c}\hfill \mathit{w}=\sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})\varphi \left(x_i\right)\end{array}$$

(23)

Thus the regression function is given by:

$$\begin{array}{c}\hfill f\left(\mathit{X}\right)=\sum _{i=1}^n({\alpha }_i-{\alpha }_i^{*})k({\mathit{x}}_i,\mathit{X})+b\end{array}$$

(24)

The learning algorithms of support vector regression need to solve the optimization problem (Equations (18) and (19)), which becomes more complex and requires much computing time when a large dataset is considered. In order to reduce the algorithmic complexity, an equivalent least squares support vector regression (LS-SVR) is introduced [56]. In LS-SVR, the inequality constrains are replaced by equality ones, significantly reducing computation time.

3.3.2. Least Squares Support Vector Regression

The optimization problem for LS-SVR can be described as:

$$\begin{array}{c}\hfill \underset{\mathit{w},b,\mathit{e}}{min}J(\mathit{w},\mathit{e})=\frac{1}{2}{\mathit{w}}^{\mathsf{T}}\mathit{w}+\frac{1}{2}\gamma \sum _{i=1}^n{e}_i^2\end{array}$$

(25)

where J is the loss function and $\gamma$ is the adjustable constant, subject to, $\forall i$:

$$\begin{array}{c}\hfill y_i={\mathit{w}}^{\mathsf{T}}\varphi \left({\mathit{x}}_i\right)+b+e_i\end{array}$$

(26)

where $\varphi$ is the nonlinear mapping function in kernel space, b is the bias term and $e_i$ is the error variable.

The Lagrangian function of the optimization problem is:

$$\begin{array}{c}\hfill L=J(\mathit{w},\mathit{e})-\sum _{i=1}^n{\alpha }_i[{\mathit{w}}^{\mathsf{T}}\varphi \left({\mathit{x}}_i\right)+b+e_i-y_i]\end{array}$$

(27)

where ${\alpha }_i$ is Lagrange multiplier. The partial derivatives of L are:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}\hfill \frac{\partial L}{\partial \mathit{w}}& =0\Rightarrow \mathit{w}=\sum _{i=1}^n{\alpha }_i\varphi \left({\mathit{x}}_i\right)\hfill \\ \hfill \frac{\partial L}{\partial b}& =0\Rightarrow \sum _{i=1}^n{\alpha }_i=0\hfill \\ \hfill \frac{\partial L}{\partial e_i}& =0\Rightarrow {\alpha }_i=\gamma e_i\hfill \\ \hfill \frac{\partial L}{\partial {\alpha }_i}& =0\Rightarrow {\mathit{w}}^{\mathsf{T}}\varphi \left({\mathit{x}}_i\right)+b+e_i-y_i=0\hfill \end{array}\right.\end{array}$$

(28)

After eliminating w and $e_i$ from Equation (28), we can write the problem in matrix form:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}0& {\mathrm{𝟙}}_n^{\mathsf{T}}\\ {\mathrm{𝟙}}_n& \mathit{K}+\frac{1}{\gamma }\mathit{I}\end{array}\right]\left[\begin{array}{c}b\\ \mathit{\alpha }\end{array}\right]=\left[\begin{array}{c}0\\ \mathit{y}\end{array}\right]\end{array}$$

(29)

where:

$$\begin{array}{c}\hfill \mathit{K}(i,j)=k({\mathit{x}}_i,{\mathit{x}}_j)=\varphi {\left({\mathit{x}}_i\right)}^{\mathsf{T}}\varphi \left({\mathit{x}}_j\right)\end{array}$$

(30)

Then the function estimation of LS-SVR is:

$$\begin{array}{c}\hfill \mathit{y}=f\left(\mathit{X}\right)=\sum _{i=1}^n{\alpha }_{i}k(\mathit{X},x_i)+b\end{array}$$

(31)

Among the kernels that can be used in support vector regression (linear, polynomial and radial basis function kernels), we have decided for the radial basis function (RBF) kernel to model and forecast GHI (Figure 5). This kernel, which is more flexible than the linear and polynomial kernels, is capable of representing complex relationship between data and is well-suited for nonlinear modelling. The RBF kernel is given by:

$$\begin{array}{c}\hfill k({\mathit{x}}_i,{\mathit{x}}_j)=exp\left(-\frac{{\parallel {\mathit{x}}_i-{\mathit{x}}_j\parallel }^2}{2{\delta }^2}\right)\end{array}$$

(32)

where $\delta$ is the length-scale. Herein, $\delta$ and $\gamma$ are estimated using the gridsearch method.

3.4. Artificial Neural Networks

Artificial neural networks are nonlinear data-driven self-adaptive approaches widely used for time series forecasting [57], in particular for solar resource forecasting [13,26,58,59]. Indeed, ANN, which are inspired by the neural structure of the human brain, are powerful tools for regression. They can process problems involving nonlinear and complex data even if these data are imprecise and noisy. An artificial neural network is fully defined by its architecture (or topology), its parameters (named weights, or synaptic weights, and biases) and the algorithm used for training.

While feedforward neural networks are the most widely used for time series forecasting tasks, recurrent neural networks are also extensively used now due to their ability to learn dependencies between time steps of data [60]. Convolutional neural networks, often used for image classification and computer vision problems, have the ability to learn and automatically extract features from raw input data [60]. Thus, they can also be applied to time-series forecasting tasks. The capabilities of these ANN types can be combined to build hybrid models, such as CNN associated with long short-term memory neural networks that seek to harness the ability of each model type [35].

In the present work, a multilayer perceptron, i.e., a feedforward neural network, and a LSTM neural network will be used to forecast GHI. Both networks are described in the sequel. MLP and LSTM neural network have been broadly used for time-series forecasting and have proven good performances [57,61]. Despite their enhanced feature learning ability, convolutional neural networks cannot learn temporal dependencies in data. Nevertheless, the use of CNN and hybrid models seems to be promising in the application of ANN to time-series forecasting. As this paper doesn’t aim to provide an exhaustive study about ANN, CNN and hybrid models are not considered for GHI forecasting.

3.4.1. Multilayer Perceptron

The multilayer perceptron is a feedforward artificial neural network historically used for time series forecasting. In a MLP with ℓ hidden layers, there is no feedback and the information flows through connecting pathways, in only one direction, from the input layer to the output layer. For the input layer, $h^0\left(\mathit{x}\right)=\mathit{x}$, where $\mathit{x}$ is an input vector. For the network’s hidden layers, with $L=1,\cdots ,\ell$:

$$\begin{array}{c}\hfill a^L\left(\mathit{x}\right)={\mathit{b}}^L+{\mathit{W}}^L{h}^{L-1}\left(\mathit{x}\right)\end{array}$$

(33)

and:

$$\begin{array}{c}\hfill h^L\left(\mathit{x}\right)=\varphi \left(a^L\left(\mathit{x}\right)\right)\end{array}$$

(34)

where ${\mathit{b}}^L$ is the bias vector of layer L, ${\mathit{W}}^L$ is the weight matrix of layer L and $\varphi$ is the hidden neurons’ activation function.

The sigmoid function was historically the most widely used activation function in hidden layers since it is differentiable and enables to keep values in the interval $\left[0,1\right]$. However, using this function is problematic since its gradient is very close to 0 when $\left|\mathit{x}\right|$ is not close to 0. Because this can cause trouble during training, in particular with deep architectures, the sigmoid function was supplanted by the rectified linear unit (or ReLU). As an interesting feature, the ReLU function, which is not differentiable in 0, has a sparsification effect. It is used in our network. So:

$$\begin{array}{c}\hfill \varphi \left(a^L\left(\mathit{x}\right)\right)=max(0,a^L\left(\mathit{x}\right))\end{array}$$

(35)

For the network’s output layer, with $L=\ell +1$:

$$\begin{array}{c}\hfill a^{\ell +1}\left(\mathit{x}\right)={\mathit{b}}^{\ell +1}+{\mathit{W}}^{\ell +1}{h}^{\ell }\left(\mathit{x}\right)\end{array}$$

(36)

and:

$$\begin{array}{c}\hfill h^{\ell +1}\left(\mathit{x}\right)=\psi \left(a^{\ell +1}\left(\mathit{x}\right)\right)\end{array}$$

(37)

where ${\mathit{b}}^{\ell +1}$ is the bias vector of layer $\ell +1$, ${\mathit{W}}^{\ell +1}$ is the weight matrix of layer $\ell +1$ and $\psi$ is the output neurons’ activation function (which is linear).

Figure 6 shows a graphical view of the MLP model’s architecture for GHI forecasting.

3.4.2. Long Short-Term Memory

LSTM networks, introduced by [62], are recurrent neural networks developed to overcome the gradient vanishing problem that occurs when training traditional recurrent neural networks [63,64]. LSTM networks are capable of learning sequences of observations. This may make them neural networks well suited for forecasting. Initially, an LSTM unit was composed of a cell, an input gate, and an output gate. Later on, the forget gate has been introduced to allow the LSTM to reset its own state [65]. Let us denote by i, f, g, o the input gate, forget gate, input node, and output gate respectively. These gates and nodes are shortly described below. The input node, the input gate and the forget gate are respectively defined as follows:

$$\begin{array}{cc}\hfill {\mathit{g}}^k& =\varphi ({\mathit{W}}^{gx}{\mathit{x}}^k+{\mathit{W}}^{gh}{\mathit{h}}^{k-1}+{\mathit{b}}_g)\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill {\mathit{i}}^k& =\sigma ({\mathit{W}}^{ix}{\mathit{x}}^k+{\mathit{W}}^{ih}{\mathit{h}}^{k-1}+{\mathit{b}}_i)\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill {\mathit{f}}^k& =\sigma ({\mathit{W}}^{fx}{\mathit{x}}^k+{\mathit{W}}^{fh}{\mathit{h}}^{k-1}+{\mathit{b}}_f)\hfill \end{array}$$

(40)

where ${\mathit{x}}^k$ is the current input vector, ${\mathit{h}}^{k-1}$ is the hidden state at the previous time step, ${\mathit{W}}^{gx}$, ${\mathit{W}}^{ix}$, ${\mathit{W}}^{fx}$ are input weights, ${\mathit{W}}^{gh}$, ${\mathit{W}}^{ih}$, ${\mathit{W}}^{fh}$ are recurrent weights, ${\mathit{b}}_g$, ${\mathit{b}}_i$, ${\mathit{b}}_f$ are biases, $\sigma$ is the sigmoid function, and $\varphi$ is the hyperbolic tangent function.

At the heart of each memory cell, there is a node with linear activation. This internal state has a self-connected recurrent edge, often called the constant error carousel, with the following unit weight:

$$\begin{array}{c}\hfill {\mathit{s}}^k={\mathit{g}}^k\odot {\mathit{i}}^k+{\mathit{s}}^{k-1}\odot {\mathit{f}}^k\end{array}$$

(41)

where ⊙ is the Hadamard product.

The output gate is described by:

$$\begin{array}{cc}\hfill {\mathit{o}}^k& =\sigma ({\mathit{W}}^{ox}{\mathit{x}}^k+{\mathit{W}}^{oh}{\mathit{h}}^{k-1}+{\mathit{b}}_o)\hfill \end{array}$$

(42)

and:

$$\begin{array}{cc}\hfill {\mathit{h}}^k& =\varphi \left({\mathit{s}}^k\right)\odot {\mathit{o}}^k\hfill \end{array}$$

(43)

Figure 7 shows a graphical view of the LSTM model’s architecture for GHI forecasting.

3.4.3. Training Algorithm

Adam (for adaptive moment estimation), a well-known adaptive learning rate optimization algorithm [66], has been used to train both the MLP and LSTM neural networks. Adam performs first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. It is computationally efficient, has very little memory requirements and is well suited for large problems in terms of data or parameters. It is also well suited for problems with very noisy or sparse gradients. Finally, the hyperparameters have intuitive interpretations and tuning is rather simple [66].

Dropout [67] works by probabilistically removing, or “dropping out”, inputs to a layer, which may be input variables in the data sample or activations from a previous layer. It has the effect of simulating a large number of networks with different topologies and, in turn, making neurons generally more robust to the inputs. Such a mechanism improves the generalization ability of deep artificial neural networks significantly, even though its computational cost is low, and reduces overfitting (which leads to improved generalization ability). That is why dropout is nowadays the most popular regularization method. Its main drawback is that training is slowed down and, usually, the number of epochs is increased.

For the MLP and LSTM models, we tried our best to optimize the results, and after testing many architectures we obtained the topologies depicted in Figure 6 and Figure 7.

4. Results

This section presents the results and is divided into two parts. The first part presents the criteria used to assess the models’ performance: the normalized root mean square error, the dynamic mean absolute error and the coverage width-based criterion. A discussion about the results based on the obtained criteria values and examples of forecasts shown in Figure 8, Figure 9 and Figure 10 is conducted in the second part.

4.1. Performance Criteria

The different models developed in this paper were compared using the popular root mean square error normalized by the mean of data (nRMSE):

$$\mathrm{nRMSE}=\frac{\sqrt{\frac{1}{n_{*}}{\sum }_{i=1}^{n_{*}}{\left(y_{*i}-{\widehat{y}}_{*i}\right)}^2}}{\frac{1}{n_{*}}{\sum }_{i=1}^{n_{*}}{y}_{*i}}$$

(44)

where ${\mathit{y}}_{*}\in {\mathbb{R}}^{n_{*}\times 1}$ is the test data and ${\widehat{\mathit{y}}}_{*}\in {\mathbb{R}}^{n_{*}\times 1}$ is the forecast.

The nRMSE is a well-known criterion in the scientific literature and assesses the error between the forecasts and test data without giving an adequate quantification of the failures due to mismatches in the time index. Indeed, this temporal error can lead to the misestimation of some events, such as the production peak or ramps. That is why the dynamic mean absolute error (DMAE) [68], which accounts for temporal error (misalignments) and absolute magnitude error simultaneously, is also used to evaluate the models’ performance. The DMAE error is a trade-off between the temporal distorsion index and the mean absolute error. It involves two parameters: $\lambda$ that determines the weight of the MAE for the considered temporal distortion and c that controls the allowed temporal distorsion. Herein, $\lambda =0.1$ and c = 5%. For more details, the interested reader is referred to [68].

As the confidence intervals are calculated and displayed for all models, the quality of prediction intervals is assessed. Criteria such as the prediction interval coverage probability (PICP) or the prediction interval normalized average width (PINAW) can be used for the assessment of prediction intervals [69]. The PICP evaluates whether the target values lie within the constructed prediction intervals limits and is defined by [69]:

$$\begin{array}{c}\hfill \mathrm{PICP}=\frac{1}{n_{*}}\sum _{i=1}^{n_{*}}{ϵ}_i\end{array}$$

(45)

where $n_{*}$ is the number of test data and ${ϵ}_i$ is a Boolean variable, which is equal to 1 if the target value $y_{*i}$ lie within the prediction interval limits and 0 otherwise.

Mathematically, ${ϵ}_i$ is defined as follows:

$${ϵ}_i=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{y}_{*i}\in [L_i,U_i],\hfill \\ 0\hfill & \mathrm{if}{y}_{*i}\notin [L_i,U_i].\hfill \end{array}\right.$$

(46)

where $y_{*i}$ is a test datum and $L_i$ and $U_i$ are respectively the lower and upper bounds of the prediction interval.

The PINAW assesses the width of prediction intervals and is defined by [69]:

$$\begin{array}{c}\hfill \mathrm{PINAW}=\frac{1}{n_{*}R}\sum _{i=1}^{n_{*}}(U_i-L_i)\end{array}$$

(47)

where R used for normalization is the difference between the maximum and minimum of the test data.

The assessment of prediction intervals based only on PICP criterion or PINAW index is inaccurate, as forecast with a large prediction interval can lead to a high PICP. However, such a forecast is not as well as another one that has a narrow prediction interval (small PINAW value) and a high PICP. That is why the combinational index of these two criteria proposed in [70], named coverage width-based criterion (CWC) that carries out information about the width and coverage probability of prediction intervals is interesting to evaluate the sharpness of the forecast. This index is defined by:

$$\begin{array}{c}\hfill \mathrm{CWC}=\mathrm{PINAW}\left(1+\gamma \left(\mathrm{PICP}\right)·exp\left(-\eta ·(\mathrm{PICP}-\mu )\right)\right)\end{array}$$

(48)

where $\mu$ is the nominal confidence level associated with the prediction intervals, and $\eta$ is a parameter that magnifies the difference between PICP and $\mu$. Herein, $\eta =10$ and $\mu =0.95$. The function $\gamma$ is defined by:

$$\gamma =\left\{\begin{array}{cc}1\hfill & \mathrm{if}\mathrm{PICP}<\mu \hfill \\ 0\hfill & \mathrm{if}\mathrm{PICP}⩾\mu .\hfill \end{array}\right.$$

(49)

In this paper, besides the nRMSE and DMAE used to evaluate the models’ forecasting performance, the CWC was used to assess the quality of models’ prediction intervals. GPR naturally provides a predictive distribution, from which confidence intervals are obtained. However, the other considered methods—LS-SVR and ANN—do not. For LS-SVR, the method developed in [71] for construction of confidence intervals based on the central limit theorem for linear smoothers combined with bias correction and variance estimation is used to obtain the confidence intervals. Concerning the ANN models, the bootstrap method [72,73] that allows training neural networks with different parameter initializations in order to estimate the model uncertainty is used to quantify the uncertainty associated with the forecasts (here 30 repetitions were used for both the MLP and LSTM models).

4.2. Forecasting Results

The models’ forecasting skill was assessed by comparing the forecasts over different time horizons ranging from 10 $\min$ to 4 $\mathrm{h}$. The aforementioned criteria were calculated for each horizon and for each model. Numerical results for the considered forecast horizons are stored in

Table 2. nRMSE values for all the models included in the comparative study.

Forecast Horizon	10 $\mathbf{min}$	1 $\mathbf{h}$	4 $\mathbf{h}$
Scaled persistence	0.2020	0.3435	0.4628
LS-SVR	0.2197	0.3305	0.4317
LSTM	0.2016	0.3089	0.3995
MLP	0.1985	0.3210	0.4449
$k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR	0.2057	0.3140	0.4020
$k_{\mathrm{SE}\text{-}\mathrm{ARD}}$-based GPR	0.2079	0.3287	0.4285

,

Table 3. DMAE values for all the models included in the comparative study.

Forecast Horizon	10 $\mathbf{min}$	1 $\mathbf{h}$	4 $\mathbf{h}$
Scaled persistence	0.04951	0.08201	0.13481
LS-SVR	0.05582	0.08463	0.14753
LSTM	0.03317	0.05587	0.08956
MLP	0.02948	0.07248	0.15173
$k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR	0.03419	0.05677	0.08912
$k_{\mathrm{SE}\text{-}\mathrm{ARD}}$-based GPR	0.04416	0.06718	0.09816

and

Table 4. CWC values (lower is better) for all the models included in the comparative study.

Forecast Horizon	10 $\mathbf{min}$	1 $\mathbf{h}$	4 $\mathbf{h}$
LS-SVR	0.5660	1.1136	9.8391
LSTM	0.4734	0.8283	2.1642
MLP	0.4609	0.7031	2.2884
$k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR	0.4574	0.6026	2.8651
$k_{\mathrm{SE}\text{-}\mathrm{ARD}}$-based GPR	0.4966	0.7610	8.8394

.

As can be seen in Table 2, at the lowest forecast horizon (10 $\min$), all the models developed in this study gave good and similar results, with a slight advantage to the ANN models, except the LS-SVR model which gave the worst result. As the forecast horizon increased, the superiority of the machine learning models became clear. They surpassed the scaled persistence model, whose performance degraded quickly as the forecast horizon increased. When looking at the nRMSE values of the machine learning models for all horizons, it can be noticed that the performances of these models were very similar, but the LSTM model and the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel outperformed the others as the forecast horizon increased. Table 2 also shows that, at the lowest horizon (10 $\min$), the MLP model gave the best results but gave the worse results at the highest horizon (4 $\mathrm{h}$).

The values of DMAE for all models are stored in Table 3. Based on this criterion, the LS-SVR model and the scaled persistence model gave the worst results for all horizons compared to the other models. This criterion combined the temporal distorsion between the forecasts and test data with the absolute magnitude error. For lower horizons, the scaled persistence model gave good results according to its nRMSE values (Table 2) and its performance at the lowest horizon ($10\min$) was practically similar to the performance of ANN models ($\mathrm{nRMSE}=20.20\%$) and slightly better than the performance of GPR models. However, there were some time lags between the forecasts given by the scaled persistence model and the test data (Figure 8, Figure 9 and Figure 10), which are reflected in its DMAE values. Table 3 shows that, at the lowest horizon, the MLP model gave the best results, but for all horizons, the best performing models were the LSTM model and $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR model.

Table 4 stores the CWC values for the LS-SVR, ANN, and GPR models. All these models gave high coverage probability at the lowest horizon (10 $\min$), with an advantage to the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel which, besides, had narrow confidence intervals as demonstrated by its CWC value. When the forecast horizon increased, the quality and accuracy of forecasts degraded for each model. The rapid augmentation of the CWC values indicated these effects. At the highest forecast horizon (4 $\mathrm{h}$), the ANN models gave high PICP values, that is why they had small CWC values, but in terms of quality and accuracy of forecasts, when looking at the nRMSE values, the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel ($\mathrm{nRMSE}\simeq 0.40$) was better than the MLP model ($\mathrm{nRMSE}\simeq 0.44$). The ANN models also had large confidence intervals at the highest horizon, which were reflected by their forecasts displayed in Figure 10. For each forecast horizon, the LS-SVR model gave the worst CWC value compared to the other models. As a result, by examining its confidence intervals combined with its nRMSE values, we can conclude that it was difficult for this model to follow its counterparts’ performances.

Some examples of 10 $\min$, 1 $\mathrm{h}$ and 4 $\mathrm{h}$ forecasts are displayed in Figure 8, Figure 9 and Figure 10. A period of 6 days in the whole test database (from 28 November 2019 to 3 December 2019), having both clear and cloudy days, was chosen. At the lowest forecast horizon, the scaled persistence model was excellent, but when the horizon increased, its performance degraded quickly. Looking at Figure 8 for the lowest forecast horizon, all the models followed the trends in the test data quite well, except the LS-SVR model that did not predict well the patterns in overcast days. When the horizon increased, the performances of all forecasting models degraded. However, the figures show that the LSTM model and the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel outperformed the other models. That is especially evident in Figure 10, at the highest forecast horizon, where despite the deterioration of their performances, these two models followed pretty well the trends in test data.

Figure 10 also shows that while the LSTM model gave better results than others during cloudy days, during clear days the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel prevailed. The MLP model and the GPR model based on $k_{\mathrm{SE}\text{-}\mathrm{ARD}}$ did not follow well the trend of days when the forecast horizon increased. The LS-SVR model converged to a smooth signal at the highest forecast horizon. This was not surprising because its kernel was suitable for smooth functions’ representation and could not correctly model both large-scale and small-scale variations in the GHI signal. Among the ANN models, the LSTM model, which could learn long-term dependencies in the data, outperformed the MLP model. The large confidence intervals, when the forecast horizon got longer, demonstrated the models’ performance degradation.

When examining the models’ performances based on the obtained criteria values and examples of forecasts shown in Figure 8, Figure 9 and Figure 10, it yielded that the two best models for GHI forecasting were the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel and the LSTM model. Indeed, these two models outperformed their counterparts considered in this study. However, as the LSTM model and the $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR model gave similar results, it was hardly easy to distinguish between them through this work. As the GHI data used in this paper exhibited more clear days than overcast days, one would tend to choose the GPR model based on $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$ kernel, which outperformed the LSTM model during clear days and gave less good results than the LSTM model during cloudy days, for GHI forecasting for our location (i.e., Font-Romeu-Odeillo-Via, in the Occitania region (southern France)).

Table 5. Comparison of models’ computation time and complexity (number of parameters).

Model	Training Time ($\mathbf{min}$)	Number of Parameters	Execution Time for One Forecast ($\mathbf{s}$)
LS-SVR	133	26,457	4.12
LSTM	20	127,459	0.05
MLP	10	58,635	0.02
$k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR	209	26,457	12.59
$k_{\mathrm{SE}\text{-}\mathrm{ARD}}$-based GPR	200	26,457	12.59

stores the models’ training time, the number of parameters involved in each model, and the time needed to perform one forecast for each model. Although many parameters were involved in ANN models, they were fast to train and their forecasts were almost instantaneous. Concerning GPR models, they were very time-consuming in the training phase and needed more time to provide forecasts. Indeed, the generation of a forecast by a GPR model required an inversion of a matrix as big as the training dataset is large. However, there are many sparse approaches in the literature [74] that can be used to reduce time complexity and memory requirement while keeping the same performances for an in situ implementation.

5. Conclusions

This paper aims to shed light on the use of machine learning to forecast global horizontal irradiance (from which photovoltaic power generation can be inferred) using endogenous data. To do so, a comparative study of popular machine learning methods (artificial neural networks, support vector regression and Gaussian process regression) for GHI forecasting is conducted at different time horizons, covering intrahour and intraday cases. The models are developed using a 2-year GHI database with a 10 $\min$ time step. Usually, in machine learning studies, the time step is 1 $\mathrm{h}$, which leads to significant simplification of GHI dynamics. In addition, contrary to developing a specific model for each forecast horizon, as shown in many studies in the literature and which can be computationally demanding when many horizons are considered, we made the choice of multi-horizon forecasting models. The scaled persistence model is also included in the comparative study as a reference model. Two criteria, the dynamic mean absolute error (DMAE) and the coverage width-based criterion (CWC), are used in addition to the popular root mean square error normalized by the mean of data (nRMSE) in order to conduct an in-depth analysis of the models’ performance.

The results show that all the machine learning-based models outperform the scaled persistence model and among these models, the long short-term memory (LSTM) model and the $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR model outperform their counterparts. However, it is difficult to distinguish between these two models because their performances are very similar. The conclusion arising from the examination of the forecasts’ temporal evolution is that the LSTM model provides good forecasts for cloudy days, contrary to the case of clear-sky days where the $k_{\mathrm{RQ}\text{-}\mathrm{ARD}}$-based GPR model outperforms the LSTM model.