1. Introduction
When supplying energy into the electricity grid, it is important to know the expected output from solar energy systems. While a point forecast of the expected output is valuable, it is only the expected value, whereas a probabilistic forecast gives a range of the most likely values of the output. A probabilistic forecast provides information about all expected outputs and allows one to asses a wide range of uncertainties and that can in turn improve decision making. A probabilistic forecast can be thought of as the error bounds of the forecast. These error bounds are also known as prediction intervals.
Probabilistic forecasting is becoming more prevalent in the renewable energy forecasting literature. Note also that one of the priorities of the International Energy Agency Task 16 on “Solar resource for high penetration and largescale applications” is developing the best methods for probabilistic forecasting of solar radiation. For a detailed literature review on probabilistic forecasting, please refer to our earlier work [
1] on probabilistic forecasting of solar irradiation. Since then, several studies have been reported in the literature. Please note that a glossary describing the symbols used is given at the end of the paper in
Section 6.
Lauret et al. [
2] develop three probabilistic forecasts based on nonparametric methods of linear quantile regression. Scolari et al. [
3] develop ultrashortterm (from 500millisecond to 5min) probabilistic forecasts of global horizontal irradiance. They use a nonparametric method based on kmeans clustering of historical data according to certain influential variables; average clearsky index and clearsky index variability. Chu and Coimbra [
4] develop very shortterm (5, 10, 15 and 20min) probabilistic forecasts of direct normal irradiance (DNI) based on a knearest neighbor ensemble (kNNEn) method. An assessment of machine learning techniques for intrahour point and probabilistic forecasting is performed by [
5].
David et al. [
6] develop probabilistic forecasts on time scales of ten minutes to six hours of global horizontal irradiance. They use a clearsky model to model the annual and diurnal cycles and a combination of the auto regressive moving average (ARMA) model to describe the serial correlation and the parametric generalized auto regressive conditional heteroscedasticity (GARCH) model to model the volatility.
Trapero [
7] develops onestepahead hourly uncertainty forecasts of global horizontal irradiation (GHI) using kernel density estimates, GARCH and Single Exponential Smoothing methods, and a combination of the before mentioned methods. They find that the combination of methods is a good compromise between coverage and interval width.
Voyant et al. [
8] uses two persistent and four machine learning point forecasting methods for onestepahead hourly GHI. The probabilistic forecast (one to six hours) method is based on the bootstrap sampling of a training set using the kfold method, together with the cumulative distribution function (CDF).
It is important to note that most of the studies in the literature use a clearsky model to encapsulate the climatology (detrending) of GHI. We instead use Fourier series to model the climatology.
In this paper, we develop a new probabilistic forecasting method. Our new probabilistic forecasting method extends our previous bootstrap method [
1,
9] to a case of an exponentially decaying heteroscedastic model for tracking dynamics in solar radiance. For the remainder of this paper we refer to our old method as
unconditional and our new method as
conditional. The unconditional method catered for the global systematic variation in variance of solar radiation, whereas the conditional method also caters for the local variation in variance—in other words the conditional heteroscedastic nature. One could say that the
unconditional picks up the climate variation in variance, while the
conditional picks up the variation because of the weather. The performance of the
unconditional method was very good, except in one aspect. Since the prediction interval construction was based solely on bootstrapping errors for the particular time of day and year, the present weather conditions were not taken into account. Thus, on a systematically clear day, the prediction intervals, although they exhibited the correct coverage criteria, they were much wider, less sharp, than one would hope. The
conditional method presented here takes into account the conditions on the day as well as the time of day and year, and thus the intervals are sharper. Please note that even though the analysis is performed on hourly data similar procedures can be used for shorter time scales. In particular, they will be adapted to be used for forecasting output on a fiveminute time scale for solar farms in Australia, under an Australian Renewable Energy Agency (ARENA) funded grant.
Boland and Soubdhan [
10] use Fourier series for the climate variation, and then an autoregressive model to forecast the residual series after removal of the seasonal component. In that paper prediction intervals are formed for the forecasts at the three sites studied using an ARCH model technique. However, they do not take into account the interplay of systematic and conditional changes in variance that is integral to the treatment here.
This paper is organized as follows. A description of the data is given in
Section 2.
Section 3 describes both the point forecasting model and its performance. In
Section 4 we describe our conditional probabilistic forecasting method, and report on its performance and compare the results to the literature in
Section 5. The paper is concluded with a discussion of our findings and ideas for future work in
Section 6.
3. Point Forecast
We begin with a description of our onestepahead point forecasting model. We use Fourier analysis to account for the seasonality and an autoregressive model to account for the serial correlation. Our hourly GHI model for Mildura is given by:
where
${F}_{t}$ is a seasonal component,
${A}_{t}$ is an autoregressive component, and
${Z}_{t}$ is a noise such that
$E{Z}_{t}=0$,
$E{Z}_{t}{Z}_{l}=0$ if
$t\ne l$ and
$E{Z}_{t}^{2}={\sigma}_{t}^{2}$. That is,
${Z}_{t}$ may be heteroscedastic. The autoregressive component
${A}_{t}$ is a linear combination of previous time steps.
We use power spectrum analysis to identify the significant frequencies in the Fourier component
${F}_{t}$. We use 11 significant frequencies at cycles 1, 2, 364, 365, 366, 729, 730, 731, 1094, 1095 and 1096 cycles per year. These correspond to onceayear (1) and twiceayear (2) cycles, onceaday (365), twiceaday (730) and threetimesaday cycles (1095), as well as the beat frequencies for the onceaday (364, 366), twiceaday (739, 731) and threetimesaday cycles (1094, 1096). As [
12] points out, the beat frequencies, also known as sidebands, are used to modulate the amplitude to suit the time of year. For our three locations,
${F}_{t}$ explains between 80–85% of the total variance.
We then use an autoregressive model of order 3, AR(3), to model the serial structure in the residuals after ${F}_{t}$ has been removed. The final step was to zero any night time values; solar altitudes less than zero degrees.
Similar point forecasting models were also developed for Adelaide and Darwin but are not shown here. For a detailed description of our point forecasting method, the reader is encouraged to read our previous paper [
1].
4. Probabilistic Forecasting
We previously developed a computationally efficient and datadriven method for shortterm probabilistic forecasting of solar radiation, using a nonparametric bootstrapping method and a map of sun positions [
1]. The idea behind using this method was to account for the heteroscedasticity of the white noise in Equation (
1).
The hourly daytime errors (noise) ${Z}_{t}$ from the insample forecast are placed into a twodimensional array ${B}_{i,j}$ according to the sun position (determined by sun elevation and solar hour angle), for a given hour. The rows i correspond to sun elevations in increments of ten degrees and the columns j correspond to solar hour angles in increments of fifteen degrees. We did this to take care of the systematic variation in variance in the GHI time series. That is, the variance in GHI differs throughout the day (higher in the middle of the day compared to the beginning and end of the day) and throughout the year (higher in summer compared to winter). The result is a twodimensional array, where each element is a bin of errors corresponding to a specific sun position.
We generate prediction intervals using nonparametric resampling (with replacement). To generate a $(1\alpha )100\%$ prediction interval for a particular hour, the empirical $\alpha /2$ and $(1\alpha /2)$quantiles from the bin in the twodimensional array corresponding to the sun position for the hour are calculated. The empirical $\alpha /2$ and $(1\alpha /2)$quantiles are then added to the point forecast ${\widehat{I}}_{t}$, resulting in lower and upper prediction intervals for the hour. Our method develops prediction intervals without imposing any parametric assumptions on the underlying distribution of GHI. As an example, suppose we wish to construct 95% prediction intervals for our forecasts. We first determine, given the time of day and day of the year, which bin the forecast is referring to, and use the error distribution in that bin for our calculations. For this error distribution we determine the 2.5% and 97.5% quantiles for this empirical distribution. We then add these values to the forecasted value of the solar irradiation determined by the point forecast method to construct our prediction interval.
For a detailed description of our previous probabilistic forecasting method, the reader is encouraged to consult our previous paper [
1].
The conditional method for generating a probabilistic forecast described in this paper, extends our unconditional method to a case of an exponentially decaying heteroscedastic model for tracking dynamics in solar radiance. The unconditional method catered for the global systematic variation in variance of solar radiation, whereas the conditional method also caters for the local variation in variance.
The problem with the unconditional method is that while it caters for the variance GHI can exhibit for a particular time of year/time of day (global), it does not cater for the variance in GHI that is currently evident at time t (local). The conditional method also takes into the account the current variance. For example, if the current variance at time t is small, then the conditional method will generate prediction intervals for time t + 1 that are narrower than otherwise would be generated by the unconditional method.
The final errors are uncorrelated but dependent. Uncorrelated refers to the fact that the means at each time step are not connected, but the series can still be dependent since dependence between the variances remains. This is exemplified by the squared error terms, a proxy for variance, being correlated. This characteristic is the socalled autoregressive conditional heteroscedastic (ARCH) effect. Usually when this happens one uses an ARCH or GARCH model for forecasting the variance.
However, we found that instead an exponential smoothing form performed better than an ARCH or GARCH model:
where
${\psi}_{t}^{2}$ is the variance at time
t.
Since we are forecasting the variance, and then constructing a prediction interval using this forecast, we must perform the forecast assuming the noise is normally distributed, which is not true. Therefore, we first had to use a normalizing transformation, then forecast the variance, construct the prediction intervals, and then transform back. Templeton [
15] describes a method for transforming a nonnormal variable to normal. In each bin of errors, we have distributions that are nonnormal. We transform the errors to normal for each bin independently. We first find for each error its percent rank, in other words how far along the empirical CDF it lies. The result is uniformly distributed probabilities. We then apply the inverse normal transformation to these probabilities, resulting in values from the standard normal distribution. We can then use these squared normal errors as proxies for the variance and proceed to construct a forecast model.
The method for generating the lower and upper prediction intervals is shown in Algorithm 1.
The final step in generating prediction intervals is to impose sensible upper and lower limits. We impose a lower bound limit of zero on the lower prediction interval. We restrict the upper prediction interval using the Bird clearsky model [
16] and then add twenty percent for cloud enhancement [
17].
Figure 2,
Figure 3 and
Figure 4 compares 95% prediction intervals using the unconditional and conditional probabilistic forecasting methods, for clear and cloudy days, for Adelaide, Darwin, and Mildura respectively. A
clearsky day is where there is little evidence of the transient effects of clouds causing no fluctuations in GHI. A
cloudy day is where there is some evidence of the transient effects of clouds causing fluctuations in GHI.
Algorithm 1: Algorithm for generating (100$\alpha $) prediction intervals using the conditional method. 

For clearsky days, the observed GHI values fall within the prediction intervals with unconditional probabilistic forecasting method providing narrower prediction intervals for all three locations. For cloudy days, most of the observed GHI values fall within the prediction intervals with the conditional probabilistic forecasting method providing narrower prediction intervals for all three locations. In the next section we numerically examine the performance of our conditional probabilistic forecasting method.