Irradiance Nowcasting by Means of Deep-Learning Analysis of Infrared Images

Abstract

This work proposes and evaluates a method for the nowcasting of solar irradiance variability in multiple time horizons, namely 5, 10, and 15 min ahead. The method is based on a Convolutional Neural Network structure that exploits infrared sky images acquired through an All-Sky Imager to estimate the range of possible values that the Clear-Sky Index will possibly assume over a selected forecast horizon. All data available, from the infrared images to the measurements of Global Horizontal Irradiance (necessary in order to compute Clear-Sky Index), are acquired at SolarTech^LAB in Politecnico di Milano. The proposed method demonstrated a discrete performance level, with an accuracy peak for the 5 min time horizon, where about 65% of the available samples are attributed to the correct range of Clear-Sky Index values.

1. Introduction

Power generation from a renewable energy source (RES) is becoming increasingly popular to meet the ever-growing global energy demand and to limit fossil fuel-related greenhouse gas emissions. In detail, over the last few decades, solar and wind have experienced significant growth in terms of installed capacity together with continuous technological improvement [1].

Regarding solar energy and, more specifically, photovoltaic (PV) technology, its rapid growth has faced multiple technical obstacles, such as low solar cell efficiency, and economic problems, such as high cost of materials [2]. Theoretical analyses show that solar energy alone, with adequate technology to collect and store it, could meet the energy demand of the entire world [3]. For example, global energy consumption recorded in 2017 was around 18.1 TW and is projected to be around 28 TW by 2050, while land-based solar energy in one year is estimated at 23,000 TW. Furthermore, solar energy is a renewable source, and its exploitation for energy production does not cause polluting emissions and greenhouse gases [4].

The energy produced by a RES is characterized by fluctuations, which are dependent on various environmental and meteorological parameters. Hence, a high penetration of this type of source into the electrical system is challenging [5]. In a scenario where renewable energies play a fundamental role in the energy supply, a reliable forecast of energy production is essential to properly manage the electrical system [6]. For this reason, RES energy production forecasting is one of the most popular topics within the scientific community today [7]. Solar energy forecasting is useful for regulating the routine operation of electrical grids and microgrids with integrated solar power generation [8,9], enabling imbalances in energy production and consumption to be minimized [10]. Furthermore, it is a fundamental tool during the identification of the potential location for the photovoltaic installation and the subsequent planning phase [11].

Different forecasting horizons can be identified. They are highly dependent on the forecasting target. In particular, for power production, two different classifications can be identified: the first one is related to the operation and maintenance of the grid, while the second one is closely related to electricity market definitions [12,13]. By analyzing [14], it is possible to identify three main horizons: the long-term forecasting, corresponding to the day-ahead forecast, the mid-term from 1 day down to 6 h, the short-term from 6 h to 30 min, and nowcasting, which includes the forecast horizons under 30 min. The nowcasting time horizon, the central topic of the present work, is crucial in solving grid stability issues and optimizing the microgrid’s energy management systems [15]. Moreover, it is vital in regulating the electricity market whenever a solar power production facility is connected to the electrical grid [16]. In nowcasting, the object of the solar forecast is the solar irradiance, which directly influences the power produced by PV modules.

Nowcasting methods can be roughly classified into physical and data-driven models [17]. Three main types of solar radiation forecasting methods are considered in the scientific literature: numerical weather forecasting (NWP) models, appropriate for long-term forecasting; satellite techniques, appropriate for time horizons up to 8 h ahead; ground techniques, appropriate for nowcasting. NWP models are based on a set of differential equations that describe the physical phenomena of interest. In order to achieve an accurate forecast, they require a proper definition of initial conditions [18]. Furthermore, images captured from a satellite are useful for forecasting tasks, providing an overview of large-scale weather systems and their movement. Ground-based techniques exploit All-Sky Imagers (ASIs), cameras capable of acquiring pictures of the sky through an hemispherical mirror, which are helpful in estimating cloudiness [19]. The ASI can be configured for different operational environments and network designs, from a standalone edge computing model to a fully integrated node in a distributed, cloud-computing-based microgrid [20].

The most challenging part of nowcasting is the analysis of transient clouds and hence the estimation of type, cover, and movement [21]. These challenges are addressed by computer vision [22], a branch of computer science dedicated to the emulation of human visual system capabilities that allows computers to recognize elements in pictures and videos in the same manner as people. In recent years, thanks to the development of artificial intelligence and deep learning, this field took great leaps forward that enabled the ability to perform several tasks that previously could not be accomplished without artificially complex methods [23]. A deep-learning tool that achieved excellent results in several works from scientific literature is the Convolutional Neural Network (CNN) [24]. In computer vision tasks, CNNs have several advantages over traditional Neural Networks because they are specifically optimized in extracting and learning features from images [17].

The aim of this work is to exploit full-sky infrared images to predict, by means of a CNN, the Clear-Sky Index (CSI) in different time horizons, namely 5, 10, and 15 min. In this work, all available images are acquired by an ASI installed at the SolarTech^LAB in Politecnico di Milano. Images are acquired in RGB format once per minute: this sampling rate is limited by the available hardware. In order to reduce the dataset required for the training of convolutionary neural networks, here, it was, therefore, decided to crop the images in the portion around the sun. This helps the applicability of the method because in many cases it is not possible to have a sufficiently large dataset.

The structure of the present article is organized as follows: Section 2 describes the available data and the characteristics of the forecast model developed; Section 3 reports the performance level achieved in irradiance nowcasting; Section 4 draws conclusions from the presented work.

2. Solar Irradiance Forecasting Procedure

In this section, all the steps of the procedure adopted to perform the forecast of solar irradiation are discussed, from preprocessing the dataset to the development of the deep-learning model. This section is structured as follows: Section 2.1 and Section 2.2 describe the available data and the preprocessing stages, and Section 2.3 describes the structure of the adopted CNN.

Available data consist of infrared sky images captured by specific ASIs and Global Horizontal Irradiation (GHI) measurements. In detail, they are acquired for all the days comprised between September 2019 and April 2020, with a one-minute time step and considering only the daylight hours.

2.1. Image Processing Techniques

The available images undergo a specific pre-processing procedure with two objectives. The first is to obtain a format suitable for processing with a CNN template. The second goal is to be able to reduce the number of samples required for CNN training.

The instrumentation in SolarTech^LAB is capable of acquiring sky images with a multi-spectral approach. Figure 1, for instance, reports an example of a same sky view acquired by two different sensors in optical and infrared spectral regions, respectively. However, only the infrared images are included in the dataset exploited in the present work since they are less prone to saturation issues and allow effectively identifying the sun position.

From previous studies, it appears that the most significant areas for nowcasting are those around the sun. In fact, these areas are the ones responsible for direct radiation. Starting from this consideration, in the available infrared images, a square region surrounding the position of the sun is cropped, as shown in Figure 2. This operation allows reducing the computational burden of the training phase, both for the number of images required and the computational time.

The available infrared images undergo specific preprocessing that aims to isolate the region surrounding the sun’s location and a cleaning procedure that aims to exclude rainy conditions from the data involved in the CNN development, and then the forecast model is tested.

2.2. Classification Phase

Once the images are properly treated, they are classified according to two different criteria. The first criterion aims at dividing the images according to the presence of precipitation. Specifically, two different groups of images can be identified, representing, respectively:

• Dry conditions: clear sky conditions, characterized by the absence of significant cloud coverage, and covered sky conditions, characterized by the presence of significant cloud coverage without precipitation, are considered together as a single group.
• Rainy conditions, characterized by significant cloud coverage and the presence of water droplets on the hemispherical mirror surfaces of ASIs.

In Figure 3, examples of clear sky (a), covered sky (b), and rainy (c) conditions are reported.

Images related to rainy conditions are identified thanks to a rain gauge installed in the weather station of the SolarTech^LAB and are removed from the dataset involved in the solar irradiance forecast.

The second classification criterion consists of dividing the images into five different groups on the basis of the Clear-Sky Index value, defined as [25,26]:

$$\mathrm{CSI}{|}_t=\frac{{\mathrm{GHI}}_m{|}_t}{{\mathrm{GHI}}_{cs}{|}_t}$$

(1)

In the equation, ${\mathrm{GHI}}_m$ is the the measured Global Horizontal Irradiation, while ${\mathrm{GHI}}_{cs}$ is the Global Horizontal Irradiation in clear sky conditions. In other words, CSI corresponds to a normalized GHI, which allows removing the effects of seasonal changes from the GHI itself [27].

In more detail, images are grouped into five distinct classes based on strict threshold values of CSI, as reported in

Table 1. Threshold values of CSI for classes definition.

Class	CSI Range	Sky Condition
1	0 < CSI < 0.2	Overcast
2	0.2 < CSI < 0.45	Partially cloudy
3	0.45 < CSI < 0.75	Partially cloudy
4	0.75 < CSI < 1	Clear
5	CSI > 1	Over-irradiance

. “Over-irradiance” is a particular condition where solar radiation multiple reflections between the ground and direct beam could result in greater solar radiation levels on the pyranometer with respect to Clear Sky conditions.Hence, the CSI, which is the ratio of measured irradiance at ground level to the estimated irradiance in clear sky conditions at ground level, could go beyond 1, as shown in [28].

Figure 4 depicts the overall double-step classification scheme that allows preprocessing the data.

2.3. Model Development

In the present work, a deep-learning model for image classification, namely a CNN, is developed. The idea of CNNs is derived from studies on the brain’s visual cortex. Each of the neurons dedicated to vision processes information derived from a limited region of the visual field only. Moreover, the receptive fields of different neurons may overlap, and, all together, they cover the entire visual field. This structure, capable of detecting complex patterns in any region of the visual field, inspired the researcher to develop a Neural Network architecture that gradually evolved into the current CNN [29].

In further detail, the typical CNN structure is capable of learning features from input images through a sequence of convolutional and pooling layers [30]. The convolutional layer is the crucial building block of CNNs: neurons in this type of layer are not connected to every pixel in the input image, only to pixels in their corresponding receptive fields. The weight of a neuron is represented by a filter (or convolution kernel) that, when applied to the image, is able to extract features from it. During the training phase, the convolutional layer learns which is the best-suited convolution kernel to fulfill a specific task [31]. On the other hand, the pooling layer has the goal of subsampling the input image in order to reduce the computational load, the memory usage, and the number of network parameters to be tuned. As in convolutional layers, each neuron is connected to a restricted region of the previous layer. Moreover, neurons in this layer do not have weights: all they do is aggregate the inputs according to a specific aggregation function, such as “max” or “mean”. After being processed in the cascade of convolutional and pooling layers, the information flow is flattened, i.e., it is structured in a suitable format to be further processed.

After the feature learning step, the flowing information is flattened (properly structured in order to be further processed) and delivered to a fully connected layer that performs the classification step.

The structure of the CNN adopted in the present work is presented in Figure 5. Hidden patterns are extracted from images by means of two couples of convolutional and pooling layers. Then, a flattening layer and a fully connected layer perform the feature-based classification. Each convolutional layer adopts 30 convolution kernels with a 4 × 4 squared shape and a stride equal to 1, while each pooling layer exploits pooling kernels with a 4 × 4 squared shape and a stride equal to 4. The number of kernels, their shape, and the number of neurons in the fully connected layer are chosen through a sensitivity analysis procedure that allows selecting a proper configuration capable of achieving reasonable performance.

Concerning the available infrared images, 70% are exploited as the training set, while another 15% constitutes the validation set. The remaining 15% of data is used as the test set.

3. Results and Discussion

In this section, the forecasting performance achieved with the proposed method is reported considering all three different forecasting horizons considered, i.e., 5, 10 and 15 min, respectively.

Firstly, the instrumentation used in this case study is described, then the numerical results are presented.

3.1. Case Study

Weather parameters are monitored at the SolarTech^LAB with a meteorological station equipped with a solar irradiance sensor, temperature and humidity sensors, a wind speed and direction sensor, and a rain collector. Solar irradiance is measured with two secondary standard pyranometers for the measurement of the total irradiance on horizontal and 30° tilt planes. In addition, a pyranometer with a shadow band is available for measuring the diffuse irradiance.

The main characteristics of the sensors, together with the temperature measuring equipment, are described in

Table 2. Detailed characeristics of weather station.

Irradiance sensors	Pyranometer (LSI, DPA252)
Standard	Secondary standard ISO 9060
Measurements range (W m${}^2$)	<2000
Spectral range	300–3000 nm
Total achievable daily uncertainty	<2%
Directional response	<±5.4 Wm${}^2$
Thermal drift	<2%
Temperature sensor	Pt100 1/3 B (DIN EN 60751)
Measurements range	[−30 °C, +70 °C]
Uncertainty	0.2 °C (at 0 °C)
Resolution	0.04 °C
Response time (T90)	3 min: with filter;
	20 s: without filter

.

The meteorological station performs ambient conditions measurements every ten seconds. The average, maximum, minimum, and standard deviation of the values measured by the sensors are calculated every minute, and these values are stored in a database.

The SkyCam system comprises a long-wavelength infrared camera placed over a hemispherical mirror reflecting the sky. It thus provides full-sky coverage, equivalent to a 180° × 360° field-of-view. The SkyCam uses a curved mirror with a diameter of 35 cm, coated with chrome. It is designed to be dust-tight, to resist liquid ingress, and to operate in the −20 to +60 °C ambient temperature range. The resolution of the raw images is 640 × 480 pixels. The sky-imager possesses its own mechanical-optical arrangement and orientation once deployed. A geometrical calibration is therefore performed at each site to retrieve the optical intrinsic parameters of the sky-imager. The calibration process is fully automated and uses an automatic algorithm based on the solar position captured during clear-sky days over full sun paths to calibrate the pixels. In addition to the geometric calibration, automated detection of the mask is performed to distinguish between useful pixels and useless pixels obstructing the field-of-view of the imager. The mask detection procedure automatically distinguishes between cold clear sky areas and hot radiant objects. The entire geometric calibration/mask detection procedure generally takes at most seven days—the average time needed to gather enough clear-sky image statistics.

Finely determining where each pixel points to on any given part of the sky (in terms of azimuthal and zenithally angles) is mandatory to detect possible shading obstacles. In our case study, the chosen location is the Department of Energy rooftop at Politecnico di Milano University campus; therefore, shading objects (as trees and buildings) are avoided as the location is in the middle, and the surroundings have no shading objects. Finally, technical features of the camera, such as ISOs, maintenance procedures, and calibrations, are industrial property of the manufacturer, and we are not allowed to disclose them.

3.2. Numerical Results

The confusion matrices for all the forecasting horizons are presented in Figure 6, Figure 7 and Figure 8. In each matrix, a generic cell in row i and column j reports the number of samples belonging to class i that are assigned to class j during classification. The diagonal cells (green in the Figure) represent the correctly classified samples, while the cells outside the diagonal (red and orange in the Figure) represent samples assigned to the wrong class. The orange cells represent an error of one single class, which is more acceptable because, in many cases, it is due to the sharp divisions of the classes. The right vertical line represents the precision of the method, while the bottom line the recall. Finally, the cell in the bottom right contains information on the accuracy of the prediction.

Figure 6 represents the confusion matrix calculated over a time horizon of 5 min. The overall accuracy level achieved is equal to 65.79% of samples correctly classified. Figure 7 represents the confusion matrix calculated over a time horizon of 10 min. The accuracy obtained over this time horizon is equal to 65.02%. Figure 8 represents the confusion matrix calculated over a time horizon of 15 min. Considering this time horizon, only 56.25% of the total number of samples are correctly classified. It is observable that class 1 (the most populated one) is always recognized without committing any error, while class 5 (the least populated one) is not recognized at all by the model. As expected, the accuracy of classification decreases when the time horizon considered increases. In further detail, the recorded performance drop is very limited when increasing the time horizon from 5 to 10 min (lower than 1%), while it is significant when increasing the time horizon from 10 to 15 min (almost 9%).

Figure 9, Figure 10 and Figure 11 report the sorted daily classification performances for each of the considered time horizons (5, 10, and 15 min), from the best to the worst. Therefore, all the daily classification performances, from the best to the worst, can be observed. In all the diagrams, the average performance level is highlighted by an orange dotted line. In this way, the daily performances above and below the average level can be clearly identified.

As observed, in a few cases, all the samples are correctly classified, reaching peaks of 100% in the daily classification accuracy. On the other hand, there are days characterized by a very low classification accuracy, where only a bunch of samples are correctly classified.

4. Conclusions

The present work proposed a new nowcasting method to calculate the variability of solar irradiation over multiple time horizons, namely 5, 10, and 15 min ahead. The method was implemented through a Convolutional Neural Network (CNN) that uses the infrared sky images captured by an All-Sky Imager (ASI) installed at the SolarTech^LAB, Politecnico di Milano, as input. The implemented system requires two main input elements: the full-sky camera and a computer capable of processing the images. The SkyCam costs EUR 60 k (EUR 45 k for the purchase of a second-hand instrument), while 1 year’s rental costs about EUR 15–20 k. The computer with GPU for image processing costs about EUR 1.5–3 k considering the required technical characteristics. The network’s output consists of a range of possible values of the Clear-Sky Index (CSI), a parameter that considers the effect of transient clouds on the Global Horizontal Irradiation (GHI) only. The available infrared images undergo specific preprocessing, aimed at isolating the region surrounding the position of the sun. In addition, a cleaning procedure has been implemented to exclude rainy conditions from the data involved in the development of the CNN, then the forecast model was tested. The CNN exhibited a discrete performance level, with an accuracy peak over the 5 min time horizon, where 65.79% of the available samples are attributed to the correct range of CSI values. Considering larger forecast horizons, as expected, the overall classification accuracy decreases to 65.02% (10 min forecast horizon) and 56.25% (15 min forecast horizon).

Future works in this area can analyze the effect of introducing some meteorological input parameters. These data, in fact, can help nowcasting because they provide information, such as the current irradiation, very correlated with the CSI to be predicted, and the current wind direction, which can help CNN identify when clouds can cover the sun.