Analyzing Spatial Variations of Cloud Attenuation by a Network of All-Sky Imagers

Abstract

All-sky imagers (ASIs) can be used to model clouds and detect spatial variations of cloud attenuation. Such cloud modeling can support ASI-based nowcasting, upscaling of photovoltaic production and numeric weather predictions. A novel procedure is developed which uses a network of ASIs to model clouds and determine cloud attenuation more accurately over every location in the observed area, at a resolution of 50 m × 50 m. The approach combines images from neighboring ASIs which monitor the cloud scene from different perspectives. Areas covered by optically thick/intermediate/thin clouds are detected in the images of twelve ASIs and are transformed into maps of attenuation index. In areas monitored by multiple ASIs, an accuracy-weighted average combines the maps of attenuation index. An ASI observation’s local weight is calculated from its expected accuracy. Based on radiometer measurements, a probabilistic procedure derives a map of cloud attenuation from the combined map of attenuation index. Using two additional radiometers located 3.8 km west and south of the first radiometer, the ASI network’s estimations of direct normal (DNI) and global horizontal irradiance (GHI) are validated and benchmarked against estimations from an ASI pair and homogeneous persistence which uses a radiometer alone. The validation works without forecasted data, this way excluding sources of error which would be present in forecasting. The ASI network reduces errors notably (RMSD for DNI 136 W/m${}^2$, GHI 98 W/m${}^2$) compared to the ASI pair (RMSD for DNI 173 W/m${}^2$, GHI 119 W/m${}^2$ and radiometer alone (RMSD for DNI 213 W/m${}^2$), GHI 140 W/m${}^2$). A notable reduction is found in all studied conditions, classified by irradiance variability. Thus, the ASI network detects spatial variations of cloud attenuation considerably more accurately than the state-of-the-art approaches in all atmospheric conditions.

1. Introduction

Short-term forecasts of the solar resource for lead times of up to six hours ahead (i.e., nowcasts) can be of interest to plan and compensate local fluctuations of the photovoltaic (PV) production in electric power grids [1,2,3,4,5]. Additionally, a nowcast of the spatial distribution of solar irradiance and cloud attenuation for the forecast instance time (i.e., an analysis) may be of interest to support numerical weather prediction (NWP) models [6], satellite-based nowcasts [7] and to upscale the production of distributed PV installations [8].

Timeseries-based nowcasts typically require a ground-based radiometer. Their accuracy decreases rapidly with the distance of the nowcasted location from the radiometer and with the forecast lead time, in particular in more variable conditions [9]. Based on satellites, nowcasts can be provided with a spatial resolution down to 0.5 km${}^2$ at a sampling time of 5 min [10]. If a nowcast of higher temporal and spatial resolution is available, this may allow for capturing fluctuations which are relevant for distribution grids and to compensate imbalances in the fluctuating energy supply and demand in real time on a local scale. This is expected to avoid investments into grid infrastructure [11,12,13]. Corresponding regulatory incentives have already been implemented e.g., in Switzerland [14]. Various studies have investigated strategies to increase the self-sufficiency of neighborhoods and local communities [15,16,17,18] or to reduce the stress on the local grid infrastructure [19].

All-sky-imager- (ASI-) based nowcasts can have a spatial resolution of 5 m × 5 m and may cover lead times up to 25 min ahead in 1-min steps e.g., [20,21]. Systems based on up to four ASIs, located in proximity [22,23], have been validated practically. These nowcasts were used mainly to plan and optimize the production of solar power plants in which they were located. ASIs take images of the sky in which clouds are detected. The locations and velocities of the clouds are determined from image series and the clouds’ future positions are predicted. The radiative effect of clouds is modeled and finally irradiance is nowcasted. If more than one ASI is used or if cloud height is estimated with the help of external data, irradiance maps can be nowcasted for the ASIs’ vicinity.

The setup of state-of-the-art nowcasting systems allows accurate cloud observations only in a limited area. This limits the forecast domain, horizon and the accuracy towards higher lead times. Additionally, up to now, the overall accuracy of these nowcasting systems is only satisfying for certain applications e.g., [24].

In an urban distribution grid, a large number of small PV installations with a capacity of 20 kW or less can be spread over an area of e.g., 13 km × 12 km in the case of the German town Oldenburg. State-of-the-art ASI-based systems with up to four ASIs located at distances of typically less than 2.5 km [22,23] can be expected to bring increased deviations if the production of each installation in this large area needs to be nowcasted, see, e.g., [21]. A larger number of ASIs could therefore be distributed in such an urban area to achieve a higher accuracy. This suggests to use a combined evaluation which turns individual ASIs into an ASI network. An ASI network can be expected to increase the accuracy of cloud modeling as multiple perspectives on clouds are combined.

ASI networks have been used in the field of astronomy to monitor the night sky for example to locate and track fireballs [25,26] or to monitor the space weather [27]. To nowcast solar irradiance, a few ASI networks with up to 16 ASIs have been installed [28,29,30] or are planned [31].

The Eye2Sky ASI network [32] provides the experimental infrastructure for the present study. Eye2Sky is being set up in the region of Oldenburg. At present, Eye2Sky includes 29 stations distributed over an area of roughly 110 km × 100 km. Each station is equipped with an ASI. Nine of these stations are supported by rotating shadowband irradiometers (RSI) to provide direct normal (DNI), diffuse horizontal (DHI) and global horizontal irradiance (GHI) and measurements of ambient temperature and relative humidity. Eight ceilometers are included in the network. Six of these are operated by the meteorological service Deutscher Wetterdienst (DWD). Several PV plants and numerous smaller distributed PV installations are also present in the study area. In the urban area of Oldenburg, the network features a high density of 14 ASIs in an area of 13 km × 12 km.

First, studies have recently presented approaches to estimate solar irradiance or to model the atmosphere’s and clouds’ optical depth using ASI networks [28,30,33]. Chu et al. [30] used seven ASIs to estimate global, direct and diffuse irradiance pointwise at the ASIs’ locations. Instead of using a cloud modeling procedure, an irradiance map was estimated by Kriging interpolation. Mejia et al. [33] and Aides et al. [28] both followed an approach based on ray-tracing and tomographic reconstruction. These authors studied an ASI network either using simulated ASI images [33] or they presented experimental data to reconstruct an exemplary cloudy scene in a single timestamp [28]. The approach of the latter two studies accounts for the scattering processes in the atmosphere in a detailed and physically reasonable way.

This publication develops and validates a novel procedure to model the shape and optical properties of clouds and thus spatial variations of cloud attenuation by an ASI network. This procedure is intended to be applied to analyze and nowcast solar irradiance in an operational way. The ASI network approaches discussed previously [28,30,33] were not demonstrated in a practical nowcasting application. Because of the following considerations, we do not consider these approaches practical for that task. Chu et al. [30] essentially used the ASIs of their network as radiometers. Networks of radiometers have also been used in other previous studies to analyze and nowcast solar irradiance e.g., [34,35,36]. The ASIs’ potential to estimate spatial variations of cloud attenuation or solar irradiance on the ground beyond their application as radiometers was not tapped. We expect that this enhancement is important to analyze spatial variations of cloud attenuation more accurately. On the other hand, the tomographic approaches [28,33] were demonstrated only for specific conditions with a single cumulus cloud layer at a low cloud coverage. From the findings of these studies and others which followed voxel carving approaches [22], we conclude that the limited perspectives of ASIs monitoring clouds only from below and the high opacity of low layer clouds bring considerable challenges to tomographic reconstruction approaches. Consequently, in conditions with multiple cloud-layers or with a high cloud coverage, the functionality of these approaches may reduce noticeably. In a practical application and in particular in the Eye2Sky region’s climate, these cloud conditions are highly relevant. Additionally, high image quality is important for the tomographic approaches. Accordingly, in the previous studies, either ASIs equipped with a shadowband were evaluated for a single timestamp [28] or artificial, modeled images [33] were used as input. In the operational application pursued for Eye2Sky, lower image quality needs to be accepted as images of regular outdoor cameras need to be evaluated reliably at any timestamp and as the maintenance effort of the numerous cameras needs to be kept manageable. Under these conditions, especially, a beam irradiance incident on an ASI causes glare effects, complicating radiative transfer modeling [37,38,39]. Thus, instead of modeling clouds physically rigorously by tomography and radiative transfer, a more statistical approach is chosen.

We use previous operational cloud modeling procedures from the field of ASI-based nowcasting [22,23,40] as a basis for the present one. As a starting point, clouds are detected using the convolutional neural network (CNN) of [41], which was already used by a nowcasting system based on two ASIs [40]. However, in the present case, we use this method to also distinguish cloud types. The radiative transfer approaches [28,33] estimated cloud optical thickness based on pixel intensities in the ASI images. In contrast, the CNN also includes textural features of the ASI images to identify cloud classes, which indicate the clouds’ attenuation by a so-called attenuation index.

The cloud modeling procedures of [22,23,40] were only developed for up to four ASIs located in proximity and were not suited to combine the observations received from very different perspectives and with different accuracies. Compared to these approaches, the ASI network aims to mitigate errors in the cloud parameters derived from single ASIs more effectively. From the principles of error propagation, it is expected that multiple observations, i.e., maps of cloud properties, merged by accuracy-based averaging, are significantly more accurate than the most accurate of those observations alone. An accuracy-weighted average was previously used to combine analysis and nowcasted irradiance from satellites with ASI-based data and/or ground-based measurements [42,43]. Analogously, this approach is applied here to combine observations from the independent ASIs. The approach requires information on the uncertainty of the observations to be merged. The ASI observations’ uncertainties are estimated based on the result of previous validations of ASI-based systems [21,41,44,45,46].

After combining the observations from multiple ASIs, the attenuation index is expected to be a good indicator of cloud attenuation. As the used CNN can only differentiate sky and three classes of clouds and as there are still sources of error present in the procedure, a ground-based DNI measurement is included to estimate the final map of cloud attenuation by a statistical procedure. A similar statistical procedure which estimated cloud transmittance was developed in [47]. However, in the present case, we avoid a number of shortcomings. In particular, our procedure does not rely on cloud height information which is subject to errors and which is only available at rather coarse resolution. Due to this and as our method avoids a discretization of the observations, we can resolve variations of cloud attenuation in a much more graduated way. Finally, our cloud modeling procedure is validated comparing our estimations directly to those of two state-of-the-art analysis systems described by [40]. While the merging procedure can be adapted for various parameters related to clouds such as cloud velocity and DHI, attenuation index and derived cloud attenuation are the focuses in this study.

This publication is structured as follows: Section 2 introduces the experimental setup and datasets. Section 3 first introduces the reference analysis approaches and then develops the ASI network’s cloud modeling procedure. In Section 4, the new method is validated using ground-based RSIs providing DNI and with it the clear sky index of DNI ($k_{DNI}^{*}$) as well as GHI. Furthermore, the method is benchmarked against an ASI pair’s estimation as well as against a homogeneous persistence analysis of DNI and GHI derived from an RSI alone. Finally, Section 5 discusses the experimental findings and concludes the study.

2. Materials

The Eye2Sky ASI network was still under construction in the year 2020 when the experimental work of this research was conducted. Due to the ongoing extension and temporary outages, the number of available ASIs changed during the period of the experimental work. A maximum of 12 operational ASIs was achieved in the urban area of Oldenburg. This area is shown in Figure 1 and covers roughly 13 km × 12 km.

The all-sky imagers (ASIs) of Eye2Sky are surveillance cameras of type Mobotix Q25 6MP color version [48] with a fisheye lens providing $180{}^{\circ }$ field of view. This camera model has been used in previous studies on ASI-based nowcasting e.g., [20]. Furthermore, the predecessor model of this ASI type, Mobotix Q24 3MP color version has been used in prior studies for ASI-based nowcasting [22,49,50], sky condition monitoring [51,52] and to estimate solar irradiance [37,49].

Exemplarily, Figure 2 shows the ASI at station DON (left) and an image from ASI DON (right). The ASIs record an image every half and full minute. Further details on the exposure and other camera settings are described by Blum et al. [37,53]. Each ASI was calibrated geometrically by an intrinsic calibration which describes a fisheye camera’s image distortion and by the ASI’s external orientation. The ASIs’ intrinsic calibrations were determined by the procedure of Scaramuzza et al. [54]. The ASIs’ locations defined by latitude, longitude and altitude were identified in geolocated satellite images which were available in Google Maps [55]. Altitude was estimated based on the local altitude of the ground and the stations’ height over ground. The cameras’ external orientation was determined as described by Blum et al. [37].

2.1. Reference Measurements of Solar Irradiance

Three meteorological stations DON, UOL and CLO are equipped with rotating shadowband irradiometers (RSI) and measurements of relative humidity and dry bulb temperature. The locations of these stations are shown in Figure 1 and listed in

Table 1. The table gives an overview of the station IDs and geolocations of the radiometer and all-sky imager (ASI) stations of Eye2Sky which are used in this study. Additionally, the distances between the radiometer stations are given. Each radiometer station listed below is equipped with a rotating shadowband irradiometer (RSI) which uses two LI-COR LI-200 silicon pyranometers.

Station	Geolocation		Distance from [km]
ID	Lat. [${}^{\circ }$N]	Lon. [${}^{\circ }$E]	UOL	DON	CLO
AMM	53.1816	8.1456
BFE	53.1492	8.23204
CLO	53.11200	8.21004	$5.6$	$3.8$	0
DON	53.14638	8.21734	$3.8$	0	$3.8$
ETZ	53.186205	8.225326
EMS	53.13021	8.15472
FLE	53.14861	8.19747
HOL	53.13873	8.24786
JET	53.1204	8.09583
MAR	53.13918	8.21504
TIR	53.13892	8.19640
UOL	53.15342	8.16191	0	$3.8$	$5.6$

. Figure 2 (left) exemplarily shows the RSI at DON. The RSIs are of the type CSPS Twin-RSI [56]. The measurement accuracy of this instrument type was evaluated by Forstinger et al. [57]. Each radiometer station provides measurements of GHI, DHI and DNI.

Only the measurements of RSI DON are used as input to estimate DNI, DHI and GHI. UOL and CLO are reserved for validations. This holds for all estimations tested in this study. UOL and CLO are located at a distance of 3.8 km to the west and to the south of DON, respectively. The locations of DON, UOL and CLO represent the central and the peripheral parts of the urban area. These stations are therefore interesting to evaluate the ASI network’s ability to predict variations of DNI and GHI within the urban area.

All radiometric measurements were quality controlled with a method described by Geuder et al. [58]. Automatic filters were applied to guide the expert during the inspection. Measurements of the RSIs were corrected and calibrated as described by Geuder et al. [59]. To improve the homogeneity of the readings from the three RSIs, this calibration was modified by a relative calibration on-site based on selected clear-sky periods. In the calibrations and evaluations, the air mass correction function used by Geuder et al. [59] was omitted. Thus, the applied correction increases the irradiance measurements by 0.6% compared to the original correction at a sun elevation of 15° and has a smaller effect at larger sun elevations. As the present evaluation is limited to sun elevations of at least 15°, the effect is small compared to the expected errors of the analysis approaches. Furthermore, the applied calibration removes a possible bias which could be introduced by the influence of air mass.

2.2. Atmospheric Conditions and Datasets

The atmospheric conditions in Oldenburg are evaluated to investigate the analysis approaches’ performance under different conditions. DNI measurements of RSI DON are used as input to calculate hourly DNI variability classes, by the procedure of Schroedter-Homscheidt et al. [60], for the whole year 2020.

Table 2. This table summarizes the sky conditions, DNI clear sky index and DNI variability which are indicated by each of the eight DNI variability classes. These classes are defined and calculated according to Nouri et al. [40]. The table was created based on [40].

Variability Class	Sky Conditions	DNI Clear Sky Index	Variability
1	mostly clear sky	very high	low
2	almost clear sky	high	low
3	almost clear sky	high/intermediate	intermediate
4	partly cloudy	intermediate	high
5	partly cloudy	intermediate	intermediate
6	partly cloudy	intermediate/low	high
7	almost overcast	low	intermediate
8	mostly overcast	very low	low

provides a short description for each of the eight classes. The evaluation is restricted to times with a sun elevation of at least ${15}^{\circ }$. Lower sun elevations are expected to be less relevant for solar energy applications as they contribute a small share of the annual energy yield. Low sun elevations would affect the accuracy of the classification and can lead to increased errors of the ASI-based systems as the image areas close to the sun would be affected by lens distortion. Figure 3 shows the relative frequencies of the eight variability classes in the year 2020 (blue solid bars). Variability classes 6–8 are the most frequent ones and represent $65\%$ of the evaluated hours. These classes indicate a high degree of cloud coverage. In total, $35\%$ of the dataset belong to variability class 8. This means overcast is the most relevant sky condition at the site. Thus, the ASI-based system is required to perform robustly under conditions with a high cloud coverage. For the more variable classes 3, 4, 6 and 7, highly resolved analyses of GHI are of particular relevance. Around $39\%$ of the hours belong to these classes.

The year 2020 is used as measurement period to evaluate estimations of DNI and GHI. To be able to use separate datasets for model development and validation and also to keep the computation time moderate, only subsets of the whole annual dataset are evaluated. Three datasets are defined. The 30-days development dataset is used for estimations which provide a feedback for the model development. The 28-days validation dataset is used for a part of the validations if a smaller dataset is considered to be sufficient. The 92-days validation dataset is used for most validations presented in this study. Each of these datasets contains full days. As shown in Figure 3, each of these datasets was assembled to represent the DNI variability classes with similar relative frequencies as found for the dataset of the whole year 2020.

For the 30-days development dataset, roughly every 11th day of the months January to November 2020 was used. Consecutively, single days were exchanged if a reference station had an outage on a day and until the variability classes were represented by the dataset as intended. The days contained in the 28-days validation dataset are usually the days which follow those days of the 30-days development dataset. Again, single days were exchanged in the case of outages and to better represent the variability classes. The 92-days validation dataset contains all days of the 28-days validation dataset and usually the two days which follow a day of the 28-days validation dataset. As before, single days were removed or exchanged for the reasons given above. The individual dates contained in each dataset are listed in Appendix D.

3. Methods

This section presents the methods used in this publication. The ASI network’s cloud modeling procedure is developed. Furthermore, the state-of-the-are analysis approaches are introduced which are used as the basis of the model development and later as reference estimations to benchmark the ASI network against. Our development and validation focus on cloud modeling and not on forecasting. Therefore, only a lead time of 0 min (i.e., analysis) is investigated. Forecasts (i.e., predictions with lead times greater than 0 min) would contain errors related to the motion and temporal evolution of clouds. Therefore, a validation based on forecast data would be less suited to compare the deviations of cloud modeling approaches.

3.1. Reference Analysis Approaches

The state-of-the-art analysis based on an ASI pair [40] is the starting point of the developments in this study. Nouri et al. [40] additionally present a hybrid nowcast which is not used in this study as it is expected to bring a very limited advantage if only analysis data are evaluated.

The ASIs for the ASI pair were selected in a trade-off between multiple aspects. Firstly, one of the ASIs should be located right next to an RSI as measurements of DNI, DHI, relative humidity and dry bulb temperature are required as inputs to this system. A small camera distance is preferred as cloud base height (CBH) is rather low for most timestamps at this site [53]. Additionally, both ASIs should have a mostly free field of view up to a zenith angle of ${78}^{\circ }$. Based on these considerations, ASIs DON and FLE were selected to form the ASI pair DON-FLE. The location of these ASI is shown in Figure 1. The camera distance is 1.4 km. RSI DON is located directly next to ASI DON and supports the ASI pair with the required measurements. As desired, the fields of view of both ASIs are mostly free. The analysis of ASI pair DON-FLE is generated as follows.

The convolutional neural network (CNN) of Fabel et al. [41] detects cloudy pixels in the images from the ASIs DON and FLE. The stereoscopic approach of Nouri et al. [45] evaluates a series of three images from both ASIs and measures cloud motion vectors (CMVs) and CBH by cross-correlation of features in the red-channel images. Cloudy pixels observed from the ASIs’ perspectives are compared, and CBH is considered to model clouds as three-dimensional objects [22]. Ray tracing [47] yields the corresponding position of a cloud shadow on the ground. This way, a shadow map is estimated and, for each shaded grid cell, the shading cloud object is known. The radiative effect of clouds is modeled to assign DNI. From recent measurements of DNI, periods are rejected in which the DNI timeseries indicate that the RSI is shaded by a cloud [47]. From the remaining measurements, Linke turbidity as well as present and future clear sky DNI are derived. A probabilistic procedure determines the transmittance of each cloud object based on recent and historic data pairs of CBH and DNI [47]. The shadow map is transformed into a DNI map. The DNI in a grid cell is determined as the product of clear sky DNI and the transmittance of the shading cloud object, if applicable. DHI is assumed to be spatially homogeneous and persistent. The estimated DNI map together with the DHI measured at forecast instance time at DON yield a GHI map.

Homogeneous persistence will be used as a further reference approach to analyze irradiance maps. Homogeneous persistence uses a point-wise ground-based measurement of GHI and DNI and assigns the same value of GHI and DNI to all grid cells in the maps of GHI and DNI.

3.2. Development of the ASI Network’s Cloud Modeling Procedure

The ASI network’s cloud modeling procedure relies on up to twelve ASIs which are placed at distinct points in and around Oldenburg. As a starting point, sky images are segmented, undistorted, georeferenced and transformed into maps of attenuation index. The attenuation index is introduced as a parameter which indicates the presence and the expected attenuation of a potential cloud for each grid cell. As shown in

Table 3. The five cloud classes received from cloud segmentation are mapped to values of the attenuation index $AI$. These approximate values of attenuation index reflect our experience and are used in an intermediate step. Exact values of cloud attenuation are determined based on $AI$ later in Section 3.2.4.

Cloud Class	Attenuation Index AI
undefined	n.d.
low layer/thick cloud	1
mid-layer cloud	0.7
high layer/thin cloud	0.2
clear sky	0

, cloud classes are provided by the segmentation procedure for each image pixel of an individual ASI. These five cloud classes are mapped to one of five values of attenuation index which range between zero and one (see Table 3). The sources of uncertainty in the maps of the attenuation index are determined. Based on the local uncertainty of the single ASIs’ maps of the attenuation index, the merging procedure assigns a weight to the grid cell of each map and averages the weighted maps. Consecutively, a statistical procedure is applied to self-calibrate the relationship between the attenuation index and cloud attenuation. The remainder of this section describes these steps in more detail.

3.2.1. Cloud Classification and Cloud Mapping

The ASI network’s cloud modeling procedure requires a reliable cloud segmentation as input. In total, three of the cloud segmentation approaches [41,61,62] were tested for the site. In particular, a four-dimensional clear sky library (CSL) and a convolutional neural network (CNN) were seen as potentially useful approaches.

The CNN of Fabel et al. [41] represents a rather new segmentation approach. Own evaluations for the site Oldenburg qualitatively confirmed the high accuracy reported in the literature for the CNN in a different climate [41]. The CNN performed robustly under all relevant conditions. In very dark scenes, typically related to an overcast sky, the CNN’s performance was only slightly worse than under the remaining conditions. Soiling occasionally led to the faulty recognition of optically thin clouds. This is acceptable for the cloud modeling approach presented here, as these faultily detected clouds will be assigned a low attenuation. Due to these favorable test results and as the CNN allows for distinguishing cloud types, it is selected to provide the cloud segmentation for the ASI network. Unlike most prior cloud modeling approaches, the novel cloud modeling procedure will incorporate ASI-image-based cloud types as an indicator of cloud attenuation. Figure 4 shows an exemplary raw image of ASI DON (left) and the detected cloud types (right).

Up to this point, the cloud segmentation is given in the coordinate system of each ASI image. Using the known location of the ASI and with the geometrical calibrations, described in Section 2, each pixel of the segmented ASI image is projected onto a map, which represents a bottom view of the detected clouds. The height over ground of this map is fixed to the local CBH which may vary spatially. CBH is estimated by the technique presented by Blum et al. [53]. These maps and all other maps of parameters derived from the ASI network use a single coordinate system and a spatial resolution of 50 m × 50 m.

Based on the cloud segmentation, each of the map’s grid cells belongs to one of five classes listed in Table 3 (left column). This classification is converted into a floating scale ranging between zero and one, which is referred to as attenuation index $AI$ in this study. The parameter $AI$ is assigned to the five cloud classes as shown in Table 3 (right column). Each value of $AI$ is chosen to indicate roughly which attenuation A is expected for the respective cloud class. However, the relationship of $AI$ and A is superimposed by various errors, which are present in the ASI-based maps of $AI$.

3.2.2. Merging Maps of Attenuation Index

The merging operation applied to the maps of attenuation index, i.e., $A{I}_i$, from independent ASIs is introduced in the following. The georeferenced map of $A{I}_i$ received from each ASI i is weighted by the inverse of the uncertainty $u_{i,x,y}$ of the respective map. The uncertainty $u_{i,x,y}$ is estimated in Appendix A. This weighting is referred to as accuracy weighting and corresponds with the one used by Meyer et al. [42]. The weighted maps are averaged to receive the merged map $A{I}_{merged}$ of the parameter:

$$A{I}_{merged,x,y}=\frac{{\sum }_{i=1}^{N}A{I}_{i,x,y}·u_{i,x,y}^{-1}}{{\sum }_{j=1}^N{u}_{j,x,y}^{-1}}.$$

(1)

The subscript $x,y$ in Equation (1) indicates that the merging operation is applied for each grid cell $(x,y)$ independently, based on the specific values of $A{I}_{i,x,y}$, $u_{i,x,y}$ for that grid cell.

The merging operation for maps of $AI$ is summarized in Figure 5.

To simplify the example, only the two ASIs EMS and DON have been included in the visualization. ASI EMS is located 4.5 km to the southwest of ASI DON. Each ASI provides $AI$ in a circular area of the map (left column), referred to as an ASI’s observed area. The size of this area is determined by the ASI’s maximum viewing angle of ${\theta }_{max}={78}^{\circ }$ and the present CBH in the urban area (1200 m in this example). The weight $u_{i,x,y}$ (center column) decreases rapidly with the distance of a grid cell from the ASI’s location. In the merged map of $AI$ (right column), the combined field of view then covers a larger area. The transition between the maps of $AI$ received from both ASIs appears natural, i.e., continuous and smooth, in this example. Furthermore, the merged map features a finer graduation of $AI$.

Inaccuracies in the estimation of CBH present a sources of uncertainty which are accounted for implicitly by the merging procedure. Blum et al. [53] described the procedure which is used here to estimate CBH. This procedure can bring deviations, in particular, in situations with highly variable CBH e.g., in multi-cloud-layer situations. Due to an inaccurate CBH, the orthoimage of the single ASI is projected into a horizontal plane at the estimated local CBH and is expanded over an area which is either too small or too large. Consequently, in the ASI-based map, the area monitored by an ASI is stretched proportionally to the error in estimated CBH. In particular, a cloud is represented in the map of $AI$ by a zone of increased $AI$. The area of this zone and the distance of this zone’s center of mass from the ASI’s location are both overestimated if CBH is overestimated.

These errors occur for each individual ASI-based map in the same way. For state-of-the-art ASI-based systems typically relying on two or less ASIs, these errors cannot be avoided as the ASIs need to be located in close proximity, to measure CBH robustly under all relevant conditions [44,53,63]. In the ASI network, the merged map of $AI$ is automatically blurred if CBH is erroneous. This mechanism is comparable to a photograph viewing objects which are out of focus. This behavior may be advantageous and is used in the modeling of expected cloud attenuation, which is described below.

3.2.3. Clear Sky DNI

In the ASI network, clear sky DNI ($DN{I}_{clear}$) is estimated by a procedure previously used in ASI-based nowcasting e.g., [46]. This procedure identifies clear-sky periods and the corresponding clear sky DNI from recent readings of a ground-based DNI measurement. In a second step, the model of Ineichen et al. [64] is applied to identify the Linke turbidity $T_L$ which explains the timeseries of $DN{I}_{clear}$. Based on sun position and $T_L$, $DN{I}_{clear}$ can be calculated for any timestamp as long as $T_L$ remains constant. In the present study, the estimate of $DN{I}_{clear}$ is required to be as steady as possible within the period from sun rise of the evaluated day until the timestamp when spatial variations of DNI should be analyzed. Prior ASI-based systems focused on detecting clouds which have a notable attenuation e.g., [47,65]. For those systems, it can be beneficial to include the attenuation of thin clouds in $DN{I}_{clear}$. An estimation of clear sky DNI of this kind was used, e.g., by Nouri et al. [20]. Unlike that, the present method to assign cloud attenuation considers variations of DNI related to optically very thin clouds in all cases. Furthermore, by the present approach, overestimations of $DN{I}_{clear}$ are tolerated, while underestimations should be avoided.

Based on these considerations, the procedure to determine $DN{I}_{clear}$ described by Nouri et al. [46] is used. The cloud-screening algorithm [66] used by that method is applied with an adapted set of parameters as follows. Measurements of DNI and $T_L$ from the recent 120 h instead of 48 h are inspected. This longer history allows for identifying clear periods more reliably during longer phases with high cloud coverage. On the other hand, this long history will lead to an overestimated $DN{I}_{clear}$ if $T_L$ increases rapidly. This is acceptable, as an overestimation of $DN{I}_{clear}$ can be compensated by the ASI network’s approach to model cloud attenuation. From this history, the most recent 240 instead of 15 observations, which were identified as clear sky, are retained for the consecutive weighted average. This assures a more inert behavior of $DN{I}_{clear}$. Furthermore, the cloud-screening algorithm is parameterized rather strictly rejecting any $T_L$ greater than 5 instead of 13. Higher values of $T_L$ are seldom at the site and are more likely to go back to an erroneous detection of clear sky in the presence of very thin clouds.

In a future enhancement of the present procedure, it could also be considered to estimate clear sky DNI and Linke turbidity based on multiple data sources, in particular including satellite-based services such as CAMS McClear [67,68]. This may bring an advantage in some cases when the present procedure erroneously detects clear sky.

3.2.4. Assigning Cloud Attenuation

Up to this point, merged maps of $AI$ have been determined. $AI$ was introduced as an indicator of cloud attenuation based on our experience that the cloud segmentation can indicate the existence of an optically thick/intermediate/thin cloud and that this indication gets more accurate as observations from multiples ASIs are merged. This indication is only qualitative as the classes optically thick ($AI=1$)/intermediate ($AI=0.7$)/thin ($AI=0.2$) are not strictly related to a certain cloud attenuation A. Accordingly, the relationship of $AI$ and attenuation A can be nonlinear and biased. Furthermore, the relationship may not be static but depend on the segmentation procedure’s accuracy for the present cloud conditions. We now use measurements of DNI from RSI DON and calculate the local cloud attenuation A. Then, we use present, recent and historic values of A at DON to derive a relationship of $AI$ and A which is apt to the respective timestamp of the analysis. With this relationship finally, A can be estimated in each point of the observed area. This procedure adapts the relationship of $AI$ and A dynamically. Therefore, the exact values of $AI$, which were assigned to the cloud classes in the first step, have a minor influence on the method.

In the first step, the present measurement of DNI at RSI DON is evaluated. From DNI, together with clear sky DNI, $DN{I}_{clear}$, the local value of the attenuation A is calculated:

$$\begin{array}{c}\hfill A=1-\frac{DNI}{DN{I}_{clear}}.\end{array}$$

(2)

The local value of A together with $AI$ of the cloud grid cell shading the RSI is stored as new entry to a database. In the proximity of the RSI at DON, the merged map of $AI$ is influenced strongly by the maps of $AI$ received from ASI DON and nearby ASIs. For other parts of the urban area, a smaller number of ASIs contributes to the estimation of $AI$. Likewise, in more peripheral grid cells, $AI$ is determined based on larger zenith angles of the raw ASI images. These aspects may lead to a different relationship of $AI$ and A in the central and in the peripheral parts of the urban area.

The relationship of $AI$ and A reflected by the database is intended to hold for the whole urban area and if possible beyond this area. For these more peripheral locations, no separate radiometer station is included in the estimation. However, $AI$ can often be tracked from these areas to the location of RSI DON. Consequently, $AI$ of the cloud shading the grid cell of RSI DON is predicted based on ASI images from the present timestamp and also based on ASI images captured 30, 60, …, 300 s before, tracking the respective recent maps of $AI$ along the local cloud motion vectors to RSI DON. This way, in total, eleven database entries are created per timestamp. Each of these entries consists of the local values of A, $AI$ predicted for the cloud grid cell shading the reference radiometer (in this study only RSI DON), an ID of the radiometer, the timestamp based on which $AI$ is predicted and the timestamp, at which A is calculated. Each database entry is referred to as an observation in the following.

In the second step, present and previous observations of A are each assigned a weight, which reflects how closely that prior value of A is related to the present and the future value of A. Observations of the same day receive a weight depending on the forecast instance time t and the timestamp $t_{j,sameday}$ of the observation. It is defined by

$$\begin{array}{c}\hfill w_{j,sameday}={\left(1+(t-t_{j,sameday})/T_{1/2}\right)}^{-1}.\end{array}$$

(3)

$T_{1/2}=60\mathrm{s}$ is an empiric constant indicating after which time the weight of an observation reduces to $1/2$. The weight assigned to an observation of the same day, based on its age, is shown in Figure 6 (labeled single observation). The relationship between $AI$ and A changes on the short-term as A varies depending on the present cloud types and also as the quality of the maps of $AI$ may be variable depending on the present errors in the cloud segmentation and in the estimation of CBH. Figure 6 also shows the cumulated weight of observations over the age (labeled cumulated from present). The example evaluates a timestamp one hour after sun rise, given a constant rate of valid readings from the radiometer station during that hour.

Observations from before the present day are referred to as historic observations. In this study, only observations from the year 2020 are included. All M historic observations together are given a constant weight of $0.05$. Each historic observation (index $j,historic$) then receives an identical weight $w_{j,historic}=0.05/M$. The constant weight implies that there is a relationship between $AI$ and A, which holds in general. Due to the small weight, the historic observations only contribute to the estimation of cloud attenuation in the sense of fallback values. These observations are relevant directly after sun rise, in the case of outages of the included radiometer and in some cases when the cloud conditions change abruptly.

In the third step, a function $\widehat{A}$ is fitted to represent the relationship of $AI$ and A based on the weighted observations. $\widehat{A}$ is defined so that it must increase monotonically with $AI$. If a decrease is indicated by the observations, this may either be related to a small database of observations or artifacts in the cloud segmentation procedure. The exact definition of $\widehat{A}$ is given in Appendix C. Attenuation is expected to range between zero and one, provided that estimated clear sky DNI is not underestimated. Accordingly, the range of values taken on by $\widehat{A}$ is restricted to the same interval. A simple fit function is preferred. Based on these considerations, a piecewise linear function is used for the fit.

A loss function $Loss$ is defined which penalizes deviations between each observation of attenuation $A_j$ and the respective output of $\widehat{A}\left(A{I}_j\right)$, calculated from the observation of $A{I}_j$. $Loss$ corresponds to a weighted sum of squared deviations:

$$\begin{array}{c}\hfill Loss\left(\widehat{A}\right)=\sum _{j=1}^N{w}_j{\left(\widehat{A}\left(A{I}_j\right)-A_j\right)}^2.\end{array}$$

(4)

The weight $w_j$ corresponds to $w_{j,sameday}$ and $w_{j,historic}$ depending on the age of the respective observation. A numerical solver is applied to find the function $\widehat{A}$ which minimizes $Loss$. At each forecast instance time, the database of observations and related weights are updated, and the solver is applied using the parameters which minimized the loss function at the forecast instance time evaluated before as starting value. Figure 7 shows weighted observations and the resulting distinct slopes of $\widehat{A}$ (blue dash-dotted curve) for a scattered (left) and a multi cloud layer situation (right). These situations are discussed further in the following section.

Finally, a map of the attenuation is created. For this, $\widehat{A}$ is applied to each grid cell of $AI$. A ray-tracing procedure then relates grid cells from the map of A at CBH to grid cells in the DNI map at the ground’s local altitude. The ray-tracing procedure used here modifies prior schemes which projected clouds as discrete bodies to the ground (e.g., used by [47]). Due to this restriction, the prior schemes were not suited to resolve variations of cloud properties within such rigid bodies, which is important here. In contrast to Nouri et al. [47], the procedure used here links each grid cell of the ground model at its local altitude with a grid cell at the local cloud base height. Together with $DN{I}_{clear}$, the respective ground-projected value of A yields DNI in each grid cell of the ground model.

4. Results and Discussion

In this section, the characteristics and the accuracy of the developed procedure to model clouds and assign cloud attenuation are evaluated in comparison to the readings of RSIs located within the urban area of Oldenburg. First, two exemplary situations are inspected in Section 4.1. Thereafter, the method is validated and benchmarked quantitatively. The validation evaluates 92 days, which represent the atmospheric conditions of the year 2020. This dataset is described in Section 2.2 and in Appendix D. As benchmarks, estimations from ASI pair DON-FLE and homogeneous persistence are used, which were both defined in Section 3.1. Please recall that homogeneous persistence receives a measurement of DNI/GHI from a single radiometer and applies it homogeneously to all grid cells of the map. Each of the three analysis approaches includes the measurements of DNI and DHI from radiometer station DON. The validation mostly examines DNI and the respective clear sky index $k_{DNI}^{*}$. Compared to GHI, DNI is more closely related to attenuation, which is the focus of this study. However, GHI is evaluated at some points to allow a comparison of the present results to those from the literature and due to its relevance for many solar technologies. The quantitative validations first evaluate deviation metrics for the overall dataset and for subsets corresponding to eight variability classes in Section 4.2. Thereafter, the procedure’s deviations are analyzed using a quantitative definition of variability in Section 4.3. Finally, the correlation of predicted and measured DNI is analyzed by relative frequency distributions in Section 4.4.

4.1. Maps of Attenuation Index and GHI in Different Cloud Conditions

Exemplary maps of attenuation index and GHI are inspected and compared to georeferenced raw sky images. The maps of GHI from the ASI network are compared to the ones from ASI pair DON-FLE. This visual inspection should provide a first overview of the properties of the ASI network’s maps of attenuation index and irradiance.

Figure 8 studies a scattered single cloud layer situation observed on 5 June 2020, 10:22:00. In the top left panel, the sky images of all ASIs available for this timestamp were overlaid. For this purpose, the raw RGB ASI images were split into red-, green- and blue-channel images. Images of the same color channel were georeferenced and merged by the procedure developed in Section 3.2. The three georeferenced and merged maps of the color channels were stacked into a sky image map with an RGB entry per grid cell. The sky image map gives an impression of the actual sky conditions in the evaluated area of around 20 km × 20 km.

As seen in the top right panel, the ASI network mainly recognizes sky ($AI\approx 0$) and a single cloud layer with $AI\approx 1$. In the urban center, the maps of $AI$, which are given the highest weight there, coincide well leading to sharply limited cloud edges. The urban periphery is observed by fewer ASIs and at larger zenith angles. For all ASIs, an increased uncertainty is expected in these areas. In the map of $AI$, this is reflected by less clear structures.

The bottom row of Figure 8 compares the maps of GHI received from ASI pair DON-FLE (left) and ASI network (right). For comparability between ASI pair and ASI network, DHI is modeled as homogeneous in this study. In these GHI maps, the positions of ground-based RSIs are marked by red circles. The color code inside each circle indicates the local measurement. The ASI network clearly allows for predict irradiance in a larger area. Furthermore, GHI from the ASI network is more graduated in particular at cloud edges which are connected to increased uncertainty. The GHI map retrieved from the ASI pair exhibits perspective errors at larger distances from the ASIs’ locations as a comparison of undistorted raw images showed. These errors are in large parts avoided by the ASI network. Shapes of cloud shadows detected by the ASI pair appear stretched in the direction pointing away from the two ASIs. For the ASI network, a similar distortion is only visible towards the periphery of the urban area, where only few ASIs provide cloud observations. Furthermore, by the combination of multiple observations, the ASI network apparently avoids faults of the cloud segmentation, e.g., in the area around ASI EMS (marked by an arrow). The raw image of ASI EMS from that timestamp shows that the circum-solar area is obscured by a thin cirrus cloud layer. Optically thicker clouds are only visible at sun angles of more than ${20}^{\circ }$. This matches with the ASI network’s map of GHI.

Analogous to Figure 8, Figure 9 shows a complex multi cloud-layer situation observed on 27 June 2020, 11:27:00. Here, the ASI network recognizes clouds at various levels of $AI$ and correspondingly cloud attenuation (not shown) and GHI are also markedly graduated. The ASI pair on the other hand only predicts high GHI values, indicating that the urban area would only be shaded by various optically thin clouds. From an inspection of the ASI images and the GHI readings of the RSIs in the urban area, the ASI network’s GHI map qualitatively represents the distribution of cloud types and resulting attenuation and GHI realistically. While the overall GHI map is complex, the shapes of various optically thick clouds and their shadows are preserved also under these conditions. In the far west, the cloud segmentation of the western-most ASI’s sky image fails partly in the presence of very dark clouds. This western-most ASI alone covers this area. Consequently, no further ASIs can support the cloud detection and the faulty segmentation leads to an overestimation of local GHI.

4.2. Performance Overall and by Temporal Variability Class

In this and in the following subsections, the distinct analysis approaches are validated quantitatively. All data are 1-min-averages. ASI-based estimations of DNI and GHI are created at the rate of 30 s at which ASI images are recorded. A 1-min-average is then created by averaging three estimations. This averaging procedure is used for ASI network, ASI pair and homogeneous persistence. The experimental setup of this evaluation was shown in Figure 1 and described in Section 2. In the quantitative evaluations of this and the following sections, ASI UOL is excluded from the ASI network. This way, the RSI at UOL is located at a distance of 2.4 km from the closest ASI. In this section, for conciseness, only UOL is selected to compare the different approaches by the deviation metrics RMSD and bias.

RMSD and bias are visualized in Figure 10. The first column overall shows the metrics for the complete 92 day dataset. Additionally, the influence of atmospheric conditions on the accuracy of the distinct analysis approaches should be studied. For this purpose, the columns labeled $1\dots 8$ provide the deviation metrics of each approach when the dataset is filtered to times in which the respective DNI variability class has been present in the preceding 15 min. The meaning of the eight DNI variability classes was summarized in Section 2.2 and Table 2. For comparability with previous studies, in this section, temporal variability is calculated on a 15-min basis using the procedure introduced by Nouri et al. [46].

In terms of RMSD, the ASI network performs best under all conditions and clearly also for the overall dataset. As to be expected, the advantage of the ASI network over the state-of-the-art approaches depends on the atmospheric conditions. Homogeneous persistence typically provides rather accurate estimations if temporal variability of DNI is low (classes 1, 2 and 8). Interestingly, the ASI network still brings a certain benefit in these classes.

As shown in Section 2, radiometer station DON is located in the urban center of Oldenburg. The radiometer at UOL is located at a notable distance of 3.8 km to the west of radiometer DON. At a temporal resolution of 1 min, homogeneous persistence based on DON will therefore yield increased deviations in the presence of broken clouds. Meanwhile, UOL is located in an area which is covered well by the ASI pair’s analysis given the cloud heights and sun positions typical to the site. As to be expected based on this, the ASI pair and especially the ASI network have a larger advantage the more variable a situation is. For variability classes $3,4,6$, the ASI network reduces RMSD by more than $100{\mathrm{W}/\mathrm{m}}^2$ in comparison to homogeneous persistence.

Assuming the local climates to be identical at points located at distance of 3.8 km, homogeneous persistence would be expected to have zero bias for the overall dataset and a small bias for any of the variability classes. The remaining bias of homogeneous persistence, especially within the subsets for the variability classes may therefore indicate effects which are related to the limited size of the dataset and which are for the larger part not statistically significant. Having this in mind, all approaches perform similarly in terms of bias. In particular for the overall dataset, all approaches deliver a small bias of $-11$, $-4$ and 2 ${\mathrm{W}/\mathrm{m}}^2$ in the case of the ASI pair, ASI network and homogeneous persistence, respectively.

Figure 11 shows the corresponding evaluation shown in Figure 10 for GHI, however based on the smaller 28 day validation dataset. This evaluation allows for comparing analyses of GHI reported in the literature to the GHI analyses of the present ASI pair and ASI network. Qualitatively, the results for GHI coincide with the ones described above for DNI.

4.3. Influence of Spatial Variability on the Estimations’ Errors

Thus far, we evaluated the analysis approaches’ deviations over temporal variability which was derived from a single RSI. Next, the deviations are studied over spatial variability derived from two separate RSIs. Spatial variability of $k_{DNI}^{*}$ is calculated by a quantitative definition. Marquez et al. [69] defined temporal variability as the RMSD of $k_{DNI}^{*}$ derived by a persistence forecast. Analogously, here, spatial variability V is defined as the RMSD of the $k_{DNI}^{*}$ derived by homogeneous persistence for the location of interest (e.g., UOL or CLO).

Figure 12 is then created to relate the RMSD of an analysis approach to the spatial variability V: The 92 day dataset is grouped into floating 25-min windows. The window corresponding to a timestamp contains the timestamps from 25 min before until the evaluated timestamp. Thus, as many windows are yielded as there are timestamps in the dataset. For each 25-min window, a data pair of V and the RMSD of the ASI-based estimation of $k_{DNI}^{*}$ is calculated. Then, V is discretized into bins with the edges (0, $0.05$, $0.1$, …, 1). The ASI system’s RMSD is averaged over all windows for which V ranges in the respective bin (blue curve in Figure 12). Furthermore, the distribution of the ASI system’s RMSD is evaluated for each bin of V. For this, the ASI system’s RMSD is also discretized into bins with the edges (0, $0.05$, $0.1$, …, 1). Subsequently, the relative frequency of the RMSD bins is plotted for each bin of V (color coding in Figure 12). Accordingly, the sum of the relative frequency equals one in each column of the plot. To evaluate the relevance of situations with a certain level of V, the relative frequencies of the variability bins are also plotted (grey dash-dotted curve). The mean RMSD is not plotted for bins of V with an absolute frequency of less than 100. A similar graph was suggested by Schmidt et al. [21] who related the RMSD of a nowcast to temporal variability.

Figure 12 (top row) evaluates the estimations for UOL. Note that it is desired to receive a low RMSD from an ASI-based system under all conditions also if there is an increased level of variability V. Accordingly, in the plots, increased relative frequencies of the RMSD further below the main diagonal indicate a higher accuracy of the evaluated ASI-based system. As to be expected, both ASI-based estimations increasingly outperform homogeneous persistence (black dashed line) at increased V. As indicated by the plotted relative frequencies of the variability bins (grey dash-dotted line), a large share of the situations corresponds to homogeneous and, in particular overcast conditions, which are typically spatially homogeneous in terms of DNI, exhibiting $V<0.05$. The ASI pair is less accurate than homogeneous persistence up to $V\approx 0.25$ at UOL. Remarkably, the ASI network outperforms homogeneous persistence for all bins of V. The ASI network is noticeably more accurate than the ASI pair for any value of V. Additionally, the ASI network’s advantage over homogeneous persistence increases steadily with variability. Only for bins with very few occurrences can this trend not be verified. For instance, at $V=0.4$, the ASI network reduces the RMSD to $63\%$ of the RMSD of homogeneous persistence. Only for the rare cases of $V>0.7$ can the performance of the ASI network not be evaluated reliably based on the small number of occurrences.

Figure 12 (bottom row) evaluates estimations for CLO. The ASI network performs very similar at CLO and UOL. This is interesting as an ASI is located right near radiometer station CLO, whereas the nearest ASI is located 2.4 km from UOL. Consequently, the ASI network will predict DNI in the whole urban area, which is surrounded by ASIs, at a similarly high accuracy. The ASI pair in turn is clearly less accurate at CLO than at UOL. CLO is located 3.8 km south of the ASIs which form the ASI pair DON-FLE. For the cloud conditions and sun elevations typical to the site, clouds shading CLO are frequently on the edge or outside the ASI pair’s field of view leading to low quality of the ASI-pair-based estimations for this location.

Overall, it is indicated that the ASI network represents spatial variability noticeably more accurately compared to simpler approaches such as an ASI pair or homogeneous persistence based on an RSI. This supports the validity of the presented procedure to model cloud geometry and attenuation. Beyond that, the distribution of the solar resource within the area of a city can be resolved in real time at increased accuracy. As shown, the ASI network can in particular reduce the uncertainty during more extreme situations with increased spatial variability. By a single radiometer, the presence of such a situation may not even be recognized.

4.4. Correlation of Predicted and Measured DNI

The correlation of measured and estimated $k_{DNI}^{*}$ is further analyzed exemplarily for station UOL by scatter-density plots. The panels in the left column of Figure 13 provide the relative frequency for data pairs of measured and predicted $k_{DNI}^{*}$. Homogeneous persistence (top row), the ASI network (center row) and ASI pair (bottom row) are evaluated. Firstly, a notable fraction of the dataset features steady conditions, during which all estimations including homogeneous persistence (Figure 13, top, left) are well aligned with the main diagonal. Furthermore, situations with high cloud coverage and high cloud attenuation are most dominant at the site. This leads to a large share of timestamps for which either measured $k_{DNI}^{*}$ is close to zero (first row in the scatter-density plots) or predicted $k_{DNI}^{*}$ is close to zero (bottom row in the scatter-density plots). In particular in these situations, the ASI network is more accurate. If the measured or the ASI-network-based $k_{DNI}^{*}$ is in the bin $0\dots 0.05$, the counterpart (ASI-network-based, measured $k_{DNI}^{*}$) typically also ranges at a rather small value. For persistence (Figure 13, top row) and for the ASI pair (Figure 13, bottom row), this relationship is less clear. If the measured or the ASI-pair-/persistence-based $k_{DNI}^{*}$ is in the bin $0\dots 0.05$, the counterpart (ASI-pair-/persistence-based, measured $k_{DNI}^{*}$) frequently ranges at increased values of up to 1.

The panels in the right column of Figure 13 are normalized column-wise. The sum of the relative frequencies is one in each column of these scatter-density plots. This way, the plots indicate the approaches’ accuracy at a given reference $k_{DNI}^{*}$. In these plots, all analysis approaches tend to predict a very low $k_{DNI}^{*}$ and thus underestimate the parameter. This effect is again caused by the dominance of very low $k_{DNI}^{*}$ values at the site. The ASI network (center row) is most accurate in this evaluation. The ASI network’s estimation of $k_{DNI}^{*}$ is in general more aligned with the main diagonal, compared to the other approaches. A better alignment with the main diagonal is seen clearest in the range of $k_{DNI}^{*}$ in $0.05\dots 0.5$. The 25th-, 50th-, 75th-percentiles received from the distinct approaches are plotted as black dashed and dotted lines in the panels in the right column of Figure 13. A comparison of these percentiles supports this finding. In addition, the ASI network apparently reduces the frequency of outliers to some extent. This can be seen from the 5th- and 95th-percentiles shown in each panel (black solid lines).

As a summary of the scatter-density plots, the coefficient of determination $R^2$ indicates a noticeably higher correlation of the ASI network’s analysis of $k_{DNI}^{*}$, over the ones from ASI pair and homogeneous persistence, with an $R^2$ value of $0.84$, $0.73$ and $0.63$, respectively. Recall that $R^2$ indicates in this context which fraction of the variance of $k_{DNI}^{*}$ is explained by the respective estimation. Interestingly, homogeneous persistence also yields an increased value of $R^2$. This means that a variation of $k_{DNI}^{*}$ from its mean value over the complete dataset often occurs simultaneously at both radiometers (DON and UOL), in spite of the stations’ considerable distance. This reflects that the overall atmospheric conditions are often homogeneous within the urban area (e.g., rather overcast or rather low cloud coverage). However, in particular, the ASI network can improve noticeably over this benchmark.

5. Conclusions

This publication presented a novel procedure by which an ASI network models clouds and assigns cloud attenuation to the sky over every location in the analyzed area. The approach is suited to combine the information from ASIs which monitor clouds from very different perspectives. First, from the sky images of each ASI, clouds were detected and assigned to classes. Then, a map of attenuation index $AI$ was derived from each ASI image. These maps were combined by an accuracy-weighted average. The average assigned a higher weight to the grid cell of a map, the lower the local uncertainty of the map was. The local uncertainty of each map was determined based on geometric considerations and validation results reported in the literature. The merged map of $AI$ was projected to the ground according to sun position and cloud height. A probabilistic procedure finally adjusted the relationship of $AI$ and cloud attenuation in real-time, evaluating present, recent and more historic maps of $AI$ together with the cloud attenuation derived from the DNI measurement of a rotating shadowband irradiometer (RSI).

The ASI network’s procedure to model clouds, assign cloud attenuation and finally estimate DNI was benchmarked against the homogeneous persistence analysis of DNI, which used only the RSI, and against the analysis from a state-of-the-art ASI pair. By the evaluation of spatially separated measurements, this validation was carried out only based on lead time 0 s, i.e., analysis data. This allowed for examining the accuracy of the cloud modeling alone and to exclude deviations which could be caused by a cloud tracking procedure. The validation dataset consisted of 92 days which were selected to represent the atmospheric conditions in Oldenburg in the year 2020. This validation dataset is large compared to the ones used in many recent validations of ASI-based systems e.g., [21,28,33,40,41,44,45,46]. Still, the results may change to some extent if a different dataset with distinct atmospheric conditions is evaluated.

The statistical evaluation attested a noticeably increased accuracy of the ASI network over an ASI pair as well as over homogeneous persistence for the overall dataset and for any of the studied conditions. For example, for the overall dataset and DNI, the ASI network yielded an RMSD of 136 W/m${}^2$, whereas the ASI pair and homogeneous persistence yielded an RMSD of 173 W/m${}^2$ and 213 W/m${}^2$, respectively. Atmospheric conditions were distinguished by variability classes and also by a quantitative definition of variability. An inspection of the coefficient of determination ($R^2$) and of scatter-density plots confirm that the ASI network’s estimations ($R^2=0.84$) are noticeably more correlated with the reference measurement of DNI than the other systems’ estimations (ASI pair: $R^2=0.73$; homogeneous persistence: $R^2=0.63$). The validation was also evaluated GHI to allow a further comparison with the literature. Thus, the quantitative results of Section 4.2–Section 4.4 support the indication from the initial evaluation of exemplary situations in Section 4.1: The ASI network locates cloud shadows more accurately and also performs better during complex cloud conditions providing a more graduated estimation cloud attenuation. At the same time, the ASI network detects slight variations of the cloud attenuation which brings an advantage over homogeneous persistence even in comparably homogeneous conditions. Therefore, the ASI network has an advantage over the reference approaches when variations of DNI need to be predicted.

Besides DNI, GHI was also estimated. It should be noted that GHI was calculated from DNI and DHI, which was modeled as spatially homogeneous. The modeling of DHI used in this study can be enhanced using the technique of Blum et al. [37]. This would bring an additional advantage to the ASI network. In the experiments, the ASI network was compared to an exemplary ASI pair. Schmidt et al. [21] presented a validation which focused on the spatio-temporal performance of an ASI-based analysis of solar irradiance. The ASI-based system studied by Schmidt et al. [21] with its single ASI in combination with a ceilometer is expected to be comparable to the ASI pair used as benchmark here. The discussed effects seen in this validation are qualitatively in agreement with Schmidt et al. [21], who classified cloud conditions by the dominant cloud type instead of by variability classes. Based on that study, the increased negative bias seen for the ASI pair for the more variable classes $4,6,7$ is reasonable as perspective errors become more pronounced under these conditions. The ASI network reduced this bias to some extent, which also indicates that the ASI network reduces perspective errors.

Regarding the RMSD of estimated GHI, Schmidt et al. [21] found that their ASI system outperformed homogeneous persistence more clearly at large distances from the ASI and at increased variability. This effect is expected as in general the cross-correlation of irradiance in two points reduces if their distance and spatial variability increase. In that study, the ASI did not outperform homogeneous persistence at any distance for clear sky conditions or if cloud types related to low variability were present. In the present evaluation, a similar effect is found while the ASI pair still slightly outperforms homogeneous persistence in variability classes $5,8$. On the other hand, for more variable conditions, Schmidt et al.’s ASI [21] outperformed homogeneous persistence at distances larger than 1…6 km depending on the conditions. For clouds of type cumulus and at a distance from the ASI of around 3.8 km, this study found an RMSD of homogeneous persistence of around 240 W/m${}^2$, whereas the ASI yielded around 220 W/m${}^2$, both for GHI. Here, the ASI pair has a similar or rather increased accuracy, with an RMSD typically reduced by more than $10\%$ over homogeneous persistence. The strong advantage found here for the ASI network is therefore expected to also hold in comparison to other ASI-based systems besides the specific ASI pair evaluated here.

This study represents one of the first research works on the use of an ASI network to analyze solar irradiance. There is a number of enhancements that are not implemented and investigated yet. Eye2Sky was still under construction during our experiments. The cloud modeling procedure can profit from Eye2Sky’s extension. Thus, smaller errors and a higher spatial coverage may be achieved by the ASI network in the future. Furthermore, only one meteorological station was included in the analysis, and the remaining stations were reserved for validations. In the regular operation, the remaining meteorological stations of Eye2Sky should be included to further increase the ASI network’s accuracy. In a second step, Eye2Sky could be extended cost-effectively by one (e.g., photodiode-based) pyranometer per ASI station. Together with the method of [37] to measure DHI by the ASI image, this would allow for determining DHI, DNI and with it cloud attenuation comparably accurately in many points of the Eye2Sky region. These additional measurements could further support the present approach to estimate cloud attenuation. Additionally, with the help of a larger number of dispersed pyranometers, the present assignment of cloud attenuation based on the local attenuation index by a piece-wise linear fitted function may be refined by a machine learning model.

The approach presented here is versatile and is promising to predict further probabilistic quantities. Besides expected attenuation, the distribution of attenuation may also be estimated in the future. Firstly, by modifying the loss function, the approach can predict e.g., the spatial distribution of attenuation quantiles. Secondly, by logging and fitting to other quantities besides cloud attenuation and modifying the constraints to the optimization if required, the spatial distribution of e.g., ramps of photovoltaic production could also be predicted.

The ASI network’s cloud modeling procedure will be used to nowcast solar irradiance and photovoltaic production. In a future study, we will present and validate the final nowcasting system. Additionally, we will present combination strategies of the ASI network’s and satellite-based analyses and nowcasts to provide grid operators and energy traders with accurate nowcasts for the very short term and up to multiple hours ahead. Furthermore, it may be considered to use the ASI network’s analyses of cloud attenuation as input to NWP models.