Surface Diffuse Solar Radiation Determined by Reanalysis and Satellite over East Asia: Evaluation and Comparison

Abstract

Recently, surface diffuse solar radiation (R_dif) has been attracting a growing interest in view of its function in improving plant productivity, thus promoting global carbon uptake, and its impacts on solar energy utilization. To date, very few radiation products provide estimates of R_dif, and systematic validation and evaluation are even more scare. In this study, R_dif estimates from Reanalysis Fifth Generation (ERA5) of European Center for Medium-Range Weather Forecasts and satellite-based retrieval (called JiEA) are evaluated over East Asia using ground measurements at 39 stations from World Radiation Data Center (WRDC) and China Meteorological Administration (CMA). The results show that JiEA agrees well with measurements, while ERA5 underestimates R_dif significantly. Both datasets perform better at monthly mean scale than at daily mean and hourly scale. The mean bias error and root-mean-square error of daily mean estimates are −1.21 W/m² and 20.06 W/m² for JiEA and −17.18 W/m² and 32.42 W/m² for ERA5, respectively. Regardless of over- or underestimation, correlations of estimated time series of ERA5 and JiEA show high similarity. JiEA reveals a slight decreasing trend at regional scale, but ERA5 shows no significant trend, and neither of them reproduces temporal variability of ground measurements. Data accuracy of ERA5 is more robust than JiEA in time but less in space. Latitudinal dependency is noted for ERA5 while not for JiEA. In addition, spatial distributions of R_dif from ERA5 and JiEA show pronounced discrepancy. Neglect of adjacency effects caused by horizontal photon transport is the main cause for R_dif underestimation of ERA5. Spatial analysis calls for improvements to the representation of clouds, aerosols and water vapor for reproducing fine spatial distribution and seasonal variations of R_dif.

1. Introduction

Surface solar radiation (R_s) drives the global energy, water and carbon cycles of by affecting sensible and latent heat fluxes, longwave emission, water vapor and circulations in the atmosphere and the ocean [1,2,3]. Determining the variations of R_s is essential for understanding global climate changes, particularly the rate of global warming and its effects on glacial melt and sea level rise [4,5]. R_s data with different spatiotemporal resolutions are urgently required in diverse application fields, such as global numerical weather prediction, agricultural meteorology, climate monitoring and solar electricity. Moreover, the accuracy of R_s data greatly influences simulations of runoff, evapotranspiration, gross primary productivity, growth and yield of crops [6,7,8,9].

In addition to R_s, surface diffuse solar radiation (R_dif) takes on greater importance in monitoring and modeling ecosystem carbon uptake [10]. R_dif tends to increase plant productivity, as it enhances light use efficiency of plants by penetrating more radiation into deeper canopies, thus improving photosynthesis in shaded leaves [11,12,13]. It was reported that changes of R_dif affect the global land carbon sink [11], according to a high-quality R_dif dataset for quantifying its effects on carbon dynamics of terrestrial ecosystems. The fraction of R_dif is also a necessary input to agricultural models, such as the Soil Water Atmosphere Plant (SWAP), Forest Biomass, Assimilation, Allocation, and Respiration (FöBAAR) and Yale Interactive Terrestrial Biosphere (YIB), for early assessment of crop yield [14,15] or radiation-use efficiency of forests [16]. Besides, the spatially continuous high-resolution hourly ratio of R_s and R_dif is required for a comprehensive assessment of the potential of rooftop solar photovoltaics and policy-making regarding the renewable energy sector [17].

Currently, R_s products are available from four common sources, namely direct measurements of surface radiation networks [3], simulations based on radiation transfer models [18,19], estimates from reanalysis systems [18,20,21] and retrievals from satellite observations [10,22,23,24,25]. Direct surface measurements are regarded as a reliable reference for data validation from simulations, reanalysis and satellite retrievals [26,27,28,29]. However, R_dif is very rare among these products, for example, R_s measurements are attainable at 119 stations in China while only 17 of them measure R_dif [26,30]; the Global Land Surface Satellite (GLASS) provides global 5-km resolution, 3-h interval R_s but lacks an R_dif map [25]. Nonetheless, many algorithms have tried to determine the fraction of R_dif, [10,31,32]. Greuell et al. [31] retrieved global, direct and diffuse irradiance (3 km, 15 min) from Spinning Enhanced Visible and Infrared Imager (SEVIRI) observations through a physics-based and empirically adjusted algorithm. Ryu et al. [10] produced incident shortwave radiation (SW), photosynthetically active radiation (PAR) and diffuse PAR datasets (5 km, 4 day) by combining an atmospheric radiative transfer model with an artificial neural network (ANN) based on Moderate Resolution Imaging Spectroradiometer (MODIS) atmosphere and land products. To date, mature kilometer-scale hourly radiation datasets (including R_dif) with global and multiyear coverage are still rare [26,33]. Most products are generated over specific regions like Europe, North America and China. To the best of our knowledge, only two products provide multiyear hourly R_dif over East Asia, i.e., Reanalysis Fifth Generation (ERA5) provided by the European Center for Medium-Range Weather Forecasts (ECMWF) and satellite-based products produced by Jiang et al. [22] (hereafter called JiEA for short).

However, these radiation products generally contain large uncertainties. The reported root-mean-square error (RMSE) of instantaneous R_s retrievals under all-sky conditions range from 60 to 140 W/m² (~15%–30%) depending on local cloud climatology [33]. In addition, multisource products usually show inconsistent temporal trends and spatial distributions [29,34], which could hamper their applicability for assessing global brightening or dimming and local climate responses to radiation changes [5,35]. Therefore, it is necessary and important to compare different products and understand their discrepancies. Zhang et al. [29] compared four satellite products of R_s using comprehensive ground measurements at stations around the world and found that satellite estimates capture the seasonal variations of R_s well and have acceptable data accuracy at the monthly time scale, with an overestimation of approximately 10 W/m². Zhang et al. [36] evaluated two R_s estimates of global reanalyses using homogenized surface measurements in China and pointed out the pronounced overestimation of the reanalyses. The significant spatiotemporal difference of data accuracy mainly results from atmospheric factors, including cloud coverage, aerosol optical depth and water vapor content. There are large numbers of references that concentrate on the data accuracy of R_s, but to date very few studies have been devoted to the evaluation of R_dif.

The purpose of this study is to evaluate and compare R_dif estimates from ERA5 and JiEA using surface in situ measurements and to investigate the spatial pattern and seasonal variations of R_dif over East Asia. The reliability of different data is discussed at both the site level and the regional scale in combination with the spatial distribution of atmospheric factors that mostly relate to the retrievals of solar radiation. This study provides a reference for rational use of these data and opens new perspectives for improving R_dif estimation.

This paper is organized as follows. The ground measurements and diffuse radiation products used are briefly described in Section 2. Section 3 explains various validation metrics and the method for comparative analysis. Section 4 presents the results of site-level validation and analysis of spatiotemporal deviations at different time scales, followed by a discussion on the reliability of these data, especially concerning their spatial pattern, in Section 5. A conclusion is finally given in Section 6.

2. Data

2.1. Ground Measurements

The ground measurements used to evaluate R_dif estimates are obtained from two data centers: the World Meteorological Organization’s (WMO) World Radiation Data Center (WRDC) (22 stations) and that of the China Meteorological Administration (CMA) (17 stations). Figure 1 shows the geographical distribution of the selected 39 stations from WRDC and CMA, with detailed information provided in Table S1.

The WRDC is one of the recognized World Data Centers sponsored by the WMO, which centrally collects and archives radiometric data from the world to ensure the availability of these data for research by the international scientific community. Daily and monthly totals of surface energy components such as global radiation (i.e., R_s), diffuse radiation (i.e., R_dif) and radiation balance are available from the official website (http://wrdc.mgo.rssi.ru/) after a simple registration process. Daily totals of global and diffuse radiation are determined where ground measurements for all time intervals of the daytime are available, along with an auxiliary procedure to avoid undue losses due to the gaps in the data for sunrise and sunset hours. Monthly totals are the sum of the entire daily totals of the month. If less than ten days with missing records exist, a monthly mean of the available daily records is calculated, then a monthly total is calculated by multiplying the monthly mean by the number of days in the calendar month. A monthly value is not provided if missing records within the month exceed ten days. A subset of 22 WRDC stations (red circles in Figure 1), which provide at least one-year monthly series of diffuse radiation within the period from 2007 to 2014, was selected for this study.

The CMA Meteorological Information Center have released daily and monthly meteorological data at 122 routine weather stations. Radiation-related elements include net radiation, downward shortwave radiation (i.e., R_s), reflected shortwave radiation and diffuse radiation (i.e., R_dif). R_dif measurements are conducted at 17 stations (blue triangles in Figure 1). The procedure to calculate daily and monthly totals is the same as that adopted by WRDC. Additional quality control measures before release include a spatial and temporal consistency check and manual inspection and correction. Furthermore, hourly measurements of these stations are attainable from National Meteorological Science Data Center (http://data.cma.cn/) on reasonable request. Herein, hourly measurements in 2007 and 2008 and daily/monthly measurements from 2007 to 2014 of diffuse radiation at these 17 CMA stations are available for evaluation.

R_s and R_dif at stations are widely measured through thermoelectric pyranometer, which has a spectral response of 0.3–3.0 μm, a thermal effect of less than 5% and an annual stability of about 5%. Pyranometers are exposed to the sun to measure R_s. For measuring R_dif, pyranometric sensors are shaded by an additional component (e.g., shadow-ball or rotating shadow band) to prevent direct solar radiation from reaching the sensor. The shading mechanism hides the minimum of sky outside the small solid angle of the sensor to receive the maximum R_dif from the whole sky dome.

Previous studies point out that systematic errors are very common in radiation measurements due to equipment failure and operational problems [30] and that it is necessary to examine measured values carefully before subsequent utilization [28,29,37]. In this study, we first applied the physical threshold test [38] and then the method based on reconstructed data [29] to the measurements of R_s associated with the selected records of R_dif for further quality control. If the measured value of R_s failed to pass the quality check, the corresponding R_dif was eliminated. In addition, R_dif should not be larger than R_s. The numbers of valid records from each station at hourly, daily and monthly scales are listed in Table S1.

2.2. Diffuse Radiation Products

Two datasets that provide estimates of R_dif over East Asia are evaluated and compared in our study, i.e., the latest global climate reanalysis provided by ECMWF [21] and satellite-based products produced by Jiang et al. [22].

ECMWF Reanalysis Fifth Generation (EAR5) is the fifth-generation ECMWF atmospheric reanalysis of the global climate to replace the old ERA-Interim. It is produced using a 4D-Var assimilation system of ECMWF’s Integrated Forecast System (IFS), namely IFS Cycle 41r2, which guarantees significant increase in forecast accuracy and computational efficiency. The advanced system is also combined with vast amounts of historical observations to generate globally consistent time series of multiple climate variables. ERA5 provides hourly estimates of many atmospheric, land-surface and sea-state parameters together with their uncertainties at reduced spatial and temporal resolutions. The parameters used in this study involve “surface solar radiation downwards” and “total sky direct solar radiation at surface”, which represent the amount of shortwave radiation (surface direct and diffuse solar radiation) and the amount of direct radiation reaching the surface of the Earth, respectively. Estimates of R_dif can be derived by subtracting total sky direct solar radiation from surface solar radiation downwards. To date, these hourly data are available in the Climate Data Store (https://climate.copernicus.eu/climate-reanalysis) on regular latitude–longitude grids at 0.25° × 0.25° resolution from 1979 to present.

Satellite-based diffuse radiation products (hereafter, called JiEA for short) are from the work of Jiang et al. [22], where a deep learning algorithm was developed to retrieve R_s from Multifunctional Transport Satellites (MTSAT) data. They concentrate on overcoming the negative impact of spatial adjacency effects on R_dif estimation through convolutional neural networks (CNNs). Spatial adjacency effects refer to the phenomena that some photons out of the field of view are reflected by the surface then scattered by the atmosphere, thus finally entering into the field of view to change the amount of solar radiation within the field of view. CNN is used to handle this effect by gaining knowledge of the spatial distribution of clouds/aerosols from satellite image blocks. This algorithm is originally designed for estimates of R_s and further extended through a transfer learning approach for estimates of R_dif. Currently, a dataset from 2007 to 2018 is freely available from Pangaea at https://doi.pangaea.de/10.1594/PANGAEA.904136 [39]. This dataset provides gridded estimates of R_s and R_dif at 0.05° × 0.05° resolution within 71°–141°E and 15°–60°N, mainly covering East Asia. In view of the difference of spatial coverage from ERA5, East Asia in this study is referred to as the maximum overlapped extent of the two datasets.

3. Methods

3.1. Validation Metrics

Ground measurements are regarded as the reference for evaluation of R_dif from different datasets. To quantify the accuracy of R_dif estimates, a set of metrics including Pearson correlation coefficient (R), (relative) mean bias error (MBE, rMBE), (relative) mean absolute bias error (MABE, rMABE), (relative) root-mean-square error (RMSE, rRMSE), bias and absolute percentage bias (APE) are used. These metrics are defined as follows:

$$\mathrm{R}=\frac{{{\displaystyle \sum }}_{i=1}^n\left({\widehat{y}}_i-\overline{\widehat{y}}\right)\left(y_i-\overline{y}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left({\widehat{y}}_i-\overline{\widehat{y}}\right)}^2}\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(1)

$$\mathrm{MBE}=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n\left({\widehat{y}}_i-y_i\right),\mathrm{rMBE}=\mathrm{MBE}/\overline{y}$$

(2)

$$\mathrm{MABE}=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n\left|{\widehat{y}}_i-y_i\right|,\mathrm{rMABE}=\mathrm{MABE}/\overline{y}$$

(3)

$$\mathrm{RMES}=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2},\mathrm{rRMSE}=\mathrm{RMSE}/\overline{y}$$

(4)

$$\mathrm{bias}={\widehat{y}}_i-y_i,i=1,2,\dots ,n$$

(5)

$$\mathrm{APE}=\left|\frac{{\widehat{y}}_i-y_i}{y_i}\right|,i=1,2,\dots ,n$$

(6)

where $n$ is the number of data samples, $y$ means ground-measured R_dif values whose mean value is $\overline{y}$ and ${\widehat{y}}_i$ represents corresponding estimated values whose mean value is $\overline{\widehat{y}}$. R measures the strength and direction of a linear relationship between ${\widehat{y}}_i$ and $y_i$. R ranges from −1 to 1, and a closer value to 1 indicates a strong positive linear relationship between estimated and measured R_dif. MBE is the mean difference between compared variables, representing the systematic error of R_dif products to under- or overestimate. MABE is the mean of absolute differences between ${\widehat{y}}_i$ and $y_i$ and gives the average magnitude of under- or overestimation of R_dif compared to ground measurements. RMSE represents the standard deviation of the differences between ${\widehat{y}}_i$ and $y_i$. Compared to MABE, RMSE is more sensitive to outliers. To eliminate the scale-dependency (i.e., influence from numbers of samples) of these metrics, their relative values are also available through dividing the original values by the mean of the reference measurements. For temporal and spatial evaluation and comparison of R_dif, these metrics were calculated according to different grouping strategies, i.e., 12 months, 8 years, and 39 stations.

In addition, we demonstrated the probability density functions (PDFs) of bias and cumulative distribution functions (CDFs) of APE for comparison of data accuracy within different tolerance ranges of deviations. The bias indicates the under- or overestimation of each estimated R_dif value. PDF is a statistical expression that defines the probability distribution of a random variable. When PDF is graphically portrayed, the total area of an interval (expressed as bin width during statistical process) under the curve equals the probability of the random variable occurring. Herein, PDF determines the likelihood of calculated bias falling into a specific range. APE expresses the deviation of each estimate in percentage, and the associated CDF gives the proportion of APE with values less than a certain threshold.

3.2. Time Series Decomposition

Time series decomposition is a common way to identify the change of different components of interest [35,40], and it involves separating a time series into several distinct components. Three components are typically of interest, i.e., the trend, seasonal periodicity and stochastic irregular anomalies. The additive functional form has been widely used to observe the bias and errors of R_s [41,42]. It assumes that a monthly R_dif time series, $R\left(t\right)$, can be decomposed into the low-frequency climatological contributions, consisting of the long-term trends $\overline{R}\left(t\right)$, the climatological seasonal cycles $\tilde{R}\left(t\right)$ and high-frequency deviations $R^{\prime }\left(t\right):$

$$R\left(t\right)=\overline{R}\left(t\right)+\tilde{R}\left(t\right)+R^{\prime }\left(t\right)$$

(7)

where $t$ defines the length of R_dif time series. The trends $\overline{R}\left(t\right)$ describe the gradual variations and can be estimated by using moving averages or parametric regression models [3,43]. The seasonal cycles $\tilde{R}\left(t\right)$ capture level shifts that repeat systematically within the same period between successive years. The anomalies $R^{\prime }\left(t\right)$ exhibit autocorrelation and cycles of unpredictable duration. For identifiability from $\overline{R}\left(t\right)$, $\tilde{R}\left(t\right)$ and $R^{\prime }\left(t\right)$ are assumed to fluctuate around zero.

Since the periodicity of R_dif data is monthly, a 13-term moving window is used for estimating the long-term trend by setting weight 1/24 for the first and last terms and weight 1/12 for the interior terms. Then $\overline{R}\left(t\right)$ is removed from the original series to obtain the detrended time series. Assuming a stable seasonal component that has constant amplitude across the series, $\tilde{R}\left(t\right)$ can be determined by averaging detrended time series for each month over the whole period, i.e., by averaging all of the January values, then all of the February values and so on for the remaining months. Finally, $R^{\prime }\left(t\right)$ is determined by removing $\overline{R}\left(t\right)$ and $\tilde{R}\left(t\right)$ from the original time series. If only $\tilde{R}\left(t\right)$ is removed, the rest is called a deseasonalized time series $R_{\mathrm{d}}^{\text{'}}\left(t\right):$

$$R_d^{\text{'}}\left(t\right)=\overline{R}\left(t\right)+R^{\prime }\left(t\right)=R\left(t\right)-\tilde{R}\left(t\right)$$

(8)

In this study, the similarity of two time series from different datasets was measured by the Pearson correlation coefficient (Equation (1)) of their corresponding $R_{\mathrm{d}}^{\text{'}}\left(t\right)$. The significance at the 95% confidence level was obtained through an F-test on the linear regression model of the two time series. In particular, we considered the increasing/decreasing trend of different components over time. We fit a linear regression model between the components and associated time index, and the slope coefficient was regarded as the indicator of increasing/decreasing trend versus time. In addition, the 95% confidence bounds of the slope coefficient were given by the F-test on the regression model.

4. Results

4.1. Evaluation Against Ground Measurements

The hourly R_dif estimates of ERA5 and JiEA are compared with the quality-controlled ground measurements from 17 CMA stations. It is stressed that such comparisons are conducted at their original spatial resolutions. ERA5 has an overall correlation coefficient R of 0.71, a negative bias of 29.69 W/m², an MABE of 63.83 W/m² and an RMSE of 92.29 W/m², whereas these values are 0.85, 8.54 W/m², 50.43 W/m², and 66.36 W/m² for JiEA. It is apparent that R_dif estimates of JiEA correlate better than ERA5 with in situ measurements at the selected stations. Evidence comes from their density scatterplots; Figure 2a shows that more points are concentrated on the lower side of 1:1 line, while in Figure 2b almost all data pairs are symmetrically distributed around the 1:1 line. This is also the reason why ERA5 exhibits a relatively serious underestimation of R_dif, with an rMBE of 18.4%. For low-radiation estimation, ERA5 performs better than JiEA, as high-density (red) scatters are on both sides of l:1 line in Figure 2a, while they are obviously inclined to the upper side in Figure 2b. The PDF of JiEA resembles the Gaussian distribution with a mean slightly larger than zero, coinciding with the observed overestimation and the density scatterplots. Although the peak of ERA5′s PDF nears zero, the curve is significantly asymmetric, revealing a high probability of underestimation. The performance of JiEA is superior to ERA5 when setting the tolerance of absolute percentage bias lower than 0.51, while few estimates of ERA5 would exceed one (Figure 2d). At hourly scale, the time systems of R_dif estimates and ground measurements deserve attention [44]. For example, measurements at some stations might be recorded according to the local time and then converted to universal time (usually the time system of satellite acquisition and climate reanalysis) when stored into a standard database. There is consequently a change of original values due to a resampling of data series in time; in any case, returning to the original values is impossible due to the asystematic shift of a fraction of an hour before and after conversion. That change would have negative impacts on evaluation results at hourly scale. It is pointed out that this impact does not hold if we deal with daily, monthly or yearly averages or sums of solar radiation data.

Daily mean R_dif estimates are evaluated using measurements at all 39 stations from WRDC and CMA. As indicated by various metrics, the overall accuracy of JiEA exceeds that of ERA5. Similar differences to hourly-scale evaluation are observed between ERA5 and JiEA from Figure 3a–d. The PDF of JiEA is symmetrically distributed with a zero mean, while that of ERA5 indicates a high probability of a negative bias. The proportion of JiEA samples whose accuracy is higher than ERA5 reaches up to 84% (Figure 3d). However, some apparently questionable estimates exist for the results of JiEA (e.g., scatters at the lower right corner of Figure 3b). Failure of these estimates might result from the difference between ground and satellite measurements, in that ground measurements represent an average state over the sample time interval whereas only instantaneous state is manifested by satellite images [45]. For instance, when coming across fast-moving clouds, a satellite sensor may scan a cloudy sky, but ground stations are covered by cloud shadows only within a momentary period (less than sample time interval). In this case, ground measurements would be greater than satellite-based estimates. The same evaluation is conducted in four typical regions, i.e., Eastern China, Mongolian Plateau, Tibetan Plateau and Deccan Plateau, whose boundaries are defined in Figure 1. Data accuracy of JiEA is always better than ERA5 except for the Mongolian Plateau (Figure S1). Both ERA5 and JiEA achieve more accurate estimates of R_dif over the Mongolian Plateau and Tibetan Plateau than over other regions. Particularly, ERA5 seriously underestimates R_dif over the Deccan Plateau and shows a large difference compared to JiEA. This is probably due to their inappropriate representation or modeling of aerosols, clouds and their interactions with solar radiation in the atmosphere [2,46,47] for Eastern China and India where rapid economic development and high-speed urbanization have caused heavy pollution [27,48]. Besides, frequent cloudy and rainy weather in India and South China also leads to the difficulty in estimating R_dif [49,50].

The differences in data accuracy between JiEA and ERA5 are more obvious at monthly mean scale (Figure S2). JiEA almost achieves zero deviation on average (a negative MBE of 0.92 W/m² and zero-centered PDF). ERA5 underestimates most parts of the selected samples, and the largest underestimation is greater than 50 W/m². The accuracy of 94% of samples exceeds ERA5 with absolute percentage bias lower than 0.39. We also depict the PDF and CDF of JiEA after upscaling the original monthly data to 0.25° grids (dotted black lines in Figure S2c,d) and observe no significant change comparing to the original ones, suggesting that the above comparisons are hardly affected by the different spatial resolutions of the two datasets. As pointed out by previous studies [51,52,53], the evaluation results are likely affected by the spatial representativeness of ground measurements. The comparison of Figures S2b and S3 indicates that ground R_dif measurements at the selected stations are more representative for 0.05° × 0.05° spatial grids than 0.25° × 0.25°. In this regard, the deviations in comparison to ground measurements are not completely attributed to the performance of models or algorithms [33,44,54].

The monthly maximum (minimum) of R_dif appears in June/July (December) and approximates to 110 (47), 90 (36) and 107 (44) W/m² for measurements, ERA5 and JiEA, respectively. It is clear that results from JiEA are closer to the measured values than those of ERA5. At the selected 39 stations, the measured yearly R_dif is 79.78 W/m² on average, and the ratio of R_dif to R_s (173.97 W/m²) equals 45.86%. The estimates of JiEA (R_dif: 78.41 W/m², R_s: 171.95 W/m², R_dif ratio: 45.60%) are basically consistent with the measurements; on the contrary, ERA5 seems to underestimate R_dif as well as its fraction (R_dif: 63.26 W/m², R_s: 190.10 W/m², R_dif ratio: 33.28%). For the whole East Asia region, JiEA provides a mean R_dif of 71.89 W/m², accounting for 41.84% of R_s (171.81 W/m²), while ERA5′s estimate of R_dif (63.40 W/m²) only accounts for 34.78% of R_s (182.28 W/m²).

4.2. Temporal Difference of Data Accuracy

The temporal stability of data accuracy is critical for detection of the long-term trend of time series products [55,56]. One of the advantages of reanalysis products is their potential to provide geographically and physically consistent estimates of regional climate changes [57,58,59]. We illustrate the average seasonal (Figure 4a) and interannual (Figure 4b) variations of different metrics to examine the temporal consistency. Considering that the absolute amount of R_dif varies greatly among months and years, relative errors (rMABE and rRMSE) are discussed. Although the overall accuracy of ERA5 is inferior to JiEA, ERA5 shows a good robustness in time. The change of R is less than 0.1 and those of rMABE and rRMSE are less than 5% for ERA5, while the maximum disparity is doubled for JiEA. Snow/ice cover is the factor most likely to be responsible for the worse accuracy of satellite-based estimates in winter. The similarity of spectral and physical properties of cloud and surface snow covers hampers the identification of clouds and retrievals of cloud optical depths over snow/ice surface [60,61], subsequently resulting in a lower accuracy in satellite estimation of solar radiation [18,24,46]. Due to the lack of a physical basis, machine learning based methods always suffer from their dependence on the representativeness of training samples, and consequently their generalizability is limited [33,62,63]. As shown in Figure 4b, although a perfect performance is achieved in 2008, when the training set for the deep network behind JiEA is constructed, data accuracy of other years becomes much worse, with a maximum disparity of 8% with respect to rRMSE. On the contrary, the accuracy of ERA5 is relatively stable over time.

To examine whether the two datasets can capture the changing trend and seasonal cycles of R_dif, we pick out 24 stations that provide relatively complete monthly measurements for time series analysis. Very few missing values are substituted by the average of existing records of the same month. The results of time series decomposition are presented in Figure S4. ERA5 and JiEA reflect roughly similar trends that are consistent with the measured ones at most stations. However, there are issues with significantly different and even contradictory trends, such as for Ulan-Bator, Harbin, Lhasa and Urumqi. We speculate that this may be due to the combined effects of local pollution and climate change. For example, the increase of particles and aerosols in the atmospheric layer near the surface caused by air pollution actually leads to greater measured R_dif at stations, but reanalysis and satellite-based estimates do not respond to such pollution because of information loss. With respect to time series anomalies, ERA5 tends to level off, while JiEA and measurements exhibit stronger fluctuations. Specifically, the observed increasing/decreasing trends from estimates are identical with measurements at 14 and 12 stations for ERA5 and JiEA, respectively, but none of them passes the significance test at the 95% confidence level. This confirms the conclusion that neither satellite retrievals nor reanalysis can accurately reproduce the decadal variability and trend revealed by combining homogenized measurements and sunshine-duration-derived R_s [29,64]. The discrepant trends between estimates and measurements during the validation period might be attributed to inappropriate responses of models to undulated aerosols over these regions [28]. In view of aerosol’s dominant contribution to the decadal trends in R_s [65], an inclusion of aerosol variability in the reanalysis and satellite retrieval is necessary for an accurate detection of changes of R_dif, which result from the scattered solar radiation on particles in the atmosphere (aerosols).

At regional levels, the trends of ERA5 remain highly constant for all regions except the Deccan Plateau, while JiEA shows slightly decreasing trends with slopes versus times ranging from 0.03 to 0.09 W m⁻² yr⁻¹ (Figure S5). The results of JiEA are in line with the reported insignificant trend (slope = −0.03, p > 0.1) of R_s over China between 2001 and 2016 [10]. Regardless of the large difference with the measured series, the deseasonalized time series of ERA5 and JiEA correlate for most parts of East Asia (Figure S5f). The phenomenon of estimates being weakly correlated with measurements reflects the difficulty in reproducing temporal variations at fine spatial resolutions and implies that the constructed variations of R_dif from gridded products are reasonable at coarse scales.

4.3. Spatial Distribution of Biases

The data accuracy shows notable spatial differentiation at the selected stations (Figure 5). For ERA5 data, latitude holds a positive correlation with R and a negative correlation with rMABE and rRMSE (Figure 6a), with correlation coefficients of 0.70, −0.75 and −0.55, respectively. This latitudinal dependency is prevalent among radiation datasets, such as GEWEX-SRB [66], ISCCP-FD [29] and UMD-SRB [20]. Serious underestimation occurs at stations on the Deccan Plateau, followed by the Eastern China (Figure S6a), which might be attributed to the inappropriate aerosol representations [29]. Local air pollution has caused high aerosol concentrations in these regions [30,67], but representation of aerosol absorption under a cloud layer is not included in current algorithms [20,29]. Although dimming of R_s is observed in Eastern China, absorption and scattering of solar radiation by aerosols and clouds increase the fraction of diffuse radiation [30,68].

With respect to JiEA data, the latitudinal dependency is not as pronounced as for ERA5 (Figure 5 and Figure 6). R shows a positive correlation with latitude (correlation coefficient equals 0.64), along with similar spatial distribution to ERA5. However, rMABE and rRMSE are positively correlated with latitude. The linear relationship is greatly weakened when only considering CMA stations (brown dots in Figure 6a). Moreover, the spatial difference of rMBE is almost negligible in China (Figure S6b). These results support that this deep learning based method results in high robustness in space [22]. As pointed out by previous studies [22,23,24], the ability of machine learning methods depends on the representativeness of training samples; therefore, some large deviations appear at stations outside China, such as Irkutsk, Omsk, Fukuoka and Ishigakijima. It is surprising that JiEA provides satisfying R_dif estimates at stations in India; this might be due to the similarity in atmospheric scattering mechanisms with South China.

5. Discussion

The data accuracy of ERA5 and JiEA is evaluated using ground measurements, and the results show that both datasets provide acceptable estimates of R_dif at the selected stations. For research on global climate change, homogeneous data with global coverage including focal hotspot regions like the Arctic, the Antarctic, the Tibet Plateau and others are always required [69]. In the field of solar energy applications, finer spatial resolution and wider ranges of temporal resolution are usually emphasized [17,62]. The WMO Observing System Capability Analysis and Review Tool (OSCAR) collects user-defined quantitative requirements with respect to the spatial resolution, timescale, coverage and quality for downward short-wave irradiance at the Earth’s surface (Table S2). Although it is reported that the overall accuracy of R_s has entered the gate of intermediate level requirements (Break. in Table S2) [22,26,33,62], R_dif estimates from ERA5 and JiEA can only meet the minimum requirement (Thres. in Table S2) at monthly mean scale according to above evaluation results.

Last but not least, we concentrate on the spatial distribution of R_dif over East Asia. We show the annual average from 2007 to 2014 of R_dif estimates and its fraction (relative to R_s) from ERA5 and JiEA at 0.25° grids (Figure 7). It is apparent that the two datasets illustrate significantly different spatial patterns, with the largest differences on the Tibetan Plateau, Deccan Plateau and Taklimakan Desert (Figure 7e). Both the amount and ratio of JiEA are in line with the application level products of SolarGIS [70] over all of East Asia (Figure S7a,b). In contrast, the R_dif distribution of ERA5 is in agreement with the diffuse photosynthetically active radiation (PAR) from Breathing Earth System Simulator (BESS) [10], except at low latitudes (Figure S7c). In the absence of densely distributed in situ measurements, it is difficult to judge which pattern is reliable. However, subjective judgements can be made according to common sense in combination with the spatial mappings of atmospheric factors mostly related to the estimate of R_dif (Figure 8). Previous studies show that cloud parameters (cloud coverage and optical thickness) and aerosols are two of the most important factors for R_s estimation [2,71,72,73,74]. The amount of water vapor plays a vital role in radiation scattering and leads to altitudinal disparity [75,76,77]. Herein, we take MODIS-derived parameters as references, including cloud fraction (CF), cloud optical thickness (COT), aerosol optical depth (AOD) and water vapor.

Both BESS and SolarGIS confirm that ERA5 seriously underestimates R_dif at low latitudes. The large amount of R_dif is the combined result of high-density downward radiation and strong scattering effects of water vapor. An additional contribution comes from aerosols for the southern Himalayas and from clouds (high CF and middle COT) for South China. The amount of R_dif from JiEA is approximately equal to that from SolarGIS, but their diffuse ratios are discrepant, implying an underestimation of R_s by JiEA. In Sichuan Basin and the middle and lower reaches of the Yangtze River, COT and water vapor account for a large amount of R_dif. In North China, AOD occupies the dominant role in affecting the estimate of R_dif. Due to the low R_s caused by high CF, the ratio of R_dif to R_s can reach around 0.7 in these regions (Figure 7d). As indicated by site-level evaluation (Beijing, Chengdu, Wuhan and Shanghai in Figure S4), the underestimation of ERA5 seems certain. Another area of concern is the Taklimakan Desert, where both CF and COT are low but aerosols and atmospheric water vapor are high. Therefore, we believe that the high amount (~90 W/m²) and middle ratio (~0.5) of R_dif is possible. In addition, the regional average is very close to the measured values at Kashi station (Figure S4). Based on the above analysis, we are confident in the reliability of JiEA and believe that underestimation indeed occurs for ERA5 in related regions. Global simulations of surface solar radiation like ERA5 use a one-dimensional atmospheric radiative transfer model for computation efficiency. As a result, radiation retrievals are unable to tackle the adjacency effects caused by photons which are reflected by the surface out of the field of view and then scattered into the field of view by the atmosphere [78]. That effect directly results in the increase of R_dif. Neglect of adjacency effects can account for up to 5% underestimation in incident shortwave radiation on the land surface [10]. In particular, multiple reflections and scattering events off the sides of clouds lead to stronger adjacency effects and consequently to worse underestimation [33,54]. JiEA relies on a CNN-based module to capture the spatial pattern of clouds to deal with adjacency effects [22] and avoids underestimation of R_dif radiation to some degree.

On the Tibetan Plateau, ERA5 provides the highest estimates of R_dif, significantly greater than those of JiEA. An exception appears in the Tarim Basin. Regardless of overestimation or underestimation, the inner spatial distribution of R_dif estimated by ERA5 and JiEA is highly similar (Figure S5f) and agrees well with relevant atmospheric factors (Figure 8). Measurements at Golmud station that is located in the Tarim Basin support the results of JiEA, while the high similarity between observed time series at Lhasa station and ERA5′s estimates confirms the potential underestimation of JiEA on the Tibetan Plateau (Figure S4c). One cause of JiEA’s underestimation might be the excessive constraint that assumes an idealized state without diffuse radiation at the top of Mt. Everest [22]. The underestimation might also result from misidentification between ice clouds and liquid water clouds, whose radiative effects are significantly different [79,80]. The high probability of ice clouds on the Tibetan Plateau [80] tends to cause more R_dif than equivalent liquid water clouds. Previous studies demonstrate that cloud parameters (liquid/ice cloud types are inclusive) are critical in determining R_s [46,81]. This reminds us that we cannot accurately retrieve surface radiation from passive satellite signals alone, and even the best model needs to integrate atmospheric parameters. Therefore, integration of radiation transfer models and deep learning might be the next research focus.

Surface conditions may also influence the estimate of surface radiation. The most frequently mentioned one relates to snow/ice cover, which is often mistaken for clouds. In particular, retrievals of cloud optical depths over such surfaces are accompanied by large uncertainties [61]. It is even more challenging over short-lived snow or ice [33,60]. The high-level R_dif of ERA5 on the Tibetan Plateau and the Pamir Plateau is likely affected by snow/ice because observed seasonal variations of R_dif (Figure S8) are not consistent with variations of atmospheric factors (Figure S9) but show high similarity to snow/ice cover (Figure S10). Except for this specific issue, both datasets conform to common sense on how atmospheric factors influence R_dif in seasonal cycles, proving their strong ability to capture seasonal variation of R_dif at regional scale (Figure S5f).

6. Conclusions

Although R_s estimates are widely available from many radiation products, only ERA5 reanalysis and satellite-based JiEA provide estimates of R_dif over East Asia. Comprehensive evaluation and comparison are of great importance for rational use of these data and in-depth understanding of temporal trends and spatial differences of R_dif. In this study, estimates of R_dif at the surface are evaluated by comparing to quality-controlled measurements from WRDC and CMA and are mutually compared with respect to temporal variations and spatial distributions by referring to the spatial pattern of related atmospheric factors.

Hourly R_dif estimates of JiEA agree well with CMA measurements with an R of 0.85, MBE of 8.54 W/m², MABE of 50.43 W/m², and RMSE of 66.36 W/m², while ERA5 performs a little worse with an R of 0.71, negative MBE of 29.69 W/m², MABE of 63.83 W/m² and RMSE of 92.29 W/m². The performance of ERA5 is better than JiEA for low-radiation estimates. The overall accuracy of JiEA also exceeds ERA5 at daily means, with 84% of winning samples. Some problematic estimates occur for JiEA, likely due to the failure to handle extreme cases. Their performances are different in different regions. Particularly, ERA5 seriously underestimates R_dif on the Deccan Plateau. At monthly means, the RMSE of R_dif estimates decreases to 12.92 and 21.13 W/m² for JiEA and ERA5, respectively. These comparisons are hardly affected by their different spatial resolution, but the evaluation results are dependent on the spatial representativeness of ground measurement.

Data accuracy of ERA5 shows strong temporal consistency and latitudinal dependency. On the contrary, the accuracy of JiEA fluctuates in time and is robust in space. Therefore, we would like to recommend using ERA5 reanalysis data for trend detection and satellite-based JiEA for regional comparisons. Deseasonalized monthly time series of ERA5 and JiEA are highly correlated with each other but differ from the ground-observed series, indicating that gridded products are unable to reproduce temporal variability at site level. At the regional scale, we observe a slight decreasing trend of R_dif from JiEA and no trend from ERA5 within the validation period. Both time series analysis at stations and seasonal variations of spatial distribution show that ERA5 and JiEA are capable of capturing the seasonal cycle of R_dif effectively, although deviations still exist.

Notable differences of spatial distribution of R_dif from the two datasets appear on the Tibetan Plateau, where the underestimation of JiEA might be due to the misidentification between ice clouds and liquid water clouds, while the overestimation of ERA5 seems related to surface snow/ice cover. References to the spatial distribution of atmospheric factors support R_dif estimates of JiEA and confirm the general underestimation of ERA5 over East Asia. Neglect of adjacency effects caused by photon transport is regarded as the main cause for ERA5′s underestimation. Our analysis calls for the integration of physical models and new technologies (e.g., deep learning) to obtain accurate estimates of R_dif.