Evaluation of Multi-Reanalysis Solar Radiation Products Using Global Surface Observations

Abstract

Solar radiation incident at the Earth’s surface is an essential driver of the energy exchange between the atmosphere and the surface and is also an important input variable in the research on the surface eco-hydrological process. The reanalysis solar radiation dataset is characterized by a long time series and wide spatial coverage and is used in the research of large-scale eco-hydrological processes. Due to certain errors in their production process of the reanalysis of solar radiation products, reanalysis products should be evaluated before application. In this study, three global solar-radiation reanalysis products (ERA-Interim; JRA-55; and NCEP-DOE) in different temporal scales and climate zones were evaluated using surface solar-radiation observations from the National Meteorological Information Center of the China Meteorological Administration (CMA, Beijing, China) and the Global Energy Balance Archive (GEBA, Zürich, Switzerland) from 2000 to 2009. All reanalysis products (ERA-Interim; JRA-55; and NCEP-DOE) overestimated with an annual bias of 14.86 W/m², 22.61 W/m², and 31.85 W/m²; monthly bias of 15.17 W/m², 21.29 W/m², and 36.91 W/m²; and seasonal bias of 15.08 W/m², 21.21 W/m², and 36.69 W/m², respectively. In different Köppen climate zones, the annual solar radiation of ERA-Interim performed best in cold regions with a bias of 10.30 W/m² and absolute relative error (ARE) of 8.98%. However, JRA-55 and NCEP-DOE showed the best performance in tropical regions with a bias of 20.08 W/m² and −0.12 W/m², and ARE of 11.00% and 9.68%, respectively. Overall, through the evaluations across different temporal and spatial scales, the rank of the three reanalysis products in order was the ERA-Interim, JRA-55, and NCEP-DOE. In addition, based on the evaluation, we analyzed the relationship between the error (ARE) of the reanalysis products and cloud cover, aerosol, and water vapor, which significantly influences solar radiation and we found that cloud was the main cause for errors in the three solar radiation reanalysis products. The above can provide a reference for the application and downscaling of the three solar radiation reanalysis products.

1. Introduction

Solar radiation is the main energy source on the Earth’s surface and the main driving force for global atmospheric, hydrologic, ecological, and biological processes [1,2,3]. Solar radiation mainly includes direct radiation and diffuse radiation. Direct radiation is part of the energy of solar rays that attenuate after passing through the atmosphere without changing direction. Diffuse radiation refers to the part of the energy of solar rays that return to the ground after being scattered by atmospheric particles, aerosols, and other matters. Over complex terrain, reflected radiation from the surrounding terrain is also important, especially over snow surfaces. Therefore, solar radiation varies spatially due to different climatic characteristics, land cover, and topographic reliefs. For the eco-hydrological and atmospheric processes model, the solar radiation dataset is an important input [4,5,6].

There are mainly four solutions for solar radiation modeling from a global scale or local scale: the atmospheric radiation transfer model, remote sensing retrieval, calculation based on the empirical equations, and solar radiation reanalysis datasets. Atmospheric radiation transfer models have a relatively good physical mechanism, but are computationally very expensive. In the recent years, several remote sensing products were released with high spatial and temporal resolution. However, remote sensing solar radiation products are still limited by the temporal resolution of polar satellite and spatial coverage for geostationary satellite [7,8,9,10]. The method based on empirical equations rely on surface observation data of which the sparsity will limit its application [11]. In comparison, reanalysis products can provide solar radiation data with a relatively long time series and high space coverage.

The National Center for Environmental Prediction (NCEP, College Park, MD, USA), the European Centre for Medium-Range Weather Forecasts (ECMWF, Reading, UK), and the Japan Meteorological Agency (JMA, Tokyo, Japan) are three representative organizations that produce reanalysis datasets. The reanalysis products of the above organizations are widely used in studies on climate change and eco-hydrology. For example, reanalysis datasets of solar radiation are used to detect long-term climate trends [12,13,14], generate atmospheric forcing data [15], and investigate the water and energy exchanges between the Earth’s surface and the atmosphere [16,17]. As such, solar radiation is an important component of reanalysis products. However, solar radiation reanalysis products are generated by the assimilation system based on observation data and numerical weather prediction models, so it is still required to evaluate the errors and adaptability before its practical application due to the sparsity of observation sites and the model errors of the assimilation system [16,18,19,20].

Solar radiation reanalysis has been verified at different temporal and spatial scales based on ground station observation data [19,21,22,23]. Xia et al. [24] evaluated the NCEP solar radiation products in China using the China Meteorological Administration’s (CMA, Beijing, China) ground observation data, and found that the monthly bias of the NCEP reanalysis dataset was between 40 W/m² and 100 W/m². Jia et al. [25] evaluated two solar radiation reanalysis products (ERA-Interim and NCEP-DOE) using the CMA’s ground observation data in China and found that the monthly mean biases of two reanalysis datasets were 25.1 W/m² (ERA-Interim) and 46.1 W/m² (NCEP-DOE). By using observation sites taken from GEBA, Hicke et al. [26] found that the annual mean bias of NCEP-DOE was 46.55 W/m² at a global scale. Based on nine stations of CAMP/Tibet, Wang and Zeng [27] found that in the area of the Qinghai–Tibet Plateau, six reanalysis products (MERRA, NCEP-NCAR, ERA-40, ERA-Interim, CFSR, and GLDAS) overestimated ground-observation, and daily solar radiation bias ranged from about 1.56 W/m² to 5.00 W/m². The evaluations of the solar radiation reanalysis datasets in the above-mentioned studies are from aspects of monthly and annual bias at a regional scale, and few are for exploring the performance of reanalysis data from the global scale and different climate zones. As such, these studies can neither characterize the overall accuracy of reanalysis products at a global scale, nor provide strong evidence for the applicability of solar radiation datasets to different climate regions. Therefore, we evaluated three solar radiation reanalysis datasets using global ground observations, and the errors were assessed from different temporal scales (monthly, seasonal, and annual) and in different climate zones (Köppen’s climatic zones) to provide reasonable suggestions for the application of solar radiation reanalysis products at different temporal and spatial scales or climate zones.

Furthermore, the error sources of the reanalysis surface radiation products were explored in this paper. Solar radiation reaching the ground is significantly affected by the Earth’s atmosphere including cloud, aerosols, water vapor, and atmospheric gases [28,29,30], leading to a very large spatial-temporal difference in solar radiation on the ground in different regions. Therefore, the relationship between aerosol, cloud cover, and water vapor remotely-sensed datasets and the error of the solar radiation reanalysis datasets were calculated for different products and different climate zones to find the main error sources for each solar radiation reanalysis dataset in different climate zones.

The rest of the paper is organized as follows. The data source and evaluation indicator used in this study are introduced in Section 2. Section 3 evaluates three solar radiation reanalysis datasets (ERA-Interim, JRA-55, and NCEP-DOE) based on the ground station solar radiation data from two observation networks (GEBA and CMA), and Section 4 discusses the error source of the reanalysis datasets in different climate zones. Our conclusions are presented in Section 5.

2. Data and Methods

2.1. Data Source

Ground observation data from the Global Energy Balance Archive (GEBA, Zürich, Switzerland) and China Meteorological Administration (CMA, Beijing, China) between 2000 and 2009 were used to evaluate the reanalysis datasets including the ERA-Interim, NCEP-DOE, and JRA-55. The MOD08 dataset including water vapor content, cloud cover, the optical thickness of the cloud, and the optical thickness of aerosols was used to analyze the error sources of the solar radiation reanalysis dataset. Details of the datasets used in this paper are shown in

Table 1. Details of the data products used in this paper.

Data Source	Data Period	Temporal Resolution	Spatial Resolution	Reference
GEBA	1922–2008	monthly	N/A(station)	[31]
CMA	1957–present	monthly	N/A(station)	[32]
ERA-Interim	1979–present	3 h	0.75° × 0.75°	[33]
JRA-55	1958–2013	3 h	0.56° × 0.56°	[34]
NCEP-DOE	1979–present	6 h	1.9° × 1.9°	[35]
MOD08	2000–present	monthly	1° × 1°	[36]

.

2.1.1. Ground Observation Data

The GEBA is a flux observation data archive for global energy that was developed and maintained by ETH Zurich (Switzerland). The GEBA provides more than 2500 station-observed data, and the database contains 15 surface energy flux variables. The data are collected from the World Radiation Data Centre (WRDC) in St. Petersburg (Russia), meteorological service centers of various countries, multiple observation networks (BSRN, ARM, and SURFRAD), and some data records. The monthly average relative random error (root mean square error) of the GEBA observation value of solar radiation is 5%, and the annual mean error is 2% [31]. The CMA dataset shows monthly and yearly meteorological data from normal weather stations published by the National Meteorological Information Center of China and includes variables like solar radiation, temperature drop, and air temperature. The distribution of the observation sites used in this study is shown in Figure 1.

2.1.2. ERA-Interim

ERA-Interim is a global atmospheric reanalysis dataset produced by the European Centre for Medium-Range Weather Forecasts (ECMWF, Reading, UK) with data available since 1979. ERA-Interim projects began in 2006 with the aim of improving the previous reanalysis data, the ERA-40. The data assimilation system used for generating ERA-Interim data is IFS (Cy31r2) launched by ECMWF in 2006. Compared with ERA-40, the spatial resolution of the ERA-Interim dataset has improved by 0.75° (about 80 km) and the temporal resolution is 3 h. The solar radiation calculation is based on the rapid radiation transfer model (RRTM) [33].

2.1.3. JRA-55

The time coverage of JRA-55 covers 1958 to 2013. Compared with JRA-25, JRA-55 is based on a new data assimilation and prediction system and has reduced many defaults in the JRA-25 reanalysis. Its improvements include the use of a network with higher spatial resolution (0.5625°), the 4D-Var with variational bias correction (VarBC) for satellite radiation, and the import of greenhouse gases of which the density changes over time. The products thus obtained are much better than the JRA-25 products [34].

2.1.4. NCEP-DOE

NCEP-DOE was jointly developed by the National Center for Environmental Prediction (NCEP, College Park, MD, USA) and the United States Department of Energy (DOE, Washington, DC, USA), and can be deemed as the continuation of the NCEP/NCAR plan for global atmosphere reanalysis data from 1979 to the present. The numerical prediction model, assimilation scheme, and observation systems used are roughly the same as the NCEP/NCAR, but there are some existing error problems in the NCEP/NCAR, which are known to have been revised. Furthermore, it can be taken as basic data for the verification of the Second Atmospheric Model Intercomparison Project (AMIP). Its horizontal resolution is 1.9°, the temporal resolution is 6 h, and the vertical resolution is 28 layers [35].

2.1.5. MOD08

To analyze the error sources of reanalysis solar radiation datasets, we used the aerosol optical thickness, cloud cover, cloud optical thickness, and water vapor from MOD08 that is a monthly 1 × 1 degree grid average atmosphere product from MODIS. The product provides means, standard deviations, uncertainty estimates, and statistics of atmosphere parameters for fractions of pixels that satisfy some condition [36].

2.2. Evaluation Indicator

The evaluation was carried out based on the bias, absolute relative error (ARE), correlation coefficient (R), and root mean square error (RMSE) between the reanalysis data and observations. Bias was used to measure the deviation of the predicted value of the reanalysis data from the observation data. ARE can be used to compare the quality of different reanalysis data and the observation data at the same level more precisely. RMSE can be taken as another indicator for measuring the accuracy of the reanalysis data results. R can be used to represent the consistency of the reanalysis data with the observation data. In addition, the partial correlation coefficient (PCC) was used to measure the relationship between the atmospheric parameters of cloud cover, AOD, and water vapor and error (ARE) of solar radiation. PCC can measure the correlation between one climatic factor among multiple mutually influenced climatic factors and evaluate reanalysis products. The calculation equations are

$$Bias=\frac{{{\displaystyle \sum }}_{i=1}^N\left(y_i-x_i\right)}{N}$$

(1)

$$ARE=\frac{{{\displaystyle \sum }}_{i=1}^N\frac{\left|y_i-x_i\right|}{x_i}}{N}\times 100$$

(2)

$$RMSE=\sqrt{\frac{\sum {\left(y_i-x_i\right)}^2}{n}}$$

(3)

$$R=\frac{{{\displaystyle \sum }}_{t=1}^n\left(x_t-\overline{x}\right)\left(y_t-\overline{y}\right)}{\sqrt{{{\displaystyle \sum }}_{t=1}^n{\left(x_t-\overline{x}\right)}^2{{\displaystyle \sum }}_{t=1}^n{\left(y_t-\overline{y}\right)}^2}}$$

(4)

where $y_i$ refers to the predicted data; $x_i$ refers to the observation data; and $N$ refers to the number of observation sites [37].

$$PC{C}_{yu}=-\frac{r^{um+1}}{\sqrt{r^{uu}{r}^{m+1m+1}}}(y>u)$$

(5)

$$R_{xy}=[\begin{array}{ccc}\begin{array}{c}\begin{array}{cc}{R}_{11}& R_{12}\end{array}\\ \begin{array}{cc}{R}_{21}& R_{22}\end{array}\end{array}& \begin{array}{c}\dots \\ \dots \end{array}& \begin{array}{c}\begin{array}{cc}{R}_{1m}& R_{1m+1}\end{array}\\ \begin{array}{cc}{R}_{2m}& R_{2m+1}\end{array}\end{array}\\ \begin{array}{cc}⋮& ⋮\end{array}& \ddots & \begin{array}{cc}⋮& ⋮\end{array}\\ \begin{array}{c}\begin{array}{cc}{R}_{m1}& R_{m2}\end{array}\\ \begin{array}{cc}{R}_{m+11}& R_{m+12}\end{array}\end{array}& \begin{array}{c}\dots \\ \dots \end{array}& \begin{array}{c}\begin{array}{cc}{R}_{mm}& R_{mm+1}\end{array}\\ \begin{array}{cc}{R}_{m+1m}& R_{m+1m+1}\end{array}\end{array}\end{array}]$$

(6)

where $R_{xy}$ is the correlation coefficient matrix between $y$ (the $\left(m+1\right)$-th variable) and $x_1,x_2,\dots ,x_m$, and $R_{xy}^{-1}$, whose $\left(u,v\right)$-th element is $r^{uv}$, is the inverse of the matrix $R_{xy}$. Therefore, the partial correlation coefficient between the $y$ and $u$-th variable is shown as Equation (5) [38].

2.3. Köppen Climatic Classification

The accuracy of solar radiation products is affected by different climate types [39]. Therefore, the surface solar radiation needs to be evaluated in different climatic zones. The Köppen climate classification is one of the most widely used climate classification systems. It was first published by the Russian climatologist Wladimir Köppen (1846–1940). The climate classification system has been modified by his collaborators and successors. It is still in widespread use. Based on a large global data set of long-term monthly precipitation and temperature station time series, Peel et al. [40] updated the global map of the Köppen–Geiger climate classification. The updated world Köppen–Geiger climate map is freely available electronically [41]. Because of the complex classification and limited observation sites, we adopt five basic climatic zones (i.e., arid, cold, polar, temperate, and tropical), as shown in Figure 1.

3. Validation Results of Reanalysis Solar Radiation Using Ground Measurements

Solar radiation from ERA-Interim, JRA-55, and NCEP-DOE from 2000–2009 were evaluated using 804 global ground stations that were selected by quality control files from the GEBA and CMA, respectively. We compared the reanalysis solar radiation with ground observations directly because the interpolations of station measurements to reduce the scale effects between points and the grid cell will also introduce other errors [27] and the monthly representation errors of surface sites are relatively small [42]. The annual, seasonal, and monthly solar radiation were evaluated at a global scale and different climate zones, respectively. Four seasons were classified as MAM (March, April, and May), JJA (June, July, and August), SON (September, October, and November), and DJF (December, January and February).

3.1. Evaluation of Reanalysis Solar Radiation at a Global Scale

Table 2. Evaluation of three annual, seasonal, and monthly solar radiation reanalysis products at a global scale (unit of Bias and RMSE: W/m², ARE: %)

Temporal Scales		MAM	JJA	SON	DJF	Annual Mean	Monthly Mean
ERA-Interim	Bias	22.75	15.58	11.14	10.83	14.86	15.17
	ARE	19.73	15.81	17.83	23.23	11.62	19.05
	RMSE	30.80	27.35	21.23	22.86	22.00	26.82
	R	86.77	90.53	94.83	96.44	63.77	91.53
JRA-55	Bias	26.97	24.63	18.59	14.66	22.61	21.29
	ARE	21.73	18.79	22.70	27.08	16.27	22.53
	RMSE	35.41	35.25	26.88	24.88	27.79	31.34
	R	82.65	87.47	93.12	95.91	64.29	89.65
NCEP-DOE	Bias	48.67	43.91	28.36	25.80	31.85	36.91
	ARE	33.72	27.56	31.22	40.05	24.11	33.25
	RMSE	58.23	54.56	38.09	37.72	40.06	48.34
	R	67.46	80.87	88.10	92.64	53.22	82.56

shows the evaluation results of the annual, seasonal, and monthly solar radiation of three reanalysis products using ground measurements. The annual solar radiation reanalysis products showed an overestimation with biases of 14.86 W/m², 22.61 W/m², and 31.85 W/m² for the ERA-Interim, JRA-55, and NCEP-DOE, respectively. The monthly and seasonal solar radiation reanalysis also performed an overestimation. In different seasons, the bias was the lowest in DJF and the highest in MAM for all reanalysis products. Comparing the biases of three solar radiation reanalysis products at different temporal scales, we found that the bias of ERA-Interim was the lowest and that of NCEP-DOE was the highest, which indicates that ERA-Interim performed better with relatively low bias.

ARE, unlike from bias, can quantify the precision of different reanalysis data at the same level. It showed a consistency in bias at all temporal scales where ERA-Interim performed the best and NCEP-DOE performed the worst out of all products by comparing the AREs of three solar radiation reanalysis datasets. For solar radiation reanalysis with different temporal scales, three products performed best with AREs of 11.62% for ERA-Interim, 16.27% for JRA-55, and 24.11% for NCEP-DOE in annual solar radiation when compared with the monthly and seasonal mean solar radiation. Meanwhile, seasonal solar radiation performed best in JJA among the four seasons for all products. For ERA-Interim, the ARE at different temporal scales were below 20%, except in DJF, indicating that ERA-Interim performed well at different temporal scales. However, the ARE of NCEP-DOE was larger than 20%, in particular, 40.05% was shown in DJF, which indicates that NCEP-DOE had larger uncertainty, especially in DJF. As such, ERA-Interim is a better choice for the application.

RMSE is a measure of the variation of reanalysis values around the measured values. From Table 2, we can see that ERA-Interim had a smaller RMSE at all temporal scales than the other two products, indicating that ERA-Interim showed a smaller variation from the ground measurements and was more stable. The smallest RMSE occurred in DJF for JRA-55 and NCEP-DOE, but in SON for ERA-Interim across all temporal scales. The correlation coefficient, which shows the consistency between the reanalysis and ground measured solar radiation, also indicated that ERA-Interim performed the best across different temporal scales among the three reanalysis products.

Figure 2 and Figure 3 show the annual and monthly changes of bias and ARE between 2000 and 2009. Based on Figure 2, it indicates that ERA-Interim very clearly performed the best in all years. During the 10 years, the annual bias of NCEP-DOE had the largest fluctuation value of −0.49 W/m², followed by JRA-55 with a fluctuation value of −0.40 W/m². The annual bias of ERA-Interim had the smallest fluctuation value of −0.03 W/m². Based on the inter-annual change of ARE, it showed the same trend with changes of bias during 2000 and 2009; however, between the three reanalysis products, the annual changes of bias or ARE showed a different pattern. Compared to the previous year, the bias and ARE of the ERA-Interim increased in 2003 and 2008, and decreased in 2007. However, NCEP-DOE had the opposite trend when compared with ERA-Interim. JRA-55 only showed a different trend to ERA-Interim in 2003. According to Figure 3, it was found that the trend of monthly bias was opposite to the monthly ARE. The monthly bias in the three products increased from January, peaked around April, then decreased. However, the monthly ARE decreased from January, reached a valley around July, and then increased. The reason for this may be that the solar radiation in July and August was larger, leading to the smaller ARE.

Overall, ERA-Interim performed the best during the period between 2000 and 2009 when compared with JRA-55 and NCEP-DOE under all evaluation indices. At different temporal scales, annual solar radiation performed the best for all products. Among the four seasons, the three products performed best in JJA by ARE, but had the lowest bias in DJF. NCEP-DOE in DJF should be carefully used as it had a relatively large ARE of 40.05%.

3.2. Evaluation of Solar Radiation in Different Climate Zones

We used Köppen’s climate zones [40] of arid, cold, polar, temperate, and tropical to distinguish the different climate types. The annual, seasonal, and monthly solar radiation in the five climatic zones were evaluated respectively to explore the performance of the three reanalysis products in different climate zones.

Table 3. Evaluation results of the annual, seasonal, and monthly solar radiation from three reanalysis datasets in different climatic zones (unit of bias: W/m², ARE: %)

ERA-Interim
Temporal Scales	Bias							ARE
	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean
Arid	24.17	20.51	15.19	15.48	18.84	18.99	16.77	14.64	12.36	13.83	20.88	15.43	15.51	10.13
Cold	19.14	11.87	6.01	4.08	10.28	10.07	10.30	14.76	11.36	14.01	19.25	14.85	15.07	8.98
Polar	3.01	8.99	−2.56	−1.67	1.94	1.92	2.12	15.59	12.02	16.62	21.74	16.49	15.67	9.13
Temperate	27.01	16.66	13.15	14.08	17.73	17.66	17.84	20.16	14.81	16.57	23.87	18.85	18.77	13.92
Tropical	22.10	9.62	23.07	23.73	19.63	20.24	15.74	13.37	12.68	16.79	22.62	16.37	16.35	9.98
Mean	19.09	13.53	10.97	11.14	13.68	13.78	12.55	15.70	12.65	15.56	21.67	16.40	16.27	10.43
JRA-55
Temporal Scales	Bias							ARE
	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean
Arid	18.73	19.99	13.56	12.45	16.18	16.63	16.33	13.78	13.47	15.73	21.86	16.21	16.36	11.47
Cold	27.93	31.30	16.89	10.32	21.61	21.21	21.88	18.13	16.22	20.70	24.69	19.94	19.96	15.70
Polar	21.64	38.95	12.50	8.41	20.38	20.93	23.93	16.78	21.75	18.54	24.33	20.35	19.79	16.33
Temperate	33.23	25.16	22.85	18.98	25.06	25.00	26.45	23.12	17.08	22.38	28.76	22.84	22.72	19.33
Tropical	21.19	20.20	29.81	23.29	23.62	24.47	20.08	13.25	15.56	19.15	22.66	17.66	17.79	11.00
Mean	24.54	27.12	19.12	14.69	21.37	21.65	21.73	17.01	16.82	19.30	24.46	19.40	19.32	14.77
NCEP-DOE
Temporal Scales	Bias							ARE
	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean	MAM	JJA	SON	DJF	Seasonal Mean	Monthly Mean	Annual Mean
Arid	39.63	42.49	23.94	19.92	31.50	31.79	25.11	23.73	22.21	22.13	26.88	23.74	23.81	17.93
Cold	56.99	54.93	29.50	22.30	40.93	40.41	36.00	33.52	27.34	33.51	40.43	33.70	33.63	25.54
Polar	39.76	60.70	19.60	11.81	32.97	34.66	29.26	24.39	32.62	25.14	26.83	27.25	27.19	19.91
Temperate	57.37	44.79	34.42	36.25	43.21	43.04	38.23	37.08	26.63	32.15	47.87	35.93	35.63	27.93
Tropical	8.14	1.01	16.88	16.70	10.68	10.66	−0.12	13.45	16.62	19.24	22.45	17.94	18.74	9.68
Mean	40.38	40.78	24.87	21.40	31.86	32.11	25.70	26.43	25.08	26.43	32.89	27.71	27.80	20.20

shows the evaluation results for the three reanalysis datasets in different climatic zones from 2000 to 2009. We only used the bias and ARE to evaluate the performance as the bias can measure the system bias and the ARE can represent the absolute error.

ERA-Interim overestimated in all climate zones, except in the Polar region in SON and DJF, which had a negative bias. We also found that ERA-Interim showed the lowest bias in the polar region with values of 2.12, 1.92, and 1.94 W/m² for the annual, monthly, and seasonal solar radiation when compared with other climate zones. Based on the ARE, ERA-Interim performed the best in the cold region for annual solar radiation with an ARE of 8.98%. However, for monthly solar radiation, ARE in arid, cold, and polar regions were similar (about 15%) and smaller than in temperate and tropical regions with AREs of 18.77% and 16.35%, respectively. In all climate zones, ERA-Interim exhibited a smaller ARE in JJA when compared with the other seasons.

In contrast with ERA-Interim, JRA-55 overestimated in all climate zones and all temporal scales and showed a larger bias than ERA-Interim. The area with the lowest bias was the Arid region, which was also different with ERA-Interim. Based on ARE, the annual, monthly and seasonal solar radiation of JRA-55 performed better in tropical and arid regions than in other climate zones.

NCEP-DOE also showed overestimation in all times and all climate zones, except for annual solar radiation in the tropical region with a bias of −0.12 W/m². It had a very large bias in cold and temperate regions in all temporal scales. For annual, seasonal, and monthly solar radiation, NCEP-DOE showed a better performance in the tropical region with AREs of 9.68, 18.74, and 17.94%, respectively. In addition, NCEP-DOE had a large ARE with 40.43% and 47.87% in DJF in cold and temperate regions, respectively.

Overall, the solar radiation reanalysis products showed an overestimation in most seasons and climate zones. ERA-Interim had a better performance in arid, cold, and polar regions; however, JRA-55 had better accuracy in arid and tropical regions. Compared with other reanalysis datasets, NCEP-DOE had a larger bias and only had a minor error in the tropical region.

3.3. Evaluation of Reanalysis Solar Radiation products at Different Elevation Zones

We also analyzed the performance of three reanalysis datasets at different elevations. The results of bias and ARE of annual, monthly and seasonal mean solar radiation are shown in Figure 4. We classified all sites into four elevation zones, i.e., I zone (0–500 m), II zone (500–1000 m), III zone (1000–2000 m), and IV zone (more than 2000 m).

It can be found, based on Figure 4, that bias and ARE of ERA-Interim are smaller than JRA-55 and NCEP-DOE in all elevation zones and different time scales. ERA-Interim shows the largest bias at low altitude zone (I zone) for annual mean solar radiation, but at III zone for seasonal and monthly mean solar radiation. For JRA-55 and NCEP-DOE, the largest bias was shown in III zone in different time scales. In addition, we found that the bias increment is more significant for JRA-55 than NCEP-DOE and ERA-Interim as the elevation increases. Based on the ARE variations at different elevation zones from Figure 4, we can find that AREs show a decreasing trend as the elevation increases for all products in different time scales. However, annual mean JRA-55 solar radiation shows an increase for ARE at III elevation zone.

Overall, it can be found that ERA-Interim performs best at different elevations among the three products. Bias of JRA-55 increases more significantly than ERA-Interim and NCEP-DOE, and ARE of annual mean JRA-55 increases in III elevation zones which indicates that JRA-55 solar radiation is influenced by elevation.

4. Error Sources of Reanalysis Solar Radiation Products

Due to the different atmospheric factors affecting the solar radiation in different seasons, the correlation relationship analysis was carried out in four seasons [43]. The partial correlation coefficient (PCC) was used to calculate the correlation between the ARE of the solar radiation products and different atmospheric factors because the different atmospheric factors are mutually influenced. The partial correlation coefficients of eight atmospheric factors of the four seasons were obtained using monthly mean of aerosol optical thickness, cloud cover, cloud optical thickness, water vapor, and their respective standard deviations from the MODIS product of MOD08. The greater the partial correlation coefficient, the more significant the influence of the atmospheric factor on the error of the solar radiation reanalysis products.

Table 4. Influence factors of three products in the different seasons and different climatic zones.

Climatic Zones		Season
		MAM	JJA	SON	DJF
ERA-Interim	Arid	WV	C	C	WV
	Cold		C		AOD
	Polar	C		C
	Temperate	WV	C	AOD	AOD
	Tropical	WV
JRA-55	Arid	C		WV	WV
	Cold	C	C		C
	Polar
	Temperate	WV	AOD	C	WV
	Tropical			C
NCEP-DOE	Arid	C	C	C	WV
	Cold	AOD	C	WV	C
	Polar
	Temperate	WV	C	C	WV
	Tropical	AOD

Note: AOD: aerosol optical depth, C: cloud cover or cloud optical thickness, WV: water vapor.

summarizes the atmospheric factors that had the most significant partial correlation coefficient of each reanalysis product in different climate zones and seasons. The WV, C, and AOD in Table 4 represent the influence factor of water vapor, cloud cover or cloud optical thickness, and aerosol optical thickness, respectively, which had the largest PCC when compared with other climate factors.

As can be seen from Table 4, the solar radiation reanalysis was affected by aerosols, clouds, and water vapor in different climatic zones and different seasons. In MAM, the three products were greatly influenced by cloud and water vapor. In JJA and SON, they were greatly influenced by the cloud. In DJF, the three products were mainly influenced by water vapor; however, ERA-Interim was also influenced by aerosols, and JRA-55 and NCEP-DOE were also influenced by the cloud in the cold region.

In the cold region, cloud is the main influencing factor for the three products. In addition, in the polar region, due to limited data, only the performance of ERA-Interim was found to be significantly influenced by cloud. In the temperate region, the three influence factors of water vapor, cloud, and aerosol affected the three reanalysis datasets, which explained the poor performance of the three solar radiation reanalysis datasets in the temperate region from Table 3.

In addition, it was found from Table 4 and the ARE of annual solar radiation in Table 3 that the climatic zone that showed a better performance for each product had relatively few influence factors. The ERA-Interim data performed well in cold, polar, and tropical regions, and the number of significant influence factors in these climatic zones in Table 4 were small with 2, 2, and 1, respectively. The JRA-55 data had a relatively small ARE in the Tropical climatic zone, and it was also found that its only one significant influence factor was cloud. In the same way, NCEP-DOE performed best in the Tropical climatic zone with only one significant influence factor of AOD. Therefore, the influence factors in Table 4 for the solar radiation reanalysis products in different climate regions can be used as a reference for the application of solar radiation reanalysis products and factor selection for the statistical downscaling of solar radiation reanalysis.

Based on the influence factor analysis results, it was found that solar radiation reanalysis was greatly influenced by cloud overall. Therefore, we compared the annual mean cloud cover from the three reanalysis products with the MODIS cloud cover product from MOD08, and the results are shown in Figure 5. We also calculated the ARE of the cloud cover data of the three reanalysis products by comparing them with MOD08 as shown in

Table 5. Specify that ARE of the cloud cover is obtained by comparison of reanalysis with MODIS (MOD08) data (ARE unit: %; SR: solar radiation, CC: cloud cover).

Climatic Zones	ERA-Interim		JRA-55		NCEP-DOE
	ARE (SR)	ARE (CC)	ARE (SR)	ARE (CC)	ARE (SR)	ARE (CC)
Arid	10.13	19.55	11.47	25.12	17.93	26.01
Cold	8.98	16.81	15.70	33.41	25.54	36.50
Polar	9.13	17.28	16.33	37.31	19.91	41.19
Temperate	13.92	21.39	19.33	30.67	27.93	31.12
Tropical	9.98	18.00	11.00	16.64	9.68	11.47
Mean	10.43	18.61	14.77	28.63	20.20	29.26

Note: SR: solar radiation, CC: cloud cover.

. From Figure 5d of the MODIS annual cloud cover distribution, we found that there was less cloud in the arid and polar climatic zones, but showed more in the tropical climatic zone than in the other climatic zones.

Figure 5a–c show the ARE of the cloud cover from the three reanalysis datasets compared with MOD08 as the true value. It was found that the cloud cover of the reanalysis data underestimated in most regions of the world. In addition, based on Table 5, it was found that the accuracy of the ERA-Interim cloud cover was the highest (18.61%), followed by JRA-55 (28.63%), and NCEP-DOE (29.26%), which was consistent with the evaluation results of the solar radiation reanalysis.

Meanwhile, we calculated the correlation coefficient between the ARE of cloud cover and of solar radiation from the three reanalysis products in different climatic zones (Table 5), (ERA-Interim: 0.93, JRA55: 0.74, NCEP: 0.75). It showed a very large correlation coefficient, which demonstrated that the underestimation of cloud cover of the three reanalysis datasets was one of the main reasons for the overestimation of solar radiation.

According to Table 5, although the ARE of annual solar radiation from ERA-Interim (10.13%) and JRA-55 (11.47%) in the Arid climate zone were similar, the error of cloud cover in these two climate zones were very different (ERA-Interim: 19.55%, JRA-55: 25.12%), which is possibly caused by that water vapor also influenced the solar radiation of these two products from Table 4. Similarly, in the tropical region, although the error of cloud cover of the three products was different (ERA-Interim: 18.00%, JRA-55: 16.64%, NCEP-DOE: 11.47%), the error of solar radiation was similar (ERA-Interim: 9.98%, JRA-55: 11.00%, NCEP-DOE: 9.68%). As the significant influence factors of the three reanalysis products were different, i.e., they were water vapor, cloud, and aerosols for ERA-Interim, JRA-55, and NCEP-DOE, respectively. The smaller error of cloud cover of JRA-55 in the tropical climatic zone may also be the reason that the error of solar radiation was significantly smaller than in other climatic zones.

Generally, the underestimation of cloud cover was the main reason for the overestimation of solar radiation for the three reanalysis products. However, the overall interaction between the aerosols, water vapor, and cloud cover were also important for the solar radiation reanalysis product.

5. Conclusions

The current study presented the validation of the solar radiation provided by three representative global reanalysis products (ERA-Interim, JRA-55, and NCEP-DOE) using the global surface observation data at different temporal and climate zones, and the error sources of the solar radiation reanalysis products in different climate zones were also analyzed.

It was found that the three reanalysis products overestimated the solar radiation when compared with the ground observation data. Overall, the range of monthly bias between the three reanalysis products at a global scale was 15.17 W/m² to 36.91 W/m², and the range of annual bias was 14.86 W/m² to 31.85 W/m². Among the three products, the performance of ERA-Interim was the best and the performance of NCEP-DOE was the worst. Regarding different climate regions, ERA-Interim had better performance in arid, cold, polar, and tropical regions; however, JRA-55 had better accuracy in arid and tropical regions. Compared with other reanalysis datasets, NCEP-DOE had a larger bias and only had a minor error in the tropical region. It is possible that NCEP-DOE performs worst among all products because the resolution of NCEP-DOE is coarse, which leads to more uncertainty in the evaluation due to the spatial mismatch. Generally, seasonal radiation has an opposite behavior in the southern and northern hemispheres. A more detailed assessment of the reanalysis product could be conducted from the seasonal characteristics of the two hemispheres which is a possible future research direction.

The error sources of the three solar radiation reanalysis products in different climate zones were analyzed and we found that the three products were mainly influenced by the water vapor and cloud. In the cold region, cloud was the main influencing factor for the three products. In the temperate region, the effects of water vapor, cloud, and aerosols all influenced the accuracy of the three reanalysis datasets. Overall, cloud was the main influence factor for all products. The evaluation results and error sources analysis can be references for the application of solar radiation reanalysis products in different areas and its downscaling in different regions.