Validation of MERRA-2 AOT Modeling Data over China Using SIAVNET Measurement

Abstract

The Modern-Era Retrospective Analysis for Research and Applications version 2 (MERRA-2) Aerosol Optical Thickness (AOT) dataset is a consistent and comprehensive dataset combining observations from various satellite instruments and other sources with a numerical model, supporting climate studies, atmospheric modeling, air quality monitoring, and environmental research. Due to the uneven and sparse distribution of the Aerosol Robotic Network (AERONET) in China, the validation for the MERRA-2 AOT dataset over China is inadequate. The construction of the National Civil Space Infrastructure Satellite Aerosol Product Validation Network (SIAVNET) is helpful to compensate for MERRA-2 AOT dataset validation over China. The validation results show that the accuracy of the MERRA-2 AOT goes down along with the aerosol loading in the atmosphere increase. In general, when the AOT is less than 1.0, the slope can reach 0.712 with R² = 0.584. The percentage of data pairs that fall within the GCOS minimum requirement is less than 60%. Research also shows that MERRA-2 has a lower simulation quality of AOT at high altitudes than at low altitudes in China. Additionally, MERRA-2’s AOT simulation quality varies by season. Simulated quality is worst in spring, improving in subsequent seasons. During the winter season, simulations are of the highest quality.

1. Introduction

Aerosol Optical Thickness (AOT) is a measure of the attenuation of sunlight by aerosols and it can be used to derive aerosol amounts, size distributions, etc. It is determined by measuring the attenuation of sunlight caused by aerosols and is typically measured using sun photometers or satellite sensors. AOT values provide information about aerosol abundance, which is crucial for studying climate, air quality, weather forecasting, and satellite remote sensing [1,2,3,4]. By monitoring AOT over time, the aerosol variations, their sources, and their impact on human health and the environment can be understood [5,6].

MERRA-2 AOT data are a product derived from NASA’s Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2) reanalysis dataset [7,8,9]. MERRA-2 AOT data are used by scientists, researchers, and policy-makers to study air quality, climate change, and atmospheric chemistry. The data can be used to track the distribution of aerosols around the world, to study the impact of aerosols on air quality and human health, and to monitor changes in the atmosphere over time. Overall, MERRA-2 AOT data are a valuable tool for understanding the complex interactions between the atmosphere and the Earth’s surface, and it is a key component of NASA’s efforts to study the Earth’s climate and environment [10,11,12,13].

Though MERRA-2 provides useful estimates of past atmospheric conditions and climate effects, the dataset carries uncertainties due to observational errors, modeling assumptions, data assimilation techniques, limitations in spatial and temporal resolution, and the challenge of accurately representing complex atmospheric processes [14,15,16,17].

Sun-sky photometers [18] offer a useful tool for ground-based AOT measurements. The data collected from these instruments provide ground-truth measurements that can be used to check the accuracy of the MERRA-2 data. The Aerosol Robotic Network (AERONET) is an especially crucial resource for ground-based measurements [19,20]. Several works have been conducted to validate the MERRA-2 dataset in previous studies. For example, MERRA-2 AOT in Australia was found to be overestimated during dust aerosol episodes, according to Mukkavilli et al. [21]. In another study [22], a total of 400 sites were carefully selected from AERONET. The result suggested that MERRA-2 had a slight negative bias against AERONET AOT. However, one of the significant challenges in China is the relative scarcity of these monitoring sites. The lack of AERONET sites and subsequent data can limit the ability to validate the MERRA-2 dataset across the vast and diverse regions of China. In addition, when generating the MERRA-2 AOT reanalysis dataset, AERONET observation is one of the datasets used for assimilation [9]. This fact brings about potential circular reasoning issues if validating the MERRA-2 AOT data directly against AERONET AOT data, which may lead to an overestimation of the accuracy of the MERRA-2 data. Using additional, independent datasets for validation of MERRA-2 AOT is necessary.

The National Civil Space Infrastructure Satellite Aerosol Product Validation Network (SIAVNET), a network of dozens of monitoring stations in mainland China that started to collect data in 2019, was established by the Aerospace Information Research Institute, Chinese Academy of Sciences as part of China’s National Civil Space Infrastructure Medium and Long-term Development Plan [23,24]. SIAVNET improves the uneven and sparse distribution of AERONET in China, thus supporting satellite and model aerosol product validation.

In this research, we validate the MERRA-2 AOT dataset in China using various SIAVNET ground-based measurements. The following sections of the paper are organized as: the dataset and methodology used in this research are introduced in Section 2; the validation results are demonstrated in Section 3; and the discussion and conclusion are given in Section 4.

2. Data and Methodology

2.1. MERRA-2 AOT Data

The MERRA-2 AOT data product provides a global, high-resolution (0.625-degree) record of AOT from 1980 to the present [25]. The data are derived from a combination of several AOT observation sources (including satellite and ground-based measurement) and a numerical atmospheric model of GEOS-5. The data are available at different time resolutions, ranging from monthly averages to hourly data. In this research, the hourly data are selected for validation because of their correspondence reason with the time-point-specific ground measurement data. The midpoint of each hour is set as the corresponding time for the hourly data.

2.2. SIAVNET

The SIAVNET network employs high-precision CIMEL Electronique (CE)-318 sun-sky photometers, which use the direct and diffuse radiation flux density to determine aerosol physical–optical properties [26]. To ensure data quality, SIAVNET uses the same calibration method as AERONET, including centralized annual calibration of instruments against the master instrument, which is precisely calibrated using the Langley method at AERONET/PHOTONS sites [27].

SIAVNET provides aerosol products, such as AOT and Angstrom Exponent (AE), classified into Level 1.0 and Level 1.5 quality levels, consistent with the AERONET data level protocol. Level 1.0 data are the preliminary product calculated from direct solar radiation measurements, while Level 1.5 is cloud-screened Level 1.0 data [19].

2.2.1. Sites Distribution

The distribution of 21 nationwide SIAVNET sites used in this research is shown in Figure 1. The time period of the available data ranges from 2019 to 2021.

2.2.2. Data Quality

When designing the SIAVNET sites, a site named Beijing was set for the convenience of comparison purposes with the AERONET, which also has a site named Beijing_RADI in the same place. Therefore, by setting the long-term AERONET observation as a benchmark, by comparing the observation data between Beijing and Beijing_RADI, the data quality of SIAVNET sites can be evaluated. The same evaluation has been conducted in previous research [23,28]. Here we show an additional comparison between the two datasets in terms of 550 nm AOT. Though the original aerosol products do not contain the AOT at 550 nm, the 550 nm AOT is an essential parameter in climate and environmental studies. This wavelength is important mainly because it corresponds to both the maximum energy wavelength (thus important for climate study) of solar radiation and the most sensitive wavelength of human eyes (important for environmental study).

The 550 nm AOT can be calculated from AOT at other known wavelengths using the Angstrom Exponent [29] as:

$$\tau \left({\lambda }_1\right)=\tau ({\lambda }_0){\left(\frac{{\lambda }_1}{{\lambda }_0}\right)}^{-\alpha }$$

(1)

where $\tau$ is AOT, ${\lambda }_1$ is the target wavelength, ${\lambda }_0$ is the known wavelength, and $\alpha$ is the Angstrom Exponent.

However, the Angstrom Exponent may vary between different bands. The specific Angstrom Exponent cannot be provided by the original aerosol products. Alternatively, when the Angstrom Exponent is unknown, the previous equation can be derived from the following equation, as an additional AOT at known wavelength is introduced to eliminate the Angstrom Exponent from the formula:

$$\tau \left(\lambda \right)=e^{\left(ln\left(\tau \left({\lambda }_1\right)\right)+\frac{ln\left(\frac{\lambda }{{\lambda }_1}\right)ln\left(\frac{\tau \left({\lambda }_2\right)}{\tau \left({\lambda }_1\right)}\right)}{ln\left(\frac{{\lambda }_2}{{\lambda }_1}\right)}\right)}$$

(2)

where ${\lambda }_1$ and ${\lambda }_2$ are the known wavelengths, and $\lambda$ is the new target wavelength. This can be achieved by assuming the Angstrom Exponent remains the same between the two used wavelengths. Thus the closer the two used wavelengths are, the more accurate the calculated 550 nm AOT will be. In this research, the closest available wavelengths of 500 nm and 675 nm, which bracket 550 nm, are selected as ${\lambda }_1$ and ${\lambda }_2$.

Figure 2 shows the scatter plot between SIAVNET and AERONET data at the Beijing site. The time difference threshold for the matched data points is set to 15 min. High accuracy is proved for the SIAVNET equipment with a linear fitting result of y = 1.001x + 0.008 and R² equals 0.995.

2.3. MERRA-2 AOT Modeling Data Validation Method

In this research, a series of comparison diagrams are used to validate the MERRA-2 AOT Modeling Data with SIAVNET ground-based measurement.

2.3.1. Box Plot

The box plot, also known as a box-and-whisker plot, is a type of graphical representation of data that displays the distribution of a dataset. It is widely used in statistics and data visualization to show the range, central tendency, and variability of the data.

A box plot is created to visualize the distribution of the MERRA-2 AOT modeling data for all sites, grouped by intervals of the SIAVNET measurement. The intervals are created by flooring the SIAVNET measurement values to the nearest 0.1.

The box represents the interquartile range (IQR), which contains the middle 50% of the data. The bottom of the box is the first quartile (Q1) or the 25th percentile, and the top of the box is the third quartile (Q3) or the 75th percentile. The whiskers extend from the box to the minimum and maximum values within 1.5 * IQR from Q1 and Q3. In addition, the mean values of the MERRA-2 AOT modeling data for each interval are shown in the figure as red dots. The number of data points in each bin and the slope values between consecutive mean points are also displayed above the corresponding bin.

2.3.2. Density Scatter Plot

The density scatter plot is a type of graphical representation used to visualize the relationship between two variables while also showing the density of data points in different regions of the plot. A density scatter plot is useful when there are a large number of data points, making a traditional scatter plot appear cluttered or difficult to interpret.

In this research, a hexbin plot is created to visualize the relationship between the SIAVNET AOT measurement and MERRA-2 AOT modeling data. A linear regression is performed on the data, and the resulting regression line (fit line) is added to the plot. A dashed line with a slope of 1 is also added to the plot to provide a visual reference for a perfect relationship between the two variables.

The Global Climate Observing System (GCOS) defines the requirements for the characterization of data records used by climate researchers. For AOT at 550 nm, the minimum requirement to be met to ensure that data are useful is 20% or 0.06 [30]. In this research, two boundary lines corresponding to the minimum requirement are plotted along with the percentage of data points falling within the boundary lines printed on the graph.

2.3.3. Time Series Comparison

A time series plot is a type of graph used to visualize how variables change over time. It is a useful tool to give a general comparison of the variables’ trend and their relationship. In this research, a time series plot is used to compare the AOT obtained from the SIAVNET measurement and the MERRA-2 modeling, allowing us to visualize their similarities and differences over time.

3. Results and Discussion

3.1. General Validation Using All SIAVNET Sites

A general validation using all SIAVNET sites was conducted in this research using box plots and density scatter plots. Only the data pairs with a time difference between SIAVNET and MERRA-2 less than 15 min are selected and shown in the figures. Figure 3a is the box plot. The six density scatter plots represent the result with AOT value of SIAVNET observation not controlled (Figure 3b), less than 2.5 (Figure 3c), 2.0 (Figure 3d), 1.5 (Figure 3e), 1.0 (Figure 3f), and 0.5 (Figure 3g), respectively. The point numbers of Figure 3b–g are 8158, 8147, 8140, 8097, 7908, and 6847, respectively. The point number differences between the first few subfigures are small, indicating that the low aerosol loading cases are more common.

As can be seen in the general validation result, MERRA-2 AOT shows relatively high accuracy when the aerosol loading in the atmosphere is low. The accuracy of the MERRA-2 AOT goes down as the aerosol loading in the atmosphere increases. When the AOT is less than 1.0, the slope can reach 0.712 with R² = 0.584. The slope can further reach 0.773 when the AOT is less than 0.5. However, even in the case of AOT less than 0.5, the percentage of data pairs falling within the GCOS minimum requirement is still less than 60% (59.53% to be exact). When the true AOT is greater than 1.0, the simulation result is significantly lower than the ground-based measurements.

3.2. Validation for Different Sites

Along with the general validation for all SIAVNET sites, the validations for individual sites are also conducted. The results vary by site. To avoid the low reliability of the validation, only 12 sites, which have more than 200 matched data points within the time difference threshold of 15 min, are presented here. According to the elevation of the sites, they can further be categorized into two subcategories, i.e., low altitude sites, with elevation less than 500 m, and high altitude sites, with elevation greater than 1 km. Significant accuracy difference exists between these two subcategories.

3.2.1. Validation for Low Altitude Sites

The validation for 8 low altitude sites (

Table 1. Detailed information on the low altitude sites.

Site Name	Latitude	Longitude	Altitude (m)	Climate/Surface Type
Dongtinghu	29.355	113.132	100	Lakeshore
Jiangshanjiao	43.853	128.952	450	Forest
Jingyuetan	43.998	125.402	213	Cropland
Luancheng	37.892	114.689	27	Cropland
Nanjing	31.504	119.211	50	Cropland
Qingdao	35.937	120.170	41	Seashore
Qiyang	26.759	111.871	100	Shrub
Yucheng	36.831	116.570	7	Cropland

: Dongtinghu site, Jiangshanjiao site, Jingyuetan site, Luancheng site, Nanjing site, Qingdao site, Qiyang site, and Yucheng site) that meet the data point number and elevation criteria is conducted and shown (Figure 4 and

Table 2. Linear fitting results of low altitude sites.

Site Name	AOD ≤	Linear Fit	N	R2	RMSE	Within GCOS
Dongtinghu	2.5	y = 0.750x + 0.089	320	0.633	0.136	60.94%
	1.0	y = 0.803x + 0.071	310	0.646	0.117	62.26%
	0.5	y = 0.901x + 0.043	221	0.483	0.095	62.90%
Jiangshanjiao	2.5	y = 0.502x + 0.058	389	0.457	0.110	47.81%
	1.0	y = 0.567x + 0.043	382	0.435	0.102	48.43%
	0.5	y = 0.728x + 0.010	353	0.472	0.078	50.99%
Jingyuetan	2.5	y = 0.471x + 0.077	1058	0.504	0.104	60.21%
	1.0	y = 0.600x + 0.050	1040	0.531	0.094	61.15%
	0.5	y = 0.723x + 0.029	964	0.451	0.083	64.32%
Luancheng	2.5	y = 0.532x + 0.091	334	0.628	0.150	48.50%
	1.0	y = 0.783x + 0.013	311	0.695	0.103	50.80%
	0.5	y = 0.784x + 0.014	253	0.480	0.091	54.15%
Nanjing	2.5	y = 0.255x + 0.287	395	0.225	0.170	41.27%
	1.0	y = 0.472x + 0.207	351	0.320	0.152	46.15%
	0.5	y = 0.853x + 0.097	244	0.413	0.114	49.59%
Qingdao	2.5	y = 0.674x + 0.098	806	0.639	0.145	52.48%
	1.0	y = 0.815x + 0.057	771	0.654	0.128	54.47%
	0.5	y = 0.856x + 0.045	600	0.413	0.111	57.33%
Qiyang	2.5	y = 0.686x + 0.091	434	0.572	0.151	50.46%
	1.0	y = 0.774x + 0.056	418	0.579	0.129	51.44%
	0.5	y = 0.774x + 0.061	292	0.398	0.099	54.45%
Yucheng	2.5	y = 0.456x + 0.155	412	0.418	0.159	45.15%
	1.0	y = 0.674x + 0.074	389	0.487	0.137	47.81%
	0.5	y = 0.758x + 0.048	292	0.360	0.114	55.82%

) in this section. The time series comparison between SIAVNET observation and MERRA-2 simulation is given in Figure 4, showing that significant differences occur in high aerosol loading cases which is consistent with Section 3.1. The density scatter plots represent the result with a time difference of less than 15 min, meanwhile, AOT values of SIAVNET observation less than 2.5, 1.0, and 0.5 are given in Table 2.

Similar to the general validation for all SIAVNET sites, for individual sites, better linear fitting results can generally be obtained with low aerosol loading. The best slope of the linear fitting result lies in AOT value of SIAVNET observation less than 0.5 in Dongtinghu site, Jiangshanjiao site, Jingyuetan site, Luancheng site, Nanjing site, Qingdao site, Qiyang site, and Yucheng site, with the slope value of 0.901, 0.728, 0.723, 0.784, 0.853, 0.856, 0.774, and 0.758, respectively. The percentage of the data pairs falling within the GCOS minimum requirement in best slope scenario of each site are 62.90% (Dongtinghu), 50.99% (Jiangshanjiao), 64.32% (Jingyuetan), 54.15% (Luancheng), 49.59% (Nanjing), 57.33% (Qingdao), 54.45% (Qiyang), and 55.82% (Yucheng), respectively. The point numbers in such scenarios are 221 (Dongtinghu), 353 (Jiangshanjiao), 964 (Jingyuetan), 253 (Luancheng), 244 (Nanjing), 600 (Qingdao), 292 (Qiyang), and 292 (Yucheng), respectively.

3.2.2. Validation for High Altitude Sites

The validation for 4 high altitude sites (

Table 3. Detailed information on the high altitude sites.

Site Name	Latitude	Longitude	Altitude (m)	Climate/Surface Type
Guyuan	41.767	115.680	1075	Grassland
Haibei	37.611	101.313	3158	Grassland
Hami	42.084	94.921	1139	Gobi Desert
Minqin	38.575	102.984	1329	Gobi Desert

: Guyuan site, Haibei site, Hami site, and Minqin site) that meet the data point number and elevation criteria is conducted and shown (Figure 5 and

Table 4. Linear fitting results of high altitude sites.

Site Name	AOD ≤	Linear Fit	N	R2	RMSE	Within GCOS
Guyuan	2.5	y = 0.416x + 0.050	212	0.625	0.058	83.49%
	1.0	y = 0.421x + 0.049	210	0.321	0.048	83.81%
	0.5	y = 0.488x + 0.041	209	0.361	0.047	84.21%
Haibei	2.5	y = 0.412x + 0.069	831	0.230	0.069	72.44%
	1.0	y = 0.448x + 0.065	830	0.240	0.069	72.53%
	0.5	y = 0.565x + 0.051	823	0.296	0.066	73.15%
Hami	2.5	y = 0.683x + 0.099	302	0.538	0.103	57.95%
	1.0	y = 0.888x + 0.068	299	0.612	0.093	58.53%
	0.5	y = 1.115x + 0.036	290	0.587	0.086	58.28%
Minqin	2.5	y = 0.307x + 0.144	1341	0.238	0.095	52.80%
	1.0	y = 0.446x + 0.117	1332	0.306	0.090	53.15%
	0.5	y = 0.572x + 0.095	1272	0.276	0.080	54.80%

) in this section. The time series comparison between SIAVNET observation and MERRA-2 simulation is given in Figure 5. As seen in the time series, model simulations are generally higher than ground-based measurements, especially at the Minqin Gobi Desert site in spring during high dust aerosol loading events. The density scatter plots represent the result with a time difference of less than 15 min, meanwhile, AOT values of SIAVNET observation less than 2.5, 1.0, and 0.5 are given in Table 4.

Though the high-altitude sites also have generally better linear fitting results when aerosol loading is low, their accuracy is significantly lower than the low-altitude sites. The best linear fitting result lies in the AOT value of SIAVNET observation less than 0.5 in the Guyuan site, Haibei site, and Minqin site, with slopes of 0.488, 0.565, and 0.572, respectively. The percentage of the data pairs falling within the GCOS minimum requirement in the best slope scenario of each site are 84.21% (Guyuan), 73.15% (Haibei), and 54.80% (Minqin), respectively, which are higher (especially for Guyuan and Haibei) than low altitude sites. Such a higher value is due to generally low aerosol loading in the high altitude sites, not indicating higher accuracy than low altitude sites. Nevertheless, the linear fitting result in the Hami site is different from the general situation, which has a slope value greater than 1 when AOT is less than 0.5, meaning it is the only site having MERRA-2 simulation higher than SIAVNET observation. Such a phenomenon indicates that MERRA-2 product has low AOT simulation quality in (China’s) high altitude regions. The point numbers in AOT less than 0.5 scenarios are 209 (Guyuan), 823 (Haibei), 290 (Hami), and 1272 (Minqin), respectively.

3.3. Validation for Different Seasons

In addition to the general and regional validation, the validation is also conducted on a seasonal basis (Figure 6). The data pairs with a time difference between SIAVNET and MERRA-2 of less than 15 min for different seasons are used. According to the general validation result in Section 3.1, the simulation quality may also be improved when high aerosol loading data pairs are removed. However as this section focuses on seasonal variation of the simulation quality, the AOT value of SIAVNET observation is not controlled in this section.

As can be seen in the result, spring has the worst MERRA-2 AOT simulation quality with a slope of 0.405, bias of 0.147, RMSE = 0.168, and R² = 0.400. The MERRA-2 AOT simulation quality improved in the following seasons and achieved the best simulation quality in winter with a slope of 0.599, bias of 0.047, RMSE = 0.123, and R² = 0.601. The percentage of the data pairs falling within the GCOS minimum requirement in spring, summer, autumn, and winter are 52.82%, 54.90%, 54.96%, and 58.73%, respectively. The point numbers of spring, summer, autumn, and winter are 2592, 2182, 1814, and 1570, respectively.

The seasonal average of the Angstrom Exponent for data pairs in Figure 6 is given in

Table 5. Seasonal average of Angstrom Exponent for data pairs in Figure 6.

Season	Average Angstrom Exponent
Spring	0.97
Summer	1.00
Autumn	1.09
Winter	1.16

. Spring has the lowest average Angstrom Exponent of 0.97, indicating that, mainly due to the presence of dust aerosol, the particle size in spring is the largest. The Angstrom Exponent increases in the following seasons (1.00 in summer and 1.09 in autumn). Winter has the highest average Angstrom Exponent of 1.16, indicating that the particle size in winter is the smallest. There is a positive correlation between MERRA-2 AOT simulation quality and the Angstrom Exponent. And the low simulation quality in spring may be caused by the presence of dust aerosol.

4. Conclusions

In this study, we validate the MERRA-2 AOT dataset over China using various SIAVNET ground-based measurements. General validation using all SIAVNET sites indicates that the accuracy of the MERRA-2 AOT decreases as the aerosol loading in the atmosphere increases. When the AOT is less than 1.0, the slope can reach 0.712 with R² = 0.584. When the AOT is greater than 1.0, the simulation result is significantly lower than the ground-based measurements. The percentage of the data pairs falling within the GCOS minimum requirement is less than 60%. The validations for individual sites show that the MERRA-2 product has lower AOT simulation quality in (China’s) high-altitude regions than in low-altitude regions. The MERRA-2 AOT simulation quality also shows seasonal variation. Spring has the worst MERRA-2 AOT simulation quality with simulation quality improved in the following seasons and getting the best simulation quality in winter. The low simulation quality in spring may be caused by the presence of dust aerosol. HYSPLIT (Hybrid Single Particle Lagrangian Integrated Trajectory) model can be used in further studies to find out the aerosol origin and study the cause of simulation quality difference.

By introducing SIAVNET observation, this research compensates for the long-standing absence of MERRA-2 AOT dataset validation over China due to the uneven and sparse distribution of AERONET in this area. With the sustainable construction of the SIAVNET sites in the future, the validation of the MERRA-2 AOT dataset can be continually improved using datasets of longer time periods and more sites.