Evaluation and Comparison of MODIS C6 and C6.1 Deep Blue Aerosol Products in Arid and Semi-Arid Areas of Northwestern China

Abstract

The Moderate Resolution Imaging Spectroradiometer (MODIS) Deep Blue (DB) algorithm was developed for aerosol retrieval on bright surfaces. Although the global validation accuracy of the DB product is satisfactory, there are still some regions found to have very low accuracy. To this end, DB has updated the surface database in the latest version of the Collection 6.1 (C6.1) algorithm. Some studies have shown that DB aerosol optical depth (AOD) of the old version Collection 6 (C6) has been seriously underestimated in Northwestern China. However, the status of the new version of the C6.1 product in this region is still unknown. This study aims to comprehensively evaluate the performance of the MODIS DB product in Northwestern China. The DB AOD with high quality (Quality Flag = 2 or 3) was selected to validate against the 23 sites from the China Aerosol Remote Sensing Network (CARSNET) and Aerosol Robotic Network (AERONET) during the period 2002–2014. By the overall analysis, the results indicate that both C6 and C6.1 show significant underestimation with a large fraction of more than 54% of collocations falling below the Expected Error (EE = ±(0.05 + 20% AOD_ground)) envelope and with a large negative Mean Bias (MB) of less than −0.14. Furthermore, the new C6.1 products failed to achieve reasonable improvements in the region of Northwestern China. Besides, C6.1 has slightly fewer collocations than C6 due that some pixels with systematic biases have been removed from the new surface reflectance database. From the analysis of the site scale, the scatter plot of C6.1 is similar to that of C6 in most sites. Furthermore, a significant underestimation of DB AOD was observed at most sites, with the most severe underestimation at two sites located in the Taklimakan Desert region. Among 23 sites in Northwestern China, there are only two sites where C6.1 has largely improved the underestimation of C6. Furthermore, it is interesting to note that there are also two sites where the accuracy of the new C6.1 has declined. Moreover, it is surprising that there is one site where a large overestimation was observed in C6 and improved in C6.1. Additionally, we found a constant value of about 0.05 for both C6 and C6.1 at several sites with low aerosol loading, which is an obvious artifact. The significant improvements of C6.1 were observed in the Middle East and Central Asia but not in most sites of Northwestern China. The results of this study will be beneficial to further improvements in the MODIS DB algorithm.

1. Introduction

Aerosols play a crucial role in the climate system and the hydrologic cycle, and are the largest uncertainty in the radiation budget of the earth [1,2]. In addition, a large number of studies have shown that aerosols have a substantial influence on air pollution [3], visibility [4,5], human health [6,7] and global or regional climate change [8,9]. Allowing for the high spatiotemporal variability of aerosol properties, such as aerosol optical depth (AOD), satellite observations are increasingly used to obtain the distribution of AOD on both global and regional scales. However, due to the aerosol retrieve algorithm including many approximations and assumptions to the underlying surface, the accuracy of AOD obtained from the satellite shows obvious regional characteristics. Therefore, extensive evaluation of satellite-derived AOD is essential for regional application.

MODIS aboard the Terra (since 2000) and Aqua satellite (since 2002) is considered to be one of the most mature and extensively studied sensors, which can provide operational aerosol products. MODIS has 36 spectral bands spanning from 0.415 μm to 14.5 μm at three relatively fine resolutions (250 m, 500 m, 1000 m) and provides global daily products because of its wide swath (2330 km) and high temporal resolutions (two observations per day) [10].

Due to the strong reflection of the earth’s surface, there are difficulties in retrieving aerosols over land from satellite observations coupled with the earth and atmosphere. Before the Terra launch, Kaufman et al. (1997) proposed a Dark Target (DT) algorithm to retrieve aerosol properties over dense vegetation areas from MODIS [11,12]. Later, in the product of Collection 5 (C5) and Collection 6 (C6), Levy et al. (2007, 2013) extended DT to a more general dark surface with a reflectance of less than 0.25 [10,13]. In the aerosol retrieval algorithm, DT uses the linear ratio between visible (0.47 μm and 0.66 μm) and shortwave infrared (2.12 μm) bands as a priori knowledge to estimate the surface reflectance at visible bands. Due to the high surface reflectance and complex surface ratio, DT failed to cover those bright land surfaces (e.g., desert, arid and semi-arid areas). Hsu et al. (2004) developed the Deep Blue (DB) algorithm that filled in the gaps in DT [14]. DB assumes that the bright surface is dark in the deep blue band, and uses a pre-calculated clear-day surface database as a priori knowledge for estimating surface reflectance in the aerosol retrieval algorithm [14,15]. The DB algorithm was initially introduced in the MODIS C5 product, but was restricted to bright reflective surfaces. When the MODIS C6 product was released in 2013, an improved algorithm named ‘enhanced Deep Blue’ expanded coverage to all surfaces except for snow/ice and cloud mask surfaces [16]. DB aerosol products have a good performance on bright surfaces from global and regional evaluation [17,18,19,20,21,22,23,24,25]. Nevertheless, there are still some regions found to have very low accuracies [17,22,26,27,28,29,30,31,32]. For example, Mohsin JamilButt et al. (2017) found that the Aqua DB AOD was underestimated in Solar_Village and KAUST [28]. Furthermore, Sayer et al. (2013) found systematic underperformance in the Middle East [17]. To this end, DB has updated the surface database in the latest version of the Collection 6.1 (C6.1) algorithm [33]. Furthermore, the accuracy of the C6.1 aerosol product has been improved in many regions.

Northwestern China is covered by vast arid and semiarid areas and deserts, which are some of the largest dust sources in the world. The topography of Northwestern China is complex, with majestic plateaus, undulating mountains and vast basins. The land cover types in this region include a large area of desert, Gobi, grassland, and a small part of forest and snow mountains. As a result, Northwestern China is sparsely populated and largely affected by natural source aerosols. Since the development of the DB product, the aerosol optical properties in this region can be monitored from MODIS. Before being applied to regional air quality and climate studies, MODIS DB products should be fully evaluated. However, only a few attempts have been made to validate MODIS DB products in the arid and semiarid areas of Northwestern China. Tao et al. (2017) evaluated MODIS DB aerosol products of C6 in Northwestern China using only eight sites of the China Aerosol Remote Sensing Network (CARSNET) from 2003 to 2013. It was found that there was a significant underestimation of AOD at all sites and a constant low value of 0.05 occurred at several sites [26]. This preliminary study suggests that there may be an underestimation of DB AOD products in Northwestern China similar to the region of the Middle East and Central Asia. Huang et al. (2020) found that a DB AOD of C6.1 was also underestimated at one Sun–sky radiometer Observation NETwork (SONET) site of Kashi [30]. Therefore, it is urgent to understand the improvement of C6.1 products throughout Northwestern China.

In this paper, we collected 2 AERONET and 21 CARSNET sites with continuous ground-based aerosol measurements during the period 2002–2014 to systematically evaluate MODIS DB AOD products in Northwestern China. By comparing the validation results of C6 and C6.1 products, this study aims to assess whether the updates of the C6.1 DB algorithm achieve the improvement in AOD products in this region. The results of this work are expected to provide an indicator for the improvement of the DB algorithm in the region of Northwestern China.

2. Data and Method

2.1. CARSNET and AERONET Data

A global ground-based aerosol network called the Aerosol Robotic Network (AERONET) is established by NASA and its collaborators [34]. AERONET provides a long-term and continuous measurement of aerosol properties for aerosol research and validation of satellite aerosol products [18,35]. The AOD products from AERONET have a low uncertainty (~0.01–0.02) with a high temporal resolution (15 min) [36], and are processed for three levels: Level 1.0 (unscreened AOD), Level 1.5 (cloud-screened AOD), and Level 2.0 (quality-assured AOD) [37]. This paper adopts the Level 2.0 product of the latest version 3, last accessed on 22 April 2021. However, AERONET sites are sporadic in Northwestern China. In this paper, only 2 AERONET sites (AOE_Baotou and SACOL) within the region are selected, which are labeled as red dots in Figure 1 and marked with * in

Table 1. Detailed information for CARSNET and AERONET in Northwestern China (Two sites marked with * are AERONET sites, and the remaining 21 are CARSNET sites).

No.	Sites Name	Longitude	Latitude	Altitude (m)	Observation Time
	Remote sites
1	Akedala	87.97	47.12	562.0	October 2009–August 2014
2	Mt. Waliguan	100.92	36.28	3810.0	March 2009–April 2012
	Desert sites
3	Dunhuang	94.68	40.15	1139.0	June 2002–November 2014
4	Ejina	101.07	41.95	940.5	May 2002–November 2014
5	Hami	93.52	42.82	737.0	April 2002–March 2005
6	Hotan	79.93	37.13	1374.7	May 2002–March 2005
7	Jiuquan	98.48	39.77	1477.3	April 2002–March 2005
8	Minqin	103.08	38.63	1367.0	February 2004–April 2012
9	Tazhong	93.67	39.00	1099.4	January 2004–July 2014
10	Ulate	108.52	41.57	1288.0	April 2002–February 2005
11	Xilinhot	116.12	43.95	1003.0	April 2002–November 2014
12	Zhangbei	114.70	41.15	1093.4	January 2004–March 2005
13	Zhurihe	112.90	42.40	1152.0	April 2002–March 2005
14	AOE_Baotou *	109.63	40.85	1270.0	September 2013–December 2014
	Transitional regions
15	Dongsheng	109.98	39.83	1460.5	April 2002–March 2005
16	Mt. Gaolan	103.85	36.00	2161.6	June 2004–April 2012
17	Yanan	109.50	36.60	958.5	January 2004–March 2005
18	SACOL *	104.14	35.95	1965.0	July 2006–May 2013
19	Yulin	109.20	38.43	1135.0	September 2008–April 2012
	Western urban and built-up regions
20	Urumqi	87.62	43.78	935.0	April 2002–November 2014
21	Datong	113.33	40.10	1067.3	May 2002–November 2014
22	Yinchuan	106.22	38.48	1111.5	May 2002–August 2004
23	Lanzhou	103.88	36.05	1517.3	July 2002–November 2014

. Since 2002, China Meteorological Administration has started to establish CARSNET including more than 50 sites throughout China [38,39]. AOD products from CARSNET are expected to have the similar accuracy as AERONET, which has been validated by Che et al. (2009) conducting simultaneous observations between two sites in Beijing for one year [38]. In this work, 21 sites of CARSNET were collected in the study region. Considering the effect of different land surface conditions and aerosol properties on aerosol retrieval accuracy, a total of 23 sites (2 AERONET and 21 CARSNET) were roughly classified into four groups according to Che et al. (2015) [39], including: (1) remote sites (two sites), which are affected by natural source aerosol with sparsely anthropogenic activities; (2) desert sites (twelve sites), in which dust aerosol particles dominate with small anthropogenic influences; (3) transitional regions (five sites), which are affected by dust and stronger human activity than desert sites; (4) urban and built-up sites (four sites), which are affected by significant anthropogenic emissions. Figure 1 illustrates the corresponding geographical locations of 23 sites and the summary information about these sites are given in Table 1.

2.2. MODIS Deep Blue Aerosol Products

MYD04 (MODIS/Aqua) Level 2 (L2) data from 2002 to 2014, including both C6 and C6.1 DB 10 km AOD products, were collected for validation in this paper. The basic principle of DB algorithms is to utilize the pre-calculated land surface reflectance database in deep blue bands (0.412 μm), in which surface reflectance is relatively lower than those in longer bands [14,15]. The initial MODIS C5 products produced by the DB algorithm only provide aerosol results on bright surfaces. Yet, C6 products introduce the ‘second generation’ DB algorithm updated by Hsu et al. (2013) extended from bright surfaces to entire land surfaces (except snow/ice), including densely-vegetated and other dark surfaces [16]. The ‘second generation’ DB algorithm contains the improvement of cloud and snow/ice screening, a new Normalized Difference Vegetation Index (NDVI)-dependent surface reflectance, an improved aerosol model scheme, revised data quality flags and so on [16]. The latest C6.1 DB products made some further modifications on the basis of C6 products: (a) reducing artifacts in heterogeneous terrain, (b) improving surface reflectance modeling in elevated terrain, (c) updating the assumed aerosol model in some areas, and (d) updating some metadata such as Ångström exponent and so on. The modifications listed above can be found in https://modis-atmosphere.gsfc.nasa.gov/sites/default/files/ModAtmo/modis_deep_blue_c61_changes2.pdf (accessed on 22 September 2020). In this paper, only high quality (QA = 2 or 3) AOD products are selected for the validation work.

2.3. Validation Methods

Considering the difference in the sampling strategies between ground-based and satellite measurements, a spatio-temporal matching scheme should be carried out for validation purposes. Here, we adopted the revised protocol proposed by Petrenko et al. (2012) [40], which averaged the ground-based measurements within ±30 min of the Terra/Aqua satellite overpass time and the satellite observation within a 25 km radius of a selected site. A valid matching collocation requires at least three AOD values of MODIS pixels and two ground-based measurements within the spatio-temporal matchup window. Since the sun photometer does not provide the observations at 550 nm, the AOD data at 550 nm are interpolated using available AOD measurements at 440 nm and 675 nm and the Ångström exponent (α) in 440–675 nm [17,21], defined by Equation (1):

$${\alpha }_{440-675}=-\frac{ln\left({\tau }_{440}/{\tau }_{675}\right)}{ln\left(440/675\right)}$$

(1)

where ${\tau }_{\lambda }$ is the AOD at a corresponding wavelength ($\lambda$) at 440 nm and 675 nm and ${\alpha }_{440-675}$ is the Ångström exponent between the wavelengths of 440 nm and 675 nm. It should be stated that all references to ‘AOD’ indicate 550 nm in the following analysis.

To quantitatively demonstrate the accuracy of the MODIS AOD compared to the ground-based AOD, an orthogonal regression technique was applied for all the matching plots to estimate the slope and intercept, which are associated, respectively, with the error of the assumed aerosol model and the uncertainty of surface reflectance estimation [41]. Additionally, three statistical parameters of the Pearson correlation coefficient (R, Equation (2)), Root-Mean-Square-Error (RMSE, Equation (3)), and Mean Bias (MB, Equation (4)) are calculated to quantify the uncertainty [42]. Moreover, the Expected Error (EE) was used to evaluate the accuracy. The fractions of collocations within, above and below the EE envelope were then calculated to indicate the overall accuracy, over-and under-estimation of the retrieved AOD, respectively. As reported by Levy et al. (2013) [10], when the fraction of collocations falling into EE reaches 67% (about 2/3), it is considered to have achieved a satisfactory retrieval accuracy. Referring to the [16,20], Equation (5) was adopted as the EE envelope in this paper.

$$\mathrm{R}=\frac{{{\displaystyle \sum }}^{ }\left({\mathrm{x}}_i-\overline{x}\right)\left({\mathrm{y}}_i-\overline{y}\right)}{\sqrt{{{\displaystyle \sum }}^{ }{\left(\left({\mathrm{x}}_i-\overline{x}\right)\right)}^2{{\displaystyle \sum }}^{ }{\left(\left({\mathrm{y}}_i-\overline{y}\right)\right)}^2}}$$

(2)

$$\mathrm{MB}=\frac{1}{N}{{\displaystyle \sum }}_{i=1}^n\left({\mathrm{AOD}}_{{\left(\mathrm{MODIS}\right)}_{\mathrm{i}}}-{\mathrm{AOD}}_{{\left(\mathrm{ground}\right)}_{\mathrm{i}}}\right)$$

(3)

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^n{\left({\mathrm{AOD}}_{{\left(\mathrm{MODIS}\right)}_{\mathrm{i}}}-{\mathrm{AOD}}_{{\left(\mathrm{ground}\right)}_{\mathrm{i}}}\right)}^2}$$

(4)

$$\mathrm{EE}=\pm \left(0.05+20{\%\mathrm{AOD}}_{\mathrm{ground}}\right)$$

(5)

3. Results

3.1. Evaluation of MODIS C6 and C6.1 DB AOD Products in the Middle East and Central Asia

As mentioned above, several previous studies have shown that the MODIS DB products exhibit a systematic underestimation of AOD in some semi-arid and arid areas [26,27]. An improved surface model for heterogeneous terrain, particularly in areas of the Middle East and Central Asia, was added to the C6.1 DB algorithm to remove some systematic biases [20,21]. In this paper, we also validate the MODIS C6/C6.1 DB product in four AERONET sites in the Middle East and Central Asia. The geographic location of these four sites can be seen in Figure S1 (Supplementary Information). The scatterplots of the validation results are shown in Figure 2. The red and green solid lines represent the X = Y line and regression line, respectively, and the dashed lines are the upper and lower EE bounds. As can be seen from the left four scatter plots in Figure 2, the C6 DB AOD is significantly underestimated in all four sites. However, the C6.1 products have a better agreement with AERONET AOD, and collocations in the right four scatter plots are more concentrated on both sides of the Y = X line.

For the sites of IASBS in Iran and Kandahar in Afghanistan, a nearly constant value of around 0.05 was found in the scatter plots of C6 products. Both sites exhibit a serious underestimation with a large fraction of 64.63% and 96.43% collections falling below EE and with large negative MB of −0.13 and −0.219, respectively. Furthermore, a poor correlation can be seen with a low R of 0.689 and 0.556 between DB and AERONET AOD, and with only 35.37% and 3.57% falling within EE, respectively. However, the accuracy of the C6.1 DB products has been considerably improved in these two sites. By comparison, the collocations are completely lifted in the scatter plots of C6.1 with 75.79% and 64.52%, respectively, falling within EE. In addition, the correlation between the C6.1 DB and AERONET AOD had a qualitative leap with a high R of 0.93 and 0.83, respectively. In spite of the underestimation still existing in the C6.1 DB algorithm, it showed a smaller degree of underestimation, with a lower MB of −0.029 and −0.067, respectively. It was worth noting that the number of collocations of the IASBS site decreased sharply from 523 in C6 to 95 in C6.1 due to the improved Quality Assurance (QA) test removing the poor quality pixels in this area. For the Dushanbe site in Tajikistan, the scatter plot of C6 products had a good correlation with a high R of 0.863, but it appeared to have a significant underestimation with 80.04% falling below EE and a large negative MB of −0.177. Nevertheless, the C6.1 outperformed the C6 with 82.48% collocations falling within EE, and achieved a lower MB of −0.024 close to 0. Further, a higher R of 0.907 and a lower RMSE of 0.104 were found in C6.1 than in C6. For the Solar_Village site in Saudi Arabia, both C6 and C6.1 products exhibited a good performance with a higher percent of 69.50% and 65.57% collocations, respectively, falling within EE. Although the slightly higher fraction of collocations within EE for C6, C6.1 was found to have a lower RMSE and a higher correlation of R than C6. Additionally, compared with the scatter plots of Solar_Village, C6.1 removed the collocations with low AODs around 0.05 happened in C6. Conclusively, the new C6.1 DB product achieves better performance than C6 in these four sites. The update of the surface database in the C6.1 DB algorithm may be effective in the region of the Middle East and Central Asia.

3.2. Evaluation of MODIS C6 and C6.1 DB AOD Products in Northwestern China

According to the analysis in Section 3.1, the new C6.1 DB AOD products effectively improve the systematic underestimation of the C6 product in the Middle East and Central Asia. However, whether the C6.1 DB improves the substantial underestimation of the C6 DB in Northwestern China remains to be validated.

The scatter plots in Figure 3 show the overall validation results of C6 and C6.1 DB AOD products against the ground-based observations of all 23 sites in Northwestern China, respectively. In general, the scatter plots of the two products look very similar, and the values of all statistical parameters are also quite close. Both C6 and C6.1 exhibited a serious underestimation with a large fraction of more than 54% collocations falling below EE much higher than those above EE, and with a large negative MB less than −0.14. The retrieval accuracy of C6.1 is almost the same poor as that of C6, and the percentage of collocations within EE is only about 40%. As can be further seen from Figure 4, the underestimation of both C6 and C6.1 increases with increasing aerosol loading. When the ground measured AOD is less than 0.2, the MB falls in the EE envelope, and the fraction of collocations within EE exceeds 60%. When the AOD is greater than 0.2, the MB falls below EE and gradually moves away from the envelope with the increase of AOD, and the fraction of collocations within EE decreases sharply, even reaching 25%, while comparing the two products in Figure 3, the number of matched collocations for C6.1 seems to be slightly less than C6 due to the new surface database and QA tests for elevated terrain, removing some poor retrievals. Thus, the fraction of collocations within EE and all other statistical parameters such as R, RMSE, and MB for C6.1 showed slightly better than those of C6, which can be also reflected in Figure 4. In general, the MODIS DB products are seriously underestimated in Northwestern China, and the new C6.1 failed to achieve reasonable improvements in this region. Therefore, the DB algorithm still needs to be further modified in the region of Northwestern China.

3.3. Evaluation of MODIS C6 and C6.1 DB AOD Products at the Site Scale in Northwestern China

The accuracy of MODIS aerosol retrieval mainly depends on the surface reflectance estimation and aerosol model assumption [43,44]. Although the overall performance of MODIS DB products in Northwestern China is poor, there are still large differences in accuracy between different sites due to the surface estimation and aerosol model selection at each site. Therefore, this section carries out a rough evaluation on the site scale according to the four groups of rough classification of all 23 sites in Section 2.1. Figure 5, Figure 6, Figure 7 and Figure 8 plot the validation results of MODIS C6 and C6.1 DB products at each site in Northwestern China. The detailed statistical parameters of the scatter plots of each site are given in Table S1 (Supplementary Information).

3.3.1. Validation of MODIS C6 and C6.1 DB at Desert Sites

The validation results at all 12 sites in the desert region are shown in Figure 5. The scatter plots of C6 and C6.1 are similar for each site. Not surprisingly, a large degree of underestimation exists on almost all sites. This means that the C6.1 DB products have not changed significantly in these sites in desert regions. For Tazhong and Hotan, respectively, located in the central and southern edge of the Taklimakan Desert in China, both DB products exhibit significant underestimation with a particularly large fraction of more than 80% collocations falling below the EE but few above EE. Further, the underestimation at Tazhong and Hotan in all aerosol conditions may indicate an overestimation of the surface reflectance [18]. For other sites including Minqin, Ulate, AOE_Baotou, Zhurihe, Zhangbei, and Xilinhot near small deserts close to Eastern China, there is still severe underestimation in almost all sites, but they are better than Tazhong and Hotan. The rest of the sites located on the Gobi, including Hami, Jiuquan, Dunhuang, and Ejina, also appear to be also less underestimated than Tazhong and Hotan. Among all 12 desert sites, it is interesting that DB achieved the best performance at Ejina with more than 75% of collocations falling within the EE for both C6 and C6.1. It also has a high correlation coefficient and low RMSE. Although a higher retrieval accuracy and lower RMSE were found from a few collocations at AOE_Baotou, it seems that DB would be underestimated in the case of an AOD greater than 0.25, which results in a low slope far less than 1 and a large negative MB. In addition, similar to the findings of Tao et al. (2017), a constant DB AOD value of around 0.05 in low aerosol conditions appears in Hotan, Xilinhot, Zhurihe, and Ulate [26]. Moreover, it also appears in Tazhong, Minqin, Zhangbei, and Dunhuang, which can be regarded as an artifact in the DB algorithm.

3.3.2. Validation of MODIS C6 and C6.1 DB at Transitional Regions

Figure 6 shows the validation results of DB AOD at five sites on the Loess Plateau, which is a transitional region between the western desert and Gobi region and the North China Plain. The retrieval accuracy of C6 exhibits a significant underestimation at all five sites, with over 60% of retrievals falling below EE, but less than 5% of them above EE. Although the correlation R of Dongsheng, Yanan, and SACOL is greater than 0.85, they all achieve poor performance with less than 36% of collocations falling within EE. This may be due to the underestimation of AOD at lower aerosol loading, resulting in more collocations falling below EE and a large negative intercept of the linear regression. Furthermore, C6.1 at Dongsheng, Yulin, and Yanan has almost no improvements, with almost identical statistical parameters of scatter plot as C6. Surprisingly, the new C6.1 at SACOL showed a considerable improvement with a lower RMSE of 0.123 and an extremely minor underestimation of MB close to 0. Furthermore, the fraction of collocations falling within EE increases from 35.99% in C6 to 75.04% in C6.1, achieving a satisfactory retrieval accuracy. However, there exists a slight underestimation with many collocations below the Y = X line at high aerosol loading, resulting in a small slope of 0.743 of the linear regression. Therefore, due to the influence of dust particles, more absorption needs to be considered in the aerosol models in the DB algorithm in the SACOL site. In addition, C6.1 has also been improved at the Mt. Gaolan site with fractions within EE increasing from 23.55% to 48.98%, but still not achieving a satisfactory retrieval accuracy.

3.3.3. Validation of MODIS C6 and C6.1 DB at Western Urban and Built-Up Regions

The four urban and built-up sites are located in four cities of Northwest China, where the urban areas are characterized by sparse vegetation around their built-up. The comparisons of the MODIS DB AOD against CARSNET AOD in all four sites are shown in Figure 7. For each site, C6.1 shows no improvement and has a similar scatter plot to C6. All four sites exhibit severe underestimation, and the statistical parameters of the scatter plot are even slightly inferior to those of Tazhong and Hotan in the desert region. The most significant underestimation is found at the Lanzhou site, and a constant value of about 0.05 for DB AOD is also observed in the scatter plot of C6. Among four sites, it has the highest RMSE and largest negative MB, while C6.1 has been slightly improved, the constant value disappears, but it is still significantly underestimated. The pattern of the scatter plot for the Urumqi site is similar to that of the Lanzhou site, and C6.1 has been slightly improved. However, the statistical parameters of Urumqi are better than those of Lanzhou. Interestingly, compared with C6, the retrieval accuracy of C6.1 in the Yinchuan site declines greatly with more than 90% of collocations falling below EE and a higher RMSE and larger MB, but the correlation coefficient R has strangely increased and is better than that of C6. The accuracy of the Datong site also decreased slightly, and the scatter plot of C6.1 appears to have a constant value at low aerosol loading.

3.3.4. Validation of MODIS C6 and C6.1 DB at Remote Sites

Figure 8 shows the scatter plots of C6 and C6.1 DB AOD compared with the CARSNET AOD at two remote sites. For the Mt. Waliguan site located in the Tibetan Plateau, only a few collocations were matched for both C6 and C6.1. The scatter plots are similar, and statistical parameters indicate a satisfactory retrieval at this site for both products. Surprisingly, unlike the whole validation results in Section 3.2, a large overestimation of C6 is observed at Akedala with 80.71% of collocations falling above EE and a large positive MB of 0.134, which may indicate an underestimation of the surface reflectance in the DB algorithm at this site. Due to the updates of the surface database, the fraction of collocations within EE has been improved with a satisfactory fraction of 71.24% for C6.1 compared with 19.29% for C6. In addition, the RMSE decreases from 0.155 in C6 to 0.081 in C6.1, and the MB decreases from 0.134 in C6 to −0.031 in C6.1. However, there are still a significant number of collocations that appear to be underestimated at a ground-based AOD greater than 0.2, resulting in a lower correlation of R = 0.317 in C6.1 decreased from 0.713 in C6 and the linear regression line is totally off the mark with a smaller slope of 0.223.

3.4. Comparison of the C6 and C6.1 DB AOD Products at Local Scales

As mentioned above, compared to C6, C6.1 does not get much improvement. In general, both C6 and C6.1 show serious underestimation on the whole. The scatter plots and statistical parameters of most sites are similar, and only a few sites exhibit large differences. Despite all this, this section compares C6 and C6.1 to evaluate which products perform better on the local scale. Figure 9 shows the better performance of two products in terms of various evaluation criteria (i.e., R, RMSE, MB, and percentage of retrievals within EE). The detailed statistical parameters of the scatter plots of each site can be found in Table S1. In Figure 9, the sites are marked with a blue or red color, indicating that C6 or C6.1 have better performance in the individual sites, and orange indicates that C6 and C6.1 perform equally. The criteria are based on Bilal et al. (2017), in which the threshold of relative difference is within 5% for RMSE and MB, and 10% for R and the percentage of retrievals within EE) [45]. As shown in Figure 9a, for the relative difference of R, C6 and C6.1 perform equally at 18 sites, and C6.1 outperforms C6 at only four sites. For the percentage of retrievals within the EE in Figure 9b, 7 out of 23 sites show improvements for C6.1 compared to C6, and C6.1 has equal or worse performance in the other 16 sites. In terms of RMSE and MB, C6.1 outperforms C6 at eight and six sites, respectively, in Figure 9c,d, C6.1 shows equal or worse accuracy on the other sites. In conclusion, the new C6.1 does not get more improvement than C6, and there is no regular pattern for this improvement at a local scale.

4. Discussion

This paper compared the MODIS C6 and C6.1 DB aerosol products against ground-based CARSNET and AERONET AOD measurements obtained from 23 sites in Northwestern China during the period 2002–2014, and the performance of the C6.1 DB in the region was deeply reported. Additionally, the validation of MODIS C6 and C6.1 DB AOD using AERONET observations in the Middle East and Central Asia was also conducted for comparison purposes. The results indicate that the MODIS C6 DB products show an evident underestimation in arid and semiarid areas including the Middle East, Central Asia, and Northwestern China. Although the C6.1 products have improved in the Middle East and Central Asia, it shows no significant improvements in Northwestern China except for only a few sites.

The performance of MODIS C6 AOD products has been evaluated in many previous studies. Fan et al. (2017) validated and compared the MODIS C5.1 and C6 AOD products against 16 AERONET sites in China, and found that the performances of C6 DT and DB products were better than C5.1 [46]. These results indicated that the surface reflectance estimation and aerosol model assumption in arid and semiarid regions were the main sources of errors in aerosol retrievals. Tao et al. (2015) evaluated the MODIS C6 aerosol products in China and found that DT was substantially higher than DB in eastern China [27]. Moreover, DB exhibited good performance in North China but poor performance in South and Northwest China. Mhawish et al. (2017) evaluated the MODIS C6 products in the Indo-Gangetic Plain [22]. The results showed that the retrieval accuracy of DB was lower than that of DT products. Meanwhile, DB showed an obvious underestimation in the Indo-Gangetic Plain in all aerosol conditions, which are all attributed to the overestimation of the surface reflectance and the aerosol single scattering albedo (SSA) [18]. A similar situation was also found in Northwestern China in our studies. Tao et al. (2017) made a comprehensive evaluation of MODIS C6 DB products against eight CARSNET sites in desert regions of Northwestern China, demonstrating a significant underestimation in all sites [26]. The validation results of our works are consistent with this underestimation in the eight sites. Furthermore, we found the severe underestimation also exited with additional 15 sites, and more sites included in our work are more representative. Furthermore, surprisingly, we find a large overestimation of C6 DB at the Akedala site (see Figure 8), which is the opposite of that at other sites. The finding indicates that the surface database of DB algorithm may also exit an underestimation of the surface reflectance somewhere in Northwestern China.

For the DB algorithm of the new C6.1, the MODIS DB team had made some major improvements including (a) improved smoke detection masks, (b) reduced artifacts in heterogeneous terrains, (c) improved surface reflectance modeling in elevated terrains, and (d) updated regional/seasonal aerosol optical models. The modifications in the C6.1 DB products removed some negative systematic biases and improved the retrieval accuracy in the Middle East and Central Asia. However, there are still some underestimations in some other places. For example, Sharma et al. (2021) evaluated the MODIS C6.1 AOD products in New Delhi, the capital of India, and found that DB showed significant underestimation [24]. Wei et al. (2019) validated the MODIS C6.1 products in 384 sites around the world and found significant negative deviations in semi-arid mountainous areas, namely western North America, southern South America and the Middle East [21]. Currently, there is no literature to systematically evaluate the performance of MODIS C6.1 DB products in Northwestern China. In this paper, a significant underestimation of C6.1 DB is also observed in the arid and semi-arid regions of northwestern China, which may indicate the overestimation of surface reflectance and the underestimation of aerosol absorption in the DB algorithm. It is worth noting that the aerosol retrieval accuracy in just a few sites like Mt. Gaolan, SACOL and Akedala have been improved in the C6.1 DB products, but the systematic underestimation still exists in most other sites in northwestern China. The MODIS DB products in northwestern China remain to be further improved.

5. Conclusions

This study presents a comprehensive comparison and evaluation of MODIS C6 and C6.1 DB AOD products against ground-based measurements from 23 CARSNET or AERONET sites in Northwestern China during 2002–2014. The results show that the MODIS DB products of both C6 and C6.1 have poor performance in northwestern China with only about 40% of the collocations in the scatter plot falling within the EE. This indicates that the MODIS DB products in northwestern China do not achieve the requirements of the EE (67%) reported by Levy et al. (2013) [10]. Furthermore, a significant underestimation was found for both C6 and C6.1 within a large fraction of more than 54% of collocations falling below EE and with a large negative MB of less than −0.14. Additionally, the C6.1 DB shows slightly better improvement than C6 with about 4.5% more collocations within EE and a slightly lower RMSE. This indicates that there is no substantial improvement achieved in C6.1, probably due to the fact that the update of the DB algorithm in the surface database and aerosol type has been slightly modified or has not played an adequate role in Northwestern China. Generally, it is believed that the AOD retrieval accuracy of the land aerosol algorithm is more affected by the estimation of surface reflectance at low aerosol loading, and more influenced by the assumption of aerosol properties at high aerosol load. Therefore, from the validation study in this paper, the further improvement of the DB algorithm should not only consider the possible overestimation of the surface database, but also pay attention to the possible existence of special aerosol types in the region of Northwestern China.

Furthermore, the validation at the site scale shows that the scatter plot of C6.1 is similar to that of C6 in most sites. Furthermore, an obvious underestimation of DB AOD was observed in most sites. From the validation results of four groups by the rough classification of all sites, desert and urban sites show more severe underestimation, of which the most severe is at two sites in the Taklimakan Desert region. It is worth noting that the DB AOD has a constant value of around 0.05 in the scatter plot of almost half of all sites when the AOD value of ground-based measurements is below 0.5. Compared with C6 and C6.1, the degree of underestimation has been largely improved only at two sites of SACOL and Mt. Gaolan. Furthermore, interestingly, a slight decline in the accuracy of the new C6.1 was found at two sites of Datong and Yinchuan. It is surprising that a large overestimation was observed in C6 at the Akedala site and improved in C6.1, which implies that DB AOD may be overestimated in some regions not covered by the sites in this paper. In general, for C6.1, significant improvements are observed in the Middle East and Central Asia but not in Northwestern China, as expected. This study gives a comprehensive evaluation of the performance of the C6.1 DB products in Northwestern China. As a result, this study may be beneficial to further improvements in the DB retrieval algorithm.