An Examination of the SMAP Operational Soil Moisture Products Accuracy at the Tibetan Plateau

Abstract

Surface soil moisture (SSM) plays an essential role in the Earth’s water cycle and land surface processes as well as in vegetative growth, ecological health, and ecosystem properties. Particularly, information on this parameter’s spatiotemporal variability at the Tibetan Plateau is of key importance to the study of climate and the impact of climate change due to it is distinctive characteristics in this area. The present study assesses the operational SSM products provided by the SMAP (Soil Moisture Active and Passive) satellite at the Tibetan Plateau, Naqu observational station, China. In particular, the globally distributed Level 3 operational products, SPL3SMP_36km and the Enhanced Passive SSM Product SPL3SMP_9km, are evaluated in two-phases. SSM and the surface temperature estimates by SPL3SMP_36km and SPL3SMP_9km are compared against corresponding ground data available at the Naqu observation network. All in all, the examined products captured the SSM dynamics in the studied area. The results showed that precipitation is the key driving source of SSM variability. SSM fluctuated significantly and was dependent on precipitation in the studied region. Statistical metrics, such as the root mean square error (RMSE), varied for SPL3SMP_36km and SPL3SMP_9km in the ranges of 0.036–0.083 m³/m³ and 0.074–0.097 m³/m³, respectively. The unbiased RMSE (ubRMSE) was higher than the SMAP uncertainty limit (0.04 m³/m³) in most cases. This study establishes some of the causes for the different performances of SMAP products, mainly, the ancillary input dataset parameterizations, and, specifically, the surface temperature parameterization schemes of SMAP retrieval algorithm is analyzed and discussed. Our research findings highlight, among others, the usefulness of those SSM products from SMAP, particularly in mesoscale studies, providing additional useful insights into the use of those products in practice in China and globally.

1. Introduction

Soil moisture is a very important parameter of the Earth’s water cycle, making a significant contribution to the change of weather and the climate in both regional and global geographical scales [1,2]. It also exerts major control upon heat and water exchange between the atmosphere and the land components of the Earth’s system. In particular, surface soil moisture (SSM) is a key variable on the Earth’s surface that influences the interactions between the surface and the atmosphere [2,3]. It has been recognized as playing a substantial role in seasonal climate and in evaluating impacts of global climate change [4], forecasting extreme events, such as drought, frost, or floods [5]. Furthermore, it has been utilized as a key variable in various models, such as hydrological [6,7], land surface evaporation [8,9], and surface runoff [10].

Traditionally, ground instrumentation is deployed in measuring SSM at point and local scales [11,12]. Yet, it is a challenging task to characterize SSM variability in a large scale using in situ data. This is due to the large variability of SSM in both the spatial and temporal domains and its dependence on several parameters, such as rainfall, soil texture, topography, and vegetation cover, some of which may vary significantly spatially [13]. At present, a global dense ground observation network of SSM does not exist. However, SSM observation networks at regional levels are available, such as the International Soil Moisture Network (ISMN). This is a continuous SSM database that provides open SSM measurements obtained from smaller networks [14]. In addition, FLUXNET is currently the largest global observation network, providing systematic ground measurements of a series of micrometeorological and auxiliary parameters, including SSM [15].

During recent decades, Earth Observation (EO) technology has evolved dramatically, and sensors operating in the microwave (MW) part of the electromagnetic radiation spectrum are now placed in orbit, offering a promising potential to map SSM spatiotemporal variability. Specifically, sensors operating in the active MW region can deliver finer spatial resolution than the passive sensors. Yet, retrievals of SSM by active sensors are influenced, for example, by surface roughness, topography, and vegetation cover [16,17]. On the contrary, passive MW sensors offer higher temporal resolution than active sensors, resulting often in higher accuracy in SSM retrievals. Nonetheless, the spatial resolution of the passive sensors is frequently too coarse. Such spaceborne systems include the WindSat [18], the AMSR-E (Advanced Microwave Scanning Radiometer-Earth Observing System) [19], Aquarius [20], Fengyun-3 (FY3) [21], SMOS (Soil Moisture and Ocean Salinity) [22,23], and ASCAT (Advanced Scatterometer) [24]. In addition, SMAP (named for Soil Moisture Active and Passive) has been recently launched, specifically for soil moisture retrieval with a spatial resolution of ~36 km² and a 2-day revisit period. A recent review of EO-based SSM operational products is available in [25].

To address the necessity of retrieving SSM at both high resolution and accuracy utilizing EO sensors, NASA launched the SMAP satellite mission. SMAP was placed in orbit in January 2015 and started collecting data in April 2015 with a 6 am descending and 6 pm ascending orbit. SMAP has two onboard instruments: an L-band radar (1.26 GHz) and an L-band radiometer (1.41 GHz). SMAP allows one to obtain estimates of brightness temperature at 36 km resolution and backscatter at 3 km spatial resolution. L-band (1–2 GHz) was chosen since this is generally regarded as the optimal choice for monitoring topsoil moisture, as it can easily penetrate low to moderate vegetation cover and is also affected by atmospheric rather than weather conditions [26,27]. The SMAP products of SSM include active, passive, and active–passive products [28]. In 2015, an enhanced SSM product also became publicly available from SMAP [29]. Three months after the SMAP launch, its radar instrument stopped collecting data due to instrument error. Nevertheless, several studies have demonstrated that to obtain SSM globally and on a long-term basis utilizing EO data, one of the most promising avenues is that of combining passive and active MW sensing. For example, Ref. [30] proposed a method to retrieve SSM on the basis of SMAP L-band brightness temperatures and Copernicus Sentinel-1 C-band backscatter. An assessment of SSM operational products accuracy is necessary, as it provides a quality check of the product of crucial importance, so that prospective users can better facilitate their applications [31]. It also provides valuable feedback to enhance operational retrieval algorithms.

Climate change has influenced the water cycle over the globe. The response of surface soil moisture to precipitation is a major element of the water cycle in the context of global temperature increase, which manifests in progressively unpredictable rainfall events. The response of SSM to precipitation in arid and semi-arid areas is an interesting focus of studies worldwide [32]. Understanding the connection between SSM and precipitation, especially in regions such as the Tibetan Plateau, is critical to the effective use of the limited water resources available for the region. The Tibetan Plateau is located in the southeast part of China. The plateau covers approximately 2.6 million km² at elevations more than 4000 m above sea level. It is also called “the Third Pole” and the “Roof of the World”. It is of key significance to Asia and the global climate, due to it is exceptional characteristics and because it contains the drainage basins of the three longest rivers in Asia (Yellow, Mekong, and Yangtze rivers), which are crucial for agriculture and urban development. Thus, information on SSM variability over the Tibetan Plateau is of very high importance for studies linked to climate change and its impacts [33,34]. In addition, the high SSM dynamic range due to the low biomass covering the region, the low air, and its typical freeze–thaw cycle further support the selection of this area as an ideal place for validating satellite-based SSM products.

Previous studies revealed that SMAP has the lowest uncertainty amongst other soil moisture products, particularly in the Tibetan Plateau area [35,36]. Thus, the present study aims to offer an in-depth insight into the series of the SMAP’s SSM operational product’s accuracy, as such information may significantly assist in additional improvement of the product’s algorithm accuracy, especially in polar and alpine climates where there is ice reflection. The assessment of the SMAP-derived SSM retrievals values is an attempt to build an improved soil moisture estimation and a consistent soil moisture dataset at the regional level. In this research, the operational SSM products provided by SMAP (Soil Moisture Active and Passive) are assessed through a two-phase approach. SSM and the surface temperature (ST) estimates by SPL3SMP_36km and SPL3SMP_9km are compared against corresponding ground data available at the Naqu observation network.

2. Experimental Set-Up

2.1. Validation Sites

Naqu is situated in the central Tibetan Plateau around Naqu city. The Naqu network, (Figure 1) is a dense network at a high altitude (average elevation of about 4650 m asl) [37]. The terrain is hilly and annual precipitation is approximately 500 mm, approximately 75% of which is received in the period between May and October. The land cover is dominated by alpine grasslands (Figure 1).

Because of its topography, low biomass, and large, dynamic seasonal range as well as the freeze–thaw cycle, the Naqu network is considered an ideal place for the evaluation of satellite SSM retrievals and simulated SSM product validation. A total of 56 stations (SMTMN) are installed in the Naqu network, measuring SSM and temperature. These stations are deployed at three spatial scales (1.0°, 0.3°, and 0.1° ), namely large-scale networks (38 stations), medium-scale networks (22 stations), and small-scale networks (9 stations). At each station, one soil moisture and soil temperature sensor have been obliquely inserted into 0–5 cm depth, comparable to the 134 penetration by the SMAP L-band passive microwave sensor. In addition, three other sensors are inserted horizontally at depths of 10 cm, 20 cm, and 40 cm. The sample intervals are 30 min and daily time resolution. At this multi-scale station, more in situ measurement points within the 36 km/or 9 km SMAP radiometer footprint can be found. Due to its remote location, it is also less affected by radio frequency interference (RFI). In addition, it has low air density and biomass and low SSM, which exert less impact in attenuating L-band passive microwave signals in retrieving SSM. This network data has been widely used in numerous validation and scaling studies of microwave soil moisture [38,39,40,41,42,43]. This suggests that data from the Naqu network in the Central Tibetan Plateau are of high consistency

In addition, in the present study, rainfall data from the Tropical Rainfall Measuring Mission is used as complementary data to assess the product’s verification since the study area being characterized by the long rainy season may give valuable insight into further algorithm improvement. In the present study, sixteen months of SMAP SM operational data were used, compared with shorter periods in previous studies [44,45].

2.2. Satellite Data

2.2.1. SMAP SSM Product (SPL3SMP)

NSIDC provides SMAP radiometer and SSM products publicly at no cost (available from https://nsidc.org/data/smap/smap-data.html (accessed on 22 March 2019). The SMAP radiometer Level-3 SSM product has a spatial resolution of ~40 km. It is displayed at 36 km Equal-Area Scalable Earth-Grid (EASE2) [46]. Herein, the SMAP Level-3 (L3) radiometer SSM product (SPL3SMP)(version 5) and SMAP-Enhanced L3 SSM product (SPL3SMP_E) (version 2) from 1 April 2015, to 18 September 2016, has been used, available at 36 km spatial resolution [47]. A newer version, SSM version 6, which was distributed in August 2019 was not available for use at the time of the study [29]. The term SPL3SMP_36 km will be used to refer to this product at this scale.

2.2.2. SMAP-Enhanced Level-3 SSM Product

This enhanced L3 is a daily composite of SMAP Level-2 (L2) SSM half-orbit products. Enhanced radiometer SSM product (SPL3SMP_E_9km) distinguishes itself in that the SSM retrievals are evaluated on a 9 km grid. This process was performed by application of the Backus–Gilbert interpolation technique. In the present study, the SMAP-enhanced L3 SSM passive product (SPL3SMP_E_9, version 3), displayed on 9 km on global cylindrical Equal-Area Scalable Earth (EASE-Grid) version 2.0, was used. For more details, the reader can refer to [29]. SMAP-Enhanced L3 Radiometer SSM product (SPL3SMP_E) (version 3) used herein is the latest version (available via https://nsidc.org/data/SPL3SMP_E/versions/2 (accessed on 22 March 2019). A total of 56 sites from the Naqu network were used to evaluate both SMAP L3 Radiometer SSM products from 1 April 2015, to 18 September 2016.

3. Methods

3.1. Data Pre-Processing

SMAP radiometer has a 9 km/36 km spatial resolution footprint. In general, the characterization of SSM spatiotemporal variability using point-based in situ measurements is challenging, as they do not align perfectly with the SMAP footprints [48]. The spatial resolution differences may introduce uncertainties since it is impossible to represent the satellite observation’s exact scale by ground observation. Herein, to address this spatial representation issue, ground-based observations have been aggregated at the 9 km and 36 km grid scale. This is an effective approach also widely used in analogous validation experiments [34,48,49]. Subsequently, station data within the same pixel were averaged. Spatial averaging has been shown to be a simple and effective way to minimize spatial noise; hence, it produces more reliable comparisons. In our study, to perform the comparisons of SSM retrieved from EO instruments versus in situ observations, only the ground measurements matching the next overpass of the SMAP satellite were selected. The exact ground measurement values collocated to the satellite-based SSM were extracted from SMAP for each site on the basis of each site’s coordinates.

In addition, quality control was implemented on the ground data to minimize any errors in ground observations. As such, data with uncertainties, such as SSM values higher than 1 m³/m³ or less than 0 m³/m³ and positive or negative anomalies in SSM, were excluded. In addition, from any further analysis were excluded stations with missing data, i.e., 30% of a full year of data. Upon completion of this step, all sites and SMAP SSM in the Naqu network in the Central Tibetan Plateau area were averaged at each satellite overpass time. In any further analysis, only topsoil moisture measurements (at 5 cm) were used because the SMAP SSM product-sensing depth is ~5 cm of the topsoil [13,27].

3.2. Statistical Assessment

Ground measurements were compared with the satellite retrievals. The averaging technique described previously was performed to investigate the statistical agreement for each threshold. The analysis was performed independently for each scenario. Comparisons of the in situ SSM (0–5 cm) and the collocated satellite retrievals were performed in all cases. A series of statistical performance scores were adopted herein to evaluate the agreement among the datasets. These included the root mean square error (RMSE), Pearson’s correlation coefficient (RP), Spearman’s rank correlation coefficient (RS), the mean error or bias, and the standard deviation (unbiased RMSE), expressed as follows:

$$\mathrm{R}=\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}({\mathrm{SSMest}}_{\mathrm{i}}-{\overline{\mathrm{SSM}}}_{\mathrm{est}})\left({\mathrm{SSM}}_{\mathrm{ref}\left(\mathrm{i}\right)}-{\overline{\mathrm{SSM}}}_{\mathrm{ref}}\right)}{\sqrt{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}({\mathrm{SSMest}}_{\mathrm{i}}-{\overline{\mathrm{SSM}}}_{\mathrm{est}})2{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{SSM}}_{\mathrm{ref}\left(\mathrm{i}\right)}-{\overline{\mathrm{SSM}}}_{\mathrm{ref}}\right)}^2}}$$

(1)

$$\mathrm{Bias}=\overline{\left({\mathrm{SSM}}_{\mathrm{est}}-{\mathrm{SSM}}_{\mathrm{ref}}\right)}$$

(2)

$$\mathrm{RMSE}=\sqrt{\overline{({\mathrm{SSM}}_{\mathrm{est}}-{\mathrm{SSM}}_{\mathrm{ref}})²}}$$

(3)

$$\mathrm{ubRMSE}=\sqrt{{\mathrm{RMSE}}^2-\mathrm{Bias}²}$$

(4)

Subscripts i = 1 … N denotes the individual observations, “est” denotes the predicted values, and ref denotes the reference values of the observations. The horizontal bar denotes the mean value. Notably, these statistical parameters have been used in similar verification studies implemented in SSM operational products [7,27,48,50].

4. Results

This section presents the key research findings from the statistical comparisons performed between the SMAP-derived SSM products (namely the SPL3SMP_36km and SPL3SMP_9km products) versus the collocated ground measurements. A total of 56 stations from the Naqu network were used for this purpose. First, the temporal behavior of the SSM in situ and SSM satellite measurements for each grid of the two datasets were analyzed. TRMM satellite-derived precipitation data [51] are included in each time series to assist in evaluating the ability of SMAP SSM products to capture the SSM dynamics. In addition, error metrics used for quantifying the accuracy of both SPL3SMP were computed for each grid, and the results are presented in Table 1, Table 2 and Table 3. To better illustrate the SCA-V satellite algorithm’s abilities, the scatterplots of the observed and predicted SSM are also included herein (shown in Figures 3 and 6).

4.1. SMAP Level-3 Radiometer SSM Product (SPL3SM_36km)

Seasonal trends illustrated in Figure 2 show satisfactory agreement with SSM products and capture the SSM trends. The temporal variations over the investigated period as well as the rapid reaction of the SSM to weather changes are captured by in situ SSM and the SPL3SM_36km product (Figure 2).

These variations indicate that the satellite SSM estimates are highly responsive to precipitation events [48]. In situ SSM shows recognizable seasonal variations ranging between the values of 0.05 and 0.5. Frozen soil at the surface (depth 0~5 cm) from early November to May, leads to a sharp decrease in the SSM from more than 0.2 to less than 0.1. At the beginning of May, when the frost starts to melt, the SSM rapidly increases. The SPL3SM_36km product performance generally underestimates the ground measurements during frozen periods and slightly overestimates during unfrozen periods, as can be observed in Figure 2 and Table 1. A possible reason for this is the frozen surface, where the satellite performance is relatively low. In general, the grids show a satisfactory agreement with the seasonal trends. However, the SPL3SM_36km product and the in situ measurements possess similar temporal evaluations. The shape coefficients (slope and intercept) for SPL3SM_36km exhibited perfect confidence intervals (at 95% confidence), as shown in Figure 3.

Table 1 lists the error metrics results of the SPL3SM_36km product over the entire Naqu network in the Central Tibetan Plateau, summarizing the performances of SPL3SM_36km at different grids. As can be observed, the passive SPL3SM products reproduce satisfactory SSM temporal patterns and their dynamic range is also comparable to the ground measurements. However, the SPL3SM_36km product shows signs of bias in its performance. Despite the bias present, the SPL3SM_36km product shows good RMSE over most of the region (ranging between 0.036 and below 0.083 m³/m³). This is within the acceptable margin for RMSE for SSM estimations (i.e., below 0.1 m³/m³). Grid 2142 exhibits the best performance. The bias of this grid is 0.024 m³/m³, RMSE is 0.036 m³/m³, and the ubRMSE is 0.028 m³/m³, smaller than the SMAP mission accuracy target (0.04 m³/m³) [27]. Grid_2230 and Grid_2232 have the highest RMSE 0.083 and 0.113 m³/m³ due to the high negative bias of −0.067 and −0.106 m³/m³, respectively. However, notably, the SPL3SM_36km SSM product sufficiently captures the various dynamics of SSM with time, as shown by Spearman’s correlation coefficient RS in the range of (0.88–0.93) and Pearson Correlation coefficient RP in the range of (0.86–0.93). Since both correlation coefficients are high, the linear and monotonic agreement between the products and in situ measurements is high. This suggests strongly that the bias contributes heavily to the errors pointing to a systematic error.

Table 1. Satellite (SPL3SMP_36km) and ground-measured SSM comparisons at the validation sites on the Tibetan Plateau, including all the datasets from 1 April 2015 to September 2016. Units are in m³/m³.

Grid	Stations	Longitude	Latitude	Bias	RMSE	ubRMSE	MAE	RP	RS	ErrMin	ErrMax	Sat.Obser
2139	7	91.6805	31.9559	−0.021	0.075	0.072	0.062	0.86	0.88	−0.156	0.166	259
2140	20	91.6805	31.62478	−0.020	0.065	0.062	0.055	0.89	0.90	−0.146	0.180	218
2230	3	92.42738	31.62478	−0.064	0.083	0.053	0.072	0.91	0.93	−0.205	0.066	166
2141	4	91.6805	31.29487	0.047	0.069	0.051	0.055	0.90	0.91	−0.076	0.226	231
2142	3	91.6805	30.96609	0.024	0.036	0.028	0.029	0.93	0.93	−0.068	0.106	238
2232	3	92.42738	30.96609	−0.106	0.113	0.041	0.106	0.90	0.90	−0.180	0.009	168

Table 1. Satellite (SPL3SMP_36km) and ground-measured SSM comparisons at the validation sites on the Tibetan Plateau, including all the datasets from 1 April 2015 to September 2016. Units are in m³/m³.

Grid	Stations	Longitude	Latitude	Bias	RMSE	ubRMSE	MAE	RP	RS	ErrMin	ErrMax	Sat.Obser
2139	7	91.6805	31.9559	−0.021	0.075	0.072	0.062	0.86	0.88	−0.156	0.166	259
2140	20	91.6805	31.62478	−0.020	0.065	0.062	0.055	0.89	0.90	−0.146	0.180	218
2230	3	92.42738	31.62478	−0.064	0.083	0.053	0.072	0.91	0.93	−0.205	0.066	166
2141	4	91.6805	31.29487	0.047	0.069	0.051	0.055	0.90	0.91	−0.076	0.226	231
2142	3	91.6805	30.96609	0.024	0.036	0.028	0.029	0.93	0.93	−0.068	0.106	238
2232	3	92.42738	30.96609	−0.106	0.113	0.041	0.106	0.90	0.90	−0.180	0.009	168

Figure 4 shows the comparison between the SMAP SPL3SMP_36km SSM products against the ground observations for each grid. It is evident from Figure 4 that surface temperature (ST) auxiliary data shows a nearly perfect consistency with the ground measurements in terms of temporal variability. The validation measures of the ST auxiliary data are displayed in Table 2 and Figure 4. The overall RMSE of the satellite ST measurements have a range of 2.8 to 4.6 K. This range is higher than 2 K, which is still within the acceptable error range for SMAP ST products [52].

Table 2. Comparison between Satellite (SPL3SMP_36km) and observed surface temperature at the validation sites in Naqu based on the Tibetan Plateau, including all the datasets from 1 April 2015 to September 2016. Error metrics of different grids are listed below; units are in (K).

Index Point	Stations	Line	Column	Longitude	Latitude	Bias	RMSE	ubRMSE	RP
2139	7	24	48	91.6805	31.95584	1.9	3.4	2.8	93
2140	20	25	48	91.6805	31.62478	2.7	4.1	3.1	92
2230	3	25	50	92.42738	31.62478	1.7	3.8	3.4	91
2141	3	26	48	91.6805	31.29487	3.0	4.6	3.5	89
2142	3	27	48	91.6805	30.96609	0.2	2.8	2.8	94
2232	2	27	50	92.42738	30.96609	1.8	3.2	2.7	92

Table 2. Comparison between Satellite (SPL3SMP_36km) and observed surface temperature at the validation sites in Naqu based on the Tibetan Plateau, including all the datasets from 1 April 2015 to September 2016. Error metrics of different grids are listed below; units are in (K).

Index Point	Stations	Line	Column	Longitude	Latitude	Bias	RMSE	ubRMSE	RP
2139	7	24	48	91.6805	31.95584	1.9	3.4	2.8	93
2140	20	25	48	91.6805	31.62478	2.7	4.1	3.1	92
2230	3	25	50	92.42738	31.62478	1.7	3.8	3.4	91
2141	3	26	48	91.6805	31.29487	3.0	4.6	3.5	89
2142	3	27	48	91.6805	30.96609	0.2	2.8	2.8	94
2232	2	27	50	92.42738	30.96609	1.8	3.2	2.7	92

4.2. SMAP-Enhanced Level-3 Radiometer SSM Product (SPL3SM_9km)

Table 3 and Figure 5 and Figure 6 illustrate the temporal trends and correlations between in situ and SPL3SMP_9km for the entire period of the study. It can be seen in Figure 5 that the temporal evaluation of the in situ observations and the SPL3SMP_9km also show significant variations. The temporal evaluation of the ground measurements and SPL3SMP_9km product show a satisfactory seasonal agreement and similar trends in variability over time. Regarding the precipitation data, both ground observations and SPL3SMP_9km respond to the precipitation events and SSM variation trends (Figure 4). It should be noted that station observations at different grids show only small differences. Still, the SPL3SMP_9km product shows more significant differences, indicating a high sensitivity to rainfall events. In most cases, it also shows overestimation against the ground measurements (refer to Figure 5 and Figure 6). This may be associated with the fact that SSM rises rapidly between precipitation events.

As can be observed, the SPL3SMP_9km product underestimates the in situ observations from November to May, and it is relatively very consistent with the trends shown by the in situ from May to September. The SPL3SM_9km product showed high sensitivity to rainfall events, and it overestimates SSM in situ from May to September with precipitation events. It underestimates SSM generally during the frozen period from November to April (Figure 5). In contrast, different trends were observed in 2015 in comparison with 2016. The observed trends show various fluctuations in the May–September period due to the variation of the precipitation events between the two years, as shown in Figure 5a–d.

Table 3 and Figure 6 illustrate the comparison between the SPL3SMP_9km SSM estimates and the different grids’ corresponding ground measurements. As shown in Table 3, the performance of SPL3SMP_9km over different grids of the RMSE (ranging between 0.078 and 0.089 m³/m³) suggests satisfactory accuracy. The correlation coefficient is generally very high (above 80%) in most cases. Notably, the Pearson Correlation (RP) at a 95% confidence level is also high (above 80%) (Figure 5). The SPL3SMP_9km product and the in situ measurements are essentially correlated for all the grids. Compared with the entire range of the observations, the bias of the SPL3SM_9km measurements is generally negative except for Grid 34104 (the bias is 0.006). On the basis of the low ubRMSE and the high correlation coefficients, the results strongly indicate that the SCA-V algorithm returns a high correlation between the product and the ground observations.

Table 3. Comparisons and validation measures for Satellite (SPL3SMP_9km) and observed SSM in the Naqu network, including all the datasets from 1 April 2015 to September 2016. Units are in m³/m³.

Index Point	Stations	Line	Column	Longitude	Latitude	Bias	RMSE	ubRMSE	RP	RS	Sat.Obser
34104	2	94	191	91.72718	31.9144	0.006	0.081	0.081	0.82	0.82	276
34105	2	95	191	91.72718	31.83156	−0.048	0.078	0.062	0.90	0.91	280
34284	2	95	192	91.82054	31.83156	−0.032	0.082	0.076	0.83	0.85	273
34106	3	96	191	91.72718	31.74879	−0.037	0.089	0.081	0.84	0.88	285
34285	4	96	192	91.82054	31.74879	−0.068	0.097	0.069	0.86	0.89	277
34107	2	97	191	91.72718	31.6661	−0.021	0.078	0.075	0.84	0.85	289
34286	6	97	192	91.82054	31.6661	−0.023	0.074	0.070	0.87	0.90	281

Table 3. Comparisons and validation measures for Satellite (SPL3SMP_9km) and observed SSM in the Naqu network, including all the datasets from 1 April 2015 to September 2016. Units are in m³/m³.

Index Point	Stations	Line	Column	Longitude	Latitude	Bias	RMSE	ubRMSE	RP	RS	Sat.Obser
34104	2	94	191	91.72718	31.9144	0.006	0.081	0.081	0.82	0.82	276
34105	2	95	191	91.72718	31.83156	−0.048	0.078	0.062	0.90	0.91	280
34284	2	95	192	91.82054	31.83156	−0.032	0.082	0.076	0.83	0.85	273
34106	3	96	191	91.72718	31.74879	−0.037	0.089	0.081	0.84	0.88	285
34285	4	96	192	91.82054	31.74879	−0.068	0.097	0.069	0.86	0.89	277
34107	2	97	191	91.72718	31.6661	−0.021	0.078	0.075	0.84	0.85	289
34286	6	97	192	91.82054	31.6661	−0.023	0.074	0.070	0.87	0.90	281

5. Discussion

The use of remotely-sensed SSM data is crucial for assimilation, assessment forecast, and for improving the simulation skills of models to better understand land–atmosphere exchanges in regions where observation data are sparse. Comparisons of newly released SMAP SM operational data with available in situ observations are fundamental to estimating the strengths and limitations of operational products in such regions. The current study used collocated in situ records to evaluate the accuracy of two SMAP microwave SSM operational products available globally at different scales (9 km and 36 km) across a diverse terrain in a range of seasons.

The in situ measurements may be subject to differences in the spatial resolution scale between in situ points and satellite pixels. An effort was made to diminish their effects. Averaging of a few point-based in situ measurements (in most cases two–four) may fully represent the true SSM values at a scale of 36 km. As such, the differences in spatial resolution could be reduced. However, in this study, the ground station measurements were assumed to be the true values of the 9 km and 36 km footprints [48,53].

A mismatch between the satellite sensor and the measuring depth of the in situ instrument and the discrepancy between the in situ SSM measuring depth and the satellite depth of microwave penetration could explain, in part, the bias displayed in some of the grids. The sensing depth of the L-band is shallow compared with the in situ instruments [7,54]. The ground-based measurements used to validate the products are placed at a depth of approximately 5 cm and usually observe SSM at a depth of 3 to 7 cm. The effective sampling depth for SMAP’s L-band is 0–3 cm [55], and it varies with the SSM [56]. Recent studies indicate that the quality of satellite-based SSM products can be improved using upscaling and triple collocation methods [36].

Other reasons for the dry bias could be errors caused by the sensor’s measurement accuracy at the point-based in situ observations. It is expected that the soil surface will often be slightly drier than the measured layer [57]. However, this bias underestimation is often reduced by inserting the sensor at a depth of 0 to 5 cm. The bias between the SMAP SSM values and the in situ measurements is expected to be negative, while the statistical results are positive [58]. It is suggested that the negative bias is attributed mainly to the error of estimating the true SSM value across a 36 km grid.

A possible cause for the positive bias reported by this study’s results could be uncertainties resulting from ancillary data parameterizations [7,59]. In the SCA-V, brightness temperature (TB) is converted to emissivity by correcting for the effective soil temperature effect, vegetation, and surface roughness. To determine the soil dielectric constant from the emissivity of the soil, the Fresnel equation is used [60]. Subsequently, a dielectric mixing model is used to estimate SSM, given that the soil texture is known. In this process, the ancillary data, such as vegetation parameters, surface roughness, effective soil temperature, and texture, must be known beforehand to effectively apply the retrieval algorithms. Regardless of the limitations introduced by simplifying the retrieval algorithms, uncertainties in ancillary parameters introduce significant errors.

In the SMAP radiometer SSM algorithm, the GEOS-5 model [29] is used to retrieve the required ST. The validation measures of the ST auxiliary data are displayed in Table 3. The overall RMSE of the satellite ST measurements have a range of 2.8 to 4.6 K. This range is higher than the acceptable error range of 2 K for SMAP ST products [52], demonstrating that the ST auxiliary data overestimates the effective surface temperature data. Therefore, the ST inputs could be included as a source of errors for the SSM retrieval in the Naqu SMTM networks, which may also be associated with the error of estimating the true SSM value across a 36 km grid.

In this study, the products’ data version that was used consists of newly released effective ST data. It consists of an improved depth correction scheme, which is a crucial parameter in passive SSM retrieval. Accordingly, the resulting inconsistency can give a dry bias (satellite product lower than in situ SSM) of the retrieved SSM if the modeled effective ST is lower than the soil temperature detected by the radiometer. Similarly, a wet bias of the SMAP-derived SSM will occur if the model-based effective soil ST is higher than the ST measured by the radiometer.

Therefore, the case can be made that the divergence of the SMAP SSM from the in situ measurements may be affected by the underestimation or overestimation of effective soil temperature over the Tibetan Plateau. This may be caused by overlaying vegetation mismatch, soil texture, as well as high soil organic carbon contents (SOC) due to the slow decay rate in this cold, arid zone of the Tibetan Plateau. This may influence thermal/hydraulic as well as soil dielectric properties [34]. Thus, they exert less impact on effective soil surface temperature retrieval.

In terms of correlation coefficients, both SMAP products correlated well with the in situ measurements, with the coefficients varying between (0.86 and 0.93) and (0.82 and 0.90) for SPL3SMP_36km and SPL3SMP_9km, respectively. In validation studies, both RMSE and correlation are important, and both should be studied to provide an adequate assessment of the accuracy. The correlation in this study signifies the good ability of the SMAP SSM product to reliably capture and reflect the dynamics of the soil moisture, which may be very useful in mesoscale studies to assess SSM dynamics, providing a guideline concerning these products’ practical application to hydro-meteorological and hydro-climatological applications.

A new finding was observed in Figure 2a–c and Figure 5a–d. It can be seen that satellite products exhibit higher temporal variability than in situ observations. This could be due to the Naqu network station receiving rainfall mostly through the period of May to October. This rainfall accounts for 91.0% of the precipitation. Rainfall during the summer months (June to August) accounts for 60.1% of the annual precipitation, while precipitation during the winter season (November to April) accounts for just 9.0% of the annual precipitation [61]. Rainfall has a major impact on SSM. It is expected that while the surface is slightly wet, the water may stay on the topmost 0–5 cm of the soil. However, in this cold and arid zone, this increase in SM may last for long periods, as SM takes longer to evaporate or percolate. Thus, it may influence the satellite signals, as seen in Figure 2a–c and Figure 5a–d. According to [62], satellite retrievals are more sensitive to atmospheric and climate parameters (e.g., precipitation, radiation, and wind).

In summary, it is apparent that both SMAP products show a very sensitive response to precipitation events, and there is a satisfactory agreement of both the satellite-derived SSM with in situ measurements, with reported correlation coefficients greater than 80%. In contrast, the products, at times, underestimate SSM with negative bias. However, to a greater extent, the spatial averaging technique used in the study generally implies a more controlled dynamic range for SSM. This might be because the soil moisture’s actual dynamic range is largely dependent on the spatial scale under consideration. For example, [63,64,65,66] showed that SSM temporal standard deviations (e.g., 0.02 m³/m³, 0.04 m³/m³, and 0.06 m³/m³) are expected for spatial resolutions with a satellite footprint at a scale of 10–30 km.

6. Conclusions

In the present study, a thorough evaluation of two SMAP radiometer SSM higher-level products (i.e., SPL3SMP_9km and SPL3SMP_36km) in the Tibetan Plateau region was conducted. The study’s findings help to better understand the errors associated with the L-band SMAP satellite SSM product. Such verification studies assist in refining the retrieval algorithm and guiding the product user community toward better use of these data in a wide range of applications and research purposes, such as studying climate change and improving water resource management and food security. These results provide a crucial quality check of those products so that potential users can better facilitate their applications. It is, therefore, indispensable to assess the highly foreseen SMAP mission, which has been devoted to observing global SSM with unique accuracy and coverage, using the newest L-band satellite in space. To investigate the SMAP radiometer SSM product accuracy, the EO data were compared with ground observations from the Naqu-validated observational network in the central Tibetan Plateau to analyze the effect of cold, arid climatic zones on SSM retrievals. This evaluation offers valuable input to a better understanding of the product’s added value, particularly with regard to studies contributing to climate change from regional to global scales. The evaluation and inter-comparison were performed on the basis of a series of statistical metrics, including Pearson’s correlation coefficient (RP), RMSE, bias, unbiased RMSE (ubRMSE) as well as Spearman’s rank correlation coefficient (RS). The key findings from this study can be summarized as follows:

• The average ubRMSE value over the different grids ranged from 0.028 to 0.072 m³/m³ and from 0.069 to 0.081 for 9 km and 36 km, respectively, which is higher than 0.04 m³/m³ (the accuracy target of the SMAP mission). Grid_2142 of the 36 km, exhibits the best performance. The bias of this grid is 0.024 m³/m³, RMSE is 0.36, and the ubRMSE is 0.028, smaller than 0.04 m³/m³.
• SMAP radiometer SSM retrievals perform relatively well. They effectively capture the absolute SSM and accurately reflect the short-term variability in soil moisture. The values of the SMAP-derived SSM retrievals presented an overestimation on wet days, especially during precipitation events. This phenomenon often causes satellite products to exhibit higher temporal variability than ground observations.
• It has been found that the ST ranges from 2.8 to 4.6 K, which is higher than the maximum error of 2 K of the SMAP requirements. It is considered the key factor contributing to the errors of the satellite product. The ST error is considered responsible for the low accuracy of the SSM retrievals on the 9 km scale.

In future studies, to address the mismatch of the spatial scale between satellite footprint and station location as well as the vertical depth, the SMAP SSM product will be evaluated with a combination of the triple collocation method, model-based products, and dense ground validation measurements. Detailed systematic variances will be investigated. Future work can be directed towards improving the SMAP soil moisture products, soil type, and soil roughness, which will help increase their use globally in future studies.