4.1. Similarities and Differences with Other Validation Results
It is critical to assess the reliability of SM products before using them. By comparing multiple sources of SM datasets, we can obtain knowledge of the strengths and weaknesses of different products. There are two ways to evaluate remotely sensed data. One is to compare with the in-situ measurements [
20], and the other is to compare with the modeling data [
36]. Compared with other validation results, there are some similarities and differences. The SMAP product validation was based on a set of core validation sites (CVS) [
28]. The CVSs provided high-quality in-situ SM measurements and used an up-scaled method to acquire quasi-spatial SM reference data through observations at multiple locations. The validation obtained from the CVSs was better than that obtained through other conventional SM networks. The ubRMSD of most CVSs was less than 0.04 m
3/m
3, which met the target of the SMAP mission [
28]. According to previous studies, the SMAP generally captures the dynamic range of SM better than other satellite products in terms of having a higher R and a lower ubRMSD [
17,
19,
37,
38]. The ESACCI, which is a combined product merging multiple remotely sensed data sources, except the SMAP, has an advantage of long-term climate data records and increase the sampling time intervals. Moreover, the combined ESACCI may perform better than individual products in some regions [
18,
30]. Although the statistical results of individual site were inconsistent, The results indicate that the SMAP outperforms the ESACCI with higher temporal R (SMAP:0.65–0.87; ESACCI:0.43–0.74) and lower ubRMSD (SMAP:0.041–0.074m
3/ m
3; ESACCI:0.032–0.09 m
3/ m
3) with respect to in-situ measurements. Therefore, the next generation of ESACCI may consider including the SMAP in the synthetizes. Model simulation is used as a benchmark for remotely sensed data, especially in some regions, such as the US or Europe; the accuracy of precipitation data is high, and model-based SM can capture the temporal change well. However, in some areas where the accuracy of meteorological data cannot be guaranteed, satellite products are proven to be better than model simulations [
36,
39]. In this study, though the precipitation data of the NLDAS-2 are similar to the site’s measurement and the model-based data and the NLDAS captured the seasonal and anomaly time series well, the SMAP also had a high R value compared with the in-situ data in terms of seasonality and anomaly. This indicates that remotely sensed SM data can reflect the change of SM due to short-term precipitation events. In addition, the SMAP captures the drying process better than the model-based data and has less difference compared with the in-situ data. All of these show the promising potential of remotely sensed data for drought monitoring and rainfall estimation [
40,
41].
4.2. Analysis of the Possible Reasons for the Discrepancy between Different Products
Another important purpose of this study is to analyze the possible reasons for the discrepancy between the remotely sensed and modeling products. First, the accuracy of sensors used in ground measurement also affects the performance of satellite and modeling products compared with in-situ products. According to the instrument handbook [
22], the sensors were well calibrated and the uncertainty of SM measurement is within 3% of the measured values, which was far less than the difference between three products and ground-based observations In addition, various satellite products might have different infiltrate depth, depending on the soil texture and SM content [
42,
43]. For SMAP, the uncertainty caused by the depths was estimated as a range (0–5 cm) with a set of uncertainties, not a certain depth [
29]. The ESACCI is a combined product for which active and passive products have different sensing depths, but they are all defined as “surface soil moisture” and are around 5 cm [
30]. The NLDAS output 5 cm as the average SM of the first layer, which is not equal to the true 5 cm ground measured depth. The mis-match of retrieved depth were usually considered to be systematic errors, which were less connected to the temporal R and ubRMSD [
44]. For ESACCI, satellite products to be combined were firstly scaled against GLDAS Noah (Global version of NLDAS) to harmonies their climatology [
31]. Therefore, ESACCI and NLDAS estimates showed a similar wet bias during the drying period based on our results. We compared the temporal mean of 5 cm and 10 cm in-situ SM measurements as a reference. Indeed, the mean SM of 10 cm was wetter than the 5 cm data, but it was still drier than the ESACCI and NLDAS data. Besides the depth, there are other reasons for this deviation, which need further study.
According to our results, the spatial bias between the ground reference data and the three products is correlated with the SM values, especially for the modeling data. The uncertainty of precipitation data may account for that. After a rainfall event, the surface soil becomes wetter and the variation of spatial distribution increases, which is shown in
Figure 5c. Data from all three products are lower than the in-situ data. With the soil drying, the remotely sensed data decrease accordingly; moreover, the modeling data show a slow drying rate, meaning the data will be wetter than the in-situ data. Thus, the bias between modeling data and in-situ observations is strongly dependent on the changing of the SM, whereas the remotely sensed data show less correlation with the reference SM values.
Another key issue regarding differences between the SM products is the mismatch of vegetation data between remotely sensed and modeling products, which also reflect whether the site is representative. As shown in
Table 1, most sites can stand for the vegetation type of their pixel, except for Anthony, Medford. The vegetation around the site were often mixed with pasture and cultivated crop, whereas the grid cells were more dominated by one type. For example, the Lamont site is covered by wheat crop and pasture; however, the corresponding footprint pixel were most covered by the cultivated crop. Which would If alleviate the disparity between different scales. Our results indicate that systematic differences exist in remotely sensed or modeled products and ground measurements, even though the temporal dynamics are every similar. After conducting a moving mean calculation of 35 days, we found that the systemic bias mainly lay in the seasonal part. The anomaly ubRMSD decreased, whereas the anomaly R did not have a significant change.
There is further potential for improvement in SMAP SM retrievals. The improvements include use of better ancillary data (optimized vegetation water content (VWC] and better soil texture data). The vegetation canopy exerts significant effects on the soil-emitted energy [
45]. Vegetation not only attenuates signals from soil surfaces but also emits radiation itself, leading to a reduced sensitivity of brightness temperature to SM. Accordingly, the influence of vegetation must be corrected accurately before achieving reliable SM estimations. Commonly, the effects of vegetation are mainly represented by the vegetation optical depth (VOD), which characterizes the radiation attenuation caused by vegetation The VOD of SMAP is estimated from the VWC, which is calculated by using a 10-year MODIS NDVI climatology data at 1-km spatial resolution. An empirical polynomial is established to calculate VWC from NDVI. Seasonal biases of remotely sensed or modeled SM products showed that most in-situ data that include managed agriculture exhibit significant time-dependent seasonal bias. According to a previous study, the performance of satellite products is worse for sites that are dominated by cultivated crop [
28]. In general, the main vegetation type of the SGP region is cultivated crop and grassland/pasture; one site is covered by forest (Waukomis). We aggregated a 30 m land-cover map to the 9 km scale pixel and counted the crop, grassland and forest class of grid cells; we found that the main vegetation type of the footprint pixel was nearly consistent with the corresponding site. As shown in
Figure 7, climatology vegetation optical depth (VOD) cannot indicate the discrepancy between the crop and grassland in terms of the growing period. The time variation of NDVI for winter wheat differed markedly from that of natural grassland, especially in summer. The winter wheat usually matured in late April [
24]. After harvesting, the ground would be uncovered for a period of time, whereas the nature pasture continued growing. As shown in
Figure 8, t crops the NDVI reached a maximum of about 0.6 in terms of NDVI. After harvest, the NDVI decreased sharply and maintained a relatively low value in the summer. In contrast, the NDVI of natural grassland increased consistently after entering the growth of vegetation and held a high value of about 0.7 in the summer. It was noted that the NDVI of pasture might vary irregularly due to cattle grazing or regular mowing. Besides the water stored in vegetation foliage, the intercepted precipitation or dew also had an effect on microwave radiation. Whether the free water in the canopy affected the microwave emission depended on the type and physical structure of vegetation [
46], which would need to be considered in future research.