Multi-Scale Validation and Uncertainty Analysis of GEOV3 and MuSyQ FVC Products: A Case Study of an Alpine Grassland Ecosystem

Abstract

Fractional vegetation cover (FVC) products provide essential data support for ecological environmental monitoring and simulation studies. However, the lack of validation efforts of FVC products limits their applications. Based on Sentinel-2 data and intensive multi-scale measured FVC data, the accuracies of two FVC products (GEOV3 and MuSyQ) in alpine grassland ecosystems were validated through direct validation and multi-scale validation. Based on the heterogeneity of the underlying surface (HUS) of the monitoring plots, the impact of the HUS of the monitoring plots on the product validation was analyzed. The results showed that: (1) the measured data directly validated that the GEOV3 FVC product performed better than the MuSyQ FVC product; (2) the multi-scale validation method based on high-resolution reference FVC map of Sentienl-2 satellite images validated the accuracy of these two FVC products, which was higher than the accuracy directly validated by FVC measured data, leading to overestimation of the validation results; and (3) the HUS of the monitored plots has a significant impact on the FVC product validation. By quantifying the HUS of the monitored plots and removing the heterogeneous monitoring plots, the uncertainty of the validation results can be reduced. It is necessary to consider the impact of validation methods and the HUS on the validation results in future product validation.

1. Introduction

The fractional vegetation cover (FVC) refers to the percentage of the vertical projection area of the vegetation (including branches, stems, and leaves) on the ground in the total statistical area [1,2]. It is one of the crucial parameters used to describe the biophysical quantity of vegetation and characterizes the growth of terrestrial vegetation. It has been widely used in scientific research and environmental monitoring [3,4]. Therefore, the improvement in ecological environmental monitoring and related simulation work depends greatly on the accurate acquisition of FVC data [5].

Remote sensing technology has become the main technical means of obtaining regional surface FVC information due to its comprehensive coverage and strong continuous observation ability [6,7]. A host of FVC products with various spatial and temporal resolutions have currently been produced by several research teams using long time-series remote sensing image datasets, such as the CNES/POLDER [8], LSA/SAF [9], ESA/MERIS [10], CYCLOPES [11], MuSyQ [12], GEOV1 [1,13], GEOV2 [14], GEOV3 [15], and GLASS [6,16]. Numerous disciplines have made extensive use of these FVC data products. However, due to the huge differences between the inversion algorithms’ performances, input data, inversion models, and inversion processes of the different FVC products [17], there are certain differences and uncertainties in these FVC products [5]. Therefore, authenticity validation is required to improve the utilization of these FVC products [18].

Currently, the commonly used validation methods for remote sensing products include the direct validation method based on in situ sites and the multi-scale validation method based on high-resolution data [19,20]. The in situ site-based direct validation method involves comparing the FVC product’s pixel values with field reference FVC measurements [17,21]. This method is widely used to product global FVC products and is a crucial step before distribution. Remote sensing product manufacturers provide product quality to optimize the inversion method and model from a global perspective and to deliver fundamental product quality information (i.e. the root mean square error (RMSE) of the products) to global users [12,13,16]. However, producers might merely view product validation as a pre-release step, and their products’ quality validation sites are often considered and distributed around the world. These places typically are significant geographical areas with only one type of biological community, such as areas with sizable tracts of crops, grass, shrub, various tree plantation types, etc. In general, due to the lack of intensive field measurements at the regional scale, the developer teams do not conduct in-depth assessments of the quality of remote sensing products in a specific ecosystem [17]. For instance, Ding et al. validated the GEOV1 FVC product using monitoring data from multiple Australian ground stations and found that the accuracy of the GEOV1 in this region was lower than expected [22].

The multi-scale validation method based on high-resolution data first establishes the relationship between the ground elementary sampling units (ESU) and high-resolution satellite image pixels using empirical or semi-empirical models, physical models, or machine learning models [23]. Then, the inversion of the required parameters is completed, and upscaling is conducted (usually via pixel aggregation) to maintain the spatial resolution consistent with the spatial resolution of the coarse resolution FVC product [24]. Finally, the pixel values of the two images are compared to evaluate the accuracy of the remote sensing product. This validation method is often applied in product validation activities in specific regions. For example, Mu et al. and Jia et al. improved the spatial scale of the ground ESU with the help of high-resolution remote sensing images in the Heihe River Basin, China, and they validated the accuracy of the GEOV1 and GLASS FVC products, respectively, in the Heihe farmland area. Their results showed that the GEOV1 can produce a systematic overestimation of the FVC of up to 0.2 in farmland, while the RMSE of the GLASS FVC in farmland is only 0.087, indicating a relatively high accuracy [25,26].

The heterogeneity of the underlying surface (HUS) of the validation sites has frequently been disregarded in remote sensing product validation, which introduces uncertainty to the validation [27,28,29]. For instance, in regions with significant spatial heterogeneity, the ESUs that do not match the scale of satellite remote sensing pixels often have poor spatial representation capabilities. The direct validation results of the remote sensing products would undoubtedly be questionable if the ESUs are used directly to validate coarse resolution products [30]. Additionally, the accuracy of the high-resolution reference FVC is primarily impacted by the HUS in the multi-scale validation method based on high-resolution data, and including sites with significant underlying surface heterogeneity (USH) in the remote sensing inversion process will decrease the accuracy of the inversion results. [31]. Furthermore, upscaling involves a complex spatial scale transformation process [32], and the inversion errors for heterogeneous sites will be further transmitted, resulting in unreliable validation results. Therefore, both the direct validation method based on in situ sites and the multi-scale validation method based on high-resolution data should consider the HUS of the validation sites.

The alpine grassland ecosystem is distinct from other ecosystems, such as farmland and forest ecosystems [33,34]. It is mainly located in high altitude or high latitude areas, and its environment is jointly affected by the topography, complex hydrological responses, and variable climate conditions, so it is a typical fragile ecosystem [35,36]. Due to the short growing season, low temperatures, sparse population, and inconvenient regional transportation in alpine grassland ecosystems, the collection of field data is greatly limited, resulting in a lack of work on the validation of FVC products of the alpine grassland ecosystem. Based on the above problems, in this study, first a large number of multi-scale FVC field measurements were obtained, and, then, the accuracies of the GEOV3 and MuSyQ global FVC data products were evaluated using the direct validation method and the multi-scale validation method based on high-resolution data. The uncertainty of the validation results was also analyzed. The main objectives of this study were as follows: (1) to evaluate the accuracies of the GEOV3 and MuSyQ FVC products in alpine grassland ecosystems; (2) to evaluate the impact of the multi-scale validation method based on high-resolution data on the product accuracy evaluation; and (3) to explore the influence of the HUS of the measured sites on the accuracy validation of the FVC products.

2. Materials and Methods

2.1. Study Area

The Three-River Source Region (TRSR) (31°39′–36°16′N, 89°24′–102°23′E) is located in the hinterland of the Qinghai–Tibet Plateau in western China (Figure 1). It is also known as the Water Tower of China. The TRSR is the source area of the Yangtze, Yellow, and Lantsang rivers [37]. The average altitude of the study area is 4000 m, and it has a complex hydrological environment and rugged terrain characteristics. Alpine grassland (including alpine steppe and alpine meadow) is the main vegetation type, covering most of the study area.

2.2. Data Source and Pre-Processing

2.2.1. Field Data Based on UAV Imagery

We delineated 422 remote sensing monitoring plots in the TRSR from July to August 2019, and each plot was 250 m × 250 m in size. In each remote sensing monitoring plot, we evenly delineated 16 small 30 m × 30 m plots, which were used as the ESU in this study. At each remote sensing monitoring site, we acquired high-resolution aerial images through aerial photography using unmanned aerial vehicles (UAVs). The UAV used in this study was the DJI Phantom 4 Professional Edition, which uses the global positioning system/global navigation satellite system (GPS/GLONASS) dual-satellite positioning module and has a very high positioning accuracy (±1.5 m in the horizontal direction and ±0.5 m in the vertical direction) [38], so it can fly precisely and hover smoothly [39]. It also carries a three-axis gimbal and a 20-megapixel light camera with a 1-inch CMOS sensor, which can capture images of red, green, and blue band information [38]. When the UAV is flying, the three-axis gimbal keeps the camera lens pointing straight down at all times. During the field investigation, we set the route using a fragmentation monitoring and analysis with aerial photography (FragMAP) system, which controlled the UAV for autonomous flight and shooting [40]. The relative flight altitude of the UAV was set to 20 m. The spatial resolution of the aerial images was approximately 1 cm, at which vegetation and non-vegetation pixels can be clearly distinguished. This was conducive to the acquisition of real FVC data. Each remote sensing monitoring plot contained a route covering the entire plot and 16 waypoints (each waypoint corresponded to a small plot). The ground area covered by the aerial images obtained along each flight route (i.e., the monitoring plot in this study) matched the pixel scale of the Moderate Resolution Imaging Spectroradiometer (MODIS) images. The ground area covered by each aerial image (i.e., the small plot in this study, which was also the ESU) matched the spatial scale of the Landsat image pixels. The specific field sampling plan is shown in Figure 2.

The results of previous studies have shown that the image excess green index (EGI) threshold segmentation method has a high accuracy in extracting FVC information [41,42]. Therefore, the EGI threshold segmentation method was adopted to process the aerial field images and then to calculate the measured FVC data for the corresponding plots. The specific calculation process of the image EGI threshold segmentation method is as follows: (1) Calculate the EGI value of each pixel in the image. (2) The initial threshold value of EGI is set. If the EGI value of an image pixel is greater than the threshold value, it is classified as a vegetation pixel. Otherwise, it is classified as a non-vegetation pixel. (3) The classification result is superimposed with the original image, and the accuracy of the segmentation result is judged by visual interpretation. If the vegetation information of the two images cannot be matched, the initial threshold of EGI needs to be adjusted repeatedly until they match. (4) Calculate the ratio of the number of vegetation pixels to the total number of pixels, and the obtained result is used as the measured FVC value of the image. The image processing results of this study are presented in Figure 3, which shows the high accuracy from the perspective of visual interpretation. To ensure the accuracy of the image processing, each image was independently processed by two experienced researchers. If the FVC difference obtained by the two people was greater than 0.05, a third experienced researcher repeatedly processed the image until the accuracy reached the standard, to minimize the influence of human subjective factors on FVC information extraction of images [31,39,42].

2.2.2. Remote Sensing Data

Sentinel-2 Data

Sentinel-2 data downloaded for free from the European Space Agency′s website (https://scihub.copernicus.eu/ (accessed on 7 June 2022)) can be accessed in the Google Earth Engine (GEE) platform, also for free [43]. A Sentinel-2 image preprocessor was developed on the GEE platform using the JAVA Script application programming interface for Sentinel-2 image preprocessing. GEE was used to screen the Sentinel-2 images with cloud contents of less than 20% in the TRSR. Then, the cloud and cloud shadows in the Sentinel-2 images were removed based on the pixel quality to create a cloud-free Sentinel-2 surface reflectance dataset from July to August 2019. Finally, the median function was utilized to generate a single image from the image collection.

MODIS Data

The MODIS data used were the MOD13Q1 vegetation product data, which provides normalized difference vegetation index (NDVI) data with a spatial resolution of 250 m and a temporal resolution of 16 days. The MOD13Q1 data can be downloaded for free from the Land Processes Distributed Active Archive Center website (https://lpdaac.usgs.gov/ (accessed on 17 June 2022)), and are freely available on the GEE platform. All of the MOD13Q1 images of the TRSR from July to August 2019 were acquired and processed using the GEE platform. Finally, a MODIS NDVI maximum value composite (MVC) image was created via the MVC method, which was used to evaluate the HUS of the monitoring plots.

2.2.3. FVC Product Data

MuSyQ FVC

The MuSyQ FVC product data for July–August 2019 were obtained from the National Earth System Science Data Center (http://www.geodata.cn/ (accessed on 1 June 2022)). The MuSyQ FVC data were calculated based on the MuSyQ leaf area index (LAI) product and the MODIS clumping index (CI) product data, and using gap probability theory [12]. The temporal and spatial resolution of the MuSyQ FVC data were 4 d and 500 m, respectively. First, we masked the pixels of non-vegetated areas, such as water and cities, according to the quality control layer of the MuSyQ FVC product, and merely retained the pixels of high quality for subsequent comparison between FVC products. Then, a single MuSyQ image was obtained by the MVC method.

GEOV3 FVC

The GEOV3 FVC product data for July–August 2019 were obtained from the Copernicus Global Land Service Center (https://land.copernicus.eu/global/products/fcover (accessed on 22 April 2021)). The spatial and temporal resolutions of GEOV3 FVC products were 300 m and 10 d, respectively, with a time range from 2014 to the present. A neural network model generated GEOV3 FVC. The initial data from 2014 to 2019 were the reflectance data of the PROBA-VEGRTATION sensor as model input data (since 2020, replaced with Sentinel-3 observations). CYCLOPE FVC products corrected by scaling coefficients were used as neural network model training samples [44]. A single GEOV3 image was obtained by the MVC method, and its spatial resolution was resampled to 500 m via the bilinear interpolation method.

2.3. Methods

The details of the research methods and processes are presented in the flowchart in Figure 4. The direct validation of these two FVC products involved three steps. First, the accuracy of the two products was directly validated according to the measured FVC values of the monitored plots. Then, the HUS of each monitoring plot was quantified using the MOD13Q1 NDVI data, the optimal threshold of the monitoring plot was adjusted and set (0.08 in this study), and the monitoring plots exceeding this threshold were removed. Finally, the accuracy of the two FVC products were validated again to evaluate the effect of the HUS on the accuracy of the direct validation.

The multi-scale validation method based on high-resolution data comprised three steps. First, Sentinel-2 remote sensing data were introduced as high-resolution remote sensing data, the band information was extracted and combined with the ESU FVC measured data, and the random forest (RF) regression algorithm was applied to invert the high-resolution reference FVC. Second, in order to ensure the reliability of the inversion results, the Sentinel-2 NDVI data were used to quantify the HUS of each ESU in order to determine and set the optimal threshold (0.10 in this study). Then, the RF regression model was trained again using the ESU data for which the HUS did not exceed the threshold. The accuracies of the FVC inversion for the two models were compared, and the high spatial resolution reference FVC was inverted based on the optimal model. The coarse-resolution reference FVC was obtained using the bilinear interpolation method. Finally, the pixel values of the locations of the monitored plots were extracted as the FVC reference truth values to validate the accuracy of the GEOV3 and MuSyQ FVC products. Based on the validation results of the HUS of the monitored plots in the direct validation method, the monitored plots exceeding the threshold were also removed to validate the accuracy of the GEOV3 and MuSyQ FVC products again. The impact of the HUS on the accuracy of the multi-scale validation method based on high-resolution data and the impact of the multi-scale validation method based on high-resolution data on the accuracy of the FVC products was evaluated according to the direct validation results. Based on the results of this study, we provide feasible suggestions for reducing the uncertainty of FVC products and FVC product validation results.

2.3.1. General Direct Validation Method

The direct validation method uses the measured values to compare the product pixel values, and the absolute accuracy of the product can be obtained. The core of direct validation is the acquisition of ground FVC measured data. However, due to a variety of objective factors, the coverage of field stations is often limited, resulting in spatial scale mismatch between the field measurement range and the satellite remote sensing image pixels, which affects the accuracy of the direct validation. In this study, UAV aerial photography was used to obtain the FVC data for the field monitoring plots, which expanded the spatial scope of the field monitoring plots and reduced the spatial scale gap with the GEOV3 and MuSyQ FVC product pixels. Therefore, the accuracy of the product validation results was more reliable.

2.3.2. Multi-Scale Validation Method

The multi-scale validation methods based on high-resolution data usually use the field measurement data combined with the high-resolution remote sensing image to obtain the high-resolution reference image through inversion, and then upscale to match the scale of the coarse resolution product, so as to complete the validation of the product. In this study, sentinel-2 images rather than Landsat-8 images were utilized as the high-resolution remote sensing image information to assist in the ground ESU scale upgrade. The main reason for this choice is that, although the spatial resolution of the Landsat-8 image matched the spatial coverage of the ESU, the central position of the ESU may not have coincided with the central position of the corresponding Landsat-8 image pixel, which would lead to errors in the spatial scale matching. However, the spatial resolution of the Sentinel-2 images was 10 m, and each pixel represented about one-ninth of the spatial range of the ESU. Therefore, the ESU covered a complete pixel in the Sentinel-2 image, which eliminated the mismatch between the ground sampling range and the satellite pixel to the greatest extent possible. Moreover, for the purpose of eliminating the influence of the HUS on the matching results, we used the HUS test method (see Section 2.3.3) to eliminate the monitoring plots with large heterogeneity before inversion was conducted to generate the high-resolution FVC reference images in order to ensure the reliability of the inversion results. Finally, the spatial resolution of the high-resolution FVC reference image generated via inversion was resampled to 500 m using the bilinear interpolation method. The pixel values of the upscaled high-resolution reference FVC image were used as the FVC reference truth values for the monitoring plots, which were compared with the pixel values of the GEOV3 and MuSyQ products. Thereby, the evaluation results of the product validation using the multi-scale validation method based on high-resolution data were obtained.

2.3.3. Method of Evaluating the HUS of the Monitoring Plots and Surrounding Area

Previous studies suggest that NDVI can characterize vegetation density and is an excellent indication of FVC [39,45]. Thus, we selected the NDVI as the index of the HUS of the monitoring plots, summing the square of the difference between the value of the center pixel and the value of the 8 pixels around the center pixel and then dividing the resulting value by the number of surrounding pixels. Finally, taking the square root, the obtained value, named H (Equation (2)), was used to quantitatively evaluate the HUS of the monitoring plot (Figure 5). The smaller the value, the smaller the difference between the central pixel and the surrounding pixels, indicating that the surrounding area of the remote sensing monitoring plot was relatively homogeneous, and the spatial HUS was relatively small. The calculation formulas of NDVI and H are as follows:

$$NDVI=\frac{NIR-RED}{NIR+RED}$$

(1)

$$H=\sqrt{\left({\displaystyle \sum _{i=1}^8{\left(NDV{I}_i-NDV{I}_0\right)}^2}\right)/8}$$

(2)

where NIR and RED are the reflectance of near infrared band and red band, respectively; $H$ is the calculated HUS, $NDV{I}_i$ is the NDVI value of the eight pixels around the central pixel, and $NDV{I}_0$ is the NDVI value of the central pixel.

2.3.4. High-Resolution Reference FVC Data Inversion Method

Previous studies have shown that the RF regression algorithm has the highest accuracy for FVC inversion [39]. Therefore, the RF regression algorithm was used to generate high-resolution reference FVC images. Compared with other machine learning models, the RF regression model can efficiently and quickly process large datasets and has a high robustness to noise [46]. The basic idea of the RF is an integrated learning method based on bagging. Multiple decision trees are integrated into a forest, and the average of the prediction results obtained by a single tree in the forest is taken as the final prediction result. The SmileRandomForest classifier provided by the GEE platform was used, where the numberOfTrees was set to 500, the minLeafPopulation was set as one-third of the feature number of the training samples (1 in this study), and the target data were the measured FVC values of the ESUs. The driving data were set as the reflectance information of the pixel bands (blue band, red band, and near infrared band) of the Sentinel-2 images. Moreover, 70% of the samples were used to train the random forest regression model, and the remaining 30% were used to validate the accuracy of the FVC inversion.

2.3.5. Accuracy Assessment

Four statistical indicators were selected for accuracy validation, namely, coefficient of determination (R²), RMSE, relative RMSE (RRMSE), and relative bias (RBias), which were respectively used to evaluate the goodness of fit, uncertainty, relative uncertainty, and relative bias. These indicators were calculated as follows:

$$R^2=1-\frac{{\displaystyle \sum _{\mathrm{i}=1}^N{\left(F_{\mathrm{i}}-f_{{}_{{}_i}}\right)}^2}}{{\displaystyle \sum _{\mathrm{i}=1}^N{\left(F_{\mathrm{i}}-\overline{f}\right)}^2}}$$

(3)

$$RMSE=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{\left(F_i-f_i\right)}^2}}$$

(4)

$$RRMSE=\frac{RMSE}{\overline{F}}\times 100\%$$

(5)

$$RBias=\frac{\overline{f}-\overline{F}}{\overline{F}}\times 100\%$$

(6)

where $N$ is the number of samples, $F_i$ represents the measured FVC values or the inverted FVC reference truth values of the monitoring plots, $f_i$ represents the values of FVC product pixels, $\overline{f}$ represents the average values of FVC product pixels, and $\overline{F}$ is the average of the measured values or the inverted FVC reference truth values of the monitoring plots.

3. Results

3.1. Comparison of GEOV3 and MuSyQ FVC

The pixel values of GEOV3 and MuSyQ FVC products are significantly different in TRSR (

Table 1. Pixel frequencies of different FVC value intervals in FVC products (%).

FVC Products	FVC Value Intervals
	0.0~0.1	0.1~0.2	0.2~0.3	0.3~0.4	0.4~0.5	0.5~0.6	0.6~0.7	0.7~0.8	0.8~0.9	0.9~1.0
GEOV3	9.83	11.64	10.35	9.7	8.58	8.41	8.92	8.87	9.11	14.59
MuSyQ	5.82	15.91	15.76	13.39	14.11	10.97	9.97	8.32	4.67	1.07

). Compared with the MuSyQ FVC product, the GEOV3 FVC product had more pixels with FVC values of greater than 0.8 and less than 0.1. Nonetheless, the MuSyQ FVC product had more pixels with FVC values between 0.1 and 0.7. The spatial distribution pattern of the differences between the GEOV3 and MuSyQ FVC products (Figure 6) revealed that the FVC values of the GEOV3 FVC product in the southern and eastern parts of the TRSR were significantly larger than those of the MuSyQ FVC product. The negative differences and positive differences between the two FVC products in the western part of TRSR were widely distributed, and the FVC values of the GEOV3 FVC product were less than or slightly more extensive than those of the MuSyQ FVC product in this region.

3.2. Direct Validation of GEOV3 and MuSyQ FVC Products Based on the Measured Values of Remote Sensing Monitoring Plots

The direct validation results based on the measured values of the remote sensing monitoring plots showed that the GEOV3 FVC and MuSyQ FVC products had good accuracies (Figure 7a,b). However, the accuracy of the GEOV3 FVC product (R² = 0.819, RMSE (RRMSE) = 0.127 (18.1%)) was better than that of the MuSyQ FVC product (R² = 0.702, RMSE (RRMSE) = 0.128 (18.3%)). For the GEOV3 FVC product, the scattered points composed of the measured values of the remote sensing monitoring plots and the pixel values of the corresponding product were evenly distributed around the 1:1 contour line, but they were relatively scattered, and some of the points deviated significantly from the 1:1 contour line. For the MuSyQ FVC product, the scatter of the points composed of measured values and pixel values of the corresponding product deviated greatly from the 1:1 contour line. When FVC > 0.3, most of the scattered points were located below the 1:1 contour line, exhibiting an evident underestimation phenomenon.

After removing the remote sensing monitoring plots with a large HUS, the validation uncertainty of the GEOV3 FVC and MuSyQ FVC products was significantly reduced (Figure 7c,d). That of the GEOV3 FVC product (R² = 0.904, RMSE (RRMSE) = 0.088 (11.4%)) was still better than that of the MuSyQ FVC product (R² = 0.833, RMSE (RRMSE) = 0.103 (13.3%)). For the GEOV3 FVC product, the scattered points composed of measured values of remote sensing monitoring plots and pixel values for the corresponding product were distributed around the 1:1 contour line, and the distance from the 1:1 contour line was very small. For the MuSyQ FVC product, the scatter of the measured values of the remote sensing monitoring plots and the pixel values of the corresponding product greatly deviated from the 1:1 contour line; and when FVC > 0.3, most of the scattered points were located below the 1:1 contour line, i.e., they still exhibited obvious underestimation.

3.3. Validation of GEOV3 and MuSyQ FVC Products with Using the Multi-Scale Validation

3.3.1. Inversion of High-Resolution FVC

The FVC inversion results of the RF regression model (

Table 2. Inversion of FVC accuracies using different datasets.

Datasets	Number of Training Samples	R2	RMSE	Number of Testing Samples	R2	RMSE
Original	4165	0.90	0.088	1785	0.89	0.093
H < 0.10	2923	0.94	0.068	1253	0.92	0.081

) showed that the accuracy (R² = 0.92, RMSE = 0.079) of the RF regression model trained using the high-quality dataset with a small HUS was better than that of the RF regression model trained using the original dataset (test: R² = 0.89, RMSE = 0.093). In addition, the inverse FVC values of the RF regression model trained using the high-quality dataset was highly consistent with the measured FVC values (Figure 8). The scattered points composed of the FVC values retrieved using the RF model and the measured FVC values were concentrated near the 1:1 contour line, and the percentage of the scattered points that were plotted slightly farther away from the 1:1 contour line to the total number of test samples was small.

3.3.2. GEOV3 and MuSyQ FVC Validation Using a High-Resolution FVC Map

The results of the product validation using the upscaled FVC reference true values of the monitoring plots showed that the GEOV3 and MuSyQ FVC products had higher accuracies, and the accuracy of the GEOV3 FVC product (R² = 0.910, RMSE (RRMSE) = 0.089 (12.7%)) was better than that of the MuSyQ FVC product (R² = 0.794, RMSE (RRMSE) = 0.107 (15.3%)) (Figure 9a,b). For the GEOV3 FVC product, the scattered points of the upscaled FVC reference truth values of the monitoring plots and the pixel values of the GEOV3 FVC product were evenly distributed around the 1:1 contour line, and the number of scattered points that were plotted slightly farther from the 1:1 contour line was very small. For the MuSyQ FVC product, the scattered points composed of the upscaled FVC reference truth values of the monitoring plots and the pixel values of the MuSyQ FVC product were also relatively concentrated. When FVC < 0.3, the scattered points were concentrated around the 1:1 contour line; however, when FVC ≥ 0.3, most of the scattered points were located below the 1:1 contour line, exhibiting obvious underestimation.

After removing the remote sensing monitoring plots with a large HUS, the validation uncertainty of the GEOV3 FVC and MuSyQ FVC products was further reduced (Figure 9c,d). That of the GEOV3 FVC product (R² = 0.954, RMSE (RRMSE) = 0.062 (8.0%)) was still better than that of the MuSyQ FVC product (R² = 0.872, RMSE (RRMSE) = 0.087 (11.2%)). For the GEOV3 FVC product, the scatter points composed of the upscaled FVC reference truth values of the monitoring plots and pixel values for the corresponding product were almost distributed around the 1:1 contour line. For the MuSyQ FVC product, the scatter of the upscaled FVC reference truth values of the monitoring plots and the pixel values of the corresponding product were slightly farther from the 1:1 contour line when 0.3 < FVC ≤ 0.9; however, the remaining scatter points were basically concentrated around the 1:1 contour line.

3.4. Error Distribution Pattern of GEOV3 and MuSyQ FVC Products

The spatial distribution of the difference between the measured FVC values obtained from the monitoring plots with small HUS and the pixel values of the GEOV3 and MuSyQ FVC products (Figure 10 and Figure 11) showed that the pixel values of the GEOV3 products were higher than the measured FVC values of the monitored plots located in the western and southern regions of the TRSR, and were lower than the measured FVC values of the monitored plots in the northern and western parts of the TRSR. However, there was no obvious pattern between the pixel values of the GEOV3 FVC product and the measured FVC values of the monitored plots in the central region of the TRSR. For the MuSyQ FVC product, the pixel values of the MuSyQ product were lower than the measured FVC values of most of the monitored plots in the TRSR, except for a few plots in the central and western parts of the TRSR.

According to the measured FVC values for the monitoring plots, five different FVC grade intervals were defined. The average values and standard deviations of the GEOV3 and MuSyQ FVC pixels corresponding to each interval are presented in

Table 3. The five FVC value intervals divided by the measured FVC values in the monitoring plots correspond to the average ± standard deviation (A ± SD), RMSE, and RBias of the GEOV3 and MuSyQ FVC.

FVC Value Intervals	Samples	Measured FVC (A ± SD)	GEOV3			MuSyQ
			A ± SD	RMSE	RBias	A ± SD	RMSE	RBias
0.0~0.2	12	0.117 ± 0.038	0.227± 0.060	0.062	94.0%	0.211 ± 0.039	0.038	80.3%
0.2~0.4	6	0.301 ± 0.052	0.247 ± 0.030	0.030	−17.9%	0.230 ± 0.016	0.017	−23.6%
0.4~0.6	5	0.475 ± 0.064	0.373 ± 0.135	0.084	−21.5%	0.317 ± 0.117	0.060	−33.3%
0.6~0.8	14	0.742 ± 0.046	0.745 ± 0.177	0.138	0.01%	0.531 ± 0.150	0.116	−28.4%
0.8~1.0	86	0.921 ± 0.052	0.966 ± 0.069	0.061	0.05%	0.794 ± 0.104	0.088	−13.8%

. The average FVC value of the GEOV3 product in the low vegetation cover area (FVC ≤ 0.2) was about 0.22, exhibiting obvious overestimation. In the medium-low vegetation cover area (0.2 < FVC ≤ 0.8), the average values of the GEOV3 FVC product were smaller than the average values of the measured FVC in the corresponding intervals, exhibiting obvious underestimation and a more discrete phenomenon (standard deviation > 0.13). The average value of the GEOV3 FVC was higher in the high vegetation cover area. For the MuSyQ FVC product, the average value of the MuSyQ FVC in the low vegetation cover area was about 0.20, i.e., similar to that of the GEOV3 FVC product, and it also exhibited a significant overestimation. However, in the remaining four FVC grade intervals, the average values of the MuSyQ FVC were lower than the average values of the measured FVC values in the corresponding intervals, and the standard deviations were higher.

4. Discussion

4.1. Comparative Analysis of the Differences between the GEOV3 and MuSyQ FVC Products

The GEOV3 and MuSyQ FVC products have obvious spatial distribution differences in the TRSR, and there is not a systematic underestimation or overestimation among products (Table 1 and Figure 6). This difference may be related to the estimation accuracies of the GEOV3 and MuSyQ FVC products for different vegetation communities and different vegetation densities, which is similar to the conclusion of Liu et al. [5]. For example, the vegetation growth is good in the southern and eastern regions of the TRSR, where the primary vegetation type is alpine meadow and the FVC values of the GEOV3 in this region are significantly higher than those of the MuSyQ. Alpine steppe and alpine vegetation are the predominant vegetation types, but there is poor vegetation growth in the western area of the TRSR. In the western region of TRSR, the FVC values of MuSyQ may be slightly higher than those of GEOV3, which may be due to the differences in the inversion models and algorithms, training samples, and input data used for the GEOV3 and MuSyQ FVC products [47].

4.2. Assessment of the Uncertainty of the Direct Validation Method

The lack of field measurements that can match the spatial scale of satellite remote sensing pixels is one of the main reasons for the uncertainty of the direct validation method. In previous product validation studies [48,49], the spatial range of the ESU deployed on the ground was limited by the surface environment, financial and material resources, and other factors, resulting in the spatial scale of the ESU being much smaller than the spatial resolution of the remote sensing products [22]. Due to the HUS of the monitored plots and the influences of other factors, direct comparison between the measured FVC values obtained for these ESUs and the pixel values of the coarse resolution remote sensing products is a typical comparison method for pixel values verified using a single point measurement; this is prone to problems, such as spatial scale mismatch error and representative error of single point observations. As a result, the direct validation results have great uncertainty [23]. In general, the monitoring plots used for the direct validation should be located in areas with a small HUS, such as uniform vegetation species and the same vegetation growth conditions [30]. The monitored plots in these regions have a sufficient spatial representation ability on the product pixel scale. Even if the spatial range is not completely consistent with the pixel scale of the coarse spatial resolution remote sensing products, there is no need for additional scale improvement, and the actual observed values of the monitored plots can be directly used to directly compare the pixel values of the product data [50]. Therefore, by evaluating the HUS of the monitoring plots, in this study, the monitoring plots with relatively large HUS were excluded, which essentially excludes the monitoring plots with weak spatial representation at the coarse spatial resolution of the product pixel scale. Our results show that the accuracy validation of the product is significantly impacted by the HUS. Nonetheless, the uncertainty of the direct validation results can be significantly reduced using the HUS test method (Figure 7), which should be used to reduce the uncertainty of the validation results in future remote sensing product validation.

4.3. Assessment of Uncertainty of Multi-Scale Validation Method Based on High-Resolution Data

Previous studies have shown that the multi-scale validation method can effectively reduce the impact of the mismatch between the spatial range of the ground sampling and the scale of the coarse spatial resolution remote sensing pixels on the validation results [17,23,28]. A host of studies have used the multi-scale validation method to validate the accuracies of products [25,26,51]. However, the majority of these studies only reported the validation results, and few studies have evaluated the impact of the multi-scale validation method on the validation of the remote sensing product’s accuracy [28]. Nevertheless, evaluating the uncertainty of the multi-scale validation method for accurate validation of remote sensing products is essential since the validation data of FVC products is the FVC reference truth value obtained via inversion of high-resolution images and upscaling, rather than the ground measured FVC value. The influence of the uncertainty of the upscaled FVC reference truth value validation results of FVC products cannot be ignored [23,29,31].

First, high-resolution reference data are uncertain, which is mainly due to the limitations of the parameter remote sensing inversion methods and the matching error between the measured data and the high-resolution remote sensing images. At present, although machine learning algorithms have been comprehensively utilized in remote sensing inversions [6,39], regardless of the algorithm used, searching for unknown information from remote sensing information that is limited and inadequate in terms of describing the complex land surface environment is an undetermined solution process. Thus, it is an ill-posed inversion problem [52], and the retrieved land surface parameters will have certain uncertainties. The reasons for the matching errors between the measured data and the high-resolution remote sensing images were the HUS and the inconsistency between the spatial range of the ESU, and the pixel scale of the high-resolution remote sensing images, which resulted in deviations between the measured FVC values and the realistic FVC values. Chen et al. showed that, in areas with large heterogeneity, the field measurement values obtained only via sampling within a small range deviate significantly from the real values [31]. Using these measured values with poor spatial representation as the real values to match satellite remote sensing pixels will significantly increase the uncertainty of the remote sensing inversion results. However, it is relatively difficult to obtain measured FVC values corresponding to remote sensing pixels at present. Therefore, Zhang et al. proposed an optimization measure, in which the field measurement range can cover at least 2 × 2 satellite remote sensing image pixels [53]. Thus, by selecting the pixel scale of Sentinel-2 images, which was smaller than the ground ESU scale, in this study, we essentially ensured that the measured data could match the high-resolution remote sensing image pixels, reducing the spatial representation error of the ESU at the high-resolution image pixel scale via the HUS test method. Our results show that the method of combining Sentinel-2 images and the RF regression algorithm has a high accuracy in retrieving the FVC, but the accuracy of the inversion results will be affected by the mismatch between the field measurement data and remote sensing image pixels. The uncertainty of the Sentinel-2 image inversion results was significantly reduced using the HUS evaluation method (Table 2 and Figure 8). In the future, in multi-scale validation of remote sensing products, selecting remote sensing images with a higher spatial resolution and using the HUS test are practical methods of reducing the uncertainty of the inversion results for high-resolution images.

The greater the heterogeneity of the monitoring plots, the greater the uncertainty of the FVC reference true value obtained [54,55]. Therefore, we evaluated the HUS of the monitoring plots and eliminated the monitoring plots with a high HUS and large uncertainty of the FVC reference true value before conducting the validation of the remote sensing products. Our results show that the HUS has a significant impact on the upscaling transformation of high-resolution reference data, but the uncertainty of the upscaling validation results can be significantly reduced using the HUS test method (Figure 9). In this study, regardless of whether the monitoring plots had a high HUS or a relatively low HUS, the upscaling via aggregation essentially ignored the serious impact of the HUS on the inversion results and the spatial scale transformation [18,28], resulting in overestimation of the actual accuracy of the remote sensing products. Therefore, it is necessary to consider the HUS of monitoring plots, and accurately evaluate the reliability of the multi-scale validation method in future product validation studies when the multi-scale method is used, such as the ground measured data error, inversion model and algorithm errors, upscaling transformation error, and high-resolution image quality [30]. A variety of methods have been used to minimize the uncertainty [52], such as strictly implementing the strategy of one test and two matches [53], rating the uncertainty of the pixels [23], and other methods, in addition to trying to evaluate the reliability of remote sensing products using the multi-scale validation method [17].

4.4. Error Analysis of GEOV3 and MuSyQ FVC Products

Compared with the FVC measurements for the monitored plots, the GEOV3 and MuSyQ FVC products exhibited regional overestimation or underestimation to a certain extent in the TRSR (Figure 10 and Figure 11). In the eastern and southern regions of the TRSR, where there was a large amount of vegetation, the monitored plots’ measured FVC values were lower than the GEOV3 FVC pixel values (Figure 10). Our statistical results and scatter plots also showed that, in the areas with measured FVC values of greater than 0.8, the FVC values of the GEOV3 were significantly higher (Figure 7c and Table 3). This result is similar to a recent GEOV3 accuracy validation study [49]. The higher estimate may be attributed to an excessive correction factor utilized to correct CYCLOPES FVC as the GEOV3 training sample [5]. The MuSyQ FVC product exhibited obvious underestimation in the areas with measured FVC values of greater than 0.3, in contrast to the GEOV3 FVC product (Figure 7d and Table 3). This may be because the MuSyQ FVC was based on gap probability theory [12], and the exponential expression leads to the poor estimation of the FVC of the MuSyQ product, which is low when the measured FVC is >0.3. In addition, the measured FVC values of the monitoring plots in the low-FVC regions (such as the western and northeastern parts of the TRSR) were all less than the pixel values of the two FVC products (Figure 10 and Figure 11). The reason for this may be that the FVC values in these regions are low, and there is soil background interference. The spatial resolution of the satellite sensors that collected the reflectance data for these two products is too low. The reflectance information they capture is highly uncertain, and this uncertainty will be further transmitted in the subsequent inversion process, resulting in FVC overestimation [56]. For both the GEOV3 and MuSyQ FVC products, the measured FVC values of the monitored plots in the central region of the TRSR were lower than or higher than the FVC values of the two products (Figure 10a and Figure 11a). We cannot accurately judge whether this region’s GEOV3 and MuSyQ FVC values were overestimated or underestimated. The central part of the TRSR is a transition zone between high and low vegetation cover, and the high altitude, rugged terrain, complicated hydrological conditions, and variable meteorological conditions impair the accuracy of the sensor reflectance information. Therefore, the accuracy of the remote sensing inversion of the FVC is affected. Relevant studies have shown that considering environmental factors and mixed water pixels can reduce the uncertainty of parameter remote sensing inversion results [51,57]. Thus, to better predict FVC values, it may be essential to tweak the GEOV3 and MuSyQ FVC algorithms’ parameters and increase the product’s regional performance in combination with TRSR environmental elements [58]. In addition, the GEOV3 and the MuSyQ FVC were generated using other remote sensing products. Detailed accuracy validations of more remote sensing products for the TRSR are needed to improve the GEOV3 and MuSyQ FVC products. Although our validation results showed that the accuracy of the GEOV3 FVC product is better than that of the MuSyQ FVC product from an annual maximum perspective, the MuSyQ FVC product has better temporal resolution than the GEOV3 FVC product, and the temporal resolution of a product is more important for some applications. Therefore, determining how to evaluate the two products more comprehensively in the future research needs to be further studied.

5. Conclusions

The accuracy of two global FVC products (GEOV3 and MuSyQ) in the alpine grassland ecosystem in the TRSR was validated in this study using a large number of multi-scale measurement FVC data and two commonly used product validation methods (i.e., the direct validation method based on in situ site measurements and the multi-scale validation method based on high-resolution reference images). Furthermore, the uncertainty of the multi-scale validation method was evaluated, and the influence of the HUS of monitored plots on product validation was revealed. We found that the accuracy of the GEOV3 FVC product is better than that of MuSyQ from the perspective of the annual maximum, and the multi-scale validation method based on high-resolution reference images leads to overestimation of the product’s accuracy. In addition, our study highlights the fact that the HUS of the monitoring plots greatly affects the product validation. By removing the remote sensing monitoring plots with large HUS, the uncertainties of the product validations using the two validation methods were significantly reduced. Future product validation studies should carefully evaluate and minimize the factors affecting the accuracy of the product validation methods.