Evaluating the Performance of Land Use Products in Mountainous Regions: A Case Study in the Wumeng Mountain Area, China

Abstract

Mountainous regions with complex terrain often generate mixed pixels, reducing the accuracy of land use classification. This study evaluates the applicability and performance of eight commonly used Land Use Products (LUPs) with resolutions from 10 m to 1000 m in a typical mountainous area of the Wumeng Mountains, using manually vectorized high-resolution imagery as reference data. The spatial patterns of classification inconsistency and topographic influences were systematically analyzed. Results show the following: (1) All LUPs underestimate Cropland, Forest, Built-up areas, and Waters, while overestimating Grassland. Most products perform poorly, with an Overall Accuracy (OA) below 0.7. (2) Classification performance varies by land cover type and location. In complex terrains, high fragmentation causes mixed pixels, leading to inconsistency. (3) XGBoost and SHAP analyses indicate elevation and Terrain Surface Complexity as dominant factors. Accuracy increases with elevation and is higher on shady than on sunny slopes. Greater surface complexity reduces performance. (4) Spatial resolution alone does not determine performance—algorithm design, sample distribution, and data fusion are critical. All eight LUPs show poor applicability in mountainous regions. Future research should emphasize terrain-stratified sample balancing, terrain-aware algorithms, and strategies to mitigate mixed pixel effects.

1. Introduction

Mountainous regions, covering approximately 27% of the global land area, are critical zones for biodiversity, water conservation, and ecosystem services [1]. Influenced by global climate change and human activities, significant alterations in land cover have been observed in these areas [2]. Land Use and Land Cover Change (LUCC) drives biophysical alterations—such as changes in vegetation structure, albedo, soil moisture, and habitat connectivity—that in turn impact ecological environments, water resources, biodiversity, and regional climate [3], with downstream consequences for the livelihoods of mountain residents and local socio-economic development [4]. Conducting LUCC research in mountainous regions is crucial for safeguarding fragile mountain ecosystems [5], advancing sustainable development [6], and constraining climate- and hazard-related risks [7], given steep environmental gradients and high exposure to global change.

Land use data serves as the foundation for LUCC research in mountainous regions. In recent years, a variety of Land Use Products (LUPs) covering global or large regional scales have been released [8,9]. These LUPs, based on diverse remote sensing data and classification methods, provide robust support for regional and global studies. However, these products differ in several key aspects.

First, data sources and spatial resolutions vary widely, ranging from the high-resolution Gaofen series to the medium- and low-resolution MODIS and PROBA-V, with spatial resolutions from sub-meter to kilometer scales [10]. High-resolution products capture fine-scale features but are limited in spatiotemporal coverage and computational efficiency, while medium- and low-resolution products may not meet the needs of complex mountainous areas [11]. Second, classification methods and systems differ. For example, GlobeLand30 uses supervised classification with ground sample points to improve accuracy [12], Dynamic World employs deep learning with frequent updates, ESA WorldCover uses a simplified UN-LCCS system, and MCD12Q1 combines multiple strategies [13,14]. These methods often do not fully account for mountainous terrain or the unique land use patterns in such regions. Third, sample set design remains a limitation [15]. Many LUPs attempt to address regional heterogeneity using stratified or adaptive sampling, but validation datasets still rely mainly on uniformly distributed ground observations, with insufficient coverage in steep or high-relief areas [16]. This leads to unstable model performance in complex terrain. The lack of standardized validation protocols further hinders comparability across studies. Ongoing international efforts, such as the FAO Land Cover Classification System (LCCS), aim to establish harmonized frameworks for more consistent validation [17,18]. Fourth, classification accuracy is uncertain [19,20]. Reported validation accuracies generally range from 75% to 85%, but heterogeneous validation methods (e.g., varying sample sizes, scales, or class aggregation) introduce additional uncertainty, especially in mountainous regions [21,22]. Consequently, the applicability of LUPs in complex terrain remains uncertain, highlighting the need for a standardized, region-specific evaluation.

Various evaluation methods for LUPs have been developed, with commonly used approaches including spatial consistency analysis, area deviation assessment, and confusion matrix analysis [23,24,25]. Many LUCC studies employ stratified sampling with point- or neighborhood-based samples to calculate overall, producer, and user accuracies. However, sample design, coverage, and spatial scale adaptability can introduce significant uncertainty, particularly in topographically complex mountainous regions [26]. To overcome these limitations, this study employs a full-coverage reference dataset that integrates key terrain factors, enhancing the reliability and representativeness of LUP assessments in rugged terrain. Despite these advances, the performance and applicability of LUPs in mountainous regions remain insufficiently explored. To address this knowledge gap, we focus on the following two questions: (1) What is the performance of LUPs in mountainous regions? (2) How do topographic factors influence the performance of LUPs in mountainous regions?

To achieve the research objectives, this study developed a classification performance evaluation framework for LUPs in mountainous regions based on full-coverage manually vectorized reference data. Using the Wumeng Mountains as a case study, eight widely used products (10–1000 m) were systematically evaluated through multiple approaches, including confusion matrix analysis, spatial consistency assessment, and area estimation comparison, while terrain factors were incorporated to reveal the mechanisms of classification inconsistency. The main innovations of this study are threefold: (1) establishing a mountain-oriented evaluation framework that integrates full-coverage reference data, multi-method accuracy assessment, and terrain heterogeneity quantification; (2) introducing an interpretable XGBoost–SHAP approach to identify key topographic drivers and their contributions to classification inconsistency; and (3) proposing performance diagnostics and application-oriented recommendations across land cover types and terrain zones based on dominant terrain variables.

2. Materials and Methods

2.1. Overview of the Study Area

The study area is located in a local region of the Wumeng Mountains in northeastern Yunnan Province, China (Figure 1), covering an area of approximately 31 km by 28 km. The region encompasses diverse landforms, including alpine meadows, alpine valleys, hills, low mountains, small basins, and mountainous reservoir areas, making it a typical mountainous area. The elevation varies significantly, ranging from 1200 m to 3700 m above sea level. Areas with slopes greater than 15° account for about 70% of the total region. The vegetation exhibits distinct vertical zonation. Due to variations in elevation and topography, the climate within the study area is highly heterogeneous. To quantitatively characterize this “high heterogeneity”, we report terrain metrics based on gridded statistics: elevation has a mean of 2520.93 m and a standard deviation of 410.75 m (CV = 0.16, indicating moderate variability); slope has a mean of 20.20° and a standard deviation of 10.84° (CV = 0.54, indicating high variability). According to widely used global land cover datasets, such as the ESA WorldCover [27] and MODIS MCD12Q1 [28], the main land cover types in this region include Cropland, Forest, Grassland, Built-up, and Waters, with a small proportion of other land types. These characteristics provide an ideal setting for evaluating the performance of LUPs in mountainous regions.

2.2. Data and Preprocessing

2.2.1. Land Use Products

The LUPs evaluated in this study include ESA WorldCover [27], DynamicWorld [29], MCD12Q1 [28], ESA_CCI [30], GlobeLand30 [31], CLCD [32], GLC_FCS30 [33], and CNLUCC [34], which were selected as they are widely used, publicly available, and represent the most influential global and national LUPs. The basic information of these products is summarized in

Table 1. The information of the LUPs.

Product Name	Data Source(s)	Methodology	Temporal Coverage	Spatial Resolution	Overall Accuracy
ESA WorldCover	Sentinel-1, Sentinel-2	CGLS-LC Algorithm	2020–2021	10 m	74.4% (2020), 76.7% (2021)
Dynamic World	Sentinel-2	Deep Learning (FCNN)	2015–present (updated every 2–5 days)	10 m	73.80%
GlobeLand30	Landsat, GF-1	Pixel-Object-Knowledge-based (POK)	1985, 2000, 2010,2020	30 m	85.72%
CLCD	Landsat	Random Forest	1990–present	30 m	80.00%
GLC_FCS30D	Landsat	Local Adaptive RF + Time Series	1985–2022	30 m	80.88%
ESA CCI	MERIS, AVHRR	Machine Learning	1992–2015	300 m	73.00%
MCD12Q1	MODIS	Decision Tree + HMM	2001–present	500 m	74.8% ± 1.3%
CNLUCC	Landsat	Visual Interpretation + ML	1980, 1990, 1995, 2000, 2015, 2018, 2020	30 m, 1000 m	85.40%

. These LUPs differ significantly in terms of resolution, classification systems, remote sensing data sources, and classification methods. Among them, ESA WorldCover, DynamicWorld, MCD12Q1, and ESA_CCI datasets were downloaded via the Google Earth Engine (GEE) platform. GEE provides a convenient API interface, enabling easy access to these globally covered datasets and allowing for the selection and clipping of spatiotemporal ranges. GlobeLand30 can be obtained from the National Geographic Information Resource Service Directory. CLCD data is available from the official website of Wuhan University. GLC_FCS30 data is published by the Aerospace Information Research Institute, Chinese Academy of Sciences, and can be accessed on its official website. CNLUCC is provided by the Resource and Environment Science and Data Center, Chinese Academy of Sciences.

To facilitate comparison, the resampling of LUPs with varying spatial resolutions (10 m to 1000 m) prioritized geometric consistency and error minimization through four key steps: (1) reprojection to EPSG:3857 for coordinate uniformity, (2) precise alignment of pixel boundaries with the 1 m target grid to eliminate spatial offsets, (3) nearest-neighbor resampling to preserve original class integrity without interpolation artifacts, and (4) study area clipping using grid-snapped boundaries to avoid edge effects. While upsampling coarser data does not enhance spatial detail, this methodology ensures pixel-level alignment accuracy across products, critically addressing spatial mismatch errors—a major challenge in multi-source LUP comparisons—while maintaining consistent georegistration throughout the study area. We note, however, that due to the varying spatial resolutions of the LUPs, the modifiable areal unit problem (MAUP) remains an inherent limitation when comparing products, which should be considered when interpreting the results.

2.2.2. Land Use Validation Data

The land use validation data was derived through visual interpretation of high-resolution remote sensing imagery. First, based on the Wayback historical imagery from June 2020 provided by ESRI (https://livingatlas.arcgis.com/wayback/, accessed on 15 August 2024), the images were connected and saved as 1 m high-resolution data using QGIS 3.16 software. To improve temporal consistency, we further incorporated Google Earth high-resolution imagery acquired between May and August 2020, which overlaps with the baseline year of the LUPs under evaluation. Using imagery from the same growing season minimized temporal mismatches and reduced potential bias caused by seasonal land cover dynamics. The coordinate system of this imagery was also EPSG:3857. To account for mountainous terrain, product classification inconsistencies, and vectorization efficiency, we adopted the land use and land cover classification system proposed by the “China 21st Century Land Use and Land Cover Spatio-Temporal Platform” [34]. In this study, the land use types were simplified into six first-level classes: Cropland, Forest, Grassland, Built-up, Waters, and Other Land Types. This classification scheme was designed to balance classification accuracy, visual interpretation efficiency, and consistency with existing land cover products. Notably, shrubland was integrated into the “Forest” class due to its spectral and visual similarity in mountainous regions, making it difficult to distinguish clearly during visual interpretation. However, this integration may pose the risk of overestimating forest cover. To address this issue, auxiliary data such as slope and NDVI were referenced, and uncertain areas were re-examined. Likewise, barren land and snow/ice—which were rarely present in the study area—were grouped into “Other Land Types” due to their extremely limited spatial extent.

The visual interpretation process was carried out jointly by three interpreters with field survey experience. Each interpreter independently digitized their assigned subset, and the results were ultimately integrated through a consensus process. To reduce subjectivity, clear standards were established in advance, including a comprehensive consideration of spectral color, spatial patterns, contextual information (adjacent land cover types), and textural characteristics (such as patch shape and grain size). Visual interpretation has potential limitations, such as human error or ambiguity in shaded and heterogeneous areas. To ensure the quality of the validation data, topological checks, spatial overlays, and spatial queries were employed to correct errors and supplement missing data. These steps included, but were not limited to, checking spatial overlaps between classes, verifying omissions in attribute classes, and ensuring that the spatial distribution of each category was reasonable. Additionally, 5% of the polygons were randomly selected for review by a second interpreter, with discussions held on uncertain areas until consensus was reached. This process served as a means of cross-validation and peer review, further enhancing the reliability of the dataset. Finally, the vectorized land use data was converted into 1 m resolution raster data, which served as the final validation dataset, hereafter referred to as MV_LUCC (Figure 2).

2.2.3. DEM Data

The Digital Elevation Model (DEM) data was used to extract topographic parameters such as slope, aspect, and Terrain Ruggedness Index (TRI). The topographic data employed in this study is sourced from the NASADEM_HGT/001 dataset [35] provided by NASA, with a spatial resolution of 30 m. This dataset was downloaded via the Google Earth Engine (GEE) platform. After downloading, the DEM data was reprojected to the EPSG:3857 coordinate system and clipped to the extent of the study area. To facilitate the comparison of the influence of topographic factors on LUPs at different scales, all subsequent topographic factors were calculated based on the 30 m resolution DEM data and then resampled to a 1 m resolution for statistical evaluation.

2.3. Harmonization of Land Use Classification Systems

Considering the geographical characteristics of the region, the current land use status, and the classification systems of existing LUPs, this study adopts the land use classification system proposed by Liu Jiyuan et al. in the “20th Century LUCC Spatiotemporal Platform of China”. The land cover types in the study area are categorized into six classes: Cropland, Forest, Grassland, Built-up, Waters, and Other Types. To facilitate comparative analysis of the LUPs, the original land use types of each LUP were reclassified and harmonized into these six major classes. The specific reclassification rules for each land use type are detailed in

Table 2. The corresponding table of land cover types of the LUPs and the classification system of this paper.

MV_LUCC	Cropland	Forest	Grassland	Built-Up	Waters	Other Types
ESAV100	40	10, 20, 95	30	50	80	60, 70, 90, 100
DynamicWorld	4	1, 3, 5	2	6	0	7, 8
GlobeLand30	10	20	30	80	60	40, 50, 70, 90, 100
CLCD	1	2, 3	4	5	5	6, 7, 9
GLC_FCS30	10, 11, 12, 20	51–92, 120–122	130, 140	190	210, 181–187	150–153, 200–202, 220
ESA_CCI	10, 11, 12, 20,30	40, 50, 610–62, 70–72, 80–82, 90, 100, 120–122	110, 130, 140	190	160, 170 180, 210	150–153, 200–202, 220
MCD12Q1	12, 14	1–7	8–10	180	17	11, 15, 16
CNLUCC	11, 12	21–23	31–33	51–53	43	46, 66

.

2.4. Performance Evaluation Methods for LUP

2.4.1. Construction of the Confusion Matrix

In this study, the widely used confusion matrix is adopted as the core evaluation framework [22]. The pixel-level correspondence between classification results and validation data was systematically compared. Assume the study area includes $n_{ij}$ land cover types. Let $n_{ij}$ represent the number of pixels that are classified as class i in the product but are validated as class j. Then, the confusion matrix is a C × C matrix $N=[n_{ij}]$. The diagonal elements of the matrix represent correctly classified pixels, while the off-diagonal elements reflect misclassification and omission errors.

2.4.2. Overall Performance Metrics

The overall performance metrics include Area Deviation Rate (ADR) and Overall Accuracy (OA).

• ADR measures the relative error between the area of a land cover type in the product and the area of the corresponding class in the validation data. It is used to evaluate the bias in area estimation of the classification results. The formula is as follows:

  $$AD{R}_i={\displaystyle \frac{P_i-V_i}{V_i}}\times 100\%$$

  (1)
  Among them, ADR_i represents the Area Deviation Rate for class i, P_i is the area of land cover type i in the product, and V_i s the area of land cover type i in the validation data.
• OA measures the ratio of correctly classified pixels for all land cover types in the product (after reclassification) compared to the validation data, relative to the total number of pixels in the study area. The formula is as follows:

  $$OA={\displaystyle \frac{{\displaystyle \sum _{i=1}^C}{n}_{ii}}{N}}\times 100\%$$

  (2)

  where n_ii is the number of correctly classified pixels for class i and N denotes the total number of pixels in the study area.

2.4.3. Performance Indicators of a Single Land Cover Type

The performance indicators of a single land cover type include Recall (PA), Precision (UA), and F1 Score (F1).

PA is used to evaluate the proportion of correct pixels for LUP classification in each land cover type for validation data. UA is used to assess the proportion of correctly classified pixels in each land cover type of the Land Use Product. The F1 comprehensively considers the balance between each product’s PA and UA. The calculation formula is as follows:

$$P{A}_i={\displaystyle \frac{n_{ii}}{{\displaystyle \sum _{j=1}^C}{n}_{\mathrm{ji}}}}\times 100\%$$

(3)

$$U{A}_i={\displaystyle \frac{n_{ii}}{{\displaystyle \sum _{j=1}^C}{n}_{ij}}}\times 100\%$$

(4)

$$F{1}_i=2\times {\displaystyle \frac{P{A}_i\times U{A}_i}{P{A}_i+U{A}_i}}$$

(5)

where ${\displaystyle \sum _{j=1}^C}{n}_{\mathrm{ji}}$ is the total number of pixels that actually belong to class i, ${\displaystyle \sum _{j=1}^C}{n}_{\mathrm{ij}}$ is the total number of pixels classified as class i in the product, and n_ii is the number of correctly classified pixels for class i.

2.5. Analysis of the Influence of Topographic Factors on LUP Performance

The classification of land use in mountainous regions is primarily influenced by the complexity of the terrain [36]. Mountainous topography not only affects the flow of surface water and the distribution of soil but also directly impacts the distribution of land use types and classification accuracy [37]. To systematically examine how topographic conditions affect the classification performance of LUPs, this study designed a three-part analytical framework: (1) extraction of terrain factors from DEM data, (2) evaluation of factor importance using ensemble learning (XGBoost and SHAP), and (3) analysis of how terrain factors influence the spatial distribution of classification inconsistency across LUPs.

2.5.1. Terrain Factor Extraction and Description

A total of eight terrain-related variables were derived from 30 m resolution DEM data using neighborhood-based terrain analysis and geomorphological classification methods [38]. These variables comprehensively describe elevation gradients, surface complexity, and solar radiation differences across the landscape.

Table 3. Summary of topographic factors used in this study.

Factor	Description	Computation Method
Elevation	Absolute height of land surface above sea level	Directly extracted from DEM
Slope	Steepness or gradient of terrain	Derived using 3 × 3 neighborhood from DEM
Aspect	Orientation of slope relative to north	Calculated using DEM gradients in x and y directions
Curvature	Rate of change of slope (convexity/concavity)	Derived from second derivatives of DEM surface
Roughness	Local terrain variability within a window	Standard deviation of elevation values in a 3 × 3 kernel
Surface_ Complexity	Degree of fragmentation and variation in micro-topography	Calculated via Terrain Ruggedness Index (TRI) using natural breaks for classification
Hillshade	Illumination value of terrain based on solar position (slope and aspect combined)	Computed using standard hillshade model with fixed sun angle
Geomorphon	Landform classification based on local terrain patterns (e.g., ridge, valley)	Extracted using Geomorphon algorithm in r.geomorphon

summarizes the terrain factors used in this study.

2.5.2. Variable Importance Analysis Based on XGBoost and SHAP

To quantify the contribution of each terrain factor to spatial consistency in land use classification, an XGBoost regression model was trained using topographic variables as input features and the spatial consistency of LUPs as the response variable. XGBoost was selected for its strong predictive performance, built-in regularization, and native handling of missing values, which are advantageous for complex topographic features. Compared with Random Forest, XGBoost also integrates seamlessly with SHAP for model interpretation, allowing us to evaluate both the linear and non-linear influences of terrain factors. For model training, data were randomly split into training (70%) and testing (30%) sets using train_test_split with random_state = 42. To enhance reproducibility and mitigate overfitting, we applied five-fold cross-validation on the training set. The final hyperparameters were set as follows: n_estimators = 500, max_depth = 10, learning_rate = 0.05, subsample = 0.8, colsample_bytree = 0.8, reg_lambda = 10, reg_alpha = 10. Model performance was evaluated on the held-out test set using R² and nRMSE. Since tree-based models are scale-invariant, no feature scaling or normalization was applied. SHapley Additive exPlanations (SHAP) were then employed to quantify the marginal contribution of each terrain factor to spatial consistency. This machine learning-based approach provides an interpretable and robust method to evaluate both linear and non-linear influences [39] of terrain on LUP performance. It is particularly suitable for mountainous regions where terrain effects are complex and spatially heterogeneous.

2.5.3. Statistical Analysis of Dominant Terrain Factors

Based on the importance rankings generated by the SHAP values, several dominant terrain factors (e.g., slope, TRI, and aspect) were selected for further stratified analysis. The classification performance of different LUPs was statistically compared across gradients of these factors to reveal spatial patterns and model sensitivity under varying topographic conditions.

3. Results

3.1. Performance Evaluation of LUPs in Mountainous Regions

3.1.1. Area Statistical Performance of LUPs

The areas of each land cover type for the eight LUPs and the validation data were separately calculated, and the ADR between the product areas and the validation data areas were determined. The ADR for each product’s land cover type areas are shown in (Figure 3). Overall, except for Other Types, the ADR of area estimation among different products show similarities. Most products underestimate the areas of Cropland, Forest, Built-up, and Waters, while overestimating Grassland areas. This bias likely stems from coarse spatial resolution, which misses small and fragmented patches, and from differences in class definitions, especially between Cropland and Grassland. Specifically, for Cropland, except for Globeland30 and CLCD, which overestimate Cropland areas, all other products underestimate Cropland areas. Among them, GLC_FCS30 shows the most significant underestimation (128%), while CLCD is closest to the actual area (underestimating by 30%). For Forest, areas are underestimated by all products. Globeland30 and MCD12Q1 show the largest underestimation, accounting for 240% of the actual area. For Grassland, all products overestimate Grassland areas, and the overestimation is severe. Globeland30, GLC_FCS30, and MCD12Q1 show deviations as high as 242%, 255%, and 423%, respectively. For Built-up, except for DynamicWorld, Built-up areas are underestimated by all products. For Waters, except for DynamicWorld, Waters areas are underestimated by all products. For Other Types, some products did not identify bare ground in the region, with relatively large deviations for global products and fewer deviations for China-specific regional products (CLCD, CNLUCC). In summary, the ESA_V100 and CLCD exhibit relatively small area ADR, while MCD12Q1, CNLUCC, and DynamicWorld products show larger ADR.

In summary, the area estimations of the eight LUPs all exhibit significant errors (ranging from −300% to −3% and 3% to 423%). To further assess differences in area deviations among products, we applied the Kruskal–Wallis test, which indicated no significant differences (H = 4.151, p = 0.8433), suggesting broadly consistent biases across products at the overall scale. Therefore, the analytical results based on these products applied to mountainous regions will likely carry substantial uncertainties.

3.1.2. Classification Performance of LUPs in Mountainous Regions

The classification performance of different LUPs in mountainous regions exhibits substantial variation across different metrics, including UA, PA, F1 score, and OA(Figure 4). In terms of UA, Forest performs the best, with six products achieving values above 0.85, while Grassland and Other Types show notably poor performance (Figure 4a). For PA, Grassland and Forest maintain relatively high performance, with most products exceeding 0.79. In contrast, Cropland and Other Types demonstrate lower PA values (Figure 4b). The F1 scores of all products remain below 0.82, with the majority falling under 0.70, reflecting the trade-off between UA and PA across different land cover types (Figure 4c). The OA for all products is below 70%, with ESA_V100, CLCD, and GLC_FCS30 showing relatively better performance (Figure 4d). From a resolution perspective, products with a spatial resolution greater than 100 m generally stabilize around 60% OA without a clear accuracy gap, while products with a spatial resolution finer than 100 m tend to show reduced OA as resolution increases.

To visually illustrate the confusion of land cover types for each product, we created confusion matrix diagrams for each product (Figure 5). Among them, ESA_V100 exhibited notable confusion: 37.26% of Cropland was misclassified as Grassland, 26.80% of Forest as Grassland, and 28.45% of Waters as Cropland, while 29.76% of Built-up was confused with Other Types. DynamicWorld showed the most significant misclassification issues, with only 27.27% of Cropland correctly identified, and approximately 67% misclassified as Grassland, Built-up, and Other Types; 27.26% of Forest was misclassified as Grassland, and 34.57% of Grassland was assigned to Other Types. GlobeLand30 had the best classification performance for Cropland and Grassland, with over 78% accuracy for both, but severely misclassified Forest, with 49.39% of it confused with Grassland. CLCD underestimated Built-up substantially, with only 16.51% correctly identified and the majority (73%) assigned to Cropland and Grassland. GLC_FCS30 notably underestimated Cropland, with 70.78% classified as Grassland, though it had relatively high accuracy in identifying Grassland (87.52%). The confusion in ESA_CCI, MCD12Q1, and CNLUCC was severe, with significant misclassification of Cropland, Forest, Waters, Built-up, and Other Types as Grassland.

In summary, the performance of the eight products in mountainous regions is generally low, with classification accuracy typically lower than the official claims or third-party validation results. The high degree of confusion between Cropland, Forest, Grassland, and Built-up leads to underestimation of Cropland, Forest, and Built-up areas, while overestimating Grassland areas [40]. Therefore, directly using these products in mountainous regions may introduce substantial errors. For land use planning (LUP) in mountains, we recommend integrating these maps with auxiliary data such as DEM-derived terrain metrics and SAR to improve class discrimination and reduce bias.

3.1.3. Visual Comparison with High-Resolution Imagery

Figure 6 presents a visual comparison of land use classification results from eight products within typical sample areas, using high-resolution satellite imagery as reference. The comparison highlights significant differences in class boundary delineation, the ability to identify fragmented land parcels, and the detection of small patches. Among the products, ESA_V100 demonstrates a superior capability in delineating land cover boundaries and capturing class-specific spatial details, offering relatively clear and precise representations of different land types. In contrast, medium-resolution products exhibit blurred class boundaries due to the limitations imposed by coarser pixel sizes. In terms of land cover classification accuracy, ESA_V100 shows a high degree of consistency with the reference data at the class level, but frequently misclassifies Cropland near urban areas as bare land. DynamicWorld tends to overestimate Built-up areas and often misclassifies Grassland and Waters as bare surfaces. GlobeLand30 provides generally reasonable classification outcomes, but exhibits noticeable confusion between Cropland and Grassland, as well as between Grassland and Forest, and shows simplified object boundaries. CLCD significantly underestimates the extent of Built-up areas, especially in urban zones, indicating its limited capacity to detect small-scale impervious surfaces. GLC_FCS30 frequently misclassifies valley bare rock as construction land and consistently underestimates Cropland area. Low-resolution products such as ESA_CCI, MCD12Q1, and CNLUCC suffer from excessive spatial generalization and a lack of fine-scale detail, which undermines their applicability for accurate land cover mapping in heterogeneous landscapes.

3.2. Spatial Distribution Characteristics of Misclassified Pixels in LUPs

To further explore the spatial distribution characteristics of misclassified pixels in each product, the frequency of misclassification for each land cover type at each pixel location was calculated, and the distribution maps of misclassification frequency for each category across all products were generated (Figure 7).

In Cropland, most patches were misclassified, with few patches having a misclassification frequency below 4, which may be due to the fragmented distribution of Cropland caused by mountainous terrain (a). In the Forest, more than half of the area was misclassified, but the extraction consistency was relatively good around river valleys and urban areas (b). Grassland showed the highest consistency across all products, with overall low misclassification frequencies (c). Built-up areas in concentrated urban regions were well extracted, but scattered rural buildings had higher misclassification frequencies (d). Waters also showed high consistency, but misclassification frequencies were higher in urban areas and canyon regions (e). Other Types were not correctly identified in most products (f). In summary, regions with lower misclassification frequencies are mainly distributed in flat, low-altitude areas and mountainous regions near reservoirs, especially along river valleys and around urban areas. Regions with higher misclassification frequencies are generally scattered and primarily located on fragmented patches on mountain slopes.

3.3. Topographic Factors Influencing LUP Performance in Mountainous Regionss

3.3.1. Variable Importance via Ensemble Interpretation Methods

The XGBoost model achieved R² = 0.19 and nRMSE = 0.24, indicating that topography explains a limited yet non-negligible portion of LUP inconsistency. This suggests that spatial consistency is jointly shaped by terrain and non-terrain factors (e.g., training-sample design, sensor characteristics, and class schema). Despite the modest predictive power, the SHAP analysis provides meaningful insights into the relative contributions of terrain variables. The importance rankings (Figure 8) highlight Elevation as the most influential factor, reflecting its strong nonlinear interactions with other topographic features. Geomorphon and Terrain Surface Complexity also show high importance, underscoring the role of terrain morphology and heterogeneity in driving inconsistencies across LUPs. In contrast, Curvature and Roughness consistently exhibit low SHAP values, indicating limited contributions.

The SHAP summary plot further validates the dominant role of Elevation and Surface Complexity, both characterized by wide SHAP value distributions. High-elevation samples generally exhibit positive SHAP values, implying a greater likelihood of inconsistency in elevated terrain. Similarly, more complex surfaces are associated with higher disagreement among products. Slope and Aspect show moderate yet less consistent effects, while Hillshade, Geomorphon, Roughness, and Curvature display narrower distributions, suggesting more localized influence. Overall, these results underscore the primary role of vertical and surface heterogeneity in shaping land cover classification uncertainty in mountainous regions, while also highlighting the limited but interpretable explanatory capacity of terrain alone.

3.3.2. Role of Elevation Gradients

Accuracy trends across 200 m elevation intervals (Figure 9) indicate a clear positive correlation between elevation and classification accuracy for most LUPs. The average accuracy curve (red line) steadily increases with elevation, exceeding 0.8 above 3500 m. The orange dashed trend line and confidence interval further confirm this upward trajectory. Products such as ESA_V100, CLCD, GLC_FCS30, ESA_CCI, GlobeLand30, MCD12Q1, and CNLUCC all reach peak accuracy between 3000 and 3800 m. In contrast, DynamicWorld exhibits a distinct inverted-U pattern, peaking around 2000 m and declining sharply above 2500 m, suggesting challenges in distinguishing high-elevation land cover types. This pattern can be explained by land cover composition and heterogeneity: mid- and low-elevation areas typically have highly fragmented and diverse land cover (e.g., Cropland, Forest, Built-up), increasing mixed-pixel effects and reducing classification accuracy. In contrast, high-elevation zones are dominated by simpler, more homogeneous classes such as Grassland and bare rock, which reduces spectral confusion and allows higher classification accuracy. The atypical trend of DynamicWorld highlights the need for algorithmic refinement in alpine regions. In high-mountain areas, we observe pixels mapped as Waters and Bare. Based on the Dynamic World methodology and its reported uncertainties [29], two mechanisms are likely: (1) near-real-time, single-scene inference combined with residual clouds/shadows and seasonal snow can bias labels toward Waters/Snow and Ice in complex terrain; and (2) limited separability and training coverage for spectrally similar alpine classes (grass–shrub–bare–snow/ice) in the FCNN lead to over-allocation to Bare and occasional Waters. These factors plausibly explain Dynamic World’s atypical elevation pattern in our study area. Selecting elevation-specific products and optimizing performance below 2500 m is recommended to improve LUP accuracy in mountainous environments.

3.3.3. Influence of Terrain Surface Complexity

Terrain Surface Complexity was classified into five levels using natural breaks (Figure 10). Overall, most LUPs showed a negative correlation between classification accuracy and surface complexity. Cropland, Built-up areas, Waters, and bare land were especially sensitive, with accuracies dropping sharply as complexity increased. For Cropland, accuracy fell from >0.6 at Levels 1–2 to <0.2 at Level 5 across most products, reflecting difficulty in distinguishing fragmented agricultural plots. Built-up areas also exhibited steep declines, with accuracy often below 0.1 in complex zones. In contrast, Grasslands maintained the highest and most stable accuracy (0.7–0.9), and Forest remained relatively robust, particularly for CLCD, ESA_CCI, and GLC_FCS30. Water bodies showed strong declines, dropping near zero in highly complex terrain. Products like ESA_V100, CLCD, and GLC_FCS30 performed better in complex regions, while DynamicWorld and MCD12Q1 were more affected. These patterns highlight the need for finer-resolution data or terrain-adaptive algorithms in complex mountainous environments.

3.3.4. Effects of Aspect Directionality

Classification performance across different aspect classes (shady, semi-shady, semi-sunny, sunny) exhibits a clear resolution dependency (Figure 11). High-resolution products (10–30 m), such as ESA_V100, CLCD, DynamicWorld, GlobeLand30, and GLC_FCS30, generally achieve higher accuracy on shady and semi-shady slopes. For instance, CLCD shows a notable accuracy difference between shady (0.74) and sunny slopes (0.63). ESA_V100 shows a similar, albeit smaller, gap (0.68 vs. 0.63). These differences (0.05–0.2) highlight that finer spatial resolution better captures the spectral and structural contrast caused by differential illumination and vegetation cover between aspects. In contrast, coarse-resolution products (300–1000 m; e.g., MCD12Q1, CNLUCC) show minimal accuracy variation across aspects (≤0.05), indicating limited sensitivity to aspect-related features due to mixed-pixel effects and terrain smoothing.

This resolution-dependent pattern suggests that shady slopes, often more mesic and less heterogeneous, are easier to classify accurately. Sunny slopes, with greater land cover heterogeneity and spectral complexity, present more classification challenges. Furthermore, local climatic conditions may promote denser vegetation on shaded slopes, reinforcing their spectral distinctiveness. Overall, aspect directionality should be considered when applying or developing land classification products in mountainous areas.

4. Discussion

4.1. Limitations and Applicability of Existing LUPs in Mountainous Regions

Although existing LUPs are widely applied in large-scale studies, their applicability in complex mountainous environments is significantly limited. In this study, all eight evaluated LUPs exhibit OA below 0.69 in mountainous areas, substantially lower than their officially reported values, with a marked degradation in accuracy observed in regions of steep slopes. Mechanistic analysis reveals that terrain-induced spatial heterogeneity, shadow effects, and the lack of topographic correction are primary contributors to the confusion between Forest and Grassland classes. This highlights the need for implementing topographic correction procedures to mitigate illumination differences and improve class separability in mountainous areas. Although high-resolution products such as WorldCover generally perform better, their classification accuracy also declines under complex terrain conditions. This suggests that, beyond spatial resolution, factors such as the representativeness of training samples, classification algorithms, and the integration of multi-source data (e.g., SAR and DEM) play more critical roles in determining accuracy. Furthermore, most existing products are trained in low-slope or homogeneous vegetation areas, leading to substantial extrapolation bias when applied to highly complex mountainous regions, manifested by a systematic overestimation of Grassland areas and underestimation of Forest and Built-up areas. This extrapolation bias can be quantitatively assessed and corrected using approaches such as comparing class-specific accuracy across terrain strata or employing transferability metrics like accuracy drop relative to reference low-slope areas [22,41]. Therefore, the limited usability of LUPs in mountainous regions can be attributed to terrain–model mismatches, sample biases, and insufficient topographic sensitivity. Future improvements should focus on stratified sampling based on terrain and cross-regional validation to enhance model robustness and generalizability.

4.2. Spatial Scale Sensitivity of Land Use Classification in Mountainous Regions

While spatial resolution is a critical factor influencing land use classification accuracy—especially evident as the OA of the eight LUPs significantly declines when resolution exceeds 100 m, and 10–30 m products generally outperform those with a 100–500 m resolution—a deeper mechanism analysis reveals that higher resolution does not always equate to higher classification reliability [42] in mountainous regions. First, high resolution amplifies the effects of terrain shadows and image noise, resulting in comparable declines in classification accuracy on both sunny and shady slopes, indicating that increasing resolution alone cannot eliminate terrain-induced spectral confusion. Second, with increasing terrain complexity, land cover patches become more fragmented, and the number of mixed pixels rises accordingly. Although high resolution enables the detection of small-scale features, overall classification performance may still deteriorate due to intensified landscape fragmentation. In this study, we quantified class fragmentation using neighborhood-based metrics such as the number of land cover types within a local window, and found a clear inverse relationship: greater land cover diversity within a neighborhood corresponds to lower classification accuracy. This finding highlights the critical role of mixed pixel effects in reducing performance in rugged terrains. Finally, significant accuracy differences among products of the same resolution suggest that algorithmic adaptability, sample design, and multi-source data fusion (e.g., SAR and DEM integration) are equally important. In conclusion, while high spatial resolution can mitigate certain classification challenges in mountainous regions, it does not fundamentally solve them. This study focuses on the comparison of multi-scale products, and future research will explore classifications derived from imagery of consistent resolution (e.g., Landsat 30 m or Sentinel 10 m) to further improve comparability and analysis depth. Future research should focus on developing integrated optimization strategies that couple resolution with terrain complexity, through shadow correction, multi-scale feature extraction, and resolution-adaptive algorithms.

4.3. Approaches to Improve Land Use Classification Performance in Mountainous Regions

In light of the limited performance of existing LUPs in complex mountainous terrain, this study proposes five key strategies:

• SAR–Optical Targeted Fusion for Shadow-Affected Zones. Although 10–30 m imagery generally outperforms coarser (>100 m) products, similarly resolved datasets (e.g., ESA WorldCover vs. GLC_FCS30) still exhibit marked accuracy differences in steep areas, indicating that spatial resolution alone is insufficient [43]. Existing pipelines that fuse SAR backscatter with DEM and slope still suffer from shadow-induced misclassifications in karstic terrain [44]. Future work should therefore design SAR–optical fusion strategies specifically adapted to mountainous shadow zones—such as deep feature–level fusion in multi-channel networks—to improve robustness under complex illumination [45].
• Topography-Aware Algorithms: Moving Beyond Generic Terrain Inputs. Simply appending elevation or slope as ancillary variables yields limited gains. Deep learning architectures—like convolutional neural networks with spatial encoding or graph-based models—can embed detailed topographic structures via multi-resolution feature fusion and topography-aware loss functions. Hybrid CNN–Transformer models further enhance the capture of both local and global terrain heterogeneity [46].
• Stratified and Balanced Sampling Designs for Complex Terrain. Training data scarcity in highly fragmented and steep regions undermines model generalization. Terrain-stratified sampling frameworks (e.g., slope and aspect zoning) combined with class-balanced (over-)sampling ensure that rugged and shadowed areas are adequately represented [47]. Density-adaptive schemes that allocate samples according to local heterogeneity can further stabilize learning in high-complexity zones.
• Scalable Feature Engineering: Beyond Object-Based Methods. Object-based image analysis (OBIA) integrates spectral, textural, and geometric cues locally but struggles with segmentation inconsistencies and computational load at continental scales. Instead, multi-scale texture encodings, transformer-based attention mechanisms, or hierarchical context models offer more scalable, end-to-end frameworks to capture mountain spatial heterogeneity [48].

4.4. Comparison with Previous Studies and Key Contributions

Previous studies have made significant progress in validating and comparing multi-source land cover products, yet they often focus on either broad-scale assessments or relatively homogeneous regions. For example, Ji et al. [24] found that land cover products in Southwest China showed substantially lower local accuracies than their reported global values, particularly in fragmented terrains. Xu et al. [25] compared recent 10 m global products and demonstrated Overall Accuracies of 73–84%, with performance deteriorating in heterogeneous landscapes due to limited spatial detail and reference data uncertainty. Similarly, Liu et al. [49] reported that product consistency was relatively high for homogeneous surfaces such as bare land and Cropland, but declined markedly for Grassland and Forest in transitional or topographically complex areas. These findings collectively indicate that existing studies converge on the difficulty of applying LUPs in heterogeneous or rugged environments, though they largely remain descriptive, emphasizing accuracy assessment and cross-product consistency. The main contributions of this study are twofold. First, we evaluated eight LUPs in the Wumeng Mountains, finding consistent underestimation of Cropland, Forest, Waters, and Built-up areas, and overestimation of Grassland, indicating limited reliability in mountainous regions. Second, using an interpretable XGBoost + SHAP framework, we examined terrain effects, showing that elevation, slope, and surface complexity are key factors, with accuracy decreasing non-linearly as terrain complexity rises. These findings provide new mechanistic insights into how terrain drives mapping errors and offer methodological advances for improving LUP applications in mountainous regions.

5. Conclusions

In summary, this study systematically assessed the classification performance of eight commonly used LUPs in typical mountainous regions and their mechanisms influenced by topographic factors. The main conclusions are as follows: Despite their extensive application, the overall classification accuracy of the eight LUPs remains limited (OA < 0.7), and area estimations exhibit substantial biases, highlighting considerable uncertainty in mountainous environments.

• Topographic heterogeneity and mixed-pixel effects dominate errors; flat areas and valleys show lower misclassification, whereas hillside dispersed patches show higher misclassification.
• Elevation and Terrain Surface Complexity strongly affect spatial consistency, while slope and aspect have secondary influence. Medium- and high-resolution products perform better on shaded slopes, but complex terrain reduces consistency.
• High-resolution products generally outperform coarse-resolution ones, yet misclassifications persist in fragmented or shaded areas. Differences among same-resolution products indicate that training sample distribution, algorithm sensitivity, and multi-source data fusion critically affect performance.
• Although the study area is representative, cross-regional validation is needed to assess generalizability, and stratified analysis methods could be further refined. For highly complex terrain, we recommend terrain-stratified balanced sampling, SAR–optical fusion with DEM-based topographic normalization, and terrain-aware modeling to mitigate mixed-pixel and illumination effects. Future research should focus on terrain-stratified sampling, terrain-aware algorithm design, and joint optimization of multi-scale features and multi-source data. Practical challenges—data availability, computational cost, and integration complexity—must be carefully addressed for wider adoption of SAR–optical fusion.