An Interpretable Machine Learning Framework for Unraveling the Dynamics of Surface Soil Moisture Drivers

Abstract

Understanding the impacts of the spatial non-stationarity of environmental factors on surface soil moisture (SSM) in different seasons is crucial for effective environmental management. Yet, our knowledge of this phenomenon remains limited. This study introduces an interpretable machine learning framework that combines the SHapley Additive exPlanations (SHAP) method with two-step clustering to unravel the spatial drivers of SSM across Iran. Due to the limited availability of in situ SSM data, the performance of three global SSM datasets—SMAP, MERRA-2, and CFSv2—from 2015 to 2023 was evaluated using agrometeorological stations. SMAP outperformed the others, showing the highest median correlation and the lowest Root Mean Square Error (RMSE). Using SMAP, we estimated SSM across 609 catchments employing the Random Forest (RF) algorithm. The RF model yielded R² values of 0.89, 0.83, 0.70, and 0.75 for winter, spring, summer, and autumn, respectively, with corresponding RMSE values of 0.076, 0.081, 0.098, and 0.061 m³/m³. SHAP analysis revealed that climatic factors primarily drive SSM in winter and autumn, while vegetation and soil characteristics are more influential in spring and summer. The clustering results showed that Iran’s catchments can be grouped into five categories based on the SHAP method coefficients, highlighting regional differences in SSM controls.

1. Introduction

Surface soil moisture (SSM) influences climate processes by governing the distribution of precipitation into runoff, evapotranspiration, and infiltration, and by affecting the partitioning of incoming energy into latent and sensible heat fluxes [1,2,3]. SSM also plays a crucial role in the global hydrological cycle, as well as in understanding water resource management, flood generation, and climate change at local and global scales [4,5]. The significance of SSM becomes more pronounced in dry and semi-arid regions, where water scarcity is a prominent concern [6]. In these regions, variations in SSM can trigger cascading effects, influencing groundwater recharge rate [7], vegetation dynamics [8], and regional climate patterns [1]. Hence, precise monitoring and prediction of SSM, along with the investigation of influencing factors, are indispensable for sustainable water resource management, risk assessment, and mitigation of drought and other hydrometeorological hazards in these regions [9,10,11].

However, a major bottleneck in SSM monitoring, particularly in arid and semi-arid regions, is the lack of high-quality and reliable in situ data [12,13,14]. While in situ sensors provide localized, high-quality measurements, their geographic coverage is often limited due to financial, logistical, and accessibility constraints [15]. In many countries, particularly those with vast arid and semi-arid regions, installing and maintaining in situ networks is challenging [12,13,16]. Thus, validating global SSM datasets with in situ measurements is becoming increasingly important [17]. Country-specific validation of these global datasets allows for the calibration of models to regional conditions, enhancing their reliability and applicability for resource management strategies [16,18].

Understanding the factors that influence SSM is key to effective monitoring and management [14,19]. SSM is influenced by numerous variables, including precipitation [20,21], texture and organic matter content of soil [3,22], topography [23], vegetation [24], and groundwater [25]. Seasonal trends, land-use changes, and evapotranspiration rates are also critical elements affecting SSM dynamics [26,27]. Understanding these variables and their interactions is vital for developing strategies for SSM management, particularly in arid and semi-arid regions where every drop of water matters [28]. Insight into these factors can enhance the precision of hydrological models and contribute to the development of adaptive management strategies for water resources [29].

The increasing complexity and non-linearity of relationships among environmental variables affecting SSM present significant challenges for traditional modeling approaches [29,30]. To address these challenges, machine learning (ML) techniques have gained popularity in hydrological modeling due to their ability to process large datasets and capture intricate interactions among multiple predictors [31]. Among various machine learning models, Random Forest (RF) stands out for its ability to deliver consistent predictions by reducing variance without increasing prediction bias [32,33]. However, a major criticism of ML approaches is their “black-box” nature, which often limits interpretability and hinders their adoption in environmental sciences [34]. To overcome this limitation, recent studies have employed SHapley Additive exPlanations (SHAP)—a model-agnostic, game-theoretic technique that explains model predictions by quantifying the contribution of each input variable [35,36]. Unlike traditional variable importance measures, SHAP not only indicates which variables are most influential, but also reveals the direction of their effects (positive or negative), thus providing deeper insight into model behavior [34].

Therefore, the objective of this study is to develop an interpretable machine learning framework to investigate the key environmental drivers influencing SSM—including the magnitude and direction of their effects—across diverse hydro-climatic regions in Iran. We achieve this by integrating the RF model with SHAP and a two-step clustering method to identify and interpret the spatial and seasonal dynamics of key environmental drivers of SSM. Unlike previous studies that are often limited to local scales or lack interpretability, our approach provides a scalable, interpretable, and seasonally explicit analysis of SSM drivers across 609 catchments. The insights derived from this framework aim to support data-driven water resource management in arid and semi-arid regions.

2. Materials and Methods

2.1. Study Area

Iran with an area of approximately 1,648,195 km² is located in the Middle East and faces challenges related to water resource scarcity [37]. Iran is typically characterized by an arid and semi-arid climate, with an average annual precipitation of approximately 250 mm [38]. As indicated by the climate classification shown in Figure 1, a significant portion of the country falls under the category of a warm-dry climate, naturally resulting in SSM deficits [39]. In recent years, SSM deficits in Iran have been exacerbated by factors that include climate change, excessive groundwater extraction, mismanagement of surface water, and inefficient irrigation practices [40]. Such SSM shortages can lead to detrimental environmental consequences, including desertification, soil degradation, dust storms, wind erosion, and the degradation of air and water quality [41]. Therefore, it is imperative to monitor SSM and study the factors influencing it in Iran to ensure effective environmental management [17,42]. Nonetheless, SSM remains inadequately monitored in numerous regions of Iran, and the existing measurements as depicted in Figure 1, lack sufficient temporal and spatial resolution [38].

Under these circumstances, remote sensing data can offer a viable solution for monitoring SSM in Iran [43]. Consequently, it is essential to validate and assess global SSM products using in situ stations to gauge their real-world effectiveness before using them across all climatic regions in Iran. In this study, after validating the SSM products, the environmental factors affecting SSM were investigated across 609 catchments in Iran. The catchments cover a total drainage area of 1,648,195 km² [44].

These catchments are distributed across diverse climates, spanning from cold-dry to warm-humid, as illustrated in Figure 1. They exhibit different topographic characteristics, geological compositions, soil types, and vegetation, leading to diverse hydrological conditions throughout the country. Additionally, the land cover map obtained from ESA with a 10 m resolution [45,46] reveals that bare/sparse vegetation, grasslands, croplands, and forests constitute the predominant land cover in the studied catchments, accounting for 64.2%, 18.9%, 11.8%, and 1.70%, as given in Table S1. Given its vast area and predominantly arid to semi-arid climate, Iran experiences severe soil moisture deficits driven by both natural and anthropogenic factors. Its diverse topography, land cover, and climate zones make it an ideal case study for evaluating SSM dynamics. Moreover, the insights gained—particularly regarding the role of environmental drivers—can be valuable for other semi-arid regions worldwide facing similar climatic and hydrological challenges.

2.2. Datasets

2.2.1. SSM Data

Following the flowchart illustrated in Figure 2, daily in situ SSM data from agrometeorological stations were gathered from the Iran Meteorological Organization, IMO [39] for the period spanning 1 February 2006, to 31 March 2023. Unfortunately, SSM data are not widely available in numerous regions of Iran and contain several gaps [38,42]. Hence, a total of 42 stations were chosen throughout Iran, distributed as follows: 13 stations in cold-dry climates, 2 in cold-humid, 11 in warm-dry, 3 in warm-humid, 8 in moderate-dry, and 5 in moderate-humid (Figure 1). The limitation of in situ stations largely stems from financial, logistical, and accessibility challenges that hinder the establishment and maintenance of dense monitoring networks in Iran [39]. Although the in situ dataset is available from 2006, we selected the period from April 2015 to March 2023 to ensure consistency with the temporal coverage limitations of the other datasets used in this study. Considering previous studies (e.g., [15,47,48]), we utilized three primary SSM datasets, namely Soil Moisture Active Passive (SMAP), Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2), and Climate Forecast System, Version 2 (CFSv2), to validate SSM at a depth of 0–5 cm using in situ data (Table 1).

The SMAP satellite mission by NASA was launched on 31 January 2015, to globally map soil moisture and the freeze/thaw state of landscapes [49]. Although the satellite was initially equipped with both an L-band radar and an L-band radiometer, the radar instrument encountered a failure after approximately 11 weeks of operation, and the production of soil moisture data continues based solely on the radiometer measurements [47]. SMAP measurements provide direct sensing of SSM in the upper 5 cm of the soil [15,50]. In this study, we employed the SMAP Level 4 passive product [51], which features a spatial resolution of 9 km (Table 1).

Table 1. A summary of the SSM datasets used in this study.

Data Name	Institution	Examined Period	Spatial Resolution	Temporal Resolution	Reference
In situ	IMO	2015–2023	point location	daily	[39]
SMAP Level 4	NASA	2015–2023	9 km × 9 km	daily	[52]
MERRA-2	NASA	2015–2023	56 km × 70 km	daily	[53]
CFSv2	NCEP	2015–2023	22 km × 22 km	daily	[54]

Table 1. A summary of the SSM datasets used in this study.

Data Name	Institution	Examined Period	Spatial Resolution	Temporal Resolution	Reference
In situ	IMO	2015–2023	point location	daily	[39]
SMAP Level 4	NASA	2015–2023	9 km × 9 km	daily	[52]
MERRA-2	NASA	2015–2023	56 km × 70 km	daily	[53]
CFSv2	NCEP	2015–2023	22 km × 22 km	daily	[54]

MERRA-2, generated by the NASA Global Modeling and Assimilation Office (GMAO), is the most recent atmospheric reanalysis covering the modern satellite era, offering global reanalysis data spanning from 1980 to the present [53]. MERRA-2 is validated against in situ measurements in North America, Europe, and Australia, demonstrating that its performance is slightly superior to that of ERA-Interim/Land [55]. In this study, we utilized MERRA-2, with a spatial resolution of 56 km by 70 km (Table 1).

CFSv2 was made operational at the National Centers for Environmental Prediction (NCEP) in March 2011 [56,57]. The soil moisture dataset within the CFSv2 encompasses four layers (at 5, 25, 70, and 150 cm depths) and has a spatial resolution of approximately 22 km [54]. It is worth mentioning that while SMAP directly measures SSM, MERRA-2 and CFSv2 rely on model-based reanalysis data generated through physical models, observations, and data assimilation. These three SSM datasets were resampled to 0.25° × 0.25° by bilinear interpolation [19] to ensure consistency across the datasets.

2.2.2. Factors Influencing SSM

We selected the candidate factors that influence SSM based on previous studies (e.g., [3,58,59]) and data availability (Table 2, Figure S1). We used the catchment-averaged monthly data of the following parameters: precipitation, potential evapotranspiration, solar radiation, wind speed, normalized difference vegetation index (NDVI), and groundwater table depth. Additionally, time-invariant catchment attributes such as distance from water bodies, clay fraction, organic matter fraction, elevation, and topography roughness index were included (Table 2). The mean monthly precipitation for the studied catchments was computed using the Radial Basis Function (RBF) interpolation method [60] in Python (v 3.11.5). This was based on daily precipitation data collected from 422 synoptic stations between 2015 and 2023 from IMO [39]. The mean monthly potential evapotranspiration for the studied catchments was determined using the MODIS global evapotranspiration product MOD16A2 [61] at a 500 m resolution.

Solar radiation and wind speed data were obtained from the ERA5-Land dataset [62,63] at a spatial resolution of 11 km. Vegetation dynamics were analyzed using the NDVI derived from Sentinel-2 data [64] at 10 m resolution. The NDVI has proven to be a suitable index for detecting vegetation changes, particularly within arid and semi-arid regions [7]. To generate a time series of groundwater table depth, we utilized monthly data from 11,003 observation wells collected by the Iran Water Resources Management Company [44]. We derived the average monthly groundwater table depth for each catchment using the RBF interpolation method.

Table 2. Summary of the candidate factors considered in this study that impact the SSM.

Type	Dataset	Source	Spatial Resolution	Reference
Dynamic	Precipitation	In situ observations	point location	[39]
	Potential evapotranspiration	MODIS	500 m	[61]
	Solar radiation	ERA5-Land	11 km	[62]
	Wind speed	ERA5-Land	11 km	[62]
	Normalized difference vegetation index	Sentinel-2	10 m	[64]
	Groundwater table depth	In situ observations	point location	[44]
Static	Distance from water bodies	NOAA	-	[65]
	Clay fraction	SoilGrids	250 m	[66]
	Organic matter fraction	SoilGrids	250 m	[66]
	Elevation	ALOS AW3D30	30 m	[67]
	Topography roughness index	ALOS AW3D30	30 m	[67]

Table 2. Summary of the candidate factors considered in this study that impact the SSM.

Type	Dataset	Source	Spatial Resolution	Reference
Dynamic	Precipitation	In situ observations	point location	[39]
	Potential evapotranspiration	MODIS	500 m	[61]
	Solar radiation	ERA5-Land	11 km	[62]
	Wind speed	ERA5-Land	11 km	[62]
	Normalized difference vegetation index	Sentinel-2	10 m	[64]
	Groundwater table depth	In situ observations	point location	[44]
Static	Distance from water bodies	NOAA	-	[65]
	Clay fraction	SoilGrids	250 m	[66]
	Organic matter fraction	SoilGrids	250 m	[66]
	Elevation	ALOS AW3D30	30 m	[67]
	Topography roughness index	ALOS AW3D30	30 m	[67]

The mean distance of each catchment from water bodies was determined by utilizing the water bodies’ data [65] and the Euclidean Distance method in ArcGIS [68]. The SoilGrids250m dataset [66] was employed to extract the clay and organic matter fractions within the upper vadose zone. Finally, we calculated the mean elevation and topography roughness index for the studied catchments using an ALOS DEM with a 30 m resolution [67] and Focal Statistics tools in ArcGIS. All factors influencing SSM were extracted using the Google Earth Engine platform [69], Python (v 3.11.5), and ArcGIS software (v 10.7.1). All SSM-influencing factors, derived from datasets with varying spatial resolutions, were resampled to a common resolution of 0.25° × 0.25° by bilinear interpolation to ensure consistency with the SSM datasets. Additional details regarding the factors influencing SSM and their correlation are provided in the Supplementary Material.

2.3. Methods

2.3.1. Statistical Metrics

The performance of global SSM products was assessed using the Root Mean Squared Error (RMSE), Relative Bias (RBias), Kendall’s Tau (τ), and Kling–Gupta efficiency (KGE′, Figure 2) [70,71,72,73]:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{N}}{({\mathrm{Tar}}_{\mathrm{i}}-{\mathrm{Ref}}_{\mathrm{i}})}^2}}{\mathrm{N}}}}$$

(1)

$$\mathrm{RBias}={\displaystyle \frac{{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{N}}({\mathrm{Tar}}_{\mathrm{i}}-{\mathrm{Ref}}_{\mathrm{i}})}}{{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{N}}{\mathrm{Ref}}_{\mathrm{i}}}}}$$

(2)

$$\mathsf{\tau }={\displaystyle \frac{{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle {\sum }_{\mathrm{j}=1}^{\mathbf{n}}\mathrm{sign}({\mathrm{Tar}}_{\mathrm{i}}-{\mathrm{Tar}}_{\mathrm{j}})\mathrm{sign}({\mathrm{Ref}}_{\mathrm{i}}-{\mathrm{Ref}}_{\mathrm{j}})}}}{\mathrm{n}(\mathrm{n}-1)}}$$

(3)

where N represents the number of samples, Ref is the reference values (i.e., in situ data), and Tar is the target values (i.e., SMAP, MERRA-2, and CFSv2 data) for each record (i). Furthermore, we utilized the KGE′ statistic, initially introduced by [74] and subsequently modified by [75]. KGE′ balances the contributions of correlation, bias, and variability terms as follows [70]:

$$\mathrm{KGE}=1-\sqrt{{(\mathrm{r}-1)}^2+{(\mathsf{\beta }-1)}^2+{(\mathsf{\gamma }-1)}^2}$$

(4)

$$\mathsf{\beta }={\displaystyle \frac{{\mathsf{\mu }}_{\mathrm{Tar}}}{{\mathsf{\mu }}_{\mathrm{Ref}}}}$$

(5)

$$\mathsf{\gamma }={\displaystyle \frac{{\mathrm{CV}}_{\mathrm{Tar}}}{{\mathrm{CV}}_{\mathrm{Ref}}}}={\displaystyle \frac{{\mathsf{\sigma }}_{\mathrm{Tar}}/{\mathsf{\mu }}_{\mathrm{Tar}}}{{\mathsf{\sigma }}_{\mathrm{Ref}}/{\mathsf{\sigma }}_{\mathrm{Ref}}}}$$

(6)

where r represents the correlation coefficient between the Ref and Tar datasets, γ is the variability ratio, β is the bias ratio, μ is the mean SSM, and CV is the coefficient of variation that represents the standard deviation. The better values have higher KGE [71]. The results of these metrics are illustrated in Figure 3, where τ, KGE′, RMSE, and RBias are presented in sub-panels (a) through (d), respectively, for the period from 1 April 2015 to 31 March 2023.

2.3.2. Random Forest (RF)

Due to its ability to produce consistent predictions by reducing variance without increasing bias [32], we chose RF to model SSM in 609 catchments across Iran. The RF model was initially introduced by Leo Breiman in 2001 [76]. The RF algorithm does not necessitate any alterations, conversions, or modifications to the input data, and it autonomously handles missing values [77,78]. An RF model comprises a multitude of decision trees designed to be as uncorrelated as possible [79]. To create an uncorrelated collection of trees, the RF employs bagging and feature randomization during the construction of the decision trees. This implies that each tree is trained on a random sample drawn from the training set with replacement, a technique known as bootstrapping [80]. Additionally, each tree is limited to a random subset of the available features [77].

In this study, we used the RandomForestRegressor from the machine learning library sklearn [81,82] for the Python programming language. For enhanced convergence speed and to mitigate the impact of local extremes on training, the input variables undergo normalization to a range of 0.1 to 0.9 before the training process. As some models encounter issues when inputs are normalized between 1 and 0, we chose to normalize the input variables using the adjusted min-max method within the range of 0.1 to 0.9 [83]. In this study, satellite-based SSM products were first evaluated against in situ observations from 42 agrometeorological stations for the period 2015–2023. The product demonstrating the highest agreement was selected to train the RF model over 609 catchments. Although the dataset covers the period from 2015 to 2023, our analysis did not involve direct time-series modeling. Instead, we aggregated the data seasonally, calculating the average SSM and associated predictors for each season. This seasonal averaging approach effectively removed the temporal sequence from the data, making serial autocorrelation irrelevant to both our modeling and validation processes. First, 80% of the data (487 catchments) was randomly allocated for training, and the remaining 20% (122 catchments) was reserved for testing. Then, 10-fold cross-validation was applied solely to the training set for hyperparameter tuning, serving as a safeguard against model overfitting [79]. Finally, the RF model’s performance in estimating SSM across different seasons was evaluated on the reserved test set using R-squared (R²), RMSE, and Mean Absolute Error (MAE).

2.3.3. SHAP

The SHAP method represents a model-agnostic game-theoretic technique for interpreting machine learning models [35]. Unlike conventional methods that only quantify the influence of input variables on the model output, SHAP can reveal whether each variable exerts a positive or negative impact on the model [34]. In other words, SHAP can analyze an individual prediction by considering it as a composite result of the combined effects of each input variable on the output value (i.e., the predicted value). This approach allows users to gain insight into the magnitude and direction of each variable’s influence on the output [35,84]. In recent years, SHAP has been widely applied to environmental data to improve the interpretability of ML models (e.g., [85,86,87,88]). Using a pre-trained machine learning model denoted as M and a set of input variables x = {x₁, …, x_q}, SHAP employs an explanation model E to ascertain the individual influence of each variable on the behavior of model M [36]. SHAP is expressed as

$$\mathrm{E}={\mathsf{\varphi }}_0+{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{q}}{\mathsf{\varphi }}_{\mathrm{i}}{\mathrm{t}}_{\mathrm{i}}}$$

(7)

$${\mathsf{\varphi }}_{\mathrm{i}}(\mathrm{M},\mathrm{x})={\displaystyle {\sum }_{\mathrm{t}\subseteq \mathrm{x}}\frac{\left|\mathrm{t}\right|!(\mathrm{q}-\left|\mathrm{t}\right|-1)!}{\mathrm{q}!}}\left[\mathrm{M}(\mathrm{t})-\mathrm{M}(\mathrm{t}\backslash \mathrm{i})\right]$$

(8)

where q represents the number of input variables, t is the variable simplification, ϕ_i ∈ R represents the contribution of each variable to the machine learning model, and \ is the difference-set notation for set operations [34,36]. Recent enhancements to SHAP have focused on optimizing TreeExplainer for tree-based models, reducing computational costs while improving accuracy. Notably, ref. [89] proposed polynomial-time algorithms for computing SHAP values by leveraging the structural properties of tree ensembles, enabling exact attributions without the exponential complexity typically associated with Shapley value calculations. Earlier, ref. [90] introduced Fast TreeSHAP, an algorithm that accelerates SHAP value computation by up to 2.5 times through strategic caching and precomputation, significantly improving scalability for large datasets. Additionally, ref. [91] developed a functional decomposition approach for gradient-boosted trees, which separates main effects and interactions to enhance the interpretability of SHAP outputs. These methodological advancements have made TreeExplainer both more efficient and more robust in capturing detailed feature contributions. Accordingly, in our study involving ensemble tree models, we employed these improved SHAP implementations to ensure scalable and reliable feature attribution.

Two major types of SHAP plots are beeswarm and waterfall plots. In a beeswarm plot, features are displayed on the vertical axis and SHAP values on the horizontal axis. Each point represents a SHAP value for a specific feature in an individual sample. Red points indicate high feature values, while blue points indicate low ones. Red points located on the right suggest that high feature values increase SSM, whereas their presence on the left implies that high values reduce SSM. Similarly, blue points on the right suggest that low feature values increase SSM, while those on the left indicate that a decrease in the feature leads to lower SSM, as illustrated in Figure 7. In a waterfall plot, red bars represent features that contribute to increasing the predicted SSM, while blue bars indicate features that reduce it. The horizontal length of each bar reflects the magnitude of the feature’s impact on the prediction relative to the model’s baseline, as shown in Figure 8.

2.3.4. Cluster Analysis

A two-step cluster analysis was performed on the SHAP model outputs to enhance our spatial understanding of the effects of factors influencing SSM within the studied catchments. In other words, employing a two-step cluster analysis can offer spatial interpretations for the factors that affect SSM, utilizing the outcomes generated by RF and SHAP. Compared to k-means and balanced iterative reducing and clustering using hierarchies (BIRCH), the two-step cluster analysis offers several advantages. These include its ability to handle both categorical and continuous variables, automatically determine the optimal number of clusters, and scale effectively for large datasets [92,93]. We follow the methodology for the two-step cluster analysis outlined by [94]. Only a summary of the technique is provided here, and rather, the reader is directed to [94] for a detailed description of the approach. The method involves two steps: (1) whole records are probed by distance to construct a classification tree, where records in the same tree node are most similar [95]; (2) nodes are classified using the cohesion technique and clustering results are evaluated using the Bayesian information criterion (BIC) or the Akaike information criterion (AIC), which determine the structure of the final cluster [96].

3. Results and Discussion

3.1. Performances of SSM Products

Figure 3 shows the validation results of the three global SSM datasets based on data from 42 in situ stations between April 2015 and March 2023. The findings suggest that, among the evaluated datasets, SMAP stands out with the highest median values for τ and KGE (0.740 and 0.690). It also exhibits the lowest median values for RMSE and RBias (0.068 and 0.030). Following SMAP, the MERRA-2 product shows median values of τ and KGE at 0.684 and 0.604, along with median RMSE and RBias values of 0.085 and 0.034, respectively. CFSv2 shows the lowest median values for both τ and KGE (0.550 and 0.500), and it also has the highest median values for RMSE and RBias (0.113 and 0.059) among the three products, as illustrated in Figure 3. These findings indicate that SMAP provides superior performance in estimating SSM across Iran compared to the other datasets.

The performance of SMAP aligns with the findings of the study conducted by [97], which reported that SMAP has a global average anomaly correlation of 0.76. Ref. [15] evaluated eight global root zone soil moisture products (0–1 m depth) across the globe. Their findings indicated that SMAP, MERRA-2, JRA-55, and ERA-5 consistently showed stronger correlations with in situ root zone soil moisture measurements compared to GLDAS, NCEP R1, and NCEP R2. Ref. [47] validated SMAP SSM using core validation sites. They reported that the SMAP radiometer-based SSM product meets its expected performance, achieving an unbiased RMSE of 0.04 m³/m³ for volumetric SSM. It is noteworthy that global evaluations have not incorporated the in situ SSM data from Iran. Few studies have explicitly focused on evaluating SSM in Iran. These studies either concentrate on a particular local area, such as the Lake Urmia Basin [98,99] or cover a short period in Iran (i.e., 2015–2016, as demonstrated in [42]). Additionally, some studies were solely concerned with validating a single product (e.g., [17,100,101]). Ref. [42] validated SSM products from SMAP, SMOS, and AMSR2, using 23 in situ stations in Iran from 2015 to 2016. Their results pointed to SMAP as the best-performing satellite-based product. Also, ref. [100] stated that SMAP has a strong capacity for SSM data retrieval in Iran.

3.2. Spatial-Temporal Pattern of SSM

We calculated SSM for 609 catchments in Iran using the SMAP dataset, which demonstrated its optimal performance (Figure 4). Figure 4 illustrates the mean catchment-averaged daily SSM across 609 studied catchments from April 2015 to March 2023, delineated by different seasons and based on the SMAP dataset. The mean SSM for different seasons reveals distinct patterns, with winter having the highest median at 0.175 m³/m³, followed by spring at 0.160 m³/m³, while autumn and summer exhibit lower medians of 0.096 m³/m³ and 0.081 m³/m³, respectively. These findings indicate notable seasonal variations in SSM within the studied catchments. Ref. [102] stated that many catchments in Iran lack natural moisture, especially during the summer, leading to a heightened demand for irrigation in agriculture during this season. According to [103], differences in soil moisture levels between dry (summer) and wet (winter) conditions are more pronounced in the upper surface layers (0 to 20 cm) when compared to deeper layers. Figure 4 also demonstrates that catchments with SSM exceeding 0.20 m³/m³ are predominantly located in northern, northwestern, western, and southwestern Iran, primarily in regions characterized by cold-humid and moderate-humid climates. In contrast, catchments with SSM below 0.05 m³/m³ are primarily concentrated in central and southeastern Iran, which are characterized by warm-dry climates.

3.3. SSM in Different Land Covers

Numerous studies have shown that changes in SSM exhibit varying characteristics when subjected to different land cover types (e.g., [104,105,106]). The boxplots of SSM within the six primary land cover categories in Iran, as determined by the land cover map of ESA with a 10 m resolution [45] for the period spanning 1 April 2015 to 31 March 2023, are displayed in Figure 5a. The results reveal significant variations in median SSM across different land covers. Specifically, we found that forests exhibited the highest SSM with a median value of 0.180 m³/m³, followed by grasslands at 0.148 m³/m³, croplands at 0.148 m³/m³, shrubland at 0.117 m³/m³, built-up areas at 0.107 m³/m³, and bare/sparse vegetation at 0.093 m³/m³. SSM in various land-use patterns along the lower Bhavani River in India has been studied [107]. The study concluded that SSM is higher in forested areas compared to fallow land and built-up areas. Figure 5a also indicates that grasslands exhibit the highest soil moisture variability, while bare/sparse vegetation shows the least diversity across Iran.

Figure 5b indicates a time series of SSM for various land cover types in Iran, spanning from 1 April 2015 to 31 March 2023. The results show that in the early months of 2019, especially in March and April, SSM reached higher levels compared to the same months in the preceding and subsequent years (Figure 5b). The analysis of the precipitation time series in Iran (as shown in Figure 5b) for the years 2015–2023 reveals that the increase in SSM during those particular months is due to the increase in precipitation. This finding aligns with previous research. A study has demonstrated that a significant increase in precipitation during the early months of 2019 led to a rise in the water level of Lake Urmia in northwestern Iran [108]. Due to the heavy and unprecedented precipitation event between mid-March and April 2019, widespread flooding events affected 25 out of the 31 provinces in Iran [73,109,110]. These events resulted in more than 77 human fatalities and caused approximately USD 2.2 billion in damages.

3.4. RF and SHAP

Figure 6 illustrates the comparison of the SSM data obtained by SMAP vs. RF model performance across different seasons during the testing phase. The findings indicate that RF can yield SSM estimations with R² values of 0.89, 0.83, 0.70, and 0.75 for the winter, spring, summer, and autumn seasons, respectively. Corresponding RMSE values are 0.076, 0.081, 0.098, and 0.061 m³/m³, while MAE values are 0.058, 0.060, 0.076, and 0.047 m³/m³, demonstrating consistent performance across seasons. The decrease in R² values observed during summer is likely due to the significant shortage of SSM during this season compared to others, notably winter (as shown in Figure 4). The good performance of the RF method in estimating SSM in this study aligns with the findings of previous research. For example, in the semi-arid region of West Khorasan-Razavi province in Iran, ref. [111] utilized several machine learning algorithms for SSM estimation. Their study concluded that the RF method provided the most precise results. Another study [112] conducted a study comparing spectral and spatial-based approaches to map local SSM variations in the Balikhli-Chay watershed in northwestern Iran. Their findings revealed that the RF approach outperformed others, demonstrating the highest level of performance in SSM modeling.

Machine learning techniques are often considered black-box models, which limits their interpretability regarding the process of making predictions [113]. SHAP offers a way to understand the influence of each feature on the model’s outputs [114]. Figure 7 presents beeswarm plots from the SHAP analysis for various seasons. The findings reveal that the primary factors influencing SSM vary from season to season (Figure 7). In the winter season, the key factors that exert the most influence on SSM are precipitation, distance from water bodies, solar radiation, clay fraction, potential evapotranspiration, and elevation, respectively (Figure 7a). Winter is typically associated with increased precipitation in many of Iran’s catchments [115,116]. Previous research demonstrated a direct contribution of precipitation to SSM (e.g., [117,118]). The proximity of the studied catchments to water bodies, such as the Caspian Sea and the Persian Gulf, plays a vital role in determining SSM during the winter. For example, the distance from water bodies can influence relative humidity, which consequently impacts SSM [33,119]. Solar radiation with an inverse impact on SSM tends to be lower during the winter due to shorter days and reduced sunlight, which affects the rate of moisture evaporation from the soil as reported by [120]. So, lower solar radiation in winter can help maintain higher SSM.

Recently, the spatial and temporal variability of soil moisture and its influencing factors across the northern agricultural regions of China have been investigated [121]. Their findings indicated that soil moisture exhibited a negative correlation with temperature and sunshine duration, while showing a positive correlation with precipitation and relative humidity. In a study [105], it is demonstrated that as soil depth increases, the influence of natural factors on soil moisture anomalies gradually diminishes, whereas the impact of human-related factors becomes more pronounced. This suggests that human activities exert a stronger influence on deeper soil layers. A study [122] examined the driving factors of soil moisture in the Heihe River Basin and found that during months with low soil moisture, land cover and elevation were the main influencing factors. In contrast, during months with higher soil moisture, the NDVI and land surface temperature played the primary roles.

As the fourth important factor influencing SSM in winter, the clay fraction can enhance the soil’s water-holding capacity [3,123,124]. Clay soils retain moisture more effectively, contributing to higher SSM levels. One of the other factors affecting SSM is potential evapotranspiration, which tends to be lower during the winter in the studied catchments due to cooler temperatures. This reduction in potential evapotranspiration can contribute to SSM preservation. Ref. [33] highlighted potential evapotranspiration as a key determinant in estimating root zone soil moisture within the Raam catchment in the Netherlands. Ref. [125] analyzed the spatiotemporal variability of soil moisture and its dominant driving factors, showing that evapotranspiration played a more significant role in tropical areas. Ref. [126] examined the long-term evolution of soil moisture and its driving factors across China’s agroecosystems. Their findings revealed that in the plateau mountain and temperate continental climate zones, relative soil moisture was primarily influenced by temperature and precipitation, respectively. In temperate humid regions, climate change emerged as the dominant controlling factor, while in subtropical humid zones, grain output exhibited a negative impact on relative soil moisture.

Elevation, identified as the sixth most influential factor, has a negative impact on SSM during winter. Catchments at different elevations may experience variations in temperature and slope, which can influence SSM retention [127,128]. The analysis of the published data on seven potential factors influencing the temporal stability of soil water content indicated that the influence of these factors appears to be interconnected rather than solely driven by a single dominant factor [129]. These results align with hydrological theory, which states that increased precipitation and reduced radiation contribute to higher SSM during winter.

As the spring season commences and plants and trees in Iran start to grow, the significance of NDVI, distance from water bodies, clay fraction, and organic matter fraction becomes more pronounced than precipitation (Figure 7b). In other words, during spring, land cover and soil characteristics take on greater importance compared to winter. On the other hand, increased plant growth and the decomposition of organic materials, driven by increased temperatures, can lead to an increase in soil organic matter content in spring. This, in turn, notably influences the soil’s ability to retain water. Based on [3], an increase in organic matter content results in an enhanced water-holding capacity due to the inherent affinity of organic matter for water. Ref. [102] investigated long-term spatiotemporal variations in SSM and vegetation indices across Iran. They concluded that NDVI exerts a noteworthy influence on the spatiotemporal variations in SSM. Ref. [130] estimated agricultural farm SSM using spectral indices and demonstrated that NDVI and land surface temperature possess substantial potential for extracting valuable SSM information. In spring, the increasing importance of NDVI and organic matter aligns with established ecohydrological understanding, as vegetation and organic matter enhance infiltration and water retention capacity.

With the significant decrease in precipitation in summer, the dominant factors affecting SSM are proximity to water bodies, clay fraction, potential evapotranspiration, NDVI, organic matter fraction, and elevation (Figure 7c). These findings reveal that the influence of the clay fraction on SSM reaches its peak during this dry season, surpassing the impact seen in other seasons. Catchments with a substantial clay content can effectively retain SSM during this period. In summer, the dominance of the clay fraction highlights the importance of soil texture in arid climates. This aligns with the water balance concept, which suggests that soils with higher water-holding capacity—such as those rich in clay or organic matter—are better able to buffer against drying.

As autumn arrives and precipitation begins, there is a shift in the hierarchy of factors influencing SSM fluctuations, with precipitation emerging as the predominant factor impacting SSM during this season (Figure 7d). It is worth noting that additional factors, such as groundwater table depth, wind speed, and topography roughness index, exert a negative influence on SSM. These factors have a relatively minor impact on SSM in comparison to other factors. The low influence of groundwater table depth on SSM can be attributed to the average depth of the groundwater table in the studied catchment, approximately 31 m (as indicated in Figure S1f). The impact of groundwater depth on SSM tends to become significant mainly in catchments with shallower groundwater tables, particularly in winter. Overall, the SHAP-derived feature importance patterns align well with established hydrological processes, including seasonal water inputs, storage dynamics, and land surface–atmosphere interactions. This confirms the model’s ability to reflect not only statistical relationships but also meaningful hydrological behavior.

Ref. [88] applied explainable transfer learning to predict subsurface soil moisture in the Yellow River Basin. Their findings, based on SHAP values, revealed that as the model was transferred from arid to humid regions, the influence of evapotranspiration-related factors declined significantly. Additionally, the effect of precipitation no longer increased with its amount, while the influence of SSM became more prominent. Also, ref. [131] conducted a comparative analysis of machine learning models for soil moisture estimation using high-resolution remote sensing data in the ShanDian River basin. Their SHAP-based analysis revealed that elevation was the most influential feature across all models, exerting a negative impact on soil moisture—indicating that higher elevations correspond to lower soil moisture levels. This aligns with the well-known pattern in which elevation influences precipitation distribution and runoff generation, ultimately reducing soil moisture at higher altitudes.

Figure 8 displays the SHAP waterfall plots for four representative catchments (i.e., Gorgan, Mahabad, Hamun, and the Lut Desert). The analysis reveals distinct regional patterns. In Gorgan and Mahabad, precipitation, clay fraction, and distance from water bodies appear as the most influential features, all contributing significantly to SSM prediction. This aligns with their relatively wetter climates and proximity to major water bodies—the Caspian Sea near Gorgan and Lake Urmia near Mahabad. Conversely, in the Hamun and Lut Desert regions, distance from water bodies shows a strong negative contribution, reflecting their remoteness from surface water sources. Additionally, the clay content in these arid regions further suppresses SSM estimates. These SHAP-based insights emphasize the importance of both climatic inputs and static geographic variables in shaping regional SSM dynamics and enhance the interpretability of the model for hydrological assessments across diverse environments.

3.5. Cluster Analysis

The SHAP model yielded a large number of coefficients for the 609 studied catchments, presenting a challenge in terms of interpretation. Therefore, a methodology is needed to categorize the non-stationarity results of factors influencing SSM across seasons in Iran. In this study, cluster analysis was used to process SHAP coefficients. Following the clustering process, we observed a discernible resemblance in the factors affecting catchments grouped within the same category. When comparing these categories, the results show substantial distinctions in the various factors across catchments. In SPSS software (v 24), the two-step clustering algorithm automatically assesses the suitability of segmenting catchments into multiple categories by considering lower BIC values. Once the optimal number of clusters is determined, more precise insights into the distinctions among categories can be obtained. The results of clustering allow us to understand how each factor influences SSM. In this study, we assessed the clustering quality using the BIC to ensure the reliability of the results. For each season, BIC values were calculated for a range of cluster numbers (from 1 to 10). The lowest BIC consistently occurred at five clusters, with only minimal changes observed beyond this point. Consequently, SPSS automatically identified five clusters as the optimal solution across all four seasons. The corresponding BIC values are presented in

Table 3. BIC values for different numbers of clusters across seasons. Boldface indicates the optimal BIC value.

Number of Clusters	BIC
	Winter	Spring	Summer	Autumn
1	2607	2607	2607	2607
2	2173	1952	2012	2059
3	1959	1706	1753	1707
4	1803	1594	1579	1514
5	1549	1432	1320	1268
6	1546	1428	1330	1260
7	1549	1432	1345	1291
8	1573	1440	1384	1328
9	1602	1453	1425	1378
10	1638	1480	1467	1428

.

Table 4. Clustering results of SHAP values for different seasons. P: precipitation, PET: potential evapotranspiration, SR: solar radiation, NDVI: normalized difference vegetation index, DWB: distance from water bodies, CF: clay fraction, OMF: organic matter fraction, and E: elevation. The SHAP values are multiplied by 1000.

Season	Class	P	DWB	SR	CF	PET	E
Winter	1	34.79	0.61	4.05	5.63	2.67	−0.21
	2	−36.45	−13.00	2.87	−0.59	−0.45	0.02
	3	3.19	19.96	−2.10	2.19	−0.81	1.42
	4	2.57	−14.18	−7.06	1.45	−1.73	−1.58
	5	−47.93	3.53	−6.71	−12.87	−2.06	1.11
		NDVI	DWB	CF	OMF	P	SR
Spring	1	37.69	1.15	5.63	8.79	4.71	1.82
	2	12.72	13.93	6.31	−2.80	−1.18	1.85
	3	−14.65	−3.57	3.69	−4.19	−4.17	−3.42
	4	−1.35	−8.68	4.55	−4.46	3.58	2.02
	5	−35.12	−1.56	−17.79	−3.20	−5.07	−1.97
		DWB	CF	PET	NDVI	OMF	E
Summer	1	6.92	4.72	18.13	1.77	1.26	0.08
	2	10.63	−8.47	1.32	−0.95	−1.14	0.25
	3	−12.85	3.59	−3.01	1.96	1.32	−0.57
	4	−13.4	−5.15	−2.50	−2.14	−1.64	−0.62
	5	8.47	3.83	−3.15	0.38	0.24	0.90
		P	DWB	PET	SR	E	CF
Autumn	1	−8.40	−5.55	−2.73	−1.03	−2.25	0.68
	2	12.68	−0.15	−4.29	0.07	0.04	0.47
	3	−8.54	3.59	−1.83	0.34	1.96	0.54
	4	−20.97	−3.00	−3.21	−2.15	0.55	−3.35
	5	11.52	3.92	13.74	4.62	0.26	0.49

presents the average SHAP values of the six primary factors for every season and cluster. Figure 9 displays the spatial distribution of the clustering results for various catchments. Features with significant absolute values carry a more pronounced influence on the SSM. These figures serve as valuable tools for comprehending the spatial distribution of each catchment type and identifying the factors that exert the most substantial influence on each catchment’s SSM.

The clustering results for winter are shown in Figure 9a,e. The first catchment type is located mainly in northwestern and western Iran. In these catchments, the factors that most significantly affect SSM are, in respective order, precipitation, clay fraction, solar radiation, potential evapotranspiration, distance from water bodies, and elevation, as shown in Figure 9a,e and Table 4. Ref. [132] investigated the spatial and temporal variations in SSM with respect to topographic and meteorological factors in Ardabil province, which falls into the first catchment category of this study. Their research underscored a significant correlation between SSM and variables such as precipitation. The second catchment type is primarily found in central, eastern, and northeastern Iran. The most influential factors affecting SSM in these catchments are precipitation, distance from water bodies, solar radiation, clay fraction, potential evapotranspiration, and elevation. The third catchment category is predominantly located in southern and northern Iran, in proximity to the Persian Gulf and the Caspian Sea. Within this catchment type, the most pivotal factor affecting SSM is the distance from water bodies, with precipitation, clay fraction, solar radiation, elevation, and potential evapotranspiration following in significance. In the fourth catchment category, similar to the third catchment type, the primary factor influencing SSM is the distance from water bodies, albeit with an inverse effect. This is followed by solar radiation, precipitation, potential evapotranspiration, elevation, and clay fraction (Table 4).

The fifth catchment category is distributed in southeastern Iran. In order of influence, the most significant factors on SSM are precipitation, clay fraction, solar radiation, distance from water bodies, potential evapotranspiration, and elevation. The clustering results for the spring season are shown in Figure 9b,f. In this season, vegetation and organic matter fraction have a greater impact on SSM than in winter. The first catchment type is located mainly in northern, northwestern, and western Iran. Within these catchments, the factors exerting the most substantial influence on SSM are, in order of impact, NDVI, organic matter fraction, clay fraction, precipitation, solar radiation, and distance from water bodies, as demonstrated in Figure 9 and expounded upon in Table 4.

The second catchment type is mainly distributed in southern, southwestern, and northern Iran. The most influential factors affecting SSM are distance from water bodies, NDVI, clay fraction, organic matter fraction, solar radiation, and precipitation. In the spring of 2020, Ref. [133] collected 394 surface soil samples in Golestan province located in northern Iran, a region categorized within the second catchment type of this study. The findings of their research demonstrated a strong correlation between NDVI and SSM. The third catchment category is primarily located in southern and central Iran. Within this catchment category, the most dominant factor affecting SSM is the NDVI, with organic matter fraction, precipitation, clay fraction, distance from water bodies, and solar radiation subsequently ranking in significance. In the fourth catchment category, the predominant factor influencing SSM is the distance from water bodies. This is followed by clay fraction, organic matter fraction, precipitation, solar radiation, and NDVI (Table 4). The fifth catchment category is distributed in southeastern and eastern Iran. The most significant factors influencing SSM are NDVI, clay fraction, precipitation, organic matter fraction, solar radiation, and distance from water bodies.

Figure 9c,g and Table 4 indicate the clustering results for the summer season. During this season, the influence of distance from water bodies and clay fraction on SSM is more pronounced compared to other seasons. The first catchment type is located in northern, northwestern, and northeastern Iran around the Caspian Sea. In these catchments, the most significant influences on SSM are associated with potential evapotranspiration, proximity to water bodies, clay fraction, NDVI, organic matter fraction, and elevation (Figure 9c,g and Table 4). The second catchment type is mainly located in southern and southeastern Iran around the Persian Gulf and the Gulf of Oman. In this category, distance from water bodies is the most important factor, followed by clay fraction, potential evapotranspiration, organic matter fraction, NDVI, and elevation. In the third and fourth catchment categories, the hierarchy of influencing factors closely mirrors that of the second catchment type, except that NDVI exerts a more significant influence than the organic matter fraction. The fifth catchment category is primarily located in southeastern and eastern Iran. The most influential factors are distance from water bodies, clay fraction, potential evapotranspiration, elevation, NDVI, and organic matter fraction (Table 4).

The clustering results for autumn are shown in Figure 9d,h and Table 4. As precipitation begins in Iran, it becomes the predominant factor affecting SSM in most classes. The first catchment category is primarily located in central and eastern Iran. Within these catchments, the factors exerting the most substantial influence on SSM are precipitation, distance from water bodies, potential evapotranspiration, elevation, solar radiation, and clay fraction, as demonstrated in Figure 9d,h and described in Table 4. The second catchment type is mainly distributed in southwestern and western Iran. The factors that most significantly affect SSM are precipitation, potential evapotranspiration, clay fraction, distance from water bodies, solar radiation, and elevation. The third catchment category is primarily located in southern (near the Persian Gulf) and northeastern Iran. In this catchment type, precipitation is the most influential factor on SSM, followed by the distance from water bodies, elevation, potential evapotranspiration, clay fraction, and solar radiation. In the fourth catchment category, the primary influencing factors on SSM are precipitation, clay fraction, potential evapotranspiration, distance from water bodies, solar radiation, and elevation (Table 4). Finally, the fifth catchment category is distributed in northern Iran near the Caspian Sea. The most significant factors on SSM are potential evapotranspiration, precipitation, distance from water bodies, solar radiation, clay fraction, and elevation.

4. Conclusions

In this study, we unraveled the impact of the spatial non-stationarity of the critical environmental factors influencing SSM. To this end, we have introduced a framework that combines the SHAP technique with a two-step clustering analysis to provide spatial interpretations for machine learning models such as RF. Given the limited availability of reliable in situ SSM data in Iran, we initially validated the global SSM datasets (SMAP, MERRA-2, and CFSv2) at a depth of 0–5 cm, against available in situ measurements. While overfitting, sensitivity to hyperparameters, and the presence of correlated covariates are common concerns in machine learning applications, we implemented several strategies to address these challenges. Aggregating the data at a seasonal scale helped reduce noise and the risk of overfitting. To further improve model robustness, we used k-fold cross-validation for hyperparameter tuning and evaluated RF performance using multiple statistical metrics (R², RMSE, MAE). Additionally, we carefully examined the predictor variables to minimize the impact of multicollinearity. These measures collectively enhanced the reliability and generalizability of our results. The main conclusions are as follows:

(1) Results of the validation analysis demonstrated that among the datasets, SMAP exhibited the highest median correlation and the lowest median RMSE compared to in situ stations. Hence, it is recommended for applications such as hydrological modeling, water resources management, and drought monitoring in Iran, where SSM data are scarce.

(2) Investigation of SSM across different land cover types in Iran revealed significant variations. Specifically, forests and bare/sparse vegetation regions exhibited the highest and lowest SSM with median values of 0.180 m³/m³ and 0.093 m³/m³, respectively. These findings highlight the importance of understanding the spatial distribution of SSM across different land cover types, which can have implications for various environmental and ecological processes.

(3) The results indicated that the RF model can produce SSM estimates with R² values of 0.89, 0.83, 0.70, and 0.75 for the winter, spring, summer, and autumn seasons, respectively. Corresponding RMSE values are 0.076, 0.081, 0.098, and 0.061 m³/m³, while MAE values are 0.058, 0.060, 0.076, and 0.047 m³/m³, demonstrating consistent performance across seasons. These findings highlight the importance of seasonal investigation of SSM. Due to limited SSM availability in the dry season, machine learning models exhibit reduced prediction accuracy.

(4) The findings of the SHAP model and two-step cluster analysis indicated that SSM in winter and autumn is primarily influenced by climatic factors. In contrast, SSM in spring and summer is largely controlled by vegetation and soil characteristics. These findings highlight the dynamic nature of SSM and how it is influenced by different environmental factors across seasons. Understanding these seasonal variations is essential for effective SSM management and prediction in diverse regions.