Estimating Soil Moisture Using Multimodal Remote Sensing and Transfer Optimization Techniques

Highlights

What are the main findings?

• A multimodal fusion framework integrating SAR, optical, topographic, and meteorological data achieved high-precision soil moisture estimation ($R^2$ = 0.8956), significantly outperforming single-modality methods.
• An intermediate fine-tuning strategy applied to a large dataset (10,571 images and 3772 samples) substantially enhanced the model’s generalization and transferability across diverse agro-ecological zones.

What are the implications of the main findings?

• The framework enables the generation of reliable, field-scale soil moisture maps, providing a practical tool for precision irrigation and site-specific water management to enhance drought resilience.
• This approach offers a scalable and robust solution for operational soil moisture monitoring in heterogeneous landscapes, supporting sustainable water resource management from farm to regional scales.

Abstract

Surface soil moisture (SSM) is essential for crop growth, irrigation management, and drought monitoring. However, conventional field-based measurements offer limited spatial and temporal coverage, making it difficult to capture environmental variability at scale. This study introduces a multimodal soil moisture estimation framework that combines synthetic aperture radar (SAR), optical imagery, vegetation indices, digital elevation models (DEM), meteorological data, and spatio-temporal metadata. To strengthen model performance and adaptability, an intermediate fine-tuning strategy is applied to two datasets comprising 10,571 images and 3772 samples. This approach improves generalization and transferability across regions. The framework is evaluated across diverse agro-ecological zones, including farmlands, alpine grasslands, and environmentally fragile areas, and benchmarked against single-modality methods. Results with RMSE 4.5834% and $R^2$ 0.8956 show consistently high accuracy and stability, enabling the production of reliable field-scale soil moisture maps. By addressing the spatial and temporal challenges of soil monitoring, this framework provides essential information for precision irrigation. It supports site-specific water management, promotes efficient water use, and enhances drought resilience at both farm and regional scales.

1. Introduction

Global water resources are under increasing pressure, driving the need for more precise and efficient agricultural water management practices [1,2]. In arid and semi-arid regions, accurate estimation of SSM is especially important for irrigation planning, drought monitoring, and crop yield forecasting [3,4,5].

For soil moisture measurement methods, they are mainly divided into traditional methods and methods based on remote sensing image inversion. Traditional in-situ methods, such as manual sampling and ground-based sensor networks, provide reliable measurements at specific sites but are constrained by high costs and limited spatial coverage [6,7]. Global monitoring efforts, including the International Soil Moisture Network (ISMN), reveal substantial observational gaps, particularly in regions like Africa and parts of South America [8]. These limitations highlight the need for scalable, cost-effective solutions for monitoring soil moisture over large areas. Remote sensing has emerged as a vital tool for filling these data gaps, offering synoptic and timely information at regional to global scales [9,10]. Optical sensors in the visible and near-infrared (VIS/NIR) range capture surface reflectance and perform well in bare or sparsely vegetated regions [11,12]. Thermal infrared (TIR) sensors infer SSM indirectly through thermal inertia, making them suitable for areas with strong diurnal temperature variations [13]. Microwave sensors are particularly effective due to their sensitivity to soil dielectric properties and their ability to penetrate vegetation and clouds, offering reliable observations in various weather conditions [14,15,16].

However, a single sensor is susceptible to environmental interference, often resulting in spatio-temporally inconsistent inversion outcomes, which makes it difficult to directly deploy in operational applications [17]. To overcome the limitations of single-sensor systems, recent advances have focused on integrating data from multiple sources [18,19,20]. Combining optical, radar, and UAV-based imagery allows models to leverage complementary information, improving accuracy and generalizability [21,22,23,24]. This multimodal fusion approach enhances the model’s ability to represent complex environmental conditions and leads to more precise soil moisture estimates [25,26,27,28], and delivers more reliable data products for agriculture [29,30]. However, despite these advances, current multimodal methods still face notable challenges. Multimodal methods often struggle with temporal consistency and regional transferability, limiting their use in long-term water planning. Integrating data from heterogeneous sources is also complicated by differences in spatial resolution and revisit frequency, making harmonization essential for accurate modeling.

In addition, soil moisture dynamics are further complicated by nonlinear interactions among soil properties, vegetation, and meteorological conditions. This complexity makes it difficult to design models that perform well across diverse environments. Machine learning and deep learning approaches have shown promise in addressing these challenges. They can model nonlinear relationships, process large datasets, and generate high-resolution soil moisture estimates [31,32,33]. However, their effectiveness is constrained by the scarcity of high-quality, spatially dense training data. To mitigate this limitation, intermediate task fine-tuning has emerged as a promising strategy. This involves pretraining model on large, related datasets to learn transferable features, followed by fine-tuning on smaller, task-specific datasets. This two-step approach improves model generalization and performance in data-scarce environments [34,35,36], and has proven effective in cross-modal applications where labeled data are limited [37,38].

Nevertheless, the transferability of soil moisture models across agro-ecological zones remains uncertain. Most fusion approaches have been developed and validated within single, relatively uniform regions, with limited testing in ecologically fragile or agriculturally important areas such as alpine ecosystems and intensively farmed zones [31,39]. Variability in climate, vegetation, soil texture, and hydrology introduces additional uncertainty, underscoring the need to evaluate how these factors affect model performance.

To bridge the gap between advanced remote sensing and actionable water management decisions, this study proposes a multimodal framework. The primary goal is to develop a robust tool that can directly support agricultural water management activities. The specific objectives are to: (1) Enhance soil moisture estimation accuracy in regions with limited ground-based observations, aiding real-time drought monitoring and irrigation decision-making. (2) Quantify the added value of integrating multimodal data in generating actionable insights beyond those offered by single-source models. (3) Test the framework across diverse agro-ecological zones to assess its robustness and potential for broad application, from intensively cultivated farmland to environmentally sensitive regions. (4) Achieve high-resolution soil moisture retrieval by leveraging the synergistic integration of multi-source satellite data, thereby overcoming the spatial resolution limitations of existing coarse-resolution products and enabling field-scale monitoring capabilities essential for precision agriculture and localized water resource management. The results of this study will provide valuable scientific insights and technical support for region-specific agricultural water management, drought monitoring, and ecological conservation, particularly enhancing soil moisture monitoring in complex environments.

2. Materials and Methods

2.1. Datasets

This study employs a multimodal dataset (see

Table 1. Multimodal Input Data.

Level	Type	Product	Resolution	Frequency	Band Information
PIXEL	Multispectral	Landsat-8	30 m	16 Days	B1-B11
	Multispectral	Sentinel-2	10–60 m	5 Days	B1-B12
	SAR	Sentinel-1	10 m	6 Days	VV, VH for ascending/descending orbit (C-band)
	SAR	Gaofen	1 m	N/A	VV, VH, HV, HH (L-band)
	DEM	Aster	30 m	N/A	elevation, slope
IMAGE	Vegetation	MODIS	500 m	4 Days	FPAR, LAI
	Meteorological Reanalysis	era5	11,132 m	1 Month/Day	temperature, precipitation
	Geoposition	N/A	N/A	N/A	cyclic encoding of latitude and longitude
	Time	N/A	N/A	N/A	cyclic encoding of month

) that integrates diverse sources of environmental information to capture the complex dynamics of soil moisture. To effectively manage the structural and functional differences among data types, inputs are organized into two complementary categories based on their spatial characteristics and their role in soil moisture estimation: pixel-level modalities and image-level modalities. This organizational framework not only facilitates systematic data preprocessing and alignment but also guides the design of the fusion architecture, ensuring that each data source contributes optimally to model performance.

Pixel-level modalities provide spatially explicit raster data where each pixel represents distinct surface properties that vary across the landscape. These data require precise geometric co-registration and are spatially aligned within a 1.28 km × 1.28 km window (128 × 128 pixels at 10 m resolution), with each pixel corresponding across sources within the fixed geographic extent. This window size was selected based on compatibility with Sentinel-1’s native 10 m resolution and computational efficiency for processing large datasets. This category includes synthetic aperture radar (SAR) imagery, multispectral optical data, and digital elevation models (DEM). SAR data, obtained from Sentinel-1 and Gaofen satellites, play a central role in directly sensing soil moisture through their sensitivity to soil dielectric properties and their ability to penetrate vegetation and clouds under various weather conditions. Multispectral imagery from Landsat-8 [40] and Sentinel-2 [41] complements SAR by capturing surface reflectance across visible, near-infrared, and shortwave infrared bands, enabling detailed characterization of vegetation cover, land use, and surface conditions that indirectly influence soil moisture distribution. Topographic features derived from DEM, including elevation and slope, are essential for modeling the terrain-driven redistribution of water through surface runoff and subsurface flow pathways.

Image-level modalities, in contrast, represent contextual environmental conditions or metadata that characterize the entire scene rather than individual pixels. These include spatially aggregated climate statistics, vegetation indicators, and scalar metadata such as geolocation and observation timing. Since these features are either spatially homogeneous across the study window or serve as scene-level attributes, they do not require pixel-level alignment and are incorporated through global pooling operations in the model architecture. Vegetation indices, specifically Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) derived from MODIS products [42], provide indirect indicators of soil moisture status through their linkage to vegetation health and canopy structure. Meteorological data [43,44] from the ERA5 reanalysis product capture atmospheric forcing on soil moisture dynamics, with daily and monthly records of temperature and precipitation enabling the model to account for both immediate and lagged meteorological influences. Additionally, geospatial coordinates (latitude and longitude) and temporal information (month of observation) are encoded in a cyclic manner to help the model recognize seasonal trends and geographic variability in soil moisture patterns.

The distinction between pixel-level and image-level modalities reflects their different contributions to soil moisture estimation. Pixel-level data capture fine-scale spatial heterogeneity in surface and subsurface properties, which is critical for generating high-resolution soil moisture maps suitable for field-scale applications. Image-level data, meanwhile, provide the broader environmental context necessary for interpreting local observations within regional climatic and phenological frameworks. By integrating these complementary information sources through an attention-based fusion mechanism, the framework leverages the strengths of each modality while mitigating their individual limitations. A comprehensive summary of each data type, including spatial resolution, temporal frequency, and band information, is provided in Table 1. The location of the aforementioned open-source data has been indicated in Appendix A.

2.1.1. SSM from a Large-Scale Dataset

The first soil moisture labels used in this study are derived from two key sources: WoSIS [45] and SMAP [46]. These datasets have fundamentally different characteristics: WoSIS provides static, high-resolution (250 m) soil property measurements derived from global field observations at six depth intervals, while SMAP delivers dynamic, daily global soil moisture estimates at coarse resolution (9 km) from the SPL3SMP-E Level 3 product. WoSIS captures fine-scale spatial heterogeneity in soil physical properties but lacks temporal dynamics, whereas SMAP captures temporal variability in surface soil moisture but at insufficient spatial resolution for field-scale applications.

To combine the strengths of both datasets, we apply the NNDiffuse image fusion algorithm [47], treating WoSIS as a high-resolution (panchromatic) input and SMAP as a low-resolution (multispectral) input. This output is then upsampled to align with the spatial resolution of Sentinel-1 imagery, producing label maps with consistent resolution across modalities. The fusion process is carried out using ENVI (v5.6) software. For convenience, the fused dataset is hereafter referred to as the SMAP-WoSIS Fusion (SWF).

The fusion process proceeds as follows: For each location and date in our dataset, we extract the corresponding SMAP observation temporally aligned with the Sentinel-1 acquisition date; The static WoSIS data provides the spatial structure that guides the downscaling of SMAP’s temporal signal; The NNDiffuse algorithm is implemented in ENVI software to generate the fused product; The output is upsampled to 10 m resolution to align with Sentinel-1’s spatial grid. This fusion successfully combines SMAP’s temporal fidelity with WoSIS’s spatial precision, while eliminating physically unrealistic moisture assignments over water bodies present in the original coarse SMAP data.

We acknowledge that this fusion introduces label uncertainty, as the resulting SWF estimates reflect regional-scale patterns rather than point-level ground truth. To address this limitation, we implement a two-stage training strategy: intermediate pretraining on the large-scale SWF dataset to learn general soil moisture patterns, followed by fine-tuning on high-quality in-situ measurements to achieve site-specific accuracy.

Input features include all modalities listed in Table 1, except Gaofen data, which is excluded due to limited global availability. The final dataset consists of 10,571 co-registered samples. The large-scale dataset spans four years from 2017 to 2020, with samples distributed across all 12 months to capture seasonal variability in soil moisture dynamics. The temporal distribution shows relatively balanced monthly coverage, with slightly higher representation during the growing season (July–August). This multi-year, multi-season sampling strategy ensures that the model learns soil moisture patterns across diverse climatic conditions, phenological stages, and meteorological forcing scenarios. The temporal alignment uses Sentinel-1 SAR acquisition dates as the reference timestamp. For all other modalities, we search for the nearest available observation within a temporal window before and after the SAR acquisition date. If no valid observation is found within the search window (due to cloud cover for optical data or data gaps), the corresponding modality is marked as missing and handled by the model’s missing data mechanism. The geographic distribution of these samples is illustrated in Figure 1.

2.1.2. SSM from a Small-Scale Dataset

The second category of soil moisture labels used in this study is derived from in-situ measurements collected across several representative regions in China. These include Wangkui County, the Tibetan Plateau, and the Heihe River Basin. The geographic distribution of these sampling sites is shown in Figure 2. This dataset spans a variety of climate zones and land surface types, which offers ground-truth observations.

In the Wangkui region, data were collected in July 2021 using manual sampling techniques. Soil samples were weighed before and after oven-drying to calculate the GWC. For the Tibetan Plateau, soil moisture data were obtained from several long-term observation networks [48], including: Naqu network, Maqu network, and additional sites such as Ali and Pali. These networks cover a wide range of meteorological and hydrological environments across the plateau, with observation periods extending from July 2010 to August 2016. The Heihe Basin dataset was compiled from two separate field projects [49]: The first includes gravimetric measurements from the upper reaches of the basin, collected between July 2013 and July 2014. The second contains time-series records from the Babao River sub-basin, covering time from July 2013 to December 2017.

All in-situ soil moisture measurements were collected at depths ranging from 0–5 cm below the surface. For the Tibetan Plateau and Heihe River Basin datasets, the long-term observation networks provide soil moisture measurements at multiple depths (5 cm, 10 cm, 20 cm, 40 cm, and 50 cm layers), and we extracted the 5 cm depth measurements as ground truth labels, as this depth is commonly used in agricultural and hydrological applications and represents the near-surface soil moisture layer relevant for vegetation water uptake and evapotranspiration processes. For the Wangkui dataset, soil samples were collected at four depths (2 cm, 5 cm, 10 cm, and 20 cm layers), and we similarly selected the 5 cm depth measurements for consistency.

Soil moisture values used in this study are calculated based on standard definitions, GWC (gravimetric water content) and VWC (volumetric water content). By introducing the soil bulk density parameter ${\rho }_{\mathrm{soil}}$, GWC can be converted to VWC as:

$$\begin{array}{c}\hfill VWC=GWC\times {\rho }_{soil}\times 100\%\end{array}$$

(1)

For consistency and comparability across sites and measurement techniques, all soil moisture observations were converted to VWC. After strict quality control and data cleaning, the final in-situ dataset contains 3772 valid records. As with the large-scale dataset, these VWC values are used as supervised labels, and the input features include the multimodal remote sensing and meteorological variables listed in Table 1. One notable distinction in the small-scale dataset is that SAR data for the Wangkui region were acquired from the Gaofen satellite.

2.2. Data Processing

To ensure consistency in spatial resolution, temporal alignment and data format across all sources, a standardized preprocessing workflow was implemented before model development. Each data type was handled according to its specific characteristics, as outlined below:

• Sentinel-1 served as the reference dataset, setting the standard for observation dates, spatial resolution and coordinate reference system. All other data sources were aligned to match this baseline. Sentinel-1 imagery is acquired in ascending and descending orbits, with each viewing direction capturing two polarization channels, resulting in four bands per observation.
• Gaofen imagery, which lacks built-in preprocessing, was radiometrically calibrated using the method proposed by [50]. This step was essential for extracting reliable radar backscatter intensities.
• Sentinel-2 offers two product levels: Level-1C and Level-2A. Although Level-2A data provide improved atmospheric correction, they were not always available for the entire study period. Consequently, these two sources are used in a complementary fashion to enhance spectral and spatial representation in remote sensing analyses.
• Landsat-8 provides both Level-1 and Level-2 data, too. To ensure adequate coverage while maintaining data quality, we also prioritized Level-2 data and used Level-1 data where necessary.
• DEM data were obtained from the ASTER GDEM dataset. In addition to elevation, terrain slope was derived to incorporate topographic features relevant to soil moisture distribution.
• ERA5 climate reanalysis data were aligned with Sentinel-1 acquisition dates. Daily and monthly averages of temperature and precipitation were extracted to capture short- and mid-term atmospheric influences on soil moisture.
• Time: seasonality was represented by encoding the acquisition month of each Sentinel-1 image using a cyclic transformation. This allowed the model to account for seasonal patterns affecting SSM.
• Geolocation information, specifically the latitude and longitude of the image center, was also encoded cyclically. This enabled the model to learn spatial patterns and regional dependencies.

All pixel-level modalities are resampled to a target resolution of 10 m using bilinear interpolation [51], and reprojected to match the spatial reference system of Sentinel-1. This ensured consistent spatial alignment across all data sources. Temporal alignment was also prioritized: for each observation, data from all modalities were synchronized as closely as possible to Sentinel-1 acquisition dates. In particular, for ERA5 and optical datasets, observations closest in time to Sentinel-1 captures were selected to enhance inter-modal consistency.

During preprocessing, invalid or missing values—caused by factors such as cloud cover, sensor noise, or acquisition failure—were systematically identified. These error indicators vary by modality (e.g., 0 in Sentinel-2, –inf in Sentinel-1). A binary validity mask was generated for each image, where a value of 1 denotes valid data and 0 represents missing or corrupted pixels. These masks were incorporated as additional model inputs during training, ensuring that only valid pixels contributed to the loss calculation and gradient updates.

To manage large-scale data acquisition and preprocessing efficiently, this study developed an automated system combining Python (v3.8) and the GEE API. For the large-scale dataset, the workflow begins with Sentinel-1 metadata, which serves as the reference for retrieving corresponding remote sensing observations and lower-resolution soil moisture labels. Anchoring the pipeline to Sentinel-1 ensures spatial consistency between inputs and labels. For the smaller in-situ dataset, the workflow starts from ground measurement metadata, including timestamp and coordinates. These are indexed in JSON files to guide the retrieval of multimodal satellite inputs that are spatially and temporally aligned with each soil sampling event.

2.3. Methods

We adopt a two-stage intermediate task fine-tuning strategy to train the soil moisture estimation model. Additionally, a multimodal fusion mechanism is integrated to effectively combine complementary information from diverse input sources, improving the model’s ability to capture complex relationships across modalities.

This research adopts ConvNeXt v2 as the backbone architecture for soil moisture estimation [52]. Given the sparse and noisy nature of remote sensing data, such as invalid regions due to cloud cover or sensor gaps, the model uses submanifold sparse convolution, a core component of ConvNeXt v2. Unlike standard dense convolutions, sparse convolutions operate only on valid pixels (i.e., where the validity mask equals 1), eliminating unnecessary computation in empty regions.

To adapt ConvNeXt V2 for regression rather than classification, architectural modifications are applied as follows (illustrated in Figure 3):

• The original classification head is replaced with a single-channel linear output layer to predict soil moisture for each pixel.
• The loss function is changed from cross-entropy to Mean Absolute Error, which better suits the continuous nature of soil moisture values.
• U-Net-style skip connections [53] are added between encoder and decoder stages to preserve spatial details. Feature maps are concatenated channel-wise to retain fine-grained information during upsampling.

Before entering the backbone network, input data from multiple modalities are fused to capture complementary information. For image-level modalities, features are concatenated into a unified 1D vector. For pixel-level modalities, we apply an attention-based fusion strategy using the Squeeze-and-Excitation (SE) module [54] to adaptively weight channels by importance. The fusion process proceeds as follows:

• Global average pooling is applied to each feature map $X\in {\mathbb{R}}^{B\times C\times H\times W}$ to obtain a channel-wise descriptor:

  $$\begin{array}{c}\hfill z_c={\displaystyle \frac{1}{H\times W}}\sum _{i=1}^H\sum _{j=1}^W{X}_c(i,j)\end{array}$$

  (2)
• The descriptor z is passed through a two-layer MLP ($FC1$ and $FC2$) with ReLU and sigmoid activations to produce the attention weights $\omega$:

  $$\begin{array}{c}\hfill \omega =\sigma \left(W_2·ReLU\left(W_1·z\right)\right)\end{array}$$

  (3)
• Each channel is reweighted by its corresponding attention score:

  $$\begin{array}{c}\hfill X^{\prime }c=Xc·\omega \end{array}$$

  (4)
• A $1\times 1$ convolution is applied to reduce the reweighted feature map to a single-channel representation per modality, allowing unified downstream processing. The complete multimodal fusion workflow is illustrated in Figure 4.

Once single-channel representations are extracted for all input modalities, they are jointly processed within the ConvNeXt V2 backbone to enable integrated feature learning across modalities. This study employs a progressive transfer learning paradigm:

• Pretraining stage: The model is initialized with ConvNeXt V2 weights pretrained on the MMEarth dataset [55], which is specifically designed for multimodal Earth observation tasks and has demonstrated strong performance in various settings.
• Intermediate training stage: The model is further trained on a large-scale and automatically curated soil moisture dataset using SWF labels. This global dataset enables the model to learn generalizable relationships between multimodal remote sensing inputs and soil moisture dynamics.
• Fine-tuning stage: Finally, the model is fine-tuned on a high-quality, small-sample in-situ dataset based on field measurements. The higher label fidelity in this stage enhances the model’s accuracy in specific geographic regions and supports better adaptation to real-world conditions.

While the SWF and in-situ datasets differ in label origin, they are consistent in terms of input modalities, spatial resolution, and preprocessing procedures.

3. Results

This study investigates the effectiveness of multimodal input for soil moisture estimation through a series of controlled experiments. A total of eight model configurations were designed to evaluate the influence of data fusion strategies and architectural choices on performance. The configurations include unimodal SAR input (a), pixel-level multimodal fusion (b), and a combined pixel- and image-level fusion strategy (c). Additionally, a group (d) was included in the experiments, which did not use the intermediate fine-tuning strategy for training. Due to convergence issues with the dataset, this group only includes results from training with SAR data.

Each group has two sub-variants: one with skip connections for better spatial detail recovery (’1’), and one without to assess their impact (’2’). This design allows for evaluating not only the contribution of modalities but also the architectural effect of skip connections. During preliminary trials, we observed that under small-sample training conditions, only SAR-based unimodal models (Group a) demonstrated consistent convergence. In contrast, models with multiple input modalities (Groups b and c) often encountered feature conflicts, which impeded effective training and model stability. These conflicts likely stem from inconsistencies across modalities and limited training data, which constrain the model’s ability to learn coherent joint representations. As a result, only SAR-based architectures were employed during the intermediate fine-tuning stage (Group d), where training stability and convergence were prioritized. This decision ensures a more reliable learning process while leveraging the strong generalization capabilities of SAR data.

To comprehensively evaluate model performance in predicting soil moisture, we employ four widely used statistical metrics: root mean square error (RMSE), mean absolute error (MAE), coefficient of determination ($R^2$), and unbiased root mean square error (ubRMSE).

3.1. Intermediate Task Fine-Tuning

This stage incorporates channel-wise fusion to integrate complementary features from multiple remote sensing modalities. As illustrated in Figure 5, the fusion process enhances spatial consistency across inputs while preserving distinctive information from each source.

As part of the intermediate task fine-tuning strategy, a fusion process was first applied to soil moisture images. The results, shown in Figure 6, demonstrate that the fused image retains the spectral characteristics of the original SMAP data while significantly improving spatial resolution and visual clarity. Prior to fusion, the low-resolution SMAP images cover broad areas, including non-soil surfaces such as lakes and rivers. This leads to the assignment of soil moisture values over water bodies, which is physically inaccurate. In contrast, the WoSIS dataset, with its finer spatial granularity, better delineates soil-covered regions and excludes water surfaces, offering a more realistic depiction of soil distribution. The NNDiffuse algorithm facilitates this enhancement by applying a nonlinear diffusion mechanism. This process improves the fused image’s spatial precision without compromising spectral integrity. Furthermore, a known limitation of the algorithm is that it does not impute values in areas with missing data. As a result, any blank regions present in the WoSIS image remain unprocessed in the final fused output, and are also reflected in the downsampled SMAP image.

During the initial training phase on the large-scale dataset, the model’s performance across different configurations is summarized in

Table 2. Test Results Using SWF Label.

Group	${\mathit{R}}^2$	RMSE (%)	MAE (%)	ubRMSE (%)
a1	0.1206	13.333	10.975	11.153
a2	0.1082	13.427	10.862	12.328
b1	0.2462	12.344	10.078	11.145
b2	0.2106	12.633	10.285	11.462
c1	0.3072	11.749	9.442	11.295
c2	0.2941	11.946	9.6159	11.327

. The best-performing model achieved an $R^2$ of 0.3072 and an RMSE of 11.749%, representing more than an 150% improvement over the SAR-only baseline (Group a). The addition of skip connections consistently improved predictive performance across all architectures. The effect was most pronounced in Group b (pixel-level multimodal fusion), where $R^2$ increased by 16.9% with skip connections enabled. However, performance gains began to plateau as more modalities were added. From Group a to Group b, $R^2$ improved by 0.1256, while the transition from Group b to Group c yielded a smaller increase of just 0.0610. This trend highlights the need to balance computational overhead with diminishing returns when incorporating additional data sources in practical applications. Despite the improvements, the optimal configuration still exhibited an ubRMSE of 11. 295%, indicating the presence of residual error.

3.2. Soil Moisture Estimation Results

To further improve model accuracy and generalizability, this stage incorporates in-situ soil moisture measurements. Figure 7 displays the prediction results, while the corresponding quantitative metrics are summarized in

Table 3. Test Results Using In-situ Data.

Group	${\mathit{R}}^2$	RMSE (%)	MAE (%)	ubRMSE (%)
a1	0.6556	8.3248	6.6762	7.0931
a2	0.6024	8.9441	7.1550	6.8219
b1	0.8454	5.5781	4.5622	5.5780
b2	0.8329	5.7994	4.7828	5.7616
c1	0.8956	4.5834	3.8412	4.4282
c2	0.8701	5.1133	4.3475	5.0135
d1	0.4529	10.4920	8.4348	9.9923
d2	0.3989	10.9971	8.8655	10.7433

. The experimental findings confirm that the inclusion of skip connections consistently enhances model performance, particularly in terms of RMSE and $R^2$, by preserving spatial features and improving gradient flow. As more data modalities are integrated, the model continues to improve across all evaluation metrics. Comparing Group a1 (SAR-only input) to Group b1 (SAR combined with other pixel-level modalities), results show significant gains: $R^2$ increases by approximately 0.20 and RMSE decreases by 2.8%; Comparing Group b1 and Group c1, we find: a further 0.05 increase in $R^2$ and RMSE reduced by 1.0%.

A comparison between the intermediate fine-tuning stage (see Table 2) and the final fine-tuning stage (see Table 3) reveals substantial performance improvements. For the same model architecture, the $R^2$ value in Group a2 increased by up to 456.7%, highlighting the critical importance of high-quality, accurate labels in soil moisture retrieval tasks. The benefits of multimodal remain consistent across training stages. Specifically, the improvement from Group a1 to Group c1 during final fine-tuning yields a $\Delta R^2=0.2400$, demonstrating the effectiveness of integrating diverse data sources in enhancing model generalization and predictive accuracy. In addition, detailed error analysis confirms that fine-tuning with in-situ data significantly reduces ubRMSE by 36.4% to 60.8%, effectively mitigating the systematic bias introduced during large-scale pretraining. Both RMSE and MAE also show consistent reductions across configurations, further affirming the utility of in-situ supervision in refining model outputs. While the Group d experiments, conducted under data-limited conditions, exhibited an overall drop in performance, the presence of skip connections continued to provide noticeable benefits. This reinforces the robustness of the intermediate task fine-tuning strategy, which helps preserve generalization capability even when training data is scarce.

3.3. Accuracy of Soil Moisture Estimates over Different Soil Types

This research categorizes in-situ soils into three types based on water sources and spatial distribution patterns: Wangkui, Heihe, and Tibet. First, when examining the performance of individual regions, the error distribution in Wangkui is notably concentrated, with the boxplot closely aligned with the zero-error line. In contrast, the error distribution in Tibet falls between the Wangkui and Heihe regions, with a slightly larger box range. On the other hand, the Heihe region exhibits much larger errors, with a wider box and extended upper whisker range. This indicates a dispersed error distribution with numerous high-error outliers.

Analysis of the three figures reveals a clear trend: as additional modalities are incorporated, errors gradually decrease, particularly in Heihe, where error values decline significantly. In Tibet, the inclusion of modalities narrows the error range. Numerically, in the Heihe region, the error (represented by MAE) decreases from 0.1004 to 0.0592 and further to 0.0509 as modalities are added. Similarly, in Tibet and Wangkui, the MAE values are 0.0701 → 0.0515 → 0.0424 and 0.0416 → 0.0403 → 0.0351, respectively. Moreover, the addition of modalities not only reduces errors but also helps identify and correct outliers. In Heihe, although errors remain substantial, the number of outliers decreases significantly, and the error distribution becomes more concentrated. This suggests that modality integration improves error estimation in regions with complex water sources, helping to reduce retrieval errors despite inherent spatial variability.

4. Discussion

4.1. Contribution of Intermediate Task Fine-Tuning Strategy

Before fine-tuning, the retrieval performance of SWF-derived soil moisture was evaluated, and the results indicated relatively low accuracy. After incorporating the in-situ dataset for fine-tuning, the multimodal model achieved a substantial improvement in accuracy (Figure 7). Compared with the findings of [56], the performance of our method was lower before fine-tuning but surpassed theirs after fine-tuning with in-situ measurements. In their study, the reported $R^2$ values ranged from 0.49 to 0.7056, with RMSE values between 3.1% and 5.0%. In contrast, our multimodal approach achieved an $R^2$ of 0.8956 and an RMSE 4.58%. These results highlight the critical role of ground-based measurements in model training, while also underscoring the limitations of relying solely on downscaled remote sensing data for soil moisture retrieval.

Prior to incorporating in-situ data, the network exhibited suboptimal training performance. One contributing factor is the quality of the training labels [57]. SWF-derived soil moisture estimates reflect regional-scale averages rather than localized point-level ground truth. Their relatively coarse spatial resolution likely introduced uncertainty during model training, reducing the network’s ability to capture fine-grained, site-specific soil moisture variability. This limitation constrains the overall predictive accuracy and demonstrates why in situ calibration is indispensable for improving retrieval performance.

4.2. Role of Multimodal Approach

The Group a experiment, which relies solely on SAR data, highlights the unique ability of SAR to penetrate through vegetation and soil layers, providing valuable insights into subsurface moisture. However, despite this penetrating capability, SAR has limited effectiveness in capturing detailed surface information, particularly in regions with diverse landforms or land use types, where surface characteristics significantly influence soil moisture dynamics [58]. Moreover, SAR backscatter is sensitive to multiple confounding factors, such as soil type and topography, which further constrains its reliability when used in isolation. The Group b experiment demonstrates that integrating optical and topographic data with SAR substantially enhances model performance. Optical imagery contributes rich surface-level spectral information, capturing vegetation cover, water bodies, and land use, while topographic data provides geometric attributes such as elevation, slope, and aspect, which are critical for understanding the movement and retention of water across the landscape [59]. The inclusion of these modalities leads to a marked improvement in estimation accuracy. The Group c experiment further advances performance by introducing pixel-level modalities in addition to image-level features.These contextual inputs enrich the model with environmental background information that aids the interpretation of soil moisture. Meteorological variables allow the model to adjust predictions based on weather dynamics [60], while temporal indices help capture seasonal variability in soil moisture patterns [61]. Vegetation attributes also play a vital role in describing how soil retains or releases water [62], offering a more complete understanding of moisture dynamics.

Overall, these findings underscore the importance of environmental context in modeling the spatio-temporal variability of soil moisture, particularly in heterogeneous or seasonally dynamic regions. The complementary nature of multimodal inputs strengthens model robustness and adaptability across diverse landscapes and climatic conditions. Nevertheless, incorporating an excessive number of modalities can increase computational costs and introduce potential interference among modalities, sometimes leading to fluctuations in training loss.

4.3. Underlying Causes of Soil Type Differences

Different soil types exert a significant influence on the accuracy of soil moisture inversion due to their distinct formation mechanisms and hydrological characteristics [63]. However, relatively few studies have systematically investigated the role of soil type in soil moisture estimation [56]. From the overall results (Figure 8), the three soil types all achieved satisfactory retrieval accuracy, but with notable differences. The Wangkui soils, primarily dependent on natural precipitation and supplemental irrigation, exhibit relatively uniform spatial distribution. This stability in soil moisture dynamics allows the model to capture regional patterns effectively. In contrast, the Heihe soils are jointly influenced by river irrigation and groundwater [64], leading to pronounced spatial heterogeneity and high sensitivity to both natural and anthropogenic disturbances. This highlights the limitations of existing models in capturing complex hydrological processes and spatial variability. The Tibet soils, replenished by precipitation, snowmelt, and thawing permafrost, represent an environment with strong heterogeneity [65]. Notably, when pixel-level and image-level modalities were jointly introduced, retrieval accuracy in Tibet improved significantly, suggesting that multimodal feature integration can mitigate uncertainties caused by complex hydrological conditions.

A closer comparison of different input strategies (Figure 8a–c) further illustrates these distinctions. In Wangkui, where hydrological processes are simple and spatial distributions uniform, the MAE remained consistently low across all three input settings. The box plots displayed few outliers and narrow distributions, indicating that SAR-only input was sufficient to achieve reliable accuracy. Additional pixel- or image-level inputs yielded gains but is limited. In contrast, in Heihe and Tibet regions with complex hydrological processes and strong heterogeneity, the marginal benefits of multimodal inputs were far greater. For example, in Heihe, SAR-only input yielded an MAE of 0.1004, whereas combined inputs reduced the error to approximately 0.0509. The upper bounds and number of outliers in the error distribution also decreased substantially, indicating that multimodal features provided effective constraints for capturing local heterogeneity caused by river irrigation and groundwater recharge. A similar trend was observed in Tibet: multimodal integration nearly halved the error, with both the overall box height and number of outliers significantly reduced, demonstrating improved model stability and robustness under conditions of high heterogeneity and multiple water sources.

From an application perspective, these different outcomes have important implications. In hydrologically stable and spatially homogeneous regions (e.g., Wangkui), SAR-only input can deliver satisfactory inversion accuracy with lower computational and data costs. However, in highly heterogeneous regions (e.g., Tibet and Heihe), single-source input is insufficient to capture complex moisture dynamics, and multimodal, multi-scale data fusion becomes essential for ensuring reliability. These findings suggest that large-scale applications should adopt a “region-specific” strategy: lightweight SAR-only models in stable areas to maximize efficiency, and fusion-based models in complex regions to preserve accuracy. Such an adaptive approach offers a balanced solution between cost and performance.

4.4. Main Perspectives of the Multimodal Approach

Nevertheless, several sources of error remain. Some errors arise from the inherent uncertainties of the data. For instance, SWF-derived soil moisture estimates have relatively coarse spatial resolution, reflecting regional averages rather than point-level ground truth, which introduces label noise [66]. In situ datasets also vary in acquisition methods and conditions, leading to inconsistencies in depth measurements [67]. Moreover, multimodal data are not always complete, and missing features were sometimes replaced with masks during training. The strong performance across all three regions ($R^2$ = 0.8956, RMSE = 4.58%) suggests that the model successfully captures soil moisture dynamics despite depth variations. The consistency of results across sites with different target depths (2 cm vs. 5 cm) indicates that the multimodal approach provides robust estimates of soil moisture. However, in regions with highly complex hydrological processes and multiple recharge sources (e.g., Tibet and Heihe), soil moisture dynamics are subject to large temporal variability and strong external influences such as irrigation and groundwater replenishment. These factors make it difficult for the model to achieve complete convergence, thereby contributing to higher residual errors.

While ideally all measurements would be at a uniform depth, the framework does not rely solely on SAR backscatter. The integration of vegetation indices, meteorological variables, and topographic features provides information about soil water dynamics that extends beyond the immediate SAR sensing depth. These auxiliary variables help bridge the gap between surface backscatter and deeper soil moisture.

Future research can be pursued in several directions. First, incorporating more advanced deep learning architectures and adaptive modality selection mechanisms may help reduce redundancy and enhance model robustness [31]. Second, coupling multi-temporal and multi-scale datasets could provide a more comprehensive representation of soil moisture dynamics [68]. Third, integrating physical constraints into the modeling process would ensure that predictions remain consistent with the underlying principles of hydrological processes [69].

5. Conclusions

This study proposed an integrated framework for soil moisture estimation that fuses SAR, optical, topographic, and meteorological data with an intermediate fine-tuning strategy. The results demonstrated that data fusion can substantially improve estimation accuracy, thereby enabling more reliable field-scale soil moisture maps that support informed irrigation planning and water allocation. The fine-tuning strategy proved effective in translating coarse satellite information into high-resolution, management-ready outputs, showing strong scalability for operational applications. Moreover, the framework exhibited robustness across heterogeneous landscapes, from stable agricultural regions to hydrologically complex basins, thus supporting adaptive management under diverse conditions. The observed sensitivity to soil type further highlights the need for region-specific strategies, where SAR-only models may suffice in stable areas while multi-source integration is essential in complex terrains. Overall, this research provides both a methodological advance and a practical pathway toward data-driven, resilient, and efficient agricultural water management, contributing to sustainable use of water resources.