Soil Moisture Content Prediction Using Gradient Boosting Regressor (GBR) Model: Soil-Specific Modeling with Five Depths

Abstract

Monitoring soil moisture content (SMC) remains challenging due to its spatial and temporal variability. Accurate SMC prediction is essential for optimizing irrigation and enhancing water use efficiency. In this research, a Gradient Boosting Regressor (GBR) model was developed and validated to predict SMC in two soil textures, loam and silt loam, using meteorological data from Internet of Things (IoT) sensors and gravimetric SMC field measurements collected from five different depths. The statistical analysis revealed significant variation in SMC across depths in loam soil (p < 0.05), while silt loam exhibited more stable moisture distribution. The GBR model demonstrated high performance in both soil textures, achieving R² values of 0.98 and 0.94 for silt loam and loam soils, respectively, with low prediction errors (RMSE 0.85 and 0.97, respectively). Feature importance analysis showed that precipitation and humidity were the most influential features in loam soil, while solar radiation had the highest impact on prediction in silt loam soil. Soil depth also showed a significant contribution to SMC prediction in both soils. These results highlight the necessity for soil-specific modeling to enhance SMC prediction accuracy, optimize irrigation systems, and support water resources management approaches aligning with SDG6 objectives.

1. Introduction

Soil moisture content (SMC) plays a crucial role in hydrological and ecological processes that affect energy, water, and carbon cycles, including evaporation [1], transpiration, diversity [2], and rainfall–runoff in various ecosystems [3]. Understanding the spatiotemporal patterns of SMC is essential for water resources management, mitigating the effects of agricultural drought [4], addressing water scarcity, and improving crop production [5,6]. Moreover, the collection of high-quality soil moisture data is significant for scientific research purposes, such as weather forecasting, flood prediction, drought monitoring [7,8], precision agriculture [9], and irrigation management [10].

As the demand for high-precision soil moisture data increases, there is a growing need to develop reliable in situ soil moisture sensor networks that balance cost, complexity, and low power consumption for effective soil moisture monitoring [11]. The advancement of sensing technologies, including Internet of Things (IoT), has enabled real-time data exchange between connected devices using a wireless sensor network, allowing for constant monitoring of various environmental parameters in real time [12]. Recent studies have demonstrated the applicability of IoT-based approaches in agriculture for environmental monitoring [13], controlling greenhouse gas emissions [14], irrigation optimization, weather monitoring, product traceability, and intelligent equipment management [15]. Notably, IoT sensors address the limitations of traditional soil moisture measurement techniques, which are often time consuming and lack real-time monitoring capabilities, by providing real-time, low-maintenance, and scalable solutions [16]. In addition to in situ measurements, remote sensing approaches utilizing satellite images have been widely used for SMC monitoring and predicting model development [17]. The integration of meteorological data and soil properties monitoring in SMC prediction provides a better understanding of its variability and the types of interactions between SMC and these parameters.

Accurate SMC prediction is important for enhancing irrigation management and optimizing crop production [18]. Traditional statistical methods often struggle to capture the complex, nonlinear, and multivariate interactions among soil, environmental, and meteorological factors that influence SMC [19]. In contrast, machine learning (ML) algorithms can handle these complexities and offer high potential for accurate SMC prediction [20,21]. ML models have the capability to utilize the high-resolution dataset provided by IoT-based sensors to enhance prediction accuracy of SMC. Among these, the Gradient Boosting Regressor (GBR) algorithm has demonstrated strong performance in SMC prediction due to its ability to combine multiple weak learners (typically decision trees) to iteratively minimize loss functions, thereby improving both accuracy and interpretability [22]. For instance, Zambudio et al. [23] utilized two years of data to train a GBR model and artificial neural network (ANN) to predict SMC in top surface layers. The results showed that the GBR model performed effectively in predicting SMC with a 0.01 learning rate, 5 max depth, and 350 estimators, achieving a mean square error (MSE) of 0.027 and a 20% difference in maximum between measured and predicted values. Similarly, Ren et al. [24] utilized XGBoost to predict soil moisture, highlighting its high performance in comparison with other models, with a correlation coefficient of 0.69 and an accuracy of 88%. The primary predictors were air relative humidity, maximum air temperature, and total precipitation. Consideration of soil characteristics, particularly during the 2022 drought, led to improved prediction accuracy.

However, existing research had a lack of soil texture-specific and depth-specific modeling, which considers significant factors regarding their strong impact on SMC. For instance, soils with higher silt and clay content generally exhibit higher water-holding capacity compared to sandy soils, which leads to higher SMC [25]. Additionally, soil depth has a crucial impact on SMC, as deeper layers often retain more moisture due to lower evaporation and higher water retention. Fu et al. [26] demonstrated this in their study utilizing the water balance equation, showing that predictive model performance varied with depth, achieving lower RMSE values at 100 cm and 200 cm compared to shallow depths.

In response to these developments, international frameworks such as the UN-Water 2023 Annual Report and the EU’s “Farm to Fork” strategy have emphasized the significance of adaptive modern water resource management approaches that align with Sustainable Development Goal 6 (SDG6): Water and Sanitation for All [27]. The integration of IoT and ML in precision crop production contributes to these goals by allowing for better resource management, enabling data-driven decision making [28].

This research developed and validated soil texture-specific Gradient Boosting Regressor (GBR) models to predict SMC in two distinct soil textures, loam and silt loam. These soils differ in their moisture retention characteristics, where the loam soil exhibits moderate water-holding capacity due to its balanced sand, silt, and clay proportions, while silt loam retains more moisture because of its finer texture and higher silt content [29]. High-resolution meteorological data were collected using calibrated IoT-based sensors for model training. The key objectives of this research are to evaluate GBR model performance using multiple evaluation metrics; to identify optimal model hyperparameters by performing 5-fold cross-validation and grid search; and to identify the most influential predictors of SMC in each soil texture utilizing feature importance analysis. These will allow for better understanding of soil moisture dynamics in each soil texture, as well as better optimization of irrigation schedules and enhanced water resource use efficiency, aligning with SDG6 goals for sustainable water management.

2. Materials and Methods

2.1. Experimental Site and Data Collection

This field experiment was conducted during the maize vegetation season at a 23-hectare field belonging to Széchenyi István University, located in Mosonmagyaróvár, Hungary. The site lies within the Little Hungarian Plain, an alluvial plain of the Leitha River, characterized by relatively flat topography with a slight slope of approximately 5% and elevation ranging from 133 to 138 m above sea level. This gradient allows for natural surface drainage while minimizing erosion risks. The region experiences a temperate climate with moderate precipitation, typical of Central Europe. During the study period, from May to October, total precipitation was approximately 400 mm, and the average air temperature ranged from 18.5 °C to 21.3 °C. Based on USDA soil classification taxonomy [30] and laboratory particle size distribution analysis using the soil texture triangle, three primary soil textures were identified in the field: loam, sandy loam, and silt loam (Figure 1). These soil textures represent more than 60% of the soils in this region [29]. The field is an alluvial plain of the Leitha River located within the little Hungarian plain and is characterized by relatively flat topography. The field has a slight slope of 5%, with elevation varying between 133 and 138 m above sea level. This slight gradient allows natural surface drainage while minimizing erosion risks. For this study, two soil textures—loam and silt loam—were selected for detailed analysis. Their specific physical and chemical properties are presented in

Table 1. Soil properties averages and ranges of soil content of sand, silt, and clay in the five depths in the two studied soil textures with mean SMC.

Soil Texture	pH	Sand % Range	Silt % Range	Clay % Range	Mean SMC %	Std of SMC %	Bulk Density
Loam	7.614	40.99–49.01	40.91–45.97	10.10–14.50	13.79	3.94	1.538
Silt loam	7.45	12.36–15.31	63.70–70.19	17.40–21	16.95	5.07	1.52

. The field is managed under a cereal-based crop rotation system, which includes winter wheat, spring barley, maize, and soybeans.

2.2. Soil Sampling and Gravimetric Measurements

From the two selected soil textures, 300 soil samples were collected between June and October 2023 using a hand auger (Figure 2B). Sampling was conducted at five depths (5, 20, 40, 60, and 80 cm) from three sampling points near the sensor installations, following a random sampling grid approach. Soil samples were collected at 10 different dates spaced two weeks apart, resulting in a total of 150 samples from each soil texture.

Soil moisture content was calculated using the gravimetric method based on dry weight [31]. The samples from each depth and soil texture were placed in pre-weighed aluminum containers and dried in an oven at 105 °C until a consistent weight was achieved. The containers were then weighed again to calculate dry weight of the soil. The soil moisture content (θ) was calculated using the following Equation (1):

$$\mathit{\theta }={\displaystyle \frac{\mathit{M}\mathit{w}}{\mathit{M}\mathit{d}}}\times 100$$

(1)

where

θ = Soil Moisture Content (%);

Mw = Mass of water (g) = (Wet weight − Dry weight);

Md = Mass of dry soil (g).

2.3. Meteorological Data Collection Utilizing IoT Sensor

Meteorological data were collected using an IoT-based meteorological sensor (Boreas Ltd., EcoLogger Boreas datalogger unit BEL-06, BHS-06 for temperature and humidity, BWS/s-06 Wind Speed Sensor, BIS-06 global solar radiation, and BPD-06 for precipitation) (Figure 2A). The manufacturer claims an accuracy of ±0.2 °C for temperature, ±2% for humidity, and ±0.1 mm for precipitation. To minimize the measurement error that could affect SMC prediction, the meteorological sensor was calibrated to ensure the accuracy of data. The data were collected at 10–15-min intervals using LoRaWAN, a low power consumption device with the capability of transmitting sensor signals up to 30–40 km away on level terrain. Meteorological parameters, including precipitation (mm), temperature (°C), humidity (%), wind speed (km/h), and solar radiation (W/m²), were used to examine how these meteorological factors influence variations in soil moisture content, considering the effects of both soil depth and soil texture. These features were selected based on their influence on soil moisture dynamics, where precipitation is considered the primary source of soil water and SMC increased with increasing precipitation [32]. Humidity, on the other hand, reflects the air moisture content, influencing the evaporation rate from the surface soil where increasing the humidity reduces the moisture losses by evaporation [33]. Solar radiation also has a significant impact on SMC depletion by its effect on available energy for evaporation and transpiration. Temperature also plays a crucial role in SMC variation; evaporation increases at higher temperatures. Also, wind speed has a role in enhancing moisture losses in the soil surface due to the increase in air movement [34,35]. The data collected represent the average of the field and were assigned to all sampling locations. Also, the data were synchronized with the dates of soil sampling to ensure accurate pairing of meteorological and SMC observations.

2.4. Data Preprocessing and Feature Engineering

Data preprocessing was performed utilizing Python (version 3.10.12) [36], and meteorological data were synchronized with the SMC data. Categorical variables such as soil texture were encoded, and missing data were handled by imputation. Data preparation, such as transformation, cleaning, and statistical testing to ensure uniformity and prevent inaccurate or missing values, was conducted using Python. Six predictive features were selected to train the model including depth (cm), precipitation (mm), temperature (°C), humidity (%), wind speed (km/h), and solar radiation (W/m²), and the target variable is SMC%. These features were chosen based on their availability from field sensors and their well-documented influence on soil moisture dynamics as reported in previous studies, highlighting their agricultural relevance in irrigation management, crop water requirements, and soil water balance assessments [37,38].

To determine whether SMC varied significantly across the five depths while accounting for repeated sampling at the same times across time, Repeated Measures ANOVA (Equation (2)) was performed independently for each soil texture using the statsmodels library’s AnovaRM. The subject was identified using sampling points, and the within-subject element was depth (cm). The statistical significance level was set to p < 0.05.

Using Python, a descriptive statistical analysis was conducted to determine the mean soil moisture content (SMC%), standard deviation (SD), and coefficient of variation (CV%) for each soil texture and depth.

$${\mathit{Y}}_{\mathit{i}\mathit{j}\mathit{k}}=\mathit{\mu }+{\mathit{\alpha }}_{\mathit{i}}+{\mathit{S}}_{\mathit{j}}+{\mathit{\epsilon }}_{\mathit{i}\mathit{j}\mathit{k}}$$

(2)

where

$Y_{ijk}$ = SMC measurement at depth i, samplisng point j, time k.

Μ = Overall mean, ${\alpha }_i$ = Fixed effect of depth, $S_j$ = Random effect of sampling point.

${\epsilon }_{ijk}$ = Residual error.

The coefficient of variation (CV%) was calculated using the following formula:

$$\mathbf{C}\mathbf{V}\left(\mathbf{\%}\right)={\displaystyle \frac{\mathit{S}\mathit{D}}{\mathit{M}\mathit{e}\mathit{a}\mathit{n}}}\times 100$$

(3)

2.5. Development and Optimization of GBR Model

A Gradient Boosting Regressor (GBR) model was used to predict SMC due to its ability to capture the nonlinear interactions between studied features. GBR considers the form of ML algorithms that produce prediction models by integrating multiple weak learners, mainly decision trees, in a sequential manner. The procedure begins with an initial model, and the succeeding models are trained to optimize the previous models’ errors, which effectively minimizes the loss function utilizing gradient descent approaches. This allowed us to develop a robust model with the ability to process complex patterns in data and enhance accuracy [39,40].

The model was trained for the two soil textures (loam and silt loam) separately to increase soil-specific prediction; the models were developed using the scikit-learn library; and the models were optimized using GridSearchCV for optimizing hyperparameters, including n_estimators, learning_rate, max_depth, and min_sample_split. A 5-fold cross-validation was utilized for performance validation to reduce overfitting and enhance robustness of the models. The dataset was split into 70% training and 30% testing sets. Feature importance analysis was carried out to determine the most important features that impact the SMC prediction in each soil texture. All modeling, analysis, and visualization were performed using Python libraries such as pandas, numpy, scikit-learn, seaborn, and matplotlib. The methodology used was developed to ensure the model’s robustness, explainability, and applicability.

2.6. Performance Evaluation Measures

Five metrics were calculated both on the hold-out test set and by cross-validation to quantify the performance of the models.

1-

MSE is the average of the squared differences between predicted and observed values [41].

$$\mathit{M}\mathit{S}\mathit{E}={\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mathit{n}$}\right.}\sum _{\mathit{t}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{t}}-\widehat{\mathit{y}}\mathit{t}\right)}^2$$

(4)

2-

The root means squared error (RMSE) [41] is as follows:

$$\mathit{R}\mathit{M}\mathit{S}\mathit{E}=\sqrt{{\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mathit{n}$}\right.}\sum _{\mathit{t}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{t}}-\widehat{\mathit{y}}\mathit{t}\right)}^2}$$

(5)

3-

Mean Absolute Error (MAE) calculates the average of the absolute differences between predictions and actual observations, as shown in Equation (6) [41]:

$$\mathit{M}\mathit{A}\mathit{E}={\displaystyle \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\mathit{n}$}\right.}\sum _{\mathit{t}=1}^{\mathit{n}}\left|{\mathit{y}}_{\mathit{t}}-\widehat{\mathit{y}}\mathit{t}\right|$$

(6)

where $n$ is the total number of observations in the dataset, t is the time index or sequence index of an observation, $y_t$ is the observed value at time t, and $\widehat{y}t$ is the predicted value by the model at time t.

4-

The Nash Sutcliffe efficiency metric (NSE) [42] is as follows:

$$\mathit{N}\mathit{S}\mathit{E}=1-{\displaystyle \frac{\sum _{\mathit{i}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{i}}-{\widehat{\mathit{y}}}_{\mathit{i}}\right)}^2}{\sum _{\mathit{i}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{i}}-\overline{\mathit{y}}\right)}^2}}$$

(7)

where $n$ is the total number of observations in the dataset; ${\mathit{y}}_{\mathit{i}}$ is the observed value at observation i; ${\widehat{\mathit{y}}}_{\mathit{i}}$ is the predicted value at observation i; and $\overline{\mathit{y}}$ is the mean of observed values.

5-

R-squared (R²) is a statistical metric that represents the proportion of the total variance in the observed data that is explained by the model. Higher R² indicates less difference between observed and predicted values [43].

$${\mathit{R}}^2=1-{\displaystyle \frac{\sum _{\mathit{t}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{t}}-\widehat{\mathit{y}}\mathit{t}\right)}^2}{\sum _{\mathit{t}=1}^{\mathit{n}}{\left({\mathit{y}}_{\mathit{t}}-\mathit{y}̲\right)}^2}}$$

(8)

3. Results and Discussion

3.1. Statistical Analysis Results for the Two Soil Textures

The descriptive statistical analysis (

Table 2. Descriptive statistical analysis results for both soil textures at five depths.

Soil Texture	Depth (cm)	Mean SMC (%)	SD (%)	CV (%)
Loam	5	12.8	3.08	24
Loam	20	16.19	3.51	21.7
Loam	40	14.66	3.76	25.6
Loam	60	12.61	4.39	34.8
Loam	80	12.68	4.94	39
Silt loam	5	14.99	2.62	17.5
Silt loam	20	18.27	3.51	19.2
Silt loam	40	18.03	4.11	22.8
Silt loam	60	16.97	7.23	42.6
Silt loam	80	16.47	7.87	47.8

) shows that in loam soil, the highest mean SMC (16.19%) was found at 20 cm with the least variability (CV = 21.7%), while the deepest layer (80 cm) had the most variability (CV = 39%) ranging from 5.49% to 20.77%. In silt loam soil, SMC was generally higher at all depths, with the highest mean SMC at 20 cm (18.27%). However, the deepest layers (60 cm and 80 cm) demonstrated higher variations (CV = 42.6% and 47.8%, respectively) with SMC ranging from 7.92% to 26.58%, indicating less consistent moisture retention at depth.

The repeated measures ANOVA revealed substantial variation in SMC across depths in both soil textures (p < 0.05). The variation in SMC across depths (Figure 3) highlights the more stable behavior of silt loam soil in the upper layers and the higher variability observed in loam soil at deeper layers.

The observed patterns confirmed the significant impact of depth on SMC variability, where deeper layers in both soil textures exhibited higher CV%, indicating less predictable moisture retention likely due to soil structural changes and lower infiltration rates. These findings highlight the impact of soil depth on SMC variability, emphasizing the importance of considering both soil texture and depth during monitoring [44,45] and the need for considering soil texture and depth in irrigation planning and management to optimize water resources and increase water use efficiency.

3.2. Model Performance Results

The results show that GBR model performance was highly effective in predicting SMC in both soil textures (Figure 4). The model performance was evaluated using test data and validated by using 5-fold cross-validation. In silt loam soil, the model achieved the highest accuracy, with an R² of 0.98, MSE of 0.73, RMSE of 0.85, MAE of 0.45, and NSE of 0.95 (Figure 5), reflecting the model’s consistency and generalization capability. In loam soil, the model performance was slightly lower, likely due to the higher variation in SMC values compared with silt loam, with an R² of 0.94, MSE of 0.94, RMSE of 0.97, MAE of 0.73, and NSE of 0.94. These results highlight the impact of soil texture on predictive model performance, where less prediction accuracy was shown in loam soil due to its higher variability in SMC compared with silt loam soil where its balanced structure and high water retention had a significant role in enhancing model accuracy. These results emphasize the need for soil-specific modeling to enhance SMC prediction [46] and the necessity for integrating more physical soil properties into models for improving prediction reliability across different soil textures.

3.3. Feature Importance Analysis

To better understand the influence of meteorological features and soil depth on SMC prediction in the two examined soil textures (loam and silt loam), as well as to improve the model’s interpretability, feature importance analysis was performed for both soil textures using the GBR model. The results reveal that the impact of the analyzed variables varied with soil texture. In loam soil, precipitation and humidity were the most influential features in prediction, with values of 31% for both (Figure 6). Soil depth had a substantial impact on SMC, with an importance value of 27%, as loam soil retains water effectively due to its balanced particle size distribution. Precipitation and humidity also played crucial roles in influencing SMC in loam soil. In contrast, wind speed and solar radiation had minimal impact, both with importance values below 6%. These results emphasize the significance of considering precipitation, humidity, and soil depth when monitoring and predicting SMC in loam soils, particularly as precipitation directly contributes to replenishing soil moisture [47], and humidity indirectly affects soil air water dynamics [48]. Conversely, in silt loam soil, the results emphasize the importance of solar radiation in SMC prediction with a value of 46%, followed by wind speed and depth with values of 26% and 24%, respectively, highlighting the role of radiation-driven evapotranspiration in controlling moisture variation in silt loam soils due to their balanced texture and its high water-holding capacity compared with loam soil and the role of wind in increasing evapotranspiration.

These findings are consistent with previous research, highlighting the critical role of environmental variables such as precipitation and solar radiation in influencing soil moisture dynamics and demonstrating the complexity of soil–water interactions in the studied soils [49]. Additionally, integrating the effects of soil texture and depth provided further insights into their significance in driving the fluctuations and dynamics of SMC [29].

The results also emphasize the importance of incorporating multiple features in model training for SMC prediction across different soil textures, including vegetation indices, which offer valuable information on soil–plant interactions and their impact on soil moisture patterns [19,50]. Increasing the number of sampling locations across the field would further improve the spatial representativity of the dataset, thereby enhancing the model’s robustness and generalizability. The model effectively captured the relationships between the selected input variables and SMC in both soil textures, highlighting its potential to reduce the need for extensive field sampling and support cost-efficient soil moisture monitoring.

These findings show the complex interactions of meteorological conditions and depth with SMC [51]. In contrast, satellite-based models frequently have higher spatial resolution but limited soil penetration and temporal alignment with ground conditions, since they utilize passive microwaves signals and downscaling approaches to predict SMC in the surface soil layers [52]. These findings highlight the significance of soil-specific water management systems and real-time monitoring to improve crop production precision.

4. Conclusions

This study demonstrated that the Gradient Boosting Regressor (GBR) model is a reliable and accurate approach for predicting SMC in two distinct soil textures, loam and silt loam, using soil depth and meteorological data as input features. The GBR models achieved high prediction accuracy in both soil textures, with R² values exceeding 0.94, and the highest performance observed in silt loam soil. These findings emphasized the robustness and generalization capability of the GBR model in capturing nonlinear interactions between SMC, depth and meteorological parameters, despite the limited sample size and spatial coverage.

The repeated measures ANOVA confirmed significant variation in SMC across depths, particularly in loam soil, where deeper layers exhibited higher variability. In contrast, silt loam soil exhibited higher overall SMC with more stable distribution across depths. The feature importance analysis provided further insights, revealing that precipitation and humidity were the most influential variables in loam soil (both contributing approximately 31%), while solar radiation was the dominant factor in silt loam soil (46%). Soil depth also had a significant effect on SMC prediction in both soil textures, emphasizing the need to consider depth-specific patterns in modeling and monitoring efforts.

These findings underscore the importance of incorporating both meteorological variables and soil-specific characteristics in SMC modeling. The model’s ability to achieve high accuracy with limited data supports its potential to reduce the need for extensive field sampling, contributing to more efficient data-driven decision making for improving water use efficiency and irrigation management, in alignment with SDG 6 goals on sustainable water use and ecosystem protection.

Additionally, the study emphasizes the necessity of soil texture-specific modeling combined with depth-sensitive predictions and explainable feature analysis. However, the study acknowledges the limitation regarding the assumption of independence in statistical analysis, which was addressed by applying a repeated measures design. Future research should further strengthen the modeling framework by incorporating more repeated sampling, integrating additional soil physical properties, and considering the impact of crop growth stages through remote sensing data to enhance the accuracy and applicability of SMC prediction models.