Vis–NIR Spectroscopy Characteristics of Wetland Soils with Different Water Contents and Machine Learning Models for Carbon and Nitrogen Content

Abstract

Soil nutrient detection in wetlands is critical for rapidly and effectively managing these ecosystems. Our objective was to provide a methodological framework for identifying optimal data processing methods and machine learning model for predicting soil organic carbon (SOC) and total nitrogen (TN) content using Vis–NIR spectroscopy, under the confounding influence of varying soil moisture. Soil samples (474) were collected from the Shaanxi Yellow River Wetland Provincial Nature Reserve with five moisture levels (0, 5, 10, 20, and 30%). Using a Vis–NIR spectroscopy system (ASD FS4 spectrometer), soil organic carbon (SOC) and total nitrogen (TN) were detected within the 350–2500 nm spectral range. Machine learning models were established using the Random Forest model (RF), eXtreme Gradient Boosting (XGBoost), and Partial Least Squares Regression (PLSR). The results indicated: (1) spectral reflectance values increased as soil moisture content decreased, with the 0% moisture model being consistently more accurate; (2) models for SOC and TN on first-derivative spectra had higher accuracy; and (3) the RF exhibited higher inversion accuracy and stability (R² = 0.30–0.69). (4) The SHAP analysis confirmed 1865 nm and 1419 nm as the most contributory bands for SOC and TN prediction respectively, validating the RF model’s spectral interpretation capability.

1. Introduction

Carbon and nitrogen in soil are pivotal to the biogeochemical cycles of terrestrial ecosystems and plant growth, serving as key indicators of soil quality [1]. Soil carbon components aid in the formation and decomposition of organic matter, improve soil structure, support microbial activity and diversity, and enhance soil carbon storage, thereby mitigating climate change [2,3]. Soil nitrogen provides essential nutrients for plant growth, bolstering soil fertility through the nitrogen cycle and fostering ecosystem productivity [4,5]. To understand the critical role of soil in global material cycles, it is necessary to quantitatively assess soil carbon and nitrogen contents and their management [6]. Soil, as a natural continuum, exhibits significant spatial heterogeneity in the distribution of soil organic carbon (SOC) and total nitrogen (TN) [7]. Traditional laboratory chemical measurement methods are expensive, prone to generating chemical waste that causes environmental pollution, and are not feasible for large-scale applications [8,9]. Visible–near infrared Vis–NIR spectroscopy technology can efficiently and rapidly acquire information on various soil nutrients. With low operational costs and strong adaptability, it provides robust technical support for measuring soil nutrient content [10]. The principles of reflectance spectroscopy in soil science are closely related to the interaction between electromagnetic waves and soil components [11,12]. Different wavelengths of soil reflectance spectra exhibit specific optical responses to soil components such as carbon and nitrogen, enabling the estimation of their content [13,14,15]. In recent years, Vis–NIR spectroscopy technology has increasingly been applied in soil science, including soil classification [16], nutrient content estimation [17], moisture measurement [18], and pollutant detection [19], becoming a reliable tool for investigating soil-related issues.

However, the collection of soil spectra is influenced by numerous factors, of which soil moisture is one of the most critical [20,21,22]. Previous studies have shown that when collecting in situ spectra in the field, the presence of soil pores and soil microaggregates causes the soil to absorb moisture from the air. This moisture increases the soil absorbance, increases the light refracted from the soil surface to the air, and decreases the light reflected. Consequently, this leads to measurement errors in the soil spectral reflectance and makes it challenging to establish high-accuracy prediction models for soil property content [23,24,25].

In recent years, researchers worldwide have conducted extensive studies on the relationship between soil moisture and soil spectral reflectance. It is widely accepted that the relationship between soil moisture content and spectral reflectance is nonlinear. Specifically, when the soil moisture content was below field capacity, there was a negative correlation between soil spectral reflectance and soil moisture content, with reflectance values decreasing as moisture content increased. Conversely, when the soil moisture content exceeds the field capacity, there is a positive correlation, with the reflectance values increasing as the moisture content increases [26]. suggested that the significant influence of soil moisture on spectral reflectance may obscure the spectral characteristics of other substances [27]. Morganc et al. found that rewetting air-dried soil samples reduced the prediction accuracy of SOC and inorganic carbon owing to the impact of soil moisture [28]. discovered that the predictive ability of VIS–NIR for soil properties declined when the soil moisture content exceeded 10% [29]. reported that using oven-dried soil in the lab to estimate soil carbon, nitrogen, phosphorus, potassium contents, and pH showed a slight improvement in prediction accuracy with the removal of moisture [30]. indicated that the prediction accuracy for organic matter, available nitrogen, and available potassium significantly decreased with increasing soil moisture [31]. predicted the soil TN content using soil samples with eight moisture levels and concluded that moisture affects the predictive performance of the models [32].

Although research on the impact of soil moisture on near-infrared spectroscopy has made some progress, different scholars have conducted detailed analyses and thorough explorations to predict certain soil parameters (such as soil organic matter, available nitrogen, and available potassium). However, research on the effect of soil moisture on the prediction of SOC and TN is limited. The objectives of this study were to: (1) systematically evaluate the impact of soil moisture on the prediction accuracy of SOC and TN; (2) compare the effectiveness of spectral preprocessing techniques, specifically raw and first-derivative (FD) spectra; (3) assess the performance and stability of three machine learning algorithms—eXtreme Gradient Boosting (XGBoost), Random Forest (RF), and Partial Least Squares Regression (PLSR)—across five soil moisture levels (0%, 5%, 10%, 20%, and 30%); and (4) interpret the prediction mechanisms of the optimal model using SHapley Additive exPlanations (SHAP) to identify the most contributory spectral bands.

While the impact of soil moisture on Vis–NIR spectra is well-documented, a systematic comparison of prediction models across a controlled moisture gradient, combined with interpretable machine learning for SOC and TN in wetlands, remains limited. The novelty of this study lies in this comprehensive framework. Specifically, compared to earlier works, which often focused on specific moisture levels or correction techniques, our study provides a direct and systematic evaluation of model performance and interpretability across five defined moisture levels (0–30%), explicitly quantifying the accuracy loss with increasing moisture and identifying the most robust modeling strategy for such conditions.

2. Materials and Methods

2.1. Study Site

The Shaanxi Yellow River Wetland Provincial Nature Reserve is located in the eastern part of the Guanzhong Plain in Shaanxi Province (110°10′–110°36′ E, 34°36′–35°40′ N), within the Weinan city region. It encompasses the Yellow River channel, floodplains, and the confluence areas of the Yellow River, Wei River, and Luo River in Hancheng, Heyang, Dali, Huayin, and Tongguan, covering a total area of 45,986 ha. It is the largest riverside wetland in the Yellow River Basin in China. The reserve primarily aims to protect the natural environment of the wetland ecosystem and the related biota dependent on it. The terrain within the protected area is flat, and the soil types include saline soil, new soil, and swamp soil, with a soil pH value between 7.66–10.56 The climate is warm temperate continental semi-humid monsoon, with an average annual temperature of 13.5 °C and an average annual precipitation of 529–574 mm [33]. The reserve boasts rich plant resources, including 287 species of seed plants from 236 genera and 70 families, two species of gymnosperms from one family, 69 species of monocots from 17 families, and 216 species of dicots from 52 families. Dominant plant species within the wetland communities include common reed (Phragmites australis), and oriental cattail (Typha orientalis), typical of herbaceous marshes.

2.2. Soil Sample Collection

The sampling was conducted between August and September 2022. Soil samples were collected from a depth of 0–30 cm at each sampling site, thoroughly mixed, and bagged. The sampling points were spaced more than 50 m apart, resulting in 474 sampling points. The final collection included 116, 117, 120, and 121 samples from bare, reed (Phragmites australis), nutgrass (Cyperus rotundus), and cattail (Typha orientalis) wetlands, respectively. The sampling areas were consistent within the Yellow River Wetland Provincial Nature Reserve, providing ample data to support our results.

2.3. Chemical Determination of Soil Carbon and Nitrogen Content

SOC content was determined using the potassium dichromate-ferrous sulphate titration method, whereas TN content was measured using the semi-micro Kjeldahl method.

The descriptive statistics of the measured SOC and TN contents for all 474 samples are as follows: SOC ranged from 0.29 to 13.69 g/kg with a mean ± standard deviation of 3.32 ± 2.03 g/kg; TN ranged from 0.09 to 1.57 g/kg with a mean ± standard deviation of 0.40 ± 0.24 g/kg.

2.4. Spectral Data Measurement of Soil Samples with Different Moisture Contents

Gravel, insects, plant roots, and leaf debris were manually removed from the freshly collected moist soil samples and placed in aluminium boxes that had been pre-dried and weighed, recording the weight as M1. We dried the samples in an oven for 24 h at a temperature of 105 °C, ensuring that all moisture was completely removed. After drying, the samples were weighed again, and the dry weight was recorded as M2. The difference M2-M1 represented the weight of the soil sample in the aluminium box, denoted M. Fully dried soil samples were labelled as samples with 0% moisture content. Subsequently, we added 0.05 M, 0.10 M, 0.20 M, and 0.30 M of distilled water to each sample to prepare soil samples with 0%, 5%, 10%, 20%, and 30% moisture content, respectively.

Using an ASD FS4 (Analytical Spectral Devices, Inc., Boulder, CO, USA) equipped with a soil reflectance probe, the spectral data of soil samples with different moisture gradients were collected in a dark room across the 350–2500 nm wavelength range. The prepared soil samples with varying moisture content were poured into preprepared black Petri dishes and the surface of each soil sample was smoothed with a knife. The soil reflectance probe was positioned vertically 5 cm above the sample surface, ensuring that the receiving area of the probe did not exceed the diameter of the dish, to minimise the influence of external light on the spectral reflectance of the soil sample. The instrument was preheated for 30 min before calibration with a white reference panel, and a white reference correction was performed before the spectral measurement of each soil sample. Ten spectral curves were collected for each soil sample, and the arithmetic average of these 10 curves was used as the actual reflectance spectral data to reduce measurement errors.

The moisture levels investigated in this study (0–30%) encompass a range relevant to field conditions in wetlands. Field capacity for many mineral soils typically falls between 20–30%. Therefore, our gradient from 0% to 30% covers conditions from completely dry to near or at field capacity, which are commonly encountered in field monitoring and remote sensing scenarios. This allows our findings to be directly applicable to understanding and mitigating moisture effects in practical applications.

2.5. Soil Spectral Data Preprocessing

Extract spectral data using Viewspec Pro software (Version 5.6), and export the average of 10 spectral data collected at each sample point as the reflectance spectral data for that sample point. To avoid redundancy caused by overly narrow bands, the spectral data were smoothed by averaging every 10 adjacent bands. To improve inversion accuracy, the raw spectral reflectance was transformed using the first-derivative (FD) method [34]. The formula is:

$$\mathrm{F}\mathrm{D}\mathrm{R}\left({\lambda }_i\right)={\displaystyle \frac{R\left({\lambda }_{i+1}\right)-R\left({\lambda }_{i-1}\right)}{∆\lambda }}$$

In the formula, ${\lambda }_{\mathfrak{i}}$ represents the wavelength of each band, $\mathrm{F}\mathrm{D}\mathrm{R}\left({\lambda }_i\right)$ is the FD value of wavelength ${\lambda }_i$, and $∆\lambda$ represents the wavelength values from band $i$ to band $i$ + 1.

2.6. Selection of Characteristic Bands

The Successive Projections Algorithm (SPA) is a feature selection method aimed at improving the robustness and predictive power of multivariate regression models by iteratively selecting variables that best explain the data characteristics [35]. The SPA begins by randomly selecting a variable from the dataset, then projects the remaining variables onto the orthogonal complement space of the current variable set. The variable with the largest projection length is added to the variable set. This process is repeated until the number of selected variables reaches a predetermined value n. Using raw spectral reflectance and FD spectral reflectance as independent variables and soil carbon and nitrogen values as dependent variables, SPA calculations were implemented using MATLAB R2023a. The calculated wavelengths were used as characteristic bands to predict soil carbon and nitrogen values. To determine the optimal number of characteristic bands (N), the SPA was run for a range of variable subset sizes. For each candidate subset selected by SPA, a preliminary PLSR model was constructed, and the corresponding Root Mean Square Error (RMSE) was calculated. The final number of variables (n = 20) was chosen as it corresponded to the point where the RMSE reached a minimum or a stable plateau, following the standard optimization procedure for SPA.

2.7. Construction and Accuracy Verification of Machine Learning Models

To ensure that both the calibration and validation sets were representative of the overall distribution of soil carbon and nitrogen content, a stratified sampling method was employed. Specifically, the entire dataset of 474 samples was first sorted in ascending order based on their Soil Organic Carbon (SOC) and Total Nitrogen (TN) content values. The sorted data was then systematically partitioned into four substrata by selecting every third sample. Finally, approximately three-quarters of the samples from each substratum were assigned to the calibration set, while the remaining one-quarter formed the validation set. XGBoost, RF, and PLSR prediction models were constructed for soil ecological stoichiometric characteristics. Training and validation of the three regression models were implemented using the Scikit-learn library in Python 3.9.

The accuracy of the machine learning models was evaluated by comparing the coefficients of determination ($R^2$), Root Mean Square Error (RMSE), and Mean Absolute Error (MAE). A larger R² and smaller RMSE and MAE indicate a higher prediction accuracy of the model [36,37].

$$R^2={\displaystyle \frac{\sum _{i=1}^n{(y-y_i)}^2}{\sum _{i=1}^n{(\overline{y}-y_i)}^2}}$$

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\displaystyle \frac{\sum _{i=1}^n{(y-y_i)}^2}{n}}}$$

$$\mathrm{M}\mathrm{A}\mathrm{E}={\displaystyle \frac{\sum _{i=1}^n\left|y_i-y\right|}{n}}$$

where $y$ represents the measured values of the soil element content, $y_i$ represents the model-predicted values of the soil element content, $\overline{y}$ is the average of the measured soil element content values, and n is the number of samples.

To ensure the robustness and optimal performance of the machine learning models, key hyperparameters were tuned. The key hyperparameters and their search ranges ultimately employed for each model are as follows:

eXtreme Gradient Boosting (XGBoost): The search ranges for key parameters were: the number of estimators (n_estimators) from 0 to 300; maximum tree depth (max_depth) selected from [3, 5–7, 9, 12, 15]; minimum child weight (min_child_weight) selected from [1, 3, 5, 7]; the regularization parameter (gamma) selected from [0, 0.05–0.1, 0.3, 0.5, 0.7, 0.9, 1]; and the subsampling ratios for instances (subsample) and features (colsample_bytree) both selected from [0.6, 0.7, 0.8, 0.9, 1].

Random Forest (RF): The search ranges for key parameters were: the number of trees (n_estimators) from 0 to 300; maximum depth (max_depth) selected from [3, 5–7, 9, 12, 15]; the number of features considered for splitting (max_features) from 5 to 20; the minimum samples required to split a node (min_samples_split) from 2 to 11; and the minimum samples required to be at a leaf node (min_samples_leaf) from 1 to 11.

Partial Least Squares Regression (PLSR): The number of latent components (n_components) was set to 2, as this value was determined to optimally explain the variance in the calibration dataset while avoiding overfitting.

Savitzky–Golay (SG) Filter: Spectral smoothing was applied using an SG filter with a window length set to 9 and a polynomial order of 3. This configuration effectively reduced high-frequency noise while preserving the essential shape of the spectral curves.

2.8. Model Interpretation Using SHAP Analysis

To interpret the prediction mechanisms of the optimal machine learning model and identify the most influential spectral bands, SHapley Additive exPlanations (SHAP) analysis was employed. SHAP is a game-theoretic approach that assigns each feature an importance value for a particular prediction, quantifying its contribution to the model’s output. The analysis was performed specifically on the best-performing Random Forest model under 0% moisture conditions using the SHAP Python library. The mean absolute SHAP value was calculated across all samples in the validation set to determine the overall importance of each wavelength. This analysis allowed for the identification of the key spectral bands driving the predictions of SOC and TN content, as well as the direction of their influence.

3. Results

3.1. Characteristics of Soil Spectral Curves Under Different Moisture Contents

The spectral data of the wetland soils with different moisture contents in the 350–2500 nm range were averaged, the reflectance data were smoothed, and the noise bands were removed using an SG filter. This resulted in the average reflectance spectra for wetland soils with five different moisture contents, as shown in Figure 1. The figure shows that the reflectance spectra of the soils with the five moisture gradients generally increased with wavelength, exhibiting a similar overall trend. However, specific differences were evident at the absorption peaks at approximately 1400 and 1900 nm, where the depth of the absorption peaks increased with increasing soil moisture content. Additionally, differences were observed in the overall reflectance levels; as the soil moisture content increased, the soil spectral reflectance decreased. Notably, the absorption peak at 2200 nm is almost identical across the five moisture levels and did not vary with moisture content, as did the previous two absorption peaks. Quantitatively, the average reflectance at 800 nm decreased from approximately 0.35 at 0% moisture to below 0.20 at 30% moisture, representing a reduction of over 40%. The absorption depth at 1400 nm and 1900 nm, key water absorption features, increased markedly with moisture content.

3.2. Continuous Projection Algorithm for Feature Bands

The SPA reduces multicollinearity among variables by iteratively selecting them through successive projections, thereby enhancing the robustness of the model. This method identifies the most representative variables, improving the predictive accuracy and interpretability of multivariate regression models. Compared to other feature selection methods, SPA efficiently performs dimensionality reduction on large-scale Vis–NIR spectroscopy data with high computational efficiency. The results of the feature bands selected using SPA are shown in Figure 2.

To achieve a good balance in terms of information retention, model stability, and generalisation ability and to enhance the effectiveness of feature band extraction and the accuracy of subsequent analysis, we set the minimum value of the number of selected variables, n, to 20. Consequently, we obtained feature bands for the original and FD spectral data for the five different moisture levels.

3.3. Accuracy Evaluation of Soil Element Content Model Validation Based on Full-Band Analysis of Different Moisture Contents

Based on the full-band raw and FD spectral data and the measured SOC and TN values, XGBoost, RF, and PLSR models were established for wetland soils with five moisture content gradients using 357 training samples. The model validation results are shown in Figure 3.

First, the prediction effectiveness of the three models for the SOC and TN elements was compared. The $R^2$ range for SOC was 0.18 to 0.58, whereas for soil TN, the $R^2$ range was 0.14 to 0.60, indicating that the prediction accuracy was relatively better for soil TN. Comparing the three machine learning models, the $R^2$ range for XGBoost was 0.23 to 0.53, for RF was 0.24 to 0.58, and for PLSR was 0.14 to 0.47. Among these, RF demonstrated the best prediction accuracy.

By comparing the accuracy performance of the raw and FD spectral data transformation types across different models, it was found that the $R^2$ values of the three models based on the FD spectra were higher than those based on the raw spectra for full-band models with different moisture gradients.

Comparing the inversion results of the models set for the five soil moisture gradients in this study, it was found that the inversion model based on the 0% moisture gradient had higher accuracy and stability than the models for the other four moisture gradients. In addition, the inversion accuracy of the models for the other four moisture gradients decreased as the moisture content increased.

3.4. Accuracy Evaluation of Soil Element Content Model Validation Based on Different Moisture Content Characteristic Bands

Based on the characteristic bands selected by SPA, raw and FD spectral data, and measured SOC and TN values, XGBoost, RF, and PLSR models were established for wetland soils with five moisture content gradients using 357 training samples. The model validation results are listed in

Table 1. Model validation accuracy evaluation of soil organic carbon and total nitrogen based on spectral characteristics of different moisture content bands.

Elements	Water Content	Model	Original			First Derivative
			Validation Set			Validation Set
			R2	RMSE	MAE	R2	RMSE	MAE
SOC/(g/kg)	0%	XGBoost	0.473	1.149	0.989	0.527	1.065	0.922
		RF	0.582	1.074	0.906	0.598	1.010	0.867
		PLSR	0.340	1.322	1.152	0.467	1.151	0.972
	5%	XGBoost	0.415	1.316	1.139	0.475	1.239	1.032
		RF	0.482	1.255	1.063	0.532	1.167	0.956
		PLSR	0.295	1.425	1.219	0.452	1.244	1.053
	10%	XGBoost	0.368	1.346	1.147	0.457	1.266	1.065
		RF	0.403	1.310	1.124	0.479	1.251	1.036
		PLSR	0.244	1.483	1.209	0.384	1.318	1.098
	20%	XGBoost	0.341	1.389	1.135	0.397	1.339	1.129
		RF	0.354	1.364	1.172	0.418	1.296	1.089
		PLSR	0.246	1.481	1.240	0.373	1.331	1.078
	30%	XGBoost	0.227	1.484	1.249	0.249	1.455	1.187
		RF	0.255	1.449	1.190	0.311	1.394	1.145
		PLSR	0.185	1.527	1.269	0.334	1.378	1.120
TN/(g/kg)	0%	XGBoost	0.577	0.119	0.090	0.648	0.116	0.092
		RF	0.606	0.129	0.106	0.693	0.114	0.090
		PLSR	0.322	0.162	0.138	0.590	0.128	0.104
	5%	XGBoost	0.389	0.148	0.118	0.569	0.122	0.098
		RF	0.479	0.131	0.105	0.585	0.135	0.103
		PLSR	0.269	0.171	0.139	0.539	0.138	0.110
	10%	XGBoost	0.311	0.167	0.134	0.492	0.134	0.109
		RF	0.394	0.158	0.130	0.516	0.133	0.112
		PLSR	0.205	0.182	0.151	0.464	0.141	0.119
	20%	XGBoost	0.296	0.170	0.136	0.435	0.152	0.121
		RF	0.327	0.165	0.136	0.427	0.150	0.125
		PLSR	0.196	0.183	0.154	0.369	0.156	0.128
	30%	XGBoost	0.256	0.174	0.137	0.397	0.160	0.125
		RF	0.308	0.169	0.136	0.457	0.150	0.113
		PLSR	0.142	0.187	0.156	0.294	0.170	0.135

.

First, comparing the prediction accuracy of the three models for SOC and TN, the $R^2$ range for SOC was 0.18 to 0.59, whereas for soil TN, the $R^2$ range was 0.14 to 0.69, indicating that the prediction accuracy was relatively better for soil TN. Comparing the three machine learning models, the $R^2$ range for XGBoost was 0.22 to 0.64, for RF was 0.25 to 0.69, and for PLSR was 0.14 to 0.59. The results showed that RF had the most stable and accurate prediction accuracy, and the accuracy of all three models improved compared to the full-band models.

Comparing the accuracy performance of the raw and FD spectral data transformation types across different models, the $R^2$ values of the three models based on the FD spectra for characteristic band models with different moisture gradients were higher than those based on the raw spectra.

Comparing the inversion results of the models set for the five soil moisture gradients in this study, the inversion model based on the 0% moisture gradient had higher accuracy and stability than the models for the other four moisture gradients. In addition, the inversion accuracy of the models for the other four moisture gradients decreased as the moisture content increased. Therefore, the RF model based on the FD spectral data of the characteristic bands for 0% moist soil exhibited higher inversion accuracy and stability.

Furthermore, as shown in Figure 4, the scatter plots of SOC and TN estimates using the two spectral characteristic bands and three models at the 0% moisture gradient indicated that the RF validation set points were more concentrated around the 1:1 line. This suggests that the RF model has higher accuracy than the other two models.

3.5. Interpretation of Prediction Mechanisms via SHAP Analysis

The SHAP analysis was applied to the optimal Random Forest model (0% moisture, FD spectra) to elucidate the contribution of individual spectral bands to the prediction of SOC and TN. The results indicated that 1865 nm was the most influential band for SOC prediction, where lower reflectance contributed more significantly to higher predicted values (Figure 5). For TN prediction, 1419 nm was identified as the most contributory band, with higher reflectance exerting a stronger negative contribution to the predicted value (Figure 6).

4. Discussion

While the inverse relationship between soil moisture and spectral reflectance is a well-established phenomenon, the primary novelty of this work lies not in rediscovering this relationship, but in providing a systematic, methodological framework to address its confounding effects. This study moves beyond mere observation by rigorously evaluating and identifying the optimal combination of spectral preprocessing, machine learning algorithms, and interpretability tools to maximize the prediction accuracy of soil carbon and nitrogen under variable moisture conditions.

4.1. Impact of Soil Moisture on the Accuracy of Soil Nutrient Retrieval

The spectral reflectance characteristics of the soil are influenced by its texture and physicochemical properties [38]. The complex composition, physicochemical properties, structure, and functions of soil result in a significant amount of interference in spectral information [39]. Notably, during the acquisition of soil spectral information, soil moisture, clay content, salinity, and temperature are the primary factors causing the appearance of absorption peaks in the spectra [40,41]. Soil moisture content affects spectral reflectance; specifically, as soil moisture content increases, soil reflectance gradually decreases. In this study, after converting the original soil spectra to FD data, Vis–NIR spectroscopy fitting models for SOC and TN content were constructed using the XGBoost, RF, and PLSR methods. The dry-soil model with 0% moisture content exhibited the highest fitting accuracy. This phenomenon may occur because soil moisture masks the spectral features of soil nutrients (such as carbon and nitrogen), leading to differences in spectral bands and affecting soil spectral reflectance.

In addition, because soil moisture has a high absorption coefficient in the near-infrared band, it significantly interferes with the detection of soil nutrient content. Under different moisture conditions, the influence of moisture on the spectrum varied across bands, but significant differences were observed at approximately 1400, 1900, and 2200 nm. This is because of the well-defined water absorption bands near 1400 and 1900 nm [42]. When the soil moisture content is low, its surface roughness may increase, leading to more scattered light being reflected towards the detector, thereby increasing the reflectance. Conversely, when the soil moisture content is high, the dielectric constant of the soil increases, affecting the interaction between the soil and electromagnetic waves, and leading to more light absorption and reduced reflectance. These findings are similar to those of Muller et al. [43]. and Croft et al. [44], suggesting that the lower the soil moisture content, the stronger the ability to predict soil carbon and nitrogen contents using soil spectral features. Future research should focus on eliminating the impact of moisture on the Vis–NIR spectral estimation of soil elements.

4.2. Differences in Element Content Inverted from Vis–NIR Spectroscopy Data

Vis–NIR spectroscopy technology determines the chemical composition and relative content of substances based on spectral reflectance to identify materials. Early studies indicated that visible spectra are primarily produced by outer electron transitions, whereas near-infrared spectra are mainly influenced by molecular vibrations, reflecting the composition and structure of molecules [45]. Soil carbon and nitrogen content are considered soil properties that can enhance spectral reflectance intensity [46]. The successful prediction of carbon and nitrogen contents is mainly because of their direct spectral response in the Vis–NIR spectrum, which is caused by the overtones and combinations of N-H, O-H, C-H+C-H, and C-H+C-C [47,48,49]. Therefore, using Vis–NIR spectroscopy technology to establish models for soil carbon and nitrogen content allows for the rapid estimation of these contents with high prediction accuracy.

4.3. Differences in Inversion of Different Vis–NIR Spectroscopy Data of Wetland Soil

Vis–NIR spectroscopy data are vast and exhibit strong correlations between bands. Before performing data analysis with Vis–NIR spectroscopy data, feature selection methods were used to extract characteristic bands. Eliminating bands unrelated to carbon and nitrogen content could significantly reduce model complexity. This is an important aspect of simplifying Vis–NIR spectroscopy data models [50,51,52]. Using SPA, the number of bands in the raw and FD spectral data was reduced to approximately 20, greatly decreasing the data dimensionality. Additionally, the characteristic bands of the raw and FD spectral data were mainly concentrated in the peaks and troughs where reflectance changes were more pronounced (Figure 2), indicating that the characteristic bands extracted by SPA retained the spectral features well. The models built based on the characteristic bands calculated using SPA showed significantly higher accuracy than the models that did not use SPA. The use of the SPA significantly reduced the model complexity and computational load while improving the model accuracy. Therefore, combining SPA with machine-learning models could be an effective method for rapidly predicting soil carbon and nitrogen content.

4.4. Interpretation of Prediction Mechanisms Through SHAP Analysis

The band contributions for Soil Organic Carbon (SOC) and Total Nitrogen (TN) prediction by the Random Forest model under 0% moisture conditions were analyzed using SHAP values (Figure 5 and Figure 6). The SHAP analysis identified 1865 nm and 1419 nm as the most influential bands for SOC and TN prediction, respectively. It is critical to note that these wavelengths coincide with strong water absorption features. Therefore, their high importance in the model likely reflects the residual coupling between soil moisture and the spectral features of carbon and nitrogen constituents, rather than representing direct, fundamental spectral features of SOC or TN molecules themselves. This underscores the challenge of disentangling moisture effects from nutrient signals, even in models trained on dried samples (0% moisture), and highlights that these models may still be leveraging covariances related to the soil’s water-holding capacity or organic matter structure.

For TN inversion,1419 nm was identified as the most influential band, where higher reflectance exerts a stronger negative contribution to the predicted value 795 nm is the next most contributory band, with higher reflectance yielding a positive contribution to the prediction.

4.5. Impact of Machine Models on the Accuracy of Wetland Soil Carbon and Nitrogen Inversion

This study utilised three regression models to explore the inversion of soil carbon and nitrogen content. The RF and PLSR models have been widely used to predict soil elemental content, confirming their versatility (

Table 2. Summary of literature results on the prediction of soil carbon and nitrogen elements. Abbreviations: R², Coefficient of determination; RMSE, Root Mean Square Error (units: g/kg for TN and SOC unless otherwise stated); RPD, Ratio of Performance to Deviation; OR, Original Reflectance; FD, First Derivative; PLSR, Partial Least Squares Regression; RF, Random Forest.

Element	Accuracy	Data Type	Model	Author
TN	$R^2$ = 0.921 RMSE = 0.086 RPD = 2.59	Full Spectra-OR	PLSR	Li et al., 2019 [53]
	$R^2$ = 0.915 RMSE = 0.089 RPD = 2.51	SPA-OR
TN	$R^2$ = 0.757 RMSE = 0.235	OR	RF	Lin et al., 2022 [49]
TN	$R^2$ = 0.355 RMSE = 0.019 RPD = 1.245	OR	PLSR	Pechanec et al., 2021 [56]
SOC	$R^2$ = 0.950	FD	RF	Wang et al., 2022 [62]
SOC	$R^2$ = 0.440 RMSE = 0.070 RPD = 1.570	OR	PLSR	Mondal et al., 2019 [63]
SOC	$R^2$ = 0.740 RMSE = 0.159 RPD = 1.780	FD	PLSR	Ribeiro et al., 2021 [64]

). The XGBoost model, an emerging ensemble learning algorithm known for its fast training speed and high fitting accuracy, has been successfully applied in fields such as salinity inversion [53] and chlorophyll-a concentration inversion [54] but is less commonly used in soil nutrient inversion studies. The RF model is an ensemble learning algorithm primarily used for classification and regression tasks. It trains and predicts samples by constructing multiple decision tree models [55]. When building each tree, Random Forest randomly selects a specific number of features at each node split, rather than using all features, which increases model diversity and reduces overfitting [56,57]. The PLSR model combines the advantages of principal component analysis, correlation analysis, and linear regression, effectively removing multicollinearity and handling high-dimensional data [58,59]. The XGBoost model is an ensemble learning algorithm based on tree models, optimized and improved using the gradient boosting algorithm. It introduces regularization terms to control model complexity and prevent overfitting. Additionally, it can automatically handle missing data by learning the optimal split direction for missing values, thereby enhancing the model’s robustness [60,61].

The comparatively lower predictive accuracy of the Partial Least Squares Regression (PLSR) model, relative to the tree-based ensemble methods, can be primarily attributed to its inherent linearity. PLSR is designed to identify linear relationships between spectral data and soil properties. However, the introduction of varying soil moisture creates complex, non-linear interactions and overlaps within the spectral data, which a linear model struggles to adequately capture. In contrast, tree-based models like Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) excel in this context due to their inherent capability to model these non-linear relationships and complex feature interactions through their hierarchical, decision-making structure. Between the two non-linear ensemble methods, the Random Forest (RF) model demonstrated superior robustness and overall accuracy compared to XGBoost. This can be explained by key architectural differences. RF employs “bagging” (Bootstrap Aggregating), which builds each tree on a random subset of data and features. This introduces high model diversity and effectively reduces variance, making it particularly robust to noisy data and complex spectral patterns. While XGBoost is a powerful “boosting” algorithm that sequentially corrects errors, it can be more sensitive to such noise and may require more extensive hyperparameter tuning to achieve optimal generalization. The inherent stability of the RF algorithm likely rendered it more adaptable to the challenging spectral variations induced by moisture in this study.

This study demonstrated R² values ranging from 0.30 to 0.69 for the best-performing models (Table 1), which are comparatively lower than the high R² values reported in some previous studies under ideal dry conditions (Table 2). This discrepancy does not indicate a failure of our methodology, but rather underscores the central challenge addressed in our work: the confounding effect of soil moisture. The wide range of R² values in the literature, from as low as 0.44 to as high as 0.95, can be attributed to several critical factors:

• Soil Moisture Content: This is the most significant factor. Studies achieving near-perfect prediction were almost exclusively conducted on oven-dried or air-dried soils. By eliminating moisture’s masking effect, these models capture the purest spectral signal of SOC and TN. In contrast, our study intentionally introduced moisture gradients (0–30%) to simulate a more realistic field scenario, where water absorption features overwhelm the weaker spectral features of carbon and nitrogen, inevitably leading to a reduction in predictive accuracy.
• Sample Size and Diversity: The representativeness of the calibration dataset greatly influences model performance. Studies with a large number of samples (n) covering a wide range of soil types, textures, and land uses tend to build more robust but potentially less precise models (with lower maximum R²), as they must account for greater inherent variability. Studies on smaller, more homogeneous datasets can achieve very high accuracy for that specific context but may lack generalizability.
• Model and Preprocessing Choices: The choice of algorithm and spectral preprocessing significantly impacts results. Nonlinear models like RF and XGBoost, as demonstrated in our work, are better suited to handle the complex, non-linear interactions introduced by moisture compared to linear models like PLSR.

Therefore, the key contribution of this study is not in achieving the highest possible R² under ideal conditions, but in systematically quantifying the performance of different models across a gradient of soil moisture—a prevalent and disruptive condition in real-world applications. Our finding that the RF model on first-derivative spectra of dry soil yielded the most accurate and stable predictions provides a critical benchmark and a practical methodological framework for future applications where sample drying is feasible. For in-situ sensing, our results clearly delineate the expected performance loss with increasing moisture, which is vital for setting realistic expectations in precision agriculture and environmental monitoring.

5. Conclusions

This study systematically evaluated the optimal strategy for predicting soil carbon and nitrogen content under varying moisture conditions. By comparing different preprocessing techniques and machine learning models on soil samples with five moisture levels, we established the soil carbon and nitrogen content at different moisture levels. The conclusions are as follows.

• The soil spectral reflectance values gradually increased as the soil moisture content decreased, with the 0% moisture content prediction model consistently exhibiting better accuracy than other moisture levels.
• By comparing the validation accuracy of models based on raw and FD spectra, it was found that the estimation models for SOC and TN content built on FD spectra had higher accuracy.
• The RF model based on SPA-selected characteristic bands had a validation $R^2$ range of 0.30–0.69, demonstrating higher inversion accuracy and greater stability.
• The SHAP analysis confirmed 1865 nm and 1419 nm as the most contributory bands for SOC and TN prediction respectively, validating the RF model’s spectral interpretation capability.

In conclusion, the principal contribution of this study is methodological. We systematically demonstrated that the combination of first-derivative spectral preprocessing, the Random Forest algorithm, and feature selection via SPA presents the most robust framework for estimating SOC and TN in wetland soils when confronted with the challenge of variable moisture content. Furthermore, the application of SHAP analysis provided critical, interpretable insights into the prediction mechanisms, identifying key spectral bands and thereby validating the model’s decision-making process. This integrated approach offers a practical and insightful pathway for enhancing the application of Vis–NIR spectroscopy in real-world scenarios where soil moisture cannot be strictly controlled.