3.2. Spatial Trend Modeling
Each model utilized different terrain attributes for SOM and CEC predictions. A backwards stepwise linear model selection was used for the UK model. Based on the following equations (Equations (6) and (7)), the backwards stepwise model selected TWI, TPI, and MrRTF for SOM, and TPI and MrVBF for CEC predictions.
From a pedological standpoint, Equations (6) and (7) reveal meaningful relationships between terrain and SOM or CEC. Equation (6) shows that SOM is positively correlated to TWI or wet/low-lying areas of the landscape, while it is negatively correlated to TPI and MrRTF or higher/steeper areas of the landscape. Similarly, CEC (Equation (7)) is negatively correlated with TPI, but positively correlated with MrVBF or lower landscape positions.
Cubist utilized all four terrain attributes for SOM, and only TPI and MrVBF for CEC predictions. Out of ten models generated by Cubist for SOM and CEC predictions, we selected the models with the lowest prediction error. For example, the SOM model (Equation (8)) was applicable in 123 locations where the average SOM was 4.09% with a prediction error of 0.70%. The CEC model (Equation (10)) applied to the 123 training locations had a mean value of 20.12 cmol
c kg
−1 and predication error of 2.84 cmol
c kg
−1. The Cubist model provided slightly different models for SOM prediction based on the combination of four terrain attributes but produced identical models for CEC. Following are examples of Cubist models for SOM and CEC predictions.
The Cubist model also provided the relative importance (RI) of the terrain attributes in developing conditions rules (if then else rules) and in developing multivariate linear function. In SOM prediction, the RI of the four terrain attributes was 54% (TPI), 37% (MrRTF), 35% (TWI), and 34% (MrVBF) in the MLR function and 54% for TWI in rule conditions. Therefore, TWI is the most important predictor for the Cubist model for SOM prediction.
For CEC, the Cubist model did not estimate conditional rules and returned only a single regression equation (Equation (10)). Not surprisingly, due to the lack of conditional rules, the equation from Cubist is almost identical with the one from stepwise linear regression (Equation (7)). Subsequently, the UK and Cubist predictive models produced similar results for CEC (
Table 3). This suggests that for our study area, the additional complexity of Cubist models is unwarranted as the relationship between soil properties and landscape can be described by a single regression equation without the need for complex conditional rules.
Random forest used all four terrain attributes for predicting both SOM and CEC. The
varImpPlot function in the
randomForest package 4.6.14 [
51] shows the importance of terrain attributes for SOM and CEC predictions. For the RF predictions the most important terrain attributes for SOM predictions were TPI and TWI, while for CEC predictions TPI and MrVBF (
Figure 4). Overall, TPI was the most important variable and MrRTF was the least important variable in all selected models.
Both the UK and Cubist models show that SOM and CEC increase as TWI and MrVBF increase and decrease as TPI and MrRTF values increase. Even though, RF utilized all four terrain attributes for SOM and CEC predictions, TPI and MrVBF were among the most important variables for both SOM and CEC, similar to UK and Cubist (Equations (6)–(10)). Ref. [
17] reported that among the five terrain attributes (elevation, TWI, plan curvature, total catchment area, and channel network base level), TWI and total catchment area closely related to the soil organic carbon content in flat slope areas. They also stated that besides these two terrain attributes, soil nutrient indicators were other environmental covariates that showed close correlation with soil organic carbon content. Ref. [
43] found that wetness index and MrVBF were among the most important terrain attributes that influenced spatial distribution of SOM. They also reported that beside terrain attributes, land use, soil type and precipitation were other environmental variables that influence SOM distribution. Ref. [
54] also documented the influence of land use, soil type, and geology on soil organic carbon distribution. They also noted that after elevation, TWI was the second most important terrain attribute in soil organic carbon prediction. The influence of soil type, geology, elevation, and slope on soil organic carbon distribution in wet cultivated fields was also reported by [
11] who found that most of the soil organic carbon variation at the surface (0–10 cm) was explained by topographic attributes, while in the subsurface (10–50 cm) it was best estimated by soil texture classes.
TWI had correlation of 0.49 with SOM, 0.43 with CEC, −0.56 with TPI, 0.01 with MrRTF, and 0.67 with MrVBF. TPI had correlation of −0.51 with SOM, −0.53 with CEC, −0.52 with MrVBF, and 0.19 with MrRTF. MrVBF had correlation of 0.46 with SOM, 0.52 with CEC, and 0.04 with MrRTF. MrRTF showed a correlation of −0.21 with SOM, −0.06 with CEC. Suleymanov et al. (2021) [
33] reported 0.5 correlation between SOM and MrRTF and MrRTF was a key attribute for mapping soil organic carbon and thickness of humus. Ref. [
15] also observed a correlation of 0.5 between soil properties including soil organic carbon and auxiliary information. Ref. [
20] reported lower correlation (<−0.38) between DEM and soil attributes including CEC, and moderate to strong correlation (<0.60) between satellite bands of synthetic soil image and soil attributes. The legacy soil map showed better correlation (0.5) with CEC compared to Landsat 5 TM derived attributes (<0.44) in the flat landscape of a semiarid region of Brazil [
14]. Similar to [
17], we found that based on the whole dataset, SOM has a strong correlation with CEC (0.73).
3.4. Predictive Model Performance
Based on the 4 evaluation metrics (
Table 5) and scatter plots of measured versus predicted SOM (
Figure 5) and CEC (
Figure 6), we found no major differences in the prediction performance of all three models.
Also [
19] found no major difference and reported similar R
2 and RMSE values for soil organic carbon estimation based on Cubist and RF. Similar conclusion was reported by [
17], who noted that RF and Cubist showed similar predictive performance in soil organic carbon estimation.
For the calibration data, RF had lower RMSE, lower bias, higher R
2, and higher concordance than UK and Cubist for both SOM and CEC predictions. With RF, this was expected due to the ensemble approach, which can result in low bias and variance [
59,
60]. Ref. [
17] reported that predicted and observed values of soil organic carbon on the RF scatter plot were closer to the central or 1:1 line compared to Cubist, artificial neural network, support vector machine, and multiple linear regression models. For the RF models, however, there was a significant change in model performance between calibration and evaluation data. Thus, the performance decreased by half during the evaluation and was comparable with the performance of UK and Cubist. For example, for SOM predictions, the R
2 of the RF prediction were 0.90 for the calibration dataset and decreased to 0.45 for the evaluation dataset which was similar to UK (0.44) and Cubist (0.45). This significant change between calibration and evaluation performance is strong evidence that the RF models may be over optimistic. This highlights one of the issues with modeling for DSM: if models are not evaluated rigorously (i.e., using independent evaluation rather than leave-one-out), model performance estimates may be overly optimistic. Ref. [
43] also documented higher R
2 and lower RMSE based on calibration dataset compared to validation dataset. Therefore, we agree that an independent validation dataset is necessary for DSM prediction performance evaluation [
54,
61].
According to the scatter plots for SOM (
Figure 5a–c) and CEC (
Figure 6a–c), all three models tended to over predict at low values and under predict at high values. This behavior is less pronounced for RF models on the calibration data, but it is apparent for all models on the evaluation dataset. This lack of performance for all models may be due to several factors, which are discussed below.
One reason for this poor correlation may be the due to the history of the study location. ACRE serves as a research and education facility and consists of many smaller individual fields that are managed under highly variable practices (e.g., multiple tillage systems, nutrient application rates, and crop rotations). For example, ACRE is crisscrossed by a grid of roads and grassed field boundaries that are on average 20 cm higher than the adjoining fields, and by a dense network of underground drainage tiles. The impact of the roads on terrain attributes is evident as linear features in the terrain attribute maps (
Figure 2). The high variation in management may lead to higher soil variability from field to field than expected. Training models using samples from highly variable fields can limit their predictive performance outside the sample areas [
62]. To account for the effects of variable management history, we would need to incorporate environmental covariates that describe previous management of each field into our modeling framework. This sort of detailed field-by-field management history has not been recorded for the entire farm. Recent research from [
63] conducted on a long-term soil monitoring network in Switzerland has highlighted the role of land use change, crop rotations and site conditions on soil organic carbon dynamics. Further research is needed to identify suitable covariates that describe agriculture management history [
64].
Compounding the land use and management history, the relatively flat topography with subtle topographic variation (on average 1% slope based on a 3 × 3 pixel window) may also have contributed to relatively poor performance of all models. In many environments, chemical properties of surface soil and SOM are highly variable spatially, and distinct variations are often found within short distances of meters and/or decimeters [
65,
66]. Thus, the intensive land use and management history combined with relatively flat terrain may have diluted the influence of terrain in the distribution of SOM and CEC leading to poorer than expected model performance. Although the R
2 for evaluation were relatively low, ranging from 0.39 to 0.45 in our study, they were comparable with other studies that considered terrain/climatic data only [
16,
50,
67]. Ref. [
16], argues that for quantitative soil spatial models, R
2 values of 0.5 and less are not uncommon. The fact that models were based only on terrain attributes supports the idea that topography at this scale is still one of the major factors for predicting soil properties despite management. The RMSE values of all three models were lower compared to [
11], which used RF for soil organic carbon prediction. They found a higher RMSE (1.72%) for the surface soil (0–10 cm) and lower (0.43) at the subsurface (10–50 cm). This suggests that the spatial distribution patterns of soil organic carbon particularly at the topsoil are highly variable. Ref. [
14] reported that based on validation dataset, geostatistical model (i.e., Cokriging) provided better results (R
2 = 0.57 and RMSE = 7.22) compared to the RF (R
2 = 0.47 and RMSE = 7.89) in CEC prediction. However a higher performance of kriging over RF and cubist models on the prediction of soil organic carbon at field-scale was also reported by [
68]. Unlike in our study, a high-sampling density would have favored kriging in their case.
3.6. Predictive Models versus SSURGO
When comparing maps of SSURGO SOM and CEC to DSM maps, all maps generally show a similar trend: high SOM and CEC occurred on lower landscape positions (
Figure 7 and
Figure 8). Where these maps differ from SSURGO is in the extent of regions of high SOM and CEC and the level of detail within SSURGO map units. In SSURGO, the regions or map units of high SOM and CEC are larger in extent compared to the DSM maps. Generally, SSURGO overrepresented the areas with high SOM and CEC (
Figure 7 and
Figure 8). For example, SSURGO representative values had a median SOM of 4.9% compared to 4.1 and 4.2% for DSM maps (
Table 6). Similarly, the SSURGO mean values for CEC had a median of 28.5% while DSM maps had a median between 19.4 and 20.0% (
Table 6). Additionally, the standard deviation of SSURGO SOM (1.2%) and CEC (6.6%) maps are higher compared to DSM Maps, which is less than 0.8% for SOM and 2.8–3.3% for CEC (
Table 6). Some of the major reasons for the overrepresentation of the areas with high SOM and CEC are the scale 1:15,840 [
24] of SSURGO mapping and the design of map units. The density of the point data combined with high resolution topographic data provides a more detailed map of the SOM and CEC distribution compared to SSURGO. Also, SSURGO SOM and CEC capture the variability of the soil property within a larger and much more generalized map units thus tend to overrepresent the ranges and their extent within individual soil polygons.
One interesting area of agreement between SSURGO and DSM maps is for CEC predictions in the southern quarter of the study area. In this area, both SSURGO and DSM models predicted the lowest CEC values. Even though sampling points were not concentrated at this part of the study site (
Figure 1), DSM models still managed to predict these regions of low CEC. Low DSM-derived CEC predictions likely resulted from the low TPI in the study areas (see the spatial trend modeling section). While SSURGO was not developed using TPI specifically, SSURGO mapping did rely heavily on relationships between soils and slope positions, which TPI captures numerically. Agreement between DSM-predicted CEC and SSURGO maps highlights the importance of soil-landscape relationships in soil spatial distributions, even at field scale.
We compared SOM and CEC predicted by DSM techniques to SOM and CEC from the SSURGO soil map. According to
Table 5, the prediction performance (i.e., R
2 and concordance) of SSURGO was analogous to the DSM prediction, however, SSURGO has higher bias and RMSE particularly for CEC prediction. It is also interesting that SSURGO show slightly better results for validation data when compared to calibration data. We also compared SOM and CEC contents predicted by DSM to the SOM and CEC contents within each map unit from SSURGO (
Figure 9). Both SOM and CEC show that the three predictive models follow similar prediction trends in each of the SSURGO mapping units. Based on visual interpretation of boxplots (
Figure 9a–c), the results of our models for SOM are consistent with the estimates from eight SSURGO mapping units; exceptions were CwB2, McS2, RoB, SwA, TfB, and TmA. SSURGO underrepresented the SOM for these map units while the other models predicted greater concentrations of SOM. Generally, SSURGO had a wider range in SOM and CEC values when compared to the prediction models (
Figure 9). This was particularly the case for CEC estimates. The prediction of our models for CEC is consistent with only few of the SSURGO mapping units see: RcA, RoB, SwA, TfB, and TmA (
Figure 9d–f). Based on visual inspection of the boxplots (
Figure 9d–f), however, for most of the mapping units, our models either over- or under-predicted CEC.
There are several reasons for the inconsistencies of model predictions with SSURGO. First, SSURGO has inherent limitations; the soil variability is represented using aggregated polygon map units with one to four named components plus inclusions of other soils or non-soils areas that do not explicitly capture the underlying spatial variability of soils within polygons [
69]. Thus, these inclusions reduce the purity of the map units and impact interpretation and modeling [
70]. Second, the procedure for SOM analysis differed between the datasets. The Walkley-Black method was used for the SSURGO data, while the loss-on-ignition (LOI) method was used for our collected data. Due to incomplete digestion of soil organic carbon, the Walkley-Black method usually underestimates SOM [
71,
72]. Additionally, the SSURGO values might have been impacted by errors introduced by the spline interpolation. Third, the SSURGO database was developed based on historical soil survey data and may not accurately reflect the current status of soil properties, particularly SOM and CEC, which are relatively dynamic and altered by various factors such as land management, climate change, and seasonal variability [
9,
73,
74]. Additionally, the data were produced over different time periods and therefore inherit inconsistencies [
4]. A fourth reason for the inconsistency is that the surveyors who collected data for SSURGO may not have had enough soil observations for building their mental models of soil variability as the detailed scale that lidar may suggest. A fifth reason for the inconsistency is that SSURGO values are not purely derived from laboratory analysis, instead the data may have resulted from a combination of laboratory measurements and field observations of expert soil scientists [
38,
75]. Due to these shortcomings, using SSURGO data in quantitative modeling and/or for monitoring soil carbon stocks sequestration could be misleading, particularly at the farm scale for highly variable soils in post glaciated landscapes.