Cross-Regional Hyperspectral Estimation of Soil Organic Carbon in Eurasian Black Soils Using an Optimal Spectral Feature Set

Abstract

Soil organic carbon (SOC) plays a critical role in the global carbon cycle and agroecosystem productivity. However, existing hyperspectral inversion models often exhibit significant predictive biases when applied across large geographic scales, primarily due to the spatial heterogeneity of pedogenic environments and background mineralogy. This study proposes a cross-regional SOC prediction method based on an optimal spectral feature set (SOC-OSFS). Leveraging laboratory hyperspectral and SOC data from 17,730 samples collected across the black soil regions of Northeast China and Europe, a core spectral feature set comprising 31 diagnostic bands was extracted using the competitive adaptive reweighted sampling (CARS) algorithm combined with the successive projections algorithm (SPA). Although this SOC-OSFS accounts for merely 1.55% of the original full-spectrum dimensionality (31 out of 2000 bands), it demonstrated robust analytical capability in local modeling across all study regions, yielding coefficients of determination (R² = 0.6714–0.8854). When transferring the prediction model calibrated in the core source domain (n = 10,000) to the other seven independent typical black soil target domains, the direct cross-regional prediction consistently reduced the root mean square error (RMSE) by over 15% compared to that of the full-spectrum models. By further incorporating 20% of the local background samples for intercept correction, the cross-regional predictive accuracy was substantially improved; the goodness-of-fit for the Northeast China target domains increased sharply (maximum R² = 0.8567), and the European target domains, which feature substantially different pedogenic environments, were successfully corrected from negative to positive linear fits. This study validates the efficacy of extracting physiochemically meaningful spectral bands in mitigating the interference caused by spatial heterogeneity, thereby providing a mechanistically grounded and practically viable framework for large-scale SOC estimation via remote sensing.

1. Introduction

As an invaluable global agricultural soil resource, black soil not only serves as a crucial foundation for ensuring food security but also acts as a vital carbon sequestration pool in terrestrial ecosystems [1,2,3,4]. However, influenced by climate change and long-term agricultural cultivation, the surface soil organic carbon (SOC) in black soil regions is facing a severe risk of depletion [3,5,6,7]. Fluctuations in its carbon sink capacity are intricately linked to the sustainability of regional agriculture and feedback to the climate system [8,9]. Therefore, exploring SOC estimation methods that transcend geographical boundaries holds significant practical implications for the conservation and dynamic monitoring of black soil resources [10,11].

In recent years, hyperspectral remote sensing has made substantial progress in SOC inversion [12,13,14]. However, existing estimation models often exhibit inherent spatial limitations when applied to study areas with large geographical spans [15,16,17,18]. This degradation in predictive performance primarily stems from the spatial heterogeneity of soil spectral responses: the interactive effects of varying pedogenic parent materials, soil textures, and background mineralogy across different geographical units lead to significant shifts of soil samples within the spectral feature space [19,20]. Because current modeling paradigms heavily rely on locally representative samples, they are prone to predictive biases when encountering heterogeneous pedogenic environments, which to some extent constrains the cross-regional synergistic utilization of soil spectral library resources [17,21,22].

Regarding hyperspectral estimation methods, early studies mostly attempted to simplify the quantitative estimation of SOC by constructing empirical spectral indices [23,24]. Although these methods possess a certain degree of physical interpretability, their simplistic structures make it difficult to comprehensively capture the latent physicochemical characteristics within high-dimensional spectral data [24,25,26]. With the evolution of data processing technologies, the research focus has gradually shifted towards spectral feature selection [27,28,29]. However, within a high-dimensional spectral space containing thousands of bands, a massive number of redundant variables not only capture the characteristic responses of organic matter but are also heavily confounded by environmental background noise dominated by soil moisture, iron oxides, and clay minerals [30,31,32,33]. How to effectively decouple these varying background interferences and extract feature bands supported by explicit physicochemical mechanisms—thereby mitigating the spatial heterogeneity issues caused by pedogenic differences—remains a crucial research direction that requires in-depth exploration in current hyperspectral soil property inversion [31,32,34].

Recently, various state-of-the-art techniques, including advanced deep learning architectures and complex transfer learning frameworks, have been increasingly applied to enhance hyperspectral SOC predictions [22,35,36,37,38]. While these sophisticated data-driven models have achieved high predictive accuracy at local or regional scales, with reported R² values often exceeding 0.80 under controlled conditions [39,40,41], they often lack explicit incorporation of physicochemical mechanisms underlying soil spectral responses [37,42]. Furthermore, when directly extrapolated to macro-scales (e.g., transcontinental levels), these highly complex models frequently suffer from severe performance degradation due to unmitigated systematic biases caused by varying pedogenic environments [43,44,45]. In contrast to these computationally intensive and data-demanding paradigms, which primarily rely on statistical learning from high-dimensional spectral space, our study advocates for a more transparent and mechanism-driven approach. By explicitly decoupling background noise through an optimized feature selection strategy prior to modeling, and coupling it with a highly efficient local sample fine-tuning (spiking) technique, our framework provides a robust and practically feasible solution for mitigating transcontinental prediction biases.

To address the limited spatial applicability of existing models [22], this study takes the black soil regions of Northeast China and Europe as research targets, aiming to explore the spectral response patterns of SOC across geographical boundaries and provide a feature extraction-based framework for cross-regional inversion. Rather than developing new fundamental algorithms, the primary contribution of this study lies in the integrated application of established methods to mitigate systematic biases in cross-regional SOC prediction. First, this study introduces the competitive adaptive reweighted sampling (CARS) and successive projections algorithm (SPA) to strip away environmental background noise from the full-spectrum data [46], extracting an optimal spectral feature set for SOC that is supported by the physicochemical mechanisms of macromolecular functional groups [47,48,49]. Second, by comparing the spectral distributions of the full-spectrum data and the optimal feature set via principal component analysis (PCA) [18,50,51], we verify the role of feature band extraction in suppressing background mineral interference and achieving cross-regional feature space realignment. Finally, multi-algorithm comparative experiments are designed to evaluate the performance of the cross-regional prediction method, which is based on the optimal spectral feature set combined with a small number of local target samples for intercept correction (spiking) [16,34,52,53]. This study aims to provide a novel and practical framework for mitigating the impact of pedogenic environmental differences on hyperspectral inversion models, by integrating physically interpretable spectral features with cross-regional transfer strategies for large-scale SOC estimation.

2. Materials

2.1. Study Areas

To investigate the stability of spectral characteristics of soil organic carbon (SOC) across different geographical spatial scales, this study focused on typical black soil regions to construct a hierarchical soil spectral dataset encompassing three progressive scales: “local–regional–intercontinental.”

The Hailun (HL) region, located in the Northeast China black soil belt, was selected as the core study area. Situated in the core zone of cold-region black soils in the Songnen Plain, the primary soil type in this region is typical thin-layer Chernozem [54]. The HL dataset comprises 10,000 topsoil samples, featuring an extensive sample size and a comprehensive SOC concentration gradient, thus serving as the primary source domain for spectral feature extraction.

Seven other typical black soil domains were selected as cross-regional target (validation) sets, including four in Northeast China and three in Europe. The target regions in Northeast China encompass Baoqing (BQ), Qiqihar (QQH), Fuyuan-Fujin (FYFJ), and Jilin (JL), with soil types covering typical black soils, sandy black soils, and meadow black soils (Phaeozems). The European black soil target regions comprise Austria (AT), the Czech Republic (CZ), and Hungary (HU), predominantly characterized by Chernozems [54].

2.2. Data

A total of 17,526 samples from the Northeast China black soil belt were collected during field campaigns in 2017, including HL (n = 10,000), BQ (n = 6361), QQH (n = 651), FYFJ (n = 397), and JL (n = 117). Specifically, the raw dataset in the HL region was subjected to strict quality control to remove samples with anomalous spectral responses and SOC measurement errors, resulting in a final dataset of 10,000 valid samples for subsequent analysis. The SOC contents of these soil samples were analyzed using the laboratory potassium dichromate volumetric method (DZ/T 0279), which shares the same analytical principle as the Walkley-Black method [55,56]. To ensure the consistency of spectral quality [31,32,57], soil sample preparation (air-drying, grinding, and sieving) and spectral measurements strictly followed standardized laboratory protocols under consistent environmental conditions, effectively minimizing the influence of soil moisture variability and surface heterogeneity on spectral measurements [2,45,58]. Spectral measurements were acquired using two distinct spectrometers: the ASD FieldSpec^® 3 (FS3) and the ASD FieldSpec^® 4 (FS4) (Analytical Spectral Devices, Inc., Boulder, CO, USA). To eliminate systematic instrumental biases, the Internal Soil Standard (ISS) method was applied to align the FS4 data to the FS3 standard [31,32,58,59], thereby minimizing potential systematic discrepancies between instruments and ensuring cross-sensor comparability.

The 204 European black soil samples utilized in this study were extracted from the LUCAS 2009 topsoil database [60], comprising AT (n = 40), CZ (n = 29), and HU (n = 135). The spectral data of these samples were resampled to a 1-nm interval to align spectrally with the Chinese dataset.

The spatial distribution of the aforementioned 17,730 black soil sampling sites is illustrated in Figure 1. For all spectral data, the marginal noisy bands (350–400 nm and 2401–2500 nm) were truncated, retaining the core wavelength range of 401–2400 nm. Subsequently, the Savitzky–Golay (SG) smoothing filter was applied across the full spectral dimensions to enhance the signal-to-noise ratio [61].

3. Methods

3.1. Spectral Transformation Methods

To enhance data robustness and isolate the optimal spectral responses of SOC, this study compared four spectral transformation forms: raw reflectance (R), first derivative (R′), continuum removal (C), and absorbance (A) [48]. Preliminary experimental results based on the core study area (HL) (Figure 2) demonstrated that the Partial Least Squares Regression (PLSR) and Random Forest (RF) models achieved the highest accuracy and the lowest error after applying the first derivative transformation. Consistent with previous findings [62], the first derivative (R′) effectively mitigates the baseline drift inherent in black soil backgrounds and amplifies the characteristic absorption peaks of SOC, thereby improving the sensitivity of the spectra to SOC content. Consequently, the first derivative (R′) was selected as the foundational spectral form for subsequent feature extraction and model construction. The calculation is expressed as Equation (1):

$$R^{\prime }\left({\lambda }_i\right)={\displaystyle \frac{R\left({\lambda }_{i+1}\right)-R\left({\lambda }_{i-1}\right)}{2\Delta \lambda }}$$

(1)

where ${\lambda }_i$ represents the wavelength of the $i$-th band; $R^{\prime }\left({\lambda }_i\right)$ is the first derivative value at wavelength ${\lambda }_i$; $R\left({\lambda }_{i+1}\right)$ and $R\left({\lambda }_{i-1}\right)$ are the original reflectance values at the wavelengths immediately adjacent to ${\lambda }_i$; and $\Delta \lambda$ is the wavelength interval between adjacent bands.

3.2. Feature Space and Feature Band Extraction

3.2.1. Feature Space Construction and Visualization

To quantitatively assess the similarities and disparities in SOC spectral responses across different regions, Principal Component Analysis (PCA) was applied to the first derivative (R′) spectra. The first two principal components (PC1 and PC2) were extracted, as they accounted for the largest proportion of variance. These components were then used to construct a two-dimensional feature space. To intuitively visualize the heterogeneity of the cross-regional data distribution, samples from the core study area (HL) were used as the baseline. Samples from other regions were then overlaid as colored scatter points. By comparing the degree of overlap and the direction of shift of samples from various regions within the PC1-PC2 coordinate system, the impact of regional soil variations on spectral responses was qualitatively evaluated.

3.2.2. Feature Band Extraction Method

High-dimensional hyperspectral data often suffer from redundancy and multicollinearity. To address this issue, a two-stage hybrid feature extraction framework was developed, combining Competitive Adaptive Reweighted Sampling (CARS) and the Successive Projections Algorithm (SPA). This approach aims to filter out interference signals driven by environmental backgrounds (e.g., dynamic moisture changes and non-specific minerals) and extract universally applicable feature bands. Initially, CARS was employed for global coarse screening, utilizing an exponentially decreasing function to rapidly eliminate uninformative variables. Subsequently, SPA was introduced for local refinement, employing orthogonal projections in a vector space to maximally eradicate collinear redundancy among the bands. Finally, the selected subset of feature bands was mapped onto the mean spectral curve of the soil samples to mechanistically validate whether they precisely correspond to the characteristic absorption bands of SOC-related core functional groups.

3.3. SOC Estimation Models and Accuracy Evaluation Metrics

3.3.1. SOC Estimation Model Construction

To verify the robustness of the extracted feature bands, a gradient model matrix encompassing linear and nonlinear, as well as shallow and deep architectures, was constructed. Specifically, Partial Least Squares Regression (PLSR), a classical model for soil spectral modeling, was selected as the linear baseline model [63,64], while Random Forest (RF), a traditional machine learning algorithm, served as the nonlinear baseline model [65]. To further unearth the application potential of the feature bands, two prominent deep learning architectures were incorporated: Convolutional Neural Networks (CNN) and Long Short-Term Memory networks (LSTM). The CNN focused on extracting local spatial-spectral features [66], whereas the LSTM emphasized analyzing long-range dependencies within the spectral sequences [67].

3.3.2. Model Evaluation Metrics

To accurately quantify the evolution of model predictive performance, an accuracy evaluation system was established using four metrics: Coefficient of Determination ($R^2$), Root Mean Square Error (RMSE), Ratio of Performance to Deviation (RPD), and Bias. To eliminate the randomness associated with sample partitioning, all models were subjected to 50 Monte Carlo simulations to verify their spatial predictive stability. The calculation formulas for these metrics are as follows:

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(\widehat{y_i}-y_i\right)}^2}{{\sum }_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(2)

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\left(\widehat{y_i}-y_i\right)}^2}$$

(3)

$$RPD={\displaystyle \frac{SD}{RMSE}}$$

(4)

$$Bias={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\left(\widehat{y_i}-y_i\right)$$

(5)

where $n$ represents the total number of samples; $y_i$ and $\widehat{y_i}$ denote the measured and predicted values, respectively; $\overline{y}$ is the mean of the measured values; and $SD$ is the standard deviation of the measured values.

4. Results

4.1. SOC Data Distribution and Spatial Heterogeneity Analysis

Prior to feature extraction and cross-regional model construction, a statistical analysis of the SOC content in the core study area (HL) and the other seven validation regions was conducted, with the results presented in

Table 1. Descriptive statistics of soil organic carbon (SOC) content for the source and target domains.

Classification	Region	N	Min (%)	Max (%)	Mean (%)	SD (%)	CV (%)	Skewness
Source Domain	HL	10,000	0.17	8.8	2.51	0.7	28.09	1.24
CN Regions	BQ	6361	0.41	20.3	2.6	1.03	39.59	2.7
	QQH	651	0.95	7.97	3.18	1.28	40.28	0.79
	FYFJ	397	0.67	11.45	2.56	1.22	47.69	2.66
	JL	117	0.42	12.59	2.81	1.83	65.21	1.96
EU Regions	HU	135	0.55	3.58	1.96	0.58	29.79	−0.07
	AT	40	1.36	5.17	1.9	0.71	37.16	2.91
	CZ	29	0.58	3.74	1.84	0.78	42.56	0.94

. As the core study area, the HL dataset exhibits robust statistical representativeness, comprising 10,000 samples with a broad SOC concentration gradient ranging from 0.17% to 8.80%, a mean of 2.51%, and a coefficient of variation (CV) of 28.09%. This distribution—characterized by a wide gradient, massive sample size, and moderate variability—provides robust data support for the subsequent research.

In contrast, the validation regions exhibit pronounced spatial heterogeneity. Within the Northeast China black soil belt, Baoqing (BQ), serving as the largest validation region (n = 6361), reached a maximum SOC value of 20.30%, significantly expanding the upper concentration limit of SOC in this region; the JL region demonstrated an exceptionally high CV of 65.21%, reflecting extreme local spatial dispersion; furthermore, the mean SOC in Qiqihar (QQH) was 3.18%, markedly higher than that in the HL region, reflecting an overall elevated distribution pattern. Among the European validation regions (AT, CZ, HU), SOC contents were generally lower. The mean values were clustered between 1.84% and 1.96%. In addition, the distribution patterns differed substantially from those observed in the Chinese regions. While the Chinese black soil regions generally exhibited a long-tailed positive skewness, the Hungarian (HU) dataset displayed a skewness of −0.07, indicating a slight negative skew. These macroscopic statistical discrepancies objectively reflect the significant heterogeneity in pedogenic environments and background properties across Eurasian black soils.

Collectively, these distributional results demonstrate that black soil SOC across different geographical spaces exhibits strong background disparities and spatial variations, which constitute the primary source of error when directly utilizing full-spectrum data for cross-regional remote sensing inversion.

4.2. Feature Band Extraction Results

4.2.1. Two-Stage Hybrid Feature Extraction Based on CARS-SPA

The full-spectrum data of the core study area (HL) comprised 2000 variables. This study applied a two-stage hybrid feature extraction method combining CARS and SPA, whose error evolution and iterative optimization process are illustrated in Figure 3. Initially, CARS was applied for global coarse screening. During this phase, the RMSECV first decreased and then increased sharply. This pattern reflects the progressive elimination of uninformative variables. After 50 iterations, the error reached a global minimum of 0.2999 when 858 variables were retained. Although this stage discarded over 50% of the bands, the retained subset still harbored high dimensional redundancy.

Subsequently, SPA was employed for secondary dimensionality reduction. As shown in Figure 3b, the error curve dropped sharply at first before gradually plateauing as the number of feature variables increased. Based on geometric distance calculations, an optimal “elbow point” emerged at a variable dimension of N = 31 (RMSECV = 0.3499). Beyond this threshold, the accuracy gains from introducing new bands tended to saturate. Considering both model fitting accuracy and cross-regional generalization capability, and to effectively prevent feature overfitting to the HL samples, these 31 critical bands were ultimately finalized as the SOC Optimal Spectral Feature Set (SOC-OSFS). Accounting for merely 1.55% of the original full-spectrum dimensionality, the SOC-OSFS efficiently purified the target core spectral information while stripping away the vast majority of background noise.

4.2.2. Validation of SOC-OSFS Spectral Responses

Mapping the 31 feature bands of the SOC-OSFS onto the mean spectral curve of the samples revealed a clear association between their spatial distribution and known soil organic matter absorption regions, as detailed in Figure 4. Within the visible region (Vis, 401–700 nm), the SOC-OSFS captured three bands: 430, 433, and 447 nm. These bands are commonly associated with electronic transition absorption features of chromophores (e.g., humic substances) within soil organic matter, serving as an intuitive physical response interval reflecting the color characteristics and organic carbon enrichment of black soils.

In the shortwave near-infrared region (SWIR-1, 1100–1700 nm), the extracted feature bands were 1356 nm and 1406 nm. Located adjacent to the 1400 nm absorption peak—typically dominated by bound water and mineral structural hydroxyls—these bands are located near regions commonly attributed to first overtone absorptions of aliphatic C-H and O-H bonds within the macromolecular structure of soil organic matter. The longwave near-infrared region (SWIR-2, 2161–2386 nm) hosted the densest distribution of SOC-OSFS bands, encompassing 18 bands including 2197, 2220, and 2307 nm. This interval has been widely reported as a key spectral region associated with aliphatic C-H and N-H bonds within complex macromolecules such as lignin, cellulose, and nitrogen-containing polypeptides that constitute soil organic matter. The dense band distribution in this region indicates a strong correlation between spectral reflectance in this frequency band and the spatial variation of SOC content in black soils.

Furthermore, it is worth noting that several selected bands (e.g., 696 nm, 1045 nm, and the cluster between 1820 nm and 1992 nm) fall outside these three primary highlighted intervals. Rather than representing direct organic absorption peaks, these additional bands were strategically retained by the SPA algorithm to serve as essential baseline anchors. Specifically, bands such as 696 nm and 1045 nm assist in mitigating the spectral interference from iron oxides and pedogenic parent materials, while the bands surrounding the 1900 nm region (e.g., 1820–1992 nm) are crucial for decoupling the dynamic background noise associated with soil moisture and clay minerals. By capturing and filtering out these environmental background signals, the feature set effectively eliminates collinear redundancy and enhances the model’s robustness against the spatial heterogeneity of pedogenic environments.

Overall, the 31 bands extracted by the SOC-OSFS showed strong consistency with spectral regions reported to be associated with key functional groups of soil organic matter, forming an optimal feature subset for SOC inversion utilizing only 1.55% of the data dimensionality. It should be noted that these interpretations are based on spectral consistency with previous studies, rather than direct chemical validation.

4.3. SOC-OSFS Feature Space Realignment and Local Analytical Validation

4.3.1. SOC-OSFS Feature Space Validation

To evaluate the evolution of feature distributions before and after the extraction of SOC-OSFS, this study compared the PCA-derived spectral feature spaces of the full spectrum and the SOC-OSFS, as shown in Figure 5 and Figure 6. Based on the 401–2400 nm full-spectrum PCA projection (Figure 5), the HL samples (gray scatter points) exhibited a broad distribution centered around the origin in the feature space, forming a foundational data support with high coverage. In the subplots, the blue scatter points representing the validation regions displayed varying degrees of spatial shifting. The sample points of BQ and QQH highly overlapped with the HL background; however, FYFJ and JL shifted locally towards the negative axes of PC1 and PC2, respectively. In the European regions (AT, CZ, HU), which possess vastly different pedogenic environments, this phenomenon was even more pronounced, showing a significant systematic shift entirely towards the negative axis of the first principal component.

Compared to the full-spectrum PCA projection, the SOC-OSFS-based PCA projection (Figure 6) demonstrated a pronounced convergence trend. The overlap between the blue scatter points of the seven validation regions and the gray HL background increased remarkably. Most notably, the three European regions (AT, CZ, HU)—which previously exhibited the most severe deviation—shifted significantly closer to the center of the core study area. This phenomenon robustly demonstrates that extracting the SOC-OSFS effectively facilitates the realignment of spectral samples from diverse geographical regions within a low-dimensional feature space.

4.3.2. Cross-Regional Local Modeling Validation Using SOC-OSFS

To quantitatively assess the independent analytical capability of the SOC-OSFS across different regions, local prediction models were constructed for all eight study areas, with accuracy statistics presented in

Table 2. Performance of optimal local prediction models across regions based on SOC-OSFS.

Region	N	Best Model	R2	RMSE (%)
HL	10,000	CNN	0.88	0.1634
QQH	651	CNN	0.8853	0.431
JL	118	RF	0.8285	0.659
FYFJ	397	RF	0.8360	0.513
BQ	6361	CNN	0.8854	0.459
HU	135	RF	0.7612	0.24
CZ	29	RF	0.6876	0.363
AT	40	RF	0.6714	0.337

(full evaluation metrics are detailed in Supplementary Table S1).

The results indicate that local models built upon the SOC-OSFS achieved stable predictive accuracies across all regions. In the Northeast China black soil regions with relatively abundant sample sizes, the models showed high goodness-of-fit, where the optimal CNN models for HL, QQH, and BQ all achieved an $R^2$ exceeding 0.88, while the optimal RF models for JL and FYFJ reached $R^2$ values of 0.8285 and 0.8360, respectively. In the European regions with limited sample sizes (AT, n = 40; CZ, n = 29; HU, n = 135), the SOC-OSFS-based RF models also demonstrated strong fitting performance, yielding $R^2$ values of 0.6714, 0.6876, and 0.7612, while constraining the RMSE to 0.337%, 0.363%, and 0.240%, respectively.

4.4. Evaluation of Cross-Regional Universal Prediction Method Based on SOC-OSFS

4.4.1. Model Prediction Results in the Core Study Area

This section compares the modeling performance of the full spectrum and SOC-OSFS in the HL region, as shown in

Table 3. Comparison of modeling accuracy between full-spectrum and SOC-OSFS in the core study area (HL).

Model	Feature	R2	RMSE (%)	RPD	Bias
CNN	SOC-OSFS	0.8800	0.1634	2.8868	0.0257
CNN	Full-Spectrum	0.8028	0.2445	2.2521	−0.0813
LSTM	SOC-OSFS	0.8674	0.1718	2.7464	0.0044
LSTM	Full-Spectrum	0.8678	0.2002	2.7508	0.0172
PLSR	SOC-OSFS	0.8169	0.2019	2.3368	0.032
PLSR	Full-Spectrum	0.8783	0.1921	2.8663	0.0277
RF	SOC-OSFS	0.8732	0.168	2.8087	0.0125
RF	Full-Spectrum	0.8764	0.1936	2.8445	0.011

.

Using full-spectrum data, the traditional PLSR and RF models achieved relatively stable fitting accuracies. Conversely, when processing 2000 high-dimensional variables, the deep learning models (CNN and LSTM) were hindered by redundant spectral information, resulting in accuracy metrics inferior to those of traditional models. When the input features were streamlined to the SOC-OSFS, the predictive performance of the CNN model was fully unleashed, achieving the optimal performance among the four models with its $R^2$ increasing to 0.8800 and RMSE decreasing to 0.1634. Although the full-spectrum PLSR model also exhibited high fitting accuracy ($R^2$ = 0.8783), this was achieved at the cost of high model complexity associated with processing high-dimensional data. In contrast, the feature-selected CNN model maintained peak predictive accuracy utilizing only a minimal number of bands, further validating the efficacy of the SOC-OSFS as a robust foundation for cross-regional joint inversion.

4.4.2. Comparison of Cross-Regional Prediction Methods

Using the CNN model as a representative example, this section compares the application effects of three strategies across the remaining seven independent validation regions: Full Spectrum Direct Prediction, SOC-OSFS Direct Prediction, and SOC-OSFS Combined with 20% Local Sample Spiking. To explicitly verify the statistical robustness of the predictions and assess the distributions of the 50 Monte Carlo simulations, paired-samples t-tests were conducted (complete statistical metrics for all models, including standard deviations, 95% confidence intervals, and exact p-values, are comprehensively documented in Supplementary Table S3). The model accuracy statistics are presented in

Table 4. Performance comparison of the representative model (CNN) across verification regions under three cross-regional prediction methods.

Region	Metrics	Full Spectrum	SOC-OSFS	SOC-OSFS 20% Spiking
QQH	R2	0.5238	0.6684	0.8567 **
	RMSE (%)	0.8838	0.7375	0.4791 **
FYFJ	R2	−0.7559	−0.2070	0.8357 **
	RMSE (%)	1.6176	1.3411	0.4659 **
BQ	R2	−0.2489	0.2918	0.7419 **
	RMSE (%)	1.1504	0.8662	0.5270 **
JL	R2	0.2264	0.3946	0.7170 **
	RMSE (%)	1.6046	1.4195	0.7831 **
CZ	R2	−3.2861	−2.5164	0.5051 **
	RMSE (%)	1.5956	1.4453	0.5242 **
HU	R2	−4.7995	−2.1126	0.4919 **
	RMSE (%)	1.4008	1.0262	0.4111 **
AT	R2	−1.5917	−1.1244	0.2715 **
	RMSE (%)	1.1227	1.0164	0.3920 **

Note: The reported values are the robust means derived from 50-iteration Monte Carlo simulations. The asterisks (**) denote that the performance improvement achieved by the 20% spiking strategy is statistically highly significant (p < 0.01) compared to the non-spiking strategies, based on paired-samples t-tests.

(complete metrics for other models are in Supplementary Table S2), and the evolutionary trajectories of model performance ($R^2$) and the scatter distributions of predictions are illustrated in Figure 7 and Figure 8, respectively.

Initially, when applying the Full Spectrum Direct Prediction, model accuracies across all validation regions plummeted significantly. As shown in Figure 7, the starting points of the trajectories for this method were at extremely low levels. Particularly in the three European validation regions (AT, CZ, HU), the $R^2$ values for the full-spectrum models were entirely negative (Figure 7e–g). Observing the gray scatter points and dashed lines in Figure 8, the predicted values of the full-spectrum models deviated severely from the 1:1 reference line, exhibiting obvious systematic biases.

In contrast, utilizing the SOC-OSFS Direct Prediction yielded varying degrees of error reduction across all validation regions. Compared to the full-spectrum direct prediction approach, the $R^2$ trajectory of this proposed method shifted upward globally across all validation areas (Figure 7). For instance, in QQH, the $R^2$ improved from 0.5238 to 0.6684 (Table 4), and the RMSE decreased by over 15% in the vast majority of validation regions. In Figure 8, the blue dashed lines representing the SOC-OSFS direct prediction visibly converged towards the 1:1 ideality line compared to the gray dashed lines.

To further rectify the systematic intercept shifts in cross-regional predictions, this study introduced the SOC-OSFS 20% Spiking strategy. As demonstrated in Table 4 and Figure 7, under the guidance of a small volume of local sample data, predictive accuracy was drastically enhanced, displaying a distinct upward climbing trajectory. The fine-tuned CNN models achieved high goodness-of-fit in the Northeast China black soil regions, elevating the $R^2$ of QQH and FYFJ to 0.8567 and 0.8357, respectively (Table 4). As verified by the paired t-tests, the accuracy improvements achieved by the 20% spiking strategy were statistically highly significant (p < 0.01, denoted by ** in Table 4) across all target regions compared to the non-spiking strategies. Furthermore, as detailed in Table S3, the CNN model maintained extremely narrow 95% confidence intervals even in regions with smaller validation samples, quantitatively confirming the robust stability and reliability of its cross-regional estimations. In the European regions, this method successfully corrected the models back to positive linear fits. This accuracy improvement is especially intuitive in the scatter plots of Figure 8; the originally deviated fitting trajectories, after fine-tuning (red solid lines), became highly aligned with the 1:1 ideal reference line (black solid lines), and the convergence of the scatter distributions improved massively. Consequently, these results demonstrate that the SOC-OSFS combined with a minor local sample spiking strategy can effectively overcome the biases induced by geographical heterogeneity, realizing robust synergistic cross-regional SOC prediction.

5. Discussion

5.1. Physicochemical Mechanisms and Feature Space Realignment Underlying the Optimal Spectral Feature Set for SOC

This study constructed the optimal spectral feature set for soil organic carbon (SOC-OSFS) based on competitive adaptive reweighted sampling and the successive projections algorithm. The efficacy of this feature set is manifested not only in mathematical dimensionality reduction, but also in its robust support from explicit physicochemical response mechanisms.

As demonstrated in Section 4.2, the SOC-OSFS successfully reduced the full spectrum to merely 31 core bands (1.55% of the original dimensionality). The specific bands extracted in the visible region (e.g., 430, 433, and 447 nm) correspond directly to the electronic transitions of chromophores in humic substances. This result strongly aligns with the findings of Ben-Dor and Viscarra Rossel et al. [68], confirming that the feature set successfully captured the stable color-producing macromolecules formed during the evolution of black soil organic matter. Furthermore, key bands were selected in the SWIR-1 (1356 and 1406 nm) and SWIR-2 regions (e.g., 2197, 2220, and 2307 nm). These bands avoid the strong water absorption feature near 1400 nm. At the same time, they capture overtone responses of aliphatic C–H and N–H bonds in complex macromolecules such as lignin and cellulose. This spectral behavior is highly consistent with the classical mechanisms of soil organic matter absorption established by Stenberg et al. and Dalal & Henry [69,70].

Furthermore, it is worth noting that several selected bands (e.g., 696 nm, 1045 nm, and the cluster between 1820 nm and 1992 nm) fall outside these three primary highlighted intervals. Rather than representing direct organic absorption peaks, these additional bands were strategically retained by the SPA algorithm to serve as essential baseline anchors. Specifically, bands such as 696 nm and 1045 nm assist in mitigating the spectral interference from iron oxides and pedogenic parent materials [12], while the bands surrounding the 1900 nm region (e.g., 1820–1992 nm) are crucial for decoupling the dynamic background noise associated with soil moisture and clay minerals [29,48,69,71].

The purification of the aforementioned physicochemical mechanisms was intuitively corroborated during the actual cross-regional validation. When utilizing full-spectrum data, samples from diverse geographical regions exhibited severe deviations within the feature space. This is mainly because the full spectrum contains a large amount of non-target background information. These signals originate from soil parent materials, iron oxides, and clay minerals. Their strong spatial variability across regions masks the true spectral response of SOC. Conversely, when exclusively utilizing the optimal spectral feature set, samples from disparate geographical regions achieved a high degree of realignment within the feature space. This result demonstrates that extracting spectral response intervals with explicit physicochemical significance can effectively eliminate the interference of environmental background noise. This provides a novel perspective for research on feature band extraction in hyperspectral remote sensing inversion of SOC; namely, in future model construction, extracting such universal features can substantially mitigate the spatial heterogeneity issues induced by differences in pedogenic environments.

5.2. Application Potential of the Universal Cross-Regional Prediction Method

In the cross-regional prediction validation, full-spectrum models experienced severe accuracy degradation, or even complete failure, when applied to target domains with substantially different pedogenic environments. The root cause of this failure lies in the full-spectrum models’ severe overfitting to the specific soil matrices and redundant environmental variations in the core study area, thereby losing their spatial generalization capability [16,72]. In contrast, the optimal spectral feature set effectively reduced errors during direct cross-regional prediction, fully confirming its powerful filtering effect against environmental noise and its ability to preserve valid SOC spectral features that transcend geographical boundaries.

However, influenced by climate and long-term agricultural activities, different geographical units possess inherent intercept discrepancies in their background SOC concentrations; relying solely on feature extraction is insufficient to compensate for this systematic statistical shift. Validation results demonstrated that, building upon the optimal spectral feature set, introducing merely 20% of local samples for model fine-tuning (i.e., spiking) could elevate the predictive accuracy of each validation region to a level approaching that of independent local modeling [52,69]. This performance proves that the feature set possesses robust universality and plasticity in cross-regional predictions.

Overall, this cross-regional prediction framework shows strong practical potential. By establishing a foundational model based on the optimal spectral feature set, realizing high-accuracy spatial inversion in unknown geographical regions requires only the collection of a minimal number of field samples for local intercept correction. This approach drastically curtails the dense sampling costs required for large-scale predictions, offering a highly efficient solution for the large-scale remote sensing inversion and mapping of SOC.

5.3. Challenges and Limitations

Although the proposed method—combining the optimal spectral feature set with local sample fine-tuning—substantially improved cross-regional predictive accuracy, it still faces certain challenges in practical applications and subsequent mechanistic explorations.

First, in certain validation regions with extremely complex pedogenic backgrounds, the absolute accuracy of the fine-tuned models still has room for further improvement. Although the relatively small European validation sets were carefully selected for their pedological representativeness across Central Europe and exhibited statistical robustness in our uncertainty analysis, relying on limited cohorts inherently carries a risk of unstable estimations. In addition, potential temporal inconsistencies between datasets may introduce further uncertainty, although their impact is mitigated by the use of standardized laboratory measurement conditions. Future research could consider introducing samples covering a broader range of climate zones and soil types (such as expanding to more balanced European and Pan-Eurasian datasets) into the foundational dataset, evaluating a wider diversity of feature extraction and modeling algorithms, or employing advanced approaches that fuse neural network models with physical prior mechanisms, to further bolster the model’s capacity to represent spatial heterogeneity.

Second, this study was primarily conducted using laboratory hyperspectral data under ideal conditions; genuinely extrapolating this optimal spectral feature set to airborne or satellite-borne hyperspectral remote sensing platforms still necessitates overcoming complex external environmental interferences. In actual field remote sensing observations, factors such as soil surface roughness, dynamic variations in natural moisture content, crop residue coverage, and multiple atmospheric scatterings can all attenuate or even mask the faint spectral responses of SOC [73,74].

6. Conclusions

This study proposed and validated a cross-regional prediction method for soil organic carbon (SOC) based on an optimal spectral feature set, using datasets from the black soil regions of Northeast China and Europe. The primary conclusions are drawn as follows:

First, this study established the SOC-OSFS, which is supported by explicit physicochemical mechanisms. Based on CARS and SPA, a feature subset comprising 31 core bands was extracted from the full spectrum. Accounting for merely 1.55% of the original data dimensionality, this feature set precisely locked onto the spectral response intervals reflecting macromolecular functional groups such as humic chromophores, aliphatic C-H bonds, and N-H bonds. This feature set preserves key structural information while reducing interference from environmental moisture.

Second, this study confirmed the critical role of feature dimensionality reduction in decoupling environmental background noise and achieving spatial realignment. Full-spectrum data are highly susceptible to interference from non-target background signals such as iron oxides and clay minerals, which leads to significant shifts of cross-regional spectral samples within the feature space. However, based on the extracted optimal spectral feature set, black soil samples from diverse pedogenic backgrounds successfully overcame the impacts of geographical spatial differences and achieved feature space realignment, thereby mitigating the obstacles posed by spatial heterogeneity to the inversion models.

Finally, this study validated the cross-regional prediction method based on the optimal spectral feature set coupled with local sample correction. When encountering heterogeneous pedogenic environments, full-spectrum prediction models suffer from significant accuracy degradation due to overfitting to local data. In contrast, utilizing the optimal spectral feature set combined with intercept correction and model fine-tuning (spiking) using only 20% of local target samples can effectively overcome cross-regional prediction biases. This method elevated the predictive accuracy of each independent validation region to a level approaching that of independent local modeling, demonstrating excellent spatial generalization capability.

Overall, this study not only provides a novel perspective for resolving spatial heterogeneity issues in hyperspectral SOC inversion but also offers an effective method for large-scale SOC estimation. These findings provide a practical framework for large-scale SOC estimation and support future applications in digital soil mapping and carbon accounting using hyperspectral remote sensing. Future work should focus on improving model generalization by incorporating more diverse soil and climate datasets and further enhancing the transferability of laboratory-derived spectral features to satellite-scale applications.