A Feature Engineering and XGBoost Framework for Prediction of TOC from Conventional Logs in the Dongying Depression, Bohai Bay Basin

Abstract

Total organic carbon (TOC) is a critical parameter for evaluating shale source rock quality and hydrocarbon generation potential. However, accurate TOC estimation from conventional well logs remains challenging, especially in data-limited geological settings. This study proposes an optimized XGBoost model for TOC prediction using conventional logging data from the Shahejie Formation in the Dongying Depression, Bohai Bay Basin, China. We systematically transform four standard logs—resistivity, acoustic transit time, density, and neutron porosity—into 165 candidate features through multi-scale smoothing, statistical derivation, interaction term creation, and spectral transformation. A two-stage feature selection process, combining univariate filtering and recursive feature elimination and further refined by principal component analysis, identifies ten optimal predictors. The model hyperparameters are optimized via Bayesian search within the Optuna framework to minimize cross-validation error. The optimized model achieves an R² of 0.9395, with a Mean Absolute Error (MAE) of 0.3392, a Root Mean Squared Error (RMSE) of 0.4259, and a Normalized Root Mean Squared Error (NRMSE) of 0.0604 on the test set, demonstrating excellent predictive accuracy and generalization capability. This study provides a reliable and interpretable methodology for TOC characterization, offering a valuable reference for source rock evaluation in analogous shale formations and sedimentary basins.

1. Introduction

The successful exploitation of shale resources in North America has stimulated global interest in unconventional hydrocarbon development, particularly within the terrestrial basins of China [1,2,3]. These shale oil and gas reservoirs are characterized by organic-rich source rocks in which hydrocarbons are generated and retained without significant secondary migration [4,5,6]. Total organic carbon (TOC) is a fundamental indicator of source-rock quality and hydrocarbon generation potential, and its accurate estimation along wellbores is essential for evaluating shale productivity and sweet-spot distribution [7,8,9,10,11].

Currently, two main approaches are used to determine TOC in petroleum exploration: direct laboratory measurement and indirect estimation [12,13]. The direct approach involves analyzing core samples in laboratories, providing accurate first-hand data for reservoir characterization. Although widely applied, this method is costly, time-consuming, and spatially limited [14,15,16]. Moreover, measurements are usually performed under ambient rather than in situ reservoir conditions, and their accuracy depends strongly on stress restoration procedures and instrumental precision [17,18].

The indirect approach relies on geophysical data, particularly conventional well logs, to provide continuous TOC estimation [19,20]. Traditional empirical methods such as the ΔlogR technique are valued for simplicity and low cost but depend heavily on subjective calibration and fixed empirical parameters, which limits their transferability across formations with different lithologies and maturities [21]. Classical machine-learning methods, including linear regression and support vector machines, are easy to implement and require relatively small datasets; however, they lack systematic optimization strategies, often leading to suboptimal parameter configurations and reduced prediction accuracy [22,23]. Ensemble methods such as Random Forest (RF) combine multiple decision trees to enhance prediction robustness and mitigate overfitting, yet their reliance on random feature selection may overlook significant nonlinear patterns in well-log data [24,25]. Deep-learning models such as Convolutional Neural Network (CNN) can capture complex nonlinear relationships between well logs and TOC but typically demand large training samples. In many oilfields, the number of cored wells is limited, constraining the practical applicability of deep-learning approaches [26,27,28,29,30].

Machine learning has therefore become a key data-driven tool for petroleum studies. Among various algorithms, the Extreme Gradient Boosting (XGBoost) model has shown outstanding predictive capability and robustness [31,32]. The XGBoost algorithm is a scalable tree-boosting method developed from gradient decision trees [33]. Through improvements in sparsity awareness, a theoretically justified weighted-quantile sketch for approximate learning, and computational optimizations including cache handling, data sharding, and compression, XGBoost achieves higher prediction accuracy and stronger generalization performance than traditional gradient-boosting algorithms and tree-based ensembles like Random Forest, owing to its gradient-guided boosting mechanism and more sophisticated regularization techniques [34,35]. These advantages have led to successful applications in production forecasting, lithology identification, and reservoir characterization [36,37]. Nevertheless, XGBoost models constructed with default or suboptimal hyperparameters often show limited adaptability to specific datasets, highlighting the need for systematic optimization strategies [38,39,40].

Recent advances in automated optimization frameworks such as Optuna allow efficient exploration of hyperparameter spaces through Bayesian optimization [41]. Optuna’s tree-structured Parzen estimator (TPE) enables adaptive and efficient parameter tuning while maintaining low computational cost [42]. Although the XGBoost model and Bayesian optimization have been applied in various geoscientific contexts, their combined use for TOC estimation from conventional well logs—particularly when integrated with systematic feature engineering and multi-stage feature selection—has not yet been comprehensively explored [43].

To address this need, this study develops an optimized XGBoost model for TOC prediction from conventional well logs. A Random Forest model is implemented as a baseline to benchmark the performance and highlight the efficacy of the proposed gradient-boosting framework. The proposed approach integrates systematic feature engineering, multi-stage feature selection, and Bayesian hyperparameter optimization using the Optuna framework to enhance model performance [44]. In this workflow, four conventional logs are expanded into a large set of derived attributes through statistical, interaction, and spectral transformations to better capture the nonlinear relationships between petrophysical parameters and TOC. A data-driven selection strategy is then employed to identify the most informative predictors, while Bayesian optimization efficiently explores the hyperparameter space to achieve an optimal model configuration. The model is trained and validated using core–log paired samples from the Shahejie Formation in the Dongying Depression, Bohai Bay Basin, China. Overall, this study provides a practical and interpretable machine-learning approach for TOC estimation, offering methodological insights and a transferable reference for similar shale and sedimentary basins.

2. Background

2.1. Passey Method

The ΔlogR technique is a well-established method for estimating TOC from conventional well logs [7]. This approach is based on the observation that in organic-rich source rocks, the separation between resistivity and sonic porosity logs increases due to the presence of hydrocarbon-generating organic matter.

Mathematically, the ΔlogR method is defined as:

$$\Delta \log R = {\log }_{10}\left({\displaystyle \frac{R}{R_0}}\right) - K \times \left(\Delta t-\Delta t_0\right)$$

(1)

where $R$ and $\Delta t$ represent measured resistivity (Ω·m) and sonic transit time (μs/ft), respectively; $R_0$ and $\Delta t_0$ are their baseline values in water-saturated, organic-lean shale intervals; and $K$ is a scaling coefficient, typically set to 0.02. In practice, a porosity log (commonly sonic) is overlain on a resistivity curve using predefined scales, with the baseline defined by parallel trends in clean intervals. The vertical separation in organic-rich zones is then quantified and empirically calibrated to TOC using:

$$\mathrm{TOC} = \Delta \log R \times {10}^{\left(2.297 - 0.1688 \times \mathrm{LOM}\right)}$$

(2)

where LOM represents the Level of Organic Maturation. Gamma ray logs often assist in identifying clean shale intervals and constraining baseline selection [16].

Despite practical advantages—such as straightforward implementation and the provision of continuous TOC profiles where core data are unavailable—the ΔlogR method possesses significant limitations. Its reliance on fixed scaling coefficients (K ≈ 0.02) may not be universally applicable across diverse geological settings [19,45]. Accuracy is highly sensitive to subjective baseline selection, and the method does not systematically integrate complementary information from other logs (e.g., density or neutron porosity), constraining its predictive performance and generalizability to new depositional environments [8,20].

2.2. Extreme Gradient Boosting

XGBoost is a scalable and highly efficient implementation of the gradient boosting framework, renowned for its computational speed and model performance [32]. As an ensemble learning method, it constructs a strong predictive model by sequentially integrating multiple weak learners, typically decision trees, with each new tree designed to correct the residuals from the previous ensemble.

The algorithm operates on the principle of gradient descent, where the model is optimized by minimizing a predefined loss function. The distinctive feature of XGBoost lies in its regularization-enhanced objective function, which incorporates both a loss term and a complexity control term. This formulation effectively balances model accuracy and generalization capability, significantly reducing the risk of overfitting [32,40]. The objective function at the t-th iteration can be expressed as:

$${\mathit{Obj}}^{\left(t\right)} = \sum _{i=1}^n l\left(y_i,{\widehat{y}}_i^{\left(t-1\right)} + f_t\left(x_i\right)\right) + \sum _{k=1}^t \Omega \left(f_k\right)$$

(3)

where $l$ denotes the differentiable loss function, $f_t$ represents the tree model at iteration $t$, and $\Omega$ is the regularization term that penalizes model complexity.

Hyperparameters in XGBoost are critical for controlling model complexity and preventing overfitting. Specifically, max_depth controls tree depth; learning_rate scales tree contributions; subsample and colsample_bytree govern the random sampling ratios for training instances and features, respectively. reg_alpha and reg_lambda correspond to L1 and L2 regularization terms on weights, while min_child_weight defines the minimum sum of instance weight needed in a child node [32,38]. The strategic configuration of these parameters enables optimal model performance across diverse datasets and problem domains.

3. Geological Settings

This study focuses on the Dongying Depression, a significant sedimentary unit situated within the Jiyang Depression of the Bohai Bay Basin. This basin, formed on a Paleozoic continental basement, is a prominent Mesozoic–Cenozoic continental rift system on China’s eastern margin [46,47], and its architectural framework is defined by six primary depressions, including the Jiyang Depression. As a third-order negative tectonic unit, the Dongying Depression spans approximately 90 km in an east–west direction and 65 km north–south, covering a total area of about 5700 km² [48].

Structurally, the depression exhibits a complex semi-graben configuration, characterized by boundary normal faults, deep subsags, internal fault blocks, and transitional zones [49]. It is bounded by the Chenjiazhuang and Binxian uplifts to the north, the Gaoqing Uplift to the west, the Luxi Uplift to the south, and the Jiaodong Uplift to the east. East–west extensional tectonics have exerted a strong influence on the evolution of sedimentary systems in the region [46,50]. The depression experienced active rifting during the Late Mesozoic to Early Tertiary, followed by a phase of post-rift subsidence throughout the Cenozoic era. Internally, the depression can be subdivided into four subsags—Niutitang, Boxing, Minfeng, and Lijin—and three structural belts: the central anticlinal belt, the southern slope belt, and the northern slope belt. The Liye 1 well, analyzed in this study, is situated on the eastern slope of the Lijin subsag, within an open and structurally segmented belt of the Dongying Depression (Figure 1) [51,52,53,54,55,56].

The Cenozoic stratigraphic framework of the Dongying Depression consists of, in ascending order, the Kongdian (Ek), Shahejie (Es), Dongying (Ed), Guantao (Ng), Minghuazhen (Nm) formations, and Quaternary (Q) deposits (Figure 2) [46]. The Shahejie Formation is further divided into four members, namely Es₄, Es₃, Es₂, and Es₁ from base to top, with the Es₃ and Es₄ members recognized as the primary source rock intervals in the region [57,58].

Depositional environment analysis indicates that the Es₃ member accumulated in a deep lacustrine setting under freshwater conditions, while the Es₄ member was influenced by saline lake phases [59,60]. The combination of anoxic bottom waters, enhanced organic productivity, and reduced sedimentation rates promoted the development of substantial source rock thicknesses in these units. The lithological column shows that the basal Es₃ primarily comprises oil shale, calcareous mudstone, and dark mudstone, while its middle-upper sections are marked by interbedded gray mudstone and sandstones. For the Es₄ member, the lower part features red sandstone and mudstone with saline layers, and the upper part is characterized by gray mudstone, ocher mudstone, thin limestone, and oil shale [61]. Within this sequence, the lower Es₃ (EsL 3) hosts an approximately 100 m thick oil shale, representing a major source rock in the region [62].

Figure 1. (A) Schematic diagram depicting the geographic location of the Bohai Bay Basin. (B) Map showing the regional position of the Dongying Sag (modified after Hu et al. [54]). (C) Schematic illustrating the structural framework and sampling well distribution in the Dongying Sag (modified after Zeng et al. [55]).

Figure 1. (A) Schematic diagram depicting the geographic location of the Bohai Bay Basin. (B) Map showing the regional position of the Dongying Sag (modified after Hu et al. [54]). (C) Schematic illustrating the structural framework and sampling well distribution in the Dongying Sag (modified after Zeng et al. [55]).

Figure 2. Composite stratigraphic column of the Paleogene succession in the Dongying Sag (modified after Chen et al. [62]). The column illustrates the following units: Dongying Formation (Ed); Shahejie Formation (Es), including its members 1 through 4 (Es₁, Es₂, Es₃, Es₄) with their lower (L), middle (M), and upper (U) subdivisions; and Kongdian Formation (Ek) with members 1 and 2 (Ek₁, Ek₂).

Figure 2. Composite stratigraphic column of the Paleogene succession in the Dongying Sag (modified after Chen et al. [62]). The column illustrates the following units: Dongying Formation (Ed); Shahejie Formation (Es), including its members 1 through 4 (Es₁, Es₂, Es₃, Es₄) with their lower (L), middle (M), and upper (U) subdivisions; and Kongdian Formation (Ek) with members 1 and 2 (Ek₁, Ek₂).

4. Methodology

4.1. Samples

The dataset used in this paper was derived from the rigorously quality-controlled sample set established by Wang et al. [63]. As documented in their original work, to ensure high data quality, the original 146 core samples from the EsL 3 and EsU 4 members underwent a strict refinement process. First, systematic depth errors were corrected by correlating core gamma ray measurements with well logs. Subsequently, to eliminate potential anomalies caused by experimental errors or contamination, a statistical screening based on the ±3.0 standard deviation criterion was applied. Data points falling outside this range were identified as outliers and removed. This pre-processing, as performed by Wang et al., refined the initial 146 samples to the final set of 125 high-quality samples, which we adopted as the foundation for developing the XGBoost model in this paper.

4.2. Input Selection

The dataset utilized in this study, derived from the Liye 1 well [63], consisted of five available conventional logging curves for the analyzed interval: natural gamma ray (GR), resistivity (RT), acoustic transit time (AC), bulk density (DEN), and neutron porosity (CNL). These logs represent the full suite of primary organic-sensitive indicators provided in the source dataset.

The response of these key well logs provides diagnostic criteria for identifying organic-rich intervals [10]. Specifically, increases in TOC content amplify the physical contrast between the organic matter and the matrix, resulting in more pronounced log anomalies characterized by significantly higher RT and AC, as well as lower DEN. This mechanism enables effective source rock characterization and TOC estimation. Given that single-log responses are susceptible to mineralogical and environmental influences, a multi-parameter integration is crucial for obtaining robust TOC predictions.

Crossplot analysis combined with Pearson correlation coefficient (r) analysis was employed to evaluate the relationships between different logging parameters and TOC content [13,64]. The Pearson correlation coefficient is calculated as:

$$r={\displaystyle \frac{{\sum }_{i=1}^n \left(x_i-\bar{x}\right)\left(y_i-\bar{y}\right)}{\sqrt{{\sum }_{i=1}^n (x_i-\bar{x}{)}^2{\sum }_{i=1}^n (y_i-\bar{y}{)}^2}}}$$

(4)

where the subscript $i$ denotes the sample index, $x_i$ and $y_i$ represent the well-logging parameter and the core-measured TOC value of the $i$-th sample, $\bar{x}$ and $\bar{y}$ are their arithmetic means, and $n$ is the total number of samples.

(1)

Resistivity Logging (RT)

Mudstone intervals are characterized by low resistivity, associated with the high conductivity of their constituent matrix and pore fluids. However, the presence of non-conductive hydrocarbons (resistivity: 10⁵–10⁹ Ω·m) generates significant positive resistivity anomalies. This explains the moderate positive correlation between RT and TOC (Figure 3), with r = 0.39 for the training set and r = 0.37 for the test set, confirming that elevated resistivity is associated with higher organic content [7].

(2)

Density Logging (DEN)

The bulk density of organic matter and hydrocarbons (1.1–1.4 g/cm³) is substantially lower than that of common rock-forming minerals (e.g., quartz: 2.65 g/cm³). Consequently, bulk density decreases as organic matter concentration increases. This robust inverse relationship is evidenced by the notable negative correlation between DEN and TOC (Figure 4), with r = −0.51 (training set) and r = −0.57 (test set) [13].

(3)

Acoustic Logging (AC)

While acoustic transit time in mudstones normally decreases with compaction, the presence of organic matter (with high acoustic slowness ~ 524.9 μs/m) increases it. This principle accounts for the observed strong positive correlation between AC and TOC (Figure 5), with r = 0.48 (training set) and r = 0.59 (test set), indicating that higher transit times signify organic enrichment [7,13].

(4)

Neutron Logging (CNL)

In principle, the hydrogen-richness of organic materials should lead to elevated neutron porosity readings. However, the observed correlation between CNL and TOC was weak to moderate, with considerable scatter in the data (Figure 6). The notable discrepancy between the training (r = 0.28) and test (r = 0.63) sets strongly suggests that the neutron log response is significantly influenced by other hydrogen-bearing components (e.g., clay-bound water) [8,13]. This variability indicates that CNL provides useful but non-unique information for TOC prediction.

(5)

Gamma Ray Logging (GR)

The relationship between GR and TOC remains contentious. In this study, GR exhibited a negligible correlation with TOC, as shown by the random scatter in the crossplot (Figure 7) and extremely low r-values (0.04 for training; 0.19 for test) [7,16]. This indicates that gamma ray activity here is controlled by factors unrelated to organic matter (e.g., clay mineralogy, detrital minerals) [45]. It is important to note that this weak correlation is recognized as a specific characteristic of the lacustrine source rocks in the Shahejie Formation. Given this weak and inconsistent relationship, which was deemed likely to introduce noise and undermine model robustness, GR was consequently excluded from the input feature set.

4.3. Model Development Workflow

Considering the limited availability of well-log parameters (RT, DEN, CNL, and AC) and the complex nonlinear relationships between petrophysical properties and TOC, XGBoost was adopted as the core learning algorithm [32]. The model construction process encompasses three integrated components: feature engineering, feature selection, and hyperparameter optimization, aiming to achieve a balance between predictive accuracy and model interpretability. As illustrated in Figure 8, the overall framework begins with well log data input and preprocessing, and proceeds through a systematic feature engineering process that expands the four raw parameters into 165 candidate features [44]. Specifically, this expansion was achieved by applying to the raw logs a structured set of mathematical transformations, including multi-scale Gaussian smoothing, gradient calculation, statistical aggregation, and frequency-domain analysis. This procedure aimed to construct a comprehensive candidate feature pool capable of capturing the complex non-linear petrophysical responses and stratigraphic trends that raw logs alone cannot fully represent. Crucially, this expanded set serves primarily as a search space for the subsequent feature selection process, ensuring that only the most relevant predictors are retained to prevent overfitting given the limited dataset size. This is followed by a two-stage feature selection procedure utilizing univariate F-test filtering and Recursive Feature Elimination (RFE) to identify an optimal subset of 10 features, and automated hyperparameter tuning of the XGBoost model using Optuna, culminating in the training of the final model with the optimal configuration [41]. The following subsections detail each component of this process.

4.3.1. Feature Engineering

To address the limitation of having only four conventional well logs (RT, DEN, CNL, and AC) and to enable the model to capture complex nonlinear relationships associated with organic richness, a systematic feature engineering scheme was developed to expand the original parameters into 165 derived features (Figure 9). The purpose of generating these 165 features is to establish a comprehensive candidate pool. This strategy creates a broad search space for the subsequent feature selection process, allowing the model to distill effective predictors from high-dimensional data while ensuring statistical robustness despite the limited sample size [65].

Temporal features were generated through Gaussian smoothing with $\sigma =1.2$ and multi-scale rolling window analysis:

$$x_i^{\mathrm{smooth}} = \sum _j x_j\exp \left(-{\displaystyle \frac{(i-j{)}^2}{2{\sigma }^2}}\right)$$

(5)

where $x_j$ is the original log value at depth $j$, and $i$ is the target depth. Rolling statistics including mean, median, and quartiles (e.g., DEN_smooth, AC_smooth, CNL_smooth) were computed using window sizes of 3, 5, 7, and 10 samples to suppress noise while preserving formation boundaries.

The significance of this is twofold. The significance of this is twofold. First, regarding the window size, considering the standard logging sampling interval of 0.125 m, a 10-sample window corresponds to a vertical resolution of approximately 1.25 m. While this might smooth out high-frequency noise, it is physically appropriate for capturing the macroscopic background trends of thick source rock intervals. Second, the multi-scale rolling window analysis enables the model to simultaneously capture features of both thick beds (via larger windows) and thin interbeds (via smaller windows), thereby enhancing the model’s robustness and its adaptability to geological variations at different scales [66].

Dynamic features were derived to capture gradient strength and directional changes in petrophysical responses:

$$\Delta x_i = x_i - x_{i-1}$$

(6)

$${\Delta }^2{x}_i=\Delta x_i-\Delta x_{i-1}$$

(7)

where $\Delta x_i$ represents the rate of change between successive depths and ${\Delta }^2{x}_i$ captures acceleration. Metrics such as RT_median7 (median resistivity over a 7-sample window) detect stratigraphic transitions.

Their significance lies in their high sensitivity to “inflection points” or “abrupt changes” in the logs. These gradient features effectively delineate lithological boundaries, sharp changes in petrophysical properties, or stratigraphic sequence boundaries, which are often critical locations for the development or termination of source rocks.

Nonlinear transforms including polynomial terms (squares, cubes, square roots) and transcendental functions (logarithmic) were applied to represent complex dependencies. Logarithmic transformation of acoustic transit time (AC_log) enhances sensitivity to weak variations and enables recognition of nonlinear petrophysical relationships.

The rationale is that the relationship between log responses and TOC is often inherently nonlinear. For instance, the logarithmic transformation of acoustic transit time can enhance the model’s sensitivity to subtle variations and better fit the nonlinear compaction trends, thus empowering the model to learn these complex underlying petrophysical relationships.

Statistical features characterize local heterogeneity through quartile-based attributes and skewness:

$${\mathrm{IQR}}_w ={ Q}_{75,w} - Q_{25,w}$$

(8)

$${\mathrm{Skew}}_w={\displaystyle \frac{1}{w}}\sum _{j=1}^w {\left({\displaystyle \frac{x_j-\bar{x}}{{\sigma }_x}}\right)}^3$$

(9)

where $Q_{75,w}$ and $Q_{25,w}$ denote the 75th and 25th percentiles, $\bar{x}$ is the window mean, and ${\sigma }_x$ is the window standard deviation. Attributes such as AC_q25_5 and AC_q25_3 quantify thin-layer variability in organic-rich facies.

These statistics are significant because they serve as powerful indicators of formation heterogeneity. For example, in organic-rich laminated shales, the local variability of log responses, as represented by the interquartile range (IQR), might be greater, and skewness can reflect distributional asymmetry, providing the model with crucial clues for identifying specific sedimentary microfacies.

Interaction features including cross-log ratios (DEN/AC) and multiplicative terms (RT times DEN, RT times DEN times AC) encode combined effects of density reduction, acoustic slowness increase, and resistivity variation characterizing organic-rich intervals.

Their importance stems from their ability to capture the synergistic effects of multiple petrophysical properties. For instance, source rocks typically exhibit a combination of low density from organic matter, high acoustic transit time from under-compaction, and high resistivity from hydrocarbons. While an anomaly in a single log may be ambiguous, a combined feature integrating all three provides a much stronger and more definitive indicator of organic-rich intervals.

Frequency features were extracted via Fast Fourier Transform:

$$\mathrm{FFT}\left(x\right) = \sum _{k=0}^{n-1} x_k\exp \left(-{\displaystyle \frac{2\pi ikj}{n}}\right)$$

(10)

where $x_k$ is the log value at sample $k$, $n$ is the total number of samples, and $i$ is the imaginary unit. Maximum and mean FFT amplitudes reflect cyclic depositional patterns in lacustrine environments.

The value of these features is their ability to identify potential periodicity or cyclicity in sedimentation. In lacustrine environments, factors like climate cycles often lead to rhythmic depositional patterns, which manifest as specific frequency components in well logs. This provides the model with valuable sedimentological context.

Distance features standardize measurements and identify anomalous intervals using Z-scores:

$$Z_i = {\displaystyle \frac{x_i - \bar{x}}{s_x}}$$

(11)

$$D_i=\mid x_i-\bar{x}\mid$$

(12)

where $x_i$ is the log value at depth $i$, $\bar{x}$ is the column mean, and $s_x$ is the standard deviation.

The purpose of these features is not only to eliminate the influence of different log scales but also to measure the deviation of a data point from the global mean. This allows the model to effectively identify “anomalous” intervals that may correspond to high TOC values.

Overall, the systematic approach expanded the four original logs into 165 derived attributes providing comprehensive coverage of temporal patterns, gradients, nonlinear behavior, local variability, and cross-log interactions [66]. This extensive feature engineering lays a necessary foundation for the subsequent feature selection process.

4.3.2. Feature Selection

Given the high dimensionality of the engineered feature space comprising 165 variables, a two-stage selection pipeline was implemented to distill the most informative predictors [67].

In the first stage, univariate F-test filtering evaluated linear correlations with TOC, retaining the 60 most significant features. This preliminary screening effectively eliminated noisy variables and streamlined subsequent processing [68].

The second stage employed RFE with a Random Forest regressor, iteratively removing the least important features based on ensemble rankings. To determine the optimal number of retained features, principal component analysis (PCA) was subsequently applied to assess the cumulative explained variance and intrinsic dimensionality of the candidate feature subsets. As shown in Figure 10, the analysis revealed that 10 features captured the primary modes of variation while preserving over 91% of the cumulative variance, providing an effective trade-off between information retention and model parsimony. This final configuration ensured the preservation of meaningful multivariate and nonlinear relationships in the logging measurements while effectively mitigating overfitting risk.

This methodology led to substantial dimensionality reduction—from 165 to 10 features—while preserving essential geophysical characteristics, thus improving computational efficiency and robustness for subsequent model development.

4.3.3. Hyperparameter Optimization

The hyperparameters of the XGBoost model were optimized using a Bayesian approach implemented with the Optuna framework and a TPE sampler [41,42]. The optimization process comprised 80 trials exploring a predefined search space: n_estimators (100–500), max_depth (3–10), learning_rate (0.01–0.3), subsample (0.6–1.0), colsample_bytree (0.6–1.0), reg_alpha (0–15), reg_lambda (0–15), and min_child_weight (1–15). The objective was to minimize the fivefold cross-validation root mean squared error (RMSE) on the full training set consisting of 100 samples and 10 selected features.

RMSE was selected as the optimization metric over R² due to its stronger penalty on large prediction errors, which is particularly critical for TOC estimation where outlier predictions can significantly impact reservoir evaluation decisions [69,70]. This metric choice ensures the model prioritizes reducing absolute prediction errors rather than merely maximizing variance explained.

Prior to optimization, all features were standardized using quantile transformation to a normal distribution to ensure stable convergence and comparable scales. The optimization identified an optimal configuration with n_estimators = 384, max_depth = 3, learning_rate = 0.035, subsample = 0.870, colsample_bytree = 0.739, reg_alpha = 0.002, reg_lambda = 13.011, and min_child_weight = 6, which achieved a minimum cross-validation RMSE of 0.8352 (Figure 11).

Using this optimal configuration, the final XGBoost model was trained on the training dataset.

4.3.4. Model Evaluation Metrics

To assess the predictive accuracy and robustness of the developed model, five quantitative metrics were defined and applied to both training and test sets. Models with higher $R^2$ and lower MAE, RMSE and variance of absolute error indicate better predictive performance [70].

The coefficient of determination ($R^2$), mean absolute error (MAE), and root mean squared error (RMSE) are defined as:

$$R^2 = 1 - {\displaystyle \frac{{\sum }_{i=1}^n (y_i-{\widehat{y}}_i{)}^2}{{\sum }_{i=1}^n (y_i-\bar{y}{)}^2}}$$

(13)

$$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n \mid y_i-{\widehat{y}}_i\mid$$

(14)

$$\mathrm{RMSE} =\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n (y_i-{ \widehat{y}}_i{)}^2}$$

(15)

The variance of absolute error (σ²) is computed as:

$${\sigma }^2 = {\displaystyle \frac{1}{n}}{\sum }_{i=1}^n (\mid e_i\mid - \mid \bar{e}\mid {)}^2$$

(16)

To facilitate comparison across different TOC value ranges, the normalized RMSE (NRMSE) was computed as:

$$\mathrm{NRMSE} = {\displaystyle \frac{\mathrm{RMSE}}{y_{\mathrm{m}\mathrm{a}\mathrm{x}}-y_{\mathrm{m}\mathrm{i}\mathrm{n}}}}$$

(17)

In the above equations, $y_i$ represents measured TOC values, ${\widehat{y}}_i$ represents predicted TOC values, $\bar{y}$ is the mean of measured TOC values, $n$ is the number of samples, $\mid e_i\mid$ represents the absolute prediction error for each sample, $\mid \bar{e}\mid$ is the mean absolute error, and $y_{\mathrm{m}\mathrm{a}\mathrm{x}}$ and $y_{\mathrm{m}\mathrm{i}\mathrm{n}}$ represent the maximum and minimum measured TOC values in the dataset.

R² measures the proportion of variance explained by the model, ranging from 0 to 1. MAE provides a linear measure of average prediction error magnitude and is less sensitive to outliers than quadratic error measures. RMSE penalizes larger prediction errors more heavily, offering greater sensitivity to extreme deviations [69,70,71]. The variance of absolute error quantifies the consistency and stability of model predictions across the dataset—lower variance indicates more uniform prediction errors, suggesting greater model robustness and reliability. NRMSE is dimensionless and enables direct comparison across datasets with different TOC value ranges.

All metrics were computed for both training and test sets to comprehensively assess model accuracy and generalization capability.

5. Results and Discussion

5.1. Hyperparameter Optimization Convergence

The hyperparameter space was systematically explored through 80 trials using the Optuna framework with TPE sampling. The convergence behavior of the cross-validation RMSE during this process already illustrated in Figure 10. Characteristic fluctuations in RMSE values were observed as the algorithm probed diverse parameter configurations, reflecting the expected exploration–exploitation dynamics of the Bayesian optimization strategy.

The optimal parameter set was identified during the optimization process, achieving a minimum cross-validation RMSE of 0.8352, which corresponds to approximately 12% of the total TOC value range in this dataset. Such an error level indicates the model’s capability to deliver predictions sufficiently precise for practical TOC profiling.

This convergence pattern confirms the efficacy of the Bayesian optimization approach. The TPE sampler adaptively balanced exploration of uncharted hyperparameter regions with focused exploitation of promising combinations, efficiently guiding computational resources toward optimal solutions. Compared to conventional grid or random search methods, this adaptive strategy enables more rapid identification of optimal parameter sets. The systematic RMSE minimization ensured the final model was tuned for minimal prediction error while maintaining robust generalization capability, as evidenced in the subsequent model prediction performance

5.2. Feature Importance Analysis

Feature importance analysis based on XGBoost’s intrinsic gain metric revealed hierarchical contributions of the 10 selected engineered features for TOC prediction (Figure 12). AC_smooth (Gaussian-smoothed acoustic log) dominated with approximately 43.9% of total feature importance, reflecting the fundamental petrophysical principle that organic-rich intervals exhibit elevated acoustic transit times due to the high acoustic slowness of kerogen (~525 μs/m), making acoustic response a primary indicator of organic matter abundance [7].

DEN_smooth (Gaussian-smoothed density log) ranked second with 32.7% importance, underscoring that organic-rich shale intervals characteristically exhibit lower matrix densities due to the presence of organic matter (density typically <1.2 g/cm³, compared to quartz at 2.65 g/cm³) [12,18]. Together, AC_smooth and DEN_smooth jointly account for approximately 76.6% of total model gain, demonstrating that multi-scale noise suppression via Gaussian smoothing is critical for reliable TOC estimation and constitutes the primary signal pathway in the learned predictive model.

The remaining eight predictors comprise a hierarchy of statistical aggregations and nonlinear transformations: AC_q25_3 (7.2%), RT_median7 (5.7%), RT_q25_5 (3.1%), RT_ma10 (1.9%), DEN_q25_7 (1.8%), AC_q75_7 (1.7%), AC_ma7 (1.3%), and AC_median3 (0.8%). The recurrent appearance of quartile-based attributes (AC_q25_3, RT_q25_5, DEN_q25_7, AC_q75_7, RT_median7) reflects the model’s reliance on distribution and central tendency characteristics within rolling windows, capturing thin-layer heterogeneity and subtle variations in organic matter distribution. Resistivity-derived features (RT_median7, RT_q25_5, RT_ma10) collectively contribute approximately 8.7%, providing complementary information on hydrocarbon saturation and pore fluid variations associated with organic-rich intervals [7,14]. The moving average feature (AC_ma7) and median-based aggregation (AC_median3) further demonstrate the value of multi-scale temporal smoothing in isolating meaningful geological signals from high-frequency noise.

Through systematic feature engineering that expanded the original four-log dataset into 165 candidate attributes, followed by two-stage feature selection refined via PCA, the model distilled 10 core predictors that capture the essential multi-scale patterns governing TOC variability. The overwhelming dominance of smoothed acoustic and density logs, supplemented by statistically aggregated rolling windows and resistivity attributes, underscores that effective TOC estimation in shale formations depends primarily on capturing noise-robust density and acoustic responses, with additional insights provided by resistivity-based constraints on pore fluid properties.

5.3. Model Performance

To systematically assess the predictive capability of machine learning and empirical approaches for TOC estimation, we developed and evaluated multiple prediction models using a standardized dataset (100 training and 25 test samples). The comparative analysis encompassed the empirical Passey method, RF, and two XGBoost variants (baseline and optimized), with a CNN model reported by Wang et al. [63] included for reference. All models were evaluated using identical metrics (R², NRMSE, and variance of absolute error) on the test set to ensure fair comparison.

As shown in

Table 1. Performance comparison of TOC prediction methods.

Method	Test R2	NRMSE	${\mathit{\sigma }}^2$
Passey Method	0.4136	0.1881	0.634
RF	0.7075	0.1329	0.298
CNN (Wang et al. [63])	0.8283	0.1010	0.176
XGBoost (Baseline)	0.7889	0.1129	0.240
XGBoost (Optimized)	0.9395	0.0604	0.109

, the optimized XGBoost model demonstrates superior predictive performance across all evaluation metrics compared to benchmark methods. The model achieved R² = 0.9395 and NRMSE = 0.0604 on the test set, with improvements of 127.3% over the Passey method, 32.8% over RF, 18.9% over baseline XGBoost, and 13.4% over CNN. Notably, the optimized XGBoost attained the lowest variance of absolute error at 0.109, substantially lower than competing methods: Passey method 0.634, RF 0.298, CNN 0.176, and baseline XGBoost 0.240. This indicates consistent prediction accuracy across the sample range.

The Passey method exhibited the most limited performance with R² = 0.4136 and NRMSE = 0.1881, as shown in Figure 13, attributable to reliance on universal empirical relationships that lack dataset-specific calibration. Random Forest achieved moderate performance with R² = 0.7075 and a notable generalization gap of 0.2057, indicative of overfitting. As shown in Figure 14A,B, the model demonstrated high training accuracy of 0.9132 but a substantial performance decline on the test set. The CNN model yielded intermediate results with R² = 0.8283 and NRMSE = 0.1010 on the test set, as presented in Figure 14C,D. Although outperforming Random Forest, its performance remained below the optimized XGBoost approach. The baseline XGBoost model developed without feature engineering attained R² = 0.7889 with a generalization gap of 0.1037, as depicted in Figure 14E,F.

The optimized XGBoost demonstrated superior generalization with a minimal gap of 0.0331, comparing train R² of 0.9726 to test R² of 0.9395 as shown in Figure 14G,H. This indicates effective model stability and robust performance across training and evaluation phases. On the test set, additional performance metrics include MAE of 0.3392 and RMSE of 0.4259, representing 46.3% and 46.4% reductions compared to the baseline XGBoost model, respectively. Also, the reduced prediction variance of 0.109 demonstrates that comprehensive feature engineering and hyperparameter optimization enhanced both prediction accuracy and consistency across different TOC value ranges. Such consistency is essential for reliable subsurface estimation in practical applications. Nevertheless, it is crucial to interpret these high-performance metrics within the context of the dataset size. While the minimal generalization gap implies that overfitting has been effectively mitigated, the evaluation on a relatively small independent test set could potentially limit the statistical certainty of the model’s transferability to broader geological settings. Therefore, these results primarily serve as a successful validation of the proposed feature engineering and optimization framework, while further verification on larger, multi-well datasets remains a necessary step for establishing universal applicability.

6. Conclusions

This study successfully developed and validated an optimized XGBoost model for predicting TOC from conventional well logs in the Shahejie Formation. The model achieved R² = 0.9395 and NRMSE = 0.0604 on an independent test set, demonstrating substantial improvement over established benchmarks including the empirical Passey method, Random Forest baseline, and a previously published CNN model. The minimal generalization gap of 0.0331 confirms model stability and effective mitigation of overfitting, attributable to integrated systematic feature engineering, rigorous feature selection, and Bayesian hyperparameter optimization.

However, the current evaluation is constrained by a limited independent test set of 25 samples, which restricts the statistical robustness of findings. Future work should prioritize external validation using larger, geologically diverse datasets to assess model robustness across different wells and formations. Additional investigation of stratigraphic constraints to refine predictions in complex lithological sequences and advanced ensemble techniques to enhance predictive consistency would further advance the framework.