A Hybrid Deep Learning Approach for Small-Sample TOC Prediction in Saline Lacustrine Shale

Abstract

Total organic carbon (TOC) is a key parameter for geological sweet spot optimization and resource evaluation. However, accurate TOC prediction remains challenging under small-sample conditions because conventional physical and machine learning methods are commonly limited by insufficient labeled data. To address this problem, this study proposes a hybrid deep learning framework that integrates a convolutional autoencoder with a BP neural network (CAE-BPNN). In this framework, the CAE is first used to learn representative feature expressions from abundant unlabeled logging data, and the learned encoder is then transferred to the supervised prediction stage, where labeled samples are used for TOC prediction through the BPNN. The dataset includes 177 measured TOC samples and 1388 unlabeled logging samples from the Fengcheng Formation shale in Well MY1, in the Mahu Sag, and data from Well MY2 are used for independent validation. Model performance is evaluated using five-fold cross-validation with the coefficient of determination (R²) and root mean square error (RMSE) as metrics. For Well MY1, CAE-BPNN achieves the best performance, with R² = 0.89 and RMSE = 0.061, outperforming CNN (R² = 0.85, RMSE = 0.075), GBDT (R² = 0.83, RMSE = 0.076), RF (R² = 0.81, RMSE = 0.082), and BPNN (R² = 0.77, RMSE = 0.085). In the independent validation using Well MY2, CAE-BPNN also shows superior predictive performance, with R² = 0.81 and RMSE = 0.367. These results indicate that unlabeled logging data can effectively enhance feature representation and improve TOC prediction accuracy under limited labeled-sample conditions. The proposed method provides an effective solution for small-sample TOC prediction and offers a reliable basis for movable oil evaluation using the oil saturation index (OSI), as well as a reference for predicting other geological parameters such as S1, S2, and porosity.

1. Introduction

The exploration and development of shale oil have become the hottest research topic in the global oil and gas field industry. It is estimated that the recoverable reserves of shale oil in the world are much higher than those of conventional oil. However, compared with marine shale oil in the United States, lacustrine shale oil is mainly developed in China, which is characterized by frequent thin interbeds, strong heterogeneity, and complex organic matter enrichment patterns [1,2,3,4,5]. These characteristics pose significant challenges for geological sweet spot evaluation and resource potential assessment in shale oil. TOC is a key organic geochemical indicator for micro-migration evaluation and for identifying effective source rocks [6,7]. Accurate determination of TOC is critical for geological sweet spot evaluation in shale oil. However, TOC is typically measured in the laboratory. Consequently, acquiring continuous along-profile TOC variation requires extensive regional coring over large longitudinal extents. This process demands substantial time and financial investment. Moreover, constrained by various practical limitations, the quantity of core samples obtained is often insufficient to meet the requirements of tasks demanding substantial TOC data [8,9,10,11]. Since the change in TOC can be reflected by a variety of logging curves (GR, SP, AC, etc.) [12], the acquisition of logging data is easier. Therefore, it is an effective method to predict TOC by establishing the relationship model between logging curves and TOC [13,14,15].

TOC prediction approaches include physical models and data-driven methods. Passey et al. [16] proposed the widely used ΔlogR method. The method first defines two baselines where resistivity and porosity logs coincide and then computes TOC. Subsequently, many researchers have continuously improved ΔlogR to enhance the prediction accuracy. Passey et al. [17] improved the ΔlogR method, significantly enhancing its predictive performance in over-mature reservoirs. Zhao et al. [18] replaced the theoretical baseline with an approximate linear baseline based on the change of the slope of acoustic wave and resistivity with time, thereby improving the prediction effect of ΔlogR. Based on the dynamic baseline theory, Zhu et al. [19] calculated two ΔlogR values by using the acoustic curve with TOC information and the theoretical acoustic curve, and proposed a double-difference ΔlogR method. Zhou et al. [20] proposed the ΔLogR-GR method by combining the ΔlogR method and the natural gamma method. Although the above traditional methods and improved methods are simple and easy to use and have certain interpretability, this series of methods also have some shortcomings: (1) subjectivity in selecting the required parameters; (2) excessive dependence on a single logging curve, so that it cannot fully tap the hidden information of the logging curve; (3) when dealing with complex nonlinear relationships, there are problems of insufficient capacity and heavy workload.

With the advent of the era of artificial intelligence (AI), AI has also made many achievements in the field of oil and gas, such as sedimentary reservoirs, logging evaluation, geophysical exploration, reservoir engineering, and CO₂ storage [21,22,23,24,25,26]. Nowadays, the methods of predicting TOC are basically data-driven [27,28]. The commonly used methods include Gaussian regression, RF, support vector machine, GBDT, BPNN, and CNN [21,29,30,31]. Many scholars have applied various combinations of optimization algorithms and traditional models to regression prediction, so a variety of hybrid models have emerged, including the combination of the Bayesian regularization algorithm and BPNN [32]; the particle swarm optimization algorithm combined with support vector machines [33]; the combination of the genetic algorithm and BPNN [34]; the deep belief network combined with long short-term memory networks [35]; and the combination of principal component analysis technology and neural networks [36]. In the past, when doing TOC prediction tasks, whether it is a single model, a traditional model based on different optimization algorithms for parameter adjustment, or a traditional model using feature processing algorithms, most of them belong to the category of supervised learning. In the prediction of TOC, although the results of these models are very good, their performance is highly dependent on the quantity and quality of labeled data. However, geological experimental data are often not easy to expand, and data quality problems are difficult to fully control, which may lead to insufficient learning and training of the model. In addition, when these models are applied to new areas with large differences in geological conditions, the stability and generalization ability of the models are often insufficient, so that the accuracy will be greatly reduced. The above situations are generally the common problems of previous TOC prediction models. Therefore, it is urgent to solve the problem of insufficient model generalization ability and prediction accuracy of small-sample TOC prediction tasks.

The quality of the model prediction results is directly related to the feature engineering [37,38]. Especially for the small-sample regression prediction task, the processing of features is more important [39]. The logging curve used to predict TOC is easy to obtain, the amount of data is large, and it is related to many physical parameters. The natural and unsupervised learning characteristics are appropriate. The advantage of unsupervised learning is that it can independently learn the potential information of unlabeled data (logging data without measured TOC). It focuses on deep mining at the feature level, which can extract more sufficient and effective feature expression from the original data. Therefore, considering the use of logging curves without measured TOC as the input of unsupervised learning, it is expected to break through the dependence on measured labels and extract more valuable deep information for TOC prediction. To this end, this study proposes a convolutional autoencoder-BP neural network (CAE-BPNN) method, which is an end-to-end learning framework that combines unsupervised learning and supervised learning, making full use of the advantages of unsupervised learning. This method is mainly aimed at small-sample learning tasks to obtain more accurate prediction results and significant generalization ability, and it is rarely applied to TOC prediction. Because the geological logging data has the characteristics of time series correlation and local distribution, convolution can extract the important characteristics of logging data [40]. Therefore, in the unsupervised stage of this study, an autoencoder based on convolution operation is adopted. First, the model learns the effective representation of the input data, and then automatically extracts the most discriminative features through the reconstruction process [41]. Subsequently, these learned features are input into the back-end TOC predictor. The predictor adopts a dual-path structure: the main part performs the main prediction task based on the BPNN, while the branch part introduces the residual network to enhance the representation ability of the model and optimize the training process. This end-to-end hybrid model has reference significance for solving the problem of physical parameter prediction under small-sample conditions, and also provides innovative technical support for the quantitative evaluation of TOC in lacustrine shale oil.

Compared with existing TOC prediction methods, the proposed CAE-BPNN framework is designed to address geological heterogeneity and limited labeled samples in saline lacustrine shale oil reservoirs. Conventional ΔlogR-based methods are physically interpretable but sensitive to baseline selection and local petrophysical assumptions [16,17,18,19,20], whereas data-driven methods such as RF, GBDT, BPNN, CNN, SVM, and GPR generally depend on sufficient labeled TOC samples and may become unstable when laboratory measurements are sparse [21,27,28,29,30,31]. Recent small-sample and semi-supervised studies have demonstrated the potential of unlabeled logs for representation learning [40,42,43]. Building on this concept, this study integrates unsupervised CAE pretraining, supervised BPNN fine-tuning, and a residual branch to learn transferable logging representations from 1388 unlabeled samples while using only 177 measured TOC samples for calibration. This framework is applied to the Fengcheng Formation shale in the Mahu Sag, where strong reservoir heterogeneity and limited training samples challenge conventional models. Its prediction accuracy and generalization ability are evaluated across two wells and compared with RF, GBDT, CNN, and BPNN using R² and RMSE. The results show that CAE-BPNN can reduce the effects of heterogeneity and label scarcity, providing a practical approach for small-sample prediction of TOC and related geological parameters.

2. Convolutional Autoencoder-BP Neural Network Model

In this study, a two-stage learning framework is employed to integrate unsupervised feature extraction with supervised TOC prediction. The end-to-end regression model, which jointly leverages labeled and unlabeled logging data and integrates a CAE with a BPNN, incorporates features learned during the unsupervised stage, thereby mitigating label scarcity in small-sample settings and enhancing the model’s generalization. The workflow is illustrated in Figure 1.

(1)

Data preparation stage: The dataset comprises labeled samples with TOC annotations and unlabeled logging data.

(2)

CAE training stage: The model is first initialized, after which the CAE is trained on unlabeled data to learn intrinsic feature relationships, reconstruct the input features, and minimize the reconstruction loss between the input and its reconstruction (Figure 1a).

(3)

TOC prediction stage: In the backbone, the pretrained CAE is fine-tuned on the labeled data, and the encoder outputs are fed into a BPNN to yield the backbone prediction; in the residual branch, a skip connection is applied to estimate the residual and generate the residual prediction. The final prediction is obtained by summing the backbone and residual outputs (Figure 1b).

2.1. Convolutional Autoencoder

Building on the traditional autoencoder, the CAE (Figure 1a) is an unsupervised learning framework well suited for feature dimensionality reduction, extraction, and denoising. Its architecture consists of two mirror-symmetric components: an encoder and a decoder. The encoder uses convolutional operations to compress the input into a compact latent-space representation. Subsequently, the decoder employs transposed convolutions to reconstruct the original input from this latent representation, which may require cropping to align spatial dimensions [41,44]. Training minimizes the discrepancy between the original input and its reconstruction, typically measured by the mean squared error (MSE) loss.

(1)

Encoder: The encoder maps the input to a low-dimensional latent space that captures principal feature relationships and preserves the salient information of the original data.

$$z=f_{\mathrm{encoder}}\left(\mathrm{Conv}\left(X\right)\right)$$

(1)

where X denotes the input data; Conv denotes the convolution; z denotes the potential space.

(2)

Decoder: The decoder maps the low-dimensional latent representation back to the original feature space to reconstruct the input data.

$$\widehat{X}=f_{\mathrm{decoder}}\left(\mathrm{DeConv}\left(z\right)\right)$$

(2)

where$\widehat{X}$denotes the output of the reconstruction; DeConv denotes the transposed convolution.

(3)

Training objective: Minimize the MSE between the input and the reconstructed output, thereby enabling the model to learn intrinsic relationships among features.

$$L_{MSE}=|X-\widehat{X}{|}^2$$

(3)

where $L_{MSE}$ denotes the MSE loss function.

2.2. BP Neural Network Model

BPNN constitutes a multi-layer feedforward architecture that employs error backpropagation for parameter optimization. This network utilizes a dual mechanism: forward propagation transmits input signals to generate outputs, while backward propagation distributes error signals to update network weights [45]. BPNN is adept at handling complex nonlinear relationships encountered in engineering research and demonstrates robust performance in diverse applications including data classification, clustering, and regression analysis.

A typical BPNN structure comprises an input layer, one or more hidden layers, and an output layer (Figure 1b). Neurons within each layer are densely interconnected with all neurons in the subsequent layer, forming a fully connected topology. The output from a preceding layer serves as the input to the following layer. The core training principle relies on the error backpropagation algorithm, which iteratively adjusts connection weights and neuron thresholds to minimize the discrepancy between predicted and target outputs. During training, if the network output deviates significantly from the expected value, the input data is reprocessed. The algorithm systematically refines the connection strengths and thresholds between the input and hidden layers and between the hidden and output layers until the prediction error falls within an acceptable tolerance, signifying convergence. Networks trained via this method exhibit strong nonlinear mapping capabilities and possess a flexible structure adaptable to various problem domains.

3. An Application Example for Shale Oil

3.1. Field Background

The Junggar Basin (Figure 2a), located in northwestern China, is a key junction of three paleoplates and has experienced multiple tectonic cycles. Multiple superposed source-rock sets across its structural units collectively control the basin’s extensive, complex hydrocarbon distribution [46]. As one of the basin’s most active sags, the Mahu Sag (Figure 2b) occupies the northwestern part of the Central Sag and covers approximately 5000 km² [47]. It is bounded by the Baikouquan Fault Zone to the west and the Xiazijie Fault Zone to the east; adjacent uplifts include the Dabasong and Zhongguai uplifts, with the Xiayan and Sansanquan uplifts farther east and the Shuitan Uplift to the northeast [42].

This study focuses on the Lower Permian Fengcheng Formation (P₁f) source rocks in the Mahu Sag (Figure 2c). Deposited in a fan-delta–lacustrine system, the formation is widespread, thick, and wedge-shaped, thickening northwestward. Stratigraphically, it comprises three members (ascending): P₁f₁, P₁f₂, and P₁f₃. P₁f₁ mainly consists of massive basalt, dolomitic mud shale, and argillaceous siltstone; local sodium carbonate indicates incipient alkaline-lake deposition. P₁f₂ is dominated by argillaceous dolomite, dolomitic shale, argillaceous siltstone, and alkali-bearing shale; abundant alkaline minerals mark the peak of alkaline-lake evolution. The lower P₁f₃ comprises interbedded argillaceous dolomite and dolomitic shale, grading upward into coarse fan-delta clastic; reduced dolomite and alkaline minerals indicate the waning of alkaline-lake conditions [43,48]. Overall, the Fengcheng Formation records an alkaline lacustrine setting with frequent laminated mud-shale interbeds, pronounced rhythmicity, and maturation from low to high during hydrocarbon generation and expulsion, establishing prerequisites for a prolific petroleum system.

3.2. Related Work

Well logging is the most widely used technique in petroleum exploration. Its high vertical resolution allows lithology to be identified from logging curves; variations in logging responses correlate with organic matter content [12,14,15]. To predict TOC accurately, the effects of different logging curves must be analyzed, and key curves selected as input features.

During feature selection, core depths must be corrected. Depth-measurement errors arise from different compression coefficients of the cable and drill pipe, causing mismatches between core and logging depths [30]. Feature selection follows two criteria: prior experience and R². R² quantifies the correlation between logging curves and TOC. Together, these criteria determine the final input features. R² can be calculated using the following equation:

$$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2}{{\sum }_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(4)

where$y_i$denotes the true value; $\widehat{y_i}$denotes the predicted value; $\overline{y}$ denotes the mean value for all samples.

It should be emphasized that the R² analysis was used as an initial screening tool rather than as the sole criterion for feature selection. A low single-variable linear R² does not necessarily indicate the absence of information in a multivariate nonlinear model, because weakly correlated logs may still contain complementary geological signals or interact nonlinearly with other curves. Therefore, GR, AC, CNL, and DEN were retained by jointly considering linear correlation, logging availability, data quality, petrophysical meaning, and previous TOC-logging studies [12,15,27,28]. The four curves reflect organic-matter enrichment, lithology/mineral composition, porosity, compaction, and hydrogen-bearing components from different perspectives. Nevertheless, the limited number of input channels constrains the representation capacity of the CAE, and future work should add mutual information, SHAP-based feature attribution, variance inflation factor analysis, and sensitivity tests to further quantify nonlinear information content and multicollinearity [27,28,49].

The correlation between TOC and logging curves is measured by R² (Figure 3). Sonic travel time, neutron porosity, and borehole caliper show positive correlations with TOC, whereas gamma ray, resistivity, and density show negative correlations. These patterns accord with local lithology, organic matter occurrence, and reservoir evolution. The R² for AC and TOC is 0.75 (Figure 3d). Organic rich shale contains abundant organic pores and microcracks, so compressional waves attenuate and travel time increases. The R² for CNL and TOC is about 0.41 (Figure 3e), reflecting a higher hydrogen index. Organic matter, especially kerogen and adsorbed hydrocarbons, is hydrogen-rich and oxygen-poor, which yields apparently high neutron porosity. The R² between CAIL and TOC is about 0.27 (Figure 3a), consistent with washouts in organic rich layers caused by low drill ability. The R² values for DEN, GR, and RT with TOC are about 0.14, 0.03, and 0.07, respectively (Figure 3b,c,f), indicating low sensitivity due to mineral composition, fluid properties, and thermal evolution in the Fengcheng Formation. Although GR and DEN show relatively weak single-variable linear correlations with TOC in the Fengcheng Formation, they were retained because GR, AC, CNL, and DEN are widely recognized as effective conventional logs for TOC prediction and can jointly reflect shale content, lithology/mineral composition, acoustic response, hydrogen-bearing components, porosity, compaction, and organic-matter enrichment; thus, weak linear correlation does not necessarily exclude their complementary value in a multivariate nonlinear model [12,15,27,31]. Based on these results and previous work, the selected input features included GR, AC, CNL, and DEN.

After identifying the required input features, the dataset was organized. The data were split into labeled and unlabeled sets based on the presence of measured TOC (Figure 4). Both sets included the selected logging curves. The labeled set contained 177 Fengcheng Formation samples with measured TOC and corresponding logs. The unlabeled set contained logs for 1388 Fengcheng Formation samples and no measured TOC. Before model training, missing values and outliers were checked. Input features were standardized to unify scale, yielding zero mean and unit standard deviation. To avoid potential information leakage during cross-validation, all data-driven preprocessing and model-fitting procedures were controlled within each training fold. Specifically, the mean and standard deviation used for feature standardization were estimated only from the training subset and then applied to the corresponding validation subset. CAE pretraining and supervised fine-tuning were performed using training-fold data only, and the validation fold was used only for model evaluation. The main model settings were fixed before final cross-validation and kept consistent across folds to reduce fold-specific overfitting. Standardization can be calculated using the following equation:

$$X={\displaystyle \frac{x-\mathsf{\mu }}{\mathsf{\sigma }}}$$

(5)

where x denotes the input data; μ represents the mean; $\mathsf{\sigma }$ denotes the standard deviation; X denotes standardized data.

In this study, the small-sample problem is quantitatively defined by the limited number of labeled samples available for supervised learning: only 177 samples have measured TOC labels, whereas 1388 logging samples do not have TOC measurements. The labeled samples account for approximately 11.3% of all MY1 logging samples used in the learning workflow. This imbalance is typical in geological prediction because laboratory geochemical measurements are much more expensive and less continuous than well logs [8,9,10,11]. The unlabeled samples were therefore used only for unsupervised representation learning by the CAE, whereas supervised TOC calibration was performed using the labeled samples.

To avoid information leakage during cross-validation, all data-driven preprocessing steps should be performed within each training fold. Specifically, the mean and standard deviation used for feature standardization should be fitted only on the training subset and then applied to the corresponding validation subset. CAE pretraining, hyperparameter selection, and BPNN fine-tuning should also be conducted using training-fold data only; the validation fold should be used only for model evaluation.

In the evaluation stage of TOC prediction, five-fold cross-validation is used (Figure 5), and root mean square error and R² serve as evaluation metrics. Cross-validation reduces dependence on a single partition and uses limited data more fully, producing more stable and reliable estimates. RMSE measures the difference between predictions and observations and is sensitive to outliers, which aids assessment in extreme cases. A smaller RMSE indicates better predictive performance. R² reflects the goodness of fit. Values closer to one indicate a better fit to the data. Using both metrics provides a more comprehensive and reliable evaluation of predictive ability. R² was shown in Equation (4). The RMSE can be calculated using the following equation:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{\mathrm{n}}}\sum _{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i}}-\widehat{{\mathrm{y}}_{\mathrm{i}}}\right)}^2}$$

(6)

where ${\mathrm{y}}_{\mathrm{i}}$ denotes the true value; $\widehat{{\mathrm{y}}_{\mathrm{i}}}$denotes the predicted value.

3.3. Building Convolutional Autoencoder-BP Neural Network Model

A CAE BPNN model is proposed. In the unsupervised stage, unlabeled logging data are used, and a CAE performs feature learning. The encoder input layer takes four logging curves: AC, CNL, GR, and DEN. As shown in Figure 6a, the convolutional part has four layers. The kernel size is 3. The numbers of neurons are 64, 32, 16, and 8. The decoder mirrors the encoder and contains four transposed convolution layers. The numbers of neurons are 8, 16, 32, and 64. The output layer reconstructs the input. Each layer is followed by BatchNorm, ReLU, MaxPool, and Dropout (0.2). The final layer uses Tanh to constrain the reconstruction range.

The architecture was designed to balance feature-representation capability and overfitting risk under limited labeled-sample conditions. The four-layer 1D convolutional structure enables hierarchical extraction of local logging-response patterns, while the kernel size of 3 captures short-range vertical variations without introducing excessive parameters. Dropout and weight decay are used as regularization strategies to improve model robustness. In addition, the residual branch preserves part of the original logging information and improves gradient propagation during training. Alternative sequential architectures, such as LSTM and Transformers, were not adopted in this study because the number of labeled TOC samples is limited and the input feature dimension is relatively small. These models generally require larger datasets to robustly learn temporal dependencies and may increase the risk of overfitting under small-sample conditions.

In the supervised stage, the encoder is initialized from the trained autoencoder and fine-tuned on labeled data, while the decoder is discarded; the encoder output then serves as the input to the BPNN. The BPNN (Figure 6b) comprises four fully connected layers. The numbers of neurons are 32, 128, 64, and 32. Each layer uses BatchNorm, ReLU, MaxPool, and Dropout (0.2). The output layer predicts TOC (main). In parallel, the original features pass through a residual branch (Figure 6c) composed of three convolutional layers [50,51]. The numbers of neurons are 32, 16, and 4, and the activation is Relu. The residual branch predicts TOC (res), and the final TOC prediction is obtained by summing the main and residual outputs.

During training and validation, five-fold cross-validation (Figure 5), the AdamW optimizer, gradient clipping, and an early stopping mechanism with patience set to 30 improve reliability and stability and help prevent overfitting. A cosine annealing learning rate scheduler decreases the learning rate periodically during training, simulates cooling, and helps escape local optima.

(1)

The input features are standardized as described in Equation (5).

(2)

The encoder applies four layers of 1D convolution:

$$z=f_{\mathrm{encoder}}\left({\mathrm{Conv}}_4\left(\cdots {\mathrm{Conv}}_1\left(x\right)\right)\right)$$

(7)

$$h_m=\mathrm{MaxPool}\left(\mathrm{Dropout}\left(\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left(\mathrm{BatchNorm}\left(\mathrm{Conv}\left(h_{m-1}\right)\right)\right)\right)\right)$$

(8)

$$h_0=x$$

(9)

where x denotes the standardized features of the input; BatchNorm denotes the batch normalization; Dropout denotes the regularization; MaxPool denotes the maximum pooling; ReLU denotes the activation function; Conv denotes the convolution.

(3)

The decoder mirrors the encoder. Four layers of transposed convolution are applied, and the final layer uses Tanh to output the reconstructed feature:

$$d_L=f_{\mathrm{decoder}}\left({\mathrm{DeConv}}_4\left(\cdots {\mathrm{DeConv}}_1\left(z\right)\right)\right)$$

(10)

$$\widehat{x}=\mathrm{Tan}\mathrm{h}\left(\mathrm{Linear}\left(d_L\right)\right)$$

(11)

where DeConv denotes the transposed convolution; Linear denotes the linear processing.

(4)

The autoencoder is trained to minimize a weighted loss between the input and the reconstructed output. MSE is calculated as described in Equation (3).

After training the autoencoder, the encoder is initialized from it and fine-tuned on labeled data, while the decoder is discarded. The BPNN receives the encoded features from the encoder as inputs and processes them through four fully connected layers. The output layer predicts TOC.

(5)

The TOC predictor comprises a prediction backbone and a residual branch. The final prediction is obtained by summing the outputs of the two components.

$$y_{\mathrm{main}}=f_{\mathrm{fc}}\left(\mathrm{Flatten}\left(z\right)\right)$$

(12)

$$f_n=\mathrm{Dropout}\left(\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left(\mathrm{Linear}\left(f_{n-1}\right)\right)\right)$$

(13)

$$f_0=z$$

(14)

$$y_{\mathrm{res}}=W_r\cdot x^{\prime }+b_r$$

(15)

$$\widehat{y}=y_{\mathrm{main}}+y_{\mathrm{res}}$$

(16)

where $y_{\mathrm{main}}$ denotes the main branch prediction; $y_{\mathrm{res}}$ denotes the prediction of residual branch; Flatten denotes flattening; $b_r$ denotes the bias parameter; $W_r$ denotes the weight; $\widehat{y}$ denotes the final predicted value.

(6)

Finally, R² and RMSE are used to evaluate the model; R² was shown in Equation (4) and RMSE was shown in Equation (6).

The CAE BPNN model is implemented in PyTorch and Scikit Learn.

All experiments were conducted on a standard workstation equipped with an Intel CPU, 32 GB RAM, and an NVIDIA GPU. The model was implemented using Python 3.12, PyTorch 2.4.0, and Scikit-learn 1.5.0. Because the dataset contains only 177 labeled TOC samples and 1388 unlabeled logging samples, the computational cost was moderate. A complete five-fold training and validation process required only several minutes, with peak GPU memory usage below approximately 2 GB, indicating that the proposed framework can be implemented without high-performance computing resources.

3.4. Building BP Neural Network Model

A BPNN without autoencoder derived features was implemented to evaluate the effectiveness of the CAE BPNN. The architecture matches the BPNN described in Section 3.3. As shown in Figure 7, features learned by the autoencoder are not used at input. Standardized features are provided directly to the model.

3.5. Building Convolutional Neural Network Model

A CNN is used for TOC prediction. CNN can perform well across domains due to local connectivity and feature amplification [31,52]. A deep regression model is designed with multi-layer one-dimensional convolution and fully connected layers. Input features are the optimized set selected in Section 3.2. As shown in Figure 8, the front end uses a three-layer 1D convolutional network. The numbers of kernels are 64, 128, and 128, and the kernel size is 3. Each layer is followed by BatchNorm, ReLU, MaxPool. After flattening, the output feeds a three layer fully connected network. The numbers of neurons are 128, 64, and 32, and the activation is Relu. The final layer is a linear output for TOC regression.

(1)

The input features are standardized as described in Equation (5).

(2)

Three layers of 1D convolution are applied:

$$H_m=\mathrm{Relu}\left(\mathrm{BatchNorm}\left(\mathrm{Conv}\left(X\right)\right)\right)$$

(17)

$$P_m=\mathrm{Dropout}\left(\mathrm{MaxPool}\left(H_m\right)\right)$$

(18)

where X denotes the standardized features; BatchNorm denotes the batch normalization; Dropout denotes the regularization; Relu denotes the activation function; MaxPool denotes the maximum pooling.

(3)

The output from the final convolution and pooling is flattened, followed by a three-layer fully connected network for prediction:

$$F_0=\mathrm{Relu}\left(\mathrm{Linear}\left(\mathrm{Flatten}\left(P_L\right)\right)\right)$$

(19)

$$F_n=\mathrm{Dropout}\left(\mathrm{Relu}\left(\mathrm{Linear}\left(F_{n-1}\right)\right)\right)$$

(20)

$$\widehat{y}=\mathrm{Linear}\left(f_L\right)$$

(21)

where Flatten denotes flattening; Linear denotes the linear processing; $\widehat{y}$ denotes the predicted value of TOC.

(4)

Finally, R² and RMSE are used to evaluate the model; R² was shown in Equation (4) and RMSE was shown in Equation (6).

The CNN model is implemented in PyTorch and Scikit Learn.

3.6. Building Machine Learning Model

In addition, other machine learning models were used for TOC prediction, including RF and GBDT.

3.6.1. Building Random Forest Model

RF is an ensemble machine learning model (Figure 9a). Its decision trees run in parallel, which yields fast computation [53,54]. It has proven effective across many domains. It tolerates outliers and noisy data [55]. Therefore, a multi-parameter RF regression model with tunable hyperparameters is designed for TOC prediction in nonlinear geological logging data. Model inputs are the selected logging curves from Section 3.2. The final core parameters are obtained via cross-validation and GridSearch and are listed in

Table 1. Parameters for the RF and GBDT models.

	n_estimators	max_depth	min_samples_split	min_samples_leaf	max_features	Subsample
RF	200	15	5	2	‘sqrt’	/
GBDT	300	5	10	4	/	0.8

. The model uses the Bagging ensemble to average predictions from multiple decision trees.

(1)

The input features are standardized as described in Equation (5).

(2)

The RF regressor consists of multiple decision trees. Each tree is trained on a subset drawn from the dataset by Bootstrap sampling. At each node split, a random subset of features is evaluated to find the best split, and the tree then produces a prediction. The final prediction is the average of all tree predictions:

$$\widehat{y}={\displaystyle \frac{1}{M}}\sum _{m=1}^M{f}_m\left(\mathit{x}\right)$$

(22)

$$\theta =\arg \underset{\theta \in \mathsf{\Theta }}{\min } {\mathrm{MSE}}_{\mathrm{CV}}\left(\theta \right)$$

(23)

where $f_m\left(\mathit{x}\right)$denotes the prediction of a single tree; M denotes the total number of trees; $\widehat{y}$ denotes the predicted value of TOC; $\mathsf{\Theta }$ denotes the parameter space; $\mathrm{CV}$ denotes cross-validation; $\theta$ denotes the hyperparameter combination; arg min denotes the operation of the parameter when the minimum value is taken.

(3)

Finally, R² and RMSE are used to evaluate the model; R² was shown in Equation (4) and RMSE was shown in Equation (6).

The RF model is implemented in Scikit Learn.

3.6.2. Building Gradient Boosting Decision Tree Model

GBDT (Figure 9b) is an ensemble learning model that builds multiple decision trees through serial iterations. At each iteration, the newly added tree learns the residuals between current predictions and observed values. By gradually correcting these residuals, overall predictive performance improves [56]. Each subsequent tree focuses on errors left by the existing ensemble. This serial training mechanism enables continuous improvement in prediction performance. Accordingly, GBDT is used for TOC prediction. Model inputs are the logging curves selected in Section 3.2. Core parameters are listed in Table 1. The model adopts a residual fitting strategy, training one tree per iteration to accumulate and refine predictions.

(1)

The input features are standardized as described in Equation (5).

(2)

The model is initialized.

$$F_0\left(x\right)=\overline{y}$$

(24)

where $F_0\left(x\right)$ denotes the initial predicted value; $\overline{y}$ denotes the mean value of the training set target.

(3)

Iterative training includes residual computation, new tree fitting, model updating, and final prediction.

$$r_i^{\left(m\right)}=y_i-F_{m-1}\left(x_i\right)$$

(25)

$$h_m\left(x\right)\approx r_i^{\left(m\right)}$$

(26)

$$F_m\left(x\right)=F_{m-1}\left(x\right)+\gamma h_m\left(x\right)$$

(27)

$$\widehat{y}=F_M\left(x\right)=F_0\left(x\right)+\sum _{m=1}^M\gamma h_m\left(x\right)$$

(28)

where $r_i^{\left(m\right)}$ denotes the residual; $y_i$ denotes the true value of TOC; $F_{m-1}\left(x_i\right)$ denotes the predicted value of the TOC; $h_m\left(x\right)$ denotes the output of the tree; $\gamma$ denotes the learning rate; $\widehat{y}$ denotes the final predicted value.

(4)

Finally, R² and RMSE are used to evaluate the model; R² was shown in Equation (4) and RMSE was shown in Equation (6).

The GBDT model is implemented in Scikit Learn.

4. Results

4.1. Geochemical Characteristics

Organic matter abundance is a key indicator of source rock quality (Figure 10a). TOC and hydrocarbon generation potential (PG) are commonly used metrics for assessing organic matter abundance [6]. Among 177 samples, TOC ranges from 0.28% to 1.16%, with a mean of 0.61%. TOC varies widely and is unevenly distributed. PG ranges from 0.70 to 9.71 mg/g, with a mean of 2.46 mg/g. TOC is positively correlated with PG. By TOC criteria for continental source rocks in saline lake basins, most P₁f shale cores are good source rocks.

Kerogen is an insoluble product of organic matter degradation and a principal precursor of oil and gas [57]. Shale kerogen types in P₁f are classified based on pyrolysis and TOC (Figure 10b). For 177 shale samples, T_max ranges from 414.5 to 451.11 °C, with a mean of 437.86 °C. HI ranges from 152.32 to 436.89 mg/g, with a mean of 281.50 mg/g. Type II₁ kerogen predominates and shows strong oil generation potential.

Vitrinite reflectance R_o (Figure 10c) and T_max are reliable indicators of thermal maturity [6]. R_o and T_max are used to evaluate the thermal maturity of P₁f shale (Figure 10c,d). T_max ranges from 414.5 to 451.11 °C, with a mean of 437.86 °C. R_o ranges from 0.71% to 0.89%, with a mean of 0.80%. Overall, Fengcheng Formation shale is in the mature stage.

4.2. Different Models’ Performance

Models for TOC prediction included GBDT, RF, CNN, CAE-BPNN, and BPNN. Predicted TOC values were evaluated. Performance results for each model are shown in Figure 11.

4.2.1. BP Neural Network Model Performance

In Section 3.4, a BPNN was constructed and shows the weakest TOC prediction performance. R² is 0.77 (Figure 11a) and RMSE is 0.085 (Figure 11f). BPNN is prone to local optima and overfitting with high-dimensional, complex data.

4.2.2. Random Forest Model Performance

In Section 3.6, an RF model was built. R² is 0.81 (Figure 11b) and RMSE is 0.082 (Figure 11f). RF performs slightly better than BPNN. With high-dimensional, small, or uneven data and outliers, RF can overfit or bias, limiting performance. Overall accuracy and generalization remain below other models.

4.2.3. Gradient Boosting Decision Tree Model Performance

Based on the GBDT in Section 3.6, final evaluation was conducted. R² is 0.83 (Figure 11c) and RMSE is 0.076 (Figure 11f), outperforming BPNN and RF, and slightly below CNN.

4.2.4. Convolutional Neural Network Model Performance

In Section 3.5, a CNN was built and performs well in TOC prediction. R² is 0.85 (Figure 11d) and RMSE is about 0.075 (Figure 11f), better than BPNN, RF, and GBDT. Compared with traditional machine learning and fully connected networks, CNN better captures complex structure, nonlinearity, and anomalies. However, performance remains constrained by small-sample regression.

4.2.5. Convolutional Autoencoder-BP Neural Network Model Performance

The CAE-BPNN in Section 3.3 delivers the best TOC prediction, highlighting the benefit of combining abundant unlabeled with limited labeled data. R² is 0.89 (Figure 11e) and RMSE is 0.061 (Figure 11f), outperforming GBDT, RF, CNN, and BPNN. A convolutional autoencoder reduces dimensionality and extracts deep features from high-dimensional logging data, effectively suppressing redundancy and noise. Compared with BPNN (Figure 11a,f), the unsupervised stage markedly improves modeling efficiency and prediction accuracy. Relative to the best single model, CNN, CAE-BPNN further strengthens feature representation and final prediction performance.

As shown in Figure 12, across the P₁f₁ to P₁f₃ interval in the composite log, CAE BPNN aligns most closely with measured TOC, with advantages concentrated in the main P₁f₂ section and at the P₁f₁ to P₁f₂ and P₁f₂ to P₁f₃ contacts. It preserves peak and trough fidelity in thin interbeds and noisy zones, avoids over-smoothing and lag, and exhibits smaller bias in low TOC intervals while capturing high TOC spikes better than CNN, RF, GBDT, and BPNN.

4.3. Validation of the Generalization Ability of the Model

Previous model evaluations were conducted mainly using Well MY1, which limited the assessment of model stability and generalization. To provide an independent validation, Well MY2 was selected as an external test well. Geologically, both Well MY1 and Well MY2 are located in the Fengcheng Formation shale of the Mahu Sag and share a similar saline lacustrine depositional background, providing comparable stratigraphic and lithological conditions for TOC prediction. However, as an independent well, Well MY2 exhibits differences in TOC distribution and vertical logging responses relative to Well MY1, reflecting lateral heterogeneity within the Fengcheng Formation. Therefore, Well MY2 is suitable for evaluating the transferability and preliminary generalization ability of the proposed CAE-BPNN model.

The same five models, including CAE-BPNN, CNN, GBDT, RF, and BPNN, were applied to Well MY2, and the evaluation metrics described in Section 3 were used for comparison. The CAE-BPNN model achieved the best performance, with an R² of 0.81 and an RMSE of 0.367 (Figure 13e,f). In comparison, CNN obtained an R² of 0.76 and an RMSE of 0.432 (Figure 13c,f), GBDT achieved an R² of 0.76 and an RMSE of 0.423 (Figure 13b,f), RF yielded an R² of 0.77 and an RMSE of 0.405 (Figure 13d,f), and BPNN produced an R² of 0.69 and an RMSE of 0.516 (Figure 13a,f). These results indicate that CAE-BPNN maintains relatively stable predictive performance in an independent well and outperforms the other models under similar geological conditions. Nevertheless, because the external validation is based on only one additional well, the generalization ability of the model should be regarded as preliminary and requires further verification using more wells and different geological intervals.

4.4. Movability Evaluation of Shale Oil

The oil saturation index (OSI) is a primary indicator of shale oil mobility. Previous studies indicate that an OSI greater than 100 mg/g marks a threshold for mobile oil and favorable development potential [47,58]. A high-precision TOC prediction model based on CAE-BPNN provides a robust basis for mobility evaluation. As shown in Figure 14, the OSI ranges from 14.29 to 517.18 mg/g, with a mean of 110.57 mg/g; more than half of the samples exceed 100 mg/g, indicating generally good mobility. Depth-wise analysis using predicted TOC shows that low TOC intervals coincide with high OSI values above 100 mg/g, implying better shale oil mobility under low organic matter abundance. This pattern is consistent with the low TOC and high mobility relationship documented for P₁f in the Mahu Sag. To verify this relationship, the hydrocarbon expulsion potential approach was used to quantify micro-transported hydrocarbons, denoted ΔQ [47]. As shown in Figure 14, ΔQ ranges from negative 403.2 to 981.86 mg/g, with a mean of 188.49 mg/g. Positive ΔQ indicates hydrocarbon expulsion, whereas negative ΔQ indicates external hydrocarbon charging. Spatial correlation with predicted TOC shows that expulsion intervals generally correspond to high TOC and low OSI; conversely, charging intervals are mostly associated with low TOC and high OSI. From a migration dynamics perspective, low TOC intervals act as dominant hydrocarbon acceptors, indicating good shale oil mobility.

4.5. Additional Robustness Checks and Methodological Limitations

Although the fold-averaged R² and RMSE values indicate that CAE-BPNN achieves the best overall performance among the tested models, the comparison should be interpreted cautiously under small-sample conditions. In this study, model performance was mainly evaluated using five-fold cross-validation and independent-well validation. Because fold-wise residual distributions were not further analyzed, the reported metric differences are interpreted as empirical performance improvements rather than formal evidence of statistical significance. Future studies based on larger multi-well datasets should incorporate repeated cross-validation or bootstrap resampling to estimate confidence intervals and conduct paired statistical tests between CAE-BPNN and baseline models [59].

The contribution of unlabeled data is another important methodological issue. The comparison between BPNN without CAE and CAE-BPNN demonstrates that representation learning from unlabeled logging data can improve TOC prediction. However, the effective minimum amount of unlabeled data is not expected to be a universal fixed percentage. Instead, it depends on geological representativeness, stratigraphic coverage, logging quality, and the distribution of logging responses within the target interval. For CAE pretraining, the unlabeled samples should sufficiently cover the major variations in lithology, mineral composition, porosity, compaction, and organic-matter-related logging responses. In this study, all available high-quality unlabeled samples from the target interval were used to maximize geological coverage and reduce sampling bias. Future work will further evaluate sample-efficiency curves using different proportions of unlabeled data in larger and more diverse datasets.

Model interpretability also remains a limitation of the present framework. Feature importance analysis, SHAP analysis, sensitivity analysis, and latent-space visualization can provide additional evidence for determining whether the CAE-learned features are relevant to TOC prediction rather than merely reconstructing input noise. SHAP has been widely used for model-agnostic feature attribution [49], while autoencoder-based representation learning is generally designed to capture compact latent structures from unlabeled data [41,44,60]. Integrating these quantitative interpretation tools with geological response mechanisms will help clarify the contribution of different logging curves and the geological significance of the learned latent features.

In addition, the representation capacity of the CAE is constrained by the information contained in the four selected logging curves. Increasing the number of unlabeled samples can improve the stability of the learned data distribution and vertical response patterns, but it cannot introduce geological information that is absent from the input variables. Therefore, further improvement of the model may require the incorporation of additional logging curves, core measurements, seismic attributes, mineralogical data, or geological constraints. Such multi-source information would enhance the physical basis, representation capacity, and geological interpretability of the proposed CAE-BPNN framework.

5. Discussion

This study addresses TOC prediction for continental shale oil in saline lacustrine basins as a small-sample regression problem. A CAE-BPNN framework was developed by coupling unsupervised pretraining with supervised fine-tuning, and its performance was compared with GBDT, RF, CNN, and a BPNN without CAE pretraining. The results show that CAE-BPNN achieved the best predictive performance among the tested models, with higher R² and lower RMSE in Well MY1 and better performance in the independent validation using Well MY2. However, because the independent validation was conducted using only one additional well, the generalization ability of the proposed model should be regarded as preliminary rather than broadly demonstrated.

Although the ΔlogR method and its improved versions proposed by Passey et al., Zhao et al., and Zhu et al. are widely used industrial approaches for TOC estimation, their performance commonly depends on baseline selection, maturity-related parameters, resistivity–porosity separation, and local petrophysical calibration [16,17,18,19]. These assumptions may introduce uncertainty in saline lacustrine shale reservoirs with strong lithological heterogeneity, complex mineral composition, and frequent thin interbeds. In contrast, the proposed CAE-BPNN framework does not rely on a single resistivity–porosity separation or manually defined baselines. Instead, it integrates multiple conventional logging curves and uses unlabeled logging data to learn nonlinear feature representations before supervised TOC calibration. Therefore, CAE-BPNN provides a practical alternative for small-sample prediction tasks in heterogeneous shale intervals, where measured TOC data are limited but continuous logging data are relatively abundant.

The improved performance of CAE-BPNN can be mainly attributed to its hybrid learning strategy. Conventional machine learning models, such as RF and GBDT, can capture nonlinear relationships but may be sensitive to feature combinations and hyperparameter settings under small-sample conditions. Standard deep learning models, such as CNN and BPNN, usually require sufficient labeled data for effective training and may suffer from overfitting when measured TOC samples are limited. In comparison, CAE-BPNN first uses unlabeled logging data to learn the intrinsic distribution and coupled responses of the selected logging curves, and then transfers the pretrained encoder to the supervised TOC prediction stage. This strategy reduces the dependence on labeled samples and provides more stable feature initialization for downstream regression.

The end-to-end structure is another advantage of the proposed framework. Unlike independent two-stage workflows, such as clustering followed by regression, the CAE-BPNN framework allows the learned representations to be further adjusted during supervised fine-tuning. As a result, the extracted features are not completely separated from the TOC prediction objective. This joint learning process improves the task relevance of feature representation and helps the model make more efficient use of limited supervision information. In small-sample regression tasks, this is particularly important because the scarcity of labeled data is the main factor limiting model robustness and prediction accuracy.

Compared with data augmentation, unsupervised pretraining provides a more direct way to exploit the abundant unlabeled logging data. Data augmentation generally generates variants of labeled samples according to predefined transformation rules, and its effectiveness depends on whether these transformations are geologically reasonable. In contrast, unsupervised learning can directly learn the potential structure of unlabeled logging data and reduce reliance on expensive and time-consuming laboratory measurements. In this study, CAE was selected instead of PCA, K-means clustering, or self-organizing maps because CAE can learn nonlinear representations through convolutional operations and can preserve local vertical response patterns in logging curves. The bottleneck structure of the autoencoder also encourages compact feature representation, which is more suitable for subsequent regression prediction than purely linear projection or discrete clustering.

The predicted TOC profiles also provide useful support for shale oil mobility evaluation. TOC is a key parameter for calculating the oil saturation index and for evaluating hydrocarbon micro-migration. Therefore, continuous TOC prediction based on logging data can provide important input for qualitative and quantitative shale oil mobility assessment. Local TOC anomalies that are inconsistent with the geological background may also provide auxiliary evidence for identifying favorable intervals. Thus, high-precision TOC prediction by CAE-BPNN has potential value for shale oil sweet spot evaluation.

Nevertheless, the proposed method still has several limitations. First, the independent validation was performed using only one additional well because multi-well datasets with complete logging curves and measured TOC samples are difficult to obtain under the constraints of project data availability and confidentiality. Therefore, the current validation provides preliminary evidence of transferability, and further testing using more wells and different geological intervals is needed. Second, the CAE in this study was trained using only four conventional logging curves, which inevitably limits the dimensionality and complexity of the latent representation. Therefore, the role of CAE should be interpreted as unsupervised pretraining, denoising, nonlinear feature combination, and stable initialization under small-sample conditions, rather than as extraction of highly complex high-dimensional features. Although dropout, weight decay, batch normalization, early stopping, and supervised fine-tuning were used to reduce the risk of simple input memorization, reconstruction-quality analysis and latent-feature visualization were not included in the present study. Future work will further evaluate reconstruction errors, visualize latent features, and combine feature-attribution methods to clarify the geological significance of CAE-learned representations.

The effective minimum amount of unlabeled data was not defined as a fixed percentage in this study, because it depends on geological representativeness, stratigraphic coverage, logging quality, and the distribution of logging responses. For CAE pretraining, the unlabeled samples should sufficiently cover the target shale interval and include the main variations in lithology, mineral composition, porosity, compaction, and organic-matter-related logging responses. In this study, all available high-quality unlabeled samples from the target interval were used to maximize geological coverage and reduce sampling bias. Future work will further evaluate sample-efficiency curves using different proportions of unlabeled data in larger multi-well datasets.

Variational autoencoders are another important class of generative representation-learning models. Recent studies have applied VAEs to geophysical inverse problems by constructing low-dimensional probabilistic priors, representing multimodal posterior solutions, and performing stochastic inversion [61,62,63]. In this study, a deterministic CAE was adopted because the task is a compact four-log regression problem and the labeled TOC data are limited. However, VAE- or conditional VAE-based frameworks may be valuable for future uncertainty quantification and probabilistic TOC prediction.

High-precision TOC prediction and shale oil mobility evaluation using CAE-BPNN indicate the potential of this framework for shale oil sweet spot assessment. Sweet spot evaluation commonly requires geochemical parameters, such as S1, S2, and Tmax, which are usually obtained from laboratory tests. Because these parameters are also related to logging responses, the proposed CAE-BPNN framework may be extended to predict other geological parameters with appropriate architectural adjustments. In the future, a multi-target prediction framework integrating TOC, S1, S2, porosity, and other key parameters could provide more consistent and comprehensive support for shale oil sweet spot evaluation.

6. Conclusions

In this paper, different regression prediction methods, including machine learning methods and deep learning methods, are used to compare the predicted TOC values with the measured TOC values. aThrough the comprehensive evaluation of various models, a representative model is selected. Finally, the CAE-BPNN model is determined, which adopts an end-to-end structure of unsupervised feature extraction-supervised fine-tuning. is found to be the optimal prediction method for TOC of shale oil in Fengcheng Formation of the Mahu Sag. Through systematic experimental verification and geological application analysis, the main conclusions are obtained as follows:

(1)

CAE-BPNN achieved the highest TOC prediction accuracy in P1f (R² is 0.89; RMSE is 0.061). It outperforms GBDT, RF, CNN, and BPNN by leveraging unlabeled logs via CAE to learn expressive features, delivering strong performance under small-sample conditions.

(2)

Accurate TOC prediction and verified generalization provide a practical basis for shale-oil mobility evaluation. TOC-driven OSI assessment corroborates the low-TOC and high-mobility pattern in the Mahu Sag, further quantified by micro-migration analysis, which supports identification of favorable intervals for exploration and development.

(3)

CAE-BPNN has good scalability and can be applied to the prediction of geological sweet spot evaluation parameters such as S1, S2 and Tmax. Future research will focus on further study the evaluation and prediction model of shale oil geological sweet spots, and make the developed model have certain geological interpretation.