Stratigraphic Correlation of Well Logs Using Geology-Informed Deep Learning Networks

Abstract

Stratigraphic correlation plays a crucial role in reservoir characterization. However, it is often time-consuming and heavily dependent on geological expertise. To address this issue, we propose a novel method called CMT-enhanced Hiformer, which integrates convolutional neural networks meet vision transformers (CMT) and hierarchical multi-scale representations using transformers (Hiformer). First, the architecture of CMT-enhanced Hiformer fuses the advantages of convolutional neural networks and transformers, effectively extracting complex features from well logs and capturing both local and global dependencies via a well-designed attention mechanism. Next, a geological constraint with regularization parameters is incorporated into the loss function. The new loss function promotes the accuracy of stratigraphic boundaries. The proposed method was validated using data from the Shuanghe oil field in central China. Specifically, the model achieved a maximum F1 score of 0.8857 and a precision of 0.8865 on the blind test dataset, demonstrating its robustness and high classification accuracy. Moreover, we conducted ablation studies and performed a detailed comparison with state-of-the-art deep learning models. The results demonstrate that the proposed method significantly improves the accuracy and efficiency of stratigraphic correlation.

1. Introduction

Stratigraphic correlation is a fundamental task in reservoir characterization, aimed at establishing an isochronous stratigraphic framework to provide data support for the spatial representation of reservoir bodies [1,2]. Well log data serve as one of the core datasets for conducting stratigraphic correlation. Essentially, it involves an in-depth analysis and comparison of the geological information embedded within logging curves, identifying stratigraphic units of various scales based on characteristic values and morphological variations of the curves. Traditional stratigraphic correlation based on well logs primarily relies on manual interpretation, with the accuracy of results heavily dependent on the expertise and experience of geoscientists. However, as the number of wells increases, stratigraphic correlation becomes increasingly labor-intensive and time-consuming [3].

To address these challenges, various semiautomated and automated stratigraphic correlation methods have been proposed, including signal decomposition techniques, statistical methods, and machine learning approaches. Signal decomposition methods focus on processing well log data by decomposing a single log curve into multiple modes, with each mode representing stratigraphic features at different scales. These methods include wavelet analysis [4,5], variational mode decomposition (VMD) [6], cluster analysis [7], principal component analysis (PCA) [8,9], and pattern recognition techniques [10]. By extracting multiple data components from the log curves, these approaches provide a more intuitive representation of multi-scale stratigraphic characteristics, thereby enhancing the accuracy of stratigraphic correlation. Statistical methods treat well log curves as data sequences and delineate stratigraphic units by analyzing the correlations between these sequences. Representative techniques in this category include dynamic time warping (DTW) [11,12,13], cross-correlation analysis [14], maximum likelihood estimation [15], and grey relational analysis [16].

In recent years, machine learning techniques have been increasingly applied to stratigraphic division and correlation tasks, achieving notable progress. Initially, methods like genetic algorithms were used for biostratigraphic correlation but were computationally intensive and time-consuming [17]. To improve interpretation efficiency, researchers have increasingly adopted machine learning algorithms, such as support vector machines, adaptive boosting, gradient boosting, and random forests, that leverage annotated datasets to achieve faster and more consistent stratigraphic analysis [18,19,20]. With the development of deep learning, more powerful tools emerged for automated geological analysis. Models like convolutional neural networks (CNNs) and recurrent neural networks have been increasingly adopted for stratigraphic correlation [21,22,23,24,25]. Furthermore, the strengths of CNNs, including parameter sharing, translation invariance, and direct learning from raw inputs, enable effective performance in classification, segmentation, and regression tasks [26,27,28,29,30,31]. However, CNNs face limitations in modeling long-range dependencies, which can require deep architectures and increase computational cost. To address these limitations, transformer-based architectures have been introduced, leveraging self-attention mechanisms to capture both local and global dependencies, making them ideal for sequential modeling tasks such as well log interpretation [32,33,34,35,36,37,38].

This paper addresses two key challenges in stratigraphic correlation of well logs: enhancing the quality of feature extraction through algorithm fusion and improving prediction accuracy through the design of geology-informed loss function. First, we propose a geology-informed deep learning network for stratigraphic correlation of well logs. The deep learning algorithm architecture, namely CMT-enhanced HiFormer, leverages the strengths of both CNNs and transformers [39,40]. To improve prediction accuracy, we incorporate geological constraints into the loss function. Next, we discuss the details of the implementation of the training process and conduct ablation studies, which evaluate the effectiveness of the different modules in our CMT-enhanced HiFormer and different regularization weights of the loss function. The well-trained model is then applied to perform stratigraphic correlation of well logs acquired from the Shuanghe oil field in central China. Representative well tops are analyzed to quantitatively assess the performance of the proposed method. Finally, we discuss the effect of the number and distribution of trained wells on prediction accuracy, where the broad applications of our model are also discussed. At the end, we provide brief conclusions of this study.

2. Methodology

2.1. The Architecture of CMT-Enhanced Hiformer

The simplified structure of our proposed CMT-enhanced Hiformer is presented in Figure 1. The proposed model architecture primarily consists of two components: a feature extraction module and a decoding module. The feature extraction module comprises one CMTSTEM layer, two CMTBLOCK layers, and two Pooling layers, while the decoding module incorporates two CONVUP modules designed to restore both the resolution and dimension of feature maps extracted from well log data. Detailed explanations about the key modules are provided as follows.

2.1.1. CMTSTEM Module

As shown in Figure 1a, the CMTSTEM module serves as the front layer of our suggested model in this study. The input well log data are passed through a 3 × 3 convolutional layer with stride 2 for initial feature extraction and downsampling. This reduces the spatial resolution of the feature map while preserving important local feature information. Such fine-grained features can overcome the inability concerning transformer structure. Subsequently, two consecutive 3 × 3 convolutional layers with a kernel size of $3\times 3$ and a stride of 1 further extract deep features from the downsampled feature map without altering its spatial dimensions. Each layer integrates GELU activation and batch normalization (BN), forming a hierarchical feature processing architecture. This design provides subsequent operations with feature representations that simultaneously maintain local receptive fields and achieve spatial compression.

2.1.2. CMTBLOCK Module

As depicted in Figure 1b,c, the CMTBLOCK is one of the key components of CMT [40], designed to effectively capture and represent complex features with an enhanced attention mechanism. The modified module starts with a 3 × 3 depth-wise convolution (DWConv) and includes a well-tailored attention mechanism from the Hiformer-MHSA (HMHSA). The typical multi-head self-attention refines the learning process by utilizing the extracted features. In addition to generating fine-grained feature maps $P^s$, it also offers the query $CL{S}^s$ for the HMHSA to further combine the low-level representation $P^l$. This wide range of features withstands the physical constraint of stratigraphical layers. The HMHSA module utilizes cross-attention to effectively guide feature map fusion. As illustrated in Figure 1f, “P” represents the segmented local data for capturing local stratigraphic features and “CLS” is a learnable token added at the “P” sequence start to aggregate global information for classification. The HMHSA takes the output feature map $P^s$ and tokens $CL{S}^s$ from the LMHSA to guide the fusion stage. Another input $P^l$ is the raw feature map simply after convolution. During the fusion stage, we utilize the weighted embedding representations $CL{S}^s$ to function as the query for fusion, where the value is the original feature maps generated by simple downsampling to capture global features.

Specifically, the precise segmented predictions emphasize the importance of boundary points, highlighting the challenges of relying solely on the transformer model. Conversely, the CNN-based model loses the global captured features. Hence, the Cross Attention module, as shown in Figure 1f, takes the resultant smallest $P^s$ and largest $P^l$ levels as inputs and employs a cross-attention mechanism to fuse information across scales, addressing the inconsistency and low precision problems. Finally, the IRFFN module reformulates the feature map into the required output size.

2.1.3. CONVUP Module

The CONVUP module is a decoder component designed for feature map upsampling. It processes input feature maps through a series of convolutional layers, GroupNorm, and ReLU activation functions, while incorporating bilinear interpolation for upscaling. For the CMTBLOCK1 and CMTBLOCK2 stages, the CONVUP module first educes channel dimensions and extracts features via convolution, then enlarges spatial dimensions through upsampling. Finally, these two modules work collaboratively to upsample the deep features and fuse them with shallow features, ultimately restoring high-resolution output and achieving the transformation from low-resolution high-level semantic features to high-resolution detailed features.

2.2. Loss Function with Geological Constraint

The first loss function used in this study is the commonly used cross-entropy (CE) loss, defined as

$$Los{s}_{CE}(y,p)=-\sum _{k=0}^{k=N_k}{y}_k·log\left(p_k\right)$$

(1)

where k denotes the stratigraphic class and $N_k$ represents the total number of stratigraphic classes. $y_k$ indicates the value of the ground true label for class k, i.e., if the sample belongs to class k, then $y_k$ is 1; otherwise, it is 0. $p_k$ represents the predicted probability for class k, which is the k-th element of the probability distribution output by the well-trained model.

Our suggested geological constraint (GC) is defined as

$$Los{s}_{GC}(\overline{y},\overline{p})=-\sum _{l=0}^{l=N_l}{\overline{y}}_l·log\left({\overline{p}}_l\right)$$

(2)

where l represents the index of the stratigraphic boundary, ranging from 0 to $N_l$, where the boundary positions are sequentially numbered from 1 to $N_l$ and non-boundary positions are 0. $N_l$ denotes the number of stratigraphic boundaries. ${\overline{y}}_l$ denotes the ground true value for the stratigraphic boundary of class l and ${\overline{p}}_l$ represents the probability predicted by the model for boundary l.

The position of the stratigraphic boundary is the key point of the loss function we proposed in this study. Focusing on optimizing the prediction of different stratigraphic boundary positions reduces confusion between different horizons in the model. Then, our loss function in this study can be defined as

$$Los{s}_{total}={\lambda }_{1}Los{s}_{CE}+{\lambda }_{2}Los{s}_{GC}$$

(3)

where ${\lambda }_1$ and ${\lambda }_2$ are two regularization weights. The selection of ${\lambda }_1$ and ${\lambda }_2$ will be explained in Section 4.

3. Data Introduction and Implementation Details

3.1. Study Area and Training Data Preparation

The Nanxiang Basin is bordered to the north by the Qinling Mountains and to the south by the Daba Mountains. It is a Meso-Cenozoic continental intermountain rift basin that formed during the late Yanshanian period. The sedimentary basin is controlled by faults along its periphery, covering a total area of 17,000 km² [41]. The Nanxiang Basin includes four depressions and four uplifts. Among them, the Biyang Depression in the southeast is the main area for oil and gas exploration. Influenced by the late Yanshanian stretching, it forms three structural belts from north to south: the northern slope belt, the central deep depression belt, and the southern steep slope belt presenting a general characteristic of being higher in the north and lower in the south [42], presented in Figure 2.

More than ten oil fields have been discovered in the Biyang Depression. This study focuses on the northern block of the Shuanghe Oil Field, which features a nose-like structure with a northwest-southeast trend, tilting towards the southeast, and a steeper western wing than the eastern wing. The Shuanghe Oil Field develops the Pingyuan Formation, Shangsi Formation, Hetaoyuan Formation, Dacangfang Formation, and Yuhuangding Formation from top to bottom [43]. Among these, the third member of the Hetaoyuan Formation is the main oil-bearing layer. This research target is the second sandstone group of the third member of the Hetaoyuan Formation, with a burial depth of 1350–2400 m. This layer consists primarily of gravel rock, gravelly sandstone, conglomerate sandstone, and sandstone, forming a mixed sandstone-conglomerate complex deposited in the fan delta front. The sand bodies are distributed in a fan shape from southeast to northwest, with wide distribution and significant sediment thickness. The sand bodies gradually thin out and pinch out from southeast to northwest [44].

The well location map of the study area is presented in Figure 3. The gray points in this image indicate well boreholes, where 100 wells are used for building the well dataset in this study, named W1, W2, …, and W100. When utilizing DL models for stratigraphic correlation, we can randomly select wells for model training, validation, and blind testing. Note that we select three well logs in this study, i.e., Gamma Ray (GR), Spontaneous Potential (SP), and Resistivity (RT), and we do not pay much attention to the sensitivity analysis of well logs. Note that, to test the effectiveness of our model, we randomly select 30% of the dataset in Figure 3 as our training data (indicated by the red points) and 5 wells as our validation data, while the rest are used as our blind testing data.

3.2. Implementation Details of Training Process

All networks use the PyTorch 2.2.1 deep learning library, Python 3.8, trained on a workstation with two NVIDIA GeForce RTX 3090 GPUs, equipped with a dual-channel Intel 6354 CPU. The SGD optimizer is selected with a learning rate scheduler. The learning rate is set with an initialization of 0.001. We train the base models for a maximum of 20 epochs, saving the model with the best validation loss during model training.

To evaluate the performance of difference models, confusion matrices defined in Equation (4) are utilized in this study.

$$\begin{array}{cc}\hfill & Precision={\displaystyle \frac{TP}{TP+FP}},\hfill \\ \hfill & Recall={\displaystyle \frac{TP}{TP+FN}},\hfill \\ \hfill & F1=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}\hfill \end{array}$$

(4)

where True Positives (TP) are instances where positive samples are correctly identified as positive, reflecting the model’s ability to detect the target condition, True Negatives (TN) are cases where negative samples are accurately classified as negative, demonstrating the model’s effectiveness in recognizing the absence of the target condition, False Positives (FP) occur when the model incorrectly classifies negative samples as positive, leading to false detections, and False Negatives (FN) arise when positive samples are mistakenly classified as negative, resulting in missed detections. These metrics offer a thorough assessment of the model’s classification performance, providing insights into its precision and recall across various classes.

4. Ablation Study

4.1. The Comparisons of Different Modules

In this section, we first implement an ablation study on different models within our proposed CMT-enhanced Hiformer, i.e., comparing the effectiveness of CMT, Hiformer, and CMT-enhanced Hiformer. As mentioned above, we randomly select 30 wells as our training data and 5 wells as our validation data, while the others are selected as the blind test dataset. The training loss curves of CMT, Hiformer, and our suggested CMT-enhanced Hiformer are presented in Figure 4. Note that we normalize all these three training losses to make a clear comparison. We obtain three well-trained and convergent models after model training, which can be easily used for model testing. Moreover, the final loss values of our suggested model are smaller than those of the other two models, indicating that our model shows a better convergence result.

Afterward, we apply these three models to the blind test dataset. The precision and F1 scores are, respectively, calculated and indicated in

Table 1. The precision and F1 of the blind test dataset by using CMT, Hiformer, and CMT-enhanced Hiformer.

Methods	Precision	F1
CMT	0.8365	0.8357
Hiformer	0.8277	0.8271
CMT-enhanced Hiformer	0.8717	0.8708

. Comparing these quantitative values, it can be easily found that our model shows better performance than the other two models by obtaining higher precision and F1. Moreover, we randomly select a well from the blind test dataset, i.e., W29, and then apply these three models for stratigraphic division. The results are presented in Figure 5c–e, while Figure 5a shows three well logs, i.e., GR, SP, and RT, and Figure 5b shows the ground truth stratigraphic classification labels. The red dashed lines indicate the sequence boundaries of W29. By comparing these images, we have several main observations. First, CMT shows an inaccurate stratigraphic division result even in the middle of a certain sequence, indicated by the red arrow. Second, there are incorrect predicted results at the sequence boundaries, denoted by the blue arrows in Figure 5c,d. Our model provides a more accurate result in Figure 5e by comparing it with the other two models, demonstrating the effectiveness of our fused model.

4.2. The Comparisons of Different Regularization Weights

We validate the effectiveness of our proposed geological constraint (GC) in this section. Moreover, we discuss in detail the selection of two regularization weights ${\lambda }_1$ and ${\lambda }_2$ assigned to the cross-entropy (CE) loss and our GC loss in Equation (3), while satisfying the constraint ${\lambda }_1+{\lambda }_2$ = 1. Note that, when selecting ${\lambda }_1$ = 1 and ${\lambda }_2$ = 0, it indicates that we only use the cross-entropy loss defined in Equation (1) for model training and our GC loss is not used for model training.

As the weight of our GC loss increases in

Table 2. The ablation study of our suggested geological constraint and two regularization parameters by applying our CMT-enhanced Hiformer to the blind test dataset.

Weight	Precision	F1
${\lambda }_1$ = 1,        ${\lambda }_2$ = 0	0.8717	0.8708
${\lambda }_1$ = 0.95,        ${\lambda }_2$ = 0.05	0.8433	0.8414
${\lambda }_1$ = 0.90,        ${\lambda }_2$ = 0.10	0.8571	0.8567
${\lambda }_1$ = 0.85,        ${\lambda }_2$ = 0.15	0.8735	0.8731
${\lambda }_1$ = 0.80,        ${\lambda }_2$ = 0.20	0.8865	0.8857
${\lambda }_1$ = 0.75,        ${\lambda }_2$ = 0.25	0.8759	0.8749
${\lambda }_1$ = 0.70,        ${\lambda }_2$ = 0.30	0.8489	0.8480

, i.e., ${\lambda }_2$ increases from 0 to 0.3, the model’s performance generally improves on the test set, despite some fluctuations with certain weight allocations. Specifically, when ${\lambda }_1$ = 0.8 and ${\lambda }_2$ = 0.2, the highest accuracy (0.8865) and F1 score (0.8857) on the blind test dataset are achieved. The quantitative results are worse than those of ${\lambda }_1$ = 0.8 and ${\lambda }_2$ = 0.2. These results indicate that the introduction of our GC loss enhances the robustness and generalization ability of our suggested model, particularly in terms of improved classification accuracy and consistency, demonstrating that our proposed GC loss shows a positive impact on the overall results by effectively stabilizing gradients during the learning process.

Afterward, we first utilize ${\lambda }_1$ = 1.0 and ${\lambda }_2$ = 0 for training our CMT-enhanced Hiformer and then ${\lambda }_1$ = 0.8 and ${\lambda }_2$ = 0.2, i.e., without and with our geological constraint. After model convergence, we randomly select a well from the blind test dataset, i.e., W25, whose well logs are denoted in Figure 6a. The ground truth stratigraphic classification label is shown in Figure 6b, while the predicted results via our suggested CMT-enhanced Hiformer without and with our GC loss are indicated in Figure 6c,d. The red dashed lines indicate the sequence boundaries of W25. Our CMT-enhanced Hiformer can realize stratigraphic division based on well logs. However, the introduction of our geological constraint effectively improves the accuracy of stratigraphic division, especially at the sequence boundaries indicated by the red arrows. These images in Figure 6 verify the validity of our suggested GC loss in this study.

5. Stratigraphic Correlation Results

We calculate the average differences between the ground truth and predicted well tops (in meters) of the blind test dataset by using SegNet, CMT, Hiformer, CMT-enhanced Hiformer, and CMT-enhanced Hiformer with GC loss, as indicated in

Table 3. The average differences between the ground truth and predicted well tops (in meters) of the blind test dataset by using SegNet, CMT, Hiformer, CMT-enhanced Hiformer, and CMT-enhanced Hiformer with GC loss.

Methods	Average Differences (m)
SegNet	2.91
CMT	2.23
Hiformer	2.54
CMT-enhanced Hiformer	1.85
CMT-enhanced Hiformer with GC	1.39

. Our suggested model obtains the smallest differences among these four models. Moreover, the proposed geological constraint promotes our model with 0.46 m, verifying its effectiveness. In addition, we select two representative well tops, i.e., Class 1 and Class 3, to calculate the statistical errors of the predicted results by using different methods, as denoted in Figure 7. By comparing these images, it can still be seen that our proposed model and geological constraint can effectively promote the performance of automatic stratigraphic correlation.

Next, we calculate the structural deep maps of Class 1’s top surfaces by using the predicted and ground truth well tops, as presented in Figure 8. These images are, respectively, calculated by using the results of (a) SegNet, (b) CMT, (c) Hiformer, and our suggested CMT-enhanced Hiformer (d) without and (e) with our GC loss, while Figure 8f is calculated by using the ground truth labels. By comparing these images, we can easily find that our result in Figure 8e is the closest one to the result in Figure 8f, further demonstrating the effectiveness of our suggested method in this study.

After predicting the stratigraphic division results, we show a well section parallel to the sediment source direction and a well section perpendicular to the sediment source direction, as denoted in Figure 9 and Figure 10. The dashed blue lines denote the stratigraphic correlation results of our CMT-enhanced Hiformer without GC loss, while the solid red lines indicate those of our CMT-enhanced Hiformer with GC loss. By comparing these stratigraphic correlation results, we can find that both methods can perform automatic stratigraphic correlation of well logs with high accuracy, verifying the effectiveness of our suggested CMT-enhanced Hiformer. Moreover, after introducing our GC loss, we can interpret the stratigraphic boundaries more accurately, as indicated by the red arrows in Figure 9 and Figure 10, further demonstrating the availability of our GC loss.

6. Discussion

We suggest a CMT-enhanced Hiformer with the geological constraint for stratigraphic correlation of well logs. On the one hand, based on Hiformer, we propose to replace its encoder by using the feature extraction module in CMT. This promotes the capability for extracting the local features of well logs and then different scale stratigraphic unit analysis is realized. On the other hand, the introduction of the geological constraint improves the accuracy of the formation interfaces, especially the formation boundaries. The applications and tests on the dataset at the Shuanghe Oil Field demonstrate the effectiveness of our model. However, the predicted accuracy and the generalization property depend on the number of training wells and their distribution.

The percentage of the training wells is the most important factor that affects the prediction accuracy. To further test the robustness of our suggested model, we randomly select different amounts of wells for model training, i.e., from 10% to 60%. The validation dataset is also built by randomly selecting 5 wells, while the other wells are used for building the blind test dataset. After model training, we apply different models to the blind test dataset and calculate the precision and F1, as shown in Figure 11. By comparing these values, we have several observations. First, no matter how many training wells we used, our suggested CMT-enhanced Hiformer with GC shows a superior classification effect compared to CMT-enhanced Hiformer without GC. This indicates that our proposed geological constraint promotes the classification effectiveness of our CMT-enhanced Hiformer. Second, with the reduction of training wells used, i.e., from 30% to 10%, the performances of all models degrade significantly, indicating the great influence of the size of the training dataset for successfully implementing DL models. Moreover, the increase in model accuracy is limited when the training percentage increases from 30% to 60% (3.53%), not so obvious as those from 10% to 30% (7.92%). Furthermore, more training wells lead to more training time. To balance the training efficiency and the prediction accuracy, we select 30% wells as our training dataset and implement the detailed comparisons in this study.

For the latter, the accuracy of the model also depends on the distribution of training wells. In practical applications, different positions on the reservoir plane are affected by the source control, the deposition characteristics are different, and the logging response characteristics are also different. To improve the accuracy and generalization ability of the model, it is necessary to select wells in different reservoir locations as training datasets.

Finally, we focus on the automatic stratigraphic correlation of well logs in this study. Nevertheless, the suggested model is actually a classified or segmented model that can also be used for addressing other seismic interpretation tasks, such as fault detection and horizon picking. Certainly, we should build specialized losses for promoting the performance of DL models when solving different issues.

7. Conclusions

In this paper, we propose a geology-informed deep learning network called CMT-enhanced Hiformer for stratigraphic correlation of well logs. The network architecture integrates the advantages of convolutional neural networks and transformers, along with a mixed loss function composed of cross-entropy and a geological constraint. The proposed method was validated using data from the Shuanghe oil field in central China, achieving a maximum F1 score of 0.8857 and a precision of 0.8865 on the blind test dataset. Experiments show that CMT-enhanced Hiformer significantly improves the accuracy and efficiency of stratigraphic correlation and outperforms the state-of-the-art counterparts.