DS Net: A Dual-Coded Segmentation Network Leveraging Large Model Prior Knowledge for Intelligent Landslide Extraction

Abstract

Landslides are characterized by their suddenness and destructive power, making rapid and accurate identification crucial for emergency rescue and disaster assessment in affected areas. To address the challenges of limited landslide samples and data complexity, a landslide identification sample library was constructed using high-resolution remote sensing imagery combined with field validation. An innovative Dual-Coded Segmentation Network (DS Net), which realizes dynamic alignment and deep fusion of local details and global context, image features and domain knowledge through the multi-attention mechanism of Prior Knowledge Integration (PKI) module and Cross-Feature Aggregation (CFA) module, significantly improves the landslide detection accuracy and reliability. To objectively evaluate the performance of the DS Net model, four efficient semantic segmentation models—SegFormer, SegNeXt, FeedFormer, and U-MixFormer—were selected for comparison. The results demonstrate that DS Net achieves superior performance (overall accuracy = 0.926, precision = 0.884, recall = 0.879, and F1-score = 0.882), with metrics that are 3.5–7.1% higher than the other models. These findings confirm that DS Net effectively improves the accuracy and efficiency of landslide identification, providing a critical scientific basis for landslide prevention and mitigation.

1. Introduction

China is a country prone to frequent geological disasters [1,2]. According to data from the National Bureau of Statistics, hundreds of thousands of geological disasters occurred in China between 2007 and 2022, with landslides ranking as the most common type. Landslides are characterized by their wide distribution, high frequency, and complex causes and mechanisms. Frequent landslides can lead to a variety of disasters and accidents, including traffic disruptions, river blockages, farmland damage, building destruction, village burial, and human and animal casualties. These events have become one of the most severe geological threats, posing significant risks to the safety of local residents’ lives and property, the sustainable development of regional economic and social health, and the safety of major engineering construction and operations.

Therefore, it is urgent to conduct research on the distribution and activity of landslide disasters, as high-precision identification and extraction of landslides form the foundation for studying landslide disasters [3,4,5,6]. Such efforts can provide critical references for landslide disaster prevention and mitigation research. However, landslides are often triggered by external environmental or geological tectonic factors, leading to fragmented terrain and inaccessible sites [7]. These conditions make it challenging for traditional landslide identification methods to accurately determine the specific location, scale, and quantity of landslide disasters [8]. Moreover, traditional methods lack standardized criteria for landslide identification, require extensive manpower and resources, and suffer from inefficiency. With the significant advancements in remote sensing platforms and sensor technology, high-resolution remote sensing imagery has been widely applied in geological disaster identification and investigation [9,10]. Consequently, using optical remote sensing imagery for high-precision landslide identification has become a key focus and a prominent research area in landslide disaster studies.

Deep learning has been developing rapidly in the field of image recognition and has attracted significant attention from researchers [11]. Compared to traditional machine learning, deep learning can achieve strong feature extraction and representation capabilities by organically combining single-layer feature extraction modules [12]. Compared to natural images, remote sensing images have more complex semantic and background information, making the application of Convolutional Neural Networks (CNN) to remote sensing images a hot topic [13]. In recent years, the research of computer vision (CV) technology in the field of landslide recognition has made important breakthroughs, which significantly improves the accuracy and efficiency of geologic disaster monitoring. At the data level, the fusion of multi-source remote sensing data have become the mainstream direction, Chen et al. [14] realized the feature fusion of DEM and remote sensing image data by utilizing the feature fusion of two-branch network data in Sichuan–Tibet area, which enhanced the robustness of intelligent landslide detection modeling. At the algorithmic level, the introduction of novel deep learning frameworks such as Transformer [15] and meta-learning [16] effectively solves the problem of model accuracy under small sample conditions. However, DNNSs still have many shortcomings in meeting the demands of remote sensing applications. On the one hand, the performance of DNNs is usually affected by the time-consuming and laborious labeling process and single-task limitations (e.g., each visual recognition task usually requires a separate DNN to be trained); on the other hand, the labeled data of specific sensor images are very scarce [17,18].

The Al large model, also known as “foundation model”, refers to a large-scale pre-trained model based on deep neural network architecture. This type of model mainly adopts Transformer and other advanced architectures, and is trained through massive data, with the following significant features: 1. Powerful transfer learning ability, which can be adapted to a variety of application scenarios; 2. excellent generalization performance; and 3. wide application potential. As the core technology in the field of artificial intelligence, the base model has become an important support for computer vision, natural language processing and other applications [19,20]. The rapid advancement of task-specific foundation models has revolutionized image segmentation. Notably, the Segment Anything Model (SAM) [21] established the first promptable segmentation framework with zero-shot generalization, though its class-agnostic design limits domain-specific precision. the Mask2Former [22] unified various segmentation tasks through masked attention transformers, demonstrating particular strength in boundary-sensitive applications. Domain-adapted variants like MedSAM [23] (for medical images) and RSRefSeg [23] (for remote sensing) proved the value of injecting domain knowledge into foundation models. Currently, intelligent recognition research primarily learns from training samples to extract and internalize the a priori knowledge in the data. However, the efficiency and effectiveness of this process are highly dependent on the number and quality of training samples [24,25]. When dealing with complex and data-scarce problems such as landslide prediction, the a priori knowledge that the model can extract is inevitably limited by the number of available samples. Due to the episodic and unpredictable nature of landslide events, collecting relevant samples is often extremely challenging and results in a relatively limited dataset. Therefore, even if the model architecture is advanced and the algorithm is optimized, without sufficient diversity and quantity of training samples, the model’s ability to learn landslide feature patterns and recognition rules will remain partial and insufficient. Thus, maximizing the extraction and effective utilization of a priori knowledge is the top priority of this study. The main content of this study includes the following points:

(1)

A library of landslide identification samples was created using high-resolution remote sensing imagery and landslide boundary data obtained through field validation.

(2)

The large model has billions or even hundreds of billions of parameters. It is trained by inputting large amounts of data, enabling the computer to acquire human-like “thinking” abilities and perform a variety of complex tasks, including image generation. In this paper, we design DSNet, which realizes the dynamic alignment and deep fusion of local details with global context, image features with domain knowledge, and significantly improves the accuracy and reliability of landslide detection through the multi-attention mechanism of Prior Knowledge Integration (PKI) module and Cross-Feature Aggregation (CFA) module.

(3)

Test the model’s prediction results on the landslide sample set under different data augmentation modes to determine the optimal data augmentation strategy for landslide identification within the study area.

2. Materials and Methods

2.1. Study Area

Beichuan Qiang Autonomous County is located in the transition zone from the northwestern edge of the Sichuan Basin to the western Sichuan Plateau (Figure 1). It falls under the jurisdiction of Mianyang City and has a land area of 3082.72 km² [26]. The county borders Jiangyou City, Anzhou District, Mao County, Songpan County, and Pingwu County, and is situated in the Longmen Mountain region, a transitional area between the Sichuan Basin and the Qinghai–Tibet Plateau. Tectonically, it is located at the junction of the front and back ranges of the Longmen Mountains [27]. The entire area is mountainous, characterized by long gullies, steep slopes, and a complex geological environment. The Yingxiu-Beichuan fault, which triggered the Wenchuan earthquake, lies in the southeastern part of the study area. Along the fault zone, Silurian phyllite and sandstone, Carboniferous carbonate rocks, and loose Quaternary deposits are widely exposed on both sides of rivers and valleys. These highly weathered rocks provide abundant material sources for geological disasters. Moreover, the area is part of the well-known Lutou Mountain heavy rainfall region, which experiences abundant precipitation. The geographical location, topography, geomorphology, and geological and tectonic conditions of the study area make landslides highly frequent in this region.

2.2. Production of a Landslide Identification Database

The model training process often requires a large amount of labeled sample data. From the schematic diagram of the spatial distribution of landslides in Beichuan County (Figure 2), it can be observed that earthquake-induced landslides in the Beichuan area are primarily distributed in the southeastern and parts of the central regions of the county. Their distribution is significantly influenced by tectonic lines and rivers, while landslide development in the northern and western regions of the county is relatively weaker.

Using high-resolution Google Earth images with a spatial resolution of 0.5 m, combined with the spectral characteristics of landslides in the study area, we applied visual interpretation to construct landslide identification sample data. The landslide contours were used as boundaries to outline landslide surface vector data and generate label data. The process of sample library production included vector editing, field assignment, vector-to-raster conversion, label generation, data segmentation, and sample dataset division. The detailed process of sample library production is shown in Figure 3.

(1)

Based on the existing landslide interpretation signs, the vector editing tool in ArcGIS 10.8 was used to edit the remote sensing images in TIFF format (Figure 2), and the boundaries were manually outlined through visual interpretation. Since the accuracy of boundary outlining directly affects the quality of the landslide sample library, to improve the precision of landslide samples and the accuracy of model training, it is necessary to strictly control vector outlining errors during the process. The error range of the outlined vector boundaries should be kept within 1–2 pixels.

(2)

ArcGIS software was used to assign attributes to the landslide interpretation data by adding a “label” attribute field to the outlined landslide boundary data. The landslide attribute field was assigned a value of “1”, while the non-landslide background was assigned a value of “0”.

(3)

Python 3.6, combined with the GDAL library, was used to convert vector files into raster files based on the assigned attribute field values. The relevant information was read from the raster files to create binary maps, ultimately generating the label data required for constructing the vegetation sample library. Additionally, it was ensured that the generated raster data matched the number of rows and columns in the image data.

(4)

Due to limited computer memory, the entire remote sensing image contains too much information to be used directly as input for the network model. Therefore, the remote sensing image and its corresponding landslide labeling data must be divided into several smaller images to ensure compatibility with the model for training. To simplify the processing of remote sensing images and enable the network model to better extract detailed landslide features while ensuring effective convergence, the dataset was divided into images of 256 × 256 pixels.

(5)

After processing through the sample segmentation operation, a total of 4879 high-resolution images were obtained, forming the remote sensing landslide sample library. To ensure sample diversity and make the dataset as varied as possible, the data were randomly divided into training, validation, and test sets in a ratio of 6:2:2. The training set, consisting of 2927 images, was used for model training to extract features. The validation set, comprising 976 images, was used to evaluate the model during training, while the test set, also consisting of 976 images, was used to assess the model’s performance.

2.3. Methods

2.3.1. Dual-Coded Segmentation Network (DSNet) Architecture

In this study, we propose a Dual-coded Segmentation Network (DSNet) that integrates prior knowledge features through a dual-encoder single-decoder architecture. The network consists of three key components: 1. Swin Transformer Encoder: It extracts multi-scale and hierarchical features through shifted window-based self-attention, capturing both local and global image characteristics. 2. CLIP-based Encoder: It leverages the pretrained CLIP large model [28] to enhance feature representation, combining its strong visual-semantic understanding with domain-specific adaptation for landslide identification. 3. Hierarchical Decoder: It utilizes progressive upsampling with skip connections to recover spatial details while maintaining feature consistency across scales. The workflow of the model is illustrated in Figure 4.

First, the core of CLIP lies in its dual-tower architecture (image encoder and text encoder) with stacked Transformers, which aligns image-text features through contrastive learning during training. In this study, we exclusively employ the image encoder architecture of CLIP. In Figure 4, the green cube in CLIP’s output represents the high-level features extracted through multiple Transformer layers, while the blue cube denotes the corresponding global features. The local information captures fine-grained details such as cracks, soil texture, or small deformations, while the global information provides broader contextual understanding, such as terrain shape and overall landslide morphology. Simultaneously, the Swin model, with its hierarchical structure and ability to model long-range dependencies, is employed to capture rich feature information from the images, ensuring comprehensive representation of complex landslide scenes.

Next, the PKI (Prior Knowledge Integration) module is introduced to effectively integrate the local and global information extracted by CLIP. The core equation of PKI is shown in Equation (1). By incorporating domain-specific knowledge about landslides, such as common deformation patterns or geological characteristics, the PKI module generates a priori knowledge tailored to landslide analysis. This step ensures that the model focuses on the most relevant features for landslide identification.

$$MA(x_G,x_L,x_L)=Softmax({\displaystyle \frac{L(E(x_G))\odot L{(x_L)}^T}{\sqrt{d_k}}})\otimes L(x_L)$$

(1)

Here, MA is Multi-Head Attention; X_G and X_L represent the global and local features output by CLIP, respectively; E() represents the dimension expansion of the matrix; L() is the fully connected layer; and $\sqrt{d_k}$ is a scaling factor to prevent the gradient from disappearing due to large dot product values.

Subsequently, in the CFA (Cross-Feature Aggregation) module (Equations (2) and (3)), the image features extracted by the Swin model are deeply fused with the a priori knowledge obtained from the PKI module. This fusion process enriches the feature representation with landslide-specific information, enhancing the model’s ability to distinguish landslides from other geological phenomena. The result is a robust and semantically meaningful feature set that improves the accuracy and reliability of landslide image analysis.

$$CFA(x)=SimAM(Softmax({\displaystyle \frac{L(x)\odot L{(x)}^T)}{\sqrt{d_k}}})\otimes L(x))$$

(2)

$$SimAM(x)=Sigmoid({\displaystyle \frac{1}{E}})\odot x$$

(3)

Here, L() is the fully connected layer; $\sqrt{d_k}$ is a scaling factor to prevent the gradient from disappearing due to large dot product values; and E is an energy function associated with the input features, usually computed as a significance measure for each position or channel in the feature map.

In the process of information integration and fusion, we skillfully utilize the multi-head attention mechanism [29] to enhance the interaction between different types of information. The core of this mechanism lies in its ability to dynamically model relationships between global and local features through scaled dot-product attention, as shown in Equation (4):

$$\mathit{Attention}(q,k,v)=\mathit{softmax}({\displaystyle \frac{q{k}^T}{\sqrt{d_k}}})v,$$

(4)

where q (Query) is derived from global information, while k (Key) and v (Value) are projected from local information.

This design allows the model to focus on the most relevant local details guided by the broader context, thereby improving the extraction of landslide prior information. By leveraging the attention mechanism, the model can dynamically weigh the importance of local features within the global context, ensuring a more accurate and comprehensive representation of landslide characteristics.

In the fusion stage of image features and prior knowledge, the roles are reversed: landslide prior information is used as the query (q), and image feature information serves as the key (k) and value (v). This reversal enables the model to align the extracted image features with the domain-specific prior knowledge, further strengthening the representation of landslide-related information within the image features. The multi-head attention mechanism facilitates fine-grained interactions between the prior knowledge and image features, allowing the model to highlight regions or patterns that are most indicative of landslides. This two-stage, attention-driven approach ensures that the final feature representation is both contextually aware and semantically enriched, significantly improving the model’s performance in landslide detection and analysis.

In addition, we have innovatively incorporated the SimAM (Simple Attention Module) [30] into the CFA module. SimAM is an attention model based on twin network architecture, which processes paired input data through a two-branch structure with shared weights to efficiently capture long-distance dependencies and cross-sample interactions. To further enhance spatial-channel feature selection, we extend SimAM with a 3D Weight mechanism. The core of SimAM lies in the use of cross-attention to compute the correlation between two input features, where one branch serves as a Query and the other branch provides the Key and Value, generates the attention weights and fuses the features through a similarity matrix. This mechanism assigns unique weights to neurons, enabling the model to focus more precisely on information that is critical to the task at hand while effectively ignoring irrelevant or less important parts. Finally, through the decoder’s step-by-step decoding process, the model successfully generates an accurate landslide mask.

2.3.2. Swin Transformer

Historically, convolutional neural networks (CNNs) have dominated the field of computer vision. Since AlexNet made a splash in the 2012 ImageNet competition [31], CNNs have been the model of choice for a variety of vision-based tasks. Given the success of Transformers in natural language processing, researchers have begun to explore their application in image processing. Compared to traditional convolutional neural networks, Transformer-based models leverage their multi-head attention mechanism to establish relationships between different pixels, thereby better extracting feature information in a global context. Vision Transformer (ViT), as a representative application of Transformers in image processing, has achieved outstanding results, surpassing traditional CNNs in the field of image recognition [32,33,34]. This demonstrates that Transformers can, to some extent, replace CNNs in image processing and hold great promise for future applications. However, the pixel resolution of images is significantly more complex than the text structure in a paragraph, and traditional Transformer models calculate attention globally, applying attention computations across the entire image. As a result, computational complexity becomes extremely high when processing high-resolution images.

To address the aforementioned issues, the Swin Transformer (Shifted Window Transformer) [35] modifies the original Multi-Head Self-Attention (MSA) structure of the Transformer and introduces a Window-based MSA (W-MSA) structure. This approach divides an image into multiple non-overlapping windows and computes attention within each window independently, significantly reducing computational complexity compared to the whole-image attention computation in ViT. Meanwhile, to overcome the limitation of feature transfer between different windows, the Swin Transformer further proposes a Shifted Window-based MSA (SW-MSA) structure. This method facilitates the transfer of feature information between windows by shifting their positions, thereby ensuring the network’s recognition accuracy. Experimental results have demonstrated that the Swin Transformer achieves top performance across various image classification tasks, showcasing its outstanding capabilities. The specific structure is shown in Figure 5.

Similarly to ViT, images entering the Swin Transformer are first divided into chunks using the patch partition structure and flattened along the channel dimension. These feature maps are then processed through four stages to extract feature information. In the first stage, each chunk undergoes a linear transformation before being fed into two consecutive Swin Transformer blocks. In the subsequent stages, to further extract features and reduce the size of the feature maps, the feature maps are downsampled using patch merging before being fed into the next Swin Transformer block.

2.3.3. Evaluation Indicators

Overall accuracy (OA), precision, F1_score, and recall were used as model evaluation metrics [36,37]. To calculate these four metrics, the following criteria were defined for the landslide identification problem, t: TP (True Positive), in which the pixels classified as landslide are correctly predicted as landslide; FP (False Positive), in which the pixels classified as non-landslide are incorrectly predicted as landslide; FN (False Negative), in which the pixels classified as landslide are incorrectly predicted as non-landslide; and TN (True Negative), in which the pixels classified as non-landslide are correctly predicted as non-landslide [38].

In summary, the above metrics are calculated as follows:

$$OA={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}$$

(5)

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(6)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(7)

$$F1\_score=2\times {\displaystyle \frac{\mathrm{Pr}ecision\times \mathrm{Re}call}{\mathrm{Pr}ecision+\mathrm{Re}call}}$$

(8)

2.3.4. Implementation Details

The experiments in this paper were conducted on the Windows 11 x64 Edition operating system, which provides a stable foundation with enhanced security features and performance optimizations. For deep learning frameworks and acceleration, the experiments utilize Pytorch version 2.1.1, complemented by Pytorch-CUDA version 11.8, to enable GPU acceleration, significantly enhancing model training speed. The training process is powered by an NVIDIA GeForce RTX4090 GPU with 24 GB of video memory, supporting large-scale model training and high-resolution image processing tasks. Regarding model training parameters, 100 training epochs are conducted for experiments involving the Swin Transformer and Dual-Coded Segmentation Network to thoroughly explore the model’s learning capabilities and mitigate overfitting. The batch_size is uniformly set to 8 to balance memory utilization and convergence speed. The network input consisted of remote sensing images with dimensions of 3 × 256 × 256, adhering to the standard input format required by most deep learning models. The Stochastic Gradient Descent (SGD) [39] optimizer is utilized as the optimizer. During the training process, it is necessary to gradually reduce the learning rate to improve the model. In this paper, we use the callback function ReduceLROnPlateau in Pytorch, which can automatically adjust the learning rate during training and is conducive to finding a balanced position between the training speed of the model and the efficiency of the training, which is achieved by reducing the learning rate after every N training rounds. In this paper, we try different learning rates, including 1 × 10⁻¹, 1 × 10⁻², 1 × 10⁻³, 1 × 10⁻⁴, 1 × 10⁻⁵, and the training results confirm that the learning rate of 1 × 10⁻⁴ is the most effective. Therefore, in this paper, the initial learning rate is set to 0.0001; N is 10, i.e., if the training accuracy does not decrease after 10 rounds of training, a new learning rate is introduced into the model for training. The newly set learning rate is to reduce the size of the initial learning rate by 20%. This study quantifies the computational efficiency based on the model parameters, average training time for each epoch, and floating-point operations (FLOPs), with the experimental results detailed in

Table 1. Computational resource metrics of DSNet.

Model	Parameters (M)	Average Training Time (s/Epoch)	FLOPs (G)
DSNet	238.14	415	198.92

.

3. Results and Analysis

3.1. Analysis of Ablation Experiment Results

In this study, a set of ablation experiments was conducted to investigate the impact of the backbone network on the experimental results. ResNet50, ResNet101, and the Swin Transformer were selected as encoders and backbone networks, respectively, to assess their influence on the feature extraction performance of the newly constructed DS Network. The effectiveness of the backbone network significantly determines the feature extraction capabilities of the encoder. A deep learning network model with strong learning ability relies heavily on whether the backbone network possesses robust feature extraction capabilities. Regarding the selection of ResNet models, both ResNet50 and ResNet101 are well-established network architectures that have been validated through extensive experiments and have demonstrated excellent performance across various computer vision tasks, including image classification, object detection, and semantic segmentation. Compared to ResNet18 and ResNet34, ResNet50, and ResNet101 have greater network depth, enabling them to better capture high-level image features. However, they are faster to train and easier to optimize than deeper models such as ResNet152. Therefore, in this study, ResNet50 and ResNet101, as representative ResNet architectures, were chosen as encoders with their fully connected layers removed. The validation results using the Swin Transformer as the encoder are presented in Figure 6.

The experimental results show that, compared to the DS Network using ResNet50 and ResNet101 as backbone networks, the DS Net with the Swin Transformer as the backbone network not only achieves a steady improvement in validation accuracy, reaching a higher level, but also exhibits relatively lower training loss.

The Swin Transformer model demonstrates significant performance advantages, particularly in comparison to the classic ResNet50 and ResNet101 models [40]. From the perspective of validation accuracy, the accuracy curve of the Swin Transformer exhibits a steady upward trend, quickly surpassing the baseline performance of ResNet50. Moreover, as training progresses, it further widens the gap with ResNet101, eventually stabilizing at a relatively high accuracy level. This improvement highlights the Swin Transformer’s remarkable advancements in feature extraction and model generalization capabilities.

Meanwhile, observing the changes in the loss, the loss curve of the Swin Transformer exhibits a smoother and more rapidly descending trajectory. In the early stages, its loss value is significantly lower than that of the ResNet family of models, indicating that the Swin Transformer can converge more quickly to the vicinity of the optimal solution. As the number of training rounds increases, this advantage becomes even more pronounced, with the loss curve continuing to decline and eventually reaching a value lower than both ResNet50 and ResNet101. This directly highlights the Swin Transformer’s efficiency and stability during the optimization process. Additionally, it demonstrates that while increasing network depth can enable the extraction of more features, it may also lead to training challenges such as gradient vanishing or gradient explosion, which can adversely affect the model’s convergence speed and segmentation performance.

3.2. Visual Analysis of Experimental Results

The Swin Transformer demonstrated the best overall performance as the backbone network in the ablation experiments. Therefore, in this subsection, the prediction results of the DS Net and the Swin Transformer are visualized and compared individually. The results of the evaluation metrics are presented in

Table 2. Evaluation metrics for the DS Net compared to the Swin Transformer.

Models	OA	P	Recall	F1_Score
Swin Transformer	0.895	0.843	0.814	0.828
DS Net	0.926	0.884	0.879	0.882

.

As shown in the table, a comprehensive evaluation of the two advanced models, Swin Transformer and DS Net, clearly demonstrates the significant advantages of DS Net. Specifically, DS Net achieves an OA of 0.926, surpassing the Swin Transformer’s 0.895. Additionally, DS Net attains a P of 0.884, compared to the Swin Transformer’s 0.843, showcasing its superior ability to accurately identify pixels in landslide areas. More importantly, DS Net significantly outperforms the Swin Transformer on the key metric of recall, with a value of 0.879 compared to the Swin Transformer’s 0.814. This indicates that DS Net is better equipped to comprehensively identify landslide areas in actual events, effectively reducing the risk of missed detections. The F1_score, combining P and recall, further confirms the superiority of DS Net, with a score of 0.882, far exceeding the Swin Transformer’s 0.828. This highlights DS Net’s robust and consistent performance in landslide identification tasks. These substantial performance improvements can be attributed to DS Net’s innovative network design, which incorporates a priori knowledge extracted and fused through the CLIP model. This enables DS Net to more effectively capture the complex features of landslide areas, ensuring its exceptional performance.

3.3. Comparison with Other Semantic Segmentation Models

To evaluate the performance of the DS Net-based landslide recognition model more objectively, we incorporated findings from the backbone network ablation experiments, which showed that CNN-based models significantly underperformed compared to the Swin Transformer. Therefore, four representative efficient semantic segmentation models were selected for comparison in this study: 1. SegFormer [41] uses a hierarchical transformer encoder and a lightweight MLP decoder to significantly reduce computational complexity while maintaining accuracy. 2. SegNeXt [42] innovatively fuses convolution and attention mechanisms, and its Convolutional Attention Module (CAM) excels in edge preservation and detail segmentation. 3. FeedFormer [43] creates the first feed-forward transformer architecture, replacing the standard attention layer with a cascading feed-forward network to dramatically increase processing speed and memory efficiency. 4. U-MixFormer [44] is an improved hybrid transformer based on U-shaped architecture, which is especially suitable for complex scene segmentation tasks by optimizing local and global feature fusion. The selection of these advanced models provided a comprehensive and rigorous frame of reference for DS Net’s performance evaluation.

During the implementation of the study, the Beichuan County landslide extraction dataset, which is unified and contains rich landslide disaster information, was used to systematically train all the models (including DS Net) involved in the comparison. This dataset was chosen not only because it can comprehensively reflect the actual situation of landslide extraction but also because it can rigorously test the identification and generalization abilities of each model in a complex and changing environment. By constructing a unified training framework and testing process, this study ensured that all models were evaluated under the same conditions, thus effectively excluding potential interference from external factors. In terms of accuracy performance on the test set,

Table 3. Comparison of DS Net results with other semantic segmentation networks.

Models	OA	P	Recall	F1_Score
FeedFormer	0.890	0.829	0.814	0.821
SegFormer	0.889	0.811	0.837	0.824
SegNeXt	0.890	0.825	0.822	0.823
U-MixFormer	0.891	0.836	0.808	0.822
DS Net (ours)	0.926	0.884	0.879	0.882

details the key evaluation metrics of each model, including OA, P, recall, and F1_score. Through comparative analysis, it can be clearly seen that the DS Net model achieves significantly better results than the other comparative models in all evaluation metrics. Specifically, DS Net’s OA is as high as 0.926, a metric that is higher than those of FeedFormer, SegFormer, SegNeXt, and U-MixFormer by 3.6%, 3.7%, 3.6%, and 3.5%, respectively. Its P is 0.884, which is 5.5%, 7.3%, 5.9%, and 4.8% higher than those of the other models. Its recall rate reaches 0.879, which is 6.5%, 4.2%, 5.7%, and 7.1% higher compared to the other models, while its F1_score is 0.882, outperforming FeedFormer, SegFormer, SegNeXt, and U-MixFormer by 6.1%, 5.8%, 5.9%, and 6.0%, respectively.

In order to demonstrate more intuitively the difference in the performance of the models in the landslide identification task, this study also provides a visual output of the model comparison, as shown in Figure 7.

In Figure 7, we can closely observe the recognition results produced by different models for the same complex landslide area. These results not only visualize the performance differences among the models but also reveal their technical limitations and advantages in handling specific geological features. Leveraging its superior design, the DS Net model significantly outperforms other comparative models in recognizing the precise boundaries of landslides, capturing landslide-specific features with remarkable detail, and accurately distinguishing landslide areas from non-landslide areas. By extracting multi-scale hierarchical features and local-global information of the images through the Swin and CLIP models and employing multi-head and SimAM attention mechanisms for in-depth feature fusion and focusing on key information, the DS Net model achieves highly accurate landslide mask generation. The effectiveness of this network can be attributed to its advanced feature fusion strategy, fine-grained information processing mechanism, and its ability to precisely capture critical landslide information.

Compared to the excellent performance of DS Net, the other four semantic segmentation models in the comparison, despite their expertise in specific domains, exhibited relatively similar overall performance levels in the task of landslide identification. This suggests potential deficiencies in their ability to model landslide-specific attributes. For instance, FeedFormer, an innovative sequence-to-sequence model based on the Transformer architecture, excels in domains such as natural language processing by effectively handling large volumes of sequential data with its self-attention mechanism. However, its application in semantic segmentation, particularly in the highly specialized task of landslide recognition, appears limited. This limitation stems primarily from the extreme complexity and diversity of landslide terrain, which exceeds the scope of features that the model can learn from a limited and potentially biased training dataset. Combined with constraints in generalization ability, FeedFormer struggles to make accurate predictions when encountering variant landslide features. Additionally, its attention mechanism may fail to adequately focus on critical landslide details, such as minute cracks and soil disturbances, leading to incomplete recognition results. SegFormer, another lightweight Transformer-based semantic segmentation model, leverages a self-attention mechanism to capture long-distance dependencies in images, aiming to enhance global information comprehension and improve segmentation accuracy. However, when faced with the highly similar spectral and textural features of landslides and vegetation, SegFormer encounters difficulties in distinguishing between these visually similar categories. In areas with dense vegetation and complex landslide morphology, its prediction results often fail to accurately delineate boundaries between landslides and vegetation cover, revealing its limitations in addressing closely related categories. SegNeXt and U-MixFormer face challenges more closely tied to their structural characteristics. SegNeXt, a convolutional neural network-based model, enhances spatial attention with a multi-scale convolutional attention module. However, its conventional convolutional operations may fall short in capturing the subtle but crucial spectral and textural differences between landslides and their surroundings, particularly vegetation. This limitation becomes especially evident in cases where landslide edges are blurred or intertwined with vegetation. On the other hand, U-MixFormer, a hybrid model combining U-Net and Transformer advantages, theoretically should improve segmentation accuracy through hierarchical feature extraction and global contextual understanding. However, in practice, its decoder may struggle to effectively differentiate between landslide and vegetation features when fusing multi-level encoder outputs. Similarly, its hybrid attention mechanism may fail to sufficiently focus on key landslide areas, resulting in inaccurate recognition. These observations indicate that even models integrating multiple advanced technologies require further optimization in structural design and feature extraction strategies to overcome the challenges of capturing and distinguishing complex terrain features. This is essential to meet the high-precision recognition demands of highly specialized tasks like landslide detection in practical applications.

4. Discussion

4.1. Comparison and Analysis of Different Data Enhancement Patterns

Data enhancement is a powerful data processing technique that applies diverse transformations to segmented images [45,46], such as rotation, flipping, adding noise, and blurring [47,48,49] (as shown in Figure 8). While it does not substantially introduce brand new data, it effectively simulates various possible changes in images under practical conditions, exponentially increasing the diversity and value of the dataset at a lower cost. Of particular importance is ensuring that corresponding labels undergo consistent transformations alongside the augmented image data. Labels are crucial in supervised learning as they guide the model to learn how to extract the correct outputs from input images. For example, if a rotation operation is applied to an image, its corresponding labels must also be rotated in the same manner to maintain the alignment between image content and label information. This synchronized transformation of labels is a critical factor in ensuring the effectiveness of data enhancement. Failure to maintain this consistency could result in the model learning incorrect information, ultimately affecting its performance.

Considering that this study uses a homemade landslide identification dataset, it may not be quantitatively superior to other public datasets. To prevent the model from overfitting due to the limited number of images in the dataset, data enhancement techniques were employed to expand the original dataset. For this purpose, methods such as image rotation, contrast adjustment, and chromaticity adjustment were primarily adopted. These transformation operations enhance the model’s ability to recognize different viewpoints, mirror morphologies, noise interference, and blurring effects in images. This significantly improves the model’s generalization ability, enabling it to exhibit stronger adaptability and robustness when dealing with new data or complex scenarios. Data enhancement is an indispensable and important method in deep learning and computer vision research. In this study, training sample sets with different enhancement levels were used to statistically analyze the changes in predictive indicators for various models on the test set. The results are shown in Figure 9.

The key performance indicators (OA, P, F1_score, recall) of DSNet exhibit a gradual decline as the dataset enhancement multiplier increases. From the feature perspective, although data enhancement increases data diversity and improves the model’s generalization ability to unseen samples, excessive enhancement can compromise key structural and detailed information within the images, particularly the subtle features critical to segmentation tasks. While the Swin Transformer and CLIP encoders in DSNet excel at capturing multi-scale, multi-level, and global contextual information, they depend on high-quality features in the original images for accurate encoding. When these features are distorted by over-enhancement, the encoder’s feature extraction capability may be severely impacted, resulting in features that no longer accurately reflect the content of the original image, thereby degrading segmentation performance. From the model learning perspective, the new samples introduced through data augmentation are not always consistent with the original task distribution. Over-enhanced images may deviate significantly from the original data, causing the model to learn features or noise unrelated to the task. This “noise learning” phenomenon disrupts the model’s ability to identify and utilize key features, leading to poorer performance on the test set. Such interference is especially pronounced in landslide identification tasks like those using DANet, which rely heavily on a priori knowledge for precise segmentation. When employing data enhancement techniques, it is crucial to carefully balance the intensity of enhancement against model performance to avoid issues such as feature distortion, noise learning, and overfitting caused by excessive enhancement. Future research can explore more intelligent data enhancement strategies, such as dynamic enhancement guided by model feedback or task-oriented enhancement methods. Additionally, developing more robust feature extraction and segmentation models will help address the challenges posed by data augmentation. Moreover, conducting in-depth investigations into the impact of data enhancement on model learning mechanisms represents an essential direction for future research.

4.2. The Model Adaptation Capability of DSNet on the Bijie Dataset

The practical value of landslide identification models largely depends on their ability to adapt to different environments. In order to verify the practical effectiveness of the DSNet proposed in this study, the publicly available landslide dataset [50] in Bijie, Guizhou, is used to carry out the adaptability analysis. The area is located in the slope zone of the transition from the Tibetan Plateau to the eastern hills, with an altitude of 457–2900 m, large relative altitude difference, many steep slopes, abundant rainfall (average annual rainfall of 849–1399 mm), and fragile ecological environment. Meanwhile, Bijie City is located on the Yunnan–Guizhou Plateau, and topographically belongs to the confluence of two or three levels of stairs, with a large relative difference in elevation within the region. The topography and precipitation conditions in Bijie laid the foundation for the development of landslides. Examples of landslides in the dataset are shown in Figure 10.

In this study, the Bijie landslide dataset was allocated into training, validation and test sets in the ratio of 6:2:2 using a randomized partitioning strategy, and the data expansion operation was implemented on the training samples. The network parameters of all models are fixed during the experiment, and the optimal weights obtained from pre-training are directly called for inference analysis, and the final model test results are shown in

Table 4. DS Net generalization results in Bijie dataset compared to other semantic segmentation networks.

Models	OA	P	Recall	F1_Score
FeedFormer	0.955	0.758	0.844	0.780
SegFormer	0.956	0.734	0.865	0.794
SegNeXt	0.951	0.772	0.842	0.772
U-MixFormer	0.961	0.772	0.850	0.809
DS Net (ours)	0.969	0.823	0.868	0.845

and Figure 11. DSNet outperforms other models overall.

As can be seen from Table 4, loading the optimal weights of the pre-trained model can effectively improve the model’s performance. This is due to the fact that the pre-training weights have already learned rich feature representations through historical data, and using them as initialization parameters for the new task can enable the model to quickly obtain strong landslide identification capability. Even on a completely new set of landslide datasets, it is possible to achieve excellent recognition test results by mining image features with a short training period.

Figure 11 compares the prediction results of different models (FeedFormer, SegFormer, SegNeXt, U-MixFormer, and DS Net) in the Bijie landslide identification task. Comparison of the prediction results between real labels and each model shows that DS Net has a more complete detection of the landslide region, fewer FN and FP, and more accurate boundary segmentation, and the overall performance is better than the other models. This indicates that the DS Net proposed in this study is able to learn landslide features more efficiently, thus improving the recognition accuracy.

4.3. Application of DS Net in Other Scenarios

In order to verify the generalizability of the DSNet, the publicly available GVLM dataset [51] is selected as the research subject in this study. The GVLM dataset contains 17 pairs of VHR (very high-resolution) imagery acquired through the Google Earth service at a spatial resolution of 0.59 m. As shown in

Table 5. Details of the GVLM dataset.

No.	Locations	Image Size	Triggers
1	A Luoi, Vietnam	7346 × 4096	Rainfall
2	Asakura, Japan	5632 × 3584	Earthquake
3	Askja, Iceland	4151 × 2763	Snow and glacier melting
4	Big Sur, United States	1748 × 1748	Loose soil and rock splitting
5	Chimanimani, Zimbabwe	10,808 × 7424	Tropical cyclone
6	Jiuzhaigou, China	5888 × 6313	Earthquake
7	Kaikoura, New Zealand	4977 × 3897	Earthquake
8	Kodagu, India	8704 × 6912	Rainfall
9	Kupang, Indonesia	1946 × 1319	Rainfall
10	Kurucasile, Turkey	8192 × 4608	Flood
11	Los Lagos, Chile	8533 × 4077	Glacier melting and rainfall
12	Osh, Kyrgyzstan	8860 × 7193	Melting snow and rainfall
13	Santa Catarina, Brazil	4864 × 3072	Torrential rain
14	Shimen, China	1861 × 1749	Rainfall
15	Taitung, China	3840 × 3840	Typhoon and rainfall
16	Tbilisi, Georgia	5588 × 5632	Flood
17	Tenejapa, Mexico	4200 × 1301	Hurricane

, the dataset covers diverse landslide scenarios in different countries and regions in six continents: Asia, Africa, North America, South America, Europe, and Oceania. The dataset covers a total area of 163.77 km², effectively making up for the fact that the existing public landslide database is limited to only a few cities or regions [52].

The dataset contains 17 subsets covering landslides in different geographical areas. These landslides show significant differences in size, morphology, phenology, and land cover type (some landslide samples are shown in Figure 12). Due to the above reasons, it causes its remote sensing images to exhibit a high degree of spectral heterogeneity and variation in radiation intensity. Landslides are triggered by a variety of mechanisms, including rainfall, earthquakes, floods, hurricanes, snowmelt, and rock loosening. The major land cover types involved in GVLM dataset can be categorized into three groups: (1) man-made surfaces (roads, buildings, and residential areas); (2) vegetation cover (farmland, forests, shrubs, and grasslands); and (3) bodies of water (snow, ice, rivers, and oceans). The high diversity characteristics of this dataset provide a solid foundation for evaluating the generalization ability of the DSNet proposed in this study.

The entire GVLM dataset was used as a test set for direct prediction. The DSNet generalization results are compared with other semantic segmentation models as shown in

Table 6. DS Net generalization results in GVLM dataset compared to other semantic segmentation networks.

Models	OA	P	Recall	F1_Score
FeedFormer	0.879	0.645	0.795	0.712
SegFormer	0.895	0.683	0.816	0.743
SegNeXt	0.797	0.476	0.873	0.616
U-MixFormer	0.889	0.674	0.785	0.725
DS Net (ours)	0.897	0.661	0.856	0.745

. DSNet outperforms other models overall.

This study demonstrates, in detail, the prediction results for A Luoi, Vietnam, as shown in Figure 13. DSNet effectively identified most of the landslide areas in the experimental area. DSNet still achieved excellent recognition performance under complex terrain conditions, with a precision of 78.38%, an overall accuracy of 99.25%, and an F1 score maintained at a high level of 74.04%.

These quantitative indicators provide sufficient evidence of the model’s core competencies. Even when faced with the challenge of cross-regional application, the model still successfully captured key features of the landslides. Particularly noteworthy is that the model is able to effectively distinguish image features that are different from those of Beichuan County, showing good generalization performance. The test results show that the overall recognition effect of DSNet is poor for relatively small landslides, which indicates the direction of the model’s subsequent improvement. Undeniably, the identification case in Figure 11 proves that the modeling framework is able to accurately target landslide locations and provide reliable technical support for disaster assessment. The practical value of this study is to confirm the potential of the DSNet model as a new generation landslide identification tool. Although there are adaptability differences in different geologic environments, its core recognition framework exhibits excellent performance and provides a new technical path for geologic hazard monitoring. In the future, the model is expected to achieve a wider range of applications by continuously optimizing the training strategy and enhancing the feature learning capability.

4.4. Limitations and Future Work

This study proposes a dual encoding segmentation network (DSNet), which extracts multi-scale image features and global-local information, respectively, through dual-path encoding of Swin Transformer and CLIP model. When integrated into landslide identification a priori knowledge using PKI module, the deep fusion of features and knowledge is realized by the multi-attention and SimAM in the CFA module, which finally outputs a high-precision landslide mask. Finally, the landslide mask is output through hierarchical decoding, showing significant advantages in recall (0.879) and F1-score (0.882) indicators, providing a new paradigm for landslide detection in complex terrain.

An in-depth analysis of the prediction results in Figure 12 of the generalization experiment reveals that the model is still deficient in the precise delineation of landslide boundaries, especially when dealing with landslides that are relatively small in size and exhibit elongated morphology. This limitation mainly stems from the following three factors: 1. Insufficient coverage of small-scale elongated landslide samples in the current training dataset; 2. The pixel features presented by this type of landslides in remote sensing imagery are relatively weak; 3. The elongated morphology leads to fragmentation of edge features, which makes it difficult for the model to establish a continuous and accurate boundary representation.

To address the limitations of the above model in elongated landslide boundary identification. In the future, we will carry out a systematic improvement study from two levels: data collection and algorithm optimization. At the data level, the focus is on supplementing typical elongated landslide samples. By using a combination of UAV tilt photography and ground-based LiDAR scanning, three-dimensional topographic data with centimeter-level accuracy were acquired, and a system of quantitative indexes for landslide morphology features was established, including morphological parameters such as aspect ratio, curvature, and slope direction. Meanwhile, the sample augmentation algorithm based on generative adversarial network is developed to generate synthetic samples of different sizes and morphology ratios through parameterized control, especially for the targeted enhancement of extreme morphology landslides with a length of more than 80 m and an aspect ratio of more than 5:1. In terms of algorithm improvement, the focus will be on how to transform morphological parameters such as landslide aspect ratio and strike into attentional weights to strengthen the feature extraction capability for slender structures. At the same time, a post-processing module based on the active contour model is developed to combine deep learning output with geometric constraint optimization to ensure that the boundary prediction results are consistent with both image characteristics and geological morphological laws.

5. Conclusions

In this study, an efficient and accurate landslide identification method was developed, featuring a Dual-coded Segmentation Network that integrates features extracted from a priori knowledge. This network combines the Swin Transformer and the CLIP large models as dual encoders and utilizes a hierarchical up-sampling technique for decoding, enabling comprehensive capture of multi-scale, multi-layer, and global information from landslide images. The results of comparison and ablation experiments reveal the following:

• The DS Network with the Swin Transformer as the encoder demonstrates better accuracy compared to ResNet50 and ResNet101 in ablation experiments;
• The DS Net model excels in landslide identification tasks, significantly outperforming comparison models such as SegFormer, SegNeXt, FeedFormer, and U-MixFormer across all evaluation metrics, with specific improvements ranging from 3.5% to 7.3%;
• In the landslide identification task, excessive data enhancement can disrupt key image features and introduce noise, leading to performance degradation of the DSNet model. Therefore, the relationship between enhancement intensity and model performance must be carefully balanced.
• While DSNet achieves superior performance (Recall = 0.879, F1 = 0.882), its current limitation lies in accurately segmenting small, elongated landslides due to dataset gaps and weak edge features. Future work will combine UAV/LiDAR-based 3D morphology augmentation and GAN-generated synthetic samples with algorithm-level enhancements integrating aspect-ratio-guided attention and active contour post-processing.