LD-DETR: A Multi-Scale Landslide Detection and Segmentation Network for Complex Geological Environments

Highlights

What are the main findings?

• LD-DETR improves multi-scale landslide detection and instance segmentation in high-resolution remote sensing imagery.
• The proposed method achieves better detection performance than the baseline model and demonstrates good transferability across different landslide datasets.

What are the implications of the main findings?

• The proposed framework provides an approach for landslide detection and instance segmentation in high-resolution remote sensing imagery.
• The proposed method enables parameter-efficient adaptation of Transformer-based landslide detection models through low-rank fine-tuning.

Abstract

Landslides are among the most destructive natural hazards. High-resolution remote sensing imagery provides an effective means for landslide detection and mapping, supporting geological hazard assessment and early warning. However, existing methods still face challenges in complex mountainous scenes with heterogeneous land cover and geological conditions. These challenges include irregular segmentation boundaries caused by diverse landslide shapes, small-scale detection difficulties due to background interference, and foreground–background class imbalance that hinders effective model optimization. To address these issues, we propose LD-DETR, a network for landslide detection and segmentation. An Atrous Spatial Pyramid Pooling (ASPP) module is integrated into the segmentation head to improve multi-scale feature representation, enabling better segmentation of landslides with varying spatial sizes. To alleviate class imbalance caused by the dominance of non-landslide pixels, a weighted Tversky Loss is introduced to improve the learning of hard samples. Low-Rank Adaptation (LoRA) is further combined with orientation-aware data augmentation to enable parameter-efficient adaptation, reducing the proportion of trainable parameters of LD-DETR to 41.07% while keeping the pre-trained backbone parameters frozen. Experiments on the Bijie and Luding Landslide Dataset (BLLD) demonstrate that LD-DETR is competitive with the baseline model and several YOLO-based approaches. The proposed method achieves an F1-score of 95.18% and a Recall of 95%, while the segmentation task reaches an mAP of 59.28%, slightly higher than the RF-DETR baseline. Furthermore, a limited-sample adaptation experiment on the Jiuzhai landslide dataset demonstrates the transferability of the proposed method across different geographic regions. These results indicate that the proposed framework provides competitive landslide detection and segmentation performance and offers an approach for fine-grained landslide mapping using remote sensing imagery.

1. Introduction

Landslides represent a profoundly destructive class of geological hazards [1]. Their triggering mechanisms are highly complex, primarily encompassing slope degradation from persistent heavy rainfall, seismic disturbances to underlying geological structures, and anthropogenic destabilization via infrastructure development and mining [2]. These events pose a substantial threat to both human life and socioeconomic infrastructure. For effective emergency response and regional risk assessment, acquiring accurate spatial distributions of affected areas in a timely manner is paramount [3]. Recently, fueled by ongoing breakthroughs in computer vision and the increasing accessibility of high-resolution satellite imagery, deep learning-driven intelligent detection frameworks have rapidly emerged as a critical research frontier [4]. Compared with traditional paradigms, these neural architectures can autonomously extract discriminative features from large-scale datasets [5]. By combining computational efficiency with robust analytical capabilities, they provide an effective approach for large-scale, high-precision automated landslide mapping [6,7]. This study focuses on automatic landslide detection and instance segmentation from high-resolution remote sensing imagery, aiming to accurately identify and delineate landslide-affected areas for rapid post-disaster mapping and hazard assessment.

In recent years, deep learning has become an important approach for landslide detection and segmentation. Existing methods can generally be grouped into two categories: object detection and semantic segmentation [8]. Object detection methods, such as Faster R-CNN and the YOLO series, localize and classify landslides using bounding-box regression, providing a balance between detection accuracy and computational efficiency [9]. To address challenges such as large variations in landslide size, complex morphology, and class imbalance, researchers have improved detection frameworks through feature extraction, multi-scale feature fusion, and loss function design. For example, Hou et al. [10] improved YOLOX by introducing VariFocal Loss and optimizing the feature extraction network to reduce missed detections of small and morphologically complex landslides. Fu et al. [11] incorporated a Swin Transformer into the Mask R-CNN backbone, enabling the model to perform both object detection and instance segmentation. Semantic segmentation predicts the class of each pixel and is therefore more suitable for delineating landslide boundaries with irregular shapes [12,13]. Recent studies have mainly focused on network architecture design, multimodal feature fusion, and lightweight models. Lu et al. [14] proposed a dual-encoder U-Net that combines spectral and topographic information through feature-level fusion. More recent studies have further integrated attention mechanisms, pre-trained models, and multi-scale feature fusion to improve segmentation performance in complex environments [15,16,17]. Although convolutional neural network (CNN)-based methods have achieved good performance in landslide detection and segmentation, their ability to model long-range contextual information is still limited by the local receptive field, which has motivated the application of Transformer-based models to landslide recognition.

With the development of Vision Transformer (ViT), Transformer-based methods have gradually been introduced into landslide recognition tasks [18]. Compared with convolutional neural networks, which rely on local receptive fields for feature extraction, Transformers model dependencies between different image regions through self-attention, thereby improving the representation of global contextual information. On this basis, Transformer-based models have been applied to landslide detection and segmentation and have led to several architectural variants. In landslide detection and segmentation, one line of research integrates Transformers with convolutional networks to enhance feature representation. Lv et al. [19] proposed ShapeFormer, which adopts Pyramid Vision Transformer (PVT) as the backbone and introduces a shape-aware branch to improve the representation of boundary-related features. Chen et al. [20] designed CTDBlock by combining ConvNeXt and Swin Transformer in a parallel manner, enabling complementary learning of local feature extraction and global dependency modeling, and using a gating mechanism to fuse multi-level features. Another line of work is based on the DETR framework. DEtection TRansformer (DETR) formulates object detection as a set prediction problem and directly outputs detection results through a Transformer encoder–decoder architecture, using bipartite matching for training, which enables an end-to-end detection pipeline [21]. Based on this framework, Zhu et al. [22] proposed Deformable DETR, which replaces global attention with sparse sampling around reference points to improve multi-scale feature modeling. Subsequently, Lv et al. [23] proposed RT-DETR, which improves efficiency by decoupling intra-scale interactions and cross-scale fusion within a hybrid encoder and introduces an uncertainty-based query selection strategy to improve prediction quality. Robinson et al. [24] further proposed RF-DETR, which uses DINOv2 as the backbone and applies neural architecture search (NAS) to balance accuracy and computational cost, while supporting both object detection and instance segmentation tasks.

Despite the progress of deep learning in landslide detection, existing generic frameworks still face challenges in complex geological environments and for landslides with varying morphologies. On the one hand, variations in landslide scale make it difficult for a single network to balance the preservation of global structural information with the capture of fine edge details, which may lead to discontinuities in feature representation. On the other hand, in wide-area remote sensing scenarios, non-landslide regions dominate most of the imagery, resulting in a highly imbalanced distribution of samples. This imbalance can hinder the learning of under-represented landslide samples and increase the likelihood of missed detections. Limited labeled geological data makes it challenging to fully fine-tune large pre-trained models while achieving effective adaptation to domain-specific characteristics. To address these challenges, this study adapts the RF-DETR framework for landslide detection and segmentation by incorporating task-specific improvements. The resulting model, termed LD-DETR, integrates three main components:

• An Atrous Spatial Pyramid Pooling (ASPP) module is incorporated into the network to enhance multi-scale feature extraction and improve landslide boundary delineation.
• Training is optimized using a weighted Tversky loss to reduce the influence of foreground–background sample imbalance.
• LoRA is combined with orientation-aware data augmentation to enable parameter-efficient fine-tuning under limited labeled data conditions.

2. Materials and Methods

2.1. Datasets

Deep learning-based landslide detection methods generally rely on a large number of high-quality annotated remote sensing images. However, due to the sporadic occurrence of geological hazards and the high cost of data acquisition, available landslide samples remain relatively limited [25]. Datasets collected from a single region often lack sufficient diversity for training models with strong generalization capability, which may restrict their applicability in real-world scenarios. To increase the diversity of training samples, this study integrates the publicly available Bijie Landslide Dataset [3] and Luding County Landslide Dataset [26] to construct the BLLD dataset. These two datasets complement each other in terms of geological environment, image sources, spatial resolution, and landslide scales, providing a more diverse benchmark for model training and evaluation. Both datasets use the overall spatial extent of landslides as the annotation target. The annotations were mainly generated based on geomorphological characteristics observed in high-resolution remote sensing images, including landslide scarps, displaced materials, accumulation zones, exposed soil/rock regions, and textural variations, together with historical landslide records for comprehensive interpretation [27,28]. This annotation strategy aims to represent the spatial extent of landslides as completely as possible. In this study, the original manual annotations from publicly available datasets were directly adopted as the ground truth, without redefining or modifying the annotated boundaries. The geographic locations and spatial distributions of the Bijie and Luding landslide datasets are shown in Figure 1.

Figure 1. Spatial overview of the Bijie and Luding landslide datasets: (a) geographical location of Luding County; (b) landslide sample distribution in the Luding dataset; (c) geographical location of Bijie City; and (d) landslide sample distribution in the Bijie dataset.

The Bijie landslide dataset developed by Ji et al. [3] covers Bijie City in northwestern Guizhou Province, China. Located in the transition zone between the Tibetan Plateau and the eastern hilly region, this area is highly susceptible to landslide hazards. The study area covers approximately 26,853 km², with elevations ranging from 457 m to 2900 m. The dataset contains RGB images acquired by the TripleSat satellite from May to August 2018, with a spatial resolution of 0.8 m, together with a digital elevation model (DEM) with an elevation accuracy of 2 m. During the annotation process, geological experts identified landslide locations based on historical disaster inventories and manually delineated landslide boundaries using ArcGIS 10.2. The delineation of landslide boundaries was mainly based on geomorphological characteristics observed in high-resolution remote sensing images, including landslide scarps, the extent of displaced materials, exposed soil or rock surfaces, accumulation zones, and textural variations. Historical landslide records were also considered for comprehensive interpretation, rather than determining landslide boundaries solely on the basis of vegetation changes. In the dataset, each sample is defined as the region within a 40 m outward expansion of the landslide bounding box. The dataset contains 770 independent landslide instances, and their accuracy has been verified through field investigations. The subset includes 770 positive samples containing landslides and 2003 negative samples covering complex backgrounds such as mountainous areas, villages, and roads. These negative samples are used to improve the model’s ability to distinguish landslide features and reduce false detections. However, relying solely on data from a specific geographic region may not be sufficient to fully evaluate the model’s generalization capability across diverse conditions.

To further evaluate the model’s detection performance in complex environments and multi-scale landslide conditions, the Luding County landslide dataset [26] was introduced in this study. Luding County is located in the southeastern part of Garze Tibetan Autonomous Prefecture, Sichuan Province, China, and lies within the transition zone between the Sichuan Basin and the Tibetan Plateau. The region is characterized by steep terrain, fractured rock masses, and active tectonic processes, resulting in frequent occurrences of landslides and collapses. The dataset was constructed using Google Earth imagery collected between 2020 and 2022, with spatial resolutions ranging from 0.4 to 0.6 m. Before annotation, the remote sensing images were processed through image registration, geometric and atmospheric correction, and radiometric calibration. The processed images were then cropped and organized to meet the input requirements of the network. The annotations were manually generated by professional interpreters using ArcMap 10.2. Approximately 50 boundary points were collected along each landslide perimeter to delineate the landslide regions and generate pixel-level labels for landslide and background areas. The determination of landslide boundaries considered multiple identifiable features in remote sensing images, including landslide scarps, the extent of displaced materials, accumulation zones, exposed surfaces, and surface texture variations, ensuring that the annotations accurately represented the spatial extent of landslides. This subset contains 230 carefully identified landslide instances, each with a corresponding label. Compared with the Bijie dataset, the Luding dataset includes more small-scale landslides located in complex background environments, which increases the difficulty of extracting fine-grained landslide features for the model.

The integrated BLLD dataset contains a total of 3003 images, including 1000 positive samples and 2003 negative samples. Figure 2 provides representative sample examples and statistical analyses of data distribution. Figure 2a presents various typical negative samples containing complex background conditions that may cause false detections, including buildings, water bodies, terraced fields, and mountain roads. Figure 2b shows positive landslide samples together with their corresponding pixel-level masks. To quantitatively characterize the scale and morphological properties of landslide targets, the geometric attributes of the ground-truth bounding boxes were statistically analyzed. Figure 2c illustrates the multi-scale spatial distribution of landslide bounding boxes, revealing considerable variations in the relative size of landslide targets within the images. Figure 2d presents a two-dimensional density histogram of the normalized bounding box widths and heights. The distribution further indicates that landslide targets exhibit significant anisotropy in terms of aspect ratios. Overall, the BLLD dataset contains diverse landslide morphologies and background conditions, providing a comprehensive data foundation for training and evaluating the proposed model.

Figure 2. Representative samples and statistical characteristics of the integrated BLLD dataset: (a) representative negative samples with complex background conditions; (b) representative landslide samples and their corresponding pixel-level masks; (c) multi-scale distribution of ground-truth bounding boxes; and (d) two-dimensional density histogram of normalized bounding box widths and heights.

However, since the BLLD dataset is primarily collected from limited geographic regions, its geological conditions, imaging characteristics, and landslide types may not fully represent the diversity of real-world scenarios. Therefore, the experimental results obtained on this dataset mainly reflect the model performance within the corresponding data distribution. To further evaluate the cross-regional adaptability of the proposed model, the Jiuzhai landslide dataset [29] was introduced for additional validation. The original images of this dataset were collected by the Sichuan Surveying and Mapping Center using unmanned aerial vehicles (UAVs) from August to September 2017. The dataset contains 5925 images with a spatial resolution of up to 0.2 m. The Jiuzhai region is characterized by alpine canyon terrain, with bedrock mainly consisting of limestone and shale. The vegetation is dominated by alpine coniferous forests, and the landslides in this area were mainly triggered by earthquakes. Compared with the BLLD source-domain dataset constructed in this study, the Jiuzhai dataset differs substantially in terms of sensor type, spatial resolution, and geological environment. The complex post-earthquake terrain conditions provide a challenging test scenario for evaluating the model’s adaptability to unseen environments. The key parameters of the three datasets are summarized in Table 1 to highlight the distribution differences between the source and target domains.

Table 1. Characteristics of the source and target landslide datasets.

Characteristics	Bijie (Source)	Luding (Source)	Jiuzhai (Target)
Data Source	TripleSat Satellite	Google Earth	UAV
Spatial Resolution	0.8 m	0.4–0.6 m	0.2 m
Total Images	770	230	5925
Image Size	Variable (61–1239 pixels)	Variable (103–759 pixels)	$512\times 512$ pixels
Instances per Image	1.0 (Avg), 1 (Max)	1.0 (Avg), 2 (Max)	1.5 (Avg), 6 (Max)
Object Scale Range (pixels2)	120–1,118,923	2009–206,684	1–261,121

2.2. Methods

DETR (DEtection TRansformer), introduced by Carion et al. in 2020 [21], applies the Transformer architecture to object detection. Unlike YOLO-based [30,31] detectors that rely on predefined anchors and local feature extraction, DETR formulates object detection as a set prediction problem through attention-based feature modeling. Based on this framework, several improved variants, including Deformable DETR and DINO [22,23], have been developed to address limitations such as slow convergence and insufficient performance on small objects. RF-DETR [24] further extends the DETR framework by improving feature representation and multi-scale modeling while retaining the DETR-based prediction paradigm. RF-DETR provides a suitable foundation for landslide detection due to its Transformer-based feature representation. However, directly applying the original RF-DETR framework to landslide scenarios remains challenging due to limited labeled samples, large variations in landslide scales, and complex background interference. Therefore, this study adapts the RF-DETR framework for landslide detection and segmentation. The original DETR-based prediction architecture and DINOv2 backbone are retained, while orientation-aware data augmentation, LoRA-based [32] fine-tuning, an ASPP [33] module, and Weighted Tversky Loss [34] are introduced to improve the model adaptation to landslide remote sensing imagery.

2.2.1. LD-DETR

Figure 3 presents the overall architecture of LD-DETR, which is adapted from the RF-DETR framework for landslide detection and segmentation. The RF-DETR framework adopts a Transformer-based set prediction architecture with DINOv2 as the backbone encoder. The extracted features are used for object localization and pixel-level segmentation. The pretrained DINOv2 parameters serve as the feature-extraction foundation, while LoRA adapters are introduced into the backbone to enable task-specific adaptation during training. To reduce the domain discrepancy between natural image pre-training and landslide remote sensing imagery, a LoRA-based fine-tuning strategy is adopted. LoRA adapters are inserted into selected layers of the DINOv2 backbone, while the original pretrained parameters remain frozen. Orientation-aware data augmentation, including random discrete rotation and vertical flipping, is applied during preprocessing to increase the diversity of training samples and improve adaptation to different terrain orientations. For the segmentation branch, an ASPP module is introduced before the segmentation head to enhance multi-scale feature representation. By aggregating contextual information from different receptive fields, ASPP helps capture landslides with varying spatial scales. The Weighted Tversky Loss is adopted to alleviate foreground-background imbalance and improve segmentation of small and irregular landslide regions.

Figure 3. Overall architecture of the LD-DETR model. Orientation-aware augmentation at the input stage and LoRA adapters embedded within the Transformer architecture constitute a synergistic optimization strategy. In the segmentation branch, an ASPP module is introduced to enhance multi-scale perception. Additionally, a Weighted Tversky Loss replaces the standard loss function to improve learning on hard examples.

2.2.2. Orientation-Aware Augmentation and LoRA-Based Adaptation

Existing DETR-based frameworks commonly employ geometric augmentation strategies such as random horizontal flipping and scale variation during training. However, landslides in remote sensing imagery may appear with different orientations due to variations in image acquisition conditions. Therefore, augmentation strategies that increase orientation diversity can help reduce the model’s dependence on specific spatial arrangements of landslide targets. To address this issue, we introduce an augmentation strategy that combines orientation-aware data augmentation at the data level with LoRA-based adaptation at the model level. The original multi-scale augmentation strategy of DETR is retained, and an orientation-aware augmentation pipeline is introduced to increase the diversity of landslide samples. The adopted geometric transformations are designed according to the imaging characteristics of overhead remote sensing imagery rather than the physical movement of landslides. Because remote sensing images are acquired from a nadir perspective, landslide bodies may appear at different orientations in the image plane due to variations in acquisition conditions. Accordingly, images and their corresponding annotations are randomly rotated by 0°, 90°, 180°, or 270° during training. Compared with arbitrary-angle rotation, these discrete rotations preserve the original image content while avoiding black padding artifacts. In addition, random vertical flipping is introduced alongside the standard horizontal flipping. In overhead remote sensing imagery, image orientation is primarily determined by the sensor acquisition geometry and map projection rather than by an intrinsic upward or downward direction in the image plane. Therefore, horizontal and vertical flipping preserve the morphological characteristics of landslide boundaries, textures, and spatial patterns while increasing the diversity of training samples. The combination of these geometric transformations reduces the model’s dependence on specific image orientations and encourages it to learn orientation-invariant landslide features.

To achieve parameter-efficient fine-tuning, this study adopts LoRA to adapt the pre-trained DINOv2 encoder. LoRA adapters with a rank of 16 are inserted into the Query, Key, and Value projection layers of the multi-head self-attention modules, as well as the FC1 and FC2 layers of the MLP modules across the 12 Transformer Blocks of the encoder, involving a total of 60 linear layers. During training, the original pre-trained parameters of the encoder are frozen, and only the LoRA parameters and task-specific modules are optimized, thereby reducing the number of parameters that need to be updated during training.

The fundamental premise of LoRA [32] involves freezing the pre-trained weights $W_0\in {\mathbb{R}}^{d\times k}$ and approximating the weight update $\Delta W$ by introducing two low-rank matrices, $A\in {\mathbb{R}}^{r\times k}$ and $B\in {\mathbb{R}}^{d\times r}$ (where the rank $r\ll min(d,k)$). During fine-tuning, the modified forward pass for an input feature x is formulated as follows:

$$h=W_{0}x+\Delta Wx=W_{0}x+BAx$$

(1)

where h denotes the output feature vector. $W_0$ remains frozen, and only A and B are optimized. The detailed internal structure and forward propagation mechanism are illustrated in Figure 4.

Figure 4. Schematic diagram of the LoRA module. By introducing a zero-initialized low-rank bypass alongside the frozen pre-trained weights, the method significantly reduces the number of trainable parameters while preserving the generalization capability of the model.

The LoRA fine-tuning strategy is combined with the orientation-aware data augmentation described above. By applying random discrete rotations and vertical flipping, the augmentation exposes the model to training samples with diverse spatial orientations and geometric variations, thereby improving its ability to recognize landslides under different terrain conditions. Together with the parameter-efficient adaptation provided by LoRA, this training strategy facilitates knowledge transfer from the general visual domain to the geohazard domain, improving the robustness of landslide detection in complex terrain while maintaining efficient fine-tuning.

2.2.3. ASPP Segmentation Head Optimization for Multi-Scale Landslides

Landslides in complex geological environments exhibit considerable variations in spatial scale, shape, and aspect ratio. The dataset contains both small-scale landslide instances and large, elongated landslide regions (Figure 2c,d). The original RF-DETR segmentation head mainly relies on stacked Depthwise ConvBlocks to decode high-level semantic features. Although this design provides an efficient feature decoding process, the relatively limited receptive field may restrict the representation of landslides with large-scale variations. As a result, small landslide targets with weak boundary information may be difficult to distinguish from surrounding backgrounds, while large and irregular landslide regions may suffer from incomplete or discontinuous segmentation. To improve the representation capability of the segmentation branch for multi-scale landslide targets, an Atrous Spatial Pyramid Pooling (ASPP) module is introduced into the segmentation head to enhance multi-scale feature representation. ASPP aggregates contextual information through parallel atrous convolution branches with different receptive fields. Compared with standard convolution, atrous convolution expands the sampling interval of convolution kernels by introducing dilation rates, allowing the receptive field to be enlarged without increasing the kernel size. This enables the model to capture both local details and broader contextual information from landslide regions.

For a 2D feature map x of landslide data, the atrous convolution operation with a $K\times K$ convolution kernel is applied to extract contextual information. By introducing the dilation rate r, the sampling interval of the convolution kernel is expanded, enabling the convolution operation to capture a wider range of contextual information without increasing the kernel size. The mathematical formulation of the 2D atrous convolution is defined as follows [33]:

$$y(i,j)={\displaystyle \sum _{m=0}^{K-1}}{\displaystyle \sum _{n=0}^{K-1}}w(m,n)x\left(i+r(m-c),j+r(n-c)\right)$$

(2)

where $w(m,n)$ represents the convolution kernel weights, r denotes the dilation rate that controls the spacing between adjacent sampling locations, and $c=(K-1)/2$ is the center index of the convolution kernel. When $r=1$, the operation is equivalent to standard convolution. As the dilation rate increases, the receptive field of the convolution expands, allowing the model to capture broader contextual information from landslide regions while maintaining the number of learnable parameters.

Based on this mechanism, the ASPP module is integrated into the segmentation head to enhance multi-scale feature representation. The modified segmentation head consists of five parallel branches (Figure 5). One branch adopts a $1\times 1$ convolution to preserve local spatial details, while three parallel atrous convolution branches employ $3\times 3$ kernels with different dilation rates. Dilation rates of 6, 12, and 18 are adopted to provide progressively expanded receptive fields for contextual feature aggregation. The influence of different dilation rate configurations is further evaluated through a sensitivity analysis in Section 4.1.1 to verify the rationality of this parameter selection. A Global Average Pooling branch is introduced to capture image-level contextual information, which provides additional semantic cues for distinguishing landslide regions from spectrally similar background areas. The outputs of the five branches are concatenated and fused through a $1\times 1$ convolution to generate the final feature representation for segmentation prediction. Through this multi-branch design, the segmentation head is able to integrate local details and contextual information across different spatial scales, thereby improving the representation of landslide regions with diverse morphologies and sizes.

Figure 5. ASPP module architecture integrating multi-level atrous convolutions and global context.

2.2.4. Optimization Strategy with Tversky Loss for Class Imbalance

Accurately delineating landslides at the pixel level presents specific challenges in instance segmentation due to scale variations, irregular morphological features, and complex surrounding terrains. This task is further complicated by class imbalance. Foreground pixels account for 14.2% and 16.9% in the Bijie and Luding subsets, averaging approximately 15.0% across the BLLD dataset (Table 2). This background-dominated distribution biases network optimization toward the majority class, limiting foreground feature learning, especially in landslide regions with ambiguous boundaries or relatively limited spatial extent. The RF-DETR model employs a combination of binary cross-entropy loss and Dice loss for instance segmentation. Binary cross-entropy is sensitive to background-dominated distributions [21]. While Dice loss helps mitigate class imbalance by maximizing regional overlap, it assigns equal weights to false positives and false negatives. In landslide emergency applications, a missed detection is generally more critical than a false alarm, as it can lead to incomplete hazard assessments. Tversky loss is integrated into the instance segmentation branch, while binary cross-entropy is retained for training stability. By adjusting the weighting of false positives and false negatives, Tversky loss enhances foreground sensitivity and reduces missed detections, thereby improving segmentation performance under complex landslide conditions.

Table 2. Pixel-level foreground–background statistics of the landslide datasets.

Dataset	Foreground Pixels	Background Pixels	Foreground Ratio
Bijie	7.11 Mpx	43.15 Mpx	14.2%
Luding	3.54 Mpx	17.42 Mpx	16.9%
Total (BLLD)	10.65 Mpx	60.57 Mpx	14.96%

Tversky Loss [34] serves as a generalized form of Dice Loss, introducing asymmetric hyperparameters $\alpha$ and $\beta$ to modulate the penalties assigned to False Positives and False Negatives. Let p denote the predicted probability map and G represent the ground truth labels; the definition of the Tversky Index ($TI$ ) is as follows:

$$TI(P,G,\alpha ,\beta )={\displaystyle \frac{|P\cap G|}{|P\cap G|+\alpha |P\setminus G|+\beta |G\setminus P|+\epsilon }}$$

(3)

where $|P\cap G|$ denotes True Positives (TP), representing the number of correctly identified landslide pixels; $|P\setminus G|$ denotes False Positives (FP), representing the number of background pixels misclassified as landslides; and $|G\setminus P|$ denotes False Negatives (FN), representing the number of actual landslide pixels missed by the model. $\alpha$ and $\beta$ serve as the weighting coefficients for False Positives and False Negatives, respectively, satisfying $\alpha +\beta =1$. We set $\beta >\alpha$ to reinforce the penalty for missed detections. $\epsilon$ represents a numerical smoothing term, utilized to avoid division by zero and prevent gradient oscillation during the early stages of training.

Based on these definitions, the Tversky Loss function is formulated as follows:

$$L_{Tversky}=1-TI(P,G,\alpha ,\beta )$$

(4)

Unlike Dice Loss, which corresponds to $\alpha =\beta =0.5$, Tversky Loss allows different penalties to be assigned to false positives and false negatives. Here, $\alpha$ and $\beta$ are set to 0.3 and 0.7, respectively, to place greater emphasis on reducing missed landslide pixels. The influence of different $\alpha$ and $\beta$ combinations is further evaluated in Section 4.1.2, where the selected parameter setting is compared with alternative configurations.

To maintain stable pixel-level optimization, the original BCE Loss [35] is jointly optimized with Tversky Loss. The BCE Loss is defined as follows:

$$L_{BCE}(P,G)=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}[g_i\log \left(p_i\right)+(1-g_i)\log (1-p_i)]$$

(5)

where N denotes the total number of pixels, and $p_i$ and $g_i$ represent the predicted probability and ground truth label of the i-th pixel, respectively.

Finally, the improved total segmentation loss function $L_{seg}$ is formulated as a linear combination of the weighted BCE Loss and Tversky Loss:

$$L_{seg}={\lambda }_{bce}{L}_{BCE}(P,G)+{\lambda }_{tversky}{L}_{tversky}(P,G,\alpha ,\beta )$$

(6)

where $\lambda$ controls the contribution of Tversky Loss to the overall optimization.

$\lambda$ is set to 2.0. The influence of different $\lambda$ values is analyzed in Section 4.1.3, and the adopted setting is selected based on the corresponding sensitivity analysis.

3. Experimental Results and Analysis

3.1. Experimental Setup

All models were trained and evaluated on an NVIDIA GeForce RTX 3090 GPU with 24 GB of VRAM using the PyTorch 2.11.0 framework. The experiments were conducted on the integrated BLLD dataset described in Section 2.1. The Bijie and Luding datasets were independently partitioned into training, validation, and testing subsets at a ratio of $7:1:2$ while maintaining the original proportions of landslide and non-landslide samples within each dataset. The corresponding subsets were then merged to form the final training, validation, and testing sets of the BLLD dataset, as summarized in Table 3. During training, in addition to the standard DETR augmentations, the orientation-aware data augmentation strategy described in Section 2.2.2 was employed to expose the model to more diverse sample variations during each training epoch. In this study, each image sample from the BLLD dataset was used directly as an individual sample, without further cropping from large-scale remote sensing scenes. Therefore, the dataset partitioning was performed at the image level. To further evaluate the transferability of the proposed method beyond the random data split within the BLLD dataset, an additional cross-region adaptation experiment was conducted on the Jiuzhai landslide dataset, which was not involved in the construction of the BLLD dataset. The experimental protocol is described in Section 4.2. Unless otherwise stated, all quantitative results reported in this paper were obtained from a single training run under the experimental settings described above.

Table 3. Partition and sample statistics of the BLLD dataset.

Subset	Ratio	Landslide Samples	Non-Landslide Samples	Total
Training	70%	700	1402	2102
Validation	10%	100	200	300
Test	20%	200	401	601

3.2. Performance Evaluation Criteria

To evaluate the performance of the LD-DETR model in landslide detection and segmentation tasks, Precision (P), Recall (R), $F_1$-score ($F_1$), and Average Precision ($A{P}_{50}$, $mA{P}_{50:95}$) were selected as the core evaluation metrics, all of which were calculated at the instance level following the standard MS COCO evaluation protocol [36].

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(7)

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(8)

$$F_1=2\times {\displaystyle \frac{P\times R}{P+R}}$$

(9)

$$AP={\int }_0^{1}P\left(R\right)dR$$

(10)

$$IoU={\displaystyle \frac{Area(B_p\cap B_{gt})}{Area(B_p\cup B_{gt})}}$$

(11)

where $TP$ (True Positives) denotes the number of landslides correctly identified by the model, $FP$ (False Positives) represents the number of non-landslide areas misclassified as landslides, and $FN$ (False Negatives) refers to the number of actual landslides missed by the model. Additionally, $B_p$ and $B_{gt}$ represent the predicted region and the ground truth annotated region, respectively. Note that for the detection task, these regions refer to bounding boxes, whereas for the segmentation task, they correspond to pixel-level masks.

$$A{P}_{50}{=AP|}_{IoU=0.50}$$

(12)

$$mA{P}_{50:95}={\displaystyle \frac{1}{10}}{\displaystyle \sum _{i=0}^9}A{P}_{0.5+0.05i}$$

(13)

Here, $A{P}_{50}$ denotes the Average Precision at an IoU threshold of 0.50, while $mA{P}_{50:95}$ represents the mean Average Precision over IoU thresholds from 0.50 to 0.95. Following the MS COCO evaluation protocol, Average Precision is computed from the precision–recall curve and is therefore adopted as the primary evaluation metric for both object detection and instance segmentation in this study.

3.3. Results and Comparison with Representative Detection and Segmentation Methods

Figure 6 presents representative inference results of LD-DETR on the BLLD test set under different geological conditions. The model is able to localize and segment landslides in various scenarios, including mountainous regions, highway slopes, and reservoir banks. The predicted instance masks generally follow the irregular landslide boundaries and remain continuous for elongated or fragmented landslide bodies, with confidence scores above 0.90 in the examples shown. For visually similar background regions such as terraced fields, bare farmland, and riverbanks, no obvious false positives are observed in these examples. These qualitative results illustrate the detection and segmentation performance of LD-DETR on representative BLLD test samples. The transferability of the proposed method to a different study area is further evaluated through the cross-region adaptation experiments on the Jiuzhai landslide dataset presented in Section 4.2.

Figure 6. Visualization of landslide detection and segmentation results using LD-DETR in typical complex scenarios.

We compared LD-DETR with several representative object detection and instance segmentation models to further evaluate its performance on the BLLD dataset. The comparison included the YOLO series, Cascade Mask R-CNN, and the baseline RF-DETR. Table 4 summarizes the backbone architectures, trainable parameter numbers, and quantitative performance of all compared models, while Figure 7 and Figure 8 present the corresponding qualitative comparisons. Since the proposed method adopts a LoRA-based parameter-efficient fine-tuning strategy, the number of trainable parameters is reported to facilitate the comparison of parameter efficiency across different models.

Table 4. Performance comparison of different deep learning models on the landslide dataset.

Model	Backbone	Total Params (M)	Trainable Params (M)	Detection					Segmentation
				P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
YOLOv8-M	CSPDarknet	27.29	27.29	91.14	89.85	94.33	59.60	90.49	91.32	90.79	93.95	56.52	91.05
YOLOv11-M	C3k2 + C2PSA	22.42	22.42	88.17	82.74	90.51	54.86	85.37	90.03	84.26	91.73	49.93	87.05
Cascade Mask R-CNN	Swin-Tiny	80.45	80.45	87.56	88.00	93.60	53.10	87.78	85.57	86.00	91.20	48.70	85.79
YOLOv12-N	Attention-Centric	2.77	2.77	88.99	81.22	88.99	53.64	84.93	87.58	82.35	88.62	50.54	84.89
YOLOv12-M	Attention-Centric	21.92	21.92	87.84	90.35	94.54	62.21	89.08	91.54	87.87	94.34	55.19	89.66
YOLOv12-L	Attention-Centric	28.75	28.75	91.94	87.31	94.02	64.34	89.57	92.50	87.82	94.74	56.37	90.10
YOLOv12-X	Attention-Centric	64.50	64.50	92.16	83.49	90.62	60.13	87.61	92.48	86.29	91.51	56.37	89.28
RF-DETR	DINOv2 ViT	33.83	33.83	87.83	93.00	95.32	63.43	90.34	92.52	91.00	96.84	59.10	91.76
LD-DETR (Ours)	DINOv2 ViT	37.28	15.31	95.37	95.00	97.79	66.81	95.18	96.30	96.00	97.42	59.28	96.15

Figure 7. Performance comparison of LD-DETR with existing instance segmentation methods on the landslide instance segmentation task.

Figure 8. Performance comparison of LD-DETR with representative object detection models on the landslide detection task.

In the landslide detection task, LD-DETR demonstrates improved performance compared with the evaluated methods, achieving a precision of 95.37%, a recall of 95.00%, an $A{P}_{50}$ of 97.79%, and an $mA{P}_{50:95}$ of 66.81%, with an overall $F_1$ score of 95.18%. Compared with Cascade Mask R-CNN, LD-DETR improves the precision by 7.81%, the $F_1$ score by 7.40%, and the $mA{P}_{50:95}$ by 13.71%. Relative to the baseline RF-DETR, the proposed method increases the precision from 87.83% to 95.37% and improves the $F_1$ score by 4.84%. Compared with the YOLO series, LD-DETR also achieves better performance on the evaluated detection metrics. When compared with YOLOv12-X, which has a larger parameter scale, LD-DETR improves recall and $F_1$ score by 11.51% and 7.57%, respectively.

In the instance segmentation task, LD-DETR achieves competitive performance among the compared methods, obtaining the highest $A{P}_{50}$ and $mA{P}_{50:95}$ values of 97.42% and 59.28%, respectively. Compared with the YOLOv12-X model, LD-DETR improves recall by 9.71%, $F_1$ score by 6.87%, and $A{P}_{50}$ by 5.91%. Compared with the baseline RF-DETR, LD-DETR achieves a slightly higher $A{P}_{50}$, increasing from 96.84% to 97.42%, while improving the $F_1$ score from 91.76% to 96.15%. The improvement in $F_1$ score is accompanied by increases in both precision by 3.78% and recall by 5.00%. LD-DETR achieves these results with 15.31 M trainable parameters (Table 4), demonstrating the parameter-efficient adaptation strategy adopted in this study. The contributions of individual components are further analyzed through the ablation experiments in Section 4.1.

Figure 9 illustrates the visual comparison between LD-DETR and the baseline RF-DETR, YOLOv12 series, and Cascade Mask R-CNN under three representative landslide scenarios, including small-scale, large-scale, and slender landslide cases. These examples are selected to qualitatively analyze the prediction behavior of different models under varying target characteristics. In small-scale landslide scenarios, where target regions occupy relatively few pixels and exhibit weak visual features, the baseline RF-DETR fails to identify the landslide instance, resulting in a missed detection. In comparison, LD-DETR successfully detects the target with a confidence score of 0.80 and generates a prediction region consistent with the landslide location. In large-scale landslide scenarios, where targets contain complex internal textures and irregular boundaries, several comparison methods produce incomplete segmentation results. In contrast, LD-DETR generates a more continuous mask with better coverage of the landslide region. In slender landslide scenarios, elongated targets with complex geometric structures may increase the difficulty of localization and segmentation. Although RF-DETR detects the target, it produces an incomplete mask boundary and a relatively low confidence score of 0.57. LD-DETR provides a more complete prediction of the elongated landslide region and achieves a confidence score of 0.89. However, these qualitative comparisons only provide visual evidence of model behavior, and the overall performance improvement is mainly supported by the quantitative results reported in the previous experiments.

Figure 9. Qualitative comparison of detection and segmentation results between LD-DETR and other models on representative test samples.

3.4. Qualitative Visualization Analysis

We employed attention visualization to qualitatively analyze the regions emphasized by the model during inference. The visualization is intended to provide an intuitive interpretation of the prediction process and to complement the quantitative experimental results presented in the previous sections. Unlike gradient-based Grad-CAM methods, the Transformer-based architecture provides attention information that can be used to qualitatively analyze feature responses. The attention weights were extracted from the last Transformer decoder layer during inference, as this layer directly interacts with the final object queries before prediction. The attention weights from different heads were averaged to obtain a unified attention map. The generated attention maps were then resized to the original image resolution and normalized using min–max normalization before being visualized as heatmaps. These heatmaps provide a qualitative interpretation of the spatial regions emphasized by the model during landslide detection. The heatmaps are used only as qualitative evidence and should be interpreted together with the quantitative results reported in the ablation and sensitivity analyses.

Representative attention visualization results are presented in Figure 10. Under complex background conditions containing vegetation textures and exposed rocks (Figure 10a), RF-DETR exhibits relatively dispersed activation responses, with some high-response regions appearing outside the landslide areas. In comparison, LD-DETR shows more concentrated responses around the annotated landslide regions, indicating that the proposed framework can produce more discriminative feature representations in such scenarios. However, the attention visualization provides only qualitative evidence of changes in feature distribution, and the quantitative improvements are mainly supported by the detection and segmentation results reported in the ablation experiments. For landslide examples at different spatial scales (Figure 10b), RF-DETR shows weaker and less concentrated responses in some small-scale or low-contrast regions, whereas LD-DETR maintains relatively consistent activation within the landslide areas. The improvement in multi-scale representation and segmentation performance is further verified by the quantitative evaluation results of the ASPP module and Tversky Loss in the ablation study. For elongated landslide examples shown in Figure 10c, RF-DETR produces fragmented activation patterns in some narrow and continuous landslide regions, while LD-DETR exhibits more continuous responses along the target structures. The observed differences are consistent with the motivation of introducing multi-scale feature aggregation and improved segmentation optimization; however, the heatmap visualization itself does not directly demonstrate the contribution of individual components. The effectiveness of each component is evaluated separately through ablation experiments, which yield quantitative results that further support the performance improvements of the proposed method. These visualization results provide an intuitive comparison of the attention patterns learned by the two models. The effectiveness of the proposed components is quantitatively evaluated through the ablation and sensitivity analyses in Section 4.1.

Figure 10. Comparison of attention heatmaps between LD-DETR and RF-DETR on representative landslide examples: (a) complex background scene; (b) landslides with different spatial scales; (c) elongated landslide morphology.

4. Discussion

4.1. Ablation Experiment

To comprehensively evaluate the effectiveness of the proposed method, a series of ablation experiments were conducted on the BLLD dataset. The ablation study includes five parts. First, a component-wise ablation study is performed to evaluate the contribution of each proposed component. Subsequently, sensitivity analyses are conducted for Tversky Loss parameters ($\alpha$ and $\beta$), the Tversky Loss weight $\lambda$, and the ASPP dilation rates. Finally, a parameter efficiency analysis is presented to evaluate the effect of the LoRA-based fine-tuning strategy.

4.1.1. Component-Wise Ablation Study

To evaluate the contribution of each proposed component, ablation experiments were conducted on the BLLD dataset. The original RF-DETR with the DINOv2 backbone and the standard Dice/BCE loss was adopted as the baseline model. Based on this baseline, orientation-aware data augmentation, LoRA-based fine-tuning, weighted Tversky Loss, and the ASPP module were introduced individually or in combination. The experimental results are summarized in Table 5.

Table 5. Ablation study of different modules on the BLLD dataset.

Model	Detection					Segmentation
	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
RF-DETR	87.83	93.00	95.32	63.43	90.34	92.52	91.00	96.84	59.10	91.76
RF-DETR + DataAug	94.59	93.00	97.51	65.52	93.79	94.12	96.00	97.19	59.15	95.05
RF-DETR + LoRA	95.83	92.00	96.47	64.38	93.88	93.75	90.00	96.09	59.02	91.84
RF-DETR + DataAugLoRA	96.48	91.00	97.48	65.41	93.66	96.48	91.00	97.63	59.76	93.66
RF-DETR + TverskyLoss	92.41	97.00	98.41	66.45	94.65	95.74	90.00	94.19	59.42	92.78
RF-DETR + ASPP	94.70	95.00	96.75	63.82	94.85	96.03	96.00	97.51	57.89	96.01
LD-DETR	95.37	95.00	97.79	66.81	95.18	96.30	96.00	97.42	59.28	96.15

The baseline RF-DETR achieved an $F_1$ score of 90.34% for object detection and 91.76% for instance segmentation. After introducing orientation-aware data augmentation alone, the detection P increased from 87.83% to 94.59%, and the detection $F_1$ score improved to 93.79%. The segmentation $F_1$ score increased from 91.76% to 95.05%. When LoRA fine-tuning was applied independently, the detection P increased to 95.83%, and the detection $F_1$ score improved to 93.88%. The segmentation performance remained comparable to the baseline, with the segmentation $F_1$ score increasing slightly from 91.76% to 91.84%. Combining orientation-aware data augmentation with LoRA resulted in additional improvements. Compared with the baseline model, the combined strategy increased the detection P from 87.83% to 96.48% and improved the detection $F_1$ score from 90.34% to 93.66%. The segmentation $F_1$ score increased to 93.66%. Replacing the standard loss with weighted Tversky Loss further improved the model performance under foreground-background imbalance. Compared with the baseline, the detection R increased from 93.00% to 97.00%, while the segmentation $F_1$ score increased from 91.76% to 92.78%. An ASPP module was subsequently introduced before the segmentation head to enhance multi-scale feature representation. Compared with the baseline, the detection $F_1$ score increased from 90.34% to 94.85%, and the segmentation $F_1$ score increased from 91.76% to 96.01%. However, the segmentation $mA{P}_{50:95}$ slightly decreased from 59.10% to 57.89%. This difference indicates that ASPP mainly improves region completeness and feature representation, as reflected in the improved segmentation $F_1$ score, while the stricter IoU-based metric remains sensitive to boundary alignment and mask localization accuracy. Therefore, the contribution of ASPP is mainly reflected in improving segmentation consistency for landslide regions with different spatial characteristics rather than uniformly increasing all evaluation metrics.

After integrating all proposed components, LD-DETR achieved the best results among the evaluated configurations. The detection $F_1$ score reached 95.18%, representing an improvement of 4.84 percentage points compared with the baseline. The segmentation $F_1$ score increased from 91.76% to 96.15%, and the segmentation $mA{P}_{50:95}$ reached 59.28%.

4.1.2. Sensitivity Analysis of Tversky Parameters $\alpha$ and $\beta$

To verify the impact of the asymmetric penalty parameters in the Tversky Loss, three parameter configurations were evaluated: $(\alpha ,\beta )=(0.4,0.6)$, $(0.3,0.7)$, and $(0.2,0.8)$, with the results shown in Table 6. As $\beta$ increases, the penalty for false negatives becomes stronger; however, increasing $\beta$ does not necessarily lead to better performance. When adopting $(0.3,0.7)$, the model obtains relatively higher values of Recall, $A{P}_{50}$, $mA{P}_{50:95}$, and $F_1$ in object detection, as well as the highest $mA{P}_{50:95}$ in instance segmentation, suggesting that this configuration provides a relatively balanced performance in detection and segmentation. Compared with $(0.3,0.7)$, the configuration $(0.4,0.6)$ provides a weaker penalty for false negatives, while $(0.2,0.8)$ improves segmentation Precision but results in decreased overall detection and segmentation performance, suggesting that assigning an excessively large penalty to false negatives may negatively affect the balance between detection and segmentation optimization. Therefore, this paper ultimately adopts $(\alpha ,\beta )=(0.3,0.7)$ as the parameter configuration for the Tversky Loss.

Table 6. Sensitivity analysis of Tversky loss parameters $\alpha$ and $\beta$.

Loss Configuration	$\mathit{\alpha }$	$\mathit{\beta }$	Detection					Segmentation
			P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
Baseline (BCE + Dice)	0.5	0.5	87.83	93.00	95.32	63.43	90.34	92.52	91.00	96.84	59.10	91.76
Tversky	0.4	0.6	95.10	89.81	95.16	61.82	92.38	95.10	89.81	94.66	59.12	92.38
Tversky	0.3	0.7	92.41	97.00	98.41	66.45	94.65	95.74	90.00	94.19	59.42	92.78
Tversky	0.2	0.8	89.72	88.89	94.52	64.58	89.30	97.06	91.67	96.34	58.46	94.29

4.1.3. Sensitivity Analysis of the Tversky Loss Weight $\lambda$

To analyze the impact of the Tversky Loss weight on model performance, under the fixed condition of $\alpha =0.3$ and $\beta =0.7$, experiments were conducted by setting $\lambda =0.5,1.0,2.0$, and $3.0$, respectively. The results are shown in Table 7. When $\lambda$ is relatively small, the contribution of the Tversky Loss is limited, which may reduce the effectiveness of learning discriminative landslide features; as the weight increases, the detection performance gradually improves. When $\lambda =2.0$, the evaluated metrics reach their highest values among the tested settings: the Recall, $A{P}_{50}$, $mA{P}_{50:95}$, and $F_1$ score for object detection all reach their optimum, while high instance segmentation accuracy is maintained. Continuing to increase $\lambda$ to 3.0 results in decreased performance in both detection and segmentation, suggesting that an excessive weight assigned to the Tversky Loss may negatively affect multi-task optimization. Therefore, this paper ultimately selects $\lambda =2.0$ as the weight coefficient for the Tversky Loss.

Table 7. Effect of different Tversky loss weights $\lambda$ on detection and segmentation performance.

Loss Configuration	$\mathit{\lambda }$	Detection					Segmentation
		P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
Baseline (BCE + Dice)	–	87.83	93.00	95.32	63.43	90.34	92.52	91.00	96.84	59.10	91.76
Tversky	0.5	91.82	93.52	96.71	63.06	92.66	96.23	94.44	97.50	59.19	95.33
Tversky	1.0	95.19	91.67	96.92	63.94	93.40	92.31	88.89	94.65	57.49	90.57
Tversky	2.0	92.41	97.00	98.41	66.45	94.65	95.74	90.00	94.19	59.42	92.78
Tversky	3.0	85.10	95.37	98.17	63.97	89.94	89.91	95.37	97.86	57.02	92.56

4.1.4. Sensitivity Analysis of ASPP Dilation Rates

To investigate the influence of ASPP dilation rates on model performance, a parameter sensitivity analysis was conducted using different dilation rate combinations, as shown in Table 8. The results demonstrate that the selection of dilation rates affects both detection and segmentation performance. Compared with the default configuration of (6, 12, 18), the configurations of (2, 4, 8) and (8, 16, 24) achieve lower overall performance. Among the evaluated configurations, (6, 12, 18) achieves relatively better overall performance and is therefore adopted as the default setting for the ASPP module in this study.

Table 8. Detection and segmentation performance under different ASPP dilation rate configurations.

ASPP Module	Dilation Rates	Detection					Segmentation
		P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
No ASPP	–	87.83	93.00	95.32	63.43	90.34	92.52	91.00	96.84	59.10	91.76
ASPP	(2, 4, 8)	90.91	92.00	92.59	60.06	91.45	89.09	90.00	91.64	48.05	89.54
ASPP	(6, 12, 18)	94.70	95.00	96.75	63.82	94.85	96.03	96.00	97.51	57.89	96.01
ASPP	(8, 16, 24)	94.34	92.00	96.31	59.66	93.16	89.57	95.00	90.98	46.03	92.20

4.1.5. Parameter Efficiency Analysis of LoRA

LoRA was introduced into the pretrained DINOv2 encoder while keeping the original encoder parameters frozen. LoRA adapters with a rank of 16 were inserted into the Query, Key, and Value projection layers of the multi-head self-attention modules, as well as the FC1 and FC2 layers of the MLP modules in all 12 Transformer blocks (Table 9). During training, only the LoRA parameters, along with the task-specific modules, were updated. Compared with full fine-tuning, introducing LoRA increased the total number of model parameters from 33.83 M to 35.05 M because of the additional adapter parameters, while reducing the proportion of trainable parameters from 100% to 37.32%. For the complete LD-DETR model, which additionally includes the ASPP module and the segmentation branch, the trainable parameter ratio was 41.07%, as listed in Table 9. The detection and segmentation performance of the LoRA-based model is presented in Section 3.3. Combined with the parameter statistics in Table 9, the results indicate that LoRA reduces the number of trainable parameters while preserving comparable detection and segmentation performance.

Table 9. Comparison of model parameters and trainable parameters for different model configurations.

Model	Encoder Frozen	Total Params (M)	Trainable Params (M)	Trainable Ratio	Rank
RF-DETR	×	33.83	33.83	100%	–
RF-DETR + LoRA	✓	35.05	13.08	37.32%	$r=16$
RF-DETR + ASPP	×	36.07	36.07	100%	–
RF-DETR + TverskyLoss	×	33.83	33.83	100%	–
LD-DETR	✓	37.28	15.31	41.07%	$r=16$

4.2. Cross-Regional Generalization Experiments

In this section, we further evaluate the generalization capability of the proposed method in cross-regional scenarios. Compared with the Bijie and Luding subsets of the BLLD dataset used for training, the external CAS-Jiuzhai dataset differs in terms of imaging sensors, spatial resolution, target scale, and landslide spatial distribution. These differences may introduce domain shift between the training and testing data, providing a more challenging scenario for evaluating the model’s adaptability to unseen regions and different remote sensing conditions.

This study further employs a limited-sample domain adaptation strategy to evaluate the cross-regional generalization capability of the proposed model. Initialized with weights pre-trained on the BLLD dataset, the model was subsequently fine-tuned on the Jiuzhai target domain dataset. To investigate the model’s adaptability under limited-annotated data conditions, 200 images were randomly selected from the Jiuzhai dataset, containing 5925 images, to form the training set, representing 3.4% of the total dataset. The remaining images were divided into a validation set and a test set containing 926 and 4799 images, corresponding to 15.6% and 81.0% of the dataset, respectively. Representative qualitative results of the LD-DETR model on the Jiuzhai test set after limited-sample adaptation are presented in Figure 11. The figure includes different landslide scenarios, such as large-scale, elongated, densely distributed, and small-scale landslides. For small-scale landslides, zoom-in views are additionally provided to facilitate a more detailed observation of the predicted instance segmentation results and their correspondence with the ground-truth annotations.

Figure 11. Qualitative results of LD-DETR on the CAS-Jiuzhai test set after target-domain adaptation. (a) Large-scale landslides, (b) elongated landslides, (c,d) densely distributed landslides in complex background areas, and (e–h) small-scale landslide examples with zoom-in views. The red boxes indicate the regions selected for detailed visualization. The corresponding enlarged views provide a detailed comparison between the predicted instance segmentation masks.

The experimental results on the Jiuzhai target-domain dataset indicate that the proposed LD-DETR maintains a certain level of cross-regional generalization capability despite differences in geographic regions and data distributions. The model is able to detect and segment landslide targets across different scales and spatial distributions. To further evaluate the effectiveness of the proposed method, LD-DETR was compared with the baseline RF-DETR model. The quantitative results show that LD-DETR improves the $mA{P}_{50:95}$ by 1.72% for object detection and 1.22% for instance segmentation compared with the baseline model, while also achieving improvements in several other evaluation metrics. The quantitative results for object detection and instance segmentation are summarized in Table 10.

Table 10. Cross-regional performance comparison between LD-DETR and RF-DETR on the CAS-Jiuzhai dataset.

Model	Detection					Segmentation
	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_1$	P	R	${\mathit{AP}}_{\mathbf{50}}$	${\mathit{mAP}}_{\mathbf{50}\mathbf{:}\mathbf{95}}$	${\mathit{F}}_{\mathbf{1}}$
RF-DETR	68.71	59.00	62.02	39.67	63.49	71.03	54.00	57.46	28.89	61.36
LD-DETR	70.00	60.00	63.34	41.39	64.62	71.36	56.00	58.88	30.11	62.75

4.3. Limitations and Future Work

Although LD-DETR achieves favorable performance in landslide detection and instance segmentation on the BLLD dataset, several limitations should be acknowledged. The training and validation data used in this study are primarily derived from the Bijie and Luding regions. Although an additional experiment on the Jiuzhaigou UAV dataset provides preliminary evidence of the model’s transferability, its generalization ability across broader geographic regions, geological conditions, and imaging scenarios has not yet been comprehensively validated. Therefore, further evaluation on more diverse regional datasets is needed before the proposed method can be considered broadly applicable.

This study primarily relies on high-resolution optical remote sensing imagery for landslide detection. Although optical imagery provides rich textural and geometric information, it is inherently limited by cloud cover, illumination variations, and adverse weather conditions. Future work could explore the multimodal fusion of optical imagery with Synthetic Aperture Radar (SAR) data and Digital Elevation Models (DEM) to fully leverage the complementary information from different sensors, thereby improving the stability and reliability of landslide identification in complex environments. The introduction of multimodal data typically increases model complexity; however, LoRA, as a parameter-efficient fine-tuning method, possesses the potential to be extended to multimodal architectures. Future studies could explore applying LoRA within a multimodal feature fusion framework by fine-tuning only the newly added modules while keeping the majority of pre-trained parameters frozen. This approach balances multimodal information fusion with parameter-efficient fine-tuning, providing a feasible strategy for the efficient adaptation of multimodal landslide detection models.

5. Conclusions

To address the detection of multi-scale landslides in complex terrains, this study proposes the LD-DETR model, which incorporates the ASPP module, Tversky Loss, and the LoRA strategy for multi-scale landslide detection and instance segmentation in remote sensing imagery. Experimental results indicate that LD-DETR achieves better overall performance than RF-DETR and the YOLO series on the Bijie and Luding datasets, with improvements observed in both precision and recall. Evaluation on the Jiuzhai landslide dataset suggests that the proposed method has a certain degree of generalization capability for cross-region landslide detection and instance segmentation. Results obtained from multiple datasets indicate that the proposed method is applicable to landslide detection tasks in complex background environments.