Automatic Detection of Forested Landslides: A Case Study in Jiuzhaigou County, China

Abstract

Landslide detection and distribution mapping are essential components of geohazard prevention. For the extremely difficult problem of automatic forested landslide detection, airborne remote sensing technologies, such as LiDAR and optical cameras, can obtain more accurate landslide monitoring data. In practice, however, airborne LiDAR data and optical images are treated independently. The complementary information of the remote sensing data from multiple sources has not been thoroughly investigated. To address this deficiency, we investigate how to use LiDAR data and optical images together to develop an automatic detection model for forested landslide detection. First, a new dataset for detecting forested landslides in the Jiuzhaigou earthquake region is compiled. LiDAR-derived DEM and hillshade maps are used to mitigate the influence of forest cover on the detection of forested landslides. Second, a new deep learning model called DemDet is proposed for the automatic detection of forested landslides. In the feature extraction component of DemDet, a self-supervised learning module is proposed for extracting geometric features from LiDAR-derived DEM. Additionally, a transformer-based deep neural network is proposed for identifying landslides from hillshade maps and optical images. In the data fusion component of DemDet, an attention-based neural network is proposed to combine DEM, hillshade, and optical images. DemDet is able to extract key features from hillshade images, optical images, and DEM, as demonstrated by experimental results on the proposed dataset. In comparison to ResUNet, LandsNet, HRNet, MLP, and SegFormer, DemDet obtains the highest mean accuracy, mIoU, and F1 values, namely 0.95, 0.67, and 0.777. DemDet is therefore capable of autonomously identifying the forest-covered landslides in the Jiuzhaigou earthquake zone. The results of landslide detection mapping reveal that slopes along roads and seismogenic faults are the most crucial areas requiring geohazard prevention.

1. Introduction

Landslide disasters have been the object of attention of researchers worldwide. Vegetation typically conceals landslides in the mountainous regions of western China. For instance, the Ms7.0 Jiuzhaigou earthquake on 8 August 2017 caused thousands of landslides in Jiuzhaigou County, which is a popular tourist destination in southwest China. The majority of landslides are forested, indicating that forested landslides have a significant impact on the local ecological environment by affecting the hillside and forest. In images derived from optical remote sensing, forested landslides are typically obscured by vegetation. Ways of identifying forested landslides and investigating the effects of forested landslides on the local ecological environment are enduring concerns.

Generally, costly and time-consuming manual field surveys are required to detect forested landslides. Nevertheless, landslides are influenced by a number of factors, including climate change [1], groundwater level [2], soil property [3], and abrupt rainstorms [4]. Can we autonomously identify forested landslides [5]? Extensive research has been conducted on automatic landslide detection using remote sensing data. Existing landslide detection models can be divided into four categories based on the remote sensing data sources: optical image, light detection and ranging (LiDAR), interferometric synthetic aperture radar (InSAR), and multimodal-based models [6,7,8]. The following is a concise overview of these models.

The images captured by optical remote sensing are highly visible. Historically, geohazard specialists have relied primarily on optical remote sensing images to identify landslides. Baum et al. [9] introduced surface deformation in order to distinguish slow-moving landslides. Ge et al. [10] constructed an emergency landslide detection network using CenterNet and ResNet50. Yu et al. [11] proposed a neural network for landslide detection with multi-scale landslide features and deconvolution. The proposed model utilized multiple spatial resolution remote sensing images. Two enhanced YOLOX (Exceeding You Only Look Once) methods for image-based remote sensing landslide detection have been proposed [12,13]. In the meantime, Saba et al. [14] compared pixel, sub-pixel, and object-based image analysis techniques for the detection of coseismic landslides. DeepLabv3+ was used by Morales et al. [15] to identify landslides from Sentinel-2 optical images. Das and Wegmann [16] evaluated artificial neural network, random forest, and support vector machine techniques for landslide detection across satellite sensors using an artificial neural network, a random forest, and a support vector machine. Fu et al. [17] proposed an enhanced Mask R-CNN (Mask Region-Based Convolutional Neural Network) algorithm for detecting seismic landslides. Attention-based deep neural networks were used by Amankwah et al. [18] to identify landslides from bitemporal satellite imagery. Yang and Xu [19] identified earthquake-induced landslides in Hokkaido using U-Net++. Existing models for automatic landslide detection rely heavily on optical remote sensing images. Due to the fact that forest cover obscures landslides in optical images, these models are not suitable for detecting landslides in forested areas.

Light detection and ranging is an increasingly essential technology for identifying forested landslides in LiDAR-based models. Alizadeh et al. [20] proposed a hybrid analytic network process and artificial neural network model for assessing the urban earthquake hazard. Nikolakopoulos et al. [21] proposed the acquisition geometry of an unmanned aerial vehicle for landslide mapping and monitoring. Syzdykbayev et al. [22] utilized persistent homology to detect landslides from LiDAR data. Fang et al. [23] proposed an enhanced U-Net model for the detection of historical landslides. Azmoon et al. [24] used high-resolution multi-temporal digital elevation model (DEM) data to detect landslides. Cai et al. [25] detected slow-moving landslides using satellite SAR observations and an airborne LiDAR digital surface model. Xu et al. [26] studied landslide displacements using InSAR and airborne LiDAR. Liu et al. [27] utilized LiDAR to detect landslides caused by earthquakes. Mezaal et al. [28] detected landslides using Dempster–Shafer and airborne LiDAR. The efficacy of the deep learning models proposed by Huang et al. [29], Huang et al. [30] for predicting the susceptibility of landslides is highly significant. There are multiple models that use LiDAR to detect landslides. Unresolved is the question of how to construct new models to make optimal use of LiDAR data for landslide detection in forested areas.

Synthetic aperture radar interferometry is widely used to monitor the deformation of active landslides for InSAR-based models. Calo et al. [31] used ground-monitoring and SAR interferometric techniques to assess the activity of a large landslide in southern Italy. Crosetto et al. [32] used interferometric SAR for monitoring the Vallcebre landslide. Wasowski and Bovenga [33] investigated the issues of investigating landslides and unstable slopes with satellite multi-temporal interferometry. Chen et al. [34] proposed a semantic segmentation model with InSAR to identify active landslides. Fu et al. [35] utilized YOLOv3 and InSAR phase-gradient layering to detect landslides that moved slowly. Nava et al. [36] used a method of image classification based on deep learning to enhance landslide detection. Nava et al. [37] identified landslides using attention U-Net and SAR data. Hussain et al. [38] utilized InSAR time series for landslide detection and inventory update. Sentinel-1 data and InSAR technology were utilized by Dai et al. [39] to detect active landslides. Existing InSAR-based landslide detection relies heavily on satellite SAR, which is susceptible to geometric distortion in high mountain areas.

With regard to the multimodal fusion models, optical remote sensing and radar are used to create landslide mapping [7,40]. Ganerød et al. [41] used five existing machine learning models with Sentinel-1, Sentinel-2, DEM, and slope images to detect landslides. Bhuyan et al. [42] proposed a transfer learning strategy with attention U-Net to generate a landslide inventory map from remote sensing images. Jin et al. [43] proposed multisource data fusion analysis for identifying landslides. The data sources are DEM, geological mapping data, and river distribution data. Landslide4Sense is a publicly accessible landslide dataset created by [44]. Various prominent landslide detection models were evaluated using the dataset. However, multimodal deep learning has been a prominent research direction in the deep learning community. Since the translation of images, audio, and natural language is of considerable interest, numerous multimodal methodologies for deep learning have been proposed. Stable Diffusion [45], DALL-E [46], and CCR-Net [47] are some examples. Several sources provide a comprehensive review of multimodal deep learning techniques [48,49,50]. New multimodal neural networks have tremendous potential for enhancing landslide detection performance.

In addition, little research has been conducted on the automatic detection of forested landslides. Object-oriented image analysis methods [51,52,53,54], random forest [55], artificial neural networks Gorsevski et al. [56], Dempster–Shafer [28], and persistent homology [22] methods have been used to identify forested landslides from LiDAR data. Traditional models based on machine learning rely on manual feature engineering. It is intriguing to investigate deep learning-based automatic feature extraction techniques, such as transformer and self-supervised learning, for forested landslide detection.

In conclusion, numerous landslide detection algorithms rely on remote sensing images that are severely impacted by cloud and vegetation cover. Automatic multimodal-based landslide detection research is still in its inception. For improved landslide detection, an effective multimodal fusion method must be developed. Many extant deep learning-based methods use convolutional neural networks (CNN) as their network architecture. It is beneficial to investigate new deep learning models for improved landslide detection [44]. The disparity between forested landslide detection and multimodal deep learning is substantial.

This paper proposes a new deep learning model for detecting forested landslides to address this deficiency. The LiDAR-derived hillshade is used to eliminate the effect of forest cover. In addition, the new multimodal deep learning model utilizes hillshade maps, optical images, and DEM to simultaneously identify forested landslides. The proposed model generates a landslide mapping that identifies the most important areas for landslide monitoring and ecological environmental protection.

2. Materials and Methods

2.1. Introduction to Study Area

Figure 1 depicts the study area, which is situated in Jiuzhaigou County, Sichuan Province, China. This region has dense vegetation and a complex topography. Several landslides are dispersed across Jiuzhaigou [57]. A significant proportion of the landslides are covered by vegetation. Airborne LiDAR is used to collect point cloud data, which are able to remove vegetation cover. The LiDAR system is Leica ALS80-HP. Its viewing angle is 60${}^{\circ }$; wavelength is 1550 nm; scan frequency is 50 to 550 KHz; focal length is 45 mm; and the density of the point cloud is greater than 50 points per square meter. The spatial resolution of the LiDAR data collected is 0.5 m. Based on these newly collected data, the landslide database of the study area is updated by geological specialists.

The DEM is derived using the Global Mapper software from LiDAR data. DEM contains precise altitude data with a wealth of 3D geometric features, including points, lines, faces, directions, gradients, and curvatures. DEM can be used to extract landslide detection features, such as slope, aspect, and curvature. A new deep learning model is proposed to autonomously extract geometric features from DEM. Additionally, the LiDAR-derived DEM is used to generate hillshade maps obtained by computer-simulated sunlight shining on mountains. Each pixel of the hillshade represents the intensity of the light at that location. The configurations for generating hillshade in the Global Mapper software are as follows: the number of light sources is 12, and the altitude is 30${}^{\circ }$, thereby reflecting the surface of forested landslides. In hillshade maps, the texture and boundary of forested landslides become visible.

In order to make a fair comparison, the remote sensing data are separated into two distinct portions, as depicted in Figure 1. The area bounded by the yellow polygon is the testing area, while the remaining area is the training area. The seven channels of the remote sensing data represent DEM, hillshade, and optical images, respectively.

2.2. A Dataset for Forested Landslide Detection

There are no open datasets for forested landslide detection. This paper proposes a new dataset for detecting forested landslides. The detailed procedure is described below. Initially, the dataset comprises LiDAR and optical images from the Jiuzhaigou earthquake zone. Due to the issue of vegetation cover, the extant landslide mapping results from Tang et al. [58] are revised based on the new data. The result of this paper’s landslide mapping includes 3245 landslides, including 1487 new landslides and 1758 ancient landslides. To develop deep learning-based landslide detection models, the input data should be separated into training and testing sets. In certain instances, random division can result in overfitting Tang et al. [58]. To circumvent this issue, the study area has been meticulously divided into two distinct training/testing areas.

The original remote sensing data are too large to be incorporated into the models for landslide detection. Consequently, they are separated into tiny patches. Nevertheless, the conventional grid clipping procedure will destroy some landslides. To address this issue, the center clip approach for protecting landslides is introduced. For each landslide within the training area, a 1024 × 1024 image patch is extracted. Noting that the patch center is identical to the landslide, each landslide is entirely preserved in at least one patch. Finally, a novel dataset for forested landslide detection is efficiently compiled. The number of patches in the training set is 1279, and the number of patches in the testing set is 354, while the train/test ratio is approximately 80% to 20%.

2.3. A New Deep Learning Model for Forested Landslide Detection

This study proposes DEM and hillshade-based landslide detection (DemDet), a new deep neural network for detecting forested landslides. A geometric feature extraction module for DEM, a transformer-based encoder for hillshade and optical image feature extraction, and a decoder for multimodal feature fusion are depicted in Figure 2. This subsection continues with additional information.

2.3.1. Geometric Feature Extraction Network for DEM

DEM contains abundant 3D geometric features that have the potential to enhance the precision of landslide detection. To autonomously extract their features, a geometric feature extraction is proposed. The structure of the geometric feature extraction network is depicted in Figure 2.

(1) ResNet extracts low-level raw features of the DEM. Every residual module is made up of two 3 × 3 convolutions. An equal width convolution is introduced at the boundaries of the input DEM to prevent the loss of boundary dimensions. A skip connection is used to circumvent the DemDet gradient vanishing problem.

(2) Deformation convolution is implemented to derive more complex features. The standard convolution reception field is a rectangle, which is not the geometry of many ground objects. In contrast, deformation convolution learns a pixel shift to construct a reception field that is irregular and closer to the geometry of ground objects.

(3) A 1 × 1 convolution that is simple yet effective is used to predict the labels of the input DEM. In DEM, it is difficult to annotate landslides. Due to the poor visibility of DEM, this study utilizes some well-known DEM-derived landslide detection features, such as slope, aspect, and curvature, to guide the geometric feature extraction model. To our knowledge, this is a novel method for explicitly incorporating geohazard domain knowledge into a deep learning model.

The definitions of slope, aspect, and curvature [59] are shown in Equations (1) to (6), where A is a small DEM patch of size 3 × 3; $c_x$ and $c_y$ are the size of the pixel; $d_x$ and $d_y$ are the horizontal and vertical shift of the central pixel of A; and $d_{xx},d_{yy}$, and $d_{xy}$ are the horizontal shift of $d_x$, horizontal shift of $d_y$, and vertical shift of $d_y$, respectively.

$$A=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right]$$

(1)

$$d_x=\frac{(a_{13}+2a_{23}+a_{33})-(a_{11}+2a_{21}+a_{31})}{8c_x}$$

(2)

$$d_y=\frac{(a_{31}+2a_{32}+a_{33})-(a_{11}+2a_{12}+a_{13})}{8c_y}$$

(3)

$$Y_{slope}=\frac{180}{\pi }arctan\left(\sqrt{d_x^2+d_y^2}\right)$$

(4)

$$Y_{aspect}=\left\{\begin{array}{c}\frac{180}{\pi }arctan(-\frac{d_y}{d_x}),d_x>0\hfill \\ \frac{180}{\pi }arctan(-\frac{d_y}{d_x}+\pi ),d_y\ge 0,d_x<0\hfill \\ \frac{180}{\pi }arctan(-\frac{d_y}{d_x}-\pi ),d_y<0,d_x<0\hfill \\ \frac{\pi }{2},d_y>0,d_x=0\hfill \\ -\frac{\pi }{2},d_y<0,d_x=0\hfill \end{array}\right.$$

(5)

$$Y_{curv}=-\frac{d_{xx}{d}_x^2+2d_{xy}{d}_x{d}_y+d_{yy}{d}_y^2}{(d_x^2+d_y^2){(d_x^2+d_y^2+1)}^{\frac{3}{2}}}$$

(6)

2.3.2. Hillshade and Optical Image Feature Extraction Network

Images of hillshade can be regarded as a type of optical image devoid of forest coverage. The study area’s hillshade map is a massive image. Each pixel is strongly dependent on its neighbors. The transformer is able to discover long-term pixel dependencies within an image patch. In comparison to the convolutional neural network, the reception field of the transformer is greater. The encoder of the well-known transformer-based semantic segmentation model SegFormer [60] is introduced to automatically extract image features from hillshade and optical images.

SegFormer is a transformer-based semantic segmentation model comprised of an encoder and a decoder. Tang et al. [58] utilized SegFormer to effectively identify coseismic landslides. DemDet utilizes SegFormer’s encoder to derive features from hillshade and optical images. The SegFormer encoder is made up of transformer blocks, which are composed of three components: patch embedding, self-attention, and position embedding. From the input data, the patch embedding module extracts feature maps. The position embedding module uses the remote sensing data’s geological position information. There are additional details available in references [58,60].

2.3.3. Decoder for Multimodal Feature Fusion

The combination of DEM and hillshade or optical images produces a 3D terrain model. Different remote sensing data sources provide complementary information for the detection of landslides. To maximize the utility of DEM, hillshade, and optical images, it is necessary to combine multimodal data. However, the majority of existing methods for landslide detection treat multi-source remote sensing data independently. This paper proposes a new decoder that combines multimodal features, such as DEM, hillshade, and optical image features. Figure 2 shows the structure of the proposed decoder.

(1) Concatenation of multimodal features and global pooling. The concat operation is used to consolidate DEM, hillshade, and optical image feature maps. A total of 768 channels represent hillshade features, 64 channels represent DEM features, and the remaining 768 channels represent optical features.

(2) Channel attention mechanism. The channel attention mechanism, which consists of two stages, is used to learn an importance factor for each channel. Prior to calculating the channel importance factor, the values of each channel are aggregated using a global aggregating operation. Second, a multiplication operation is used to designate each channel’s channel importance factor.

(3) Special attention mechanism. The use of spatial attention increases the probability of landslide pixels while decreasing the probability of non-landslide pixels. Horizontal attention and vertical attention are used to learn a correlation factor from the horizontal and vertical directions of the feature map. An average pooling operation, a convolution layer, and a sigmoid function are used to implement the spatial attention mechanism.

2.3.4. Loss Function

The loss function of the DemDet model is composed of two parts: DEM loss and landslide classification loss. The definition is shown in Equation (7):

$$\mathcal{L}={\mathcal{L}}_{dem}+{\mathcal{L}}_{cls}.$$

(7)

(1) Loss function of geometric feature extraction model for DEM. The loss function of the geometric feature extraction model ${\mathcal{L}}_{dem}$ consists of three parts, as shown in Equation (8), where ${\mathcal{L}}_{slope},{\mathcal{L}}_{aspect},$ and ${\mathcal{L}}_{curv}$ are the mean square error (MSE) between slope, aspect, and curvature, with their predictions. The definitions of ${\mathcal{L}}_{slope},{\mathcal{L}}_{aspect},$ and ${\mathcal{L}}_{curv}$ are shown in Equations (9) to (11). H and W are the height and width of the DEM. The labels of the DEM are slope, aspect, and curvature, which are calculated using Equations (1) to (6).

$${\mathcal{L}}_{dem}={\mathcal{L}}_{slope}+{\mathcal{L}}_{aspect}+{\mathcal{L}}_{curv}$$

(8)

$${\mathcal{L}}_{slope}=\frac{1}{HW}\sum _{i=1}^H\sum _{j=1}^W{(Y_{slope}(i,j)-{\widehat{Y}}_{slope}(i,j))}^2$$

(9)

$${\mathcal{L}}_{aspect}=\frac{1}{HW}\sum _{i=1}^H\sum _{j=1}^W{(Y_{aspect}(i,j)-{\widehat{Y}}_{aspect}(i,j))}^2$$

(10)

$${\mathcal{L}}_{curv}=\frac{1}{HW}\sum _{i=1}^H\sum _{j=1}^W{(Y_{curv}(i,j)-{\widehat{Y}}_{curv}(i,j))}^2$$

(11)

(2) Loss function of landslide classification. The loss function of the landslide classification is the cross-entropy between labels and predictions. Equation (12) shows the definition of cross-entropy, where $\delta$ is the Softmax function:

$${\mathcal{L}}_{cls}=\frac{1}{HW}\sum _{i=1}^H\sum _{j=1}^W{Y}_{cls}(i,j)log\left(\delta \left({\widehat{Y}}_{cls}(i,j)\right)\right).$$

(12)

Mean intersection over union (mIoU), precision, recall, F1, and accuracy are also used to evaluate the landslide detection models. mIoU is the ratio between the overlapped area and union of the labels. These metric definitions can be found in reference [58].

2.3.5. A Two-Stage Training Strategy

Algorithm 1 depicts the DemDet training and testing procedure. DemDet’s training procedure consists of two stages: feature extraction and multimodal fusion. First, feature extraction is accomplished. In order to train an image feature extraction branch, optical images are utilized. Then, we use hillshade images to train a second image feature extraction branch. In order to train the geometric feature extraction module, DEM data are utilized. Then, the trained feature extraction branch parameters are serialized to persistent storage. Second, we deal with multimodal integration by loading and correcting the branches of feature extraction. Then, we use the multimodal input data to train the DemDet decoder. In other words, the decoder module performs multimodal fusion at the feature level.

Algorithm 1: The training and test process of landslide detection using DemDet.

Input:  Optical images, hillshade images, DEM, and their labels. Output:  Pixel-wise landslide prediction results. 1: #Train the image feature extraction and geometric feature extraction modules. 2: Initialize the image feature extraction module of DemDet by the pre-trained SegFormer model. 3: Fine-tune the image feature extraction model using the optical and hillshade images. Then, the trained model is serialized to persistent storage. 4: Train the geometric feature extraction module using DEM data. Then, the trained model is also serialized to persistent storage. 5: #Train the decoder of DemDet 6: for $i=1$ to 10,000 do 7:    Randomly choose a batch of training data, i.e., two groups of optical images, hillshade images, and DEMs. 8:    Load the pre-trained parameters of the image feature extraction and geometric feature extraction modules. Then, fix the parameters of these two modules. 9:    Conduct forward process of the image feature extraction and geometric feature extraction modules. 10:    Train the decoder of DemDet. 11:    Calculate the loss function of DemDet using Equation (7). 12:    Conduct back-propagation. 13:    Update the parameters of DemDet. 14: end for 15: #Test 16: Use the trained DemDet model to predict landslides in the test images.

3. Results

Numerous experiments were performed to demonstrate the efficacy and superiority of DemDet. Detailed experimental configurations and outcomes are displayed below.

3.1. Experimental Configurations

The experimental apparatus consists of a deep learning server with an NVIDIA A100 80 GB GPU, an Intel Xeron Gold 5218 CPU, and 128 GB memory. All the methods are implemented in PaddleSeg, an open source deep learning package (https://github.com/PaddlePaddle/PaddleSeg, accessed on 1 June 2023). Four well-known landslide detection models, HRNet [61], SegFormer [60], ResUNet [62], and LandsNet [63], are compared. Additionally, a shallow model MLP (multi-layer perceptron) [64] is also compared. The batch size and number of iterations are 2 and 10,000, respectively. The DemDet training procedure is depicted in Algorithm 1.

3.2. Evaluation of Accuracy

DemDet is a multimodal deep learning model that identifies forested landslides using hillshade images, optical images, and DEM. It is necessary to examine the landslide detection precision of each individual remote sensing data source.

3.2.1. The Accuracy of Single Optical Images

The experimental outcomes are presented in

Table 1. Comparison of landslide detection accuracy using optical images.

Model	Class	IoU	Precision	Recall	F1
MLP	background	0.909	0.944	0.936	0.939
	landslide	0.020	0.061	0.036	0.045
ResUNet	background	0.884	0.944	0.933	0.938
	landslide	0.035	0.063	0.075	0.068
LandsNet	background	0.943	0.945	0.978	0.962
	landslide	0.041	0.137	0.056	0.080
HRNet	background	0.922	0.952	0.967	0.959
	landslide	0.117	0.247	0.182	0.209
SegFormer	background	0.930	0.953	0.975	0.964
	landslide	0.124	0.314	0.189	0.236

, which illustrates the accuracy of identifying landslides and backgrounds in relation to all comparison techniques. SegFormer obtains the highest performance on all evaluation metrics for the landslide class, with IoU and F1 values of 0.124 and 0.236, respectively. HRNet has the second greatest IoU and F1 scores. SegFormer increases HRNet’s mIoU and F1 by 5.9% and 12.0%, respectively. Despite this, the mIoU and F1 of SegFormer with optical images continue to be low. A probable explanation is that optical remote sensing images cannot detect the majority of forested landslides. Therefore, it is necessary to identify alternative data sources to resolve the forest cover problem.

3.2.2. The Accuracy of Single DEM-Derived Hillshade Images

Table 2. Comparison of landslide detection accuracy using DEM-derived hillshade images.

Model	Class	IoU	Precision	Recall	F1
MLP	background	0.906	0.934	0.943	0.938
	landslide	0.122	0.160	0.133	0.145
ResUNet	background	0.932	0.963	0.963	0.964
	landslide	0.242	0.368	0.380	0.390
LandsNet	background	0.938	0.963	0.973	0.968
	landslide	0.259	0.405	0.376	0.423
HRNet	background	0.904	0.985	0.917	0.950
	landslide	0.320	0.455	0.468	0.461
SegFormer	background	0.934	0.974	0.958	0.966
	landslide	0.336	0.450	0.570	0.503

displays the per-class accuracy of all comparison models for DEM-derived hillshade images as the data source. SegFormer obtains the highest IoU, Recall, and F1 for the landslide class, namely 0.366, 0.57, and 0.503, respectively. HRNet has the second-highest level of precision. Compared to HRNet, SegFormer increases IoU and F1 by 5% and 9.1%, respectively.

According to Table 1 and Table 2, hillshade images increase the landslide detection accuracy of optical images by 170%. It is intriguing that hillshade images detect forested landslides significantly better than optical images. The reason is that DEM, which is derived from LiDAR, is used to obtain hillshade. LiDAR is capable of penetrating through vegetation. Additionally, SegFormer obtains the highest degree of precision on both optical and hillshade images. DemDet therefore uses SegFormer to simultaneously extract features from optical images and hillshade images.

3.2.3. Effectiveness of the Geometric Feature Extraction Module for DEM

To visualize the effectiveness of the geometric feature extraction module, Figure 3 depicts the mean square error curves of slope, aspect, and curvature on the test dataset. As interactions increase, the MSE of slope, aspect, and curvature decreases steadily. Minimum MSE values are 0.18, 0.54, and 0.62, respectively. Consequently, the geometric feature extraction module typically converges to the geometric labels’ ground truth. The extracted features are a superset of slope, aspect, and curvature that includes novel DEM features. The geometric feature extraction module effectively incorporates domain knowledge into the deep learning model, providing a general method for incorporating domain knowledge into deep learning models.

3.3. Comparing the Mean Accuracy of Landslide Detection Methods

Table 3. Comparison of average landslide detection accuracy.

Model	Data Source	mIoU	Precision	Recall	Accuracy	F1
MLP	Optical	0.464	0.502	0.486	0.892	0.492
	Hillshade	0.514	0.547	0.538	0.916	0.541
ResUNet	Optical	0.460	0.503	0.504	0.885	0.503
	Hillshade	0.587	0.665	0.671	0.939	0.677
LandsNet	Optical	0.484	0.541	0.517	0.927	0.521
	Hillshade	0.598	0.684	0.674	0.938	0.695
HRNet	Optical	0.519	0.599	0.574	0.923	0.584
	Hillshade	0.612	0.720	0.692	0.929	0.705
SegFormer	Optical	0.527	0.634	0.582	0.931	0.600
	Hillshade	0.635	0.712	0.764	0.937	0.735
SegFormer	Hillshade+DEM	0.659	0.763	0.752	0.949	0.757
DemDet	Optical+Hillshade+DEM	0.676	0.800	0.751	0.955	0.773

displays the mean accuracy values for each model of landslide detection. For each comparison method, the mean value of all optical image metrics and hillshade image metrics is determined. In addition, the proposed DemDet method uses optical images, hillshade images, and DEM as its data source. SegFormer is the backbone network of DemDet. All evaluation metrics of hillshade images are significantly superior to those of optical images, indicating that hillshade images effectively remove a significant amount of vegetation from the surface of landslides. DemDet attains the maximum mIoU, precision, accuracy, and F1 values, namely 0.676, 0.8, 0.955, and 0.773. SegFormer, which uses hillshade images and DEM, is the second-best method. DemDet enhances mIoU and F1 by 2.5% and 2.1% over SegFormer. A plausible explanation is that DEM provides DemDet with crucial geometric features. DemDet effectively identifies forested landslides using multimodal data (i.e., hillshade images, optical images, and DEM).

3.4. Visualization Results

The predicted landslides of the test area are depicted in Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 to facilitate a visual comparison of DemDet for various data sources. Figure 4 depicts the results of visualizing randomly selected image segments. Forested landslides are identified by the white areas in these images. The initial column displays manually annotated landslides. The remaining columns display the results of DemDet’s detection of forested landslides using optical images, hillshade images, and “optical+hillshade+DEM”, respectively. Figure 8’s second column demonstrates that many landslide components cannot be distinguished in optical images. Compared to the second column, the majority of pixels corresponding to landslides can be identified in hillshade images. Few environments are incorrectly identified as landslides. From the fourth column, it is clear that DemDet with all three data sources obtains the best visualization results. Specifically, the landslide boundary is more distinct, and the number of incorrectly identified landslides is reduced. In conclusion, the visualization results indicate that hillshade images are superior to optical images for identifying forested landslides. DemDet makes extensive use of hillshade images, optical images, and DEM for detecting forested landslides.

Figure 5, Figure 6, Figure 7 and Figure 8 illustrate the entire test location. The red polygons are the boundaries of landslides. Figure 5 depicts the labels of forested landslides, while Figure 6 depicts the results of DemDet’s optical image visualization. Optical remote sensing images do not reveal the majority of the landslides in the forest. Figure 7 depicts the DemDet visualization results with hillshade images. The majority of forested landslides can be correctly identified from high-resolution hillshade images, but there are some false positives, such as bare rocks and rockfalls. Figure 8 depicts the results of DemDet’s visualization with hillshade images, optical images, and DEM. The majority of forest-based landslides have been appropriately identified. Additionally, DEM assists DemDet in removing the majority of false positives. Some landslides’ boundaries are inaccurate. A possible reason is that human activities change the features of landslides.

4. Discussion

4.1. Multimodal Landslide Detection Model

Four aspects of the proposed multimodal landslide detection model are discussed, including the data for forested landslide detection, the efficacy of hillshade and DEM, and multimodal landslide monitoring data fusion.

Firstly, automatic detection of forested landslides has been an issue for a very long time. The majority of existing landslide detection models, such as ResUNet and LandsNet, use remote sensing images to identify landslides [65]. The experimental results presented in Table 1 demonstrate that the performance of landslide detection models based on a single optical image is weak. Forested landslides necessitate alternative strategies to resolve the forest cover issue. We observed that LiDAR technology is capable of removing a substantial portion of forest cover [26]. This study uses LiDAR data to construct a model for detecting forested landslides. The question of how to maximize the use of LiDAR data for landslide detection in forested areas is still an unresolved issue [22]. To address this issue, LiDAR-derived DEM and hillshade are utilized to construct a model for detecting forested landslides.

Secondly, this study employs hillshade maps derived from LiDAR for the automatic detection of forested landslides. The use of hillshade to identify solitary landslides has been the subject of some research [25]. The primary benefit of the proposed model is that it can autonomously identify forested landslides from hillshade maps, thereby meeting the requirements for large-scale landslide detection. The results presented in Table 1 and Table 2 demonstrate that the accuracy of autonomous landslide detection models is significantly greater than that of optical images; specifically, the accuracy of hillshade-based models is 1.7 times greater than that of optical image-based models.

Thirdly, the DEM’s efficacy in landslide detection is contested. Ghorbanzadeh et al. [66] found that DEM does not enhance the accuracy of landslide detection models. This study proposes a geometric feature extraction network for DEM data to address this issue. The experimental outcomes presented in Table 2 and Table 3 demonstrate the efficacy of DEM and the geometric feature extraction network. Slope, aspect, and curvature are introduced to force the DEM feature network to acquire crucial geometric characteristics. In other words, this is a general method for incorporating geohazard domain expertise into deep learning models. In addition, a two-stage training strategy for the DEM feature extraction network was proposed in this study. This tactic also enhances the efficacy of DEM.

Lastly, different forms of landslide monitoring data provide complementary information for landslide detection. Existing landslide detection models rely on a singular data source type. The question of how to maximize the use of data from multiple sources remains unanswered [41]. This study proposes a multimodal fusion network for merging optical images, hillshade images, and DEM, which conducts feature-level fusion of multimodal remote sensing data, as a solution to the aforementioned problem. The experimental results presented in Table 3 demonstrate that DemDet with multimodal data obtains the highest level of precision.

4.2. Future Studies

Future research should investigate new models and interpretations for the detection of forested landslides. Firstly, DEM and hillshade are extracted from the LiDAR point cloud. Hopefully, forested landslides could be identified directly from raw LiDAR data. In addition, deep learning models are poorly explicable. Considering the grave hazard that landslides pose to human lives and infrastructures, the decision basis and conditions of landslide detection models must be determined.

Secondly, the aim of automatic landslide detection is to identify the landslides’ boundaries as accurately as possible. It is interesting to explore new landslide detection models, which gives more accurate prediction for landslide boundaries.

Lastly, landslides devastate the catastrophe area’s geological environment. The results of landslide detection and mapping reveal the areas that are important for landslide protection. To safeguard the geological environment more effectively, new technologies for landslide monitoring and protection should be investigated.

5. Conclusions

This study investigated the significant issue of how to autonomously identify forested landslides. A new dataset based on LiDAR and optical images was developed. LiDAR-derived hillshade maps were then incorporated into the model for landslide detection. Compared to optical images, hillshade images reduced the impact of forest cover on landslides by a significant amount. DemDet also designed two novel neural networks, a geometric feature extraction network for DEM and a multimodal fusion network for integrating DEM, hillshade images, and optical images, in order to maximize the use of multimodal data. Extensive experimental results demonstrated that multimodal data can enhance forested landslide detection performance. The accuracy of landslide detection using hillshade is 1.7 times greater than that of using optical images. The results of landslide detection and mapping indicate that the geologically significant areas are located on slopes close to roads and seismogenic faults.