Hybrid Integration of Bagging and Decision Tree Algorithms for Landslide Susceptibility Mapping

Abstract

Landslides represent a significant global natural hazard, threatening human settlements and the natural environment. The primary objective of the study was to develop a landslide susceptibility modeling approach that enhances prediction accuracy and informs land-use planning decisions. The study utilized a hybrid ensemble-based methodology to improve prediction accuracy and effectively capture the complexity of landslide susceptibility patterns. This approach harnessed the power of ensemble models, employing a bagging algorithm with base learners, including the reduced error pruning decision tree (REPTree) and functional tree (FT) models. Ensemble models are particularly valuable because they combine the strengths of multiple models, enhancing the overall performance and robustness of the landslide susceptibility prediction. The study focused on Yanchuan County, situated within the hilly and gully region of China’s Loess Plateau, known for its susceptibility to landslides, using sixteen critical landslide conditioning factors, encompassing topographic, environmental, and geospatial variables, namely elevation, slope, aspect, proximity to rivers and roads, rainfall, the normalized difference vegetation index, soil composition, land use, and more. Model performances were evaluated and verified using a range of metrics, including receiver operating characteristic (ROC) curves, trade-off statistical metrics, and chi-square analysis. The results demonstrated the superiority of the integrated models, particularly the bagging FT (BFT) model, in accurately predicting landslide susceptibility, as evidenced by its high area under the curve area (AUC) value (0.895), compared to the other models. The model excelled in both positive predictive rate (0.847) and negative predictive rate (0.886), indicating its efficacy in identifying landslide and non-landslide areas and also in the F-score metric with a value of 0.869. The study contributes to the field of landslide risk assessment, offering a significant investigation tool for managing and mitigating landslide hazards in Yanchuan County and similar regions worldwide.

1. Introduction

Landslides represent a significant natural disaster phenomenon triggered by various mechanisms within diverse geomorphic contexts, such as heavy rainfall and seismic activity, often with complex and poorly understood causes. The consequences of landslides pose substantial threats to human civilization, resulting in immeasurable damage and impacting economic development. In many countries, landslides are a leading cause of injuries and economic losses [1]. China faces annual geological disasters related to landslides that cannot be underestimated, leading to risks to human life and property and the depletion of natural resources [2,3,4]. Therefore, it is evident that effective monitoring and prediction of landslides are imperative to minimize losses and mitigate the aftermath of these disasters. Various methods exist for predicting and preventing landslides, with landslide susceptibility mapping emerging as a valuable strategy to mitigate landslide-related damage. Landslide susceptibility refers to the assessment of the likelihood or probability of landslides occurring in a particular area based on geological, seismo–tectonic, and environmental factors. While our models incorporate multiple variables to predict landslide susceptibility accurately, the spatial predictions are subject to uncertainties due to potential data quality variations and the dynamic nature of the environmental factors. Acknowledging these uncertainties is vital for carefully interpreting the models’ outputs, especially when applied to land-use planning and risk management decisions. Future research should focus on improving data quality, exploring more sophisticated modeling techniques, and incorporating uncertainty analysis methods, such as Monte Carlo simulations, to quantify and reduce these uncertainties, thereby enhancing the reliability of spatial predictions [5,6,7].

In recent years, there has been a growing focus among experts on landslide susceptibility analysis [8,9,10,11]. Geographic information systems (GIS), recognized as an essential tool for landslide susceptibility mapping, have gained widespread popularity in spatial modeling for landslides [12]. With the advancements in GIS and remote sensing (RS) technologies, a significant amount of data has become available, while the development of sophisticated algorithms and the increased computational power have opened up numerous opportunities for research and applications in various fields [8,13]. Machine learning techniques for predicting landslide susceptibility have been steadily replacing traditional statistical methods, a trend that has gained significant attention from experts and scholars alike [6,14,15]. When assessing landslide susceptibility, quantitative analysis has emerged as a more practical, objective, and favored approach [16,17]. Machine learning algorithms have proven invaluable by mitigating the limitations of subjective assumptions, offering enhanced accuracy and effectiveness compared to expert-based theoretical models and statistical analyses [18,19]. Machine learning methodologies provide a reliable means of interpreting and processing large datasets in a manner analogous to engineering methods [6,20]. These techniques enable the exploration of intricate relationships between landslide occurrences and various conditioning factors within extensive datasets [21,22]. A multitude of machine learning methods, including decision trees (DT) [19,23,24], function trees (FT) [25,26,27], logistic regression (LR) [12,28,29,30], support vector machines (SVM) [31,32,33], frequency ratios [30,34,35], multiple regression [36,37,38], evidential belief functions [13,39,40,41], weight of evidence [42,43,44,45], artificial neural networks (ANN) [46,47,48,49], Naïve Bayes trees (NBT) [50,51,52], kernel logistic regression [53,54,55], boosted regression trees [56,57], classification and regression trees (CART) [23,56,58], and random forest (RF) [59,60,61], have been successfully employed in landslide susceptibility analysis and zoning.

Despite the successful implementation of the above methods, recent research concerning the accuracy and reliability of models has highlighted the limitations of relying solely on single machine learning algorithms when confronted with the superior performance of hybrid machine learning methods, underscoring the growing significance of ensemble models [62]. Integration technology has emerged as a pivotal advancement in this domain [4,19,63,64,65]. The primary advantage of integrated models lies in their heightened recognition capabilities and their ability to deliver more accurate and robust predictions compared to individual models [64,66]. Notably, the rotation forest-functional trees (RFFT) aggregate model surpasses the performance of standalone SVM and FT models, showcasing superior prediction abilities [67]. Similarly, the fusion of ANN, SVM, and maximum entropy (MaxEnt) models introduces a novel hybrid integration method for spatial landslide susceptibility prediction in the Wanyuan area, as reported by Chen et al. [64]. This is also the case for Pham et al. [58], who presented a hybrid machine learning model based on random subspace classification and regression trees, which have been compared and evaluated against SVM, CART, NBT, and LR models for landslide susceptibility prediction, consistently demonstrating excellent performance. Furthermore, the recurrent error pruning tree (REPTree) model, in conjunction with various integration technologies such as bagging, multi-boss, rotation forest, and random subspace, has yielded a novel integrated model with improved performance [68]. The advance of ensemble and hybrid models has been highlighted by Hong et al. [69]. The authors conducted a study in the Guangchang district of Jiangxi province, China, with the research aimed to assess and compare the performance of various ensemble techniques, including AdaBoost, bagging, and rotation forest, alongside the base classifier of the J48 decision tree (JDT), for landslide susceptibility prediction. The results revealed that all models exhibited high predictive capabilities (AUC > 0.8), with JDT combined with rotation forest demonstrating the highest accuracy (AUC = 0.855).

In this context, our research aimed to develop a landslide susceptibility model for Yanchuan County, situated in the hilly and gully region of the Loess Plateau in northern Shaanxi, China. This area is prone to landslides, which pose substantial threats to human life and the environment. To address this challenge, we introduced a hybrid ensemble-based methodology combining machine learning models, including REPTree and FT, with the bagging algorithm. Our comprehensive approach to modeling landslide susceptibility and evaluating the proposed models sets our study apart. While previous studies have examined the performance of FT, REPT, and bagging models individually, our research combines these models in a novel ensemble approach, leading to more accurate landslide susceptibility predictions. The evaluation approach includes the use of various metrics such as receiver operating characteristic (ROC) curves, trade-off statistical metrics (positive predictive rate (PPR), negative predictive rate (NPR), sensitivity, specificity, etc.), and chi-square analysis for an overall evaluation. This comprehensive approach enhances prediction accuracy and provides valuable insights into landslide susceptibility, supporting more informed land-use and planning decisions. By offering a more comprehensive assessment of model performance and verification using a range of metrics, our research provides a powerful tool for assessing landslide risks in the study area and other regions with similar geological and environmental characteristics.

2. Study Area

Yanchuan County is situated in the northeastern part of the Yan’an region, within the northern reaches of Shaanxi Province, China. Its geographical coordinates range from 109°36′20” to 110°26′44” east longitude and from 36°37′15” to 37°5′55” north latitude, encompassing a total land area of approximately 1985 km² (Figure 1). The topography of this region exhibits significant variations in altitude, with elevations ranging from 492 m to 1392 m. The terrain presents a distinct pattern, with higher elevations prevailing in the northwest and gradually decreasing towards the southeast. The terrain predominantly slopes from the northwest to the southeast. Notably, the steepest slopes in this area exhibit angles reaching up to 68.30°, while a substantial portion, approximately 58.19% of the total area, comprises slopes with angles less than 20°. Areas characterized by slope angles between 20° and 60° encompass roughly 41.80% of the study area.

The study area, situated in the hilly and gully region of the Loess Plateau in northern Shaanxi, experiences a temperate continental monsoon climate characterized by higher elevations in the northwest and lower altitudes in the southeast. Several geological and environmental factors contribute to the development of unstable slopes in Yanchuan County, primarily owing to substantial loess accumulation, loose soil structure, severe soil erosion, intricate gully networks, rugged terrain, and significant human engineering activities. Consequently, landslides in this region predominantly manifest as loess and soil landslides characterized by traction movements. These unstable slopes tend to concentrate along larger riverbanks, convex and linear slopes, and the northern loess beam area. In high and steep slopes, cracks resulting from differential weathering often precipitate landslide disasters. Landslides occur in all directions but show a pronounced concentration on northwest-facing shady slopes (45–90°, 135–180°, and 225–300°), contributing to the asymmetric distribution of landslides on both sides of valleys. Concerning the geometric characteristics of landslides, the main length varies between 40 and 160 m, the width varies between 60 and 200 m, and the thickness is mainly concentrated in 2 to 25 m.

3. Materials and Methods

In this study, we adopted a comprehensive modeling approach to assess and compare landslide susceptibility in the Yanchuan area of China, using single tree-based models and hybrid-bagging-based models. Specifically, REPTree and FT were the two single tree-based models, whereas using the bagging method models and having as base learns the REPTree and FT models, including their combined forms, bagging REPTree (BREPTree) and bagging FT (BFT). These models were evaluated using a range of assessment metrics, including ROC curves and trade-off statistical metrics.

DTs are predictive models that segment data into subsets based on feature value differences, constructing a tree for decision-making. The result is a tree with decision nodes and leaf nodes, where each node represents a feature in the instance to be classified, and each branch represents a decision rule, ultimately leading to a leaf node that indicates the outcome. The REPTree model refines this by pruning less accurate branches to enhance prediction. The model uses a regression tree that prunes using reduced error pruning and is capable of learning from both nominal and numerical data. FT enhances the decision-making process by constructing a DT that classifies samples using the gain ratio as the split criterion. This allows for the integration of logistic regression functions at the leaves or nodes of the tree, combining the interpretability of the DT with the predictive power of logistic regression. Bagging, short for bootstrap aggregating, is an ensemble machine learning technique that improves the stability and accuracy of machine learning algorithms. It works by creating multiple versions of a predictor model (such as a DT) and then combining them to form an aggregated model with better performance. Bagging involves training each model on a random distribution of the dataset and then combining the models by averaging the output (for regression) or voting (for classification). Our modeling approach involved considering the strengths and weaknesses of each model to select the most suitable one for landslide susceptibility assessment. Further explanation concerning models will follow in the following paragraphs.

The proposed methodology follows a five-step analysis, which involves: (i) constructing the landslide and non-landslide inventory, (ii) data selection and classification, (iii) correlation attribute evaluation process, certainty factor analysis and normalization (iv), applying the tree-based and hybrid models to produce the four landslide susceptibility models and estimating the learning and predictive performance of the four models (v), generate the susceptibility maps and calculate the relative landslide density. A detailed description of our modeling methods is provided below, complemented by Figure 2, which illustrates the flowchart of our hybrid-ensemble model approach.

3.1. First Step of Analysis

The accuracy and reliability of a landslide susceptibility model are greatly influenced by the quality of the landslide inventory map, which is essential for recording vital information about various aspects of landslides, including their types, occurrence timing, and precise locations [70,71,72,73]. A fundamental assumption underlying landslide susceptibility modeling is that future landslides will exhibit patterns similar to those observed in the past [74,75]. Consequently, the composition of the landslide inventory map primarily draws from historical records, field surveys, and the interpretation of remote sensing imagery, resulting in a comprehensive dataset comprising a total of 311 distinct landslide areas. It is important to note that these areas have been represented as single points, referencing the centroid of each respective landslide region. To develop the landslide susceptibility model, we employed random sampling techniques, selecting 218 landslide disaster points, which account for 70% of the dataset, using the Subset Features in the Geostatistical Analyst toolbox of the ArcGIS suite. The remaining 93 points, constituting 30% of the dataset, were set aside for model verification purposes. Non-landslide areas were identified through a combination of methods. These methods included random point selection within the study area, ensuring that these points did not coincide with known landslide locations. Additionally, satellite imagery analysis of Landsat 8 images, obtained from the International Scientific and Technical Data Mirror Site of the Computer Network Information Center, Chinese Academy of Sciences, available at http://www.gscloud.cn (accessed on 22 August 2023), was employed to pinpoint regions characterized by stable vegetation cover, flat terrain, or solid bedrock—criteria that, in theory, denote areas less susceptible to landslides and thus suitable for classification as non-landslide areas. The spatial distribution of the landslide and non-landslide datasets is illustrated in Figure 3.

3.2. Second Step of Analysis

Selection and Classification of Landslide—Related Variables

The occurrence of landslide disasters in Yanchuan County is linked to a combination of local geological, topographic, and environmental factors [71,76]. The selection of appropriate conditioning factors for landslide susceptibility modeling is a critical step, as these factors may exhibit interdependencies that could impact predictive accuracy [77,78,79]. Consequently, this study carefully selected 16 landslide conditioning factors to create thematic maps for landslide susceptibility modeling (Figure 2), which include slope, aspect, elevation, plan curvature, profile curvature, slope length, the topographic position index (TPI), terrain ruggedness index (TRI), convergence index (CI), distance to rivers, distance to roads, rainfall, the normalized difference vegetation index (NDVI), soil type, lithology, and land use. The main data sources for the 16 landslide—related factors included digital elevation model (DEM) data, Landsat 8 Operational Land Imager imagery with a 30 m by 30 m resolution, a 1:200,000 geological map, 1:100,000 land use/land cover map, and 1:1,000,000 soil type map. Utilizing geospatial transformation functions available in the ArcGIS suite, we categorized the 16 landslide conditioning factors into thematic layers, distinguishing between continuous and discontinuous data.

In more detail, from the DEM file (resolution 30 × 30 m) obtained by the International Scientific and Technical Data Mirror Site of the Computer Network Information Center, Chinese Academy of Sciences, available at http://www.gscloud.cn, slope, aspect, plan and profile curvature, slope length, TPI, TRI, and CI were constructed using the ArcGIS suite and the Spatial Analyst Tools toolbox. For these continuous raster files, the slope length, TPI, TRI, and CI were reclassified into five classes using the natural break classification method having the same resolution (30 × 30) as the DEM file. For slope, aspect, elevation, plan and profile curvature, distance from the river, distance from roads, and rainfall, we applied a manual classification scheme, and the files were reclassified in the same resolution (30 × 30) as the DEM file. The slope was classified into seven classes, aspect and elevation into nine classes, plan and profile curvature into three classes, distance from the river and distance from roads into five classes, and rainfall into six classes. Soil type, lithology, and land use were digitized from the available maps and transformed into a 30 × 30 resolution raster format to match the rest of the files. The NDVI and land use maps were generated by processing Landsat 8 satellite images using ENVI 5.1 software. The Landsat 8 images, which had a spatial resolution of 30 × 30 m, were also sourced from the same website as the ASTER Global DEM data, accessed on 22 August 2023. The NDVI was classified into five classes according to the natural break classification scheme, whereas land use was classified into six classes. Finally, the rainfall map was created using comprehensive rainfall data that covered the study area and reclassified into six classes. These data were collected and compiled in collaboration with the Meteorological Bureau of the region. Each of the raster format files was weighted according to the certainty factor values, as discussed in Section 3.3.2 (Figure 4).

3.3. Third Step of Analysis

3.3.1. Correlation Attribute Evaluation—Importance Analysis

In landslide susceptibility modeling, the selection of landslide conditioning factors significantly influences landslide susceptibility. Therefore, selecting the landslide conditioning factors and evaluating the prediction ability to optimize the spatial analysis of landslide susceptibility is necessary. The influence of landslide conditioning factors on the landslide susceptibility model is not certain. If there are factors that are not conducive to modeling, they must be eliminated to ensure the good quality of the model. In this research, 16 factors were selected, and their prediction ability was evaluated using the CorrelationAttributeEval (correlation attribute evaluation) (CAE) method. A compatible ranking technique was used to rank the overall attributes of each factor evaluation. Afterward, Spearman’s rank correlation coefficient was used to quantify the correlation between the factors, indicating the correlation between each factor [65]. To ascertain the reliability of the selected landslide susceptibility factors, this study adopts the 10-fold cross-validation attribute selection models to assess the importance of each factor.

3.3.2. Certainty Factor Analysis

The certainty coefficient (CF) method employed in this study serves as a robust tool for assessing the correlation between various conditioning factors and landslide occurrences [71,80,81,82]. Despite its simplicity, this method is known for its high accuracy in evaluating landslide susceptibility. Its accuracy stems from the assumption that geological disasters that have occurred and those that may occur in the future will manifest under the same geological conditions. The CF calculation formula is as follows [71] (Equation (1)):

$$CF=\left\{\genfrac{}{}{0pt}{}{\frac{(PPa-PPs)}{PPa\times (1-PPs)}}{\frac{(PPa-PPs)}{PPs\times (1-PPa)}}|\genfrac{}{}{0pt}{}{if PPa\ge PPs}{if PPa<PPs}\right\}$$

(1)

where PPa represents the conditional probability of landslides in a specific impact factor classification “a”. It is calculated as the ratio of landslide points within classification “a” to the total area covered by that classification. On the other hand, PPs signifies the probability of historical landslides across the entire study area. It is calculated as the ratio of historical landslide points to the total area of the study. The CF values fall within the range of −1 to 1. When CF values are between zero and 1, it indicates that landslides are more likely to occur in the geological environment corresponding to the specific classification. The closer the CF value is to 1, the higher the sensitivity of that unit to landslides. Conversely, when CF values are between −1 and zero, it suggests that landslides are less likely to occur in the corresponding geological environment.

3.4. Fourth Step of Analysis

The fourth step of analysis involved the implementation of the single-based tree models, the REPTree and FT, and the hybrid bagging-based REPTree and FT models. The hybrid bagging-based models are used as base learners for the two single-based tree models. The purpose of this step is to compare the performance of the four models in terms of (ROC curves, accuracy, sensitivity, specificity, the F-score, Matthews correlation coefficient (MCC), true skill statistic (TSS), and root-mean-square error (RMSE).

3.4.1. Reduced Error Pruning Decision Tree

The REPTree model serves as a crucial element within our modeling approach, effectively merging the foundational principles of the DT with error pruning reduction (REP). At the heart of the REPTree algorithm lies the concept of information gain, aimed at simplifying the often-intricate DT model structure to attain objectives of minimizing variance and enhancing prediction accuracy [83,84].

The maximum information gain ratio is calculated as follows [24] (Equation (2)):

$$GainRatio\left(x,A\right)=\frac{Entropy\left(A\right)-{\displaystyle {\sum }_{i=1}^n\frac{\left|A_i\right|}{\left|A\right|}}Entropy\left(A_i\right)}{-{\displaystyle {\sum }_{i=1}^n\frac{\left|A_i\right|}{\left|A\right|}}{\log }_2\frac{\left|A_i\right|}{\left|A\right|}}$$

(2)

where in our case, attribute x representing various landslide conditioning factors is sourced from the training subset A_i (i = 1, 2, …, n) within the landslide training dataset A. These factors include slope, aspect, elevation, plan curvature, profile curvature, slope length, topography position index, terrain ruggedness index, convergence index, distance to rivers, distance to roads, rainfall, NDVI, soil, lithology, and land use.

Error reduction pruning (REP) is based on the principles of empirical risk minimization, as established by empirical observations and analyses [85,86]. By combining with the DT framework, REP effectively minimizes errors, ensuring the highest level of accuracy possible [87]. To mitigate the risk of excessive complexities during the learning process, REP employs a post-pruning technique that strategically removes branches and leaves that make only marginal contributions to the model. This process is essential for reducing the overall size of the DT model, optimizing its efficiency, and maintaining robust predictive accuracy [82].

3.4.2. Function Trees

The FT, an algorithm derived from the DT, represents a fusion of discriminant functions and a multivariate DT, initially proposed by Gama [27]. The FT model has found extensive application in landslide modeling across various studies [52,60,88]. FT’s primary objective is to construct a DT capable of distinguishing samples (x, y) within the training dataset A, where “y” denotes the output value, either 0 or 1, signifying the presence or absence of landslides.

In the FT model, each sample navigates from the root node to a leaf node, and functions are constructed at each node to expand the attribute set at decision nodes. The decision-making tests performed at each node determine the sample’s path, ultimately leading to classification into functions or constants at the leaves [27].

The FT employs the gain ratio as the split criterion to select input attributes for splitting. To prevent overfitting, it utilizes the standard C4.5 pruning technique [88]. FT employs logistic regression functions for calculations to handle internal node fission (oblique cracking) and predict outcomes at leaf nodes [89]. In this context, W_i represents the coefficient of the i-th component in the input vector A_i (i = 1, 2, …, n), and P(x) signifies the probability prediction value. The posterior probability of a leaf node, P(x), is computed as follows [90] (Equations (3) and (4)):

$$f_{y_i}(x)={\displaystyle \sum _{i=1}^{16}{W}_i{x}_i+W_0}$$

(3)

$$P(x)=\frac{{\mathrm{e}}^{2f_{y_i}(x)}}{1+{\mathrm{e}}^{2f_{y_i}(x)}}$$

(4)

3.4.3. Bagging

The bagging model represents an intelligent integrated machine learning technique pioneered by Breiman, grounded in the principles of clustering and bootstrapping [69,91]. As one of the most widely adopted combinatorial construction methods, the bagging algorithm plays a pivotal role in acquiring more precise and robust models [69,91]. Its primary function revolves around enhancing classification accuracy by diminishing the variance associated with classification errors [69,91]. The bagging model possesses a keen aptitude for detecting and analyzing fluctuations within training data, rendering it exceptionally effective in the realm of landslide susceptibility modeling. This capability significantly elevates the model’s predictive prowess [68,92].

The bagging algorithm is an ensemble machine learning technique that combines the predictions from multiple base models to improve overall predictive performance. The theoretical expression of the bagging estimator can be described as follows:

Let us assume a dataset D containing N samples (D = {x₁, x₂, …, x_N}) and their corresponding labels (target values) Y = {y₁, y₂, …, y_N}. There is also a base model (e.g., decision tree and neural network) denoted as M, and the purpose is to train B different instances of this base model to create an ensemble.

For each of the B base models, a new dataset (D_i) is generated by randomly sampling N samples with replacement from the original dataset D. This process creates multiple bootstrapped datasets. The next step is to train each B base model independently on these bootstrapped datasets. Each model M_i (i = 1 to B) is trained on a different dataset D_i. To make predictions, you apply each of the B models to a new input sample x (where x is not necessarily part of the training data). Each model M_i produces its prediction, denoted as ŷ_i. Finally, you aggregate the predictions from all B models to obtain the final prediction. Common aggregation methods include taking the majority vote for classification tasks (e.g., counting the most frequent class among the predictions) or averaging the predictions for regression tasks. Mathematically, for a classification problem with binary outcomes (0 or 1), you can express the bagging estimator as follows:

Given B base models M_i, i = 1 to B, each trained on bootstrapped datasets D_i, you can calculate the final prediction as (Equation (5)):

$$\mathrm{ŷ}=\sum {\mathrm{ŷ}}_{\mathrm{i}}$$

(5)

where ŷ is the final prediction, ŷ_i is the prediction from the i-th base model, and B is the total number of base models in the ensemble.

The bagging algorithm effectively reduces overfitting, improves model stability, and enhances predictive performance by combining the diversity of individual models trained on different data subsets.

3.4.4. Receiver Operating Characteristic (ROC) Curves

The area under the ROC curve (AUC) serves as a crucial metric obtained through the integration of the ROC curve. It plays a pivotal role in assessing the overall performance of the model and facilitating comparisons among various models [93,94]. The ROC curve is widely embraced as a tool to assess the quality of probability prediction systems [71,95]. It presents a graphical representation where the horizontal axis (X-axis) portrays 1-Specificity (the false positive rate), while the vertical axis (Y-axis) represents Sensitivity (the true positive rate) [71,95]. The ROC curve and its corresponding AUC value are instrumental in quantifying the likelihood of event occurrence and are pivotal in diagnosing and evaluating landslide susceptibility. The ROC curve takes shape by establishing the relationship between the true and false positive rates within a binary classification system. Notably, ROC curves exhibit a unique appeal as they do not heavily rely on the class ratio, making them impervious to discrepancies between the training and validation datasets [65]. AUC values provide a definitive measure of ROC curve analysis, offering a specific numeric representation of the model’s quality on the training dataset and its predictive capabilities on the validation dataset. AUC values range from 0 to 1, with a curve closer to the upper left corner indicating superior model performance [71]. Larger AUC values denote a more accurate model with enhanced performance. An AUC value of 1 signifies the highest level of model accuracy, an ideal achievement. Generally, AUC values are categorized into distinct ranges: excellent (0.9 to 1), very good (0.8 to 0.9), good (0.7 to 0.8), average (0.6 to 0.7), and poor (0.5 to 0.6) [96]. When the AUC value falls between 0 and 0.5, it implies that the classification needs to be inverted [97].

3.4.5. Trade-Off Statistical Metrics

A confusion matrix for in-depth analysis was constructed to achieve a more precise evaluation of the four models. Within this matrix, several fundamental statistical indicators are utilized: TP (true positive), indicating the number of pixels correctly classified as landslide occurrences; TN (true negative), representing the number of pixels accurately classified as non-landslide areas; FP (false positive), denoting the number of pixels incorrectly identified as landslides when they are not; and FN (false negative), indicating the number of pixels wrongly classified as non-landslide areas when they are indeed landslides. Furthermore, within the evaluation step, a range of trade-off statistical metrics from the confusion matrix, including the positive predictive rate (PPR), negative predictive rate (NPR), sensitivity, specificity, accuracy, F-score, Matthews correlation coefficient (MCC), true skill statistics (TSS), and root-mean-square error (RMSE) were calculated. Trade-off statistical metrics refer to a set of metrics used to assess the balance between different aspects of model performance, particularly in binary classification problems like landslide susceptibility mapping. These metrics help evaluate how well a model performs in terms of making trade-offs between different types of errors (e.g., false positives and false negatives) and various aspects of classification, such as sensitivity, specificity, positive predictive rate (PPR), negative predictive rate (NPR), accuracy, and more. These metrics provide a more nuanced understanding of a model’s performance by considering both its ability to correctly identify positive cases (landslides) and negative cases (non-landslides) and the potential trade-offs between them, providing a more robust assessment of model performance beyond the ROC curve and AUC values as being reported in previous research [65,98,99,100]. The following Equations (6)–(15) provide the mathematical expression of the above comprehensive evaluation metrics.

$$AUC=\frac{{\displaystyle \sum TP}+{\displaystyle \sum TN}}{P+N}$$

(6)

$$PPR=\frac{TP}{TP+FP}$$

(7)

$$NPR=\frac{TN}{TN+FN}$$

(8)

$$Sensitivity=\frac{TP}{TP+FN}$$

(9)

$$Specificity=\frac{TN}{TN+TP}$$

(10)

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(11)

$$F-score=\frac{2TP}{2TP+FP+FN}$$

(12)

$$MCC=\frac{\left(TP\times TN\right)-\left(FP\times FN\right)}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}$$

(13)

$$TSS=\frac{TP}{TP+FN}+\frac{TN}{TN+FP}-1$$

(14)

$$RMSE=\sqrt{\frac{{\displaystyle \sum _{k=1}^m\left(X^{\left(0\right)}\left(k\right)-{\widehat{X}}^{\left(0\right)}\left(k\right)\right)}}{m}}$$

(15)

where m is a value that corresponds to the sum of the total number of non-landslides and the number of landslides. In the model, $X^{\left(0\right)}\left(k\right)$ is the observed value of the landslide and ${\widehat{X}}^{\left(0\right)}\left(k\right)$ is the predicted value of the landslide.

3.4.6. Pairwise Comparison Based on Chi-Square Test (χ² Test)

In the present study, the Chi-Square test (χ² test) was used to perform pairwise comparisons of the four models (BREPTree, REPTree, FT, and BFT) to determine whether there are significant differences in their performance based on a categorical outcome. The first action is to set up the hypotheses. Specifically, the null hypothesis (H0) assumes that there is no significant difference in the performance between the two models being compared, whereas the alternative hypothesis (H1) assumes that there is a significant difference in performance between the two models being compared.

The comparison was based on the performance of each model’s correct and incorrect predictions, with degrees of freedom set to 1 and significance level α = 0.05, thus χ²_critical value set to 3.841. The chi-square statistic, χ², is calculated as (Equations (15) and (16)):

$${\chi }^2=\sum \frac{{(O-{\rm E})}^2}{{\rm E}}$$

(16)

where O is the observed frequency and E is the expected frequency under the null hypothesis.

If χ² > χ²_critical, the null hypothesis is rejected, which means that there is a significant difference in performance between the two models; otherwise, the null hypothesis is not rejected, which means that there is no significant difference in performance between the two models.

3.4.7. Landslide Susceptibility Mapping

The final step in our proposed methodology involves the creation of landslide susceptibility maps, a crucial component of our analysis. In more detail, within this final step, the landslide susceptibility index (LSI) is calculated for every pixel, utilizing the powerful capabilities of ArcGIS 10.5 software. ArcGIS offers several techniques for map classification, including Jenks natural breaks, geometric intervals, equal intervals, standard deviation, quantile, and manual methods [101,102]. Our study employed the Jenks natural breaks classification method to reclassify the results obtained from the four different landslide susceptibility models. The natural breaks method was particularly useful for continuous data to identify meaningful classes based on natural groupings inherent in the data. This classification process assigns each model’s output to one of five distinct susceptibility levels, encompassing very low, low, medium, high, and very high categories. This final susceptibility map visually represents landslide susceptibility across the study area, offering valuable insights for effective landslide risk assessment and management.

4. Results

Importance of Landslide Conditioning Factors

The initial output of the proposed methodology focuses on assessing the predictive capability and significance of each landslide-related factor. The average merit (AM) assigned to each factor serves as an indicator of its importance in the context of landslide occurrences. Notably, there is a standard deviation (Std.) associated with each average merit (AM), reflecting the degree of variability in their importance assessments. The results underscore a positive correlation among the 16 conditioning factors, collectively contributing to the enhancement of landslide susceptibility modeling. These factors, characterized by positive correlations, remain pivotal for further analysis (

Table 1. Predictive ability and importance of landslide-related factors—average merit values.

Landslide-Related Factor	Average Merit (AM)	Standard Deviation (Std.)
Slope	0.612	±0.012
Terrain Ruggedness Index	0.456	±0.015
Distance to Rivers	0.262	±0.019
Convergence Index	0.187	±0.012
Aspect	0.182	±0.011
Rainfall	0.176	±0.012
Land Use	0.172	±0.011
Plan Curvature	0.159	±0.009
Profile Curvature	0.157	±0.007
Slope Length	0.140	±0.011
Distance to Roads	0.116	±0.014
Elevation	0.115	±0.116
NDVI	0.088	±0.010
Lithology	0.071	±0.017
Soil	0.029	±0.014
Topography Position Index	0.029	±0.014

). Among the 16 landslide conditioning factors, the analysis reveals that the slope (AM = 0.612, Std. = ±0.012) and terrain ruggedness index (AM = 0.456, Std. = ±0.015) exhibit the highest predictive capabilities. Following closely are the distance to rivers (AM = 0.262, Std. = ±0.019), convergence index (AM = 0.187, Std. = ±0.012), aspect (AM = 0.182, Std. = ±0.011), rainfall (AM = 0.176, Std. = ±0.012), Land Use (AM = 0.172, Std. = ±0.011), Plan Curvature (AM = 0.159, Std. = ±0.009), profile curvature (AM = 0.157, Std. = ±0.007), slope length (AM = 0.140, Std. = ±0.011), distance to roads (AM = 0.116, Std. = ±0.014), elevation (AM = 0.115, Std. = ±0.116), NDVI (AM = 0.088, Std. = ±0.010), lithology (AM = 0.071, Std. = ±0.017), soil (AM = 0.029, Std. = ±0.014), and topography position index (AM = 0.029, Std. = ±0.014). These findings provide valuable insights into the significance and relative contributions of these factors to landslide susceptibility modeling.

Concerning the analysis of the CF value and the three landslide factors with the most predictive ability, the “21–30” and “31–40” slope classes exhibit the highest percentages of landslide occurrences, followed by the “41–50” class. Conversely, the “51–60” and “61–68.30” slope classes show no landslides, and their negative certainty factors suggest lower confidence in landslide occurrences in these ranges. Ranging from 0 to 75.75, each interval provides details on the number and percentage of landslides within the specified TRI range, the total number of pixels in the domain, the percentage of the domain covered, and the associated certainty factor. Notably, the TRI intervals demonstrate varying degrees of terrain complexity, with higher TRI values indicating increased ruggedness. The CF analysis reveals a strong trend between terrain ruggedness and landslides, with higher certainty factors in intervals such as 8.62–12.48, 12.48–17.82, and 17.82–75.75, suggesting a positive correlation between rugged terrain and landslide susceptibility. Concerning proximity to rivers, a positive correlation between the proximity to rivers and landslide susceptibility, with higher certainty factors in intervals such as 0–200 m and 200–400 m, suggests an increased likelihood of landslides in closer proximity to rivers. Conversely, as the distance from rivers increases in the 400–600 m, 600–800 m, and > 800 m intervals, the certainty factors become negative, indicating a perceived decrease in landslide susceptibility with greater distances from rivers. The rest of the CF values for each class and each factor are documented in

Table 2. Certainty factor values.

Factors	Class	Certainty Factor	Factors	Class	Certainty Factor
Slope (°)	0–10	−1.000	Terrain Ruggedness Index	0–5.05	−0.964
	11–20	−0.964		5.05–8.62	−0.464
	21–30	0.563		8.62–12.48	0.501
	31–40	0.598		12.48–17.82	0.568
	41–50	0.656		17.82–75.75	0.574
	51–60	−1.000	Convergence Index	−99.34–−27.72	−1.000
	61–68.30	−1.000		−27.72–−6.92	−0.332
Aspect	Flat	−1.000		−6.92–4.63	0.173
	North	0.340		4.63–24.66	0.010
	Northeast	0.108		24.66–97.05	−0.864
	East	0.192	Distance to rivers (m)	0–200	0.446
	Southeast	0.050		200–400	0.392
	South	−0.333		400–600	−0.380
	Southwest	−0.551		600–800	−0.304
	West	−0.144		>800	−0.412
	Northwest	0.134	Distance to roads (m)	0–300	0.096
Elevation (m)	492–600	−1.000		300–600	0.452
	601–700	−0.372		600–900	0.192
	701–800	−0.608		900–1200	−0.120
	801–900	0.080		>1200	−0.089
	901–1000	−0.163	Rainfall (mm/yr)	<460	0.342
	1001–1100	0.207		460–470	0.231
	1101–1200	0.173		470–480	0.224
	1201–1300	−0.442		480–490	−0.109
	1301–1392	0.349		490–500	−0.238
Plan curvature	−9.17–−0.05	0.024		>500	−0.534
	−0.05–0.05	−1.000	NDVI	−0.22–0.10	−0.167
	0.05–9.75	0.087		0.11–0.15	−0.186
Profile curvature	−12.83–−0.05	0.029		0.16−0.20	0.150
	−0.05–0.05	−1.000		0.21−0.25	0.007
	0.05–14.85	0.063		0.26–0.51	0.012
Slope Length	0–39.86	−0.262	Soil	Calcaric Cambisol	0.000
	39.86–112.33	0.290		Eutric Cambisol	−0.105
	112.33–202.91	0.251		Calcaric Fluvisol	0.084
	202.91–329.73	−0.259		Rendzic Leptosol	−0.484
	329.73–923.97	−0.618	Lithology	Alluvial and Pluvial deposits	−0.224
Topographic Position Index	−87.9–−12.52	0.112		Yellowish brown eolian loess	−0.108
	−12.52–−4.81	−0.049		Brown eoalian loess with paleosoil interlayers	−0.155
	−4.81–2.32	−0.050		Red-brown clay with calcareous nodules	0.544
	2.32–10.03	0.010		Mudstone, sandstone, siltstone, shale	0.165
	10.03–63.45	0.087	Landuse	Farm land	0.064
				Forest land	0.063
				Grassland	−0.079
				Water	−1.000
				Construction land	0.544
				Unused land	−1.000

.

Passing to the implementation of the models, the first action was to set the optimal parameters for each model.

Table 3. Optimal values of structural parameters.

Models	Parameters
Bagging	The number of iterations: 10; the number of execution slots (threads) to use for constructing the ensemble: 1; seed: 1
REPTree	Seed: 1; number of folds: 3; the minimum proportion of the variance: 0.001; the maximum tree depth: −1; the minimum total weight of the instances in a leaf: 2
FT	Number of iterations for LogitBoost: 15; the minimum number of instances at which a node is considered for splitting: 15

provides the values of the structural parameters that were applied to the predictive models using a trial-and-error procedure using accuracy as the evaluation metric. Specifically, the bagging model performed 10 iterations, using only one execution slot for constructing the ensemble, indicating a sequential processing approach. For the REPTree model, the tree construction process used 3-fold cross-validation to evaluate splits and a minimum proportion of variance for considering a split set at 0.001, allowing for fine-grained splits. The maximum tree depth was unrestricted (−1), enabling the tree to grow as needed, while the minimum total weight (sum of instances’ weights) in a leaf was set to 2, preventing overly small leaves. The FT model was configured for 15 iterations of LogitBoost, considering splitting nodes if they contain at least 15 instances, focusing on significant data partitions, and avoiding overfitting on small data subsets.

The results of the ROC curve analysis for both the training and validation datasets revealed important insights into model performance (Figure 5 and

Table 4. Parameters of ROC curves using the training data.

Test Variables	BREPTree Model	REPTree Model	FT Model	BFT Model
ROC Curve Area	0.853	0.798	0.795	0.851
Standard Error	0.018	0.022	0.022	0.019
95% Confidence Interval	0.816–0.889	0.755–0.842	0.752–0.838	0.814–0.888
p-value	<0.0001	<0.0001	<0.0001	<0.0001

and

Table 5. Parameters of ROC curves using the validation data.

Test Variables	BREPTree Model	REPTree Model	FT Model	BFT Model
ROC Curve Area	0.886	0.842	0.856	0.895
Standard Error	0.025	0.029	0.030	0.025
95% Confidence Interval	0.836–0.936	0.785–0.900	0.797–0.914	0.845–0.945
p-value	<0.0001	<0.0001	<0.0001	<0.0001

). In the success rate curve (Figure 5a), the BREPTree model demonstrated the highest AUC value of 0.853, closely followed by the BFT model with an AUC of 0.851. The REPTree model achieved an AUC of 0.798, and the FT model recorded an AUC of 0.795. Furthermore, the BREPTree model exhibited the smallest standard error (0.018), underscoring the precision and reliability of its AUC estimate. The narrow 95% confidence interval (0.816 to 0.889) and the highly significant p-value (“<0.0001”) further supported the robustness of the BREPTree model. In the prediction rate curve (Figure 5b), the BFT model outperformed all others with the largest AUC value of 0.895, followed by the BREPTree model with an AUC of 0.886. The FT model exhibited an AUC of 0.856, while the REPTree model achieved an AUC of 0.842. Importantly, the BFT and BREPTree models exhibited the same and minimal standard error (0.025), suggesting a high confidence level in their AUC estimates. The significance levels of these parameters, as indicated by the low p-values, underscore the robustness of the models. These findings collectively demonstrate that all models in the study perform reasonably well, with the hybrid BFT model consistently exhibiting the highest AUC values, indicating its superior predictive accuracy.

The significance of model verification in the context of landslide susceptibility assessment cannot be overstated, as it serves as the crucial step in ensuring the reliability and accuracy of the predictive models. In this study, the verification process employed a comprehensive set of trade-off statistical metrics, effectively quantifying the performance of the four models. The results (

Table 6. Validation of results using cutoff-dependent metrics.

Metrics	BREPTree	REPTree	FT	BFT
TP	79	81	75	83
TN	72	66	76	78
FP	21	27	17	15
FN	14	12	18	10
PPR	0.790	0.750	0.815	0.847
NPR	0.837	0.846	0.809	0.886
Sensitivity	0.849	0.871	0.806	0.892
Specificity	0.774	0.710	0.817	0.839
Accuracy	0.812	0.790	0.812	0.866
F-score	0.819	0.806	0.811	0.869
MCC	0.625	0.588	0.624	0.732
TSS	0.624	0.581	0.623	0.731
RMSE	0.245	0.274	0.388	0.343

) highlight the strengths and capabilities of each model in predicting landslides and non-landslide areas, ultimately guiding decision-making processes related to landslide risk management. The positive predictive rate (PPR) assesses the models’ ability to correctly identify areas prone to landslides, with the BFT model achieving the highest PPR (0.847), followed by the FT model (PPR = 0.815). Furthermore, the BFT model excels in the negative predictive rate (NPR) in correctly identifying non-landslide areas, with an NPR value of 0.886. Regarding landslide classification, sensitivity measures the models’ capacity to accurately detect actual landslide occurrences, and once again, the BFT model achieves the highest value of 0.892, followed by the REPTree model (0.871). The BFT model has the highest value in specificity (0.839), a metric that evaluates the models’ effectiveness in correctly classifying non-landslide areas, followed by the FT model (0.817). Accuracy, as a holistic measure of correctness, underlines the BFT model’s dominance, achieving the highest accuracy score of 0.866, followed by the BREPTree model (0.812), the FT model (0.812), and the REPTree model (0.790). Concerning the F-score, which balances precision and recall, the BFT model’s performance is the highest (F-score = 0.869), whereas BFT demonstrates the highest MCC (0.732), a vital metric for binary classifications. The TSS, a metric that emphasizes sensitivity and specificity, corroborates the BFT model’s robustness, with a TSS score of 0.731, followed by the BREPTree model (0.624). Lastly, the RMSE, quantifying prediction errors, showcases the BREPTree model as the most accurate in terms of landslide susceptibility prediction, with the lowest RMSE value of 0.245, followed by the REPTree model (0.274).

Table 7. Pairwise comparison of models using chi-square.

Pair	BREPTree vs. REPTree	BREPTree vs. FT	BREPTree vs. BFT	REPTree vs. FT	REPTree vs. BFT	FT vs. BFT
Chi-square	11.205	9.264	0.0230	0.0214	8.436	9.547
p-value	0.001	0.002	0.879	0.884	0.004	0.002
Significance	Yes	Yes	No	No	Yes	Yes

compares the four models, BREPTree, REPTree, FT, and BFT, using the chi-square test. This statistical analysis is crucial for determining whether there are significant differences in the performance of these models, which can provide valuable insights into their relative strengths and weaknesses. The chi-square values in the table quantify the degree of dissimilarity between each pair of models. When comparing BREPTree to REPTree, the chi-square value is 11.205, indicating a significant difference in their performance (p-value = 0.001). Similarly, when comparing the BREPTree to the FT, the chi-square value is 9.264, signifying a substantial performance difference (p-value = 0.002). However, when comparing BREPTree to BFT, the chi-square value is only 0.0230, with a high p-value of 0.879, suggesting no significant difference in performance. Comparing REPTree to FT yields a chi-square value of 0.0214 and a p-value of 0.884, indicating no significant difference in performance between these two models. Conversely, when comparing REPTree to BFT, the chi-square value is 8.436, with a p-value of 0.004, signifying a significant performance difference. Finally, when comparing the FT to the BFT, the chi-square value is 9.547, also indicating a significant difference in their performance (p-value = 0.002).

The resulting susceptibility maps illustrate the spatial distribution of landslide susceptibility across the study area (Figure 6a–d). For the REPTree model, the majority of the area falls under the very low susceptibility category (58.74%), followed by low (7.92%), medium (6.08%), high (26.46%), and very high (0.81%) susceptibility. Similarly, the BREPTree model designates the largest portion of the area as very low susceptibility (58.99%), with the rest classified as low (6.20%), medium (9.81%), high (12.13%), and very high (12.87%) susceptibility. In the case of the FT model, the susceptibility classes consist of very low (56.40%), low (7.08%), medium (9.18%), high (11.31%), and very high (16.03%). Finally, the BFT model categorizes the study area into very low susceptibility (45.52%), low (11.11%), medium (13.47%), high (15.99%), and very high susceptibility (13.91%) (Figure 7,

Table 8. Landslide susceptibility zoning.

Models	Very Low Susceptibility	Low Susceptibility	Moderate Susceptibility	High Susceptibility	Very High Susceptibility
REPT	58.74	7.92	6.08	26.46	0.81
FT	56.40	7.08	9.18	11.31	16.03
BREPT	58.99	6.20	9.81	12.13	12.87
BFT	45.52	11.11	13.47	15.99	13.91

). Each of the four landslide susceptibility maps reveals that while the very low susceptibility class dominates the landscape, there are variations in the overall distribution of susceptibility levels among the models. Notably, the bagging algorithm employed in the integrated models (BREPTree and BFT) demonstrates its effectiveness by improving the predictive accuracy and fine-tuning the susceptibility classification.

5. Discussion

Landslides represent a worldwide natural disaster, a significant threat to human life, the man-made and natural environment. These characteristics command the accurate spatial prediction of their occurrence and evolution, a process that is the initial crucial step in landslide disaster risk assessment, essential for informed land-use and planning decisions [103]. As already discussed, identifying landslide-prone areas is not an easy task. Traditional statistical approaches are gradually giving way to machine learning techniques for landslide susceptibility prediction since machine learning algorithms have proven invaluable in mitigating the limitations of subjective assumptions, offering enhanced accuracy and effectiveness compared to expert-based models and traditional statistical analyses. However, recent research highlights the limitations of relying solely on single machine learning algorithms, emphasizing the growing significance of ensemble models. As widely acknowledged, integrated models tend to outperform individual models in general [104,105]. In this context, the current study introduces a five-step methodology approach with the main objective of delivering two hybrid bagging-based models, BDT and BREPTree, which had a REPTree and FT model as base learners, respectively. These hybrid models were evaluated and compared to estimate the most accurate and reliable model. In more detail, the methodology commences with the construction of both landslide and non-landslide inventories, selects and classifies the various landslide-related factors, and incorporates the crucial aspects of correlation attribute evaluation, certainty factor analysis, and normalization, which collectively aid in identifying and quantifying the influences of these factors on landslide susceptibility. It deploys the tree-based and hybrid models to create the four distinct landslide susceptibility models and evaluates their learning and predictive performance with varying statistical metrics and chi-square tests [106,107]. The certainty factor method was applied to identify the trends, patterns, and potential association between landslide factors and landslide manifestation, providing a means to highlight their contribution to landslide susceptibility [108].

Based on the correlation attribute evaluation (CAE) method and the average merit information (AM) values, it has been affirmed that all 16 landslide conditioning factors exert a positive and varying influence on landslide susceptibility, underscoring their individual contributions. Slope, TRI, and the distance to rivers have been identified as having the highest predictive capabilities. These factors are well-known as significant factors in landslide susceptibility modeling due to their direct physical and hydrological connections with landslide occurrence. They are all physical characteristics of the terrain that are directly related to hydrological patterns and landslide susceptibility. Steeper slopes generally pose a higher risk of landslides, rugged terrain can increase landslide potential, and proximity to rivers can affect soil moisture levels and slope stability [7,22,76]. The above factors, along with CI, aspect, and rainfall, are significant factors related to water-related issues. Specifically, in most cases, rivers with high-volume water flow are associated with areas of higher landslide susceptibility [109]. These interconnections highlight how accurately predicting landslide-prone areas is vital for managing and mitigating potential disruptions to water systems. Landslide occurrences can lead to significant consequences such as blockage of rivers and streams, contamination of water supplies from soil and debris, and alterations in hydrological patterns, affecting water availability. By addressing these factors in landslide susceptibility models, we can provide crucial insights for effective water resource management, particularly in mitigating the impacts of landslides on water flow and quality.

Concerning the low AM values of lithology and soil type covers, the study area is characterized by a high degree of geological uniformity; the diversity and influence of lithology and soil types on landslide susceptibility might be less pronounced than they would be in areas with more geological diversity. In regions where geological features are relatively homogenous, other factors, such as topography or hydrological conditions, may play a more pivotal role in triggering landslides.

Among the factors influencing landslide susceptibility, certain classes within each variable exhibit notably higher certainty factor values. Notably, the class ranging from 40 to 50 degrees for slope holds the highest certainty factor at 0.656, suggesting a strong positive correlation with landslide susceptibility. Similarly, in terms of aspect, the “North” class stands out with a certainty factor of 0.340, indicating an inclination towards increased susceptibility. Elevation-wise, areas higher than 1300 m demonstrate the highest certainty factor of 0.349. Other influential factors include the terrain ruggedness index, where the class ranging from 17.82 to 75.75 exhibits a substantial certainty factor of 0.574. Additionally, classes within distance to rivers (0–200 m) and distance to roads (300–600 m) also show heightened certainty factors of 0.446 and 0.392, respectively, emphasizing their significance in landslide susceptibility assessment. Red-brown clay with calcareous nodules and construction land show the highest certainty factor values of 0.544.

The main idea behind using bagging with base learners is to reduce overfitting and variance, thus improving the overall model’s generalization performance. It can lead to better predictive accuracy and stability, especially when individual base learners might have different strengths and weaknesses. The bagging approach builds multiple classifiers through random sampling with replacement in parallel, meaning that each model is constructed independently of the others. Our study’s hybrid approach leverages the strengths of both base learners and the bagging method to create a more powerful predictive model. In general, bagging approaches are more effective when base models appear unstable or prone to overfitting. The evaluation of the hybrid models using both training and validation datasets yielded valuable insights into their respective strengths and reliability. The high AUC values, minimal standard errors, narrow confidence intervals, and significant p-values collectively indicated the models’ effectiveness in predicting landslide susceptibility. The robustness and reliability of the BFT model stem from the combined benefits of bagging’s variance reduction and error compensation, along with the functional trees’ ability to capture complex data patterns. This synergy makes BFT models particularly effective for tasks like landslide susceptibility prediction, where the complexity and variability of the data require sophisticated modeling techniques. Comparing the concept of FBT models with RF, which are known for their robustness and ability to handle diverse datasets, BFT extends the bagging concept by incorporating FT, which can capture complex nonlinear relationships within the data (

Table 9. Description of the four models: REPT, FT, BREPT, and BFT.

Model	Description	Relevance in Bagging Approach and Landslide Susceptibility Prediction
REPT	Reduced error pruning tree (REPTree): A fast decision tree learner that builds and prunes a tree using reduced error pruning. Efficient in various classification tasks.	As a base learner, REPT can be prone to overfitting, but when used within a bagging framework (as in BREPT), its stability and accuracy are enhanced, making it more suitable for complex tasks like landslide susceptibility prediction.
FT	Functional tree (FT): Integrates logistic and linear regression models at the leaves of the tree, providing nuanced classification or regression outputs.	FT’s ability to capture complex data patterns makes it a strong candidate for ensemble methods. It is effective in hybrid models due to its detailed approach to data analysis.
BREPT	Bagged REPTree: An ensemble of REPT models created using bagging. Each model in the ensemble is trained on a different subset of the data.	Bagging reduces overfitting and variance in REPT, thus enhancing its predictive accuracy and stability. This is critical for predicting phenomena with high variability, like landslides.
BFT	Bagged functional tree: Employs bagging with FT models as base learners. Each model is trained on a distinct data subset.	Combines bagging’s variance reduction and error compensation with FT’s complex data pattern recognition. Highly effective in landslide susceptibility prediction due to its robustness and reliability, as indicated by high AUC values and minimal standard errors.

). Similar to our findings, Zhao and Chen [71] report that the BFT model was the best optimization ensemble model in their study among the FT, Dagging-FT, and Rotation Forest-FT models, and it could be used as an advantageous and promising method for landslide susceptibility modeling. Also, Peng et al. [110] implemented an advanced random subspace (RS) and FT-coupled model and compared its performance with BFT, CART, and NBT models. Their findings agreed with ours in terms of higher accuracy achieved from hybrid models; however, they found that the RS-FT model was more accurate compared to BFT.

In conclusion, while other bagging-based tree models, like RF, appear as promising methods and are widely applicable, BFT may outperform them when there are complex, nonlinear relationships to uncover. However, as in any classification problem addressed through machine learning techniques, it is advisable to employ multiple methods with either similar or preferably diverse learning mechanisms. This approach allows for a comprehensive examination of the problem and facilitates a relative comparison between the performance metrics of each model to identify the optimal model.

In more detail, the analysis of various metrics reveals that the BFT model emerges as the top performer in landslide susceptibility assessment. BFT exhibits the highest values across key metrics such as accuracy, sensitivity, specificity, and the F-score, providing evidence of its ability to make reliable predictions and maintain a balanced approach in identifying both positive and negative landslide cases. Notably, BFT excels in reducing false positives and false negatives, a critical aspect in landslide susceptibility modeling where erroneous predictions can have significant consequences. Additionally, the BFT model’s high MCC and TSS scores underscore its robustness and discriminatory power in handling complex datasets. Moreover, it achieves the lowest RMSE, indicating minimal prediction errors. Considering these findings, the BFT model is highly recommended for landslide susceptibility assessment in the study area, as it consistently demonstrates superior performance and reliability, promising more accurate and effective risk assessment and management strategies.

The observation that none of the models exceeded a 90% F-score, along with similar trends in other metrics, can be attributed to several factors intrinsic to modeling complex natural phenomena such as landslides. These factors encompass potential data-related limitations, including the dataset’s resolution, coverage, or completeness, as well as the inherent challenges posed by accurately modeling environmental risks. This situation underscores the inherent difficulty in attaining extremely high accuracy levels in the prediction of natural events. It also highlights the ongoing necessity to enhance both the modeling methodologies and the data quality used for such analyses.

Even if our study provides significant insights, it is important to acknowledge its limitations. The reliance on specific data sets and environmental factors may limit the generalizability of our findings. Future studies should aim to address these limitations by incorporating a broader range of data and testing the models in varied geographical contexts. Also, while our study has concentrated on surface geomorphological features as primary indicators for landslide susceptibility in China’s Loess Plateau, we recognize that human activities can significantly influence the activation of landslides. It is well-documented that land use and land cover alterations, such as deforestation, construction, and agricultural practices, can destabilize slopes and precipitate landslide events. In our analysis, land use and land cover data and distance from road networks were utilized; however, more detailed aspects of human activity, including urban development, infrastructure expansion, and intensive agriculture, were not explicitly included. We acknowledge that this is an important dimension of landslide susceptibility and that the exclusion of detailed human activity-related factors could be seen as a limitation of the current study. Future research should aim to integrate these factors to provide a more holistic understanding of the conditions leading to landslide occurrences. Incorporating detailed data on human activities will enhance the accuracy of predictive models and contribute to more effective risk management and mitigation strategies in regions similar to Yanchuan County. Furthermore, while our models incorporate multiple variables to predict landslide susceptibility accurately, the spatial predictions are subject to uncertainties due to potential data quality variations and the dynamic nature of the environmental factors. Acknowledging these uncertainties is vital for carefully interpreting the models’ outputs, especially when applied to land-use planning and risk management decisions. Future research should focus on improving data quality, exploring more sophisticated modeling techniques, and incorporating uncertainty analysis methods, such as Monte Carlo simulations, to quantify and reduce these uncertainties, thereby enhancing the reliability of spatial predictions. Future work may include a comprehensive project aimed at integrating an uncertainty analysis into the landslide susceptibility modeling, which will focus on quantifying the confidence levels associated with the susceptibility predictions by implementing advanced statistical techniques such as bootstrapping and Bayesian inference. This will allow us to identify zones where model predictions are less certain, thereby guiding targeted data collection to refine model inputs and improve predictive accuracy.

The potential to extend our methodology into a spatio-temporal model offers an exciting avenue for advancing landslide susceptibility assessment. By incorporating multi-date landslide inventories and analyzing changes over time, a spatio-temporal model could capture the dynamics of landslide susceptibility, reflecting how it evolves in response to changes in environmental conditions and human activities. This approach would enable the identification of trends and patterns in landslide occurrences, providing insights into areas that are becoming more susceptible over time. Such a model would be particularly useful for monitoring the effectiveness of mitigation measures, planning future land use, and adapting to climate change impacts. Incorporating temporal data requires robust methodologies to handle the added complexity, including time-series analysis and dynamic modeling techniques, which could significantly enhance our understanding of landslide risks and inform more proactive management strategies.

In conclusion, the landslide susceptibility maps generated by the four models using the natural breaks method effectively depict susceptibility zoning across the entire county, offering valuable insights. The diversity in susceptibility distributions among the four models underscores the significance of considering different modeling approaches. The successful application of the bagging algorithm, particularly in the integrated models, demonstrates its potential to enhance the accuracy and reliability of landslide susceptibility assessments, ultimately contributing to informed decision-making and the reduction of landslide risks in the study area. The results underscore the suitability of the selected models for the study area and suggest that, from a probabilistic standpoint, most regions in Yanchuan County are relatively safe in terms of landslide hazards.

6. Conclusions

The present study addresses the critical need for accurate landslide prediction, a vital step in disaster risk assessment and land-use planning. Focusing on the landslide-prone Yanchuan County, the research integrates advanced techniques and comprehensive analyses to develop landslide susceptibility models. Our research introduced a hybrid ensemble-based methodology, developing four distinct landslide susceptibility tree-based models: REPTree, BREPTree, FT, and BFT. These models were rigorously evaluated and compared to identify the most accurate and reliable model for landslide susceptibility assessment. Yanchuan County’s unique geographical and environmental characteristics, characterized by loess accumulation, soil erosion, gully networks, rugged terrain, and human activities, contribute to unstable slopes and landslide occurrences. The selection and evaluation of 16 landslide conditioning factors affirmed their positive influence on landslide susceptibility, underscoring their individual significance. The study’s findings underscore the significance of various conditioning factors, including slope, terrain ruggedness, distance to rivers, rainfall, and land uses, in influencing regional landslide occurrences. Current research highlights the power of integrating methodologies by utilizing ensemble models, particularly the bagging-enhanced REPTree and FT models (BREPTree and BFT). These hybrid models outperform their individual counterparts, emphasizing the value of ensemble techniques in improving prediction accuracy and model reliability. The evaluation metrics, including ROC curves, AUC values, and trade-off statistical metrics, confirm the superiority of the BFT model in accurately predicting landslide susceptibility. The study’s detailed analyses contribute to the specific understanding of landslides in Yanchuan County and shed light on the broader application of ensemble models in landslide susceptibility mapping. Adopting integrated ensemble models, particularly the combination of bagging with the REPTree and FT models to create the BREPTree and BFT models, proved advantageous. This approach effectively reduced data dimensionality and enhanced sensitivity to data variations, resulting in more optimized models. The chi-square analysis conducted on the various model pairs has provided valuable insights into their relative performance in predicting landslide susceptibility. The results highlight significant differences in performance between certain model pairs, notably BREPTree vs. REPTree, BREPTree vs. FT, and REPTree vs. BFT. These findings underscore the importance of selecting the most suitable model based on statistical evidence when tackling landslide susceptibility assessments. Interestingly, no statistically significant differences were observed when comparing BREPTree to BFT and REPTree to FT, indicating that these model pairs exhibit similar predictive capabilities. Each of the four landslide susceptibility maps generated in this study highlights the prevalence of areas classified as having very low susceptibility to landslides. This finding indicates that a significant portion of Yanchuan County is at a relatively low risk of experiencing landslides. However, it is crucial to recognize that there are noteworthy distinctions in the distribution of susceptibility levels across the landscape when comparing the four models. However, the need for further investigation is worth noting, especially in understanding the differences in model performance. This ongoing pursuit of optimization reflects the dynamic nature of predictive modeling and opens avenues for future research in landslide susceptibility assessment in Yanchuan County and similar regions globally. In summary, our research highlights the significance of ensemble models and the potential of the BFT model in improving landslide susceptibility prediction. It also underscores the continuing need for research in this field to enhance our understanding and preparedness for landslide-related disasters.