From Expert-Based Evaluation to Data-Driven Modeling: Performance-Based Flood Susceptibility Mapping

Abstract

Floods are natural disasters that cause significant socioeconomic and environmental losses in both urban and rural areas. Within the framework of spatial planning, precautionary measures against flood hazards can be developed using analytical approaches based on different modeling techniques. In this study, flood-prone areas in the Melen Basin, Türkiye, were identified and mapped using five statistical methods, namely Frequency Ratio (FR), Shannon Entropy (SE), Evidential Belief Function (EBF), and the hybrid models EBF–SE and EBF–FR. The analysis was conducted using a flood inventory and environmental datasets covering the period 2019–2024, including elevation, slope, aspect, land use, plan and profile curvature, drainage density, distance to river, curve number, long-term average precipitation, geological formation, soil depth, topographic wetness index, sediment transport, and stream power index. Model performances were evaluated using the Receiver Operating Characteristic (ROC) curve and the Area Under the Curve (AUC). The results indicate that the SE method achieved the highest predictive performance (AUC = 0.979), followed by FR (0.974), EBF–SE (0.972), EBF–FR (0.968), and EBF (0.966). According to the FR and SE models, elevation, lithology, and slope were identified as the most influential factors in flood occurrence. In the evaluation of the success index of the models, the following values were determined according to their size: EBF–SE (96.0), SE (94.4), EBF (91.8), FR (81.9), and EBF–FR (79.4). In the classification of flood sensitivity maps, Natural Breaks (Jenks) is the most successful method according to the success index. The findings demonstrate that data-driven and hybrid models can effectively support flood risk assessment and provide valuable input for land-use planning and flood risk management.

1. Introduction

In recent decades, evidence has grown that climate change is intensifying the global hydrological cycle [1]. Global climate change is expected to increase the likelihood of events such as drought, flooding, and wildfires [2]. In this context, the most common natural disaster worldwide is flash floods caused by heavy rainfall [3,4,5]. Europe has been facing increasing climate threats such as heat waves, droughts, floods, and rising sea levels, in addition to broken climate records in recent years. Urgent action is needed to address the increased risk of flooding and make infrastructure climate-resilient [6]. According to Emergency Events Database (EM-DAT) data, 64% of disasters are natural in origin, primarily floods (22%) and storms (18%). In 2024, floods caused 2288 deaths, affected 113.2 million people, and resulted in a total economic loss of $19.6 billion [7]. The frequency and severity of floods in Europe are increasing, and concerns are growing that human activities may be having a significant impact on this change [8]. Increased population and asset density in urban development areas and watersheds have heightened flood risk, making accurate flood risk assessment critical [9,10]. Therefore, a key challenge is to identify areas vulnerable to floods, which are natural disasters and cannot be completely prevented, and to reduce potential damage through appropriate management strategies [11].

Flood sensitivity maps form the basis of flood hazard, vulnerability, and risk analysis, so it is crucial that they are produced in the most accurate and reliable manner possible [11,12,13]. Although flash floods cannot be prevented with flood sensitivity mapping, losses can be reduced by identifying risk areas through flood forecasting and implementing appropriate structural and non-structural measures [14,15]. Five main approaches are used in the preparation of flood sensitivity maps: Multi-Criteria Decision Analysis (MCDA), statistical methods, physically-based models, deep learning, and machine learning techniques [16]. Flood risk assessment involves multiple factors, often with conflicting characteristics. A series of complex processes are involved in alternative selection, ranking, and comparison. Multi-factor decision-making techniques are widely preferred for simplifying and streamlining the application and integrating different indicators [17]. In addition, although limited data and field information complicate flood damage assessments, advances in remote sensing, information technology, and GIS modeling tools make it possible to predict flood formation with high accuracy and precision [18,19,20].

Two approaches commonly stand out in flood sensitivity analyses: statistical and knowledge-based methods. Frequency Ratio (FR), Evidential Belief Function (EBF) and Shannon Entropy Index (SE) are two-variable techniques that are highly effective and widely accepted for fragility analysis [21,22]. Statistical methods such as the Frequency Ratio (FR) analyze the relationship between flood events and certain environmental factors based on historical data [23]. Shannon Entropy and Multi-Criteria Decision Analysis (MCDA) are common techniques used to objectively determine criterion weights in flood hazard mapping [24,25]. Thus, comparing these methods is essential for identifying the most suitable approach for a given basin.

Statistical modeling facilitates easier management of the process from inputs to outputs and enables the processing of large datasets in a short time in a GIS environment [26,27]. Statistically based spatial models based on various factors used in sensitivity analysis are increasingly being used to determine flood risk. These multi-criteria models are cheaper and faster to implement than traditional hydrological models and can reveal the general spatial distribution of risks across large areas [28]. The success of the flood maps created depends on a good understanding of the flood process, the determination of the effects of relevant factors, and the selection of an appropriate model [29]. Methods alone may be insufficient for estimating flood sensitivity due to the limited representation of input data. At this point, hybrid strategies overcome this problem by combining the advantages of different methods to produce more accurate results [30]. However, since there is no consensus on the best model, new models need to be developed and tested [31].

The Melen Basin research area has experienced numerous flood disasters throughout history due to its natural structure and characteristics. The largest of these was the flood disaster in 1961, which submerged 90 villages and agricultural lands, killed thousands of animals, and severed transportation links between Istanbul and Ankara. For this reason, although the Hasanlar Dam was constructed in Düzce province between 1965 and 1972 for flood control and irrigation purposes, flooding has continued to this day [32]. Despite the long history of flooding, the lack of a comprehensive flood inventory for the Melen Basin in recurring floods throughout history and the insufficient comparison of statistical approaches in disaster management efforts are significant factors. Accordingly, the novelty of this study lies in the selection of the classification method (natural, quantile, etc.) for the flood sensitivity map. In this context, a success index is defined to select the classification method that best predicts flood-prone and non-flood-prone points. This study aims to produce a flood susceptibility map with minimized bias using the EBF, FR, and SE methods. Consequently, the study is significant in terms of testing the performance of area-specific models using ROC curves and current flood data and comparing the results with previous studies.

2. Materials and Methods

2.1. Study Area

The study consists of four stages: identifying flood factors through literature review, compiling an inventory of past floods, applying models, and model validation. The methodology covering all stages of the study pertaining to the basin-based analysis is presented in Figure 1.

The Melen Basin is a sub-basin of the Western Black Sea Basin, one of Türkiye’s 25 river basins. The study area is located between 41°5′00″ and 40°40′00″ north latitude and 30°50′00″ and 31°40′00″ east longitude, covering an area of 2446 km² (Figure 2).

The Great Melen River, which flows into the Black Sea, flows through the Western Black Sea Region, including the provincial borders of Düzce, Bolu, and Sakarya. It is a sub-basin of the Western Black Sea Basin and covers 80% of the province of Düzce. The Akçakoca Mountains are located to the north of the basin, the Bolu Mountains to the east, and the western extensions of the Abant Mountains to the southeast and south [33].

According to the General Directorate of Meteorology, Düzce Province has a temperate and humid climate based on meteorological data for the period 1959–2025. The annual average temperature is 13.3 °C, and the annual average total rainfall is approximately 837 mm. The distribution of rainfall throughout the year and the occurrence of short-term heavy rainfall increase the region’s susceptibility to flooding.

The steep slopes in the basin and the flat areas at the bottom of the plain have formed alluvial fans. Low-gradient and meandering rivers create shallow beds, particularly around Lake Efteni, increasing the risk of flooding [34,35]. In the distribution of past floods by month, 23 of the 28 recorded flood events occurred between April and August, accounting for 82% of all disasters. It is understood that flood disasters are more closely linked to sudden rainfall during the spring and summer months [36].

2.2. Preparation of Flood Inventory Map

Flood inventory maps are used to record the location, date, and type of floods, thereby forming the basis for sensitivity analyses [37]. By analyzing the frequency of past flood events, the locations of floodplains, and the relationships between environmental factors, future flood risks can be predicted [38]. Flood inventory data also plays an important role in validating flood susceptibility maps, with flood conditioning factors used as independent variables [39]. The flood inventory was compiled by evaluating different data sources together. The flood area data used in the study was obtained from Sentinel-2 satellite images and other satellite images of the spread areas of floods that occurred in 2019, 2021, 2022 and 2023. To ensure the accuracy of the maps, the location points of structures identified as slightly damaged, moderately damaged, severely damaged, and destroyed as a result of damage assessments conducted by the Ministry of Environment, Urbanisation, and Climate Change following the aforementioned flood years were utilised. This approach aims to enhance the verifiability and reproducibility of inventory data. Since this point data model will not be used for training, it will be used entirely for testing purposes. Within this scope, 437 points with flooding and the same number of points without flooding were used for verification (Figure 3). To reduce potential spatial dependence between flood and non-flood samples, non-flood points were selected independently from the flood inventory polygons and were spatially separated from mapped flood locations. Among the 437 flood points, only 163 points fall within the delineated flood polygons, while the remaining flood samples are located outside these polygon boundaries, representing additional flood-affected locations identified from field evidence and ancillary sources. Therefore, the non-flood dataset is independent of both the flood polygons and flood point locations. Although a fully spatially blocked validation framework was not implemented, the adopted sampling strategy aimed to minimize spatial overlap and reduce bias related to spatial autocorrelation.

2.3. Flood Conditioning Factors

Understanding the formation of flash floods, examining flash flood forecasters, and selecting the appropriate ones is a critical step [40]). In this study, a literature review on flood sensitivity mapping (FSM) and the status of the study area was conducted, including elevation (E), slope (S), aspect (A), plan curvature (Pc), profile curvature (Prc), drainage density (Dd), distance to the river (Dri), curve number (Cn), land use, long-term average rainfall (R), normalized difference vegetation index (NDVI), lithology (L), soil depth (Sd), topographic wetness index (TWI), sediment transport index (STI), and stream power index (SPI).

Conditioning factors and digital elevation models were obtained from institutions’ open access data. Each flood conditioning factor was prepared using ArcGIS 10.7 software, and the factors obtained were reclassified using the natural breaks method. The visuals of the prepared flood conditioning factors are presented in Figure 4. The data sources from which the layers used in the flood sensitivity analysis were derived are detailed in

Table 1. Research database and derived data.

	Data Format	Data Source	Spatial Resolution	Derived Data
Digital elevation model (dem)	Raster	Alos Palsar DEM (https://search.asf.alaska.edu/, accessed on 16 February 2024)	12.5 × 12.5 m	Elevation, Slope, Aspect, Plan curvature, Profile curvature, TWI, SPI, STI
Landsat 8 image	Raster	National Academy of Sciences in the USA	30 × 30 m	Normalized Vegetation Difference Index (NDVI)
Sentinel 2 image	Raster	European Space Agency (ESA)	10 × 10 m	Land cover
Digital soil map	Vector	Ministry of Agriculture and Forestry	1/25.000	Curve number, Soil depth
Flows	Vector	EU-Hydro River Network Database	2.5 × 2.5 m	Distance from river, Drainage density
Rainfall		General Directorate of Meteorology		Long-term average precipitation,
Geology	Raster	General Directorate of Mineral Exploration and Research	10 × 10 m	Lithology

.

2.4. Evidential Belief Function (EBF) Model

The EBF model is a statistical method based on the two-variable Dempster-Shafer Theory (DST) that is widely used in mapping water resources and natural disasters [41]. DST is effective in modeling deviations from uncertainty and certainty, providing additional flexibility in hypothesis testing by defining uncertainty in probabilistic models [42].

Flood probability classification is performed based on the flood conditioning factors and the function equation result ranges. In the assessment, the occurrence of a flood is estimated using Bel (Degree of Belief) and Pls (Degree of Plausibility), while the non-occurrence of a flood is estimated using Dis (Degree of Disbelief) and Unc (Degree of Uncertainty) [11,27,43,44,45]. A high belt value indicates a high probability of flooding, while a low belt value indicates a low probability of flooding [22,46,47]. According to these indicators, the results in the function equations are evaluated in the range of 1–6 [22,41,48,49,50,51,52].

$${\mathit{B}\mathit{e}\mathit{l}}_{\mathit{C}\mathit{i}\mathit{j}}={\displaystyle \raisebox{1ex}{${\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{B}}$}\!\left/ \!\raisebox{-1ex}{$\sum _{\mathit{j}=\mathbf{1}}^{\mathit{m}}{\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{B}}$}\right.}$$

(1)

$${\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{B}}={\displaystyle \frac{\frac{\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}} \cap \mathit{D}\right)}{\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}\right)}}{\raisebox{1ex}{$\mathit{N}\left(\mathit{D}\right) - \left(\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}} \cap \mathit{D}\right)\right)$}\!\left/ \!\raisebox{-1ex}{$\mathit{N}\left(\mathit{T}\right) - \mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}\right)$}\right.}}$$

(2)

$${\mathit{D}\mathit{i}\mathit{s}}_{\mathit{C}\mathit{i}\mathit{j}}={\displaystyle \raisebox{1ex}{${\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{D}}$}\!\left/ \!\raisebox{-1ex}{$\sum _{\mathit{j}=1}^{\mathit{m}}{\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{D}}$}\right.}$$

(3)

$${\mathit{W}}_{\mathit{C}\mathit{i}\mathit{j}\mathit{D}}={\displaystyle \frac{\frac{\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}\right) - \mathit{N}({\mathit{C}}_{\mathit{i}\mathit{j}} \cap \mathit{D})}{\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}\right)}}{\raisebox{1ex}{$\mathit{N}\left(\mathit{T}\right) - \mathit{N}\left(\mathit{D}\right) - \left(\mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}) - \mathit{N}({\mathit{C}}_{\mathit{i}\mathit{j}} \cap \mathit{D}\right)\right)$}\!\left/ \!\raisebox{-1ex}{$\mathit{N}\left(\mathit{T}\right) - \mathit{N}\left({\mathit{C}}_{\mathit{i}\mathit{j}}\right)$}\right.}}$$

(4)

$$\mathit{U}\mathit{n}\mathit{c}=\left[\mathbf{1}-{\mathit{B}\mathit{e}\mathit{l}}_{\mathit{C}\mathit{i}\mathit{j}}-{\mathit{D}\mathit{i}\mathit{s}}_{\mathit{C}\mathit{i}\mathit{j}}\right]$$

(5)

$$\mathit{P}\mathit{l}\mathit{s}=\left[\mathbf{1}-{\mathit{D}\mathit{i}\mathit{s}}_{\mathit{C}\mathit{i}\mathit{j}}\right]$$

(6)

Equation abbreviations:

N(Cij Ո D): Flood pixel density in class D

N(Cij): The total number of floods in the area

N(D): Number of pixels in class D

N(T): The total number of pixels in the area

WCijB: The ratio of the conditional probability of B given Cij to the probability of the B condition

WCijD: The ratio of the probability of the condition that D does not exist given the existence of Cij to the probability of the condition D

2.5. Shannon Entropy Method

The Shannon Entropy method is a theoretical concept that measures uncertainty or randomness in a system and quantitatively shows how much information an event or variable contains [53]. This method is widely used in determining weight criteria for natural disasters such as landslides and floods [24]. The following equations are used in entropy weight calculations [54,55,56]:

$${\mathbf{P}}_{\mathbf{i}\mathbf{j}}={\displaystyle \raisebox{1ex}{${\mathbf{F}\mathbf{R}}_{\mathbf{i}\mathbf{j}}$}\!\left/ \!\raisebox{-1ex}{$\sum _{\mathbf{İ}=\mathbf{1}}^{\mathbf{m}}{\mathbf{F}\mathbf{R}}_{\mathbf{i}\mathbf{j}}$}\right.}$$

(7)

$${\mathbf{E}}_{\mathbf{j}}=\left({\displaystyle \frac{-\mathbf{1}}{{\mathbf{log}}_{\mathbf{2}}\left({\mathit{m}}_{\mathit{j}}\right)}}\right)\times \sum _{\mathbf{i}=\mathbf{1}}^{\mathbf{m}}{\mathbf{P}}_{\mathbf{i}\mathbf{j}}{\mathbf{log}}_{\mathbf{2}}{\mathbf{P}}_{\mathbf{i}\mathbf{j}}$$

(8)

$${\mathbf{W}}_{\mathbf{S}\mathbf{E}}^{\mathbf{j}}=\left(\mathbf{1}-{\mathbf{E}}_{\mathbf{j}}\right)\sum _{\mathbf{j}=\mathbf{1}}^{\mathbf{n}}\left(\mathbf{1}-{\mathbf{E}}_{\mathbf{j}}\right)$$

(9)

Equation abbreviations:

m: The number of subclasses for each factor

n: Total number of factors

Pij: Probability density

FRij: Frequency ratio

Ej: Entropy value

WJSE: Entropy weight

2.6. Frequency Ratio Method

The Frequency Ratio (FR) method is a bivariate analysis technique that reveals the spatial distribution of past flood events and the statistical relationship between factors affecting flood formation. With this method, the probability of a flood event occurring in a specific class of each factor is calculated by comparing it to the general distribution of that class in the area. The following equation is used in frequency ratio value calculations [57,58].

$$\mathit{R}={\displaystyle \frac{\left[\frac{\mathbf{N}\mathbf{p}\mathbf{i}\mathbf{x}\left(\mathbf{S}\mathbf{X}\mathbf{i}\right)}{\sum _{\mathit{i}=\mathbf{1}}^{\mathit{m}}\mathbf{S}\mathbf{X}\mathbf{i}}\right]}{\left[\frac{\mathbf{N}\mathbf{p}\mathbf{i}\mathbf{x}\left(\mathbf{X}\mathbf{j}\right)}{\sum _{\mathit{j}=\mathbf{1}}^{\mathit{n}}\mathbf{X}\mathbf{j}}\right]}}$$

(10)

Equation abbreviations:

Npix(SXi): The number of flood pixels in factor class i

Npix(SXj): The number of pixels in factor i

$$\mathbf{R}\mathbf{F}={\displaystyle \frac{\left(\mathbf{F}\mathbf{R}\mathbf{i}\mathbf{j}\right)}{\sum \mathbf{F}\mathbf{R}\mathbf{j}}}$$

(11)

Equation abbreviations:

FRij: The frequency ratio of class i for factor j

$\sum \mathrm{F}\mathrm{R}\mathrm{j}$: The sum of the frequency ratio values for factor j classes

$${\mathbf{F}\mathbf{S}\mathbf{M}}_{\mathbf{F}\mathbf{R}}={\displaystyle \frac{{\sum }_{\mathbf{i}=\mathbf{0}}^{\mathbf{n}}\mathbf{P}\mathbf{R}\mathbf{i}}{\mathbf{R}\mathbf{F}\mathbf{i}}}$$

(12)

Equation abbreviations:

FSM_FR: An FSM based on FR

PRi: The estimated ratio of factor j

RFi: The relative frequency of factor class i

n: Rhe number of factors

The FR value is a parameter for assessing the contribution of the relevant factor class to flood formation. If the FR value of the relevant class is greater than 1, it contributes more than the average; if it is less than 1, its contribution is considered to be lower than the average [59].

2.7. Ensemble Models

In addition to the three main methods used to minimise error rates in the research, these two hybrid methods, created by combining different models, were also applied in the field.

Firstly, the Evidential Belief Function (EBF) Model and the Frequency Ratio Model (FR) were combined. With this hybrid model, the weights of the factors were normalised within the 0–1 range using the prediction rate values from the frequency ratio method, and then combined with the subclasses of the EBF model to create a sensitivity map.

$${\mathrm{F}\mathrm{S}\mathrm{M}}_{\mathrm{E}\mathrm{B}\mathrm{F}-\mathrm{S}\mathrm{E}}={SE}_w\times {EBF}_{bel}$$

(13)

Equation abbreviations:

FRpr: The prediction rate value of the factor in the frequency ratio method

EBFbel: In the Evidential Belief Function method, the belief value

SEw: Rhe weight value of the factor in the Shannon Entropy method

Secondly, a hybrid method was applied that was used together in all three models

The following equation was used for the i’th class of each factor:

$$S_i=\sqrt{ {EBF}_i\times {FR}_i}$$

(14)

Equation abbreviations:

FRi: Normalised Frequency Ratio value (in the range 0–1)

EBFi: Normalised Evidential Belief Function value

Si: Rhe combined score for the relevant class

Thirdly, the prediction rate in the Frequency Ratio method and the factor weights in the Shannon Entropy method were normalised, and the following equation was used for each factor:

$$W_j={\displaystyle \frac{{PR}_j+{SE}_j}{2}}$$

(15)

Equation abbreviations:

PRj: The normalised prediction rate value obtained from the Frequency Ratio method,

SEj: The factor weight obtained using the Shannon Entropy method,

Wj: It is the combined weight of the factor.

$${\mathrm{F}\mathrm{S}\mathrm{M}}_{\mathrm{E}\mathrm{B}\mathrm{F}-\mathrm{F}\mathrm{R}}=\sum _{j=1}^n{(S}_{ij}\times W_j)$$

(16)

Equation abbreviations:

n: Number of factors,

Sij: The combined score of factor j’s i-th class,

Wj: The factor weight.

2.8. Selection of Appropriate Factors for Flood Sensitivity Analysis

Due to the diversity of factors affecting floods, ensuring the control of multiple linearity is a priority. Simplifying the model reduces complexity; otherwise, efficiency in calculations decreases and the risk of overfitting increases [55,60]. In this context, the Variance Inflation Factor (VIF) and Tolerance (TOL) are important indices used in identifying linear relationships [61].

The ‘Multicollinearity Test’ was applied in the study to select appropriate factors, and linear relationships between variables were avoided to ensure objective results. To determine the level of multilinearity among the factors affecting flood sensitivity in the field, 437 flood points were used.

A total of 874 points were identified, comprising 437 non-flooded points randomly selected from areas at least 800 m from streams, with an elevation above 310 m, drainage intensity below 1.2, slope gradient above 6.3, and lithological structure not consisting of alluvium. Thresholds have been determined by considering the relative frequency values in the frequency ratio method. When examining the relative frequency values of flood events in the study area, it is observed that they are generally concentrated in areas close to river channels, with low elevation, low slope, and widespread alluvial units. To verify the accuracy of these points, we conducted a visual examination using water spread, change, and availability maps obtained from the Global Surface Water dataset [62].

Using ArcGIS 10.7 software, the Extract Multi Values to Points tool in Spatial Analyst Tools was employed to extract data from each thematic layer. The VIF and Tolerance values of the factors were determined using SPSS 17.0 software based on the acceptance values. In terms of acceptance values, tolerance is the inverse of the variance inflation factor (VIF) (1 − R²). If the Variance Inflation Factor (VIF) value is above 5–10 or the tolerance value (TOL) is below 0.1–0.2 (R² ≈ 0.8–0.9), it is accepted that there is a multicollinearity problem between the variables [63]. TOL and VIF values are calculated using the following equation [64].

$${\mathbf{T}\mathbf{O}\mathbf{L}=\mathbf{1}-\mathbf{R}}_{\mathbf{v}}^{\mathbf{2}}$$

(17)

$$\mathbf{V}\mathbf{I}\mathbf{F}={\displaystyle \frac{\mathbf{1}}{\mathbf{T}\mathbf{O}\mathbf{L}}}$$

(18)

The tolerance value (TOL) in the equation ${\mathrm{R}}_{\mathrm{V}}^2$ represents the coefficient of determination of the regression of the explanatory variables on all other explanatory variables. If the Spearman correlation between the effect factor pairs exceeds a value of 0.7, it is considered to be ‘high linearity’ [65].

2.9. Classification of Flood Sensitivity Maps

Flood sensitivity is determined through analyses based on multiple factors, yielding different layers and, consequently, maps. According to the analysis results, grouping has been performed based on the acceptance of specific value ranges. The purpose of these classifications is to ensure that the results are clearer and easier to interpret [66]. If the histogram of the flood sensitivity map is uniform, equal interval and quantile methods are recommended; if it is symmetric, standard deviation is recommended; if it is right-skewed or left-skewed, the geometric interval method is recommended; if it is bimodal, the natural breaks method is recommended [67]. Within this scope, there are five main methods used in the ArcGIS environment [67,68,69]:

Natural breaks (Jenks) method: Classes are based on natural groupings found in the data. Group boundaries are statistically determined when there are relatively large jumps in sensitivity data values.

Quantile method: Homogeneous areas are created by assigning the same number of cells to each class. When sensitivity values are not evenly distributed, these areas are divided into intervals of equal size. This classification method is highly suitable for linearly distributed data. When data is scattered, data with large numerical differences can be classified by placing them in the same category.

Equal interval method: The equal interval method facilitates analysis by dividing the data range into classes of equal size. However, if the data distribution is not homogeneous, some classes may be overly dense, while others may be empty or contain very little data.

Geometrical interval method: A technique used in classifying continuous data that strikes a balance between equal interval, natural breaks, and quantile methods. By determining class intervals in a balanced and consistent manner, it takes into account both changes in intermediate values and extreme values, thereby producing smoother and more detailed maps.

The natural breaks method is used when there are significant jumps in the data values [70,71,72]. If the data distribution is close to normal, equal interval or standard deviation-based methods should be preferred for classification. However, if the distribution is skewed positively or negatively, quantile or natural break-based classifiers are more appropriate [73,74].

In order to determine the most appropriate classification method to be applied in this study, a histogram analysis was first conducted to examine the distribution of the data. According to these classification methods, a total of 874 points were identified, comprising 437 flood points and 437 non-flood points selected at random. Flood sensitivity map data has been grouped into very low, low, moderate, high and very high categories, and the percentage distribution falling into these classes has been presented. According to the success index equation, the method with the lowest absolute value among the differences obtained was determined, and the analyses were classified accordingly. In the success index equation, the higher the result (D), the more successful the classification method is considered to be.

$$D=100-\left|P_{flood}^{m+h+vh}-P_{noflood}^{l+vl}\right|$$

(19)

Equation abbreviations:

${\mathrm{P}}_{\mathrm{f}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}}^{\mathrm{m}+\mathrm{h}+\mathrm{v}\mathrm{h}}$: Percentage of medium+, high+ and very high classes at points affected by flooding

${\mathrm{P}}_{\mathrm{n}\mathrm{o}\mathrm{f}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}}^{\mathrm{l}+\mathrm{v}\mathrm{l}}$: The percentage of very low + low classes in non-flooded areas

2.10. Validation of Flood Susceptibility Maps

A single criterion is not sufficient for validating the performance of flood sensitivity models. The validity and superiority verification of the classification results is calculated using four different reliable indices [75]. The performance of sensitivity maps can be determined using the TP (true positive), TN (true negative), FP (false positive) and FN (false negative) values and the equations for ‘Specificity, Sensitivity, Accuracy, Precision, Recall’ [76].

These assessments were carried out in the study and, in addition, the accuracy of the prediction models was measured during the analysis phase. The Receiver Operating Characteristic (ROC) curve was created and the areas under the curve (AUC) were determined. The area under the ROC curve, or AUC, defines the probability of a correct prediction. It is a measure of the model’s success in distinguishing between correct and incorrect pixels [22]. It is a two-dimensional curve with true-positive rates (sensitivity) on the x-axis versus false-positive rates (1–specificity) on the y-axis [77]. In the evaluation, the area under the curve (AUC) value, which measures the model’s ability to correctly distinguish flood formation, ranges between 0.5 and 1. It is accepted that as the AUC value approaches 1, the accuracy of the model increases, and as it approaches 0.5, the predictive power of the model weakens [18,78].

$$\mathbf{S}\mathbf{p}\mathbf{e}\mathbf{c}\mathbf{i}\mathbf{f}\mathbf{i}\mathbf{t}\mathbf{y}={\displaystyle \frac{\mathbf{T}\mathbf{N}}{\mathbf{F}\mathbf{P}+\mathbf{T}\mathbf{N}}}$$

(20)

$$\mathbf{S}\mathbf{e}\mathbf{n}\mathbf{s}\mathbf{i}\mathbf{t}\mathbf{i}\mathbf{v}\mathbf{i}\mathbf{t}\mathbf{y}={\displaystyle \frac{\mathbf{T}\mathbf{P}}{\mathbf{F}\mathbf{N}+\mathbf{T}\mathbf{P}}}$$

(21)

$$\mathbf{A}\mathbf{c}\mathbf{c}\mathbf{u}\mathbf{r}\mathbf{a}\mathbf{c}\mathbf{y}={\displaystyle \frac{\mathbf{T}\mathbf{N}+\mathbf{T}\mathbf{P}}{\mathbf{F}\mathbf{P}+\mathbf{T}\mathbf{P}+\mathbf{F}\mathbf{N}+\mathbf{T}\mathbf{N}}}$$

(22)

$$\mathbf{P}\mathbf{r}\mathbf{e}\mathbf{c}\mathbf{i}\mathbf{s}\mathbf{i}\mathbf{o}\mathbf{n}={\displaystyle \frac{\mathbf{T}\mathbf{P}}{\mathbf{F}\mathbf{P}+\mathbf{T}\mathbf{P}}}$$

(23)

Equation abbreviations:

P: Total number of pixels affected by flooding

N: Total number of pixels without flooding

TP: True positive values

TN: True negative values

FP: False positive values

FN: False negative values

3. Results

3.1. Multicollinearity Analysis of Factors Affecting Flooding

In Multicollinearity Analysis, the Variance Inflation Factor (VIF) and Tolerance Values (TOL) were determined, and the Pearson Correlation Test (PCC) was applied. According to the analyses, it was determined that the VIF values of all factors were below 5 and the TOL values were above 0.1. Among the flood conditioning factors, the Variance Inflation Factor (VIF) value is highest for the Stream Power Index (SPI), Topographic Wetness Index (TWI), and Slope (S), and lowest for Aspect (A), Sediment Transport Index (STI), and Rainfall (R) factors (

Table 2. Multicollinearity assessment of flood conditioning factors.

Conditioning Factors	Variance Inflation Factor Value (VIF)	Tolerance Value (TOL)
Rainfall (R)	1.090	0.917
Land cover (Lc)	1.645	0.608
Normalized difference vegetation index (NDVI)	1.508	0.663
Topographic wetness index (TWI)	2.815	0.355
Lithology (L)	1.448	0.690
Soil depth (Sd)	1.696	0.590
Elevation (E)	2.331	0.429
Sediment Transport Index (STI)	1.068	0.937
Drainage density (Dd)	2.097	0.477
Stream Power Index (SPI)	2.822	0.354
Distance to river (Dri)	1.786	0.560
Slope (S)	2.722	0.367
Curve number (Cn)	1.252	0.799
Aspect (A)	1.051	0.951
Profile curvature (Prc)	1.484	0.674
Plan curvature (Pc)	1.848	0.541

).

According to the Pearson Correlation Test, the highest correlations are between drainage density (Dd) and distance to river (Dri), normalized difference vegetation index (NDVI) and land cover (Lc), and profile curvature (Prc) and aspect (A), respectively. According to the Correlation Test Matrix, it has been accepted that there is no “multicollinearity” problem among the independent variables in this range of values (Figure 5).

3.2. Selecting the Classification Method

Classification methods in flood sensitivity mapping significantly affect the success rate of the applied model. Selecting a classification method with a high success rate is critical. In validating the method, it is important to include non-flooded points in the predictions equal to the percentage of predicted flooded points. For this reason, the success index (Equation 16) was applied in selecting the classification method specifically for the Melen Basin research area. Prediction percentages for 437 flood-prone points and 437 non-flood-prone points were determined according to five classification methods (Figure 6 and Figure 7,

Table 3. Success index of classification method based on Evidential Belief Function (EBF), Frequency Ratio (FR) and Shannon Entropy (SE).

No	Method	Classification Method	Flood	Non-Flood	Success Index
1	Evidential Belief Function (EBF)	Natural Breaks	90.6	98.8	91.8
		Quantile	96.1	62.2	66.1
		Geometrical	71.8	100	71.8
		Equal	96.1	62.7	66.6
2	Frequency Ratio (FR)	Natural Breaks	79.4	97.5	81.9
		Quantile	97.7	61.4	63.7
		Geometrical	97.7	66.1	68.4
		Equal	77.6	100	77.6
3	Shannon Entropy (SE)	Natural Breaks	91.9	97.5	94.4
		Quantile	96.3	61.2	64.9
		Geometrical	95.9	66.1	70.5
		Equal	70.3	100	70.3
4	Evidential Belief Function (EBF)–Shannon Entropy (SE)	Natural Breaks	91.9	95.9	96.0
		Quantile	95.9	60.4	64.5
		Geometrical	95.8	61.6	65.8
		Equal	64.7	99.5	65.2
5	Evidential Belief Function (EBF)–Frequency Ratio (FR)	Natural Breaks	79.4	100	79.4
		Quantile	95.9	64.4	68.5
		Geometrical	95.9	63.6	67.7
		Equal	76.9	100	76.9

).

As a result, the Natural Breaks classification method has a high success index in the EBF, FR, SE, EBF–SE, and EBF–FR methods. The highest success index was achieved in the EBF, FR, EBF–FR, SE, and EBF–SE methods, respectively. The success index of the Natural Breaks classification method is highest in the Evidential Belief Function (EBF), Frequency Ratio (FR), and Shannon Entropy (SE) methods.

3.3. Flood Conditioning Factors, Weight and Frequency Values

The weights and frequencies of factors contributing to flood formation are important parameters. The Frequency Ratio (FR) method was used in the study to determine the relationship between flood events and flood conditioning factors.

Frequency ratio values have been calculated for each flood conditioning factor class (Figure 7). Additionally, the Relative Frequency (RF) values of factors and subclasses have been determined based on all flood classification methods.

When comparing the weights of the factors in the Shannon Entropy and Frequency Ratio methods, it is observed that their importance rankings are similar. The most important factors in the Shannon Entropy method are, in order, Elevation (E), Lithology (L), Slope (S), and Drainage density (Dd). In the Frequency Ratio method, the same ranking was obtained, and it was determined that the importance values of the Litology (L) and Slope (S) factors were the same (0.106) (Figure 8 and Figure 9).

3.4. Melen Basin Flood Susceptibility Maps

In flood sensitivity mapping, sensitivity values have been evaluated and classified into five categories: very low, low, moderate, high, and very high.

Table 4. Proportional distribution of flood-prone areas by criteria and value classes based on different methods.

Method	Criterion	Flood Sensitivity Value Classes
		Very Low	Low	Moderate	High	Very High
Frequency Ratio (FR)	Value range	0–0.137	0.138–0.325	0.326–0.514	0.515–0.682	0.683–1
	Area ratio (%)	69	14	4	12	3
Shannon Entropy (SE)	Value range	5.505–9.567	9.568–13.63	13.64–20.29	20.3–27.77	27.78–46.94
	Area ratio (%)	34	37	13	10	6
Evidential Belief Function (EBF)	Value range	0.712–1.368	1.369–2.114	2.115–3.516	3.517–4.977	4.978–8.318
	Area ratio (%)	43	29	13	11	5
Evidential Belief Function (EBF)–Shannon Entropy (SE)	Value range	0.0537–0.0924	0.0925–0.133	0.134–0.198	0.199–0.276	0.277–0.482
	Area ratio (%)	32	39	13	11	5
Evidential Belief Function (EBF)–Frequency Ratio (FR)	Value range	0.019–0.0992	0.0993–0.208	0.209–0.321	0.322–0.424	0.425–0.62
	Area ratio (%)	68	14	4	12	2

shows that the spatial distribution of flood sensitivity classes differs significantly depending on the method used. The FR and EBF–FR models concentrated a large portion of the study area in the ‘very low’ sensitivity class (69% and 68%), allocating a more limited number of areas to the “high” and ‘very high’ classes. In contrast, the SE model produced a more balanced distribution among sensitivity classes; the ‘low’ (37%) and ‘very low’ (34%) classes were particularly prominent, while the proportion of the ‘very high’ class (6%) was higher than with the other methods. The EBF model exhibited a moderate distribution, concentrating on the low classes (43–29%) while allocating more areas to the high sensitivity classes. Among the hybrid approaches, the EBF–SE model represented the low sensitivity class at the highest rate (39%), while the EBF–FR model showed a pattern similar to the FR model.

Flood susceptibility maps were created separately for each method based on the success index (Figure 10).

3.5. Model Validation Assessment

The classification performance of the three main methods and two hybrid models used in the study was evaluated using the Area Under the Curve (AUC) method (Figure 10) and various statistical metrics and success criteria (

Table 5. Comparison of model performance based on parameters.

		Primary Methods			Hybrid Models
No	Parameter	EBF	FR	SE	EBF–FR	EBF–SE
1	Tp	396	347	402	347	340
2	Tn	432	437	426	437	437
3	Fp	41	90	35	90	97
4	Fn	6	0	11	0	0
5	Sensitivity	0.99	1.0	0.97	1.0	1.0
6	Specifivity	0.91	0.83	0.92	0.83	0.82
7	Accuracy	0.95	0.90	0.95	0.90	0.89

). According to these accuracy values, the most successful model is the Evidential Belief Function (EBF) method.

The success values of the FR, EBF–FR (0.90) and SE, EBF–SE (0.89) models are seen to be very close. All models are successful in terms of sensitivity values. The model with the highest specificity, accuracy, and precision values is the Evidential Belief Function (EBF).

In terms of the success index ratio, the AUC value of 0.979 makes the SE method the most efficient. According to the ranking, this is followed by the FR method and then the EBF–SE, EBF–FR, and EBF methods. As a result, it can be seen that all applied models are successful and that the SE method is more successful than the others (Figure 11).

4. Discussion

Changes in land use and unplanned construction affect the distribution of flood hazards; assessing the flood sensitivity of watersheds, climate, and sensitive areas contributes to preventing loss of life and property [79].

In terms of flood sensitivity factors, the Frequency Ratio (FR) and Shannon Entropy (SE) methods are the most effective for elevation factors, and this result has been demonstrated in many studies [55,80,81,82]. The Relative Frequency (RF) value in the lower class of the elevation factor is 97.54, belonging to the class with the highest value (33–310). The lower class of the slope factor (0–6.30) has the highest RF ratio at 94.15. Accordingly, the finding that low-lying and gently sloping areas are more susceptible to flooding is consistent with previous studies [11,83,84,85,86]. Another effective conditioning factor in the frequency ratio method is lithology. The fact that the Melen Basin center is an alluvial plain and that most of the floods occur in the plain highlights this factor. The importance of the lithology factor is also supported by previous studies [38,87].

According to the frequency ratio method, the elevation criterion is the most important criterion in flood formation, followed by slope, lithology, and drainage density. In the Shannon Entropy (SE) method, elevation is again the most important criterion. Therefore, the ranking of the factors is the same for the first four factors in both methods, and these factors have also been emphasized in previous studies [21,26,88,89,90].

In flood sensitivity mapping, it is important to pay close attention to which method is used for classification, as well as the methods themselves [70]. In the selection of classification methods, the Natural Breaks method has been found to be the most successful classification method. This result is consistent with the method preferred in other studies in the literature [91,92,93,94,95,96]. However, the process of selecting the classification method to be applied in mapping in the study constitutes the original part of the work and, in this respect, differs from the existing literature.

In the study, the Shannon Entropy (SE) model had the highest success rate in terms of AUC value, while the Frequency Ratio model ranked second. This finding is consistent with the results of previous studies [46,97]. In the study, individual models (FR, SE) were more successful than mixed models, similar to the results of Ghosh’s (2022) study [98].

The results indicate that the Geometrical and Quantile classification methods tend to show wider flood spread areas. While this situation enables the successful prediction of flooded areas using these methods, it leads to relatively low performance in the accurate classification of non-flooded areas. In contrast, the Natural Break method was able to distinguish between flooded and non-flooded areas more evenly and showed the highest success rate in correctly predicting both classes. This finding highlights the importance of a balanced approach in selecting classification methods for flood susceptibility maps.

The determination of flood inundation areas was based on disaster-prone area decisions announced by the Disaster and Emergency Management Presidency, using Google Earth and Sentinel satellite imagery. However, these boundaries may sometimes be set more broadly for security purposes. Similarly, Sentinel satellite images taken several days after the flood event or due to cloud cover may not accurately identify the flood spread areas. These circumstances determine the sensitivity limits of the study. These gaps can be filled by subsequent studies using drone images taken during or after the flood, information gathered from local residents, field observations, and on-site assessments.

Uncertainty Analysis and Limitations

Various sources of uncertainty should be considered when interpreting the results of this study. Temporal changes in land use and climatic conditions may also cause flood susceptibility patterns to change in the future. Therefore, the results obtained represent current conditions and should not be considered as definitive long-term predictions.

Sentinel-2’s 5-day revisit interval may limit its ability to record key stages of fast-evolving hydrological events, particularly flash floods [99]. Furthermore, cloud cover during the hours and days when the flood reaches its peak level also limits the ability to identify the floodplain.

A limitation of this study is that spatially blocked validation was not conducted; therefore, some degree of spatial autocorrelation may still affect the reported performance metrics. Future research may apply event-wise or block-wise validation frameworks to further assess model transferability across independent regions.

The FR, SE, and EBF models depend on historical flood inventory data for calibration and validation. In regions with limited records, insufficient data can lead to uncertain results and reduce the applicability of these models [100]. The use of higher resolution data in future studies, the creation of longer-term flood records, and the comparison of different models may contribute to reducing uncertainties.

5. Conclusions

This study has demonstrated that flood-prone areas in the Melen Basin can be reliably identified using statistical and hybrid methods to mitigate the destructive effects of flood disasters on residential areas. As a result of this study, different statistical and hybrid approaches for determining flood sensitivity in the Melen Basin were compared, and the scenario providing the highest prediction accuracy was identified. Evaluations conducted among models created by integrating the EBF, FR, and SE methods showed that the SE (AUC = 0.979) and FR (AUC = 0.974) methods demonstrated higher performance than hybrid methods according to ROC/AUC analysis.

The flood susceptibility maps obtained were tested using different classification methods, and it was proven that the Natural Break classification was the most successful method in all cases. According to the flood susceptibility maps, flood susceptibility was found to be significantly high in the middle and lower sections of the Melen Basin. In particular, low-gradient flat areas and areas with increased settlement pressure have been identified as the most sensitive areas.

According to the results of the sensitivity map, the FR and EBF–FR models grouped most of the area into the ‘very low’ class, while the SE and EBF-based approaches presented a more balanced distribution, highlighting the high sensitivity areas more clearly.

The susceptibility maps produced provide decision-makers with a scientific basis for risk reduction, land use planning, and sustainable watershed management at the regional scale. Testing this approach comparatively with machine learning-based models in future studies will contribute to the development of the method.