Flood Hazard Mapping Using the Flood and Flash-Flood Potential Index in the Buzău River Catchment, Romania

Round 1

Reviewer 1 Report

The paper is well-written and presents an interesting topic with scientific method. Methodologies and references fill this topic and result to be relevant. In my opinion such a work can be considered for publication. I suggest only revisiting the manuscript to avoid typos.

I am very sorry for the delay in the revision.

Regards

Author Response

Thank you for the constructive comment. We reviewed the manuscript and we hope we improved the overall quality.

Reviewer 2 Report

This manuscript presents a new approach to create flood hazard maps based on two new index (based on geo-spatial data). Generally speaking, the approach/method and results would be of interest to readers of Water, however, the manuscript lacks many essential components that must be included before it is suitable for publications:

Insufficient literature review: the authors do not provide sufficient review of previous work in the field. There are numerous, recent papers on similar approaches to flood hazard mapping (many in MDPIs database). The authors must frame their approach in the context of existing work, to demonstrate the gap in current research, and why their particular approach is necessary and useful. Specifically, the introduction section (where many paragraphs are only a single sentence long) only briefly mention previous work, but this needs to be expanded to address the issues mentioned above. Methods: the author's claim (line 89) that the river catchment is representative of the entire Romanian watershed system: they must provide evidence and justification for this claim.  Methods: more detail is needed for every aspect of the method; it is very unbalanced in its current form. The authors provide the equation for the circularity ratio (which is of minor importance) but very few other equations are provided (especially for the ANNs). Methods: The DEM resolution seems too coarse to be useful for a flash flood index. The authors should discuss the impact of using this data on the results (in terms of suitability and uncertainty). The method for the ANN application is incomplete and needs further refinement. Why did the authors choose a 70-30 ratio for training-testing? Can the authors demonstrate that both subsets had similar statistical properties? How was the training stopped? Was an early-stopping procedure used? The authors should explore different combinations of this datasplit? Was the subset randomly sampled or block sampled? Which training algorithm was used? I would recommend the authors use a "multi-start" training approach where the ANN is retrained x1000 times to get an ensemble of models rather than a single model (this is best practice for ANN use). A single model will result in deterministic results (likely in a local minima) which limits the generalisability of the approach. A binary output is used for the ANN: is a simple MLP the best approach for this system? A categorical ANN might be better. No model performance results are presented for the ANN - this is necessary for any modelling approach. More detail should be provided on why each variable was split into four classes. What is the impact of arbitrarily choosing each class? More detail is needed on how the spatial association between variables is determined - this is not clear. The authors should also investigate the correlation between input variables.  The weights of the classes (Eqn 4): how were the weights determined. The authors should present a formal analysis on how the weights were determined, e.g., pairwise comparison. Unclear how "variable importance" (in Fig 8) is determined. This should be explained in detail in the methods. Similarly, the pseudo-probabilities in Fig 9 need an explanation. Tables 2 and 3 are too long and repeat much of the information presented in the text. The authors use "flood risk" and "flood hazard" interchangeably in the manuscript: they should only use hazard based on their method. No risk is being calculated. The discussion section is too short and should be expanded. The manuscript is very poorly written and presented. Numerous grammatical issues exist. Many of the figures in the text are not referenced properly (see for example, Line 112).

Author Response

Insufficient literature review: the authors do not provide sufficient review of previous work in the field. There are numerous, recent papers on similar approaches to flood hazard mapping (many in MDPIs database).

 

Thank you for the constructive comment. We have added more recent and relevant papers.

 

The authors must frame their approach in the context of existing work, to demonstrate the gap in current research, and why their particular approach is necessary and useful. Specifically, the introduction section (where many paragraphs are only a single sentence long) only briefly mention previous work, but this needs to be expanded to address the issues mentioned above.

 

Thank you for the observation, we reviewed the Introduction section, therefore we have added the following paragraphs:

Line 42: European rivers are specifically analyzed in accordance with the European Flood Directive 2007/60 and the Directive 2008/94/EC of the European Parliament and of the Council, published in the Official Journal of the European Union, whilst statistical, hydraulic and GIS techniques are used for hazard and flood mapping [4–6].

Line 48: Climate change impact studies on flood risk are mostly conducted at a river basin or regional scale [8,9].

Line 66: Various methods are commonly used to map flood sensitivity. Recent methods such as multicriteria evaluation [16], decision tree analysis (DT) [17], fuzzy theory [18,19], weight of samples (WoE) [20], artificial neural networks (ANN) [21–23], frequency ratio (FR) [24] and logistic regression (LR) approaches [25], have been widely used by many researchers.

 

Methods: the author's claim (line 89) that the river catchment is representative of the entire Romanian watershed system: they must provide evidence and justification for this claim.

 

Thank you for the observation. The methodology presented in the study was applied for the Buzau river catchment, but it could be used for any European river catchment, with some exceptions (areas with limestone or strongly anthropized river catchments).

 

Methods: more detail is needed for every aspect of the method; it is very unbalanced in its current form.

 

Thank you for the constructive observation. We have detailed the methods proposed in the present study. More detail on the methods will be found in the answers to the following questions.

 

The authors provide the equation for the circularity ratio (which is of minor importance) but very few other equations are provided (especially for the ANNs).

 

We reviewed the manuscript and we have detailed the methods used in the study, respecting the observations. More detail on the methods will be found in the answers to the following questions.

 

Methods: The DEM resolution seems too coarse to be useful for a flash flood index. The authors should discuss the impact of using this data on the results (in terms of suitability and uncertainty).

 

The present study uses open-source data. The data obtaining process represents an important obstacle for researchers which aim at developing new methodologies. We consider that using a 25m resolution, it does not affect the relevance of the obtained results. We discussed the impact of using open-source data in the Conclusions section. Thus, we added the following paragraph:

Previous studies [80–82] analyze and present flood forecasts at a resolution of 100 m, but in order to determine and validate the areas prone to this natural hazard it is crucial to have data at a high resolution [47]. Similar approaches have been tested using satellite imagery at different spatial resolutions [83], alongside with various image processing techniques. Such approaches show a great potential in the areas where ground observations are rare or lacking.

 

The method for the ANN application is incomplete and needs further refinement.

 

Thank you for the observation. We reviewed the theoretical component and the applicability of method, therefore we added the following paragraphs:

Line 311: The Multilayer Perceptron is an artificial neural network (ANN) used in function approximation and pattern recognition and is made up of three components (Figure 5) [66]. Artificial Neural Networks represent a simple way to mimic the neural system of the human brain, in which through various samples, in this case the training samples, one can recognize data unseen before, take decisions and solve problems regarding the spatial relationship / association between input variables and the presence or absence of a certain phenomenon [34,67,68]. An MLP is based on the backpropagation algorithm, a supervised learning technique [66,69]. The neurons, represented by the variables/factors used in the analysis, are knows as Input layers and are connected to the Hidden layers through a neural connection which holds the weights of the hidden layers. The connection of the input and output layers with each neuron of the hidden and output layers is represented through the following equation (3) [70]:

where Nh represents the neurons in the hidden layer, ω_ij(¹) represents the weight of the connection between the neuron x_i and the input layer and the neuron of the second layer, ω_o(⁰) is the bias variable which prevents the parameter a_j from becoming the value zero.

In order to avoid the overfitting of the neural network, the selection of the number of hidden neurons represents an important step as it controls the accuracy level of the network, proportionate to the noise level [76,77]. The number of neurons is determined based on the following equation (4):

where N_h represents the number of hidden neurons and V represents the number of flood or flash-flood conditioning variables. Thus, the FPI-MLP model has 29 hidden neurons, whilst the FFPI-MLP has 27 hidden neurons. Based on the weight of each connection, the output layers generate an output decision with the values of 1 and 0. The output decisions are determined as follows (5):

where O_d is the output decision, ωj and ω0 represent the connection weights and h is the hidden layer.

 

Line 412: After the FR of each flood or flash-flood conditioning variable was computed and normalized, each factor was reclassified based on the Ratio + values. The values were normalized using the following equation (6):

where nv is the standardized value, v represents the used variable, r is the limit of the range value and l is the limit of the standardization range.

The role of hybrid models is to develop more accurate methods and reduce the potential disadvantages of the more traditional methods.

 

Why did the authors choose a 70-30 ratio for training-testing? Can the authors demonstrate that both subsets had similar statistical properties?

 

Thank you for this comment. As there isn’t an ideal split ratio and as many other studies propose the use of a 70 – 80 % for the training data and 20 – 30 % for the testing data, we chose the 70 – 30 % split ratio which is a practice often used in Machine Learning or in similar research (Costache, R. Flash-flood Potential Index mapping using weights of evidence, decision Trees models and their novel hybrid integration. Stoch. Environ. Res. Risk Assess. 2019, 33, 1375–1402., Cao, C.; Xu, P.; Wang, Y.; Chen, J.; Zheng, L.; Niu, C. Flash Flood Hazard Susceptibility Mapping Using Frequency Ratio and Statistical Index Methods in Coalmine Subsidence Areas. Sustainability 2016, 8, 948.)

 

How was the training stopped? Was an early-stopping procedure used? The authors should explore different combinations of this datasplit? Was the subset randomly sampled or block sampled? Which training algorithm was used? I would recommend the authors use a "multi-start" training approach where the ANN is retrained x1000 times to get an ensemble of models rather than a single model (this is best practice for ANN use). A single model will result in deterministic results (likely in a local minima) which limits the generalisability of the approach. A binary output is used for the ANN: is a simple MLP the best approach for this system? A categorical ANN might be better. No model performance results are presented for the ANN - this is necessary for any modelling approach.

 

Thank you for your constructive observations and recommendations. We retrained the Neural Network using a multi-start approach, 1000 maximum training epochs, 30 validation thresholds and a gradient descent optimization algorithm.

Line 373: The MLP model for each index was trained using 1000 maximum training epochs and 30 validation thresholds. Each model used a multi-start approach, which consists in running multiple models in parallel. The MLP model used in the present study used a gradient descent optimization algorithm and a sigmoid activation function. The backpropagation algorithm is using the gradient descent to look for the minimum function each in weight space [78]. The sigmoid activation function used for the backpropagation algorithm is defined as follows, where c is an arbitrarily selected constant and 1/c is its reciprocal which is known as the temperature parameter [78] (7):

The overall classification accuracy for the FPI-MLP is 90.91% whilst for the FFPI-MLP is 86.03%. The percentage of correct observations per predicted values (0 - non-flood / non-torrential areas; 1 – flood / torrential) for the FPI-MLP are 86.41% for the non-flood areas and 90.52% for the flood areas whilst for the FFPI-MLP the percentages are 82.35% for the non-torrential areas and 89.72% for the torrential areas.

More detail should be provided on why each variable was split into four classes. What is the impact of arbitrarily choosing each class? More detail is needed on how the spatial association between variables is determined - this is not clear.

 

The variables have been classified as such as similar approaches have been used in previous research and offered good results. The final maps have been classified as well into four classes. By using 5 classes, we have noticed that the coverage of the very low class was insignificant, therefore we decided to group the very low class with the low class.

 

The authors should also investigate the correlation between input variables.  The weights of the classes (Eqn 4): how were the weights determined. The authors should present a formal analysis on how the weights were determined, e.g., pairwise comparison. Unclear how "variable importance" (in Fig 8) is determined. This should be explained in detail in the methods.

 

Thank you for the constructive comment. The weights were determined using the Pairwise comparison matrix. We added the Pairwise matrix for both indices in order to support our affirmation. In this context, the variable importance is the weight of each variable used in the analyze.

 

Pairwise matrix FPI

 

Pairwise matrix FFPI

 

Similarly, the pseudo-probabilities in Fig 9 need an explanation.

Thank you for your observation. We have attached the pseudo-probabilities for both indices. We consider that adding the following images would make the article a bit more difficult to follow.

 

 

Tables 2 and 3 are too long and repeat much of the information presented in the text.

 

Thank you for the constructive comment. We chose to leave tables in the current form as we wanted to provide an in-depth overview of the resulted parameters of each variable used in the study. Even though some of the information repeats, the variables provide different results.

 

The authors use "flood risk" and "flood hazard" interchangeably in the manuscript: they should only use hazard based on their method. No risk is being calculated.

 

Thank you for the comment, we have changed the terminology. This “confusion” was made as the terms hazard and risk refer to the same thing in our native language.

 

The discussion section is too short and should be expanded.

 

We have reviewed the discussion section and we added the following paragraphs:

Previous studies [80–82] analyze and present flood forecasts at a resolution of 100 m, but in order to determine and validate the areas prone to this natural hazard it is crucial to have data at a high resolution [47]. Similar approaches have been tested using satellite imagery at different spatial resolutions [83], alongside with various image processing techniques. Such approaches show a great potential in the areas where ground observations are rare or lacking. We consider that using a 25m resolution, it does not affect the relevance of the obtained results.

The outputs of the analysis, make this study relevant as other studies [32,35,36,38,40,47,72] propose the computation of only one index for creating hazard maps. Thus, the developed models constitute a support for assisting the decisions taken regarding the management and the elaboration of public policies which aim at mitigating natural risks.

The obtained results show the need to complete similar approaches [47,72,84] with new variables, which will increase the relevance of the advanced modelling techniques.

 

The manuscript is very poorly written and presented. Numerous grammatical issues exist.

 

Thank you for the constructive comment. We reviewed the manuscript and we hope we improved the overall quality.

 

Many of the figures in the text are not referenced properly (see for example, Line 112).

 

Thank you for the observation, we reviewed the figure referencing. The error occurred due to an error of the text editor which automatically referenced the figures.

Round 2

Reviewer 2 Report

The author's have made several revisions based on my initial review which has improved the manuscript. However, I do think there are a few items that have not been addressed in the revised version:

training-testing split: the author's do not provide any information if the data in each subset have similar statistical properties. If the test dataset is not representative of the training, there are limits to the applicability of the method. While one reference to a previous study that uses the 70%-30% split, the author's should note that this ratio is not easily generalisable and should be checked for every study/dataset. additional details on ANN: the author's have provided basic information on the ANN use as requested. However, I had recommended using a multi-start approach and the author's response to this request is confusing: the number of epochs is not the same as the number of multi-start iterations. Epochs typically refers to the number of trials needed to converge the backpropagation algorithm. Multi-start refers to re-running the training using a different initial weights and biases. If the number of epochs is indeed 1000, why was there a need for 30 "validation thresholds"? It is not clear how the training stopped due to the unclear description of the method. If a multi-start approach was indeed used, the results are not included in the revised manuscript. How does the multi-start approach impact the results? the author's response references some figures, however, these were not correctly included in the form and thus, I was unable to review them. Similarly, the information related to the "pairwise" comparison in the author's response is missing. No reference to this pairwise approach can be found in the manuscript. 

Author Response

Dear reviewer, I have attached below the follow up to your comments and suggestions. I thank you for them and hope that my article will receive your approval.

Author Response File: Author Response.docx

Round 3

Reviewer 2 Report

Thank you for the revisions.

Ideally, the pairwise information should be in the manuscript. I only see the following sentence: "The resulted weights for each index were compared and validated through a Pairwise comparison matrix." and not the tables provided in the response.