Multistep Flood Inundation Forecasts with Resilient Backpropagation Neural Networks: Kulmbach Case Study

Round 1

Reviewer 1 Report

This paper presents a machine learning based inundation map forecasting approach. I believe some additions would make this study stronger. Here are my comments;

Section 2.1, as someone who knows what an artificial neural network is and how backpropagation, or some backpropagation variations such as rprop for that matter, works I understand the ANN description here. However, it needs to be more comprehensive with more references from the literature to cover this information adequately. I recommend authors to refer to at least some recent hydroscience-deep learning review papers if not milestone studies and reference books in deep learning literature.

Authors go into detailing the rpop in section 2.1, which in my opinion, is unnecessary. I'm not saying omit this part, it just does not seem important for this study, particularly, it does not seem as important to make its way to the title. Another viewpoint regarding the use of resilient backprop is that the choice is not justified. If one uses a tool, and make a case for one's study using the tool's name in their title, they must at least discuss why one chose that tool instead of other alternatives. For instance, why not others? Current references are only used to say rprop is efficient, that doesn't make others deficient. What I am asking is why rprop was used? Why not others? If rprop would be justified and shown that it brings efficiency or performance to the effort, then using it in title would also be justified.

As a machine learning practitioner, I must say that from my perspective, this approach to inundation map forecasting is not the first to come in mind. This does not necessarily mean the results are bad since an ANN with a hidden layer is a universal approximator, and in theory, it has the capability to learn anything if enough memory is given. What I mean is that building a separate ANN for each grid and for each interval means that you have a * b individual ANNs, a being the number of grids and b being the number of intervals, this approach is not what I was thinking when I first read the problem definition. I, for instance, would convert this task into a matrix to matrix prediction task and would use CNNs or GANs to tackle this problem instead of numerous individual networks. I don't expect the authors to change what they've done. This is science and I expect some justifications for methodology choices. Instead of just saying "This is what we've done", they should say "This is what we've done because of this and that".

Author Response

Reviewer 1

This paper presents a machine learning based inundation map forecasting approach. I believe some additions would make this study stronger. Here are my comments;

R1Q1: Section 2.1, as someone who knows what an artificial neural network is and how backpropagation, or some backpropagation variations such as rprop for that matter, works I understand the ANN description here. However, it needs to be more comprehensive with more references from the literature to cover this information adequately. I recommend authors to refer to at least some recent hydroscience-deep learning review papers if not milestone studies and reference books in deep learning literature

RA: We referred to the recent hydroscience deep learning review papers and added more ANN descriptions.

L77: “Artificial neural networks are algorithms applied to map features into a series of outputs. Through a structure of input, output and intermediate hidden layers, artificial neural networks can learn data relationships between input and output data [22].”

L81: “One of the most widely used ANN is the multi-layer perceptron (MLP) [24]. The MLP consists of highly interconnected neurons organized in layers to process information. The neurons in one layer are fully connected to each neuron in the next layer. Each connection is then assigned a weight.”

L85: “An activation function is used to transfer the results from the hidden layers to the output layer, and a loss function is applied to measure the fit of the neural network to a set of input-output data pair.”

R1Q2: Authors go into detailing the rpop in section 2.1, which in my opinion, is unnecessary. I'm not saying omit this part, it just does not seem important for this study, particularly, it does not seem as important to make its way to the title. Another viewpoint regarding the use of resilient backprop is that the choice is not justified. If one uses a tool, and make a case for one's study using the tool's name in their title, they must at least discuss why one chose that tool instead of other alternatives. For instance, why not others? Current references are only used to say rprop is efficient, that doesn't make others deficient. What I am asking is why rprop was used? Why not others? If rprop would be justified and shown that it brings efficiency or performance to the effort, then using it in title would also be justified.

RA: L103: “Berkhahn et al. [19] compared the training algorithms for hyperparameter tuning showed that the resilient backpropagation is more efficient than conventional backpropagation and Levenberg–Marquardt in maximum inundation prediction. ”

Please refer also to R2Q4.

R1Q3: As a machine learning practitioner, I must say that from my perspective, this approach to inundation map forecasting is not the first to come in mind. This does not necessarily mean the results are bad since an ANN with a hidden layer is a universal approximator, and in theory, it has the capability to learn anything if enough memory is given. What I mean is that building a separate ANN for each grid and for each interval means that you have a * b individual ANNs, a being the number of grids and b being the number of intervals, this approach is not what I was thinking when I first read the problem definition. I, for instance, would convert this task into a matrix to matrix prediction task and would use CNNs or GANs to tackle this problem instead of numerous individual networks. I don't expect the authors to change what they've done. This is science and I expect some justifications for methodology choices. Instead of just saying "This is what we've done", they should say "This is what we've done because of this and that".

RA: The ANN we applied has two hidden layers not one.

L90:”Between the input layer and the output layer, the ANN has 2 hidden layers with 10 nodes per layer.”

L100: “Some alternative artificial network structures, such as convolutional neural network (CNN), could not applied in this study. The network size required a very large number of hypermeters which was beyond the memory capacity of a PC for forecasting purposes [23].”

Author Response File: Author Response.docx

Reviewer 2 Report

Review of the manuscript

“Multi-Step Flood Inundation Forecasts with Resilient Backpropagation Neural Networks: Kulmbach Case Study” by Lin et al. submitted to Water

This work uses the resilient backpropagation neural networks to forecast multiple time step ahead (1h, 2h, 3h, 4h, and 5h) based on the synthetic event database of inundation map. The model was trained and tested via 180 synthetic events that were generated by the HEC-RAS and FloodEvac models.

Although the initial idea is good, and there is merit in the work, it is my opinion that the current version of the manuscript needs to improve significantly. The Introduction is not thorough, the description of the data is very short, and the methodology and its approach are not clear.  The results and discussion as the authors presented have some important points that need to be fixed.

In general, the quality of the writing is unfortunately insufficient to be published in this journal and the authors should consider a general revision of the text to improve the readability.

Details and suggestions on how to improve this work are presented below.

Major points

Abstract. The abstract should be rewritten to show summarize the results and findings. Some minor issues are also mentioned specifically in the minor point section.

Introduction. The background information is quite short and not thorough. Are there only the data-driven methods for flood forecasts? The authors should clarify how this work differs from previous attempts. What is your research question? After reading Introduction section, I do not understand “a threshold value” meaning as the authors proposed.

Methods. I am curious about Resilient Backpropagation Neural Networks that is the main tool in this study. However, in the Methods section, I only find the information by two equations (1) and (2). Is it enough to present the theory of the method? It is hard to follow section 2.1 (L70-93) and section 3.2. Due to the important role of data in the proposed method, I suggest the authors should reorganize the sections to see clearly the data information and how the data implemented into the model. How the ANN models were calibrated to choose the hyper-parameters? What are the criteria for choosing one-third of the synthetic event database in the testing stage?

Sections 2.2 and 2.3 (Figure 3). I tried to understand the proposed approach for linking the trained ANN models at the different lead times (3h, 6h, 9h, and 12h) and their application in real-time forecasting with lead time 1h, 2h, 3h, 4h, and 5h. Unfortunately, it is not clear to me. What is the meaning of the terms such as first time interval 0, first interval?

Study area and database. This section includes very short descriptions of the study area and data information. It is hard to understand the characteristics of the study area, database generation of synthetic events and its application in the study.

Results. Figures 5, 7, 9 show the discharge curves of five streams, however, there are seven curves that are mentioned in the captions. How about two remaining discharge curves? The threshold value should be presented in the figures.

In this study, the authors compare the ANN-based inundations map with the map generated by the FloodEvac Tool model. How about the reliability of the FloodEvac Tool model in this study? Was the model calibrated and verified in the previous study?

In the sub-sections of section 4 (4.1, 4.2, and 4.3) and section 5 (5.1, 5.2, and 5.3), there is a repetition of the subsection names. The authors only mentioned the name of the tables and figures in most of the subsections of section 4. Then, the authors repeated the names again, made some analysis and discussion based on the results of the tables and figures in the subsections of section 5. I suggest that the sections should be combined into the section “Results and Discussion”. The Discussion section needs to improve to see the findings of this study due to the authors mainly describe the results in the current version.

Figure 11. Surprisingly, the performance of the 12-h forecast model is quite good (even is the best one of the models) in at least two historical events in terms of CSI and POD. I may have missed something, but in my understanding, the accuracy of the forecast model with short lead-time would be better than that with long lead-time. This phenomenon is consistent with the results in Tables 5, 6, and 7 of the manuscript, which point out at L 304-305 by the authors. Please explain and discuss the results in detail.

There are many portions of the paper that have significant readability issues, for example, at L30-34, L 35-38, L 56-62, L 121-126, L 130-139, L 183-187, L397-399, L 407-409,  L 424-430. Some of the issues are with sentence structure, specifically with several sentence fragments attached together into one sentence. Other times, the language is vague or confusing. I suggest that the manuscript should be rewritten thoroughly.

Minor points

Line 17. It is hard to understand “forecast of the first intervals (time 0)”

Line 20. “for verify …” should be rewritten, for example “for verifying …”

Lines 22-23: What is the real events?

Line 30: “hydrological and most predominantly hydrodynamic model are computationally expensive”

Line 38: “a real time forecasting”

Line 39: “a higher performance for modelling 39 nonlinear systems in recent years”, what did the authors mention to compare?

Line 56: “an ANN framework”. It might change to “an ANN-based framework”

Line 60-61: “Our hypothesis is the ANN trained with different forecast lead times can provide 60 comparable multi-step forecast as hydraulic models, within much shorter time consummation, from 61 several hours (hydraulic model) to several seconds (ANN), with a comparable accuracy”. It is hard to understand.

Line 136-137: “the real-time forecast is performed with the ANN models trained to forecast with lead times of 1h, 2h, 3h, 4h and 5h”

Line 189: “the first sub-section” means section 4.1?

Line 190: “The second sub-section” means section 4.2?

Tables 1, 2, and 3. Are the highlights necessary when the tables contain not much information (4 rows only)?

What is the meaning of x-axis and y-axis in Figures 6, 8, and 10?

Figure 7. What time for new starting point? 

L 419-422: This paragraph should be moved to the Conclusion section.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Authors addressed all my comments regarding the first version of manuscript and made necessary changes. I suggest publication of this manuscript.