Detection of Flood Extent Using Sentinel-1A/B Synthetic Aperture Radar: An Application for Hurricane Harvey, Houston, TX

Round 1

Reviewer 1 Report

Overview

The manuscript describes the use of images from remote sensing sources to detect flooded areas by using different methods to process these images. The original contribution of this study to the investigated subject is not clearly stressed. The authors should better highlight the innovative features of the study. The novelty and originality of the contents are not clearly identified. Another limitation is that just one flood event has been investigated to test the accuracy of the proposed methodology. Furthermore, more details and analyses should be provided about methods and evaluation of results.

I therefore believe that the actual form of the manuscript needs major revisions before the publication in Remote Sensing.

Major comments

1) The title of the paper does not fully reflect the contents of the study, given that the machine learning approach is not so preeminent throughout the study. Moreover, in my opinion, the title should report that the study is an application for a specific case study.

2) A more detailed review about past literature on the topic issue of this study (i.e., mapping of flood areas by remote sensing sources and methods to process images) should be added.

3) The novelty of the study and proposed approaches is not clear. Which is the innovation of the present study? Which methods do represent an original innovation or application (with respect to the state-of-the-art)?

4) Which operational rules can be deduced from this study? Could the outcomes be used for a timely real-time (or near-real-time) evacuation plan? Could a prediction tool be developed for an operational use in real-time (or near-real-time), based on the discussed investigation? Which images could be available actually in real-time (or near-real-time) during a flood event (given that the used images refer to different times)?

5) The different sources of data are not available at the same time during the event. This issue opens a discussion theme: could the proposed methodology be applied to different remote sensing sources according to their availability, for instance by using a source for the pre-flood condition and another source for the post-flood condition? Is this possible in the operational practice? This issue would be worth to be discussed if the aim of the study is to progress towards the goals of using different sensors and resolutions for effectively mapping floods with timing proper for real time (or near-real-time) flood-risk emergency responses.

6) The meaning of “real-time” and  “near-real-time” should be briefly introduced with respect to the field of application of data and methods proposed in the present study.

7) It seems that the analysis of results is based just on a subjective analysis by visual inspection of images. An objective statistical evaluation should be added to test the accuracy of classification by proposed methods.

8) The significance of discussion and conclusions is dampened by the limited available dataset (just one case study) and the lack of an objective statistical evaluation.

Minor comments

Lines 21-23: The significance of these statements is dampened by the different reference time of images used within the study to evaluate the accuracy at identifying flooded areas.

Line 133: The contents of Table 1 could be of poor significance. Please consider to remove Table 1 and summarise the sense of this collection of scenes in the text.

Lines 195-196: It is not clear to what refer with “a separate data set for ... etc.”

Lines 238-239: Details should be provided to justify this choice. Could authors support this separation of the dataset by citing proper references?

Author Response

Thank you for your thorough review, it has helped us to greatly improve the paper.  Our responses are in italics between your comments, below.

Overview

The manuscript describes the use of images from remote sensing sources to detect flooded areas by using different methods to process these images. The original contribution of this study to the investigated subject is not clearly stressed. The authors should better highlight the innovative features of the study. The novelty and originality of the contents are not clearly identified. Another limitation is that just one flood event has been investigated to test the accuracy of the proposed methodology. Furthermore, more details and analyses should be provided about methods and evaluation of results.

Thank you, we have endeavored to stress the significance and originality of the study, in the Discussion and Conclusions, and provided additional details throughout the manuscript.

I therefore believe that the actual form of the manuscript needs major revisions before the publication in Remote Sensing.

Major comments

1) The title of the paper does not fully reflect the contents of the study, given that the machine learning approach is not so preeminent throughout the study. Moreover, in my opinion, the title should report that the study is an application for a specific case study.

Agreed, we have revised the title accordingly.

2) A more detailed review about past literature on the topic issue of this study (i.e., mapping of flood areas by remote sensing sources and methods to process images) should be added.

Thank you, we have added additional details to the Introduction.

3) The novelty of the study and proposed approaches is not clear. Which is the innovation of the present study? Which methods do represent an original innovation or application (with respect to the state-of-the-art)?

The innovations here are the production of higher resolution SAR flood maps, the application of the thresholding method to Sentinel-1 for the first time, comparison of those results with earlier methods (DFO, NDWI), and the ability to incorporate them into ongoing flood monitoring using DisasterAware. We have tried to incorporate those points into this version of the paper in the Abstract, Introduction and Conclusions.

4) Which operational rules can be deduced from this study? Could the outcomes be used for a timely real-time (or near-real-time) evacuation plan? Could a prediction tool be developed for an operational use in real-time (or near-real-time), based on the discussed investigation? Which images could be available actually in real-time (or near-real-time) during a flood event (given that the used images refer to different times)?

We have added text in the Conclusions about the integration into DisasterAware and the Model of Models approach, which is the approach for implementing this research.  We also discuss potential sources of additional data for future applications that would shorten the temporal resolution and improve our ability to do near real-time or even real-time forecasting.

5) The different sources of data are not available at the same time during the event. This issue opens a discussion theme: could the proposed methodology be applied to different remote sensing sources according to their availability, for instance by using a source for the pre-flood condition and another source for the post-flood condition? Is this possible in the operational practice? This issue would be worth to be discussed if the aim of the study is to progress towards the goals of using different sensors and resolutions for effectively mapping floods with timing proper for real time (or near-real-time) flood-risk emergency responses.

This methodology certainly could be used with different sources, both pre- and post-flood.  We have added a short discussion to this effect in the Conclusions, in conjunction with (4) above and suggestions from Reviewer 2.

6) The meaning of “real-time” and “near-real-time” should be briefly introduced with respect to the field of application of data and methods proposed in the present study.

Agreed.  We have added text to clarify the difference for SAR flood mapping in the revised version, lines 56-57.

7) It seems that the analysis of results is based just on a subjective analysis by visual inspection of images. An objective statistical evaluation should be added to test the accuracy of classification by proposed methods.

Because, as you note above, the different data sources are not available on the same dates during the event, a direct comparison is difficult. However, we have applied a statistical evaluation, using the Sentinel-2 derived NDWI data for comparison, and those results are now incorporated into the paper (see Results and Discussion).

8) The significance of discussion and conclusions is dampened by the limited available dataset (just one case study) and the lack of an objective statistical evaluation.

Agreed.  We have included a statistical evaluation in the revised manuscript, as described above, and included a discussion of both the limitations of one case study and recommended steps for expanded studies in the future, in the Discussion and Conclusions.

Minor comments

Lines 21-23: The significance of these statements is dampened by the different reference time of images used within the study to evaluate the accuracy at identifying flooded areas.

Agreed, we have revised the abstract to reflect the results of the statistical analysis.

Line 133: The contents of Table 1 could be of poor significance. Please consider to remove Table 1 and summarise the sense of this collection of scenes in the text.

Agreed, removed.

Lines 195-196: It is not clear to what refer with “a separate data set for ... etc.”

Agreed, revised to say ‘scheme, providing an additional data set…’

Lines 238-239: Details should be provided to justify this choice. Could authors support this separation of the dataset by citing proper references?

There is no fixed rule to set the ratio between training and validation data. Typically, the ratio is between 70%/30% to 90%/10%, and practical rule is that a larger dataset allows for a larger ratio. Here we have approximately 10,000 tiles, so we chose 90% of the tiles for training. We revised the sentence to reflect this and included the following reference.

Zhang, E., Liu, L., Huang, L., & Ng, K. S. (2021). An automated, generalized, deep-learning-based method for delineating the calving fronts of Greenland glaciers from multi-sensor remote sensing imagery. Remote Sensing of Environment, 254, 112265.

Reviewer 2 Report

In my opinion the manuscript concerns important issue i.e. detection flood extents using machine learning methods and satellite data over Houston area. I am delighted that remote sensing techniques used by the Authors could be applied for mapping flood extents and investigating false/negative positives. The study proved feasibility of application of optical as well as radar data in order to map flood extents.

However I have some major recommendations needed to be improve in the manuscript:
1. Regarding introduction: I would suggest to add one/few sentences on application of microwave images from other satellite missions i.e. European ERS-1/2, ENVISAT ASAR as well as Japanese ALOS PALSAR for recognizing flood extents over large areas covering vegetation, urban, arable lands . Please find below annotations that could be improved the text in the introduction:

a) Turlej K., Bartold M., Lewiński S., 2010, Analysis of extent and effects caused by the flood wave in May and June 2010 in the Vistula and Odra River Valleys, Geoinformation Issues, Vol. 2, No. 1(2), pp. 49-57. doi:10.34867/gi.2010.5 (http://bc.igik.edu.pl/Content/48/PDF/006.pdf)

b) Dabrowska-Zielinska K., Budzynska M., Tomaszewska M., Bartold M., Gatkowska M., Malek I., Turlej K., Napiorkowska M., 2014, Monitoring Wetlands Ecosystems Using ALOS PALSAR (L-Band, HV) Supplemented by Optical Data: A Case Study of Biebrza Wetlands in Northeast Poland, Remote Sensing, Vol. 6(2), pp. 1605-1633. (https://www.mdpi.com/2072-4292/6/2/1605)

2. Regarding methodology and radar data: Did you try to take Radarsat 1&2 archives into account for mapping wetlands regarding heavy cloud coverage and low availability of S1/S2 data? Radarsat archives could be taken from longer period than period you analyzed 2016-2019.
Please find Radarsat 1&2 archives with high spatial/temporal resolution, available at this link:
https://earth.esa.int/eogateway/catalog/radarsat-1-2-full-archive-and-tasking

3. As far as we know Sentinel-1B has been still malfunctioning since December (https://spacenews.com/hope-fading-for-recovery-of-european-radar-imaging-satellite/). What are your recommendations to use auxilliary data for filling radar data gaps from Sentinel-1 mission?

4. Regarding methodology and thresholding. Did you try to use other thresholding methods (e.g. Otsu algorithm) in order to compare the results of thresholding you applied in the manuscript?

5. The results must be improved with statistics and quantified outcomes i.e. false positives/false negatives. We shouldn't desribe the results on the basis of visual interpretation and use such words as "similar", "very similar", "better" etc. What do you mean "similar" or "very similar" and what is the difference between them? It absolutely must be quantified in stats form (table, descriptive stats etc). The same issue concerns classfification, what is accuracy of recognized classess? I was trying to investigate false positives however the stats better show the over/underestimations than the map results.

6. Last but not least: regarding Discussion and Conclusions. I realize mapping flood extent in USA is crucial for monitoring environmental hazards. Like in Poland being under temperate climate we try to diversify satellite-based images from many missions: Radarsat (Canada), TerraSar (DLR), ALOS-2 (Japan). I think it should be mentioned that Sentinel constellation and Landsat mission highly improved temporal and spatial mapping flood extents (the Authors proved it taking S-1 and S-2 account), on the other hand we need to remember on many satellite data could support or even fulfill requirements in buildin' remotely sensed system especially for regions being hig-risk of flood.

Author Response

Thank you for your thorough review, it has helped us to greatly improve the paper.  Our responses are in italics between your comments, below.

In my opinion the manuscript concerns important issue i.e. detection flood extents using machine learning methods and satellite data over Houston area. I am delighted that remote sensing techniques used by the Authors could be applied for mapping flood extents and investigating false/negative positives. The study proved feasibility of application of optical as well as radar data in order to map flood extents.

Thank you.

However I have some major recommendations needed to be improve in the manuscript:

1. Regarding introduction: I would suggest to add one/few sentences on application of microwave images from other satellite missions i.e. European ERS-1/2, ENVISAT ASAR as well as Japanese ALOS PALSAR for recognizing flood extents over large areas covering vegetation, urban, arable lands.

Agreed, we have included additional details on both satellites and flood detection methods and have included the references below.

Please find below annotations that could be improved the text in the introduction:

57. a) Turlej K., Bartold M., Lewiński S., 2010, Analysis of extent and effects caused by the flood wave in May and June 2010 in the Vistula and Odra River Valleys, Geoinformation Issues, Vol. 2, No. 1(2), pp. 49-57. doi:10.34867/gi.2010.5 (http://bc.igik.edu.pl/Content/48/PDF/006.pdf).
58. b) Dabrowska-Zielinska K., Budzynska M., Tomaszewska M., Bartold M., Gatkowska M., Malek I., Turlej K., Napiorkowska M., 2014, Monitoring Wetlands Ecosystems Using ALOS PALSAR (L-Band, HV) Supplemented by Optical Data: A Case Study of Biebrza Wetlands in Northeast Poland, Remote Sensing, Vol. 6(2), pp. 1605-1633. (https://www.mdpi.com/2072-4292/6/2/1605).
59. Regarding methodology and radar data: Did you try to take Radarsat 1&2 archives into account for mapping wetlands regarding heavy cloud coverage and low availability of S1/S2 data? Radarsat archives could be taken from longer period than period you analyzed 2016-2019.
    Please find Radarsat 1&2 archives with high spatial/temporal resolution, available at this link:
    https://earth.esa.int/eogateway/catalog/radarsat-1-2-full-archive-and-tasking.

Agreed, these methods could be applied to the Radarsat1&2 data sets.  However, while the Radarsat1 data has been freely available through the Alaskan Satellite Facility (ASF) archive, acquisition of Radarsat2 data generally requires submission of a proposal, for limited images.  In addition, Radarsat1 data is not available over Houston, TX for the Hurricane Harvey time period.  Finally, neither is available in GRD format, which these methods have been adapted for as input – their use would require additional modifications to their amplitude images.  However, future studies should investigate the applicability of these data sets in the preparation of longer time series, and we have discussed this in the Conclusions.

3. As far as we know Sentinel-1B has been still malfunctioning since December (https://spacenews.com/hope-fading-for-recovery-of-european-radar-imaging-satellite/). What are your recommendations to use auxiliary data for filling radar data gaps from Sentinel-1 mission?

Agreed, the loss of Sentinel-1B will impact the potential for near real-time image acquisition.  Other satellites could provide images to supplement the missing acquisitions, such as RCM C-band data or from the upcoming L-band NISAR mission. Future studies will incorporate that L-band data into these methods, and we have discussed this in the Conclusions.

4. Regarding methodology and thresholding. Did you try to use other thresholding methods (e.g. Otsu algorithm) in order to compare the results of thresholding you applied in the manuscript?

We did analyze the data using both the LM and Otsu thresholding algorithms.  The results were remarkably similar for this flood event and, because the primary goal was to compare three different algorithms, we did not include those results in this paper.

5. The results must be improved with statistics and quantified outcomes i.e. false positives/false negatives. We shouldn't describe the results on the basis of visual interpretation and use such words as "similar", "very similar", "better" etc. What do you mean "similar" or "very similar" and what is the difference between them? It absolutely must be quantified in stats form (table, descriptive stats etc). The same issue concerns classification, what is accuracy of recognized classess? I was trying to investigate false positives however the stats better show the over/underestimations than the map results.

Agreed, we have incorporated a statistical analysis into the Results section, comparing the three new methods with the NDWI results.  We have included a discussion of the potential limitations, particularly the fact that they are acquired on different dates, in the Discussion and Conclusions.

6. Last but not least: regarding Discussion and Conclusions. I realize mapping flood extent in USA is crucial for monitoring environmental hazards. Like in Poland being under temperate climate we try to diversify satellite-based images from many missions: Radarsat (Canada), TerraSar (DLR), ALOS-2 (Japan). I think it should be mentioned that Sentinel constellation and Landsat mission highly improved temporal and spatial mapping flood extents (the Authors proved it taking S-1 and S-2 account), on the other hand we need to remember on many satellite data could support or even fulfill requirements in buildin' remotely sensed system especially for regions being hig-risk of flood.

Agreed, we have incorporated this into the Conclusions.

Reviewer 3 Report

The authors present a comparison of several methods for characterizing flood inundation, using a combination of SAR remote sensing data and machine learning methods. Each method is applicable under certain conditions or constraints: cloud free/cloud cover conditions, resolution, temporal coverage, spatial scales, real‐time or near real‐ time flood mapping

I suggest the authors to provide, if possible, a Multiple-Criteria Analysis (MCA) in order to put into evidence "the most preferred alternative(s)" or the “nondominated” alternative(s).

Author Response

Thank you for your thorough review.  Our responses are in italics between your comments, below.

The authors present a comparison of several methods for characterizing flood inundation, using a combination of SAR remote sensing data and machine learning methods. Each method is applicable under certain conditions or constraints: cloud free/cloud cover conditions, resolution, temporal coverage, spatial scales, real‐time or near real‐ time flood mapping.

Agreed, thank you.

I suggest the authors to provide, if possible, a Multiple-Criteria Analysis (MCA) in order to put into evidence "the most preferred alternative(s)" or the “nondominated” alternative(s).

We agree that a statistical comparison of the methods is necessary and we have provided one in the Results section, and have incorporated the F1 multi-class classifier into that analysis.  However, a larger MCA analysis, providing decision trees, for example, is outside the scope of this work.  We do discuss the potential application of such an analysis in the Conclusions, in conjunction with the current plans for implementing these methods in an operational system.

Round 2

Reviewer 1 Report

Authors have addressed reviewer's comments.

Reviewer 2 Report

The manuscript has been improved properly. Thank you