Validating the Quality of Volunteered Geographic Information (VGI) for Flood Modeling of Hurricane Harvey in Houston, Texas

Round 1

Reviewer 1 Report

The manuscript discusses the quality of crowdsourced data obtained during a flooding event (due to Hurricane Harvey) in terms of its usability to determine flood depth and extent. The authors have used FEMA derived flood plain, HEC-RAS modeled flood inundation outputs and USGS high water marks during the event. Although the methodology is explained in detail, the manuscript needs to address the following topics:

1.     The manuscript refers citizen science, crowdsourcing and VGI interchangeably, but they are not the same. Specifically, citizen science requires specific training for user participation to collect high quality data unlike crowdsources, and VGI requires voluntary submission of geographic information. The authors need to clearly discuss the difference and introduce what UFlood is and how the data are collected by the platform.

2.     The comparison with FEMA floodplain can result in errors as they are derived from historic events and with climate change, the floodplains don’t reflect the severity of events or their magnitude often. The authors need to acknowledge these issues in the writeup. Furthermore, HEC_RAS does not account for storm events unlike FEMA flood plain.

3.     The contribution of the manuscript needs to be clearly mentioned. While crowdsourced data could be used for areas where other data sources may not be available, their quality is problematic as the authors have pointed out. So, there should be discussion about the limitations of these data not just best practices of using them.

4.     The authors were able to use additional data sets for comparison, but often this is not the case with other countries. So, a discussion about the usability of the crowdsourced data should be mentioned.

Author Response

Reviewers

Authors

The manuscript discusses the quality of crowdsourced data obtained during a flooding event (due to Hurricane Harvey) in terms of its usability to determine flood depth and extent. The authors have used FEMA derived flood plain, HEC-RAS modeled flood inundation outputs and USGS high water marks during the event. Although the methodology is explained in detail, the manuscript needs to address the following topics:

1. The manuscript refers citizen science, crowdsourcing and VGI interchangeably, but they are not the same. Specifically, citizen science requires specific training for user participation to collect high quality data unlike crowdsources, and VGI requires voluntary submission of geographic information. The authors need to clearly discuss the difference and introduce what UFlood is and how the data are collected by the platform.

We appreciate the reviewer’s comment. The abstract and introduction are revised with distinction among crowdsourcing, VGI and citizen science. As stated in the abstract, we see the U-Flood data as VGI, because the volunteers are not well-trained in a citizen science project but they were instructed to voluntarily contribute inundated streets in various formats through the online platform. The title is also revised to properly reflect that.

2. The comparison with FEMA floodplain can result in errors as they are derived from historic events and with climate change, the floodplains don’t reflect the severity of events or their magnitude often. The authors need to acknowledge these issues in the writeup. Furthermore, HEC_RAS does not account for storm events unlike FEMA flood plain.

In this study, we used the FEMA floodplain modeled specifically for the Harvey flood (USGS, 2018). As stated in the paper, “[t]he FEMA floodplain WD data, however, is only available from August 27, 2017, to September 1, 2017 (but without August 31). Thus, this study compared the modeled floodplain of HEC-RAS against the FEMA floodplain on September 1, 2017, the only matched date between the two datasets of FEMA and U-Flood.”. While the effect of climate change doesn’t apply here, there are some uncertainties using FEMA modeled floodplain as the reviewer rightfully pointed out. We discussed the comparison of U-Flood and FEMA floodplain in this context in lines 360-372 & 393-398. Files of Harvey floodplain from FEMA are available here: https://disasters.geoplatform.gov/publicdata/NationalDisasters/2017/HurricaneHarvey/Data/DepthGrid/FEMA/Riverine_Modeled_Preliminary_Observations/

United States Geological Survey, 2018. Post-Harvey report provides inundation maps and flood details. Communication and Publishing division, Published on July 9, 2018. https://www.usgs.gov/news/national-news-release/post-harvey-report-provides-inundation-maps-and-flood-details-largest, accessed on May 2, 2023.

3. The contribution of the manuscript needs to be clearly mentioned. While crowdsourced data could be used for areas where other data sources may not be available, their quality is problematic as the authors have pointed out. So, there should be discussion about the limitations of these data not just best practices of using them.

The contribution of this study is articulated in the first paragraph of Conclusions section (i.e. lines 468-479). The limitations are elaborated for HEC-RAS and FEMA data (lines 365-370, 393-395), U-Flood data (lines 373-391, 407-409, 415-420).

4. The authors were able to use additional data sets for comparison, but often this is not the case with other countries. So, a discussion about the usability of the crowdsourced data should be mentioned.

Good suggestion. A discussion about the usability of such data is inserted in lines 493-501.

Reviewer 2 Report

The manuscript validated crowdsourced flood reports during Hurricane Hurvey by comparing with simulated flood depth from various sources (HEC-RAS, FEMA and USGS). With the growing volume of crowdsourced data comes the necessity to ascertain the reliability of such datasets to be implemented in model improvement. This study addresses this issue and can be of interest to the readers. Some suggestions to improve the manuscript is listed below:

1.       Here are some other studies on validating crowdsourced flood data, which might be relevant:

Ostermann, Frank, and Laura Spinsanti. "Context analysis of volunteered geographic information from social media networks to support disaster management: A case study on forest fires." International Journal of Information Systems for Crisis Response and Management (IJISCRAM) 4.4 (2012): 16-37.

Praharaj, Shraddha, et al. "Assessing trustworthiness of crowdsourced flood incident reports using Waze data: a Norfolk, Virginia case study." Transportation research record 2675.12 (2021): 650-662.

2.       Line 172-177: Why simulate worst flood condition instead of what happened in reality as the authors want to validate crowdsourced data?

3.       Line 195: How was water depth extracted from DEM?

4.       Line 216-218: 184 within one segment or across all the segments? It is not clear how the max elevation comes into the calculation.

5.       Line 272-273: What does the FEMA water depth represent? Maximum flood depth on a day?

6.       Line 297: Why is the 1m difference considered small by the authors? What was the range of water depths on the floodplain? On a roadway, 1m should not be considered minor.

7.       Line 426: Fix the error.

Author Response

Reviewers

Authors

The manuscript validated crowdsourced flood reports during Hurricane Hurvey by comparing with simulated flood depth from various sources (HEC-RAS, FEMA and USGS). With the growing volume of crowdsourced data comes the necessity to ascertain the reliability of such datasets to be implemented in model improvement. This study addresses this issue and can be of interest to the readers. Some suggestions to improve the manuscript is listed below:

1. Here are some other studies on validating crowdsourced flood data, which might be relevant:

Ostermann, Frank, and Laura Spinsanti. "Context analysis of volunteered geographic information from social media networks to support disaster management: A case study on forest fires." International Journal of Information Systems for Crisis Response and Management (IJISCRAM) 4.4 (2012): 16-37.

Hung, Kuo-Chih, Mohsen Kalantari, and Abbas Rajabifard. "Methods for assessing the credibility of volunteered geographic information in flood response: A case study in Brisbane, Australia." Applied Geography 68 (2016): 37-47.  

Safaei-Moghadam, Arefeh, David Tarboton, and Barbara Minsker. "Estimating the likelihood of roadway pluvial flood based on crowdsourced traffic data and depression-based DEM analysis." Natural Hazards and Earth System Sciences 23.1 (2023): 1-19.

Praharaj, Shraddha, et al. "Assessing trustworthiness of crowdsourced flood incident reports using Waze data: a Norfolk, Virginia case study." Transportation research record 2675.12 (2021): 650-662.

We thank this reviewer for the fine suggestion and deep understanding on this topic throughout the comments. These studies are relevant and have been incorporated into the revised manuscript.

2. Line 172-177: Why simulate worst flood condition instead of what happened in reality as the authors want to validate crowdsourced data?

The FEMA floodplain uses USGS stream records and high water marks (HWM) to simulate the Harvey floodplain. The HWM records imply the maximum flood depth experienced at that location. Moreover, some USGS stream gauges were damaged during Hurricane Harvey, and there is no continual stream record available for hydraulic simulation. In disaster research, it is not uncommon to use the “worst” case scenario for mitigation and preparation purposes. Therefore, this study used the HWM and peak discharge from USGS stream gauge with the assumption to simulate the worst flood conditions for comparison. We acknowledged that this may differ from the “reality”, as in day-to-day fluctuation at a finer temporal interval, and stated the uncertainty in lines 204-206.

3. Line 195: How was water depth extracted from DEM?

Some methodological details were simplified to meet the word limits, we appreciate the opportunity to clarify it. First, the modeled floodplain was overlaid with the lidar-derived DEM. A zonal maximum value of the DEM was used to infer the water surface elevation (WSE) of the floodplain. Assuming the WSE is flat within the floodplain, the water depth is simply the difference between the WSE and DEM at any location within the floodplain. The section is revised accordingly in lines 192-196 for clarification.

4. Line 216-218: 184 within one segment or across all the segments? It is not clear how the max elevation comes into the calculation.

Similar to the procedure in the above response, the maximum elevation represents the WSE of inundated street segments, “which was again subtracted from DEM to estimate the WD along the U-Flood inundated street segment” (lines 216-219). Hope this clarifies the calculation.

5. Line 272-273: What does the FEMA water depth represent? Maximum flood depth on a day?

Yes

6. Line 297: Why is the 1m difference considered small by the authors? What was the range of water depths on the floodplain? On a roadway, 1m should not be considered minor.

In this research, the WD in the floodplain ranges between 0 - 12.53 m (Figure 6). To visualize the WD differences between HEC-RAS & U-Flood in Figure 8, the word “small” for the WD diff between -1 – 1 m class was used in a relative sense to the 1-2 m & >2 m classes. Within the -1 – 1 m class, most WD differences are distributed in the lower and middle portion. Anyway, we agree with the reviewer that 1 m difference is not “small” in absolute sense and that line was revised accordingly.

7. Line 426: Fix the error.

It is supposed to be a reference to Figure 11, it was ok in the .docx but missing in the .pdf. Fixed.

Reviewer 3 Report

Dear authors,

Thanks for your submission. The comments are as follows:

1. In figure 1, please reviese it by followsing MDPI instruction https://www.mdpi.com/journal/hydrology/instructions. This was not a standard thematic map(ie. latitude and longitude, compass and et al.).

2. In line 141, the location of figure 2 was error. 

3. In flow chart, what's the innovation of this manuscript? Especially in methodology.

4. In section 3, how to validate the result accuracy?

5. What's the scientific quetion of this manuscript? In section 4, what's effective measure for flood control and disaster reduction?

Author Response

Reviewers

Authors

1. In figure 1, please reviese it by followsing MDPI instruction https://www.mdpi.com/journal/hydrology/instructions. This was not a standard thematic map(ie. latitude and longitude, compass and et al.).

This map has a legend, a north arrow, a scale bar, an inset map, a neat line and labels commonly found in a thematic map. Latitude/longitude is not used because it is projected in state plane coordinate system appropriate for spatial analysis and minimize map distortion in the local area. The figure conforms with MDPI instructions with regards to figure preparation.

2. In line 141, the location of figure 2 was error.

Revised, thanks.

3. In flow chart, what's the innovation of this manuscript? Especially in methodology.

The flow chart (Figure 4) documents the methodology of this research to enable scientific replicability. The contribution of this paper lies in assessing the quality of VGI (i.e. U-Flood data) for floodplain modeling, not in the methodology.

4. In section 3, how to validate the result accuracy?

As stated in lines 178-181, “[t]he quality of U-Flood data was examined by comparing a) the water depth (WD) among the HEC-RAS modeled floodplain, FEMA flood map, USGS HWM & U-Flood data (RQ1) and b) the extent of the modeled floodplain (RQ2). This study compared various flood datasets to answer the research questions (Table 3).”

5. What's the scientific quetion of this manuscript? In section 4, what's effective measure for flood control and disaster reduction?

As stated in lines 86-90, “[t]he research questions of this study include: 1) Are there any significant differences in the water depth among the H&H model (i.e. HEC-RAS), authorized reference (i.e. FEMA) and VGI (i.e. U-Flood data)? 2) Are there any significant differences in the inundated areas between the HEC-RAS modeled floodplain and U-Flood data observations?”

While the research can benefit flood control and disaster reduction by providing useful insights to assist and improve real-time flood modeling, this research has no effective measure for these unintended goals.

Round 2

Reviewer 3 Report

Dear authors,

Thanks for re-submission. 

All reviesed content are correct.