Framework, Procedure, and Tools for Comprehensive Evaluation of Sustainable Stormwater Management: A Review

Round 1

Reviewer 1 Report

This paper presents an extensive literature review of comprehensive evaluation system for stormwater management, related policies/decision support tools and research, a comprehensive performance and benefit evaluation framework for sustainable stormwater management in most developing countries.

In my opinion, the article is overdeveloped. The authors have devoted a lot of time to gathering all this information, but besides, their suggestions related to the implementation of these policies or research are very limited. Therefore, a secondary aim (provide recommendations for further decision support tool development) is limited to a minimum. A great advantage of this article are the tables with collected information and diagrams. The disadvantage is the extensive content, without in-depth interpretation. It is recommended to shorten descriptions and instead try to achieve the second goal.

Specific comments

It is proposed to change the name of chapter 3.1.3 to Comprehensive ..... "Studies on" is unnecessary;

Line 214: there is Li, et al. [43] and Li, et al. [44] but should be Li, et al. [43,44];

Line 219: there is SUDs but should be SUDS

Line 232: there is "....evaluation on environmental benefits, economic benefits and social benefits.", better would be "..evaluation on environmental , economic and social benefits."

For what purpose the authors place some Indicators without reflection in the analyzed tools / strategies for stormwater management (see Supplementary table 2 and Table 3)

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

General Comment:

This study well summarizes stormwater management (SWM) practices and evaluation approaches from various perspectives based on extensive literature review. The suggested evaluation framework incorporates multiple faucets of SWM benefits, and the review of eight simulation models provides an insight into the applicability of model use to projects in China. However, my major concerns are: 1) how the proposed procedure for comprehensive evaluation (Figure 3) are linked to the suggested evaluation framework (Table 1), and 2) readability of the manuscript. The detail suggestions to improve the manuscript are listed as follow:

Detailed Comments:

Overall: I strongly suggest the authors to improve the readability of this manuscript through professional edits. Some sentences are not complete and convey unclear messages.

Line 28-29: The reduction of stormwater infiltration and evaporation, generation of massive stormwater runoff, decrease of groundwater recharge and continuous aggravation of non-point source pollution are examples of hydrologic changes, not results.

Line 32: Why SWM is important to mitigate climate change impacts? Please provide explanation about how climate change, along with urbanization, can exacerbate the current hydrologic challenges in a city.

Line 44: What does ‘it’ indicate?

Line 49-50: This is not a complete sentence.

Line 52: Replace ‘And’ to ‘Furthermore’ or ‘In addition.’

Line 54: Delete ‘Also.’

Line 65: … a clear vision of stormwater management ‘for developing countries.’

Line 66: This paragraph delivers additional information to the previous paragraph. I would replace ‘On the other hand’ to ‘In addition.’

Line 69: for the evaluation ‘process’ (not results).

Line 86-87: I would suggest ‘… facilitate development of suitable goals and indicators for measuring the benefits of sustainable urban stormwater management at varying regional conditions.’

Line 89: delete ‘in.’

Line 98-105: Which keywords were used for literature search? How many papers were selected? What databases are used (in addition to Web of Science and ScienceDirect)? What types of literature were reviewed (e.g. peer-reviewed journal article, conference proceeding, governmental report, book, etc.)? Please provide more detailed methodological approaches that you take to conduct literature review.

Line 133: The term ‘Sponge City’ has been used in China only. For readers from other countries, the definition and concept of Sponge City is needed.

Line 139: … ‘evaluation processes and methods’…

Line 177: To be consistent with Figure 1, the title of subsection shall be SWM practices of ‘national’ government and institutions.

Line 182-184: It should be noted that the USEPA requires the National Pollutant Discharge Elimination System (NPDES) permits for any construction sites larger than an acre in USA. To obtain the permit, constructors are mandated to submit the Stormwater Pollution Prevention Plans (SWPPP). Most states require constructors to track the results of LID monitoring and submit reports to verify compliance. Yet, it is true that no comprehensive framework has been developed to guide performance measurement at a federal level. This session should be elaborated more with intensive literature review.

Line 192-193: This sentence is awkward.

Line 226-227: This is not a complete sentence.

Line 237-239: More literature-based evidence is needed to argue that the current evaluation system in developing countries focus on water-related goals and indicators only. More references should be included.

Line 264-265: Please elaborate on this.

Line 272: This is not a complete sentence.

Line 274-275: Please elaborate on this.

Line 303: It is not clear what ‘watershed wide impact’ indicates. What specific benefits can be addressed at a watershed scale?

Line 329-332: I would say that the warning system itself cannot be parts of social engagement. Rather, the examples are social networking and community information sharing when the disaster hits.

Table 1: There is no reason to place ‘indicators’ into two separate columns and several cells. I would rather recommend you listing the indicators in a cell for each evaluation objective. Additionally, the list of sub-indicators can be added to the current table as examples of performance measures (e.g., volume and peak flow reduction under ‘runoff quantity control’).

• ‘System meets the design objective’: Please be more specific. Meeting the target volume or peak flow reduction goals for a design storm will be an example.
• ‘Suitable planting scheme’: I would rather say 'system layout/structure' which includes not only vegetation scheme but also medium depth, layer configuration, soil composition, etc. (See Sohn, W., Kim, J. H., Li, M. H., & Brown, R. (2019). The influence of climate on the effectiveness of low impact development: A systematic review. Journal of environmental management, 236, 365-379.
• ‘Technical advancement’: Please elaborate on this. This is a broad term which can include the concepts of the other two indicators: system operation intelligence and adoption of innovative equipment.

Line 378: Please provide what ASSC stands for. Abbreviations should be defined in parentheses the first time they appear in the manuscript.

Table S2: This table shall be embedded in the manuscript, rather than being a supplementary table.

Line 412-413: Please elaborate on this.

Figure 3: How is the evaluation procedure narrated in Figure 3 linked to Table 1? It is not clear whether those multiple measures emphasized in Table 1 can be quantified through the process explained in Figure 3.

You previously mentioned that "The construction objectives and evaluation standards currently established by Sponge City mostly focus on the operational capacity of the stormwater system and the ecological benefits that it could bring. Little is mentioned to assess whether stormwater management initiatives are applied in the appropriate context such as site condition economy, technology, society, management, and urban development." Please clarify how the suggested evaluation procedure allows measurement of economy, technology, society, management, and urban development.

Line 433: Is this evaluation procedure only for hypothetical SWM projects? If it is a built project, monitoring performance of SWM facilities can be a better option over simulation/modelling. Please elaborate on the purpose of developing the framework.

Line 472-473: This is not a complete sentence.

Line 476: A dot is needed after a complete sentence.

Line 536: I would say ‘applicability’ rather than ‘application.’

Line 537-538: It is unclear what the benefit or value of a decision support tool will be if it is neither applicable nor possible. Could you provide an example?

Line 564: I would say ‘applicability’ rather than ‘accessible.’

Table 2: The meaning of abbreviations used in Table 2 can be noted right below the table for readers.

• Under ‘MUSIC’: User friendly manner and interface; MUSIC-Link could check the design program according to local construction specifications online’ is rather an ability of the tool, not an evaluation capacity.
• Under UrbanBEATS: What kinds of algorithm is used? Why is it inspiring?
• Under DAnCE4Water: What philosophy does the model adopt? Why is it inspiring?

Table S3: As shown in the table, the reviewed eight decision support tools cannot measure most indicators other than surface runoff control, system performance, economic sustainability, environmental governance, and disaster resistance. Your proposed procedure for comprehensive evaluation (Figure 3) should be then revised to meet the goals narrated in Table 1. Instead of focusing on hydrologic models only, a broader approach is required to embrace measurement methods of diverse indicators such as social engagement and urban development.

Conclusion section: Please state the limitation of this study.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

This manuscript has been significantly improved, but I still suggest minor revision before publication.

One of the major concerns was readability. I am glad that the authors worked to improve it, though I can still find many grammatical errors and awkward sentence structure. I highly recommend authors to contact worldwide ESL specialists for professional academic editing. Please proofread the manuscript very carefully. The common errors are:

• Missing conjunctions such as ‘and/or’ for word lists (e.g., Lines 117-120; 273-274; 278; 334-335; 353; 382-383; 390-391; 558-559; 573-574; 581-582; 594-595; 620-621; 732-733; etc.).
• Be consistent in using either American or British English.
• Be consistent in the use of numerals or spelled-out numbers (e.g., numbers in Lines 102, 132, 200, 429, and 430 can be spelled out).
• Abbreviations should be defined in parentheses the first time they appear in the manuscript (e.g., Line 610 – SWMM; Line 736 – IUWSM).
• Incomplete sentence (e.g., Lines 262-263).
• Inconsistent grammatical structure (e.g., Table 1 – Please use all sentences or all fragments, not a mixture).
• Typo (e.g., Figure 4 – ‘majority’ and ‘monetary’).
• Improper preposition.

The other detail suggestions to improve the manuscript are listed as follow:

Lines 28-39: While the issues of urbanization and hydrologic changes are emphasized in the problem statement, the sustainable SWM is suggested as a solution to climate change mitigation. The linkage between problems and solutions is still vague.

Lines 32-36: I would suggest ’… aggravation of non-point source pollution, leading to a major challenge of urban stormwater management (SWM). Given these challenges, the green-infrastructure-based sustainable SWM has been...’

Line 67: Please provide some examples of context-specific evaluation systems.

Lines 104-105: Try to minimize redundancy.

Line 116: Google Scholar is not a database but a search engine.

Lines 119- 121: Semi-colons should be consistently used.

Lines 120-121: Please elaborate on this.

Figure 1: The newly added figure should be cited in the main text.

Line 154: Sponge City

Lines 154-155: This is the first time that the term ‘Sponge City’ has been introduced in the main text. Please provide the rationale behind your selection of China as a case study. How is it representative of developing countries? How is the concept of Sponge City similar to that of GI, LID, SUSD, etc.?

Lines 168-173: This statement can be moved to the location where Sponge City was first introduced.

Line 345: Could you emphasize that this subsection focuses on the evaluation framework at a site-specific scale, while the previous section also examines the evaluation frameworks at a national level?

Line 251: I would suggest ‘… the environmental, economic, and social benefits …’

Lines 306-307: It will be nice to emphasize that this runoff control indicator measures the site-scale impacts to differentiate from the watershed-wide impact indicator that is introduced later.

Line 307: Replace ‘Include’ to ‘including.’

Line 315-318: I would suggest ‘… meeting the specific design goals as well as local technical specifications and standards that are subject to site conditions. The site adaptability, including the conditions of land use, soil, topography, and groundwater, is another key criterion for …’

Line 326: Please elaborate on this. Do does the layout flexibility mean? Maintenance cost is more related to the 'economic sustainability' that you introduced in the following paragraph.

Line 339: Replace ‘SWM’ to ‘SWM technique.’

Line 356: You can emphasize that you are focusing on hydrologic impact here. Otherwise, the following indicators such as biodiversity, habitat restoration, and others can also be considered the watershed wide impact.

Line 362: I would suggest ‘… alleviating water scarcity problems by increasing water storage capacity of land and reusing harvested rainwater.'

Lines 373-374: Please elaborate on this. Grey infrastructure, one of the conventional SWM techniques, also helps to reduce flood impacts through rapid stormwater discharge away from pollution sources.

Lines 385-387: This sentence is awkward. Please reword it.

Line 403: ‘… a decrease of … ‘

Line 404: ‘… accessibility to …’

Lines 409-410: Please elaborate on this. This indicator better belongs to the ‘urban resilience’ based on your explanation.

Table 1:

• Pipe leakage control: Please elaborate on this. How does SWM techniques control pipe leakage?
• Reduction of water supplied by water plant: do you mean the potable water treatment plant?
• Preparation and response for extreme climates: preparation for and response to extreme climates.

Line 459: replace ‘reduce’ to ‘reduced.’

Lines 461-462: I would suggest ‘…in related research for site-scale evaluation.’

Lines 463-464: replace ‘It further’ to ‘The results in Table 2 also.’

Line 475: delete ‘relevant.’

Line 517: add a comma before ‘etc.’

Line 559: replace ‘modelling’ to ‘model simulation.’

Lines 566-567: What do you mean? Can’t be quantitative indicators characteristics and comprehensive?

Lines 568-570: This sentence is relevant to neither the previous nor the following statement.

Line 573: ‘… be used only for…’

Line 576: delete ‘of the value.’

Lines 592-595: Isn't this applicable to MCA only, not BCA? If it is, this statement should be moved right after explanation of MCA.

Lines 595-597: Shouldn't be this step prior to normalizing indicators? Without indicator scores, how can normalization proceed?

Lines 615-616: This is a common issue for any simulation models not just for MUSIC.

Line 616: replace ‘In addition’ to ‘Nevertheless.’

Line 618: What do you mean ‘in the specific evaluation indicators?’

Line 624: … of the decision support tool for urban SWM.

Line 640: What principles?

Line 654: … to other developing countries.

Line 733: replace ‘is’ to ‘are.’

Table 3:

• The column of ‘Comprehensive evaluation ability of SWM’: I would suggest using consistent language used in Table 1. Also, please use semicolons to separate indicators.
• ‘This could provide some inspirations for the SWM simulation…’: This sentence is awkward. Please reword it.
• ‘User friendly manner and interface’: This is the ease of program use, not relating to any of comprehensive evaluation abilities indicated in Table 1.

Lines 742-743: Could you label out which specific countries other than China were reviewed to develop this evaluation framework or benefit from your suggested framework? That would help enhance generalization of your study results.

Author Response

Please see the attachment.

Author Response File: Author Response.docx