Modeling the Influence of Public Risk Perceptions on the Adoption of Green Stormwater Infrastructure: An Application of Bayesian Belief Networks Versus Logistic Regressions on a Statewide Survey of Households in Vermont

Round 1

Reviewer 1 Report

This work offers a useful alternative on how to analyze survey data using BBN. I have some general comments to share below, with more specific comments in the attached PDF.

1. The title of the paper states that it is a comparison between Logistic Regression and BBN techniques. Research objective 2 also states that a comparison of the two approaches will be presented. Yet, a real comparison of the two approaches is only presented as the last subsection in the Method's section, and not in the Results. I suggest moving that section to Results. However, and more importantly, the authors do not refer to the Coleman et al. (2018) work that clearly used Logistic Regression to analyze the same dataset. Why is that paper not referred to at all, and why not make a more explicit comparison between that paper and this work?
2. The work assumes that the reader is well-versed with BBN, and results are presented with minimal background beyond a general presentation of when BBN is useful. I believe there is an opportunity to educate the reader of the value of BBN, some basic principles, and how results given in this work might be interpreted. Please see PDF with more specific comments.
3. There are no studies cited in the conclusions section. The results of this work needs to be compared and contrasted with others.
4. Figures 3 to 8 are poorly captioned and poorly presented in the narrative. Unless the reader is familiar with this specific model output, little is offered in terms of what the figures depict - or their point.
5. Figure 1 is also not captioned appropriately - which is BBN and which is logistic regression? Side-by-side plots have differing axes labeling, tick marks and even font sizes. Please make them consistent, and consider overlaying the two graphs - assuming that they contain similar information. What is ROC index? It is shown in Figure 1, but not explained in the narrative text, or described in terms of what it signifies.

Comments for author File: Comments.pdf

Author Response

Responses were directly added to the PDF. Revisions are made in text and figures based on suggestions.

Author Response File: Author Response.pdf

Reviewer 2 Report

• Overall comments about article:
  • Methods section is thorough and helps guide reader unfamiliar with machine learning through the authors’ thought process
  • Text in the results section is clarifying and enlightening but gets muddied by confusing figures
  • Clarifying tables like Table 1 are very helpful for reader understanding
• In-line edits have been made to the PDF and attached to this review; edits focus on clarifying text to ensure the manuscript is more accessible to a broader stormwater audience
• Authors switch between GSI/GSIs/GIS and use “infrastructures”, they should edit the text to only refer to green stormwater infrastructure (works for both single and plural) and use only GSI
• Listed hypotheses are awkwardly placed at the end of the introduction; manuscript could be improved by moving to methods and citing papers that lead to the formation of each hypothesis since these are not explicitly called out in introduction; could also add paragraph to introduction to pull in these sources so when the hypotheses are presented the reader already has context
• Add clarifying and larger text to Figure 1, suggested edits provided in annotated PDF
• Manuscript would benefit from having a table or figure that lists all the variables included in the analysis
• Figure 3-8 + tiles in Figure 9-13 are not reader-friendly; update to the have text associated with the binary numbers, update tile titles to have spaces (Federal is misspelled), layout of tiles within figures are visually awkward, could these be condensed into a table?
• General question: the manuscript refers to “adoption of GSI” for every scenario, are all these households assumed to have installed GSI? Or do some merely live where previous owners or landlords have installed GSI? Using “adopt” in that context could be misleading. A table/figure that shows the spread of input data in the methods section could be helpful for comprehension

Comments for author File: Comments.pdf

Author Response

Responses were directly added to the PDF. Revisions are made in text and figures based on suggestions.

Author Response File: Author Response.pdf

Reviewer 3 Report

2020 Review of Ren et al. Modeling the Influence of Public Risk Perceptions on the Adoption of Green Stormwater Infrastructure: An Application of Bayesian Belief Networks Versus Logistic Regressions on A Statewide Survey Of Households In Vermont.

 

Overall Comments

Overall, this is a very interesting analysis, which should provide useful and informative results for city, county, and other watershed and stormwater management planners.  The paper is well organized and is not too complex to understand.  Most of my comments below are about adding details here and there to clarify sentences or other sections, particularly the figure captions, for the reader.  

 

One general comment I have about analysis and/or discussion is that I did not noticed whether the level of urbanization (such as percent impervious surface cover) was a variable used in this analysis and this is typically a strong determinate in whether GSI is needed, particularly in more urban areas, due to excess runoff from high impervious surface cover, known to cause “urban stream syndrome.”  Could you please, at the minimum, address this with a few sentences in your discussion?  The paragraph in lines 400-402 might be a good place to add this.  It would be interesting to see how your BBN would be impacted if percent impervious surface cover or percent urbanization was added as a variable, but you could possibly just use your county, residence type, or residence size as surrogates for the level of urbanization and not need to redo the analysis.

 

Line by Line comments

 

Line 40. “Risk perception” is a term that should be defined.  Adding a sentence to describe what is meant by risk perceptions would be helpful, since the term is used later in the paper. 

 

Lines 83 & 84.  Do you mean “GSI” instead of “GIS”?

 

Line 83. It may help to start a new paragraph when you begin to talk specifically about Vermont.

 

Lines 103-111. It would help to provide a few references on supervised and unsupervised learning.

 

Line 115. Can you define “responsibility attribution?” Possibly by just adding a parenthetical description right after.    

Lines 139-141. This seems like an important sentence and it would help to expand upon this by providing more details on one or more sentences.  Because right now, this concept is not as clear as it should be.

 

Line 141.  Should this be “trials” and not “trails”

 

Line 149. This part: “in the influence chains of the network,” is a little unclear on what you mean.  Can you add a few words to better describe what is meant by “influence changes”?

 

Line 187. Mention the specific type of unsupervised learning technique used (e.g. cluster analysis) here, or if it is mentioned later, the add a note to say something like “specific types are listed are below.”

 

Line 195. Same as previous question: can you name the specific type of supervised learning method you are using here?

 

Line 195.  What is the “behavior question” and what does “Q14” stand for?  These need to be better defined or referenced.  Is there a list of 14 different questions somewhere?  Also, because Q14 seems to be important in all BBN figures, it is important to make sure it is clearly defined and how it is used. 

 

Line 198.  “Target question.” Can you add “(e.g. example of target questions)”

 

Line 206. Unneeded parenthetical before “MDL”

 

Lines 225-236. This material should be moved to the results section.

 

Line 259. What is meant by “the inference”?   It would be clearer to state “the BBN in Figure 2…”

 

Line 260.  It would help if “roof diversion,” was used as the label in Figure 2 instead of just “current_diversion.”  Otherwise, it isn’t obvious that this is a “roof” diversion.    

 

Lines 260-262.  It would help if you could reference Figure 2 at the end of the sentence to remind the reader you are still talking about this figure.

 

Lines 262-264.  There seems to be a lot of information in Figure 3 but results for only one part of Figure 3 is mentioned.  Can you provide some more results for the other aspects of Figure 3?

 

Lines 268-275. It would help to have each sentence in this paragraph reference a specific chart in Figure 4 or 5.  Such as ending each sentence with (Figure 4a) or (Figure 5d). 

 

Lines 289-290.  It would help to have each sentence in this paragraph reference a specific chart in Figure 6.  Such as ending the sentence with (Figure 6a) or (Figure 6d). 

 

Lines 294-299. It would help to have each sentence in this paragraph reference a specific chart in Figure 7 or 8.  Such as ending the sentence with (Figure 7a) or (Figure 8b). 

 

Line 311. It may help to define what is meant by “positive answer” here or in the methods.  It is not completely intuitive to know that “positive answer” means the respondent said yes to the question.     

 

Lines 311-356. Like my above comments, it would help the reader understand these results if you could point the reader to the specific component/chart within each figure.  Because it is hard to tell if the results you are describing are based on the network diagram or the charts below the diagram.  These figures are not intuitive and the more details you can provided, the better. 

 

Lines 361-362.  I don’t see “county” listed in Table 1.  Please clarify. 

 

Comments on the Figures

 

Figure 1.  This figure needs some improvement.  It is not clear which charts correspond to the BBN and which charts correspond to the logistic regression.  The labels go from G14A to Q14H, but E and F are missing. It would help to have the charts on the left be similar to the charts on the right.  Currently, they have a different scale: the left charts go from 0-1 and the right charts go from 0-100.  Also, the right chart uses red, while the lefts charts are black and white.  Lastly, the text on the right panels is too small to read. 

 

Figure 3.  There is a lot of information in this figure and it is hard to understand.  Can you add some more description to the figure caption to help the reader know how to interpret this figure?  For example, what does “yes” or “no” mean or “1” vs. “0” mean? 

 

Figure 4.  Like my comment for Figure 3, please add some more details to the caption to help the reader know how to interpret this figure.  Also, can you add labels to each of the charts in this figure (e.g. a through g).  And, if the figures in the same column are supposed to correspond with each other, please make that more clear.

 

Figure 5.  My comments for Figures 3 and 4 apply to Figure 5 as well.

 

Figure 6.  My comments for Figures 3 and 4 apply to Figure 6 as well.

 

Figure 7.  My comments for Figures 3 and 4 apply to Figure 7 as well.

 

Figure 8.  My comments for Figures 3 and 4 apply to Figure 8 as well.

 

Figures 9-13.  Like above figures, please add more to the caption to briefly describe how these figures and charts should be interpreted.  Also, please add labels (e.g. a, b, c, …) to the different charts within each of these figures (this will help the reader better follow along with the text in the results section). 

Author Response

Please see the attached document for the responses. Revisions were made in text and figures based on suggestions.

Author Response File: Author Response.docx