Up-Scaling Fuel Hazard Metrics Derived from Terrestrial Laser Scanning Using a Machine Learning Model

Round 1

Reviewer 1 Report

Manuscript ID: remotesensing-2136199

Title: Upscaling fuel hazard metrics derived from Terrestrial Laser Scanning

In this study, the authors have performed a study using machine-learning-based method for estimating fuel metrics for operational fire management decisions. Authors have provided a very nice introduction and clearly introduced their objectives. The subject is worthy of investigation. Results are interesting and can be helpful for the investigation of fire weather danger. I have some comments before recommending the manuscript for publication: 

1.     Authors may consider changing the title of the manuscript to highlight they have used machine learning models. ML aspect of the study became evident to me only after going to methods section. I leave this to authors decision.

2.     Please discuss the drawbacks of previous studies connecting it with the current study to justify the novelty of work clearly.

3.     How do authors justify the validation of the current study and, please briefly discuss the different models used in the previous study and their advantages/disadvantages to justify the current work?

4.     Please also discuss the future directions of the work.  

5.     Analysis seems reasonable and even the results seem reasonable and have been analyzed sensibly. However, my experience is that models are often wrong. Here, for the results in this paper to have any meaning at all, one must trust the models upon which they are based. We are not presented with any reason to trust them. Their output is basically treated like observed data. Please discuss

6. How does the result of this study compare with those from traditional weather-based models, or how ML approved the previous approach? Please provide a comprehensive discussion on this.

7.     Authors could present some results and comparison with the previous studies

Author Response

Please see the attached response document.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript “Upscaling fuel hazard metrics derived from Terrestrial Laser Scanning” submitted to the journal Remote Sensing describes how random forest models built using terrestrial lidar (TLS) and visual fuel assessments perform when modeled using Sentinel data.  I feel that this paper is overall well written and interesting because of the lack within the literature of the fusion between TLS and satellite data.  However, I have some few suggestions on ways to improve the manuscript (please see below):

1)      Consistently capitalize “Random Forest” or “random forest” throughout the manuscript.

2)      What was the study sample size per your TLS and OFHAG methodologies?  I’m assuming that your sample size would be larger than the number of study sites (N=7).  Sample size would really impact the quality of your models, especially given the variability among study sites.  Please make sure to indicate somewhere the total number of samples.  You should also indicate in the discussion that some (maybe much) of the error in your study is caused by the relatively small sample size.

3)      Please explain in greater detail how your random forest models were tuned.  What hyperparameters were used?

4)      Why didn’t you test for model importance of each variable?  You could potentially make the model more parsimonious, more accurate, and quicker to run if you dropped variables with low model importance (e.g., less than 2%).  At the very least it allows you to determine and explain why your model performance.  Additionally, it might identify some variables (perhaps soil characteristics) that lead to model overfitting.  I know that the rule of thumb is that random forest models don’t overfit, but this is not entirely true depending on variable characteristics.

5)      Withholding test data and comparing them to the outputs is map accuracy.  How did you evaluate map accuracy?  You should specifically explain your methods of assessing accuracy within the Methods section.

6)      Additionally, all statistical methods discussed in the results section should be explained in the methods section.  For example, fuel cover estimates from TLS and the visual assessments are not explained.

7)      Below are some additional line x line comments.

Line X Line comments:

12: End the abstract with a concluding sentence that states why these results are important, and/or how they will be used in the future.

20-21: Risk from fuel conditions?

22-32: Maybe merge this paragraph with the next one.  It would also be good to explain that by modifying the fuel inputs that go into CFD's that you can potentially model the fire behavior for a given area, including the effect of modifications.

74: Spacing between “canopy cover” and citation.

119: What is a bark hazard?  It is not a common term outside of Australia.

155: How was the threshold determined?

182-184: This needs more explanation.

238-239: Which compositing method was used?

245: Delete “of” before “fuel”.

256: Capitalize “Random”.

Figure 3 caption: “Near” should not be capitalized.

295: A correlation of 0.6 is not a strong correlation.  It is at best a low-moderate correlation.

307-309: The “Where the near..” sentence is not currently complete.  Please rewrite.

349: “South East” should be “Southeast”.

366-367: This sentence doesn’t currently make sense as written.

373-374: Commonly used hazard ratings should be explained more in the introduction or here.  I don't know the commonly used hazard metrics in Australia.

382: Maybe use “site placement” instead of “sites”.

400-403: Integration of what information?

424-425: Is there a citation for the statement “Of these two technologies, it is believed that TLS is a preferred technique”?

430-431: This needs more details or a citation for an example with landscape specific sampling.

Author Response

Please see the attached response document.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Authors have answered all of my comments. I recommend the manuscript for publication. 

Author Response

Thank you for your time. Your comments and suggestion have contributed to an improved manuscript.

Reviewer 2 Report

The manuscript “Upscaling fuel hazard metrics derived from Terrestrial Laser

Scanning using a machine learning model” is much improved.  Below are a few minor comments/questions by which to improve the manuscript.

Line X Line comments:

142-144: Maybe combine the sentences and change them to: “Bark hazard is associated with the loose bark on tree trunks and branches, and can create extreme fuel hazards.”

520:  The abbreviation “RF” is not used in the paper.  Please remove from the abbreviations.

Figure A8:  There are image anomalies on the right side of the image.  Are these actually there are just within the image for some reason?  If they’re actually there, what is causing these problems?

Figure A9: Similar problem to A8 with image anomalies in the center of the image.  Please see A8 for questions.

Author Response

The authors would like to thank the reviewer for the valuable suggestions.
The suggestions offered have been helpful and we appreciate the thoughtful
comments made by the reviewer which have helped improve the overall clarity
and completeness of this paper. We hope that our responses to comments
provide clarity on the importance of our contribution to the literature.

Specific responses (normal text) to editor and reviewer’s
comments and suggestions for the paper (in bold) are provided below. Changes made in the main manuscript are in blue italics text.

We have provided line numbers for the new text added in the manuscript.

The manuscript “Upscaling fuel hazard metrics derived from Terrestrial
Laser Scanning using a machine learning model” is much improved.
Below are a few minor comments/questions by which to improve the manuscript.
Line X Line comments:

1) 142-144: Maybe combine the sentences and change them to: “Bark
hazard is associated with the loose bark on tree trunks and branches,
and can create extreme fuel hazards.”

Thank you for the suggestion. We have combined the sentences as
follows:

Line 139-140
From: Where bark hazard is associated with the
potential ignition of loose bark on the tree trunk and
branches. In Australian conditions it can present
levels of fuel hazard up to extreme.
To: Bark hazard is associated with the loose
bark on tree trunks and branches, and can create
extreme fuel hazards.

2) 520: The abbreviation “RF” is not used in the paper. Please
remove from the abbreviations.

We have removed the abbreviation ”RF”.

3) Figure A8: There are image anomalies on the right side of the
image. Are these actually there are just within the image for some
reason? If they’re actually there, what is causing these problems?

The authors would like to thank reviewer for the feedback.
These anomalies were mainly caused due to (1) the low resolution
environmental inputs (soil and climate). No variations were
observed in soil data within the plot. (2) The method used to randomly
sample plot locations within the burn units in this study is
currently implemented by Forest Fire Management Victoria staff
and was developed to balance the capture of useful information and
efficiency of data capture. As such plots were situated at least
100m and no more than 500m away from the road edge (to reduce
edge effect whilst balancing human safety factors) with the number
of plots scaled to the size of the burn unit. For ’Toorloo Arm’ site,
plots (8) were mainly towards the west of the site. The determination
of site placement did not consider changes in the landscape
factors that are used to assist in the scaling up of TLS data points.
This likely caused the Random Forest predictions to fall within a
narrower band than the overall fuel conditions at each location.
These anomalies were observed where no scans were taken. Therefore,
it is recommended that sampling locations should be guided
by environmental differences across the landscape. It is likely that
location of plots spanning a greater range of fuel conditions will be
obtained and in turn model accuracy estimates will likely improve.
We provided discussion around this point in discussion section.

4) Figure A9: Similar problem to A8 with image anomalies in the
center of the image. Please see A8 for questions.

Please see our response to comment 3.

The authors would like to thank once again to the reviewer for the suggestions
and valuable time invested.