Managing Rockfall Hazard on Strategic Linear Stakes: How Can Machine Learning Help to Better Predict Periods of Increased Rockfall Activity?

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper uses now common machine learning tools in an attempt to better predict rockfalls along a road corridor, with the objective of making the geohazard management process (e.g., durations of road closures) more efficient. I did not see any problems with the way the authors implemented the machine learning models. The results show that the machine learning models work somewhat well. 

Here is my primary concern with the paper: Perhaps I have misread, but my understanding of the paper is that the machine learning models, while performing somewhat well, do not work as well as the old-fashioned use of human expert opinion. From the perspective of a hazard manager, why would I implement a method that does not work as well as the existing procedure? I think the authors need to explain in a compelling manner why their work represents a publishable advance in knowledge if it does not work as well as the older and simpler expert method. Or, change the focus of the paper to the superiority of a traditional expert based approach to machine learning. Perhaps the value of machine learning will eventually be an ability to predict the volume or number of rockfalls on a given day, not simply whether any rockfalls occur, as the authors suggest for future work. Are the authors able to do that with the data they currently possess? If so, it should be incorporated into a new version of the paper.

Secondarily, I am not sure if the paper is appropriate for a journal called Sustainability. It seems like a reasonable geohazard paper that, if strengthened by using machine learning models that do a better job than human experts, would be fully appropriate for journals like Engineering Geology or Natural Hazards. If the paper is to be suitable for Sustainability, the authors need to do a better job of explaining clearly to readers how their work contributes to sustainability. As the paper is currently written, readers are left to guess about sustainability. I think the authors try to suggest that a good rainfall-based rockfall model would decrease the number or length of protective structures necessary, which might contribute to sustainability because it might require less concrete and steel, but they are not clear about that and, in the end, their methods are not an improvement over the existing expert method.

Also secondarily, the paper is far longer than necessary. This is in part because the authors have included an introduction to basic machine learning concepts in their manuscript. For example, they devote an entire paragraph to an explanation of the confusion matrix (which is well known). That is unnecessary. The authors should simply cite one or two introductory papers or books and give the briefest overview possible in their paper. Based on the amount of substantive content, the final paper should be closer to 20 typeset pages than the current 32 typeset pages.

Finally, there are some minor issues with the figures. For example, Fig. 5a and 5c seem like they show the same variables over the same ranges of values, but for different times, and thus should be the same size. But, they are different sizes and it appears that the title of Fig. 5c is truncated. Likewise, the "as" at the beginning of the second line of the title of Fig. 5d is truncated. The authors also use "Nb" in the axis labels for several figures. I am guessing that is an abbreviation for Number, but it is a non-standard abbreviation and it would be better to simply use the entire word Number. Those things should be easy to fix.

Comments on the Quality of English Language

The authors do a better job in English than I would do in French, and I admire them for that. However, I think the paper could be shortened considerably (see previous comment regarding paper length) by editing the text for efficiency and economy of phrasing. The authors would do well to ask a technical editor with expert-level fluency in English help them refine and more concisely phrase the paper, which would help to shorten it. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

1. The paper lacks innovation, and there are already a large number of use cases available for deep learning in areas such as landslides. The input parameters are too few and the model predictions are not very accurate.

 

2. In the section 4.2, the treatment cannot be called model optimisation, it is more biased towards the treatment of datasets, like dataset segmentation, n-fold cross validation are very common in deep learning.

 

3. The presentation of Fig. 5 is obscure and the data is not sufficiently presentable, resulting in a failure to adequately present the conclusions of the paragraph.

 

4. In lines 841-844, “In details, we can see (Figure 11) that the bagged tree, DNN and logistic regression are slightly better in term of recall than the expert model whatever the study period and the analyzed characteristics (number of rockfalls or mass)……” Please describe with specific numerical values.

Comments on the Quality of English Language

Minor editing of English language required

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Dear author, I recently had the opportunity to review your manuscript titled “Managing Rockfall Hazard on Strategic Linear Stakes: How Can Machine Learning Help to Better Predict Periods of Increased Rockfall Activity?” I appreciate the valuable insights you’ve provided in your research.

·        Please suggest avenues for future research. Could additional features (e.g., soil properties, slope characteristics) improve model performance?

·        What are the practical implications of your findings for transportation infrastructure managers? How can they use these predictions to enhance risk management strategies?

·        I found some relevant papers that you can cite them in your work like: you can cite this paper in your work: “Machine Learning-Based Rockfalls Detection with 3D Point Clouds, Example in the Montserrat Massif (Spain)”

 

·        I  noticed that the manuscript is quite extensive. Could you please consider condensing it to a more concise form?

You can use some numbering methods like 1-a, 1-b ,... for the figures that you provided.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

Introduction and study site, are clearly described.

Here figure 1 could be improved and have a better definition, for example, part b identifies a 90 m high cliff while in text is said to be between 100 and 200 m. And in part d) geology identification is not so clear, perhaps it is the scale, but the lithology structure is not appreciated well. Besides figure cannot show the volumes of rockfall, it is something that has to be assumed.

Data section

It is well described but information should be presented before introducing it into the text, for example, parameter Bt, Nt or Mt are introduced in line 222, but it is not really explained what they are. Your find its meaning when reading the next sections of the article. This must be revised.

Figure 2 could be clearer if both series were draw in the same graph.

It is clear that rain is an adequate parameter for the study, but rainwater can flow inside rock mass due to permeability conditions, (rock fractures). This could be an influence factor and depends on the rock mass properties and it should be taken into account. Probably rock mass conditions also influence in the rockfalls but there is not zone discretization in the study if not necessary an explanation is required.

Machine learning models

The ML models are identified and described properly. Figure 7 really helps to the understanding. Metrics are also clear and well exposed.

Calculations and results

This section is according to the information presented in the article. Mass study perhaps it is not clear since rock mass falls depends on the size of the of rock fragments and this study is not focused on the structure of rock geological units. To get a better understanding of this problem further geological studies should be done.

Discussion

The discussion section is wide argued. Perhaps since there are four ML models at the end of the section it should be identified which of them has a better implementation to the problem an under what circumstances. Considering authors text it seems that depending on the period considered, 2000-2006 or 2009-2018, different ML models give better approach so it is not clear what should be use for future studies. All ML models has always to be used?

Conclusion

It is according to what is expressed in the article, but if some corrections are made it should be taken it into account.

Overall it is an interesting approach to a difficult characterization problem. It is an interesting work that could be improved if some geological information is also introduced as variables of the problem.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 5 Report

Comments and Suggestions for Authors

the manuscript explores the contribution of machine learning to the prediction of rock falls on linear infrastructures (roads, railways, etc.), the study is well done except for a few remarks that need to be rectified :

1- The introduction contains sentences from the absract.

2- Figures can be improved (Fig 7 and 9).

3- Based on the results obtained, wich is the most effective model for predicting rockfalls ?

4- which factors (real and statistical) influence the results of each model ?

5- I prefer to put the geological map in the first manuscript section, if you can put the road on the geological map that would also be ideal.

6- Figure number 6 is not clear, i suggest to change it with a simple graphs that show the relation between Rainfall conditions on days with rockfalls.

The methodology, résultats and interpretations are well developed.

You've focused your model on rainfall as the determining factor, but geology also changes from one sector to another along the road studied. In a future study, I suggest making sections of the road according to the geological context of each part, then running the model in each part (to fix the geological parameter).

Comments on the Quality of English Language

The quality of English is good.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors appear to have addressed my major concerns. Their data appear reasonable and their application of existing machine learning methods correct. As someone who has spent decades working on geohazard and risk assessment, I still doubt their conclusion that the machine learning approach is a significant improvement over human expert judgement in this situation (this is not to discount machine learning, just the practical utility of its results and promise compared to existing methods in this specific geohazard problem). However, I do not think the utility of the method in practical situations should necessarily prevent publication of a methodologically correct paper based on  good data, and others may find the authors' approach and results useful.