Damage Detection and Level Classification of Roof Damage after Typhoon Faxai Based on Aerial Photos and Deep Learning
Round 1
Reviewer 1 Report
This paper presented a method used to detect the damaged roof after Typhoon Faxai based on aerial photo and deep learning and classify it damage level, The study used the neural network Mask R-CNN to detect the damage of different object. Although this is an interesting paper, which is well organized and provide a relatively comprehensive investigation. It can be considered for the publication in the Journal of Applied sciences-Basel. However, there are some drawbacks and minor revision is needed.
(1) This paper is about damage detection based deep learning. Therefore, to emphasize this, I suggest the authors to review the diverse studies on the application of the vibration-based method under environmental such as “Integration of Time Series and Neural Network under Varying Temperatures”.
(2) In the paper, the loss value is reduced using the stochastic gradient descent, in my opinion, swarm Intelligence optimization algorithms are widely used in the field of optimization, such as WOA (Superposition of Modal Flexibility Curvature and Whale Optimization Algorithm), and MFO (Damage Identification of Bridge Structures Considering Temperature Variations-Based SVM and MFO). These algorithms have shown great optimization performance and they may be combined with neural network and deep learning to update their network parameters.
(3) Minor spell check is required, such as the ‘train’ in Figure 8 which should be ‘training’.
(4) The advantage of deep learning used in the field of damage detection of the damaged roof after Typhoon should be highlighted in the section of introduction and summary.
Author Response
Please see the attachment.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The paper presents a detection and level classification model of roof damage. The observation data are aerial photos and the core model is a Mask-RCNN. This work involves four aspects: (1) data collection and manual labeling, (2) training Mask-RCNN model, (3) detecting roof outline, blue tarps, and roof completely destroyed with the trained Mask-RCNN, (4) determining damage levels based on the detection results.
The model design of this paper is reasonable and the experimental results are acceptable. However, there are some issues to be clarified in the further version.
1. K fold cross validation should be used to evaluate the proposed model. In the current manuscript, 18 aerial photos are used to train, while 5 aerial photos are used to test. There is a risk that the difficulty of 5 testing photos is lower than the training photos. K fold cross validation can provide a more reliable evaluation.
2. The time consumption of the proposed model should be detailed. Line 560 stated that “…, while time spent on the automatic detection in the later of the automatically detection is neglected”. What does the word “neglected” mean? Authors should quantitatively evaluate the time consumption of each step. For example, for a specific hardware configuration, how many seconds are required to solve a 10000x10000 aerial photo with the trained Mask-RCNN? What about the speed of damage level classification introduced in Figure 14?
3. In line 467, the authors stated that “In Table 6, the accuracy and F value of the classification of each damage level are calculated separately”. However, Table 6 only shows the number of correct and false samples. The accuracy and F value should be shown in this table.
4. The flowchart in Figure 2 is unclear and inaccurate.
(1) Unclear: In an actual application, model training is an offline process which is generally separated from model inference and testing. However, Figure 2 describes model training and testing as a high-coupling process. It would be more intuitive to show the training and testing with two separate flowcharts.
(2) Inaccurate: Some judgment criteria in diamonds are not reasonable. For example, “Whether trained model meets the required accuracy” and “Whether result fits the actual level of roof damage” are sufficient termination conditions. A model may fail to meet the expected accuracy even if we have collected a large training set and set reasonable parameters. Authors should rethink all the judgment criteria carefully.
5.The flowchart in Figure 14 is redundant. For example, the judgement “Whether blue tarps detected” after “non-object (without completely destroyed)” is redundant because it does not influence the classification result. Furthermore, the flowchart can be simplified by performing “Whether roof completely destroyed detected” as the first judgement. The authors should revise this flowchart to simplify its structure.
6.Line 486 stated that “Equation (12) is used to …”. However, the manuscript has no definition of Equation (12).
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 3 Report
Please see the attached file.
Comments for author File: 
Comments.pdf
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Reviewer 4 Report
Dear authors,
you have written interesting article according to my opinion, and to my field of interest. English language is fine and doesn’t need any improvements. You didn’t cite all the literature. I didn’t find any major flaws in the article, i.e. in the methodology, presentation and results.
The good points of the article are:
- interesting topic regarding quick damage assessment after typhons for residents and their need for quick dispatch of house repair and disaster insurance works
According to my opinion, the weak points of your article are:
- to my opinion the influence of obtained aerial photos is not explained thoroughly
- design of the article
- all references are not cited in the article
The improvement should be done in the following:
Ad. 1) I think that the quality of obtained aerial photos is not thoroughly explained in the article, i.e. in the sections 3.1 and 3.2. It is not clear at what flight height they were taken, what is the aerial photos i.e. the GSD (ground sampling distance). Further, are they taken with planes or Drones. Furthermore, how was the digital orthophoto made, with what software?
These facts are important for your research, i.e. for the training and detection of damage roofs according to my opinion and I think that they should be explained.
Ad 2.) The design of the article i.e., the titles of the sections should be renamed according to my opinion:
- Section 3: Materials (aerial photo) to Materials
- Section 6: Comparison with other classification methods to Discussion
- Section 7: Summary to Conclusions
Ad. 3)
The references 12, 14, 23, 24, 25 are not cited in the article. I could find them Please have a look at it and correct if necessary.
Suggestion by “lines”:
- line 19 – please replace the F value with The…
- line 59 – please replace Field with field
- line 164 – please delete the brackets and aerial photo – it is clear from section 3.1 Used aerial photos
- line 183 - please replace Area with Aerial
- line 229 – what is the relevance of figure 5? If there is relevance, please explain above the figure from line 220.
- line 354 vs. line 363 – figures 9 and 10 have the same title. I suppose that the figure 10 should have been named Result of validation in training
- line 378 – Table 5 – are there any units for these indices, i.e. for accuracy and precision?
Best regards.
Author Response
Please see the attachment.
Author Response File: Author Response.docx
Round 2
Reviewer 2 Report
All my concerns were discussed and responded well.
Reviewer 3 Report
I have no other concerns.
