A High-Precision Ensemble Model for Forest Fire Detection in Large and Small Targets

Round 1

Reviewer 1 Report

The authors present a paper in which two of their proposed models (WSB and WSS) are used for forest fire detection. The combination of two models makes it possible to identify small forest fires as well as large ones, according to the research work carried out by the authors. To evaluate the proposed models, the authors use standard metrics for evaluating artificial neural networks. The proposed document is interesting, but I have a few comments and suggestions. Authors use many abbreviations in the text and it would be good practice to write the full meaning of each abbreviation when it is first used in the text. Line 39 says "China has a total forest area of 210,000 square kilometers, accounting for 21.7% of the country's total area" - the numbers need to be double-checked. It will be interesting if the authors in their future papers compare the results of their approach with the results achieved by other researchers.

Author Response

Authors’ Reply to Reviewers

Thank you for giving us the helpful comments and suggestions, which helps us to improve the quality of the manuscript. In the revised version, we have revised their concerns and changed made in the revised manuscript are red in color. The followings are our replies to the reviewer’s comments: the words in blue are the questions of the reviewers, and the words in black are our replies.

Reviewer #4:

1. Authors use many abbreviations in the text and it would be good practiceto write the full meaning of each abbreviation when it is first used in the text.

Reply to Question 1: This is a good question and we have made necessary changes.

We have revised the references in the text. For each abbreviation used for the first time in the article, we have indicated its full meaning.

2. Line 39 says "China has a total forest area of 210,000 square kilometers.accounting for 21.7% of the country's total area" - the numbers need to be double-checked.

Reply to Question 2: This is a good question and we have made necessary changes.

The figures in the original article are with according to the data of 2018 in the first reference, in order to improve the real-time data, it is now changed to the latest China's forest area and coverage in 2022 "As of 2022, China's forest area is 231 million hectares, and the forest coverage rate reaches 24.02% [1]" The reference is changed to [1] Bulletin of China's land greening status in 2022. Land Greening,2023(03):6-11.

Author Response File: Author Response.docx

Reviewer 2 Report

Dear authors, the paper is very well written.

I have followed it quite well because the subject matter is close to me, but not so much the tools used.

From my point of view I think that the tools should be described more, so that everything is more justified.

I agree with the authors that it is difficult to obtain real data, which makes it difficult to validate these models. Even so, I understand the effort they have made in the validation. 

I miss comparison of results with other similar works to see the consistency of the proposed model.

Author Response

Authors’ Reply to Reviewers

Thank you for giving us the helpful comments and suggestions, which helps us to improve the quality of the manuscript. In the revised version, we have revised their concerns and changed made in the revised manuscript are red in color. The followings are our replies to the reviewer’s comments: the words in blue are the questions of the reviewers, and the words in black are our replies.

Reviewer #2:

1. I have followed it quite well because the subject matter is close to me, but not so much the tools used. From my point of view I think that the tools should be described more, so that everything is more justified.

Reply to Question 1: This is a good question and we have made necessary changes.

This paper describes the deep learning framework and hardware configuration required for model building in this paper in subsection 4.1. experimental environment and parameter setting. In the future, we plan to deploy the model on UAVs or surveillance cameras for real-time forest fire monitoring. By combining the model with UAVs or surveillance cameras, we can monitor forest fires over a wider area and take timely measures to minimize the harm of fires to the environment and humans. This extension will give our model greater utility and adaptability to be applied in different scenarios and environments. We will continue to work on improving and optimizing the model to ensure its stability and performance when deployed in UAVs or surveillance cameras. We look forward to validating and demonstrating the model's potential for forest fire monitoring in real-world applications in the future.

2. I agree with the authors that it is difficult to obtain real data, which makes it difficult to validate these models. Even so, I understand the effort they have made in the validation. 

Reply to Question 2: This is a good question and we have made necessary changes.

We appreciate your feedback and understanding of the challenges involved in obtaining real data for validating forest fire detection models.

3. I miss comparison of results with other similar works to see the consistency of the proposed model.

Reply to Question 3: This is a good question and we have made necessary changes.

This paper has been supplemented with comparison tests with earlier models as shown in the table below. We compare the WSB model, WSS model, and WSB-WSS model built in this paper with the existing mainstream target detection models. Although the accuracy of the integrated WSB-WSS model is slightly improved compared to the WSB model and the WSS model, the results of the comparison experiments show that the accuracy of the WSB model and the WSS model built in this paper has been greatly improved compared to the existing mainstream models. And the WSB-WSS model improves the accuracy rate again on this basis, and the detection effect is obviously better than the existing mainstream model. In addition, this paper aims to solve the problem of simultaneously realizing high-accuracy detection of forest fires with large and small targets, and in the future we will also conduct research on forest fire smoke.

Author Response File: Author Response.docx

Reviewer 3 Report

The paper addresses the very important and challenging topic of forest fire and current detection models. The authors utilized  two weakly supervised machine learning (ML) models to alleviate the current models’ shortcomings to achieve higher detection accuracy in both of small and large complex environments.  The authors claim that by integrating the two ML models, where one is for small size and the other for large size fires, with the aid of a new edge loss function, regardless of the size of the fire area the hybrid model will perform much better in detecting the fire size and location. Yet, the new model enhancement over the original two models is very low, one digit only, which is the major problem with this paper, if such enhancement is the target goal of the paper! Another point is, most fires small and grows large, hence the need for small fire detection is much more important than waiting until it becomes large! Better yet, instead od waiting to detect, which might be too late, it is much more useful and life saving to “early predict” fires (at small initial areas, before spreading). Runing both model in parallel will cover both the small- and large-scale fires without the need to hybrid!

My major issue with this paper is the confusion of the following para (very long one sentence!):

“Therefore, this paper integrates the two models based on the WBF method and designs a more efficient forest fire detection model WSB_WSS, which simultaneously realizes high-precision large-target forest fire detection and small-target forest fire detection, with small-target forest fire detection accuracy up to 83.3%, and large-target forest fire detection accuracy up to 93.5%, which enhances the detection accuracy compared with that of the original basic network model by 3.1% respectively, 4.2%.”

Where is the original basic network model  results are published to be able to conclude the obtained enhance enhancements of 3.1% and 4.2%?! Which I could not find it anywhere in the paper!!!

Another major comment to the authors:

From line 26 up to line 32, all I can compare to is the difference between is the old 82.4% versus the new 83.3% for small scale fire, with a very slight improvement of 0.9% ONLY; and the old 92.8% versus the new 93.5% for large scale fire, with a very slight improvement of 0.7% ONLY!

My question to the authors is:

Is it really worth it to publish journal paper, spending that much time in the writing/analyzing of many algorithms to achieve ONLY 0.9% and 0.7%?

You need to read the paper again to filter out many English (technical writing) writing errors, e.g., in line 456: "83.3% for small target forest fire detection accuracy and 93.5% for large target forest fire...."

Author Response

Authors’ Reply to Reviewers

Thank you for giving us the helpful comments and suggestions, which helps us to improve the quality of the manuscript. In the revised version, we have revised their concerns and changed made in the revised manuscript are red in color. The followings are our replies to the reviewer’s comments: the words in blue are the questions of the reviewers, and the words in black are our replies.

Reviewer #3:

1. The paper addresses the very important and challenging topic of forest fire and current detection models. The authors utilized  two weakly supervised machine learning (ML) models to alleviate the current models’ shortcomings to achieve higher detection accuracy in both of small and large complex environments.  The authors claim that by integrating the two ML models, where one is for small size and the other for large size fires, with the aid of a new edge loss function, regardless of the size of the fire area the hybrid model will perform much better in detecting the fire size and location. Yet, the new model enhancement over the original two models is very low, one digit only, which is the major problem with this paper, if such enhancement is the target goal of the paper! Another point is, most fires small and grows large, hence the need for small fire detection is much more important than waiting until it becomes large! Better yet, instead od waiting to detect, which might be too late, it is much more useful and life saving to “early predict” fires (at small initial areas, before spreading). Runing both model in parallel will cover both the small- and large-scale fires without the need to hybrid!

Reply to Question 1: This is a good question and we have made necessary changes.

The forest fire detection model WSB_WSS established in this paper can achieve high-precision detection of small-target forest fires, timely detection of early forest fires and timely control of the spread of forest fires; it can also realize high-precision detection of large-target forest fires, provide more accurate information about fires for the fire-fighting commanding department, and help the commanding department to reasonably allocate fire-fighting resources. Through scientific resource allocation, it can maximize the efficiency of fire fighting, reduce the waste of resources, and protect the ecological environment and natural resources as much as possible. Therefore, the detection of both large target fire and small target fire is very important.

In addition, this paper has been supplemented with comparative experiments as follows. We compare the WSB model, WSS model, and WSB-WSS model established in this paper with the existing mainstream target detection models. Although the accuracy of the integrated WSB-WSS model is slightly improved compared to the WSB model and WSS model, the results of the comparison experiments show that the accuracy of the WSB model and WSS model built in this paper has been greatly improved compared to the existing mainstream models. And the WSB-WSS model improves the accuracy rate again on this basis, and the detection effect is obviously better than the existing mainstream model.

2. My major issue with this paper is the confusion of the following para (very long one sentence!): “Therefore, this paper integrates the two models based on the WBF method and designs a more efficient forest fire detection model WSB_WSS, which simultaneously realizes high-precision large-target forest fire detection and small-target forest fire detection, with small-target forest fire detection accuracy up to 83.3%, and large-target forest fire detection accuracy up to 93.5%, which enhances the detection accuracy compared with that of the original basic network model by 3.1% respectively, 4.2%.” Where is the original basic network model  results are published to be able to conclude the obtained enhance enhancements of 3.1% and 4.2%?! Which I could not find it anywhere in the paper!!!

Reply to Question 2: This is a good question and we have made necessary changes.

We have revised the long sentences as well as checked the sentences throughout the text and the revised sentences are as follows:

Therefore, this paper integrates the two models based on the WBF method and designs a more efficient forest fire detection model WSB_WSS, which simultaneously realizes high-accuracy large-target forest fire detection and small-target forest fire detection. The model WSB_WSS designed in this paper has an accuracy of 83.3% for small-target forest fire detection and 93.5% for large-target forest fire detection, which is significantly better than the existing mainstream models.

In addition, the phrase "original basic network model" in the previous sentence was a misrepresentation due to our mischaracterization; in the previous sentence, it referred to the YOLOv5 model that we used for the comparison test. The description has been corrected in the revised sentence.

3. From line 26 up to line 32, all I can compare to is the difference between is the old 82.4% versus the new 83.3% for small scale fire, with a very slight improvement of 0.9% ONLY; and the old 92.8% versus the new 93.5% for large scale fire, with a very slight improvement of 0.7% ONLY! Is it really worth it to publish journal paper, spending that much time in the writing/analyzing of many algorithms to achieve ONLY 0.9% and 0.7%?

Reply to Question 3: This is a good question and we have made necessary changes.

This paper has been supplemented with comparison experiments to compare the WSB model, WSS model, and WSB-WSS model built in this paper with existing mainstream target detection models. Although the accuracy of the integrated WSB-WSS model is only improved by 0.7 and 0.9 compared to the WSB model and WSS model, the results of the comparison experiments show that the accuracy of the WSB model and WSS model built in this paper has been greatly improved compared to the existing mainstream models. The WSB-WSS model improves the accuracy rate again on this basis, and the detection effect is obviously better than the existing mainstream model, so the research conducted in this paper is very meaningful.

4. You need to read the paper again to filter out many English (technical writing) writing errors, e.g., in line 456:"83.3% for small target forest fire detection accuracy and 93.5% for large target forest fire...."

Reply to Question 4: This is a good question and we have made necessary changes.

We have corrected the error here and checked the English throughout the text for changes. The revised description is below:

The integrated model achieves 83.3% detection accuracy for small target forest fires and 93.5% detection accuracy for large target forest fires, which is 3.1% and 4.2% higher than the detection accuracy of the network model without the addition of each module, respectively.

Author Response File: Author Response.docx

Reviewer 4 Report

enclosed

Comments for author File: Comments.docx

Author Response

Authors’ Reply to Reviewers

Thank you for giving us the helpful comments and suggestions, which helps us to improve the quality of the manuscript. In the revised version, we have revised their concerns and changed made in the revised manuscript are red in color. The followings are our replies to the reviewer’s comments: the words in blue are the questions of the reviewers, and the words in black are our replies.

Reviewer #4:

1. Authors have provided a neat introduction about the forest fire detection.yet, they lack in addressing the limitations in the existing related works. It would be better if the authors prepare a comparison of earlier works. It would be interesting if the authors detect the smoke.

https://doi.org/10.1016/j.jnlssr.2023.06.002

Sumathi, D., Alluri, K. (2021). Deploying Deep Learning Models for Various Real-Time Applications Using Keras. In: Prakash, K.B., Kannan, R., Alexander, S., Kanagachidambaresan, G.R. (eds) Advanced Deep Learning for Engineers and Scientists. EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-66519-7_5

Reply to Question 1: This is a good question and we have made necessary changes.

This paper has been supplemented with comparison tests with earlier models as shown in the table below. We compare the WSB model, WSS model, and WSB-WSS model built in this paper with the existing mainstream target detection models. Although the accuracy of the integrated WSB-WSS model is slightly improved compared to the WSB model and the WSS model, the results of the comparison experiments show that the accuracy of the WSB model and the WSS model built in this paper has been greatly improved compared to the existing mainstream models. And the WSB-WSS model improves the accuracy rate again on this basis, and the detection effect is obviously better than the existing mainstream model. In addition, this paper aims to solve the problem of simultaneously realizing high-accuracy detection of forest fires with large and small targets, and in the future we will also conduct research on forest fire smoke.

2. Motivation and research gap analysis is found to be generic which the authors have to narrow down to justify their contribution.

Reply to Question 2: This is a good question and we have made necessary changes.

Although the innovations in this paper are not very outstanding, this paper realizes high-precision detection of both small-target forest fires and large-target forest fires at the same time. The high-precision detection of small-target forest fires is conducive to the early detection and control of forest fires, realizing early monitoring of forest fires, effectively protecting the forest ecological environment, and maintaining biodiversity and ecological balance. The high-precision detection of large-target forest fires can provide firefighters with important information such as the location of the fire, the spread of the fire, and the environment of the fire scene, which can help firefighters to quickly understand the fire situation and make scientific decisions, providing strong support for firefighters' work and guaranteeing their safety and work efficiency.

3. Authors have not mentioned properly about the labeling strategies. How the area is identified as small or large? What could be the threshold level of the fire to identify in that ?

Reply to Question 3: This is a good question and we have made necessary changes.

According to the definition of the international organization SPIE, the small target is the target area less than 80 pixels in the 256×256 image, that is, less than 0.12% of 256×256 is a small target, this is the definition of the relative size. Another is the definition of absolute size, according to the definition of COCO dataset , the size of the target less than 32 * 32 pixels can be considered as a small target. Targets larger than 96*96 pixels are considered as large targets. In this paper, the absolute size definition is used to categorize the large and small targets.

In addition, the threshold of fire is defined according to the confidence of the model output (its value ranges from 0-1), the model in this paper adopts a fixed threshold of 0.6, and when the confidence score exceeds 0.6, it is determined as a forest fire target.

4. The report also lacks acronyms for important topics like WSB and WSS.

Reply to Question 4: This is a good question and we have made necessary changes.

We have revised the references in the text. For each abbreviation used for the first time in the article, we have indicated its full meaning.

5. What is the actual size of the target? How the authors determine it? As for as my view, the size of the target might vary. If that is the case how does the authors fix it over in the model? It is not clearly mentioned. In line 440, the authors have mentioned , but still the clarity is missing.

Reply to Question 5: This is a good question and we have made necessary changes.

In our forest fire detection model, the actual size of the target may vary depending on the particular application and dataset. As you correctly mentioned, the size of the target may vary in real-world scenarios. In our model, we use a flexible approach to handle different target sizes. To determine the target size, we use anchor boxes or default bounding boxes during training. These anchor boxes have predefined aspect ratios and scales that cover a range of possible target sizes. During training, the model learns to predict the offset and scaling of these anchor boxes to accurately localize targets of different sizes. In this paper, the absolute scale definition is used to categorize size targets. According to the COCO dataset definition, a target with size less than 32*32 pixels can be considered as a small target. Targets larger than 96*96 pixels are considered as large targets.

6. The authors have used attention mechanisms which might increase the computational complexity. How they will overcome?

Reply to Question 6: This is a good question and we have made necessary changes.

We used two attention mechanisms in our study, the SE attention mechanism and the SimAM attention mechanism. In this study, the increased computational complexity with the addition of the attention mechanism is not much, but it can effectively improve the precision and accuracy of the model. However, in order to balance the speed and accuracy, we used different methods in the two models to improve the efficiency of model detection. For example, BiFPN is used, which is more resource-efficient than the base FPN. We will follow up with further research on model lightweighting as well.

7. It would be still better if the authors would have deployed the models on other datasets

Reply to Question 7: This is a good question and we have made necessary changes.

We experimented the model we studied in the dataset of forestry pests and diseases as well. The following is the recognition result graph, which has good recognition effect on both large and small targets of pests and diseases. As shown in the figure for the recognition of tea tree pests, we can see from the figure that the model built in this paper can still accurately recognize the small targets of pests and diseases.

8. Comparison with the earlier works are missing.

Reply to Question 8: This is a good question and we have made necessary changes.

The comparison experiments between the model in this paper and the earlier mainstream target detection models are added, and the results of the experiments are shown in the following table. We compare the WSB model, WSS model, and WSB-WSS model developed in this paper with the existing mainstream target detection models. Although the accuracy of the integrated WSB-WSS model is slightly improved compared to the WSB model and the WSS model, the results of the comparison experiments show that the accuracy of the WSB model and the WSS model built in this paper has been greatly improved compared to the existing mainstream models. And the WSB-WSS model improves the accuracy rate again on this basis, and the detection effect is obviously better than the existing mainstream model.

9. The authors did not write much about the small and large target object. In fig 13(d) the large trunk fire is not detected as of my viewpoint. But it has been written that the model detects. Could you provide more insights?

Reply to Question 9: This is a good question and we have made necessary changes.

Fig 13(d) shows the detection results of the WSB model, through the picture we can observe that the WSB model is detecting the occurrence of fires (the forest fire confidence level reaches 0.87, but the detection frame does not have a comprehensive coverage of target fires. In this paper, we use this picture to show the deficiency of WSB in recognizing large target forest fires.

10. How BiFPN attention varies form CBAM attention ? Regarding the FPN, obviously it consumes more time . How the authors will resolve this?

Reply to Question 10: This is a good question and we have made necessary changes.

BiFPN is a feature pyramid network rather than a traditional attention mechanism; CBAM attention is an attention module for feedforward convolutional neural networks.

BiFPN is a combined top-down and low-up feature pyramid network that can suppress or enhance feature representation for different detection features according to cross-scale weights.The goal of BiFPN is to provide multi-scale feature pyramids to better capture targets of different sizes and resolutions.

The CBAM attention mechanism consists of two sub-modules, a Channel Attention Module for enhancing features in the channel dimension and a Spatial Attention Module for enhancing features in the spatial dimension.The main goal of CBAM attention is to improve the convolutional block within the feature representation in order to better capture the feature relationships in the spatial and channel dimensions.

The BiFPN structure is based on FPN.BiFPN removes nodes with no feature fusion and small contribution and adds new channels between the original input and output nodes. The model in this paper uses BiFPN, which saves the consumption of resources and fuses more feature information than FPN.

11. A dynamic mechanism is being followed in adjusting the gradients. How the harmful or low quality gradient is identified by the model? What could be the threshold value?

Reply to Question 11: This is a good question and we have made necessary changes.

The outlier degree of the anchor box is characterized by the ratio of  to :

A small outlier degree means that the anchor box is highquality.

Because low-quality gradients are usually generated by low-quality anchor frames, instead of setting a threshold to determine what is a harmful gradient, we assign different gradient gains for anchor frames of different qualities, and smaller gradient gains for anchor frames with larger outliers. This effectively avoids low quality examples from having a large impact on the gradient.

Author Response File: Author Response.docx