Deep Learning Beehive Monitoring System for Early Detection of the Varroa Mite

Round 1

Reviewer 1 Report

Manuscript ID: signals-1749339

Title:  Proposed beehive detection system supported by a deep learning algorithm for the early detection of the Varroa mite

Authors: Voudiotis et al.

General comments: The article proposes a novel deep learning algorithm for early detection of Varroa mite infection in bee hives. The approach has advantages, however these should be discussed vis-à-vis other approaches being reported or in use. The structure of manuscript may need some changes if journal style guidelines permit. For example, The introduction should contain the overview of other published work in the similar domain, while methodology section should include details about the hardware setup and configuration, software-algorithms and details of calculations and statistical analysis used. The outcome should be mentioned in the results followed by discussion and conclusions. Several concerns are indicated in the PDF file of the manuscript, while some important section-specific comments are detailed below.

Section specific comments

Title:            The title of the manuscript may be may be modified to make it more focussed and to avoid repetitive terms, e.g., ‘detection

Suggestion: Deep learning algorithm-based beehive monitoring system for early detection of the Varroa mite

Abstract:  Abstract looks fine, however there is scope for improvement, particularly the efficiency of detection detailed in the results section is not getting highlighted.  Use of abbreviations may be minimized in the abstract.

Keywords:    The number of keywords and style (alphabetical or not) should be as per the journal format. Use of abbreviation as keywords may be avoided, if essential use the expanded form of the keyword with abbreviation in parenthesis.

Introduction: Introduction section is appropriate w.r.t. to length, information content and background. The abbreviated terms should be avoided at first use, instead the expanded from with abbreviation in the parenthesis should be used.  At certain places, minor changes have been indicated (see PDF file for details), of which few are listed below:

Line 28:          ‘anthectic’ may be corrected to ‘aesthetic’

Line 28:       ‘apis mellifera’ should be corrected to scientific notation ‘Apis mellifera’

Section 2:    This section details some of the methods/approaches used for management of Varroa mite including chemical treatment as well as novel technological approaches involving computational and image processing based methods. The details of chemical management of the disease may be somewhat minimized. While the modern methods/approaches, the focus of this article may be highlighted a little bit more in this section. Several recent reports, which might be appropriate for this article are missing. For example, Chazette et al., doi: 10.1109/SSCI.2016.7850001, 2016, Mrozek et al.,  Appl. Sci. 2021; Bilik et al., Sensors, 2021; Sevin et al., Turk. J. Vet. Anim. Sci. 2021. Authors may update this section by citing few more and relevant publications.

Line 60:       ‘Varroa mite alevienation’ or ‘Varroa mite alienation’, kindly check?

Section 3:     This section describes the Varroa mite detection system, which is the focus of this article. However, this lacks information on similar previous reports, and how the proposed system addresses the concerns (if any) of previous approaches. This may be mentioned in a brief manner, and a detailed comparative analysis can be part of discussion section.

                     In lines 131, 137, 148 (check other places also), ‘Figure 1’ is mentioned as ‘Figure ??’, which should be corrected.

Lines 144-148:   Correct designation for ‘GHz’ and ‘MHz’ may be used.

Line 164:       Check if use of ‘offline’ is correct, or ‘online’ is appropriate here.

Section 4:     Section 4 is fine, however few things may be improved. Table 1 contain certain abbreviations/designations, which may be expanded in the text or added as foot notes

Section 5:    Section 5 deals with actual experimental setup and analysis. There are several formulas with terms that are not defined appropriately. If feasible Table 2, 3, and 4 may be organized as a easy comparison of parameters across the approaches. If possible it should integrate the advantages, disadvantages (if any) and efficiency of proposed Varroa mite detection approach compared to other standard methods in use or in literature.

Conclusions: Overall this section is fine, and may need minor changes (indicated in the PDF file)  

Figures: The figure legends/captions should have been a little bit made more informative.  

Figure 1: The quality of Figure 1 (showing the device setup) may be enhanced.

Figure 2 and Figure 3 can be part of a composite image (two panels), where the different steps can be shown with actual real image data to make it more informative for the users

Tables: The Tables are good for presenting the comparative picture of different approaches used in the study.

Comments for author File: Comments.pdf

Author Response

Thank you for reviewing our manuscript. Please find our amendments to your comments below.

Comment 1: The title of the manuscript may be may be modified to make it more focused and to avoid repetitive terms, e.g., 'detection

Suggestion: Deep learning algorithm-based beehive monitoring system for early detection of the Varroa mite

Response 1: The manuscript title has been modified according to the reviewer's suggestion. Thank you.

Comment 2: Abstract looks fine, however, there is scope for improvement, particularly the efficiency of detection detailed in the results section is not getting highlighted.  The use of abbreviations may be minimized in the abstract.

Response 2: Added appropriate sentence highlighting the total efficiency of the detection process presented in terms of detection accuracy of both bees and (step 1) then of bees having the varroa mite attached (step 2). Abbreviations have been removed from the abstract.

Comment 3: The number of keywords and style (alphabetical or not) should be as per the journal format. Use of abbreviation as keywords may be avoided, if essential use the expanded form of the keyword with abbreviation in parenthesis.

Response 3: The keywords have been added using the journal format (see https://www.mdpi.com/2624-6120/3/2/21/htm), and keywords abbreviations have been put in parenthesis.

Comment 4: Introduction: Introduction section is appropriate w.r.t. to length, information content and background. The abbreviated terms should be avoided at first use, instead the expanded from with abbreviation in the parenthesis should be used.  At certain places, minor changes have been indicated (see PDF file for details), of which few are listed below:

Line 28:          'anthectic' may be corrected to 'aesthetic'

Line 28:       'apis mellifera' should be corrected to scientific notation 'Apis mellifera'

Response 4: The errors in this section have been corrected. In addition, the abbreviated terms first used have been written using their expanded form.

Comment 5: Section 2:    This section details some of the methods/approaches used for management of Varroa mite including chemical treatment as well as novel technological approaches involving computational and image processing based methods. The details of chemical management of the disease may be somewhat minimized. While the modern methods/approaches, the focus of this article may be highlighted a little bit more in this section. Several recent reports, which might be appropriate for this article are missing. For example, Chazette et al., doi: 10.1109/SSCI.2016.7850001, 2016, Mrozek et al.,  Appl. Sci. 2021; Bilik et al., Sensors, 2021; Sevin et al., Turk. J. Vet. Anim. Sci. 2021. Authors may update this section by citing few more and relevant publications.

Line 60:       'Varroa mite alevienation' or 'Varroa mite alienation', kindly check?

Response 5: Following your comment, we have reduced our reference to the chemical methods of the varroa alienation, focused more on the technological ones, and added the references you suggested.

In line 60, we meant alienation, and we have corrected it.

Comment 6: Section 3:     This section describes the Varroa mite detection system, which is the focus of this article. However, this lacks information on similar previous reports, and how the proposed system addresses the concerns (if any) of previous approaches. This may be mentioned in a brief manner, and a detailed comparative analysis can be part of discussion section.

In lines 131, 137, 148 (check other places also), 'Figure 1' is mentioned as 'Figure ??', which should be corrected.

Lines 144-148:   Correct designation for 'GHz' and 'MHz' may be used.

Line 164:       Check if use of 'offline' is correct, or 'online' is appropriate here.

Response 6: The mentioned errors in this section have been corrected. In line 164, offline use was incorrect and has been fixed.

Comment 7:  Section 4 is fine, however few things may be improved. Table 1 contain certain abbreviations/designations, which may be expanded in the text or added as foot notes

Response 7:  The abbreviations used in Table 1 have been expanded and removed from the abbreviations Table at the end of the paper. Abbreviations have also been expanded in their manuscript's first appearance.

Comment 8: Section 5:    Section 5 deals with actual experimental setup and analysis. There are several formulas with terms that are not defined appropriately. If feasible Table 2, 3, and 4 may be organized as a easy comparison of parameters across the approaches. If possible it should integrate the advantages, disadvantages (if any) and efficiency of proposed Varroa mite detection approach compared to other standard methods in use or in literature.

Response 8: A separate subsection has been added 5.4

Comment 9: Conclusions: Overall this section is fine, and may need minor changes (indicated in the PDF file)  

Response 9: The Conclusions section has been thoroughly checked for typo errors and minor mistakes.

Comment 10: Figures: The figure legends/captions should have been a little bit made more informative.  

Figure 1: The quality of Figure 1 (showing the device setup) may be enhanced.

Figure 2 and Figure 3 can be part of a composite image (two panels), where the different steps can be shown with actual real image data to make it more informative for the users

Tables: The Tables are good for presenting the comparative picture of different approaches used in the study.

Response 10:  The figure captions have been corrected. The quality of Figure 1 has been enhanced to 300dpi. Figures 2 and 3 remained separate since it will be hard to add side by side and maintain the images' clearness.

Reviewer 2 Report

The paper proposes a beehive detection system using a deep learning algorithm. In terms of application, the paper has great potential; but it requires some issues to be fixed before acceptance. 

1. Quantitative results are required in the abstract.

2. The advantage of the proposed method is required to explain in the abstract.

3. The organization of the paper should be improved: Introduction, Related works, proposed method, Results and discussion, Conclusion. Please organize them accordingly.

4. The background and application of other pre-trained deep learning models are required. For this, we could explain the background of deep learning models, followed by pre-trained deep learning models and their applications.After that, we can focus on lightweight models such as mobile net models For the deep learning models and applications, you may explain the following papers:

https://www.hindawi.com/journals/cin/2021/2158184/ 

After the explanation of the deep learning background, you may explain the pre-trained mobile net deep learning models:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0264586  

Based on such discussion, you may explain your proposed work. This makes your content organization smooth and coherent. This also helps understand why mobile net-like architecture is preferred.  

5. Are there any state-of-the-art methods for the comparison? It is always good to compare with them if there are any methods in the literature.  

6. It is also good to include other performance metrics such as precision, recall, and f1-score.

Author Response

Thank you for reviewing our manuscript. Please find our amendments to your comments below.

Comment 1:  Quantitative results are required in the abstract.

Response 1: Quantitative results are mainly part of the conclusions sections. An appropriate sentence indicating the total accuracy of the proposed CNN detection algorithm has been added to the Abstract.

Comment 2: The advantage of the proposed method is required to explain in the abstract.

Response 2: The proposition advantages:  The end-node device camera module is placed inside the brood box. It is equipped with offline detection in remote areas of limited network coverage or online imagery data transmission and mite detection over the cloud. Using a smart algorithm that tries to identify bees inside the brood frames carrying the mite in real-time. Also, the Varroa detection accuracy of our proposition has been mentioned.  

Comment 3: The organization of the paper should be improved: Introduction, Related work, proposed method, Results and discussion, Conclusion. Please organize them accordingly.

Response 3: The organization of the paper has been improved according to the reviewer's suggestions. Results and discussion are presented separately to each experimental scenario subsections, overall system evaluation, and cross-comparison with existing detection systems mentioned in the literature (5.1-5.4). 

Comment 4: The background and application of other pre-trained deep learning models are required. For this, we could explain the background of deep learning models, followed by pre-trained deep learning models and their applications. After that, we can focus on lightweight models such as mobile net models for the deep learning models and applications, you may explain the following papers:

https://www.hindawi.com/journals/cin/2021/2158184/ 

After the explanation of the deep learning background, you may explain the pre-trained mobile net deep learning models:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0264586

Based on such discussion, you may explain your proposed work. This makes your content organization smooth and coherent. This also helps understand why mobile net-like architecture is preferred.  

 Response 4: Two additional paragraphs (1,2) were added at the beginning of Section 4, mentioning the background and application of deep learning models with additional references. Additionally, the Related work has been expanded, focusing specifically on CNN-based Varroa mite detection systems (paragraphs 6,7,8).

Comment 5: Are there any state-of-the-art methods for the comparison? It is always good to compare with them if there are any methods in the literature.  

Response 5: A separate subsection has been added 5.4. Also, existing state-of-the-art methods compared are mentioned in the Introduction section 2.2 , paragraphs 6,7,8 (camera-based detection implementations in the literature), apart from other implementations that use sensors and sound, mentioned.

Comment 6: It is also good to include other performance metrics such as precision, recall, and f1-score.

 Response 6: A paragraph has been added to section 5.3 explaining why Recall and F1-Score have not been used.

Reviewer 3 Report

The paper proposed by Voudiotis and colleagues presents an innovative solution for the detection of Varrona mite. Authors also evaluated memory demanding which represents an important but sometimes undervalued aspect. Although the paper is well written some small points have been found.

Minor Concerns

1)In the manuscript, when Authors referred a figure it appears "??", this can be due to wrong letex compiling, please provide correction to this aspect.

2) It is reasonable that the method and implementation is new but a comparison between the performances of the architectures you implemented and the available methods present in the literature is welcome. This can be also useful to better explain the limits of the study and the possible future research directions.

Author Response

Thank you for reviewing our manuscript. Please find our amendments to your comments below.

Comment 1: In the manuscript, when Authors referred a figure, it appears "??", this can be due to wrong latex compiling, please provide correction to this aspect.

Response 1: The numbering of the figures has been corrected

Comment 2: It is reasonable that the method and implementation is new but a comparison between the performances of the architectures you implemented, and the available methods present in the literature is welcome. This can be also useful to better explain the limits of the study and the possible future research directions.

 Response 2: A separate subsection has been added 5.4. Also, existing state-of-the-art methods compared are mentioned in the Introduction section 2.2 , paragraphs 6,7,8 (camera-based detection implementations in the literature), apart from other implementations that use sensors and sound, mentioned.

Round 2

Reviewer 2 Report

1. While looking at the SOTA comparison, it is seen that the proposed method is weak and higher detection time. This shows that it is not lightweight and not useful in real applications. It is not convincing.

2. The comparison with other pre-trained models such as Efficientnet and light architectures is required to show more robust performance as it failed to outperform others. 

3. The background of deep learning models based on the suggested papers in the previous

review has been discarded by the authors. The reviewer suggested explaining basic deep learning models and their applications. Please explain the suggested papers in relation to the related work. The reviewer is not convinced with the current response.

Author Response

Thank you very much for giving us the opportunity to further improve our manuscript. Please find attached our responses to your comments:

Comment 1:  While looking at the SOTA comparison, it is seen that the proposed method is weak and higher detection time. This shows that it is not lightweight and not useful in real applications. It is not convincing.

Response 1: Appropriate justification paragraph has been added to section 5, paragraph 4, Line 506

Comment 2: The comparison with other pre-trained models such as Efficientnet and light architectures is required to show more robust performance as it failed to outperform others. 

Response 2: A paragraph has been added mentioning the results of the EfficientNet model and why it has been excluded in section 5, paragraph 5, Line 512

Comment 3: The background of deep learning models based on the suggested papers in the previous

review has been discarded by the authors. The reviewer suggested explaining basic deep learning models and their applications. Please explain the suggested papers in relation to the related work. The reviewer is not convinced with the current response.

The background and application of other pre-trained deep learning models are required. For this, we could explain the background of deep learning models, followed by pre-trained deep learning models and their applications.After that, we can focus on lightweight models such as mobile net models For the deep learning models and applications, you may explain the following papers:

https://www.hindawi.com/journals/cin/2021/2158184/ 

After the explanation of the deep learning background, you may explain the pre-trained mobile net deep learning models:

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0264586  

Based on such discussion, you may explain your proposed work. This makes your content organization smooth and coherent. This also helps understand why mobile net-like architecture is preferred.  

Response 3: Additional paragraph has been added in section 4, Third paragraph, Line 274, and the https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0264586  

reference has been added, to explain the use of lightweight models.

The authors also included in section 4 the paper of Minaee, Shervin and Boykov, Yuri Y. and Porikli, Fatih and Plaza, Antonio J and Kehtarnavaz, Nasser and Terzopoulos, Demetri, with title: Image Segmentation Using Deep Learning: A Survey,

As a more suitable reference to the deep learning background and algorithms used.

Round 3

Reviewer 2 Report

Accepted. Please proofread the manuscript before publication.