Plant Disease Diagnosis in the Visible Spectrum

Round 1

Reviewer 1 Report

Dear Authors  

The manuscript is dedicated to the investigation of the automatical identification and diagnosis of plants diseases by images and neural network application. Generally, the manuscript is interesting and written accurately enough. But there are some mistakes, so I recommend to verify and correct the manuscript before the next submission. The below recommendations can help to identify weak places which can be improved.  

1. It is recommended to carefully verify the formatting according to the journal template file, because there are a lot of small mistakes, for example:

• unnecessary points at the end of the title, the author's list, paragraphs' titles in lines 107, 160, 173, 243, 244
• the title should be written as own names, i.e. each word with a capital letter,
• the affiliation part doesn't have the e-mails; the unnecessary sign ":" is in the first affiliation
• missed backspace in keywords (...agriculture;Gaussian...), some of the words are written from the capital, which in some cases is not necessary,
• figures mentioned in the text have to be written from the capital letter (lines 97, 213, 259),
• missed comma in Fig. 2 (...GMM 3) Segmentation...),
• The formulas do not have numeration. Punctuate equations as regular text (as a part o sentence, see the journal template file); i.e the comma at the end of the formula, "Where:" from the small letter, commas of the list of formula parts explanation etc. As an example, see the formula in line 221, but the commas have to be added at the end of the sentence (...as follows.) and in the list.
• missed idents on the beginning of some in some lines (222, 226 etc.)

2. Lines 23-31: sign ":" could be unnecessary in these sentences - for verification.

3. The manuscript mentioned an average diagnosis time equals 22 seconds. Аor the first view, it could seem that the time is not high, but let's imagine that the automatic system is moving in the field with some speed, which can be around 10-20 km/h, and gathering the data for the neural network. In such a case, this time will not allow getting instantaneous identification and also remember the location of the plant. So it is difficult to agree with the statement in line 87 that it is a fast method.

Described above and significantly long time are a result of a low number of probes (images), which is equal to 23 according to line 104. Normally, the number of images should be taken a few hundred; it is a minimum recommended value. If the neural network is really trained, it performs the task and identification very quickly; the time is measured in milliseconds. All these allow to suppose that the neural network was not sufficiently trained because of the luck of the data. So, I recommend to implement a higher number of images and correct the manuscript.

 Also, the paper presents a very good review of different investigations but does not compare what the time was reached in other researches mentioned in the reference list. I suggest comparing this parameter and add a short description in the manuscript.

4. Line 46: the abbreviation NN is clear for understanding, but it does not state in the text as (CNN), (GMM) etc. Generally, (NN) is stated but in line 244, which is too late - for verification. 

5. Figure 3: some elements in the figure are not visible enough ("ho")

6. Figures 5 and 6: the numbers are too small, so I suggest to increase them.

7. Lines 261-275 represent a single sentence with formula description - for correction as was mentioned in p. 1 for formulas. In addition, as a suggestion, maybe it will be better to place Fig. 7 after line 283 to avoid division of formula and its description.

8. Figures 8-12: missed commas - for verification.

9. Author Contribution: according to the journal template, the initials have to be used in this section. I recommend to correct according to the journal requirements.

10. Data Availability Statement: currently, the link does not work correctly (error 404).

11. Reference: correct references 1 and 2 (authors list). Close mistakes with the author list could be found in all references. Also, it is recommended to verify small mistakes in all the references (for example, in reference 4 the link goes outside of the page; in reference 7, the link is given as "7http")

Sincerely,

Reviewer 

Author Response

Reviewer#1 Comment

The manuscript is dedicated to the investigation of the automatical identification and diagnosis of plants diseases by images and neural network application. Generally, the manuscript is interesting and written accurately enough. But there are some mistakes, so I recommend to verify and correct the manuscript before the next submission. The below recommendations can help to identify weak places which can be improved.

Author response: Thanks for your commentaries. The response to your comments is given below: 

Reviewer#1, Comment 1.-

It is recommended to carefully verify the formatting according to the journal template file, because there are a lot of small mistakes, for example:

• unnecessary points at the end of the title, the author's list, paragraphs' titles in lines 107, 160, 173, 243, 244 
• the title should be written as own names, i.e. each word with a capital letter, 
• the affiliation part doesn't have the e-mails; the unnecessary sign ":" is in the first affiliation 
• missed backspace in keywords (...agriculture;Gaussian...), some of the words are written from the capital, which in some cases is not necessary, 
• figures mentioned in the text have to be written from the capital letter (lines 97, 213, 259), 
• missed comma in Fig. 2 (...GMM 3) Segmentation...),
• The formulas do not have numeration. Punctuate equations as regular text (as a part o sentence, see the journal template file); i.e the comma at the end of the formula, "Where:" from the small letter, commas of the list of formula parts explanation etc. As an example, see the formula in line 221, but the commas have to be added at the end of the sentence (...as follows.) and in the list. 
• missed idents on the beginning of some in some lines (222, 226 etc.) 

Author action:  All errors have been corrected. 

  Reviewer#1, Comment 2.-

Lines 23-31: sign ":" could be unnecessary in these sentences - for verification.

 Author action:  The error has been corrected.

Reviewer#1, Comment 3.-

The manuscript mentioned an average diagnosis time equals 22 seconds. Аor the first view, it could seem that the time is not high, but let's imagine that the automatic system is moving in the field with some speed, which can be around 10-20 km/h, and gathering the data for the neural network. In such a case, this time will not allow getting instantaneous identification and also remember the location of the plant. So, it is difficult to agree with the statement in line 87 that it is a fast method.

Described above and significantly long time are a result of a low number of probes (images), which is equal to 23 according to line 104. Normally, the number of images should be taken a few hundred; it is a minimum recommended value. If the neural network is really trained, it performs the task and identification very quickly; the time is measured in milliseconds. All these allow to suppose that the neural network was not sufficiently trained because of the luck of the data. So, I recommend to implement a higher number of images and correct the manuscript.

 Also, the paper presents a very good review of different investigations but does not compare what the time was reached in other researches mentioned in the reference list. I suggest comparing this parameter and add a short description in the manuscript.

Author response:

We understand your concern, there is some confusion regarding the article, it is a combination of image processing methods that uses a simple neural network to perform plant diagnostics.

Therefore, the estimated time of 22 seconds is the combination of all the methods themselves, reaching that average in the best of cases with a time of ###. and at worst it could give up to ###. We also understand that if a single convolutional network would be used that learns the variations of images, but there is no data set that meets all these characteristics that we want. For this reason, a new data set with noisy images and the help of manual segmentation and survey evaluation is proposed to validate this data set and compare it with our results.

Regarding the times, as suggested the times in the tables are now presented in milliseconds, it is important to clarify that the NN was trained on HSV values of white, black, green, yellow, and brown, so the 23 images of plants do not represent the training set, because the NN is used exclusively for the identification of colors and the diagnosis is performed with a combination of methods.

The proposed methodology is the combination of the use of a neural network and an adequate segmentation which allows good results with small databases, but with high variability, in particular this database has images with controlled and uncontrolled background and considers different types of individual plants. This methodology is designed for a mobile phone application.

A completely new subsection has been created to compare our results with those of other authors, where 4954 new images were analyzed in order to compare the performance on that data set to a previous work that we consider a benchmark. 

Author action: 

 the times in the tables are now presented in milliseconds.

A completely new subsection has been created to compare our results with those of other authors, where 4954 new images were analyzed in order to compare the performance on that data set to a previous work that we consider a benchmark. 

Reviewer#1, Comment 4.-

Line 46: the abbreviation NN is clear for understanding, but it does not state in the text as (CNN), (GMM) etc. Generally, (NN) is stated but in line 244, which is too late - for verification.

Author response: Thanks for your commentary. An earlier presentation of the NN abbreviation was put in the introduction.

Author action:  add in line 40 the abbreviation in section introduction. 

Reviewer#1, Comment 5.-                        

Figure 3: some elements in the figure are not visible enough ("ho").  

Figures 5 and 6: the numbers are too small, so I suggest to increase them.

Author action: The image was corrected.

In figures 5 and 6, the size of the numbers is increased.

Reviewer#1, Comment 7.-

Lines 261-275 represent a single sentence with formula description - for correction as was mentioned in p. 1 for formulas. In addition, as a suggestion, maybe it will be better to place Fig. 7 after line 283 to avoid division of formula and its description.

Author action: The error has been corrected.

Reviewer#1, Comment 8.-

Figures 8-12: missed commas - for verification.

Author action: The error has been corrected.

Reviewer#1, Comment 9.-

 Author Contribution: according to the journal template, the initials have to be used in this section. I recommend to correct according to the journal requirements.

Author action: Done.

Reviewer#1, Comment 10.-

Data Availability Statement: currently, the link does not work correctly (error 404). It can be accessed.

Author action: now the GitHub is public and you can see the page..

Reviewer#1, Comment 11.-

Reference: correct references 1 and 2 (authors list). Close mistakes with the author list could be found in all references. Also, it is recommended to verify small mistakes in all the references (for example, in reference 4 the link goes outside of the page; in reference 7, the link is given as "7http")

Author response: Those suggestions are now implemented.

Author action: we create a BibTeX   for the references and recheck all the references.

Author Response File: Author Response.pdf

Reviewer 2 Report

I thank the authors for their effort in carrying out the work. The topic discussed is very interesting. However, in my opinion, the format of the manuscript is inadequate. The work describes in detail a methodology for the diagnosis of diseases and presents some results without establishing a work plan. The results are not compared with those of other authors. Thus it is difficult to establish the characteristics of the presented method and draw some conclusions. In my opinion, it would be necessary to establish an objective of the work, a work plan, a discussion of the results and some conclusions based on said results.

Author Response

Reviewer#2 Comment

I thank the authors for their effort in carrying out the work. The topic discussed is very interesting. However, in my opinion, the format of the manuscript is inadequate. The work describes in detail a methodology for the diagnosis of diseases and presents some results without establishing a work plan. The results are not compared with those of other authors. Thus it is difficult to establish the characteristics of the presented method and draw some conclusions. In my opinion, it would be necessary to establish an objective of the work, a work plan, a discussion of the results and some conclusions based on said results.

Author response: Thanks for your commentaries.

we changed the introduction was modified to give a proposed organization of work, as well as to better understand the problem.

As suggested a set of 4954 images were analyzed in order to compare it to a previous work cited in this proposal, we consider that work a benchmark and with that comparison, adequate conclusion may be drawn by the reader. A complete new subsection has been created to compare our results with those of other authors.

and we mention it in the conclusions with respect to the results that were obtained.

Author action:

We add the organization of the paper.

Add a new subsection has been created to compare our results.

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear authors,

I find your manuscript interesting, but I think it will need a lot of improvement to be published. Here are my specific comments:

1) The used dataset is really small. Since you have gathered the data online, it seems it would be easy to gather 200 more samples. You should get to at least a hundred.

2) When you have done the previous. Use a proper testing-validation split. You state, for example on lines 165-166 that "0.16 was selected because it is optimal for the used data." You should use one part of the data to select parameters and another part to test the model.

3) You fail to justify the use of the neural network in colour coding. First of all, chapters 2.3.1 and 2.4.1 should be combined to be able to convey the reasoning for the use of the neural network. You state, on lines 252-253, that the use of neural network helps in colour coding in non-ideal situations. I would like to have a more detailed explanation of this, preferably by an additional experiment in the paper or a citation of such an experiment. Also, explain the training process of the used neural network. If you used the same images as in the results for training, you should refer to my point 2.

4) In the Results section, the usage of survey for the general public comes out of nowhere. Explain it in the methods. I do not fully understand the selection%, ranking, dimension or processing time metrics either. I suspect some of these relate to the survey and some to your image processing protocol, but explain them in more detail. For example, the ranking 1-4 is a complete mystery to me based on your explanation. Also, you claim that you do not have access to specialists. Why? I think many researchers on the field would evaluate the 20-200 photos you send them and write a chapter or two in discussion for a name in the paper. It should not take more than a day.

5) Since your dataset was so small, you should have appended all segmentations and result images as an appendix. If you increase the dataset size significantly as I suggested, you should show the readers some of the best segmentations, some of the worst, and some average ones. In Figures 8 - 11, you have not disclosed what numbers they refer to in Table 1.

6) You should have a discussion chapter where you discuss the results in light of previous research, and the strengths and weaknesses of this manuscript compared to them.

7) You capitalize Figure, Table, etc, and internet is written with a small 'i'.

I feel this is too much work for Major revision, so I am going to suggest rejection with encouragement for resubmitting.

With the improvement suggestions done, I have a feeling that this could be a good article

Best regards

Author Response

Reviewer#3 Comment

I find your manuscript interesting, but I think it will need a lot of improvement to be published. Here are my specific comments:

Author response: Thanks for your commentaries.   The response to your comments is given below: 

Reviewer#3, Comment 1.-

The used dataset is really small. Since you have gathered the data online, it seems it would be easy to gather 200 more samples. You should get to at least a hundred.

Author response:

As suggested, we increase the data set, 4954 extra images from the PlantVillage data set were analyzed.

Author action:

Add a new subsection has been created to compare our results.

Reviewer#3, Comment 2.-

When you have done the previous. Use a proper testing-validation split. You state, for example on lines 165-166 that "0.16 was selected because it is optimal for the used data." You should use one part of the data to select parameters and another part to test the model.

Author response:

The value 0.16 is a threshold to segment the background. that gives very good results. this is 0.016, but we wrote 0.16 by mistake, which is now corrected. As suggested,

this threshold of 0.016 calculated from the original data set is also used to determine the method of segmentation for the 4954 extra images, in order to test that threshold. This is an image processing criterion and it was optimized, it has nothing to do with training and the purpose of the work is to give a diagnosis based on image processing.

Author action:

Add a new subsection has been created to compare our results.

This paragraph has been rewritten for better understanding. In line 166.

Reviewer#3, Comment 3.-

You fail to justify the use of the neural network in colour coding. First of all, chapters 2.3.1 and 2.4.1 should be combined to be able to convey the reasoning for the use of the neural network. You state, on lines 252-253, that the use of neural network helps in colour coding in non-ideal situations. I would like to have a more detailed explanation of this, preferably by an additional experiment in the paper or a citation of such an experiment. Also, explain the training process of the used neural network. If you used the same images as in the results for training, you should refer to my point 2.

Author response:

We used a NN for color identification, after the image is segmented in order to calculate the portion of the plant that has a disease associated with that shade of color, also in the segmentation strategy with saliency the NN is used to codify those pixels which may be part of the plant based on the assumptions described in section 2.3.1.

Regarding the color coding in non-ideal situations, as of this moment it is considered future work, but in order to clarify and provide further explanation, a citation with a specific page is provided. 

Author action:

Reviewer#3, Comment 4.-

In the Results section, the usage of survey for the general public comes out of nowhere. Explain it in the methods. I do not fully understand the selection%, ranking, dimension or processing time metrics either. I suspect some of these relate to the survey and some to your image processing protocol, but explain them in more detail. For example, the ranking 1-4 is a complete mystery to me based on your explanation. Also, you claim that you do not have access to specialists. Why? I think many researchers on the field would evaluate the 20-200 photos you send them and write a chapter or two in discussion for a name in the paper. It should not take more than a day.

Author response:

The metrics mentioned are not part of the survey, but generated with the results of the survey. The survey is composed of 23 questions with 4 answers per question, only one answer belongs to the diagnosis generated by our proposal. Regarding the ranking, rank 1 means that the option generated by our method was the most selected diagnosis out of the 4 possible choices by the people who took the survey. On the other hand, rank 4 means that the option generated by our method was the least selected diagnosis out of the 4 possible choices by the people who took the survey. 

Author action:

 The definition of  the concept is  extended to give clarity

 in Line 299.

Reviewer#3, Comment 5.-

Since your dataset was so small, you should have appended all segmentations and result images as an appendix. If you increase the dataset size significantly as I suggested, you should show the readers some of the best segmentations, some of the worst, and some average ones. In Figures 8 - 11, you have not disclosed what numbers they refer to in Table 1.

Author response:

The appendix for the original data set of 23 images is available, and we tested the methods in an extra data base of 4954 in order to compare the performance with an existing benchmark work specifically trained on that related data set. A complete new subsection has been created to compare our results with those of other authors. 

Author action:

Images have been added to GitHub and are commented on in the article.

Reviewer#3, Comment 6.-

 You should have a discussion chapter where you discuss the results in light of previous research, and the strengths and weaknesses of this manuscript compared to them.

Author response:

As we have added an extra performance evaluation a comparison with a benchmark work is now available and also a discussion of those results. A complete new subsection has been created to compare our results with those of other authors.

Author action:

Add a new subsection has been created to compare our results.

Reviewer#3, Comment 7.-

 You capitalize Figure, Table, etc, and the internet is written with a small 'i'.

Author action:

Those changes are now implemented.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Dear authors
 
The manuscript is significantly improved, and the results are much better. 
It will be interesting to see the application and test. 
 
Sincerely,
reviewer

Author Response

Reviewer#1Comment

The manuscript is significantly improved, and the results are much better. 
It will be interesting to see the application and test. 

Author response:                        

 We appreciate your valuable comments, we hope in the near future we can create an application for cell phones.

We present a new version of the work in view of the comments and suggestions from the others reviewers.

Author Response File: Author Response.pdf

Reviewer 2 Report

In my opinion, the section 3.0.1 should be removed and its content included in discussion, although the final part would be better in materials and methods. It is necessary to indicate the references of the cited works. If possible, it would be convenient for the authors to indicate objectives and a work plan.

Author Response

Reviewer#2Comment

In my opinion, the section 3.0.1 should be removed and its content included in discussion, although the final part would be better in materials and methods. It is necessary to indicate the references of the cited works. If possible, it would be convenient for the authors to indicate objectives and a work plan.

Author response:                        

 We appreciate your valuable comments,  suggested changes were made.

Author action:  

We present a new version of the work in view of the comments and suggestions from the others reviewers.

Section 3.0.1 was removed and its content included in the discussion,  the final part was placed in materials and methods. A new section was created with objectives and a work plan.

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear authors,

I find you did your changes hastily. The manuscript did not improve significantly, and the methodological improvement is marginal. Take a step back from, and try to make it a coherent package, where everything relevant is documented clearly in order to be evaluated and possibly replicated. I have some specific examples. You must understand that these are by no means my only concerns regarding the article:

1) The citation [39] for neural network use in colour classification is not a peer reviewed study, but rather someones masters project.

2) I think you are shooting "a fly with a cannon" when you use neural network in colour classification. It as a problem should not require use of neural network. What is your motive here?

3) The survey is a method, not a result. Describe it in detail in methods section. I feel I would not understand the survey without seeing the actual survey as an appendix.

4) In the beginning of the results you still talk of a data set of 23 images, even though it was supposed to be raised to over 4000 images. Clarify.

I still believe this manusript could be published provided that you take the time to do the necessary improvements.

Best regards,

Author Response

Reviewer#3 Comment

Dear authors,

I find you did your changes hastily. The manuscript did not improve significantly, and the methodological improvement is marginal. Take a step back from, and try to make it a coherent package, where everything relevant is documented clearly in order to be evaluated and possibly replicated. I have some specific examples. You must understand that these are by no means my only concerns regarding the article:

1) The citation [39] for neural network use in colour classification is not a peer reviewed study, but rather someones masters project.

Author response:                        

We present a new version of the work, in view of the comments on the use of the neural network, it was decided to use another color classification method. We chose to use HSV thresholds because of their simplicity and robustness for the identification of the considered colors.

Author action:  

The citation [39] was removed.

We use HSV thresholds instead of the   NN. 

2) I think you are shooting "a fly with a cannon" when you use neural network in colour classification. It as a problem should not require use of neural network. What is your motive here?

Author response:  

In view of the comments on the use of the neural network, it was decided to use another color classification method. We chose to use HSV thresholds because of their simplicity and robustness for the identification of the considered colors.

Author action:  

We use HSV thresholds instead of the NN. 

3) The survey is a method, not a result. Describe it in detail in methods section. I feel I would not understand the survey without seeing the actual survey as an appendix.

Author response:  

Given the feedback regarding the survey and the performance of the proposal when compared to the benchmark, we decided to remove the survey and focus on highlighting the results of the plant village set; the results of the special case of 23 images with challenging backgrounds remain available as an annex.

Author action:  

Remove the survey and present the results as supplementary material.

4) In the beginning of the results you still talk of a data set of 23 images, even though it was supposed to be raised to over 4000 images. Clarify.

Author response:  

It was clarified in the text that the data set is 4977 images.

Author action:  

It was clarified in the text that the data set is 4977 images.

Author Response File: Author Response.pdf

Round 3

Reviewer 3 Report

Dear authors,

the improvement is significant, I have no further comments.

best regards