Remote Sensing Image Information Quality Evaluation via Node Entropy for Efficient Classification
Round 1
Reviewer 1 Report (Previous Reviewer 1)
All my previous concerns have been solved. The current version is acceptable.
Author Response
Dear reviewer:
Thanks for your encouragement and good suggestions again.
Sincerely,
Yue Yang
Reviewer 2 Report (Previous Reviewer 2)
I thank the authors for improving the article. I had a hard time understanding the article due to English language problems. This has improved dramatically. Please find my comments and suggestions below:
- Please consider reformulating equation (3). I would write that \beta = d_q, where q is the index in the ascending sequence D. The equation for q is then q = floor( n^2 / 4 ). This makes it easier to read. I had to reread it several times to understand the equation in its current form.
- Chapter 3.1 "node selection module":
* How many node images are picked? How do you decide? Are there as many node images as classes for the labeled data? Are the nodes computed per class (will not work for unlabeled data)?
* How do you avoid nodes that are very close together? Do you remove images after they are assigned to a node? If so, this is very close to any clustering algorithm (k-means or k-nn).
- Chapter 4 "Experiments":
* In your experiments what are f and d (equation 1)? Is the embedding the ResNet-18 feature vector after training? How large is it?
* Do you use a pre-trained ResNet and then use Transfer Learning or do you start every training from scratch for 200 epochs?
- Figure 6: Why are there multiple node images per class? Are these the unlabeled or labeled node images?
- Chapter 5.2 "Reasons":
You claim that the node images are on the decision boundaries of the learning algorithm. Can you verify this experimentally? I would expect that the classification score/confidence for images on decision boundaries is very low. I'd like to see such an experiment in the article.
- Furthermore, to verify your results transparently and apply your findings to new datasets I would recommend releasing your source code!
There are still a few language issues. Please carefully proofread the article, again.
For example:
* Lines 91-93 in the introduction are hard to understand.
* Lines 150-152: I don't understand what is meant here.
* Lines 159-164 are hard to read.
Author Response
Dear reviewer:
Thank you for giving us a second chance to revise. In this revision, we mainly made the following modifications. Please see the attachment.
Sincerely,
Yue Yang
Author Response File: 
Author Response.pdf
Reviewer 3 Report (Previous Reviewer 3)
The authors have addressed all my concerns. I suggest accept this paper in its present form.
Author Response
Dear reviewer:
Thanks for your encouragement and good suggestions again.
Sincerely,
Yue Yang
Round 2
Reviewer 2 Report (Previous Reviewer 2)
I thank the authors again for improving the article.
I can follow and understand the algorithm, now.
Please find my comments and suggestions below:
- Source Code: I believe it is great that the authors are willing to release their code. Please add the link to Github to the Data Availability Statement.
- Figure 7: Can you please use the same scale bar for the left and right matrix. Then results are easier to compare and understand. Is it possible to normalize the confusion matrix per row, as you have a differing number of sample images per class?
Furthermore, can you discuss what is shown by the confusion matrix? Images with high node entropy should be classified incorrectly more often (away from the diagonal). The maximum, however, is for most classes still at the diagonal. The only exception is class 4 which is very likely misclassified as class 0. Ideally, there would be more examples like class 4.
For comparison can you also select 100 random images and show the confusion matrix?
Please modify the figure and discuss it in Section 5.2.
- There are still a few language issues. The changes introduced new ones. Please carefully proofread the article, again.
For example:
Line 180-182: Reformulate those sentences. e.g. "If the node entropy is large, the image is located close to a decision boundary".
Line 213: Reformulate to something like "We chose to start the training from zero, like most active learning methods to avoid uncertainty introduced by transfer learning <cite a work that does so>".
Line 274 - 277: This is a very long sentence, which makes it hard to understand. Consider reformulating it.
Line 283: [We] select 100 images ...
Author Response
Dear reviewer:
Thanks for your encouragement. In this revision, we mainly made the following modifications. Please see the attachment. Thank you very much for your good suggestions again.
Sincerely,
Yue Yang
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
In this manuscript, the authors propose an information quality evaluation method of remote sensing image samples based on node entropy. Combining this method with the active learning strategy can provide good guidance for the collection and annotation of remote sensing image datasets. The method of this manuscript is novel. The comparative experiment and ablation experiment can illustrate the effectiveness of the method. However, some details need to modify. Suggest minor revisions.
1. In the "Introduction" section, the connection between active learning and high information quality images is abrupt. It suggests to add some supplementary instructions to facilitate readers' understanding.
2. In the "Related Work" section, there are errors in writing and typesetting, which are recommended to be revised. For example, "deep learningxia2017aid".
3. In the "The Proposed Method" section, there are errors in the section number, which are recommended to be revised. For example, lines 132 and 133.
4. In the "The Proposed Method" section, the transition is rigid and lacks explanation, for example, why is the image with high node entropy located in the center of the labeled node sample.
5. The full text of the manuscript needs careful editing, with special attention to English grammar, spelling, and sentence structure.
Reviewer 2 Report
The article has serious flaws and seems unfinished. It especially has problems in the use of the English language up to a point where it is not understandable anymore.
Language:
For example line 25, and 26 on page 1 are completely not understandable. Similar issues can be found throughout the whole manuscript which makes it hard to read and impossible to understand.
Other issues:
- There is a broken reference on line 70, page2.
- I don't understand equation (3). Is there a typo?
- The technique explained in chapter 3, seems to be very close to k-means or similar clustering algorithms. Please explain the difference.
- For me, it is completely not clear, how the equations of chapter 3 are used in the experiments. However, this might be related to the language issue.
Please fix the language problems in the article.
Reviewer 3 Report
This paper proposes a remote sensing image quality evaluation method based on node entropy. The method includes a node selection module and a remote sensing image quality evaluation module. The results of this study may contribute to provide low-cost guidance for remote sensing image collection and labeling. The topic of this study is interesting, while there are some concerns that the authors should address before it can be considered for publication.
(1) My major concern is whether the method proposed in this paper is useful for all the remote sensing data? The authors should further discuss or clarify the possible differences of quality evaluation for different remote sensing data.
(2) I suggest the authors add more detail information about the remote sensing data used in this article.
(3) Some mechanism explanations should be better added to further explain the remote sensing data inaccuracy related to quality evaluation method in this study.
(4) A paragraph of limitation discussion should be added to clarify the limitation or uncertainty of data and methods in this current study. For example, the uncertainty of remote sensing data (Decuyper et al., 2020; Shen et al., 2022) may affect the research results.
(5) Some writing of this study should be further improved. For example, Lines 243-245, it is not a complete sentence.
Spatio-temporal assessment of beech growth in relation to climate extremes in Slovenia–An integrated approach using remote sensing and tree-ring data. Agricultural and Forest Meteorology, 2020, 287, 107925.
Vegetation greening, extended growing seasons, and temperature feedbacks in warming temperate grasslands of China. Journal of Climate, 2022, 35, 5103-5117.
