Double Deep Q-Network for Hyperspectral Image Band Selection in Land Cover Classification Applications
Round 1
Reviewer 1 Report
This study presents a novel approach, Double Deep Q-Network, for Hyperspectral Image Band Selection in Coastal Land Cover Classification Applications. This is potentially a good article and fits well with the aims and scopes of Remote Sensing. Some minor improvements could be considered before possible publication:
Abstract: Good.
Line 24.
OA>>Overall Accuracy?
Please also state other indicators of accuracy assessment. It would also be good to describe the actual accuracies instead the difference in accuracies.
Introduction
The introduction is very good; the authors demonstrate a thorough knowledge of the published literature and highlight the importance and background to carry out this investigation.
It lacks the actual context of the problems. The authors should discuss the background of the problem being solved, especially in the first two paragraphs. Giving a note on the existing Hyperspectral Data may not be suitable.
The rest of the introduction is good, first two paragraphs should be better placed in the section.
Methods
Methods are technically strong and well explained.
Experimental Results and Discussion
Table 1. Could be better presented and the placement is wrong, the table caption is appearing later. Please revised and replace.
There were 20 classes for the land cover, did I miss something about the reduction of classes to 15 as mentioned in Table 1?
Figure 3: Figure 3C should be Figure 3D. There is a very slight difference between the proposed method and existing methods which also have reasonably high accuracy (Figure 3A and 3D). This is also indicated in Table 3. The proposed method is not the best solution always, how could authors justify this?
Figure 4: The number of bands selected by each algorithm should also be mentioned.
Figure 5: Would it be possible to overlay Figure 5B over Figure 5A?
PRISMA Experiment: Pre-processing, type of the data (L1 / L2), and removing water absorption bands are important to describe before experimenting, as the accuracy may vary with pre-processing methods applied.
Figure 7. Sentinel-2 data is not described in the text, what is the context of the S2 data here? It might not be good to compare 10 m and 30 m spatial resolution.
Conclusion
No Comments. Authors could briefly discuss the possible challenges and aspects for loss or gain in accuracy while applying this algorithm in other than coastal environments or this algorithm specific to the coastal environment.
Author Response
Abstract
Comment 1: Line 24.OA>>Overall Accuracy?
Response: Thanks for your kind suggestion. It has been revised.
Comment 2: Please also state other indicators of accuracy assessment. It would also be good to describe the actual accuracies instead the difference in accuracies.
Response: Thanks for your kind suggestion. It has been revised and other evaluation indicators have been added.
Introduction
Comment 3: The introduction is very good; the authors demonstrate a thorough knowledge of the published literature and highlight the importance and background to carry out this investigation. It lacks the actual context of the problems. The authors should discuss the background of the problem being solved, especially in the first two paragraphs. Giving a note on the existing Hyperspectral Data may not be suitable. The rest of the introduction is good, first two paragraphs should be better placed in the section.
Response: Thanks for your kind suggestion. The first paragraph and parts of the second paragraph have been deleted. The rest of the second paragraph and the third paragraph were combined to introduce the advantages and disadvantages of the current hyperspectral band selection methods.
Methods
Methods are technically strong and well explained.
Experimental Results and Discussion
Comment 4: Table 1. Could be better presented and the placement is wrong, the table caption is appearing later. Please revised and replace.
Response: Thanks for your kind suggestion. Table 1 has been modified and re-designed.
Comment 5: There were 20 classes for the land cover, did I miss something about the reduction of classes to 15 as mentioned in Table 1?
Response: Thanks for your kind suggestion. Indian Pines dataset has 16 classes, this dataset was downloaded from MultiSpec© | Tutorials (purdue.edu). The classes in Table 1 were displayed according to the available classes.
Comment 6: Figure 3: Figure 3C should be Figure 3D. There is a very slight difference between the proposed method and existing methods which also have reasonably high accuracy (Figure 3A and 3D). This is also indicated in Table 3. The proposed method is not the best solution always, how could authors justify this?
Response: Thanks for your kind suggestion. The label of Figure 3 has been modified. Hyperspectral dimensionality reduction methods can be categorized into feature extraction methods and band selection methods. Feature extraction methods (PCA, mvPCA, ICA) have good outcomes, but the extracted features can’t be explained and they are not image bands which makes feature extraction methods are less attractive in practice. In Indian Pines dataset, features extraction methods have strong advantages with using SVM verification. Clustering methods, the proposed method and other reinforcement learning methods are band selection methods, the selected bands can be explained physically and they can be easily and efficiently applied in practice. In the three experimental datasets, overall accuracy (OA) of the proposed method is higher than other band selection methods, and the reasons of why average accuracy (AA) occasionally appears low are explained in Lines 425-438.
Comment 7: Figure 4: The number of bands selected by each algorithm should also be mentioned.
Response: Thanks for your kind suggestion. This has been revised.
Comment 8: Figure 5: Would it be possible to overlay Figure 5B over Figure 5A?
Response: Thanks for your kind suggestion. The structure of the paper has been modified. Figure 5 is now Figure 4. Figure 4A is the PRISMA image, Figure 4B is the Sentinel-2 image. Because PRISMA and Sentinels-2 were captured almost at the same time (no much visual differences if displayed in RGB colors) therefore Figure4B has been removed.
Comment 9: PRISMA Experiment: Pre-processing, type of the data (L1 / L2), and removing water absorption bands are important to describe before experimenting, as the accuracy may vary with pre-processing methods applied.
Response: Thanks for your kind suggestion. This advice is very important and critical in deed. We used Level 1 products. There is no pre-process about PRISMA dataset and Washington DC Mall dataset and didn’t remove water absorption bands. Some experts advised to keep all bands because this study is to select bands among all available bands. The Indian Pine dataset removes 20 water absorption bands. We have added the description in “C) PRISMA dataset” of section 3 “Dataset”.
Comment 10: Figure 7. Sentinel-2 data is not described in the text, what is the context of the S2 data here? It might not be good to compare 10 m and 30 m spatial resolution.
Response: Thanks for your kind suggestion. Sentinel-2 has been added to the description in “D) Sentinel-2 MRS” of section 3 “Dataset”. Although the resolutions are different it would still be interesting to show the classification differences due to both Sentinel-2 and PRISMA were captured only 10 minutes time apart.
Conclusion
Comment 11: No Comments. Authors could briefly discuss the possible challenges and aspects for loss or gain in accuracy while applying this algorithm in other than coastal environments or this algorithm specific to the coastal environment.
Response: Thanks for your kind suggestion. According to your suggestion, some analysis discussions were added in the conclusion section.
Reviewer 2 Report
This paper proposed a new band selection method for hyperspectral image data. The method can obtain higher classification accuracy compared with other methods. However, some issues should be improved.
Main issues:
1. There may have some logical problems in the Introduction. The author said the proposed method is a partially supervised method (semi-supervised or supervised?) However, why too many un-supervised band selection method characteristics are introduced in detail (lines 82 to 100)? As the description from lines 75 to 100, readers will think the study may propose an un-supervised BS method.
2. Some expressions for the experimental results are not rigorous. For example, it can be seen in Figure 3 (B) that PCA and mvPCA have higher OA than the proposed method when using the SVM-RBF classifier. So "the proposed method has achieved the highest OA when RF, SVM-RBF and KNN were used as the classifiers when 5 to 60 bands were selected." may not be proper. The bold fonts in Tables 3, 4, and 5 are confusing, some are the highest results, but some are not. Please check and revise them.
Minor issues:
1. Line 44. "This research mainly focused on hyperspectral Images." can be deleted.
2. Line 46. "spectroscopy" can be replaced by "technology".
3. Line 48. The sentence can be revised to "It has a total of 242 spectral bands that acquire images with 30 m spatial resolution and 10 m spectral resolution, respectively."
4. Line 73. "This paper focuses on the BS study for dimensionality reduction." can be deleted.
5. "research" can be replaced by "community".
6. Lines 136-138. For the contribution (3), does it have significant meaning? The proposed method can also be used for other datasets, why the PRISMA data are introduced there? On the other hand, other BS methods can be used for the PRISMA data. Therefore, I do not think the contribution (3) is meaningful.
7. Line 274. Only the loss function can be used to evaluate the model metrics?
8. Line 287. There should be a space between "20" and "m". Please fix other similar errors in the article.
9. Line 322. Please fix "f eature".
10. Line 364. The format of Table 1 has some errors.
11. Lines 405-406. Does it have some experimental results that indicate the over fitted for CNN?
12. Line 426. The full name should be given in the first place it appears.
13. Does the hyperspectral data of PRISMA is public? Where can it be downloaded?
14. Lines 538-540. One possible reason for the OA increase is that the signal-to-noise ratio increase when you use averaging. The reason and detailed introduction can be found in the paper (Tradeoffs in the Spatial and Spectral Resolution of Airborne Hyperspectral Imaging Systems: A Crop Identification Case Study. DOI: 10.1109/TGRS.2021.3096999.)
Author Response
Main issues:
Comment 1: There may have some logical problems in the Introduction. The author said the proposed method is a partially supervised method (semi-supervised or supervised?) However, why too many un-supervised band selection method characteristics are introduced in detail (lines 82 to 100)? As the description from lines 75 to 100, readers will think the study may propose an un-supervised BS method.
Response: Thanks for your kind suggestion. In this study, band selection is based on DRL method. itself is an unsupervised process. The core of deep reinforcement learning is the reward function which guides the learning. This paper uses labeled data to calculate the reward value of each band according to land cover classification results. This reward value was then used in the reward function of deep reinforcement learning. This proposed method indirectly uses labeled data to guide the learning, therefore it is different from the supervised methods and semi-supervised methods. We will take your advice to think it more deeply in our research.
Comment 2: Some expressions for the experimental results are not rigorous. For example, it can be seen in Figure 3 (B) that PCA and mvPCA have higher OA than the proposed method when using the SVM-RBF classifier. So "the proposed method has achieved the highest OA when RF, SVM-RBF and KNN were used as the classifiers when 5 to 60 bands were selected." may not be proper. The bold fonts in Tables 3, 4, and 5 are confusing, some are the highest results, but some are not. Please check and revise them.
Response: Thanks for your kind suggestions. Hyperspectral dimensionality reduction can be categorized into feature extraction and band selection. PCA, mvPCA and ICA which are actually feature extraction methods and they have higher OA in some cases, but the extracted features can’t be regarded as selected bands and therefore they are less attractive in practice. The proposed method is purely a band selection method based DRL, it has the highest OA among band selection methods on all three datasets. “RF, SVM-RBF and KNN” has been changed to “RF, KNN and CNN”. Because of the structure of the paper has been extensively modified, Tables 3, 4, and 5 are now renamed to Tables 4, 5, 6, respectively. In these tables, the highest scores which are from band selection methods were highlight.
Minor issues:
Comment 1: Line 44. "This research mainly focused on hyperspectral Images." can be deleted.
Response: Thanks for your kind suggestion. This sentence has been deleted.
Comment 2: Line 46. "spectroscopy" can be replaced by "technology".
Response: Thanks for your kind suggestion. There are too many descriptions about hyperspectral images. The Line 45 – Line 50 have been deleted.
Comment 3: Line 48. The sentence can be revised to "It has a total of 242 spectral bands that acquire images with 30 m spatial resolution and 10 m spectral resolution, respectively."
Response: Thanks for your kind suggestion. The Lines 45 –50 have been deleted.
Comment 4: Line 73. "This paper focuses on the BS study for dimensionality reduction." can be deleted.
Response: Thanks for your kind suggestion. This sentence has been deleted.
Comment 5: "research" can be replaced by "community".
Response: Thanks for your kind suggestion. This sentence has been replaced.
Comment 6: Lines 136-138. For the contribution (3), does it have significant meaning? The proposed method can also be used for other datasets, why the PRISMA data are introduced there? On the other hand, other BS methods can be used for the PRISMA data. Therefore, I do not think the contribution (3) is meaningful.
Response: Thanks for your kind suggestion. PRISMA is a newly available hyperspectral satellite and we haven’t found any published materials about the band selection of PRISMA, so when we developed our method, we have the intention to apply to PRISMA data. The selected PRISMA scene is just 10 minutes apart from Sentinel-2 in the Chongming Island area, so we think this is a valuable data source and suitable for comparation. The structure of the paper has been extensively modified. the contribution (3) has been revised.
Comment 7: Line 274. Only the loss function can be used to evaluate the model metrics?
Response: Thanks for your kind suggestion. “The proposed DDQN based BS method” section has been revised. We have added the description about the structure of methods and the parameters.
Comment 8: Line 287. There should be a space between "20" and "m". Please fix other similar errors in the article.
Response: Thanks for your kind suggestion. There are two places about “spatial resolution”. these have been revised.
Comment 9: Line 322. Please fix "f eature".
Response: Thanks for your kind suggestion. this has been revised.
Comment 10: Line 364. The format of Table 1 has some errors.
Response: Thanks for your kind suggestion. Table 1 has been modified and re-designed.
Comment 11: Lines 405-406. Does it have some experimental results that indicate the over fitted for CNN?
Response: Thanks for your kind suggestion. We used the CNN method which is from RLSBS/波段选择_A3C_固定波段 at main · jiefeng0109/RLSBS · GitHub. It was described in Lines 527-528. The experimental data is small and CNN method is easy to over-fit in a small amount of data. The baseline and proposed method can be downloaded from GitHub - hy-shou/bandselection: Hyperspectral Band Selection in Coastal Land Cover Classification.
Comment 12: Line 426. The full name should be given in the first place it appears.
Response: Thanks for your kind suggestion. We have checked and revised.
Comment 13: Does the hyperspectral data of PRISMA is public? Where can it be downloaded?
Response: Thanks for your kind suggestion. PRISMA data can be applied to download, once approved the data can be downloaded from Mission Selection Form (asi.it) for free. This is user's manual (how to apply): Microsoft Word - PRISMA User Manual_Is1_1.docx (asi.it)
Comment 14: Lines 538-540. One possible reason for the OA increase is that the signal-to-noise ratio increase when you use averaging. The reason and detailed introduction can be found in the paper (Tradeoffs in the Spatial and Spectral Resolution of Airborne Hyperspectral Imaging Systems: A Crop Identification Case Study. DOI: 10.1109/TGRS.2021.3096999.)
Response: Thanks for your kind suggestion. This is a very interesting paper, we found that our experiment results were consistent with the experimental results from this paper. We have cited this paper.
Reviewer 3 Report
1. What is the reward mechanism? and it improves the performance, clarify, and justify.
2. what is the difference between a double Q and a double deep Q network?
3. The contributions of this research in the current manuscript are not crystal clear. I suggest adding bullet-wise contributions in the last paragraph of the introduction section.
4. A complete framework with each step explained visually is missing from the paper. The Figure is just a working flow of dataset splitting without any visual contents to show the proposed system in action for input to the final output. I strongly recommend adding a complete self-explainable framework.
5. I cannot make it clear what is the difference between the proposed method and the existing DRL method. The authors should reorganize the manuscript to make it clearer and more logical.
6. Include a problem formulation section in your paper & define the problem mathematically.
7. Discuss the computational complexity of the proposed method.
8. Hyper-parameter setting of the proposed network should be explained properly and justified.
9. Authors should justify the need of using "band selection" for the problem solved in the paper.
10. Error-based performance indicators can also be used to evaluate the performance of the proposed approach.
11. Need to share the system configuration where you applied these tests.
12. Need to show the significance of your model's improvement comparing to other models.
13. Is your dataset and codes available? If not, why? and if yes, please add a link to the codes and dataset so that your study could be repeatable for other researchers.
14. Application part of the proposal is not well defined. Where would your solution find a use? Please discuss future aspects, or if you already have applied present this application.
15. Extension to literature will be appreciated: DOI: 10.3390/electronics11091328, DOI: 10.1016/j.seta.2022.102275. Please cite in section II behind the CNN and DL to enrich the literature.
Author Response
Comment 1: What is the reward mechanism? and it improves the performance, clarify, and justify.
Response: Thanks for your kind suggestion. “Reward Functions” section describes in detail. The reward function determines the direction of training. If the action results in good results, a big reward was given, and on the contrary, a negative or small reward was given. The goal of the algorithm is to obtain maximize rewards from a series of selected actions. This study compares different reward functions. The results of different reward mechanisms have been shown in “Reward Function Comparison” section.
Comment 2: what is the difference between a double Q and a double deep Q network?
Response: Thanks for your kind suggestion. The basic algorithm of DRL is Q-Learning. The core is to train q-table, which is used to select the next action with the largest reward in the current state. When the state is continuous, q-table cannot be exhaustive, DQN (Deep Q-network) is introduced, q-table is replaced by a deep neural network (which is called Q-network). However, when calculating the reward of a certain action, the reward is also determined by the reward of the subsequent action sequence, as shown in Formula 9, which leads to a relatively large reward estimated by the Q-network. The target Q-network was introduced in DDQN (double deep Q network), one network (Q-network) is used to calculate the action of obtaining the maximum reward value, which is the training network. The other network (target Q-network) calculates the reward under this action as the target. The algorithm section is shown in the text for details. “a double Q” and “a double deep Q network” is the same method.
Comment 3: The contributions of this research in the current manuscript are not crystal clear. I suggest adding bullet-wise contributions in the last paragraph of the introduction section.
Response: Thanks for your kind suggestion. One contribution of this study is the use of DDQN, and another contribution is that the labelled data of land cover was introduced into the reward function to make the classification more accurate. This is modified and added to the introduction section.
Comment 4: A complete framework with each step explained visually is missing from the paper. The Figure is just a working flow of dataset splitting without any visual contents to show the proposed system in action for input to the final output. I strongly recommend adding a complete self-explainable framework.
Response: Thanks for your kind suggestion. The framework was added to the end of “Method” section.
Comment 5: I cannot make it clear what is the difference between the proposed method and the existing DRL method. The authors should reorganize the manuscript to make it clearer and more logical.
Response: Thanks for your kind suggestion. The other studies based on DQN. Due to some drawbacks of DQN, this study uses the DDQN model instead, and the “Method” section has described in detail about both DQN and DDQN. The framework was added for more intuitive display.
Comment 6: Include a problem formulation section in your paper & define the problem mathematically.
Response: Thanks for your kind suggestion. The problem we want to tackle is to select the most suitable bands from many (several hundreds) available hyperspectral bands, which is hard to write as mathematical equations or definitions. It’s more like a real scenario application which requires many processing steps (band selection, classification etc.). We hope we have described the problems adequately at the beginning and through the paper.
Comment 7: Discuss the computational complexity of the proposed method.
Response: Thanks for your kind suggestion. The time complexity has been added to the "The proposed DDQN based BS method" section.
Comment 8: Hyper-parameter setting of the proposed network should be explained properly and justified.
Response: Thanks for your kind suggestion. The core of the DDQN algorithm is the reward function that affects the training direction. The Q-network is only a simple two-layer full connection, this is very simple. This study focuses on the reward function, and without updating Q-network. The experiment used standard parameters and tuned the number of iterations. We have updated the algorithm structure and parameters settings description according to your suggestions。
Comment 9: Authors should justify the need of using "band selection" for the problem solved in the paper.
Response: Thanks for your kind suggestion. The huge volume of hyperspectral data significantly increases computational inefficiency and also causes storage problems. In order to reduce the dimension and improve the efficiency, DDQN is introduced to select the most important bands for classification applications. Compared with other band selection methods, DDQN has several advantages which were shown and compared during several experiments.
Comment 10: Error-based performance indicators can also be used to evaluate the performance of the proposed approach.
Response: Thanks for your kind suggestion. Based on the previous research and similar band selection methods, the overall accuracy (OA), average accuracy assessments (AA) and kappa coefficient (Kappa) were commonly used for evaluation, so we stick with those indicators so far, we may explore error-based indicators in our further investigations.
Comment 11: Need to share the system configuration where you applied these tests.
Response: Thanks for your kind suggestion. It is introduced in the last sentence of the section "The proposed DDQN based BS method".
Comment 12: Need to show the significance of your model's improvement comparing to other models.
Response: Thanks for your kind suggestion. One of the key contributions of our study was to introduce better reward functions, which is introduced in the first part of the experiment. Another contribution was that the DDQN was used for band selection during DRL. In the experiments, this new method was compared with other similar band selection methods using three public available datasets.
Comment 13: Is your dataset and codes available? If not, why? and if yes, please add a link to the codes and dataset so that your study could be repeatable for other researchers.
Response: Thanks for your kind suggestion. the data can be download from MultiSpec© | Tutorials (purdue.edu), and Mission Selection Form (asi.it). All of datasets used are publicly available. Our code can be downloaded from GitHub - hy-shou/bandselection: Hyperspectral Band Selection in Coastal Land Cover Classification.
Comment 14: Application part of the proposal is not well defined. Where would your solution find a use? Please discuss future aspects, or if you already have applied present this application.
Response: Thanks for your kind suggestion. We have the final application goal in our mind, that is to use the proposed method to selected optimal subset of bands from hyperspectral images and then apply land classification using only selected bands. The band selection method is used to solve the data redundancy problem, so it is not obvious for classification application but it is an essential pre-processing step. This study compares and analyzes the influence of band selection on the application of land cover.
Comment 15: Extension to literature will be appreciated: DOI:10.3390/electronics11091328, DOI: 10.1016/j.seta.2022.102275. Please cite in section II behind the CNN and DL to enrich the literature.
Response: Thanks for your kind suggestion. We have read the paper of DOI:10.3390/electronics11091328. This paper uses convolutional capsule and bi-directional gated recurrent unit networks to extract spectral features from the speech signals, which is very interesting to us and could be used in hyperspectral feature extraction in future. We have cited this paper. The second paper (DOI: 10.1016/j.seta.2022.102275) uses CNN-DeepESN, which does not a direct linkage to our problems at this stage, we need to study further. Thanks for letting us know these two papers.
Round 2
Reviewer 2 Report
Please note that Chinese appears in the revised manuscript.
Author Response
Thanks for your kind suggestion. we have revised.
Reviewer 3 Report
The authors addressed my comments and recommendations successfully. Good Luck!
Author Response
Thank you.
