Application of a Convolutional Neural Network for the Detection of Sea Ice Leads
, Steven A. Ackerman 1, Yinghui Liu 2, Jeffrey R. Key 2 and Iain L. McConnell 3
Round 1
Reviewer 1 Report
- The paper suffers lack of literature review. Please add more relevant papers in a specific section "related works". There is much recent work in this area. I strongly suggest including these papers and concretely position your method.
N. Asadi, K. A. Scott, A. S. Komarov, M. Buehner, and D. A. Clausi, “Evaluation of a neural network with uncertainty for detection of ice and water in sar imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, pp. 247–259, 2021.
Han, Y.; Liu, Y.; Hong, Z.; Zhang, Y.; Yang, S.; Wang, J. Sea Ice Image Classification Based on Heterogeneous Data Fusion and Deep Learning. Remote Sensing2021,13, 592
Salman Khaleghian, Habib Ullah, Thomas Kræmer, Torbjørn Eltoft, Andrea Marinoni, “Deep Semi-Supervised Teacher-Student Model based on Label Propagation for Sea Ice Classification”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing PP(99):1-1
- In semantic segmentation, the accuracy and F1 are not always enough. IoU metric is also very important. Therefore, please add this metric to your results section.
- Please add several ground truth samples (labeled data) with inputs and network outputs.
- The method is not described clearly, and I am afraid the paper cannot be published in this form. It is better to have a figure to illustrate a big picture of your method and training and validation dataset.
- You train two different models on two different input data (satellite data). Have you combined data? It seems you consider them as a wholly separated model. Would you please describe your intention clearly?
- Is it an active learning method? Or the iterative process is used to generate training data and then train a model based on them? Describe it. Which model and data are used in this process?
- Have you used this process for the legacy methods as well? Your comparison should be on the same ground and fair.
- How about overfitting? How many samples did you have for training?
- Training, validation, and test data are not described clearly.
- Have you used any augmentation?
- What is your patch size to train the network? Is it 7000x7000 or 512x512?
- Page 3, line 134:
- Please indicate the number of tiles (patches) you have in your dataset for each class?
- How many tiles (patches) have you used for training, validation, and test?
- You mentioned you use the validation set to make your training set? Why? Are you using your validation set for training?
- Does your training data come from a specific area? How about test and validation data? Please describe.
- Although U-Net is a famous network architecture, please carefully describe the network architecture's hyperparameters and training process. Number layers in Encoder and Decoder path, filter size, and etc. Learning rate (any changes during training), batch size
- How did you find the hyperparameters? Please update the manuscript.
- researchers should be able to replicate your results.
- Page 4, line 155: What do you mean by 30 steps over 200 epochs?
- Did you mask open water manually? How about the larger lead in ice?
- What is your spatial resolution in figure 6 and FigureA1?
Author Response
Reponses to Review #1 on “Application of a convolutional neural network for the detection of sea ice leads”
Reviewer’s comments are in italics, and responses are in bold.
- The paper suffers lack of literature review. Please add more relevant papers in a specific section "related works". There is much recent work in this area. I strongly suggest including these papers and concretely position your method.
- Asadi, K. A. Scott, A. S. Komarov, M. Buehner, and D. A. Clausi, “Evaluation of a neural network with uncertainty for detection of ice and water in sar imagery,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 1, pp. 247–259, 2021.
Han, Y.; Liu, Y.; Hong, Z.; Zhang, Y.; Yang, S.; Wang, J. Sea Ice Image Classification Based on Heterogeneous Data Fusion and Deep Learning. Remote Sensing2021,13, 592
Salman Khaleghian, Habib Ullah, Thomas Kræmer, Torbjørn Eltoft, Andrea Marinoni, “Deep Semi-Supervised Teacher-Student Model based on Label Propagation for Sea Ice Classification”, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing PP(99):1-1
Thank you, references to these 3 papers have been added.
- In semantic segmentation, the accuracy and F1 are not always enough. IoU metric is also very important. Therefore, please add this metric to your results section.
Yes, the F1 score is not always enough. The IoU metric is equivalent to the provided CSI skill score. We added in the revised manuscript, “critical success index (CSI, which also known as Intersection over Union or IoU)”.
- Please add several ground truth samples (labeled data) with inputs and network outputs.
“Ground truth” of sea ice leads is not readily available. Hand analysis is the next best thing. The Appendix contains figures that illustrate the full comparison. The “Data Availability Statement” section mentions all input and output files available on an ftp server.
- The method is not described clearly, and I am afraid the paper cannot be published in this form. It is better to have a figure to illustrate a big picture of your method and training and validation dataset.
The AI (U-Net) method has already been described in literature, reference 18. The intent here is to describe a new application of that existing method rather than provide a robust description of a previously published model. Some text and a table have been added.
- You train two different models on two different input data (satellite data). Have you combined data? It seems you consider them as a wholly separated model. Would you please describe your intention clearly?
The intention is to show that the same technique can be applied to either satellite independently. The two satellites span different time periods. The first MODIS (on Terra) launched in 1999 and nearing end of life; the first VIIRS launched in 2011. Some text has been added to the Conclusions section. Yes, it would be possible to develop a third model, one that uses input from both (or either) satellite. For brevity, this was an option not presented in this paper. A follow-up paper will address long-term analysis of the combined product from both satellites, and this will be a topic of discussion in that paper.
- Is it an active learning method? Or the iterative process is used to generate training data and then train a model based on them? Describe it. Which model and data are used in this process?
No, this is not active learning method. Additional text has been added to described the iterative method.
- Have you used this process for the legacy methods as well? Your comparison should be on the same ground and fair.
The validation methods in the paper represent new work and predate the legacy method. The legacy method is not AI based, more background description of the legacy method has been added.
- How about overfitting? How many samples did you have for training?
6000 samples were used. Further description has been added.
- Training, validation, and test data are not described clearly.
More detail has been added and method section has been revised and rearranged for clarity.
- Have you used any augmentation?
Discussion of augmentation has been expanded.
- What is your patch size to train the network? Is it 7000x7000 or 512x512?
The model is trained on 512x512 samples; the entire arctic domain is on the order of 7000x7000 km. This has been clarified in the revised manuscript.
- Page 3, line 134: Please indicate the number of tiles (patches) you have in your dataset for each class?
Each model was trained with 6,000 tiles, each tiles contains a mixture of lead and lead-free pixels. This has been clarified in the revised manuscript.
- How many tiles (patches) have you used for training, validation, and test?
Each model was trained with 6,000 tiles. Some text for clarification
- You mentioned you use the validation set to make your training set? Why? Are you using your validation set for training?
Methods are described to keep validation and training separate, the description has been rearranged and updated for clarity.
- Does your training data come from a specific area? How about test and validation data? Please describe.
The whole Arctic domain is used, text added for clarity.
- Although U-Net is a famous network architecture, please carefully describe the network architecture's hyperparameters and training process. Number layers in Encoder and Decoder path, filter size, and etc. Learning rate (any changes during training), batch size
More details, including a table and references have been added.
- How did you find the hyperparameters? Please update the manuscript.
Added a reference and Table 1.
- researchers should be able to replicate your results.
See “Data availability Statement” section; files and reference code are available on an ftp server
- Page 4, line 155: What do you mean by 30 steps over 200 epochs?
Each epoch has 30 steps; 6,000 samples are used, with 30 each of the 200 epochs
- Did you mask open water manually? How about the larger lead in ice?
Open water is excluded by using a sea ice concentration mask [27]. The AI is not programed with any explicit lead size criteria that would prevent the detection of larger leads.
- What is your spatial resolution in figure 6 and FigureA1?
The native resolution is 1km; images are resampled to fit inside the paper document. Full resolution imagery has been uploaded to the publisher.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Dear authors,
Thanks you for an interesting manuscript.
The manuscript presents a study of detecting sea ice leads using machine learning with the convolutional neural network approach. The outcome of the study provides insight into the advantages of using AI compared to traditional image analysis.
Generally, the manuscript is well written and presents the results well but it lacks clarification on a few points. Especially regarding the design of the study with the choices of training versus validation data. It is difficult to asses whether the datasets are truly independent. More clear description regarding this would strengthen the analysis.
One limitation of the study is the lack of validation with independent data sources. This point is stated in the manuscript but could perhaps be even more clear. It would benefit the study to investigate options for “real” validation.
The structure of the manuscript could also be improved. Mainly Section 2 Materials and Methods need to be reorganized (see specific comments below).
Further detailed suggestions for improvements are listed below with reference to line numbers of the manuscript:
L. 103-116 This part describes the pre-processing of the data. This should be moved to the beginning of the section
L.117-133 This part describes the training dataset development and should also be moved up in the section
L. 134-150 Would it be possible to exclude the training data from the results?
L. 155-156 Please define binary cross entropy. There are similar metrics used throughout the analyses that need to be defined and/or refer to relevant references.
L. 199-200 … dependency, which …as cloudy and this is…
L. 201-216 This part needs to be clarified and metrics need to be defined and explained better. Could you refer to a few appropriate references?
L. 214-216 If the F1 Score is the most suited, why consider the others? Perhaps elaborate on this point.
L. 324-326 This is a very important point – but no validation is done in this study. Perhaps comment on the implications for the study. Perhaps elaborate more on this in the Discussion section.
Author Response
Please see the attachment
Author Response File: 
Author Response.docx
Reviewer 3 Report
This manuscript presents a novel approach to utilize remote sensing images and trainable AI to detect leads. Compared to the visual detection method, the new method seems to be more efficiency. The manuscript is organized well and the methods are well described. So, my recommendation for this manuscript is accept after minor revision.
Some suggestions or comments:
- The figures should be placed nearby the corresponding descriptions in the text for the convenience of the readers.
- High precision elevations from altimeters such as ICESat-2, Operation IceBridge can be used for validations of the quality of leads detection results.
- To obtain long-time series leads product, how to remove the effect of cloud cover.
- How about leads detection with this approach for the months of May to September? Due to the existence of melt ponds.
Author Response
Please see the attachment
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
Authors significantly improve and clarify the manuscript.
Adding a lead mask (a labeled patch) in figure 2 beside input and result could be helpful.
Author Response
Panel C has been added to Figure 2 to illustrate the lead labels.
