Quality Assessment of SAR-to-Optical Image Translation
Round 1
Reviewer 1 Report
Authors have made an effort to improve the readibility of the manuscript. Also Engligh language has been improved.
I have no further comments.
Author Response
Thanks very much for your comment.
Reviewer 2 Report
The authors have done an excellent job to address the reviewer comments and I would recommend the manuscript as suitable for publication.
Author Response
Thanks very much for your comment.
Reviewer 3 Report
The authors introduced a study to combine image quality assessment (IQA) with SAR-to-optical image translation for pursuing a suitable evaluation approach. Extensive comparisons of the IQA models are performed and evaluated for the optimization of image restoration and scene classification. However, there are some comments that should be addressed by the authors.
1- The authors should give the advantages and disadvantages of the current related work presented in related works.
2- The authors should write the major contributions to the field at the end of related works.
3- However, the work can be considered as an exploratory method in the evaluation and optimization of SAR-to-optical image translation. The contribution is just an assessment of the image quality in terms of quality metrics and accuracy of some existed models using different deep CNN models such as Inception, VGG, SqueezeNet, and ResNet. For example, Table 5 shows that the Feature-Guided SAR-to-optical image translation (FGGAN) model proposed in “Zhang, J., Zhou, J., and Lu, X. (2020). Feature-Guided SAR-to-Optical Image Translation. IEEE Access, 8, 633 70925-70937” achieved the best classification accuracy result. The authors should give some experiments to show the effect of IQA on the results of image restoration and image classification. For instance, the author can compute the accuracy of image classification before and after performing the IQA. Statistical analysis of the effect of IQA on the results should also be presented to see if the difference in the results is significant or not.
4- English writing needs to be revised for example: “three dataset” should be changed to “three datasets” in Subsection “4.1. Datasets”.
Author Response
Dear Reviewer,
Thanks very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. Response-to-reviewer which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: 
Author Response.pdf
Reviewer 4 Report
In the manuscript a new approach to the assessment of SAR-to-optical image translation was proposed. This group of method is important not only for the evaluation of translation quality but also for optimisation of algorithms.
The topic is important and the contribution is original and relevant.
My minor remark is that the way how features are extracted is not exactly explained. Chapter 2.3 does not contain this information, so title of this chapter Image Feature Extraction is not adequate. It is a short paragraph which lists several CNNs but I miss explanation what signals from these networks are taken as features. In the next part the Authors say that they used as features ReLU layers in front of each max-pooling layer in VGG but how features are extracted from Inception, SqueezeNet (line 496)?
In Line 29 Authors say that SAR images are not suitable for visual interpretation. However, is some applications SAR images can be interpreted directly, for example deformations of terrain such as subsidence troughs can be detected by a man directly from D-InSAR images. Could the authors comment on possibilities of using their method to transform InSAR images to facilitate human interpretation?
Author Response
Dear Reviewer,
Thanks very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. Response-to-reviewer which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
Thank you for revising your manuscript according to the suggested comments. I see an improvement in the manuscript. So, it could be accepted for publication.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
Article: Quality Assessment of SAR-to-Optical Image Translation
Authors: J. Zhang, J. Zhou, M Li, T. Yu
Overall: The authors have a done thorough research in the very interesting and relevant space of SAR-to-Optical image translation. As more SAR sensors are being deployed, enabling methodologies to enhance the use of SAR becomes more important, so it is not strictly an experts-only domain. The comparison of methods and experimental setup documented in this paper is well constructed and delivered. This research effort will be a valuable contribution to the sensing community. Congratulations on a job well done. The more detailed comments below provide suggestions for word/phrase choices and some additional things to think about and incorporate into the paper.
Line 13: Phrase Choice: ‘has become a blank’ --> ‘is a research gap’
Line 55: Consider using ‘network architectures and algorithms’
Line 86: ‘A scene classification…’
Line 90: ‘…transform one kind of representation…’
Section 3: Please lay out the use and provide supporting examples of ‘low frequency’ and ‘high frequency’ to clarify for the reader. There is also some mixed use of frequency with regards to signal noise and feature extraction, ensure this is clarified throughout the manuscript.
Line 148: ‘These are:’
Line 149: Add comma after ‘translation module’
Line 169: Please explain the basis of the value N = 70.
Line 178: ‘…D* represent the generator and discriminator, respectively, when…’
Line 224: …regarded as the result of an image restoration…’
Line 182: ‘…images as similar to the reference at low-frequencies’
Line 224: ‘…regarded as the result of an image restoration…’
Line 227: Define ‘EDSR’
Line 256: Define ‘DNN’ on first use
Line 266: Perhaps note the ‘SEN1-2 dataset’ is derived from 1) ESA’s Sentinel-1 C-Band SAR using the ground-range detected (GRD) product, collected in interferometric wide swath (IW) mode, and constrained to VV polarity; and 2) ESA’s Sentinel-2 multi-spectral imagery constrained to bands 4, 3, and 2 (red, green, and blue channels). I think the little bit of information will help set the dataset context for the paper.
Line 269: ‘covering’ -> ‘cover’
Line 271: Word Choice: ‘displayed’ -> ‘presented’
Line 279: Word Choice: ‘numbers’ -> ‘values’
Line 280/Table 1 Caption: ‘brackets’ should be ‘parentheses’
Table 1: ‘Salt Noise’ should be ‘Speckle Noise’
Line 287: ‘brackets’ should be ‘parentheses’
Line 302/Figure 5: Suggest to spell out ‘OP’ to ‘Optical’ since it isn’t defined anywhere, or define in the text within Section 4.3.
Figure 5: Consider labeling the rows ‘(a),(b),(c)…’ then define the scene category within the figure caption. Additionally, rows 2 and 4 images from pix2pix à Op are dark and hard to visualize. Please consider lightening these images so the features can be better viewed.
Line 312: ‘…difficult to recognize in SAR images.’
Line 339: Is ‘bricks’ the intended word here?
Line 355: Word Choice: ‘Destination’ -> ‘intent’ or ‘objective’
Line 360: Word Choice: ‘direction’ -> ‘orientation’
Figure 6: Consider labeling the rows ‘(a),(b),(c)…’ then define the scene category within the figure caption.
Figure 9: I don’t believe the IQA metric ‘TRA’ was defined and discussed in the text. Please include.
Line 433: ‘PSNR’ – please define on first use.
Section 5: What needs to be considered with regards to SAR polarizations (i.e., single to quad polarizations)? Additionally, it would be valuable to gain author perspectives on what might be expected with these methodologies in other collection modes outside of IW. Perhaps this is a future work effort.
Line 544: Word Choice: ‘part’ -> ‘section’
Figure B1: Please lighten the images on row 3 from CycleGAN -> OP These are hard to interpret as is.
Author Response
Dear Reviewer,
Thank you very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. The cover letter which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The paper addresses an important topic in remote sensing image interpretation and feature identification from SAR data – namely how good are the reconstructions and what metrics are used to measure quality. The major contributions of the paper are the development of a model for SAR-to-optical image translation, comparison of perceptual IQA models, and summarizing translation properties in terms of applications. I find the technical content to be well thought out and the organization of the study to be reasonable. The selection of models seems defensible, though some justification of why specific models were selected may be argued for.
Unfortunately, the manuscript is not ready to be published in an English language journal as the writing is often inappropriate and, in some cases, unintelligible. Starting in the abstract and continuing throughout, I found it slow going to understand what was being said, often reading a sentence multiple times to get the meaning and in some cases I’m not sure that I got it. I am sensitive to the fact that English may not be the first language for the authors, but feel a rewrite is necessary. I encourage the authors to get assistance rewriting the language and resubmit the paper.
Some specific comments:
Line 9 – “The interpretation of Synthetic Aperture Radar (SAR) images is a challenging work…” The very first line of the abstract has improper syntax. That sets the tone for the entire paper.
Line 57 - “Inspired by [16],..” would be much clearer if you used the name and what they did. As it is written it requires the reader to go to the reference, read enough to understand what they did and how it inspired the author. Better to simply state it in a few words.
Line 66 - Figure 1. has a misspelling “Distoration”, I assume is supposed to be distortion.
Line 116 – “Feature Similarity (FSIM) [21] thinks that the importance of pixels…” . This an inappropriate use of the term “think”.
Lines 147-158 – I find this paragraph to be generally well written with the exception of the final sentence “ Finally, the generated results closest to the real ones are obtained.” I think this means that you used the generated results that are most similar to the real images?
Author Response
Dear Reviewer,
Thank you very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. The cover letter which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: Author Response.docx
Reviewer 3 Report
In this paper, several image-to-image translation models are investigated to achieve a SAR-to-optical image translation, with the aim to improve SAR images interpretation and feature extraction. The generated results are evaluated based on feedback from visual inspection. An extensive comparison between different algorithms is performed.
Even if the methodology is convincing and the experiments interesting, in this Reviewer opinion the manuscript should undergo a massive revision by the authors, especially concerning the section of the experimental results. In fact in this reviewer opinion the experimental results are described in a very messy way and many information indispensable for their correct interpretation and understading are lacking.
Also English must be properly checked and improved by the authors.
All this affects a lot the comprehension and also the legibility of the manuscript, which appears to be hardly readable and unclear. For these reasons due to the considerable amount of work to be done, in this reviewer opinion the manuscript in the present form is not suitable for publication.
All my detailed comments and suggestions are in the attached file.
Comments for author File: Comments.pdf
Author Response
Dear Reviewer,
Thank you very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. The cover letter which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: Author Response.docx
Reviewer 4 Report
This manuscript compares several SAR-to-optical methods and several IQA metrics used for an image restoration CNN. Overall, it is a good manuscript but the fact that we are looking at evaluating two problems at the same time, translation and restoration, makes it a little bit hard to Follow. There are also numerous typos and some sentences are difficult to understand, an english proofreading is recommended before publication. Other comments:
- In Figure 1: Traditional Distoration
- In Figure 5: explain what OP means
- Figure 6: put the name of the methods involved for each row
- The first paragraph of section 4.4 needs to be clarify, it is not clear what are the experiments here. There are at least four terms that are used here: initial, original and reference images.
- Line 354: Guassian Blur
- Line 355: Destination of the experiment... ?
- Line 358: 'Then, we select superior ones to assess the translation result..' not clear, do you mean the maximum value?
- Line: 425: 'the vertical axis denotes IQA models used to evaluate restoration performance' I think you mean IAQ metrics here, please check.
- There are no mentions of Table 3 in the text.
- Line 442-452 and Table 4: I don't understand the meaning of the term feature in this context. You are talking about scene classification here, right?
Author Response
Dear Reviewer,
Thank you very much for your constructive comments concerning our manuscript. The comments are valuable and helpful for revising and improving our paper, as well as the important guiding significance to our researches. We have studied the comments carefully and have made corrections which we hope meet with approval. All revisions use the "Track Changes" function in Microsoft Word, so that changes are easily visible. The cover letter which provides a point-by-point response to the comments is in the attachment.
Once again, thank you very much for your comments and suggestions.
Best regards,
Jiexin Zhang et al.
Author Response File: Author Response.docx
Round 2
Reviewer 2 Report
Needs to be re-written for English publication.
Reviewer 3 Report
I thank the authors for taking into account my comments and suggestions. I think that authors have made an effort to improve the manuscript. In particular results now are more clearly presented.
I would have preferred that authors cleaned up the manuscript of the old text which was raplaced by the new one (red and underlined) to facilitate reading.
Finally I found that figure 5d has no longer red and green rectangles as described in the text. Please check.
