Cycle Generative Adversarial Network Based on Gradient Normalization for Infrared Image Generation

Round 1

Reviewer 1 Report

In this paper, we propose a gradient normalization based cycle GAN for infrared image generation. To improve generation performance, they introduce a ResNet backbone in the generator, incorporate the channel and spatial attention mechanism to enhance the feature perception and involve the gradient normalization to avoid model collapse. Their experiments demonstrate that the method is better than the original cycle GAN.

However, I have the several major concerns:

1. The paper has many grammar errors and is very hard to read.

Some sentences are as long as 3 lines in the paper. Even in the title, "gradient normalisation" is also a typo.

2. The motivation and the paper target are vague.

In the introduction, the authors aim to solve the model collapse issue by using gradient normalization. However, in the related work, they did not introduce any  gradient normalization literature, while the differences between their method and cycle GAN are introduced. Then, what is the problem you want to solve?

3. The proposed method is unclear.

In the section 2.2 , the channel and spatial attention in Figure 2 fail to show how they works. And what are the symbol /operator of "x" and "~" in the figure? It should be detailed in its caption. 

In the section 2.3, eq.(1) has its "=" inside the norm of gradient of f(x)_hat.  In eq.(2), it is lack of the definition of |f(x)| . In eq.(3), \xi(x) is used to approximate |f(x)|. However, \xi(x) is also undefined.

4. Experiments are not sufficient for evaluation.

In Table. 2, what's the model of "Ours"? How are the differences between "Ours" and other methods?  CycleGAN+resnet_9blocks shows a obvious decrement by comparing to CycleGAN+resnet_6blocks. Is it caused by the overfitting? How many blocks you used in the "Ours". 

Additionally, although the experiments do show some improvements of the proposed method, it is lack of the comparison among the state-of-the-art methods. 

The ablation study shows the proposed method is better than the original cycle GAN. Then, does it works for other unsupervised GAN methods?

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The paper is a report on training a generative network which constructs an infrared image from the corresponding visible image. Using a basic CycleGAN network, it adds three refinements: a residual network in the generating part; incorporating an attention mechanism there; finally adding a gradient normalization at the discriminator part. In Section 3.3 it is checked that all three parts are necessary to achieve the claimed performance figures. To make the paper more available I recommend to enhance Sections 2.3 and 2.4. These sections contain numerous typos and misprints which should be corrected. There are other minor typographical / grammatical errors listed below. With those changes the paper can be accepted for publication.

Line 18: "we propose" ... "is proposed" Use only one proposed

Line 27: add space after the period (.)

Line 33, "method, *the* proposed" (lower case T)

Line 43: "texture detail*s*, as well"

Line 52: "is one of" (add space)

Line 53: "the widely ???? in deep learning" Please add a noun.

Line 54: replace period by comma

Line 58: add space: "idea,* *however"

Line 61: add space: "research,* *in order"

Line 85: "network*.* In the" (replace the semicolon)

Line 88: Replace "Although" by "However," 

Line 88: "enough*. *Arjovsky"  period is at the wrong place

Line 102: "*T*aigman" Capital T

Line 111: "then it is then from" => "then from"

Line 114: "*StarGAN" use capital S

Line 115-116: The sentence with "x" fields should be rewritten. Since x-(x-1)=1, it makes no sense.

Line 116: *S*tarGAN - use capital S

Line 125: "few samples*. T*he method"

Line 197: what formula F(X)=X does mean?

Line 198: in formula (1) the closing norm sign is missing before =

Line 210: The letter E in the formulas (4) and (5) are for Expected value. The explanation following these lines are confusing. Please rewrite.

Line 224: E is not a sample, it denotes expected value. The samples are x and y.

Line 237: In formula (6) the "-x" is missing in the first part.

Line 311: insert space before " observation"

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

After revision, the paper has a better presentation and is more readable. Also one more state-of-the-art method is added for comparison, which is good to show the improvement of the proposed method. However some of my concerns are still not be solved yet.

1. In the introduction, authors claimed that "The gradient-based normalization method proposed in current study focuses on solving the aforementioned problems of easy model collapse and lack of prominent  texture detail information in the generated infrared images to improve the quality of image generation and obtain realistic infrared images with realistic effects.", which means this is their major contribution. However, in their experiments, they did not show any result to compare with eq.(1),(2) and (3). Although it has the comparison between normalized gradient and no normalized gradient, and it achieves a better performance with adding the gradient normalization, the differences among what they proposed in (3) and eq.(1) and (2) are unclear. I would like to see the comparison among them.

2.  The settings of \xi(x) in their experiments is still unclear. What is a universal term or a constant in your experiment?

3. Please separate your proposed method from the related work section into a new section to introduce your method.

4. Although CUT is a state-of-the-art, it is better to simply describe it in your experimental settings. So reader can easily know what CUT is.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

All my remarks have been addressed.

Author Response

Dear reviewer,
    Thank you for your latter and for the reviewers' comments concerning our manuscript entitled"cycle".Those comments are all valuable and very helpful for revising and improving our paper, as well as the important guiding significance to our researches.
    We would like to thank you again for taking the time to review our manuscript.

Author Response File: Author Response.docx