A Robust CNN for Malware Classification against Executable Adversarial Attack

Round 1

Reviewer 1 Report

Minor concerns:

Format: margins need to adjusting; table 1 & 3 ae incomplete.

What does ‘Adversarial attack’ mean?

Dataset:

1-      Malekal website is not working (https://malwaredb.malekal.com). Please put the dataset at another safe place (e.g., Google drive).

2-      Why did you select this dataset? Who used it before?

Adam optimizer: What is it? And why did you choose?

Major concerns:

The author should add three sections or subsections to discuss the following points:

First: secure design principles (SDPs) [1]. Any software-level security solution is expected to employ SDPs that are applied to be a solution to security problems of all system types. Did you implement SDPs or not, at least some of them, if not all? I recommend the author to mention for this point in detail.

Second: ‘Threats to Validity’ should be added, and discuss the three types of such threats: internal validity, construct validity, and external validity

Third:  there is a need to compare your work with the recent ones that used CNN. Would yours outperform theirs? I mean, in terms of performance multi-metrics (i.e., f-score, accuracy, etc.).

[1] Exploring How to Apply Secure Software Design Principles, IEEE Access ( Volume: 10), 2022.

Author Response

Dear Editors,

Thank you so much for your kindness and warm help! Please find the response to the reviewers' comments and the revised paper (in PDF). As only one file is allowed, I merged two files into a single PDF file. 

We highly appreciate the constructive comments by all reviewers. Their comments helped us to enhance our paper in depth. In our opinion, the last three comments on SDP are somewhat difficult because the goal of this study is to disclose the possibility of creating an adversarial malware program that not only succeeds in misleading the target malware detection model but also remains malicious (preserving its original malicious functions). The requirements of SDP are recently popular in the software security domain and also related to malware. However, the research of SDP in the malware domain is our future work and is talked about only in the last section of our paper. We are so pleased that the reviewer's comments are insightful to our study. Therefore, all comments are highly valuable and appreciated by us. 

Thank you so much for your hard work! I wish you a good day!

Sincerely yours,

Dr. Yunchun Zhang

School of Software, Yunnan University, China

Author Response File: Author Response.pdf

Reviewer 2 Report

Authors at work:

1) propose a robust global max pooling-based CNN for detecting and classifying Windows malware;

2) design an executable adversarial attack targeting on the Windows PE header files while the maximum amount of distortions is predefined;

3) propose a header-aware CNN to improve the robustness of the trained malware classifier (based on Euclidean distance-based normalizer).

 IMPORTANT: some tables (Tables 1 and 3) have truncated (invisible) content. This makes the review difficult. Requires mandatory improvement.

Content-wise:

• It should be clearly stated in the text (and maybe even in the Summary) that the presented method applies ONLY to PE files in the Windows operating system

• Figures should be explained more precisely in the text (e.g. Fig. 1 and its elements; Fig. 2 – without reference in the text; …)

• In Fig. 3, the difference between GAP and GMP should be better visible (e.g. using bold and adding abbreviations in the drawing will be sufficient)

• DOI should be added to all possible references.

Editorials:

• There are too many abbreviations defined in the Summary. This should be avoided. If a defined abbreviation is not used again, it must NOT be defined. It only makes sense to define: CNN and GMP. In addition, these and the rest of the abbreviations must be defined (even redefined) in the text the first time they are used.

• Improve the formatting of figures (e.g. Fig. 1)

• The diagrams in Fig. 4 are difficult to read. They should be corrected.

Author Response

Dear Editors,

Thank you so much for your consideration and chances for us! We highly appreciate the reviewer's comments. According to the comments, we revised our paper point-by-point and provided a detailed response in the attachment. Please find the response and the revised paper (with revisions in blue color). If there is any problem, please do not hesitate to let me know. Your help and kindness are special to our growth in academic research. Thanks again!

I wish you a happy day!

Sincerely yours,

Dr. Yunchun Zhang

School of Software, Yunnan University, China.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Regrettably, it appears that several of my previous suggestions remain unaddressed. To expedite the process and ensure clarity, I advise organizing the revisions into three distinct sections or subsections to address the following points:

1. Threats to Validity: every experiment is susceptible to threats to validity, which must be acknowledged and addressed. If there is uncertainty regarding these threats, consulting the following reference would be beneficial for clarification.

Experimentation in Software Engineering. Wohlin C, et al, 2012.

2. Secure Design Principles (SDPs): it is imperative to incorporate secure design principles into your solution. If these principles have not been implemented, it is crucial to explicitly acknowledge this deficiency. Despite your assertion of their inclusion in the previous section, such mention is not true (just claim that the authors added but in reality you thy did not). This oversight reflects poorly on the quality of the work. For more info about SDPs, the authors can consult the following reference:

Exploring How to Apply Secure Software Design Principles, IEEE Access ( Volume: 10), 2022.

3. Brief Comparison with Existing Works: a concise comparison between your work and existing literature is necessary. This comparison should highlight the distinguishing features of your solution, especially in contrast to prior works. While you may choose to summarize your earlier work from 2023 here, it is important to ensure clarity and brevity in the comparison.

Furthermore, it is noted that the URL provided for the dataset (https://www.malekal.com) remains inaccessible. I encountered an error when attempting to access the site, as illustrated in the attached screenshot.

Author Response

Thank you so much for these constructive comments! All comments are highly appreciated by us. We revised our paper and gave point-by-point responses. Please find the revised paper, together with responses, in the attachment. 

Author Response File: Author Response.pdf

Reviewer 2 Report

I would like to thank the authors for their corrections.

Author Response

Thank you so much for reviewing our paper and provide us with so many constructive comments! Please find our response and revised paper in the attachment. I wish you have a good day!

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

‘Threats to validity’ is not related to software engineering as the authors said in the first para in Section 5.5; it is related to the experimental research instead.

Anyway, to save time, please do the following in this section:

-       Replace the title of Section 5.5 with “Limitations and Secure Design Principles”.

-       remove the first paragraph completely, “Since … follows.”

Author Response

Thank you so much for your comments! We revised our paper accordingly and  please find the revised paper in the attachment. Thank you so much  again!

Author Response File: Author Response.pdf