MalCaps: A Capsule Network Based Model for the Malware Classification

Round 1

Reviewer 1 Report

The paper proposes a malware detection method based on capsule network architecture with hyper-parameter optimized convolutional layers. The proposed method aims to overcome the limitations of convolutional neural network. The proposed method is evaluated on the Microsoft Malware Classification Challenge dataset with good results.

Comments:

1. A reference for image given in Figure 1 is required.
2. Since this is not a first paper, which proposed using capsule networks for malware detection (see, for example, Wang et al., “A Novel Malware Detection and Classification Method Based on Capsule Network”) you should clearly define the novelty of your approach and its differences from other authors’ work.
3. An overview of related works needs to discuss other deep learning and convolutional neural network based approaches on malware recognition such as “Detection of malware on the internet of things and its applications depends on long short-term memory network”, “Hybrid malware classification method using segmentation-based fractal texture analysis and deep convolution neural network features”, and “Ensemble‐based classification using neural networks and machine learning models for windows pe malware detection”.
4. Present a detailed representation of the proposed method as a block (or flow) diagram.
5. I do not think that the method for transforming malware byte files into images, which you have described in section 3.2, is novel. Other authors use the same or very similar methods to transform malware into images as well.
6. Figure 4: mathematical formulas at the bottom part of the figure are not clear. Can you move them to the main text of the paper?
7. For evaluation, you should also use the AUC (Area Under Curve) metric, which works better for the imbalanced datasets.
8. Why you have decided to stop the training of Le-net5 network at 25 epochs, and the MalCaps model at 50 epochs? Explain.
9. Add more works, which reported the results on the same dataset, to Table 5. For example, Narayan in “Ensemble Malware Classification System Using Deep Neural Networks” (2020), reported better results than you have achieved on the same dataset while using the ensemble approach.
10. Present the confusion matrix of your classification results. Discuss most typical misclassifications.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

 In this paper, we draw on the idea of image classification in the field of computer vision and propose a novel malware detection method based on capsule network architecture with hyper-parameter optimized convolutional layers(MalCaps), which overcomes CNNs limitations by removing the need for a pooling layer and introduces capsule layers.

The study is interesting and motivations are good. However, the paper is not well structured; the methodology section is too short and not clear.

The literature review section presents useful cited papers.

I am confident if the authors resubmit the paper by reviewing the methodology, the document would acquire more quality.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Traditional deep learning architectures, such as CNNs, do not take into account spatial hierarchies between functions. The concept presented is based on a grayscale image, resulting from malware transformation. Then, a capsule network based on dynamic routing is used to detect and classify the image. Thus the value of the article lies mainly in proposing a new method, alternative to CNN architecture, and showing the effectiveness of this method based on the MMCC dataset.

Explanation of the malware detection approaches is not essential to the content of this article, as several good reviews of the methods and approaches are available. This section of the article could be limited without sacrificing content and it is enough to refer to one of the available overviews, eg [3].

The authors introduce deep learning in malware detection and describe the methods based on image classification. They introduce the Capsule Network-Based Model used for the image classification area. They adopted the model for malware detection as their own innovative approach. However, it is not precise.

Firstly, there are several approaches using image classification in malware detection that should be shortly referred to. For example detection of malicious code variants using CNN and greyscale images were presented [2]. In the work, the malicious code was converted into a visual grayscale image. The images were identified and classified using a convolutional neural network (CNN) that could extract the features of the malware images automatically.

Secondly, the authors themselves considered such an application already in 2019 [1]. In this approach, the malware was transformed into a color image the based capsule network was used to detect and classify the color image. The authors indicate that experimental results showed a higher level of detection accuracy than SVM and classical CNN. Authors should refer to this article and previous effort and accurately describe the differences and progress.

Moreover, the discussion of the results [line 469] indicates that "One approach is to process the images in color, rather than greyscale, which allows feature details to be preserved that do not available in purely a greyscale image represents the malware byte files". Referring to [1], such an approach (classification of color images) has probably been verified by them in previous works.

For this reason, the Overall Merit of this article was rated low as it was not precisely indicated whether the article actually describes the new results. If the authors have worked on similar solutions, this should be described in detail. If there are other works using a similar approach, this should also be described.

Minor issues:
Most of the equations are in low resolution. It should be repleaded with vectorized representation.

---------------
[1] Wang S., Zhou G., Lu J., Zhang F. (2019) A Novel Malware Detection and Classification Method Based on Capsule Network. In: Sun X., Pan Z., Bertino E. (eds) Artificial Intelligence and Security. ICAIS 2019. Lecture Notes in Computer Science, vol 11632. Springer, Cham. https://doi.org/10.1007/978-3-030-24274-9_52
[2] W. Li, R. Zhang, and Q. Wen, "A malicious code variants detection method based on self-attention" inProc.20206thInternationalConferenceonComputerandTechnologyApplications, ser. ICCTA '20. New York, NY, USA: Association for Computing Machinery, 2020, pp. 51-56. [Online]. Available: https://doi.org/10.1145/3397125.3397145
[3] Tight Arms Race: Overview of Current Malware Threats and Trends in Their Detection. Available from: https://www.researchgate.net/publication/348091850_Tight_Arms_Race_Overview_of_Current_Malware_Threats_and_Trends_in_Their_Detection [accessed Apr 26 2021].

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors have addressed all my comments and improved the paper. I have no further comments and recommend the paper to be accepted.

Reviewer 2 Report

In this paper, we draw on the idea of image classification in the field of computer vision and propose a novel malware detection method based on capsule network architecture with hyper-parameter optimized convolutional layers(MalCaps), which overcomes CNNs limitations by removing the need for a pooling layer and introduces capsule layers.
The study is interesting and motivations are good. However, the paper is not well structured; the methodology section is too short and not clear. 
The literature review section presents useful cited papers.

I am confident if the authors resubmit the paper by reviewing the methodology, the document would acquire more quality. 

Reviewer 3 Report

The reviver's comments were satisfactorily addressed. The presented content is an improvement of the methods used so far and is one more step in the research process in the area.