On deceiving malware classification with section injection

We investigate how to modify executable files to deceive malware classification systems. This work's main contribution is a methodology to inject bytes across a malware file randomly and use it both as an attack to decrease classification accuracy but also as a defensive method, augmenting the data available for training. It respects the operating system file format to make sure the malware will still execute after our injection and will not change its behavior. We reproduced five state-of-the-art malware classification approaches to evaluate our injection scheme: one based on GIST+KNN, three CNN variations and one Gated CNN. We performed our experiments on a public dataset with 9,339 malware samples from 25 different families. Our results show that a mere increase of 7% in the malware size causes an accuracy drop between 25% and 40% for malware family classification. They show that a automatic malware classification system may not be as trustworthy as initially reported in the literature. We also evaluate using modified malwares alongside the original ones to increase networks robustness against mentioned attacks. Results show that a combination of reordering malware sections and injecting random data can improve overall performance of the classification. Code available at https://github.com/adeilsonsilva/malware-injection.


INTRODUCTION
Malware -a short term for malicious software -is described by Sikorski et al. [37] by their action: Any software that does something that causes harm to a user, computer, or network can be considered malware [. . .] These applications, purposedly built with intentions of reading, copying, or modifying information from computer systems -often without user consent -pose a high threat for modern information systems [24,28,39].The early detection of such malware is vital to minimize their effects on an organization, or even among regular users.
In this work we discuss strategies related to the classification (i.e. which kind of malware is it?) of malware samples using only their raw bytes as inputs to machine learning algorithms.These strategies can be seen as part of the static analysis of samples, a very important stage in a malware detection pipeline, in which is necessary to provide the classification without executing the file being analyzed.It is important to stress that these methodologies are not to be used as the solely strategy to detect malware samples, but as the first one in a multi-step chain of procedures.Despite that, due to their fast execution times and lack of human interaction, they are still an integral part of such a pipeline [10,27].
We present here a simple way to modify a software file to deceive systems built to classify malware examples into families.Our method builds upon the idea of injecting bytes into the executable file [3].We seek to insert bytes in different parts of a malware.By doing so, we aim to deceive malware classifiers and preserve the original functionality while hindering the detection of injected data.To accomplish that, we create rules of injection that respect the file format of the operating system the malware will infect.We can not only define how many bytes we inject but also how they spread over the file.More importantly, we explore two approaches: Random injection: inserting random bytes, so that we do not require any knowledge about the systems to be deceived Adversarial injection: inserting bytes taken from families different from the sample being evaluated The classification approaches evaluated in this work are based on methods that learn straight from the raw bytes of the file, ranging from methodologies that reinterpret the sample as an grayscale image up to preprocessing each sample as a 1D vector in their execution [5,6,9,10,17,19,23,29,33,38,42].We want to evaluate the vulnerability of these variants to the already known adversarial examples [16], an approach with increasing popularity in the literature, especially in the context of malware [2,3,6,13,14,17,19,26].There are some limitations that must be observed, tough, since the pertubations added to malware samples must be drawn from a discrete domain.It differs from other types of data, such as images.Also, executable files have strict standards, which means byte ordering is relevant in some parts of the file.As mentioned earlier, we tried to conform our manipulations to the expected stantards in order to preserve the functionality of the malware samples.
The rest of this paper is presented as follows: in Section 3 we compare our methodologies to other present in the literature.In Section 4 we present how we generate and add data between sections of a PE file.Section 5 discusses the machine learning algorithms evaluated in this paper for malware classification.In Section 6 we discuss our evaluation strategies and their results, finishing with our conclusions in Section 7.
Our contributions can be summarized as follows: (1) We provide a framework to inject data into PE files that leverages all the alignments required to preserve its functionality.It can inject any sort of data (either random or from a different file) in multiple positions of the file, not only at the end (padding).(2) We evaluate how different deep neural networks architectures proposed for malware classification behave in multiclass classification scenarios.We want to assess the difficulties behind separating a given sample from other samples from the same kind.
(3) We evaluate how the aforementioned networks behave when dealing with injected samples.Our goal here is to assess how our attacks impact the classification of these networks, in regards both of the location and also the amount of injected data.

BACKGROUND
Modern operating systems, such as Linux and Microsoft Windows, use the concept of sections to read an executable file, load it to the memory, and run its instructions.It is necessary to know and follow the file format specifications to be able to insert data in different parts of an executable and preserve its functionality.Since there are differences between the files accepted by each operating system, designing a system-agnostic injection scheme is impracticable.For this reason, we focus on the Microsoft Windows' PE32 format, as it is the only one included in publicly available malware datasets [29,35].
As shown in Figure 1, PE32 sections provide information about the executable, such as its instructions (".text"), its variables(".data"),and resources it uses (".rsrc").Following this layout, our strategy consists of injecting non-executable sections like ".data" to the file.This way, the set of instructions does not change, and the only way to decide whether an injected section is in use or not is through execution.

RELATED WORKS
We start by presenting previous works which performed malware classification.We also discuss other methodologies for malware perturbation and what authors have done to minify these effects.

Malware classification
Nataraj et al. [29] presented a method to transform software into images and classify them according to their malware family.In this context, a family is a set of software files with high similarity of instructions and behavior.Follow up works explored this idea using different feature extraction (e.g., GIST, Local Binary Patterns, Scale-Invariant Feature Transform) and classification (e.g., Support Vector Machines, K-Nearest Neighbours (KNN)) methods [1,25].
The growth in Deep Learning research led to the exploration of neural networks for malware classification.Recent works applied different architectures for this task, either by extracting features from the file (e.g., system calls, imported libraries, functions in use) [4,5,31,36] or by using the raw bytes from the data as input [18,33,34].Some of them achieve high classification accuracy by training Convolutional Neural Networks (CNNs) from scratch [25,38], or by using prior knowledge from a CNN pretrained on a large dataset [9,10,42] like ImageNet [15].Malware detection was also exploited in the form of a binary classification by considering all malware files as one class and samples of benign software as the other one [9,33].These networks, however, are vulnerable to adversarial attacks.In this work we explore different different architectures -KNN+GIST as proposed by Nataraj et.al.
It is worth mentioning that there are not many relevant public datasets for training malware classifiers, which makes comparing different works a more subtle task.Malimg [29], BIG 2015 [35] and EMBER [4] are the most notable ones.Since our injection method require reading the file header, BIG 2015 [35] dataset is not possible because the samples have their headers stripped.EMBER [4], on the other hand, does not provide raw byte values straight away.Since they provide SHA-256 values taken from file contents, a reverse search in malware indexing services is needed in order to retrieve their raw bytes.In Table 1 we aggregate state of the art methods for malware detection and classification by their technique and the dataset it used.Multilayer SIFT malimg [29], BIG 2015 [35] Agarap e Pepito (2018) [1] GRU + SVM malimg [29] Le (2018) [23] CNN-BiLSTM BIG 2015 [35] Chen (2018) [9] Inception-V1 [40] malimg [29], BIG 2015 [35] Chen (2020) [10] Inception-V1 [40] Private

Malware injection
Adversarial attacks consist of adding tiny changes to the input data to alter its classification result and are usually not easily perceived by humans.But, arbitrarily modifying software files without changing its behavior is impossible.Even verifying if a modification does not affect a software's response is an undecidable problem.Thus, if someone arbitrarily alters a malware to change its classification results, there is a chance it will no longer pose a threat to the system.Despite that fact, there exists in literature some possible attacks that retain their functionalities.They are illustrated in Figure 2. Different works exploited adversarial attacks in the malware domain.Grosse et al. [17] and Al-Dujaili et al. [2] extracted static features from malware files and used the Fast Gradient Sign Method (FGSM) [16] to modify these feature vectors and form adversarial samples.Notwithstanding, these approaches do not guarantee that it is possible to alter the malware file to produce the adversarial feature vector while maintaining the original functionality.Therefore, they may not have a practical use.
Anderson et al. [3] explores a black box attack against a reinforcement learning model, where the agent actions are taken from a list of modifications that includes manipulating existing bytes but also adding ones between sections or even creating new sections.No further information is provided regarding the constraints on these injections.It fits "Section Injection" and "Content Manipulation" categories illustrated in Figure 2. It achieves evasion rates up to 16% against a Gradient Boost Decision Tree (GBDT ) [4] model.
Khormali et al. [19] focused on injecting bytes to the executable files' end, an unreachable area during execution.It fits "File Padding" category illustrated in Figure 2. As the operating system will not execute it, not even read it in some cases, it does not affect the malware behavior.These bytes can either be generated by FGSM or be parts of other malware.Nevertheless, extra bytes at the end of the file may be easy to detect and discard before the classification.Besides, this approach requires access to the model or training data used by the classification system, which may not be available in a real attacking scenario.
Demetrio et al. [13] proposes a black-box attack called GAMMA (Genetic Adversarial Machine learning Malware Attack), a method that queries a given malware classifier and, based on the output, draws from a set of functionality-preserving manipulations that changes malware samples iteratively.GAMMA is evaluated against two malware classifiers, Malconv [33] -a shallow neural networkand GBDT [4].Its proposed methods fit all categories illustrated in Figure2, despite not detailing how some of those are achieved.
Lucas et al. [26] also employ functionality preserving techniques.They extend binary rewriting techniques such as in-place randomization (IPR) [30] -where the binary is disassembled and some of its instructions are rewritten -and code displacement (Disp) [21] where the disassembled version is also used, but with the intent of moving instructions between sections, fitting into "Content Manipulation" category illustrated in Figure 2.They apply these attacks in an interactive manner and evaluate them against three neural networks, achieving a misclassification rate of over 80% in some scenarios.
Benkraouda et al. [6] proposes a framework that mixes a mask generator to highlight the bytes that are possible to manipulate while retaining executability, adversarial example generation using Carlini-Wagner (CW) attack [7] and an optimization step that iteratively modifies the masked bytes by comparing the generated adversarial data to a set of known instructions.It fits the "Content Manipulation" category illustrated in Figure2.The attack is evaluated against a three-layer CNN, achieving an attack success rate of up to 81.8%.A shortcoming of this method is the time it takes to generate its samples, reaching over six hours for a single sample in some cases.
Demetrio et al. [14] introduce the RAMEN framework, an extensive library with multiple attacks for malware classification.They present three novel attacks -Full DOS, Extend and Shift -all of them capable of modifying the binary sample while keeping its functionality.The novel attacks are evaluated against MalConv [33], DNN with Linear (DNN-Lin) and ReLU (DNN-ReLU) [12] and GBDT [4], being misclassified by the neural networks, but not being able to evade the decision tree since it does not rely only on static data.
Our attack scheme -Section Injection -is also explored by Anderson et al. [3] and Demetrio et al. [13], as one possible method in their pipelines, but no further information is provided regarding the constraints for this injection.It can also be seen as an ensemble of Extend and Shift methods proposed by Demetrio et al. [14] and the padding methods discussed by Khormali et al. [19].The byte modifications presented by Lucas et al. [26] can also be integrated in our method, leading to the injection of perturbed sections instead of random ones.
Regarding the data used to evaluate the attacks, most works listed here used some sort of private dataset, either by collecting samples from malware hosting services or expanding public ones -Benkraouda et al. [6] merged malimg [29] and benign samples from Architecture Object Code Dataset (AOCD) [11], Khormali et al. [19] used BIG 2015 [35] and also formed a private IoT dataset.A summary of the functionality preserving attacks can be found in Table 2.

DATA INJECTION
To comply with a realistic usage scenario, we inject one or more sections filled with arbitrary bytes before any processing being done for classification purposes, as illustrated in Figure 3.We explain how the proposed injection process works in the following sections, and we show how we built the malware classifiers used in our experiments in Section 5.

File header
The first step in our injection scheme is to obtain information about the input file by reading its header.Table 3 lists the flags that are relevant to us.After inserting a new section, we need to increment the flag    and update the flag    accordingly to preserve the malware functionality.We pick the injected section's index  by drawing a number in the interval [0,   ].Sections 0 to  − 1 remain in place, and sections  to    − 1 are shifted one position forward so that we can insert the new section in -th place.

Section header
A section header is composed of 40 contiguous bytes.These bytes specify what the loader needs to handle this section.Table 2 shows the bytes that we fill when creating a new section.We refer to a flag of the -th section as   .First, we generate eight random printable characters (ASCII table values between 33 and 126) as    .After that, we set       using Equation 1: with  being the number of bytes we want to add.We always set  as a multiple of  so that null padding is unnecessary. is usually 512 bytes, but it varies according to compilation options.  is set as in Equation 2, if the -th section is the last one.Or else as in Equation 3, and we add     to   , ∀ > .
On memory, we inject sections after every other section to avoid having to update instructions that use memory offsets and preserve the execution path.We set    using Equation 4: where  is the index of the last section on memory.This way, we correctly align the injected section according to .   is set to 0, as we do not want to take memory space.Thus, multiple runs of this injection process produce sections pointing to the same address.In our tests, this does not affect execution.We finish our header by setting ℎ  as a read-only section with initialized data.

Injected data
In our work, injected data is a sequence of random bytes.As we have control of the section structure, we could insert pieces from other executables or adversarial examples created using FGSM as other works in the literature [19].However, we do not do that because we assume we have no access to models and training data used by malware classifiers.Besides, our results show that our simple strategy is enough to affect the performance of a state-of-the-art malware classification approach substantially.

Workarounds
We found some challenges when applying this method to an arbitrary PE32 file.Instead of constraining the input files, we dealt with the problems as they appeared.Some malware instances, usually packed or obfuscated, have multiple contiguous virtual sections that do not exist on disk, only on memory.For those cases, we had to adjust the  in injected data to make sure it points to a valid physical section.Furthermore, malware sections are not always correctly aligned with the  flag.To avoid fixing existing sections, we only inject data before correctly aligned ones.

MALWARE CLASSIFICATION
As can be seen in Figure 3, this process is divided into two parts: image generation and classification.The former is described in Section 5.1.The latter is carried out with various approaches: (1) GIST+KNN [29], which holds state-of-the-art performance for handcrafted methods (2) Le-CNN, Le-CNN-LSTM, Le-CNN-BiLSTM [23], three similar models that uses resizing of the input data to a fixed number of bytes (3) Malconv [33], a model that truncates the first 1MB and perform classification with it.The trained models are respectively described in Sections 5.2.1 and 5.2.2.

Image generation
We transform an executable into an image following Chen's adaptation [9] of Nataraj et al.'s specifications [29].We treat every byte as a grayscale pixel, and we break the file into image rows by using a fixed width, which is set according to the file size (see Table 5).We discard the last row if it is incomplete.The result is illustrated in Figure 1.[29] to the best of our abilities.To do so, we resize our images to 64 × 64 pixels, extract 320-dimensional GIST descriptors, and then classify it using KNN with  = 3.
In this work we chose different CNN strategies to understand how they perform against data injection: (1) Le et al [23] presents three models.A simple model with three 1D-CNN layers before a fully connected layer, refered to as Le-CNN.A second model with a LSTM layer before the fully connected one, refered to as Le-CNN-LSTM.A third model with a bidirectional LSTM before the fully connected layer, referered to as Le-CNN-BiLSTM.For all of them we employ the same input size of 10k bytes, a batch size of 512, and train the model for at most 60 epochs (early stopping if the accuracy does not improve for 10 epochs).(2) Raff et al [33] presents the model refered to as Malconv.
This model employs a gated convolution network, i.e., an embedding layer followed by two separate 1D-CNN layers that are multiplied and passed on for two fully connected layers.For this model we use training protocol similar to Lucas et al [13]; input size of 1MB, training for a total of 10 epochs without early stopping, with a batch size of 16 due to memory constraints.

Injection Attacks with Random Data
To evaluate malware classification before and after code injection, we use malimg [29] dataset.It has 9,339 malware samples from 25 families.We randomly split the dataset into three parts: training (80%), validation (10%), and test (10%).We use the training and validation sets to perform the CNN training and combine them as a single gallery for the KNN search.
For testing, we insert  new sections with  ×  bytes at random parts of each test malware, with and  varying from 1 to 5, totaling 25 different injection scenarios.We repeat training/testing experiments three times for each model and show average results in Figure 4.
We can see that the way we inject multiple sections affects the results.For instance, despite the amount of data being the same, four sections with  bytes impact more the performance than two sections with 2 ×  or one section with 4 ×  bytes.Thus, dividing a portion of data in more parts and spreading them over the file is more effective in deceiving the classifier than having a few large sections.
The biggest drop occurred when we injected five sections with 5 ×  bytes.As most samples have  = 512 and the average malware size is 177kB, our injection approach accounts for an approximate 7% increase in file size and misclassification rates ranging between 25% and 40%.Figure 5 illustrates the misclassification differences between the test set with original samples and a set with injected samples.
It is worth noting that some of these families share similar traits.For instance, families Autorun.K, Malex.gen!J, Rbot!gen, VB.AT and Yuner.A are all packed using UPX packer.Some families are variants of the same kind of malware, such as C2Lop.P and C2Lop.gen!g, Swizzor.gen!I and Swizzor.gen!E.It is expected that confusion concentrates around those variants [29].
We can see that all models suffer to correctly classify these variants, even before data injection.Le-CNN-BiLSTM model, as seen in Figures 5c and 5d does not learn how to correctly identify samples from a packed family, e.g."Autorun.K".One behavior is clear in KNN and MalConv models: their tendencies to misclassify samples as "Autorun.K", "C2LOP.gen!g" and "C2LOP.P".Those families share the samples with higher average size in bytes, at 513k, 383k and 523k respectively.Since Le-CNN-BiLSTM resizes everything to 10k bytes, this error is less prevalent with this model.In the same manner, MalConv has these classes as the ones with less misclassifications in the injected set.Considering its 1MB input, those are the samples where padding is used the least.
In Figure 6 we can see how the trained models lose precision after section injection.Due to the imbalanced nature of the dataset, this is illustrated by Precision-Recall curves.
We can see that the handcrafted method is the least precise in this scenario, being followed by Le-CNN, MalConv, Le-CNN-LSTM and Le-CNN-BiLSTM respectively.

Injection Attacks with Adversarial Data
What if instead of adding random data we use bytes that appear in samples from other classes?We evaluate this kind of attack in this section, this time focusing on the most impactful injection scenario, i.e., 5 sections with 5 × . Figure 6c displays the difference that injecting with adversarial data imposes.
Comparing with random injection results seen in Figure 6b, we can see that all models had its average precision decreased -KNN+GIST by 25.44%, Le-CNN by 15.75%, Le-CNN-LSTM by 15.91%, Le-CNN-BiLSTM by 11.56% and MalConv by 5.62%.That might be an indication that MalConv is learning more discriminative features from the samples and it is deceived for reasons other than the kind of data being injected, since it becomes the model with highest average precision despite losing more accuracy than Le-CNN-BiLSTM (Figure 4).

Evaluating Samples without Header
Here we evaluate the possibility of training our models striping the header of the samples, similarly to what is employed in BIG 2015 [35].Figure 7 illustrates the results for samples without the header.
All models rely heavily on the samples header in order to perform classification, losing precision even before data injection as seen in 7a.Only Le-CNN-BiLSTM increased its precision by 0.0026 in this scenario.All models became less robust to data injection, losing precision significantly when compared to complete executables in 6b.Despite that, MalConv is the only model with similar average precision to previous scenarios.

Defending Against Data Injection
Multiple strategies were evaluated in order to make these models more robust against data injection, by focusing on the kind of data available during training.

Augmentation.
A solution proposed in the literature [8,32,41] is to augment the data used for training.Three strategies were initially evaluated: (1) Section reordering: since our injection scheme adds new sections in a random position among the existing one, the first augmentation idea was to reorder the sections on the training section.By doing this we wanted to check if the model could be more robust against data injection without  8h, 8i, was also evaluated.Comparing with the original results in Figure 6 we can see that this may be a good defense strategy as well.
As shown by Figure 8, some models were improved by these augmentation strategies, even tough still vulnerable.

Binary Data.
All experiments mentioned here were also performed in a binary dataset.We collected samples from a clean Windows 10 Virtual Machine.In this dataset the models were barely affected by data injection.We believe something similar to the mentioned by Raff et al [33] also happened in our dataset: models were learning "Microsoft vs Non-Microsoft" instead of "Benign vs Malign", as shown by Figure 9.For these tests we evaluated the models performance on the original test set and against malware-only versions of the dataset injected with both random and adversarial data.
6.4.3Scaling models.Some of the challenges involved in building more robust models are closely related to the available data -highly imbalanced number of samples, sample size variation intra and extra families, packing and obfuscation of samples -but those are not the only concerns.Increasing architecture size does not necessarily lead to more robust models, one example being that finetuning a large model like Inception-V1 [40] as done by Chen et al [9,10] is also vulnerable to section injection, as seen in a preliminary comparison against KNN+GIST, illustrated in Figure 10.Current results point in the direction of combining text processing techniques with convolutional layers, as made by MalConv [33]    with its Embedding layer and Le-CNN-BiLSTM [23] with the recurrent layer after convolutional ones.An open challenge regarding these approaches is related to their input sizes: MalConv truncates data larger than a given size, which requires choosing between discarding relevant data and using more computational resources to process larger samples; CNN-BiLSTM interpolates its input to a fixed size, possibly removing relevant byte relationships in some regions of the file.

CONCLUSIONS AND FUTURE WORK
In this work we proposed a new method to inject data into malware files in order to change its classification when analyzed by and automatic malware classification system.With a mere 7% file size increase, we dropped the accuracy of five classifiers on par with the state-of-the-art -namely GIST+KNN [29], MalConv [33] and Le-CNN [23] and two other variations -between 25% and 40%.The obtained results seems promising and we think this method can be improved to be robust enough for a larger scale of scenarios.There are some points researchers using this method need to be aware of: • The usage of CNNs is gaining momentum in this research field literature [9,10,19,23,38,42].This work shows a simple technique that is able to make the accuracy in such CNNs drop in almost 50% by adding small perturbations to a malware file.We could observe that methods such as Gated CNN [33] or combining CNN with LSTM [23] can be more robust against the data injection presented here.• A deeper understanding of how the operating system loads executable files to memory usually helps malware creators.
During preliminary tests we were able to see that some file format rules are flexible and malware authors do not follow all of them.It includes files with section headers missing or some sections not aligned to the required flags.We tried our best to keep our generated examples in accordance to the format specified.Malware creators might not have this mentality, so that should be considered when building neural networks with the purpose of detecting malware files that rely on static features from the file.• Our results show that data dispersion might be just as important as the amount of data being injected.We can use this idea to conduct a more directed attack using our method together with the method proposed by Khormali et al. (2019) [19], injecting FSGM generated sections in any position of the file.
As mentioned in Section 6.4, augmenting the training set with injected samples might not be enough to prevent section injection attacks, nor only increasing architecture size.Further investigation is required on how to transform the input for the models in a way that only relevant data for the classification is kept.Current experiments point in the direction that instead of relying on a fixed preprocessing method -like truncating or interpolating -more dynamic approaches should be investigated, such as Attention based methods.

Figure 1 :
Figure 1: Illustration of the sections of a PE32 executable file.

Figure 2 :
Figure 2: Illustration of the differences among attacks to image-based malware classifiers.Leftmost (red) square displays our approach.Blue squares displays previous attacks.

Figure 3 :
Figure 3: Flowchart of an image-based malware classification system (blue lines).Red lines replace the dashed blue line in our data injection scheme.

Figure 4 :
Figure 4: Average accuracy of malware classification under different injection scenarios.Different colors represent the amount of injected sections.

Figure 6 :
Figure 6: Precision Recall curves in the original test set 6a and when 5 sections of 5 ×  bytes are injected, 6b with random bytes and adversarial bytes in 6c.Each color represents a different model.

Figure 7 :
Figure 7: Precision Recall curves in the original test set 7a and 7b when 5 sections of 5 ×  bytes are injected, both versions without file header.

Figure 8 :
Figure 8: Precision Recall curves for tests with augmentation.8a, 8d and 8g displays results on test sets with original samples.8b, 8e and 8h displays results for datasets injected with random data.8c, 8f and 8i display results for injection with adversarial data.5 sections of 5 ×  bytes are injected in all cases.Each color represents a different model.
(a) Results on unmodified binary dataset (b) Results on random injected malware (c) Results on adversarial-injected malware

Figure 9 :
Figure 9: Precision Recall curves when 5 sections of 5 ×  bytes are injected, 9b with random bytes and adversarial bytes in 9c.Each color represents a different model.

Figure 10 :
Figure 10: Average accuracy of malware classification under different injection scenarios.Solid lines show results for Inception, and dashed lines for GIST+KNN.Different colors represent the amount of injected sections.

Table 1 :
Summary of different malware classification techniques.

Table 2 :
Summary of functionality preserving attacks against PE32 malware classification.

Table 3 :
Flags in the header of PE32 files.

Table 4 :
Flags in section headers of PE32 files.