Image-Based Approach to Intrusion Detection in Cyber-Physical Objects

Abstract

Recently, approaches based on the transformation of tabular data into images have gained a lot of scientific attention. This is explained by the fact that convolutional neural networks (CNNs) have shown good results in computer vision and other image-based classification tasks. Transformation of features without spatial relations to images allows the application of deep neural networks to a wide range of analysis tasks. This paper analyzes existing approaches to feature transformation based on the conversion of the features of network traffic into images and discusses their advantages and disadvantages. The authors also propose an approach to the transformation of raw network packets into images and analyze its efficiency in the task of network attack detection in a cyber-physical object, including its robustness to novel and unseen attacks.

1. Introduction

The wide application of deep neural networks is explained by their ability to reveal hidden non-linear relations between the analyzed attributes. They have turned out to be extremely efficient in such tasks as computer vision, and speech recognition, i.e., tasks where parameters have explicit spatial relations. Currently, there are many studies devoted to the problem of transformation of tabular data without such relations into images in order to apply deep neural models and take advantage of pre-trained models [1,2,3].

The input data in network intrusion detection tasks are usually represented by a vector of numerical values. There are only a few approaches that use image-based features to detect network attacks [4,5,6], and all of them use the transformation of tabular data, such as statistics on network flows extracted from PCAP (Packet Capture) files. To the best of the authors’ knowledge, there are no approaches that are based on direct conversion of raw binary packets into images [7], though when used as a data preprocessing step, this transformation has proved its efficiency in cyber security tasks, such as malware detection and authorship attribution [8,9,10,11]. Moreover, Alrabaee et al. [8] and Kaur et al. [9] demonstrated that analysis models trained on such images are able to detect the authors of a code, even if different techniques that could mask authorship, such as code obfuscation or compiler optimization settings are applied. Rong et al. [10] used the conversion of raw session network traffic into fixed-size RGB images as a preprocessing step, for training a deep transfer learning model on the basis of ResNet-50 [12] in order to detect unseen malware samples.

These results motivated the authors to analyze the conversion of raw network packets into images as a data preprocessing step in the process of network attack detection, and to evaluate the performance of deep neural networks that are trained on this type of dataset. The results of the research were first presented at the 15th International Symposium on Intelligent Distributed Computing (IDC 2022), where the authors introduced the approach and primary experimental results [13]. This paper provides an extended analysis of related research and a detailed description of the approach, including the key challenges that should be addressed when constructing images from raw network packets. It also presents an evaluation methodology for assessing the robustness of the trained classifier to unseen attacks.

Thus, the authors’ contributions are as follows:

• An approach to the preprocessing of the network data in PCAP format into images;
• An evaluation of the impacts of different settings of the image generation procedure on the efficiency of the analysis models, namely, ResNet34 and MobileNetV3-small, that are used in the feature extraction mode, as well as the CNN constructed by the authors.

The novelty of this research consists in a novel approach to data preprocessing that is based on a transformation of raw packets into images, which allows the achievement of high accuracy in attack detection.

This paper is structured as follows. Section 2 reviews approaches to intrusion detection, including ones that use the transformation of data into images. Section 3 describes the proposed approach to attack detection with a particular focus on the image generation step. Section 4 details the methodology of the experiments and discusses the obtained results. Section 5 summarizes the obtained results and defines the direction of future work.

2. Related Work

Cyber intrusion detection is a highly relevant area today, and it continues to develop. Over recent years, researchers have studied and proposed various approaches to the detection of known types of cyber attacks; these approaches have ranged from knowledge-based methods (signature-based methods, scenario description languages, finite-state machines, Petri nets, expert systems, and model checking [14]) to machine-learning-based methods (decision trees [15], support vector machines [16], Bayesian networks [17], Bayesian methods [18], multivariate adaptive regression (MAR) splines [19], clustering algorithms [20], and regression algorithms [21]) and other advanced computational intelligence methods (neural networks [22,23], genetic algorithms [24], fuzzy logic [25], immune systems [26], and swarm intelligence [7]). For the detection of anomalies (unknown types of cyber intrusions), researchers have also proposed numerous techniques based on wavelet analysis [27], statistical analysis [28], entropy analysis [29], spectral analysis, fractal analysis [30], and cluster analysis [7,31].

The proposed methods have demonstrated good results for the detection of known types of cyber intrusions, but are still limited in the detection of unknown types of attacks (anomalies). Anomaly detection methods use input data, such as network traffic or event logs, for feature extraction, and they construct the normal behavior profile on that basis. This profile is then compared with new activity profiles to detect anomalies. New approaches to feature extraction have been proposed as some of possible ways to overcome limitations in the detection of unknown types of intrusions. Thus, the transformation of features into images as a feature extraction procedure has demonstrated promising results in various object detection tasks in other areas, such as genetics.

In [3], the authors proposed an algorithm called DeepInsight, which converts various non-image data into images, and they used this to train a CNN to differentiate among phenotypes or categories. They tested different kinds of non-image datasets, including RNA-seq, vowel, text, and artificial datasets and they obtained promising results.

In [32], the authors tested their approach in drug sensitivity prediction scenarios by using synthetic and pharmacological datasets. They researched the limitations of CNNs in relation to predictive modeling. The authors proposed a novel feature representation approach called REFINED (representation of features as images with neighborhood dependencies). The idea consisted of the transformation of high-dimensional vectors into images, which were then used for CNN deep learning. The peculiarity of this approach was its use of embedded feature extraction. The authors generated a concise feature map in the form of a two-dimensional image using a Bayesian metric multi-dimensional scaling approach in order to minimize the pairwise distance values and, thus, to consider the similarities between features. The experiments demonstrated that obtained REFINED CNN outperforms such commonly used approaches as artificial neural networks, random forests, support vector machines, elastic nets, and linear regressions, and such state-of-the-art methods as Deep–Resp–Forest [33] and heterogeneous graph networks [34] on a synthetic dataset, NCI60 drug response dataset [35], and heterogeneous Genomics of Drug Sensitivity in Cancer (GDSC) dataset [36] in terms of predictive accuracy, statistical significance, and robustness.

In [37], the authors transform tabular data into images to predict anti-cancer drug response. Their algorithm, image generator for tabular data (IGTD), assigns features to pixel positions to locate similar features close to each other in the image minimizing the distance between the pixels that correspond to the features with minimum distance between them. The authors used the Cancer Therapeutics Response Portal v2 (CTRP) [38] and the Genomics of Drug Sensitivity in Cancer (GDSC) (https://www.cancerrxgene.org/, accessed on 1 October 2022) datasets to demonstrate on the experiments that CNNs trained using the output images of their algorithm outperform CNNs trained using another image representations and prediction models trained using the original tabular data in predicting anti-cancer drug response.

Thus, the transformation of the network data (PCAP packets) to image as a feature extraction procedure looks promising. It usually incorporates the following stages:

1.

Extraction of the numerical and nominal features from the PCAP packets.

2.

Transformation of the numerical and nominal features to image.

There are several research papers that implement this procedure to generate image-based features and further use obtained images to learn convolutional neural networks (CNN) for intrusion detection tasks [4,5,6,39,40,41].

Thus, in [4] authors use a convolutional neural network pre-trained model VGG-16 [42]. They transform 41 network features from NSL-KDD data [43] as follows: (1) normalize features; (2) extend their number from 41 to 121 to generate a grayscale image with size 11 × 11; (3) duplicate the single color channel of the generated image for each color channel and reshape to 224 × 224 × 3 because VGG-16 uses RGB image as input. The authors obtained anomaly detection accuracy of 89.30% for KDDTest+ and 81.77% for KDDTest-21 in case of binary classification task.

In [44], the authors also use Visual Geometry Group pre-trained model (VGG-19). After that they use a hybrid deep neural network based on CNN and long short-term memory (LSTM) to extract features from network traffic. The final VGG-19 + Hybrid CNN-LSTM model allows obtaining an accuracy of 98.86% while classifying attacks within the network intrusion benchmark dataset.

In [5], the authors use the MobileNetV2 convolutional neural network model [45] to detect attacks in binary and multi-class modes. They transform features extracted from the UNSW-NB15 dataset [46], containing labeled PCAP packets, into a 16 × 16 grayscale image as follows: each pixel corresponds to some feature value. For example pixel with coordinates (3, 13) is set to 255 if the packet uses HTTP protocol. The authors obtained the trained model’s accuracy 97%. The best accuracy was obtained for Generic, Fuzzers types of attacks and normal traffic. In [6], the authors trained their own CNN model on the NSW-NB15 dataset first and then used the pre-trained model to detect attacks in the NSL-KDD dataset. They also converted network features into images. The authors obtained up to 99.82% of detection rate on the KDDTest-21 dataset.

The research [47] deserves attention as the authors applied another approach to image generation—they transform texts (HTTP messages) to images on the character level: each character was represented by a pixel. After that, they trained a convolutional auto-encoder to detect anomalies in HTTP messages. Through the experiments, the authors demonstrated that the suggested approach outperforms traditional unsupervised methods, such as isolation forest and one-class support vector machine, in the anomaly detection task.

The analysis of the related works showed that transformation of non-image data into images with further training of a CNN has the following essential advantages: finding hidden relationships in attributes and their values; robustness in unseen objects detection (it is essential to detect unknown attacks); extracting basic knowledge from limited datasets [13].

Though researchers from different subject domains have adopted the transformation of various data types into images as the feature extraction procedure, in cyber security and security incidents detection in particular, the researchers work mainly with numerical and nominal features extracted from the PCAP packets and rarely transform them into images.

This research investigates a direct transformation of the raw PCAP packets into images. This approach has demonstrated high performance in the malware analysis tasks where the researchers analyze, usually, raw binaries [8,9,48]. The most interesting capabilities of such transformation were demonstrated in [8,9]. The authors showed that the features extracted from the malware images allowed detection of the malicious code compiled with different compiler settings, and obfuscation techniques.

3. A Proposed Approach to Image-Based Feature Extraction

The key idea of a proposed approach is a transformation of raw (binary) network packets into grayscale images that serve as an input to a classification module. This idea relies on the hypothesis that the model trained on images is protocol independent and robust to different even unseen types of network attacks because it is not required to calculate protocol-specific features, and CNNs are able to reveal hidden non-linear relations between attributes. Thus, the proposed approach to network intrusion detection consists of the following steps:

1.

An extraction of the raw (binary) packets from the PCAP files.

2.

A transformation of each packet into a grayscale image.

3.

An attack detection based on the analysis of the images.

The schema of the approach is given in Figure 1.

The grayscale images have only one channel that carries information about luminous intensity only, thus the pixels encode the amount of light only that varies from 0 to 255. In this case, the procedure of the network packet transformation is a quite straight-forward process. It does not require any extraction of such attributes as protocol type, service type, and others. Each network packet is treated as a binary sequence, which can be split into a sequence of bytes. Then, each byte is mapped into a grayscale level according to the following rule:

$$\begin{array}{c}0\times 0⟶0(\mathrm{black}\mathrm{color})\\ \cdots \\ 0\times FF⟶255(\mathrm{white}\mathrm{color})\end{array}$$

(1)

Figure 2 shows the process of byte conversion to a grayscale image.

Each grayscale image can be considered as a two-dimensional matrix, and it is necessary to solve two tasks when constructing it:

1.

To define a width and a height of the matrix, i.e., size of an image to be generated.

2.

To select a way how pixels are filled during the image generation.

The task of the image size definition is not trivial as soon as the length of network packets varies significantly. In malware analysis, the most common approach to construct the image from the binary code is to set a constant image width and a variable height that depends on the length of the malware code. In major cases, the generated image requires implementation of a resizing operation to fit the input data format of an analysis model. For example, the format of the input data of the majority of the pre-trained deep neural networks is a square image (N × N). Thus, if a particular pre-trained neural network is used to detect malicious packets then the size of the image is defined by the input format of this neural network. The authors suggest using another option. It consists of an analysis of the packet length distribution in the test dataset, and a selection of the image size based on a calculated statistical measure such as a mean packet size, maximum or minimum packet size, median, etc. It is proposed to use Formula (2) to determine the image size $S_{image}$, where $P_{stat}$ stands for a statistical parameter characterizing packet lengths’ distribution.

$$S_{image}=ceil(e^{ln(P_{stat})/2})$$

(2)

In all cases except one, when the maximum packet length is selected to determine the image size, the packets are cropped when the image is generated. In order to preserve all information about the network packets, it is recommended to use the maximum possible size of the packet as a measure to determine the image size. If the packets have a smaller size, the extra bytes are filled up with $0\times 00$.

There are several approaches to laying out the pixels within the image.

• $Linear$$layout$. The pixel matrix is filled up row by row. This is the most widely used approach.
• $Spiral$$layout$. The filling of the pixel matrix is started from the center and continues in a spiral form. This technique is suitable for square matrices.
• $Zig$–$zag$$layout$. This type of image construction is suggested in [49]. The pixels are filled up in the zigzag form. The underlying idea of such a layout relates to the fact that when the image size is greater than the length of the packet, the majority of pixels will be padded with zeros, however, the zig–zag filling of the pixel matrix with further discrete cosine transform can preserve and enforce existing patterns in low frequency part of the frequency domain.

Figure 3 shows pixel filling schemes for each approach and shows the corresponding examples of the generated images.

The attack detection could be performed by either pre-trained models such as VGG-16, or ResNet, or by a specially trained neural network. The authors studied both options, and the experiments were performed with two pre-trained models (ResNet34 [12] and MobileNetV3-small [50]) and one own CNN model.

4. Experiments and Discussion

To evaluate the approach, Secure Water Treatment (SWaT) dataset [51] was selected. This dataset is generated using a test bed that models a large water treatment facility with a six-staged technological process. The communication part of the SWAT test bed is represented by a layered communication network, programmable logical controllers, Supervisory Control and Data Acquisition (SCADA) server and a workstation, and a repository with historic data. The architecture of the test bed allows remote connection to the facility infrastructure by the operational personnel. The dataset has several versions that vary in types of collected data, attacks performed, and overall duration of the test bed functioning. To perform experiments, the SWaT.A6_Dec 2019 version of the dataset was selected. It consists of several PCAP files with network traffic and historical data from sensors in a .csv file. This dataset describes 3 h of normal functioning and 1 h during which 9 attacks of two different types were implemented. These attacks are targeted to extract historical data or disrupt the functioning of the sensors and actuators. Figure 4 shows the timeline of the attacks in the dataset with their type specification.

To evaluate the ability of the approach to detect attacks, the following experiment scenario was developed. The first stage included the analysis of the package length distribution and the choice of possible image size. In the second stage, the performance of the pre-trained models, as well as the impact of the different pixel layout schemes on their performance was evaluated. In the third stage, the experiments with the own CNN were performed. They included an analysis of the impact of image size, as well as the size of the training set on the model performance. The authors evaluated also its capability to detect unseen attacks. In order to implement this task, three different datasets were formed on the basis of the initial dataset. They are schematically shown in Figure 4: two datasets, namely (test/train dataset) and validation dataset 1 were formed from the first part of the SWAT dataset, and contained one attack type only, while the third dataset (validation dataset 2) was formed on the basis of the second part of the SWaT dataset and contained both types of attacks. The test/train dataset was used to train and assess the efficiency of all analysis models included in the experiments.

To assess the efficiency of the attack detection, the accuracy, F1-measure, recall, and precision metrics were used.

Choice of image size. The analysis of the network packet length distribution revealed that there is almost no difference in statistical characteristics for normal and abnormal packets—the maximum length of the malicious packets is slightly less than a normal one.

Table 1. Statistics on packet length distribution.

Packet Type	Mode	Median	80 Percentile	99 Percentile	Min	Max
Normal	64	90	128	633	60	19,034
Attack	64	86	128	633	60	14,888

summarizes the obtained results, and Figure 5 shows the distribution of the packet lengths for normal and abnormal traffic. Using Formula (2), the maximum image size is determined as $138\times 138$.

Performance evaluation of the pre-trained deep neural networks. In this scenario, a pre-trained model is used in the feature extraction task, and the weights of the final layer only are updated in order to make predictions. Two pre-trained neural networks, namely ResNet34 and MobileNetV3-small were selected. Both of these networks were trained on the ImageNet dataset, which contains images and is usually used in the object detection task. Their key characteristics such as the classification accuracy and the number of FLOPs are given in

Table 2. Characteristics of the selected pre-trained models.

Pre-Trained Model	Input Image Size	Top-1 Accuracy	Top-5 Accuracy	FLOPs (Millions)	Num. of Trained Parameters
ResNet34 [12]	224 × 224	73.31	91.42	21.8	1026
MobileNetv3-small [50]	224 × 224	67.67	87.40	2.5	2050

. It is obvious that the accuracy of the ResNet34 is higher than one of the MobileNetV3-small, however, the MobileNetV3-small is a lightweight neural network designed for mobile CPUs with low computational resources. It should be noted that the classification accuracy of these neural networks is higher than the ones that are used in [5,39].

The experiments were performed with models from PyTorch model hub [52], and the following parameters were used in the experiments to train the model:

• Number of epochs: 7;
• Optimizer: SGD (Stochastic gradient descent);
• Loss function: CrossEntropyLoss;
• Learning rate: 0.001;
• Batch size: 16.

It should be noted that an attack with a duration of 15 min is described by 22 million network packets. To train the model, the dataset was reduced to 1 million packets, and balanced to keep the ratio of normal packets (70%) and attack packets (30%). The obtained results are given in

Table 3. Experimental results with pre-trained models and different pixel layout schemes.

Pre-Trained Model	Pixel Layout Scheme	Accuracy	Precision	Recall	F1-Measure
ResNet34 [40]	linear	0.56	0.51	0.51	0.49
	spiral	0.59	0.53	0.52	0.51
	zig-zag	0.64	0.55	0.57	0.57
MobileNetv3-small [41]	linear	0.54	0.49	0.47	0.48
	spiral	0.52	0.49	0.5	0.48
	zig-zag	0.56	0.5	0.52	0.44

. It is obvious that the performance of the classifiers is low and comparable to the accuracy of a random classifier. The calculated metrics are slightly higher for the ResNet34 classifier with zig–zag pixel layout, but still not acceptable.

The authors assume that there are few reasons for such results. First of all, in the experiments, two pre-trained models were used in feature selection mode, i.e., all layers were “freezed”, and the weights of the last layer only were updated during the training. The number of the training epochs was not large and, perhaps, not enough to train the model. Another possible reason is a significant difference between the train dataset and Image net dataset that was used to train MobileNetV3-small and ResNet3 models. These models are trained to detect real-world objects, while the images generated from network traffic are totally different, in major cases, they are mostly black images. Moreover, the size of the initially generated image is smaller than that is required for neural network input, and it is re-scaled using nearest interpolation operation [53]. Thus, the usage of the pre-trained models in feature extraction mode is not efficient, they require significant fine-tuning, i.e., updating all neural network weights. This process could be resource-exhaustive, especially for large models such as ResNet34. That is why authors believe that a possible way of transfer learning application could be as presented in [6]. The model is trained on some public labeled dataset with network attacks, and then fine-tuned on some private or other dataset. Additionally, in this case feature selection mode of the pre-trained models would be efficient as the subject domain of the datasets used to train and fine-tune the model is the same.

Experiments with CNN designed for the task. To test the approach to feature extraction, the authors also developed their own CNN with the following architecture:

• 1 input layer;
• 3 convolutional layers;
• 3 max pooling layers;
• 1 flatten layer;
• 1 dense layer;
• 1 output layer.

This network was trained for 10 epochs and demonstrated significantly higher results. The experiments also showed that they do not depend on image size and the size of the training dataset.

Table 4. Accuracy of the CNN trained with different initial parameters of the training dataset.

Size of the Test Dataset	Image Size	Metric Used to Define Image Size	Accuracy
1 million	10 × 10	Median (and mode)	0.99
	26 × 26	99 Percentile	0.99
	138 × 138	Maximum size	0.99
10 millions	138 × 138	Maximum size	0.99

gives brief results of the CNN training with different initial parameters, and Figure 6 provides detailed information on the training process for the case when the CNN was trained on the 10 millions of images. The best performance of the model is achieved on the 5–6 epoch training, then the precision and the recall metrics slightly deteriorate. The recall parameter is high, it equals 0.98, however, the precision is low—it is slightly higher than 0.7. The precision value is more important in the attack detection task, as it considers false positive cases, and the low value of precision indicates the high rate of false positive cases.

The last series of experiments was devoted to the evaluation of the CNN ability to detect novel and unseen attacks. In this experiment two datasets, validation dataset 1 and validation dataset 2, were used. In the case of the first dataset the authors tried to evaluate the robustness of the model to detect similar attacks that could vary in such attributes as packet timestamps and checksums. The goal of the second dataset application is to assess the model robustness to novel and unseen attacks. It should be noted that the considered attacks are different in their origin, the target of the first attack is to retrieve data from the repository with archive data, while the target of the second attack is to disrupt the functioning of the sensors and actuators. The obtained results are shown in

Table 5. Performance of the CNN on different validation datasets.

Validation Dataset	F1-Measure	Precision	Recall
Validation dataset 1	0.78	0.64	0.98
Validation dataset 2 (with novel attack type)	0.75	0.62	0.95

. Interestingly, they are almost similar for both validation datasets. The precision metric is slightly lower on the test/train dataset, but it is almost the same for both types of attacks, however, the recall metric that characterizes the true positive rate is high indicating that the classifier detected almost all samples with the attack. The latter looks very promising and stimulating to continue further research on the suggested approach to network traffic pre-processing. The main problem consists in decreasing the false positive rate while maintaining the high recall rate. The possible solution of this task is changing the parameters of the image construction procedure, for example, selecting the mode of network packet lengths as a base parameter to define image size. We also compared the performance of the CNN model to the performance of the Random Forest (RF) model that is often used in attack detection tasks as it usually demonstrates high performance and is characterized by low requirements to computational resources [54,55]. It was trained on test/train dataset, and then evaluated on the validation dataset 2. The performance of the RF on the dataset with a known attack was high, reaching 99% of the accuracy and F1-measure, however, when applied to dataset with an unseen attack the accuracy decreased to 63%, thus indicating about low generality of the model.

Thus, a certain bottleneck of the suggested approach consists in selecting the image size that is used to generate an image. Though the performed experiments showed that it does not impact the accuracy much, it is required to have a priori information on statistical characteristics of package length distribution in order to define image size. The authors also consider that it is necessary to evaluate the impact of image size more thoroughly, for example, by performing experiments on different datasets that include different types of attacks, and by evaluating how the most important pixels (features) are located within the image. Secondly, though experiments showed that the pre-trained models in feature extraction mode do not demonstrate high efficiency, in a fine-tuning mode they could have high accuracy, and in this case the selection of the image size would depend also on the input format of the selected model.

5. Conclusions

Approaches for feature extraction based on the transformation of the raw data into images have attracted recently a lot of research attention. This paper investigated the approach to attack detection that uses the transformation of network traffic into grayscale images.

The analysis of the related works showed that currently there are only a few research papers devoted to this problem, and they use a feature vector that is extracted from the PCAP files to generate an image. This paper suggests creating grayscale images directly from raw network packets that are extracted from PCAP files. The main motivation for such a solution is that the model trained on images could be robust to novel and unseen types of network attacks.

To evaluate the efficiency of the approach, a series of experiments were performed. They included performance assessment of the pre-trained models, such as MobileNetV3-small and ResNet34, training and testing our own CNN on different datasets with different types of attacks.

The SWaT dataset was used as a test dataset, which describes the functioning of a smart water treatment facility and contains such attacks as historical data extraction and sensor disruption.

The implemented experiments showed that the performance of the pre-trained models when they are used in feature extraction mode is low and comparable to the performance of the random classifier. The CNN which was created by authors demonstrated quite a high positive detection rate even when detecting unseen attacks. In future research, we plan to hyper-tune the CNN to enhance the results.

The identified drawbacks of the approach, including the high false positive rate of the classifier and the image size selection, defined the direction of future research work. They include evaluation of the pixel’s importance in the context of their layout within the image in order to determine optimal image size. Another direction of the future research relates to the investigation of another technique for image construction, which assumes the generation of one image for a series of packets.