Detection of Adversarial DDoS Attacks Using Generative Adversarial Networks with Dual Discriminators

Abstract

DDoS (Distributed Denial of Service) has emerged as a serious and challenging threat to computer networks and information systems’ security and integrity. Before any remedial measures can be implemented, DDoS assaults must first be detected. DDoS attacks can be identified and characterized with satisfactory achievement employing ML (Machine Learning) and DL (Deep Learning). However, new varieties of aggression arise as the technology for DDoS attacks keep evolving. This research explores the impact of a new incarnation of DDoS attack–adversarial DDoS attack. There are established works on ML-based DDoS detection and GAN (Generative Adversarial Network) based adversarial DDoS synthesis. We confirm these findings in our experiments. Experiments in this study involve the extension and application of the GAN, a machine learning framework with symmetric form having two contending neural networks. We synthesize adversarial DDoS attacks utilizing Wasserstein Generative Adversarial Networks featuring Gradient Penalty (GP-WGAN). Experiment results indicate that the synthesized traffic can traverse the detection systems such as k-Nearest Neighbor (KNN), Multi-Layer Perceptron (MLP) and Random Forest (RF) without being identified. This observation is a sobering and pessimistic wake-up call, implying that countermeasures to adversarial DDoS attacks are urgently needed. To this problem, we propose a novel DDoS detection framework featuring GAN with Dual Discriminators (GANDD). The additional discriminator is designed to identify adversary DDoS traffic. The proposed GANDD can be an effective solution to adversarial DDoS attacks, as evidenced by the experimental results. We use adversarial DDoS traffic synthesized by GP-WGAN to train GANDD and validate it alongside three other DL technologies: DNN (Deep Neural Network), LSTM (Long Short-Term Memory) and GAN. GANDD outperformed the other DL models, demonstrating its protection with a TPR of 84.3%. A more sophisticated test was also conducted to examine GANDD’s ability to handle unseen adversarial attacks. GANDD was evaluated with adversarial traffic not generated from its training data. GANDD still proved effective with a TPR around 71.3% compared to 7.4% of LSTM.

1. Introduction

By flooding malicious traffic, DoS attacks deplete a targeted system’s network bandwidth and computing resources, preventing the target system from providing legitimate users with regular services. DDoS attacks are becoming increasingly widespread. DDoS attacks seize control of a huge number of infiltrated computers, known as botnets, and conduct synchronized attacks on the victim system, as can be seen in Figure 1. Together with the emergence and accomplishment of revolutionary Internet technologies, DDoS attacks are escalating in frequency, magnitude and sophistication. Organizations face possible network threats that could have serious consequences for their operations, such as downtime, data breaches, or even extortion demands from attackers [1].

Figure 1. DDoS attack with the botnet.

As indicated in [2], appropriate DDoS mitigation procedures should be taken upon the occurrence of DDoS attacks. Before any mitigation strategies can be implemented, DDoS attacks must be detected essentially. DDoS attacks were initially detected by traffic engineers using programmed rules. This approach appears to have fallen behind the dynamic and evolving nature of DDoS attacks. As ML (Machine Learning) and DL (Deep Learning) exhibit their enormous potential in a variety of fields, academia and industry are investigating the feasibility of applying ML/DL to detect DDoS. Notable successes have been achieved, as documented in [3]. Human experts or specific feature selection schemes need to choose features for classification in ML. On the other side, feature engineering is a vital built-in piece of DL. Some successful stories on DL for detecting DDoS will be analyzed in Section 2.

When it comes to DDoS detection, both ML and DL demand labeled traffic for training. DDoS detection system’s performance is determined by the quality of the training set. A new type of DDoS attack referred to as adversarial DDoS attack could pose new challenges to traditional approaches. Since Szegedy et al. [4] first proposed the concept of adversarial examples, their work has aroused interest among researchers in the field of adversarial attacks. In the Internet of Things era, the network has become an ideal target for cyber attackers. Hackers often exploit vulnerabilities in artificial intelligence to bring out cyberattacks. Despite the AI communities’ indefatigable endeavors to establish defensive barricades, the number of adversarial attacks is escalating dramatically, and their posing threats proceed to grow. Furthermore, as the economic benefits sprout, the number of adversarial attacks will keep rising. A secure cyber security environment can only be created through the continuous development of deep learning patterns, as suggested by Han et al. [5].

As stated in [6], attacking and defending adversaries is an iterative, evolutionary process. Adversary data makers evolve new adversarial attacks by exploring new vulnerabilities and improving algorithms, while defenders analyze the characteristics of alternative threats and employ suitable methods to provide effective defenses against adversarial attacks.

The Generative Adversarial Network (GAN) [7] is well-known for creating fictitious but realistic-looking data, such as image generation. We believe GAN can also create fraudulent but legitimate-looking traffic, which causes DDoS detection systems to be confused. This study is a corroboration of such a belief. We synthesize adversarial DDoS attacks employing Wasserstein Generative Adversarial Networks [8] featuring Gradient Penalty (GP-WGAN) [9]. As we shall see in Section 4, synthesized traffic can evade DDoS detection systems such as k-Nearest Neighbour, Multi-Layer and Perceptron Random Forest.

This study has its contribution in the proposal of a novel framework, named Generative Adversarial Networks with Dual Discriminators (GANDD), for the detection of adversarial DDoS attacks. The proposed GANDD has a unique design featuring dual discriminators. One discriminator provides the ability to detect adversarial DDoS attacks, while the other distinguishes legitimate traffic from conventional DDoS traffic. The additional discriminator is designed to target adversary DDoS traffic. To evaluate GANDD’s effectiveness, adversarial DDoS traffic synthesized by GP-WGAN is used to train GANDD. Trained GANDD outperformed DNN (Deep Neural Network), LSTM (Long Short-Term Memory), and GAN, with a TPR (True Positive Rate) of 84.3%. A more sophisticated test was also conducted to examine GANDD’s ability in handling unseen adversarial attacks. GANDD was evaluated with adversarial traffic generated from different training data. GANDD still proved effective with a TPR of around 71.3% compared to 7.4% of LSTM.

The proposed GANDD has its novelty and contribution both in structure and application. To our best knowledge, there are only a small number of studies working on GAN with dual discriminators, such as that in the works of Nguyen et al. [10] and Zhang et al. [11]. Nonetheless, these studies developed parallel dual discriminators that worked separately and focused on increasing the quality and diversity of synthesized data generated by the generator, mainly for image processing applications. Meanwhile, our approach has a novel and distinct design with consecutive dual discriminators that complement each other to defend against adversarial DDoS attacks effectively.

The rest of this paper is structured as follows. In Section 2, previous works and GAN are reviewed briefly. Section 3 embodies out the framework for the proposed approach. The results of the experiment are then reported in Section 4. Finally, some conclusions are made in Section 5.

2. Related Work

2.1. ML and DL for DDoS Detection

Various machine learning technologies have been employed to detect DDoS attacks, primarily as classifiers. There are k-Nearest Neighbours (KNN), Support Vector Machine (SVM), Random Forest (RF), Naïve Bayes Classifier, Artificial Neural Network (ANN), and Density-Based Spatial Clustering of Applications with Noise (DBSCAN), to name a few. With SVM, a hyperplane is established in the transform domain to classify unseen data using labeled training data. Cheng et al. [12] developed an IAI (IP Address Interaction Feature) model that can distinguish normal from abnormal traffic flows and assisting in the rapid and precise detection of attack flows. KNN locates the incoming data’s k nearest neighbors. The classification of the incoming data is decided by a majority of these k neighbors. Vu [13] utilized KNN to classify the state of networks focusing on each stage of DDoS attacks and obtain optimistic outcomes. The Naïve Bayes classifier is a probabilistic classifier based on Bayes’ theorem and the assumption of predictor independence. According to a Naïve Bayes classifier, a particular feature in a class is unrelated to any other feature. Based on the mean and standard deviation of network packets, Fadil et al. [14] used the NB method for predicting the existence of DDoS attacks and obtained precise results. An RF is a collection of decision trees. The classification is determined by the majority of the outcomes of individual decision trees. Wang et al. [15] asserted that the RF algorithm could achieve good classification performance and the best feature subset by using well-calculated key features in DDoS data. DBSCN finds high-density core samples and expands clusters from them. It is ideal for data with clusters of similar density. Dincalp et al. [16] utilized the DBSCAN clustering algorithm to deal with attack vector diversity. In their experiments, the proposed system performs well with chosen characteristics. ANNs are artificial neural networks that emulate biological neural networks. ANNs use the back-propagation technique to learn the mapping function from labeled data. Ahanger et al. [17] introduced an ANN-based DDoS detection system that detects DDoS attacks with a 99.8% detection accuracy.

In DDoS detection, there are also successful stories for DL. Li et al. [18] integrate Long Short-Term Memory (LSTM) and Bayesian approaches to detect DDoS attacks. LSTM is well suited to time-domain events with long intervals and delays. The author employs LSTM to determine the confidence index of DDoS attacks and then implements the Bayesian method to improve detection accuracy. Yang et al. [19] adopt the autoencoder for detecting DDoS attacks. Autoencoder is an unsupervised training multilayer neural network. During the training process, it filters out less relevant information and background noise and retains essential information. It’s essentially a form of feature selection. Training and operation times can be greatly accelerated.

2.2. Generative Adversarial Networks

GAN, first proposed by Goodfellow et al. in 2014 [7], demonstrates a wide variety of applicability in recent years. A GAN has a symmetric fabrication consisting of two functional blocks, generator (G) and discriminator (D), as shown in Figure 2. These two blocks act as participant in game theory engage against each other. The generator aims at generating adversarial data to deceive the discriminator. Conversely, the discriminator is in charge of determining validity. As a GAN converges after training, G is supposed to generate synthetic data that is indiscernible from the real ones. At the same time, D intends to distinguish whether the input comes from G or the original training set. They both want to maximize their chances of winning and minimize the opponent’s chances on each move. The loss function of GAN can be expressed in Equation (1).

$$\underset{G}{\mathit{min}} \underset{D}{\mathit{max}}V\left(D,G\right)={\mathbb{E}}_{p_{data}\left(x\right)}\left[logD\left(x\right)\right]+{\mathbb{E}}_{p_z\left(z\right)}\left[log(1-D\left(G\left(z\right)\right)\right]$$

(1)

where $x$ is the data, $z$ is the input noise to the generator, $p_{data}$ is the data distribution, $p_z$ is the noise distribution, D is the discriminator and G is the generator.

Figure 2. A generative adversarial network’s functional block diagram.

Despite its success in various fields, GAN training and convergence still remain to be challenging for practitioners. The original GAN is predisposed to convergence issues, such as loss functions failing to furnish training direction, lacking variety in generated data, and so on. Wasserstein GAN (WGAN) [8] proposes the Wasserstein distance to overcome the gradient vanishing problem in original GANs. The WGAN is determined to be more stable in the training process, with fewer collapsing cases.

Even though WGAN improves on the original GAN, it still has issues with data quality and divergence. The Weight Clipping strategy adopted by different WGAN has resulted in these difficulties. The Lipschitz constraint must be enforced using the Weight Clipping strategy. As a result, the WGAN weights may converge to critical values, degrading generation efficiency or even causing the model to collapse. Gradient Penalty-WGAN (GP-WGAN) proposed the Gradient Penalty strategy to address this issue [9]. GP-WGAN learns a uniformly distributed gradient after using the Gradient Penalty strategy. The Gradient Penalty strategy improves the stability of WGAN training and provides higher-quality data generation.

For the detection of outliners in data mining applications, Li et al. [20] obtained interesting results with the introduction of “RCC Dual-GAN” featuring two separate GANs. The first GAN performs the data distribution reference, while the second one responds to overfitting constraints. Meanwhile, we had taken a novel and radically different approach by entrenching an additional discriminator in a unified GAN model. Accordingly, the proposed framework becomes more comprehensive and coherent, as shown in the experiments.

2.3. Adversarial DDoS Attacks

Hackers and researchers envision GAN holds potential to bring up new type of DDoS attack-adversarial DDoS attacks. In other words, GANs may be used to produce malicious but legitimate-looking traffic. Trend Micro Inc. reported adversarial DDoS assaults at CYBERSEC-2019. Hackers employ Adversarial Machine Learning (AML) technology to discover the vulnerabilities in machine learning-based detection systems and confuse them by adversarial traffic. Studies are devoted to the impacts of adversarial DDoS attacks. Least Square GAN (LSGAN) was proposed in [21] to generate artificial traffic based on the observation that DDoS attacks closely resemble regular flash crowds. Up to 99% of DDoS traffic generated by LSGAN was misclassified as legit flash crowds. In [22], an exciting architecture for adversarial DDoS attacks called MalGAN is presented. A black box detector and two neural networks are included. The black box detector serves as a victim system. The black-box detector is used to classify the instances generated by the generator. The classification results are then fed to the discriminator. When the discriminator could no longer recognize among adversarial attacks and benign, the system converges.

For the time being, detection of adversarial DDoS attacks is still in its early stages, but the AI community has made significant efforts to deal with the adversarial DDoS attacks. Kolosnjaji et al. [23] have proposed a gradient-based evading attack. The authors append a set of bytes to the end of the malicious binary without breaking functionality, and then compute the gradient to generate adversarial examples. The limitation of this work is using dataset is not large enough for comprehensive results. Song et al. [24] presented a method for creating and testing actual malware that could be used to carry out the elusive attack. A designed action set and verification function were used to create the adversarial attack. However, the methods of defense and the framework’s robustness have received less attention. Ebrahimi et al. [25] proposed MalRNN to automatically generate adversarial malware variants. MalRNN is a novel DL model that acquires data through system sampling and learns the language model of benign malware binaries by using RNN. The study is hindered by the fact that its method for avoiding antivirus is simple.

2.4. Detection of Adversarial DDoS Attacks

Even now, detection of adversarial DDoS attacks is still in its early stages. Adversarial DDoS attacks mislead ML-based detection systems by GAN-generated traffic. This problem can be addressed with an Adversarial Detection Module (ADM) [26] in the loop. Adversary DDoS attacks are captured and logged by ADM. Logged and labelled traffic can be used to incrementally retrain the detection system. The obstacle in previous approaches is that human data engineers perform the role of ADM. The discriminating and tagging a labor intensive and time-consuming process. The most significant contribution of this research is the introduction of an additional discriminator to automate the process of capturing and tagging adversarial DDoS traffic.

3. Proposed Approach

3.1. Synthesis of Adversarial DDoS Attacks Using GP-WGAN

For the generation of legitimate-looking adversarial DDoS attack traffic, we adopt the GP-GWAN architecture with normal, and DDoS traffic as training data, as shown in Figure 3. The generator is fed with DDoS traffic to ensure the training direction will not distract during the training phase. It also reduces the chance of the model collapsed. The ML-based IDS in Figure 3 is the DDoS detection system we try to deceive, such as RF, KNN and MLP. The output of the IDS is used to train the discriminator. The two networks contend with each other. As converged, the generator is ready to launch adversarial DDoS attacks.

Figure 3. A GP-WGAN architecture for the synthesis of adversarial DDoS attacks.

The generator is the most critical part of the architecture. GP-WGAN, instead of the original GAN, is adopted for its stability in training and immunity to model collapsing. The generator is designed for the generation of deceiving traffic. To this aim, it is required for the discriminator to provide the ML-based detectors with adequate feedback, based on which network parameters can be adjusted, such that the generated traffic is capable of deceiving the ML-based detectors. As converged, generated traffic appears to be legitimate to the detection system. However, they are actually adversarial DDoS attacks on the boundary between regular and malicious traffic. The loss function of the generator is defined as follows:

$$L\left(G\right)={\mathbb{E}}_{A,N}D\left(G\left(A,N\right)\right)$$

(2)

where A represents the attack traffic in the training set, N is the generator’s noise input, G is the generator, and D is the discriminator.

In order to generate deceiving adversarial DDoS traffic, it is needed to minimize Equation (2). The discriminator is designed to emulate ML-based detectors. Therefore, the training process is to minimize the loss function based on the output of ML-based detectors, as given in Equation (3). In the meantime, we also feed the discriminator with adversarial DDoS data to facilitate the generation of traffic capable of deceiving the ML-based detectors.

$$L\left(D\right)={\mathbb{E}}_B\left(D\left(B\right)\right)-{\mathbb{E}}_A\left(D\left(A\right)\right)$$

(3)

where B represents the normal traffic, and A is the attack traffic identified by the ML-based detectors.

The discriminator’s objective is to imitate the ML-based detector. It gives feedback to the generator to generate adversarial traffic that defeats the discriminator. The resulted in traffic should be undetectable to the target ML-based detector. Algorithm 1 presents the operation of the GP-WGAN.

Algorithm 1 Training of the GP-WGAN.

Input: G: generator, D: discriminator, IDS: ML-based DDoS detectors B: benign traffic, A: DDoS traffic A, N: noise for adversarial disturbance Output: Trained GP-WGAN Train IDS with B and A for n_epochsdo    // Training of G    $L\left(G\right)\leftarrow {\mathbb{E}}_{A,N}D\left(G\left(A,N\right)\right)$    Update G according to the gradient of G’s loss function    // Training of D    // B_pred: benign traffic predicted by IDS    // A_pred: attack traffic predicted by IDS    B_pred, A_pred $\leftarrow IDS\left(G\left(A,N\right)\right)$    $L\left(D\right)\leftarrow {\mathbb{E}}_{B\_pred}\left(D\left(B\_pred \right)\right)-{\mathbb{E}}_{A\_pred}\left(D\left(A\_pred \right)\right)$    Update D according to the gradient of D’s loss function end for

3.2. Detection of Adversarial DDoS Attacks Using GAN with Dual Discriminators (GANDD)

The proposed GANDD is a GAN with dual discriminators that can provide a solution for detecting adversarial DDoS attacks, as illustrated in Figure 4. The additional discriminator employs the ADM approach, which is in charge of adversarial DDoS traffic discrimination. The traffic generated by the GP-WGAN is included in the GANDD’s training set. During operation, the ADM discriminator inspects incoming traffic before directing it to the regular discriminator for further examination.

Figure 4. The architecture of GANDD.

In a standard GAN, the generator uses samples from the problem domain to generate new instances to take down the discriminator. For GANDD, the generator is fed with both general DDoS traffic and adversarial samples generated from GP-WGAN to ensure the detection of adversarial DDoS traffic. The generator’s loss function is defined as follows:

$$L\left(G\right)={\mathbb{E}}_{A,N,E}\{D2[D1\left(G\left(A,E,N\right)\right]\}$$

(4)

where A is regular attacks, N is the noise input to the generator, E is adversarial attacks, D1 is the adversarial attack discriminator and D2 is the regular attack discriminator.

In the training of discriminator D1, we adopted adversarial DDoS traffic generated by GP-WGAN. Thereby, we will have the correct gradient when contending with the generator. Regular traffic is also taken into account when training discriminator D2 to endow it with the capability to distinguish between legitimate and malicious traffic. The discriminator’s loss function is defined as follows:

$$L\left(D\right)={\mathbb{E}}_E\left(D1\left(E\right)\right)-{\mathbb{E}}_{B,A}\left(D2\left(A,B\right)\right)$$

(5)

where E is the adversarial attack, D1 is the adversarial discriminator, B is the benign traffic, A is the regular attack and D2 is the regular discriminator. $L_D$ is fed back to the generator for its parameter adjustment. Algorithm 2 presents the operation of the GANDD.

Algorithm 2 Algorithm for GANDD

Input: G: generator, D1: discriminator1, D2: discriminator2, B: benign traffic, N: noise, A: normal DDoS Attack, E: adversarial attack generated by GP-WGAN Output: Trained GANDD

for n_epochs do    //Training of G    $L\left(G\right)\leftarrow {\mathbb{E}}_{A,N,E}\{D2[D1\left(G\left(A,E,N\right)\right]\}$    Update G according to the gradient of G’s loss function    //Training of D    //S: traffic predicted by D1    $S\leftarrow D1\left(G\left(A,E,N\right)\right]$    if S is not an adversarial attack then      if S is benign then       $B\leftarrow B+S$      else       $A\leftarrow A+S$    $L\left(D\right)\leftarrow {\mathbb{E}}_E\left(D1\left(E\right)\right)-{\mathbb{E}}_{B,A}\left(D2\left(A,B\right)\right)$    UpdateD1 and D2 according to the gradient of D’s loss function end for

It is well known that GANs are difficult to train. The situation is more difficult in GANDD due to its dual discriminators. Individual discriminator drives the training toward a different direction, resulting in instability in the training process. After the intensive empirical study, we observe that the ReLU activation function is prone to gradient vanishing. We replace the ReLU by PReLU, which is capable of self-adaptation.

4. Experiments Results and Discussion

A series of experiments are implemented to verify that the proposed approaches are feasible and effective. The followings statements will be examined in turn in the subsequent experiments:

• ML-based DDoS detection is highly effective
• Adversarial DDoS attacks can penetrate ML-based DDoS detection
• The proposed GANDD is capable of detecting adversarial DDoS attacks

Experiments are conducted on a Windows 10 personal computer with an Intel i7-8700 CPU, 32GB RAM and an NVIDIA RTX 2060 graphics card. The GP-WGAN, for the synthesis of adversarial DDoS attacks, is implemented with PyTorch. The GANDD framework for detecting adversarial DDoS detection is implemented in TensorFlow paired with Keras.

Two well-embraced datasets, NSL-KDD [27] and CIC-IDS2017 [28], are adopted in this study. They are datasets presented by the Canadian Institute for Cybersecurity for the study of intrusion detection. The NSL-KDD dataset is an optimized version of the KDD’99 [29] dataset, which removes redundant and duplicate data from KDD’99. The CIC-IDS2017 dataset contains benign and the most up-to-data common attacks, it also includes the results of the network traffic analysis using CICFlowMeter [30] with labelled flows based on the time stamp, source, and destination IPs, source and destination ports, protocols and attack (CSV files). We focus on DDoS traffic and use them for training ML-based DDoS detection and the synthesis of adversarial DDoS attacks.

A number of performance indices are employed for comparative study. Referring to Table 1, the definitions of True Positive Rate (TPR), False Positive Rate (FPR) and F1-score are given in Equations (6)–(8), respectively. The Receiver Operating Characteristic (ROC) curve is used to evaluate the impact of adversarial DDoS attacks and the effectiveness of the proposed GANDD in the detection of DDoS attacks.

$$TPR=\raisebox{1ex}{$TP$}\!\left/ \!\raisebox{-1ex}{$\left(TP+FN\right)$}\right.$$

(6)

$$FPR=\raisebox{1ex}{$FP$}\!\left/ \!\raisebox{-1ex}{$\left(FP+TN\right)$}\right.$$

(7)

$$\mathrm{F}1\_\mathrm{score}=\raisebox{1ex}{$2.TP$}\!\left/ \!\raisebox{-1ex}{$\left(2TP+FP+FN\right)$}\right.$$

(8)

Table 1. Confusion matrix.

Actual Predicted	Attack	Normal
Attack	TP (True Positive)	FP (False Positive)
Normal	FN (False Negative)	TN (True Negative)

4.1. Synthesis of Adversarial DDoS Attacks Using GP-WGAN

Literature reports that ML had been applied to the detection of DDoS with satisfactory achievement. This subsection examines the capability of DDoS detection of RF, KNN, and SVM on NSL-KDD and CIC-IDS2017. Training data are labelled as legitimate or malicious beforehand. 80% of the samples are used for training, and 20% are used for validation. The results of applying RF, KNN and SVM on NSL-KDD and CIC-IDS2017 are reported in Table 2 and Table 3, respectively. A higher TPR and F1-score value is an indication of better detection capability. We can see that all three ML-based DDoS detection approaches are considered effective. This finding is consistent with those in previous studies.

Table 2. ML-based DDoS detection on NSL-KDD.

Model	TPR	FPR	F1-Score
RF	0.901	0.066	0.918
KNN	0.868	0.124	0.913
SVM	0.880	0.152	0.860

Table 3. ML-based DDoS detection on CIC-IDS2017.

Model	TPR	FPR	F1-Score
RF	0.942	0.039	0.951
KNN	0.914	0.094	0.909
SVM	0.873	0.173	0.843

In addition to the TPR and the F1-score, the ROC curve is another useful performance metric to assess the detection and classification accuracy. For a ROC curve, a larger Area Under the Curve (AUC) indicates better detection accuracy. Figure 5 shows that RF has the best detection rate, followed by KNN and then SVM.

Figure 5. ROC curves of ML-based DDoS detection.

4.2. Synthesis of Adversarial DDoS Attacks Using GP-WGAN

GAN is well recognized in the generation of realistic-looking fake images. We expect that GAN can also be used to generate legitimately-like malicious traffic, i.e., adversarial DDoS attacks. We adopt the GP-WGAN architecture discussed in Section 3 with a network configuration listed in Table 4 and training parameters given in Table 5. Gradient Penalty is incorporated to stabilize the training process. The activation function of the neural networks adopts Leaky ReLU instead of sigmoid to improve the robustness of the training process and to prevent the phenomena of vanishing gradient.

Table 4. Network configuration of GP-WGAN.

Layer	Configuration
Input (Noise)	(None, 20)
Input (Normal DDoS)	(None, 41)
Concatenate Input (Normal DDoS, Noise)	(None, 61)
Dense	(None, 32)
Leaky ReLU	(None, 32)
Dense	(None, 8)
Leaky ReLU	(None, 8)
Dense	(None, 2)
Leaky ReLU	(None, 2)
Input (Noise)	(None, 20)
Input (Normal DDoS)	(None, 41)

Table 5. Training parameter settings of GP-WGAN.

Epoch/Batch Size	Learning Rate	Optimizer	Lambda	CRITIC_ITERS
500/1024	0.000139	Adam	10	5

After training, the adversarial DDoS attacks synthesized by the GP-WGAN are directed to ML-based DDoS detectors to observe their capability in detecting adversarial DDoS attacks. We experiment with RF, KNN, and SVM on adversarial DDoS attacks trained using NSL-KDD and CIC-IDS2017. As shown in Table 6, the TPRs drop dramatically to near zero. It implies that adversarial DDoS attacks can easily penetrate ML-based DDoS detectors without being detected. The ROC curves in Figure 6 also report ML-based DDoS detectors’ inability to deal with adversarial DDoS attacks. We can conclude that adversarial DDoS attacks are a substantial breach of conventional ML-based DDoS detectors. Furthermore, the adopted GP-WGAN is highly effective in synthesizing adversarial DDoS attacks.

Table 6. TPRs of ML-based DDoS detectors on adversarial DDoS attacks.

	NSL-KDD	CIC-IDS2017
RF	0.024	0.097
KNN	0.013	0.066
SVM	0.006	0.004

Figure 6. ROC curves of ML-based DDoS detection on adversarial DDoS attacks.

Conventional ML/DL-based DDoS detection is trained using standard DDoS datasets. They can successfully deal with conventional DDoS attacks. However, synthesized adversarial attacks are legitimate-like malicious traffic. To figure out the reason behind the severe impact brought about by adversarial DDoS attacks, we virtualize the distribution of training samples and adversarial DDoS attacks for SVM. As revealed in Figure 7, synthesized adversarial DDoS attacks lay along the discrimination hyperplane of SVM. Such a distribution renders SVM unable to make correct decisions and therefore degrades its detection ability. In other words, conventional ML/DL approaches lack the ability in detecting adversarial DDoS attacks.

Figure 7. Visualization of SVM on adversarial DDoS attacks.

4.3. Detection of Adversarial DDoS Attacks with GANDD

In recognizing the potential threats of adversarial DDoS attacks, we develop the GANDD, presented in Section 3, as a countermeasure. The adversarial DDoS attacks synthesized by the GP-WGAN can be used to train the proposed GANDD to endow it with the ability in detecting adversarial DDoS attacks. As explained in Section 3, the additional discriminator in GANDD is designed to detect adversarial DDoS attacks. The second discriminator is used to discriminate legitimate traffic and conventional DDoS attacks. The network configurations and the training parameter settings for the proposed GANDD are given in Table 7, Table 8 and Table 9.

Table 7. Configuration of D1 of GANDD.

Layer	Configuration
Input (Adversarial DDoS Attack)	(None, 41)
Dense	(None, 32)
PReLU	(None, 32)
Dense	(None, 32)
PReLU	(None, 8)
Dense	(None, 8)
Dropout	(None, 8)
Dense	(None, 2)
PReLU	(None, 2)

Table 8. Configuration of D2 of GANDD.

Layer	Configuration
Input (Conventional DDoS Attack, Benign)	(None, 41)
Dense	(None, 32)
PReLU	(None, 32)
Dense	(None, 32)
PReLU	(None, 8)
Dense	(None, 8)
Dropout	(None, 8)
Dense	(None, 2)
PReLU	(None, 2)

Table 9. Training parameter settings of GANDD.

Epoch/Batch Size	Learning Rate	Optimizer	Lambda	CRITIC_ITERS
500/1024	0.000144	Adam	10	5

A fundamental difficulty in training GANs is the disorientation of training direction caused by the gradients vanishing phenomenon. The GANDD model in this article employs the PReLU activation function to let the model adjust the gradient’s direction on its own, avoiding the issue of information loss. We also add dropout layers to prevent the overfitting problem. The optimization function of the proposed GANDD additionally incorporates Gradient Penalty as a constraint to make the entire training process more stable. Experiment results reveal that the given model configurations effectively reduce the gradient disorientation problem and therefore stabilize the training process.

For the sake of comparative study, we have Deep Neural Network (DNN), LSTM and standard GAN included in the subsequent experiments. We first examine how these DL technologies perform on conventional DDoS attacks. As reported in Table 10 and Table 11. All four deep learning technologies under investigation perform well in detecting conventional DDoS attacks on both NSL-KDD and CIC-IDS2017. Among them, the proposed GANDD has the best detection rate. Figure 8 shows the ROC curves of different DL-based DDoS detection.

Table 10. DL-based DDoS detection on NSL-KDD.

Model	TPR	FPR	F1-Score
DNN	0.924	0.068	0.929
LSTM	0.957	0.031	0.962
GAN	0.941	0.058	0.942
GANDD	0.985	0.041	0.971

Table 11. DL-based DDoS detection on CIC-IDS2017.

Model	TPR	FPR	F1-Score
DNN	0.913	0.085	0.914
LSTM	0.928	0.063	0.932
GAN	0.955	0.096	0.925
GANDD	0.983	0.053	0.963

Figure 8. ROC curves of DL-based DDoS detections on conventional DDoS attacks.

We now concentrate our attention on the impact of adversarial DDoS attacks. Adversarial DDoS traffic synthesized by the GP-WGAN is used to train the GANDD and test all four DL technologies. As reported in Table 12, the TPRs of DNN, LSTM and GAN drop dramatically, showing their inability to deal with adversarial DDoS attacks. This is because the substitute neural network’s capability of generalization is degraded by the adversarial DDoS attack’s atypical probability distribution. On the other hand, the proposed GANDD can still maintain a TPR value of 0.843. This indicates that the proposed GANDD can be an effective solution to challenging adversarial DDoS attacks. Figure 9 shows the ROC curves of different DL-based DDoS detection on adversarial DDoS attacks. GANDD’s ROC curve has a high AUC value and steep slope, indicating its superiority over DNN, LSTM and GAN.

Table 12. TPRs of DL-based DDoS detections on adversarial DDoS attacks.

	DNN	LSTM	GAN	GANDD
TPR	0.087	0.117	0.106	0.843

Figure 9. ROC curves of DL-based DDoS detections on adversarial DDoS attacks.

A more challenging and more realistic scenario is considered in the following experiment. The GANDD is trained with adversarial DDoS traffic synthesized using CIC-IDS2017 but tested with adversarial DDoS traffic synthesized using NSL-KDD. Such an experiment setting is to examine the GANDD’s ability in handling unseen adversarial DDoS attacks. The results are given in Table 13 and Figure 10. The performance of DL-based approaches degrades even further. Even the best case, LSTM, has a TPR as low as 0.074. Although the TPR of GANDD decreases, the ROC curve of GANDD also exhibits a drop in both steepness and AUC. GANDD still maintains a TPR value of 0.73. This result reflects the elusiveness of unseen adversarial DDoS attacks. However, GANDD is still considered effective compared to the other approaches.

Table 13. TPRs of DL-based DDoS detections on unseen adversarial DDoS attacks.

	DNN	LSTM	GAN	GANDD
TPR	0.022	0.074	0.066	0.713

Figure 10. ROC curves of DL-based DDoS detections on unseen adversarial DDoS attacks.

The menace and evasiveness of adversarial attacks generated by GP-WGAN are evident according to Table 6. They can breach many commonly used ML-based defense systems. As the best, RF has a TPR as low as 9.7%. Similarly, Table 12 shows that DL-based defenses are also ineffective with regards to adversarial attacks. Even the best one, LSTM, the TPR is only 11.7%. On the contrary, the proposed GANDD demonstrates its defensive capabilities with a TPR as high as 84.3%. Even for a more challenging and realistic scenario of cordoning unseen adversarial attacks, GANDD still proved effective with a TPR around 71.3% compared to the 7.4% of LSTM. We arrive at the same conclusion in examining Figure 9 and Figure 10. With a high AUC of 0.814 and 0.717, respectively, GANDD has steep ROC curves indicating that its sensitivity increases more rapidly than specificity. On the other hand, due to the low TPR rates, the ROC curves of the other DL-based approaches tend to be closer to the diagonal, implying lower sensitivity and higher specificity.

Hu et al. [22] introduced an exciting GAN-based architecture for DDoS attacks called MalGAN, neglecting the ML black-box detection model. MalGAN’s attack has excellent infiltration power except obstacle from Random Forest, where TPRs remain at 80%. Meanwhile, adversarial attacks from GP-WGAN overcome most ML defending systems including RF, as shown in Table 6. Furthermore, Hu’s research also does not propose an adequate defence model like GANDD.

Based on the original GAN model, Li et al. [20] obtained interesting results with the introduction of the “RCC Dual-GAN” featuring two separate GANs. The first model performs the data distribution reference, while the second model responds to overfitting constraints. On the other hand, we had taken a novel and radically different approach by entrenching an additional discriminator in a unified GAN model. Accordingly, the proposed GANDD becomes more comprehensive and coherent, as shown in the analysis above.

5. Conclusions

This study investigates the potential threat of a new type of DDoS attack known as adversarial DDoS attacks. The adopted GP-WGAN architecture could indeed generate legitimate-looking malicious traffic. According to experimental results, the synthesized adversarial DDoS attacks can easily penetrate ML-based detection systems, such as RF, KNN, and SVM. This phenomenon is a sobering reminder that adversarial DDoS attacks require urgent countermeasures. The GANDD architecture proposed in this study is a response to this new threat. The experimental results determine that adversarial DDoS attacks can be efficiently intercepted with an additional discriminator for capturing and tagging adversarial DDoS traffic. The GANDD can achieve TPRs of values 0.985, 0.843 and 0.713 on conventional DDoS attacks, adversarial DDoS attacks, and unseen adversarial DDoS, respectively. The proposed GANDD is considered adequate, although there is still room for improvement for unseen adversarial DDoS. The incorporation of open-set recognition technology could be a promising direction that deserves further investigation.