Machine Learning Techniques to Detect a DDoS Attack in SDN: A Systematic Review

: The recent advancements in security approaches have signiﬁcantly increased the ability to identify and mitigate any type of threat or attack in any network infrastructure, such as a software-deﬁned network (SDN), and protect the internet security architecture against a variety of threats or attacks. Machine learning (ML) and deep learning (DL) are among the most popular techniques for preventing distributed denial-of-service (DDoS) attacks on any kind of network. The objective of this systematic review is to identify, evaluate, and discuss new efforts on ML/DL-based DDoS attack detection strategies in SDN networks. To reach our objective, we conducted a systematic review in which we looked for publications that used ML/DL approaches to identify DDoS attacks in SDN networks between 2018 and the beginning of November 2022. To search the contemporary literature, we have extensively utilized a number of digital libraries (including IEEE, ACM, Springer, and other digital libraries) and one academic search engine (Google Scholar). We have analyzed the relevant studies and categorized the results of the SLR into ﬁve areas: (i) The different types of DDoS attack detection in ML/DL approaches; (ii) the methodologies, strengths, and weaknesses of existing ML/DL approaches for DDoS attacks detection; (iii) benchmarked datasets and classes of attacks in datasets used in the existing literature; (iv) the preprocessing strategies, hyperparameter values, experimental setups, and performance metrics used in the existing literature; and (v) current research gaps and promising future directions.


Introduction
With the increasing demand for high-quality multimedia content, software-defined networking (SDN) has been proposed as the future of internet architecture. In this network paradigm, the control plane (which is the brains of the network) and the data plane (which is the muscle) are decoupled [1]. SDN models include SDN controllers, as well as southbound and northbound APIs. This architecture provides a programmable and centralized network that can dynamically provision services [2]. OpenFlow (OF) is a standard and open protocol used in SDN that explains how a centralized controller configures and governs the control layer in the network. The data in SDN is kept in Mac tables and routing tables and is handled by various sophisticated switching and routing protocols. These tables are utilized to create the forwarding plane in traditional networks [3].
Today's society relies heavily on the internet, which is essential for economic transactions, education, and communication. However, along with its many benefits, the internet has experienced an increase in criminal activity, such as hacking, spreading false information, and denial-of-service (DoS) attacks. A DoS attack occurs when a legitimate service, system or network is made inaccessible to its intended users. A DDoS attack, a subcategory of DoS attacks, involves an attacker breaching multiple computing systems in order to disrupt a specific target's regular traffic [4].
Defending against DoS and DDoS attacks is more challenging in SDN than in traditional networks. These types of attacks have become significant threats to computer networks, causing a decline in network performance by consuming available resources and disabling services. An effective DoS/DDoS attack intentionally depletes resources and prevents hosts from accessing the targeted service. In SDNs, a DoS/DDoS attack can overwhelm the control plane, data plane, or control plane bandwidth, potentially bringing down the entire network. An attack on the data plane may consume all of the OpenFlow switch's limited flow table RAM, resulting in the discarding of packets and the inability to install newly received flow rules. DoS/DDoS attacks on the data plane may also involve the generation of a large number of new flows that do not correspond to flow table entries. These packets are buffered by the switch, and if the buffer fills up, the entire packet is sent to the controller rather than just the headers through packet-in messages. This can cause delays in installing new flow rules and higher communication bandwidth use [5].
The primary distinction between DoS and DDoS assaults is that DoS utilizes many internet connections to take the victim's computer network offline, whereas DDoS assaults use a network of devices controlled by the attacker. DDoS assaults are more challenging to detect and trace because they are launched from various locations, and the attack volume used is enormous. DDoS assaults are carried out differently from DoS attacks, which are often carried out via a script or a DoS tool like Low-Orbit Ion Cannon. Types of DOS attacks include buffer overflows, ICMP floods, teardrop attacks, and flooding assaults, whereas types of DDOS attacks include volumetric attacks, fragmentation attacks, application layer attacks, and protocol attacks. DDoS assaults are more destructive than DoS attacks because they involve several systems, making it more challenging for security teams and products to pinpoint the source of the attack [6].
The aforementioned examples highlight the requirement for a reliable approach to identifying DDoS assaults. DDoS assaults may be detected using a variety of approaches, including statistical analysis, ML/DL, etc. Among these, deep learning approaches are the most effective at identifying DDoS assaults. The following are shortcomings of the alternative approaches that have been studied to date: • Statistical Methods Limitations: The limitations of various DDoS detection approaches have been studied, including statistical and machine learning (ML) methods. Statistical methods are based on past network flow information, which may not accurately describe current network traffic due to evolving hostile network flows. Such techniques rely heavily on user-defined criteria, which need to be able to change dynamically in order to keep up with changes in the network. Statistical techniques such as entropy and correlation require a significant amount of computational effort, making them unsuitable for real-time detection [7]. ML methods work effectively on a small amount of data and determine the statistical properties of attacks before classifying or valuing them. However, they require routine model updates to reflect changes in attack patterns, and certain algorithms can take a very long time to test [8]. • Machine Learning (ML) Limitations: Even when applying ML principles to a tiny quantity of data, it can function quite effectively. The ML first determines the assault's statistical properties before classifying or valuing them. Additionally, it requires routine model updates in order to reflect changes in attack patterns [9]. ML techniques address this problem by decomposing it into manageable subproblems, addressing those subproblems, and then providing the full solution. ML algorithms typically require a short amount of time to train and a considerably longer amount of time to test [10].
DL techniques can effectively identify DDoS attacks, as the data can be classified and the features extracted using DL algorithms, unlike in ML which needs to extract the features in different algorithms before inserting them into the model. In today's security environment, a detection system that can handle data unavailability is a necessity. Although labels for valid traffic are frequently accessible, labels for malicious traffic are less common. DL methods are capable of extracting information from incomplete data [11], and are appropriate for recognizing low-rate assaults. To recognize low-rate assaults, historical data are necessary, which DL techniques use to discover long-term relationships of temporal patterns [12]. As a result, in circumstances where such data are available DL techniques can be very helpful. During the training phase, DL methods perform intricate mathematical operations across a variety of hidden layers and parameters [13]. Quantum computing has shown great promise in a variety of fields, including artificial intelligence (AI), cybersecurity, and medical research. Quantum computing can help AIs to solve more complicated issues by speeding up computation. It can be used with both SML and DL models for quick training or other enhancements. By addressing complicated issues that need vast datasets and are demanding to process, quantum computing can enhance the capabilities of deep learning [14,15].
Compared to other review studies in the literature, Table 1 illustrates that the majority of these studies have not provided a comprehensive evaluation of the preparation techniques, benefits, and types of attacks used in the analyzed datasets. In contrast, our systematic study presents an extensive review of various deep-learning techniques for detecting DDoS attacks. Through this research, we have identified a gap in the literature, namely, that a comprehensive evaluation of deep learning methods for DDoS detection remains lacking. Our study contributes to addressing this gap by providing a comprehensive review and analysis of the strengths and weaknesses of different deep learning approaches for detecting DDoS attacks. As such, our review provides valuable insights into the current state-of-the-art in DDoS attack detection using deep learning techniques.
We reviewed DDoS assaults detection systems based on DL techniques in this research using the SLR protocol, and offer the following findings: • Based on common criteria, modern DDoS attack detection technologies involving deep learning algorithms have been identified and grouped. • The methodology, benefits, and drawbacks of current ML/DL systems for detecting DDoS assaults have been outlined. • The different kinds of assaults in the datasets utilized in recent studies as well as the accessible DDoS benchmarked datasets have been compiled. • The core of our review was focused on data pre-processing techniques, hyperparameter adjustments, testing configurations, and the quality measures used by current ML/DL systems for DDoS attack detection. • The main purpose of the study was to identify areas for future research in this field and to highlight current research gaps.
The remainder of this review is structured as follows: the SLR protocol is explained in Section 2; Section 3 discusses current ML/DL methods that have been employed in the literature for detection of DDoS assaults; in Section 4, the methodology, advantages, and disadvantages of different studies are discussed; the available benchmarked DDoS datasets and classes of attacks in the datasets commonly used in the literature are described in Section 5; preprocessing techniques and hyperparameters are described in Section 6; in Section 7, the research gaps in the current literature are shown; finally, in Section 8, our conclusions are explained and future prospects are explored.

Systematic Literature Review (SLR) Protocol
This paper presents a systematic literature review (SLR) conducted between 2018 and 2022 focusing on detection of DDoS attacks using DL methods. The SLR method used in this study adheres to the recommendations made in [21], providing a comprehensive approach to understanding the literature on the subject. Unlike previous review papers, this study includes an analysis of the preparation techniques, advantages, and different types of attacks used in various datasets. The output of the SLR is a collection of research publications organized according to the taxonomy of the DL techniques utilized. By identifying research limitations in the body of literature, this study offers exciting new options for future research. Overall, this paper presents a rigorous and novel approach to a systematic review of DDoS attack detection techniques. The research protocol summary is shown in Figure 1, and is described in detail below.

Research Questions
The main objective of a systematic review is to address research questions by analyzing data extracted from previous studies. The research questions addressed in the present work include:

Search Strategy
An effective search strategy is essential for any systematic survey. In this study, a carefully selected set of databases was used to mine the relevant literature. Two search phases were conducted between 2018 and 2022. The first phase searched four databases: ACM, IEEE Explore, Springer, and Science Direct. The second phase added Google Scholar in order to ensure that all relevant material was included. To refine the search string, pilot research was conducted. From the search results, ten highly referenced and relevant articles were selected.
One such search term that was used in several digital libraries with little alteration was (DDoS attack detection using DL approaches OR DDoS attack detection using ML approaches OR Detection of DDoS attacks using DL OR Detection of DDoS attacks using ML). Using "filtering choices", we were able to improve the outcomes from the selected digital libraries. Figure 2 shows the flow of the various phases of the survey protocol.

Study Selection Criteria
The main objective of the research selection process was to identify relevant literature addressing the defined research questions while excluding any irrelevant material. To this end, inclusion and exclusion criteria were applied; these encompassed research papers that built upon earlier relevant studies. In stage 1, we took the first 1000 items from the second search phase and combined them with the 3039 entries from the first search phase to create 4039 entries. In stage 2, 170 duplicate entries were eliminated. After stage 2, articles were removed in accordance with their titles (3126), abstracts (581), and complete texts (118), respectively. In the end, (44) research articles were chosen. Studies that were unrelated to established research topics were eliminated using the inclusion and exclusion criteria. The following definitions describe the inclusion/exclusion criteria:

Reference Checking
The references from the (32) studies that were retained after scanning the whole manuscripts were evaluated to make sure that no significant work had been missed. The (76) papers that contributed to their conclusions were then evaluated more thoroughly based on the title, abstract, and full article using the same inclusion and exclusion criteria as previously. Articles based on titles (11), abstracts (51), and entire articles (12) were removed in the next rounds. Of the papers found via reference checking, (74) entries were removed, resulting in only two additional papers.

Data Extraction
After examining the entire manuscripts, pertinent information was collected based on our research questions. The collected information from each study was used to complete a templated form. The title, technique, datasets used, number of features, recognition of attack and genuine classes, preprocessing techniques, testing configuration for enhancement of the model, evaluation methods, advantages and disadvantages of the model, and a summary were all used to critically evaluate the final set of articles in order to condense the answers to our research questions. The fields used for data extraction are detailed in Table 2.

Most Up-to-Date ML/DL Techniques for Detecting DDoS Attacks
The field of ML is a subfield of artificial intelligence (AI) that encompasses all techniques and algorithms that allow computers to automatically learn from big datasets by applying mathematical models. Decision Tree (DT), K-Nearest Neighbor (KNN), Artificial Neural Network (ANN), Support Vector Machine (SVM), K-Means Clustering, Fast Learning Networks, Ensemble Methods, and others are the most popular ML methods used for DDoS detection in SDN (sometimes called Shallow Learning). The brief explanations of each category are as follows: • Decision Tree (DT): a fundamental supervised ML method that leverages a set of rules to classify and predict data using regression. The model is structured as a tree with nodes, branches, and leaves, where each node represents a feature or characteristic. Each leaf on the branch denotes a possible outcome or a class label, and the branch itself signifies a decision or a rule. The DT algorithm automatically selects the optimal attributes for tree construction and performs pruning to eliminate unnecessary branches and prevent overfitting [22]. • K-Nearest Neighbor (KNN): the K-Nearest Neighbor (KNN) algorithm is a simple supervised ML method that uses the concept of "feature similarity" to classify a given data sample. By determining a sample's identity based on its neighbors and how far away it is from them, KNN can effectively determine the class of a data sample. The value of the KNN algorithm's k parameter can have an impact on its performance, and selecting a k value that is too small or too large can lead to overfitting or incorrect categorization of the sample case. To improve the detection rate of attacks in the minority class, researchers using the most recent benchmark dataset, CSE-CIC-IDS2018, have applied the Synthetic Minority Oversampling Technique (SMOTE) to overcome the dataset imbalance issue when evaluating the performance of various ML algorithms, including KNN [23]. • Support Vector Machine (SVM): Support Vector Machine (SVM) is a supervised ML method that uses the max-margin separation hyperplane in n-dimensional feature space as its foundation. It can be used to solve both linear and nonlinear issues, employing kernel functions to address the latter. The goal of SVM is to first translate a low-dimensional input vector into a high-dimensional feature space using the kernel function, then to use the support vectors create an optimal maximum marginal hyperplane that serves as a decision boundary. By correctly identifying the benign and harmful classes, the SVM method can be used to identify DDoS attacks with greater efficiency and accuracy [24]. • K-Mean Clustering: the goal behind clustering is to group together sets of data that are very similar in order to divide the data into meaningful clusters or groups. One popular iterative ML technique that learns without supervision is K-Mean clustering.
Here, K denotes a dataset's total number of centroids (cluster centers). Distance is typically measured when allocating specific data points to a cluster. The main goal is to decrease the total distance between each data point and its associated centroid within a cluster [25]. • Artificial Neural Network (ANN): the functioning of the human nervous system serves as the inspiration for the supervised ML algorithm known as ANN. It consists of neurons (nodes), which are processing units, and the connections that link them together. The organization of these nodes includes an input layer, several hidden levels, and an output layer. A backpropagation algorithm is employed by ANNs as a learning method. The capacity of the ANN approach to perform nonlinear modeling by learning from larger datasets is its key benefit. However, the fundamental difficulty with training ANN models is the lengthy procedure required, as its complexity can hinder learning and result in less than ideal results [26]. • Ensemble methods: the main idea behind ensemble techniques is to learn in an ensemble fashion in order to benefit from the use of multiple classifiers. Each classifier has its own advantages and disadvantages; for example, they may be good at spotting a certain kind of attack and bad at spotting other kinds. By training several classifiers, ensemble techniques can combine several weak classifiers to create a single stronger classifier, which is typically selected using a voting mechanism [27].
DL is a type of ML used in AI that has the ability to learn from both supervised and unstructured data [28]. DL models are known as Deep Neural Networks or Deep Neural Learning, as the technology makes use of multi-layer networks. Neurons connect the levels and stand in for the mathematical calculations behind learning processes [29]. As seen in Figure 3, the three main processes that make up most ML/DL methods are: (i) the data preparation phase, (ii) te training phase, and (iii) the testing phase. The dataset is initially preprocessed for each of the suggested solutions in order to convert it into a form that the algorithm can use. Typically, this phase involves normalization and coding. The dataset may need be cleaned, which occurs during this step if necessary. Duplicate entries and entries with missing data are removed. The training dataset and testing dataset are created by randomly dividing the preprocessed data into two halves. Typically, nearly all (80%) of the initial dataset size is typically made up of the training dataset, with the remaining amount (20%) constituting the testing dataset. In the subsequent training phase, the ML or DL algorithm is taught using the training dataset. The proportion of the dataset that is used and the complexity of the model being trained affect how long it takes the algorithm to learn. Due to their intricate and sophisticated structure, DL models often require a longer training period than ML models. After training, models are tested using the testing dataset, with performance being assessed based on the predictions made by the model. In the case of DDoS detection models, this takes the form of network traffic instances being classified as either benign (normal) or attack instances. DL techniques can be divided into five groups: hybrid learning, semi-supervised learning, supervised instance learning, and supervised sequence learning. A succinct summary of each category is provided below: Supervised instance learning: Supervised Instance l = Learning uses the flow of instances [18]. For training purposes, it makes use of labeled instances. The most popular techniques in this area are: • Deep Neural n = Networks (DNN): a fundamental DL structure that allows the model to learn at multiple levels. It comprises several hidden layers, along with input and output layers. DNNs are used to simulate complex nonlinear functions. The addition of more hidden layers improves the model's abstraction level, expanding its potential. For classification purposes, the output layer consists of one fully connected layer and a softmax classifier. The Rectified Linear Unit (ReLU) function is commonly used as the activation function for the hidden layer [30,31].
• Convolutional Neural Network (CNN): a CNN is a DL structure that is well suited for image and signal data. All CNNs have an input layer, a stack of convolutional and pooling layers for feature extraction, a fully connected layer, and a softmax classifier in the classification layer. CNNs have achieved great progress in the realm of computer vision, and can perform supervised feature extraction and classification functions for DDoS detection tasks [32].
Supervised sequence learning: in supervised sequence learning, a series of flows are used; when learning from a set of inputs, this form of model keeps track of the prior input states in its memory. The most popular models of this kind include: • Recurrent Neural Networks (RNN): RNNs were developed to improve upon the capabilities of traditional feed-forward neural networks and model sequence data. Input, hidden, and output units make up an RNN, with the hidden units acting as the memory elements. In order reach a decision, each RNN unit considers both the current input and the results of prior inputs. RNNs are commonly used in a wide range of fields, such as semantic comprehension, handwriting prediction, voice processing, and human activity identification [33]. RNNs can be used for feature extraction and supervised categorization in DDoS detection. However, RNNs can only manage sequences up to a certain length before running into short-term memory problems [34]. • Long Short-Term Memory (LSTM): LSTM is a DL structure that has successfully addressed the challenges of RNNs. An LSTM network is composed of different memory cells or blocks. The following cell receives both the hidden state and the cell state through three mechanisms known as gates, specifically, forget, input, and output gates [35]. The memory blocks may choose which data to recall or ignore. A forget gate eliminates information from the current input that the LSTM no longer requires [36]. The output gate is responsible for extracting pertinent data from the current input and processing it as an output. Finally, the input gate is responsible for adding inputs to the cell state [37].
Semi-supervised learning: semi-supervised learning involves using unlabeled data in the pre-training stage of the algorithm. This approach trains a model using both labeled and unlabeled data. In this case, the features are extracted using an autoencoder and classification is performed using various deep or shallow machine learning models. AutoEncoding (AE) is a common deep-learning method that belongs to the unsupervised neural network family. By learning the best features, AE aims to match the output to the input as closely as possible. Although the dimensions of the hidden layers are often smaller than those of the input layer, an autoencoder has input and output layers of the same dimension. Symmetric encoder-decoder operation is a key aspect of AE. Stacked AE, Sparse AE, and Variational AE are three different versions of AE [13].
Other learning methods: this group includes transfer learning, in which a pre-trained model from a repository is used in a transfer learning technique [13]. In these cases, researchers use deep learning techniques train models on one attack domain before applying them to another.
This section has provided a thorough summary of the most popular ML and DL algorithms for DDoS detection systems. Figure 4 illustrates the taxonomy of current ML/DL-based DDoS detection approaches.

Methodologies, Strengths, and Weaknesses
The specifics of the most popular ML and DL algorithms used to create an effective DDoS detection model are described in this section, along with basic techniques for AIbased DDoS detection. Both supervised and unsupervised methods are used in ML and DL. In supervised algorithms, data need to be labeled prior to use. Unsupervised algorithms, on the other hand, use unlabeled data to extract important characteristics and details. The methodologies, advantages, and disadvantages of studies using these approaches are summarized in Table 3.    Combined a stacked sparse AE to learn features with a DNN for categorizing network data The results indicated an accuracy of 98.92%. The suggested approach worked well to address issues with feature learning and overfitting, as the AE was trained with random training data samples to perform feature learning and overfitting was avoided by employing the sparsity parameter Performed an offline study rather than evaluating the most recent datasets. Additionally, the suggested model could not compute the detection time.
Moha. et al. [41] Combined the LSTM and Bayes techniques The results revealed that, the performance indicators decreased only slightly with the new data, and the outcomes were positive Assaults that are unsuited for real-time applications may take the LSTM-BA longer to identify. Comparing the suggested model to the current DeepDefense approach, the accuracy was only improved by 0.16%. IP addresses were transformed into actual vectors using feature hashing, and the preprocessing time was not computed using the BOW.

RoopakM et al. [39]
Employed multi-objective optimization, namely, the NSGA approach F1-score value of 99.36% and high accuracy of 99.03%. Additionally, The outcomes demonstrated that the suggested model outperformed earlier studies. When compared to previous DL approaches, the training time was cut by an astounding eleven times The majority of the cutting-edge methods used in this article did not use the CI-CIDS2017 dataset; therefore, the analogy appears inappropriate.
Elsa et al. [42] Combined AE and RNN to produce DDoS-Net for identifying DDoS assaults in SDNs The results indicated that DDoS-Net performed better than six traditional ML techniques (DT, NB, RF, SVM, Booster, and LR) in terms of accuracy, recall, precision, and F1score. The proposed method obtained 99% accuracy and an AUC of 98.8 The dataset used offline analysis, and multiclass classification was not carried out Nugraha et al. [40] A DL-based strategy was proposed to identify sluggish DDoS assaults in SDNs using a CNN-LSTM model When compared to cutting-edge algorithms, the suggested policy-based GPDS algorithm outperformed them in terms of anti-jamming performance

Available Benchmarked DDoS Datasets and Classes of Attacks in Datasets
The datasets and attack class types utilized by the studies that were examined for DDoS attack detection are listed in Table 4. eight datasets (KDD Cup99, Kyoto 2006+, NSL-KDD, UNSW-NB15, CIC-IDS2017, CSE-CIC-IDS2018, SCX2012, and CICDDoS2019) were utilized across the majority of studies. Following is a description of these datasets.
KDD Cup99: One of the most well-known and often used datasets for IDS is KDD Cup99. It contains about five million and two million recordings for training and testing, respectively. Each recording has 41 distinct characteristics or properties, and is classified as an attack or as normal data. Four categories of attacks are established for the recordings, namely, Denial of Service (DoS), Probe, Remote to Local (R2L), and User to Root (U2R) [80].  [81] 2020 RBF Generated dataset DDoS RoopakM et al. [38] 2019 MLP, CNN, LSTM C7 DoS, Probe, R2L, U2R Mohammad et al. [41] 2019 LST, BN I2 DDoS RoopakM et al. [39] 2020 CNN, LSTM C7 DDoS ElsayedMS et al. [42] 2020 RNN, AE C9 DDoS Nugraha et al. [40] 2020 CNN, LSTM generated DDoS He et al. [78] 2020 LANN generated DDoS Chen et al. [79] 2022 CICDDoS2019: Sharafaldin et al. [86] created the CICDDoS2019 dataset (2019). More than 80 traffic characteristics were taken from the original information by using the CICFlowMeter-V3 program to extract the features. The CICDoS2019 contains typical DDoS assaults that are safe and current. This dataset, which was created using actual traffic, contains a variety of DDoS assaults created utilizing TCP/UDP protocols [87]. Table 4 lists the preprocessing techniques, hyperparameter settings, test configurations, and performance metrics employed by the current ML/DL algorithms for DDoS attack detection. At the beginning, the data are preprocessed. Preprocessing the data is essential, as it changes raw data into a structure that improves the model's capacity for learning [88]. Table 5 in this research provides an overview of preprocessing techniques employed in the body of the literature.    Hyperparameters are crucial because they directly affect how ML training algorithms behave. Prior to training the model, certain hyperparameter values must be chosen, which calls for specialized expertise and experience. There are two approaches for hyperparameter tuning, namely, manual search and automated search techniques. In a manual search, values for the hyperparameters are chosen manually. The automated search technique is similar to grid search, however, the grid search approach is more expensive. Another approach, known as a random search, has been introduced to address the grid search issue. Examples of hyperparameters include the number of epochs, batch size, learning rate, training algorithm, amount of layers, amount of neurons in each layer, etc.

Preprocessing Techniques, Hyperparameter Settings, Experimental Configurations, and Performance Metrics
An experimental setup includes information about the program, dataset, physical hardware, and other aspects of the experimentation process. As the training and testing timeframes rely on the hardware setup, it is particularly crucial. Due to the complexity of ML/DL algorithms, suitable hardware configurations are needed.
The performance indicators are the most popular measures defined in this section. For binary classification, the typical performance measurements are accuracy, recall, precision, F1-score, AUC, etc.
The confusion matrix is described as the overview of outcomes foreseen by the categorization model. It includes (True Positive TP), True Negative (TN), False Positive (FP), and False Negative (FN) [89].
The true positive rate (TPR) is determined following Equation (1) Additionally, it may be known as the recall or sensitivity [90], and ought to be as high as possible.
Precision is determined following Equation (2) by checking how many of the positive classes that the model adequately predicted are really positive [91].
Following Equation (3), accuracy is defined as the percentage of true predictions made by the model across all classes. The highest level is preferable. Its formula is as follows [91]: The FPR or False Positive Rate is shown in Equation (4) [90]; it measures the percentage of negative occurrences that the model incorrectly forecast as positive.
The percentage of positive cases incorrectly anticipated as negative cases is known as the false negative rate (FNR), and is determined as shown in Equation (5) [90] The TNR or True Negative Rate is shown in Equation (6); Specificity is another name for it. It is described as the percentage of adverse events accurately foreseen as adverse [90].
It is challenging to compare two models if one has great recall and low accuracy or vice versa. The F1-score is therefore used to compare them. It is employed to simultaneously assess memory and precision [92]. Equation (7) is used to compute the F1-score: Efficiency at different threshold levels for classification problems is known as the AUC-ROC curve. A model makes more accurate predictions if the AUC is near 1 [93].

Research Gaps in the Existing Literature
The research gaps detailed below were identified through our thorough assessment of the literature.

•
Insufficiently large datasets: due to the potential loss of reputation or money, the majority of victim organizations are reluctant to disclose information regarding attacks undertaken against them. Furthermore, there are no complete databases in the public domain that include all traffic kinds, including genuine, low rate, high rate, and flash traffic [37,39,40,42,[75][76][77]81]. Therefore, experimental settings are necessary to provide extensive datasets for the thorough validation of DDoS detection methodologies. • Access to skewed datasets: occurrences of DDoS attacks are typically highly skewed in comparison to genuine events in the datasets currently available [37,39,40,53- Insufficient effort on unknown data or zero-day attacks: when the instruction and assessment datasets contain the same traits or patterns, ML models are able to function well. However, ML-based algorithms are unable to accurately detect unknown threats in real-life situations, where attacks may be launched using novel patterns. As a result, these models must be frequently updated in order to account for novel and untested assaults [53]. • Using an offline dataset for evaluation: the majority of the research we reviewed used offline datasets to assess deep learning models [37,39,42,[55][56][57][58][59][61][62][63][64][65][66][67]69,70,72,73,75,77,78,81]. The implementation of these models in actual networks remains a work in progress. Real-time evaluation of models would be highly beneficial for adequate verification.

•
No deployment of automated real-time defense models: most DDoS assaults overwhelm the target site in a relatively short amount of time, and network managers are often unable to automatically identify and fight back against these attacks. The primary cause of this is that defense strategies themselves become susceptible to DDoS assaults based on floods. Therefore, high-speed and computationally efficient DDoS solutions are needed in order to automatically stop those attacks.

Conclusions and Future Directions
It may be quite difficult to distinguish between DDoS assaults with various rates and patterns and normal traffic. Over the years, many effective ML/DL methods for DDoS attack detection have been suggested by different researchers. Sadly, however, the applicability of these techniques is severely constrained due to attackers constantly changing their attack tactics. Findings involving the SLR protocol are evaluated and drawn from in this review in order to assess the state-of-the-art DDoS assault detection systems based on ML/DL approaches. The literature has been summarized in Section 4 in accordance with the suggested taxonomy for DDoS attack detection using ML/DL techniques, with each study's respective advantages and disadvantages listed. The accuracy rate reported in much of the literature is over 99%. Because the majority of these studies assessed their models using offline data analysis for evaluation and comparison, certain metrics for performance may vary in a real-world or production settings. In particularly, we note that existing papers have generally not employed the same DS or assessment techniques, making comparisons between their results difficult.
On the ISCX2012 dataset, the LSTM, CNN, and LSTM-Bayes techniques all showed accuracy lower than 98.8% [70,71]. On the NSL-KDD dataset, only the CNN technique in [63] demonstrated accuracy above 99%, and it required complex calculations to achieve this.
We can conclude from this study that the most commonly used preprocessing techniques are BOW, Z-score normalization, one-hot encoding, and min-max normalization.
Another conclusion of our review relates to performance metrics. Of the reviewed studies, 29 employed accuracy measurements for evaluating their techniques, compared to 22 studies each using the precision, recall, and F1-score metrics and six studies each using the FPR and AUC metrics. These findings are shown in Figure 6; it can be seen that the majority of papers did not report the testing/training time for their methodologies, despite the fact that these measurements are crucial for system implementation in real-world or production settings. With respect to future research directions, our discoveries on ML/DL techniques for DDoS attack detection point towards the following paths for further study: • Lack of actual implementation of ML/DL systems: most research focusing on analysis of these models has neglected the crucial need to evaluate the performance of these models in the real-time situations where DDoS attacks actually occur. There remains a pressing need for ML/DL models that have been verified using real-world scenarios.
• ML/DL models rely on dynamically and frequent updating: models that can be dynamically and routinely updated in accordance with new types of attacks are a necessity due to constant and rapid changes in attack patterns, and is a crucial element in today's world of quickly developing new technologies that carry with them more sophisticated threats. However, no such DL models are available in the literature. • Requirement for lightweight ML/DL models: lightweight models are necessary for networks such as the Internet of Things, MANETS, and wireless sensor networks, as these have limited computational power and memory and are highly susceptible to security threats. In the future, it is expected to become increasingly necessary to develop effective and portable DL models for these contexts. • Need for appropriate datasets: the current datasets lack diversity in terms of the types of attacks and quality of the data recordings they contain, leading to biased detection systems that are unable to identify all types of attacks. It is essential to have sufficient datasets in order to ensure accurate and effective detection models.
To close, addressing these areas of research is important in order to realize significant advances in this field and bridge the gaps that currently exist in the literature.

Conflicts of Interest:
The authors declare that they have no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: