SOE: A Multi-Objective Traffic Scheduling Engine for DDoS Mitigation with Isolation-Aware Optimization

Abstract

Distributed Denial-of-Service (DDoS) attacks generate deceptive, high-volume traffic that bypasses conventional detection mechanisms. When interception fails, effectively allocating mixed benign and malicious traffic under resource constraints becomes a critical challenge. To address this, we propose SchedOpt Engine (SOE), a scheduling framework formulated as a discrete multi-objective optimization problem. The goal is to optimize four conflicting objectives: a benign traffic acceptance rate (BTAR), malicious traffic interception rate (MTIR), server load balancing, and malicious traffic isolation. These objectives are combined into a composite scalarized loss function with soft constraints, prioritizing a BTAR while maintaining flexibility. To solve this problem, we introduce MOFATA, a multi-objective extension of the Fata Morgana Algorithm (FATA) within a Pareto-based evolutionary framework. An $ϵ$-dominance mechanism is incorporated to improve solution granularity and diversity. Simulations under varying attack intensities and resource constraints validate the effectiveness of SOE. Results show that SOE consistently achieves a high BTAR and MTIR while balancing server loads. Under extreme attacks, SOE isolates malicious traffic to a subset of servers, preserving capacity for benign services. SOE also demonstrates strong adaptability in fluctuating attack environments, providing a practical solution for DDoS mitigation.

1. Introduction

A Distributed Denial-of-Service (DDoS) attack [1] is a malicious act in which attackers leverage a large number of compromised devices to generate massive traffic floods toward a target network, rendering its services unavailable. According to report by Cloudflare [2], in 2024, more than 21.3 million DDoS attacks were mitigated, representing a 53% year-over-year increase. Some of these attacks reached traffic volumes of several terabits per second—enough to overwhelm even the most resilient network infrastructures. The consequences of DDoS attacks often entail financial losses, service disruptions, and damage to brand reputation [3]. It is estimated that the average loss per incident ranges from USD400,000 to USD 500,000, including costs related to downtime, emergency response, and mitigation efforts. Furthermore, such attacks may erode service continuity, thereby undermining an organization’s market position and customer trust in the long term.

In recent years, in response to the escalating threat of DDoS attacks, both academic and industrial communities have proposed various defense mechanisms to mitigate their destructive impact on critical infrastructure. Commonly adopted techniques include traffic scrubbing [4], rate limiting [5], blackhole routing [6], Content Delivery Networks (CDNs) [7], and Web Application Firewalls (WAFs) [8]. Specifically, traffic scrubbing relies on high-performance equipment to identify and filter malicious traffic, rate limiting constrains abnormal access behaviors by setting thresholds for connection frequency or bandwidth, blackhole routing diverts suspicious traffic to null addresses to prevent resource exhaustion, CDNs alleviate pressure on origin servers by caching content at edge nodes, and WAFs are designed to detect and intercept application-layer attacks. These methods have proven effective to some extent in mitigating conventional attacks and collectively form the foundational framework of today’s DDoS defense systems.

However, the continuous evolution of DDoS attack techniques poses significant challenges to existing defense systems. First, attackers now adopt advanced evasion techniques that mimic legitimate user behavior, making it difficult for traditional signature-based detection mechanisms to accurately identify malicious requests. Second, it has become increasingly difficult to distinguish benign traffic from malicious traffic in real time, especially under the prevalence of application-layer attacks, where their behavioral patterns tend to converge, further diminishing detection accuracy. Moreover, the widespread deployment of Internet of Things (IoT) devices—many of which suffer from inherent security vulnerabilities—has enabled attackers to form massive botnets [9]. These botnets can launch high-intensity distributed attacks within seconds, generating instantaneous traffic surges that can swiftly cripple targeted services. More critically, once an attack penetrates the defense perimeter, existing mechanisms often lack the adaptability and dynamic response capacity required for timely mitigation and recovery, leading to prolonged service outages. These challenges highlight the insufficiency of traditional defenses in meeting the high availability and adaptability demands of modern network environments, thereby underscoring the urgent need for intelligent, multi-objective, and systematically adaptive DDoS mitigation solutions.

To address the aforementioned challenges, we propose a multi-objective optimization-based scheduling strategy named SchedOpt Engine (SOE), which aims to achieve efficient trade-offs among several conflicting DDoS defense objectives. Unlike conventional optimization strategies that focus on a single objective—such as maximizing benign traffic throughput or entirely discarding malicious traffic—SOE performs holistic modeling to identify the optimal balance between security and availability, thereby ensuring robust and efficient system performance under attack conditions. Multi-objective optimization is particularly crucial in DDoS scenarios. For example, traditional threshold-based filtering mechanisms may block part of the attack traffic but risk discarding legitimate user bursts. In contrast, SOE is network state-aware and dynamically adjusts its scheduling policies to preserve genuine user requests to the greatest extent possible. Specifically, SOE formulates traffic scheduling under DDoS conditions as a multi-objective optimization problem constrained by limited resources, accurately capturing the coexistence dynamics of benign and malicious traffic and converting them into a solvable scheduling task. Unlike approaches that rely on a single node, SOE makes localized and intelligent scheduling decisions across multiple servers, laying a strong foundation for enhancing attack resilience in distributed environments. Additionally, we enhanced the original FATA algorithm as a multi-objective variant named MOFATA, which drives the refined decision-making capabilities of SOE. The designed composite fitness function simultaneously optimizes three core metrics—the benign traffic acceptance rate (BTAR), malicious traffic interception rate (MTIR), and server load-balancing degree—while also reinforcing the system’s capability to isolate malicious traffic. This method demonstrates empirical superiority over traditional single-objective or static strategies. For instance, threshold-based filtering may block attack traffic but often suffers from high false-positive rates, mistakenly discarding legitimate requests. In contrast, SOE powered by MOFATA adaptively responds to changing network conditions and dynamically tunes its scheduling strategies to strike an optimal balance between security and performance. It significantly improves attack resilience while ensuring efficient resource utilization, outperforming traditional scheduling and filtering techniques.

We conducted extensive simulations across various realistic attack scenarios to validate the effectiveness and scalability of SOE. The results show that SOE significantly improves BTAR, successfully intercepts a large volume of malicious traffic, and maintains balanced loads across servers in networks of varying scales. When facing high volumes of malicious traffic, SOE is capable of isolating the attack load to a small subset of nodes, thereby preserving the service capacity of the majority of servers. Furthermore, SOE exhibits rapid responsiveness, promptly adjusting strategies in response to highly volatile malicious traffic to maintain overall system stability. This scenario-driven and systematic evaluation provides theoretical insights for real-world system design, suggesting that adopting the proposed adaptive scheduling strategy can significantly enhance the DDoS resilience of critical infrastructure.

Key Contributions of This Work:

• We propose the SchedOpt Engine (SOE), a scheduling framework that focuses on maintaining system operability through intelligent traffic allocation, even when malicious traffic cannot be fully intercepted. The underlying MOFATA algorithm provides powerful multi-objective optimization capabilities, significantly enhancing scheduling performance.
• We design a composite loss function that jointly considers benign traffic protection and malicious traffic isolation. Even in the absence of complete interception, the isolation mechanism safeguards a majority of system resources, thereby improving both the benign traffic acceptance rate (BTAR) and the malicious traffic interception rate (MTIR).
• SOE exhibits excellent adaptability, maintaining stable scheduling and promptly responding to malicious traffic surges, which ensures continuous and stable system operation under volatile attack conditions.

Organization of the Paper:

The remainder of this paper is organized as follows. Section 2 reviews related work on network traffic scheduling and multi-objective optimization algorithms. Section 3 models the DDoS-aware traffic scheduling problem, introduces the proposed fitness function, and elaborates on the construction of the MOFATA algorithm. Section 4 conducts various algorithmic analyses on MOFATA, including time complexity analysis, scalability analysis, and parameter sensitivity analysis. Section 5 presents comprehensive experiments to evaluate the BTAR, MTIR, and server load balancing under varying attack intensities and resource constraints and assesses the responsiveness and stability of SOE in dynamic environments. Section 6 concludes the paper by summarizing key findings and discussing potential future research directions.

2. Related Work

In recent years, network scheduling technologies have been extensively studied to address issues such as traffic allocation [10], load balancing [11], and quality of service (QoS) [12] assurance under constrained network resources. Early-stage scheduling algorithms primarily focused on single-objective optimization, such as maximizing throughput or minimizing latency. Algorithms such as the round-robin [13], weighted round-robin [14], and shortest queue-first [15] algorithms have been widely adopted in distributed server and data-center environments. With the advent of cloud computing and large-scale networks, researchers have shifted their focus to dynamic load distribution across multiple nodes, leading to the development of distributed load balancing [16] and adaptive scheduling strategies [17]. In DDoS defense systems, network scheduling—particularly under Software-Defined Networking (SDN) [18,19] architectures—has emerged as a key enabler for dynamic defense and optimized resource allocation. The centralized control and global visibility provided by SDN offer a solid foundation for the flexible deployment and rapid response of traffic scheduling algorithms. Current research focuses on the integration of multidimensional scheduling strategies with dynamic path planning to identify and redirect malicious traffic, thereby mitigating its impact on critical resources. For instance, Ref. [20] proposed a DDoS mitigation scheme that utilizes multidimensional metrics—including residual bandwidth, flow table entries, and path length—for path planning, which outperforms traditional methods such as the Equal-Cost Multi-Path (ECMP) [21] and K-Shortest Paths (KSP) [22] algorithms. Additionally, Ref. [23] employed adaptive scheduling of data collection and detection tasks, which not only reduces resource consumption but also significantly improves detection timeliness and accuracy. These studies demonstrate the significant potential of SDN in fine-grained traffic control, improved network resilience, and dynamic DDoS mitigation, thereby laying the technical foundation for the development of next-generation network security systems with intelligent scheduling capabilities.

While the aforementioned scheduling techniques perform well under typical conditions, they face multiple challenges in the context of DDoS attacks. First, malicious traffic often masquerades as legitimate requests, and traditional upstream identification and scrubbing mechanisms are insufficient for complete detection, resulting in a mixed stream of benign and malicious traffic entering the scheduling module. This uncertainty substantially increases the complexity of devising effective scheduling strategies. Second, due to constraints such as limited server processing capacity and network bandwidth, the system must perform efficient scheduling under traffic overload conditions. This introduces a two-fold challenge: ensuring quality of service for benign traffic while minimizing resource consumption by malicious traffic—two inherently conflicting goals. In addition, the system must also maintain load balancing across servers to avoid local resource bottlenecks. Under extreme attack intensities, it must further apply traffic isolation mechanisms to direct malicious traffic toward a small subset of nodes, thereby preserving service availability on the remaining servers. Consequently, the scheduling problem evolves into a complex multi-objective optimization task involving dynamic trade-offs among four conflicting objectives: maximizing benign traffic transmission, minimizing malicious traffic occupancy, improving server load balancing, and isolating malicious traffic. This problem is further aggravated by the highly dynamic nature of DDoS attacks, where the proportion, pattern, and distribution of malicious traffic frequently fluctuate. Traditional static or rule-based scheduling methods often struggle to adapt in a timely manner. As a result, scheduling under DDoS conditions is no longer a single-objective optimization problem but a typical multi-objective, constrained, and dynamic optimization challenge. To address this, the introduction of multi-objective optimization algorithms offers a promising direction for achieving flexible strategy trade-offs and adapting dynamically to system state changes, thereby improving both scheduling efficiency and system resilience while preserving benign traffic. This also explains the growing interest in multi-objective evolutionary optimization algorithms within the domains of network scheduling and DDoS defense in recent years.

Multi-objective optimization (MOO) serves as a foundational methodology for addressing problems involving multiple conflicting objectives. Such problems generally do not have a single optimal solution but, instead, yield a set of Pareto-optimal solutions that reflect trade-offs among competing objectives. Traditional scalarization-based methods, such as weighted sum and $ϵ$-constraint techniques, are relatively easy to implement but suffer from limitations when dealing with non-convex Pareto fronts. In contrast, population-based evolutionary algorithms such as NSGA-II [24] and SPEA2 [25] utilize non-dominated sorting and diversity preservation mechanisms, exhibiting strong adaptability and search capability, particularly for high-complexity, nonlinear optimization problems. In the field of network security, Roopak et al. [26] proposed a multi-objective feature selection approach for IoT-based DDoS detection, employing an enhanced NSGA-II algorithm combined with a jumping gene operator. The method optimizes six criteria: feature relevance, redundancy, the number of features, classification accuracy, recall, and precision. Experimental results showed that nearly 90% of the features could be eliminated while achieving a detection accuracy of 99.9%. Sun et al. [27] explored the application of Pareto optimization in multi-objective reinforcement learning and proposed the Pareto Defense Strategy Selection Simulator (PDSSS), which integrates multi-objective zero-sum games with reinforcement learning to assist security agents in selecting optimal strategies across multiple defense objectives. The results indicated that scalarization strategies based on the satisficing trade-off method (STOM) outperformed traditional linearly weighted approaches and significantly improved policy decision-making. In summary, network scheduling in the DDoS defense domain is shifting from traditional single-objective approaches to multi-objective, adaptive, and intelligent paradigms. Multi-objective optimization algorithms, with their remarkable ability to handle high-dimensional, complex scenarios, are increasingly becoming core tools for countering large-scale network attacks and constructing resilient scheduling frameworks. This trend also provides a solid theoretical and methodological foundation for the design of future intelligent network defense systems.

3. Problem Formulation and Methodology

3.1. Problem Modeling

When facing DDoS attacks, the complexity of the traffic scheduling problem primarily lies in maximizing the service delivery of benign traffic under limited network resources while suppressing the performance degradation caused by malicious attacks. To address this, the DDoS traffic scheduling problem is formulated as a variant of the multidimensional bin-packing or multiple-knapsack problem, which is recognized as a classic NP-hard problem. In this model, each server is abstracted as a bin with a fixed capacity (C), and each traffic source is treated as an item to be packed. Each item comprises both benign traffic, which is assigned a positive value, and malicious traffic, which is penalized as a negative value; however, both consume limited server resources. The core objective of the model is to pack as many high-value (benign) items as possible under constrained server capacities while minimizing the impact of malicious traffic and simultaneously achieving high load balancing and strong traffic isolation capability.

In our model, consider a network with N traffic sources, where each source (i) carries a benign traffic volume ($L_i$) and a malicious traffic volume ($A_i$). In a multi-server scheduling scenario, suppose there are M servers, each with identical service capacity denoted by C. When traffic source i is assigned to server j (denoted as $x_i=j$), server j receives total incoming traffic of $L_i+A_i$ from source i. Due to capacity limitations, each server can serve, at most, C units of benign traffic; any excess beyond this capacity will be discarded. The topological diagram is shown in Figure 1.

The formalized objective function is defined as Equation (1). This objective reflects the design principle of prioritizing benign traffic under constrained resources while suppressing the servicing of malicious traffic under the constraint that each traffic source is assigned to exactly one server.

$$max\sum _{j=1}^{M}min\left(C,\sum _{i:x_i=j}{L}_i\right),$$

(1)

where the $min\left(\right)$ function ensures that the amount of benign traffic served by each server does not exceed its physical capacity (C).

As NP-hard problems have no known polynomial-time exact solutions and exhibit exponential growth in computational complexity with increasing problem size—especially in the presence of multiple objectives and complex constraints—exact algorithms are theoretically infeasible and practically limited by computational time and resource constraints. As a result, heuristic, metaheuristic, and approximation algorithms have emerged as effective alternatives for tackling such problems. These algorithms are capable of providing high-quality approximate solutions within reasonable computational time, striking a balance between efficiency and accuracy and, thus, meeting the demands of complex system optimization in practical engineering contexts. To this end, this study introduces a heuristic search method aimed at fast convergence, with the goal of identifying near-optimal traffic allocation strategies that quickly determine scheduling decisions while satisfying key DDoS defense objectives—namely, maximizing benign traffic throughput, balancing server loads, and minimizing the service of malicious traffic.

3.2. Fitness Function

To accurately evaluate the effectiveness of each traffic allocation scheme (X), a composite fitness function is constructed, integrating primary performance objectives with several auxiliary constraints. The core component of the function measures the actual volume of benign traffic successfully served by each server and is defined as Equation (2). This objective function captures the fundamental goal of maximizing benign traffic throughput under limited-resource conditions.

$$F_{\mathrm{normal}}\left(X\right)=\sum _{j=1}^{M}min\left(C,\sum _{i:x_i=j}{L}_i\right),$$

(2)

where C denotes the maximum processing capacity of each server, $L_i$ represents the benign traffic volume of request i, and $x_i$ denotes the index of the server assigned to request i.

To enhance the overall efficiency of resource allocation, a load imbalance penalty term is introduced to prevent the overloading of certain servers while others remain underutilized. The load imbalance degree (LID) is defined as Equation (3). In the fitness function, this term is penalized by subtracting $\alpha ·$LID, thereby encouraging balanced traffic distribution and promoting trade-offs between system performance and fairness.

$$\mathrm{LID}={\displaystyle \frac{\underset{j}{max}\mathrm{Load}j-\underset{j}{min}\mathrm{Load}j}{\frac{1}{M}\sum {j=1}^M{\mathrm{Load}}_j}},$$

(3)

$$\mathrm{Load}j=\sum i:x_i=j(L_i+A_i),$$

(4)

where Equation (4) denotes the total incoming traffic assigned to server j and $A_i$ represents the malicious traffic volume of request i.

Although malicious traffic contributes no positive value, its consumption of resources can significantly impair the service quality of benign traffic. To address this, two additional penalty terms are introduced: The first is a penalty for insufficient benign traffic delivery, defined as Equation (5). The second is a penalty for malicious traffic serviced by the system, which measures the degree to which malicious requests are processed, defined as Equation (8). This term quantifies the portion of malicious traffic served by server j and is incorporated into the overall fitness function via the $\beta$ coefficient to discourage allocation strategies that respond excessively to malicious requests.

$$P_{\mathrm{legit}}\left(X\right)=w_{\mathrm{legit}}·{\left[max\left(0,T_{\exp }-T_{\mathrm{legit}}\left(X\right)\right)\right]}^2,$$

(5)

$$T_{\mathrm{legit}}\left(X\right)=\sum _{j=1}^{M}min\left(C,\sum _{i:x_i=j}{L}_i\right),$$

(6)

$$T_{\exp }=min\left(\sum _{i=1}^N{L}_i,M·C\right),$$

(7)

where, $T_{\mathrm{benign}}\left(X\right)$ denotes the actual amount of benign traffic served under allocation scheme X, and $T_{\exp }$ represents the expected upper bound of service. The coefficient $w_{\mathrm{legit}}$ is a tunable weight used to enhance the optimization drive toward preserving benign traffic.

$$P_{\mathrm{malicious}}\left(X\right)=max\left(0,min(C,L_j+A_j)-min(C,L_j)\right).$$

(8)

To further strengthen the system’s resilience against attacks, an isolation reward term is introduced. Under resource-constrained conditions, an ideal scheduling strategy should concentrate attack traffic on a small subset of nodes to preserve the service capacity of critical servers for benign requests.

The ratio of benign traffic served by server j is defined as Equation (9). To quantify the isolation effect, the a reward term is designed as Equation (10). This metric captures the disparity in service purity among servers—a greater difference indicates that malicious traffic is effectively concentrated on specific nodes, which benefits the overall scheduling outcome. The significance of this term is controlled by the weight coefficient ($\delta$).

$$R_j={\displaystyle \frac{min\left(C,{\sum }_{i:x_i=j}{L}_i\right)}{min\left(C,{\sum }_{i:x_i=j}(L_i+A_i)\right)+\epsilon }},$$

(9)

where $ϵ$ is a small constant to prevent division by zero.

$$F_{\mathrm{iso}}=-\left({\max }_j{R}_j-{\mathrm{avg}}_j{R}_j\right).$$

(10)

Accordingly, the final fitness function is expressed as Equation (11). This fitness function not only evaluates the service capability for benign traffic but also systematically integrates key factors such as load balancing, attack suppression, and traffic isolation. It thereby constructs a unified and flexible evaluation framework that serves as a theoretical and algorithmic foundation for intelligent scheduling in DDoS environments.

$$F\left(X\right)=F_{\mathrm{normal}}\left(X\right)-\alpha ·DI-\beta ·P_{\mathrm{attack}}\left(X\right)-\gamma ·P_{\mathrm{legit}}\left(X\right)-\delta ·F_{\mathrm{iso}},$$

(11)

where weighting parameters $\alpha$, $\beta$, $\gamma$, and $\delta$ can be tuned experimentally to achieve optimal performance.

3.3. Methodology

3.3.1. Fata Morgana Algorithm (FATA)

The Fata Morgana Algorithm (FATA) [28] is adopted as the core optimization engine in the SOE framework. The FATA algorithm is a heuristic optimization algorithm inspired by the physical phenomenon of Fata Morgana mirages. It employs a two-tiered search strategy based on the behavior of light propagation, combining population-level and individual-level searches to enhance global exploration and local convergence efficiency. The algorithm operates in two phases. In the first phase, inspired by definite integral principles, it simulates the propagation of multiple light beams through a non-uniform medium to perform global guidance over the entire population. A dynamic light-filtering mechanism is introduced at this stage to evaluate the fitness of beam paths within the objective space, thereby strengthening the algorithm’s population-wide exploratory capability. In the second phase, trigonometric principles are employed to simulate light refraction and reflection at boundary interfaces, guiding individuals to adjust their positions along new directions and thereby improving the precision and convergence speed of local search. Through this physical analogy mechanism, the FATA algorithm achieves an effective synergy between global exploration and local exploitation during the search process, offering efficient solution-space navigation for complex multi-objective scheduling problems.

The FATA algorithm simulates the formation process of Fata Morgana mirages to construct its population-based search mechanism, as shown in Figure 2. As illustrated in Figure 2, a ship emits two types of light in such a phenomenon. Most light (“other light” in Figure 2) does not undergo significant refraction during propagation and, thus, fails to produce visible mirages. In contrast, a distinct class of light becomes strongly refracted when passing through air layers with density gradients, ultimately forming observable mirage images. This light is referred to as mirage light in this study. Effective discrimination between these two light types is essential for identifying the global optimum ($x_{\mathrm{best}}$) during the optimization process. To this end, the FATA algorithm introduces a light-beam quality evaluation mechanism based on definite integral theory to assess the overall adaptability of the population. First, the FATA algorithm calculates the fitness of each individual according to the defined fitness function and ranks the population accordingly, as shown in Figure 3. Then, following the principles of definite integration (see Equations (12) and (13)), the FATA algorithm performs integration over the individual fitness curve ($f\left(x\right)$) to compute the population’s adaptive area (S) and uses Equation (14) to select high-quality individuals for the subsequent evolutionary process. As shown in Figure 2 and Figure 4, the FATA algorithm applies differentiated strategies to distinct light populations: For a random number ($\mathrm{rand}>P$), the light group is considered to have traversed a non-uniform medium and, thus, possesses guiding capability. In this case, the population is re-initialized using Equation (15) to enhance search diversity. Otherwise, the light is assumed to have followed non-refracted paths, and the algorithm invokes the individual search strategy derived from light propagation principles to update positions and guide the population toward locally optimal regions.

$$y=f\left(x\right)=\sum _{j=0}^n{c}_j{\phi }_{j}x,$$

(12)

where $c_j$ and ${\phi }_j$ are parameters.

$$s={\int }_a^{b}f\left(x\right)dx\approx {\displaystyle \frac{b-a}{n}}·\left({\displaystyle \frac{y_0+y_1}{2}}+{\displaystyle \frac{y_1+y_2}{2}}+\dots +{\displaystyle \frac{y_{n-1}+y_n}{2}}\right).$$

(13)

This is the population quality fitting function ($f\left(x\right)$), with points on the curve expressed as $(x_i,y_i)$ and $i\in [1,n]$.

$$P={\displaystyle \frac{S-S_{worst}}{S_{best}-S_{worst}}},$$

(14)

where P is the quality factor of the light population, $S_{worst}$ represents the quality of the worst population, and $S_{best}$ represents the quality of the best population.

$$x_i^{\mathrm{next}}=L_b+(U_b-L_b)·rand,rand>P,$$

(15)

where x is the light individual, $x^{next}$ is the new individual, $U_b$ represents the upper limit of the individual position, and $L_b$ represents the lower limit of the individual position.

In the FATA algorithm, the light propagation principle is executed after the mirage light-filtering phase and serves as the core strategy for individual-level search. It is designed to facilitate local exploitation within the search space and uncover potential local optima. As illustrated in Figure 4, light selected from the filtering stage is emitted from the source point and undergoes a sequence of refraction and reflection behaviors. This process defines three core local search mechanisms in the FATA algorithm: Refraction Phase I, Refraction Phase II, and Total Internal Reflection. These behaviors collectively enable the algorithm to coordinate exploration and convergence within local neighborhoods.

Refraction Phase I. In the initial phase, the individual light (x) propagates from an optically dense medium into a less dense medium, undergoing its first refraction due to the non-uniform refractive index of the environment. This process alters the light’s direction vector and propagation intensity, which corresponds to the generation of a new candidate solution within the search space. As illustrated in Figure 5, the incidence angle ($i_1$) is smaller than the refraction angle ($i_2$), indicating an outward expansion in the search direction. In this phase, the FATA algorithm updates the individual’s position based on Equations (16), (17) and (20) and applies a partial reflection strategy to modulate the search range.

$$x_i^{\mathrm{next}}=x_{\mathrm{best}}+{\mathbf{x}}_i·Par{a}_1,rand\le P\mathrm{and}rand<q,$$

(16)

where $x_{best}$ is the current best individual.

$$Par{a}_1={\displaystyle \frac{sin\left(i_1\right)}{C·cos\left(i_2\right)}}=tan\left(\theta \right),$$

(17)

where $i_1$ is the incidence angle and $i_2$ is the refraction angle.

Refraction Phase II. After the initial refraction phase, the light undergoes a second refraction at a randomly selected position to further adjust its propagation direction. As illustrated in Figure 6, the incidence angle ($i_3$) remains smaller than the refraction angle ($i_4$), and the refraction index parameter (Para2) is dynamically adjusted in response to the continuous change in medium density. In this strategy, the FATA algorithm generates a new individual ($x_{\mathrm{next}}$) based on the current light ($x_f$) and a randomly selected individual ($x_{\mathrm{rand}}$) from the search space, with the aim of enhancing local perturbation and search diversity. This process is described by Equations (18)–(20).

$$x_i^{\mathrm{next}}=x_{\mathrm{rand}}+\left[0.5·(\alpha +1)(U_b-L_b)-\alpha x_i\right]·Par{a}_2,rand\le P\mathrm{and}rand\ge q,$$

(18)

where $x_r \mathrm{and}$ denotesrandom individuals and $\alpha$ is the reflectance of the reflection strategy, which controls the pattern of change in the light individual.

$$Par{a}_2={\displaystyle \frac{cos\left(i_5\right)}{C·sin\left(i_6\right)}}={\displaystyle \frac{1}{tan\left(\theta \right)}},$$

(19)

where $i_5$ is the incidence angle and $i_6$ is the reflection angle.

$$q={\displaystyle \frac{fi{t}_i-fi{t}_{\mathrm{worst}}}{fi{t}_{\mathrm{best}}-fi{t}_{\mathrm{worst}}}},$$

(20)

where q is the individual quality factor, $fi{t}_i$ represents the fitness of the current individual (x), $fi{t}_{worst}$ represents the fitness of the worst individual, and $fi{t}_{best}$ represents the fitness of the best individual.

Total Internal Reflection. When the refraction angle gradually increases beyond the critical angle, the light undergoes total internal reflection within the non-uniform medium and is redirected in the opposite direction. This mechanism constitutes the third-phase search strategy in the FATA algorithm, encouraging the population to migrate toward unexplored regions, thereby enhancing the algorithm’s global search capability. As shown in Figure 7, the incidence angle ($i_5$) equals the reflection angle ($i_6$), demonstrating the directional symmetry of the reversal. In the figure, point $O(x_0,0)$ denotes the center of the search interval $[Lb,Ub]$, and points E and F represent the horizontal projections of the incidence and reflection points, respectively. Under this strategy, individual x is transformed into $x_{\mathrm{next}}$ and redirected in the opposite direction to ensure escape from local optima. This process is defined by Equations (21)–(24).

$${\mathbf{x}}_i^{\mathrm{next}}={\mathbf{x}}_f=0.5·(\alpha +1)(U_b+L_b)-\alpha \mathbf{x},$$

(21)

where $x_f$ is the individual reflected by the total internal reflection strategy.

$$\alpha ={\displaystyle \frac{F}{E}},$$

(22)

where E and F are the distances of the incident and refracted light to the horizontal plane, respectively.

$${\mathbf{x}}_0-{\mathbf{x}}_f={\displaystyle \frac{\mathbf{F}·(\mathbf{x}-{\mathbf{x}}_0)}{E}},$$

(23)

$${\mathbf{x}}_0={\displaystyle \frac{U_b-L_b}{2}}+L_b={\displaystyle \frac{U_b+L_b}{2}}.$$

(24)

3.3.2. MOFATA

As the FATA algorithm was originally designed as a single-objective optimization algorithm, it lacks the capacity to simultaneously address multiple critical goals in complex network defense tasks—such as preserving benign traffic, intercepting malicious flows, and balancing resource loads. To overcome this limitation, we extend its original loss formulation to a four-dimensional objective vector, which shown in Equation (25), representing distinct dimensions of system performance. Specifically, $f_1\left(x\right)$ measures the effectiveness of benign traffic servicing, $f_2\left(x\right)$ quantifies the degree of load imbalance across servers, $f_3\left(x\right)$ and $f_4\left(x\right)$ capture the proportion of benign traffic and malicious traffic that is allowed to pass through, and $f_5\left(x\right)$ assesses the system’s ability to isolate malicious traffic. This vectorized formulation unifies multiple performance metrics into a dimensionless mathematical representation, allowing the problem to be rigorously addressed under a multi-objective optimization framework.

$$\mathbf{f}\left(x\right)=[f_1\left(x\right),f_2\left(x\right),f_3\left(x\right),f_4\left(x\right),f_5\left(x\right)].$$

(25)

In SOE, each candidate solution is initially represented as a discrete N-dimensional vector ($x\in \mathsf{\Omega }$), as shown in Equation (26), indicating the assignment of traffic source i to one of M servers. This discrete representation faithfully reflects the inherently discrete nature of real-world resource allocation decisions. However, to leverage the efficient global search capabilities of the FATA algorithm in continuous domains, we relax the discrete decision variables by mapping each $x_i$ to a continuous variable defined over the interval of $[1,M]$. The search and update processes are performed in this relaxed continuous domain, after which the solution is discretized via rounding to generate executable scheduling schemes.

$$x=[x_1,x_2,\dots ,x_N],x_i\in 1,2,\dots ,M.$$

(26)

To achieve effective trade-offs among multiple conflicting objectives, the concept of Pareto optimality is introduced. A multi-objective solution set is constructed through a non-dominated sorting mechanism. Specifically, in the continuous search domain, each candidate solution (x) corresponds to an objective vector as defined in Equation (27).

$$F\left(x\right)=\left(f_1\left(x\right),f_2\left(x\right),\dots ,f_m\left(x\right)\right),$$

(27)

where each objective function ($f_i\left(x\right)$) represents performance dimensions such as benign traffic assurance, malicious traffic interception, load balancing, and attack isolation.

According to the classical Pareto dominance rule, for any two solutions (x and y), x is said to dominate y (denoted by $x\prec y$) if Equations (28) and (29) hold.

$$f_i\left(x\right)\le f_i\left(y\right)\mathrm{for}\mathrm{all}i,$$

(28)

$$f_j\left(x\right)<f_j\left(y\right)\mathrm{for}\mathrm{at}\mathrm{least}\mathrm{one}j.$$

(29)

To enhance the algorithm’s ability to discriminate fine-grained performance differences and avoid premature convergence or policy degeneration, we incorporate the concept of $ϵ$-dominance. Under this strategy, a solution x $ϵ$-dominates solution y only if Equation (30) holds:

$$f_i\left(x\right)\le f_i\left(y\right)+ϵ\mathrm{for}\mathrm{all}i,\mathrm{and}\exists j:f_j\left(x\right)<f_j\left(y\right)-ϵ.$$

(30)

Based on these principles, non-dominated sorting is applied to partition the population into multiple Pareto fronts. Tournament selection and crowding distance mechanisms are subsequently used to maintain solution diversity and promote a well-distributed set of high-quality solutions across all objectives. This hybrid approach effectively combines the discrete characteristics of the scheduling decision with the powerful search capability of continuous optimization. The integration of $ϵ$-dominance further enhances the global exploration capacity and fine-resolution performance discrimination of the proposed MOFATA algorithm. The pseudocode of MOFATA is presented as Algorithm 1:

Algorithm 1 MOFATA: Multi-objective Extension Based on FATA

Require:  $PopSize$: population size; $MaxGen$: maximum generations; $Fitness\left(x\right)=[f_1\left(x\right),f_2\left(x\right),\dots ,f_5\left(x\right)]$: multi-objective fitness function; $\epsilon$: $\epsilon$-dominance threshold; M: number of servers; FATA() function (baseline optimizer) Ensure:  Final Pareto front of discrete scheduling vectors   1: Initialize population $\{x_i\in {\mathbb{R}}^N\mid i=1,\dots ,PopSize\}$ via FATA()   2: for all $x_i$ in population do   3:     Evaluate fitness vector $f\left(x_i\right)$   4: end for   5: Perform non-dominated sorting → obtain initial Pareto front $P_0$   6: for $t=1$ to $MaxGen$ do   7:     Execute one FATA generation to update population $\left\{x_i\right\}$   8:     for all $x_i$ in population do   9:         Discretize $x_i$ by rounding to nearest integer in ${[1,M]}^N$ 10:         Evaluate multi-objective fitness $f\left(x_i\right)$ 11:     end for 12:     Apply $\epsilon$-dominance filtering and non-dominated sorting 13:     Use crowding distance to maintain diversity 14:     Update Pareto front with current non-dominated individuals 15: end for 16: return Final Pareto front

4. Algorithmic Analysis

4.1. Time Complexity Analysis

Unlike classical deterministic optimization algorithms, metaheuristic and multi-objective heuristic methods such as MOFATA do not provide theoretical guarantees of convergence to the global optimum within a fixed number of iterations. This inherent uncertainty arises from the reliance on stochastic search operators—such as mutation, crossover, and random selection—as well as the highly complex and rugged nature of the solution landscape. As a result, the number of generations required to reach an optimal or near-optimal solution can vary dramatically across problem instances and is generally unpredictable. Without a provable convergence bound, it becomes infeasible to derive a closed-form expression for the overall algorithmic time complexity from initialization to the global optimum. Therefore, the standard practice in the analysis of such algorithms is to focus on the computational complexity per generation. By decomposing the main computational components executed in each generation, we can provide a detailed estimation of MOFATA’s per-generation time complexity. This approach enables a practical evaluation of the algorithm’s scalability and helps to identify potential computational bottlenecks in large-scale or high-concurrency deployment scenarios.

Let P, N, and M denote the population size, the number of traffic sources (decision variables), and the number of servers, respectively. The key computational components per generation are summarized as follows:

• Fitness Evaluation: Each individual represents a scheduling decision vector of length N, mapping traffic sources to servers. Evaluating a single individual involves computing four objectives: BTAR, MTIR, Load Imbalance Degree (LID), and traffic isolation ability. Each of these objectives requires the aggregation of server-level statistics over all N sources. Thus, the per-individual cost is $\mathcal{O}\left(N\right)$, and the total cost for the population is $\mathcal{O}(P·N)$.
• Non-Dominated Sorting: MOFATA adopts the fast non-dominated sorting algorithm introduced in NSGA-II, which has an average-case complexity of $\mathcal{O}(M·P^2)$, where M is the number of objectives. This operation identifies Pareto fronts used for survival selection.
• Crowding Distance Assignment: For each objective, solutions are sorted to compute crowding distances, contributing $\mathcal{O}(M·PlogP)$ to the per-generation cost.
• Selection and Variation: Tournament selection and crossover/mutation operations scale linearly with P, yielding $\mathcal{O}\left(P\right)$ complexity, which is relatively negligible compared to the above.

Therefore, the total time complexity per generation is approximated as Equation (31). This theoretical estimation forms the basis for the empirical scalability evaluation presented in Section 4.2.

$$\mathcal{O}(P·N+M·P^2).$$

(31)

4.2. Scalability Analysis

To validate the theoretical complexity estimates presented in Section 4.1, we conducted a series of scalability experiments under varying problem sizes. Specifically, we examined how increasing the number of traffic sources (N) and the number of servers (M) affects the computational cost and optimization performance of MOFATA. We tested MOFATA with a fixed population size of $P=30$ and varied $N\in \{5000,10,000,20,000,30,000\}$ and $M\in \{10,100,500,1000\}$. Each configuration was run for 50 generations, and we recorded the average per-generation runtime, as well as the final values of the BTAR and MTIR, as well as solution-set quality metrics such as the Generational Distance (GD) [29], Inverted Generational Distance (IGD) [29], $ϵ$ indicator ($ϵ$) [30], and spacing [29], which measure convergence to the true front, coverage of the true front, worst-case domination gap, and the distribution uniformity of the obtained solutions, respectively. The quality metrics are explained as follows:

• The generational distance (GD) measures the average Euclidean distance from each solution in the obtained solution set (S) to the nearest point on the true Pareto front ($P^{*}$). It is defined as follows:

  $$\mathrm{GD}={\displaystyle \frac{1}{\left|S\right|}}\sqrt{\sum _{i=1}^{\left|S\right|}{d}_i^2},$$

  (32)

  where $\left|S\right|$ denotes the number of solutions in S and $d_i={min}_{p^{*}\in P^{*}}\parallel s_i-p^{*}\parallel$ represents the minimum Euclidean distance from solution $s_i\in S$ to any point ($p^{*}\in P^{*}$). A lower GD value indicates better convergence to the true Pareto front.
• The inverted generational distance (IGD) computes the average distance from each point in the reference front ($P^{*}$) to its nearest member in the obtained solution set (S):

  $$\mathrm{IGD}={\displaystyle \frac{1}{|P^{*}|}}\sum _{j=1}^{|P^{*}|}\underset{x\in S}{min}\parallel x-p_j^{*}\parallel ,$$

  (33)

  where $|P^{*}|$ is the number of points in $P^{*}$ and $p_j^{*}$ denotes the j-th point of $P^{*}$. Lower IGD values indicate both better convergence and diversity.
• The epsilon indicator ($ϵ$) reflects the minimum additive value ($ϵ$) required such that every point in the reference front ($P^{*}$) is weakly dominated by at least one point in the obtained set (S). Formally,

  $$ϵ=inf\left\{ϵ\in \mathbb{R}\mid \forall p^{*}\in P^{*},\exists s\in S:f_i\left(s\right)\le f_i\left(p^{*}\right)+ϵ,\forall i\right\},$$

  (34)

  where $f_i(·)$ denotes the i-th objective function. A smaller $ϵ$ value implies stronger dominance of S over $P^{*}$.
• Spacing quantifies the distribution uniformity of solutions in S:

  $$\mathrm{Spacing}={\displaystyle \frac{1}{\left|S\right|-1}}\sum _{i=1}^{\left|S\right|}\left|d_i-\overline{d}\right|,$$

  (35)

  where $d_i={min}_{s_j\in S,j\ne i}\parallel s_i-s_j\parallel$ is the minimum distance from $s_i$ to its nearest neighbor in S and $\overline{d}$ is the mean of all $d_i$. Smaller Spacing values indicate more uniform distribution.

As shown in Figure 8, the average per-generation runtime remained nearly constant in the smaller-scale settings ($N=5000,M=10$ and $N=10,000,M=100$), despite the increase in problem size. This plateau effect can be attributed to the fact that the algorithm’s computational workload in these configurations does not fully utilize the available hardware resources—such as CPU cache, memory bandwidth, or thread-level parallelism. As a result, the measured runtime is primarily dominated by fixed system-level overheads rather than by the algorithm’s intrinsic complexity. In contrast, as the problem scale increases further (N = 20,000, M = 500 and N = 30,000, and M = 1000), the runtime begins to grow more significantly. In these larger settings, the computational demand likely exceeds the system’s resource saturation threshold, causing the runtime to better reflect the theoretical time complexity (Equation (31)). This observation confirms that MOFATA’s scalability becomes evident under high-concurrency conditions, where hardware utilization more accurately amplifies the cost of increasing decision and objective dimensions.

Despite the increase in runtime at larger problem scales, the key optimization metrics—BTAR and MTIR—remain stable, as illustrated in Figure 9. However, the solution-set quality, as assessed by GD, IGD, the $ϵ$ indicator, and spacing, shows noticeable degradation as N and M grow. This is mainly because a larger and higher-dimensional search space makes it more difficult for a fixed-size population to cover the true Pareto front, resulting in reduced diversity and convergence. Nevertheless, this decline in solution-set quality does not significantly impact BTAR or MTIR. In practical scheduling, it is sufficient for the algorithm to consistently identify high-quality solutions in relevant regions of the objective space, even if global front approximation weakens. Thus, MOFATA remains robust and effective for real-world DDoS traffic scheduling, even as the problem scale increases.

These findings validate the complexity-driven scalability concerns raised in Section 4.1 but also highlight MOFATA’s resilience: even in large-scale settings, it continues to produce reliable scheduling outcomes. This robustness makes it suitable for real-time traffic control in dynamic DDoS environments.

4.3. Parameter Sensitivity Analysis

The proposed multi-objective framework introduces four control parameters—$\alpha$, $\beta$, $\gamma$, and $\delta$—each of which governs a distinct optimization objective:

• $\alpha$ regulates the penalty for DI, encouraging uniform traffic distribution across servers.
• $\beta$ penalizes the acceptance of malicious traffic, promoting a more aggressive filtering strategy and increasing the MTIR.
• $\gamma$ penalizes the rejection of legitimate traffic, thereby favoring higher BTAR values.
• $\delta$ incentivizes the isolation of malicious flows, encouraging the scheduler to concentrate attack traffic on a small set of designated isolation nodes, as reflected by the isolation effectiveness ($F_{iso}$).

A series of controlled experiments are conducted to examine the system’s sensitivity to each parameter. By varying one parameter at a time while holding the others fixed, we characterize the influence of each on key performance metrics, as shown in

Table 1. Influence of each parameter on key performance metrics.

Parameter	When Increased	When Decreased
$\alpha$	Leads to improved load balancing but may weaken attack isolation	Allows more clustering, which may enhance isolation at the cost of balance
$\beta$	Improves MTIR by aggressively filtering malicious flows but may reduce BTAR	Raises BTAR but potentially allows more malicious traffic through
$\gamma$	Boosts BTAR by reducing false negatives but can reduce MTIR	Improves MTIR while possibly denying legitimate traffic
$\delta$	Increases attack clustering effectiveness but may lead to greater load imbalance	Results in more dispersed attack flow handling and lower isolation clarity

:

This analysis reveals two inherent conflicts among the optimization objectives. First, there exists a trade-off between security and availability governed by $\beta$ and $\gamma$. Elevating $\beta$ improves MTIR but lowers BTAR, while increasing $\gamma$ has the opposite effect. Second, a similar tension exists between global load balancing and attack flow isolation, controlled by $\alpha$ and $\delta$. While $\delta$ promotes the aggregation of malicious traffic into specific servers, enhancing attack containment, $\alpha$ enforces fairness and resists concentrated load allocation, thereby undermining the isolation effect.

Given these trade-offs, optimal parameter values depend on the specific deployment context. For instance, security-critical environments such as enterprise firewalls may prefer high $\beta$ and $\delta$ values to aggressively suppress attacks and maximize isolation while setting $\gamma$ lower to tolerate occasional service disruption. In contrast, availability-sensitive systems, such as e-government portals, benefit from high $\gamma$ values and moderate $\alpha$ values, ensuring minimal service disruption while still preserving acceptable fairness. Balanced configurations, suitable for public cloud gateways, may set all parameters around mid-level values to maintain a compromise between conflicting objectives. Additionally, honeypot-based defensive setups may prioritize $\delta$ to maximize attack traffic aggregation, whereas service clusters under high load pressure may benefit from high $\alpha$ values to enforce a balanced load distribution.

To further elucidate the interactions between security and availability objectives, we constructed a dual-indicator heatmap based on the ($\beta$, $\gamma$) parameter pair, which directly governs the trade-off between MTIR and BTAR, as shown as Figure 10. In this visualization, each grid cell corresponds to a unique configuration of $\beta$ and $\gamma$ and encodes two performance metrics simultaneously: the upper portion of each cell represents the BTAR, while the lower portion indicates the MTIR. This layered encoding facilitates direct comparison of the competing objectives under varying parameter values. The heatmap reveals a distinct antagonism between the two metrics: higher $\beta$ values generally enhance MTIR by encouraging more aggressive filtering, while higher $\gamma$ values improve BTAR by making the scheduler more tolerant to ambiguous flows. However, it is evident that no single configuration simultaneously maximizes both. For instance, configurations with $\beta =0.9$ and $\gamma =0.1$ yield high MTIR but low BTAR, whereas $\beta =0.1$ and $\gamma =0.9$ exhibit the reverse. The intermediate region—where both parameters are set to moderate values—produces balanced trade-offs. This visual confirmation supports the necessity of Pareto-aware parameter tuning and emphasizes that prioritizing one objective inevitably involves compromise on the other.

As shown in Figure 11, the heatmap was constructed to investigate the interplay between global load balancing and malicious traffic isolation under varying settings of $\alpha$ and $\delta$. Each cell in the heatmap encodes two key metrics: the upper portion represents the average load of isolation nodes, reflecting the degree to which malicious traffic is successfully concentrated; the lower portion represents the negated LID among non-isolation nodes, which serves as a proxy for benign load distribution fairness. The use of the negated LID ensures that, for both indicators, higher values consistently indicate more desirable outcomes, facilitating intuitive visual interpretation. This heatmap reveals a clear trade-off structure. As $\delta$ increases under low $\alpha$ values, the system tends to concentrate attack flows on a limited number of servers, which results in higher average isolation-node loads. However, as $\alpha$ increases, the scheduler begins to enforce stricter global balancing constraints, which undermines the isolation mechanism. In such cases, malicious traffic becomes partially redistributed, reducing the average load on isolation nodes.

In conclusion, the analysis visually confirms the presence of competing objectives within the proposed scheduling framework and reinforces the necessity of adaptive, scenario-aware parameter selection strategies for balancing system availability, security, and resource fairness under adversarial conditions.

5. Experiment

5.1. Experiment Setup

We constructed a simulated experimental environment in which a total of M servers were configured, each with a processing capacity of C. The experiments utilized authoritative DDoS attack datasets from the Kaggle platform [31], including several widely recognized subsets: CSE-CIC-IDS2018-AWS, CICIDS2017, and the CIC DoS Dataset (2016), comprising approximately 13 million traffic records in total. The parameter settings of the optimization algorithms are shown in

Table 2. MOFATA Hyperparameter Settings.

Category	Parameter	Value
Evolutionary Settings	Population size ($PopSize$)	30
	Maximum generations ($MaxGen$)	50
	Problem dimensionality (N)	35,000
Search Space	Variable bounds ($Lb$, $Ub$)	${[1,M]}^N$
	Server capacity (C)	1.2 × total benign traffic
Multi-objective Strategy	$\epsilon$-dominance threshold	0.01
	Fitness weights	$\alpha =0.53$, $\beta =0.5$, $\gamma =0.5$, $\delta =0.57$

. Two categories of comparison methods were included: (1) classical multi-objective optimization algorithms such as MOGWO [32] and MOAHA [33] and (2) the original single-objective FATA algorithm prior to our proposed enhancements. The experiments were designed to evaluate performance across three dimensions: (1) benign traffic acceptance rate and malicious traffic interception capability, (2) server load balancing and malicious traffic isolation, and (3) system responsiveness and stability under dynamic attack scenarios. For each experiment, both a single-objective fitness function and a composite fitness function were tested in order to assess the effectiveness of the composite formulation in optimizing overall system performance. All experiments were repeated five times, and the best result from each trial set was recorded as the basis for final evaluation.

5.2. Evaluation of Benign Traffic Acceptance Rate (BTAR) and Malicious Traffic Interception Rate (MTIR)

This experiment aims to evaluate whether SOE can significantly improve the benign traffic acceptance rate (BTAR) while effectively identifying and intercepting malicious traffic under DDoS attack scenarios, thereby ensuring both service quality and system security. A simulated DDoS attack environment was constructed, with attack traffic ratios set to 10%, 30%, 50%, and 70%. The system was deployed with 20 homogeneous server nodes, each having the same processing capacity, to approximate realistic server performance constraints. Benign and malicious traffic was generated according to predefined distributions. The total server processing capacity was set to be 1.2 times the total volume of benign traffic, while the addition of malicious traffic caused the total incoming requests to exceed system capacity, creating resource contention. To evaluate the model’s performance under steady-state attack conditions, both the total traffic volume and attack intensity were kept constant to ensure consistency and comparability across test scenarios.

Two key performance metrics were considered:

• BTAR (Benign Traffic Acceptance Rate): This metric denotes the proportion of benign requests that are successfully scheduled and served, defined as Equation (36).

  $$\mathrm{BTAR}={\displaystyle \frac{N_{\mathrm{legit},\mathrm{passed}}}{N_{\mathrm{legit},\mathrm{total}}}},$$

  (36)

  where $N_{\mathrm{legit},\mathrm{passed}}$ is the number of benign requests that received service and $N_{\mathrm{legit},\mathrm{total}}$ is the total number of benign requests received by the system. A higher BTAR indicates fewer false negatives and stronger service continuity.

• MTIR (Malicious Traffic Interception Rate): This metric evaluates the system’s ability to detect and block malicious traffic, defined as Equation (37).

  $$\mathrm{MTIR}={\displaystyle \frac{N_{\mathrm{attack},\mathrm{blocked}}}{N_{\mathrm{attack},\mathrm{total}}}},$$

  (37)

  where $N_{\mathrm{attack},\mathrm{blocked}}$ denotes the number of malicious requests that were successfully intercepted and $N_{\mathrm{attack},\mathrm{total}}$ is the total number of malicious requests that entered the system. A higher MTIR reflects stronger defense capability and more effective mitigation of service degradation caused by attacks.

As shown in Figure 12, under a low attack ratio of 10%, SOE successfully scheduled the majority of benign traffic to the appropriate target servers, achieving a BTAR of 97.74% and an MTIR of 95.42%. The remaining malicious traffic was minimal and could be absorbed by the residual system capacity. As the attack ratio increased to moderate intensity levels (30–50%), SOE still maintained high performance, with BTAR values of 95.42% and 93.85% and corresponding MTIR values of 93.12% and 95.24%, respectively. When the attack ratio reached 70%, part of the system’s resources were consumed by malicious traffic, resulting in some benign requests being dropped. Consequently, BTAR slightly decreased to 92.34% and MTIR to 92.67%, yet the overall performance still outperformed all comparison methods.

Furthermore, with the use of the composite fitness function, as shown in Figure 13, MOGWO, MOAHA, and FATA all exhibited significant performance improvements—BTAR increased by 4.99% to 19.10%, and MTIR improved by 3.65% to 30.63%. This demonstrates the effectiveness of the composite fitness function in enhancing an algorithm’s ability to optimize both BTAR and MTIR. Despite these improvements, SOE based on MOFATA consistently achieved superior results across all attack intensities. Compared to other multi-objective algorithms, it improved BTAR by 0.57% to 5.38% and MTIR by 0.09% to 4.08%. Relative to single-objective algorithms, the gains were even more pronounced—BTAR increased by 14.51% to 22.29% and MTIR by 14.93% to 41.73%.

In summary, SOE demonstrates strong robustness and adaptability across varying levels of attack intensity, effectively preserving benign traffic continuity while simultaneously identifying and suppressing malicious flows. Notably, even under high-intensity adversarial conditions, SOE maintained high scheduling efficiency and interception accuracy, validating its practical utility and superiority in DDoS mitigation scenarios. Additionally, the results show that regardless of the underlying optimization algorithm, the introduction of a composite fitness function consistently improves both BTAR and MTIR, further confirming the suitability of this formulation for traffic scheduling under DDoS attack conditions.

5.3. Evaluation of Load-Balancing Degree and Malicious Traffic Isolation Effect

This experiment aims to evaluate whether SOE can achieve effective traffic load balancing in multi-server environments, ensuring that, under low attack intensity, the defense strategy does not cause local bottlenecks or single-point overloads due to imbalanced resource allocation. Under high attack intensity, SOE should be able to concentrate malicious traffic onto a limited number of nodes, thereby preserving the service availability of critical servers for legitimate requests. In addition, the experiment investigates whether variations in the number of servers affect SOE’s scheduling performance, thereby assessing its scalability and stability under different resource scales. The experimental setup included configurations of 5, 10, 15, and 20 servers, with the malicious traffic ratio fixed at 10% and 70%. The primary metric of interest is the load-balancing degree, defined as the standard deviation of the loads across all servers, which reflects the overall equilibrium of the system. A smaller value indicates a more uniform load distribution across servers and a more effective scheduling strategy.

As shown in Figure 14, Figure 15, Figure 16 and Figure 17 and

Table 3. Standard deviation of server load. Among them, MOAHA, MOGWO, and FATA employ a single-objective fitness function. (The lower, the better).

Malicious Traffic Ratio	Number of Servers	SOE	MOAHA	MOGWO	FATA
10%	5	2.57	8.74	21.26	28.16
	10	2.48	15.71	17.74	17.30
	15	3.23	23.65	21.31	15.17
	20	2.41	22.68	21.68	24.01
70%	5	7.03	12.82	8.05	14.16
	10	6.23	11.53	12.47	9.62
	15	6.34	13.62	12.13	16.79
	20	6.54	12.79	15.13	13.48

and

Table 4. Standard deviation of server load. All algorithms employ a composite fitness function. Bolded text represents the standard deviation of the load across the remaining servers after excluding fully loaded servers. (The lower, the better).

Malicious Traffic Ratio	Number of Nodes	SOE	MOAHA	MOGWO	FATA
10%	5	2.42	4.16	2.72	11.41
	10	2.48	2.91	3.59	7.28
	15	3.23	3.92	5.39	7.51
	20	2.41	4.68	3.36	6.84
70%	5	7.03	6.57	8.27	8.59
		0.56	2.61	1.38	6.05
	10	6.23	6.99	8.15	8.12
		2.72	3.59	5.04	6.71
	15	6.32	6.50	6.40	5.22
		2.73	5.20	4.99	4.01
	20	5.97	6.23	7.40	6.51
		3.51	4.83	6.55	6.03

, under a 10% malicious traffic ratio, SOE consistently maintained good load balancing across different server network scales, with the standard load deviation ranging from 2.41 to 3.23. In contrast, when the composite fitness function was not applied, alternative methods exhibited significantly higher load variance, with standard deviations ranging from 8.74 to 24.01. After incorporating the composite fitness function, the load-balancing performance of MOAHA, MOGWO, and FATA improved, with their standard deviations reduced to the range of 2.91 to 11.41. Under high-intensity attack conditions (70% malicious traffic), SOE was able to strategically redirect residual, non-intercepted malicious traffic to a limited number of server nodes, which operated at full capacity to isolate the attacks. This approach preserved the remaining servers for normal service delivery. In contrast, other methods without load-balancing optimization failed to perform effective traffic redirection, leading to simultaneous overloads or even crashes across multiple servers. Experimental results showed that SOE achieved a standard load deviation of 6.23 to 7.03 under these conditions, outperforming most comparison algorithms. Even when enhanced with the composite fitness function, the standard deviation of other methods generally remained in the range of 5.22 to 8.59. After excluding the fully-loaded isolation nodes, SOE’s standard load deviation further decreased to 0.56 to 2.28, demonstrating its exceptional fine-grained load-balancing capability. In summary, SOE not only demonstrates effective load balancing under light attack conditions but also strategically sacrifices a small number of nodes under severe attacks to maintain overall system stability and service continuity. Its consistently superior performance across all standard load deviation evaluations highlights SOE’s intelligent scheduling capability and system resilience under extreme stress conditions.

5.4. Comparison with Supervised Learning-Based Models

To provide a comprehensive evaluation of SOE’s advantages in traffic scheduling, we incorporated two representative supervised learning algorithms—XGBoost and Random Forest—as machine learning baselines. Unlike conventional flow classification models, these baselines were trained to directly predict the target server assignment for each flow, learning a mapping from flow-level features to server indices. The experiments utilized the same real-world DDoS dataset as previous sections, with the data randomly partitioned into training (70%), validation (15%), and test (15%) subsets. Hyperparameter tuning was conducted on the validation set, and final performance was reported on the test set to ensure fairness and reproducibility. All models were evaluated in a five-server environment under two typical DDoS scenarios: a low-intensity attack (10% malicious traffic) and a high-intensity attack (70% malicious traffic). As shown in Figure 18, under the 10% attack scenario, XGBoost and Random Forest achieved BTAR values of 79.03% and 76.94%, respectively—substantially lower than SOE’s 97.74%—with corresponding MTIR values of 80.35% and 83.10%, compared to SOE’s 95.42%. In the 70% attack setting, their BTAR values dropped further to 73.64% and 70.77%, with MTIR values at 62.75% and 65.98%, both well below SOE’s 92.34% and 92.67%. Additionally, as shown in

Table 5. Standard deviation of server load. Comparison with XGBoost and Random Forest. (The lower, the better).

Malicious Traffic Ratio	SOE	XGBoost	Random Forest
10%	2.57	13.92	13.21
70%	7.03	18.49	20.10

, analysis of the server load distribution revealed that both machine learning-based approaches often produced severe load imbalance, leaving certain nodes overloaded while others were underutilized, ultimately decreasing overall throughput. These results indicate that, while supervised learning models can partially approximate traffic scheduling, their effectiveness and robustness are significantly outperformed by the optimization-driven SOE framework, especially in complex and dynamic DDoS environments.

Although both models exhibited satisfactory predictive accuracy during training, their underlying structural design restricts their practical utility in real-world scheduling. Specifically, these methods lack explicit modeling of global coordination and system-level constraints, which are essential for effective multi-objective optimization in DDoS mitigation. As a result, their inference outputs, while often feasible at the individual flow level, fail to achieve robust trade-offs across competing objectives such as BTAR and MTIR. This limitation leads to suboptimal scheduling decisions, especially in complex or dynamic network environments. In contrast, the SOE framework employs a composite optimization approach that dynamically balances multiple objectives and adapts to changing attack patterns, resulting in consistently superior system-level performance.

5.5. Evaluation of Dynamic Scenarios

This experiment aims to evaluate the stability of SOE under dynamically changing DDoS attack environments. Specifically, it examines whether SOE can rapidly respond and promptly adjust its scheduling strategy in the presence of significant fluctuations in attack intensity or traffic patterns. Additionally, the experiment assesses whether the overall performance remains stable and whether the scheduling strategy avoids frequent or drastic oscillations—thereby validating SOE’s practicality and deployability in real-world scenarios. To this end, a time-evolving simulation scenario was designed, in which the attack process was divided into four representative stages to emulate typical conditions such as normal operation, peak attack periods, traffic pattern transitions, and attack mitigation phases. Within each stage, the proportion of malicious traffic fluctuates within a narrow range to better reflect the dynamic nature of real-world attack scenarios. At time $T_0$, the system is in a normal or mildly attacked state, with the malicious traffic ratio fluctuating between 8% and 13%; at time $T_1$, the attack intensity surges, initiating a peak attack phase (68–73%); at time $T_2$, the attack sources change, leading to a sudden shift in the traffic distribution; and at time $T_3$, the attack intensity drops significantly, returning to a relatively low level (28–33%). Throughout all stages, we continuously monitored SOE’s benign traffic assurance (BTAR) and malicious traffic interception capability (MTIR), with particular attention to performance fluctuations under dynamic conditions. To quantify this, the standard deviations of BTAR and MTIR within each stage were introduced as evaluation metrics, capturing the system’s scheduling stability under abrupt variations.

In the dynamic scheduling experiments, we pre-trained nine models, each corresponding to a specific malicious traffic ratio ($r_k\in 10\%,20\%,\dots ,90\%$). At runtime, SOE continuously estimates the current attack intensity ($r_t$) and selects the model whose pre-trained ratio ($r_k$) is closest to $r_t$. This similarity matching is achieved by rounding $r_t$ to the nearest available $r_k$, ensuring the scheduling strategy remains closely aligned with the current attack level. While this nearest-neighbor matching was adopted for experimental consistency, the framework can be easily extended to finer granularity to further enhance adaptability in real-world deployments.

$$k^{*}=\underset{k}{argmin}\left|r_t-r_k\right|,r_k\in \{0.1,0.2,\dots ,0.9\}$$

(38)

As shown in Figure 19 and

Table 6. Illustration of the standard deviation of BTAR and MTIR across different time periods. (The lower, the better).

	Time	SOE	MOAHA	MOGWO	FATA
Benign Traffic	$T_0$	1.20	1.03	1.02	1.26
	$T_1$	1.15	1.49	1.05	1.94
	$T_2$	1.24	1.37	1.09	1.24
	$T_3$	1.34	1.29	2.47	1.47
	$T_0-T_3$	4.56	4.19	5.39	6.71
Malicious Traffic	$T_0$	1.20	1.03	1.02	1.26
	$T_1$	1.15	1.49	1.05	1.70
	$T_2$	2.28	1.76	1.26	0.85
	$T_3$	1.88	1.43	1.48	1.17
	$T_0-T_3$	2.79	2.27	2.66	12.17

, across the four stages ($T_0$ to $T_3$), SOE maintained a BTAR ranging from 80.72% to 96.49% and an MTIR between 82.05% and 94.17%. Within each stage, the standard deviation of BTAR ranged from 1.15 to 1.34, and that of MTIR ranged from 1.15 to 2.28. This indicates that while SOE exhibited a slight performance drop immediately following the initial malicious traffic shift in each stage, it was able to rapidly recover and improve performance in subsequent scheduling cycles. MOAHA and MOGWO, which also employed the composite fitness function, achieved relatively stable scheduling under the same scenario, but their BTAR and MTIR values were consistently lower than those of SOE. Furthermore, as shown in Figure 20, SOE responded to each change in attack conditions within one second, demonstrating its capability for real-time analysis and scheduling. In summary, SOE exhibited strong stability and adaptability under dynamically changing attack conditions. It consistently ensured both service quality and system security across all stages, highlighting its practical value and scheduling robustness in dynamic DDoS mitigation scenarios.

6. Conclusions and Discussion

This study proposes a novel scheduling engine—SchedOpt Engine (SOE)—designed for DDoS attack environments, with the goal of ensuring the continuous availability of benign traffic, even when traditional filtering mechanisms fail. The traffic scheduling problem is formulated as a multi-objective optimization task, targeting an optimal trade-off among four conflicting goals: maximizing the benign traffic acceptance rate (BTAR), maximizing the malicious traffic interception rate (MTIR), improving server load balancing, and enhancing malicious traffic isolation. To achieve this, a composite loss function was constructed that integrates the aforementioned objectives into a unified optimization framework using soft constraints. This avoids the rigidity associated with hard rules while ensuring strong penalization when benign traffic guarantees are insufficient. In terms of algorithm design, we developed a multi-objective optimization procedure based on an improved version of the FATA algorithm, referred to as MOFATA, which produces a set of Pareto-optimal scheduling strategies. These strategies offer network administrators a diverse set of scheduling options, allowing them to flexibly select the most appropriate performance trade-off according to system-specific priorities. Extensive simulations in near-realistic network environments demonstrate that SOE maintains high levels of BTAR and MTIR, even under high attack ratios, reflecting its strong adaptability in scheduling. Notably, SOE can redirect the majority of malicious traffic to a small subset of server nodes, sacrificing limited resources to preserve the operational integrity and efficiency of the remaining system. Furthermore, under dynamic attack scenarios, SOE is capable of promptly adjusting its scheduling strategies in response to rapid fluctuations in attack intensity, exhibiting excellent stability and adaptability. Overall, SOE advances both the theoretical modeling and practical algorithmic design of multi-objective traffic scheduling under DDoS conditions and provides a viable path toward building practical and scalable network defense solutions.

Although SOE demonstrates notable performance advantages in multi-objective scheduling optimization, it still exhibits certain limitations, particularly in terms of real-time responsiveness. As the algorithm requires multi-objective evaluations, non-dominated sorting, and fitness computations for a large number of candidate solutions in each iteration, its computational overhead increases substantially with the growth of network scale and attack intensity. In high-concurrency environments, where low-latency scheduling is essential, the current convergence efficiency may not suffice to guarantee real-time service continuity. Therefore, future work should focus on accelerating the optimization process by incorporating parallel computing frameworks, incremental fitness evaluation techniques, or model compression strategies to improve execution efficiency.

Moreover, current experiments are primarily based on simulation data. Although they span various malicious traffic ratios and network configurations, they still fall short of fully capturing the intricate and dynamic characteristics of real-world networks. To address this, a small-scale yet representative physical testbed is planned, featuring multiple server nodes, routing infrastructure, and traffic generation modules. By deploying both the attack simulation components and the scheduling engine within this platform, we aim to comprehensively evaluate SOE’s adaptability and defensive efficacy under realistic traffic flows, adversarial behaviors, and system-level disturbances. Furthermore, this platform will support a detailed analysis of the algorithm’s performance in terms of resource consumption, deployment complexity, and fault tolerance—providing a robust basis for engineering deployment and scalable adoption in real-world network environments.