Replacing the Genetic Algorithm with Multi-Objective Bacterial Foraging Optimization in XCS

Abstract

This article is positioned into a cross-section of machine learning, cybersecurity, and nature-inspired domains. This article’s main objective is to use eXtended Classifier System (XCS), a known adaptive Reinforcement Learning (RL) algorithm, and alter it to use the Bacterial Foraging Optimization Algorithm (BFOA) instead of its original Genetic Algorithm component. This modification transforms XCS into a multi-criteria optimization system (BFOA-XCS) through evaluation of classifier fitness across accuracy, stability, and variance reduction while simultaneously using weighted-sum scalarization. In this way, the method leverages BFOA’s chemotactic search and population dynamics. The proposed BFOA-XCS integration was validated in two experimental phases. First, evaluations across 19 benchmark machine learning datasets demonstrated that Improved BFOA (IBFOA)-XCS achieves the best Friedman ranking among all XCS variants (marginally significant at α = 0.10, supported by medium-to-large effect sizes), with notable variance reduction (15.2 percent) over standard GA-XCS. Second, in a dynamic cybersecurity simulation environment with six attack scenarios, all XCS variants significantly outperformed three of five deep RL baselines (Deep Q-Network (DQN), Q-Learning, and Policy Gradient (REINFORCE)) with large statistical effect sizes. Proximal Policy Optimization (PPO) and Soft Actor–Critic (SAC) achieved higher overall rewards but at substantially greater computational cost: PPO at 5.3× and SAC at 26.1× the XCS compute time per run (2 min 8 s and 10 min 26 s, respectively, vs. 24 s for XCS). The results demonstrate that rule-based XCS with BFOA optimization offers a compelling alternative to neural approaches for cybersecurity defense, combining competitive performance with interpretable policies and substantially lower computational requirements.

1. Introduction

The modern world runs on the power of knowledge provided by information and technology [1], which are crucial for different social, biological, and economic systems. Technology systems that enable such modern operations heavily rely on the leading artificial intelligence technology [2]. Specifically, computer research fields of computational intelligence, machine learning (e.g., Reinforcement Learning (RL) adaptive systems), and nature-inspired systems [3], the cross-section of which our work is positioned, are playing an essential role in tackling an ever-increasing complexity of the world of information.

In 1976 [4], Holland introduced foundational ideas in adaptation and evolutionary computation that later influenced Learning Classifier Systems (LCS). Even after many decades, these systems are still the subject of intense research [5]. The first LCS architecture was Cognitive System One (CS-1) [6], which utilizes two main components: RL, where the system learns based on the rewards received when executing actions in the environment, and evolutionary mechanisms. Now, the eXtended Classifier System (XCS) [7] is one of the most known and investigated algorithms in the LCS group [8]. The XCS population presents a model-free solution to the problem when it has evolved (i.e., the learning process has been completed). It is structured as a set of classifiers (population), where a classifier can be seen as an “if condition then action” structure. The condition is usually a binary representation of the (part of) environment (e.g., large-scale dynamic IoT environment [9]), and the action is a viable choice a system can perform in that environment (e.g., control inputs (actions) for a spherical robot [10]).

The pioneered evolutionary, nature-based Genetic Algorithm (GA) optimization component first introduced in CS-1 was crucial to its operation. Today, the GA is integral to many algorithms from the LCS group, and the field has also evolved into a world-known standalone research domain. But the GA from the group of evolutionary-based algorithms (e.g., another well-known algorithm from this group is differential evolution [11]) is not the only viable optimization solution. There are many other metaheuristic algorithm groups with various valuable options. Groups such as Swarm-based algorithms (SBA) (e.g., ant colony optimization [12]), Physics/Chemistry-based algorithms (e.g., black hole algorithm [13]), Human-based Algorithms (e.g., teaching–learning-based optimization [14]), Ecosystem-based algorithms (e.g., artificial ecosystem-based optimization [15]), to name just a few, can be utilized. The pool of choices is vast [16] because solving both artificial benchmarks and real-world engineering problems requires a diverse set of techniques, as implied by the No Free Lunch theorems (i.e., no single optimization algorithm performs best across all problem classes) [17].

In this work, the Bacterial Foraging Optimization Algorithm (BFOA) is utilized, which mimics the collective foraging behavior of bacteria and, therefore, belongs to the SBA group of nature-based optimization algorithms [18]. The selection of the BFOA algorithm was carefully considered, taking multiple factors into account. The first consideration was to use the algorithm, which would be a worthy substitute for the GA component [19] found in the XCS algorithm. For example, current research is promising, and modified BFOA has been found to have a greater convergence speed than the original GA in the real-problem industry setting [20]. The second consideration was to have an algorithm with great local and global optimization ability [21] (i.e., with the main line of thought being that the movement mechanisms of bacteria in BFOA would serve exceptionally well for (local) action adaptivity). The third consideration was to avoid GA’s sensitivity to the initial selection of its hyperparameter values (i.e., evolutionary algorithms are sensitive to correct initial settings [22]) and its premature convergence [23] to suboptimal solutions due to loss of genetic diversity).

The main contributions of this article are as follows:

• Replacing the GA component in XCS with BFOA, extending the LCS research domain with a bio-inspired swarm intelligence algorithm. To the best of the authors’ knowledge, this is the first integration of BFOA with the LCS. The integration pattern itself is independent of the specific swarm optimizer chosen. BFOA is one instantiation, and the same template can in principle accommodate other population-based optimizers.
• Introducing a weighted multi-criteria fitness function that leverages BFOA’s population dynamics to simultaneously evaluate classifier accuracy, prediction stability, and variance reduction. These capabilities were not natively supported by the standard single-objective GA operator, which evaluates and recombines classifiers in pairs rather than as a coordinated population.
• Comprehensive empirical validation across two experimental domains: static classification on 19 machine learning datasets and dynamic cybersecurity defense in a simulated network environment with six distinct attack scenarios, demonstrating that BFOA-XCS achieves consistent ranking improvements over standard XCS, supported by medium-to-large effect sizes (Cohen’s d = 0.623–0.803) and a 15.2% variance reduction.
• Comparative evaluation against modern deep RL (DRL) baselines (Deep Q-Network (DQN), Q-Learning, Policy Gradient (REINFORCE), Proximal Policy Optimization (PPO), and Soft Actor–Critic (SAC)), demonstrating that XCS with BFOA optimization significantly outperforms three of five neural baselines (DQN, Q-Learning, and Policy Gradient) while offering interpretable decision rules, warmup-free competence, and substantially lower computational cost per training run than the top-performing neural agents (PPO at 5.3× and SAC at 26.1× the XCS compute time).

Throughout this article, BFOA-XCS refers jointly to all XCS variants utilizing BFOA-based optimization, including standard BFOA, Adaptive BFOA, and Improved BFOA (IBFOA), unless a specific variant is indicated.

This article’s structure is as follows. Section 2 presents relevant works related to the domains of LCS and nature-inspired heuristics. Section 3 describes the methods and methodology, detailing the XCS algorithm, the BFOA variants, their integration (BFOA-XCS), and the cybersecurity simulation environment. Section 4.1 presents Experiment 1 on benchmark machine learning datasets and reports the statistical analyses of operator performance. Section 4.2 presents Experiment 2 in the dynamic cybersecurity defense setting and compares XCS variants against DRL baselines. Section 5 discusses the interpretation of results and this study’s limitations. Section 6 outlines directions for future work. Finally, Section 7 concludes the article.

2. Related Domains

This section provides a brief overview of the LCS domain, specifically the XCS subdomain and the Nature-Inspired Metaheuristics domain, with a focus on the works related to the BFOA.

2.1. Learning Classifier Systems: The eXtended Classifier Systems Subdomain

LCS is a group of rule-based algorithms that can be built upon and draw from the cross-section of domains such as RL [24], Q-Learning [25], R-Learning [26], evolutionary optimization [27], and many others. The word ‘can’ is used to emphasize that not all algorithms within the LCS group employ all these techniques at the same time. LCSs are utilized in the applicative fields ranging from data mining [28] to aviation management [29], across different tasks such as classifications [30] or domains of imperfect information [31].

Over the decades, the research field of LCS has matured from the CS-1 algorithm into two main groups called Pittsburgh and Michigan style [27,32]. The Pittsburgh style evolves a population of complete rule sets (i.e., the whole set is the solution) [33]. In contrast, the Michigan style evolves individual rules within a population, enabling rule-level specialization and adaptation [34]. The LCS group now holds many algorithm sub-branches. Most notable is XCS, which is most widely used and belongs to the Michigan style [35], while others range from relatively simple systems such as the zeroth level classifier system (ZCS) [36]; powerful eXtended Classifier System with Function Approximation [37], which can approximate complex multi-dimensional function surfaces by utilization a mosaic of simpler functions; and Anticipatory Learning Classifier Systems, which also meets the criteria for eXplainable Artificial Intelligence [38].

The current open research directions in XCS are extensive. For example, there is a constant need for research on advanced classifier representations [39]. Another field of research belongs to memory-based XCS systems, which have shown promise in improving the efficiency of rule discovery and application, in complex real-world scenarios, such as maze navigation and traffic problems in wireless networks [40]. One of the essential focuses of the research is also the evolutionary process [41]. It can be argued that the optimizer is essential, but so far, it has not been researched as much as LCS’s learning components [42]. Therefore, a need for further research advancements in the XCS evolutionary process, as in this work, is a logical (evolutionary) step.

2.2. Nature-Inspired Metaheuristic: Bacterial Foraging Optimization Algorithm

Nature-inspired metaheuristic algorithms are optimization techniques that model natural systems and have seen a worthy rise in research in recent years [43]. By mimicking biological and physical processes, these algorithms efficiently solve complex problems and can be classified by their inspiration sources (e.g., bio-based, ecosystem-based, social-based, physics-based, chemistry-based, mathematics-based, and hybrid families) [44]. Among these, the BFOA stands out as a notable example of an SBA group.

Escherichia coli is one of the most well-studied bacterial species and is commonly found as a member of the healthy human gut microbiome [45]. The BFOA, as the name suggests, is based on bacterial foraging behavior, which is also known as chemotaxis [46], and in simple terms describes bacterial movements directed towards beneficial chemicals (stimulus) [47] while avoiding harmful substances. Other steps of the BFOA algorithm cover bacterial reproduction, swarming, elimination, and dispersal, which are described in more detail in Section 3, and are under constant research improvements. For example, IBFOA, which enhances both the chemotaxis and elimination–dispersal mechanisms, has been shown to outperform the original BFOA in terms of convergence speed and accuracy [48].

The selection of BFOA for the present work is grounded in two structural properties of the algorithm that align with the requirements of the XCS classifier population. First, BFOA’s chemotactic step generates small, continuous-valued displacements by the tumble-and-swim mechanism, which makes it well suited for detailed refinement of classifier action parameters once XCS has produced a candidate population, without disrupting the binary condition representation that XCS relies on for state matching. Second, BFOA evaluates a population of bacteria as a coordinated whole at each chemotactic iteration, which enables a multi-criteria fitness function (Section 3.3.2) to be assessed at the population level, a property that the standard GA, with its pairwise crossover and mutation semantics, does not natively support. These two properties motivate the integration mechanism proposed in Section 3.3. The integration pattern itself is independent of the specific swarm optimizer chosen; comparative evaluation of more recent swarm or evolutionary alternatives instantiated under the same template is a natural direction for future work.

2.3. Reinforcement Learning in Cybersecurity

The intersection between RL and cybersecurity is a fast-developing research area because cyber threats grow in sophistication and frequency [49]. Traditional defense mechanisms based on rule- or signature-based detection, along with static firewall rules, are increasingly insufficient [50]. Approaches that utilize RL offer the ability to adapt and respond to dynamic threat environments in real time, making them attractive for autonomous cyber defense [51].

DRL is one of the dominant computational approaches within cybersecurity. DRL methods such as DQN have been applied to intrusion detection [52], vulnerability management [53], and automated threat response [54]. More advanced architectures have also been explored, ranging from actor–critic methods, such as PPO and SAC for multi-step defense scenarios [55], to multi-agent-inspired and federated RL frameworks designed to protect large-scale network infrastructures [56].

Despite this progress, the existing body of work is dominated by neural network-based approaches. Rule-based learning systems, which offer inherent interpretability (a property that is highly desirable and useful in security-critical domains where understanding the rationale behind defensive decisions is essential), remain largely unexplored in this context. While only limited works have applied LCS variants to intrusion detection as a classification task [57], the use of LCS as an adaptive RL agent for closed-loop, dynamic network defense remains largely unexplored.

Furthermore, to the best of our knowledge, prior cybersecurity uses of LCS have not progressed beyond the standard GA evolutionary component, and the combination of LCS with other nature-inspired optimizers therefore appears underexplored in cybersecurity settings. This gap is particularly important because cybersecurity defense environments often exhibit large, structured, discrete action spaces, where rule-based representations can provide interpretable, modular decision structures. By contrast, many deep RL approaches typically rely on neural function approximation for value/policy estimation, which may trade interpretability for representational power.

In the next section, a transition is made from an overview of the related research domains to a detailed examination of the methods, including the operational mechanisms of XCS and BFOA, their integration, and the cybersecurity simulation environment.

3. Methods and Methodology

This section presents the methods and methodology behind the proposed BFOA-XCS system. It begins with a description of the XCS algorithm and its main execution cycle, followed by the BFOA and its variants. The integration of BFOA into XCS, including the multi-objective classifier fitness function, is also explained in detail. This section concludes with a description of the cybersecurity simulation environment designed for the experimental evaluation.

3.1. eXtended Classifier System

The XCS algorithm is a rich learning framework that combines adaptivity, generalization, and scalability. Its RL component uses a Q-learning-style temporal-difference update [58], although alternative formulations such as R-learning have also been shown viable [26]. These properties make XCS a versatile algorithm within the LCS family.

The XCS algorithm used in this research was implemented manually, line by line transformation from the pseudocode to Java code, following the original algorithmic description by Butz and Wilson [58]. No external libraries or code generation tools were used for the XCS implementation and its GA component.

3.1.1. Knowledge Representation

XCS has knowledge represented through utilization of classifiers organized in four sets. The population [P] contains all classifiers that exist at a given time step t and represents the system’s accumulated knowledge about the problem. The match set [M] is formed by selecting all classifiers from [P] whose conditions match the current environmental state σ. The action set [A] is a subset of [M] containing all classifiers that advocate the action selected for execution, and the [A]₋₁ is a copy of the action set from the previous time step, used for multi-step problems where reward assignment must be propagated backward through sequential decisions.

Each classifier consists of a condition (typically a binary string with don’t-care symbols ‘#’), an action, and several parameters: the reward prediction p, the prediction error ε, the fitness F, the experience exp, the numerosity num, and the time stamp ts of the last GA invocation. The don’t-care symbol ‘#’ enables generalization. A classifier with more ‘#’ symbols in its condition matches a wider set of environmental states, allowing XCS to discover maximally general but accurate rules.

3.1.2. Main XCS Cycle

The main operational cycle of XCS is presented in Algorithm 1.

Algorithm 1 Pseudocode of the XCS main reinforcement learning cycle (adapted from [58])

initialize reinforcement_program rp initialize environment env initialize XCS    // create population [P]; connect to evaluation function procedure RUN_XCS()     1. do {     2.      σ = get sensory input from env     3.      generate [M] from [P] using σ    // covering may be activated     4.      generate PA from [M]     5.      act = select action according to PA     6.      execute act in env     7.      ρ = get reward from rp     8.      if [A]−1 is not empty then     9.          P = ρ−1 + γ·max(PA)   10.          update set [A]-1 using P, possibly deleting in [P]   11.          → run GA in [A]−1, possibly inserting in [P] // replaced by BFOA (Section 3.3)   12.      end if   13.      if rp signals end-of-problem then   14.          P = ρ   15.          update set [A] using P, possibly deleting in [P]   16.          → run GA in [A], possibly inserting in [P] // replaced by BFOA (Section 3.3)   17.          clear [A]−1   18.      else   19.          [A]−1 = [A]   20.          ρ−1 = ρ   21.          σ−1 = σ   22.      end if   23. } while (termination criteria from rp are not met) end procedure

The cycle begins by receiving sensory input σ from the environment (line 2). The match set [M] is formed from [P] using σ, with a covering mechanism activated if no classifiers match the current state (line 3). A prediction array PA is then computed from [M], aggregating the fitness-weighted predictions for each possible action (line 4). An action is selected according to PA, typically using a greedy strategy that balances exploration and exploitation (line 5). The selected action is executed in the environment (line 6), and a reward ρ is obtained from the reinforcement program (line 7).

If a previous action set [A]₋₁ exists (line 8), the system computes an update target P using the previous reward and the discounted maximum of the current prediction array (line 9, corresponding to Q-learning’s temporal difference update). The classifiers in [A]₋₁ are then updated using P, with deletion of excess classifiers possible in [P] (line 10). The genetic algorithm is then applied to [A]₋₁ (line 11), with offspring potentially inserted into [P]. When the reinforcement program signals the end of an episode (line 13), the current action set [A] is updated directly with the final reward (lines 14–16), and [A]₋₁ is cleared (line 17). Otherwise, the current state is stored for the next iteration (lines 19–21).

Lines 11 and 16 (marked with →) represent the evolutionary optimization step where, in standard XCS, the genetic algorithm is invoked. The GA is triggered only when the average time since its last invocation in the action set exceeds a threshold θ_GA. This evolutionary step is the component that is replaced by BFOA in the proposed BFOA-XCS system, as detailed in Section 3.3.

3.2. Bacterial Foraging Optimization Algorithm

In this section, the BFOA and the variants of this algorithm used in the experiments are described. First, the standard BFOA is introduced. Next, the Adaptive BFOA is outlined, followed by the IBFOA. In each case, specific mechanisms of the original algorithm are modified to address known limitations.

3.2.1. Standard BFOA

The BFOA, originally proposed by Passino [18], mimics the natural foraging behavior of Escherichia coli bacteria through several biologically inspired steps. Chemotaxis involves the movement of bacteria towards beneficial chemicals and away from harmful ones through a sequence of tumble (random direction change) and swim (continued movement in a successful direction) operations. The reproduction phase retains and multiplies the bacteria that perform best while replacing the weaker half of the population. Elimination and dispersal introduce diversity by randomly relocating bacteria with a fixed probability, preventing premature convergence to local optima. These steps are nested in a three-level loop structure that enables the algorithm to balance local exploitation (chemotaxis) with global exploration (elimination-dispersal).

There are numerous variations and interpretations of the BFOA since its original inception, and their pseudocode representations vary. For example, BFOA with added social learning, which directs bacteria to tumble towards better solutions in the chemotactic steps [59], or an enhanced BFOA variant employing a novel step-size evolution to overcome fixed step-size limitations and improve computing efficiency [60]. The pseudocode of Algorithm 2 therefore serves as an overview representation of the core concepts, effectively capturing the algorithm’s essential chemotaxis, reproduction, and elimination-dispersal processes as implemented in this work.

Algorithm 2 Pseudocode of the standard Bacterial Foraging Optimization Algorithm (based on [18])

Input: N: Number of bacteria Nc: Number of chemotactic steps Nr: Number of reproduction steps Ned: Number of elimination-dispersal steps Ns: Swim length (max swim steps per tumble) Pe: Probability of elimination LB, UB: Search space bounds F: Evaluation (fitness) function Output: Best solution found

1.    Initialize N bacteria at random positions within [LB, UB] 2.    Evaluate health of each bacterium using F 3.    for ed = 1 to Ned do            // Elimination-Dispersal 4.        for rs = 1 to Nr do            // Reproduction 5.            for cs = 1 to Nc do           // Chemotaxis 6.                for each bacterium i do 7.                    Generate random tumble direction Δ 8.                    Normalize: φ = Δ/||Δ|| 9.                    Move: xnew = xi + C·φ        // C = step size 10.                  Evaluate F(xnew) 11.                  if F(xnew) < F(xi) then        // Improvement found 12.                      xi = xnew            // Accept new position 13.                      for s = 1 to Ns do         // Swim in same direction 14.                          xswim = xi + C·φ 15.                          if F(xswim) < F(xi) then 16.                              xi = xswim 17.                          else break          // Stop swimming 18.                      end for 19.                  end if 20.              end for (bacterium) 21.          end for (chemotaxis) 22.          Sort bacteria by health (ascending) 23.          Replace worst N/2 bacteria with copies of best N/2    // Reproduction 24.      end for (reproduction) 25.      for each bacterium i do           // Elimination-Dispersal 26.          if random() < Pe then 27.              Relocate xi to random position within [LB, UB] 28.          end if 29.      end for 30. end for (elimination-dispersal) 31. return best solution found

3.2.2. Adaptive BFOA

The Adaptive BFOA variant addresses a well-known limitation of the standard BFOA, which is a reliance on fixed hyperparameter values throughout the optimization process. In the standard algorithm, the step size C, the swim length N_s, and the elimination probability P_e remain constant, which can lead to suboptimal performance when the search landscape changes as the algorithm converges. The Adaptive BFOA overcomes this by introducing a self-tuning parameter control based on an improvement-rate feedback mechanism.

The variant modifies only the chemotaxis phase (Algorithm 2, lines 6–20) while inheriting the reproduction and elimination-dispersal steps from the standard BFOA. After each chemotaxis cycle, the algorithm computes the aggregate relative improvement across all bacteria. Based on this metric, three parameters are adapted simultaneously:

• Step size C: If the algorithm detects improvement (aggregate improvement > 0.01), the step size is increased to expand the search. When the population stagnates, it decays gradually. If stagnation persists beyond a threshold (five consecutive cycles without progress), the step size is reset to a balanced midpoint value, acting as a recovery mechanism.
• Swim length N_s: Increases alongside step size growth during productive phases and decreases during stagnation, bounded between minimum and maximum values.
• Elimination probability P_e: Increases during stagnation (promoting exploration through random relocation) and decreases during productive phases (preserving promising solutions).

This adaptation scheme enables the Adaptive BFOA to transition between broad exploration in early search phases and fine-grained exploitation as the population converges without requiring manual parameter tuning for different problem domains.

3.2.3. Improved BFOA (IBFOA)

The IBFOA introduces two structural modifications to the standard algorithm: Lévy flights in the chemotaxis phase and an elitism mechanism that preserves the best solution found across all iterations.

In the standard BFOA, bacteria move using a fixed step size C, producing uniform displacements during tumble operations. IBFOA replaces this with Lévy flight-based step sizes, generated using the Mantegna algorithm [61] with a stability exponent α = 1.5. Lévy flights produce step sizes drawn from a heavy-tailed distribution, where most steps are small (enabling local refinement) but occasional large jumps occur (enabling escape from local optima). This property makes IBFOA particularly effective in multimodal fitness landscapes where the standard BFOA’s fixed step size may be insufficient for escaping suboptimal regions.

The second modification is a global elitism mechanism. Throughout the optimization process, IBFOA maintains a record of the best solution ever found across all bacteria and all iterations. This elite solution is updated whenever any bacterium discovers a position with superior fitness, ensuring that the best known solution is never lost due to subsequent reproduction or elimination-dispersal operations.

Additionally, IBFOA simplifies the optimization loop by omitting the reproduction phase. Instead of sorting and replacing bacteria based on accumulated health, IBFOA relies on Lévy flights and elitism to maintain population quality. This simplification, combined with the absence of the computationally expensive swarming phase based on gradients used in the standard BFOA, makes IBFOA inherently faster while maintaining competitive optimization performance.

Table 1. Comparison of BFOA variants.

Feature	Standard BFOA	Adaptive BFOA	IBFOA
Step size	Fixed	Self-tuning (improvement-based)	Lévy flight (heavy-tailed)
Swim length	Fixed	Adaptive (increases/decreases)	Fixed
Elimination probability	Fixed	Adaptive (based on stagnation)	Fixed
Reproduction	Sort and replace	Sort and replace (inherited)	Omitted
Elitism	None	None	Global best preserved
Stagnation recovery	None	Parameter reset after threshold	Lévy jumps escape local optima
Swarming	Optional (based on gradient)	Optional (inherited)	Not used

summarizes the key differences between the three BFOA variants used in this work.

3.3. BFOA-XCS Integration

This section describes how BFOA replaces the GA within the XCS algorithm. The integration mechanism covers the mapping from classifier to the bacterium and the optimization cycle. Next, a detailed description is provided of the multi-objective fitness function that evaluates each bacterium during optimization.

Figure 1 illustrates the integration architecture, showing the data flow between XCS’s action set and the BFOA optimizer.

3.3.1. Replacing GA with BFOA in XCS

In standard XCS, the GA is invoked at lines 11 and 16 of Algorithm 1, where it applies selection, crossover, and mutation operators to the classifiers in the current action set [A]. In the proposed BFOA-XCS system, this GA invocation is replaced with a call to the BFOA optimizer, while the remainder of the XCS cycle remains unchanged (that includes [M] formation, PA generation, action selection, and Q-learning parameter updates). The integration operates through a three-phase process: encoding, optimization, and decoding.

When BFOA is triggered, each classifier in the action set [A] is converted into a bacterium whose position is a real-valued vector in a continuous search space. The position vector has D = L + 1 dimensions, where L is the condition length of the classifier and the additional dimension represents the classifier’s action value. Condition bits are encoded as follows: a don’t-care symbol ‘#’ is mapped to 0.5, a bit value of 0 is mapped to 0.25, and a bit value of 1 is mapped to 0.75. The action value is normalized to [0, 1] using the minimum and maximum values of the target attribute from the dataset. All dimensions are thus bound within [0, 1], creating a uniform search space for BFOA’s chemotactic movement.

With the bacteria population initialized from the [A] classifiers, the selected BFOA variant (Standard, Adaptive, or IBFOA) executes its full optimization cycle as described in Section 3.2. Each bacterium is evaluated using the fitness function described in Section 3.3.2. The number of bacteria equals the number of classifiers in the [A], meaning that BFOA directly optimizes the existing classifier population rather than generating new individuals. This is a fundamental difference from the GA approach, where crossover produces new offspring. BFOA instead adjusts the existing classifiers’ parameters through continuous movement in the search space.

After optimization completes, the optimized bacterium positions are mapped back to the classifiers. In the current implementation, only the action values are updated from the optimized positions, while the classifier conditions are preserved. This constitutes a deliberate separation of concerns within the XCS learning cycle: BFOA serves as an action-parameter optimizer that refines what each classifier proposes, while XCS’s covering mechanism and condition matching logic retain full responsibility for structural exploration, that is, generating new rule conditions that match previously unseen states. This separation avoids the disruptive effects that continuous-valued perturbations would have on XCS’s binary condition representation, where small numerical changes to encoded condition bits could fundamentally alter which states a classifier matches. By contrast, the standard GA modifies both conditions and actions through crossover, which can disrupt well-fitted classifier conditions in mature populations. BFOA’s restriction on action-parameter optimization is therefore not a limitation but an architectural advantage: it enables fine-grained refinement of classifier actions without risking the structural integrity of the rule population. For classification tasks, the optimized continuous action value is snapped to the nearest valid class label to ensure consistency with the discrete action space. For regression tasks, the continuous value is applied directly.

Unlike the GA, which is triggered based on the average time stamp of classifiers in the action set (the θ_GA threshold), BFOA is invoked at a configurable frequency (i.e., every f iteration of the XCS main cycle). This design choice reflects the different computational profiles of the two approaches: the GA operates with lightweight crossover and mutation on two selected parents, whereas BFOA executes a full nested optimization loop (N_ed × N_r × N_c chemotactic steps across all bacteria). Running BFOA at a reduced frequency (e.g., every 50 iterations) balances the computational cost while providing periodic population- level optimization that the GA’s operators at the individual level cannot achieve.

Algorithm 3 presents the pseudocode of the BFOA-XCS integration procedure, which replaces the GA calls at lines 11 and 16 of Algorithm 1.

Algorithm 3 Pseudocode of the BFOA integration into the XCS cycle (replaces GA at Algorithm 1, lines 11 and 16)

procedure RUN_BFOA(actionSet [A], sensory input σ)     1. if [A] is empty then return     2. N = number of classifiers in [A]     3. D = condition length + 1             // Map classifiers into a continuous-valued search space so BFOA’s             // chemotactic movement can operate on the action set     4. for each classifier cli in [A] do     5.      for each condition bit j do     6.          if cli.condition[j] = ‘#’ then pos[i][j] = 0.5     7.          else if cli.condition[j] = 1 then pos[i][j] = 0.75     8.          else pos[i][j] = 0.25     9.      end for   10.      pos[i][D] = normalize(cli.action, targetMin, targetMax)   11. end for         // Refine the action-set population through chemotaxis and         // population dynamics under the multi-criteria fitness   12. Select BFOA variant (Standard or Adaptive or IBFOA)   13. Initialize optimizer with pos, bounds = [0, 1]D, fitness function F   14. optimizedpos = optimizer.optimize()         // Write optimized action values back to the classifiers. Conditions         // are deliberately left unchanged to preserve XCS’s binary rule         // representation and avoid disrupting mature populations.   15. for each classifier cli in [A] do   16.      actionraw = denormalize(optimizedpos[i][D], targetMin, targetMax)   17.      if classification task then   18.          cli.action = snapToNearestClass(actionraw)   19.      else   20.          cli.action = actionraw   21.      end if   22. end for end procedure

3.3.2. Multi-Objective Classifier Fitness Function

A key advantage of integrating BFOA into XCS is the ability to evaluate classifier quality across multiple objectives simultaneously. In the standard GA-XCS, classifier fitness is derived solely from prediction accuracy. Classifier fitness is derived from prediction accuracy, which is computed from the classifier’s prediction error ε [58]. While effective, this single-objective approach does not account for the consistency or stability of classifier predictions over time. The proposed BFOA-XCS system introduces a multi-objective fitness function F that evaluates each bacterium (classifier) across three complementary objectives: prediction accuracy, prediction stability, and variance reduction.

The combined fitness is computed as a weighted sum:

F = w_a × S_accuracy + w_s × S_stability + w_v × S_variance,

(1)

where w_a = 0.60, w_s = 0.25, and w_v = 0.15 are the objective weights (Σw = 1), and the three score components S_accuracy, S_stability, and S_variance are defined below. This weighted-sum formulation constitutes a scalarized multi-criteria fitness rather than a Pareto-based multi-objective optimizer. The scalarized approach was chosen for computational efficiency, as evaluating a Pareto front within each BFOA chemotactic step would be prohibitively expensive for the inner-loop optimization setting. Since BFOA minimizes its objective function, the final cost passed to the optimizer is 1 − F, ensuring that higher-quality classifiers receive lower cost values.

Prediction Accuracy (S_accuracy) component measures how correctly a classifier’s action (predicted class or regression value) matches the expected output across the data instances to which the classifier’s condition applies. For classification tasks, accuracy is the fraction of correctly predicted instances. For regression tasks, it is derived from the normalized prediction error. The accuracy score is bounded in [0, 1], where 1.0 represents perfect prediction.

Prediction Stability (S_stability) component measures the consistency of a classifier’s predictions over a sliding window of recent evaluations. The fitness function maintains a Fitness Accumulator, a sliding window of size W (default W = 75) that records normalized prediction values. The stability score is computed as:

S_stability = 1 − min(1, σ²_w/0.25),

(2)

where σ²_w is the sample variance of the predictions within the window. A stability score of 1.0 indicates that the classifier consistently produces the same prediction, while a score near 0.0 indicates erratic behavior. Normalization constant 0.25 corresponds to the maximum possible variance for values bounded in [0, 1].

Variance Reduction (S_variance) component provides an additional penalty for prediction volatility using an Exponential Moving Average (EMA) of the variance. The EMA variance is updated incrementally:

V_ema = (1 − α)·(V_ema + α·δ²),

(3)

where α is the smoothing parameter (default α = 0.01) and δ = x_t − μ_ema is the deviation of the current prediction x_t from the EMA mean μ_ema. The variance score is then:

S_variance = 1 − min(1, V_ema/λ²),

(4)

where λ (default λ = 0.08) is a normalization parameter. The EMA-based variance provides a temporally smooth signal that responds gradually to changes, complementing the window-based stability score.

Accuracy alone can be misleading when a classifier achieves high accuracy sporadically but produces inconsistent predictions across different data presentations. The stability component rewards classifiers that produce consistent output, while the variance component penalizes long-term prediction drift. Together, these three objectives guide BFOA toward classifier populations that are not only accurate but also reliable. This multi-objective evaluation is made possible by BFOA’s population level optimization. Because BFOA evaluates all classifiers within [A] as a coordinated population of bacteria, it naturally supports simultaneous assessment across multiple fitness dimensions. By contrast, the standard GA evaluates individual classifiers in isolation, making multi-objective population level optimization less straightforward.

The weights w_a = 0.60, w_s = 0.25, w_v = 0.15 were established through a two-stage preliminary empirical study conducted prior to the main experimental grid. In the first stage, a grid sensitivity check across eight weight triples (8 triples × 2 runs × 2000 iterations) confirmed that mean accuracies vary by only 0.0017 across the candidate range, establishing that the operator comparison is robust to weight choice within this range. In the second stage, a four-configuration ablation on the Breast Cancer dataset (4 configurations × 3 runs × 3000 iterations) compared an accuracy-only baseline (1.00, 0.00, 0.00), a +Stability configuration (0.75, 0.25, 0.00), the proposed (0.60, 0.25, 0.15) triple, and a high-stability variant (0.50, 0.35, 0.15). All four configurations produced statistically indistinguishable mean accuracies, but the (0.60, 0.25, 0.15) triple produced the lowest run-to-run standard deviation (a 17.1% reduction relative to the accuracy-only baseline), precisely the property the variance-reduction component is intended to deliver. The selection criterion was therefore: among configurations of equivalent mean accuracy, the triple producing the lowest run-to-run variance is selected. A supplementary replication across four datasets from the main experimental grid (Breast Cancer, Iris, Heart Disease, Wine Quality) further confirmed that the alternative weight triple used in earlier code versions (0.85, 0.10, 0.05) is empirically inferior, losing by 6.21% accuracy on Heart Disease relative to the proposed triple. Accuracy receives the dominant weight to ensure that prediction quality remains the primary optimization objective, consistent with XCS’s accuracy-based fitness philosophy. Stability receives a secondary weight to promote consistent classifier behavior, and variance reduction receives a tertiary weight as a smoothing regularizer. These weights were held constant across all experiments in this work.

3.4. Cybersecurity Simulation Environment

To evaluate BFOA-XCS in a dynamic, multi-step RL setting, a research cybersecurity simulation environment was designed, which will be presented in this subsection. The environment models a network under active attack, where a defending agent (i.e., algorithm in testing) must select protective actions at each time step to prevent the spread of threats across the network. This section describes the network topology, state representation, action space, attack dynamics, reward structure, and scenario profiles.

3.4.1. Network Topology

The simulated network consists of 15 nodes arranged in a medium-size topology comprising three servers, ten workstations, and two routers (Figure 2). Nodes are connected through network edges, forming a graph structure where attack propagation follows the adjacent relationships. Each node maintains three properties relevant to the defense task: a compromised flag (indicating active infection), a defended flag (indicating active protection), and a list of known vulnerabilities, which are visually indicated with yellow markers in the figure.

3.4.2. State Representation

At each time step, the environment state is encoded as a feature vector of fixed size of 100 dimensions. The first five features capture global network status: the fraction of compromised nodes, the fraction of defended nodes, the normalized time step, the fraction of vulnerable (undefended, uncompromised) nodes, and the normalized cumulative damage. The remaining features encode per-node information, with four features per node: compromise status (binary), defense status (binary), normalized vulnerability count, and normalized connection degree. All features are bounded in [0, 1]. Each continuous feature is converted into a 4-bit binary representation for XCS’s condition matching, where the [0, 1] range is partitioned into 16 equal intervals, and the feature value is encoded as the corresponding 4-bit binary code. This produces a total condition length of 400 bits (100 features × 4 bits) for each classifier.

3.4.3. Action Space

The defending agent selects from a discrete action space of 90 actions, composed of 6 action types applied to each of the 15 network nodes (6 × 15 = 90). The action types are:

• Defend Node: Removes compromise and applies defense. Rewards correctly targeting compromised (+10) or vulnerable (+3) nodes and penalizes defending nodes that are already safe (−2 for safe, −1 for already defended).
• Isolate Node: Disconnects a node from the network. Rewards isolating compromised nodes (+7), penalizes isolating clean nodes (−3).
• Patch Node: Removes all vulnerabilities from a node and applies defense. Rewards patching vulnerable nodes (+5).
• Monitor Node: Places surveillance on a node. Small reward for monitoring suspicious nodes (+1).
• Restore Node: Reconnects a previously isolated node. Rewards restoring isolated nodes (+3).
• Do Nothing: No action is taken. Receives a constant penalty (−2) to discourage passivity.

This large, structured action space (90 discrete actions) represents a key characteristic of the experimental setting and is a challenging environment for many agents (e.g., neural-network agents that rely on softmax action selection) but should show as a strength for XCS, which uses rules as its modus operandi, where each classifier directly maps a state condition to a specific action.

3.4.4. Attack Dynamics

The environment simulates ongoing cyber-attacks through three concurrent threat vectors and a defense-decay mechanism that operate at each time step (Figure 3,

Table 2. Per-step threat vectors and defense-decay mechanism in the cybersecurity simulation environment. Each mechanism is evaluated independently at every environment step (Section 3.4.4). Symbols β_vul, β_clean, α_ext, γ_vul, and δ_decay denote per-step probabilities whose scenario-specific values are reported in Table 3.

Mechanism	Effect on Network State	Per-Step Dynamics	Immunity/Mitigation	Reference
Lateral spread	Compromised node attempts to compromise an adjacent neighbor	βvul if neighbor has known vulnerabilities; βclean otherwise (βvul > βclean)	Isolated and defended neighbors are immune	MITRE ATT&CK Lateral Movement (TA0008) [62]
External attacks	An external attacker compromises an undefended, vulnerable node directly	αext on each undefended, vulnerable node, independent of internal spread	Defended nodes are immune	MITRE ATT&CK Initial Access (TA0001) [62]
Vulnerability emergence	A new vulnerability appears on a node, simulating the discovery of a new exploit during the episode	γvul on each undefended, uncompromised node	Defended and compromised nodes do not acquire new vulnerabilities in this step	CVE/NVD vulnerability disclosure [63]; vulnerability-lifecycle literature [64,65]
Defense decay	An active defense lapses, returning the node to an undefended state	δdecay on each defended node	None: defense decay applies to all defended nodes	Static-defense-obsolescence framing in moving target defense literature [66]

). At every step t, the simulator updates the network state in a fixed order. First, lateral spread propagates compromise from infected nodes to their neighbors according to per-edge probabilities β_vul if the neighbor is vulnerable or β_clean if the neighbor is clean. Nodes that are isolated or currently defended are immune. Second, external attacks compromise undefended vulnerable nodes with probability α_ext per step, modeling pressure from outside the network. Third, new vulnerabilities emerge on uncompromised, undefended nodes with probability γ_vul per step, capturing the continuous discovery of latent software flaws. Finally, the defense-decay mechanism causes each defended node to lose its defensive posture with probability δ_decay per step, reflecting the finite lifetime of patches, monitoring rules, and operator attention. Once all four mechanisms have been applied to the state at step t, the defender agent observes the resulting state and selects one of six action types (Defend, Isolate, Patch, Monitor, Restore, or Do Nothing), across the fifteen network nodes, producing the network state at step t + 1.

Parameter symbols (β_vul, β_clean, α_ext, γ_vul, δ_decay) correspond to the per-step probabilities defined in Table 2, with scenario-specific values listed in

Table 3. Scenario profile parameters for the cybersecurity simulation environment. The Mixed scenario combines elements of all five preceding attack types using intermediate parameter values. It represents a realistic multi-vector threat environment rather than introducing a new attack type.

Parameter	DDoS	Intrusion	Ransomware	Port Scan	Data Exfil.	Mixed
Spread rate (vulnerable), βvul	0.1	0.06	0.15	0.04	0.12	0.08
Spread rate (clean), βclean	0.03	0.02	0.04	0.01	0.03	0.02
External attack rate, αext	0.04	0.01	0.01	0.06	0.02	0.03
Defense attack rate, δdecay	0.02	0.05	0.03	0.04	0.03	0.03
New vulnerability rate, γvul	0.04	0.08	0.02	0.12	0.04	0.06
Initial compromised	2	1	1	0	1	1
Initial vulnerable (%)	40	50	20	60	30	35
Terminal threshold (%)	60	60	55	60	60	60

.

3.4.5. Reward Structure

The reward function combines three components at each time step. First, an action reward provides immediate feedback based on the appropriateness of the selected action (as described in the previous subsection). Second, a state-change reward penalizes new node compromises (−8 per newly compromised node) and rewards successful remediation (+3 per cleaned node), with a constant time penalty (−0.3) to encourage efficient defense. Third, an episode termination reward is applied when the episode ends: a terminal penalty (−30) if the network reaches the compromise threshold or a survival bonus (up to +30, scaled by remaining network health) if the agent successfully maintains the network.

An episode terminates when either the fraction of compromised nodes exceeds the scenario-specific threshold (55–60%, depending on scenario) or the maximum step count (500) is reached.

3.4.6. Scenario Profiles

To evaluate agent adaptability across diverse threat landscapes, six distinct scenario profiles were designed, each with different attack rates, initial conditions, and dynamics. Table 3 summarizes the key parameters for each scenario.

4. Experimental Evaluation

This section presents the empirical evaluation of the proposed BFOA-XCS integration through two complementary experiments. Experiment 1 evaluates the BFOA-XCS variants on 19 benchmark datasets spanning classification, regression, and cybersecurity domains, establishing the general-purpose behavior of the proposed optimizer against the standard GA and a no-optimization baseline. Experiment 2 evaluates BFOA-XCS in a dynamic, multi-step cybersecurity defense setting against five deep reinforcement-learning baselines, demonstrating applicability beyond static supervised tasks. The two experiments share methodological choices (statistical validation, hyperparameter handling, hardware) but differ in task type, baselines, and evaluation metrics, motivating separate setup and results subsections within this unified evaluation chapter.

4.1. Experiment 1: Benchmark Machine Learning Datasets

The first experiment evaluates the BFOA-XCS integration on static classification and regression tasks using established benchmark datasets. The purpose is twofold: to verify that replacing the GA with BFOA does not degrade XCS’s operational capabilities and to assess whether BFOA’s multi-objective population optimization provides measurable improvements in prediction accuracy, stability, and variance reduction compared to the standard GA and the no-optimization baseline.

4.1.1. Experimental Setup

The experiment was conducted across 19 datasets spanning three categories: seven classification datasets, five regression datasets, and seven cybersecurity datasets (

Table 4. Datasets used in Experiment 1.

No.	Dataset	Source	Category	Instances	Target	Classes
1	Breast Cancer	UCI	Classification	569	diagnosis	2
2	German Credit	UCI	Classification	1000	class	2
3	Heart Disease	UCI	Classification	297	class	2
4	Ionosphere	UCI	Classification	351	class	2
5	Iris	UCI	Classification	150	class	3
6	Neurofibromatosis	Other	Classification	296	class	2
7	Wine	UCI	Classification	178	class	3
8	Auto MPG	UCI	Regression	392	mpg	41 bins
9	Boston Housing	UCI	Regression	506	MEDV	41 bins
10	Concrete Strength	UCI	Regression	1030	Strength	41 bins
11	Paddy Yield	Other	Regression	2789	yield	41 bins
12	Wine Quality	UCI	Regression	1143	quality	6 1
13	NSL-KDD (Binary) 2	Cyber	Cybersecurity	25,192	label	2
14	NSL-KDD (Multi) 2	Cyber	Cybersecurity	25,194	attack_cat	5
15	UNSW-NB15 (Binary) 2	Cyber	Cybersecurity	16,466	label	2
16	UNSW-NB15 (Multi) 2	Cyber	Cybersecurity	16,466	attack_cat	10
17	KDD Test+	Cyber	Cybersecurity	22,544	label	2
18	Ransomware (RG)	Cyber	Cybersecurity	2675	RG	2
19	Ransomware (Family)	Cyber	Cybersecurity	2675	family	41 bins 1

¹ Wine Quality was auto-detected as a six-class classification task despite its regression label due to having only six unique target values. Ransomware Family (41 unique families) was handled by the regression binning. ² 20% stratified random subset of the full dataset, used to maintain computational feasibility across the 5600-experiment grid.

). The majority of classification and regression datasets originate from the UCI Machine Learning Repository, supplemented by additional domain-specific datasets (Neurofibromatosis, Paddy Yield). The classification datasets range from well-known benchmarks (Iris, Breast Cancer, Wine) to more challenging problems (Ionosphere, Neurofibromatosis). The regression datasets cover diverse prediction tasks including housing prices, material strength, and wine quality. Regression targets were discretized into 41 equal-width bins, transforming the original regression task into an ordinal classification problem. This approach was chosen because XCS’s rule-based action matching and action set formation require discrete action labels. While bin-based accuracy does not capture ordinal proximity (e.g., a prediction one bin away is penalized the same as one twenty bins away), it provides a consistent evaluation metric across all dataset types in our experimental grid.

The primary purpose of the regression datasets in this study is comparative (i.e., evaluating relative operator performance rather than absolute regression quality), which mitigates this limitation. The cybersecurity datasets include two industry standard intrusion detection benchmarks (NSL-KDD (Network Security Laboratory–KDD; revised KDD Cup 1999 intrusion detection benchmark dataset) and UNSW-NB15 (University of New South Wales Network-Based 2015 intrusion detection dataset)), using 20% stratified random subsets to maintain computational feasibility across the experimental grid, evaluated in both binary and multiclass settings, one additional intrusion detection benchmark (KDD Test+), and a ransomware dataset evaluated with two different targets (binary detection and family classification).

For all datasets, numeric features (excluding the target column) were normalized using min–max scaling, with the minimum and maximum values computed from the training split and propagated to the test and validation splits to prevent information leakage. The target column was excluded from the state representation during encoding. Categorical features were encoded using binary encoding with dataset-specific category maps. For the Ransomware dataset, per-dataset column exclusions were applied to prevent target leakage: the ‘family’ column was excluded when predicting ‘RG’, and the ‘RG’ column was excluded when predicting ‘family’. Additionally, identifier columns (e.g., IDs, filenames, timestamps) were excluded globally across all datasets. Missing values in the Neurofibromatosis dataset (~13% of instances) were handled through the encoder’s don’t-care mechanism, which maps missing values to generalized classifier conditions.

Five XCS operator configurations were compared: NONE (no optimization: XCS runs without any evolutionary or swarm-based operator), GA (standard genetic algorithm), BFOA, Adaptive BFOA, and IBFOA. Each configuration was evaluated across four P# (don’t-care probability) values: 0.33, 0.50, 0.60, and 0.70. Each configuration was executed 10 times with different random seeds. Classification and cybersecurity datasets were evaluated at both 5000 and 10,000 training iterations, while regression datasets used 5000 iterations. This yielded a total of 5600 experiments (19 datasets × 5 operators × varying configurations × 10 runs).

All datasets were split using stratified sampling with a 70/20/10 train/test/validation ratio. XCS was configured with a population size of 500 classifiers, exploration rate of 0.10, crossover probability of 0.80, mutation probability of 0.04, GA threshold θ_GA = 25, and subsumption threshold of 20 with both action set and GA subsumption enabled. The holdout test set (20%) was used for all reported accuracy values.

BFOA parameters were configured as follows: invocation frequency every 100 iterations, step size C = 0.02, chemotactic steps N_c = 25, reproduction steps N_r = 3, elimination-dispersal steps N_ed = 1, swim length N_s = 4, bacteria population capped at 64, swarming disabled, and a warmup period of 20% of total iterations (during which BFOA is not invoked, allowing XCS to build an initial classifier population). The multi-objective fitness function was enabled with weights w_a = 0.60, w_s = 0.25, and w_v = 0.15 as described in Section 3.3.2. An additional ablation run with multi-objective fitness disabled, which is an accuracy-only evaluation, was performed under otherwise identical conditions to isolate the contribution of the multi-objective fitness function.

Statistical validation followed the methodology recommended by Demšar [67] for comparing multiple classifiers across multiple datasets: the Friedman test for global ranking with the Iman–Davenport adjustment and the Wilcoxon signed-rank test for targeted pairwise comparisons. Effect sizes were quantified using Cohen’s d [68]. Variance reduction was computed as the percentage decrease in prediction standard deviation relative to the GA baseline. The Friedman test significance level was set to α = 0.10 a priori, consistent with the exploratory nature of this study: the primary goal is to identify whether BFOA optimization variants warrant further investigation as GA replacements, rather than to confirm a pre-registered hypothesis at the more conservative α = 0.05 level. Operator comparisons in this experiment rely on convergent evidence across multiple indicators (Friedman ranking, pairwise Wilcoxon, effect sizes, and an ablation study) rather than on any single test.

All experiments were conducted on a desktop PC equipped with an Intel Core i7-9700 processor and 32 GB of RAM, running Windows 11 Pro. The software environment consisted of OpenJDK 22, with IntelliJ IDEA 2025.3.1 (Community Edition) used as the development environment.

4.1.2. Results and Analysis

This section presents the results of the benchmark experiment, organized into four parts: the overall operator ranking, per-dataset accuracy analysis, variance reduction analysis, and an ablation study of the multi-objective fitness function.

Overall Operator Ranking

Table 5. Friedman test rankings across 19 datasets (lower rank = better, MO = true).

Rank	Operator	Avg. Rank
1	IBFOA	2.395
2	NONE	2.553
3	BFOA	3.000
4	GA	3.368
5	Adaptive BFOA	3.684

presents the Friedman test results for all five operators across the 19 datasets. The Friedman test yielded χ² = 8.895 with p = 0.064, marginally significant at the a priori α = 0.10 level (justified in Section 4.1.1). The Iman–Davenport corrected statistic was F = 2.386 with p = 0.059. The consistent ranking pattern and the supporting effect-size analysis (discussed below) provide convergent evidence of operator differences.

The key finding is that IBFOA achieves the best Friedman ranking, while the standard GA ranks fourth out of five operators. All three BFOA variants (BFOA, Adaptive BFOA, and IBFOA) together hold ranks 1, 3, and 5, with IBFOA and standard BFOA both outranking the GA. The NONE baseline ranks second, which indicates that the standard GA optimization in XCS does not consistently improve upon the unoptimized baseline across diverse datasets, a finding that supports the motivation for exploring alternative optimization approaches.

Per-Dataset Accuracy

The per-dataset accuracies in

Table 6. Best holdout accuracy per operator per dataset (MO = true). Bold indicates the best operator for each dataset.

Dataset	NONE	GA	BFOA	Adaptive BFOA	IBFOA
Breast Cancer	0.762	0.758	0.720	0.717	0.772
German Credit	0.688	0.704	0.687	0.687	0.697
Heart Disease	0.766	0.764	0.729	0.741	0.751
Ionosphere	0.759	0.777	0.741	0.754	0.739
Iris	0.937	0.920	0.883	0.837	0.883
Neurofibromatosis	0.559	0.544	0.566	0.548	0.556
Wine	0.800	0.718	0.777	0.747	0.741
Auto MPG	0.381	0.379	0.381	0.381	0.381
Boston Housing	0.518	0.517	0.518	0.519	0.518
Concrete Strength	0.563	0.563	0.563	0.563	0.563
Paddy Yield	0.328	0.328	0.328	0.328	0.328
Wine Quality	0.512	0.433	0.524	0.537	0.539
NSL-KDD (Binary)	0.534	0.544	0.543	0.532	0.534
NSL-KDD (Multi)	0.365	0.368	0.394	0.372	0.365
UNSW-NB15 (Binary)	0.449	0.452	0.462	0.462	0.449
UNSW-NB15 (Multi)	0.288	0.249	0.211	0.198	0.211
KDD Test+	0.681	0.569	0.655	0.618	0.654
Ransomware (RG)	0.885	0.785	0.863	0.836	0.875
Ransomware (Family)	0.772	0.772	0.771	0.771	0.771

are presented under oracle configuration selection: for each operator–dataset pair, the (P#, iteration count) configuration that produced the highest mean accuracy over 10 runs is selected post hoc, and the corresponding holdout accuracy is reported. Table 6 therefore presents upper-bound per-operator performance for illustrative comparison only and is not the basis for the statistical claims of this paper. The statistical analyses elsewhere in Section 4.1.2 (Friedman rankings, effect sizes, and variance reduction) are computed across all configurations without post hoc selection, ensuring that the evidential claims are not affected by configuration-level optimistic bias.

In terms of dataset wins (highest accuracy per dataset), NONE achieved the best accuracy on 10 of 19 datasets, GA on 6, BFOA on 6, Adaptive BFOA on 5, and IBFOA on 5 (ties counted for every operator that achieves the best value). Two patterns underlie these counts. First, on the three regression datasets (Auto MPG, Concrete Strength, Paddy Yield), all operators converge to nearly identical accuracies and therefore tie, contributing equally to every operator’s total. Second, NONE’s lead in unique wins (i.e., outright best, not shared) comes from several cybersecurity datasets (KDD Test+, Ransomware RG, UNSW-NB15 Multi) and a few classification datasets (Iris, Wine), where the unoptimized baseline outperforms every evolutionary and swarm-based operator. This pattern suggests that, on datasets where XCS’s native learning mechanisms already find near-optimal rules, adding optimization provides minimal benefit.

However, where optimization does make a difference, BFOA variants outperform the GA. On Breast Cancer, IBFOA achieves the highest accuracy (0.772 vs. GA’s 0.758). On Wine Quality, IBFOA (0.539) substantially outperforms GA (0.433). On NSL-KDD Multiclass, standard BFOA (0.394) outperforms both GA (0.368) and NONE (0.365). On Neurofibromatosis, BFOA (0.566) achieves the best result, where GA (0.544) underperforms even the baseline.

Effect Sizes and Pairwise Comparisons

While the Wilcoxon signed-rank tests comparing each operator against GA did not yield individually significant p-values (p = 0.220–1.000), the Cohen’s d effect sizes reveal consistent and meaningful differences in favor of all operators over GA (

Table 7. Pairwise comparisons against GA (MO = true).

Comparison	Wilcoxon p	Cohen’s d	Magnitude	Direction
GA vs. NONE	0.220	0.803	Large	NONE > GA
GA vs. BFOA	0.457	0.693	Medium	BFOA > GA
GA vs. Adaptive BFOA	1.000	0.623	Medium	Adaptive BFOA > GA
GA vs. IBFOA	0.433	0.737	Medium	IBFOA > GA

).

All four comparisons show medium-to-large effect sizes favoring the alternative operator over GA. The largest effect size is observed for NONE vs. GA (d = 0.803), suggesting that the standard GA may, in some cases, disrupt otherwise functional classifier populations through its crossover and mutation operations. Among the BFOA variants, IBFOA shows the strongest effect (d = 0.737), followed by BFOA (d = 0.693) and Adaptive BFOA (d = 0.623). Importantly, these effect sizes are computed across all 19 datasets using the best configuration per dataset, reflecting a robust comparison that is not sensitive to individual dataset selection.

Variance Reduction

A central claim of the multi-objective fitness function (Section 3.3.2) is that it simultaneously optimizes accuracy, stability, and variance reduction.

Table 8. Variance reduction relative to GA (MO = true).

Operator	Variance Reduction	Interpretation
NONE	4.3%	Slight reduction
BFOA	4.7%	Slight reduction
Adaptive BFOA	1.8%	Slight reduction
IBFOA	15.2%	Notable reduction

presents the variance reduction achieved by each operator relative to GA.

IBFOA achieves the most substantial variance reduction at 15.2% relative to GA, meaning that IBFOA’s predictions are considerably more consistent across datasets and runs. All operators show positive variance reduction relative to GA, indicating that the GA’s stochastic crossover and mutation operators introduce prediction instability that the alternatives avoid.

Multi-Objective Fitness Ablation

To isolate the contribution of the multi-objective fitness function, a complete ablation experiment was conducted under identical conditions with the multi-objective fitness disabled (accuracy-only evaluation).

Table 9. Multi-objective fitness ablation: MO = true vs. MO = false.

Metric	MO = True	MO = False
Friedman χ2	8.895	6.200
Friedman p-value	0.064	0.185
Best Friedman rank	IBFOA (2.395)	NONE (2.447)
GA Friedman rank	4th (3.368)	4th (3.158)
Operator	Variance reduction (MO = true)	Variance reduction (MO = false)
NONE	4.3% ↓	−1.7% (increase)
BFOA	4.7% ↓	−5.2% (increase)
Adaptive BFOA	1.8% ↓	−9.3% (increase)
IBFOA	15.2% ↓	6.7% ↓

↓ indicates a reduction in prediction variance relative to the GA baseline (the desired outcome). Values labeled (increase) indicate higher variance than GA.

compares the key metrics between both conditions.

The ablation reveals three important findings. First, the multi-objective fitness function produces a substantially stronger statistical signal, with the Friedman p-value improving from 0.185 (not significant) to 0.064 (marginally significant). Second, with multi-objective fitness enabled, IBFOA is promoted from second to first in the Friedman ranking, while without it, the unoptimized baseline (NONE) ranks first, indicating that the accuracy-only fitness landscape is too flat for the metaheuristic optimizers to provide meaningful differentiation. Third, and most critically, the multi-objective fitness function delivers consistent variance reduction across all operators, whereas under accuracy-only evaluation, three of four operators (NONE, BFOA, Adaptive BFOA) exhibit increased variance relative to GA. This confirms that the stability and variance components of the fitness function (Equations (2)–(4) in Section 3.3.2) are operating as intended, rewarding stable classifier populations and penalizing erratic prediction behavior.

Summary of Key Findings

The first experiment establishes four key results. First, replacing GA with BFOA does not degrade XCS’s performance. IBFOA achieves the top Friedman ranking across 19 datasets (p = 0.064, marginally significant at α = 0.10), while GA consistently ranks fourth. Second, the multi-objective fitness function substantially enhances the effectiveness of BFOA-based optimization, improving the Friedman p-value from 0.185 to 0.064 and producing consistent variance reduction. Third, the effect sizes confirm that all operators outperform GA with medium-to-large effects (Cohen’s d = 0.623–0.803), demonstrating that the GA component is the weakest optimization choice for XCS across diverse problem domains. Fourth, IBFOA emerges as the strongest variant, combining the best Friedman rank with the largest variance reduction (15.2%), validating the combination of Lévy flights and global elitism for classifier population optimization.

4.2. Experiment 2: Dynamic Cybersecurity Defense

The second experiment evaluates BFOA-XCS in a fundamentally different setting: a dynamic, multi-step cybersecurity simulation where an intelligent defender must make sequential decisions against ongoing attack scenarios. Unlike the static classification and regression tasks of Experiment 1, this environment requires each agent to learn a defensive policy through online interaction with a simulated network under six distinct attack profiles. The experiment compares the twenty XCS agent configurations against six DRL baselines, providing insight into how rule-based and neural approaches perform in structured, high-dimensional discrete action spaces.

4.2.1. Experimental Setup

The experiment uses the cybersecurity simulation environment described in Section 3.4, with the six scenario profiles defined in Table 2: DDoS flooding, network intrusion, ransomware propagation, port scanning reconnaissance, data exfiltration, and a mixed-attack scenario that combines elements of all five. The network topology consists of 15 nodes (three servers, ten workstations, two routers), resulting in a state vector of 100 continuous dimensions (five global features: fraction of compromised nodes, fraction of defended nodes, normalized time step, fraction of vulnerable nodes, and normalized cumulative damage; plus four per-node features for each node: compromise status, defense status, normalized vulnerability count, and normalized connection degree) and a discrete action space of 90 actions (six defensive action types, Defend Node, Isolate Node, Patch Node, Monitor Node, Restore Node, and Do Nothing, applied to each of the 15 nodes).

The simulation environment is intended as a controlled abstraction that captures the dynamics most directly relevant to closed-loop defensive policy learning rather than as a faithful replica of an enterprise network. Aspects such as encrypted traffic, heterogeneous device configurations, partial observability beyond the modeled state vector, and adversarial co-adaptation are not modeled (see Section 5.2).

Twenty XCS agent configurations were tested: five operators (NONE, GA, BFOA, Adaptive BFOA, IBFOA) crossed with four P# values (0.33, 0.50, 0.60, 0.70). Each XCS agent was configured with a population of 500 classifiers, 4 bits per feature (producing 400-bit conditions), and an exploration rate decaying from 0.10 to 0.01 (decay factor 0.995 per episode). BFOA parameters were adjusted relative to Experiment 1 to suit the dynamic environment: the invocation frequency was set to every 50 episodes (resulting in twenty BFOA optimizations per 1000-episode run, compared to every 100 iterations in Experiment 1), and the inner loop budget was increased to N_c = 50, N_r = 4, N_ed = 2 (compared to N_c = 25, N_r = 3, N_ed = 1 in Experiment 1). These changes double the optimization cadence and increase the per-trigger computational effort, providing more thorough population refinement in the dynamic setting. The multi-objective fitness function was enabled with the same weights as Experiment 1 (w_a = 0.60, w_s = 0.25, w_v = 0.15).

Six DRL baselines were included for comparison. All neural agents were implemented from scratch in Java with manual backpropagation, as no production-ready pure Java implementations of PPO or SAC exist, to the best of the authors’ knowledge. Existing open-source PPO/SAC implementations (e.g., RLlib, version 2.55.1, part of the Ray framework) are available only in Python/C++. Implementation correctness was verified through a diagnostic test suite covering reward normalization stability, loss convergence, and action distribution analysis across multiple reward scales, with both PPO and SAC passing all eight diagnostic tests.

The baseline configurations used in the experiment were:

• DQN (Deep Q-Network): learning rate 0.001, discount factor γ = 0.99, ε-greedy exploration with ε = 0.1.
• Q-Learning: tabular with learning rate 0.1, γ = 0.95, ε = 0.1, and state discretization into 10 bins.
• Policy Gradient (REINFORCE): the foundational policy gradient algorithm [69], learning rate 0.001, γ = 0.95.
• PPO (Proximal Policy Optimization): learning rate 0.001, γ = 0.99, clip ε = 0.2, GAE-λ (Generalized Advantage Estimation) = 0.95, 4 epochs per update. For PPO, the clip-region check in the surrogate objective uses a non-strict comparison: the unclipped policy gradient is applied whenever surr1 ≤ surr2, ensuring that the gradient flows when the importance ratio lies inside the clip interval [1 − ε, 1 + ε], including at its boundary. This is consistent with the surrogate objective specified by Schulman et al. (2017) [70].
• SAC (Soft Actor–Critic, discrete variant): actor learning rate 0.00015, critic learning rate 0.0003, γ = 0.99, initial entropy coefficient α = 0.2 with automatic tuning, target entropy ratio 0.5, Polyak τ = 0.005, replay buffer capacity 10,000, batch size 64, warmup 128 steps.
• Random: uniformly random action selection (lower-bound baseline).

All neural baseline agents use feedforward architectures with Leaky ReLU activations (α = 0.01) in hidden layers and linear output layers. DQN uses a single network with architecture [100, 100, 50, 90] (input, two hidden layers, output). Policy Gradient uses the same architecture for its policy network. PPO uses separate actor [100, 100, 50, 90] and critic [100, 100, 50, 1] networks. SAC uses larger networks due to its five-network architecture: actor [100, 200, 100, 90] and four critic networks (two online, two target) each [100, 200, 100, 90]. All networks use He initialization and element-wise gradient clipping (clip value 1.0). Input normalization (Welford’s online algorithm for running mean and variance) and reward normalization (Welford’s with clip to ±10) are applied in DQN, PPO, and SAC. DRL hyperparameters were set to standard values from the original publications without environment-specific tuning, consistent with the XCS configurations that also used fixed hyperparameters across all scenarios.

Each of the 26 agent configurations (twenty XCS and six baselines) was run ten times per scenario with fixed random seeds (base seed 42), resulting in 1560 total runs (26 × 6 × 10). Each run consisted of 1000 training episodes with a maximum of 300 steps per episode. The primary performance metric is the mean cumulative reward across all episodes, which reflects the agent’s overall policy quality including the learning trajectory. The total experiment runtime was approximately 18 h 49 min. Statistical validation employed the same methodology as Experiment 1: the Friedman test with Iman–Davenport correction for global ranking, the Nemenyi post hoc test for multiple comparisons, the Wilcoxon signed-rank test for targeted pairwise comparisons, and Cohen’s d for effect size quantification. The Wilcoxon test uses 60 per-run paired observations (six scenarios × ten runs) to ensure sufficient statistical power.

4.2.2. Results and Analysis

This section presents the results of the cybersecurity defense experiment, organized into three parts: overall agent ranking, per-scenario analysis, and the sensitivity to P# (don’t-care probability) configuration.

Overall Agent Ranking

Table 10. Overall agent ranking in the cybersecurity simulation (1560 runs), sorted by mean cumulative reward. Only the top 10 XCS configurations (positive mean reward) and all 6 baselines are shown. The remaining 10 XCS configurations (P# = 0.33 and P# = 0.70) achieved negative mean rewards and occupy Friedman ranks 13–22.

Rank	Agent	Mean Reward	Std	Friedman Rank	Avg Time
1	PPO	+307.57	283.53	1.167	2 m 8 s
2	SAC	+110.65	140.39	1.833	10 m 26 s
3	Adaptive BFOA-XCS_P0.50	+6.60	43.60	4.833	24 s
4	GA-XCS_P0.50	+6.42	44.20	6.500	24 s
5	IBFOA-XCS_P0.50	+6.41	44.66	6.500	24 s
6	NONE-XCS_P0.50	+6.05	44.26	7.000	24 s
7	BFOA-XCS_P0.50	+5.82	43.45	7.000	24 s
8	IBFOA-XCS_P0.60	+3.78	41.06	8.167	16 s
9	GA-XCS_P0.60	+3.70	40.74	7.500	16 s
10	NONE-XCS_P0.60	+3.57	40.11	7.667	16 s
11	BFOA-XCS_P0.60	+3.09	40.50	9.333	16 s
12	Adaptive BFOA-XCS_P0.60	+2.52	40.12	10.500	16 s
23	DQN	−28.34	43.87	23.000	38 s
24	Q-Learning	−47.98	21.77	24.000	<1 s
25	Random	−80.12	20.29	25.167	<1 s
26	Policy Gradient	−86.12	11.22	25.833	4 s

presents the overall ranking of all 26 agent configurations by mean cumulative reward, alongside the Friedman test rankings across the six scenarios. The Friedman test yielded χ² = 136.45 with p ≈ 0 (Iman–Davenport F = 50.35, p ≈ 0), confirming highly significant differences among agents.

PPO achieves the highest overall performance, with a mean reward of +307.57 and the best Friedman rank (1.167), followed by SAC (mean +110.65, rank 1.833). All XCS variants at P# = 0.50 achieve positive mean rewards (+5.82 to +6.60) and occupy Friedman ranks 4.833 to 7.000, forming a third tier. Among the XCS configurations, Adaptive BFOA-XCS at P# = 0.50 achieves the best Friedman rank (4.833), followed by GA-XCS (6.500), IBFOA-XCS (6.500), NONE-XCS (7.000), and BFOA-XCS (7.000). DQN is the best-performing of the remaining DRL baselines (rank 23.000, mean −28.34) but scores 34 points below the weakest XCS P# = 0.50 variant. Q-Learning (rank 24.000, mean −47.98), Random (rank 25.167, mean −80.12), and Policy Gradient (rank 25.833, mean −86.12) occupy the bottom of the ranking. The Friedman ranking visualization is presented in Figure 4.

Per-Scenario Analysis

Table 11. Per-scenario mean cumulative reward for selected agents.

Scenario	SAC	Adaptive BFOA-XCS	BFOA-XCS	NONE-XCS	DQN	PPO	Random
DDoS	+25.41	−26.64	−27.30	−28.50	−41.54	+87.45	−68.75
Intrusion	+197.88	+80.20	+80.85	+81.85	−0.45	+528.36	−54.20
Ransomware	+265.83	+37.09	+32.86	+34.97	+1.37	+563.30	−119.80
Port Scan	−6.11	−53.04	−53.40	−54.17	−60.96	−29.73	−83.51
Data Exfil.	+106.88	+12.12	+11.30	+11.46	−24.99	+284.72	−82.70
Mixed	+74.03	−10.13	−9.42	−9.30	−43.50	+411.33	−71.77

presents the mean reward for selected agents across all six scenarios. Figure 5 provides the corresponding visual comparison.

Three performance tiers emerge consistently across the scenarios. The first tier consists of PPO and SAC, which achieve the highest rewards across most scenarios. PPO leads in four of the six scenarios (Intrusion +528.36, Ransomware +563.30, Data Exfiltration +284.72, Mixed +411.33), with SAC leading on DDoS (+25.41) and Port Scan (−6.11). Both top neural agents exhibit particularly strong performance on the most learnable scenarios, such as Ransomware and Intrusion. The second tier comprises all XCS variants at P# = 0.50, which achieve positive rewards on three of six scenarios (Intrusion, Ransomware, and Data Exfiltration) and moderately negative rewards on the others. The third tier contains DQN, Q-Learning, Random, and Policy Gradient, which achieve negative rewards across nearly all scenarios, with mean overall rewards in the −28 to −86 range.

Within the XCS tier, the per-scenario differences between operators are small. On Intrusion, NONE-XCS (+81.85) slightly outperforms BFOA-XCS (+80.85) and Adaptive BFOA-XCS (+80.20), suggesting that, for high-signal scenarios, XCS’s native learning mechanisms suffice. On Ransomware, Adaptive BFOA (+37.09) outperforms all other XCS operators, while on Data Exfiltration, Adaptive BFOA (+12.12) also leads. The most challenging scenario for all agents is Port Scan, where even SAC achieves only −6.11. The stealthy, low-signal nature of reconnaissance attacks makes defensive decision-making inherently difficult.

P# Sensitivity

The don’t-care probability (P#) has a strong and consistent effect on performance across all XCS operators. P# = 0.50 produces the best results (mean +5.82 to +6.60), followed by P# = 0.60 (+2.52 to +3.78). Both P# = 0.33 and P# = 0.70 yield negative mean rewards. At P# = 0.33, classifiers are too specific, producing narrow rules that fail to generalize across the diverse states encountered in the simulation. At P# = 0.70, classifiers are too general, producing overly broad rules that cannot discriminate between meaningful state differences. The optimal range of P# = 0.50–0.60 produces classifiers with balanced generality. That is specific enough to capture decision-relevant state features while general enough to transfer across related states within each scenario.

Classifier Population Analysis

To address scalability and population complexity associated with the 400-bit condition representation used in Experiment 2, three per-run classifier-population diagnostics were extracted from the cybersecurity-simulation logs at P# = 0.50 (the configuration that produced the best results), aggregated across the 30 (operator, scenario) cells: the final classifier population size [P], the mean condition specificity (the fraction of non-don’t-care bits per classifier), and the mean match-set size [M] in the steady-state (final 100) episodes of each run. The full per-cell results are reported in

Table 12. Per-cell classifier-population diagnostics at P# = 0.50 (30 operator × scenario combinations, 10 runs per cell). Population size equilibrates at the configured cap of 500 (zero variation across runs); condition specificity is consistent with the P# = 0.50 design point (≈0.5). Match-set size is approximately 6–8% of the population across all cells.

Operator	Scenario	Final Population Size	Mean Condition Specificity	Mean Match-Set Size
NONE	DDoS	500	0.4994 ± 0.0010	38.06 ± 1.41
NONE	Intrusion	500	0.5003 ± 0.0014	34.33 ± 0.89
NONE	Ransomware	500	0.5005 ± 0.0013	30.29 ± 1.29
NONE	Port Scan	500	0.5001 ± 0.0007	40.35 ± 1.37
NONE	Data Exfiltration	500	0.5000 ± 0.0015	34.59 ± 1.21
NONE	Mixed	500	0.4996 ± 0.0009	36.25 ± 0.82
GA	DDoS	500	0.4999 ± 0.0010	37.76 ± 1.92
GA	Intrusion	500	0.4998 ± 0.0009	33.96 ± 0.69
GA	Ransomware	500	0.5000 ± 0.0010	29.67 ± 0.69
GA	Port Scan	500	0.5001 ± 0.0014	40.70 ± 1.03
GA	Data Exfiltration	500	0.5001 ± 0.0009	34.25 ± 0.73
GA	Mixed	500	0.5001 ± 0.0011	36.43 ± 1.20
BFOA	DDoS	500	0.4997 ± 0.0011	38.61 ± 1.45
BFOA	Intrusion	500	0.5000 ± 0.0010	33.64 ± 0.92
BFOA	Ransomware	500	0.5003 ± 0.0009	29.66 ± 1.27
BFOA	Port Scan	500	0.5004 ± 0.0009	39.89 ± 1.33
BFOA	Data Exfiltration	500	0.5002 ± 0.0011	34.55 ± 1.04
BFOA	Mixed	500	0.4993 ± 0.0011	36.68 ± 1.01
Adaptive BFOA	DDoS	500	0.5003 ± 0.0008	37.70 ± 1.58
Adaptive BFOA	Intrusion	500	0.4994 ± 0.0008	33.74 ± 1.00
Adaptive BFOA	Ransomware	500	0.4996 ± 0.0012	29.54 ± 1.20
Adaptive BFOA	Port Scan	500	0.5000 ± 0.0013	39.91 ± 1.52
Adaptive BFOA	Data Exfiltration	500	0.4997 ± 0.0011	34.03 ± 0.87
Adaptive BFOA	Mixed	500	0.5010 ± 0.0012	36.25 ± 0.66
IBFOA	DDoS	500	0.5002 ± 0.0008	37.53 ± 1.78
IBFOA	Intrusion	500	0.4996 ± 0.0008	33.68 ± 0.85
IBFOA	Ransomware	500	0.4998 ± 0.0010	30.16 ± 0.63
IBFOA	Port Scan	500	0.4997 ± 0.0009	40.16 ± 1.00
IBFOA	Data Exfiltration	500	0.4999 ± 0.0010	34.44 ± 1.01
IBFOA	Mixed	500	0.5001 ± 0.0011	36.64 ± 0.93

. The summary findings follow.

First, the average final classifier population size was 500 with zero variation across cells. XCS’s deletion-pressure mechanism reaches and holds the configured cap of 500 in every run, confirming that the 400-bit condition representation does not cause population growth beyond the design budget. Second, the average condition specificity was 0.5 (with a maximum cross-cell deviation of 0.0001), indicating that, on average, approximately 200 of the 400 bits per classifier condition are informative while the remainder are don’t-care wildcards, precisely the value expected at P# = 0.50 under uniform covering, confirming that the representation is not pathologically sparse and that no optimization operator drifts it from the design point. Third, the average [M] size in the steady-state episodes was 35.45 classifiers (i.e., approximately 7.1% of the population matches a typical state, with cell-level means ranging from 29.5 (Ransomware) to 40.7 (Port Scan)). This is well above the percent-level threshold below which sparse-match collapse would be a concern, and the moderate cross-scenario spread reflects the differing state-distribution diversity of the scenarios rather than any structural pathology.

The three metrics are statistically indistinguishable across the five optimization operators (per-operator means differ by at most 0.5 classifiers in [M] size and at most 0.0001 in specificity, both well below within-cell noise). This operator-agnostic structure is consistent with the separation-of-concerns architecture described in Section 3.3.1: because BFOA optimization refines classifier actions rather than conditions, the population-level structural statistics are determined by XCS’s covering and matching mechanisms and not by the choice of optimization operator. Together, these three diagnostics directly imply the following: the population is bounded, the conditions are not degenerate, and the [M] is dense enough for meaningful action selection.

4.2.3. Comparison with Deep Reinforcement Learning Baselines

This section analyzes the performance differences between XCS and the DRL baselines, addressing three aspects: statistical significance, the architectural factors underlying each agent’s performance, and the computational cost trade-off.

Statistical Significance

Table 13. Pairwise comparisons against BFOA-XCS_P0.50 (Wilcoxon signed-rank, n = 60 per-run pairs).

Comparison	Wilcoxon p	Cohen’s d	Magnitude	Winner
SAC vs. BFOA-XCS	<0.001	1.000	Large	SAC
DQN vs. BFOA-XCS	<0.001	0.776	Medium	BFOA-XCS
Q-Learning vs. BFOA-XCS	<0.001	1.552	Large	BFOA-XCS
PPO vs. BFOA-XCS	<0.001	1.179	Large	PPO
Policy Gradient vs. BFOA-XCS	<0.001	2.873	Large	BFOA-XCS
Random vs. BFOA-XCS	<0.001	2.513	Large	BFOA-XCS
NONE-XCS vs. BFOA-XCS	0.868	0.005	Negligible	—
NONE-XCS vs. Adaptive BFOA-XCS	0.284	0.012	Negligible	—

presents the Wilcoxon signed-rank tests and Cohen’s d effect sizes for key pairwise comparisons against BFOA-XCS_P0.50 (the representative XCS variant).

BFOA-XCS significantly outperforms three of five DRL baselines (DQN, Q-Learning, Policy Gradient), with medium-to-large effect sizes (d = 0.776–2.873), and also significantly outperforms the Random baseline (d = 2.513). Both PPO and SAC significantly outperform BFOA-XCS in pairwise comparison (PPO: Wilcoxon p < 0.001, d = 1.179; SAC: Wilcoxon p < 0.001, d = 1.000), with large effect sizes in both cases. The Nemenyi post hoc test (critical difference CD = 17.505 at α = 0.01) confirms that XCS variants significantly outperform the bottom-tier DRL baselines (DQN, Q-Learning, Random, Policy Gradient) and that PPO and SAC significantly outperform the same DRL baselines. Notably, however, neither PPO nor SAC significantly outperforms any XCS variant at P# ∈ {0.50, 0.60} under Nemenyi correction (maximum rank difference 9.333 for PPO and 8.667 for SAC, both well below CD = 17.505), meaning that, while PPO and SAC clearly lead in pairwise Wilcoxon comparison, the advantage over XCS does not survive the conservative multiple-comparison correction. Within the top tier of agents (PPO, SAC, and XCS variants at P# ∈ {0.50, 0.60}), no pair is statistically distinguishable at α = 0.01 under Nemenyi.

Within the XCS family, pairwise Wilcoxon tests show no significant differences between operators (p = 0.220–0.868, d = 0.005–0.014), indicating that the choice of optimization operator has a negligible impact on overall performance in the dynamic environment. This contrasts with Experiment 1, where IBFOA achieved a meaningfully better Friedman rank and variance reduction, suggesting that the dynamic environment’s stochasticity dominates any operator-level differences.

Architectural Analysis

The performance hierarchy observed in this experiment reflects fundamental architectural differences between the agent types.

SAC achieves the second-highest performance (mean +110.65) due to three key architectural advantages for this environment. First, as an off-policy algorithm, SAC reuses past experiences via a replay buffer, allowing it to learn from each transition multiple times. That is a critical advantage given the 300,000-step training budget (1000 episodes × 300 steps). Second, SAC’s entropy regularization explicitly encourages exploration of the 90-action space, preventing premature convergence to suboptimal defensive strategies. Third, SAC’s twin Q-networks and automatic temperature tuning provide stable value estimation in the presence of the environment’s inherent stochasticity. Research on discrete SAC has noted that these advantages come with known trade-offs: Q-value underestimation and performance instability can emerge in discrete settings, and the computational overhead of maintaining five neural networks (actor, two critics, two target critics) plus a replay buffer is substantial.

XCS variants form a consistent second tier, achieving positive mean rewards despite a fundamentally different approach. XCS operates through a population of interpretable IF–THEN rules, each matching a subset of states and proposing defensive actions. This rule-based architecture provides three practical advantages. First, XCS exhibits warmup-free competence: the covering mechanism generates reasonable rules from the first episode, producing positive-reward behavior without a warmup period or replay buffer. In contrast, SAC requires 128 warmup steps before training begins. Second, XCS decisions are inherently interpretable. The matching rules can be inspected to understand why a particular defensive action was selected. Third, XCS’s computational footprint is substantially lower, as discussed under Computational Cost in Section 4.2.

DQN occupies a middle ground (mean −28.34), showing that a simpler off-policy architecture with a replay buffer can learn basic defensive patterns but lacks SAC’s entropy-driven exploration needed to navigate the 90-action space effectively. DQN’s ε-greedy exploration (ε = 0.1) provides only a 10% chance of random action selection, which is insufficient for discovering non-obvious defensive strategies across 90 actions.

PPO achieves the highest mean reward in this experiment (+307.57) after the implementation correction described next. The original PPO implementation contained a strict-inequality at the PPO-Clip branch selector (“if (surr1 < surr2)”) that zeroed the policy gradient throughout the unclipped region of the importance ratio, where surr1 equals surr2 bitwise within the [1 − ε, 1 + ε] interval. The canonical PPO-Clip surrogate objective of Schulman et al. (2017) [70] requires the unclipped policy gradient to flow at the equality boundary, which we ensure via non-strict comparison (“if (surr1 ≤ surr2)”). With this implementation, PPO’s clipping and GAE advantage estimation provide stable on-policy updates in the 90-action discrete cybersecurity environment, producing per-scenario means from +87.45 (DDoS) to +563.30 (Ransomware). Policy Gradient, in contrast, performs worse than the Random baseline (mean −86.12 vs. −80.12). As an on-policy algorithm without a clipping or trust-region mechanism, Policy Gradient discards collected experience after each gradient update and applies the raw policy gradient with no bound on policy drift between updates; in an environment with 90 discrete actions and only 1000 training episodes, this combination of sample inefficiency and unbounded update magnitudes prevents the policy from reaching a viable working point within the training budget. The contrast between PPO and Policy Gradient highlights that the clipping mechanism, and not the on-policy framing alone, determines whether a policy-gradient method can navigate a large discrete action space within a limited training budget.

Q-Learning (mean −47.98) fails because tabular Q-learning with state discretization into 10 bins cannot represent the 100-dimensional continuous state space with sufficient resolution, resulting in severe state aliasing where perceptually different network conditions map to the same discretized state.

Computational Cost

Table 14. Computational cost versus performance. Time is the average wall-clock time per training run across all six scenarios. Overall Mean is the mean cumulative reward across all six scenarios and ten runs.

Agent	Avg Time/Run	Overall Mean (Cumulative Reward)	Cost Ratio vs. XCS
SAC	10 m 26 s	+110.65	26.1×
PPO	2 m 8 s	+307.57	5.3×
DQN	38 s	−28.34	1.6×
XCS variants (P# = 0.50 avg)	24 s	+6.26	1.0× (baseline)
Policy Gradient	4 s	−86.12	0.2×
Q-Learning	<1 s	−47.98	—
Random	<1 s	−80.12	—

presents the average computation time per run for each agent type, alongside the overall mean reward. Figure 6 illustrates the cost–performance trade-off.

PPO and SAC, the two top-performing agents, both come at substantial computational cost compared to XCS: PPO at an average 5.3× the XCS compute time per run (2 min 8 s vs. 24 s) and SAC at 26.1× (10 min 26 s vs. 24 s). PPO’s per-run cost reflects that the corrected agent successfully defends the network across most of the 300 steps of each episode in most scenarios, increasing the total number of environment–agent interactions per run. Agents that fail quickly terminate episodes early and therefore incur lower wall-clock cost per run despite their poor performance. SAC’s cost ratio varies by scenario, ranging from 15.8× on Ransomware (where SAC’s longer episodes involve more complex state transitions) to 62.0× on Port Scan (where XCS’s lightweight rule matching resolves quickly on low-activity states). The high SAC cost arises from its five-network architecture (actor + two critics + two target critics), replay buffer sampling, and per-step backpropagation through multiple networks.

DQN requires 1.6× the computation time of XCS (38 s vs. 24 s) while achieving substantially worse performance (mean −28.34), representing one of the least favorable cost–performance positions among the DRL baselines. Policy Gradient and Q-Learning are computationally cheaper but offer no performance advantage.

The computational cost difference has practical implications for cybersecurity deployment. In operational environments, defensive policies must be learned or updated in real time as network conditions and attack patterns evolve. An XCS agent can complete a full 1000-episode training cycle in approximately 24 s, enabling rapid policy adaptation. PPO’s 2 min training cycle and SAC’s 10 min training cycle, while producing higher-reward policies, may be prohibitive in time-critical scenarios. Furthermore, XCS’s rule-based representation allows incremental online updates (each defensive interaction refines the rule population), whereas PPO and SAC both require batched training from collected trajectories or replay buffers. These operational considerations (interpretability, warmup-free competence, and training speed) position XCS as a practical alternative to neural approaches in resource-constrained or explainability-critical cybersecurity settings.

5. Discussion

5.1. Interpretation of Results

This work introduced the first integration of BFOA with the XCS, replacing the standard GA component with an SBA guided by a weighted multi-criteria fitness function. The results from two complementary experiments reveal both the strengths and the contextual boundaries of this approach.

The benchmark experiment (Experiment 1) demonstrates that BFOA, particularly the IBFOA variant with Lévy flights and global elitism, is a viable and often superior replacement for the GA in XCS. IBFOA achieved the best Friedman ranking across 19 datasets (rank 2.395 vs. GA’s 3.368), while GA consistently ranked fourth out of five operators. All pairwise comparisons showed medium-to-large effect sizes favoring the alternatives over GA (Cohen’s d = 0.623–0.803), and IBFOA delivered a 15.2% reduction in prediction variance relative to GA. The finding that the unoptimized XCS baseline (NONE) ranked second (i.e., outperforming the GA) suggests that the GA’s crossover and mutation operators can be counterproductive in the XCS context, disrupting well-fitted classifier populations rather than refining them. This observation aligns with the broader criticism of genetic operators in mature classifier populations, where random recombination of two good classifiers does not necessarily produce a better one. BFOA’s chemotactic search, by contrast, performs gradual, directional refinement of individual classifier parameters, which is better suited to the fine-tuning phase of XCS evolution.

The strong performance of the NONE baseline in Experiment 1 further supports this interpretation: when XCS’s native learning mechanisms already produce well-fitted classifiers, applying GA’s structural operators can degrade population quality rather than improve it, whereas BFOA’s refinement-only approach avoids this risk entirely.

It is important to note that BFOA’s superior ranking is achieved despite receiving substantially fewer optimization invocations than GA. In standard XCS, the GA is triggered whenever the average time stamp of classifiers in the action set exceeds the threshold θ_GA = 25, resulting in approximately 200 GA invocations per 5000-iteration run in Experiment 1. By contrast, BFOA is invoked at a fixed frequency of every 100 iterations, yielding approximately 50 invocations per run. Thus, BFOA receives roughly 4× fewer optimization opportunities than GA but still achieves better Friedman rankings and larger effect sizes. This suggests that BFOA’s per-invocation efficiency (i.e., optimizing the entire action set population through coordinated chemotactic movement rather than producing two offspring from two parents) more than compensates for its reduced invocation frequency.

The ablation study comparing multi-objective fitness (MO = true) against accuracy-only evaluation (MO = false) revealed that multi-objective fitness is not merely beneficial but essential for BFOA to outperform the GA. Without multi-objective fitness, the Friedman p-value degraded from 0.064 to 0.185 (not significant), IBFOA dropped from first to second place behind NONE, and three of four operators exhibited increased variance relative to GA. This indicates that the fitness landscape based only on the accuracy is too flat for metaheuristic optimizers to find meaningful differentiation (i.e., the accuracy differences between classifiers are too small to guide chemotactic search effectively). By incorporating stability and variance components (Equations (2)–(4)), the multi-objective fitness function creates a richer optimization surface with stronger gradients, enabling BFOA’s population dynamics to produce measurably better classifier populations. The 0.60/0.25/0.15 weighting scheme (accuracy/stability/variance) proved effective across all 19 datasets without per-dataset tuning, suggesting that the balance is reasonably robust.

The cybersecurity simulation experiment (Experiment 2) tested the BFOA-XCS integration in a fundamentally different setting: sequential decision-making under dynamic attack conditions with a 90-action discrete space. Here, all XCS variants achieved positive mean rewards (+5.82 to +6.60) and significantly outperformed four of five DRL baselines (DQN, Q-Learning, PPO, and Policy Gradient) with large effect sizes (d = 0.78–2.87). However, within the XCS family, the pairwise differences between operators were negligible (d = 0.005–0.014, p = 0.22–0.87), in sharp contrast to Experiment 1 where IBFOA clearly dominated. This suggests that the dynamic environment’s stochasticity (i.e., varying attack intensities, probabilistic infection spread, and episode-to-episode variability) dominates any subtle classifier population improvements that BFOA can provide. The covering mechanism, which generates new rules on-the-fly when no existing classifier matches the current state, appears to be the primary driver of XCS’s performance in this setting, with the optimization operator playing a secondary role.

PPO achieved the highest overall reward in Experiment 2 (mean +307.57), with SAC second (mean +110.65) and XCS variants forming the next tier (mean +6.60 for the best XCS variant). Both PPO and SAC benefit from architectural advantages well-matched to this environment: PPO’s clipped surrogate objective stabilizes on-policy updates under the high variance of the 90-action discrete space, while SAC’s off-policy replay buffer, entropy regularization, and twin Q-networks provide sample-efficient learning from the 300,000-step training budget. However, both top neural agents come at substantial computational cost compared to XCS: PPO at 5.3× and SAC at 26.1× the XCS compute time per run (2 min 8 s and 10 min 26 s, respectively, vs. 24 s for XCS). Furthermore, neither PPO’s nor SAC’s superiority over XCS reached significance under the conservative Nemenyi multiple-comparison correction (maximum rank difference 9.333 for PPO vs. XCS variants at P# ∈ {0.50, 0.60} and 8.667 for SAC, both well below CD = 17.505), meaning that, while PPO and SAC clearly outperform XCS in pairwise comparison, the advantage is not statistically separable when the full set of 26 agent configurations is considered simultaneously. The cost–performance trade-off is a critical consideration for operational cybersecurity deployment, where XCS’s ability to complete a full training cycle in 24 s, with interpretable rule-based decisions and warmup-free competence from the first episode, may outweigh PPO’s and SAC’s higher reward in time-critical or explainability-critical scenarios.

Policy Gradient (REINFORCE) performed worse than the Random baseline (mean −86.12 vs. Random’s −80.12). This failure reflects a genuine architectural mismatch: as an on-policy algorithm without a clipping or trust-region mechanism, Policy Gradient discards collected experience after each gradient update and applies the raw policy gradient to whatever samples were available, with no bound on policy drift between updates. In an environment with 90 discrete actions and 1000 training episodes, this sample inefficiency combined with unbounded update magnitudes prevents the policy from reaching a viable working point within the training budget. PPO, by contrast, retains the on-policy framing but uses its clipped surrogate objective to bound updates: this is precisely what allows PPO to achieve the top performance in the same environment. The contrast between PPO and Policy Gradient in this experiment highlights that the clipping mechanism, not the on-policy framing alone, is what determines whether a policy-gradient method can navigate a large discrete action space within a limited training budget.

The don’t-care probability (P#) emerged as the most influential hyperparameter in the cybersecurity experiment, with P# = 0.50 consistently producing the best results across all operators. This parameter controls the balance between classifier specificity and generality: at P# = 0.33, classifiers are too specific to generalize across the diverse states encountered in different attack scenarios, while at P# = 0.70, classifiers are too general to discriminate between meaningful state differences. The optimal range of P# = 0.50–0.60, where each condition bit has a roughly equal probability of being “don’t care” or specific, produces classifiers that can match related states within a scenario while maintaining enough specificity to differentiate between distinct network conditions. This finding is consistent with the XCS literature on the generalization pressure, where overly specific or overly general populations lead to suboptimal performance.

5.2. Limitations

Several limitations should be acknowledged. First, the benchmark evaluation (Experiment 1) used 20% stratified subsets for the larger cybersecurity datasets (NSL-KDD, UNSW-NB15) due to computational constraints. Preliminary experiments on full-size datasets revealed that BFOA’s evaluation cost scales with dataset size (a single BFOA run on the CICIDS dataset (~191,000 instances) required approximately 52 h), making full-size experiments computationally infeasible under the current evaluation approach. Addressing this through evaluation sampling strategies or lazy evaluation represents a direction for future work. Second, the cybersecurity simulation environment used in Experiment 2 is a controlled abstraction of a network under active attack, not a faithful replica of an enterprise network. Several aspects of real-world network defense are deliberately not modeled, including encrypted traffic and the corresponding constraints on inspection-based detection; heterogeneous devices with widely varying configurations, operating systems, and patch states; partial observability beyond the 100-dimensional state vector exposed to the agent; and adversarial co-adaptation in which the attack policy itself evolves in response to the defender’s actions. The simulator was scoped to support the 1560-run experimental grid while preserving the dynamics most directly relevant to closed-loop defensive policy learning (lateral spread, external attacks, vulnerability emergence, defense decay, and terminal compromise thresholds). Third, the DRL baselines (PPO, SAC) were implemented from scratch in Java without established deep learning frameworks, as no production-ready pure-Java implementations exist. While implementation correctness was verified through diagnostic testing and external code review, these implementations may not achieve the same training efficiency as optimized Python/C++ implementations using graphics processing unit acceleration and autograd. Fourth, the Friedman test in Experiment 1 yielded a marginally significant p-value (0.064), which does not meet the conventional α = 0.05 threshold. While the consistent ranking pattern, supporting effect sizes, and the ablation study provide convergent evidence, a larger number of datasets or more iterations may be needed to achieve conventional significance. Fifth, the SAC implementation uses two critic networks (“twin Q-networks”) intended to mitigate Q-value overestimation through the clipped-double-Q operator, but the two critics share an identical pseudo-random seed at initialization and therefore produce numerically correlated value estimates throughout training. The clipped-double-Q operator’s variance-reduction benefit is consequently weakened in the present implementation, although the SAC architecture’s other components (off-policy replay, entropy regularization, soft target updates) remain unaffected and SAC’s overall second-place ranking is robust to this caveat.

6. Future Work

Numerous promising extensions emerge from this work. First, the action space scaling behavior of XCS versus the top-tier neural agents (PPO and SAC) warrants investigation because, as the discrete action space grows beyond 90 actions (e.g., in larger network topologies with 400 or more actions), both PPO’s clipped surrogate objective and SAC’s softmax-based exploration may degrade, while XCS’s covering mechanism scales more gracefully. Second, the multi-objective fitness function could be extended with additional components such as classifier diversity or coverage breadth, potentially producing more specialized classifier populations for complex environments. Third, the BFOA-XCS framework could be evaluated in adversarial multi-agent settings where both attacker and defender employ adaptive strategies, testing whether BFOA’s population dynamics can track non-stationary optimal policies. Fourth, the scaling limitation on large datasets could be addressed through approximate evaluation functions (e.g., evaluating classifiers on random subsets of the training data rather than the full dataset), which would reduce BFOA’s per-trigger cost from O(N × population) to O(k × population) for a fixed sample size k. Finally, an investigation of BFOA-XCS in continuous or hybrid action spaces, leveraging BFOA’s natural suitability for continuous optimization, could extend the framework’s applicability beyond discrete cybersecurity actions. Validating the framework against established cybersecurity simulation benchmarks would further strengthen the external validity of these findings. A further promising direction is the development of a hybrid BFOA–GA operator that combines the GA’s crossover-based structural exploration with BFOA’s chemotactic local refinement, as hybrid genetic–bacterial foraging approaches have shown strong results in global optimization [71]. In addition, the hyperparameter-optimization strategy used in [72] (hybrid GA/PSO) could be applied to BFOA-XCS to jointly tune XCS hyperparameters (population size, P#, GA threshold) and BFOA inner-loop parameters (chemotactic steps, reproduction count, step size), extending the method toward a more general predictive tool. Lastly, the same integration pattern could be instantiated with more recent swarm and evolutionary optimizers (for example grey wolf optimization, whale optimization algorithm, or differential evolution variants) to test whether the architectural advantages described in Section 2.2 generalize across other population-oriented methods or are specific to BFOA’s chemotactic dynamics.

7. Conclusions

This article presents the first integration of BFOA with XCS, replacing the standard GA component with an SBA guided by a weighted multi-criteria fitness function. The integration pattern proposed here is a structural change to the XCS optimization step that is independent of the specific swarm algorithm chosen. BFOA is one instantiation, and the empirical results presented in this work demonstrate that the pattern is productive in both static and dynamic learning regimes. Three BFOA variants (standard BFOA, Adaptive BFOA with self-tuning parameters, and IBFOA with Lévy flights and global elitism) were implemented and evaluated alongside a novel multi-objective fitness function that simultaneously optimizes classifier accuracy, prediction stability, and variance reduction.

Two complementary experiments validated the proposed BFOA-XCS integration. In the first experiment, across 19 benchmark datasets spanning classification, regression, and cybersecurity domains, IBFOA achieved the best Friedman ranking among all five XCS operator configurations, while the standard GA consistently ranked fourth. All operators outperformed GA with medium-to-large effect sizes (Cohen’s d = 0.623–0.803), and IBFOA delivered a 15.2% reduction in prediction variance. The ablation study confirmed that the multi-objective fitness function is essential: without it, the Friedman test lost significance (p = 0.185 vs. 0.064), and three of four operators exhibited increased variance relative to GA. These results establish that BFOA’s chemotactic search is better suited than GA’s crossover-mutation operators for refining mature XCS classifier populations, particularly when guided by a multi-objective fitness landscape.

In the second experiment, conducted in a dynamic cybersecurity simulation with six attack scenarios and a 90-action discrete space, all XCS variants significantly outperformed three of five DRL baselines (DQN, Q-Learning, Policy Gradient), with large effect sizes (d = 0.78–2.87, p < 0.001). PPO achieved the highest overall reward through its clipped surrogate objective and Generalized Advantage Estimation, with SAC second through off-policy learning, entropy-regularized exploration, and experience replay. Both top neural agents came at substantial computational cost: PPO at 5.3× and SAC at 26.1× the XCS compute time per run (2 min 8 s and 10 min 26 s, respectively, vs. 24 s for XCS). Notably, Policy Gradient performed worse than random action selection, confirming that on-policy methods without trust-region or clipping mechanisms face a fundamental architectural mismatch in large discrete action spaces with limited training budgets. XCS’s practical advantages (i.e., interpretable IF–THEN rule-based decisions, warmup-free competence from the first episode without replay buffers, and rapid training cycles enabling real-time policy adaptation) position it as a compelling alternative to neural approaches in resource-constrained or explainability-critical cybersecurity deployments.

Together, these findings demonstrate that swarm intelligence, specifically bacterial foraging, offers a viable and in several respects superior alternative to evolutionary operators in LCS. The BFOA-XCS framework generalizes across both static supervised learning tasks and dynamic RL environments, and the multi-objective fitness mechanism provides a principled approach to producing stable, low-variance classifier populations.