Enhancing Rule Construction in Grammatical Evolution with Advanced Crossover Techniques

Abstract

Many real-world problems can be formulated as classification tasks in machine learning and addressed using a wide range of advanced algorithms developed for this purpose. Among these approaches, methods based on Grammatical Evolution (GE) can also be employed, such as GenClass, a technique designed to automatically construct classification rules. Although GenClass has been successfully applied to a variety of classification problems, its evolutionary search process may suffer from limitations such as entrapment in local minima of the error function and premature convergence, which can prevent the method from discovering high-quality classification rules. The present work proposes an enhanced version of GenClass through the introduction of advanced crossover techniques operating on the integer-valued chromosomes employed by the method, with the aim of improving the evolutionary search process and reducing classification error. The proposed crossover operators follow two different strategies. Some of them are purely stochastic, introducing greater diversity into the evolutionary search, whereas others exploit information already generated during the Grammatical Evolution process to perform targeted exchanges of genetic material between chromosomes. In this way, the latter operators attempt to guide the evolutionary search toward more promising regions of the search space while preserving useful information discovered during previous generations. The proposed techniques were evaluated on a diverse collection of real-world classification datasets, including several problems characterized by a large number of patterns and/or a large number of classes. The experimental results demonstrate that the Standard crossover strategy provides the most consistent improvement over the original GenClass method, reducing the average classification error from 16.13% to 13.42% across 31 classification datasets. This improvement remains statistically significant after excluding four related EEG-derived datasets and applying a correction for multiple comparisons. Among the three investigated crossover strategies, the simplest approach, Standard, achieved the lowest average classification error, whereas the more elaborate grammar-guided Targeted and TargetedWorst variants did not provide further improvements.

1. Introduction

Genetic Algorithms (GAs) are stochastic optimization methods whose foundations can be traced back to the pioneering work of Holland [1]. They constitute an important class of evolutionary computation techniques [2], drawing inspiration from the principles of natural evolution. A typical Genetic Algorithm starts with a population of candidate solutions to the optimization problem under consideration. This population is progressively improved over successive generations through evolutionary operators such as selection, crossover, and mutation [3,4,5]. These operators enable the exploration of the search space while promoting the propagation of promising solutions throughout the population. Due to their flexibility and ability to address complex optimization problems without requiring explicit gradient information, Genetic Algorithms have been successfully applied across a broad range of domains. Representative applications include network optimization [6], robotics [7,8], energy-related problems [9,10], medical applications [11], and agricultural problems [12], among many others.

Grammatical Evolution (GE) [13] is a specialized form of Genetic Algorithms that employs integer-valued chromosomes. Each chromosome consists of a sequence of integer codons that are used to select among the production rules defined by a given Backus–Naur Form (BNF) grammar [14]. Through a genotype-to-phenotype mapping process, these codons guide the derivation of syntactically valid expressions or programs. The primary objective of Grammatical Evolution is therefore to evolve programs or expressions in a target language whose syntax is specified by the underlying grammar.

During genotype-to-phenotype mapping, chromosome codons are read sequentially. When the end of the chromosome is reached before the derivation is completed, the mapping process returns to the beginning of the chromosome (wrapping). The maximum number of wrappings is set to two. If the derivation cannot be completed within this limit, the chromosome is considered invalid and is assigned a large fitness value. Mathematical expressions are evaluated using the FunctionParser library, which detects invalid mathematical operations, such as division by zero or logarithms with non-positive arguments. In such cases, an evaluation error is reported, and the corresponding chromosome is penalized with a large fitness value rather than using protected mathematical operators.

Grammatical Evolution has been successfully applied to a wide range of real-world problems across diverse scientific and engineering domains. Representative applications include data fitting [15,16], credit classification [17], network intrusion detection [18], the solution of differential equations [19], drinking water quality monitoring [20], and the automatic construction of optimization methods [21]. GE has also been employed in trigonometric problems [22], automatic music composition [23], neural network design [24,25], and computer games [26,27]. Additional applications can be found in energy demand modeling [28], combinatorial optimization [29], security and cryptography [30], automatic decision tree construction [31], analog circuit design [32], taxonomy discovery from Wikipedia [33], algorithmic trading [34], bioinformatics [35], and human glycemia modeling [36]. The diversity of these applications demonstrates the flexibility of Grammatical Evolution as a general-purpose evolutionary framework for addressing complex problems in different domains.

Despite the extensive application of Grammatical Evolution across different domains, the design of effective crossover operators remains an important research challenge. In particular, the indirect genotype-to-phenotype mapping employed by GE introduces additional considerations regarding the preservation and exchange of useful genetic material. Previous studies have investigated conventional, grammar-aware, and structure-preserving crossover mechanisms, as well as crossover-based local improvement within hybrid evolutionary algorithms. These approaches and their relationship to the methods proposed in the present study are discussed in Section 2.

Among the techniques that have been proposed based on Grammatical Evolution is a recently introduced rule construction method [37], for which a software implementation [38] is also publicly available through the application’s website https://github.com/itsoulos/GenClass/(accessed on 2 October 2026). This method automatically constructs programs for data classification using a representation that closely resembles the C programming language. Consequently, the generated classification programs are human-readable and can be easily inspected and interpreted, providing a considerable advantage in applications where model interpretability is important. The rule construction method has been successfully applied to a variety of real-world classification problems, including pollution-related applications [39] and network-related problems [40], among others.

The aforementioned method employs Grammatical Evolution to evolve classification programs that minimize the classification error on a given dataset. More specifically, the classification error can be defined as

$$E_C\left(C\right)=100\times {\displaystyle \frac{1}{M}}{\displaystyle \sum _{i=1}^M}\mathbb{I}\left(C\left({\mathbf{x}}_i\right)\ne y_i\right)$$

(1)

where the corresponding training set is defined as $T=\left\{({\mathbf{x}}_1,y_1),({\mathbf{x}}_2,y_2),\dots ,({\mathbf{x}}_M,y_M)\right\}.$ Here, M denotes the number of patterns in the training set, ${\mathbf{x}}_i$ represents the input feature vector of the (i)-th pattern, and $y_i$ denotes its corresponding target class label. The function $C\left({\mathbf{x}}_i\right)$ represents the class predicted by the classification program generated through Grammatical Evolution, while ($\mathbb{I}(·)$) is the indicator function, which takes the value 1 when its argument is true and 0 otherwise. Therefore, $E_C\left(C\right)$ expresses the percentage of training patterns that are incorrectly classified by the generated program.

Although the aforementioned technique has been evaluated on a variety of classification problems, several aspects of its evolutionary search process may still benefit from further investigation and improvement. One of the most important issues is premature convergence, where the population rapidly loses genetic diversity and converges before sufficiently exploring the search space. As a consequence, potentially useful genetic material may disappear from the population, reducing the ability of the evolutionary process to discover alternative and potentially more effective classification rules.

Another important limitation is the tendency of the evolutionary search to become trapped in local minima of the classification error function, or, more generally, to stagnate in suboptimal regions of the search space. In such cases, the evolutionary operators may fail to generate sufficiently diverse or beneficial modifications to the existing chromosomes, preventing the algorithm from escaping these regions and discovering solutions with lower classification error. These limitations become particularly important in complex classification problems characterized by large or highly irregular search spaces, where effective exploration and preservation of population diversity are essential for obtaining high-quality classification rules. Consequently, improving the search capabilities of the method and enhancing the exchange of useful genetic information among candidate solutions constitute important directions for further improving its classification performance.

In the present work, advanced crossover techniques are introduced in order to alleviate the aforementioned limitations and improve the search capabilities of the original method. These techniques are applied at each generation to a relatively small number of chromosomes selected randomly from the genetic population. Restricting their application to a subset of the population allows the proposed operators to enhance the exploration of the search space while limiting their application to a selected number of chromosomes. However, the additional crossover operations introduce computational overhead, which depends on the selected operator and the dataset, as demonstrated in the experimental analysis.

The randomly selected chromosomes are subjected to additional crossover operations designed to generate new combinations of genetic material and, consequently, potentially more effective classification programs. The introduction of such combinations can increase population diversity and provide the evolutionary process with additional opportunities to escape from suboptimal regions of the search space. This mechanism is particularly important when the population has begun to converge and the conventional evolutionary operators are no longer able to produce sufficiently diverse candidate solutions.

Furthermore, the random selection of chromosomes ensures that the additional crossover operations are not systematically restricted to a particular subset of the population. In this way, different candidate solutions can participate in the enhanced crossover process during successive generations, while the majority of the population continues to evolve according to the standard evolutionary procedure. The proposed strategy therefore attempts to achieve a balance between exploration and exploitation: it introduces additional diversity and explores new combinations of genetic material, while preserving the evolutionary information already accumulated by the population.

In the present work, three crossover strategies are considered: Standard, Targeted and TargetedWorst crossover. Standard crossover follows a purely stochastic strategy, in which the exchanged chromosome regions are selected without exploiting information obtained from the evaluation of the generated classification programs. Targeted crossover, on the other hand, makes use of information collected during the evolutionary process to identify chromosome regions where crossover may be more beneficial. TargetedWorst extends this idea by focusing the crossover operation on regions associated with poorly performing components of the corresponding classification program. Thus, the three strategies represent progressively more informed crossover mechanisms, ranging from purely random recombination to crossover operations guided by the performance of the evolved solutions. A detailed description of each strategy is provided in the following subsections.

The remainder of this article is organized as follows. Section 2 reviews previous research on crossover operators in Grammatical Evolution and discusses the relationship between the proposed techniques and hybrid evolutionary and memetic algorithms. Section 3 introduces the original GenClass rule construction method and describes the proposed crossover strategies in detail. Section 4 presents the classification datasets, experimental results, and corresponding statistical analysis. Finally, Section 5 summarizes the main findings and conclusions of the study.

2. Related Work

2.1. Crossover Operators in Grammatical Evolution

Crossover is one of the fundamental operators of evolutionary computation, enabling the exchange of genetic material between candidate solutions. In Grammatical Evolution (GE), however, crossover presents additional challenges because chromosomes are indirectly mapped to executable expressions or programs through a formal grammar. Consequently, exchanging chromosome segments may affect not only the genetic representation but also the grammatical derivation and the resulting program structure.

O’Neill et al. [41] investigated conventional and homologous crossover operators in GE, examining the effects of recombination on the genotype-to-phenotype mapping process. Their study highlighted the importance of crossover-induced disruption and demonstrated that homologous crossover does not necessarily outperform conventional one-point crossover. These findings indicate that increasing the structural sophistication of recombination does not automatically lead to improved evolutionary performance.

Harper and Blair [42] introduced grammar-informed crossover mechanisms designed to exploit information obtained during grammatical derivation. Their Matching Crossover operator identifies crossover positions associated with corresponding grammatical symbols, whereas Left Hand Side (LHS) Replacement Crossover exchanges complete expansions associated with matching non-terminal symbols. These approaches represent important developments toward grammar-aware recombination, particularly by distinguishing between matching grammatical positions and preserving complete grammatical structures.

Another relevant research direction involves the use of crossover as a local improvement mechanism rather than exclusively as a population-level reproductive operator. Tsoulos et al. [43] introduced Local Crossover for GE, employing repeated recombination and fitness-based acceptance to improve individual chromosomes. This approach demonstrates how crossover can be incorporated into an additional refinement procedure, allowing beneficial genetic modifications to accumulate during the evolutionary process.

The present study builds upon these research directions by investigating three additional crossover strategies within the GenClass rule construction framework. The Standard strategy employs repeated one-point crossover with strict fitness-based acceptance. The Targeted strategy exploits genotype-to-phenotype mapping information to identify active codons associated with compatible grammatical categories, while selecting donor chromosomes from an elite subset of the population. TargetedWorst extends this approach by incorporating class-specific classification performance to prioritize modifications associated with poorly performing classes.

Unlike crossover operators that exchange complete grammatical expansions, the proposed Targeted strategies transfer bounded codon blocks between grammatically compatible positions. Therefore, they do not guarantee the preservation of complete grammatical substructures or equivalent program behavior after remapping. Their objective is instead to investigate whether grammatical compatibility, restricted genetic exchange, and classification-performance information can improve the effectiveness of crossover- based refinement.

2.2. Hybrid Evolutionary and Memetic Algorithms

Hybrid evolutionary algorithms combine population-based evolutionary search with additional optimization mechanisms to improve search effectiveness. A particularly important category consists of memetic algorithms, in which evolutionary operators are complemented by local improvement procedures. Such approaches attempt to combine the global exploration capabilities of evolutionary algorithms with the exploitation capabilities of local search.

Local improvement is commonly associated with mutation-based neighborhood exploration or numerical optimization procedures. However, crossover can also serve as a mechanism for local refinement by generating candidate solutions through recombination with other members of the population. Lozano et al. [44] investigated crossover hill-climbing within memetic algorithms, demonstrating the relevance of recombination-based improvement procedures in hybrid evolutionary optimization.

Within GE, Local Crossover [43] provides a related example in which repeated recombination is used to refine chromosomes through fitness-based acceptance. This establishes a direct connection between grammatical evolution and memetic search strategies, since crossover is employed not only to generate offspring for subsequent generations but also to improve existing candidate solutions.

The methods investigated in the present work can be interpreted within this broader memetic framework. The original GenClass evolutionary process retains its conventional selection, crossover, and mutation operators, while an additional crossover-based refinement stage is applied to a limited number of chromosomes during each generation. Candidate modifications are accepted only when they improve the corresponding training fitness, and accepted improvements are immediately available for subsequent refinement trials.

An important distinction from approaches in which local improvement dominates the evolutionary search is that the proposed framework limits the number of chromosomes subjected to additional refinement. This design aims to supplement the population-based evolutionary process without replacing its conventional search mechanisms. Nevertheless, the additional operations introduce computational overhead, making the relationship between classification performance and computational effort an important aspect of the experimental evaluation.

From this perspective, the principal contribution of the present study is not the introduction of grammar-aware crossover or crossover-based local search as entirely new concepts. Rather, it is the systematic investigation of additional crossover refinement strategies within a grammatical-evolution-based rule construction method. In particular, the study examines whether grammatical-category matching, elite donor selection, bounded codon exchange, and class-specific performance guidance provide measurable advantages over simpler stochastic recombination.

The experimental comparison of Standard, Targeted, and TargetedWorst also provides evidence concerning the trade-off between operator complexity and practical effectiveness. This is particularly relevant to the design of hybrid and memetic evolutionary algorithms, where more informed local search mechanisms may introduce additional computational cost without necessarily producing superior solutions.

3. Materials and Methods

This section first provides a detailed description of the rule construction method and subsequently introduces the proposed crossover techniques designed to enhance its evolutionary search process.

3.1. The Rule Construction Method

The rule construction method, hereafter referred to as GenClass, generates classification rules expressed in a C-like language according to the BNF grammar presented in Algorithm 1. The grammar shown in the algorithm is an extended representation of the original BNF grammar, in which each alternative production rule associated with a non-terminal symbol is assigned a corresponding production index. These indices are subsequently used during the genotype-to-phenotype mapping process to determine the production rules selected by the integer-valued chromosomes of GenClass.

Although Algorithm 1 illustrates the grammar used to construct a binary classification rule, GenClass supports multi-class classification through an ordered sequence of rules. For a classification problem involving C classes, the chromosome is divided into C − 1 consecutive segments of equal length, each encoding a separate Boolean condition generated using the BNF grammar. These conditions are combined into an if–else-if structure, where each condition corresponds to one of the first C − 1 classes. The final class serves as the default class when none of the preceding conditions is satisfied. Since the conditions are evaluated sequentially, the first satisfied condition determines the predicted class, thereby avoiding conflicts between multiple matching rules.

The genotype-to-phenotype mapping process employed by GenClass is illustrated in Figure 1. Each chromosome consists of a sequence of integer-valued codons, which are sequentially used to select production rules from the BNF grammar. Starting from the initial non-terminal symbol $⟨S⟩$, the mapping process identifies the next non-terminal to be expanded and determines the corresponding production rule according to

$$r=c_{i}modN,$$

where $c_i$ denotes the current integer codon, N is the number of alternative production rules available for the corresponding non-terminal symbol, and r is the index of the selected production. The selected production is then applied and the mapping process proceeds to the next non-terminal symbol using the next chromosome codon. This procedure is repeated until no non-terminal symbols remain and a complete syntactically valid classification program is generated.

Algorithm 1 BNF grammar employed by the GenClass method.

<S> ::= if (<BEXPR>) CLASS=0 else CLASS=1                (0) <BEXPR> ::= <VARIABLE><BOOLOP><EXPR>                     (0)       | !(<BEXPR>)                            (1)       | <VARIABLE><BOOLOP><EXPR>&<BEXPR>                (2)       | <VARIABLE><BOOLOP><EXPR>|<BEXPR>                (3) <BOOLOP> ::= >                                 (0)       | >=                                 (1)       | <                                 (2)       | <=                                 (3) <EXPR> ::= (<EXPR><BINARYOP><EXPR>)                      (0)      | <FUNCTION>(<EXPR>)                          (1)      | <TERMINAL>                              (2) <BINARYOP> ::= +                               (0)         | -                               (1)         | *                               (2)         | /                               (3) <FUNCTION> ::= sin | cos | exp | log                (0-3) <TERMINAL> ::= <VARIABLE>                           (0)         | <DIGITLIST>.<DIGITLIST>                   (1)         | (-<DIGITLIST>.<DIGITLIST>)                  (2) <VARIABLE> ::= x1 | x2 | ... | xD                 (0-D-1) <DIGITLIST> ::= <DIGIT>                            (0)         | <DIGIT><DIGIT>                         (1)         | <DIGIT><DIGIT><DIGIT>                    (2) <DIGIT> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9   (0-9)

Figure 1. Illustration of the genotype-to-phenotype mapping process employed by GenClass. Integer-valued codons are sequentially mapped to BNF production rules using the modulo operation until a complete classification program is generated.

As an illustrative example, Figure 1 demonstrates the successive mapping of an integer chromosome to the production rules of Algorithm 1. For instance, when the non-terminal $⟨BEXPR⟩$ is expanded, four alternative productions are available. Therefore, for a codon value of 7, production $7mod4=3$ is selected. Similarly, the remaining codons are successively consumed to expand the resulting non-terminal symbols until the final classification rule is obtained. In this way, the integer-valued genotype is transformed into an executable and human-readable C-like classification program.

The main steps of the algorithm that produces classification programs are listed subsequently.

1.

Initialization step.

(a)

Initialize the Genetic Algorithm parameters: $N_c$, the population size (number of chromosomes); $N_g$, the maximum number of generations; $p_s$, the selection rate; and $p_m$, the mutation rate.

(b)

Generate the initial population of $N_c$ chromosomes, $g_i,i=1,\dots ,N_c$, where each chromosome consists of a sequence of randomly generated integer- valued codons.

(c)

Set $t=0$, the generation counter.

2.

Fitness calculation step.

(a)

For $i=1,\dots ,N_c$ do

i.

Map each chromosome $g_i$ to its corresponding classification program $C_i$ through the genotype-to-phenotype mapping process defined by the BNF grammar presented in Algorithm 1.

ii.

Evaluate the generated classification program $C_i$ on the training set and compute its classification error. The resulting error value, denoted by $f_i$, is used as the fitness value of the corresponding chromosome $g_i$.

(b)

End For

3.

Application of genetic operations.

(a)

Selection: Rank the chromosomes according to their fitness values in ascending order. The $(1-p_s)N_c$ chromosomes with the lowest fitness values are directly transferred to the next generation without modification, while the remaining $p_s{N}_c$ chromosomes are replaced by new offspring generated through the crossover and mutation operators.

(b)

Crossover: Generate pairs of offspring by applying the one-point crossover operator. For each pair of offspring, denoted as $O_1$ and $O_2$, two parent chromosomes, $P_1$ and $P_2$, are selected from the current population using tournament selection. A crossover point is then randomly selected along the chromosomes, and the genetic material located after this point is exchanged between the two parents. Consequently, $O_1$ inherits the first part of $P_1$ and the second part of $P_2$, whereas $O_2$ inherits the first part of $P_2$ and the second part of $P_1$. A graphical illustration of the one-point crossover operation is provided in Figure 2.

Figure 2. An example of the one-point crossover method.

(c)

Mutation: For each gene $g_{i,j}$ of chromosome $g_i$, a random number $r\in [0,1]$ is generated. If $r\le p_m$, where $p_m$ denotes the mutation rate, the corresponding gene is replaced by a randomly generated integer value within the predefined range of admissible codon values. Otherwise, the gene remains unchanged.

4.

Termination check step.

(a)

Set $t=t+1$

(b)

If $t\le N_g$then go to Fitness Calculation step else terminate.

3.2. The Proposed Crossover Methods

The additional crossover procedures investigated in this work can be interpreted as crossover-based local improvement mechanisms embedded within a conventional evolutionary algorithm. Their repeated application, together with the immediate acceptance of fitness-improving modifications, follows a greedy hill-climbing strategy in which candidate neighbors are generated through recombination with other population members. Consequently, the overall framework is closely related to hybrid evolutionary and memetic algorithms, as discussed in Section 2.2.

In the present work, additional crossover techniques are introduced with the aim of improving the performance of GenClass and enhancing its ability to escape from local minima of the classification error function. These crossover operations are applied at each generation of the evolutionary process to a subset of the current population. More specifically, a number of chromosomes are randomly selected from the population and genetic material from each selected chromosome is exchanged with that of another randomly selected chromosome. In this way, new combinations of genetic material are introduced into the population, potentially increasing population diversity and allowing the evolutionary search to explore previously unvisited regions of the search space. The number of chromosomes subjected to these additional crossover operations at each generation is denoted by $N_K$ throughout the remainder of this work.

The proposed procedure is incorporated into the evolutionary algorithm described in Section 3.1 and is executed at each generation after the standard genetic operators of selection, crossover, and mutation have been applied. In this way, the additional crossover mechanism operates as an enhancement of the original evolutionary process without modifying its basic structure.

The proposed crossover techniques, denoted throughout this work as Standard, Targeted, and TargetedWorst, differ in the strategy employed to select and exchange genetic material between chromosomes. A detailed description of each technique is provided in the following subsections.

The Standard crossover strategy is closely related to the Local Crossover operator previously introduced by Tsoulos et al. [43]. Both methods employ repeated one-point recombination with fitness-based acceptance and sequential improvement of selected chromosomes. The differences concern the selection of chromosomes for refinement, the acceptance criterion, and the computational effort allocated to the refinement stage. Therefore, Standard is considered here as an adapted crossover-based refinement strategy rather than a fundamentally new local-search operator. It also serves as a reference for evaluating the additional grammatical and class-performance guidance introduced by Targeted and TargetedWorst.

The configurations also differ substantially in the allocation of search effort. The additional procedure performs $N_{K}k=200$ candidate evaluations per generation, against $p_s{N}_c=450$ offspring produced by the conventional operators. In the earlier study the corresponding ratio is reversed by two orders of magnitude, since the refinement stage dominates the evolutionary update. The additional crossover examined here therefore operates as a supplementary refinement within an otherwise conventional evolutionary search, rather than as its principal search mechanism.

Targeted and TargetedWorst approaches introduce grammatical-category matching, elite donor selection, bounded codon exchange and, for TargetedWorst, class-performance-guided selection of the modified region. The use of grammatical information to guide recombination is itself not new. Harper and Blair [42] introduced Matching Crossover, which aligns corresponding grammatical positions, and LHS Replacement Crossover, which exchanges complete expansions of matching non-terminals. However, unlike complete-expansion replacement, the Targeted procedures transfer bounded codon blocks beginning at grammatically matched positions and therefore do not guarantee the preservation of complete grammatical structures or of equivalent program behaviour after remapping.

3.2.1. The Standard Crossover Method

In the Standard crossover technique, for each of the $N_K$ selected chromosomes, denoted by $g_c$, a subset of k chromosomes is randomly selected from the current population,

$$S=\{g_{r_1},g_{r_2},\dots ,g_{r_k}\}.$$

The chromosome $g_c$ is then successively combined with each chromosome $g_{r_i},i=1,\dots ,k$, using one-point crossover. For each pair $(g_c,g_{r_i})$, a crossover point is randomly selected and the corresponding genetic material is exchanged to produce a candidate offspring chromosome $g_{\mathrm{new}}$. The fitness of $g_{\mathrm{new}}$ is subsequently evaluated using its classification error on the training set. If $g_{\mathrm{new}}$ achieves a lower fitness value than the current $g_c$, it replaces $g_c$; otherwise, the current chromosome remains unchanged. Importantly, whenever such an improvement occurs, the updated chromosome becomes the new $g_c$ and is used in the subsequent crossover operation with $g_{r_{i+1}}$. Therefore, the procedure performs a sequential improvement of $g_c$, allowing beneficial genetic material obtained from successive crossover operations to be accumulated. After all k chromosomes in S have been examined, the final improved version of $g_c$ is retained in the population.

The complete procedure of the Standard crossover technique is summarized in Algorithm 2. The method follows a greedy acceptance strategy, since a newly generated chromosome replaces the current chromosome only when it achieves a lower classification error. Furthermore, each accepted chromosome is immediately used in the subsequent crossover operation, allowing improvements obtained during successive crossover trials to be accumulated.

The proposed crossover operator generates up to two candidate offspring using one-point crossover. The first offspring is constructed by combining the prefix of the selected chromosome with the suffix of a randomly selected chromosome. If this offspring improves the fitness of the selected chromosome, it is accepted and replaces the original chromosome. Otherwise, a second offspring is constructed by exchanging the corresponding chromosome segments in the opposite direction. The second offspring is accepted only if it improves the fitness of the selected chromosome. If neither offspring yields an improvement, the original chromosome remains unchanged. Thus, at most one offspring is accepted during each crossover iteration.

Algorithm 2 Standard crossover technique.

Require:  Population $G=\{g_1,g_2,\dots ,g_{N_c}\}$, number of selected chromosomes $N_K$, number of crossover trials k Ensure:  Updated population G  1: Randomly select $N_K$ chromosomes from G  2: for each selected chromosome $g_c$ do  3:     Randomly select a subset $S=\{g_{r_1},g_{r_2},\dots ,g_{r_k}\}$ from G  4:     for $i=1$ to k do  5:         Randomly select a one-point crossover position  6:         Generate $g_{\mathrm{new}}$ by applying one-point crossover between $g_c$ and $g_{r_i}$  7:         Evaluate the fitness $f\left(g_{\mathrm{new}}\right)$  8:         if $f\left(g_{\mathrm{new}}\right)<f\left(g_c\right)$ then  9:            $g_c\leftarrow g_{\mathrm{new}}$ 10:         end if 11:     end for 12: end for 13: return G

3.2.2. The Targeted Crossover Method

In the Targeted crossover technique, the exchange of genetic material is guided by information collected during the genotype-to-phenotype mapping process. More specifically, the mapping procedure records the positions of the active codons of each chromosome together with their semantic role in the generated classification program. These codons may correspond to different grammatical components, such as functions, binary operators, Boolean operators, variables, and constants.

For each selected chromosome $g_c$, the Targeted technique performs k crossover trials. At each trial, an active codon position is randomly selected as the target of the crossover operation. A donor chromosome $g_r$ is then randomly selected from the best fraction $\rho$ of the current population, and the recorded mapping information is used to identify an active codon in $g_r$ having the same semantic type as the selected codon of $g_c$. Instead of performing an unrestricted one-point crossover, the Targeted technique transfers a bounded block of genetic material between the corresponding positions. The size of this block is determined from the distances to the next active codons in the two chromosomes and is additionally restricted by a predefined maximum block size $B_{\max }$. In this way, the transferred genetic material is associated with similar grammatical components of the classification programs represented by the two chromosomes.

An active codon is defined as a chromosome position actually accessed during the genotype-to-phenotype mapping process. The proposed Targeted crossover operator records these positions together with their associated semantic categories, including functions, operators, variables, and constants. A target codon is selected from this trace, and a compatible donor codon of the same semantic category is selected from an elite chromosome. The exchanged block length is determined by the distance from each selected position to the next active codon with a greater chromosome index, subject to the maximum block size and chromosome boundaries. Consequently, when active codons occupy consecutive chromosome positions, the exchanged block consists of a single codon. The operator therefore performs semantically guided codon-level or short-block exchanges rather than guaranteeing the exchange of complete grammatical subtrees.

The resulting chromosome $g_{\mathrm{new}}$ is subsequently mapped to its corresponding classification program and its fitness is evaluated on the training set. If

$$f\left(g_{\mathrm{new}}\right)<f\left(g_c\right),$$

the new chromosome replaces $g_c$; otherwise, the current chromosome remains unchanged. As in the Standard technique, an accepted improvement is immediately retained and the updated chromosome is used in subsequent crossover trials. The main difference from Standard crossover is therefore that the exchange of genetic material is not determined exclusively by randomly selected crossover points. Instead, information obtained during the mapping process is exploited to perform crossover between semantically related regions of the participating chromosomes. This targeted exchange is intended to increase the probability of producing meaningful modifications to the generated classification programs while preserving useful genetic structures already discovered during the evolutionary process. The complete procedure of the Targeted crossover technique is summarized in Algorithm 3.

3.2.3. The TargetedWorst Crossover Method

The TargetedWorst crossover technique extends the Targeted approach by additionally exploiting information about the classification performance of the generated program. While Targeted crossover may select an active semantic codon from the chromosome without considering the performance of individual classes, TargetedWorst attempts to concentrate the crossover operation on the chromosome region associated with the class exhibiting the poorest classification performance. The motivation behind this strategy is to modify preferentially those parts of the generated classification program that are more likely to contribute to the observed classification errors, while leaving better-performing components less affected.

The worst-performing class is identified using the class-specific classification error computed on the training set. More specifically, for each class c, the classification error is defined as the percentage of incorrectly classified training patterns belonging to that class. The class with the highest error rate is selected as the worst-performing class. This criterion is equivalent to minimizing class-specific recall and avoids directly favoring majority classes based on their absolute number of misclassified patterns. Nevertheless, the estimated error rates of classes containing relatively few training patterns may exhibit higher variability, which represents a potential limitation of this selection strategy.

For each selected chromosome $g_c$, the corresponding classification program is evaluated and the class exhibiting the worst classification performance is identified. The genotype-to-phenotype mapping information is subsequently used to determine the active codons associated with the corresponding part of the classification program. An active codon is then selected from this region and its semantic type is determined. As in the Targeted technique, a donor chromosome $g_r$ is selected from the elite fraction of the population, and an active codon having the same semantic type is identified in the donor chromosome. This establishes semantically compatible positions between the two chromosomes and allows genetic material from a high-quality solution to be transferred specifically to a poorly performing region of $g_c$.

Algorithm 3 Targeted crossover technique.

Require:  Population $G=\{g_1,g_2,\dots ,g_{N_c}\}$, number of selected chromosomes $N_K$, number of crossover trials k, elite fraction $\rho$, maximum block size $B_{\max }$ Ensure:  Updated population G  1: Randomly select $N_K$ chromosomes from G  2: $N_e\leftarrow \max (2,\lfloor \rho N_c\rfloor )$  3: for each selected chromosome $g_c$ do  4:     for $i=1$ to k do  5:         Obtain the codon trace of the current chromosome $g_c$  6:         Construct the set $A_c$ of active semantic codons corresponding to functions, binary operators, Boolean operators, variables, and constants  7:         if $A_c$ is empty then  8:            break  9:         end if 10:         Randomly select a target codon $a_c\in A_c$ 11:         Let $t_c$ be the semantic type of $a_c$ 12:         Randomly select a donor chromosome $g_r$ from the best $N_e$ chromosomes, with $g_r\ne g_c$ 13:         Obtain the codon trace of $g_r$ 14:         Construct the set $A_r$ of active codons in $g_r$ having semantic type $t_c$ 15:         if $A_r$ is empty then 16:            continue 17:         end if 18:         Randomly select a compatible donor codon $a_r\in A_r$ 19:         Determine $L_c$, the distance from $a_c$ to the next active codon in $g_c$ 20:         Determine $L_r$, the distance from $a_r$ to the next active codon in $g_r$ 21:         $L\leftarrow \min (L_c,L_r,B_{\max })$ 22:         Create $g_{\mathrm{new}}\leftarrow g_c$ 23:         Copy L codons from $g_r$, starting at $a_r$, to the corresponding block of $g_{\mathrm{new}}$ starting at $a_c$ 24:         Evaluate the fitness $f\left(g_{\mathrm{new}}\right)$ 25:         if $f\left(g_{\mathrm{new}}\right)<f\left(g_c\right)$ then 26:            $g_c\leftarrow g_{\mathrm{new}}$ 27:         end if 28:     end for 29: end for 30: return G

A bounded block of genetic material is subsequently transferred from the donor chromosome to $g_c$. As in Targeted crossover, the size of the transferred block is determined using the active-codon structure of the participating chromosomes and is restricted by the predefined maximum block size $B_{\max }$. The resulting chromosome $g_{\mathrm{new}}$ is then mapped to its corresponding classification program and evaluated on the training set. If

$$f\left(g_{\mathrm{new}}\right)<f\left(g_c\right),$$

the candidate chromosome replaces $g_c$; otherwise, the modification is discarded. Accepted improvements are immediately retained and are therefore available to subsequent crossover trials.

Consequently, TargetedWorst introduces an additional level of guidance compared with Targeted crossover. Targeted determines the exchanged genetic material according to semantic information obtained during genotype-to-phenotype mapping, whereas TargetedWorst combines this information with class-specific classification performance in order to focus the search on chromosome regions associated with poorly performing parts of the generated classifier. The complete procedure is summarized in Algorithm 4.

Algorithm 4 TargetedWorst crossover technique.

Require:  Population $G=\{g_1,g_2,\dots ,g_{N_c}\}$, number of selected chromosomes $N_K$, number of crossover trials k, elite fraction $\rho$, maximum block size $B_{\max }$ Ensure:  Updated population G  1: Randomly select $N_K$ chromosomes from G  2: $N_e\leftarrow \max (2,\lfloor \rho N_c\rfloor )$  3: for each selected chromosome $g_c$ do  4:     for $i=1$ to k do  5:         Evaluate the class-specific performance of $g_c$  6:         Identify the worst-performing class w  7:         Let C be the number of classes  8:         $N_R\leftarrow C-1$  9:         if $w<N_R$ then 10:            Determine the chromosome segment $R_w$ associated with class w 11:         else 12:            Set $R_w$ to the entire chromosome 13:         end if 14:         Obtain the codon trace of the current chromosome $g_c$ 15:         Construct the set $A_c$ of active semantic codons of $g_c$ that belong to $R_w$ 16:         if $A_c$ is empty then 17:            continue 18:         end if 19:         Randomly select a target codon $a_c\in A_c$ 20:         Let $t_c$ be the semantic type of $a_c$ 21:         Randomly select a donor chromosome $g_r$ from the best $N_e$ chromosomes, with $g_r\ne g_c$ 22:         Obtain the codon trace of $g_r$ 23:         if $w<N_R$ then 24:            Determine the donor segment $R_w^{\left(r\right)}$ associated with class w 25:         else 26:            Set $R_w^{\left(r\right)}$ to the entire donor chromosome 27:         end if 28:         Construct the set $A_r$ of active codons in $g_r$ that belong to $R_w^{\left(r\right)}$ and have semantic type $t_c$ 29:         if $A_r$ is empty then 30:            continue 31:         end if 32:         Randomly select a compatible donor codon $a_r\in A_r$ 33:         Determine $L_c$, the distance from $a_c$ to the next active codon in $g_c$ 34:         Determine $L_r$, the distance from $a_r$ to the next active codon in $g_r$ 35:         $L\leftarrow \min (L_c,L_r,B_{\max })$ 36:         Restrict L to remain within the corresponding class segments and chromosome boundaries 37:         Create $g_{\mathrm{new}}\leftarrow g_c$ 38:         Copy L codons from $g_r$, starting at $a_r$, to $g_{\mathrm{new}}$, starting at $a_c$ 39:         Evaluate the fitness $f\left(g_{\mathrm{new}}\right)$ 40:         if $f\left(g_{\mathrm{new}}\right)<f\left(g_c\right)$ then 41:            $g_c\leftarrow g_{\mathrm{new}}$ 42:         end if 43:     end for 44: end for 45: return G

Figure 3 presents the general flowchart of the proposed method. The process starts with the generation of the initial population of integer-valued chromosomes. Each chromosome is mapped to a classification program through the BNF grammar, and the program is evaluated on the training set, which gives the fitness of the chromosome. The standard genetic operators of selection, one-point crossover and mutation are then applied to the population. After them, the additional crossover procedure is executed. A small number of chromosomes is randomly selected and, for each of them, a number of crossover trials is performed. In every trial, the technique in use, Standard, Targeted or TargetedWorst, produces a new candidate chromosome, which is mapped to a classification program through the grammar and evaluated. If the candidate achieves a lower error than the current chromosome, it replaces it, otherwise the current chromosome is kept. The process continues with the next trial until all the trials of all the selected chromosomes are completed. Finally, the termination criterion is checked. If it is not satisfied, the algorithm returns to the mapping and evaluation of the new population, otherwise it terminates and returns the best classification program.

Figure 3. Flowchart of the proposed GenClass method with the additional crossover techniques.

4. Results

4.1. Experimental Datasets

The datasets employed in the experimental evaluation were obtained from two widely used public repositories: the UCI Machine Learning Repository [45] and the KEEL Dataset Repository [46]. Both repositories provide a broad collection of benchmark datasets commonly used for the experimental evaluation and comparison of machine learning and classification algorithms. The datasets were accessed on 3 October 2026. The used datasets are the following:

• Alcohol. This dataset contains data collected from experiments investigating patterns of alcohol consumption and is used as a classification benchmark in the present study [47].
• Appendicitis. The Appendicitis dataset is a binary classification dataset containing 106 instances and 7 numerical features, used to predict the presence or absence of acute appendicitis based on clinical measurements [48].
• The Beed dataset. The Bangalore EEG Epilepsy Dataset (BEED), presented by Banu et al. [49], is an EEG dataset developed for epileptic seizure detection and classification. The dataset contains 8000 patterns, each represented by 16 features.
• Dermatology. The Dermatology dataset [50] contains clinical and histopathological measurements used for the differential diagnosis of dermatological diseases.
• Gene. The Gene dataset, also known as the Splice-junction Gene Sequences dataset, contains 3175 DNA sequences represented by 120 binary features. The dataset is used for the classification of splice-junction boundaries into three classes: exon–intron (EI) boundaries, intron–exon (IE) boundaries, and sequences containing neither type of boundary [51].
• Hayes-Roth. The Hayes-Roth dataset is an artificial classification dataset comprising three classes and was originally designed to evaluate the behavior of prototype-based classifiers. The hobby attribute was generated randomly and therefore acts as a noise-inducing feature in the classification problem [52].
• Hepatitis. The Hepatitis dataset contains clinical information collected from patients affected by hepatitis and includes a combination of integer- and real-valued attributes describing various patient characteristics and medical measurements [53].
• HouseVotes. The House Votes dataset contains voting records of members of the U.S. House of Representatives on a set of congressional issues. The classification task aims to distinguish representatives according to their political party affiliation based on their recorded votes [54].
• Ionosphere. The Ionosphere dataset contains radar measurements collected from signals transmitted through the ionosphere. The classification task aims to distinguish between radar returns that provide evidence of structure in the ionosphere and those that do not [55,56].
• Lymphography [57], which is a dataset with four distinct classes.
• Nursery. The Nursery dataset was derived from a hierarchical decision model originally developed to rank applications for admission to nursery schools. The model was used in Ljubljana, Slovenia, during the 1980s, when high demand for nursery schools required an objective procedure for evaluating and explaining admission decisions. The corresponding classification task involves assigning nursery school applications to different priority categories based on a set of socioeconomic and family-related attributes [58].
• Optdigits. The Optical Recognition of Handwritten Digits (Optdigits) dataset contains digitized representations of handwritten numerals obtained from optical scans. The corresponding classification task aims to recognize and assign each handwritten sample to one of the ten digit classes, ranging from 0 to 9 [59].
• Penbased. The Pen-Based Recognition of Handwritten Digits dataset contains samples of handwritten digits collected from 44 writers. Each digit is represented using its $(x,y)$ coordinate information, resampled to eight points and encoded as a fixed-length feature vector of 16 attributes. The corresponding classification task aims to assign each sample to one of the ten digit classes, ranging from 0 to 9 [60].
• Phoneme. The Phoneme dataset contains acoustic measurements extracted from speech signals. The corresponding classification task aims to distinguish between nasal and oral phonemes based on their acoustic characteristics [61].
• PirVision. The PirVision dataset contains sensor measurements collected for occupancy detection. The corresponding classification task aims to determine occupancy status based on data acquired from passive infrared (PIR) sensing devices [62].
• Satimage. The Satimage dataset contains multispectral satellite image data representing different types of terrain. The corresponding classification task aims to assign image regions to different land-cover classes based on their spectral characteristics [63].
• Segment. The Segment dataset contains image regions characterized by a set of numerical features extracted from outdoor images. The corresponding classification task aims to assign each region to one of several image categories based on its visual characteristics [64].
• Shuttle. The Shuttle dataset was originally developed to derive comprehensible classification rules for determining the conditions under which automatic landing control of a spacecraft would be preferable to manual control. The corresponding classification task involves distinguishing among different operational conditions based on a set of numerical measurements [65].
• Spambase. The Spambase dataset contains features extracted from email messages, including word and character frequencies as well as information related to sequences of capital letters. The corresponding binary classification task aims to distinguish spam from non-spam emails [66].
• Spiral, The Spiral dataset is a synthetic binary classification problem consisting of 1000 two-dimensional instances equally distributed between two classes. The instances are generated using parametric equations that form two intertwined spiral patterns, resulting in a highly nonlinear classification problem [67].
• Student. The Student dataset contains educational data collected from 587 students over two consecutive academic years in ten non-university educational centers. The attributes describe factors such as academic performance, student abilities, motivation, and methodological and assessment preferences. The corresponding classification task aims to categorize students according to their educational characteristics and performance [68].
• Splice dataset, which is a biological classification benchmark consisting of 3190 DNA sequences, each represented by 60 categorical features. The task is to classify sequences into three categories: exon–intron boundaries (EI), intron–exon boundaries (IE), and non-splice junctions (N) [69].
• Texture. The Texture dataset is designed for the classification of different visual textures and comprises 11 texture classes, including grass lawn, pressed calf leather, handmade paper, raffia, and cotton canvas. Each pattern corresponds to a pixel represented by 40 numerical features obtained from the estimation of fourth-order modified moments computed at four different orientations: 0°, 45°, 90°, and 135°. The corresponding classification task aims to assign each pattern to one of the 11 texture categories [70].
• Thyroid. The Thyroid dataset contains clinical measurements related to thyroid gland function. The corresponding classification task aims to determine whether a patient exhibits normal thyroid function or suffers from hyperthyroidism or hypothyroidism, resulting in a three-class classification problem [71].
• The TwoNorm dataset is a synthetic 20-dimensional binary classification problem originally introduced by Breiman. The patterns of each class are generated from multivariate normal distributions with unit covariance matrices and symmetric mean vectors. Specifically, the two class means are $(a,a,\dots ,a)$ and $(-a,-a,\dots ,-a)$, where $a=2/\sqrt{20}$ [72].
• Vehicle. The Vehicle dataset is designed to classify vehicle silhouettes into four different categories using a set of numerical features extracted from their shapes. The silhouettes were obtained from vehicles viewed from different angles, making the classification task dependent on distinguishing characteristic geometric properties despite variations in viewing orientation [73].
• Yeast. The Yeast dataset contains information describing proteins from Saccharomyces cerevisiae. Each protein is represented by eight numerical features derived from sequence-based measurements. The corresponding classification task aims to predict the cellular localization site of each protein among ten possible classes [74].
• EEG. The EEG dataset is a medical dataset containing electroencephalographic recordings obtained from healthy subjects and patients with epilepsy under different physiological and pathological conditions [75,76]. The original dataset consists of five subsets, denoted by Z, O, N, F, and S, corresponding to different recording conditions and brain states. From these subsets, four classification problems were constructed for the experimental evaluation: Z_F_S, ZO_NF_S, Z_O_N_F_S, and ZONF_S.

The characteristics of each dataset are outlined in Table 1.

Table 1. The main characteristics of each dataset.

Dataset	Features	Patterns	Classes
ALCOHOL	154	476	4
APPENDICITIS	7	106	2
BEED	16	8000	4
DERMATOLOGY	34	358	6
GENE	120	3175	3
HAYES ROTH	5	160	3
HEPATITIS	19	80	2
HOUSEVOTES	16	232	2
IONOSPHERE	34	351	2
LYMPHOGRAPHY	18	148	4
NURSERY	8	12,960	5
OPTDIGITS	64	5620	10
PENBASED	16	10,992	10
PHONEME	5	5404	2
PIRVISION	56	15,302	3
SATIMAGE	36	6435	7
SEGMENT	19	2310	7
SHUTTLE	9	58,000	7
SPAMBASE	57	4597	2
SPIRAL	2	2000	2
SPLICE	60	3190	3
STUDENT	5	587	4
TEXTURE	40	5500	11
THYROID	21	7200	3
TWONORM	20	7400	2
VEHICLE	18	846	4
YEAST	8	1484	10
Z_F_S	21	300	3
Z_O_N_F_S	21	500	5
ZO_NF_S	21	500	3
ZONF_S	21	500	2

4.2. Experimental Results

The proposed method was implemented in C++. For each dataset, a 10-fold cross-validation procedure was employed using predefined training and test partitions. Each experiment was repeated 30 times independently for every fold, resulting in a total of 300 runs per dataset and method. In each run, the classification model was constructed and optimized exclusively using the training partition, while its classification error was evaluated on the corresponding held-out test partition. The final classification error reported in the experimental tables was calculated as the arithmetic mean of the 300 test classification errors. The test partitions were not used during the evolutionary optimization process. The code used in the experiments with GenClass is a modifed version of the original algorithm located in the repository https://github.com/itsoulos/QGenClass/(accessed on 24 October 2026). The values used for the experimental parameters are listed in Table 2.

Table 2. Values for the parameters in experiments.

Parameter	Meaning	Value
$N_c$	Number of chromosomes	500
$N_g$	Number of allowed generations	500
$N_s$	Chromosome length	200
$N_t$	Tournament size	20
w	Maximum wrapping	2
$p_s$	Selection rate	0.9
$p_m$	Mutation rate	0.05
$N_K$	Crossover items	10
k	Number of crossover trials	20
$\rho$	Elite fraction for donor selection	0.20
$B_{\mathrm{m}\mathrm{a}\mathrm{x}}$	Maximum crossover block size	16

The notation used in Table 3 is described below:

Table 3. Classification error (%) of the proposed method (STANDARD) and of the other classifiers on the 31 datasets. STANDARD is the GenClass method with the proposed additional crossover procedure, while CLASS is the original GenClass method without it. The last row (AVERAGE) reports the mean error over all datasets, and lower values indicate better performance.

Dataset	RBFCON	CART	SVM	RBF	NEAT	DNN	ADAM	BFGS	GEN	CLASS	STANDARD
ALCOHOL	20.43%	23.83%	23.40%	49.38%	66.80%	39.04%	57.78%	41.50%	39.57%	16.70%	11.70%
APPENDICITIS	17.80%	16.00%	21.00%	12.23%	17.20%	17.30%	16.50%	18.00%	18.10%	16.47%	12.80%
BEED	43.01%	38.17%	32.17%	65.12%	67.94%	64.45%	70.38%	75.24%	46.19%	31.93%	27.45%
DERMATOLOGY	39.80%	7.14%	11.14%	59.48%	32.43%	24.26%	26.14%	52.92%	30.58%	5.69%	2.80%
GENE	50.86%	30.19%	9.21%	37.11%	52.68%	33.06%	35.17%	38.12%	34.38%	8.67%	7.29%
HAYES ROTH	40.08%	37.59%	55.39%	64.36%	50.15%	44.65%	59.70%	37.33%	56.18%	24.31%	21.08%
HEPATITIS	13.13%	21.25%	16.25%	18.38%	39.75%	16.65%	16.25%	21.38%	15.00%	15.50%	12.00%
HOUSEVOTES	4.61%	4.33%	7.32%	6.13%	10.89%	3.13%	7.48%	7.13%	6.62%	3.47%	2.96%
IONOSPHERE	12.83%	9.14%	6.12%	16.22%	19.67%	12.57%	16.64%	15.29%	15.14%	8.10%	8.74%
LYMPHOGRAPHY	24.57%	22.86%	22.71%	25.50%	33.70%	24.07%	39.79%	35.43%	28.42%	22.67%	18.86%
NURSERY	44.24%	17.94%	24.62%	77.10%	64.11%	59.73%	40.94%	41.40%	32.14%	19.65%	22.27%
OPTDIGITS	62.29%	32.94%	26.62%	71.97%	72.56%	64.34%	72.32%	67.13%	67.81%	27.15%	14.77%
PENBASED	60.33%	18.00%	26.81%	81.69%	78.58%	73.50%	77.41%	82.48%	84.01%	24.93%	17.79%
PHONEME	15.59%	18.60%	21.56%	23.32%	22.43%	24.34%	29.43%	15.58%	15.55%	18.89%	16.82%
PIRVISION	8.14%	1.15%	2.05%	24.58%	11.84%	18.28%	18.38%	16.94%	14.31%	6.53%	4.38%
SATIMAGE	32.97%	23.12%	10.86%	40.19%	76.19%	72.30%	73.22%	71.73%	54.54%	18.47%	16.14%
SEGMENT	32.47%	17.45%	30.46%	59.68%	66.72%	32.04%	49.75%	68.97%	57.72%	15.21%	9.57%
SHUTTLE	23.25%	0.36%	0.32%	32.83%	29.32%	2.27%	21.29%	42.07%	20.86%	1.11%	1.28%
SPAMBASE	10.54%	13.18%	9.33%	29.35%	35.77%	11.30%	48.05%	18.16%	6.37%	8.97%	7.28%
SPIRAL	41.03%	41.70%	54.30%	44.87%	48.66%	45.24%	47.67%	47.99%	48.66%	35.65%	31.25%
SPLICE	28.37%	15.83%	14.61%	40.27%	55.54%	48.18%	75.93%	30.64%	30.97%	7.68%	7.62%
STUDENT	12.43%	7.93%	4.70%	5.49%	10.20%	6.93%	5.13%	7.14%	5.61%	6.86%	5.72%
TEXTURE	46.87%	35.44%	24.55%	74.52%	70.82%	70.78%	56.33%	59.25%	54.00%	20.01%	10.71%
THYROID	6.35%	0.86%	5.42%	10.62%	18.06%	7.42%	3.43%	5.18%	7.65%	1.24%	0.83%
TWONORM	12.90%	15.58%	2.15%	2.18%	13.54%	2.16%	5.58%	9.22%	9.59%	6.77%	6.57%
VEHICLE	70.24%	41.07%	42.02%	78.06%	64.05%	59.22%	74.41%	72.99%	60.24%	36.55%	35.83%
YEAST	65.83%	50.68%	61.80%	67.14%	71.09%	68.87%	57.69%	77.28%	69.13%	45.95%	44.35%
Z_F_S	4.68%	5.00%	27.33%	13.16%	38.41%	9.27%	47.81%	39.37%	10.73%	7.39%	6.00%
Z_O_N_F_S	44.28%	27.80%	46.12%	48.70%	77.08%	67.80%	78.79%	65.67%	64.81%	30.32%	25.32%
ZO_NF_S	5.47%	5.20%	24.40%	9.02%	43.75%	8.50%	47.43%	43.04%	21.54%	5.03%	3.69%
ZONF_S	1.82%	1.60%	16.40%	4.03%	5.44%	2.52%	11.99%	15.62%	4.36%	2.04%	2.00%
AVERAGE	28.94%	19.42%	21.97%	38.47%	44.04%	33.37%	41.57%	40.01%	33.25%	16.13%	13.42%

• DATASET denotes the dataset used in the corresponding experiment.
• RBFCON: this method was presented recently [77] to construct Radial Basis Function (RBF) neural networks, with the assistance of Grammatical Evolution. The parameters used for this method are presented in Table 2.
• CART denotes the Classification and Regression Trees (CART) algorithm [78] used to construct binary decision trees by recursively partitioning the feature space. For classification tasks, the splits are typically selected using the Gini impurity criterion, while tree pruning can be employed to reduce overfitting.
• SVM: Support Vector Machines (SVMs) [79] were also used here and was implemented using the LIBSVM library [80].
• RBF denotes a Radial Basis Function (RBF) neural network [81,82] with $H=10$ hidden nodes.
• NEAT denotes the NeuroEvolution of Augmenting Topologies (NEAT) method [83].
• DNN denotes a deep neural network implemented using the Tiny-DNN library, publicly available at the project’s GitHub repository https://github.com/tiny-dnn/tiny-dnn (accessed on 27 October 2026). The network was trained using the AdaGrad optimization algorithm [84].
• ADAM denotes a neural network with $H=10$ hidden nodes trained using the Adam optimization algorithm [85].
• BFGS denotes a neural network with $H=10$ hidden nodes trained using Powell’s variant of the BFGS optimization method [86].
• GEN denotes a neural network with $H=10$ hidden nodes whose weights were optimized using a Genetic Algorithm. The Genetic Algorithm employed the same parameter settings reported in Table 2.
• CLASS denotes the original GenClass rule construction method without the additional crossover mechanism proposed in this work. The parameters used for this method are presented in Table 2.
• STANDARD denotes GenClass enhanced with the proposed Standard crossover method. The parameters used for this method are presented in Table 2.
• The row AVERAGE reports the mean classification error across all datasets for the corresponding method.

Figure 4 compares the average classification errors of the evaluated methods with the majority-class baseline. The STANDARD method achieves the lowest average classification error (13.42%), followed by TARGETED (15.08%), TARGETEDWORST (15.42%), and the original CLASS method (16.13%). In contrast, the majority-class baseline yields an average error of 56.23%. Although some competing classifiers perform worse than the majority-class baseline on individual datasets, the four GenClass variants consistently achieve lower classification errors across all 31 datasets.

Figure 4. Average classification error (%) of the evaluated classification methods across 31 benchmark datasets. The reported values represent the unweighted mean of the dataset-level classification errors. Lower values indicate better classification performance. The majority-class classifier is included as a simple reference baseline.

To assess the influence of the additional computational effort introduced by the crossover mechanism, we performed a supplementary comparison between STANDARD and the original CLASS method under equal fitness evaluation budgets. For each corresponding experimental run, the total number of fitness evaluations performed by STANDARD was recorded and subsequently used as the maximum evaluation budget for CLASS.

The results are presented in Table 4. STANDARD achieved an average classification error of 13.71%, compared with 14.21% for CLASS. Although STANDARD retained a small numerical advantage, the difference was not statistically significant according to the Wilcoxon signed-rank test ($p\approx 0.608$).

Table 4. Classification error (%) of STANDARD and CLASS under equal fitness evaluation budgets. The bold notation is used for the lowest values.

Dataset	STANDARD	CLASS-EQUAL
Alcohol	10.43	12.13
Appendicitis	15.00	14.00
Beed	25.68	27.78
Dermatology	6.00	5.14
Gene	7.35	6.85
Hayes-Roth	27.69	22.31
Hepatitis	13.75	17.50
Housevotes	3.48	3.89
Ionosphere	10.00	7.43
Lymography	24.29	22.86
Nursery	19.10	18.41
Optdigits	13.29	16.94
Penbased	14.52	21.68
Phoneme	16.73	17.54
Pir	1.40	1.19
Satimage	15.26	15.23
Segment	9.05	10.35
Shuttle	1.05	0.20
Spambase	6.93	7.76
Spiral	28.90	34.20
Splice	7.21	6.90
Student	7.45	5.73
Texture	8.36	12.91
Thyroid	0.64	0.96
Twonorm	5.57	2.82
Vehicle	35.83	36.07
Wine	8.82	8.82
Yeast	46.83	45.15
Z_F_S	6.33	6.67
Z_O_N_F_S	25.40	28.80
ZONF_S	2.80	2.20
Mean	13.71	14.21

These findings suggest that the additional computational effort associated with the crossover mechanism may contribute to the improvements observed in the original generation-based comparison. Nevertheless, the proposed method maintains a slightly lower average classification error when both approaches are evaluated under equal fitness evaluation budgets.

To further examine whether the observed improvements can be attributed to the proposed crossover mechanism rather than to the increased computational budget, an additional experiment was conducted using a restart-based baseline. In this experiment, the original CLASS method was executed twice independently with different random seeds, without the additional crossover mechanism. The total fitness evaluation budget was divided between the two executions, and the best solution was selected according to its training fitness. Consequently, the restart-based baseline was allocated the same total number of fitness evaluations as STANDARD.

The results of the equal-budget comparison are summarized in Table 5. As shown in the table, STANDARD achieved the lowest average classification error (14.15%), followed by MUTATION-LS (14.22%), equal-budget CLASS (14.53%), and the restart-based baseline (14.97%). The restart-based approach outperformed STANDARD on 10 datasets, whereas STANDARD achieved lower classification errors on 18 datasets. The two methods produced identical results on one dataset. A paired Wilcoxon signed-rank test yielded p-values of 0.1271 and 0.4118 for the comparisons of the restart-based approach against STANDARD and equal-budget CLASS, respectively. Neither comparison revealed a statistically significant difference at the 0.05 significance level.

Table 5. Comparison of the average classification errors obtained by STANDARD-EQUAL, MUTATION-LS, CLASS-EQUAL, and CLASS-RESTART under equal total fitness evaluation budgets.

Method	Average Classification Error (%)
STANDARD-EQUAL	14.15%
MUTATION-LS	14.22%
CLASS-EQUAL	14.53%
CLASS-RESTART	14.97%

An additional statistical comparison was performed between STANDARD and MUTATION-LS to investigate whether the proposed crossover mechanism provides an advantage over mutation-based local search under the same computational budget. MUTATION-LS achieved lower classification errors on 11 datasets, whereas STANDARD performed better on 16 datasets, with identical results observed on two datasets. The paired Wilcoxon signed-rank test yielded a p-value of 0.6830, indicating that the difference between the two methods was not statistically significant at the 0.05 significance level.

The results of the computational-budget-controlled experiments indicate that STANDARD achieved the lowest average classification error among the four examined methods. However, the differences between STANDARD and the restart-based baseline, as well as between STANDARD and MUTATION-LS, were not statistically significant. These findings suggest that the proposed crossover mechanism may offer a modest practical advantage under equal computational budgets, but they do not establish its statistical superiority over alternative strategies that allocate the same number of fitness evaluations. Consequently, the improvements observed in the main experimental evaluation should be interpreted in conjunction with these additional controlled comparisons.

Statistical analysis. The comparison of the methods was based on the average classification error of each method on each dataset, so that every dataset provides one observation and $N=31$ observations are available for each method. All tests were performed at the significance level $\alpha =0.05$. For every table, the Friedman test was first applied in order to examine the hypothesis that all the methods of the table perform equally. For each dataset the methods are ranked from the lowest to the highest error, with ties receiving the average rank, and the test compares the mean ranks of the methods. The result is also reported with the Iman-Davenport correction, which is less conservative, and the strength of the differences is expressed by Kendall’s coefficient of concordance $W={\chi }^2/\left(N(k-1)\right)$, where k is the number of methods. The Nemenyi test was then used to compare all the methods through their mean ranks, and two methods are considered different when their mean ranks differ by more than the critical difference $CD$. In addition, specific pairs of methods were compared with the two-sided Wilcoxon signed-rank test for paired samples, applied to the differences of the errors on the individual datasets, where zero differences are discarded and tied absolute differences receive average ranks. The p-values of the Wilcoxon tests were adjusted with the Holm method separately for each table, over the family of comparisons of that table. These are the ten comparisons of STANDARD with each of the other methods in Table 3, and all the pairs of methods in Table 6 (three comparisons) and in Table 7 (six comparisons). Finally, for Table 7, the one-sided Page trend test was used to examine whether the error decreases as $N_K$ increases, together with the Spearman correlation between $N_K$ and the error in each dataset, whose sign was summarized with a sign test.

Table 6. Classification error (%) of the three crossover strategies.

Dataset	STANDARD	TARGETED	TARGETEDWORST
ALCOHOL	11.70%	17.65%	14.04%
APPENDICITIS	12.80%	16.40%	15.40%
BEED	27.45%	29.48%	31.93%
DERMATOLOGY	2.80%	6.12%	5.14%
GENE	7.29%	9.05%	8.20%
HAYES ROTH	21.08%	22.00%	25.38%
HEPATITIS	12.00%	16.25%	16.25%
HOUSEVOTES	2.96%	3.72%	3.04%
IONOSPHERE	8.74%	7.77%	7.77%
LYMPHOGRAPHY	18.86%	25.00%	21.86%
NURSERY	22.27%	21.44%	21.02%
OPTDIGITS	14.77%	19.59%	19.88%
PENBASED	17.79%	15.10%	23.85%
PHONEME	16.82%	17.86%	17.86%
PIRVISION	4.38%	4.47%	6.53%
SATIMAGE	16.14%	17.52%	18.14%
SEGMENT	9.57%	12.86%	11.26%
SHUTTLE	1.28%	1.35%	0.94%
SPAMBASE	7.28%	8.22%	8.10%
SPIRAL	31.25%	35.30%	34.83%
SPLICE	7.62%	8.12%	8.15%
STUDENT	5.72%	5.92%	6.45%
TEXTURE	10.71%	17.23%	16.59%
THYROID	0.83%	1.35%	1.53%
TWONORM	6.57%	3.90%	5.00%
VEHICLE	35.83%	39.64%	37.03%
YEAST	44.35%	45.22%	47.77%
Z_F_S	6.00%	6.40%	7.33%
Z_O_N_F_S	25.32%	28.12%	30.66%
ZO_NF_S	3.69%	3.08%	4.04%
ZONF_S	2.00%	1.28%	2.04%
AVERAGE	13.42%	15.08%	15.42%

Table 7. Experiments with the number that participate in the crossover procedure.

Dataset	${\mathit{N}}_{\mathit{K}}=2$	${\mathit{N}}_{\mathit{K}}=5$	${\mathit{N}}_{\mathit{K}}=10$	${\mathit{N}}_{\mathit{K}}=15$
ALCOHOL	12.55%	13.19%	11.70%	15.11%
APPENDICITIS	15.60%	14.40%	12.80%	15.80%
BEED	27.30%	26.78%	27.45%	25.47%
DERMATOLOGY	4.74%	4.86%	2.80%	4.00%
GENE	8.39%	7.29%	7.29%	6.91%
HAYES ROTH	20.77%	19.54%	21.08%	21.39%
HEPATITIS	14.00%	11.50%	12.00%	10.00%
HOUSEVOTES	3.63%	2.78%	2.96%	3.28%
IONOSPHERE	7.60%	7.31%	8.74%	8.80%
LYMPHOGRAPHY	23.71%	23.00%	18.86%	20.57%
NURSERY	20.33%	22.16%	22.27%	18.76%
OPTDIGITS	17.16%	14.56%	14.77%	13.99%
PENBASED	19.95%	16.54%	17.79%	14.22%
PHONEME	16.99%	16.21%	16.82%	16.00%
PIRVISION	8.49%	6.33%	4.38%	5.35%
SATIMAGE	15.91%	15.04%	16.14%	14.93%
SEGMENT	11.95%	9.91%	9.57%	9.44%
SHUTTLE	1.15%	0.62%	1.28%	0.24%
SPAMBASE	7.70%	7.98%	7.28%	7.33%
SPIRAL	36.80%	33.55%	31.25%	30.35%
SPLICE	7.46%	7.18%	7.62%	7.34%
STUDENT	6.11%	4.92%	5.72%	6.70%
TEXTURE	12.87%	12.73%	10.71%	12.44%
THYROID	1.57%	1.06%	0.83%	0.76%
TWONORM	5.62%	5.31%	6.57%	8.01%
VEHICLE	37.62%	34.81%	35.83%	33.69%
YEAST	46.21%	45.69%	44.35%	44.01%
Z_F_S	7.00%	6.47%	6.00%	5.67%
Z_O_N_F_S	27.28%	27.00%	25.32%	23.80%
ZO_NF_S	3.68%	3.32%	3.69%	3.40%
ZONF_S	1.96%	1.44%	2.00%	1.16%
AVERAGE	14.58%	13.66%	13.42%	13.19%

Table 3 presents the classification errors obtained by STANDARD, the original GenClass method (CLASS), and the additional classification methods considered in this study across the 31 datasets. Lower values indicate better classification performance. STANDARD achieved an average classification error of 13.42%, compared with 16.13% for CLASS, corresponding to a reduction of 2.71 percentage points (approximately 16.8% relative reduction). Among the external reference methods, the lowest average errors were obtained by CART (19.42%), SVM (21.97%), and RBFCON (28.94%), while the remaining neural network methods ranged from 33.25% (GEN) to 44.04% (NEAT). STANDARD obtained the lowest error on 19 of the 31 datasets, followed by SVM (5 datasets), CART (3), GEN (2), RBF (1), and RBFCON (1), whereas CLASS did not obtain the lowest error on any dataset. These findings provide evidence of improved classification performance within the GenClass framework under the examined experimental configuration. The remaining classifiers are included as external reference methods, rather than as part of an exhaustive comparison of supervised learning algorithms.

The critical difference of the Nemenyi test is $CD={\displaystyle \frac{q_{\alpha }}{\sqrt{2}}}\sqrt{{\displaystyle \frac{k(k+1)}{6N}}}$, where $q_{\alpha }$ is the critical value of the studentized range distribution for k methods, and it therefore depends on the number of methods compared. For each pair of methods, the size of the difference was estimated with the Hodges–Lehmann estimator, which is the median of the pairwise averages of the paired differences of the errors, and its 95% confidence interval was derived from the Wilcoxon signed-rank test using the normal approximation. The effect size of each Wilcoxon test was expressed as $r=Z/\sqrt{n}$, where Z is the standardized statistic of the test and n is the number of datasets with non-zero difference. The confidence intervals of the mean rank and of the mean change in error in Figure 7 were computed with the percentile bootstrap, using 10,000 resamples of the datasets with replacement.

Notation for Figure 5, Figure 6 and Figure 7. Each figure has four panels and a header with the Friedman test, the Iman-Davenport correction and Kendall’s W.

Figure 5. Statistical comparison of the proposed method (STANDARD) against the other classifiers over 31 datasets. Asterisks denote the significance of the Holm-adjusted Wilcoxon signed-rank test: *** $p<0.001$.

Figure 6. Statistical comparison of the three crossover strategies. (STANDARD, TARGETED, TARGETEDWORST). Asterisks denote the significance of the Holm-adjusted Wilcoxon signed-rank test: ** $p<0.01$, *** $p<0.001$, ns: not significant ($p\ge 0.05$).

• Box plots (top left): The box is the interquartile range ($IQR$), the thick line is the median, the whiskers extend up to $1.5\times IQR$, the diamond is the mean, the dots are the datasets and the grey lines join the same dataset.
• Pairs “A vs. B” (top right): A is the first method named and B the second. The dot is the Hodges-Lehmann median paired difference, error(B) − error(A), with its 95% confidence interval. Right of the dashed line, A has lower error. The label gives the significance stars of the Holm-adjusted Wilcoxon test and the wins/ties/losses of A.
• Mean ranks (bottom left): Mean Friedman rank (1 = best) $\pm CD/2$, where $CD$ is the Nemenyi critical difference. Non-overlapping bars differ at $\alpha =0.05$.
• Bottom right: In Figure 5 and Figure 6, the distribution of error(B) − error(A) over the datasets for each pair (blue: A better, red: B better). In Figure 7, the mean change in error relative to $N_K=2$ (upper) and the mean rank (lower) as a function of $N_K$, with bootstrap 95% confidence intervals.
• Stars: *** $p<0.001$, ** $p<0.01$, * $p<0.05$, ns = not significant (Holm-adjusted).

The statistical analysis of Table 3 is summarized in Figure 5. The Friedman test rejects the hypothesis that all eleven methods perform equally (${\chi }^2\left(10\right)=164.49$, $p=3.8\times {10}^{-30}$), a result confirmed by the Iman-Davenport correction ($F(10,300)=33.91$, $p=9.9\times {10}^{-44}$), with a substantial effect size (Kendall’s $W=0.53$). STANDARD obtains the best mean rank (1.87), followed by CLASS (2.97), CART (3.90) and SVM (4.73), while all the other methods have mean ranks above 5.5. In the paired comparisons, STANDARD has lower error than every other method, with Holm-adjusted Wilcoxon signed-rank p-values between $1.2\times {10}^{-5}$ and $1.0\times {10}^{-4}$ and wins in 25 to 31 of the 31 datasets. The Hodges-Lehmann estimates of the median paired difference range from approximately 2.4 percentage points (CLASS) to 30.4 percentage points (NEAT), with 5.2 percentage points against CART and 8.0 percentage points against SVM. None of the 95% confidence intervals includes zero, and the effect sizes are large ($r\ge 0.72$, where values around 0.5 or above are conventionally considered large). The comparisons with CLASS and CART require a careful reading. Their bars overlap with that of STANDARD in the mean-rank panel, which means that the conservative Nemenyi test does not separate them from STANDARD, since the differences in mean rank (1.10 and 2.03) are smaller than the critical difference (2.71). The paired Wilcoxon test, which also takes into account the magnitude of the differences, nevertheless detects a consistent improvement (28 wins and 3 losses against CLASS, 26 wins and 5 losses against CART). STANDARD is worse than CLASS only on IONOSPHERE, NURSERY and SHUTTLE, and worse than CART on NURSERY, PIRVISION, SHUTTLE, Z_F_S and ZONF_S. The competitor with the most individual wins against STANDARD is SVM, which achieves lower error on six datasets (IONOSPHERE, PIRVISION, SATIMAGE, SHUTTLE, STUDENT and TWONORM), although the overall difference remains significant in favor of STANDARD (25 wins and 6 losses).

To examine the potential influence of dependent benchmark datasets, an additional statistical analysis was conducted by excluding the four EEG-derived datasets, which originate from the same underlying recordings. The Wilcoxon signed-rank test was applied to the remaining 27 datasets. The STANDARD method retained a statistically significant advantage over CLASS $\left(p=0.54\times {10}^{-6}\right)$. In contrast, the differences between TARGETED and CLASS ($p=0.11105$) and between TARGETEDWORST and CLASS ($p=0.04215$) were not statistically significant after applying the Holm correction for multiple comparisons. These findings indicate that the statistically significant improvement of STANDARD over the original GenClass method persists after excluding the related EEG datasets, whereas the evidence supporting improvements from the other two crossover strategies is less conclusive.

4.3. Crossover Methods

Table 6 isolates the effect of the crossover strategy. STANDARD, in which the exchanged chromosome regions are selected purely stochastically, achieves the lowest average error (13.42%), whereas the two strategies that exploit information collected during the evolutionary process are less effective, with 15.08% for TARGETED and 15.42% for TARGETEDWORST. STANDARD obtained the lowest error in 24 of the 31 datasets, compared with 5 for TARGETED and 3 for TARGETEDWORST (a tie between the two targeted strategies in IONOSPHERE is counted for both). The average gain of STANDARD is 1.66 percentage points over TARGETED and 2.0 percentage points over TARGETEDWORST, which correspond to relative reductions of 11.0% and 13.0%, respectively. The two targeted strategies differ little from each other. Hence, in these experiments, guiding the exchange of genetic material with information from the evolutionary process did not improve upon the purely random exchange.

Figure 6 presents the statistical analysis of Table 6. The Friedman test indicates significant differences among the three strategies (${\chi }^2\left(2\right)=22.13$, $p=1.6\times {10}^{-5}$, Iman-Davenport $F(2,60)=16.65$, $p=1.8\times {10}^{-6}$), with a moderate effect size ($W=0.36$). STANDARD has the best mean rank (1.32), against 2.27 for TARGETED and 2.40 for TARGETEDWORST. Since the critical difference is 0.60, the bar of STANDARD does not overlap with those of the other two strategies, while the bars of TARGETED and TARGETEDWORST overlap. The paired tests agree with this picture. STANDARD is significantly better than TARGETED (25 wins and 6 losses, median difference 1.62 percentage points, 95% CI $[0.58,2.55]$, $p=0.0018$, $r=0.60$) and than TARGETEDWORST (27 wins and 4 losses, median difference 1.93 percentage points, 95% CI $[1.08,2.80]$, $p=1.6\times {10}^{-4}$, $r=0.73$). On the contrary, TARGETED and TARGETEDWORST cannot be distinguished (15 wins, 3 ties and 13 losses, median difference approximately 0.1 percentage points, 95% CI $[-0.48,0.95]$, $p=0.64$, $r=0.09$).

To further assess the effectiveness of the three crossover strategies, we performed pairwise comparisons against the original CLASS method using two-sided Wilcoxon signed-rank tests. For the complete set of 31 datasets, the p-values were $1.381\times {10}^{-6}$ for STANDARD, 0.022385 for TARGETED, and 0.030285 for TARGETEDWORST. Since four EEG-derived datasets originate from the same underlying recordings, we additionally repeated the analysis after excluding these related problems. For the remaining 27 datasets, the corresponding p-values were $9.537\times {10}^{-6}$, 0.106917, and 0.040013, respectively. After applying the Holm correction for the three comparisons, only STANDARD retained statistical significance at the 0.05 level in the analysis excluding the EEG-derived datasets. These findings indicate that STANDARD provides the most robust improvement over the original CLASS method, whereas the evidence supporting the two targeted strategies is less conclusive.

4.4. Number of Crossing Items

The value $N_K=10$ was used as the default configuration in the main experiments without performing a dataset-specific parameter optimization procedure. The experiments presented in this subsection constitute a sensitivity analysis intended to investigate the influence of the number of chromosomes participating in the crossover operation. Although $N_K=15$ achieved the lowest average classification error, the default configuration was retained for the main comparisons rather than selecting the best-performing value separately for each dataset.

Table 7 reports the results of the proposed method when $N_K$ = 2, 5, 10 and 15 chromosomes participate in the crossover. The column $N_K$ = 10 corresponds to the default value of Table 2 and coincides with the STANDARD column of Table 6. The average error decreases monotonically as $N_K$ increases, from 14.58% for $N_K$ = 2 to 13.66% for $N_K$ = 5, 13.42% for $N_K$ = 10 and 13.19% for $N_K$ = 15. The setting $N_K$ = 15 gave the lowest error in 17 of the 31 datasets, $N_K$ = 5 and $N_K$ = 10 in 7 datasets each, and $N_K$ = 2 in none. Increasing $N_K$ from 2 to 15 reduces the average error by 1.39 percentage points, whereas each of the two further increases beyond $N_K$ = 5 changes it by less than 0.3 percentage points.

The identical results obtained by Targeted and TargetedWorst on the Hepatitis and Phoneme datasets can be attributed to their binary classification structure. In the proposed chromosome representation, a problem with C classes contains $C-1$ explicit classification rules, with the remaining class handled by a default rule. Consequently, for binary classification problems, the single explicit rule spans the entire chromosome, making the class-specific crossover restriction of TargetedWorst equivalent to the unrestricted selection employed by Targeted.

For the BEED dataset, the identical classification error reported for CLASS and TargetedWorst (31.93%) should not be interpreted as algorithmic equivalence. BEED contains four classes, and TargetedWorst performs additional class-directed crossover operations that are absent from the CLASS baseline. Supplementary implementation checks confirmed that these operations were executed and produced accepted fitness improvements. However, the numerical coincidence in the original results has not been independently reproduced.

Figure 7 presents the corresponding statistical analysis. The Friedman test detects significant differences among the four settings (${\chi }^2\left(3\right)=21.97$, $p=6.6\times {10}^{-5}$, Iman-Davenport $F(3,90)=9.28$, $p=2.1\times {10}^{-5}$), although with a smaller effect size than in the previous comparisons ($W=0.24$). The mean rank is the worst for $N_K=2$ (3.35) and the best for $N_K=15$ (1.87), with $N_K=5$ (2.27) and $N_K=10$ (2.50) in between. The one-sided Page trend test, which examines whether the error decreases as $N_K$ increases, confirms a significant trend ($z=4.08$, $p=2.3\times {10}^{-5}$). In agreement, the Spearman correlation between $N_K$ and the error is negative in 23 of the 29 datasets where it is non-zero (median correlation $-0.6$, sign test $p=0.0023$). The pairwise comparisons show, however, that this trend is mainly due to the weakness of the smallest setting. The value $N_K=2$ has significantly higher error than $N_K=5$ ($p=3.8\times {10}^{-4}$), $N_K=10$ ($p=0.0047$) and $N_K=15$ ($p=0.0039$), with median differences of approximately 0.8, 1.0 and 1.3 percentage points, respectively. None of the differences among $N_K=5$, 10 and 15 is statistically significant (adjusted $p=0.61$ for 5 versus 10, 0.20 for 5 versus 15 and 0.61 for 10 versus 15), and their confidence intervals include zero. Therefore, a very small number of chromosomes in the crossover is detrimental, whereas beyond $N_K=5$ any further benefit is small and is not statistically established in these experiments, although $N_K=15$ has the best mean rank.

Figure 7. Effect of the number of chromosomes ($N_K$) participating in the crossover on the proposed method. Asterisks denote the significance of the Holm-adjusted Wilcoxon signed-rank test: ** $p<0.01$, *** $p<0.001$, ns: not significant ($p\ge 0.05$).

Figure 7. Effect of the number of chromosomes ($N_K$) participating in the crossover on the proposed method. Asterisks denote the significance of the Holm-adjusted Wilcoxon signed-rank test: ** $p<0.01$, *** $p<0.001$, ns: not significant ($p\ge 0.05$).

The figure shows the average execution time of the Standard technique on the Dermatology dataset for $N_K$ = 2, 5, 10 and 15. The time increases with $N_K$, from 104.4 s for $N_K=2$ to 302.6 s for $N_K=15$, which is about 2.9 times higher. The increase is approximately linear up to $N_K=10$, at about 19 s per additional chromosome, and becomes slower from $N_K=10$ to $N_K=15$, at about 9 s per additional chromosome. The observed execution-time increase is not uniform across the examined values of $N_K$. Although the increase becomes smaller between $N_K=10$ and $N_K=15$, the available measurements do not establish the underlying cause of this behavior. Consequently, no specific explanation for the change in the slope of the execution-time curve can be confirmed from these results alone as outlined in Figure 8.

Figure 8. Execution time of the STANDARD crossover technique on the Dermatology dataset for different values of $\left(N_K\right)$. The baseline configuration $(N_K=0)$ required 81.80 s, while execution time increased to 302.60 s for $(N_K=15)$. The baseline value represents a single execution, whereas the remaining values correspond to the previously reported average execution times.

For the Dermatology dataset, the default value $N_K=10$ requires 255.6 s, whereas $N_K=15$ requires 302.6 s, representing an increase of approximately 18%. This dataset is used solely to illustrate the computational cost associated with increasing $N_K$ and should not be considered representative of the overall classification performance. Across all 31 datasets, $N_K=15$ achieves the lowest average classification error and the best mean rank, although its advantage over $N_K=5$ and $N_K=10$ is not statistically significant after correction for multiple comparisons.

4.5. Computational Cost Analysis

To provide a more comprehensive assessment of the computational cost of the proposed crossover operators, additional experiments were conducted on three relatively large datasets: Nursery, Pirvision, and Shuttle. Both wall-clock execution time and the total number of fitness evaluations were measured for the original CLASS configuration and the STANDARD, TARGETED, and TARGETEDWORST crossover variants. All configurations were evaluated using identical experimental settings and the same random seed, with $N_K=0$ for CLASS and $N_K=10$ for the crossover variants.

Table 8 reports the measured execution times and total fitness evaluation counts. The results reveal substantial differences in computational cost among the examined approaches. STANDARD consistently exhibits the highest overhead, requiring approximately 3.49–3.93 times the execution time of CLASS. In contrast, TARGETED introduces a more moderate overhead, with execution-time ratios ranging from 1.40 to 1.67. TARGETEDWORST exhibits dataset-dependent behavior, with execution-time ratios ranging from 0.64 to 1.85.

Table 8. Wall-clock execution time and total fitness evaluations for CLASS and the three crossover variants on Nursery, Pirvision, and Shuttle. The baseline uses $N_K=0$, while the crossover variants use $N_K=10$. Results correspond to one run per configuration.

Dataset	Method	Time (s)	Fitness Evaluations
Nursery	CLASS	1230.76	1,595,309
	STANDARD	4293.11	5,564,200
	TARGETED	1755.92	2,038,902
	TARGETEDWORST	2275.75	2,079,594
Pirvision	CLASS	1984.78	2,072,441
	STANDARD	7805.84	6,776,246
	TARGETED	2771.94	2,676,918
	TARGETEDWORST	1267.48	2,160,418
Shuttle	CLASS	4750.54	1,783,945
	STANDARD	18,632.64	6,335,527
	TARGETED	7912.64	2,481,313
	TARGETEDWORST	7386.76	2,364,263

The number of fitness evaluations generally follows a similar pattern, although execution time is not strictly proportional to the evaluation count. This suggests that factors beyond the number of evaluations may also influence total execution time. In particular, STANDARD introduces a substantial increase in both execution time and fitness evaluations.

An additional baseline measurement was performed on the Dermatology dataset with $N_K=0$. The recorded execution time was 81.80 s, with 2,041,800 fitness evaluations. The previously reported time of 255.60 s at $N_K=10$ is approximately 3.12 times this baseline; this comparison is valid only if both measurements were obtained under equivalent experimental conditions.

These findings demonstrate that the computational overhead of the proposed crossover strategies depends on the operator and dataset and can be substantial, particularly for STANDARD. The additional crossover operations therefore represent a trade-off between classification performance and computational cost rather than a computationally negligible modification. Because these measurements correspond to a single execution per configuration, they are intended to characterize computational overhead rather than establish statistically significant timing differences.

4.6. Crossover Ablation Study

To examine whether the observed behavior of targeted crossover is attributable to donor selection, exchange geometry, or local grammar-based matching, we conducted a controlled ablation study. Six crossover configurations were considered:

• Standard, the original standard crossover using donors sampled from the population and tail exchange;
• Standard + Elite, which retains the standard exchange but restricts donor selection to the top 20% of the population;
• Standard + Block, which retains population-wide donor selection but limits the exchanged segment to at most 16 codons;
• Targeted, which combines elite donor selection, a maximum exchanged block length of 16 codons, and local grammar-based matching;
• Targeted + Full, which replaces elite donor selection in Targeted with population-wide sampling; and
• TargetedWorst, the corresponding targeted variant using its alternative recipient-selection strategy. These controls examine individual design choices but do not constitute a complete factorial isolation of all possible interactions.

Experiments used ten classification datasets (Dermatology, Hepatitis, Housevotes, Ionosphere, Optdigits, Segment, Spambase, Thyroid, Vehicle, and Yeast). For each dataset and configuration, 30 runs were performed using the same set of random seed values, giving 1800 runs in total. All runs were constrained to exactly 100,000 fitness evaluations, with a maximum of 1000 generations. The number of crossover items was 10; the targeted variants used 20 crossover iterations and a maximum block length of 16 codons. The results refer to one predefined training/test split per dataset, rather than a complete ten-fold cross-validation. Consequently, the conclusions are conditional on the selected splits. The recorded test classification errors are reported as mean ± sample standard deviation across the 30 runs.

Table 9 shows that no single operator is best on every dataset. Targeted+Full achieves the lowest mean classification error on five datasets, while Standard performs best on Dermatology and Standard+Block on Hepatitis. The relatively small differences between several configurations should not be interpreted as evidence of universal superiority.

Table 9. Classification error (%, mean ± standard deviation over 30 runs) under an equal budget of 100,000 fitness evaluations per run. Lower values are better.

Dataset	Standard	Std. + Elite	Std. + Block	Targeted	Targeted + Full	TargetedWorst
Dermatology	$6.48\pm 5.19$	$10.67\pm 6.75$	$9.05\pm 5.26$	$6.86\pm 5.49$	$8.19\pm 5.45$	$7.24\pm 5.02$
Hepatitis	$21.25\pm 16.46$	$25.42\pm 16.89$	$16.25\pm 13.19$	$24.58\pm 14.86$	$22.50\pm 15.19$	$24.58\pm 14.86$
Housevotes	$4.06\pm 1.10$	$4.35\pm 0.00$	$4.20\pm 0.79$	$4.06\pm 1.10$	$3.48\pm 1.77$	$4.06\pm 1.10$
Ionosphere	$13.14\pm 3.80$	$13.71\pm 4.34$	$12.67\pm 4.09$	$11.52\pm 3.71$	$11.62\pm 3.26$	$11.52\pm 3.71$
Optdigits	$43.62\pm 5.69$	$44.86\pm 5.00$	$43.74\pm 5.97$	$43.14\pm 5.01$	$42.98\pm 3.72$	$46.36\pm 4.69$
Segment	$27.56\pm 6.46$	$29.60\pm 6.06$	$26.93\pm 6.40$	$25.57\pm 4.85$	$25.02\pm 5.46$	$25.32\pm 7.19$
Spambase	$10.09\pm 2.12$	$10.61\pm 2.10$	$9.51\pm 2.25$	$9.69\pm 1.95$	$8.69\pm 1.84$	$9.69\pm 1.95$
Thyroid	$4.79\pm 1.61$	$4.25\pm 2.04$	$5.00\pm 1.64$	$4.48\pm 1.78$	$3.51\pm 1.78$	$4.20\pm 1.78$
Vehicle	$39.29\pm 5.99$	$42.54\pm 8.11$	$42.78\pm 7.84$	$40.71\pm 8.08$	$39.64\pm 6.12$	$39.33\pm 6.63$
Yeast	$55.59\pm 4.50$	$57.18\pm 4.03$	$54.48\pm 3.55$	$54.84\pm 3.70$	$55.09\pm 8.14$	$53.81\pm 4.33$

As shown in Table 10, the targeted configurations produce more accepted improvements per crossover fitness evaluation than the standard configurations. In particular, Targeted yields approximately 2.41 times the normalized acceptance count of Standard. However, these are not equal numbers of crossover evaluations: the fixed total evaluation budget also covers other algorithmic operations, and the crossover implementations use different numbers of trials and evaluations. The normalized ratio is therefore an operational efficiency measure, not direct proof that grammar-based matching alone causes the difference. TargetedWorst also records substantially more skipped trials, indicating that some attempted operations terminate without a crossover fitness evaluation.

Table 10. Crossover statistics aggregated over all ten datasets and 30 runs per dataset. Accepted improvements and skipped trials are mean counts per run; the last column is the ratio of total accepted improvements to total crossover fitness evaluations, multiplied by 1000.

Operator	Accepted/Run	Skipped/Run	Accepted/1000 Evals
Standard	965.5	0.0	12.18
Standard + Elite	810.0	0.0	10.22
Standard + Block	904.0	0.0	11.40
Targeted	629.8	489.2	29.33
Targeted + Full	636.1	560.7	29.67
TargetedWorst	479.8	1355.1	23.06

For a dataset-level statistical comparison, the six operators were ranked according to their mean classification error on each of the ten datasets. A Friedman test identified an overall difference (${\chi }^2\left(5\right)=18.878$, $p=0.00203$). The mean ranks (smaller is better) were 2.20 for Targeted + Full, 2.85 for TargetedWorst, 3.05 for Targeted, 3.50 for Standard, 3.90 for Standard + Block, and 5.50 for Standard+Elite. Five pre-specified pairwise comparisons were subsequently performed using two-sided Wilcoxon signed-rank tests on the ten paired dataset means, with Holm correction: Standard versus Targeted ($p_{\mathrm{adj}}=1.0000$), Standard versus Standard + Elite ($p_{\mathrm{adj}}=0.0488$), Standard versus Standard + Block ($p_{\mathrm{adj}}=1.0000$), Targeted versus Targeted + Full ($p_{\mathrm{adj}}=0.6406$), and Targeted versus TargetedWorst ($p_{\mathrm{adj}}=1.0000$). Only the Standard versus Standard + Elite comparison met the 0.05 threshold after correction, with Standard achieving the lower classification error on average. These tests use datasets as the paired analysis units, rather than treating all 300 runs per operator as independent datasets.

The ablations suggest that restricting donors to the elite subset is not beneficial by itself and that limiting exchange length does not guarantee improved generalization. Local grammar-based matching may increase the frequency of accepted local modifications, but the present controls cannot establish an isolated causal effect of semantic matching. Importantly, matching grammar-derived types at the exchange site is a local constraint: changing the derivation can alter how subsequent codons are consumed and interpreted, so global semantic equivalence or validity of the entire downstream derivation is not guaranteed solely by the local match. These limitations motivate interpreting the crossover mechanisms as interacting design choices rather than as independently validated sources of performance improvement.

Mechanistic interpretation of local semantic compatibility.

An important limitation of grammar-guided crossover arises from the sequential nature of genotype-to-phenotype mapping in Grammatical Evolution. Although the Targeted operator attempts to identify compatible crossover positions based on grammar-derived semantic information, such compatibility is inherently local and does not guarantee preservation of the overall derivation structure.

For example, consider a grammar containing the productions <expr> ::= <expr> <op> <expr> | <var> | <func>(<expr>). Replacing a codon that selects <var> with one selecting <expr> <op> <expr> introduces additional non-terminal symbols into the derivation. As a result, subsequent codons may be consumed in different grammatical contexts, potentially selecting different production rules and producing substantially different phenotypes.

This phenomenon implies that exchanging locally compatible codon blocks does not necessarily preserve the semantic relationships of the complete generated expression. The resulting disruption may offset the expected benefits of semantic guidance and provides a plausible explanation for the absence of consistent improvements in classification performance. Importantly, this interpretation is a mechanistic hypothesis supported by the properties of GE mapping rather than a directly measured causal effect in the present experiments.

4.7. Convergence and Population Diversity Analysis

To further investigate the evolutionary behavior of the proposed crossover operators, a convergence and population diversity analysis was conducted on four representative classification datasets: Dermatology, Ionosphere, Spambase, and Vehicle. These datasets include binary and multiclass classification problems, allowing the crossover strategies to be examined under different chromosome structures. Each configuration was evaluated over 30 independent runs using a fixed budget of 100,000 fitness evaluations.

Population diversity was quantified using the normalized average pairwise Hamming distance between chromosomes. This measure expresses the proportion of differing codon positions across pairs of individuals in the population. Higher values indicate greater genotypic diversity, whereas lower values indicate increasing population homogeneity. The convergence curves show the mean best-so-far fitness, and the diversity curves show the mean population diversity. Shaded regions indicate one standard deviation around the mean.

Figure 9 presents the convergence and diversity trajectories for all four datasets. The left column shows best-so-far fitness against the number of fitness evaluations; the right column shows normalized population diversity. Since the fixed evaluation budget may interrupt the final generation, the diversity curves are restricted to evaluation ranges with observations from all 30 runs for each operator, avoiding comparisons based on unequal sample counts.

Figure 9. Convergence and population diversity of the Standard, Targeted, and TargetedWorst crossover operators on Dermatology, Ionosphere, Spambase, and Vehicle. The left column presents mean best-so-far fitness, while the right column presents mean normalized average pairwise Hamming distance. Results are aggregated over 30 independent runs per operator. Shaded regions represent the mean ± one standard deviation. Diversity curves are restricted to evaluation ranges with complete observations across all runs, whereas convergence curves extend to the fixed budget of 100,000 fitness evaluations.

On Dermatology, TargetedWorst exhibits relatively rapid initial fitness improvements. Both targeted variants maintain greater population diversity than Standard over substantial portions of the search. At 60,000 fitness evaluations, the diversity differences between Standard and Targeted and between Standard and TargetedWorst are statistically significant (Holm-adjusted Wilcoxon $p=3.18\times {10}^{-5}$ and $p=9.94\times {10}^{-5}$, respectively). No statistically significant diversity difference was observed between the two targeted variants. Although TargetedWorst obtained the best mean final fitness in this experiment, pairwise tests did not identify statistically significant differences in final fitness.

On Ionosphere, Targeted achieves significantly better final fitness than Standard. The recorded convergence and diversity trajectories of Targeted and TargetedWorst coincide. On Spambase, the targeted variants maintain greater population diversity than Standard over substantial portions of the search, but the final-fitness difference between Standard and Targeted is not statistically significant. The two targeted variants again exhibit identical recorded trajectories.

Vehicle provides a multiclass case in which the targeted variants generally maintain greater genotypic diversity than Standard, while the observed final-fitness difference between Standard and Targeted is not statistically significant. Thus, increased diversity does not necessarily translate into better final optimization performance under the available evaluation budget.

An implementation-specific characteristic explains the identical behavior of Targeted and TargetedWorst observed on the binary datasets. For a problem with C classes, the chromosome contains $C-1$ explicit classification rules, with the remaining class represented by the default branch. TargetedWorst restricts crossover positions to the segment corresponding to the worst-performing class, except when that class is the default class, in which case the entire chromosome is considered. When $C=2$, the single explicit-rule segment already spans the entire chromosome. Therefore, the class-specific restriction is ineffective for either possible worst-performing class, and TargetedWorst can select the same candidate crossover positions as Targeted. With otherwise equivalent selection and exchange operations and the same random seed, the operators can follow identical evolutionary trajectories, as observed on Ionosphere and Spambase.

Overall, Figure 9 indicates that the targeted crossover variants often preserve greater genotypic diversity, particularly on Dermatology, Spambase, and Vehicle, without consistently producing statistically significant improvements in final fitness. The effects are dataset-dependent. Furthermore, higher genotypic diversity should not be interpreted as direct evidence of improved generalization performance or a causal explanation for differences in classification accuracy.

4.8. Interpretability and Program Complexity

Interpretability was evaluated through the complexity of the final, expressed classification programs. Four complementary measures were considered:

• Program length in lexical tokens.
• The number of distinct input features referenced.
• The number of relational comparisons.
• The number of conditional statements.

These measures characterize different aspects of the readability of the generated rule systems: a program may be syntactically long without introducing additional decision branches or input features. We analyzed the final logged programs of the Standard, Targeted, and TargetedWorst crossover variants on ten datasets. For each dataset–operator combination, up to 30 independently seeded runs were available. In total, 898 programs were successfully extracted: 298 for Standard and 300 for each of Targeted and TargetedWorst. The Standard results include 29 runs forDermatology and 29 for Yeast; the remaining combinations contain 30 runs. The measurements describe the final logged best programs, not the mean complexity of the evolving populations. This information regarding rule complexity is outlined in Table 11.

Table 11. Complexity of final evolved programs. Values are mean ± standard deviation across independent runs. Program length is measured in lexical tokens; F is the number of distinct features, C the number of relational comparisons, and I the number of conditional statements.

Dataset	Method	n	Tokens	F	C	I
Dermatology	Standard	29	105.76 ± 18.32	12.79 ± 3.02	8.45 ± 1.97	5.00
Dermatology	Targeted	30	94.47 ± 17.88	11.83 ± 2.29	7.77 ± 1.65	5.00
Dermatology	TargetedWorst	30	96.43 ± 18.44	12.30 ± 2.09	7.73 ± 1.55	5.00
Hepatitis	Standard	30	56.03 ± 28.71	7.33 ± 2.68	3.80 ± 1.83	1.00
Hepatitis	Targeted	30	55.90 ± 30.52	7.87 ± 2.42	4.17 ± 1.84	1.00
Hepatitis	TargetedWorst	30	55.90 ± 30.52	7.87 ± 2.42	4.17 ± 1.84	1.00
Housevotes	Standard	30	44.57 ± 23.25	4.50 ± 2.69	2.63 ± 1.65	1.00
Housevotes	Targeted	30	35.57 ± 13.52	4.80 ± 2.01	2.60 ± 1.59	1.00
Housevotes	TargetedWorst	30	35.57 ± 13.52	4.80 ± 2.01	2.60 ± 1.59	1.00
Ionosphere	Standard	30	54.67 ± 28.10	6.60 ± 3.04	3.60 ± 2.04	1.00
Ionosphere	Targeted	30	51.90 ± 17.00	6.43 ± 2.06	3.70 ± 1.42	1.00
Ionosphere	TargetedWorst	30	51.90 ± 17.00	6.43 ± 2.06	3.70 ± 1.42	1.00
Optdigits	Standard	30	162.23 ± 18.19	20.93 ± 3.38	13.13 ± 1.91	9.00
Optdigits	Targeted	30	147.13 ± 12.33	18.90 ± 2.68	12.03 ± 1.59	9.00
Optdigits	TargetedWorst	30	144.37 ± 16.25	19.30 ± 2.69	12.13 ± 1.43	9.00
Segment	Standard	30	132.57 ± 19.12	12.00 ± 1.78	9.40 ± 1.89	6.00
Segment	Targeted	30	117.73 ± 13.16	10.87 ± 1.46	8.70 ± 1.47	6.00
Segment	TargetedWorst	30	119.27 ± 12.27	11.53 ± 2.05	8.90 ± 1.49	6.00
Spambase	Standard	30	72.30 ± 33.92	10.33 ± 3.92	6.00 ± 2.82	1.00
Spambase	Targeted	30	72.77 ± 28.17	11.17 ± 3.28	6.87 ± 2.56	1.00
Spambase	TargetedWorst	30	72.77 ± 28.17	11.17 ± 3.28	6.87 ± 2.56	1.00
Thyroid	Standard	30	79.27 ± 26.96	7.67 ± 2.51	4.63 ± 1.83	2.00
Thyroid	Targeted	30	70.10 ± 22.42	7.60 ± 2.57	5.00 ± 2.00	2.00
Thyroid	TargetedWorst	30	75.67 ± 23.03	8.33 ± 1.75	5.47 ± 1.53	2.00
Vehicle	Standard	30	115.63 ± 24.02	10.77 ± 2.61	6.87 ± 2.71	3.00
Vehicle	Targeted	30	94.93 ± 22.02	9.90 ± 2.50	5.60 ± 2.18	3.00
Vehicle	TargetedWorst	30	91.03 ± 17.33	9.87 ± 1.91	5.60 ± 1.96	3.00
Yeast	Standard	29	159.24 ± 14.86	7.83 ± 0.47	11.38 ± 1.86	9.00
Yeast	Targeted	30	156.80 ± 14.26	7.67 ± 0.48	10.87 ± 1.59	9.00
Yeast	TargetedWorst	30	154.47 ± 14.30	7.87 ± 0.35	10.73 ± 1.17	9.00

To assess paired differences in final program length, runs were matched by dataset and random seed and compared using the Wilcoxon signed-rank test. The Holm procedure was applied to the family of 20 length comparisons (Standard versus Targeted and Standard versus TargetedWorst for each of the ten datasets). After correction, Standard generated significantly longer programs on Optdigits, Segment, and Vehicle relative to both alternatives (Table 12). No comparison of the number of distinct features or relational comparisons remained significant after the corresponding Holm corrections. The number of conditional statements was identical across the three variants within each dataset in these extracted results. Thus, longer expressions did not necessarily entail more decision branches.

Table 12. Holm-adjusted Wilcoxon p-values for the significant paired comparisons of final program length. The positive mean differences indicate longer Standard programs.

Dataset	Comparison	Mean Difference	Adjusted p
Optdigits	Standard–Targeted	15.10	0.03490
Optdigits	Standard–TargetedWorst	17.87	0.00654
Segment	Standard–Targeted	14.83	0.01714
Segment	Standard–TargetedWorst	13.30	0.01163
Vehicle	Standard–Targeted	20.70	0.00987
Vehicle	Standard–TargetedWorst	24.60	0.00462

• Examples of evolved rules.

The following examples are taken verbatim from the final logged programsfor the Ionosphere dataset (seed 1). Both runs used a strict budget of 100,000 fitness evaluations. Standard reached a training classification error of 8.23\% and a test error of 17.14%, whereas Targeted reached 5.06% and 11.43%, respectively. The TargetedWorst program was identical to the Targeted program for this seed. These examples illustrate the form and complexity of the learned expressions, but should not be interpreted as a representative performance comparison from a single seed.

• Standard:
• if(!(x1<=cos((sin(x33)*x17))&x18>((-8.55)+(-1.2))|x2*x23>=x7&x9>x24|x5<sin(x20)&!(x19<cos(((x3*(x27*sin(x29)))-(-9.40)))))) CLASS=0.00
• else
• CLASS=1.00

• Targeted (also TargetedWorst):
• if(!(x1<=cos((sin(x33)*x9))|!(!(x5<=(x21*x16)|!(x16>=sin((-89.27))&!(x29<=cos((-9.0)))))))) CLASS=0.00
• else
• CLASS=1.00

• Program growth and bloat.

Additional illustrative evolutionary traces were examined for the original GenClass baseline and Standard on Shuttle and Pirvision over 2000 generations. In these individual runs, Standard yielded longer final programs, but program length fluctuated and sometimes decreased during optimization rather than increasing monotonically. Importantly, these traces are not controlled equal-evaluation comparisons: Standard used 6,335,527 versus 1,783,945 fitness evaluations on Shuttle and 6,776,246 versus 2,072,441 on Pirvision. Moreover, the recorded lengths refer to reported best programs, not population-wide mean genome or phenotype sizes. Accordingly, the evidence supports the narrower finding that some greedy crossover variants increase final expressed rule length; it does not establish the presence or absence of population-level genetic programming bloat. A definitive bloat study would require matched fitness-evaluation budgets, multiple independent baseline runs, and population-level complexity measurements throughout evolution.

4.9. Influence of Dataset Characteristics on Classification Performance

To investigate whether the benefit of the proposed crossover mechanism depends on dataset characteristics, we examined all 31 datasets listed in Table 1. For each dataset, the improvement was defined as $\Delta E=E_{\mathrm{CLASS}}-E_{\mathrm{STANDARD}}$, where E denotes the mean test classification error (in percentage points) reported in Table 3. Thus, positive values indicate lower error for STANDARD. Spearman’s rank correlation was used to assess associations between $\Delta E$ and the numbers of patterns, input features, and classes. Each dataset constituted one observation ($N=31$); repeated runs were not treated as independent dataset-level observations.

The association between improvement and dataset size was weak ($\rho =-0.087$, $p=0.643$), as was the association with the number of features ($\rho =0.071$, $p=0.706$). The number of classes showed a positive association ($\rho =0.395$, nominal $p=0.0277$), suggesting that larger improvements may occur in problems with more classes. However, this association did not remain statistically significant after Holm correction across the three correlations. The result should therefore be considered exploratory rather than evidence of a robust dependence on class count.

We also separated the datasets into binary ($N=9$) and multiclass ($N=22$) problems. STANDARD reduced the mean classification error relative to CLASS on 8 of the 9 binary datasets and 20 of the 22 multiclass datasets. The average improvements were 1.72 and 3.12 percentage points, respectively. Nevertheless, a two-sided Mann–Whitney U test did not identify a statistically significant difference in dataset-level improvements between these groups ($p=0.267$).

The previously mentioned results do not reveal a clear monotonic relationship between the advantage of STANDARD and the number of patterns or features. Although the class-count analysis suggests a potentially interesting trend, the available dataset-level evidence is insufficient to establish that the proposed crossover mechanism systematically benefits more from a larger number of classes. Moreover, these observational comparisons do not isolate dataset size or class count from other differences in problem difficulty and structure.

4.10. Analysis of Accepted Crossover Improvements

To quantify the behavior of the six crossover variants, we recorded the number of accepted crossover improvements at each generation in 30 independent runs on the Dermatology dataset. All runs used the same limit of 100,000 fitness evaluations. An accepted improvement denotes an improvement accepted by the crossover mechanism; it does not necessarily indicate a new global-best individual. Table 13 reports the mean and sample standard deviation across runs. The per-generation measure was computed for each run by dividing its total number of accepted improvements by its number of completed generations, before aggregating across runs.

Table 13. Accepted crossover improvements on the Dermatology dataset (mean ± sample standard deviation over 30 independent runs). The last column normalizes by the number of crossover evaluations, not by the total fitness-evaluation budget.

Crossover Variant	Accepted/Run	Accepted/Generation	Accepted/1000 Cross. Evals
Standard	$968.40\pm 192.93$	$24.46\pm 4.76$	$12.26\pm 2.41$
Standard + Elite	$817.57\pm 196.46$	$20.62\pm 4.78$	$10.34\pm 2.43$
Standard + Block	$887.27\pm 144.49$	$22.47\pm 3.55$	$11.24\pm 1.80$
Targeted	$555.90\pm 85.48$	$5.62\pm 0.86$	$27.19\pm 4.30$
Targeted + Full	$558.37\pm 75.82$	$5.63\pm 0.77$	$27.23\pm 3.90$
TargetedWorst	$294.40\pm 84.40$	$2.97\pm 0.85$	$15.19\pm 4.23$

Standard produced the largest number of accepted improvements per generation ($24.46\pm 4.76$), followed by Standard + Block ($22.47\pm 3.55$) and Standard + Elite ($20.62\pm 4.78$). The targeted variants produced fewer accepted improvements per generation, partly because they performed fewer crossover evaluations within each generation. Accordingly, we also report accepted improvements per 1000 crossover evaluations. By this measure, Targeted and Targeted + Full achieved approximately 27.2 accepted improvements, compared with 12.3 for Standard. This difference concerns acceptance frequency, not necessarily the magnitude of improvement or final classification accuracy.

To examine how acceptance evolves during the search, we divided the common 100,000-fitness-evaluation budget into ten intervals. For each run, the accepted improvements and crossover evaluations of a generation were assigned to the interval containing that generation’s ending cumulative fitness-evaluation count. Figure 10 shows the resulting mean counts. The Standard variants generally exhibit a reduction in the number of accepted improvements from the early to the later intervals. The targeted variants exhibit more irregular bin counts.

Figure 10. Mean number of accepted crossover improvements in successive 10,000-fitness-evaluation intervals on Dermatology (30 runs per variant). Whole generations are assigned to the interval in which they finish; therefore, short-term fluctuations may reflect bin boundaries rather than changes in search behavior.

Figure 11 accounts for the different numbers of crossover evaluations in each interval. The Standard acceptance frequency falls from approximately 17.3 to 10.4 accepted improvements per 1000 crossover evaluations between the first and last intervals. In contrast, Targeted and Targeted + Full remain at approximately 25–29 accepted improvements per 1000 crossover evaluations across most intervals. Thus, although the targeted variants accept fewer improvements per generation, they exhibit a higher acceptance frequency per crossover evaluation.

Figure 11. Accepted crossover improvements per 1000 crossover evaluations across the fitness-evaluation budget. Each point is the ratio of the mean accepted improvements to the mean crossover evaluations within the corresponding interval (30 runs per variant). The binning uses generation endpoints.

These findings describe the acceptance dynamics on Dermatology and should not be interpreted as evidence of superiority across all datasets. In particular, acceptance frequency alone does not establish whether a method yields better final classification performance.

5. Conclusions

This work introduced an additional crossover procedure into the GenClass method. At each generation, after the conventional genetic operators have been applied, $N_K$ randomly selected chromosomes undergo k crossover trials. A candidate chromosome replaces the current chromosome only when it achieves a strictly lower training classification error. Three variants, denoted Standard, Targeted, and TargetedWorst, were evaluated on 31 classification datasets. Standard applies one-point crossover using donors selected from the population. It reduced the average classification error of the original method from 16.13% to 13.42%, corresponding to a relative reduction of approximately 16.8%, and obtained the lowest error on 19 of the 31 datasets among the eleven compared methods. Its average error was also lower than those of the additional reference classifiers, including CART (19.42%) and SVM (21.97%). The improvement over the original method remained statistically significant in the sensitivity analysis that excluded four related EEG-derived datasets and applied Holm correction for multiple comparisons.

The comparison of the three variants revealed that additional grammatical and class-specific guidance did not necessarily improve classification performance. Targeted and TargetedWorst select donor chromosomes from an elite fraction of the population and exchange bounded blocks of codons at semantically compatible positions. TargetedWorst further restricts the modified region according to the worst-performing class. Nevertheless, Standard achieved a lower average classification error than both targeted variants. The respective mean errors were 13.42% for Standard, 15.08% for Targeted, and 15.42% for TargetedWorst, compared with 16.13% for the original GenClass method. In the reported pairwise comparisons, Standard outperformed both targeted variants, whereas no statistically significant difference was established between Targeted and TargetedWorst. Comparisons against the original method indicated that Standard provided the most robust improvement under the more conservative sensitivity analysis; the evidence supporting the targeted variants was less conclusive.

A possible explanation for these findings concerns the different recombination mechanisms. Standard selects donors from the entire population and exchanges chromosome segments following randomly selected crossover points, whereas the targeted variants select donors from the best 20% of the population and transfer blocks of at most $B_{\max }$ codons. The additional convergence and population diversity experiments described in Section 4.7, conducted on Dermatology, Ionosphere, Spambase, and Vehicle, showed that the targeted variants often maintained greater genotypic diversity than Standard, as measured by normalized average pairwise Hamming distance. However, this higher diversity did not consistently translate into better final fitness or lower classification error. Thus, the lower average classification error achieved by Standard in the present experiments cannot be explained simply by greater preservation of genotypic diversity. Donor selection, the extent of exchanged genetic material, and the balance between exploration and exploitation are plausible contributing factors, but the present results do not establish a causal relationship between these mechanisms and generalization performance. Donor selection, the extent of exchanged genetic material, and the balance between exploration and exploitation are plausible contributing factors, but the present results do not establish a causal relationship between these mechanisms and generalization performance.

The convergence experiments also clarified an implementation-specific property of the targeted operators on binary classification problems. When the number of classes is two, the single explicitly encoded rule occupies the entire chromosome; consequently, the class-based restriction of TargetedWorst does not narrow the set of eligible crossover positions relative to Targeted. This explains the identical trajectories observed for these operators on the binary datasets considered in the convergence analysis. Their behavior can differ on multiclass problems, where the chromosome contains separate rule- encoding segments.

The number of chromosomes subjected to additional crossover also influenced performance, particularly at its smallest tested value. The setting $N_K=2$ performed significantly worse than the larger settings, whereas differences among $N_K=5$, 10, and 15 were small and not statistically significant. Although $N_K=15$ yielded the lowest average error and the best mean rank, its advantage over the other moderate settings was not statistically established. Therefore, a moderate number of selected chromosomes appears sufficient under the examined experimental configuration.

Several limitations should be acknowledged. The additional procedure requires up to $N_{K}k$ candidate fitness evaluations per generation, corresponding to 200 evaluations for the parameter configuration used in the experiments, while the conventional genetic operators generate $p_s{N}_c=450$ offspring per generation. The parameters k, $\rho$, and $B_{\max }$ were kept fixed, and their individual effects were not systematically examined. The conclusions are therefore restricted to the evaluated datasets, parameter settings, and classification tasks. Moreover, four EEG-derived benchmark problems originate from related recordings and cannot be treated as fully independent observations. Although a sensitivity analysis excluding these problems was performed, the remaining datasets do not necessarily represent a random sample of all classification tasks.

It should also be emphasized that the objective of this study is to assess the effects of additional crossover strategies within the GenClass framework, rather than to establish GenClass as a universally superior classification method. The comparisons with other classifiers provide supplementary performance references and should not be interpreted as a comprehensive benchmark of supervised learning algorithms. Different classification methods may be more suitable for different datasets and application domains.

An additional limitation concerns the availability of run-level observations. For the main cross-validation experiments, the individual results of the 30 independent runs per fold were not retained, preventing the estimation of run-to-run standard deviations and within-dataset statistical tests from those experiments. In contrast, run-level observations were retained for the supplementary convergence and population diversity experiments reported in Section 4.7. Accordingly, small differences in average classification error should be interpreted cautiously. Furthermore, because the greedy acceptance criterion uses training classification error, the additional refinement may increase the risk of overfitting, particularly on small datasets such as Hepatitis and Appendicitis. The present diversity measurements describe genotypic variation and do not, by themselves, demonstrate improved generalization.

Future research should extend the convergence and population diversity analysis to more datasets and investigate the effects of the elite fraction $\rho$, maximum block size $B_{\max }$, and alternative donor-selection policies. Hybrid operators combining random and grammatically guided exchanges could also be examined. In addition, the number of selected chromosomes $N_K$ and the number of crossover trials k could be adapted dynamically during evolution; values of $N_K$ greater than 15 merit investigation because the largest tested setting achieved the best mean rank without a statistically significant advantage. Further studies should retain run-level results across the full cross-validation protocol, assess generalization and overfitting more directly, and report computational cost alongside predictive performance. Finally, parallel implementations and applications to larger datasets, regression tasks, and other Grammatical Evolution methods would help establish the broader applicability of the proposed crossover procedure.