Local Crossover: A New Genetic Operator for Grammatical Evolution

Abstract

The presented work outlines a new genetic crossover operator, which can be used to solve problems by the Grammatical Evolution technique. This new operator intensively applies the one-point crossover procedure to randomly selected chromosomes with the aim of drastically reducing their fitness value. The new operator is applied to chromosomes selected randomly from the genetic population. This new operator was applied to two techniques from the recent literature that exploit Grammatical Evolution: artificial neural network construction and rule construction. In both case studies, an extensive set of classification problems and data-fitting problems were incorporated to estimate the effectiveness of the proposed genetic operator. The proposed operator significantly reduced both the classification error on the classification datasets and the feature learning error on the fitting datasets compared to other machine learning techniques and also to the original models before applying the new operator.

1. Introduction

Genetic algorithms are stochastic optimization algorithms originated in the work of Holland [1]. They belong to a wide area of optimization algorithms called evolutionary techniques [2]. Genetic algorithms initiate by formulating candidate solutions of the objective problem. These solutions are evolved through a series of processes that mimic natural evolution, such as selection, crossover and mutation [3,4,5]. The genetic algorithms were incorporated in a variety of problems, such as networking problems [6], problems arising in robotics [7,8], energy problems [9,10], medicine problems [11,12], agriculture problems [13], etc.

Grammatical Evolution [14] is an integer-based genetic algorithm, where each chromosome represents a series of production rules derived from a Backus–Naur form (BNF) grammar [15]. Grammatical Evolution can be utilized to produce programs in the provided programming language. This method was used in a series of cases derived from real-world problems, such as data fitting [16,17], credit classification [18], detection of network attacks [19], solving differential equations [20], monitoring the quality of drinking water [21], construction of optimization methods [22], application in trigonometric problems [23], composition of music [24], constructing neural networks [25,26], computer games [27,28], problems regarding energy demand [29], combinatorial optimization [30], security topics [31], automatic construction of decision trees [32], circuit design [33], discovering taxonomies in Wikipedia [34], trading algorithms [35], bioinformatics [36], modeling glycemia in humans [37], etc.

Grammatical Evolution has been extended by many researchers in the recent bibliography. Among these extensions there are works, such as the Weighted Hierarchical Grammatical Evolution [38], which proposed a novel technique to map genotypes to phenotypes, or the method of Structured Grammatical Evolution [39,40]. Also, O’Neill et al. suggested the usage of shape grammars in the Grammatical Evolution for evolutionary design [41]. Also, the $\pi$Grammatical Evolution method [42] was suggested as a modification of the Grammatical Evolution, where a position-independent mapping was proposed. Another interesting work was the incorporation of Particle Swarm Optimization (PSO) [43] to create expressions in Grammatical Evolution. This method was named Grammatical Swarm [44,45] in the relevant literature. Moreover, the method of Probabilistic Grammatical Evolution [46] has been introduced recently, where a stochastic procedure was used as a mapping mechanism. Recently, the optimization method of the Fireworks algorithm [47] was applied as a learning algorithm for the Grammatical Evolution procedure [48]. Contreras et al. suggested the combination of Grammatical Evolution and some ideas from interval analysis to solve problems with uncertainty [49]. Grammatical evolution has also been extended using programming techniques [50,51] or Christiansen grammars [52].

Additionally, many researchers have developed and published open source software for Grammatical Evolution, such as the graphical user interface (GUI) application of Grammatical Evolution in Java (GEVA) [53], a Java implementation called jGE [54], an R implementation of Grammatical Evolution under the name gramEvol [55], the GRAPE software version 1.0 that implemented Grammatical Evolution in Python [56], the GeLab [57] software that suggested a Matlab toolbox for Grammatical Evolution, a software which produced classification programs with Grammatical Evolution called GenClass [58], the QFc software version 1.0 [59] was incorporated for feature construction, etc.

In this work, a new genetic operator for Grammatical Evolution is introduced, which is derived from the one crossover technique. The new genetic operator is stochastically applied to the genetic population, randomly selecting a set of chromosomes on which to apply it. For each randomly selected chromosome, a group of chromosomes is stochastically formed from the current genetic population. Afterwards, a one-point crossover operation is performed between the selected chromosome and each of the generated groups to search for a lower value of the fitness function. The new genetic operator was applied in two distinct cases of Grammatical Evolution methods: in the rule construction technique introduced recently [60] and in the neural network construction technique [61]. These machine learning tools were applied on some classification and regression datasets proposed in the recent literature, and the experimental results indicated a reduction in classification or regression error from the application of the new genetic operator.

The main components of the proposed technique are outlined below:

• The method can be applied as a genetic operator to all problems solved by the Grammatical Evolution technique, and the only information it exploits is the fitness function of the problem.
• The method has no dependence on the grammar of the objective problem.
• By using an application rate, the user can require fewer or more applications of the new operator.
• Although this operator requires significant computing time for its execution, its application between chromosomes can be made using parallel techniques, since there is no dependency between its successive applications.
• Simple linear operations are required for its implementation, such as crossing a point between chromosomes.
• The new genetic operator could theoretically be applied to other forms of genetic algorithms beyond Grammatical Evolution.

The following sections have this structure: in Section 2, the basic principles of Grammatical Evolution are discussed as well as the current genetic operator. In Section 3, the used datasets and the experimental results are presented, and in Section 4, some conclusions are discussed in detail.

2. Materials and Methods

2.1. Basics of the Grammatical Evolution Method

Grammatical Evolution considers chromosomes as a set of production rules in the provided BNF grammar. BNF grammar is usually denoted as a set $G=\left(N,T,S,P\right)$ with the following assumptions:

• N defines the set of non-terminal symbols.
• T is a set that contains the terminal symbols.
• S is a non-terminal symbol that corresponds to the start symbol of the grammar.
• P is the set of the rules used during production. These rules are in the form $A\to a$ or $A\to aB,A,B\in N,a\in T$. In the Grammatical Evolution procedure, a sequence number is assigned to every production rule.

The grammar used for the rule machine learning method is outlined in Figure 1 and the corresponding grammar for the neural network construction technique is displayed in Figure 2.

The notation < > is used to enclose non-terminal symbols. The numbers at the end of the production rules represent the sequence number of each rule. The parameter d is the dimension of the used dataset (number of features). Grammatical Evolution produces valid expressions by starting from the starting symbol S and by following the production rules. The method selects the following production rules according to the following scheme:

• The next element denoted as V from the chromosome to be processed is retrieved.
• The next production rule is selected according to the formula.

  $$\mathrm{Rule}=V\mathrm{mod}\mathrm{NR}$$

  (1)

  where the value NR represents the total number of production rules for the under processing non-terminal symbol.

For a dataset with three features $\left(x_1,x_2,x_3\right)$, the following possible expression could be created for the case of the rule method:

$$\mathrm{if}\left(x_1>2+sin\left(x_3\right)\right)\mathrm{value}=1+cos\left(x_2\right)\mathrm{else}\mathrm{value}=1+x_1$$

The terminal symbol value is used to denote the final outcome of the rule method. For the same case, the following constructed neural network could be created:

$$\mathrm{NNC}\left(x\right)=2.45\mathrm{sig}\left(1.9x_1+3.11x_3+2.5\right)+5.9\mathrm{sig}\left(10.8x_2+6.25\right)$$

This grammar of Figure 2 is able to produce artificial neural networks using this form:

$$N\left(\overrightarrow{x},\overrightarrow{p}\right)=\sum _{i=1}^H{p}_{(n+2)i-(n+1)}\sigma \left(\sum _{j=1}^n{x}_j{p}_{(n+2)i-(n+1)+j}+p_{(n+2)i}\right)$$

(2)

The vector $\overrightarrow{x}$ indicates the input vector and the vector $\overrightarrow{p}$ stands for the parameter set of the neural network. The integer constant H denotes the number of weights or processing units. These networks have one processing level, but the grammar could be easily extended to produce neural networks of additional levels. The function $\mathrm{sig}\left(x\right)$ used in the previous example denotes the sigmoid function given by

$$\mathrm{sig}\left(x\right)={\displaystyle \frac{1}{1+exp(-x)}}$$

(3)

used in the majority of cases of neural networks. Of course, the Grammatical Evolution procedure can construct neural networks with different activation functions or neural networks that use a mix of activation functions.

2.2. The Genetic Algorithm

The two machine learning techniques, in which the new genetic operator was applied, follow a series of similar execution steps, which are presented in detail below:

• Initialization step.

  (a)

  Set $k=0$ as the generation counter.

  (b)

  Denote with $N_g$ the generations allowed and with $N_c$ the amount of chromosomes in the genetic population.

  (c)

  Denote with $p_s$ the used selection rate $\left(p_s\le 1\right)$ and with $p_m$ the used mutation rate $\left(p_m\le 1\right)$.

  (d)

  Set as $p_{cr}$ the rate for the application of the new crossover operator, where $p_{cr}\le$ 1.

  (e)

  Set as $N_{cr}$ the amount of chromosomes that will be chosen for each chromosome where the new crossover operator will be applied.

• Fitness step.

  (a)

  For  $i=1,\dots ,N_c$  do

  • Set as $f_i$ the corresponding fitness value of chromosome i. For the case of the rule creation model, the grammar of Figure 1 is applied, while for the case of neural network construction, the grammar of Figure 2 is utilized.

  (b)

  EndFor

• Genetic operations.

  (a)

  Apply the selection procedure: In the first phase, a sorting is performed for the members of the genetic population according to the fitness of each chromosome. The $\left(1-p_s\right)\times N_c$ best of these are transmitted unchanged to the next generation, while the rest will be replaced by chromosomes produced through crossover and mutation.

  (b)

  Execute the crossover procedure: During this procedure, $p_s\times N_c$ offsprings are produced from the population under processing. For every set $\left(\tilde{z},\tilde{w}\right)$ of produced children, two distinct chromosomes $(z,w)$ are chosen from the current population with tournament selection. The offsprings $\left(\tilde{z},\tilde{w}\right)$ are formulated using the one-point crossover procedure, which is graphically outlined in Figure 3.

  (c)

  Perform the mutation procedure. During this procedure, a random number $r\in [0,1]$ is selected for each element of every chromosome, and this element is changed randomly if $r\le p_m$.

  (d)

  Apply the new crossover operator: For each chromosome $g_i,i=1,\dots ,N_c$, a random number $r\in [0,1]$ is selected. If $r\le p_{cr}$, then we execute the procedure described in Section 2.3 on $g_i$.

• Termination step.

  (a)

  Set $k=k+1$.

  (b)

  If $k\le N_g$ go to Step 2, else terminate.

The previous algorithm is also graphically illustrated in the flowchart of Figure 4.

2.3. The New Crossover Operator

The new crossover operator executes the one-point crossover method on a selected chromosome using a set of chromosomes that was selected randomly from the current population. The steps of the proposed operator are outlined below:

• Set as g the chromosome where the operator will be applied and as $f_g$ the corresponding fitness value.
• Create the set $C=\left\{x_1,x_2,\dots ,x_{N_{cr}}\right\}$ of $N_{cr}$ randomly selected chromosomes.
• For  $i=1,\dots ,N_{cr}$  do

  (a)

  Perform one-point crossover between g and $x_i$. This procedure produces the offsprings $g_1$ and $g_2$ with associated fitness values $f_{g_1}$ and $f_{g_2}$.

  (b)

  If  $f_{g_1}\le f_g$  then

  • $g=g_1$

  (c)

  else if  $f_{g_2}\le f_g$  then

  • $g=g_2$

  (d)

  Endif

• EndFor

3. Results

The suggested genetic operator was tested on a set of classification and regression datasets obtained from the recent bibliography and relevant websites. The internet sources for these datasets are the following websites:

• The UCI dataset repository, https://archive.ics.uci.edu/ml/index.php (accessed on 10 October 2024) [62];
• The Keel repository, https://sci2s.ugr.es/keel/datasets.php (accessed on 10 October 2024) [63];
• The Statlib URL http://lib.stat.cmu.edu/datasets/ (accessed on 10 October 2024).

3.1. The Used Classification Datasets

A wide series of datasets presenting classification problems is used here:

• Appendictis proposed in [64].
• Australian dataset proposed in [65].
• Balance dataset [66] used in a series of psychological experiments.
• Circular dataset, which is a dataset produced artificially.
• Cleveland dataset [67,68].
• Dermatology dataset [69], which is related to dermatological deceases.
• Ecoli dataset, which is related to protein problems [70].
• Fert dataset for the detection of possible relations between fertility and sperm concentration.
• Haberman dataset, which is used for breast cancer detection.
• Hayes roth dataset [71].
• Heart dataset [72], which is proposed for the detection of heart diseases.
• HeartAttack dataset, which is a medical dataset related to heart diseases.
• HouseVotes dataset [73], which contains the votes in the U.S. House of Representatives for various cases.
• Glass dataset, which contains glass component analysis for glass pieces that belong to six classes.
• Liverdisorder dataset [74], which is a medical dataset for the detection of liver disorders.
• Mammographic dataset [75], which is used in breast cancer detection.
• Parkinsons dataset, which is proposed in [76].
• Pima dataset [77], which is used in the detection of diabetes.
• Popfailures dataset [78], which is to do with climate-related measurements.
• Regions2 dataset, which is proposed in [79].
• Saheart dataset [80], which is proposed to detect heart diseases.
• Segment dataset [81], which contains information regarding image processing.
• Spiral dataset, which is a dataset created artificially.
• Student dataset [82], which is a dataset for measurements in schools.
• Wdbc dataset [83], which is used in cancer detection.
• Wine dataset, which contains information about wines [84,85].
• Eeg datasets, which is a medical dataset that contains measurements from various experiments regarding EEG [86]. The following subdatasets were extracted from this dataset: Z_F_S, Z_O_N_F_S, ZO_NF_S and ZONF_S.
• Zoo dataset [87], which is proposed to estimate the category of some animals.

3.2. The Used Regression Datasets

A series of datasets from this category was utilized in the current work:

• Abalone dataset [88].
• Airfoil dataset, which is provided by NASA [89].
• BK dataset [90], which is related to the prediction of points in a basketball game.
• BL dataset, which is related to calculations from electricity experiments.
• Baseball dataset, which is used to estimate the income of baseball players.
• Concrete dataset [91], which is a dataset related to the durability of cements in public works.
• Dee dataset, which is related to the prices of electricity.
• FY, which is a dataset that contains measurements for the longevity of fruit flies.
• HO dataset, which is provided from the the STALIB repository.
• Housing dataset [92].
• Laser dataset, which contains measurements from laser experiments.
• LW dataset, which contains measurements regarding the weight of babies.
• MORTGAGE, which contains economic measurements from USA.
• MUNDIAL, which is used from the STALIB repository.
• PL dataset, which is used from the STALIB repository.
• QUAKE dataset, which is used in measurements from earthquakes.
• REALESTATE, which is from the STALIB repository.
• SN dataset, which is used in an experiment related to trellising and pruning.
• Treasury dataset, which is a dataset regarding the economy of USA.
• TZ dataset, which is founded in the STALIB repository.
• VE dataset, which is from the STALIB repository.

3.3. Experimental Results

ANSI C++ was used to code the software for the conducted experiments using also the Optimus optimization environment, which is available from https://github.com/itsoulos/GlobalOptimus/ (accessed on 10 October 2024). The experiments were executed 30 times for each case, and the random number generator was initialized using different seeds in every execution. For the case of classification problems, the average classification error as measured on the test set was recorded and, for the case of regression datasets, the average regression error as measured on the test set was recorded. The validation of experiments was carried out using the method of ten-fold cross-validation.

Table 1. This table presents the values used for the experimental parameters, which are used in the conducted experiments.

Parameter	Meaning	Value
$N_c$	Chromosomes used	500
$N_g$	Maximum number of generations	200
$p_s$	Crossover rate	0.10
$p_m$	Mutation rate	0.05
$p_{cr}$	New crossover rate	0.05
$N_{cr}$	New crossover items	100
H	Weights for the neural network	10

depicts the experimental settings for the performed experiments.

Table 2. This table shows the average classification error as calculated on the corresponding test set for the classification datasets using all machine learning models.

DATASET	BFGS	GEN	RULE	NNC	RULE_CROSS	NNC_CROSS
APPENDICITIS	18.00%	24.40%	14.70%	13.70%	14.80%	14.40%
AUSTRALIAN	38.13%	36.64%	14.27%	14.51%	14.46%	14.71%
BALANCE	8.64%	8.36%	28.79%	22.11%	17.47%	14.32%
CIRCULAR	6.08%	5.13%	13.25%	13.64%	9.12%	7.49%
CLEVELAND	77.55%	57.21%	48.24%	50.10%	47.52%	49.21%
DERMATOLOGY	52.92%	16.60%	43.77%	25.06%	38.00%	12.92%
ECOLI	69.52%	54.67%	55.18%	47.82%	53.48%	49.15%
FERT	23.20%	28.50%	17.40%	19.00%	17.50%	19.20%
HABERMAN	33.10%	27.80%	27.03%	28.03%	26.53%	28.37%
HAYES-ROTH	56.54%	35.85%	39.39%	35.93%	38.08%	24.08%
HEART	39.44%	26.41%	20.30%	15.78%	19.41%	15.33%
HEARTATTACK	46.67%	29.03%	23.63%	19.33%	23.70%	18.73%
HOUSEVOTES	7.13%	7.00%	3.48%	3.65%	4.51%	3.22%
GLASS	69.95%	55.09%	58.10%	57.10%	54.81%	53.82%
IONOSPHERE	13.37%	18.03%	15.06%	11.12%	14.14%	9.25%
LIVERDISORDER	42.59%	37.09%	37.09%	33.71%	35.68%	31.24%
MAMMOGRAPHIC	29.54%	16.33%	19.00%	17.78%	18.10%	17.12%
PARKINSONS	27.58%	16.58%	13.47%	12.21%	13.37%	11.47%
PIMA	35.59%	34.21%	27.85%	27.99%	27.30%	25.95%
POPFAILURES	5.24%	4.17%	5.44%	6.74%	5.02%	6.41%
REGIONS2	36.28%	33.53%	29.13%	25.52%	29.26%	24.46%
SAHEART	37.48%	34.85%	30.20%	30.52%	31.00%	28.64%
SEGMENT	68.97%	46.30%	71.51%	54.99%	61.99%	35.82%
SPIRAL	47.99%	47.67%	50.06%	48.39%	49.08%	48.04%
STUDENT	4.90%	6.75%	11.08%	5.78%	7.23%	5.06%
TRANSFUSION	25.59%	24.01%	25.19%	25.34%	24.46%	24.44%
WDBC	29.91%	7.87%	7.66%	6.95%	6.43%	6.48%
WINE	59.71%	22.88%	15.35%	14.35%	12.47%	9.88%
Z_F_S	39.37%	24.60%	16.40%	14.17%	8.77%	10.23%
Z_O_N_F_S	79.04%	64.26%	53.64%	49.18%	44.60%	42.30%
ZO_NF_S	43.04%	21.54%	14.10%	14.14%	8.39%	9.12%
ZONF_S	15.62%	4.36%	2.76%	3.14%	2.06%	2.70%
ZOO	12.10%	10.20%	14.80%	9.20%	11.10%	5.70%
AVERAGE	36.39%	26.91%	26.28%	23.54%	23.93%	20.58%

outlines the results for the performed experiments on the classification datasets while

Table 3. This table shows the experimental results when the regression datasets were used and the numbers stand for the average regression error as measured on the corresponding test set.

DATASET	BFGS	GEN	RULE	NNC	RULE_CROSS	NNC_CROSS
ABALONE	6.38	7.17	7.36	5.05	5.32	4.63
AIRFOIL	0.003	0.001	0.003	0.003	0.002	0.002
BK	0.36	0.26	0.02	2.32	0.037	0.15
BL	1.09	2.23	2.53	0.021	0.023	0.40
BASEBALL	119.63	64.60	65.64	59.85	61.35	58.75
CONCRETE	0.023	0.001	0.013	0.008	0.009	0.005
DEE	2.36	0.47	0.43	0.26	0.32	0.23
FY	0.19	0.65	0.041	0.058	0.046	0.049
HO	0.62	0.37	0.019	0.017	0.019	0.014
HOUSING	97.38	35.97	47.99	26.35	26.74	19.10
LASER	0.03	0.084	0.055	0.024	0.032	0.019
LW	0.26	0.54	0.012	0.011	0.013	0.017
MORTGAGE	8.23	0.40	0.20	0.30	0.13	0.21
MUNDIAL	0.05	1.22	0.038	4.47	0.049	0.76
PL	0.11	0.03	0.056	0.045	0.035	0.036
QUAKE	0.09	0.12	1.13	0.045	0.73	0.046
REALESTATE	128.94	81.19	104.74	76.78	92.49	69.77
SN	0.16	0.20	0.025	0.026	0.026	0.024
TREASURY	9.91	0.44	0.15	0.47	0.12	0.30
TZ	0.21	0.097	0.036	5.04	0.035	0.061
VE	1.92	2.43	0.028	6.61	0.043	0.084
AVERAGE	17.99	9.45	10.98	8.94	8.93	7.35

holds the results for the mentioned regression datasets. The following notations are used in the experimental tables:

• The column BFGS represents the incorporation of the BFGS method [93] to train an artificial neural network with H hidden nodes.
• The column GEN outlines the experimental results using a genetic algorithm [94] to train an artificial neural network with H hidden nodes. The values for the critical parameters of the genetic algorithm are included in Table 1.
• The column RULE refers to the simple rule construction method [60] without the incorporation of the new crossover operator.
• The column NNC refers to the method of neural network construction [61]. This method was applied without the new operator.
• The column RULE_CROSS represents the application of the new crossover operator and the rule construction machine learning model.
• The column NNC_CROSS depicts the results for the neural construction method with the assistance of the new genetic operator.
• The average error is depicted in the row under the name AVERAGE.

There is a clear indication from the experimental results that there is a significant reduction in the mean error using the new genetic operator in both machine learning models. In a wide range of datasets, the proposed technique drastically reduces the error of data classification or fitting, as it is also represented in Figure 5 and Figure 6. These graphs show the number of datasets in which the application of the new genetic operator resulted in a drastic reduction in the corresponding error.

Furthermore, the box plots for the classification cases are shown in Figure 7 and Figure 8 for the rule construction model and the network construction model, respectively.

Box plots for the same comparisons as deduced from the results using the series of regression datasets are shown in Figure 9 and Figure 10, respectively.

These figures confirm the significant improvement brought about by the use of the new operator in the effectiveness of the two techniques that utilize Grammatical Evolution. This improvement appears to be greater in the datasets used in data fitting.

Also, a statistical comparison was performed using the two machine learning methods and the enhanced ones that use the new crossover operator. This comparison was performed using the classification datasets, and it is shown in Figure 11.

Moreover, an additional test was executed in order to estimate the effectiveness of the new crossover rate parameter denoted as $p_{cr}$. In this experiment, the rule construction machine learning model was applied on the classification datasets using a series of values for $p_{cr}$, and the results are shown in

Table 4. The effect of a series of values of $p_{cr}$ to the rule model with application on the classification datasets.

DATASET	RULE	${\mathit{p}}_{\mathit{cr}}=0.025$	${\mathit{p}}_{\mathit{cr}}=0.05$	${\mathit{p}}_{\mathit{cr}}=0.075$
APPENDICITIS	14.70%	15.80%	14.80%	15.10%
AUSTRALIAN	14.27%	13.96%	14.46%	14.03%
BALANCE	28.79%	20.18%	17.47%	18.07%
CIRCULAR	13.25%	11.00%	9.12%	9.78%
CLEVELAND	48.24%	48.24%	47.52%	46.07%
DERMATOLOGY	43.77%	38.60%	38.00%	36.00%
ECOLI	55.18%	52.49%	53.48%	48.83%
FERT	17.40%	16.70%	17.50%	18.50%
HABERMAN	27.03%	27.57%	26.53%	26.87%
HAYES-ROTH	39.39%	35.69%	38.08%	36.77%
HEART	20.30%	20.48%	19.41%	20.37%
HEARTATTACK	23.63%	22.83%	23.70%	22.53%
HOUSEVOTES	3.48%	3.48%	4.51%	3.13%
GLASS	58.10%	55.62%	54.81%	52.76%
IONOSPHERE	15.06%	15.14%	14.14%	14.14%
LIVERDISORDER	37.09%	34.79%	35.68%	33.50%
MAMMOGRAPHIC	19.00%	18.34%	18.10%	17.90%
PARKINSONS	13.47%	13.95%	13.37%	13.21%
PIMA	27.85%	27.80%	27.30%	27.84%
POPFAILURES	5.44%	5.33%	5.02%	5.32%
REGIONS2	29.13%	28.82%	29.26%	28.00%
SAHEART	30.20%	30.00%	31.00%	30.18%
SEGMENT	71.51%	67.36%	61.99%	63.91%
SPIRAL	50.06%	50.42%	49.08%	49.60%
STUDENT	11.08%	7.50%	7.23%	6.07%
TRANSFUSION	25.19%	24.20%	24.46%	24.68%
WDBC	7.66%	5.79%	6.43%	6.41%
WINE	15.35%	15.47%	12.47%	13.59%
Z_F_S	16.40%	11.63%	8.77%	9.10%
Z_O_N_F_S	53.64%	47.14%	44.60%	44.04%
ZO_NF_S	14.10%	10.50%	8.39%	8.42%
ZONF_S	2.76%	2.64%	2.06%	2.14%
ZOO	14.80%	11.30%	11.10%	8.70%
AVERAGE	26.28%	24.57%	23.93%	23.50%

.

Looking at the table of results, one can see a significant decrease in the average classification error when the application rate of the genetic operator increases from 2.5% to 5%. However, the rate of reduction of the average error decreases significantly when the application rate increases to 7.5%. This finding reinforces the idea of implementing the new genetic operator at a rate of 5%.

Another experiment was performed in order to estimate the importance of the parameter $N_{cr}$, which controls the number of chromosomes participating in the new crossover operator. In this experiment, the neural network construction method was tested on the classification datasets using a series of values for the parameter $N_{cr}$, while the parameter $p_{cr}$ was fixed to 2.5%. The outcomes obtained from this experiment are depicted in

Table 5. The effects of the parameter $N_{cr}$ to the NNC machine learning model. The classification datasets were used in this experiment. In all experiments, the value $p_{cr}$ was set to 0.025.

DATASET	NNC	${\mathit{N}}_{\mathit{cr}}=25$	${\mathit{N}}_{\mathit{cr}}=50$	${\mathit{N}}_{\mathit{cr}}=100$
APPENDICITIS	13.70%	14.00%	14.50%	14.70%
AUSTRALIAN	14.51%	14.46%	14.13%	13.97%
BALANCE	22.11%	22.29%	17.76%	18.05%
CIRCULAR	13.64%	11.90%	9.38%	8.46%
CLEVELAND	50.10%	49.69%	48.90%	49.17%
DERMATOLOGY	25.06%	20.51%	18.20%	16.29%
ECOLI	47.82%	47.79%	47.39%	47.52%
FERT	19.00%	18.70%	19.20%	18.70%
HABERMAN	28.03%	28.27%	28.43%	26.70%
HAYES-ROTH	35.93%	31.54%	27.77%	27.69%
HEART	15.78%	15.07%	16.00%	14.67%
HEARTATTACK	19.33%	20.13%	19.73%	18.50%
HOUSEVOTES	3.65%	3.30%	3.26%	3.13%
GLASS	57.10%	55.38%	54.62%	54.29%
IONOSPHERE	11.12%	10.63%	10.71%	9.89%
LIVERDISORDER	33.71%	32.03%	32.53%	31.12%
MAMMOGRAPHIC	17.78%	17.72%	17.64%	17.12%
PARKINSONS	12.21%	12.53%	12.79%	11.58%
PIMA	27.99%	27.26%	27.68%	26.09%
POPFAILURES	6.74%	6.33%	6.91%	6.35%
REGIONS2	25.52%	26.20%	25.47%	24.82%
SAHEART	30.52%	30.61%	29.81%	29.58%
SEGMENT	54.99%	53.07%	49.24%	42.90%
SPIRAL	48.39%	48.08%	48.20%	48.34%
STUDENT	5.78%	5.40%	5.20%	4.10%
TRANSFUSION	25.34%	25.26%	24.80%	24.47%
WDBC	6.95%	6.82%	7.39%	6.59%
WINE	14.35%	11.82%	11.77%	9.88%
Z_F_S	14.17%	12.60%	13.50%	9.98%
Z_O_N_F_S	49.18%	48.20%	46.24%	44.73%
ZO_NF_S	14.14%	12.72%	12.18%	10.42%
ZONF_S	3.14%	3.18%	2.82%	2.58%
ZOO	9.20%	8.20%	8.10%	7.50%
AVERAGE	23.54%	22.78%	22.19%	21.21%

.

The lowest average classification error is observed for $N_{cr}=100$; however, no major changes are observed in the classification errors as the parameter increases. Furthermore, we expected the average execution time to increase as the value $N_{cr}$ increases, and this is demonstrated in Figure 12, where the computation time for the neural network construction method is plotted with respect to the $N_{cr}$.

The average computation time increases dramatically as the critical parameter $N_{cr}$ increases, which is something that is expected, since the crossings increase significantly with the increase in this parameter. This dramatic increase in required execution time can be reduced by incorporating modern parallel libraries. These programming techniques may include the application of the Message Passing Interface (MPI) library [95] or the incorporation of the OpenMP library [96].

4. Conclusions

A new genetic operator for tasks based on Grammatical Evolution is introduced in this article. This operator is applied to randomly selected chromosomes of the genetic population. On each application, a group of randomly selected chromosomes is formulated for every chromosome, and one-point crossover is executed between each member of the group and the selected chromosome, aiming to reduce the associated fitness value. In order to measure the effectiveness of the new operator, it was applied with success in two machine learning models from the recent bibliography that utilizes the Grammatical Evolution method:

• A rule construction method that constructs rules in language similar to the C programming language for data classification or regression problems.
• A method that constructs artificial neural networks.

Several datasets from the literature were used to test the proposed method. In the vast majority of cases, the application of the new genetic operator resulted in a drastic reduction in the corresponding classification or data fitting error. Furthermore, to asses the sensitivity of the proposed method to some critical parameters, more experiments were conducted in which these parameters were changed inside a wide range of values. Boosting these values improves the performance of machine learning methods by applying the new genetic operator but only up to a point. In addition, an increase in the chromosomes involved in the genetic operator has a significant increase in the required execution time, as was also seen in the performed experiments. However, with the use of recent techniques that can utilize modern parallel computing structures, this additional time can be significantly reduced.

Improvements of the proposed operator in future studies may include the application of the new crossover in other machine learning methods based on Grammatical Evolution, a parallel implementation of the operator or even the usage of this operator in other tasks involving Genetic Algorithms.