Simultaneous Feature Selection and Support Vector Machine Optimization Using an Enhanced Chimp Optimization Algorithm

Abstract

Chimp Optimization Algorithm (ChOA), a novel meta-heuristic algorithm, has been proposed in recent years. It divides the population into four different levels for the purpose of hunting. However, there are still some defects that lead to the algorithm falling into the local optimum. To overcome these defects, an Enhanced Chimp Optimization Algorithm (EChOA) is developed in this paper. Highly Disruptive Polynomial Mutation (HDPM) is introduced to further explore the population space and increase the population diversity. Then, the Spearman’s rank correlation coefficient between the chimps with the highest fitness and the lowest fitness is calculated. In order to avoid the local optimization, the chimps with low fitness values are introduced with Beetle Antenna Search Algorithm (BAS) to obtain visual ability. Through the introduction of the above three strategies, the ability of population exploration and exploitation is enhanced. On this basis, this paper proposes an EChOA-SVM model, which can optimize parameters while selecting the features. Thus, the maximum classification accuracy can be achieved with as few features as possible. To verify the effectiveness of the proposed method, the proposed method is compared with seven common methods, including the original algorithm. Seventeen benchmark datasets from the UCI machine learning library are used to evaluate the accuracy, number of features, and fitness of these methods. Experimental results show that the classification accuracy of the proposed method is better than the other methods on most data sets, and the number of features required by the proposed method is also less than the other algorithms.

1. Introduction

With society gradually becoming digitalized, the question of how to extract useful information effectively from complex and huge data has become the focus of research in recent years. Machine learning is one of the important research fields of information recognition and pattern recognition in data sets [1]. It has been widely used due to its strong data processing ability. Machine learning is evolving from computer science [2], which is a multidisciplinary and interdisciplinary major. It covers probability theory, statistics, approximate theory, and complex algorithms, and it can design efficient and accurate prediction algorithms [3]. It can be divided into supervised learning and unsupervised learning according to the learning mode [4]. SVM is one of the most widely used supervised learning algorithms. Vapnik and other researchers built the cornerstone of SVM [5]. It is widely used in pattern recognition [6,7], text recognition [8], biomedical [9,10,11], imaging medicine [12], anomaly detection [13] and in other fields. SVM classifies feature spaces into two categories and defines a non-parametric method for finding decision boundaries [14]. SVM has shown excellent performance in many problems [15]. Using the support vector machines to solve problems can improve generalization performance, improve computational efficiency, reduce running time, and produce a very accurate classification model [16,17]. The Support Vector Machines can solve both linear and nonlinear classification problems. When the data set is linearly inseparable, the nonlinear problem is usually transformed into a linear problem. Then, the data set can be used to construct an optimal hyperplane in the feature space, so that the feature space can be divided into two classes more easily [18]. However, when machine learning processes high-dimensional data sets, there will be problems such as noise and redundant data [19]. In this case, feature selection [20] can be used to reduce the number of features.

Feature selection is to select the optimal feature subset from the original data set [21]. According to the form of feature selection, the feature selection methods can be divided into the following three types: filter, wrapper, and embedded [22,23,24]. The wrapper-based approach is used in this article. The wrapper method, which relies on the classification method [25], is more efficient than the other two, but its computational intensity is relatively greater [26]. The filter method [27] uses numerous indicators to evaluate and select high-order features according to their discriminant attributes [28]. After that it will select the subset with the richest information [29]. The main idea of the wrapper approach is to treat the selection of subsets as a search optimization problem. At first it will generate different combinations, then evaluate them, and finally compare them with other combinations. The wrapper method is often used in feature selection problems due to the superiority of its calculation results, and the wrapper method can directly interact with classifier [30]. The wrapper method can be effectively combined with the meta-heuristic optimization algorithm so to achieve better practical application effect.

In recent years, many researchers have combined optimization algorithms with SVM to solve problems. Influenced by this, this paper tries to combine ChOA with SVM for feature selection and parameter optimization. ChOA is a new meta-heuristic optimization algorithm proposed by Khishe and Mosavi in 2019 [31] which is inspired by the hierarchy mechanism of chimps in nature when they are hunting. It can solve the problem of slow convergence and local optimum when it is solving the serious dimensional problems. However, the ChOA still has some shortcomings. Firstly, the population diversity of the algorithm is insufficient in the initial stage. Secondly, there is a risk of falling into local optimal in the final search stage. Therefore, an Enhanced Chimp Optimization Algorithm is proposed in this paper. Three strategies have been introduced successively, including highly disruptive polynomial mutation [32], Spearman’s correlation coefficient, and Beetle Antennae Search Algorithm [33]. In the initial stage, HDPM is used to increase the population’s variety, which improved the detection ability of the optimization algorithm. Due to the updating of the position of the chimp being determined by each level of chimp, this paper will improve the global search ability of chimps by improving the position updating ability of lower level chimps. The algorithm firstly introduces the Spearman’s rank correlation coefficient to calculate the distance between low grade and high grade chimp. For the lower chimps that are far away from the higher chimps, this paper will introduce the beetle antenna search algorithm. It can make the chimps with low fitness achieve the visual ability, thus they can change their movement direction according to the surrounding environment. This strategy improves the local and global search ability of the chimps with lower fitness. On this basis, this paper proposes an EChOA-SVM model. This model is used for feature selection and parameter optimization at the same time to obtain good classification accuracy and performance. The main contributions of this paper are summarized as follows:

• An Enhanced Chimp Optimization Algorithm is proposed to solve the shortcomings of the ChOA and make it better applied to feature selection problems.
• HDPM strategy is introduced in the initial stage to enhance the population diversity.
• Spearman’s rank correlation coefficient helps to identify the chimps that need to be improved. Then BAS is introduced to improve the positions updating ability.
• The EChOA-SVM model is used for feature selection and SVM parameter optimization simultaneously. The model is evaluated by 17 benchmark data sets in UCI machine learning library [34]. In order to verify the effectiveness of this method, it is compared with seven optimization algorithms, such as ChOA [31], GWO [35], WOA [36], ALO [37], GOA [38], MFO [39], and SSA [40].

The structure of this article is as follows. Section 2 reviews relevant literatures. Section 3 briefly introduces the basic principle of the ChOA. Section 4 proposes a new EChOA, then introduces the related theory of EChOA-SVM. In Section 5, compared and analyzed the experimental results.

2. Literature Review

Optimization algorithms have been widely used in different fields such as medicine, multi-objective optimization, data classification, feature selection, and Support Vector Machine optimization. Zhao and Zhang proposed a Learning-based Evolutionary Multi-objective Algorithm [41]. By comparing five algorithms, the proposed algorithm is significantly superior to the other compared algorithms in determining the convergence and the approximation of the Pareto front. Dulebenets proposed an Adaptive Polyploid Memetic Algorithm to solve the cross-docking terminal vehicle scheduling problem [42]. This algorithm can assist the correct operation planning of CDT. Liu, Wang and Huang proposed an Alternative Algorithm to deal with Multiobjective optimization problems [43] which is proven to be better than other many-objective evolutionary algorithms. Furthermore, it is easily extended to solve constrained multi-objective optimization problems. Junayed et al., established an optimization model and solution algorithm to optimize the factory-in-a-box supply chain [44]. Gianni et al., Established specific rules or formulas that can distinguish bacterial from viral meningitis by machine learning methods [45]. The method even achieved 100% accuracy in detecting bacterial meningitis. Panda and Majhi used the Salp Swarm Algorithm to train the Multilayer Perceptron for data classification [46]. Compared with other classical optimization algorithms, the results show that this method is very advantageous.

This paper mainly studies the application of optimization algorithm in feature selection and support vector machine optimization. Feature selection is to select the optimal feature subset from the original data set, which can be regarded as an optimization problem [47]. In order to obtain a better classification accuracy, it is necessary to input the optimal SVM parameters into the feature subset. The selection of the feature subset will also affect the parameters of SVM. Therefore, in order to obtain the ideal classification accuracy, both feature selection and optimization of SVM parameters are required [48]. Huang and Wang [49] proposed a GA algorithm for feature selection and support vector machine parameter optimization simultaneously. Compared with other methods, the proposed method can achieve higher classification accuracy with fewer features. Shih-Wei Lin et al. [50] proposed a feature selection and parameter optimization method based on SA, and compared the method with grid algorithm. The results showed that the proposed method could significantly improve the classification accuracy. Heming Jia and Kangjian Sun proposed an IBMO-SVM classification model [51]. This model can help SVM to find the optimal feature subset and parameters at the same time. Compared with other optimization algorithms, this model shows good performance on both low and high dimensional data sets. The ECHOA-SVM model, which is proposed in this paper, is inspired by these methods.

3. Chimp Optimization Algorithm (ChOA)

ChOA is a mathematical model that is based on intelligent diversity [31]. Driving, chasing, blocking and attacking are accomplished by four different types of chimps, which are accomplished by attackers, obstacles, chasers, and drivers. The four hunting steps are completed in two stages. The first stage is the exploration stage, and the second stage is the exploitation stage. The exploration stage includes driving, blocking, and chasing the prey. As for the exploitation stage, it has to attack the prey. In which, the driving and chasing are represented by Equations (1) and (2).

$$d=\left|c\cdot x_{prey}(t)-m\cdot x_{chimp}(t)\right|$$

(1)

$$x_{chimp}(t+1)=x_{prey}(t)-a\cdot d$$

(2)

where $x_{prey}$ is the vector of prey position, $x_{chimp}$ is the vector of chimp position, t is the number of current iterations, $a$, $c$, $m$ are coefficient vectors and they can be obtained through Equations (3)–(5).

a = 2 · f · r₁ – f

(3)

c = 2 · r₂

(4)

$$m=Chaotic\_value$$

(5)

where $f$ non-linearly declined from 2.5 to 0, r₁ and r₂ is the random number between 0 and 1, and $m$ is the chaotic vector. The dynamic coefficient $f$ can be selected for different curves and slopes, thus chimps can use different abilities to search the prey.

Chimps can update their positions based on the other chimps, and this mathematical model can be represented by Equations (6) and (8).

$$\begin{array}{l}{d}_{Attac\ker }=\left|c_1{x}_{Attac\ker }-m_{1}x\right|\\ d_{Barrier}=\left|c_2{x}_{Barrier}-m_{2}x\right|\\ d_{Chaser}=\left|c_3{x}_{Chaser}-m_{3}x\right|\\ d_{Driver}=\left|c_4{x}_{Driver}-m_{4}x\right|\end{array}$$

(6)

$$\begin{array}{l}{x}_1=x_{Attac\ker }-a_1(d_{Attac\ker })\\ x_2=x_{Barrier}-a_2(d_{Barrier})\\ x_3=x_{Chaser}-a_3(d_{Chaser})\\ x_4=x_{Driver}-a_4(d_{Driver})\end{array}$$

(7)

$$x\left(t+1\right)=\frac{x_1+x_2+x_3+x_4}{4}$$

(8)

4. Enhanced Chimp Optimization Algorithm (EChOA)

4.1. Highly Disruptive Polynomial Mutation (HDPM)

The highly disruptive polynomial mutation is the improved version of the polynomial mutation method. It can solve the shortcoming that the polynomial mutation method may fall into local optimum when the variable is close to the boundary [32]. Equations (9)–(12) show the process of HDPM changes the $x_i$

$${\delta }_1=\frac{x_i-lb}{ub-lb}$$

(9)

$${\delta }_2=\frac{ub-x_i}{ub-lb}$$

(10)

$${\delta }_k=\left\{\begin{array}{l}{[(2r)+(1-2r)\ast {(1-{\delta }_1)}^{{\eta }_m+1}]}^{\frac{1}{{\eta }_m+1}-1},r\le 0.5\\ 1-{[2(1-r)+2(r-0.5)\ast {(1-{\delta }_2)}^{{\eta }_m+1}]}^{\frac{1}{{\eta }_m+1}},otherwise\end{array}\right.$$

(11)

$$x_i=x_i+{\delta }_k(ub-lb)$$

(12)

where ub and lb represent the upper and lower boundaries of the search space. $r$ is a random number between 0 and 1. ${\eta }_m$ is a distribution exponential, which is a non-negative number. As can be seen from the above formula, HDPM can explore the entire search space.

4.2. Spearman’s Rank Correlation Coefficient

Spearman’s rank correlation coefficient is a statistical index to reflect the degree of the relationship between two groups of variables, which is figured upon a level basis. The formula for calculating as Equation (13).

$$\rho =1-\frac{6{\displaystyle \sum d_i^2}}{n(n^2-1)}$$

(13)

where $d_i$ is the difference of grades among each pair of samples, and n is the dimension of the series. If the absolute value of the correlation coefficient is equal to 1, then the two series are monotonically correlated; otherwise, they are uncorrelated.

4.3. Beetle Antennae Search Algorithm (BAS)

The long-horned beetle has two too long antennae, which can combine the scent of prey to expand detection range and act as a protective alarm mechanism [33]. The beetle explores nearby areas by swinging its antennae on one side of its body to accept the smell. The beetle will move toward to the side where it detects a high odor concentration, as it is shown in Figure 1. The Beetle Antennae Search Algorithm is designed based on this property of beetles.

The search direction of beetles is represented by Equation (14).

$$\overrightarrow{b}=\frac{rnd(k,1)}{‖rnd(k,1)‖}$$

(14)

where $\overrightarrow{b}$ is the direction vector of the beetle, rnd is a random function, and k is the dimension of the position.

Next, Equation (15) respectively presents the search behaviors on the left and right sides to simulate the activity tracking of the beetle.

$$\begin{array}{l}{x}_l=x^t+d^t\overrightarrow{b}\\ x_r=x^t-d^t\overrightarrow{b}\end{array}$$

(15)

where $x_l$ and $x_r$ respectively represent the locations within the left and right search areas, $x^t$ represents the beetle’s position at $t$th time instant, and $d^t$ represents the perceived length of the antenna, which will gradually decrease with the passage with time.

The position update of the beetle can be represented by Equation (16).

$$x^t=x^{t-1}+{\delta }^t\overrightarrow{b}sign(f(x_r)-f(x_l))$$

(16)

where $\delta$ is the step size of the search, which initial scope should be equal to the search area, and the sign represents the sign function. The odor concentration at x is expressed by $f(x)$, which is also known as the fitness function.

4.4. Improvement Strategy

Although the original ChOA divides the chimps into four different levels to complete the hunting process, the algorithm still has two obvious defects when applied to higher dimensional problems. The first disadvantage is that in the initial stage of the population it lacks the population diversity. The other disadvantage is there is a risk of falling into the local optimal in the final search stage. Therefore, this article proposes an Enhanced Chimp Optimization Algorithm, which can be better applied to feature selection problems.

Three strategies including HDPM, Spearman’s rank correlation coefficient, and BAS algorithm are introduced into EChOA based on the original algorithm. First, the HDPM strategy is introduced to enhance the population diversity in the initialization phase. The traditional polynomial mutation (PM) has almost no effect when the variable is close to the boundary. The HDPM strategy uses Equation (12) to generate the mutation location of the chimp, which helps further to explore the regions and boundaries of the initial space. In this way, even if the variables are on the search boundary, the search space can be fully utilized to ensure the diversity of the population. Second, the Spearman’s rank correlation coefficient between the driver and attacker chimp is calculated. As it can be seen from Equation (7), position update is jointly determined by all chimps of different grades. By improving the position of the lower chimp, the population can avoid falling into local optimal effectively. The distance between the driver and the attacker can be determined by calculating the Spearman’s rank correlation coefficient of the two by Equation (13). The two chimp species are negatively or uncorrelated when Spearman’s rank correlation coefficient is less than or equal to 0. This will show whether the driver and the attacker are close or far away. Finally, the BSA is introduced for the chimp which is far away from the attacker. The position of the chimp is improved by using Equation (16), which makes it acquire the visual ability. Thus, they can judge the surrounding environment and decide the direction of movement. This increases the performance of the driver chimp and avoids local optimal.

4.5. EChOA for Optimizing SVM and Feature Selection

EChOA-SVM firstly uses optimization algorithm to search the optimal feature subset, and then classify the feature subset by SVM. In the feature selection, the positions of population are separated into 0 and 1 by logical function. If the feature corresponds to 1, the feature will be selected; if the feature corresponds to 0, the feature will not be selected. Since the selection of kernel function and its parameters is related to the performance of SVM, in order to obtain better classification accuracy, EChOA is also needed to optimize the parameters. The EChOA-SVM model generates the primary agents through the penalty parameter c and the nuclear parameter g, and calculates the fitness value to evaluate the chimp’s positions. Finally outputs the optimal value of c, g and the best accuracy to obtain the final classification result.

The description of the EChOA-SVM is shown as follows. The flow chart is shown in Figure 2.

• Initialize the population, position and fitness value of the chimps.
• SVM randomly generates the values of c and g, and then calculates the initial fitness value.
• The training model is obtained by training set in support vector machine.
• Reorder the population to get the agent position and fitness value after sorting.

5. Experimental Results and Discussion

5.1. Datasets Details

The implementation of the proposed algorithm is done using Matlab. Seventeen data sets from the University of California at Irvine (UCI) machine learning repository [34] has been selected to evaluate the proposed EChOA-SVM approach.

Table 1. List of data sets.

Data Set	Features	Instances	Classes
Balance	4	625	3
Bupa	6	345	2
Coimbra	9	116	2
Dermatology	34	358	6
Diagnostic	10	683	4
Divorce	18	170	2
Glass	10	214	6
Heart	13	270	2
Iris	4	150	3
Knowledge	5	258	4
Liver	6	345	2
Lymphography	18	148	8
Sonar	60	208	2
Teaching Assistant Evaluation (TAE)	5	151	3
Transfusion	4	748	2
Wine	13	178	3
Zoo	16	101	7

describes the number of instances, the number of features, and the number of classes of the data set. Seventeen data sets of different sizes and latitudes are selected to observe the performance of the algorithm at different scales.

All the experimental series are carried out on Matlab R2016a, and the computer is configured as Intel(R) Core (TM) i5-1035G1 CPU @ 1.00 GHz 1.19 GHz, using Microsoft Windows 10 system. Each experiment is run ten times independently to reduce the random influence. In addition, the general parameters are set as follows: the population size is 30, and each run is set to 100 iterations as stopping criteria.

5.2. Parameters Setting

In order to verify the optimization performance of EChOA-SVM, this paper compares the algorithm with different meta-heuristic algorithms such as ChOA [31], GWO [35], WOA [36], ALO [37], GOA [38], MFO [39] and SSA [40].

All settings and parameters that are used in the experiments for all algorithms are presented in

Table 2. Parameter settings of different algorithms.

Algorithm	Parameters	Value
EChOA	ηm	1
	C	2
	K	0.95
	f	[0, 2.5]
	m	Gauss chaotic
ChOA	f	[0, 2.5]
	m	Gauss chaotic
GWO	α	[0, 2]
WOA	α	[0, 2]
	a2	[−2, −1]
ALO	Number of search agents	30
	Number of iterations	100
GOA	cMin	0.00001
	cMax	1
	Number of search agents	30
	Number of iterations	100
MFO	α	[−2, −1]
	b	1
	t	[−2, −1]
SSA	c1	[2 × 10−16, 2]
	c2	[0, 1]
	c3	[0, 1]

.

5.3. Results and Discussion

In order to compare the different feature selection methods with the method proposed in this paper, three indexes are used in this paper [53].

• Accuracy: Accuracy represents the ratio between the number of correct classification and the actual number, which is reflecting the accuracy of the classifier recognition results. It is an important index to evaluate algorithm performance;
• The number of features: The number of features reflects ability of eliminate redundancy. It shows whether the method can find the optimal feature subset;
• Fitness value: Fitness value can reflect the advantages and disadvantages of the solution selected by the classifier. The better fitness value can get the better the solution.

Table 3. Mean and standard deviation of classification accuracy for all data sets with applying feature selection.

		EChOA	ChOA	GWO	WOA	ALO	GOA	MFO	SSA
Balance	mean	8.9853 × 10−1	8.9073 × 10−1	8.8640 × 10−1	8.9376 × 10−1	8.9120 × 10−1	8.9120 × 10−1	7.7344 × 10−1	8.9120 × 10−1
	std	6.4825 × 10−3	4.1199 × 10−3	1.5513 × 10−2	2.6349 × 10−3	1.2413 × 10−16	1.2413 × 10−16	6.5830 × 10−2	1.2413 × 10−16
Bupa	mean	6.8217 × 10−1	6.1943 × 10−1	6.6380 × 10−1	6.4002 × 10−1	6.2106 × 10−1	6.2260 × 10−1	6.2378 × 10−1	6.2204 × 10−1
	std	5.8192 × 10−3	7.1396 × 10−3	3.6625 × 10−2	3.3470 × 10−2	3.9715 × 10−3	3.2047 × 10−3	8.5538 × 10−3	5.7270 × 10−3
Coimbra	mean	6.5348 × 10−1	6.5348 × 10−1	7.2414 × 10−1	7.0518 × 10−1	6.2930 × 10−1	6.0818 × 10−1	5.7930 × 10−1	6.0858 × 10−1
	std	3.8460 × 10−3	7.1953 × 10−3	5.2065 × 10−2	9.1744 × 10−2	2.8621 × 10−2	1.7562 × 10−2	2.3106 × 10−2	1.1583 × 10−2
Dermatology	mean	8.6590 × 10−1	8.5010 × 10−1	9.6087 × 10−1	9.4638 × 10−1	7.5922 × 10−1	7.5752 × 10−1	6.0560 × 10−1	8.0838 × 10−1
	std	1.2205 × 10−2	6.9936 × 10−3	2.1811 × 10−2	3.1929 × 10−2	2.4481 × 10−2	4.6042 × 10−2	8.1800 × 10−2	2.6113 × 10−2
Diagnostic	mean	9.6607 × 10−1	9.5450 × 10−1	9.6516 × 10−1	9.6430 × 10−1	9.5842 × 10−1	9.5754 × 10−1	9.3558 × 10−1	9.5842 × 10−1
	std	6.1598 × 10−3	4.4034 × 10−3	3.7997 × 10−3	6.0828 × 10−3	1.2969 × 10−3	2.0683 × 10−3	9.6469 × 10−3	1.2969 × 10−3
Divorce	mean	9.8044 × 10−1	9.7768 × 10−1	9.7650 × 10−1	9.7298 × 10−1	9.1764 × 10−1	9.4236 × 10−1	8.9176 × 10−1	9.1412 × 10−1
	std	6.3689 × 10−3	2.6386 × 10−3	4.1719 × 10−3	1.4765 × 10−2	2.1989 × 10−2	1.9255 × 10−2	6.6920 × 10−3	3.1768 × 10−3
Glass	mean	6.8067 × 10−1	6.7290 × 10−1	9.8813 × 10−1	7.5796 × 10−1	6.3924 × 10−1	6.5048 × 10−1	5.8410 × 10−1	6.7010 × 10−1
	std	1.1775 × 10−2	1.2341 × 10−2	2.3159 × 10−3	7.1049 × 10−1	1.5296 × 10−2	2.5891 × 10−2	4.9679 × 10−2	3.7157 × 10−2
Heart	mean	8.4230 × 10−1	8.3207 × 10−1	7.8594 × 10−1	7.4666 × 10−1	8.0370 × 10−1	7.9738 × 10−1	7.4740 × 10−1	8.0888 × 10−1
	std	4.7606 × 10−3	8.5448 × 10−3	2.2868 × 10−2	8.8013 × 10−2	6.4086 × 10−3	9.3041 × 10−3	3.8271 × 10−2	1.7667 × 10−2
Iris	mean	9.6400 × 10−1	9.5337 × 10−1	9.6268 × 10−1	9.6268 × 10−1	9.5734 × 10−1	9.4934 × 10−1	9.3068 × 10−1	9.4268 × 10−1
	std	8.9443 × 10−3	1.1547 × 10−2	3.6697 × 10−3	3.6697 × 10−3	8.9592 × 10−3	7.5910 × 10−3	4.0173 × 10−2	3.6697 × 10−3
Knowledge	mean	9.5675 × 10−1	9.5087 × 10−1	8.5426 × 10−1	8.9692 × 10−1	9.5350 × 10−1	9.4660 × 10−1	9.0542 × 10−1	9.5428 × 10−1
	std	1.8385 × 10−3	2.3714 × 10−3	7.9861 × 10−2	4.6400 × 10−2	2.7577 × 10−3	1.2371 × 10−2	3.9133 × 10−2	3.2630 × 10−3
Liver	mean	6.6937 × 10−1	6.3383 × 10−1	6.6844 × 10−1	6.4350 × 10−1	6.3248 × 10−1	6.2490 × 10−1	6.2142 × 10−1	6.2610 × 10−1
	std	3.2716 × 10−3	1.3395 × 10−2	1.6609 × 10−2	6.4846 × 10−3	1.4266 × 10−2	4.2024 × 10−3	5.9997 × 10−3	0.0000
Lymphography	mean	5.1556 × 10−1	5.0900 × 10−1	4.3778 × 10−1	4.3512 × 10−1	4.7432 × 10−1	4.7568 × 10−1	4.2838 × 10−1	4.7566 × 10−1
	std	4.1004 × 10−3	7.7942 × 10−3	3.2275 × 10−2	2.8920 × 10−2	1.1088 × 10−2	2.0061 × 10−2	1.5544 × 10−2	1.6980 × 10−2
Sonar	mean	8.4137 × 10−1	8.2373 × 10−1	8.0578 × 10−1	8.3368 × 10−1	7.2978 × 10−1	7.8076 × 10−1	7.4040 × 10−1	7.1252 × 10−1
	std	9.4701 × 10−2	4.0880 × 10−2	4.4297 × 10−2	3.1638 × 10−2	5.6004 × 10−2	1.0441 × 10−1	1.2143 × 10−1	1.1694 × 10−1
TAE	mean	6.1110 × 10−1	6.0840 × 10−1	5.7084 × 10−1	5.8940 × 10−1	6.0290 × 10−1	6.1132 × 10−1	5.9206 × 10−1	6.0174 × 10−1
	std	2.3812 × 10−3	1.1533 × 10−3	2.9963 × 10−3	1.6228 × 10−2	5.4282 × 10−3	1.7858 × 10−2	1.7293 × 10−2	9.0329 × 10−3
Transfusion	mean	7.7713 × 10−1	7.6380 × 10−1	7.7218 × 10−1	7.7004 × 10−1	7.6200 × 10−1	7.6414 × 10−1	7.6280 × 10−1	7.6392 × 10−1
	std	4.1429 × 10−3	3.1177 × 10−3	9.3082 × 10−3	6.1788 × 10−3	0.0000	4.7852 × 10−3	1.7889 × 10−3	2.6631 × 10−3
Wine	mean	8.5562 × 10−1	9.5503 × 10−1	9.3147 × 10−1	9.0674 × 10−1	6.9776 × 10−1	6.4616 × 10−1	4.5506 × 10−1	7.7416 × 10−1
	std	3.7552 × 10−2	1.4873 × 10−2	1.6064 × 10−2	4.0183 × 10−2	1.4329 × 10−1	1.0406 × 10−1	6.0622 × 10−2	1.1504 × 10−1
Zoo	mean	9.8119 × 10−1	9.8010 × 10−1	9.2377 × 10−1	9.3565 × 10−1	9.6139 × 10−1	9.5743 × 10−1	9.0892 × 10−1	9.6337 × 10−1
	std	3.1307 × 10−3	9.0554 × 10−4	3.4310 × 10−2	1.9383 × 10−2	8.6684 × 10−3	1.3241 × 10−2	2.1790 × 10−2	1.1479 × 10−2

shows the mean and the standard deviation of the accuracy. Figure 3 presents the boxplot charts of the accuracy for eight used data sets. The boxplots are shown for the seventeen values of classification accuracy that are given by each algorithm at the end of the SVM training. The boxplots indicate the improved performance of EChOA for optimizing the SVM.

The performance of EChOA is better than other optimization algorithms in terms of average accuracy. EChOA obtains the best performance on 13 data sets out of the 17 data sets, which accounts for 76.5% of the total data sets. For the other four data sets, EChOA also has strong competitiveness. Although EChOA do not achieve the highest accuracy in Coimbra, Dermatology and Glass, its classification accuracy ranked the third among the eight optimization algorithms, just next to GWO and WOA. It is noteworthy that the accuracy of EChOA is 10 percent higher than ChOA on Bupa data sets. It is 12 percent higher than GWO on the Knowledge data set and 18 percent higher than GWO on the Lymphography data set. At the same time, it can be seen from the box chart that each optimization algorithm generates ten accuracies in the process of running each data set ten times. From the perspective of their distribution, the accuracy of EChOA is very stable. The accuracy of the EChOA optimization algorithm hardly changes significantly in the 17 data sets. Especially in the Coimbra and Heart data sets, the EChOA optimization algorithm is still very stable in the case that the accuracy of almost all optimization algorithms varies significantly each time. All these show the superiority of the EChOA algorithm.

Table 4. Mean and standard deviation of the number of features for all data sets with applying feature selection.

		EChOA	ChOA	GWO	WOA	ALO	GOA	MFO	SSA
Balance	mean	2.3333	2.6667	3.2000	2.8000	3.8000	2.8000	3.2000	3.4000
	std	5.7735 × 10−1	5.7735 × 10−1	1.3038	7.1274 × 10−1	4.4721 × 10−1	8.3666 × 10−1	4.4721 × 10−1	8.9443 × 10−1
Bupa	mean	2.0000	1.6667	3.0000	2.8000	1.2000	1.6000	2.2000	1.4000
	std	0.0000	5.7735 × 10−1	1.8708	1.7889	4.4721 × 10−1	5.4772 × 10−1	4.4721 × 10−1	5.4772 × 10−1
Coimbra	mean	1.2000	1.4000	4.0000	4.2000	1.6000	1.4000	1.8000	1.4000
	std	4.4721 × 10−1	5.4772 × 10−1	1.7321	1.6432	8.9443 × 10−1	5.4772 × 10−1	8.3666 × 10−1	5.4772 × 10−1
Dermatology	mean	7.6667	8.6667	1.4000 × 10	1.4400 × 10	1.1000 × 10	1.0800 × 10	1.3000 × 10	9.8000
	std	5.7735 × 10−1	5.7735 × 10−1	6.2450	3.1305	7.0711 × 10−1	1.0954	2.9155	1.6432
Diagnostic	mean	1.6667	2.3333	4.0000	4.4000	1.8000	2.2000	3.6000	2.0000
	std	5.7735 × 10−1	2.3333	1.8708	2.0736	4.4721 × 10−1	4.4721 × 10−1	8.9443 × 10−1	0.0000
Divorce	mean	2.0000	2.6000	7.4000	8.4000	3.8000	3.0000	6.4000	4.0000
	std	7.0711 × 10−1	5.4772 × 10−1	1.1402	2.4083	8.3666 × 10−1	1.5811	1.6733	7.0711 × 10−1
Glass	mean	3.3333	4.3333	5.3333	5.0000	4.4000	4.6000	5.2000	3.6000
	std	5.7735 × 10−1	5.7735 × 10−1	2.5166	4.8000	1.6733	1.3416	1.3038	1.8166
Heart	mean	2.6667	3.0000	6.2000	5.6000	4.2000	4.4000	4.0000	4.0000
	std	5.7735 × 10−1	1.0000	1.4832	2.0736	4.4721 × 10−1	5.4772 × 10−1	1.5811	1.4142
Iris	mean	1.2000	2.3333	2.2000	2.0000	1.8000	1.4000	1.4000	1.6000
	std	4.4721 × 10−1	5.7735 × 10−1	8.3666 × 10−1	7.0711 × 10−1	8.3666 × 10−1	5.4772 × 10−1	8.9443 × 10−1	5.4772 × 10−1
Knowledge	mean	1.7500	2.3333	2.2000	3.8000	2.0000	2.2000	2.2000	2.0000
	std	5.0000 × 10−1	5.7735 × 10−1	1.7889	1.0954	0.0000	4.4721 × 10−1	8.3666 × 10−1	0.0000
Liver	mean	1.0000	1.3333	4.6000	3.2000	1.6000	1.2000	2.4000	1.2000
	std	1.0000	5.7735 × 10−1	8.9443 × 10−1	8.3666 × 10−1	5.4772 × 10−1	4.4721 × 10−1	5.4772 × 10−1	4.4721 × 10−1
Lymphography	mean	6.4000	7.3333	7.8000	9.4000	7.6000	7.2000	8.8000	7.4000
	std	5.4772 × 10−1	5.7735 × 10−1	2.6833	2.6077	5.4772 × 10−1	1.9235	1.6432	1.1402
Sonar	mean	1.4667 × 10	1.7333 × 10	2.6000 × 10	3.3600 × 10	2.4600 × 10	2.6400 × 10	2.8600 × 10	2.3400 × 10
	std	2.0817	2.3094	2.9155	4.0373	2.5100	4.1593	3.1305	2.3022
TAE	mean	1.6667	2.3333	2.2000	2.4000	3.0000	3.0000	3.4000	2.8000
	std	5.7735 × 10−1	1.1547	4.4721 × 10−1	5.4772 × 10−1	0.0000	0.0000	8.9443 × 10−1	4.4721 × 10−1
Transfusion	mean	3.0000	2.3333	2.4000	2.8000	2.0000	2.2000	2.0000	2.2000
	std	0.0000	1.1547	5.4772 × 10−1	8.3666 × 10−1	7.0711 × 10−1	1.0954	0.0000	8.3666 × 10−1
Wine	mean	2.6000	6.0000	3.8000	5.8000	3.4000	4.0000	4.6000	3.0000
	std	8.4327 × 10−1	1.0000	1.7512	1.7512	2.0736	2.0000	1.5166	1.0000
Zoo	mean	5.0000	6.2500	5.7000	7.4000	6.2000	6.2000	7.3000	6.3000
	std	6.6667 × 10−1	9.5743 × 10−1	3.1990	3.2042	8.6684 × 10−3	7.8881 × 10−1	1.4944	1.2517

shows the mean and the standard deviation of the number of the features. Figure 4 presents the histogram of the number of features for all used data sets. We can intuitively see that the number of features required by EChOA is almost below the average of the number of features of all the algorithms. There are only two data sets do not get the best results (Bupa and Transfusion), which accounts for only 11.8 percent of the total. In Liver data set EChOA even requires only one feature. It shows that EChOA can use space more efficiently. In the analysis of classification accuracy, it has been found that EChOA’s classification accuracy is lower than GWO and WOA in Coimbra, Dermatology and Glass. However, it is not difficult to find in Table 4 that the number of features required by EChOA in these three data sets is less than GWO and WOA. Especially in the Dermatology data set, EChOA required nearly seven fewer features than GWO and WOA. It can be seen in Figure 4, in Coimbra, Dermatology, Divorce and Liver data sets, there are two or three optimization algorithms with far higher features counts than the other optimization algorithms, but EChOA never had this situation. It should be noticed that the number of features required by CHOA is much higher than that of EChOA in Dermatology and Wine data sets, which also indicates the superiority of EChOA over CHOA and other algorithms in the quantitative aspect of features.

Table 5. Mean and standard deviation of fitness values for all data sets with applying feature selection.

		EChOA	ChOA	GWO	WOA	ALO	GOA	MFO	SSA
Balance	mean	1.1070 × 10−1	1.1437 × 10−1	1.3420 × 10−1	1.1218 × 10−1	1.1720 × 10−1	1.1470 × 10−1	2.3226 × 10−1	1.1620 × 10−1
	std	1.9079 × 10−3	1.4434 × 10−3	3.0616 × 10−2	1.4595 × 10−3	1.1180 × 10−3	2.0917 × 10−3	6.4041 × 10−2	2.2361 × 10−3
Bupa	mean	3.3020 × 10−1	3.7293 × 10−1	3.3786 × 10−1	3.6106 × 10−1	3.7214 × 10−1	3.7282 × 10−1	3.7646 × 10−1	3.7248 × 10−1
	std	7.6531 × 10−3	9.8150 × 10−4	3.5019 × 10−2	3.2445 × 10−2	7.6026 × 10−4	9.3113 × 10−4	8.5483 × 10−3	9.3113 × 10−4
Coimbra	mean	3.4442 × 10−1	3.4464 × 10−1	2.7756 × 10−1	2.9656 × 10−1	3.6810 × 10−1	3.9710 × 10−1	4.1846 × 10−1	3.8880 × 10−1
	std	3.7090 × 10−3	7.3310 × 10−3	4.9871 × 10−2	8.9737 × 10−2	2.8289 × 10−2	3.4474 × 10−2	2.3793 × 10−2	1.0957 × 10−2
Dermatology	mean	1.3580 × 10−1	1.5097 × 10−1	4.2833 × 10−2	5.7340 × 10−2	2.4160 × 10−1	2.4320 × 10−1	3.9430 × 10−1	1.9258 × 10−1
	std	1.2653 × 10−2	7.1066 × 10−3	2.0294 × 10−2	3.1561 × 10−2	2.4253 × 10−2	4.5528 × 10−2	8.1597 × 10−2	2.6259 × 10−2
Diagnostic	mean	3.7460 × 10−2	4.2600 × 10−2	3.8480 × 10−2	3.9760 × 10−2	4.2980 × 10−2	4.4240 × 10−2	6.7380 × 10−2	4.3180 × 10−2
	std	5.6241 × 10−3	0.0000	2.8639 × 10−3	6.3830 × 10−3	1.4738 × 10−3	2.4006 × 10−3	1.0348 × 10−2	1.2969 × 10−3
Divorce	mean	3.0560 × 10−2	2.3240 × 10−2	2.7420 × 10−2	3.1460 × 10−2	8.3660 × 10−2	5.8720 × 10−2	1.1072 × 10−1	8.7260 × 10−2
	std	4.3206 × 10−3	2.5938 × 10−3	4.3568 × 10−3	1.4762 × 10−2	2.2144 × 10−2	1.9400 × 10−2	7.4221 × 10−3	2.9458 × 10−3
Glass	mean	3.1980 × 10−1	3.2820 × 10−1	1.4633 × 10−2	2.4464 × 10−1	3.6154 × 10−1	3.5062 × 10−1	4.1692 × 10−1	3.3020 × 10−1
	std	1.1755 × 10−2	1.2429 × 10−2	2.5166 × 10−3	2.9145 × 10−1	1.4406 × 10−2	2.6367 × 10−2	4.8918 × 10−2	3.8471 × 10−2
Heart	mean	1.6360 × 10−1	1.6877 × 10−1	2.1670 × 10−1	2.5512 × 10−1	1.9754 × 10−1	2.1170 × 10−1	2.5312 × 10−1	1.9228 × 10−1
	std	8.6603 × 10−4	8.9489 × 10−3	2.2128 × 10−2	8.7301 × 10−2	6.7122 × 10−3	1.7476 × 10−2	3.8069 × 10−2	1.8465 × 10−2
Iris	mean	3.8140 × 10−2	4.8700 × 10−2	3.9460 × 10−2	4.1960 × 10−2	4.5240 × 10−2	5.3160 × 10−2	7.2140 × 10−2	5.9260 × 10−2
	std	7.7661 × 10−3	1.1432 × 10−2	2.8183 × 10−3	2.8183 × 10−3	6.8178 × 10−3	6.6842 × 10−3	4.1966 × 10−2	3.6150 × 10−3
Knowledge	mean	4.7150 × 10−2	4.9667 × 10−2	1.4868 × 10−1	1.0966 × 10−1	5.0020 × 10−2	5.1460 × 10−2	9.8040 × 10−2	4.9260 × 10−2
	std	1.9000 × 10−3	5.7735 × 10−4	7.9223 × 10−2	4.6659 × 10−2	2.7225 × 10−3	8.1076 × 10−3	3.7952 × 10−2	3.2153 × 10−3
Liver	mean	3.5083 × 10−1	3.6473 × 10−1	4.9214 × 10−1	3.5930 × 10−1	3.6650 × 10−1	4.9214 × 10−1	3.7932 × 10−1	3.7214 × 10−1
	std	1.8717 × 10−3	1.2240 × 10−2	2.6909 × 10−1	7.1554 × 10−3	1.3778 × 10−2	2.6909 × 10−1	6.5933 × 10−3	7.6026 × 10−4
Lymphography	mean	4.8520 × 10−1	4.9017 × 10−1	5.6088 × 10−1	5.6442 × 10−1	5.2464 × 10−1	5.2308 × 10−1	5.7058 × 10−1	5.2320 × 10−1
	std	2.7386 × 10−4	7.5692 × 10−3	3.2668 × 10−2	2.7232 × 10−2	1.0873 × 10−2	2.0454 × 10−2	1.6228 × 10−2	1.6372 × 10−2
Sonar	mean	1.6070 × 10−1	1.7740 × 10−1	1.9664 × 10−1	1.7028 × 10−1	2.7160 × 10−1	2.2144 × 10−1	2.6176 × 10−1	2.8852 × 10−1
	std	9.2529 × 10−2	4.0124 × 10−2	4.4221 × 10−2	3.1281 × 10−2	5.5205 × 10−2	1.0308 × 10−1	1.2014 × 10−1	1.1603 × 10−1
TAE	mean	3.8797 × 10−1	3.9147 × 10−1	4.2888 × 10−1	4.1128 × 10−1	3.9280 × 10−1	4.1524 × 10−1	4.1066 × 10−1	3.9240 × 10−1
	std	1.8448 × 10−3	2.3094 × 10−3	2.9516 × 10−3	1.6103 × 10−2	0.0000	2.0485 × 10−2	1.8461 × 10−2	8.9443 × 10−4
Transfusion	mean	2.3203 × 10−1	2.3800 × 10−1	7.7218 × 10−1	2.3462 × 10−1	2.4060 × 10−1	7.6414 × 10−1	2.3880 × 10−1	2.3754 × 10−1
	std	5.0143 × 10−3	1.7321 × 10−4	9.3082 × 10−3	4.5329 × 10−3	1.7678 × 10−3	4.7852 × 10−3	2.4900 × 10−3	2.0780 × 10−3
Wine	mean	1.4506 × 10−1	4.9100 × 10−2	7.1150 × 10−2	9.6780 × 10−2	3.0180 × 10−1	2.8960 × 10−1	5.4302 × 10−1	2.2590 × 10−1
	std	3.6823 × 10−2	1.4126 × 10−2	1.6038 × 10−2	3.9600 × 10−2	1.4065 × 10−1	2.5621 × 10−2	6.0944 × 10−2	1.1317 × 10−1
Zoo	mean	2.1740 × 10−2	2.3525 × 10−2	7.9040 × 10−2	6.8210 × 10−2	4.2120 × 10−2	4.6050 × 10−2	9.4730 × 10−2	4.0220 × 10−2
	std	3.1362 × 10−3	6.1847 × 10−4	3.2862 × 10−2	1.8403 × 10−2	9.0828 × 10−3	1.3499 × 10−2	2.1928 × 10−2	1.1204 × 10−2

shows the mean and the standard deviation of the fitness. Figure 5 presents the line chart of the fitness for six used data sets. The lines in Figure 5 represents the convergence curves of corresponding algorithm. The fitness value of EChOA is the minimum in all the 12 data sets, and the convergence time is very short. The convergence speed of Coimbra, Dermatology, Diagnostic, Divorce and Heart data sets is all higher than ChOA, and some fitness values are also lower than ChOA. At the same time, EChOA convergence curve is relatively flat, in the performance of each data set is relatively good, which are almost below to the average fitness value, does not appear similar to GOA (e.g., Liver) in a certain data set fitness value too much higher than the other algorithms. As it can be seen from Figure 5, the convergence curve of EChOA in the Heart data set is smoother than other algorithms, which shows that the local search ability of the algorithm has been greatly improved. EChOA algorithm can achieve the maximum objective function value when the number of iterations is less than 10 on Dermatology and Divorce data sets, which is earlier than other optimization algorithms. This indicates that the convergence rate of EChOA is improved. As you can see, EChOA is better and more stable than most algorithms.

6. Conclusions

This work presents a novel hybrid method for optimizing SVM based on the EChOA. The proposed approach is able to tune the parameters of the SVM kernel and at the same time can find the best accuracy. Experimental results show that the proposed algorithm is effective in improving the classification accuracy of SVM. The experimental results show that the EChOA algorithm has certain advantages over the other seven optimization algorithms in terms of accuracy, feature number and fitness value, and its comprehensive performance is relatively stable. All these indicate that the EChOA algorithm is very competitive.

Although EChOA can effectively improve the exploitation and exploration of ChOA, EChOA introduces three strategies at the same time that increase the complexity of the algorithm. Therefore, it is necessary to consider how to reduce ineffective improvement strategies to alleviate this problem. In subsequent studies, some parallel strategies such as co-evolutionary mechanism can be introduced.

In the future research, EChOA can be applied in multi-objective problems, social manufacturing optimization, and video coding optimization. Simultaneously, the ECHOA-SVM model that proposed in this paper can be studied in a larger scale. It can also be applied to other practical problems like data mining.