A Modified Artificial Bee Colony Algorithm Based on the Self-Learning Mechanism

Bao Pang 1, Yong Song 2,*, Chengjin Zhang 2, Hongling Wang 1 and Runtao Yang 2 1 School of Control Science and Engineering, Shandong University, Jinan 250061, China; pang_bao11@163.com (B.P.); wanghongling720409@163.com (H.W.) 2 School of Mechanical, Electrical and Information Engineering, Shandong University at Weihai, Weihai 264209, China; cjzhang@sdu.edu.cn (C.Z.); runtao-sd@163.com (R.Y.) * Correspondence: songyong@sdu.edu.cn; Tel.: +86-0631-5682389


Introduction
In recent years, swarm intelligence algorithms have received a wide spread attention. The artificial bee colony (ABC) algorithm is a relatively new approach that was proposed by Karaboga [1,2], motivated by the collective foraging behavior of honey bees. In the process of foraging, the bees need to find the place of food source with the highest nectar amount. In ABC system, artificial bees search in the given search space and the food sources represent possible solutions for the optimisation problems. The bees update the candidate solutions by means of solution search equation (SSE) and if the new solution is better than the previous one in their memory, they memorize the new position and forget the previous one. Due to its simplicity and ease of implementation, the ABC algorithm has captured much attention and has been applied successfully to a variety of fields, such as classification and function approximation [3], feature selection [4], inverse modelling of a solar collector [5], electric power system optimization [6], multi-objective optimisation [7], complex network optimization [8], transportation energy demand [9], large-scale service composition for cloud manufacturing [10], job-shop scheduling problem with no-wait constraint [11], respiratory disease detection from medical images [12].
Although the ABC algorithm has been widely used in different fields, some researchers have also pointed out that the ABC algorithm suffers from low solution accuracy and poor convergence performance. To solve the optimization problem, the intelligent optimization algorithm should combine global search methods, used to locate the potential optimal regions, with local search methods, used to fine-tune the candidate solutions, to balance exploration and exploitation process. However, exploration strategies and exploitation strategies contradict each other and to achieve good performance, they should be well balanced. While the SSE of ABC, which is used to generate new candidate solutions based on the information of the present solutions, does well in exploration but poorly in exploitation, which results in the poor convergence rate. Thus, many related and improved ABC algorithms have been proposed [13][14][15][16].
Inspired by a operator of global best (gbest) solution in particle swarm optimization (PSO) algorithm [17], Zhu and Kwong proposed a modified ABC algorithm called gbest-guided ABC (GABC) algorithm to improve the exploitation [18]; the gbest term in the modified SSE can drive the new candidate solution towards the global best solution. Although the GABC algorithm accelerated the convergence rate, the exploration performance decreased. Therefore, how to balance exploration and exploitation has become the main goal in the ABC research. Inspired by differential evolution [19], Gao and Liu introduced a new initialization approach and proposed an improved SSE which is based on that the bee searches only around the best solution of the previous iteration to improve the exploitation; by hybridizing the original SSE and the improved SSE with the fixed selective probability, the new search mechanism obtains better performance [20]. After that, based on the two SSEs, Gao and Liu proposed the modified ABC (MABC) algorithm which excludes the onlooker and scout bee stage. In MABC, a selective probability was introduced to balance exploration of the original SSE and exploitation of the improved SSE [21]; if the new candidate solution obtained using the improved SSE is worse than the original one, the bee uses the original SSE to generate a new candidate solution with a certain probability. To well balance exploration and exploitation, Akay and Karaboga constructed an adaptive scaling factor (SF) which regulates the range of parameter in SSE by using Rechenberg's 1/5 mutation rule; a smaller SF makes the candidate solution fine-tuned with a small steps while causing slow convergence rate and the bigger SF speeds up the search, but it reduces the exploitation performance [22]. In the original SSE in ABC algorithm, since the guidance of the last two term may be in opposite directions, it may cause an oscillation phenomenon. To overcome the oscillation phenomenon, Gao et al. presented a new SSE with two different candidate solutions selected from the solution space; moreover, an orthogonal learning strategy was developed to discover more effective information from the search experiences and to get more promising and efficient candidate solutions [23]. When the candidate solutions converge to the similar points, the SSE can cause a stagnation behavior during the search process, that means the value of the new candidate solution is the same with the value of the current solution. To overcome stagnation behavior of the algorithm, Babaoglu proposed a novel algorithm called distABC algorithm based on the distributed solution update rule, which uses the mean and standard deviation of the selected two solution to obtain a new candidate solution [24].
The above methods have achieved some progress, but there still exist some problems. The GABC algorithm improved the exploitation, but the exploration decreased. Even though MABC algorithm used two SSEs to balance exploration and exploitation, the selection mechanism and the fixed selective probability cannot adapt to the changing environment. Moreover, when the global best solution trapped in local optimum, GABC and MABC algorithm cannot escape from the local optimum effectively. The distABC algorithm overcame stagnation behavior, but distABC does poorly in exploitation. For population-based optimization methods, it is desirable to encourage the individuals to wander through the entire search space at the initial phase of the optimization; on the other hand, it is very important to fine-tune the candidate solutions in the succeeding phases of the optimization [25]. However, one SSE of original ABC algorithm cannot balance two aspects. Therefore, this paper proposes an achievable ABC algorithm which uses five SSEs as the candidate operator pool. The same with the SSE in ABC algorithm, the first SSE uses a solution selected randomly from the population to maintain population diversity and it emphasizes the exploration. Inspired by the PSO algorithm, the second SSE takes advantage of the information of the global best solution to guide the new candidate solution towards the global best solution. Therefore, the second SSE can improve the exploitation. To achieve good performance, the third SSE combines the above two SSEs which means that a randomly selected solution and the global best solution are all used in the SSE to balance exploration and exploitation. It seems that the global optimal solution is most likely around the best solution of the previous iteration. Therefore, the fourth SSE is the same with the one proposed in MABC algorithm which is based on that the bee searches only around the best solution of the previous iteration to improve the exploitation. When the candidate solutions trapped in local optimum, the above SSEs cannot escape from the local optimum effectively. To solve such problem, this paper proposes a novel SSE with Lévy flight step-size which can generate smaller step-size with high frequency and bigger step-size occasionally. The fifth SSE cannot only balance exploration and exploitation but also escape from the local optimum effectively. The five SSEs have both advantages and disadvantages and in order to make full use of the advantages of each SSE, this paper proposes a simple self-learning mechanism, wherein the SSE is selected according to the previous success ratio in generating promising solutions at each iteration. The SSE with a high success ratio means that the SSE can generate a better candidate solution with a large probability. Therefore, the self-learning mechanism cannot only select the appropriate SSE to generate new candidate solution but also adapt to the changing environment.
The following sections are organized as follows. Section 2 outlines the reviews of the classical ABC algorithm. Section 3 introduces the proposed self-learning mechanism (SLABC) algorithm. In Section 4, experiments are carried out to verify the effectiveness of SLABC algorithm based on nine benchmark functions in terms of t-test. Section 5 presents and discusses the experimental results. Finally, the conclusion is drawn in Section 6.

Classical ABC Algorithm
In the ABC algorithm, the colony of artificial bees contains three groups of bees: employed bees, onlooker bees and scouts bees [2]. Half of the colony consists of the employed bees, and another half consists of the onlooker bees. The scouts bees are transmuted from the inactive employed bees and then abandon their food source to search a new food source. Employed bees explore the food source in the search space and pass the food information to onlooker bees. Onlooker bees select the good food sources from those found by employed bees and further search the foods around the selected food source. The positions of the food sources are initialized in the search space and food sources present possible solutions for the optimization problem. There are SN solutions, where SN denotes the size of employed bees or onlooker bees. Suppose x i = {x i,1 , x i,2 , ..., x i,D } is the position of the ith solution and D is the number of dimension to be optimized. The flow of ABC algorithm is shown as follows.

Employed Bee Stage
At this stage, each employed bee search around the given solution x i and let v i = x i . In the update process, the new candidate solution v i = {v i,1 , v i,2 , ..., v i,D } is produced by using SSE as follows: where k ∈ {1, 2, ..., SN} and j ∈ {1, 2, ..., D} are randomly chosen indexes; k has to be different from i; φ i,j is a random number in the range [−1, 1]. Then, a greedy selection mechanism is applied between x i and v i to select a better solution. After all the employed bees complete their searches, they share the solution information to the onlooker bees.

Onlooker Bee Stage
According to the fitness value of each solution, onlooker bees calculate the probability value P i associated with that solution, where f it i is the fitness value of solution i and SN is the number of solutions. Based on P i and roulette wheel selection method, an onlooker bee selects one solution to update. After selecting the solution x i , the onlooker bee update it by using Equation (1) and the greedy selection mechanism is also used to select a better solution. At this stage, only the selected solution can be updated and the better solutions may be updated many times.

Scouts Bee Stage
If a solution cannot be improved further at least limit times, this solution is assumed to be abandoned and a new solution will be produced randomly in the search space to replace the abandoned one. This operation can be defined as follows:

SLABC Algorithm
To improve the performance, the SLABC algorithm uses five SSEs as the candidate operator pool. One of the SSEs with Lévy flight step-size cannot only balance exploration and exploitation but also avoid trapping in the local optimum. This section first introduces the Lévy flight in detail.

Lévy Flight Step-Size
A Lévy flight is a random walk and the step-size satisfies a probability distribution which can be expressed as follows [26]: where s is the step-size with 1<λ ≤ 3. Lévy flight can generate smaller step-size with high frequency and generate larger step-size occasionally. In the search process, the bee with a large step-size can reach anywhere of the entire search space to locate the potential optimal solution; when the bees are trapped in the local optimum, the large step-size can make the bees escape from the local optimum. The bees with a small step-size tend to fine-tune the current solution to obtain the optimal solution. The foraging behaviors of many creatures in nature satisfy Lévy flight, such as albatrosses' foraging flight trajectory [27,28] and drosophilas' intermittent foraging flight trajectory [29]. Viswanathan et al. suggest that Lévy flight is an optimal search strategy when the target sites are sparse and distributed randomly [26]. This paper uses the method proposed by [30] to calculate Lévy flight step-size: where β ∈ [0.3, 1.99], u and v are two normal stochastic variables with standard deviation σ u and σ v .
where the notation Γ(z) is gamma function. If the real part of the complex number z is positive (Re(z) > 0), then the integral converges absolutely. As shown in Figure 1, Lévy flight with a mix of large step-size and small step-size can balance exploration and exploitation. Therefore, SLABC introduces Lévy flight step-size to the modified SSEs to improve the performance.

The Modified Solution Search Equations
To solve the optimization problem, the intelligent optimization algorithm should combine global search methods with local search methods to balance exploration and exploitation. However, exploration strategies and exploitation strategies contradict each other and one SSE in ABC algorithm cannot balance two aspects. Therefore, this paper proposes an achievable ABC algorithm which uses five SSEs as the candidate operator pool.
Following the classical ABC algorithm, SLABC employs the original SSE as the first search equations to improve exploration of SLABC algorithm.
To improve exploitation, the second SSE introduces the global best solution to guide the new candidate solutions towards the global best solution.
where x gbest is the global best solution. Generally speaking, there may be a better solution around the global best solution; therefore, we set random number c 2 in the range [0.75, 1.25]. The Equation (9) is good at exploration and Equation (10) is good at exploitation, therefore to well balance exploration and exploitation, the third SSE combine the two equations as follows.
where r2 ∈ {1, 2, ..., SN} are randomly chosen indexes and r2 is different with i; c 3 ∈ [−0.5, 0.5] and c 4 ∈ [0.5, 1.5] are random numbers. It seems that the global optimal solution is most likely around the best solution of the previous iteration. Therefore, the fourth SSE is the same with the one proposed in MABC algorithm which is based on that the bee searches only around the best solution of the previous iteration to improve the exploitation. v where r3 and r4 are mutually different random integer indices selected from {1, 2, ..., SN}; c 5 is a random number in the range [−0.5, 0.5].
Because the above four SSEs are based on the current solutions, when the present solutions converge to the similar point or are trapped in local optimum, the above SSEs cannot escape from the local minimum effectively. Therefore, this paper introduces Lévy flight step-size to the last SSE to solve this problem. v where s is the Lévy flight step-size which can be calculated in Equation (5). Different from the GABC, MABC algorithm, the range of the weight c 3 , c 4 , c 5 in Equations (11) and (12) with the g best term are all reduced and the interval length are set to 1. Therefore, the new generated candidate solution can be in a smaller range and the accuracy of the solution will be improved. Moreover, to make the candidate solution nearer to the global best solution, the range of the weight c 2 in Equation (10) is further reduced and interval length are set to 0.5.
After all the employed (onlooker) bees produce the new candidate solutions using the improved SSEs, then, a greedy selection mechanism is used to select a better solution between x i and v i .

Self-Learning Mechanism
The five SSEs are regarded as the candidate operator pool and different operator is more effective at different stage. At each iteration, each employed (onlooker) bee will select a SSE from the candidate operator pool to update the corresponding solution. This paper proposes a simple self-learning mechanism to realize such optimal choice, wherein the SSE is selected according to the previous success ratio in generating promising solutions at each iteration.
In the self-learning mechanism, each SSE is assigned to a probability: success ratio. It is defined as where S k denotes the counter that records the number of successful updating times of the k-th SSE, where the new candidate solution is better than the old one; T k is the total number of updating times of the k-th SSE is selected; Srat k is the success ratio of the k-th SSE. At each iteration, each SSE k is selected according to the success ratio Srat k through roulette wheel selection. The new candidate solutions are then produced by the selected SSE. In the initialization stage, each success ratio Srate k is given an equal selection probability. It is well known that at the early stage of the optimization, the bees are inclined to locate the potential optimal regions by wandering through the entire search space. Conversely, most of the bees are apt to fine-tune the present solutions to obtain the global optimal solution at the latter stage of the optimization. Therefore, in order to avoid the interference between the early stage and the latter stage, this paper divides the whole optimization process into two stages. At each stage, S k , T k and the success ratio Srate k all will be initialized.

Description of the SLABC Algorithm
The pseudo code of the SLABC algorithm can be described as follows (Algorithm 1):

Algorithm 1
Initialize the population of the bees as N and SN = N/2; set the number of trials as limit. Randomly generate SN points x i (i = 1, 2, ..., SN) in the search space to form an initial solution. Find the global best solution gbest and the its position x gbest from the SN points. Set the maximum number of function evaluations, Max.FE; S k = 1, T k = 1 and Srate k = 0.2.
Initialize S k , T k and Srate k . End If Employed bee stage: Randomly choose a SSE k from the candidate strategy pool through roulette wheel selection and count T k = T k + 1. Update the candidate solution v i using the selected SSE.

End If End For
Update the gbest and x gbest and calculate the probability value P i using Equation (2). Onlooker bee stage: for i = 1 to SN Select one solution to update based on P i and roulette wheel selection. The update process is same as that in the employed bee stage.

End For
Update the gbest and x gbest and calculate the probability value P i using Equation (2).

Scout stage:
If trial(i) > limit Initialize x i with a new randomly generated point in the search space.

Experimental Setup
To investigate the performance of the SLABC algorithm, 9 benchmark functions shown in Table 1 were used, including four unimodal functions ( f 1 − f 4 ) and five multimodal functions ( f 5 − f 9 ). In Table 1, D denotes the dimensions of the solution space and 30, 60, and 100 dimensions are used in the present paper. The unimodal functions can be used to analyse the convergence rate of the algorithms. The multimodal functions are commonly used to show whether the algorithms can escape from the local optimum effectively.
The effectiveness of the proposed SLABC algorithm was evaluated by comparing its results with other related algorithms, such as ABC [2], GABC [20], MABC [21], and distABC [24]. To make a fair comparison among ABCs, all the algorithms were tested using the same parameters: the population size N = 100, limit = 100, the maximum number of iterations, Max.FE = 2000. Additionally, other specific parameters of each comparison algorithm are same as in ABC [2], GABC [20], MABC [21], and distABC [24]. In order to ensure the experiment results stability, we repeated each algorithm for 20 times and average the results.

Unimodal
Sphere

Experimental Results
This paper uses five indexes to analysis the experimental results: the best solution (Best), median solution (Median), worst solution (Worst), average solution (Mean), and standard deviation (Std). Tables 2-4 show the experimental results obtained by each algorithm in the 20 independent runs. The results suggest that SLABC offers the higher solution accuracy on almost all the functions except function f 4 with D = 30, 60, 100. It can be seen from the formulation of Rosenbrock function ( f 4 ) that the first term 100(x i+1 − x i ) 2 mainly influence the function value, which means that only the values among different dimensions in the candidate solutions are almost the same, the function value is smaller. Using the mean and standard deviation of the selected two solution, the distABC algorithm obtained the new candidate solutions, in which the values among different dimensions are of uniform size. Therefore, based on the distributed solution update rule, distABC algorithm gets the best results on Rosenbrock function ( f 4 ).
Moreover, in the case of functions f 5 , f 6 , f 8 with D = 30, the solution accuracy of SLABC are equal with the best of other algorithms. The Rastrigin function ( f 5 ), Griewank function ( f 6 ) and Schwefel's Problem 2.26 function ( f 8 ) are multimodal functions and are easy to obtain the optimal solutions with D = 30. As the dimension gets higher, the difficulty of obtaining the optimal solution increases gradually. However, the solution accuracy of SLABC are better than the functions f 5 , f 6 , f 8 with D = 60 and D = 100, which proves that SLABC algorithm outperforms the other algorithms.        As shown in these figures, the self-learning mechanism selected different SSE in solving different functions and different functions have different optimal combination of SSEs. In the figures, the coordinate 2 is corresponding to the second SSE at the early stage and the coordinate 7 is corresponding to the second SSE at the latter stage. In most of the functions, the second SSE (coordinate 2 and coordinate 7) has the highest success rate and the fifth SSE has the lowest success rate. The second SSE with the global best solution can improve exploitation. At the early stage of the optimization, the bees are inclined to locate the promising regions by wandering through the entire search space. At the latter stage, the bees are busy to fine-tune the present solutions to obtain the global optimal solution. Therefore, these solutions may trap in the local minimum at the latter stage of the optimization process. The fifth SSE is mainly to make the bees escape from the local minimum. Because functions f 1-f 4 are unimodal functions, the fifth SSE with lévy flight step-size has a lower success ratio at the latter stage. However, functions f 5-f 9 are multimodal functions which have many local minimum in the search space. The higher success ratio indicates that the SSE with lévy flight step-size can escape from the local optimum effectively to improve search efficiency.

Comparison Regarding the t-Test
This section mainly analyzes the experimental data and study whether the experimental results are statistically significantly different between SLABC and other algorithm. The F-test was used to analyze the homogeneity of variances and two-tailed t-test was used to determine if two sets of data are significantly different from each other. The sample size and number of degrees of freedom were set as 20 and 38, respectively. The confidence level was set to 95%. In the F-test, F 0.05 (19,19) = 2.17 and if the F-value is larger than 2.17, the corresponding experimental results are heterogeneity of variance. If the p-value received in the t-test is less than 5%, it illustrates that the corresponding experimental results are significantly different. Table 5 lists the results of t-tests between SLABC and the best results of the other algorithms regarding the indexes "Mean" for different benchmark functions (listed are the F-value, p-value, and the significance of the results). "YES" indicates that the experimental results are significantly different between SLABC and the best one in other algorithms. "NO" suggests that no significant difference between the results of SLABC and of other algorithms.
From Tables 2-5, although other algorithms (distABC) have higher solution accuracy on function f 4 with D = 30, 60, 100, there exist no significant difference between SLABC and distABC. The same as function f 6 with D = 100 and function f 8 with D = 60, there exist no significant difference between SLABC and other algorithm. Moreover, SLABC and other algorithm obtain the same optimal solution on functions f 5, f 6, f 8 with D = 30.
Based on the results of statistical tests in Table 5, almost most of the results have obvious differences and it is clear that the results of SLABC algorithm are significantly better than the results of other algorithms.

Discussion
In the previous sections, comparative results of ABC, GABC, MABC and distABC were presented. In this section, we offer a thorough analysis on the proposed SLABC algorithm and all the algorithms were tested using the same parameters with Section 4 .
When constructing the SSE, the value range of the coefficients were adjusted to improve the performance of the SLABC algorithm. This section constructed two SLABC algorithm variants (SLABC1, SLABC2) and experiments on a set of 6 benchmark functions were carried out to clearly show how these coefficients influence the performance in various optimization problems. In SLABC1, the value range of the coefficient c 1 , c 2 , c 3 , c 4 , c 5 are reduced to half of the corresponding value range in SLABC. In SLABC2, the value range of the coefficient c 1 , c 2 , c 3 , c 4 , c 5 are increased to double of the corresponding value range in SLABC. The new generated candidate solutions can appear in a large range using the SLABC2 algorithm and can appear in a small range using the SLABC1 algorithm. From Table 6, the SLABC2 is worse than SLABC and SLABC1, which shows that the increased value range reduced the performance of the SLABC algorithm. Moreover, SLABC is better than SLABC1 except function f 4 , which means that too small value range can reduce solution accuracy. Through the analysis of experimental data, the changes of the value range can influence the performance of SLABC algorithm and only the appropriate value range can generate the better solutions.
Five SSEs are used to solve the optimization problem and each SSE has different effect. For example, the fifth SSE with Lévy flight step-size can help the artificial bees escape from the local optimum effectively. However, when solving the optimization problem of unimodal function, the fifth SSE is ineffective. This section constructed another SLABC algorithm variants (SLABC3 without the fifth SSEs) to illustrate the effect of such combination. As can be seen from Table 6, SLABC3 obtained the better solution than SLABC on the unimodal functions f 1 , f 2 , f 3 . Therefore, we can choose combination of different SSEs to solve different optimization problems in the future applications. Best 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 Median 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 f 5 Worst 0.00 × 10 0 0.00 × 10 0 9.95 × 10 −1 0.00 × 10 0 0.00 × 10 0 Mean 0.00 × 10 0 0.00 × 10 0 4.97 × 10 −2 0.00 × 10 0 0.00 × 10 0 Std 0.00 × 10 0 0.00 × 10 0 2.22 × 10 −1 0.00 × 10 0 0.00 × 10 0 Best 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 Median 2.00 × 10 −14 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 0.00 × 10 0 In order to avoid the interference between the early stage and the latter stage, SLABC algorithm divides the whole optimization process into two stages. We can divide the optimization process into much more stages to further reduce interference between each stage. To show the effect of such division, this section constructed another SLABC algorithm variants (SLABC4) which divides the whole optimization process into ten stages. As can be seen from Table 6, SLABC4 obtained the better solution than SLABC on most of the functions. Dividing the optimization process into several stages can improve the performance of SLABC algorithm and how to divide the optimization process remains to be a problem should be further studied.
This paper proposes a self-learning mechanism to select the appropriate SSE according to the previous success ratio. Such mechanism is a reinforcement mechanism and other optimization algorithms can construct novel algorithms variants to improve the performance by using the self-learning mechanism.

Conclusions
This paper proposes an improved ABC algorithm based on the self-learning mechanism with five SSEs as the candidate operator pool. Among them, one SSE is good at exploration; other two SSEs are good at exploitation; another SSE intends to balance the two aspects. The last SSE with Lévy flight step-size can avoid trapping in local optimal solution. Meanwhile, a simple self-learning mechanism is proposed, wherein the SSE is selected according to the previous success ratio in generating promising solutions at each iteration. Experiments verify that the proposed SLABC algorithm can improve search efficiency and speed up the convergence rate.