Adaptive Mutation Dynamic Search Fireworks Algorithm

The Dynamic Search Fireworks Algorithm (dynFWA) is an effective algorithm for solving optimization problems. However, dynFWA easily falls into local optimal solutions prematurely and it also has a slow convergence rate. In order to improve these problems, an adaptive mutation dynamic search fireworks algorithm (AMdynFWA) is introduced in this paper. The proposed algorithm applies the Gaussian mutation or the Levy mutation for the core firework (CF) with mutation probability. Our simulation compares the proposed algorithm with the FWA-Based algorithms and other swarm intelligence algorithms. The results show that the proposed algorithm achieves better overall performance on the standard test functions.


Introduction
Fireworks Algorithm (FWA) [1] is a new group of intelligent algorithms developed in recent years based on the natural phenomenon of simulating fireworks sparking, and can solve some optimization problems effectively.Compared with other intelligent algorithms such as particle swarm optimization and genetic algorithms, the FWA adopts a new type of explosive search mechanism, to calculate the explosion amplitude and the number of explosive sparks through the interaction mechanism between fireworks.
However, many researchers quickly find that traditional FWA has some disadvantages in solving optimization problems; the main disadvantages include slow convergence speed and low accuracy, thus, many improved algorithms have been proposed.So far, research on the FWA has concentrated on improving the operators.One of the most important improvements of the FWA is the enhanced fireworks algorithm (EFWA) [2], where the operators of the conventional FWA were thoroughly analyzed and revised.Based on the EFWA, an adaptive fireworks algorithm (AFWA) [3] was proposed, which was the first attempt to control the explosion amplitude without preset parameters by detecting the results of the search process.In [4], a dynamic search fireworks algorithm (dynFWA) was proposed which divided the fireworks into core firework and non-core fireworks according to the fitness value and adaptive adjustment of the explosion amplitude for the core firework.Based on the analysis of each operator of the fireworks algorithm, an improvement of fireworks algorithm (IFWA) [5] was proposed.Since the FWA was proposed, it has been applied to many areas [6], including digital filter design [7], nonnegative matrix factorization [8], spam detection [9], image identification [10], mass minimization of trusses with dynamic constraints [11], clustering [12], power loss minimization and voltage profile enhancement [13], etc.
The aforementioned dynFWA variants can improve the performance of FWA to some extent.However, the inhibition of premature convergence and solution accuracy improvement are still challenging issues that require further research on dynFWA.
In this paper, an adaptive mutation dynamic search fireworks algorithm (AMdynFWA) is presented.In AMdynFWA, the core firework chooses either Gaussian mutation or Levy mutation based on the mutation probability.When it chooses the Gaussian mutation, the local search ability of the algorithm will be enhanced, and by choosing Levy mutation, the ability of the algorithm to jump out of local optimization will be enhanced.
The paper is organized as follows.In Section 2, the dynamic search fireworks algorithm is introduced.The AMdynFWA is presented in Section 3. The simulation experiments and analysis of the results are given in detail in Section 4. Finally, the conclusion is summarized in Section 5.

Dynamic Search Fireworks Algorithm
The AMdynFWA is based on the dynFWA because it is very simple and it works stably.In this section, we will briefly introduce the framework and the operators of the dynFWA for further discussion.
Without the loss of generality, consider the following minimization problem: The object is to find an optimal x with a minimal evaluation (fitness) value.
In dynFWA, there are two important components: the explosion operator (the sparks generated by the explosion) and the selection strategy.

Explosion Operator
Each firework explodes and generates a certain number of explosion sparks within a certain range (explosion amplitude).The numbers of explosion sparks (Equation ( 2)) are calculated according to the qualities of the fireworks.
For each firework X i , its explosion sparks' number is calculated as follows: where y max = max (f (X i )), m is a constant to control the number of explosion sparks, and ε is the machine epsilon to avoid S i equal to 0. In order to limit the good fireworks that do not produce too many explosive sparks, while the poor fireworks do not produce enough sparks, its scope S i is defined as.
where a and b are fixed constant parameters that confine the range of the population size.
In dynFWA, fireworks are divided into two types: non-core fireworks and core firework, and the core firework (CF) is the firework with the best fitness, and is calculated by Equation (4).
The calculations of the amplitude of the non-core fireworks and the core firework are different.The non-core fireworks' explosion amplitudes (except for CF) are calculated just as in the previous versions of FWA: where y min = min f (X i ), A is a constant to control the explosion amplitude, and ε is the machine epsilon to avoid A i equal to 0. However, for the CF, its explosion amplitude is adjusted according to the search results in the last generation: where A CF (t) is the explosion amplitude of the CF in generation t.In the first generation, the CF is the best among all the randomly initialized fireworks, and its amplitude is preset to a constant number which is usually the diameter of the search space.
Algorithm 1 describes the process of the explosion operator in dynFWA.

Algorithm 1. Generating Explosion Sparks
Calculate the number of explosion sparks S i Calculate the non-core fireworks of explosion amplitude A i Calculate the core firework of explosion amplitude Where the operator % refers to the modulo operation, and X min k and X max k refer to the lower and upper bounds of the search space in dimension k.

Selection Strategy
In dynFWA, a selection method is applied, which is referred to as the Elitism-Random Selection method.In this selection process, the optima of the set will be selected firstly.Then, the other individuals are selected randomly.

Adaptive Mutation Dynamic Search Fireworks Algorithm
The mutation operation is an important step in the swarm intelligence algorithm.Different mutation schemes have different search characteristics.Zhou pointed out that the Gaussian mutation has a strong local development ability [14].Fei illustrated that the Levy mutation not only improves the global optimization ability of the algorithm, but also helps the algorithm jump out of the local optimal solution and keeps the diversity of the population [15].Thus, combining the Gaussian mutation with the Levy mutation is an effective way to improve the exploitation and exploration of dynFWA.
For the core firework, for each iteration, two mutation schemes are alternatives to be conducted based on a probability p.The new mutation strategy is defined as: where p is a probability parameter, X CF is the core firework in the current population, and the symbol ⊗ represents the dot product.Gaussian() is a random number generated by the normal distribution with mean parameter mu = 0 and standard deviation parameter sigma = 1, and Levy() is a random number generated by the Levy distribution, and it can be calculated with the parameter β = 1.5 [16].The value of E varies dynamically with the evolution of the population, with reference to the annealing function of the simulated annealing algorithm, and the value of E is expected to change exponentially, and it is calculated as follows: where t is the current function evaluations, and T max is the maximum number of function evaluations.
To sum up, another type of sparks, the mutation sparks, are generated based on an adaptive mutation process (Algorithm 2).This algorithm is performed N m times, each time with the core firework X CF (N m is a constant to control the number of mutation sparks).

Algorithm 2. Generating Mutation Sparks
Set the value of mutation probability p Find out the core firework X CF in current population Calculate the value of E by Equation ( 8) Where d is the number of dimensions, X min is the lower bound, and X max is the upper bound.As Figure 1 shows, the Levy mutation has a stronger perturbation effect than the Gaussian mutation.In the Levy mutation, the occasional larger values can effectively help jump out of the local optimum and keep the diversity of the population.On the contrary, the Gaussian mutation has better stability, which improves the local search ability.For the core firework, for each iteration, two mutation schemes are alternatives to be conducted based on a probability p.The new mutation strategy is defined as: , () ' (7) where p is a probability parameter, XCF is the core firework in the current population, and the symbol ⊗ represents the dot product.Gaussian() is a random number generated by the normal distribution with mean parameter mu = 0 and standard deviation parameter sigma = 1, and Levy() is a random number generated by the Levy distribution, and it can be calculated with the parameter β = 1.5 [16].The value of E varies dynamically with the evolution of the population, with reference to the annealing function of the simulated annealing algorithm, and the value of E is expected to change exponentially, and it is calculated as follows: where t is the current function evaluations, and Tmax is the maximum number of function evaluations.
To sum up, another type of sparks, the mutation sparks, are generated based on an adaptive mutation process (Algorithm 2).This algorithm is performed Nm times, each time with the core firework XCF (Nm is a constant to control the number of mutation sparks).

Algorithm 2. Generating Mutation Sparks
Set the value of mutation probability p Find out the core firework XCF in current population Calculate the value of E by Equation ( 8) Where d is the number of dimensions, Xmin is the lower bound, and Xmax is the upper bound.As Figure 1 shows, the Levy mutation has a stronger perturbation effect than the Gaussian mutation.In the Levy mutation, the occasional larger values can effectively help jump out of the local optimum and keep the diversity of the population.On the contrary, the Gaussian mutation has better stability, which improves the local search ability.
The flowchart of the adaptive mutation dynamic search fireworks algorithm (AMdynFWA) is shown in Figure 2.

Simulation Settings
Similar to dynFWA, the number of fireworks in AMdynFWA is set to five, the number of mutation sparks is also set to five, and the maximum number of sparks in each generation is set to 150.
In the experiment, the function of each algorithm is repeated 51 times, and the final results after 300,000 function evaluations are presented.In order to verify the performance of the algorithm proposed in this paper, we use the CEC2013 test set [16], including 28 different types of test functions, which are listed in Table 1.All experimental test function dimensions are set to 30, d = 30.

Simulation Settings
Similar to dynFWA, the number of fireworks in AMdynFWA is set to five, the number of mutation sparks is also set to five, and the maximum number of sparks in each generation is set to 150.
In the experiment, the function of each algorithm is repeated 51 times, and the final results after 300,000 function evaluations are presented.In order to verify the performance of the algorithm proposed in this paper, we use the CEC2013 test set [16]  Finally, we use the Matlab R2014a software on a PC with a 3.2 GHz CPU (Intel Core i5-3470), 4 GB RAM, and Windows 7 (64 bit).

Study on the Mutation Probability p
In AMdynFWA, the mutation probability p is introduced to control the probability of selecting the Gaussian and Levy mutations.To investigate the effects of the parameter, we compare the performance of AMdynFWA with different values of p.In this experiment, p is set to 0.1, 0.3, 0.5, 0.7, and 0.9, respectively.
Table 2 gives the computational results of AMdynFWA with different values of p, where 'Mean' is the mean best fitness value.The best results among the comparisons are shown in bold.It can be seen that p = 0.5 is suitable for unimodal problems f1 − f5.For f6 − f20, p = 0.3 has a better performance than the others.When p is set as 0.1 or 0.9, the algorithm obtains better performance on f21 − f28.
The above results demonstrate that the parameter p is problem-oriented.For different problems, different p may be required.In this paper, taking into account the average ranking, p = 0.3 is regarded as the relatively suitable value.To assess the performance of AMdynFWA, AMdynFWA is compared with enhanced fireworks algorithm (EFWA), dynamic search fireworks algorithms (dynFWA), and adaptive fireworks algorithm (AFWA), and the EFWA parameters are set in accordance with [2], the AFWA parameters are set in accordance with [3], and the dynFWA parameters are set in accordance with [4].
The probability p used in AMdynFWA is set to 0.3.For each test problem, each algorithm runs 51 times, all experimental test function dimensions are set as 30, and their mean errors and total number of rank 1 are reported in Table 3.
The results from Table 3 indicate that the total number of rank 1 of AMdynFWA (23) is the best of the four algorithms.Figure 3 shows a comparison of the average run-time cost in the 28 functions for AFWA, EFWA, dynFWA, and AMdynFWA.The results from Figure 3 indicate that the average run-time cost of EFWA is the most expensive among the four algorithms.The time cost of AFWA is the least, but the run-time cost of AMdynFWA is almost the same compared with AFWA.The run-time cost of AMdynFWA is less than that of dynFWA.Taking into account the results from Table 3, AMdynFWA performs significantly better than the other three algorithms.
To evaluate whether the AMdynFWA results were significantly different from those of the EFWA, AFWA, and dynFWA, the AMdynFWA mean results during the iteration for each test function were compared with those of the EFWA, AFWA, and dynFWA.The T test [17], which is safe and robust, was utilized at the 5% level to detect significant differences between these pairwise samples for each test function.
The ttest2 function in Matlab R2014a was used to run the T test, as shown in Table 4.The null hypothesis is that the results of EFWA, AFWA, and dynFWA are derived from distributions of equal mean, and in order to avoid increases of type I errors, we correct the p-values using the Holm's method, and order the p-values for the three hypotheses being tested from smallest to largest, and we then have three T tests.Thus, the p-value 0.05 is changed to 0.0167, 0.025, and 0.05, and then the corrected p-values were used to compare with the calculated p-values, respectively.Where the p-value is the result of the T test.The '+' indicates the rejection of the null hypothesis at the 5% significance level, and the '-' indicates the acceptance of the null hypothesis at the 5% significance level.
Table 5 indicates that AMdynFWA showed a large improvement over EFWA in most functions.However, in Unimodal Functions, AMdynFWA is not significant when compared with AFWA and dynFWA.In Basic Multimodal Functions and Composition Functions, the AMdynFWA also showed a large improvement over AFWA and dynFWA.Figure 4 shows the mean fitness searching curves of the 28 functions for EFWA, AFWA, dynFWA, and AMdynFWA.

Comparison of AMdynFWA with Other Swarm Intelligence Algorithms
In order to measure the relative performance of the AMdynFWA, a comparison among the AMdynFWA and the other swarm intelligence algorithms is conducted on the CEC2013 single objective benchmark suite.The algorithms compared here are described as follows.
The above four algorithms use the default settings.The comparison results of ABC, DE, CMS-ES, SPSO2011, and AMdynFWA are presented in Table 6, where the 'Mean error' is the mean error of the best fitness value.The best results among the comparisons are shown in bold.ABC beats the other algorithms on 12 functions (some differences are not significant), which is the most, but performs poorly on the other functions.CMA-ES performs extremely well on unimodal functions, but suffers from premature convergence on some complex functions.From Table 7, the AMdynFWA ranked the top three (22/28), which is better than the other algorithms (except the DE), and in terms of average ranking, the AMdynFWA performs the best among these five algorithms

Comparison of AMdynFWA with Other Swarm Intelligence Algorithms
In order to measure the relative performance of the AMdynFWA, a comparison among the AMdynFWA and the other swarm intelligence algorithms is conducted on the CEC2013 single objective benchmark suite.The algorithms compared here are described as follows.
(1) Artificial bee colony (ABC) [18]: A powerful swarm intelligence algorithm.6, where the 'Mean error' is the mean error of the best fitness value.The best results among the comparisons are shown in bold.ABC beats the other algorithms on 12 functions (some differences are not significant), which is the most, but performs poorly on the other functions.CMA-ES performs extremely well on unimodal functions, but suffers from premature convergence on some complex functions.From Table 7, the AMdynFWA ranked the top three (22/28), which is better than the other algorithms (except the DE), and in terms of average ranking, the AMdynFWA performs the best among these five algorithms on this benchmark suite due to its stability.DE and ABC take the second place and the third place, respectively.The performances of CMS-ES and the SPSO2011 are comparable.

Conclusions
AMdynFWA was developed by applying two mutation methods to dynFWA.It selects the Gaussian mutation or Levy mutation according to the mutation probability.We apply the CEC2013 standard functions to examine and compare the proposed algorithm AMdynFWA with ABC, DE, SPSO2011, CMS-ES, AFWA, EFWA, and dynFWA.The results clearly indicate that AMdynFWA can perform significantly better than the other seven algorithms in terms of solution accuracy and stability.Overall, the research demonstrates that AMdynFWA performed the best for solution accuracies.
The study on the mutation probability p demonstrates that there is no constant p for all the test problems, while p = 0.3 is regarded as the relatively suitable value for the current test suite.A dynamic p may be a good choice.This will be investigated in future work.

Figure 1 .
Figure 1.The value produced by the Levy mutation and Gaussian mutation.

Figure 1 . 17 Figure 2 .Algorithm 3 .
Figure 1.The value produced by the Levy mutation and Gaussian mutation.

( 2 )
Standard particle swarm optimization (SPSO2011) [19]: The most recent standard version of the famous swarm intelligence algorithm PSO.(3) Differential evolution (DE) [20]: One of the best evolutionary algorithms for optimization.(4) Covariance matrix adaptation evolution strategy (CMA-ES) [21]: A developed evolutionary algorithm.The above four algorithms use the default settings.The comparison results of ABC, DE, CMS-ES, SPSO2011, and AMdynFWA are presented in Table

Table 1 .
, including 28 different types of test functions, which are listed in Table 1.All experimental test function dimensions are set to 30, d = 30.CEC2013 test set.

Table 2 .
Mean value and average rankings achieved by AMdynFWA with different p, where the 'mean' indicates the mean best fitness value.

Table 3 .
Mean errors and total number of rank 1 achieved by EFWA, AFWA, dynFWA, and AMdynFWA.

Table 4 .
T test results of AMdynFWA compared with EFWA, AFWA and dynFWA.

Table 5 .
Total number of significance of AMdynFWA compared with EFWA, AFWA and dynFWA.

Table 7 .
Statistics of rank (SR) and average rankings (AR).