Multi-Leader Comprehensive Learning Particle Swarm Optimization with Adaptive Mutation for Economic Load Dispatch Problems

Particle swarm optimization (PSO) is one of the most popular, nature inspired optimization algorithms. The canonical PSO is easy to implement and converges fast, however, it suffers from premature convergence. The comprehensive learning particle swarm optimization (CLPSO) can achieve high exploration while it converges relatively slowly on unimodal problems. To enhance the exploitation of CLPSO without significantly impairing its exploration, a multi-leader (ML) strategy is combined with CLPSO. In ML strategy, a group of top ranked particles act as the leaders to guide the motion of the whole swarm. Each particle is randomly assigned with an individual leader and the leader is refreshed dynamically during the optimization process. To activate the stagnated particles, an adaptive mutation (AM) strategy is introduced. Combining the ML and the AM strategies with CLPSO simultaneously, the resultant algorithm is referred to as multi-leader comprehensive learning particle swarm optimization with adaptive mutation (ML-CLPSO-AM). To evaluate the performance of ML-CLPSO-AM, the CEC2017 test suite was employed. The test results indicate that ML-CLPSO-AM performs better than ten popular PSO variants and six other types of representative evolutionary algorithms and meta-heuristics. To validate the effectiveness of ML-CLPSO-AM in real-life applications, ML-CLPSO-AM was applied to economic load dispatch (ELD) problems.


Introduction
Optimization problems are commonly found in science and engineering applications.Nowadays, these optimization problems are getting more and more complex.Traditional optimization methods such as least square approximation, gradient descent and Newton methods belong to single point optimizers and need gradient information.Hence, most of the traditional optimizers are unfit for complex multimodal problems and non-differentiable optimization problems [1].To cope with complex optimization problems, several swarm intelligence (SI) algorithms such as particle swarm optimization (PSO) [2,3], ant colony optimization (ACO) [4], artificial bee colony (ABC) [5], cuckoo search algorithm (CS) [6,7], grey wolf optimizer (GWO) [8], et al. have been proposed over the past decades.Brezočnik et al. [9] reviewed the recent development ofSI algorithms.As one of the widely used SI algorithms, PSO is easy to implement and converges fast.Since its inception, PSO has attracted great interest from the evolutionary computation (EC) community, and theoretical Energies 2019, 12, 116 2 of 27 researchers [10][11][12][13], and real-life applications [14][15][16][17][18] of PSO have been reported over the past two decades.Zhang et al. [19] have summarized the recent advancements in PSO.
Canonical PSO moves the particle by the attractive force from the global best position (Gbest) and the particle's own personnel best position (Pbest).This mechanism can a obtain high convergence rate, however, the canonical PSO suffers from premature convergenceon complex multimodal problem.In the early stage of optimization, the diversity of canonical PSO is lost quickly and the algorithm's exploration performance weakens, while in the latter stage, the particles crowd in the neighborhood of Gbest and slow down the algorithm's convergence rate.
Among the aforementioned four categories of improved PSO variants, the learning strategy has a significant impact on the performance of PSO.A lot of PSO variants with novel learning strategies have been proposed recently.For example, inspired by learning principles found in human cognitive psychology, Tanweer et al. [1,39,40] proposed self-regulating particle swarm optimization (SRPSO).In SRPSO, a self-regulating inertia weight is employed by the best particle for better exploration, and self-perception of the global search direction is employed by the rest of particles for intelligent exploitation of the solution space.Qin et al. [41] proposed particle swarm optimization with inter-swarm interactive learning strategy (IIL-PSO).Mo et al. [42] transferred an attractive and repulsive interacting mechanism into PSO and proposed attractive and repulsive fully informed particle swarm optimization based on the modified fitness model (ARFIPSOMF).To solve the premature convergence problem in traditional PSO, Dong et al. [43] proposed an opposition based particle swarm optimization with adaptive mutation strategy (AMOPSO).Wang et al. [44] proposed committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems.Ye et al. [45] presented multi-swarm particle swarm optimization with dynamic learning strategy (PSO-DLS).Dong et al. [46] presented a supervised learning and control method to improve particle swarm optimization algorithms.Yang et al. [12] proposed segment-based predominant learning particle swarm optimizer for large-scale optimization through letting several predominant particles guide the learning of a particle.Cao et al. [11] proposed neighbor-based learning particle swarm optimizer (NLPSO) with short-term and long-term memory for dynamic optimization problems.Wang et al. [47] introduced hybrid particle swarm optimization algorithm using adaptive learning strategy (ALPSO).
To discourage premature convergence, Liang et al. [48] proposed comprehensive learning particle swarm optimization (CLPSO).CLPSO uses a novel comprehensive learning strategy whereby all other particle's Pbest is used to update a particle's velocity.The comprehensive learning strategy of CLPSO is widely used to obtained high performance on multimodal problems, however, CLPSO cancels the global learning component and converges slowly on unimodal functions.Nasir et al. [49] retained the global learning component for CLPSO to achieve a high convergence rate, nevertheless, all the particles learning from the sole Gbest in the social learning component may impair the algorithm's exploration.Hence to enhance exploitation of CLPSO without significantly weakening its exploration, a multi-leader (ML) strategy is combined with CLPSO and the resultant algorithm is called multi-leader comprehensive learning particle swarm optimization (ML-CLPSO).ML-CLPSO constructs a candidate leader set by using a group of top ranked particles and enables each particle to learn from a randomly selected leader from the candidate leader set in the social learning part.The candidate leader set and the leaders of the particles are refreshed dynamically during the optimization to mitigate premature convergence.To activate the stagnated particles, an adaptive mutation (AM) strategy is incorporated into ML-CLPSO and the resultant algorithm is referred to as multi-leader comprehensive learning particle swarm optimization with adaptive mutation (ML-CLPSO-AM).The adaptive mutation employs the distribution information of the candidate leader set to transfer the stagnated particle to a potentially promising area.To evaluate the impact of multi-leader and adaptive mutation strategies, they were combined with CLPSO both separately and jointly.The latest CEC2017 test suite was adopted to test the performance of the proposed methods.The test results were compared with ten representative PSO variants, six evolutionary algorithms (EAs) and meta-heuristics.The ten selected PSO variants are CLPSO [48], comprehensive learning particle swarm optimization with Gbest (CLPSO-G) [49], particle swarm optimization with constriction factor (PSO-cf) [50], fully informed particle swarm (FIPS) [23], fitness-distance-ratio based particle swarm optimization (FDR-PSO) [28], social learning particle swarm optimization (SL-PSO) [51], enhanced leader particle swarm optimization (ELPSO) [52], ensemble particle swarm optimizer (EPSO) [53], genetic learning particle swarm optimization (GL-PSO) [54], and heterogeneous comprehensive learning particle swarm optimization (HCLPSO) [55].The six EAs and meta-heuristics include adaptive differential evolution with optional external archive (JADE) [56], artificial bee colony (ABC) [5], evolution strategy with covariance matrix adaptation (CMA-ES) [57], grey wolf optimizer (GWO) [8], across neighbor search (ANS) [58] and salp swarm algorithm (SSA) [59].The test results showed that the ML strategy can enhance exploitation of CLPSO without significantly weakening its exploration, while the AM strategy can activate the stagnated particles to further improve the performance of ML-CLPSO.The ML strategy and AM strategy are compatible, and combining both of them with CLPSO, the resultant ML-CLPSO-AM yields high performance.ML-CLPSO-AM outperformed ten popular PSO variants, six EAs and meta-heuristics in the CEC 2017 test suite.The test results for ELD problems showed that ML-CLPSO-AM is applicable to real-life optimization problems.
The rest of this paper is organized as follows: Section 2 reviews the related work, Section 3 introduces the methodologies, Section 3 reports the experimental results, Section 5 examines the application to ELD problems, and Section 6 concludes the paper.

Canonical PSO
In canonical PSO, every candidate solution is regarded as a particle.The particles fly in the search space to find the optimal solution by the attractive force from Pbests and Gbest.The velocities and positions of the particles are updated according to Equations ( 1) and (2) [60].
where I = 1, 2, . . ., Ps stands for the index of particle.Ps denotes the swarm size.The velocity and position vector of the ith particle are denoted by Vi = (v i,1 , v i,2 , . . ., v i,D ) and Xi = (x i,1 , x i,2 , . . ., x i,D ), respectively.D is the dimensionality.Ω is the inertia weight, c 1 and c 2 are two acceleration coefficients.r 1 and r 2 are two uniformly distributed random numbers within the range of [0,1].The velocity of canonical PSO is composed of three parts, namely the velocity of the previous iteration, the cognition learning part and the social learning part.In canonical PSO, the particle learns from its own Pbest to preserve swarm diversity in the cognition learning part and learns from Gbest to enhance exploitation in the social learning part.The whole swarm learns from the sole Gbest and can obtain high convergence speed, however, this also causes rapid loss of swarm diversity and premature convergence.The canonical PSO only uses the information of the particle's own Pbests and Gbest to move the particles, while the information of other elite neighbor particles is not utilized to enhance exploration, therefore, it does not perform well on complex multimodal problems.

CLPSO
To enhance exploration of PSO, Liang et al. [48] proposed comprehensive learning particle swarm optimization (CLPSO).CLPSO introduced a novel comprehensive learning (CL) strategy where each dimension of one particle can potentially learn from different particles' Pbests.The velocity of the particles in CLPSO is updated according to Equation (3).
The comprehensive learning list fi = [fi(1), fi(2), . . ., fi(D)] defines which particles' Pbests the ith particle should follow.rand i,d ∈[0,1] is a random number.The Pbest fi(d),d records each dimension of one particle should learn from which particles' Pbest, and the decision is determined by probability Pc, which is referred to as CL probability.The Pc is generated according to Equation (4) [48].
where a and b are two constants.a = 0.05, b = 0.45 [48].A random number is generated for every dimension of the ith particle.If this random number is below Pc(i), the relevant dimension will learn from the better particle's Pbest within two tournament selected particles, otherwise, the relevant dimension will learn from its own Pbest.If the fitness value of one particle ceases to improve for a predefined number of continuous iterations, CLPSO will refresh the exemplar of that particle.The predefined number is called the refreshing gap.CLPSO enlarges the search range of PSO and exhibits high performance on multimodal problems.Because CLPSO has no independent global learning component, it converges slowly on unimodal problems.To improve the performance of CLPSO, Lynn et al. [55] employed two sub-groups of swarms to focus solely on either exploration or exploitation and proposed heterogeneous comprehensive learning particle swarm optimization (HCLPSO).HCLPSO employs CLPSO and CLPSO-G for the exploration sub-group and the exploitation sub-group, respectively.The velocity of CLPSO-G is updated according to Equation (5).
where rand1 i,d , rand2 i,d ∈[0, 1] are two random numbers.CLPSO-G added a global learning component to the original CLPSO, so it converges relatively faster.The acceleration coefficients are linearly adjusted in CLPSO-G to obtain better performance.Lynn et al. [53] also adopted the CLPSO-G component in ensemble particle swarm optimizer (EPSO).

The Social Learning Leader
In the social learning part, the particles learn from other particles' Pbests.In this study, the social learning exemplar is referred to as the leader.The selection and utilization of the leader is very important to the performance of PSO.Chen et al. [61] transferred the aging mechanism to PSO and proposed particle swarm optimization with an aging leader and challengers (ALC-PSO).In ALC-PSO, the leader is assigned with limited life of iterations, and the life of the leader is adjusted dynamically during the evolution process according to the evolutionary state of the particles.Once the leader gets too old, a challenger is generated to challenge the old leader.ALC-PSO can mitigate premature convergence to some extent, however, the whole swarm sharing the sole leader may cause the rapid loss of diversity.Zhang et al. [56] presented a novel DE/current-to-pbest/1 mutation strategy.The DE/current-to-pbest/1 employs top 100p% individuals in the current swarm to generate a trial vector of differential evolution (DE).The test results prove that the DE/current-to-pbest/1 performs better than DE/current-to-best/1 and thus it is widely used in many DE variants.Borrowing from DE/current-to-pbest/1, the idea of employing the top 100p% particles' Pbest instead of Gbest to guide the motion of particles may yield promising improvements.Yu et al. [62] proposed multiple learning PSO with space transformation perturbation (MLPSO-STP).In MLPSO-STP the particles learn from the top 100p% particles and mean positions of the current swarm.The test results prove MLPSO-STP performs better than classical PSO variants on selected benchmark functions.Tanweer et al. [1] divided the whole swarm into mentors, independents and mentees according to the particle's fitness value and the Euclidian distance between the particle and Gbest.The high-quality mentors guide low quality mentees and the manner of learning different dimensions from different particle is adopted.

Mutation
To enhance swarm diversity and ease premature convergence, the mutation factor of genetic algorithms (GA) [63] can be transferred into the PSO algorithm.To improve the performance of PSO on multimodal problems, Higashi et al. [64] combined particle swarm optimization with Gaussian mutation.The test results showed that the proposed PSO with Gaussian mutation performed better than both GA and PSO on the tested problems.Wei et al. [65] employed Gaussian mutation to increase the diversity of PSO and decrease the risk of plunging into local optima.To mitigate premature convergence, Jordehi [52] employed five succeeding stages of mutations to the swarm leader per iteration, and proposed enhanced leader particle swarm optimization (ELPSO).The Gaussian mutation, Cauchy mutation, dimension by dimension opposition-based mutation, all dimension opposition-based mutation and DE-based mutation are applied to the swarm leader to improve its fitness value.The test results proved that ELPSO obtains high performance in terms of accuracy, scalability and convergence rate.The above mentioned publications indicate that combining mutation with PSO can enhance diversity and improve the performance of PSO on multimodal problem.For more PSO variants with mutation operators, please refer to [41,[66][67][68][69].

Multi-Leader Strategy
Section 2.3 analyzed the importance of the social learning leader.In the widely used global version PSO, Gbest is the sole leader.However, if the whole swarm learns from the sole leader, this may cause the swarm to quickly lose diversity and suffer from premature convergence in multimodal problems.On the other hand, CLPSO and FIPS have no independent component learning from Gbest, so they converge too slowly on unimodal problems.To enhance the exploitation of CLPSO without weakening its exploration, the ML strategy is combined with CLPSO; the resultant PSO algorithm is referred to as ML-CLPSO.In the ML strategy, a group of top ranked particles constitutes the candidate leader set and the candidate leader set is updated dynamically.In the initialization, each particle selects a leader randomly from the candidate leader set.The velocity of ML-CLPSO is updated according to Equation (6).
where Leader(i) is an index of particle, it records the ith particle, which should learn from that particle's Pbest in the social learning part.The learning behavior of ML-CLPSO is divided into two parts, the comprehensive learning part and the social learning part.The particles learn from CL exemplars in the comprehensive learning part to enhance exploration and learn from high quality leaders in the social learning part to enhance exploitation.Each particle learns from a different leader and the leader of the particle is updated dynamically during the optimization process.When the particle's fitness value has not been improved within a refreshing gap, the particle's leader will be updated together with its comprehensive learning exemplar.To minimize problems, the improvement in the fitness value means one particle finds a solution with a lower fitness value than its Pbest.When updating the leader of one particle, the particles are sorted by fitness value and a group of top ranked particles constructs the new candidate leader set.The particle's new leader is selected from the new candidate leader set by random selection.Rantanweera et al. [70] pointed out "in population-based optimization methods, it is desirable to encourage the individuals to wander through the entire search space, without clustering around local optima, during the early stages of the optimization.On the other hand, during the latter stages, it is very important to enhance convergence toward the global optima, to find the optimum solution efficiently."Hence, to enhance exploration in the early stage and enhance exploitation in the latter stage, the linearly adjusting inertia weight and acceleration coefficients are adopted.ω and c 1 are initialized with high value and decreases linearly, while c 2 is started with low value and increases linearly.The pseudo code of ML-CLPSO is presented in Algorithm 1.In the initialization, the swarm size, search range, position, velocity, Pbest and Gbest of particles are initialized.The inertia weight and acceleration coefficients are also assigned with initial values.The ω, c 1 and c 2 are linearly adjusted according to Table 1 in Section 4.2.The ct and Max_FES denote the count of currently consumed function evolutions (FEs) and the maximum FEs.Ps is the swarm size.The X(i), Pbest(i) and Pbestval(i) stand for the ith particle's position, Pbest and fitness value, respectively.fit(•) is the function for calculating the fitness value.CLS is an integer array for recording the indexes of candidate leader particles.r 1 ∈[0,1] is a random number.sort(•) and ceil(•) are two Matlab functions.The sort(•) function sorts the input array in ascending order and the ceil(•) function returns the input variable's nearest integer in the direction of positive infinity.Stag1 and Stag2 are two independent integer arrays to count the continuous number of iterations without improving the particles' fitness values.Stag1m is the refreshing gap for updating the CL exemplar and leader.Stag2m is the mutation gap, which will be introduced in Section 3.2.N L is the size of the candidate leader set.The alternative steps are only executed when the AM strategy is employed.In ML-CLPSO, the leader of one particle is updated together with its comprehensive learning exemplar.When one particle's leader needs updating, ML-CLPSO will sort the particles by fitness value to refresh the candidate leader set, then select the particle's new leader randomly from the candidate leader set.

Adaptive Mutation Strategy
Because PSO adopts greedy selection to update Pbests, in complex multimodal problems some particles may consume many continuous iterations (for example, 30 iterations) without improving their fitness value.These particles are called stagnated particles.The stagnated particles waste numerous Fes without moving their Pbests towards a promising area, thereby, the performance of the PSO algorithm is degenerated.To activate the stagnated particles, the AM strategy is introduced.If one particle is detected in stagnation, the AM strategy will generate a new Pbest for the stagnated particle.Unlike traditional mutation strategy, the AM strategy accepts the new Pbest without considering its fitness value.The mean position of the candidate leader set and the normalized mean velocity are employed for the AM strategy.After mutation, the stagnated particle has a chance to search for a new promising area.The new Pbest of the stagnated particle is generated according to Equations ( 7)- (9).8) where Leader d denotes the mean position of the candidate leader set, which may locate the approximate promising area.η is a scale factor to adjust the amplitude of mutation, and it will be discussed in Section 4.4.N(0,1) stands for a normal distributed random number with the standard deviation of 1. v norm denotes the normalized mean velocity in the current swarm.The velocity information of the current swarm is employed to adjust the amplitude of the AM [71].The AM strategy is described in Algorithm 2.
Algorithm 2. The adaptive mutation module Generate a new Pbest for the ith particle according to Equations ( 7)-(9) 6 Pbestval(i) = fit(Pbest(i)); The integer Stag2(i) introduced in Algorithm 1 is used to detect the stagnated particles.Stag2m is the mutation gap.Commonly, the Stag2m is much bigger than Stag1m to avoid impairing the convergence of algorithm.Once Stag2(i) is bigger than the mutation gap Stag2m, the AM may be triggered to generate a new Pbest for the ith particle.Pam is a linearly decreasing probability used to determine whether to execute AM for the ith particle.r 2 ∈[0,1] is a uniformly distributed random number.If r 2 is smaller than Pam, the AM is executed and a new Pbest is generated.The stagnated particle accepts the new Pbest without considering its fitness value.The AM can transfer the stagnated particle to a potentially promising area by employing the distribution information of the candidate leader set.Conducting AM frequently may impair the convergence of PSO, hence a linearly decreasing probability is employed to control AM.In the early stage, the AM is executed with a higher probability to enhance exploration, while in the latter stage, the AM is carried out with a lower probability to avoid impairing the convergence of the algorithm.The amplitude of mutation is adjusted by the normalized mean velocity.Algorithm 2 is conducted in the alternative step *3 of Algorithm 1. ML-CLPSO-AM executes all the alternative steps in Algorithm 1.The characteristics of ML strategy and AM strategy will be tested in the following section.

Characteristics of ML and AM Strategies
To investigate the characteristics of the ML and AM strategies, two benchmark functions of the CEC2017 test suite were employed (unimodal function f 2 and multimodal function f 15 ).ML-CLPSOs with different leader size were tested and their diversity curve and convergence curve are given in Figure 1.In this study, the diversity was evaluated by the mean distance between the particles' position and the centroid of the current swarm.The CLPSO, CLPSO-G were also included in this comparison test.The parameter settings of PSO variants were in accordance with Table 1.The leader size of ML-CLPSO was set to 4, 6, 8, 10 and 12.The leader size of ML-CLPSO-AM was set to 10.In Figure 1 the ML-CLPSO(*) denotes ML-CLPSO with the leader size in the bracket, for example ML-CLPSO(4) denotes ML-CLPSO with a leader size of 4. one particle's fitness value ceases improving, its Pbest may stay in the same position unless a better position is found by the particle, hence the diversity of CLPSO-G is higher than CLPSO-AM in the latter stage.The diversity curves of ML-CLPSO (10) and ML-CLPSO-AM are almost overlapping, that is, for unimodal problems, the effect of the AM strategy on the diversity of ML-CLPSO-AM is not significant.Figure 1b shows that CLPSO-G converges quickly because it contains a global learning component, while CLPSO converges a bit more slowly.ML-CLPSOs converge more slowly than CLPSO in the early stage, however, they surpass CLPSO in the middle stage.In the early stage, ML-CLPSO with its smaller leader size converges faster.In the latter stage, ML-CLPSO(6) and ML-CLPSO(8) surpass ML-CLPSO(4).Figure 1b shows that employing moderate leader size leads to better performance on unimodalf2.In the end stage, the convergence rate of ML-CLPSO-AM increases significantly and outperforms ML-CLPSO (10) because the AM strategy activates the stagnated particles to explore the potentially promising area and hence, the performance of ML-CLPSO-AM is improved.Because the ML strategy dynamically allocates suitable leaders for the particles, ML-CLPSOs can make good use of the information from the high-quality particles to achieve better performance than CLPSO-G.By adopting the AM strategy, CLPSO-AM and ML-CLPSO-AM yield better performance than the corresponding CLPSO-G and ML-CLPSO (10), respectively.Figure 1a shows that on unimodal function f 2 , CLPSO keeps the highest diversity, CLPSO-G keeps relatively lower diversity, and the diversity of ML-CLPSOs is higher than CLPSO-G but lower than CLPSO.ML-CLPSO has bigger leader size and keeps slightly higher diversity.The test results show that increasing the leader size is helpful to enhance diversity.The diversity of CLPSO-AM is approximately equal to that of CLPSO-G in the early stage, while in the latter stage, the diversity of CLPSO-AM is lower than CLPSO-G.This is because the AM strategy encourages the motion of the stagnated particles by transferring them to explore the possible promising area while in CLPSO-G, if one particle's fitness value ceases improving, its Pbest may stay in the same position unless a better position is found by the particle, hence the diversity of CLPSO-G is higher than CLPSO-AM in the latter stage.The diversity curves of ML-CLPSO (10) and ML-CLPSO-AM are almost overlapping, that is, for unimodal problems, the effect of the AM strategy on the diversity of ML-CLPSO-AM is not significant.Figure 1b shows that CLPSO-G converges quickly because it contains a global learning component, while CLPSO converges a bit more slowly.ML-CLPSOs converge more slowly than CLPSO in the early stage, however, they surpass CLPSO in the middle stage.In the early stage, ML-CLPSO with its smaller leader size converges faster.In the latter stage, ML-CLPSO(6) and ML-CLPSO(8) surpass ML-CLPSO(4).Figure 1b shows that employing moderate leader size leads to Energies 2019, 12, 116 9 of 27 better performance on unimodalf 2 .In the end stage, the convergence rate of ML-CLPSO-AM increases significantly and outperforms ML-CLPSO (10) because the AM strategy activates the stagnated particles to explore the potentially promising area and hence, the performance of ML-CLPSO-AM is improved.Because the ML strategy dynamically allocates suitable leaders for the particles, ML-CLPSOs can make good use of the information from the high-quality particles to achieve better performance than CLPSO-G.By adopting the AM strategy, CLPSO-AM and ML-CLPSO-AM yield better performance than the corresponding CLPSO-G and ML-CLPSO (10), respectively.
Figure 1c shows that on multimodal function f 17 , similar tounimodal function f 2 , CLPSO preserves the highest diversity.The diversity of ML-CLPSOs is higher than CLPSO-G but lower than CLPSO.By increasing the leader size, ML-CLPSO is able to preserve more diversity.Compared with unimodal functions, more significant diversity differences can be observed for multimodal functions.Since the AM strategy transfers the stagnated particles to possible promising area, the diversity of CLPSO-AM and ML-CLPSO-AM are relatively lower than the corresponding CLPSO-G and ML-CLPSO (10). Figure 1d shows that in the early stage of optimization, CLPSO and CLPSO-G converge quickly, however, they are surpassed by ML-CLPSOs.ML-CLPSOs achieve almost the same mean error in the end.The convergence curves of CLPSO-AM and CLPSO-G are almost overlapping.Compared with ML-CLPSO(10), ML-CLPSO-AM converges faster and generates a lower mean error.The test results suggest that the solution accuracy is not only dependent on leader size, but also has something to do with the characteristics of the tested problem.
This group of tests show that the ML strategy can enhance the exploitation of CLPSO without significantly impairing its exploration.By using an ML strategy, the ML-CLPSOs yield lower mean error than both CLPSO and CLPSO-G.The leader size has a significant impact on the diversity and performance of ML-CLPSO; increasing the leader size preserves more diversity while adopting a proper leader size can obtains a good trade-off between exploration and exploitation.Because ML-CLPSO employs linear decreasing ω, c 1 and linear increasing c 2 , in the early stage, the comprehensive learning part plays the key role in enhancing exploration while in the latter stage, the social learning part plays a leading role in speeding up the convergence rate.Hence, ML-CLPSO converges relatively slowly in the early stage but it converges relatively fast in the latter stage.Regarding the AM strategy, CLPSO-AM performs better than CLPSO-G on f 2 while it performs almost the same as CLPSO-G on f 17 .The AM strategy can activate the stagnated particle to search potential promising areas.With both ML and AM strategies, ML-CLPSO-AM outperforms ML-CLPSO(10) on both f 2 and f 17 .For more tests of ML and AM strategies refer to Section 4.3.1.

Test Problems
To test the performance of the proposed ML-CLPSO-AM, the latest CEC2017 test suite on single objective real-parameter numerical optimization [72] was employed.The test suite is referred to as CEC2017 test suite hereafter.The CEC2017 test suite contains thirty benchmark functions, including three unimodal functions, seven simple multimodal functions, ten hybrid functions and ten composition functions.The exact equations of CEC2017 test suite are not allowed to be used.
According to the instruction of CEC2017 test suite, the maximum FEs Max_FES = 10,000*D and the search range is [−100, 100].In this study, the number of dimensions was set as D = 30.Each algorithm was run for 51 runs and the mean error, average rank and Wilcoxon signed rank test [73][74][75] were employed to compare the performance of the involved algorithms.

Parameters Settings
To test the effectiveness of the proposed ML and AM strategy, three PSO variants were proposed in this study, namely, ML-CLPSO, CLPSO-AM, and ML-CLPSO-AM.ML-CLPSO is a combination of ML strategy and CLPSO; CLPSO-AM is developed by transferring AM strategy into CLPSO-G; Energies 2019, 12, 116 10 of 27 ML-CLPSO-AM is merges ML and AM strategies simultaneously into CLPSO.Ten PSO variants, six EAs and meta-heuristics were tested and compared with the proposed methods, including five state-of-the-art PSO variants, five latest PSO algorithms, two classic EAs and three recently reported meta-heuristics.CLPSO-G inherits the comprehensive learning strategy of CLPSO and added a component of learning from Gbest to speed up the convergence [55].Other involved peer PSO algorithms have been introduced as mentioned in Sections 1 and 2. The adaptive differential evolution with optional external archive (JADE) [56] adopts a new mutation strategy "DE/current-to-pbest" with an optional external archive and updating control parameters in an adaptive manner.Artificial bee colony (ABC) [5] simulates the intelligent foraging behavior of a honeybee swarm.Evolution strategy with covariance matrix adaptation (CMA-ES) [57] solves the rotation problem by adopting covariance matrix adaptation strategy.The grey wolf optimizer (GWO) [8] simulates the leadership hierarchy and hunting mechanism of grey wolves in nature.Across neighbor search (ANS) [58] utilizes a group of individuals to search the solution space cooperatively.A set of superior solutions found by individuals is maintained and refreshed dynamically.The Salp swarm algorithm (SSA) [59] simulates the swarming behavior of salps when navigating and foraging in oceans for solving optimization problems.
All the algorithms employ the relevant author suggested parameters configuration.The parameter settings of the PSO algorithms are given in Table 1.EPSO [53] 2017 Reference [53] 4.3.Comparison Tests

Comparison Test of Different Strategies
To show the effects of ML and AM strategies on CLPSO, a comparison test of CLPSO with the ML, AM or both strategies was conducted, and the test results are given in Table 2.The best results are highlighted in bold.
The test results in Table 2 show that on three unimodal functions (f 1 -f 3 ), CLPSO, ML-CLPSO-AM and CLPSO-AM yield the best solutions on f 1 , f 2 and f 3 , respectively.Because of the ML strategy, ML-CLPSO obtains a higher performance.It achieves a lower mean error than CLPSO and CLPSO-G on two (f 2 , f 3 ) and three (f 1 -f 3 ) functions, respectively.CLPSO-AM performs better than CLPSO-G on f 2 and f 3 , performs worse on f 1 .Compared with ML-CLPSO, ML-CLPSO-AM performs better on f 2 , and performs worse on f 1 and f 3 .The Wilcoxon signed-rank test results show that ML-CLPSO-AM performs significantly better than CLPSO, CLPSO-G, ML-CLPSO, CLPSO-AM on 2, 1, 1 and 2 functions, respectively.The ML strategy improves the performance of CLPSO significantly, while the AM strategy doesn't exhibit obvious advantage.
On seven simple multimodal functions (f 4 -f 10 ), ML-CLPSO-AM yields the best solutions on six functions (f 5 -f 10 ).CLPSO-AM achieves the best solution on f 4 .Compared with CLPSO and CLPSO-G, ML-CLPSO obtains lower mean error on four (f 5 , f 7 -f 9 ) and five functions (f 5 -f 9 ), respectively.Compared with CLPSO-G, CLPSO-AM yields lower mean error on four functions Energies 2019, 12, 116 11 of 27 (f 4 , f 6 , f 7 , f 9 ).The Wilcoxon signed rank test results show ML-CLPSO-AM outperforms CLPSO, CLPSO-G, ML-CLPSO on all seven functions.Compared with CLPSO-AM, ML-CLPSO-AM wins on six functions.It only performs worse than CLPSO-AM on f 4 .The test results reveal that the ML strategy can improve the performance of CLPSO, and the AM strategy can further improve the performance of ML-CLPSO.
On ten hybrid functions (f 11 -f 20 ), ML-CLPSO-AM achieves the best solutions on five functions (f 11 , f 16 , f 17 , f 19 , f 20 ), CLPSO generates the best solutions on f 13 and f 15 , ML-CLPSO achieves the best solutions on f 12 and f 14 , and CLPSO-AM generates the best solution on f 18 .Compared with CLPSO and CLPSO-G, ML-CLPSO yields lower mean error on five (f 11 , f 12 , f 14 , f 16 , f 17 ) and ten functions (f 11 -f 20 ), respectively.Compared with CLPSO-G, CLPSO-AM yields lower mean error on four functions (f 13 , f 14 , f 16 , f 18 ).According to the Wilcoxon signed rank test results, ML-CLPSO-AM performs significantly better than ML-CLPSO and CLPSO-AM on six (f 11 , f 15 -f 18 , f 20 ) and five functions (f 11 , f 15 -f 17 , f 20 ), respectively.ML-CLPSO-AM and CLPSO-AM obtaineda similar performance on f 19 .ML-CLPSO performs almost the same as CLPSO while it performs much better than CLPSO-G.For the AM strategy, the performance of CLPSO-AM is similar to CLPSO-G while ML-CLPSO-AM performs slightly better than ML-CLPSO and CLPSO-AM.Of ten complex composition functions (f 21 -f 30 ), ML-CLPSO-AM achieves the best solution on seven functions (f 21 -f 25 , f 27 , f 29 ), CLPSO and ML-CLPSO yield the best solution on two (f 25 , f 26 ) and three functions (f 22 , f 28 , f 30 ), respectively.Compared with CLPSO and CLPSO-G, ML-CLPSO generates lower mean error on eight (f 21 -f 24 , f 27 -f 30 ) and ten functions (f 21 -f 30 ), respectively.Compared with CLPSO-G, CLPSO-AM achieves lower mean error on five functions (f 23 , f 24 , f 27 , f 29 , f 30 ).The Wilcoxon signed rank test shows that the performance of ML-CLPSO-AM is significantly better, the same and significantly worse than ML-CLPSO on four (f 21 , f 23 , f 24 , f 29 ), three (f 22 , f 25 , f 27 ) and three functions (f 26 , f 28 , f 30 ), respectively.Compared with CLPSO-AM, ML-CLPSO-AM performed significantly better, the same, and significantly worse on six (f 21 -f 24 , f 26 , f 29 ), two (f 25 , f 27 ) and two functions (f 28 , f 30 ).The test results show that the ML strategy improves the performance of CLPSO significantly, and after combining the AM strategy with ML-CLPSO, the resultant ML-CLPSO-AM performs better than ML-CLPSO and CLPSO-AM.
On all thirty functions, ML-CLPSO-AM, CLPSO, ML-CLPSO and CLPSO-AM achieve the best solutions on nineteen, five, five and three functions, respectively.ML-CLPSO achieves lower mean errors than CLPSO and CLPSO-G on nineteen and twenty-eight functions, respectively.Compared with CLPSO-G, CLPSO-AM achieves lower mean errors on fifteen functions.Compared with CLPSO-AM, ML-CLPSO achieves lower mean errors on twenty-six functions.ML-CLPSO-AM performs significantly better than CLPSO, CLPSO-G, ML-CLPSO and CLPSO-AM on twenty-two functions, twenty functions, eighteen functions and nineteen functions, respectively.
The test results indicate that with the ML strategy, ML-CLPSO performs much better than CLPSO and CLPSO-G, although if incorporating the AM strategy into CLPSO-G, the resultant CLPSO-AM does not show significant advantage over CLPSO-G.The reason is that CLPSO-G cannot preserve sufficient diversity to mitigate premature convergence.Combining ML and AM strategies with CLPSO simultaneously, the resultant ML-CLPSO-AM performs better than both ML-CLPSO and CLPSO-AM.The ML strategy makes good use of high-quality particles to enhance exploitation of CLPSO without impairing its exploration significantly.The AM strategy can further improve the performance of ML-CLPSO by transferring the stagnated particles to potentially promising area.The supplementary file show that the computational complexity of ML-CLPSO-AM is almost the same with CLPSO, while the computing time of ML-CLPSO-AM is shorter than CLPSO (Figure S1).
On ten complex composition functions (f 21 -f 30 ), ML-CLPSO-AM achieves the best solutions on seven functions (f 21 -f 25 , f 27 , f 29 ).HCLPSO generates the best performance on f 25 and f 26 , FIPS, FDR-PSO and SL-PSO wins the best performance on f 22 , f 28 and f 30 , respectively.On f 22 , FIPS, SL-PSO, EPSO, GL-PSO, HCLPSO and ML-CLPSO-AM generate almost the same mean error.On f 25 , all the PSO variants yields almost the same mean error except PSO-cf.Compared with PSO-cf, FIPS and FDR-PSO, ML-CLPSO-AM is only outperformed by FDR-PSO on f 28 and f 30 .ML-CLPSO-AM only performs worse than SL-PSO, EL-PSO, EPSO, GL-PSO and HCLPSO on one (f 30 ), zeros, three (f 26 , f 28 , f 30 ), zeros, two functions (f 26 , f 30 ), respectively.ML-CLPSO-AM performs better than any other peer PSO algorithms on complex composition functions.
In this group of comparison test, ML-CLPSO-AM performs better than all the involved PSO variants on simple multimodal functions and composition functions.On unimodal functions, ML-CLPSO-AM is outperformed by EL-PSO, EPSO and HCLPSO.On hybrid functions, ML-CLPSO-AM is secondary to EPSO.
To evaluate nine PSO variants' overall performance on 30 CEC2017 functions, they were ranked by mean error in ascending order (the lower rank the better) and the results are given in Table 4.
It can be seen in Table 4 that ML-CLPSO-AM ranks first on most of simple multimodal functions and composition functions, while on unimodal functions and hybrid functions, ML-CLPSO-AM achieves a moderate rank.ML-CLPSO-AM does not perform well on f 13 , f 14 , f 18 and f 19 .EPSO and HCLPSO showed the highest performance on unimodal functions and hybrid functions, respectively.ML-CLPSO-AM hits the best solutions on more than half of the tested functions.The average ranks of PSO-cf, FIPS, FDR-PSO, SL-PSO, EL-PSO, EPSO, GL-PSO, HCLPSO and ML-CLPSO-AM on 30 CEC2017 functions are 8.233, 6.800, 5.333, 4.033, 6.367, 3.367, 4.667, 3.133 and 2.900, respectively.Evaluating the overall performance of nine PSO algorithms by average ranks, the order is ML-CLPSO-AM, HCLPSO, EPSO, SL-PSO, GL-PSO, FDR-PSO, EL-PSO, FIPS and PSO-cf.The overall performance of ML-CLPSO-AM is better than other comparison PSO algorithms.To evaluate the convergence rates of different PSO variants, their convergence curves on six representative functions are given in Figure 2. Six functions are unimodal function f 2 , simple multimodal functions f 7 , hybrid function f 12 and f 17 , composition function f 23 and f 29 .
Figure 2a shows that on unimodal function f 2 , EPSO and GL-PSO converged fast from the beginning, because EPSO contains a global version HPSO-TAVC component while GL-PSO employs the genetic operator to breed high qualified examples to guide the evolution of the particles.With an independent subgroup to enhance exploitation, HCLPSO performs well too.PSO-cf, FIPS and EL-PSO fell behind.Since ML-CLPSO-AM enhances diversity to explore the search space sufficiently in the early stage, it converges more slowly in the early stage.However, ML-CLPSO-AM gradually changes over to enhance exploitation in the optimization process by linearly adjusting inertia weight and acceleration coefficients, thus it speeds up convergence rate and surpasses a few PSO variants.Figure 1a shows that the convergence rate of ML-CLPSO-AM is accelerated significantly at about Energies 2019, 12, 116 15 of 27 240,000 FEs, for the AM strategy can activate the stagnated particles to explore the possible promising areas.The similar phenomenon can be seen in Figure 2b,d,e.In the end, ML-CLPSO-AM achieves the lowest mean error.The AM strategy can improve the performance of ML-CLPSO-AM significantly on f 2 .
Figure 2b shows that on simple multimodal function f 7 , GL-PSO converges fast at the beginning, FDR-PSO, EPSO and HCLPSO speed up the convergence rate and yield almost the same mean error with GL-PSO.SL-PSO and FIPS lag behind.Though ML-CLPSO-AM converges relatively slowly from the beginning, the convergence rate accelerates significantly at about 230,000 FEs and it outperforms the other peer PSO variants.
On hybrid functions, Figure 2c shows that on f 12 , GL-PSO and EPSO converge fast.FDR-PSO keeps a steady convergence rate and catches up with GL-PSO and EPSO in the end.PSO-cf and FIPS converge slowly.Though ML-CLPSO-AM converges relatively slowly at the beginning, it surpasses PSO-cf, FIPS, EL-PSO and SL-PSO in the middle stage.Figure 2d shows that on f 17 , SL-PSO and HCLPSO converge fast, FIPS, FDR-PSO, EL-PSO, EPSO and GL-PSO follow HCLPSO and achieve almost the same mean error.PSO-cf is trapped in local optima at about 30000 FEs and cannot yield high accuracy.After about 100000 FEs, the convergence curves of other peer PSO algorithms are almost horizontal.They are trapped in local optima and converge slowly thereafter.Because of the AM strategy, ML-CLPSO-AM escapes from local optima and outperforms other PSO variants.
On composition functions, Figure 2e shows on f 23 SL-PSO converges fast.GL-PSO, HCLPSO and FDR-PSO follow SL-PSO and generate a slightly higher mean error than SL-PSO.EL-PSO and FIPS fall behind.ML-CLPSO-AM converges relatively slowly from the beginning, however, at about 150000 FEs, ML-CLPSO-AM speeds up convergence speed significantly by AM strategy and surpasses other peer PSO variants.Figure 2f shows on f 29 , SL-PSO, EPSO and HCLPSO converges relatively fast.PSO-cf and EL-PSO lag behind.ML-CLPSO-AM keeps steady convergence rate and outperforms other peer PSO variants in the middle stage.
Energies 2018, 11, x FOR PEER REVIEW 15 of 27 from the beginning, the convergence rate accelerates significantly at about 230,000 FEs and it outperforms the other peer PSO variants.On hybrid functions, Figure 2c shows that on f12, GL-PSO and EPSO converge fast.FDR-PSO keeps a steady convergence rate and catches up with GL-PSO and EPSO in the end.PSO-cf and FIPS converge slowly.Though ML-CLPSO-AM converges relatively slowly at the beginning, it surpasses PSO-cf, FIPS, EL-PSO and SL-PSO in the middle stage.Figure 2d shows that on f17, SL-PSO and HCLPSO converge fast, FIPS, FDR-PSO, EL-PSO, EPSO and GL-PSO follow HCLPSO and achieve almost the same mean error.PSO-cf is trapped in local optima at about 30000 FEs and cannot yield high accuracy.After about 100000 FEs, the convergence curves of other peer PSO algorithms are almost horizontal.They are trapped in local optima and converge slowly thereafter.Because of the AM strategy, ML-CLPSO-AM escapes from local optima and outperforms other PSO variants.
On composition functions, Figure 2e shows on f23 SL-PSO converges fast.GL-PSO, HCLPSO and FDR-PSO follow SL-PSO and generate a slightly higher mean error than SL-PSO.EL-PSO and FIPS fall behind.ML-CLPSO-AM converges relatively slowly from the beginning, however, at about 150000 FEs, ML-CLPSO-AM speeds up convergence speed significantly by AM strategy and surpasses other peer PSO variants.Figure 2f shows on f29, SL-PSO, EPSO and HCLPSO converges relatively fast.PSO-cf and EL-PSO lag behind.ML-CLPSO-AM keeps steady convergence rate and outperforms other peer PSO variants in the middle stage.In general, because ML-CLPSO-AM employs linearly adjusted inertia weight and acceleration coefficients to regulate its exploration and exploitation during the optimization process, it converges relatively slowly in the early stage, however, the convergence rate speeds up in the middle stage and achieves high accuracy in the end.The ML strategy can enhance the exploitation of CLPSO without weakening its exploration significantly.The AM strategy can transfer the stagnated particles to search potentially promising areas, hence, ML-CLPSO-AM achieves high accuracy on complex multimodal problems.

Comparison Test with EAs and Meta-Heuristics
In this section, ML-CLPSO-AM is compared with six EAs and meta-heuristics, namely, JADE, ABC, CMA-ES, ABC, GWO, ANS and SSA.All the involved algorithms employ the freely available source codes and authors' suggested parameter configurations.The best test results are highlighted in bold.Table 5 shows that ML-CLPSO-AM yields the best solution on twelve functions, JADE, CMA-ES, ABC and ANS achieve the best solution on seven, seven, four and two functions, respectively.Compared with JADE, ML-CLPSO-AM wins, ties with and loses on sixteen, threeand eleven functions, respectively.ML-CLPSO-AM performs better than JADE on unimodal functions and composition functions, performs similarly to JADE on simple multimodal and hybrid functions.Compared with ABC, ML-CLPSO-AM wins on twenty three functions, loses on seven functions.ABC adopts a one dimension updating strategy and yields the best solution on four composition functions (f21, f24-f26).Compared with CMA-ES, ML-CLPSO-AM wins, ties with and loses on eighteen, two, and twelve functions, respectively.CMA-ES generates the best solution on all three unimodal functions.On simple multimodal functions and composition functions, ML-CLPSO-AM outperforms CMA-ES, while on unimodal and hybrid functions, CMA-ES performs better than ML-CLPSO-AM.Compared with GWO, ML-CLPSO-AM wins on all thirty functions.Compared with ANS, ML-CLPSO-AM wins on twenty three functions, ties on two functions and loses on five functions.Compared with SSA, ML-CLPSO-AM only loses on two functions (f3, f14).ML-CLPSO-AM performs better than ABC, GWO, ANS and SSA on unimodal functions, simple multimodal functions, hybrid functions and composition functions.This group of tests show that ML-CLPSO-AM is competitive compared to six EAs and meta-heuristics.In general, because ML-CLPSO-AM employs linearly adjusted inertia weight and acceleration coefficients to regulate its exploration and exploitation during the optimization process, it converges relatively slowly in the early stage, however, the convergence rate speeds up in the middle stage and achieves high accuracy in the end.The ML strategy can enhance the exploitation of CLPSO without weakening its exploration significantly.The AM strategy can transfer the stagnated particles to search potentially promising areas, hence, ML-CLPSO-AM achieves high accuracy on complex multimodal problems.

Comparison Test with EAs and Meta-Heuristics
In this section, ML-CLPSO-AM is compared with six EAs and meta-heuristics, namely, JADE, ABC, CMA-ES, ABC, GWO, ANS and SSA.All the involved algorithms employ the freely available source codes and authors' suggested parameter configurations.The best test results are highlighted in bold.Table 5 shows that ML-CLPSO-AM yields the best solution on twelve functions, JADE, CMA-ES, ABC and ANS achieve the best solution on seven, seven, four and two functions, respectively.Compared with JADE, ML-CLPSO-AM wins, ties with and loses on sixteen, threeand eleven functions, respectively.ML-CLPSO-AM performs better than JADE on unimodal functions and composition functions, performs similarly to JADE on simple multimodal and hybrid functions.Compared with ABC, ML-CLPSO-AM wins on twenty three functions, loses on seven functions.ABC adopts a one dimension updating strategy and yields the best solution on four composition functions (f 21 , f 24 -f 26 ).Compared with CMA-ES, ML-CLPSO-AM wins, ties with and loses on eighteen, two, and twelve functions, respectively.CMA-ES generates the best solution on all three unimodal functions.On simple multimodal functions and composition functions, ML-CLPSO-AM outperforms CMA-ES, while on unimodal and hybrid functions, CMA-ES performs better than ML-CLPSO-AM.Compared with GWO, ML-CLPSO-AM wins on all thirty functions.Compared with ANS, ML-CLPSO-AM wins on twenty three functions, ties on two functions and loses on five functions.Compared with SSA, ML-CLPSO-AM only loses on two functions (f 3 , f 14 ).ML-CLPSO-AM performs better than ABC, GWO, ANS and SSA on unimodal functions, simple multimodal functions, hybrid functions and composition functions.This group of tests show that ML-CLPSO-AM is competitive compared to six EAs and meta-heuristics.

Parameter Sensitive Analysis
To evaluate the effects of parameters setting on the performance of ML-CLPSO-AM, four representative functions are tested with different parameters setting.In each group of tests, only one parameter is assigned with different value, while the other parameters are set in accordance with Table 1.Each parameter setting on one function is tested for 51 independent runs and the mean error is normalized within the range of [0,1] by dividing the maximum mean error on the tested function.In other words, the lower the normalized error the better.
Figure 3a shows that the composition function f 23 is not sensitive to the leader size.On unimodal function f 1 , the leader size N L = 4 shows the lowest mean error.On simple multimodal functions f 7 , the mean error decreases as the leader size increases.The test results show that a smaller leader size favors unimodal functions while a bigger leader size favors multimodal functions.N L = 10 achieves good performance on f 1 and f 15 ; therefore, N L = 10 is the best choice for the leader size.
Figure 3b shows thatf 23 is insensitive to the refreshing gap, while f 7 achieves almost the same mean error except for the refreshing gap Stag1m = 2. Stag1m = 6 yields the lowest mean errors on f 1 , f 15 and performs better than the other settings, so Stag1m = 6 is employed as the default value of the refreshing gap.A proper refreshing gap can update the particle's comprehensive learning exemplar and social learning leader in a timely way when one particle's fitness value stops improving.
Figure 3c shows that f 7 and f 23 are insensitive to the mutation gap.The mutation gap Stag2m = 40 yields the lowest mean error on f 1 and a relatively low mean error on f 15 , hence, Stag2m = 40 is adopted as the default value of the mutation gap.Too big a mutation gap cannot activate the stagnated particles in time, while too small a mutation gap may impair the algorithm's convergence.
Figure 3d shows that the f 7 and f 23 are insensitive to scale factor as well.The scale factor η = 0.6 achieves the lowest mean error on f 1 and f 15 , hence, η = 0.6 is the best choice of scale factor.A proper scale factor can improve the efficiency of the AM strategy.
Figure 3d shows that the f7 and f23 are insensitive to scale factor as well.The scale factor η = 0.6 achieves the lowest mean error on f1 and f15, hence, η = 0.6 is the best choice of scale factor.A proper scale factor can improve the efficiency of the AM strategy.

Problem Definition
The static economic load dispatch (ELD) problem is one of the major optimization problems in power system management.The objective of ELD problem is to minimize the fuel cost of generating units during the specific operation period.The constraints of the power balance, generator, ramp rate and prohibited operating zones are considered in the ELD problem.

Objective Function
The objective functions of ELD problem can be modeled as a quadratic functionas follows in Equation (10). Minimize: where , …,NG is the formulation of cost functions corresponding to the ith generator.ai, bi and ciare cost coefficients of generator.Pi stands for the ith generator's real power output (in MW) in the period t.NG denotes the number of total generating units.

Problem Definition
The static economic load dispatch (ELD) problem is one of the major optimization problems in power system management.The objective of ELD problem is to minimize the fuel cost of generating units during the specific operation period.The constraints of the power balance, generator, ramp rate and prohibited operating zones are considered in the ELD problem.

Objective Function
The objective functions of ELD problem can be modeled as a quadratic functionas follows in Equation (10).
where f i (P i ) = a i P 2 i + b i P i + c i , i = 1, 2, 3, . . ., N G is the formulation of cost functions corresponding to the ith generator.a i , b i and c i are cost coefficients of generator.P i stands for the ith generator's real power output (in MW) in the period t.N G denotes the number of total generating units.

Cost Functions with Valve Point Effect
A rippling effect is produced for the steam admission throughthe valve in a turbine, hence it is more practical to take the valvepoint effect into aconsideration of the fuel cost function.The cost function for a generator with valve point loading effects can be expressed in follows in Equation (11).
where e i and f i are cost coefficients of valve point loading effects.The constriants of ELD problems are as follows: Power Balance Constriants The power balance constraints expressed in Equation ( 12) are based on the equilibrium between total system generation and total system loads (P D ) and losses (P L ).

∑
N G i=1 P i = P D + P L (12) where P L is calculated according to B-coefficients in the followingEquation (13).

Generator Constraints
Each generating unit in the power system has a lower and upper bound.The generator constraints are represented by a couple of inequality constraints as below: where P min i and P max i correspond to the lower and upper bounds for the power outputs of the ith generator.

Ramp Rate Limits
In practical circumstances, the ramp rate limit restricts the operating range of the online generators when changing-over the generator between two operating periods.The generation may increase or decrease within the relevant upper and lower ramp rate limits.The ramp rate limits are as follows: where P t−1 i is the power output of the ith generator at the previous hour and UR i and DR i are the upper and the lower ramp rate limits, respectively.

Prohibited Operating Zones
The generating units have certain prohibited operating zones due to the physical limitations of the machine components or instability such as steam valve or shaft bearings vibration.Hence, discontinuities are produced in the cost objective function by the prohibited operating zones.These zones should be avoided to economize the power production.The prohibited operating zones can be expressed in the following inequality: P i ≤ P PZ and P i ≥ P PZ (17) where P PZ and P PZ are the lower and upper limits of the prohibited zone for the ith generating unit, respectively.
In this study, three ELD problems were tested.Case 1 is a 6-unit system with transmission losses and valve point effects, the power demand is 1263 MW.Case 2 is a 15-unit system with transmission losses and valve point effects, its power demand is 2630 MW.Case 3 is a 40-unit system without transmission loss, its power demand is 10,500 MW.The experiments for ML-CLPSO-AM and other PSO algorithms were carried out according to the requirements of CEC2011.The maximum FEs was set as 150,000 FEs.The population size and leader size of ML-CLPSO-AM were set as 100 and 25, respectively.Other parameter configurations are in accordance with Table 1.Each algorithm was run for 25 independent runs and the test results are given in Sections 5.2 and 5.3.

Comparison of PSO Algorithms on ELDProblem
In this section, nine PSO algorithms, namely, PSO-cf, FIPS, FDR-PSO, SL-PSO, ELPSO, EPSO, GL-PSO, HCLPSO and ML-CLPSO-AM were tested on an ELD problem.The mean, the standard deviation, the maximum and the minimum costs of 25 independent runs are presented in Table 6, the rank of mean costs is given in Table 7.The row of "average" and "overall" denote the mean rank and the overall rank achieved by one algorithm on three ELD problems, respectively.The best results are highlighted in bold.The test results in Tables 6 and 7 show that EPSO, HCLPSO and ML-CLPSO-AM achieve the lowest mean cost on the 6-unit ELD problem.ML-CLPSO-AM generates the lowest mean cost on 15-unit and 40-unit ELD problems.EPSO and ML-CLPSO-AM yield the lowest minimum cost on the 6-unit ELD problem, ML-CLPSO-AM yields the lowest minimum cost on 15-unit and 40-unit ELD problems.The rank of mean cost in Table 7 shows that EPSO, HCLPSO and ML-CLPSO-AM tie for first on the 6-unit ELD problem.ML-CLPSO-AM ranks first on 15-unit and 40-unit ELD problems.HCLPSO generates the lowest standard deviation on 6-unit and 15-unit ELD problems.The standard deviation of ML-CLPSO-AM is quite close to HCLPSO on 6-unit and 15-unit ELD problems.ML-CLPSO-AM yields the lowest standard deviation on the 40unit ELD problem.Table 7 shows that the average ranks of PSO-cf, FIPS, FDR-PSO, SL-PSO, ELPSO, EPSO, GL-PSO HCLPSO and ML-CLPSO-AM on three ELD problems are 8.667, 4.333, 6.000, 5.667, 6.667, 2.333, 5.333, 2.667 and 1.000, respectively.Evaluating the overall performance of nine PSO algorithms by average rank, the order is ML-CLPSO-AM, EPSO, HCLPSO, GL-PSO, FIPS, FDR-PSO, EL-PSO and PSO-cf.The test results indicate that overall, ML-CLPSO-AM achieves the highest performance on ELD problems.
On a 15-unit system, the optimization results in Table 9 show that *RD-PSO yields the lowest best cost, and ACSS yields the lowest mean cost and the maximum cost.HGPSO generates the highest best cost, mean cost and the maximum cost.The best cost of ML-CLPSO-AM is lower than SA, MTS, SOH-PSO, HGPSO and DE, and slightly higher than *RD-PSO and ACSS.The performance of ML-CLPSO-AM is very close to *RD-PSO and ACSS.Table 12 reports the best optimization results generated by ML-CLPSO-AM.The best cost is $32,694.1960.The total power production and power loss are 2659.5114 and 29.4981, respectively.Subtracting the power loss, the actual power output is 2630.013MW.On a 40-unit system, Table 10 shows that MCSA achieves the lowest best cost, mean cost and maximum cost.*RD-PSO yields the highest best cost, mean cost and the maximum cost.The best cost of ML-CLPSO-AM is lower than *RD-PSO, but higher than ACO, CEP, EMA, MCSA, SDE and FPSOGSA.With the increase in unit number, the performance of ML-CLPSO-AM deteriorates.The best optimization results of ML-CLPSO-AM in Table 13 indicate that the total power production and fuel cost are 10,499.9561and 1,271,288.4367,respectively.

Figure 1 .
Figure 1.The diversity curve and convergence curve of ML-CLPSOs.Figure 1.The diversity curve and convergence curve of ML-CLPSOs.

Figure 1 .
Figure 1.The diversity curve and convergence curve of ML-CLPSOs.Figure 1.The diversity curve and convergence curve of ML-CLPSOs.

Figure 2 .
Figure 2. Convergence curves of the representative functions.

Figure 2 .
Figure 2. Convergence curves of the representative functions.
(a) Effects of the leader size.(b) Effects of the refreshing gap.(c) Effects of the mutation gap.(d) Effects of the scale factor.

Note: the line with *1, *2, *3 are
alternative steps for the AM strategy, which will be introduced in Section 3.2.They are skipped in ML-CLPSO.

Table 1 .
Parameter settings of PSO algorithms.

Table 2 .
Test results of CLPSO with different strategies.

Table 3 .
Comparison test results with PSO algorithms.

Table 4 .
Rank based analysis of mean performance among PSO algorithms.

Table 5 .
Comparison test results with EAs and meta-heuristics.

Table 6 .
Comparison test results of PSO algorithms on ELD problems.

Table 7 .
Rank of mean cost among PSO algorithms on ELD problems.

Table 11 .
Best solution achieved by ML-CLPSO-AM on a 6-unit system.

Table 12 .
Best solution achieved by ML-CLPSO-AM on 15 units system.

Table 13 .
Best solution achieved by ML-CLPSO-AM on a 40-unit system.