A Hybrid Multi-Step Probability Selection Particle Swarm Optimization with Dynamic Chaotic Inertial Weight and Acceleration Coefficients for Numerical Function Optimization

Abstract

As a meta-heuristic algoriTthm, particle swarm optimization (PSO) has the advantages of having a simple principle, few required parameters, easy realization and strong adaptability. However, it is easy to fall into a local optimum in the early stage of iteration. Aiming at this shortcoming, this paper presents a hybrid multi-step probability selection particle swarm optimization with sine chaotic inertial weight and symmetric tangent chaotic acceleration coefficients (MPSPSO-ST), which can strengthen the overall performance of PSO to a large extent. Firstly, we propose a hybrid multi-step probability selection update mechanism (MPSPSO), which skillfully uses a multi-step process and roulette wheel selection to improve the performance. In order to achieve a good balance between global search capability and local search capability to further enhance the performance of the method, we also design sine chaotic inertial weight and symmetric tangent chaotic acceleration coefficients inspired by chaos mechanism and trigonometric functions, which are integrated into the MPSPSO-ST algorithm. This strategy enables the diversity of the swarm to be preserved to discourage premature convergence. To evaluate the effectiveness of the MPSPSO-ST algorithm, we conducted extensive experiments with 20 classic benchmark functions. The experimental results show that the MPSPSO-ST algorithm has faster convergence speed, higher optimization accuracy and better robustness, which is competitive in solving numerical optimization problems and outperforms a lot of classical PSO variants and well-known optimization algorithms.

1. Introduction

With the development of scientific research, engineering technology and social economy, optimization issues have gradually become high dimensionality, high level and great difficulty. Conventional optimization has become increasingly unsuitable for dealing with these optimization problems. In order to adapt to the upgrading of optimization problems better, people have a very urgent need for the upgrading of optimization technology. In recent years, the development of optimization algorithms has been a very prevalent research direction. In optimization algorithms, meta-heuristic optimization methods play a vital role, such as particle swarm optimization (PSO) [1] grey wolf optimizer (GWO) [2] the whale optimization algorithm (WOA) [3], differential evolution (DE) [4] gravitational search algorithm (GSA) [5] moth-flame optimization (MFO) [6] biogeography-based optimization (BBO) [7] sine cosine algorithm (SCA) [8] krill herd algorithm (KH) [9] artificial bee colony (ABC) [10] ant lion optimizer (ALO) [11]

Among these well-known optimization algorithms, the PSO algorithm has attracted considerable attention as a classic swarm intelligence algorithm. The PSO algorithm is inspired by the predation behavior of birds. Since its introduction in 1995, it has undergone many improvements to form many PSO variants, which are now widely used in various fields of science and society, such as feature selection [12] artificial intelligence [13] wireless sensor network [14] energy management system [15] public resource construction [16] and so on.

The PSO algorithm has the advantages of having a simple principle, few required parameters, fast convergence and easy realization. However, the search direction of PSO is to approach the global optimum, which makes the information exchange in the group in a single direction, so that the particles quickly gather in a small search area, resulting in the poor swarm diversity [17], which makes it easy to fall into local extremes and get poor convergence accuracy. The swarm diversity information can reflect the distribution of the entire particle swarm. The lack of swarm diversity will cause the swarm to converge prematurely to some local optimums. At present, the two most widely used measures of swarm diversity are population distribution entropy [18] and population average particle distance [19] The algorithm performs cluster analysis on the individuals in the population, and then the population distribution entropy can be obtained, which reflects the particle distribution of the population in each area of the search space. For each evolutionary generation of the population, the average particle distance of all individuals in the entire population must be calculated, which expresses the dispersion degree of the individual in the population.

To overcome the above shortages, researchers have conducted a lot of research and made improvements on PSO to optimize the algorithm performance. The first direction is an updated mechanism improvement, such as comprehensive learning PSO (CLPSO) [20] orthogonal multi-swarm cooperative PSO [21] enhanced particle swarm optimization with levy flight (PSOLF) [22] hybrid chaotic quantum behaved particle swarm optimization algorithm (HCQPSO) [23] Qin et al. [24] proposed a deep-learning-driven PSO algorithm (DLD-PSO). It introduced the matrix of weights which are extracted from the deep learning prediction model into the PSO update mechanism. These improved mechanisms can effectively speed up the convergence speed and improve the search ability of the method. The second direction is parameter improvement, that is, improving the inertia weight and acceleration coefficients. If the global searching capability is strong, the convergence speed would be fast, and it is less likely to be limited by the local minimum, but the convergence accuracy is low. The strong local search capability is the opposite [25] To find a balance between global search and local search, the values of inertia weight and acceleration coefficient are particularly important. In [26] Tian et al. introduced sigmoid-based acceleration coefficients and established an appropriate ratio between exploration and exploitation, which successfully achieved a balance between global search and local search. In [27] Arasomwan et al. introduced chaos mechanism and adaptive strategy to the inertia weight, which used the regularity, randomness and ergodicity of chaos and avoided premature convergence. In [28] Taherkhani et al. obtained different adaptive inertial weight based on the optimal position and distance in particle history, which improved the convergence accuracy and speed of the algorithm. The third direction is to combine PSO with other algorithms. Zhang et al. [29] introduced the DE-PSO algorithm, which combined PSO and DE to realize the algorithm using a differential operator and enrich the swarm diversity. Garg [30] proposed a hybrid PSO-GA algorithm by incorporating genetic operators into PSO. The balance between exploration and exploitation abilities have been further improved. Javidrad et al. [31] used the simulated annealing algorithm (SA) as a local search mechanism, which improves the convergence behavior of PSO, forming a PSO-SA hybrid algorithm.

Although these PSO variants inherit the advantages of PSO and overcome some shortcomings of PSO, which makes them increasingly have advantages over conventional optimization, the deficiencies of being easily trapped in a local optimal solution and lacking swarm diversity still exist [32] which leads to the unsatisfactory in addressing complex optimization problems with different characteristics. The above methods only improve one aspect of PSO, like convergence speed, premature convergence and the balance between exploration and exploitation. Therefore, these methods are unable to deal with more complicated optimization problems.

In this paper, a hybrid multi-step probability selection particle swarm optimization with sine chaotic inertial weight and symmetric tangent chaotic acceleration coefficients called MPSPSO-ST is proposed. MPSPSO-ST is a comprehensively improved algorithm that can modify many aspects of PSO for numerical function optimization as follows:

• The multi-step probability selection process can enhance the search ability of particles and avoid premature convergence, which also has a positive effect on convergence speed.
• The sine chaotic $\omega$ and symmetric tangent chaotic $c_1,c_2$ enrich the swarm diversity and achieve a better balance between the exploration and exploitation ability, which offers higher convergence accuracy.

The remainder of this paper is organized as follows: Section 2 presents the related theory about PSO. Section 3 describes the proposed MPSPSO-ST algorithm in detail. In Section 4, several well-known optimization methods and PSO variants are adopted to verify the performance of MPSPSO-ST in numerical optimization. Finally, Section 5 consists of the conclusions drawn in this paper and the summary of future work.

2. Related Theory about PSO

PSO is a kind of swarm intelligence optimization algorithm, which is derived from the study of bird swarm predation behavior. Each particle in the particle swarm represents a possible solution to a problem. Through the simple behavior of individual particles and the interaction of information within the group, the intelligence of solving problems is achieved.

PSO is a population-based search algorithm. An individual in the bird swarm is abstracted into a particle. Each particle searches for the optimal solution in the search space separately. The optimal solution that each particle can currently search for is recorded as the current individual optimal value, and then the individual optimal value is shared with other particles in the entire particle swarm. The optimal individual extreme value is found as the current global maximum of the whole particle swarm. All particles in the particle swarm continually update their positions and speeds based on the current individual extremes found by themselves and the current global optimal solution shared by the entire particle swarm. The particle will gradually approach the optimal position and finally find the global optimum during the iterative process.

When searching in D-dimensional target search space, each particle has two parameters, position and velocity. The position of the $i$th particle is represented as $x_i^{}=(x_{i1,}{x}_{i2,}{\dots }_{,}{x}_{iD})$, $i=(1,2,\dots ,N)$, where N is the number of population. The velocity of the $i$th particle is represented as $v_i=(v_{i1,}{v}_{i2,}{\dots }_{,}{v}_{iD})$, $i=(1,2,\dots ,N)$. The velocity and position of the particles are iterated according to the following iterative equations:

$$v_i^{(t+1)}=v_i^t+c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)+c_2\cdot r_2\cdot (gbes{t}_{}^t-x_i^t)$$

(1)

$$x_i^{(t+1)}=x_i^t+v_i^{(t+1)}$$

(2)

$x_i^t$ indicates the position of the $t$th iteration of the $i$th particle. $v_i^t$ denotes the speed of the $t$th iteration of the $i$th particle.$c_1$ and $c_2$ are two positive constants, generally $c_1=c_2=2$; $r_1$ and $r_2$ are random numbers in the interval [0, 1]. Each particle is evaluated for fitness value $f(x_i)$ by objective function $f$. Each particle position represents a solution to the problem, which has a fitness value $f(x_i)$, given by the objective function $f$. During each iteration process, the particles memorize the position with the best fitness value by comparing the fitness value of each position and update $pbest,gbest$. $pbes{t}_i^t$ is the current individual optimal position of the particle and $gbes{t}^t$ is the current global optimal position of the entire particle swarm. Self-cognition component term $c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)$ pushes the particles to move toward their own best positions found so far. Social cognitive component term $c_2^{}\cdot r_2\cdot (gbes{t}^t-x_i^t)$ encourages the particles to move toward the global best position found currently. $v_i\in [-v_{\max },v_{\max }]$, and $v_{\max }$ is a constant. $v_{\max }$ often set to four in practice [33]. When the particle velocity exceeds the interval, let the particle velocity $v_i$ be equal to the upper or lower bound of the interval ($-v_{\max }$ or $v_{\max }$) so that the particle velocity is controlled within a reasonable range. Equations (1) and (2) are called the basic PSO algorithm.

In [34] Shi and Eberhart introduced inertial weight $\omega$ into the memory term in Equation (1) and found that it can balance the global search ability and local search ability better. The improved iterative equations are formulated as follows:

$$v_i^{(t+1)}=\omega \cdot v_i^t+c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)+c_2\cdot r_2\cdot (gbes{t}_{}^t-x_i^t)$$

(3)

$$x_i^{(t+1)}=x_i^t+v_i^{(t+1)}$$

(4)

where $\omega$ is a linear decreasing weight strategy defined as follows:

$$\omega ={\omega }_{\max }-\frac{M_j}{M_{\max }}\cdot ({\omega }_{\max }-{\omega }_{\min })$$

(5)

where ${\omega }_{\min }$ and ${\omega }_{\max }$ are the initial and final values of the inertial weight, respectively, $M_j$ is the current iteration number and $M_{\max }$ is the maximum iteration number.

Since the inertial weight $\omega$ improves the performance significantly, scholars take it as the standard PSO algorithm, which has become the basis of the current research and improvement of PSO. The pseudo-code of the standard PSO algorithm is shown in Figure 1. Inspired by this linear decreasing $\omega$, some articles have proposed different strategies on inertial weight [28,35].

3. Hybrid Multi-Step Probability Selection Particle Swarm Optimization with Sine Chaotic Inertial Weight and Symmetric Tangent Chaotic Acceleration Coefficients (MPSPSO-ST)

The PSO algorithm currently has shown great effectiveness on various optimization problems, but it still has a drawback as lacking diversity in the search process. Due to premature convergence, the search is prone to stagnate and fall into local optimal solutions [36,37]. Based on the in-depth study of PSO, this paper proposes three modifications to improve the performance of PSO.

3.1. Hybrid Multi-Step Probability Selection Particle Swarm Optimization (MPSPSO)

According to Equation (3), the particle velocity updated in the PSO algorithm is determined by three terms: inertia term $v_i^t$, self-cognition component term $c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)$ and social cognitive component term $c_2^{}\cdot r_2\cdot (gbes{t}^t-x_i^t)$. Inspired by human behavior, when humans do things, they sometimes perform habitually (the velocity update only includes inertia term $v_i^t$, as shown in Equation (6)), sometimes they sum up their own experience (the velocity update includes inertia term $v_i^t$ and self-cognition component term $c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)$, as shown in Equation (7)), and sometimes they summarize all the information and then do things (the basic PSO algorithm is this way, as shown in Equation (1)) [38].

Previous studies have shown that the length and direction of $v_i^t$ are coupled with $pbes{t}_i^t$ and $gbes{t}^t$, resulting in a slow update of $pbes{t}_i^t$ [39]. Gao et al. [38] proposed a variant named PSO-MP to decompose the single-step velocity update formula in the standard PSO into the three-steps update and then choose the best one. This method that updating step-by-step first and then selecting the optimal value to update can achieve decoupling of $v_i^t$, $pbes{t}_i^t$ and $gbes{t}^t$, respectively. Thereby, the search ability of this algorithm is enhanced.

In order to increase the diversity and randomness of the particle population, this paper proposes a PSO variant using hybrid multi-step probability selection, namely MPSPSO. The main idea of this variant is to decompose the single-step velocity update formula in the standard PSO into four velocity update formulas. In the early iterations, we directly select the one with the best fitness value of four velocity update formulas. While in the later iterations, we probabilistically select one of the four velocity update formulas using the roulette wheel selection mechanism according to the fitness value. Finally, the optimal solution is obtained. This algorithm can not only achieve the decoupling of $v_i^t$, $pbes{t}_i^t$ and $gbes{t}^t$ but also enhance the randomness and diversity of optimization, thereby effectively avoiding falling into local optimums, so that it can improve the search accuracy and efficiency. The detailed introduction about MPSPSO is presented as follows.

1. Calculate the particle velocity and position by Equations (3) and (4), where $x_i^{t+1}$ is the vector sum with $x_i^t$, $\omega \cdot v_i^t$, $c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)$ and $c_2^{}\cdot r_2\cdot (gbes{t}^t-x_i^t)$, as shown in Figure 2.

As can be seen from Figure 2, in the standard PSO, only $x_i^t$ and $x_i^{t+1}$ are used to update the particle position. $x_1^{t+1}$, $x_2^{t+1}$ and $x_3^{t+1}$ are used as temporary points only for the generation of $x_i^{t+1}$. These three points may be better than others, but they are ignored. Therefore, the velocity update Equation (3) is divided into four steps in this paper, and three temporary points are iterative objects which can be selected. The equations are described as follows:

$$v_1^{t+1}=\omega \cdot v_i^t,x_1^{t+1}=x_i^t+v_1^{t+1}$$

(6)

$$v_2^{t+1}=v_1^{t+1}+c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t),x_2^{t+1}=x_i^t+v_2^{t+1}$$

(7)

$$v_3^{t+1}=v_1^{t+1}+c_2\cdot r_2\cdot (gbes{t}_{}^t-x_i^t),x_3^{t+1}=x_i^t+v_3^{t+1}$$

(8)

$$v_4^{t+1}=v_1^{t+1}+c_1\cdot r_1\cdot (pbes{t}_i^t-x_i^t)+c_2\cdot r_2\cdot (gbes{t}_{}^t-x_i^t),x_4^{t+1}=x_i^t+v_4^{t+1}$$

(9)

2. If iteration $<0.9\times t_{\max }$($t_{\max }$ is the maximum iteration number), choose the position with the best fitness given by the objective function $f$ (in this paper, we all solve $\min f(x)$ problem, so the smaller $f(x)$ is, the better the fitness is) among the above equations as the final position $x_i^{t+1}$. The corresponding velocity is taken as the final velocity $v_i^{t+1}$.

The process is described as follows:

$$v_i^{t+1}=\{\begin{array}{cc}{v}_1^{t+1},& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_1^{t+1})\\ v_2^{t+1},& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_2^{t+1})\\ v_3^{t+1},& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_3^{t+1})\\ v_4^{t+1},& otherwise\end{array}$$

(10)

$$x_i^{t+1}=\{\begin{array}{cc}{x}_1^{t+1},& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_1^{t+1})\\ x_2^{t+1}& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_2^{t+1})\\ x_3^{t+1}& if\min \left\{f(x_1^{t+1}),f(x_2^{t+1}),f(x_3^{t+1}),f(x_4^{t+1})\right\}=f(x_3^{t+1})\\ x_4^{t+1}& otherwise\end{array}$$

(11)

The rationality and effectiveness of this step-by-step process are reflected in the refinement of the particle search trajectory. Firstly, the particles move by Equation (6) according to the speed and inertia of the previous step. If the search direction in the previous step is a direction that is closer to the best solution, then this position may be a better point. On the basis of $x_1^{t+1}$, move along the individual extreme value direction to $x_2^{t+1}$ by Equation (7) to refine the local search; move to $x_3^{t+1}$ along the global extreme value direction by Equation (8) to refine the global search; or comprehensively considering the individual extreme value and the global extreme value, move to $x_4^{t+1}$ along the individual extreme value direction and the global extreme value direction by Equation (9) as the standard PSO. Such a step-by-step search operation can take the temporary points that were initially ignored as optional objects and achieve greater use of potential information so that the algorithm has multi-selected opportunities to find the optimal solution more efficiently.

3. After iterating to 90% of $t_{\max }$, according to the fitness of the four positions from Equations (6)–(9), given by the objective function $f$, we probabilistically select one of the four positions as the update particle position $x_i^{t+1}$ of this generation using a roulette wheel selection mechanism. The particle velocity corresponding to this position is taken as the update particle velocity $v_i^{t+1}$. Roulette wheel selection [40] is widely used in improving various population-based algorithms such as GA [41] DE [42] which all achieve satisfactory results. Inspired by this, we propose the above idea.

Roulette wheel selection mechanism, also known as a proportional selection method, the basic idea of which is that the probability of an individual being selected is proportional to its fitness, that is, the size of the objective function value. The smaller the objective function value of the particle position is, the better the particle fitness is, the higher the probability of being selected is. The specific operations are as follows:

• Calculate the objective function value $f(i=1,2,3,4)$ of each of the four particle positions.
• Due to it that the smaller $f(x)$ is, the better the fitness of the position is, so the probability of being selected should be inversely proportional to the function value of the position. From this, the probability of each position being selected is calculated as follows:

  $$P(x_i)=1-\frac{f(x_i)}{{\displaystyle \sum _{j=1}^{4}f(x_j)}}$$

  (12)
• Calculate the cumulative probability of each position:

  $$q_i={\displaystyle \sum _{j=1}^{i}P(x_j)}$$

  (13)

The cumulative probability diagram is in Figure 3 as follows:

• Randomly generate a uniformly distributed random number $r$ in the interval [0, 1].
• When $r$ satisfies $r<q_1$, select position $x_1^{t+1}$; when $r$ satisfies $q_{k-1}<r\le q_k$, select position $x_k^{t+1}$.

After iterating to 90% of $t_{\max }$, the reason for using the roulette wheel selection mechanism is that the whole swarm are now close to the global optimum, but the swarm diversity in the later iterations is poor, and the point with the best fitness in this generation may not always approach the best solution quickly after iterative calculations, so the roulette wheel selection mechanism is used at the later stage of the iteration to select the final update point $x_i^{t+1}$. Possibility of selecting good particle positions for this generation is high but not absolutely and other particle positions may also be selected, which makes the choice of search direction more diverse. This operation enriches the swarm diversity and ensures that the ability to jump out of local extreme value is effectively modified.

Similar to the next iteration, each particle will modify its speed and position within each iteration until the optimal solution is found or all iterations are terminated. MPSPSO successfully avoids premature convergence and greatly increases the possibility of finding the optimal solution.

3.2. Sine Chaotic Inertia Weight $\omega$

The inertia weight plays a vital role in the convergence behavior of PSO [34,43]. Chaos is a nonlinear dynamic phenomenon, which has regularity, randomness and ergodicity. Due to these characteristics, chaos has become a popular optimization method today. Optimizing the search by chaos mechanism is superior to stochastic search [44,45]. It has the ability not to repeatedly go through all states in a certain range, which can meet the needs of $\omega$ for different functions to the greatest extent [46]. The authors of [47] compared 15 classic methods of inertial weight, and the result indicates that chaotic inertial weight is the best strategy to improve accuracy.

Previous research has shown that the sine function plays a great role in the adjustment of $\omega$ [48,49]. Combining the sine function and chaos mechanism, this paper uses a sine iterator to directly generate the sequence to achieve the sine chaotic inertial weight.

A sine iterator is a type of chaotic sequence generator. Mathematically, its basic form is as follows:

$$X_{n+1}=a{x}_n^2\sin (\pi x_n)$$

(14)

When $a=2.3$, $X_0=0.7$, its simple form is as follows:

$$X_{n+1}=\sin (\pi x_n)$$

(15)

It generates chaotic sequences in (0, 1).

Therefore, this paper proposes the following update form of $\omega$:

$${\omega }_{t+1}=\varphi \times \sin (\pi {\omega }_t)+\tau$$

(16)

The initial value ${\omega }_1$ is an arbitrary number in [0, 1], and $\varphi ,\tau$ are constants.

In this paper, we take $\varphi =0.9$, $\tau =0$. With the progress of the iterative process, $\omega$ can traverse almost all values in the interval of [0.2, 0.9] approximately. The proposal of sine chaotic inertial weight can enhance the global exploration of the algorithm and strengthen the ability of avoiding local optimum.

3.3. Symmetric Tangent Chaotic Acceleration Coefficients $c_1,c_2$

The cognitive component $c_1$ and social component $c_2$ are very important for the algorithm to find the optimal solution accurately and efficiently. Usually, the cognitive component and social component are given equal weight $c_1=c_2=2$. With a large cognitive component and a small social component at the beginning, particles are allowed to move around the search space during the early stages. On the other hand, a small cognitive component and a large social component allow the particles to converge to the global optimum in the latter part of the optimization process [50]. Logistic mapping in the chaos mechanism has attracted much attention owing to better optimization performance, such as randomness and ergodicity. Based on the above two points, this paper proposes symmetric tangent chaotic acceleration coefficients.

The formula of Logistic mapping is $z=\mu \times z\times (1-z)$, and chaos performance is best when $\mu =4$. This paper firstly proposes the symmetric tangent acceleration coefficient, and then introduces the chaotic term. The equations of the symmetric tangent acceleration coefficients are as follows:

$$c_1=-\partial \times m^2\times \tan [\frac{\pi }{8}\times (1+m^2)]+\theta$$

(17)

$$c_2=-\partial \times (1-m{)}^2\times \tan [\frac{\pi }{8}\times (1+(1-m{)}^2)]+\theta$$

(18)

where $m=\frac{t}{t_{\max }}$, $t$ is the current number of iterations, $t_{\max }$ is the maximum number of iteration, and $\partial ,\theta$ are constants.

Then add chaotic terms to Equations (17) and (18): firstly, take a random number in (0,1), then generate the Logistic mapping sequence. We propose the symmetric tangent chaotic acceleration coefficients, which are defined by Equations (19) and (20).

$$c_1=-\partial \times m^2\times \tan [\frac{\pi }{8}\times (1+m^2)]+\theta +\rho \times z$$

(19)

$$c_2=-\partial \times (1-m{)}^2\times \tan [\frac{\pi }{8}\times (1+(1-m{)}^2)]+\theta +\rho \times z$$

(20)

where $\rho$ is a constant.

This paper takes $\partial =0.2$, $\theta =1.5$ and $\rho =0.1$. The proposal of the symmetric tangent acceleration factor enables the algorithm to balance the global search in the early stage of iteration and the local search in the later stage of iteration better. Meanwhile, the proposal of chaotic terms makes the acceleration coefficients have chaotic characteristics while maintaining the original change trend.

Summarizing the above description, the pseudo-code of MPSPSO-ST proposed in this paper is shown in Figure 4:

4. Experimental Results and Discussion

In this study, we used 20 well-known multi-dimensional classical benchmark functions, i.e., objective function $f$, to perform a large number of experiments to evaluate the performance of MPSPSO-ST proposed in this paper. These test functions were adopted by extensive literature [51,52]. We conducted two groups of experiments: in the first group, we compared the search performance of MPSPSO-ST with standard PSO, basic PSO, MPSPSO. In the second group, we compared MPSPSO-ST with three PSO variants (chaos particle swarm optimization (CPSO), particle swarm optimization with the nonlinear dynamic acceleration coefficients (PSO-NDAC), adaptive inertia weight and acceleration coefficients PSO (AIWCPSO)) and three well-known similar optimization algorithms (DE, MFO, SCA).

The Appendix A lists 20 classic benchmark functions adopted for this experiment to test the performance of MPSPSO-ST. These test functions are divided into two types. The first type contains eight unimodal functions and the second type contains 12 multimodal functions. There are many well-known functions, like $f_{10}$(Michalewicz),$f_{11}$(Griewank),$f_{13}$(levy) and $f_{19}$(Zakharov).$f_{10}$(Michalewicz) has d! local minima. The parameter m defines the steepness of the valleys and ridges and a larger m leads to a more difficult search. $f_{11}$(Griewank) has many widespread local minima, which causes more difficulties in searching for the global optimum. $f_{13}$(levy) and $f_{19}$(Zakharov) are both widely used in the optimization field because they are classical. Dim, Range and f_min represent the space dimensions of solution, the range of function variation and the minimum value of the function, that is, the optimal solutions, respectively.

4.1. Comparison of MPSPSO-ST with Standard PSO, Basic PSO and MPSPSO

The parameter settings for MPSPSO-ST, Standard PSO, Basic PSO and MPSPSO are shown in

Table 1. Parameter settings for MPSPSO-ST and Standard PSO, Basic PSO and MPSPSO.

Algorithm	Population Size	Iteration	Run Times	Parameter Settings
Standard PSO	40	500	20	$c_1=c_2=2,\omega =0.9~0.4,V_{\max }=6$
Basic PSO	40	500	20	$c_1=c_2=2,\omega =1,V_{\max }=6$
MPSPSO	40	500	20	$c_1=c_2=2,\omega =0.9~0.4,V_{\max }=6$
MPSPSO-ST	40	500	20	$c_1=-\partial \times m^2\times \tan [\frac{\pi }{8}\times (1+m^2)]+\theta +\rho \times z,$$c_2=-\partial \times (1-m{)}^2\times \tan [\frac{\pi }{8}\times (1+(1-m{)}^2)]+\theta +\rho \times z,$${\omega }_{t+1}=\varphi \times \sin (\pi {\omega }_t)+\tau ,V_{\max }=6$

. The experimental results obtained by testing the 20 benchmark functions in the Appendix A are shown in

Table 2. Experimental results for MPSPSO-ST, Standard PSO and Basic PSO. MPSPSO on 20 classical test functions.

Function	Algorithm	The Best	The Worst	Mean	S.d.
$f_1$	Standard PSO	4.5678 × 10−4	2.1867 × 10−2	5.0366 × 10−3	2.0787 × 10−2
	Basic PSO	9.0029 × 101	1.7626 × 102	1.3408 × 102	8.0903 × 101
	MPSPSO	1.2001 × 10−8	9.9103 × 10−7	2.6957 × 10−7	1.1904 × 10−6
	MPSPSO-ST	5.6834 × 10−16	1.9825 × 10−13	2.0811 × 10−14	1.9440 × 10−13
$f_2$	Standard PSO	0.1209	0.7743	0.2986	0.6429
	Basic PSO	56.7585	142.5399	102.9728	104.4540
	MPSPSO	0.0306	0.1479	0.0640	0.1217
	MPSPSO-ST	0.0191	0.0732	0.0396	0.0644
$f_3$	Standard PSO	4.3416 × 101	1.1851 × 102	7.6565 × 101	9.3955 × 101
	Basic PSO	2.7751 × 102	8.5694 × 102	4.8829 × 102	5.5224 × 102
	MPSPSO	1.8069 × 100	1.4074 × 101	5.7546 × 100	1.5430 × 101
	MPSPSO-ST	7.9038 × 10−3	2.3292 × 10−1	7.0004 × 10−2	2.5658 × 10−1
$f_4$	Standard PSO	1.9400 × 10−7	4.0724 × 10−3	4.4258 × 10−4	4.1605 × 10−3
	Basic PSO	7.3786 × 10−2	1.2261 × 100	6.5420 × 10−1	1.3869 × 100
	MPSPSO	4.6486 × 10−21	1.7877 × 10−16	1.2995 × 10−17	1.7277 × 10−16
	MPSPSO-ST	1.8112 × 10−49	2.6982 × 10−40	1.9660 × 10−41	2.6120 × 10−40
$f_5$	Standard PSO	1.02591	12.0564	4.08932	12.80900
	Basic PSO	5.2064 × 104	1.2287 × 105	8.5260 × 104	9.1677 × 104
	MPSPSO	0.66672	3.14810	1.26960	3.71450
	MPSPSO-ST	0.66667	2.03490	0.79497	1.70870
$f_6$	Standard PSO	2.5432 × 10−3	2.4399 × 10−2	7.9220 × 10−3	2.4535 × 10−2
	Basic PSO	8.2531 × 101	1.6702 × 102	1.3159 × 102	8.8401 × 101
	MPSPSO	1.5414 × 10−8	1.0729 × 10−6	2.3913 × 10−7	1.1474 × 10−6
	MPSPSO-ST	6.1306 × 10−16	5.1809 × 10−13	5.0002 × 10−14	5.2461 × 10−13
$f_7$	Standard PSO	3.1183 × 10−2	2.5205 × 10−1	1.0527 × 10−1	2.7090 × 10−1
	Basic PSO	1.3802 × 103	2.3578 × 103	1.8650 × 103	1.2075 × 103
	MPSPSO	5.7551 × 10−7	7.9424 × 10−5	8.8537 × 10−6	7.6769 × 10−5
	MPSPSO-ST	5.7456 × 10−16	7.9810 × 10−14	2.7045 × 10−14	1.0692 × 10−13
$f_8$	Standard PSO	4.7811 × 10−4	4.7318 × 10−2	7.0241 × 10−3	4.8574 × 10−2
	Basic PSO	6.0663 × 101	1.3860 × 102	1.0412 × 102	1.0270 × 102
	MPSPSO	2.9684 × 10−13	3.2834 × 10−10	5.1454 × 10−11	3.7774 × 10−10
	MPSPSO-ST	7.5201 × 10−31	5.7745 × 10−26	7.5066 × 10−27	6.5486 × 10−26
$f_9$	Standard PSO	0.29987	0.49987	0.43487	0.25593
	Basic PSO	1.10070	1.5999	1.34680	0.57279
	MPSPSO	0.29987	0.49987	0.42487	0.27839
	MPSPSO-ST	0.29987	0.49987	0.36518	0.25683
$f_{10}$	Standard PSO	−1.7207 × 10−9	−9.1788 × 10−10	−1.2473 × 10−9	9.7966 × 10−10
	Basic PSO	−9.3338 × 10−10	−6.8522 × 10−10	−8.0715 × 10−10	3.4338 × 10−10
	MPSPSO	−2.3176 × 10−9	−1.1251 × 10−9	−1.8146 × 10−9	1.4152 × 10−9
	MPSPSO-ST	−3.0052 × 10−9	−2.4403 × 10−9	−2.7811 × 10−9	7.4879 × 10−10
$f_{11}$	Standard PSO	1.4317 × 10−5	3.2392 × 10−2	9.4530 × 10−3	3.6992 × 10−2
	Basic PSO	1.0250 × 100	1.0392 × 100	1.0335 × 100	1.9501 × 10−2
	MPSPSO	2.0319 × 10−8	3.6910 × 10−2	1.0222 × 10−2	4.3936 × 10−2
	MPSPSO-ST	3.9968 × 10−15	8.0685 × 10−2	2.0720 × 10−2	1.1602 × 10−1
$f_{12}$	Standard PSO	1.1716 × 10−5	1.0372 × 10−1	5.2801 × 10−3	1.0100 × 10−1
	Basic PSO	4.1217 × 100	6.0335 × 100	5.2257 × 100	2.4996 × 100
	MPSPSO	1.4159 × 10−9	1.0367 × 10−1	1.5550 × 10−2	1.6555 × 10−1
	MPSPSO-ST	9.1102 × 10−17	3.9616 × 100	8.3068 × 10−1	4.6458 × 100
$f_{13}$	Standard PSO	3.4293	6.0435	4.5710	3.6773
	Basic PSO	124.6619	182.6454	150.8625	74.4685
	MPSPSO	2.3077	5.9331	3.7067	3.5467
	MPSPSO-ST	2.4172	11.536	5.3201	11.0171
$f_{14}$	Standard PSO	7.409 × 10−6	4.3215 × 10−4	8.4032 × 10−5	4.1803 × 10−4
	Basic PSO	1.6260 × 100	4.1097 × 100	2.4549 × 100	2.8544 × 100
	MPSPSO	6.8133 × 10−11	2.3492 × 10−8	2.4795 × 10−9	2.4440 × 10−8
	MPSPSO-ST	7.6029 × 10−17	2.8983 × 10−3	1.6939 × 10−4	2.8349 × 10−3
$f_{15}$	Standard PSO	2.5207 × 10−2	2.0130 × 10−1	7.7555 × 10−2	2.3124 × 10−1
	Basic PSO	2.7387 × 101	3.6274 × 101	3.1328 × 101	1.0927 × 101
	MPSPSO	4.5570 × 10−5	4.0095 × 10−3	1.1036 × 10−3	4.9777 × 10−3
	MPSPSO-ST	6.5547 × 10−8	3.0083 × 10−4	3.6657 × 10−5	3.3579 × 10−4
$f_{16}$	Standard PSO	3.0000	3.0000	3.0000	6.7349 × 10−15
	Basic PSO	3.0099	3.4032	3.1322	5.5096 × 10−1
	MPSPSO	3.0000	3.0000	3.0000	1.9357 × 10−15
	MPSPSO-ST	3.0000	3.0000	3.0000	0
$f_{17}$	Standard PSO	0.9980	7.8740	2.6270	9.4631
	Basic PSO	0.9980	7.8740	2.0372	6.5904
	MPSPSO	0.9980	2.9821	1.4449	2.9697
	MPSPSO-ST	0.9980	0.9980	0.9980	0
$f_{18}$	Standard PSO	4.3426 × 10−4	1.1096 × 10−3	8.6577 × 10−4	7.7543 × 10−4
	Basic PSO	5.8557 × 10−4	1.9988 × 10−3	1.2394 × 10−3	1.6428 × 10−3
	MPSPSO	3.0749 × 10−4	1.0349 × 10−3	6.5378 × 10−4	1.4993 × 10−3
	MPSPSO-ST	3.0749 × 10−4	1.0383 × 10−3	4.0910 × 10−4	1.0838 × 10−3
$f_{19}$	Standard PSO	1.8802 × 10−17	5.3438 × 10−15	9.9794 × 10−16	7.0982 × 10−15
	Basic PSO	1.6320 × 100	4.5516 × 100	2.8228 × 100	3.6775 × 100
	MPSPSO	8.1831 × 10−38	3.5528 × 10−33	2.2073 × 10−34	3.4617 × 10−33
	MPSPSO-ST	3.2993 × 10−58	2.8784 × 10−55	5.5092 × 10−56	3.2355 × 10−55
$f_{20}$	Standard PSO	−3.3220	−3.2031	−3.2744	0.2605
	Basic PSO	−3.1587	−2.5910	−2.9131	0.8005
	MPSPSO	−3.3220	−3.2031	−3.2566	0.2645
	MPSPSO-ST	−3.3220	−3.2031	−3.2982	0.2126

. The convergence diagrams of the four algorithms are shown in Figure 5, Figure 6 and Figure 7. The values in the diagrams are the mean best results of 20 runs of the whole swarm so far.

Figure 5 clearly shows that compared to the other three algorithms, MPSPSO-ST has very prominent advantages, which offers the fastest convergence speed and the strongest optimization accuracy. From Figure 6a,d,e,f, we observe that for multimodal functions $f_{10}$, $f_{15}$, $f_{17}$, $f_{18}$, MPSPSO-ST owns faster convergence speed and better optimization performance compared with the other three algorithms, especially these advantages are more outstanding for $f_{10}$, $f_{15}$, $f_{17}$. However, for multimodal functions $f_{11}$, $f_{13}$ in Figure 6b,c, the performance of MPSPSO-ST is not as good as that of MPSPSO and Standard PSO. Nevertheless, MPSPSO-ST can maintain a faster convergence rate when the final optimization results of MPSPSO-ST and MPSPSO, Standard PSO are quite close. As seen from Figure 7, the convergence speed and global search capability of MPSPSO-ST are the most competitive for multimodal functions $f_{19}$, $f_{20}$.

The information in Table 2 can reflect the above conclusions more specifically and accurately. The indicators in Table 2 are the best values (the best), the worst value (the worst), mean value (mean), standard deviation (S.D.) of MPSPSO-ST and Standard PSO and Basic PSO, MPSPSO in 20 independent running experiments. Among all unimodal functions $f_1~f_8$ and multimodal functions $f_9$, $f_{10}$, $f_{15}$, $f_{16}$, $f_{17}$, $f_{18}$, $f_{19}$ and $f_{20}$, MPSPSO-ST overtakes all other methods in terms of convergence rate and optimization accuracy. Especially for multimodal functions $f_{16}$ and $f_{17}$, MPSPSO-ST can all reach the theoretical optimal value in a few iterations and the S.D. of MPSPSO-ST can reach 0. It means that MPSPSO-ST can achieve theoretical optimal value in each of 20 independent experiments, which shows MPSPSO-ST has steadier search ability than the other three algorithms during the search process. For multimodal functions $f_{11}$, $f_{12}$ and $f_{14}$, the optimization performance of MPSPSO-ST is not optimal, but it is the best at The best, indicating that MPSPSO-ST has obtained the best optimization results in the optimization of these test functions but the overall has not been maintained. Meanwhile, it can be seen that the standard deviation of MPSPSO-ST is smaller than the other three algorithms in a majority of test functions, illustrating that it has stronger robustness and stability.

As seen from Table 2, Figure 5, Figure 6 and Figure 7, we can conclude that the proposed MPSPSO-ST owns better optimization capability compared with Standard PSO, Basic PSO and MPSPSO, which indicates that MPSPSO-ST has excellent ability in solving numerical optimization problems.

4.2. Comparison of MPSPSO-ST with CPSO, PSO-NDAC, AIWCPSO, DE, MFO and SCA

In this section, we have designed comparisons of MPSPSO-ST with several classic improved PSO variants (chaos particle swarm optimization (CPSO) [53] particle swarm optimization with the nonlinear dynamic acceleration coefficients (PSO-NDAC) [32] AIWCPSO [54]) and other well-known classic optimization algorithms (differential evolution algorithm (DE), moth-flame optimization algorithm (MFO) and sine cosine algorithm (SCA)). CPSO realizes chaotic searching on the current global best individual using chaos mechanism. PSO-NDAC proposes the nonlinear dynamic acceleration coefficients to add to PSO. AIWCPSO designs an inertia weight strategy that the adaptive inertia weight is adjusted according to the updated particles of the previous generation. DE is a group-based adaptive global optimization algorithm, which belongs to an evolutionary algorithm. MFO is a swarm intelligence optimization algorithm, the main inspiration of which is the navigation method of moths in nature called transverse orientation. SCA creates multiple initial random candidate solutions and requires them to fluctuate outwards or towards the best solution based on sine and cosine functions. The parameter settings for each algorithm are shown in

Table 3. Parameter settings for MPSPSO-ST and particle swarm optimization with the nonlinear dynamic acceleration coefficients (PSO-NDAC), chaos particle swarm optimization (CPSO), AIWCPSO, moth-flame optimization (MFO), sine cosine algorithm (SCA) and differential evolution (DE).

Algorithm	Population Size	Iteration	run times	Parameter Settings
PSO-NDAC	40	500	20	$\begin{array}{l}{c}_1=-2\times m^2+2.5,c_2=0.5\times (1-m{)}^2+2.5\times m\\ (m=\frac{t}{t_{\max }}),\omega =0.9~0.4,V_{\max }=6\end{array}$
CPSO	40	500	20	$c_1=c_2=2,\omega =0.9~0.4,\mu =4,V_{\max }=6$
AIWCPSO	40	500	20	$c_1=c_2=2,\omega =0.9~0.4,V_{\max }=6$
MFO	40	500	20	t is random number in the range[−2, 1]
SCA	40	500	20	$r_1=4~0$, $r_2$ is a random number in the range [0, 2π], $r_3$ is a random number in the range [0, 2], $r_4$ is a random number in the range [0, 1]
DE	40	500	20	F=0.3, CR=0.5
MPSPSO-ST	40	500	20	$c_1=-\partial \times m^2\times \tan [\frac{\pi }{8}\times (1+m^2)]+\theta +\rho \times z,$$c_2=-\partial \times (1-m{)}^2\times \tan [\frac{\pi }{8}\times (1+(1-m{)}^2)]+\theta +\rho \times z,$${\omega }_{t+1}=\varphi \times \sin (\pi {\omega }_t)+\tau ,V_{\max }=6$

.

The experimental results obtained by the test functions listed in the Appendix A are shown in

Table 4. Experimental results for MPSPSO-ST and PSO-NDAC, CPSO, AIWCPSO, MFO, SCA and DE on 20 classical test functions.

Function	Algorithm	The Best	The Worst	Mean	S.D.
$f_1$	PSO-NDAC	5.1300 × 10−8	3.3867 × 10−5	2.6932 × 10−6	3.2888 × 10−5
	CPSO	2.5381 × 10−1	3.3147 × 101	7.9537 × 100	4.6559 × 101
	AIWCPSO	1.6492 × 10−6	2.8529 × 10−5	8.0015 × 10−6	2.8947 × 10−5
	MFO	5.3151 × 10−1	1.0000 × 104	5.0147 × 102	9.7458 × 103
	SCA	1.2638 × 10−15	2.2184 × 102	1.1287 × 101	2.1604 × 102
	DE	6.1421 × 10−3	8.6630 × 101	1.3950 × 101	9.8238 × 101
	MPSPSO-ST	8.7871 × 10−16	2.5463 × 10−13	8.1821 × 10−14	3.3863 × 10−13
$f_2$	PSO-NDAC	0.02018	0.09354	0.05649	0.08375
	CPSO	0.05965	2.81610	1.25650	3.50140
	AIWCPSO	0.03554	0.08707	0.05680	0.05934
	MFO	0.05698	29.70930	3.39012	31.62590
	SCA	0.00815	0.31364	0.06917	0.37656
	DE	0.03722	0.20296	0.10343	0.19898
	MPSPSO-ST	0.01611	0.05752	0.03274	0.04851
$f_3$	PSO-NDAC	8.8027 × 100	8.9787 × 101	2.7801 × 101	8.7341 × 101
	CPSO	2.4701 × 101	1.0485 × 102	6.0563 × 101	9.0587 × 101
	AIWCPSO	1.9160 × 101	1.1344 × 102	4.5443 × 101	9.0026 × 101
	MFO	1.7315 × 103	5.3692 × 104	1.9764 × 104	5.0273 × 104
	SCA	2.1712 × 100	1.2644 × 104	1.9731 × 103	1.8014 × 104
	DE	5.8248 × 103	2.5593 × 104	1.2039 × 104	2.2101 × 104
	MPSPSO-ST	4.1427 × 10−2	5.9152 × 10−1	2.1145 × 10−1	7.7382 × 10−1
$f_4$	PSO-NDAC	4.0015 × 10−25	2.6450 × 10−19	3.5505 × 10−20	3.5317 × 10−19
	CPSO	2.5551 × 10−4	8.0566 × 10−3	2.2690 × 10−3	8.7752 × 10−3
	AIWCPSO	3.0713 × 10−18	3.1036 × 10−12	2.2484 × 10−13	3.1787 × 10−12
	MFO	8.7526 × 10−14	2.4125 × 10−7	1.2382 × 10−8	2.3484 × 10−7
	SCA	1.7091 × 10−56	1.3950 × 10−3	6.9753 × 10−5	1.3597 × 10−3
	DE	1.0221 × 10−18	3.9932 × 10−6	2.0004 × 10−7	3.8918 × 10−6
	MPSPSO-ST	1.8781 × 10−49	6.6562 × 10−41	1.1225 × 10−41	9.4197 × 10−41
$f_5$	PSO-NDAC	6.6668 × 10−1	6.5562 × 100	1.6663 × 100	7.0782 × 100
	CPSO	1.1582 × 100	3.4957 × 104	1.0897 × 104	5.4315 × 104
	AIWCPSO	6.6749 × 10−1	4.7360 × 100	1.3134 × 100	5.3422 × 100
	MFO	1.9549 × 100	7.2585 × 104	1.0531 × 104	1.1194 × 105
	SCA	6.6713 × 10−1	7.4345 × 101	4.5179 × 100	7.1707 × 101
	DE	3.0014 × 100	1.5603 × 103	1.3819 × 102	1.6222 × 103
	MPSPSO-ST	6.6667 × 10−1	3.5021 × 100	1.0923 × 100	3.7675 × 100
$f_6$	PSO-NDAC	1.7100 × 10−8	1.4269 × 10−5	2.1761 × 10−6	1.4714 × 10−5
	CPSO	1.4766 × 100	3.7322 × 101	1.1397 × 101	3.7236 × 101
	AIWCPSO	1.2828 × 10−6	1.7912 × 10−5	6.8441 × 10−6	2.1667 × 10−5
	MFO	5.0435 × 10−1	9.9013 × 103	1.0440 × 103	1.3243 × 104
	SCA	4.3620 × 100	1.4545 × 101	5.5499 × 100	9.8103 × 100
	DE	6.2442 × 10−10	1.5620 × 102	2.3732 × 101	1.9729 × 102
	MPSPSO-ST	1.0678 × 10−15	4.8348 × 10−13	7.7464 × 10−14	5.0284 × 10−13
$f_7$	PSO-NDAC	4.4619 × 10−7	6.0106 × 10−5	8.5332 × 10−6	6.1709 × 10−5
	CPSO	5.7821 × 101	1.1138 × 103	5.0476 × 102	1.3492 × 103
	AIWCPSO	3.5964 × 10−6	7.4185 × 10−5	2.1497 × 10−5	8.3628 × 10−5
	MFO	3.0001 × 10−2	1.5001 × 103	4.5025 × 102	2.0993 × 103
	SCA	1.8083 × 10−22	1.9105 × 101	1.0553 × 100	1.8618 × 101
	DE	7.3789 × 10−5	1.4108 × 101	1.3408 × 100	1.4138 × 101
	MPSPSO-ST	4.6508 × 10−16	1.1855 × 10−12	1.8800 × 10−14	1.5397 × 10−12
$f_8$	PSO-NDAC	7.8413 × 10−14	1.4895 × 10−9	1.4322 × 10−10	1.6302 × 10−9
	CPSO	1.2651 × 10−2	2.7074 × 100	8.0086 × 10−1	3.2157 × 100
	AIWCPSO	6.6641 × 10−11	1.6291 × 10−7	1.1648 × 10−8	1.6120 × 10−7
	MFO	6.5346 × 10−6	1.3422 × 101	1.3437 × 100	1.3419 × 101
	SCA	6.8560 × 10−28	1.4863 × 100	7.8368 × 10−2	1.4465 × 100
	DE	2.9936 × 10−7	3.3894 × 10−2	2.9902 × 10−3	3.2996 × 10−2
	MPSPSO-ST	2.2820 × 10−31	2.5041 × 10−26	3.3096 × 10−27	2.8730 × 10−26
$f_9$	PSO-NDAC	0.29987	0.59987	0.41987	0.36332
	CPSO	0.20245	0.89988	0.59128	0.87943
	AIWCPSO	0.19987	0.59987	0.41994	0.38963
	MFO	1.29987	12.19990	5.56991	17.25210
	SCA	0.09988	4.55470	0.61124	4.18660
	DE	0.29987	1.39990	0.50236	1.23470
	MPSPSO-ST	0.29987	0.49987	0.38987	0.27928
$f_{10}$	PSO-NDAC	−2.9410 × 10−9	−1.8034 × 10−9	−2.3857 × 10−9	1.1856 × 10−9
	CPSO	−1.1181 × 10−9	−8.6693 × 10−10	−9.9583 × 10−10	2.7155 × 10−10
	AIWCPSO	−2.8340 × 10−9	−1.6520 × 10−9	−2.4220 × 10−9	1.1758 × 10−9
	MFO	−2.6658 × 10−9	−2.0384 × 10−9	−2.3715 × 10−9	6.6030 × 10−10
	SCA	−7.1394 × 10−10	−1.1393 × 10−10	−2.6959 × 10−10	7.6044 × 10−10
	DE	−2.1077 × 10−9	−1.6630 × 10−9	−1.8151 × 10−9	4.4156 × 10−10
	MPSPSO-ST	−3.0193 × 10−9	−2.3895 × 10−9	−2.7955 × 10−9	6.9722 × 10−10
$f_{11}$	PSO-NDAC	1.8235 × 10−8	1.1349 × 100	5.8798 × 10−1	2.1714 × 100
	CPSO	1.2241 × 10−2	7.0711 × 10−1	2.4554 × 10−1	1.0575 × 100
	AIWCPSO	4.8976 × 10−6	2.7095 × 10−2	8.7549 × 10−3	3.7409 × 10−2
	MFO	5.0123 × 10−1	9.1002 × 101	9.9449 × 100	1.2071 × 102
	SCA	7.8826 × 10−15	2.5680 × 100	4.5262 × 10−1	2.7232 × 100
	DE	9.1420 × 10−4	2.0907 × 100	6.3608 × 10−1	2.9071 × 100
	MPSPSO-ST	6.6613 × 10−16	9.3347 × 10−2	1.9044 × 10−2	9.7708 × 10−2
$f_{12}$	PSO-NDAC	5.7978 × 10−11	1.0370 × 10−1	1.0369 × 10−2	1.3911 × 10−1
	CPSO	2.9995 × 10−1	2.0577 × 100	7.9385 × 10−1	2.0733 × 100
	AIWCPSO	5.5656 × 10−8	3.1096 × 10−1	5.1833 × 10−2	3.7374 × 10−1
	MFO	2.0323 × 100	1.4052 × 101	7.0648 × 100	1.6366 × 101
	SCA	5.4712 × 10−1	2.0995 × 101	1.8558 × 100	1.9818 × 101
	DE	1.4196 × 10−1	3.7376 × 104	1.8701 × 103	3.6428 × 104
	MPSPSO-ST	6.2512 × 10−17	3.9495 × 100	1.1213 × 100	5.1966 × 100
$f_{13}$	PSO-NDAC	2.3083	5.7135	3.5316	3.5264
	CPSO	10.3682	56.6080	24.7796	48.5283
	AIWCPSO	1.6613	4.6190	3.0787	4.1172
	MFO	2.8600	56.0777	18.1767	66.0427
	SCA	26.0094	34.1472	29.7011	10.6788
	DE	0.0038	0.8927	0.4323	1.0218
	MPSPSO-ST	2.4172	8.6693	5.3316	9.3605
$f_{14}$	PSO-NDAC	1.1023 × 10−9	2.4166 × 10−6	2.7192 × 10−7	2.7198 × 10−6
	CPSO	3.6970 × 10−2	1.1233 × 100	3.4541 × 10−1	1.1816 × 100
	AIWCPSO	2.6544 × 10−8	2.3484 × 10−6	4.0181 × 10−7	2.6397 × 10−6
	MFO	6.7555 × 10−1	1.1573 × 102	2.4571 × 101	1.2484 × 102
	SCA	8.2348 × 10−24	5.5803 × 10−2	3.0040 × 10−3	5.4329 × 10−2
	DE	1.7097 × 10−4	6.8415 × 10−1	5.6675 × 10−2	6.5169 × 10−1
	MPSPSO-ST	1.7084 × 10−17	1.1430 × 10−4	1.1812 × 10−5	1.4598 × 10−4
$f_{15}$	PSO-NDAC	5.6249 × 10−4	3.7831 × 10−1	2.7575 × 10−2	3.6192 × 10−1
	CPSO	2.9421 × 100	1.7870 × 101	1.0966 × 101	2.0972 × 101
	AIWCPSO	3.3407 × 10−4	3.0759 × 10−3	8.7949 × 10−4	3.3632 × 10−3
	MFO	1.9292 × 10−2	2.2203 × 101	6.5223 × 100	2.8871 × 101
	SCA	4.9150 × 10−5	7.1893 × 10−1	3.7656 × 10−2	6.9966 × 10−1
	DE	5.4837 × 10−7	3.3238 × 10−2	4.0703 × 10−3	3.2331 × 10−2
	MPSPSO-ST	1.0274 × 10−8	9.8253 × 10−4	1.7602 × 10−4	1.4203 × 10−3
$f_{16}$	PSO-NDAC	3.0000	3.0000	3.0000	1.8841 × 10−15
	CPSO	3.0008	3.4160	3.0888	5.3477 × 10−1
	AIWCPSO	3.0000	3.0000	3.0000	1.9357 × 10−15
	MFO	3.0000	3.0000	3.0000	8.2725 × 10−15
	SCA	3.0000	3.0010	3.0002	1.0726 × 10−3
	DE	3.0000	3.0000	3.0000	1.8310 × 10−15
	MPSPSO-ST	3.0000	3.0000	3.0000	0
$f_{17}$	PSO-NDAC	0.99800	1.99200	1.09740	1.33360
	CPSO	4.14576	28.82710	13.89500	20.24470
	AIWCPSO	0.99800	5.92880	1.78910	6.63320
	MFO	0.99800	5.92880	1.59210	5.30030
	SCA	0.99801	2.98210	1.09930	1.93180
	DE	0.99801	10.76320	2.08246	11.49000
	MPSPSO-ST	0.99800	0.99800	0.99800	0
$f_{18}$	PSO-NDAC	3.0749 × 10−4	1.0028 × 10−3	5.9124 × 10−4	1.1285 × 10−3
	CPSO	6.6030 × 10−4	4.0282 × 10−2	6.4056 × 10−3	5.0951 × 10−2
	AIWCPSO	3.0749 × 10−4	1.5941 × 10−3	7.4898 × 10−4	1.5257 × 10−3
	MFO	3.7221 × 10−4	1.6554 × 10−3	9.5160 × 10−4	1.7842 × 10−3
	SCA	8.1546 × 10−4	1.6696 × 10−3	1.3903 × 10−3	9.2161 × 10−4
	DE	3.1525 × 10−4	4.6017 × 10−3	1.1767 × 10−3	3.9121 × 10−3
	MPSPSO-ST	3.0749 × 10−4	1.0371 × 10−3	3.8036 × 10−4	9.7776 × 10−4
$f_{19}$	PSO-NDAC	1.5069 × 10−33	7.4158 × 10−30	6.4473 × 10−31	7.2887 × 10−30
	CPSO	2.1384 × 10−2	2.2297 × 101	5.7052 × 100	3.8606 × 101
	AIWCPSO	2.2815 × 10−30	1.4904 × 10−27	2.6907 × 10−28	1.8522 × 10−27
	MFO	3.3701 × 10−23	1.3478 × 10−19	3.2070 × 10−20	2.0313 × 10−19
	SCA	3.3833 × 10−48	5.0261 × 10−12	2.5131 × 10−13	4.8989 × 10−12
	DE	8.7979 × 10−27	8.2996 × 10−2	5.9547 × 10−3	8.4550 × 10−2
	MPSPSO-ST	3.3480 × 10−58	4.6387 × 10−55	3.2719 × 10−56	4.4551 × 10−55
$f_{20}$	PSO-NDAC	−3.3220	−3.2031	−3.2804	0.2536
	CPSO	−3.2242	−2.6097	−3.0177	0.6943
	AIWCPSO	−3.3220	−3.2031	−3.2744	0.2605
	MFO	−3.3220	−3.1376	−3.2351	0.2619
	SCA	−3.0134	−2.9619	−2.9892	0.0738
	DE	−3.3220	−3.2030	−3.2799	0.2521
	MPSPSO-ST	−3.3220	−3.2031	−3.2982	0.2127

. The convergence diagrams of the seven algorithms are shown in Figure 8, Figure 9 and Figure 10, and the values in the diagrams are the mean best results in 20 runs in the whole swarm so far.

As seen from Figure 8, MPSPSO-ST has an undeniable advantage in terms of convergence rate and optimization accuracy for unimodal functions $f_1,f_3,f_4,f_6$ compared with the other six algorithms. Figure 9a–c,f show that MPSPSO-ST also has the best convergence rate and solution accuracy for multimodal functions $f_7$, $f_8$, $f_{10}$, $f_{15}$. Through Figure 9d,e, we find that the search performance of MPSPSO-ST is not optimal for multimodal functions $f_{11}$, $f_{14}$. For multimodal function $f_{11}$, MPSPSO-ST has the best performance except for AIWCPSO. Furthermore, in the case where the optimization effect of MPSPSO-ST is very close to the optimization effect of AIWCPSO, MPSPSO-ST has a faster convergence speed to the global optimal solution than AIWCPSO. For multimodal functions $f_{17}$, $f_{18}$, $f_{19}$, $f_{20}$ in Figure 10, MPSPSO-ST outperforms the other six methods.

As shown in Table 4, for multimodal functions $f_{11}$ and $f_{12}$, MPSPSO-ST is not the best but The best it can find is the greatest compared to other algorithms, which shows that MPSPSO -ST has the ability to find the best results in 20 independent experiments but it has not been maintained in general. Besides, for multimodal functions $f_{16}$, $f_{17}$, the theoretical optimal value can be successfully found each time by MPSPSO-ST in 20 independent experiments, which indicates that the search performance of MPSPSO-ST is very stable. Therefore, we can see that MPSPSO-ST has prominent advantages in robustness and stability.

Combining Table 4 and Figure 8, Figure 9 and Figure 10, among the 20 classical benchmark functions $f_1~f_{20}$, referring to two indicators, mean (mean) and standard deviation (S.D.) in Table 4, MPSPSO-ST is significantly superior to the other six algorithms in searching better global optimal value for these 16 test functions $f_1~f_{10}$, $f_{15}~f_{20}$. In addition, although the final optimization result of MPSPSO-ST is inferior to AIWCPSO for multimodal function $f_{11}$, it is extremely close to AIWCPSO in terms of optimization effect and has the fastest convergence speed, and its convergence accuracy is better than the five other algorithms except for AIWCPSO, so MPSPSO-ST is used as the most efficient optimization algorithm. For multimodal functions $f_{11}$, $f_{14}$, AIWCPSO shows the best performance. For multimodal function $f_{12}$, PSO-NDAC has the best performance. The performance of AIWCPSO and PSO-NDAC is similar in other test functions, second only to MPSPSO-ST, so AIWCPSO and PSO-NDAC are tied as the second most effective optimization algorithm. DE is listed as the third most effective algorithm, achieving the best solution on multimodal function $f_{13}$. Although SCA has not got the best solution in the optimization of any test function, from the indicator The best in Table 4, SCA has found better optimal values than other algorithms in 20 independent experiments for unimodal function $f_7$ and multimodal function $f_{14}$, but the lack of maintenance leads to a general mean. So SCA is listed as the fourth most influential optimization algorithm. For CPSO and MFO, they are unable to find a better global optimal solution than all other algorithms in all test functions.

To sum up, it can be concluded that MPSPSO-ST offers superior overall performance among all seven algorithms, followed by AIWCPSO and PSO- NDAC, then DE, then SCA, and finally CPSO and MFO.

In general, each category used to assess a specified behavior in the optimization algorithm. The unimodal functions are used to assess the convergence rate of the algorithm since they contain a single extreme solution in the search domain ($f_1~f_8$). Meanwhile, the multimodal functions are used to evaluate the ability of the algorithm to avoid the local point and reach to global solution since they contain more than an extreme solution ($f_9~f_{20}$) 52In the above two experiment sections, among 20 test functions applied in this paper, MPSPSO-ST is significantly superior to other comparison algorithms for all unimodal functions and most multimodal functions, which reflects that MPSPSO-ST has an excellent convergence speed and the ability to get rid of local extreme values.

5. Conclusions

In this paper, we propose the MPSPSO-ST algorithm, which can facilitate the algorithm performance of the traditional PSO. Firstly, we propose a hybrid multi-step probability selection update mechanism, i.e., the MPSPSO algorithm. Secondly, based on the MPSPSO algorithm, in order to achieve a good balance between exploration and exploitation to optimize the performance of the algorithm, we design the sine chaotic inertial weight and symmetric tangent chaotic acceleration coefficients, which are integrated into the MPSPSO-ST algorithm. Finally, to appraise the effectiveness of the MPSPSO-ST algorithm, we conduct extensive experiments with 20 classic benchmark functions and the experiments consisted of two groups. The first group of experiments is the comparisons of MPSPSO-ST with the Standard PSO, Basic PSO and MPSPSO. The experimental results show that the hybrid multi-step probability selection update mechanism proposed in this paper is very effective. The sine chaotic inertial weight and the symmetric tangent chaotic acceleration coefficients proposed on this basis can effectively maintain swarm diversity in the search process, which enhances the performance of MPSPSO-ST significantly. The second group of experiments is the comparisons of the MPSPSO-ST algorithm with three classical improved PSO variants (CPSO, PSO-NDAC, AIWCPSO) and three well-known classic optimization algorithms (DE, MFO, SCA). The experimental results show that the MPSPSO-ST algorithm surpasses all other six algorithms obviously on the majority of the classic benchmark functions, with faster convergence speed, higher optimization accuracy, and greater stability robustness, which is successful in solving numerical optimization tasks. MPSPSO-ST has the basic characteristics of PSO, simple, easy to realize and other advantages while taking the search accuracy and search efficiency into account. It can further avoid premature convergence to a certain extent. Besides, MPSPSO-ST does not introduce any new parameters, making it more versatile and easier to operate and realize. Indeed, the proposed modification implies increasing complexity of the algorithm, but it is worthwhile in order to solve complex numerical optimization problems more efficiently.

Therefore, the MPSPSO-ST algorithm proposed in this paper is an excellent choice for solving complex numerical optimization problems. In the future, we will further develop the strategies of parameter tuning and study the application of the MPSPSO-ST algorithm in practical problems.