A Particle Swarm Algorithm Based on a Multi-Stage Search Strategy

Particle swarm optimization (PSO) has the disadvantages of easily getting trapped in local optima and a low search accuracy. Scores of approaches have been used to improve the diversity, search accuracy, and results of PSO, but the balance between exploration and exploitation remains sub-optimal. Many scholars have divided the population into multiple sub-populations with the aim of managing it in space. In this paper, a multi-stage search strategy that is dominated by mutual repulsion among particles and supplemented by attraction was proposed to control the traits of the population. From the angle of iteration time, the algorithm was able to adequately enhance the entropy of the population under the premise of satisfying the convergence, creating a more balanced search process. The study acquired satisfactory results from the CEC2017 test function by improving the standard PSO and improved PSO.

PSO was proposed by Kennedy and Eberhart [7] in 1995 as a population-based heuristic optimization algorithm. With the advantages of a simple implementation, high efficiency, and few parameters, it is widely used in fields such as path planning [8], image segmentation [9], neural networks [10,11], data prediction [12], and noise control [13]. However, PSO is prone to getting trapped in local optima and lacks search accuracy in late iterations. To address these issues, PSO has been improved from four main points of view.
(1) Parameter tuning. Gunasundari et al. [14] proposed velocity-bounded Boolean PSO (VbBoPSO) based on binary PSO (BPSO), in which particles are initialized with random binary positions and velocities, with the velocity constrained to a specific range of values [15], to explore more regions and obtain better convergence. Sen et al. [16] proposed adaptive modified particle velocity PSO (MPV-PSO). A method to balance the search capability of PSO was proposed, using a strategy of linearly reducing the inertia weights [17]. Ratnaweera et al. [18] proposed a self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients (HPSO-TVAC), which varies the acceleration coefficients to improve the search capabilities. Zhan et al. [19] proposed adaptive particle swarm optimization (APSO) with adaptive control parameters. Chen et al. [20] proposed chaotic dynamic weight particle swarm optimization (CDW-PSO), using inertia quantities with chaotic mapping to modify the search direction.
(2) Learning strategy. PSO is prone to getting trapped in local optima, and the search accuracy is not sufficient in late iterations because it uses only two experiences to guide particle learning. Improving this aspect of the learning strategy has attracted much attention. Liang et al. [21] proposed a dynamic multi-swarm particle swarm optimizer (DMS-PSO) with a dynamic neighborhood structure, in which the learning of each particle is no longer restricted to one population. Liu et al. [22] proposed a hierarchical simple time hierarchy strategy (THSPSO) algorithm using different learning strategies in different search phases. Zhan et al. [23] proposed orthogonal learning particle swarm optimization (OLPSO) with orthogonal learning, in which each particle obtains useful information from its own historical best experience and that of its neighbors. Xu et al. [24] proposed twoswarm learning particle swarm optimization (TSLPSO) based on dimensional learning, which constructs a learning paradigm for each particle by learning each dimension of its individual optimal position from the corresponding dimension of the population optimal position. Finally, Li et al. [25] proposed a learning strategy based on the collaboration of multiple populations to achieve information sharing and co-evolution among populations.
(3) Topology. Kennedy [26] pointed out that the use of topology is effective for population-based algorithms. In structured populations, information is often exchanged between closely linked individuals based on fitness and topological relationships as a way to slow down convergence. Mendes et al. [27] proposed fully informed particle swarm optimization (FIPSO), which uses a fully informed strategy in which each particle is updated based on the historical best experience of its neighbors. Janson et al. [28] constructed a dynamically changing tree topology in which each particle learns from its parent, effectively using the information of each particle.
(4) Algorithm crossover. Hybrid algorithms are a key research area to improve the performance of PSO algorithms. They incorporate operators such as crossover, selection, mutation, and choice to improve the search quality of the population individuals and the general efficiency of the algorithm. For example, the use of genetic operators can improve population diversity and convergence to the global optimum. Hybrid algorithms can better escape local optima and overcome certain inherent drawbacks associated with single algorithms. Zhang et al. [29] proposed differential mutation and novel social learning PSO (DSPSO), which combined four differential variation operations with social learning particle swarm optimization (SLPSO). Nasiraghdam et al. [30] proposed a new approach based on a hybrid genetic algorithm and PSO. Related studies [31][32][33][34] showed that a hybrid PSO algorithm incorporating other evolutionary algorithms not only improves population diversity but prevents premature convergence and increases the probability of finding a global optimal solution.
In summary, the key to PSO improvement lies in balancing diversity and convergence, preventing premature convergence to a local optimum, and improving local exploitation. Previous work used entropy as an evaluation index to improve the algorithm but stopped at using entropy to constrain the population traits and did not explore a more optimal search process. Others divided up the search strategy using both new strategies or vastly different ones. This paper proposes a multi-stage search strategy to fully maintain the population diversity for the global search in the early stage and modifies the update formula in the late stage to help jump out of the local optima, improving the local search ability. While improving the algorithm exploitation ability and search accuracy, the original algorithm converges after adding the strategy. This strategy can be used as an improved operator and implemented in heterogeneous comprehensive learning particle swarm optimization (HCLPSO) and TSLPSO. Both of these algorithms divide sub-populations to balance exploration and exploitation; however, adding a new balancing strategy can have adverse effects. This paper explores these two representative algorithms, which are compatible with the new strategy, to illustrate the effectiveness of the strategy. The experimental results show that this strategy can effectively improve the probability of the algorithm converging to the global optimal solution.

Particle Swarm Optimization
PSO tends to a global convergence through the cooperation and competition of particles in the search space. The particle velocity and position updates are determined by the best position found by the corresponding particle (pbest) and the best position found by the whole population up to the current iteration of the algorithm (gbest). Let the total number of particles be N, the dimension D, and the maximum number of iterations maxiter. Then, the velocity of the ith particle at moment t is . , x im (t)] T , the best position found by the corresponding particle at moment t is pbest i (t) = [p i1 (t), p i2 (t), . . . , p im (t)] T , and the best position found by the whole population is gbest = [g 1 , g 2 , . . . , g m ] T . The particles are iteratively searched according to the following formula: where the inertia weight coefficients are the learning factors. r 1 and r 2 are random numbers between [0,1]. The two factors c 1 and c 2 , known as "acceleration coefficients", are positive constants commonly used to determine when the cognition speed of the ith particle is accelerated towards pbest and gbest, respectively. The symbol ω denotes the inertia weight parameter which was originally developed to address the velocity explosion problem.

TSLPSO Algorithm
Currently, TSLPSO is one of the algorithms that achieves superior results on the congress on evolutionary computation (CEC) test function [24], in which one sub-population uses the learning paradigm constructed by the dimensional learning strategy to guide the local search of particles, and the other uses the learning paradigm constructed by the integrated learning strategy to guide the global search. The learning paradigm constructed by dimensional learning can guide the particles to search in better regions; however, if the majority of particles are near a local optimum and are trapped there, then premature convergence occurs. To solve this problem, the integrated learning strategy is introduced to enhance population diversity and help particles escape local optima.

HCLPSO Algorithm
In HCLPSO, the population is divided into two sub-populations. The first aims to enhance exploration and the second to enhance exploitation. In both sub-populations, samples are generated using an integrated learning (CL) strategy with learning probability Pc to generate random numbers for each dimension of the particles and to compare them with their respective learning probability Pc i values. The velocity of the exploration subpopulation is updated as and the velocity of the exploitation sub-population as where indicates whether the i-th particle of each dimension d follows its own or another's pbest. When rand d i > Pc i , f i (d) = 1; when rand d i < Pc i , then two particles, namely X j and X k , are randomly selected from the sub-population of the ith particle. If fitness (X j ) ≤ fitness (X k ), then f i (d) = j; otherwise, f i (d) = k.
Since the exploring particles are not allowed to access the information of the exploiting particles, there is no information flow from the exploiting sub-population to the exploring sub-population. To prevent loss of diversity, exploring and exploiting individ-

Multi-Stage Search Strategy
The paper uses the multi-stage search strategy to improve the search results based on the original mechanisms of PSO, TSLPSO, and HCLPSO. The multi-stage search strategy is described below.
Before population iteration, each dimension of each particle is given an additional attribute Ra, the value of which is 1 or −1. As shown in Figure 1, blue signifies that the Ra attribute value of the particle is 1, and red signifies that the value of the particle is −1. In subsequent action behavior judgments, this value determines whether two particles are mutually attracted or repelled, i.e., particles attract each other with the opposite Ra value, but repel each other if they are the same. In the first stage, a certain proportion of better adapted particles are selected, and these will not be attracted or repelled, as indicated by the black box at the edge of the particles in Figure 1. The remaining particles will be affected by the better adapted particle swarm, except for those that change their position according to the original algorithm, i.e., those without the black border in the figure, and the strength of the influence is determined by the parameter pow. To prevent the phenomenon of premature clustering, the Euclidean distance between particles is calculated as when the distance is less than the threshold, the particles with poor fitness will be bounced away.
Since the exploring particles are not allowed to access the information of the exploiting particles, there is no information flow from the exploiting sub-population to the exploring sub-population. To prevent loss of diversity, exploring and exploiting individuals do not interact. Inertia weights and dynamic acceleration coefficients are used for both sub-populations.

Multi-Stage Search Strategy
The paper uses the multi-stage search strategy to improve the search results based on the original mechanisms of PSO, TSLPSO, and HCLPSO. The multi-stage search strategy is described below.
Before population iteration, each dimension of each particle is given an additional attribute Ra, the value of which is 1 or −1. As shown in Figure 1, blue signifies that the Ra attribute value of the particle is 1, and red signifies that the value of the particle is −1. In subsequent action behavior judgments, this value determines whether two particles are mutually attracted or repelled, i.e., particles attract each other with the opposite Ra value, but repel each other if they are the same. In the first stage, a certain proportion of better adapted particles are selected, and these will not be attracted or repelled, as indicated by the black box at the edge of the particles in Figure 1. The remaining particles will be affected by the better adapted particle swarm, except for those that change their position according to the original algorithm, i.e., those without the black border in the figure, and the strength of the influence is determined by the parameter pow. To prevent the phenomenon of premature clustering, the Euclidean distance between particles is calculated as when the distance is less than the threshold, the particles with poor fitness will be bounced away.
In the second stage, no operation is applied to the particle population. In the third stage, only the Ra of the optimal particle is kept unchanged, and all other particles have the opposite property value of the optimal particle in each dimension, i.e., they will be attracted by it. In the velocity update Formula (1), particles are pulled by the gbest and pbest points. This seems similar to the attraction and repulsion of this strategy, but in fact, the PSO easily traps in local optima because particles are influenced by the current gbest point in the early stage, and the poor search accuracy in the later stage is caused by the attraction of the pbest point. Existing improvement algorithms make dynamic linear changes to learning factors c1 and c2 in the velocity update Formula (1) to achieve the expected im- In the second stage, no operation is applied to the particle population. In the third stage, only the Ra of the optimal particle is kept unchanged, and all other particles have the opposite property value of the optimal particle in each dimension, i.e., they will be attracted by it.
In the velocity update Formula (1), particles are pulled by the gbest and pbest points. This seems similar to the attraction and repulsion of this strategy, but in fact, the PSO easily traps in local optima because particles are influenced by the current gbest point in the early stage, and the poor search accuracy in the later stage is caused by the attraction of the pbest point. Existing improvement algorithms make dynamic linear changes to learning factors c 1 and c 2 in the velocity update Formula (1) to achieve the expected improvement. However, from another perspective, the improvement strategy is independent of the velocity and position update Formulae (1) and (2), and this study sought to improve the algorithm in terms of exploration and exploitation and to control the shape of the population by relatively direct and simple means. From the analysis of the experimental results in Section 6, it can be seen that the algorithm's performance has a potential point of The position and velocity of all particles are updated according to velocity and position update Formulate (1) and (2), the position of the lagging particles will change according to the fitness value of the particles, and the direction of movement is determined by the Ra value. The Ra value is an inherent property of each particle: its positive or negative value determines repulsion or attraction between particles, i.e., particles attract those with the opposite Ra value, but repel those with the same. Each particle has a Ra value for each dimension.
As a result of the randomness of Ra, a lag particle action also has randomness. When the force value pow is large enough, the new interaction force between particles can dominate the movement of the lag particle swarm. The force value is calculated as where str is a constant parameter that determines pow, which changes the speed of the particle; the "searchrange" is limited by X min and X max . The particle itself has a speed, which means that these particles may be pushed away from the gbest point, may be close to the gbest point, or may oscillate in its vicinity. However, since the size of the lag particle population is fixed, only a certain number of particles will act to make up for the difference. If a lag particle improves its fitness beyond that of other particles after iteration, then other particles will replace it as the lag particle population. Put differently, this mechanism ensures that part of the particle swarm is exploited locally, and the rest is explored globally; this study sought a balance between them.
To limit premature convergence, which is the tendency to trap in local optima, the Euclidean distance between particles is incorporated in stage 1. Once the distance between any two particles reaches a threshold, the particle with the lag fitness value will be selected and a force applied to it in the opposite direction with respect to the gbest point, pushing it away from the region in which it was originally located. This strategy coexists with the strategy repulsion and attraction mentioned above, and they affect each other.

Particle Action Definition for Stage 2
As shown in Figure 2, stage 2 is mainly used to judge the population traits and control the transition of the stage; in this study, no additional actions were performed on the particles. By mapping the diversity and convergence of the improvement algorithm, in this study, the parameters were set to control the end of stage 1 and the beginning of stage 3. Stage 2 did not originally exist in our design, but it was found that the experimental results could be improved by incorporating stage 2. Therefore, stage 2 could not be abandoned. provement. However, from another perspective, the improvement strategy is independent of the velocity and position update Formulae (1) and (2), and this study sought to improve the algorithm in terms of exploration and exploitation and to control the shape of the population by relatively direct and simple means. From the analysis of the experimental results in Section 6, it can be seen that the algorithm's performance has a potential point of balance between exploration and exploitation, and thus a higher probability of obtaining the optimal solution.

Particle Action Definition for Stage 1
The position and velocity of all particles are updated according to velocity and position update Formulae (1) and (2), the position of the lagging particles will change according to the fitness value of the particles, and the direction of movement is determined by the Ra value. The Ra value is an inherent property of each particle: its positive or negative value determines repulsion or attraction between particles, i.e., particles attract those with the opposite Ra value, but repel those with the same. Each particle has a Ra value for each dimension.
As a result of the randomness of Ra, a lag particle action also has randomness. When the force value pow is large enough, the new interaction force between particles can dominate the movement of the lag particle swarm. The force value is calculated as where str is a constant parameter that determines pow, which changes the speed of the particle; the "searchrange" is limited by . The particle itself has a speed, which means that these particles may be pushed away from the gbest point, may be close to the gbest point, or may oscillate in its vicinity. However, since the size of the lag particle population is fixed, only a certain number of particles will act to make up for the difference. If a lag particle improves its fitness beyond that of other particles after iteration, then other particles will replace it as the lag particle population. Put differently, this mechanism ensures that part of the particle swarm is exploited locally, and the rest is explored globally; this study sought a balance between them.
To limit premature convergence, which is the tendency to trap in local optima, the Euclidean distance between particles is incorporated in stage 1. Once the distance between any two particles reaches a threshold, the particle with the lag fitness value will be selected and a force applied to it in the opposite direction with respect to the gbest point, pushing it away from the region in which it was originally located. This strategy coexists with the strategy repulsion and attraction mentioned above, and they affect each other.

Particle Action Definition for Stage 2
As shown in Figure 2, stage 2 is mainly used to judge the population traits and control the transition of the stage; in this study, no additional actions were performed on the particles. By mapping the diversity and convergence of the improvement algorithm, in this study, the parameters were set to control the end of stage 1 and the beginning of stage 3. Stage 2 did not originally exist in our design, but it was found that the experimental results could be improved by incorporating stage 2. Therefore, stage 2 could not be abandoned.

Particle Action Definition for Stage 3
In stage 3, we redefined the additional behavior of the particles, maintaining the Ra value of the current particle closest to the gbest point, and making the Ra values of all other Entropy 2021, 23, 1200 6 of 20 particles the opposite, i.e., all other particles will be attracted by the particle near the gbest point, the power force decreases with the number of iterations, and it will decay to 0 at the end of the iteration.
where "stage" is the value of the parameter that divides the stage and EFS represents fitness evaluations.
It should be noted that the optimal particle may change, so the pow of each particle will be recalculated after each iteration, and there is no Ra influence between other particles except the optimal particle.
Unlike the standard PSO, in which particles are influenced by the information of the pbest and gbest points, in stage 3, particles are influenced by the information of the current optimal particle position, which does not overlap with the gbest point position. The subtle differences between the two can be seen in Figure 3. The addition of a new guidance factor can help improve the convergence performance of PSO, and such a definition reduces the problem of poor search accuracy caused by the traction of the pbest point in the late stage of the search. Moreover, the current optimal particle position and the gbest point jointly determine the search direction to improve convergence performance. In stage 3, we redefined the additional behavior of the particles, maintaining the Ra value of the current particle closest to the gbest point, and making the Ra values of all other particles the opposite, i.e., all other particles will be attracted by the particle near the gbest point, the power force decreases with the number of iterations, and it will decay to 0 at the end of the iteration.
where "stage" is the value of the parameter that divides the stage and EFS represents fitness evaluations.
It should be noted that the optimal particle may change, so the pow of each particle will be recalculated after each iteration, and there is no Ra influence between other particles except the optimal particle.
Unlike the standard PSO, in which particles are influenced by the information of the pbest and gbest points, in stage 3, particles are influenced by the information of the current optimal particle position, which does not overlap with the gbest point position. The subtle differences between the two can be seen in Figure 3. The addition of a new guidance factor can help improve the convergence performance of PSO, and such a definition reduces the problem of poor search accuracy caused by the traction of the pbest point in the late stage of the search. Moreover, the current optimal particle position and the gbest point jointly determine the search direction to improve convergence performance.

Particle Property Change History
Ra is randomly attached to each particle at the beginning of phase 1 and will change in the following situations: (1) A particle distance determination is triggered in stage 1. The Ra of the less adapted of two particles will be reassigned with the same value as the other particle; (2) At the beginning of stage 3, a particle that is not the best adapted will have an Ra value opposite to that of the best particle; (3) When the optimal particle changes in stage 3, repeat 2.

Inter-Particle Action
The interactions between particles are summarized as follows, where repulsion and attraction are reflected by the change in the direction of velocity and magnitude, which directly acts on velocity change: (1) In all phases, on the basis of the original algorithm velocity and position update formula, particles that are equivalent to each other will be subject to attraction from the gbest point; (2) In stage 1, the lag particle will be attracted by the pow in Formula (7) from the better particle population; (3) In stage 1, the particle with the most lag of two close particles will be subject to repulsive force; (4) In stage 3, the non-optimal particle will be attracted by the optimal particle.

Particle Property Change History
Ra is randomly attached to each particle at the beginning of phase 1 and will change in the following situations: (1) A particle distance determination is triggered in stage 1. The Ra of the less adapted of two particles will be reassigned with the same value as the other particle; (2) At the beginning of stage 3, a particle that is not the best adapted will have an Ra value opposite to that of the best particle; (3) When the optimal particle changes in stage 3, repeat 2.

Inter-Particle Action
The interactions between particles are summarized as follows, where repulsion and attraction are reflected by the change in the direction of velocity and magnitude, which directly acts on velocity change: (1) In all phases, on the basis of the original algorithm velocity and position update formula, particles that are equivalent to each other will be subject to attraction from the gbest point; (2) In stage 1, the lag particle will be attracted by the pow in Formula (7) from the better particle population; (3) In stage 1, the particle with the most lag of two close particles will be subject to repulsive force; (4) In stage 3, the non-optimal particle will be attracted by the optimal particle.

Framework of RaPSO Algorithm
Input: Tmax is the maximum number of iterations, and Dim is the number of dimensions, Ra: denotes additional properties, and pow denotes force parameters. Output: GbestValue is the global optimal solution, and a and b are constants for control phase division. Figure 4 is the framework of the RaPSO algorithm (Algorithm 1).
7 update the particle velocity v and position x again. 8 update pbest, gbest. 9 Distance detection, adjusting Ra. 10 else If t > b/Tmax do 11 adjust the Ra of the non-optimal particle opposite to the optimal particle. 12 Calculate the additional offset of the particle position based on pow and Ra. 13 update the particle velocity v and position x again. 14 end 15 end Figure 4 is the framework of the RaPSO algorithm.   1 Initialize a population, initialize particle velocity, position, fitness; initialize PBest, GBest. 2 Randomly assign particles Ra 3 for t = 1 to Tmax do 4 Update the velocity v of the particles according to the particle population update formula; 5 If t < a/Tmax do 6 calculate the additional offset of the particle position based on pow and Ra 7 update the particle velocity v and position x again. 8 update pbest, gbest. 9 Distance detection, adjusting Ra. 10 else If t > b/Tmax do 11 adjust the Ra of the non-optimal particle opposite to the optimal particle. 12

Entropy and Convergence
Calculate the additional offset of the particle position based on pow and Ra. 13 update the particle velocity v and position x again. 14 end 15 end

Entropy and Convergence
The paper discusses the effectiveness of the strategy in terms of entropy and convergence. An increase in population confusion often means a decrease in convergence performance, but if it increases the likelihood that the population searches for the optimal value without affecting convergence, i.e., it produces small fluctuations in the convergence curve, it will help improve the algorithm. We chose population entropy and DBscan as reference tools to draw figures to establish that the multi-stage search strategy worked.

Population Entropy
Entropy is a measure of the degree of chaos in a system and population entropy is an indicator of group diversity. The general definition of entropy may not apply in high-dimensional space; therefore, we chose population entropy as the evaluation criterion of algorithm convergence. In this paper, we assume that the number of particles is M, and the best individual position of the particles is f_pb. Calculating population entropy can be divided into three steps: Step 1: Calculate the minimum fitness of the particle's historical best position, f min = min(f_pb), and maximum fitness, f max = max(f_pb). The interval of consideration is [f min , f max ]; Step The larger the computational results of population entropy, the more chaotic the particle distribution in space and the less convergent the algorithm.
We compared PSO and the improved algorithm RaPSO using CEC2017-fun10 as the objective function and by conducting experiments in 10-dimensional space. The fitness value of Figure 5a is higher than that of Figure 5b. As can be seen from Figure 5a, the population entropy value of PSO basically has a continuous decreasing trend, and PSO has nearly no effective exploration in the middle and at the end of the algorithm iteration, after which the fitness value no longer changes. This also demonstrates that PSO is easily trapped in local optima. Figure 5b shows the historical change in the population entropy after adding the stage search strategy, from which we can see that the algorithm is in a more chaotic state from beginning to end, and the population entropy is obviously larger than that of PSO at each moment. However, the experimental results are better than those depicted in Figure 5a, which shows that, under the premise of algorithm convergence, increasing population entropy is beneficial to particles in terms of searching for the optimal solution, i.e., increasing population entropy at the beginning of an iteration can help in global exploration, while increasing it at the end of the iteration can benefit local exploitation.
Entropy 2021, 23, x FOR PEER REVIEW 8 The paper discusses the effectiveness of the strategy in terms of entropy and co gence. An increase in population confusion often means a decrease in convergenc formance, but if it increases the likelihood that the population searches for the op value without affecting convergence, i.e., it produces small fluctuations in the co gence curve, it will help improve the algorithm. We chose population entropy and D as reference tools to draw figures to establish that the multi-stage search strategy wo

Population Entropy
Entropy is a measure of the degree of chaos in a system and population entropy indicator of group diversity. The general definition of entropy may not apply in dimensional space; therefore, we chose population entropy as the evaluation criter algorithm convergence. In this paper, we assume that the number of particles is M the best individual position of the particles is f_pb. Calculating population entropy c divided into three steps: Step 1: Calculate the minimum fitness of the particle's historical best position, f min(f_pb) , and maximum fitness, f max = max(f_pb). The interval of considerati [f min , f max ]; Step Step 3: Calculate the population entropy, The larger the computational results of population entropy, the more chaotic th ticle distribution in space and the less convergent the algorithm.
We compared PSO and the improved algorithm RaPSO using CEC2017-fun10 objective function and by conducting experiments in 10-dimensional space. The f value of Figure 5a is higher than that of Figure 5b. As can be seen from Figure 5 population entropy value of PSO basically has a continuous decreasing trend, and has nearly no effective exploration in the middle and at the end of the algorithm iter after which the fitness value no longer changes. This also demonstrates that PSO is trapped in local optima. Figure 5b shows the historical change in the population en after adding the stage search strategy, from which we can see that the algorithm i more chaotic state from beginning to end, and the population entropy is obviously than that of PSO at each moment. However, the experimental results are better than depicted in Figure 5a, which shows that, under the premise of algorithm converg increasing population entropy is beneficial to particles in terms of searching for the mal solution, i.e., increasing population entropy at the beginning of an iteration can in global exploration, while increasing it at the end of the iteration can benefit loc ploitation.

DBscan
DBscan can measure the degree of population disorder from the perspective of density. It defines clusters as the maximum set of densely connected points, which can divide regions with sufficient density into clusters and find clusters of arbitrary shapes in the spatial database of noise. The DBscan calculation method is described as follows: Step 1: Define the field radius e and the threshold of core point; Step 2: Start from one particle and find all particles that have a density connection. Jump out of this cluster and start to find particles again from the other particle; Step 3: Repeat step 2 until all particles have been traversed. We chose the conditions for the population entropy experiment and plotted Figure 6. From the figure, we can see that DBscan can depict the clustering phenomenon of the population more graphically than population entropy. The PSO algorithm enters a high degree of clustering after 400 iterations, where the particle population cannot jump out of the local optimum due to insufficient guidance information. Similarly, it can be seen that the overall convergence performance of the population remains unchanged after adding the strategy, while the degree of confusion is improved.
Entropy 2021, 23, x FOR PEER REVIEW 9 DBscan can measure the degree of population disorder from the perspective of sity. It defines clusters as the maximum set of densely connected points, which can d regions with sufficient density into clusters and find clusters of arbitrary shapes i spatial database of noise. The DBscan calculation method is described as follows: Step 1: Define the field radius e and the threshold of core point; Step 2: Start from one particle and find all particles that have a density conne Jump out of this cluster and start to find particles again from the other particle; Step 3: Repeat step 2 until all particles have been traversed. We chose the conditions for the population entropy experiment and plotted F 6. From the figure, we can see that DBscan can depict the clustering phenomenon population more graphically than population entropy. The PSO algorithm enters a degree of clustering after 400 iterations, where the particle population cannot jump o the local optimum due to insufficient guidance information. Similarly, it can be seen the overall convergence performance of the population remains unchanged after ad the strategy, while the degree of confusion is improved.

Experimental Design Method
We chose 29 test functions in CEC2017 as objective functions for optimiz Among them, Fun1 is single-peaked, Fun3-Fun10 are multi-peaked, Fun11-Fun2 mixed, and Fun21-Fun30 are composite. The details are shown in the Table 1.

Experimental Design Method
We chose 29 test functions in CEC2017 as objective functions for optimization. Among them, Fun1 is single-peaked, Fun3-Fun10 are multi-peaked, Fun11-Fun20 are mixed, and Fun21-Fun30 are composite. The details are shown in the Table 1.

Experimental Parameter Selection
The 29 test functions of CEC2017 were selected as target functions for optimization tests, and experiments were conducted with a population size N = 100, repeat = 30, and D = 10, 30. Other parameter settings are shown in Table 2. In order to achieve a better comparison, the parameters were all selected based on previous experiments [24].  After adding the improvement mechanism, there were three additional controllable parameters in each algorithm: str, prop, and stage. In order to ensure the accuracy of parameter selection, the number of experimental repetitions was increased to 100.

The Parameter Str
The parameter str affects the magnitude of the strength of all repulsions and attractions in stages 1 and 3. The algorithm used in the experiment was the standard PSO. With the other parameters unchanged, Fun5, Fun14, and Fun23 in CEC2017 were selected as objects for the comparison experiments. The x-axis in Figure 7 represents the value of parameter str, and the y-axis represents the average fitness after normalization. The value range of the x-axis was obtained after a large number of experiments. The experimental range was far from that. It was the range obtained after preliminary screening. The sub-graph in the Figure 7 is a detailed display of the x-axis from 20 to 30. The results from Figure 7 show that str = 25 was chosen to obtain better results.

The Parameter Prop
The parameter prop determines the proportion of poorly adapted particles in stage 1. Because only Fun5 in CEC2017 was selected as the test function, the fitness value of the y-axis was unprocessed. Other parts are similar to those used in the experiment in Section 5.2.1 Based on the experimental results, a value of 0.5 was chosen in Figure 8, meaning that half of the particles were attracted to or repelled by the other half.

The Parameter Prop
The parameter prop determines the proportion of poorly adapted particles in stage 1. Because only Fun5 in CEC2017 was selected as the test function, the fitness value of the y-axis was unprocessed. Other parts are similar to those used in the experiment in 5.2.1. Based on the experimental results, a value of 0.5 was chosen in Figure 8, meaning that half of the particles were attracted to or repelled by the other half.

The Parameter Stage
The parameter stage is the basis for deciding the division of the three stages. Similarly, we used standard PSO as the experimental algorithm in 10 dimensions. The test function was Fun25 in CEC2017. Based on the experimental results in Table 3, this was set to 0.1, which means that the first stage accounts for 40% of the iterations, the second stage 20%, and the third stage 40%.

The Parameter Prop
The parameter prop determines the proportion of poorly adapted particles i 1. Because only Fun5 in CEC2017 was selected as the test function, the fitness valu y-axis was unprocessed. Other parts are similar to those used in the experiment i Based on the experimental results, a value of 0.5 was chosen in Figure 8, meaning t of the particles were attracted to or repelled by the other half.

The Parameter Stage
The parameter stage is the basis for deciding the division of the three stage larly, we used standard PSO as the experimental algorithm in 10 dimensions. T function was Fun25 in CEC2017. Based on the experimental results in Table 3, this to 0.1, which means that the first stage accounts for 40% of the iterations, the secon 20%, and the third stage 40%.

TSLPSO Based on Multi-Stage Search Strategy
Tables 6 and 7 is the experiment data of results of TSLPSO and RaTSLPSO in the case of 10/30 dimensions according to the parameters mentioned in Section 5.2. The data in bold in the tables is preferred. 5.03 × 10 2 5.14 × 10 2 5.08 × 10 2 2.56 × 10 0 5.03 × 10 2 5.10 × 10 2 5.07 × 10 2 1.74 × 10 0 F6 6.00 × 10 2 6.00 × 10 2 6.00 × 10 2 2.72 × 10 −5 6.00 × 10 2 6.00 × 10 2 6.00 × 10 2 3.44 × 10 −9 F7 7.12 × 10 2 7.23 × 10 2 7.19 × 10 2 2.82 × 10 0 7.08 × 10 2 7.23 × 10 2 7.19 × 10 2 3.43 × 10 0 F8 8.04 × 10 2 8.12 × 10 2 8.08 × 10 2 2.25 × 10 0 8.03 × 10 2 8.10 × 10 2 8.07 × 10 2 1.79 × 10 0 F9 9.00 × 10 2 9.02 × 10 2 9.00 × 10 2 4.43 × 10 −1 9.00 × 10 2 9.00 × 10 2 9.00 × 10 2 2.73 × 10   Tables 8 and 9 is the experiment data of results of HCLPSO and RaHCLPSO in the case of 10/30 dimensions according to the parameters mentioned in Section 5.2. The data in bold in the tables is preferred.    Figure 9 shows various test functions of more significant optimization effects from the six tables in Section 5.3. The x-axis is the fitness evaluation number (Fes, 10 dimensions equal 10 5 ; 30 dimensions equal 3 × 10 5 ). The y-axis is the average fitness value. Different colors represent different original algorithms. Solid lines represent algorithms with the strategy and dotted lines represent those without. According to the figure, algorithms adopting the strategy achieve lower (better) fitness and better search performance. Since the strategy increases population disorder during early iterations, the phenomena of slower convergence in early iterations and a poorer fitness value could be observed in certain circumstances in the figure; however, these phenomena are often improved at the end of search. Full development in the preliminary stage lays a more solid foundation for later exploration, and may in turn yield better fitness values. According to Figure 9, the red dotted curve of PSO is always the first stabilized, which means that PSO is more likely to be trapped in local optima. Both TSLPSO and HCLPSO can improve this. However, the assistance of the new strategy can produce better results. According to the figure, the strategy has a more significant optimizing effect on PSO and its two improved algorithms. The same conclusion is drawn in 10 dimensions and 30 dimensions, which further verifies the effectiveness of the strategy as regards the scope of the application.

Conclusions
A relatively direct and simple method is proposed herein to address problems such as premature convergence and poor searching precision with PSO, i.e., improving particle swarm diversity in early iterations of the algorithm and introducing guidance for the best particle location in the population at the end of the iteration to enhance searching precision. The concept was based on PSO and its two improved algorithms and acts as a multistage search strategy operator. It guides the particle swarm to different goals in different stages, improves the comprehensive exploration and development ability, and upgrades population disorder without changing the convergence performance of the algorithm too much, thus effectively improving the particle's ability to jump out of local optima. Particle swarm behavior before and after the application of the strategy was compared and described, and the effectiveness of the strategy was verified with the population entropy and DBscan tools. The contrast experiment indicated the selection process for three parameters in the proposed strategy. Moreover, according to experimental results of the algorithms on the test functions, the strategy can improve search performance.
In future work, we hope to mathematically demonstrate that the selected parameter values are optimal, prove the general applicability of the strategy, and demonstrate the adequacy and necessity of the divided stages. The effectiveness of the proposed strategy was experimentally demonstrated, but a mathematical analysis and proof are required.

Conclusions
A relatively direct and simple method is proposed herein to address problems such as premature convergence and poor searching precision with PSO, i.e., improving particle swarm diversity in early iterations of the algorithm and introducing guidance for the best particle location in the population at the end of the iteration to enhance searching precision. The concept was based on PSO and its two improved algorithms and acts as a multi-stage search strategy operator. It guides the particle swarm to different goals in different stages, improves the comprehensive exploration and development ability, and upgrades population disorder without changing the convergence performance of the algorithm too much, thus effectively improving the particle's ability to jump out of local optima. Particle swarm behavior before and after the application of the strategy was compared and described, and the effectiveness of the strategy was verified with the population entropy and DBscan tools. The contrast experiment indicated the selection process for three parameters in the proposed strategy. Moreover, according to experimental results of the algorithms on the test functions, the strategy can improve search performance.
In future work, we hope to mathematically demonstrate that the selected parameter values are optimal, prove the general applicability of the strategy, and demonstrate the