Adaptive-Uniform-Experimental-Design-Based Fractional-Order Particle Swarm Optimizer with Non-Linear Time-Varying Evolution

: An adaptive-uniform-experimental-design-based fractional particle swarm optimizer (AUFPSO) with non-linear time-varying evolution (NTE) is proposed. A particle swarm optimizer (PSO) is an excellent evolutionary algorithm due to its simple structure and rapid convergence. Nevertheless, PSO has notable drawbacks. Although many proposed methods and strategies have enhanced its e ﬀ ectiveness and performance, PSO is limited by its tendency to fall into local optima and its tendency to obtain di ﬀ erent solutions in each search (i.e., its weak robustness). Introducing fractional-order calculus in PSO (FPSO) can correct the order of the velocity derivative for each particle, which enhances the diversity and algorithmic e ﬀ ectiveness. This study used NTE of the order of the velocity derivative, inertia weight, cognitive parameter, and social parameter in an FPSO used to search for a global optimal solution. To obtain the best combination of FPSO and NTE, an adaptive uniform experimental design (AUED) method was used to deal with this essential issue. The AUED method integrates a uniform layout with the best combination phase and a stepwise ratio to assist in selecting the best combination for FPSO-NTE. Experimental applications in 15 global numerical optimization problems conﬁrmed that the AUFPSO-NTE had a better performance and robustness than existing PSO-related algorithms.


Introduction
Particle swarm optimization (PSO), which was first proposed in 1995 [1], is a swarm intelligence computational technique inspired by animal behavior, such as birds' flocking. Because of its many advantages, including fast convergence, simple structure, and high accuracy, PSO is used to solve optimization problems, e.g., the travelling salesman problem [2], and problems in many industrial and engineering domains [3], e.g., image processing [4], clouding computing [5], power systems [6,7], the algorithm proposed in this study applied the underlying concept of the DAUED method by using DAUED to select the best combination of four constant coefficients. Because a data-driven approach was not used, DAUED was renamed to an adaptive uniform experimental design (AUED) method for this study. The AUFPSO-NTE algorithm automatically obtained the best combination of four constant coefficients because AUED was integrated in FPSO-NTE.
This paper is organized as follows. FPSO and the proposed FPSO-NTE are briefly described in Section 2. Section 3 describes how AUED was used in FPSO-NTE to search for the best combination. Section 4 presents and discusses the experimental and simulation results. Finally, Section 5 concludes the study.

FPSO and FPSO-NTE
Solteiro Pires and coworkers [34,35] first proposed FPSO, which introduced a fractional-order derivative in PSO for rearranging and modifying the order of the velocity derivative. In a fractional-order derivative, common definitions include the Riemann-Liouville definition, the Caputo definition, and the Grünwald-Letnikov definition [55][56][57]. In the FPSO proposed by Solteiro Pires and coworkers [34,35], the order of the velocity derivative is derived from the Grünwald-Letnikov definition, as shown below: where D is the derivative operator, λ is the fractional order of the derivative, h is the difference operator, Γ is the Gamma function, and x(t) is the object for the derivative. For an application in discrete time, Expression (1) can be approximated using: where T and r are the sampling period and the truncation order, respectively. The update expression of the velocity for PSO is: V i (t + 1) = V i (t) + c 1 ·r 1 P l i (t) − p i (t) + c 2 ·r 2 (P g (t) − p i (t)).
The update expression of the position for PSO is: where i = 1, 2, . . . , S; S is the number of particles; t is the current iteration; λ is the fractional order of the derivative; V i (t) is the velocity; c 1 and c 2 are the cognitive and social coefficients, respectively; p i (t) is the current position; P i l (t) and P g (t) are the position of the self-best solution in the current iteration and the position of the global solution in the current iteration, respectively; and r 1 and r 2 are random constants between 0 and 1. In Expression (3), the position term p i of (P i l −p i ) is rewritten as follows using fractional-order derivation: According to Solteiro Pires et al. [34] and Gao et al. [36], r is set to 4 to obtain the best balance of convergence rate and accuracy. Additionally, tests show that an r larger than 4 obtains the same results. Therefore, the updated expression of the velocity of the FPSO is: To improve the PSO performance and effectiveness, Ko et al. [16] and Cui and Zeng [58] introduced a non-linear NTE in PSO. Gao et al. [36] introduced a non-linear time-varying inertia weight in FO-DPSO. In the current study, NTE is integrated in FPSO to enhance the performance and effectiveness. The resulting FPSO-NTE is expressed as follows: where where t and t max are the current iteration and the maximum of iteration, respectively. The constant coefficient λ influences the algorithmic performance, and coefficients α, β, and γ influence ω, c 1 , and c 2 , respectively. The problem is how to obtain the best parameter value for the constant coefficients λ, α, β, and γ. The solution proposed in this study is to use AUED to obtain the best combination of constant coefficient values.

AUED-Based FPSO (AUFPSO) with NTE
This study used the AUED method in the proposed FPSO-NTE algorithm to assist in the search for the best combination of the four constant coefficients. The three main steps of AUED method are initializing, performing ten-level uniform layout experiments, and calculating the parameter range for the next ten-level uniform layout. The initialization step includes selection of the parameters to be optimized, their ranges, and the solution accuracy. At this time, a suitable ten-level uniform layout, the output, a stepwise ratio, and the stop condition are also selected. The second main step can then be executed. Ten levels of the parameter range and ten levels of the solution accuracy are defined, and the ten levels are assigned to the ten-level uniform layout. The ten-level uniform layout experiments can then be performed, and the experimental results are recorded. After the ten-level uniform layout experiments in this stage are completed, the best combination obtained according to the output is an optimal or near-optimal value. The range for each parameter is then calculated according to the best combination and the stepwise ratio. The second and third main steps are repeated until the stop condition is met.
For a clear understanding of how the AUED method is applied in the proposed FPSO-NTE algorithm, the detailed steps of the method are given below.

A.
Initialization of the AUED method in the proposed FPSO-NTE algorithm Step 1. Define the experimental parameters as the four constant coefficients λ, α, β, and γ. For each parameter, set the range from 0 to 2, and set the solution accuracy to 0.0001. Step 2. Set the experimental output as the fitness value.
Step 3. Set the stepwise ratio to 0.8.
Step 4. Select a suitable ten-level uniform layout of U 10 (10 4 ), as shown in Table 1.
Step 5. Repeat steps 1-4 until the objective value is reached or until the fitness value does not obtain a near-objective value in two consecutive ten-level uniform layout experiments. B.
Perform the ten-level uniform layout experiments Step 1. The ranges for each parameter are divided into ten discrete values according to the chosen ten-level uniform layout of U 10 (10 4 ).
Step 2. Assign ten discrete values of each parameter into the chosen ten-level uniform layout of U 10 (10 4 ), shown as Table 2.
Step 3. Perform this process 15 times for each ten-level uniform layout experiment and record the average as the output. Table 2.

C.
Update the search range for next ten-level uniform experiments Step 1. For each parameter, calculate the search range according to the best combination in this stage and the stepwise ratio (0.8). The updated Algorithm 1 is shown below.
Step 2. Return to main step B and execute the experimental steps until the stop condition is met. where PARA_NO is the total number of parameters; LB and UB are the upper and lower bounds for each experimental parameter, respectively; LT and UT are temporary values of LB and UB, respectively; LEVEL is the level value; EXP_NO is the total number of experiments in the uniform layout; BEST is the best parameter value for the best combination obtained by the uniform layout experiments in this stage; and SWR is a stepwise ratio.
The following example demonstrates the use of the updated algorithm when the upper and lower bounds for each parameter are initially set to 2 and 0, respectively, the number of parameters was 4, and the number of experiments was 10. The first ten-level uniform layout experiments indicated that the best combination [P 1 P 2 P 3 P 4 ] was [0.2553 0.4514 0.5556 1.2455]. The stepwise ratio was set to 0.8. Here, the first parameter value was used to explain how to calculate a new range for the next ten-level uniform layout experiments stage. When the first ten-level uniform layout experiments stage was completed, LB and UB were 0 and 2, respectively. At first, LT and UT were 0 and 2, respectively, due to LB and UB. The best parameter was 0.2553, and the stepwise ratio was 0.8. Therefore, LB = 0.2553 − (2 − 0) × 0.8 ÷ 2 = − 0.5447 and UB = 0.2553 − (2 − 0) × 0.8 ÷ 2 = 1.0553. However, since LB was lower than LT, LB must be corrected to LT, and the new range for the first parameter was 0 to 1.0553. This was because the original range was set to 0 to 2. Therefore, we could know that LB for the next uniform layout experiments must be equal to or more than LT. In the same way, UB for the next uniform layout experiments must have been equal to or less than UT. Next, the range was divided into ten levels, and a discrete value was calculated for each level. The discrete values were calculated as follows: 0 1173 for the second level, and so on. The discrete value for the tenth level was calculated as 0 + ((1.0553 − 0) ÷ (10 − 1) × (10 − 1)) = 1.0533. Table 3 shows the levels obtained after the updated algorithm was executed in this instance.

Simulation Results and Comparisons
This section presents the results for the 15 global numerical optimization problems in Table 4, which were used for the performance evaluations of the proposed AUFPSO-NTE algorithm. In Example (1), functions f 1 -f 7 were used to compare the performance of the proposed AUFPSO-NTE, the PSO-FOV (PSO with the fractional-order velocity) proposed by Solteiro Pires et al. [34], the FPSO improved by Solteiro Pires et al. [35] from PSO-FOV, the modified PSO (MPSO) proposed by Shi and Eberhart [11], and the PSO proposed by Kennedy and Eberhart [1]. In Example (2), functions f 5 and f 8 -f 11 were used to compare the performance of the proposed AUFPSO-NTE, the FPSO-based algorithms proposed by Gao et al. [36], and the PSO proposed by Gao et al. [36]. In Example (3), functions f 5 , f 8 , f 10 , f 11 , and f 12 were used to compare between the AUFPSO-NTE, the adaptive fractional-order Darwinian PSO (AFO-DPSO) proposed by Guo et al. [38], and the PSO-based algorithms described in Guo et al. [38]. In Example (4), functions f 5 , f 8 , f 10 , f 11 , and f 13 -f 15 were used to compare between the AUFPSO-NTE, the fractional-order Darwinian PSO (FDPSO) proposed by Hosseini et al. [39] and the PSO-based algorithms described in Hosseini et al. [39]. The simulations were run on a Windows 10 personal computer with a core i7-6700M, a 3.4 GHz CPU, and 8 GB RAM.

Name Definition Solution Space Optimal Value
Quartic The FPSO was the first developed, called PSO-FOV (PSO with the fractional-order velocity), and improved by Solteiro Pires and coworkers [34,35], the MPSO introduced a time-varying inertia weight [11], and PSO was first proposed by Kennedy and Eberhart [1]. All three algorithms were compared with the proposed AUFPSO-NTE. Table 5 shows the number of dimensions (D n ) that functions f 1 to f 7 were set to with D n = 2, 4, 2, 2, 30, 2, and 4, respectively; the number of particles (S) was set to S = 10; and the number of iterations (I) was set to 200. Table 6 shows the parameter settings for the proposed AUFPSO-NTE and for the FPSO, PSO-FOV, MPSO, and PSO. In the AUFPSO-NTE, the minimum weight (ω min ) and maximum weight (ω max ) were set to 0.4 and 0.9, respectively. The minimum cognitive coefficient (c 1min ) and maximum cognitive coefficient (c 1max ) were set to 0 and 2, respectively. The minimum social coefficient (c 2min ) and maximum social coefficient (c 2max ) were set to 0 and 2, respectively. In the MPSO, ω min and ω max were set to 0.4 and 0.9, respectively, and c 1 and c 2 were both set to 2. For FPSO, PSO-FOV and PSO, c 1 and c 2 were both set to 2. The maximum velocity (V max ) was defined as the maximum position minus minimum position, and the minimum velocity (V min ) was defined as negative V max . Table 7 shows the best combinations of the constant coefficients λ, α, β, and γ for each benchmark function of the proposed AUFPSO-NTE. For comparisons, the constant coefficient λ and results are obtained by the algorithms developed by Solteiro Pires and coworkers [34,35]. Tables 8 and 9 show the constant coefficient λ for each benchmark function of the FPSO [35] and PSO-FOV [34], respectively. In Tables 7-9, λ is the fractional order of the derivative. In Table 7, α, β, and γ are coefficients that influence ω, c 1 , and c 2 , respectively.   Table 10 shows the performance comparison results for the proposed AUFPSO-NTE and for FPSO [35], PSO-FOV [34], MPSO [11], and PSO [1]. The table shows the best solution, mean, and standard deviation (S.D.) that each algorithm obtained for f 1 -f 7 in 30 independent trials. In Example (1), all algorithms except PSO obtained the best solutions for f 1 , f 3 , f 4 , and f 6 . Only the proposed AUFPSO-NTE and FPSO obtained the best solution for f 5 [35]. Additionally, only the proposed AUFPSO-NTE obtained the best solution for f 2 and f 7 . In terms of the mean and S.D., the proposed AUFPSO-NTE outperformed FPSO [35], PSO-FOV [34], MPSO [11], and PSO [1].

Example (2): Proposed AUFPSO-NTE in Comparison with FVFP-PSO, FP-PSO, FV-PSO, and PSO
Example (2) compared the performance of the proposed AUFPSO-NTE with the standard PSO and with three modifications of PSO proposed by Gao et al. [36]: particle swarm optimization with the fractional-order velocity and the fractional-order position (FVFP-PSO), particle swarm optimization with the fractional-order position (FP-PSO), and particle swarm optimization with the fractional-order velocity (FV-PSO). Table 11 shows that the number of dimensions for functions f 5 and f 8 -f 11 was set to D n = 10, the number of particles was set to S = 30, and the number of iterations was set to N = 300. Table 12 shows the parameter settings for the proposed AUFPSO-NTE, FVFP-PSO, FP-PSO, FV-PSO, and PSO. Parameters ω min , ω max , c 1min , c 1max , c 2min , and c 2max of the proposed AUFPSO-NTE were set as in Example (1). In FVFP-PSO, FP-PSO, and FV-PSO, the fractional-order velocities and positions were affected by factors ε and ζ. Notably, factors ω, ε, and ζ underwent time-varying evolution in FVFP-PSO, FP-PSO, and FV-PSO. Table 12 also shows that c 1 and c 2 were 1. In Table 13, λ is the fractional order of the derivative and α, β, and γ are coefficients that influence ω, c 1 , and c 2 , respectively. Table 13 shows the best combinations of constant coefficients λ, α, β, and γ for each benchmark function of the proposed AUFPSO-NTE.     Table 14 compares the mean values obtained using the proposed AUFPSO-NTE, FVFP-PSO, FP-PSO, FV-PSO, and PSO developed by Gao et al. [36] for f 5 and f 8 -f 11 in 100 independent trials. Table 14 shows that the means obtained by the proposed AUFPSO-NTE were better than those obtained using FVFP-PSO, FP-PSO, FV-PSO, and PSO.   [38]. The AFO-DPSO introduces fractional-order velocity into a Darwinian PSO algorithm and includes a mutation mechanism to overcome premature convergence, NCPSO improves the chaos-PSO algorithm, FO-DPSO is a fractional-order Darwinian PSO, APSO provides adaptive PSO to enable automatic control of parameters, DPSO is Darwinian particle swarm optimization, and HPSO combines the concept of evolutionary computation with PSO.
In Table 15, the D n for functions f 5 , f 8 , f 9 , f 11 , and f 12 were set to 30, the number of particles was set to S = 30, and the number of iterations was set to N = 1000. Tables 16 and 17 show the parameter settings  for the proposed AUFPSO-NTE and for the AFO-DPSO, NCPSO, FO-DPSO, FPSO, APSO, DPSO, HPSO, and PSO. Parameters ω min , ω max , c 1min , c 1max , c 2min , and c 2max of the proposed AUFPSO-NTE were set as in Example (1). For AFO-DPSO, ω was 1; c 1 and c 2 were 1.5 to 2.5, respectively; and δ was 0.05 to 0.1. For NCPSO, ω was 0.7298, and c 1 and c 2 were both 1.4962. For FO-DPSO and FPSO, ω was 0.9, λ was 0.632, and c 1 and c 2 were both 1.5. The APSO parameters were automatically set. In DPSO, ω was set to 0.9. In HPSO, ω min and ω max were set to 0.2 and 0.8, respectively, and c 1 and c 2 were both set to 2.5. In PSO, ω min and ω max were set to 0.4 and 0.9, respectively, and c 1 and c 2 were both set to 2. Table 18 illustrates the best combinations of constant coefficients λ, α, β, and γ for each benchmark function of the proposed AUFPSO-NTE. In Table 18, λ is the fractional order of the derivative and α, β, and γ are coefficients which influence ω, c 1 , and c 2 , respectively.

Function
Number of Dimension (D n )  In Tables 19 and 20, the mean values obtained by the AUFPSO-NTE are compared with those obtained using AFO-DPSO, NCPSO, FO-DPSO, FPSO, APSO, DPSO, HPSO, and PSO given by Guo et al. [38]. Mean values obtained using each PSO-based algorithm for f 5 , f 8 , f 9 , f 11 , and f 12 in 30 independent trials were recorded. Tables 19 and 20 show that the means obtained using the proposed AUFPSO-NTE were better than those obtained using AFO-DPSO, NCPSO, FO-DPSO, FPSO, APSO, DPSO, HPSO, and PSO. In Guo et al. [38], the performance was evaluated in terms of variance in the optimum (12):

Number of Iterations (I)
where f i is the ith fitness value and f avg is the mean fitness value for 30 independent trials. The f max is a normalization factor. When |f i − f avg | > 1, f max is the maximum (|f i − f avg |); otherwise, f max = 1.  The variance in the optimum was also used to compare the performance of the proposed AUFPSO-NTE with algorithms given by Guo et al. [38]. Tables 21 and 22 [38], GAPSO (genetic algorithm-PSO) [59], HFPSO (hybrid firefly algorithm and PSO) [60], FPSO, and PSO developed by Hosseini et al. [38]. The HAFPSO introduces the concept of hunter-attack into the FODPSO and the GAPSO is a compound optimizer that introduces the crossover and mutation strategy of GA into PSO. The HFPSO combines the firefly optimization algorithm and PSO. Table 23 shows that the number of dimensions for functions f 5 , f 8 , f 10 , f 11 , and f 13 -f 15 was set to D n = 50, the number of particles was set to S = 30, and the number of iterations was set to N = 1000. Parameters ω min , ω max , c 1min , c 1max , c 2min , and c 2max of the proposed AUFPSO-NTE were set as in Example (1). Except for the fractional order value (λ) in HAFPSO being 0.6, the parameter settings for the HAFPSO, GAPSO, HFPSO, FPSO, and PSO were not mentioned. In Table 24, λ is the fractional order of the derivative and α, β, and γ are coefficients that influence ω, c 1 , and c 2 , respectively. Table 24 shows the best combinations of the constant coefficients λ, α, β, and γ for each benchmark function of the proposed AUFPSO-NTE.   Table 25 shows the performance comparison results obtained using the proposed AUFPSO-NTE, and obtained using HAFPSO [38], GAPSO [59], HFPSO [60], FPSO, and PSO developed by Hosseini et al. [38] for f 5 , f 8 , f 10, f 11 , and f 13 -f 15 in 100 independent trials. The table shows that the means and S.D. obtained using the proposed AUFPSO-NTE were better than those obtained using HAFPSO [38], GAPSO [59], HFPSO [60], FPSO, and PSO. Overall, the performance of the proposed AUFPSO-NTE was better than others in Example (4).

Conclusions
This study applied the AUED method to enhance the performance and effectiveness of the FPSO-NTE algorithm. Use of the AUED method in the proposed FPSO-NTE algorithm enabled a rapid automatic search for the best combination of four constant coefficients, namely λ, α, β, and γ. The major contribution of this paper was the use of AUED to improve the performance of the algorithm and to obtain a robust output. The above experimental and simulation results indicate that the proposed AUFPSO-NTE algorithm achieved a higher solution accuracy compared to the FPSO and PSO-FOV proposed by Solteiro Pires and coworkers [34,35], the FPSO proposed by Gao et al. [36], the AFO-DPSO proposed by Guo et al. [38], the HAFPSO proposed by Hosseini et al. [39], and the PSO-based algorithm described by them [34][35][36]38,39]. Examples demonstrated that the solutions obtained using the proposed AUFPSO-NTE algorithm were more consistent, i.e., more robust. Therefore, we conclude that the proposed AUFPSO-NTE algorithm had a superior effectiveness and performance.

Conflicts of Interest:
The authors declare no conflict of interest.