Heterogeneous Cooperative Bare-Bones Particle Swarm Optimization with Jump for High-Dimensional Problems
Abstract
:1. Introduction
2. Background Knowledge
2.1. The Particle Swarm Optimization (PSO) Algorithm
Algorithm 1 Standard PSO Algorithm. | |
1: | |
▹: # of Particles | |
2: | |
3: | |
4: while Termination condition does not meet do | |
5: for to do | |
6: for to n do | ▹n: Dimension of Problem |
7: | |
8: | |
9: | ▹Using Equation (1) |
10: end for | |
11: | |
▹Using Equation (2) | |
12: if then | |
13: | ▹Update |
14: if then | |
15: | ▹Update |
16: end if | |
17: end if | |
18: end for | |
19:end while | |
20: return | ▹Best Found Solution |
2.2. The Bare-Bones PSO (BBPSO) Algorithm
2.3. The Cooperative Approach
2.4. The Jumping Strategy
3. Heterogeneous Cooperative BBPSO with Jumping (HCBBPSO-Jx) Algorithms
3.1. The Cooperative BBPSO (CBBPSO) Step
3.2. The BBPSO with Jumping (BBPSO-Jx) Step
Algorithm 2 HCBBPSO-Jx Algorithms (for x, C: Cauchy, G: Gaussian). | |
1: | ▹Set parameters |
▹K: split factor, : # of maximum allowable update failure, : jump scaling factor, n: dimension of given problem | |
2: | |
3: | |
▹For spliting into solution vector components for CBBPSO | |
4: | |
▹Initialize swarms P and swarm Q, : Counter vector for update failure | |
5: while Termination condition does not meet do | |
6: | |
Cooperative BBPSO Step | |
7: for to K | |
▹For s-th swarm with dimensions if and dimensions if | |
8: | ▹ if and if |
9: for to do | ▹For each particle |
10: if then | |
11: | ▹Update components (or particle) of swarms P |
12: if then | |
13: | ▹Update components of swarms P ( or components of context vector) |
14: end if | |
15: end if | |
16: end for | |
17: | ▹Moving swarm using Equations (3)–(5) for swarms P |
18: end for | |
BBPSO with Jump Step | |
19: for to do | ▹For each particle |
20: if then | ▹Perform original BBPSO |
21: | ▹Moving swarm using Equations (3)–(5) for swarm Q |
22: else | ▹Perform jump |
23: | |
24: | ▹ for Cauchy jump or for Gaussian jump |
25: end if | |
26: for to K do | ▹Information exchange from swarms P to swarm Q |
27: | ▹ if and if |
28: | ▹Integer uniform distribution with range |
29: | |
30: end for | |
31: if then | |
32: | ▹Update vector (or particle) of swarm Q |
33: if then | |
34: | ▹Update vector of swarm Q |
35: end if | |
36: else | |
37: | |
38: end if | |
39: end for | |
40: for to n do | ▹Information exchange from swarm Q to swarms P |
41: | ▹Integer uniform distribution with range |
42: | |
43: end for | |
44: | |
45:end while | |
46: return | ▹Best found solution |
3.3. The Steps Involving Cooperation by Exchanging Information between the CBBPSO and BBPSO-Jx Algorithms
4. Comparative Simulations
4.1. The Simulation Environment and Setup
4.2. The Results of Comparing the Simulations Performed by the PSO Algorithm Variants
4.3. The Results of Simulations Using the HCBBPSO-JG Algorithm for the CEC 2010 Benchmark Functions
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Lei, G.; Shao, K.R.; Guo, Y.; Zhu, J.; Lavers, J.D. Improved Sequential Optimization Method for High Dimensional Electromagnetic Device Optimization. IEEE Trans. Magn. 2009, 45, 3993–3996. [Google Scholar]
- Xue, Y.; Tang, T.; Liu, A.X. Large-Scale Feedforward Neural Network Optimization by a Self-Adaptive Strategy and Parameter Based Particle Swarm Optimization. IEEE Access 2019, 7, 52473–52483. [Google Scholar] [CrossRef]
- Qiao, W.; Yang, Z. Modified Dolphin Swarm Algorithm Based on Chaotic Maps for Solving High-Dimensional Function Optimization Problems. IEEE Access 2019, 7, 110472–110486. [Google Scholar] [CrossRef]
- Gardeux, V.; Chelouah, R.; Siarry, P.; Glover, F. EM323: A line search based algorithm for solving high-dimensional continuous non-linear optimization problems. Soft Comput. 2011, 15, 2275–2285. [Google Scholar] [CrossRef]
- Shan, S.; Wang, G.G. Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct. Multidisc. Optim. 2010, 41, 219–241. [Google Scholar] [CrossRef]
- Wang, H.; Rahnamayan, S.; Wu, Z. Parallel differential evolution with self-adapting control parameters and generalized opposition-based learning for solving high-dimensional optimization problems. J. Parallel Distrib. Comput. 2013, 73, 62–73. [Google Scholar] [CrossRef]
- Wang, S.; Liu, G.; Gao, M.; Cao, S.; Guo, A.; Wang, J. Heterogeneous comprehensive learning and dynamic multi-swarm particle swarm optimizer with two mutation operators. Inf. Sci. 2020, 540, 175–201. [Google Scholar] [CrossRef]
- Borowska, B. Genetic Learning Particle Swarm Optimization with Interlaced Ring Topology. In International Conference on Computational Science; Lecture Notes in Computer Science; Krzhizhanovskaya, V.V., Závodszky, G., Lees, M.H., Dongarra, J.J., Sloot, P.M.A., Brissos, S., Teixeira, J., Eds.; Springer: Cham, Switzerland, 2020; Volume 12141, pp. 136–148. [Google Scholar]
- Kennedy, J.; Eberhart, R. Particle Swarm Optimization. In Proceedings of the International Conference on Neural Networks, Perth, WA, Australia, 27 November–1 December 1995; pp. 1942–1948. [Google Scholar]
- Shi, Y.; Eberhart, R. A Modified Particle Swarm Optimizer. In Proceedings of the IEEE World Congress on Computational Intelligence, Anchorage, AK, USA, 4–9 May 1998; pp. 69–73. [Google Scholar]
- Clerc, M.; Kennecy, J. The Particle Swarm—Explosion, Stability, and Convergence in a Multidimensional Complex Space. IEEE Trans. Evol. Comput. 2002, 6, 58–73. [Google Scholar] [CrossRef] [Green Version]
- Shi, Y.; Eberhart, R.C. Empirical Study of Particle Swarm Optimization. In Proceedings of the IEEE Cngress on Evolutionary Computation, Washington, DC, USA, 6–9 July 1999; pp. 1945–1950. [Google Scholar]
- Shi, Y.; Eberhart, R.C. Parameter Selection in Particle Swarm Optimization. Evol. Program. VII 1998, 1447, 591–600. [Google Scholar]
- Trelea, I.C. The particle swarm optimization algorithm: Convergence analysis and parameter selection. Inf. Process. Lett. 2003, 85, 317–325. [Google Scholar] [CrossRef]
- Coelho, L.S.; Krohling, R.A. Predictive Controller Tuning Using Modified Particle Swarm Optimization Based on Cauchy and Gaussian Distributions. Adv. Soft Comput. 2005, 32, 287–298. [Google Scholar]
- Krohling, R.A.; Hoffmann, F.; Coelho, L.S. Co-evolutionary Particle Swarm Optimization for Min-Max Problems using Gaussian Distribution. In Proceedings of the IEEE Cngress on Evolutionary Computation, Portland, OR, USA, 19–23 June 2004; pp. 959–964. [Google Scholar]
- Krohling, R.A.; Mendel, E. Bare Bones Particle Swarm Optimization with Gaussian or Cauchy Jumps. In Proceedings of the IEEE Cngress on Evolutionary Computation, Trondheim, Norway, 18–21 May 2009; pp. 3285–3291. [Google Scholar]
- Lee, J.-W.; Lee, J.-J. Gaussian-Distributed Particle Swarm Optimization: A Novel Gaussian Particle Swarm Optimization. In Proceedings of the IEEE International Conference on Industrial Technology, Cape Town, South Africa, 25–28 February 2013; pp. 1122–1127. [Google Scholar]
- Kennedy, J. Bare bones particle swarms. In Proceedings of the IEEE Swarm Intelligence Symposium, Indianapolis, IN, USA, 26 April 2003; pp. 80–87. [Google Scholar]
- Van den Bergh, F.; Engelbrecht, A.P. A Cooperative approach to particle swarm optimization. IEEE Trans. Evol. Comput. 2004, 8, 225–239. [Google Scholar] [CrossRef]
- Krohling, R.A. Gaussian particle swarm with jumps. In Proceedings of the IEEE Cngress on Evolutionary Computation, Edinburgh, UK, 2–5 September 2005; pp. 1226–1231. [Google Scholar]
- Blackwell, T. A Study of Collapse in Bare Bones Particle Swarm Optimization. IEEE Trans. Evol. Comput. 2012, 16, 354–372. [Google Scholar] [CrossRef]
- Al-Rifaie, M.M.; Blackwell, T. Cognitive Bare Bones Particle Swarm Optimisation with Jumps. Int. J. Swarm Intell. Res. 2016, 7, 1–31. [Google Scholar] [CrossRef] [Green Version]
- Tang, K.; Li, X.; Suganthan, P.N.; Yang, Z.; Weise, T. Benchmark Functions for the CEC’2010 Special Session and Competition on Large Scale Global Optimization; Technical Report; Nat. Inspir. Comput. Appl. Lab., USTC: Hefei, China, 2009; Available online: http://nical.ustc.edu.cn/cec10ss.php (accessed on 17 October 2011).
- Zhan, Z.-H.; Zhang, J.; Li, Y.; Chung, H.S.-H. Adaptive Particle Swarm Optimization. IEEE Trans. Syst. Man Cybern.-Part B Cybern. 2009, 39, 1362–1381. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liang, J.J.; Qin, A.K.; Suganthan, P.N.; Baskar, S. Comprehensive Learning Particle Swarm Optimizer for Global Optimization of Multimodal Functions. IEEE Trans. Evol. Comput. 2006, 10, 281–295. [Google Scholar] [CrossRef]
Algorithm | Parameter | Value |
---|---|---|
HCBBPSO-Jx | Number of particles in Swarms P () | 25 |
(x = C or G) | Split factor (K) | 50 |
Number of particles in Swarm Q () | 25 | |
Jump scaling factor () | 1.1 | |
Number of maximum allowable update failure () | 5 | |
CPSO-HK [20] | Number of particles in each algorithm (CPSO-SK and PSO respectively) | 25 |
Split factor (K) | 6 | |
Weight for previous velocity (w) | 0.72 | |
Coefficient of cognition term () | 1.49 | |
Coefficient of social term () | 1.49 | |
BBPSOjumpx [17] | Number of particles | 50 |
(x = C or G) | Jump scaling factor () | 1.1 |
Number of maximum allowable update failure () | 5 | |
APSO [25] | Number of particles | 50 |
Weight for previous velocity | 0.9 | |
Coefficient of cognition term | 2 | |
Coefficient of social term | 2 | |
CLPSO [26] | Number of particles | 50 |
0.9 | ||
0.4 | ||
Coefficient of cognition term | 1.49445 | |
Coefficient of social term | 1.49445 | |
Refreshing gap (m) | 7 | |
(, : Lower and upper bound of i-th dimension) | 0.2() |
Ranking | 1st | 2nd | 3rd | 4th | 5th | 6th | 7th |
---|---|---|---|---|---|---|---|
Algorithm | HCBBPSO-JG | HCBBPSO-JC | BBPSOjumpC | BBPSOjumpG | CLPSO | CPSO6 | APSO |
Total Score | 5474 | 5357 | 4276 | 4233 | 4131 | 2517 | 2212 |
Ranking | 1st | 2nd | 3rd | 4th | 5th | 6th | 7th | 8th | 9th | 10th |
---|---|---|---|---|---|---|---|---|---|---|
Points | 25 | 18 | 15 | 12 | 10 | 8 | 6 | 4 | 2 | 1 |
Algorithm | ||||||||
---|---|---|---|---|---|---|---|---|
APSO | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (−) | (∼) | (+) | |
BBPSOjumpC | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (−) | (−) | (∼) | |
BBPSOjumpG | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (−) | (−) | (∼) | |
CLPSO | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (−) | (+) | (+) | (−) | (−) | (+) | |
CPSO-H6 | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (∼) | (−) | (+) | |
HCBBPSO-JC | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (∼) | (∼) | (∼) | (∼) | (∼) | (∼) | (+) | |
HCBBPSO-JG | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Total # of +/∼/− | 5/1/0 | 4/1/1 | 5/1/0 | 5/1/0 | 0/2/4 | 0/2/4 | 4/2/0 |
Algorithm | ||||||||
---|---|---|---|---|---|---|---|---|
APSO | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (+) | (+) | (+) | |
BBPSOjumpC | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (∼) | (+) | (−) | (−) | (+) | (+) | (+) | |
BBPSOjumpG | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (−) | (−) | (+) | (+) | (+) | |
CLPSO | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (∼) | (+) | (+) | (−) | (+) | (+) | (+) | |
CPSO-H6 | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (+) | (+) | (+) | (+) | (+) | (+) | (+) | |
HCBBPSO-JC | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
p-value | ||||||||
(Sign) | (∼) | (∼) | (∼) | (∼) | (∼) | (∼) | (∼) | |
HCBBPSO-JG | Best | |||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Total # of +/∼/− | 3/3/0 | 5/1/0 | 3/1/2 | 2/1/3 | 5/1/0 | 5/1/0 | 5/1/0 |
Algorithm | |||||||
---|---|---|---|---|---|---|---|
APSO | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (+) | (+) | (+) | (+) | (+) | (+) | |
BBPSOjumpC | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (−) | (−) | (+) | (+) | (+) | (+) | |
BBPSOjumpG | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (−) | (−) | (+) | (+) | (+) | (+) | |
CLPSO | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (+) | (−) | (+) | (+) | (+) | (+) | |
CPSO-H6 | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (+) | (+) | (+) | (+) | (+) | (+) | |
HCBBPSO-JC | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
p-value | |||||||
(Sign) | (∼) | (+) | (+) | (∼) | (∼) | (∼) | |
HCBBPSO-JG | Best | ||||||
Median | |||||||
Worst | |||||||
Mean | |||||||
Std | |||||||
Total # of +/∼/− | 3/2/1 | 3/0/3 | 6/0/0 | 5/1/0 | 5/1/0 | 5/1/0 |
1000D | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
1000D | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
1000D | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std | ||||||||
Best | ||||||||
Median | ||||||||
Worst | ||||||||
Mean | ||||||||
Std |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, J.; Kim, W. Heterogeneous Cooperative Bare-Bones Particle Swarm Optimization with Jump for High-Dimensional Problems. Electronics 2020, 9, 1539. https://doi.org/10.3390/electronics9091539
Lee J, Kim W. Heterogeneous Cooperative Bare-Bones Particle Swarm Optimization with Jump for High-Dimensional Problems. Electronics. 2020; 9(9):1539. https://doi.org/10.3390/electronics9091539
Chicago/Turabian StyleLee, Joonwoo, and Won Kim. 2020. "Heterogeneous Cooperative Bare-Bones Particle Swarm Optimization with Jump for High-Dimensional Problems" Electronics 9, no. 9: 1539. https://doi.org/10.3390/electronics9091539