# An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data

^{*}

## Abstract

**:**

## 1. Introduction

- We propose a novel feature selection method based on an embedding of the proposed asymmetric chaotic map in CSO. The proposed method can deal with high-dimensional problems effectively. This paper is the first work combining asymmetric chaotic CSO.
- The proposed method is compared with wrapper feature selection methods based on other swarm optimization algorithms in terms of classification accuracy and the selected feature subset’s compactness.
- A graphical competitive magic quadrant is used to depict the ability of the proposed method.

## 2. Materials and Methods

#### 2.1. Competitive Swarm Optimization (CSO)

#### 2.2. Chaos

#### 2.3. Simulated Annealing (SA)

_{0}= 2 * |N|, where |N| is the number of attributes in each dataset, and the cooling time is scheduled by T = 0.93 * T, as in Reference [4]. The pseudocode of SA is shown in Algorithm 1.

Algorithm 1: The pseudocode of the SA algorithm | |

1: | T_{0} = 2*|N|, where |N| is the number of attributes for each dataset |

2: | BestSol =${\mathit{S}}_{\mathit{i}}$ |

3: | $\mathit{\delta}$(BestSol) =$\mathit{\delta}({\mathit{S}}_{\mathit{i}}$) //$\mathit{\delta}$ indicates the quality of the solution |

4: | whileT > T_{0} |

5: | generate at random a new solution TrailSol in a neighbor of ${\mathit{S}}_{\mathit{i}}^{\mathbf{\prime}}$ |

6: | calculate $\mathit{\delta}$(TrailSol) |

7: | if($\mathit{\delta}$(TrailSol) > $\mathit{\delta}$(BestSol)) |

8: | ${\mathit{S}}_{\mathit{i}}^{\mathbf{\prime}}$ = TrailSol |

9: | BestSol = TrailSol |

10: | $\mathit{\delta}({\mathit{S}}_{\mathit{i}}$) = $\mathit{\delta}$(TrailSol) |

11: | $\mathit{\delta}$(BestSol) = $\mathit{\delta}$(TrailSol) |

12: | else if ($\mathit{\delta}$(TrailSol) = $\mathit{\delta}$(BestSol)) |

13: | Calculate |TrailSol } and | BestSol | |

14: | if(|TrailSol| < |BestSol|) |

15: | ${\mathit{S}}_{\mathit{i}}^{\mathbf{\prime}}$= TrailSol |

16: | BestSol = TrailSol |

17: | $\mathit{\delta}({\mathit{S}}_{\mathit{i}}$) = $\mathit{\delta}$(TrialSol) |

18: | $\mathit{\delta}$(BestSol) = $\mathit{\delta}$(TrailSol) |

19: | end if |

20: | else // accepting a worse solution |

21: | Calculate $\mathit{\theta}$ = $\mathit{\delta}$(TrailSol) − $\mathit{\delta}$ (BestSol) |

22: | Generate a uniform random number, P = [0,1] |

23: | if(P $\le {\mathit{e}}^{-\mathit{\theta}/\mathit{T}}$) |

24: | ${\mathit{S}}_{\mathit{i}}^{\mathbf{\prime}}$ = TrailSol |

25: | $\mathit{\delta}({\mathit{S}}_{\mathit{i}}$) = $\mathit{\delta}$(TrialSol) |

26: | end if |

27: | end if |

28: | T = 0.93 * T //update temperature |

29: | end while |

30: | Output BestSol |

Algorithm 2: The pseudocode of competitive swarm optimization (CSO) followed by SA (CSO-SA) | |

1: | t = 0; |

2: | randomly initialize particles P(0); //(*) |

3: | while the termination condition is not satisfied do |

4: | Calculate the fitness of each particle in P(t); |

5: | U = P(t), P(t + 1) = $\left\{\right\}$; |

6: | while U is not empty do |

7: | uniformly random choose two particles ${\mathit{X}}_{\mathbf{1}}\left(\mathit{t}\right),{\mathit{X}}_{\mathbf{2}}\left(\mathit{t}\right)$ from U; |

8: | if $\mathit{f}\left({\mathit{X}}_{\mathbf{1}}\left(\mathit{t}\right)\right)\le \mathit{f}\left({\mathit{X}}_{\mathbf{2}}\left(\mathit{t}\right)\right)$ then |

9: | ${\mathit{X}}_{\mathit{w}}\left(\mathit{t}\right)={\mathit{X}}_{\mathbf{1}}\left(\mathit{t}\right),{\mathit{X}}_{\mathbf{l}}\left(\mathit{t}\right)={\mathit{X}}_{\mathbf{2}}\left(\mathit{t}\right)$; |

10: | else |

11: | ${\mathit{X}}_{\mathit{w}}\left(\mathit{t}\right)={\mathit{X}}_{\mathbf{2}}\left(\mathit{t}\right),{\mathit{X}}_{\mathbf{l}}\left(\mathit{t}\right)={\mathit{X}}_{\mathbf{1}}\left(\mathit{t}\right)$; |

12: | end if |

13: | add ${\mathit{X}}_{\mathit{w}}\left(\mathit{t}\right)$ into P(t + 1) |

14: | randomly generate ${\mathit{R}}_{\mathbf{1}}^{\mathit{t}},{\mathit{R}}_{\mathbf{2}}^{\mathit{t}},{\mathit{R}}_{\mathbf{3}}^{\mathit{t}}$ //(*) |

15: | update ${\mathit{X}}_{\mathbf{l}}\left(\mathit{t}\right)$ using (1) and (2); |

16: | add the updated ${\mathit{X}}_{\mathbf{l}}\left(\mathit{t}+\mathbf{1}\right)$ to P(t + 1) |

17: | Remove ${\mathit{X}}_{\mathbf{1}}\left(\mathit{t}\right),{\mathit{X}}_{\mathbf{2}}\left(\mathit{t}\right)$ from U; |

18: | end while |

19: | t = t + 1; |

20: | Use the SA algorithm (Algorithm 1) to find a better gbest |

21: | t = t + 1; |

22: | end while |

Note: (1) The symbol “//” is a comment, (2) if the algorithm is without SA, then line numbers 20 and 21 are removed. |

## 3. The Proposed Asymmetric Chaotic Competitive Swarm Optimization

#### 3.1. The Proposed Asymmetric (Right-Skewed) Chaotic Map

#### 3.2. The Process Approach

#### 3.3. An Asymmetric Chaotic CSO

Algorithm 3: An asymmetric chaotic competitive swarm optimization (ACCSO) | |

1: | The pseudocode of Algorithm 3 is as same as Algorithm 2. |

2: | There are three modifications in Algorithm 2. |

3: | The first modification is at line number 2: a population of N particles is initialized using the proposed asymmetric chaos map. |

4: | The second change is at line number 14: the three vectors ${R}_{1}^{t},{R}_{2}^{t},\mathrm{and}{R}_{3}^{t}$ are randomly generated vectors within ${\left[0,1\right]}^{n}$ using the proposed asymmetric chaos function. |

5. | Line numbers 20 and 21 of Algorithm 2 will be removed. The rest remains the same. |

#### 3.4. An Asymmetric Chaos CSO Followed by SA (ACCSO-SA)

Algorithm 4: An asymmetric chaotic competitive swarm optimization followed by SA (ACCSO-SA) | |

1: | The pseudocode of Algorithm 4 is as the same as Algorithm 3, but line numbers 20 and 21 are uncommented. |

2: | The rest remains the same. |

## 4. Experimental Results

#### 4.1. Dataset

#### 4.2. Parameter Settings

#### 4.3. Comparison of CSO-Kent, CSO-Neuron, and ACCSO

#### 4.4. Comparison of ACCSO and ACCSO-SA

#### 4.5. Comparison with Other Algorithms

#### 4.6. Magic Quadrant

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Hossain, E.; Khan, I.; Un-noor, F.; Sikander, S.S.; Sunny, S.H. Application of Big Data and Machine Learning in Smart Grid, and Associated Security Concerns: A Review. IEEE Access
**2019**, 7, 13960–13988. [Google Scholar] [CrossRef] - Rong, M.; Gong, D.; Gao, X. Feature Selection and Its Use in Big Data: Challenges, Methods, and Trends. IEEE Access
**2019**, 7, 19709–19725. [Google Scholar] [CrossRef] - Al-Tashi, Q.; Abdulkadir, S.J.; Rais, H.M.; Mirjalili, S.; Alhussian, H. Approaches to Multi-Objective Feature Selection: A Systematic Literature Review. IEEE Access
**2020**, 8, 125076–125096. [Google Scholar] [CrossRef] - Jia, H.; Li, J.; Song, W.; Peng, X.; Lang, C. Spotted Hyena Optimization Algorithm With Simulated Annealing for Feature Selection. IEEE Access
**2019**, 7, 71943–71962. [Google Scholar] [CrossRef] - Dhiman, G.; Kumar, V. Spotted hyena optimizer: A novel bio-inspired based metaheuristic technique for engineering applications. Adv. Eng. Softw.
**2017**, 114, 48–70. [Google Scholar] [CrossRef] - Brezočnik, L.; Fister, I.; Podgorelec, V. Swarm intelligence algorithms for feature selection: A review. Appl. Sci.
**2018**, 8, 1521. [Google Scholar] [CrossRef] [Green Version] - Cheng, R.; Jin, Y. A Competitive Swarm Optimizer for Large Scale Optimization. IEEE Trans. Cybern.
**2014**, 45, 191–204. [Google Scholar] [CrossRef] - Sun, C.; Ding, J.; Zeng, J.; Jin, Y. Regular research paper a fitness approximation assisted competitive swarm optimizer for large scale expensive optimization problems. Memetic Comput.
**2018**, 10, 123–134. [Google Scholar] [CrossRef] - Xiong, G.; Shi, D. Orthogonal learning competitive swarm optimizer for economic dispatch problems. Appl. Soft Comput.
**2018**, 66, 134–148. [Google Scholar] [CrossRef] - Ling, T.; Zhan, Z.-H.; Wang, Y.; Wang, Z.-J.; Yu, W.-J.; Zhang, J. Competitive Swarm Optimizer with Dynamic Grouping for Large Scale Optimization. In Proceedings of the 2018 IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 2655–2660. [Google Scholar]
- Too, J.; Abdullah, A.R.; Saad, N.M. Binary Competitive Swarm Optimizer Approaches for Feature Selection. Computation
**2019**, 7, 31. [Google Scholar] [CrossRef] [Green Version] - Gu, S.; Cheng, R.; Jin, Y. Feature selection for high-dimensional classification using a competitive swarm optimizer. Soft Comput.
**2018**, 22, 811–822. [Google Scholar] [CrossRef] [Green Version] - Eshtay, M.; Faris, H.; Obeid, N. Improving Extreme Learning Machine by Competitive Swarm Optimization and its application for medical diagnosis problems. Expert Syst. Appl.
**2018**, 104, 134–152. [Google Scholar] [CrossRef] - Huang, S.; Tao, M. Competitive swarm optimizer based gateway deployment algorithm in cyber-physical systems. Sensors
**2017**, 17, 209. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Liu, B.; Wang, L.; Jin, Y.-H.; Tang, F.; Huang, D.-X. Improved particle swarm optimization combined with chaos. Chaos Solitons Fractals
**2005**, 25, 1261–1271. [Google Scholar] [CrossRef] - Dong, S.; Dong, Z.; Ma, J.; Chen, K. Improved PSO algorithm based on chaos theory and its application to design flood hydrograph. Water Sci. Eng.
**2010**, 3, 156–165. [Google Scholar] - Gandomi, A.H.; Yun, G.J.; Yang, X.S.; Talatahari, S. Chaos-enhanced accelerated particle swarm optimization. Commun. Nonlinear Sci. Numer. Simul.
**2013**, 18, 327–340. [Google Scholar] [CrossRef] - Wang, G.G.; Deb, S.; Gandomi, A.H.; Zhang, Z.; Alavi, A.H. Chaotic cuckoo search. Soft Comput.
**2016**, 20, 3349–3362. [Google Scholar] [CrossRef] - Kohli, M.; Arora, S. Chaotic grey wolf optimization algorithm for constrained optimization problems. J. Comput. Des. Eng.
**2017**, 5, 458–472. [Google Scholar] [CrossRef] - Saha, S.; Mukherjee, V. A novel chaos-integrated symbiotic organisms search algorithm for global optimization. Soft Comput.
**2017**, 22, 3797–3816. [Google Scholar] [CrossRef] - Wang, Y.; Li, H.; Gao, H.; Kwong, S. A Chaotic Based Artificial Bee Colony Algorithm. In Proceedings of the 2018 Fifth HCT Information Technology Trends (ITT), Dubai, UAE, 28–29 November 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 165–169. [Google Scholar]
- Talbi, E.-G. A Taxonomy of Hybrid Metaheuristics. J. Heuristics
**2002**, 8, 541–564. [Google Scholar] [CrossRef] - Feng, J.; Zhang, J.; Zhu, X.; Lian, W. A novel chaos optimization algorithm. Multimed. Tools Appl.
**2017**, 76, 17405–17436. [Google Scholar] [CrossRef] - Gao, S.; Vairappan, C.; Wang, Y.; Cao, Q.; Tang, Z. Gravitational search algorithm combined with chaos for unconstrained numerical optimization. Appl. Math. Comput.
**2014**, 231, 48–62. [Google Scholar] [CrossRef] - Kirkpatrick, S.; Gelatt, C.D.; Vecchi, M.P. Optimization by simulated annealing. Science
**1983**, 220, 671–680. [Google Scholar] [CrossRef] [PubMed] - Wang, G.-G.; Guo, L.; Gandomi, A.H.; Alavi, A.H.; Duan, H. Simulated Annealing-Based Krill Herd Algorithm for Global Optimization. Abstr. Appl. Anal.
**2013**, 2013, 213853. [Google Scholar] [CrossRef] [Green Version] - Emary, E.; Zawbaa, H.M.; Hassanien, A.E. Binary grey wolf optimization approaches for feature selection. Neurocomputing
**2016**, 172, 371–381. [Google Scholar] [CrossRef] - Nasa-ngium, P.; Sunat, K. Impacts of Linnik Flight Usage Patterns on Cuckoo Search for Real-Parameter Global Optimization Problems. IEEE Access
**2019**, 7, 83932–83961. [Google Scholar] [CrossRef]

**Figure 2.**(

**a**) Histogram of asymmetric chaotic sequences, and (

**b**) chaotic sequences generated by the proposed chaotic map, Equation (5).

**Figure 3.**Comparison between the proposed asymmetric chaotic CSO algorithm and other algorithms based on the accuracy (

**a**) and the number of features (

**b**).

No. | Dataset Name | Number of Instances | Number of Attributes | Number of Classes |
---|---|---|---|---|

1 | Iris | 150 | 4 | 3 |

2 | Lirbras | 360 | 90 | 15 |

3 | ORL | 200 | 1024 | 40 |

4 | Colon | 62 | 2000 | 2 |

5 | Madelon | 2600 | 500 | 2 |

6 | Lymphoma | 96 | 4026 | 9 |

7 | Glioma | 50 | 4434 | 4 |

8 | Prostate_GE | 102 | 5966 | 2 |

9 | Nci9 | 60 | 9172 | 9 |

10 | Carcinom | 174 | 9182 | 11 |

11 | Pixraw10P | 100 | 10,000 | 10 |

12 | Orlraw10P | 100 | 10,304 | 10 |

**Table 2.**Comparison of the average number of selected features and accuracy among the proposed and related work methods.

Dataset | Attribute | Accuracy | ||||
---|---|---|---|---|---|---|

CSO-Kent | CSO-Neuron | ACCSO | CSO-Kent | CSO-Neuron | ACCSO | |

Iris | 2 | 2 | 2 | 0.9600 | 0.9600 | 0.9467 |

Lirbras | 41.5 | 43.4 | 43.1 | 0.7924 | 0.7844 | 0.8196 |

Orl | 503.7 | 506.8 | 504.4 | 0.8572 | 0.8930 | 0.7968 |

Colon | 988.4 | 970 | 973.6 | 0.7806 | 0.7161 | 0.7796 |

Madelon | 249.9 | 259 | 278.51 | 0.6900 | 0.6985 | 0.8247 |

Lymphoma | 1961 | 1953 | 1975.4 | 0.9167 | 0.9792 | 0.9000 |

Glioma | 2185.9 | 2185.8 | 2179.7 | 0.6027 | 0.9280 | 0.8453 |

Prostate_ge | 2957.1 | 2981.2 | 2916.4 | 0.8595 | 0.7804 | 0.9033 |

Nci9 | 4816.5 | 4848.2 | 4768.4 | 0.8567 | 0.8680 | 0.8333 |

Carcinom | 4560.9 | 4561.4 | 4555.7 | 0.8808 | 0.9011 | 0.9042 |

Pixraw10p | 4852.8 | 4856.2 | 4876.7 | 0.9800 | 1.0000 | 1.0000 |

Orlraws10p | 5061.8 | 5074.4 | 5002.5 | 0.9400 | 0.8800 | 0.9400 |

Sum all | 28,181.5 | 28,241.4 | 28,076.41 | 10.1166 | 10.3887 | 10.4935 |

Ranking | 2 | 3 | 1 | 3 | 2 | 1 |

**Table 3.**Classification accuracy and ranking comparison between the proposed and related work methods.

Dataset | PSO | GWO | SOS | BAT | CSO | CSO-Kent | CSO-Neuron | ACCSO | ACCSO-SA |
---|---|---|---|---|---|---|---|---|---|

Iris | 0.9600 | 0.9600 | 0.9556 | 0.9562 | 0.9733 | 0.9600 | 0.9600 | 0.9467 | 0.9600 |

Lirbras | 0.7789 | 0.8133 | 0.7912 | 0.7914 | 0.8278 | 0.7924 | 0.7844 | 0.8196 | 0.8289 |

ORL | 0.864 | 0.8050 | 0.7756 | 0.7730 | 0.8720 | 0.8572 | 0.893 | 0.7968 | 0.9200 |

Colon | 0.8094 | 0.8645 | 0.7333 | 0.7373 | 0.7097 | 0.7806 | 0.7161 | 0.7796 | 0.7419 |

Madelon | 0.6935 | 0.7018 | 0.7163 | 0.7067 | 0.7011 | 0.6900 | 0.6985 | 0.8247 | 0.6926 |

Lymphoma | 0.9958 | 0.8958 | 0.9396 | 0.8250 | 0.9792 | 0.9167 | 0.9792 | 0.9000 | 0.8542 |

Glioma | 0.8400 | 0.8000 | 0.6863 | 0.6943 | 0.7200 | 0.6027 | 0.9280 | 0.8453 | 0.7840 |

Prostate_ge | 0.8331 | 0.7451 | 0.8357 | 0.8437 | 0.8235 | 0.8595 | 0.7804 | 0.9033 | 0.8863 |

Nci9 | 0.6200 | 0.6800 | 0.7685 | 0.6067 | 0.6133 | 0.8567 | 0.8680 | 0.8333 | 0.7340 |

Carcinom | 0.8076 | 0.9103 | 0.7075 | 0.6997 | 0.9356 | 0.8808 | 0.9011 | 0.9042 | 0.9011 |

Pixraw10p | 0.9400 | 0.9400 | 0.9524 | 0.9523 | 0.9800 | 0.9800 | 1 | 1 | 0.9600 |

Orlraws10p | 0.9600 | 0.9400 | 0.8984 | 0.8989 | 0.9400 | 0.9400 | 0.8800 | 0.9400 | 0.9240 |

Average | 10.1023 | 10.0558 | 9.7604 | 9.4852 | 10.0755 | 10.1166 | 10.3887 | 10.4935 | 10.187 |

Ranking | 5 | 7 | 8 | 9 | 6 | 4 | 2 | 1 | 3 |

**Table 4.**Average number of selected features and ranking comparison between the proposed and related work methods.

Dataset | PSO | GWO | SOS | BAT | CSO | CSO-Kent | CSO-Neuron | ACCSO | ACCSO-SA |
---|---|---|---|---|---|---|---|---|---|

Iris | 1 | 2 | 1 | 1 | 1 | 2 | 2 | 2 | 2 |

Lirbras | 42.8 | 46.2 | 43 | 47 | 42.4 | 41.5 | 43.4 | 43.1 | 42.4 |

Orl | 513.4 | 497.8 | 532 | 516 | 505.6 | 503.7 | 506.8 | 504.4 | 497 |

Colon | 990.6 | 1012 | 1036 | 967 | 948 | 988.4 | 970 | 973.6 | 993.2 |

Madelon | 253.6 | 245.4 | 257 | 239 | 258.2 | 249.9 | 259 | 278.51 | 245.8 |

Lymphoma | 2029.8 | 1993.6 | 1992 | 1921 | 1951.4 | 1961 | 1953 | 1975.4 | 1940.2 |

Glioma | 2155 | 2172.8 | 2204 | 2246 | 2162.6 | 2185.9 | 2185.8 | 2179.7 | 2200.8 |

Prostate_ge | 2942 | 2943.8 | 2971 | 2959 | 2931.4 | 2957.1 | 2981.2 | 2916.4 | 2941.2 |

Nci9 | 4804.8 | 4725.8 | 4942 | 4674 | 4798 | 4816.5 | 4848.2 | 4768.4 | 4834.2 |

Carcinom | 4552.4 | 4551.2 | 4593 | 4591 | 4577.2 | 4560.9 | 4561.4 | 4555.7 | 4524.6 |

Pixraw10p | 4854.4 | 4857 | 5011 | 4988 | 4868 | 4852.8 | 4856.2 | 4876.7 | 4855.6 |

Orlraws10p | 5048.8 | 5106.6 | 5145 | 5190 | 5108.4 | 5061.8 | 5074.4 | 5002.5 | 5043.4 |

Average | 28,188.6 | 28,154.2 | 28,727 | 28,339 | 28,152.2 | 28,181.5 | 28,241.4 | 28,076.41 | 28,120.4 |

Ranking | 6 | 4 | 9 | 8 | 3 | 5 | 7 | 1 | 2 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pichai, S.; Sunat, K.; Chiewchanwattana, S.
An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data. *Symmetry* **2020**, *12*, 1782.
https://doi.org/10.3390/sym12111782

**AMA Style**

Pichai S, Sunat K, Chiewchanwattana S.
An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data. *Symmetry*. 2020; 12(11):1782.
https://doi.org/10.3390/sym12111782

**Chicago/Turabian Style**

Pichai, Supailin, Khamron Sunat, and Sirapat Chiewchanwattana.
2020. "An Asymmetric Chaotic Competitive Swarm Optimization Algorithm for Feature Selection in High-Dimensional Data" *Symmetry* 12, no. 11: 1782.
https://doi.org/10.3390/sym12111782