A Survey on Particle Swarm Optimization for Association Rule Mining

Abstract

Association rule mining (ARM) is one of the core techniques of data mining to discover potentially valuable association relationships from mixed datasets. In the current research, various heuristic algorithms have been introduced into ARM to address the high computation time of traditional ARM. Although a more detailed review of the heuristic algorithms based on ARM is available, this paper differs from the existing reviews in that we expected it to provide a more comprehensive and multi-faceted survey of emerging research, which could provide a reference for researchers in the field to help them understand the state-of-the-art PSO-based ARM algorithms. In this paper, we review the existing research results. Heuristic algorithms for ARM were divided into three main groups, including biologically inspired, physically inspired, and other algorithms. Additionally, different types of ARM and their evaluation metrics are described in this paper, and the current status of the improvement in PSO algorithms is discussed in stages, including swarm initialization, algorithm parameter optimization, optimal particle update, and velocity and position updates. Furthermore, we discuss the applications of PSO-based ARM algorithms and propose further research directions by exploring the existing problems.

1. Introduction

Association rule mining (ARM), as one of the core techniques of data mining (DM), can discover potentially valuable association relationships between itemsets from complex and fuzzy mixed data. The Apriori algorithm is one of the classical algorithms for ARM, which was first proposed by Agrawal et al. [1] and has gained the attention and research of many scholars in the past two decades. Akbar et al. [2] reviewed the research on ARM algorithms based on evolutionary computation published before 2019 and classified the evolutionary ARM algorithms into four major categories: evolutionary-based, swarm intelligence-based, physics-inspired, and hybrid approaches. Minakshi et al. [3] systematically assessed numerical association rule mining (NARM) algorithms to help scholars select a suitable algorithm when performing NARM mining. Djenouri et al. [4] studied the application of various metaheuristics algorithms in frequent itemset mining (FIM) and high-utility itemset mining (HUIM), including genetic algorithms (GA), ant colony optimization (ACO) algorithms, artificial bee colony (ABC), and particle swarm optimization (PSO). Arani et al. [5] also used a metaheuristic-based clustering mechanism when analyzing cloud workloads. In addition, Gan et al. [6] combined the design of convolutional neural networks (CNN) to address the shortcomings of traditional mining models that cannot extract temporal data effectively by constructing a new temporal association rule mining model. Similar papers include [7,8]. Badhon et al. [9] developed a systematic and structured analysis of ARM algorithms in terms of different aspects, including chromosome representation, genetic operators, and initial swarms. In addition, Kheirollahi et al. [10] combined the clustering algorithm with the PSO algorithm, which can be applied to recognize effective, good clusters and extract strong rules among reservoir data. Datta et al. [11] introduced the concept of flexible dissociation to propose a significant ARM framework with high associativity. In response to the rapid growth of data, Sanjay et al. [12] proposed a distributed ARM algorithm called adaptive-miner, which can dynamically change the mining algorithm according to the characteristics of the dataset. Among them, the PSO algorithm has already received the attention of many scholars at home and abroad since it was proposed because of its few adjustable parameters, high efficiency, easy implementation, and fast convergence speed. The current research shows that ARM using the PSO algorithms has higher operational efficiency compared with other data mining algorithms and can generate more accurate and valuable rules [13,14,15,16,17,18,19,20,21].

However, recent papers reviewed only a small fraction of the PSO-based ARM algorithms; however, there have been many papers published in recent years. Therefore, we aimed to provide a more comprehensive and multi-faceted survey of emerging research. We expect this survey can provide a reference for researchers in the field to help them understand the state-of-the-art PSO-based ARM algorithms. We downloaded all the articles containing PSO or ARM keywords in recent years, removed the less cited and less relevant ones, and selected the remaining 117 papers for use in our survey.

To analyze these papers more systematically and comprehensively, we divided the ARM algorithms according to the different pattern types. They were fuzzy association rule mining, rare association rule mining, numerical association rule mining, quantitative association rule mining, binary association rule mining, high-utility association rule mining, and classification association rule mining, and they are shown in blue in the inner circle of Figure 1. In addition, we compared them from three perspectives: biologically inspired, physically inspired, and other algorithms, and listed the key information of each algorithm under the different perspectives, as shown in the outer circle of Figure 1. The specific description and summary of the characteristics of each algorithm can be found in Section 3 of the article.

The remaining paper is organized as follows: In Section 2, the concept of ARM is described in detail, and it is divided into three categories: classification association rule mining (CARM) [22], fuzzy association rule mining (FARM) [23,24] and numerical association rule mining (NARM) [15,25,26,27]; their algorithm evaluation metrics are described in three sections. In Section 3, different algorithms in ARM are compared and categorized into three groups: bio-inspired, physics-inspired, and other algorithms, and in each group, a paragraph describing the algorithms and a summary of their features are provided. The process of the PSO-based ARM algorithm is described in detail in Section 4, including the binary conversion of data and the encoding process. Section 5 discusses the current status of research on the ARM algorithms based on PSO algorithms from four perspectives: swarm initialization, algorithm parameter optimization, optimal particle update, and velocity and position update. The applications of the PSO algorithm in ARM are summarized in Section 6. Finally, the paper concludes with concluding statements and future aspects in Section 7.

2. Association Rule Mining

2.1. Definition of ARM

The definition of association rules was first introduced by Agrawal in 1993: $I=\left\{i_1,i_2,\cdots ,i_n\right\}$ is called the itemsets, where $i_j\in \left\{0,1\right\}$ is called an item, $D=\left\{d_1,d_2,\cdots ,d_m\right\}$ is called a database, and $d_k$ is called a transaction. A transaction is an itemset, i.e., a transaction is a subset $d_k\subseteq I$ of a transaction, and each transaction has a unique ID. Each rule consists of two different itemsets, $X$ and $Y$, where $X$ is called the premise, and Y is called the conclusion. Rules are represented as $X\to Y$, and $X,Y\subseteq I$.

2.2. Mode Types for ARM

2.2.1. CARM

CARM is a widely used method in the real world. Compared with traditional ARM, CARM has the characteristics of high classification accuracy and adaptability, etc. Its mining is mainly divided into the following processes: in the preprocessing stage, if the data are continuous values, they need to be discretized; then, class association rules that satisfy the minimum support and confidence are mined in the database, and the generated association rules are pruned. Finally, the classifier is used to predict the data objects of unknown classes. Currently, this model is widely used in several fields, such as medicine and biology.

2.2.2. FARM

For traditional ARM, its itemsets are often discretized by dividing the intervals into continuous values [28], but the method suffers from the drawback of dividing the intervals too hard. Therefore, Chan et al. [23] proposed the concept of fuzzy association rules to divide the interval into multiple fuzzy sets by softening the division boundary with fuzzy sets and obtaining the probability of an element on a fuzzy set based on the fuzzy affiliation function to provide a smooth transition between the members and nonmembers in the itemset.

Let the itemset set $I=\left\{i_1,i_2,\cdots ,i_n\right\}$, database $D=\left\{t_1,t_2,\cdots ,t_m\right\}$, and each element $i_j$ in the itemset correspond to a fuzzy set $F_{ik}=\left\{f_{ik}^1,f_{ik}^2,\cdots ,f_{ik}^l\right\}$. The fuzzy association rule for mining is that if $T$ is $A$, then $F$ is $B$. Where, $T=\left\{t_1,t_2,\cdots ,t_p\right\}$, $F=\left\{f_1,f_2,\cdots ,f_p\right\}$ are the itemsets, $A=\left\{a_1,a_2,\cdots ,a_p\right\}$, $B=\left\{b_1,b_2,\cdots ,b_p\right\}$ are the fuzzy sets corresponding to T and F, respectively. The rule premise $X=T$ is $A$, and the conclusion $Y=F$ is $B$. This is expressed as: when $X$ is satisfied, it can be concluded that Y is also satisfied. For this purpose, we use the significance factor and the certainty factor to determine whether the rule is interesting or not.

Significance Factor: To generate a fuzzy association rule, it is first necessary to obtain the set of k items with a significance factor above a user-specified threshold, denoted as $S_{<T,A>}$, which is calculated as follows:

$$S_{<T,A>}=\frac{{\displaystyle {\sum }_{d_k\in D}{\displaystyle {\prod }_{t_i\in T}\left\{{\alpha }_{a_j}(d_k[t_i])\right\}}}}{Total(D)}$$

(1)

$${\alpha }_{a_j}\left(d_k\left[t_j\right]\right)=\{\begin{array}{ll}{m}_{a_j}\in A\left(d_k\left[t_j\right]\right)\hfill & , if m_{a_j}>w\hfill \\ 0\hfill & , otherwise\hfill \end{array}$$

(2)

where $Total(D)$ indicates the total number of transactions in database D, ${\alpha }_{aj}(d_k\left[t_j\right])$ indicates the membership grade of $t_j$ in the $k$th records.

Certainty Factor: After the first step of calculation to obtain a large set of K terms with significance factors above a given value $<Z,C>$, a deterministic factor is introduced to measure whether the rule is interesting or not, which is calculated as follows:

$$C_{<<T,A>,<F,B>>}=\frac{{\displaystyle {\sum }_{d_k\in D}{\displaystyle {\prod }_{z_i\in Z}\left\{{\alpha }_{c_i}\left(d_k\left[z_i\right]\right)\right\}}}}{{\displaystyle {\sum }_{d_k\in D}{\displaystyle {\prod }_{t_j\in T}\left\{{\alpha }_{a_j}\left(d_k\left[t_j\right]\right)\right\}}}}$$

(3)

$${\alpha }_{c_i}\left(d_k\left[z_i\right]\right)=\{\begin{array}{ll}{m}_{c_i}\in c\left(d_k\left[z_i\right]\right)\hfill & , if m_{c_i}>w\hfill \\ 0\hfill & , otherwise\hfill \end{array}$$

(4)

$$Z=T\cup F,C=A\cup B$$

(5)

2.2.3. NARM

In the real world, data are often represented in the form of numerical values, and the process of finding the association rules in numerical data is called NARM. In solving the NARM problem, most of the existing studies are divided into three methods: optimization, discretization, and distribution. Among them, the optimization methods can be divided into several sub-methods, such as swarm intelligence-based and evolution-based; the discretization methods are for discrete-continuous values, which can be further divided into supervised and unsupervised, static and dynamic, and other sub-methods; and the distribution methods include the mean, median, and variance. The division methods are shown in Figure 2.

2.3. Evaluation Metrics for ARM

2.3.1. Basic Evaluation Indicators

To select the valuable rules from the possible frequent itemsets, some metrics are needed to filter and sift them. In most studies, the strength of a rule is described by support and confidence.

Support: Support indicates the frequency of the itemsets in the database. The support of the association rule $X\to Y$ is the ratio of the number of transactions in the database that also contain the itemsets $X$ and $Y$ to the total number of transactions.

$$Support\left(X\to Y\right)=P\left(X\cap Y\right)=\frac{X\cap Y}{all}$$

(6)

Confidence: The confidence level is used to measure the confidence level of a rule. The confidence level of the association rule $X\to Y$ is the ratio of the number of transactions in the database that contain both $X$ and $Y$ and the total number of transactions that contain $X$.

$$Confidence\left(X\to Y\right)=P\left(X|Y\right)=\frac{P\left(X\cap Y\right)}{P\left(X\right)}$$

(7)

When $Support(X\to Y)$ and $Confidence\left(X\to Y\right)$ are both greater than the minimum support threshold and the minimum confidence threshold, if $Lift\left(X\to Y\right)>1$, then $X\to Y$ is a strong association rule, and vice versa.

2.3.2. Quantitative Evaluation Indicators

Most researchers use support and confidence to evaluate rules. The inherent drawback of this is that rare data items with less than the minimum support cannot be studied, and the support–confidence-based evaluation system generates a huge number of invalid rules; thus, other evaluation metrics need to be introduced to further mine the rules of interest to users [29].

Correlativity: Correlation reflects the degree of interdependence between X and Y. If $Correlativity\left(X\to Y\right)<1$, then it means that $X\to Y$ is an invalid strong correlation rule, and if $Correlativity\left(X\to Y\right)=1$, then $X$ and $Y$ are independent of each other.

$$Correlativity\left(X\to Y\right)=\frac{P\left(X|Y\right)}{P\left(Y\right)}=\frac{Confidence\left(X\to Y\right)}{Support\left(Y\right)}$$

(8)

Effect: The degree of influence reflects the degree of correlation between $X$ and $Y$. It can filter invalid association rules and generate meaningful positive and negative association rules. $\left|Effect(X\to Y)\right|$ is larger, indicating a high degree of $X\to Y$ correlation.

$$Effect(X\to Y)=\frac{Confidence(X\to Y)-Support(Y)}{Max\left\{Confidence(X\to Y),Support(Y)\right\}}$$

(9)

Time: The timeliness reflects the value of time between $X$ and $Y$ and can help decision makers remove redundant or already outdated information. The temporal validity of the rules decreases exponentially with increasing time intervals. The temporal validity of support and confidence is calculated as shown in Equations (10) and (11), respectively.

$$Time\_Support(X\to Y)={\displaystyle \sum _{i=1}^{n}Support(X_i)\cdot e^{-\alpha \cdot \left|t_i-t_0\right|}}$$

(10)

$$Time\_Confidence(X\to Y)={\displaystyle \sum _{i=1}^{n}Confidence(X_i)\cdot e^{-\alpha \cdot \left|t_i-t_0\right|}}$$

(11)

where $X_i$ denotes the $i$th transaction in the X events and $t_i$ denotes the time when the $i$th transaction in the $X$ events occurred.

2.3.3. Qualitative Evaluation Indicators

Utility: Utility reflects the potential usefulness of some attributes in the rule and their importance, and $Utility\in \left[0,1\right]$. When $Utility=1$, it means the attribute is the most important, and vice versa.

Concision: Simplicity is used to measure the comprehensibility of the association rule and is related to the arithmetic average of the number of items and the abstraction level of the antecedent items of the rule, and the weighted arithmetic average of the precedent items, whose weights depend on the importance of the antecedent and precedent items. The higher the number of items in a rule, the lower its comprehensibility, and the higher the level of abstraction contained in a rule, the lower its comprehensibility. The greater the simplicity of a rule, the more interesting it is to the user.

3. Comparison of Different ARM Algorithms

Due to the features of association rules in high-dimensional spaces, many scholars obtain better solutions through iterative heuristic processes. In this paper, we divided the ARM methods based on the heuristic algorithms into three main groups: bio-inspired, physics-inspired, and other algorithms, and provided a paragraph describing the algorithms and a summary of their features in each group.

3.1. Bio-Inspiration

3.1.1. Evolution-Based Algorithms

Some evolution-based algorithms are shown in

Table 1. ARM, based on evolutionary algorithm.

Algorithms	Authors	Technology	Advantages	Disadvantages
DE	Wang et al. [30]	Adaptive adjustment F and Cr	Enhanced algorithmic global search capability	High memory overhead
	Altay et al. [31]	DE-SCA, Multi-target,	Performs well on datasets with few attributes and many instances	High algorithmic complexity
	Guan et al. [32]	HGDE	Increased swarm diversity and high stability	Unable to handle high-dimensional data items
GA	Menaga et al. [33]	GAPPARM	High practicality	Easy to cause data loss
	Lin et al. [34]	Clustering the data	High accuracy and performance	Single objective
	Neysiani et al. [35]	Identify similarity between itemsets by association rules	High-quality rules	Long execution time
BBO	Giri et al. [36]	LGBBO-RuleMiner	High accuracy and a simple algorithm	Only for single target problems
	Arani et al. [37]	K-means, Bayesian learning	Reduce the delay, SLA violation ratio, cost, and energy consumption	High CPU usage
CSA	Mo et al. [38]	DMARICA	Short execution time, Highly scalable	Less integrity of generation rules
AIS	Husain et al. [39]	CLONALG	High accuracy	Accuracy is positively correlated with the number of iterations
	Danilo et al. [40]	CLONALG-GA	Performs better in sparse datasets	Not suitable for dense datasets

.

Evolutionary algorithms are based on biological evolutionary mechanisms and are mostly inspired by the biological evolution of nature, including natural selection and genetics, which have higher robustness and wide adaptability, and efficiently solve complex problems that are difficult to solve in traditional optimization algorithms.

3.1.2. Swarm Intelligence-Based Algorithm

Some algorithms based on swarm intelligence are shown in

Table 2. ARM, based on swarm intelligence algorithm.

Algorithms	Authors	Technology	Advantages	Disadvantages
PSO	Tyagi et al. [41]	Multi-target, MOPSO-ARM	Performs well on sparse data	Calculated overload, low efficiency
	Kuo et al. [42]	Adaptive Archive Grid, MOPSO	Automatically finds the best interval between datasets; no data preprocessing is required.	Fewer targets to consider
	Baro et al. [14]	Guided search strategy	High-quality rules and short calculation times	Lower average fitness values
	Agarwal et al. [43]	Multi-objective	Balancing global and local search capabilities	Not applicable to quantitative association rules
	Moslehi et al. [44]	GA-PSO	No predefined minimum support and confidence levels are required	Generate redundant rules
CSO	Devi et al. [45]	HGWCSO-ETSVM, Min-Max	High accuracy and precision	Reduced overall system performance
AMO	Son et al. [46]	ARM-AMO	Reduces the time and memory required for frequent itemset generation	Lower quality rules
BAT	Kamel et al. [27]	MSBARM, loop strategy	Rule quality is higher than other ways algorithm	Unable to handle large databases
FA	Pradeep et al. [47]	BFFO-TA, Feature Selection	No redundant rules are created	Not applicable to multi-objective problems
GWO	Pazhaniraa et al. [48]	BGWO-HUI	Low time complexity	High memory overhead
	Yildirim et al. [49]	Adaptive multi-target intelligent search, MOGWO	For discrete, quantitative, and mixed datasets with high comprehensibility	Not suitable for distributed datasets
	Chantar et al. [50]	SVM, Elite Cross, BGWO	High accuracy	High time complexity
ABC	Akbar et al. [51]	ABC4ARH	Highly efficient and stable	Less scalability and higher complexity
HHO	Turabieh et al. [52]	HHO-KNN	Enhanced ability to think outside the local optimum	Less stability
ALO	Dong et al. [53]	ALO-ARM	The optimized search process has high efficiency	Not suitable for large datasets

.

Inspired by the behavior of insect clusters, herds of animals, flocks of birds, etc., the swarm intelligence algorithm can change the search direction through collaboration and information sharing among individuals of the swarm; it has a faster convergence rate and can obtain a better solution than evolutionary algorithms; the poor performance of individuals does not affect the solution of the problem by the whole swarm.

3.2. Physics-Inspired

A physics-inspired algorithm is a swarm optimization algorithm that models certain physical objective laws. Individuals of the swarm collaborate and exchange information with each other in the search space through rules inspired by some physical principles. Some physically inspired algorithms are shown in

Table 3. ARM, based on the physics-inspired algorithm.

Algorithms	Authors	Technology	Advantages	Disadvantages
SA	Nawaz et al. [54]	HUIM-SA	HUIM-SA varies linearly with the number of iterations	Lack of mutation mechanisms
	Holman et al. [55]	GA-SA, Confusion Matrix	High accuracy	Accuracy depends on dataset size
TS	Chou et al. [56]	QTS	Obtain more rules	Algorithm performance degrades when the dataset is too small
CRO	Abir et al. [57]	CRO	Low algorithm complexity; no need to specify minimum support	Less stability, calculated overload
GSA	Taradeh et al. [58]	HGSA, Crossover, variation	Fast convergence and high-quality rules	Not applicable to mixed datasets

.

3.3. Other Algorithms

Some of the algorithms are shown in

Table 4. ARM, based on other heuristic algorithms.

Algorithms	Authors	Technology	Advantages	Disadvantages
FA	Eva et al. [59]	Combining k-means for web data clustering	Faster convergence; better performance at the minimum distance; shorter computation time	The placement strategy of the initial center of mass needs to be further optimized
MA	Ting et al. [24]	Optimizing the affiliation function in FARM	Improve the search capability of the algorithm and obtain high fuzzy support	Less scalability
SCA	Laith et al. [60]	SCAGA	Maximum classification accuracy with minimal attributes	The imbalance between global search capability and local search capability
BSO	Djenouris et al. [61]	HBSO-TS	Short execution time	Poor results in large datasets
	Ma et al. [62]	PLBSO	Parallelized processing with low computational costs	Less performance when dealing with quantitative rules
	Youcef et al. [63]	GBSO-Miner	Adaptable to large text and graphical databases	Thread divergence exists
EO	Malik et al. [64]	DEOA-CRM	High-quality rules and highly interpretable algorithms	Poor performance in some datasets

.

Other heuristic algorithms are inspired by natural phenomena, mathematical principles, man-made activities, etc. They include the fireworks algorithm, brainstorming algorithm, backtracking search algorithm, and sine and cosine algorithms, etc.

4. Particle Swarm Optimization Algorithms

4.1. Standard PSO Algorithms

The particle swarm optimization (PSO) algorithm [65] is an intelligent optimization algorithm studied by Kenney and Eberhart in 1995 for the swarm motion behavior of bird and fish flocks, and the basic idea is to seek the global optimal solution through mutual collaboration and information sharing among individuals. In this model, an individual is regarded as a particle and a flock of birds as a particle swarm. Suppose there are m particles in a D-dimensional target search space, where the position of the $i$th particle ($i=1,2,3,\dots ,m$) is denoted as $X_i=\left(x_i^1,x_i^2,\cdots ,x_i^D\right)$, i.e., the position of the $i$th particle in the D-dimensional target search space is $X_i^D$. Substituting $X_i$ into the objective function calculates its fitness value and measures its merit. The best position experienced by each particle is denoted as $P_i=\left(p_i^1,p_i^2,\cdots ,p_i^D\right)$, and the best position experienced by the whole particle swarm is denoted as $P_g=\left(p_g^1,p_g^2,\cdots ,p_g^D\right)$. The velocity of particle $i$ is denoted as $V_i=\left(v_i^1,v_i^2,\cdots ,v_i^D\right)$. Particle $i$ updates itself in each iteration by Equations (12) and (13) for $P_i$,$P_g$.

$$v_i^{d+1}=w{v}_i^d+c_1{r}_1\left(p_i^{}-x_i^d\right)+c_2{r}_2\left(p_g^{}-x_i^d\right)$$

(12)

$$x_i^{d+1}=x_i^d+\alpha v_i^d$$

(13)

where $i=1,2,3,\dots ,m$, $d=1,2,3,\dots D$, w is the inertia weight, $v_i^d$ is the velocity of the $i$th particle at the $d$th iteration, $c_1,c_2$ is a nonnegative constant, $r_1,r_2$ is an internally uniformly distributed random number, and $\alpha$ is a controlling factor that controls the velocity weight. The pseudocode is shown in Algorithm 1.

Algorithm 1 Procedure of standard PSO algorithm
1:	for each particle $i=1,2,\dots ,N$ do
2:	(a) Initialize the particle’s position and velocity.
3:	(b) Evaluate the particle’s fitness value;
4:	(c) Update the particle’s Pbest;
5:	(d) Update the swarm’s Gbest;
6:	while termination criteria is not met do
7:	for each particle $i=1,2,\dots ,N$ do
8:	(a) Update particle’s velocity using Equation (12);
9:	(b) Update particle’s position using Equation (13);
10:	(c) Evaluate particle’s fitness value;
11:	if (x[i]) < f(Pbest[i]) then
12:	Update the best known position of particle i: Pbest[i]=x[i];
13:	if (Pbest[i]) < f(Gbest) then
14:	Update the swarm’s best known position: Gbest=Pbest[i];
15:	(d) t = t + 1;
16:	return Gbest

Among them, we illustrate some of the more important parameters.

Maximum velocity $V_{\max }$: In each dimension, the velocity of the particle $v_i\in \left(-V_{\max },V_{\max }\right)$ reduces the possibility of the particle leaving the search space. Where $V_{\max }=k\cdot x_{\max }$$\left(0.1<k<1\right)$, the purpose is to limit the maximum distance the particle can move.

Inertia weight w: The larger the w, the greater the global searchability and the easier it is for the particle to escape the local extremes; however, it also reduces search efficiency and makes converging more difficult, and vice versa. $w$ usually varies linearly ($0<w<1$), and the variation equation is shown in Equation (14).

$$w_i=w_{\max }-\frac{w_{\max }-w_{\min }}{ite{r}_{\max }}\cdot iter$$

(14)

where $ite{r}_{\max }$ is the maximum number of iterations and $iter$ is the number of current iterations.

4.2. Improved PSO Based on Algorithm Design

4.2.1. Swarm Initialization

The main function of swarm initialization is to generate several particles randomly, which has a significant impact on the diversity and convergence of the algorithm. Wang et al. [66] used the roulette wheel selection method and non-expectation coding vector construction strategy to initialize the swarm and select the initial optimized value of the next-generation swarm by a certain probability. Equation (15) represents the probability of itemset I being selected, Equation (16) represents the hetero set of the coding vector and Equations (17)–(19) represent the velocity update in the generated coding vector:

$$S_i=\frac{fitnes{s}_i}{{\displaystyle {\sum }_{k=1}^{\left|SHUI\right|}fitnes{s}_k}}$$

(15)

$$BitDiff(Ve{c}_i,Ve{c}_j)=\left\{num|1\le num\le len,b_{num}(Ve{c}_i)\oplus b_{num}(Ve{c}_j)=1\right\}$$

(16)

$$v_i=v_{i1}+v_{i2}+v_{i3}$$

(17)

$$v_{i2}=\left[\left|BitDiff(p_i,pbes{t}_i)\right|\cdot r_1\right]$$

(18)

$$v_{i3}=\left[\left|BitDiff(p_i,pbes{t}_i)\right|\cdot r_2\right]$$

(19)

where |SHUI| denotes the number of elements in the set SHUI, $Fitnes{s}_i$ denotes the adaptation value of the $i$th itemset, $b_{num}(Ve{c}_i)$ denotes the value of $Ve{c}_i$ in position num, and ⊕ denotes the iso-or operation.$v_{i1}$ defaults to 1.

Kalka et al. [67] introduced the chemical reaction optimization (CRO) algorithm to quickly generate the optimal set of solutions as the initial solution of the PSO algorithm and use the selection operator to improve the quality of the optimal solution and enhance the overall performance of the algorithm. Equation (20) is used to determine whether the generated latest solution is accepted after colliding through two chemical reactions:

$$f(R_j)={\phi }_1\cdot M{S}_R+{\phi }_2\cdot E{C}_{Rj}+{\phi }_3\cdot \epsilon$$

(20)

where ${\phi }_1+{\phi }_2+{\phi }_3=1$.

Hematpour et al. [68] introduced a nonlinear dynamical chaotic system to initialize the swarm positions. Equation (21) represents the first class of Chebyshev polynomials, $\alpha \in (0,2)$, and Equation (22) represents the second class of Chebyshev polynomials, which can be simplified to a logistic mapping, $\alpha \in (\frac{1}{2},\infty )$, and $\alpha \in (\frac{1}{3},3)$ in Equation (23):

$${\mathsf{\Phi }}_2^{(1)}=\frac{{\alpha }^2\cdot {(2x-1)}^2}{4x(1-x)+{\alpha }^2\cdot {(2x-1)}^2}$$

(21)

$${\mathsf{\Phi }}_2^{(2)}=\frac{4\cdot {\alpha }^2\cdot x\cdot (1-x)}{1+4\cdot ({\alpha }^2-1)\cdot x\cdot (1-x)}$$

(22)

$${\mathsf{\Phi }}_2^{(1,2)}=\frac{{\alpha }^2\cdot x\cdot {(4x-3)}^2}{{\alpha }^2\cdot x\cdot {(4x-3)}^2+(1-x)\cdot {(4x-1)}^2}$$

(23)

Li et al. [69] used an S-type transfer function to initialize the swarm position by converting the inertia weights based on feature weighting into probability values. Let $z$ be the variable, Equation (24) represents the inertia weight normalization process, Equation (25) represents the conversion of $z$ to $z\prime$ by a standard logistic function, Equation (26) represents the conversion of $z\prime$ to reduce the upper limit of $z\prime$, and p represents the initial swarm position:

$$z=\frac{2\cdot \sigma \cdot (w-w_{\min })}{w_{\max }-w_{\min }}-\sigma$$

(24)

$$z^{\prime }=\frac{1}{1+e^{-z}}$$

(25)

$$p=\lambda \cdot z^{\prime }$$

(26)

where $p\in (0,\lambda )$, $\lambda \in (0.5,1)$, $z\in [-\sigma ,\sigma ]$, $\sigma \in R^{+}$.

Wang et al. [70] used the principal component analysis (PCA) technique to map high-dimensional data to low-dimensional data, retaining variance information and ranking it by priority, and initializing the mixture by selecting significant features. Equation (27) represents the normalization process of the PCA mean and variance, Equation (28) represents the PCA mapping the original data into new coordinates to maximize the variance, and Equation (29) represents the swarm position obtained after eigendecomposition of the covariance matrix:

$$\{\begin{array}{l}u=\frac{1}{m}\cdot {\displaystyle {\sum }_{i=1}^m{x}^{(i)}}\hfill \\ {\sigma }_j^2=\frac{1}{m}\cdot {\displaystyle {\sum }_{i=1}^m{x}_j^{(i)}}\hfill \end{array}$$

(27)

$$\frac{1}{m}\cdot {{\displaystyle {\sum }_{i=1}^m(x^{(i)}{}^{{}^T}\cdot u)}}^2=\frac{1}{m}\cdot {{\displaystyle {\sum }_{i=1}^m(u^T\cdot x^{(i)}\cdot x^{(i)}{}^{{}^T})}}^2=u^T\cdot (\frac{1}{m}\cdot {\displaystyle {\sum }_{i=1}^m{x}^{(i)}\cdot x^{(i)}{}^{{}^T}})\cdot u$$

(28)

$$y^{(i)}=\left[\begin{array}{c}{u}_1^T\cdot x^{(i)}\\ u_2^T\cdot x^{(i)}\\ ⋮\\ u_k^T\cdot x^{(i)}\end{array}\right]$$

(29)

where $x^{\left(i\right)}$ denotes the position of the $i$th particle in the swarm, $j$ denotes the $j$th dimension of $x^{\left(i\right)}$, u denotes the mean of the $i$th particle, ${\sigma }^j$ denotes the standard deviation of the $j$th dimension, and $y^{(i)}$ denotes the new position generated by the $i$th particle.

Bangyal et al. [71] introduced the Well technique with the help of low deviation sequences to make it applicable to high-dimensional optimization problems. Sobia et al. [72] reviewed a variety of swarm initialization strategies for the PSO algorithms and analyzed and compared each strategy in detail by the QRS method, including the performance of SO-PSO, H-PSO, WE-PSO, etc. in 15 benchmark tests. It was found that KE-PSO performs better in high-dimensional problems.

4.2.2. Algorithm Parameter Optimization

In the PSO algorithms, algorithm parameters play a great role in balancing the global search ability and local searchability, which can effectively improve swarm diversity and speed up algorithm convergence. Common parameter optimization includes inertia weights w, learning factors c1 and c2, the introduction of other factors, etc. Tian et al. [73] adaptively controlled the inertia weights through the fitness function, and the interaction of the two helps to control the swarm diversity. Equation (30) is used to assess swarm diversity, and Equation (31) is used to calculate inertia weights:

$$f_{Div}(t)={\log }_{10}(\frac{1}{m}\cdot {\displaystyle {\sum }_{j=t-m+1}^t{(f_{avg}(j)-f_{Asvg}(t))}^2})$$

(30)

$$w(t)=\{\begin{array}{ll}{w}_{\min }+(w_{\max }-w_{\min })\cdot f_{Aavg}(t)\hfill & , f_{Div}(t)<\alpha \hfill \\ w_{\min }+(w_{\max }-w_{\min })\cdot {(\frac{T-t}{T})}^2\hfill & , f_{Div}(t)\ge \alpha \hfill \end{array}$$

(31)

where $f_{avg}(t)$ denotes the mean value of adaptation in iteration $t$.

Li et al. [74] divided the swarm into three subsets by the fitness value and assigned different inertia weights, and Equation (32) represents the process of dynamically updating the inertia weights by introducing the cloud variation method when the particle fitness value is lower than the average fitness value of the swarm:

$$w=0.9-0.7\cdot e^{-\frac{{(f_i-Ex)}^2}{2{(E{n}^{’})}^2}}$$

(32)

Zhang et al. [75] introduced a multi-stage strategy to improve the inertia weights through the cosine function. Equations (33)–(35) show the improvement process:

$$w_{\cos }(t)=\frac{(w_{ini}+w_{fin})}{2}+\frac{(w_{ini}-w_{fin})}{2}\cdot \cos (\frac{I(t)\cdot \pi }{t_{\max }})$$

(33)

$$I(t+1)=I(t)+a,I(1)=0$$

(34)

$$a=\{\begin{array}{ll}{a}_1\hfill & , \frac{I(t)}{6}\le t_{\max }\hfill \\ a_2\hfill & , \frac{t_{\max }}{6}<I(t)\le \frac{5 \cdot t_{\max }}{6}\hfill \\ a_3\hfill & , \frac{5 \cdot t_{\max }}{6}<I(t)\le t_{\max }\hfill \end{array}, where \frac{1}{6\cdot a_1}+\frac{1}{3\cdot a_2}+\frac{1}{6\cdot a_3}=1$$

(35)

Ankit et al. [76] introduced a mapping function based on binomial probability and adaptively updated the inertia weights. Equations (36) and (37) represent the updating process:

$$w=w_{\min }+(w_{\max }-w_{\min })\cdot {\displaystyle {\sum }_{k=0}^X\left(\begin{array}{l}N\hfill \\ k\hfill \end{array}\right)\cdot p^k\cdot q^{N-k}}$$

(36)

$$X=X(i,t)={\displaystyle {\sum }_{i=1}^{N}I(i,t)}$$

(37)

where $I(i,t)$ denotes the $i$th particle at the time $t$ position function, $X$ denotes the total number of particles passing through the binomial probability improvement position, p denotes the improvement success probability, q denotes the failure probability, the particle swarm is independent of each other, and $p=q=0.5$.

Komarudin et al. [77] combined the multi-dimensional direct fuzzy signature, parametric feedback success counter, and CBPE methods to transform the fuzzy process into aggregation equations for the adaptive updating of inertia weights. Equation (38) represents the structural hierarchy of the inertia weights, Equation (39) represents the best particle to reach the minimization goal, Equation (40) represents the best particle percentage, Equation (41) represents the particle distribution in the search space, and Equation (42) represents the NCBPE normalization process for measuring the best fitness value of the solution:

$$w=\left[\begin{array}{c}{w}_{\max }\\ \left[\begin{array}{c}{w}_{\min }\\ \left[\begin{array}{c}PS\\ \left[\begin{array}{c}diversity\\ di{v}_{\max }\end{array}\right]\\ NCBPE\end{array}\right]\end{array}\right]\end{array}\right]$$

(38)

$$S{C}_i(t)=\{\begin{array}{ll}1\hfill & , f(x_{id}(t))<f(P_{id}(t))\hfill \\ 0\hfill & , else\hfill \end{array}$$

(39)

$$PS(t)=\frac{1}{N}\cdot {\displaystyle {\sum }_{i=1}^{N}S{C}_i(t)}$$

(40)

$$diversity=\frac{1}{N}\cdot {\displaystyle {\sum }_{i=1}^N\sqrt{{\displaystyle {\sum }_{d=1}^D{(x_{id}-g_d)}^2}}}$$

(41)

$$NCBPE=\frac{CBPE-CBP{E}_{\min }}{CBP{E}_{\max }-CBP{E}_{\min }}$$

(42)

Fan et al. [78] analyzed a variety of improved inertia weights and proposed a time-varying adaptive inertia weighting strategy, and Equation (43) represents the update strategy:

$$w(t)=(w_{\max }-w_{\min })\cdot \frac{f{(P_{nbest})}_{avg}}{f{(P_{tbest})}_{avg}}+\frac{w_{\max }\cdot t}{MaxIter}$$

(43)

where $f{(P_{nbest})}_{avg}$ and $f{(P_{tbest})}_{avg}$ denote the average fitness values of the global best position and the local best position in iteration $t$.

Chrouta et al. [79] proposed a factor selection strategy algorithm, FMSPSO, based on particle swarm number selection to adaptively adjust different parameter selection strategies during a swarm search to improve the algorithm’s ability to find global search and avoid premature convergence, which is based on the problem that MSPSO fails to obtain the optimal solution due to premature convergence during the search process. Equation (44) represents the adaptive inertia weight updating strategy, and Equation (45) represents the learning factor updating strategy:

$$w_s=(w_{ini}-w_{end})\cdot \mathrm{ar}\tanh (\delta \cdot {(1-\frac{S}{S_{\max }})}^m)+w_{ini}$$

(44)

$$c=(c_{ini}-c_{end})\cdot \left[1-{\left(\frac{k}{k_{\max }}\right)}^{\frac{s^m}{e^{s^m}}}\right]+c_{ini}$$

(45)

where $w_{ini}=0.4$, $w_{end}=0.9$, $\delta$ and $m$ are control factors, and $s$ is the number of current iterations.

Jeremiah et al. [80] introduced a shrinkage factor in the search process to ensure a smooth convergence of the algorithm to the global optimal solution to resolve the problem of premature convergence of the algorithm. They introduced a gravitational search algorithm to improve the global search ability of the algorithm and enhanced the stagnant particle activity through an adaptive response strategy. Equation (46) represents the shrinkage factor update strategy based on the cosine function, and Equation (47) represents the inertia weight update strategy. Equation (48) represents the relationship between the parameters:

$$k(t)=\frac{\cos (\frac{\pi }{G_{\max }}\cdot T+2.5)}{4}$$

(46)

$$\{\begin{array}{ll}w=w_{end}+(W_{start}-w_{end})\cdot (1-\frac{T}{G_{\max }})\hfill & , if P_{gd}\ne x_{id}\hfill \\ w=w_{end}\hfill & , if P_{gd}=x_{id}\hfill \end{array}$$

(47)

$$w\ge \frac{c_1+c_2}{2}-\frac{1}{k(t)}$$

(48)

4.2.3. Optimal Particle Update

The self-renewal of particles in the PSO algorithm varies with the local optimal position $P_i$ and the global optimal position $P_g$, which has a greater ability to guide the swarm evolution. Li et al. [81] combined the gray wolf optimization (GWO) algorithm with the worst particle for the evolution strategy and the optimal particle perturbation strategy to improve the swarm diversity, and the elite swarm guidance strategy to guide the particle swarm for enveloping the search and enhance the global searchability. The perturbation strategy is shown in Equations (49) and (50):

$$N_{best}(t)=sign(rand-0.5)\cdot e^{(-avg\_r)\cdot G_{best}(t)+G_{best}(t)}$$

(49)

$$G_{best}(t)=\{\begin{array}{ll}{N}_{best}(t)\hfill & , if(fit(N_{best}(t))<fit(G_{best}(t)))\hfill \\ G_{best}(t)\hfill & , otherwise\hfill \end{array}$$

(50)

where $avg\_r$ denotes the average distance of all particles to the center position. $N_{best}(t)$ denotes the position after perturbation.

Keshavamurthy et al. [82] used a stochastic inertia weight strategy to balance the global and local search and adjusted the inertia weights by simulating the annealing probability, and the algorithm gradually approached the optimal solution as the temperature decreased. Equations (51)–(53) represent the inertia weight update strategy:

$$Tt=\frac{f_{avg}^t}{f_{best}^t}-1=\frac{\frac{1}{m}{\displaystyle {\sum }_{i=1}^m{f}_i^t}}{f_{best}^t}-1$$

(51)

$$p=\{\begin{array}{ll}1\hfill & , if \min \underset{1\le i\le m}{f_i^{t-k}}\le \min \underset{1\le i\le m}{f_i^t}\hfill \\ e^{-(\frac{\min \underset{1\le i\le m}{f_i^{t-k}} - \min \underset{1\le i\le m}{f_i^t}}{Tt})}\hfill & , if \min \underset{1\le i\le m}{f_i^{t-k}}>\min \underset{1\le i\le m}{f_i^t}\hfill \end{array}$$

(52)

$$w=\{\begin{array}{ll}{\alpha }_1+\frac{r}{2}\hfill & , p\ge r\hfill \\ {\alpha }_2+\frac{r}{2}\hfill & , p<r\hfill \end{array}$$

(53)

where Tt denotes the cooling temperature, p denotes the simulated annealing probability, $k$ is an integer multiple of $t$, $r$ is a random number within [0,1], α₁, α₂ are constants within [0,1], and ${\alpha }_1>{\alpha }_2$.

Han et al. [83] used the Karush–Kuhn–Tucker proximity measure to determine the contribution of optimally ranked particles to swarm convergence and introduced a gravitational search algorithm and an adaptive reset velocity strategy to enhance swarm diversity and avoid falling into local extremes. Equation (54) calculates the mass of the particle, and Equation (55) calculates the force of the $j$th particle on the $i$th particle to calculate the value of the particle’s contribution to the swarm convergence in Equation (56) to decide whether to guide other particles as the optimal particle. Equations (57) and (58) represent the stalled particle generation process:

$$m_i^t=1-\frac{0.9\cdot fi{t}_i^t}{wors{t}^t}$$

(54)

$$f_{ij}^d(t)=G(t)\cdot \frac{m_i^t+m_j^t}{R_{ij}+\epsilon }\cdot (x_j^d(t)-x_i^d(t))$$

(55)

$$a_i^d(t)=\frac{{\displaystyle {\sum }_{j\in kbest,j\ne 1}ran{d}_j\cdot f_{ij}^d(t)}}{m_i^t}$$

(56)

$$pbes{t}_{i,d}=\mathsf{\Phi }+sdamp\cdot v_{mean}$$

(57)

$$v_{mean}=\frac{1}{N}\cdot {\displaystyle {\sum }_{i=1}^N\sqrt{\frac{1}{D}\cdot {\displaystyle {\sum }_{j=1}^D{(v_{i,d})}^2}}}$$

(58)

where $fi{t}_i^t$ denotes the fitness value of the $i$th particle in the $t$th iteration, $wors{t}^t$ denotes the fitness worst value, $R_{ij}$ denotes the Euclidean distance between particles $i$ and $j$, $G(t)$ denotes the gravitational coefficient, and $ran{d}_j$ is a random number uniformly distributed between [0,1].

Miao et al. [84] sorted each particle in the current swarm by the nonDS function to obtain the kth iteration of the non-dominated particle set and optimally combined it with the kth iteration of the particle set to obtain the current non-dominated particle set $Ar{c}_k$, where the particles choose their personal best position, $Pbest$, in $Ar{c}_k$ according to the Euclidean distance. This effectively improves the quality of $P_{best}$ and prevents the algorithm from falling into local extremes. Equation (59) calculates the degree of influence of the optimal particles, and Equation (60) calculates Att, the size of which is proportional to the degree of influence:

$$dist(x,y)=\{\begin{array}{ll}Att\hfill & , if x=y\hfill \\ -Att\hfill & , otherwise\hfill \end{array}$$

(59)

$$Att=At{t}_{\max }-(At{t}_{\max }-At{t}_{\min })\cdot {(1-\frac{gen}{G_{\max }})}^5$$

(60)

where $gen$ denotes the current iteration number and $G_{\max }$ denotes the maximum iteration number.

Zhu et al. [85] show that, based on the elastic collision principle and the law of momentum conservation, the worst particle of the swarm fitness value collides with the best particle of the fitness, and the worst particle obtains the best particle velocity. Equations (61) and (62) represent the calculation of the worst particle velocity, Equation (63) represents the particle velocity update strategy, and Equation (64) represents the particle position update strategy:

$$v_1=Z\cdot e^{-{(x_{worst}-x_{rand})}^2}$$

(61)

$$Z=\frac{1}{2}\cdot ({(1-2\cdot \frac{t}{T})}^3+1)$$

(62)

$$\{\begin{array}{l}{m}_1{v}_1=m_1{v}_1^{\prime }+m_2{v}_2^{\prime }\hfill \\ \frac{1}{2}\cdot m_1{v}_1^2=\frac{1}{2}\cdot m_1{v}_1^{\prime 2}+\frac{1}{2}\cdot m_2{v}_2^{\prime 2}\hfill \end{array}\Rightarrow \{\begin{array}{l}{v}_1^{\prime }=\frac{m_1 - m_2}{m_1 + m_2}\cdot v_1\hfill \\ v_2^{\prime }=\frac{2 \cdot m_1}{m_1 + m_2}\cdot v_2\hfill \end{array}$$

(63)

$$\{\begin{array}{l}{X}_{worst}^{\prime }=X_{worst}+v_1^{\prime }\hfill \\ X_{best}^{\prime }=X_{best}+v_2^{\prime }\hfill \end{array}$$

(64)

where $X_{worst}$ denotes the particle with the worst adaptation value, $X_{rand}$ denotes the random particle position, $T$ denotes the nonlinear factor, $m_1$ denotes the adaptation value of the worst particle, and $m_2$ denotes the adaptation value of the best particle.

Fu et al. [86] introduced the RBF model to generate the global optimum and individual optimum through MATLAB’s fmincon toolbox. While the existence of a large number of uncertain particles in the search space easily leads to local convergence of the algorithm, the authors also introduced the MaxEI method for pre-screening to measure the particle potential and uncertainty; thus, the global optimum has both maximum potential and maximum uncertainty.

4.2.4. Speed and Position Update

In the particle swarm optimization algorithms, velocity and position are properties specific to particles. Song et al. [87] combined successive higher Bessel curves and adaptive fractional-order velocities with particle evolution states to apply perturbations to the swarm to enhance its ability to jump out of local constraints. Equation (65) denotes the average distance of the $i$th particle from $g_{best}$ in the $k$ iterations, Equation (66) denotes the kth iteration and the swarm evolution state, Equation (67) denotes the swarm evolution adaptive adjustment factor, and Equations (68) and (69) denote the particle velocity and position update formulas:

$$d_i^k=\frac{1}{S-1}{\displaystyle {\sum }_{j=1,j\ne i}^S\sqrt{{{\displaystyle {\sum }_{k=1}^D(x_i^k-x_j^k)}}^2}}$$

(65)

$$E_f^k=\frac{d_{gb}^k-d_{\min }^k}{d_{\max }^k-d_{\min }^k}$$

(66)

$$\alpha =0.9-\frac{1}{1+e^{-E_f^k}}\cdot \frac{k}{k_{\max }}$$

(67)

$$\begin{array}{l}{v}_i^{k+1}=\alpha \cdot v_i^k+\frac{1}{2}\cdot \alpha \cdot (1-\alpha )\cdot v_i^{k-1}+\frac{1}{6}\cdot \alpha \cdot (1-\alpha )\cdot (2-\alpha )\cdot v_i^{k-2}\\ +\frac{1}{24}\cdot \alpha \cdot (1-\alpha )\cdot (2-\alpha )\cdot (3-\alpha )\cdot v_i^{k-3}\\ +c_1\cdot r_1\cdot (x_{ib}^k-x_i^k)+c_2\cdot r_2\cdot (x_{gb}^k-x_i^k)\end{array}$$

(68)

$$x_i^{k+1}=x_i^k+v_i^{k+1}$$

(69)

where $d_{\max }^k,d_{\min }^k$ denotes the maximum and minimum values of the distance from a particle to other particles in the swarm, and $d_{gb}^k$ denotes the average distance of the $g_{best}$.

Tian et al. [73] introduced the particle with the best comprehensive evaluation of the swarm fitness value and diversity in the current iteration as the latest guiding particle to guide the swarm evolution, together with the local and global optimum. Equation (70) represents the comprehensive evaluation of the $i$th particle, and Equation (71) represents the velocity update under the latest guiding particle:

$$f_{Com}(i)=d_1\cdot f_{Fit}(i)+d_2\cdot f_{Dis}(i)$$

(70)

$$v_i^d(t+1)=w\cdot v_i^d(t+1)+c_1\cdot r_1\cdot (p_i^d-x_i^d(t))+c_2\cdot r_2\cdot (g_{}^d-x_i^d(t))+c_3\cdot r_3\cdot (f_{Com}^d(i)-x_i^d(t))$$

(71)

where $d_1=d_2=0.5$ and $d_1+d_2=1$, $f_{Fit}(i)$ denotes the $i$th particle fitness value, and $f_{Dis}(i)$ denotes the $i$th particle diversity evaluation, i.e., the distance to the central particle $c_1=c_2=c_3=1.49445$.

Mohamed et al. [88] and Suman et al. [89] introduced the GWO algorithm to improve the global search ability of the algorithm. Equation (72) represents $\alpha ,\beta ,\delta$ the distance between the three wolves and other individuals, Equation (73) represents the current position of the three wolves, and Equation (74) represents the current gray wolf position:

$$\left\{\begin{array}{l}\overrightarrow{D_{\alpha }}=\left|\overrightarrow{C_1}\cdot \overrightarrow{X_{\alpha }}-\overrightarrow{X}\right|\hfill \\ \overrightarrow{D_{\beta }}=\left|\overrightarrow{C_2}\cdot \overrightarrow{X_{\beta }}-\overrightarrow{X}\right|\hfill \\ \overrightarrow{D_{\delta }}=\left|\overrightarrow{C_3}\cdot \overrightarrow{X_{\delta }}-\overrightarrow{X}\right|\hfill \end{array}\right\}$$

(72)

$$\left\{\begin{array}{l}\overrightarrow{X_1}=\left|\overrightarrow{X_{\alpha }}-\overrightarrow{A_1}\cdot \left(\overrightarrow{D_{\alpha }}\right)\right|\hfill \\ \overrightarrow{X_2}=\left|\overrightarrow{X_{\beta }}-\overrightarrow{A_2}\cdot \left(\overrightarrow{D_{\beta }}\right)\right|\hfill \\ \overrightarrow{X_3}=\left|\overrightarrow{X_{\delta }}-\overrightarrow{A_3}\cdot \left(\overrightarrow{D_{\delta }}\right)\right|\hfill \end{array}\right\}$$

(73)

$$\overrightarrow{X}(t+1)=\frac{\overrightarrow{X_1}+\overrightarrow{X_2}+\overrightarrow{X_3}}{3}$$

(74)

where $\overrightarrow{A}=2\cdot a\cdot r_1-a$, $\overrightarrow{C}=2\cdot r_2$, and a is a convergence factor that decreases from 2 to 0 with iteration and $r_1,r_2$ are random numbers within [0,1].

Zhao et al. [84] used Euclidean distance to quantify the leading effect of the current optimal particle on evolution and to update the particle velocity and position. Equations (75) and (76) represent the velocity update process, and Equation (77) represents the particle position update probability:

$$V_{i(p,q)}^{k+1}=W\cdot V_{i(p,q)}^k+C_1\cdot r_1\cdot dist(Pbes{t}_i^k(p),q)+C_2\cdot r_2\cdot dist(Gbes{t}_{}^k(p),q)$$

(75)

$$V_{i(p,q)}^{k+1}=\{\begin{array}{ll}{V}_{\max }\cdot \frac{V_{i(p,q)}^{k+1}}{\left|V_{i(p,q)}^{k+1}\right|}\hfill & , if\left|V_{i(p,q)}^{k+1}\right|\ge V_{\max }\hfill \\ V_{i(p,q)}^{k+1}\hfill & , otherwise\hfill \end{array}$$

(76)

$$P^{k+1}(x_{i,p}^{k+1}=q)=\frac{sigmoid(V_{i(p,q)}^{k+1})}{{\displaystyle {\sum }_{l=1}^{M}sigmoid(V_{i(p,l)}^{k+1})}}$$

(77)

where dist() denotes the distance between the global optimum and the individual optimum for evaluating the particle’s importance.

Xu et al. [90] introduced variation, crossover, and selection operations of differential evolution to enhance the information exchange among swarms: Equation (78) for the variation factor, Equation (79) for the crossover operation, Equation (80) for the variation operation, and Equation (81) for the particle velocity update:

$$x_i^{son}=x_{ra}+0.5\cdot 2^{e^{1-\frac{it\max }{it\max +1-n}}}\cdot (x_{rb}-x_{rc})(i=1,2,\cdots ,N)$$

(78)

$$u_{i,d}^{son}=\{\begin{array}{ll}{x}_{i,d}^{son}\hfill & , if(rand<CR) or (d=d_{rand})\hfill \\ x_{i,d}^{}\hfill & , otherwise\hfill \end{array}$$

(79)

$$x_i^{son}=\{\begin{array}{ll}{u}_i^{son}\hfill & , if(f(u_i^{son})<f(x_i))\hfill \\ x_i\hfill & , otherwise\hfill \end{array}$$

(80)

$$v_{id}^{n+1}=w\cdot v_{id}^n+c_1\cdot r_1\cdot (P_{id}^n-x_{id}^n)+c_2\cdot r_2\cdot (P_{gd}^n-x_{id}^n)+c_3\cdot r_3\cdot (P_{cd}^n-x_{id}^n)$$

(81)

where ra, rb, rc, and $d_{rand}$ denote the random numbers within $[0,N]$, CR denotes the crossover rate and $P_{cd}^n$ denotes the best communication location obtained after the variation, crossover, and selection.

Su et al. [91] introduced a time factor to overcome the problem that causes the traditional FOPSO algorithm to easily fall into local extremes, which reduces the overdependence of the algorithm on the fractional-order α. The time factor, T, can change adaptively with the swarm parameters, and when T is large, the algorithm has a strong global search capability and helps to obtain the global optimum, and vice versa. Equation (82) represents the time factor change process, and Equation (83) represents the position update process of the improved algorithm:

$$T=\frac{1}{2}\cdot 2^{e^{-\frac{a_t}{1-a_t}}}, where a_t=\frac{\min (f_{gd},f_t)}{\max (f_{gd},f_t)}$$

(82)

$$x_{id}(t+1)=x_{id}(t)+v_{id}(t)\cdot T$$

(83)

where $a_t$ denotes the swarm diversity, $f_{gd}$ denotes the global optimum, and $f_t$ denotes the average fitness value in iteration $t$.

5. Association Rule Mining Based on PSO

5.1. Algorithm Principle

5.1.1. ARM Process

The particle swarm optimization (PSO) algorithm for ARM mainly consists of two stages: preprocessing and rule mining. Its basic flow is shown in Figure 3.

The preprocessing stage mainly completes the binary data conversion. The rule mining stage is the process of an iterative search and the continuous updating of particle positions and velocities in the search space by the PSO algorithm after data encoding and initialization of the swarm to finally obtain the best rules.

5.1.2. Binary Conversion

The standard PSO algorithm is mainly used for continuous real-valued optimization problems, while in real life, it is mostly used for discrete or combinatorial optimization problems. To address this issue, Kennedy and Eberhart proposed the discrete binary particle swarm optimization algorithms (BPSO) in 1997, which use the sigmoid() function to limit the position and velocity to the {0,1} interval, and where the particle position is calculated iteratively, as shown in Equation (84).

$$x_i^{d+1}=\{\begin{array}{cc}1& , r<sigmoid(v_i^{d+1});\\ 0& , r\ge sigmoid(v_i^{d+1});\end{array}$$

(84)

where $r$ is a random number between (0,1) and $sigmoid\left(v_i^{d+1}\right)=\frac{1}{1+e^{-v_i^{d+1}}}$.

The process of binary conversion of transaction data by the above means is shown in Figure 4.

Among them, the original database has five transaction records, which are $T_1,T_2,\cdots ,T_5$, and these five transaction records are converted into binary form and stored. Take $T_5$ as an example, where the transaction record contains $I_1,I_3,I_4$, so its corresponding units are all 1, and the other units are 0. The converted transaction record is 10110.

5.1.3. Encoding

There are two main ways to encode data: the Michigan approach and the Pittsburgh approach. Among them, the Pittsburgh approach corresponds to a set of rules per chromosome and is suitable for solving classification problems. The Michigan approach corresponds to a single rule per chromosome and is more straightforward than the former, with a shorter rule syntax. Let us take the Michigan method as an example: Let there be N data items in the dataset, and each data item has two parts, and each part has two possible values that take the value of 1 or 0, as shown in Figure 5.

The dimension of each particle is 2N, and the data item X refers to whether the item is included in the rule. Y refers to whether the item is included in the result of the rule. For example, if the value $I_i$ is 11, then the $i$th item appears in the rule and is included in the result of the rule. The position representation of the rule is shown in Figure 6.

5.1.4. Search for the Best Particle

First, we need to define a fitness function for evaluating the importance of each particle. By using Equation (84) as the objective function:

$$Fitness(i)=Confidence(i)+Support(i)$$

(85)

where $Confidence(i)$ and $Support(i)$ denote the confidence and support of the ith particle, respectively. The objective of this function is to maximize the confidence and support of the particle; the stronger this support, the stronger the strength of the association, implying that this is an important association rule.

Second, we also need to generate an initial population, and the particles in the population are called initial particles. Finally, we need to set boundary conditions to constrain the range of motion of the particles and obtain the global best particle when $Gbest$ and $Pbest$ are iterated until the termination condition is reached. The boundary conditions include calculating the Euclidean distance of the particles, etc. The termination conditions can be set by setting a fixed number of iterations.

5.2. Application

In dealing with complex, realistic optimization problems, heuristic algorithms are inspired by nature and modeled by simulating the behavior of individuals or swarms, which have the advantage of not relying on specific problems, while such algorithms are not constrained by the form of optimization functions in the search process and are highly adaptable, efficient, and easy to implement. Compared with other algorithms, heuristic algorithms show obvious advantages in solving ARM problems. Among the heuristic algorithms, the genetic algorithm (GA) has a long convergence time due to the lack of individual memory and blind search without direction, and it shows inefficiency and difficulty in convergence in high-dimensional data. The ant colony optimization (ACO) algorithm uses pheromone to decide the search direction, which consumes a lot of time for a solution construction and also has the problem of a long convergence time and is prone to stagnation in high-dimensional data. The particle swarm optimization (PSO) algorithm is efficient and simple to implement, converges quickly, and can adjust the search direction by iteration, which is significantly better than other heuristics in ARM problems [92]. PSO algorithms are widely used in ARM, mainly in the fields of privacy protection, recommender systems, risk assessment, and medicine. The correlation is shown in Figure 7.

5.2.1. Privacy Protection

The rapid growth of the Web is accompanied by a dramatic increase in data, and more and more companies or organizations are sharing information by sharing data to gain more profits. To protect the sensitive information implicit in the data from leakage, users need to modify the individual numerical information of the database while sharing the data without compromising the validity of the data. Kalyani et al. [93] proposed an IFHARM algorithm based on PPARM, which includes identifying SAR, clustering SAR by similarity metric, identifying sensitive transactions, removing and rescanning the converted database, and performing error rule verification. Among them, the authors use the result of each item deletion on non-sensitive data as its impact factor to identify SAR to minimize the loss of non-sensitive association rules and ensure the high utility of the data. Jangra et al. [94] proposed the VIDPSO algorithm to solve the rapid performance degradation of evolutionary algorithms in dense datasets, which prioritizes the presence of more sensitive patterns in data and identifies SAR based on its sensitive patterns to avoid the failure of sensitive information hiding due to the transformation of sensitive frequent itemsets into infrequent itemsets by removing too much data. Suma et al. [95] developed a new method by integrating fractional calculus (FC) and the salp swarm algorithm (SSA) for data sanitization using privacy-preserved data. Krisgnamoorthy et al. [96] used different methods to perturb the dataset to generate a large number of solutions without affecting the validity of the data, which were then sequentially optimized by the PSO algorithm until the support was lower than the minimum support, at which point, the loop was closed, effectively reducing the number of modifications and avoiding the generation of spurious rules.

5.2.2. Recommender Systems

Analyzing consumers’ previous behaviors can provide users with personalized products or valuable information, and recommendation systems attempt to recommend goods to potential customers by applying data mining techniques. Among them, the shopping basket problem is a typical example of ARM, and Guo et al. [97] proposed an improved PSO algorithm based on the gray model for dynamic ARM of supermarket shopping data. Furthermore, since the introduction of the background values of the gray model tends to lead to a decrease in prediction accuracy, the authors introduced a secondary search mechanism based on the PSO algorithm to optimally solve the background values of the gray model at different moments, which significantly improves the local search ability of the algorithm and avoids premature particle convergence. To obtain rich product manufacturing knowledge from enterprise historical data to reflect the mapping relationship between a product’s design and manufacturing, Kou [98] proposed an ARM method based on a binary PSO algorithm, which encodes association rules and generates a certain number of particle swarms, obtains N rules through multiple iterations, integrates multiple indicators, such as reliability, relevance, and comprehension, and adds a similarity function to compare rules two by two, while eliminating the poor rules to achieve efficient rule mining.

Web data often implies a large amount of web search information. Kaur et al. [99] proposed a hybrid algorithm, HPSO-TS-ARM, to obtain the potential user information among them, which obtains web data through the PSO algorithm, arranges data information by the TAO algorithm, and then performs ARM. Its computation time and efficiency are better than other algorithms, and it has been validated in several datasets. Gangurde et al. [100] used the Weblog features related to Web mining to construct a Web recommendation model, introduced a PSO algorithm, and used a feedforward counter model to obtain association rules using a single data iteration technique, which was used as an adaptation value to execute the recommendation model, and the execution time was significantly reduced. Tyagi et al. [41] solved the low accuracy and data sparsity problems, considering support and confidence as the objectives for proposing the MOPSO algorithm. The algorithm can effectively filter out rules with significant association relationships, and its accuracy and precision are also better than other algorithms when the data is sparse. Jiang et al. [101] proposed an association rule method for generating sentiment designs based on online customer reviews for a large number of online user reviews on major websites by mining the sentiment dimension opinion of user reviews. This method can mine more reliable and interesting rules than the traditional PSO algorithm by performing sentiment analysis on customer reviews to achieve accurate user sentiment recommendations.

5.2.3. Risk Assessment

Credit risk management is one of the most important concerns of financial companies, including the collection of relevant information about borrowers, which is used to decide whether to approve their credit applications. Lanzarini et al. [102] proposed the LVQ-PSO algorithm by combining competitive neural networks with the PSO algorithm, which mines association rules by preprocessing the original data with the LVQ method and binary PSO algorithm. The simplified rules can assist the decision-maker in deciding whether to grant a loan or not. Priya et al. [103] proposed a novel PSO-based fuzzy associative classifier (PaSOFuAC) to detect phishing websites and avoid financial losses caused by users not recognizing phishing websites.

As the most convenient means of transportation, the flight process of aircraft is highly susceptible to environmental, equipment, and other complex factors, and the number of hazards increases. Li et al. [104] designed a weighted multiple swarm PSO algorithm for hazard source analyses based on the background of civil aviation safety risk assessments. The preprocessing stage assigns weights to items in the hazard source database and defines the candidate association rules for hazard sources; the analysis stage introduces the swarm interaction mechanism and uses a parallel search mode to generate the association rules for hazard sources and their generation causes, which provides important theoretical support for hazard source identification and prediction. The multiple swarms’ parallel search mode provides an interaction mechanism between the swarms to avoid swarms falling into local extremes due to the lack of information exchanged. Inspired by an online sequence learning machine, She et al. [105] used a single hidden layer feedforward neural network to train hazard source data and update the output weights of the learning machine in real-time to realize the online identification of hazard sources. The paper discards the traditional ARM algorithm and uses the ACO-ARM algorithm to mine the most frequent term and the PSO algorithm to initialize the pheromone concentration, which effectively avoids the algorithm’s initialization error and improves the system’s response speed.

5.2.4. Medical Field

In the medical field, a pathological diagnosis is mostly derived from physicians’ treatment experience due to the lack of quantitative criteria. ARM can extract features from multiple factors, classify and identify them, and extract the hidden information behind them from medical records to provide physicians with accurate scientific opinions. To improve the applicability and classification accuracy of the algorithm in processing medical data, Ripon et al. [106] combined the PSO algorithm with an artificial neural network to automatically analyze the dataset to achieve an accurate classification of tuberous sclerosis by ARM. To overcome the problem of excessive data dimensionality, Choubey et al. [107] introduced the PCA-PSO algorithm for a feature reduction of raw data to significantly reduce the computation time while ensuring prediction classification accuracy.

In terms of disease prediction, Karsidani et al. [108] proposed ANFIS-PSO to predict whether a patient has coronary artery disease effectively. To deal with the continuous and rapid growth of data volume problems, Mangat et al. [109] proposed a dynamic particle swarm-based rule mining classifier to achieve adaptive prediction of the expected life expectancy of patients with thoracic diseases by dynamically creating swarms and updating the algorithm parameters. Raja et al. [110] proposed a data mining strategy-based prediction model for type II diabetes mellitus by combining the PSO algorithm and fuzzy clustering mean with medical metrics, such as sensitivity, specificity, and accuracy, which are widely used in medicine to assess the performance of the system, and found that the prototype was 8.26% more accurate than traditional methods.

To address the existing problems of inefficient and time-consuming diagnoses of heart diseases, Alkeshuosh et al. [111] used a standard PSO algorithm to obtain heart disease rules from the original dataset and optimized them to generate the best rules. Mao Jie et al. [112] proposed the PSO-SVM algorithm based on support vector machines (SVM) for heart disease predisposing factor detection, which uses ARM to extract the disease features and train the feature dataset. Each PSO-SVM classifier performs the corresponding fitness function, formulated by a classified/total, to iteratively update the velocity and position of each particle until it obtains the global best value. The algorithm is tested on the UCI Cleveland dataset using confidence levels as an indicator and has better classification performance compared to traditional PSO algorithms.

6. Conclusions and Challenges

As an emerging field in ARM, PSO algorithms are becoming more mature through continuous exploration and research by many scholars and are widely used in privacy protection, recommender systems, risk assessment, and medicine. This section shows a comparative analysis of the different PSO algorithms, and the objective analysis of existing methods is shown in

Table 5. An objective analysis of the PSO algorithm.

Objective	Thesis
Convergence	[66,67,73,75,79,80,82,83,86,87,91,113]
Versatility	[66,68,73,81,83,84,87,90,113,114,115,116,117]
High-dimensional data	[70,71,72,115,116,117]
Result accuracy	[46,73,81,84,116,117]
Balance exploration and utilization	[67,69,74,77,78,80,82,85,86,114]

.

The advantages of the particle swarm optimization (PSO) algorithm over other algorithms are: (1) It has uncertainty, which is significantly better than the deterministic algorithms in solving certain aspects of a problem and is more advantageous in finding the global optimal solution; (2) it has better adaptability—the PSO algorithm is a bionic optimization algorithm based on multiple intelligences and is highly robust; (3) it is evolutionary and can preserve the optimal solution information in each iteration round.

The above research shows that the PSO algorithm still has great development space and application value in ARM, but with the expansion of data scale, the difficulty of mining effective association rules from large-scale data using the PSO algorithm gradually increases, and this field will still face great challenges in future research.

7. PSO Future Prospects

The future research directions of particle swarm optimization algorithms may be as follows:

(1)

PSO algorithms converge prematurely and are prone to falling into local extremes, which mainly lie in the imbalance between global search and local search capabilities. Nevertheless, many improvement algorithms have been proposed, such as the QPSO algorithm based on the cloud model, the adaptive multi-objective PSO algorithm based on Gaussian mixed variance and elite learning, etc. However, the two are relatively contradictory, and how to dynamically maintain the balance between them in the search process to obtain the optimal solution based on the actual search results or how to measure that the two have reached the best balance needs further study.

(2)

Due to the exponential growth of high-dimensional data search space and low data relevance, search efficiency decreases rapidly. In addition to the existing “divide and conquer” method and the introduction of other algorithms, the use of feature extraction methods to remove redundant data and natural computational methods, such as nonlinear dimensionality reduction, can also be introduced to reduce redundancy using the maximum linear irrelevance group. Although significant progress has been made in handling large amounts of data, there is still much room for exploration in the ARM models for handling high-speed data.

(3)

Inertia weights and learning factors are important parameters in PSO algorithms. Improvements to inertia weights include linearly decreasing inertia weights, fuzzy inertia weights, random inertia weights, etc. Improvements in learning factors include shrinkage factors, synchronous learning factors, asynchronous learning factors, etc. In the future, we can continue to optimize these two parameters and consider whether the two affect each other and their mutual influence weights.

(4)

Many scholars focus more on the swarm initialization problem, including the M-class random method, fixed-value setting method, two-step method, hybrid method, etc. However, the method is too simple in how to divide the swarm size and lacks a scientific and reasonable effective division, which will limit the speed of particle movement when the swarm is too small and, thus, lead to early convergence. When the swarm is too large, it leads to a too large search space and reduces the performance and efficiency of the algorithm.

(5)

When there are missing or wrong datasets, there is difficulty in guaranteeing the PSO algorithm’s accuracy and performance without artificial preprocessing, i.e., having certain fault tolerance.