ELPSO-C: A Clustering-Based Strategy for Dimension-Wise Diversity Control in Enhanced Leader Particle Swarm Optimization

Abstract

In high-dimensional optimization, particle swarm optimization (PSO) algorithms often suffer from premature convergence due to stagnation in certain dimensions. This study proposes an enhanced variant, ELPSO-C, which integrates dimension-wise convergence detection with adaptive exploration mechanisms. By applying agglomerative clustering to inter-particle velocity diversity, ELPSO-C identifies dimensions showing signs of stagnation and selectively reintroduces diversity through targeted mutation strategies. The algorithm preserves global search capability while reducing unnecessary perturbation in well-explored dimensions. Experimental results on a suite of 18 benchmark functions across various dimensions demonstrate that ELPSO-C consistently achieves superior performance compared to existing PSO variants, especially in high-dimensional and complex landscapes. These findings suggest that dimension-aware adaptation is an effective strategy for improving PSO’s robustness and convergence quality.

1. Introduction

Particle swarm optimization (PSO) was originally proposed by Kennedy and Eberhart as a stochastic optimization algorithm inspired by the collective behavior observed in bird flocking and fish schooling [1,2]. Due to its ability to balance exploration and exploitation using simple mathematical operations, PSO has become one of the most widely used metaheuristic algorithms in computational intelligence [3]. Its advantages include high search efficiency, ease of implementation, and few control parameters, making it particularly suitable for a wide range of practical applications. These applications span various fields such as machine learning [4], control systems [5], electric power systems [6], and financial engineering [7]. As a result, PSO has been extensively studied and continuously improved over the past decades.

One of the most critical issues in PSO is its tendency to prematurely converge to local optima, especially when dealing with complex multimodal objective functions [7,8]. In such scenarios, particles often lose diversity and become trapped in limited regions of the search space, preventing the algorithm from sufficiently exploring other promising areas. This phenomenon arises because the social and cognitive components in the PSO update equations tend to rapidly align particle trajectories, leading to loss of exploration capability as the search progresses [9]. As a result, maintaining adequate diversity within the swarm is essential for achieving global search performance and avoiding stagnation [10].

To address the limitations of standard PSO in handling local optima, the Enhanced Leader Particle Swarm Optimization (ELPSO) algorithm was proposed [11]. ELPSO enhances the leader particles by applying a multi-stage mutation strategy, thereby increasing the algorithm’s ability to escape from local minima. Specifically, it adopts diverse mutation mechanisms depending on the current stage of convergence, such as Gaussian, Cauchy, and Lévy-based perturbations, to reinforce the global exploration capability of the best-performing individuals. Although ELPSO improves search accuracy and robustness through leader enhancement, it does not explicitly control the diversity of the entire swarm. As a result, convergence may still occur prematurely when diversity loss propagates among non-leader particles.

To overcome the limitations of ELPSO, this study proposes a novel method called ELPSO-C, which introduces a clustering-based diversity control mechanism at the dimension level. The core idea of ELPSO-C is to monitor and manage the diversity of the swarm on a per-dimension basis, rather than treating the swarm as a whole. By applying clustering to the particles’ distribution in each dimension, ELPSO-C identifies which dimensions are undergoing premature convergence, characterized by reduced diversity. Adaptive operations are then applied selectively to those dimensions in order to restore and maintain diversity. By integrating the leader-enhancing mutation mechanisms of ELPSO with dimension-wise diversity control over the entire swarm, ELPSO-C achieves improved balance between global exploration and local exploitation, thus enhancing overall optimization performance.

The remainder of this paper is organized as follows. Section 2 provides a review of related work in the field of particle swarm optimization and diversity control strategies. Section 3 describes the proposed method, ELPSO-C, in detail, including its clustering-based mechanism and dimension-wise adaptation. Section 4 presents numerical experiments conducted to evaluate the performance of the proposed method using benchmark functions. Finally, Section 5 concludes the paper with a summary of findings and future research directions.

2. Related Work

Particle swarm optimization (PSO), originally proposed by Kennedy and Eberhart [1,2], is a stochastic, population-based optimization algorithm inspired by the collective behavior of biological swarms. Owing to its simplicity, computational efficiency, and few control parameters, PSO has emerged as one of the most widely adopted metaheuristic algorithms. It has been successfully applied in various domains, including machine learning, control systems, electric power systems, and engineering design [3,5,6,7].

A major limitation of standard PSO lies in its tendency to prematurely converge in complex multimodal landscapes, where particles lose diversity and become trapped in local optima [8,10]. The velocity and position of each particle are updated at each iteration according to the following equations:

$$v_i(t+1)=\omega v_i(t)+c_1{r}_1(pbes{t}_i-x_i(t))+c_2{r}_2(gbest-x_i(t)),$$

(1)

$$x_i(t+1)=x_i(t)+v_i(t+1),$$

(2)

where $\omega$ denotes the inertia weight, $c_1$ and $c_2$ are the acceleration coefficients, and $r_1$, $r_2$ are random variables uniformly drawn from $[0,1]$. These equations demonstrate how particle movements are guided by their personal best and the global best positions, potentially resulting in a rapid loss of diversity as the search progresses.

In addition to standard PSO and ELPSO frameworks, various studies have explored leader-based or cooperative enhancements to PSO. For example, Li et al. [12] proposed a multi-component PSO algorithm with a leader learning mechanism that combined four PSO strategies in a cooperative strategy pool to improve structural damage detection accuracy at the expense of a higher computational effort. Chen et al. [13] introduced ALC-PSO, where the global-best leader was assigned a lifespan and replaced by challengers as it ages, helping to prevent premature convergence and maintain swarm diversity. Wang et al. [14] applied PSO as an optimization tool for tuning gain parameters in leader–follower cooperative control laws of multi-agent systems, demonstrating PSO’s utility in distributed control design.

Several recent PSO variants have adopted Gaussian-based strategies to enhance swarm diversity and mitigate premature convergence. For instance, Chen et al. [15] proposed an Adaptive PSO with Gaussian Perturbation and Mutation (AGMPSO), where Gaussian noise was injected into the global best particle when stagnation was detected. Wang et al. [16] introduced an Adaptive Dimensional Gaussian Mutation PSO (ADGMPSO), which applied Gaussian perturbation to elite particles on a per-dimension basis with adaptive control. Han et al. [17] developed a Group Learning PSO (GLPSO) that incorporated adaptive Gaussian noise into particle velocities to promote exploration. While these methods apply Gaussian perturbations globally or at the level of selected particles, they do not explicitly identify stagnated dimensions across the swarm. In contrast, our approach performs a clustering-based analysis of particle positions to detect stagnated dimensions and selectively reset their values using a targeted diversity control strategy. This dimension-wise mechanism enables more precise intervention, which is particularly beneficial in high-dimensional optimization problems.

To overcome this issue, various improvements have been proposed. Enhanced Leader PSO (ELPSO) [11] applies multi-stage mutation strategies—such as Gaussian, Cauchy, and opposition-based mutations—to enhance the exploration ability of leader particles. While ELPSO increases robustness against local optima, it does not explicitly manage diversity at the dimension level. Clustering techniques have also been integrated into swarm and evolutionary computation frameworks to support adaptive diversity control and structure-aware search [4,18]. In particular, hierarchical agglomerative clustering using complete linkage helps to identify structural patterns in the search space and to guide localized adaptation. Building on these concepts, the proposed ELPSO-C combines leader-enhancing mutation mechanisms with clustering-based dimension-wise control to improve search diversity, particularly in high-dimensional and complex optimization problems.

In addition to clustering-based approaches, recent studies have applied domination landscape concepts to better understand and exploit the structure of local optima in swarm and evolutionary optimization. Hao et al. [19] first proposed the domination landscape (DL) as a directed graph of solutions ordered by dominance relationships, providing a hierarchical view of optima in the search space. Building on this idea, later studies have applied DL analysis in swarm algorithms to preserve solution diversity and avoid premature convergence. For example, Li et al. [20] employ a DL-based matrix to compare successive fitness landscapes in dynamic environments, allowing their evolutionary swarm to detect shifts in the distribution of optima and reuse or diversify solutions accordingly. Similarly, Hou et al. [21] incorporate DL theory into particle swarm optimization, using dominance relations to guide particles towards multiple high-quality peaks rather than a single dominant solution, thereby maintaining diversity and reducing the risk of entrapment in local optima. These approaches show that modeling the fitness landscape as a dominance hierarchy helps characterize the landscape’s multi-optima structure and inform strategies (like diversity preservation and knowledge transfer) that align with the goals of the current study.

Other well-known swarm intelligence algorithms such as Artificial Bee Colony (ABC), Gray Wolf Optimizer (GWO), and Tree-Seed Algorithm (TSA) have also been proposed and widely applied to various continuous optimization problems [22,23,24]. ABC is inspired by the foraging behavior of honey bees and balances exploration and exploitation through the roles of employed and onlooker bees. GWO mimics the hierarchical leadership and hunting strategies of gray wolves, effectively balancing local and global search. TSA models the propagation behavior of tree seeds, combining random and directional steps to switch between different search modes. While these algorithms have shown strong performance on many problems, our proposed methods are built on the PSO framework with a specific focus on dimension-wise diversity control and adaptive intervention strategies.

2.1. Enhanced Leader Particle Swarm Optimization (ELPSO)

ELPSO is a variant of the standard PSO algorithm designed to mitigate premature convergence. It enhances global search capabilities by incorporating multi-stage mutation strategies into the behavior of leader particles, thereby promoting swarm diversity. This section outlines the motivation and conceptual basis of ELPSO, details its mutation strategy, and presents the overall algorithmic structure.

2.1.1. Motivation and Concept

A fundamental limitation of standard PSO is its tendency to suffer from premature convergence, particularly in complex multimodal search spaces. As particles are iteratively attracted toward their personal best ($p_i^{\mathrm{best}}$) and the global best ($g^{\mathit{best}}$) solutions, the diversity of the swarm often diminishes rapidly. This loss of diversity reduces the ability of the algorithm to escape local optima and conduct sufficient exploration. A key contributing factor to this issue is the dominant influence exerted by the leader particles. Once a leader emerges, other particles tend to follow its trajectory too closely, resulting in the entire swarm collapsing into a limited region of the search space [8,11].

To overcome the limitations of standard PSO, Jordehi proposed the Enhanced Leader Particle Swarm Optimization (ELPSO) algorithm [11]. The core idea of ELPSO is to reinforce the role of leader particles through a multi-stage mutation strategy designed to prevent their stagnation, which often leads to premature convergence around suboptimal regions. Unlike conventional PSO, in which particles update their positions solely based on deterministic equations, ELPSO adaptively applies different mutation types to leader particles depending on their search state. These mutations introduce diverse perturbations to leader positions, thereby preserving exploration potential and enabling the swarm to escape local optima. This approach helps maintain higher swarm diversity throughout the search process and enhances the algorithm’s ability to identify global optima in complex landscapes.

ELPSO preserves the core structure of standard PSO while enhancing search performance by modifying the behavior of leader particles. Through the use of adaptive mutation strategies, ELPSO offers greater flexibility and improved global exploration compared to earlier PSO variants, making it an effective approach for mitigating premature convergence and stagnation. However, a notable limitation of ELPSO is its lack of explicit mechanisms for controlling diversity at the dimension level. Although the behavior of leader particles is diversified, the algorithm does not monitor or preserve variation along individual dimensions. This shortcoming motivated the development of ELPSO-C, which extends ELPSO by introducing clustering-based, dimension-wise diversity control.

2.1.2. Multi-Stage Mutation Strategy

To mitigate the stagnation of leader particles in standard PSO, ELPSO employs a multi-stage mutation strategy consisting of five distinct operators: Gaussian mutation, Cauchy mutation, dimension-wise opposition-based mutation, global opposition-based mutation, and DE-based mutation [11]. Instead of relying on a single perturbation method, ELPSO adaptively selects operators based on the degree of stagnation. Milder mutations are applied during early stagnation, whereas stronger perturbations are triggered under prolonged stagnation. This composite strategy enhances diversity and achieves a better balance between global exploration and local exploitation.

Gaussian Mutation

Gaussian mutation perturbs each component $x_d$ of a particle’s position vector x by adding Gaussian noise:

$$x_d^{\mathrm{new}}=x_d+\mathcal{N}(0,{\sigma }^2(t)),$$

(3)

where $\mathcal{N}(0,{\sigma }^2\left(t\right))$ denotes a normal distribution with mean zero and time-dependent variance ${\sigma }^2\left(t\right)$. Following [11], the standard deviation $\sigma \left(t\right)$ is defined as:

$$\sigma \left(t\right)={\sigma }_0\left(1-{\displaystyle \frac{t}{T}}\right),$$

(4)

where ${\sigma }_0$ is the initial standard deviation, t is the current iteration number, and T is the maximum number of iterations. This schedule progressively reduces the mutation strength, allowing for finer local search in the later stages of the optimization process.

Cauchy Mutation

Cauchy mutation introduces perturbations drawn from the Cauchy distribution, which has heavier tails than the Gaussian distribution. This property enables occasional large jumps, thereby increasing the likelihood of escaping local optima:

$$x_d^{\mathrm{new}}=x_d+\mathcal{C}(0,\gamma (t)),$$

(5)

where $\mathcal{C}(0,\gamma (t\left)\right)$ denotes a Cauchy distribution with a time-dependent scale parameter $\gamma \left(t\right)$. Following [11], the scale parameter is defined as:

$$\gamma \left(t\right)={\gamma }_0\left(1-{\displaystyle \frac{t}{T}}\right),$$

(6)

where ${\gamma }_0$ is the initial scale, and T is the total number of iterations.

Dimension-Wise Opposition-Based Mutation

In the ELPSO framework, opposition-based mutation generates an opposite solution based on the current position [11]. For each component $x_d$ of the position vector within bounds $[a_d,b_d]$, the opposite value is computed as:

$$x_d^{\mathrm{opp}}=a_d+b_d-x_d.$$

(7)

In this study, we introduce a variant in which the opposition operation is selectively applied to a subset of dimensions $\mathcal{D}\subset \{1,\dots ,D\}$:

$$x_d^{\mathrm{new}}=\left\{\begin{array}{cc}{a}_d+b_d-x_d\hfill & \mathrm{if}d\in \mathcal{D},\hfill \\ x_d\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(8)

We refer to this variant as dimension-wise opposition-based mutation. This operator is newly introduced in the present study and was not part of the original ELPSO framework proposed in [11]. While inspired by the general concept of opposition-based learning, it applies the transformation selectively to a subset of dimensions, thereby providing more targeted diversity enhancement.

Global Opposition-Based Mutation

This mutation applies the opposition-based transformation to all dimensions simultaneously:

$$x_d^{\mathrm{new}}=a_d+b_d-x_d\forall d\in \{1,\dots ,D\}.$$

(9)

It aggressively relocates a particle to its mirrored position and is particularly useful when the current search region is deemed unpromising.

Differential Evolution-Based Mutation

Inspired by the differential evolution (DE) algorithm, this mutation creates new candidate positions using vector differentials between randomly selected particles:

$$x^{\mathrm{new}}=x_r+F·(x_s-x_t),$$

(10)

where $x_r$, $x_s$, and $x_t$ are randomly selected individuals from the population, and F is a scaling factor. This operator introduces global exploration capability into the PSO framework.

Each mutation type is triggered based on a stagnation counter or predefined performance criteria. The strength and frequency of application are adaptively controlled to maintain a balance between convergence and exploration.

To systematically integrate the multi-stage mutation strategy into the ELPSO framework, a staged and conditional logic is adopted. Specifically, each leader particle maintains a stagnation counter that tracks the number of consecutive iterations without fitness improvement. When this counter exceeds a predefined threshold, mutation operators are invoked in a sequential manner. Initially, a mild operator, such as Gaussian or Cauchy mutation, is applied. If stagnation persists, more aggressive strategies—such as dimension-wise or global opposition-based mutation—are introduced. For prolonged stagnation, the differential evolution-based mutation is employed to enhance global exploration.

This adaptive staging is implemented using conditional branching and loop structures within the algorithm. The appropriate mutation stage is selected based on the degree of stagnation, enabling the strategy to effectively balance intensification and diversification.

From an implementation perspective, each mutation operation must be followed by a boundary correction to ensure feasibility:

$$x_d^{\mathrm{new}}=\min \{\max \{x_d^{\mathrm{new}},a_d\},b_d\},\forall d\in \{1,\dots ,D\},$$

(11)

where $[a_d,b_d]$ denotes the valid search range for each dimension. Furthermore, the stagnation counter should be reset upon improvement in fitness, and redundant application of the same mutation stage should be avoided. These considerations contribute to both algorithmic stability and computational efficiency.

Each mutation operator serves a distinct purpose in the search process. Gaussian mutation is well suited for early-stage global exploration due to its moderate perturbation range. Cauchy mutation, with its heavy-tailed distribution, enables occasional large jumps that help escape boundary traps. Dimension-wise opposition-based mutation facilitates local refinement in selected dimensions, while global opposition-based mutation aggressively repositions particles across the entire search space. DE-based mutation introduces directional variation and promotes long-range exploration. These operators are selectively applied based on the degree of stagnation and their expected impact on swarm diversity.

Other strategies for maintaining diversity in swarm-based optimization include techniques such as random immigrants [25], adaptive parameter control [26], and hybridization with differential evolution [18]. The random immigrants approach introduces new particles with random positions into the swarm to refresh diversity, although it may disrupt convergence if not carefully managed. Adaptive parameter schemes adjust coefficients such as inertia weight and acceleration constants during the optimization process, promoting a better balance between exploration and exploitation. Compared to these methods, ELPSO explicitly targets stagnation in leader particles and leverages conditional multi-stage mutation for focused perturbation. While ELPSO enhances flexibility and resilience to local optima, it does not directly control swarm-wide diversity at the dimension level—an issue addressed by our proposed ELPSO-C method.

2.1.3. Algorithmic Structure

The ELPSO algorithm extends the canonical PSO by incorporating mutation mechanisms that are triggered based on stagnation detection. The procedure of the algorithm is as follows:

Step 1: 

Initialization: Initialize the positions $x_i\left(0\right)$ and velocities $v_i\left(0\right)$ of all particles uniformly at random within the search space. Set the personal best $p_i^{\mathrm{best}}=x_i\left(0\right)$, and determine the initial global best $g^{\mathrm{best}}$. Set the inertia weight $\omega ={\omega }_{max}$, and initialize stagnation counters to zero.

Step 2: 

Fitness Evaluation: Evaluate the fitness of each particle. If a particle’s current fitness is better than its $p_i^{\mathrm{best}}$, update $p_i^{\mathrm{best}}$. Update $g^{\mathrm{best}}$ if any particle improves upon it.

Step 3: 

Velocity and Position Update: Update each particle’s velocity and position using Equations (1) and (2), respectively.

Step 4: 

Inertia Weight Update: Linearly decrease the inertia weight over time using:

$$\omega \left(t\right)={\omega }_{max}-\left({\displaystyle \frac{{\omega }_{max}-{\omega }_{min}}{T}}\right)·t,$$

(12)

where t is the current iteration, and T is the maximum number of iterations. Here, ${\omega }_{max}$ and ${\omega }_{min}$ denote the upper and lower bounds of the inertia weight, respectively. A larger $\omega$ in early iterations promotes global exploration, while a smaller $\omega$ in later stages encourages local exploitation near promising regions.

Step 5: 

Stagnation Detection: For each leader particle, track the number of consecutive iterations without fitness improvement. If this count exceeds a predefined threshold, mark the particle as stagnant.

Step 6: 

Mutation Strategy Application: Apply the appropriate mutation strategy to stagnant leader particles. Strategies are selected progressively based on the stagnation duration.

Step 7: 

Boundary Constraint Handling: If any position component after velocity or mutation updates falls outside its feasible range, project it back using a constraint-handling rule such as clipping or reflection.

Step 8: 

Termination Check: If the iteration count t reaches T or a stopping criterion is satisfied (e.g., convergence threshold), return $g^{\mathrm{best}}$ as the solution. Otherwise, increment t and return to Step 2.

2.2. Clustering Techniques in Optimization

Clustering techniques, widely used in data mining and pattern recognition, have also been employed in evolutionary computation to enhance both adaptive behavior and search efficiency. In swarm-based optimization, clustering serves as a tool for analyzing structural properties of particles, such as spatial distribution and behavioral similarity. By leveraging clustering-based insights, one can design more effective mechanisms for diversity maintenance, convergence acceleration, or solution space decomposition.

This section provides an overview of hierarchical agglomerative clustering, explains the complete-linkage method as a representative distance criterion, and discusses its application to dimension-wise analysis within the ELPSO-C framework.

Clustering techniques have become powerful tools in metaheuristic optimization, used to analyze and control the distribution of particles or solutions in the search space. Among these, hierarchical clustering is especially valuable because it reveals structured relationships in the data, providing interpretable insights into convergence behavior and diversity patterns. In this study, we adopted hierarchical clustering as a core mechanism for diversity control in particle-based optimization, using its structure to guide adaptive exploration.

Hierarchical clustering can be broadly categorized into two approaches: agglomerative and divisive. In this study, we focused on the agglomerative strategy, which constructs a hierarchy in a bottom-up manner. Initially, each data point is treated as an individual cluster. Then, in an iterative process, the two closest clusters are merged at each step based on a defined linkage criterion. This process continues until all points are grouped into a single cluster, thereby forming a nested hierarchy of clusters.

The algorithmic procedure of hierarchical agglomerative clustering (HAC) consists of the following iterative steps [27]: initializing each data point as an individual cluster, computing the pairwise distance matrix, merging the two closest clusters, and updating the distance matrix accordingly. This process is repeated until all data points are grouped into a single cluster. The entire clustering progression can be represented as a dendrogram, a tree-like diagram that visually illustrates the nested relationships among clusters and reveals the structure of the data at various levels of granularity.

The choice of linkage criterion, which defines the distance between clusters, has a significant impact on the outcome of hierarchical clustering. Common linkage methods include single linkage, complete linkage, and average linkage, each influencing the clustering structure differently. In this study, we adopted the complete-linkage method, which defines the distance between clusters as the maximum distance between their elements. This approach is particularly effective in suppressing the formation of elongated or chained clusters, thereby mitigating the tendency toward local concentration in the search space.

Agglomerative Clustering Procedure

Step 1: 

Initialize each data point as an individual cluster $C_1,C_2,\dots ,C_n$, and compute the initial distance matrix $DIS=\left[dis(C_i,C_j)\right]$ between all clusters. For clusters $C_i$ and $C_j$, represented by their respective centroid vectors $x^{\left(i\right)}$ and $x^{\left(j\right)}$, the normalized Euclidean distance is defined as:

$$dis(C_i,C_j)=\sqrt{\sum _{d=1}^D{\left({\displaystyle \frac{x_d^{\left(i\right)}-x_d^{\left(j\right)}}{s_d}}\right)}^2},$$

(13)

where D is the dimensionality of the data, and $s_d$ denotes the standard deviation along the dth dimension.

Step 2: 

According to the selected linkage criterion—such as complete linkage—identify the pair of clusters $(C_i,C_j)$ with the smallest distance $dis(C_i,C_j)$ in the matrix $DIS$.

Step 3: 

Merge clusters $C_i$ and $C_j$ to form a new cluster $C_{ij}$.

Step 4: 

Recalculate the distances between the newly formed cluster $C_{ij}$ and all remaining clusters, and update the distance matrix $DIS$.

Step 5: 

Repeat Steps 2–4 until all data points are grouped into a single cluster or the desired number of clusters is achieved.

3. Proposed Method: ELPSO-C

This section presents an improved algorithm, named ELPSO-C (ELPSO with Clustering), which introduces a clustering-based mechanism into the original ELPSO framework [11]. The key idea behind the proposed method is to focus on dimension-wise diversity during the search process and to perform clustering on the dimensions based on their respective search characteristics. By identifying dimensions that have prematurely converged or are approaching stagnation, the algorithm adaptively applies specialized update operations to maintain diversity and facilitate more effective exploration. Through this mechanism, ELPSO-C aims to prevent suboptimal convergence and improve global optimization performance. The proposed method combines the leader enhancement strategy of ELPSO with a clustering operation applied to the entire swarm, thereby achieving both wider coverage of the search space and improved solution accuracy.

3.1. Dimension-Wise Clustering Process

In particle swarm optimization (PSO), one of the primary causes of premature convergence is the loss of diversity in certain dimensions of the solution space. Since PSO updates each particle’s position independently across dimensions, the convergence behavior can vary from one dimension to another. Particularly in high-dimensional optimization problems, some dimensions may stagnate earlier than others, leading to reduced exploration capacity and eventual search stagnation. As the dimensionality increases, the influence of each individual dimension on the overall objective value diminishes, which in turn increases the susceptibility of the algorithm to premature convergence in a subset of dimensions. To address this issue, it is essential to monitor and manage the search dynamics on a per-dimension basis. This motivates the incorporation of dimension-wise analysis and control mechanisms into the PSO framework.

To address the above issue, the proposed method introduces a dimension-wise clustering strategy based on the diversity characteristics of each dimension. Specifically, diversity metrics such as the mean and variance of particle velocities are computed for each dimension and used to characterize its search status. By applying agglomerative clustering to the constructed feature vectors, dimensions exhibiting similar search behavior are grouped together. Among these, clusters containing dimensions with low diversity are identified as being close to stagnation. The algorithm then selectively applies specialized update operations to those dimensions in order to reinvigorate the search activity. This selective intervention enables adaptive control of the search process without uniformly perturbing all dimensions.

Unlike conventional PSO variants that primarily rely on global swarm-level indicators, the proposed method emphasizes localized, dimension-wise analysis and adaptation. This clustering-based mechanism enables the algorithm to detect and mitigate premature convergence at the level of individual dimensions, thereby maintaining diversity more effectively. By restricting intervention to only those dimensions exhibiting signs of stagnation, ELPSO-C avoids unnecessary perturbations and promotes targeted exploration. This promotes a more effective balance between global exploration and local exploitation, particularly in high-dimensional search spaces. Ultimately, the integration of clustering and dimension-wise adaptation enhances the algorithm’s robustness and search efficiency.

3.1.1. Data Construction and Metrics

To evaluate the convergence status of each dimension, the proposed method transforms the velocity vectors of all particles into a dimension-wise representation. Let $v_i^k=[v_{i1}^k,v_{i2}^k,\dots ,v_{id}^k]$ denote the d-dimensional velocity vector of the ith particle at iteration k. The velocity matrix ${\mathbf{V}}^k\in {\mathbb{R}}^{n\times d}$ at iteration k, consisting of all particles’ velocities, is expressed as:

$${\mathbf{V}}^k=\left[\begin{array}{cccc}{v}_{11}^k& v_{12}^k& \dots & v_{1d}^k\\ v_{21}^k& v_{22}^k& \dots & v_{2d}^k\\ ⋮& ⋮& \ddots & ⋮\\ v_{n1}^k& v_{n2}^k& \dots & v_{nd}^k\end{array}\right]$$

(14)

For each dimension $j\in \{1,\dots ,d\}$, we extract a vector $v_j^{k^{\prime }}\in {\mathbb{R}}^n$ by collecting the absolute values of the velocity components across all n particles at iteration k. Specifically,

$$v_j^{k^{\prime }}={\left[|v_{1j}^k|,|v_{2j}^k|,\dots ,|v_{nj}^k|\right]}^T,\mathrm{for}j=1,2,\dots ,d.$$

(15)

Each of these vectors is then summarized by its mean and variance, resulting in a two-dimensional feature vector for each dimension. Specifically, for each $j\in \{1,\dots ,d\}$, we define:

$${\mathrm{data}}_j^k=\left[\mathrm{ave}\left(v_j^{k^{\prime }}\right),\mathrm{var}\left(v_j^{k^{\prime }}\right)\right]$$

(16)

These feature vectors ${\mathrm{data}}_j^k\in {\mathbb{R}}^2$ form the basis for clustering dimensions according to their diversity characteristics.

3.1.2. Agglomerative Clustering Procedure

This subsection describes the first type of clustering applied in the proposed method, referred to as clustering (i). It groups the dimensions of the solution space based on their diversity characteristics, using agglomerative clustering. This allows the algorithm to capture and utilize similarities in convergence behavior among dimensions.

To detect dimensions that are close to convergence or exhibiting stagnation, the proposed method performs agglomerative clustering on the search behavior of each dimension. Each ${\mathrm{data}}_j^k\in {\mathbb{R}}^2$ serves as a feature vector representing the behavior of dimension j at iteration k. To group dimensions with similar diversity characteristics, agglomerative clustering is applied to these feature vectors using the standardized Euclidean distance. The clustering process adopts the complete-linkage criterion, in which the distance between two clusters is defined as the maximum pairwise distance between their constituent elements. This approach results in a hierarchical structure of clusters, from which a predefined number of clusters, $n_c$, are extracted for subsequent analysis.

By transforming particle-level velocity data into dimension-wise feature vectors, this clustering mechanism enables the identification of stagnating dimensions and supports adaptive diversity control in the proposed ELPSO-C framework.

In the proposed method, two types of clustering are used in tandem. Clustering (i) groups dimensions based on their diversity features, enabling the identification of structurally similar dimensions. These clusters are then ranked and utilized in clustering (ii), which guides adaptive diversity control by determining which clusters should be targeted for intervention. This separation of analysis and control allows the method to selectively enhance diversity in stagnating dimensions while avoiding unnecessary modifications in well-explored areas.

3.1.3. Cluster Ranking Strategy

To identify which clusters are likely to represent stagnating dimensions, a ranking score is assigned to each cluster based on statistical summaries of the diversity metrics. For each cluster, we compute the average of the means and the average of the variances across all dimensions contained within the cluster. Then, each cluster is ranked separately in ascending order according to these two criteria: lower average mean and lower average variance indicate a higher likelihood of convergence.

Each cluster receives a rank score for its average mean and another for its average variance. These two rank scores are summed to obtain a final composite score. Clusters with lower total scores are considered more prone to premature convergence and are assigned higher priority in subsequent diversity control operations.

Table 1. Cluster ranking based on mean and variance.

Cluster	Rank by Avg. Mean	Rank by Avg. Variance	Total Score	Rank
Cluster1	1	1	2	1st
Cluster2	4	4	8	4th
Cluster3	2	3	5	2nd
Cluster4	3	2	5	3rd

illustrates an example of the ranking procedure when the number of clusters is $n_c=4$.

By using this simple yet effective ranking mechanism, the proposed method can prioritize intervention in the most critical dimensions—those that are most likely to have stagnated—while avoiding unnecessary operations in well-explored dimensions. This forms the basis for the subsequent adaptation strategies described in the following subsections.

3.1.4. Cluster-Based Strategy Application

This subsection describes clustering (ii), which utilizes the clusters obtained through clustering (i) and the subsequent ranking process. Based on the ranked clusters, diversity control operations such as velocity reinitialization are selectively applied to the most stagnated clusters. In contrast to clustering (i), which focuses on dimension-wise grouping, clustering (ii) guides the application of control strategies at the cluster level.

Based on the ranking described in the previous section, the proposed method selectively applies diversity control mechanisms to the top-ranked clusters that exhibit the lowest diversity. Let $n_c$ denote the total number of clusters, and suppose that the top $\lceil n_c/2\rceil$ clusters are identified as convergence-prone. For each such cluster l, a relative convergence coefficient ${\alpha }_l$ is computed to quantify the level of stagnation compared to the overall swarm. This coefficient is defined as:

$${\alpha }_l={\displaystyle \frac{\mathrm{ave}\left(\mathrm{ave}\left(v_{k,j}^{\prime }\right)|j\in \mathrm{cluster}l\right)}{\mathrm{ave}\left(\mathrm{ave}\left(v_{k,j}^{\prime }\right)|j=1,\dots ,D\right)}},$$

(17)

Here, $v_{n_{c,j}}^{k^{\prime }}$ denotes the absolute velocity vector in dimension j belonging to cluster l, and $\mathrm{ave}(·)$ denotes the arithmetic mean. The numerator represents the average of the mean absolute velocities within the cluster, while the denominator is the mean over all dimensions.

This coefficient provides an intuitive measure of convergence within each cluster. A low value of ${\alpha }_l$ suggests that the particles’ movement along the corresponding dimensions has significantly slowed, indicating that the search in these directions is stagnating. In contrast, high ${\alpha }_l$ values imply ongoing exploration. Thus, ${\alpha }_l$ effectively highlights which clusters are in danger of premature convergence and should be targeted by diversity-enhancing operations.

When ${\alpha }_l$ is sufficiently small, indicating that the corresponding cluster has significantly lower diversity than the overall swarm, a reinitialization operation is performed to restore exploration capability. Specifically, the velocity components of all particles in the dimensions belonging to the selected clusters are randomly reset within the defined search bounds:

$$v_{i,l}^k=\mathrm{rand}[x_{min,j},x_{max,j}],i=1,\dots ,n,l=1,\dots ,\lceil n_c/2\rceil$$

(18)

This reinitialization process injects diversity into dimensions deemed to be stagnating, thereby preventing the swarm from becoming trapped in local optima and enhancing the robustness of the search process.

3.1.5. Clustering Application Conditions

To prevent unnecessary computational overhead and ensure that clustering is only performed when beneficial, the proposed method incorporates specific conditions for triggering the clustering process. These conditions are designed to detect stagnation and loss of diversity in the search process.

The first condition evaluates the relative change in the global best fitness value between two consecutive iterations:

$${\displaystyle \frac{|f({\mathrm{gbest}}^{k-1})-f({\mathrm{gbest}}^k)|}{|f({\mathrm{gbest}}^{k-1})|}}<0.001$$

(19)

This condition, referred to as Condition (A), is satisfied when the improvement in the objective function is below a specified threshold, indicating stagnation in the global search.

The second condition, Condition (B), ensures that the velocity-based diversity across dimensions is sufficiently high before initiating clustering:

$$\mathrm{var}(\mathrm{ave}\left(v_1^{k^{\prime }}\right),\dots ,\mathrm{ave}\left(v_D^{k^{\prime }}\right))>0.0001$$

(20)

This condition verifies that there is still meaningful variability in the average absolute velocity values across dimensions, suggesting that clustering could effectively differentiate between stagnated and active dimensions.

Conversely, when the variability in average velocities is low, Condition (C) is used to determine if velocity reinitialization should be prioritized instead:

$$\mathrm{var}(\mathrm{ave}\left(v_1^{k^{\prime }}\right),\dots ,\mathrm{ave}\left(v_D^{k^{\prime }}\right))\le 0.0001$$

(21)

This condition implies that most dimensions have converged to similar velocity profiles, and injecting diversity through reinitialization is more effective than clustering.

By using these conditions adaptively, the proposed method balances computational efficiency with effective diversity maintenance throughout the optimization process.

3.1.6. Summary of Clustering-Based Adaptation

The preceding subsections have described a clustering-based mechanism to enhance the performance of PSO by detecting and mitigating premature convergence in specific dimensions. The process consists of six key steps:

• Compute the absolute velocity vectors $v_j^{k^{\prime }}$ for each dimension j.
• Construct feature vectors ${\mathrm{data}}_j^k$ consisting of the mean and variance of $v_j^{k^{\prime }}$.
• Perform agglomerative clustering on ${\mathrm{data}}_j^k$ using standardized Euclidean distance and complete linkage to form $n_c$ clusters.
• Rank the clusters based on their average mean and variance values.
• Apply control strategies (e.g., inertia adjustment or velocity reinitialization) to the top $\lceil n_c/2\rceil$ clusters based on ranking.
• Use predefined conditions to determine whether clustering or reinitialization should be executed in each iteration.

This clustering-based adaptation strategy enables dimension-wise control that is both targeted and efficient. It balances the need for diversity preservation with computational cost and allows ELPSO-C to adaptively regulate its search behavior depending on the convergence state of individual dimensions.

3.2. ELPSO-C Algorithm Summary

The entire procedure of the proposed ELPSO-C algorithm can be summarized as follows:

Step 1 

Initialization: Randomly initialize each particle’s position vector $x_i^0$ and velocity vector $v_i^0$ within the search bounds $[x_{min},x_{max}]$.

Step 2 

Evaluation: Evaluate the fitness value of each particle.

Step 3 

Update personal best: For each particle, if the current position yields a better fitness than the personal best $pbes{t}_i^k$, update $pbes{t}_i^k$.

Step 4 

Update global best: Identify the global best $gbes{t}^k$ among all $pbes{t}_i^k$.

Step 5 

Apply mutation strategies: Apply mutation operators (e.g., Gaussian, Cauchy, dimension-wise opposition, full opposition, differential mutation) to generate new candidates. Replace a particle’s position if the mutated candidate improves upon $gbes{t}^k$.

Step 6 

Perform clustering (i) and cluster ranking: Generate data based on the velocity vectors and apply agglomerative clustering. Rank the resulting $n_c$ clusters based on diversity metrics.

Step 7 

Apply clustering-based control (ii):

Step 7-1 

If both Conditions (19) and (20) are satisfied, apply clustering (i) and proceed with Step 7-3.

Step 7-2 

If Conditions (19) and (21) are satisfied, apply clustering (ii).

Step 7-3 

If Condition (19) is not satisfied, update the velocity using Equation (1).

Step 8 

Update position: Update each particle’s position using Equation (2).

Step 9 

Termination check: If the termination criterion is met, stop. Otherwise, increment k and return to Step 2.

This stepwise structure integrates both conventional PSO operations and the proposed clustering-based adaptive strategies to effectively balance exploration and exploitation throughout the search process.

4. Numerical Experiments

This section presents numerical experiments to evaluate the effectiveness of the proposed method, ELPSO-C, in solving optimization problems with various characteristics. Specifically, the performance of ELPSO-C is compared against conventional PSO and ELPSO as well as other evolutionary algorithms, using a set of benchmark functions [28] that include both unimodal and multimodal problems.

To investigate the performance across different levels of problem complexity, the experiments are conducted using benchmark functions with dimensionalities of 10, 30, 50, and 100. The performance is evaluated in terms of convergence accuracy and robustness over multiple independent runs.

The remainder of this section is organized as follows. Section 4.1 describes the experimental settings, including benchmark functions, parameter values, and evaluation metrics. Section 4.2 presents the results obtained on each benchmark function and compares the performance of different algorithms. Section 4.3 provides a discussion based on the experimental results, followed by concluding remarks.

4.1. Experimental Settings

The results of our comparative experiments involve multiple benchmark functions, various dimensional settings, and multiple algorithmic variants, with detailed statistical measures such as mean and standard deviation. To ensure transparency and allow readers to fairly assess the performance of the proposed method, we present the results in detailed form. While this leads to longer tables, it preserves essential information necessary for rigorous comparison. Although other swarm intelligence algorithms such as Artificial Bee Colony (ABC), Gray Wolf Optimizer (GWO), and Tree-Seed Algorithm (TSA) are well established and have demonstrated strong performance in continuous optimization, we did not include them in our numerical comparisons. This is due to practical limitations in implementation and parameter calibration, which are necessary for fair and reproducible evaluation. Instead, we focused on comparing with baseline methods (PSO, GA) and structurally related PSO variants (ELPSO, ACOR), which share similar search mechanisms and allow for more direct performance assessment of the proposed dimension-wise diversity strategies.

To evaluate the performance of the proposed ELPSO-C algorithm, we conducted numerical experiments using a set of 18 benchmark functions widely employed in the field of evolutionary computation. These benchmark functions are selected to cover a diverse range of characteristics, including unimodal and multimodal landscapes, separable and non-separable variables, and different levels of difficulty. A complete list of the benchmark functions is provided in

Table 2. List of benchmark functions used in the experiments.

ID	Function Name	Modality	Properties
F1	Sphere	Unimodal	Separable
F2	Weighted sphere	Unimodal	Separable
F3	Rosenbrock	Unimodal	Non-separable
F4	Schwefel’s Problem 2.22	Unimodal	Separable
F5	Rastrigin	Multimodal	Separable
F6	Ackley	Multimodal	Separable
F7	Griewank	Multimodal	Separable
F8	Schwefel	Multimodal	Non-separable
F9	Shifted sphere	Unimodal	Separable, Shifted
F10	Shifted weighted sphere	Unimodal	Separable, Shifted
F11	Shifted Rastrigin	Multimodal	Separable, Shifted
F12	Shifted Griewank	Multimodal	Separable, Shifted
F13	Shifted and rotated Bent Cigar	Unimodal	Non-separable, shifted, rotated
F14	Shifted and rotated Zakharov	Unimodal	Non-separable, shifted, rotated
F15	Shifted and rotated Rosenbrock	Unimodal	Non-separable, shifted, rotated
F16	Shifted and rotated Rastrigin	Multimodal	Non-separable, shifted, rotated
F17	Shifted and rotated Schaffer’s F7	Multimodal	Non-separable, shifted, rotated
F18	Shifted and rotated Griewank	Multimodal	Non-separable, shifted, rotated

.

The detailed definitions, search domains, and global minima of all benchmark functions listed in Table 2 are provided in Appendix A. Each function was tested under four different dimensional settings: 10, 30, 50, and 100. For each combination of function and dimensionality, the optimization process was independently repeated 30 times to account for the stochastic nature of the algorithms. The performance was evaluated in terms of the mean and standard deviation of the final objective values obtained after a fixed number of iterations.

The parameters of PSO, ELPSO, and ELPSO-C were uniformly configured unless otherwise noted. The swarm size was set to 40, and the maximum number of iterations was fixed at 5000. The inertia weight $\omega$ was linearly decreased from 0.9 to 0.5 over iterations. The acceleration coefficients $c_1$ and $c_2$ were both set to 2.0. For ELPSO and ELPSO-C, the stagnation threshold for triggering mutation operations was set to 50 consecutive iterations without improvement.

For ELPSO-C, the number of clusters was fixed at five unless otherwise stated. Agglomerative clustering with complete linkage was applied based on normalized metrics of velocity diversity in each dimension. The top $\lceil n_c/2\rceil$ clusters ranked by convergence tendency were selected for diversity-enhancing operations.

All algorithms were implemented in Python 3.11 and executed on a machine equipped with an Intel Core i5 processor and 32 GB of RAM. Each function evaluation strictly adhered to the defined search domain, and boundary handling was performed using the saturation method. To ensure a fair comparison across all methods, we adopted standardized parameter settings for each algorithm. These settings are summarized in

Table 3. Parameter settings used in the experiments.

Method	$\mathit{\omega }$	${\mathit{c}}_1$, ${\mathit{c}}_2$	n	${\mathit{k}}_{max}$	${\mathit{n}}_{\mathit{c}}$	${\mathit{h}}^1$	${\mathit{s}}^1$	Other
PSO [2]	0.7	1.5, 1.5	100	10,000	–	–	–	–
PSO-C (proposed)	0.7	1.5, 1.5	100	10,000	4	–	–	–
ELPSO [11]	0.7	1.5, 1.5	100	4900	–	1	2	$F=1.2$
ELPSO-C (proposed)	0.7	1.5, 1.5	100	4900	4	1	2	$F=1.2$
GA [29]	–	–	100	10,000	–	–	–	$P_c=0.8$, $P_m=0.02$
ACOR [30]	–	–	100	10,000	–	–	–	$t=50$, $\xi =0.85$

Note: n is the population size; $k_{max}$ is the maximum number of iterations; $n_c$ is the number of clusters used in cluster-based variants; $h^1$ and $s^1$ are stagnation thresholds used to trigger mutation stage switching in ELPSO; F is the scaling factor for DE-based mutation. In GA, $P_c$ is the crossover probability and $P_m$ is the mutation probability. In ACOR, t is the archive size, and $\xi$ is the scaling factor for the standard deviation of the Gaussian kernel.

.

4.2. Results and Performance Comparison

Numerical experiments were conducted on the benchmark functions described above with dimensionalities $D=10,30,50,$ and 100. In the following sections, the results for each benchmark function are presented in terms of the average, maximum, minimum, standard deviation, and average computation time of the best solution found in each of the 30 trials. The experimental results for each of the 18 benchmark functions are shown in Appendix B using both tables and figures.

To ensure fair and reproducible comparisons, each algorithm was executed independently 30 times per benchmark function, and both mean and standard deviation are reported. Although statistical significance testing such as the Wilcoxon signed-rank test is not included in the current analysis, the presented descriptive statistics offer sufficient resolution to compare the relative performance of algorithms. Statistical testing will be considered as a valuable enhancement in future work. Each figure in Appendix B presents a box-plot-like visualization for each function, where the box represents the range between the mean and the mean ± standard deviation, and the whiskers indicate the maximum and minimum values of the best solutions obtained. If the value of mean minus standard deviation becomes negative, it is replaced with the smallest objective value among all solutions found by any method for the corresponding benchmark function, to ensure proper visualization on logarithmic scales.

4.3. Discussion on the Experimental Results

The experimental results obtained from the 18 benchmark functions—including unimodal and multimodal functions—clearly demonstrate the effectiveness of the proposed ELPSO-C algorithm, especially in higher-dimensional optimization problems. However, ELPSO-C did not always achieve the best performance on all test functions. In particular, for low-dimensional or unimodal problems, the advantage of dimension-wise clustering was less pronounced. In such cases, the additional computational overhead introduced by clustering may outweigh its benefit, leading to comparable or slightly inferior results compared to ELPSO or standard PSO. These results are consistent with the design intention of ELPSO-C, which primarily targets high-dimensional and complex search spaces.

Below is a summarized discussion categorized into low-dimensional (10 dimensions), medium-dimensional (30 and 50 dimensions), and high-dimensional (100 dimensions) scenarios.

4.3.1. Low-Dimensional Problems (10D)

For most of the low-dimensional functions, ELPSO-C achieved competitive or superior performance compared to PSO, PSO-C, ELPSO, GA, and ACO_R. In particular, ELPSO and ELPSO-C consistently outperformed other methods on unimodal functions (e.g., shifted sphere and weighted sphere functions). This superior performance can be attributed to the effective integration of diversity control via dimension-wise clustering and the leader-enhancement strategies introduced in ELPSO. However, for multimodal and rotated functions (e.g., Rastrigin, Griewank), ELPSO-C showed mixed results, reflecting its moderate capability in escaping local optima in lower dimensions, where the search space is relatively confined.

4.3.2. Medium-Dimensional Problems (30D and 50D)

In medium-dimensional scenarios, ELPSO-C demonstrated substantial improvements over other approaches, particularly in multimodal and complex rotated functions such as the shifted and rotated Zakharov, Rosenbrock, and Rastrigin functions. The introduction of dimension-wise clustering significantly enhanced the exploration capacity by identifying stagnating dimensions and adaptively reinitializing particle velocities, thus maintaining higher diversity. While ELPSO showed good results, its performance was further improved by incorporating clustering operations, emphasizing the complementary nature of leader-enhanced mutation strategies and dimension-wise diversity control.

4.3.3. High-Dimensional Problems (100D)

The effectiveness of ELPSO-C was most evident in high-dimensional problems, where it consistently achieved the best performance across nearly all benchmark functions tested. This is due to the intensified importance of diversity control strategies in higher-dimensional spaces, where premature convergence and stagnation become increasingly problematic. Specifically, dimension-wise clustering proved effective in preventing premature convergence by identifying and reactivating dimensions with reduced diversity, thus significantly enhancing global search capabilities. This result underscores the strength of the proposed adaptive clustering-based diversity enhancement strategies combined with ELPSO’s leader mutation mechanisms.

4.3.4. Summary

In summary, the experimental outcomes validate the efficacy of the proposed ELPSO-C algorithm. Its integrated approach of dimension-wise clustering for diversity preservation and leader enhancement through multiple mutation strategies significantly improves the optimization performance, particularly in complex and high-dimensional optimization problems.

5. Conclusions

This study proposed an enhanced swarm-based optimization algorithm, ELPSO-C, to address premature convergence in high-dimensional problems. The key idea was to apply agglomerative clustering to dimension-wise velocity-based diversity features, enabling selective intervention in stagnating dimensions. This mechanism maintained diversity without disrupting well-explored dimensions.

The effectiveness of ELPSO-C was validated through experiments on 18 benchmark functions, including rotated and shifted multimodal problems. ELPSO-C consistently outperformed baseline algorithms, especially in high-dimensional landscapes, by achieving lower average and variance of objective values with reasonable computational cost.

While the proposed method shows promising performance, some limitations remain. The choice of clustering parameters (e.g., number of clusters) may require tuning depending on problem characteristics. The diversity metric does not directly reflect sensitivity to the objective function, which may limit its effectiveness in ill-conditioned problems. Moreover, the clustering operation introduces additional computational overhead. Future work will focus on developing adaptive mechanisms for determining clustering parameters and incorporating objective-function sensitivity or gradient information into the feature vectors. We also plan to explore lightweight or incremental clustering approaches for real-time applications. Additional directions include evaluating the impact of particle count on convergence and performance, integrating gradient-based strategies (e.g., Nesterov acceleration), and conducting statistical significance tests (e.g., t-test or Wilcoxon test) for performance validation. Lastly, extending the comparative study to include recent metaheuristics such as the Special Relativity Search Algorithm (SRSA) [31] or the Whale Optimization Algorithm (WOA) would further assess the robustness of ELPSO-C.