Parallel Optimized Particle Swarm Optimization: An Efficient Parallel Approach for Neural Network Training

Abstract

Global optimization has become a cornerstone of modern machine learning, particularly for the challenging task of training high-dimensional neural network architectures. Despite the prevalence of established metaheuristics, traditional approaches frequently struggle with the irregular and multimodal error surfaces inherent in complex training landscapes. This paper proposes the parallel optimized particle swarm optimization (Parallel OPSO), a multi-population parallel framework specifically engineered to enhance the training efficiency and predictive accuracy of neural networks across classification and regression tasks. At its core, OPSO integrates several advanced mechanisms: a simulated annealing-based acceptance criterion to escape local minima, adaptive inertia weights that dynamically respond to stagnation by injecting momentum, and a hybrid Broyden–Fletcher–Goldfarb–Shanno (BFGS) local search phase for fine-grained convergence. While OPSO defines this underlying algorithmic model, Parallel OPSO represents its execution across parallel multi-population architectures. By partitioning the swarm into interacting subpopulations and leveraging modern multi-core architectures, Parallel OPSO significantly reduces training time, with prominent performance gains primarily manifesting on highly challenging and multimodal optimization tasks. Experimental evaluations demonstrate that while standard algorithms may perform adequately on simpler loss landscapes, Parallel OPSO achieves superior convergence robustness and accuracy when navigating demanding and highly irregular error surfaces. These results underscore Parallel OPSO as a highly efficient and scalable framework for optimizing neural network parameters in complex search spaces.

1. Introduction

The training of artificial neural networks (ANNs) represents one of the most critical and challenging tasks in modern machine learning and computational intelligence [1,2]. From a mathematical perspective, training an ANN is formulated as a large-scale global optimization problem [3], where the objective is to minimize an error function by adjusting the network’s weights and biases. The global optimization problem is defined as

$$x^{*}=\arg \underset{x\in S}{\min }f\left(x\right)$$

(1)

where the search space S is defined as the high-dimensional hyper-rectangle:

$$S=[a_1,b_1]\otimes [a_2,b_2]\otimes \cdots \otimes [a_n,b_n]$$

In the context of neural network training, the objective function $f\left(\mathbf{x}\right)$ represents the loss function (e.g., mean squared error or cross-entropy), and the vector $\mathit{x}\in {\mathbb{R}}^n$ contains the network’s parameters. This optimization landscape is highly non-linear, non-convex, and multi-modal, containing an immense number of local minima [4]. Apart from neural network training, this mathematical formulation finds critical applications in a variety of real-world problems, such as physics [5,6], chemistry [7], economics [8], and medicine [9,10]. Historically, optimization methods are divided into two main categories: deterministic and stochastic. Deterministic approaches, such as Interval methods [11,12] or gradient-based algorithms (e.g., backpropagation, stochastic gradient descent, Adam), rely on mathematical proofs or gradient derivatives to navigate the search space [13]. However, in high-dimensional and non-convex neural network landscapes, gradient-based methods are heavily prone to getting trapped in saddle points or sub-optimal local minima and are highly sensitive to initial weight configurations [4,13,14]. To overcome these limitations, stochastic and metaheuristic methods have been widely adopted. Guided by randomness, these methods do not guarantee finding the absolute global minimum in polynomial time, but they consistently locate high-quality solutions with remarkable implementation simplicity [15].

Researchers have proposed numerous stochastic algorithms, including controlled random search [16], simulated annealing [17,18], differential evolution [19], particle swarm optimization (PSO) [20,21], ant colony optimization [22], and genetic algorithms [23]. Furthermore, modern computational paradigms have successfully ported these algorithms to multi-core and GPU architectures to accelerate high-dimensional optimization [24,25], with comprehensive surveys available in recent literature [26,27].

Among these metaheuristics, particle swarm optimization (PSO) has emerged as a highly efficient population-based algorithm inspired by the social behavior of bird flocking [28]. In PSO, a population of candidate solutions, called particles, explores the search space S. Each particle i is characterized by its current position ${\mathit{x}}_{\mathit{i}}$ and its velocity ${\mathit{v}}_{\mathit{i}}$. The dynamic movement of each particle in a D-dimensional space is governed by updating its velocity and position at each iteration step [21,29] according to the following formulation:

$$v_{i,d}=\omega ·v_{i,d}+c_1·r_1·({\mathbf{p}}_{\mathrm{best},i,d}-x_{i,d})+c_2·r_2·({\mathbf{g}}_{\mathrm{best},d}-x_{i,d})$$

(2)

$$x_{i,d}=x_{i,d}+v_{i,d}$$

(3)

where $\omega$ is the inertia weight introduced to control the impact of the previous velocity [30], $c_1$ and $c_2$ are the cognitive and social acceleration coefficients, and $r_1,r_2\sim U(0, 1)$ are uniformly distributed random numbers. The vector ${\mathbf{p}}_{\mathrm{best},i}$ denotes the personal best position discovered by particle i, while ${\mathbf{g}}_{\mathrm{best}}$ represents the global best position found by the entire swarm. Despite its success, classical PSO can suffer from premature convergence and high computational costs when optimizing complex neural network architectures, falling into local minima during the exploitation phase [20,21]. Many researchers focus on dynamically adjusting the inertia weight $\omega$ [31,32] or integrating hybrid and parallel implementation techniques [33,34] to tackle complex numerical optimization problems. However, limited attention has been given to the simultaneous enhancement of scalability, multi-population parallel evolution, and adaptive convergence control in large-scale optimization environments. To address these challenges, this paper introduces the parallel optimized particle swarm optimization (Parallel OPSO) method, specifically engineered to enhance the training efficiency and predictive accuracy of neural networks. The proposed algorithm is a multi-population parallel framework that partitions the swarm into interacting subpopulations, leveraging modern multi-core architectures to significantly reduce execution time. More specifically, OPSO defines the core sequential optimization method that incorporates the enhanced search mechanisms, whereas Parallel OPSO extends this core engine into a cooperative, multi-threaded multi-population paradigm.

The novelty of Parallel OPSO lies in the integration of several advanced mechanisms within a unified parallel framework:

• Simulated Annealing Acceptance Criterion: A stochastic acceptance criterion is integrated to allow particles to occasionally accept worse positions, enabling the swarm to escape stubborn local minima [17,18].
• Adaptive Inertia Weight and Momentum: The inertia weight $\omega$ dynamically responds to swarm stagnation by injecting momentum, balancing wide-range exploration and local exploitation based on the algorithm’s real-time performance [32].
• Hybrid Local Search Phase: Incorporates a deterministic BFGS quasi-Newton method to refine promising candidate solutions and significantly enhance convergence accuracy toward the global optimum [35].

The main contributions of this work are summarized as follows:

• A parallel cooperative extension of the PSO algorithm, where the swarm is partitioned into multiple subpopulations evolving simultaneously on multi-core architectures using the OpenMP framework to enhance exploration in large-scale search spaces.
• A dynamic propagation mechanism that enables structured and controlled information exchange among the independent subpopulations, improving convergence speed while maintaining overall population diversity.
• The incorporation of adaptive inertia weight $\omega$ based on the framework introduced in [32], which dynamically responds to swarm stagnation by injecting momentum, effectively balancing global exploration and local exploitation based on the algorithm’s real-time performance.
• The integration of a simulated annealing-based stochastic acceptance criterion [17,18], which allows particles to occasionally accept worse positions, providing a robust mechanism to escape stubborn local minima.
• The incorporation of a hybrid local search phase utilizing the deterministic BFGS quasi-Newton method [35] to significantly improve convergence accuracy and speed up the localization of the exact global optimum.
• A comprehensive experimental evaluation spanning both large-scale global optimization benchmark problems and demanding neural network training tasks for classification and regression, demonstrating that Parallel OPSO consistently converges to more robust solutions compared to state-of-the-art algorithms.

The remainder of this article is organized as follows: Section 2 outlines the foundational background of classical PSO alongside the detailed mathematical formulation of the proposed Parallel OPSO; Section 4 presents the experimental setup, benchmark functions, and comparative performance analysis; and finally, Section 5 summarizes our key findings, provides concluding remarks, and outlines directions for future research.

2. Background Materials

This section begins with the main aspects of the original PSO method and continues by presenting in detail the main parts of the proposed method.

2.1. Artificial Neural Networks and Training Process

Artificial neural networks (ANNs) are computational models inspired by biological nervous systems, structured to approximate complex non-linear functions and identify high-dimensional data patterns [1,13]. An ANN typically comprises interconnected processing units (neurons) organized into sequential layers: an input layer that accepts raw feature vectors, one or more hidden layers that extract hierarchical representations, and an output layer that delivers final estimations or classifications [13]. Each inter-neuron connection is weighted by an adjustable parameter, and individual nodes apply non-linear activation functions to modulate incoming signals [13].

The training process of an artificial neural network involves iteratively modifying its internal weights and biases to minimize the discrepancy between predicted outputs and known ground-truth targets [1,13]. This optimization mechanism operates through two primary computational stages:

• Forward Propagation: Input data propagate sequentially through the network layers, where weighted sums are computed and passed through non-linear activation functions to generate a prediction [1,13].
• Backward Propagation (Backpropagation): The output error is quantified using a predefined loss function, and error gradients with respect to network parameters are calculated utilizing the chain rule [13,14].

Conventionally, gradient-based optimization techniques update connection weights iteratively along the error landscape [14,35]. Because neural network training landscapes are typically high-dimensional, non-convex, and prone to severe stagnation in local attractors [4], advanced global optimization and meta-heuristic frameworks are frequently applied to optimize or assist in training neural network architectures [2,15,26].

2.2. Classical Particle Swarm Optimization (PSO)

The classical particle swarm optimization (PSO) algorithm, originally introduced by Kennedy and Eberhart [29], is a population-based stochastic optimization technique inspired by the social behavior of bird flocking or fish schooling. The algorithm explores an n-dimensional continuous search space $𝒮\subset {\mathbb{R}}^n$ using a population of m interacting agents called particles. Each particle i ($i\in \{1,2, \dots ,m\})$ is characterized by its dynamic state parameters, consisting of an n-dimensional position vector ${\mathbf{x}}_i={\left[x_{i1},x_{i2}, \dots ,x_{in}\right]}^T$ and a corresponding velocity vector ${\mathbf{v}}_i={\left[v_{i1},v_{i2}, \dots ,v_{in}\right]}^T.$ During the optimization process, each particle maintains a memory of its own historical personal best position discovered so far, denoted as ${\mathbf{p}}_i={\left[p_{i1},p_{i2}, \dots ,p_{in}\right]}^T$. Concurrently, the swarm tracks the absolute global best position achieved by any particle in the entire population, defined as ${\mathbf{p}}_{\mathrm{gbest}}={\arg \min }_{i\in \{1,\dots ,m\}}f\left({\mathbf{p}}_i\right)$, where $f(·)$ represents the objective function under minimization. To balance global exploration and localized exploitation, a trade-off extensively evaluated in recent metaheuristic literature [2,3], Shi and Eberhart [30] introduced the inertia weight factor $\omega$. At each discrete evolutionary epoch t, the dynamic trajectories of the particles are updated according to the following stochastic kinematic equations:

$${\mathbf{v}}_i(t+1)=\omega \left(t\right)·{\mathbf{v}}_i\left(t\right)+c_1·{\mathbf{r}}_1·\left({\mathbf{p}}_i\left(t\right)-{\mathbf{x}}_i\left(t\right)\right)+c_2·{\mathbf{r}}_2·\left({\mathbf{p}}_{\mathrm{gbest}}\left(t\right)-{\mathbf{x}}_i\left(t\right)\right)$$

(4)

$${\mathbf{x}}_i(t+1)={\mathbf{x}}_i\left(t\right)+{\mathbf{v}}_i(t+1)$$

(5)

where $\omega \left(t\right)$ is the inertia weight, $c_1$ and $c_2$ denote the cognitive and social acceleration coefficients, respectively, and ${\mathbf{r}}_1,{\mathbf{r}}_2\sim \mathcal{U}(0, 1)$ represent independent uniformly distributed random vectors injected to maintain stochastic diversity. Despite its mathematical simplicity and high convergence speed, the classical formulation is prone to stagnation in complex multimodal landscapes due to rapid loss of population diversity, a challenge frequently highlighted in modern optimization benchmarks [20,36]. The main parts of the original PSO method are listed in Algorithm 1.

Algorithm 1: The classical PSO algorithm.

3. The Proposed Method

3.1. Parallel Optimized Particle Swarm Optimization Algorithm

The distributed framework of the optimized particle swarm optimization (OPSO) algorithm is engineered by partitioning the global agent population into K distinct subpopulations, and it is presented graphically in Figure 1. Each subpopulation operates as an autonomous evolutionary unit allocated to an individual processing element, enabling simultaneous execution across a shared-memory environment [37].

Figure 1. Schematic representation of the Parallel OPSO framework architecture. The central OpenMP shared-memory region manages thread spawn and execution across K autonomous subpopulation islands (Island 1 through Island K). Red directional arrows illustrate the dynamic inter-island migration topologies, depicting both single-target routing (1-to-1, 1-to-N) and multi-island coordination mechanisms (N-to-1, N-to-N) used for periodic elite candidate solution exchange.

Before initiating the optimization cycle, the multi-swarm configurations are established, and the global termination strategy ($\mathrm{TS}$) is selected during the initialization phase (Algorithm 2). To govern decentralized convergence behavior and manage interaction dynamics between the concurrently evolving swarms, the architecture introduces a set of control features: a topological propagation method ($\mathrm{NM}$), a generational communication rate ($\mathrm{NR}$), a volumetric quantity of migrated solutions ($\mathrm{NP}$), and a localized termination criterion ($\mathrm{TC}$) [34,38].

Algorithm 2: Optimized and parallel particle swarm optimization (OPSO)—initialization phase.

   The communication frequency across the multi-swarm layout is controlled by the propagation rate $\left(\mathrm{NR}\right)$, which establishes the exact number of generational steps between consecutive data distributions. Low values for $\mathrm{NR}$ enforce tight coupling and frequent interaction, a configuration that can accelerate convergence velocities but risks collapsing population diversity [38,39]. Conversely, high values for $\mathrm{NR}$ grant the individual islands extended periods of decoupled execution, fostering independent exploration of unmapped solution regions.

In a similar manner, the volume of shared elite information is dictated by the parameter $\mathrm{NP}$, which defines the number of top-performing vectors transmitted during a migration step. Expanding $\mathrm{NP}$ strengthens the collective exploitation capability by injecting high-quality geometric coordinates into target swarms; however, excessive data injection can induce premature convergence by steering all independent groups toward identical local optima [38,39,40]. Within this framework, the specific bounds for $\mathrm{NR}$ and $\mathrm{NP}$ are derived through systematic empirical tuning to maintain a healthy balance between localized search refinement and global space exploration, with the precise values detailed in the Experimental Results section.

During every generational step, the system monitors each subpopulation to determine if it has met its localized termination criterion $\left(\mathrm{TC}\right)$, which is evaluated independently via an advanced convergence heuristic known as the similarity check [41]. The total number of stagnated units is then cross-referenced with a macro-level decision threshold, denoted as Q. This threshold mathematically operationalizes the chosen system-wide termination strategy $\left(\mathrm{TS}\right)$ to govern the macro-level optimization loop [38,41]:

$$Q=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}TS=ANY\hfill \\ \lfloor K/2\rfloor +1,\hfill & \mathrm{if}TS=MAJORITY\hfill \\ K,\hfill & \mathrm{if}TS=ALL\hfill \end{array}\right.$$

(6)

Following the calculation of the threshold metrics, the subpopulations proceed with their parallel evolutionary operations, executing localized kinematics updates and simulated annealing leader adjustments as detailed in Algorithm 4. At generational intervals matching the propagation rate $\left(\mathrm{NR}\right)$, the main execution loop (Algorithm 3) triggers the designated migration topology $\left(\mathrm{NM}\right)$ to distribute $\mathrm{NP}$ elite candidate vectors across the multi-threaded network [34,38,40].

Algorithm 3: Optimized and parallel particle swarm optimization (OPSO)—main evolutionary loop (continues from Algorithm 2).

Algorithm 4: ProcessIsland(k): parallel fitness evaluation and particle update for island k (invoked from the parallel region of Algorithm 3).

The iteration counter increments, and the status of the threshold criterion Q is evaluated. If the conditions for global termination are not fulfilled, the concurrent optimization sequence continues. Upon satisfying the macro-level stopping condition, the framework (Algorithm 3) invokes a memetic refinement sequence, executing a rigorous local search directly around the global best solution $g_{\mathrm{best}}$ to secure high-precision mathematical accuracy before returning the final optimized result [42].

From an architectural standpoint, mapping each subpopulation to an isolated thread or processing core facilitates true concurrent execution. Synchronization barriers are strictly confined to the scheduled propagation phases, allowing the swarms to evolve independently during intermediate generations and significantly minimizing memory access latencies [34]. Resource utilization is optimized through the selection of the global termination strategies (ANY, MAJORITY, ALL), which prevent thread starvation and idle processing states by eliminating the requirement for all processing elements to terminate at the same time. The ANY strategy, in particular, permits rapid continuation by halting the global loop the moment early convergence is detected on any single thread via the similarity metric [41]. While selecting the ANY global termination strategy ($TS=\mathrm{ANY}$) prioritizes computational efficiency and drastically reduces execution time by eliminating idle processor states, it inherently carries a potential risk of premature termination if a single isolated subpopulation converges early to a local attractor. Within our framework, this risk is directly mitigated by two structural safeguards: (i) the mandatory warm-up phase ($t\ge t_{min}=50$), which suppresses localized termination diagnostics during early stochastic iterations, and (ii) the continuous inter-island migration mechanism ($N\text{-}\mathrm{to}\text{-}N$), which routinely injects high-quality solution coordinates across all subpopulations to maintain global diversity and prevent single-island stagnation. Furthermore, the computational footprint of the migration phase remains small, as it restricts communication to a limited set of elite solution coordinates. This brief coordination phase introduces negligible latency relative to the overall objective function evaluation costs, and it significantly enhances optimization performance by enabling effective information sharing across the parallel architecture [34,38].

3.2. Inter-Swarm Communication Protocol and Migration Topologies

Within the cooperative execution phase of the parallel framework, the communication layer facilitates the distribution of high-quality parameter configurations among the concurrent islands. When the scheduled migration interval conditions are satisfied, the elite solution vectors identified by a specific subpopulation are transmitted across the shared-memory space to overwrite the lowest-performing coordinates of the designated target swarms. The foundational objective of this information routing scheme is to optimize cross-island exploration, ensuring that promising topological discoveries are rapidly assimilated into the broader optimization landscape within the main evolutionary loop (Algorithm 3) to enhance overall convergence efficiency [34,38].

Unlike conventional distributed metaheuristics that rely on static and rigid structural layouts, such as ring, star, or fully mesh migration topologies, the proposed framework introduces a localized, dynamically configurable migration model tailored for high-dimensional execution spaces. Conventional geometric topologies impose strict synchronization locks and significant thread contention penalties as the execution scale grows, which frequently induces severe memory access latencies within shared-memory architectures [34,38]. To avoid structural complexity and eliminate excessive communication overhead, the parallel architecture relies on a versatile routing protocol operating across four discrete configuration states:

• One-to-One ($1\mathrm{to}1$): A single source subpopulation, selected uniformly at random, migrates its historical elite parameter states exclusively to a single target subpopulation, which is also designated through a random uniform distribution.
• One-to-All ($1\mathrm{to}N$): A randomly isolated subpopulation serves as a central broadcaster, transmitting its optimal geometric updates to refresh the search vectors of all other $K-1$ active swarms simultaneously.
• All-to-One ($N\mathrm{to}1$): Every active computational swarm routes its current top-performing vectors toward a single, randomly targeted subpopulation, consolidating global directional data into a localized spatial window to catalyze structural exploitation.
• All-to-All ($N\mathrm{to}N$): A dense execution mode where every individual subpopulation synchronizes and exchanges its finest candidate solutions with the historical memory matrix of every other swarm in the network.

By adjusting the combination of these topological routing strategies $\left(\mathrm{NM}\right)$ alongside the volumetric payload parameter $\left(\mathrm{NP}\right)$, the OPSO framework maintains a fluid balance between regional exploitation and global population diversity [34,38]. This dynamic coordination mitigates structural stagnation risks while preserving the low coordination footprint required for efficient execution on multi-core compute clusters.

3.3. Adaptive Inertia Weight Mechanism

To achieve a self-regulating balance between global exploration and localized exploitation, each autonomous subpopulation discards static or strictly time-dependent parameter scaling. Instead, an adaptive inertia weight mechanism is implemented [31] within the per-island parallel update procedure (Algorithm 4), adapting the phenotypic stagnation-tracking framework introduced in the ideal particle swarm optimization (IPSO) architecture [32]. This framework evaluates the state of the swarm by monitoring the phenotypic convergence rate across consecutive generations. When the global fitness gradient flattens, indicating that the multi-swarm trajectory is confined within a local attractor, the inertia weight parameter is dynamically adjusted to modulate particle momentum.

Operationally, the system calculates the absolute differential progress of the collective fitness landscape at every iteration. If the fitness deviation between sequential states falls below a predefined threshold, a stagnation counter increments. By calculating the ratio of cumulative stagnant states against the elapsed generational counter, a dynamic constraint coefficient is derived. Consequently, when the subpopulation actively discovers novel structural minima, the velocity profile maintains a high exploratory momentum. Conversely, when severe multi-island stagnation is detected, the inertia scales down to its minimum operating boundary, dampening particle velocities to focus computational resources strictly on high-precision local exploitation [31,32].

Mathematical Formulation

The evolutionary state monitoring is governed by evaluating the global fitness volume change at each generation t, tracking structural stagnation vectors as formalized in the IPSO framework [32]:

$$\delta \left(t\right)=\left|{\displaystyle \sum _{i=1}^m}{f}_i\left(t\right)-{\displaystyle \sum _{i=1}^m}{f}_i(t-1)\right|$$

(7)

where m denotes the number of particles within the subpopulation, and $f_i\left(t\right)$ represents the objective cost value of the i-th particle at generation t. The localized stagnation state indicator $\zeta \left(t\right)$ is defined as

$$\zeta \left(t\right)=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\delta \left(t\right)=0\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(8)

Using the real-time feedback from $\zeta \left(t\right)$, the cumulative stagnation tracking matrix $S_{\delta }\left(t\right)$ and its corresponding density ratio $C_{\delta }\left(t\right)$ are computed dynamically:

$$S_{\delta }\left(t\right)={\displaystyle \sum _{k=1}^t}\zeta \left(k\right)$$

(9)

$$C_{\delta }\left(t\right)={\displaystyle \frac{S_{\delta }\left(t\right)}{t}}$$

(10)

The adaptive inertia weight $\omega \left(t\right)$ supplied to the velocity kinematic updating core (Algorithm 4) is formally governed by the bound-constrained formulation [32], building upon the dynamic inertia weight control concepts introduced in [30]:

$$\omega \left(t\right)=max\left({\omega }_{min},min\left({\omega }_{max},{\omega }_{max}-C_{\delta }\left(t\right)·({\omega }_{max}-{\omega }_{min})\right)\right)$$

(11)

where ${\omega }_{max}$ and ${\omega }_{min}$ represent the predefined boundary coefficients governing the dynamic momentum spectrum.

3.4. Adaptive Simulated Annealing Acceptance Criterion

To balance the exploitation of local optima with the necessity of maintaining exploratory pressure, a stochastic acceptance mechanism is integrated into the leader update phase of each island. Traditional PSO variants typically employ a greedy selection rule, which often leads to rapid convergence toward suboptimal regions in non-convex, high-dimensional landscapes [21,28]. To mitigate this, our approach implements an adaptive simulated annealing acceptance criterion during the subpopulation update (Algorithm 4), allowing the sub-population leader (${\mathit{P}}_{\mathrm{best},k}$) to occasionally accept inferior solutions based on a temperature-dependent probability inspired by the classic simulated annealing framework [5,17,18].

Let $\Delta =f\left({\mathbf{X}}_{\mathrm{best},k}\right)-f\left({\mathbf{P}}_{\mathrm{best},k}\right)$ represent the fitness difference between the current best particle within sub-cluster $S_k$ and the recorded sub-cluster leader. The probability $P_{\mathrm{acc}}$ of accepting a potentially worse configuration is defined as

$$P_{\mathrm{acc}}=\left\{\begin{array}{cc}1.0,\hfill & \mathrm{if}\Delta <0\hfill \\ exp\left(-{\displaystyle \frac{\Delta }{T·{\overline{\Delta }}_k+ϵ}}\right),\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(12)

where T represents the current normalized temperature under a linear cooling schedule, calculated as $T=max\left(1.0-\mathrm{progress},{10}^{-4}\right)$ to enforce a minimum thermal floor, withprogressdenoting the relative execution progress ($t/t_{max}$). To ensure robustness across diverse objective function scales, an adaptive scaling factor ${\overline{\Delta }}_k$ is maintained locally for each sub-cluster k, updating incrementally via an exponential moving average (EMA) as ${\overline{\Delta }}_k\leftarrow 0.9·{\overline{\Delta }}_k+0.1·\Delta$ for every positive fitness delta observed. The term $ϵ={10}^{-9}$ serves as a numerical stability constant to prevent division by zero.

The parameters governing the thermal schedule were selected based on preliminary sensitivity tests balancing exploration and computational stability. A dynamic linear cooling schedule provides a smooth transition from stochastic global sampling in early stages to deterministic local convergence near termination. The thermal floor (${10}^{-4}$) avoids numerical underflow, while the EMA smoothing coefficient of $0.9$ ensures that scale updates reflect long-term landscape trends without overreacting to transient gradient spikes.

This mechanism facilitates a transition from a high-exploration phase at the onset of the optimization process, where the acceptance of inferior moves enables the algorithm to escape local attractors, to a high-exploitation phase as T approaches zero, effectively narrowing the search towards the global optimum [17,18]. By decoupling the leader update from strict greedy selection, the algorithm exhibits enhanced resilience against the “saddle point” problem in high-dimensional spaces [4], ensuring a more thorough traversal of the search space before settling into a terminal configuration [11,12]. This adaptive control, combined with the parallel island model paradigm, provides a robust defense against premature stagnation without the prohibitive computational overhead associated with global, non-localized simulated annealing schedules [32,34,39,40].

3.5. Hybrid Local Search Phase

To balance global stochastic exploration with high-precision exploitation, Parallel OPSO incorporates a memetic hybrid local search phase [42] based on the deterministic Broyden–Fletcher–Goldfarb–Shanno (BFGS) quasi-Newton algorithm [35]. During execution, the local search procedure is stochastically triggered on the global best position ${\mathbf{g}}_{\mathrm{best}}$ within the main evolutionary loop (Algorithm 3) with a predefined probability $p_{ls}$$(\mathtt{local}\_\mathtt{search}\_\mathtt{rate}).$ In our experimental configuration, $p_{ls}$ was set to $0.0$ during the main execution iterations. This choice is intentional: extensive preliminary tuning revealed that applying full gradient-based BFGS evaluations at every generation incurs prohibitive computational overhead due to numerical finite-difference gradient estimations. By restricting BFGS refinement exclusively as a final deterministic post-processing step on ${\mathbf{g}}_{\mathrm{best}}$, Parallel OPSO preserves high overall computational throughput while ensuring high-precision terminal convergence.

To prevent race conditions during parallel island evaluations, this stochastic triggering and the subsequent update are synchronized across threads using OpenMP critical sections (#pragma omp critical). Additionally, a final deterministic refinement step is mandatorily executed inside the $\mathtt{done}\left(\right)$ routine upon meeting the swarm stopping criteria, applied strictly to the global best position ${\mathbf{g}}_{\mathrm{best}}$, once the swarm reaches full convergence, defined by a similarity check criterion requiring 25 consecutive unchanged iterations per island. When invoked, each local search phase is allocated a maximum budget of 2000 iterations (function evaluations). The BFGS procedure [35] terminates when any of the following stopping criteria are satisfied: (i) no further reduction in the objective function value can be achieved during line search, or (ii) the maximum iteration limit of 2000 evaluations is reached.

The BFGS algorithm iteratively refines the target position vector ${\mathbf{x}}_k$ along a search direction ${\mathbf{p}}_k=-H_k\nabla f\left({\mathbf{x}}_k\right)$, where the gradient vector $\nabla f\left({\mathit{x}}_k\right)$ is numerically approximated via finite differences and $H_k$ represents an approximate inverse Hessian matrix. At each local step k, $H_k$ is dynamically updated using the spatial difference vector ${\mathbf{s}}_k={\mathbf{x}}_{k+1}-{\mathbf{x}}_k$ and the gradient difference vector ${\mathbf{y}}_k=\nabla f\left({\mathbf{x}}_{k+1}\right)-\nabla f\left({\mathbf{x}}_k\right)$ according to

$$H_{k+1}=(I-{\rho }_k{\mathbf{s}}_k{\mathbf{y}}_k^T)H_k(I-{\rho }_k{\mathbf{y}}_k{\mathbf{s}}_k^T)+{\rho }_k{\mathbf{s}}_k{\mathbf{s}}_k^T$$

(13)

where ${\rho }_k={\left({\mathbf{y}}_k^T{\mathbf{s}}_k\right)}^{-1}$. The refined candidate position replaces ${\mathbf{g}}_{\mathrm{best}}$ strictly under a greedy scheme, updating the global optimum only if $f\left({\mathit{x}}_{\mathrm{refined}}\right)<f\left({\mathbf{g}}_{\mathrm{best}}\right)$. This gradient-based coupling guarantees superlinear local convergence characteristics, refining the stochastic candidate solutions of OPSO to exact numerical optima.

3.6. Subpopulation Termination Criterion via Similarity Check

In parallel multi-swarm architectures, determining the optimal execution boundary for each autonomous subpopulation is critical. Halting an island prematurely deprives the global swarm of valuable population diversity and high-quality coordinates during migration phases [39,40]. Conversely, allowing a stagnated subpopulation to execute until the global maximum iteration limit leads to wasted processor cycles and unnecessary energy consumption [34,38]. To resolve this trade-off, each individual island k dynamically monitors its own search trajectory during its iteration updates (Algorithm 4) via a localized termination state $\left({\mathrm{TC}}_{\mathrm{k}}\right)$.

To ensure sufficient initial exploration and avoid false positive termination signals caused by early stochastic fluctuations, a mandatory warm-up period is enforced. Localized convergence diagnostics remain inactive until the generation counter satisfies:

$$t\ge t_{min},\mathrm{where}{t}_{min}=50$$

(14)

Upon crossing this threshold, the algorithm tracks the fitness progression of island k at every generation t by calculating the absolute difference between its current and previous local best objective values:

$$\Delta f_k\left(t\right)=\left|f\left({\mathbf{L}}_{\mathrm{best},k}\left(t\right)\right)-f\left({\mathbf{L}}_{\mathrm{best},k}(t-1)\right)\right|$$

(15)

To evaluate whether $\Delta f_k\left(t\right)$ reflects genuine mathematical convergence rather than a transient plateau, the framework embeds the similarity check heuristic proposed by Tsoulos et al. [41]. Conventional termination metrics often rely on instantaneous point-to-point differences, making them highly sensitive to noise in complex fitness landscapes. Specifically, island k is declared locally converged ($T{C}_k=\mathrm{true}$) when the average objective improvement falls below a predefined tolerance threshold $\epsilon$:

$${\displaystyle \frac{1}{W}}{\displaystyle \sum _{m=0}^{W-1}}\Delta f_k(t-m)<ϵ$$

(16)

In contrast, the similarity check evaluates the convergence behavior over a sliding historical window of W = 25 consecutive iterations. This window size was determined through empirical sensitivity trials evaluating window sizes in the range W in [10, 40] across five representative datasets (Wine, Pima, WDBC, Spiral, and Ecoli). As summarized in the Table 1, a setting of W = 25 offered the optimal baseline balancing convergence reliability and computational speed. Smaller window sizes (W < 20) risk premature island termination due to transient objective plateaus—as evidenced by elevated error rates on datasets such as Wine (19.12%). Conversely, excessively large windows ($W\ge 30$) yield no tangible accuracy gains and, in cases like Wine and WDBC, significantly degrade classification precision while systematically escalating computational overhead up to 1.06 s per fold. Consequently, W = 25 represents a robust trade-off, ensuring stability against false positive termination without incurring unnecessary computational latency.

Table 1. Sensitivity analysis of similarity check window parameter (W).

Dataset	Error W = 10	Error W = 20	Error W = 25	Error W = 30	Error W = 40	Time W = 10	Time W = 20	Time W = 25	Time W = 30	Time W = 40
Wine	19.12%	17.88%	14.12%	45.88%	45.59%	0.672 s	0.764 s	0.891 s	0.853 s	1.060 s
Pima	26.45%	25.99%	25.73%	26.43%	26.43%	0.647 s	0.783 s	0.886 s	0.807 s	1.062 s
WDBC	5.61%	7.45%	6.18%	12.48%	11.55%	0.725 s	0.738 s	0.888 s	0.857 s	1.061 s
Spiral	41.84%	41.35%	42.08%	42.01%	42.00%	0.676 s	0.863 s	0.887 s	0.954 s	1.062 s
Ecoli	52.15%	53.61%	53.97%	51.15%	54.64%	0.664 s	0.733 s	0.769 s	0.958 s	1.058 s

From an implementation perspective, each parallel worker thread maintains an isolated heuristic evaluator instance (threadSimilarity[k]). This design eliminates thread contention and lock overheads, enabling fully decoupled convergence checks across processing cores [34,37]. The aggregate count of stagnated islands is continuously tracked as

$$S_{\mathrm{term}}={\displaystyle \sum _{k=1}^K}\mathbb{I}({\mathrm{TC}}_k=\mathrm{true})$$

(17)

where $\mathbb{I}(·)$ denotes the indicator function. When $S_{\mathrm{term}}\ge Q,$ the global termination strategy $\left(\mathrm{TS}\right)$ evaluated in the main loop (Algorithm 3) is triggered [38,41]. Finally, the maximum iteration count $t_{max}$ serves as an absolute safety ceiling ($t\ge t_{max}$), guaranteeing bounded computational time under all execution conditions.

3.7. Pseudocode and Algorithmic Summary

The complete metaheuristic workflow of the proposed parallel optimized particle swarm optimization algorithm, incorporating the parallel migration topologies, adaptive inertia weight control mechanisms, simulated annealing leader acceptance criteria, and hybrid local search refinement, is synthesized in the following pseudocode, presented across three parts to preserve legibility: the initialization phase (Algorithm 2), the main evolutionary loop (Algorithm 3), and the per-island parallel update procedure invoked within it (Algorithm 4):

Algorithm 2 performs the sequential initialization of the swarm, the islands, and the initial leaders. The resulting state is then evolved by the main loop of Algorithm 3, which repeatedly invokes, within its OpenMP parallel region, the per-island update procedure detailed separately in Algorithm 4.

Additionally, Algorithm 3 relies on two key sub-routines: $\mathtt{UpdateIslandLeader}\left(\right)$ updates the leader of each island using a Simulated Annealing-based acceptance criterion to escape local minima, while $\mathtt{Propagate}\left(\right)$ manages inter-island communication by migrating candidate solutions across subpopulations according to the specified topology.

Algorithm 4 specifies the computations carried out independently and concurrently for each island during every generation of the main loop, namely the fitness evaluation and personal/global best updates, followed by the particle movement, velocity clamping, and bound enforcement steps.

3.8. Computational Complexity Analysis

The computational complexity of the proposed algorithm is analyzed per iteration, considering the number of particles N, the dimensionality of the search space D, and the number of subpopulations (islands) K. Let $n=N/K$ be the number of particles per island. The total complexity per iteration is dominated by the evaluation of the objective function, the velocity/position updates, the hybrid local search procedure, and the communication overhead. The breakdown is as follows:

• Objective Function Evaluation: The primary cost is associated with the evaluation of the objective function. In each iteration, N evaluations are performed. If $f_{cost}$ denotes the computational cost of the objective function $f\left(\mathbf{x}\right)$, the complexity is $\mathcal{O}(N·f_{cost})$. Due to the OpenMP parallelization, this is distributed across K threads, reducing the execution time to $\mathcal{O}({\displaystyle \frac{N}{K}}·f_{cost})$ per thread, assuming uniform load balancing [34,37].
• Particle Update Mechanism: The velocity and position update operations for all N particles across D dimensions, executed concurrently within the subpopulation update procedure (Algorithm 4), require $\mathcal{O}(N·D)$ time. Similar to the objective function, this is parallelized as $\mathcal{O}({\displaystyle \frac{N}{K}}·D)$ per subpopulation [30,34].
• Hybrid Local Search Phase: The deterministic BFGS quasi-Newton refinement is stochastically invoked on the global best solution within the main evolutionary loop (Algorithm 3) with probability $p_{ls}$. Let $L_{\mathrm{evals}}$ denote the average number of objective function evaluations required by the BFGS gradient calculations and line search per local execution. The computational cost of matrix updates for a D-dimensional search space is $\mathcal{O}\left(D^2\right)$. Thus, the amortized cost per iteration for the hybrid refinement phase is $\mathcal{O}\left(p_{ls}·(L_{\mathrm{evals}}·f_{\mathrm{cost}}+D^2)\right)$ [35,42].
• Inter-Island Propagation: The migration mechanism triggered within the main execution loop (Algorithm 3) involves sorting particles by fitness within subpopulations and transferring them. Sorting n particles takes $\mathcal{O}(nlogn)$. Given that the migration occurs every P iterations (propagation rate), the amortized complexity per iteration for the migration phase is $\mathcal{O}\left({\displaystyle \frac{K·nlogn}{P}}\right)$ [34,39,40].
  Combining the above, the total computational complexity per iteration $T_t$ is expressed as

  $$T_t\approx \mathcal{O}\left({\displaystyle \frac{N}{K}}(f_{\mathrm{cost}}+D)+p_{ls}·\left(L_{\mathrm{evals}}·f_{\mathrm{cost}}+D^2\right)+{\displaystyle \frac{Nlog\left(\frac{N}{K}\right)}{P}}\right)$$

  (18)
  As P is typically significantly larger than 1 (e.g., $P=10$) and the local search activation probability is set to zero ($p_{ls}=0.0$), any potential local search computational overhead is completely eliminated, leaving only negligible propagation overhead relative to objective function evaluations. Consequently, the algorithm exhibits linear scaling with respect to the number of particles and dimensions, while parallel efficiency is optimized by the factor K, allowing the algorithm to maintain high performance in high-dimensional optimization problems [27,34,38].

4. Results

4.1. Experimental Datasets

The datasets used in the conducted experiments were downloaded from the following online databases:

• The UCI website. https://archive.ics.uci.edu/ (accessed on 24 June 2026) [43]
• The Keel website. https://sci2s.ugr.es/keel/datasets.php (accessed on 24 June 2026) [44].

The datasets used in the conducted experiments are shown in Table 2.

Table 2. Summary and main characteristics of the experimental datasets.

Dataset	Dimension	Features	Classes
Appendictis dataset, which is a medical dataset [45]	106	7	2
The Alcohol dataset, concerning alcohol consumption, was introduced in [46]	125	6	4
Australian dataset [47] used in economics	690	14	2
The Balance dataset, related to various psychological experiments, was introduced in [48]	625	4	3
Circular dataset, which represents an artificial dataset with two classes	2000	2	2
Cleveland dataset, that denotes a medical dataset regarding heart diseases [49,50]	297	13	5
Dermatology dataset [51], a medical dataset	358	34	6
The Ecoli dataset, used for detecting protein localization problems, was introduced in [52]	336	7	8
The Fert dataset, related to sperm concentration disorders	100	9	2
Haberman, a medical dataset related to the presence of breast cancer	306	3	2
Hayes roth dataset [53]	160	4	3
Heart dataset [54], a medical dataset regarding heart diseases.	270	13	2
HeartAttack, a medical dataset related to heart diseases	138	8	2
HouseVotes dataset [55].	232	16	2
The Ionosphere dataset, used for the classification of radar signals from the ionosphere, was introduced in [56,57]	351	34	2
Liverdisorder dataset [58], which is a medical dataset	345	6	2
Lymography dataset [59]	148	18	4
Mammographic dataset [60], which is a medical dataset regarding the detection of breast tumors	830	5	2
The Optdigits dataset, used for the recognition of handwritten digits from optical scans	5620	64	10
Parkinsons dataset, which is a medical dataset regarding the Parkinson’s Disease [61]	195	22	2
Phoneme, a dataset related to sound.	5404	5	2
Pima dataset, which is a medical dataset used widely [62]	768	8	2
PirVision dataset [63], that contains occupancy detection data	10,000	8	2
Popfailures dataset [64], used in various experiments regarding climate	540	18	2
Spiral dataset, which was produced artificially and contains two classes	1000	2	2
Regions2 dataset [65]	300	5	2
Saheart dataset [66], a medical dataset used to detect the presence of heart diseases	462	9	2
Satimage dataset, related to satellite images	6435	36	7
Segment dataset [67], which is an image processing dataset	2310	19	7
The Sonar dataset [68]	208	60	2
Spambase dataset, related to the detection of spam emails [69]	4597	57	2
Statheart, which is a medical dataset on heart diseases	270	13	2
The Student dataset, containing data collected from various school-related studies, was originally presented in [70]	395	30	3
Transfusion dataset [71]	748	4	2
The WDBC dataset, a medical dataset for breast tumor diagnosis, was originally presented in [72]	569	30	2
The Wine dataset, used for wine quality classification, was originally presented in [73,74]	178	13	3
The Zoo dataset, used for animal classification, was originally presented in [75]	101	16	7
The Eeg dataset, used for EEG classification with the following cases: Z_F_S, ZO_NF_S and ZONF_S [76]	500	21	3

4.2. Experimental Settings

The software used in this work was entirely coded in ANSI C++ with the support of the OPTIMUS environment, downloaded from https://github.com/itsoulos/OPTIMUS/(accessed on 24 June 2026) [77]. All the experiments were conducted on an AMD Ryzen 5950X with 128GB of RAM and the operating system used was Debian Linux. During the experimental evaluation, a 10-fold cross-validation procedure was applied, in which each dataset was divided into ten subsets (folds). In each validation round, 90% of the instances were used for training, whereas the remaining 10% were reserved for testing. To ensure statistical reliability, each experimental configuration was independently executed 30 times, with a different initialization seed assigned to the random number generator for each run. The random numbers required during the experiments were generated using the ‘drand48()‘ function provided by the C standard library. The average classification error is defined as

$$E_C\left(N\left(\overrightarrow{x},\overrightarrow{w}\right)\right)=100\times {\displaystyle \frac{{\sum }_{i=1}^N\left(\mathrm{class}\left(N\left(\overrightarrow{x_i},\overrightarrow{w}\right)\right)\ne y_i\right)}{M}}$$

(19)

where the set T represents the associated test set, with $T=\left(x_i,y_i\right),i=1, \dots ,M$. The settings used in the conducted experiments are shown in Table 3.

Table 3. Values for the experimental parameters.

Parameter	Meaning	Value
$N_c$	Number of particles/chromosomes	500
$N_g$	Maximum number of generations	500
k	Number of weights	10
$NM$	Propagation method	N to N
K	Number of subpopulations	10
$NR$	Propagation rate	10
$NP$	Volumetric quantity of migrated solutions	1
$TS$	Global termination strategy	ANY
$c_1$	Cognitive Constant	2.0
$c_2$	Social Constant	1.0
w	Inertia Weight Bounds	[0.4, 0.9]
$T_0$	Simulated Annealing Initial Temperature	1.0
$T_{min}$	Simulated Annealing Minimum Temperature	0.0001
T	Simulated Annealing Cooling Schedule	$1.0-\mathrm{progress},\mathrm{where}\mathrm{progress}={\displaystyle \frac{\mathrm{iter}}{\mathrm{Ng}}}$
p	Simulated Annealing Acceptance Probability	$exp\left(-{\displaystyle \frac{\Delta }{T·\overline{\Delta }+{10}^{-9}}}\right)$
$\overline{\Delta }$	Simulated Annealing Average Error	${\overline{\Delta }}_{\mathrm{new}}=0.9·{\overline{\Delta }}_{\mathrm{old}}+0.1·\Delta$
$r_{1,}{r}_2$	Random Stochastics	$r_1, r_2\sim \left|N(0, 1)\right|,r_1, r_2\in [0, 2]$
$\mathit{LocalSearchRate}$	Local Search Probability	0.00

Moreover, the following notation is used in the tables presenting the experimental results:

• The column DATASET indicates the name of the used dataset.
• The column BFGS is used for the application of the BFGS optimization procedure [78] to train an artificial neural network [79,80] with 10 processing nodes.
• The column NEAT [81] represents the results obtained by the NEAT (NeuroEvolution of Augmenting Topologies) method.
• The column RBF denotes the usage of a radial basis function (RBF) network [82,83] with 10 processing nodes.
• The column GEN denotes the incorporation of a genetic algorithm [84] with the same parameters as in the proposed method. This algorithm was used to train an artificial neural network with 10 processing nodes.
• The column OPSO stands for the results obtained by the incorporation of the proposed method.
• The row labeled AVERAGE presents the mean classification error obtained across the entire collection of datasets.

4.3. Parallel Performance and Scalability Analysis

To thoroughly evaluate the computational advantage of the proposed parallelized scheme, benchmarking experiments were conducted across all 20 datasets using 1, 2, 5, and 10 threads. Table 4 summarizes the execution metrics, including wall-clock runtime ($T_p$), speed-up ratio ($S_p=T_1/T_p$), parallel efficiency ($E_p=S_p/p$), and CPU utilization (%).

Table 4. Parallel performance metrics including Wall-Clock Time ($T_p$ in seconds), Speed-up ($S_p$), Parallel Efficiency ($E_p$), and CPU Utilization across different thread configurations ($p\in \{1, 2, 5, 10\}$).

Dataset	Wall-Clock Time ${\mathit{T}}_{\mathit{p}}$ (s)				Speed-Up ${\mathit{S}}_{\mathit{p}}$				Parallel Efficiency ${\mathit{E}}_{\mathit{p}}$				CPU Utilization (%)
	1T	2T	5T	10T	1T	2T	5T	10T	1T	2T	5T	10T	1T	2T	5T	10T
Appendicitis	2.26	1.82	1.69	0.89	1.00	1.24	1.34	2.54	100%	62.1%	26.7%	25.4%	100%	199%	497%	990%
Alcohol	2.33	1.86	1.84	0.73	1.00	1.25	1.27	3.19	100%	62.6%	25.3%	31.9%	100%	199%	497%	986%
Circular	2.41	1.86	1.38	0.63	1.00	1.30	1.75	3.83	100%	64.8%	34.9%	38.3%	100%	199%	496%	982%
Cleveland	2.33	2.02	0.94	0.62	1.00	1.15	2.48	3.76	100%	57.7%	49.6%	37.6%	100%	199%	496%	982%
Dermatology	2.35	1.88	1.76	0.51	1.00	1.25	1.34	4.61	100%	62.5%	26.7%	46.1%	100%	199%	497%	984%
Ecoli	2.98	1.91	1.80	0.61	1.00	1.56	1.66	4.89	100%	78.0%	33.1%	48.9%	99%	199%	498%	985%
Fertility	2.40	1.90	1.78	0.91	1.00	1.26	1.35	2.64	100%	63.2%	27.0%	26.4%	100%	199%	497%	990%
Haberman	2.32	1.91	1.33	0.91	1.00	1.21	1.74	2.55	100%	60.7%	34.9%	25.5%	99%	199%	497%	991%
Heart	2.41	1.86	0.90	0.51	1.00	1.30	2.68	4.73	100%	64.8%	53.6%	47.3%	100%	199%	496%	983%
Heartattack	2.46	1.88	1.79	0.53	1.00	1.31	1.37	4.64	100%	65.4%	27.5%	46.4%	100%	199%	497%	983%
Ionosphere	2.48	1.93	1.80	0.53	1.00	1.28	1.38	4.68	100%	64.2%	27.6%	46.8%	99%	199%	497%	983%
Liverdisorder	2.43	1.99	1.74	0.59	1.00	1.22	1.40	4.12	100%	61.1%	27.9%	41.2%	100%	199%	497%	981%
Lymography	2.35	1.90	0.81	0.48	1.00	1.24	2.90	4.90	100%	61.8%	58.0%	49.0%	100%	199%	494%	982%
Mammographic	2.44	1.90	1.76	0.59	1.00	1.28	1.39	4.14	100%	64.2%	27.8%	41.4%	100%	199%	497%	981%
Parkinsons	2.38	1.86	0.96	0.69	1.00	1.28	2.48	3.45	100%	64.0%	49.6%	34.5%	100%	199%	494%	948%
Pima	3.44	1.56	1.07	0.69	1.00	2.21	3.21	4.99	100%	110.5%	64.2%	49.9%	100%	199%	495%	987%
Spiral	3.54	1.39	1.02	0.70	1.00	2.55	3.47	5.06	100%	127.5%	69.4%	50.6%	100%	198%	491%	961%
Saheart	2.81	1.38	0.81	0.61	1.00	2.04	3.47	4.61	100%	102.0%	69.4%	46.1%	99%	198%	494%	985%
WDBC	3.27	2.15	0.85	0.72	1.00	1.52	3.85	4.54	100%	76.0%	77.0%	45.4%	100%	199%	495%	985%
Wine	2.41	1.87	1.78	0.83	1.00	1.29	1.35	2.90	100%	64.5%	27.0%	29.0%	100%	199%	498%	990%

As observed in Table 4, expanding the thread pool leads to a consistent reduction in total execution time across all datasets. For instance, on the Spiral dataset, wall-clock time drops from 3.54 s (1 thread) to 0.70 s (10 threads), achieving a speed-up factor of 5.06 times. Notably, several datasets (e.g., Pima, Spiral, and Saheart) exhibit super-linear speed-up under 2 threads ($E_p>100\%$), which is attributed to localized cache efficiency advantages and stochastic early convergence in the island subpopulations.

As thread counts scale up to 10, CPU utilization closely matches full capacity (~980–990%), demonstrating effective thread utilization. However, parallel efficiency shows an expected decrease, settling between 25% and 50% across most datasets. This behavior is primarily driven by inter-island communication overhead. During periodic migration phases, thread synchronization barriers and data exchange mechanisms (e.g., solution transfer and memory locks) introduce latency that scales with the number of concurrent subpopulations. Nevertheless, the substantial reduction in total runtime confirms that the parallel implementation delivers substantial performance gains, ensuring fast execution for complex optimization runs.

4.4. Comparison with Other Methods

Table 5 reports the per-dataset classification error achieved by four baseline machine-learning models (BFGS, NEAT, RBF, GEN) and by the proposed OPSO algorithm under three thread/population configurations ($T=1$, $T=2$, $T=5$), evaluated across 40 benchmark datasets.

Table 5. Comparison with other machine learning methods on the series of classification datasets. Reported values are the average classification error (%) on the test set for each dataset/model combination.

DATASET	BFGS	NEAT	RBF	GEN	OPSO $(\mathit{T}=1)$	OPSO $(\mathit{T}=2)$	OPSO $(\mathit{T}=5)$
APPENDICITIS	18.00%	17.20%	12.23%	18.10%	22.30%	23.30%	22.80%
ALCOHOL	41.50%	66.80%	49.38%	39.57%	17.66%	20.21%	22.13%
AUSTRALIAN	38.13%	31.98%	34.89%	32.21%	29.41%	27.38%	24.43%
BALANCE	8.64%	23.14%	33.53%	8.97%	7.90%	7.32%	6.94%
CIRCULAR	6.08%	35.18%	5.98%	5.99%	4.20%	4.36%	4.34%
CLEVELAND	77.55%	53.44%	67.10%	51.60%	50.79%	46.14%	45.69%
DERMATOLOGY	52.92%	32.43%	62.34%	30.58%	9.60%	10.08%	9.97%
ECOLI	69.52%	43.44%	59.48%	54.67%	51.73%	51.97%	50.67%
FERT	23.20%	15.37%	15.00%	28.50%	25.70%	24.90%	25.50%
HABERMAN	29.34%	24.04%	25.10%	28.66%	30.13%	29.50%	28.97%
HAYES ROTH	37.33%	50.15%	64.36%	56.18%	32.92%	34.85%	35.00%
HEART	39.44%	39.27%	31.20%	28.34%	27.33%	23.11%	21.11%
HEARTATTACK	46.67%	32.34%	29.00%	29.03%	32.43%	24.10%	19.93%
HOUSEVOTES	7.13%	10.89%	6.13%	6.62%	7.09%	5.91%	6.96%
IONOSPHERE	15.29%	19.67%	16.22%	15.15%	15.14%	15.03%	14.86%
LIVERDISORDER	42.59%	30.67%	30.84%	31.11%	35.71%	32.38%	31.94%
LYMOGRAPHY	35.43%	33.70%	25.50%	28.42%	26.22%	25.50%	24.93%
MAMMOGRAPHIC	17.24%	22.85%	21.38%	19.88%	17.05%	17.02%	17.06%
OPTDIGITS	67.13%	72.56%	71.97%	67.81%	56.78%	48.49%	49.39%
PARKINSONS	27.58%	18.56%	17.41%	18.05%	17.21%	15.58%	16.37%
PHONEME	15.58%	22.34%	23.32%	15.55%	14.93%	15.11%	15.46%
PIMA	35.59%	34.51%	25.78%	32.19%	33.12%	30.47%	27.88%
PIRVISION	16.94%	11.84%	24.58%	14.31%	9.31%	10.54%	9.07%
POPFAILURES	5.24%	7.05%	7.04%	5.94%	7.87%	6.92%	6.83%
REGIONS2	36.28%	33.23%	38.29%	29.39%	30.10%	30.13%	31.11%
SAHEART	37.48%	34.51%	32.19%	34.86%	35.63%	34.41%	33.87%
SATIMAGE	71.73%	76.19%	40.19%	54.54%	47.58%	40.01%	47.47%
SEGMENT	68.97%	66.72%	59.68%	57.72%	22.35%	15.21%	17.74%
SONAR	25.85%	34.10%	27.90%	22.40%	23.10%	22.10%	19.80%
SPAMBASE	18.16%	35.77%	29.35%	6.37%	6.50%	6.59%	6.19%
SPIRAL	47.99%	48.66%	44.87%	48.66%	44.81%	43.01%	41.91%
STATHEART	39.65%	44.36%	31.36%	27.25%	28.04%	23.89%	22.22%
STUDENT	7.14%	10.20%	5.49%	5.61%	6.00%	5.68%	5.68%
TRANSFUSION	25.84%	24.87%	26.41%	24.87%	24.35%	24.92%	24.45%
WDBC	29.91%	12.88%	7.27%	8.56%	8.13%	5.80%	4.95%
WINE	59.71%	25.43%	31.41%	19.20%	22.30%	15.77%	11.82%
Z_F_S	39.37%	38.41%	13.16%	10.73%	19.84%	14.17%	9.00%
ZO_NF_S	43.04%	43.75%	9.02%	21.54%	21.98%	12.44%	10.08%
ZONF_S	15.62%	5.44%	4.03%	4.36%	3.66%	3.02%	2.68%
ZOO	10.70%	20.27%	21.93%	9.50%	4.80%	3.40%	3.90%
AVERAGE	33.79%	32.61%	29.56%	26.32%	23.29%	21.27%	20.78%

A closer inspection of the per-dataset results reveals that the advantage of OPSO over the baseline models is far from uniform, but instead concentrated in a specific subset of datasets where the baseline learners struggle markedly. On datasets such as SEGMENT, DERMATOLOGY, ALCOHOL, Z_F_S, and ZO_NF_S, the baseline models yield error rates ranging from roughly 0.31 to 0.69, whereas all three OPSO variants converge to substantially lower values, often below 0.20 and in several cases below 0.10. This pattern suggests that OPSO is particularly effective at navigating the more irregular or multimodal error landscapes that appear to defeat gradient-based or single-pass heuristics such as BFGS and NEAT, rather than offering a uniform, dataset-independent improvement. Examining the behavior across the three OPSO configurations ($T=1, 2, 5$) shows that increasing the number of threads tends to produce a progressive, though not strictly monotonic, refinement of the solution. Datasets such as WINE, Z_F_S, and ZO_NF_S display a clear, consistent decrease in error from $T=1$ to $T=5$, indicating that additional parallel search threads continue to yield tangible benefits even after the first configuration has already improved substantially over the baselines. In a smaller number of cases, such as FERT and SEGMENT, the lowest error is instead observed at $T=2$ rather than $T=5$, pointing to occasional diminishing or slightly non-monotonic returns once the population has already reached a favorable region of the search space. Not every dataset conforms to this favorable pattern, however. On HABERMAN, POPFAILURES, LIVERDISORDER, and TRANSFUSION, the OPSO variants perform on par with or even slightly worse than the best baseline model, with differences generally confined to the second or third decimal place. These are, notably, datasets on which the baseline models themselves already achieve comparatively low and closely clustered error rates, suggesting that once a classification problem is “easy” or nearly linearly separable, the additional search capacity offered by OPSO yields little to no further gain. At the other extreme, datasets such as CLEVELAND, SATIMAGE, ECOLI, and OPTDIGITS remain uniformly difficult for every method tested, with error rates staying above 0.40 regardless of the algorithm used, an indication that the difficulty of these multi-class problems lies in the intrinsic complexity of the data itself rather than in any shortcoming specific to a particular optimization strategy.

To test this impression quantitatively rather than relying on qualitative inspection alone, we computed, for each of the 40 benchmark datasets, the standard deviation of the four baseline models’ error rates (BFGS, NEAT, RBF, GEN) trained on the same architecture and data splits, and used it as a proxy for the ruggedness of the underlying error landscape: since these four methods explore the search space through fundamentally different mechanisms (gradient descent, neuro-evolution, radial-basis fitting, and a genetic algorithm), a wide spread among their results is indirect evidence of a landscape with many competing local optima, rather than of measurement noise. We then computed the Spearman rank correlation between this ruggedness proxy and OPSO’s improvement over the mean baseline error across datasets. As shown in Figure 2, this correlation is strong and highly significant ($\rho$ = 0.71, p < 0.0001), confirming quantitatively that OPSO’s advantage grows systematically with the degree of disagreement among the baseline optimizers. A comparable, though somewhat weaker, positive correlation is also observed with the number of classes in each dataset ($\rho$ = 0.62, p < 0.0001), consistent with the intuition that decision boundaries become more complex, and hence the error surface more irregular, as the number of classes increases. In contrast, dataset size, feature count, and the feature-to-instance ratio show no significant association with OPSO’s improvement ($\rho$ ≤ 0.30, p > 0.05 in all cases), indicating that OPSO’s advantage is driven specifically by landscape irregularity rather than by dataset scale or dimensionality per se.

Figure 2. Correlation between OPSO’s improvement over the mean baseline error and baseline-method disagreement (landscape-ruggedness proxy) across the 40 benchmark datasets.

A statistical comparison of the OPSO against the other machine learning methods is depicted in Figure 3 and in Table 6. Across the 40 benchmark datasets, the Friedman test revealed statistically significant differences among the seven models under comparison $\left({\chi }^2=88.36,df=6,p<0.001\right)$, with a Nemenyi critical difference of 1.425. The OPSO variant with $T=5$ threads achieved the lowest mean classification error (20.78% ± 13.53) and the best average rank (2.16), closely followed by OPSO ($T=2$) (21.27% ± 13.18), the difference between the two not reaching statistical significance (paired t-test, p = 0.135). From a practical standpoint, while $T=5$ offers a marginal numerical advantage, $T=2$ provides a significantly better trade-off between predictive performance and computational resource efficiency, making $T=2$ the more cost-effective default setting in hardware-constrained environments. In contrast, OPSO ($T=1$) (23.29%) and all four baseline models, GEN (26.32%), RBF (29.56%), NEAT (32.61%), and BFGS (33.79%), differed significantly from the best-performing variant (p < 0.001 in every case), providing strong evidence for the superiority of the proposed OPSO algorithm, particularly under the $T=2$ and $T=5$ configurations, over conventional approaches.

Figure 3. Mean classification error of OPSO ($T=1, 2, 5$) versus baseline machine-learning models. Asterisks (*) denote a statistically significant pairwise difference (Holm-corrected *** $p<0.001$); ns denotes a non-significant pair.

Table 6. Statistical comparison of classification error, average ranking, and significance tests (Wilcoxon and paired t-test) for the six machine learning models relative to proposed method as the reference model.

Model	Group	Mean Error	SD Error	Min Error	Max Error	Avg Rank	Mean Diff vs. OPSO (T5)	Wilcoxon p	Paired p	Significance
OPSO T5	Proposed (OPSO)	20.78	13.53	2.68	50.67	2.16	0	Reference	Reference	Reference
OPSO T2	Proposed (OPSO)	21.27	13.18	3.02	51.97	2.67	−0.49	0.0351	0.1351	ns
OPSO T1	Proposed (OPSO)	23.29	14.13	3.66	56.78	3.58	−2.52	0.0001	0.0002	***
GEN	Baseline	26.32	16.84	4.36	67.81	4.08	−5.55	0	0.0001	***
RBF	Baseline	29.56	18.46	4.03	71.97	4.44	−8.78	0.0002	0.0002	***
NEAT	Baseline	32.61	17.56	5.44	76.19	5.38	−11.83	0	0	***
BFGS	Baseline	33.79	19.81	5.24	77.55	5.7	−13.01	0	0	***

Friedman test: chi-squared = 88.36, df = 6, p-value = 0. Nemenyi critical difference (alpha = 0.05): 1.425. Reference model for pairwise tests: OPSO_T5 (auto-selected as the best-performing OPSO variant). Significance vs. reference (paired t-test): *** p < 0.001, n.s. = not significant.

4.5. Ablation Study

To systematically evaluate the individual contribution and synergistic impact of each core component within the proposed OPSO framework, an ablation study was conducted. Three distinct variants were created by disabling one core mechanism at a time: (i) without subpopulation migration, (ii) without simulated annealing (SA) acceptance criterion, and (iii) without adaptive inertia weight. All algorithmic variants were evaluated under identical experimental conditions using 10-fold cross-validation across ten benchmark datasets (Circular, Pima, WDBC, Wine, Spiral, Ionosphere, Lymography, Liver Disorder, Heart, and Mammographic). Table 7 summarizes the mean classification error (%) across all evaluated datasets.

Table 7. Ablation study comparing the mean classification error (%) and standard deviation across 10-fold cross-validation for the proposed OPSO framework and its constituent algorithm variants.

Dataset	Full OPSO	Without Migration	Without SA Acceptance Criterion	Without Adaptive Inertia
Circular	4.33%	4.45%	4.46%	4.50%
Pima	26.53%	28.03%	26.91%	27.50%
WDBC	4.63%	5.11%	8.78%	4.96%
Wine	12.29%	14.29%	13.35%	16.00%
Spiral	41.91%	42.61%	42.03%	42.63%
Ionosphere	14.37%	15.14%	14.60%	14.60%
Lymography	24.93%	26.57%	25.07%	26.43%
Liver Disorder	34.07%	34.59%	34.97%	33.21%
Heart	20.22%	20.56%	20.55%	21.33%
Mammographic	16.96%	17.23%	16.99%	17.07%
Mean	20.02%	20.86%	20.77%	20.82%

Analysis of Components:

• Simulated Annealing Acceptance Criterion: The inclusion of the Simulated Annealing probabilistic acceptance mechanism proves vital for preventing premature convergence, particularly in non-convex and complex search spaces. On the WDBC dataset, disabling the SA mechanism leads to a severe performance degradation, with the mean classification error escalating from 4.63% to 8.78% (an absolute increase of 4.15 percentage points). This demonstrates the essential role of SA in allowing the algorithm to probabilistically accept non-improving states during early search phases, thereby escaping local minima.
• Subpopulation Migration: Eliminating the migration mechanism consistently impairs performance, increasing the overall average classification error across all datasets from 20.02% to 20.86%. Subpopulation migration plays a decisive role in maintaining structural diversity and promoting information exchange across island subpopulations. The absence of migration noticeably degrades performance on datasets such as Wine (error rising from 12.29% to 14.29%), Lymography (from 24.93% to 26.57%), Pima (from 26.53% to 28.03%), and Ionosphere (from 14.37% to 15.14%).
• Adaptive Inertia Weight: Dynamic adjustment of the inertia weight provides the necessary balance between global exploration and fine-grained local exploitation. Disabling adaptive inertia results in an overall mean error increase to 20.82%. The impact is most prominent on the Wine dataset, where classification error increases significantly from 12.29% to 16.00% (+3.71 percentage points), as well as on Heart (rising from 20.22% to 21.33%) and Lymography (rising from 24.93% to 26.43%). Without adaptive inertia, particle velocity updates fail to optimize search trajectories as convergence progresses.
• Overall Framework Synergy: The full OPSO framework achieves the lowest overall mean classification error (20.02%) across all ten benchmark datasets, outperforming every ablated variant. These empirical findings confirm that subpopulation migration, simulated annealing acceptance criterion, and adaptive inertia operate synergistically to ensure high search stability, robustness against premature stagnation, and superior generalization capability across diverse problem domains.

4.6. Experiments with the Propagation Method

Table 8 reports the per-dataset classification error obtained by the proposed OPSO algorithm under four alternative solution propagation strategies (NtoN, 1to1, 1toN, Nto1), evaluated across the same 40 benchmark datasets.

Table 8. Experiments using the proposed method and a series of propagation techniques. The number of processing threads was set to $T=5$.

DATASET	N to N	1 to 1	1 to N	N to 1
APPENDICITIS	22.80%	23.90%	23.30%	23.60%
ALCOHOL	22.13%	17.78%	20.85%	19.79%
AUSTRALIAN	24.43%	24.29%	25.19%	22.32%
BALANCE	6.94%	7.71%	8.08%	7.26%
CIRCULAR	4.34%	4.30%	4.25%	4.35%
CLEVELAND	45.69%	45.66%	47.59%	47.24%
DERMATOLOGY	9.97%	10.06%	10.40%	9.83%
ECOLI	50.67%	51.76%	50.33%	51.58%
FERT	25.50%	24.60%	25.00%	25.40%
HABERMAN	28.97%	29.27%	28.70%	29.53%
HAYES ROTH	35.00%	35.00%	37.31%	35.54%
HEART	21.11%	21.41%	20.18%	20.45%
HEARTATTACK	19.93%	21.53%	20.64%	21.13%
HOUSEVOTES	6.96%	6.48%	7.48%	7.61%
IONOSPHERE	14.86%	15.17%	15.37%	14.89%
LIVERDISORDER	31.94%	33.21%	32.47%	33.50%
LYMOGRAPHY	24.93%	26.07%	27.14%	26.21%
MAMMOGRAPHIC	17.06%	17.12%	17.19%	17.13%
OPTDIGITS	49.39%	49.04%	46.47%	53.38%
PARKINSONS	16.37%	15.95%	15.16%	15.90%
PHONEME	15.46%	14.91%	14.93%	14.17%
PIMA	27.88%	26.59%	26.92%	26.87%
PIRVISION	9.07%	10.52%	7.93%	9.15%
POPFAILURES	6.83%	6.80%	6.61%	7.04%
REGIONS2	31.11%	30.73%	29.40%	30.24%
SAHEART	33.87%	33.65%	33.96%	33.15%
SATIMAGE	47.47%	52.18%	53.19%	54.45%
SEGMENT	17.74%	20.68%	19.07%	18.94%
SONAR	19.80%	22.60%	21.15%	21.90%
SPAMBASE	6.19%	6.52%	6.33%	6.09%
SPIRAL	41.91%	42.48%	42.22%	41.72%
STATHEART	22.22%	21.89%	23.11%	20.37%
STUDENT	5.68%	6.23%	5.90%	6.18%
TRANSFUSION	24.45%	24.42%	24.37%	24.29%
WDBC	4.95%	5.30%	6.02%	5.73%
WINE	11.82%	14.24%	13.30%	14.65%
Z_F_S	9.00%	14.40%	10.63%	13.20%
ZO_NF_S	10.08%	10.64%	8.12%	10.16%
ZONF_S	2.68%	3.22%	2.90%	3.56%
ZOO	3.90%	4.50%	4.10%	5.50%
AVERAGE	20.78%	21.32%	21.08%	21.35%

The most immediately striking behavior in this table is the remarkable consistency of the four propagation strategies on the majority of datasets: for cases such as BALANCE, CIRCULAR, MAMMOGRAPHIC, TRANSFUSION, and PHONEME, the four strategies produce error values that agree to within one or two hundredths, and in some instances within a few thousandths. This tight clustering indicates that, for a large share of the benchmark problems, the specific topology through which good solutions are shared among islands has essentially no bearing on the final outcome, implying that the search process converges to a comparable solution regardless of how information circulates once the population size and number of iterations are held fixed.

A markedly different behavior emerges, however, on a smaller set of more demanding datasets. SATIMAGE shows the widest spread among all strategies (from 0.3717 to a high of 0.5445), and comparable, if less extreme, spreads appear for SEGMENT, WINE, Z_F_S, and ZO_NF_S. On most of these harder cases, the full-connectivity strategy (NtoN) tends to sit at or near the lowest error observed, consistent with the intuition that richer, more thorough mixing of good solutions across islands becomes increasingly valuable as the underlying optimization landscape grows more rugged or deceptive, while restricted propagation schemes may leave promising regions of the search space insufficiently exploited.

At the same time, no single propagation strategy dominates uniformly across the remaining, moderately difficult datasets: Nto1 achieves the lowest error on AUSTRALIAN, PHONEME, and STATHEART, 1to1 is best on HOUSEVOTES, and 1toN performs best on PARKINSONS and HEART, with the margins separating the four strategies in each of these cases remaining small. This idiosyncratic distribution of “wins” across strategies, rather than a consistent ranking, suggests that for datasets of intermediate difficulty the choice of propagation topology interacts with dataset-specific characteristics in ways that are not easily predictable from the properties of the strategy alone, reinforcing the impression that propagation topology becomes a decisive factor mainly at the extremes of problem difficulty rather than across the board.

A statistical presentation of the classification error of OPSO under four solution propagation strategies is presented in Figure 4.

Figure 4. Mean classification error of OPSO under four solution propagation strategies. Asterisks (*) denote a statistically significant pairwise difference (Holm-corrected $p<0.05$); ns denotes a non-significant pair.

A statistical comparison of the four propagation strategies relative to NtoN is provided in Table 9.

Table 9. Statistical comparison of the four propagation strategies relative to NtoN.

Propagation	Mean Error	SD Error	Min Error	Max Error	Avg Rank	Mean Diff vs. NtoN	Wilcoxon p	Paired p	Significance
NtoN	21.21	13.42	2.68	50.67	2.14	0	Reference	Reference	Reference
1toN	21.52	13.65	2.9	53.19	2.51	−0.31	0.1781	0.1931	n.s.
1to1	21.75	13.51	3.22	52.18	2.73	−0.54	0.0241	0.0411	*
Nto1	21.76	13.96	3.56	54.45	2.62	−0.55	0.0874	0.0571	n.s.

Friedman test: chi-squared = 4.589, df = 3, p-value = 0.2045. Nemenyi critical difference (alpha = 0.05): 0.751. Reference strategy for pairwise tests: NtoN (full propagation). Significance vs. NtoN (paired t-test): * p < 0.05, n.s. = not significant.

The statistical comparison for the obtained results is presented in Figure 5 and in Table 10. The Friedman test indicated an overall statistically significant effect of population size on algorithmic performance (${\chi }^2$ = 9.28, df = 3, p = 0.026, n = 40), with a Nemenyi critical difference of 0.742. A population size of $N_c=2000$ yielded the lowest mean error (19.86% ± 13.09) and the best average rank (2.08), while $N_c=1000$ did not differ significantly from this reference (20.51%, p = 0.140). Smaller populations, however, performed significantly worse: $N_c=500$ (20.78%, p = 0.013) and, more markedly, $N_c=200$ (21.45%, p = 0.009). confirmed that this effect is driven primarily by the smallest population size ($N_c=200$ vs. $N_c=1000$: paired t-test p = 0.011 (Holm-corrected); vs. $N_c=2000$: p = 0.045 (Holm-corrected)), whereas the differences among $N_c=500$, $N_c=1000$, and $N_c=2000$ did not remain significant after correction, suggesting diminishing returns beyond a population size of approximately 500–1000 individuals.

Figure 5. Mean classification error of OPSO under four population sizes. Asterisks (*) denote a statistically significant pairwise difference (Holm-corrected * p < 0.05, ** p < 0.01); ns denotes a non-significant pair.

Table 10. Statistical comparison of the four population sizes relative to $N_c$ = 2000.

${\mathit{N}}_{\mathit{c}}$	Mean Error	SD Error	Min Error	Max Error	Avg Rank	Mean Diff vs. ${\mathit{N}}_{\mathit{c}}$ = 2000	Wilcoxon p	Paired p	Significance
2000	19.86	13.09	2.96	50.76	2.08	0	Reference	Reference	Reference
1000	20.51	13.63	2.86	52.88	2.48	−0.65	0.1428	0.1401	n.s.
500	20.78	13.53	2.68	50.67	2.5	−0.91	0.0087	0.0132	*
200	21.45	13.86	3.16	58.17	2.95	−1.59	0.0005	0.0089	**

Friedman test: chi-squared = 9.28, df = 3, p-value = 0.0258. Nemenyi critical difference (alpha = 0.05): 0.742. Reference model for pairwise tests: $N_c$ = 2000 (auto-selected as the best-performing population size). Significance vs. reference (paired t-test): ** p < 0.01, * p < 0.05, n.s. = not significant.

4.7. Experiments with the Number of Particles

Table 11 reports the per-dataset classification error obtained by the proposed OPSO algorithm under four different population sizes $\left(N_c=200, 500, 1000, 2000\right)$, evaluated across the same 40 benchmark datasets.

Table 11. Experiments using the proposed method and a series of values for the number of particle $N_c.$ The propagation method was set to $NtoN$ and the number of processing threads was set to $T=5$. Reported values are the average classification error (%) on the test set for each dataset/population-size combination.

DATASET	${\mathit{N}}_{\mathit{c}}=200$	${\mathit{N}}_{\mathit{c}}=500$	${\mathit{N}}_{\mathit{c}}=1000$	${\mathit{N}}_{\mathit{c}}=2000$
APPENDICITIS	23.70%	22.80%	21.60%	21.40%
ALCOHOL	20.42%	22.13%	18.09%	14.89%
AUSTRALIAN	23.21%	24.43%	24.78%	23.13%
BALANCE	7.19%	6.94%	7.48%	7.36%
CIRCULAR	4.35%	4.34%	4.52%	4.56%
CLEVELAND	45.41%	45.69%	45.79%	45.72%
DERMATOLOGY	13.29%	9.97%	9.92%	10.03%
ECOLI	53.70%	50.67%	49.27%	50.76%
FERT	24.30%	25.50%	27.60%	26.50%
HABERMAN	28.43%	28.97%	29.37%	28.97%
HAYES ROTH	36.00%	35.00%	36.23%	37.69%
HEART	21.14%	21.11%	19.89%	17.78%
HEARTATTACK	21.63%	19.93%	19.47%	19.33%
HOUSEVOTES	6.61%	6.96%	6.70%	7.30%
IONOSPHERE	14.71%	14.86%	15.72%	13.94%
LIVERDISORDER	34.06%	31.94%	32.47%	32.47%
LYMOGRAPHY	25.86%	24.93%	24.36%	25.36%
MAMMOGRAPHIC	16.78%	17.06%	17.29%	17.26%
OPTDIGITS	44.06%	49.39%	44.43%	47.46%
PARKINSONS	15.79%	16.37%	16.26%	15.05%
PHONEME	15.06%	15.46%	14.59%	15.17%
PIMA	29.03%	27.88%	26.62%	27.04%
PIRVISION	11.11%	9.07%	7.72%	8.69%
POPFAILURES	6.78%	6.83%	7.04%	6.82%
REGIONS2	29.15%	31.11%	29.18%	27.79%
SAHEART	33.91%	33.87%	33.00%	32.67%
SATIMAGE	58.17%	47.47%	52.88%	37.17%
SEGMENT	19.83%	17.74%	16.46%	17.21%
SONAR	21.35%	19.80%	21.65%	20.15%
SPAMBASE	6.35%	6.19%	5.96%	6.33%
SPIRAL	42.04%	41.91%	42.41%	41.29%
STATHEART	23.96%	22.22%	21.96%	19.41%
STUDENT	6.50%	5.68%	5.45%	5.55%
TRANSFUSION	24.57%	24.45%	24.62%	24.16%
WDBC	5.61%	4.95%	5.39%	4.43%
WINE	14.12%	11.82%	10.59%	11.41%
Z_F_S	13.47%	9.00%	10.57%	8.73%
ZO_NF_S	8.68%	10.08%	6.38%	6.34%
ZONF_S	3.16%	2.68%	2.86%	2.96%
ZOO	4.50%	3.90%	3.90%	4.30%
AVERAGE	21.45%	20.78%	20.51%	19.86%

For a substantial portion of the datasets, increasing the population size from $N_c=200$ to $N_c=2000$ produces a visible, and in several cases fairly consistent, downward trend in error. HEART (from 0.2114 to 0.1778), STATHEART (from 0.2396 to 0.1941), HEARTATTACK (from 0.2163 to 0.1933), and WDBC (from 0.0561 to 0.0443) all display this steady improvement, indicating that a larger population provides these problems with a genuinely richer sampling of the search space that translates into more accurate final solutions rather than merely additional computational overhead.

A second, equally prominent behavior is the near-total insensitivity of certain datasets to population size. On MAMMOGRAPHIC, TRANSFUSION, HABERMAN, SAHEART, CLEVELAND, BALANCE, and CIRCULAR, the error stays within a narrow band of no more than roughly one hundredth across all four population sizes, suggesting that for these particular problems the search already converges to essentially the same solution with as few as 200 individuals, and that the additional computational cost associated with larger populations offers no discernible practical benefit for this class of dataset.

Finally, a smaller subset of datasets displays behavior that runs counter to the general trend, either through pronounced instability or through outright deterioration with a larger population. SATIMAGE stands out with the widest fluctuation observed anywhere in the table (ranging non-monotonically from 0.5817 down to 0.3717), while FERT (rising from 0.243 to 0.276 before partially recovering) and HAYES ROTH (lowest at $N_c=500$, then rising steadily to 0.3769 at $N_c=2000$) actually perform worse, on balance, with a larger population. This behavior indicates that the relationship between population size and solution quality is not universally monotonic, and that for certain datasets, likely those with noisier or more deceptive fitness landscapes, an overly large population may dilute selective pressure or slow convergence within a fixed number of iterations, underscoring that population size should ideally be tuned per problem rather than fixed at a single “optimal” value for all datasets.

The statistical comparison for the obtained results is presented in Figure 5 and in Table 10. The Friedman test indicated an overall statistically significant effect of population size on algorithmic performance (${\chi }^2$ = 9.28, df = 3, p = 0.026, n = 40), with a Nemenyi critical difference of 0.742. A population size of $N_c=2000$ yielded the lowest mean error (19.86% ± 13.09) and the best average rank (2.08), while $N_c=1000$ did not differ significantly from this reference (20.51%, p = 0.140). Smaller populations, however, performed significantly worse: $N_c=500$ (20.78%, p = 0.013) and, more markedly, $N_c=200$ (21.45%, p = 0.009). The complete, Holm-corrected pairwise comparisons confirmed that this effect is driven primarily by the smallest population size ($N_c=200$ vs. $N_c=1000$: paired t-test p = 0.011 (Holm-corrected); vs. $N_c=2000$: p = 0.045 (Holm-corrected)), whereas the differences among $N_c=500$, $N_c=1000$, and $N_c=2000$ did not remain significant after correction, suggesting diminishing returns beyond a population size of approximately 500–1000 individuals.

4.8. Benchmarking Against Island Model PSO Variant

To address the comparison with parallel island-based metaheuristics, the proposed OPSO algorithm was evaluated against the island model PSO variant proposed by [34] across 25 benchmark classification datasets under an identical parallel configuration of $T=5$ execution threads/islands. Table 12 summarizes the classification error rates (%) and performance differences ($\Delta =\mathrm{Proposed}-\mathrm{Variant}$) between the two methodologies.

Table 12. Classification error rate (%) comparison between OPSO and island model PSO variant [34] for $T=5$ threads across 25 classification datasets.

Dataset	Proposed Method (%) T = 5	Island Model PSO Variant (%) T = 5	Difference ($\mathbf{\Delta }$)
Alcohol	16.38	45.32	−28.94
Australian	22.74	34.58	−11.84
Circular	4.17	19.51	−15.34
Cleveland	44.48	61.76	−17.28
Dermatology	8.31	40.51	−32.20
Ecoli	44.88	55.68	−10.80
Fert	24.40	36.60	−12.20
Haberman	27.33	31.57	−4.24
Hayes-roth	37.23	41.85	−4.62
Heart	17.74	31.52	−13.78
HeartAttack	20.57	32.54	−11.97
Lymography	25.57	32.50	−6.93
Mammographic	17.45	19.75	−2.30
Optdigits	44.47	65.57	−21.10
Parkinsons	14.47	27.79	−13.32
Pima	25.45	37.09	−11.64
Regions2	29.27	38.08	−8.81
Saheart	32.09	39.67	−7.58
Satimage	32.77	47.49	−14.72
Segment	16.54	57.44	−40.90
Sonar	21.25	27.75	−6.50
Spambase	6.30	20.54	−14.24
StatHeart	19.41	29.85	−10.44
Wdbc	4.00	8.87	−4.87
Wine	11.59	24.12	−12.53

Performance Analysis

As evidenced by the empirical results, the proposed method consistently outperforms the island model PSO variant [34] across all 25 benchmark datasets.

The primary insights derived from this comparative analysis are summarized below:

• Overall Error Reduction: OPSO achieves an average classification error rate of 22.75%, compared to 36.32% for the baseline model [34], yielding a substantial average absolute error reduction of 13.56%.
• High-Dimensional and Complex Datasets: The most pronounced performance gains are observed in high-dimensional and complex datasets. Notably, OPSO achieves significant error reductions on Segment ($\Delta$ = —40.90%), Dermatology ($\Delta$ = —32.20%), Alcohol ($\Delta$ = —28.94%), and Optdigits ($\Delta$ = —21.10%).
• Exploration versus Exploitation Efficiency: While traditional island-based PSO variants [34] often suffer from premature convergence due to static topologies or fixed migration policies, OPSO’s dynamic interaction mechanisms facilitate a superior balance between global exploration and local exploitation across parallel threads. This capability enables OPSO to escape sub-optimal local regions more effectively.
• Consistency Across Domains: The performance superiority remains robust across diverse application domains, ranging from medical diagnostics (heart, Parkinsons, dermatology) to image and feature classification (Optdigits, Satimage, segmentation).

5. Conclusions

This paper introduced OPSO, a parallel, multi-population optimization algorithm for classification tasks, and evaluated it systematically against four established machine-learning models across 40 benchmark datasets drawn from the UCI and KEEL repositories. Across three thread/population configurations, OPSO consistently and significantly outperformed all four baseline models, with the $T=5$ configuration achieving the lowest overall mean error and the $T=2$ configuration performing statistically indistinguishably from it. Importantly, this advantage was not uniform across datasets: it was most pronounced on problems exhibiting irregular or multimodal error surfaces, precisely the class of problems where gradient-based or single-pass heuristics are known to struggle, while on datasets that were already comparatively easy for the baseline models, OPSO offered little to no additional benefit. This pattern suggests that the proposed method’s principal strength lies not in universally improving classification accuracy, but in providing a more capable search mechanism specifically for the harder, more deceptive regions of the problem space.

The two supplementary experiments clarified which internal design choices of OPSO meaningfully affect its performance and which do not. Increasing the population size produced a statistically significant improvement, though with clearly diminishing returns beyond roughly 500–1000 particles, and with a subset of datasets showing negligible or even mildly non-monotonic sensitivity to this parameter, indicating that population size, while important, should be regarded as a tunable, problem-dependent setting rather than a fixed universal choice. By contrast, the solution propagation strategy used to share promising candidates among subpopulations had no statistically significant overall effect, with the full-connectivity strategy (NtoN) offering only a marginal, largely non-significant advantage over more restricted topologies. Taken together, these results position OPSO as a robust and practically competitive alternative to conventional classifier-training approaches, whose effectiveness is driven more by the number and diversity of its parallel search threads and the richness of its populations than by the specific manner in which information is exchanged among them.

The present study, while comprehensive in its comparison across baseline models and internal configurations, leaves several directions open for further investigation. First, Section 4.4 provides quantitative support for this observation by correlating OPSO’s improvement with a baseline disagreement proxy for landscape ruggedness; extending this analysis with direct measures computed from the raw feature space, such as class-overlap indices or sampled loss-surface ruggedness metrics, would allow the conditions under which OPSO is expected to excel to be predicted a priori with even greater precision. Second, the non-monotonic or even adverse effect of increasing population size observed on a small number of datasets (e.g., FERT, HAYES ROTH) suggests that an adaptive or self-tuning population size, adjusted dynamically during the search based on convergence behavior, may outperform any single fixed value and merits dedicated investigation. Third, although the propagation strategy was found to have limited overall impact, its comparatively larger effect on the hardest benchmark problems (e.g., SATIMAGE, SEGMENT) indicates that hybrid or adaptive propagation schemes, which shift between restricted and fully connected topologies depending on the observed diversity or stagnation of the population, could yield further improvements specifically on difficult problems, and should be explored in future work. Finally, extending the evaluation beyond the 40 datasets considered here to larger-scale, higher-dimensional, or streaming classification problems, and benchmarking OPSO against a broader range of modern evolutionary and swarm-based optimizers, would help establish the generality of the conclusions reached in this study and clarify the computational trade-offs involved in deploying OPSO at scale.