Performance Analysis of Partitioned Step Particle Swarm Optimization in Function Evaluation

Abstract

The partitioned step particle swarm optimization (PSPSO) introduces a two-fold searching mechanism that increases the search capability of Particle Swarm Optimization. The first layer involves the γ and λ, values which are introduced to describe the current condition of characteristics of the searched solution that diversifies the particles when it is converging too much on some optima. The second layer involves the partitioning of particles that tries to prevent premature convergence. With the two search mechanisms, the PSPSO presents a simpler way of making the particles communicate with each other without too much compromise of the computational time. The proposed algorithm was compared with different variants of particle swarm optimization (PSO) using benchmark functions as well as the IEEE 10-unit unit commitment problem. Results proved the effectiveness of PSPSO with different functions and proved its competitive advantage in comparison with published PSO variants.

1. Introduction

Heuristic, which means “to find” or “to discover by trial and error”, is a way of producing an acceptable solution to a complex problem in a reasonable time [1]. Moreover, metaheuristic algorithms are the further development of heuristic algorithms. In the literature, there is no clear distinction between these algorithms, and the terms are usually interchangeable. Although heuristic methods use randomness to search for the solution, these methods are flexible enough to search through non-smooth space where mathematical methods have difficulty [2].

Metaheuristic algorithms, at times called intelligent search methods [3], are methods patterned on certain behaviors in our environment [4]: genetic algorithm (GA), which was based on evolution; particle swarm optimization (PSO) and firefly algorithm (FA) are examples of swarm intelligence; gravitational search algorithm (GSA) is a physics-based algorithm; teaching-learning based optimization (TLBO) and harmony search(HS) are based on human behavior and a never-ending list of algorithms that are patterned to different behaviors of organisms.

The success of a metaheuristic algorithm depends on how it balances the exploration and exploitation capability of each algorithm [5]. Exploration ensures the diversity of the solution by searching on a global scale, while exploitation focuses on the search on the local region where good solutions are. Algorithms provide different weights on these two terms. Another unique characteristic of algorithms is how they perform their random walk that determines a new and, hopefully, a better solution. Certain components of the algorithm that also helps in its movement, such as the inertia weight in PSO or the differential weight in differential evolution (DE) [6], the circling movement around the target in whale algorithm [7], and the use of a quantum bit to store the direction and magnitude of rotation angle in quantum-inspired algorithms (QEA) [8]. Mostly these random walks provide a unique movement that helps in the exploration and exploitation.

1.1. Review of Related Algorithms

The three algorithms discussed below are the inspiration of the partitioned step particle swarm optimization (PSPSO). The concepts and behavior of these algorithms are used as a guide to develope the PSO variant.

1.1.1. Genetic Algorithm

John Holland’s GA in 1975 became the foundation of modern evolutionary intelligent search methods [1]. It is based on Darwinian theory on natural selection–crossover, mutation, and selection. Each individual in the population is composed of chromosomes that are encrypted with data and are, then, evolved through recombination with other particles or mutation. These individuals are then evaluated according to their fitness and randomly selected on the basis of this fitness value as their probability of being selected to generate the next population or children. Because of the complexity of the GA, finding the global optimal in a wide variety of functions has been a great success but at the expense of large calculation time [3].

1.1.2. Particle Swarm Optimization

PSO was introduced by Kennedy and Eberhart [9]. This algorithm simulates the social behavior of flocking birds and schooling fish. It uses the autobiographical memory of the particle (pbest), described as the inclination of the individual to return to the place that is more satisfying in the past, and the target (gbest). Each particle represents a solution for the problem and the collective of particles is called the swarm. For every iteration, the velocity of the particles is determined on the basis of the distance of a particle to its current best (pbest), overall best (gbest), and previous direction. The new position of the swarm is determined by moving the particles according to their calculated velocities [1].

The structure of PSO seems simple enough with its few parameters and easy implementation. However, PSO usually suffers from premature convergence [2] or being stuck at the local optima since it always moves towards the pbest and gbest, which may simply be just the local optima [3]. It has problems with scattering its particles or exploration. Hence, balancing exploration and exploitation needs to be studied for PSO.

1.1.3. Harmony Search

HS was introduced by Geem, Lee, and Loganathan [10]. It was inspired by how the music player improvises music to create better harmony. A generation of harmony can be done in three basic steps: (1) playing by memory, (2) playing a variation of the harmony, or (3) composing new harmony. With its simple structure, fast convergence, and a good balance between exploration and exploitation [5], HS has been widely implemented and combined with other algorithms such as random search [10], PSO [11], virus optimization algorithm [12], etc.

The format of HS is not quite as similar to the other algorithms, but concepts have some equivalent to other algorithms, such as GA’s elitism and mutation and PSO’s simplicity. In HS, the harmony is composed of different melodies (solution). The harmony is divided into harmony memory, a variation of the memory, and new melodies through initializing the percentage of each of these components. Better melodies are, then, saved in the harmony memory every iteration [1]. This concept of harmony division is done using a random selector variable that dictates what type of harmony a certain solution is, which is the inspiration for the proposed partitioned step particle swarm optimization (PSPSO).

From the discussions above, it can be observed that there are four main components in an algorithm: the step, elitism, crossover, and mutation. In GA, each of these four components can easily be distinguished in its algorithm as they are separated. In HS, the step is merged with memory, variation, and composition, representing elitism, crossover, and mutation, respectively, in one equation. In a standard PSO, the representation of each of these components is not too obvious; the velocity computation is the step, within this step is its elitism (pbest and gbest) and its limited mutation (the random multiplier on the pbest and gbest). The crossover done by PSO is only with its pbest and gbest–this means that particles only move toward the particle’s best value and the current global best. While this can mean that each particle independently operates, lack of knowledge about other pbest always maintains the velocity of the particle a little towards the gbest. This is the main reason why PSO suffers from premature convergence. Therefore, the PSO particles should learn how to communicate with other particles to improve their search capability.

The PSPSO introduces a fast way of incorporating crossover and mutation without compromising the computational time. Two layers of search are also introduced. The first layer handles the direction of the inertia weight that is responsible for the partial mutation in PSO. The second is through a modified step equation that partitions the crossover of pbest and gbest, and pbest and another pbest. The proposed algorithm is then compared to different variants of the PSO in terms of the function’s characteristics and computational time.

This paper analyzes the effectivity of PSPSO in solving different benchmark problems as it was not done when it was first introduced by the authors in their paper [13]. This paper provides a better understanding as to how and why it is capable of solving such problems through conducting further tests on the PSPSO. Furthermore, only PSO variants were tested in this paper because we wanted to show its improvement on the PSO algorithm. The paper is organized as follows. Section 1.2 discusses in detail the components of PSO and its known variants found in the literature. Section 2 discusses the PSPSO algorithm and testing methods. Parameter setting results and function evaluation results are discussed in Section 3. Section 4 discusses the possible implications of the use of this algorithm. Lastly, Section 5 discusses the conclusion.

1.2. Particle Swarm Optimization Components

There are several studies conducted that focuses on the enhancement of the PSO algorithm through the following components.

1.2.1. Velocity Rule

The velocity rule is how the particles of the algorithm determine their next update or next step. Originally, PSO [9] uses three components for the step; the previous velocity, the particle best (pbest), and the global best (gbest). The inertia weight variant later introduced by Shi and Eberhart [14] is the recent widely used variant that follows the equation

$$ve{l}_{new}=\omega ve{l}_{old}+c_{1}rand\left(pbest-x\right)+c_{2}rand\left(gbest-x\right)$$

(1)

where c₁ and c₂ are the cognitive and cooperative constants, respectively; x is the particle position, ω is the inertia weight, and vel is the velocity of the particle.

However, as discussed in Bonyadi and Michalewics [15], the weighted velocity variant discussed above is locally convergent and may suffer from premature convergence. Other papers tried to mitigate this problem by changing the value of the weight and/or changing the velocity rule. One notable method is removing the pbest component and just use the previous velocity and the global best for the update [16,17] which showed better results, especially in multimodal functions.

Another unique variant uses an additional component to the original PSO. Weight-Improved Crazy PSO (WICPSO) [18] uses a bad experience component (pworst) to help the particle move away from the not so good position.

1.2.2. Inertia Weight

From the discussion of Shi and Eberhart [16], the velocity is intended to expand the search space, that the particles have in terms of ability to explore a new area. The inertia weight is responsible for controlling the previous velocity, thereby, controlling the movement of particles toward the previous velocity. In addition, this inertia weight is also responsible for balancing the global search (toward gbest) and local search (toward pbest).

Because of the inertia weight’s control on the exploration of the PSO, several PSO variants were performed to modify the value of ω. The crazy PSO (CPSO) [14] uses a probability of craziness to determine if the new velocity will be used or if it would be a random number from zero to the pre-set maximum velocity. For both CPSO and weight-improved PSO (WIPSO) [19], the value of ω changes on the basis of the current iteration that decreases as the iteration increases. This decrease in the change of the ω value is necessary to make sure that the algorithm converges. The generalized PSO (GPSO) [20], likewise, changes in the inertia weight depending on the iteration. However, instead of a craziness component, the GPSO uses the previous gbest and current gbest to decide how much of the previous inertia would be used. However, if the GPSO got stuck in a local minimum, the inertia would not change. The GPSO compensates this by including the position of a random particle and a velocity of another random particle as the fourth and fifth terms in their velocity update, leading to too much exploration.

The adaptive PSO (APSO) has been presented in several pieces of literature as the consideration of the current state of particles or sparseness of the solution for exploration. Li et al. [21] used the distances between particles to compute the local sparseness degree to select the exemplars of each group of particles as pbest particles. Although Li et al.’s APSO seems a good way to select pbest particles, the presented algorithm itself uses long complicated steps. In the work of Han et al. [22], the APSO used additional constant parameters to adjust the degree of sparseness than then adjusted the value of ω. Han et al.’s variant seems to be promising because the particles were moved according to the current state of the solution rather than being limited by a range of ω or relying on randomness as in like the CPSO and WIPSO.

1.2.3. Crossover

Seeing a crossover operation in PSO is not usual because the velocity update itself seems to be the crossover operator—a crossover with its pbest and gbest [9]. In the paper of Chen et al. [17], discusses that the crossover enhances the information sharing between particles and prevents premature convergence. They have proposed the PSO with Crossover Operation (PSOCO), which uses two types of crossover operation, arithmetic crossover and differential evolution crossover. They also combined the concept of exemplars as a replacement strategy for stagnated value and competitive selection from differential evolution resulting in a faster convergence. However, arriving at the solution takes longer than usual due to the use of multiple procedures and function evaluation in every iteration.

2. Partitioned Step Particle Swarm Optimization

2.1. Theory

The PSPSO incorporates adaptive components and crossover on the basis of the current state of the particles. In Li et al.’s [21] concept of APSO, the adaptive component is based on the congestion and distribution of particles. For the PSPSO, these two conditions are reformulated as the λ and γ components, acting similar to congestion and distribution components, respectively. The λ component indicates the sparseness of the solution in the current iteration; the larger the λ, the much closer are the fitness values. The γ component indicates how far is the fitness is the current best solution; the larger the value, the more near the particle is to the current best solution.

$$\lambda =\raisebox{1ex}{$f{\left(x\right)}_{min}$}\!\left/ \!\raisebox{-1ex}{$f{\left(x\right)}_{max}$}\right.$$

(2)

$$\gamma =1-\frac{f\left(x\right)-f\left(gbest\right)}{f{\left(x\right)}_{max}-f\left(gbest\right)}$$

(3)

$$\omega =\lambda \gamma \frac{\left(MaxIter-i\right)}{MaxIter}$$

(4)

Equations (2)–(4) show the computation of the three components, λ, γ, and ω, respectively, where x is particle position, f(x) is the fitness function, MaxIter is the maximum iteration, and i is the iteration number. At the beginning of the algorithm when the sparseness is still high, the particles will try to improve themselves by moving towards the known better solutions, pbest and gbest. When the particles suddenly try to converge together, ω will be high, which means that the algorithm will do more exploration. In other studies, the ω is referred to as the “mutation” of PSO; when the ω is high, the algorithm searches more the solution space. This means that the PSPSO will try to use the previous trajectory more to look for other solutions or to diversify.

Traditional random walk for PSO includes the previous velocity, pbest, and gbest. Nonetheless, some research suggests that including gbest makes the algorithm converge prematurely. To remedy this, the probability of using gbest is decreased at the start of the iterations using the partition (p) (5). Even though the use of gbest is reduced, the cooperative component will remain throughout the iteration through the use of a random particle’s pbest (pbest_rand). The velocity (6) would have two types of particles, the crossover of pbest and gbest, and the crossover of pbest and pbest_rand. The vector representation for velocity (6) and particle movement update (7) is seen in Figure 1a.

$$p=rand<\left(\frac{i}{MaxIter}\right)$$

(5)

$$ve{l}_{new}=\omega ve{l}_{old}+c_{1}rand\left(pbest-x\right)+c_{2}rand\left(p gbest+\left(1-p\right)pbes{t}_{rnd}-x\right)$$

(6)

$$par{t}_{new}=par{t}_{old}+ve{l}_{new}$$

(7)

Figure 1b shows the flowchart of the PSPSO, with the blue boxes indicating the modified equations for the PSPSO.

2.2. Parameter Setting

There are only two parameters for PSPSO—cognitive-cooperative constants (c₁ and c₂) and the number of particles. These parameters are determined by performing a brutal search, first, on the constants, then the number of parameters. The search range for the constants is from 1.5 to 2.5 with 0.25 increments, while the number of particles (N) is 30 to 60 with increments of 10. The mean, standard deviation, and computational time for the 30 runs of all the test functions are recorded. The parameter setting that has the best or lowest results in terms of mean, standard deviation, and computational time is then chosen.

2.3. Performance Tests

2.3.1. PSO Variants

Five PSO variants were used to compare the performance of the proposed PSPSO, namely, PSO [16], CPSO [14], PSOCO [17], WICPSO [18], APSO [21], and scout PSO (ScPSO) [23]. Each of these algorithms focus on a unique component of PSO, either in the velocity rule, inertia weight, crossover, or mutation. The CPSO introduced the craziness probability to adjust the inertia weight. WICPSO improved the CPSO by introducing the pworst component in the velocity rule but was not tested for global search. Li et al.’s APSO considered the relative positions of the particles in the movement. Lastly, the ScPSO uses the mutation criteria based on the artificial bee colony (ABC) algorithm.

All the variants were coded in MATLAB R2020a on an Intel i5 Core 3.2GHz with 8GB RAM. The initialized particles use the Halton quasirandom point set for uniform distribution on the solution space and uniformity of all algorithms. Parameter settings for each of the algorithms are shown in

Table 1. Parameters for compared algorithms.

Algorithms	Parameter Setting
PSO	c1 = c2 = 2, ωmin = 0.4, ωmax = 0.9
CPSO	c1 = c2 = 2, ωmin = 0.4, ωmax = 0.9
PSOCO	c = 1.49618, ω = 0.7298, CR = 0.05, G = 7
WICPSO	c1max = 2.2, c1min = 1.5, c1max = 2.2, c1min = 1.5, ωmin = 0.4, ωmax = 0.9
APSO	c1 = c2 = 2, L = 3, C = 1
ScPSO	c1 = c2 = 2, ωmin = 0.4, ωmax = 0.9

. The number of particles (N) and maximum iteration is set to 50 and 1000*dimension, respectively.

2.3.2. Benchmark Functions

The list of unconstrained functions can be found in

Table 2. Benchmark functions (Unconstrained).

No.	Function Name	Characteristics	Range
f1	Sphere	C, D, S, Sc, U	[−10, 10]
f2	Rosenbrock	C, D, NS, Sc, U	[−30, 30]
f3	Chung Reynolds	C, D, PS, Sc, U	[−100, 100]
f4	Dixon and Price	C, D, NS, Sc, U	[−10, 10]
f5	Powell Singular	C, D, NS, Sc, U	[−4, 5]
f6	Step	DC, ND, S, Sc, U	[−100, 100]
f7	Cosine Mixture	DC, ND, S, Sc, M	[−1, 1]
f8	Ackley 1	C, D, NS, Sc, M	[−35, 35]
f9	Alpine	C, ND, S, NSc, M	[−10, 10]
f10	Egg Holder	C, D, NS, Sc, M	[−512, 512]
f11	Griewank	C, D, NS, Sc, M	[−100, 100]
f12	Pathological	C, D, NS, NSc, M	[−100, 100]
f13	Weierstrass	C, D, S, Sc, M	[−0.5, 0.5]
f14	Schwefel	C, D, S, NSc, M	[−500, 500]
f15	Pinter	C, D, NS, Sc, M	[−100, 100]
f16	Qing	C, D, S, Sc, M	[−500, 500]
f17	Quintic	C, D, S, NSc, M	[−10, 10]
f18	Stretched V Sine Wave Function	C, D, NS, Sc, U	[−10, 10]
f19	Styblinski-Tang	C, D, NS, NSc, M	[−5, 5]
f20	Trigonometric 2	C, D, NS, Sc, M	[−500, 500]
Nomenclature: Continuous (C), Differentiable (D), Discontinuous (DC), Multimodal (M), Non-Differentiable (ND), Non-Separable (NS), Non-Scalable (NSc), Partially Separable (PS), Separable (S), Scalable (Sc), and Unimodal (U)

, with its corresponding property taken from [24]. Moreover, five benchmarks constrained optimization problems were also used for the testing, as shown in

Table 3. Benchmark functions (Constrained).

No.	Function Name	Dimension
c1	Rosenbrock [31]	2
c2	Bird Problem [32]	2
c3	Townsend [33]	2
c4	G01 [34]	13
c5	G02 [34]	20

.

2.3.3. Unit Commitment Problem

The PSPSO is, likewise, applied to solve the IEEE 10-unit unit commitment (UC) problem, which is a non-convex, large-scale, non-linear, and mixed-integer combinatorial problem. There are several attempts to solve the IEEE10-unit UC problem in literature using WICPSO [18], hybrid genetic-imperialist competitive algorithm (HGICA) [25], GA [26], quantum-inspired binary grey wolf optimizer (QBGWO) [27], binary artificial sheep algorithm (BASA) [28], binary grey wolf optimizer (BGWO) [29], and MILP-based stochastic multi-objective UC (SMOUC) using one scenario and single objective base case [30].

The UC problem deals with the 24-h scheduling of generators, deciding on when a generator turns on or off (unit commitment decision) and what amount of power should be scheduled for each generator (economic dispatch decision) in order to have the least operating cost. The generator constants and load are listed in

Table 4. Generator data.

Gen	MAX OP (kW)	MIN OP (kW)	a ($)	b ($/MW)	c ($/MW2)	Hot Start Cost (HSC)	Cold Start Cost (CSC)	Cold Start Hour (CSH)	MUT (h)	MDT (h)	tini (h)	Ramp Up/ Down (kW)
1	455	150	1000	16.19	0.00048	4500	9000	5	8	8	8	160
2	455	150	970	17.26	0.00031	5000	10,000	5	8	8	8	160
3	130	20	700	16.60	0.00200	550	1100	5	5	4	−5	100
4	130	20	680	16.50	0.00211	560	1120	5	5	4	−5	100
5	162	25	450	19.70	0.00398	900	1800	6	6	4	−6	100
6	80	20	370	22.25	0.00712	170	340	2	3	3	−3	60
7	85	25	480	27.74	0.00079	260	520	2	3	3	−3	60
8	55	10	660	25.92	0.00413	30	60	0	1	1	−1	40
9	55	10	665	27.27	0.00222	30	60	0	1	1	−1	40
10	55	10	670	27.79	0.00173	30	60	0	1	1	−1	40

and

Table 5. Load Data.

Hour	1	2	3	4	5	6	7	8	9	10	11	12
Load	700	750	850	950	1000	1100	1150	1200	1300	1400	1450	1500
Hour	13	14	15	16	17	18	19	20	21	22	23	24
Load	1400	1300	1200	1050	1000	1100	1200	1400	1300	1100	900	800

.

Table 6. Ten-unit unit commitment problem.

Objective$\hspace{1em}\hspace{1em}\min \left({{\displaystyle \sum }}_{t=1}^{T}Fuel\left(t\right)+SU\left(t\right)\right)$	6-a
s.t. $Fuel\left(t\right)={\displaystyle \sum _{i=1}^{ngen}}U{C}_i\left(t\right)\left(a_i{P}_{DG,i}^2\left(t\right)+b_i{P}_{DG,i}\left(t\right)+c_i\right)$	6-b
${\displaystyle \sum _i^{ngen}}{P}_{DG,i}\left(t\right)=P_L\left(t\right)$	6-c
$L{B}_i\left(t\right)\le P_{DG,i}\left(t\right)\le U{B}_i\left(t\right)$ $\left|P_{DG,i}\left(t\right)-P_{DG,i}\left(t-1\right)\right|\le Rm{p}_{DG,i}$	6-d 6-e
$t_{ON,i}\le MU{T}_i ; t_{OFF,i}\le MD{T}_i$	6-f
$S{U}_i\left(t\right)=\left\{\begin{array}{cc}HS{C}_i& MD{T}_i\le t_{OFF,i}\le MD{T}_i+CS{H}_i\\ CS{C}_i& MD{T}_i+CS{H}_i<t_{OFF,i}\end{array}\right.$	6-g
${\displaystyle \sum _{i=1}^{ngen}}U{C}_i\left(t\right)\left(P_{max,i}\left(t\right)-P_{DG,i}\left(t\right)\right)\ge SR\left(t\right)$	6-h

lists the objective function and constraints of the UC problem. The objective, as found in Table 6-a, is to minimize sum of the fuel cost and start-up (SU) cost of the generators for the entire period. It is subject to seven system constraints. The total fuel cost at each hour t is equal the summation of the fuel cost of each generator i given their fuel constants (a, b, and c) (Table 6-b). UC_i is the commitment of generator i; a UC value of 1 means that the generator is turned on, and 0 means the generator is turned off. The power balance of the system must be maintained at each hour such that the summation of the power generated by each generator P_DG must be equal to the load demand P_L at a particular hour (Table 6-c). The generator power output should not exceed the maximum (UB) and minimum (LB) operating limits of each generator (Table 6-d). The ramping up and down of power should not exceed the generator’s ramping limits Rmp_DG (Table 6-e). The generators have a minimum up time (MUT) and minimum down time (MDT) where the generator cannot turn off and turn on, respectively (Table 6-f). The shutdown costs of each generator depend on the cold start hour (CSH) of each generator (Table 6-g). Finally, the scheduled reserve (10% of the load) at each hour (Table 6-h) should be available.

2.3.4. Function Evaluation Tests

The test is conducted by running the function evaluation 30 times with 30, 40, and 50 unknown variables. The number of iterations is set to 1000*dimension for all algorithms. The average of the runs of each algorithm is recorded. The average value, standard deviation, and running time for each function are, likewise, recorded.

The best algorithm for each function characteristic is determined by obtaining the average ranking of the values of every algorithm for every function characteristic. The overall best algorithm is determined by simply obtaining the average ranking of each algorithm for all functions.

3. Data and Results

3.1. Variables and Parameter Setting

Table 7. Parameter setting results using functions with zero as global minima

(a) N = 50				(b) C1 = C2 = 1.50
C	Mean	Std Dev	Time (s)	N	Mean	Std Dev	Time(s)
1.00	65.6091	14.3104	8.773	30	1.0940	5.1989	5.303
1.25	30.4968	29.3233	8.020	40	0.0546	0.0282	6.353
1.50	0.0447	0.0006	7.625	50	0.0446	0.0006	7.591
1.75	2.5755	0.7062	9.687	60	0.0445	0.0002	8.868
2.00	13,050.09	15,406.5	10.373	70	0.0444	7.16 × 10−6	10.131
2.25	5.69 × 1029	3.11 × 1030	10.358	80	0.0445	1.37 × 10−4	11.622
2.50	1.07 × 1036	2.64 × 1036	10.107	90	0.0445	9.62 × 10−5	12.995

Note: In bold are the results for the chosen c and N parameters.

shows the results for varying c₁ or c₂ value from 1.00 to 2.50 (with 0.25 increments) while making the number of particles constant at 50 and varying the number of particles N (with 10 increments) while keeping the c₁ or c₂ value constant at 1.50. These are evaluated using all functions in Table 2 and Table 3 with zero as optimal value.

On the basis of the data in Table 7a, we can conclude that 1.50 is the most suitable constant parameter since it has the lowest mean, standard deviation, and computational time. Increasing the c value increases all the criteria values. Decreasing its value also increases the all the criteria values but not as much as increasing the c value.

Using 1.5 as the constant value, as seen in Table 7b, N = 50 can be selected as the number of parameters for PSPSO, since increasing N does not give appreciable change in the mean and standard deviation. Although N = 70 has the lowest mean and standard deviation, the increase in computational time is large compared to the decrease in mean and standard deviation.

3.2. PSPSO

Figure 2 shows the change in the fitness f(x) of a selected random particle with its ω, γ and λ. Notice that when the diversity of the particles is high during the start of the iteration (high λ and γ), ω will also be in its higher value. When the particle moves closer to the best solution, the value of ω increases, which means that the algorithm tries to explore more and does not come too near the best solution at the earlier iterations. This can happen in any iteration, which can be seen in the pulses in ω. These pulses will be larger when a bad solution for a particle is found, moving the particle away from the bad solution by giving a larger ω. Convergence is also assured as the value of ω decreases as the iteration increases.

3.3. Benchmark Functions

Figure 3 shows the average convergence graph for the 30 runs for Rosenbrock, Pinter, G02, and Pathological functions of PSPSO compared with other PSO variants. In almost all functions, the PSPSO was able to find the solution before 25% of the total iterations. Of all the benchmark functions only in (f12) Pathological and (f19) Styblinski– Tang was the PSPSO not able to converge in less than 25% of the total iterations. However, for the Pathological function, the PSPSO was still able to outperform the others in finding a better solution in the end.

The mean cost, standard deviation (std) of the costs, and average computational time in seconds of the 30 runs are recoded in

Table 8. Results of benchmark unconstrained functions (D = 30).

		PSPSO	PSOCO	CPSO	WICPSO	APSO	ScPSO	PSO			PSPSO	PSOCO	CPSO	WICPSO	APSO	ScPSO	PSO
f1	mean	0.0 × 100	5.6 × 10-296	6.7 × 102	7.0 × 104	3.9 × 102	1.4 × 10-208	3.3 × 102	f11	mean	1.3 × 10-3	0.0 × 100	1.9 × 10-1	1.8 × 101	2.8 × 10-1	1.9 × 10-2	1.8 × 10-2
	std	0.0 × 100	0.0 × 100	2.5 × 103	2.2 × 103	1.8 × 103	0.0 × 100	1.8 × 103		std	3.0 × 10-3	0.0 × 100	6.8 × 10-1	6.3 × 10-1	7.2 × 10-1	2.5 × 10-2	2.0 × 10-2
	time	2.597	5.746	2.84	3.264	1.805	2.423	1.778		time	4.741	10.487	5.596	5.933	3.107	4.845	3.837
f2	mean	5.0 × 10-4	2.5 × 10-2	1.7 × 104	8.9 × 105	4.6 × 104	1.5 × 105	1.1 × 105	f12	mean	2.2 × 100	3.6 × 100	5.5 × 100	4.3 × 100	5.3 × 100	6.4 × 100	6.4 × 100
	std	1.6 × 10-3	3.5 × 10-2	2.5 × 104	7.2 × 104	5.7 × 104	7.6 × 104	7.2 × 104		std	6.9 × 10-1	3.9 × 10-1	1.0 × 100	4.7 × 10-1	8.7 × 10-1	7.5 × 10-01	7.0 × 10-1
	time	2.766	6.956	3.028	3.4	1.957	2.76	2.058		time	4.678	16.267	8.084	5.603	3.875	6.406	5.487
f3	mean	0.0 × 100	0.0 × 100	3.3 × 106	4.9 × 109	3.3 × 106	1.0 × 107	3.3 × 106	f13	mean	0.0 × 100	6.9 × 10-1	2.3 × 100	5.2 × 101	7.1 × 100	3.6 × 100	3.2 × 100
	std	0.0 × 100	0.0 × 100	1.8 × 107	3.2 × 108	1.8 × 107	3.1 × 107	1.8 × 107		std	0.0 × 100	4.6 × 10-1	2.5 × 100	6.1 × 10-01	4.5 × 100	3.0 × 100	3.5 × 100
	time	2.712	5.459	2.796	3.292	1.867	2.488	1.798		time	58.365	305.531	94.078	76.381	86.413	92.411	91.694
f4	mean	6.7 × 10-1	3.2 × 10-1	3.3 × 104	1.7 × 106	2.4 × 101	5.7 × 104	4.5 × 104	f14	mean	1.2 × 10-219	5.6 × 10-165	7.3 × 102	1.9 × 1040	6.1 × 102	7.7 × 102	7.2 × 102
	std	0.0 × 100	3.3 × 10-1	5.0 × 104	3.1 × 104	3.8 × 101	6.5 × 104	6.1 × 104		std	0.0 × 100	0.0 × 100	2.1 × 102	1.4 × 1040	2.1 × 102	2.2 × 102	1.9 × 102
	time	2.665	9.563	3.536	3.435	1.992	2.889	2.214		time	3.616	8.159	4.998	4.067	2.298	3.884	2.982
f5	mean	2.3 × 10-7	3.5 × 10-4	5.7 × 102	5.0 × 103	6.5 × 101	9.4 × 102	8.9 × 102	f15	mean	0.0 × 100	2.8 × 101	2.3 × 103	1.8 × 104	1.6 × 103	1.9 × 103	2.3 × 103
	std	1.5 × 10-7	1.2 × 10-4	3.4 × 102	6.0 × 102	6.8 × 101	6.7 × 102	6.9 × 102		std	0.0 × 100	1.5 × 102	1.1 × 103	2.1 × 102	1.3 × 103	9.4 × 102	9.5 × 102
	time	6.293	19.669	7.419	6.363	5.773	6.217	5.314		time	10.028	19.846	8.521	11.07	6.231	7.882	7.23
f6	mean	0.0 × 100	0.0 × 100	1.0 × 103	7.0 × 104	3.6 × 102	3.3 × 102	6.7 × 102	f16	mean	1.3 × 10-21	1.2 × 10-28	1.4 × 10-28	2.2 × 1011	9.7 × 10-11	1.5 × 10-28	1.5 × 10-28
	std	0.0 × 100	0.0 × 100	3.1 × 103	1.5 × 103	1.8 × 103	1.8 × 103	2.5 × 103		std	7.3 × 10-21	0.0 × 100	2.9 × 10-29	2.2 × 109	5.2 × 10-10	5.1 × 10-29	5.5 × 10-29
	time	3.698	9.278	6.12	5.21	2.908	4.684	3.67		time	4.017	9.733	4.133	4.227	2.341	3.385	2.55
f7	mean	-1.8 × 100	-1.8 × 100	-1.8 × 100	-1.8 × 100	-1.8 × 100	-1.8 × 100	-1.8 × 100	f17	mean	1.5 × 10-17	0.0 × 100	1.6 × 10-15	2.3 × 105	5.3 × 10-1	4.6 × 10-16	2.0 × 10-15
	std	9 × 10-16	9 × 10-16	9 × 10-16	9 × 10-16	9 × 10-16	9 × 10-16	9.0 × 10-16		std	8.1 × 10-17	0.0 × 100	2.3 × 10-15	1.6 × 104	1.9 × 100	1.1 × 10-15	3.7 × 10-15
	time	0.094	0.252	0.089	0.11	0.073	0.091	0.065		time	9.248	64.709	21.968	23.553	21.128	20.731	19.947
f8	mean	3.7 × 10-15	3.8 × 10-15	2.0 × 101	2.0 × 101	1.8 × 101	2.0 × 101	2.0 × 101	f18	mean	0.0 × 100	2.3 × 10-37	2.0 × 10-38	3.0 × 10-1	0.0 × 100	2.3 × 10-38	2.6 × 10-38
	std	6.5 × 10-16	9.0 × 10-16	2.1 × 10-3	8.2 × 10-4	5.1 × 100	1.5 × 10-3	1.7 × 10-03		std	0.0 × 100	9.9 × 10-37	4.2 × 10-38	9.2 × 10-2	0.0 × 100	4.5 × 10-38	4.6 × 10-38
	time	4.015	9.64	7.242	5.462	3.545	5.993	4.942		time	0.118	0.335	0.109	0.13	0.077	0.114	0.088
f9	mean	5.8 × 10-20	4.2 × 10-9	5.2 × 100	5.3 × 101	4.1 × 10-1	6.0 × 100	5.3 × 100	f19	mean	-1.1 × 103	-1.2 × 103	-1.2 × 103	-4.1 × 102	-1.1 × 103	-1.2 × 103	-1.2 × 103
	std	3.2 × 10-19	1.6 × 1008	4.2 × 100	3.6 × 10-14	1.0 × 100	4.0 × 100	4.3 × 100		std	1.7 × 101	4.2 × 10-13	1.0 × 101	7.9 × 100	4.1 × 101	1.1 × 101	1.1 × 101
	time	4.089	11.099	5.508	5.503	3.254	4.73	3.74		time	9.642	29.122	9.991	10.628	3.294	9.697	8.744
f10	mean	-959.6	-958	-959.6	-959.6	-959.6	-943.5	-951.6	f20	mean	1.0 × 100	1.0 × 100	1.7 × 104	1.8 × 106	1.7 × 104	1.7 × 104	4.5 × 100
	std	5.8 × 10-13	5.4 × 100	5.8 × 10-13	2.8 × 10-5	5.8 × 10-13	6.1 × 101	4.4 × 101		std	1.0 × 10-16	9.2 × 10-17	6.3 × 104	4.1 × 104	6.3 × 104	6.3 × 104	3.4 × 100
	time	0.096	0.267	0.09	0.113	0.078	0.096	0.07		time	5.278	10.321	6.108	15.027	3.591	5.658	4.728

Note: time is in seconds.

and

Table 9. Results of benchmark constrained functions.

		PSPSO	PSOCO	CPSO	WICPSO	APSO	ScPSO	PSO
c1	mean	7.90 × 10−28	8.25 × 10−6	2.53 × 10−29	1.65 × 10−3	1.00 × 10−29	2.92 × 10−28	6.23 × 10−28
	std	4.28 × 10−27	1.30 × 10−5	7.39 × 10−29	1.33 × 10−3	4.00 × 10−29	8.95 × 10−28	1.89 × 10−27
	time	0.091	0.224	0.069	0.091	0.059	0.075	0.05
c2	mean	−1.07 × 102	−1.07 × 102	−1.07 × 102	−1.07 × 102	−1.10 × 102	−1.07 × 102	−1.07 × 102
	std	2.81 × 10−14	3.82 × 10−14	2.84 × 10−14	2.27 × 10−1	3.90 × 10−14	3.07 × 10−14	3.02 × 10−14
	time	0.127	0.277	0.084	0.11	0.072	0.089	0.064
c3	mean	−2.13 × 100	−2.08 × 100	−2.13 × 100	−2.13 × 100	−2.10 × 100	−2.13 × 100	−2.13 × 100
	std	1.17 × 10−15	1.42 × 10−1	8.96 × 10−16	3.83 × 10−4	6.00 × 10−16	9.72 × 10−16	1.06 × 10−15
	time	0.095	0.285	0.084	0.109	0.081	0.09	0.065
c4	mean	−1.50 × 101	−1.50 × 101	−9.77 × 100	−8.15 × 100	−1.10 × 101	−6.63 × 100	−6.43 × 100
	std	0.00 × 100	0.00 × 100	2.21 × 100	2.61 × 100	2.40 × 100	1.69 × 100	1.43 × 100
	time	1.207	3.389	1.557	1.298	0.841	1.36	1.027
c5	mean	−7.96 × 10−1	−7.83 × 10−1	−5.71 × 10−1	−1.54 × 10−1	−7.20 × 10−1	−6.00 × 10−1	−6.32 × 10−1
	std	1.10 × 10−2	8.73 × 10−3	1.31 × 10−1	9.32 × 10−3	9.80 × 10−2	1.50 × 10−1	1.28 × 10−1
	time	4.2	10.678	3.923	4.018	3.301	3.746	3.221

Note: time is in seconds.

for each PSO variant for every benchmark function. The results of Table 8 and Table 9 are summarized in

Table 10. Ranking of each PSO variant (D = 30).

	PSPSO	PSOCO	CPSO	WICPSO	APSO	ScPSO	PSO
Unconstrained function property
Continuous	1	2	3	7	4	6	4
Discontinuous	1	1	6	7	4	3	4
Differentiable	1	2	3	7	5	5	4
Non-differentiable	1	2	5	7	3	4	5
Separable	2	1	4	7	6	3	5
Non-separable	1	2	3	7	4	6	5
Scalable	1	2	3	7	4	6	5
Non-scalable	2	1	3	7	6	5	4
Unimodal	1	2	4	7	3	5	5
Multimodal	2	1	3	7	6	5	4
Constrained functions	1	4	2	7	3	4	6
Overall Ranking in Function Evaluation	1	2	3	7	3	5	5
Overall Ranking in Computational Time	3	7	5	6	1	4	2

by ranking the performance of each algorithm. It can be seen that in almost all functions that the PSPSO was ranked 1 except for separable, non-scalable, and multimodal functions where it ranked second. The PSOCO ranked second on almost all functions, performing better in discontinuous, separable, non-scalable, and multimodal functions but performed poorly in constrained function. Overall, the PSPSO ranked first, followed by the PSOCO, CPSO, and APSO tied on third, ScPSO and PSO on fifth, and lastly WICPSO.

Figure 4a shows the average ranking of the algorithms with increasing dimension, where PSPSO almost remained first in rank in function evaluation with a slightly improved performance. It can be noted that both the APSO and PSPSO, both with downward slope, had better performance when the dimension was increased. This can also mean the other way around, that the other algorithms’ performance worsened, especially for the CPSO with an increased average ranking (upward slope). Figure 4b shows the average computational time of every variant with increasing dimension. It also shows that the computational times of PSPSO, CPSO, WICPSO, ScPSO, and PSO were not that far from each other, even in the increase in dimension. On the other hand, PSOCO was found to be always more than twice the computational time of the other variants. The computational time PSO variants are magnified in Figure 4c, and the effect in the increase in dimension is seen linear for all but with different slopes. The proposed PSPSO had the lowest slope followed by APSO, PSO, ScPSO, WICPSO, CPSO, and PSOCO. Although the average computational time for PSPSO was the lowest, its overall ranking for computational time was just a third, as seen in Table 10. This means that not in every function evaluation was solved the fasted by the PSPSO, and that most of the time APSO or PSO solved a function faster than PSPSO.

3.4. Unit Commitment Performance

Table 11. Unit Commitment schedule and costs.

hour\unit	1	2	3	4	5	6	7	8	9	10	Fuel Cost	SU Cost
1	455	245	0	0	0	0	0	0	0	0	13,683.13	0
2	455	295	0	0	0	0	0	0	0	0	14,554.50	0
3	455	370	0	0	25	0	0	0	0	0	16,809.45	900
4	455	455	0	0	40	0	0	0	0	0	18,597.67	0
5	455	390	0	130	25	0	0	0	0	0	20,020.02	560
6	455	360	130	130	25	0	0	0	0	0	22,387.04	1100
7	455	410	130	130	25	0	0	0	0	0	23,261.98	0
8	455	455	130	130	30	0	0	0	0	0	24,150.34	0
9	455	455	130	130	85	20	25	0	0	0	27,250.86	860
10	455	455	130	130	162	33	25	10	0	0	30,057.22	60
11	455	455	130	130	162	73	25	10	10	0	31,915.33	60
12	455	455	130	130	162	80	25	43	10	10	33,889.36	60
13	455	455	130	130	162	33	25	10	0	0	30,057.22	0
14	455	455	130	130	85	20	25	0	0	0	27,250.86	0
15	455	455	130	130	30	0	0	0	0	0	24,150.34	0
16	455	310	130	130	25	0	0	0	0	0	21,513.66	0
17	455	260	130	130	25	0	0	0	0	0	20,641.82	0
18	455	360	130	130	25	0	0	0	0	0	22,387.04	0
19	455	455	130	130	30	0	0	0	0	0	24,150.34	0
20	455	455	130	130	162	33	25	10	0	0	30,057.22	490
21	455	455	130	130	85	20	25	0	0	0	27,250.86	0
22	455	455	0	0	145	20	25	0	0	0	22,735.32	0
23	455	425	0	0	0	20	0	0	0	0	17,645.16	0
24	455	345	0	0	0	0	0	0	0	0	15,427.42	0
											559,844.16	4090
								Total Cost ($)			563,934.16

shows the result of the UC from the PSPSO with a total UC cost of $563,934.16. The total fuel cost is $559,844.16 and the total start-up cost is $4,090. This result is the same commitment of units with the other published literatures but with discrepancies in the dispatch and/or costs.

Table 12. Benchmark unit commitment (UC) problem.

(a) Without Ramp Constraint
Approach	Best ($)	Average ($)	Worst ($)	Avg. Time (s)	Max Iter	n
WICPSO [18]	563,937.69	-	-	39.6	100	10
HGICA [25]	563,935.31	-	-	2.44	32	55
GA [26]	563,937.69	-	-	-	300	64
QBGWO [27]	563,936.30	563,963.30	563,936.30	62.3	500	30
BASA [28]	563,937	563,937	563,937	-	-	-
BGWO [29]	563,937.30	563,937.30	563,937.30	66.15	500	30
MILP-SMOUC [30]	56,3937	-	-	-	-	-
PSPSO	563,934.17	564,112.10	564,357.63	3.57	1000	50
(b)With Ramp Constraint
Approach	Best ($)	Average ($)	Worst ($)	Avg. Time (s)	Max Iter	n
HGICA [25]	564,186.63	-	-	1.61	32	55
MILP- SMOUC [30]	564,326.80	-	-	1.08	-	-
PSPSO	564,111.36	564,178.22	564,563.00	5.9	1000	50

shows the results of the PSPSO compared with algorithms in the literature, which also solved the 10-unit UC problem. Although the maximum iteration is 1000, the algorithm terminated around the 400th iteration with an average time of 3.57 s. The best result of the PSPSO reached the minimum solution found in the literature. The average cost of the algorithm was 104.33 more than the best cost, which was a 0.018% difference. Moreover, some algorithms were three times slower than the proposed PSPSO, and only one algorithm was faster than the proposed.

Table 13. System Cost ($) and Computational Time (sec) of the 10-unit UC problem with increased. Dimension compared with other PSO variants.

	PSO	CPSO	PSOCO	WICPSO	PSPSO
10-unit DUC
best	564,112	564,278	566,485	563,974	563,934
average	564,966	564,892	569,874	564,623	564,039
worst	566,891	566,847	572,136	566,127	564,214
time	24.05	21.93	35.86	19.94	7.24
30-unit DUC
best	1,124,723	1,125,119	1,134,358	1,124,639	1,124,663
average	1,126,983	1,126,516	1,140,327	1,126,166	1,125,221
worst	1,131,985	1,129,734	1,144,489	1,128,543	1,126,375
time	46.01	42.26	65.07	38.14	27.80
40-unit DUC
best	2,249,498	2,250,868	2,273,559	2,246,659	2,245,909
average	2,254,239	2,255,014	2,282,112	2,249,593	2,248,930
worst	2,257,990	2,260,150	2,290,050	2,252,665	2,250,830
time	96.10	87.31	131.00	86.45	85.65
60-unit DUC
best	3,379,482	3,376,946	3,407,486	3,367,881	3,368,371
average	3,384,379	3,385,574	3,420,476	3,373,715	3,371,589
worst	3,392,098	3,392,595	3,430,814	3,379,109	3,375,929
time	164.85	152.47	217.43	176.86	146.83
80-unit DUC
best	4,508,627	4,509,984	4,550,595	4,495,549	4,495,053
average	4,517,806	4,521,129	4,567,651	4,502,553	4,497,875
worst	4,533,652	4,536,629	4,585,982	4,509,585	4,500,931
time	235.21	214.21	295.07	240.66	235.68
100-unit DUC
best	5,633,268	5,643,919	5,691,819	5,622,310	5,619,793
average	5,652,797	5,657,421	5,719,049	5,630,848	5,626,222
worst	5,669,649	5,669,151	5,826,662	5,636,712	5,631,809
time	318.28	280.12	368.28	309.69	319.69

shows the results of the 10-unit system multiples with the termination criteria changed to maximum iteration for all algorithms. In almost all cases, the PSPSO performed the best. However, in the 20-unit and 60-unit systems, the WICPSO was able to outperform PSPSO in finding the best cost, yet PSPSO still was the best in terms of the average cost, worst cost, and computational time. The performance of WICPSO was already expected because it has already been used to solved this test system in the literature, but the fact that PSPSO almost always demonstrates the best performance in all the multiples, costs, and computational time makes PSPSO a viable and competitive algorithm in solving a unit commitment problem.

4. Discussion

The PSPSO showed very competitive results with the other known variants of PSO in both search and computational time. The PSPSO outperformed CPSO, WICPSO, ScPSO, and PSO on all function properties except non-scalable functions. Although PSOCO’s search for the global optimum was better than the other variants, PSOCO had the worst computational time, which at times was twice the time of a standard PSO due to its multiple crossover operation, which is computationally taxing. The PSPSO, on the other hand, computed faster compared to other variants. This may have been due to the algorithm’s freedom to direct the particles. Combining this concept with a crossover with another particle would make the particles move more intelligently on the solution space. This suggests that PSPSO can be an alternative to PSO and its variants.

5. Conclusions

In the literature, it has always been the disadvantage of PSO of it having no crossover operation, aside from its crossover to its global best solution, in terms of preventing it from premature convergence. Although some variants tried to include a crossover operation, the operational time is usually compromised, as in the case of PSOCO. The proposed PSPSO’s γ and λ values summarize the characteristics of the current set of particles so that all particles will gain some information about other particles as well and continually search for a better solution. Combining these constants with the partitioned velocity rule may provide an alternative crossover operation that does not compromise the computational time.

The results have proven the effectiveness of PSPSO compared to variants without the crossover operation in different types of functions. The computational speed is likewise competitive with published variants. Even with the increased dimension, the effectiveness of PSPSO is still maintained with a slight improvement in computational time. The PSPSO was also proven effective in solving the IEEE 10-unit problem and is competitive with other algorithms found in the literature.