A Compact Cat Swarm Optimization Algorithm Based on Small Sample Probability Model

Abstract

In this paper, a compact cat swarm optimization algorithm based on a Small Sample Probability Model (SSPCCSO) is proposed. In the same way as with previous algorithms, there is a tracking mode and a searching mode in the processing of searching for optimal solutions, but besides these, a novel differential operator is introduced in the searching mode, and it is proved that this could greatly enhance the search ability for the potential global best solution. Another highlight of this algorithm is that the gradient descent method is adopted to increase the convergence velocity and reduce the computation cost. More importantly, a small sample probability model is designed to represent the population of samples instead of the normal probability distribution. This representation method could run with low computing power of the equipment, and the whole algorithm only uses a cat with no historical position and velocity. Therefore, it is suitable for solving optimization problems with limited hardware. In the experiment, SSPCCSO is superior to other compact evolutionary algorithms in most benchmark functions and can also perform well compared to some population-based evolutionary algorithms. It provides a new means of solving small sample optimization problems.

1. Introduction

Lately, compact evolutionary algorithms (cEA) have rapidly developed. In 1999, Georges R. Harik et.al proposed a novel compact genetic algorithm (cGA) [1]. It mimics the behavior of a simple GA with standard binary coding for order-one problem. It could almost obtain the same performance as a standard GA under this simple representation. Besides these, it engendered an idea that it is possible to use a probability distribution to represent populations, in order to reduce memory usage. Inspired by cGA, Ernesto Mininno et al. proposed a real-valued Compact Genetic Algorithms (rcGA) [2]. It firstly employed a normal probabilistic distribution model to describe the statistic features of all of the samplings. Individuals could be generated directly from this normal probabilistic distribution model. The most successful highlight of rcGA lies in that it employed effective updating rules to update the parameters of the normal probabilistic distribution functions (PDF). A compact Differential Evolutionary algorithm (cDE) [3] was presented by Ernesto Mininno et al. in 2011. Though it was based on the same normal probabilistic model, it inherited the essential features of differential evolutionary (DE). The efficient performance together with modest requirements made it suitable for the environment with small computational power. After cDE, cPSO [4] was proposed in 2013. Unlike the other PSO versions, it stored neither the positions nor the velocities, and only a particle was employed in the whole algorithm; what is more, it also employed a normal probabilistic model to simulate the swarm’s behavior. This modest representation enables cPSO to run in those devices with limited computational power or limited memory. In 2018, Ming Zhao [5] proposed a novel compact cat swarm optimization algorithm based on a differential method, with better performance than some similar algorithms.

These compact evolutionary algorithms employed a probability distribution to explicitly represent the population of the solutions. Normally, a normal distribution function is introduced. Instead of a large number of populations and variables, only the expectation and variance of the representative probability model are adopted, a type of probability model and a particle are adopted and a few variables and limited run spaces are required; that is, a compact idea is used to design the algorithm. It is known to all that a good distribution is equivalent to linkage learning [6,7]. A normal distribution model suits the simulation of those samplings with a large size [8,9]. Obviously, not all of the samplings could be described by normal distribution. There are still some samplings with non-normal distribution. If we employ a normal distribution to simulate them compulsively, the performance will be barely satisfactory. Therefore, the problems with a small size could probably be described by a special non-normal distribution.

Inspired by the literature [1,2,3,4,5], we expected to find another non-normal probabilistic distribution model to represent the samplings, which would have a different mean and the variance under the different parameters. Updating the rules for the mean and the variance could help the probabilistic model to generate highly effective solutions. Giving overall consideration to the features of some non-normal probability models, a gamma probabilistic distribution function is employed in this study.

Meanwhile, the corresponding evolutionary algorithm will be considered. It will cooperate with the gamma probabilistic model to try to find the best solution for the optimization problems. Chu et al. [10] proposed a cat swarm optimization (CSO) algorithm in 2007. A novel combination searching strategy was employed in CSO, in comparison with the corresponding evolutionary algorithms, a higher performance was shown in the standard test functions. Then, Tsai and his group [10] developed it further and proposed some improved versions, such as parallel cat swarm optimization (PCSO) [11] and reinforced parallel cat swarm optimization (EPCSO) [12]. It was also widely used in some application domains with a pretty good performance [13,14,15]. It is population-based. There is still no population-less version for CSO. So, in this paper, we also select CSO to combine the gamma probabilistic distribution, which is a Small Sample Probability Model. In order to reduce computing costs and the velocity up convergence rate, a gradient descent method is introduced to the seeking mode of CSO. Thus, a novel compact cat swarm optimization scheme with a Small Sample Probability Model (SSPCCSO) is proposed. We will employ this novel algorithm to test whether it could solve some problems of a small size.

The remainder of this research is organized as follows: Section 2 presents the sampling mechanism and cat swarm optimization. In Section 3, the proposed compact cat swarm optimization with gamma distribution and gradient descent method is discussed in detail. Section 4 displays the experimental results, and Section 5 summarizes this study.

2. Related Work

In this section, the sampling mechanism based on real-valued coding is presented in detail, and the cat swarm optimization algorithm is also introduced.

2.1. The Sampling Mechanism Based on Real-Valued

As mentioned above, the main feature of compact algorithms is population-less, see [1,2,3,4]; a virtual population based on probabilistic model is introduced to represent the populations. In real-valued coding compact algorithms [2,3,4], solutions are generated through this probabilistic model. Mean and variance are parameters for the probabilistic density function (PDF). A little modification for the two parameters will affect the solutions in the next generation. So, this probabilistic model is named as Perturbation Vector (PV). It is encoded as Formula (1):

$$P{V}^t=[u^t,{\delta }^t]$$

(1)

where $\mu$ and $\sigma$ are, respectively, mean and standard deviation values of the corresponding PDF; top t is generations.

Without losing generality, all of designed variable x should be normalized to interval [−1,1], for a PDF, its corresponding Cumulative Distribution Function (CDF) may not be equal to 1 because some of the variables will be out of [−1,1]. An error function [15] is introduced to solve this problem; thus, the truncated PDF is presented as Formula (2):

$$PDF(truncated(x_i))=\frac{\sqrt{\frac{2}{\pi }}{e}^{-\frac{{(x_i-u_i)}^2}{2{({\sigma }_i)}^2}}}{{\sigma }_i(erf(\frac{u_i+1}{\sqrt{2}{\sigma }_i})-erf(\frac{u_i-1}{\sqrt{2}{\sigma }_i}))}$$

(2)

where $x_i$ is the i-th dimension of designed variable $x$; $u_i \mathrm{and} {\delta }_i$ are mean and variance associated with $x_i$. The corresponding CDF value can be obtained through Formula (3). When a Cumulative Distribution Function value is generated, the corresponding x could be calculated by computing the inverse function of Formula (3). The sampling mechanism of PV can be seen in Figure 1.

$$CDF(x_i)={\displaystyle \underset{-1}{\overset{1}{\int }}PDF(truncated x_i)}{d}_{x_i}$$

(3)

The sampling procedure could be described as follows. Firstly, a random number in [0,1] is generated under a normal distribution, and it will be taken as a Cumulative Distribution Function value; then, computing the inverse function of the Cumulative Distribution Function, the calculated value is $x_i$. $x_i$ is a needed solution, however, the solution is not generated directly, it was obtained based on e evolutionary computation and evolutionary computation. The sampling mechanism could be interpreted as Figure 1.

In the real process of sampling, in order to reduce the calculated cost, an approximate computing for the designed x [i] is implemented, by means of the Chebyshev polynomials [16].

Another highlight for the sampling mechanism of the compact evolutionary algorithms with real-value is updating the rules for mean and variance. When two individuals are compared, the winner indicated the one with better fitness, and the other is the loser. A more effective solution would be expected to be generated from PV through updating mean and variance. The updating rules are shown as Formulas (4) and (5):

$${\mu }^{t+1}(i)={\mu }^t(i)+\frac{1}{N_p}\left[winner(i)-loser(i)\right]$$

(4)

$${\left[{\sigma }^{t+1}(i)\right]}^2={\left[{\sigma }^t(i)\right]}^2+{\left[{\mu }^t(i)\right]}^2-{\left[{\mu }^{t+1}(i)\right]}^2+\frac{1}{N_p}\left\{{\left[winner(i)\right]}^2-{\left[loser(i)\right]}^2\right\}$$

(5)

N_p is the size of the virtual population, and top t is the generations, i is a dimension of the designed variable x. The details can be seen in the literature [2,3,4].

2.2. Cat Swarm Optimization (CSO)

Inspired by the behavior of cats, Chu and Tsai [10] proposed the cat swarm optimization algorithm in 2007. A combination of two search logics is employed in this algorithm, i.e., the seeking mode and the tracing mode. All of the cats will be divided into two groups before the iterations. Just like PSO, its update rules are very similar to the traditional Particle Swarm Optimization (PSO) algorithm; the designed cat represents a solution for the project to be solved, and each designed cat has its own position and velocity. The solutions are updated by the cat’s position and velocity, and estimated based on its degree of adaptability for the project; the global best will be chosen and conduct all of the cats to seek its next position and velocity. From the perspective of biological group behavior, this is obviously different from the particle swarm optimization algorithm. The details of the CSO algorithm will be introduced in the following section.

2.2.1. Seeking Mode

The number of cat populations in the seeking mode is decided by a parameter GR (group rate), which, normally, is set to be 0.98 [8]. When the cats are in seeking mode, the GR is used as a minor tune-up for the cats’ position, and does change their velocities. The following steps will be implemented.

Firstly, every cat will copy its own position many times, according to the size of the parameter SMP, and the position will be stored in the corresponding seeking mode pool (SMP) unit. Then, each position in the SMP will be recalculated by a mutagenic operator, a dimension of the expected variable $x_i$ could be chosen to mutate, and the range of variation would be decided by a random number, which is up to 20% of $x_i$. The mutation operation is described as Formula (6):

$$x_i=x_i+\Delta x_i$$

(6)

The position with the best fitness in the SMP will be chosen to update $x_i$.

2.2.2. Tracing Mode

The evolutionary process for the cats in tracing mode is similar to the particle in the PSO algorithms. However, they are still somewhat different. Each cat in tracing mode will only trace the cat with the global best fitness to update its own velocity and position. The particles in PSO [16] trace both the global best individual and the local best individual. The updating rule for the tracing mode can be expressed as the Formulas (7) and (8):

$${\mathrm{v}}_k\left(\mathrm{t} +1\right)=\omega \cdot {\mathrm{v}}_k\left(\mathrm{t}\right)+Const\cdot random\cdot [x_{gb}\left(\mathrm{t}\right)-x_k\left(\mathrm{t}\right)]$$

(7)

$$x_k(t+1)=x_k(t)+v_k(t+1)$$

(8)

where $x_{gb}$ is the position of the cat with the best fitness; $x_k$ is the position of $ca{t}_k$; t is the generation for iterations. $Const$ is a constant and $random$ is a random number in [0,1].

The cat in seeking mode will be compared with the cat with the best fitness in tracing mode; the winner would be chosen to update the variable. The final $x_{gb}$ is the required solution.

3. The Proposed Compact Cat Swarm Optimization Scheme Based on Small Sample Probability Model (SSPCCSO)

In this section, a novel compact swarm optimization scheme based on the Small Sample Probability Model will be proposed. First, a sampling mechanism with a new gamma distribution model will be introduced in Section 3.1, then a new differential operation will be implemented in seeking mode. Another highlight, a gradient descent method, will also be presented in Section 3.2. Section 3 states the tracing mode, and Section 4 introduces the SSPCCSO.

3.1. Virtual Population and Sampling Mechanism with Real-Valued

The main feature of the compact evolutionary optimization algorithms is population-less. A probabilistic model is employed to represent the distribution of the solution sets, instead of processing an actual population. A gamma distribution model is employed to act as the Perturbation Vector (PV). The PV is also a $n\times 2$ matrix and it is expressed as Formula (1). As is mentioned above, the PV is introduced to generate a new individual. The sampling mechanism is the same as the PV with a normal distribution (see Figure 1).

Normally, a gamma probability density function (PDF) and its CDF [17] are presented as Formula (9) or (10):

$$f\left(x;k,\theta \right)=\frac{1}{\Gamma \left(k\right){\theta }^k}{x}^{k-1}{e}^{-x/\theta }$$

(9)

$$F\left(x;k,\theta \right)=\underset{0}{\overset{x}{\int }}f\left(u;k,\theta \right)du=\frac{1}{\Gamma \left(k\right)}\gamma (k,\frac{x}{\theta })$$

(10)

where $k$ and $\theta$ are two parameters for Gamma $PDF$; the $PDF$ curve is shown as Figure 2 [13].

The lower incomplete Gamma function is defined by $\gamma \left(s,x\right)=\underset{0}{\overset{x}{\int }}{t}^{s-1}{e}^{-t}dt$ (note that the upper incomplete Gamma function is $\Gamma \left(s,x\right)=\underset{x}{\overset{\infty }{\int }}{t}^{s-1}{e}^{-t}dt$, and the ordinary Gamma function is defined as $\Gamma \left(s\right)=\Gamma \left(s,0\right)=\gamma (s,\infty )$). Thus, the mean and variance for gamma $PDF$ is calculated by $E\left[X\right]=k\theta$ and $Var\left[X\right]=k{\theta }^2$.

Most of the variables are in the interval [0,20]. We define [0,20] as an all solution domain. However, there may still be some potential solutions out of [0,20], so an error function must be employed to map those potential solutions out of [0,20] into [0,20].

Without losing generality, all of the variables should be normalized to [−1,1]. As discussed above, we define the variables x in [0,20], then new variable will be mapped as $y=10\left(x+1\right)$. Thus, we consider the truncated Gamma distribution to [0,20], and then:

$$\begin{array}{ll}f\left(10(x+1);k,\theta \right)& =\frac{{10}^{k-1}}{\Gamma \left(k\right){\theta }^k}{(x+1)}^{k-1}{e}^{-10(x+1)/\theta }\\ & =\frac{1}{10\Gamma \left(k\right){(\frac{\theta }{10})}^k}{(x+1)}^{k-1}{e}^{-(x+1)/(\frac{\theta }{10})}\\ & =\frac{1}{10\Gamma \left(k\right)t^k}{(x+1)}^{k-1}{e}^{-(x+1)/t}\\ & =\frac{1}{10}f\left(x+1;k,t\right)\end{array}$$

(11)

$$F\left(10\left(x+1\right);k,\theta \right)=\frac{1}{\Gamma \left(k\right)}\gamma \left(k,\frac{10\left(x+1\right)}{\theta }\right)=\frac{1}{\Gamma \left(k\right)}\gamma \left(k,\frac{x+1}{t}\right)$$

(12)

Thus, the new distribution truncated on [−1,1] is represented by Formula (13):

$$\begin{array}{ll}PDF\left(truncated\right)& =\frac{1}{10}\frac{f\left(x+1;k,\theta \right)}{\frac{1}{\Gamma \left(k\right)}\gamma \left(k,\frac{2}{\theta }\right)}\\ & =\frac{1}{10\gamma \left(k,\frac{2}{\theta }\right){\theta }^k}{(x+1)}^{k-1}{e}^{-(x+1)/t}\end{array}$$

(13)

Then, it could ensure that any of the solutions out of [−1,1] could map to [−1,1], see Figure 3. Its CDF is presented by Formula (14):

$$\begin{array}{ll}CDF\left(truncated\right)& =\frac{\frac{1}{\Gamma \left(k\right)}\gamma \left(k,\frac{x+1}{\theta }\right)}{\frac{1}{\Gamma \left(k\right)}\gamma \left(k,\frac{2}{\theta }\right)}\\ & =\frac{\gamma \left(k,\frac{x+1}{\theta }\right)}{\gamma \left(k,\frac{2}{\theta }\right)}\end{array}$$

(14)

According to Formulas (13) and (14), the mean and variance could be represented by $\mu =\frac{k\theta }{10}$ and $\sigma =\frac{k{\theta }^2}{100}$.

The sampling mechanism of a designed variable could be described as follows:

First, the scheme will generate a random number $p$ between 0 and 1, according to the uniform distribution model, and the parameters ${\mu }_{\mathrm{i}}$ and ${\sigma }_{\mathrm{i}}$ for the Perturbation Vector will be Initialized (${\mu }_{\mathrm{i}}=0$ and ${\sigma }_{\mathrm{i}}=\lambda$). This $p$ is the Cumulative Distribution Function (CDF) value for the expected variable $x$, it is a Cumulative Distribution Function value for the corresponding PDF, then the inverse function of CDF in rand (0, 1) is introduced, according to Formula (14). Finally, a new $x[i]$ will be obtained.

Apparently, according to the definition of normal distribution, its solutions’ domain should be [$-\infty ,+\infty$], but real projects are limited to a specific domain, The optimal solution obtained by updating the rules may be not located in the definition domain, an error will be generated for mapping an infinite space to a finite space; the mapping function is introduced to solve the error for mapping. Because of the particularity of the Cumulative Distribution Function for Gamma distribution, all of the solutions almost locate in [0,20]. The mapping problem turns out to be a finite domain into another finite domain, the error disappears automatically, thus, an error function is not required.

The Perturbation Vector Updating Rule

The PV of the virtual population is designed to create new solutions; the parameters of PV could be updated to create more significant individuals. The updating rules for PV are also the same as in the literature [2,3,4]. They are expressed as Formulas (4) and (5). There are two very important vectors, $winner$ and $loser$, in Formulas (4) and (5), in which winner indicates the individual with best fitness when two solutions are compared. From Formulas (4) and (5), $\mu$ and $\sigma$ are updated by an vector $\frac{1}{N_p}(winner-loser)$, this vector could adjust the values of $\mu$ and $\sigma$. Apparently, the new vector $\frac{1}{N_p}(winner-loser)$ conducts the forward orientation of $\mu$; thus, the $\mu$ would approach the $winner$, in order to obtain the next solution more effectively; $\sigma$ is designed to conduct the step size, when the current solution is far from the best solution, a large size $\sigma$ could be used. When the current solution is close to the best solution, a small $\sigma$ would be used. Thus, a new solution in the next iteration would be generated more effectively by this updating rule. This can be seen in Figure 4.

3.2. Seeking Mode

Compared with CSO, the SSPCCSO has two highlights: one is the differential operator, and the other is the gradient descent method. The details about these will be presented in this section.

3.2.1. Differential Operator

The cat in seeking mode will update its position by another new way. For the seeking mode of CSO, the position of a cat will be updated through Formula (6), while in SSPCCSO, a differential operator is introduced to enhance the search ability of the cat for the local best solution, and we call this differential operator the Exploration Vector (EV), it is presented as Formula (15):

$$x=x+F\ast (winner-loser)$$

(15)

where $F\in [1/N_p,0.2]$ is a scale factor which controls the length of the exploration vector $(winner-loser)$.

It had to re-mention the vector $(winner-loser)$, the updating rule for seeking in the CSO is described as Formula (6); according to Formula (6), x will be changed from the view of x itself, that is to say, the updating vector ($x+\Delta x$) is only a simple amplification or reduction in the scalar on its direction. It can find the better solution based on two directions, it can be seen on the left of Figure 5, according to the proposed Formula (15), where it could search for the best solution in all directions. It can also be seen on the right of Figure 5.

(Winner-loser) is an exploration vector, it could face in all of the potential directions, and it has challenges in seeking for a better solution, see on the right of Figure 5.

With reference to the literature [10,11,12], the proportion for cats in the seeking mode and cats in the tracing mode is 98:2, and the size of memory pool is five times of the current cat. However, in the proposed SSPCCSO, a cat is only in seeking mode or in tracing mode. In order to mimic the search logic of the CSO algorithm, the updating rule would be implemented 245 times in each iteration; thus, the computing cost may be too high to be accepted. A gradient descent method [18,19] is introduced to reduce the computing cost and obtain the real local solution for the designed x. The details will be discussed in the following section.

3.2.2. Gradient Descent Method

As mentioned in the previous section, the updating rule in seeking mode will be implemented many times, and the process will be run with a higher computing cost. It may not be accepted for some of the engineering problems. With full consideration of these factors, a gradient descent method (GD) [19,20] is introduced. Firstly, a convergence rate (CR) is shown as Formula (16):

$$CR=\frac{\left|fitness(t+2)-fitness(t+3)\right|}{\left|fitness(t)-fitness(t+1)\right|}$$

(16)

where $fitness(t)$ is the fitness for the cat in $t$ generation. When $CR<1$, it means that the acquired solution begins to converge [20].

When the selection for the local best solution is being implemented, and the termination condition is not met, and meanwhile the local best cat is not updated for many generations, the rest loop would be unnecessary. Even though the local best solution is frequently to obtain a differential operator, sometimes this local best solution is not actually the local best solution. Based on these factors, much unnecessary computing cost could be reduced, and a more effective solution could be found with less time. Thus, a gradient is introduced and is defined as Formula (17).

$$\nabla f=[\frac{\partial f}{\partial x_1},\frac{\partial f}{\partial x_2},\dots ,\frac{\partial f}{\partial x_n}]$$

(17)

where $n$ is the dimension of designed variable $x$. It is known to all that the real local best solution of variable x will be quickly obtained, according to the gradient vector of the function, and it can be calculated by Formula (18):

$$x(t+1)=x(t)+\nabla f\cdot d$$

(18)

where $d$ is the step length, and it could firstly be set as the vector (winner-loser), then it can be optimized by processing $max(f(x_i+\nabla f\cdot d)$. The flow chart of the steepest gradient descent method is shown as Figure 6.

When the evolutionary procedure of the designed x goes into the gradient descent phase, the computing cost will be highly reduced, and the real local best solution will be quickly obtained.

3.3. Tracing Mode

The cat’s behavior in tracing mode is simulated to the particle in the PSO algorithm, but each cat only traces the cat with the global best position to update its own velocity and position. The updating rule of tracing mode could be presented as Formulas (7) and (8). All of the parameters are the same to the original CSO [9].

The combination of seeking mode and tracing mode could ensure that the cat swarm optimization algorithm converges quickly and prevents the solution from the local optimum.

A cat with the best fitness will be chosen to compare which is the final best solution. When the iterations meet the termination condition, the final $xgb$ is the solution for the problem.

3.4. The Procedure and Pseudo Code for SSPCCSO

For the proposed SSPCCSO, normally, all of the variables would be mapped into the intervals [−1,1], $\mu$ and $\sigma$ are initialized as ${\mu }_{\mathrm{i}}=0$ and ${\sigma }_{\mathrm{i}}=\lambda$, a random number in the range [−1,1] is chosen as the global best $xgb$. Then, the cat would be randomly grouped into a mode.

In the iteration phase, when the cat is in tracing mode, a local best solution $x_{lb}$ is generated from PV, the cat’s position and velocity is updated by Formulas (7) and (8), and the comparison between $x_{lb}$ and $cat.x^{t+1}$ is used to determine which is winner and which is loser. Then, winner and loser are applied to updating the PV. Another comparison between the $cat.x^{t+1}$ and $xgb$ is used to decide who is $xgb$. When the cat is in seeking mode, firstly, a new candidate solution $x_{lb}$ from PV would compare with $cat.x$; it is also employed to determine the winner and loser, as winner and loser in this case are applied to update $cat.x^{t+1}$ by Formula (12) and PV. These steps would be reduplicated for many times in a loop. When this loop is stagnation, a gradient descent method is involved to reduce the computing cost and find the real local best. No matter which mode the cat is in, a $xgb$ would be chosen in each run. For the sake of clarity, the pseudo code for SSPCCSO is shown in Figure 7.

4. Experimental Results and Analysis

With reference to the literature [4,20], the SSPCCSO would be tested on 47 benchmark functions, which include those where their coordinates had been transformed and their shifted [21]. The composition test functions [22] are also introduced; all of the benchmark functions are listed in the Appendix A.

SSPCCSO is a compact optimization algorithm based on small size samplings. Firstly, we select rcGA, cDE and cPSO as the compared algorithms. As SSPCCSO is also a member of evolutionary algorithms, traditional DE [23], PSO [24] and CSO [9] should be considered. From the perspective of saving memory, ISPO [25] is an indispensable object; it should be one of the objects of comparison with SSPCCSO.

All of the experiments were carried out on a personal computer with MATLAB language, which is equipped with Pentium (R) dual core E6600 CPU, 3.06 GHz and 2.96 gb RAM. The operation system is set at windows XP platform. Each comparison algorithm will select a set of parameters through which the best results can be obtained. With reference to the literature [4,26], all of the parameters for each compared algorithm are listed in

Table 1. Selected parameters list for all compared algorithms in this projection.

Algorithm	Parameters	Literature	Algorithm	Parameters	Literature
rcGA	$N_p=300$	[1]	DE	$N_p=60,F=0.5,Cr=0.9$	[23]
cDE	$\begin{array}{l}{N}_p=300,F=0.5\\ Cr=0.3\end{array}$	[2]	PSO	$\begin{array}{l}{\varphi }_1=-0.2,{\varphi }_2=-0.07,{\varphi }_3=3.74\\ {\gamma }_1={\gamma }_2=1,N_p=60\end{array}$	[24]
cPSO	$\begin{array}{l}{\varphi }_1=-0.2,{\varphi }_2=-0.07,{\varphi }_3=3.74\\ {\gamma }_1={\gamma }_2=1,N_p=300\end{array}$	[3]	ISPO	$\begin{array}{l}A=1,P=10,B=2,S_f=4\\ H=30,\epsilon \epsilon =1.0\times {10}^{-5}\end{array}$	[25]
CSO	$\begin{array}{l}{N}_p=60,c1=c2=2\\ W=0.9\end{array}$	[10]	SSPCCSO	$\begin{array}{l}w=-0.4,c1=2,c2=-0.07\\ N_p=300\end{array}$

. For all of the real population-based algorithms, the population size is 60, for all of the virtual population-based algorithms, the population size is 300. To achieve a truly fair comparison for all of the compared algorithms, all of the algorithms involved in the comparison were evaluated by taking the average value after running for over 30 times. In all of the tables, each value represents the corresponding mean value and the standard deviation value calculated for each comparison algorithm within 30 times, and “+”, “−” and ”=“ have the same implication as in the literature [2].

The remains of this section are described as below: first, a comparison for memory usage is listed in

Table 2. Running memory space for all compared algorithms.

Algorithm	Components	Memory Slots
ISPO	Single individual, 1 global best	2
rcGA	One individual, persistent elitism, 1 sampling	4
cDE	3 sampling, 1 global best	4
cPSO	1 sampling, 5 persistent variables	5
SSPCCSO	The same to cPSO	5
PSO	Population-based, history and current individuals	2NP
DE	Population-based, current individuals only	NP
CSO	Population-based, history and current individuals	2NP

. Then, the comparisons for the memory-saving algorithms are presented. Next to this, the comparisons between the population-based algorithms and SSPCCSO will be shown; the analyses of the results for SSPCCSO are summarized in the final section.

4.1. Comparison for Memory Usage

The proposed SSPCCSO scheme adopts a gamma probability distribution model to represent the population, and only a cat is used. The cat has the same data structure as the particle of cPSO, so it will also have the same memory space as the cPSO, in which only five persistent variables are required for storing the whole algorithm. The memory usage status for all of the compared algorithms can be seen in Table 2.

From the data in Table 2, it can be seen that SSPCCSO, cPSO, cDE, rcGA and ISPO have modest memory requirements. They belong to compact optimization algorithms. In relation to memory usage, the SSPCCSO is better than the CSO which is population-based.

4.2. Comparisons for Compact Optimization Algorithms

The experimental data in

Table 3. Comparison for memory-saving algorithms.

Function	rCGA	cDE	ISPO	cPSO	W	SSPCCSO
fu1	1.427 × 104 ± 9.27 × 103	8.73 × 10−28 ± 1.86 × 10−28	8.437 × 10−31 ± 3.31 × 10−31	6.471 × 101 ± 2.28 × 101	+	6.170 × 10−3 ± 1.05 × 10−3
fu2	2.851 × 104 ± 6.58 × 103	3.778 × 103 ± 1.85 × 103	1.184 × 101 ± 5.92 × 100	2.560 × 103 ± 2.37 × 103	+	3.625 × 102 ± 7.02 × 102
fu3	1.282 × 109 ± 1.58 × 109	1.291 × 102± 1.84 × 102	2.026 × 102 ± 3.28 × 102	1.320 × 105 ± 7.46 × 104	+	5.776 × 10−1 ± 7.01 × 100
fu4	1.874 × 101 ± 3.59 × 10−1	8.694 × 10−2 ± 2.97 × 10−1	1.942 × 101 ± 1.57 × 10−1	3.728 × 100 ± 3.71 × 10−1	+	5.574 × 10−1 ± 3.07 × 10−2
fu5	6.434 × 10−3 ± 1.31 × 10−2	4.289 × 10−3 ± 1.38 × 10−2	1.124 × 101 ± 1.77 × 101	9.63 × 10−8 ± 3.07 × 10−8	−	1.613 × 10−2 ± 2.37 × 10−3
fu6	1.963 × 102 ± 2.85 × 101	7.944 × 101 ± 1.48 × 101	2.548 × 102± 4.23 × 101	2.94 × 101 ± 7.94 × 100	+	2.399 × 10−2 ± 4.09 × 10−3
fu7	2.312 × 103 ± 2.47 × 103	4.983 × 103 ± 3.78 × 103	2.254 × 103 ± 8.62 × 102	4.614 × 102 ± 2.40 × 102	+	2.991 × 102 ± 1.75 × 10−1
fu8	3.194 × 103 ± 8.01 × 102	1.673 × 103 ± 4.48 × 102	5.768 × 103 ± 5.38 × 102	3.160 × 103 ± 9.75 × 102	+	1.248 × 101 ± 1.90 × 100
fu9	1.008 × 104 ± 2.35 × 103	8.548 × 103 ± 2.14 × 103	2.755 × 104 ± 6.08 × 103	1.344 × 104 ± 1.74 × 103	−	1.111 × 105 ± 5.29 × 104
fu10	3.697 × 105 ± 1.78 × 105	4.265 × 104 ± 2.35 × 104	4.326 × 103± 4.54 × 103	1.040 × 106 ± 1.16 × 105	+	9.390 × 105 ± 1.13 × 104
fu11	1.851 × 101 ± 4.37 × 10−1	1.708 × 100 ± 1.11 × 100	1.948 × 101 ± 1.89 × 10−1	3.699 × 100 ± 3.53 × 10−1	+	8.328 × 10−2 ± 8.23 × 10−2
fu12	5.769 × 10−2 ± 1.05 × 10−1	2.395 × 10−1 ± 2.03 × 10−1	0.001 × 10−1± 0.01× 100	9.567 × 10−8 ± 2.69 × 10−8	+	1.018 × 10−8± 2.61 × 10−9
fu13	2.154 × 102 ± 3.96 × 101	1.314 × 102 ± 1.87 × 101	2.566 × 102 ± 4.15 × 101	3.924 × 101 ± 2.31 × 101	−	2.70 × 102 ± 1.81 × 10−5
fu14	3.246 × 101 ± 4.53 × 100	2.988 × 101 ± 3.47 × 100	4.777 × 101 ± 4.34 × 100	3.943 × 101 ± 1.15 × 100	−	7.142 × 102 ± 2.37 × 10−1
fu15	5.251 × 100 ± 5.19 × 100	2.315 × 10−16 ± 5.65 × 10−16	1.184 × 10−6 ± 2.89 × 10−17	1.778 × 100 ± 4.27 × 10−1	+	9.427 × 10−3 ± 6.15 × 10−3
fu16	−1.001 × 102 ± 4.43 × 10−9	−1.001 × 102 ± 1.63 × 10−9	−1.001 × 102 ± 8.38 × 10−15	−1.001 × 102 ± 8.45 × 10−5	=	−1.001 × 102 ± 0.00 × 100
fu17	1.452 × 100 ± 1.88 × 100	2.817 × 10−23 ± 3.16 × 10−23	9.994 × 10−1 ± 1.56 × 100	1.702 × 100 ± 7.08 × 10−1	+	9.518 × 10−5 ± 1.57 × 10−6
fu18	−5.485 × 10−1 ± 1.11 × 100	−1.150 × 100 ± 4.98 × 10−16	−2.258 × 10−1 ± 1.28 × 100	−1.030 × 100± 7.56 × 10−1	−	−4.104 × 10−1± 8.97 × 10−4
fu19	4.338 × 102 ± 4.75 × 101	2.603 × 102 ± 3.04 × 101	4.044 × 102 ± 4.15 × 101	4.403 × 101 ± 3.44 × 101	−	4.500 × 102 ± 2.60 × 10−3
fu20	−1.517 × 101 ± 2.76 × 100	−3.347 × 101 ± 1.87 × 100	−3.348 × 101 ± 1.64 × 100	−2.063 × 101 ± 2.33 × 100	−	−1.988 × 101 ± 2.33 × 10−1
fu21	8.372 × 103 ± 1.62 × 103	5.343 × 103 ± 8.47 × 102	9.679 × 103 ± 1.09 × 103	4.784 × 103 ± 1.09 × 103	−	1.42 × 100 ± 2.26 × 100
fu22	2.014 × 101 ± 1.48 × 10−1	1.787 × 101 ± 2.89 × 10−1	1.951 × 101 ± 7.51 × 10−2	3.899 × 10−1 ± 5.19 × 10−1	+	7.139 × 10−2 ± 4.17 × 10−2
fu23	1.645 × 102 ± 2.36 × 101	4.042 × 101 ± 1.41 × 101	1.247 × 10−13± 1.01 × 10−14	4.657 × 10−2± 2.39 × 10−2	=	7.319 × 10−2 ± 5.17 × 10−3
fu24	8.488 × 104 ± 8.14 × 103	2.941 × 103 ± 1.59 × 103	1.252 × 10−30± 3.09 × 10−31	6.918 × 10−2 ± 2.54 × 10−2	−	8.961 × 10−3 ± 1.46 × 10−2
fu25	−6.349 × 10−3± 3.24 × 10−4	−9.161 × 10−3 ± 6.27 × 10−4	−4.551 × 10−3 ± 3.79 × 10−4	−7.85 × 10−1 ± 1.60 × 10−14	−	0.000 × 100 ± 0.00 × 100
fu26	−2.178 × 101 ± 3.09 × 100	−4.938 × 101 ± 3.54 × 100	−6.556 × 101 ± 3.18 × 100	−2.920 × 101 ± 2.53 × 100	=	−3.970 × 101 ± 1.52 × 10−1
fu27	2.524 × 105 ± 2.58 × 104	1.051 × 104 ± 6.31 × 103	3.498 × 10−30± 8.75 × 10−31	2.217 × 10−2 ± 4.04 × 10−3	−	1.033 × 10−2 ± 1.39 × 10−2
fu28	1.166 × 103 ± 7.35 × 101	4.218 × 102 ± 3.72 × 101	7.942 × 102 ± 7.69 × 101	8.776 × 10−3 ± 2.88 × 10−3	=	6.631 × 10−3 ± 1.24 × 10−4
fu29	6.906 × 1010 ± 1.38 × 1010	5.643 × 108 ± 4.98 × 108	3.503 × 102 ± 3.91 × 102	1.220 × 102 ± 2.81 × 101	+	1.286 × 100 ± 2.50 × 100
fu30	1.297 × 1011 ± 2.63 × 1010	7.066 × 1010 ± 1.19 × 1010	9.702 × 109 ± 3.26 × 109	4.928 × 106 ± 6.56 × 105	−	1.109 × 108 ± 3.44 × 108
fu31	2.148 × 104 ± 2.51 × 103	1.842 × 104 ± 1.29 × 103	1.971 × 104 ± 1.28 × 103	1.045 × 104 ± 2.94 × 103	−	4.989 × 104 ± 8.38 × 104
fu32	1.591 × 103 ± 1.27 × 103	1.062 × 10−5 ± 9.78 × 10−6	2.684 × 10−30± 4.75 × 10−31	1.531 × 10−2 ± 3.80 × 10−3	=	1.774 × 10−2 ± 2.89 × 10−2
fu33	1.258 × 102 ± 6.44 × 100	8.948 × 101 ± 6.18 × 100	1.773 × 102 ± 5.91 × 100	7.370 × 101 ± 3.32 × 100	+	−9.9 × 101 ± 0.00 × 100
fu34	5.331 × 1010 ± 3.51 × 1010	8.041 × 109± 4.88 × 109	2.476 × 102 ± 2.13 × 103	4.896 × 105 ± 2.21 × 105	+	1.790 × 100 ± 2.67 × 100
fu35	9.384 × 102 ± 1.78 × 102	5.578 × 102 ± 8.53 × 101	1.612 × 103 ± 2.32 × 102	6.701 × 102 ± 6.36 × 101	+	1.046 × 10−2± 1.66 × 10−2
fu36	7.462 × 102 ± 2.32 × 102	2.422 × 102 ± 8.72 × 101	−1.273 × 102± 3.77 × 100	−1.082 × 102 ± 4.21 × 100	−	3.137 × 10−2 ± 6.55 × 10−2
fu37	5.5078 × 102 ± 1.83 × 10−1	5.478 × 102 ± 9.65 × 10−1	5.498 × 102 ± 4.64 × 10−2	5.492 × 102 ± 2.51 × 10−1	+	6.525 × 10−2 ± 4.01 × 10−2
fu38	−1.201 × 103 ± 4.77 × 101	−1.407 × 103 ± 3.24 × 101	−1.267 × 103 ± 5.18 × 101	−1.284 × 103 ± 3.90 × 101	−	0.000 × 100 ± 0.00 × 100
fu39	6.157 × 104 ± 1.54 × 104	4.98 × 10−27 ± 4.22 × 10−27	1.445 × 10−30± 5.58 × 10−31	4.314 × 10−3 ± 1.24 × 10−3	−	8.704 × 10−3 ± 2.16 × 10−4
fu40	7.518 × 104 ± 1.08 × 104	3.316 × 104 ± 8.12 × 103	5.665 × 102 ± 2.19 × 102	4.375 × 100 ± 9.83 × 10−1	+	1.463 × 10−1 ± 2.02 × 10−1
fu41	1.044 × 1010 ± 4.34 × 109	1.098 × 103 ± 1.86 × 103	2.575 × 102 ± 3.11 × 102	8.941 × 101 ± 5.26 × 101	−	1.397 × 100 ± 5.03 × 100
fu42	1.949 × 101 ± 2.59 × 10−1	8.003 × 100 ± 4.31 × 100	1.949 × 101± 1.48 × 10−1	1.277 × 100 ± 3.68 × 10−1	+	7.812 × 10−2 ± 3.72 × 10−2
fu43	2.978 × 10−1± 3.723 × 10−1	1.354 × 10−1 ± 2.31 × 10−1	6.857 × 100 ± 1.06 × 101	1.084 × 100 ± 3.16 × 10−1	+	2.039 × 10−2± 3.72 × 10−2
fu44	4.707 × 10−3 ± 7.38 × 10−3	0.001 × 100 ± 0.01 × 100	0.001 × 100 ± 0.01 × 100	0.001 × 100 ± 0.01 × 100	=	0.001 × 100 ± 0.01 × 100
fu45	4.258 × 104 ± 4.15 × 104	2.534 × 104 ± 6.28 × 103	4.066 × 103 ± 9.66 × 102	5.051 × 101 ± 4.28 × 101	+	1.433 × 101 ± 1.92 × 101
fu46	2.368 × 104 ± 3.45 × 103	2.01 × 104 ± 3.04 × 103	3.776 × 104 ± 6.47 × 103	2.320 × 104 ± 3.38 × 103	+	3.577 × 103 ± 8.68 × 103
fu47	2.087 × 106 ± 7.97 × 105	4.588 × 105 ± 1.69 × 105	1.589 × 104 ± 1.74 × 104	1.395 × 106 ± 1.14 × 106	−	3.749 × 106 ± 2.47 × 106

display the results for the SSPCCSO and other compared compact bio-inspired algorithms. In all of the experimental results of the 47 test functions, the SSPCCSO exhibited quite good performance. Compared with the cPSO, SSPCCSO is outperformed on 30 test functions, on the contrary, the cPSO exceeds SSPCCSO only on 17 benchmark functions. Among all of the memory-saving algorithms, SSPCCSO is out-performed by the other compared algorithms over 24 functions. From the view of the mathematics method, it lies in two factors, first of all, a differential operator in seeking mode is introduced to substitute for the original mutation operator. The difference between a solution $cat.x[i]$ and another variable may generate a moving direction, it may be a 360 degree angle transformation for the existing solution; the magnitude for the $cat.x[i]$ variation is decided by the size of the vector $(winner-loser)$. The potentially more efficient solution around $cat.x[i]$ will be found according to this searching method, which was similar to the cDE algorithm. Secondly, SSPCCSO also kept the search logic of the PSO, that is to say, the proposed SSPCCSO has the search ability of both the cPSO and cDE. It combines these two searching abilities; thus, it is not surprising that its search performance exceeds these two algorithms.

4.3. Comparison between the Corresponding Population-Based Algorithms and SSPCCSO

In addition to the comparison with memory saving algorithms, another group of comparisons between memory-saving and non-memory-saving algorithms are also used to test the performance of the algorithms; The comparison between sspccso, CSO, PSO and de will be arranged in this group of experiments.

Table 4. Comparison among SSPCCSO, CSO, PSO and DE.

Benmark	DE	PSO	W	CSO	W	SSPCCSO
fu1	8.269 × 101 ± 1.90 × 101	1.095 × 104 ± 2.30 × 103	−	0.000 × 100± 0.00 × 100	−	6.170 × 10−3 ± 1.05 × 10−3
fu2	3.063 × 104 ± 3.70 × 103	4.232 × 104 ± 1.84 × 103	+	0.000 × 100± 0.00 × 100	−	3.625 × 102 ± 7.02 × 102
fu3	2.715 × 100± 1.11 × 106	1.103 × 109 ± 5.07 × 108	+	2.890 × 101 ± 1.394 × 10−2	−	5.776 × 10−1 ± 7.01 × 100
fu4	4.072 × 101 ± 1.98 × 10−1	1.639 × 101 ± 1.21 × 100	+	0.001 × 100± 0.00 × 100	−	5.574 × 10−1 ± 3.07 × 10−2
fu5	7.195 × 101 ± 9.73 × 100	0.001 × 100± 0.001 × 100	−	0.001 × 100± 0.01 × 100	−	1.613 × 10−2 ± 2.37 × 10−3
fu6	2.151 × 102± 9.08 × 100	2.887 × 102± 3.28 × 101	+	0.001 × 100± 0.00 × 100	−	2.399 × 10−2 ± 4.09 × 10−3
fu7	2.408 × 105 ± 4.96 × 104	1.321 × 105 ± 1.03 × 104	+	2.990 × 102 ± 0.00 × 100	=	2.991 × 102 ± 1.75 × 10−1
fu8	6.328 × 103 ± 2.36 × 102	6.677 × 103 ± 6.44 × 102	−	3.161 × 103 ± 9.76 × 102	−	1.248 × 100± 1.90 × 100
fu9	1.633 × 104 ± 1.13 × 103	1.305 × 104 ± 3.17 × 103	+	1.344 × 104 ± 1.74 × 103	+	1.111 × 105 ± 5.29 × 104
fu10	8.508 × 105± 9.24 × 104	9.716 × 105 ± 1.57 × 105	−	3.222 × 106 ± 1.68 × 105	+	9.390 × 105 ± 1.13 × 104
fu11	4.217 × 100 ± 1.58 × 10−1	1.707 × 101± 1.73 × 100	+	−1.84 × 10−6 ± 0.01 × 100	−	8.328 × 10−2 ± 8.23 × 10−2
fu12	6.536 × 101± 1.02 × 101	1.139 × 101 ± 3.08 × 101	+	9.568 × 10−8± 2.68 × 10−8	=	1.018 × 10−8 ± 2.61 × 109
fu13	2.586 × 102±1.12 × 101	3.155 × 102± 2.19 × 101	+	2.701 × 102 ± 0.01 × 100	=	2.70 × 102± 1.81 × 10−5
fu14	4.003 × 101 ± 1.09 × 100	3.966 × 101± 1.18 × 100	−	7.049 × 102± 2.22 × 100	=	7.142 × 102 ± 2.37 × 10−1
fu15	7.443 × 10−2 ± 1.89 × 10−5	4.083 × 100 ± 2.23 × 100	+	0.001 × 100± 0.01 × 100	−	9.427 × 10−3 ± 6.15 × 10−3
fu16	−9.942 × 10−8± 1.08 × 10−1	−1.001 × 102± 0.01 × 100	=	−1.001 × 102± 8.46 × 10−5	=	−1.000 × 102± 0.00 × 100
fu17	9.424 × 10−8± 5.16 × 10−8	1.046 × 101 ± 5.08 × 100	−	1.631 × 100 ± 5.84 × 10−1	−	9.518 × 10−5 ± 1.57 × 10−6
fu18	−1.151 × 100± 3.37 × 10−7	3.502 × 103 ± 9.85 × 103	+	5454 × 10−1 ± 3.27 × 10−1	+	−4.104 × 10−1 ± 8.97 × 10−4
fu19	4.701 × 102 ± 1.44 × 101	6.107 × 102 ± 3.43 × 101	+	4.501 × 102± 0.01 × 100	=	4.500 × 102 ± 2.60 × 10−3
fu20	−1.278 × 101 ± 4.28 × 10−1	−1.936 × 101 ± 1.72 × × 100	+	−1.103 × 101 ± 1.08 × 100	+	−1.988 × 101 ± 2.33 × 10−1
fu21	1.268 × 104 ± 3.62 × 102	9.691 × 103 ± 1.14 × 103	−	4.785 × 103 ± 1.48 × 102	−	1.42 × 100 ± 2.26 × 100
fu22	1.828 × 101 ± 4.24 × 101	2.004 × 101 ± 3.75 × 10−1	+	0.001 × 100± 0.01 × 100	−	7.139 × 10−2 ± 4.17 × 10−2
fu23	1.611 × 102 ± 6.39 × 100	1.815 × 101 ± 1.01 × 101	+	4.658 × 10−2± 2.38 × 10−2	−	7.319 × 10−2 ± 5.17 × 10−3
fu24	2.386 × 104 ± 3.48 × 103	6.501 × 104 ± 9.68 × 103	+	0.001 × 100± 0.01 × 100	−	8.961 × 10−3 ± 1.46 × 10−2
fu25	−1.119 × 10−2± 1.29 × 10−3	−7.493 × 10−3 ± 1.06 × 10−3	−	0.001 × 100 ± 0.01 × 100	=	0.000 × 100 ± 0.00 × 100
fu26	−1.589 × 101 ± 5.26 × 10−1	−2.677 × 101 ± 2.01 × 100	−	−1.978 × 101 ± 1.43 × 100	=	−3.970 × 101 ± 1.52 × 10−1
fu27	8.899 × 104 ± 8.79 × 103	1.925 × 104 ± 1.93 × 104	+	0.001 × 100± 0.01 × 100	−	1.033 × 10−2 ± 1.39 × 10−2
fu28	1.177 × 103 ± 2.54 × 101	1.279 × 103± 4.45 × 101	+	0.001 × 100± 0.01 × 100	−	6.631 × 10−3 ± 1.24 × 10−4
fu29	2.636 × 1010 ± 5.09 × 109	3.854 × 1010 ± 1.42 × 1010	+	9.899 × 101± 1.85 × 100	−	1.286 × 100 ± 2.50 × 100
fu30	1.477 × 1011 ± 1.27 × 1010	1.017 × 1011 ± 1.96 × 1010	+	0.001 × 100± 8.36 × 100	−	1.109 × 108 ± 3.44 × 108
fu31	3.026 × 104 ± 4.79 × 102	2.368 × 104 ± 1.89 × 103	−	1.046 × 104± 2.95 × 103	−	4.989 × 104 ± 8.38 × 104
fu32	2.094 × 105 ± 1.64 × 104	1.138 × 104 ± 1.68 × 103	+	1.532 × 10−2± 3.81 × 10−3	−	1.774 × 10−2 ± 2.89 × 10−2
fu33	1.186 × 102 ± 2.84 × 100	1.407 × 102 ± 1.28 × 101	+	7.371 × 103 ± 3.33 × 100	+	−9.9 × 101 ± 0.00 × 100
fu34	8.197 × 1010 ± 1.12 × 1010	7.765 × 1010 ± 2.17 × 1010	+	9.898 × 102 ± 2.25 × 10−2	+	1.790 × 100 ± 2.67 × 100
fu35	1.398 × 103 ± 4.25 × 101	1.055 × 103 ± 1.48 × 102	+	0.001 × 100± 0.01 × 100	−	1.046 × 10−2 ± 1.66 × 10−2
fu36	1.567 × 103 ± 1.34 × 102	1.242 × 103 ± 2.45 × 102	+	1.083 × 102 ± 4.22 × 100	+	3.137 × 10−2 ± 6.55 × 10−2
fu37	5.507 × 102 ± 1.25 × 10−1	5.508 × 102 ± 1.71 × 10−1	+	5.493 × 102± 2.51 × 10−1	+	6.525 × 10−2 ± 4.01 × 10−2
fu38	−1.056 × 103 ± 1.09 × 101	−1.283 × 103 ± 2.18 × 102	−	−1.285 × 103± 3.91 × 101	−	0.000 × 100 ± 0.00 × 100
fu39	8.338 × 103 ± 1.12 × 103	1.201 × 103 ± 2.18 × 102	+	0.001 × 100± 0.01 × 100	−	8.704 × 10−3 ± 2.16 × 10−4
fu40	8.969 × 104 ± 7.75 × 103	1.701 × 104 ± 3.08 × 103	+	0.001 × 100± 0.01 × 100	−	1.463 × 10−1 ± 2.02 × 10−1
fu41	2.142 × 109 ± 5.71 × 108	1.784 × 107 ± 5.52 × 106	+	4.898 × 101 ± 1.38 × 10−2	−	1.397 × 100± 5.03 × 100
fu42	1.364 × 101 ± 4.46 × 10−1	6.878 × 100 ± 4.73 × 10−1	+	0.001 × 100± 0.01 × 100	−	7.812 × 10−2 ± 3.72 × 10−2
fu43	3.711 × 10−2 ± 3.27 × 101	2.555 × 10−2 ± 4.98 × 101	−	0.001 × 100± 0.01 × 100	−	2.039 × 10−2 ± 3.72 × 10−2
fu44	4.684 × 102 ± 1.35 × 101	0.001 × 100± 0.01 × 100	=	0.001 × 100± 0.01 × 100	=	0.001 × 100 ± 0.01 × 100
fu45	2.539 × 106 ± 2.09 × 105	1.134 × 106 ± 2.17 × 103	+	0.001 × 100± 0.01 × 100	−	1.433 × 101 ± 1.92 × 101
fu46	3.152 × 104 ± 1.18 × 103	1.888 × 104 ± 2.17 × 103	+	0.0010 × 100± 0.01 × 100	−	3.577 × 103 ± 8.68 × 103
fu47	4.675 × 106 ± 2.19 × 105	2.183 × 106± 4.01 × 105	−	1.694 × 107 ± 4.31 × 106	+	3.749 × 106 ± 2.47 × 106

presents the comparison results of the 47 test functions. SSPCCSO did better in 10 benchmarks, even when only one cat was employed. This status also happens in comparison between DE [23], cDE [2], PSO [24] and cPSO [4]. Obviously, Compared with population-based bio-inspired algorithms, the search ability of a single individual in SPCCSO is limited, and there is no large population collective cooperative search. However, the performance of SPCCSO still exceeds other algorithms.

4.4. Comparison against Swarm-Based Version Algorithms Based on Iterations and Solution

Another indicator of the algorithm’s performance is the convergence rate; after several iterations, different algorithms will converge to different results. A comparison of the convergence results based on the same test function and the same number of iterations is shown in

Table 5. The convergence results on test function 1 based on the same iterations.

Iterations	PSO	CSO	cPSO	SSPCCSO
I 100	8202.6317	0.000000	55.695	5.809
I 200	3754.9483	0.000000	19.852	4.277
I 1000	1417.4688	0.000000	0.61648	0.03692
I 2000	1414.2868	0.000000	0.36894	0.000032

. Test function 1 [22] is selected.

Table 5 shows that the SSPCCSO exceeds the other compared algorithms with a faster convergence rate, except for CSO. It ensures gradual convergence in the early iterations.

Table 6. The convergence results on test function 4 based on the same iterations.

Iterations	PSO	CSO	cPSO	SSPCCSO
I 100	14.38578	8.88 × 10−16	6.3458	10.602
I 200	12.03479	8.88 × 10−16	4.6725	4.942
I 500	10.31688	8.88 × 10−16	3.3370	3.824
I 1000	9.05237	8.88 × 10−16	2.3393	1.427

shows that the SSPCCSO is also much better than the PSO and CPSO in terms of the convergence results

Due to space constraints, no more comparison results are displayed. This situation also could be obtained with other test functions results.

Because of too many local cycles in the seeking mode, there is not any advantage shown in the computing costs. However, the gradient descent method introduced can make up for this shortcoming. It can end unnecessary calculations in advance, so the algorithm achieves a better performance in less running time.

5. Conclusions

A novel compact cat swarm optimization scheme based on gradient descent is proposed in this study. It kept the search logic of CSO, but introduced a gradient descent method into the scheme to seek for the optimal solution. According to the experimental results, this scheme could greatly reduce the computing costs. It also outperformed all of the relative compact optimization algorithms in most of the test benchmark functions. More significantly, its design is based on gamma probability distribution for solving small size sampling problems, so it probably suggests a new solution for optimization problems with small sampling.