Running-Time Analysis of Brain Storm Optimization Based on Average Gain Model

Abstract

The brain storm optimization (BSO) algorithm has received increased attention in the field of evolutionary computation. While BSO has been applied in numerous industrial scenarios due to its effectiveness and accessibility, there are few theoretical analysis results about its running time. Running-time analysis can be conducted through the estimation of the upper bounds of the expected first hitting time to evaluate the efficiency of BSO. This study estimates the upper bounds of the expected first hitting time on six single individual BSO variants (BSOs with one individual) based on the average gain model. The theoretical analysis indicates the following results. (1) The time complexity of the six BSO variants is $\mathrm{O}\left(\sqrt{n}\right)$ in equal coefficient linear functions regardless of the presence or absence of the disrupting operator, where n is the number of the dimensions. Moreover, the coefficient of the upper bounds on the expected first hitting time shows that the single individual BSOs with the disrupting operator require fewer iterations to obtain the target solution than the single individual BSOs without the disrupting operator. (2) The upper bounds on the expected first hitting time of single individual BSOs with the standard normally distributed mutation operator are lower than those of BSOs with the uniformly distributed mutation operator. (3) The upper bounds on the expected first hitting time of single individual BSOs with the $U\left(-\frac{1}{2},\frac{1}{2}\right)$ mutation operator are approximately twice those of BSOs with the $U(-1,1)$ mutation operator. The corresponding numerical results are also consistent with the theoretical analysis results.

1. Introduction

The swarm intelligence algorithm is one of the nature-inspired optimization algorithms that simulates the behavior of biological groups in nature [1,2,3]. Over the past two decades, many different types of swarm intelligence algorithms have been proposed, such as particle swarm optimization (PSO) [4,5], ant colony optimization (ACO) [6,7], artificial bee colony (ABC) [8,9], and brain storm optimization (BSO) [10,11,12]. Different from traditional methods, these algorithms solve problems by simulating the behavior of animal or human groups, with higher flexibility and adaptability.

BSO, a novel swarm intelligence algorithm, is inspired by the human brainstorming process. It is a continuous evolutionary algorithm that simulates the collective behavior of human beings. In recent years, BSOs have seen various practical applications in power systems [13,14,15,16], aviation design [17,18,19], mobile robot path planning [20,21], antenna design [22], financial optimization [23,24,25], and many other fields [26,27,28,29,30].

In addition, the theoretical analysis of BSO is also very important, especially for the practical application of BSO. The theoretical analysis benefits researchers in enabling them to understand the mechanism of the algorithm in guiding its design, improvement, and application in practice. The theoretical analysis can be divided into convergence analysis and running-time analysis. Zhou et al. [31], Qiao et al. [32], and Zhang et al. [33] have performed corresponding convergence analyses on BSO. The BSO–CAPSO algorithm, proposed by Alkmini [1], effectively enhances the computational efficiency of BSO through hybridization with chaotic accelerated particle swarm optimization. However, there are few works on the running-time analysis of BSO.

Some theoretical methods have been proposed as general analysis tools to investigate the running time of random heuristic algorithms, including the fitness level method [34], drift analysis method [35], switch analysis method [36], wave model [37], etc. These methods are mainly used to analyze discrete random heuristic algorithms. In contrast, fewer theoretical analysis results have been obtained for continuous random heuristic algorithms [38,39,40]. However, a large number of practical application problems are continuous. Therefore, the running time of continuous random heuristic algorithms has important research significance. To analyze the running time of continuous random heuristic algorithms, Huang et al. [41] proposed the average gain model.

Huang et al. [41,42] and Zhang et al. [43] used an average gain model to evaluate the expected first hit time of the ($1+1$)-evolutionary algorithm (($1+1$) EA), evolutionary strategy (ES), and covariance matrix adaptation evolution strategy (CMA-ES). The concept of the first hit time refers to the minimum number of iterations required before the algorithm finds an optimal solution [35]. The expected first hit time represents the average number of iterations needed to find the optimal solution, which actually reflects the average time complexity of the algorithm [44]. Wang [45] employed the average gain model to analyze the computational efficiency of the proposed swarm intelligence algorithm and provided theoretical evidence for its effectiveness. Therefore, the expected first hit time is a core metric in runtime analysis. Based on the average gain model, the expected first hit time of BSO is deeply analyzed in this paper.

The core of BSO consists of three key components: clustering, interruption, and update. The mutation operator plays an important role in BSO and is included in the interrupt operation and update operation. Specifically, the mutation operator helps the algorithm to jump out of the local optimal solution and further explore a broader search space by introducing random factors in the search process. For example, Zhan et al. [46] proposed an improved BSO (MBSO) in which the mutation operator employs a novel thought difference strategy (IDS). This strategy takes advantage of the thought differences among individuals in the group and increases the diversity of the group by introducing random factors, thus increasing the probability of the algorithm finding the global optimal solution. In addition, El-Abd [47] improved the step equation of the mutation operator and improved the performance of BSO by adjusting the step size and distribution. This improvement helps the algorithm to balance the local search and the global search better, so that the algorithm can find the global optimal solution more effectively when solving complex optimization problems.

The time complexity of the single individual BSO is analyzed in this paper based on the research process from simple to complex. The single individual BSO without the disrupting operation is the same as the (1+1) ES [41]. However, the corresponding results can explain the influence of the mutation operator and disrupting operator on the time complexity of BSO. In this paper, we choose the three most classic and representative distributions, $N(0,1)$, $U\left(-\frac{1}{2},\frac{1}{2}\right)$, and $U(-1,1)$, as the analysis objects for the mutation operator. Therefore, six BSO variants are obtained as the analyzed algorithms based on the combination of three mutation operators and the presence or absence of a disrupting operator.

The remainder of this paper is organized as follows. Section 2 introduces the process of BSO and the mathematical model of running-time analysis for BSO. Section 3 provides the theoretical analysis results for the running-time analysis of three different BSO variants. Section 4 presents the corresponding experimental results to evaluate the theoretical results. Finally, Section 5 concludes the paper.

2. Mathematical Model for Running-Time Analysis of BSO

2.1. Brain Storm Optimization

BSO was proposed by Shi [48,49] in 2011, and it is simple in concept and easy to implement. BSO can be simplified as follows.

In Steps 4 and 5, the new solution is generated by $\tilde{x}=x+\Delta$, where x is the original individual, $\tilde{x}$ is the newly generated individual, and $\Delta$ is a vector generated according to the mutation operator. In this paper, we focus on the running time of BSO with three different mutation operators. The same mutation operators are used to generate new individuals in both Steps 4 and 5. The superposition of different mutation operators is not considered in this work.

To accurately observe the effects of the disrupting and updating operations on the running time of BSO, we select a single individual form of BSO as the analyzed object. The single individual BSO framework simplifies the effect of the population size, which helps to evaluate the effect of the disrupting and updating operations on the running time. Furthermore, following the principle from simple to difficult, the single individual BSO is a suitable starting point for the running-time analysis of BSO. Moreover, the randomness of $\Delta$ and the design of the operations in Steps 4 and 5 are derived from the mutation operator design of evolutionary programming [50,51]. Therefore, the conclusion of this analysis will have positive implications for the study of similar mutation operators in evolutionary programming algorithms.

2.2. Stochastic Process Model of BSO

BSO can be represented as a stochastic process. In this section, we introduce the terminology for the analysis of the running time of BSO.

Definition 1 

(Hill-climbing task). Given a search space $S\subseteq R^n$ and a fitness function $f:S\to R$, the hill-climbing task is to find a solution ${\overrightarrow{x}}^{*}\in S,$ where the fitness of ${\overrightarrow{x}}^{*}$ reaches the target value H, where $f\left({\overrightarrow{x}}^{*}\right)\ge H$.

In this paper, we focus on analyzing the BSO running time in the hill-climbing task of a continuous search space.

Definition 2 

(State of BSO). The state of BSO at the t-th $(t=0,1,\dots )$ iteration is defined as $P_t=\left\{{\overrightarrow{\xi }}_1^t,{\overrightarrow{\xi }}_2^t,\dots ,{\overrightarrow{\xi }}_{\lambda }^t\right\}$, where λ is the size of the population, and ${\overrightarrow{\xi }}_1^t,{\overrightarrow{\xi }}_2^t,\dots ,{\overrightarrow{\xi }}_{\lambda }^t\in S$.

Definition 3 

(State space of BSO). The set of all possible BSO states is called the state space of BSO, denoted as

$$\Omega =S^{\lambda }=\left\{\left\{{\overrightarrow{\xi }}_1,{\overrightarrow{\xi }}_2,\dots ,{\overrightarrow{\xi }}_{\lambda }\right\}|{\overrightarrow{\xi }}_k\in S,k=1,\dots ,\lambda \right\}.$$

(1)

An optimization problem is a mapping from a decision space to an objective space, and the state space of BSO represents the corresponding decision space.

Definition 4 

(Expected first hitting time). Let ${\left\{X_t\right\}}_{t=0}^{\infty }$ be a stochastic process, where, for any $t\ge 0$, $X_t\ge 0$ holds. Suppose that $X_t$ is the Euclidean distance value of the t-th iteration state of BSO to the target solution, and the target threshold $\epsilon >0$, the first hitting time [35] of the ε-approximation solution, can be defined by

$$T_{\epsilon }=\min \{t\ge 0:X_t\le \epsilon \}.$$

(2)

Therefore, the expected first hitting time [44] of BSO can be denoted with $E\left(T_{\epsilon }\right|X_0)$.

The expected first hit time refers to the average number of iterations required for the BSO algorithm to reach the target fitness value. This metric can more accurately measure the performance of an algorithm because it takes into account the probability distribution of the algorithm over different iterations. Through this metric, we can evaluate the average time complexity of the algorithm in finding the optimal solution, so as to better understand the efficiency of the algorithm.

2.3. Running-Time Analysis of BSO Based on Average Gain Model

Inspired by drift analysis [52] and the idea of Riemann integrals [53], Huang et al. [41] proposed the average gain model. Zhang et al. [43] separated this model by introducing the concepts of the supermartingale and stopping time. Based on the former research results [41,43] of the average gain model, Huang et al. [42] proposed an experimental method to estimate the running time of the continuous evolution algorithms.

The expected one-step variation

$$\begin{array}{c}\hfill {\delta }_t=E(X_t-X_{t+1}|H_t),t\ge 0\end{array}$$

(3)

is called the average gain, where ${\left\{X_t\right\}}_{t=0}^{\infty }$ is a stochastic process, and $H_t=\sigma (X_0,X_1,\dots ,X_t)$. The $\sigma$-algebra $H_t$ contains all the events generated by $X_0,X_1,\dots ,X_t$. All the information observed from the original population to the t-th iteration is recorded in $H_t$.

Based on Definition 2, $P_t=\{{\overrightarrow{\xi }}_1^t,{\overrightarrow{\xi }}_2^t,\dots ,{\overrightarrow{\xi }}_{\lambda }^t\}$ is the state of BSO at the t-th iteration. The process of BSO in solving the hill-climbing task is considered as the gradual process of the stochastic state from the initial population to the population that contains the optimal solution. Let $f\left(P_t\right)=\max \{f\left(\overrightarrow{x}\right):\overrightarrow{x}\in P_t\}$ be the highest fitness value of individuals in $P_t$. $X_t$ is used to measure the distance of the current population to the population of the target value. $X_t=f^{*}-f\left(P_t\right)$, where $f^{*}$ is the fitness value of the optimal solution. Obviously, ${\left\{X_t\right\}}_{t=0}^{\infty }$ is a non-negative stochastic process.

The state of BSO in the $(t+1)$-th iteration $P_{t+1}$ only depends on $P_t$. In other words, the stochastic process ${\left\{P_t\right\}}_{t=0}^{\infty }$ can be modeled by a Markov chain [42]. Similarly, ${\left\{X_t\right\}}_{t=0}^{\infty }$ can also be regarded as a Markov chain. In this case, the average gain ${\delta }_t=E(X_t-X_{t+1}|H_t)$ can be simplified to ${\delta }_t=E(X_t-X_{t+1}|X_t)$. Based on Th. 2 of  [43], the expectation of $T_{\epsilon }$ of BSO can be estimated as follows.

Theorem 1. 

Suppose that ${\left\{X_t\right\}}_{t=0}^{\infty }$ is a stochastic process associated with BSO, where $X_t\ge 0$ for all $t\ge 0$. Let $h:[0,A]\to R^{+}$ be a monotonically increasing and integrable function. If $E(X_t-X_{t+1}|X_t)\ge h\left(X_t\right)$ and $X_t>\epsilon >0$, it holds for $T_{\epsilon }$ that

$$E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{X_0}\frac{1}{h\left(x\right)}x.$$

(4)

Theorem 1 shows the upper bounds on the expected first hitting time of BSO based on the average gain model. The average gain ${\delta }_t$ plays a key role in analyzing the first hitting time $T_{\epsilon }$ of the $\epsilon$-approximation solution for BSO. The higher average gain indicates a more efficient iteration of the optimization process.

3. Running-Time Analysis of BSO Instances for Equal Coefficient Linear Functions

In this section, we present the theoretical analysis results based on the average gain model to analyze the expected first hitting time of BSO for equal coefficient linear functions. The running time of BSO with three different mutation operators is analyzed from the perspective of whether the disrupting operation exists. In this paper, we refer to the BSO without a disrupting operation as BSO-I, and the BSO with a disrupting operation as BSO-II.

On this basis, the equal coefficient linear functions are selected as the research object [54,55,56]. These functions are a form of basic continuous optimization problem whose function expression is as follows:

$$\begin{array}{c}\hfill f(x_1,x_2\dots ,x_n)=k(x_1+x_2+\dots +x_n)=k\sum _{i=1}^n{x}_i,\end{array}$$

(5)

where $(x_1,x_2,\dots ,x_n)\in S$. It is assumed that the function starts from the origin and sets the target fitness value to $na$, where $a>0$. The objective of optimizing the equal coefficient linear function is to find a solution ${\overrightarrow{x}}^{*}\in S$, such that $f\left({\overrightarrow{x}}^{*}\right)\ge na$.

The mutation operator that obeys the Gaussian distribution and the uniform distribution is selected for the evaluation of BSO. The Gaussian distribution and uniform distribution are common tools for the design of mutation operators [50,51,57], so it is representative to select these two distributions as research cases.

3.1. Case Study of BSO without Disrupting Operator

Since the single individual BSO is analyzed in this paper, $\lambda$ is equal to 1 in the state $P_t=\{{\xi }_1^t,{\xi }_2^t,\dots ,{\xi }_{\lambda }^t\}$ of BSO. The BSO of a single individual has only one individual, so ${\xi }_1^t$ represents both the optimized individual and the random state of the algorithm. The procedure of single individual BSO can be described as follows when the disrupting operation does not exist (i.e., Step 4 in Algorithm 1 is ignored).

Algorithm 1 Brain Storm Optimization (BSO)

1: Initialization: Randomly generate $\lambda$ individuals (potential solutions) to form the initial population $P=\{{\xi }_1,{\xi }_2,\dots ,{\xi }_{\lambda }\}$ and evaluate the $\lambda$ individuals; 2: while fail to achieve the predetermined maximum number of iterations do 3:    Clustering: Use clustering algorithms to divide $\lambda$ individuals into m clusters; 4:    Disrupting: The mutation occurs with a certain probability, and a randomly selected cluster’s central individual is replaced by a randomly generated new individual; 5:    Updating: Randomly choose one or two clusters to create a new individual;    Compare the newly generated individual and the original individual with the same individual index. The better one will be saved as the new individual;    Update the whole population; the offspring population is recorded as $P^{\prime }=\{{\xi }_1^{\prime },{\xi }_2^{\prime },\dots ,{\xi }_{\lambda }^{\prime }\}$. Evaluate the individuals in $P^{\prime }$; 6: end while 7: Output the most optimal solution discovered.

${\overrightarrow{x}}^t=(x_1^t,x_2^t,\dots ,x_n^t)\in S,t=0,1,\dots$ is the t-th generation of the algorithm.

$$\begin{array}{c}\hfill X_t=an-f\left({\overrightarrow{x}}^t\right)=an-k(x_1^t+x_2^t,\dots +x_n^t),\end{array}$$

(6)

is defined as the Euclidean distance of the t-th iteration to the optimal solution. The gain at t-th is given by

$$\begin{array}{cc}\hfill {\eta }_t& =X_t-X_{t+1}\hfill \\ \hfill & =k(x_1^{t+1}+x_2^{t+1}+\dots +x_n^{t+1})-k(x_1^t+x_2^t+\dots +x_n^t).\hfill \end{array}$$

(7)

3.1.1. When $z_i\sim N(0,1)$

If the mutation operator obeys the standard normal distribution $N(0,1)$, the distribution function of ${\eta }_t$ is as presented by Lemma 1.

Lemma 1. 

For BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $N(0,1)$, the distribution function $F\left(u\right)=P({\eta }_t\le u)$ of the gain ${\eta }_t$ is

$$F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{1}{\sqrt{2\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{t^2}{2n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..$$

(8)

Proof. 

According to Step 4 and Step 5 of Algorithm 2, the ($t+1$)-th individual is

$$\begin{array}{c}\hfill {\overrightarrow{x}}^{t+1}=\left\{\begin{array}{cc}{\overrightarrow{x}}^t,\hfill & f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right)\hfill \\ {\overrightarrow{y}}^t,\hfill & f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right)\hfill \end{array}\right..\end{array}$$

(9)

Algorithm 2 BSO-I

1: Initialization: Randomly generate an individual $\overrightarrow{x}=(x_1,x_2,\dots ,x_n)\in R^n$ based on uniform distribution; 2: while stopping criterion is not satisfied do 3:    $\overrightarrow{y}=\overrightarrow{x}+\overrightarrow{z}$, where $\overrightarrow{z}$ is the mutation operator; 4:    if $\overrightarrow{y}$ adapts better than $\overrightarrow{x}$ then 5:       $\overrightarrow{x}$ is substituted for $\overrightarrow{y}$ 6:    end if 7: end while Output:  $\overrightarrow{x}$

According to the definition of ${\eta }_t$, where $t=0,1,\dots$,

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right)$,

$$\begin{array}{cc}\hfill {\eta }_t& =k(x_1^{t+1}+x_2^{t+1}+\dots +x_n^{t+1})-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k(x_1^t+x_2^t+\dots +x_n^t)-k(x_1^t+x_2^t+\dots +x_n^t)=0\hfill \end{array}$$

(10)

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right)$,

$$\begin{array}{cc}\hfill {\eta }_t& =k(x_1^{t+1}+x_2^{t+1}+\dots +x_n^{t+1})-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k(y_1^t+y_2^t+\dots +y_n^t)-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k[(y_1^t-x_1^t)+(y_2^t-x_2^t)+\dots +(y_n^t-x_n^t)]\hfill \\ & =k(z_1^t+z_2^t+\dots +z_n^t)=f\left({\overrightarrow{z}}^t\right)\hfill \end{array}$$

(11)

Since $z_i\sim N(0,1)$, $z_1,\dots ,z_n$ are independent of each other. All of the $z_i$ satisfy the additivity of the normal distribution, so $f\left({\overrightarrow{z}}^t\right)$ obeys the distribution of $N(0,n{k}^2)$.

Hence, the distribution function of ${\eta }_t$ is shown as $F\left(u\right)=P({\eta }_t\le u)$.

(1)

If $u<0$, according to the definition of ${\eta }_t$ where ${\eta }_t\ge 0$, it has $F\left(u\right)=0$.

(2)

If $u=0$, the probability density function of $N(0,n)$ is symmetric in the y axis, so $F\left(u\right)=P({\eta }_t\le u)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=P({\eta }_t\le u)=\frac{1}{\sqrt{2\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{t^2}{2n{k}^2}}t$.

Lemma 1 holds.    □

Theorem 2 is presented based on the above proof.

Theorem 2. 

If the mutation operator $\overrightarrow{z}$ of BSO-I obeys $N(0,1)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as follows.

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+\sqrt{2\pi n}\frac{a}{k}-\sqrt{\frac{2\pi }{n}}\frac{\epsilon }{k}.\end{array}$$

(12)

Proof. 

$$\begin{array}{c}E(X_t-X_{t+1}|X_t)=E\left({\eta }_t\right|X_t)={\int }_{-\infty }^{+\infty }uF\left(u\right)\\ ={\int }_0^{+\infty }u\left(\frac{1}{\sqrt{2\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{t^2}{2n{k}^2}}t\right)=\frac{k\sqrt{n}}{\sqrt{2\pi }}\end{array}$$

(13)

It is assumed that the algorithm starts from the origin at initialization, where ${\overrightarrow{x}}^0=(0,0,\dots ,0)$, i.e.,

$$\begin{array}{c}\hfill X_0=na-f\left({\overrightarrow{x}}^0\right)=na-k(0+0\dots +0)=na\end{array}$$

(14)

According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{na}\frac{\sqrt{2\pi }}{k\sqrt{n}}x=1+\sqrt{2\pi n}\frac{a}{k}-\sqrt{\frac{2\pi }{n}}\frac{\epsilon }{k}.\end{array}$$

(15)

Theorem 2 holds.    □

Theorem 2 indicates that for BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $N(0,1)$, the computational time complexity of BSO-I for the equal coefficient linear function is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$.

3.1.2. When $z_i\sim U\left(-\frac{1}{2},\frac{1}{2}\right)$

The uniform distribution function does not satisfy additivity like the normal distribution function. The Lindeberg–Levy center limit theorem [58] can provide an idea to find the distribution of ${\eta }_t$. The Lindeberg–Levy center limit theorem is introduced below.

Suppose that $X_n$ is a sequence of independent and identically distributed random variables with $E\left(X_i\right)=\mu$ and $\left(X_i\right)={\sigma }^2>0$; let

$$\begin{array}{c}\hfill Y_n^{*}=\frac{X_1+X_2+\dots +X_n-n\mu }{\sigma \sqrt{n}},\end{array}$$

(16)

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }P(Y_n^{*}\le y)=\Phi \left(y\right)=\frac{1}{\sqrt{2\pi }}{\int }_{-\infty }^y{e}^{-\frac{t^2}{2}}t\end{array}$$

(17)

is satisfied for any real number y.

The Lindeberg–Levy center limit theorem [58] shows that if n is sufficiently large, $Y_n^{*}\sim N(0,1)$, it has ${\sum }_{i=1}^n{X}_i\sim N(n\mu ,n{\sigma }^2)$. Generally, the case of higher dimensions requires more attention in the study of the computational time complexity of algorithms. If the mutation operator obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the distribution function of ${\eta }_t$ can be represented by Lemma 2.

Lemma 2. 

For BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the distribution function $F\left(u\right)=P({\eta }_t\le u)$ of the gain ${\eta }_t$ is

$$\begin{array}{c}\hfill F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{\sqrt{6}}{\sqrt{\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{6t^2}{n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..\end{array}$$

(18)

Proof. 

According to the definition of ${\eta }_t$, where $t=0,1,\dots$.

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right),{\eta }_t=0$.

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right),{\eta }_t=k(z_1^t+z_2^t+\dots +z_n^t)$, where $z_i\sim U\left(-\frac{1}{2},\frac{1}{2}\right)$, and $z_1,\dots ,z_n$ are independent of each other. According to the Lindeberg–Levy center limit theorem, ${\eta }_t$ obeys $N\left(0,\frac{1}{12}n{k}^2\right)$.

Hence, the distribution function of ${\eta }_t$ is $F\left(u\right)=P({\eta }_t\le u)$.

(1)

If $u<0$, $F\left(u\right)=0$.

(2)

If $u=0$, $F\left(u\right)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=\frac{\sqrt{6}}{\sqrt{\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{6t^2}{n{k}^2}}t$.

Lemma 2 holds.    □

Theorem 3 is presented based on the above proof.

Theorem 3. 

For BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as follows.

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+2\sqrt{6\pi n}\frac{a}{k}-\frac{2\sqrt{6\pi }}{\sqrt{n}}\frac{\epsilon }{k}.\end{array}$$

(19)

The proof of this theorem is based on the same principle as Theorem 2. The detailed derivation is given as follows.

Proof. 

According to Lemma 2, we have

$$\begin{array}{cc}& E(X_t-X_{t+1}|X_t)=E\left({\eta }_t\right|X_t)={\int }_{-\infty }^{+\infty }uF\left(u\right)\hfill \\ & ={\int }_0^{+\infty }u\left(\frac{\sqrt{6}}{\sqrt{\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{6t^2}{n{k}^2}}t\right)=\frac{k\sqrt{n}}{2\sqrt{6\pi }},\hfill \end{array}$$

(20)

and the algorithm starts from the origin at initialization, ${\overrightarrow{x}}^0=\left(0,0,\dots ,0\right)$, i.e., $X_t=na$. According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{na}\frac{2\sqrt{6\pi }}{k\sqrt{n}}x=1+2\sqrt{6\pi n}\frac{a}{k}-\frac{2\sqrt{6\pi }}{\sqrt{n}}\frac{\epsilon }{k}. \square \end{array}$$

(21)

Theorem 3 indicates that if the mutation operator $\overrightarrow{z}$ of BSO-I obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, its computational time complexity is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$ for the equal coefficient linear function.

3.1.3. When $z_i\sim U(-1,1)$

If the mutation operator obeys $U(-1,1)$, the distribution function of ${\eta }_t$ can be represented by Lemma 3.

Lemma 3. 

For BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $U(-1,1)$, the distribution function $F\left(u\right)=P({\eta }_t\le u)$ of the gain ${\eta }_t$ is

$$\begin{array}{c}\hfill F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{\sqrt{3}}{\sqrt{2\pi nk}}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{2n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..\end{array}$$

(22)

The proof of this lemma is based on the same principle as Lemma 2.

Proof. 

According to the definition of ${\eta }_t,t=0,1,\dots$,

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right),{\eta }_t=0$.

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right),{\eta }_t=f\left({\overrightarrow{z}}^t\right)$.

Since $z_i\sim U(-1,1)$, $z_1,\dots ,z_n$ are independent of each other, according to the Lindeberg–Levy center limit theorem, $f\left({\overrightarrow{z}}^t\right)$ obeys $N(0,\frac{1}{3}n{k}^2)$.

Hence, the ${\eta }_t$ distribution function $F\left(u\right)=P({\eta }_t\le u)$ is

(1)

If $u<0$, $F\left(u\right)=0$.

(2)

If $u=0$, $F\left(u\right)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=\frac{\sqrt{3}}{\sqrt{2\pi nk}}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{2n{k}^2}}t$.

   □

According to Lemma 3 and Theorem 1, Theorem 4 can be inferred.

Theorem 4. 

For BSO-I, if its mutation operator $\overrightarrow{z}$ obeys $U(-1,1)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+\sqrt{6\pi n}\frac{a}{k}-\sqrt{\frac{6\pi }{n}}\frac{\epsilon }{k}.\end{array}$$

(23)

The proof of this theorem is based on the same principle as Theorem 2. The proof is given as follows.

Proof. 

According to Lemma 3, we have

$$\begin{array}{c}E(X_t-X_{t+1}|X_t)=E\left({\eta }_t\right|X_t)={\int }_{-\infty }^{+\infty }uF\left(u\right)\\ ={\int }_0^{+\infty }u\left(\frac{\sqrt{3}}{\sqrt{2\pi nk}}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{2n{k}^2}}t\right)=\frac{k\sqrt{n}}{\sqrt{6\pi }},\end{array}$$

(24)

and the algorithm starts from the origin at initialization, ${\overrightarrow{x}}^0=(0,0,\dots ,0)$, i.e., $X_t=na$. According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{na}\frac{\sqrt{6\pi }}{k\sqrt{n}}x=1+\sqrt{6\pi n}\frac{a}{k}-\sqrt{\frac{6\pi }{n}}\frac{\epsilon }{k}. \square \end{array}$$

(25)

Theorem 4 indicates that if the mutation operator $\overrightarrow{z}$ of BSO-I obeys $U(-1,1)$, its computational time complexity for the equal coefficient linear function is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$.

The time complexity of BSO-I with three different mutation operators is $\mathrm{O}\left(\sqrt{n}\right)$. In the next section, we will discuss the running time of BSO considering the case with a disrupting operation.

3.2. Case Study of BSO with Disrupting Operator

Based on the average gain model, this section analyzes the upper bounds of the expected first hit time in three BSO cases. When interference operations are added to the single individual BSO, the algorithm process can be simplified as follows.

In Algorithm 3, the disrupting operations, which are executed with a small probability, are shown in Steps 3 to 6. Let $A=\left\{P{b}^{\prime }\right|P{b}^{\prime }<Pb\}$ indicate that replacement occurs, while $\overline{A}=\left\{P{b}^{\prime }\right|P{b}^{\prime }\ge Pb\}$ indicates that no replacement occurs.

Algorithm 3 BSO-II

1: Initialization: Randomly generate an individual $\overrightarrow{x}=(x_1,x_2,\dots ,x_n)\in R^n$ based on uniform distribution; 2: while stopping criterion is not satisfied do 3:    Randomly generate a value $P{b}^{\prime }$ from 0 to 1 based on uniform distribution; 4:    if $P{b}^{\prime }$ is smaller than a pre-determined probability $Pb$ then 5:        replace $\overrightarrow{x}$ with a randomly generated individual $\overrightarrow{b}=(b_1,b_2,\dots ,b_n)$ based on uniform distribution; 6:    end if 7:    if $P{b}^{\prime }<Pb$ then 8:        $\overrightarrow{y}=\overrightarrow{b}+\overrightarrow{z}$, where $\overrightarrow{z}$ is a mutation operator; 9:    else 10:      $\overrightarrow{y}=\overrightarrow{x}+\overrightarrow{z}$; 11:   end if 12:   if $\overrightarrow{y}$ has better fitness than $\overrightarrow{x}$ then 13:       replace $\overrightarrow{x}$ with $\overrightarrow{y}$ 14:   end if 15: end while Output:  $\overrightarrow{x}$

To highlight the effect of each mutation operator on the algorithm, we choose the same mutation operators of BSO-I in Steps 5 and 8 to generate new individuals. As a result, $b_i=x_i+\Delta x_i$, where mutation operator parameter $\Delta x_i$ and $z_i$ follow the same distribution.

Here, ${\overrightarrow{x}}^t=(x_1^t,x_2^t,\dots ,x_n^t)\in S$ is still the t-th individual of the algorithm. We have $X_t=an-f\left({\overrightarrow{x}}_t\right)=an-k(x_1^t+x_2^t,\dots +x_n^t)$, and the corresponding gain at t is given by ${\eta }_t=X_t-X_{t+1}=k(x_1^{t+1}+x_2^{t+1}+\dots +x_n^{t+1})-k(x_1^t+x_2^t+\dots +x_n^t)$.

3.2.1. When $z_i\sim N(0,1)$

(1)

If $P{b}^{\prime }\ge Pb$, it is the same as the result of the case with no disrupting operation in Section 3.1, and the average gain is

$$\begin{array}{c}\hfill E(X_t-X_{t+1}|X_t,\overline{A})=E\left({\eta }_t\right|X_t,\overline{A})=\frac{k\sqrt{n}}{\sqrt{2\pi }}.\end{array}$$

(26)

(2)

If $P{b}^{\prime }<Pb$ and the mutation operator obeys $N(0,1)$, the distribution function of ${\eta }_t$ is represented by Lemma 4.

Lemma 4. 

For BSO-II, if its mutation operator $\overrightarrow{z}$ obeys $N(0,1)$ and $P{b}^{\prime }<Pb$, the distribution function of the gain ${\eta }_t$ is $F\left(u\right)=P({\eta }_t\le u)$.

$$\begin{array}{c}\hfill F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{1}{2k\sqrt{\pi n}}{\int }_{-\infty }^u{e}^{-\frac{t^2}{4n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..\end{array}$$

(27)

Proof. 

According to Step 12 and Step 13 of Algorithm 3, the ($t+1$)-th individual is ${\overrightarrow{x}}^{t+1}=\left\{\begin{array}{cc}{\overrightarrow{x}}^t,\hfill & f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right)\hfill \\ {\overrightarrow{y}}^t,\hfill & f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right)\hfill \end{array}\right.$.

According to the definition of ${\eta }_t,t=0,1,\dots$,

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right),{\eta }_t=0$.

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right)$,

$$\begin{array}{cc}\hfill {\eta }_t& =k(x_1^{t+1}+x_2^{t+1}+\dots +x_n^{t+1})-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k[(b_1^t+b_2^t+\dots +b_n^t)+(z_1^t+z_2^t+\dots +z_n^t)]-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k[(x_1^t+x_2^t+\dots +x_n^t)+(\Delta x_1^t+\Delta x_2^t+\dots +\Delta x_n^t)]\hfill \\ & +k(z_1^t+z_2^t+\dots +z_n^t)-k(x_1^t+x_2^t+\dots +x_n^t)\hfill \\ & =k[(\Delta x_1^t+\Delta x_2^t+\dots +\Delta x_n^t)+(z_1^t+z_2^t+\dots +z_n^t)]\hfill \end{array}$$

(28)

Since $z_i\sim N(0,1)$, $\Delta x_i\sim N(0,1)$, and $z_1,\dots ,z_n$ are independent of each other, $\Delta x_1,\Delta x_2,\dots ,\Delta x_n$ are also independent of each other. All of $z_i$ and $\Delta x_i$ satisfy the additivity of the normal distribution. As a result, ${\eta }_t$ obeys $N(0,2n{k}^2)$.

Hence, the distribution function of ${\eta }_t$ is $F\left(u\right)=P({\eta }_t\le u)$.

(1)

If $u<0$, $F\left(u\right)=0$.

(2)

If $u=0$, $F\left(u\right)=P({\eta }_t\le u)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=P({\eta }_t\le u)=\frac{1}{2k\sqrt{\pi n}}{\int }_{-\infty }^u{e}^{-\frac{t^2}{4n{k}^2}}t$.

Lemma 4 holds. □

Based on the above proofs, we can conclude that

$$\begin{array}{c}\hfill E(X_t-X_{t+1}|X_t,A)=E\left({\eta }_t\right|X_t,A)={\int }_0^{+\infty }u\left(\frac{1}{2k\sqrt{\pi n}}{\int }_{-\infty }^u{e}^{-\frac{t^2}{4n{k}^2}}t\right)=\frac{k\sqrt{n}}{\sqrt{\pi }}\end{array}$$

(29)

Assume that the probability $Pb$ is equal to 0.2 [59]; Theorem 5 can be obtained according to Theorem 1.

Theorem 5. 

For BSO-II, if its mutation operator $\overrightarrow{z}$ obeys $N(0,1)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as follows.

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+\frac{5\sqrt{2\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{2\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}.\end{array}$$

(30)

Proof. 

$$\begin{array}{cc}E(X_t-X_{t+1}|X_t)& =P\left(\overline{A}\right)E(X_t-X_{t+1}|X_t,\overline{A})+P\left(A\right)E(X_t-X_{t+1}|X_t,A)\hfill \\ & (\mathrm{full} \mathrm{expectation} \mathrm{formula})\hfill \\ & =(1-0.2)\times E(X_t-X_{t+1}|X_t,\overline{A})+0.2\times E(X_t-X_{t+1}|X_t,A)\hfill \\ & =0.8\times \frac{k\sqrt{n}}{\sqrt{2\pi }}+0.2\times \frac{k\sqrt{n}}{\sqrt{\pi }}=\frac{4k\sqrt{n}+k\sqrt{2n}}{5\sqrt{2\pi }}\hfill \end{array}$$

(31)

The algorithm starts from the origin at initialization, ${\overrightarrow{x}}^0=(0,0,\dots ,0)$, i.e., $X_t=na$. According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{cc}E\left(T_{\epsilon }\right|X_0)& \le 1+{\int }_{\epsilon }^{na}\frac{5\sqrt{2\pi }}{4k\sqrt{n}+k\sqrt{2n}}x\hfill \\ & =1+\frac{5\sqrt{2\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{2\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}\hfill \end{array}$$

(32)

Theorem 5 holds. □

According to the proof of Theorem 5, if the mutation operator $\overrightarrow{z}$ of BSO-II obeys $N(0,1)$, its computational time complexity for the equal coefficient linear function is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$.

3.2.2. When $z_i\sim U\left(-\frac{1}{2},\frac{1}{2}\right)$

(1)

If $P{b}^{\prime }\ge Pb$, the result is the same as the case in Section 3.1 with no disrupting operation. The average gain is

$$\begin{array}{c}\hfill E(X_t-X_{t+1}|X_t,\overline{A})=E\left({\eta }_t\right|X_t,\overline{A})=\frac{k\sqrt{n}}{2\sqrt{6\pi }}.\end{array}$$

(33)

(2)

If $P{b}^{\prime }<Pb$ and the mutation operator obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the distribution function of ${\eta }_t$ is represented by Lemma 5.

Lemma 5. 

If the mutation operator of BSO-II $\overrightarrow{z}$ obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$ and $P{b}^{\prime }<Pb$, the distribution function of the gain ${\eta }_t$ is $F\left(u\right)=P\left({\eta }_t\le u\right)$.

$$\begin{array}{c}\hfill F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{\sqrt{3}}{\sqrt{\pi nk}}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..\end{array}$$

(34)

The proof of this lemma is based on the same principle as Lemma 4. The detailed derivation is given as follows.

Proof. 

According to the definition of ${\eta }_t,t=0,1,\dots$,

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right),{\eta }_t=0$.

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right),{\eta }_t=k[(\Delta x_1^t+\Delta x_2^t+\dots +\Delta x_n^t)+(z_1^t+z_2^t+\dots +z_n^t)]$

Since $z_i\sim U\left(-\frac{1}{2},\frac{1}{2}\right)$, $\Delta x_i\sim U\left(-\frac{1}{2},\frac{1}{2}\right)$, and $z_1,\dots ,z_n$ are independent of each other, $\Delta x_1,\Delta x_2,\dots ,\Delta x_n$ are also independent of each other. $f\left({\overrightarrow{z}}^t\right)$ obeys $N\left(0,\frac{1}{6}n{k}^2\right)$ according to the Lindeberg–Levy center limit theorem.

Hence, the ${\eta }_t$ distribution function $F\left(u\right)=P({\eta }_t\le u)$ is

(1)

If $u<0$, $F\left(u\right)=0$.

(2)

If $u=0$, $F\left(u\right)=P({\eta }_t\le u)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=P({\eta }_t\le u)=\frac{\sqrt{3}}{\sqrt{\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{n{k}^2}}t$.

□

Theorem 6 can be concluded based on Lemma 5 and Theorem 1.

Theorem 6. 

If the mutation operator of BSO-II $\overrightarrow{z}$ obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as follows.

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+\frac{10\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{10\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}.\end{array}$$

(35)

The proof of this theorem is based on the same principle as Theorem 5. The detailed derivation is presented as follows.

Proof. 

According to Lemma 5, we have

$$\begin{array}{cc}\hfill & E(X_t-X_{t+1}|X_t,A)=E\left({\eta }_t\right|X_t,A)={\int }_0^{+\infty }u\left(\frac{\sqrt{3}}{\sqrt{\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{n{k}^2}}t\right)=\frac{k\sqrt{n}}{2\sqrt{3\pi }}.\hfill \end{array}$$

(36)

Suppose that the probability $Pb=0.2$; according to Theorem 1, the following conclusions can be drawn:

$$\begin{array}{cc}& E(X_t-X_{t+1}|X_t)\hfill \\ & =P\left(\overline{A}\right)E(X_t-X_{t+1}|X_t,\overline{A})+P\left(A\right)E(X_t-X_{t+1}|X_t,A)\hfill \\ & =(1-0.2)\times E(X_t-X_{t+1}|X_t,\overline{A})+0.2\times E(X_t-X_{t+1}|X_t,A)\hfill \\ & =0.8\times \frac{k\sqrt{n}}{2\sqrt{6\pi }}+0.2\times \frac{k\sqrt{n}}{2\sqrt{3\pi }}=\frac{4k\sqrt{n}+k\sqrt{2n}}{10\sqrt{6\pi }}\hfill \end{array}$$

(37)

The algorithm starts from the origin at initialization, ${\overrightarrow{x}}^0=(0,0,\dots ,0)$, i.e., $X_t=na$. According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{na}\frac{10\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}x=1+\frac{10\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{10\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}. \square \end{array}$$

(38)

Theorem 6 indicates that for BSO-II, if its mutation operator $\overrightarrow{z}$ obeys $U\left(-\frac{1}{2},\frac{1}{2}\right)$, the computational time complexity of BSO-II for the equal coefficient linear function is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$.

3.2.3. When $z_i\sim U(-1,1)$

(1)

If $P{b}^{\prime }\ge Pb$, the result is the same as the case in Section 3.1 with no disrupting operation. The average gain is

$$\begin{array}{c}\hfill E(X_t-X_{t+1}|X_t,\overline{A})=E\left({\eta }_t\right|X_t,\overline{A})=\frac{k\sqrt{n}}{\sqrt{6\pi }}.\end{array}$$

(39)

(2)

If $P{b}^{\prime }<Pb$, and the mutation operator obeys $U(-1,1)$, the distribution function of ${\eta }_t$ is represented by Lemma 6.

Lemma 6. 

If the mutation operator of BSO-II $\overrightarrow{z}$ obeys $U(-1,1)$ and $P{b}^{\prime }<Pb$, its distribution function $F\left(u\right)=P({\eta }_t\le u)$ of the gain ${\eta }_t$ is

$$\begin{array}{c}\hfill F\left(u\right)=\left\{\begin{array}{cc}0,\hfill & u<0\hfill \\ \frac{1}{2},\hfill & u=0\hfill \\ \frac{\sqrt{3}}{\sqrt{4\pi nk}}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{4n{k}^2}}t,\hfill & u>0\hfill \end{array}\right..\end{array}$$

(40)

The proof of Lemma 6 is based on the same principle as Lemma 4. The proof is given as follows, which is used to support the proof of Theorem 7.

Proof. 

According to the definition of ${\eta }_t,t=0,1,\dots$,

(1)

If $f\left({\overrightarrow{y}}^t\right)\le f\left({\overrightarrow{x}}^t\right),{\eta }_t=0$.

(2)

If $f\left({\overrightarrow{y}}^t\right)>f\left({\overrightarrow{x}}^t\right),{\eta }_t=k[(\Delta x_1^t+\Delta x_2^t+\dots +\Delta x_n^t)+(z_1^t+z_2^t+\dots +z_n^t)]$

Since $z_i\sim U(-1,1)$, $\Delta x_i\sim U(-1,1)$, and $z_1,\dots ,z_n$ are independent of each other, $\Delta x_1,\Delta x_2,\dots ,\Delta x_n$ are also independent of each other. $f\left({\overrightarrow{z}}^t\right)$ obeys $N(0,\frac{2}{3}n{k}^2)$ according to the Lindeberg–Levy center limit theorem.

Hence, the ${\eta }_t$ distribution function $F\left(u\right)=P({\eta }_t\le u)$ is

(1)

If $u<0$, $F\left(u\right)=0$.

(2)

If $u=0$, $F\left(u\right)=P({\eta }_t\le u)=P({\eta }_t=0)=\frac{1}{2}$.

(3)

If $u>0$, $F\left(u\right)=P({\eta }_t\le u)=\frac{\sqrt{3}}{\sqrt{4\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{4n{k}^2}}t$.

□

Theorem 7. 

If the mutation operator of BSO-II $\overrightarrow{z}$ obeys $U(-1,1)$, the upper bound on the expected first hitting time to reach the target fitness value $na$ is derived as

$$\begin{array}{c}\hfill E\left(T_{\epsilon }\right|X_0)\le 1+\frac{5\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}.\end{array}$$

(41)

The proof of this theorem is based on the same principle as Theorem 5. The proof is given as follows.

Proof. 

According to Lemma 6, we have

$$\begin{array}{c}\hfill E(X_t-X_{t+1}|X_t,A)=E\left({\eta }_t\right|X_t,A)={\int }_0^{+\infty }u\left(\frac{\sqrt{3}}{\sqrt{4\pi n}k}{\int }_{-\infty }^u{e}^{-\frac{3t^2}{4n{k}^2}}t\right)=\frac{k\sqrt{n}}{\sqrt{3\pi }}.\end{array}$$

(42)

Suppose that the probability $Pb=0.2$; according to Theorem 1, the following conclusions can be drawn:

$$\begin{array}{cc}& E(X_t-X_{t+1}|X_t)\hfill \\ & =P\left(\overline{A}\right)E(X_t-X_{t+1}|X_t,\overline{A})+P\left(A\right)E(X_t-X_{t+1}|X_t,A)\hfill \\ & =(1-0.2)\times E(X_t-X_{t+1}|X_t,\overline{A})+0.2\times E(X_t-X_{t+1}|X_t,A)\hfill \\ & =0.8\times \frac{k\sqrt{n}}{\sqrt{6\pi }}+0.2\times \frac{k\sqrt{n}}{\sqrt{3\pi }}=\frac{4k\sqrt{n}+k\sqrt{2n}}{5\sqrt{6\pi }},\hfill \end{array}$$

(43)

The algorithm starts from the origin at initialization, ${\overrightarrow{x}}^0=(0,0,\dots ,0)$, i.e., $X_t=na$. According to Theorem 1, the upper bound on the expected first hitting time is derived as

$$\begin{array}{cc}\hfill & E\left(T_{\epsilon }\right|X_0)\le 1+{\int }_{\epsilon }^{na}\frac{5\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}x=1+\frac{5\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}. \square \hfill \end{array}$$

(44)

Theorem 7 indicates that if the mutation operator of BSO-II $\overrightarrow{z}$ obeys $U(-1,1)$, its computational time complexity for the equal coefficient linear function is $E\left(T_{\epsilon }\right|X_0)=\mathrm{O}\left(\sqrt{n}\right)$.

3.3. Summary

Overall, we summarize the theoretical analysis results of the running-time analysis of the single individual BSO in solving the n-dimensional equal coefficient linear function in six different situations. The theoretical analysis results are shown in

Table 1. Analysis of the running time of BSO in six different situations.

Algorithm	Mutation Operator $\overrightarrow{\mathit{z}}$	Display Expression for $\mathit{E}\left({\mathit{T}}_{\mathit{\epsilon }}\right|{\mathit{X}}_0)$	Time Complexity $\mathbf{T}\left(\mathit{n}\right)$
BSO-I	$N(0,1)$	$E\left(T_{\epsilon }\right|X_0)\le 1+\sqrt{2\pi n}\frac{a}{k}-\sqrt{\frac{2\pi }{n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	$E\left(T_{\epsilon }\right|X_0)\le 1+2\sqrt{6\pi n}\frac{a}{k}-\frac{2\sqrt{6\pi }}{\sqrt{n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$
	$U(-1,1)$	$E\left(T_{\epsilon }\right|X_0)\le 1+\sqrt{6\pi n}\frac{a}{k}-\sqrt{\frac{6\pi }{n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$
BSO-II	$N(0,1)$	$E\left(T_{\epsilon }\right|X_0)\le 1+\frac{5\sqrt{2\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{2\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	$E\left(T_{\epsilon }\right|X_0)\le 1+\frac{10\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{10\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$
	$U(-1,1)$	$E\left(T_{\epsilon }\right|X_0)\le 1+\frac{5\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	$\mathrm{O}\left(\sqrt{n}\right)$

. The time complexity of the single individual BSO in these six cases is $\mathrm{O}\left(\sqrt{n}\right)$. However, the coefficients in the display expressions are different.

Table 1 shows the correlation between the expected first hitting time, the dimension n, the slope k, and the parameter a. The upper bounds on the expected first hitting time of BSO-II are lower than those of BSO-I. This means that the performance of BSO-II is better than that of BSO-I in solving the equal coefficient linear function. The disrupting operation in BSO helps to reduce the running time of the algorithm. Moreover, the upper bounds on the expected first hitting time of the algorithm using the standard normal distribution mutation operator are lower than those of the algorithms with the uniform distribution mutation operator. In addition, the upper bounds on the expected first hitting time of the algorithm using the $U\left(-\frac{1}{2},\frac{1}{2}\right)$ mutation operator are approximately two times higher than those of the algorithm using the $U(-1,1)$ mutation operator.

4. Experimental Results

In Section 3, we obtain the theoretical analysis results of the expected first hitting time of single individual BSOs through the average gain model. To verify the correctness of the analysis results, numerical experiments are presented in this section.

As the number of samples increases, the arithmetic mean will gradually approach true mathematical expectations based on the Wiener–Khinchine theorem of large numbers [58]. The Wiener–Khinchine theorem of large numbers is introduced as follows.

Suppose that $X_1,X_2,\dots$ is a sequence of independent and identically distributed random variables with $E\left(X_i\right)=\mu$; for any $\epsilon >0$, the following equation will hold.

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }P\left(\left|\frac{1}{n}\sum _{i=1}^n{X}_i-\mu \right|<\epsilon \right)=1.\end{array}$$

(45)

The Wiener–Khinchine theorem of large numbers indicates that if the number of samples is sufficiently large, the mathematical expectations are approximately equal to the mean of samples $X_1,X_2,\dots ,X_n$. Therefore, we use the arithmetic mean of the first hitting time of multiple experiments to estimate the actual expected first hitting time.

In this section, the parameters of the proposed approach are set as follows. The fixing error is $\epsilon =1\times {10}^{-8}$, the initial individual is $x^0=(x_1^0,x_2^0,\dots ,x_n^0)=(0,0,\dots ,0)$, the slope is $k=1$, and the target fitness parameter is $a=10$. The problem dimension n is set from 10 to 280. BSO-I and BSO-II are conducted on the n-dimensional equal coefficient linear function for 300 runs for each dimension. The termination criterion for each experiment is that the error of the optimal solution should be below a predefined threshold $\epsilon$.

Table 2. Comparison of the estimation of the expected first hitting time and the theoretical upper bounds.

Algorithm	$\overrightarrow{\mathit{z}}$	n	10	40	70	100	130	160	190	220	250	280
BSO-I	$N(0,1)$	$1+\sqrt{2\pi n}\frac{a}{k}-\sqrt{\frac{2\pi }{n}}\frac{\epsilon }{k}$	80.27	159.53	210.72	251.66	286.80	318.07	346.51	372.79	397.33	420.44
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	79.65	159.57	212.51	252.09	287.14	316.84	345.61	373.66	398.08	418.75
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	$1+2\sqrt{6\pi n}\frac{a}{k}-\frac{2\sqrt{6\pi }}{\sqrt{n}}\frac{\epsilon }{k}$	275.59	550.17	727.49	869.32	991.04	1099.35	1197.90	1288.93	1373.94	1453.98
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	275.81	548.63	727.97	868.67	990.57	1102.96	1198.00	1288.83	1373.06	1455.95
	$U(-1,1)$	$1+\sqrt{6\pi n}\frac{a}{k}-\sqrt{\frac{6\pi }{n}}\frac{\epsilon }{k}$	138.29	275.59	364.24	435.16	496.02	550.17	599.45	644.96	687.47	727.49
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	137.27	275.45	363.95	435.81	498.34	553.30	598.41	641.36	685.46	723.91
BSO-II	$N(0,1)$	$1+\frac{5\sqrt{2\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{2\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	74.20	147.40	194.68	232.49	264.93	293.81	320.08	344.35	367.01	388.35
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	72.97	145.55	193.03	232.77	262.05	294.26	318.51	347.10	365.51	387.87
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	$1+\frac{10\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{10\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	254.58	508.16	671.91	802.89	915.30	1015.32	1106.33	1190.40	1268.90	1342.82
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	251.80	505.49	674.53	804.58	915.61	1014.22	1103.21	1190.82	1271.88	1342.12
	$U(-1,1)$	$1+\frac{5\sqrt{6\pi n}}{4+\sqrt{2}}\frac{a}{k}-\frac{5\sqrt{6\pi }}{4\sqrt{n}+\sqrt{2n}}\frac{\epsilon }{k}$	127.79	254.58	336.45	401.95	458.15	508.16	553.67	595.70	634.95	671.91
		$\widehat{E\left(T_{\epsilon }\right|X_0)}$	128.10	253.61	334.30	400.20	458.07	507.69	554.34	594.77	634.45	669.75

The points larger than the theoretical upper bounds are highlighted in boldface.

shows the numerical results of the practical expected first hitting time $\widehat{E\left(T_{\epsilon }\right|X_0)}$ and the theoretical time upper bound, where $\widehat{E\left(T_{\epsilon }\right|X_0)}=\frac{{\sum }_{i=1}^{300}{T}_{\epsilon i}}{300}$, and $T_{\epsilon i}$ is the first hitting time of the $\epsilon$-approximation solution at the i-th run.

As shown in Table 2, the experimental results strongly fit the theoretical upper bounds, indicating that the error between the theoretical upper bounds and the actual value is within $\epsilon$. The points larger than the theoretical upper bounds are highlighted in boldface. The arithmetic mean of multiple experiments is used to estimate the expected first hitting time. In the real case, only the arithmetic mean of 300 experiments is used to estimate the expected first hitting time, which allows a certain statistical error. According to the central limit theorem, the results obtained from 300 independent experiments follow a normal distribution. The null hypothesis $H_0$ means that the mean value of the expected first hitting time in 300 experiments is less than or equal to the corresponding theoretical upper bounds. The corresponding significance level $\alpha$ is $0.05$ with the T testing. Moreover, as shown in Figure 1, the actual expected running time is followed with the estimated result based on our proof. All the detailed results are shown in

Table 3. Statistical results of hypothesis testing.

Algorithm	$\overrightarrow{\mathit{z}}$	n	10	40	70	100	130	160	190	220	250	280
BSO-I	$N(0,1)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.80	0.48	0.06	0.38	0.41	0.81	0.72	0.31	0.34	0.83
		$ci$	78.41	157.86	210.57	249.84	284.76	314.52	343.03	370.82	395.13	415.87
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.44	0.79	0.42	0.60	0.57	0.10	0.49	0.51	0.61	0.27
		$ci$	273.43	545.49	724.19	864.51	986.22	1098.22	1193.50	1283.96	1367.75	1450.62
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf
	$U(-1,1)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.85	0.54	0.57	0.36	0.10	0.07	0.69	0.95	0.81	0.94
		$ci$	135.64	273.05	361.29	432.88	495.38	549.76	595.05	637.79	681.73	720.20
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf
BSO-II	$N(0,1)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.95	0.96	0.93	0.42	0.99	0.38	0.83	0.06	0.81	0.61
		$ci$	71.71	143.86	191.16	230.52	259.89	291.83	315.84	344.26	362.73	385.06
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf
	$U\left(-\frac{1}{2},\frac{1}{2}\right)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.98	0.92	0.12	0.23	0.45	0.65	0.86	0.44	0.18	0.59
		$ci$	249.47	502.38	670.91	800.79	911.60	1009.53	1098.54	1185.87	1266.44	1337.20
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf
	$U(-1,1)$	h	0	0	0	0	0	0	0	0	0	0
		p	0.37	0.77	0.92	0.84	0.52	0.60	0.37	0.66	0.59	0.83
		$ci$	126.53	251.47	331.78	397.34	455.24	504.80	551.07	591.02	630.94	665.97
			Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf	Inf

.

Table 3 provides the numerical results, where h represents the hypothetical result, p represents the p-value of the test, and $ci$ is the confidence interval. As shown in the T testing of Table 3, $h=0$ and $p>\alpha$. The null hypothesis $H_0$ is accepted at the significance level $\alpha =0.05$. Therefore, the analytic expression of the running time of BSO obtained based on the average gain model can characterize the actual upper bounds of the running time of BSO in these six BSO variants.

5. Conclusions

The running time of six BSO variants for the equal coefficient linear function is analyzed in this paper based on the average gain model. The additivity of the normal distribution and the Lindeberg–Levy center limit theorem are applied to deal with the superposition of the normal distribution mutation operator and the uniform distribution mutation operator, respectively. Furthermore, the full expectation formula is utilized to deal with the problem of individual replacement with a certain probability in the disrupting operator. This paper also concludes the upper bounds on the expected first hitting time of the single individual BSO in equal coefficient linear functions.

The analysis results show that the time complexity of BSO-I and BSO-II is $\mathrm{O}\left(\sqrt{n}\right)$ in the equal coefficient linear function. However, their coefficients are different. In the linear function with equal coefficients, the upper bound of the expected first hit time of BSO-II is smaller than that of BSO-I. In addition, the single individual BSO using the standard normally distributed mutation operator expects a lower upper bound on the first hit time than the corresponding algorithm using the uniformly distributed mutation operator. The upper bounds on the expected first hitting time of single individual BSOs with the $U(-\frac{1}{2},\frac{1}{2})$ mutation operator are approximately twice those of BSOs with the $U(-1,1)$ mutation operator.

In our future work, we will analyze the running time of the population-based BSO in the equal coefficient linear function. The running time of population-based BSOs in practical optimization problems is also an important topic. Moreover, it is crucial to extend our research to practical optimization problems that are encountered in real-world applications with complex constraints, non-linear relationships, or high-dimensional spaces.