Studying the Impact of Initialization for Population-Based Algorithms with Low-Discrepancy Sequences

Abstract

To solve different kinds of optimization challenges, meta-heuristic algorithms have been extensively used. Population initialization plays a prominent role in meta-heuristic algorithms for the problem of optimization. These algorithms can affect convergence to identify a robust optimum solution. To investigate the effectiveness of diversity, many scholars have a focus on the reliability and quality of meta-heuristic algorithms for enhancement. To initialize the population in the search space, this dissertation proposes three new low discrepancy sequences for population initialization instead of uniform distribution called the WELL sequence, Knuth sequence, and Torus sequence. This paper also introduces a detailed survey of the different initialization methods of PSO and DE based on quasi-random sequence families such as the Sobol sequence, Halton sequence, and uniform random distribution. For well-known benchmark test problems and learning of artificial neural network, the proposed methods for PSO (TO-PSO, KN-PSO, and WE-PSO), BA (BA-TO, BA-WE, and BA-KN), and DE (DE-TO, DE-WE, and DE-KN) have been evaluated. The synthesis of our strategies demonstrates promising success over uniform random numbers using low discrepancy sequences. The experimental findings indicate that the initialization based on low discrepancy sequences is exceptionally stronger than the uniform random number. Furthermore, our work outlines the profound effects on convergence and heterogeneity of the proposed methodology. It is expected that a comparative simulation survey of the low discrepancy sequence would be beneficial for the investigator to analyze the meta-heuristic algorithms in detail.

1. Introduction

The term ‘optimization’ refers to the best solution for a problem with minimum cost in aspect of memory, time, and resources. Sometimes processing time is fast but it may be using a lot of memory while sometimes the processing speed and memory both work fine but the accuracy may get affected. Optimization targets the best solution of any problem [1]. The solution is considered to be the best solution if it is satisfactory in terms of processing speed, resource utilization, and accuracy of the result [2]. Optimization algorithms are utilized to determine the problems of local and global search. A typical target behind the utilization of these optimization algorithms is to discover the optima for contribution as indicated by known inputs model that describes the problem which is to be solved [3]. Optimization algorithms have turned out to be the most generally adopted algorithms that are operational in all application areas, like, enterprises, sports, medical, agriculture, and finance [4].

Evolutionary algorithms (EAs) have been introduced and strongly employed in the different field of science and engineering tracks [5]. EAs have been broadly utilized to determine optimization problems of maximization and minimization to find best optimal value. Rather than ordinary strategies dependent on mathematical programming or formal rationales, EAs are observed to be all the more dominant and adaptable [6]. Despite this, in solving the complex optimization problems, EAs faces the problem of local optima in which the computation to be caught in nearby local optima and a resist the convergence speed, for example, complex nonlinear problems [7]. To enhance the performance of EAs and to avoid premature convergence, there is a need to develop new variants of evolutionary algorithms. Furthermore, dependent on genetic evolution procedures, several researchers have been given the task to improve existing EAs or developing new EAs. The most generally used EAs algorithms involve the genetic algorithm (GA) [8] and differential evolution [9]. DE is recognized as a simplistic yet strong evolutionary algorithm that has been utilized to tackle different hard optimization problems in various science and engineering disciplines [10]. As another component of an EA, the DE algorithm yields a comparative structure [11] by EA, which incorporates three essential basic genetic operators, i.e., mutation, crossover, and selection. These genetic operators contribute major roles to the performance of the DE [12].

The intelligent attitude of non-intelligent species like ants (going for searching food) or birds (during flying in flocks) or fish in school is termed as swarm intelligence (SI). SI inspired by the experience of ants, bees, birds, and fishes to fulfill their goals as a swarm [13]. If every member of a swarm in SI works individually without social interaction, it will become complicated to achieve their goals due to their individuality which results in lack of intelligence. However, when they cooperate with each other, their social interaction is improved, and they also interact with the environment which makes it easier for them to accomplish difficult tasks [14]. SI-based algorithms are ant colony optimization (ACO), bat algorithm (BA) [15,16,17,18], and particle swarm optimization (PSO) [19]. PSO [20] has pulled in much consideration because of its simplicity of execution and strong search abilities. It is impelled by the social foraging fashions of fish and birds that seek for food in the form of groups.

The major issue with these meta-heuristic algorithms while applying those complex numerical optimization problems is premature convergence [21]. Regardless of the nature of the non-linear problem, this issue is confronted while running a heuristic algorithm, for example, meta-heuristic algorithms like PSO [22] and DE get stuck in the local optima after little number of epochs. The population convergence fails to produce a new population of the swarm due to inappropriate animalization strategies to explore the whole search space [23]. In the field of evolutionary computing, the performance of meta-heuristics algorithms is affected by the generation of random numbers while initializing the population into the multidimensional search space [24]. The meta-heuristics algorithm tends to reach the optimum value while solving the problems in low dimensional search space. However, the performance is supposed to be insignificant when the dimensionality of the problem is high, and this causes the particles to stick in the local optima [25]. Meta-heuristics population-based algorithm initialization can be performed by using chaotic initialization [26,27,28], opposition-based initialization, and quasi-random sequences. This paper presents the impact of quasi-random sequences for the initialization of the population of the meta-heuristics algorithm.

In accordance with the optimization problem, population initialization plays a significant role in meta-heuristic algorithms. These algorithms can influence diversity, convergence, and also help to find an efficiently optimal solution. Particularly, recognizing the importance of diversity, several researchers have worked on performance for the improvement of meta-heuristic algorithms. In order to improve the convergence, rather applying the random distribution for initialization, quasi-random sequences are more useful to initialize the population [29].

Quasi-random sequences suffer from many issues while solving the problems of different dimensionality in real world [30]. Some of the sequences of quasi-random sequences give better results on large dimensions and vice versa [31,32]. Our objective is to find the most suitable quasi-random sequence for meta-heuristic algorithms, which gives superior results without considering the dimensionality problem.

Considering this fact, we have proposed three novel pseudo-random initialization strategies called WELL sequence, Knuth sequence [33], and Torus sequence to initialize the population in the search space. We initialized PSO, BA, and DE algorithm with these proposed pseudo-random strategies (WELL sequence, Knuth sequence, and Torus sequence). In our first contribution, we have compared the novel PSO technique with the simple random distribution [34] and family of low discrepancy sequences [35] on several unimodal and multi modals complex benchmark functions and training of the artificial neural network [36]. The experimental results have shown that PSO with Knuth-based initialization (KN-PSO) outperforms the other traditional PSO, PSO with Sobol-based initialization (SO-PSO), PSO with Halton-based initialization (H-PSO), PSO with Torus-based initialization (TO-PSO), and PSO with WELL-based initialization (WE-PSO) [37]. Similarly, in the second contribution, DE is initialized by these proposed pseudo-random strategies (WELL sequence, Knuth sequence, and Torus sequence) for function optimization and training of the neural network. The simulation results depict that DE with Halton-based initialization (DE-H) is superior to the standard DE, DE with Sobol-based initialization (DE-SO), DE with Knuth-based initialization (DE-KN), DE with Torus-based initialization (DE-TO), and DE with WELL-based initialization (DE-WE). BA is initialized by these proposed pseudo-random strategies (WELL sequence, Knuth sequence, and Torus sequence) for function optimization and training of the neural network. The simulation results depict that BA with WELL-based initialization (BA-WE) is superior to the standard BA, BA with Sobol-based initialization (BA-SO), BA with Halton-based initialization (BA-HA), BA with Torus-based initialization (BA-TO), and BA with Knuth-based initialization (KN-BA) [38,39,40].

The rest of the paper is organized as: Section 2 overviews the previous work. In Section 3, the different algorithm methodology is represented with six initialization strategies. Section 4 contains the experimental setup. In Section 5, the results and discussion about the implementation and comparison of algorithms using initialization techniques on sixteen benchmark tests functions are presented. Section 6 presents the comparison of PSO, BA, and DE regarding data classification. Lastly, Section 7 concludes the paper.

2. Previous Work

Many research studies proposed different variants based on initialization techniques and we have discussed some of them in detail in this chapter. Initialization of the swarm in a good way helps the PSO to search more efficiently [41]. In this work, the initialization of swarm with nonlinear simplex method (NSM) has been done. NSM requires only function evaluations without any derivatives for computation. NSM starts with initial simplex and produces sequence of steps moving the highest function value vertex in opposite direction of the lowest one. They initialized the particle with the initial simplex in the D dimensional search areas, where D + 1 vertices of the simplex are D + 1 particle of the swarm and the MSN method is applied for N-D + 1 steps for N size swarm. In this way, each particle in the swarm has the information of the region. In the last, they compared their results with simple PSO and found significant improvement. This variant was introduced by Mark Richards and Dan Ventura in 2004.

In their work [42], they proposed to use centroidal Voronoi tessellations for initializing the swarm. Voronoi tessellations is a technique of partitioning any region into compartments, each partition contains group of generators. Each partition is associated with one generator and it consists of all the particles closer to that generator. In the same way, the generators are selected for the initial position of the particle. In this way, they initialized the particle swarm optimization algorithm. They compared it with basic SPO on many benchmark functions and found improved performance in high-dimensional spaces.

Halton sampling was introduced by Nguyen Xuan Hoai, Nguyen Quang Uy, and R.I. McKay in 2007 [43]. Halton sequence is a low discrepancy deterministic sequence used to generate point in space. Halton sequence is not a fully random. To randomize, X.Wang and F.J. Hickernell proposed a new function called randomize Haltom sequence by using von Neuman–Kakutani transformation. They used this sequence to initialize the global best of the PSO. They performed a test on various benchmark functions and compared the result with the PSO and initialized with uniform random numbers. They found better performance especially for complex and smaller populations.

VC−PSO was introduced by Millie Pant et al. in 2008. They used Vandor Corput sequence for initializing the swarm for large dimensions’ search [44] (Pant, Thangaraj, Grosan, and Abraham, 2008). The Vandor Corput and Sobol sequence generates semi random number which is more suitable for computational purposes. They tested the new variant with different benchmark functions and compared the result with BPSO and SO-PSO and found significant improvement especially for large search space dimension. The main purpose of this variant is to see the performance of large search space problems. That is why they used search space with different dimensions ranging [−5.12, 5.12] to [−1000, 1000]. The performance is showing prominent when the dimension increases to [−100, 100].

SMPO was introduced by Millie Pant et al. in 2008. They used quasi-random Sobol sequence to initialize the particles instead of normal random numbers [45]. They used a new operator called systematic mutation operator which is used to improve the performance of the PSO. Instead of using the normal random number, the new operator uses the quasi-random Sobol sequence to initialize the swarm as the QRS is less random as compared to pseudorandom sequences which is helpful for computational methods. They proposed two variants, SM-PSO1 and SM-PSO2. The main difference between the two versions is that in MSPSO1, the best particle is mutated while in MS-PSO2, the worst particle is mutated. They found better results comparing with BPSO and other variants.

This work is done by Jiyong et al. 2011 [46]. In this paper, researchers proposed a new method of initialization. In their work, they added the functionality to detect automatically when the particle is prematurely converged and initializes the swarm. They also added functionality to redesign the inertia weight to balance the searching ability globally and locally. They named it IAWPSO.

This variant was proposed by P. Murugan in 2012 and applied on the transmission expansion problem to decide installation of new circuits in an increasing usage of electricity and found this variant fruitful [47]. In this work, he used the new initialization technique called population monitored for complementary magnitudes initialization. In initialization, he used decision variables. All particles are initialized with and integer within the limit of the upper and lower values of the decision variable in such a way that each particle should be unique. The initial population is created in a way that each particle can have the ability of the possible solution and they are unique. Almost 50% of the particles are opposite to another 50% considering the lower and upper limit of decision variable. The important thing in this initialization is to maintain uniqueness and diversity among the particles of the swarm generated initially.

SISP SO was introduce by Liang Yin, Xiao−Min Hu, and Jun Zhang in 2013 [48]. In this paper, the authors introduced a new initialization technique named space-based initialization strategy. In this work, they broke down each dimension of the search area into two segments, S1i and S2i. The borders of the areas are [li,(li + ui)/2] and [(li + ui)/2, ui], with each segment linked with a probability and initialized with 0.5. They applied SIS-PSO on thirteen functions and compared results with GPSO and CLPSO and found significant improvement.

This variant was introduced by Moaath Shatnawi, Mohammad Faidzul Nasrudin, and Shahnorbanun Sahran in 2017 [49]. In this work, they introduced a new variant of PSO called polar PSO. They explained that most of the distortion was occurring due to polar particles. Hence, they introduced a new method for reinitialization of the polar particles by redefining the distance based on the dimensionality of the point. By using this method, it removed the distortion occurring during the computation. He compared the results with BPSO and found some improvement.

This variant was proposed by Laxmi et al. in 2017 [50]. In this work, they used the Nawaz–Enscore–Ham heuristic technique to initialize the swarm. This variant is named PHPSO. The sequence generated by NEH jobs is placed in ascending order of the sums of their total flow time. To construct a job sequence, it depends on its initial order. The minimum TFT sequence is the current sequence for the upcoming iteration among all the sequences. The resulting population generated by the NEH method is used to initialize the population of PSO. They applied this algorithm for the no-wait flow shop scheduling problem. They compared the result with DPSO and HPSO and found the comparatively better result.

A new variant of PSO combining with stochastic gradient decent was proposed by Hayder M. Albeahdili, Tony Han, and Naz E. Islam in 2015 and named it the PSO–SGD algorithm for training the convolution neural network [51]. The proposed technique was divided into two phases. PSO was used to train and initialize the CNN parameters in the first phase. When it showed slow progress of the PSO for few iterations, SGD was used in the second phase. Additionally, they used PSO combined with the genetic algorithm (GA) which helped the particle for simulation and overcame the slowness of SGD. They applied the new algorithm on different benchmark datasets and performed well for three different datasets. The proposed technique avoided the occurrence of local optimum and premature saturation as it was in the known problem by using any single algorithm.

The authors in [52] examined the impact of initiating the initial population by excluding traditional techniques like random numbers or quasi-random numbers. The authors applied the non-linear simplex method for generating the initial population of DE, where the proposed algorithm was termed NSD. The working of the proposed algorithm is measured with twenty benchmark functions and compared with the standard DE and opposition-based DE (ODE) algorithm. Numerical results illustrate that the proposed technique enhances the convergence rate.

To tackle the thresholding problem of the image, an enhanced variant of the standard DE algorithm with a local search (termed as LED) and low discrepancy sequences is introduced [53]. Experimental results conclude that the performance of the introduced algorithm is superior for finding the optimum threshold.

For the steelmaking continuous (SCC) problem, in [54], the authors presented a novel enhanced technique of DE based on the two-step procedure for producing an initial population, as well as, a novel mutation approach. Furthermore, an incremental methodology for generating the initial population was also incorporated in DE to handle dynamic events. Computational experiments conducted with the presented approach show the effectiveness of the presented approach than others. Additionally, as per concern, in the application area, the authors utilized BA for the antenna optimization problem in [55], moreover, pan evaporation was estimated by using BA [56]. Beside this, in [57], the authors applied a new variant of DE for path-planning of mobile robots.

According to the above-mentioned studies, we conclude that the efficiency of meta-heuristic algorithms is affected by using the random number for the initialization of the population. Due to this reason, various articles used the quasi-random number sequences for the population’s initialization in meta-heuristic algorithms. However, the majority of the researchers used limited quasi-random sequences for initializing the population and did not perform any comparative analysis for their effect on the initialization of population algorithms. Similarly, Knuth, Well, and Torus sequences from quasi-random sequences are still not proposed in DE and BA for the initialization of the population. After analyzing all the literature, we found the above-mentioned gaps and try to fill it.

3. Methodology

The most important step in any meta-heuristic algorithm is to initialize its population properly. If the initialization is not proper, then it may go to search in unnecessary areas and may fail to search the optimum solution. Proper initialization is very important for any algorithm for its performance. The objective of this paper is to figure out the purity of quasi-random sequences. PSO is random in nature, so it does not have a specific pattern to ensure the global optimum point. Therefore, by taking the advantage of this randomness and considering this fact, we proposed three novel quasi-random initialization strategies called WELL sequence, Knuth sequence, and Torus sequence to initialize the population in the search space. We initialized the PSO, DE, and BA algorithm with these proposed pseudo-random strategies (WELL sequence, Knuth sequence, and Torus sequence). We have compared the novel techniques with the simple random distribution and family of low discrepancy sequences on several unimodal and multi modals complex benchmark functions and training of the artificial neural network. A brief description of quasi sequences approaches and proposed algorithms using WELL sequence, Knuth sequence, and Torus sequence for PSO, DE, and BA are discussed in below.

It has been stated above; the goal of this study is to analyze the purity of low discrepancy sequences. Therefore, we compare the proposed algorithm based on WELL, Torus, and Knuth distribution with the simple PSO, BA, and DE based on pseudo-random uniform distribution and other low discrepancy distributions based on the Sobol sequence and Halton sequence.

3.1. Low Discrepancy Sequences

Discrepancy is the measure of how uniform the numbers are distributed. Consider the set of points P = (x₁, x₂, ……, x_n) be set of n points in s dimensions in [0, 1)s. For a vector y = (y₁, y₂, …, y_s) [0, 1)s, let J be:

$$j={\displaystyle \prod _{i=1}^s{[0,y_i)}^s}$$

(1)

Although there exit other measures of discrepancy, the star discrepancy is commonly used. A low value of discrepancy means more uniform distribution in space.

3.1.1. Uniform Random Numbers

Random numbers are generated through a pseudo random sequence by pursuing uniform-distribution [44], which can be typified using the probability-density function of constant uniform-distribution. Given below is the probability-density function in (2) as:

$$f\left(t\right)=\{\begin{array}{c}\frac{1}{p-q} for p<t<q\\ 0 for t<p or t>q\end{array}$$

(2)

where u and v describe the features that fit the maximum likelihood. At the edge of u and v, the cost of f(w) is unproductive because of 0 impacts on the integrals of f(w)dw at any range. The likelihood function of assessment helps to simulate the assessment of features of maximum likelihood, likelihood function of assessment is given below in (3) as:

$$l\left(p,q|t\right)=n\log \left(q-p\right)$$

(3)

3.1.2. Sobol

The Sobol sequence is firstly introduced by a Russian mathematician, Sobol [45]. Then, reconstruct the coordinates. Coordinates have liner recurrence relation for each dimension. Let the non-negative instance s containing a binary expression in (4) where s:

$$a=a_1{2}^0+a_2{2}^1+a_3{2}^2+\dots +a_z{2}^{z-1}$$

(4)

Then, the ith instance of the D dimension can be generated using the (5):

$$x_i^D=i_1{v}_1^D+i_2{v}_2^D+\dots +i_z{v}_z^D$$

(5)

$\mathrm{where} v_1^D$ represents the binary function which is followed by the D dimension and ith direction instance and these direction instances can be generated using the (6):

$$V_k^D=c_1{v}_{k-1}^D+c_2{v}_{k-2}^D+\dots +c_z{v}_{z-1}^D+\left(\frac{v_{i-z}^D}{2^z}\right)$$

(6)

where c_q is a polynomial coefficient where i > q.

3.1.3. Halton

Halton sequences was carried out by J. Halton [43] and can be considered as the enhanced version of Van Dar Corput (Gentle, 2006). Halton sequence constructs random points pattern by using the base as coprime. Halton sequences: The pseudo code to generate Haltom sequences is as follow:

Halton Sequences:

• //input: Initial index = s and base = coprime
• //output: instances = h
• Set the interval over

• min = 0
• max = 1

For each iteration k1, k2, k3…kn:do

• For each particle p1, p2, p3, …, pn

–

max = max/coprime

–

min = min + max ∗ smodb

–

s = s/b

Return h

3.1.4. WELL

WELL equi-distributed long-period linear (WELL) sequence was proposed in [58]. Initially, it was carried out as updated version of the Mersenne twister algorithm. The algorithm for generating the WELL distribution is given as:

WELL Sequences

• WELL ():
• $t_0=\left(m_x\&v_{k,r-1}\right)+\left(m_x\&v_{k,r-2}\right)$
• $t_1=\left(A_0{v}_{k,0}\right)+\left(A_1{v}_{k,m_1}\right)$
• $t_2=\left(A_2{v}_{k,m_2}\right)+\left(A_3{v}_{k,m_3}\right)$
• $t_3=t_2+t_1$
• $t_4=t_0{A}_4+t_1{A}_5+t_2{A}_6+t_3{A}_7$
• $v_{k+1,r-1}=v_{k,r-2 }\& m_x$
• $for i-\to r-2\dots \mathrm{..2} do v_{k+1,i=}{v}_{k,i-1}$
• $v_{k+1,1=}{t}_3$
• $v_{k+1,0=}{t}_4$
• $Return y_{k=}{v}_{k,0}$

The algorithm stated above describes the general recurrence for the WELL distribution. The description for the algorithm is as: x and r two integers with the interval of r > 0 and 0 < x < k and k = r ∗ w − x, where w is the weight factor of distribution. A₀ to A₇ represent the binary matrix of size r ∗ w having r bit block. m_x describes the bit mask that holds the first w − x bits. t₀ to t₇ are temporary vector variables.

3.1.5. Knuth

As discussed above, inbuilt library function is used, Knuth(x(min,)xmax) to generate Knuth sequences random points. Following is the pseudo code to generate Knuth sequences. Knuth sequence is designed and was proposed by the authors in [33]. Following is the pseudo code to generate Knuth sequences.

• To shuffle an array a of n elements (indices 0…n − 1):
• F orifrom0ton − 2do
• jrandomintegersuchthati ≤ j < n
• exchangea[i]anda[j]

3.1.6. Torus

Torus is a geometric term and was firstly used by the authors in [59] to generate a Torus mesh for the geometric coordinate system. In game development, Torus mesh is commonly used and can be generated using the left hand coordinate system or right hand coordinate system. The shape for the Torus at 1D, 2D, and 3D are circle, donut, and 2D rectangle, respectively. The Torus in 3D can be represented by the following (7)–(9):

$$a\left(\theta ,\delta \right)=\left(D+r\cos \theta \right)\cos \delta ,$$

(7)

$$b\left(\theta ,\delta \right)=\left(D+r\cos \theta \right)\sin \delta ,$$

(8)

$$c\left(\theta ,\delta \right)=r\sin \delta ,$$

(9)

where the angles of circles are θ, δ and D is the distance from tube center to Torus center, r denotes to the radius of 6circle. Inspired by this mesh having Torus, low discrepancy sequences have been generated that were initialized with the prime series as Torus effect. In (10), the mathematical notation for Torus series is shown:

$${\alpha }_k=\left(f\left(k\sqrt{s_1}\right),\dots ,f\left(k\sqrt{s_d}\right)\right),.$$

(10)

where s₁ denotes the series of ith prime number and f is a fraction which can be calculated by f = a − floor(a). Due to the prime constraints, the dimension for the Torus is limited to the 100,000 only if we use parameter prime in Torus function. For more than 100,000 dimensions, the number must be provided through manual way.

In Figure 1, uniform random distribution; Figure 2, Sobol distribution; Figure 3, Halton distribution; Figure 4, WELL distribution; Figure 5, Knuth distribution; Figure 6, Torus distribution are presented by bubble plot, in which y-axis is representing the random values and the x-axis is showing the relevant index of the concerned point in the table. We made our first major contribution to this study first by introducing three novel methods of initialization of population: WE-PSO, KN-PSO, and TO-PSO. The algorithm shows the flow chart of the proposed distribution-based PSO initialization.

The Algorithm 1 shows the flow chart of the proposed distribution-based PSO initialization.

Algorithm 1 Proposed Pseudo Code of PSO Using Novel Method of Initialization

Initialize the swarm Set epoch count $I=0$, population size $N_z$, Dimension of the problem $D_z$, $w_{max}$ and $w_{min}$ For each particle $P_z$. Initialize $x_z,$ as $x_z= \mathrm{WELL}, \mathrm{Knuth}, \mathrm{Torus} \left(xmin,xmax\right)$ Initialize the Particle velocity as, $v_z=Rand(xmin,xmax$. ) Compute the fitness score $f_z$ Set global best position $g_z^{best}$ as $max(f_1,f_2,f_3\dots ..f_z$) where $f_z\in globally optimal fitness$ Set local best position $p_z^{best}$ as $max(f_1,f_2,f_3\dots ..f_z$) where $f_z\in locally optimal fitness$ Compare the current particle’s fitness score $x_z$ in the swarm and its old local best location $p_z^{best}$ If the current fitness score $x_z$ is greater than $p_z^{best}$, then substitute $p_z^{best}$, with $x_z$; else retain the $x_z$ unchanged Compare the current particle’s fitness score $x_z$ in the swarm and its old global best location $g_z^{best}$ If the current fitness score $x_z$. is greater than $g_z^{best}$, then substitute $g_z^{best}$, with $x_z$; else retain the $x_z$ unchanged Compute $v_{z+1}$ → updated velocity vector Compute $x_{z+1}$ → updated position vector Go to step 2; If the stopping criteria does not met; else terminate

In our other contribution in the paper, we introduced three novel methods of initialization of population: DE-WE, DE-KN, and DE-TO. The Algorithm 2 shows the flow chart of the proposed distribution-based DE initialization.

Algorithm 2 Proposed Pseudo Code of DE Using Novel Method of Initialization

Input:xi = (xi,1, xi,2, xi,3, ..., xi,D), Population size ‘N-P’, Problem Size ‘D’, Mutation Rate ‘F’, Crossover Rate ‘C-R’; Stopping Criteria {Number of Generation, Target}, Upper Bound ‘U’, Lower Bound ‘L’, Output: xi = Global fitness vector with minimal fitness value 1. Pop = Initialize of Paraments (N-P, D, U, L); a. Generate initial population Using WELL,Knuth,Torus 2. While (Stopping Criteria ≠ True) do 3. Best Vector = Evaluate Pop (Pop); 4. vx = Select Rand Vector (Pop); 5. I = Find Index Vector (vx); 6. Select Rand Vector (Pop,v1,v2,v3) where v1 ≠ v2 ≠ v3 ≠ vx 7. vy = v1, + F(v2−v3) 8. For (i = 0; i++; i < D−1) 9. If (randj [0, 1) < C-R) Then 10. U[i] = vx [i] 11. else 12. U[i] = vy [i] 13. End For loop 14. If (Cost Fun Vector(U) ≤ Cost Fun Vector (vx)) Then 15. Update Pop (U, I, Pop); 16. End IF 17. End While 18. Retune Best Vector;

In our last contribution in the paper, we introduced three novel methods of initialization of population: BA-WE, BA-KN, and BA-TO. The Algorithm 3 shows the flow chart of the proposed distribution-based BA initialization.

Algorithm 3 Proposed Pseudo Code of BA Using Novel Method of Initialization

(1) Bat-Initialization();Using WELL,Knuth,Torus (2) E = newly_evluated_population; (3) $f_{min}=$ current_solution(best); (4) While termination_condition_not_meet do (5) fori = 1 to population do (6) $x^t=$ compute_solution(best); (7) if rand(0,1) > $r_i$ then (8) $x^t=$ update_the_current-solution(best) (9) end if {searching locally} (10) if$f_{new}=$ compute_new_solution($x^t)$; (11) E = E + 1; Addition in evaluation (12) if$f_{new}<f_{i }$and N(0,1) < $A_i$ then (13) $x_i=x_t;$ (14) $f_i=f_{new};$ (15) end if {Simulation annealing} (16) $f_{min}=$ explore_for_best_solution(best); (17) end for (18) end while

4. Experimental Setup

To achieve the effective working of the algorithms, it is compulsory to adjust the parameters coupled with all approaches to their most suitable value. Largely, these parameters are observed before the implementation of the algorithm and maintain uniformity throughout the execution. In various studies, it is suggested that the most appropriate methodology for selecting the parameters of any algorithm is predicted through exhaustive experiments for obtaining the optimal parameters. In this study, the experimental setting of the parameters is employed in

Table 1. Experimental setting of parameters.

Parameter	Value
Search Space	[100, −100]
Dimensions	10	20	30
Iterations	1000	2000	3000
Population size	50
Number of PSO Runs	10

and the parameters setting of the algorithms in

Table 2. Parameters setting of parameters.

Algorithm	Parameters
PSO	c1=c2=1.49, w = linearly decreasing
BA	$N_p$=40, $r_{ij}^t\in \left[0,1\right]$, $A_{ij}^t\in \left[0,2\right]$
DE	F ∈ [0.4, 1], CR ∈ 0.6

, respectively, which is on the basis of the literature stated in this section. Along with this, objective functions and their details are in

Table 3. Function table with characteristics.

Sr.#	Function Name	Objective Function	Search Space	Optimal Value
01	Sphere	$Minf(x)={\displaystyle \sum _{i=1}^D{x}_i^2}$	$-5.12\le x_i\le 5.12$	0
02	Rastrigin	$Minf(x)=10D+{{\displaystyle \sum _{i=1}^D\left[x_i^2--10\cos \left(2\pi x\right)\right]}}_i$	$-5.12\le x_i\le 5.12$	0
03	Axis parallel hyper-ellipsoid	$Minf(x)={\displaystyle \sum _{i=1}^D\left(i.x_i^2\right)}$	$-5.12\le x_i\le 5.12$	0
04	Rotated hyper ellipsoid	$Minf(x)={\displaystyle \sum _{i=1}^D{\displaystyle \sum _{j=1}^i\left(x_j^2\right)}}$	$-65.536\le x_i\le 65.536$	0
05	Moved Axis	$Minf(x)={\displaystyle \sum _{i=1}^{D}5i.x_i^2}$	$-5.12\le x_i\le 5.12$	0
06	Sum of different power	$Minf(x)={\displaystyle \sum _{i=1}^D{\left|x_i\right|}^{(i+1)}}$	$-1\le x_i\le 1$	0
07	ChungReynolds	$Minf(x)={\left({\displaystyle {\sum }_{i=1}^D{x}_i^2}\right)}^2$	$-100\le x_i\le 100$	0
08	Csendes	$Minf(x)={\displaystyle \sum _{i=1}^D{x}_i^6}\left(2+\sin \frac{1}{x_i}\right)$	$-1\le x_i\le 1$	0
09	Schaffer	$Minf(x)=0.5+\frac{{\sin }^2{(x_1^2+x_2^2)}^2-0.5}{1+0.001{(x_1^2+x_2^2)}^2}$	$-100\le x_i\le 100$	0
10	Schumer_Steiglitz	$Minf(x)={\displaystyle \sum _{i=1}^D{x}_i^4}$	$-100 \le xi \le 100$	0
11	Schwefel	$Minf(x)={\left({\displaystyle \sum _{i=1}^D{x}_i^2}\right)}^{\alpha }$	$-100\le x_i\le 100$	0
12	Schwefel1.2	$Minf(x)={\displaystyle \sum _{i=1}^D{\left({\displaystyle \sum _{j=1}^i{x}_j}\right)}^2}$	$-100\le x_i\le 100$	0
13	Schwefel 2.21	$Minf(x)=\underset{1\le i\le D}{\max }\left|x_i\right|$	$-100\le x_i\le 100$	0
14	Schwefel 2.22	$Minf(x)={\displaystyle \sum _{i=1}^D\left|x_i\right|}+{\displaystyle \prod _{i=1}^D\left|x_i\right|}$	$-100\le x_i\le 100$	0
15	Schwefel 2.23	$Minf(x)={\displaystyle \sum _{i=1}^D{x}_i^{10}}$	$-10\le x_i\le 10$	0
16	Zakharov	$Minf(x)={\displaystyle \sum _{i=1}^D{x}_i^2}+{\left(\frac{1}{2}{\displaystyle \sum _{i=1}^{n}i{x}_i}\right)}^2+{\left(\frac{1}{2}{\displaystyle \sum _{i=1}^{n}i{x}_i}\right)}^4$	$-5\le x_i\le 10$	0

. The search space boundary is [−100, 100] population size kept 50 with 10 number of runs. Further, 10, 20, and 30 dimensions are used for 1000, 2000, and 3000 iterations, respectively.

5. Results and Discussion

This section briefly describes the simulation results of the proposed approaches and their graphical representation. Primarily, this section is divided into three sub-sections, where each sub-section is specifically dedicated to EAs simulation results such as PSO, DE, and BA, respectively. In addition to this, each EA is also examined through the statistical tests which are also stated in sub-sections.

5.1. Discussion on PSO Results

The simulation was simulated in C + + and applied on a computer using the C + + language on the computer having the Windows 10 operating system with the specifications of 8 Gigabyte ram and 2.3 GHz Core (M) 2 Duo CPU processor. In order to measure the execution of the proposed approaches, WELL-based PSO (WE-PSO), Torus-based PSO (TO-PSO), and Knuth-based PSO (KN-PSO), a group of fifteen non-linear benchmark test functions were utilized to do the comparison of WE-PSO, TOPSO, and KN-PSO with standard PSO, SO-PSO, and HPSO. These functions are normally applied to investigate the performance of any technique. Hence, in our study, we used them to examine the optimization outcomes of the quasi-random-based approach of WE-PSO, TO-PSO, KN-PSO, SO-PSO, and H-PSO. The list of those functions is available in Table 3. In Table 1, D(Dimensions) shows the dimensionality of the problem, S (Search Space) represents the interval of the variables, I(Iterations), Pop(Population size) and in Table 2, f_min denotes the common global optimum minimum value. The parameters for the simulation use c1 = c2 = 1.45, inertia weight w is used in the interval [0.9, 0.4], and swarm size is 50. For simulation, the function dimensions are D = 10, 20, and 30 and a maximum number of epochs is 3000. All techniques were applied to similar parameters for comparatively effective results. In order to check the performance of each technique, all algorithms were tested for 30 runs.

The purpose of this study continues to observe whereby the unique characteristics of experimental results rely on dimensions of these standard benchmark functions.

The objective of this study is to find the most suitable initializing approach for the PSO and during the first experiment, the proposed WE-PSO, TO-PSO, and KN-PSO with other approaches SO-PSO, H-PSO, and standard PSO was investigated. The objective of the second simulation is to find the nature of the dimension regarding standard function optimization. Lastly, the simulation results of WE-PSO, TO-PSO, and KN-PSO were compared with standard PSO, SO-PSO, and H-PSO. In the rest of the paper, simulation results are discussed in detail.

Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21 and Figure 22 contain the graphical representation of the comparisons of the proposed WE-PSO, TO-PSO, and KN-PSO with standard PSO, H-PSO, and SO-PSO. In the x-axis, dimensions of the problem 10, 20, and 30 are presented while the y-axis represents the mean best against each dimension of the problem.

We can see that the majority of the figures contain a better convergence curve for KN-PSO on functions F1, F2, F3, F4, F5, F6, F17, F8, F9, F10, F11, F12, F13, F14, and F15 over WE-PSO, TO-PSO, H-PSO, SO-PSO, and standard PSO on all dimensions comprehensively. The other proposed approach TO-PSO provides better results over WE-PSO on functions F1, F2, F4, F6, F17, F8, F9, F10, F14, F15, and F16 and beats on all functions for PSO, SO-PSO, and TO-PSO.

In this simulation, PSO is initialized with the WELL sequence (WE-PSO), Torus sequence (TO-PSO), and Knuth sequence (KN-PSO) instead of uniform distribution. The variant proposed WE-PSO, TO-PSO, and KN-PSO is compared with other initialized approaches, Sobol sequence (SO-PSO), Halton sequence (H-PSO), and standard PSO. The experimental results give superior results in higher dimensions for KN-PSO on other SO-PSO, H-PSO, PSO, and proposed approach TO-PSO and WE-PSO.

The core objective of this simulation setup is to find the superiority of results depending upon the dimension of the functions that are to be optimized. In experiments, three dimensions for benchmark functions D = 10, D = 20, and D = 30 were used. Simulation results are presented in

Table 4. Comparative results for all PSO-based approaches on 16 standard benchmark functions.

Functions	DIM × Itr	PSO	SO−PSO	H−PSO	TO−PSO	WE−PSO	KN−PSO
		Mean	Mean	Mean	Mean	Mean	Mean
F1	10 × 1000	2.33 × 10−74	2.74 × 10−76	3.10 × 10−77	5.57 × 10−78	5.91 × 10−78	0.0000 × 10+00
	20 × 2000	1.02 × 10−84	8.20 × 10−88	1.76 × 10−90	1.30 × 10−90	4.95 × 10−90	3.14001 × 10−217
	30 × 3000	1.77 × 10−26	7.67 × 10−20	4.13 × 10−32	1.25 × 10−51	1.30 × 10−42	8.91595 × 10−88
F2	10 × 1000	4.97 × 10−01	4.97 × 10−01	7.96 × 10−01	3.98 × 10−01	2.98 × 10−01	−8602.02
	20 × 2000	8.17 × 10+00	6.47 × 10+00	3.58 × 10+00	2.89 × 10+00	3.11 × 10+00	−31,433.3
	30 × 3000	1.01 × 10+01	9.86 × 10+00	9.45 × 10+00	8.16 × 10+00	7.76 × 10+00	−60,711.8
F3	10 × 1000	8.70 × 10−80	1.79 × 10−79	4.87 × 10−79	3.91 × 10−82	4.40 × 10−81	0.0000 × 10+00
	20 × 2000	2.62144	7.86432	2.62144	7.07 × 10−90	1.78 × 10−89	4.78718 × 10−237
	30 × 3000	2.62 × 10+01	1.57 × 10+01	1.05 × 10+01	7.70 × 10−35	3.87 × 10−57	1.57084 × 10−97
F4	10 × 1000	4.46 × 10−147	3.86 × 10−147	9.78 × 10−145	7.29 × 10−148	1.24 × 10−150	0.0000 × 10+00
	20 × 2000	3.14 × 10−155	9.27 × 10−154	2.75 × 10−159	5.14 × 10−158	4.96 × 10−159	0.0000 × 10+00
	30 × 3000	1.82 × 10−133	2.36 × 10−135	8.53 × 10−130	3.13 × 10−138	2.54 × 10−136	1.6439 × 10−228
F5	10 × 1000	4.35 × 10−79	8.95 × 10−79	2.43 × 10−78	2.04 × 10−80	2.20 × 10−80	0.0000 × 10+00
	20 × 2000	1.31 × 10+01	3.93 × 10+01	1.31 × 10+01	3.54 × 10−89	3.12 × 10−89	2.39359 × 10−236
	30 × 3000	1.31 × 10+02	7.86 × 10+01	5.24 × 10+01	3.85 × 10−34	1.94 × 10−56	2.9093 × 10−87
F6	10 × 1000	1.70 × 10−61	4.45 × 10−64	7.29 × 10−66	2.46 × 10−66	4.62 × 10−66	3.04226 × 10−318
	20 × 2000	3.25 × 10−112	4.39 × 10−112	5.01 × 10−109	2.56 × 10−115	4.45 × 10−113	8.59557 × 10−277
	30 × 3000	7.21 × 10−135	4.10 × 10−124	1.51 × 10−134	6.22×10−137	6.96 × 10−135	2.33033 × 10−223
F7	10 × 1000	2.96 × 10−157	2.39 × 10−157	1.28 × 10−157	4.89 × 10−159	2.47 × 10−163	0.0000 × 10+00
	20 × 2000	8.79 × 10−177	1.77 × 10−184	3.49 × 10−183	3.09 × 10−187	3.41 × 10−186	0.0000 × 10+00
	30 × 3000	1.23 × 10−82	1.25 × 10−116	5.99 × 10−130	5.01 × 10−135	4.60 × 10−134	8.03288 × 10−175
F8	10 × 1000	4.39 × 10−200	1.98 × 10−194	4.51 × 10−197	1.26 × 10−202	8.99 × 10−201	4.9228 × 10−67
	20 × 2000	1.57 × 10−20	1.04 × 10−93	1.10 × 10−148	2.84 × 10−157	4.09 × 10−151	4.5887 × 10−16
	30 × 3000	1.89 × 10−09	4.54 × 10−10	1.14 × 10−08	1.40 × 10−10	1.34 × 10−09	2.2334 × 10−08
F9	10 × 1000	5.49 × 10−01	1.30 × 10−01	2.02 × 10−01	1.26 × 10−01	1.42 × 10−01	0.824968
	20 × 2000	2.05 × 10+00	7.83 × 10−01	6.83 × 10−01	5.84 × 10−01	4.32 × 10−01	4.56265
	30 × 3000	1.12 × 10+00	9.99 × 10−01	9.56 × 10−01	9.06 × 10−01	9.12 × 10−01	7.25675
F10	10 × 1000	2.23 × 10−138	2.23 × 10−138	4.35 × 10−137	1.02 × 10−140	1.10 × 10−139	0.0000 × 10+00
	20 × 2000	3.79 × 10−148	7.87 × 10−149	4.19 × 10−147	3.78 × 10−151	8.73 × 10−153	0.0000 × 10+00
	30 × 3000	4.43 × 10−126	7.52 × 10−133	1.57 × 10−128	2.03 × 10−134	1.38 × 10−133	2.26229 × 10−221
F11	10 × 1000	3.75 × 10−187	1.57 × 10−192	2.15 × 10−191	5.57 × 10−198	8.99 × 10−198	0.0000 × 10+00
	20 × 2000	5.29 × 10−193	2.53 × 10−195	8.45 × 10−195	8.45 × 10−195	9.83 × 10−197	0.0000 × 10+00
	30 × 3000	4.82 × 10−154	8.84 × 10−159	5.49 × 10−168	2.04 × 10−170	5.75 × 10−173	9.00586 × 10−278
F12	10 × 1000	1.13 × 10−01	1.67 × 10−02	2.28 × 10−02	4.78 × 10−03	2.89 × 10−03	2.739 × 10−12
	20 × 2000	1.39 × 10+01	5.03 × 10+00	2.95 × 10+00	1.28 × 10+00	1.67 × 10+00	7.819 × 10+00
	30 × 3000	7.45 × 10+00	1.22 × 10+01	8.74 × 10+00	2.94 × 10+00	4.94 × 10+00	2.239 × 10+01
F13	10 × 1000	8.04 × 10−26	8.01 × 10−27	3.59 × 10−27	1.24 × 10−27	1.41 × 10−27	0.0000 × 10+00
	20 × 2000	1.42 × 10−08	2.64 × 10−11	3.29 × 10−10	2.99 × 10−10	2.14 × 10−12	0.0000 × 10+00
	30 × 3000	6.20 × 10−03	1.41 × 10−03	9.36 × 10−03	1.12 × 10−03	1.41 × 10−03	0.0000 × 10+00
F14	10 × 1000	3.62 × 10−38	3.62 × 10−38	5.92 × 10−36	6.92 × 10−39	1.95 × 10−38	7.78286 × 10−197
	20 × 2000	6.27 × 10−10	1.38 × 10−09	7.91 × 10−13	2.49 × 10−12	1.17 × 10−13	6.6163 × 10−12
	30 × 3000	2.56 × 10−06	4.80 × 10+01	1.34 × 10−06	5.40 × 10−11	4.88 × 10−09	9.3032 × 10−06
F15	10 × 1000	1.10 × 10−294	3.19 × 10−301	2.78 × 10−307	1.94 × 10−307	3.21 × 10−308	6.26612 × 10−138
	20 × 2000	6.16 × 10−271	5.09 × 10−276	3.74 × 10−270	1.60 × 10−276	4.85 × 10−268	1.29033 × 10−25
	30 × 3000	3.08 × 10−207	1.04 × 10−200	8.12 × 10−209	2.34 × 10−215	3.06 × 10−212	2.27 × 10−06
F16	10 × 1000	5.4835385	8.5299 × 10	3.3074 × 10−16	1.224803	8.3354 × 10−07	2.26476 × 10−27
	20 × 2000	83.467	1.6344	0.18037	49.16841	5.1322	7.17014 × 10−72
	30 × 3000	265.90708	282.1864	45.0408	133.9679	67.0301	5.45179 × 10−251

. From these simulation results, it was found that functions having larger dimensions were tougher to optimize and it can be seen from the Table 4 when dimension size id is D = 20 and D = 30 and our proposed approach KN-PSO ^shows belter result on higher dimensions on other approaches WE-PSO, TO-PSO, standard PSO, H-PSO, and SO-PSO.

KN-PSO is compared with the other approaches like WE-PSO, TO-PSO, SO-PSO, H-PSO, and standard PSO where each technique true value is presented for comparison with other techniques for the same nature of problems. Standard benchmark functions are presented in the Table 3 and their parameter settings are also shown in the Table 1. Table 4 shows that with dimension D-30, KN-PSO is more superior and outperforms in convergence than the WE-PSO, TO-PSO, standard PSO, SO-PSO, and H-PSO. The comparative analysis can be seen from Table 4 that with smaller dimension size, standard PSO performs well (D = 10); while the size of the dimension increases, KN-PSO outperforms in convergence significantly. Hence, KN-PSO is best for higher dimensions. The experimental results from Table 4 show that KN-PSO outclassed the results of WE-PSO, TO-PSO, SO-PSO, H-PSO, and traditional PSO for all functions. It can be seen that the TO-PSO outperformed the results of the other techniques in all functions on SO-PSO, H-PSO, standard PSO; while in the other approaches, H-PSO performs better on functions f4, f1, f2 for 20D but H-PSO gives overall poor result on higher dimensions and SO-PSO gives slightly better results on functions f8, f9, f15 on 10-D but worst results on larger dimensions. Standard PSO did not provide better results. Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 depict that WE-PSO outperforms in simulation results than other approaches for solving the standard benchmark tests functions for dim size D = 10, D = 20, D = 30.

To validate the numerical results, mean ranks obtained by Kruskal–Wallis and Friedman tests for KN-PSO, WE-PSO, TO-PSO, SO-PSO, HA-PSO, and standard PSO are given in

Table 5. Mean ranks obtained by Kruskal–Wallis and Friedman tests for all.

Approaches	Friedman Value	p-Value	Kruskal–Wallis	p-Value
PSO	39.09	0.001	39.33	0.001
SO-PSO	37.47	0.001	38.39	0.001
H-PSO	38.50	0.001	38.91	0.001
TO-PSO	41.79	0.000	42.67	0.000
WE-PSO	41.88	0.000	42.50	0.000
KN-PSO	18.24	0.001	23.31	0.002

.

5.2. Discussion on DE Results

Population initialization is a vital factor in the evolutionary computing-based algorithm, which considerably influences the diversity and convergence. In order to improve the diversity and convergence, rather applying the random distribution for initialization, quasi-random sequences are more useful to initialize the population. In this paper, the capability of DE was extended to make it suitable for the optimization problem by introducing new initialization techniques: Knuth sequence-based (DE-KN), the Torus-based sequence-based (DE-TO), and the WELL sequence-based (DE-WE) by using low discrepancies sequence, Torus to solve the optimization problems in large dimension search spaces.

For global optimization, the most considerable variety of benchmark problems can be used. All benchmark problems have their own individual abilities and the variety of detailed characteristics of such functions explains the level of complexity for benchmark problems. For the efficiency analysis of the above-mentioned optimization algorithms, Table 3 displays the benchmark problems that are utilized. Table 3 explains the following contents of benchmark problems: name, range, domain, and formulas. In this study, those benchmark problems are incorporated, which have been extensively utilized in the literature for conveying a deep knowledge of the performance related to the above-mentioned optimization algorithms.

To measure the effectiveness and robustness of optimization algorithms, benchmark functions are applied. In this study, fifteen computationally-expensive black box functions are applied with their various abilities and traits. The purpose to utilize these benchmark functions is to examine the effectiveness of the above-mentioned proposed approaches.

In this section, a comparison among the low discrepancies sequence, methods, is performed with each other with reference to capabilities and efficiency with the help of highdimensional fifteen benchmark functions. Nevertheless, the whole performance of optimization algorithms varies on the basis of setting parameters, and also with other testing criteria. Benchmark problems may be embeded to demonstrate the performance of the low discrepancies sequence approaches, at various complex levels.

Table 6. Comparative results for all DE-based approaches on 16 standard benchmark functions.

Functions	DIM × Iter	DE	DE−H	DE−S	DE−TO	DE−WE	DE−KN
F1	10 × 1000	1.1464 × 10−44	2.1338 × 10−44	5.8561 × 10−44	7.4117 × 10−45	7.4827 × 10−39	5.7658 × 10−39
	20 × 2000	3.3550 × 10−46	7.2338 × 10−46	1.3545 × 10−45	1.2426 × 10−45	9.6318 × 10−45	7.1501 × 10−45
	30 × 3000	8.8946 × 10−47	1.2273 × 10−45	9.4228 × 10−46	1.6213 × 10−46	6.2007 × 10−46	5.7425 × 10−46
F2	10 × 1000	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00
	20 × 2000	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00
	30 × 3000	1.8392 × 10+01	1.1846 × 10+01	1.8871 × 10+01	3.7132 × 10−01	5.0821 × 10+00	6.6313 × 10+00
F3	10 × 1000	5.00325 × 10−44	1.5019 × 10−38	9.3956 × 10−44	4.7807 × 10−44	1.6251 × 10−38	1.3411 × 10−38
	20 × 2000	2.56987 × 10−45	4.1485 × 10−44	1.5339 × 10−44	3.0262 × 10−45	9.5984 × 10−44	1.3606 × 10−43
	30 × 3000	1.01692 × 10−45	2.7349 × 10−45	4.0581 × 10−45	4.5726 ×10−45	4.5686 × 10−45	5.4659 × 10−45
F4	10 × 1000	5.81825 × 10−42	3.0950 × 10−36	2.2300 × 10−41	1.6903 × 10−41	1.1331 × 10−36	3.8869 × 10−36
	20 × 2000	2.70747 × 10−43	1.0658 × 10−41	1.6730 × 10−42	1.3490 × 10−42	1.3094 × 10−41	6.0053 × 10−42
	30 × 3000	2.99887 × 10−43	1.4032 × 10−42	4.4442 × 10−42	5.9186 × 10−43	4.6922 × 10−43	1.4829 × 10−42
F5	10 × 1000	1.65318 × 10−43	4.7939 × 10−38	7.0329 × 10−43	4.8106 × 10−43	4.3219 × 10−38	3.5770 × 10−38
	20 × 2000	1.39082 × 10−44	3.6325 × 10−43	4.2191 × 10−44	2.7448 × 10−44	5.8557 × 10−43	1.4008 × 10−43
	30 × 3000	6.07162 × 10−45	1.7557 × 10−44	1.6295 × 10−44	2.0582 × 10−44	8.6773 × 10−45	4.2285 × 10−44
F6	10 × 1000	7.8201 × 10−96	3.8819 × 10−96	9.7956 × 10−96	2.3292 × 10−95	8.4774 × 10−94	2.8037 × 10−95
	20 × 2000	1.6847 × 10−125	8.6880 × 10−124	5.9005 × 10−122	8.7800 × 10−123	3.7438 × 10−124	1.3947 × 10−124
	30 × 3000	2.4533 × 10−140	1.5487 × 10−139	5.7211 × 10−138	4.4492 × 10−137	6.5749 × 10−140	3.4442 × 10−137
F7	10 × 1000	8.0217 × 10−75	7.3243 × 10−67	5.7807 × 10−66	1.0243 × 10−73	1.9035 × 10−67	1.4359 × 10−65
	20 × 2000	4.0682 × 10−71	1.5037 × 10−70	1.5747 × 10−69	1.0623 × 10−70	5.5546 × 10−70	2.3507 × 10−70
	30 × 3000	8.5895 × 10−68	6.6009 × 10−68	3.3919 × 10−67	2.6036 × 10−67	1.1587 × 10−67	2.1901 × 10−67
F8	10 × 1000	7.0221 × 10−120	3.4271 × 10−108	2.7718 × 10−108	6.3092 × 10−118	3.9423 × 10−106	9.9394 × 10−108
	20 × 2000	5.2096 × 10−108	7.7158 × 10−89	1.4732 × 10−106	8.8720 × 10−107	3.4490 × 10−107	2.2539 × 10−106
	30 × 3000	1.2538 × 10−98	1.8071 × 10−98	1.1085 × 10−95	7.2462 × 10−98	2.5375 × 10−99	5.8040 × 10−98
F9	10 × 1000	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00
	20 × 2000	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00
	30 × 3000	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00	0.0000 × 10+00
F10	10 × 1000	1.3459 × 10−75	2.6493 × 10−66	2.6884 × 10−66	3.6168 × 10−67	3.8397 × 10−67	1.8408 × 10−66
	20 × 2000	3.0478 × 10−71	1.6106 × 10−69	5.5253 × 10−69	2.7746 × 10−70	5.3662 × 10−70	5.0931 × 10−70
	30 × 3000	8.2514 × 10−68	1.0937 × 10−66	4.1120 × 10−67	1.3055 × 10−67	3.5397 × 10−68	6.073 × 10−67
F11	10 × 1000	2.3417 × 10−42	1.2483 × 10−41	1.3726 × 10−41	6.3337 × 10−42	4.8161 × 10−42	7.34640 × 10−42
	20 × 2000	8.4769 × 10−44	3.5140 × 10−43	3.3777 × 10−43	2.4721 × 10−43	1.9553 × 10−43	3.6961 × 10−43
	30 × 3000	3.6888 × 10−44	6.9938 × 10−44	2.5123 × 10−43	1.4710 × 10−43	4.0019 × 10−44	3.9503 × 10−43
F12	10 × 1000	2.3304 × 10+00	4.4354 × 10+00	3.4520 × 10+00	5.1229 × 10+00	3.8782 × 10+00	2.7840 × 10+00
	20 × 2000	3.1768 × 10+04	3.9596 × 10+04	3.8814 × 10+04	2.9488 × 10+04	4.1181 × 10+04	4.0914 × 10+04
	30 × 3000	1.1760 × 10+06	1.0300 × 10+06	1.3402 × 10+06	1.2008 × 10+06	1.0916 × 10+06	1.0160 × 10+06
F13	10 × 1000	1.3940 × 10−65	1.3756 × 10−64	3.1956 × 10−66	9.3609 × 10−64	5.4864 × 10−63	9.2695 × 10−63
	20 × 2000	2.0163 × 10−111	8.5333 × 10−110	8.5260 × 10−111	3.9836 × 10−109	5.0102 × 10−115	4.4624 × 10−110
	30 × 3000	1.4146 × 10−156	4.3434 × 10−156	4.4702 × 10−154	4.3862 × 10−151	1.0781 × 10−153	1.0142 × 10−149
F14	10 × 1000	9.1259 × 10−24	2.1900 × 10−23	2.5559 × 10−23	2.9039 × 10−23	1.9174 × 10−23	3.3427 × 10−23
	20 × 2000	2.6867 × 10−25	3.8631 × 10−25	1.5177 × 10−24	5.5714 × 10−25	4.5049 × 10−25	5.6503 × 10−25
	30 × 3000	5.9241 × 10−26	8.6401 × 10−26	8.4348 × 10−26	1.4630 × 10−25	9.7932 × 10−26	1.4921 × 10−25
F15	10 × 1000	1.0493 × 10−185	4.0276 × 10−181	5.0331 × 10−182	3.1770 × 10−183	1.1698 × 10−180	2.6563 × 10−182
	20 × 2000	2.9407 × 10−159	9.9152 × 10−159	2.1401 × 10−158	9.0345 × 10−156	3.8871 × 10−158	8.0144 × 10−160
	30 × 3000	4.6769 × 10−138	1.0737 × 10−137	7.0544 × 10−138	8.0376 × 10−138	4.9091 × 10−139	1.1054 × 10−137
F16	10 × 1000	1.8635 × 10−04	1.8109 × 10−02	4.9798 × 10−02	5.8605 × 10−04	1.4858 × 10−02	3.7220 × 10−02
	20 × 2000	1.1032 × 10+00	1.6605 × 10+00	1.7157 × 10+00	1.4875 × 10+00	1.5697 × 10+00	1.2008 × 10+00
	30 × 3000	2.8283 × 10+01	2.2049 × 10+01	2.9388 × 10+01	2.8205 × 10+01	2.5794 × 10+01	2.9526 × 10+01

contains the experimental simulation results on benchmark functions. The exhaustive statistical results are explained in

Table 7. Mean ranks obtained by Kruskal–Wallis and Friedman tests for all.

Approaches	Friedman Value	p-Value	Kruskal–Wallis	p-Value
DE	63.74	0.000	65.11	0.000
DE-H	59.31	0.000	60.41	0.000
DE-S	64.01	0.000	65.05	0.000
DE-TO	63.76	0.000	65.35	0.000
DE-WE	63.35	0.000	63.93	0.000
DE-KN	63.33	0.000	64.06	0.000

. From Figure 23, Figure 24, Figure 25, Figure 26, Figure 27, Figure 28, Figure 29, Figure 30, Figure 31, Figure 32, Figure 33, Figure 34, Figure 35, Figure 36, Figure 37 and Figure 38, the experimental results of constrained benchmark test functions are only exhibited by having the surface with D = 10, 20, and 30. The experimental results of this work may not contemplate the entire competency of the new proposed low discrepancies sequence in accordance with all the possible conditions.

The core objective of this section is to review the consequences of tested optimization approaches in high dimensionality, regarding the accuracy and reliability of achieved solutions at the time of solving complex and computationally-expensive optimization problems. From Figure 23, Figure 24, Figure 25, Figure 26, Figure 27, Figure 28, Figure 29, Figure 30, Figure 31, Figure 32, Figure 33, Figure 34, Figure 35, Figure 36, Figure 37 and Figure 38, the performance of following methods: DE-KN, DE-WE, DE-TO, DE-S, DE-H, and traditional DE is compared for sixteen benchmark functions. In the graphs, the horizontal axis displays the total number of iterations, while on the other hand, the vertical axis displays the mean value of objective functions at the fixed number of function computations. Correspondingly, the value achieved in each iteration operates as a performance measure. As a result, the exploitation ability of traditional DE is moderately low, particularly for high-dimensional problems. The results are also disclosed that traditional DE, DE-S, DE-H are only effective in performance, while they tackle with expensive design problems having low dimensionality.

Beside this, DE-TO has excellent control on the high dimensionality problems than other methods in spite of complexity and the superficial topology of the examined problems. Figure 23, Figure 24, Figure 25, Figure 26, Figure 27, Figure 28, Figure 29, Figure 30, Figure 31, Figure 32, Figure 33, Figure 34, Figure 35, Figure 36, Figure 37 and Figure 38 show the achievements of traditional DE, DE-S, DE-H, DE-KN, and DE-WE algorithms with to regard to their efficiency and capability. The results demonstrate that DE-TO outperforms in higher dimensionality problems. By summarizing it, the dimensionality strongly influences the working of most algorithms, however, it is observed that DE-TO is more consistent during the increment of dimensions of the problem. Due to this consistency of DE-TO, it is proven that the DE-TO algorithm has greater capability of exploration.

For statistical comparison, a widely known mean ranks obtained by Kruskal–Wallis and Friedman tests is implemented to compare the implications between the DE-TO algorithm and other algorithms in DE-KN, DE-WE, DE-S, DE-H and standard DE are given in the Table 7.

5.3. Discussion on BA Results

The initialization technique plays a vital role in evolutionary and swarm-based stochastic algorithms. As, the traditional BA is not good in the process of global search. Therefore, the performance of BA can be increased by assigning the robust initial fitness to the particles. This may cause the enhancement in the diversity of swarm. For improving the performance of BA in terms of minimizing the global solution, we proposed three novel techniques of population initialization: the Knuth sequence-based BA (BA-KN), the Torus sequence-based-BA (BA-TO), and the WELL sequence-based BA (BA-WE).

For evolutionary comparison, the proposed techniques: Knuth sequence-based BA(BA-KN), the Torus sequence-based BA (BA-TO), and the WELL sequence-based BA (BA-WE) are compared with BA with Halton distribution (BA-HA), BA with Sobol distribution (BA-SO), are tested on sixteen standard benchmark functions and compared with the standard BA. The experimental results showed that the proposed techniques, BA based on Halton distribution (BA-HA), BA with Sobol distribution (BA-SO), and BA based on Torus sequence (BA-TO), perform better as compared to the standard BA, where among these three proposed techniques, (BA-TO) outperforms the others such as (BA-HA) and the (BA-SO). The convergence curves for all the techniques are presented in Figure 39, Figure 40, Figure 41, Figure 42, Figure 43, Figure 44, Figure 45, Figure 46, Figure 47, Figure 48, Figure 49, Figure 50, Figure 51, Figure 52, Figure 53 and Figure 54. The results depict that proposed techniques can help BA to avoid from premature convergence and to find the optimum solution quickly in the search space. The performance of the proposed techniques (BA-KN), (BA-WE), and (BA-TO) are compared with the standard BA and (BA-SO), (BA-HA), and (BA-TO) on standard benchmark functions with different dimensional size. It can be concluded from results that quasi-random sequences are best to create the random number sequences for BA and also for other population-based algorithms.

In this work, the primary concern is to reach the optimal solution, which is 0 in the ideal case. We investigated the different distribution approaches such as Knuth, WELL, Torus, Sobol, Halton, and random to initialize the BA for ensuring the swarm diversity in the very initial stage of the process. It is observed from

Table 8. Comparative results for all BA-based approaches on 16 standard benchmark functions.

		BA	BA−SO	BA−HA	BA−TO	BA−WE	BA−KN
F#	DIM × Iter	Mean	Mean	Mean	Mean	Mean	Mean
F1	10 × 1000	1.59 × 10−07	1.03 × 10−07	1.32 × 10−07	8.95 × 10−08	0.63202	0.88186
	20 × 2000	1.02 × 10−84	8.20 × 10−88	1.76 × 10−90	1.30 × 10−90	4.95 × 10−90	3.14001 × 10−217
	30 × 3000	1.77 × 10−26	7.67 × 10−20	4.13 × 10−32	1.25 × 10−51	1.30 × 10−42	8.91595 × 10−88
F2	10 × 1000	4.13 × 10+01	3.17 × 10+01	1.82 × 10+01	3.55 × 10+01	37.9883	40.8852
	20 × 2000	1.06 × 10+02	5.78 × 10+01	1.15 × 10+02	9.47 × 10+01	140.2023	147.8938
	30 × 3000	2.05 × 10+02	1.46 × 10+02	1.83 × 10+02	1.69 × 10+02	271.307	275.8626
F3	10 × 1000	5.93 × 10−07	4.70 × 10−07	3.99 × 10−07	4.0 × 10−07	2.9125	5.2009
	20 × 2000	1.57 × 10−06	1.05 × 10−06	1.29 × 10−06	1.05 × 10−06	49.3011	72.4834
	30 × 3000	3.53 × 10−06	3.48 × 10−06	3.27 × 10−06	2.20 × 10−06	197.4826	257.9855
F4	10 × 1000	2.19 × 10+05	1.11 × 10+05	1.66 × 10−14	2.07 × 10+02	9.268	13.5548
	20 × 2000	2.56 × 10+07	2.42 × 10+07	2.31 × 10+07	1.50 × 10+07	160.0394	255.3367
	30 × 3000	1.43 × 10+08	1.38 × 10+08	3.30 × 10+08	1.34 × 10+08	656.5592	946.3934
F5	10 × 1000	2.81 × 10−06	2.67 × 10−06	1.69 × 10−06	2.47 × 10−06	19.7651	22.0461
	20 × 2000	7.43 × 10−06	5.77 × 10−06	6.25 × 10−06	5.33 × 10−06	250.8679	293.7174
	30 × 3000	1.59 × 10−05	1.43 × 10−05	1.50 × 10−05	9.59 × 10−06	1029.0595	1277.0077
F6	10 × 1000	6.19 × 10−04	4.54 × 10−04	3.89 × 10−04	3.39 × 10−04	10.1173	10.1222
	20 × 2000	7.96 × 10−04	6.32 × 10−04	7.77 × 10−04	5.78 × 10−04	20.2119	20.1467
	30 × 3000	1.05 × 10−03	1.01 × 10−03	1.01 × 10−03	6.65 × 10−04	30.3623	30.2845
F7	10 × 1000	15.8966	18.851	18.2544	15.3465	16.9429	18.4835
	20 × 2000	839.1846	686.8456	762.1919	690.0657	876.2518	496.7506
	30 × 3000	4892.6864	4877.7072	4476.0152	4482.3035	5361.4808	3860.4327
F8	10 × 1000	0.023455	0.01557	0.018152	0.016735	0.024264	0.020123
	20 × 2000	0.45222	0.44101	0.39478	0.39719	0.39445	0.24522
	30 × 3000	1.7266	1.0969	1.3512	1.3717	1.4375	0.99905
F9	10 × 1000	4.1394	3.8003	3.8687	4.0739	3.4024	3.9516
	20 × 2000	8.587	8.8019	8.5686	8.585	8.8319	8.5709
	30 × 3000	13.0878	13.502	13.4188	13.2291	13.2835	13.4514
F10	10 × 1000	6.66 × 10−15	3.31 × 10−15	1.90 × 10−15	2.43 × 10−15	1.3357	2.0298
	20 × 2000	3.65 × 10−15	2.03 × 10−15	1.55 × 10−15	1.12 × 10−15	24.9415	53.16
	30 × 3000	1.71 × 10−15	1.06 × 10−15	1.06 × 10−15	9.24 × 10−16	115.9262	318.7949
F11	10 × 1000	218.1498	113.8805	105.5027	58.8987	69.5656	52.7079
	20 × 2000	31293.8096	17609.1493	20760.6498	17638.3139	26832.6985	13990.4835
	30 × 3000	651165.7416	338621.8857	323118.6791	268752.5102	432441.3838	213235.6129
F12	10 × 1000	2.96 × 10+03	2.21 × 10+03	2.27 × 10+03	1.49 × 10+03	253.722	272.7033
	20 × 2000	2.21 × 10+04	1.28 × 10+04	1.49 × 10+04	5.93 × 10+03	11265.9616	9512.9456
	30 × 3000	2.65 × 10+05	7.06 × 10+04	1.65 × 10+05	7.19 × 10+04	71723.2776	67828.8796
F13	10 × 1000	1.4453	1.2827	1.2766	1.298	1.3884	1.3271
	20 × 2000	2.7303	2.8404	2.746	2.7973	2.9688	2.6632
	30 × 3000	3.9975	3.9993	4.2588	4.0509	4.2646	3.9435
F14	10 × 1000	3.83 × 10+06	7.39 × 10+06	2.33 × 10+05	2.69 × 10+04	4.71 × 10+08	4.23 × 10+08
	20 × 2000	7.28 × 10+19	1.55 × 10+17	1.30 × 10+20	1.69 × 10+18	7.78 × 10+19	2.94 × 10+19
	30 × 3000	5.15 × 10+33	3.30 × 10+31	8.00 × 10+32	1.21 × 10+30	1.99 × 10+33	5.50 × 10+33
F15	10 × 1000	2.40 × 10−36	1.11 × 10−37	2.08 × 10−37	2.19 × 10−37	5.27 × 10−02	3.76 × 10−02
	20 × 2000	1.50 × 10−37	4.91 × 10−38	5.70 × 10−38	4.72 × 10−39	1.83 × 10+02	1.30 × 10+02
	30 × 3000	1.39 × 10−37	2.20 × 10−38	7.43 × 10−38	1.15 × 10−38	1.08 × 10+04	2.04 × 10+04
F16	10 × 1000	4.3197	3.2767	3.0513	2.8833	4.0923	3.0495
	20 × 2000	21.9881	24.5093	22.608	2.8833	23.917	22.102
	30 × 3000	89.0053	80.3004	74.4391	66.7812	67.7061	63.0376

that Knuth distribution-based BA initialization gives better results as compared to the other quasi-random sequences. To validate the numerical results, mean ranks obtained by Kruskal–Wallis and Friedman tests for BA-KN, BA-WE, BATO, BA-SO, BA-HA, and standard BA are given in the

Table 9. Mean ranks obtained by Kruskal–Wallis and Friedman tests for all.

Approaches	Friedman Value	p-Value	Kruskal–Wallis	p-Value
BA	44.88	0.000	46.15	0.000
BA-SO	44.82	0.000	46.00	0.000
BA-HA	40.29	0.000	40.90	0.000
BA-TO	44.71	0.000	45.16	0.000
BA-WE	40.12	0.001	32.67	0.005
BA-KN	39.53	0.000	32.32	0.006

.

6. Comparison of PSO, BA, and DE Regarding Data Classification

6.1. NN Classifications with PSO-Based Initialization Approaches

For further verification of performance of proposed algorithms TO-PSO, WE-PSO, and KN-PSO, a comparative study for real world benchmark datasets problem is tested for training of the neural network. We performed experiments using seven benchmark datasets (Diabetes, Heart, Wine, Seed, Vertebral, Blood Tissue, and Mammography) exerted from the worldwide famous machine-learning repository of UCI. Training weights are initialized within interval [−50, 50]. Accuracy of the feed forward neural network is tested in the form of root mean squared error (RMSE).

Table 10. Characteristics of UCI benchmarks DataSets.

S. No	Data Set	Continuous	Nature	No. of Inputs	No. of Classes
1	Diabetes	8	Real	8	2
2	Heart	13	Real	13	2
3	Wine	13	Real	13	3
4	Seed	7	Real	7	3
5	Vertebral	6	Real	6	2
6	Blood Tissue	5	Real	5	2
7	Memo Graphy	6	Real	6	2

shows the characteristics of the datasets used.

Discussion

The multi-layer feed forward neural network is trained with the back propagation algorithm, standard PSO, SO-PSO, H-PSO, and proposed TO-PSO, KN-PSO, and WE-PSO. Comparison of these training approaches tested on real classification problem datasets are taken from the UCI repository. The cross validation method is used to compare the performances of different classification techniques. In the paper, the k-fold cross validation method used for the comparison of classification performances for the training of the neural network with back propagation, standard PSO, SO-PSO, H-PSO, and proposed TO-PSO, KN-PSO, and WE-PSO is used. The k-fold cross validation method was proposed and used in the experimental with value k = 10. The dataset divided into 10 chunks where each chunk of data contains the same proportion of each class of dataset. One chunk is used as testing while nine chunks are used as the training phase. The experimental results of algorithms such as with back propagation, standard PSO, SOPSO, H-PSO, and proposed TO-PSO, KN-PSO, and WE-PSO are compared with each other on seven well-known real datasets taken from UCI and their performances are evaluated. In

Table 11. Results of 10-fold classification rates of ANN-training methods in 7 datasets for accuracy.

S. No	Data Sets	Type	BPA	PSONN	SO-PSONN	H-PSONN	TO-PSONN	WE-PSONN	KN-PSONN
			Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc
1	Diabetes	2-Class	65.3%	69.1%	69.1%	71.6%	73.3%	74.1%	78.5%
2	Heart	2-Class	68.3%	72.5%	67.5%	72.5%	77.5	77.5%	79%
3	Wine	3-Class	62.17%	61.11%	66.66%	67.44%	69.44%	69.6%	72%
4	Seed	3-Class	70.56%	77.77%	84.44%	77.77%	88.88%	91.11%	93%
5	Vertebral	2-Class	84.95%	92.85%	92.85%	92.85%	94.64%	94.64%	96%
6	Blood Tissue	2-Class	73.47%	78.6%	78.66%	70%	82.66%	84%	87%
7	Memo Graphy	2-Class	71.26%	76.66%	63%	85%	88.88%	96.66%	98%

, the simulation results show that the training of the neural network with the KN-PSO algorithm outperforms in accuracy and is capable to provide good classification accuracy than the other traditional approaches. The KN-PSO algorithm may be used effectively for data classification and statistical problems in the future as well. Figure 55 represents the accuracy graph for seven datasets.

The classification testing accuracy were imported from Microsoft Excel Spreadsheet to the software RStudio version 1.2.5001 to get assurance of the winner approach statistically among all the other approaches. The testing accuracy of all seven variants of PSONN were analyzed by the one-way ANOVA test and post-hoc Tukey’s multi-comparison test [60] having a 0.05 significance level.

Table 12. One-way ANOVA results of PSO variants.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Among groups	1318.2	6	219.697	2.3676	0.04639

depicts the results of one-way ANOVA of the testing accuracy of classification data. The significance value in Table 11 is 0.04639 which is less than 0.05, giving evidence that there is a significant difference among all variants of PSONN with a 95% confidence level. According to this, the variants of PSONN are significantly distinct from each other. Figure 56 represents the graph of one-way ANOVA results, which conclude that KN-PSONN significantly outperforms than all other variants of PSONN. Figure 57 represents the results of multi-comparisons of PSONN variants through the post-hoc Tukey’s test. The resultant graph depicts that the KN-PSONN variant is significantly different from all other variants. According to the results in Figure 55, KN-PSONN is proved statistically different from all other approaches of PSONN with a 95% confidence level.

6.2. NN Classifications with DE-Based Initialization Approaches

The proposed approaches, DE-KN, DE-TO, and DE-WE and family of low discrepancy sequences, are extremely suitable for tackling global optimization problems. A comparative study for real-world benchmark datasets problems is tested for the training of the neural network. We performed experiments using seven benchmark datasets (Diabetes, Heart, Wine, Seed, Vertebral, Blood Tissue, and Mammography) exerted from the worldwide famous machine-learning repository of UCI. Training weights are initialized within the interval [−50, 50]. Accuracy of the feed-forward neural network is tested in the form of root mean squared error (RMSE).

Discussion

The multi-layer feed-forward neural network is trained with the back propagation algorithm, standard DE, DE-S, DE-H, and proposed DE-TO, DE-KN, and DE-WE. For this goal, we prepared the multi-layer feed-forward neural network utilizing the process of weight optimization. The performance of the DE, DE-S, DE-H, DE-TO, DE-KN, and DE-WE and state of the art NN algorithms are examined on 10 well-known datasets which have been taken directly from the worldwide UCI repository of machine learning. The features of those informational indexes are given in Table 10. These features include the total units participated against each dataset, the number of total input instances, the dataset nature, and the number of classes against each dataset i.e., binary class problem or multi-class problem. The impact of increasing the number of target classes is independent as the proposed strategy is purely concerned with weight optimization rather feature selection or reducing high dimensionality. The 10-fold cross validation method has been carried out for the training and testing process. The experimental results of algorithms such as with back propagation, standard DE, DE-S, DE-H, DE-WE, DE-TO, and DE-KN are compared with each other on seven well-known real datasets taken from UCI and their performances are evaluated. In

Table 13. Results of 10-fold classification rates of ANN-training methods in 7 datasets for accuracy.

S. No	Data Sets	Type	BPA	DENN	SO-DENN	H-DENN	TO-DENN	KN-DENN	WE-DENN
			Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc
2	Diabetes	2-Class	65.3%	66.1%	68.16%	69.6%	71.30%	67.17%	75.50%
3	Heart	2-Class	68.3%	70.5%	72.5%	71.5%	74.50%	72.56%	76.34%
4	Wine	3-Class	62.17%	64.7%	65.19%	66.20%	66.59%	68.25%	70.51%
5	Seed	3-Class	70.56%	75.16%	75.29%	75.77%	82.13%	86.76%	91.54%
6	Vertebral	2-Class	84.95%	87.13%	89.26%	91.15%	93.64%	90.17%	96.25%
7	Blood Tissue	2-Class	73.47%	76.23%	74.16%	72..21%	84.76%	81.34%	86.45%
9	Memo Graphy	2-Class	71.26%	74.39%	68.37%	82.45%	86.17%	96.66%	99.21%

, the simulation results show that training of neural networks with the DE-H algorithm outperforms in accuracy and is capable of providing the good classification accuracy than the other traditional approaches. The DE-H algorithm may be used effectively for data classification and statistical problems in the future as well. Figure 58 represents the accuracy graph for seven datasets.

To prove the experimental results statistically, the testing accuracy of classification datasets were loaded to the software RStudio (1.2.5001 version). The classification results of seven approaches of DE were tested along with the one-way ANOVA statistical test and post-hoc Tukey’s pair-wise comparison statistical test [60] using significance level 0.05. The findings of the classification dataset with one-way ANOVA are illustrated in

Table 14. One-way ANOVA results of DE variants.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Among groups	1180	6	196.672	2.8453	0.02043

, where the significance = 0.02043 is less than the above-mentioned threshold of significance level. The findings in Table 14 prove that there are significant dissimilarities in all variants of DE with a 95% confidence level. Figure 59 demonstrates the graph of one-way ANOVA which gives the evidence that H-DE is significantly better than other approaches of DE. Figure 60 models the findings of pairwise comparisons of DE approaches with the post-hoc Tukey’s statistical test. The simulated graph describes that the H-DE approach is statistically significant dissimilar as compared to other approaches of DE having a 95% confidence level.

6.3. NN Classifications with BA-Based Initialization Approaches

The multi-layer feed forward neural network is trained with the back propagation algorithm, standard BA, BA-SO, BA-H, and proposed BA-TO, BA-KN, and BA-WE. The comparison of these training approaches is tested on real classification problem datasets taken from the UCI repository. The cross validation method used to compare the performances of different classification techniques. In the paper, the k-fold cross validation method used for comparison of classification performances for the training of the neural network with back propagation, standard BA, BA-SO, BA-H, and proposed BA-TO, BAKN, and BAWE is used. The k-fold cross validation method was proposed and used in the experimental with value k = 10. The dataset is divided into 10 chunks where each chunk of data contains the same proportion of each class of dataset. One chunk is used as testing while nine chunks are used as training phase. The experimental results of algorithms such as with back propagation, standard BA, BA-SO, BA-H, and proposed BA-TO, BA-KN, and BA-WE are compared with each other on seven well-known real datasets taken from UCI and their performances are evaluated. In

Table 15. Results of 10-fold classification rates of ANN-training methods in for 7 datasets for accuracy.

S. No	Data Sets	Type	BPA	BANN	SO-BANN	H-BANN	TO-BANN	KN-BANN	WE-BANN
			Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc	Ts. Acc
1	Diabetes	2-Class	65.31%	66.23%	67.40%	67.28%	69.62%	71.39%	72.68%
2	Heart	2-Class	68.34%	69.39%	69.11%	69.65%	70.12%	73.19%	72.47%
3	Wine	3-Class	62.17%	63.7%	63.22%	65.53%	67.33%	69.27%	69.08%
4	Seed	3-Class	70.56%	72.29%	72.97%	74.41%	77.76%	84.53%	81.54%
5	Vertebral	2-Class	84.95%	86.47%	86.39%	89.72%	90.11%	92.38%	94.19%
6	Blood Tissue	2-Class	73.47%	75.28%	75.23%	72.21%	84.76%	81.34%	83.19%
7	Memo Graphy	2-Class	71.26%	73.17%	71.29%	74.71%	79.23%	91.32%	94.34%

, the simulation results show that training of the neural network with the BA-KN algorithm outperforms in accuracy and is capable of providing good classification accuracy than the other traditional approaches. The BA-WE algorithm may be used effectively for data classification and statistical problem in the future as well. Figure 61 represents the accuracy graph for seven datasets.

For giving the evidence of simulation results, the results of seven variants of BA initialization obtained from classification were examined by the statistical tests such as one-way ANOVA and post-hoc Tukey tests [60] for pair-wise likeliness (pair-wise comparisons) under the condition of 0.05 significance level. The outcomes of the one-way ANOVA test are presented in

Table 16. One-way ANOVA results of BA variants.

Parameter	Relation	Sum of Squares	df	Mean Square	F	Significance
Testing Accuracy	Among groups	845.8	6	140.967	2.5113	0.03623

, which show the significance level less than 0.05 is 0.03623. The outcomes of Table 16 revealed that there is significant divergence in all initialization variants of BA with a 95% confidence level. Figure 62 displays the graph of one-way ANOVA which gives proof that KN-BANN is significantly superior to the other initialization variants of BANN. Figure 63 displays the outcomes of pair-wise likeliness comparisons (pair-wise comparisons) of initialization variants of BANN by using post-hoc Tukey’s statistical test. The plotted graph shows that the KN-BANN initialization variant is statistically divergent than other initialization variants of BANN having a 95% confidence level.

7. Conclusions

This paper introduces the new WELL sequence, Knuth sequence, and Torus sequence pseudorandom initialization strategies that are used to initialize the population in PSO, BA, and DE algorithms. Using the low discrepancy sequence family, the theoretical validation of the suggested methods is assessed on a robust suite of benchmark test functions and artificial neural network learning. The results of the simulation show that the use of the low discrepancy sequence family preserves the swarm’s diversity, increases the pace of convergence, and identifies a better swarm area. The suggested low discrepancy sequence families contain wider diversity and improved local searchability. The experimental findings indicate that KN-PSO, BA-KN, and DE-H have excellent convergence precision and improved avoidance of local optima. The proposed methods are contrasted with both a random distribution family of low discrepancy sequence approaches and traditional algorithms for PSO, BA, and DE, producing better performance. According to our analysis, an inference can be drawn that the quasi-random sequence for all population-based algorithms is substantially stronger and more feasible. Our goal is to work on higher-dimensional problems and constrained optimization problems for future perspectives. Moreover, we have not improved other additional algorithm operators such as mutations in this study. However, the results of such operators on low discrepancy sequences would be fascinating to examine. The main goal of this research is to extend to other stochastic meta-heuristic algorithms that establish the future directions of our work.