Refining the Eel and Grouper Optimizer with Intelligent Modifications for Global Optimization

Abstract

Global optimization is used in many practical and scientific problems. For this reason, various computational techniques have been developed. Particularly important are the evolutionary techniques, which simulate natural phenomena with the aim of detecting the global minimum in complex problems. A new evolutionary method is the Eel and Grouper Optimization (EGO) algorithm, inspired by the symbiotic relationship and foraging strategy of eels and groupers in marine ecosystems. In the present work, a series of improvements are proposed that aim both at the efficiency of the algorithm to discover the total minimum of multidimensional functions and at the reduction in the required execution time through the effective reduction in the number of functional evaluations. These modifications include the incorporation of a stochastic termination technique as well as an improvement sampling technique. The proposed modifications are tested on multidimensional functions available from the relevant literature and compared with other evolutionary methods.

1. Introduction

The goal of global optimization method is to discover the global minimum of a continuous multidimensional function, and it is defined as

$$x^{\ast }=\arg \underset{x\in S}{min}f\left(x\right)$$

(1)

with S as follows:

$$S=\left[a_1,b_1\right]\times \left[a_2,b_2\right]\times \dots \left[a_n,b_n\right]$$

The function $f\left(x\right)$ is defined as $f:S\to R,S\subset R^n$, and the set S denotes the bounds of x. In recent years, many researchers have published important reviews on global optimization [1,2,3]. Global optimization is a technique of vital importance in many fields of science and applications, as it allows us to find the optimal solution to problems with multiple local solutions. In mathematics [4,5,6,7], it is used to solve complex mathematical problems; in physics [8,9,10], it is used to analyze and improve models that describe natural phenomena; in chemistry [11,12,13], it analyzes and designs molecules and chemical diagnostic tools; and in medicine [14], it analyzes and designs therapeutic strategies and diagnostic tools.

The methods that aim to discover the global minimum has two main categories, deterministic [15,16,17] and stochastic [18,19,20]. In the first category, there are techniques aimed at identifying the total minimum with some certainty, such as interval methods  [21,22], and they are usually distinguished by their complex implementation. The vast majority of global optimization algorithms belong to stochastic methods that have simpler implementation and can also be applied to large-scale problems. Recently, Sergeyev et al. [23] published a systematic comparison between deterministic and stochastic methods for global optimization problems.

An important branch of the stochastic methods are the evolutionary methods, that attempt to mimic a series of natural processes. Among these methods one can find the Differential Evolution method [24,25], Particle Swarm Optimization (PSO) methods [26,27,28], Ant Colony optimization methods [29,30], Genetic algorithms [31,32], the Exponential Distribution Optimizer [33], the Brain Storm Optimization method [34], etc. Additionally, since in recent years there has been an extremely widespread increase in parallel computing units, many research studies have proposed evolutionary methods that exploit modern parallel processing units [35,36,37].

Among the evolutionary techniques, one finds a large group of methods that have been explored intensively in recent years, the so-called swarm intelligence algorithms. These methods [38,39,40] are inspired by the collective behavior of swarms. These algorithms mimic systems in which candidate solutions interact locally and cooperate worldwide to discover the global minimum of any problem. These algorithms are very important tools for dealing with complex optimization problems in many applications.

In addition to the previously mentioned Particle Swarm Optimization and Ant Colony techniques that are also included in swarm intelligence algorithms, other methods that belong to this category are the Fast Bacterial Swarming Algorithm (FBSA) [41], the Fish Swarm Algorithm [42], the Dolphin Swarm Algorithm [43], the Whale Optimization Algorithm (WOA) algorithm [44,45,46,47], the Tunicate Swarm Algorithm [48], the Salp Swarm algorithm (SSA) algorithm [49,50,51,52], the Artificial Bee Colony algorithm [53], etc. These methods simulate a series of complex interactions between biological species [54], such as the following:

1.

Naturalism, where two species can live without affecting each other.

2.

Predation, where one creature dies by feeding another.

3.

Parasitism, where one species can cause harm to another.

4.

In competitive mode, the same or different organizations compete for resources.

5.

Mutualism [55,56,57], when two organisms have a beneficial interaction.

Among swarm intelligence algorithms, one can find the Eel and Grouper (EGO) algorithm, which is inspired by the symbiotic interaction and foraging strategy of eels and groupers in marine ecosystems. Bshary et al. [58] consider that target ingestion, something observed in eels and groupers, is a necessary condition for interspecific cooperative hunting to occur. Intraspecific predation could increase the hunting efficiency of predators by mammals. According to Ali Mohammadzadeh and Seyedali Mirjalili the EGO optimization algorithm [59] generates a set of random answers, then stores the best answers found so far, allocates them to the target point, and changes the answers with them. As the number of iterations increases, the limits of the sine function are changed to enhance the phase of finding the best solution. This method stops the process when the iteration exceeds the maximum number. Because the EGO optimization algorithm generates and boosts a collection of random responses, it has the advantage of increased local optimum discovery and avoidance compared to individual methods. According to Ali Mohammadzadeh and Seyedali Mirjalili, the algorithm’s capabilities extend to NP-hard problems in wireless sensor networks [60], IoT [61], logistics [62], smart agriculture [63], bioinformatics [64], and machine learning [65] in various fields such as programming, image segmentation [66], electrical circuit design [67], feature selection, and 3D path planning in robotics [68].

This paper introduces some modifications to the EGO algorithm in order to improve its efficiency. The proposed amendments are presented below:

• The addition of a sampling technique based on the K-means method  [69,70,71]. The sampling points facilitate finding the global minimum of the function in the most efficient way. Additionally, by applying this method, nearby points are discarded. The initialization of the population in evolutionary techniques is a crucial factor which can push these techniques to more efficiently locate the global minimum, and in this direction, a multitude of research works have been presented in recent years, such as the work of Maaranen et al. [72], where they apply quasi-random sequences to the initial population of a genetic algorithm. Likewise, Paul et al. [73] suggested a method for the initialization of the population of genetic algorithms using a Vari-begin and Vari-diversity (VV) seeding method. Ali et al. proposed a series of initialization methods for the Differential Evolution method [74]. A novel method that initializes the population of evolutionary algorithms using clustering and Cauchy deviates is suggested in the work of Bajer et al. [75]. A systematic review of initialization techniques for evolutionary algorithms can be found in the work of Kazimipour et al. [76].
• Using a termination technique that is developed with random measurements. Each time the algorithm is repeated, the minimum value is recorded. When this remains constant for a predefined number of iterations, the process is terminated. Therefore, the method terminates without wasting execution time in iterations, avoiding unnecessary consumption of computing resources. There are several methods found in the recent bibliography to terminate optimization methods. An overview of methods used to terminate evolutionary algorithms can be found in the work of Jain et al. [77]. Also, Zielinski et al. outlined some stopping rules used particularly in the Differential Evolution method [78]. Recently, Ghoreishi et al. published a literature study concerning various termination criteria on evolutionary algorithms [79]. Moreover, Ravber et al. performed extended research on the impact of the maximum number of iterations on the effectiveness of evolutionary algorithms [80].
• The application of randomness in the definition of the range of the positions of candidate solutions.

The rest of this paper is divided into the following sections: in Section 2, the proposed method is fully described, in Section 3 the experimental results and statistical comparisons are outlined and finally, in Section 4, some conclusions and guidelines for future improvements are discussed.

2. The Proposed Method

The main steps of the proposed algorithm are discussed in this section. Also, the mentioned modifications are fully described.

2.1. The Main Steps of the Algorithm

The EGO optimization algorithm starts by initializing a population consisting of “search agents” that search to find the optimal solution. At each iteration, the position of the “prey” (optimal solution) is calculated. Agent positions are adjusted based on random variables and their distance based on the optimal position. At the end of each iteration, the current solutions are compared, and it is decided whether the algorithm should continue or terminate. The steps of the proposed method are provided in Algorithm 1. Also, the algorithm is presented as a series of steps in the flowchart of Figure 1. Using a flowchart and algorithm simultaneously enhances visual perception and provides a detailed analysis of logic and processes.

Algorithm 1 EGO algorithm

Initialization step. 1. Define as $N_c$ the number of elements in the search agents 2. Define as $N_g$ the maximum number of allowed iterations. 3. Initialize randomly the search agents $x_i,i=1,\dots ,N_c$ in set S. 4. Set $t=0$, the iteration counter. 5. Set $s_r=0$, the starvation rate of the algorithm. 6. Set $m=1$; this parameter influences how the variables $f_1,f_2$ are defined, which in turn affects the calculation of new positions. When $m=2$, it introduces randomness to the range of positions before the update, while in the inactive state, the range remains fixed. Calculation step. 1. While termination criteria are not met do (a) Update variables a and $s_r$: $a=2-2{\displaystyle \frac{t}{N_g}}$ $s_r=100{\displaystyle \frac{t}{N_g}}$ (b) Compute the fitness of each search agents. (c) Sort all solutions according to their fitness values. (d) Set $XP$, the estimated position of the prey. (e) For  $i=1,\dots ,N_c$  do i. Update random variables $r_1,r_2,r_3,r_4,C_1,C_2,b$ : $r_1$ and $r_2$ are random numbers in $[0,1]$ $r_3=(a-2)r_1+2$ $r_4=100r_2$ $C_1=2\ast a\ast r_1-a$ $C_2=2\ast r_1$ $b=a\ast r_2$ Select a random position $v\in \left\{1,\dots ,N_c\right\}$ if $\left(r4\le s_r\right)$ then set $XE=C_{2}XP$ else $XE=position\left[randomindexv\right]$ ii. Create the vector $y=\left[y_{1,}{y}_2,\dots ,y_n\right]$ with the following procedure iii. For  $j=1,\dots ,n$  do A. Set $X_1=e^{b{r}_3}sin\left(2\pi r_3\right)C_1\left|\mathbf{X}\mathbf{E}-\mathbf{X}\mathbf{P}\right|+\mathbf{X}\mathbf{E},$ where $\mathbf{X}\mathbf{E}=C_2{x}_{i,j}$ B. Set $X_2=x_{i,j}+C_1\left|x_{i,j}-\mathbf{X}\mathbf{P}\right|$ C. if $m=1$ then set $f_1=0.8, f_2=0.2$ else set $f_1$ to a random number in $[0,2]$ and $f_2$ to a random number in $[-2,2]$ D. Set $p\in [0,1]$ a random number. E. if $\left(p\le 0.5\right)$ then set $x_{i,j}={\displaystyle \frac{f_1{X}_1+f_2{X}_2}{2}}$ else set $x_{i,j}={\displaystyle \frac{f_2{X}_1+f_1{X}_2}{2}}$ iv. EndFor (f) End For Termination check step (a) Set $t=t+1$ (b) If $t\ge N_g$ terminate. (c) Else (d) Calculate the stopping criterion proposed in the work of Charilogis [81]. In the similarity stopping rule, at every iteration t, the absolute difference between the current located global minimum $f_{min}^{\left(t\right)}$ and the previous best value $f_{min}^{(t-1)}$ is calculated: $${\delta }^t=\left|f_{\min }^{\left(t\right)}-f_{\min }^{(t-1)}\right|$$ (2) The algorithm terminates when ${\delta }^t\le ϵ$ for $N_k$ consecutive iterations, where $ϵ$ is a small positive value. 2. End While

The main modifications introduced by the proposed algorithm are the following:

1.

The members of the population are initialized using a procedure that incorporates the K-means algorithm. This procedure is fully described in Section 2.2. The purpose of this sampling technique is to produce points that are close to the local minima of the objective problem through systematic clustering. This potentially significantly reduces the time required to complete the technique. The samples used in the proposed algorithm are the calculated centers of the K-means algorithm. This sampling technique has also been utilized in some recent papers related to Global Optimization techniques, such as the extended version of the Optimal Foraging algorithm [82] or the new genetic algorithm proposed by Charilogis et al. [83].

2.

The second modification proposed is the stopping rule invoked at every step of the algorithm. This rule measures the similarity between the best fitness values obtained between consecutive iterations of the algorithm. If this difference takes low values for a consecutive number of iterations, then the algorithm may not be able to find a lower value for the global minimum and should stop. This stopping rule has been applied in recent years in various methods such as in the work of Charilogis and Tsoulos, which presented an improved parallel PSO method [84], the work of Kyrou at al., which suggested an improved version of the Giant-Armadillo optimization method [85], or the recent work of Kyrou et al., which proposed an extended version of the Optimal Foraging Algorithm [82]. Of course, this method is general enough for application in any global optimization procedure.

3.

The third modification is the m flag, which controls the randomness in the range of candidate solutions. When this value is set to 2, then the critical parameters $f_1,f_2$ are calculated using random numbers.

The overall procedure that outlines the basic steps of the method and the added modifications is shown in Algorithm 2.

Algorithm 2 The basic steps of the algorithm accompanied with the proposed modifications

1. Initialize the population members using a procedure incorporating the K-means algorithm, shown in Section 2.2. 2. Compute the fitness of each search agent . 3. Sort all solutions according to their fitness. 4. Calculate the similarity stopping rule proposed in [81]. If the termination criteria are not satisfied then go to step 2; Else terminate and return the best solution. End if

2.2. The Used Sampling Procedure

The used sampling procedure incorporated in this work initially generates samples from the objective problem. Then, using the K-means method, only the estimated centers are selected as samples for the proposed algorithm. This technique, due to James MacQueen [71], is one of the best-known clustering algorithms in the broad research community, both in data analysis and in machine learning [86] and pattern recognition [87]. The algorithm aims to divide a data set into K clusters. The K-means algorithm tries to divide the data into groups in such a way that the internal points of each group are as close as possible to each other. At the same time, it tries to place the central points of each group in positions which are as representative as possible of the points of their group. Over the years, a series of variants of this algorithm have been proposed, such as the genetic K-means algorithm [88], the unsupervised K-means algorithm [89], the fixed-centered K-means algorithm [90], etc. A review of K-means clustering algorithms can be found in the work of Oti et al. [91]. Next, the basic steps of the algorithm are provided in Algorithm 3. A flowchart of the K-means procedure is also depicted in Figure 2.

Algorithm 3 K-means algorithm

1. Initialization (a) Set k, the number of clusters. (b) Obtain randomly the initial samples $x_i,i=1,\dots ,N_m$. (c) Set $S{=}_j\left\{\right\}$, from $j=1,\dots ,k$. 2. Repeat (a) For every point $x_i,i=1,\dots ,N_m$ do i. Set $j^{\ast }={\mathrm{argmin}}_{m=1}^k\left\{D\left(x_i,c_m\right)\right\}$. $j^{\ast }$ is the nearest center from $x_i$. ii. Set $S_{j^{\ast }}=S_{j^{\ast }}\cup \left\{x_i\right\}$. (b) End For (c) For every center $c_j,j=1\dots k$ do i. Set $M_j$ the number of samples in $S_j$. ii. Update the center $c_j$ as $$c_j={\displaystyle \frac{1}{M_j}}\sum _{x_i\in S_j}{x}_i$$ (d) End For 3. Terminate when $c_j$ no longer changes. 4. The final samples of the algorithm are the centers $c_j$.

3. Results

This section begins with a detailed description of the functions that were used in the experiments, followed by an analysis of the experiments performed and comparisons with other global optimization techniques.

3.1. Test Functions

The test functions used in the experiments were suggested in a series of relative works [92,93] and they originated in a series of scientific fields. Also, these objective functions were studied in various publications [94,95,96,97,98]. Also, a series of function found in [99] were used as test functions. The used functions were defined as follows:

• Ackley’s function:

  $$f\left(x\right)=-aexp\left(-b\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{x}_i^2}\right)-exp\left({\displaystyle \frac{1}{n}}\sum _{i=1}^{n}cos\left(c{x}_i\right)\right)+a+exp\left(1\right)$$

  with a = 20.0.
• Bf1 (Bohachevsky 1) function:

$$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1\right)-{\displaystyle \frac{4}{10}}cos\left(4\pi x_2\right)+{\displaystyle \frac{7}{10}}$$

• Bf2 (Bohachevsky 2) function:

  $$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1\right)cos\left(4\pi x_2\right)+{\displaystyle \frac{3}{10}}$$
• Bf3 (Bohachevsky 3) function:

  $$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1+4\pi x_2\right)+{\displaystyle \frac{3}{10}}$$
• Branin function:

  $$f\left(x\right)={\left(x_2-{\displaystyle \frac{5.1}{4{\pi }^2}}{x}_1^2+{\displaystyle \frac{5}{\pi }}{x}_1-6\right)}^2+10\left(1-{\displaystyle \frac{1}{8\pi }}\right)cos\left(x_1\right)+10$$

  with $-5\le x_1\le 10,0\le x_2\le 15$.
• Camel function:

  $$f\left(x\right)=4x_1^2-2.1x_1^4+{\displaystyle \frac{1}{3}}{x}_1^6+x_1{x}_2-4x_2^2+4x_2^4,x\in {[-5,5]}^2$$
• Easom function:

  $$f\left(x\right)=-cos\left(x_1\right)cos\left(x_2\right)exp\left({\left(x_2-\pi \right)}^2-{\left(x_1-\pi \right)}^2\right)$$

  with $x\in {[-100,100]}^2.$
• Equal maxima function, defined as:

  $$f\left(x\right)={sin}^6\left(5\pi x\right)$$
• Exponential function, with the following definition:

  $$f\left(x\right)=-exp\left(-0.5\sum _{i=1}^n{x}_i^2\right),-1\le x_i\le 1$$
  The values $n=4,8,16,32$ were used in the conducted experiments.
• F9 test function:

  $$f\left(x\right)=-\sum _{i=1}^n\left(10+9cos\left(2\pi k_i{x}_i\right)\right)$$

  with $x\in {[0,1]}^n.$
• Extended F10 function:

  $$f\left(x\right)=\sum _{i=1}^n\left(x_i^2-10cos\left(2\pi x_i\right)\right)$$
• F14 function:

  $$f\left(x\right)={\left({\displaystyle \frac{1}{500}}+\sum _{j=1}^{25}{\displaystyle \frac{1}{j+{\sum }_{i=1}^2{\left(x_i-a_{ij}\right)}^6}}\right)}^{-1}$$
• F15 function:

  $$f\left(x\right)=\sum _{i=1}^{11}{\left(a_i-{\displaystyle \frac{x_1\left(b_i+b_i{x}_2\right)}{b_i^2+b_i{x}_3+x_4}}\right)}^2$$
• F17 function:

  $$\begin{array}{cc}\hfill f\left(x\right)& =\left(1+{\left(x_1+x_2+1\right)}^2\times \left(19-14x_1+3x_1^2-14x_2+6x_1{x}_2+3x_2^2\right)\right)\times \hfill \\ \hfill & \left(30+{\left(2x_1-3x_2\right)}^2\times \left(18-32x_1+12x_1^2+48x_2-36x_1{x}_2+27x_2^2\right)\right)\hfill \end{array}$$
• Five-uneven-peak trap function:

  $$f\left(x\right)=\left\{\begin{array}{cc}80(2.5-x)\hfill & 0\le x<2.5\hfill \\ 64(x-2.5)\hfill & 2.5\le x<5.0\hfill \\ 64(7.5-x)\hfill & 5.0\le x<7.5\hfill \\ 28(x-7.5)\hfill & 7.5\le x<12.5\hfill \\ 28(17.5-x)\hfill & 12.5\le x<17.5\hfill \\ 32(x-17.5)\hfill & 17.5\le x<22.5\hfill \\ 32(27.5-x)\hfill & 22.5\le x<27.5\hfill \\ 80(x-27.5)\hfill & 27.5\le x\le 30\hfill \end{array}\right.$$
• Himmelblau’s function:

  $$f\left(x\right)=200-{\left(x_1^2+x_2-11\right)}^2-{\left(x_1+x_2^2-7\right)}^2$$

  with $x\in {[-6,6]}^2$.
• Griewank2 function:

  $$f\left(x\right)=1+{\displaystyle \frac{1}{200}}\sum _{i=1}^2{x}_i^2-\prod _{i=1}^2{\displaystyle \frac{cos\left(x_i\right)}{\sqrt{\left(i\right)}}},x\in {[-100,100]}^2$$
• Griewank10 function, given by the equation

  $$f\left(x\right)=\sum _{i=1}^n{\displaystyle \frac{x_i^2}{4000}}-\prod _{i=1}^{n}cos\left({\displaystyle \frac{x_i}{\sqrt{i}}}\right)+1$$

  with $n=10$.
• Gkls function [100]: The function $f\left(x\right)=Gkls(x,n,w)$ is a test function proposed in [100] with w local minima. The values $n=2,3$, and $w=50$ were used in the conducted experiments.
• Goldstein and Price’s function

  $$\begin{array}{ccc}\hfill f\left(x\right)& =& \left[1+{\left(x_1+x_2+1\right)}^2\right.\hfill \\ & & \left(19-14x_1+3x_1^2-14x_2+6x_1{x}_2+3x_2^2\right)]\times \hfill \\ & & [30+{\left(2x_1-3x_2\right)}^2\hfill \\ & & \left(18-32x_1+12x_1^2+48x_2-36x_1{x}_2+27x_2^2\right)]\hfill \end{array}$$

  with $x\in {[-2,2]}^2$.
• Hansen’s function: $f\left(x\right)={\sum }_{i=1}^{5}icos\left[(i-1)x_1+i\right]{\sum }_{j=1}^{5}jcos\left[(j+1)x_2+j\right]$, $x\in {[-10,10]}^2$ .
• Hartman 3 function:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^3{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^3$ and $a=\left(\begin{array}{ccc}3& 10& 30\\ 0.1& 10& 35\\ 3& 10& 30\\ 0.1& 10& 35\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{ccc}0.3689& 0.117& 0.2673\\ 0.4699& 0.4387& 0.747\\ 0.1091& 0.8732& 0.5547\\ 0.03815& 0.5743& 0.8828\end{array}\right)$$
• Hartman 6 function:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^6{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^6$ and $a=\left(\begin{array}{cccccc}10& 3& 17& 3.5& 1.7& 8\\ 0.05& 10& 17& 0.1& 8& 14\\ 3& 3.5& 1.7& 10& 17& 8\\ 17& 8& 0.05& 10& 0.1& 14\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{cccccc}0.1312& 0.1696& 0.5569& 0.0124& 0.8283& 0.5886\\ 0.2329& 0.4135& 0.8307& 0.3736& 0.1004& 0.9991\\ 0.2348& 0.1451& 0.3522& 0.2883& 0.3047& 0.6650\\ 0.4047& 0.8828& 0.8732& 0.5743& 0.1091& 0.0381\end{array}\right)$$
• Potential function: this function represents the energy of a molecular conformation of N atoms. The interaction of these atoms is determined by the Lennard–Jones potential [101]. The definition of this potential is:

  $$V_{LJ}\left(r\right)=4ϵ\left[{\left({\displaystyle \frac{\sigma }{r}}\right)}^{12}-{\left({\displaystyle \frac{\sigma }{r}}\right)}^6\right]$$
  For the conducted experiments, the values $N=3,5$ were used.
• Rastrigin’s function.

  $$f\left(x\right)=x_1^2+x_2^2-cos\left(18x_1\right)-cos\left(18x_2\right),x\in {[-1,1]}^2$$
• Rosenbrock function.

  $$f\left(x\right)=\sum _{i=1}^{n-1}\left(100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2\right),-30\le x_i\le 30.$$
  The values $n=4,8,16$ were incorporated in the conducted experiments.
• Shekel 5 function.

$$f\left(x\right)=-\sum _{i=1}^5{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\end{array}\right)$

• Shekel 7 function.

$$f\left(x\right)=-\sum _{i=1}^7{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 3& 5& 3\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\end{array}\right)$.

• Shekel 10 function.

$$f\left(x\right)=-\sum _{i=1}^{10}{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 5& 3& 3\\ 8& 1& 8& 1\\ 6& 2& 6& 2\\ 7& 3.6& 7& 3.6\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\\ 0.7\\ 0.5\\ 0.6\end{array}\right)$.

• Sinusoidal function, defined as:

  $$f\left(x\right)=-\left(2.5\prod _{i=1}^{n}sin\left(x_i-z\right)+\prod _{i=1}^{n}sin\left(5\left(x_i-z\right)\right)\right),0\le x_i\le \pi .$$
  The values $n=4,8,16$ were incorporated in the conducted experiments.
• Schaffer’s function:

  $$f\left(x\right)=\sum _{i=1}^n{\left(\sum _{j=1}^i{x}_j\right)}^2$$
• Schwefel221 function:

  $$f\left(x\right)=418.9829n+\sum _{i=1}^n-x_{i}sin\left(\sqrt{\left|x_i\right|}\right)$$
• Schwefel222 function:

  $$f\left(x\right)=\sum _{i=1}^n\left|x_i\right|+\prod _{i=1}^n\left|x_i\right|$$
• Shubert function:

  $$f\left(x\right)=-\prod _{i=1}^n\sum _{j=1}^{5}jcos\left((j+1)x_i+j\right)$$

  with $x\in {[-10,10]}^n$
• Sphere function:

  $$f\left(x\right)=\sum _{i=1}^n{x}_i^2$$
• Test2N function:

  $$f\left(x\right)={\displaystyle \frac{1}{2}}\sum _{i=1}^n{x}_i^4-16x_i^2+5x_i,x_i\in [-5,5].$$
  The values $n=4,5,6,7$ were incorporated for the conducted experiments.
• Test30N function:

  $$f\left(x\right)={\displaystyle \frac{1}{10}}{sin}^2\left(3\pi x_1\right)\sum _{i=2}^{n-1}\left({\left(x_i-1\right)}^2\left(1+{sin}^2\left(3\pi x_{i+1}\right)\right)\right)+{\left(x_n-1\right)}^2\left(1+{sin}^2\left(2\pi x_n\right)\right)$$
  For the conducted experiments, the values $n=3,4$ were used.
• Uneven decreasing maxima function:

  $$f\left(x\right)=exp\left(-2log\left(2\right){\left({\displaystyle \frac{x-0.08}{0.854}}\right)}^2\right){sin}^6\left(5\pi \left(x^{{\displaystyle \frac{3}{4}}}-0.05\right)\right)$$
• Vincent’s function:

  $$f\left(x\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}sin\left(10log\left(x_i\right)\right)$$

  with $x\in {[0.25,10]}^n$.

Also, the dimension for each problem, as used in the experiments, is shown in

Table 1. The dimension for every function used in the experiments.

Function	Dimension
Ackley	$n=2$
Bf1	$n=2$
Bf2	$n=2$
Bf3	$n=2$
Branin	$n=2$
Camel	$n=2$
Easom	$n=2$
EQUAL_MAXIMA	$n=1$
EXP	$n=4,8,16,32$
EXTENDEDF10	$n=2$
FIVE_UNEVEN	$n=1$
F9	$n=2$
F14	$n=2$
F15	$n=4$
F17	$n=2$
HIMMELBLAU	$n=2$
GKLS	$n=2,3$
GRIEWANK2	$n=2$
GRIEWANK10	$n=10$
HANSEN	$n=2$
HARTMAN3	$n=3$
HARTMAN6	$n=6$
POTENTIAL	$n=9,15$
RASTRIGIN	$n=2$
ROSENBROCK	$n=4,8,16$
SHEKEL5	$n=4$
SHEKEL7	$n=4$
SHEKEL10	$n=4$
SHUBERT	$n=2,4,8$
SCHWEFEL221	$n=2$
SCHWEFEL222	$n=2$
SPHERE	$n=2$
TEST2N	$n=4,5,6,7$
SINU	$n=4,8,16$
TEST30N	$n=3,4$
UNEVEN_MAXIMA	$n=1$
VINCENT	$n=2,4,8$

.

3.2. Experimental Results

A series of global optimization methods were applied to the mentioned test functions. All experiments were performed 30 times using different seeds for the random generator and the average value of function calls was measured. The used software was coded in ANSI C++ using the freely available OPTIMUS optimization environment v1.0, available from https://github.com/itsoulos/OPTIMUS (accessed on 27 September 2024). The experiments were executed on a Debian Linux system with an AMD Ryzen 5950X processor and 128 GB of RAM. In all cases, the BFGS [102] local optimization method was used at the end of each global optimization technique to ensure that an actual minimum was discovered by the global optimization method. The values for all experimental parameters are shown in

Table 2. The values used for every parameter of the used algorithms.

PARAMETER	MEANING	VALUE
$N_c$	Number of chromosomes/particles	200
$N_g$	Maximum number of allowed iterations	200
$N_m$	Number of samples for the K-means	$10\times N_c$
$N_k$	Number of maximum iterations for the stopping rule	5
$p_s$	Selection rate of the genetic algorithm	0.1
$p_m$	Mutation rate of the genetic algorithm	0.05

.

The experimental results for the test functions and a series of optimization methods are shown in

Table 3. Experimental results using the incorporated optimization methods. The numbers in parentheses denote the standard deviation for the number of function calls.

Function	Genetic	PSO	DE	EGO	EEGO
ACKLEY	6749 (869)	6885 (1108)	10,220 (1342)	8714 (1009)	4199 (768)
BF1	4007 (308)	4142 (397)	8268 (299)	4762 (379)	3228 (356)
BF2	3794 (350)	3752 (302)	7913 (320)	4299 (325)	2815 (310)
BF3	3480 (266)	3306 (261)	10270 (294)	3747 (303)	2501 (218)
BRANIN	2376 (136)	2548 (142)	4101 (559)	2659 (150)	1684 (180)
CAMEL	2869 (195)	2933 (180)	5609 (530)	3317 (229)	2262 (295)
EASOM	1958 (67)	1982 (92)	2978 (344)	2235 (196)	1334 (133)
EQUAL_MAXIMA	2651 (131)	1499 (176)	2374 (117)	2013 (157)	1286 (121)
EXP4	2946 (196)	3404 (201)	5166 (430)	3392 (313)	2166 (243)
EXP8	3120 (187)	3585 (212)	5895 (615)	3347 (288)	2802 (304)
EXP16	3250 (234)	3735 (206)	6498 (784)	3345 (233)	3279 (416)
EXP32	3561 (233)	3902 (233)	7606 (475)	3332 (250)	3430 (401)
EXTENDEDF10	4862 (959)	3653 (329)	5728 (976) (0.87)	4737 (428)	2609 (321)
FIVE_UNEVEN	3412 (422) (0.67)	3913 (378) (0.87)	4042 (391) (0.14)	5006 (519) (0.97)	3849 (388) (0.90)
F9	2604 (197)	1888 (306)	2271 (364)	2748 (333)	1439 (229)
F14	6686 (1503)	5498 (550)	5279 (1988) (0.63)	9228 (3725) (0.94)	6063 (1835)
F15	4373 (624)	6696 (856)	5874 (1880) (0.80)	7342 (1441)	4397 (1031)
F17	3667 (267)	3805 (333)	10441 (1435)	4057 (339)	2766 (368)
HIMMELBLAU	2481 (27)	1013 (34)	6636 (282)	1718 (84)	1119 (81)
GKLS250	2280 (184)	2411 (158)	3834 (416)	3332 (203)	1603 (150)
GKLS350	2613 (269)	2234 (225)	3919 (469)	2493 (233)	1298 (178)
GOLDSTEIN	3687 (278)	3865 (356)	6781 (483)	4015 (323)	2784 (290)
GRIEWANK2	4501 (918)	3076 (218) (0.73)	7429 (1472)	4682 (586)	2589 (532) (0.96)
GRIEWANK10	6410 (1264) (0.97)	8006 (732)	18490 (1716)	8772 (1138)	7435 (900)
HANSEN	3210 (458)	2856 (208)	4185 (627)	3789 (864)	2484 (417)
HARTMAN3	2752 (188)	3140 (215)	5190 (294)	3078 (267)	1793 (231)
HARTMAN6	3219 (217)	3710 (235)	5968 (548)	3583 (309)	2478 (282)
POTENTIAL3	4352 (461)	4865 (468)	6118 (1047)	6027 (793)	4081 (391)
POTENTIAL5	7705 (892)	9183 (1180)	9119 (570)	9968 (1171)	8886 (1146)
RASTRIGIN	4107 (729)	3477 (348)	6216 (428)	4201 (401)	2304 (302)
ROSENBROCK4	3679 (393)	6372 (549)	8452 (643)	6137 (720)	4019 (324)
ROSENBROCK8	5270 (514)	8284 (849)	11,530 (1632)	8569 (872)	6801 (633)
ROSENBROCK16	8509 (1026)	11,872 (1018)	17,432 (1738)	11,777 (1268)	11,996 (1331)
SHEKEL5	3325 (227)	4259 (305)	6662 (974)	3948 (340)	2495 (310)
SHEKEL7	3360 (283)	4241 (300)	6967 (1035)	4043 (379)	2432 (240)
SHEKEL10	3488 (240)	4237 (268)	6757 (897)	3932 (355)	2516 (326)
SHUBERT2	3567 (413)	2123 (188)	3526 (885)	3622 (446)	2300 (527)
SHUBERT4	3358 (380)	1823 (166)	3067 (699)	3593 (674)	1967 (323) (0.97)
SHUBERT8	3569 (357)	2348 (203)	3120 (452) (0.94)	2862 (383)	2267 (296)
SCHAFFER	18,787 (3105)	15,176 (2401)	6315 (1548)	28,679 (6267)	23,531 (5904)
SCHWEFEL221	2667 (416)	2529 (163)	5415 (1161)	3426 (787)	2203 (478)
SCHWEFEL222	33,725 (4809)	42,898 (6978)	12,200 (1737)	51,654 (7150)	38,876 (5659)
SPHERE	1588 (23)	1521 (12)	3503 (199)	1642 (38)	1162 (84)
TEST2N4	3331 (392)	3437 (223)	6396 (999)	3695 (411)	2277 (464)
TEST2N5	4000 (815)	3683 (287)	6271 (1017)	4234 (636)	2734 (479) (0.96)
TEST2N6	4312 (901) (0.93)	3781 (241)	5410 (822) (0.93)	4599 (589)	2905 (832) (0.86)
TEST2N7	4775 (905) (0.90)	4060 (312)	7074 (1523) (0.97)	5146 (767)	3559 (763) (0.73)
SINU4	2991 (298)	3504 (231)	5953 (1509)	3478 (463)	2005 (332)
SINU8	3442 (293)	4213 (309)	6973 (1637)	4420 (560)	3158 (467)
SINU16	4320 (458)	5019 (312)	6979 (634)	7033 (1017)	5891 (553)
TEST30N3	3211 (1207)	4610 (1615)	6168 (1297)	3971 (1348)	2362 (884)
TEST30N4	3679 (1435)	4629 (1708)	7006 (2745)	4908 (1333)	2978 (1719)
UNEVEN_MAXIMA	2969 (283)	2729 (292)	2393 (319)	2972 (288)	1560 (208)
VINCENT2	12,779 (563)	1797 (140) (0.87)	6216 (1152)	2094 (148)	1834 (388) (0.90)
VINCENT4	19,385 (1179)	1830 (190) (0.67)	4691 (1019) (0.83)	2674 (244)	2697 (1443) (0.64)
VINCENT8	19,882 (2548)	2717 (256)	4417 (1118) (0.77)	3423 (358)	4368 (2253) (0.77)
SUM	297,650	268,654	288,413	320,469	231,856

, where the following applies to this table:

• The column function represents the used test function.
• The column genetic represents the usage of a genetic algorithm [31,32] into the test function. This genetic algorithm was equipped with $N_c$ chromosomes, and the maximum number of generations was set to $N_g$. The modified version of Tsoulos [103] was used as a genetic algorithm here.
• The column PSO represents the incorporation of a Particle Swarm Optimizer [27,28] into each test function. This algorithm had $N_c$ particles, and the maximum number of allowed iterations was set to $N_g$. For the conducted experiments, the improved PSO method proposed by Charilogis and Tsoulos was used [81].
• The column DE refers to the Differential Evolution method [24,25].
• The column EGO represents the initial method without the modifications suggested in this work.
• The column EEGO represents the usage of the proposed method. The corresponding settings are shown in Table 2.
• The row sum is used to measure the total function calls for all problems.
• If a method failed to find the global minimum over all runs, this was noted in the corresponding table with a percentage enclosed in parentheses next to the average function calls.

The same termination method, as detailed in Section 2.1, was used in all global optimization techniques in order to be able to evaluate all techniques fairly and with the same rules. In Figure 3, we present the total function calls of every optimization method graphically. The proposed method had excellent results compared to the other optimization techniques according to the experiments conducted. As we can observe, it had the least number of calls among all other techniques.

Furthermore, as the experimental results clearly indicate, the modified version outperformed the original EGO method in terms of average and function calls and this is depicted in Figure 4, which outlines box plots for the mentioned methods.

A box plot between all the used methods is depicted in Figure 5.

Moreover, a statistical comparison of all used methods is outlined in Figure 6.

According to Table 3 and Figure 6, the EEGO method proved to be more efficient, as it consistently required fewer function calls compared to the DE, PSO, genetic, and EGO methods, and these differences were statistically significant (p < 0.05). For example, in the Ackley function, EEGO required 4199 calls, while DE required 10.220, PSO 6885, and the genetic algorithm 6749, with a p-value < 0.05, showing that EEGO is significantly more efficient. Similar examples are observed in the BF1 function, where EEGO had 3228 calls, while DE required 8268 (p < 0.05), and in the Branin function, where EEGO required 1684 calls, compared to 4101 for DE (p < 0.05). Overall, EEGO showed the best performance in most cases, with statistically significant differences in function calls, as it effectively reduced the number of calls compared to the other methods. While PSO and the genetic algorithm performed better than DE in some instances, they still lagged significantly behind EEGO. The p-values confirmed that these differences were statistically significant, indicating that EEGO was the most efficient method overall.

One more experiment was performed with the ultimate goal of measuring the importance of K-means sampling in the proposed method. The results for this experiment are outlined in

Table 4. A series of sampling techniques was used in the proposed method. The number in parentheses denotes the standard deviation for the number of function calls.

Function	Uniform	Triangular	Maxwell	K-means
ACKLEY	6118 (736)	5912 (741)	5986 (845)	4199 (768)
BF1	4513 (423)	4318 (368)	4055 (346)	3228 (356)
BF2	3959 (333)	3879 (363)	3587 (339)	2815 (310)
BF3	3506 (286)	3344 (254)	3129 (298)	2501 (218)
BRANIN	2282 (162)	2131 (174)	2066 (167)	1684 (180)
CAMEL	3156 (251)	2919 (244)	2848 (261)	2262 (295)
EASOM	1756 (138)	1650 (140)	1321 (106)	1334 (133)
EQUAL_MAXIMA	1445 (130)	1293 (115)	1165 (139)	1286 (121)
EXP4	3438 (335)	3273 (232)	3194 (330)	2166 (243)
EXP8	3432 (284)	3387 (332)	3152 (266)	2802 (304)
EXP16	3369 (349)	3326 (329)	3291 (224)	3279 (416)
EXP32	3216 (274)	3225 (207)	3344 (374)	3430 (401)
EXTENDEDF10	4304 (683)	3913 (596)	3718 (673)	2609 (321)
FIVE_UNEVEN	4685 (427)	4330 (474)	4358 (544)	3849 (388)
F9	1958 (168)	1681 (188)	1878 (186)	1439 (229)
F14	7552 (1377)	7573 (2049)	6148 (724)	6063 (1835)
F15	6806 (1065)	6466 (722)	6543 (974)	4397 (1031)
F17	3805 (326)	3700 (404)	3543 (236)	2766 (368)
HIMMELBLAU	1333 (91)	1173 (72)	1114 (102)	1119 (81)
GKLS250	2268 (177)	2023 (170)	1778 (208)	1603 (150)
GKLS350	2151 (291)	1841 (190)	2069 (893)	1298 (178)
GOLDSTEIN	3855 (291)	3731 (335)	3530 (309)	2784 (290)
GRIEWANK2	4310 (1205)	4510 (1310)	4035 (995)	2589 (532)
GRIEWANK10	8640 (1106)	8773 (1265)	8232 (911)	7435 (900)
HANSEN	3329 (653)	3071 (466)	2734 (521)	2484 (417)
HARTMAN3	2849 (240)	2673 (246)	2678 (317)	1793 (231)
HARTMAN6	3456 (344)	3249 (330)	3119 (299)	2478 (282)
POTENTIAL3	4554 (511)	5095 (469)	3928 (509)	4081 (391)
POTENTIAL5	8356 (702)	10,032 (1078)	7504 (996)	8886 (1146)
RASTRIGIN	3310 (414)	3187 (336)	2751 (600)	2304 (302)
ROSENBROCK4	6566 (719)	6353 (742)	5588 (599)	4019 (324)
ROSENBROCK8	8379 (864)	8717 (882)	7783 (782)	6801 (633)
ROSENBROCK16	11,921 (1389)	12,471 (1025)	11,677 (1212)	11,996 (1331)
SHEKEL5	3946 (333)	3731 (398)	3859 (460)	2495 (310)
SHEKEL7	3990 (361)	3646 (442)	3944 (326)	2432 (240)
SHEKEL10	3836 (316)	3630 (385)	3694 (379)	2516 (326)
SHUBERT2	3288 (562)	3212 (728)	2631 (373)	2300 (527)
SHUBERT4	3116 (548)	2919 (514)	2499 (422)	1967 (323)
SHUBERT8	2815 (500)	2810 (513)	2337 (296)	2267 (296)
SCHAFFER	55,131 (10,062)	40,715 (9866)	60,717 (9950)	23531 (5904)
SCHWEFEL221	2724 (406)	2901 (547)	2799 (505)	2203 (478)
SCHWEFEL222	53,118 (7011)	54,593 (8763)	55,354 (7330)	38,876 (5659)
SPHERE	1346 (68)	1188 (72)	1084 (82)	1162 (84)
TEST2N4	3345 (416)	3233 (401)	2867 (253)	2277 (464)
TEST2N5	3937 (757)	3742 (584)	3094 (278)	2734 (479)
TEST2N6	4008 (718)	4473 (1060)	3266 (297)	2905 (832)
TEST2N7	4545 (1169)	4612 (1046)	3549 (421)	3559 (763)
SINU4	3128 (410)	2879 (240)	3559 (750)	2005 (332)
SINU8	4126 (339)	3767 (410)	5637 (964)	3158 (467)
SINU16	6774 (1166)	5977 (601)	7739 (1791)	5891 (553)
TEST30N3	3704 (1289)	3384 (1083)	3175 (1012)	2362 (884)
TEST30N4	4262 (1805)	4327 (1838)	3491 (920)	2978 (1719)
UNEVEN_MAXIMA	1877 (236)	1810 (239)	1666 (189)	1560 (208)
VINCENT2	1598 (119)	1482 (133)	1849 (121)	1834 (388)
VINCENT4	2471 (229)	2282 (167)	2296 (206)	2697 (1443)
VINCENT8	3074 (524)	2797 (201)	2883 (423)	4368 (2253)
SUM	324,736	307,329	315,835	231,856

and the following sampling methods were used:

1.

The column uniform represents the usage of uniform sampling in the current method.

2.

The column triangular stands for the usage of the triangular distribution [104] for sampling.

3.

The column Maxwell represents for the application of the Maxwell distribution [105] to produce initial samples for the used method.

4.

The column K-means represents the usage of the method described in Section 2.2 to produce initial samples for the used method.

The initial distributions play a critical role in a wide range of applications, including optimization, statistical analysis, and machine learning. Table 4 presents the proposed distribution alongside other established distributions. The uniform distribution is widely used due to its ability to evenly cover the search space, making it suitable for initializing optimization algorithms [106]. The triangular distribution is applied in scenarios where there is knowledge of the bounds and the most probable value of a phenomenon, making it useful in risk management models [107]. The Maxwell distribution, although originating from physics, finds applications in simulating communication networks, where data transfer speeds can be modeled as random variables [108]. Finally, the K-means method is used for data clustering, with K-means++ initialization offering improved performance compared to random distributions, particularly in high-dimensional problems [109]. As observed in Table 4, the choice of an appropriate initial distribution can significantly affect the performance of the algorithms that utilize them.

In the scatter plot in Figure 7, the critical parameter “p” was found to be very small, leading to the rejection of the null hypothesis and indicating that the experimental results were highly significant.

In addition, in order to investigate the impact that the choice of sampling method had on the optimization method, an additional experiment was performed, in which the execution time of the proposed method was recorded with different sampling techniques for the ELP function, whose dimension was varied between 5 and 50. This function is defined as:

$$f\left(x\right)=\sum _{i=1}^n{\left({10}^6\right)}^{{\displaystyle \frac{i-1}{n-1}}}{x}_i^2$$

where the parameter n defines the dimension of the function. The results from this experiment are graphically outlined in Figure 8.

The proposed sampling method significantly increased the required execution time, since an iterative process was required before starting the algorithm. The next sampling procedure in required execution time appeared to be the Maxwell sampling, but it was not significantly different compared to uniform sampling.

4. Conclusions

The article proposed some modifications to the EEGO optimization method, aimed to improve the overall performance and reduce the needed function calls to discover the global minimum. The first modification concerned the application of a sampling technique based on the K-means method. This technique allowed us to significantly minimize the number of function calls to find the global minimum and further improved the accuracy with which it is located. In particular, the application of the K-means method accelerated the finding of a solution, as it more efficiently located the points of interest in the search space and led to the fastest convergence to the global minimum. Compared to other methods based on random distributions, the proposed technique proved its superiority, especially in multidimensional and complex functions.

The second proposed amendment concerned the termination rule based on the similarity of solutions during iterations. The main purpose of this rule was to stop the optimization process when the iterated solutions were too close to each other, thus preventing pointless iterations that did not provide any significant improvement. It therefore avoided wasting computing time in cases where the process was already very close to the desired result. The use of the termination rule significantly improved the efficiency of the algorithm.

Furthermore, one more improvement was suggested in this research paper. This optimization added randomness to the generation of new candidate solutions from the old ones aiming to better explore the search space of the objective problem in search of the global minimum.

To verify the effectiveness of the new method, a series of experiments were performed on a large group of objective problems from the recent literature. In these experiments both the efficiency and speed improvement of the original technique was measured, and a comparison of the speed of the new method was made in relation to other known techniques from the relevant literature. Furthermore, a series of extensive experiments were carried out to study the dynamics of the proposed initialization technique as well as the additional time required by its implementation. A number of useful conclusions were drawn from the execution of these experiments. First of all, the new method significantly improved the efficiency and speed of the original method. Furthermore, the experiments revealed that the new method required a significantly lower number of function calls on average than other global optimization methods. In addition, the sampling method proved to be highly efficient in finding the global minimum and significantly reduced the required number of function calls compared to other initialization techniques. The additional time required by the new initialization method was noticeable compared to other techniques, but the gains it brought were equally significant.

Future extensions of the proposed technique could be the use of parallel programming techniques to speed up the overall process, such as the MPI programming technique [110] or the integration of the OpenMP library [111], as well as the use of other termination techniques that could potentially speed up the termination of the method.