EOFA: An Extended Version of the Optimal Foraging Algorithm for Global Optimization Problems

Abstract

The problem of finding the global minimum of a function is applicable to a multitude of real-world problems and, hence, a variety of computational techniques have been developed to efficiently locate it. Among these techniques, evolutionary techniques, which seek, through the imitation of natural processes, to efficiently obtain the global minimum of multidimensional functions, play a central role. An evolutionary technique that has recently been introduced is the Optimal Foraging Algorithm, which is a swarm-based algorithm, and it is notable for its reliability in locating the global minimum. In this work, a series of modifications are proposed that aim to improve the reliability and speed of the above technique, such as a termination technique based on stochastic observations, an innovative sampling method and a technique to improve the generation of offspring. The new method was tested on a series of problems from the relevant literature and a comparative study was conducted against other global optimization techniques with promising results.

1. Introduction

The basic goal of global optimization is to find the global minimum by searching for the appropriate scope of each problem. Primarily, a global optimization method aims to discover the global minimum of a continuous function, and it is defined as

$$x^{*}=\arg \underset{x\in S}{min}f\left(x\right)$$

(1)

with S defined as

$$S=\left[a_1,b_1\right]\times \left[a_2,b_2\right]\times \dots \left[a_n,b_n\right]$$

Global optimization is a core part of applied mathematics [1] and computer science [2]. Additionally, it finds application in various fields such as physics [3,4,5], chemistry [6,7,8], economics [9,10], medicine [11,12], etc. Optimization methods are divided into deterministic and stochastic [13]. Deterministic methods guarantee finding the global optimal solution and are usually applied to small problems [14]. The most commonly used techniques in the first category are the so-called interval techniques [15,16], which divide the initial domain of the function until they discover a promising subset that may contain the global minimum. On the other hand, stochastic methods do not guarantee finding the global minimum and are mainly applied to large problems, but they are inefficient when nonlinear equality constraints are involved. Stochastic methods include a variety of techniques, such as Controlled Random Search methods [17,18,19], Simulated Annealing methods [20,21,22], methods based on the Multistart technique [23,24], etc. Furthermore, in the recent literature, a variety of hybrid methods have appeared that combine different optimization techniques [25,26]. Furthermore, due to the widespread usage of parallel programming architectures, a variety of methods have been proposed that take advantage of such systems [27,28,29,30].

The above techniques present various problems that have had to be addressed by researchers, such as, for example, the initialization of the possible solutions, the termination rules, etc. A number of researchers have dealt with the topic of the initialization of candidate solutions and have proposed techniques such as the repulsion method [31], an adaptive sampling method using the Kriging model [32], Monte Carlo-based sampling methods [33], a method which utilizes an Radial Basis Function (RBF) network [34] to obtain samples [35] etc. Additionally, in the direction of termination methods, a series of techniques have been proposed, such as sequential stopping rules [36], a stopping rule based on asymptotic considerations [37], a stopping rule that utilizes the likelihood of obtaining the specific set of solutions [38], etc.

An important group of stochastic techniques that has attracted the interest of many researchers is the group of evolutionary techniques. The branch of evolutionary computing includes the study of the foundations and applications of computational techniques based on the principles of natural evolution. Evolution in nature is responsible for the “design” of all living things on earth and the strategies they use to interact with each other. This branch of methods includes Differential Evolution methods [39,40], Particle Swarm Optimization (PSO) methods [41,42,43], Ant Colony optimization methods [44,45], genetic algorithms [46,47], etc. Zhu and Zhang proposed a novel method in this area that tackles optimization problems with a strategy based on animal foraging behavior [48]. This Optimal Foraging Algorithm (OFA) is a swarm-based algorithm motivated by animal behavioral ecology. Furthermore, Fu et al. proposed a variation of the OFA algorithm with the incorporation of the Differential Evolution technique [49]. Additionally, Jian and Zhu proposed a modification of the OFA algorithm with direction prediction [50] in order to enhance the performance of the original algorithm. Recently, Chen Ding and Guang Yu Zhu introduced an improved quasi-adversarial optimal social behavior search algorithm (QOS-OFA) to handle optimization problems [51].

The current work introduces a series of modifications to the OFA algorithm in order to speed up the process and to increase the effectiveness of the algorithm. These modifications include the following:

• The incorporation of a sampling technique that utilizes the K-means method [52]. The points that are sampled will more accurately and quickly lead to the global minimum of the function and additional points that might have been close enough using another sampling technique are discarded.
• The use of a termination technique based on stochastic observations. At each iteration of the algorithm, the smallest functional value is recorded. If this remains constant for a predetermined number of iterations, then the method terminates. This way, the method will terminate in time without needlessly wasting computing time on iterations that will simply produce the same result.
• The improvement of the offspring produced by the method using a few steps of a local minimization method. In the current work, a BFGS variant [53] was used as a local search procedure.

The rest of this paper is divided into the following sections: in Section 2, the proposed method is fully described. In Section 3, the benchmark functions are presented as well as the experimental results, and finally in Section 4 some conclusions and guidelines for future work are discussed.

2. The Proposed Method

The OFA algorithm uses nature as a source of inspiration. The OFA algorithm has as its main purpose the evolution of the population using some iterative steps which will be presented in more detail below. The quality of the initial population can be improved, allowing the search space to be explored more efficiently. The first major modification introduced to the above technique is the use of a sophisticated technique of sampling points from the objective function. This sampling is based on the K-means technique and is presented in detail in Section 2.2. The second modification is to apply a few steps of a local search procedure in offspring creation if an offspring is not consistent with the bounds $[a,b]$ of the objective function $f\left(x\right)$. With this modification, it is ensured that a new point will be within the field definition of the function on the one hand, and on the other hand, it will have a slightly lower value than its predecessor. The third modification is to use an efficient termination rule to terminate the global optimization method effectively. This termination rule is outlined in Section 2.3.

2.1. The Main Steps of the Algorithm

The main steps of the used global optimization method are the following:

• Initialization step.

  (a)

  Define as $N_c$ the number of elements in the population.

  (b)

  Define as $N_g$ the maximum number of allowed iterations.

  (c)

  Initialize randomly the members of the population in set S. This forms the initial population denoted as P. The sampling is performed using the procedure in Section 2.2.

  (d)

  Create the QOP population, which stands for the quasi-opposite population of P, by calculating the quasi-opposite positions of P.

  (e)

  Set $t=0$, the iteration number.

  (f)

  Select the $N_c$ fittest solutions from the population set $\left\{P,\mathrm{QOP}\right\}$ using the QOBL method proposed in [51].

• Calculation step.

  (a)

  Produce the new offspring solutions $x^{t+1}$ based on the ranking order using the following equation:

  $$x_j^{t+1}=\left\{\begin{array}{c}\begin{array}{cc}{x}_j^t+K\left(t\right)\times \left(r_1-r_2\right)\times \left(x_j^t-x_b^t\right)+\left(1-K\left(t\right)\right)\times \left(x_1^t-x_r^t\right)& ,j=2,\dots ,N_c\\ x_1^t+K\left(t\right)\times \left(r_1-r_2\right)\times \left(x_1^t-x_{N_c}^t\right)& ,j=1\end{array}\hfill \end{array}\right.$$

  (2)

  where $r_1,r_2$ are random numbers in $[0,1]$ and $x_r^t$ is a randomly selected solution from the population. The function $K\left(t\right)$ is defined as

  $$K\left(t\right)=cos\left({\displaystyle \frac{\pi t}{2N_g}}\right)$$

  (3)

  (b)

  For $j=1,\dots ,N_c$, do

  i.

  If $x^{t+1}\notin S$ then $x^{t+1}=LS\left(x^t,B_i\right)$. This step applies a limited number of steps of a local search procedure LS(x) to the current point. The parameter $B_i$ denotes the number of steps for the local search procedure. In the current work, the BFGS local optimization procedure is used, but other local search methods could be employed such as Gradient Descent [54], Steepest Descent [55], etc.

  ii.

  if ${\displaystyle \frac{{\lambda }_j^{t+1}\times f\left(x^{t+1}\right)}{1+(t+1)\times {\lambda }_j^{t+1}}}<{\displaystyle \frac{f\left(x^t\right)}{t}}$ then select $x^{t+1}$ for the next population else select $x^t$. The values ${\lambda }_j^{t+1}$ are uniformly distributed random numbers in $[0,1]$.

  (c)

  End For.

  (d)

  Sort all solutions in the current population from the best to worst according to the function value.

• Termination check step.

  (a)

  Set $t=t+1$.

  (b)

  Calculate to stopping rule in the work of Charilogis [56]. This termination method is described in Section 2.3.

  (c)

  If the termination criteria are not met then go to Calculation step, else terminate.

The steps of the proposed method are also graphically shown in Figure 1.

2.2. The Proposed Sampling Procedure

The proposed sampling technique initially produces a series of samples from the objective function and, subsequently, through the application of the K-means technique, only the located centers are considered as the final samples. The method was introduced by James MacQueen [57] and it is a clustering algorithm that has been used in data analysis and machine learning in a variety of research papers [58,59]. The algorithm seeks to estimate the centers of possible teams in a set of samples and its main steps are listed as follows:

• Define as k the number of clusters.
• Draw randomly $N_m$ initial points $x_i,i=1,\dots ,N_m$ from the objective function.
• Assign randomly each point $x_i,i=1,\dots ,N_m$ in a cluster $S_j,j=1,\dots ,k$.
• For every cluster $j=1\dots k$ do

  (a)

  Set as $M_j$ the number of points in $S_j$.

  (b)

  Compute the center of the cluster $c_j$ as

  $$c_j={\displaystyle \frac{1}{M_j}}\sum _{x_i\in S_j}{x}_i$$

• EndFor.
• Repeat

  (a)

  Set $S_j=\left\{\right\},j=1\dots k$.

  (b)

  For each point $x_i,i=1,\dots ,N_m$ do

  i.

  Set $j^{*}={\mathrm{argmin}}_{m=1}^k\left\{D\left(x_i,c_m\right)\right\}$. The function $D(x,y)$ is the Euclidean distance of points $(x,y)$.

  ii.

  Set $S_{j^{*}}=S_{j^{*}}\cup \left\{x_i\right\}$.

  (c)

  EndFor.

  (d)

  For each center $c_j,j=1\dots k$ do

  i.

  Update the center $c_j$ as

  $$c_j={\displaystyle \frac{1}{M_j}}\sum _{x_i\in S_j}{x}_i$$

  (e)

  EndFor.

• If there is no significant change in centers  $c_j$ terminate the algorithm and return the k centers as the final set of samples.

2.3. The Termination Rule Used

At every iteration t, the difference between the current best value $f_{min}^{\left(t\right)}$ and the previous best value $f_{min}^{(t-1)}$ is computed:

$${\delta }^{\left(t\right)}=\left|f_{\min }^{\left(k\right)}-f_{\min }^{(k-1)}\right|$$

(4)

The algorithm terminates when ${\delta }^{\left(t\right)}\le ϵ$ for a series of predefined consecutive iterations $N_k$, where $ϵ$ is a small positive number, for example ${10}^{-6}$.

3. Results

This section will begin with a detailed description of the functions that will be used in the experiments, followed by an analysis of the experiments performed and comparisons with other global optimization techniques.

3.1. Test Functions

The functions used in the experiments have been proposed in a series of relative works [60,61] and they cover various scientific fields, such as medicine, physics, engineering, etc. Furthermore, these objective functions have been used by many researchers in a variety of publications [62,63,64,65,66]. The definitions of these functions are given below:

• Ackley function:

  $$f\left(x\right)=-aexp\left(-b\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{x}_i^2}\right)-exp\left({\displaystyle \frac{1}{n}}\sum _{i=1}^{n}cos\left(c{x}_i\right)\right)+a+exp\left(1\right)$$

  where a = 20.0.
• Bf1 (Bohachevsky 1) function:

$$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1\right)-{\displaystyle \frac{4}{10}}cos\left(4\pi x_2\right)+{\displaystyle \frac{7}{10}}$$

• Bf2 (Bohachevsky 2) function:

  $$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1\right)cos\left(4\pi x_2\right)+{\displaystyle \frac{3}{10}}$$
• Bf3 (Bohachevsky 3) function:

  $$f\left(x\right)=x_1^2+2x_2^2-{\displaystyle \frac{3}{10}}cos\left(3\pi x_1+4\pi x_2\right)+{\displaystyle \frac{3}{10}}$$
• Branin function: $f\left(x\right)={\left(x_2-{\displaystyle \frac{5.1}{4{\pi }^2}}{x}_1^2+{\displaystyle \frac{5}{\pi }}{x}_1-6\right)}^2+10\left(1-{\displaystyle \frac{1}{8\pi }}\right)cos\left(x_1\right)+10$ with $-5\le x_1\le 10,0\le x_2\le 15$.
• Camel function:

  $$f\left(x\right)=4x_1^2-2.1x_1^4+{\displaystyle \frac{1}{3}}{x}_1^6+x_1{x}_2-4x_2^2+4x_2^4,x\in {[-5,5]}^2$$
• Easom function:

  $$f\left(x\right)=-cos\left(x_1\right)cos\left(x_2\right)exp\left({\left(x_2-\pi \right)}^2-{\left(x_1-\pi \right)}^2\right)$$

  with $x\in {[-100,100]}^2.$
• Exponential function, defined as

  $$f\left(x\right)=-exp\left(-0.5\sum _{i=1}^n{x}_i^2\right),-1\le x_i\le 1$$
  In the conducted experiments, the values $n=4,8,16,32$ were used.
• ExtendedF10 function:

  $$f\left(x\right)=\sum _{i=1}^n\left(x_i^2-10cos\left(2\pi x_i\right)\right)$$
• F12 function:

  $$\begin{array}{cc}\hfill f\left(x\right)& ={\displaystyle \frac{\pi }{n}}\left(10sin\left(\pi y_1\right)+\sum _{i=1}^{n-1}\left({\left(y_i-1\right)}^2\left(1+10{sin}^2\left(\pi y_{i+1}\right)\right)\right)+{\left(y_n-1\right)}^2\right)+\hfill \\ \hfill & \sum _{i=1}^{n}u\left(x_i,10,100,4\right)\hfill \end{array}$$

  where

  $$y_i=1+{\displaystyle \frac{x_i+1}{4}}$$

  and

  $$u\left(x_i,a,k,m\right)=\left\{\begin{array}{c}\begin{array}{cc}k{\left(x_i-a\right)}^m,& x_i>a\\ 0,& -a<x_i<a\\ k{\left(-x_i-a\right)}^m,& x_i<-a\end{array}\hfill \end{array}\right.$$
• F14 function:

  $$f\left(x\right)={\left({\displaystyle \frac{1}{500}}+\sum _{j=1}^{25}{\displaystyle \frac{1}{j+{\sum }_{i=1}^2{\left(x_i-a_{ij}\right)}^6}}\right)}^{-1}$$
• F15 function:

  $$f\left(x\right)=\sum _{i=1}^{11}{\left(a_i-{\displaystyle \frac{x_1\left(b_i+b_i{x}_2\right)}{b_i^2+b_i{x}_3+x_4}}\right)}^2$$
• F17 function:

  $$\begin{array}{cc}\hfill f\left(x\right)& =\left(1+{\left(x_1+x_2+1\right)}^2\times \left(19-14x_1+3x_1^2-14x_2+6x_1{x}_2+3x_2^2\right)\right)\times \hfill \\ \hfill & \left(30+{\left(2x_1-3x_2\right)}^2\times \left(18-32x_1+12x_1^2+48x_2-36x_1{x}_2+27x_2^2\right)\right)\hfill \end{array}$$
• Griewank2 function:

  $$f\left(x\right)=1+{\displaystyle \frac{1}{200}}\sum _{i=1}^2{x}_i^2-\prod _{i=1}^2{\displaystyle \frac{cos\left(x_i\right)}{\sqrt{\left(i\right)}}},x\in {[-100,100]}^2$$
• Griewank10 function: the function is given by the equation

  $$f\left(x\right)=\sum _{i=1}^n{\displaystyle \frac{x_i^2}{4000}}-\prod _{i=1}^{n}cos\left({\displaystyle \frac{x_i}{\sqrt{i}}}\right)+1$$

  with $n=10$.
• Gkls function: $f\left(x\right)=\mathrm{Gkls}(x,n,w)$ is a constructed function with w local minima presented in [67], with $x\in {[-1,1]}^n$. For the conducted experiments, the values $n=2,3$ and $w=50$ were utilized.
• Goldstein and Price function:

  $$\begin{array}{ccc}\hfill f\left(x\right)& =& \left[1+{\left(x_1+x_2+1\right)}^2\right.\hfill \\ & & \left(19-14x_1+3x_1^2-14x_2+6x_1{x}_2+3x_2^2\right)]\times \hfill \\ & & [30+{\left(2x_1-3x_2\right)}^2\hfill \\ & & \left(18-32x_1+12x_1^2+48x_2-36x_1{x}_2+27x_2^2\right)]\hfill \end{array}$$

  with $x\in {[-2,2]}^2$.
• Hansen function: $f\left(x\right)={\sum }_{i=1}^{5}icos\left[(i-1)x_1+i\right]{\sum }_{j=1}^{5}jcos\left[(j+1)x_2+j\right]$, $x\in {[-10,10]}^2$.
• Hartman 3 function:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^3{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^3$ and $a=\left(\begin{array}{ccc}3& 10& 30\\ 0.1& 10& 35\\ 3& 10& 30\\ 0.1& 10& 35\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{ccc}0.3689& 0.117& 0.2673\\ 0.4699& 0.4387& 0.747\\ 0.1091& 0.8732& 0.5547\\ 0.03815& 0.5743& 0.8828\end{array}\right)$$
• Hartman 6 function:

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^6{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^6$ and $a=\left(\begin{array}{cccccc}10& 3& 17& 3.5& 1.7& 8\\ 0.05& 10& 17& 0.1& 8& 14\\ 3& 3.5& 1.7& 10& 17& 8\\ 17& 8& 0.05& 10& 0.1& 14\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{cccccc}0.1312& 0.1696& 0.5569& 0.0124& 0.8283& 0.5886\\ 0.2329& 0.4135& 0.8307& 0.3736& 0.1004& 0.9991\\ 0.2348& 0.1451& 0.3522& 0.2883& 0.3047& 0.6650\\ 0.4047& 0.8828& 0.8732& 0.5743& 0.1091& 0.0381\end{array}\right)$$
• Potential function: This function stands for the energy of a molecular conformation of N atoms that interacts via the Lennard-Jones potential [68]. The function is defined as

  $$V_{LJ}\left(r\right)=4ϵ\left[{\left({\displaystyle \frac{\sigma }{r}}\right)}^{12}-{\left({\displaystyle \frac{\sigma }{r}}\right)}^6\right]$$
  For the conducted experiments, the values $N=3,5$ were used.
• Rastrigin function:

  $$f\left(x\right)=x_1^2+x_2^2-cos\left(18x_1\right)-cos\left(18x_2\right),x\in {[-1,1]}^2$$
• Rosenbrock function:

  $$f\left(x\right)=\sum _{i=1}^{n-1}\left(100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2\right),-30\le x_i\le 30.$$
  The values $n=4,8,16$ were used in the conducted experiments.
• Shekel 5 function:

$$f\left(x\right)=-\sum _{i=1}^5{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\end{array}\right)$

• Shekel 7 function:

$$f\left(x\right)=-\sum _{i=1}^7{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 3& 5& 3\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\end{array}\right)$.

• Shekel 10 function:

$$f\left(x\right)=-\sum _{i=1}^{10}{\displaystyle \frac{1}{(x-a_i){(x-a_i)}^T+c_i}}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 5& 3& 3\\ 8& 1& 8& 1\\ 6& 2& 6& 2\\ 7& 3.6& 7& 3.6\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\\ 0.7\\ 0.5\\ 0.6\end{array}\right)$.

• Sinusoidal function, defined as

  $$f\left(x\right)=-\left(2.5\prod _{i=1}^{n}sin\left(x_i-z\right)+\prod _{i=1}^{n}sin\left(5\left(x_i-z\right)\right)\right),0\le x_i\le \pi .$$
  The values $n=4,8,16$ were used in the conducted experiments.
• Sphere function:

  $$f\left(x\right)=\sum _{i=1}^n{x}_i^2$$
• Schwefel function:

  $$f\left(x\right)=\sum _{i=1}^n{\left(\sum _{j=1}^i{x}_j\right)}^2$$
• Schwefel 2.21 function:

  $$f\left(x\right)=418.9829n+\sum _{i=1}^n-x_{i}sin\left(\sqrt{\left|x_i\right|}\right)$$
• Schwefel 2.22 function:

  $$f\left(x\right)=\sum _{i=1}^n\left|x_i\right|+\prod _{i=1}^n\left|x_i\right|$$
• Test2N function:

  $$f\left(x\right)={\displaystyle \frac{1}{2}}\sum _{i=1}^n{x}_i^4-16x_i^2+5x_i,x_i\in [-5,5].$$
  For the conducted experiments, the values $n=4,5,6,7$ were used.
• Test30N function:

  $$f\left(x\right)={\displaystyle \frac{1}{10}}{sin}^2\left(3\pi x_1\right)\sum _{i=2}^{n-1}\left({\left(x_i-1\right)}^2\left(1+{sin}^2\left(3\pi x_{i+1}\right)\right)\right)+{\left(x_n-1\right)}^2\left(1+{sin}^2\left(2\pi x_n\right)\right)$$
  The values $n=3,4$ were used in the conducted experiments.

3.2. Experimental Results

A series of experiments was performed using the test functions presented previously. Every experiment was conducted 30 times using different seeds for the random generator each time. The averages of function calls were measured and are presented in the following tables. The software was coded in ANSI C++ with the assistance of the OPTIMUS optimization package, which is freely available from https://github.com/itsoulos/OPTIMUS (accessed on 2 July 2024). The values for the experimental parameters are shown in

Table 1. Experimental settings. The numbers in cells denote the values used in the experiments for all parameters.

Parameter	Meaning	Value
$N_c$	Number of chromosomes/particles	500
$N_g$	Maximum number of allowed iterations	200
$N_m$	Number of initial samples for K-means	$10\times N_c$
$N_k$	Number of iterations for stopping rule	5
$p_s$	Selection rate for the genetic algorithm	0.1
$p_m$	Mutation rate for the genetic algorithm	0.05
$B_i$	Number of iterations for BFGS	3

. Generally, the region of attraction $A\left(x^{*}\right)$ of a local minimum $x^{*}$ is defined as

$$A\left(x^{*}\right)=\left\{x:x\in S,L\left(x\right)=x^{*}\right\}$$

(5)

where $L\left(x\right)$ is a local search procedure, such as BFGS. Global optimization methods usually find points which are in the region of attraction $A\left(x^{*}\right)$ of a local minimum $x^{*}$ but not necessarily the local minimum itself. For this reason, at the end of their execution, a local minimization method is applied in order to ensure the exact finding of a local minimum. Of course, this does not imply that the minimum that will be located will be the total minimum of the function, since a function can contain tens or even hundreds of local minima.

The following applies to

Table 2. Experimental results using different optimization methods. Numbers in cells represent average function calls.

FUNCTION	GENETIC	PSO	EOFA
ACKLEY	16,754	24,518	1346
BF1	10,466	9912	1856
BF2	10,059	9364	1738
BF3	7290	9847	1408
BRANIN	10,032	5940	1479
CAMEL	11,069	7132	1540
EASOM	10,587	4922	1304
EXP4	10,231	7382	1651
EXP8	10,622	7644	1891
EXP16	10,458	8050	2145
EXP32	10,202	8800	2165
EXTENDEDF10	10,739	14,363	1361
F12	31,277	23,705	4340
F14	14,598	26,296	1617
F15	8215	13,599	1620
F17	9202	11,851	1438
GKLS250	10,198	5488	1404
GKLS350	9861	6029	1325
GOLDSTEIN	11,901	9244	1751
GRIEWANK2	13,612	10,315	1602
GRIEWANK10	14,750	15,721	3092
HANSEN	13,053	7636	1559
HARTMAN3	10,066	6897	1664
HARTMAN6	11,119	8061	1873
POTENTIAL3	16,325	10,728	2093
POTENTIAL5	34,284	19,307	3397
RASTRIGIN	13,354	9783	1528
ROSENBROCK4	12,618	9266	2134
ROSENBROCK8	15,019	12,854	2985
ROSENBROCK16	17,150	21,074	4151
SHEKEL5	13,927	8383	2021
SHEKEL7	13,688	8491	2029
SHEKEL10	13,722	8746	2120
TEST2N4	10,522	7815	1608
TEST2N5	10,847	8393	1658
TEST2N6	11,180	9385	1782
TEST2N7	11,485	10,561	1876
SPHERE	3942	8330	1199
SCHWEFEL	4688	9037	1242
SCHWEFEL2.21	6233	7872	1286
SCHWEFEL2.22	81,980	88,227	5309
SINU4	12,920	7250	1700
SINU8	12,703	8202	2202
SINU16	12,404	10,640	2188
TEST30N3	16,692	7750	1912
TEST30N4	19,159	10,036	1820
SUM	651,203	564,846	91,409

:

• The column FUNCTION denotes the name of the objective problem.
• The column GENETIC denotes the application of a genetic algorithm to the objective problem. The genetic algorithm has $N_c$ chromosomes and the maximum number of allowed generations was set to $N_g$.
• The column PSO stands for the application of Particle Swarm Optimizer to every objective problem. The number of particles was set to $N_c$ and the maximum number of allowed iterations was set to $N_g$.
• The column EOFA represents the application of the proposed method using the values for the parameters shown in Table 1.
• The row SUM represents the sum of function calls for all test functions.

A statistical comparison of these results is also outlined in Figure 2.

As can be seen, the present method drastically reduces the required number of function calls to almost all objective functions. This, of course, also results in a significant reduction in the required computing time to find the global minimum. Furthermore, a scatter plot for the used optimization methods is shown in Figure 3, which also depicts the superiority of the current method.

Furthermore, an additional experiment was conducted in order to measure the importance of the K-means sampling in the proposed method. In this test, three different sampling methods were used in the proposed method:

• The column UNIFORM stands for the application of uniform sampling in the proposed method.
• The column TRIANGULAR represents the application of the triangular distribution [69] to sample the initial points of the proposed method.
• The column KMEANS represents the sampling method presented in the current work.

The experimental results for this test are shown in

Table 3. Experiments using different sampling techniques for the proposed method.

FUNCTION	UNIFORM	TRIANGULAR	KMEANS
ACKLEY	1803	1330	1346
BF1	2637	2146	1856
BF2	2470	2011	1738
BF3	1863	1366	1408
BRANIN	2023	1518	1479
CAMEL	2174	1671	1540
EASOM	1755	1264	1304
EXP4	2428	1844	1651
EXP8	2500	1939	1891
EXP16	2568	2026	2145
EXP32	2534	1976	2165
EXTENDEDF10	1812	1360	1361
F12	6085	4196	4340
F14	2225	1691	1617
F15	1950	1512	1620
F17	1932	1424	1438
GKLS250	1895	1403	1404
GKLS350	1789	1249	1325
GOLDSTEIN	2516	2013	1751
GRIEWANK2	2242	1711	1602
GRIEWANK10	3776	3416	3092
HANSEN	2163	1678	1559
HARTMAN3	2289	1757	1664
HARTMAN6	2541	2065	1873
POTENTIAL3	2841	2366	2093
POTENTIAL5	4029	3960	3397
RASTRIGIN	2105	1779	1528
ROSENBROCK4	3255	2949	2134
ROSENBROCK8	4059	3468	2985
ROSENBROCK16	4963	4391	4151
SPHERE	1533	1033	1199
SCHWEFEL	1635	1139	1242
SCHWEFEL2.21	1724	1230	1286
SCHWEFEL2.22	8363	8663	5309
SHEKEL5	2945	2515	2021
SHEKEL7	3008	2567	2029
SHEKEL10	3160	2683	2120
TEST2N4	2339	1804	1608
TEST2N5	2388	1847	1658
TEST2N6	2478	1962	1782
TEST2N7	2552	2044	1876
SINU4	2444	1954	1700
SINU8	2881	2388	2202
SINU16	3694	3300	2188
TEST30N3	2189	1654	1912
TEST30N4	2282	2374	1820
SUM	124,837	102,636	91,309

.

The statistical comparison for the previous experiment is graphically outlined in Figure 4. The proposed sampling method significantly outperforms the remaining sampling techniques in almost all problems. Furthermore, the proposed global optimization technique outperforms the rest of the compared techniques, even if a different sampling technique is used.

Moreover, a scatter plot representation of the used distributions with Kruskal–Wallis Ranking is depicted in Figure 5.

In this test, the p-value remains very low at any level of significance. Consequently, the null hypothesis is rejected, indicating that the experimental tests are statistically significant.

Furthermore, an additional test was carried out, where the function ELP was used to measure the effectiveness of the proposed method as the dimension increased. This function is defined as

$$f\left(x\right)=\sum _{i=1}^n{\left({10}^6\right)}^{{\displaystyle \frac{i-1}{n-1}}}{x}_i^2$$

where the parameter n defines the dimension of the function. A comparison between the genetic algorithm and the proposed method as n increases is shown in Figure 6.

The required number of calls for the current method increases at a much lower rate than in the case of the genetic algorithm. This means that the method can cope with large-scale problems without significantly increasing the required computing time.

4. Conclusions

Three modifications were proposed to the EOFA optimization method in this article. The modifications were primarily aimed at improving the efficiency and speed of the global optimization algorithm. The first modification proposed the application of a sampling technique which incorporates the K-means method [52]. With the proposed modification, the points where the sampling was achieved helped to find the global minimum with the greatest accuracy and in the least possible time. The second amendment concerned termination rules. Termination rules help terminate functions immediately without unnecessarily wasting computational time on iterations. The third modification concerned the refinement of the offspring produced using the BFGS method [53] as a local search procedure. In the experiments conducted, different sampling techniques were used for the proposed method. More specifically, the following techniques were used: Uniform, Triangular and K-means. It was found that the K-means sampling technique yields much better results than the other two techniques and the total number of calls is extremely lower.

Experiments were also performed using different optimization methods. More specifically, the following were used: genetic, PSO and EOFA. It was observed that the average number of EOFA calls is very limited compared to the other two.

Since the experimental results have been shown to be extremely promising, further efforts can be made to develop the technique in various areas. Among the future extensions of the application may be the use of parallel computing techniques to speed up the optimization process, such as the integration of MPI [70] or the OpenMP library [71].