A Parallel Implementation of the Differential Evolution Method

Abstract

Global optimization is a widely used technique that finds application in many sciences such as physics, economics, medicine, etc., and with many extensions, for example, in the area of machine learning. However, in many cases, global minimization techniques require a high computational time and, for this reason, parallel computational approaches should be used. In this paper, a new parallel global optimization technique based on the differential evolutionary method is proposed. This new technique uses a series of independent parallel computing units that periodically exchange the best solutions they have found. Additionally, a new termination rule is proposed here that exploits parallelism to accelerate process termination in a timely and valid manner. The new method is applied to a number of problems in the established literature and the results are quite promising.

1. Introduction

The task of locating the global minimum of a continuous and differentiable function $f:S\to R,S\subset R^n$ is defined as

$$x^{*}=\arg \underset{x\in S}{min}f\left(x\right).$$

(1)

The set S is defined as:

$$S=\left[a_1,b_1\right]\otimes \left[a_2,b_2\right]\otimes \dots \left[a_n,b_n\right]$$

A variety of practical problems from various research fields can be modeled as global optimization problems, such as problems from physics [1,2,3], chemistry [4,5,6], economics [7,8], medicine [9,10], etc. Many methods have been proposed to tackle the problem of Equation (1), such as controlled random search methods [11,12,13], simulated annealing methods [14,15,16], differential evolution methods [17,18], particle swarm optimization (PSO) methods [19,20,21], ant colony optimization [22,23], genetic algorithms [24,25,26], etc. Reviews of stochastic methods for global optimization problems can be found in the work of Pardalos et al. [27] or in the work of Fouskakis et al. [28].

The current work proposes a parallel implementation of the differential evolution (DE) method that aims to speed up the optimization process of this particular method and also tries to make adequate use of modern computing structures with multicore architectures. The DE method initially generates a population of candidate solutions, which iteratively evolves through the crossover process in order to discover the global minimum of the objective function. The method was applied in various research fields such as electromagnetics [29], energy consumption problems [30], job shop scheduling [31], image segmentation [32], etc. The proposed method partitions the processing into independent structural units, such as threads, and each of them acts independently. Furthermore, the new method proposes a way of communication between the different building blocks of parallel processing and a process termination technique suitably modified for parallel processing.

In the recent literature, several methods have been proposed that take full advantage of parallel processing, such as parallel techniques [33,34,35], methods that take advantage of GPU architectures [36,37,38], etc. Moreover, Weber et al. [39] proposed a parallel DE method for large-scale optimization problems using new search mechanisms for the individuals of the subpopulations. Chen et al. [40] proposed a parallel DE method for cluster optimization using modified genetic operators. Moreover, Penas et al. [41] suggested an enhanced parallel asynchronous DE algorithm for problems in computational systems biology. Recently, Sui et al. [42] proposed a parallel compact DE method applied to image segmentation.

The proposed technique is a modified version of the parallel island methodology for different evolutionary techniques [43,44]. Therefore, in the proposed technique, the initial population of agents (candidate solutions) is divided into a series of independent populations and each individual population evolves independently in a parallel computing unit, such as a thread. Populations periodically exchange information with each other, such as the lowest functional value to which they have been driven. The proposed technique uses a new differential weight calculation scheme, can use a number of different information exchange methods between the parallel computing units and furthermore proposes a new termination method of the optimization process that can take advantage of the parallelism so that the optimization terminates in time and is valid.

The rest of this article is organized as follows: in Section 2, the original DE method as well as the proposed modifications are outlined, in Section 3, the experimental test functions from the relevant literature and the associated experimental results are listed, and finally, in Section 4, some conclusions and guidelines for future research are provided.

2. Method Description

In this section, the basic DE method is first presented and then, the proposed modifications are analyzed so that it can be executed in parallel.

2.1. The Original DE Method

The original method was originally proposed by Storn [45], and it has been modified in various research papers. For example, the compact differential evolution algorithm [46,47], a self-adaptive DE [48] where the parameters of the method are modified iteratively, fuzzy logic modifications [49], etc. Moreover, a numerical study on some modifications of the DE method can be found in the work of Kaelo et al. [50]. The base DE method has the steps described below in Algorithm 1.

Algorithm 1: The steps of the base DE method

1. INPUT: (a) The population size $NP\ge$ 4. The members of this population are also called agents. (b) The crossover probability $CR\in [0,1]$. (c) The differential weight $F\in [0,2].$ 2. OUTPUT: (a) The agent $x_{best}$ with the lower function value $f\left(x_{best}\right)$. 3. Initialize all agents in S. 4. While termination criteria are not held do (a) For$i=1\dots NP$do i. Select as x the agent i. ii. Select randomly three agents $a,b,c$ with the property $a\ne b,$ $b\ne c$, $c\ne a$. iii. Select a random position $R\in \left\{1,\dots ,n\right\}$ iv. Create the vector $y=\left[y_{1,}{y}_2,\dots ,y_n\right]$ with the following procedure v. For$j=1,\dots ,n$do A. Set$r_i\in [0,1]$ a random number. B. If$r_j<\mathrm{CR}$or$j=R$then$y_j=a_j+F\times \left(b_j-c_j\right)$else$y_j=x_j$. vi. If $y\in SANDf\left(y\right)\le f\left(x\right)$ then $x=y$. vii. EndFor (b) EndFor 5. End While

2.2. Proposed Modifications

In the proposed procedure, the population of agents is segmented into N independent contiguous segments. Each section is called an island in the following text, similar to island genetic algorithms [51,52], which are very popular parallel variants of genetic algorithms. For example, if there are 10 agents and 2 islands, then agents 1–5 are assigned to island 1 and agents 6–10 to island 2. On each island, the process of differential evolution is carried out independently. Of course, there should be some mechanism for communication between the islands as well as some appropriate mechanism for terminating the overall process. The proposed Algorithm 2 is presented next.

Algorithm 2: The steps of the proposed method

1. INPUT: (a) The parameters NP, CR, F. (b) The integer parameter N, which stands for the number of islands. (c) The integer parameter $N_R$, which represents the propagation rate. (d) The integer parameter $N_I$, which represents the number of islands that should terminate in order to terminate the whole process. 2. OUTPUT: (a) The agent $x_{best}$ with the lower function value $f\left(x_{best}\right)$. 3. Initialize all agents in S. 4. Set iter = 1 5. For i = 1,..., N do in Parallel (a) Perform for every island i the step 4.a of the base DE algorithm of Section 2.1. 6. EndFor 7. If iter mod $N_R$ = 0, apply the propagation scheme of Section 2.3 to the islands. The default value used in the experiments is the “1 to 1” case. 8. Set iter = iter + 1 9. If the termination rule of Section 2.4 is not valid, goto 5. 10. Apply local search procedure to $x_{best}$. The local search procedure used in the proposed method is the BFGS variant of Powell [53].

2.3. Propagation Mechanism

During the propagation mechanism, the best values of the islands are spread to the rest by replacing their worst values. In general, there are the following propagation cases:

1.

One to one. In this case, a random island will send to another randomly selected island its best value.

2.

One to N. In this case, a random island will send its best value to all other islands.

3.

N to one. In this case, all islands will inform a randomly selected island about their best value.

4.

N to N. All islands will inform all the other islands about their best value.

2.4. Termination Rule

In the proposed termination criterion, a simple criterion is checked separately on each island. For every island i the difference

$${\delta }_i^{\left(k\right)}=\left|f_{i,\min }^{\left(k\right)}-f_{i,\min }^{(k-1)}\right|,$$

(2)

is measured at each iteration $k,$ where $f_{i,\min }^{\left(k\right)}$ is the best located function value for island i at iteration k. If ${\delta }_i^{\left(k\right)}\le ϵ$ for at least M consecutive iterations, then most likely the island i should terminate the population evolution. In the proposed technique, if the quantity of Equation (2) holds for more than $N_I$ islands, then the overall algorithm terminates.

3. Experiments

In the following, the benchmark functions used in the experiments as well as the experimental results are presented.

3.1. Test Functions

To evaluate the ability of the proposed technique to find the total minimum of functions, a series of test functions from the relevant literature [54,55] were used and are presented below.

• Bent-cigar function. The function is

  $$f\left(x\right)=x_1^2+{10}^6\sum _{i=2}^n{x}_i^2$$

  with the global minimum $f\left(x^{*}\right)=0$. For the conducted experiments, the value $n=10$ was used.
• Bf1 function. The function Bohachevsky 1 is given by the equation

  $$f\left(x\right)=x_1^2+2x_2^2-\frac{3}{10}cos\left(3\pi x_1\right)-\frac{4}{10}cos\left(4\pi x_2\right)+\frac{7}{10}$$

  with $x\in {[-100,100]}^2$.

• Bf2 function. The function Bohachevsky 2 is given by the equation

  $$f\left(x\right)=x_1^2+2x_2^2-\frac{3}{10}cos\left(3\pi x_1\right)cos\left(4\pi x_2\right)+\frac{3}{10}$$

  with $x\in {[-50,50]}^2$.
• Branin function. The function is defined by $f\left(x\right)={\left(x_2-\frac{5.1}{4{\pi }^2}{x}_1^2+\frac{5}{\pi }{x}_1-6\right)}^2+10\left(1-\frac{1}{8\pi }\right)cos\left(x_1\right)+10$ with $-5\le x_1\le 10,0\le x_2\le 15$. The value of the global minimum is 0.397887 with $x\in {[-10,10]}^2$.
• CM function. The cosine mixture function is given by the equation

  $$f\left(x\right)=\sum _{i=1}^n{x}_i^2-\frac{1}{10}\sum _{i=1}^{n}cos\left(5\pi x_i\right)$$

  with $x\in {[-1,1]}^n$. For the conducted experiments, the value $n=4$ was used.
• Discus function. The function is defined as

  $$f\left(x\right)={10}^6{x}_1^2+\sum _{i=2}^n{x}_i^2$$

  with global minimum $f\left(x^{*}\right)=0.$ For the conducted experiments, the value $n=10$ was used.
• Easom function. The function is given by the equation

  $$f\left(x\right)=-cos\left(x_1\right)cos\left(x_2\right)exp\left({\left(x_2-\pi \right)}^2-{\left(x_1-\pi \right)}^2\right)$$

  with $x\in {[-100,100]}^2$ and a global minimum of −1.0.
• Exponential function. The function is given by

  $$f\left(x\right)=-exp\left(-0.5\sum _{i=1}^n{x}_i^2\right),-1\le x_i\le 1$$
  The global minimum is located at $x^{*}=(0,0,\cdots ,0)$ with a value of $-1$. In our experiments, we used this function with $n=4,16,64$ and the corresponding functions are denoted by the labels EXP4, EXP16 and EXP64.
• Griewank2 function. The function is given by

  $$f\left(x\right)=1+\frac{1}{200}\sum _{i=1}^2{x}_i^2-\prod _{i=1}^2\frac{cos\left(x_i\right)}{\sqrt{\left(i\right)}},x\in {[-100,100]}^2$$
  The global minimum is located at the $x^{*}=(0,0,\cdots ,0)$ with a value of 0.
• Gkls function. $f\left(x\right)=Gkls(x,n,w)$ is a function with w local minima, described in [56] with $x\in {[-1,1]}^n$ and n a positive integer between 2 and 100. The value of the global minimum is −1 and in our experiments, we used $n=2,3$ and $w=50,100$.
• Hansen function. $f\left(x\right)={\sum }_{i=1}^{5}icos\left[(i-1)x_1+i\right]{\sum }_{j=1}^{5}jcos\left[(j+1)x_2+j\right]$, $x\in [-10,$${10]}^2$. The global minimum of the function is −176.541793.
• Hartman 3 function. The function is given by

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^3{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^3$ and $a=\left(\begin{array}{ccc}3& 10& 30\\ 0.1& 10& 35\\ 3& 10& 30\\ 0.1& 10& 35\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{ccc}0.3689& 0.117& 0.2673\\ 0.4699& 0.4387& 0.747\\ 0.1091& 0.8732& 0.5547\\ 0.03815& 0.5743& 0.8828\end{array}\right).$$
  The value of the global minimum is −3.862782.
• Hartman 6 function.

  $$f\left(x\right)=-\sum _{i=1}^4{c}_{i}exp\left(-\sum _{j=1}^6{a}_{ij}{\left(x_j-p_{ij}\right)}^2\right)$$

  with $x\in {[0,1]}^6$ and $a=\left(\begin{array}{cccccc}10& 3& 17& 3.5& 1.7& 8\\ 0.05& 10& 17& 0.1& 8& 14\\ 3& 3.5& 1.7& 10& 17& 8\\ 17& 8& 0.05& 10& 0.1& 14\end{array}\right),c=\left(\begin{array}{c}1\\ 1.2\\ 3\\ 3.2\end{array}\right)$ and

  $$p=\left(\begin{array}{cccccc}0.1312& 0.1696& 0.5569& 0.0124& 0.8283& 0.5886\\ 0.2329& 0.4135& 0.8307& 0.3736& 0.1004& 0.9991\\ 0.2348& 0.1451& 0.3522& 0.2883& 0.3047& 0.6650\\ 0.4047& 0.8828& 0.8732& 0.5743& 0.1091& 0.0381\end{array}\right).$$
  The value of the global minimum is −3.322368.
• High conditioned elliptic function, defined as

  $$f\left(x\right)=\sum _{i=1}^n{\left({10}^6\right)}^{\frac{i-1}{n-1}}{x}_i^2$$

  with global minimum $f\left(x^{*}\right)=0$; the value $n=10$ was used in the conducted experiments
• Potential function. The molecular conformation corresponding to the global minimum of the energy of N atoms interacting via the Lennard–Jones potential [57] was used as a test case here. The function to be minimized is given by:

  $$V_{LJ}\left(r\right)=4ϵ\left[{\left(\frac{\sigma }{r}\right)}^{12}-{\left(\frac{\sigma }{r}\right)}^6\right]$$

  (3)
  In the experiments, two different cases were studied: $N=3,5$
• Rastrigin function. The function is given by

  $$f\left(x\right)=x_1^2+x_2^2-cos\left(18x_1\right)-cos\left(18x_2\right),x\in {[-1,1]}^2$$
• Shekel 7 function.

$$f\left(x\right)=-\sum _{i=1}^7\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 3& 5& 3\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\end{array}\right)$.

• Shekel 5 function.

$$f\left(x\right)=-\sum _{i=1}^5\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\end{array}\right)$.

• Shekel 10 function.

$$f\left(x\right)=-\sum _{i=1}^{10}\frac{1}{(x-a_i){(x-a_i)}^T+c_i}$$

with $x\in {[0,10]}^4$ and $a=\left(\begin{array}{cccc}4& 4& 4& 4\\ 1& 1& 1& 1\\ 8& 8& 8& 8\\ 6& 6& 6& 6\\ 3& 7& 3& 7\\ 2& 9& 2& 9\\ 5& 5& 3& 3\\ 8& 1& 8& 1\\ 6& 2& 6& 2\\ 7& 3.6& 7& 3.6\end{array}\right),c=\left(\begin{array}{c}0.1\\ 0.2\\ 0.2\\ 0.4\\ 0.4\\ 0.6\\ 0.3\\ 0.7\\ 0.5\\ 0.6\end{array}\right)$.

• Sinusoidal function. The function is given by

  $$f\left(x\right)=-\left(2.5\prod _{i=1}^{n}sin\left(x_i-z\right)+\prod _{i=1}^{n}sin\left(5\left(x_i-z\right)\right)\right),0\le x_i\le \pi .$$
  The global minimum is located at $x^{*}=(2.09435,2.09435,\cdots ,2.09435)$ with $f\left(x^{*}\right)=-3.5$. For the conducted experiments, the cases of $n=4,8$ and $z=\frac{\pi }{6}$ were studied. The parameter z was used to shift the location of the global minimum [58].
• Test2N function. This function is given by the equation

  $$f\left(x\right)=\frac{1}{2}\sum _{i=1}^n{x}_i^4-16x_i^2+5x_i,x_i\in [-5,5].$$
  The function has $2^n$ in the specified range and in our experiments, we used $n=4,5,6,7$.
• Test30N function. This function is given by

  $$f\left(x\right)=\frac{1}{10}{sin}^2\left(3\pi x_1\right)\sum _{i=2}^{n-1}\left({\left(x_i-1\right)}^2\left(1+{sin}^2\left(3\pi x_{i+1}\right)\right)\right)+{\left(x_n-1\right)}^2\left(1+{sin}^2\left(2\pi x_n\right)\right)$$

  with $x\in [-10,10]$. The function has ${30}^n$ local minima in the specified range, and we used $n=3,4$ in the conducted experiments.

3.2. Experimental Results

To evaluate the performance of the modified version of the differential evolutionary technique, a series of experiments were performed in which the number of parallel computing units varied from 1 to 10. The freely available OpenMP library [59] was used for parallelization, and the method was coded in ANSI C++ inside the OPTIMUS optimization package available from https://github.com/itsoulos/OPTIMUS (accessed on 4 January 2023). All the experiments were conducted on an AMD Ryzen 5950X with 128 GB of RAM and the Debian Linux operating system. All the experiments were conducted 30 times using different seed for the random generator each time and averages were reported. The values for the parameters used in the DE algorithm are shown in

Table 1. Experimental settings.

Parameter	Value
NP	200 agents
Propagation	1-to-1 method
$N_R$	5 iterations
$N_I$	2 islands
CR	0.9 for the crossover probability
M	15 iterations
$ϵ$	${10}^{-4}$

. The parameter F (differential weight) was calculated as:

$$F=-\frac{1}{2}+2\times R,$$

(4)

where $R\in [0,1]$ is a random number, which was used in [60]. This random scheme for the calculation of the parameter F was used successfully to better explore the search space of the objective function. The experimental results for different numbers of threads for the test functions of the previous subsection are shown in

Table 2. Comparison of experimental results with a “1 to 1” propagation scheme. The first column represents the name of the objective function and the remaining columns are the average function calls using 1 to 10 processing threads for the proposed method.

Function	Thread 1	Threads 4	Threads 5	Threads 10
BF1	5908	5517	5310	4887
BF2	5415	5008	4888	4577
BRANIN	5467	4767	4535	3895
CIGAR10	1886	1885	1885	1875
CM4	2518	2432	2330	2243 (0.97)
DISCUS10	1818	1816	1814	1807
EASOM	1807	1802	1801	1791
ELP10	44,910	41,731	41,930	29,944
EXP4	1820	1816	1814	1806
EXP16	1838	1835	1834	1830
EXP64	1842	1840	1839	1838
GKLS250	1987	1897	1879	1818
GKLS350	2428	2373	2299	2195
GRIEWANK2	5544	4811	4612	4208
POTENTIAL3	11,121	7868	7260	5598
POTENTIAL5	24,708	15,146	13,793	8620
HANSEN	23,035	13,602	12,178	9242
HARTMAN3	3406	3198	3162	2883
HARTMAN6	7611	6172	5739	4877 (0.97)
RASTRIGIN	5642	4537	4386	3707
ROSENBROCK4	11,859	10,441	10,139	9473
ROSENBROCK8	21,640	19,536	20,560	20,654
SHEKEL5	12,491	10,247	9754	5065 (0.80)
SHEKEL7	10,755	9183	8857	6996
SHEKEL10	10,257	9002	8705	7283
SINU4	6045	5473	5301	4434
SINU8	9764	8132	7748	4523
TEST2N4	8521	7487	7404	6834
TEST2N5	10,218	8916	8715	8050
TEST2N6	11,984	10,240	10,191	9175
TEST2N7	15,674	13,983	13,341	9760
TEST30N3	3720	3379	3349	2994
TEST30N4	3728	3382	3363	3031
Average	298,267	249,454	242,715	197,913 (0.99)

. The number in the cells denote average function calls for every test function. The number in parentheses stands for the fraction of executions where the global optimum was successfully found. The absence of this number indicates that the global minimum was computed for every independent run (100% success). At the end of the table, an additional row named average has been added and shows the total number of function calls for all test functions and the average success rate in locating the global minimum.

As can be seen, as the number of computational threads increased, the required number of function calls needed to locate the global minimum decreased, with no appreciable difference in the overall reliability of the method, which remained extremely high (99–100%). In addition, to show the dynamics of the proposed methodology, it was also used in the training of an artificial neural network [61] for learning a common benchmark problem from machine learning, the wine problem [62,63]. The sums of execution times for 30 independent runs are displayed in Figure 1. As we can see, as the number of network weights increased from w = 5 to w = 20, the gain from using multiple processing threads increased as the training time decreased noticeably.

In addition, to discover whether there was differentiation using the different propagation techniques, additional experiments were performed using 10 processing threads. In each processing thread, as before, the population of each island was 20 agents. The results from these experiments are shown in

Table 3. Experiments for the proposed method using different options for the propagation method. The number of processing threads was set to 10. Numbers in cells represent average function calls for every test function.

Function	1 to 1	1 to N	N to 1	N to N
BF1	4887	4259	4209	2792
BF2	4577	4021	3917	2691
BRANIN	3895	3378	3307	2382
CIGAR10	1875	1874	1871	1873
CM4	2243 (0.97)	2173	2136 (0.97)	2030
DISCUS10	1807	1810	1808	1809
EASOM	1791	1790	1791	1789
ELP10	29,944	42,025	22,876	19,117
EXP4	1806	1807	1811	1806
EXP16	1830	1829	1828	1824
EXP64	1838	1838	1838	1836
GKLS250	1818	1812	1810	1802
GKLS350	2195	2163	2109	2011 (0.97)
GRIEWANK2	4208	3620	3514	2445 (0.80)
POTENTIAL3	5598	4445	4353	2521
POTENTIAL5	8620	7475	7025	3374
HANSEN	9242	6075	6181	3135
HARTMAN3	2883	2664	2593	2207
HARTMAN6	4877 (0.97)	4327 (0.83)	4362 (0.80)	2834 (0.57)
RASTRIGIN	3707	3213	2870	2220 (0.90)
ROSENBROCK4	9473	8294	7883	7084
ROSENBROCK8	20,654	24,470	15,919	19,272
SHEKEL5	5065 (0.80)	7556	5386 (0.93)	4456 (0.70)
SHEKEL7	6996	7207 (0.90)	6488 (0.93)	4493 (0.80)
SHEKEL10	7283	6812 (0.93)	6440	3916 (0.73)
SINU4	4434	4204	4020	2796 (0.97)
SINU8	4523	5386	4605	3341 (0.90)
TEST2N4	6834	5777	5625	3609 (0.97)
TEST2N5	8050	6695 (0.97)	6647	4179 (0.73)
TEST2N6	9175	7770 (0.93)	7660	4522 (0.53)
TEST2N7	9760	9259 (0.77)	9081	5200 (0.57)
TEST30N3	2994	2814	2653	2210
TEST30N4	3031	2797	2700	2107
Average	197,913 (0.99)	201,649 (0.98)	167,316(0.98)	129,683 (0.91)

. From the experimental results, it appears that most of the time, there were no significant changes in the total number of function calls except in the case of an “N to N” propagation. There was a significant reduction in function calls, but also a drop in reliability of the techniques from 99% to 91%. This may be because, due to the exchange of the best costs between all the islands, the total population was locked into local minima.

Furthermore, the proposed method was compared against the original differential evolution method and two variants from the relevant literature, mentioned as DERL and DELB [50]. The results from this comparison are shown in

Table 4. Comparison of the proposed method against other variants of the differential evolution technique.

Function	Proposed	Original DE	DERL	DELB
BF1	4887	5516	2881	5319
BF2	4577	5555	2895	5405
BRANIN	3895	5656	2857	4830
CIGAR10	1875	88,396	66,161	58,460
CM4	2243 (0.97)	9107	3856	6014
DISCUS10	1807	87,657	55,722	49,014
EASOM	1791	7879	7225	14,934
ELP10	29,944	33,371	9345	39,890
EXP4	1806	6027	2638	4142
EXP16	1830	26,194	25,117	11,740
EXP64	1838	26,497	27,831	18,346
GKLS250	1818	3800	1983	3706
GKLS350	2195	6206	2901	5027
GRIEWANK2	4208	6365	3325	6165
POTENTIAL3	5598	82,933	111,496	44,592
POTENTIAL5	8620	24,118	61,694	46,557
HANSEN	9242	18,470	7123	12212
HARTMAN3	2883	4655	2205	4124
HARTMAN6	4877 (0.97)	15,488	5343	7215 (0.93)
RASTRIGIN	3707	6362	3102	5704
ROSENBROCK4	9473	16,857	6679	10,411
ROSENBROCK8	20,654	56,445	17,198	22,939
SHEKEL5	5065 (0.80)	13,079	5224 (0.90)	8167
SHEKEL7	6996	12,409	4994 (0.97)	8093
SHEKEL10	7283	13,238	5240	8822
SINU4	4434	8977	3828	6052
SINU8	4523	28,871	9318	10,157
TEST2N4	6834	10,764	4529	7331
TEST2N5	8050	15,568	5917	8969
TEST2N6	9175	21,185	7613	10,648
TEST2N7	9760	28,411	9492	12,252
TEST30N3	2994	4965	2758	4693
TEST30N4	3031	5123	2688	5153
Average	197,913 (0.99)	706,144	491,178 (0.99)	477,083 (0.99)

. As is evident, the proposed technique significantly outperformed the other modifications of the different evolutionary method. This was largely due to the different differential weight calculation technique but also to the proposed termination method. The used differential weight calculation technique largely succeeded in providing a better search of the search space, while the new termination method terminated the optimization method in time. Moreover, this new termination technique was modified to perform well in parallel computing environments as well.

Moreover, a statistical comparison was performed for the proposed method and different numbers of processing threads, and the results are outlined in Figure 2. A statistical comparison was also included for the results of the proposed method against the other variations of the DE method and the corresponding plot is shown in Figure 3.

4. Conclusions

A new global optimization technique was presented in this manuscript, which can be performed in parallel computing environments. This method was based on the well-known differential evolutionary technique and partitioned the initial population of agents, so as to create independent populations executed on parallel computing units. The parallel units periodically exchanged the best values for the objective function with each other, and from the experiments carried out it was found that the most robust information exchange technique was the so-called “1 to 1”, where a randomly selected subpopulation exchanges information with another randomly selected subpopulation. Furthermore, a new termination method was proposed which could take full advantage of the parallel computing environment. With this termination rule, the decision to terminate the method could be efficiently made even by a small portion of the independent computing units.

From the experimental results, it appeared that the proposed technique could successfully find the global minimum in a series of problems from the relevant literature, and in fact, as the number of parallel processing units increased, the required number of function calls decreased. Furthermore, after experiments on a difficult problem such as the training of artificial neural networks, it was shown that the time required for the optimization process decreased dramatically with the increase of threads.

However, much can still be done to improve the methodology, such as finding a better way of communication between parallel processing units or even formulating more efficient termination criteria that exploit parallel computing environments. In addition, the proposed technique could be applied to other global optimization techniques such as genetic algorithms or particle swarm optimization.