Iterative Optimization RCO: A “Ruler & Compass” Deterministic Method

Abstract

We present a basic version of a deterministic iterative optimization algorithm that requires only one parameter and is often capable of finding a good solution after very few evaluations of the fitness function. We demonstrate its principles using a multimodal one-dimensional problem. For such problems, the algorithm could be applied with just a ruler and a compass, which is how it got its name. We also provide classical examples and compare its performance with six well-known stochastic optimizers. These comparisons highlight the strengths and weaknesses of RCO. Since this version does not address potential stagnation, it is best suited for low-dimensional problems (typically no more than ten), where each evaluation of a position in the search space is computationally expensive.

1. Introduction

In iterative optimization, and for some problems, the cost of each evaluation can be very high. Additionally, many algorithms are stochastic, meaning they need to be run multiple times. Therefore, a method that requires only one run and may find a good solution after just a few evaluations could be highly beneficial. As mentioned, this approach does not work for every problem, but it is generally worth trying before moving on to a more costly stochastic method.

The only parameter required is the lower bound, which is often reasonably well-known in practice. Furthermore, there is an optional adaptive lower bound, in which case no parameter is needed.

Several definitions of the theoretical a priori difficulty of an optimization problem (minimization here) have been proposed [1]. For example, one can consider the number of local minima and the relative size of the subspace of acceptable solutions. If this number is high and the size small, then any iterative algorithm functioning as a black box would require numerous samples of positions in the search space and, therefore, many evaluations of positions.

Consider the multimodal one-dimensional function whose landscape is shown in Figure 1, where the minimum is 0.999507 at position 2.412616. Although it is one-dimensional, its theoretical difficulty is high because it has many local minima.

Figure 1. Test landscape sinCos15 (see Appendix A.7). Many local minima, very small set of acceptable solutions.

For the algorithms tested below, we say that a solution is acceptable if its value is less than 0.9997.

For stochastic algorithms, we estimate the probability of success by running them a thousand times.

Table 1 then shows that, for these, many evaluations are needed to have a good chance of finding a satisfactory solution and, therefore, the actual difficulty (large number of evaluations) is in good agreement with the theoretical difficulty.

However, it is possible to define a deterministic (in the non-stochastic sense) algorithm, for which this agreement is challenged. On this problem, the Ruler & Compass Optimizer (RCO) is extremely efficient (see Figure 2). More generally, it is also effective on other supposedly difficult problems. Furthermore, conversely, it can be highly ineffective on problems that are supposed to be easy.

Figure 2. RCO on the test landscape sinCos15. Path followed (in red). The first position is 0. An acceptable solution is found after 20 evaluations.

Thus, the mere existence of such an algorithm largely calls into question the relevance of classical difficulty estimates, which will have to be revised, at least by specifying their field of application in the space of optimizers.

Informally, the No Free Lunch Theorem [2] allows us to say that there is no such thing as an efficient optimizer for all problems. If “efficient” is to be interpreted as “finding a solution without too much difficulty”, then what RCO suggests is a dual proposition: there is no a priori measure of difficulty valid for all optimizers.

Analyzing this conjecture is not the purpose of this study. In what follows, we will look at the detailed operation of RCO and show on other examples how its behavior is atypical. This will allow us to sketch a typology of problems for which it is interesting to use it.

Table 1. On the problem in Figure 1, metaheuristics, as expected, sometimes struggle to find an acceptable solution with a high probability, estimated over 1000 executions. Since RCO is deterministic, only one run is required.

Algorithm	Stochastic	Evaluations/Run	Success Rate
CMA-ES [3]	yes	10,000	0.12
DE [4]	yes	10,000	0.73
ACO [5]	yes	410	0.80
		4010	0.94
Jaya [6]	yes	1000	0.985
SPSO [7]	yes	180	0
		190	1
GA-MPC [8]	yes	80	0
		90	1
RCO	no	20	1

Table 1. On the problem in Figure 1, metaheuristics, as expected, sometimes struggle to find an acceptable solution with a high probability, estimated over 1000 executions. Since RCO is deterministic, only one run is required.

Algorithm	Stochastic	Evaluations/Run	Success Rate
CMA-ES [3]	yes	10,000	0.12
DE [4]	yes	10,000	0.73
ACO [5]	yes	410	0.80
		4010	0.94
Jaya [6]	yes	1000	0.985
SPSO [7]	yes	180	0
		190	1
GA-MPC [8]	yes	80	0
		90	1
RCO	no	20	1

2. RCO Principle

On our Test landscape, RCO finds a definite acceptable solution after 20 evaluations (see Figure 2).

However, to detail the iterative construction, it is better to consider a simpler example, defined by:

$$f\left(x\right)={\left(x-3\right)}^{2}forx\in \left[0,10\right]$$

(1)

where its minimum is obviously 0 on $x=3$.

For this basic version, a user-defined parameter is needed: a fixed lower bound. Note that in practice, we often know one. Moreover, RCO is in fact able to use an adaptive one, automatically estimated during the process (see Appendix A.5), but the figures are then more difficult to understand.

Here, we set this lower bound to −5, to more clearly see what happens, although −0.1 would be sufficient.

Table 2, Table 3, Table 4, Table 5 and Table 6 illustrate the process. Of course, in higher dimension D the lines become (hyper-)planes but the method is the same. At each time step, we consider D planes defined by $D+1$ points and their intersection point (it may happen that there is more than one unique intersection point; if so, we consider we are in the case 1) with the lower bound plane. There are two cases:

Table 2. Principle. Step 1.

An horizontal lower bound is defined. The “corners” are evaluated. The line connecting them intersects the lower bound beyond the search space, this intersection point is rejected and will be replaced with another one (step 2).

Table 3. Principle. Step 2.

The new position is defined so that $a/b=A/B$. As the dimension is 1, this can be achieved thanks to a “Ruler & Compass” construction. Then, the position is evaluated to define the third point.

Table 4. Principle. Step 3.

Now the line joining the two last points does intersect the lower bound on a point whose position is inside the search space. This position is then maintained and evaluated. The same process is repeated, again.

Table 5. Principle. Step 9, zoom.

… and again.

Table 6. Principle. Step 10, and zoom.

After Step 9, the line connecting the last two points is nearly horizontal and intersects the lower boundary outside of the search space. Consequently, we generate a new position using the same method as in Step 2. This occurrence becomes increasingly frequent as the positions approach the solution. Hence, the new position often approximates the mean of the last two positions, which is a favorable behavior for better approaching the solution.

• If its position lies outside the search space (possibly at infinity), then the new position is determined by a barycenter of the $2^D$ previous positions. For the calculation of the barycenter, the weight of each position is inversely proportional to the value of the function at that position. See the detail in Appendix A.2.
• If its position is within the search space, then it is kept as the new position.

Then, in the list of positions, the first one is removed and the new one is added.

3. Examples

The definitions of the problems are given in Appendix A.7. Some of them have been shifted in order to be not too easy.

3.1. Six Hump Camel Back (Appendix A.7)

The dimension of the problem is two, so the landscape has four corners. The search space is $\left[-2,2\right]\times \left[-1,1\right]$.

The minimum is −1.031628453489877. A definite lower bound is −1.1. Table 7 shows the results for increasing number of evaluations and Figure 3 the construction for 14 evaluations.

Table 7. Six Hump Camel Back.

Constructed Points (Evaluations)	Best Fitness
14	−0.957541
24	−1.030227
34	−1.030227
44	−1.031227
54	−1.031227
64	−1.031473
74	−1.031473
84	−1.031473
94	−1.031473
104	−1.031473

Figure 3. Six Hump Camel Back. Lower bound −1.1. After the construction of 14 points the result is −0.9575.

3.2. Shifted Rastrigin

(See Appendix A.7 for details).

The minimum is zero, so a definite lower bound is −0.1. Table 8 shows the results for different dimensions and different numbers of evaluations and Figure 4 the constructions for 1D and 2D landscapes after 12 and 14 evaluations, respectively.

Table 8. Shifted Rastrigin.

Dimension	Constructed Points (Evaluations)	Best Fitness
1	52	0.000557
2	1004	0.001511
3	1508	0.002548
4	50,016	0.001735
5	250,032	0.004518

Figure 4. Shifted Rastrigin.

3.3. Rosenbrock

(See Appendix A.7 for details).

The minimum is zero, so a definite lower bound is −0.1. Table 9 shows the results for dimensions two to five with an increasing number of evaluations.

Table 9. Rosenbrock.

Dimension	Evaluations	Best Fitness
2	252	0.002788
3	1508	0.001003
4	10,016	0.003756
5	15,032	0.005719

3.4. Pressure Vessel

(See Appendix A.7 for details).

There are four values to find, where the two first ones are discrete. The minimum value is 6059.714335048436, as proved in [9].

The lower bound used here is 6000. Table 10 show the results for an increasing number of evaluations. Due to the discrete variables, the algorithm encounters difficulties in converging.

Table 10. Pressure Vessel.

Evaluations	Best Fitness
1052	11,056.81
2049	6646.71
3074	6615.977
4074	6615.977
5074	6615.977
6076	6592.011

3.5. Gear Train

(See Appendix A.7 for details).

There are four values to find, all discrete. The minimum is $2.700857\times {10}^{-12}$.

Here, the lower bound is simply set to 0. As we can see in Table 11, the algorithm quickly proposes a solution that is far from optimal, and then stagnates without further improvement.

Table 11. Gear Train. Discretization is applied only at the very end to all values, so more search effort does not always imply a better result.

Evaluations	Best Fitness
116	$3.10\times {10}^{-3}$
130	$1.20\times {10}^{-3}$
173	$6.65\times {10}^{-9}$
223	$6.65\times {10}^{-9}$
1753	$6.65\times {10}^{-9}$

4. Comparison

Let us compare RCO with a slightly improved version of Standard PSO (SPSO-Dicho on my technical website [10]. To favor PSO, I have selected the best run out of five. The results are presented in Table 12. Also, refer to Figure 5 for Shifted Griewank, where the two methods appear to be equivalent, particularly for dimensions 1 to 5 and the number of evaluations, respectively set to 100, 200, 5000, 10,000, and 20,000.

Table 12. RCO vs. PSO. The better values are highlighted in bold.

Problem	Dimension	Evaluations	RCO	PSO
Six Hump Camel Back	2	54	−1.031227	−1.011030
Shifted Rastrigin	1	52	0.000557	0.510724
	2	1004	0.001511	0.4397499
	5	250,032	0.004578	0.994959
	10	251,024	35.246	1.9899
Rosenbrock	2	252	0.002788	266.94
	5	15,032	0.005719	0.0864
	10	101,024	8.996	$5.19\times {10}^{-8}$
Pressure Vessel	4	3074	6615.977	6890.347
		5074	6615.977	6821.931
		20,074	6583.75	6820.410
Gear Train	4	173	$6.65\times {10}^{-9}$	$2.35\times {10}^{-9}$
		1154	$6.65\times {10}^{-9}$	$2.35\times {10}^{-9}$
		199,752	$1.18\times {10}^{-9}$	$1.26\times {10}^{-9}$

Figure 5. Shifted Griewank. RCO and PSO are equivalent.

In lower dimensions (less than 10), RCO performs better and sometimes significantly so. However, for higher dimensions, RCO stagnates while PSO continues to find better solutions, as evidenced by Shifted Rastrigin and Rosenbrock.

5. Complexities

To assess an algorithm, it is common practice to conduct a theoretical analysis of both its space complexity and time complexity. However, as explained in [11], this approach is not always appropriate, and a more reliable method is to estimate the actual memory usage and computational time.

In this context, there are two possible scenarios at each time step:

• Solving a linear system of D equations to determine a new position with D coordinates. The required space is $D^2+D$, with a time complexity of $O\left(D^3\right)$.
• Computing a linear combination of $2^D$ positions, each with D coordinates. The required space is $O(D\times 2^D)$, and the number of multiplications is $O\left(2^D\right)$.

However, such a classical reasoning does not correspond to the behavior of the algorithm in practice. The point is that the second situation usually does not occur very often except at the end of the process. Moreover, it depends on the landscape of the problem and also on the “budget” (the maximum acceptable number of evaluations).

Let us consider, for example, the computing time per evaluation for two problems: Planes and Shifted Rastrigin. We have already seen the Shifted Rastrigin and Planes is defined by:

$$f\left(x\right)=\sum _{d=1}^D\left(x\left(d\right)-3\right)$$

(2)

Figure 6 depicts its landscape in two dimensions.

Figure 6. Two-dimensional Planes problem landscape.

As we can see in Table 13, the real computing time indeed grows exponentially, but more like ${1.35}^D$ than $2^D$.

Table 13. Computing time per iteration.

Dimension	Planes	Shifted Rastrigin
1	$2.40\times {10}^{-5}$	$1.60\times {10}^{-5}$
2	$3.20\times {10}^{-5}$	$4.80\times {10}^{-5}$
3	$4.50\times {10}^{-5}$	$5.30\times {10}^{-5}$
4	$6.09\times {10}^{-5}$	$6.69\times {10}^{-5}$
5	$8.17\times {10}^{-5}$	$8.47\times {10}^{-5}$
6	$1.05\times {10}^{-4}$	$1.05\times {10}^{-4}$
7	$1.37\times {10}^{-4}$	$1.38\times {10}^{-4}$
8	$1.80\times {10}^{-4}$	$1.78\times {10}^{-4}$
9	$2.57\times {10}^{-4}$	$2.31\times {10}^{-4}$
10	$3.91\times {10}^{-4}$	$3.15\times {10}^{-4}$
11	$5.15\times {10}^{-4}$	$4.67\times {10}^{-4}$
12	$1.04\times {10}^{-3}$	$1.09\times {10}^{-3}$

6. A Problem That RCO Cannot Solve

In some instances, even when problems are continuous and of low dimension, RCO does not perform well. For instance, let us examine the Frequency Modulated Sound Waves from the CEC 2011 competition benchmark [12]. The search space is ${\left[-6.4, 6.35\right]}^6$ and the minimum value is zero. Despite 50,064 evaluations, the best final value achieved is 24.07. This is primarily due to a considerable number of intersection positions (21,238) lying outside the search space, rendering them unacceptable.

It is worth noting that in such cases, expanding the search space might provide some benefit if feasible. For example, if the search space is expanded to ${\left[-500, 500\right]}^6$, there are 17,322 unacceptable intersections, resulting in a final best value of 17.42. However, many classical stochastic algorithms tend to discover significantly better solutions. This is because stochasticity can effectively navigate a highly chaotic landscape, as is the one of FMSW (see Figure 7). For example, the slightly improved PSO already mentioned needs just two runs to find $2.218602\times {10}^{-27}$.

Figure 7. Frequency modulated sound waves problem. Cross section on dimensions 5 and 6.

7. Conclusions and Future Works

Even in low dimensions, evaluating the objective function can sometimes be very costly. In such cases, RCO may be beneficial as it may propose a good solution after only a few evaluations. However, there are some disadvantages:

• It requires a reasonably good lower bound. That said, compared to most other methods, this is the only user-defined parameter. In practice, for real-world problems, such a bound is often known. Moreover RCO is able to automatically define an adaptive lower bound (see Appendix A.5).
• It performs poorly on some problems, even in low dimensions, when the landscape is highly chaotic.
• It does not perform well on discrete problems, particularly when all variables are discrete. However, it still appears to be usable when only some of the variables are discrete.
• Its computation time per iteration increases exponentially with the dimension of the search space. Although it does not grow as quickly as theoretically predicted, in practice, it can still be challenging to use on a laptop for high-dimensional problems.
• As with many initial presentations of iterative optimization algorithms, such as Genetic Algorithm [13], Ant Colony Optimization [14], Particle Swarm Optimization [15], and Differential Evolution [16], among others, a formal convergence analysis has not yet been provided. Specifically, while it is clear that there is no explosion effect, as seen in the original PSO version [17], it would be beneficial to more precisely identify in which cases stagnation occurs and how it can be avoided. There is, in fact, an experimental RCO version that attempts to address the issue of stagnation, though it is not particularly convincing. For instance, in the case of the 10D Rosenbrock function (see Table 12), it finds a value of 2.3 instead of 8.996, which is still significantly worse than PSO ($5.19\times {10}^{-8}$).

Appendix A.1. A Bit of Geometry

To perform the “Ruler & Compass” construction in one dimension, as depicted in  Figure 3, it is necessary to define two segments, denoted as a and b, such that a/b = A/B. This method, dating back over 2600 years to Thales of Miletus, may have slipped from your memory over time; thus, Figure A1 illustrates the procedure.

Figure A1. Ruler & Compass method to define proportional segments.

Figure A1. Ruler & Compass method to define proportional segments.

Appendix A.2. Barycenter

More generally we define the barycenter $xNew$ of the $2^D$ current positions x thanks to the values of the function f on these positions. See below the Octave/Matlab code of this calculation.

  

% Barycenter

minf=min(f);

if minf>=0 w=sum(f)-f; else w=sum(f-minf)-(f-minf); end

if w>0  xNew=sum(w’.*x)/sum(w);

else xNew=mean(x); end % Particular case, w may be null

  

Note: Of course, other formulas are possible—particularly those that use the differences between the f-values and the lower bound—but this one seems to produce good results.

Appendix A.3. Number of Minima

In black box optimization of a non-discrete function, it is theoretically impossible to know the number of (local) minima, because the set of positions contains an infinite number of points. On a digital computer, it is theoretically possible by exhaustive search, because with an $\epsilon$-machine, the number of positions is finite. However, this number will still be huge (typically $2^{53D}$ for a definition space of dimension D).

So, we have to make a comprise: an exhaustive search on a grid whose spacing is not too large (which may generate many fake minima) and not too small (too much computing time, numerical instability).

Then the following algorithm for a function f may be useful:

  

Define a grid with N positions along each dimension

For each position x evaluate $f\left(x\right)$, and also the values $f_i=f\left(x_i\right)$ of the neighbors $x_i$ (usually $2D$ except on the frontier).

If $f\left(x\right)<f_i$ for all i, then assume that x is a (local) minimum.

  

This algorithm requires about $2^{N+1}$ evaluations, but does not need a large memory size. We could evaluate just once each position and save the result, but then the memory size would be big. Furthermore, for each position, we would have to find if it has been already evaluated.

When two neighbors on the grid have the same value, the algorithm may assume they are both “minima”, as we can see in Figure A2.

By applying this algorithm, we find that the landscape of sinCos15 has 28 minima in dimension one (and 818 in dimension two), including just one global minimum (see Figure 1). Six Hump Camel Back has six minima, including two global ones. Furthermore, naturally, Parabola is unimodal. For the Frequency-modulated Sound Waves, we find that there are at least 99,325 minima. Further refinement would be too time consuming.

Figure A2. Looking for minima, the algorithm may find fake minima when two neighbors (here 2 and 4) on the grid have the same value.

Figure A2. Looking for minima, the algorithm may find fake minima when two neighbors (here 2 and 4) on the grid have the same value.

Appendix A.4. Difficulty Measures

Detailed descriptions, examples, and references can be found in [1]. As explained in this Reference, the ${\delta }_{\neg NisB}$ measure is interesting, for it implicitly takes into account not only the number of local minima but also the sizes of their attraction basins. Using a tolerance threshold is usually less discriminant and the normalized roughness is too rudimentary.

Table A1. Difficulty measures in [0, 1]. The ones of Parabola Six Hump Camel Back are given for comparison. The tolerance thresholds are relative to the global minimum value. For sinCos15, it corresponds to a value at most equal to 0.9997, as used in the Introduction.

	sinCos15	Parabola	Six Hump Camel Back	Frequency Modulated Sound Waves
Normalized roughness	0.9643	0	0.3333	>0.9999
Tolerance threshold $1.93\times {10}^{-4}$	0.9996	0.9972	0.99997	1.0
${\delta }_{\neg NisB}$	0.4706	0.0711	0.6258	0.5263

Table A1. Difficulty measures in [0, 1]. The ones of Parabola Six Hump Camel Back are given for comparison. The tolerance thresholds are relative to the global minimum value. For sinCos15, it corresponds to a value at most equal to 0.9997, as used in the Introduction.

	sinCos15	Parabola	Six Hump Camel Back	Frequency Modulated Sound Waves
Normalized roughness	0.9643	0	0.3333	>0.9999
Tolerance threshold $1.93\times {10}^{-4}$	0.9996	0.9972	0.99997	1.0
${\delta }_{\neg NisB}$	0.4706	0.0711	0.6258	0.5263

As we can see, when considering the sizes of the attraction basins, the difficulty of FMSW is estimated to be less than that of Six Hump Camel Back. This is because, although there are many local minima, each basin is very small. Therefore, many stochastic algorithms can easily escape these minima.

Appendix A.5. About the Lower Bound

As mentioned, the basic RCO needs a user-defined lower bound. Therefore, it is interesting to understand the algorithm’s sensitivity to this parameter. As depicted in Figure A3, we observe that efficiency is indeed sensitive to changes in the lower bound. Though the sensitivity is not extreme, it could still be beneficial to experiment with various values. It is worth noting that setting the lower bound to the exact minimum value, if known, may not be the optimal choice.

Figure A3. Efficiency is somewhat sensitive to the user-defined lower bound, but not excessively so.

Figure A3. Efficiency is somewhat sensitive to the user-defined lower bound, but not excessively so.

A clear example of sensitivity can be observed when attempting to solve the optimal control problem of the Bifunctional Catalyst Blend in the CEC 2011 competition [12].

As depicted in Figure A4, even a slight modification to the lower bound leads to significantly different behavior due to the very small slope. Conversely, due to the same reason, a position far from the optimal one already exhibits a value close to the minimum.

Figure A4. CEC 2011 Bifunctional Catalyst Blend optimal control problem. Twelve evaluations.

Figure A4. CEC 2011 Bifunctional Catalyst Blend optimal control problem. Twelve evaluations.

If you are unsure about the lower bound, you can utilize an adaptive one. This is evaluated as follows after initialization and at each iteration:

• if min_f>0 lower_bound=min_f*coeff; else lower_bound=min_f*(2-coeff);

where min_fis the minimum function value found so far and coeff a user-defined parameter in $\left]0, 1\right[$.

It appears that the results can occasionally be quite satisfactory, as evidenced by Table A2. Thus, it may be worthwhile to try different methods: user-defined and adaptive ones.

Table A2. Adaptive lower bound may improve the efficiency, or decrease it. The better values are highlighted in bold.

	Dimension	Evaluations	Lower Bound	Non Adaptive	Adaptive Coeff. 0.5	Adaptive Coeff. 0.1
Six Hump Camel Back	2	104	−1.1	−1.031473	−1.03157	−0.947
Shifted Rastrigin	3	1508	−0.1	0.002548	0.06729	$3.27\times {10}^{-5}$
	4	50,016	−0.1	0.00173	$1.78\times {10}^{-15}$	0
Rosenbrock	3	1508	−0.1	0.001	0.1866	0.238
	4	10,016	−0.1	0.003756	$7.52\times {10}^{-22}$	$8.94\times {10}^{-24}$
Pressure Vessel	4	6076	6000	6592	6955	$1.29\times {10}^5$
Gear Train	4	1753	0	$6.65\times {10}^{-9}$	$1.38\times {10}^{-6}$	$6.60\times {10}^{-10}$

Table A2. Adaptive lower bound may improve the efficiency, or decrease it. The better values are highlighted in bold.

	Dimension	Evaluations	Lower Bound	Non Adaptive	Adaptive Coeff. 0.5	Adaptive Coeff. 0.1
Six Hump Camel Back	2	104	−1.1	−1.031473	−1.03157	−0.947
Shifted Rastrigin	3	1508	−0.1	0.002548	0.06729	$3.27\times {10}^{-5}$
	4	50,016	−0.1	0.00173	$1.78\times {10}^{-15}$	0
Rosenbrock	3	1508	−0.1	0.001	0.1866	0.238
	4	10,016	−0.1	0.003756	$7.52\times {10}^{-22}$	$8.94\times {10}^{-24}$
Pressure Vessel	4	6076	6000	6592	6955	$1.29\times {10}^5$
Gear Train	4	1753	0	$6.65\times {10}^{-9}$	$1.38\times {10}^{-6}$	$6.60\times {10}^{-10}$

Appendix A.6. More Examples

For amusement, here are a few 1D and 2D figures illustrating how rapidly the algorithm approaches the solution, thanks to steps defined purely by geometry. Initially, they are quite large and gradually diminish in size as they approach the solution’s position.

Figure A5. Multiparaboloid. Fourteen evaluations, final value 0.38 (real minimum 0).

Figure A5. Multiparaboloid. Fourteen evaluations, final value 0.38 (real minimum 0).

Figure A6. Branin. Forty-four evaluations, final value 0.3988 (real minimum 0.397887).

Figure A6. Branin. Forty-four evaluations, final value 0.3988 (real minimum 0.397887).

Figure A7. Combination of sinus. (a) Twelve evaluations, final value −1.899584 (real minimum −1.8996). (b) Sixteen evaluations, final value −1.732.

Figure A7. Combination of sinus. (a) Twelve evaluations, final value −1.899584 (real minimum −1.8996). (b) Sixteen evaluations, final value −1.732.

Figure A8. Shifted Griewank. Fourteen evaluations, final value 0.34 (real minimum 0).

Figure A8. Shifted Griewank. Fourteen evaluations, final value 0.34 (real minimum 0).

Appendix A.7. Problem Definitions

Name	Definition	Search Space	Minimum
sinCos15	$f=2+{\prod }_{d=1}^D(cos\left(x_d\right)+$ $\left|sin\left(x_d\right)cos\left(x_d\right)\right|sin\left(15x_d\right)+x_d/10)$	${\left[0, 10\right]}^D$	0.999507 for $D=1$
Six Hump Camel Back	$\left(4-2.1x_1^2+x_1^4/3\right)x_1^2$ $+x_1{x}_2+4\left(x_2^2-1\right)x_2^2$	${\left[-2, 1\right]}^2$	—1.031628453489877
Shifted Rastrigin	$\left\{\begin{array}{c}{u}_d=x_d-10d/\left(D+1\right)\\ f=10D+{\sum }_{d=1}^D{u}_d^2-10cos\left(2\pi u_d\right)\end{array}\right.$	${\left[-10, 10\right]}^D$	0
Rosenbrock	$f={\sum }_{d=1}^{D-1}\left(100{\left(x_{d+1}-x_d^2\right)}^2+\left(1-x_{d+1}^2\right)\right)$	${\left[-100, 100\right]}^D$	0
Pressure Vessel	$\left\{\begin{array}{c}0.0193x_3-x_1\le 0\\ 0.00954x_3-x_2\le 0\\ \mathrm{1,296,000}-\pi x_3^2\left(x_4+4x_3/3\right)\le 0\\ f=0.6224x_1{x}_3{x}_4+1.7781x_2{x}_3^2\\ +x_1^2\left(3.1661x_4+19.84x_3\right)\end{array}\right.$	$\left\{\begin{array}{c}q=0.0625\\ {\left\{q,2q,\dots ,99\right\}}^2\\ \times {\left[10, 200\right]}^2\end{array}\right.$	6059.714335048436
Gear Train	${\left(f=1.0/6.931-x_1{x}_2/\left(x_3{x}_4\right)\right)}^2$	${\left\{12,60\right\}}^4$	$2.700857\times {10}^{-12}$
Frequency Modulated Sound Waves	$\left\{\begin{array}{c}\theta =\pi /50\\ y_t=\\ {\sum }_{t=1}^{100}{x}_{1}sin\left(x_{2}t\theta +x_{3}sin\left(x_{4}t\vartheta +x_{5}sin\left(x_{6}t\theta \right)\right)\right)\\ y_0=\\ {\sum }_{t=1}^{100}sin\left(5t\theta -1.5sin\left(4.8t\vartheta +2sin\left(4.9t\theta \right)\right)\right)\\ f={\left(y_t-y_0\right)}^2\end{array}\right.$	${\left[-6.4, 6.35\right]}^6$	0