# A Local Optima Network View of Real Function Fitness Landscapes

## Abstract

**:**

## 1. Introduction

## 2. Search Spaces and Local Optima Networks

#### 2.1. Local Optima Networks in Discrete Spaces

#### 2.2. LON Methodology for Real Function Optimization

**x**.

Algorithm 1 Basin Hopping |

$s\leftarrow $ generate initial solution $x\leftarrow $ minimize $\left(f\right(s\left)\right)$ while termination condition not met do $y\leftarrow $ perturb (x) $z\leftarrow $ minimize $\left(f\right(y\left)\right)$ $x\leftarrow $ acceptance (x, z) end whilereturn
$x,f\left(x\right)$ |

**while**loop is executed (line 5) The termination condition of the

**while**loop is set as a maximum number of allowed function evaluations. Each time a new minimum is found (We use the standard quasi-Newton method L-BFGS from the Python SciPy library which can also work by numerically approximating the derivatives if they are not available analytically), it is added to the set of graph vertices V (lines 4 and 9). Each new directed edge $\{x,z\}$, with direction from x to z, is added to the set of edges E (line 10) or, if the edge existed already, its weight is increased (lines 14 and 16) and the search continues from the last minimum found. Notice that self-loops simply count the number of times a perturbation followed by local minimization fell back into the basin of attraction of the minimum from which the perturbation was applied. In continuous spaces, there is a numerical precision issue that is not present in discrete spaces: when the search finds an optimum with the same function value as a previous one (to a certain precision), one must decide whether the optimum has already been found or if it is new (lines 8 and 12). The problem is easily solved by a test on the optima coordinates: the absolute value of their differences must be less than $\u03f5$ for all the coordinates for the optima to be the same. We have used $\u03f5=0.0001$. At the end the LON ${G}_{w}(V,E)$ is returned. The above sampling procedure differs from the method used in [13]. In the latter, only minima that are monotonously decreasing in objective value, i.e., only improving or equal solutions are accepted. As a consequence, the resulting sampled graphs have long linear branches.

Algorithm 2 Basin Hopping Sampling |

Require:$f\left(x\right)$, $x\in \mathcal{X},\mathcal{X}\subseteq {\mathbb{R}}^{n}$, strength p, number of sample points q1: $V=\varnothing ,E=\varnothing \phantom{\rule{0.277778em}{0ex}}\phantom{\rule{0.277778em}{0ex}}\phantom{\rule{0.277778em}{0ex}}\phantom{\rule{0.277778em}{0ex}}$ //vertices and edges sets 2: for each sampling point s in $1..q$ do3: $x\leftarrow $ minimize $\left(f\right(s\left)\right)$ 4: $V\leftarrow V\cup \left\{x\right\}$ 5: while termination condition not met do6: $y\leftarrow $ perturb ($x,p$) 7: $z\leftarrow $ minimize $\left(f\right(y\left)\right)$ 8: if $z\notin V$ then9: $V\leftarrow V\cup \left\{x\right\}$ 10: $E\leftarrow E\cup \left\{xz\right\}$ 11: ${w}_{xz}\leftarrow 1$ 12: else13: if $z=x$ then14: ${w}_{xx}\leftarrow {w}_{xx}+1$ 15: else16: ${w}_{xz}\leftarrow {w}_{xz}+1$ 17: end if18: end if19: $x\leftarrow z$ 20: end while21: end for22: $\mathbf{return}{G}_{w}(V,E)$ |

## 3. LONS of Some Common Test Functions

#### 3.1. Algorithm Performance and Problem Hardness

#### 3.2. Local Optima Networks Statistics

## 4. Results and Discussion

#### 4.1. Strength

#### 4.2. Degree Distribution Functions

#### 4.3. Paths and Distances

#### 4.4. PageRank Centrality

#### 4.5. Funnels and Function Difficulty

#### 4.6. Scaling to Higher Dimension

Algorithm 3 Multistart Minima Sampling |

Require:$f(.)$, bounding box $B=x\in {[a,b]}^{n}$, number of starting points q create empty list of minima M for $i\leftarrow 1$ to q do $s\leftarrow $ generate a random solution in B $m\leftarrow $ minimize (f(s)) if $m\notin M$ and m not out of bounds B then add m and $f\left(m\right)$ to M end ifend forreturn
M |

## 5. Conclusions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Liberti, L. Introduction to Global Optimization; LIX, Ecole Polytechnique: Palaiseau, France, 2008. [Google Scholar]
- Talbi, E.G. Metaheuristics: From Design to Implementation; John Wiley & Sons: Hoboken, NJ, USA, 2009. [Google Scholar]
- Richter, H.; Engelbrecht, A. (Eds.) Recent Advances in the Theory and Application of Fitness Landscapes; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Tomassini, M.; Verel, S.; Ochoa, G. Complex-network analysis of combinatorial spaces: The NK landscape case. Phys. Rev. E
**2008**, 78, 066114. [Google Scholar] [CrossRef][Green Version] - Verel, S.; Ochoa, G.; Tomassini, M. Local optima networks of NK landscapes with neutrality. IEEE Trans. Evol. Comput.
**2011**, 15, 783–797. [Google Scholar] [CrossRef][Green Version] - Daolio, F.; Tomassini, M.; Vérel, S.; Ochoa, G. Communities of minima in local optima networks of combinatorial spaces. Phys. A Stat. Mech. Its Appl.
**2011**, 390, 1684–1694. [Google Scholar] [CrossRef][Green Version] - Hernando, L.; Daolio, F.; Veerapen, N.; Ochoa, G. Local optima networks of the permutation flowshop scheduling problem: Makespan vs. total flow time. In Proceedings of the 2017 IEEE Congress on Evolutionary Computation (CEC), Donostia, Spain, 5–8 June 2017; pp. 1964–1971. [Google Scholar]
- Tomassini, M. Complex networks analysis of the energy landscape of the low autocorrelation binary sequences problem. Phys. A Stat. Mech. Its Appl.
**2021**, 577, 126089. [Google Scholar] [CrossRef] - Verel, S.; Liefooghe, A.; Jourdan, L.; Clarisse, C. On the structure of multiobjective combinatorial search space: Mnk-landscapes with correlated objectives. Eur. J. Oper. Res.
**2013**, 227, 331–342. [Google Scholar] [CrossRef] - Wales, D. Energy Landscapes: Applications to Clusters, Biomolecules and Glasses; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Kucharik, M.; Hofacker, I.L.; Stadler, P.F.; Qin, J. Basin hopping graph: A computational framework to characterize RNA folding landscapes. Bioinformatics
**2014**, 30, 2009–2017. [Google Scholar] [CrossRef] [PubMed][Green Version] - Vinkó, T.; Gelle, K. Basin hopping networks of continuous global optimization problems. Cent. Eur. J. Oper. Res.
**2017**, 25, 985–1006. [Google Scholar] [CrossRef] - Contreras-Cruz, M.A.; Ochoa, G.; Ramirez-Paredes, J.P. Synthetic vs. real-world continuous landscapes: A local optima networks view. In Lecture Notes in Computer Science, Proceedings of the International Conference on Bioinspired Methods and Their Applications, Brussels, Belgium, 19–20 November 2016; Springer: Cham, Switzerland, 2020; pp. 3–16. [Google Scholar]
- Reidys, C.M.; Stadler, P.F. Combinatorial landscapes. SIAM Rev.
**2002**, 44, 3–54. [Google Scholar] [CrossRef] - Thomson, S.L.; Ochoa, G.; Verel, S. Clarifying the difference in local optima network sampling algorithms. In Lecture Notes in Computer Science, Proceedings of the European Conference on Evolutionary Computation in Combinatorial Optimization (Part of EvoStar), Leipzig, Germany, 24–26 April 2019; Springer: Cham, Switzerland, 2019; pp. 163–178. [Google Scholar]
- Locatelli, M. On the multilevel structure of global optimization problems. Comput. Optim. Appl.
**2005**, 30, 5–22. [Google Scholar] [CrossRef] - Wales, D.J.; Scheraga, H.A. Global optimization of clusters, crystals, and biomolecules. Science
**1999**, 285, 1368–1372. [Google Scholar] [CrossRef] [PubMed][Green Version] - Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P. Numerical Recipes 3rd Edition: The Art of Scientific Computing; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
- Bartz-Beielstein, T.; Doerr, C.; Berg, D.; Bossek, J.; Chandrasekaran, S.; Eftimov, T.; Fischbach, A.; Kerschke, P.; Cava, W.L.; Lopez-Ibanez, M.; et al. Benchmarking in optimization: Best practice and open issues. arXiv
**2020**, arXiv:2007.03488. [Google Scholar] - Storn, R.; Price, K. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim.
**1997**, 11, 341–359. [Google Scholar] [CrossRef] - Locatelli, M. A note on the griewank test function. J. Glob. Optim.
**2003**, 25, 169–174. [Google Scholar] [CrossRef] - Broyden, C.G. Quasi-newton methods and their application to function minimisation. Math. Comput.
**1967**, 21, 368–381. [Google Scholar] [CrossRef] - Newman, M. Networks; Oxford University Press: Oxford, UK, 2018. [Google Scholar]
- Albert, R.; Jeong, H.; Barabási, A.-L. Error and attack tolerance of complex networks. Nature
**2000**, 406, 378–382. [Google Scholar] [CrossRef] [PubMed][Green Version] - Brin, S.; Page, L. The anatomy of a large-scale hypertextual web search engine. Comput. Netw. Isdn Syst.
**1998**, 30, 107–117. [Google Scholar] [CrossRef] - Leopold, P.E.; Montal, M.; Onuchic, J. Protein folding funnels: A kinetic approach to the sequence-structure relationship. Proc. Natl. Acad. Sci. USA
**1992**, 89, 8721–8725. [Google Scholar] [CrossRef][Green Version] - Ochoa, G.; Veerapen, N. Mapping the global structure of TSP fitness landscapes. J. Heuristics
**2018**, 24, 265–294. [Google Scholar] [CrossRef][Green Version] - Hansen, N.; Auger, A.; Ros, R.; Mersmann, O.; Tušar, T.; Brockhoff, D. Coco: A platform for comparing continuous optimizers in a black-box setting. Optim. Methods Softw.
**2021**, 36, 114–144. [Google Scholar] [CrossRef] - Homolya, V.; Vinkó, T. Leveraging local optima network properties for memetic differential evolution. In Advances in Intelligent Systems and Computing, Proceedings of the World Congress on Global Optimization, Metz, France, 8–10 July 2019; Springer: Cham, Switzerland, 2019; pp. 109–118. [Google Scholar]
- Ballard, A.J.; Das, R.; Martiniani, S.; Mehta, D.; Sagun, L.; Stevenson, J.D.; Wales, D.J. Energy landscapes for machine learning. Phys. Chem. Chem. Phys.
**2017**, 19, 12585–12603. [Google Scholar] [CrossRef][Green Version]

**Figure 1.**Contour lines representation of the Branin function $B(x,y)$. In the given domain the function has four global minima named $0,1,2,3$ from left to right and located at $(-\pi ,12.275),(\pi ,2.275),(3\pi ,2.475),(5\pi ,12.875)$ with $B(x,y)\approx 0.39789$.

**Figure 2.**The LON corresponding to the function Branin shown in Figure 1. Vertex labels correspond to the optima and edge thickness is drawn proportional to the corresponding edge weight, which stands for the empirical frequency of transition between the corresponding minima basins.

**Figure 3.**Contour lines plots for two-dimensional Griewank (

**left**) and Rastrigin (

**right**) functions. Darker areas contain the local minima. Note the different region sizes.

**Figure 4.**Contour lines representation of the two-dimensional Schwefel function in the domain $x\in [-500,500],y\in [-500,500]$.

**Figure 5.**Incoming strength of LON nodes. (

**Left image**): Griewank function. (

**Right image**): Rastrigin function. Graph nodes are sorted by increasing objective function value.

**Figure 6.**Degree distribution function for incoming (green curve) and outgoing (blue curve) edges. (

**Left**): Griewank function; (

**Right**): Rastrigin function, both defined in two-dimensional space.

**Figure 7.**Empirical cumulative degree distribution function of all edges for the Rastrigin LON on double logarithmic scale. The distribution is well fitted by a power law shown by the line in red except near the tail cutoff, as usual for finite and relatively small networks.

**Figure 8.**PageRank centrality of all minima versus their objective function values as measured by the frequency of visits during a random walk. Node function values increase to the right of the x-axis. (

**Left picture**): Griewank network; (

**Right picture**): Rastrigin network.

**Figure 9.**From left to right: one-dimensional Griewank, Rastrigin, and Schwefel functions showing a single-funnel landscape (Griewank, Rastrigin), and a double funnel (Schwefel). Note the different axes scales.

**Figure 10.**The sampled LON of the Schwefel function in two dimensions. Edge weights are not shown for the sake of clarity. The node corresponding to the global minimum is in blue.

**Figure 11.**The LON of the Griewank function in five dimensions in the box ${[-10,\phantom{\rule{3.33333pt}{0ex}}10]}^{5}$. Self-loops, edge weights and node labels are not shown for the sake of clarity. The node corresponding to the global minimum is in red.

**Figure 12.**Degree distribution functions for the Griewank LON in five dimensions. Green curve: incoming edges; blue curve: outgoing edges.

**Figure 13.**Griewank function at dimension $n=5$. Nodes are sorted by increasing objective function value. (

**Left image**): incoming strength. (

**Right image**): node centrality according to the Page Rank algorithm.

**Figure 14.**Degree distribution functions for the Schwefel LON in five dimensions. Green curve: incoming edges; blue curve: outgoing edges.

**Figure 15.**The Page Rank centrality of nodes in the Schwefel LON. Nodes represent function minima and are ordered by increasing objective function value.

**Table 1.**Fraction of optimization runs that found the global minimum for the Griewank and Rastrigin functions in two dimensions for DE (left column) and BH (right column). The average is over 500 optimization runs in each case. A budget of $3.0\times {10}^{3}$ function evaluations was allowed per run and the search region is ${[-30,\phantom{\rule{3.33333pt}{0ex}}30]}^{2}$. The mean number of function evaluations when the global minimum has been found is in parentheses.

Success Rates | ||
---|---|---|

DE | BH | |

Griewank | $0.464\phantom{\rule{0.166667em}{0ex}}\left(2479\right)$ | $0.050\phantom{\rule{0.166667em}{0ex}}\left(1528\right)$ |

Rastrigin | $0.930\phantom{\rule{0.166667em}{0ex}}\left(2178\right)$ | $0.980\phantom{\rule{0.166667em}{0ex}}\left(928\right)$ |

**Table 2.**The fraction of optimization runs that found the global minimum for the Griewank and Rastrigin functions for dimension $n=5$ (left half of the table) and $n=10$ (right half) using DE and BH. The averages are over 500 optimization runs in each case. A budget of $3.0\times {10}^{4}$ function evaluations has been allocated per run and the search region are, respectively, ${[-30,30]}^{5}$ and ${[-30,30]}^{10}$. The mean number of function evaluations when the global minimum has been found is in parentheses.

Success Rates | ||||
---|---|---|---|---|

n = 5 | n = 10 | |||

DE | BH | DE | BH | |

Griewank | 0.157 (25,496) | 0.080 (18,575) | 0.016 (96,573) | 0.926 (75,314) |

Rastrigin | 0.818 (14,754) | 0.352 (14,945) | 0.496 (105,202) | 0.165 (107,571) |

**Table 3.**Typical distances in the Griewank network (top line) and the Rastrigin network (bottom line). First column: diameter (largest among all shortest paths). Second column: average shortest path computed according to Equation (2) in Section 3.2. Third column: average path length to the global optimum from all other minima.

Diameter | Av. Shortest Path | Av. Path to GO | |
---|---|---|---|

Griewank | $26.423$ | $5.247$ | $3.70$ |

Rastrigin | $33.333$ | $5.905$ | $2.93$ |

**Table 4.**Fraction of optimization runs that found the global minimum for the Schwefel function using DE for dimensions $n=2,5,10$. The averages are over 500 optimization runs in each case. The search region is ${[-500,\phantom{\rule{3.33333pt}{0ex}}500]}^{n}$. The allocated budget of function evaluations is $3.0\times {10}^{3},10.0\times {10}^{3}$, and $30.0\times {10}^{3}$ for $n=2,5$ and 10, respectively. The mean number of function evaluations when the global minimum has been found is in parentheses.

Success Rates | |||
---|---|---|---|

Schwefel | n = 2 | n = 5 | n = 10 |

0.910 (1115) | 0.866 (6265) | 0.675 (26,992) |

**Table 5.**Estimated number of local minima for Schwefel, Rastrigin, and Griewank functions for dimensions $n=2,5,10$ defined in $\mathbf{x}\in {[-500,500]}^{n}$, $\mathbf{x}\in {[-30,30]}^{n}$, and $\mathbf{x}\in {[-60,60]}^{n}$, respectively.

Number of Local Minima | |||
---|---|---|---|

n = 2 | n = 5 | n = 10 | |

Schwefel | ∼50 | ∼11,000 | >85,000 |

Rastrigin | ∼3600 | >100,000 | >100,000 |

Griewank | ∼528 | >95,000 | >6000 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tomassini, M. A Local Optima Network View of Real Function Fitness Landscapes. *Entropy* **2022**, *24*, 703.
https://doi.org/10.3390/e24050703

**AMA Style**

Tomassini M. A Local Optima Network View of Real Function Fitness Landscapes. *Entropy*. 2022; 24(5):703.
https://doi.org/10.3390/e24050703

**Chicago/Turabian Style**

Tomassini, Marco. 2022. "A Local Optima Network View of Real Function Fitness Landscapes" *Entropy* 24, no. 5: 703.
https://doi.org/10.3390/e24050703