# A Survey of Advances in Landscape Analysis for Optimisation

## Abstract

**:**

## 1. Introduction

## 2. Beyond Fitness Landscapes

#### 2.1. Multiobjective Fitness Landscapes

#### 2.2. Violation Landscapes

#### 2.3. Dynamic and Coupled Fitness Landscapes

#### 2.4. Error Landscapes

## 3. Advances in Landscape Analysis

#### 3.1. Techniques for Landscape Analysis

- Technique #: the name of the technique, citation and extensions (where the technique was adapted in subsequent studies).
- Year: the year the technique was first introduced in published form.
- Focus: refers to what is measured or predicted by the technique.
- Assumptions: any significant assumptions on which the technique is based.
- Description: summary of how the technique works.
- Result: describes the form of output produced by the technique (numerical, graphical, etc.).

#### 3.2. Sampling and Robustness of Measures

## 4. Applications of Landscape Analysis

#### 4.1. Understanding Complex Problems

#### 4.2. Understanding and Explaining Algorithm Behaviour

#### 4.3. Algorithm Performance Prediction

- Bischl et al. [113] used one-sided support vector regression to predict the best-performing algorithm from a portfolio of four numerical optimisation algorithms based on ELA features. They showed that the model was able to generalise on new problem instances and predict the optimal or close to optimal algorithm from the portfolio.
- Muñoz et al. [114] used a neural network regression model to predict the performance of a CMA-ES algorithm based on landscape features and algorithm parameters. Performance was measured in terms of the number of function evaluation required and they found that the model was able to predict the relative ranking values for given algorithm-parameter combinations effectively.
- Malan and Engelbrecht [115] used decision tree models to predict failure of seven variants on the particle swarm optimisation algorithm based on landscape features. The models of five of the algorithm variants achieved testing accuracy levels above 90%.
- Liefooghe et al. [30] used a random forest regression model to predict the performance of multiobjective optimisation algorithms in combinatorial optimisation based on a combination of landscape features and problem-specific features. They later developed a decision tree model for selecting the best performing algorithm out of three multiobjective algorithms [50]. Their model was able to predict the best performing algorithm in more than 98.4% of the cases.
- Jankovic and Doerr [116] proposed a random forest regression model for predicting the performance of CMA-ES algorithms based on ELA features in a fixed-budget setting. They obtained high-quality performance prediction by combining two regression models trained to predict target precision and the logarithm of the target precision.
- Thomson et al. [117] used random forest and linear regression models to predict algorithm performance for solving quadratic assignment problems based on landscape features derived from LON sampling. They found that random forest trees performed better at prediction than linear regression.

#### 4.4. Automated Algorithm Selection

- genetic algorithms: using the fitness distance correlation landscape measure to dynamically adjust the migration period in a distributed genetic algorithm [119], selecting a crossover operator based on fitness landscape properties [120], using fitness landscape features to estimate the optimal population size [121];
- differential evolution algorithms: adapting the strategy and adjusting the control parameters based on detected landscape modality [122,123], adapting the mutation strategy based on landscape features [124,125], algorithm configuration based on exploratory landscape features with an empirical performance model [126];
- selection of CMA-ES algorithm configuration using a trained model for predicting performance based on landscape features that was shown to outperform the default setting of CMA-ES [128];
- surrogate-assisted particle swarm optimisation, where fitness landscape analysis was used to select surrogate models [129]; and
- decomposition-based multiobjective evolutionary algorithms (MOEA/D), where the addition of landscape information improved the behaviour of the adaptive operator selection mechanism [130].

## 5. Opportunities for Further Research

## 6. Conclusions

## Funding

## Conflicts of Interest

## References

- Malan, K.M.; Engelbrecht, A.P. A survey of techniques for characterising fitness landscapes and some possible ways forward. Inf. Sci.
**2013**, 241, 148–163. [Google Scholar] [CrossRef][Green Version] - Wright, S. The Roles of Mutation, Inbreeding, Crossbreeding, and Selection in Evolution. In Proceedings of the Sixth International Congress on Genetics, Ithaca, NY, USA, 24–31 August 1932; pp. 356–366. [Google Scholar]
- Stadler, P.F. Fitness Landscapes. In Biological Evolution and Statistical Physics; Michael Lässig, A.V., Ed.; Lecture Notes in Physics; Springer: Berlin/Heidelberg, Germany, 2002; Volume 585, pp. 183–204. [Google Scholar]
- Verel, S.; Liefooghe, A.; Dhaenens, C. Set-based Multiobjective Fitness Landscapes: A Preliminary Study. In Proceedings of the 13th annual Conference on Genetic and Evolutionary Computation, Dublin, Ireland, 12–16 July 2011; pp. 769–776. [Google Scholar]
- Malan, K.M.; Oberholzer, J.F.; Engelbrecht, A.P. Characterising Constrained Continuous Optimisation Problems. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 25–28 May 2015; pp. 1351–1358. [Google Scholar]
- Hordijk, W.; Kauffman, S.A. Correlation analysis of coupled fitness landscapes. Complexity
**2005**, 10, 41–49. [Google Scholar] [CrossRef] - Richter, H. Evolutionary Optimization in Spatio-temporal Fitness Landscapes. In Parallel Problem Solving from Nature—PPSN IX; Springer: Berlin/Heidelberg, Germany, 2006; pp. 1–10. [Google Scholar] [CrossRef]
- Richter, H. Coupled map lattices as spatio-temporal fitness functions: Landscape measures and evolutionary optimization. Phys. D Nonlinear Phenom.
**2008**, 237, 167–186. [Google Scholar] [CrossRef] - Yazdani, D.; Nguyen, T.T.; Branke, J. Robust Optimization Over Time by Learning Problem Space Characteristics. IEEE Trans. Evol. Comput.
**2019**, 23, 143–155. [Google Scholar] [CrossRef][Green Version] - Richter, H. Codynamic Fitness Landscapes of Coevolutionary Minimal Substrates. In Proceedings of the 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China, 6–11 July 2014; pp. 2692–2699. [Google Scholar] [CrossRef][Green Version]
- De Jong, E.D. Objective Fitness Correlation. In Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation (GECCO ’07), London, UK, 7–11 July 2007; Association for Computing Machinery: New York, NY, USA, 2007; pp. 440–447. [Google Scholar] [CrossRef]
- Richter, H. Dynamic landscape models of coevolutionary games. Biosystems
**2017**, 153–154, 26–44. [Google Scholar] [CrossRef] [PubMed][Green Version] - Choromanska, A.; Henaff, M.; Mathieu, M.; Arous, G.B.; LeCun, Y. The Loss Surfaces of Multilayer Networks. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, São Paulo, Brazil, 21–25 June 2015; pp. 192–204. [Google Scholar]
- Im, D.J.; Tao, M.; Branson, K. An empirical analysis of the optimization of deep network loss surfaces. arXiv
**2016**, arXiv:1612.04010. [Google Scholar] - Li, H.; Xu, Z.; Taylor, G.; Studer, C.; Goldstein, T. Visualizing the Loss Landscape of Neural Nets. In Advances in Neural Information Processing Systems 31; Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R., Eds.; Curran Associates, Inc.: Duchess County, NY, USA, 2018; pp. 6389–6399. [Google Scholar]
- Bosman, A.S.; Engelbrecht, A.; Helbig, M. Search Space Boundaries in Neural Network Error Landscape Analysis. In Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI), Athens, Greece, 6–9 December 2016; pp. 1–8. [Google Scholar]
- Bosman, A.; Engelbrecht, A.; Helbig, M. Fitness Landscape Analysis of Weight-Elimination Neural Networks. Neural Process. Lett.
**2018**, 48, 353–373. [Google Scholar] [CrossRef] - Bosman, A.S.; Engelbrecht, A.; Helbig, M. Loss Surface Modality of Feed-Forward Neural Network Architectures. arXiv
**2019**, arXiv:1905.10268. [Google Scholar] - Bosman, A.S.; Engelbrecht, A.; Helbig, M. Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions. Neurocomputing
**2020**, 400, 113–136. [Google Scholar] [CrossRef][Green Version] - Pitzer, E.; Affenzeller, M. A Comprehensive Survey on Fitness Landscape Analysis. In Recent Advances in Intelligent Engineering Systems; Springer: Berlin/Heidelberg, Germany, 2012; pp. 161–191. [Google Scholar] [CrossRef]
- Goldberg, D.E. Simple Genetic Algorithms and the Minimal Deceptive Problem. In Genetic Algorithms and Simulated Annealing; Davis, L., Ed.; Pitman: London, UK, 1987; Chapter 6; pp. 74–88. [Google Scholar]
- Lu, G.; Li, J.; Yao, X. Fitness-Probability Cloud and a Measure of Problem Hardness for Evolutionary Algorithms. In Evolutionary Computation in Combinatorial Optimization; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; Volume 6622, pp. 108–117. [Google Scholar]
- Ochoa, G.; Tomassini, M.; Vérel, S.; Darabos, C. A Study of NK Landscapes’ Basins and Local Optima Networks. In Proceedings of the Genetic and Evolutionary Computation Conference, Atlanta, GA, USA, 12–16 July 2008; pp. 555–562. [Google Scholar]
- Vérel, S.; Ochoa, G.; Tomassini, M. The Connectivity of NK Landscapes’ Basins: A Network Analysis. In Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems, Winchester, France, 5–8 August 2008; pp. 648–655. [Google Scholar]
- Tomassini, M.; Vérel, S.; Ochoa, G. Complex-network analysis of combinatorial spaces: The NK landscape case. Phys. Rev. E
**2008**, 78, 066114. [Google Scholar] [CrossRef] [PubMed][Green Version] - Ochoa, G.; Vérel, S.; Tomassini, M. First-Improvement vs. Best-Improvement Local Optima Networks of NK Landscapes. In Parallel Problem Solving from Nature—PPSN XI; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2010; Volume 6238, pp. 104–113. [Google Scholar]
- Vérel, S.; Daolio, F.; Ochoa, G.; Tomassini, M. Local Optima Networks with Escape Edges. In Artificial Evolution; Springer: Berlin/Heidelberg, Germany, 2011; pp. 49–60. [Google Scholar]
- Vérel, S.; Ochoa, G.; Tomassini, M. Local Optima Networks of NK Landscapes With Neutrality. IEEE Trans. Evol. Comput.
**2011**, 15, 783–797. [Google Scholar] [CrossRef][Green Version] - Herrmann, S.; Ochoa, G.; Rothlauf, F. Coarse-Grained Barrier Trees of Fitness Landscapes. In Parallel Problem Solving from Nature—PPSN XIV; Springer International Publishing: New York, NY, USA, 2016; pp. 901–910. [Google Scholar] [CrossRef][Green Version]
- Liefooghe, A.; Derbel, B.; Vérel, S.; López-Ibáñez, M.; Aguirre, H.; Tanaka, K. On Pareto Local Optimal Solutions Networks. In Proceedings of the International Conference on Parallel Problem Solving from Nature, Coimbra, Portugal, 8–12 September 2018; Springer: Berlin/Heidelberg, Germany; pp. 232–244. [Google Scholar]
- Fieldsend, J.E.; Alyahya, K. Visualising the Landscape of Multi-objective Problems Using Local Optima Networks. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’19), Prague, Czech Republic, 13–17 July 2019; ACM: New York, NY, USA, 2019; pp. 1421–1429. [Google Scholar]
- Iclanzan, D.; Daolio, F.; Tomassini, M. Data-driven Local Optima Network Characterization of QAPLIB Instances. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO ’14), Vancouver, BC, Canada, 12–16 July 2014; pp. 453–460. [Google Scholar]
- Verel, S.; Daolio, F.; Ochoa, G.; Tomassini, M. Sampling Local Optima Networks of Large Combinatorial Search Spaces: The QAP Case. In Parallel Problem Solving from Nature—PPSN XV; Springer: Cham, Switzerland, 2018; pp. 257–268. [Google Scholar] [CrossRef][Green Version]
- Adair, J.; Ochoa, G.; Malan, K.M. Local Optima Networks for Continuous Fitness Landscapes. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’19), Prague, Czech Republic, 13–17 July 2019; pp. 1407–1414. [Google Scholar]
- Ochoa, G.; Vérel, S.; Daolio, F.; Tomassini, M. Local Optima Networks: A New Model of Combinatorial Fitness Landscapes. In Recent Advances in the Theory and Application of Fitness Landscapes; Richter, H., Engelbrecht, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2014; pp. 233–262. [Google Scholar]
- Ochoa, G.; Veerapen, N. Mapping the global structure of TSP fitness landscapes. J. Heuristics
**2018**, 24, 265–294. [Google Scholar] [CrossRef][Green Version] - Thomson, S.L.; Ochoa, G.; Verel, S. Clarifying the Difference in Local Optima Network Sampling Algorithms. In Evolutionary Computation in Combinatorial Optimization; Springer International Publishing: New York, NY, USA, 2019; pp. 163–178. [Google Scholar] [CrossRef][Green Version]
- Herrmann, S.; Rothlauf, F. Predicting Heuristic Search Performance with PageRank Centrality in Local Optima Networks. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2015, Madrid, Spain, 11–15 July 2015; pp. 401–408. [Google Scholar] [CrossRef]
- Mersmann, O.; Bischl, B.; Trautmann, H.; Preuss, M.; Weihs, C.; Rudolph, G. Exploratory Landscape Analysis. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO ’11), Dublin, Ireland, 12–16 July 2011; pp. 829–836. [Google Scholar] [CrossRef]
- Kerschke, P.; Preuss, M.; Hernández, C.; Schütze, O.; Sun, J.Q.; Grimme, C.; Rudolph, G.; Bischl, B.; Trautmann, H. Cell Mapping Techniques for Exploratory Landscape Analysis. In Advances in Intelligent Systems and Computing; Springer International Publishing: Berlin/Heidelberg, Germany, 2014; pp. 115–131. [Google Scholar] [CrossRef]
- Kerschke, P.; Trautmann, H. The R-Package FLACCO for Exploratory Landscape Analysis with Applications to Multi-objective Optimization Problems. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, Canada, 24–29 July 2016; pp. 5262–5269. [Google Scholar] [CrossRef]
- Kerschke, P.; Trautmann, H. Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package Flacco. In Studies in Classification, Data Analysis, and Knowledge Organization; Springer International Publishing: Berlin/Heidelberg, Germany, 2019; pp. 93–123. [Google Scholar] [CrossRef][Green Version]
- Morgan, R.; Gallagher, M. Length Scale for Characterising Continuous Optimization Problems. In Proceedings of the 12th International Conference on Parallel Problem Solving from Nature—Part I, Taormina, Italy, 1–5 September 2012; pp. 407–416. [Google Scholar]
- Morgan, R.; Gallagher, M. Analysing and characterising optimization problems using length scale. Soft Comput.
**2015**, 21, 1735–1752. [Google Scholar] [CrossRef] - Caraffini, F.; Neri, F.; Picinali, L. An analysis on separability for Memetic Computing automatic design. Inf. Sci.
**2014**, 265, 1–22. [Google Scholar] [CrossRef] - Malan, K.M.; Engelbrecht, A.P. A Progressive Random Walk Algorithm for Sampling Continuous Fitness Landscapes. In Proceedings of the IEEE Congress on Evolutionary Computation, Beijing, China, 6–11 July 2014; pp. 2507–2514. [Google Scholar]
- Shirakawa, S.; Nagao, T. Bag of local landscape features for fitness landscape analysis. Soft Comput.
**2016**, 20, 3787–3802. [Google Scholar] [CrossRef] - Sun, Y.; Kirley, M.; Halgamuge, S.K. Quantifying Variable Interactions in Continuous Optimization Problems. IEEE Trans. Evol. Comput.
**2017**, 21, 249–264. [Google Scholar] [CrossRef] - Wang, M.; Li, B.; Zhang, G.; Yao, X. Population Evolvability: Dynamic Fitness Landscape Analysis for Population-Based Metaheuristic Algorithms. IEEE Trans. Evol. Comput.
**2018**, 22, 550–563. [Google Scholar] [CrossRef] - Liefooghe, A.; Daolio, F.; Verel, S.; Derbel, B.; Aguirre, H.; Tanaka, K. Landscape-Aware Performance Prediction for Evolutionary Multi-objective Optimization. IEEE Trans. Evol. Comput.
**2019**, 1. [Google Scholar] [CrossRef] - Verel, S.; Liefooghe, A.; Jourdan, L.; Dhaenens, C. On the structure of multiobjective combinatorial search space: MNK-landscapes with correlated objectives. Eur. J. Oper. Res.
**2013**, 227, 331–342. [Google Scholar] [CrossRef] - Liefooghe, A.; Verel, S.; Aguirre, H.; Tanaka, K. What Makes an Instance Difficult for Black-Box 0–1 Evolutionary Multiobjective Optimizers? In Artificial Evolution; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2014; Volume 8752, pp. 3–15. [Google Scholar]
- Bosman, A.S.; Engelbrecht, A.P.; Helbig, M. Progressive Gradient Walk for Neural Network Fitness Landscape Analysis. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’18), Kyoto, Japan, 15–19 July 2018; pp. 1473–1480. [Google Scholar]
- Morgan, R.; Gallagher, M. Sampling Techniques and Distance Metrics in High Dimensional Continuous Landscape Analysis: Limitations and Improvements. IEEE Trans. Evol. Comput.
**2014**, 18, 456–461. [Google Scholar] [CrossRef][Green Version] - Renau, Q.; Doerr, C.; Dreo, J.; Doerr, B. Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy. In Parallel Problem Solving from Nature—PPSN XVI; Springer International Publishing: New York, NY, USA, 2020; pp. 139–153. [Google Scholar] [CrossRef]
- Saleem, S.; Gallagher, M.; Wood, I. Direct Feature Evaluation in Black-Box Optimization Using Problem Transformations. Evol. Comput.
**2019**, 27, 75–98. [Google Scholar] [CrossRef] [PubMed] - Muñoz, M.A.; Kirley, M.; Smith-Miles, K. Analyzing Randomness Effects on the Reliability of Landscape Analysis. 2020. Available online: https://www.researchgate.net/publication/325483674_Analyzing_randomness_effects_on_the_reliability_of_Landscape_Analysis (accessed on 15 November 2020). [CrossRef]
- Pitzer, E.; Beham, A.; Affenzeller, M. Generic Hardness Estimation Using Fitness and Parameter Landscapes Applied to Robust Taboo Search and the Quadratic Assignment Problem. In Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation, Philadelphia, PA, USA, 12–16 July 2012; pp. 393–400. [Google Scholar]
- Muñoz, M.A.; Kirley, M.; Halgamuge, S.K. Exploratory Landscape Analysis of Continuous Space Optimization Problems Using Information Content. IEEE Trans. Evol. Comput.
**2015**, 19, 74–87. [Google Scholar] [CrossRef] - Moser, I.; Gheorghita, M. Combining Search Space Diagnostics and Optimisation. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012. [Google Scholar] [CrossRef]
- Malan, K.M. Landscape-Aware Constraint Handling Applied to Differential Evolution. In Theory and Practice of Natural Computing; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2018; Volume 11324, pp. 176–187. [Google Scholar]
- Janković, A.; Doerr, C. Adaptive Landscape Analysis. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Prague, Czech Republic, 13–17 July 2019. [Google Scholar] [CrossRef]
- Muñoz, M.A.; Kirley, M.; Halgamuge, S.K. Landscape Characterization of Numerical Optimization Problems Using Biased Scattered Data. In Proceedings of the IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012; pp. 1–8. [Google Scholar]
- Beham, A.; Pitzer, E.; Wagner, S.; Affenzeller, M. Integrating Exploratory Landscape Analysis into Metaheuristic Algorithms. In Computer Aided Systems Theory—EUROCAST 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2018; Volume 10671, pp. 473–480. [Google Scholar]
- Muñoz, M.A.; Smith-Miles, K. Effects of Function Translation and Dimensionality Reduction on Landscape Analysis. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC), Sendai, Japan, 12–14 June 2015; pp. 1336–1342. [Google Scholar]
- Škvorc, U.; Eftimov, T.; Korošec, P. Understanding the problem space in single-objective numerical optimization using exploratory landscape analysis. Appl. Soft Comput. J.
**2020**, 90, 106138. [Google Scholar] [CrossRef] - Scott, E.O.; Jong, K.A.D. Landscape Features for Computationally Expensive Evaluation Functions: Revisiting the Problem of Noise. In Parallel Problem Solving from Nature—PPSN XIV; Springer: Cham, Switzerland, 2016; pp. 952–961. [Google Scholar] [CrossRef]
- Werth, B.; Pitzer, E.; Affenzeller, M. Surrogate-Assisted Fitness Landscape Analysis for Computationally Expensive Optimization. In Computer Aided Systems Theory – EUROCAST 2019; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2020; pp. 247–254. [Google Scholar]
- Daolio, F.; Vérel, S.; Ochoa, G.; Tomassini, M. Local Optima Networks of the Quadratic Assignment Problem. In Proceedings of the IEEE Congress on Evolutionary Computation, Barcelona, Spain, 18–23 July 2010; pp. 1–8. [Google Scholar]
- Chicano, F.; Daolio, F.; Ochoa, G.; Vérel, S.; Tomassini, M.; Alba, E. Local Optima Networks, Landscape Autocorrelation and Heuristic Search Performance. In Parallel Problem Solving from Nature—PPSN XII; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2012; Volume 7492, pp. 337–347. [Google Scholar]
- Tayarani-Najaran, M.H.; Prügel-Bennett, A. Quadratic assignment problem: A landscape analysis. Evol. Intell.
**2015**, 8, 165–184. [Google Scholar] [CrossRef] - Prügel-Bennett, A.; Tayarani-Najaran, M. Maximum Satisfiability: Anatomy of the Fitness Landscape for a Hard Combinatorial Optimization Problem. IEEE Trans. Evol. Comput.
**2012**, 16, 319–338. [Google Scholar] [CrossRef][Green Version] - Ochoa, G.; Chicano, F. Local Optima Network Analysis for MAX-SAT. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Prague, Czech Republic, 13–17 July 2019; pp. 1430–1437. [Google Scholar] [CrossRef]
- Daolio, F.; Vérel, S.; Ochoa, G.; Tomassini, M. Local Optima Networks of the Permutation Flow-Shop Problem. Revised Selected Papers. In Artificial Evolution—EA 2013; Springer: Berlin/Heidelberg, Germany, 2013; Volume 8752, pp. 41–52. [Google Scholar]
- Hernando, L.; Daolio, F.; Veerapen, N.; Ochoa, G. Local Optima Networks of the Permutation Flowshop Scheduling Problem: Makespan vs Total Flow Time. In Proceedings of the IEEE Congress on Evolutionary Computation—CEC 2017, San Sebastian, Spain, 5–8 June 2017; pp. 1964–1971. [Google Scholar]
- Baioletti, M.; Santucci, V. Fitness Landscape Analysis of the Permutation Flowshop Scheduling Problem with Total Flow Time Criterion. In Computational Science and Its Applications—ICCSA 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2017; pp. 705–716. [Google Scholar]
- Morgan, R.; Gallagher, M. Fitness Landscape Analysis of Circles in a Square Packing Problems; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2014; pp. 455–466. [Google Scholar] [CrossRef]
- Alyahya, K.; Rowe, J.E. Landscape Analysis of a Class of NP-Hard Binary Packing Problems. Evol. Comput.
**2019**, 27, 47–73. [Google Scholar] [CrossRef][Green Version] - Ochoa, G.; Veerapen, N.; Whitley, D.; Burke, E.K. The Multi-Funnel Structure of TSP Fitness Landscapes: A Visual Exploration. In Artificial Evolution—EA 2015; Lecture Notes in Computer Science; Revised Selected Papers; Springer: Berlin/Heidelberg, Germany, 2015; Volume 9554, pp. 1–13. [Google Scholar]
- Veerapen, N.; Ochoa, G.; Tinós, R.; Whitley, D. Tunnelling Crossover Networks for the Asymmetric TSP. In Parallel Problem Solving from Nature—PPSN XIV; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2016; Volume 9921, pp. 994–1003. [Google Scholar]
- Ochoa, G.; Veerapen, N. Deconstructing the Big Valley Search Space Hypothesis. In Evolutionary Computation in Combinatorial Optimization; Springer: Cham, Switzerland, 2016; pp. 58–73. [Google Scholar] [CrossRef][Green Version]
- Tayarani-N., M.H.; Prügel-Bennett, A. An Analysis of the Fitness Landscape of Travelling Salesman Problem. Evol. Comput.
**2016**, 24, 347–384. [Google Scholar] [CrossRef][Green Version] - Tayarani-N., M.H.; Prügel-Bennett, A. Anatomy of the fitness landscape for dense graph-colouring problem. Swarm Evol. Comput.
**2015**, 22, 47–65. [Google Scholar] [CrossRef] - Ochoa, G.; Veerapen, N.; Daolio, F.; Tomassini, M. Understanding Phase Transitions with Local Optima Networks: Number Partitioning as a Case Study. In Evolutionary Computation in Combinatorial Optimization—EvoCOP; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2017; Volume 10197, pp. 233–248. [Google Scholar]
- Ventresca, M.; Ombuki-Berman, B.; Runka, A. Predicting Genetic Algorithm Performance on the Vehicle Routing Problem Using Information Theoretic Landscape Measures. In Evolutionary Computation in Combinatorial Optimization; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2013; pp. 214–225. [Google Scholar]
- Yafrani, M.E.; Martins, M.S.R.; Krari, M.E.; Wagner, M.; Delgado, M.R.B.S.; Ahiod, B.; Lüders, R. A Fitness Landscape Analysis of the Travelling Thief Problem. In Proceedings of the Genetic and Evolutionary Computation Conference, Kyoto, Japan, 15–19 July 2018; pp. 277–284. [Google Scholar]
- Caamaño, P.; Bellas, F.; Becerra, J.A.; Díaz, V.; Duro, R.J. Experimental Analysis of the Relevance of Fitness Landscape Topographical Characterization. In Proceedings of the IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012; pp. 1–8. [Google Scholar]
- Rodriguez-Maya, N.; Flores, J.J.; Graff, M. Predicting the RCGA Performance for the University Course Timetabling Problem. In Intelligent Computing Systems; Springer: Cham, Switzerland, 2016; pp. 31–45. [Google Scholar] [CrossRef]
- Haraldsson, S.O.; Woodward, J.R.; Brownlee, A.E.I.; Smith, A.V.; Gudnason, V. Genetic Improvement of Runtime and Its Fitness Landscape in a Bioinformatics Application. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’17), Berlin, Germany, 15–19 July 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 1521–1528. [Google Scholar] [CrossRef]
- Langdon, W.B.; Veerapen, N.; Ochoa, G. Visualising the Search Landscape of the Triangle Program. In European Conference on Genetic Programming—EuroGP 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2017; Volume 10196, pp. 96–113. [Google Scholar]
- Veerapen, N.; Daolio, F.; Ochoa, G. Modelling Genetic Improvement Landscapes with Local Optima Networks. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Berlin, Germany, 15–19 July 2017. [Google Scholar] [CrossRef]
- Aleti, A.; Moser, I.; Grunske, L. Analysing the Fitness Landscape of Search-based Software Testing Problems. Autom. Softw. Eng.
**2017**, 24, 603–621. [Google Scholar] [CrossRef] - Albunian, N.; Fraser, G.; Sudholt, D. Causes and Effects of Fitness Landscapes in Unit Test Generation. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, Cancún, Mexico, 8–12 July 2020; pp. 1204–1212. [Google Scholar] [CrossRef]
- Simoncini, D.; Barbe, S.; Schiex, T.; Verel, S. Fitness Landscape Analysis Around the Optimum in Computational Protein Design. In Proceedings of the Genetic and Evolutionary Computation Conference, Kyoto, Japan, 15–19 July 2018. [Google Scholar] [CrossRef]
- Jakobovic, D.; Picek, S.; Martins, M.S.R.; Wagner, M. A Characterisation of S-Box Fitness Landscapes in Cryptography. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’19), Prague, Czech Republic, 13–17 July 2019; Association for Computing Machinery: New York, NY, USA, 2019; pp. 285–293. [Google Scholar] [CrossRef][Green Version]
- Harrison, K.R.; Ombuki-Berman, B.M.; Engelbrecht, A.P. The Parameter Configuration Landscape: A Case Study on Particle Swarm Optimization. In Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand, 10–13 June 2019; pp. 808–814. [Google Scholar] [CrossRef]
- Waibel, C.; Mavromatidis, G.; Evins, R.; Carmeliet, J. A comparison of building energy optimization problems and mathematical test functions using static fitness landscape analysis. J. Build. Perform. Simul.
**2019**, 12, 789–811. [Google Scholar] [CrossRef] - van Aardt, W.A.; Bosman, A.S.; Malan, K.M. Characterising Neutrality in Neural Network Error Landscapes. In Proceedings of the 2017 IEEE Congress on Evolutionary Computation (CEC), San Sebastian, Spain, 5–8 June 2017; pp. 1374–1381. [Google Scholar]
- Mostert, W.; Malan, K.; Engelbrecht, A. Filter Versus Wrapper Feature Selection Based on Problem Landscape Features. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Kyoto, Japan, 15–19 July 2018; pp. 1489–1496. [Google Scholar] [CrossRef]
- Mostert, W.; Malan, K.M.; Ochoa, G.; Engelbrecht, A.P. Insights into the Feature Selection Problem Using Local Optima Networks. In Evolutionary Computation in Combinatorial Optimization; Springer: Cham, Switzerland, 2019; pp. 147–162. [Google Scholar] [CrossRef]
- Stapelberg, B.; Malan, K.M. Global Structure of Policy Search Spaces for Reinforcement Learning. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Prague, Czech Republic, 13–17 July 2019. [Google Scholar] [CrossRef]
- Pimenta, C.G.; de Sá, A.G.C.; Ochoa, G.; Pappa, G.L. Fitness Landscape Analysis of Automated Machine Learning Search Spaces. In Evolutionary Computation in Combinatorial Optimization; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2020; Volume 12102, pp. 114–130. [Google Scholar] [CrossRef]
- Rodrigues, N.M.; Silva, S.; Vanneschi, L. A Study of Generalization and Fitness Landscapes for Neuroevolution. IEEE Access
**2020**, 8, 108216–108234. [Google Scholar] [CrossRef] - Rodrigues, N.M.; Silva, S.; Vanneschi, L. A Study of Fitness Landscapes for Neuroevolution. In Proceedings of the IEEE Congress on Evolutionary Computation, Glasgow, UK, 19–24 July 2020. [Google Scholar]
- Watson, J.P. An Introduction to Fitness Landscape Analysis and Cost Models for Local Search. In Handbook of Metaheuristics; Gendreau, M., Potvin, J.Y., Eds.; Springer: Boston, MA, USA, 2010; pp. 599–623. [Google Scholar]
- Tari, S.; Basseur, M.; Goëffon, A. Sampled Walk and Binary Fitness Landscapes Exploration. In Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2018; pp. 47–57. [Google Scholar] [CrossRef]
- Wu, Y.; McCall, J.; Corne, D. Fitness Landscape Analysis of Bayesian Network Structure Learning. In Proceedings of the 2011 IEEE Congress of Evolutionary Computation (CEC), New Orleans, LA, USA, 5–8 June 2011; pp. 981–988. [Google Scholar]
- Nguyen, Q.U.; Nguyen, X.H.; O’Neill, M. Examining the Landscape of Semantic Similarity Based Mutation. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO ’11), Dublin, Ireland, 12–16 July 2011; pp. 1363–1370. [Google Scholar]
- Nguyen, Q.U.; Truong, C.D.; Nguyen, X.H.; O’Neill, M. Guiding Function Set Selection in Genetic Programming Based on Fitness Landscape Analysis. In Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO ’13 Companion, Amsterdam, The Netherlands, 6–10 July 2013; pp. 149–150. [Google Scholar]
- Daolio, F.; Liefooghe, A.; Verel, S.; Aguirre, H.; Tanaka, K. Global vs Local Search on Multi-Objective NK-Landscapes: Contrasting the Impact of Problem Features. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15, Madrid, Spain, 11–15 July 2015; pp. 369–376. [Google Scholar]
- Medvet, E.; Daolio, F.; Tagliapietra, D. Evolvability in Grammatical Evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17), Berlin, Germany, 15–19 July 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 977–984. [Google Scholar] [CrossRef]
- Thomson, S.L.; Ochoa, G.; Daolio, F.; Veerapen, N. The Effect of Landscape Funnels in QAPLIB Instances. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Berlin, Germany, 15–19 July 2017. [Google Scholar] [CrossRef][Green Version]
- Bischl, B.; Mersmann, O.; Trautmann, H.; Preuß, M. Algorithm Selection Based on Exploratory Landscape Analysis and Cost-sensitive Learning. In Proceedings of the Genetic and Evolutionary Computation Conference, Philadelphia, PA, USA, 13–27 July 2012; pp. 313–320. [Google Scholar]
- Muñoz, M.A.; Kirley, M.; Halgamuge, S.K. A Meta-learning Prediction Model of Algorithm Performance for Continuous Optimization Problems. In Parallel Problem Solving from Nature—PPSN XII; Lecture Notes in Computer Science; Springer: Berlin, Heidelberg, 2012; pp. 226–235. [Google Scholar]
- Malan, K.M.; Engelbrecht, A.P. Particle Swarm Optimisation Failure Prediction Based on Fitness Landscape Characteristics. In Proceedings of the 2014 IEEE Symposium on Swarm Intelligence, Orlando, FL, USA, 9–12 December 2014; pp. 1–9. [Google Scholar]
- Jankovic, A.; Doerr, C. Landscape-Aware Fixed-Budget Performance Regression and Algorithm Selection for Modular CMA-ES Variants. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, Cancún, Mexico, 8–12 July 2020; Association for Computing Machinery: New York, NY, USA, 2020; pp. 841–849. [Google Scholar] [CrossRef]
- Thomson, S.L.; Ochoa, G.; Verel, S.; Veerapen, N. Inferring Future Landscapes: Sampling the Local Optima Level. Evol. Comput.
**2020**, 28, 1–22. [Google Scholar] [CrossRef] - Kerschke, P.; Hoos, H.H.; Neumann, F.; Trautmann, H. Automated Algorithm Selection: Survey and Perspectives. Evol. Comput.
**2019**, 27, 3–45. [Google Scholar] [CrossRef] [PubMed] - Salto, C.; Alba, E.; Luna, F. Using Landscape Measures for the Online Tuning of Heterogeneous Distributed Gas. In Proceedings of the 13th Annual Conference Companion on Genetic and Evolutionary Computation, Dublin, Ireland, 12–16 July 2011; pp. 691–694. [Google Scholar]
- Picek, S.; Jakobovic, D. From Fitness Landscape to Crossover Operator Choice. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO ’14), Vancouver, BC, Canada, 12–16 July 2014; pp. 815–822. [Google Scholar]
- Gibbs, M.S.; Maier, H.R.; Dandy, G.C. Using characteristics of the optimisation problem to determine the Genetic Algorithm population size when the number of evaluations is limited. Environ. Model. Softw.
**2015**, 69, 226–239. [Google Scholar] [CrossRef] - Takahama, T.; Sakai, S. Differential Evolution with Dynamic Strategy and Parameter Selection by Detecting Landscape Modality. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012. [Google Scholar]
- Takahama, T.; Sakai, S. Large Scale Optimization by Differential Evolution with Landscape Modality Detection and a Diversity Archive. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation, Brisbane, Australia, 10–15 June 2012. [Google Scholar]
- Sallam, K.M.; Elsayed, S.M.; Sarker, R.A.; Essam, D.L. Differential Evolution with Landscape-Based Operator Selection for Solving Numerical Optimization Problems. In Proceedings in Adaptation, Learning and Optimization; Springer: Cham, Switzerland, 2016; pp. 371–387. [Google Scholar] [CrossRef]
- Sallam, K.M.; Elsayed, S.M.; Sarker, R.A.; Essam, D.L. Landscape-assisted multi-operator differential evolution for solving constrained optimization problems. Expert Syst. Appl.
**2020**, 162, 113033. [Google Scholar] [CrossRef] - Belkhir, N.; Dréo, J.; Savéant, P.; Schoenauer, M. Feature Based Algorithm Configuration: A Case Study with Differential Evolution. In Parallel Problem Solving from Nature—PPSN XIV; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2016; Volume 9921, pp. 156–166. [Google Scholar]
- Consoli, P.A.; Mei, Y.; Minku, L.L.; Yao, X. Dynamic selection of evolutionary operators based on online learning and fitness landscape analysis. Soft Comput.
**2016**, 20, 3889–3914. [Google Scholar] [CrossRef][Green Version] - Belkhir, N.; Dréo, J.; Savéant, P.; Schoenauer, M. Per Instance Algorithm Configuration of CMA-ES with Limited Budget. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17), Berlin, Germany, 15–19 July 2017; pp. 681–688. [Google Scholar]
- Yu, H.; Tan, Y.; Sun, C.; Zeng, J.; Jin, Y. An Adaptive Model Selection Strategy for Surrogate-assisted Particle Swarm Optimization Algorithm. In Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI), Athens, Greece, 6–9 December 2016. [Google Scholar]
- Kuk, J.; Goncalves, R.; Pozo, A. Combining Fitness Landscape Analysis and Adaptive Operator Selection in Multi and Many-Objective Optimization. In Proceedings of the 2019 8th Brazilian Conference on Intelligent Systems (BRACIS), Salvador, Brazil, 15–18 October 2019. [Google Scholar] [CrossRef]
- Beham, A.; Affenzeller, M.; Wagner, S. Instance-based Algorithm Selection on Quadratic Assignment Problem Landscapes. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, Berlin, Germany, 15–19 July 2017; pp. 1471–1478. [Google Scholar]
- Beham, A.; Wagner, S.; Affenzeller, M. Algorithm Selection on Generalized Quadratic Assignment Problem Landscapes. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’18), Kyoto, Japan, 15–19 July 2018; pp. 253–260. [Google Scholar]
- Bożejko, W.; Gnatowski, A.; Niżyński, T.; Affenzeller, M.; Beham, A. Local Optima Networks in Solving Algorithm Selection Problem for TSP. In Contemporary Complex Systems and Their Dependability; Advances in Intelligent Systems and Computing; Springer: Cham, Switzerland, 2018; Volume 761, pp. 83–93. [Google Scholar]
- Pavelski, L.M.; Delgado, M.R.; Kessaci, M.É. Meta-learning on Flowshop using Fitness Landscape Analysis. In Proceedings of the Genetic and Evolutionary Computation Conference, Prague, Czech Republic, 13–17 July 2019. [Google Scholar] [CrossRef][Green Version]

Technique 23: | Local optima networks (LONs) by Ochoa et al. [23] with extensions [24,25,26,27,28,29,30,31]. |

Year: | 2008 |

Focus: | Global landscape structure |

Assumptions: | Requires a complete enumeration of a discrete search space. Later extensions are based on samples to produce approximate LONs [32,33] and are adapted for continuous spaces [34]. |

Description: | A LON is a graph-based abstraction of the search space representing the global structure, where each node of the LON is a local optimum and edges between nodes represent adjacency of the basins of optima (the possibility of search transitioning from one local optimum to another). The LON model is also extended to multiobjective problems to form pareto local optimal solutions networks (PLOS-nets) [30,31]. More detail and resources on LONs can be found on the website: http://lonmaps.com. |

Result: | A graph visualisation showing the connectivity between local optima. Metrics can also be extracted from LONs such as number of optima, size of basins of attraction, shortest path to the global optimum [35], as well as funnel metrics [36,37], and PageRank centrality [38]. |

Technique 24: | Exploratory landscape analysis (ELA) by Mersmann et al. [39] with extensions [40]. |

Year: | 2011 |

Focus: | Low-level features based on small samples |

Assumptions: | Assumes a continuous search space. |

Description: | Based on a small sample of random solutions (using Latin Hypercube sampling), six classes of low level features are defined: (1) convexity, (2) y-distribution, (3) levelset, (4) meta-model, (5) local search, and (6) curvature. Features are estimations of attributes such as the probability of the objective function being linear, the skewness of the distribution of the function values, the accuracy of fitted meta models, the number of local optima identified by local search, estimated numerical gradient and so on. The standard ELA feature set was later extended to include features based on general cell mapping (GCM) [40], but these are currently limited to low-dimensional spaces. ELA is supported by an online package in R, called flacco [41,42] (https://github.com/kerschke/flacco). |

Result: | 50 numerical values for the standard ELA feature set and a further 44 values for GCM features. |

Technique 25: | Length scale distribution by Morgan and Gallagher [43] with extensions [44]. |

Year: | 2012 |

Focus: | Variation in gradient estimations across the search space |

Assumptions: | Assumes a distance metric in solution space. |

Description: | Based on a sample of solutions from a random Levy walk through the search space, the length scale (the absolute difference in fitness over the distance in space) is calculated for each pair of solutions in the sample. The length scale distribution is defined as the probability density function of length scales and is estimated using kernel density estimation on the sample of length scales. |

Result: | Plot of length scale distribution and a single value for the estimated entropy of the length scale distribution. |

Technique 26: | Codynamic landscape measures by Richter [10]. |

Year: | 2014 |

Focus: | Similarity between the objective and subjective landscapes in coevolution |

Assumptions: | Assumes a model of coevolution for fast evaluation of subjective and objective fitness values. |

Description: | Given a sample of points in the search space and two coupled and codynamic landscapes: the objective landscape (the fitness landscape of the problem) and the subjective landscape (how the coevolution perceives the problem) landscape measures are defined to quantify differences between the landscapes. |

Result: | Three numeric values at each generation, quantifying different aspects of similarity. |

Technique 27: | Degree of separability by Caraffini et al. [45]. |

Year: | 2014 |

Focus: | Nonseparability |

Assumptions: | Assumes a continuous search space and the use of the covariance matrix adaptation evolution strategy (CMA-ES) search algorithm. |

Description: | A portion of the budget of the CMA-ES algorithm is executed on the problem. After a limited number of generations, the matrix $\mathbf{C}$ evolves to estimate the covariance matrix describing the correlation between pairs of variables. The degree of separability is defined as the average of the absolute values of the Pearson correlation matrix of $\mathbf{C}$ (ignoring symmetrical and diagonal elements) after discretisation of the coefficients into classes in $0,0.2,0.4,0.6,0.8,1$. |

Result: | An index in the range $[0,1]$ where 0 indicates full separability and 1 indicates full nonseparability. |

Technique 28: | Constrained landscape metrics by Malan et al. [5]. |

Year: | 2015 |

Focus: | Constraint violation in relation to fitness |

Assumptions: | Assumes that the extent to which constraints are violated can be quantified for all solutions. |

Description: | Given a sequence of solutions based on a progressive random walk [46], with associated fitness and level of constraint violation for each solution, the following are estimated: (1) the proportion of feasible solutions in the search space (FsR), (2) the level of disjointedness between feasible areas, quantified as the ratio of feasible boundary crossings (${\mathrm{RFB}}_{\times}$), (3) the correlation between the fitness and violation (FVC), and (4) the proportion of solutions that are both high in fitness and low in constraint violation, in the form of two metrics: proportion of solutions in the top 50% percentile and 20% percentile for both fitness and violation. |

Result: | A vector of five numerical values. |

Technique 29: | Bag of local landscape features by Shirakawa and Nagao [47]. |

Year: | 2016 |

Focus: | Relative fitness patterns in local neighbourhood |

Assumptions: | Assumes a distance metric in solution space. |

Description: | Given a sample of solutions of size $\lambda $, the local neighbourhood of a solution is defined as the M nearest solutions in the sample, based on a distance metric in solution space. The $LLP$ (local landscape pattern) of a solution is a pattern number corresponding to the binary sequence characterising the relative fitness of M nearest neighbours to the current solution. The $LLP$ of ${x}_{i}$ is 0 (string of M 0’s) if all M neighbours are fitter than ${x}_{i}$, and ${2}^{M}-1$ (string of M 1’s) if ${x}_{i}$ is fitter than all M neighbours. The $Evo$ (evolvability) of ${x}_{i}$ is defined as the number of better neighbours (out of M). Histograms are constructed to characterise the distribution of $LLP$ and $Evo$ values of all solutions in the sample. |

Result: | Two vectors: BoLLP (of length ${2}^{M}$) and BoEvo (of length $M+1$), representing the normalised histograms of $LLP$ and $Evo$, respectively. Principle component analysis is used to reduce the dimensions of the vectors for analysis. |

Technique 30: | Maximum entropic epistasis (MEE) by Sun et al. [48]. |

Year: | 2017 |

Focus: | Variable interactions (direct and indirect) |

Assumptions: | Assumes a continuous search space. |

Description: | For each pair of decision variables ${x}_{i},{x}_{j}$, the interaction matrix for direct interactions ($I{M}_{d}$) is identified by calculating the maximal information coefficient (largest mutual information at different scales) between ${x}_{j}$ and the estimated partial derivative of the objective with respect to ${x}_{i}$. The $I{M}_{d}$ is then used to construct an interaction graph to map the strongly connected components to identify the indirect interactions. |

Result: | Three measures: (1) the degree of direct variable interaction (DDVI), (2) the degree of indirect variable interactions (DIVI), and (3) the degree of variable interactions (DVI). |

Year: | 2018 |

Focus: | Evolvability of a population |

Assumptions: | Assumes a population-based algorithm for sampling. |

Description: | Given a population of solutions and the set of neighbours (from one iteration of the algorithm), two metrics are defined: (1) $epp$ is the probability that a population will evolve and is estimated by calculating the proportion of neighbours that are fitter than the best solution of the current population, and (2) $eap$ is the evolutionary ability of the population, which is a quantity that increases with the absolute fitness improvement of the neighbours and decreases with the fitness diversity of the population. |

Result: | A single value $evp$ which is defined as $epp\times eap$, with a range of $[0,+\infty ]$. |

Technique 32: | Local multiobjective landscape features by Liefooghe et al. [50] including earlier contributions [51,52]. |

Year: | 2019 |

Focus: | Evolvability for multiobjective optimisation |

Assumptions: | Assumes a discrete search space. |

Description: | Given a sequence of solutions obtained through random walks and adaptive walks, features of the walk are derived from the sequence as a whole as well as the neighbourhood of solutions in terms of dominance and hypervolume improvement by neighbours. |

Result: | 26 numerical values representing local features (17 from random walk sampling and 9 from adaptive walk sampling). |

Technique 33: | Loss-gradient clouds by Bosman et al. [19]. |

Year: | 2020 |

Focus: | Basins of attraction in neural network error landscapes |

Assumptions: | Requires the numeric gradient of the loss function. |

Description: | A sample of loss values and gradient values is obtained based on a number of random, progressive gradient walks [53]. Stationary points in the sample are determined to be local minima, local maxima or saddle points based on local curvature derived from the eigenvalues of the Hessian matrix. Stagnant sequences on the walk are detected by tracking the deviation in a smoothing of the error. Two quantities are measured: (1) the average number of times that stagnation was observed, and (2) the average length of the stagnant sequence. |

Result: | A two-dimensional scatterplot of loss values against gradient values (loss-gradient cloud) and two metrics to estimate the number and extent of distinct-valued basins of attraction. |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Malan, K.M.
A Survey of Advances in Landscape Analysis for Optimisation. *Algorithms* **2021**, *14*, 40.
https://doi.org/10.3390/a14020040

**AMA Style**

Malan KM.
A Survey of Advances in Landscape Analysis for Optimisation. *Algorithms*. 2021; 14(2):40.
https://doi.org/10.3390/a14020040

**Chicago/Turabian Style**

Malan, Katherine Mary.
2021. "A Survey of Advances in Landscape Analysis for Optimisation" *Algorithms* 14, no. 2: 40.
https://doi.org/10.3390/a14020040