Special Issue "Benchmarking, Selecting and Configuring Learning and Optimization Algorithms"

A special issue of Algorithms (ISSN 1999-4893). This special issue belongs to the section "Evolutionary Algorithms and Machine Learning".

Deadline for manuscript submissions: 30 September 2020.

Special Issue Editors

Dr. Mario A. Muñoz
Guest Editor
School of Mathematics and Statistics, University of Melbourne, Melbourne, Australia
Interests: intelligent optimization methods; complex systems
Prof. Katherine Malan
Guest Editor
Department of Decision Sciences, University of South Africa, Pretoria, South Africa
Interests: computational intelligence; metaheuristics; fitness landscape analysis

Special Issue Information

Dear Colleagues,

Whenever we need to solve a computational problem, selecting and configuring an appropriate algorithm are crucial tasks. Both theoretical and empirical results demonstrate that no single algorithm can find the best possible solution for all problems within a domain with the least amount of computation. This is because each algorithm makes different assumptions about the structure of a problem, leading to strength and weaknesses which are often unknown beforehand. This is known as performance complementarity. A deep understanding of this issue is critical for heuristic algorithms, which can perform better than classical ones, solving problems that were unfeasible in the past; however, their behavior is still largely unpredictable. Therefore, an otherwise useful method would result in failures if it is used inappropriately in the wrong contexts.

Given a complex problem, automated algorithm selection and configuration involves the development of methods that would choose the most appropriate algorithm for solving that problem. Automated algorithm selection has been successfully implemented in some well-studied problem scenarios, such as the travelling salesman problem. However, there are many challenges that remain before automated algorithm selection can become a reality in the wider learning and optimization contexts. Some of these challenges involve:

  1. Constructing a robust knowledge base of empirical results, from a wide variety of benchmarks suites that are unbiased, challenging, and contain a mix of synthetically generated and real-world-like instances with diverse structural properties. Without this diversity, the conclusions that can be drawn about the expected algorithm performance in future scenarios are necessarily limited;
  2. Developing robust and efficient characterization methods can determine the structural similarities between problems, and the influence that such structure has on algorithm performance, while facilitating the analysis by the designers;
  3. Constructing selection and configuration methods that are not only accurate but also minimize the probability of making expensive mistakes.

With this call, we invite you to submit your research papers to this Special Issue, covering all aspects of automated algorithm selection and configuration. The following is a (non-exhaustive) list of topics of interest:

  • Problem characterization, such as fitness landscape analysis;
  • Experimental algorithmics for collection of reliable performance data;
  • Meta- and surrogate modeling;
  • Automated parameter selection/tuning;
  • Pipelines for automated algorithm selection;
  • Instance space analysis and algorithm footprints;
  • Benchmark collections;
  • Hyper-heuristics.

Dr. Mario A. Muñoz
Prof. Dr. Katherine Malan
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All papers will be peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Algorithms is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1000 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.


  • Algorithm selection and configuration
  • Benchmark suites
  • Experimental analysis of algorithms
  • Fitness landscape analysis and problem characterization
  • Meta-learning
  • Surrogate modeling

Published Papers

This special issue is now open for submission.
Back to TopTop