Modifications of Flower Pollination, Teacher-Learner and Firefly Algorithms for Solving Multiextremal Optimization Problems^†

Round 1

Reviewer 1 Report

Review of the manuscript ‘Modifications of flower pollination, teacher-learner and firefly

algorithms for solving multiextremal optimization problems’ by Sorokovikov & Gornov submitted to Algorithms.

Recommendation: ACCEPT.

Focus of paper: Sorokovikov & Gornov presented an approach that applies the advantages of nature-inspired algorithms for the exploration of the admissible set and gradient methods for local optimization. The optimization was evaluated through three major approaches: flower pollination, teacher-learner and firefly algorithms. The aim of the study is well defined and clearly stated: to create computational schemes that make it possible to efficiently solve global optimization problems.

Abstract is well written and clearly describes the undertaken study.

Structure: The paper is well organized with structured sections. The structure of the article conforms to an acceptable format of standard sections: Introduction, Methodology, Results, Discussion, Conclusion, References. Some sections are divided into the minor subsections for a better structure. Logic and numeration of the sections is correct and consecutive.

Introduction presents a background, defines research goals and provides a clear statement of research problem. The Introduction well describes the research

Research questions, goal and objectives are identified: Sorokovikov & Gornov aimed to develop an approach to numerical research of the problems of finding the absolute optimum, based on the use of globalized nature-inspired and local descent methods for exploration and exploitation.

Literature regarding the relevant topics is reviewed, formatted according to the journal rules and appropriately referenced. Major sources include published papers on algorithms and mathematical literature, as well as applied physics, engineering and computational chemistry. Sorokovikov & Gornov cited 48 papers. The authors reviewed the existing approximate methods of global optimization and specifically discussed the 1) metaheuristic optimization methods where the data regarding the nature and properties of the function is absent; 2) heuristic methods which are based on the intuitive approaches. Sorokovikov & Gornov compared the both approaches and stated that metaheuristic methods are based on a higher level search strategy since they combine heuristic procedures. The authors then presented the classification of metaheuristic methods and listed the existed 3 major types. Afterwards, they detailed the optimization algorithms that they used: flower pollination (FP), fireflies. Afterwards, Sorokovikov & Gornov presented a brief overview of the state-of-the-art in problem of searching in applied physics. They noted the main issue of non-convexity which leads to a huge number of local extremes of potential functions. They also mentioned the most well-known potential optimized functions, e.g., Lennard-Jones, Morse, Keating, Gupta, Dzugutov, Sutton-Chen, etc.

Research gaps and weakness in former works are described; the existing gaps are identified in the performed literature review where Sorokovikov & Gornov mentioned and described the existing methods and pointed at the lack of enough publications in this very specific area. Thus, the contribution of this work filling this gap which is well explained. The paper concerns the development of novel solutions to the optimization problems for low-potential clusters of ever-increasing dimensions which continues to be an urgent problem actual for a wide variety of disciplines: chemistry, physics, materials science and engineering, electronics, biology, pharmaceuticals, etc.

English language: excellent.

Algorithms used in this study are described and presented with detailed explanations of formulae. The research presents novel solutions and approaches regarding the multiextremal optimization problems. Equations are explained, parameters and functions are mentioned.

Methods used in the study are summarized: The authors presented a global optimization problem with parallelepiped constraints. They presented the three multiextremal optimization algorithms (flower pollination, teacher-learner, firefly), which are well developed and implemented in this manuscript. The methods and algorithms are descriptive, parameters are explained. Modifications of the existing algorithms are mentioned briefly. The study well explained the methods. These include optimization algorithms with detailed explanations on the parameters and solutions. The methodology is well structured clearly described. It includes the sufficient information to reproduce these methods in a similar research which is useful for future studies.

Motivation and research gaps are explained: Sorokovikov & Gornov presented the novel solution to the multiextremal optimization problems, because the global optimization of multimodal objective function remains one of the crucial and hardest mathematical problems. This requires the need for new approximate methods of global optimization, which is presented by Sorokovikov & Gornov in the current manuscript. In such a way, the authors well contributed to the solution of the stated problem.

Results are reported: The authors presented nature-inspired methods for global search: flower pollination, teacher-learner, and firefly algorithms and performed the numerical comparison of variants of the realized approach employing a representative collection of multimodal objective functions. Sorokovikov & Gornov used the implemented nonconvex optimization methods to solve applied problems. They performed tasks which included optimization of the low-energy metal Sutton-Chen clusters potentials with a very large number of atoms and the parametric identification of the non-linear dynamic model. Sorokovikov & Gornov obtained results which confirm the performance of the suggested algorithms. The authors presented the Algorithm 1 (FP) with detailed explanations in 11 steps; Algorithm 2 (TL) with detailed explanations in 12 steps; Algorithm 3 (FA) with detailed explanations in 10 steps (lines 102-229). The parameters and conditions are explained, the solutions are proposed. Thus, the Results are presented with clarity and include detailed description of the algorithm parameters, mathematical equations, calculations and solutions of problems, graphics and statistical plots (e.g., box-whisker graphs of data distribution). The results are relevant to the research goals and objectives of this study. The numerical study of the developed algorithms is proposed using C language. The the values of the algorithmic parameters are listed in a Table 1. Table 2 demonstrates statistics on launches: mean and standard deviation. The results highlights the major achievements of this study regarding the development of the nonconvex optimization methods. Box and whisker diagrams of algorithms illustrated 4 cases: 1) Griewank problem; 2) Rastrigin problem; 3) Schwefel problem; 4) Ackley problem.

Discussion interpreted the major outcomes of this study. The authors presented novel solutions which well contributed to the existing Sutton-Chen configurations up to 80 atoms in the Cambridge Energy Landscape Database. Sorokovikov & Gornov made a performance investigation of developed techniques for clusters of 3–80 atoms and compared the solutions obtained with the best of the known ones. The solutions obtained during the presented numerical study coincided with the known results, which proves the correctness of the used methods, The advantages of the obtained results are described and compared with other studies on algorithms (e.g., Lennard-Jones, Morse, Keating, Gupta, Dzugutov, Sutton-Chen). The Discussion described the issues of methodology and results.

Conclusion summarized the study with interpretation of facts. Sorokovikov & Gornov investigated a large problem for configurations of atoms from 81 to 100, which presents an application to physics. For each algorithm-generator and each starter method, they performed computational experiments. They used an excess of the time limit (24 hours) as a stopping criterion. They presented the results in the Table 3 which summarized the results of the numerical testing of computational technology for the task in which the cluster consisted of 85 atoms. The conclusions are appropriately stated and connected to the original questions.

Actuality, novelty and importance of the research is clear. It consists in technical approach of developing three hybrid nonconvex minimization algorithms, which are developed and implemented by the authors. Specifically, Sorokovikov & Gornov presented the modifications of the following nature-inspired methods for global search: flower pollination, teacher-learner, and firefly algorithms. Sorokovikov & Gornov used modified trust region method which is based on the main diagonal approximation of the Hessian matrix and applied for local refinement. This is a new approach presented by the authors and contributes to the finding solutions of novel methods and algorithms of search.

Logic: The clarity of the text logic and organization of the paper is sufficient. It demonstrates the consistent interpretation of the results with detailed explanations. Sorokovikov & Gornov presented a solution to the numerical investigation of the tasks of searching the absolute optimum of multimodal functions. To this end, they used the following approaches: flower pollination, teacher-learner, firefly, and modified trust region algorithms. They used hybrid methods for non-convex optimization, which are proposed and implemented in the presented manuscript. A comparison of the results with previous studies is presented.

Relevance: The manuscript perfectly meets general criteria of the significance in Algorithms journals, as Sorokovikov & Gornov presented a well-thought study regarding the novel solutions in the algorithms. The study has been conducted in accordance to the technical standards in mathematical science. Specifically, the developed algorithms were studied on a set of test problems characterized by different levels of complexity. All variants of the algorithms are based on combinations with the modification of trust region method for local search. The authors demonstrated and argued essential improvements over the original algorithms. Sorokovikov & Gornov also performed the statistical testing of the proposed algorithms which revealed that modifications of teacher-learner, and flower pollination methods outperformed those of the firefly method. It is relevant to the journal topic as corresponding to the major domain and research discipline clusters: optimization, algorithms and functions, clusters, solutions to problems using numerical methods..

Academic contribution is clear: the paper increases the knowledge in mathematics and algorithms. The paper combines technical and mathematical approaches which presents a multi-disciplinary study well deserved to be published in Algorithms.

Figures are of good quality, easy to read, relevant and suitable, well illustrate the results, relevant to the content, have sufficient resolution, appropriately described and labeled.

Recommendation: This manuscript can be ACCEPTED based on the detailed report above.

With kind regards,

- Anonymous Reviewer.

02.09.2022.

Comments for author File: Comments.pdf

Author Response

Thank you for the review!

Reviewer 2 Report

The paper can be published if the authors perform the following modifications

1. The stopping rule used is not well defined. If the authors have used the maximum number of iterations, then a more advanced stopping rule should be incorporated

2. It is not clear if the authors have improved the produced solutions using some local search method (eg. BFGS ) If not, the additional experiments should be conducted using some local search procedure

3. The set of testing functions used is quite limited. The authors should use more testing functions from the relevant litearature and report the results

Author Response

Thank you for the review. Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

In the current manuscript the authors propose 3 hybrid nonconvex minimization metaheuristic algorithms by modifying the original version of the flower pollination algorithm, the teacher-learner algorithm and the firefly algorithm. The authors test the proposed methods on academic benchmarks and on relevant applied problems.  The obtained results are better than the original versions. 

I would like the following comments to be addressed by the authors.

- In every proposed algorithm,  3-rd step there is this sentence: "perform m local descents". Could the authors be more specific about what do they mean? Is this word "descent" meaning that a gradient descent method is applied? If this is the case, isn't this step blowing up the meta-heuristic approach somehow? Otherwise, if this is not the case, could the authors explain more in details how the "descent" direction can be understood in their method?

- In step 4, for every proposed algorithm, how can the record point be the minimal argument of a set of points? Do the author mean  $arg_min { f(s_l), l=1..m}$ ? So, the record point is the minimum point for the function f evaluated at the points "s_l"?

- Algorithm 1 (FP), lines 7.2 and 7.3: the authors should specify what is the quantity "P_s" as it is not defined anywhere and it is not clear if it is somehow a "probability value" or a kind of "fitness value". The value for this parameter is set in Table 1, but it is very unclear why 0.2 and as I said, what is the meaning for this parameter... 

- Algorithm 1 (FP), line 7.3, could one of the two integer indexes (so either j_1 or j_2) be equal to the index " l "? . If this is not allowed, please specify it, otherwise, if it is, please add a short sentence to clarify it. 

- Algorithm 1 (FP), line 8: please, specify what is the quantity $\bar{p}$ as it is not defined anywhere else. 

- Algorithm 2 (TL), step 7.3: there is a typo in the sentence, "generate a random numbers", please remove the "a". 

- Algorithm 3 (FA), step 4.1, I would suggest the authors to replace the letter $\Beta_1$ with maybe something different (or simply to put a hat above it) , to avoid the confusion with the upper point of the box constraints (see Eq. (1)). The same suggestion is for line 4.3.

-What is the stopping criterion adopted? Is it only a maximum number of iterations? From the inserted tables or plots, it is not possible to understand the behaviour of the proposed strategies in terms of "stagnation points" . Did the authors observe such a behaviour? Could a different stopping criterion, like for example the computation of the swarm-diameter,  save some additional function evaluations, and hence reduce the computational cost?

- How the parameters in Table 1 have been chosen? Did the authors perform some sensitivity analysis? In that case, I would suggest to add a comment on this. 

- The problem 3.6 is not well described in my opinion. For example, what is the solution of the problem (11) and how it is connected to the minimization of the functional in (12) ? Then, why the integral is performed just in the interval [0,8]? How is the plot in figure 6 insightful? What the reader should expect to see in order to detect a good performance from a bad one?

- I think there might be a typo in Eq. (9): +e-20 ---> I would make the notation consistent and write $10^{-20}$, if this is what the authors meant.

As final comment, I would say that the paper is well written and the English is clear. The topic addressed is meaningful and sound in the scientific community. 

Author Response

Thank you for the review. Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The paper can be accepted when the authors apply their proposed work to more test functions from the relative literature

Reviewer 3 Report

The authors addressed all my comments. The paper is very well written and insightful about bio-inspired based algorithms.