Novel Algorithm for Linearly Constrained Derivative Free Global Optimization of Lipschitz Functions
Abstract
:1. Introduction
Paper Contributions and Structure
- The review of techniques proposed to tackle linearly constrained problems within the framework of DIRECT-type algorithms.
- Introduction of a novel and distinctive DIRECT-type algorithm explicitly designed for non-convex problems involving linear constraints.
- The substantial enhancement of the DIRECTGOLib v1.3 benchmark library by incorporating 34 lineary-constrained test problems.
- The provision of the novel algorithm developed as an open-source resource to ensure the full reproducibility and reusability of all results.
2. Materials and Methods
2.1. The Original DIRECT Algorithm for Box-Constrained Global Optimization
2.2. Extensions of the DIRECT Algorithm for Problems with Constraints
2.2.1. Approaches Based on Simplicial Partitioning
2.2.2. Penalty and Auxiliary Function Approaches
2.2.3. Filtering Approach
2.2.4. Alternative Approaches without Utilizing Constraint Information
3. Description of the Proposed mBIRECTv-GL Algorithm
3.1. Efficient Bijective Mapping: Construction and Methodological Details
3.2. Integrating Mapping Techniques in DIRECT-Based Framework
Selection of the Most Promising Regions Using a Two-Step-Based Approach
3.3. Description of a Novel Algorithm (mBIRECTv-GL)
Algorithm 1 The mBIRECTv-GL algorithm |
|
3.4. Convergence Properties of the mBIRECTv-GL Algorithm
4. Results and Discussions
4.1. Foundation of Solver Comparisons and Design of Experimental Setup
4.2. Analysis of the Overall Performance of Algorithms
4.3. Statistical Analysis of the Results
5. Conclusions and Future Prospects
Author Contributions
Funding
Data Availability Statement
- https://github.com/blockchain-group/DIRECTGOLib (accessed on 15 June 2023),
- https://zenodo.org/record/8046086 (accessed on 16 June 2023).
- https://github.com/blockchain-group/DIRECTGO (accessed on 15 June 2023).
Conflicts of Interest
Appendix A. Linearly Constrained Test Problems from the DIRECTGOLib v1.3 Benchmark Library
# | Name | Ref. | n | Con. | AC | Variable Bounds | Optimum |
---|---|---|---|---|---|---|---|
1 | avgasa | [66] | 8 | 10 | 3 | ||
2 | avgasb | [66] | 8 | 10 | 4 | ||
3 | biggsc4 | [66] | 4 | 13 | 3 | ||
4 | Bunnag1 | [66] | 3 | 1 | 1 | ||
5 | Bunnag2 | [66] | 4 | 2 | 1 | ||
6 | Bunnag3 | [66] | 5 | 3 | 1 | ||
7 | Bunnag4 | [66] | 6 | 2 | 1 | ||
8 | Bunnag5 | [66] | 6 | 5 | 1 | ||
9 | Bunnag6 | [66] | 10 | 11 | 3 | ||
10 | Bunnag7 | [66] | 10 | 5 | 0 | ||
11 | Bunnag8 | [66] | 5 | 1 | 0 | ||
12 | Bunnag10 | [66] | 20 | 10 | 5 | ||
13 | Bunnag11 | [66] | 20 | 10 | 5 | ||
14 | Bunnag12 | [66] | 20 | 10 | 5 | ||
15 | Bunnag13 | [66] | 20 | 10 | 0 | ||
16 | Bunnag14 | [66] | 20 | 10 | 0 | ||
17 | Bunnag15 | [66] | 20 | 10 | 2 | ||
18 | ex2_1_1 | [66] | 5 | 1 | 1 | ||
19 | ex2_1_2 | [66] | 6 | 2 | 1 | ||
20 | expfita | [66] | 5 | 2 | 0 | ||
21 | expfitb | [66] | 5 | 2 | 0 | ||
22 | expfitc | [66] | 5 | 2 | 0 | ||
23 | G01 | [66] | 13 | 9 | 6 | ||
24 | Genocop7 | [66] | 6 | 2 | 1 | ||
25 | Genocop9 | [66] | 3 | 5 | 2 | ||
26 | Genocop10 | [66] | 4 | 2 | 1 | ||
27 | Horst1 | [67] | 2 | 3 | 1 | ||
28 | Horst2 | [67] | 2 | 3 | 2 | ||
29 | Horst3 | [67] | 2 | 3 | 0 | ||
30 | Horst4 | [67] | 3 | 4 | 2 | ||
31 | Horst5 | [67] | 3 | 4 | 2 | ||
32 | Horst6 | [67] | 3 | 7 | 2 | ||
33 | Horst7 | [67] | 3 | 4 | 2 | ||
34 | hs021 | [66] | 2 | 1 | 0 | ||
35 | hs021mod | [66] | 7 | 1 | 1 | ||
36 | hs024 | [66] | 2 | 3 | 2 | ||
37 | hs036 | [66] | 3 | 1 | 1 | ||
38 | hs037 | [66] | 3 | 2 | 1 | ||
39 | hs038 | [66] | 4 | 2 | 0 | ||
40 | hs044 | [66] | 4 | 6 | 2 | ||
41 | hs076 | [66] | 4 | 3 | 1 | ||
42 | hs086 | [66] | 5 | 1 | 0 | ||
43 | hs118 | [66] | 15 | 17 | 9 | ||
44 | hs268 | [66] | 5 | 5 | 2 | ||
45 | Ji1 | [66] | 3 | 4 | 1 | ||
46 | Ji2 | [66] | 3 | 2 | 0 | ||
47 | Ji3 | [66] | 2 | 1 | 0 | ||
48 | ksip | [66] | 10 | 20 | 2 | ||
49 | Michalewicz1 | [66] | 2 | 3 | 0 | ||
50 | P9 | [66] | 3 | 9 | 2 | ||
51 | P14 | [66] | 3 | 4 | 2 | ||
52 | s224 | [66] | 2 | 4 | 1 | ||
53 | s231 | [66] | 2 | 2 | 0 | ||
54 | s232 | [66] | 2 | 3 | 2 | ||
55 | s250 | [66] | 3 | 2 | 1 | ||
56 | s251 | [66] | 3 | 1 | 1 | ||
57 | s253 | [66] | 3 | 1 | 0 | ||
58 | s268 | [66] | 5 | 5 | 2 | ||
59 | s277 | [66] | 4 | 4 | 4 | ||
60 | s278 | [66] | 6 | 6 | 6 | ||
61 | s279 | [66] | 8 | 8 | 8 | ||
62 | s280 | [66] | 10 | 10 | 10 | ||
63 | s331 | [66] | 2 | 1 | 0 | ||
64 | s340 | [66] | 3 | 1 | 1 | ||
65 | s354 | [66] | 4 | 1 | 1 | ||
66 | s359 | [66] | 5 | 14 | 4 | ||
67 | zecevic2 | [66] | 2 | 2 | 1 |
References
- Lucidi, S.; Sciandrone, M. A Derivative-Free Algorithm for Bound Constrained Optimization. Comput. Optim. Appl. 2002, 21, 119–142. [Google Scholar] [CrossRef]
- Giovannelli, T.; Liuzzi, G.; Lucidi, S.; Rinaldi, F. Derivative-free methods for mixed-integer nonsmooth constrained optimization. Comput. Optim. Appl. 2022, 82, 293–327. [Google Scholar] [CrossRef]
- Kimiaei, M.; Neumaier, A. Efficient unconstrained black box optimization. Math. Program. Comput. 2022, 14, 365–414. [Google Scholar] [CrossRef]
- Holland, J. Adaptation in Natural and Artificial Systems; The University of Michigan Press: Ann Arbor, MI, USA, 1975. [Google Scholar]
- Eberhart, R.; Kennedy, J. A new optimizer using particle swarm theory. In Proceedings of the Sixth International Symposium on Micro Machine and Human Science (MHS’95), Nagoya, Japan, 4–6 October 1995; pp. 39–43. [Google Scholar] [CrossRef]
- Kirkpatrick, S.; Gelatt, C.D.; Vecchi, M.P. Optimization by simulated annealing. Science 1983, 220, 671–680. [Google Scholar] [CrossRef]
- Bujok, P.; Kolenovsky, P. Eigen crossover in cooperative model of evolutionary algorithms applied to cec 2022 single objective numerical optimisation. In Proceedings of the 2022 IEEE Congress on Evolutionary Computation (CEC), Padua, Italy, 18–23 July 2022; pp. 1–8. [Google Scholar] [CrossRef]
- Hadi, A.A.; Mohamed, A.W.; Jambi, K.M. Single-objective real-parameter optimization: Enhanced LSHADE-SPACMA algorithm. Heuristics Optim. Learn. 2021, 906, 103–121. [Google Scholar] [CrossRef]
- Paulavičius, R.; Stripinis, L.; Sutavičiūtė, S.; Kočegarov, D.; Filatovas, E. A novel greedy genetic algorithm-based personalized travel recommendation system. Expert Syst. Appl. 2023, 230, 120580. [Google Scholar] [CrossRef]
- Jones, D.R.; Schonlau, M.; Welch, W.J. Efficient Global Optimization of Expensive Black-Box Functions. J. Glob. Optim. 1998, 13, 455–492. [Google Scholar] [CrossRef]
- Björkman, M.; Holmström, K. Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions. Optim. Eng. 2000, 1, 373–397. [Google Scholar] [CrossRef]
- Lera, D.; Sergeyev, Y.D. Lipschitz and Hölder global optimization using space-filling curves. Appl. Numer. Math. 2010, 60, 115–129. [Google Scholar] [CrossRef]
- Sergeyev, Y.D.; Kvasov, D.E. Lipschitz global optimization. In Wiley Encyclopedia of Operations Research and Management Science (in 8 Volumes); Cochran, J.J., Cox, L.A., Keskinocak, P., Kharoufeh, J.P., Smith, J.C., Eds.; John Wiley & Sons: New York, NY, USA, 2011; Volume 4, pp. 2812–2828. [Google Scholar]
- Martins, J.R.R.A.; Ning, A. Engineering Design Optimization; Cambridge University Press: Cambridgem, UK, 2021. [Google Scholar]
- Best, M.J. Portfolio Optimization; CRC Press: Boca Raton, FL, USA, 2010. [Google Scholar]
- Floudas, C.A.; Pardalos, P.M.; Adjiman, C.; Esposito, W.R.; Gümüs, Z.H.; Harding, S.T.; Klepeis, J.L.; Meyer, C.A.; Schweiger, C.A. Handbook of Test Problems in Local and Global Optimization; Springer: Berlin/Heidelberg, Germany, 2013; Volume 33. [Google Scholar]
- Jones, D.R.; Perttunen, C.D.; Stuckman, B.E. Lipschitzian Optimization Without the Lipschitz Constant. J. Optim. Theory Appl. 1993, 79, 157–181. [Google Scholar] [CrossRef]
- Jones, D.R. The Direct Global Optimization Algorithm. In The Encyclopedia of Optimization; Floudas, C.A., Pardalos, P.M., Eds.; Kluwer Academic Publishers: Dordrect, The Netherlands, 2001; pp. 431–440. [Google Scholar]
- Jones, D.R.; Martins, J.R.R.A. The DIRECT algorithm: 25 years later. J. Glob. Optim. 2021, 79, 521–566. [Google Scholar] [CrossRef]
- Sergeyev, Y.D.; Kvasov, D.E. Global search based on diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 2006, 16, 910–937. [Google Scholar] [CrossRef] [Green Version]
- Holmstrom, K.; Goran, A.O.; Edvall, M.M. User’s Guide for TOMLAB 7. Technical Report. Tomlab Optimization Inc. 2010. Available online: https://tomopt.com/docs/TOMLAB.pdf (accessed on 15 June 2023).
- Stripinis, L.; Žilinskas, J.; Casado, L.G.; Paulavičius, R. On MATLAB experience in accelerating DIRECT-GLce algorithm for constrained global optimization through dynamic data structures and parallelization. Appl. Math. Comput. 2021, 390, 125596. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. An empirical study of various candidate selection and partitioning techniques in the DIRECT framework. J. Glob. Optim. 2022, 1–31. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R.; Žilinskas, J. Penalty functions and two-step selection procedure based DIRECT-type algorithm for constrained global optimization. Struct. Multidiscip. Optim. 2019, 59, 2155–2175. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. Experimental Study of Excessive Local Refinement Reduction Techniques for Global Optimization DIRECT-Type Algorithms. Mathematics 2022, 10, 3760. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. Lipschitz-inspired HALRECT algorithm for derivative-free global optimization. J. Glob. Optim. 2023, 1–31. [Google Scholar] [CrossRef]
- Liuzzi, G.; Lucidi, S.; Piccialli, V. A direct-based approach exploiting local minimizations for the solution for large-scale global optimization problems. Comput. Optim. Appl. 2010, 45, 353–375. [Google Scholar] [CrossRef]
- Liuzzi, G.; Lucidi, S.; Piccialli, V. Exploiting derivative-free local searches in direct-type algorithms for global optimization. Comput. Optim. Appl. 2016, 65, 449–475. [Google Scholar] [CrossRef]
- Kvasov, D.E.; Mukhametzhanov, M.S. Metaheuristic vs. deterministic global optimization algorithms: The univariate case. Appl. Math. Comput. 2018, 318, 245–259. [Google Scholar] [CrossRef]
- Rios, L.M.; Sahinidis, N.V. Derivative-free optimization: A review of algorithms and comparison of software implementations. J. Glob. Optim. 2013, 56, 1247–1293. [Google Scholar] [CrossRef] [Green Version]
- Sergeyev, Y.D.; Kvasov, D.E.; Mukhametzhanov, M.S. On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget. Sci. Rep. 2018, 8, 453. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Costa, M.F.P.; Rocha, A.M.A.C.; Fernandes, E.M.G.P. Filter-based DIRECT method for constrained global optimization. J. Glob. Optim. 2018, 71, 517–536. [Google Scholar] [CrossRef]
- Finkel, D.E. MATLAB Source Code for DIRECT. 2004. Available online: http://www4.ncsu.edu/~ctk/Finkel_Direct/ (accessed on 22 March 2017).
- Liu, H.; Xu, S.; Chen, X.; Wang, X.; Ma, Q. Constrained global optimization via a DIRECT-type constraint-handling technique and an adaptive metamodeling strategy. Struct. Multidiscip. Optim. 2017, 55, 155–177. [Google Scholar] [CrossRef]
- Pillo, G.D.; Liuzzi, G.; Lucidi, S.; Piccialli, V.; Rinaldi, F. A DIRECT-type approach for derivative-free constrained global optimization. Comput. Optim. Appl. 2016, 65, 361–397. [Google Scholar] [CrossRef] [Green Version]
- Pillo, G.D.; Lucidi, S.; Rinaldi, F. An approach to constrained global optimization based on exact penalty functions. J. Optim. Theory Appl. 2010, 54, 251–260. [Google Scholar] [CrossRef]
- Gablonsky, J.M. Modifications of the DIRECT Algorithm. Ph.D. Thesis, North Carolina State University, Raleigh, NC, USA, 2001. [Google Scholar]
- Na, J.; Lim, Y.; Han, C. A modified DIRECT algorithm for hidden constraints in an LNG process optimization. Energy 2017, 126, 488–500. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. A new DIRECT-GLh algorithm for global optimization with hidden constraints. Optim. Lett. 2021, 15, 1865–1884. [Google Scholar] [CrossRef]
- Paulavičius, R.; Žilinskas, J. Advantages of simplicial partitioning for Lipschitz optimization problems with linear constraints. Optim. Lett. 2016, 10, 237–246. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. DIRECTGOLib—DIRECT Global Optimization Test Problems Library. Version v1.3, Zenodo. 2023. Available online: https://zenodo.org/record/8046086/export/hx (accessed on 15 June 2023).
- Friedman, M. The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance. J. Am. Stat. Assoc. 1937, 32, 675–701. [Google Scholar] [CrossRef]
- Hollander, M.; Wolfe, D. Nonparametric Statistical Methods, Solutions Manual; Wiley Series in Probability and Statistics; Wiley: Hoboken, NJ, USA, 1999. [Google Scholar]
- Gablonsky, J.M.; Kelley, C.T. A locally-biased form of the DIRECT algorithm. J. Glob. Optim. 2001, 21, 27–37. [Google Scholar] [CrossRef]
- Paulavičius, R.; Žilinskas, J. Analysis of different norms and corresponding Lipschitz constants for global optimization in multidimensional case. Inf. Technol. Control 2007, 36, 383–387. [Google Scholar]
- Paulavičius, R.; Žilinskas, J. Simplicial Lipschitz optimization without the Lipschitz constant. J. Glob. Optim. 2014, 59, 23–40. [Google Scholar] [CrossRef]
- Paulavičius, R.; Žilinskas, J. Simplicial Global Optimization; SpringerBriefs in Optimization; Springer: New York, NY, USA, 2014. [Google Scholar] [CrossRef]
- Finkel, D.E. Global Optimization with the Direct Algorithm. Ph.D. Thesis, North Carolina State University, Raleigh, NC, USA, 2005. [Google Scholar]
- Fletcher, R.; Leyffer, S. Nonlinear programming without a penalty function. Math. Program. 2002, 91, 239–269. [Google Scholar] [CrossRef]
- Barber, C.B.; Dobkin, D.P.; Huhdanpaa, H. The quickhull algorithm for convex hulls. ACM Trans. Math. Softw. (TOMS) 1996, 22, 469–483. [Google Scholar] [CrossRef] [Green Version]
- Becker, S. CON2VERT—Constraints to Vertices, MATLAB Central File Exchange. 2023. Available online: https://www.mathworks.com/matlabcentral/fileexchange/7894-con2vert-constraints-to-vertices (accessed on 16 May 2023).
- Huyer, W.; Neumaier, A. Global Optimization by Multilevel Coordinate Search. J. Glob. Optim. 1999, 14, 331–355. [Google Scholar] [CrossRef]
- Liu, H.; Xu, S.; Wang, X.; Wu, X.; Song, Y. A global optimization algorithm for simulation-based problems via the extended DIRECT scheme. Eng. Optim. 2015, 47, 1441–1458. [Google Scholar] [CrossRef]
- Chiter, L. Experimental Data for the Preprint “Diagonal Partitioning Strategy Using Bisection of Rectangles and a Novel Sampling Scheme”. Mendeley Data, V2. 2023. Available online: https://data.mendeley.com/datasets/x9fpc9w7wh/2 (accessed on 16 June 2023).
- Paulavičius, R.; Chiter, L.; Žilinskas, J. Global optimization based on bisection of rectangles, function values at diagonals, and a set of Lipschitz constants. J. Glob. Optim. 2018, 71, 5–20. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R.; Žilinskas, J. Improved scheme for selection of potentially optimal hyper-rectangles in DIRECT. Optim. Lett. 2018, 12, 1699–1712. [Google Scholar] [CrossRef]
- Finkel, D.E.; Kelley, C.T. Additive scaling and the DIRECT algorithm. J. Glob. Optim. 2006, 36, 597–608. [Google Scholar] [CrossRef]
- Paulavičius, R.; Sergeyev, Y.D.; Kvasov, D.E.; Žilinskas, J. Globally-biased DISIMPL algorithm for expensive global optimization. J. Glob. Optim. 2014, 59, 545–567. [Google Scholar] [CrossRef]
- Stripinis, L.; Paulavičius, R. DIRECTGO: A New DIRECT-Type MATLAB Toolbox for Derivative-Free Global Optimization. ACM Trans. Math. Softw. 2022, 48, 41. [Google Scholar] [CrossRef]
- Paulavičius, R.; Sergeyev, Y.D.; Kvasov, D.E.; Žilinskas, J. Globally-biased BIRECT algorithm with local accelerators for expensive global optimization. Expert Syst. Appl. 2020, 144, 11305. [Google Scholar] [CrossRef]
- Moré, J.J.; Wild, S.M. Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 2009, 20, 172–191. [Google Scholar] [CrossRef] [Green Version]
- Grishagin, V.A. Operating characteristics of some global search algorithms. In Problems of Stochastic Search; Zinatne: Riga, Latvia, 1978; Volume 7, pp. 198–206. (In Russian) [Google Scholar]
- Hansen, N.; Auger, A.; Ros, R.; Mersmann, O.; Tušar, T.; Brockhoff, D. COCO: A platform for comparing continuous optimizers in a black-box setting. Optim. Methods Softw. 2021, 36, 114–144. [Google Scholar] [CrossRef]
- Jusevičius, V.; Oberdieck, R.; Paulavičius, R. Experimental Analysis of Algebraic Modelling Languages for Mathematical Optimization. Informatica 2021, 32, 283–304. [Google Scholar] [CrossRef]
- Jusevičius, V.; Paulavičius, R. Web-Based Tool for Algebraic Modeling and Mathematical Optimization. Mathematics 2021, 9, 2751. [Google Scholar] [CrossRef]
- Vaz, A.; Vicente, L. Pswarm: A hybrid solver for linearly constrained global derivative-free optimization. Optim. Methods Softw. 2009, 24, 669–685. [Google Scholar] [CrossRef]
- Horst, R.; Pardalos, P.M.; Thoai, N.V. Introduction to Global Optimization; Nonconvex Optimization and Its Application; Kluwer Academic Publishers: Berlin, Germany, 1995. [Google Scholar]
Average Number of Function Evaluations | ||||||
---|---|---|---|---|---|---|
Algorithm | Fails | Overall | AC 1 | NAC 2 | ||
mBIRECTv-GL | ||||||
Lc-DISIMPLv | ||||||
Lc-DISIMPLc | ||||||
glcSolve | ||||||
DIRECT-GLc | ||||||
DIRECT-GLce |
Algorithm | |||||
---|---|---|---|---|---|
Lc-DISIMPLv | |||||
Lc-DISIMPLc | |||||
glcSolve | |||||
DIRECT-GLc | |||||
DIRECT-GLce |
Algorithm | |||||
---|---|---|---|---|---|
mBIRECTv-GL | |||||
Lc-DISIMPLv | |||||
Lc-DISIMPLc | |||||
glcSolve | |||||
DIRECT-GLc | |||||
DIRECT-GLce | |||||
p-value |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Stripinis, L.; Paulavičius, R. Novel Algorithm for Linearly Constrained Derivative Free Global Optimization of Lipschitz Functions. Mathematics 2023, 11, 2920. https://doi.org/10.3390/math11132920
Stripinis L, Paulavičius R. Novel Algorithm for Linearly Constrained Derivative Free Global Optimization of Lipschitz Functions. Mathematics. 2023; 11(13):2920. https://doi.org/10.3390/math11132920
Chicago/Turabian StyleStripinis, Linas, and Remigijus Paulavičius. 2023. "Novel Algorithm for Linearly Constrained Derivative Free Global Optimization of Lipschitz Functions" Mathematics 11, no. 13: 2920. https://doi.org/10.3390/math11132920
APA StyleStripinis, L., & Paulavičius, R. (2023). Novel Algorithm for Linearly Constrained Derivative Free Global Optimization of Lipschitz Functions. Mathematics, 11(13), 2920. https://doi.org/10.3390/math11132920