# Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Past Methods of Metamodeling for Multiobjective Optimization

## 3. A Taxonomy for Multiobjective Metamodeling Frameworks

#### 3.1. M1-1 and M1-2 Frameworks

#### 3.2. Frameworks M2-1 and M2-2

#### 3.3. M3-1 and M3-2 Frameworks

#### 3.4. Frameworks M4-1 and M4-2

#### 3.5. M5 Framework

#### 3.6. Framework M6

#### 3.7. Summary of 10 Frameworks

## 4. Adaptive Switching Based Metamodeling (ASM) Frameworks

Algorithm 1: Adaptive Swithing Framework |

#### 4.1. Performance Metric for Framework Selection

#### 4.2. Selecting a Framework for an Epoch

#### 4.3. Trust-Region Based Real-Coded Genetic Algorithms

## 5. Results and Discussion

#### 5.1. Parameter Settings

#### 5.2. Two-Objective Unconstrained Problems

#### 5.3. Two-Objective Constrained Problems

#### 5.4. Three and More Objective Constrained and Unconstrained Problems

#### 5.5. Comparison with Existing Methods

## 6. Conclusions

## Funding

## Conflicts of Interest

## References

- Cassioli, A.; Schoen, F. Global optimization of expensive black box problems with a known lower bound. J. Glob. Optim.
**2013**, 57, 177–190. [Google Scholar] [CrossRef] - Jin, Y. Surrogate-assisted evolutionary computation: Recent advances and future challenge. Swarm Evol. Comput.
**2011**, 1, 61–70. [Google Scholar] [CrossRef] - Ponweiser, W.; Wagner, T.; Biermann, D.; Vincze, M. Multiobjective Optimization on a Limited Budget of Evaluations Using Model-Assisted S-Metric Selection. In Parallel Problem Solving from Nature–PPSN X; Springer: Berlin/Heidelberg, Germany, 2008; pp. 784–794. [Google Scholar]
- Jones, D.R. A taxonomy of global optimization methods based on response surfaces. J. Glob. Optim.
**2001**, 21, 345–383. [Google Scholar] [CrossRef] - Hussein, R.; Deb, K. A Generative Kriging Surrogate Model for Constrained and Unconstrained Multi-objective Optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’16), Denver, CO, USA, 20–24 July 2016; ACM Press: New York, NY, USA, 2016. [Google Scholar]
- Deb, K.; Hussein, R.; Roy, P.; Toscano, G. Classifying Metamodeling Methods for Evolutionary Multi-objective Optimization: First Results. In Evolutionary Multi-Criterion Optimization EMO; Springer: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
- Roy, P.; Hussein, R.; Deb, K. Metamodeling for multimodal selection functions in evolutionary multi-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17), Berlin, Germany, 15–19 July 2017; ACM Press: New York, NY, USA, 2017. [Google Scholar]
- Bhattacharjee, K.S.; Singh, H.K.; Ray, T. Multi-objective optimization with multiple spatially distributed surrogates. J. Mech. Des.
**2016**, 138, 091401. [Google Scholar] [CrossRef] - Bhattacharjee, K.S.; Singh, H.K.; Ray, T.; Branke, J. Multiple Surrogate Assisted Multiobjective Optimization Using Improved Pre-Selection. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC-2016), Vancouver, BC, Canada, 24–29 July 2016. [Google Scholar]
- Emmerich, M.T.M.; Giannakoglou, K.C.; Naujoks, B. Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans. Evol. Comput.
**2006**, 10, 421–439. [Google Scholar] [CrossRef] - Byrd, R.H.; Nocedal, J.; Waltz, R.A. Knitro: An Integrated Package for Nonlinear Optimization. In Large-Scale Nonlinear Optimization; Springer US: Boston, MA, USA, 2006. [Google Scholar]
- Jin, Y.; Oh, S.; Jeon, M. Incremental approximation of nonlinear constraint functions for evolutionary constrained optimization. In Proceedings of the 2010 IEEE Congress on Evolutionary Computation (CEC-2010), Barcelona, Spain, 18–23 July 2010; pp. 1–8. [Google Scholar]
- Datta, R.; Regis, R.G. A surrogate-assisted evolution strategy for constrained multi-objective optimization. Expert Syst. Appl.
**2016**, 57, 270–284. [Google Scholar] [CrossRef] - Zhang, Q.; Liu, W.; Tsang, E.; Virginas, B. Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model. IEEE Trans. Evol. Comput.
**2010**, 14, 456–474. [Google Scholar] [CrossRef] - Knowles, J. ParEGO: A Hybrid Algorithm with On-line Landscape Approximation for Expensive Multiobjective Optimization Problems. IEEE Trans. Evol. Comput.
**2006**, 10, 50–66. [Google Scholar] [CrossRef] - Allmendinger, R.; Emmerich, M.T.; Hakanen, J.; Jin, Y.; Rigoni, E. Surrogate-assisted multicriteria optimization: Complexities, prospective solutions, and business case. J. Multi-Criteria Decis. Anal.
**2017**, 24, 5–24. [Google Scholar] [CrossRef] - Roy, P.C.; Deb, K. High Dimensional Model Representation for Solving Expensive Multi-objective Optimization Problems. In Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada, 24–29 July 2016. [Google Scholar]
- Rahat, A.A.M.; Everson, R.M.; Fieldsend, J.E. Alternative Infill Strategies for Expensive Multi-objective Optimisation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17), Berlin, Germany, 15–19 July 2017; ACM: New York, NY, USA, 2017; pp. 873–880. [Google Scholar] [CrossRef] [Green Version]
- Gómez, R.H.; Coello, C.A.C. A Hyper-heuristic of Scalarizing Functions. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’17), Berlin, Germany, 15–19 July 2017; ACM: New York, NY, USA, 2017; pp. 577–584. [Google Scholar] [CrossRef]
- Deb, K.; Hussein, R.; Roy, P.C.; Toscano, G. A Taxonomy for Metamodeling Frameworks for Evolutionary Multi-Objective Optimization. IEEE Trans. Evol. Comput.
**2018**, 23, 104–116. [Google Scholar] [CrossRef] - Hussein, R.; Roy, P.C.; Deb, K. Switching between Metamodeling Frameworks for Efficient Multi-Objective Optimization. In Proceedings of the 2018 IEEE Symposium Series on Computational Intelligence (SSCI), Bangalore, India, 18–21 November 2018; pp. 1188–1195. [Google Scholar]
- Viana, F.A.C.; Haftka, R.T.; Watson, L.T. Efficient global optimization algorithm assisted by multiple surrogate techniques. J. Glob. Optim.
**2013**, 56, 669–689. [Google Scholar] [CrossRef] - Chugh, T.; Jin, Y.; Miettinen, K.; Hakanen, J.; Sindhya, K. A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization. IEEE Trans. Evol. Comput.
**2018**, 22, 129–142. [Google Scholar] [CrossRef] [Green Version] - Zhao, D.; Xue, D. A multi-surrogate approximation method for metamodeling. Eng. Comput.
**2011**, 27, 139–153. [Google Scholar] [CrossRef] - Bhattacharjee, K.; Singh, H.; Ray, T. Multi-Objective Optimization Using an Evolutionary Algorithm Embedded with Multiple Spatially Distributed Surrogates. Am. Soc. Mech. Eng.
**2016**, 138, 135–155. [Google Scholar] - Wang, H.; Jin, Y.; Doherty, J. Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems. IEEE Trans. Cybern.
**2017**, 47, 2664–2677. [Google Scholar] [CrossRef] [Green Version] - Gaspar-Cunha, A.; Vieira, A. A Multi-Objective Evolutionary Algorithm Using Neural Networks to Approximate Fitness Evaluations. Int. J. Comput. Syst. Signal
**2005**, 6, 18–36. [Google Scholar] - Rosales-Perez, A.; Coello, C.A.C.; Gonzalez, J.A.; Reyes-Garcia, C.A.; Escalante, H.J. A hybrid surrogate-based approach for evolutionary multi-objective optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC-2013), Cancun, Mexico, 20–23 June 2013; pp. 2548–2555. [Google Scholar]
- Akhtar, T.; Shoemaker, C.A. Efficient Multi-Objective Optimization through Population-based Parallel Surrogate Search. arXiv
**2019**, arXiv:1903.02167v1. [Google Scholar] - Chugh, T.; Sindhya, K.; Hakanen, J.; Miettinen, K. A survey on handling computationally expensive multiobjective optimization problems with evolutionary algorithms. Soft Comput.
**2019**, 23, 3137–3166. [Google Scholar] [CrossRef] [Green Version] - Isaacs, A.; Ray, T.; Smith, W. An evolutionary algorithm with spatially distributed surrogates for multiobjective optimization. In Proceedings of the 3rd Australian Conference on Progress in Artificial Life, Gold Coast, Australia, 4–6 December 2007; Springer: Berlin/Heidelberg, Germany, 2007; pp. 257–268. [Google Scholar]
- Habib, A.; Singh, H.K.; Chugh, T.; Ray, T.; Miettinen, K. A Multiple Surrogate Assisted Decomposition-Based Evolutionary Algorithm for Expensive Multi/Many-Objective Optimization. IEEE Trans. Evol. Comput.
**2019**, 23, 1000–1014. [Google Scholar] [CrossRef] [Green Version] - Pan, L.; He, C.; Tian, Y.; Wang, H.; Zhang, X.; Jin, Y. A Classification Based Surrogate-Assisted Evolutionary Algorithm for Expensive Many-Objective Optimization. IEEE Trans. Evol. Comput.
**2018**, 23, 74–88. [Google Scholar] [CrossRef] - Chafekar, D.; Shi, L.; Rasheed, K.; Xuan, J. Multiobjective GA optimization using reduced models. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev.
**2005**, 35, 261–265. [Google Scholar] [CrossRef] - Peitz, S.; Dellnitz, M. A Survey of Recent Trends in Multiobjective Optimal Control—Surrogate Models, Feedback Control and Objective Reduction. Math. Comput. Appl.
**2018**, 23, 30. [Google Scholar] [CrossRef] [Green Version] - Thoman, J.; Eichfelder, G. Trust-Region Algorithm for Heterogeneous Multiobjective Optimization. SIAM J. Optim.
**2019**, 29, 1017–1047. [Google Scholar] [CrossRef] [Green Version] - Banholzer, S.; Beermann, D.; Volkwein, S. POD-Based Error Control for Reduced-Order Bicriterial PDE-Constrained Optimization. Annu. Rev. Control
**2017**, 44, 226–237. [Google Scholar] [CrossRef] [Green Version] - Díaz-Manríquez, A.; Toscano, G.; Barron-Zambrano, J.H.; Tello-Leal, E. A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms. Comput. Intell. Neurosci.
**2016**, 2016, 9420460. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Deb, K.; Agrawal, S.; Pratap, A.; Meyarivan, T. A fast and Elitist multi-objective Genetic Algorithm: NSGA-II. IEEE Trans. Evol. Comput.
**2002**, 6, 182–197. [Google Scholar] [CrossRef] [Green Version] - Singh, P.; Rossi, M.; Couckuyt, I.; Deschrijver, D.; Rogier, H.; Dhaene, T. Constrained multi-objective antenna design optimization using surrogates. Int. J. Numer. Model.
**2017**, 30, e2248. [Google Scholar] [CrossRef] [Green Version] - Koziel, S.; Bekasiewicz, A.; Szczepanski, S. Multi-objective design optimization of antennas for reflection, size, and gain variability using Kriging surrogates and generalized domain segmentation. Int. J. RF Microw. Comput. Eng.
**2018**, 28, e21253. [Google Scholar] [CrossRef] - Beck, J.; Friedrich, D.; Brandani, S.; Fraga, E.S. Multi-objective optimisation using surrogate models for the design of VPSA systems. Comput. Chem. Eng.
**2015**, 82, 318–329. [Google Scholar] [CrossRef] [Green Version] - Liao, X.; Li, Q.; Yang, X.; Zhang, W.; Li, W. Multi-objective optimization for crash safety design of vehicles using stepwise regression model. Struct. Multidiscip. Optim.
**2008**, 35, 561–569. [Google Scholar] [CrossRef] - Sreekanth, J.; Datta, B. Multi-objective management of saltwater intrusion in coastal aquifers using genetic programming and modular neural network based surrogate models. J. Hydrol.
**2010**, 393, 245–256. [Google Scholar] [CrossRef] - Arias-Montaño, A.; Coello, C.A.C.; Mezura-Montes, E. Multi-objective airfoil shape optimization using a multiple-surrogate approach. In Proceedings of the 2012 IEEE Congress on Evolutionary Computation (CEC-2012), Brisbane, Australia, 10–15 June 2012; pp. 1–8. [Google Scholar]
- D’Angelo, S.; Minisci, E.A. Multi-objective evolutionary optimization of subsonic airfoils by Kriging approximation and evolution control. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC-2005), Scotland, UK, 2–5 September 2005; pp. 1262–1267. [Google Scholar]
- Alvarado-Iniesta, A.; Cuate, O.; Schütze, O. Multi-objective and many objective design of plastic injection molding process. Int. J. Adv. Manuf. Technol.
**2019**, 102, 3165–3180. [Google Scholar] [CrossRef] - Knowles, J.; Hughes, E.J. Multiobjective Optimization on a Budget of 250 Evaluations. In Evolutionary Multi-Criterion Optimization; Springer: Berlin/Heidelberg, Germany, 2005; pp. 176–190. [Google Scholar]
- Hüsken, M.; Jin, Y.; Sendhoff, B. Structure optimization of neural networks for evolutionary design optimization. Soft Comput.
**2005**, 9, 21–28. [Google Scholar] [CrossRef] [Green Version] - Pilát, M.; Neruda, R. Improving many-objective optimizers with aggregate meta-models. In Proceedings of the 2011 11th International Conference on Hybrid Intelligent Systems (HIS), Melacca, Malaysia, 5–8 December 2011; pp. 555–560. [Google Scholar] [CrossRef]
- Le, M.N.; Ong, Y.S.; Jin, Y.; Sendhoff, B. A Unified Framework for Symbiosis of Evolutionary Mechanisms with Application to Water Clusters Potential Model Design. IEEE Comput. Intell. Mag.
**2012**, 7, 20–35. [Google Scholar] [CrossRef] - Li, F.; Cai, X.; Gao, L. Ensemble of surrogates assisted particle swarm optimization of medium scale expensive problems. Appl. Soft Comput.
**2019**, 74, 291–305. [Google Scholar] [CrossRef] - Jin, C.; Qin, A.K.; Tang, K. Local ensemble surrogate assisted crowding differential evolution. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation (CEC-2015), Sendai, Japan, 25–28 May 2015; pp. 433–440. [Google Scholar]
- Deb, K. Multi-Objective Optimization Using Evolutionary Algorithms; John Wiley & Sons: Hoboken, NJ, USA, 2001. [Google Scholar]
- Srinivas, N.; Deb, K. Multi-Objective function optimization using non-dominated sorting genetic algorithms. Evol. Comput. J.
**1994**, 2, 221–248. [Google Scholar] [CrossRef] - Jones, D.R.; Schonlau, M.; Welch, W.J. Efficient Global Optimization of Expensive Black-Box Functions. J. Glob. Optim.
**1998**, 13, 455–492. [Google Scholar] [CrossRef] - Deb, K. An efficient constraint handling method for genetic algorithms. Comput. Methods App. Mech. Eng.
**2000**, 186, 311–338. [Google Scholar] [CrossRef] - Wierzbicki, A.P. The use of reference objectives in multiobjective optimization. In Multiple Criteria Decision Making Theory and Application; Springer: Berlin/Heidelberg, Germany, 1980; pp. 468–486. [Google Scholar]
- Das, I.; Dennis, J.E. Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems. SIAM J. Optim.
**1998**, 8. [Google Scholar] [CrossRef] [Green Version] - Deb, K.; Jain, H. An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: Solving problems with box constraints. IEEE Trans. Evol. Comput.
**2014**, 18, 577–601. [Google Scholar] [CrossRef] - Zitzler, E.; Thiele, L. Multiobjective optimization using evolutionary algorithms—A comparative case study. In Proceedings of the Conference on Parallel Problem Solving from Nature (PPSN V), Amsterdam, The Netherlands, 27–30 September 1998; pp. 292–301. [Google Scholar]
- Coello, C.A.C.; Sierra, M.R. A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm. In MICAI 2004: Advances in Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2004; pp. 688–697. [Google Scholar]
- Kendall, M.G. A new measure of rank correlation. Biometrika
**1938**, 30, 81–93. [Google Scholar] [CrossRef] - Miettinen, K. Nonlinear Multiobjective Optimization; Kluwer: Dordrecht, The Netherlands, 1999. [Google Scholar]
- Roy, P.C.; Blank, J.; Hussein, R.; Deb, K. Trust-region Based Algorithms with Low-budget for Multi-objective Optimization. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’18), Kyoto, Japan, 15–19 July 2018; ACM: New York, NY, USA, 2018; pp. 195–196. [Google Scholar]
- Alexandrov, N.M.; Dennis, J.E.; Lewis, R.M.; Torczon, V. A trust-region framework for managing the use of approximation models in optimization. Struct. Optim.
**1998**, 15, 16–23. [Google Scholar] [CrossRef] - Tian, Y.; Cheng, R.; Zhang, X.; Jin, Y. PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization. IEEE Comput. Intell. Mag.
**2017**, 12, 73–87. [Google Scholar] [CrossRef] [Green Version] - Allmendinger, R.; Knowles, J. ‘Hang on a minute‘: Investigations on the effects of delayed objective functions in multiobjective optimization. In Evolutionary Multi-Criterion Optimization; Purshouse, R.C., Fleming, P.J., Fonseca, C.M., Greco, S., Shaw, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 6–20. [Google Scholar]
- Blank, J.; Deb, K. Constrained Bi-objective Surrogate-Assisted Optimization of Problems with Heterogeneous Evaluation Times: Expensive Objectives and Inexpensive Constraints; Technical Report COIN Report 2020019; COIN Laboratory, Michigan State University: East Lansing, MI, USA, 2020. [Google Scholar]

**Figure 1.**The proposed taxonomy of 10 different metamodeling frameworks for multi- and many-objective optimization. (Taken from [20]).

**Figure 2.**Two objectives and two constraints are shown for SRN problem. The combined feasible region is shown in the contour plot. PO solutions lie on the black line marked inside the feasible region.

**Figure 3.**In-fill solutions for different frameworks for SRN problem. True functions are plotted here, however, different metamodeling frameworks use different approximations to find in-fill solutions on the true PO set.

**Figure 4.**In-fill solution for a specific $\mathbf{z}$-vector to be obtained by framework M5 for SRN problem.

**Figure 7.**Non-dominated solutions of the final archive for the median run of ASM approach for two-objective ZDT and constrained problems. In all cases, a well-diversified set of near PO solutions is obtained with a limited solution evaluations.

**Figure 8.**Epoch-wise proportion of appearance of 10 frameworks within ${M}_{B}$ in 11 runs of the ASM approach for ZDT problems, TNK, and welded beam design problems indicates the use of multiple frameworks during optimization. Some problems uses some specific frameworks more frequently.

**Figure 9.**Switching among frameworks for the median IGD run of the ASM approach for ZDT2, ZDT4, and ZDT6 indicates that many frameworks are used during the optimization process.

**Figure 10.**Epoch-wise proportion of usage of 10 frameworks in 11 runs of the ASM approach for three and five-objective problems.

**Table 1.**Summary of metamodeled functions and optimization algorithms needed in each epoch for all 10 frameworks.

Frame- | Metamodeling | #Metamodels | Optimization | #Opt. |
---|---|---|---|---|

Work | Functions | Method | Runs | |

M1-1 | $({\underline{f}}_{1},\dots ,{\underline{f}}_{M})$ | $M+J$ | RGA | H |

$({\underline{g}}_{1},\dots ,{\underline{g}}_{J})$ | ||||

M1-2 | Same as above | $M+J$ | NSGA-II/III | 1 |

M2-1 | $({\underline{f}}_{1},\dots ,{\underline{f}}_{M})$ & ACV | $M+1$ | RGA | H |

M2-2 | Same as above | $M+1$ | NSGA-II/III | 1 |

M3-1 | ASF${}_{34}$ & $({\underline{g}}_{1},\dots ,{\underline{g}}_{J})$ | $H+J$ | RGA | H |

M3-2 | Same as above | $H+J$ | MM-RGA | 1 |

M4-1 | ASF${}_{34}$ & ACV | $H+1$ | RGA | H |

M4-2 | Same as above | $H+1$ | MM-RGA | 1 |

M5 | ${S}_{5}$ | H | RGA | H |

M6 | ${S}_{6}$ | 1 | N-RGA | 1 |

Problem | n | M | J | ${\mathit{N}}_{0}$ | SE${}_{max}$ | H | #Epochs |
---|---|---|---|---|---|---|---|

ZDT1 | 10 | 2 | 0 | 100 | 500 | 21 | 20 |

ZDT2 | 10 | 2 | 0 | 100 | 500 | 21 | 20 |

ZDT3 | 10 | 2 | 0 | 100 | 500 | 21 | 20 |

ZDT4 | 5 | 2 | 0 | 100 | 1000 | 21 | 43 |

ZDT6 | 10 | 2 | 0 | 100 | 500 | 21 | 20 |

OSY | 6 | 2 | 6 | 200 | 800 | 21 | 29 |

TNK | 2 | 2 | 2 | 200 | 800 | 21 | 29 |

SRN | 2 | 2 | 2 | 200 | 800 | 21 | 29 |

BNH | 2 | 2 | 2 | 200 | 800 | 21 | 29 |

WB | 4 | 2 | 4 | 300 | 1000 | 21 | 39 |

DTLZ2 | 7 | 3 | 0 | 500 | 1000 | 91 | 6 |

C2DTLZ2 | 7 | 3 | 1 | 700 | 1500 | 91 | 9 |

CAR | 7 | 3 | 10 | 700 | 2000 | 91 | 15 |

DTLZ5 | 7 | 3 | 0 | 500 | 1000 | 91 | 6 |

DTLZ4 | 7 | 3 | 0 | 700 | 2000 | 91 | 15 |

DTLZ7 | 7 | 3 | 0 | 500 | 1000 | 91 | 6 |

DTLZ2-5 | 7 | 5 | 0 | 700 | 2500 | 210 | 9 |

C2DTLZ2-5 | 7 | 5 | 1 | 700 | 2500 | 210 | 9 |

**Table 3.**IGD values obtained from all the individual frameworks and proposed switching algorithm on different test problems are presented. The best performing framework and other statistically similar frameworks are marked in bold with their p-values in the second row. For problems without any constraint, the framework Mi-1 is identical to Mi-2, hence a “-” is provided for the latter. For unconstrained problems, M5 and M6 are also identical.

Problem | M1-1 | M2-1 | M1-2 | M2-2 | M3-1 | M4-1 | M3-2 | M4-2 | M5 | M6 | ASM |
---|---|---|---|---|---|---|---|---|---|---|---|

ZDT1 | 0.00090 | - | 0.00555 | - | 0.00447 | - | 0.00537 | - | - | 0.01337 | 0.00130 |

- | - | p = 0.4701 | - | p = 0.4702 | - | p = 0.7928 | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.091 | |

ZDT2 | 0.00065 | - | 0.00062 | - | 0.00568 | - | 0.00910 | - | - | 0.72366 | 0.00055 |

p = 0.2372 | - | p = 0.2372 | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | - | |

ZDT3 | 0.00531 | - | 0.00212 | - | 0.17123 | - | 0.19050 | - | - | 0.08315 | 0.00391 |

p = 0.325 | - | - | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.369 | |

ZDT4 | 0.28900 | - | 5.43450 | - | 0.29300 | - | 0.43450 | - | - | 6.15510 | 0.39992 |

- | - | p = 8.1$\times {10}^{-5}$ | - | p = 0.4307 | - | p = 0.0126 | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.1310 | |

ZDT6 | 0.37058 | - | 0.48360 | - | 0.24192 | - | 0.47159 | - | - | 0.21327 | 0.24440 |

p = 0.2934 | - | p = 8.1$\times {10}^{-5}$ | - | p = 0.8438 | - | p = 0.0013 | - | - | - | p = 0.3933 | |

OSY | 0.15323 | 24.57940 | 0.18806 | 22.99990 | 6.26550 | 18.49200 | 4.77670 | 18.33760 | 45.18110 | 57.15870 | 0.12110 |

p = 0.2301 | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

TNK | 0.00073 | 0.04383 | 0.00082 | 0.02849 | 0.01180 | 0.03332 | 0.01121 | 0.03743 | 0.03077 | 0.03990 | 0.00080 |

- | p = 8.1$\times {10}^{-5}$ | p = 0.206 | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.494 | |

SRN | 0.13191 | 4.17160 | 1.00930 | 0.92614 | 1.06120 | 1.20480 | 1.51360 | 1.48870 | 1.28450 | 2.41710 | 0.13406 |

- | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.1891 | |

BNH | 0.07885 | 0.74425 | 0.04630 | 0.04457 | 0.23728 | 0.23923 | 0.32874 | 0.36600 | 0.23699 | 0.71300 | 0.04176 |

p = 0.0865 | p = 8.1$\times {10}^{-5}$ | p = 0.5114 | p = 0.5994 | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

WB | 0.13794 | 0.55529 | 0.23159 | 0.84746 | 0.16909 | 0.88586 | 1.39250 | 3.40770 | 0.96166 | 1.41110 | 0.08960 |

p = 0.2933 | p = 8.1$\times {10}^{-5}$ | p = 0.0126 | p = 8.1$\times {10}^{-5}$ | p = 0.1007 | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

DTLZ2 | 0.07870 | - | 0.03340 | - | 0.05377 | - | 0.05040 | - | - | 0.07736 | 0.03701 |

p = 8.1$\times {10}^{-5}$ | - | - | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.562 | |

C2DTLZ2 | 0.05130 | - | 0.03355 | - | 0.03493 | - | 0.03190 | - | 0.12403 | 0.04410 | 0.03062 |

p = 8.1$\times {10}^{-5}$ | - | p = 0.115 | - | p = 0.008 | - | p = 0.148 | - | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

CAR | 0.43510 | 0.43145 | 0.50119 | 0.29817 | 0.39809 | 0.42223 | 0.40494 | 0.44251 | 0.50061 | 0.55569 | 0.40110 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | |

DTLZ5 | 0.01960 | - | 0.00948 | - | 0.01352 | - | 0.01537 | - | - | 0.05421 | 0.01252 |

p = 8.1$\times {10}^{-5}$ | - | - | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.0605 | |

DTLZ4 | 0.05840 | - | 0.09024 | - | 0.20668 | - | 0.12570 | - | - | 0.08731 | 0.07934 |

- | - | p = 0.1203 | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 0.3933 | p = 0.425 | |

DTLZ7 | 0.11808 | - | 0.07664 | - | 0.87172 | - | 1.26300 | - | - | 0.82989 | 0.06529 |

p = 0.0187 | - | p = 0.2122 | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | - | |

DTLZ2-5 | 0.21450 | - | 0.03981 | - | 0.14401 | - | 0.14403 | - | - | 0.11028 | 0.04918 |

p = 8.1$\times {10}^{-5}$ | - | - | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | - | p = 8.1$\times {10}^{-5}$ | p = 0.595 | |

C2DTLZ2-5 | 0.17341 | - | 0.03676 | - | 0.15388 | - | 0.11669 | - | 0.29291 | 0.20842 | 0.03441 |

p = 8.1$\times {10}^{-5}$ | - | p = 0.8541 | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - |

**Table 4.**Average rank of 10 frameworks and the ASM approach on 18 problems based on Wilcoxon rank-sum test.

M1-1 | M2-1 | M1-2 | M2-2 | M3-1 | M4-1 | M3-2 | M4-2 | M5 | M6 | ASM |
---|---|---|---|---|---|---|---|---|---|---|

3.66 | 6.16 | 2.88 | 3.00 | 4.55 | 5.44 | 6.22 | 6.94 | 6.33 | 8.55 | 1.11 |

**Table 5.**Median IGD on unconstrained problems using ASM approach, and MOEA/D-EGO, K-RVEA, and CSEA algorithms. DNC is denoted as “Did not converge” within given time.

Problem | MOEA/D-EGO | K-RVEA | CSEA | ASM |
---|---|---|---|---|

ZDT1 | 0.05611 | 0.07964 | 0.95330 | 0.00130 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.0910 | |

ZDT2 | 0.04922 | 0.03395 | 1.01060 | 0.00055 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

ZDT3 | 0.30380 | 0.02481 | 0.94840 | 0.00391 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

ZDT4 | 73.25920 | 4.33221 | 12.71600 | 0.39992 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | - | |

ZDT6 | 0.51472 | 0.65462 | 5.42620 | 0.24440 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.0612 | |

DTLZ2 | 0.33170 | 0.0548 | 0.11420 | 0.03701 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.157 | |

DTLZ4 | 0.64533 | 0.0449 | 0.08110 | 0.07934 |

p = 8.1$\times {10}^{-5}$ | - | p = 0.0022 | p = 0.0380 | |

DTLZ5 | 0.26203 | 0.0164 | 0.03081 | 0.01252 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | p = 0.211 | |

DTLZ7 | 5.33220 | 0.0531 | 0.70520 | 0.06529 |

p = 8.1$\times {10}^{-5}$ | - | p = 8.1$\times {10}^{-5}$ | p = 0.1930 | |

DTLZ2-5 | 0.31221 | 0.23031 | DNC | 0.04918 |

p = 8.1$\times {10}^{-5}$ | p = 8.1$\times {10}^{-5}$ | DNC | - |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Deb, K.; Roy, P.C.; Hussein, R.
Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results. *Math. Comput. Appl.* **2021**, *26*, 5.
https://doi.org/10.3390/mca26010005

**AMA Style**

Deb K, Roy PC, Hussein R.
Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results. *Mathematical and Computational Applications*. 2021; 26(1):5.
https://doi.org/10.3390/mca26010005

**Chicago/Turabian Style**

Deb, Kalyanmoy, Proteek Chandan Roy, and Rayan Hussein.
2021. "Surrogate Modeling Approaches for Multiobjective Optimization: Methods, Taxonomy, and Results" *Mathematical and Computational Applications* 26, no. 1: 5.
https://doi.org/10.3390/mca26010005