Hybridization of Multi-Objective Deterministic Particle Swarm with Derivative-Free Local Searches

Abstract

The paper presents a multi-objective derivative-free and deterministic global/local hybrid algorithm for the efficient and effective solution of simulation-based design optimization (SBDO) problems. The objective is to show how the hybridization of two multi-objective derivative-free global and local algorithms achieves better performance than the separate use of the two algorithms in solving specific SBDO problems for hull-form design. The proposed method belongs to the class of memetic algorithms, where the global exploration capability of multi-objective deterministic particle swarm optimization is enriched by exploiting the local search accuracy of a derivative-free multi-objective line-search method. To the authors best knowledge, studies are still limited on memetic, multi-objective, deterministic, derivative-free, and evolutionary algorithms for an effective and efficient solution of SBDO for hull-form design. The proposed formulation manages global and local searches based on the hypervolume metric. The hybridization scheme uses two parameters to control the local search activation and the number of function calls used by the local algorithm. The most promising values of these parameters were identified using forty analytical tests representative of the SBDO problem of interest. The resulting hybrid algorithm was finally applied to two SBDO problems for hull-form design. For both analytical tests and SBDO problems, the hybrid method achieves better performance than its global and local counterparts.

1. Introduction

The research and development of new technologies, along with a reduction of design and production costs, are enabling factors to face the challenges imposed by the worldwide-market competition. Computer simulations have played an increasingly important role in the design process of engineering products whose efficiency is greatly affected by shape parameters, such as air-, ground-, and water-borne vehicles. For these reasons, addressing real-world complex industrial applications involves high-fidelity physics-based solvers, particularly where innovative products are pursued for which past experience is not available.

In this context, the simulation-based design (SBD) paradigm has demonstrated the ability to support the design decision-making process. The recent development of high-performance computing systems has led the SBD to integrate optimization algorithms and uncertainty quantification (UQ) methods, shifting the SBD paradigm to automatic deterministic and stochastic SBD optimization (SBDO) [1] with potential global solutions to the design problem. The computational costs associated with the solution of the SBDO problem with high-fidelity solvers remain a limiting factor for the implementation of the SBDO in industries and, in particular, small- and medium-sized enterprises. The major critical issues related to computational cost and implementation complexity of the SBDO procedure can be summarized as: (a) physical solvers are computationally expensive to run; (b) they are often combined in a multidisciplinary analysis framework that increases the computational complexity; (c) solvers are often available only as a black box; (d) solutions are often affected by non-negligible residual noise; (e) assessing design performance in a variety of operating/environmental conditions requires multiple solvers to operate; (f) the search for improved performance often faces conflicting objectives; and (g) achieving optimum design through an optimization method requires a large number of evaluations, especially for high-dimensional design spaces and, if their global exploration is attempted, making high-fidelity SBDO a very complex theoretical, algorithmic, and technical task. Despite the unquestionable and significant achievements in the field, with the capability of solving high-fidelity multidisciplinary design optimization , the design-space exploration and exploitation in the quest for global optima remains a hard-to-reach objective, due to its almost-unaffordable computational cost [2], especially when dealing with multi-objective (constrained) problems.

Real-world applications are pervaded by almost conflicting objectives and can be formulated as multi-objective (constrained) problems (MOPs) [3]. For such problems, the optimum is defined by the Pareto definition, where a set of non-dominated solutions provides the trade-off among the objectives. From the optimization view point, the Pareto optimality definition separates the roles of problem solver and decision maker. The problem solver finds one or more non-dominated solutions. The decision maker, based on her/his preferences, selects one of the non-dominated solutions. For this reason, considering when the decision maker operates, the following classification of the optimization algorithms can be drawn up: (i) without preferences, decision maker preferences are indifferent and the problem is solved by finding a single non-dominated solution; (ii) a priori, the optimization procedure is driven by the knowledge of the decision maker preferences, making provision for the acceptance of the optimal solution by the decision maker; and (iii) a posteriori, decision maker preferences are taken into account at the end of the optimization. Even though there is interest in without preferences and a priori methods (capable of providing a single non-dominated solution), the a posteriori methods have a high relevance in the SBDO context, since they are able to approximate the whole set of Pareto solutions. For this purpose, evolutionary algorithms (EAs) [4] have long been used as a posteriori methods: by adapting a group of individuals, EAs naturally allow for the solution of MOPs. Nevertheless, EAs are in general stochastic and their testing and assessment requires statistically converged results (non-dominated solution set) from multiple optimization runs, which are almost unattainable for complex industrial applications. Therefore, deterministic methods have also been developed and assessed [5,6,7].

Historically, the aeronautics industry and the associated research have played a very important role in tackling and solving SBDO for complex multidisciplinary applications. For technological and computational-efficiency reasons, researchers have focused on local optimization methods, often relying on availability of derivatives from adjoint solvers [2]. On the one hand, local methods require the knowledge of derivatives, which may require intrusive approaches (e.g., adjoint methods, unfortunately not suitable for black-box solvers) and can get easily trapped in local minima. On the other hand, global optimization methods usually do not require derivatives, but are affected by the curse of dimensionality, where the complexity and associated computational cost to solve the problem increases dramatically with the dimension of the design space. Furthermore, local algorithms investigate accurately a limited domain region, also providing proof of convergence (generally not available for global methods), whereas global methods are designed to explore the whole design domain, providing approximate solutions to the decision problem. Since in the SBDO context objectives may be noisy and/or their derivatives are often not provided by the simulation tool, derivative-free optimization algorithms [8] are often a viable option. Examples of local and global derivative-free SBDO approaches to the design problem have been investigated in the ship hydrodynamic community [9], with focus on both civil and military applications and development of dual-use technologies. In the last years, several EAs have been proposed to solve design problems, such as dragonfly [10], salp swarm [11], polar bear [12], and dolphin pod optimization [13]. Among other derivative-free global methods, particle swarm optimization (PSO) [14] represents a suitable option for the solution of SBDO problems [15]. The numerical experiences reported in literature seem to indicate that it is quite efficient in identifying the regions of the feasible set where the globally optimal solutions are located [16]. Although the multi-objective extension of PSO results in an increased algorithmic complexity, the associated increased computational cost (overhead) is orders of magnitude lower than the CPU-time associated to the typical objectives evaluations in SBDO for ship hydrodynamics, and therefore not considered an issue.

In the last decades, the optimization community has explored the possibility of combining global and local approaches, with the aim of enriching global search capability with the local search accuracy in some hybrid formulations. This kind of algorithms falls in the class of memetic algorithms (MAs), where a population-based global method is coupled with an individual learning procedure capable of performing local refinements [17]. Among other MAs, PSO has been extended to single-objective hybrid global/local formulations by local line search in [18]. In the MOPs context, PSO has been combined with several local search methods, such as scatter search [19], synchronous particle local search [20], gradient-based [21], charged system search [22], modified local search [23], multi-objective dichotomy line search [24], and optimal particle search strategy [25]. Further examples of hybrid methods in the context of multi-objective optimization can be found in [26,27]. Most MAs and PSO methods ensure global exploration and solution diversity via use of random components. These require running multiple times to achieve statistically significant results, which may be not affordable in computationally-expensive SBDO problems. Therefore, deterministic PSO formulations are considered in this work.

At the present state of the art, to the authors best knowledge, studies are still limited on memetic multi-objective deterministic derivative-free EA formulations for an effective and efficient solution of SBDO for hull-form design. In earlier work [28], the authors proposed the hybridization of a multi-objective deterministic version of the PSO algorithm (MODPSO) [6] with local searches by a deterministic derivative-free multi-objective (DFMO) [29] line search method. The resulting multi-objective deterministic hybrid algorithm (MODHA) was applied to a limited number of test cases and algorithm setups. Criteria for the activation of local searches were proposed and discussed, based on the particles velocity and the hypervolume metric (HV) [30], which is a widely-used indicator for the assessment of convergence and distribution of multi-objective problem solutions [31]. Although the study was limited to only few cases/setups, HV was found the most promising criterion to drive the hybridization within MODHA. A more extensive study was recommended on the local activation and computational-cost breakdown between global and local searches.

The objective of the present work is to show how the use of a global/local hybrid algorithm based on MODPSO and DFMO achieves better performance than stand-alone global and local algorithms in solving multi-objective SBDO for hull-form design. Specifically, the problems of interest are characterized by a number of variables of the order of 10 and up to three objectives (for instance, resistance, seakeeping, and maneuverability). Specifically, the paper advances the study on MODHA presented in [28], with focus on the use of the HV metric and the identification of the proper setup for the hybridization scheme.

The proposed hybridization scheme is driven by two parameters: the first defines the threshold for the activation of local searches, whereas the second defines the “deepness” of the local search (i.e., how many function evaluations are performed at each call of the local algorithm). To identify reasonable values for these parameters, a full factorial combination of three activation and three deepness parameters is evaluated and the performance compared. Following the guidelines in [32], the assessment was targeted towards a representative benchmark of the specific SBDO problem of interest. Specifically, the benchmark was composed by 40 analytical test problems, with two and three objectives, 1 to 12 variables, and several features of variables and functions spaces. MODHA performance was assessed by the HV bounded by the non-dominated solution set. Finally, the most promising parameter setup was applied to two SBDO problems pertaining to the shape optimization of a high-speed catamaran in waves and of a small waterplane area twin hull (SWATH) model in calm water. Comparisons with stand-alone global [6] and local [29] methods are presented and discussed.

2. Multi-Objective Optimization Problem Formulation and Definitions

The general formulation of a multi-objective minimization problem reads

$$\begin{array}{cc}\hfill \mathrm{minimize}& \mathbf{f}\left(\mathbf{x}\right)=\left\{f_m\left(\mathbf{x}\right)\right\},\mathrm{with}m=1,\cdots ,M\hfill \\ \hfill \mathrm{subject}\mathrm{to}& z_i\left(\mathbf{x}\right)\le 0,\mathrm{with}i=1,\cdots ,I\hfill \\ \hfill \mathrm{and}\mathrm{to}& h_j\left(\mathbf{x}\right)=0,\mathrm{with}j=1,\cdots ,J\hfill \\ \hfill \mathrm{and}\mathrm{to}& \mathbf{l}\le \mathbf{x}\le \mathbf{u}\hfill \end{array}$$

(1)

where $\mathbf{x}\in {\mathbb{R}}^N$ is the variables vector with N the number of variables, M is the number of objectives $f_m$, $z_i$ are the inequality constraints, $h_j$ are the equality constraints, and $\mathbf{l}$ and $\mathbf{u}$ are the lower and upper bound for $\mathbf{x}$, respectively. Defining the feasible solution set as

$$\mathcal{X}=\{\mathbf{x}\in {\mathbb{R}}^N:\mathbf{l}\le \mathbf{x}\le \mathbf{u},z_i\left(\mathbf{x}\right)\le 0\forall i\in I,h_j\left(\mathbf{x}\right)=0\forall j\in J\}$$

(2)

the theoretical solution of the MOP in Equation (1) is the locus of non-dominated feasible solutions given (both in the variable and objective function space) by the Pareto solution set $\mathcal{P}$ (see Figure 1)

$$\mathcal{P}=\left\{\right(\mathbf{x},\mathbf{p}\left)\right|\mathbf{x}\in \mathcal{X},\mathbf{p}=\mathbf{f}\left(\mathbf{x}\right),\nexists \mathbf{y}\in \mathcal{X}:\mathbf{f}\left(\mathbf{y}\right)⪯\mathbf{f}\left(\mathbf{x}\right)\}$$

(3)

where $\mathbf{f}\left(\mathbf{y}\right)⪯\mathbf{f}\left(\mathbf{x}\right)$ indicates that $\mathbf{f}\left(\mathbf{y}\right)$ dominates $\mathbf{f}\left(\mathbf{x}\right)$ (see, e.g. [33]).

In the following, the approximate and discrete solution set ${\mathcal{S}}^i$ (see Figure 1) achieved by the optimizer at the ith iteration and used for the algorithm performance assessment is defined similarly to Equation (3) as

$${\mathcal{S}}^i=\left\{\left(\mathbf{x},\mathbf{s}\right)\right|\mathbf{x}\in {\mathcal{X}}^i,\mathbf{s}=\mathbf{f}\left(\mathbf{x}\right),\nexists \mathbf{y}\in {\mathcal{X}}^i:\mathbf{f}\left(\mathbf{y}\right)⪯\mathbf{f}\left(\mathbf{x}\right)\}$$

(4)

where ${\mathcal{X}}^i$ is the subset of feasible points produced by the algorithm and available at the ith iteration.

Likewise, the approximate and discrete reference solution set $\mathcal{R}$ used for the performance analysis is defined as

$$\mathcal{R}=\{\left(\mathbf{x},\mathbf{r}\right)\in {\cup }_{n=1}^{N_{\mathrm{s}}}{\mathcal{S}}_n|\nexists (\mathbf{y},\mathbf{s})\in {\cup }_{n=1}^{N_{\mathrm{s}}}{\mathcal{S}}_n:\mathbf{s}⪯\mathbf{r}\}$$

(5)

where $N_{\mathrm{s}}$ is the number of final solution sets ${\mathcal{S}}_n$ provided by different algorithm formulations and/or setups.

Performance Metrics

The performance assessment of a multi-objective optimization algorithm can be established in terms of convergence and diversity: the former is related to the distance between $\mathcal{S}$ and $\mathcal{R}$, whereas the latter deals with the wideness and the spread of $\mathcal{S}$. A suitable convergence-diversity metric is the HV [30]. It provides the (hyper)volume dominated by the solution set $\mathcal{S}$, evaluated as follows

$$\mathrm{HV}(\mathcal{S},\mathcal{R})=\mathrm{volume}\left(\bigcup _{i=1}^{\left|\mathcal{S}\right|}{v}_i\right)$$

(6)

by using the anti-ideal point $\mathcal{A}$ of the reference solution set $\mathcal{R}$ [33] as a reference point, as shown in Figure 1.

Herein, to assess and compare the performance of different algorithms, a normalized version of HV is used

$$\mathrm{NHV}=\frac{\mathrm{HV}(\mathcal{S},\mathcal{R})}{\mathrm{HV}(\mathcal{R},\mathcal{R})}$$

(7)

To further assess the impact of the kth setting parameter on the algorithm performance, the relative variability ${\sigma }_{r,k}^2$[34] is additionally used. Considering the algorithm parameters vector as $\mathbf{t}\in \mathcal{T}$, the relative performance variability associated to its kth component is

$${\sigma }_{r,k}^2=\frac{{\sigma }_k^2}{{\sum }_{k=1}^{\left|\mathcal{T}\right|}{\sigma }_k^2}$$

(8)

where

$${\sigma }_k^2=\frac{1}{|\Gamma |}\sum _{\gamma \in \Gamma }{\left[{\widehat{\mu }}_k\left(\gamma \right)\right]}^2-{\left[\frac{1}{|\Gamma |}\sum _{\gamma \in \Gamma }{\widehat{\mu }}_k\left(\gamma \right)\right]}^2$$

(9)

with $\Gamma$ containing the positions $\gamma$ assumed by the parameter $t_k$,

$${\widehat{\mu }}_k\left(\gamma \right)=\frac{1}{\left|\mathcal{B}\right|}\sum _{b\in \mathcal{B}}\overline{\mu }\left(\mathbf{t}\right),\mathcal{B}=\{\mathbf{t}:t_k=\gamma \}\mathrm{and}\overline{\mu }\left(\mathbf{t}\right)=\frac{1}{\left|\mathcal{Q}\right|}\sum _{q\in \mathcal{Q}}{\left[\mu \left(\mathbf{t}\right)\right]}_q$$

(10)

where ${\left[\mu \left(\mathbf{t}\right)\right]}_q$ is the value of the NHV given by the parameters $\mathbf{t}$, for the problem q.

3. Multi-Objective Deterministic Hybrid Algorithm: MODHA

The proposed global/local hybrid algorithm is based on a multi-objective deterministic version of the particle swarm optimization [6] and on the derivative-free multi-objective local searches algorithm [29]. The following subsections present the global and local algorithms and finally the hybridization proposed.

3.1. Multi-Objective Deterministic Particle Swarm Optimization (MODPSO)

The PSO algorithm [14] belongs to the class of metaheuristic algorithms for single-objective derivative-free global optimization. It is based on the social-behavior metaphor of a flock of birds or a swarm of bees searching for food. The original PSO formulation makes use of random coefficients, to enhance the swarm dynamics’ variability. This approach might be too expensive in SBDO for real industrial applications, since statistically convergent results can be obtained only through extensive numerical campaigns. Therefore, a deterministic PSO is introduced in [35] and a discussion for its effective and efficient use in SBDO, with comparison to the original (stochastic) PSO, is presented in [36]. The extension of deterministic PSO to a multi-objective formulation is proposed in [37] as follows

$$\left\{\begin{array}{cc}\hfill {\mathbf{v}}_j^{i+1}& =\chi \left[{\mathbf{v}}_j^i+{\varphi }_1\left({\mathbf{h}}_j-{\mathbf{x}}_j^i\right)+{\varphi }_2\left({\mathbf{g}}_j-{\mathbf{x}}_j^i\right)\right]\hfill \\ \hfill {\mathbf{x}}_j^{i+1}& ={\mathbf{x}}_j^i+{\mathbf{v}}_j^{i+1}\hfill \end{array}\right.$$

(11)

where ${\mathbf{v}}_j^i$ and ${\mathbf{x}}_j^i$ are the velocity and the position of the jth particle at the ith iteration, $\chi$ is the constriction factor, ${\varphi }_1$ and ${\varphi }_2$ are the cognitive (or personal) and social (or global) learning rates, and ${\mathbf{h}}_j$ and ${\mathbf{g}}_j$ are the cognitive and social (personal and global) attractors. Specifically, ${\mathbf{h}}_j$ is the personal minimizer of an aggregated objective function defined as

$${\mathbf{h}}_j=\underset{{\mathbf{x}}_{h,j}}{\mathrm{argmin}}\sum _{m=1}^M{f}_m\left({\mathbf{x}}_{h,j}\right)\mathrm{with}{\mathbf{x}}_{h,j}^i\in {\mathcal{S}}_j^i$$

(12)

where ${\mathbf{x}}_{h,j}^i$ are the points of the personal non-dominated solution set ${\mathcal{S}}_j^i$ at the ith iteration, whereas ${\mathbf{g}}_j$ is the closest point to the jth particle of the solution set ${\mathcal{S}}^i$ defined as

$${\mathbf{g}}_j=\underset{{\mathbf{x}}_j}{\mathrm{argmin}}\parallel {\mathbf{x}}_j^i-\mathbf{x}\parallel \mathrm{with}\mathbf{x}\in {\mathcal{S}}^i$$

(13)

Here, a memory-based formulation of MODPSO is used, where all points $\mathbf{x}$ ever visited until the current iteration are assessed in Equation (4).

A discussion on MODPSO formulations and parameters setup for an effective and efficient use in SBDO is presented in [6]. The current MODPSO code is available at github.com/MAORG-CNR-INM/POT.

3.2. Derivative-Free Multi-Objective Local Searches (DFMO)

DFMO [29] is an a posteriori derivative-free algorithm for constrained (possibly) non-smooth multi-objective problems. The algorithm produces (or updates) a set of non-dominated solutions (rather than a single one, as it is common in the single-objective case), tending towards the $\mathcal{P}$ of the problem, as the iteration count grows.

At each iteration (see Figure 2), from each point ${\mathbf{x}}_j\in {\mathcal{S}}^i$, DFMO starts a line-search along a suitably generated direction ${\widehat{d}}_k$. If such a direction is able to guarantee “sufficient” decrease, then a “sufficiently” large step $\lambda$ along this direction is performed, allowing to (possibly) improve $\mathcal{S}$.

It is worth noting that DFMO belongs to a particular class of derivative-free methods proposed to tackle optimization problems where first order derivatives of the objective function and/or of the constraints can be neither calculated nor explicitly approximated [38]. In particular, DFMO does not perform any line-searches along approximations of gradient-related directions but rather along directions that are not affected by possible errors on the calculation of the objective function and constraints. Further details on DFMO formulation and implementation can be found in [29]. DFMO source code is available at www.iasi.cnr.it/liuzzi/DFL/index.php/home.

3.3. Hybridization Scheme

When and where from the local search starts, represent one of the critical issues when combining global and local optimization methods. Herein, the proposed hybridization scheme uses the HV metric. Defining the local search activation parameter as

$$\alpha ={\displaystyle \frac{\mathrm{HV}({\mathcal{S}}^i,{\mathcal{S}}^i)}{\mathrm{HV}({\mathcal{S}}^{i-1},{\mathcal{S}}^i)}}$$

(14)

where $\mathrm{HV}({\mathcal{S}}^i,{\mathcal{S}}^i)$ is the hypervolume associated to ${\mathcal{S}}^i$ and $\mathrm{HV}({\mathcal{S}}^{i-1},{\mathcal{S}}^i)$ is the hypervolume associated to solution set at the previous algorithm iteration ${\mathcal{S}}^{i-1}$, the hybrid algorithm starts a local search from each point of the current solution set if $\alpha <{\alpha }^{\star }$, with ${\alpha }^{\star }$ a prescribed threshold value.

The number of problem evaluations performed at each call of the local algorithm ($\mathcal{L}$) is defined as $\mathcal{L}=\omega \mathcal{Z}$, with $\omega$ the local search deepness parameter and $\mathcal{Z}$ the number of particles. Hybridization scheme block-diagram and pseudo-code are shown in Figure 3 and Algorithm 1, respectively.

Algorithm 1 MODHA pseudo-code.

1: Initialize a swarm of $\mathcal{Z}$ particles 2: while$i<$ Maximum number of iterations (Maximum budget of problem evaluations) do 3:  for $j=1,\mathcal{Z}$ do 4:   Evaluate the objective functions $\mathbf{f}$ 5:  end for 6:  Compute the personal and global non-dominated solution sets 7:  for $j=1,\mathcal{Z}$ do 8:   Identify cognitive ${\mathbf{h}}_j$ and social ${\mathbf{g}}_j$ attractors, 9:   Update particles velocity ${\mathbf{v}}_j^{i+1}$ and position ${\mathbf{x}}_j^{i+1}$ 10:  end for 11:  Evaluate the local search activation parameter ($\alpha$) 12:  if Condition for performing local search is true ($\alpha <{\alpha }^{\star }$) then 13:   Define $\mathcal{D}=2N$ coordinate directions ${\widehat{d}}_k$ 14:   Identify $\mathcal{N}$ starting points for the local search 15:   for $n=1,\mathcal{N}$ do 16:    Set the number of local search $\mathcal{L}$ to zero 17:    for $k=1,\mathcal{D}$ do 18:    while $\mathcal{L}<\omega \mathcal{Z}$ and $\lambda >{\lambda }_{min}$ do 19:     Perform one step equal to $\lambda$ along ${\widehat{d}}_k$ from the nth starting point 20:      Evaluate the objective functions $\mathbf{f}$ 21:      Set $\mathcal{L}$ to $\mathcal{L}+1$ 22:      if At least one objective function improves then 23:       Update local search solution set and go to step 18 24:      else 25:       Reduce $\lambda$ 26:      end if 27:     end while 28:    end for 29:   end for 30:   Update the non-dominated solution set ${\mathcal{S}}^i$ with local search solution set 31:  end if 32:  Update the number of problem evaluations performed 33: end while 34: Output the non-dominated solution set ${\mathcal{S}}^i$

4. Analytical Test Problems

The most promising activation and deepness parameter values are identified using a test set representative of a SBDO problem with up to three objectives and a number of variables of the order of 10. Specifically, 40 real-valued test problems were selected from the literature [39], with two and three objectives and 1 to 12 variables. The selected test problems include several features, such as convex/concave/mixed/disconnected Pareto fronts or some combination, separable/non-separable variables, Pareto one-to-one/many-to-one, and uni/multi-modality [40]. The multi-objective test problems are summarized in

Table 1. Analytical test problems.

Problem q	Name	Reference	N	M
1	Deb 4.1	[42]	2	2
2	Deb 5.3	[42]	2	2
3	Deb 5.1.3	[42]	2	2
4	DTLZ1	[43]	7	3
5	DTLZ3	[43]	12	3
6	DTLZ3n2	[43]	2	2
7	DTLZ5	[43]	12	3
8	F2	[44]	2	2
9	Far1	[40]	2	2
10	FES2	[40]	10	3
11	I2	[40]	8	3
12	I5	[40]	8	3
13	IKK1	[40]	2	3
14	IM1	[40]	2	2
15	lovison4	[45]	2	2
16	lovison5	[45]	3	3
17	lovison6	[45]	3	3
18	MOP3	[40]	2	2
19	MOP4	[40]	3	2
20	MOP6	[40]	2	2
21	Sch1	[40]	1	2
22	TKLY1	[40]	4	2
23	VU2	[40]	2	2
24	WFG4	[40]	8	3
25	ZDT6	[46]	10	2
26	Freudenstein-Roth—Multi Modal	[34]	2	2
27	Freudenstein-Roth—Sphere	[34]	2	2
28	Freudenstein-Roth—Styblinski-Tang	[34]	2	2
29	Freudenstein-Roth—Three-Hump Camel Back	[34]	2	2
30	Levy 5 —Schubert	[34]	2	2
31	Levy 10—Griewank	[34]	2	2
32	Levy 15—Ackley	[34]	2	2
33	Schubert P1—Matyas	[34]	2	2
34	Schubert P2—Exponential	[34]	2	2
35	Sphere—Booth	[34]	2	2
36	Sphere—Schubert P1	[34]	2	2
37	Sphere—Six-Hump Camel Back	[34]	2	2
38	Test Tube Holder—Ackley	[34]	2	2
39	Test Tube Holder—Schubert	[34]	2	2
40	Test Tube Holder—Schubert P1	[34]	2	2

. Note that problems $q=26,\dots ,40$ are multi-objective combination of well-known single-objective test functions.

To provide a proper comparison between different problems with different function-space range, the non-dominated solution set $\mathcal{S}$ for each problem was normalized with the functions range, therefore each non-dominated solution ${\mathbf{s}}_j\in [0,1]$ and the anti-ideal reference point for the HV is ${\left\{1\right\}}_{m=1}^M$ (see Figure 1). The code provided in [41] is used for the computation of the HV metric.

5. Simulation-Based Design Optimization Problems

Two SBDO problems were solved pertaining to hull-form optimization of a high-speed catamaran and a SWATH model. Details are provided in the following subsections.

5.1. Catamaran Problem

The stochastic (reliability-based and robust) design optimization of a high-speed catamaran in irregular head waves was solved [1]. Realistic environment conditions are associated to the North Pacific Ocean, whereas operating conditions include stochastic sea state and speed. A multi-objective problem ($M=2$) was solved, aiming at the reduction of the expected value of the mean total resistance in irregular waves ($f_1$) and the increase of the ship operational effectiveness (operability) referring to a set of motion-related constraints ($f_2$). The SBDO problem reads

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\{f_1\left(\mathbf{x}\right),-f_2\left(\mathbf{x}\right)\}}^{\mathsf{T}}\hfill \\ \hfill \mathrm{subject}\mathrm{to}& f_1\le 0;f_2\ge 0\hfill \\ \hfill \mathrm{and}\mathrm{to}& \mathbf{l}\le \mathbf{x}\le \mathbf{u}\hfill \end{array}$$

(15)

The problem was solved by means of stochastic radial-basis function (SRBF) interpolation [47] of high-fidelity unsteady Reynolds-averaged Navier–Stokes (URANS) simulations (CFDShip-Iowa v4.5 [48]). Further details on design variables, objective functions, constraints, and conditions can be found in [48]. A snapshot of the catamaran behavior in irregular head waves computed by the URANS solver is shown in Figure 4.

The inequalities in Equation (15) are handled by a penalty function, so that, if $f_1>0$ or $f_2<0$,

$$f_m=10000f_m\mathrm{for}m=1,2$$

(16)

whereas if domain bounds violation occurs

$$f_m=f_m+100\sum _{j=1}^{N}max(x_j-u_j,0)+100\sum _{j=1}^{N}max(l_j-x_j,0)\mathrm{for}m=1,2$$

(17)

Four design variables ($N=4$) control the catamaran hull global modifications, based on the Karhunen–Loève expansion (KLE) of the shape modification vector [49] based on free-form deformation. Further details can be found in [1]. The reference non-dominated solution set $\mathcal{R}$ is taken from [6].

5.2. SWATH Problem

A multi-objective optimization problem ($M=2$) was solved for calm water resistance ($f_1$) and displacement ($f_2$) at constant speed [51]. The SBDO problems reads

$$\begin{array}{cc}\hfill \mathrm{minimize}& {\{f_1\left(\mathbf{x}\right),-f_2\left(\mathbf{x}\right)\}}^{\mathsf{T}}\hfill \\ \hfill \mathrm{and}\mathrm{to}& \mathbf{l}\le \mathbf{x}\le \mathbf{u}\hfill \end{array}$$

(18)

The hydrodynamic resistance was evaluated by an adaptive multi-fidelity SRBF metamodel [52] trained with high-fidelity URANS ($\chi$navis [53]) and low-fidelity potential flow (WARP [54]) solvers. Further details on design variables, objective functions, constraints, and conditions can be found in [51]. Figure 5 shows non-dimensional pressure distribution on the SWATH torpedoes and wave elevation for the original geometry by potential flow simulation.

Four design variables ($N=4$) control SWATH hull global modifications, based on the KLE of the shape modification vector [49] based on a CAD parametric model developed with the CAESES^® software from FRIENDSHIP SYSTEMS [51]. The reference non-dominated solution set $\mathcal{R}$ is taken from [55].

6. Numerical Results

The MODPSO setup was defined as in [6]: number of particles $\mathcal{Z}$ equal to $8MN$, initialized using a Hammersley sequence sampling [56] over variables domain and boundaries; coefficients $\chi =0.721$ and ${\varphi }_1={\varphi }_2=1.655$ [57]; box constraints handled by a semi-elastic wall-type method [15].

Three threshold values for DFMO activation parameter (see Equations (14)) were set as ${\alpha }^{\star }=\{1.0,1.1,1.2\}$. Three deepness parameters (budget of local search evaluations for each call) were set as $\omega =\{1,5,10\}$. The line-search step $\lambda$ was halved at each local search step until it reached a minimum step size ${\lambda }_{min}=1$× 10${}^{-9}$. The starting local search step size was set equal to the 10% of the variables-space dimension.

The computational budget was assessed by $\nu MN$ problem evaluations (one problem evaluation involves the evaluation of each objective function) where $\nu =125\times 2^c,c\in \mathbb{N}[0,4]$ therefore ranges between $125MN$ and $2000MN$. The MODHA formulations under analysis performs $\omega \mathcal{Z}$ local search for each call, starting from the current solution set ${\mathcal{S}}^i$, and are activated by the HV threshold value $\alpha$ (see Equation (14)).

A preliminary study on the analytical test problems (see Table 1) was used to identify the most promising values of the activation and deepness parameters. The most promising parameters setup was finally applied to the two SBDO problems and compared with its global (MODPSO) and local (DFMO) counterparts, using a budget of $2000MN$ problem evaluations.

6.1. Analytical Benchmark Problems

Figure 6 shows the NHV metric versus the budget of problem evaluations. Average values are presented, conditional to the local search activation parameter $\alpha$ (Figure 6 left) and deepness parameter $\omega$ (Figure 6 right). It can be noted how MODHA effectiveness improves by using higher values of $\alpha$, meaning that the hybrid formulation is pushed to exploit local searches capability and its effectiveness is enriched by global exploration only if a significant improvement of the HV is achieved between iterations (i.e., the global exploration improves the algorithm performance if the HV improves by 10–20% between MODHA iterations). Considering the deepness parameter $\omega$, the use of a low number of local search ($\omega =1$, at each DFMO call) is more effective when a low computational budget is available. On the contrary (using higher budgets), the “deeper” is the local search, the greater is the effectiveness of the hybrid formulation.

Figure 7 shows the relative variance ${\sigma }_r^2$ of the NHV retained by each of the aforementioned parameters. The hybrid formulation is mainly affected by the local searches activation parameter $\alpha$ rather than by the actual number of local searches (related to $\omega$).

The MODHA performance is summarized in

Table 2. MODHA average performance along with comparison to MODPSO and DFMO: test problems NHV mean values and (in parenthesis) standard deviation.

MODHA	Computational Budget Coefficient $\mathit{\gamma }$
Parameters	125	250	500	1000	2000	Average
$\alpha =1.0$, $\omega =1$	0.9458	0.9641	0.9725	0.9751	0.9770	0.9669
	(9.0934 × 10${}^{-2}$)	(7.7112 × 10${}^{-2}$)	(7.3252 × 10${}^{-2}$)	(7.0314 × 10${}^{-2}$)	(6.8226 × 10${}^{-2}$)	(7.5968 × 10${}^{-2}$)
$\alpha =1.0$, $\omega =5$	0.9464	0.9651	0.9745	0.9788	0.9808	0.9691
	(9.2159 × 10${}^{-2}$)	(7.7439 × 10${}^{-2}$)	(7.1709 × 10${}^{-2}$)	(6.8668 × 10${}^{-2}$)	(6.6934 × 10${}^{-2}$)	(7.5382 × 10${}^{-2}$)
$\alpha =1.0$, $\omega =10$	0.9462	0.9643	0.9743	0.9797	0.9817	0.9692
	(9.4873 × 10${}^{-2}$)	(7.6855 × 10${}^{-2}$)	(7.1750 × 10${}^{-2}$)	(6.7123 × 10${}^{-2}$)	(6.5631 × 10${}^{-2}$)	(7.5246 × 10${}^{-2}$)
$\alpha =1.1$, $\omega =1$	0.9634	0.9704	0.9854	0.9896	0.9915	0.9800
	(5.8652 × 10${}^{-2}$)	(5.0271 × 10${}^{-2}$)	(3.1711 × 10${}^{-2}$)	(2.3128 × 10${}^{-2}$)	(1.5723 × 10${}^{-2}$)	(3.5897 × 10${}^{-2}$)
$\alpha =1.1$, $\omega =5$	0.9580	0.9682	0.9742	0.9891	0.9941	0.9767
	(7.0017 × 10${}^{-2}$)	(6.6010 × 10${}^{-2}$)	(5.6875 × 10${}^{-2}$)	(2.4307 × 10${}^{-2}$)	(1.4171 × 10${}^{-2}$)	(4.6276 × 10${}^{-2}$)
$\alpha =1.1$, $\omega =10$	0.9583	0.9617	0.9815	0.9885	0.9933	0.9767
	(7.1280 × 10${}^{-2}$)	(6.9339 × 10${}^{-2}$)	(3.8687 × 10${}^{-2}$)	(2.3486 × 10${}^{-2}$)	(1.7157 × 10${}^{-2}$)	(4.3989 × 10${}^{-2}$)
$\alpha =1.2$, $\omega =1$	0.9675	0.9741	0.9859	0.9893	0.9910	0.9815
	(5.5413 × 10${}^{-2}$)	(4.6050 × 10${}^{-2}$)	(3.1321 × 10${}^{-2}$)	(2.6102 × 10${}^{-2}$)	(1.9876 × 10${}^{-2}$)	(3.5798 × 10${}^{-2}$)
$\alpha =1.2$, $\omega =5$	0.9621	0.9710	0.9765	0.9910	0.9939	0.9789
	(6.6240 × 10${}^{-2}$)	(6.0512 × 10${}^{-2}$)	(5.1914 × 10${}^{-2}$)	(2.3339 × 10${}^{-2}$)	(1.5871 × 10${}^{-2}$)	(4.3575 × 10${}^{-2}$)
$\alpha =1.2$, $\omega =10$	0.9623	0.9664	0.9785	0.9858	0.9924	0.9771
	(6.4324 × 10${}^{-2}$)	(6.1208 × 10${}^{-2}$)	(4.3171 × 10${}^{-2}$)	(2.9605 × 10${}^{-2}$)	(2.2869 × 10${}^{-2}$)	(4.4235 × 10${}^{-2}$)
MODSPO	0.9493	0.9647	0.9702	0.9715	0.9725	0.9659
	(8.6609 × 10${}^{-2}$)	(7.6035 × 10${}^{-2}$)	(7.4521 × 10${}^{-2}$)	(7.4430 × 10${}^{-2}$)	(7.3635 × 10${}^{-2}$)	(7.6978 × 10${}^{-2}$)
DFMO	0.9530	0.9594	0.9653	0.9684	0.9708	0.9641
	(9.0301 × 10${}^{-2}$)	(8.6761 × 10${}^{-2}$)	(8.2138 × 10${}^{-2}$)	(8.0448 × 10${}^{-2}$)	(7.9084 × 10${}^{-2}$)	(8.3362 × 10${}^{-2}$)

. For each budget of problem evaluations, the NHV mean values are provided along with standard deviations. Furthermore, budget-averaged performance is also provided. The hybridization scheme with $\alpha =1.2$ and $\omega =1$ is the most effective on average, providing the highest NHV with the lowest standard deviation, in accordance with results shown in Figure 6.

Finally, MODHA with the most promising parameters setup ($\alpha =1.2$, $\omega =1$) is compared with MODPSO and DFMO. Comparison results are shown in Figure 8 and summarized in Table 2. MODHA shows better average performance than its global and local counterparts. Furthermore, the results achieved from MODHA with the most promising parameters setup are found always very close to the results achieved with the ”best” parameters setup (see Figure 8). Some illustrative examples of the non-dominated solution sets achieved by MODPSO, DFMO, and MODHA are shown in Figure 9. It can be noted how MODHA improves the quality and accuracy of the non-dominated solution sets (obtained by MODPSO and DFMO) in terms of both convergence towards the reference and diversity of the solutions.

6.2. Simulation-Based Design Optimization Problems

Figure 10 shows the non-dominated solution sets obtained for the catamaran and the SWATH problems by MODPSO, DFMO, and their hybridization MODHA with the most promising parameters setup ($\alpha =1.2$, $\omega =1$). Considering the catamaran problem (see Figure 10a), the hybrid algorithm can cover effectively the reference solution, outperforming its global and local counterparts. It can be noted that MODHA is able to accurately identify the upper right region of the reference solution set $\mathcal{R}$. Considering the SWATH problem, MODHA clearly improves the quality and accuracy of the non-dominated solution set obtained by MODPSO, whereas it is comparable with DFMO. SBDO problems results are summarized in

Table 3. SBDO problems, summary of the optimization results in terms of NHV *.

Problem	N	M	MODPSO	DFMO	MODHA
Catamaran	4	2	0.9871	0.9895	0.9935
			(915)	(998)	(1774)
SWATH	4	2	0.9732	0.9947	0.9900
			(210)	(367)	(341)

* In parenthesis is the size of the non-dominated solution set.

, showing the NHV metric along with the number of the non-dominated solutions achieved. MODHA achieves the best performance for the catamaran problem, whereas MODHA and its local counterpart DFMO are comparable for the SWATH problem.

7. Conclusions and Future Work

A global/local hybridization of the multi-objective deterministic particle swarm formulation with a local derivative-free multi-objective line-search algorithm is presented, for the effective and efficient solution of multi-objective SBDO for hull-form design. Specifically, the problems of interest are characterized by a number of variables of the order of 10 and up to three objectives. A metric-based memetic scheme is proposed. The hypervolume metric is selected for the current formulation, since it provides information on both convergence and diversity of the non-dominated solution set. The hybridization scheme is “dual” since both global and local algorithms are mutually enriched. Specifically, MODHA starts with the global exploration of the design variables space; if the HV associated to the current solution set does not increase sufficiently (by a factor $\alpha$) with respect to the previous iteration, then MODHA activates the local exploitation starting the local search from each point of the current non-dominated solution set and performing a prescribed number of problem evaluations (proportional to the parameter $\omega$). This process iterates until the budget of problem evaluations is consumed.

With the aim of identifying reasonable values for the activation ($\alpha$) and deepness ($\omega$) parameters, a full factorial combination of these was evaluated and the performance compared. The comparison was performed on a test set of 40 analytical test problems, with 2–3 objectives, 1–12 variables, and several features of variables and function spaces. The test functions were selected as representative of the specific SBDO problem of interest. The results are presented and discussed based on the normalized HV, assessing both the convergence and the diversity of the non-dominated solution set.

The performance of the hybridization scheme depends significantly on the local search activation parameter $\alpha$. The parameters setup formed by $\alpha =1.2$ and $\omega =1$ is the most effective overall. This was compared to its global and local counterpart, whose setting parameters were taken from earlier studies [6,29]. The proposed memetic algorithm showed the best performance on average. MODHA algorithm with the most promising parameters setup was finally applied to the two simulation-based design optimization problems of a high-speed catamaran (aimed at reducing the resistance and increasing the operability in realistic ocean conditions) and a SWATH model (aimed at reducing the calm water resistance and increasing the hull displacement), showing better results than global and local algorithms.

The current achievements motivate further studies and analysis of the proposed metrics-based formulation, focusing on the criterion for the selection of the local-search starting points, such as subsampling of the non-dominated solution set or the use of crowding-distance metrics. Furthermore, the analysis of the effects of the local search maximum and minimum step size (on the overall performance) will be addressed. Future work includes also the investigation of alternative multi-objective PSO formulations, such as the crowding-distance-based multi-objective PSO [58].