## 1. Introduction

In recent years, robust design optimization (RDO) [

1] has received increasing interest in engineering applications, due to its ability to provide efficient designs with a stable behavior under uncertainties of a diverse nature, such as randomly fluctuating operating conditions, geometric tolerances, and model uncertainties. Taguchi’s method [

2], relying on the simultaneous optimization of the average and variance of the stochastic cost functions, is by far the most popular RDO method, although approaches allowing accounting for rare events, such as the low-quantile [

3,

4] or the “horsetail matching” [

5] methods, have been paid significant interest recently.

The main ingredient for RDO is an uncertainty quantification (UQ) method, allowing characterizing the probability distribution functions (pdf) or, at least, the lower order statistics of the cost functionals for each proposed design, in order to select those that guarantee the best possible average performance while avoiding critical deviations when nominal design conditions are not matched. According to the RDO method in use, a single objective deterministic design problem is generally converted into a multi-objective (Pareto front) one, with the aim to optimize the average performance while avoiding critical performance loss at off-design conditions. For this reason, RDO often combines an UQ solver with evolutionary algorithms (typically, multi-objective genetic algorithms (MOGA) [

6,

7]), which are naturally suited for providing a full set of compromise solutions among the multiple objectives. On the other hand, evolutionary optimizers are generally very demanding in terms of cost function evaluations, which may require in the end a prohibitive computational effort for problems described by costly computer models, such as those encountered in computational fluid dynamics (CFD), despite the use of massive parallelization [

8,

9,

10]. In order to reduce the number of costly function calls, it is crucial to select parsimonious UQ methods and optimizers, the overall cost of RDO being typically the product of the cost of the two approaches [

11]. Past examples of RDO in CFD include various forms of UQ solvers based on non-intrusive polynomial chaos expansion [

11,

12] or surrogate models such as simplex stochastic collocation [

13] or kriging [

9]. All of them require a number of CFD solves quickly increasing with the number of uncertain parameters, and their direct coupling with MOGA optimizers is not computationally affordable for industrial applications, especially if massively parallel computers are not available.

An interesting option for reducing the cost of UQ solves is to use gradient information. A simple method for approximating statistical moments of the cost function by Taylor series expansions is the so-called first-order method of moments (MoM) [

14]. Such a method can be remarkably fast if the derivatives of the cost function with respect to the uncertain variables are readily available by means of a discrete or continuous adjoint solver [

15,

16,

17,

18,

19,

20]. Nevertheless, its accuracy is limited to Gaussian or weakly non-Gaussian processes with small uncertainties, since higher-order terms become increasingly important for strongly non-Gaussian input distributions. Some improvement can be achieved by using higher-order expansions, but these require information about higher-order sensitivity derivatives, which may represent a delicate and highly intrusive task. A more complete discussion can be found in [

21]. An alternative to MoM, better suited for high uncertainty ranges and generic pdf, consists of leveraging gradient information to construct a high-quality surrogate from a reduced number of samples. Such an approach is used for instance in gradient-enhanced kriging (GEK) surrogates [

22,

23,

24]. Once the surrogate is available, inexpensive Monte Carlo sampling on the response surface can be used to estimate the required statistics.

Massive parallelization is of great help for speeding up the RDO process [

10,

11,

25], but it is not promptly applicable for routine industrial use. A way of drastically reducing the required number of function calls consists of replacing the costly CFD or UQ solvers with surrogate models, such as radial basis functions [

26], artificial neural networks [

27], and kriging [

9,

10], approximating variations of the cost functions through the design space. Such an approach is called a surrogate-based multi-objective genetic algorithm (SMOGA).

Further reductions of computational time can be achieved by combining models with various levels of fidelity during the optimization. Such so-called multi-fidelity (MF) models [

28,

29] leverage the use of inexpensive, but low-fidelity (LF) models for efficiently exploring the design or stochastic spaces, while using parsimonious high-fidelity (HF) samples to improve model accuracy. Examples of LF models are given by coarse-grid approximations [

30,

31,

32,

33], data-fit interpolation and regression models [

34], projection-based reduced models [

35,

36], machine-learning-based models [

37], or simplified models relying on approximations of the underlying physics [

38,

39,

40]. In addition to an LF and an HF model, a correlation model is also required for combining data with various fidelity levels. A simple approach consists of linking the HF and the LF models by means of an additive correlation [

41]: given an LF model

${f}_{LF}\left(\xi \right)$ and an HF model

${f}_{HF}\left(\xi \right)$, it is assumed that

${f}_{HF}\left(\xi \right)={f}_{LF}\left(\xi \right)+\delta \left(\xi \right)$ where

$\delta \left(\xi \right)$ is an error function to be estimated. This approach is accurate enough when HF and LF models have similar scales and a good correlation, as is the case for coarse-grid approximations [

42]. As an alternative, a multiplicative correlation can be used [

43,

44,

45]:

${f}_{HF}\left(\xi \right)=\rho \left(\xi \right){f}_{LF}\left(\xi \right)$, with

$\rho \left(\xi \right)$ a constant scalar multiplier. A more comprehensive formulation combining the preceding ones was proposed in [

32]:

${f}_{HF}\left(\xi \right)=\rho {f}_{LF}\left(\xi \right)+\delta \left(\xi \right)$. This approach is considerably more robust, and it has been extensively used in conjunction with Gaussian process approximation (including kriging) and Bayesian inference [

46,

47,

48,

49]). More sophisticated correlations exist [

50,

51,

52], but they may be difficult to implement for complex problems.

In the present paper, we build on a SMOGA-based RDO technique introduced in [

10], relying on the coupling of two nested Bayesian kriging (BK) surrogates: the first one is used to compute the required statistics of the objective functions in the uncertain parameter space, while the second one is used to model the response of these statistics to the design variables. Such an approach is called “combined kriging” [

53]. An expected improvement criterion is used to update the second kriging surrogate during convergence towards the optimum. This technique has been successfully applied to the design of turbine blades for organic Rankine cycles [

9] and to the RDO of the thermodynamic cycle [

54]. Assuming that each kriging surrogate requires a number of samples approximately equal to 10 times the cardinality of the parameter space to build a reasonably accurate approximation [

55], the nested BK RDO strategy needs

$\mathcal{O}(100\times {n}_{unc}\times {n}_{des})$ function evaluations (with

${n}_{unc}$ the number of uncertain parameters and

${n}_{des}$ the number of design variables) in the first generation of the GA to generate the initial kriging surrogates for the statistical moments. Additional

$\mathcal{O}(10\times {n}_{unc})$ evaluations are required for each update of the external kriging surrogate. Such a number of function calls is still too expensive for industrial CFD problems, even for uncertain or design spaces of low to moderate dimensionality (up to about eight uncertain or design parameters). GEK surrogates can be used to reduce the number of samples for the UQ step, but GEK-based MOGA is not straightforward in the context of RDO problems, since it requires also the gradient of the statistical moments of the QoI’s pdf with respect to the design variables. Obtaining such a piece of information by using efficient adjoint methods is not a trivial task; on the other hand, finite difference approximations are easily applicable, but at the price of a considerable computational expense for high-dimensional design spaces.

This is why we propose in this work a new multi-fidelity strategy for RDO that drastically reduces the required number of function calls by leveraging an inexpensive (but low-accuracy) first-order method of moments (MoM) and a Bayesian GEK [

56]. Using gradient-based solvers allows reducing per se the number of function solves in the UQ step and to mitigate the curse of dimensionality. Here, the two models are fused together by using a methodology similar to [

57] to generate a surrogate model for the MOGA optimization that combines the efficiency of MoM and the accuracy of GEK. The surrogate is enriched based on the expected improvement criterion during MOGA convergence, as in [

9]. The required gradient information is obtained by using either continuous or discrete adjoint formulations. The new MF-RDO strategy is applied to an inexpensive test problem, i.e., the stochastic inverse design of a supersonic quasi-1D nozzle. The results show that a few GEK UQ solves are sufficient to correct the MoM solution, thus reducing the computational cost of the RDO by approximately one order of magnitude with respect to the nested BK strategy. Computational gains are expected to be even more substantial for costly industrial CFD problems.

The paper is organized as follows. In

Section 2, we present the RDO problem and the test problem. The UQ methods considered in the study are described in

Section 3.

Section 4 presents the surrogate-based and multi-fidelity RDO strategies. In

Section 5, we first apply various UQ methods to the test configuration and compare their accuracy and computational costs; afterwards, such methods are combined with a SMOGA or MF-SMOGA, and their efficiency in solving the RDO problem is assessed. Conclusions and final remarks are reported in

Section 6.

## 2. Problem Definition

Following Taguchi’s RDO method, we look for a methodology allowing optimizing a set of QoIs,

$\mathit{J}=\mathit{J}(\mathit{x},\xi )$,

$\mathit{J}\in {\mathbb{R}}^{m}$ depending on a vector of deterministic design parameters

$\mathit{x}\in {\mathbb{R}}^{{n}_{des}}$ and on a vector of uncertain parameters

$\xi \in {\mathbb{R}}^{{n}_{unc}}$. Note that some of the design parameters may also be uncertain. We formulate the RDO problem by using the expectancy and the variance of

J as measures of robustness, which leads to the solution of the two objective deterministic optimization problem in Equation (

1).

The preceding optimization problem is solved by means of an MOGA. More precisely, following our previous studies [

10,

12], we adopt the non-dominated sorting genetic algorithm (NSGA-II) of Deb et al. [

58], which provides an approximated Pareto front of optimal solutions corresponding to different trade-offs between average performance and robustness for the various QoIs at hand. For simplicity, in the following, we consider only the case of a single QoI,

$m=1$, but the approach can be extended to multiple QoIs. The required statistics of the QoIs are calculated by means of a non-intrusive UQ method, which, for CFD models, is coupled with a suitable (costly) flow solver. Thus, the first ingredient of the RDO process is an efficient UQ approach, which provides accurate approximations of

$E\left[\mathit{J}\right]$ and

$var\left[\mathit{J}\right]$ based on a set of

N deterministic samples of the solution. The UQ methods investigated in this work are described in

Section 3.

Direct coupling of the MOGA with the UQ solver is overly costly for CFD, due to the high number of function evaluations. For instance, running the MOGA with a population of

${n}_{pop}$ individuals over

${n}_{gen}$ generations and using

N samples for UQ lead to an overall number of CFD evaluations of the QoI of about

$N\times {n}_{pop}\times {n}_{gen}$. The computational cost can be greatly alleviated by running

$N\times {n}_{pop}$ deterministic runs in parallel at each NSGA generation [

8,

25], but: (i) the required number of computational cores may exceed the computational resources available, and (ii) even with a perfect parallel scaling at each generation, the turn-around time of the RDO equals at least the average cost of a CFD run multiplied by

${n}_{gen}$. To reduce the computational cost, a second (external) surrogate model is introduced to predict the response of the cost functions to the design parameters, as described in

Section 4.

#### 2.1. Test Problem: Quasi-1D Supersonic Nozzle

Various RDO strategies are assessed against an inexpensive test problem (also studied in [

10,

17]), namely the inverse design of a supersonic quasi-1D diverging nozzle. This allows validating UQ methods against MC sampling.

The nozzle geometry is assigned through the area distribution

$S\left(x\right)$ along the longitudinal axis

x. This is chosen to be of the form:

with

a,

b,

c, and

d coefficients defining the geometry. The nozzle length is set to

$L=10$. A typical nozzle geometry is depicted in

Figure 1.

For this test case, the QoI

J (Equation (

3)) is a scalar, namely the mean quadratic error of the actual pressure distribution in the nozzle

$P\left(x\right)$ with respect to the target design pressure distribution

${P}_{des}\left(x\right)$. The latter corresponds to the pressure distribution for a nozzle geometry with

$a=1.75$,

$b=0.699$,

$c=1.00$, and

$d=3.80$, a reservoir pressure

${P}_{T,in}=1$ bar, an outlet static pressure

${P}_{out}=0.6$ bar, and a gas specific heat capacity ratio

$\gamma =1.4$.

The optimization goal is to determine the design parameters

$a,\phantom{\rule{3.33333pt}{0ex}}b,\phantom{\rule{3.33333pt}{0ex}}c,\phantom{\rule{3.33333pt}{0ex}}d$ in Equation (

2) providing the best fit to the target pressure distribution under multiple uncertainties, in the sense of Equation (

1).

The flow is assumed to be governed by the steady Euler equations for quasi-1D flows (Equation (

4)):

where

$w={[\rho ,\rho v,\rho {e}_{t}]}^{T}$ and

$f\left(w\right)={[\rho v,\rho {v}^{2}+P,\rho v{h}_{t}]}^{T}$ are, respectively, the conservative variable and the physical flux vectors [

59]. In the preceding equations,

$\rho $ is the fluid density,

v is the velocity along the nozzle axis,

${e}_{t}$ and

${h}_{t}$ are the total specific energy and enthalpy, and

$K={[0,-P,0]}^{T}$. The system of equations is supplemented by the equation of state for thermally and calorically perfect gases,

$P=(\gamma -1)\rho ({e}_{T}-{v}^{2}/2)$. The governing equations are discretized by a cell-centered finite volume formulation, using Rusanov’s first-order upwind scheme for space integration. The steady state solution is computed iteratively by solving a false transient with four-stage explicit Runge–Kutta time-stepping [

59]. Characteristic boundary conditions based on Riemann invariants are imposed at the nozzle inlet and outlet. Sonic flow conditions are prescribed at the inlet, so that all Riemann invariants enter the domain. The range of variation of the total pressure is such that a shock is always created in the divergent nozzle. As a consequence, outlet flow conditions are always subsonic. In this case, we impose the outlet static pressure, which is treated as deterministic, and fixed to

${P}_{out}=0.6$ bar. Based on a preliminary mesh study, a computational grid of 300 uniformly spaced cells is used in all of the following calculations.

The system is assumed to be subject to uncertainties of various nature, specifically:

geometric tolerances on the nozzle shape, modeled by treating the shape parameters $a,\phantom{\rule{3.33333pt}{0ex}}b,\phantom{\rule{3.33333pt}{0ex}}c,\phantom{\rule{3.33333pt}{0ex}}d$ as normally distributed random variables, with mean $\mu $ and coefficient of variation $CoV=\sigma /\mu $, with $\sigma $ the standard deviation;

uncertainties in inlet total pressure ${P}_{T,in}$ described as a uniformly-distributed random variable with imposed lower and upper bounds;

uncertainties in the gas properties, here represented by the specific heat ratio $\gamma $, which is also assumed as uniformly distributed.

The characteristics of the random parameters are listed in

Table 1. In the inverse design process, the uncertain geometric parameters are also (uncertain) design variables: for this reason, their mean is not fixed, but varies within the ranges corresponding to the bounds of the design space. This means that, even for designs corresponding to the upper/lower bounds, a realization of the nozzle geometry may lie outside the prescribed limits, due to geometric tolerances.

## 6. Conclusions

In the present work, we first assessed various gradient-based methods for uncertainty quantifications, in view of their use for the robust design optimization (RDO) of CFD problems. Specifically, the capability of a Bayesian gradient-enhanced kriging surrogate model and a first-order method of moments to accurately and efficiently compute the lower order statistical moments of a quantity of interest was evaluated for an inexpensive test problem, representative of a supersonic divergent nozzle, for which the results can be compared with well-converged Monte Carlo sampling. The results show that GEK allows computational gains of a factor of two or more with respect to a Bayesian kriging surrogate not using gradient information, when the gradients are efficiently computed using an adjoint method. In the present work, both a discrete and a continuous adjoint method were used for building GEK surrogates. The first one provides more accurate results, but it is somewhat more costly and requires intrusive automatic derivation of the CFD code, which is not possible if, for instance, a commercially available CFD solver is to be employed. The continuous adjoint method allows developing a non-intrusive adjoint solver, but it is less accurate due to inconsistencies in the numerical treatment of the direct and adjoint equations. In the present implementation, the continuous adjoint solver benefits from a direct solution of the linear system of adjoint equations, and it is therefore quicker than the discrete adjoint solver, which converges by means of a pseudo-transient technique. The first-order moment method, based on either discrete or continuous adjoint gradients, is less accurate than GEK, but it still provides satisfactory estimates of the QoI for the present shocked flow problem, due to the relatively small variation ranges of the uncertain parameters.

The UQ methods are then combined with a genetic algorithm for solving the RDO problem. The computations are sped up by constructing a Bayesian kriging surrogate model of the design space. The surrogate is enriched during GA iterations by means of a multi-objective expected improvement (MOEI) infill criterion. For the test problem at hand, the RDO results are found to be similar for the BK, GEK, and MoM methods, the latter being much less expensive in terms of CPU time, but slightly less accurate than the former ones. In order to benefit from the computational efficiency of the MoM and the accuracy of the GEK UQ solvers at the same time, a multi-fidelity surrogate model is built by fusing together the low-fidelity MoM and the high-fidelity GEK. An MOEI infill is used again to enrich the surrogate during convergence, with preference for low-fidelity infills. The multi-fidelity approach successfully identifies the RDO optimum, while dividing by a factor of $3\xf74$ the computational cost with respect to GEK. Such an approach is then identified as a promising candidate for more complex RDO problems using CFD models. Further work is however required to assess its effectiveness for more realistic and complex CFD problems. For this aim, its application to the RDO of the stator row of an organic Rankine cycle turbine is underway and will be reported in the near future. In such a context, the introduction of multi-fidelity UQ solvers combining BK and/or GEK based on different numbers of samples could be an interesting future development for further speed up of the RDO procedure.