Gradient-Based, Post-Optimality Sensitivity Analysis with Respect to Parameters of State Equations

Abstract

Design optimization is a computational tool that can enable a designer to investigate the effectiveness of a design concept in an organized format. However, this design process requires the design variables, constraints, and objective function to be properly defined and expressed in mathematical forms. Post-optimality analysis thus becomes a necessary step to investigate different variations in the problem formulation and parameters to ensure that optimization produces a stable and trustworthy outcome. One efficient way to achieve this aim is to compute the local derivative of the optimized objective function with respect to the optimization problem parameters, such as bounds on the constraints and the material properties in the state equation. This method is referred to as post-optimality sensitivity analysis. In this study, we derived the post-optimal sensitivity equation to explicitly include the derivatives of state variables with respect to problem parameters and to broaden its applications to minimax and goal attainment design optimization problems.

1. Introduction

The responsibility of an engineering designer is primarily to produce a quality product that can meet performance requirements with high confidence. This design task could form a lengthy process that may require steps such as data collection, trial and error, and validation. This task could be improved, however, by taking advantage of computer technology and numerical methods for modeling, analysis, and design optimization. To achieve this aim, post-optimality analysis becomes necessary to ensure that the related design optimization problems are effectively formulated, which includes the selection of design variables, the criteria to measure product quality, and the constraints on product performance.

One specific task within post-optimality analysis is to investigate the impact of the values of problem parameters upon the results of design optimization. This can be achieved with the aid of post-optimality sensitivity analysis, which involves investigating variation in the optimization solution due to changes in problem parameters by conducting direct differentiation of the optimized objective function with respect to the parameters of concern [1].

The focus of post-optimality analysis differs from that of post-optimality sensitivity analysis investigated herein. The former places greater emphasis on the quality of the design optimization formulation and its outcome. In particular, post-optimality analysis, as stated by Gero [2], benefits the stability analysis of the optimal solution, the impact of different design decisions, and the weighing of different objective functions. Venkat et al. [3] used the Greedy Reduction algorithm to select the most preferable subset of the Pareto optimal points. Arias et al. [4] conducted post-optimality analysis to compare the operational costs of two different ${\mathrm{C}\mathrm{O}}_2$ capture plants. Cherif [5] introduced a new behavior penalty method to conduct post-optimality analysis to improve the efficiency of goal programming. Moreover, Wang et al. [6] conducted post-optimality analysis in a multi-objective design optimization problem to measure the stability of an optimal solution under variations in the design and environmental variables of wind turbines, based on the distances between Pareto points.

Post-optimality sensitivity analysis, in comparison, involves analytically producing the design derivatives of the optimization results with respect to problem parameters, which is achieved once the optimization formulation is fully specified and solved. These derivatives are conducted in the domain of the optimized design variables. Specifically, the optimized design variables that involve those specific derivatives must satisfy the Kuhn–Tucker necessary conditions. Vanderplaats and Yoshida [1] classified post-optimality sensitivity analysis methods into three major approaches: the optimality objective-based approach, the feasible search direction approach, and the necessary condition-based approach. Both Approaches 1 and 3 are based on the Kuhn–Tucker necessary conditions. However, the first approach only focuses on the derivative of the optimal objective function, whereas the latter approach includes the derivative of the objective function, the constraints, and the derivatives of the optimal design variables and the Lagrange multipliers. Numerous authors have discussed some or all of these approaches in their published engineering optimization textbooks. Onwubiko [7] presented only the first approach in his book; Haug detailed the first and third approach for post-optimality sensitivity analysis [8], as was also the case for Belegundu and Chandrupatla [9]. Conversely, Rao [10] presented all three approaches.

Fiacco and Ghaemi [11] derived the post-optimality sensitivity analysis of a nonlinear structure design problem. The bounds of constraints and the structural properties involved in the state equation are considered as problem parameters. However, the state variables are not explicitly included in the post-optimality sensitivity equation. Barthelemy and Sobieszczanski-Sobieski [12] presented the optimum sensitivity derivatives of the objective function in nonlinear programming. Diewert [13] used the first approach to derive the first-order and second-order post-optimality sensitivity analysis of a nonlinear programming problem related to economics that does not involve engineering state variables. In a similar manner, Braun and Kroo [14] used Approach 1 to investigate the sensitivity of an optimized direct operating cost of a DC-9 class airspace vehicle with respect to parameters including cruise range, takeoff field length, etc. Hart and van Bloemen Waanders [15] improved the data collected from a limited number of optimization solutions of a high-fidelity model by taking advantage of the post-optimality sensitivities of the optimized low-fidelity models with respect to model discrepancy. Enevoldsen [16] also formulated the design optimization problem into two levels. The post-optimal sensitivity analysis of the upper-level optimization problem requires the sensitivity of the reliability index presented in its constraint with respect to the design variables. Post-optimality sensitivity analysis is also widely used in reliability index-based design optimization, which is structured as a two-level optimization [17,18]. The first-order reliability method (FORM) is applied at the first level. Its objective is the reliability index of a response function, and its design variables are the random variables. The post-optimality sensitivity analysis of the reliability index is then computed with respect to the design variables of the upper level, which are treated as the problem parameters in the low-level FORM.

Jereza and colleagues [19] used the feasible search direction method, Approach 2, for optimum design sensitivity of failure probabilities with respect to filter parameters of a concrete structure modeled by linear dynamic equations with random loads. Baldomir et al. [20] also used Approach 2 to update the optimal design variables to more effectively match the targeted values of the constraints. This process is often used in the tradeoff design process [21].

Koltai and Tatay [22] applied Approach 3 in linear programming to avoid degeneracy of optimal solutions with the ranges of the coefficients of the objective and the RHS of constraints. This optimality sensitivity analysis is achieved with finite differencing of repeated optimization solutions with respect to different problem parameters. Vakilifard et al. [23] analytically derived the derivatives of the optimal objective in their paper with respect to problem parameters in the constraint equations. The derivatives of the optimal objectives for the applications presented in the paper are equal to the Lagrange multipliers of the associated constraints.

Bonnans and Shapiro [24] mathematically provided the necessary and sufficient conditions to ensure stable first- and second-order optimal sensitivity analyses of general programming problems based on Approach 3. The problem parameters in the example presented in their paper are the bounds of the constraints. Similarly, Pimay et al. [25] differentiated the Kuhn–Tucker necessary condition twice with respect to problem parameters to produce the second-order derivatives of the optimized design variables and Lagrange multipliers. Sobieskzczanski-Sobicski and colleagues [26] applied Approach 3 of optimal sensitivity analysis to decouple a large optimization problem into multiple levels to improve computational efficiency. The upper-level design variables are treated as problem parameters in a lower-level design optimization problem. The optimal sensitivities of the objective function and the design variables of a lower-level problem with respect to these problem parameters can be employed to approximate the lower-level objective function linearly in terms of the upper-level design variables presented in the upper-level design optimization formulation. This process improves computational efficiency by eliminating the lower-level design variables from the upper-level formulation.

Approach 1, as discussed above, is the most efficient method in terms of the number and degree of derivatives among the three for post-optimality sensitivity analysis. However, it is often limited to the objective function expressed in terms of problem parameters and design variables. To broaden engineering applications of Approach 1, we explicitly aim to take into account of state equations in the post-optimality sensitivity analysis. The newly derived equation is then further extended to common multi-objective problems, minimax and goal attainment. We also methodically review the search direction method and the most comprehensive approach, Approach 3. The remainder of the paper is organized as follows: In Section 2, Materials and Methods, we will review all three approaches. The focus in this case is placed on Approach 1. The detailed derivation of the post-optimality sensitivity analysis equation for a single objective problem is first presented, whereby the problem parameter is presented as part of the state equation. The same post-optimality sensitivity equation is then revised to handle minimax and goal attainment problems. In Section 3, Validation and Application, we will first use an illustrative example to validate the derived equations in Approaches 1 and 2 and demonstrate their applications. The second example is a design problem of a finned heat sink for a circuit board cooling system to minimize its weight while maximizing the heat loss rate and hydraulic diameter. The heat sink is modeled using a 2D finite element heat transfer problem. Research results and future research directions are summarized in the final section, Section 4, Conclusion: Remarks and Discussion.

2. Methodology

All three approaches for the post-optimality sensitivity analysis are reviewed in this section. The layout of this section is divided into four subsections. The most efficient approach, Approach 1, is presented in Section 2.1 and Section 2.2. In the first subsection, we focus on a standard optimization method with one single-objective function; in the second subsection, we address the minimax and goal attainment problems, which are multi-objective. The related post-optimality sensitivity equation of a single objective function with respect to a problem parameter is derived in Section 2.1, which takes state variables into consideration in the formulation.

The post-optimality sensitivity analysis of the minimax and goal attainment problems is discussed in Section 2.2. The multi-objective functions of these problems are converted into single-objective functions. These recast problems treat the new objective function as an additional design variable, which is also presented as part of the constraints. This uniqueness produces a post-optimal sensitivity equation differing from the one derived from the single-objective optimization problem presented in Section 2.1.

In the third subsection, we will review the search direction method in which the search direction is formulated and computed based on the quadratic programming problem rather than the linear programming problem presented in Reference [1]. The direct search method is suitable for tradeoff design but not for post-optimality sensitivity analysis, as it does not involve the Kuhn–Tucker necessary solution. The most comprehensive method for post-optimality sensitivity analysis is Approach 3, which will be presented in Section 2.4. The method will directly differentiate the Kuhn–Tucker necessary condition to construct the equations to solve the design derivatives of the optimal design variables and the Lagrange multipliers. The derived equation involves second-order derivatives which are characterized by computational complexity.

2.1. Approach 1: Direct Differentiation of the Augmented Objective Function

Setting ϕ, x, b, and p, respectively, as the cost function, the state variable vector, the design variable vector, and a problem parameter, a single-objective design optimization problem is expressed mathematically as follows:

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^k\end{array}& \varphi \left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)\end{array},$$

(1)

subjected to the constraints,

$$\begin{array}{c}\mathit{h}\left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)=\mathbf{0}\\ \mathit{g}\left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)\le \mathbf{0}\end{array}\},$$

(2)

and the state equation,

$$\mathit{Q}\left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)=\mathbf{0}$$

(3)

where the state variable vector, $\mathit{x}\in R^k$, is the solution of the state equation, $\mathit{Q}\in R^k$, expressed in Equation (3), which itself is a function of the design variable vector and the problem parameters. As a result, the state variable vector, $\mathit{x}\left(\mathit{b},p\right),$ is also a function of the design variables, $\mathit{b}$, and the problem parameter, $p$.

Assume that all functions in Equations (1)–(3) meet the sufficient continuity requirements and, for a given problem parameter, $p_0$, ${\mathit{b}}^{\mathit{*}}\left(p_0\right)$ is an isolated minimum of the above design problem, which satisfies the Kuhn–Tucker necessary condition at the optimal design, ${\mathit{b}}^{\mathit{*}}$,

$${\displaystyle \frac{\partial {\varphi }^{*}}{\partial \mathit{b}}}+{\displaystyle \frac{\partial {\varphi }^{*}}{\partial \mathit{x}}}{\displaystyle \frac{\partial \mathit{x}}{\partial \mathit{b}}}+{\mathit{\lambda }}^T\left({\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{b}}}+{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{x}}}{\displaystyle \frac{\partial \mathit{x}}{\partial \mathit{b}}}\right)=0.$$

(4)

The optimal objective function, ${\varphi }^{*}(\mathit{x}({\mathit{b}}^{\mathit{*}},p_0),{\mathit{b}}^{\mathit{*}},p_0)$ is expanded as

$${\varphi }^{*}(\mathit{x}({\mathit{b}}^{\mathit{*}},p_0),{\mathit{b}}^{\mathit{*}},p_0)={\varphi }^{*}+{\mathit{\lambda }}^T{\mathit{g}}^{*}$$

where ${\mathit{g}}^{*}$ represents all tight constraints defined by Equation (2), evaluated at ${\mathit{b}}^{\mathit{*}}$ and $p_0$. The vector, $\mathit{\lambda }$, is also a collection of all associated Lagrange multipliers with $\mathit{\lambda }\ge 0$. Furthermore, by taking advantage of the Kuhn–Tucker necessary condition, the derivative of the optimal objective function $\varphi (\mathit{x}({\mathit{b}}^{\mathit{*}},p_0),{\mathit{b}}^{\mathit{*}},p_0)$ with respect to the problem parameter, $p$, at $p_0$ can now be derived as

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}={\displaystyle \frac{\partial {\varphi }^{*}}{\partial p}}+{\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}+\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial \mathit{x}}}+{\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{x}}}\right){\displaystyle \frac{\partial x}{\partial p}}$$

(5)

where $\partial {\mathit{g}}^{\mathit{*}}/\partial p$ is an $m\times 1$ column vector, $\partial {\varphi }^{\mathit{*}}/\partial \mathit{x}$ is a $1\times n$ row vector, and $\partial {\mathit{g}}^{\mathit{*}}/\partial \mathit{x}$ is an $m\times n$ Jacobian matrix. Note that an asterisk marks the function evaluated at the optimal design. By partially differentiating the state equation, Equation (3), evaluated at ${\mathit{b}}^{\mathit{*}}$ with respect to the problem parameter, $p$, one can construct the following equation solved for $\partial \mathit{x}/\partial p$ directly as

$$\left({\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial \mathit{x}}}\right){\displaystyle \frac{\partial \mathit{x}}{\partial p}}=-{\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial p}}.$$

(6)

Alternatively, the adjoint variable method can be employed to rederive Equation (4) as

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}={\displaystyle \frac{\partial {\varphi }^{*}}{\partial p}}+{\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}+{\mathit{\eta }}^T{\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial p}}$$

(7)

where the adjoint variable vector, $\mathit{\eta }$, is the solution of the associated adjoint equation,

$${\left({\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial \mathit{x}}}\right)}^T\mathit{\eta }=-\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial \mathit{x}}}+{\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{x}}}\right).$$

(8)

The advantage of the adjoint variable method is that the same adjoint variable vector, $\mathit{\eta }$, in Equation (7) can be repeatedly used in Equation (6) for post-optimality sensitivity analysis with respect to different problem parameters.

The sensitivity equations derived by Approach 1 published in the literature usually include only the first two terms of Equation (4), but not those associated with state variables, $\partial \mathit{x}/\partial p$. Detailed derivation of Equations (5)–(8) can be found in the Appendix A.

2.2. Approach 1: Minimax and Goal Attainment

Some optimization problems with multi-objective functions can be revised to fit into a single-objective format. Specifically, the goal of minimax is, among all objective functions, to minimize the maximal one. Its mathematical format can be expressed as follows:

$$\begin{array}{cc}\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^n\end{array}& \begin{array}{c}\mathrm{m}\mathrm{a}\mathrm{x}\\ i=\mathrm{1,2},\dots m\end{array}\end{array}& {\{\varphi }_i\left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)\}\end{array}$$

(9)

subjected to the same constraints and the state equation stated in Equations (2) and (3). The objective function described in Equation (9) can be converted into a single-objective format by introducing an additional design variable, $Z$, as

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^n,Z\end{array}& Z\end{array}$$

(10)

subjected to the following $m$ constraints in addition to those presented in Equations (2) and (3),

$${g_{Z,i}\cong \varphi }_i\left(\mathit{x}\left(\mathit{b},p\right),\mathit{b},p\right)-Z\le 0, i=1,\dots ,m_Z.$$

(11)

Conversely, the goal attainment problem is defined with a single objective to minimize the gap between the optimized objective functions and its targeted values, ${\mathit{f}}_0$. The outcome is expected to be a design that can achieve the targeted goals, ${\mathit{f}}_0$. The problem can be described as follows:

$$\begin{array}{cc}\begin{array}{c}\mathbf{m}\mathbf{i}\mathbf{n}\\ \mathit{b},\gamma \end{array}& \gamma \end{array}$$

(12)

subjected to the following constraint set in addition to Equations (2) and (3),

$${\mathit{g}}_Z\cong \mathit{f}\left(\mathit{x},\mathit{b}\right)-\gamma \times {\mathit{w}}_t={\mathit{f}}_0.$$

(13)

The vector, ${\mathit{w}}_t$, is the pre-determined weighting vector, whereas the attainment factor, $\gamma$, is a magnitude to measure the gap between the objective functions and the targeted goals.

The objective functions presented in these two optimization formulations differ considerably from the one presented in Equations (1)–(3). First, the objective function in either Equation (10) or (12) is formulated as a design variable; secondly, it is also presented as part of the constraints. Therefore, the post-optimal sensitivity equation presented in Equation (5) is now revised as

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}={\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}+{\mathit{\lambda }}_Z^T{\displaystyle \frac{\partial {\mathit{g}}_Z^{*}}{\partial p}}+\left({\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{x}}}+{\mathit{\lambda }}_Z^T{\displaystyle \frac{\partial {\mathit{g}}_Z^{*}}{\partial \mathit{x}}}\right){\displaystyle \frac{\partial \mathit{x}}{\partial p}}$$

(14)

where the Lagrange multiplier, $\mathit{\lambda },$ is associated with the traditional format of constraints as described in Equations (2) and (3), and ${\mathit{\lambda }}_{\mathit{Z}}$ is the Lagrange multiplier associated with Equation (11) for the minimax problem and Equation (13) for the goal attainment one. The above equation can also be reformulated by introducing the adjoint variable, $\mathit{\eta }$, as

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}={\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}+{\mathit{\lambda }}_Z^T{\displaystyle \frac{\partial {\mathit{g}}_Z^{*}}{\partial p}}-\mathit{\eta }{\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial p}}$$

with the adjoint equation,

$${\left({\displaystyle \frac{\partial {\mathit{Q}}^{\mathit{*}}}{\partial \mathit{x}}}\right)}^T\mathit{\eta }=-\left({\mathit{\lambda }}^T{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{x}}}+{\mathit{\lambda }}_Z^T{\displaystyle \frac{\partial {\mathit{g}}_Z^{*}}{\partial \mathit{x}}}\right)$$

where the vector ${\mathit{Q}}^{\mathit{*}}$ is the state equation of Equation (3) evaluated at the optimal design point; i.e., ${\mathit{Q}}^{\mathit{*}}=0=\mathit{Q}(\mathit{x}({\mathit{b}}^{\mathit{*}},p),{\mathit{b}}^{\mathit{*}},p)$.

2.3. Approach 2: The Feasible Search Direction Method

The aim of the feasible search direction approach, proposed by Vanderplaats and Yoshida [1], is to find the best way to improve the existing optimal design, ${\mathit{b}}^{\mathrm{*}},$ without violating the tight constraints, ${\mathit{g}}^{\mathrm{*}}$. This is achieved by expanding the design space to include the problem parameter, $p$, as a new design variable. Consequently, both $\mathit{b}$ and $p$ are now treated as independent design variables.

Using the existing optimal design, ${\mathit{b}}^{\mathrm{*}}$, as the starting point, a new design along the steep descent direction is then found in the expanded design space by solving a linear programming problem [1], which maximizes the reduction in the objective function while not violating constraints. In the study, however, the search direction, $\mathit{s}=\left({\mathit{s}}_{\mathit{b}},s_p\right)$, is found in the expanded design space by solving a quadratic programming problem as stated below:

$$\begin{gathered} \begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ s_p,{\mathit{s}}_{\mathit{b}}\in R^n\end{array}& ∆{\varphi }^{*}+{\displaystyle \frac{1}{2}}{\mathit{s}}^T\mathit{s}=\left({{\displaystyle \frac{\partial \varphi }{\partial \mathit{b}}}}^{*T}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\mathit{s}}_{\mathit{b}}\end{array}+\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial p}}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{dp}}\right)s_p \\ +{\displaystyle \frac{1}{2}}\left({\mathit{s}}_{\mathit{b}}^T{\mathit{s}}_{\mathit{b}}+s_p^2\right) \end{gathered}$$

(15)

subject to

$$\left({{\displaystyle \frac{\partial \mathit{g}}{\partial \mathit{b}}}}^{*}+{{\displaystyle \frac{\partial \mathit{g}}{\partial \mathit{x}}}}^{*}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\mathit{s}}_{\mathit{b}}+\left({\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}+{{\displaystyle \frac{\partial \mathit{g}}{\partial \mathit{x}}}}^{*}{\displaystyle \frac{d\mathit{x}}{dp}}\right)s_p+{\mathit{g}}^{\mathit{*}}=0.$$

(16)

This feasible search direction resulting from the above equation has been widely used in gradient-based design optimization and in tradeoff design [21]. Its solution, $\mathit{s},$ comprises two parts. The first part, ${\mathit{s}}_1$, aims to reduce the objective function while maintaining the same constraint values. The second one, ${\mathit{s}}_2$, is mainly responsible for correcting constraint violations. As in Reference [1], in this study, we will focus on the first part of the solution, ${\mathit{s}}_{\mathbf{1}}=\left({\mathit{s}}_{\mathbf{1}\mathit{b}},s_{1p}\right)$, which is computed by

$${\mathit{s}}_1=-P\nabla {\varphi }^{*}$$

(17)

where the matrix, $P$, is computed by

$$P=I-(\nabla {\mathit{g}}^{\mathit{*}}){[{(\nabla {\mathit{g}}^{\mathit{*}})}^T(\nabla {\mathit{g}}^{\mathit{*}})]}^{-1}{(\nabla {\mathit{g}}^{\mathit{*}})}^T.$$

Note that both gradients, $\nabla f^{*}$ and $\nabla {\mathit{g}}^{\mathit{*}}$, are computed in the expanded design space.

Once ${\mathit{s}}_{\mathbf{1}}$ becomes available, one can compute the reduction in the objective function at the current optimal design, ${\mathit{b}}^{\mathit{*}},$ in the most efficient way as

$$∆{\varphi }^{\mathrm{*}}=\nabla {\varphi }^{\mathrm{*}}{\mathit{s}}_{\mathbf{1}}=\left({{\displaystyle \frac{\partial \varphi }{\partial \mathit{b}}}}^{\mathrm{*}T}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{\mathrm{*}T}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\mathit{s}}_{\mathbf{1}\mathit{b}}+\left({\displaystyle \frac{\partial {\varphi }^{\mathrm{*}}}{\partial p}}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{\mathrm{*}T}{\displaystyle \frac{d\mathit{x}}{dp}}\right)s_{1p}.$$

Furthermore, one can divide the above equation by $s_{1p}$ to obtain the below equation:

$${\displaystyle \frac{∆\varphi }{s_{1p}}}=\left({{\displaystyle \frac{\partial \varphi }{\partial \mathit{b}}}}^{*T}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\displaystyle \frac{{\mathit{s}}_{\mathbf{1}\mathit{b}}}{s_{1p}}}+\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial p}}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{dp}}\right).$$

(18)

Next, setting the limit of $s_{1p}$ approaching zero, the above equation is revised as the following one, which can be used to estimate the change in the objective function due to the changes in the design variables and the problem parameter as

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}=\left({{\displaystyle \frac{\partial \varphi }{\partial \mathit{b}}}}^{*T}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\displaystyle \frac{d{\mathit{b}}^{\mathit{*}}}{dp}}+\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial p}}+{{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}}^{*T}{\displaystyle \frac{d\mathit{x}}{dp}}\right)$$

(19)

where $d{\mathit{b}}^{\mathrm{*}}/dp$ is the limit of ${\mathit{s}}_{\mathbf{1}\mathit{b}}/s_{1p}$, and $d{\varphi }^{\mathrm{*}}/dp$ is the derivative of the objective function with respect to $p.$

2.4. Approach 3: Direct Differentiation of the Kuhn–Tucker Necessary Condition

The most comprehensive approach to conducting post-optimality sensitivity analysis is to directly differentiate the Kuhn–Tucker necessary conditions, Equation (4), and the tight constraints, ${\mathit{g}}^{\mathrm{*}}=\mathbf{0}$, with respect to the problem parameter, $p$, at the optimal design point, ${\mathit{b}}^{\mathrm{*}}.$ This differentiation builds an equation that can be solved for $d{\mathit{b}}^{\mathrm{*}}/dp$ and $d\mathit{\lambda }/dp$ together, which are the targeted outcome of Approach 3 [24,25,26].

$$\left[\begin{array}{cc}{\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial {\mathit{b}}^{\mathbf{2}}}}+{\mathit{\lambda }}^T{\displaystyle \frac{{\partial }^2{\mathit{g}}^{\mathit{*}}}{\partial {\mathit{b}}^{\mathbf{2}}}}& {\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial \mathit{b}}}\\ {\displaystyle \frac{\partial {{\mathit{g}}^{\mathit{*}}}^{\mathit{T}}}{\partial \mathit{b}}}& 0\end{array}\right]\left\{\begin{array}{c}{\displaystyle \frac{d{\mathit{b}}^{\mathit{*}}}{dp}}\\ {\displaystyle \frac{d\mathit{\lambda }}{dp}}\end{array}\right\}=\left\{\begin{array}{c}-\left({\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial \mathit{b}\partial p}}+{\mathit{\lambda }}^T{\displaystyle \frac{{\partial }^2{\mathit{g}}^{\mathit{*}}}{\partial \mathit{b}\partial p}}\right)\\ -{\displaystyle \frac{\partial {\mathit{g}}^{\mathit{*}}}{\partial p}}\end{array}\right\}.$$

(20)

Once the problem parameter, p, becomes a part of the state equation, differentiation of the Kuhn–Tucker necessary condition with respect to p can be complicated, as it involves second-order derivatives. However, once the state variable, $\mathit{x}$, is engaged in the problem formulation, the coefficient matrix and the loading terms of the assembled equation, Equation (20), will be a collection of various second-order derivatives. For example, the differentiations of the first and second terms of Equation (4) lead to

$${\displaystyle \frac{d}{dp}}{\left({\displaystyle \frac{\partial {\varphi }^{*}}{\partial \mathit{b}}}\right)}^T={\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial \mathit{b}\partial p}}+\left[{\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial {\mathit{b}}^{\mathbf{2}}}}+{\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial \mathit{b}\partial \mathit{x}}}{\displaystyle \frac{\partial \mathit{x}}{\partial \mathit{b}}}\right]\left({\displaystyle \frac{d{\mathit{b}}^{\mathit{*}}}{dp}}\right)+{\displaystyle \frac{{\partial }^2{\varphi }^{*}}{\partial \mathit{b}\partial \mathit{x}}}\left({\displaystyle \frac{\partial \mathit{x}}{\partial p}}\right).$$

(21)

and

$${\displaystyle \frac{d}{dp}}{\left({\displaystyle \frac{\partial \mathit{x}}{\partial \mathit{b}}}\right)}^T={\displaystyle \frac{{\partial }^2\mathit{x}}{\partial \mathit{b}\partial p}}+{\displaystyle \frac{{\partial }^2\mathit{x}}{\partial {\mathit{b}}^{\mathbf{2}}}}{\displaystyle \frac{d{\mathit{b}}^{\mathit{*}}}{dp}}.$$

(22)

which include ${\partial }^2\mathit{x}/\partial \mathit{b}\partial p$ and ${\partial }^2\mathit{x}/\partial {\mathit{b}}^{\mathbf{2}}$. The requirement for these second-order derivative terms will increase the computational complexity of applying Approach 3 for general engineering applications.

The following summary explains the common procedure for implementing the post-optimality sensitivity analysis. Based on the equations derived in Section 2, one can conduct post-optimality sensitivity analysis of the optimization design problems with respect to $p$ at $p_0$. First, the optimized design variables, ${\mathit{b}}^{\mathit{*}}$, and the associated Lagrange multipliers, $\mathit{\lambda }$, are solved at the specified value of the problem parameter, $p_0$. The state variables, $\mathit{x}$, must also be available at ${\mathit{b}}^{\mathit{*}}$ and $p_0$. While MATLAB R2023b was used in the examples in this paper, there are many other computational software applications, such as Mathematica, that could be used effectively to accomplish the post-optimality sensitivity analysis.

3. Validation Examples

Four validation examples are presented in this section. The first three examples concern illustrative state equations, and the fourth example is built on a finite element model of a 2D heat sink problem. The first example is set not only to validate the post-optimality sensitivity equations of Approach 1 derived in Section 2, but also to demonstrate the use of the derived equations to investigate the effects of the problem parameters on the optimal formulation and solution. The second example extends Approach 1 to solve a problem with a minimax objective in addition to goal attainment. Example 3 will apply Approach 2, the feasible search direction method, to resolve the problem investigated in the first example. The last example involves the use of Approach 1 to conduct post-optimality sensitivity analysis of a circuit board cooling problem involving a finned heat sink. This engineering application example is analyzed using a 2D finite element model. Its optimization design problem is formulated with multiple objectives.

3.1. Example 1: A Structural Problem

The example presented in this section is an academic example which covers broad applications of engineering problems. It has two state equations which model the static problem as well as the eigenvalue problem. Both state equations share the same stiffness matrix, which is a function of the problem parameter, $p$. Consequently, this example problem demonstrates the post-optimality sensitivity analysis of an optimization design problem with respect to problem parameters involved in the state equations. Furthermore, it provides an opportunity to form a multi-objective design optimization problem; one is associated with the static equation and the other with the eigenvalue problem.

The design problem of concern is formulated with four design variables, $b_1, b_2, b_3 \mathrm{a}\mathrm{n}\mathrm{d} b_4$; three state variables, $x_1, x_2 \mathrm{a}\mathrm{n}\mathrm{d} x_3$; and the fundamental eigenvalue, ${\mu }_1$. The optimization problem aims to minimize the work performed by the external force, $\mathit{f}$, subjected to three inequality constraints on deflection, stress, and the fundamental eigenvalue. The design optimization problem is mathematically cast as follows:

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^4\end{array}& \varphi ={\mathit{f}}^T\mathit{x}\left(\mathit{b},p\right)\end{array}$$

(23)

subject to the inequality constraints,

$$\begin{array}{c}{g}_1\left(\mathit{b},\mathit{x}\left(\mathit{b},p\right)\right)={-b}_1^2{x}_1+x_2^2\le 0\\ g_2\left(\mathit{b},\mathit{x}\left(\mathit{b},p\right)\right)={4b}_2\sin \left(x_3\right)-3x_1\le 0\\ g_3\left(\mathit{b},\mu \left(\mathit{b},p\right)\right)=0.8-4{\mu }_1\le 0\end{array}\},$$

(24)

and the bounds on the design variables,

$$0.1\le b_1, b_2,b_3, b_4\le 20$$

(25)

where $\mathit{x}={\left(\begin{array}{ccc}{x}_1& x_2& x_3\end{array}\right)}^{\mathit{T}},$ the state variables, and ${\mu }_1$, the fundamental eigenvalue, are the solutions of the following static matrix equations,

$$K\left(p\right)\mathit{x}-\mathit{f}=0=\left[\begin{array}{ccc}5b_1& -5b_1& 0\\ -5b_1& 5b_1+p{b}_2+5b_3& -5b_3\\ 0& -5b_3& 5b_3^2+5b_4^2\end{array}\right]\left\{\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right\}-\left\{\begin{array}{c}20\\ {1-b}_2\\ 15\end{array}\right\},$$

(26)

and the eigenvalue matrix equation,

$$\begin{gathered} \mathit{K}\left(p\right)\mathit{y}-\mu M\mathit{y}=0=\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em} \\ \left[\begin{array}{ccc}5b_1& -5b_1& 0\\ -5b_1& 5b_1+p{b}_2+5b_3& -5b_3\\ 0& -5b_3& 5b_3^2+5b_4^2\end{array}\right]\left\{\begin{array}{c}{y}_1\\ y_2\\ y_3\end{array}\right\}- \\ \mu \left[\begin{array}{ccc}{b}_1^2+5b_3& 0& 0\\ 0& 3b_2^2+b_4^2& 0\\ 0& 0& b_3^2+5b_1\end{array}\right]\left\{\begin{array}{c}{y}_1\\ y_2\\ y_3\end{array}\right\}. \end{gathered}$$

(27)

The problem parameter, $p$, appears in $K\left(\mathrm{2,2}\right)$ in both Equations (26) and (27). Its initial value, $p_0$, is set to be 10.

The design optimization run starts with initial design variables, ${\mathit{b}}_0^T=\left[1., 1., 1., 1.\right]$, which generates the initial objective,${\varphi }_0=163$, and encounters no constraint violations, with constraint values ${\mathit{g}}^{\mathit{T}}=\left[-1.2053,-16.7501,-1.3372\right]$. With user-provided gradients, the MATLAB built-in function, fmincon, reaches the optimal solution after 38 function evaluations. The objective function and the design variables at the optimal design solution are ${\varphi }^{*}=8.8881$ and ${\mathit{b}}^{*T}=\left[10.2718, 11.3447, 11.1417, 0.1001\right]$, combined with two Lagrange multipliers, ${\lambda }_2=0.0184$ and ${\lambda }_3=18.9916$, which correspond to the tight constraints, $g_2^{*}=g_3^{*}=0$. Note that the value of ${\lambda }_2$ is 10³ times smaller than that of ${\lambda }_3$. The design derivative of the optimality objective with respect to the parameter, $p$, is $-$0.6065. It is calculated at $p_0=10$ based on the following Equation (28), which was derived based on Equation (5) as

$${\displaystyle \frac{d{\varphi }^{\mathrm{*}}}{dp}}={\left\{\begin{array}{c}20-3{\lambda }_2\\ {1-b}_2\\ 15+{4{\lambda }_{2}b}_2\cos \left(x_3\right)\end{array}\right\}}^T{\displaystyle \frac{\partial \mathit{x}}{\partial p}}-4{\lambda }_3{\displaystyle \frac{\partial \mu }{\partial p}}.$$

(28)

Its accuracy can be validated with the results obtained via finite differencing:

$$\mathsf{\Delta }{\varphi }^{*}\cong {\displaystyle \frac{d{\varphi }^{*}}{dp}}∆p.$$

The above sensitivity equation can be used to estimate the change in the optimal objective function due to the change in parameter, $p$, as

$$\mathsf{\Delta }{\varphi }^{*}={\varphi }^{*}\left(p_0+∆p\right)-{\varphi }^{*}\left(p_0\right).$$

The results of such comparison are listed in

Table 1. Post-optimal sensitivity analysis of Example 1 at $p_0=10.0$.

Perturbation: ∆p/p	Initial Obj. ${\mathit{\varphi }}_0$	Optimal Obj. ${\mathit{\varphi }}^{\mathbf{*}}$	Active Constraints $\mathbf{at} {\mathit{b}}^{\mathbf{*}}$	Finite Diff. ∆ϕ*	Estimated (dϕ*/dp) ∆p
0.50/10.0	160.673	8.6001	$g_2,g_3$	−0.2880	−0.3031
0.25/10.0	161.813	8.7403	$g_2,g_3$	−0.1478	−0.1515
0.10/10.0	162.519	8.8280	$g_2,g_3$	−0.0601	−0.0606
0.00/10.0	163.000	8.8881	$g_2,g_3$	0.0	0.0
−0.10/10.0	163.487	8.9494	$g_2,g_3$	0.0613	0.0606
−0.25/10.0	164.234	9.0435	$g_3$	0.1554	0.1515
−0.50/10.0	165.520	9.2059	$g_3$	0.3178	0.3031

. This comparison is performed with a perturbation of the problem parameter, $p$, falling into a range of $\pm 5\%$ of $p_0$.

Note that Equation (5) of $d{\varphi }^{\mathrm{*}}/dp$ is derived based on the assumption that the involved constraints remain active due to the change in the value of $p$. However, the bottom two rows of Table 1 present cases where Equation (5) still achieves satisfactory results even though the constraint, $g_2$, is not active. For this example, it is noted that both ${\lambda }_2$ and its coefficient terms in Equation (5) are much smaller than the rest, which may explain why Equation (5) is still effective in estimating the change in the objective function due to the change in p, even though the constraint, $g_2$, is switched from active at the base design, $p_0=10,$ to non-active at the perturbed designs, $p=9.75$ and $p=9.5$. These two cases are presented in the last two rows of Table 1 with constraint values, $g_2=-0.0244$ and $g_2=-0.0717$, respectively.

To further clarify this matter, one may add an additional parameter, $\alpha$, to investigate the impact of the feasibility of the second constraint in Equation (2) as

$$g_2={4b}_2\sin \left(x_3\right)-3x_1+\alpha \le 0$$

to make $g_2$ active at the optimal design with a positive value of $\alpha$. For example, setting the values of parameters, $p=9.5$ and $\alpha =0.1$, the second and third constraints will remain active at the optimal design with an objective function, 9.206274, and the total perturbation of the optimal objective function can be computed as follows:

$$\mathsf{\Delta }{\varphi }^{*}\cong {\displaystyle \frac{d{\varphi }^{*}}{dp}}∆p+{\displaystyle \frac{d{\varphi }^{*}}{d\alpha }}∆\alpha$$

(29)

where $d{\varphi }^{*}/dp$ can be computed according to Equation (28), whereas $d{\varphi }^{*}/d\alpha$ is simply equal to ${\lambda }_2$ based on Equation (5). By setting the changes in the parameter, $∆p=-0.5$ and $∆\alpha =0.1$, one may estimate the change in the optimal objective function as

$$\mathsf{\Delta }{\varphi }^{*}\cong 0.3031+0.00184=0.30494$$

(30)

which is close to the result obtained via finite differencing between two optimization runs, 0.3182. The numerical results of Equation (30) demonstrate that whether constraint $g_2$ is active or not has little influence on the change in optimal objective functions, $\mathsf{\Delta }{\varphi }^{*}$.

3.2. Example 2: Minimax and Goal Attainment

The example problem presented herein comprises two objectives: the first objective is for the structure problem to reach the value of 10, whereas the second objective is to make the second eigenvalue seven times higher than the first one. Mathematically, the objective function can be expressed as, based on the required format for minimax optimization, Equations (12) and (13),

$$\begin{array}{cc}\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^4\end{array}& \begin{array}{c}\mathrm{m}\mathrm{a}\mathrm{x}\\ i=\mathrm{1,2}\end{array}\end{array}& \left\{\begin{array}{c}{\varphi }_1={\mathit{f}}^T\mathit{x}-10\\ {\varphi }_2={\displaystyle \frac{{\mu }_2}{{\mu }_1}}-7\end{array}\right\}\end{array}.$$

(31)

The constraints and the state equations are identical to those employed by the single-objective optimization problem as stated in Equations (24)–(27). By introducing a new design variable, Z, the multi-objective function in Equation (31) can now be converted into a single objective, $Z$, as

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^n,Z\end{array}& Z\left(\mathit{b}\left(p\right), p\right)\end{array}$$

(32)

together with two additional constraints such as, excluding those presented in Equation (24),

$$\begin{array}{c}{g}_{Z_1}\cong {\mathit{f}}^T\mathit{x}-10-Z\left(\mathit{b}\left(p\right), p\right)\le 0\\ g_{Z_2}\cong {\displaystyle \frac{{\mu }_2}{{\mu }_1}}-7-Z\left(\mathit{b}\left(p\right), p\right)\le 0\end{array}\}.$$

(33)

As indicated in the above Equations (32) and (33), the objective function, $Z$, plays not only the role of a design variable but also a bound on the constraints.

The two design objectives stated in Equation (31) can also be formulated as the targeted objectives in a goal attainment problem such as

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ \mathit{b}\in R^4,\gamma \end{array}& \gamma \left(\mathit{b}\left(p\right), p\right)\end{array}$$

(34)

subjected to the additional constraint set as

$$\begin{array}{c}{g}_{Z_1}={\mathit{f}}^T\mathit{x}-\gamma \times w_1=10\\ g_{Z_2}={\displaystyle \frac{{\mu }_2}{{\mu }_1}}-\gamma \times w_2=7\end{array}\}$$

(35)

where $\gamma$ is the attainment factor, treated as the objective function, and $w_1$ and $w_2$ are the weighting coefficients provided by the users to measure the gap between the goals and the results of the optimal objective functions.

The optimization solution processes of these two sets of problems, Equations (31)–(35), start with the same design variables as previous, ${\mathit{b}}_0=\left[1., 1., 1., 1.\right]$, combined with the same bounds, $0.1\le \mathit{b}\le 20$. Conversely, the bounds on the new design variables, Z and $\gamma$, are set to be between 0 and 20. Setting the lowest values of $Z$ and $\gamma$ to zero ensures that the optimal objectives can match the targeted values.

To achieve this aim, the optimization process is performed repeatedly by adjusting the value of the problem parameter, $p$, from 10 to 10.0215, with the aid of the post-optimality sensitivity equation, Equation (14). The results are listed in

Table 2. Post-optimality sensitivity analysis for minimax and goal attainment problems.

Method	p	${{\mathit{b}}^{\mathit{*}}}^{\mathit{T}}$	${\mathit{f}}^{\mathit{T}}\mathit{x}$	μ2/μ1	dϕ*/dp	∆${\mathit{\varphi }}^{\mathit{*}}$
Minimax	10.0000	[10.1882, 6.7067, 12.9216, 0.1010]	10.0071	7.0050	−0.0331	7.1089 × 10−4
	10.0215	[10.1952, 6.7055, 12.9201, 0.2441]	10.0000	7.0000	−0.0044	1.2008 × 10−4
Goal Attain.	10.0000	[10.1858, 6.6953, 12.9263, 0.1000]	10.0128	7.0128	−0.5958	1.2785 × 10−2
	10.0215	[10.1955, 6.7054, 12.9189, 0.2463]	10.0000	7.0000	8.1003 × 10−8	8.1003 × 10−8

. Both minimax and goal attainment achieve the targeted values of 10 and 7 for ${\mathit{f}}^T\mathit{x}$ and ${\mu }_2/{\mu }_1$, respectively. Note that the column, $∆{\varphi }^{\mathrm{*}},$ listed in Table 2 is the targeted amount of reduction in the optimal objective function evaluated as below, based on the known values of $d{\varphi }^{\mathrm{*}}/dp$ and $∆p$:

$${∆\varphi }^{*}={\displaystyle \frac{d{\varphi }^{*}}{dp}}∆p.$$

In this example, both the minimax and goal attainment problems produce very close optimal design variables, ${\mathit{b}}^{\mathit{*}}$, and are subjected to the same form of the post-optimality sensitivity equation as below:

$${\displaystyle \frac{d{\varphi }^{*}}{dp}}=-4{\lambda }_3{\displaystyle \frac{d{\mu }_1}{dp}}+{\lambda }_4{\mathit{f}}^T{\displaystyle \frac{\partial \mathit{x}}{\partial p}}+{\lambda }_5{\displaystyle \frac{\partial }{\partial p}}\left({\displaystyle \frac{{\mu }_2}{{\mu }_1}}\right).$$

However, they generate different values of $d{\varphi }^{\mathrm{*}}/dp$ at ${\mathit{b}}^{\mathrm{*}}$, as they have different values of Lagrange multipliers. The detailed results are summarized in Table 2.

3.3. Example 3: The Feasible Search Direction Method

The post-optimality sensitivity analysis of the optimal solution of the structural problem described in Section 3.1 is repeatedly investigated herein based on the feasible search direction method. The investigation starts with a problem parameter, $p=10,$ which leads to the optimal design, ${\mathit{b}}^{\mathrm{*}T}=\left[10.2718,11.3447,11.1417,0.1001\right]$ together with the post-optimality sensitivity, $d{\varphi }^{\mathrm{*}}/dp=-0.6065.$ The sensitivity of the optimal design variables with respect to $p$ at ${\mathit{b}}^{\mathrm{*}}$ is estimated by central difference with two more design optimization runs of the structural problem with $p=10.01$ and $p=9.99$. The result is computed as

$${\displaystyle \frac{d{\mathit{b}}^{\mathrm{*}}}{dp}}\approx \left[0.4950,0.6800,0.5600,-0.0050\right].$$

(36)

Based on the quadratic programming formulation, Equations (15) and (16), the search direction ${\mathit{s}}_1$ is solved at ${\mathit{b}}^{\mathrm{*}}$ and $p=10$ as

$${\mathit{s}}_1=\left[{\mathit{s}}_{1\mathit{b}} s_{1p}\right]=\left[-0.2563,-0.0541,0.1401,0.0019,-0.3390\right].$$

The above value of ${\mathit{s}}_1$ can be substituted into Equation (18) to estimate the sensitivity of the optimal design variables as

$${\displaystyle \frac{d\mathit{b}}{dp}}\approx {\displaystyle \frac{{\mathit{s}}_{\mathbf{1}\mathit{b}}}{s_{1p}}}=\left[0.7562,0.1596,-0.4132,-0.0055\right],$$

which is different from the result of Equation (36), solved via central difference. Furthermore, the sensitivity of the objective function at ${\mathit{b}}^{\mathit{*}}$ with respect to the problem parameter, p, is now computed based on Equation (19) as

$${\displaystyle \frac{d\varphi }{dp}}\approx -\left({\displaystyle \frac{\partial \varphi }{\partial \mathit{b}}}+{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}{\displaystyle \frac{d\mathit{x}}{d\mathit{b}}}\right){\displaystyle \frac{{\mathit{s}}_{\mathbf{1}\mathit{b}}}{s_{1p}}}-\left({\displaystyle \frac{\partial \varphi }{\partial p}}+{\displaystyle \frac{\partial \varphi }{\partial \mathit{x}}}{\displaystyle \frac{d\mathit{x}}{dp}}\right){\displaystyle \frac{\partial \varphi }{\partial p}}=-0.5993.$$

Its value is close to the value, −0.6065, obtained via the post-optimality sensitivity of Approach 1, Equation (28). Nevertheless, the goal of the feasible search direction method is to find the minimum design point of ${\mathit{b}}^{\mathit{*}}$ in an expanded design space which includes parameter p as an additional design variable. This new minimum design point, $\mathit{b}$, however, is not an optimal design point as it does not satisfy the Kuhn–Tucker necessary conditions defined in the original design space. In short, the feasible direct search method can be viewed as a method for post-optimality analysis, but not for post-optimality sensitivity analysis.

3.4. Example 4: Finned Heat Sink Design Optimization

A CPU for a computer with a heat transfer conduction surface area of 90 mm by 88.50 mm must be cooled to ensure that the CPU does not reach a surface contact temperature of $T=71 °\mathrm{C}$. The heat flux is constant across the contact area with the base of the heat sink. For simplicity, the thermal contact resistance is assumed to be negligible. Despite the appearance of the heat sink, it is composed of 6061-T651 Aluminum with a specific heat capacity of $C_p=0.896 \mathrm{J}/\mathrm{g}\mathrm{K}$, a thermal conductivity of $k=167 \mathrm{W}/\mathrm{m}\mathrm{K}$, a coefficient of thermal expansion of $\alpha =23.6\left({10}^{-6}\right) {\mathrm{K}}^{-1}$, and a modulus of elasticity of $E=68.9 \mathrm{G}\mathrm{P}\mathrm{a}$.

A graphical representation of the heat sink, upon which this example is based, is shown in Figure 1. The dimensions of the heat sink are also provided in Figure 2 below. The simplification of the geometry neglects the screw interference and assumes that all fins are the same length.

A single fan is used for forced cooling of the heat sink. The orientation of the fan is assumed to be such that the air is pushed lengthwise through the fins. The hydraulic diameter, $D_h$, is provided with the formula for an open channel, with airflow moving in the direction of the arrows in Figure 3. The hydraulic diameter of a single channel of the heat sink is given as

$$D_h={\displaystyle \frac{4b\left(a-2t\right)}{2\left(a-2t\right)+b}}$$

(37)

where $t_b$ is the thickness of the base of the heat sink, and $t$ is the thickness of one fin of the heat sink. The hydraulic diameter is used to measure the hydraulic volumetric efficiency. The variable, $a$, is the distance from the centerline of one fin to the centerline of the fin on the opposite side of the channel, which is approximately equal to the width of the channel. The variable, $b$, is the height of the fins.

The fan moves air over the heat sink at 58 cfm at static pressure (Arctic P9 Max—ARCTIC GMBH, Braunschweig, Germany). At zero pressure differential, air will be pushed at a velocity of approximately $v=3.24 \mathrm{m}/\mathrm{s}$ while assuming a flow area of 92 × 92 mm², which is the cross-sectional area of the fan. However, due to friction, the air velocity is zero at the bottom convection-facing surface of the heat sink and varies linearly with respect to the height above that zero-velocity boundary, until it reaches the maximum air velocity of $3.24 \mathrm{m}/\mathrm{s}$. Additionally, the flow in the heat sink is assumed to be fully developed to reach a constant speed.

The design problem of concern is a 2D section of the model shown in Figure 3, which represents a typical channel of the heat sink where $b$ and $t$ model the height and the width of the fin, and $t_b$ and $L$ model the thickness and length of the base plate. These four dimensions, $b$, $t$, $t_b$, and $L$, will be considered as design variables. Three types of boundary conditions are applied to this 2D heat sink model. The first is a temperature boundary condition applied on the bottom edge of the base of the heat sink. The center point is subjected to the highest temperature, 71 °C, which will be linearly reduced to 55 °C at the edge points of the base. The second is a zero-heat flux boundary condition applied to the left and right edges of the heat sink base, and the third is a convection boundary condition applied to the top edges of the heat sink base and all three edges of the fins. Specifically, the thermal convection condition is formulated using the following equation:

$$-k{\displaystyle \frac{\partial T}{\partial n}}=h\left(T-T_{\infty }\right) \mathrm{o}\mathrm{n} {\mathsf{\Gamma }}_h$$

(38)

where the ambient temperature, $T_{\infty }$, was set at a constant 25 °C, and the convection coefficient, $h_0$, was assumed to be 0.002924 $\mathrm{W}/\left({\mathrm{m}}^2\times \mathrm{K}\right)$. The 2D finite element equation of this heat sink model is formulated as

$$\left(K+K_h\right)\mathit{T}={\mathit{f}}_h-{\mathit{f}}_{\mathit{\infty }}$$

(39)

which is subject to the prescribed temperature boundary conditions along the base plate.

The optimization problem is formulated as a minimax problem. Therefore, the objectives are to deliver the most efficient heat sink that maximizes the cooling rate, $Q_h$, and the hydraulic diameter $D_h$, while minimizing the weight, $W_t$. The cooling rate is measured by the heat flux extracted by the heat convection of the surface exposed to the cooling flow. It is computed using the following equation:

$$Q_h={\int }_{{\mathsf{\Gamma }}_h}h\left(T-T_{\infty }\right)\varphi dA={\mathit{L}}_h^{T}h\left(\mathit{T}-T_{\infty }\right)$$

where $h$ is the heat convection coefficient, and $\mathit{T} \mathrm{a}\mathrm{n}\mathrm{d}$ ${\mathit{L}}_h$ are the surface temperature and the area of the cooling fin exposed to the air flow with temperature. The hydraulic diameter in our case is computed as

$$D_h=10b/(2.5-2t+2t_b).$$

The weight of the cooling unit is

$$W_t=5t_b+2bt.$$

Specifically, the mathematical formulation of the multi-objective design problem is stated as follows

$$\begin{array}{cc}\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{a}\mathrm{x}\\ \mathit{b}\in R^4\end{array}& \begin{array}{c}\mathrm{m}\mathrm{i}\mathrm{n}\\ i=\mathrm{1,2},3\end{array}\end{array}& \left\{\begin{array}{c}{Q}_h\left(\mathit{b},\mathit{T}\left(\mathit{b}\right)\right)\\ D_h\left(\mathit{b}\right)\\ 1/W_t\left(\mathit{b}\right)\end{array}\right\}\end{array}$$

(40)

subjected to a maximal thermal stress constraint

$$\sigma \le {\sigma }_{all}\left(\mathit{b}\right)=110 \mathrm{N}/{\mathrm{m}\mathrm{m}}^2.$$

(41)

The minimax problem defined in Equation (40) can be recast into a single objective one as

$$\begin{array}{cc}\begin{array}{c}\mathrm{m}\mathrm{a}\mathrm{x}\\ \mathit{b}\in R^4,Z\end{array}& Z\end{array}$$

(42)

subjected to three constraints

$$\begin{array}{c}{g}_1=Q_h-Z\ge 0\\ g_2=D_h-Z\ge 0\\ g_3=1/W_t-Z\ge 0\end{array}\}$$

(43)

together with the stress constraint stated in Equation (41).

The problem begins with design variables, $b=24$ mm, $t=1 \mathrm{m}\mathrm{m}$, $t_b=2.5$ mm and $Z=1$, that satisfy all constraints. The MATLAB built-in function fmincon is then applied to solve the above optimization problem. The optimal design reaches $b=22.0002 \mathrm{m}\mathrm{m}$, $t=1.0217 \mathrm{m}\mathrm{m}$, $t_b=2 \mathrm{m}\mathrm{m} ,$ and $Z=1.1009$. The optimal design raises the cooling rate by 10% from the initial 0.5829 watts to 0.6417 watts. It is, however, under the control of two tight constraints, $g_1=g_3=0$ with associated Lagrange multipliers, ${\lambda }_1=$0.9071 and ${\lambda }_3=$ 0.0929. This result implies that the optimal design achieves two objectives which maximize cooling while minimizing the weight. The comparison between the optimized dimension and the initial one is displayed in Figure 4, and the temperature distribution of the optimal design is displayed in Figure 5. The lowest temperature can be found along the top edge of the fin, 23.8862 °C, and the highest temperature is 71 °C at the center of the baseline.

Post-optimality sensitivity analysis is performed hereafter to investigate the impact of air velocity on the maximal cooling rate. The air velocity, $v,$ is related to the heat convection coefficient, $h\left(v\right)=1.16\left[10.45-v+10\sqrt{v}\right]$ [27], with a unit of $\mathrm{W}/\left({\mathrm{m}}^2\times \mathrm{K}\right)$. The post-optimality sensitivity derivative of the maximal cooling rate with respect to the air velocity, $v$, can be computed analytically based upon Equation (14), as

$${\displaystyle \frac{d{Q}_h^{\mathrm{*}}}{dh}}={\lambda }_1{\displaystyle \frac{\partial }{\partial h}}\left(L_h^{\mathrm{*}}h\left(T^{\mathrm{*}}-T_{\mathrm{\infty }}\right)\right),$$

which produces a result of $77.5564 {\mathrm{m}\mathrm{m}}^2\times \mathrm{K}$, a value close to the result obtained by conducting finite differencing between two optimization runs, 78.6594 ${\mathrm{m}\mathrm{m}}^2\times \mathrm{K}$. Estimating the change in the maximal cooling rate due to the change in air velocity can then be performed conveniently as

$${\displaystyle \frac{d{Q}_h^{*}}{dv}}=1.16\left[-1+{\displaystyle \frac{5}{\sqrt{v}}}\right]{\displaystyle \frac{d{Q}_h^{*}}{dh}}.$$

4. Conclusions: Remarks and Discussion

The goal of post-optimality sensitivity analysis is to investigate the impact of a problem parameter on the value of the optimized design objective. This aim is achieved by computing the derivative of the objective function with respect to the problem parameter at the optimized design point. The concept and application of post-optimality sensitivity analysis is not new, having been developed in the early 1980s.

The published methods can be grouped into three approaches. Approach 1 differentiates the optimized objective function first with respect to the parameter p. The derived equation is then simplified by taking the Kuhn–Tucker necessary condition into consideration. This method is the most efficient among all methods, as it involves only first-order derivatives. The second approach, Approach 2, is developed based on the feasible search direction method to find the steepest descent direction to upgrade the design variables at the optimality design point, which maximizes the objective function reduction without violating constraints. Approach 2 is in fact a method for post-optimality analysis, not for post-optimality sensitivity analysis. Approach 3 is the most comprehensive approach, which differentiates the Kuhn–Tucker necessary condition directly with respect to the problem parameter, p, to find the post-optimality sensitivities of the optimality design variables, in addition to the associated Lagrange multipliers. Approach 3 is computationally expensive as it requires solving a linear equation with the first-order derivatives of the optimal design variables and the Lagrange multipliers as unknown.

However, the authors of most published papers did not include the state variable in their derived post-optimality sensitivity equation. In order to broaden the engineering applications of post-optimality sensitivity analysis, we derived the new post-optimality sensitivity analysis equations based on Approaches 1 and 3, which explicitly includes the state variables and their first-order derivatives with respect to the problem parameter in the equations. One feasible search direction is also reformulated to count the state variable. The newly derived post-optimality sensitivity equations are validated for both single and multi-objective problems by using an illustrative example with static and eigenvalue equations. Lastly, an engineering application problem, design optimization of a finned heat sink for a circuit board cooling problem, is formulated and solved as a multi-objective problem. Post-optimality sensitivity analysis is performed thereafter to investigate the impact of air velocity on the maximal cooling rate.

Sensitivity analysis can be categorized into two groups, local vs. global [28]. The scope of this paper is limited to local sensitivity of the optimality objective at an isolated optimality design point. Global sensitivity analysis, in contrast, is mostly conducted through a statistical point of view to investigate the impact on the output due to a wide variation in the input parameters and their interactions. In other words, global sensitivity analysis is associated with uncertainty. The current methods in this study deal with deterministic problems with variation in the deterministic optimal solutions. It is desirable in the near future to thus extend the current research to include time-dependent and stochastic state equations in the post-optimality global sensitivity analysis.