Pareto-Front Optimization of Variance-Added Expected Loss with Interrelated Qualities

Abstract

In industries, particularly in quality optimization, the trade-off between model bias and variance is inevitable, reflecting the tension between accuracy and uncertainty. Traditional methods often address these aspects separately, potentially leading to suboptimal decisions. This study proposes a Pareto-front optimization framework for a variance-added expected loss function within the context of interrelated quality characteristics. By integrating multivariate quadratic loss with a variance term, our approach simultaneously captures deviation from targets (bias) and system uncertainty (variance). Unlike sequential approaches that first minimize bias and then variance—often increasing total risk—our weighted formulation flexibly adjusts for their trade-offs. This enables a more balanced and efficient optimization process that identifies solutions with lower overall risk. Through Pareto-front analysis, we reveal trade-offs between expected loss and variance, allowing users to select optimal quality designs based on their preferred bias–variance balance. Representative examples and a case study adopted from the literature validate the effectiveness and practical applicability of the proposed method.

1. Introduction

In modern industries, achieving optimal performance across multiple interrelated quality attributes is a complex, yet essential, task. Systems often face conflicting objectives, in which improving one quality indicator may deteriorate another. As industries demand higher precision and robustness, efficiently managing these trade-offs becomes increasingly critical. In a machine-learning task, for example, a trade-off between model bias and model variance is inevitable: a low-level model bias from substantial model complexity incurs high-level model variance, and vice versa. Representative quality objectives in conflict are accuracy and variance arising from predefined targets and random observations.

Traditional optimization methods typically focus on minimizing a single objective loss function, but they often fail to account for interdependencies between different quality attributes. Many real-world systems such as manufacturing, engineering, and healthcare exhibit tightly coupled quality characteristics that directly influence overall system performance. In such contexts, a more comprehensive approach is required—one that considers not only deviations from target quality but also uncertainty in system behavior by incorporating variance into overall loss.

This paper proposes a novel approach for globally optimizing interrelated quality attributes by adding variance-induced uncertainty to expected loss. The goal is to minimize not only deviation from a target quality but also uncertainty in performance. For the sake of global optimization, we use Pareto fronts to explore trade-offs between conflicting objectives, enabling decision-makers to select solutions, or decisions, that achieve the desired balance between accuracy and risk. Importantly, our method allows users to explicitly choose the ratio between bias and variance terms, providing greater flexibility to manage the trade-offs arising from specific application requirements.

In this study, we apply an alternative global criterion (GC)-based method for generating Pareto fronts to handle accuracy-induced and variance-induced qualities. This method offers increased computational efficiency compared to traditional optimization techniques and effectively handles the trade-offs between multiple objectives. We illustrate the applicability of this approach through two representative examples and demonstrate its practical use with a dataset adopted from the literature. The experiments reveal the applicability of the proposed method to a complex system with interrelated quality attributes. Our analysis shows that the optimization results outperform existing quality decisions in terms of both quality and flexibility, providing valuable insights into achieving quality optimization in industrial settings.

The proposed method presented in this paper offers a systematic framework for quality optimization in complex systems with interrelated attributes and variance. By applying this approach, decision-making processes in industries requiring simultaneous optimization of multiple quality factors can be significantly improved.

2. Related Works

In multi-objective optimization problems, it is often necessary to simultaneously consider distinct quality characteristics such as mean deviation and variance. As a result, conventional single-objective optimization techniques have limited applicability, and various multi-objective quality optimization methods have been proposed.

A well-known early approach is the two-step strategy proposed by [1], in which variance is minimized first, followed by minimization of mean deviation. This method is computationally simple and easy to interpret, making it appealing in practical settings. However, in real-world applications, most design variables influence both the mean and variance simultaneously. Thus, separating these objectives can lead to inaccurate decisions and reduce the overall predictive accuracy and interpretability of the optimization process.

Other studies have sought to mathematically incorporate interactions between multiple quality characteristics. For example, Shiau proposed a weighted sum of signal-to-noise (SN) ratios to aggregate multiple quality characteristics [2], while Tong et al. developed a multi-response SN ratio that sums weighted expected losses before converting the result to an overall performance measure [3]. More recently, a loss function explicitly considering interactions between variables has been proposed (see, e.g., [4]), offering a more theoretically robust formulation compared to earlier methods. However, these approaches share practical limitations: they rely mainly on numerical computations from existing data, do not support the setting of design variables or the generation of actionable design alternatives, and thus are generally restricted to post hoc analysis rather than active design optimization.

Building on such interaction-based loss functions, Costa and Lourenço [5] proposed a game-theoretic approach to simultaneously optimize process and product quality characteristics in bi-objective settings. By employing Stackelberg’s strategy, their method avoids subjective parameter tuning and seeks efficient trade-offs. Nonetheless, such game-theoretic frameworks often struggle to yield a solution that simultaneously satisfies all quality objectives, especially when multiple interdependent characteristics are involved. This underscores the need for methods that not only capture interactions among characteristics but also offer clear and actionable design recommendations.

To address such multi-objective optimization challenges, especially in design settings, surrogate-based approaches have gained popularity. A representative example is the efficient global optimization (EGO) algorithm [6], which uses the expected improvement (EI) criterion to sequentially explore the design space and identify optima. Ref. [7] extended this framework to multi-objective problems by introducing ParEGO, an algorithm that employs Tchebycheff scalarization to search the Pareto front. Furthermore, ref. [8] proposed an integrated approach combining multiple scalarization techniques (e.g., Tchebycheff and weighted sum) to better approximate diverse trade-offs. However, these methods inherently reduce multi-objective problems to a series of single-objective subproblems and, as a result, cannot fully capture the complex trade-offs among conflicting objectives.

In this study, we propose a multivariate quadratic loss-based Pareto optimization framework that mathematically captures the interactions between design variables and multiple quality characteristics, and simultaneously accounts for overall quality loss arising from both mean deviation and variance. In contrast to scalarization-based methods, the proposed approach directly derives interpretable Pareto-optimal solutions in the multi-objective space without reducing the problem to a single-objective formulation. For reproducibility, we provide a GitHub URL (https://github.com/KIM-sang-won/PFOVELIQ/blob/main/PFOVE__Python_code.ipynb, accessed on 4 February 2026) for the experiments.

3. Methods

3.1. Multivariate Quadratic Loss Function and Total Risk

A multivariate quadratic loss function was introduced by Pignatiello [1] and is defined as follows:

$$L\left(\mathbf{y}\right)={(\mathbf{y}-\mathit{\tau })}^T\mathbf{C}(\mathbf{y}-\mathit{\tau }),$$

(1)

where $\mathbf{y}={(y_1,\dots ,y_p)}^T$ is a $p\times 1$ vector consisting of principal components (PCs) of the ’nominal-the-best’ type, and $\mathit{\tau }={({\tau }_1,\dots ,{\tau }_p)}^T$ is a $p\times 1$ vector containing the corresponding target values. The $p\times p$ positive definite cost matrix $\mathbf{C}$ assigns weights to deviations among different PCs.

The elements of $\mathbf{C}$ are determined by defining the loss in terms of deviations. Let $z_i=y_i-{\tau }_i$ denote the deviation of the i-th characteristic from its target. For example, in a bivariate case ($p=2$), the loss function is expressed as:

$$L\left(\mathbf{y}\right)=c_{11}{z}_1^2+c_{22}{z}_2^2+2c_{12}{z}_1{z}_2,$$

in which $c_{ii}$ represents the cost coefficient associated with the squared deviation $z_i^2$, and $c_{ij}$, $i\ne j$, accounts for the interaction effect between deviations $z_i$ and $z_j$. By default, $\mathbf{y}$ is of the nominal-the-best type, but it can be adjusted to the lower-the-better or higher-the-better type by modifying $\mathit{\tau }$ accordingly.

Furthermore, Pignatiello [1] assumes that $\mathbf{y}$ follows a p-dimensional multivariate normal distribution with mean vector $\mathit{\mu }={({\mu }_1,\dots ,{\mu }_p)}^T$ and $p\times p$ variance–covariance matrix $\mathbf{\Sigma }=\left({\sigma }_{ij}^2\right)$, i.e.,

$$\mathbf{y}\sim \mathcal{N}(\mathit{\mu },\mathbf{\Sigma }),$$

(2)

where $\mathit{\mu }$ represents the expected values of the PCs, and $\mathbf{\Sigma }$ captures their variability and correlations.

The expected quadratic loss can be decomposed into bias and variance contributions:

$$\mathbf{E}\left[L\left(\mathbf{y}\right)\right]={(\mathit{\mu }-\mathit{\tau })}^T\mathbf{C}(\mathit{\mu }-\mathit{\tau })+\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right),$$

(3)

where the first term measures the deviation of the mean from the target, and the second term represents the variance contribution weighted by $\mathbf{C}$.

We define the total risk T by adding the variance contribution weighted by the parameter $\lambda$:

$$T=\mathbf{E}\left[L\left(\mathbf{y}\right)\right]+\lambda \mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right)={(\mathit{\mu }-\mathit{\tau })}^T\mathbf{C}(\mathit{\mu }-\mathit{\tau })+(1+\lambda )\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right),$$

(4)

where we used $\mathbf{E}\left[L\left(\mathbf{y}\right)\right]={(\mathit{\mu }-\mathit{\tau })}^T\mathbf{C}(\mathit{\mu }-\mathit{\tau })+\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right)$. The term $\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right)$ derives from the identity

$$\mathbf{E}\left[{(\mathbf{y}-\mathit{\mu })}^T\mathbf{C}(\mathbf{y}-\mathit{\mu })\right]=\mathrm{trace}\left(\mathbf{E}\left[{(\mathbf{y}-\mathit{\mu })}^T\mathbf{C}(\mathbf{y}-\mathit{\mu })\right]\right)=\mathrm{trace}\left(\mathbf{E}\left[\mathbf{C}(\mathbf{y}-\mathit{\mu }){(\mathbf{y}-\mathit{\mu })}^T\right]\right)=\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right).$$

Physically, it represents cost-weighted variability, implying that variance contributes to loss only when associated with non-zero costs in $\mathbf{C}$. Thus, the risk is interpreted in terms of economic consequences rather than variance alone. The parameter $\lambda$ adjusts this variance contribution, typically satisfying $\lambda \ge 0$ but allowing $\lambda \ge -1$.

Mathematically, introducing $\lambda$ generalizes the standard expected loss by relaxing the rigid 1:1 bias–variance trade-off, allowing for a flexible control and scalarization of conflicting objectives. Practically, minimizing variance is often more critical than centering the mean, as excessive variability can lead to irreversible consequences. For instance, high variance can cause matching failures in precision assembly or explosion risks in chemical processes, whereas mean shifts are often correctable. Thus, $\lambda$ allows for heavier penalization of variability, reflecting a risk-averse strategy in robust design.

3.2. Risk Derivation for Mixed Quality Characteristics

To accommodate both nominal-the-best (${\tau }_i\ne 0$) and smaller-the-better (${\tau }_i=0$) types of quality characteristics, we partition the vector $\mathbf{y}$ into two sub-vectors, ${\mathbf{y}}_1$ and ${\mathbf{y}}_2$:

$$\mathbf{y}={\left[{\mathbf{y}}_1^T{\mathbf{y}}_2^T\right]}^T,$$

where ${\mathbf{y}}_1={(y_1,\dots ,y_q)}^T$ is a $q\times 1$ vector of nominal-the-best PCs, and ${\mathbf{y}}_2={(y_{q+1},\dots ,y_p)}^T$ is a $(p-q)\times 1$ vector of smaller-the-better PCs. For simplicity, one may also scale $\mathbf{y}$ to remove unit effects in the loss function.

Following this partition, all related vectors and matrices are restructured accordingly. In particular, the mean vector $\mathit{\mu }$ and the target vector $\mathit{\tau }$ are expressed as

$$\mathit{\mu }={\left[{\mathit{\mu }}_1^T{\mathit{\mu }}_2^T\right]}^T,\mathit{\tau }={\left[{\mathit{\tau }}_1^T{\mathit{\tau }}_2^T\right]}^T,\mathrm{with}{\mathit{\tau }}_2=\mathbf{0}.$$

The covariance matrix $\mathbf{\Sigma }$ and the cost matrix $\mathbf{C}$ are also expressed in block forms, respectively:

$$\mathbf{\Sigma }=\left[\begin{array}{cc}{\mathbf{\Sigma }}_{11}& {\mathbf{\Sigma }}_{12}\\ {\mathbf{\Sigma }}_{12}^T& {\mathbf{\Sigma }}_{22}\end{array}\right],\mathbf{C}=\left[\begin{array}{cc}{\mathbf{C}}_{11}& {\mathbf{C}}_{12}\\ {\mathbf{C}}_{12}^T& {\mathbf{C}}_{22}\end{array}\right].$$

Using this structure, the multivariate quadratic loss function for $\mathbf{y}$ is defined as

$$\begin{array}{cc}\hfill L\left(\mathbf{y}\right)& ={(\mathbf{y}-\mathit{\tau })}^T\mathbf{C}(\mathbf{y}-\mathit{\tau })={\left[\begin{array}{c}{\mathbf{y}}_1-{\mathit{\tau }}_1\\ {\mathbf{y}}_2\end{array}\right]}^T\left[\begin{array}{cc}{\mathbf{C}}_{11}& {\mathbf{C}}_{12}\\ {\mathbf{C}}_{12}^T& {\mathbf{C}}_{22}\end{array}\right]\left[\begin{array}{c}{\mathbf{y}}_1-{\mathit{\tau }}_1\\ {\mathbf{y}}_2\end{array}\right]\hfill \\ & ={({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{11}({\mathbf{y}}_1-{\mathit{\tau }}_1)+{\mathbf{y}}_2^T{\mathbf{C}}_{22}{\mathbf{y}}_2+2{({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{12}{\mathbf{y}}_2.\hfill \end{array}$$

The expected multivariate quadratic loss of $\mathbf{y}$ can then be written as

$$\mathbf{E}\left[L\left(\mathbf{y}\right)\right]=\mathbf{E}\left[{({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{11}({\mathbf{y}}_1-{\mathit{\tau }}_1)\right]+\mathbf{E}\left[{\mathbf{y}}_2^T{\mathbf{C}}_{22}{\mathbf{y}}_2\right]+2\mathbf{E}\left[{({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{12}{\mathbf{y}}_2\right].$$

Expanding the first and second terms using the mean vector $\mathit{\mu }$ and variance–covariance matrix $\mathbf{\Sigma }$, we have

$$\begin{array}{cc}\hfill \mathbf{E}\left[{({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{11}({\mathbf{y}}_1-{\mathit{\tau }}_1)\right]& =\mathrm{trace}\left({\mathbf{C}}_{11}{\mathbf{\Sigma }}_{11}\right)+\mathrm{trace}({\mathbf{C}}_{11}({\mathit{\mu }}_1-{\mathit{\tau }}_1){({\mathit{\mu }}_1-{\mathit{\tau }}_1)}^T),\hfill \\ \hfill \mathbf{E}\left[{\mathbf{y}}_2^T{\mathbf{C}}_{22}{\mathbf{y}}_2\right]& =\mathrm{trace}\left({\mathbf{C}}_{22}{\mathbf{\Sigma }}_{22}\right)+\mathrm{trace}\left({\mathbf{C}}_{22}{\mathit{\mu }}_2{\mathit{\mu }}_2^T\right),\hfill \end{array}$$

and the third term can be rewritten as

$$\mathbf{E}\left[{({\mathbf{y}}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{12}{\mathbf{y}}_2\right]=\mathrm{trace}\left({\mathbf{C}}_{12}{\mathbf{\Sigma }}_{12}^T\right)+\mathrm{trace}\left({\mathit{\mu }}_2{({\mathit{\mu }}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{12}\right).$$

Combining all three terms, the expected loss becomes

$$\begin{array}{cc}\hfill \mathbf{E}\left[L\right(\mathbf{y}\left)\right]& =\mathrm{trace}\left({\mathbf{C}}_{11}{\mathbf{\Sigma }}_{11}\right)+\mathrm{trace}({\mathbf{C}}_{11}({\mathit{\mu }}_1-{\mathit{\tau }}_1){({\mathit{\mu }}_1-{\mathit{\tau }}_1)}^T)\hfill \\ & +\mathrm{trace}\left({\mathbf{C}}_{22}{\mathbf{\Sigma }}_{22}\right)+\mathrm{trace}\left({\mathbf{C}}_{22}{\mathit{\mu }}_2{\mathit{\mu }}_2^T\right)\hfill \\ & +2\mathrm{trace}\left({\mathbf{C}}_{12}{\mathbf{\Sigma }}_{12}^T\right)+2\mathrm{trace}\left({\mathit{\mu }}_2{({\mathit{\mu }}_1-{\mathit{\tau }}_1)}^T{\mathbf{C}}_{12}\right)\hfill \\ & =\mathrm{trace}\left(\mathbf{C}(\mathit{\mu }-\mathit{\tau }){(\mathit{\mu }-\mathit{\tau })}^T\right)+\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right).\hfill \end{array}$$

(5)

This can also be expressed in summation form as

$$\mathbf{E}\left[L\left(\mathbf{y}\right)\right]=\sum _{i=1}^p\sum _{j=1}^p{c}_{ij}({\mu }_i-{\tau }_i)({\mu }_j-{\tau }_j)+\sum _{i=1}^p\sum _{j=1}^p{c}_{ij}{\sigma }_{ij}^2.$$

(6)

Hence, from Equations (5) and (6), we could express the total risk, T, as 

$$\begin{array}{cc}\hfill T& =\mathbf{E}\left[L\left(\mathbf{y}\right)\right]+\lambda \mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right)=\mathrm{trace}\left(\mathbf{C}(\mathit{\mu }-\mathit{\tau }){(\mathit{\mu }-\mathit{\tau })}^T\right)+(1+\lambda )\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\right)\hfill \\ & =\sum _{i=1}^p\sum _{j=1}^p{c}_{ij}({\mu }_i-{\tau }_i)({\mu }_j-{\tau }_j)+(1+\lambda )\sum _{i=1}^p\sum _{j=1}^p{c}_{ij}{\sigma }_{ij}^2.\hfill \end{array}$$

(7)

Assuming that independent input variables $\mathbf{x}={(x_1,\dots x_k)}^T$ influence $\mathbf{y}$, the mean vector and covariance matrix are expressed as functions of $\mathbf{x}$, $\mathit{\mu }\left(\mathbf{x}\right)$ and $\mathbf{\Sigma }\left(\mathbf{x}\right)$. The bias and variance components are then defined as

$$\begin{array}{cc}\hfill f_1\left(\mathbf{x}\right)& =\mathrm{trace}\left(\mathbf{C}(\mathit{\mu }\left(\mathbf{x}\right)-\mathit{\tau }){(\mathit{\mu }\left(\mathbf{x}\right)-\mathit{\tau })}^T\right),\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill f_2\left(\mathbf{x}\right)& =\mathrm{trace}\left(\mathbf{C}\mathbf{\Sigma }\left(\mathbf{x}\right)\right),\hfill \end{array}$$

(9)

so that the total risk can be written as $T=f_1\left(\mathbf{x}\right)+(1+\lambda )f_2\left(\mathbf{x}\right)$. Naturally, the total risk $T\ge 0$ leads to the parameter, the variance weight, $\lambda \ge -1.$

The determination of the weighting parameter, $1+\lambda$, is possible in two ways depending on the decision-maker’s strategy. Subjectively, users may assign a value to $1+\lambda$ reflecting their perceived importance of bias versus variance. Alternatively, for an objective criterion, $1+\lambda$ can be derived from economic cost structures. If we define the total expected cost as $\mathbf{E}\left[\mathrm{Cos}\mathrm{t}\right]=C_{\mathrm{bias}}{f}_1\left(\mathbf{x}\right)+C_{\mathrm{var}}{f}_2\left(\mathbf{x}\right)$, where $C_{\mathrm{bias}}$ and $C_{\mathrm{var}}$ are the aggregate cost coefficients for bias and variance respectively, then normalizing the equation by $C_{\mathrm{bias}}$ yields $1+\lambda =C_{\mathrm{var}}/C_{\mathrm{bias}}$. For example, $C_{\mathrm{bias}}$ may represent process adjustment costs, while $C_{\mathrm{var}}$ corresponds to the scrap costs associated with defective products. This formulation allows the trade-off to be driven by tangible economic data when available.

3.3. Multi-Objective Optimization via Global Criterion

Ideally, one would minimize the sum of the functions for a given $\lambda$. Practically, however, minimizing both functions simultaneously is challenging. For example, field engineers typically determine a few inputs in $\mathbf{x}$ to optimize $f_1$, then determine the remaining to optimize $f_2$ sequentially.

To effectively balance both terms of bias and variance and to globally optimize them, we utilize a Pareto front that represents the comprehensive optimal trade-offs. Unlike standard optimization problems targeting a single objective, multi-objective optimization seeks a set of solutions, called a Pareto optimal set, where no solution is strictly better than the others in all objectives. These solutions form a Pareto front when visualized, allowing us to analyze and adjust $\lambda$ for an appropriate compromise.

In essence, the Global Criterion (GC) method transforms the multi-objective problem into a scalar minimization problem by measuring the relative deviation of each objective function from its ideal value. We apply an alternative global criterion (GC)-based method for generating a well-distributed Pareto front [9]. The GC function minimizes the distance to a target, defined as ${\sum }_{i=1}^n{\left(|{\hat{y}}_i-T_i|/(U_i-L_i)\right)}^{p_i}$, where ${\hat{y}}_i$ is the i-th estimated response, $T_i$ is the target (here $T_i=0$), and $U_i,L_i$ are the bounds.

Adapting this to our problem with two objective functions ($n=2$), $f_1\left(\mathbf{x}\right)$ and $f_2\left(\mathbf{x}\right)$, the GC function is formulated as:

$$\begin{array}{c}\hfill {\min }_{\mathbf{x}}{\left({\displaystyle \frac{\left|{\hat{f}}_1\left(\mathbf{x}\right)\right|}{U_1-L_1}}\right)}^{p_1}+{\left({\displaystyle \frac{\left|{\hat{f}}_2\left(\mathbf{x}\right)\right|}{U_2-L_2}}\right)}^{p_2},\end{array}$$

(10)

in which the upper ($U_i$) and lower ($L_i$) bounds are defined as the maximum and minimum values of ${\hat{f}}_i\left(\mathbf{x}\right)$, an approximation of $f_i\left(\mathbf{x}\right)$ over the dataset $\mathcal{X}$,

$$U_i=\underset{\mathbf{x}\in \mathcal{X}}{max}{\hat{f}}_i\left(\mathbf{x}\right),L_i=\underset{\mathbf{x}\in \mathcal{X}}{min}{\hat{f}}_i\left(\mathbf{x}\right).$$

We assume the function is well-behaved enough for its maximum and minimum to be attainable and computable over the dataset $\mathcal{X}$. Alternatively, these bounds can be set by technical constraints or economic conditions. Normalization by $(U_i-L_i)$ is crucial for the following reasons [10]:

• It ensures comparability across responses with different scales.
• It reduces subjectivity in weighting.
• It prevents a single criterion from dominating the optimization.
• It improves the spread of Pareto-optimal solutions.

The exponent parameters $p_1$ and $p_2$ assign priorities to the objectives, enabling the exploration of trade-offs. While $p_i$ can theoretically range from 0 to infinity, Costa et al. [9] suggest that $p_i>3$ yields diminishing returns. Our experiments confirm that limiting $p_i\le 3$ is sufficient. By varying $p_1$ and $p_2$ numerically in a grid-search manner, we can explore both convex and non-convex regions of the Pareto frontier [11].

In this research, we perform the optimization over $\mathbf{x}$ using pairs of $p_1$ and $p_2$ from a predefined grid. The resulting diverse set of solutions ${\mathbf{x}}^{\ast }$ numerically forms the Pareto front, spanning various trade-offs to offer a comprehensive view of the optimization landscape.

To solve Equation (10), we adopt the SLSQP algorithm from Python’s scipy.optimize. minimize library. As a gradient-based method, it efficiently handles equality and inequality constraints, making it well-suited for nonlinear, multi-response problems required to construct an effective Pareto front.

3.4. Procedure of Framework

To summarize the operational procedure of our framework, we propose the following four-step workflow. First, appropriate regression models are fitted for the mean, variance, and covariance of the quality characteristics using data from a designed experiment. Second, the bias loss $f_1\left(\mathbf{x}\right)$ and variance loss $f_2\left(\mathbf{x}\right)$ are computed as functions of the design variables via the fitted models. Third, the Pareto front is generated by solving the GC minimization problem across a grid of weighting exponents $p_1$ and $p_2$. Finally, the optimal operating condition is selected from the Pareto front by identifying the point that satisfies a desired risk tolerance or a specific bias-to-variance ratio.

4. Experiments

We now present two synthetic cases to demonstrate the applicability of the proposed approach.

4.1. Case 1: Bivariate Quality Characteristics ($p=2$)

In the first case, we consider one NB-type PC and one SB-type PC, i.e., $q=1$ and $p=2$:

$${\mathbf{y}}_1=\left(y_1\right),{\mathbf{y}}_2=\left(y_2\right).$$

Then, total risk $T=f_1\left(\mathbf{x}\right)+(1+\lambda )f_2\left(\mathbf{x}\right)$ in Equations (7) and (10) is defined by the functions $f_1\left(\mathbf{x}\right)$ and $f_2\left(\mathbf{x}\right)$ as follows:

$$f_1\left(\mathbf{x}\right)=\sum _{i=1}^2\sum _{j=1}^2{c}_{ij}({\mu }_i\left(x\right)-{\tau }_i)({\mu }_j\left(\mathbf{x}\right)-{\tau }_j),f_2\left(\mathbf{x}\right)=\sum _{i=1}^2\sum _{j=1}^2{c}_{ij}{\sigma }_{ij}^2\left(\mathbf{x}\right).$$

For input variables $\mathbf{x}=(x_1,x_2)$, $-1\le x_i\le 1,i=1,2$, let us assume that the estimated regression functions ${\hat{f}}_1\left(\mathbf{x}\right)$ and ${\hat{f}}_2\left(\mathbf{x}\right)$, corresponding to $f_1\left(\mathbf{x}\right)$ and $f_2\left(\mathbf{x}\right)$, are given as follows:

$${\hat{f}}_1\left(\mathbf{x}\right)={(3x_1+3x_2+6)}^2,{\hat{f}}_2\left(\mathbf{x}\right)={({(x_1+x_2)}^2+2)}^2.$$

Furthermore, we set $U_i$ and $L_i$, for each $i=1,2$, to be 0 and 100, respectively. The input variables satisfy $-1\le x_i\le 1$. We increment each parameter of $p_1$ and $p_2$ from 0.05 to 3 with step size 0.1. The initial starting point in the SLSQP algorithm is set to $(0,0)$.

Using the alternative GC-based method, we numerically obtain the Pareto frontier shown in Figure 1. In this graph, increasing the value of $\lambda$, which reflects a higher emphasis on ${\hat{f}}_2\left(\mathbf{x}\right)$, leads to solutions $\mathbf{x}$ with higher bias but lower variance. Conversely, decreasing $\lambda$ emphasizes ${\hat{f}}_1\left(\mathbf{x}\right)$, resulting in lower bias but higher variance.

Figure 1. Pareto-front solutions highlighting a solution (depicted by the red circle) that minimizes the sum of bias loss and variance loss, $\lambda =0$, for the first case.

For instance, as shown in Figure 1, when $\lambda =0$, the minimum total risk T is 17.54, achieved at ${\mathbf{x}}^{\ast }=(-0.57573,-0.57573)$, with corresponding objective values ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=6.48$ and ${\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=11.06$, which is highlighted by the red circle in Figure 1. Notice that for $\lambda =0$ the tangent line with the ratio, x-intercept/y-intercept, being 1 (thereby slope $-1$) is adopted. If $\lambda$ differs from zero in general, one can easily draw another tangent line of which the ratio, x-intercept/y-intercept, is $1+\lambda$ (thereby slope $-1-\lambda$) in Figure 1, finding corresponding objective values ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)$ and ${\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ along with input vector ${\mathbf{x}}^{\ast }$.

Here, ${\mathbf{x}}^{\ast }$ denotes the input vector that minimizes the GC-based objective function for a given value of $p_i$, where ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)$ and ${\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ represent the estimated bias and variance components, respectively, at the optimal point. In Figure 1, the red line indicates the tangent with slope $-1$ to the Pareto frontier at the point where the total risk $T={\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)+{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$, $\lambda =0$, is minimized, illustrating the trade-off between the two objectives. This suggests that decision-makers can adjust the input variables by finding a tangent line with a slope relating to their specific industry characteristics or strategic preferences.

As shown above, the construction of a Pareto front enables flexible control over the trade-off between bias and variance terms. Furthermore, let us suppose that one wants to set a ratio of the bias-loss term to the variance-loss term to a certain value in practice. Figure 2 shows three Pareto-Front solutions for specific ratios, $2:1,1:1,$ and $1:2$ of the two terms under approximation error $0.1$.

Figure 2. Pareto-front solutions, depicted by the red circles, for specific ratios of bias loss to variance loss, ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right):{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ being 1:2, 1:1, and 2:1, in the first case.

Firstly, if one sets $f_1\left(\mathbf{x}\right)$ to be twice $f_2\left(\mathbf{x}\right)$ with the ratio being $2:1$, the optimal solution is $\mathbf{x}=[-0.387,-0.387]$, with function values $f_1\left(\mathbf{x}\right)=13.52$ and $f_2\left(\mathbf{x}\right)=6.76$. As seen in Figure 2, the solution is easily extracted by drawing a line with slope $1/2$ passing through the origin. Secondly, when $f_1\left(\mathbf{x}\right)$ equals $f_2\left(\mathbf{x}\right)$, in which the ratio is $1:1$, we easily obtain the optimal solution, $\mathbf{x}=[-0.5,-0.5]$, which yields function values $f_1\left(\mathbf{x}\right)=f_2\left(\mathbf{x}\right)=9.00$. Thirdly, when $f_1\left(\mathbf{x}\right)$ is half of $f_2\left(\mathbf{x}\right)$ with a ratio of $1:2$, the optimal solution is $\mathbf{x}=[-0.595,-0.595]$ with function values $f_1\left(\mathbf{x}\right)=5.90$ and $f_2\left(\mathbf{x}\right)=11.67$. These results illustrate how different bias-variance preferences can be effectively represented by specific functional ratios along the Pareto front. We also notice that the extraction of both bias risk $f_1\left(\mathbf{x}\right)$ and variance risk $f_2\left(\mathbf{x}\right)$ with a desired ratio is hardly possible without the use of the proposed approach.

4.2. Case 2: Four-Variable Quality Characteristics ($p=4$)

In the second case, we consider the situation where there are two NB-type PCs and two SB-type PCs, i.e., $q=2$ and $p=4$:

$${\mathbf{y}}_1=(y_1,y_2),{\mathbf{y}}_2=(y_3,y_4).$$

The total risk $T=f_1\left(\mathbf{x}\right)+f_2\left(\mathbf{x}\right)$, $\lambda =0$, in Equations (7) and (10) is defined by functions $f_1\left(\mathbf{x}\right)$ and $f_2\left(\mathbf{x}\right)$ as follows:

$$f_1\left(\mathbf{x}\right)=\sum _{i=1}^4\sum _{j=1}^4{c}_{ij}({\mu }_i\left(\mathbf{x}\right)-{\tau }_i)({\mu }_j\left(\mathbf{x}\right)-{\tau }_j),f_2\left(\mathbf{x}\right)=\sum _{i=1}^4\sum _{j=1}^4{c}_{ij}{\sigma }_{ij}^2\left(\mathbf{x}\right).$$

For input variables $\mathbf{x}=(x_1,x_2,x_3)$, let us assume that ${\hat{f}}_1\left(\mathbf{x}\right)$ and ${\hat{f}}_2\left(\mathbf{x}\right)$ are expressed as follows:

$${\hat{f}}_1\left(\mathbf{x}\right)={(0.5x_1^2-0.5x_2^2+0.5x_3^2+x_1{x}_3-4)}^2,{\hat{f}}_2\left(\mathbf{x}\right)={(x_1^2-x_2^2+3x_3)}^2.$$

Similarly to the first case, the upper bounds of ${\hat{f}}_1\left(\mathbf{x}\right)$ and ${\hat{f}}_2\left(\mathbf{x}\right)$ are set to 100, and the lower bounds are set to 0. The initial starting point is $(1,1,1)$, and the input variables are constrained within the range $-1\le x_i\le 1$. The exponent parameters $p_i$ are incremented from 0.05 to 3 with step size $0.05$. Under the setting, we obtain a Pareto front as shown in Figure 3.

Figure 3. Pareto-front solutions highlighting a solution (depicted by the red circle) that minimizes the sum of bias loss and variance loss, $\lambda =0$, for the second case.

When $\lambda =0$, we achieve the minimum value of the total risk T at 13.23314, which is highlighted by the red circle (by slope $-1$) in Figure 3, with

$${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=12.11403,{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=1.11911,$$

and the corresponding input vector is ${\mathbf{x}}^{\ast }=(1,0.00209,0.01929).$

By allowing approximation error $0.1$, we similarly extract optimal settings under desired trade-off between bias loss and variance loss as shown in Figure 4.

Figure 4. Pareto-front solutions depicted by circles for specific ratios of bias loss to variance loss, ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right):{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ being 1:2, 1:1, and 2:1, in the second case.

Similarly, we extract the solutions from the Pareto front by drawing a line with the appropriate slope passing through the origin. Firstly, when the ratio $f_1\left(\mathbf{x}\right)/f_2\left(\mathbf{x}\right)$ is 2, we find the optimal solution, $\mathbf{x}=[1.00,1.00,0.71],$ which yields function values $f_1\left(\mathbf{x}\right)=9.19,f_2\left(\mathbf{x}\right)=4.59.$ Secondly, when the ratio $f_1\left(\mathbf{x}\right)/f_2\left(\mathbf{x}\right)$ is 1, the optimal solution becomes $\mathbf{x}=[1.00,1.00,0.90]$ with function values, $f_1\left(\mathbf{x}\right)=7.28,f_2\left(\mathbf{x}\right)=7.27.$ Thirdly, when the ratio $f_1\left(\mathbf{x}\right)/f_2\left(\mathbf{x}\right)$ is $1/2$, the optimal solution is $\mathbf{x}=[1.00,0.83,1.00],$ yielding function values, $f_1\left(\mathbf{x}\right)=5.49,f_2\left(\mathbf{x}\right)=10.99.$

We notice that the Pareto frontier obtained using the GC-based method contains visual gaps as shown in Figure 3 and Figure 4. Similar discontinuities have been observed in previous studies [9], which demonstrated that achieving a perfectly connected and comprehensive Pareto front is often difficult, especially in the presence of complex or nonconvex response surfaces.

5. Application to Simulation Data

In this section, we apply our approach to a simulation dataset introduced by Pignatiello [1]. This problem serves as a standard test case in multi-response optimization literature, involving two response variables, $y_1$ and $y_2$, and three input variables, $x_1,x_2,$ and $x_3$.

5.1. Simulation Data and Regression Modeling

The dataset simulates a replicated factorial experiment, enabling the estimation of mean and covariance structures essential for joint optimization. By capturing simultaneous shifts in mean and variance, the Pareto front reveals risk trade-offs that sequential approaches often overlook.

The simulation data is presented in Table 1.

Table 1. Simulation Data.

			${\mathit{y}}_1$				${\mathit{y}}_2$
$x_1$	$x_2$	$x_3$	1	2	3	4	1	2	3	4
1	1	1	104.454	105.029	99.786	104.923	76.9003	77.0322	67.989	75.7691
1	1	−1	104.120	104.802	104.203	104.335	72.9878	74.2487	73.9371	73.2824
1	−1	1	98.732	99.357	102.842	94.235	67.0955	63.6112	68.647	62.4188
1	−1	−1	100.192	99.634	100.269	100.600	67.0264	66.1779	66.5758	67.9431
−1	1	1	103.145	106.959	107.620	103.440	71.6818	76.2657	77.4958	76.3739
−1	1	−1	106.078	105.642	105.670	105.393	72.9353	72.8508	72.5768	72.3754
−1	−1	1	113.515	111.121	112.854	106.666	68.2934	68.4693	68.9576	64.7051
−1	−1	−1	109.895	109.759	110.704	109.773	67.6974	67.2374	67.962	66.9268

Using the values in each row in Table 1, we obtain the sample mean, sample variance, and sample covariance as shown in Table 2. For example, the first ${\hat{\mathit{\mu }}}_1=103.54875$ is the sample average of the four $y_1$ observations, $104.454,105.029,99.786,104.923$, in Table 1.

Table 2. Statistical Summary of Simulation Data.

${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{x}}_3$	${\hat{\mathit{\mu }}}_1$	${\hat{\mathit{\mu }}}_2$	${\hat{\mathit{\sigma }}}_1^2$	${\hat{\mathit{\sigma }}}_2^2$	${\hat{\mathit{\sigma }}}_{12}^2$
1	1	1	103.54875	74.422675	6.5059	20.4381	6.4311
1	1	−1	104.365	73.614	0.0913	0.3887	0.1642
1	−1	1	98.2915	65.443125	14.1937	9.0943	5.6163
1	−1	−1	100.17375	66.4308	0.2062	0.5586	0.2039
−1	1	1	105.541	75.9543	4.1763	7.1282	4.5582
−1	1	−1	105.69575	72.684575	0.0975	0.0758	0.0707
−1	−1	1	111.539	67.60635	10.3569	3.3087	2.4789
−1	−1	−1	110.53275	67.4559	0.1971	0.2383	0.1715

Using regression analysis, we derive the following predictive equations:

$$\begin{array}{cc}\hfill {\hat{\mu }}_1\left(\mathbf{x}\right)& =104.9-3.14x_1+2.38x_1{x}_2-0.35x_1{x}_3,\hfill \\ \hfill {\hat{\sigma }}_{11}^2\left(\mathbf{x}\right)& ={\left(1.613-0.254x_2+1.25x_3\right)}^2,\hfill \\ \\ \hfill {\hat{\mu }}_2\left(\mathbf{x}\right)& =70.45-0.349x_1+3.59x_2-0.449x_1{x}_3+0.614x_2{x}_3,\hfill \\ \hfill {\hat{\sigma }}_{22}^2\left(\mathbf{x}\right)& ={\left(1.729+0.417x_1+1.21x_3+0.263x_1{x}_3\right)}^2,\hfill \\ \\ \hfill {\hat{\sigma }}_{12}^2\left(\mathbf{x}\right)& =3.712+1.203x_1+3.555x_3+1.161x_1{x}_3.\hfill \end{array}$$

To derive the response surface models, we utilized the replicate data provided in the simulation dataset. For the mean (${\hat{\mu }}_i$) and variance (${\hat{\sigma }}_{ii}^2$) functions, we adopted the models established by Kazemzadeh et al. [12]. However, for the covariance structure (${\hat{\sigma }}_{12}^2$), which is central to our study, we explicitly calculated the sample covariance for each design point using the four replicates and subsequently fitted a second-order regression model to these computed values.

To ensure the reliability of the optimization framework, the derived regression models must meet specific statistical criteria established in quality engineering literature. Following the guidelines suggested by Joglekar and May [13] and Montgomery [14], we adopted a significance level of $p<0.05$ and a minimum coefficient of determination $R^2>0.80$ as the thresholds for model adequacy. An $R^2$ value above 0.80 generally indicates that the regression model explains a substantial proportion of the variability in the response, which is essential for accurate prediction in robust design. If the regression models fail to meet these thresholds, it implies insufficient statistical power or excessive experimental noise. In such cases, the number of experimental replicates should be increased to reduce the standard error.

In this study, we utilized the dataset from Pignatiello [1] ($n=4$). To validate whether this sample size was adequate under our proposed criteria, we evaluated the statistical significance of the derived models. As shown in Table 3, all regression models achieved high coefficients of determination ($R^2>0.91$) and were statistically significant ($p<0.05$). Specifically, the covariance model (${\hat{\sigma }}_{12}^2$), a critical component of our framework, demonstrated an $R^2$ of 0.9726 with a p-value of 0.001. These results confirm that the sample size of $n=4$ provided sufficient statistical power to capture the underlying covariance structure accurately.

Table 3. Statistical verification of the regression models ($n=4$).

Response	Regression Model	${\mathit{R}}^2$	Model p-Value
Mean $y_1$	${\hat{\mu }}_1\left(\mathbf{x}\right)$	0.9916	<0.001
Variance $y_1$	${\hat{\sigma }}_{11}^2\left(\mathbf{x}\right)$	0.9699	<0.001
Mean $y_2$	${\hat{\mu }}_2\left(\mathbf{x}\right)$	0.9866	0.004
Variance $y_2$	${\hat{\sigma }}_{22}^2\left(\mathbf{x}\right)$	0.9180	0.012
Covariance	${\hat{\sigma }}_{12}^2\left(\mathbf{x}\right)$	0.9726	0.001

5.2. Formulation of Objective Functions

In this dataset, the target vector is $\mathit{\tau }=(103,73)$. Consistent with the definition in Section 3.1 where cost coefficients are determined by the economic impact of deviations, we adopt the specific values established in the study by Pignatiello [1]:

$$\mathbf{C}=\left[\begin{array}{cc}1.0& 0.5\\ 0.5& 2.5\end{array}\right].$$

Based on these target values and cost values, the following functions are defined:

$${\hat{f}}_1\left(\mathbf{x}\right)=\sum _{i=1}^2\sum _{j=1}^2{c}_{ij}\left({\hat{\mu }}_i\left(\mathbf{x}\right)-{\tau }_i\right)\left({\hat{\mu }}_j\left(\mathbf{x}\right)-{\tau }_j\right),{\hat{f}}_2\left(\mathbf{x}\right)=\sum _{i=1}^2\sum _{j=1}^2{c}_{ij}{\hat{\sigma }}_{ij}^2\left(\mathbf{x}\right).$$

By evaluating these functions across the experimental design space, we obtain the following statistical ranges:

$$U_1=192.28,L_1=3.643,U_2=63.87,L_2=0.306.$$

To generate a Pareto frontier, we employ an alternative GC-based method with the following objective function:

$${\min }_{\mathbf{x}}{\left({\displaystyle \frac{|{\hat{f}}_1\left(\mathbf{x}\right)|}{192.28-3.643}}\right)}^{p_1}+{\left({\displaystyle \frac{|{\hat{f}}_2\left(\mathbf{x}\right)|}{63.87-0.306}}\right)}^{p_2}.$$

5.3. Pareto Frontier Analysis

Figure 5 shows a Pareto frontier generated by constraining the input variables to the range $-1\le x_i\le 1$, $i=1,2,3,$ and varying the parameters $p_1$ and $p_2$ from 0.05 to 3 in increments of 0.05. Optimization is initialized from starting point $(0,0,0)$.

Figure 5. Pareto-front solutions for various variance-risk weights, $\lambda =0,4,6$, (depicted by the three circles coming from the tangent lines) in the simulation experiment.

When $\lambda =0$, we find the minimum total risk as follows:

$$T=1.95304,{\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=0.60067,{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=1.35237,{\mathbf{x}}^{\ast }=(1,0.71001,-1).$$

For $\lambda =4$, similarly using the front, we extract minimum total risk as follows:

$$T=7.02391,{\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=1.52971,{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=1.09884,{\mathbf{x}}^{\ast }=(0.52104,0.70273,-1).$$

Finally, when $\lambda =6$, minimum total risk becomes

$$T=9.05211,{\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=2.52681,{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=0.93219,{\mathbf{x}}^{\ast }=(0.17905,0.72327,-1).$$

The three dashed lines in green, yellow, and red shown in Figure 5 represent the tangents to the Pareto frontier at each solution that minimizes the weighted sum $T={\hat{f}}_1\left(\mathbf{x}\right)+(1+\lambda ){\hat{f}}_2\left(\mathbf{x}\right),$ for $\lambda =0,4,6$, respectively.

5.4. Comparative Analysis with Sequential Strategies

A common practical approach to quality optimization is to adopt a sequential decision process. For instance, a field engineer may first minimize the bias term ${\hat{f}}_1\left(\mathbf{x}\right)$, and then reduce the variance term ${\hat{f}}_2\left(\mathbf{x}\right)$ among the solutions where bias is already minimized.

To evaluate this practical strategy, we compare it against our proposed approach under the setting $\lambda =0$, where bias and variance are treated as equally important. The optimal solution minimizing the total risk, $T={\hat{f}}_1\left(\mathbf{x}\right)+{\hat{f}}_2\left(\mathbf{x}\right),$ is indicated by the green dot in Figure 6, yielding the minimum value $T=1.95$.

Figure 6. Comparison of decision strategies between sequential decisions depicted by two stars and Pareto-based decisions depicted by the green dot for the total risk, $\lambda =0$, and by the three red circles for various trade-off ratios of bias loss to variance loss, ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right):{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ being 1:2, 1:1, and 2:1.

In contrast, when one adopts a sequential decision-making strategy, the outcomes are significantly suboptimal compared to the joint optimization approach. For instance, if one first minimizes the bias term ${\hat{f}}_1\left(\mathbf{x}\right)$ and then evaluates the variance term at that point, the results are ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=0.30$ and ${\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=14.72$, yielding a total risk of $T={\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)+{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=15.02$ for $\lambda =0$. This point is highlighted by the red star, located at the upper-left side in Figure 6. On the other hand, if one first minimizes the variance term ${\hat{f}}_2\left(\mathbf{x}\right)$ and then evaluates the bias at that point, the corresponding values are ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)=6.25$ and ${\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)=0.45$, resulting in a total risk of $T=6.70$. This point is highlighted by the yellow star, located at the lower-right side in Figure 6. Notice that the total risk, $T=1.95$, represented by the green dot in Figure 6, is far smaller than the values of $T=15.02$ and $T=6.70$ resulting from the two sequential decisions.

This comparison highlights that such a sequential optimization approach—whether prioritizing bias or variance—can lead to significantly worse solutions than those obtained by balancing both objectives jointly.

Furthermore, our approach enables flexible decision-making by allowing users to control the trade-offs between bias and variance via ratio settings. For example, we consider three target ratios: 1, 2, and 0.5. The corresponding solutions and function values are highlighted by the red circles in Figure 6 for direct comparison.

When the ratio ${\hat{f}}_1\left({\mathbf{x}}^{\ast }\right)/{\hat{f}}_2\left({\mathbf{x}}^{\ast }\right)$ is approximately 1, the optimal solution is ${\mathbf{x}}^{\ast }=[0.673,0.702,-1]$, which yields ${\hat{f}}_1={\hat{f}}_2=1.18$. When the ratio is approximately 2, the solution becomes ${\mathbf{x}}^{\ast }=[0.346,0.709,-1]$, resulting in ${\hat{f}}_1=2.01$ and ${\hat{f}}_2=1.01$. Lastly, for a ratio of 0.5, the solution is ${\mathbf{x}}^{\ast }=[0.956,0.710,-1]$, with ${\hat{f}}_1=0.66$ and ${\hat{f}}_2=1.33$.

Consequently, our method simultaneously incorporates both bias and variance through a weighted formulation, enabling a more balanced and efficient optimization process. This approach identifies solutions with lower total risk, clearly demonstrating its effectiveness in achieving a superior trade-off between bias and variance. Additionally, our method provides the flexibility to choose mixing ratios between bias and variance terms, allowing for optimization that can be customized according to specific needs.

6. Conclusions

In this study, we proposed a novel approach for optimizing interrelated quality attributes by incorporating variance into the expected loss function. Leveraging an alternative GC-based method, we successfully constructed Pareto front to visualize the trade-offs between conflicting objectives. The effectiveness and practical utility of the proposed method were demonstrated through both simplified examples and a simulation study.

Nonetheless, constructing a comprehensive Pareto frontier requires the exploration of multiple starting points, and identifying all possible nondominated solutions remains a significant challenge. Future research should focus on developing systematic strategies for selecting initial points and benchmarking the proposed approach against existing multi-objective optimization techniques. Furthermore, as the complexity of the expected loss and variance functions increases, the associated computational burden also grows. Investigating simplified formulations or alternative structures to enhance computational efficiency would be a valuable direction for future work.

In addition, it is important to explore how this method can be extended to more realistic scenarios where only a subset of input variables—such as two or three out of many—can be actively controlled. Adapting our approach to such constrained optimization settings would further broaden its applicability.

Overall, this study contributes to the field of quality optimization by offering a new perspective on managing trade-offs in complex systems. The proposed method has the potential for application across various industries, and future extensions could include higher-dimensional problems and the integration of machine learning techniques for dynamic and adaptive optimization.