Data-Driven Yield Estimation and Maximization Using Bayesian Optimization Under Uncertainty

Featured Application

This paper applies yield estimation and optimization with minimal experiments and stringent specifications in the field of quality control.

Abstract

In this paper, we propose a novel method which utilizes samples of measured product quality characteristics to efficiently estimate the probabilities of those quality characteristics being within the desired specifications and, consequently, the process yield. Specifically, when dealing with 1D Gaussian distributions, we formally prove that the proposed yield estimator asymptotically gives a lower Mean Squared Error compared to the best unbiased estimator. In order to enable maximization of yield, this novel estimator is incorporated into the framework of Bayesian Optimization which iteratively seeks controllable tool parameters under which the outgoing product yield is maximized. The newly proposed yield maximization method is demonstrated in an application involving high-fidelity simulations of a reactive ion etch chamber, a tool component commonly used in semiconductor manufacturing. The aim of these simulations was to rapidly and reliably determine tool parameters that maximize the probability of delivering desired plasma density characteristics under stochastic variations in chamber conditions. The novel yield estimation and optimization methods show superiority when the number of experimental observations is limited and the distributions of outgoing product characteristics can be approximated well by a Gaussian distribution.

1. Introduction

In advanced manufacturing, product specifications have become increasingly stringent, and relevant tools must be meticulously regulated to ensure that the outgoing product quality meets these specifications. However, even when a tool operates under identical control parameter settings, the quality of fabricated products usually still varies. As a result, some portion of the outgoing products may potentially be defective. This variability arises because tools are influenced by numerous environmental factors that cannot be observed or controlled. To minimize defective quality and maximize yields, it is crucial to efficiently address the uncertainties arising from these uncontrollable and unobservable factors.

Problems in yield assessment vary across industry sectors, including additive manufacturing [1], semiconductor fabrication [2,3], and the pharmaceutical industry [4,5]. A good yield is qualitatively defined as having quality characteristics of outgoing products that match the desired values with minimal variation. On a quantitative level, various formulations for defining yield have been considered, such as the sum of bias and variance [6,7] or the probability that the quality characteristics meet the specified constraints [8,9,10,11,12]. In cases where quality characteristics are multivariate, multi-criteria frameworks may be employed to define yield, as in [13,14]. In this manuscript, we define yield as the probability that the quality characteristic falls within the predefined specification limits (it should be noted that even when quality characteristics are multidimensional, yield defined as above is a scalar value), as discussed in [9,10].

A common approach is to treat yield as a Bernoulli parameter and estimate it via the maximum likelihood as in [9,11]. These studies regard the maximum likelihood estimation as a method using traditional Monte Carlo techniques. They propose the use of efficient sampling methods, such as importance sampling [11] or the cross-entropy method [9], each aiming for the highest possible accuracy using as few samples as possible. Nevertheless, improvement in sampling methods is only applicable when the probability density function of the quality characteristics is known. An alternative strategy for efficiently estimating yields is to assume a prior distribution for the parameter of a Bernoulli distribution and use samples to iteratively update that distribution. The study reported in [12] suggests employing a Beta distribution as the prior distribution of the Bernoulli parameter. While this approach can achieve high accuracy when provided with a precise initial yield estimation, it is often unreasonable to assume that such accurate yield estimations are available, especially without significant gathering of prior observations of the quality characteristics. Therefore, while leveraging prior knowledge can enhance accuracy with a limited number of samples, it is often preferable for estimation methods to operate without any assumptions about the true yield value, especially when one is developing a new process—i.e., when such knowledge is limited or does not exist.

To address this need, this article proposes estimating the yield by assuming and exploiting knowledge of the form of the underlying distribution of the product quality, with the standard assumption of independent and identically distributed (i.i.d.) samples. The main idea is that when one does have knowledge about the distribution form, one can first estimate its parameters and then subsequently calculate the corresponding probability of product quality falling within the desired specifications. Note that the estimator based on the Bernoulli distribution is the best unbiased estimator whose variance attains the Cramér–Rao bound [15]; however, it does not utilize any knowledge about the underlying distribution of the quality characteristic. Although the proposed estimator is generally biased, it can achieve a lower Mean Squared Error (MSE) due to the trade-off between bias and variance. In this article, we will formally prove that, under the assumptions that a scalar (univariate) quality output follows a Gaussian distribution and the desired quality specification area is an interval, the estimator based on the Gaussian distribution is asymptotically superior to the Bernoulli-based estimator in terms of the MSE metric.

Besides yield estimation from the product quality measurements, utilization of those measurements to adjust controllable parameters of the relevant manufacturing tool and thus maximize the outgoing product yield is a highly impacting, complementary problem. Such sampling-driven optimization of yield is a classical problem of optimization under uncertainty and has been examined in numerous manufacturing sectors, including pharmaceutical manufacturing [16,17], the petrochemical industry [18,19], circuit design [10,20], and material design [21], to name a few.

Regardless of the application area, the success metric for any sampling-driven yield optimization approach is the ability to determine controllable parameter settings that maximize the outgoing yield, while using as few samples of the outgoing product quality as possible. One highly promising approach for addressing this classical problem of optimization under uncertainty is Bayesian Optimization (BO). In the field of BO, various forms of uncertainties and objective functions have been considered, as discussed in [22,23,24,25,26,27]. Similar problem formulations to the one considered in this article are proposed in [28,29], where the authors proposed a method based on a criterion they named probability threshold robustness measure, with the underlying assumption that the environmental factors are observable. However, in many situations, the environmental factors are not observable, which necessitates a different approach; this need will be addressed in this paper. More specifically, to maximize the probability that the quality characteristic falls within the desired specifications, we propose incorporating the novel yield estimator introduced in this paper into a BO framework. Unlike previous approaches modeling empirical probabilities directly such as [27], our method employs Gaussian Process Regression (GPR) to model the parameters of the quality distribution. To address noise effects caused by the unobservable and uncontrollable environmental factors, a repeated experimentation strategy will be employed, similar to how BO was performed in [25,27].

In summary, the contributions of this article are as follows:

• The introduction of a yield estimator based on the estimates of the mean and variance parameters of the distribution characterizing the outgoing product quality which are then used to estimate the probability that the outgoing quality falls within a desired specification interval.
• A mathematical proof of asymptotic superiority in terms of MSE of the aforementioned yield estimator compared to the traditionally utilized yield estimator based on estimating the Bernoulli distribution parameter under the assumption that the outgoing product quality variables follow an uncorrelated Gaussian distribution.
• The introduction of a BO algorithm which uses the aforementioned mean and variance parameter-based estimator of yield to tune controllable process parameters in a way that maximizes the probability that the random variable characterizing the outgoing product quality falls within the desired specifications.

The newly proposed approaches to yield estimation and optimization were evaluated in simulations of a plasma chamber reactor, which is a highly complex system commonly used in semiconductor manufacturing. The aim was to use the novel methods to iteratively drive the controllable tool parameters towards a setting at which key plasma parameters could be kept within the desired specifications with maximum probability, given the inherent uncertainties present in the plasma chamber reactor.

The remainder of this article is structured as follows: In Section 2.1, we introduce a yield estimator based on estimating the mean and variance of the distribution of the outgoing product quality, and we introduce several theorems characterizing that estimator and proving its superiority over the Bernoulli-based yield estimator when the outgoing product quality characteristics follow an uncorrelated Gaussian distribution. In Section 2.2, we formalize the optimization problem for sampling-driven maximization of yield and propose two BO algorithms to solve it—one using the newly proposed yield estimator, and the other employing the traditional yield estimator. In Section 3.1, we comparatively evaluate the performance of the yield estimators. First, we examine problems where quality characteristics follow Gaussian distributions and empirically validate the theorems presented in Section 2.1. Second, we estimate the probability of plasma parameters falling within the specifications in the presence of random condition variations in a plasma reactor, with plasma behaviors being simulated using a commercial high-fidelity simulator. Similar to Section 3.1, in Section 3.2, we assess the effectiveness of the proposed BO-based yield maximization algorithms for scenarios where the outgoing quality characteristics follow Gaussian distributions, as well as for a scenario in which that is not the case. Finally, Section 4 concludes this article and offers suggestions for possible future work.

2. Materials and Methods

2.1. Estimation of Yields

• A. Problem formulation

Let $y\in \mathrm{R}$ be a random variable representing a quality characteristic and let $S=\left[a,b\right]$ denote the desired specification interval for $y$. The yield is defined as $p_{\mathrm{*}}=Pr\left(y\in \left[a,b\right]\right)$, i.e., as the probability that the quality characteristic falls within the desired specification limits. Suppose $y$ follows a Gaussian distribution with unknown mean ${\mu }_{\mathrm{*}}$ and variance ${\sigma }_{\mathrm{*}}^2$. In addition, let us observe $N$ independent samples $\{y_1,\dots ,y_N\}$ of the outgoing quality, where

$$\begin{array}{c}\begin{array}{r}{y}_n\stackrel{i.i.d}{\sim }\mathcal{N}\left({\mu }_{*},{\sigma }_{*}^2\right), n\in \left\{\mathrm{1,2},\dots ,N\right\}.\end{array} \end{array}$$

(1)

The objective is to accurately estimate the probability $p_{\mathrm{*}}$ via some estimator $\widehat{p}$ from the samples $\{y_1,\dots ,y_N\}$. The accuracy of the estimator will be assessed via its MSE, defined as

$$\begin{array}{c}\begin{array}{r}{\mathbb{E}}_{\left\{y_1,\dots ,y_N\right\}}\left[{\left(p_{*}-\widehat{p}\right)}^2\right].\end{array} \end{array}$$

(2)

Obviously, estimators with a smaller MSE are generally regarded as better estimators. Note that our formulation of yield differs from outage error probability discussed in estimation theory as in [30]. Outage error probability is the probability that the error of the estimate exceeds a certain threshold. In our formulation, yield itself is the estimation target expressed as a probability.

• B. Yield estimator based on estimating Bernoulli distribution parameter

By focusing on whether a sample $y_n$ satisfies the specifications or not, yield estimation can be formulated as the problem of estimating the parameter of Bernoulli distribution, which describes the yield as in [31]. The maximum likelihood is given by

$$\begin{array}{c}\begin{array}{r}{\widehat{p}}_1={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}{\mathbf{1}}_S\left(y_n\right),\end{array} \end{array}$$

(3)

where ${\mathbf{1}}_S$ is the indicator function defined as

$$\begin{array}{c}\begin{array}{r}{\mathbf{1}}_S(y)=\left\{\begin{array}{ll}1& y\in S\\ 0& y\notin S\end{array}\right..\end{array} \end{array}$$

(4)

The MSE of ${\widehat{p}}_1$ is easily derived as

$$\begin{array}{c}\begin{array}{r}\mathbb{E}\left[{\left({\widehat{p}}_1-p_{*}\right)}^2\right]={\displaystyle \frac{1}{N}}{p}_{*}\left(1-p_{*}\right).\end{array} \end{array}$$

(5)

Note that ${\widehat{p}}_1$ is the best unbiased estimator in the sense that it achieves the minimum Mean Squared Error.

• C. Yield estimator based on estimates of mean and variance parameters

Under the assumption that $y$ follows a Gaussian distribution, we can estimate the parameters of that distribution and subsequently estimate the required probability $p_{\mathrm{*}}$. Let us use the unbiased estimators for the mean and variance of Gaussian distributions, which are, respectively,

$$\begin{array}{cc}\hfill \widehat{\mu }& ={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}{y}_n\hfill \\ \hfill {\widehat{\sigma }}^2& ={\displaystyle \frac{1}{N-1}}{\displaystyle \sum _{n=1}^N}{\left(y_n-\widehat{\mu }\right)}^2.\hfill \end{array}$$

(6)

Using those estimators, we can construct the estimator of the probability $p_{\mathrm{*}}$ as follows:

$$\begin{array}{cc}\hfill {\widehat{p}}_2& =Pr\left(y\in \left[a,b\right]|y\sim \mathcal{N}\left(\widehat{\mu },{\widehat{\sigma }}^2\right)\right)\hfill \\ & ={\displaystyle \frac{1}{\sqrt{2\pi {\widehat{\sigma }}^2}}}{\int }_a^{b}exp\left(-{\displaystyle \frac{1}{2{\widehat{\sigma }}^2}}{\left(z-\widehat{\mu }\right)}^2\right) dz.\hfill \end{array}$$

(7)

Here, let us define a function $F\left(a,b,\mu ,V\right)$ as

$$\begin{array}{c}\begin{array}{r}F(a,b,\mu ,V)={\displaystyle \frac{1}{2}}\left(erf\left({\displaystyle \frac{b-\mu }{\sqrt{2V}}}\right)-erf\left({\displaystyle \frac{a-\mu }{\sqrt{2V}}}\right)\right),\end{array} \end{array}$$

(8)

where $\mathrm{e}\mathrm{r}\mathrm{f}\left(\cdot \right)$ is the error function defined by

$$\begin{array}{c}\begin{array}{r}erf(z)={\displaystyle \frac{2}{\sqrt{\pi }}}{\int }_0^z{e}^{-t^2}dt.\end{array} \end{array}$$

(9)

Then, the estimator ${\widehat{p}}_2$ can be expressed as

$$\begin{array}{c}\begin{array}{r}{\widehat{p}}_2=F\left(a,b,\widehat{\mu },{\widehat{\sigma }}^2\right).\end{array} \end{array}$$

(10)

In order to calculate the MSE of ${\widehat{p}}_2$, we use the fact that $\widehat{\mu }$ and ${\widehat{\sigma }}^2$ independently follow $\mathcal{N}\left({\mu }_{\mathrm{*}},{\displaystyle \frac{{\sigma }_{\mathrm{*}}^2}{N}}\right)$ and ${\displaystyle \frac{{\sigma }_{\mathrm{*}}^2}{N-1}}{\chi }_{N-1}^2$, respectively (e.g., see Chapter 6 of [32]). Then, the MSE of the estimator ${\widehat{p}}_2$ is calculated as follows:

$$\mathbb{E}[{\left({\widehat{p}}_2-p_{\mathrm{*}}\right)}^2]={\mathbb{E}}_{Y,Z}[F{\left(a,b,Z,Y\right)}^2]-2p_{\mathrm{*}}{\mathbb{E}}_{Y,Z}[F\left(a,b,Z,Y\right)]+p_{\mathrm{*}}^2$$

(11)

where distributions of $Y$ and $Z$ satisfy

$$\begin{array}{c}\begin{array}{rrr}& {\displaystyle \frac{\left(N-1\right)}{{\sigma }_{*}^2}}Y\sim {\chi }_{N-1}^2 ; Z\sim \mathcal{N}\left({\mu }_{*},{\displaystyle \frac{{\sigma }_{*}^2}{N}}\right).& \end{array} \end{array}$$

(12)

Although the MSE of the estimates is mainly discussed in the main text, confidence intervals of the estimates are also important in applications. The derivation of the confidence intervals is explained in Appendix A.

• D. Comparison of MSEs of two estimators

If $\left[a,b\right],{\mu }_{\mathrm{*}},{\sigma }_{\mathrm{*}}^2$ and $N$ are known, then the performance of the estimators becomes clear via calculation of ($5$) and (11). However, in reality, ${\mu }_{\mathrm{*}}$ and ${\sigma }_{\mathrm{*}}^2$ are never known. Here, we formulate and prove the theorem that shows the following: when the number of samples $N$ is sufficiently large, metric (11) is smaller than ($5$) for any $\left[a,b\right],{\mu }_{\mathrm{*}}$ and ${\sigma }_{\mathrm{*}}^2$.

First, let us apply the delta method (numerous statistics textbooks describe the delta method; e.g., one can refer to [33] or [34] for details of this method) to the MSE of the estimator ${\widehat{p}}_2$ in order to derive its asymptotic behavior in the following Lemma.

Lemma 1 (Asymptotic MSE of ${\widehat{p}}_2$).

Suppose that we are given the specification interval $\left[a,b\right]$ and $N$ independent samples $y_1,\dots ,y_N$, where $y_n\sim \mathcal{N}\left({\mu }_{*},{\sigma }_{*}^2\right), n\in \left\{\mathrm{1,2},\dots ,N\right\}$. Let $p_{*}$ denote the target probability that $y\in \left[a,b\right]$ if $y\sim \mathcal{N}\left({\mu }_{*},{\sigma }_{*}^2\right)$, i.e., let

$$\begin{array}{c}{p}_{*}=Pr\left(y\in \left[a,b\right]|y\sim \mathcal{N}\left({\mu }_{*},{\sigma }_{*}^2\right)\right).\end{array}$$

(13)

 Then, for the Gaussian parameter-based estimator ${\widehat{p}}_2$ defined in ($7$), the corresponding MSE of ${\widehat{p}}_2$ can be expressed as follows:

$$\begin{array}{c}\begin{array}{l}\begin{array}{ll}& \mathbb{E}\left[{\left({\widehat{p}}_2-p_{*}\right)}^2\right]\\ & ={\displaystyle \frac{1}{4\pi {\sigma }_{*}^{2}N}}{e}^{-{\displaystyle \frac{1}{{\sigma }_{*}^2}}{\left(a-{\mu }_{*}\right)}^2}\left(2{\sigma }_{*}^2+{\left(a-{\mu }_{*}\right)}^2\right)\\ & +{\displaystyle \frac{1}{4\pi {\sigma }_{*}^{2}N}}{e}^{-{\displaystyle \frac{1}{{\sigma }_{*}^2}}{\left(b-{\mu }_{*}\right)}^2}\left(2{\sigma }_{*}^2+{\left(b-{\mu }_{*}\right)}^2\right)\\ & -{\displaystyle \frac{1}{4\pi {\sigma }_{*}^{2}N}}{e}^{-{\displaystyle \frac{1}{2{\sigma }_{*}^2}}\left({\left(a-{\mu }_{*}\right)}^2+{\left(b-{\mu }_{*}\right)}^2\right)}\left(4{\sigma }_{*}^2+2\left(a-{\mu }_{*}\right)\left(b-{\mu }_{*}\right)\right)\\ & +\mathcal{O}\left({\displaystyle \frac{1}{N^2}}\right)\end{array}\end{array} \end{array}$$

(14)

For details of the derivation, please refer to Appendix B.

Theorem 1 (comparison of asymptotic MSEs).

Under the same premises as in Lemma 1, we have

$$\begin{array}{c}\begin{array}{r}\underset{N\to \infty }{lim}{\displaystyle \frac{\mathbb{E}\left[{\left({\widehat{p}}_1-p_{*}\right)}^2\right]-\mathbb{E}\left[{\left({\widehat{p}}_2-p_{*}\right)}^2\right]}{\frac{1}{N}}}\ge 0.\end{array} \end{array}$$

(15)

The strategy for the proof of this theorem is to observe the difference in the MSE portions associated with the factor $1/N$ as a function of parameters $a$ and $b$ and to compute extrema of this function. The details of the proof are enclosed in Appendix C.

The following corollary can be easily derived based on the inequality in ($15$) in Theorem 1.

Corollary of Theorem 1 (comparison of asymptotic MSEs).

Asymptotically, the MSE of ${\widehat{p}}_1$ is equal to or greater than that of ${\widehat{p}}_2$.

It should be noted that in the process of proving Theorem 1, the following proposition is derived.

Proposition 1 (the maximum difference between MSEs of ${\widehat{p}}_1$ and ${\widehat{p}}_2$ for a centered manufacturing process).

Consider the case where the true mean parameter corresponds to the center of the specification interval, i.e., $\left[a,b\right]=\left[{\mu }_{*}-w\sqrt{{\sigma }_{*}^2},{\mu }_{*}+w\sqrt{{\sigma }_{*}^2}\right]$. Under the same premises as Lemma 1 and Theorem 1, the difference in asymptotic MSEs between yield estimators ${\widehat{p}}_1$ and ${\widehat{p}}_2$ reaches its maximum when $\left[a,b\right]=\left[{\mu }_{*}-w_{*}\sqrt{{\sigma }_{*}^2},{\mu }_{*}+w_{*}\sqrt{{\sigma }_{*}^2}\right]$, where $w_{*}$ is the solution of

$$\begin{array}{c}\sqrt{2\pi }\left(1-2erf\left({\displaystyle \frac{w_{*}}{\sqrt{2}}}\right)\right)-\left(2w_{*}-2w_{*}^3\right)e^{{\displaystyle \frac{-w_{*}^2}{2}}}=0.\end{array}$$

(16)

 That solution can be determined to be $w_{*}\approx 0.48$ and, in that case, the maximum value for $\underset{N\to \infty }{lim}{\displaystyle \frac{\mathbb{E}\left[{\left({\widehat{p}}_1-p_{*}\right)}^2\right]-\mathbb{E}\left[{\left({\widehat{p}}_2-p_{*}\right)}^2\right]}{1/N}}$ can be found to be

$$\underset{N\to \infty }{lim}{\displaystyle \frac{\mathbb{E}\left[{\left({\widehat{p}}_1-p_{*}\right)}^2\right]-\mathbb{E}\left[{\left({\widehat{p}}_2-p_{*}\right)}^2\right]}{1/N}}\fallingdotseq 0.17.$$

• E. Yield estimator based on parameters of uncorrelated multivariate Gaussian distribution

Theorem 1 can be easily extended to the case of multivariate quality characterization, with quality variable vectors following an uncorrelated multivariate Gaussian distribution. Suppose that a vector of quality characteristics $\mathit{y}\in {\mathrm{R}}^D$ follows an uncorrelated multivariate Gaussian distribution as below,

$$\begin{array}{c}\begin{array}{r}\mathit{y}\sim \mathcal{N}\left({\mathit{\mu }}_{*},\mathit{diag}\left({\sigma }_{*,1}^2,{\sigma }_{*,2}^2,{\sigma }_{*,D}^2\right)\right).\end{array}\end{array}$$

(17)

The yield estimator based on the Bernoulli distribution parameter is the same as ($\mathbf{3}$), though the specification area $\mathit{S}$ is now defined as a set product over each dimension of the specification area, i.e.,

$$\begin{array}{c}\begin{array}{r}\mathit{S}=\left[a_1,b_1\right]\times \left[a_2,b_2\right]\times \dots \times \left[a_D,b_D\right].\end{array} \end{array}$$

(18)

On the other hand, the yield estimator based on the estimates of the mean and variance components of the quality variable vector $\mathit{y}$ can be expressed as

$$\begin{array}{c}\begin{array}{r}{\widehat{p}}_2=\mathit{Pr}\left(\mathit{y}\in \mathit{S}|\mathit{y}\sim \mathcal{N}\left(\widehat{\mathit{\mu }},\mathit{diag}\left({\widehat{\sigma }}_1^2,\dots ,{\widehat{\sigma }}_D^2\right)\right)\right),\end{array} \end{array}$$

(19)

where

$$\begin{array}{c}\begin{array}{cc}\hfill {\widehat{\mu }}_d& ={\displaystyle \frac{1}{N}}{\displaystyle \sum _{n=1}^N}{y}_{n,d}\hfill \\ \hfill {\widehat{\sigma }}_d^2& ={\displaystyle \frac{1}{N-1}}{\displaystyle \sum _{n=1}^N}{\left(y_{n,d}-{\widehat{\mu }}_d\right)}^2.\hfill \end{array}\end{array}$$

(20)

Extension of Theorem 1 for this case is given below.

Theorem 2 (comparison of asymptotic MSEs for yield estimators based on samples of quality characteristics drawn from a multivariate uncorrelated Gaussian distribution).

Let the desired specification area be $\mathit{S}=\left[a_1,b_1\right]\times \left[a_2,b_2\right]\times \dots \times \left[a_D,b_D\right]$ and let us observe $N$ independent vectorial samples $\{{\mathit{y}}_1,\dots ,{\mathit{y}}_N\}$ drawn from the multivariate uncorrelated Gaussian distribution $\mathcal{N}\left({\mathit{\mu }}_{*},\mathit{diag}\left({\sigma }_{*,1}^2,{\sigma }_{*,2}^2,{\sigma }_{*,D}^2\right)\right)$. Let $p_{*}$ denote the yield we are trying to estimate, i.e.,

$$\begin{array}{c}\begin{array}{r}{p}_{*}=Pr\left(\mathit{y}\in \mathit{S}| \mathit{y}\sim \mathcal{N}\left({\mathit{\mu }}_{*},\mathit{diag}\left({\sigma }_{*,1}^2,{\sigma }_{*,2}^2,{\sigma }_{*,D}^2\right)\right)\right)\end{array} .\end{array}$$

(21)

 Then, the Bernoulli parameter-based yield estimator ${\widehat{p}}_1$ defined by ($3$) and the Gaussian parameter-based estimator ${\widehat{p}}_2$ defined by ($19$) satisfy the following: 

$$\underset{N\to \infty }{lim}{\displaystyle \frac{\mathbb{E}\left[{\left({\widehat{p}}_1-p_{*}\right)}^2\right]-\mathbb{E}\left[{\left({\widehat{p}}_2-p_{*}\right)}^2\right]}{1/N}}\ge 0$$

The approach to the proof is to independently consider $D$ estimators for each dimension of the quality variable vector and to utilize the result of Theorem 1 for each of those dimensions.

• F. Discussion

The proposed methodology for estimating yield based on estimating the parameters of the underlying distribution from the available samples is applicable to any distribution form, provided that a parametric distribution can be assumed and that the probability over the specification region can be computed. Nevertheless, formally proving the benefits of exploiting the distribution form and estimating its parameters may be challenging for correlated multivariate Gaussian distributions or for non-Gaussian distributions. In particular, when the distribution is heavy-tailed, estimating its parameters is difficult, and parameter estimation accuracy strongly affects yield estimation. A future challenge is to improve and provide guarantees for yield estimation by leveraging theoretical guarantees on parameter estimation (e.g., [35]).

In this article, we indeed formally only analyze the case when the underlying distribution of the outgoing quality is of a multivariate uncorrelated Gaussian form. However, there are application areas, such as semiconductor manufacturing processes, where the random behavior of outgoing quality could significantly deviate from a Gaussian distribution form. In these cases, the superiority or inferiority of ${\widehat{p}}_1$ and ${\widehat{p}}_2$ depends on the sample size $N$, the actual underlying distribution form, and the specification interval.

Generally, ${\widehat{p}}_2$ tends to be superior to ${\widehat{p}}_1$ for small sample sizes $N$. This is because the available empirical clues for estimating yield probability from the Bernoulli distribution describing whether samples are falling into the specification area or not are insufficient when the sample size is small. In such cases, although not entirely accurate, assumption of Gaussianity helps ${\widehat{p}}_2$ compensate for the scarcity of samples. On the other hand, when $N$ is large, ${\widehat{p}}_1$ will be superior to ${\widehat{p}}_2$, in general, because ${\widehat{p}}_1$ is a consistent estimator [36], while ${\widehat{p}}_2$ is not. To select the preferred estimator, the sample size $N$ for which the superiority of ${\widehat{p}}_2$ to ${\widehat{p}}_1$ flips is an important threshold. Unfortunately, derivation of that threshold is practically impossible without knowledge of the true distribution. Nevertheless, based on Theorem 1, it is reasonable to claim that the more closely the distribution of the outgoing quality resembles a Gaussian distribution, the more pronouncedly ${\widehat{p}}_2$ is superior to ${\widehat{p}}_1$ for larger $N$. Note that if Gaussianity is strongly violated and the true distribution is multimodal, the relative performance of ${\widehat{p}}_2$ and ${\widehat{p}}_1$ is expected to be reversed for very small sample sizes (e.g., $N$ = 2).

From a practical viewpoint, selecting a subset of important dimensions to measure—rather than reducing the sample size $N$—can reduce measurement cost while maintaining estimation accuracy. Simple heuristics based on collinearity or formal subset selection frameworks [37] can be used. Combining these selection methods with the newly proposed yield estimator may improve the trade-off between measurement cost and estimation accuracy, particularly when the selected features are approximately regarded as uncorrelated.

2.2. Bayesian Optimization for Maximizing Yields

A classical problem in manufacturing is determining the settings of controllable parameters of the tool which maximize the outgoing yield. In this section, we propose a yield optimization method which incorporates the yield estimators described in the previous section into a Bayesian Optimization (BO) framework.

• A. Problem formulation

We consider the optimization problem

$$\begin{array}{c}\begin{array}{r}\underset{x\in \mathcal{X}}{\mathit{max}}\hspace{0.33em}Pr\left(\mathit{y}\in \mathit{S}|\mathit{y}\sim \mathcal{D}\left(\mathit{x}\right)\right),\end{array} \end{array}$$

(22)

where $\mathit{x}$ is a vector of control parameters, $\mathcal{D}\left(\mathit{x}\right)$ denotes the control parameter-dependent distribution of the vector of quality characteristics $\mathit{y}$, $\mathcal{X}$ is the domain of control parameters and $\mathit{S}$ is the desired multivariate specification area.

For a given candidate solution ${\mathit{x}}_i$ of the vector of control parameters, let us assume that we can observe $N$ independent samples ${\mathit{y}}_1,{\mathit{y}}_2,\dots {\mathit{y}}_N$, of the vector of quality characteristics, where ${\mathit{y}}_n\sim \mathcal{D}\left({\mathit{x}}_{\mathrm{i}}\right)$. The objective is to iteratively move towards a solution $\widehat{\mathit{x}}$ such that $\mathrm{P}\mathrm{r}\left(\mathit{y}\in \mathit{S}|\mathit{y}\sim \mathcal{D}\left(\widehat{\mathit{x}}\right)\right)$ becomes as high as possible and to do that in as few iterations as possible (i.e., using as few experiments as possible). Algorithm 1 shown below summarizes the proposed optimization procedure. In each iteration, it encompasses two major functionalities: (i) suggestion of the next iteration’s control parameter (step 3 in Algorithm 1), and (ii) estimation of the control parameter settings which maximize the outgoing yield based on observations available at that iteration (step 6 in Algorithm 1). Details of the implementation of this procedure will be discussed below.

Algorithm 1 Bayesian procedure for solving optimization problem

Input: An algorithm $\mathcal{A}$ initial observations ${\mathcal{H}}_{\mathrm{init}}$, the number of observations per iteration $N$, and maximum iteration $I_{\max }$. Output: Estimated best control parameters for each iteration $\widehat{X}=({\widehat{\mathit{x}}}_1,{\widehat{\mathit{x}}}_2,\dots ,{\widehat{\mathit{x}}}_{I_{\max }})$ 1: $\mathcal{H}\leftarrow {\mathcal{H}}_{\mathrm{init}}$ 2: for $i=1$ to $I_{\max }$ do 3:     Algorithm $\mathcal{A}\left(\mathcal{H}\right)$ suggests the next control parameter ${\mathit{x}}_i$. 4:     Observe the $N$ outputs for ${\mathit{x}}_i$, $\left({\mathit{x}}_i,\left({\mathit{y}}_i^{\left(1\right)},{\mathit{y}}_i^{\left(2\right)},\dots ,{\mathit{y}}_i^{\left(N\right)}\right)\right)$. 5:     Update the history of observations $\mathcal{H}\leftarrow \mathcal{H}\bigcup \left({\mathit{x}}_i,\left({\mathit{y}}_i^{\left(1\right)},{\mathit{y}}_i^{\left(2\right)},\dots ,{\mathit{y}}_i^{\left(N\right)}\right)\right)$ 6:     Algorithm $\mathcal{A}\left(\mathcal{H}\right)$ suggests the best control parameter, ${\widehat{\mathit{x}}}_i$. 7:  $\widehat{X}\leftarrow \widehat{X}\bigcup {\widehat{\mathit{x}}}_i$. 8: end for

• B. Bayesian Optimization using GPR modeling empirical yields

In this subsection, we describe the BO algorithm for solving ($22$) which relies on the GPR estimator of the yield, with yields being modeled from experiments using the empirical estimator of the Bernoulli distribution parameter, as described in Section 2.1-B. Let us define $f_{*}\left(\mathit{x}\right)=Pr\left(\mathit{y}\in \mathit{S}|\mathit{y}\sim \mathcal{D}\left(\mathit{x}\right)\right)$. Suppose that we have an available record of iterations of control parameter vectors $\mathit{x}$ and the corresponding observations of the outgoing quality vectors $\mathit{y}$ in the form

$$\begin{array}{c}\begin{array}{cc}\hfill \mathcal{H}& =\left\{\left({\mathit{x}}_1,\left({\mathit{y}}_1^{\left(1\right)},{\mathit{y}}_1^{\left(2\right)},\dots {\mathit{y}}_1^{\left(N_1\right)}\right)\right)\dots \right.\hfill \\ & \left({\mathit{x}}_i,\left({\mathit{y}}_i^{\left(1\right)},{\mathit{y}}_i^{\left(2\right)},\dots {\mathit{y}}_i^{\left(N_i\right)}\right)\right)\dots \hfill \\ & \left.\left({\mathit{x}}_I,\left({\mathit{y}}_I^{\left(1\right)},{\mathit{y}}_I^{\left(2\right)},\dots {\mathit{y}}_I^{\left(N_I\right)}\right)\right)\right\}.\hfill \end{array} \end{array}$$

(23)

For each setting of control parameters ${\mathit{x}}_i$, we can use quality observations ${\mathit{y}}_i^{\left(k\right)},k=1,2,\dots ,N_i$ to empirically estimate the yield under those control settings using the simple formulation

$$\begin{array}{c}\begin{array}{r}\bar{f}\left({\mathit{x}}_i\right)={\displaystyle \frac{1}{N_i}}{\displaystyle \sum _{n=1}^{N_i}}{\mathbf{1}}_{\mathit{S}}\left({\mathit{y}}_i^{\left(n\right)}\right).\end{array} \end{array}$$

(24)

Then, one can model the yield as it depends on the control parameter vector $\mathit{x}$ by training a GPR to model $f_{*}\left(\mathit{x}\right)$ using control vector–yield estimate pairs $\left({\mathit{x}}_1,\bar{f}\left({\mathit{x}}_1\right)\right),\left({\mathit{x}}_2,\bar{f}\left({\mathit{x}}_2\right)\right),\dots ,\left({\mathit{x}}_I,\bar{f}\left({\mathit{x}}_I\right)\right)$ obtained from the available data $\mathcal{H}$. Let ${\mathrm{GPR}}_f\left(\mathit{x}|\mathit{f}\right)$ denote the GPR function modeling the expected value and variance of the yield $f\left(\mathit{x}\right)$, given observations $\mathit{f}={\left[f\left({\mathit{x}}_1\right),f\left({\mathit{x}}_2\right),\dots f\left({\mathit{x}}_I\right)\right]}^{\top }$ of the yield at ${\mathit{x}}_i,i=\mathrm{1,2},\dots ,I$. Let us denote $\stackrel{\mathit{‾}}{\mathit{f}}={\left[\bar{f}\left({\mathit{x}}_1\right),\bar{f}\left({\mathit{x}}_2\right),\dots ,\bar{f}\left({\mathit{x}}_I\right)\right]}^{\mathrm{\top }}$. From the set of control parameters ${\mathcal{X}}_I=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I\}$, let us select the vector $\widehat{\mathit{x}}$ which maximizes the posterior expectation of the yield, as modeled by ${\mathrm{GPR}}_f\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{f}}\right),$ i.e.,

$$\begin{array}{c}\widehat{\mathit{x}}=\underset{{\mathcal{X}}_I}{\mathit{argmax}}{ \mathbb{E}}_f\left[f|f\sim {\mathrm{GPR}}_f\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{f}}\right)\right],\end{array}$$

(25)

and let ${\widehat{f}}_{\mathrm{*}}$ be the expected yield, as modeled by ${\mathrm{GPR}}_f\left(\widehat{\mathit{x}}|\stackrel{\mathit{‾}}{\mathit{f}}\right)$, i.e.,

$${\widehat{f}}_{*}={\mathbb{E}}_f\left[f|f\sim {\mathrm{GPR}}_f\left(\widehat{\mathit{x}}|\stackrel{\mathit{‾}}{\mathit{f}}\right)\right].$$

(26)

Now, as in any BO procedure, the vector of control parameters $\mathit{x}$ in the next iteration needs to be pursued via a trade-off between exploration and exploitation enabled by that iteration [38]. Following the approach of [16,21], one can use Expected Improvement (EI) as the acquisition function and obtain the next candidate vector of controllable parameters ${\mathit{x}}_{I+1}$ as

$$\begin{array}{c}\begin{array}{r}\begin{array}{rr}{\mathit{x}}_{I+1}& =\underset{\mathit{x}\in \mathcal{X}}{\mathit{argmax} }\mathrm{EI}\left(\mathit{x}\right)\end{array},\end{array} \end{array}$$

(27)

where

$$\begin{array}{c}\mathrm{E}\mathrm{I}\left(\mathit{x}\right)={\mathbb{E}}_f\left[max\left(f-{\widehat{f}}_{*},0\right)|f\sim {\mathrm{GPR}}_f\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{f}}\right)\right]\end{array}$$

(28)

is the acquisition function which, for a given vector of control parameters $\mathit{x}$, expresses the expected improvement in the yield relative to the current best expected yield ${\widehat{f}}_{\mathrm{*}}$.

Nevertheless, the EI acquisition function ($28$) does not account for the uncertainties of the observations $\stackrel{\mathit{‾}}{\mathit{f}}.$ In order to address this problem, following [39], in this paper we employ the so-called Noisy Expected Improvement (NEI) acquisition function which incorporates the uncertainty of observations $\stackrel{\mathit{‾}}{\mathit{f}}$ into the selection of the next candidate solution ${\mathit{x}}_{I+1}$, thus achieving general superiority over the EI acquisition function-based iterations described by (26). For completeness, NEI acquisition function-based selection of the next iteration for the candidate solution ${\mathit{x}}_{I+1}$ is given below.

Let us introduce EI acquisition function ${\alpha }_{\mathrm{EI}}\left(\mathit{x}|\mathit{f}\right)$ for any set of observations $\mathit{f}$ of yield for control parameter vectors ${\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I$, i.e.,

$$\begin{array}{c}{\alpha }_{\mathrm{EI}}\left(\mathit{x}|\mathit{f}\right)={\mathbb{E}}_f\left[max\left(f-{\widehat{f}}_{*},0\right)|f\sim {\mathrm{GPR}}_f\left(\mathit{x}|\mathit{f}\right)\right].\end{array}$$

(29)

Obviously, $\mathrm{EI}\left(\mathit{x}\right)={\alpha }_{\mathrm{EI}}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{f}}\right).$ Let us note that as modeled by ${\mathrm{GPR}}_f\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{f}}\right)$, the vector of yields for control vector ${\mathit{x}}_1,{\mathit{x}}_2\dots ,{\mathit{x}}_I$ can be modeled as a random vector ${{\mathit{f}}_{\mathbf{GPR}}=\left[f_{\mathrm{GPR}}\left({\mathit{x}}_1\right) f_{\mathrm{GPR}}\left({\mathit{x}}_2\right) \dots f_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\right]}^T$ which follows a vectorial normal distribution:

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{f}_{\mathrm{GPR}}\left({\mathit{x}}_1\right)\\ ⋮\\ f_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\end{array}\right]& \sim \mathcal{N}\left({\mathit{m}}^{f_{\mathrm{GPR}}},{\mathit{S}}^{f_{\mathrm{GPR}}}\right)\hfill \\ \hfill m_i^{f_{\mathrm{GPR}}}& =\mathbb{E}\left[{\mathrm{GPR}}_f\left({\mathit{x}}_{\mathit{i}}|\stackrel{\mathit{‾}}{\mathit{f}}\right))\right], i=\mathrm{1,2},\dots ,I\hfill \\ \hfill S_{ij}^{f_{\mathrm{GPR}}}& =\mathbb{C}\mathbb{o}\mathbb{v}\left[{\mathrm{GPR}}_f\left({\mathit{x}}_{\mathit{i}}|\stackrel{\mathit{‾}}{\mathit{f}}\right),{\mathrm{GPR}}_f\left({\mathit{x}}_{\mathit{j}}|\stackrel{\mathit{‾}}{\mathit{f}}\right))\right], i,j=\mathrm{1,2},\dots ,I\hfill \end{array}$$

(30)

Then, the NEI acquisition function can be defined as

$$\begin{array}{c}\mathrm{N}\mathrm{E}\mathrm{I}\left(\mathit{x}\right)={\mathbb{E}}_{{\mathit{f}}_{\mathbf{GPR}}}\left[{\alpha }_{\mathrm{EI}}\left(\mathit{x}|{\mathit{f}}_{\mathbf{GPR}}\right)\right]. \end{array}$$

(31)

Following [39], Monte Carlo sampling needs to be utilized in order to evaluate the expectation over ${\mathit{f}}_{\mathbf{GPR}}$ in the $\mathrm{NEI}\left(\mathit{x}\right)$ acquisition function described by ($31$). Finally, selection of the next candidate vector of controllable parameters is accomplished by

$$\begin{array}{c}\begin{array}{rr}{\mathit{x}}_{I+1}& =\underset{\mathit{x}\in \mathcal{X}}{\mathit{argmax}} \mathrm{NEI}\left(\mathit{x}\right).\end{array} \end{array}$$

(32)

• C. Bayesian Optimization using GPR modeling parameters of the distribution of quality characteristics

Let us consider the situation where for a given setting of control parameters $\mathit{x}$, the outgoing quality characteristics $y$ follow a normal distribution (For simplicity, in this section, we consider the case in which quality output $y$ is a scalar. Please note that following the discussions in Section 2.1-E, one can straightforwardly extend the approach described in this section to the multivariate case when the vector of quality characteristics $\mathit{y}$ follows an uncorrelated multivariate Gaussian distribution.) $\mathcal{N}\left({\mu }_{\mathrm{*}}\left(\mathit{x}\right),{\sigma }_{\mathrm{*}}^2\left(\mathit{x}\right)\right)$. In that case, a control parameter vector $\mathit{x}$ determines the mean and variance of the distribution $\mathcal{D}\left(\mathit{x}\right)$, and the yield for that control parameter setting can be written as

$$\begin{array}{c}\begin{array}{r}Pr\left(y\in S|y\sim \mathcal{N}\left({\mu }_{*}\left(\mathit{x}\right),{\sigma }_{*}^2\left(\mathit{x}\right)\right)\right).\end{array} \end{array}$$

(33)

Based on the results from Section 2.1, which show that the yield estimator relying on the estimation of Gaussian distribution parameters outperforms that relying on the empirical estimation of the Bernoulli distribution parameter, let us apply the GPR framework to model the mean and variance parameters of the distribution of the outgoing product quality with the available data $\mathcal{H}$, i.e.,

$$\begin{array}{c}\begin{array}{cc}\hfill \bar{\mu }\left({\mathit{x}}_i\right)& ={\displaystyle \frac{1}{N_i}}{\displaystyle \sum _{n=1}^{N_i}}{y}_i^{\left(n\right)}\hfill \\ \hfill {\bar{\sigma }}^2\left({\mathit{x}}_i\right)& ={\displaystyle \frac{1}{N_i-1}}{\displaystyle \sum _{n=1}^{N_i}}{\left(y_i^{\left(n\right)}-\bar{\mu }\left({\mathit{x}}_i\right)\right)}^2.\hfill \end{array} \end{array}$$

(34)

Following [40], variances ${\bar{\sigma }}^2\left({\mathit{x}}_i\right)$ are transformed using the natural $\mathrm{l}\mathrm{o}\mathrm{g}$ function in order to ensure that the range for the GPR model estimating those transformed variances comprises the entire set of real numbers. Hence, GPRs modeling the mean and variance parameters will be trained on the pairs of control vectors and estimated distribution parameters as below,

$$\begin{array}{c}({\mathit{x}}_1,\left(\bar{\mu }\left({\mathit{x}}_1\right),\mathit{log}\left[{\bar{\sigma }}^2\left({\mathit{x}}_1\right)\right]\right),\left({\mathit{x}}_2,\left(\bar{\mu }\left({\mathit{x}}_2\right),\mathit{log}\left[{\bar{\sigma }}^2\left({\mathit{x}}_2\right)\right]\right)\right),\dots ,\hfill \\ \left({\mathit{x}}_I,\left(\bar{\mu }\left({\mathit{x}}_I\right),\mathit{log}\left[{\bar{\sigma }}^2\left({\mathit{x}}_I\right)\right]\right)\right)).\hfill \end{array}$$

(35)

The variance of the estimator $\bar{\mu }\left({\mathit{x}}_i\right)$ can be expressed and approximated as

$$\begin{array}{c}\begin{array}{c}\begin{array}{cc}\hfill \mathbb{VAR}\left[\bar{\mu }\left({\mathit{x}}_i\right)\right]& =\mathbb{VAR}\left[{\displaystyle \frac{1}{N_i}}{\displaystyle \sum _{n=1}^{N_i}}{y}_i^{\left(n\right)}\right]\hfill \\ & ={\displaystyle \frac{1}{N_i}}{\sigma }_{*}^2\left({\mathit{x}}_i\right)\fallingdotseq {\displaystyle \frac{1}{N_i}}{\bar{\sigma }}^2\left({\mathit{x}}_i\right),\hfill \end{array}\end{array} \end{array}$$

(36)

while the variance of the estimator $log\left[{\bar{\sigma }}^2\left({\mathit{x}}_i\right)\right]$ can be approximated using the delta method as follows [34]:

$$\begin{array}{c}\begin{array}{r}\mathbb{VAR}\left[log\left[{\bar{\sigma }}^2\left({\mathit{x}}_i\right)\right]\right]\fallingdotseq {\displaystyle \frac{\mathbb{V}\mathbb{A}\mathbb{R}\left[{\bar{\sigma }}^2\left({\mathit{x}}_i\right)\right]}{{\left(\mathbb{E}\left[{\bar{\sigma }}^2\left({\mathit{x}}_i\right)\right]\right)}^2}}={\displaystyle \frac{2}{N_i-1}}.\end{array} \end{array}$$

(37)

Let ${\mathrm{GPR}}_{\mu }\left(\mathit{x}|\mathit{\mu }\right)$ denote the GPR function modeling the mean parameter $\mu \left(\mathit{x}\right)$, given the observations of the estimated parameters $\mathit{\mu }=\left[\mu \left({\mathit{x}}_1\right),\mu \left({\mathit{x}}_2\right),\dots ,\mu \left({\mathit{x}}_I\right)\right]$. Similarly, let ${\mathrm{GPR}}_{log\left[{\sigma }^2\right]}\left(\mathit{x}|\mathit{\theta }\right)$ be the GPR function modeling the logarithm of the true variance parameter $\theta \left(\mathit{x}\right)=log\left[{\sigma }^2\left(\mathit{x}\right)\right]$, given the observations of the estimated parameters $\mathit{\theta }={\left[log\left[{\sigma }^2\left({\mathit{x}}_1\right)\right] log\left[{\sigma }^2\left({\mathit{x}}_2\right)\right] \dots log\left[{\sigma }^2\left({\mathit{x}}_I\right)\right]\right]}^{\top }$.

Then, let us denote with $\mathcal{F}\left(\mathit{x}|\mathit{\mu },\mathit{\theta }\right)$ the distribution of the estimated yield, which can be expressed as

$$\begin{array}{r}\begin{array}{cc}\hfill \mathcal{F}\left(\mathit{x}|\mathit{\mu },\mathit{\theta }\right)& =Pr\left(Y\in S|Y\sim \mathcal{N}\left(\tilde{\mu },{\tilde{\sigma }}^2\right)\right)\hfill \\ \hfill \tilde{\mu }& \sim {\mathrm{GPR}}_{\mu }\left(\mathit{x}|\mathit{\mu }\right)\hfill \\ \hfill log\left[{\tilde{\sigma }}^2\right]& \sim {\mathrm{GPR}}_{log\left[{\sigma }^2\right]}\left(\mathit{x}|\mathit{\theta }\right)),\hfill \end{array}\end{array}$$

(38)

and let $\stackrel{\mathit{‾}}{\mathit{\mu }}={\left[\bar{\mu }\left({\mathit{x}}_1\right),\bar{\mu }\left({\mathit{x}}_2\right),\dots ,\bar{\mu }\left({\mathit{x}}_I\right)\right]}^{\mathrm{\top }}$ and $\stackrel{\mathit{‾}}{\mathit{\theta }}={\left[log\left[{\bar{\sigma }}^2\left({\mathit{x}}_1\right)\right],log\left[{\bar{\sigma }}^2\left({\mathit{x}}_2\right)\right],\dots ,log\left[{\bar{\sigma }}^2\left({\mathit{x}}_I\right)\right]\right]}^{\mathrm{\top }}$. Now, from the set of control parameters ${\mathcal{X}}_I=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I\}$, let us select vector $\widehat{\mathit{x}}$ which maximizes the posterior expectation of the yield modeled by $\mathcal{F}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)$, i.e.,

$$\begin{array}{c}\begin{array}{r}\widehat{\mathit{x}}=\underset{{\mathit{x}\in \mathcal{X}}_I}{\mathit{argmax}}{\mathbb{E}}_f\left[f|f\sim \mathcal{F}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)\right],\end{array} \end{array}$$

(39)

and let ${\widehat{f}}_{\mathrm{*}\mathrm{*}}$ be the expected value as per model $\mathcal{F}\left(\widehat{\mathit{x}}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right),$ i.e.,

$$\begin{array}{c}\begin{array}{r}{\widehat{f}}_{**}={\mathbb{E}}_f\left[f|f\sim \mathcal{F}\left(\widehat{\mathit{x}}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)\right].\end{array}\end{array}$$

(40)

The next iteration ${\mathit{x}}_{I+1}$ of the candidate solution for the vector of control parameters is determined as

$$\begin{array}{c}\begin{array}{rr}{\mathit{x}}_{I+1}& =\underset{\mathit{x}\in \mathcal{X}}{\mathit{argmax}}{\mathrm{EI}}_{\mathcal{F}}\left(\mathit{x}\right),\end{array} \end{array}$$

(41)

where

$$\begin{array}{c}\begin{array}{rr}{\mathrm{EI}}_{\mathcal{F}}\left(\mathit{x}\right)& ={\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)\end{array} \end{array}$$

(42)

is the acquisition function which, for a given vector of control parameters $\mathit{x},$ expresses the expected improvement in the yield relative to the inferred best expected yield ${\widehat{f}}_{\mathrm{*}\mathrm{*}}$.

A major challenge associated with the calculation of ($39$) and ($41$) is that $\mathcal{F}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)$ is not an analytically tractable Gaussian distribution. Therefore, following [40], Monte Carlo sampling from the distribution $\mathcal{F}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)$ is proposed for evaluating the expected value of the yield in ($39$) and the acquisition function ${\mathrm{EI}}_{\mathcal{F}}\left(\mathit{x}\right)$ for any vector $\mathit{x}$ of the control parameters.

Let us now introduce ${\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|\mathit{\mu },\mathit{\theta }\right)$ for any set of observed estimated parameters $\mathit{\mu }$ and $\mathit{\theta }$ for control parameter vectors ${\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I$, i.e.,

$$\begin{array}{c}\begin{array}{rr}{\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|\mathit{\mu },\mathit{\theta }\right)& ={\mathbb{E}}_f\left[max\left(f-{\widehat{f}}_{**},0\right)|f\sim \mathcal{F}\left(\mathit{x}|\mathit{\mu },\mathit{\theta }\right)\right]\end{array}. \end{array}$$

(43)

Obviously, ${\mathrm{EI}}_{\mathcal{F}}\left(\mathit{x}\right)={\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|\stackrel{\mathit{‾}}{\mathit{\mu }},\stackrel{\mathit{‾}}{\mathit{\theta }}\right)$. It can be noted that the mean parameters for control vector ${\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I$ can be modeled as random vector ${\left[{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_1\right),{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_2\right),\dots ,{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\right]}^T$ as below:

$$\begin{array}{c}\begin{array}{c}\begin{array}{cc}\hfill \left[\begin{array}{c}{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_1\right)\\ ⋮\\ {\mu }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\end{array}\right]& \sim \mathcal{N}\left({\mathit{m}}^{{\mu }_{\mathrm{GPR}}},{\mathit{S}}^{{\mu }_{\mathrm{GPR}}}\right)\hfill \\ \hfill m_i^{{\mu }_{\mathrm{GPR}}}& =\mathbb{E}\left[{\mathrm{GPR}}_{\mu }\left({\mathit{x}}_i|\stackrel{\mathit{‾}}{\mathit{\mu }}\right))\right],i=\mathrm{1,2},\dots ,I\hfill \\ \hfill S_{ij}^{{\mu }_{\mathrm{GPR}}}& =\mathbb{C}\mathbb{o}\mathbb{v}\left[{\mathrm{GPR}}_{\mu }\left({\mathit{x}}_i|\stackrel{\mathit{‾}}{\mathit{\mu }}\right)),{\mathrm{GPR}}_{\mu }\left({\mathit{x}}_j|\stackrel{\mathit{‾}}{\mathit{\mu }}\right))\right]\hfill \end{array}\end{array} \end{array}$$

$$\begin{array}{c}i,j=\mathrm{1,2},\dots ,I \end{array}$$

(44)

Similarly, the logarithm of variance parameters for control vector ${\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_I$ can be modeled as random vector ${\left[{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_1\right),{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_2\right),\dots {\theta }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\right]}^T$ following the distribution

$$\begin{array}{c}\begin{array}{c}\begin{array}{cc}\hfill \left[\begin{array}{c}{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_1\right)\\ ⋮\\ {\theta }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\end{array}\right]& \sim \mathcal{N}\left({\mathit{m}}^{{\theta }_{GPR}},{\mathit{S}}^{{\theta }_{GPR}}\right)\hfill \\ \hfill m_i^{{\theta }_{GPR}}& =\mathbb{E}\left[{\mathrm{GPR}}_{log\left[{\sigma }^2\right]}\left({\mathit{x}}_i|\stackrel{\mathit{‾}}{\mathit{\theta }}\right))\right],i=\mathrm{1,2},\dots ,I\hfill \\ \hfill S_{ij}^{{\theta }_{GPR}}& =\mathbb{Cov}\left[{\mathrm{GPR}}_{log\left[{\sigma }^2\right]}\left({\mathit{x}}_i|\stackrel{\mathit{‾}}{\mathit{\theta }}\right),{\mathrm{GPR}}_{log\left[{\sigma }^2\right]}\left({\mathit{x}}_j|\stackrel{\mathit{‾}}{\mathit{\theta }}\right))\right],\hfill \end{array}\end{array} \end{array}$$

$$\begin{array}{c}i,j=\mathrm{1,2},\dots ,I. \end{array}$$

(45)

If we denote ${\mathit{\mu }}_{\mathrm{GPR}}={\left[{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_1\right),{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_2\right),\dots ,{\mu }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\right]}^{\mathrm{\top }}$ and ${\mathit{\theta }}_{\mathrm{GPR}}={\left[{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_1\right),{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_2\right),\dots ,{\theta }_{\mathrm{GPR}}\left({\mathit{x}}_I\right)\right]}^{\mathrm{\top }}$, then the NEI acquisition function is defined as

$$\begin{array}{c}\begin{array}{rr}{\mathrm{NEI}}_{\mathcal{F}}\left(\mathit{x}\right)& ={\mathbb{E}}_{{\mathit{\mu }}_{\mathrm{GPR}},{\mathit{\theta }}_{\mathrm{GPR}}}\left[{\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|{\mathit{\mu }}_{\mathrm{GPR}},{\mathit{\theta }}_{\mathrm{GPR}}\right)\right]\end{array}.\end{array}$$

(46)

Finally, the selection of ${\mathit{x}}_{I+1}$ is accomplished as

$$\begin{array}{c}\begin{array}{rr}{\mathit{x}}_{I+1}& =\underset{\mathit{x}\in \mathcal{X}}{\mathit{argmax}}{\mathrm{NEI}}_{\mathcal{F}}\left(\mathit{x}\right).\end{array} \end{array}$$

(47)

In order to calculate ${\mathrm{NEI}}_{\mathcal{F}}\left(\mathit{x}\right)$, Monte Carlo sampling is required for evaluating the expectation of ${\alpha }_{{\mathrm{EI}}_{\mathcal{F}}}\left(\mathit{x}|{\mathit{\mu }}_{\mathrm{GPR}},{\mathit{\theta }}_{\mathrm{GPR}}\right)$ over ${\mathit{\mu }}_{\mathrm{GPR}}$ and ${\mathit{\theta }}_{\mathrm{GPR}}$, which means that the calculation cost for NEI becomes higher than that for EI. Nevertheless, the calculation of Monte Carlo sampling is apparently parallelizable in nature. Therefore, multiple cores are able to compensate for the burden of increase in calculations.

• D. Discussion

In this section, we discuss the suitability of the BO algorithms described in Section 2.2-B,C. For problems where control parameters determine means and variances of Gaussian distributions, we deduce from Theorem 1 that applying GPRs to the parameters of Gaussian distributions is better than applying the GPR concept to the empirical, Bernoulli distribution-based estimators of yield. It is difficult to discuss the suitability and performance of the proposed approaches for the cases where the distributions of the outgoing quality are not Gaussian. Nevertheless, drawing an analogy with the discussion in Section 2.1-F, when the sample size $N$ and the number of iterations is large, applying GPR to empirical probabilities is preferable due to the consistency of the corresponding estimator. On the other hand, when the sample size $N$ and the number of iterations are small, applying GPRs to the parameters of the Gaussian distributions may be preferable, as the assumption of the Gaussian form of the outgoing quality characteristics compensates for the scarcity of samples.

Furthermore, when the specification area is narrow, applying GPRs to parameters of Gaussian distributions is expected to outperform applying the GPR concept to Bernoulli distribution-based estimators of yield, especially when the number of iterations is small. Essentially, if quality characteristics corresponding to a certain set of control parameters fall near the boundaries of the specification areas, these control parameters should be regarded as somewhat promising, even if the empirical probability that the quality characteristics satisfy the specification is zero. Applying GPRs to distribution parameters incorporates this intuitive insight. On the other hand, when GPR is applied to empirical probabilities, the algorithm cannot exploit the proximity of the quality characteristics to the specification area.

Finally, we comment on computational cost. Applying GPRs to the parameters of Gaussian distributions requires additional computation because Monte Carlo sampling is needed (see Section 2.2-C). The cost of these Monte Carlo calculations grows roughly with the number of samples and with the dimensionality of the characteristic space; so, high-dimensional problems or large sample sizes can become computationally demanding. When a single workstation is insufficient, the embarrassingly parallel nature of Monte Carlo sampling allows for straightforward distribution of independent samples (or independent control parameter evaluations) across multiple workstations.

3. Results

3.1. Experiments with Various Methods for Estimating Yields

In this section, we evaluate the performance of the yield estimators discussed in Section 2.1. In Section 3.1-A, we analyze the performance of yield estimators when the distribution of the outgoing quality is exactly Gaussian, while in Section 3.1-B, we consider the distribution of the outgoing quality not exactly following the Gaussian distribution form.

• A. Estimation of yield when quality characteristic follows Gaussian distribution

Let us assume that a quality characteristic follows the normal distribution $\mathcal{N}\left(0,1\right)$ and that the desired interval for that characteristic is $\left[-w^{\mathrm{*}},+w^{\mathrm{*}}\right]$, where $w^{\mathrm{*}}$ is the solution of

$$\begin{array}{c}\sqrt{2\pi }\left(1-2erf\left({\displaystyle \frac{w_{*}}{\sqrt{2}}}\right)\right)-\left(2w_{*}-2w_{*}^3\right)e^{{\displaystyle \frac{-w_{*}^2}{2}}}=0,\end{array}$$

(48)

as described in Proposition 1. This situation is illustrated in Figure 1. MSEs for the classical yield estimator based on Bernoulli distribution ${\widehat{p}}_1$ and the newly proposed yield estimator ${\widehat{p}}_2$ for different sample sizes $N$ are shown in Figure 2 and

Table 1. Empirical squared errors of estimators (Mean ± STD).

N	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_1$	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_2$
2	1.15 × ${10}^{-1}$ ± 1.25 × ${10}^{-1}$	6.93 × ${10}^{-2}$ ± 1.01 × ${10}^{-1}$
4	5.68 × ${10}^{-2}$ ± 7.27 × ${10}^{-2}$	2.39 × ${10}^{-2}$ ± 4.87 × ${10}^{-2}$
8	2.89 × ${10}^{-2}$ ± 3.90 × ${10}^{-2}$	9.30 × ${10}^{-3}$ ± 1.90 × ${10}^{-2}$
16	1.43 × ${10}^{-2}$ ± 1.99 × ${10}^{-2}$	4.16 × ${10}^{-3}$ ± 7.91 × ${10}^{-3}$
32	7.01 × ${10}^{-3}$ ± 9.86 × ${10}^{-3}$	1.92 × ${10}^{-3}$ ± 3.23 × ${10}^{-3}$
64	3.57 × ${10}^{-3}$ ± 5.05 × ${10}^{-3}$	9.44 × ${10}^{-4}$ ± 1.48 × ${10}^{-3}$
128	1.83 × ${10}^{-3}$ ± 2.57 × ${10}^{-3}$	4.54 × ${10}^{-4}$ ± 6.64 × ${10}^{-4}$

. We see that the theoretical expressions for relevant MSEs, namely ($5$) and (11), coincide with the empirical results.

The advantage of the newly proposed yield estimator is statistically supported by one-sided hypothesis testing using the bootstrap method, as shown in

Table 2. One-sided hypothesis testing using the bootstrap method (number of resamples = ${10}^4$).

N	One-Sided Alternative Hypothesis	p-Value	Effect Size (Difference in Average)	95% CI of Difference in Squared Error (Percentiles from 2.5 to 97.5)
2	Squared error of ${\widehat{p}}_2$ is less than that of ${\widehat{p}}_1$.	<${10}^{-16}$	−0.0460	[−0.04912, −0.04284]
128	Squared error of ${\widehat{p}}_2$ is less than that of ${\widehat{p}}_1$.	<${10}^{-16}$	−0.00138	[−0.00143, −0.001327]

, for both $N=2$ and $N=128$. In both cases, sufficiently small p-values are obtained. In addition, the portion of MSE related to the $1/N$ factor, as characterized by ($14$), predicts the MSE of ${\widehat{p}}_2$ when $N$ is large, and the MSE of ${\widehat{p}}_2$ is smaller than that of ${\widehat{p}}_1$, as stipulated by Theorem 1.

• B. Estimation of Yield Expressed as Probabilities of Electron Densities and Temperatures Falling Within Specifications

In this subsection, we evaluate the performance of the classical yield estimator ${\widehat{p}}_1$ and the newly proposed yield estimator ${\widehat{p}}_2$ when the distribution of the outgoing process quality is not exactly Gaussian. This study will be performed through examination of the problem of ensuring plasma stability in an Inductively Coupled Plasma (ICP) chamber, which is a system widely utilized in semiconductor manufacturing.

ICP chambers use coils to generate a strong electric field within a vacuum chamber and thus generate plasmas. In the simulations considered in this paper, Ar and ${\mathrm{O}}_2$ gases are ionized to create plasmas, where Ar ions mainly provide mechanical energy for sputtering, while ${\mathrm{O}}_2$ ions enhance chemical reactions. Maintaining key plasma parameters within desired specification limits is crucial for ascertaining the desired quality of outgoing wafers, as those parameters have significant effects on the surface modifications of wafers. Specifically, following [41], this article focuses on the electron density and temperature among plasma parameters.

• (1) Configuration of simulations:

COMSOL software package (version: 6.2) is utilized to perform simulations, with the model details available in [42]. A schematic sketch of the chamber considered in this paper is shown in Figure 3. It is assumed that one is able to set powers of the four coils to distinct values and thus generate an electric field within the chamber. From the gas inlet, Ar and ${\mathrm{O}}_2$ gases flow into the chamber and leave the chamber via the outlet. Random variations in the Ar/${\mathrm{O}}_2$ concentration ratios occur due to the effects of residual gases inside the chamber, while random fluctuations of pressures attributes arise from uncertainties in the control actions of pressure controllers. Given inputs, COMSOL simulations provide the stationary spatial fields of electron densities and temperatures across the chamber, based on which measurements of electron densities and temperatures are assumed to be obtained via a plasma absorption probe installed at a specific location in the chamber. A plasma absorption probe is a device capable of measuring local absolute electron density [43] and can be used to estimate electron temperature [44]. The radius of the dielectric tube used in such probes is typically a few millimeters [43,45]. Therefore, for this simulation, probe measurements were modeled as the spatial average of the plasma parameters over an area of 1 cm × 1 cm, with the aim of keeping these measurements within some a priori specified intervals. The objective of the numerical experiments in this section is to accurately estimate the probability that the measurements of electron densities and temperatures will fall within the desired specification intervals.

• (2) Probability of electron density falling within specification limits:

In this subsection, we compare the performance of the classical yield estimator ${\widehat{p}}_1$ and the newly proposed yield estimator ${\widehat{p}}_2$ in terms of their performance in estimating the probability that the sensed electron density falls within the desired specification interval. Stochastic variations in chamber conditions are simulated by randomly disturbing inputs of the simulator, and an empirical distribution of electron densities is acquired by running simulation replications with these disturbed inputs. The settings used in the simulations are summarized in

Table 3. Input parameters of simulations for constructing true distribution of electron density.

Item	Type	Value/Distribution
Power (Pw1-4)	Fixed	100 [W]
Ratio of O2 at inlet (xO2)	Varied	$\mathcal{N}\left(0.6,{\left(0.1\right)}^2\right)$
Pressure (p0)	Varied	$\mathcal{N}\left(0.04,{\left(0.01\right)}^2\right)$ [torr]
Pressure at outlet (outlet_p)	Fixed	0 [torr]

. In order to ensure that the parameters stay within the valid range, we clip the values of xO2 to fall within $\left[0,1\right]$ and ensure that the values of p0 are positive after sampling from the Gaussian distribution. A total of 1000 simulation replications were run and, from the valid simulation runs, the spatial average of electron density was calculated for the area assumed to be occupied by the probe.

Figure 4 shows the histogram of electron densities obtained using the above-described simulations, with the desired specification interval superimposed over the true distribution. While the distribution illustrated in Figure 4 appears to be similar to a Gaussian distribution in terms of its mono-modality, the Shapiro–Wilk test [46] implies that it does not follow a Gaussian form ($p=2.64\times {10}^{-12}$). This is consistent with the fact that, in semiconductor manufacturing, nonlinear effects and substantial noise often lead to non-Gaussian distributions of quality characteristics. Nevertheless, we will still attempt to estimate the probability that electron density falls within the desired specification interval by estimating Gaussian distribution parameters from $N$ independent, identically distributed samples, as in ($7$).

In Figure 5 and

Table 4. Empirical squared errors of estimators (Mean ± STD).

N	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_1$	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_2$
2	9.26 × ${10}^{-2}$ ± 1.20 × ${10}^{-1}$	5.58 × ${10}^{-2}$ ± 1.06 × ${10}^{-1}$
4	4.66 × ${10}^{-2}$ ± 6.21 × ${10}^{-2}$	2.27 × ${10}^{-2}$ ± 4.19 × ${10}^{-2}$
8	2.40 × ${10}^{-2}$ ± 3.34 × ${10}^{-2}$	1.12 × ${10}^{-2}$ ± 1.72 × ${10}^{-2}$
16	1.18 × ${10}^{-2}$ ± 1.67 × ${10}^{-2}$	6.16 × ${10}^{-3}$ ± 8.32 × ${10}^{-3}$
32	5.81 × ${10}^{-3}$ ± 8.36 × ${10}^{-3}$	4.18 × ${10}^{-3}$ ± 4.73 × ${10}^{-3}$
64	2.86 × ${10}^{-3}$ ± 3.99 × ${10}^{-3}$	3.48 × ${10}^{-3}$ ± 3.24 × ${10}^{-3}$
128	1.43 × ${10}^{-3}$ ± 2.00 × ${10}^{-3}$	3.19 × ${10}^{-3}$ ± 2.32 × ${10}^{-3}$

, the MSEs of yield estimators ${\widehat{p}}_1$ and ${\widehat{p}}_2$ for the above-mentioned probability are shown. As discussed in Section 2.1-F, the MSE of the proposed estimator ${\widehat{p}}_2$ is smaller than that of the classical estimator ${\widehat{p}}_1$ until a certain threshold value of the sample size $N$ is reached, beyond which the superiority flips and the MSE of ${\widehat{p}}_1$ becomes smaller. This finding is supported by one-sided hypothesis tests using the bootstrap method for $N=2$ and $N=128,$ as in

Table 5. One-sided hypothesis testing using the bootstrap method (number of resamples = ${10}^4$).

N	One-Sided Alternative Hypothesis	p-Value	Effect Size (Difference in Average)	95% CI of Difference in Squared Error (Percentiles from 2.5 to 97.5)
2	Squared error of ${\widehat{p}}_2$ is less than that of ${\widehat{p}}_1$.	<${ 10}^{-16}$	−0.0369	[−0.0400, −0.0337]
128	Squared error of ${\widehat{p}}_1$ is less than that of ${\widehat{ p}}_2$.	<${ 10}^{-16}$	−0.00176	[−0.00183, −0.00170]

. For $N=2$, the test indicates that the empirical squared errors of ${\widehat{p}}_2$ are significantly smaller than those of ${\widehat{p}}_1$. Conversely, for $N=128$, the test indicates that the empirical square errors of ${\widehat{p}}_1$ are significantly smaller than those of ${\widehat{p}}_2$.

• (3) Probability of electron densities and temperatures falling within specifications:

This subsection evaluates the performance of the classical estimator ${\widehat{p}}_1$ and the newly proposed estimator ${\widehat{p}}_2$ in the simulations when the outgoing quality characteristics are multivariate.

Suppose that the desired specification intervals for electron density and temperature are given. As mentioned before, both electron density and temperature are measured as averages over the areas covered by the probes schematically shown in Figure 3. As in the previous subsection, we simulate the variability in chamber conditions by randomly disturbing inputs of the simulator, which yields the true distribution of output variables. Figure 6 depicts the true joint distribution of electron densities and temperatures, along with the desired specification area. Simple visual inspection indicates that, though mono-modal, the true distribution does not follow a Gaussian form and is also correlated, which does not match the assumptions from Theorem 2 in Section 2.1-E.

A comparison of yield estimators ${\widehat{p}}_1$ and ${\widehat{p}}_2$ is shown in Figure 7 and

Table 6. Empirical squared errors of estimators (Mean ± STD).

N	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_1$	$\mathbf{Empirical} \mathbf{Squared} \mathbf{Errors} \mathbf{of} {\widehat{\mathit{p}}}_2$
2	8.81 × ${10}^{-2}$ ± 1.19 × ${10}^{-1}$	5.07 × ${10}^{-2}$ ± 8.96 × ${10}^{-2}$
4	4.41 × ${10}^{-2}$ ± 6.06 × ${10}^{-2}$	2.34 × ${10}^{-2}$ ± 3.53 × ${10}^{-2}$
8	2.26 × ${10}^{-2}$ ± 3.14 × ${10}^{-2}$	1.31 × ${10}^{-2}$ ± 1.50 × ${10}^{-2}$
16	1.12 × ${10}^{-2}$ ± 1.57 × ${10}^{-2}$	8.29 × ${10}^{-3}$ ± 8.82 × ${10}^{-3}$
32	5.53 × ${10}^{-3}$ ± 8.01 × ${10}^{-3}$	6.10 × ${10}^{-3}$ ± 5.97 × ${10}^{-3}$
64	2.71 × ${10}^{-3}$ ± 3.81 × ${10}^{-3}$	4.90 × ${10}^{-3}$ ± 4.20 × ${10}^{-3}$
128	1.34 × ${10}^{-3}$ ± 1.86 × ${10}^{-3}$	4.35 × ${10}^{-3}$ ± 3.05 × ${10}^{-3}$

. Similar to what we observed in the scalar case, the MSE of the proposed yield estimator ${\widehat{p}}_2$ is smaller than that of the classical yield estimator ${\widehat{p}}_1$ until a certain threshold sample size $N$ is reached, after which the performance of ${\widehat{p}}_1$ and ${\widehat{p}}_2$ flips, as discussed in Section 2.1-F. As in the scalar case, the hypothesis test using the bootstrap method indicates that the proposed estimator ${\widehat{p}}_2$ is significantly preferable for $N=2$, whereas the classical yield estimator ${\widehat{p}}_1$ is significantly preferable for $N=128$, as shown in

Table 7. One-sided hypothesis testing using the bootstrap method (number of resamples = ${10}^4$).

N	One-Sided Alternative Hypothesis	p-Value	Effect Size (Difference in Average)	95% CI of Difference in Squared Error (Percentiles from 2.5 to 97.5)
2	Squared error of ${\widehat{p}}_2$ is less than that of ${\widehat{p}}_1$.	<${ 10}^{-16}$	−0.0373	[−0.0402, −0.0344]
128	Squared error of ${\widehat{p}}_1$ is less than that of ${\widehat{ p}}_2$.	<${ 10}^{-16}$	−0.00301	[−0.00309, −0.00294]

.

3.2. Numerical Experiments for Maximizing Yield

In this section, we compare the performance of BO algorithms for maximizing the yield of the outgoing product quality. Specifically, we focus on the difference between applying the GPR paradigm to directly model the empirical yields, as described in Section 2.2-B, versus applying GPRs to model parameters of the Gaussian distributions of the outgoing quality, as described in Section 2.2-C.

In Section 3.2-A, we compare the performance of the yield estimation and optimization algorithms for a synthetic problem in which the outgoing quality follows a Gaussian distribution. In Section 3.2-B, we consider yield estimation and optimization in ICP chamber simulations. Finally, in Section 3.2-C, we use ICP chamber simulations to perform a set of sensitivity studies regarding the preferability of different estimation and optimization algorithms for that specific problem.

• A. Optimization of control parameters which determine parameters of Gaussian distribution

In this subsection, we consider a synthetic problem of determining the control input for a process which will result in a maximal outgoing yield under the assumption that the outgoing quality follows a Gaussian distribution. More precisely, the problem is defined as follows. The domain of the control input parameter is $\mathcal{X}=\left[-3,+3\right]$ and is discretized into $1000$ points. The distribution of the outgoing quality $y$ is assumed to be distributed as $\mathcal{N}\left(\mu \left(x\right),{\sigma }^2\left(x\right)\right)$, where

$$\begin{array}{c}\begin{array}{r}\mu \left(x\right)=2x^2\end{array} \end{array}$$

(49)

and

$$\begin{array}{c}\begin{array}{rr}{\sigma }^2\left(x\right)& =\left\{\begin{array}{ll}-2x^2+10-4x& \left(0.1\le x\right)\\ 0.1& \left(x<0.1\right)\end{array}\right..\end{array} \end{array}$$

(50)

The desired specification interval for the outgoing quality variable $y$ is assumed to be $S=\left[-2,+2\right]$. Figure 8 illustrates the dependency of the Gaussian distribution parameters and the corresponding yield on the control parameter $x$. It can be noted that though the mean $\mu \left(x\right)$ of the outgoing quality takes the value of 0 when $x=0$, which corresponds to the center of the specification interval $S$, the variance ${\sigma }^2\left(x\right)$ of the outgoing quality is relatively large when $x=0$. Therefore, the maximum yield is realized when $x\fallingdotseq 0.72$, for which, even though the expected value of the outgoing quality is not centered in the middle of the specification interval, the variance ${\sigma }^2\left(x\right)$ is relatively small. This synthetic problem is an example that demonstrates that it is inadequate to only focus on the mean parameter for optimization of the yield.

Two versions of the BO algorithm are used for solving the problem described above. In one version, labeled in the remainder of the paper as BO PROB, GPR is employed to model the empirical yields obtained via estimation of the relevant Bernoulli parameter, as described in Section 2.2-B. In the other BO version, labeled as BO DIST in the remainder of this paper, GPRs are employed to model the mean and variance parameters of the outgoing product quality, as described in Section 2.2-C. Both BO PROB and BO DIST exploit the NEI acquisition functions, as discussed in Section 2.2. In our experiments, we use RBF (squared-exponential) kernels for BO PROB and BO DIST. As new observations are collected, the kernel length-scale parameters are optimized by maximizing the marginal log-likelihood. We impose a log-normal prior on each length-scale with lnℓ ∼ $\mathcal{N}$ (2 + 0.5d, ${\sigma }^2=3$), where d denotes the number of control parameters. Detailed configurations are available in the Supplementary Codebase.

A set of four initial guesses for the control parameter x were randomly generated following a Sobol Sequence [47], and $N=8$ samples of the outgoing quality $y$ were generated for each of those control parameters. Starting from that, BO PROB and BO DIST iterative procedures were run following Algorithm 1. In order to evaluate the stochastic behavior of the resulting BO methods, we performed 16 trials of the optimization procedures, with the progression of resulting quantiles of the yield suggested by the algorithms depicted in Figure 9 and

Table 8. Yields suggested by Bo methods as iterations progressed.

Percentile	25		50		75
Method	BO DIST	BO PROB	BO DIST	BO PROB	BO DIST	BO PROB
Iteration
4	0.501	0.429	0.528	0.501	0.539	0.524
9	0.536	0.482	0.542	0.511	0.543	0.524
14	0.529	0.476	0.539	0.496	0.541	0.524
19	0.525	0.484	0.538	0.500	0.541	0.528
24	0.534	0.482	0.540	0.516	0.541	0.534
29	0.531	0.479	0.540	0.498	0.542	0.534
34	0.534	0.497	0.540	0.526	0.542	0.534
39	0.536	0.500	0.540	0.522	0.542	0.539
44	0.531	0.497	0.540	0.522	0.542	0.541
49	0.535	0.497	0.542	0.525	0.543	0.535

.

As a tendency, one can see that as the iterations progress, the yields obtained using the suggested control parameters become higher. However, this tendency is not monotone due to the random nature of the problem and the BO procedure, as well as the limited number of trials. Inspection of the results corresponding to the 75th percentile shows little difference between BO DIST and BO PROB because both methods realize the yield close to the highest possible value. However, progressions of the 25th percentiles show the superiority of BO DIST in the sense that it realizes a relatively high yield very early in the iteration process. As per the discussions in Section 2.2-D, the superiority of the BO DIST method is reasonable because the outgoing quality y in this synthetic problem actually follows a Gaussian distribution.

We present additional experiments in Appendix D that examine the effects of kernel selection and the differences between the EI and NEI acquisition functions.

• B. Optimization of control parameters to maximize probability that electron densities and temperatures are within specifications

In this subsection, we study an optimization problem in which the objective is to maximize the yield defined as the probability that electron densities and temperatures in an ICP chamber fall within the desired specification limits. Similarly to what was performed in the previous subsection, we compare the performance of the BO PROB and BO DIST algorithms.

For this purpose, we use the simulation environment described in Section 3.1-B, with the basic simulation settings depicted in Figure 3 and discussed in Section 3.1-B-1. In order to derive the empirical distributions of electron densities and temperatures for each setting of control parameters, one needs to conduct a prohibitively large number of simulations. To address this challenge, we used the simulation environment to construct a Machine Learning (ML)-based surrogate model capable of predicting measurements of electron densities and temperatures based on a limited number of simulation results. Specifically, a Random Forest regression with z-score normalization [48] was employed because of its well-documented ability to achieve high accuracy even with a limited number of training samples [49]. This model was trained using the results of 2000 COMSOL simulation runs conducted for input parameters sampled using the Sobol Sequence method from the domain described in

Table 9. Domain of input parameters used to generate training data for Random Forest-based surrogate model of ICP chamber.

Item	Range
Coil Power (PW1)	10 [W]
Coil Powers (PW2, PW3)	(0, 300) [W]
Coil Powers (PW4)	250 [W]
Ratio of O2 at inlet (xO2)	(0.7, 0.9)
Pressure (p0)	(0.01, 0.02) [torr]
Pressure at outlet (outlet_p)	(0, 0.02) [torr]

, thus pursuing an informative training set via a limited number of simulations.

Table 10. Input parameters for simulations in order to construct true distribution of electron densities and temperatures.

Item	Type	Domain/Value/Distribution
Power1 (Pw1)	Fixed	10 [W]
Power2 (Pw2)	Control	10–250 [W]
Power3 (Pw3)	Control	10–250 [W]
Power4 (Pw4)	Fixed	250 [W]
Ratio of O2 at inlet (xO2)	Randomly Varied	$N\left(0.8,{\left(0.01\right)}^2\right)$
Pressure (p0)	Randomly Varied	$\mathcal{N}\left(0.015,{\left(0.0005\right)}^2\right)$ [torr]
Pressure at the outlet (outlet_p)	Randomly Varied	$\mathcal{N}\left(0.01,{\left(0.0005\right)}^2\right)$ [torr]

summarizes the input settings for the problem. The parameters labeled as Control are iteratively selected through the optimization algorithms, while the parameters labeled as Randomly Varied correspond to the uncontrollable inputs whose variations lead to variability in chamber conditions and, consequently, randomness in the measured electron densities and temperatures. As for the parameters labeled as Fixed, their values are not varied as inputs into the ML surrogate model. The domain of each control parameter, i.e., the domains of values for each of the two controllable power coils, is discretized into 20 points. The goal of the BO procedures is to determine control parameters that would maximize the probability of electron densities and temperatures measured in the probe areas illustrated Figure 3 being within desired specification limits. Specification limits for the electron densities and temperatures are summarized in

Table 11. Specification of electron densities and temperatures.

Item	Range
Electron density $\left(1/{\mathrm{m}}^3\right)$	$\left(1.8\times {10}^{16},2.0\times {10}^{16}\right)$
Electron temperature (eV)	$\left(3.8,4.0\right)$

.

To clarify the relationship between surrogate-based evaluations and real experiments, we summarize their correspondences in

Table 12. Correspondence between surrogate model experiments and real experiments.

Aspect	Surrogate Model	Real Experiments
Control variables/operating windows	Coils (see Table 3 for details)	Same conditions
Measurement cost /response lag	Negligible per eval/fast surrogate; (0.1~1 [s] per evaluation on workstation)	Measurement of sensors for processing wafers /repetition of wafer processing (tens of minutes ~ hours)
Yield metrics	Probability both density/temp within specification limits	Same metrics

. While the surrogate model and simulations are designed to mimic the measurements obtained from the physical system and therefore produce data that can be treated with the same evaluation metrics, real experiments require repeated measurements at fixed settings to directly observe variability arising from environmental factors and measurement noise. A typical real experiment loop consists of performing repeated measurements at a chosen setting, training a Gaussian Process (GP) on the acquired data, computing the acquisition function, and selecting the next setting sequentially. Because real experiments incur substantial time, material, and operational costs, a pragmatic workflow is to first screen and evaluate promising methods using surrogate models and then validate selected candidates on the actual equipment.

Figure 10 illustrates the relationship between control parameters and the probabilities that electron densities and temperatures fall within the specification intervals, as determined by the surrogate ML model with empirical samples from the domain described in Table 10.

A set of four initial guesses for the pair of coil powers were randomly selected following a Sobol Sequence, and $N=8$ samples of electron densities and temperatures were produced by the surrogate ML model for each of those pairs. Starting from that, BO PROB and BO DIST iterative procedures were run following Algorithm 1 in order to optimize the pair of controllable coil powers. In order to evaluate the stochastic behavior of the resulting BO methods, we ran 16 trials of the optimization procedures, with the progression of the 25th, 50th and 75th percentiles of the yields obtained by the BO algorithms depicted in Figure 11 and

Table 13. Yields suggested by Bo methods for each iteration.

Percentile	25		50		75
Method	BO DIST	BO PROB	BO DIST	BO PROB	BO DIST	BO PROB
Iteration
4	0.283	0.297	0.426	0.335	0.621	0.365
9	0.479	0.282	0.507	0.336	0.689	0.381
14	0.507	0.340	0.621	0.369	0.689	0.492
19	0.455	0.371	0.587	0.447	0.638	0.492
24	0.483	0.356	0.507	0.447	0.621	0.524
29	0.503	0.356	0.621	0.424	0.638	0.493
34	0.507	0.356	0.587	0.471	0.638	0.510
39	0.507	0.382	0.621	0.492	0.621	0.621
44	0.507	0.387	0.507	0.471	0.621	0.570
49	0.507	0.356	0.621	0.450	0.621	0.621

. For one trial, approximately 10 s and 80 s were necessary for BO PROB and BO DIST, respectively, in this evaluation setting with the standard workstation.

One can see that when the number of iterations is small, the BO DIST approach which relies on estimating means and variances of the underlying distributions achieves higher yields than the BO PROB method which relies on directly modeling empirical yields. As discussed in Section 2.2-D, this may be attributed to the scarcity of experiments compensated for by the assumption of a Gaussian form. If one looks at the progression of the 75th percentiles, BO PROB improves its performance as iterations progress, while the BO DIST method deteriorates in the later iterations. This may be attributed to the incorrect Gaussianity assumption, leading to the BO DIST method being informed by incorrect yield estimations, as discussed in Section 2.1-F.

• C. Sensitivity studies for BO performance in ICP chamber simulations

As discussed in Section 2.2-D, the preferability of the newly proposed BO DIST algorithm, which relies on GPRs modeling the Gaussian distribution parameters, depends on the properties of the underlying problem. Here, we perform two sensitivity studies in order to illustrate the following:

• When the desired specification area is narrower, the newly proposed algorithm works more favorably.
• When the number of experiments is smaller, the newly proposed algorithm works more favorably.

Table 14. The specification of electron densities and temperatures for a sensitive study for the strictness of specification.

Item	STRICT	LENIENT
Electron density $\left(1/{\mathrm{m}}^3\right)$	$\left(1.8\times {10}^{16},1.9\times {10}^{16}\right)$	$\left(1.8\times {10}^{16}, 2.1\times {10}^{16}\right)$
Electron temperature (eV)	$\left(3.8,3.9\right)$	$\left(3.8,4.1\right)$

shows the specification intervals used in numerical experiments investigating the influence of the strictness of the desired specification on BO performance. In the case labeled as STRICT, the specified intervals of electron density and temperature are narrow, and the maximal possible yield is approximately 0.52. In the case labeled as LENIENT, the specification intervals are wider, with the maximal possible yield being approximately 0.71.

The number of experiments per iteration $N$ was set to be 8 and the initial settings of control parameters were randomly selected, after which the BO PROB and BO DIST iterative procedures were run following Algorithm 1 in order to maximize the yield via optimization of the pair of controllable coil powers. The performance of each algorithm was investigated using the ratio of the yield determined by that algorithm vs. the actual maximal possible yield for the problems. For each BO algorithm, we performed 32 trials of the optimization procedure and calculated the medians of the aforementioned yield ratios, which are depicted in Figure 12.

It can be seen that when the specifications are strict, the BO DIST algorithm relying on the newly proposed concept of GPRs modeling the mean and variance parameters of the outgoing quality outperforms the classical approach relying on the GPR modeling empirical yields. On the other hand, when the specifications are lenient, the classical approach achieves high yields for a relatively smaller number of iterations. As discussed in Section 2.2-D, these observations may be attributed to the fact that the newly proposed algorithm takes into consideration the proximity of the observed characteristics to the desired specifications, thus facilitating better yield estimates from a smaller number of samples.

Under the same input parameter settings and specification limits as those considered in the previous subsection (input parameter settings listed in Table 10 and specification limits summarized in Table 11), we evaluated the performance of the optimization algorithms for different sample sizes $N$ available at each iteration in the BO algorithms. A small sample size $N=4$ and a large sample size of $N=256$ were considered, with 32 random trials of the BO procedures conducted. Figure 13 reports the progression of medians of the resulting ratios of the yields produced by the BO PROB and BO DIST methods versus the maximal possible yield.

One can see that when the sample size $N$ is small, the BO DIST approach relying on the newly proposed concept of GPRs modeling the parameters of the Gaussian distribution performed better. On the other hand, when the sample size $N$ is large, the classical approach relying on the GPR modeling empirical yield becomes superior, except at the beginning of the iterations. As discussed in Section 2.2-D, the effectiveness of the proposed algorithm for small sample sizes $N$ can be attributed to the ability of the Gaussian distribution assumption to compensate for a limited number of experiments, even when the Gaussianity assumption does not hold exactly true. In contrast, the classical BO PROB algorithm shows its strength with the large sample size $N$ due to the consistency of the Bernoulli distribution-based yield estimator—a property that the yield estimator proposed in this paper does not possess.

4. Conclusions

In this paper, we introduced novel methodologies for estimating and maximizing yield in a manufacturing process, with yield being defined as the probability that quality characteristics fall within the desired specifications. The newly proposed yield estimator relies on estimating the expected value and variance of the outgoing product quality, based on which a Gaussian distribution form is assumed to calculate the yield. It was mathematically proven that if the outgoing product quality characteristics follow an uncorrelated multivariate Gaussian distribution, the newly proposed yield estimator achieves asymptotically lower MSE values compared to the classical yield estimator, which directly estimates the relevant Bernoulli distribution parameters. Additionally, we integrated this yield estimator into a BO framework in order to effectively optimize yield in the presence of uncertainties in the relevant manufacturing processes.

Theoretical insights regarding the properties of the novel yield estimator were verified in a synthetic example of yield estimation and optimization. In addition, the BO-based yield optimization procedures using the novel and traditional yield estimators were evaluated through high-fidelity simulations of a reactive ion etch chamber. The objective of these simulations was to determine tool parameters that maximize the probability of delivering desired electron density and temperature characteristics in plasma. BO using the newly proposed yield estimation algorithm showed advantages when the number of experiments was limited and when the desired specifications were strict, with advantages being more prevalent when the underlying distribution of the outgoing quality characteristics can be approximated well with an uncorrelated multivariate Gaussian distribution.

Looking ahead, future research should emphasize applications of the proposed method to experiments performed with physical equipment. Furthermore, the proposed method can be applied to estimate and maximize yields of various outgoing characteristics in different fields of manufacturing. From the perspective of theoretical developments, extending our methodologies to account for distributions beyond uncorrelated multivariate Gaussian distributions is essential. It will enable one to handle the critical question of when and to what degree the incorporation of prior knowledge about the underlying distribution form leads to a reduction in MSE when the distribution form differs from the uncorrelated multivariate Gaussian distributions.