A Penalized Orthogonal Kriging Method for Selecting a Global Trend

Abstract

A kriging regression model is a popular and effective type of surrogate model in computer experiments. A significant challenge arises when the mean function of the model includes polynomial terms with unknown coefficients, leading to identifiability problems and potentially unreliable results. To overcome this problem, Plumlee and Joseph (2018) introduced an orthogonal kriging model. Variable selection for kriging models has been widely considered by researchers in computer experiments. In this paper, we introduce a new method for combining orthogonal kriging with penalized variable selection. Furthermore, an efficient algorithm is given to select the correct mean function. The simulation results and an example study with real data show that the proposed method is superior to others in variable recognition rate and prediction accuracy.

1. Introduction

The rapid development of computers has made it possible to simulate some physical processes through computer experiments and has helped us deal with these complex practical problems efficiently. Sacks et al. [1,2] proposed the use of stochastic processes to model deterministic output responses. Kriging models, first proposed by [3], have become a widely used class of surrogate models for deterministic approximation; some empirical studies have shown that kriging is superior to other interpolation techniques [4]. Simpson et al. [5] found that kriging is suitable for the approximation of higher-order and lower-dimensional functions and has high accuracy for nonlinear problems, which makes this type of model attractive to researchers.

A kriging model is called an ordinary kriging model if its mean function has only one constant, and it is called a universal kriging (UK) model if its mean function is partially assumed to have some known variables. Many studies suggest benefits to using a more complex mean function. However, overfitting will occur when the mean function contains many candidate predictors [6,7]. In such cases, variable selection is proposed to avoid overfitting problems for the UK model [8]; the commonly used methods of variable selection are the Bayesian method [9,10] and penalty likelihood method [11,12]. Žilinskas [13,14] have thoroughly examined the axiomatics regarding the application of statistical models to interpret deterministic data. To choose the mean function of a kriging model using the penalty likelihood method, an optimization algorithm may be constructed on the basis of a statistical model. Park [15] proposed an improved penalized blind lasso kriging method that could avoid the need for iterative calculations, but their approach does not account for correlations between sample points. Zhao et al. [16] addressed this limitation with a modified penalized blind kriging (MPBK) method.

A significant challenge arises when the mean of a UK model includes polynomial terms with unknown coefficients, leading to an identifiability problem. If the coefficient estimator of the mean function is poor, then even if the predictions are still good, the interpretability of the mean function is lost. Many researchers have tackled this identifiability problem in numerous studies [17,18,19]. For example, in model calibration, due to the existence of parameter identification problems, we cannot correctly interpret the actual meaning of each variable from the coefficient of the mean function, and sometimes we even misrecognize the part of the mean function, which results in the opposite conclusion. Some scholars have proposed that the identification problem could be overcome by orthogonalizing the mean function and a Gaussian process [20,21,22]. Plumlee and Joseph [23] proposed an orthogonal Kriging model based on previous studies. Recently, Wang et al. [24] proposed a model averaging method for orthogonal kriging models, and the proposed method achieved prediction advantages over others. In brief, the orthogonal kriging model greatly improves the accuracy of parameter estimation and is becoming increasingly meaningful for variable selection problems.

Due to the problem of parameter identification, this study incorporates a variable selection mechanism into the orthogonal Kriging model via penalized likelihood functions. The rest of the paper is structured as follows. The penalized orthogonal Kriging model is proposed in Section 2. Section 3 investigates its performance through simulation studies. A real engineering example is presented in Section 4. Finally, Section 5 concludes the work.

2. Penalized Orthogonal Kriging Model

2.1. Orthogonal Kriging Model

We consider the output response $y\left(x\right)$, where x represents a d-dimensional vector-valued input in D. A popular modeling approach is UK, which assumes that $y\left(x\right)$ is a sum of a trend function $\mu \left(x\right)$ and a stochastic process $z\left(x\right)$. We express the response $y\left(x\right)$ as follows:

$$y\left(x\right)=\mu \left(x\right)+z\left(x\right)=f^T\left(x\right)\beta +z\left(x\right),$$

where $z\left(x\right)$ is a stochastic process with mean zero and covariance $cov(z\left(x\right),z\left(x^{\prime }\right))={\sigma }^{2}R(x,x^{\prime })$, $f\left(x\right)={(f_1\left(x\right),\cdots ,f_k\left(x\right))}^T\in R^k$ is a known basis function vector, and $\beta ={({\beta }_1,{\beta }_2,\cdots ,{\beta }_d)}^T$ is the unknown parameter vector. In many publications on computer experiments, the choice of correlation function is not unique; examples include the exponential correlation function, the $Mat\acute{e}rn$ correlation function, and the Gaussian correlation function.

A significant challenge arises when the mean function of UK includes polynomial terms with unknown coefficients, leading to identifiability problems. To overcome these problems, Plumlee and Joseph [23] proposed an orthogonal Gaussian model, in which the Gaussian process term is orthogonal to the mean function term. If we let $h\left(x\right)={\int }_{X}R(x,\zeta )f\left(\zeta \right)d\zeta$, $H\left(x\right)={\int }_X{\int }_{X}R({\zeta }^{\prime },\zeta )f{\left({\zeta }^{\prime }\right)}^{T}f\left(\zeta \right)d\zeta d{\zeta }^{\prime }$, and $R_{*}(x,x^{\prime })=R(x,x^{\prime })-h{\left(x\right)}^T{H}^{-1}h\left(x^{\prime }\right)$, the orthogonal kriging model is written as

$$y\left(x\right)=f^T\left(x\right)\beta +z^{*}\left(x\right),$$

(1)

where ${\int }_{X}f\left(x\right)z_{*}\left(x\right)dx=0$, $z_{*}\left(x\right)$ is a stochastic process with mean zero and covariance $co{v}_{*}(x,x^{\prime })={\sigma }^2{R}_{*}(x,x^{\prime })$.

In this paper, we only consider the following Gaussian correlation function:

$$R(x,x^{\prime })=exp\left(-\sum _{j=1}^d{\displaystyle \frac{|x_j-x_j^{\prime }{|}^2}{{\psi }_j^2}}\right).$$

(2)

Suppose the set $K_i$ is a subset of $\{1,2,\cdots ,d\}$, and let $\mathrm{erf}\left(x\right)={\displaystyle \frac{2}{\sqrt{\pi }}}{\int }_0^x{e}^{-t^2}dt$ be the Gaussian error function. According to [23], the orthogonal version $R_{*}(x,x^{\prime })$ of $R(x,x^{\prime })$ in (2) can be stated as follows:

$$R_{*}(x,x^{\prime })=R(x,x^{\prime })-\sum _{i=1}^p\prod _{j\in K_i}{\displaystyle \frac{A_j\left(x_j\right)A_j\left(x_j^{\prime }\right)}{I{A}_j}}\prod _{j\in K_i}{\displaystyle \frac{B_j\left(x_j\right)B_j\left(x_j^{\prime }\right)}{I{B}_j}},$$

where

$$\begin{array}{cc}\hfill & B_j={\displaystyle \frac{\sqrt{\pi }{\psi }_j}{2}}\{erf({\displaystyle \frac{x+1}{{\psi }_j}})-erf({\displaystyle \frac{x-1}{{\psi }_j}})\},\hfill \\ \hfill & A_j={\displaystyle \frac{{\psi }_j^2}{2}}\{exp(-{({\displaystyle \frac{x+1}{{\psi }_j}})}^2)-exp(-{({\displaystyle \frac{x-1}{{\psi }_j}})}^2)\}+x{B}_j(x),\hfill \\ \hfill & I{A}_j={\displaystyle \frac{{\psi }_j^4}{6}}\{1-exp(-{({\displaystyle \frac{4}{{\psi }_j}})}^2)-{\displaystyle \frac{{\psi }_j^2}{3}}erf\{3-exp({\displaystyle \frac{4}{{\psi }_j^2}})\}+{\displaystyle \frac{2\sqrt{\pi }{\psi }_j}{3}}erf({\displaystyle \frac{2}{{\psi }_j}}),\hfill \\ \hfill & I{B}_j=2\sqrt{\pi }{\psi }_{j}erf({\displaystyle \frac{2}{{\psi }_j}}))-{\psi }_j^2\{1-exp(-{({\displaystyle \frac{4}{{\psi }_j}})}^2)\}.\hfill \end{array}$$

Given some sampled points $x_i$, $y_i=y\left(x_i\right)$ is the corresponding output, $i=1,\cdots ,N$. Let $Y={(y_1,y_2,\cdots ,y_N)}^T$, $F={(f^T\left(x_1\right),...,f^T\left(x_N\right))}^T$, $Z^{*}={(z^{*}\left(x_1\right),z^{*}\left(x_2\right),\cdots ,z^{*}\left(x_N\right))}^T$, $\psi =({\psi }_1,\cdots ,{\psi }_d)$, and $R_{*}\left(\psi \right)={\left[R_{*}(x_i,x_j)\right]}_{N\times N}$. For the simplicity, we can use the matrix notation of model (1):

$$Y=F\beta +Z^{*},$$

where $Z^{*}$ follows $N(0,{\sigma }^2{R}_{*}\left(\psi \right))$. Then the corresponding log-likelihood function in (1) after removing the constant term is

$$l(\beta ,{\sigma }^2,\psi )=-{\displaystyle \frac{N}{2}}log\left({\sigma }^2\right)-{\displaystyle \frac{1}{2}}log\left|R_{*}\right|-{\displaystyle \frac{1}{2{\sigma }^2}}{(Y-F\beta )}^T{R}_{*}^{-1}(Y-F\beta ).$$

(3)

The parameter estimation can be expressed as

$$\widehat{\beta }={\left(F^T{R}_{*}^{-1}\left(\widehat{\psi }\right)F\right)}^{-1}{F}^T{R}^{-1}\left(\widehat{\psi }\right)Y,$$

$${\widehat{\sigma }}^2={\displaystyle \frac{1}{N}}{(Y-F\widehat{\beta })}^T{R}_{*}^{-1}\left(\widehat{\psi }\right)(Y-F\widehat{\beta }),$$

$$\widehat{\psi }=arg\underset{\psi }{min}\left\{\begin{array}{c}Nlog\widehat{{\sigma }^2}+log\left|R_{*}\left(\psi \right)\right|\end{array}\right\}.$$

Therefore, the best linear unbiased predictor (BLUP) for $y\left(x\right)$ at a new point x is

$$\widehat{y}\left(x\right)=f^T\left(x\right)\widehat{\beta }+R_{*}(x,D)R_{*}^{-1}\left(\widehat{\psi }\right)(Y-F\widehat{\beta }),$$

where the vector $R_{*}(x,D)={(R_{*}(x,x_1),\cdots ,R_{*}(x,x_N))}^T$ represents the correlation between the sample point $x_i(i=1,\cdots ,N)$ and the predicted point x.

2.2. Penalized Orthogonal Kriging Model

Zhao et al. [16] described a modified penalized blind kriging (MPBK) method for efficiently selecting a global trend model in a universal kriging model. In this subsection, we propose a model in which a variable selection procedure is incorporated in the orthogonal kriging model and the variable selection is achieved using the penalized likelihood. We call it the penalized orthogonal kriging model (POK model for short).

The objective of variable selection is to detect important variables. A penalty function is included in the likelihood that is generally used in variable selection; examples include the lasso [25], the elastic net (EN) penalty [26], the minimax concave penalty (MCP for short) [27], etc. On this basis, we can estimate the coefficients $\beta$ by maximizing the following penalized log-likelihood function:

$$\begin{array}{cc}\hfill Q(\beta ,{\sigma }^2,\psi )& =\ell (\beta ,{\sigma }^2,\psi )-\sum _{j=1}^p{P}_{\lambda }\left(\left|{\beta }_j\right|\right)\hfill \\ \hfill & =-{\displaystyle \frac{N}{2}}lg\left({\sigma }^2\right)-{\displaystyle \frac{1}{2}}lg\left|R_{*}\right|-{\displaystyle \frac{1}{2{\sigma }^2}}{(Y-F\beta )}^T{R}_{*}^{-1}(Y-F\beta )-P\left(\beta \right),\hfill \end{array}$$

where $-P\left(\beta \right)$ is a penalty function. When $\psi$ and ${\sigma }^2$ are estimated, $\beta$ can be given for the penalty function as follows:

$${\widehat{\beta }}_{*}=arg\underset{\beta }{min}\{{\displaystyle \frac{1}{2\widehat{{\sigma }^2}}}{(Y-F\beta )}^T{\widehat{R}}_{*}^{-1}(Y-F\beta )+P\left(\beta \right)\},$$

where ${\widehat{R}}_{*}=R^{*}(\widehat{\psi })$.

We use the lasso, EN, and MCP as penalty functions in this paper. The EN penalty function is

$$P_{\alpha ,\lambda }\left(\beta \right)=\lambda ((1-\alpha ){∥\beta ∥}^2+\alpha {∥\beta ∥}_1),$$

which is a convex combination of ridge regression and the lasso; $\lambda$ and $\alpha$ are penalty parameters. When $\alpha =1$, the EN penalty function is the lasso. The MCP penalty function is

$$P_{\alpha ,\lambda }\left(\beta \right)=\left\{\begin{array}{cc}\lambda {∥\beta ∥}_1-{\displaystyle \frac{{∥\beta ∥}^2}{2\alpha }},\hfill & {∥\beta ∥}_1\le \alpha \lambda ,\hfill \\ {\displaystyle \frac{1}{2}}\alpha {\lambda }^2,\hfill & {∥\beta ∥}_1>\alpha \lambda ,\hfill \end{array}\right.$$

where $\lambda$ and $\alpha$ are penalty parameters.

A penalty function is included in the likelihood, which faces the challenging work of estimating the parameters in the POK models. Since the penalty term is not a function of $\psi$ and ${\sigma }^2$, the maximum likelihood estimate of ${\sigma }^2$ can be expressed as follows:

$${\widehat{\sigma }}_{*}^2={\displaystyle \frac{1}{N}}{(Y-F{\widehat{\beta }}_{*})}^T{R}_{*}^{-1}(\psi )(Y-F{\widehat{\beta }}_{*}),$$

Substituting the values of ${\widehat{\beta }}_{*}$ and ${\widehat{\sigma }}_{*}^2$ into $Q(\beta ,{\sigma }^2,\psi )$, the maximum likelihood estimate of $\psi$ is

$${\widehat{\psi }}_{*}=arg\underset{\psi }{min}\left\{Nlog{\widehat{{\sigma }_{*}}}^2+log\left|R_{*}\left(\psi \right)\right|\right\}.$$

The POK predictor, which has the same form as the OUK predictor, is

$${\widehat{y}}_{*}\left(x\right)=f^T\left(p\right){\widehat{\beta }}_{*}+r_{*}{R}_{*}^{-1}\left({\psi }_{*}\right)(Y-F{\widehat{\beta }}_{*}),$$

(4)

where $r_{*}={(R_{*}(p_1,p),···,R_{*}(p_N,p))}^T$.

For the lasso, EN, or MCP as the penalty function, the relevant parameter estimation and algorithm steps can be obtained in a similar way. An iteratively reweighted least angle regression (IRLARS) is performed, as expressed in Algorithm 1.

Algorithm 1: IRLARS algorithm for POK.

Step 0: Set up initial values for ${\widehat{\beta }}_{*.0}^2$, ${\widehat{\sigma }}_{*.0}^2$, and ${\widehat{\psi }}_{*.0}$, and let the counter $l=0$. While none of the termination conditions is satisfied, do Step 1: Decompose $R{\left({\widehat{\psi }}_{*,l}\right)}^{-1}/{\widehat{\sigma }}_{*.l}^2=A_{*}^T{A}_{*}$, update $y^{*}=A_{*}y$, and $F^{*}=A_{*}F$.  The optimal penalty parameters $\alpha$ and $\lambda$ are obtained by cross-validation. Thus, $${\widehat{\beta }}_{*,l+1}=arg\underset{\beta }{min}\{{∥\tilde{Y}-F^{*}\beta ∥}^2+P_{\alpha ,\lambda }\left(\left|\beta \right|\right)\}.$$ Step 2: Use the ${\widehat{\beta }}_{*,l+1}$ estimated in Step 2 to obtain ${\widehat{\sigma }}_{*,l+1}^2$, and ${\widehat{\psi }}_{*,l+1}$. $${\widehat{\psi }}_{*,l+1}=arg\underset{\psi }{min}\left\{Nlog{\widehat{\sigma }}_{*,l+1}^2+log\left|R_{*}\left(\psi \right)\right|\right\}.$$ Step 3: Repeat Step 1 and Step 2 until ${\widehat{\psi }}_{*}$ converges. Step 4: The POK predictor is obtained by substituting the estimated parameters into (4).

In the IRLARS algorithm, the optimal penalty parameters $\lambda$ and $\alpha$ are obtained through cross-validation (CV). The sample is randomly divided into K subsets of approximately equal size; among them, (K-1) subsets are used to construct a model, and the left-out subset is used to validate the model. Suppose we are given two parameter sets $C_{\lambda }$ and $C_{\alpha }$ of $\lambda$ and $\alpha$, respectively. For the given $(\lambda ,\alpha )\in C_{\lambda }\times C_{\alpha }$, we randomly split the sample into K subsets of approximately equal size; (K-1) of the subsets are used as training data to obtain the penalty estimator ${\widehat{\beta }}_{*}$, and the left-out sample is used as testing data to validate the results. The K-fold mean square CV error is as follows:

$$CV(\lambda ,\alpha )=\sum _{k=1}^K{\parallel y_{\left(k\right)}-{\widehat{y}}^{-k}\parallel }^2/N,$$

where $y_{\left(k\right)}$ is the response in the left-out sample and ${\widehat{y}}^{-k}$ is the BLUP of $y_{\left(k\right)}$ by (4). Then, the optimal tuning parameters are as follows:

$$(\widehat{\lambda },\widehat{\alpha })=argmi{n}_{(\lambda ,\alpha )\in C_{\lambda }\times C_{\alpha }}CV(\lambda ,\alpha ).$$

To balance the prediction accuracy and computational complexity, the 10-fold CV method is used in this paper.

3. Numerical Simulation Study

In this section, the performance of the proposed POK method is compared with that of MPBK methods using two examples from [16].

In simulation studies, performance is evaluated in two aspects: the accuracy of variable selection and the degree of prediction accuracy. The accuracy of variable selection is measured by the active effect identified rate (AEIR), the inactive effect identified rate (IEIR), and the mean size (MS) of variables in the mean function in the repeated simulations. Prediction accuracy is measured by the mean value of the root mean square prediction error (MRMSPE) and the standard deviation of the root mean square prediction error (SDRMSPE), calculated based on the randomly generated testing data.

The simulation studies were carried using the POK and MPBK methods with the lasso, EN, and MCP as penalty functions. MPBK.MCP, MPBK.MCP, and MPBK.EN denote the MPBK method with the lasso, MCP, and EN, respectively, as penalty functions. Furthermore, POK.L, POK.MCP, and POK.EN denote the POK method with the lasso, MCP, and EN, respectively, as penalty functions. In the simulation study, 1000 sample points were randomly selected as testing data, and six methods are evaluated based on 500 simulations.

Example 1

(linear function model from [12]). The known function is defined on a twelve-dimensional (p = 12) input space ${[0,1]}^{12}$, where the first six variables $x_1$, …, $x_5,x_6$ have decreasing effects on the on the computer experiment’s output, and the coefficients of the remaining variables $x_7$, …, $x_{12}$ are zero. The function is

$$y\left(x\right)=0.4x_1+0.3x_2+0.2x_3+0.1x_4+0.05x_5+0.01x_6+\epsilon ,$$

(5)

where $\epsilon \sim N(0,{\sigma }_{\epsilon }^2)$, ${\sigma }_{ϵ}=0.05$. The experimental designs were generated through Latin hypercube sampling with $p=12$ dimensions, and the responses were generated using (5) independently for training data. Here, we consider sample sizes of $N=50$, $80$, and $100$, respectively.

The variable selection performances of the six methods are shown in

Table 1. Data simulation results for Example 1.

Sample Size	Method	MPBK.L	MPBK.MCP	MPBK.EN	POK.L	POK.MCP	POK.EN
N = 50	AEIR (%)	91.93	99.62	80.12	90.55	99.90	79.28
	IEIR (%)	55.45	99.45	14.70	54.45	99.90	18.17
	MS	8.78	11.94	5.69	8.70	11.99	5.85
	MRMSPE ($\times {10}^{-2}$)	5.77	5.85	5.63	5.72	5.79	5.61
	SDRMSPE ($\times {10}^{-2}$)	0.39	0.39	0.38	0.33	0.37	0.34
N = 80	AEIR (%)	92.80	99.32	82.18	92.10	99.80	81.75
	IEIR (%)	53.52	99.25	11.17	53.20	99.80	13.35
	MS	8.78	11.91	5.60	8.72	11.98	5.71
	MRMSPE ($\times {10}^{-2}$)	5.47	5.51	5.39	5.40	5.45	5.33
	SDRMSPE ($\times {10}^{-2}$)	0.57	0.57	0.58	0.22	0.56	0.22
N = 100	AEIR (%)	93.12	99.70	82.92	92.77	100.00	82.72
	IEIR (%)	54.07	99.70	10.05	53.07	99.98	11.58
	MS	8.83	11.96	5.58	8.75	11.99	5.66
	MRMSPE ($\times {10}^{-2}$)	5.38	5.40	5.31	5.33	5.35	5.27
	SDRMSPE ($\times {10}^{-2}$)	0.44	0.44	0.44	0.20	0.19	0.19

and Figure 1, Figure 2 and Figure 3. According to Table 1 and Figure 2, MPBK.L, MPBK.MCP, POK.L, POK.MCP, POK.Lm and POK.EN exhibit more pronounced overfitting tendencies than MPBK.EN and POK.EN, resulting in substantially higher MS and IEIR values. This indicates that MPBK.EN and POK.EN achieve better identification in variable selection.

The simulation results in Table 1 and Figure 1 demonstrate that POKs achieve higher AEIRs and lower MRMSEPs and SDRMSPEs than their MPBK counterparts across all three penalty functions, despite showing similar patterns of behavior. Notably, POK.EN has the smallest MRMSEP and SDRMSPE. In summary, POK.EN demonstrates superior overall performance in linear function scenarios, excelling in both variable selection accuracy and prediction quality compared to alternative methods.

Example 2

(borehole function ([28])).

$$y\left(x\right)=2\pi x_3(x_4-x_6){\left\{\begin{array}{c}log\left({\displaystyle \frac{x_2}{x_1}}\right)(1+2{\displaystyle \frac{x_3{x}_7}{log(x_2/x_1{x}_1^2{x}_8)}}+{\displaystyle \frac{x_3}{x_5}})\end{array}\right\}}^{-1}.$$

(6)

The input space is a rectangular interval [0.05, 0.015] × [100, 5000] × [63,070, 115,600] × [990, 1110] × [63.1, 116] × [700, 820] × [1120, 1680] × [9855, 12,045], and the samples were generated through Latin hypercube sampling with $p=8$ dimensions and sample sizes of $N=50$, $80$, and $100$. We set the mean value function as a linear function,

$$\mu \left(x\right)={\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_8{x}_8.$$

The variable selection performances of six methods are shown in

Table 2. Data simulation results for example 2.

Sample Size	Method	MPBK.L	MPBK.MCP	MPBK.EN	POK.L	POK.MCP	POK.EN
N = 50	MS	7.03	8.00	7.94	5.51	8.00	7.99
	MRMSPE	11.47	11.48	11.48	6.30	4.63	4.62
	SDRMSPE	0.64	0.63	0.63	1.14	1.11	1.10
N = 80	MS	7.02	8.00	7.94	5.54	8.00	8.00
	MRMSPE	11.07	11.08	11.08	2.98	2.16	2.15
	SDRMSPE	0.48	0.47	0.48	0.57	0.25	0.25
N = 100	MS	7.02	8.00	7.94	5.54	8.00	7.99
	MRMSPE	10.95	10.96	10.95	2.38	1.82	1.81
	SDRMSPE	0.46	0.46	0.47	0.36	0.19	0.19

and Figure 3. Table 2 demonstrates that POK.L exhibits larger MRMSEPs than POK.MCP and POK.EN, while the latter two methods behave similarly, perhaps due to the strong compression effect of the lasso penalty, and all three POK methods demonstrate significantly smaller MRMSEP values than their MPBK counterparts. Figure 3 provides further evidence of this phenomenon, demonstrating POK.MCP and POK.EN’s consistent superiority in prediction accuracy across diverse parameter configurations.

4. Real Data Example

In this section, our proposed method is applied to analyze an engineering problem called piston slap noise. Piston slap is an undesirable engine noise caused by the secondary motion of the piston. Through computer experiments, six factors are varied to reduce piston slap noise. These six factors are cylinder liner ($x_1$), location of peak pressure ($x_2$), skirt length ($x_3$), skirt profile ($x_4$), skirt oval ovality ($x_5$), and pin offset ($x_6$). The relevant dataset for the example comes from [29], which includes 100 observations with six input variables.

The candidate variable set includes all linear main effects, quadratic main effects, and two-factor interaction effects, resulting in a total of 72 basis variables. We perform 5-fold cross-validation, using one of the five subsets as the test set and the rest as the training set. Six methods are compared, and the simulation results are given in

Table 3. Data simulation results for piston slap noise.

Method	MPBK.L	MPBK.MCP	MPBK.EN	OPBK.L	OPBK.MCP	OPBK.EN
MS	22.26	54.40	23.80	22.60	66.60	32.80
MRMSPE	0.93	1.24	0.91	0.37	0.98	0.38
SDRMSPE	0.65	0.32	0.63	0.21	0.41	0.22

.

As displayed in Table 3, POK models outperform MPBK models in both prediction accuracy and model simplicity. Specifically, OPBK.L and OPBK.EN achieve the smallest prediction errors. These results indicate that the POK framework not only enhances predictive capability but also yields more parsimonious models.

5. Conclusions

Variable selection for Kriging models has been widely considered by researchers in computer experiments. Since accurate estimation of mean functions is important in many application, variable selection plays an important role in solving the overfitting problem and improving the prediction accuracy of models. This paper introduces a variable selection method using POK. The simulation results indicate that the POK model shows great advantages in both variable recognition rate and prediction accuracy.