2.1. Kriging Surrogate Model
The Kriging model was initially created in geology to forecast and estimate mineral deposits. Later, Jerome Sacks et al. introduced it into computer experiment approximation modeling, which led to its widespread application and broad recognition in engineering design and optimization [
35]. In this paper, only a brief overview of the Kriging model is provided; the detailed derivation can be found in Sacks et al. [
35].
For a set of design points
with corresponding function values
, a simple Kriging model can be constructed as follows:
Here,
represents the mean of the Gaussian process, and
denotes the normally distributed error term with zero mean and variance
. Most regression models assume that the errors at two points
x(i) and
x(j) are independent. In contrast, the Kriging model assumes that the error terms
and
are correlated, with the correlation depending on the distance between the two points, as expressed in Equation (2).
Here, denotes the dimension of the design variables, while and are the parameters to be determined.
In addition, the parameters
in the Kriging model are estimated from the sampling points. The Kriging model also provides not only the predicted value at an unknown point but also the mean squared error of the prediction. The specific formulations are given in Equations (3) and (4).
and
In the equations, is a matrix with entry , is an n-dimensional vector with entry , is the vector of the n observed function values, and is an n-dimensional vector of ones.
2.3. Principles of Efficient Global Optimization (EGO)
The Efficient Global Optimization (EGO) algorithm typically consists of two stages. In the first stage, an initial set of sample points is chosen through a sampling strategy to build the Kriging surrogate model. In the second stage, the Expected Improvement (EI) criterion is used to identify the most promising points, with the Kriging model being updated in each iteration to progressively approximate the global optimum. The procedure of the EGO algorithm is summarized in Algorithm 1.
| Algorithm 1: The framework of the EGO algorithm |
| Require: Initial design set (X, Y) |
| Ensure: The best observation (xmin, ymin) |
| 1: while the stop condition is not met, do |
| 2: Building a Kriging model based on the current design set (X, Y) |
| 3: xnew = argmax EI(x) |
| 4: Evaluating xnew with the real function |
| 5: X←X ∪ xnew |
| 6: Y←Y ∪ ynew |
| 7: ymin ←min(Y) |
| 8: xmin ←x ∈ X: y(x) = ymin |
| 9: end while |
Due to the specific properties of the EI function, its value is zero at sampled points and positive at unsampled points. By selecting the point with the highest EI value in each iteration, the EGO process ensures that the update point can be any location other than the sampled points, thereby guaranteeing convergence. However, a major limitation of the EI criterion is that it can only generate one new sample point per iteration. This restricts the EGO algorithm from performing parallel evaluations of computationally expensive models in the second stage, and the efficiency of the EI criterion decreases in the later stages of optimization, leading to poor convergence performance. Given the widespread use of parallel computing frameworks in modern engineering applications and the growing demand for reducing computational time in practical optimization processes, this study extends the EGO algorithm by incorporating the MP criterion and an improved MSE criterion. This enhancement enables the addition of multiple sample points in each iteration, thereby better exploiting parallel computing resources to further reduce computational time. The details of this approach are discussed in the following subsection.
In the traditional EGO algorithm, the second sample point cannot be updated without first evaluating the initial point, since rebuilding the Kriging surrogate model requires the updated data from the first sample. As a result, the traditional EGO global optimization algorithm evaluates designs sequentially rather than in parallel. To address this limitation, the present study incorporates new infill criteria, namely the MP and MSE criteria, to improve the efficiency of the EGO algorithm. In addition, the search space of the MSE criterion is constrained, as defined in Equation (10). Specifically, new sample points are generated within a certain spatial radius centered on the current optimal solution. For a d-dimensional problem, the initial sample points are set to 10d, evenly spread across the entire design space. The search radius is defined as half of the spatial distance between adjacent initial samples, centered on the optimal solution identified by the current surrogate model. This setting primarily aims to balance the search range and accuracy during the sample addition process.
Here,
denotes the current best sample value, and
represents the spatial radius centered on
. The procedure of this method can be summarized as Algorithm 2.
| Algorithm 2: The framework of the parallel EGO algorithm |
| Require: Initial design set (X, Y) |
| Ensure: The best observation (xmin, ymin) |
| 1: while the stop condition is not met, do |
| 2: Building a Kriging model based on the current design set (X, Y) |
| 3: xnew1 = argmin MP(x) |
| 4: xnew2 = argmax EI(x) |
| 5: xnew3 = argmax MSE(x) |
| 6: Evaluating xnew1, xnew2 and xnew3 with the real function |
| 7: X←X ∪ xnew1∪xnew2∪xnew3 |
| 8: Y←Y ∪ ynew1∪ ynew2∪ ynew3 |
| 9: ymin ←min(Y) |
| 10: xmin ←x ∈ X: y(x) = ymin |
| 11: end while |
2.4. Method Validation
Yan et al. [
45] systematically compared the optimization efficiency of the Efficient Global Optimization (EGO) algorithm with several representative optimization algorithms, including the Radial Basis Function (RBF) method, the DIviding RECTangles (DIRECT) algorithm, and Differential Evolution (DE). The optimization efficiency was assessed in terms of the number of objective function evaluations. As shown in
Table 1, the EGO algorithm consistently required fewer function evaluations than the other algorithms on most benchmark functions, demonstrating higher optimization efficiency. The only exception was the Hartman 6 function, for which the EGO algorithm required slightly more function evaluations than the RBF method. Overall, these results indicate that the EGO algorithm generally achieves higher optimization efficiency than the compared algorithms.
To highlight the limitations of individual infill criteria, the single-sample update strategies based on MSP, EI, and MSE were compared with the parallel infill strategies MSP–EI, MSP–MSE, and EI–MSE, in which one and two samples were added per iteration, respectively. The benchmark function , which possesses both a local minimum and a global minimum within the range of [0, 10], was adopted for validation. For clarity, the initial design consisted of only three sample points at 0, 5, and 10, corresponding to objective function values of 0, 4.7946, and 5.4402, respectively.
Figure 1 illustrates the convergence curves obtained using the single infill criteria. The results indicate that the MSP criterion exhibits poor convergence performance and remains trapped in a suboptimal region even after numerous iterations. This behavior arises because the initial surrogate model, established from the three initial samples, predicts the minimum objective value at x = 0. As the MSP criterion always selects the optimum of the current surrogate model for further sampling, the search process is confined to the vicinity of the current optimum and is unable to explore unexplored regions. Consequently, MSP behaves as a purely local search strategy and lacks the ability to escape from local optima. By comparison, the EI and MSE criteria converge to the global optimum after only 5–6 iterations. The primary reason is that both criteria incorporate global exploration mechanisms when determining new sampling locations. Therefore, they can effectively balance local exploitation and global exploration, enabling the optimization process to escape local optima and identify the true global optimum with significantly fewer iterations.
Figure 2 shows the convergence histories of the parallel infill criteria. It is evident that all parallel infill strategies reach the global minimum after only 2–3 iterations. In comparison with the single-point infill strategies, the number of iterations required for convergence is reduced by nearly half, while the total number of sample evaluations remains unchanged. This improvement is attributed to the simultaneous addition of multiple samples during each iteration, which enhances the information acquisition efficiency of the surrogate model. Consequently, parallel infill criteria offer substantially higher optimization efficiency and are particularly advantageous in applications where adequate computational resources are available for parallel evaluations.
To further evaluate the robustness and reliability of the proposed parallel infill strategies, five dual-point sampling approaches based on the MSP–EI, MSP–MSE, MSP–PI, MSP–WEI, and MSP–LCB criteria were investigated. The well-known Goldstein–Price (GP) benchmark function, given in Equation (11), was employed for performance assessment.
Figure 3 depicts the response surface of the GP function within the design space [−2, 2] together with the distribution of the 20 initial sample points generated using Latin Hypercube Sampling (LHS).
Figure 4 illustrates the convergence behavior of the five dual-point sampling strategies based on different parallel infill criteria. It can be seen that the MSP–EI and MSP–PI strategies begin to converge rapidly only after approximately 20 iterations. By contrast, MSP–LCB, MSP–MSE, and MSP–WEI exhibit a significantly faster convergence rate, approaching the optimum within the first 10 iterations. After this stage, these strategies mainly focus on global exploration of the design space before gradually reaching the final optimum. Despite the differences in convergence rates, all five strategies successfully converge within approximately 40 iterations. This observation demonstrates the strong robustness and reliability of the proposed parallel infill framework. By integrating a local exploitation criterion with a global exploration criterion, the parallel infill strategies effectively maintain a balance between exploitation and exploration, leading to stable and consistent optimization performance.
The foregoing results have verified the effectiveness and stability of the dual-sample parallel infill strategies. Motivated by these observations, a novel three-point parallel infill criterion, denoted as MSP–EI–MSE, is further developed in this study. The proposed strategy simultaneously incorporates the MSP, EI, and MSE criteria to select three candidate samples at each iteration, with the aim of further improving sampling efficiency while maintaining optimization accuracy.
A distinctive feature of this method is the introduction of constraints on the MSE criterion, which ensures faster convergence. To demonstrate the characteristics of the proposed technique, the test function was selected for comparison with the unconstrained MSE criterion. The minimum of this function is located at , with . The convergence condition was defined as achieving an error within ±0.2% of the minimum value. The initial sample points were set at 0, 0.5, and 1, with the corresponding values being −0.0445, −0.1229, and 0.7343, respectively.
For both the constrained and unconstrained MSE search regions, the Kriging surrogate models established from the initial sample points are shown in
Figure 5. At this stage, the error is −8.137%, which does not satisfy the convergence condition; thus, the sample points must be further updated. Under the constrained MSE criterion, the first iteration searches in the vicinity of the current optimum, ensuring high fitting accuracy around the best point. As a result, the global optimum is identified after the first iteration, with an error of −0.102% relative to the true minimum, which satisfies the convergence condition (error < 0.2%). In contrast, for the unconstrained MSE criterion, the first iteration expands the search region. While this improves global accuracy, it reduces convergence efficiency, yielding an error of −0.370% after the first iteration. Therefore, a second iteration is required to meet the convergence condition.
In summary, although the unconstrained MSE criterion explores a wider search space and can improve the overall accuracy of the surrogate model, the sampled regions are not necessarily located near the true optimum. As a result, additional iterations are required, leading to unnecessary consumption of computational resources and time.
Table 2 demonstrates the reliability of the proposed method in terms of efficiency and robustness through standard test functions.
Table 3 compares the average number of iterations and variance of the two methods across 11 benchmark functions, as presented in
Table 1. The results show that the proposed EGO algorithm significantly outperforms the EW-LCB method, with an improvement in efficiency of 22.46% and a reduction in variance of 45.14%. Further verification can be found in previously published articles [
46].