The Impact of Different Parallel Strategies on the Performance of Kriging-Based Efficient Global Optimization Algorithms

Abstract

A parallel efficient global optimization (EGO) algorithm with a pseudo expected improvement (PEI) multi-point sampling criterion, proposed in recent years, is developed to adapt the capabilities of modern parallel computing power. However, a comprehensive and clear discussion on the impact of different point-filling strategies on the optimization performance of the parallel EGO algorithm is still lacking, limiting its theoretical reference for practical applications and technological advancements. To address this gap, this study comprehensively investigates the optimization performance of the parallel EGO algorithm based on the PEI multi-point sampling criterion by analyzing the impact of different point-filling strategies under kriging surrogate models of varying fidelity. Therefore, nine benchmark test functions with different optimization problem characteristics were selected as optimization test objects, and the results were systematically analyzed from the perspectives of convergence performance, optimization efficiency, and algorithmic diversity. The analysis results indicate that the higher-fidelity kriging surrogate model enhances the stability of the parallel EGO algorithm in terms of convergence performance, optimization efficiency, and algorithmic diversity.

1. Introduction

In modern industrial design, engineers frequently encounter optimization tasks involving computationally expensive black-box problems, such as finite element analysis models and computational fluid dynamics models. With the rapid advancement of the algorithm field, numerous derivative-free and derivative-based algorithms have been developed to address this challenge [1,2]. Among them, traditional population-based algorithms such as particle swarm optimization, as well as emerging nature-inspired metaheuristic algorithms, are known to be highly reliable for optimizing black-box problems [3,4]. However, the direct application of such algorithms typically requires a large number of evaluations of the objective function, which is impractical for time-sensitive industrial product design.

To mitigate the high computational cost associated with directly applying intelligent algorithms to engineering optimization problems, surrogate-based optimization (SBO) methods have gained increasing attention over the past two decades [5]. Figure 1 illustrates the typical workflow of SBO methods. Leveraging the predictive capability of surrogate models and their low computational cost, the number of expensive objective function evaluations in the optimization process can be significantly reduced, thereby improving the overall efficiency of the optimization workflow. SBO methods have been extensively studied and applied to various problems, including aerodynamic performance optimization [6,7] and structural optimization [8,9].

The kriging-based efficient global optimization (EGO) algorithm is one of the most successful SBO methods [10,11]. The kriging surrogate model not only provides predictions of the objective function but also quantifies the uncertainty of these predictions. Accordingly, the EGO algorithm incorporates an adaptive sampling criterion, known as expected improvement (EI), to guide the selection of new sample points. On one hand, the sampling criterion identifies candidate points with the highest probability of improvement; on the other hand, it enhances the predictive accuracy in the vicinity of the selected points. The optimization process terminates once the stopping criterion is met. The traditional EGO algorithm has been extensively studied and extended to address not only simple single-objective optimization problems [12,13] but also complex challenges such as multi-objective optimization [14,15,16], numerical noise optimization [17,18], constrained optimization [19,20,21], and gradient-based optimization [22]. However, a major limitation of the classical EGO algorithm is that it identifies only a single promising candidate point per optimization cycle. For automotive aerodynamic optimization problems, a complete point-filling process requires approximately 8 h of computational time [18], including 5 h for CFD numerical simulations on a 576-core supercomputer and 3 h for mesh preparation and queuing delays prior to evaluation [18]. Under such circumstances, when only one design sample point can be evaluated per optimization cycle, despite abundant available computing resources, the approach fails to effectively utilize the parallel computing capacity. This limitation substantially restricts the potential reduction in wall-clock time and consequently hinders optimization efficiency improvement.

To overcome this limitation, researchers have explored various approaches to extend the classical serial EGO algorithm [23,24,25], enabling the generation of multiple candidate points within a single optimization cycle. As a result, the overall optimization process can be significantly accelerated by reducing the number of required optimization cycles. Among various multi-point sampling criteria, the pseudo expected improvement (PEI) criterion exhibits relatively high reliability. It employs additional influence functions (IFs) to approximate the impact of newly sampled points on the original EI criterion, thereby avoiding costly evaluations of new points and frequent updates of the kriging model. Performance evaluations have demonstrated that the EGO algorithm based on the PEI criterion exhibits strong robustness and effective global optimization capability. However, it is known that a major limitation of the parallel EGO algorithm based on the PEI criterion is that as the number of candidate points increases within a single optimization cycle, the improvement in parallel optimization efficiency gradually slows down. A possible explanation proposed by researchers is that as more IFs are multiplied by the initial EI function, the reliability of the PEI function gradually decreases. Consequently, the likelihood of identifying high-quality candidate points diminishes. Currently, no systematic study has been conducted to investigate the impact of the number of candidate points per optimization cycle on the performance of the EGO algorithm. Key aspects such as the quality of the converged solution, convergence speed, and variations in algorithmic diversity remain unexplored. Furthermore, existing numerical studies are based on the ordinary kriging (OK) model, without exploring how the number of candidate points affects the performance of the EGO algorithm under different kriging models, such as the gradient-enhanced kriging (GEK) model.

This study aims to enhance the practical applicability and further development of the EGO algorithm based on the PEI criterion. A comparative analysis was conducted on nine analytical functions with different characteristics to investigate the impact of the number of candidate points on the performance of the PEI-based EGO algorithm using both the OK and GEK models. A systematic and in-depth evaluation was performed from multiple perspectives, including global optimization capability, convergence speed, optimization efficiency in achieving the desired convergence criteria, algorithmic diversity, and the balance between global exploration and local exploitation.

The remainder of this paper is organized as follows: Section 2 provides a brief overview of the methodology of the parallel EGO algorithm and details the design of the comparative numerical experiments. Section 3 presents the experimental results and discussion. Finally, Section 4 presents the conclusions and recommendations.

2. Materials and Methods

2.1. Ordinary Kriging

Kriging was first introduced by Sacks et al. as a surrogate for expensive computational models [26]. Given n initial training points, the response of the OK model can typically be expressed as follows:

$$\widehat{y}\left(x\right)=\widehat{\mu }+\widehat{ϵ}\left(x\right)$$

(1)

Here, $\widehat{\mu }$ represents the mean term, which is determined by solving a weighted least squares problem:

$$\widehat{\mathit{\mu }}={\left({\mathit{F}}^{\mathit{T}}{\mathit{R}}^{\mathbf{-}\mathbf{1}}\mathit{F}\right)}^{\mathbf{-}\mathbf{1}}{\mathit{F}}^{\mathit{T}}{\mathit{R}}^{\mathbf{-}\mathbf{1}}\mathit{y}$$

(2)

Here, $\mathit{F}$ is the regression matrix representing the zero-order basis function. In the OK model, the polynomial vector of the basis function can be expressed as follows:

$$\mathit{F}=\mathbf{1}=\underset{\mathit{n}}{{\underbrace{\left[\mathbf{1},\cdots ,\mathbf{1}\right]}}^{\mathit{T}}}$$

(3)

The correlation matrix $\mathit{R}$ characterizes the spatial correlation among all training points:

$$\mathit{R}=\left[\begin{array}{ccc}\mathit{\psi }\left({\mathit{x}}^{\mathbf{1}},{\mathit{x}}^{\mathbf{1}}\right)& \dots & \mathit{\psi }\left({\mathit{x}}^{\mathbf{1}},{\mathit{x}}^{\mathit{n}}\right)\\ ⋮& \ddots & ⋮\\ \mathit{\psi }\left({\mathit{x}}^{\mathit{n}},{\mathit{x}}^{\mathbf{1}}\right)& \dots & \mathit{\psi }\left({\mathit{x}}^{\mathit{n}},{\mathit{x}}^{\mathit{n}}\right)\end{array}\right]$$

(4)

Here, the Gaussian function is selected as the modeling function for spatial correlation:

$$\psi \left(x^i,x^j\right)=exp\left[\sum _{k=1}^n-{\theta }_k{\left(‖x_k^i-x_k^j‖\right)}^2\right]$$

(5)

${\theta }_k$ is a critical hyper-parameter in kriging, and its value directly influences the quality of the kriging fit. As seen from the expression, ${\theta }_k$ scales the influence of the $k_{th}$ design variable on the correlation between training points. Figure 2 illustrates the variation in correlation with respect to the distance between training points under different hyper-parameter values. The determination of hyper-parameters requires optimization using the maximum likelihood estimation (MLE) method to achieve the optimal fitting trend for the training points.

The error term $\widehat{ϵ}\left(x\right)$ in Equation (1) is treated as a realization of a stochastic process and can be expressed as follows:

$$\widehat{ϵ}\left(x\right)={\mathit{w}}^T\mathit{r}\left(x\right)$$

(6)

Here, the correlation vector $\mathit{r}$ describes the spatial correlation between the training points and the prediction point:

$$\mathit{r}=\left[\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathbf{1}}\right),\dots ,\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathit{n}}\right)\right]$$

(7)

The determination of $\mathit{w}$ requires solving a system of linear equations:

$$\mathit{w}={\mathit{R}}^{\mathbf{-}\mathbf{1}}\left(\mathit{y}\mathbf{-}\mathit{F}\widehat{\mathit{\mu }}\right)$$

(8)

The final kriging predictor can be expressed as follows:

$$\widehat{\mathit{y}}\left({\mathit{x}}^{\mathit{n}\mathbf{+}\mathbf{1}}\right)=\widehat{\mathit{\mu }}+{\mathit{w}}^{\mathit{T}}\mathit{r}$$

(9)

The mean square error of the prediction made by the kriging predictor is given by the following:

$$s^2\left(x^{n+1}\right)={\widehat{\sigma }}^2\left[1-{\mathit{r}}^T{\mathit{R}}^{-1}\mathit{r}+{\displaystyle \frac{{\left(1-{\mathit{F}}^T{\mathit{R}}^{-1}\mathit{r}\right)}^2}{{\mathit{F}}^T{\mathit{R}}^{-1}\mathit{F}}}\right]$$

(10)

Here, ${\widehat{\sigma }}^2$ represents the predicted system variance:

$${\widehat{\sigma }}^2={\displaystyle \frac{1}{n}}{\left(\mathit{y}-\mathit{F}\mu \right)}^T{\mathit{R}}^{-1}\left(\mathit{y}-\mathit{F}\mu \right)$$

(11)

The objective function values at the unknown points can be predicted by solving the linear equation systems (2), (8), (10), and (11).

2.2. Gradient-Enhanced Kriging

If the objective function values and sensitivity information of the corresponding points are available, incorporating gradient information into the construction of the kriging model can significantly improve the accuracy of fitting [27]. Figure 3 presents an example of fitting the two-dimensional Griewank function using both OK and GEK, with the same number of training points. Clearly, the latter achieves higher fitting accuracy. The GEK model is an extension of the OK model. To fit the GEK model, the correlation terms are also extended to include the gradient terms. First, the correlation matrix is expanded as follows:

$$\mathit{R}=\left[\begin{array}{cc}{\mathit{Q}}^{\mathbf{1},\mathbf{1}}& {\mathit{Q}}^{\mathbf{1},\mathbf{2}}\\ {\left({\mathit{Q}}^{\mathbf{1},\mathbf{2}}\right)}^{\mathit{T}}& {\mathit{Q}}^{\mathbf{2},\mathbf{2}}\end{array}\right]$$

(12)

where ${\mathit{Q}}^{1,1}$ is the correlation matrix that does not include gradient information:

$${\mathit{Q}}^{1,1}=\left[\begin{array}{ccc}\psi \left(x^1,x^1\right)& \dots & \psi \left(x^1,x^n\right)\\ ⋮& \ddots & ⋮\\ \psi \left(x^n,x^1\right)& \dots & \psi \left(x^n,x^n\right)\end{array}\right]$$

(13)

${\mathit{Q}}^{1,2}$ is the extended correlation matrix that includes the first-order gradient information of $\mathit{R}$:

$${\mathit{Q}}^{1,2}=\left[\begin{array}{ccc}{\displaystyle \frac{\partial \psi \left(x^1,x^1\right)}{\partial x^1}}& \dots & {\displaystyle \frac{\partial \psi \left(x^1,x^n\right)}{\partial x^n}}\\ ⋮& \ddots & ⋮\\ {\displaystyle \frac{\partial \psi \left(x^n,x^1\right)}{\partial x^1}}& \dots & {\displaystyle \frac{\partial \psi \left(x^n,x^n\right)}{\partial x^n}}\end{array}\right]$$

(14)

${\mathit{Q}}^{2,2}$ is the extended correlation matrix that includes the second-order gradient information of $\mathit{R}$:

$${\mathit{Q}}^{2,2}=\left[\begin{array}{ccc}{\displaystyle \frac{{\partial }^2\psi \left(x^1,x^1\right)}{\partial x^1\partial x^1}}& \dots & {\displaystyle \frac{{\partial }^2\psi \left(x^1,x^n\right)}{\partial x^1\partial x^n}}\\ ⋮& \ddots & ⋮\\ {\displaystyle \frac{{\partial }^2\psi \left(x^n,x^1\right)}{\partial x^n\partial x^1}}& \dots & {\displaystyle \frac{{\partial }^2\psi \left(x^n,x^n\right)}{\partial x^n\partial x^n}}\end{array}\right]$$

(15)

The spatial correlation between the prediction point and the training points is also extended in the same manner as follows:

$$\mathit{r}=\left[\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathbf{1}}\right),\dots ,\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathit{n}}\right),{\displaystyle \frac{\mathbf{\partial }\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathbf{1}}\right)}{\mathbf{\partial }{\mathit{x}}^{\mathbf{1}}}},\dots ,{\displaystyle \frac{\mathbf{\partial }\mathit{\psi }\left({\mathit{x}}^{\mathit{n}+\mathbf{1}},{\mathit{x}}^{\mathit{n}}\right)}{\mathbf{\partial }{\mathit{x}}^{\mathit{n}}}}\right]$$

(16)

Similarly, the extension of the function value vector also includes the gradient information of the response:

$$\mathit{y}=\left[\mathit{f}\left({\mathit{x}}^{\mathbf{1}}\right),\dots ,\mathit{f}\left({\mathit{x}}^{\mathit{n}}\right),{\displaystyle \frac{\mathbf{\partial }\mathit{f}\left({\mathit{x}}^{\mathbf{1}}\right)}{\mathbf{\partial }{\mathit{x}}^{\mathbf{1}}}},\dots ,{\displaystyle \frac{\mathbf{\partial }\mathit{f}\left({\mathit{x}}^{\mathit{n}}\right)}{\mathbf{\partial }{\mathit{x}}^{\mathit{n}}}}\right]$$

(17)

The extended expression for the regression matrix $\mathit{F}$ of the zero-order basis function is given by

$$\mathit{F}=\left[\mathbf{1} \mathbf{0}\right]=\underset{\mathit{n}+\mathit{n}\times \mathit{n}}{{\underbrace{\left[\mathbf{1},\cdots ,\mathbf{1},\mathbf{0},\cdots ,\mathbf{0}\right]}}^{\mathit{T}}}$$

(18)

The mean square error of the prediction made by the GEK predictor is given by

$$s^2\left(x^{n+1}\right)={\widehat{\sigma }}^2\left[1-{\mathit{r}}^T{\mathit{R}}^{-1}\mathit{r}+{\displaystyle \frac{{\left(1-{\mathit{F}}^T{\mathit{R}}^{-1}\mathit{r}\right)}^2}{{\mathit{F}}^T{\mathit{R}}^{-1}\mathit{F}}}\right]$$

(19)

The predicted system variance is given by

$${\widehat{\sigma }}^2={\displaystyle \frac{1}{n\left(1+n\right)}}{\left(\mathit{y}-\mathit{F}\mu \right)}^T{\mathit{R}}^{-1}\left(\mathit{y}-\mathit{F}\mu \right)$$

(20)

The objective function values at the unknown points can be predicted by solving the linear equation systems (2), (8), (19), and (20). It is important to note that, because GEK improves fitting accuracy by augmenting the correlation vectors and matrices, the computational cost of constructing the GEK model increases accordingly. However, this cost remains relatively low compared to the expensive evaluation experiments and the wall-clock time costs associated with practical engineering optimization cycles.

2.3. Expected Improvement Multi-Points Infill Criterion

Kriging not only predicts the objective function values at unknown points but also provides the uncertainty of the predictions. This makes kriging well-suited to be paired with an acquisition function to enhance optimization efficiency. The kriging predictor treats the predicted objective function value at an unknown point as a realization of a stochastic process, which follows a normal distribution with a mean of $\widehat{y}\left(x\right)$ and a variance of $s^2\left(x\right)$. Based on this, the criterion for identifying unknown points with potential for improvement can be expressed as [10]

$$EI\left(x\right)=\left(y_{\mathit{min}}-\widehat{y}\left(x\right)\right)\mathit{\Phi }\left({\displaystyle \frac{y_{\mathit{min}}-\widehat{y}\left(x\right)}{s\left(x\right)}}\right)+s\left(x\right)\varphi \left({\displaystyle \frac{y_{\mathit{min}}-\widehat{y}\left(x\right)}{s\left(x\right)}}\right)$$

(21)

Here, $\mathit{\Phi }\left(x\right)$ and $\varphi \left(x\right)$ represent the cumulative distribution function and the probability density function of the standard normal distribution, respectively. The EI function consists of two terms: the first term represents local exploitation near the current optimal value, while the second term represents global exploration in sparsely sampled regions. Therefore, the EI function maintains a perfect balance between local exploitation and global exploration. However, a significant limitation of the EI function is that it can only identify one candidate point within a single optimization cycle. This prevents it from fully utilizing external parallel computing resources to reduce the wall-clock time cost during the optimization process. To overcome this limitation, the PEI criterion proposes using an additional influence function to simulate the impact of the updated points on the behavior of the original EI function, thereby eliminating the costly evaluation of the update points and the Kriging model update within a single optimization cycle. As illustrated in Figure 4, a comparative analysis of the approximation and discrepancies between the EI and PEI functions during point infill on a one-dimensional Griewank’s function reveals several key observations. Figure 4a demonstrates that the EI function derived from the updated kriging model and the PEI function obtained using the IF function exhibit generally consistent variation trends. The IF function effectively captures the characteristic behavior of the EI function, where the influence of updated points on the EI function diminishes with increasing distance from the update location in the design space. More notably, in this particular case, the PEI function demonstrates superior convergence performance compared to its EI counterpart. This enhanced performance stems precisely from the subtle differences introduced by the approximation process, which enable the PEI function to overcome the well-documented limitation of conventional EI functions—their tendency to become trapped in local optima. The observed phenomenon suggests that the intentional approximation employed in the PEI formulation not only maintains the essential properties of the EI function but also introduces beneficial characteristics that improve global search capability. The expression for the PEI criterion is given by [24]

$$PEI\left(x,q-1\right)=EI\left(x\right)\times IF\left(x,x^{n+1}\right)\times \cdots \times IF\left(x,x^{n+q-1}\right)$$

(22)

where $q$ represents the number of candidate points to be identified within a single optimization cycle. The IF function is used to simulate the impact of the updated points on the initial EI function and can be computed using the following expression:

$$IF\left(x,x^{n+1}\right)=1-Corr\left[Y\left(x\right),Y\left(x^{n+1}\right)\right]$$

(23)

Here, $x^{n+1}$ denotes the position of the candidate point, and $Corr\left[.\right]$ is the spatial correlation function used in the construction of kriging. The value of the PEI function at the update points is zero, and as the distance between the unknown point and the update point increases, the value of the PEI function gradually increases to one. This means that the EI function is significantly affected in the vicinity of the update point, while areas far from the update point are not noticeably impacted. This characteristic ensures that the variation of the PEI function across the entire search space closely approximates the influence of the update points on the initial EI function. In high-dimensional problems, where functional relationships exhibit extreme complexity, the current IF function considers only spatial correlations between sample points and therefore captures relatively limited information. During the approximation process, the IF function may potentially lose many critical pieces of information. Consequently, whether the IF function can perfectly approximate the true EI function in high-dimensional spaces remains an open question that warrants further investigation.

2.4. Kriging-Based Parallel EGO Algorithm System

The combination of the kriging surrogate model with the expected improvement sampling criterion constitutes an efficient global optimization (EGO) algorithm. Figure 5 presents the framework of the parallel EGO algorithm system based on the kriging surrogate model. First, a design of experiment (DOE) method is used to generate the initial sample points for evaluation. Then, computationally expensive experiments are conducted to evaluate the objective values of each sample point, forming the initial training dataset for fitting the kriging model. Next, an external optimizer is employed to optimize the hyper-parameters of the kriging model, thereby constructing the surrogate model. Subsequently, based on the current kriging model and the multi-points sampling criterion, the external optimizer is utilized to identify multiple promising candidate points. The objective values of these candidate points are then re-evaluated. If the optimization of the objective values meets the predefined requirements, the optimization process terminates. Otherwise, the kriging model is updated, and the iterative optimization continues until the stopping criterion of the optimization process is satisfied.

2.5. Materials for Numerical Simulation Tests

To comprehensively investigate the impact of the number of infill points on the performance of the kriging-based EGO algorithm system, a comparative experimental study was conducted under different levels of infill points. The optimization performance of the EGO algorithm system based on both the OK model and the GEK model was systematically evaluated.

Table 1. Details of benchmark test functions.

Benchmark Test Functions	Design Spaces	Optimum
$\mathrm{F}1\left(\mathrm{x}\right)={\displaystyle \sum _1^{D-1}}{(1-x_i)}^2+100{(x_{i+1}-x_i^2)}^2$	$\mathrm{x}\in {\left[-5,10\right]}^D$	0
$\mathrm{F}2\left(\mathrm{x}\right)=1+{\displaystyle \frac{1}{4000}}{\displaystyle \sum _1^D}{x}_i^2-{\displaystyle \prod _{i=1}^D}\mathit{cos}\left({\displaystyle \frac{x_i}{\sqrt{i}}}\right)$	$\mathrm{x}\in {\left[-50,50\right]}^D$	0
$\mathrm{F}3\left(\mathrm{x}\right)=1+{\displaystyle \sum _1^D}{\displaystyle \frac{z_i^2}{4000}}-{\displaystyle \prod _{i=1}^D}\mathit{cos}\left({\displaystyle \frac{z_i}{\sqrt{i}}}\right)-500,z={\Lambda }^{100}{M}_1{\displaystyle \frac{600(x-o)}{100}}$	$\mathrm{x}\in {\left[-100,100\right]}^D$	−500
$\mathrm{F}4\left(\mathrm{x}\right)={\displaystyle \sum _1^D}\left(z_i^2-10\mathit{cos}(2\pi z_i\right)+10)-400,z={\Lambda }^{10}{T}_{asy}^{0.2}(T_{osz}({\displaystyle \frac{5.12\left(x-o\right)}{100}}))$	$\mathrm{x}\in {\left[-100,100\right]}^D$	−400
$\mathrm{F}5\left(\mathrm{x}\right)={\displaystyle \sum _1^D}\left(z_i^2-10\mathit{cos}\left(2\pi z_i\right)+10\right)-300,$ $z={M_1\Lambda }^{10}{M_{2}T}_{asy}^{0.2}(T_{osz}(M_1{\displaystyle \frac{5.12\left(x-o\right)}{100}}))$	$\mathrm{x}\in {\left[-100,100\right]}^D$	−300
$\mathrm{F}6\left(\mathrm{x}\right)=418.9829\times D-{\displaystyle \sum _1^D}g\left(z_i\right)-100,$ $z={\Lambda }^{10}\left({\displaystyle \frac{1000\left(x-o\right)}{100}}\right)+4.209687462275036e+002,$ $g\left(z_i\right)=\left\{\begin{array}{c}{z}_i\mathit{sin}\left({\left|z_i\right|}^{{\displaystyle \frac{1}{2}}}\right),if\left|z_i\right|\le 500\\ (500-mod(z_i,500))\mathit{sin}\left(\sqrt{\left|500-mod\left(z_i,500\right)\right|}\right)-{\displaystyle \frac{({z_i-500)}^2}{10000D}},if{z}_i>500\\ (mod\left(\left|z_i\right|,500\right)-500)\mathit{sin}\left(\left(\sqrt{\left|mod\left(\left|z_i\right|,500\right)-500\right|}\right)\right)-{\displaystyle \frac{({z_i+500)}^2}{10000D}},if{z}_i<500\end{array}\right.$	$\mathrm{x}\in {\left[-100,100\right]}^D$	−100
$\mathrm{F}7\left(\mathrm{x}\right)=418.9829\times D-{\displaystyle \sum _1^D}g\left(z_i\right)+100,$ $z={\Lambda }^{10}{M}_1\left({\displaystyle \frac{1000\left(x-o\right)}{100}}\right)+4.209687462275036e+002,$ $g\left(z_i\right)=\left\{\begin{array}{c}{z}_i\mathit{sin}\left({\left|z_i\right|}^{{\displaystyle \frac{1}{2}}}\right),if\left|z_i\right|\le 500\\ (500-mod(z_i,500))\mathit{sin}\left(\sqrt{\left|500-mod\left(z_i,500\right)\right|}\right)-{\displaystyle \frac{({z_i-500)}^2}{10000D}},if{z}_i>500\\ (mod\left(\left|z_i\right|,500\right)-500)\mathit{sin}\left(\left(\sqrt{\left|mod\left(\left|z_i\right|,500\right)-500\right|}\right)\right)-{\displaystyle \frac{({z_i+500)}^2}{10000D}},if{z}_i<500\end{array}\right.$	$\mathrm{x}\in {\left[-100,100\right]}^D$	100
$\mathrm{F}8\left(\mathrm{x}\right)=g_1\left(g_2\left(z_1,z_2\right)\right)+g_1\left(g_2\left(z_2,z_3\right)\right)+\cdots +g_1\left(g_2\left(z_{D-1},z_D\right)\right)+g_1\left(g_2\left(z_D,z_1\right)\right)+500,$ $g_1\left(x\right)=1+{\displaystyle \frac{1}{4000}}{\displaystyle \sum _1^D}{x}_i^2-{\displaystyle \prod _{i=1}^D}\mathit{cos}\left({\displaystyle \frac{x_i}{\sqrt{i}}}\right),g_2\left(x\right)={\displaystyle \sum _1^{D-1}}{\left(1-x_i\right)}^2+100{\left(x_{i+1}-x_i^2\right)}^2,$ $z=M_1{\displaystyle \frac{5\left(x-o\right)}{100}}+1$	$\mathrm{x}\in {\left[-100,100\right]}^D$	500
$\mathrm{F}9\left(\mathrm{x}\right)={\displaystyle \sum _1^n}\left\{{\omega }_i\left[{\lambda }_i{g}_i\left(x\right)+{bias}_i\right]\right\}+800,n=3,{\omega }_i={\displaystyle \frac{w_i}{{\sum }_{i=1}^n{w}_i}},$ $w_i={\displaystyle \frac{1}{\sqrt{{\sum }_{j=1}^D{\left(x_j-o_{ij}\right)}^2}}}\exp \left(-{\displaystyle \frac{{\sum }_{j=1}^D{\left(x_j-o_{ij}\right)}^2}{2D{\sigma }_i^2}}\right),\sigma =\left[\mathrm{20,20,20}\right],\lambda =\left[\mathrm{1,1},1\right],$ $bias=\left[\mathrm{0,100,200}\right],g_{1-3}:Schwefel’sfunction$	$\mathrm{x}\in {\left[-100,100\right]}^D$	800

presents nine benchmark functions, selected from a benchmark function suit specifically designed for evaluating the performance of optimization algorithms [28], including their mathematical formulations, design space specifications, and corresponding global optima, all of which are derived from authoritative references in the field. The tested functions all have a dimensionality of five. To reduce the impact of randomness in the initial points and stochastic parameters in the optimization process, the tested experiments were repeated 30 times using 30 sets of 40 initial points generated by optimal Latin hypercube design (OLHD) [29]. In each case, the EGO algorithm based on OK and GEK was allowed to perform an additional 240 and 120 evaluations, respectively, on the tested function. This configuration is implemented to ensure both OK-based and GEK-based EGO algorithms converge to relatively high-quality solutions, with the OK-based EGO algorithm’s iteration count set twice that of GEK-based EGO due to its comparatively slower convergence speed. In each optimization cycle, the EGO algorithm generates 2, 4, 6, 8, and 10 candidate points, respectively. This means that the EGO algorithm based on the OK model has a maximum of 120 optimization cycles, while the EGO algorithm based on the GEK model has a maximum of 60 optimization cycles.

In the numerical simulation tests, a Python (3.9.0) toolbox (0.6.2), named VAEOpy (Vehicular Aerodynamic Engineering Optimization in Python) [30], was used to fit both OK and GEK models and to perform optimization using the parallel EGO algorithm. The corresponding parameter settings are as follows [31]:

• Regression function: regpoly0.
• Correlation function: corrgauss.
• Feasible domain of hyper-parameters: [10⁻⁵, 10⁵].
• Optimizer for tuning hyper-parameters: GS-SQP; maximum number of iterations: 1000.

Optimizer for searching candidate points: MGBDE; population size: 50; maximum number of iterations: 500.

3. Experimental Results and Discussion

3.1. Comparative Analysis of the Optimization Performance of the Parallel EGO Algorithm

3.1.1. Optimization Solution Accuracy

A comparative analysis was conducted to examine the impact of different point-filling quantities on the optimization solution accuracy of the parallel EGO algorithm based on the OK model and the GEK model within a limited total number of evaluations.

Table 2. Statistical results of the converged solutions obtained by the parallel EGO algorithms based on OK and GEK models under different point-filling strategies.

Systems	Test Problems	Metrics	2 Points	4 Points	6 Points	8 Points	10 Points
OK-based	F1	Mean	4.509 × 10−1	5.669 × 10−1	6.046 × 10−1	6.907 × 10−1	7.791 × 10−1
		Std	7.865 × 10−1	6.880 × 10−1	4.560 × 10−1	6.826 × 10−1	5.662 × 10−1
		Rank (p = 0.0019)	2.00	3.03	3.30	3.10	3.57
GEK-based		Mean	4.49 × 10−2	9.18 × 10−2	1.220 × 10−1	1.415 × 10−1	2.719 × 10−1
		Std	3.70 × 10−2	7.06 × 10−2	1.172 × 10−1	9.55 × 10−2	3.565 × 10−1
		Rank (p < 0.0001)	1.87	2.57	3.03	3.53	4.00
OK-based	F2	Mean	4.642 × 10−1	4.967 × 10−1	4.551 × 10−1	4.698 × 10−1	5.096 × 10−1
		Std	1.153 × 10−1	1.118 × 10−1	8.56 × 10−2	1.151 × 10−1	1.058 × 10−1
		Rank (p = 0.2998)	2.97	3.20	2.57	2.87	3.40
GEK-based		Mean	5.479 × 10−1	5.375 × 10−1	5.501 × 10−1	5.312 × 10−1	5.635 × 10−1
		Std	1.290 × 10−1	1.304 × 10−1	1.107 × 10−1	1.094 × 10−1	1.351 × 10−1
		Rank (p = 0.3578)	3.03	2.97	2.90	2.63	3.47
OK-based	F3	Mean	−4.9934 × 102	−4.9936 × 102	−4.9940 × 102	−4.9936 × 102	−4.9938 × 102
		Std	1.858 × 10−1	1.998 × 10−1	2.199 × 10−1	1.658 × 10−1	1.921 × 10−1
		Rank (p = 0.6387)	3.27	3.03	2.73	3.17	2.80
GEK-based		Mean	−4.9518 × 102	−4.9483 × 102	−4.9400 × 102	−4.9388 × 102	−4.9359 × 102
		Std	1.9765	1.7859	2.5571	2.4633	2.2423
		Rank (p = 0.0316)	2.43	2.60	3.20	3.20	3.57
OK-based	F4	Mean	−3.9797 × 102	−3.9754 × 102	−3.9730 × 102	−3.9694 × 102	−3.9674 × 102
		Std	8.011 × 10−1	1.3583	1.6863	2.0484	1.8026
		Rank (p = 0.0023)	2.23	2.87	2.83	3.23	3.83
GEK-based		Mean	−3.9818 × 102	−3.9840 × 102	−3.9817 × 102	−3.9807 × 102	−3.9806 × 102
		Std	8.714 × 10−1	8.988 × 10−1	1.0087	1.0889	1.0464
		Rank (p = 0.0316)	3.20	2.20	2.97	3.30	3.33
OK-based	F5	Mean	−2.9684 × 102	−2.9566 × 102	−2.9585 × 102	−2.9730 × 102	−2.9666 × 102
		Std	3.7693	4.9763	4.5262	1.3679	2.1963
		Rank (p = 0.2073)	2.67	3.10	3.47	2.63	3.13
GEK-based		Mean	−2.9765 × 102	−2.9735 × 102	−2.9726 × 102	−2.9749 × 102	−2.9765 × 102
		Std	1.1496	1.2659	1.3008	9.568 × 10−1	1.0123
		Rank (p = 0.3173)	2.63	3.13	3.40	3.10	2.73
OK-based	F6	Mean	−6.444 × 101	−1.817 × 101	−4.460 × 101	−2.078 × 101	−1.944 × 101
		Std	6.382 × 101	1.0328 × 102	1.3318 × 102	9.513 × 101	1.1667 × 102
		Rank (p = 0.0086)	2.50	3.40	2.37	3.53	3.20
GEK-based		Mean	−8.770 × 101	−9.324 × 101	−8.852 × 101	−8.099 × 101	−9.426 × 101
		Std	3.885 × 101	2.106 × 101	2.939 × 101	4.975 × 101	8.0437
		Rank (p = 0.6868)	3.30	2.80	3.13	2.80	2.97
OK-based	F7	Mean	2.0104 × 102	1.7902 × 102	1.8019 × 102	1.7962 × 102	1.7694 × 102
		Std	1.4379 × 102	1.0624 × 102	1.0799 × 102	1.0126 × 102	1.0803 × 102
		Rank (p = 0.8136)	3.13	2.73	2.97	2.97	3.20
GEK-based		Mean	1.3460 × 102	1.4083 × 102	1.3083 × 102	1.3919 × 102	1.3518 × 102
		Std	6.349 × 101	6.999 × 101	5.526 × 101	6.968 × 101	6.090 × 101
		Rank (p = 0.9197)	3.23	2.93	2.97	2.87	3.00
OK-based	F8	Mean	5.0284 × 102	5.0311 × 102	5.0334 × 102	5.0351 × 102	5.0351 × 102
		Std	6.669 × 10−1	8.299 × 10−1	9.720 × 10−1	9.606 × 10−1	1.3396
		Rank (p = 0.0047)	2.17	2.57	3.40	3.37	3.50
GEK-based		Mean	5.0500 × 102	5.0480 × 102	5.0476 × 102	5.0566 × 102	5.0515 × 102
		Std	1.7700	1.3614	1.3429	2.0348	1.3429
		Rank (p = 0.6828)	2.90	2.63	2.83	3.33	3.20
OK-based	F9	Mean	9.7651 × 102	1.0291 × 103	1.0683 × 103	1.0344 × 103	1.0929 × 103
		Std	1.6718 × 102	1.3887 × 102	1.3285 × 102	1.5219 × 102	1.5950 × 102
		Rank (p = 0.0071)	2.20	2.83	3.47	2.97	3.53
GEK-based		Mean	9.5022 × 102	9.5390 × 102	9.6924 × 102	9.3789 × 102	9.4671 × 102
		Std	8.041 × 101	8.716 × 101	1.0428 × 102	7.287 × 101	8.207 × 101
		Rank (p = 0.8088)	2.97	3.17	3.20	2.90	2.77

presents the statistical results of the optimized solutions for all test cases, along with the Friedman ranking across the test cases [32]. A p-value less than 0.05 indicates statistically significant differences in solution accuracy between different point-filling strategies at the specified significance level, whereas values exceeding this threshold demonstrate no statistically significant performance variations.

The results indicate that, for the case based on the OK model, filling two update points per optimization cycle achieved the most accurate converged solutions for test functions F1, F4, F6, F8, and F9. Furthermore, the significance levels (p-values) reveal statistically significant differences in solution accuracy among different point-filling strategies for these test functions. Additionally, for F5, it also obtained the second-highest ranked approximate optimal solution in terms of accuracy. The observed phenomenon may be attributed to the high-dimensional, multimodal characteristics of these functions, where a relatively small number of filling points effectively avoids local optima traps while maintaining convergence efficiency by preventing excessive exploration. In contrast, as the number of filled points increased to 10, the algorithm yielded the least accurate converged solutions for almost all test functions, except for F3 and F6. This suggests that the stability of the optimization performance of the parallel EGO algorithm based on the OK model deteriorates as the number of filled points increases. However, overall, as the number of filled points increased from 2 to 10, the change in the accuracy of the converged solutions did not follow a strictly linear decline; instead, there were fluctuations within certain intervals. For test functions F2 and F6, the performance with six filling points proves superior to that with four points. This phenomenon similarly manifests in functions F5 and F9, where eight filling points demonstrate better performance than six. The underlying rationale may lie in the fact that intermediate quantities (6–8 points) in certain specific problems enable the parallel EGO algorithm to escape particular types of local optima traps while maintaining an optimal balance between global exploration and local exploitation.

It is noteworthy that the impact of point-filling quantity on parallel EGO algorithm performance varies significantly across different functions. For instance, the performance differences between various filling quantities nearly disappear when comparing the original Griewank’s function (F2) with its rotated version (F3). This phenomenon likely stems from how rotation weakens the original function’s variable coupling structure (as illustrated in Figure 6, showing 3D maps of both versions), thereby reducing optimization process sensitivity to sampling strategies. A similar pattern emerges between the original Schwefel’s function (F6) and its rotated counterpart (F7). In contrast, for Rastrigin functions (F4, F5), the algorithm’s performance remains highly sensitive to filling quantities regardless of rotation, with two-point filling demonstrating consistent performance—a characteristic potentially related to the function’s strong multimodality (visualized in Figure 7’s 3D maps). An appropriate filling quantity balances avoidance of local optima traps with maintained exploration efficiency. These findings suggest that-filling strategy selection should consider objective function landscape characteristics: more flexible approaches may suit rotation-sensitive functions (e.g., Griewank-class), while rotation-invariant problems (e.g., Rastrigin-class) may benefit from the robust two-point-filling strategy.

For the case based on the GEK model, the optimization performance advantage of the strategy with two filled points decreased. It only achieved the best convergence accuracy on test functions F1, F3, and F5, while performing relatively poorly on several other test functions. Particularly, the convergence accuracy obtained on F2, F6, and F7 was the worst. Although the strategy with a point-filling number of 10 still performs poorly on most test functions compared to other filling strategies, the overall accuracy of its convergence solutions has improved, with a reduction in the differences. Additionally, except for F1 and F3, the variation in the accuracy of the convergence solutions does not exhibit a linear decrease as the point-filling number increases. It is worth noting that, based on the statistical decision metric p-value, the difference in the accuracy of the converged solutions has significantly reduced, compared to the case based on OK model, except for F1 and F3. This indicates that the stability of the optimization performance of the parallel EGO algorithm based on GEK model has been better ensured.

3.1.2. Convergence Speed of Optimization

To investigate the impact of different point-filling quantities on the convergence speed of the parallel EGO algorithm, Figure 8 and Figure 9 present the average convergence curves of the parallel EGO algorithm based on the OK and GEK models, respectively, under different point-filling strategies.

From Figure 8, it can be observed that for the case based on the OK model, the convergence speed varies significantly across different point-filling quantities for almost all test functions, except for F3. A point-filling quantity of two exhibits an absolute performance advantage in terms of convergence speed for test functions F1, F4, F6, F8, and F9. This conclusion can be drawn from the fact that the average convergence curve for a point-filling quantity of two remains consistently below the other average convergence curves throughout the optimization process. In contrast, the average convergence speed for a point-filling quantity of 10 is relatively slower across most test functions. The average convergence speed for other point-filling quantities exhibits a certain degree of fluctuation between these two cases. This result is generally consistent with the findings on the accuracy of the optimized solutions presented above. This indicates that the optimization performance of the parallel EGO algorithm based on OK model is most reliable when the point-filling quantity is set to two. However, as the point-filling quantity gradually increases to 10, the stability of the parallel EGO algorithm’s optimization performance deteriorates significantly.

From the average convergence curves in Figure 9, it can be observed that in the case of the GEK model, the impact of different point-filling quantities on the convergence speed of the parallel EGO algorithm differs only in the early stages of the optimization process for almost all test functions, except for F3 and F8. Moreover, during this stage, the impact on the average convergence speed exhibits a trend that is essentially linearly correlated with the increase in point-filling quantity. As the optimization process progresses, the differences in average convergence speed among different point-filling quantities gradually diminish. The potential reason for this phenomenon is that, in the early stages of the optimization process, fewer point-filling quantities lead to more frequent updates of the surrogate model. This provides more reliable design space information for the EGO algorithm’s optimization. As the optimization process progresses and considering that GEK model has superior fitting performance compared to OK model, it can rapidly compensate for the loss of design space information due to fewer surrogate model updates in the early stages of the optimization process. This result also indicates that the convergence performance of the parallel EGO algorithm based on the GEK model is more stable and reliable. It is noteworthy that both OK-based and GEK-based parallel EGO algorithms demonstrate relatively poor convergence performance on test function F8 (rotated expanded Griewank plus Rosenbrock function), exhibiting slower convergence speeds. Analysis of F8’s functional characteristics reveals its highly oscillatory and multimodal nature, which can lead to significant deviations between the PEI function and the true EI function. Consequently, the PEI criterion may prove particularly susceptible to failure when applied to functions of this category.

As evidenced in Figure 8 and Figure 9, the GEK-based parallel EGO algorithm demonstrates superior convergence performance over its OK-based counterpart on test functions F1, F4–F7, and F9. This enhancement stems from GEK’s effective utilization of gradient information from the same number of sample points, enabling more comprehensive exploration of the design space when identifying promising candidate points. However, for functions F2, F3, and F8 (all belonging to the Griewank family), the GEK-based approach shows no performance advantage and even exhibits degraded convergence compared to the OK-based method. This phenomenon may be attributed to the Griewank functions’ characteristic local oscillations (illustrated in Figure 6), where gradients exhibit violent fluctuations near local optima. The GEK model’s strong dependence on gradient information may cause it to overemphasize local curvature at the expense of global trends, thereby misleading the search direction. Consequently, OK-based parallel EGO is recommended for problems exhibiting such functional characteristics. Furthermore, while the GEK model possesses powerful fitting capabilities, practical applications must consider the accessibility of gradient information, and potential noise contamination in gradient data—both critical factors when selecting GEK-based approaches.

3.2. Comparative Analysis of the Optimization Efficiency of the Parallel EGO Algorithm

In practical performance optimization problems, the primary source of time consumption is the wall-clock time cost. In a single optimization cycle, the wall-clock time cost primarily consists of three components: experimental model preparation, waiting gap, and experimental evaluation (Figure 10). An effective approach to significantly improving optimization efficiency is to achieve the desired optimization objective with as few optimization cycles as possible. Therefore, the multi-point infill criterion was proposed based on this principle.

To investigate the impact of different infill point quantities on the optimization efficiency of the parallel EGO algorithm,

Table 3. The statistics of the number of cycles required to reach the expected stopping criterion under different point-filling strategies using parallel EGO algorithms based on OK and GEK, respectively.

Systems	Test Problems	Stopping Criterions	Number of Cycles	2 Points	4 Points	6 Points	8 Points	10 Points
OK-based	F1	Expected optimum < 1.0	Median	91.00	48.00	36.50	27.50	24.00
			Mean	92.77	49.33	35.00	26.83	22.23
			Std	16.15	7.49	4.70	4.03	2.36
GEK-based			Median	15.50	11.00	7.00	6.00	5.00
			Mean	15.97	11.47	8.07	6.90	6.17
			Std	4.14	4.30	2.22	1.70	1.93
OK-based	F2	Expected optimum < 0.7	Median	32.00	19.50	13.00	10.00	9.00
			Mean	39.10	23.43	13.13	11.17	10.23
			Std	29.74	16.36	7.32	6.89	5.59
GEK-based			Median	27.50	9.50	8.00	8.00	5.50
			Mean	26.93	13.80	10.23	8.43	7.20
			Std	19.17	9.35	5.78	3.47	3.19
OK-based	F3	Expected optimum < −490	Median	2.00	2.00	2.00	2.00	2.00
			Mean	1.93	1.93	1.93	1.93	1.93
			Std	0.25	0.25	0.25	0.25	0.25
GEK-based			Median	8.00	6.00	6.00	7.00	4.00
			Mean	11.20	7.57	7.53	7.23	5.33
			Std	10.50	5.63	5.04	4.59	3.09
OK-based	F4	Expected optimum < −395	Median	36.50	23.00	14.50	15.00	14.00
			Mean	40.83	29.03	19.70	16.23	14.53
			Std	18.88	16.17	10.81	7.59	6.32
GEK-based			Median	7.00	4.00	4.00	3.50	3.00
			Mean	9.60	6.50	5.17	4.63	4.20
			Std	8.19	4.88	3.74	3.06	2.45
OK-based	F5	Expected optimum < −294	Median	44.50	21.00	15.00	13.00	11.00
			Mean	53.13	26.63	19.37	13.83	13.47
			Std	29.76	16.28	11.57	5.22	5.93
GEK-based			Median	10.00	6.50	5.00	5.00	5.00
			Mean	13.13	8.13	5.93	5.40	4.77
			Std	9.96	4.81	3.44	2.33	1.76
OK-based	F6	Expected optimum < −50	Median	51.50	52.50	23.00	25.00	17.00
			Mean	58.50	42.60	24.17	21.20	16.90
			Std	34.35	19.00	11.22	8.91	6.69
GEK-based			Median	6.00	4.00	4.00	3.00	3.00
			Mean	14.10	8.07	7.03	6.03	4.97
			Std	14.14	7.23	5.94	4.41	2.81
OK-based	F7	Expected optimum < 200	Median	59.00	28.50	22.50	17.50	13.50
			Mean	69.33	36.00	24.20	19.03	15.53
			Std	42.81	19.93	12.49	9.89	7.33
GEK-based			Median	14.50	6.50	6.00	6.00	6.50
			Mean	23.80	12.60	8.90	7.27	7.03
			Std	20.94	10.54	6.35	4.29	3.52
OK-based	F8	Expected optimum < 505	Median	41.50	26.00	15.00	12.00	13.50
			Mean	38.03	24.93	16.17	12.57	12.60
			Std	25.14	18.38	11.61	8.33	8.01
GEK-based			Median	60.00	20.00	19.00	15.00	12.00
			Mean	42.20	17.87	14.00	11.27	9.00
			Std	24.33	11.43	7.73	5.66	4.34
OK-based	F9	Expected optimum < 1100	Median	50.00	27.50	40.00	16.50	21.00
			Mean	61.43	35.20	29.83	19.00	18.07
			Std	39.14	20.69	12.97	9.44	6.84
GEK-based			Median	5.00	4.00	3.50	3.00	3.00
			Mean	14.27	8.27	7.27	5.53	4.70
			Std	17.96	9.17	6.68	4.18	2.85

presents the relevant statistics on the number of optimization cycles required to achieve the predefined convergence criteria for the parallel EGO algorithm based on both OK and GEK models. For the case based on the OK model, the number of optimization cycles continuously decreases with an increasing number of infill points for almost all test functions, except for F3. This result is generally consistent with the findings presented in [24]. However, the reduction in the number of cycles required for the parallel EGO algorithm to reach the convergence criterion does not strictly follow a proportional relationship with the increase in the number of infill points. For example, on F1, taking the parallel EGO algorithm with an infill point count of 2 as the baseline, the parallel EGO algorithm with an infill point count of 4 accelerates the search by approximately 92.77/49.33 ≈ 1.88 times. However, for a strictly linear speedup, the expected factor should be 2. A possible explanation given in previous studies is that as the sequential selection process progresses within a single optimization cycle, an increasing number of IFs are multiplied by the initial EI criterion, making the PEI criterion increasingly unreliable. It is noteworthy that for F5, the parallel EGO algorithm with a point-filling number of 4 achieved an acceleration factor of approximately 53.13/26.63 ≈ 2.00 times.

To explain this phenomenon, Figure 11 illustrates the process of the PEI criterion searching for multiple potential improvement points within an optimization cycle. Strictly speaking, the fundamental principle of the PEI criterion remains a sequential process, rather than a true parallel point-filling criterion (its parallel concept is based on the parallelization of external computational resources, rather than the parallelization of the algorithm itself). The initial potential improvement point is identified using the original EI criterion, while subsequent points depend on the approximation accuracy between the current PEI function and the true EI function. When the discrepancy between PEI and true EI remains relatively small, the search process benefits from more reliable information. Conversely, significant deviations may lead to information loss, potentially misleading the EGO algorithm’s exploration direction. As the candidate selection process progresses, these approximation errors accumulate, ultimately affecting the overall optimization convergence rate. A truly parallel candidate selection approach would require completely independent selection of multiple points within each optimization cycle. Interestingly, the observed linear relationship between efficiency improvement and increased point-filling quantity in certain special cases may stem from how the influence function-induced differences between PEI and true EI functions provide beneficial algorithmic diversity. This diversity enhances the probability of escaping local optima while promoting faster convergence to higher-quality solutions.

For the case based on the GEK model, compared to the parallel EGO algorithm with the OK model, the number of optimization cycles with the same number of point fillings significantly decreased in almost all test functions, except for F3 and F8. Because the fitting performance of the GEK model has been significantly improved, as a result, the effective spatial information relied upon during the search for potential improvement points is more reliable, which facilitates the convergence of the parallel EGO algorithm. However, the redundancy of the spatial information relied upon by multiple points in a single optimization cycle also increases, which means the bottleneck for achieving optimization efficiency improvements occurs more quickly than in the case of the OK model.

3.3. Comparative Analysis of the Optimizing Diversity of the Parallel EGO Algorithm

3.3.1. Population Diversity

When an optimization algorithm is prone to getting trapped in local optima, strategies can be employed to provide population diversity, helping the algorithm escape local optima and thereby increasing the probability of finding higher-quality solutions [33]. The classical EGO algorithm inherently achieves a perfect balance between local exploitation and global exploration; however, it is also prone to getting trapped in local optima. The introduction of the parallel EGO algorithm not only enhances the efficiency of the entire optimization process but also, by multiplying the corresponding number of IFs after the original EI function, can be seen as increasing the diversity of the optimization. Therefore, the impact of different point-filling quantities on the performance of the parallel EGO algorithm is analyzed from the perspective of algorithmic diversity. The calculation method for population diversity is as follows:

$${Pop}_{div}\left(l\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n\sqrt{\sum _{j=1}^D{\left({\overline{pos}}_j-{pos}_{i,j}\left(l\right)\right)}^{{\displaystyle \frac{1}{2}}}},{\overline{pos}}_j={\displaystyle \frac{1}{n}}\sum _{i=1}^n{pos}_i\left(l\right)$$

(24)

where n and D represent the total population size and the number of design variables, respectively. ${pos}_{i,j}\left(l\right)$ represents the position of the $i$ individual in the $j$ dimension, and ${\overline{pos}}_j$ denotes the average position of all individuals in the $j$ dimension. A low population diversity indicates that the search is exploring the design space near the local optimum. In contrast, a high population diversity suggests that the search is exploring regions of the design space farther from the local optimum.

Figure 12 and Figure 13 present the average population diversity curves of the parallel EGO algorithm based on the OK model and the GEK model, respectively, under different point-filling quantities during optimization process. For the case based on OK model, the parallel EGO algorithm with a larger point-filling quantity generally exhibits greater population diversity in the early stages of optimization, indicating that it explores a larger design space. As the optimization process progresses, population diversity gradually decreases. Meanwhile, the rate of decline in population diversity is relatively slower for the parallel EGO algorithm with a larger point-filling quantity. This also explains why its overall solution convergence speed is relatively slower (Figure 8). However, the advantage of having greater population diversity is that it can facilitate the parallel EGO algorithm in escaping local optima, thereby increasing the probability of discovering higher-quality optimal solutions. For example, in the case of test functions F2, F3, F5, and F7, higher-quality solutions were found when using a larger number of filled points.

For the case based on the GEK model, overall, the impact of different point-filling numbers on the population diversity of the parallel EGO algorithm is reduced, except for test functions F1, F3, and F8. Combining the average convergence curves for the corresponding test functions in Figure 6, it can also be observed that the differences in the impact of different point-filling numbers on the convergence speed of the parallel EGO algorithm are minimal. This may be due to the fact that the spatial design information provided by GEK to the parallel EGO algorithm is sufficiently reliable, which makes the search directions for potential improvement points more well-defined.

3.3.2. Exploitation and Exploration

Almost all global optimization algorithms aim to achieve a reasonable balance between local exploitation and global exploration, and the EGO algorithm is no exception. To evaluate the impact of different point-filling numbers on the balance between local exploitation and global exploration in the parallel EGO algorithm, the percentage change trends of exploitation–exploration throughout the optimization process were calculated using the following formula:

$$\mathit{\text{exploration}}\left(l\right)=\left({\displaystyle \frac{{\mathit{Pop}}_{\mathit{div}}\left(l\right)}{\mathit{max}\left({\mathit{Pop}}_{\mathit{div}}\left(l\right)\right)}}\right)\times 100\%$$

(25)

$$\mathit{\text{exploration}}\left(l\right)=\left({\displaystyle \frac{‖{\mathit{Pop}}_{\mathit{div}}\left(l\right)-\mathit{max}\left({\mathit{Pop}}_{\mathit{div}}\left(l\right)\right)‖}{\mathit{max}\left({\mathit{Pop}}_{\mathit{div}}\left(l\right)\right)}}\right)\times 100\%$$

(26)

Here, ${\mathit{Pop}}_{\mathit{div}}$ is defined in Equation (24) and represents the diversity of the entire population.

Figure 14 and Figure 15 show the trends of the exploration–exploitation ratio over the process of optimization for the parallel EGO algorithm based on the OK and GEK models, respectively, under different point-filling quantities. Overall, it can be observed that the parallel EGO algorithm with different point-filling quantities maintains a high level of exploration during the early stages of optimization, allowing the algorithm to explore a broader design space for global optima. As the optimization progresses, the exploration rate gradually decreases while the exploitation rate increases, enabling the algorithm to continue improving the solution quality in the vicinity of local optima. For the case based on the OK model, as shown in Figure 8, the point-filling quantity of 2 consistently achieves the best convergence speed on F1, F4, F6, F8, and F9. Therefore, with a point-filling quantity of 2, the parallel EGO algorithm better maintains a reasonable balance between local exploitation and global exploration. This conclusion can be drawn from the trend of the exploration–exploitation curves in Figure 14, as the exploration–exploitation curves for the case with two points remain smaller and larger than those for the other point configurations throughout almost the entire optimization process.

For the case based on the GEK model, as shown in Figure 9, except for F1, F3, and F8, where there is a significant difference in convergence speed between different point-filling quantities, the convergence speed for the other test functions only shows slight differences in the early stages of optimization. As the optimization iterations progress, the convergence speed remains nearly consistent. Therefore, as shown in Figure 15, the differences in the impact of different point-filling quantities on the balance between local exploitation and global exploration of the parallel EGO algorithm are negligible for these test functions. This indicates that the GEK model provides greater reliability and stability to the parallel EGO algorithm.

4. Conclusions

This study conducts an in-depth investigation into how the optimization performance of the Kriging-based parallel EGO algorithm with the PEI multi-point sampling criterion is affected by different numbers of points filled in a single optimization cycle. Meanwhile, to obtain comprehensive numerical simulation results and conduct an in-depth comparative analysis, nine benchmark test functions with different optimization problem characteristics were selected as optimization targets. The point-filling levels within a single optimization cycle were set to five values: 2, 4, 6, 8, and 10. Different fidelity surrogate models were then constructed using both OK and GEK. Finally, the optimization tasks were performed using the parallel EGO algorithm with the PEI multi-point sampling criterion. Finally, a systematic comparative analysis of the results was conducted from the perspectives of the optimization performance, efficiency, and diversity of the parallel EGO algorithm, leading to the following conclusions:

(a).

The impact of different point-filling quantities on the optimization performance of the parallel EGO algorithm based on the OK model is significant. Filling two points per optimization cycle ensures the most stable optimization performance of the parallel EGO algorithm. However, as the number of filled points increases, the stability of the optimization performance gradually declines. In contrast, for the GEK model, the influence of different point-filling quantities on the optimization performance of the parallel EGO algorithm is only noticeable in the early stages of optimization. As the optimization process progresses, the performance differences gradually diminish.

(b).

For the parallel EGO algorithm based on the OK model, optimization efficiency improves as the point-filling quantity increases. However, the rate of improvement gradually slows with the increasing number of filled points. In the case of GEK, this slowdown in optimization efficiency becomes even more pronounced.

(c).

The point-filling quantity has a significant impact on the optimization diversity of the parallel EGO algorithm based on the OK model. In most cases, filling two points per optimization cycle ensures a reasonable level of optimization diversity and maintains a balance between local exploitation and global exploration. In contrast, for the GEK model, the influence of different point-filling quantities on the diversity of the parallel EGO algorithm is less pronounced.

(d).

For practical implementation, it is recommended to employ a smaller number of filling points per optimization cycle during the initial stages when design space information remains relatively limited. However, as iterations progress, gradually increasing the filling quantity may enhance algorithmic convergence speed and improve overall optimization efficiency.

Although the introduction of multi-point sampling criteria can significantly improve the efficiency of the EGO algorithm in solving practical optimization problems, its fundamental principle still follows a sequential point selection strategy within a single optimization cycle. Therefore, achieving true parallelization of the EGO algorithm requires decoupling the selection process of potential improvement points and reducing their mutual dependence. In the future, further research will focus on developing more efficient and stable point-filling strategies to better meet the needs of practical engineering optimization problems.