Active Learning-Based Kriging Model with Noise Responses and Its Application to Reliability Analysis of Structures

Abstract

This study introduces a reliability analysis methodology employing Kriging modeling enriched by a hybrid active learning process. Emphasizing noise integration into structural response predictions, this research presents a framework that combines Kriging modeling with regression to handle noisy data. The framework accommodates either constant variance of noise for all observed responses or varying, uncorrelated noise variances. Hyperparameters and the variance of the Kriging model with noisy data are determined through maximum likelihood estimation to address inherent uncertainties in structural predictions. An adaptive hybrid learning function guides design of experiment (DoE) point identification through an iterative enrichment process. This function strategically targets points near the limit-state approximation, farthest from existing training points, and explores candidate points to maximize the probability of misclassification. The framework’s application is demonstrated through metamodel-based reliability analysis for continuum and discrete structures with relatively large degrees of freedom, employing subset simulations. Numerical examples validate the framework’s effectiveness, highlighting its potential for accurate and efficient reliability assessments in complex structural systems.

1. Introduction

Ensuring the resilience of modern society demands structural systems to attain a requisite level of reliability throughout the phases of design and maintenance decision making. The implementation of robust procedures is crucial to pre-empt unexpected failures, averting potential catastrophic damage or losses. Consequently, the design processes of structures and engineering systems necessitate adept strategies for reliability assessment and optimization. However, the inherent need for repeated evaluations of system performance in reliability assessment often leads to substantial computational expenses. This challenge is particularly pronounced in cases where complex, high-dimensional models are indispensable for an accurate depiction of system performance. In response to these challenges, this dissertation proposes the incorporation of active learning frameworks. These frameworks leverage adaptive surrogate models, trained on computational simulation data, to facilitate reliability assessment in the design and evaluation of intricate structural systems under diverse uncertainties. The ensuing sections delve into the conceptualization and application of these frameworks, elucidating their efficacy in managing the intricacies of complex systems. Structural reliability methods [1,2] are a set of techniques used to assess the probability of failure of complex systems, particularly those related to civil engineering, mechanical engineering, aerospace, and other fields where structural integrity is crucial. These methods are employed to account for uncertainties that arise from various sources, such as design, manufacturing [3], environmental factors, and operating conditions [4].

Addressing uncertainties is crucial in the design process to ensure the reliability of engineering systems and mitigate the risk of unexpected failures that could lead to catastrophic damage or losses. Effective strategies are needed for quantifying and optimizing uncertainties in the design of structures and other engineering systems. Reliability-based design optimization (RBDO) [5,6] and reliablity-based topology optimization (RBTO) [7,8] have emerged as comprehensive approaches to achieving optimal designs with a low failure probability, ensuring that the likelihood of violating specified constraints remains below a predefined target level. The integration of diverse reliability analysis methods has been a subject of study in this context, and readers interested in a comprehensive overview can refer to [9] for a state-of-the-art review on RBD/TO and its applications.

To estimate reliability, defined as the probability of failure for considered performance or constraint functions, various methods have been developed. Monte Carlo simulation (MCS), a simulation method, involves sampling the joint distribution of state variables to calculate failure probability [10,11]. Despite their versatile applicability to complex problems, simulation methods like MCS can be computationally expensive due to the need for a large number of samples. The trade-off between precision and computational cost is a key consideration. Strategies such as stratified sampling and Latin hypercube sampling have been employed to enhance efficiency [12]. Lin and Su [13] introduced an explicit time-domain method-based MCS for assessing the first-passage system dynamic reliability of jacket platforms under specific sea conditions, aiming for computational efficiency.

Reliability analysis often employs approximation methods to estimate failure probability by locally approximating the limit-state function around a reference point. While these methods like FORM (first-order reliability method) and SORM (second-order reliability method) offer efficiency with a low number of model evaluations, they may face challenges with complex and nonlinear limit-state functions. Wang et al. [14] introduced an uncertain-polar coordinates SORM capable of handling hybrid variables, demonstrating reduced computational complexity while maintaining precision. Chun [15] combined SORM with complex-step derivative approximation for sensitivity analysis and design optimization. FORM, as a computationally efficient first-order approximation method, captures linear behavior near the design point but may lose accuracy with strongly nonlinear functions. SORM refines FORM by extending the approximation to second-order statistics, offering a more accurate representation of the limit-state function. While SORM enhances accuracy, it incurs higher computational costs, particularly with an increasing number of input random variables, making it valuable for precise representations of complex limit-state functions.

Metamodel-based adaptive methods [16,17,18] are a class of techniques used in structural reliability analysis that involve the iterative construction of surrogate models, known as metamodels, which approximate the limit-state function in the immediate vicinity of the limit-state surface. These metamodels are adaptively refined to improve their accuracy, and this refinement process continues until a predefined convergence criterion, typically related to the accuracy of the failure probability, is met. One potent metamodeling approach is polynomial chaos expansion (PCE) [19,20], which aims to furnish a functional approximation of a computational model by representing it spectrally using a well-constructed basis of polynomial functions. Kriging [16,21], also known as Gaussian process modeling, is a statistical interpolation technique that harnesses the power of Gaussian processes to interpolate a diverse set of complex functions. It is particularly effective in capturing intricate relationships within data and is widely used in various fields for tasks such as predictive modeling and optimization. Lee and Park [22] introduced a metamodeling approach, combining nonstationary Kriging and a support vector machine to predict the stochastic eigenvalues of brake systems. In the approach, Gibbs’s nonstationary kernel was used with step-wise hyperparameters to account for the heterogeneity of the stochastic eigenvalues. In the work by Yang et al. [23], a sampling strategy is introduced to address system reliability-based design optimization problems involving multiple simulation models. Initially, the system uncertainty across the entire domain is characterized through the cumulative distribution function (CDF) of the systems. Subsequently, a new active learning function, known as the generalized expected system improvement, is developed to predict the enhancement in system uncertainty. To address high-dimensional problems using Kriging, Sadoughi et al. [24] employ an adaptive approach to update the surrogate model. This involves sequential exploration–exploitation with a dynamic trade-off in the important dimension or region. The application of the Kriging-based approach for predicting system responses or estimating reliability has found utility in various fields, encompassing geotechnical engineering [25], structural engineering [26], mechanical engineering [27,28], and medicine [29].

This study primarily focuses on the Kriging modeling approach, considering both homoscedastic noise (constant and uniform variance for all observed responses) and heteroscedastic noise (independent noise with varying variances for each observed response without correlation). The response prediction model for Kriging is defined, and the conditional mean and variance of Kriging predictors are derived. Hyperparameters and the variance of Kriging with noise are determined through maximum likelihood estimation by solving optimization problems. The study introduces an adaptive hybrid learning function, where design of experiment (DoE) points for training a Kriging model are adaptively identified through a hybrid enrichment process. The hybrid learning function begins by identifying points in the vicinity of the limit-state approximation farthest from existing training points and subsequently explores the region expected to be close to the limit-state surface by maximizing the probability of misclassficiation. The rationale for employing this hybrid learning process is to effectively capture the significant nonlinearity inherent in the limit-state function, which might be overlooked when choosing DoE points exclusively in local regions. To estimate the reliability of a limit-state function, subset simulation is integrated into the proposed framework. The main contributions of this study include incorporating noise into the structural response for Kriging modeling, proposing a hybrid active learning process, and applying metamodel-based reliability analysis to continuum and discrete structures with relatively large degrees of freedom using subset simulations.

The subsequent sections of this paper are structured as follows. Section 2 provides the theoretical background on failure probability. Section 3 explores the principles and methods of Kriging modeling. Section 4 introduces the proposed Kriging modeling with noise and hybrid active learning processes. The effectiveness of the proposed reliability analysis framework is validated through three numerical examples in Section 5. Section 6 presents numerical applications of the proposed method to truss and continuum structures with a large number of random variables. Finally, Section 7 encapsulates the conclusions drawn from this study.

2. Theoretical Background

2.1. Limit-State Function

A limit-state function is a mathematical expression that defines a limit or boundary separating the safe or non-failure state from the failure state of a structure or system. It is a fundamental concept used to assess the safety and reliability of structures or components subject to various uncertainties, such as material properties, loads, dimensions, and environmental conditions. Consider a system in which the state is represented by a random vector of variables $\mathbf{X}$∈$D_{\mathbf{X}}$⊂${\mathbb{R}}^M$. Two distinct domains, $D_s$ and $D_f$, can be defined within the state space $D_{\mathbf{X}}$, corresponding to the safe and failure regions, respectively. In essence, the system is deemed to be in a state of failure if the current state x is within $D_f$, and it is considered to be operating safely if x is within $D_s$. This classification allows for the development of a limit-state function, referred to as $g\left(\mathbf{X}\right)$. The surface in M dimensions defined by $g\left(\mathbf{X}\right)=0$ is termed the limit-state surface, serving as the dividing line between the safe and failure domains illustrated in Figure 1.

2.2. Probability of Failure

Failure probability, often denoted as $P_f$, is a critical concept in engineering, risk assessment, and reliability analysis that represents the likelihood of a system, component, or structure failing to meet its specified performance or safety criteria under given conditions. The failure probability $P_f$ can be written as:

$$P_f=\mathbb{P}(g\left(\mathbf{X}\right)<0)$$

(1)

The failure probability $P_f$, when the vector of random variables X is described by a joint probability density function (PDF) $f_{\mathbf{X}}\left(\mathbf{x}\right)$, is calculated as:

$$P_f={\int }_{D_f=\left\{\mathbf{x}:g\left(\mathbf{x}\right)<0\right\}}{f}_{\mathbf{X}}\left(\mathbf{x}\right)d\mathbf{x}$$

(2)

In Equation (2), which calculates the failure probability $P_f$, the integration domain is determined by the region where the limit-state function $g\left(\mathbf{X}\right)$ is greater than or equal to zero, as indicated by the inequality $g\left(\mathbf{X}\right)\ge 0$. However, this region is typically implicitly defined by the limit-state function itself.

In the general case, the computational complexity associated with solving Equation (2) can be significant, particularly when it involves the execution of a computational model with respect to the vector of random variables X. For a comprehensive review on conventional techniques in structural reliability and their practical applications, the reader is directed to the work by Ditlevsen and Madsen [1] and Melchers and Beck [30].

3. Kriging (Gaussian Process) Model

Kriging is a stochastic interpolation algorithm that is based on the assumption that the model output, denoted as $\mathrm{𝒴}=\Gamma \left(\mathbf{x}\right)$, is a realization of a Gaussian process. This Gaussian process is indexed by the input variables $\mathbf{x}$∈$D_{\mathbf{X}}$⊂${\mathbb{R}}^M$. A Kriging model can be written as

$$\mathrm{𝒴}\approx {\Gamma }^{Krig}\left(\mathbf{x}\right)={\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)+{\sigma }^2\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )$$

(3)

where ${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)$ is the mean value of the Gaussian process. This Gaussian process consists of a set of functions $f_{j,j=1,...,q}$, each associated with corresponding coefficients $m_{j,j=1,...,q}$. The parameter ${\sigma }^2$ denotes the variance of this Gaussian process. The covariance function (also known as the “kernel” function) is described as $\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )$, and it characterizes a stationary Gaussian process. This stationary Gaussian process maintains a zero mean and unit variance and is defined by a covariance function $\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )$, which depends on hyperparameters denoted as $\Theta$.

In the simple Kriging model, no estimation on $\mathbf{m}$ in Equation (3) is performed and the coefficients $\mathbf{m}$ are set to 1. Consequently, the simple Kriging model is written as

$${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)=\sum _{j=1}^q{f}_j\left(\mathbf{x}\right)$$

(4)

In ordinary Kriging, ${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)$ assumes a constant but unknown value.

$${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)=m_0$$

(5)

The linear Kriging model assumes that ${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)$ is a linear combination of an arbitrary function as

$${\mathbf{m}}^T\mathbf{f}\left(\mathbf{x}\right)=m_0+\sum _{j=1}^M{m}_j{x}_j$$

(6)

where M denotes the number of input variables.

3.1. Response Prediction

In the metamodeling, it is of interest to predict a response, $\widehat{\mathbf{𝒴}}={\Gamma }^{Krig}\left({\mathbf{x}}_{\ast }\right)$, for a new data point ${\mathbf{x}}_{\ast }$, with the knowledge of the observed experimental design ${\chi }_D={\left\{{\mathbf{x}}^{\left(1\right)},{\mathbf{x}}^{\left(2\right)},...,{\mathbf{x}}^{\left(n\right)}\right\}}^T$ and its associated model responses ${\mathbf{𝒴}}_D={\left\{y\left({\mathbf{x}}^{\left(1\right)}\right),...,y\left({\mathbf{x}}^{\left(n\right)}\right)\right\}}^T={\left\{{\Gamma }^{Krig}\left({\mathbf{x}}^{\left(1\right)}\right),...,{\Gamma }^{Krig}\left({\mathbf{x}}^{\left(n\right)}\right)\right\}}^T$. The Kriging metamodel offers predictions that rely on the Gaussian characteristics of the underlying process. The Gaussian assumption postulates that the vector created by the prediction at point ${\mathbf{x}}_{\ast }$, represented as $\widehat{\mathbf{𝒴}}$, and the actual model responses ${\mathbf{𝒴}}_D$ follow a joint Gaussian distribution defined in a vector form [16,31,32] by

$$\begin{array}{c}\left\{\begin{array}{c}\widehat{\mathbf{𝒴}}\left({\mathbf{x}}_{\ast }\right)\\ {\mathbf{𝒴}}_D\end{array}\right\}\sim N_{n+1}\left(\left\{\begin{array}{c}{\mathbf{f}}^T\left({\mathbf{x}}_{\ast }\right)\mathbf{m}\\ \mathcal{F}\mathbf{m}\end{array}\right\},{\sigma }^2\left\{\begin{array}{cc}1& {\mathbf{r}}^T\left({\mathbf{x}}_{\ast }\right)\\ \mathbf{r}\left({\mathbf{x}}_{\ast }\right)& \kappa \end{array}\right\}\right)\end{array}$$

(7)

where $\mathcal{F}$ is the Kriging design matrix and each of its elements is computed as follows:

$${\mathcal{F}}_{ij}=f_j\left({\mathbf{x}}^{\left(i\right)}\right),i=1,...,n;j=1,...,q$$

(8)

where $\mathbf{r}\left({\mathbf{x}}_{\ast }\right)$ represents the vector of cross-correlations between the prediction point ${\mathbf{x}}_{\ast }$ and each of the observations $\mathbf{x}$, with its individual elements being expressed as $r_i=\kappa \left({\mathbf{x}}_{\ast },{\mathbf{x}}^{\left(i\right)};\Theta \right),i=1,...,n$. The element of covariance matrix $\mathit{\kappa }$ is calculated as

$${\kappa }_{ij}=\kappa \left({\mathbf{x}}^{\left(i\right)},{\mathbf{x}}^{\left(j\right)};\Theta \right),i,j=1,...,n$$

(9)

The Gaussian-process-based estimation of responses, $\widehat{\mathbf{𝒴}}$, at a new (prediction) point ${\mathbf{x}}_{\ast }$, and the corresponding prediction variance ${\sigma }_{\widehat{Y}}^2\left({\mathbf{x}}_{\ast }\right)$ and mean ${\mu }_{\widehat{Y}}\left({\mathbf{x}}_{\ast }\right)$ are determined as the conditional on the observed points $\left(\mathbf{x}\right)$.

$${\mu }_{\widehat{Y}}\left({\mathbf{x}}_{\ast }\right)=\mathbf{f}{\left({\mathbf{x}}_{\ast }\right)}^T\widehat{\mathbf{m}}+\mathbf{r}{\left({\mathbf{x}}_{\ast }\right)}^T{\mathit{\kappa }}^{-1}({\mathbf{𝒴}}_D-\mathcal{F}\widehat{\mathbf{m}})$$

(10)

$${\sigma }_{\widehat{Y}}^2\left({\mathbf{x}}_{\ast }\right)={\sigma }^2\left(1-\mathbf{r}{\left({\mathbf{x}}_{\ast }\right)}^T{\mathit{\kappa }}^{-1}\mathbf{r}\left({\mathbf{x}}_{\ast }\right)+\mathbf{v}{\left({\mathbf{x}}_{\ast }\right)}^T{\left({\mathcal{F}}^T{\mathit{\kappa }}^{-1}\mathcal{F}\right)}^{-1}\mathbf{v}\left({\mathbf{x}}_{\ast }\right)\right)$$

(11)

where $\widehat{\mathbf{m}}$, the generalized least square estimate of $\mathbf{m}$ [33], is determined by

$$\begin{array}{cc}\hfill \widehat{\mathbf{m}}& ={\left({\mathcal{F}}^T{\mathit{\kappa }}^{-1}\mathcal{F}\right)}^{-1}{\mathcal{F}}^T{\mathit{\kappa }}^{-1}{\mathbf{𝒴}}_D\hfill \\ \hfill \mathbf{v}\left({\mathbf{x}}_{\ast }\right)& ={\mathcal{F}}^T{\mathit{\kappa }}^{-1}\mathbf{r}\left({\mathbf{x}}_{\ast }\right)-\mathbf{f}\left({\mathbf{x}}_{\ast }\right)\hfill \end{array}$$

(12)

Figure 2 illustrates the predictive mean, variance, and confidence interval of the Gaussian process given the observed data. In Kriging, a confidence interval establishes a range of values within which the actual response value is expected to lie. This interval serves as a measure of the uncertainty inherent in the predictions generated by the Kriging model. The significance of the confidence interval lies in its ability to enhance comprehension regarding the reliability of the model predictions and to evaluate the level of confidence one can place in the estimated values.

3.2. Covariance Function

The covariance function $\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )$ plays a critical role in Kriging predictors by encapsulating the underlying assumptions about the approximation function. This function quantifies the extent of “similarity” between observations and newly acquired data points, with its value dependent on the spatial separation between these input points. Consequently, the proximity of input points to each other implies a higher likelihood of similar outcomes or responses. The covariance functions under consideration are of a stationary nature, implying that their characteristics solely rely on the relative position between their two inputs. Furthermore, these covariance functions are one-dimensional in nature and are defined for a pair of one-dimensional inputs denoted as $\mathbf{x}$ and ${\mathbf{x}}^{\prime }$, both belonging to the real number set ($\mathbb{R}$), and are parameterized by $\Theta \in \mathbb{R}$. For instance, the linear, exponential, and Gaussian covariance functions [16] can be written as

$$\begin{array}{cc}\hfill \mathrm{Linear}:\tilde{\kappa }(x,x^{\prime };\Theta )& =\max \left(0,1-\frac{\overline{d}}{\Theta }\right)\hfill \\ \hfill \mathrm{Exponential}:\tilde{\kappa }(x,x^{\prime };\Theta )& =\exp \left(-\frac{\overline{d}}{\Theta }\right)\hfill \\ \hfill \mathrm{Gaussian}:\tilde{\kappa }(x,x^{\prime };\Theta )& =\exp \left(-0.5·{\left(\frac{\overline{d}}{\Theta }\right)}^2\right)\hfill \end{array}$$

(13)

where $\overline{d}=∥x-x^{\prime }∥$ denotes the distance between the two input points x and $x^{\prime }$. The Matérn function, widely employed as a covariance function and utilized in this study, is represented as follows [31]:

$$\mathrm{Matérn}:\tilde{\kappa }(x,x^{\prime };\Theta )=\left(1+\sqrt{3}·\frac{\overline{d}}{\Theta }\right)·\exp \left(-\sqrt{3}·\frac{\overline{d}}{\Theta }\right)$$

(14)

In cases where the input dimension M exceeds one, multi-dimensional covariance functions can be formed by employing one of the methods, the separable and ellipsoidal function construction approaches, which are expressed as:

$$\begin{array}{cc}\hfill \mathrm{Ellipsoidal}:\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )& =\tilde{\kappa }\left(\sqrt{\sum _{i=1}^M{\left(\frac{{\overline{d}}_i}{{\Theta }_i}\right)}^2}\right),{\overline{d}}_i=∥x_i-x_i^{\prime }∥\hfill \\ \hfill \mathrm{Separable}:\kappa (\mathbf{x},{\mathbf{x}}^{\prime };\Theta )& =\prod _{i=1}^M\tilde{\kappa }(x_i,x_i^{\prime },{\Theta }_i)\hfill \end{array}$$

(15)

3.3. Maximum Likelihood Estimation

For the creation of a Kriging model, the hyperparameters denoted as $\Theta$ are typically unknown and therefore necessitate estimation. This estimation process involves solving an optimization problem, the specifics of which depend on the chosen estimation method. In the case of the maximum likelihood estimation (MLE) method [31], the objective is to determine the set of Kriging parameters and appropriate variables in a manner that maximizes the likelihood of the observed data ${\mathbf{𝒴}}_D={\left\{{\Gamma }^{krig}\left({\mathbf{x}}^{\left(1\right)}\right),{\Gamma }^{krig}\left({\mathbf{x}}^{\left(2\right)}\right),...,{\Gamma }^{krig}\left({\mathbf{x}}^{\left(n\right)}\right)\right\}}^T$. In cases where no response noise exists, the likelihood function $\mathcal{L}$ based on the multivariate normal PDF can be expressed as:

$$\mathcal{L}(\mathbf{m},{\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)=\frac{det{\left(\mathit{\kappa }\right)}^{-0.5}}{{\left(2\pi {\sigma }^2\right)}^{0.5n}}·\exp \left(-0.5{({\mathbf{𝒴}}_D-\mathbf{F}\mathbf{m})}^T{\left({\sigma }^2\mathit{\kappa }\right)}^{-1}({\mathbf{𝒴}}_D-\mathbf{F}\mathbf{m})\right)$$

(16)

The hyperparameter $\Theta$ is determined by solving the optimization problem expressed as:

$$\underset{\Theta }{max}\left(\log \mathcal{L}(\mathbf{m},{\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)\right)$$

(17)

4. Framework for Reliability Analysis Incorporating Noisy Response Prediction

4.1. Response Prediction with Noise

Constructing the predictor for Kriging models applicable to structures with noisy variance entails introducing the following framework. It is assumed that the observed response incorporates noise with a known variance, denoted as $\epsilon$, adhering to a zero-mean Gaussian distribution $\sim \mathcal{N}(0,{\Sigma }^{\ast })$, where ${\Sigma }^{\ast }$ is the noise covariance matrix, and can be expressed as:

$$\mathrm{𝒴}={\Gamma }^{krig}\left(\mathbf{x}\right)+\epsilon$$

(18)

Substituting Equation (18) into Equation (7), the formulation of the response prediction model can be expressed as:

$$\begin{array}{c}\left\{\begin{array}{c}\widehat{\mathbf{𝒴}}\left({\mathbf{x}}_{\ast }\right)\\ {\mathbf{𝒴}}_D\end{array}\right\}\sim N_{n+1}\left(\left\{\begin{array}{c}{\mathbf{f}}^T\left({\mathbf{x}}_{\ast }\right)\mathbf{m}\\ \mathcal{F}\mathbf{m}\end{array}\right\},\left\{\begin{array}{cc}{\sigma }^2& {\sigma }^2{\mathbf{r}}^T\left({\mathbf{x}}_{\ast }\right)\\ {\sigma }^2\mathbf{r}\left({\mathbf{x}}_{\ast }\right)& {\sigma }^2\kappa +{\Sigma }^{\ast }\end{array}\right\}\right)\end{array}$$

(19)

Based on Equation (19), the conditional mean and variance of Kriging predictors are then derived:

$${\mu }_{\widehat{Y}}\left({\mathbf{x}}_{\ast }\right)=\mathbf{f}{\left({\mathbf{x}}_{\ast }\right)}^T\tilde{\mathbf{m}}+{\left({\sigma }^2\mathbf{r}\left({\mathbf{x}}_{\ast }\right)\right)}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}({\mathbf{𝒴}}_D-\mathcal{F}\tilde{\mathbf{m}})$$

(20)

$${\sigma }_{\widehat{Y}}^2\left({\mathbf{x}}_{\ast }\right)={\sigma }^2-{\left({\sigma }^2\mathbf{r}\left({\mathbf{x}}_{\ast }\right)\right)}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}\left({\sigma }^2\mathbf{r}\left({\mathbf{x}}_{\ast }\right)\right)+\tilde{\mathbf{v}}{\left({\mathbf{x}}_{\ast }\right)}^T{\left({\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}\mathcal{F}\right)}^{-1}\tilde{\mathbf{v}}\left({\mathbf{x}}_{\ast }\right)$$

(21)

where

$$\tilde{\mathbf{v}}\left({\mathbf{x}}_{\ast }\right)={\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}{\sigma }^2\mathbf{r}\left({\mathbf{x}}_{\ast }\right)-\mathbf{f}\left({\mathbf{x}}_{\ast }\right)$$

(22)

4.2. Estimates of the Hyperparameters with Noise

The optimization problem of maximum likelihood estimation in Equation (17) without noise can be reformulated as an equivalent minimization problem in natural logarithm form, expressed as:

$$\underset{\Theta ,\mathbf{m},{\sigma }^2}{min}\left(-\log \mathcal{L}(\mathbf{m},{\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)\right)$$

(23)

Based on the first-order optimality conditions for unconstrained optimization, the following derivations can be obtained:

$$\begin{array}{cc}\hfill {\nabla }_{\mathbf{m}}log\mathcal{L}(\mathbf{m},{\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)& =\frac{1}{2{\sigma }^2}({\mathcal{F}}^T{\kappa }^{-1}{\mathbf{𝒴}}_D-{\mathcal{F}}^T{\mathit{\kappa }}^{-1}\mathcal{F}\mathbf{m})=\mathbf{0}\hfill \\ \hfill \frac{\partial log\mathcal{L}(\mathbf{m},{\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)}{\partial {\sigma }^2}& =-\frac{n}{2}+\frac{1}{2{\sigma }^2}{({\mathbf{𝒴}}_D-\mathcal{F}\mathbf{m})}^T{\mathit{\kappa }}^{-1}({\mathbf{𝒴}}_D-\mathcal{F}\mathbf{m})=0\hfill \end{array}$$

(24)

From Equation (24), the generalized least-squares estimates, $\tilde{\mathbf{m}}$ and ${\tilde{\sigma }}^2$, of $\mathbf{m}$ and ${\sigma }^2$ are obtained as

$$\begin{array}{cc}\hfill \tilde{\mathbf{m}}& =\mathbf{m}(\Theta )={\left({\mathcal{F}}^T{\mathit{\kappa }}^{-1}\mathcal{F}\right)}^{-1}\left({\mathcal{F}}^T{\mathit{\kappa }}^{-1}{\mathbf{𝒴}}_D\right)\hfill \\ \hfill {\tilde{\sigma }}^2& ={\sigma }^2(\Theta )=\frac{1}{n}{({\mathbf{𝒴}}_D-\mathcal{F}\tilde{\mathbf{m}})}^T{\mathit{\kappa }}^{-1}({\mathbf{𝒴}}_D-\mathcal{F}\tilde{\mathbf{m}})\hfill \end{array}$$

(25)

With the generalized least-square estimate, the optimization problem in Equation (23) can be simplified as

$$\underset{\Theta }{min}-\log \mathcal{L}(\Theta ;{\mathbf{𝒴}}_D)$$

(26)

When the noise is homoscedastic, a constant and uniform variance for all observed responses is indicated, and when it is heteroscedastic, independent noise with varying variances for each observed response without correlation is signified. In the presence of homoscedastic or heteroscedastic noise with a known variance, the maximum likelihood function can be expressed using the PDF of the multivariate normal distribution as follows:

$$\mathcal{L}(\mathbf{m},{\sigma }^2,{\sigma }_{\ast }^2,\Theta ;{\mathbf{𝒴}}_D)=\frac{det{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-0.5}}{{\left(2\pi \right)}^{0.5n}}·\exp \left(-0.5{({\mathbf{𝒴}}_D-\mathcal{F}\mathbf{m})}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}({\mathbf{𝒴}}_D-\mathcal{F}\mathbf{m})\right)$$

(27)

The generalized least-square estimate of $\mathbf{m}$ is a function dependent on $\Theta ,{\sigma }^2,{\sigma }_{\ast }^2$. Employing the first optimality condition, the subsequent expressions are derived:

$${\nabla }_{\mathbf{m}}log\mathcal{L}(\mathbf{m},{\sigma }^2,{\sigma }_{\ast }^2,\Theta ;{\mathbf{𝒴}}_D)=\frac{1}{2}({\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}{\mathbf{𝒴}}_D-{\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}\mathcal{F}\mathbf{m})=\mathbf{0}$$

(28)

Thus, the generalized least square estimate $\tilde{\mathbf{m}}$ is computed by

$$\tilde{\mathbf{m}}=\mathbf{m}(\Theta )={\left({\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}\mathcal{F}\right)}^{-1}\left({\mathcal{F}}^T{({\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast })}^{-1}{\mathbf{𝒴}}_D\right)$$

(29)

In contrast to the noise-free scenario, the variance estimate ${\tilde{\sigma }}^2$ cannot be analytically derived because of the implicit terms ${\sigma }^2\mathit{\kappa }+{\Sigma }^{\ast }$. It needs to be determined by solving the optimization problem.

$$\underset{\Theta ,{\sigma }^2}{min}-\log \mathcal{L}({\sigma }^2,\Theta ;{\mathbf{𝒴}}_D)$$

(30)

This study utilizes sequential quadratic programming [34] for solving the optimization problem. The forward finite difference approach is adopted to compute gradients of the objective function with respect to variables of interest.

4.3. Adaptive Hybrid Learning Function

Constructing an accurate Kriging model requires a sufficiently large number of DoE points. However, this necessity often leads to significant computational costs, especially when the original function’s evaluation is computationally expensive. To address this challenge and facilitate effective enrichment in DoE, recent years have witnessed the development of various adaptive methods [16,35,36,37]. These methods employ a sequential point addition approach based on information from previous iterations. In essence, adaptive methods iteratively construct Kriging models that approximate the limit-state function in close proximity to the limit-state surface. These models undergo adaptive refinement through the inclusion of additional limit-state function evaluations in their experimental designs until a convergence criterion, tied to the accuracy of the probability of failure, is satisfied. The significance of the misclassification error within the adaptive refinement stages of the Kriging surrogate model hinges on the relative importance of the associated locations. In reliability scenarios, domains critical to accurate failure probability and reliability index estimation, particularly those containing the design point, assume paramount importance. Consequently, adaptive refinement strategies are inclined to prioritize these critical domains. Specifically, a learning function is employed to identify points that, upon integration into the experimental design, can most effectively reduce the error introduced by the surrogate in estimating the failure probability. A diverse array of learning functions has been put forth in the literature [38,39]. For example, Echard et al. [35] suggested the integration of Kriging and MCS in a method known as AK-MCS. This approach initially creates Monte Carlo sample points in accordance with the PDF. The evaluation of the significance of each Monte Carlo sample relies on past simulation outcomes, with the learning function being explicitly defined as

$$U_{LF}\left(x\right)=\frac{\left|{\mu }_{\tilde{G}}\left(x\right)\right|}{{\sigma }_{\tilde{G}}\left(x\right)}$$

(31)

where ${\mu }_{\tilde{G}}\left(x\right)$ signifies the Kriging mean prediction and ${\sigma }_{\tilde{G}}\left(x\right)$ represents the associated standard deviation. In AK-MCS, the next simulation point is determined by selecting the Monte Carlo sample that minimizes the learning function in Equation (31) from the candidate pool $\mathcal{Q}$.

In this study, a hybrid active learning process is introduced to identify multiple points in the DoE during each enrichment process. This learning function integrates the DoE max-min function by Moustapha and Sudret [40] with a K-means clustering algorithm [41]. The max-min learning function seeks to identify samples in the vicinity of the limit-state approximation that are farthest from existing training points. The next point to be added to the DoE is determined by

$${\widehat{x}}^{\ast }=\arg \underset{d\in {\mathcal{Q}}^{{}^{\prime }}}{max}\underset{i=1,...,N}{min}∥d-x^{\left(i\right)}∥,{\mathcal{Q}}^{{}^{\prime }}=\left\{d\in \mathcal{Q}:\left|{\mu }_{\tilde{G}}\left(d\right)\right|<\gamma \text{-}\mathrm{quantile}\mathrm{of}\left|{\mu }_{\tilde{G}}\left(d\right)\right|\right\}$$

(32)

In adherence to the min-max learning function approach, enrichment occurs sequentially, adding one DoE point at a time, and updating the min-max distance after the inclusion of each point. To identify the multiple K number of DoE points in each iterative enrichment process, the K-means clustering algorithm can be employed [42]. The proposed framework integrates the weighted K-means clustering algorithm. K-means clustering begins by defining a subset of the candidate pool $\mathcal{Q}$ as

$$\underline{\mathcal{Q}}=\left\{\mathbf{x}\in \mathcal{Q}:-\alpha ·{\sigma }_{\tilde{G}}\left(\mathbf{x}\right)\le {\mu }_{\tilde{G}}\left(\mathbf{x}\right)\le \alpha ·{\sigma }_{\tilde{G}}\left(\mathbf{x}\right)\right\}$$

(33)

where $\alpha$ defines the confidence intervals. After the $K-K_{min}^{max}$ number of DoE points is determined by Equation (32), then candidate points $\widehat{\mathbf{x}}\in \underline{\mathcal{Q}}$ for DoE are employed to compute weights according to:

$$\mathrm{weight}:w\left(\widehat{\mathbf{x}}\right)=\Phi \left({\left(\frac{\left|{\mu }_{\tilde{G}}\left(\widehat{\mathbf{x}}\right)\right|}{{\sigma }_{\tilde{G}}\left(\widehat{\mathbf{x}}\right)}\right)}^2\right)$$

(34)

Using the computed weights, the ($K-K_{min}^{max}$) number of weighted centroids of clusters denoted by $\widehat{{\mathbf{wx}}_c}=\left\{{\widehat{wx}}_{c_1},{\widehat{wx}}_{c_2},...,{\widehat{wx}}_{c_{K-K_{min}^{max}}}\right\}$ is determined by the K-means clustering algorithm. Subsequently, a point $\widehat{x}\in \underline{\mathcal{Q}}$ that is closest to each weighted centroid is chosen as a point in the DoE.

In the proposed hybrid learning process, candidate points are initially generated in addition to the $n_0$ initial DoE points. From this pool of candidates, points falling within the $\gamma$-quantile of $\left|{\mu }_{\tilde{G}}\left(d\right)\right|$ are selected. These chosen points are then utilized to calculate individual distances to each point in the initial DoE. Subsequently, the point with the farthest distance is determined as a DoE point and added to the initial set. This iterative process continues until the specified $K_{min}^{max}$ is met, with the recommended setting being a half or one-third of the number of points, denoted as K, to add within an enrichment step. Following this, K-means clustering algorithms are employed to identify the $K-K_{min}^{max}$ number of weights and centroids of points within a subset of the candidate pool $\mathcal{Q}$. The next step involves selecting $K-K_{min}^{max}$ points closest to the computed centroids of clusters, which are then added to the DoE. In summary, the proposed hybrid learning function initially identifies points near the limit-state approximation that are farthest from existing training points, followed by an exploration of the region expected to be close to the limit-state surface. This bidirectional process ensures that the newly added DoE points, upon completion of the enrichment process, include those distant from the existing DoE to explore unknown regions. Simultaneously, the search is directed toward samples relatively close to the limit-state function, significantly impacting the failure probability. Figure 3 depicts the hybrid learning process for the limit-state function $g(x_1,x_2)=5-2x_1+0.2x_2^3$, where $x_1$ and $x_2$ represent uncorrelated standard normal random variables.

4.4. Surrogate Reliablity Analysis

In conducting reliability analysis with the Kriging model, this study employs the simulation approach. Given that MCS may demand a substantial number of limit-state function evaluations to converge with an acceptable level of accuracy, especially when the probability of failure is small, subset simulation emerges as a viable option. Originating from the work of Au and Beck [43,44], this technique is crafted to mitigate the computational burden by addressing a sequence of simpler reliability problems with prescribed failure thresholds. In the subset approach, the failure domain is restructured as a decreasing sequence of failure domains as

$$D_f=D_{f_1}\supset D_{f_2}\supset \cdots \supset D_{f_m},D_{f_k}=\left\{\mathbf{x}:g\left(\mathbf{x}\right)\le {\xi }_k\right\}$$

(35)

where ${\xi }_k$ denotes a series of decreasing threshold values such that ${\xi }_1>{\xi }_2>\cdots >{\xi }_k$ in relation to the limit-state function. The failure probability is then computed as a product of a sequence of conditional probabilities as

$$P_f=P\left(\bigcap _{k=1}^m\mathbb{P}\left(D_{f_k}\right)\right)=\left(\prod _{i=1}^{m-1}\mathbb{P}\left(D_{f_{i+1}}\right|D_{f_i})\right)·\mathbb{P}\left(D_{f_1}\right)$$

(36)

The concept behind subset simulation is to estimate the failure probability by assessing these quantities. Estimating the failure probability using simulation approaches becomes more challenging as the failure probability decreases. In general, the smaller the probability of failure, the greater the number of simulation samples required to observe the failure event and calculate the failure probability. As discussed by Au and Beck [43], appropriately assigning intermediate failure events allows the conditional probability in Equation (36) to be made sufficiently large, facilitating efficient evaluation through simulation approaches. In this study, a threshold value of 0.1 (${\xi }_k=0.1$), as indicated in [43], is adopted to compute conditional probabilities.

With a sample of size $N_s$ of the input random vector $\mathbf{X}$, MCS can be employed to compute $\mathbb{P}\left(D_{f_1}\right)$ in Equation (36) as

$$\mathbb{P}\left(D_{f_1}\right)\approx \frac{1}{N_s}\sum _{i=1}^{N_s}{I}_{D_1}\left({\mathbf{x}}_i^s\right)$$

(37)

where ${\mathbf{x}}_i^s$ denotes generated samples according to the PDF and $I_{D_1}$ is the binary indicator function. The binary indicator function returns “1” if the limit-state function is negative or zero and “0” otherwise. To enhance the efficiency of sampling in the computation of the remaining conditional probability, this study employs the modified Metropolis method proposed by Au and Beck [43], utilizing Markov chain simulation [45].

The convergence is assessed based on the relative increment in the reliability index or the failure probability. The convergence criterion, expressed in terms of the change in reliability index, is formulated as

$$\frac{\left|{\widehat{\beta }}^{\left(k\right)}-{\widehat{\beta }}^{(k-1)}\right|}{{\widehat{\beta }}^{\left(k\right)}}\le {ϵ}_{\widehat{\beta }}$$

(38)

where ${\widehat{\beta }}^{\left(k\right)}$ represents the estimated reliability index at the k-th iteration and ${ϵ}_{\widehat{\beta }}$ denotes a prescribed tolerance value.

4.5. Implementation Details of the Proposed Framework

The algorithm and implementation details of the proposed framework are outlined as follows and illustrated in Figure 4:

• Step 1. Construct the initial DoE: Generate $n_0$ samples $\mathbf{X}={\left\{{\mathbf{x}}^{\left(1\right)},...,{\mathbf{x}}^{\left(n_0\right)}\right\}}^T$ using Latin hypercube sampling (LHS) in the standard normal space.
• Step 2. Evaluate the limit-state function responses: Evaluate the limit-state function on the sample points $\mathbf{𝒴}={\left\{g\left({\mathbf{x}}^{\left(0\right)}\right),...,g\left({\mathbf{x}}^{\left(n_0\right)}\right)\right\}}^T$.
• Step 3. Construct a Kriging model: Establish a Kriging model using the initial DoE $\left\{\mathbf{X},\mathbf{𝒴}\right\}$
• Step 4. Perform reliability analysis: Utilize the subset simulation method to estimate the failure probability $P_f$ with the current Kriging model.
• Step 5. Conduct hybrid active learning process: Generate a candidate pool and select sample points within the $\gamma \text{-}\mathrm{quantile}\mathrm{of}\left|{\mu }_{\tilde{G}}\left(d\right)\right|$. Subsequently, identify the farthest points from the selected sample points to the current DoE points. The Kriging model can be updated either before moving to the next step or after completing multiple enrichment processes. Proceed to the next step to identify multiple enrichment points. Utilize the K-means clustering algorithm to determine centroids of clusters. Finally, identify candidate points that are closest to each centroid of the cluster.
• Step 6. Complete enrichment process: Based on identified points $\widehat{\mathbf{x}}$ and the corresponding limit-state function responses $\mathbf{𝒴}\left(\widehat{\mathbf{x}}\right)={\left\{g\left({\widehat{\mathbf{x}}}^{\left(0\right)}\right),...,g\left({\widehat{\mathbf{x}}}^{\left(K\right)}\right)\right\}}^T$, update the experimental design of the Kriging model.
• Step 7. Check if the convergence condition, such as stability in the reliability index or reaching the maximum number of DoE points added, is satisfied. If the condition is met, conclude the process with the estimated failure probability; otherwise, return to Step 5.

5. Numerical Example

The procedural steps and performance of the proposed framework are elucidated through numerical examples in this study. Specifically, one example highlights a scenario characterized by a small failure probability and high nonlinearity, where the nonlinearity is represented by a large curvature. A second numerical example is presented to showcase the method’s effectiveness in addressing challenges associated with multiple failure regions within the context of reliability analysis. In both instances, the initial DoE size, denoted as $n_0$, is set to 10, and the stopping criterion for the hybrid learning process is defined by the stability of the reliability index. The accuracy and efficiency of various methods are compared using the parameter $N_{call}$, which represents the total number of simulations for MCS or the number of actual function evaluations required to estimate the failure probability for FORM and SORM. For HAK and AK-MCS, $N_{call}$ encompasses the sum of the initial DoE points and the number of simulations during the active learning process. This comprehensive comparison aims to assess and contrast the performance of the different methods under consideration.

5.1. Performance of the Hybrid Learning Function

As an illustrative numerical example in this paper, let us examine a highly nonlinear limit-state function given by

$$g\left(\mathbf{x}\right)=(c-1)-x_2+\exp (-x_1^2/10)+{(x_1/5)}^4$$

(39)

where $x_1$ and $x_2$ are uncorrelated random variables following normal distribution ($x_1\sim N(2,0.1),x_2\sim N(1,0.15)$), and c is an integer parameter.

To compute the failure probability, the proposed framework employs subset simulations and hybrid active learning approaches with $K_{min}^{max}=3$, and the addition of five DoE points within each enrichment step ($K=5$). The performance of Kriging-based reliability analysis is compared using the AK-MCS method [35] with one DoE point enrichment at each iteration. Figure 5 presents the predicted limit-state functions obtained using the proposed framework and AK-MCS during the active learning process. Despite similarities in the predicted limit-state functions between the two methods, it is noteworthy that AK-MCS required more design points for its construction and subsequent reliability analysis. Additionally, Figure 6 illustrates the convergence history of the failure probability ($c=2$) computed by subset simulations in the proposed method and MCS in AK-MCS. The results of the failure probability demonstrate reasonable similarity.

A parametric study is conducted by varying the parameter c from 3 to 5. The accuracy and efficiency of the different methods (MCS; FORM; SORM; AK-MCS) are compared to the proposed framework in terms of the failure probability $P_f$, the coefficient of variation (c.o.v) of the failure probability ${\Delta }_{P_f}$, and the generalized reliability index $\beta$.

In contrast to FORM, which exhibits less accurate results, and even SORM, which shows significant errors for the case of $c=4$ due to high nonlinearity, the proposed framework demonstrates consistent accuracy, yielding results akin to the reference value (MCS) for all cases summarized in

Table 1. Comparative analysis of results obtained through the proposed HAK and several other reliability analysis methods for the nonlinear limit-state function.

Case	Method	${\mathit{P}}_{\mathit{f}}$	${\Delta }_{{\mathit{P}}_{\mathit{f}}}$	$\mathit{\beta }$	${\mathit{N}}_{\mathit{c}\mathit{a}\mathit{l}\mathit{l}}$
$c=2$	FORM	$3.61\times {10}^{-2}$	-	1.798	97
	SORM	$4.03\times {10}^{-2}$	-	1.747	109
	MCS	$5.14\times {10}^{-2}$	0.014	1.632	${10}^5$
	AK-MCS	$4.94\times {10}^{-2}$	0.014	1.651	35$({10}^5$)
	HAK	$5.15\times {10}^{-2}$	0.010	1.630	25
$c=3$	FORM	$3.31\times {10}^{-3}$	-	2.715	145
	SORM	$2.54\times {10}^{-3}$	-	2.802	157
	MCS	$4.46\times {10}^{-3}$	0.03	2.615	2 $\times {10}^5$
	AK-MCS	$4.40\times {10}^{-3}$	0.048	2.620	35$({10}^5$)
	HAK	$4.51\times {10}^{-3}$	0.017	2.611	25
$c=4$	FORM	$1.31\times {10}^{-4}$	-	3.651	192
	SORM	$7.89\times {10}^{-5}$	-	3.778	204
	MCS	$1.51\times {10}^{-4}$	0.037	3.615	5 $\times {10}^6$
	AK-MCS	$1.37\times {10}^{-4}$	0.049	3.6393	42$({10}^6$)
	HAK	$1.57\times {10}^{-4}$	0.023	3.604	25
$c=5$	FORM	$2.07\times {10}^{-6}$	-	4.604	349
	SORM	$1.05\times {10}^{-6}$	-	4.743	361
	MCS	$2.08\times {10}^{-6}$	0.102	4.618	5 $\times {10}^7$
	AK-MCS	-	-	-
	HAK	$2.25\times {10}^{-6}$	0.030	4.587	25

. Notably, the performance of the proposed method remains robust even in scenarios with low failure probability, attributed to the effectiveness of active learning and subset simulations. However, it is observed that MCS requires a high number of samples to achieve a certain degree of c.o.v (${\Delta }_{P_f}$). AK-MCS (with ${10}^6$ samples) was unable to estimate the lower failure probability for $c=5$.

5.2. Series System with Two Failure Regions

The efficacy of the proposed method is evaluated in the case involving multiple failure regions. To illustrate this, a series system with three design points is considered, and the corresponding limit-state function is defined as per the formulation presented in [38,46].

$$g(x_1,x_2)=min\left\{\begin{array}{c}(c-1)-x_2+\exp (-x_1^2/10)+{(x_1/5)}^4\\ c^2/2-x_1{x}_2\end{array}\right\}$$

(40)

where $x_1$ and $x_2$ denote uncorrelated standard normal random variables and c is an integer parameter. This limit-state function is known to have three design points: ${\mathrm{x}}_1^{\ast }={(0,c)}^T,{\mathrm{x}}_2^{\ast }={(c/\sqrt{2},c/\sqrt{2})}^T$, and ${\mathrm{x}}_3^{\ast }={(-c/\sqrt{2},-c/\sqrt{2})}^T$. In this study, various traditional learning functions, including the $U_{LF}$ function in Equation (31) [35], expected feasibility function (EFF) [47], and constrained min-max (CMM) function [40], are employed to construct a Kriging model and estimate the failure probability of the series system. Their performances are compared with the results obtained from the proposed HAK approach. As part of the assessment, EFF and $U_{LF}$ are employed both with and without K-means clustering algorithms, serving as active learning functions for performance evaluation. Figure 7 illustrates the final experiment designs and Kriging predictions using several learning algorithms, each initiated with two randomly generated initial DoE points (Case A and Case B). In Case A, the EFF and CMM approaches encounter challenges in accurately establishing the Kriging model, leading to imprecise failure probability estimates, possibly attributed to the presence of multiple failure points. In contrast, the proposed HAK approach adeptly constructs the Kriging model, demonstrating improved accuracy. $U_{LF}$ exhibits relatively better performance when compared to EFF and CMM in this scenario. In Case B, both HAK and the Kriging model with the $U_{LF}$ have estimate failure probabilities close to the reference values from MCS. The Kriging model with EFF and CMM identifies both failure regions but exhibits a less precise estimate of failure probability. Moreover, as shown in Figure 7c,e, Kriging models based on $U_{LF}$ and EFF without the K-means clustering algorithm demonstrate less accurate predictions of the limit-state function compared to those with the K-means clustering scheme.

Table 2. Comparative analysis of results obtained using HAK and various other reliability analysis methods for the series system example.

Method	${\mathit{P}}_{\mathit{f}}$	${\Delta }_{{\mathit{P}}_{\mathit{f}}}$	${\mathit{N}}_{\mathit{c}\mathit{a}\mathit{l}\mathit{l}}$
HAK	$3.51\times {10}^{-3}$	0.018	50
MCS	$3.47\times {10}^{-3}$	0.017	${10}^6$
Meta-IS [38]	$3.54\times {10}^{-3}$	<0.05	644

presents the estimated failure probabilities by the proposed HAK method, MCS, and metamodel-based importance sampling (Meta-IS) [38].

5.3. A Highly Nonlinear Oscillator

A numerical example of a nonlinear oscillator subjected to a rectangular load pulse is depicted in Figure 8. The proposed method is employed to compare numerical results with other approaches. The limit-state function, as described in [35], is expressed as:

$$G(c_1,c_2,m,R,t_1,F_1)=3R-\left|\frac{2F_1}{m{\omega }_o^2}·sin\left(\frac{{\omega }_o{t}_1}{2}\right)\right|$$

(41)

where $c_1$ and $c_2$ denote the initial stiffness of nonlinear springs, respectively, while m represents the mass of the oscillator and ${\omega }_o=\sqrt{(c_1+c_2)/m}$. $t_1$ and $F_1$ correspond to the duration and amplitude of the load pulse, respectively. The parameter R is the displacement at which one of the springs yields. The second term in Equation (41) represents the maximum displacement of the oscillator system. Types of distributions of random variables and parameters are in

Table 3. Distribution type and parameters of the random variables for the nonlinear oscillator example.

Random Variables	Distribution	Mean	Standard Deviation
m	Gaussian	1	0.05
$c_1$	Gaussian	1	0.1
$c_2$	Gaussian	0.1	0.01
R	Gaussian	0.5	0.05
$t_1$	Gaussian	1	0.2
$F_1$	Gaussian	1	0.2

.

Table 4. Comparative analysis of results: HAK versus various reliability analysis methods for the nonlinear oscillator example.

Method	${\mathit{P}}_{\mathit{f}}$	${\Delta }_{{\mathit{P}}_{\mathit{f}}}$	$\mathit{\beta }$	${\mathit{N}}_{\mathit{C}\mathit{a}\mathit{l}\mathit{l}}$
FORM	0.0311	-	1.865	41
SORM	0.0287	-	1.901	128
MCS	0.0286	0.025	1.902	${10}^5$
AK-MCS	0.0294	0.018	1.889	$85\left({10}^5\right)$
IS	0.0288	0.03	1.899	2670
HAK	0.0286	0.018	1.902	46

demonstrates the effectiveness of the HAK method in handling the dimension, yielding accurate results.

6. Numerical Application

The application of the proposed framework to engineering systems, including the truss structure and the Messerschmitt-Bolkow-Blohm (MBB) beam subjected to multiple loads, is presented. The analysis explores the influence of dimensions of random variables, distribution types, and computational efficiency on the effectiveness of the proposed method under uncertainties in material properties and magnitudes of loads.

6.1. Truss Bridge Structure

In this section, the proposed reliability analysis framework is employed to assess the probability of failure in a truss structure under the influence of multiple point loads. These point loads are categorized into two distinct groups: $F_1$ and $F_2$ forces, which are applied to the joints on the bottom chords, and the $F_3$ force group, which is applied to the joints on the top chords. A visual representation of this configuration can be found in Figure 9. The force groups, denoted as $F_1$, $F_2$, and $F_3$, are modeled as random variables and are assumed to follow a lognormal distribution. Additionally, the three moduli of elasticity, $E_1$ for the bottom chords, $E_2$ for the top chords, and $E_3$ for the diagonal members, are also treated as random variables and are assumed to follow a Gumbel distribution. The statistical characteristics of these moduli, including their means and moments, are succinctly presented in

Table 5. Distribution type and parameters of the random variables considered for the reliability analysis of the truss bridge structure.

Random Variables	Distribution	Mean	Moments
$E_1$ (BC)	Gumbel	29,000 ksi	2900 ksi
$E_2$ (TC)	Gumbel	29,000 ksi	5800 ksi
$E_3$ (DG)	Gumbel	29,000 ksi	8700 ksi
$F_1$	Lognormal	30 kips	9 kips
$F_2$	Lognormal	50 kips	10 kips
$F_3$	Lognormal	60 kips	6 kips

. Moreover, Figure 10 provides a visual representation of the correlation matrix, illustrating the inter-relationships between the various random variables involved in the reliability analysis.

In the initial phase, a deterministic structural analysis is performed on the truss structure, incorporating the cross-sectional areas outlined in

Table 6. Cross sectional areas of the truss bridge structure and threshold values of probabilistic constraints.

Area			Threshold
BC	TC	DG	${\Delta }_v$	${\sigma }_{aw}$
36 in${}^2$	25 in${}^2$	9 in${}^2$	1.0 in	45 ksi

. This analysis utilizes the mean values of the forces and modulus of elasticity with the objective of determining both the maximum vertical displacement and normal stress within the structure. The results of this deterministic analysis are depicted in Figure 11. Then, probabilistic constraints are established for both the maximum displacement and stress. These constraints are characterized by specific threshold values, where the maximum displacement is limited to ${\Delta }_v=1.0$ inch, and the allowable stress is set at ${\sigma }_{aw}=45$ ksi. Therefore, the limit-state function is written as:

$$\begin{array}{cc}\hfill g\left(\mathbf{X}\right):\mathrm{displacement}& =\left\{{\Delta }_v-max\left(u\left(\mathbf{X}\right)\right)\le 0\right\}\hfill \\ \hfill g\left(\mathbf{X}\right):\mathrm{stress}& =\left\{{\sigma }_{aw}-max\left(\right|\sigma \left(\mathbf{X}\right)\left|\right)\le 0\right\}\hfill \end{array}$$

(42)

The results of reliability analysis presented in

Table 7. Reliability analysis results for the truss bridge structure obtained through various approaches without noise.

Limit-State Function	Method	${\mathit{P}}_{\mathit{f}}$	${\Delta }_{{\mathit{P}}_{\mathit{f}}}$	$\mathit{\beta }$	${\mathit{N}}_{\mathit{c}\mathit{a}\mathit{l}\mathit{l}}$
Stress	FORM	$3.76\times {10}^{-3}$	-	2.673	48
	SORM	$3.95\times {10}^{-3}$	-	2.656	128
	MCS	$3.92\times {10}^{-3}$	0.05	2.659	${10}^5$
	AK-MCS	$4.11\times {10}^{-3}$	0.05	2.643	52
	IS	$4.22\times {10}^{-3}$	0.05	2.634	1048
	HAK	$4.02\times {10}^{-3}$	0.02	2.650	43
Displacement	FORM	$9.19\times {10}^{-2}$	-	1.329	40
	SORM	$9.35\times {10}^{-2}$	-	1.320	120
	MCS	$9.43\times {10}^{-2}$	0.01	1.315	${10}^5$
	AK-MCS	$9.51\times {10}^{-2}$	0.01	1.311	166 (${10}^5$)
	IS	$9.44\times {10}^{-2}$	0.04	1.314	1040
	HAK	$9.41\times {10}^{-2}$	0.01	1.316	68

, utilizing FORM, SORM, MCS, AK-MCS, IS, and the proposed HAK method, indicate that the overall computational cost is lower for FORM, but it yields less accurate results. In contrast, the other approaches produce results that closely align with each other.

In the exploration of noise impact on the truss structure problem, noise, denoted as $\epsilon$ with a variance of ${\sigma }_{\mathrm{noise}}^2$, is introduced. This study assumes that observed responses, including normal stress, are subject to noise. Specifically, varying variances of noise are added to a truss member where the maximum stress is observed. The proposed framework, outlined in Section 4, is applied to estimate the failure probability of the stress limit-state function in the presence of noise. Four different variances (${\sigma }_{\mathrm{noise}}^2=\left\{5.06,9.92,20.25,45.56\right\}{\mathrm{kips}}^2$) of Gaussian noise corresponding to coefficients of variation (c.o.v) of 0.05, 0.07, 0.1, and 0.15, respectively, are considered. After five implementations of the proposed framework for each noise variance case, the average failure probabilities are computed. The minimum and maximum failure probabilities for each variance case are also recorded to demonstrate the probability range. For a comprehensive comparison, reliability analysis results of AK-MCS, which does not consider regression progress to account for noise in the reliability analysis, are presented. It is worth noting that methods employing linearization (FORM) and curvature (SORM) to approximate the limit-state function face challenges in estimating failure probability. Specifically, the FORM method resulted in a 50% failure probability, and SORM failed to converge to a stable result. Figure 12 illustrates the ranges of estimated failure probability using the proposed method. The proposed framework demonstrates commendable ability in estimating failure probability under varying noise conditions introduced to stress responses. However, the failure probability range computed by AK-MCS for each variance case significantly increases as the variance rises. When noise is considered in the Kriging modeling and active learning process, a more stable outcome (lower range of failure probability) is observed with the proposed method.

6.2. Computational Efficiency in Reliability Analysis of Continuum Structure

A continuum structure is considered as a continuous medium for analytical purposes, especially in the field of structural engineering. This concept is prominently applied when modeling and analyzing deformable bodies like beams, plates, and solids, treating them as if they were infinitely divisible. Typically, the continuum structure is discretized using finite elements for numerical analysis, with the accuracy of the finite element analysis being influenced by the mesh size. To evaluate the computational efficiency of the proposed framework in the reliability analysis of a continuum structure with varying mesh sizes, the MBB beam discretized with polygonal elements is investigated. Unstructured polygonal meshes are constructed using Voronoi tessellations, and polygonal finite element analysis is conducted for computing structural response and reliability analysis. Figure 13 illustrates the discretization of the beam with 500 polygonal elements ($N_{elem}=500$), including its loading and boundary conditions. Consider that the point loads $F_1,F_2,\mathrm{and},F_3$ are random variables following different types of marginal distributions, as summarized in

Table 8. Distribution type and parameters of the random variables considered for the reliability analysis of the continuum structure.

Random Variables	Distribution	Mean	Moments
$E_i,i=1,...,n_E$	Lognormal	29,000 ksi	$(0.05·i)·$ 29,000 ksi
$F_1$	Normal	3 kips	0.45 kips
$F_2$	Lognormal	4 kips	0.6 kips
$F_3$	Gamma	5 kips	0.75 kips

. Furthermore, three random variables ($E_1,E_2,E_3$), each following the lognormal distribution with the different second moment, are randomly assigned to polygonal elements shown in Figure 13b. Consequently, each finite element is associated with one modulus of elasticity random variable. During the assembly of the stiffness matrix in the finite element setting, each element’s stiffness matrix is influenced by the assigned modulus of the elasticity random variable. The three groups of polygonal elements, categorized based on the assigned random variable of modulus of elasticity, have a similar number of elements. For a comprehensive computational study, the continuum domain is discretized with different numbers of polygonal elements (500, 1000, 2000, 3000, 5000, 10,000, 20,000 finite elements). The three moduli of elasticity are randomly but uniformly assigned over the entire domain, ensuring a similar distribution of elements for each modulus. Figure 14 illustrates the distribution of the modulus of elasticity. The limit-state function is expressed in terms of compliance, which is a scalar value obtained from the product of the displacement vector and the applied force vector:

$$g\left(\mathbf{x}\right)=\left\{C_{max}-\underline{\mathbf{u}}{\left(\mathbf{x}\right)}^T\underline{\mathbf{f}}\left(\mathbf{x}\right)\le 0\right\};\underline{\mathbf{K}}\left(\mathbf{x}\right)\underline{\mathbf{u}}\left(\mathbf{x}\right)=\underline{\mathbf{f}}\left(\mathbf{x}\right)$$

(43)

where $\underline{\mathbf{K}},\underline{\mathbf{f}}$, and $\underline{\mathbf{u}}$ denote the stiffness matrix, force vector, and displacement vector, respectively. The threshold value $C_{\max }$ is set to 1.0 in this problem. Figure 15a presents a comparison of failure probabilities obtained by the proposed method (HAK), FORM, SORM, and AK-MCS. Three independent HAKs and AK-MCSs were performed, and the failure probabilities obtained were averaged. An increase in the failure probability is observed across all approaches. Notably, as the number of finite elements increases in the linearized elasticity problem, more accurate analysis results are achieved. Specifically, the compliance increases and converges to a more accurate value with finer mesh discretization. Consequently, an increase in the failure probability is anticipated. However, it is essential to acknowledge that the distribution of random variables of modulus of elasticity for each different number of finite elements (as shown in Figure 14) may also influence the failure probability, potentially impacting the results illustrated in Figure 15a. Figure 15b illustrates the normalized computational time. The normalization is performed with time obtained using the SORM for $N_{Elem}=5000$ (the degrees of freedom (DoF) is 19,320). It is important to highlight that the failure probability and computational cost obtained through AK-MCS reflect the results achieved using the Kriging model with 100 DoE points obtained during the enrichment process, reaching the prescribed maximum added points for all cases in this study. In contrast, the proposed method identified the Kriging model with approximately 38 DoE points. Consequently, the increase in computational cost is not significantly influenced by the mesh sizes in the proposed framework. It is worth noting that, for relatively small mesh sizes, SORM and AK-MCS exhibit faster computational times, given that the active learning process in the proposed method, involving the initial DoE, is relatively expansive. However, as the mesh size increases, resulting in a substantial rise in the computational cost for finite element analysis, there is an overall increase in computation time. Nevertheless, in the proposed method, once the Kriging model is established and trained through the hybrid active learning process, the reliability evaluation process becomes computationally effective and efficient.

To examine the influence of material uncertainty on the failure probability, various material groups ($E_i,i=1,...,5$) are introduced. Specifically, five beam structures, discretized with the same mesh using 1000 polygonal elements, are considered, each assigned a different set of $E_i$ values, as depicted in Figure 16a. The study utilizes the same distribution types for force and modulus of elasticity random variables in Table 8. Figure 16b illustrates the changes in failure probability corresponding to varying numbers of modulus of elasticity groups assigned to polygonal elements. As anticipated, the failure probability increases with the number of material property random variables. This escalation is primarily attributed to the higher standard deviations of modulus of elasticity for a greater number of elasticity groups. Furthermore, due to the nonlinearity of the limit-state function, FORM exhibits a less accurate estimate of the failure probability.

7. Conclusions

This paper introduces a reliability analysis methodology employing surrogates enriched by a hybrid learning process. The hybrid learning focuses on regions near limit-state approximation candidate samples farthest from existing training points, maximizing the probability of misclassifications. The resulting Kriging model is constructed based on identified DoE points. The proposed method proves accurate and efficient through successful applications to diverse reliability problems, including highly nonlinear limit-state functions and complex structures. When compared to traditional learning functions, the HAK scheme exhibits reliable performance. Notably, as reliability analysis problems scale up, the computational costs of both structural and reliability analyses become substantial. Traditional simulation-based approaches show a significant increase in computational time for larger problems, while the proposed Kriging approach, coupled with subset simulations and the hybrid active learning process, demonstrates notable computational efficiency. This efficiency facilitates exploration of diverse scenarios, including those with random material property distributions and real-world complexities, highlighting the versatility and applicability of the proposed method.

For future research directions, the current study primarily addressed component failure probabilities with a single limit-state function. However, in practical scenarios, system reliability often takes precedence as it can describe realistic and practical failure modes and conditions. Nevertheless, system reliability analysis generally entails higher computational costs and more complex steps to consider the statistical dependence of random variables on the performance of limit-state functions, identify the system failure domain, and estimate system failure probabilities. Thus, future research will be dedicated to exploring system reliability analysis using Gaussian process modeling. Additionally, investigating reliability-based design and topology optimization employing Kriging is identified as another promising avenue for future research.