RMFGP: A Rotated Multi-Fidelity Gaussian Process Framework for Supervised Dimension Reduction

Abstract

High-dimensional surrogate modeling with limited high-fidelity data poses a major challenge in uncertainty quantification. Classical supervised dimension reduction methods often fail in this setting due to insufficient accurate observations, while low-fidelity data are abundant but biased. In this work, we propose a Rotated Multi-Fidelity Gaussian Process (RMFGP) framework that enables reliable dimension reduction and surrogate construction under severe data scarcity. The proposed method integrates nonlinear multi-fidelity Gaussian process regression with sliced average variance estimation (SAVE) to iteratively identify informative input directions. Low-fidelity data are first used to extract coarse structural information, which is exploited to rotate the input space prior to multi-fidelity model training. Predictions generated by the trained RMFGP surrogate are then used to refine the dimension reduction, allowing accurate estimation of the central sufficient dimension reduction subspace even when high-fidelity data are scarce. A Bayesian active learning strategy based on predictive uncertainty is further incorporated to adaptively select new high-fidelity samples. Numerical examples, including stochastic partial differential equations, demonstrate that RMFGP significantly improves prediction accuracy, convergence, and uncertainty propagation compared to existing Gaussian process-based dimension reduction approaches, while requiring substantially fewer high-fidelity evaluations.

1. Introduction

High-dimensional uncertainty quantification problems arise frequently in scientific computing and engineering, where quantities of interest depend on a large number of uncertain parameters. In such settings, the number of model evaluations required to accurately explore the input space grows rapidly with dimension, leading to the well-known curse of dimensionality. This challenge is particularly severe when accurate model evaluations are computationally expensive, as is often the case for simulations governed by partial differential equations. Dimension reduction techniques play a central role in alleviating this difficulty by identifying low-dimensional structures that capture the dominant input–output dependence.

A common formulation of supervised dimension reduction assumes that a scalar response Y depends on the input $X\in {\mathbb{R}}^p$ only through a low-dimensional linear subspace, i.e.,

$$Y=f\left(X\right)\approx g\left({\beta }^{T}X\right),$$

(1)

where $\beta \in {\mathbb{R}}^{p\times d}$ with $d\ll p$. The span of $\beta$ is referred to as a sufficient dimension reduction (SDR) subspace, and the smallest such subspace is known as the central subspace [1,2]. A variety of methods have been developed to estimate this structure, including sliced inverse regression (SIR) [3], sliced average variance estimation (SAVE) [4], contour regression [5], and related approaches [6,7,8,9,10,11,12]. These techniques are effective when sufficient high-quality labeled data are available, but their performance can deteriorate rapidly when accurate observations are scarce.

In many practical applications, data are available from models of varying fidelity. Low-fidelity models are typically inexpensive to evaluate but may be biased or incomplete, while high-fidelity models provide accurate information at a significantly higher computational cost. In such multi-fidelity settings, classical supervised dimension reduction methods often fail because the number of high-fidelity samples is insufficient to reliably estimate conditional expectations or variances. Bayesian approaches have been proposed to mitigate data scarcity, such as the Bayesian inverse regression framework in [13], which employs Gaussian process regression to approximate the likelihood and infer dimension-reduced structure. However, these methods typically assume a single fidelity level and can degrade when accurate observations are extremely limited or when data originate from heterogeneous sources.

Multi-fidelity modeling provides a principled framework to combine information from models of differing accuracy and cost. Gaussian process-based multi-fidelity methods, beginning with the autoregressive formulation of Kennedy and O’Hagan [14] and its recursive implementation by Le Gratiet and Garnier [15], have been widely used for surrogate modeling and uncertainty quantification. More recently, nonlinear information fusion strategies based on deep or autoregressive Gaussian processes have been developed to capture complex relationships between fidelity levels [16]. While these approaches have demonstrated strong predictive performance, they primarily focus on forward surrogate accuracy and uncertainty propagation, and do not explicitly address supervised dimension reduction in high-dimensional input spaces.

Several recent works have explored dimension reduction in related contexts, including active subspace methods [6,17,18] and Gaussian process-based kernel dimension reduction techniques [8]. Although effective in moderate-dimensional problems, these methods typically require either a sufficiently large number of high-fidelity samples or direct optimization of high-dimensional projection operators. When both the input dimension is large and high-fidelity data are scarce, such optimization problems become ill-posed due to the large number of parameters involved.

In this work, we propose a Rotated Multi-Fidelity Gaussian Process (RMFGP) framework that addresses these challenges by tightly coupling nonlinear multi-fidelity Gaussian process modeling with supervised dimension reduction and Bayesian active learning. The central idea is to exploit abundant low-fidelity data to extract coarse structural information about the input–output relationship using SAVE, and to use this information to rotate the input space prior to multi-fidelity model training. A nonlinear multi-fidelity Gaussian process surrogate is then constructed in the rotated space, enabling effective information fusion across fidelity levels. By sampling from the trained surrogate, the dimension reduction procedure is iteratively refined, allowing reliable estimation of the central subspace even when the number of high-fidelity samples is small (Figure 1).

To further improve efficiency, a Bayesian active learning strategy driven by predictive uncertainty is incorporated to adaptively enrich the high-fidelity dataset. Depending on user requirements, the proposed framework supports either a rotated surrogate model for improved predictive accuracy or a reduced-order surrogate obtained via a two-stage dimension reduction strategy that combines Bayesian information criterion-based dimension selection with Gaussian process kernel optimization.

The main contributions of this work are summarized as follows:

• Multi-fidelity–informed supervised dimension reduction. We develop a dimension reduction framework that leverages nonlinear multi-fidelity Gaussian process surrogates to enable reliable estimation of the central sufficient dimension reduction subspace under severe high-fidelity data scarcity.
• Rotated multi-fidelity Gaussian process model. A rotated multi-fidelity Gaussian process (RMFGP) is proposed, in which the input space is iteratively rotated using directions identified by supervised dimension reduction and refined through surrogate-generated samples, rather than treated as a fixed preprocessing step.
• Two-stage dimension reduction strategy under data scarcity. To address the ill-posedness of directly learning high-dimensional projection operators with limited high-fidelity data, we introduce a two-stage dimension reduction procedure that combines RMFGP-based rotation with Gaussian process kernel-based optimization, with intrinsic dimension selected via a Bayesian information criterion.
• Integration of Bayesian active learning in multi-fidelity settings. A Bayesian active learning strategy based on predictive uncertainty is incorporated to adaptively select new high-fidelity samples, improving both surrogate accuracy and dimension reduction quality while minimizing computational cost.
• Applications to stochastic partial differential equations. The proposed framework is validated on a range of numerical examples, including stochastic partial differential equations, demonstrating improved prediction accuracy, convergence behavior, and uncertainty propagation compared to existing Gaussian process-based dimension reduction methods.

The remainder of the paper is organized as follows: Section 2 introduces the necessary background on Gaussian process regression, multi-fidelity modeling, supervised dimension reduction, and active learning. Section 3 presents the proposed RMFGP framework and associated algorithms. Section 4 demonstrates the performance of the method through several numerical examples. Conclusions and directions for future work are given in Section 5.

2. Background and Preliminaries

2.1. Gaussian Process Regression

Gaussian process regression (GPR) provides a flexible Bayesian framework for approximating unknown functions from data. The presentation in this section follows [19]. Let $z:{\mathbb{R}}^p\to \mathbb{R}$ denote an unknown scalar-valued function, and suppose that noisy observations are available in the form

$$y_i=z\left(x_i\right)+{\epsilon }_i,i=1,\dots ,n,$$

(2)

where ${\epsilon }_i$ represents independent Gaussian noise.

In the Gaussian process framework, the latent function $z\left(x\right)$ is modeled as a stochastic process,

$$z\left(x\right)\sim \mathcal{GP}(0,k(x,x^{\prime };\theta )),$$

(3)

where $k(·,·;\theta )$ is a positive-definite covariance kernel parameterized by hyperparameters $\theta$. This prior specification encodes assumptions about the smoothness and correlation structure of the function.

Let $\mathbf{X}={[x_1,\dots ,x_n]}^T$ and $\mathbf{y}={[y_1,\dots ,y_n]}^T$ denote the training inputs and outputs. Under the Gaussian likelihood assumption, the hyperparameters $\theta$ can be estimated by maximizing the marginal log-likelihood,

$$logp(\mathbf{y}\mid \mathbf{X},\theta )=-{\displaystyle \frac{1}{2}}log\left|\mathbf{K}\right|-{\displaystyle \frac{1}{2}}{\mathbf{y}}^T{\mathbf{K}}^{-1}\mathbf{y}-{\displaystyle \frac{n}{2}}log\left(2\pi \right),$$

(4)

where $\mathbf{K}$ is the covariance matrix with entries $K_{ij}=k(x_i,x_j;\theta )$.

Given a new input $x^{\ast }$, the posterior predictive distribution of $z\left(x^{\ast }\right)$ is Gaussian,

$$p(z^{\ast }\mid \mathbf{y},\mathbf{X},x^{\ast })=\mathcal{N}\left({\mu }^{\ast }\left(x^{\ast }\right),{\sigma }_{\ast }^2\left(x^{\ast }\right)\right),$$

(5)

with mean and variance given by

$$\begin{array}{cc}\hfill {\mu }^{\ast }\left(x^{\ast }\right)& ={\mathbf{k}}_{\ast n}{\mathbf{K}}^{-1}\mathbf{y},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\sigma }_{\ast }^2\left(x^{\ast }\right)& =k(x^{\ast },x^{\ast })-{\mathbf{k}}_{\ast n}{\mathbf{K}}^{-1}{\mathbf{k}}_{\ast n}^T,\hfill \end{array}$$

(7)

where ${\mathbf{k}}_{\ast n}=[k(x^{\ast },x_1),\dots ,k(x^{\ast },x_n)]$.

The posterior mean provides a surrogate prediction of the unknown function, while the posterior variance naturally quantifies predictive uncertainty, a feature that will be exploited later for active learning and multi-fidelity modeling.

2.2. Multi-Fidelity Gaussian Process Models

Multi-fidelity modeling aims to combine information from models or experiments of varying accuracy and computational cost. In many scientific computing applications, low-fidelity models are inexpensive to evaluate but may exhibit bias or missing physics, whereas high-fidelity models provide accurate predictions at a substantially higher cost. Gaussian process-based multi-fidelity models provide a flexible probabilistic framework for fusing such heterogeneous information.

2.2.1. Linear Autoregressive Multi-Fidelity Models

A widely used formulation is the linear autoregressive model proposed by Kennedy and O’Hagan [14]. Let ${\left\{Z_t\left(x\right)\right\}}_{t=1}^r$ denote a sequence of Gaussian processes corresponding to increasing levels of fidelity, where $t=r$ represents the highest fidelity. The autoregressive relationship between successive fidelity levels is given by

$$Z_t\left(x\right)={\rho }_t{Z}_{t-1}\left(x\right)+{\delta }_t\left(x\right),t=2,\dots ,r,$$

(8)

where ${\rho }_t$ is a scalar correlation parameter and ${\delta }_t\left(x\right)$ is a Gaussian process independent of $Z_{t-1}\left(x\right)$, modeling the discrepancy between fidelity levels.

This construction implies a Markov property across fidelity levels and allows the high-fidelity response to be expressed as a linear transformation of the lower-fidelity model plus a correction term. To reduce computational complexity, Le Gratiet and Garnier [15] proposed a recursive implementation under a nested experimental design, transforming the original multi-fidelity problem into a sequence of standard Gaussian process regressions.

Under this formulation, the predictive mean and variance at fidelity level t can be computed recursively as

$$\begin{array}{cc}\hfill {\mu }_t^{\ast }\left(x^{\ast }\right)& ={\rho }_t{\mu }_{t-1}^{\ast }\left(x^{\ast }\right)+{\mu }_{{\delta }_t}+{\mathbf{k}}_{\ast n_t}^{\left(t\right)}{\mathbf{K}}_t^{-1}\left({\mathbf{y}}_t-{\rho }_t{\mu }_{t-1}^{\ast }\left({\mathbf{x}}_t\right)-{\mu }_{{\delta }_t}\right),\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill {\sigma }_t^{2\ast }\left(x^{\ast }\right)& ={\rho }_t^2{\sigma }_{t-1}^{2\ast }\left(x^{\ast }\right)+k_t(x^{\ast },x^{\ast })-{\mathbf{k}}_{\ast n_t}^{\left(t\right)}{\mathbf{K}}_t^{-1}{\left({\mathbf{k}}_{\ast n_t}^{\left(t\right)}\right)}^T,\hfill \end{array}$$

(10)

where ${\mathbf{K}}_t$ denotes the covariance matrix at fidelity level t.

While effective in problems with strong linear correlations between fidelity levels, this formulation can perform poorly when the relationship between models is highly nonlinear or varies across the input space.

2.2.2. Nonlinear Information Fusion

To address these limitations, Perdikaris et al. [16] introduced a nonlinear autoregressive multi-fidelity Gaussian process model, also referred to as a nonlinear autoregressive Gaussian process (NARGP). In this framework, the relationship between fidelity levels is expressed as

$$Z_t\left(x\right)=g_t\left(x,Z_{t-1}\left(x\right)\right)+{\delta }_t\left(x\right),$$

(11)

where $g_t(·,·)$ is an unknown nonlinear function modeled as a Gaussian process and ${\delta }_t\left(x\right)$ is an independent Gaussian process capturing residual discrepancies.

By augmenting the input space with predictions from the lower-fidelity model, this formulation allows flexible nonlinear correlations between fidelity levels to be learned from data. The resulting surrogate can be interpreted as a deep Gaussian process with a restricted architecture [20,21]. Since the posterior distribution is no longer Gaussian for $t\ge 2$, predictive moments are typically approximated using Monte Carlo sampling.

Nonlinear multi-fidelity Gaussian process models have demonstrated improved robustness and predictive accuracy in a wide range of applications, particularly when linear correlations between fidelity levels are weak or input-dependent. In the present work, this nonlinear information fusion framework serves as a fundamental building block for the proposed RMFGP methodology.

2.3. Supervised Dimension Reduction Methods

Supervised dimension reduction aims to identify low-dimensional structures in the input space that preserve the dependence between inputs and outputs. Let $X:\mathrm{\Omega }\to {\mathbb{R}}^p$ be a random input vector and let $Y:\mathrm{\Omega }\to \mathbb{R}$ denote a scalar response. A central concept in supervised dimension reduction is the notion of a sufficient dimension reduction (SDR) subspace.

Definition 1

(Sufficient dimension reduction). A subspace $\mathcal{S}\subset {\mathbb{R}}^p$ is called a sufficient dimension reduction subspace for Y given X if

$$Y\perp \perp X\mid {\mathbf{B}}^{T}X,$$

(12)

where the columns of $\mathbf{B}\in {\mathbb{R}}^{p\times d}$ form a basis of $\mathcal{S}$ and $d<p$.

Among all SDR subspaces, the smallest one in the sense of set inclusion is referred to as the central subspace, denoted by ${\mathcal{S}}_{Y|X}$ [22]. Once an estimate of the central subspace is obtained, the original high-dimensional regression problem can be approximated by

$$Y=f\left(X\right)\approx g\left({\mathbf{B}}^{T}X\right),$$

(13)

where $g:{\mathbb{R}}^d\to \mathbb{R}$ is an unknown low-dimensional function.

A variety of classical methods have been developed to estimate the central subspace, including sliced inverse regression (SIR) [3] and sliced average variance estimation (SAVE) [4]. These methods rely on discretizing the range of the response variable and estimating conditional moments of standardized inputs.

2.3.1. Sliced Inverse Regression

Sliced inverse regression (SIR) estimates the central subspace by exploiting variations in the conditional mean $\mathbb{E}[X\mid Y]$. After standardizing the input variables, the response Y is partitioned into a finite number of slices, and the covariance of the slice-wise conditional means is used to construct an eigenvalue problem whose leading eigenvectors provide an estimate of the SDR directions.

The computational procedure of SIR adopted in this work is summarized in Algorithm 1. While computationally efficient, SIR may fail to detect symmetric or higher-order dependencies between the inputs and the response.

Algorithm 1 Sliced Inverse Regression (SIR)

1: Compute the sample mean and sample variance: $$\hat{\mu }=E_n\left(X\right),\hat{\sigma }=va{r}_n\left(X\right).$$ and compute the standardized random vectors $$Z_i={\hat{\Sigma }}^{-1/2}(X_i-\hat{\mu }),i=1,\dots ,n.$$ 2: Discretize Y as $\hat{Y}={\sum }_{h=1}^{H}hI(Y\in J_h)$, where a collection of intervals $\{J_1,\dots ,J_h\}$ is a partition of $Y_i$. 3: Approximate $E[Z|\hat{Y}\in J_h]$ or $E\left[Z\right|Y\in J_h]$ by $$E_n\left(Z\right|Y\in J_h)={\displaystyle \frac{E_n\left[ZI(Y\in J_h)\right]}{E_n\left[I(Y\in J_h)\right]}},l=1,\dots ,H$$ 4: Approximate $var\left[E\left(Z\right|\hat{Y}\right)]$ by $$M=\sum _{h=1}^{H}E\left[I(Y\in J_h)\right]E_n\left(Z\right|Y\in J_h)E_n\left(Z^T\right|Y\in J_h)$$ 5: Let ${\hat{v}}_1,\dots ,{\hat{v}}_d$ be the first d eigenvectors of M, let ${\hat{\beta }}_k={\hat{\Sigma }}^{-1/2}{\hat{v}}_k$,$k=1,\dots ,d$. The SDR predictors are $[{\hat{\beta }}_1^T(X_1-\hat{\mu }),\dots ,{\hat{\beta }}_d^T(X_d-\hat{\mu })]$.

2.3.2. Sliced Average Variance Estimation

Sliced average variance estimation (SAVE) extends SIR by incorporating second-order conditional information through the conditional variance $\mathrm{Var}(X\mid Y)$. Following [23], the SAVE matrix is constructed as

$${\mathbf{M}}_{\mathrm{SAVE}}={\displaystyle \frac{1}{H}}\sum _{h=1}^H{\left[{\mathbf{I}}_p-\widehat{\mathrm{Var}}(Z\mid Y\in {\mathcal{J}}_h)\right]}^2,$$

(14)

where Z denotes the standardized input, ${\left\{{\mathcal{J}}_h\right\}}_{h=1}^H$ is a partition of the response space, and ${\mathbf{I}}_p$ is the identity matrix.

The full computational steps of SAVE used throughout this paper are summarized in Algorithm 2. Compared to SIR, SAVE is capable of detecting a broader class of dependencies, including symmetric relationships between inputs and outputs. For this reason, SAVE is adopted as the primary supervised dimension reduction tool in the proposed framework, although other SDR methods could also be incorporated.

Algorithm 2 Sliced Average Variance Estimation (SAVE)

1: Standardize $X_1,\dots ,X_n$ to obtain $Z_i$ as in Algorithm 1. 2: Discretize Y as $\hat{Y}={\sum }_{h=1}^{H}hI(Y\in J_h)$, where a collection of intervals $\{J_1,\dots ,J_h\}$ is a partition of $Y_i$. 3: For each slice $J_h$, compute the sample conditional variance of Z given $Y\in J_h$: $$va{r}_n\left(Z\right|\hat{Y}=h)={\displaystyle \frac{E_n\left[Z{Z}^{T}I(\hat{Y}=h)\right]}{E_n\left[I(\hat{Y}=h)\right]}}$$ 4: Compute the sample version of M: $$M=H^{-1}\sum _{h=1}^H{E}_{n}I(\hat{Y}=h)[I_p-va{r}_n(Z|\hat{Y}=h){]}^2$$ 5: Let ${\hat{v}}_1,\dots ,{\hat{v}}_d$ be the first d eigenvectors of M, let ${\hat{\beta }}_k={\hat{\Sigma }}^{-1/2}{\hat{v}}_k$,$k=1,\dots ,d$. The SDR predictors are $[{\hat{\beta }}_1^T(X_1-\hat{\mu }),\dots ,{\hat{\beta }}_d^T(X_d-\hat{\mu })]$, where $\hat{\mu }=E_n\left(X\right)$.

2.3.3. Selection of Reduced Dimension

A key challenge in supervised dimension reduction is determining the intrinsic dimension d of the central subspace. In this work, the reduced dimension is selected using the Bayesian information criterion (BIC) proposed in [24]. Let ${\lambda }_1\ge {\lambda }_2\ge \dots \ge {\lambda }_p$ denote the eigenvalues of the SAVE-based matrix ${\mathbf{M}}_{\mathrm{SAVE}}+{\mathbf{I}}_p$. The BIC score is defined as

$$G\left(k\right)={\displaystyle \frac{n}{2}}\sum _{l=k+1}^p\left(log{\lambda }_l+1-{\lambda }_l\right)-C_n{\displaystyle \frac{k(2p-k+1)}{2}},$$

(15)

where $C_n$ is a sequence satisfying the conditions specified in [24]. The estimated reduced dimension is then given by

$$d=arg\underset{1\le k\le p-1}{max}G\left(k\right).$$

(16)

In the present work, Algorithms 1 and 2 are applied either directly to observed data or to surrogate-generated samples, depending on the stage of the proposed RMFGP framework. While effective in classical settings, the direct application of SAVE and BIC can be unreliable when the number of accurate observations is small. This limitation motivates the surrogate-assisted dimension reduction strategy developed in the subsequent sections.

2.4. Bayesian Active Learning

Bayesian active learning, also referred to as sequential experimental design, aims to improve model accuracy by adaptively selecting new data points that are expected to provide the greatest information gain [25]. In problems where accurate observations are expensive or time-consuming to obtain, active learning plays a crucial role in achieving reliable inference with limited computational resources.

In the context of this work, observations are available from multiple fidelity levels. Let

$${\mathcal{D}}_L={\left\{(x_i^L,y_i^L)\right\}}_{i=1}^{N_L},{\mathcal{D}}_H={\left\{(x_i^H,y_i^H)\right\}}_{i=1}^{N_H}$$

(17)

denote the low-fidelity and high-fidelity datasets, respectively, where typically $N_L\gg N_H$. A multi-fidelity Gaussian process surrogate is constructed using the combined dataset $\mathcal{D}=\{{\mathcal{D}}_L,{\mathcal{D}}_H\}$, as described in Section 2.2.

The objective of active learning in this setting is to enrich the high-fidelity dataset ${\mathcal{D}}_H$ by selectively querying new high-fidelity observations from a predefined candidate pool. In this work, the candidate pool is chosen to be the set of low-fidelity input locations ${\left\{x_i^L\right\}}_{i=1}^{N_L}$, which are inexpensive to evaluate and already available. At each iteration, a new high-fidelity sample location is selected by maximizing an acquisition function,

$$x_{N_H+1}^H=arg\underset{x\in {\mathcal{D}}_L}{max}a\left(x\right).$$

(18)

Among the various acquisition functions proposed in the literature, such as expected improvement or probability of improvement [26,27,28], we adopt the predictive variance of the multi-fidelity Gaussian process surrogate,

$$a\left(x\right)={\sigma }_{\ast }^2\left(x\right),$$

(19)

where ${\sigma }_{\ast }^2\left(x\right)$ denotes the posterior predictive variance at input x. This choice is motivated by the fact that predictive variance naturally quantifies epistemic uncertainty in Gaussian process models and directly reflects regions of the input space where additional high-fidelity information is most beneficial.

Once the new high-fidelity observation $(x_{N_H+1}^H,y_{N_H+1}^H)$ is obtained, it is added to the training dataset, and the multi-fidelity surrogate is retrained. This process is repeated until a stopping criterion is satisfied, such as reaching a prescribed prediction error threshold on a validation set or exhausting a predefined computational budget.

In the proposed framework, Bayesian active learning serves two complementary purposes. First, it improves the predictive accuracy of the multi-fidelity surrogate model with minimal additional high-fidelity evaluations. Second, by enhancing surrogate quality in regions of high uncertainty, it indirectly improves the reliability of surrogate-assisted supervised dimension reduction procedures employed in subsequent sections.

3. Rotated Multi-Fidelity Gaussian Process Framework

3.1. Problem Setting

We consider a supervised regression problem in which a scalar quantity of interest depends on a high-dimensional input. Let

$$f:{\mathbb{R}}^p\to \mathbb{R}$$

(20)

denote an unknown high-fidelity response function, where the input dimension p may be large. We assume that evaluations of f are expensive and that only a limited number of accurate observations are available.

In addition to the high-fidelity model, we assume access to one or more low-fidelity models that provide approximate evaluations of the response at substantially lower computational cost. For clarity of exposition, we focus on the case of two fidelity levels, although the proposed framework can be extended naturally to more general multi-fidelity settings. Let

$${\mathcal{D}}_H={\left\{(x_i^H,y_i^H)\right\}}_{i=1}^{N_H},{\mathcal{D}}_L={\left\{(x_i^L,y_i^L)\right\}}_{i=1}^{N_L}$$

(21)

denote the high-fidelity and low-fidelity datasets, respectively, with $N_H\ll N_L$ in typical applications.

A central assumption of this work is that the high-fidelity response admits an approximate low-dimensional structure. Specifically, we assume that there exists a matrix $\mathbf{A}\in {\mathbb{R}}^{d\times p}$ with $d\ll p$ such that

$$f\left(x\right)\approx g\left(\mathbf{A}x\right),$$

(22)

where $g:{\mathbb{R}}^d\to \mathbb{R}$ is an unknown function. The row space of $\mathbf{A}$ defines the central sufficient dimension reduction subspace introduced in Section 2.3. Identifying this subspace is critical both for reducing the effective dimensionality of the problem and for constructing accurate surrogate models. Note that throughout this work, we use A to denote rotation matrices and M to denote final reduction matrices, while B in Section 2 refers generically to SDR directions.

The primary challenge addressed in this work arises from the severe scarcity of high-fidelity data. Classical supervised dimension reduction techniques, such as SIR or SAVE, rely on an accurate estimation of conditional moments and typically require a sufficient number of high-fidelity observations. When $N_H$ is small, a direct application of these methods becomes unreliable. At the same time, although low-fidelity data are abundant, they may contain bias or missing information and cannot be used directly to infer the central subspace of the high-fidelity response.

The goal of this work is therefore twofold. First, we aim to develop a reliable strategy for identifying the central dimension reduction subspace of the high-fidelity model by exploiting both low- and high-fidelity data. Second, we seek to construct an accurate surrogate model for the high-fidelity response that can be used for prediction and uncertainty quantification with minimal reliance on expensive model evaluations.

To achieve these objectives, we introduce a Rotated Multi-Fidelity Gaussian Process (RMFGP) framework that tightly integrates multi-fidelity Gaussian process modeling, supervised dimension reduction, and Bayesian active learning. The subsequent subsections describe the construction of the RMFGP model and the associated dimension reduction and sampling strategies in detail.

3.2. Input Rotation via Supervised Dimension Reduction

The first step of the proposed RMFGP framework is to identify informative directions in the input space and use them to define a rotated coordinate system. At this stage, the objective is not to perform aggressive dimension reduction, but rather to improve the representation of the input space so that subsequent multi-fidelity surrogate modeling can be carried out more effectively.

Since low-fidelity data are typically abundant and inexpensive to obtain, we begin by applying supervised dimension reduction to the low-fidelity dataset. Specifically, the sliced average variance estimation (SAVE) method introduced in Section 2.3 is applied to the low-fidelity observations ${\mathcal{D}}_L={\left\{(x_i^L,y_i^L)\right\}}_{i=1}^{N_L}$. The computational procedure follows Algorithm 2, resulting in an estimate of dominant directions that characterize the dependence between low-fidelity inputs and outputs.

Let ${\mathbf{A}}_L\in {\mathbb{R}}^{p\times p}$ denote the orthogonal matrix whose columns are the eigenvectors obtained from the SAVE matrix constructed using the low-fidelity data, ordered according to decreasing eigenvalues. This matrix defines a rotation of the original input space,

$$\tilde{x}={\mathbf{A}}_L^{T}x,$$

(23)

where $\tilde{x}$ represents the rotated input coordinates. All available data, including low-fidelity, high-fidelity, and validation inputs, are transformed using the same rotation.

It is important to emphasize that this step performs a rotation rather than a truncation of the input space. All p dimensions are retained, and no information is discarded at this stage. The purpose of the rotation is to align the coordinate system with directions that are informative for the low-fidelity response, thereby improving the conditioning of subsequent multi-fidelity Gaussian process models.

Although the low-fidelity model may contain bias or missing physics, its response often captures coarse structural information about the underlying input–output relationship. By exploiting this information through SAVE-based rotation, the input space is reorganized so that informative directions are concentrated in the leading coordinates, while less influential directions are pushed toward higher-indexed components. This reorganization facilitates more effective information fusion across fidelity levels in later stages of the RMFGP framework.

The rotated input representation serves as the foundation for constructing the nonlinear multi-fidelity Gaussian process surrogate described in the next subsection. As will be shown, this initial rotation can be further refined using surrogate-generated predictions, leading to improved estimation of the central subspace associated with the high-fidelity response.

3.3. Rotated Multi-Fidelity Gaussian Process Construction

After rotating the input space as described in Section 3.2, we construct a nonlinear multi-fidelity Gaussian process surrogate in the rotated coordinates. This surrogate serves as the core modeling component of the proposed RMFGP framework and provides the probabilistic foundation for subsequent dimension refinement and active learning.

Let $\tilde{x}={\mathbf{A}}_L^{T}x$ denote the rotated input associated with $x\in {\mathbb{R}}^p$. The low-fidelity and high-fidelity datasets in the rotated space are given by

$${\tilde{\mathcal{D}}}_L={\left\{({\tilde{x}}_i^L,y_i^L)\right\}}_{i=1}^{N_L},{\tilde{\mathcal{D}}}_H={\left\{({\tilde{x}}_i^H,y_i^H)\right\}}_{i=1}^{N_H}.$$

(24)

We adopt the nonlinear autoregressive multi-fidelity Gaussian process (NARGP) formulation introduced in [16] to fuse information across fidelity levels. In the rotated space, the high-fidelity response is modeled as

$$Z_H\left(\tilde{x}\right)=g\left(\tilde{x},Z_L\left(\tilde{x}\right)\right)+\delta \left(\tilde{x}\right),$$

(25)

where $Z_L\left(\tilde{x}\right)$ denotes the low-fidelity Gaussian process surrogate, $g(·,·)$ is an unknown nonlinear function modeled as a Gaussian process, and $\delta \left(\tilde{x}\right)$ is an independent Gaussian process capturing residual discrepancies between fidelity levels.

The low-fidelity surrogate $Z_L\left(\tilde{x}\right)$ is first trained using ${\tilde{\mathcal{D}}}_L$ by standard Gaussian process regression. Predictions from this surrogate are then used as additional inputs to train the high-fidelity model $Z_H\left(\tilde{x}\right)$ using ${\tilde{\mathcal{D}}}_H$. This construction allows the multi-fidelity surrogate to learn complex, input-dependent correlations between fidelity levels while preserving a coherent probabilistic interpretation.

We emphasize that the defining feature of the RMFGP model is the coupling between input rotation and multi-fidelity Gaussian process regression. Unlike standard NARGP models constructed in the original input space, the RMFGP surrogate operates in a rotated coordinate system informed by supervised dimension reduction. This coupling improves the alignment between informative input directions and the covariance structure of the Gaussian process, leading to enhanced predictive performance and numerical stability in high-dimensional settings.

Since the posterior distribution of the high-fidelity surrogate is no longer Gaussian under the nonlinear autoregressive formulation, predictive moments are approximated using Monte Carlo sampling. In particular, for a given input $\tilde{x}$, samples of $Z_L\left(\tilde{x}\right)$ are first drawn from the low-fidelity posterior and then propagated through the Gaussian process $g(·,·)$ to generate samples of $Z_H\left(\tilde{x}\right)$. The predictive mean and variance are subsequently estimated from these samples.

The resulting RMFGP surrogate provides both point predictions and uncertainty quantification for the high-fidelity response. These uncertainty estimates play a crucial role in guiding Bayesian active learning and in enabling surrogate-assisted supervised dimension reduction, as described in the following subsections.

3.4. Iterative Refinement and Active Learning

The rotated multi-fidelity Gaussian process constructed in Section 3.3 provides an initial probabilistic surrogate for the high-fidelity response in the rotated input space. However, because the initial rotation is obtained solely from low-fidelity data, it may not accurately reflect the central subspace associated with the high-fidelity model. To address this limitation, the RMFGP framework employs an iterative refinement strategy that combines surrogate-assisted supervised dimension reduction with Bayesian active learning.

3.4.1. Surrogate-Assisted Refinement of Rotation

Once the RMFGP surrogate is trained, it can be used to generate additional response samples at arbitrary input locations. In particular, surrogate predictions are evaluated at the low-fidelity input locations ${\left\{x_i^L\right\}}_{i=1}^{N_L}$, yielding a set of surrogate-generated high-fidelity responses

$${\hat{\mathcal{D}}}_H={\left\{(x_i^L,{\hat{y}}_i^H)\right\}}_{i=1}^{N_L}.$$

(26)

These surrogate-generated samples are inexpensive to obtain and provide an approximation of the input–output relationship of the high-fidelity model across the input domain.

The SAVE algorithm (Algorithm 2) is then reapplied using the surrogate-generated dataset ${\hat{\mathcal{D}}}_H$ to estimate an updated set of dimension reduction directions. Let ${\mathbf{A}}^{\left(k\right)}$ denote the rotation matrix obtained at iteration k. The input space is subsequently rotated according to

$${\tilde{x}}^{\left(k\right)}={\left({\mathbf{A}}^{\left(k\right)}\right)}^{T}x,$$

(27)

and the RMFGP surrogate is retrained in the updated rotated coordinates.

This surrogate-assisted refinement procedure allows information from the limited high-fidelity data to be propagated throughout the input space via the multi-fidelity surrogate, enabling more reliable estimation of the central subspace than would be possible using the high-fidelity observations alone. In practice, only a small number of refinement iterations is required before the rotation stabilizes.

3.4.2. Integration of Bayesian Active Learning

To further enhance surrogate accuracy and refinement reliability, Bayesian active learning is incorporated within the iterative framework. At each iteration, the predictive uncertainty of the RMFGP surrogate is evaluated over the candidate pool of low-fidelity input locations. New high-fidelity samples are selected by maximizing the predictive variance, as described in Section 2.4, and added to the high-fidelity dataset.

The inclusion of actively selected high-fidelity samples serves two complementary purposes. First, it improves the local accuracy of the RMFGP surrogate in regions of high uncertainty. Second, it enhances the quality of surrogate-generated samples used for supervised dimension reduction, leading to more accurate estimation of rotation directions in subsequent iterations.

The combined process of surrogate-assisted rotation refinement and Bayesian active learning results in a feedback loop in which improved surrogates lead to improved dimension reduction, which in turn leads to improved surrogate construction. This coupling distinguishes the RMFGP framework from conventional multi-fidelity Gaussian process models and classical supervised dimension reduction methods, which typically treat surrogate modeling and dimension reduction as independent tasks.

The iterative refinement procedure is terminated when a prescribed stopping criterion is satisfied, such as stabilization of the estimated dimension reduction directions, convergence of surrogate prediction error, or exhaustion of the available computational budget.

3.5. Two-Stage Dimension Reduction Strategy

The iterative refinement procedure described in Section 3.4 yields a sequence of rotation matrices that progressively align the input coordinates with directions that are informative for the high-fidelity response. While this rotation significantly improves surrogate modeling and information fusion, it does not by itself reduce the dimensionality of the input space. In many applications, however, constructing a reduced-order surrogate model is desirable for efficiency, interpretability, and scalability.

A direct approach to supervised dimension reduction would be to estimate a projection matrix $\mathbf{A}\in {\mathbb{R}}^{d\times p}$ by optimizing a Gaussian process kernel or likelihood function in the original high-dimensional space. When the number of high-fidelity samples is limited, such an approach is often ill-posed due to the large number of parameters involved. To address this challenge, the RMFGP framework adopts a two-stage dimension reduction strategy that decouples the identification of informative directions from the final reduction of dimensionality.

3.5.1. Stage I: RMFGP-Based Rotation

In the first stage, the RMFGP framework is used to identify and refine a rotation of the input space through surrogate-assisted SAVE and active learning, as described in Section 3.2, Section 3.3 and Section 3.4. Let ${\mathbf{A}}^{(\ast )}\in {\mathbb{R}}^{p\times p}$ denote the final rotation matrix obtained upon convergence of the iterative refinement procedure. This rotation concentrates the dominant input–output dependence into the leading coordinates of the transformed input

$$\tilde{x}={\left({\mathbf{A}}^{(\ast )}\right)}^{T}x,$$

(28)

while retaining the full dimensionality of the problem.

The primary role of this stage is to reorganize the input space so that the effective dimensionality of the response is reduced, even though all p coordinates are preserved. By doing so, the subsequent dimension reduction step can be carried out in a significantly better-conditioned coordinate system.

3.5.2. Stage II: Reduced-Order Surrogate Construction

In the second stage, a reduced-order surrogate model is constructed by selecting a subset of the rotated coordinates. The intrinsic dimension d of the reduced representation is determined using the Bayesian information criterion described in Section 2.3, applied to the SAVE-based matrix computed from surrogate-assisted data. Once d is selected, the reduced input is defined as

$${\tilde{x}}_r={({\tilde{x}}_1,\dots ,{\tilde{x}}_d)}^T,$$

(29)

corresponding to the leading d components of the rotated input vector.

A Gaussian process surrogate is then trained using ${\tilde{x}}_r$ as input. In this reduced space, kernel hyperparameter optimization becomes well-posed due to the substantially lower dimensionality. This allows for the construction of an accurate and efficient reduced-order surrogate for the high-fidelity response.

3.5.3. Algorithmic Summary

The proposed two-stage strategy offers several advantages. First, it avoids the direct optimization of high-dimensional projection operators using limited high-fidelity data. Second, it exploits the strengths of multi-fidelity modeling to guide dimension reduction in a data-efficient manner. Finally, it provides flexibility to the user: depending on application requirements, one may either use the rotated full-dimensional surrogate for maximum accuracy or employ the reduced-order surrogate for improved computational efficiency.

The complete RMFGP procedure, including input rotation via supervised dimension reduction, nonlinear multi-fidelity Gaussian process construction, surrogate-assisted refinement, Bayesian active learning, and the two-stage dimension reduction strategy described above, is summarized in Algorithm 3. This algorithm serves as a practical implementation of the RMFGP framework and highlights the interaction between surrogate modeling, dimension reduction, and adaptive sampling in high-dimensional, data-scarce multi-fidelity settings.

Algorithm 3 Rotated multi-fidelity Gaussian process model (RMFGP)

1: Input: low-fidelity data sets $\{X_L,y_L\}$, high-fidelity set $\{X_H,y_H\}$ and validation data set $\{X_T,y_T\}$, threshold $\xi$, maximum iteration number I, reduction parameter $flag=0$ or 1. 2: Apply SAVE method Algorithm 2 only on the low-fidelity data to compute the first rotation matrix $A_T$ to extract the principle direction information in the low-fidelity data. Then apply $A_T$ to all $X=(X_L,X_H,X_T)$. 3: Perform NARGP on new training data $\{\widehat{X_L},y_L\}$, $\{\widehat{X_H},y_H\}$ to get the prediction $\widehat{y_T}$ at $\widehat{X_T}$. Then perform SAVE method again on new $\{\widehat{X_T},\widehat{y_T}\}$ to compute the rotation matrix $\hat{A}$ and apply $\hat{A}$ to all Xs. 4: Check whether the threshold of the generalization error meet. If not, perform Bayesian active learning method to locate $x^{\ast }$ where the prediction variance achieves maximum. Sample $\{x^{\ast },y_H^{\ast }\}$ and add them into the high-fidelity training set. 5: Repeat step 2 and 3 until the generalization error threshold fulfilled or the maximum iteration number is reached. 6: Compute the rotation matrix $M_1=A_T{\prod }_i{\hat{A}}_i$, and build the rotated model. 7: If dimension reduction parameter $flag=1$, compute the intrinsic dimension d through BIC and compute the reduction matrix ${\hat{M}}_1$ consisting of the first s principle columns of $M_1$, where $d<s<p$. 8: Apply Gaussian processes dimension reduction technique to compute reduction matrix $M_2$ which is $s\times d$. Then build the reduced model with final reduction matrix $M={\hat{M}}_1{M}_2$.

4. Numerical Results

In this section, we present four numerical examples to demonstrate the effectiveness of the proposed RMFGP framework in high-dimensional, data-scarce settings. We first assess the accuracy of the estimated rotation or reduction matrix using the subspace distance metric introduced in [24],

$$m(\mathbf{A},\hat{\mathbf{A}})={\parallel \mathbf{P}-\hat{\mathbf{P}}\parallel }_F,$$

(30)

where $\mathbf{A}$ and $\hat{\mathbf{A}}$ denote the true and estimated central subspace matrices, respectively; $\mathbf{P}$ and $\hat{\mathbf{P}}$ are the corresponding projection matrices; and ${\parallel ·\parallel }_F$ denotes the Frobenius norm. To highlight the benefit of the proposed approach, we compare the reduction matrix computed by RMFGP (with $\mathtt{flag}=1$) against the GP-SAVE method introduced in [13], using the same number of high-fidelity samples.

Once the rotation or reduction matrix is obtained, new training and test datasets are generated to construct and evaluate Gaussian process surrogate models. To ensure a fair comparison, two cases are considered depending on the reduction parameter $\mathtt{flag}$. When $\mathtt{flag}=0$, the proposed RMFGP model and standard Gaussian process regression (GPR) are trained on the rotated and original input spaces, respectively. When $\mathtt{flag}=1$, the reduced-order surrogate constructed using RMFGP is compared against GP-SAVE, where the same reduced dimension is used.

Model accuracy is quantified using the relative error

$$e={\displaystyle \frac{\parallel u-\hat{u}{\parallel }_2}{{\parallel u\parallel }_2}},$$

(31)

where u and $\hat{u}$ denote the exact and predicted high-fidelity responses evaluated on the test set.

4.1. Linear Example: Poisson’s Equation

We first consider a linear multi-fidelity example in which the high-fidelity and low-fidelity models are related through additive high-dimensional perturbations. This example represents a simple setting in which the correlation between fidelity levels is predominantly linear.

Specifically, we define

$$\begin{array}{cc}\hfill f_H\left(x\right)& =sin\left(\pi (x_1+x_3)\right)+sin\left(\pi (x_1+x_2)\right)+2,\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill f_L\left(x\right)& =f_H\left(x\right)+x_3{x}_4{x}_5{x}_6,\hfill \end{array}$$

(33)

where $x_i$, $i=1,\dots ,6$, are independent random variables uniformly distributed on $[0,1]$. Both functions can be interpreted as solutions of Poisson’s equation

$${\nabla }^{2}f=h\left(x\right),$$

(34)

with forcing terms

$$\begin{array}{cc}\hfill h_H\left(x\right)& =2{\pi }^{2}sin\left(\pi (x_1+x_3)\right)+2{\pi }^{2}sin\left(\pi (x_1+x_2)\right),\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill h_L\left(x\right)& =h_H\left(x\right)+{\displaystyle \frac{1}{6}}\left(x_3^3{x}_4{x}_5{x}_6+x_3{x}_4^3{x}_5{x}_6+x_3{x}_4{x}_5^3{x}_6+x_3{x}_4{x}_5{x}_6^3\right).\hfill \end{array}$$

(36)

From the analytical form of $f_H$, it is evident that the true intrinsic dimension is $d=2$. The corresponding central subspace is spanned by ${\beta }_1={(1,0,1,0,0,0)}^T$ and ${\beta }_2={(1,1,0,0,0,0)}^T$. The total input dimension is $p=6$. In this experiment, we use $N_L=200$ low-fidelity samples and $N_T=500$ test points. For each case, the initial number of high-fidelity samples is set to $N_H-10$, and two iterations of Bayesian active learning are performed, with five new high-fidelity samples added per iteration.

Table 1. Accuracy of the dimension reduction matrix measured by the metric $m(A,\hat{A})$ with different numbers of high-fidelity samples for linear examples.

	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$	${\mathit{N}}_{\mathit{H}}=40$
RMFGP ($flag=1$)	0.133262	0.112705	0.066355	0.043337
GP-SAVE	0.396780	0.243222	0.221707	0.202306

Note: Bold numbers indicate the lowest error among all methods for each fixed $N_H$.

reports the subspace distance $m(\mathbf{A},\hat{\mathbf{A}})$ obtained with different numbers of high-fidelity samples when $\mathtt{flag}=1$. Both RMFGP and GP-SAVE exhibit consistent improvement as $N_H$ increases. However, RMFGP achieves substantially higher accuracy across all sample sizes. This improvement can be attributed to the effective exploitation of low-fidelity information and the refinement enabled by Bayesian active learning, which together enhance the estimation of the central subspace under limited high-fidelity budgets.

Table 2. BIC: $G\left(k\right)$ for linear examples.

	$k=1$	$k=2$	$k=3$	$k=4$	$k=5$	$k=6$
$G\left(k\right)$	0.936692	0.959209	0.927219	0.894635	0.860916	0.826732

Note: Bold number indicates the highest BIC value.

presents the values of the Bayesian information criterion $G\left(k\right)$. The criterion attains its maximum at $k=2$, correctly identifying the intrinsic dimension $\hat{d}=2$, consistent with the analytical structure of $f_H$.

Table 3. Relative error e of the RMFGP model compared to the standard GP for linear examples. If $flag=0$, the inputs are simply rotated by the rotation matrix from the RMFGP model before fed into a new GP surrogate model. It is compared to a standard GP model. If $flag=1$, the inputs are reduced to dimension $d=2$. For comparison, the inputs for the standard GP are reduced to dimension $d=2$ by a reduction matrix computed by the SAVE method using the same number of high-fidelity training points.

	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$	${\mathit{N}}_{\mathit{H}}=40$
RMFGP ($flag=0$)	0.051382	0.030260	0.019012	0.007531
GP	0.060513	0.043898	0.027694	0.023358
RMFGP ($flag=1$)	0.051358	0.039532	0.026999	0.020309
GP-SAVE	0.084310	0.077906	0.072945	0.066365

Note: Bold numbers indicate the relative error of the proposed method for each fixed $N_H$.

summarizes the relative prediction errors on the test set. When $\mathtt{flag}=0$, RMFGP trained on the rotated inputs consistently outperforms standard GPR trained on the original inputs. When $\mathtt{flag}=1$, the reduced-order RMFGP surrogate also achieves lower errors than GP-SAVE. In RMFGP, the input dimension is first reduced from $p=6$ to $s=3$, followed by a further reduction to $d=2$ using Gaussian process-based dimension reduction. This hierarchical reduction alleviates the burden of hyperparameter optimization and allows accurate surrogate construction with fewer high-fidelity samples.

Figure 2 illustrates the mean squared error as a function of $N_H$. In both the rotated and reduced settings, RMFGP exhibits faster convergence and lower error levels than the comparison methods. Figure 3 further confirms this observation through correlation plots at $N_H=30$, where RMFGP predictions align closely with the ideal correlation line, while the comparison methods exhibit larger deviations.

4.2. Nonlinear Example

The second example demonstrates the capability of the proposed RMFGP framework to model nonlinear problems and nonlinearly correlated relationships between low- and high-fidelity data that cannot be captured by linear autoregressive multi-fidelity models.

We consider the following pair of functions:

$$\begin{array}{cc}\hfill f_H\left(x\right)& =exp\left(0.2\sum _{i=1}^{10}{x}_i\right),\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill f_L\left(x\right)& =x_4{f}_H\left(x\right),\hfill \end{array}$$

(38)

where $x_i$, $i=1,\dots ,10$, are independent random variables uniformly distributed on $[0,1]$. From the analytical expressions of $f_H$ and $f_L$, it follows that the intrinsic dimension of the low-fidelity model is two, whereas the intrinsic dimension of the high-fidelity model is one. Such a discrepancy may arise in practical applications when low-fidelity models contain additional noise or spurious dependencies that are absent in the high-fidelity response.

The original input dimension is $p=10$, and the true central subspace of the high-fidelity model is spanned by the vector ${\beta }_1={(1,1,\dots ,1)}^T$. In this experiment, we use $N_L=200$ low-fidelity training samples and $N_T=500$ test points. For all cases, the initial number of high-fidelity samples is set to $N_H-5$. The Bayesian active learning scheme is then employed to add two high-fidelity samples in the first iteration and three additional samples in the second iteration, after which the stopping criterion is satisfied.

Table 4. Accuracy of the dimension reduction matrix measured by the metric $m(A,\hat{A})$ with different numbers of high-fidelity samples for nonlinear examples.

	${\mathit{N}}_{\mathit{H}}=10$	${\mathit{N}}_{\mathit{H}}=15$	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$
RMFGP ($flag=1$)	0.375618	0.217788	0.032944	0.019125
GP-SAVE	1.234884	1.004467	0.177209	0.080237

Note: Bold numbers indicate the lowest error among all methods for each fixed $N_H$.

reports the subspace distance $m(\mathbf{A},\hat{\mathbf{A}})$ defined in (30) for different numbers of high-fidelity samples. Both RMFGP and standard Gaussian process regression exhibit consistent improvement as $N_H$ increases. However, RMFGP achieves significantly higher accuracy in estimating the central subspace with substantially fewer high-fidelity samples. In this example, the low-fidelity data contain complete but noisy information about the high-fidelity response, and the proposed RMFGP framework effectively exploits this structure to mitigate the impact of noise.

The Bayesian information criterion values $G\left(k\right)$ are reported in

Table 5. BIC: $G\left(k\right)$ for nonlinear examples.

	$k=1$	$k=2$	$k=3$	$k=4$	$k=5$
$G\left(k\right)$	0.962213	0.935511	0.906146	0.876473	0.846605
	$k=6$	$k=7$	$k=8$	$k=9$	$k=10$
$G\left(k\right)$	0.816363	0.785284	0.753946	0.722162	0.689382

Note: Bold number indicates the highest BIC value.

. The criterion attains its maximum at $k=1$, correctly identifying the intrinsic dimension $\hat{d}=1$, consistent with the analytical structure of $f_H$. This result demonstrates the ability of RMFGP to reliably identify intrinsic dimensionality even when low-fidelity data introduce additional spurious directions.

When the reduction parameter $\mathtt{flag}=1$, the RMFGP framework first reduces the input dimension from $p=10$ to $s=3$ using the RMFGP-based rotation. A subsequent Gaussian process-based dimension reduction step is then applied to further reduce the dimension from $s=3$ to $d=1$.

Table 6. Relative error e of the RMFGP model compared to the standard GP for nonlinear examples.

	${\mathit{N}}_{\mathit{H}}=10$	${\mathit{N}}_{\mathit{H}}=15$	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$
RMFGP ($flag=0$)	0.037386	0.008324	0.001096	0.000810
GP	0.171120	0.139623	0.034945	0.024370
RMFGP ($flag=1$)	0.080942	0.045254	0.006634	0.003374
GP-SAVE	0.175399	0.133578	0.026675	0.021621

Note: Bold numbers indicate the relative error of the proposed method for each fixed $N_H$.

and Figure 4 present the relative error and mean squared error as functions of $N_H$ for four different surrogate models. In all cases, RMFGP achieves lower prediction errors than the comparison methods. In particular, the reduced-order RMFGP surrogate exhibits superior accuracy in the small-sample regime, highlighting the effectiveness of the proposed framework in high-dimensional, data-scarce nonlinear settings. Figure 5 further confirms this observation through correlation plots at $N_H=20$, where RMFGP predictions align closely with the ideal correlation line, while the comparison methods exhibit larger deviations.

Figure 6 is the prediction plot at $N_H=20$. The black star line is the exact prediction with $x_d$ computed by the true dimension reduction matrix. The red circle line obtained by RMFGP ($flag=1$) fits the curve well. The blue square line has a large error at some locations of $x_d$. The successful estimation of the central subspace as well as the predictions on the test set prove the ability of our method to exclude the effect of noise with a relatively small set of highly accurate data, which is useful in many real-world applications where high-fidelity data are expensive or hard to collect.

4.3. Advection Equation

The third example is aimed at evaluating the performance of RMFGP for stochastic partial differential equation problems in [29]. Consider the one-dimensional advection equation with random input

$${\displaystyle \frac{\partial }{\partial t}}u(x,t;\xi )+{\displaystyle \frac{a}{4}}\sum _{i=1}^5{\xi }_i{\displaystyle \frac{\partial }{\partial x}}u(x,t;\xi )=0,$$

(39)

with the initial condition

$$u(x,0;\xi )=sin\left(\pi (x+1)\right)+1,$$

(40)

where a is a constant coefficient, $x\in [0,1]$, and $\xi =({\xi }_1,\dots ,{\xi }_5)\in {[0,1]}^5$ is a random vector.

Under this setting, an analytical high-fidelity solution is available and is denoted by $u_H$,

$$u_H(x,t;\xi )=sin\left(\pi \left(x-{\displaystyle \frac{a}{4}}t\sum _{i=1}^5{\xi }_i+1\right)\right)+1.$$

(41)

The low-fidelity data are generated from

$$u_L(x,t;\xi )=sin\left(\pi \left(x-{\displaystyle \frac{a}{4}}t\sum _{i=3}^5{\xi }_i+1\right)\right)+1.$$

(42)

Here, $\xi$ is generated by i.i.d. uniformly distributed random variables on ${[0,1]}^5$, and the constant a is fixed to be 1. Compared to the previous examples, the low-fidelity model in this setting contains missing information, since it depends only on $({\xi }_3,{\xi }_4,{\xi }_5)$ rather than the full input vector.

In this experiment, the high- and low-fidelity function values are computed at $x=0.5$ and $t=1$. From the analytical expression of $u_H$, the true reduced dimension is $d=1$, and the corresponding reduction matrix is $\mathbf{A}=\mathrm{span}\left\{{\beta }_1\right\}$ with ${\beta }_1={(1,1,1,1,1)}^T$.

Table 7. BIC: $G\left(k\right)$ for the advection equation.

	$k=1$	$k=2$	$k=3$	$k=4$	$k=5$
$G\left(k\right)$	0.875394	0.846415	0.816565	0.786153	0.755256

Note: Bold number indicates the highest BIC value.

reports the BIC values $G\left(k\right)$; the criterion attains its maximum at $k=1$, indicating that the estimated intrinsic dimension is $\hat{d}=1$.

For the numerical experiments, the number of low-fidelity training samples is set to $N_L=200$, and the number of test samples is $N_T=500$. For all cases with different $N_H$, the experiment starts from $(N_H-10)$ high-fidelity samples, and the Bayesian active learning procedure adds five high-fidelity samples per iteration, with two iterations performed before termination.

Table 8. Error of the dimension reduction matrix measured by the metric $m(A,\hat{A})$ with different numbers of high-fidelity samples for the advection equation.

	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$
RMFGP ($flag=1$)	0.249067	0.114606	0.072898	0.066792
GP-SAVE	1.410894	1.269664	0.483262	0.450924

Note: Bold numbers indicate the lowest error among all methods for each fixed $N_H$.

summarizes the accuracy of the estimated central subspace using the metric $m(\mathbf{A},\hat{\mathbf{A}})$. As expected, both methods improve as $N_H$ increases; however, RMFGP ($\mathtt{flag}=1$) consistently outperforms GP-SAVE across all sample sizes. In particular, when the number of high-fidelity samples is insufficient, GP-SAVE may fail to identify the central subspace accurately.

The relative errors reported in

Table 9. Relative error e of the RMFGP model compared to the standard GP for the advection equation.

	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$
RMFGP ($flag=0$)	0.645469	0.084900	0.068312	0.061210
GP	1.025144	0.964663	0.705019	0.672353
RMFGP ($flag=1$)	0.277659	0.260726	0.216323	0.084661
GP-SAVE	0.743488	0.665575	0.406130	0.384018

Note: Bold numbers indicate the relative error of the proposed method for each fixed $N_H$.

and the MSE curves in Figure 7 further confirm this behavior. RMFGP achieves smaller prediction errors and exhibits faster convergence than the baseline GP model. The correlation plots in Figure 8 and the prediction plots in Figure 9 also demonstrate that RMFGP yields improved generalization performance and more accurate regression behavior than the comparison approach. Overall, this example illustrates that RMFGP can successfully approximate the central subspace in a stochastic differential equation problem, even when the low-fidelity model contains missing information.

4.4. Elliptic Equation

The final example illustrates the performance of the proposed RMFGP framework for a more challenging stochastic partial differential equation. We consider the one-dimensional elliptic equation with a random, high-order coefficient

$$-{\displaystyle \frac{d}{dx}}\left(a(x;\xi ){\displaystyle \frac{du(x;\xi )}{dx}}\right)=1,x\in (0,1),$$

(43)

subject to homogeneous Dirichlet boundary conditions

$$u(0;\xi )=u(1;\xi )=0.$$

(44)

The diffusion coefficient $a(x;\xi )$ depends on a random input vector $\xi =({\xi }_1,\dots ,{\xi }_6)$ and is defined differently for the high- and low-fidelity models as

$$\begin{array}{cc}\hfill a_H(x;\xi )& ={\displaystyle \frac{1}{{\xi }_1+sin\left(x({\xi }_1+{\xi }_2+{\xi }_3+{\xi }_4)\right)+1}},\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill a_L(x;\xi )& ={\displaystyle \frac{1}{0.1+sin\left(x({\xi }_1+{\xi }_2+{\xi }_3+{\xi }_4)\right)+1}}.\hfill \end{array}$$

(46)

The random variables ${\xi }_i$ are assumed to be independent and uniformly distributed on $[0,1]$. Compared with the high-fidelity model, the low-fidelity model introduces a systematic bias through the constant shift in the denominator, while retaining partial information about the stochastic structure of the coefficient.

For this elliptic problem, a deterministic solution representation exists and is given by

$$u(x;\xi )=u(0;\xi )+{\int }_0^x{\displaystyle \frac{a(0;\xi )u^{\prime }(0;\xi )-y}{a(y;\xi )}}dy.$$

(47)

Applying the boundary conditions yields

$$a(0;\xi )u^{\prime }(0;\xi )={\displaystyle \frac{{\displaystyle {\int }_0^1\frac{y}{a(y;\xi )}dy}}{{\displaystyle {\int }_0^1\frac{1}{a(y;\xi )}dy}}}.$$

(48)

The integrals appearing in the solution are evaluated using highly accurate numerical quadrature. Unlike the previous examples, no closed-form analytical expression for $u(x;\xi )$ is available.

In this experiment, the solution is evaluated at the spatial location $x=0.7$. The exact central subspace cannot be inferred directly from the governing equation. Instead, it is approximated using the classical SAVE method applied to a large reference dataset consisting of 10,000 high-fidelity samples. The intrinsic dimension is determined via the Bayesian information criterion, and the corresponding BIC values $G\left(k\right)$ are reported in

Table 10. BIC: $G\left(k\right)$ for the elliptic equation.

	k = 1	k = 2	k = 3	k = 4	k = 5	k = 6
G(k)	0.701881	0.732303	0.709489	0.685960	0.662142	0.637976

Note: Bold number indicates the highest BIC value.

. The criterion attains its maximum at $k=2$, indicating that the true reduced dimension is $\hat{d}=2$.

The number of low-fidelity training samples is fixed at $N_L=200$, and the number of test samples is set to $N_T=500$. For all cases with different values of $N_H$, the experiment starts with $(N_H-5)$ high-fidelity samples. Two iterations of Bayesian active learning are performed, with two samples added in the first iteration and three samples added in the second.

Table 11. Error of the dimension reduction matrix measured by the metric $m(A,\hat{A})$ with different numbers of high-fidelity samples for the elliptic equation.

	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$
RMFGP ($flag=1$)	0.159129	0.141972	0.122867	0.120856
GP-SAVE	0.921045	0.387880	0.250663	0.238877

Note: Bold numbers indicate the lowest error among all methods for each fixed $N_H$.

summarizes the accuracy of the estimated central subspace using the metric $m(\mathbf{A},\hat{\mathbf{A}})$.

Table 12. Relative error e of the RMFGP model compared to the standard GP for the elliptic equation.

	${\mathit{N}}_{\mathit{H}}=20$	${\mathit{N}}_{\mathit{H}}=25$	${\mathit{N}}_{\mathit{H}}=30$	${\mathit{N}}_{\mathit{H}}=35$
RMFGP ($flag=0$)	0.015076	0.007989	0.004557	0.001545
GP	0.060633	0.008441	0.006939	0.004409
RMFGP ($flag=1$)	0.014057	0.011540	0.011276	0.010667
GP-SAVE	0.059538	0.026454	0.020915	0.014822

Note: Bold numbers indicate the relative error of the proposed method for each fixed $N_H$.

reports the relative prediction errors for different surrogate models. Figure 10 presents the mean squared error as a function of the number of high-fidelity samples, while Figure 11 shows the corresponding correlation plots at $N_H=25$. Across all metrics, RMFGP consistently outperforms the comparison Gaussian process models and GP-SAVE, exhibiting behavior similar to that observed in the previous examples.

As the number of high-fidelity samples increases, RMFGP with $\mathtt{flag}=0$ (rotated inputs without truncation) achieves the best predictive performance. This observation reflects the fact that accurate identification of the principal directions improves the quality of the surrogate.

Finally, the uncertainty quantification analysis is also performed for this model. With the dimension reduction matrix M obtained by the RMFGP model with $flag=1$, a new Gaussian process surrogate can be built by pre-processing all training data with M to reduce the input dimension to $d=2$. Then, 2000 samples for $\xi$ are drawn from an i.i.d uniformly distribution. Figure 12 presents the average of means and standard deviations(std) of those 2000 cases along with 50 different x in $[0,1]$. The x-axis represents the indexes of different x values. The y-axis is the average mean for $\left(a\right)$ and std for $\left(b\right)$ in each case. The ground truth is the black line. The green diamond line is obtained by the pure SAVE method if we give large enough training data, i.e., high-fidelity data samples. Here, we give 10,000 samples in order to obtain these results. Note that there are only 35 high-fidelity samples in the RMFGP model. As shown in Figure 12a, all four methods have similar performance. However, Figure 12b shows that our RMFGP method has a smaller std compared to the GP-SAVE method. This shows RMFGP is more confident about the predictions.

5. Conclusions

In this work, we proposed a Rotated Multi-Fidelity Gaussian Process (RMFGP) framework for surrogate modeling, dimension reduction, and uncertainty quantification in high-dimensional settings with severely limited high-fidelity data. The central idea of the proposed approach is to tightly integrate multi-fidelity Gaussian process modeling, supervised dimension reduction, and Bayesian active learning within a unified iterative framework. By exploiting abundant low-fidelity data to guide an initial rotation of the input space and refining this rotation through surrogate-assisted dimension reduction and adaptive sampling, RMFGP enables reliable identification of the central subspace associated with the high-fidelity response.

A key feature of RMFGP is the distinction between input rotation and dimensional truncation. Rather than performing aggressive dimension reduction directly from scarce high-fidelity data, the proposed two-stage strategy first reorganizes the input space to concentrate informative directions and then constructs reduced-order surrogates in a well-conditioned coordinate system. This design alleviates the ill-posedness commonly encountered in high-dimensional Gaussian process regression and allows for accurate surrogate construction with substantially fewer high-fidelity evaluations.

The effectiveness of the RMFGP framework was demonstrated through a sequence of numerical examples, including linear and nonlinear algebraic models as well as stochastic partial differential equations. Across all test cases, RMFGP consistently achieved more accurate estimation of the central subspace, improved predictive performance, and more reliable uncertainty quantification compared to standard Gaussian process regression and GP-SAVE methods, particularly in the small-sample regime. The results also highlight the benefit of Bayesian active learning in enhancing both surrogate accuracy and dimension reduction quality under limited computational budgets.

Several directions for future research remain open. The present work focuses on problems with moderate input dimensionality and smooth response functions; extensions to problems with discontinuities or sharp gradients would be of interest. In addition, while the current framework employs Gaussian process surrogates, the rotation and refinement strategy could be combined with other probabilistic or operator-learning models. Finally, extending RMFGP to handle time-dependent problems and more complex multi-physics systems represents a promising direction for further investigation.