Uncertainty Quantification for Numerical Solutions of the Nonlinear Partial Differential Equations by Using the Multi-Fidelity Monte Carlo Method

Abstract

The Monte Carlo simulation is a popular statistical method to estimate the effect of uncertainties on the solutions of nonlinear partial differential equations, but it requires a huge computational cost of the deterministic model, and the convergence may become slow. For this reason, we developed the multi-fidelity Monte Carlo (MFMC) methods based on data-driven low-fidelity models for uncertainty analysis of nonlinear partial differential equations. Firstly, the nonlinear partial differential equations are transformed into ordinary differential equations (ODEs) by using finite difference discretization or Fourier transformation. Then, the reduced dimension model and discrete empirical interpolation method (DEIM) are coupled to construct effective nonlinear low-fidelity models in ODEs system. Finally, the MFMC method is used to combine the output information of the high-fidelity model and the low-fidelity models to give the optimal estimation of the statistics. Experimental results of the nonlinear Schrodinger equation and the Burgers’ equation show that, compared with the standard Monte Carlo method, the MFMC method based on the data-driven low-fidelity model in this paper can improve the calculation efficiency significantly.

1. Introduction

Nonlinear partial differential equations are often used to describe mathematical models of physical, mechanical and engineering problems. However, the complexity of the real world and the limitations of people’s cognition, the mathematical models and numerical calculation processes of partial differential equations have various sources of uncertainties, such as uncertain parameters, initial and boundary conditions and a variety of model forms in the same process [1]. In recent years, uncertainty quantification [2,3] has become an emerging discipline in the aerospace and defense security industries, and it has attracted great attention in many cutting-edge areas, including academia and engineering applications.

Among the common uncertainty quantification methods, the Monte Carlo (MC) method is simple, easy to implement and robust [4]. However, the convergence speed of MC method is slow, especially in the application for the uncertainty analysis of high-dimensional problems of numerical methods for partial differential equations (e.g., finite element, finite difference and the Laplace adomian decomposition method [5]), which limits the application in some uncertainty quantification problems. Bonvin et al. [6] proposed a type of MC variance reduction technique, which ameliorates the error partially. However, there is no comprehensive consideration to reduce error, improve convergence speed and optimize the computational cost. Ng and Willcox [7] first proposed a multi-fidelity Monte Carlo method (MFMC) to estimate the mean using the control variate technique. They used efficient low-fidelity models to reduce the computational cost of high-fidelity models in uncertain optimization. Moreover, Peherstorfer et al. [8,9,10,11] extended the multi-fidelity method by using a large number of low-fidelity models and optimized the model allocation according to its corresponding authenticity and cost. In addition, Fleeter et al. [12] proposed a hybrid multi-fidelity method under a reasonable calculation cost, which improved the accuracy of the uncertainty quantification of cardiovascular dynamics. Recently, uncertainty quantification based on multi-fidelity Monte Carlo method has been successfully applied in many fields, as can be seen in [13,14,15,16,17,18,19].

In this work, we aim to extend the application of MFMC to the field of nonlinear partial differential equations, such as the nonlinear Schrodinger (NLS) equation and Burges’ equation, using some of the latest data-driven low-fidelity models, to improve the computational efficiency. As we know, in the framework of MFMC method, the high-fidelity models are selected according to the type and parameter characteristics of the actual problems, and the low-fidelity models are computationally cheap approximations of the underlying high-fidelity models. In recent years, several different types of low-fidelity models, for example, projection-based models [20,21,22], surrogate models [23], up-scaled models [24], have been applied to the MFMC framework.

Here, for the NLS equation, inspired by Ref. [19], the proper orthogonal decomposition (POD) method [22,25,26] coupled with discrete empirical interpolation method (DEIM) [27,28,29,30] is used to construct an optimal low fidelity model, which can deal with the complexity of higher order nonlinear terms of the NLS equation. Otherwise, for Burgers’ equation, we attempt to construct another kind of the low-fidelity model by using the operator reduction (OR) method [31,32,33], combined with DEIM for the parameterized Burgers’ equation. In particular, the quality of MFMC approach based on data-driven low-fidelity models is determined by its Pearson correlation coefficient relative to the high-fidelity model. Even if the accuracy of the data-driven low-fidelity models cannot be guaranteed, our MFMC estimator is unbiased for the output statistics of the high-fidelity models.

This paper is organized as follows. In Section 2, the MFMC algorithm and how to construct low-fidelity models are introduced. In Section 3, the MFMC method is used to solve the NLS equation with initial value uncertainty and the Burgers’ equation with parameter uncertainty, and the numerical results are compared with those calculated by the standard MC method. Conclusions are presented in Section 4.

2. Multi-Fidelity Monte Carlo Method

In this section, we first present the estimation models of MC method, and then describe the framework of MFMC. Then, we discuss how to construct appropriate and effective low-fidelity models of nonlinear partial differential equations.

2.1. Monte Carlo Method

Let $d\in N$ and define the input domain $D\in R^d$ and the output domain $Y\in R$. Suppose the function is $f:D\to Y$, that maps an input $z\in D$ to an output $y\in Y$. Let $\omega$ be the sample space and $Z:\omega \to D$ a random variable with range $D\subset R^d$. Independent and identically distributed realizations of $Z$ are denoted as $z_1,\dots ,z_m\in D$, where $m\in N$. We consider the $k$ models $f^{\left(1\right)},\dots ,f^{(k)}$ for $i=1,\dots ,k$. The high-fidelity model denoted as $f^{\left(1\right)}$ and lower-fidelity models are $f^{\left(2\right)},\dots ,f^{(k)}$ with $k\in N$. The variance $Var[f^{(i)}\left(Z\right)]$ of $f^{(i)}\left(Z\right)$ is denoted as:

$${\sigma }_i^2=Var[f^{(i)}\left(Z\right)],$$

(1)

The Monte Carlo method draws $m$ independent and identical distribution sample $z_1,\dots ,z_m\in D$ of $Z$ and the estimated expectation of MC estimation is:

$${\overline{y}}_m^{(i)}=\frac{1}{m}{\displaystyle \sum _{j=1}^m{f}^{(i)}}(z_j),$$

(2)

The MC estimator ${\overline{y}}_m^{(i)}$ is an unbiased estimator of $E\left[f^{\left(i\right)}\left(Z\right)\right]$. If the variance ${\sigma }_i^2\in R$, then the mean-square error (MSE) of the estimator ${\overline{y}}_m^{(i)}$ with respect to $E\left[f^{\left(i\right)}\left(Z\right)\right]$ is given by the estimator variance

$$e({\overline{y}}_m^{(i)})=E\left[{(E[f^{(i)}(Z)]-{\overline{y}}_m^{(i)})}^2\right]=\frac{{\sigma }_i^2}{m}$$

(3)

2.2. Multi-Fidelity Monte Carlo Method

2.2.1. Control Variables in Multi-Fidelity Estimation

The multi-fidelity method takes the random variable $f^{\left(2\right)}(Z),\dots ,f^{(k)}(Z)$ generated by the low-fidelity models $f^{\left(2\right)},\dots ,f^{(k)}$ as the control variable, to estimate the statistics of the random variable $f^{(1)}(Z)$ of the high-fidelity model $f^{\left(1\right)}$. Let the number of model calculations $m={[m_1,\dots ,m_k]}^T\in N^k$ be a vector with integer components $0<m_1\le \dots \le m_k$ and

$$z_1,\dots ,z_{m_k}\in D,$$

(4)

so let $m_k$ be independent and identically distributed realizations of the random variable $Z$. For $i=1,\dots ,k$, evaluate model $f^{\left(i\right)}$ at the $m_i$ realizations $z_1,\dots ,z_{m_i}$ of (4) to obtain

$$f^{(i)}(z_1),\dots ,f^{(i)}(z_{m_i}),$$

The component $m_i$ of $m$ is the number of calculations of model $f^{\left(i\right)}$ for $i=1,\dots ,k$. Derive the Monte Carlo estimate ${\overline{y}}_{m_i}^{(i)}$ as in (2) from the $m_i$ model evaluations $f^{(i)}(z_1),\dots ,f^{(i)}(z_{m_i})$ for $i=1,\dots ,k$. Additionally, the Monte Carlo estimate ${\overline{y}}_{m_{i-1}}^{(i)}$ computed from the $m_{i-1}$ model evaluations $f^{(i)}(z_1),\dots ,f^{(i)}(z_{m_{i-1}})$ for $i= 2,\dots ,k$. The high-fidelity model expectation estimation is $s=E[f^{(1)}(Z)]$. The MFMC estimate $\widehat{s}$ of $s$ is then

$$\widehat{s}={\overline{y}}_{m_1}^{(1)}+{\displaystyle \sum _{i=2}^k{\alpha }_i}({\overline{y}}_{m_i}^{(i)}-{\overline{y}}_{m_{i-1}}^{(i)}),$$

(5)

where ${\alpha }_2,\dots ,{\alpha }_k\in R$ are control variable coefficients and ${\overline{y}}_{m_i}^{(i)}$ is the Monte Carlo estimate using model $i$ and the first $m_i$ the number of calculations of model in the MFMC approach. The estimator (5) of MFMC is to correct the estimation of high-fidelity by using the sum of difference estimation of low-fidelity models. It can be seen from (5) that the estimation of MFMC method depends on number of model calculations $m\in N^k$ and coefficients ${\alpha }_2,\dots ,{\alpha }_k\in R$. In [8] (lemma3.1) show that the MFMC estimator is an unbiased estimator of $s$.

Consider the Monte Carlo estimates of ${\overline{y}}_{m_i}^{(l)}$ and ${\overline{y}}_{m_j}^{(t)}$ with $1\le i,j,l,t\le k$, the variance $Var[\widehat{s}]$ of the MFMC estimator $\widehat{s}$ is:

$$Var[\widehat{s}]=\frac{{\sigma }_1^2}{m_1}+{\displaystyle \sum _{i=2}^k(\frac{1}{m_{i-1}}}-\frac{1}{m_i})({\alpha }_i^2{\sigma }_i^2-2{\alpha }_i{\rho }_{1,i}{\sigma }_1{\sigma }_i),$$

(6)

where ${\rho }_{1,i}^2>0$ is the Pearson correlation coefficient of random variables $f^{(1)}(Z)$ and $f^{(i)}(Z)$ for $i=2,\dots ,k$. The Pearson correlation coefficient is ${\rho }_{ij}=\frac{Cov[f^{(i)}\left(Z\right),f^{(j)}\left(Z\right)]}{{\sigma }_i{\sigma }_j}$.

2.2.2. Multi-Fidelity Monte Carlo Framework

The MSE of MFMC method is estimated by the number of model calculations $m$ and the coefficients ${\alpha }_2,\dots ,{\alpha }_k$, and the cost $p \in R_{+}$ of estimation $\widehat{s}$ is equal to the calculated budget. Since $e\left(\widehat{s}\right)=Var\left[\widehat{s}\right]$, it is sufficient to minimize the variance $Var\left[\widehat{s}\right]$ of the MFMC estimator. Therefore, the objective of the MFMC method is to minimize the variance of multi-fidelity estimation $\widehat{s}$ under a given computational budget $p^{\ast }\in R_{+}$, which can be achieved by solving of the optimization problem:

$$\begin{gathered} \underset{m\in R^k,{\alpha }_{2,}\dots ,{\alpha }_k\in R}{\arg \min }Var[\widehat{s}] \\ \mathrm{s}.\mathrm{t}{m}_{i-1}-m_i\le 0 \\ -m_1\le 0 \\ {w}^{T}m=p \end{gathered}$$

(7)

where the control variable coefficients ${\alpha }_2,\dots ,{\alpha }_k\in R$ and the number of model calculations $m_1,\dots ,m_k$ are variables The inequality constraints $m_{i-1}-m_i\le 0$ for $i=2,\dots ,k$ and $-m_1\le 0$ ensure $0<m_1\le m_2\le ,\dots ,\le m_k$. Note that we will show $m_1^{\ast }>0$ for the optimal $m_1^{\ast }$ and therefore $m^{\ast }\in R_{+}^k$. The equality constraint $w^{T}m=p$ guarantees that the cost of the corresponding MFMC estimator is equal to the computation budget $p \in R_{+}$. Peherstorf et al. [8] proved that the optimization problem in Equation (7) has unique approximate optimal solution under certain conditions of the high-fidelity and low-fidelity models. Let the $k$ models $f^{\left(1\right)},\dots ,f^{(k)}$ satisfy $\left|{\rho }_{1,1}\right|>\dots >\left|{\rho }_{1,k}\right|$ and the costs $w_1,\dots ,w_k$ satisfy the ratios $\frac{w_{i-1}}{w_i}>\frac{{\rho }_{1,i-1}^2-{\rho }_{1,i}^2}{{\rho }_{1,i}^2-{\rho }_{1,i+1}^2}$, for $i=2,\dots ,k$. The optimal control variable coefficients are:

$${{\alpha }^{\ast }}_i=\frac{{\rho }_{1,i}{\sigma }_1}{{\sigma }_i},i=2,\dots ,k$$

(8)

The optimal numbers of model calculations are:

$${m_1}^{\ast }=\frac{p}{w^T{r}^{\ast }},{m_i}^{\ast }={m_1}^{\ast }{r_{\mathrm{i}}}^{\ast }$$

(9)

where $r^{\ast }={[{r_1}^{\ast },\dots ,{r_k}^{\ast }]}^T\in R_{+}^k$, ${r_i}^{\ast }=\sqrt{\frac{w_1({\rho }_{1,i}^2-{\rho }_{1,i+1}^2)}{w_i(1-{\rho }_{1,2}^2)}}$, and $p\in R_{+}$ is the calculation budget. The global minimum solution of Equation (7) is $\left(m^{\ast },{\alpha }_2^{\ast },\dots ,{\alpha }_k^{\ast }\right)$. The MFMC framework is shown in Figure 1. Using the number of model calculations $m^{\ast }$ and the coefficients ${\alpha }_2^{\ast },\dots ,{\alpha }_k^{\ast }$, the MSE $e({\widehat{s}}^{\ast })$ of the multi-fidelity estimation ${\widehat{s}}^{\ast }$ is

$$e({\widehat{s}}^{\ast })=\frac{{\sigma }_1^2}{p}{\left({\displaystyle \sum _{i=1}^k\sqrt{w_i({\rho }_{1,i}^2-{\rho }_{1,i+1}^2)}}\right)}^2$$

(10)

Figure 1. The MFMC framework.

Since the cost of MC estimate ${\overline{y}}_m^{(1)}$ is $p=m{w}_1$, $m$ is the numbers of model calculations and $w_1$ is the input calculation cost. The MSE of the MC estimator ${\overline{y}}_m^{(1)}$ is rewritten as:

$$e({\overline{y}}_m^{(1)})=\frac{{\sigma }_1^2}{m}=\frac{{\sigma }_1^2}{p}{\omega }_1$$

(11)

If we compare the MSE of the MFMC with the MSE of the standard MC estimator, it holds that $e({\widehat{s}}^{\ast })<e({\overline{y}}_m^{(1)})$, if and only if

$${\displaystyle \sum _{i=1}^k\sqrt{\frac{w_i}{w_1}({\rho }_{1,i}^2-{\rho }_{1,i+1}^2)}}<1$$

(12)

The low-fidelity models lead to a MFMC estimator with a smaller MSE than the MSE of the standard MC estimator with the same budget $p$.

2.3. Data-Driven Low-Fidelity Models of Nonlinear Partial Differential Equations

The uncertainty analysis framework of MFMC estimation needs to combine the output information of high-fidelity model and low-fidelity models to accelerate the estimation of statistics. In this paper, based on the data-driven low-fidelity models, the nonlinear partial differential equation is converted into ODEs system according to the type and parameter characteristics of the actual problem equations. Then, the reduced dimension model and DEIM are coupled to construct effective nonlinear low-fidelity models in ODEs system.

First, the nonlinear partial differential equations are transformed into ordinary differential equations in numerical simulation of partial differential equations. We can use the finite difference discretization or Fourier transformation to obtain the following nonlinear ordinary differential equations:

$$\frac{d}{dt}y(t)=Ay(t)+F(y(t))+B(u(t)),$$

(13)

where $t\in \left[0,T\right]$ is time, $y(t)={\left[y_1(t),\dots ,y_N(t)\right]}^T\in R^N$ is the state vector or solution of the initial condition $y(0)=y_0\in R^N$. The operator $A\in R^{N\times N}$ is the approximate matrix of linear space differential operator. The matrix $B\in R^{N\times p}$ is the input matrix, and $F$ is the nonlinear function calculated on the $y(t)$ component with $F={\left[F_1(y_1(t)),\dots ,F_N(y_N(t))\right]}^T$.

2.3.1. POD Coupled with DEIM

Consider a set of snapshots $\{y(t_1),\dots ,y(t_M)\}\in R^N$ and the corresponding snapshot matrix $Y=[y(t_1),\dots ,y(t_M)]\in R^{N\times M}$. Let $V=[v_1,\dots ,v_n]\in R^{N\times n}$ be the POD basis in $n$ dimensional space. The POD basis vector $V\in R^{N\times n}$ of the matrix $V$ is the $n$ largest left singular vectors of the matrix $Y$. The POD reduced-order system is obtained by Galerkin projection [22,26]:

$$\frac{d}{dt}\tilde{y}(t)=\tilde{A}\tilde{y}(t)+V^{T}F(V\tilde{y}(t))+\tilde{B}(u(t)),$$

(14)

where the reduced operator is $\tilde{A}=V_n^{T}A{V}_n\in R^{n\times n}$, the reduced input operator is $\tilde{B}=V_n^{T}B\in R^{n\times p}$ and state vector is $y(t)=V\tilde{y}(t)$.

We use POD method to construct low-fidelity models. Although the dimension of solving complex systems can be reduced, the complexity of higher order nonlinear terms may be as challenging as the original equation. DEIM [30,33] provides an effective calculation method for evaluating nonlinear terms.

Select $m\in N$ vectors as columns stored in DEIM as base vector $U\in R^{N\times m}$. The nonlinear terms are approximately $F(y(t))=Uc(t)$, where $U=[t_1,\dots ,t_{\mathrm{m}}]$ and $c(t)$ are the corresponding coefficient vectors. Then DEIM selects $m$ pairs of different interpolation points $p_1,\dots ,p_m\in \{1,\dots ,N\}$ and assumes that DEIM interpolation point matrix $P=[e_{p_1},\dots ,e_{p_m}]\in R^{N\times m}$, where $e_i\in {\{0,1\}}^N$ is the $i$-th standard unit vector. The coefficient vector $c(t)$ can be uniquely determined by $P^{T}F(y(t))=P^{T}Uc(t)$. The DEIM interpolation points of nonlinear function $F$ are composed of tuples $(U,P)$ defined by DEIM basis $U$ and DEIM interpolation point matrix $P$. The DEIM approximate of nonlinear function $F$ is estimated at the state vector $y(t)$ as:

$$F(y(t))\approx U{(P^{T}U)}^{-1}{P}^{T}F(y(t)),$$

(15)

By coupling POD (14) and DEIM (15), the POD-DEIM low-fidelity models are:

$$\tilde{A}\tilde{y}(t)+V^{T}F(U{(P^{T}U)}^{-1}{P}^{T}F(y(t)))+\tilde{B}(u(t))=0.$$

(16)

2.3.2. Operator Reduction

Another strategy of constructing the low-fidelity models is operator reduction [22,31]. We first construct a reduced space from trajectories of the full-model states, and the original operator is replaced by the reduced operator to construct the low-fidelity models of nonlinear partial differential equations.

Let $V_n\in R^{N\times n}$ be a matrix composed of the first $n\in N$ POD basis vectors. The projection state of the full-model state $y(t)\in R^N$ is $\widehat{y}(t)=V_n^{T}y(t)$. Let $Y(t)=[y_1(t),\dots ,{y_K(t)]}^T\in R^{K\times N}$ be the trajectory and the input matrix be $U(t)=[u(t),\dots ,{u_K(t)]}^T\in R^{K\times p}$. Consider the time step $t_1,\dots ,t_K$ with time step $\delta t$. For $j=1,\dots ,K$, let ${\dot{\widehat{y}}}_j\in R^n$ be the approximation of the derivative $\frac{d}{dt}\widehat{y}(t_j)$ of the projection state $\widehat{y}(t_j)$ at time $t_j$ that converges to $\frac{d}{dt}\widehat{y}(t_j)$ under $L^2$ norm when $\delta t\to 0$. Suppose approximates the operators $A\in R^{N\times N}$, $B\in R^{N\times p}$ and $F\in R^{N\times s}$ by the reduced operators $\widehat{A}\in R^{n\times n}$, $\widehat{B}\in R^{n\times p}$ and $\widehat{F}\in {\mathrm{R}}^{n\times s}$. The reduction operators satisfy the equation:

$${\dot{\widehat{y}}}_j=\widehat{A}{\widehat{y}}_j+\widehat{F}{\widehat{y}}_j+\widehat{B}{u}_j,j=1,\dots ,K,$$

(17)

We define the reduced operators $\widehat{A}\in R^{n\times n}$, $\widehat{B}\in R^{n\times p}$ and $\widehat{F}\in {\mathrm{R}}^{n\times s}$ as the solution of the optimization problem.

$$\underset{\widehat{A}\in R^{n\times n},\widehat{B}\in R^{n\times p},\widehat{F}\in R^{n\times s}}{minimize}\underset{j=1}{\overset{K}{\Sigma }}{\Vert {\dot{\widehat{y}}}_j-\widehat{A}{y}_j-\widehat{F}{\widehat{y}}_j-\widehat{B}{u}_j\Vert }_2^2.$$

(18)

Let $D=[\widehat{Y},U,{\widehat{Y}}^{(1)},\dots ,{\widehat{Y}}^{(n)}]\in R^{K\times (n+p+s)}$ be the data matrix, where $\widehat{Y}$ is the projection trajectory, $U$ is the input matrix, we define the matrix by using the finite difference scheme ${\dot{\widehat{y}}}_j=\frac{{\widehat{y}}_j-{\widehat{y}}_{j-1}}{\delta t},j=1,\dots ,K,$

$$R={[{\dot{\widehat{y}}}_1,\dots ,{\dot{\widehat{y}}}_K]}^T\in R^{K\times n},$$

(19)

The final optimization problem (18) is transformed into the least-square problem with the data matrix $D$ and the matrix $O$:

$$\underset{O\in R^{n\times (n+p+s)}}{minimize}\Vert D{O}^T{-R\Vert }_F^2,$$

(20)

The solution is $O=[\widehat{A},\widehat{B},\widehat{F}]\in R^{n\times (n+p+s)}$.

3. Numerical Results

In this section, we give two numerical examples of nonlinear partial differential equations, using MFMC with data-driven low-fidelity models and the initial value uncertainty of NLS equation and the parameter uncertainty of Burgers’ equation are studied, respectively.

3.1. Uncertainty Analysis of NLS Equation with Initial Value

3.1.1. Estimate Analysis

NLS equation is a basic equation in quantum mechanics [33,34]. It can be used to describe the propagation phenomena and principles of optical solitons in fibers, the propagation of optical pulses in dispersion and nonlinear media and the motion of superconducting electrons in electromagnetic fields. We consider the NLS nonlinear partial differential equation:

$$\{\begin{array}{l}i\frac{\partial u}{\partial t}+\frac{1}{2}\frac{{\partial }^{2}u}{\partial x^2}+u{\left|u\right|}^2=0\\ u(x,0)=u_0(x)\end{array}$$

(21)

where $u$ is a complex number function. Let $\mathsf{\Omega }=(-20,20)$ be the spatial domain and the time be $t\in [0,2\pi ]\subset R$. We assume that the initial value is an uncertain input variable. In the NLS equation, the initial condition satisfies $u_0(x)=N^{\ast }sech(x)+Z$, $N=2$ is the double soliton order, $x\in \mathsf{\Omega }$ is the propagation distance and $Z$ obeys the normal distribution for the uncertainty analysis. Rewriting (21) by Fourier transformation, gives the set of differential equations

$$\frac{\partial \widehat{u}}{\partial t}=-\frac{i}{2}{k}^2\widehat{u}+i\widehat{{\left|u\right|}^{2}u}$$

(22)

where $-\frac{i}{2}{k}^2\widehat{u}$ is linear term and $i\widehat{{\left|u\right|}^{2}u}$ is nonlinear term.

To investigate the performance of our multi-fidelity estimation method based on data-driven low-fidelity models. We use the fourth- and fifth-order Runge–Kutta method to solve Equation (22), and the numerical solutions of NLS Equation are regarded as the high-fidelity model $f^{(1)}$. The data-driven low-fidelity model of NLS equation is used to reduce the dimension for the numerical solutions by POD method and select three POD basis functions. Then, DEIM is used to reduce the dimension of higher order nonlinear terms and we obtain a type of the low-fidelity model $f^{(2)}$. Table 1 shows the information for calculating these models with 1000 samples.

Table 1. Number of samples per model, coefficients and costs for NLS equation.

Models	$\mathit{\alpha }$	$\mathit{\omega }$	$\mathit{m}$
High-fidelity (FD) $f^{(1)}$	1	480	204
Low-fidelity (POD + DEIM) $f^{(2)}$	0.2082	3	698

We show the Pearson correlation coefficients and variances of the high- and low-fidelity models for NLS equation in Table 2. The two statistics are estimated using 100 evaluations of the high-fidelity and low-fidelity model. We can see that the low-fidelity model is highly correlated with a correlation coefficient of 0.9989. Note that the correlation is high enough and the low-fidelity model is cheap enough for the multi-fidelity estimator to be worthwhile.

Table 2. The Pearson correlation coefficient and variances of high- and low-fidelity models for NLS equation.

Models	${\mathit{\rho }}_{\mathit{i}}$	${\mathit{\sigma }}_{\mathit{i}}^{\mathbf{2}}$
High-fidelity (FD) $f^{(1)}$	1	214.1663
Low-fidelity (POD + DEIM) $f^{(2)}$	0.9989	214.3100

We are interested in the expectation $E[f(Z)]$ of the NLS equation if the initial value is uncertain. We choose the sample output function is $f(Z)={\displaystyle {\int }_{\mathsf{\Omega }}\Vert u(x,T;u_0(Z))\Vert dx}$ and the time is $T=\pi$. The Monte Carlo estimate ${\overline{y}}_m^{(1)}$ of $E[f^{(1)}(Z)]$ is computed from $n=1\times {10}^3$ realizations and is used to estimate the MSE. We use the high-fidelity and the low-fidelity models listed in Table 1 to estimates the mean, variance and sensitivity index of output function for the NLS equation.

We compare the MC and MFMC expectation estimators in terms of their MSE that is estimated from 20 replicates as

$$\widehat{e}=\frac{1}{20}{\displaystyle \sum _{i=1}^{20}(E[f^{(1)}(Z)]}-{\widehat{s}}_i{)}^2$$

(23)

where $E[f^{(1)}(Z)]$ is the reference expectation estimator and ${\widehat{s}}_i$ is either an MFMC estimator (10) or an MC estimator (11). The reference expectation $E[f^{(1)}(Z)]$ is computed from $n=1\times {10}^3$ samples and we obtain $E[f^{(1)}(Z)]=450.1967$.

Figure 2 reports the estimated MSE (23) of the standard MC estimators and the MFMC estimators of budgets from $p={10}^3$ to ${10}^5$. Compared with the standard MC method, the MSE of our MFMC method is smaller. That is, the MFMC method can be significantly better than the standard MC. The multi-fidelity method based on data-driven low-fidelity model has the same convergence speed as the MC method. Under the same budget ${10}^5$, MSE estimated by the MFMC method is about 30 times lower than the MC method. For the same convergence rate, this means that the speed of the MC estimation is increased by 30 times. In other words, the MSE estimation of MFMC is 0.1 when the calculation budget is 300, but the budget of MC is 9000 with the same value of MSE.

Figure 2. The budgets of estimated MSE for the standard MC and MFMC estimators range from $p={10}^3$ to ${10}^5$.

To further study the stability of the MFMC method, Figure 3a shows that the MSE of the standard MC and MFMC estimator changes with runtime when the numbers of samples are 100, 500 and 1000. We can see, with the increase of running time, that MSE decreases under different sample numbers, indicating that MFMC is stable in the time evolution direction. On the other hand, the MSE of the MFMC is estimated by the number of model evaluations $m$ and the coefficients ${\alpha }_2,\dots ,{\alpha }_k$, while the number of model evaluations $m$ and coefficients $\alpha$ are determined by the sample correlation coefficients and sample variances. Table 3 give the sample correlation coefficients computed from 100, 500 and 1000 samples. When the number of samples increases, the sample correlation coefficients change only slightly. Table 4 list the sample variance calculated by the same samples as before. Since calculating coefficients $\alpha$, only the sample variance ratios ${\sigma }_1/{\sigma }_i,i=1,2$ affect the value of coefficients. When increasing the samples 100 to 500, the ratios ${\sigma }_1/{\sigma }_i$ also change very little. Hence, the disturbance of sample variances and sample correlation coefficients has minimal effect on MSE estimated by MFMC method. Figure 3b reports the proportion plot of the number of model calculations $m$ for MFMC estimator when the sample numbers are 100, 500 and 1000. As the number of samples increases, the change of percentage is small, and it is confirmed that small perturbations in the sample estimates lead to small changes in the number of model evaluations $m$.

Figure 3. (a) Comparison the estimated MSE of our MFMC estimators from 100, 500 and 1000 samples, respectively; (b) Proportion plot of the number of model calculations $m$ for MFMC estimator when the sample numbers are 100, 500 and 1000.

Table 3. Comparison of correlation coefficient using 100, 500 and 1000 samples, respectively.

Models	100 Samples	500 Samples	1000 Samples
High-fidelity (FD) $f^{(1)}$	1.0000	1.0000	1.0000
Low-fidelity (POD + DEIM) $f^{(2)}$	0.1904	0.1968	0.2082

Table 4. Comparison of sample variances by 100, 500 and 1000 samples, respectively.

Models	100 Samples	500 Samples	1000 Samples
High-fidelity (FD) $f^{(1)}$	212.3323	218.7528	232.9878
Low-fidelity (POD + DEIM) $f^{(2)}$	317.5544	336.1835	337.2713
Ratios $\left({\sigma }_1/{\sigma }_2\right)/\%$	0.6686	0.6507	0.6908

3.1.2. Sensitivity Analysis

In uncertainty quantification analysis, another concern is the sensitivity for initial value of random input to output variation. We use the Sobol index [35,36,37] to analyze the contribution of each input variable to the output variance. Figure 4 displays the main and total sensitivities box plots of the estimators repeated 100 times under different computational budgets. We note that the MFMC estimator can accurately predict the contribution of each input variable compared with the standard MC estimator. When the low-fidelity model is used, the sensitivity index of the estimator decreases, and the variance of the estimator is also reduced.

Figure 4. (a,b) Comparison of the MFMC and standard MC estimates of main or total effect sensitivities for the NLS Equation (18) with computational budgets $p=100$. (c,d) Comparison of the MFMC and standard MC estimates of main or total effect sensitivities for the NLS Equation (21) with computational budgets $p=1000$.

3.2. Parameter Uncertainty Analysis of Burgers’ Equation

3.2.1. Estimate Analysis

Let $\mathsf{\Omega }=[0,1]\subset R$ be the spatial domain and the time be $t\in [0,1]\subset R$. We consider the Burgers’ partial differential equation [25,38]:

$$\{\begin{array}{l}\frac{\partial x}{\partial t}+x\frac{\partial x}{\partial \omega }-\mu \frac{{\partial }^{2}x}{\partial {\omega }^2}=0\\ x_0(\mu )=0\end{array},$$

(24)

where $x$ is the state at spatial coordinate $\omega \in \mathsf{\Omega }$, and $\mu$ is the input uncertainty parameter, which obeys the uniform distribution in the domain $[0.1,1]$ for uncertainty analysis. The boundary conditions are $x(0,t;\mu )=1$ and $x(1,t;\mu )=-1$ for $t:[0,1]\subset R$. We use the finite difference method to discretize the Burgers’ Equation (24) in an iso-spacing grid with 129 points of the spatial domain $\mathsf{\Omega }$. We obtain parameterized nonlinear ordinary differential equations:

$$\frac{d}{dt}y(t;\mu )=A(\mu )y(t;\mu )+F(\mu )y(t:\mu )+B(\mu )u(t)$$

(25)

where $\mathbf{A}(\mu )\in R^{N\times N}$ is the operator of Burgers’ equation linear term, $F(\mu )\in {\mathrm{R}}^{N\times N(N+1)/2}$ is the operator corresponding to the nonlinear term, and $B(\mu )\in R^{N\times P}$ is the operator of boundary condition.

To develop a multi-fidelity estimator based on the data-driven low-fidelity model of nonlinear partial differential equations. The numerical solutions of Equation (24) are used as a high-fidelity model $f^{(1)}$. Then, the data-driven low-fidelity models are obtained by using operator reduction coupled with DEIM. Two types of low-fidelity models are constructed, respectively. The values of random parameter space for one of the low-fidelity model $f^{(2)}$ are calculated by DEIM of high fidelity models. We construct other reduction model from 129 to 20 dimensions by using the operator reduction of the full-model. The value of random parameter space is calculated by DEIM for reduction model, and we obtain the low-fidelity model $f^{(3)}$. The information of each multi-fidelity model, calculated using 20,000 samples, is reported in Table 4.

In the uncertainty quantification analysis of nonlinear partial differential equation, we are interested in the expected $E[f(\mu )]$ of random parameter. The sample output function is $f(\mu )={\displaystyle {\int }_0^{1}x(w,T;\mu )dw},$ and the time is $T=0.5$. The MC estimate ${\overline{y}}_m^{(1)}$ of $E[f^{(1)}(\mu )]$ is computed from $n=2\times {10}^4$ and is used to estimate the MSE. The MC method is compared with the MFMC estimator under the given calculation budgets. It is hoped that the optimal MFMC method can obtain satisfactory estimation. We use the high-fidelity and the low-fidelity models as shown in Table 5 to estimates the mean, variance and sensitivity of output function for the Burgers’ equation, where the low-fidelity models use the operator reduction method and the DEIM for the parameterization Burgers’ equation.

Table 5. Number of samples per model, coefficients and costs for Burgers’ equation.

Models	$\mathit{\alpha }$	$\mathit{\omega }$	$\mathit{m}$
High-fidelity (FD) $f^{(1)}$	1	385	2597
Low-fidelity (DEIM) $f^{(2)}$	0.0233	0.001	8475
Low-fidelity (OR + DEIM) $f^{(3)}$	0.0221	$1\times {10}^{-4}$	11,784

$\alpha$$\omega$$m$ High-fidelity (FD) $f^{(1)}$$\mathrm{Low}-\mathrm{fidelity} \left(\mathrm{DEIM}\right) f^{(2)}$$\mathrm{Low}-\mathrm{fidelity} (\mathrm{OR}+\mathrm{DEIM}) f^{(3)}$$1\times {10}^{-4}$ The high-fidelity model $f^{(1)}$, low-fidelity models $f^{(2)}$ and $f^{(3)}$ are divided into three groups for comparative study. Figure 5 displays the estimated MSE (23) of those three models, respectively. The MSE of the MFMC estimator achieves a speedup of up to two orders of magnitude compared to the standard MC estimator. In the three numerical experiments, when the cost is ${10}^6$, the MSEs estimated by the multi-fidelity models are $1.3844\times {10}^{-5}$, $1.3648\times {10}^{-5}$ and $3.4297\times {10}^{-6}$ in sequence. Therefore, the MSE of the models are further decreased when the calculation budget for output function expectation is increased.

Figure 5. The estimated MSE of the standard MC estimators and the MFMC estimators of budgets from to ${10}^6$. (a) MSE estimation of $f^{(1)}$, $f^{(2)}$. (b) MSE estimation of $f^{(1)}$, $f^{(3)}$. (c) MSE estimation of $f^{(1)}$, $f^{(2)}$, $f^{(3)}$.

Compared with the standard MC method, under the same computational budget, the MFMC can be superior to the MC for each statistical quantity. Figure 6 shows the relative share of each model in the total number of samples. The shares of the models vary by orders of magnitude between the high-fidelity and low-fidelity models, reflecting their correlations and costs. It should be noted that the relative share of the model is not related to the calculation budget $p$, because all components of $m$ are linearly proportional to $p$.

Figure 6. The total number of samples distribution in high-fidelity and low-fidelity models.

3.2.2. Sensitivity Analysis

To further analyze the parameter uncertainty law of Burgers’ equation, the sensitivity of the parameter is studied. Figure 7 reports the standard MC and MFMC estimator, respectively, in $p={10}^2$ and $p={10}^4$ computational budget, repeated $100$ times estimates of the main sensitivity and total sensitivity box plots. It is worth noting that compared with the standard MC estimation, the sensitivity index of the MFMC method estimator decreases and the estimated variance is also reduced. Therefore, in all cases, compared with the standard MC estimation, the multi-fidelity estimation has smaller variance with the budget increases. When the true value of the sensitivity index is closer to zero, the variance of the estimator is similar as the order of sensitivity magnitudes.

Figure 7. (a,b) Comparison of the MFMC and standard MC estimates of main or total effect sensitivities for the Burgers’ Equation (24) with computational budgets $p=100$. (c,d) Comparison of the MFMC and standard MC estimates of main or total effect sensitivities for the Burgers’ Equation (24) with computational budgets $p=10,000$.

4. Conclusions

We developed the MFMC based on the data-driven low-fidelity models of nonlinear partial differential equations in order to solve the problem of high computational cost in the uncertainty quantification of numerical solutions. The main strategy is that the reduced dimension model and DEIM are coupled to construct effective nonlinear low-fidelity models in ODEs system. Our method combined the output information of high-fidelity models and low-fidelity models to speed up the estimation of statistics, so that the calculation cost is lower and the MSE is smaller.

The presented MFMC approach is used to calculate the statistical information, including mean, variance and sensitivity for the NLS equation with the initial value uncertainty and Burgers′ equation with parameter uncertainty. The results shows that the MFMC estimation with the data-driven low-fidelity models can effectively reduce the MSE and predict the output statistics well under the condition of multiple numbers of sample points for nonlinear partial differential equations. It provides a good solution for solving the uncertainty quantification problem of multivariate in physics, mechanics and engineering technologies. However, it should be pointed out that the key point of MFMC method is the necessity of having suitable low-fidelity models for our high-fidelity models. For an unknown complex system, how to construct an appropriate low-fidelity model needs further research and discussion.