Gaussian Process Based Expected Information Gain Computation for Bayesian Optimal Design

Abstract

Optimal experimental design (OED) is of great significance in efficient Bayesian inversion. A popular choice of OED methods is based on maximizing the expected information gain (EIG), where expensive likelihood functions are typically involved. To reduce the computational cost, in this work, a novel double-loop Bayesian Monte Carlo (DLBMC) method is developed to efficiently compute the EIG, and a Bayesian optimization (BO) strategy is proposed to obtain its maximizer only using a small number of samples. For Bayesian Monte Carlo posed on uniform and normal distributions, our analysis provides explicit expressions for the mean estimates and the bounds of their variances. The accuracy and the efficiency of our DLBMC and BO based optimal design are validated and demonstrated with numerical experiments.

1. Introduction

As acquiring data in experiments is generally computationally demanding and time-consuming, maximizing the informativeness of experimental data is of crucial importance. For example, in the area of climate science, a complex system with various stochastic inputs is integrated to represent the real climate situation. Usually, the locations and the time of putting sensors to collect climate observations are optional. With limited resources, careful selections of sensor placements are required (see Reference [1] for a detailed discussion). Many works focus on finding experimental data carrying more information, and this topic is usually referred to as optimal experimental design (OED) [2]. In this paper, we consider the OED problem in the context of the Bayesian inverse problem. Given a forward problem, the inverse problem is to infer parameters inherent of the forward model through a set of design points and the corresponding responses. In the Bayesian setting, the parameters of interest are viewed as random variables, and hence the posterior distribution of the parameters can be obtained via the Bayes’ rules [3,4,5,6]. In linear cases, the OED problem includes various criteria such as the D-optimal design criterion and the A-optimal design criterion. The D-optimal design criterion seeks to maximize the determinant of the information matrix of the design, whereas the A-optimal design criterion considers minimizing the trace of the inverse of the information matrix which results in minimizing the average variance [7,8,9].

We focus on the decision theoretic approach that considers maximizing the expectation of Kullback-Leibler (KL) divergence from the posterior distribution to the prior distribution [10]. This decision theoretic approach is a nonlinear generalization of the Bayesian D-optimal criterion [11]. The objective function we want to maximize, the expectation of KL divergence, is also referred to as the expected information gain (EIG) over parameters. Computing EIG is usually a challenging problem, since it does not have an analytical form for nonlinear problems in general. In the literature, the following attempts have been made for this problem. Efficient surrogates for the forward models are introduced to make the problem tractable [1,12,13,14,15]. Recently, as the rise of novel computational methods, new approaches are actively developed to evaluate the EIG, including the quasi-Monte Carlo method [16] and the layered importance sampling [17] in the context of the focused optimal design. As the goal of this kind of optimal design is to find the maximizer of EIG, it is crucial to apply efficient optimization strategies. In Reference [12], a curve fitting surrogate of Monte Carlo experiments is proposed to result in an efficient optimization scheme. For the Bayesian optimal design problem focusing on the risk from an optimal terminal decision, the Bayesian optimization (BO) approach is proposed to find the minimizer of the risk [9]. We note that, while BO is proposed for the Bayesian optimal design problem for minimizing the risk in Reference [9], BO is considered for maximizing the EIG in this work. In addition, efficient approximate coordinate exchange strategies are proposed for Bayesian design in Reference [18]. For intractable likelihood models, Gaussian process (GP) models are built to emulate the likelihood function in Reference [19]. Gradient-based optimization methods are proposed to compute the maximizer of EIG in References [13,14]. Review of modern computational methods for decision theoretic optimal experimental design is provided in Reference [20].

The main purpose of this work is to propose an efficient Gaussian process (GP) based Bayesian optimal design strategy, where Bayesian Monte Carlo (BMC) and Bayesian optimization (BO) which are both based on GP are used [21,22,23]. As the Monte Carlo simulation for EIG involves an inner layer simulation and an outer layer simulation (see Reference [14]), we develop a novel efficient double-loop Bayesian Monte Carlo (DLBMC) method, which employs BMC [24,25,26] for both layers. However, the EIG is generally computationally expensive and its gradient information is typically not given explicitly. We propose a BO method [27,28,29] to compute the maximizer of EIG, where the gradient information of EIG is not required. To summarize, the contributions of this work are three-fold: first we develop a novel DLBMC to efficiently compute EIG; second we analyze the BMC for the normal and the uniform distributions; third we propose BO to obtain the maximizer of EIG.

This paper is organized as follows. In Section 2, we review the Bayesian optimal experimental design problem and formulate the expected information gain (EIG) criterion. In Section 3, we derive a double-loop Bayesian Monte Carlo estimator for the EIG and propose a Bayesian optimization approach to obtain the maximizer of the approximated EIG. Detailed analysis of BMC for the normal and the uniform distributions are conducted in this section. In Section 4, we demonstrate the efficiency of our GP based Bayesian optimal design with three test problems. Section 5 concludes the paper and provides discussions of the advantages and disadvantages of our algorithm.

2. Formulation of Experimental Design

In this section, we review the setting of the Bayesian optimal design problem following the presentation in [14]. In the Bayesian setting, the unknown parameters are viewed as random variables. Let $(\Omega ,\mathcal{F},\mathcal{P})$ be a probability space, where $\Omega$ is a sample space, $\mathcal{F}$ is a $\sigma$-field, and $\mathcal{P}$ is the probability measure on $(\Omega ,\mathcal{F}).$ Let $\mathbf{\theta }:\Omega \to {\mathbb{R}}^{n_{\theta }}$ denote the parameters of interest, where $n_{\theta }$ is the dimension of the unknown parameters. Assume that $\mathbf{\theta }$ is associated with a prior measure $\mu$ on ${\mathbb{R}}^{n_{\theta }}$ satisfying $\mu (A)=\mathcal{P}({\mathbf{\theta }}^{-1}(A))$ for $A\in {\mathbb{R}}^{n_{\theta }}$. Throughout this paper, we assume that all the random variables have densities with respect to the Lebesgue measure. Let $\mathit{d}\in \mathcal{D}\subset {\mathbb{R}}^{n_d}$ denotes the design variable, where $n_d$ is the number of design variables and $\mathcal{D}$ denotes the design space. Let $\mathit{y}\in {\mathbb{R}}^{n_y}$ denote the response associated with $\mathit{d}$ where $n_y$ is the dimension of response. The inference of $\mathbf{\theta }$ can be obtained based on the prior distribution and observations via Bayes’ rule,

$$\underset{\mathrm{Posterior}}{\underbrace{p(\mathbf{\theta }|\mathit{d},\mathit{y})}}=\frac{\underset{\overbrace{p(\mathit{y}|\mathbf{\theta },\mathit{d})}}{\mathrm{Likelihood}}\underset{\overbrace{p(\mathbf{\theta }|\mathit{d})}}{\mathrm{Prior}}}{\underset{\mathrm{Evidence}}{\underbrace{p(\mathit{y}|\mathit{d})}}}.$$

(1)

The likelihood function is often determined by a deterministic forward model and a statistical model for measurement of model noises. Here we model the relation of the design variable and the observation by a deterministic model $\mathit{G}(\mathbf{\theta },\mathit{d})$ and additive Gaussian noises $\mathbf{ϵ}$,

$$\mathit{y}=\mathit{G}(\mathbf{\theta },\mathit{d})+\mathbf{ϵ},\mathbf{ϵ}\sim \mathcal{N}(\mathbf{0},{\sigma }^2\mathit{I}),$$

(2)

where $\mathit{G}$ is the forward model. In many practical problems, the forward model is computationally expensive, and its explicit form is not given. So we can just view it as a black box whose internal structure is unknown, whereas we can generate noisy responses given fixed design variables and parameters.

Following the decision theoretic approach [10], we set the utility function as the KL divergence from the posterior distribution to the prior distribution,

$$u(\mathit{d},\mathit{y},\mathbf{\theta })=D_{\mathrm{KL}}(p(\mathbf{\theta }|\mathit{d},\mathit{y}\parallel p(\mathbf{\theta })))=u(\mathit{d},\mathit{y}).$$

(3)

This term is actually independent of $\mathbf{\theta }$. Noting that $u(\mathit{d},\mathit{y})$ is a function of both $\mathit{d}$ and $\mathit{y}$, therefore we further take expectation of u over $\mathit{y}$ to define the expected information gain (EIG):

$$\begin{array}{c}\hfill U(\mathit{d})={\int }_{\mathcal{Y}}u(\mathit{d},\mathit{y})p(\mathit{y}|\mathit{d})\mathrm{d}\mathit{y}={\int }_{\mathcal{Y}}{\int }_{\Theta }p(\mathbf{\theta }|\mathit{d},\mathit{y})\log \left[\frac{p(\mathbf{\theta }|\mathit{d},\mathit{y})}{p(\mathbf{\theta })}\right]\mathrm{d}\mathbf{\theta }p(\mathit{y}|\mathit{d})\mathrm{d}\mathit{y}.\end{array}$$

Then the optimal experimental design is to find a design point which maximizes the expected utility, that is,

$${\mathit{d}}^{\star }=\arg \underset{\mathit{d}\in \mathcal{D}}{\max }U(\mathit{d}).$$

(4)

A double-loop Monte Carlo (DLMC) estimator of EIG is proposed in [30]. Rewrite $U(\mathit{d})$ as

$$\begin{array}{cc}\hfill U(\mathit{d})& ={\int }_{\mathcal{Y}}{\int }_{\Theta }p(\mathbf{\theta }|\mathit{d},\mathit{y})\log \left[\frac{p(\mathbf{\theta }|\mathit{d},\mathit{y})}{p(\mathbf{\theta })}\right]\mathrm{d}\mathbf{\theta }p(\mathit{y}|\mathit{d})\mathrm{d}\mathit{y}\hfill \\ \hfill & ={\int }_{\mathcal{Y}}{\int }_{\Theta }\{\log \left[p(\mathit{y}|\mathbf{\theta },\mathit{d})\right]-\log \left[p(\mathit{y}|\mathit{d})\right]\}p(\mathit{y},\mathbf{\theta }|\mathit{d})\mathrm{d}\mathbf{\theta }\mathrm{d}\mathit{y},\hfill \end{array}$$

and note that $p(\mathbf{\theta }|\mathit{d})=p(\mathbf{\theta })$, since the specification of $\mathit{d}$ does not provide further information about inference of $\mathbf{\theta }.$ Then, the DLMC method approximates $U(\mathit{d})$ as

$$U(\mathit{d})\approx \frac{1}{n_{\mathrm{out}}}\sum _{i=1}^{n_{\mathrm{out}}}\left[\log (p({\mathit{y}}^{(i)}|{\mathbf{\theta }}^{(i)},\mathit{d}))-\log (p({\mathit{y}}^{(i)}|\mathit{d}))\right],$$

(5)

where ${\mathbf{\theta }}^{(i)}$ are drawn from the prior $p(\mathbf{\theta })$, and ${\mathit{y}}^{(i)}$ are drawn from the conditional distribution $p(\mathit{y}|\mathbf{\theta }={\mathbf{\theta }}^{(i)},\mathit{d})$ (i.e., the likelihood), and hence $p({\mathit{y}}^{(i)}|\mathit{d})$ can be estimated via the importance sampling technique,

$$p({\mathit{y}}^{(i)}|\mathit{d})={\int }_{\Theta }p({\mathit{y}}^{(i)}|\mathbf{\theta },\mathit{d})p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }\approx \frac{1}{n_{\mathrm{in}}}\sum _{j=1}^{n_{\mathrm{in}}}p({\mathit{y}}^{(i)}|{\mathbf{\theta }}^{(i,j)},\mathit{d}).$$

(6)

Combining (6) and (5) yields a biased estimator $\tilde{U}(\mathit{d})$ of $U(\mathit{d})$. However, if we sample ${\{{\mathbf{\theta }}^{(i,j)}\}}_{j=1}^{n_{\mathrm{in}}}$ for every $y^{(i)}$ ($i=1,\dots ,n_{\mathrm{out}}$), the complexity of this method is $\mathcal{O}(n_{\mathrm{out}}{n}_{\mathrm{in}})$. To reduce the computational cost, a sample reuse technique is employed in [14]. That is, for every $\mathit{d}$, we drawn a fresh batch from the prior ${\{{\mathbf{\theta }}^{(k)}\}}_{k=1}^{n_{\mathrm{out}}}$ and use this set for both the outer Monte Carlo and the inner Monte Carlo (i.e., ${\mathbf{\theta }}^{(·,j)}={\mathbf{\theta }}^{(k)}$). Consequently, the complexity is reduced to $\mathcal{O}(n_{\mathrm{out}})$.

3. GP Based Framework for Bayesian Optimal Design

Our main framework for Bayesian optimal design is based on two powerful tools according to Gaussian processes: the Bayesian Monte Carlo (BMC) method and the Bayesian optimization (BO) method. In this section, we first review BMC and conduct the analysis of BMC for the normal and the uniform distributions, and then present our novel double loop Bayesian Monte Carlo (DLBMC) for EIG. After that, we propose BO to find the maximizer of the approximated EIG. Finally, we review the classical Markov chain Monte Carlo (MCMC) method for Bayesian parameter inference.

3.1. Bayesian Monte Carlo

Consider the integral problem $I:={\int }_{\mathcal{X}}f(x)p(x)\mathrm{d}x,$ where $p(x)$ is the density of x, $f(x):{\mathbb{R}}^n\to \mathbb{R}$ is the integrand, and $\mathcal{X}$ is the support of x, that is, the integration domain. The idea of BMC is to formulate an integral problem into a Bayesian inference problem by placing a prior over the integrand f and to obtain the posterior distribution of f conditioning on the data collected. A natural way of putting a prior over function is through a Gaussian process, which is completely characterized by its mean function $\mu (x)$ and kernel function $k(x,x^{\prime })$, that is, $f\sim \mathcal{GP}(\mu (·),k(·,·)).$ A commonly used choice for the kernel function is the Gaussian kernel (or known as squared exponential kernel), $k(x,x^{\prime })={\sigma }_f^2\exp (-\parallel x-x^{\prime }{\parallel }_2^2/(2l^2))$, where both ${\sigma }_f$ and l are hyperparameters of the kernel function. The choice of hyperparameters affects the result to a large extent. Therefore the hyperparameters need to be determined carefully. Having collected noisy observations $D={\{x^{(i)},f^{(i)}\}}_{i=1}^N$, where $f^{(i)}=f(x^{(i)})+{ϵ}^{(i)}$ with Gaussian noises ${ϵ}^{(i)}\sim \mathcal{N}(0,{\sigma }^2),$ a Gaussian process provides a posterior distribution for an arbitrary new point $x^{\star }$, $f(x^{\star })|D,x^{\star }\sim \mathcal{N}({\mu }_N(x^{\star }),{\sigma }_N(x^{\star })),$ with mean function ${\mu }_N(x^{\star })$ and variance function ${\sigma }_N(x^{\star })$ given by

$$\begin{array}{cc}\hfill & {\mu }_N(x^{\star }):=k_N{(x^{\star })}^T{(K_N+{\sigma }^{2}I)}^{-1}{f}_N,\hfill \\ \hfill & {\sigma }_N(x^{\star }):=k(x^{\star },x^{\star })-k_N{(x^{\star })}^T{(K_N+{\sigma }^{2}I)}^{-1}{k}_N(x^{\star }),\hfill \end{array}$$

where $k_N(x^{\star })={\left[k(x^{\star },x^{(i)})\right]}_{N\times 1}$, $K_N={\left[k(x^{(i)},x^{(j)})\right]}_{N\times N}$, and $f_N={\left[f^{(i)}\right]}_{N\times 1}.$ In some special cases (For example, when x is distributed with Gaussian and the kernel is a Gaussian kernel [25]), GP allows us to estimate the integration in a closed form, of which the posterior mean is given by,

$$\begin{array}{cc}\hfill {\mathbb{E}}_{f|D}\left[I\right]& ={\int \int }_{\mathcal{X}}f(x)p(x)\mathrm{d}xp(f|D)\mathrm{d}f={\int }_{\mathcal{X}}\left[\int f(x)p(f|D)\mathrm{d}f\right]p(x)\mathrm{d}x\hfill \\ \hfill & ={\int }_{\mathcal{X}}{\mathbb{E}}_{f|D}(f)p(x)\mathrm{d}x={\int }_{\mathcal{X}}{\mu }_N(x)p(x)\mathrm{d}x={\int }_{\mathcal{X}}{k}_N{(x)}^T{(K_N+{\sigma }^{2}I)}^{-1}{f}_{N}p(x)\mathrm{d}x\hfill \\ \hfill & ={\int }_{\mathcal{X}}{k}_N{(x)}^{T}p(x)\mathrm{d}x{(K_N+{\sigma }^{2}I)}^{-1}{f}_N=z^T{(K_N+{\sigma }^{2}I)}^{-1}{f}_N,\hfill \end{array}$$

(7)

where $z:={\int }_{\mathcal{X}}{k}_N{(x)}^{T}p(x)\mathrm{d}x$, and the posterior variance is given by

$$\begin{array}{cc}\hfill & {\mathbb{V}}_{f|D}\left[I\right]={\mathbb{E}}_{f|D}\{{[I-{\mathbb{E}}_{f|D}(I)]}^2\}=\int {\left[{\int }_{\mathcal{X}}f(x)p(x)\mathrm{d}x-{\int }_{\mathcal{X}}{\mu }_N(x^{\prime })p(x^{\prime })\mathrm{d}{x}^{\prime }\right]}^{2}p(f|D)\mathrm{d}f\hfill \\ \hfill & ={\int \int }_{\mathcal{X}\times \mathcal{X}}\int {[f(x)-{\mu }_N(x^{\prime })]}^{2}p(f|D)\mathrm{d}fp(x)p(x^{\prime })\mathrm{d}x\mathrm{d}{x}^{\prime }={\int \int }_{\mathcal{X}\times \mathcal{X}}\mathbb{V}(f(x))p(x)p(x^{\prime })\mathrm{d}x\mathrm{d}{x}^{\prime }\hfill \\ \hfill & ={\int \int }_{\mathcal{X}\times \mathcal{X}}k(x,x^{\prime })p(x)p(x^{\prime })\mathrm{d}x\mathrm{d}{x}^{\prime }-z^T{(K_N+{\sigma }^{2}I)}^{-1}z.\hfill \end{array}$$

(8)

Note that in the above analytical expression of posterior mean and variance, when the hyperparameters and the kernel function are given, z and K are determined by ${\{x^{(i)}\}}_{i=1}^N$ and they are independent of the observations ${\{f^{(i)}\}}_{i=1}^N$. Therefore, the computation procedure for the posterior mean and variance of the BMC estimator proceeds the following two steps: first, an input sample set ${\{x^{(i)}\}}_{i=1}^N$ is generated, and z and K are computed; second, the corresponding observation set ${\{f^{(i)}\}}_{i=1}^N$ is collected, and the posterior mean and variance are computed through (7) and (8) respectively. Next, when x is an uniform or Gaussian random vector, we provide detailed derivations for the posterior mean and variance of BMC estimators.

Theorem 1

(Bayesian Monte Carlo for the standard Gaussian distribution). Consider the integral $I={\int }_{\mathcal{X}}f(x)p(x)dx$, where x is the standard Gaussian random variable vector in $\mathcal{X}={\mathbb{R}}^n$. The prior mean function is assumed to be a zero function, and the kernel is assumed to be the Gaussian kernel $k(x,x^{\prime })={\sigma }_f^2\exp (-\parallel x-x^{\prime }{\parallel }_2^2/(2l^2))$ with predetermined hyperparameters ${\sigma }_f$ and l. Having collected noisy observations $D={\{x^{(i)},f^{(i)}\}}_{i=1}^N$ where $f^{(i)}=f(x^{(i)})+{ϵ}^{(i)}$ and ${ϵ}^{(i)}\sim \mathcal{N}(0,{\sigma }^2)$, the posterior mean and variance of BMC are given by

$$\begin{array}{cc}\hfill & {\mathbb{E}}_{f|D}\left[I\right]=z^T{(K_N+{\sigma }^{2}I)}^{-1}{f}_N,\hfill \\ \hfill & {\mathbb{V}}_{f|D}\left[I\right]={\sigma }_f^2{\left(\frac{l^2}{l^2+2}\right)}^{n/2}-z^T{(K_N+{\sigma }^{2}I)}^{-1}z,\hfill \end{array}$$

where $K_N={\left[k(x^{(i)},x^{(j)})\right]}_{N\times N}$, $f_N={[f^{(1)},\dots ,f^{(N)}]}_{N\times 1}$, and the components of z are

$$z_i={\sigma }_f^2{\left(\frac{l^2}{l^2+1}\right)}^{n/2}\exp \left(-\frac{\parallel x^{(i)}{\parallel }^2}{2(l^2+1)}\right),$$

(9)

for $i=1,\dots ,N.$

Proof. 

The components of z can be computed analytically, for $i=1,\dots ,N,$

$$\begin{array}{cc}\hfill z_i& ={\int }_{\mathcal{X}}{\sigma }_f^2\exp \left(-\frac{\parallel x-x^{(i)}{\parallel }_2^2}{2l^2}\right)\frac{1}{\sqrt{{(2\pi )}^n}}\exp \left(-\frac{x^{T}x}{2}\right)\mathrm{d}x\hfill \\ \hfill & =\frac{{\sigma }_f^2}{\sqrt{{(2\pi )}^n}}{\int }_{\mathcal{X}}\exp \left[-\frac{1}{2}{x}^T\frac{(l^2+1)I}{l^2}x+\frac{x^T{x}^{(i)}}{l^2}-\frac{\parallel x^{(i)}{\parallel }^2}{2l^2}\right]\mathrm{d}x\hfill \\ \hfill & ={\sigma }_f^2{\left|\frac{l^2}{l^2+1}I\right|}^{1/2}\exp \left[-\frac{\parallel x^{(i)}{\parallel }^2}{2l^2}-\frac{\parallel x^{(i)}{\parallel }_2^2}{2l^2(l^2+1)}\right]\hfill \\ \hfill & ={\sigma }_f^2{\left(\frac{l^2}{l^2+1}\right)}^{n/2}\exp \left(-\frac{\parallel x^{(i)}{\parallel }_2^2}{2(l^2+1)}\right).\hfill \end{array}$$

Moreover, the variance of BMC is

$$\begin{array}{cc}\hfill {\mathbb{V}}_{f|D}\left[I\right]& ={\int \int }_{\mathcal{X}\times \mathcal{X}}k(x,x^{\prime })p(x)\mathrm{d}xp(x^{\prime })\mathrm{d}{x}^{\prime }-z^T{K}_N^{-1}z\hfill \\ \hfill & ={\int }_{\mathcal{X}}\frac{{\sigma }_f^2}{{(2\pi )}^{n/2}}{(2\pi )}^{n/2}{\left|\frac{l^2}{l^2+1}I\right|}^{1/2}\exp \left(-\frac{\parallel x^{\prime }{\parallel }_2^2}{2l^2}+\frac{\parallel x^{\prime }{\parallel }_2^2}{2l^2(l^2+1)}\right)p(x^{\prime })\mathrm{d}x-z^T{K}_N^{-1}z\hfill \\ \hfill & ={\sigma }_f^2{\left(\frac{l^2}{l^2+1}\right)}^{n/2}{\int }_{\mathcal{X}}\frac{1}{{(2\pi )}^{n/2}}\exp \left(-\frac{l^2+2}{2(l^2+1)}{\parallel x^{\prime }\parallel }_2^2\right)\mathrm{d}x-z^T{K}_N^{-1}z\hfill \\ \hfill & ={\sigma }_f^2{\left(\frac{l^2}{l^2+1}\right)}^{n/2}\frac{1}{{(2\pi )}^{n/2}}{(2\pi )}^{n/2}{\left(\frac{l^2+1}{l^2+2}\right)}^{n/2}-z^T{K}_N^{-1}z\hfill \\ \hfill & ={\sigma }_f^2{\left(\frac{l^2}{l^2+2}\right)}^{n/2}-z^T{K}_N^{-1}z.\hfill \end{array}$$

 □

We note that Theorem 1 is presented in References [22,25], but we give the above detailed proof for completeness.

Theorem 2

(Bayesian Monte Carlo for the uniform distribution). Consider the integral $I={\int }_{\mathcal{X}}f(x)p(x)dx$, where x is a random vector uniformly distributed in the hypercube $\mathcal{X}=[l_1,r_1]\times \cdots \times [l_n,r_n]$. The prior mean function is assumed to be a zero function, and the kernel is assumed to be the Gaussian kernel $k(x,x^{\prime })={\sigma }_f^2\exp (-\parallel x-x^{\prime }{\parallel }_2^2/(2l^2))$ with predetermined hyperparameters ${\sigma }_f$ and l. Having collected noisy observations $D={\{x^{(i)},f^{(i)}\}}_{i=1}^N$ where $f^{(i)}=f(x^{(i)})+{ϵ}^{(i)}$ and ${ϵ}^{(i)}\sim \mathcal{N}(0,{\sigma }^2)$, the posterior mean of BMC and the upper bound of variance of BMC are given by

$$\begin{array}{cc}\hfill & {\mathbb{E}}_{f|D}\left[I\right]=z^T(K_N+{\sigma }^{2}I)f,\hfill \\ \hfill & {\mathbb{V}}_{f|D}\left[I\right]<\frac{{\sigma }_f^2\sqrt{{(2\pi l^2)}^n}}{|\mathcal{X}|}-z^T(K_N+{\sigma }^{2}I)z,\hfill \end{array}$$

where $K_N={\left[k(x^{(i)},x^{(j)})\right]}_{N\times N}$, $f_N={[f^{(1)},\dots ,f^{(N)}]}_{N\times 1}$, and the components of z are given by

$$z_i=\frac{{\sigma }_f^2}{|\mathcal{X}|}\prod _{j=1}^n(\Phi (x_j;x_j^{(i)},l,r_j)-\Phi (x_j;x_j^{(i)},l,l_j)),$$

(10)

for $i=1,\dots ,N,$ with $\Phi (x;\mu ,\sigma ,t)$ being the cumulative distribution function (CDF) of the Gaussian distribution $\mathcal{N}(\mu ,{\sigma }^2)$.

Proof. 

The components of z can be computed analytically, for $i=1,\dots ,N,$

$$\begin{array}{cc}\hfill z_i& ={\int }_{\mathcal{X}}k(x,x^{(i)})p(x)\mathrm{d}x=\frac{{\sigma }_f^2}{|\mathcal{X}|}{\int }_{\mathcal{X}}\exp \left(-\frac{\parallel x-x^{(i)}{|\parallel }^2}{2l^2}\right)\mathrm{d}x\hfill \\ \hfill & =\frac{{\sigma }_f^2}{|\mathcal{X}|}{\int }_{\mathcal{X}}\exp \left(-\frac{{\sum }_{j=1}^n{(x_j-x_j^{(i)})}^2}{2l^2}\right)\mathrm{d}x=\frac{{\sigma }_f^2}{|\mathcal{X}|}\prod _{j=1}^n{\int }_{l_j}^{r_j}\exp \left(-\frac{{(x_j-x_j^{(i)})}^2}{2l^2}\right)\mathrm{d}{x}_j\hfill \\ \hfill & =\frac{{\sigma }_f^2}{|\mathcal{X}|}\prod _{j=1}^n(\Phi (x_j;x_j^{(i)},l,r_j)-\Phi (x_j;x_j^{(i)},l,l_j)),\hfill \end{array}$$

where $\Phi (x;\mu ,\sigma ,t)={\int }_{\infty }^t\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{{(x-\mu )}^2}{2{\sigma }^2}\right)\mathrm{d}x$ is the CDF of the Gaussian distribution $\mathcal{N}(\mu ,{\sigma }^2)$ and $v_j$ denotes the j-th component of the vector v. However, since the double integral of the Gaussian density function has no analytical form, the variance of the estimator cannot be obtained, and we give an upper bound of the variance,

$$\begin{array}{cc}\hfill {\mathbb{V}}_{f|D}\left[I\right]& ={\int }_{\mathcal{X}}{\int }_{\mathcal{X}}k(x,x^{\prime })p(x)\mathrm{d}xp(x^{\prime })\mathrm{d}{x}^{\prime }-z^T{K}_N^{-1}z<{\int }_{\mathcal{X}}{\int }_{{\mathbb{R}}^n}k(x,x^{\prime })p(x)\mathrm{d}xp(x^{\prime })\mathrm{d}{x}^{\prime }-z^T{K}_N^{-1}z\hfill \\ \hfill & =\frac{{\sigma }_f^2\sqrt{{(2\pi l^2)}^n}}{|\mathcal{X}|}{\int }_{\mathcal{X}}p(x^{\prime })\mathrm{d}{x}^{\prime }-z^T{K}_N^{-1}z=\frac{{\sigma }_f^2\sqrt{{(2\pi l^2)}^n}}{|\mathcal{X}|}-z^T{K}_N^{-1}z.\hfill \end{array}$$

 □

3.2. Estimating the Expected Information Gain Using Double-Loop BMC

In general, the value of the EIG has no closed form and has to be approximated via numerical methods. Based on the idea of BMC for efficiently evaluating integrals, we develop a double-loop BMC (DLBMC) scheme to approximate the EIG.

Letting $e(\mathit{y},\mathit{d})=p(\mathit{y}|\mathit{d})$, $g(\mathit{d},\mathit{y},\mathbf{\theta })=\{\ln [p(\mathit{y}|\mathbf{\theta },\mathit{d})]-\ln [p(\mathit{y}|\mathit{d})]\}p(\mathit{y}|\mathbf{\theta },\mathit{d})=\{\ln [p(\mathit{y}|\mathbf{\theta },\mathit{d})]-\ln [e(\mathit{y},\mathit{d})]\}p(\mathit{y}|\mathbf{\theta },\mathit{d})$, and $h(\mathit{d},\mathit{y})={\int }_{\Theta }g(\mathit{d},\mathit{y},\mathbf{\theta })p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }$, it is known that $U(\mathit{d})$ can be rewritten as

$$\begin{array}{cc}\hfill U(\mathit{d})& ={\int }_{\mathcal{Y}}{\int }_{\Theta }\{\ln \left[p(\mathit{y}|\mathbf{\theta },\mathit{d})\right]-\ln \left[e(\mathit{y},\mathit{d})\right]\}p(\mathit{y}|\mathbf{\theta },\mathit{d})p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }\mathrm{d}\mathit{y}\hfill \\ \hfill & ={\int }_{\mathcal{Y}}{\int }_{\Theta }g(\mathit{d},\mathit{y},\mathbf{\theta })p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }\mathrm{d}\mathit{y}={\int }_{\mathcal{Y}}h(\mathit{d},\mathit{y})\mathrm{d}\mathit{y}.\hfill \end{array}$$

First, we consider the straightforward detailed calculation of $U(\mathit{d})$ for any fixed $\mathit{d}$. To compute $U(\mathit{d})={\int }_{\mathcal{Y}}h(\mathit{d},\mathit{y})\mathrm{d}\mathit{y}$, we need samples ${\{{\mathit{y}}^{(i)},h^{(i)}:=h(\mathit{d},{\mathit{y}}^{(i)})\}}_{i=1}^{n_{\mathrm{out}}}$, where ${\mathit{y}}^{(i)}\sim \mathcal{U}(\mathcal{Y})$ for $i=1,\dots ,n_{\mathrm{out}}$, $n_{\mathrm{out}}$ denotes the sample size of the outer layer. To compute $h^{(i)}:={\int }_{\Theta }g(\mathit{d},{\mathit{y}}^{(i)},\mathbf{\theta })p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }$, samples ${\{{\mathbf{\theta }}^{(i,j)},g^{(i,j)}:=g(\mathit{d},{\mathit{y}}^{(i)},{\mathbf{\theta }}^{(i,j)})\}}_{j=1}^{n_{\mathrm{in}}}$ are needed, where $n_{\mathrm{in}}$ denotes the sample size of the inner layer. Again, $g^{(i,j)}=\{\ln \left[p({\mathit{y}}^{(i)}|{\mathbf{\theta }}^{(i,j)},\mathit{d})\right]-\ln \left[p({\mathit{y}}^{(i)}|\mathit{d})\right]\}p({\mathit{y}}^{(i)}|{\mathbf{\theta }}^{(i,j)},\mathit{d})$ also involves another integration $p({\mathit{y}}^{(i)}|\mathit{d})={\int }_{\Theta }p({\mathit{y}}^{(i)}|\mathit{d},\mathbf{\theta })p(\mathbf{\theta })\mathrm{d}\mathbf{\theta }$, and therefore samples ${\{{\mathbf{\theta }}^{(i,j)},p({\mathit{y}}^{(i)}|\mathit{d},{\mathbf{\theta }}^{(i,j)})\}}_{j=1}^{n_{\mathrm{in}}}$ are needed and ${\mathbf{\theta }}^{(i,j)}$ are generated from the prior. We propose using the BMC method to evaluate integrals ${\{e^{(i)}\}}_{i=1}^{n_{\mathrm{out}}}$, ${\{h^{(i)}\}}_{i=1}^{n_{\mathrm{out}}}$, and U. So far, there are two problems. First, the computation complexity for the forward model is $\mathcal{O}(n_{\mathrm{in}}{n}_{\mathrm{out}})$, which grows fast with the increase of the problem dimension. Second, since we usually have no prior knowledge of the support $\mathcal{Y}$ of $\mathit{y}$, we can not uniformly sample ${\{{\mathit{y}}^{(i)}\}}_{i=1}^{n_{\mathrm{out}}}$.

To overcome these obstacles, we employ the sample reuse technique [14] that sets ${\mathbf{\theta }}^{(·,j)}={\mathbf{\theta }}^{(j)}$, and the computational complexity is reduced to $\mathcal{O}(n_{\mathrm{in}})$. Besides, it allows us to generate samples ${\{{\mathbf{\theta }}^{(j)}\}}_{j=1}^{n_{\mathrm{in}}}$ in advance, and we can use the corresponding forward model outputs to estimate $\mathcal{Y}$—suppose the corresponding forward model values are given by ${\{G^{(j)}=\mathit{G}(\mathit{d},{\mathbf{\theta }}^{(j)})\}}_{j=1}^{n_{\mathrm{in}}}$, and then $\mathcal{Y}$ can be approximated by $\tilde{\mathcal{Y}}:=[\min (G)-\sigma ,\max (G)+\sigma ]$ where $G={[G^{(1)},\dots ,G^{n_{\mathrm{in}}}]}^T$. In this way, we can sample ${\{{\mathit{y}}^{(i)}\}}_{i=1}^{n_{\mathrm{in}}}\sim \mathcal{U}(\tilde{\mathcal{Y}})$. Intuitively speaking, the approximated $\tilde{\mathcal{Y}}$ is slightly smaller than the actual field $\mathcal{Y}$, and consequently, bias is induced in the estimator. With increased sample size $n_{\mathrm{in}}$, $\tilde{\mathcal{Y}}$ can be captured more accurately and the bias can be reduced.

In the process of estimating U, we propose using the BMC method to compute e, h and U. Since two layers of integration are involved, let the hyperparameters of BMC for the inner layer and the outer layer be $\{l_{\mathrm{in}},{({\sigma }_f)}_{\mathrm{in}}\}$ and $\{l_{\mathrm{out}},{({\sigma }_f)}_{\mathrm{out}}\}$ respectively. In our previous discussion about BMC in (7)–(8), z and K can be computed, once the input sample set ${\{x^{(i)}\}}_{i=1}^N$ is given. Therefore, for computational simplicity, after ${\{{\mathbf{\theta }}^{(j)}\}}_{j=1}^{n_{\mathrm{in}}}$ and ${\{{\mathit{y}}^{(i)}\}}_{i=1}^{n_{\mathrm{out}}}$ are generated, we can compute $\{z_{\mathrm{in}},K_{\mathrm{in}}\}$ and $\{z_{\mathrm{out}},K_{\mathrm{out}}\}$ ahead. Taking the prior being the standard normal distribution for example, $z_{\mathrm{in}}$ and $K_{\mathrm{in}}$ are given by,

$$\begin{array}{cc}\hfill & {(z_{\mathrm{in}})}_j={({\sigma }_f)}_{\mathrm{in}}^2{\left(\frac{l_{\mathrm{in}}^2}{l_{\mathrm{in}}^2+1}\right)}^{n_{\theta }/2}\exp \left(-\frac{\parallel {\mathbf{\theta }}^{(j)}{\parallel }_2^2}{2(l_{\mathrm{in}}^2+1)}\right),\mathrm{for}j=1,\dots ,n_{\mathrm{in}},\hfill \\ \hfill & K_{\mathrm{in}}=[{({\sigma }_f)}_{\mathrm{in}}^2\exp (-\parallel {\mathbf{\theta }}^{(i)}-{\mathbf{\theta }}^{(j)}{\parallel }_2^2/(2l_{\mathrm{in}}^2){)]}_{n_{\mathrm{in}}\times n_{\mathrm{in}}}.\hfill \end{array}$$

Details of our DLBMC method for estimating U are summarized in Algorithm 1. Note that we only use the mean estimates of the DLBMC estimator in the following. The variance of DLBMC is potentially useful, but as discussed in Section 3.1, the variance of BMC typically does not have a closed form, and we are not able to derive a closed form for the variance of DLBMC in this work. We will consider the variance of DLBMC in our future work. It is also possible to consider other numerical integration methods to compute EIG, for example, the sparse grid quadrature rules [31,32] and their combination with physical model reduction techniques [33], but they are out of the scope of this paper.

Algorithm 1 Double-Loop Bayesian Monte Carlo (DLBMC) for estimating EIG

1: Input: Design points $\mathit{d}$, prior $p(\mathbf{\theta })$, standard deviation of noise $\sigma$, hyperparameters $\{l_{\mathrm{in}},{({\sigma }_f)}_{\mathrm{in}}\}$ and $\{l_{\mathrm{out}},{({\sigma }_f)}_{\mathrm{out}}\}$. 2: Data preparation: Sample ${\{{\mathbf{\theta }}^{(j)}\}}_{j=1}^{n_{\mathrm{in}}}\sim p(\mathbf{\theta })$ and compute $G^{(j)}=\mathit{G}(\mathit{d},{\mathbf{\theta }}^{(j)})$ for $j=1,\dots ,n_{\mathrm{in}}$. 3: Sample ${\{{\mathit{y}}^{(i)}\}}_{i=1}^{n_{\mathrm{out}}}\sim \mathcal{U}(\min (G)-\sigma ,\max (G)+\sigma )$. 4: Compute $\{z_{\mathrm{in}},K_{\mathrm{in}}\}$ and $\{z_{\mathrm{out}},K_{\mathrm{out}}\}$. 5: for$i=1,\dots ,n_{\mathrm{out}}$do 6:     for $j=1,\dots ,n_{\mathrm{in}}$ do 7:         Compute the likelihood $f^{(i,j)}=p({\mathit{y}}^{(i)}|{\mathbf{\theta }}^{(j)},\mathit{d})$. 8:     end for 9:     Let $f^{(i)}={[f^{(i,1)},\dots ,f^{(i,n_{\mathrm{in}})}]}^T$. 10:     Compute the evidence $e^{(i)}=z_{\mathrm{in}}^T{K}_{\mathrm{in}}^{-1}{f}^{(i)}$. 11:     for $j=1,\dots ,n_{\mathrm{in}}$ do 12:         $g^{(i,j)}=[\log (f^{(i,j)})-\log (e^{(i)})]f^{(i,j)}.$ 13:     end for 14:     Let $g^{(i)}={[g^{(i,1)},\dots ,g^{(i,n_{\mathrm{in}})}]}^T$. 15:     Compute $h^{(i)}=z_{\mathrm{in}}^T{K}_{\mathrm{in}}{g}^{(i)}$. 16: end for 17: Let $h={[h^{(1)},\dots ,h^{(n_{\mathrm{out}})}]}^T$. 18: Compute $\widehat{U}(\mathit{d})=z_{\mathrm{out}}^T{K}_{\mathrm{out}}^{-1}h.$ 19: Output: the estimated EIG $\widehat{U}(\mathit{d})$.

3.3. Bayesian Optimization

The ultimate goal of the optimal experimental design problem is to find the optimizer ${\mathit{d}}^{\star }$ in (4). In this problem, since the computing of the function value $U(\mathit{d})$ and the gradient $\nabla U$ is prohibitively expensive, it is challenging to apply function-value-based or gradient-based optimization methods. As the Bayesian optimization (BO) method [27,28,34,35] typically only requires a low objective function evaluation budget [36] and does not require any gradient information, it can be suitable for this problem. In this section, we give a brief review of BO and apply it to obtain the maximizer of EIG (4).

To compute the maximizer of EIG $U:{\mathbb{R}}^{n_d}\to \mathbb{R}$ (see (4)), for a given maximum number of iterations $t_{\max }$, that is, the evaluation budget, Bayesian optimization begins with putting a GP prior on $U\sim \mathcal{GP}({\mu }_0(\mathit{d}),k(\mathit{d},{\mathit{d}}^{\prime }))$, and then randomly chooses an initial point ${\mathit{d}}_1$ and collects the corresponding response $U_1=U({\mathit{d}}_1)$. Next, the posterior mean function ${\mu }_1(\mathit{d})$ and variance ${\sigma }_1(\mathit{d})$ are updated via collected data set ${\mathcal{S}}_1=\{{\mathit{d}}_1,U_1\}$. Usually ${\mathit{d}}_1$ alone is inadequate to find the maximum and therefore we need a strategy to choose the next design point. Typically, the next point is determined through maximizing an acquisition function A, that is, at t-th iteration, ${\mathit{d}}_{t+1}=\arg {\max }_{\mathit{d}}A(\mathit{d}|{\mathcal{S}}_{1:t})$. After the next point ${\mathit{d}}_2$ is obtained, we sample the objective function $U_2$. The collected data set is then augmented as ${\mathcal{S}}_{1:2}={\mathcal{S}}_1\cup \{{\mathit{d}}_2,U_2\}={\{{\mathit{d}}_i,U_i\}}_{i=1,2}$, and the posterior mean function ${\mu }_2(\mathit{d})$ and the variance function ${\sigma }_t(\mathit{d})$ are also updated. The above procedure repeats until the given maximum budget $t_{\max }$ is reached.

In the case that $\mathcal{D}$ is infinite, the process of finding the next design point ${\mathit{d}}_{t+1}=\arg {\max }_{\mathit{d}\in \mathcal{D}}A(\mathit{d}|S_{1:t})$ is demanding. However, performing global search over the discretized space is usually effective [27,37], since we assume that evaluating U is more costly than computing the GP surrogate. Therefore, the design space $\mathcal{D}$ is discretized over equidistant grids and we denote the discretized design space as $\overline{\mathcal{D}}$. Supposing we have collected data set ${\mathcal{S}}_{1:t}$, the posterior of U is a GP distribution with mean ${\mu }_t(\mathit{d})$, kernel $k(\mathit{d},{\mathit{d}}^{\prime })$ and variance ${\sigma }_t^2(\mathit{d})$,

$$\begin{array}{cc}\hfill & {\mu }_t(\mathit{d})=k_t{(\mathit{d})}^T{(K_t+{\sigma }^{2}I)}^{-1}{U}_{1:t},\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill & {\sigma }_t(\mathit{d})=k(\mathit{d},\mathit{d})-K_t{(\mathit{d})}^T{(K_t+{\sigma }^{2}I)}^{-1}{k}_t(\mathit{d}),\hfill \end{array}$$

(12)

where $k_t(\mathit{d})={\left[k(\mathit{d},{\mathit{d}}_i)\right]}_{t\times 1}$, $K_t={\left[k({\mathit{d}}_i,{\mathit{d}}_j)\right]}_{t\times t}$ and $U_{1:t}={\left[U_i\right]}_{t\times 1}$. We note the design space considered in this paper is assumed to be bounded, such that it can be directly discretized. For unbounded design spaces, an unbounded Bayesian optimization approach is developed through gradually extending regions with regularization in [38].

Choosing a proper acquisition function is crucial for the Bayesian optimization algorithm since it guides the search for the optimum. Popular choices of acquisition function include maximizing the probability of improvement (PI) [39,40], and maximizing the expected improvement (EI) in the efficient global optimization (EGO) algorithm [41,42]. A review for the selection of acquisition functions is in [27]. Suggested by [37], we apply the GP-UCB algorithm to choose the next point—the acquisition function is set to a linear combination of the posterior mean function and the posterior variance function,

$${\mathit{d}}_t=\arg \underset{\mathit{d}\in \mathcal{D}}{\max }{\mu }_{t-1}(\mathit{d})+\sqrt{{\beta }_{t-1}}{\sigma }_{t-1}(\mathit{d}),$$

where ${\mu }_{t-1}(\mathit{d})+\sqrt{{\beta }_{t-1}}{\sigma }_{t-1}(\mathit{d})$ can be considered as the upper confidence bound of the current Gaussian process. It is clear that maximizing the acquisition function ${\mu }_{t-1}(\mathit{d})+\sqrt{{\beta }_{t-1}}{\sigma }_{t-1}(\mathit{d})$ shows a trade-off between exploring the point with potential high function value and exploiting the point with high uncertainty. Here, ${\beta }_{t-1}$ is the parameter balancing exploring and exploiting. Details of our BO strategy for optimal design are shown in Algorithm 2.

We set the prior mean function to ${\mu }_0(\mathit{d})=0$, and set the kernel to be the Gaussian kernel given by $k(\mathit{d},{\mathit{d}}^{\prime })={\sigma }_f^2\exp (-\parallel \mathit{d}-{\mathit{d}}^{\prime }{\parallel }^2/(2l^2)).$ The design space $\mathcal{D}$ is discretized over equidistant grids and we denote the discretized design space as $\overline{\mathcal{D}}$. In the step of maximizing the acquisition function, ${\mathit{d}}_t$ is located through a grid search over $\overline{\mathcal{D}}$.

A natural measure in performance of the Bayesian optimization method is defined through average cumulative regret. Supposing the maximum $U({\mathit{d}}^{\star })$ is known, the instantaneous regret for iteration t is defined as $r_t=U({\mathit{d}}^{\star })-U({\mathit{d}}_t)$ and the cumulative regret $R_T$ after T iterations is defined as the sum of the instantaneous regrets $R_T={\sum }_{t=1}^T{r}_t$. Then the average cumulative regret $R_T/T$ is defined as $R_T/T={\sum }_{t=1}^T{r}_t/T$. It should be noted that neither $r_t$ nor $R_T$ can be obtained directly from the Algorithm 2. In [37], it is proven that, for finite design space $\overline{\mathcal{D}}$, setting $\delta \in (0,1)$ and ${\beta }_t=2\log (|\overline{\mathcal{D}}|t^2{\pi }^2/6\delta )$, the Bayesian optimization method is no-regret with high probability, that is, ${\lim }_{T\to \infty }{R}_T/T=0.$

Algorithm 2 Bayesian optimization (BO) for optimal design

1: Input: Design space $\mathcal{D}$ and its discretized design space $\overline{\mathcal{D}}$, prior ${\mu }_0(\mathit{d})=0$, hyperparameters $l,{\sigma }_f$ of the Gaussian kernel, hyperparameter $\delta$, and maximum number of iterations $t_{\max }$. 2: for$t=1,\dots ,t_{\max }$do 3:     Find the maximizer of the acquisition function: ${\mathit{d}}_t=\arg {\max }_{d\in \mathcal{D}}{\mu }_{t-1}(\mathit{d})+\sqrt{{\beta }_{t-1}}{\sigma }_{t-1}(\mathit{d})$. 4:     Sample the objective function $U_t=\widehat{U}({\mathit{d}}_t)$ using Algorithm 1. 5:     Augment the data set ${\mathcal{S}}_{1:t}={\{{\mathit{d}}_i,U_i\}}_{i=1}^t$. 6:     Perform Bayesian update to obtain ${\mu }_t$ and ${\sigma }_t$ over $\overline{\mathcal{D}}$ using (11) and (12) respectively. 7:     Update ${\beta }_t.$ 8: end for 9: Output: Optimal design: $d^{\star }=\arg {\max }_{t=1,\dots ,t_{\max }}{U}_t$

3.4. Bayesian Parameter Inference

After the optimal design points are selected and the corresponding noisy observations $D={\{{\mathit{d}}^{(i)},{\mathit{y}}^{(i)}\}}_{i=1}^N$ are collected, we can conduct Bayesian inference for the system parameters, that is, to assess the posterior distribution $p(\mathbf{\theta }|D)$. The posterior $p(\mathbf{\theta }|D)$ can be calculated via Bayes’ rule (1). However, as there is no closed-form for the evidence in (1) in many practical problems, the Markov chain Monte Carlo (MCMC) method is often used to generate samples of the posterior distribution. Next, we give a brief review of the MCMC algorithm.

The basic idea of MCMC is to construct a Markov chain over the state space until the chain has reached a stationary distribution, which is assumed to be a target distribution. Here our target distribution is set to be the posterior distribution $p(\mathbf{\theta }|D)$. Although there are many variants of MCMC, we focus on the Metropolis-Hastings MCMC (MH-MCMC) method [43,44,45]. The basic idea of constructing the Markov chain in MH-MCMC algorithm is that at each step, given current state $\mathbf{\theta }$, a candidate state ${\mathbf{\theta }}^{\mathrm{cand}}$ is proposed with probability $q({\mathbf{\theta }}^{\mathrm{cand}}|\mathbf{\theta })$, where $q(·|·)$ is referred to as the proposal distribution. Note that proposal distribution can be arbitrary. A commonly used proposal is the symmetric Gaussian distribution, that is, $q({\mathbf{\theta }}^{\mathrm{cand}}|\mathbf{\theta })=\mathcal{N}(\mathbf{\theta },\lambda I)$, where $\lambda$ denotes the stepsize. Whether to accept the candidate state is determined by the acceptance probability $\alpha$, given by the following formula,

$$\alpha =\min \left\{1,\frac{p({\mathbf{\theta }}^{\mathrm{cand}}|D)/q({\mathbf{\theta }}^{\mathrm{cand}}|\mathbf{\theta })}{p(\mathbf{\theta }|D)/q(\mathbf{\theta }|{\mathbf{\theta }}^{\mathrm{cand}})}=\frac{p({\mathbf{\theta }}^{\mathrm{cand}}|D)}{p(\mathbf{\theta }|D)}\right\}.$$

Note that the equation holds only when the proposal distribution is symmetric. If $p({\mathbf{\theta }}^{\mathrm{cand}}|D)/p(\mathbf{\theta }|D)>1$, it means that ${\mathbf{\theta }}^{\mathrm{cand}}$ is more possible than the current state $\mathbf{\theta }$, we accept the proposal with probability $\alpha =1$. Otherwise, we accept the proposal with probability $\alpha$. The detailed MH-MCMC algorithm is summarized in Appendix B.

4. Numerical Experiments

In this section, we consider three numerical examples: a standard linear Gaussian model, a nonlinear simple model, and a partial differential equation (PDE) model. Since the first problem has analytical expressions, we examine the performance of our method by comparing the numerical result with the exact solution. The second problem is a commonly-used test problem, and we demonstrate the efficiency of our method through it. In the third test problem, a physical system governed by the diffusion equation is considered, in which a contaminant source inversion problem is studied.

4.1. Test Problem 1: Linear Gaussian Problem

We consider the standard linear Gaussian problem in the following form,

$$G(\mathbf{\theta },\mathit{d})={\mathbf{\theta }}^T\mathit{d},\mathit{y}=G(\mathbf{\theta },\mathit{d})+\mathbf{ϵ},$$

where the noise and the prior are assumed to be Gaussian, $\mathbf{ϵ}\sim \mathcal{N}(0,{\sigma }^2),$$\mathbf{\theta }\sim \mathcal{N}(0,I_{n_{\theta }\times n_{\theta }})$ and $n_{\theta }=n_d$. The posterior distribution is a multivariate Gaussian distribution

$$\begin{array}{c}\hfill \mathbf{\theta }|\mathit{d},\mathit{y}\sim \mathcal{N}(\overline{\mathbf{\theta }},\overline{\Sigma }),\end{array}$$

where the mean and the covariance are

$$\begin{array}{c}\hfill \overline{\mathbf{\theta }}=\frac{\mathit{y}}{{\sigma }^2}\overline{\Sigma }\mathit{d},\overline{\Sigma }={\left(\frac{\mathit{d}{\mathit{d}}^T}{{\sigma }^2}+I\right)}^{-1}.\end{array}$$

The expected information gain (EIG) for $\mathbf{\theta }$ can then be given in a closed form. (The detailed deduction is shown in Appendix A.),

$$U(\mathit{d})=-\frac{1}{2}\log \left[\det \left(\overline{\Sigma }\right)\right].$$

(13)

Note that maximizing EIG is equivalent to minimizing the determinant of the posterior covariance matrix [11] (the Bayesian D-optimal design).

We consider the one-dimensional case where $n_{\theta }=1$, $d\in [0,1]$, and set the standard deviation of the noise to $\sigma =0.1$. The hyperparameters of DLBMC are set to $l_{\mathrm{in}}=0.5,{({\sigma }_f)}_{\mathrm{in}}=1,$ and $l_{\mathrm{out}}=0.2,{({\sigma }_f)}_{\mathrm{out}}=1.$ Figure 1 shows the exact EIG, the estimated EIG using DLMC with 300 samples, and the estimated EIG using DLBMC with 300 samples. It can be seen that the estimated EIG of DLBMC is more accurate and more stable than that of DLMC. Besides, for $d=0.3$, the relationship between the sample size and the bias of the DLMC and DLBMC estimators is studied. As the sample size increases, we compute the bias of the DLBMC estimator and the DLMC estimator averaging 20 trails respectively. Figure 2(Left) shows the results of the bias, where it can be seen that the DLBMC estimator converges to the true value faster than the DLMC estimator, and DLBMC is more accurate than DLMC.

As discussed in Section 3.2, the variance of the BMC estimator for the uniform distribution is not explicitly given. We use the sample variance as an alternative to compare the discrepancy of the DLMC estimator and the DLBMC estimator. Fixing the design point $d=0.5$, for different sample sizes of DLMC and DLBMC, we repeatedly compute the estimator $\widehat{U}(d)$ n times, and denote them by ${\{{\widehat{U}}^{(i)}\}}_{i=1}^n$. Let $\overline{U}(d)$ denote the mean estimator, $\overline{U}(d):=\frac{1}{n}{\sum }_{i=1}^n{\widehat{U}}^{(i)}(d)$, and then the sample variance estimator is defined as

$$s^2:=\frac{{\sum }_{i=1}^n({\widehat{U}}^{(i)}(d)-\overline{U}(d))}{n-1}.$$

As the number of repeat times n increases, we compare the sample variance of two estimators with different sample sizes for DLMC and DLBMC in Figure 2(Right). It is clear that the DLBMC estimator outperforms the DLMC estimator.

4.2. Test Problem 2: Nonlinear Simple Problem

In this section, a simple nonlinear model is tested, which is also studied in [14]. This model is written as

$$G(\theta ,d)={\theta }^3{d}^2+\theta \exp (-|0.2-d|),$$

the prior is set to $\theta \sim \mathcal{U}(0,1)$, and the standard deviation of the observation noise is set to $\sigma =0.01$. The hyperparameters of DLBMC are set to $l_{\mathrm{in}}=0.01,{({\sigma }_f)}_{\mathrm{in}}=0.2,$ and $l_{\mathrm{out}}=0.05,{({\sigma }_f)}_{\mathrm{out}}=0.2.$

Figure 3a,b show the estimated EIG using DLMC and DLBMC with 300 and 500 samples respectively. A reference solution using DLMC with ${10}^5$ samples is also compared in Figure 3a,b. Here, 20 trails of the DLMC estimator and the DLBMC estimator are generated, and Figure 3 shows the mean estimates and the intervals containing 80% of the trails. It is clear that, compared to DLMC, our DLBMC estimator has smaller variances. Compared to the reference solution, DLBMC gives biased estimation. With the increasing sample size, the extent of bias is reduced as we expect.

To illustrate the effect of optimal design, we compare the posterior distribution given by three design points. Let design A = 1 be the optimal design point, let design B = 0.2 be the local maximizer of the EIG, and let design C = 0 which has the least information since it has the least EIG value. The MH-MCMC algorithm (Algorithm A1) with $N_{\mathrm{iter}}=1000$ and $\gamma =0.2$ is used to generate samples of the posterior distribution, and kernel density estimation is used to obtain the posterior density functions from these samples. For this test problem, the ground-truth is set to 0.75. From Figure 4, it can be seen that the posterior distribution obtained through design A is the most accurate, and it has the smallest variance.

4.3. Test Problem 3: Source Inversion for the Diffusion Problem

Letting $\mathcal{D}\subset {\mathrm{R}}^2$ be a bounded and connected domain with a polygonal boundary $\partial \mathcal{D}$, the governing equation of the diffusion problem studied in this test problem is: find a random function $u(\mathit{x},\omega )\in D\times \Omega \to \mathrm{R}$, such that $\mathcal{P}$-a.e. in $\Omega$,

$$\begin{array}{cc}\hfill -{\nabla }^{2}u(\mathit{x},\omega )=f(\mathit{x},\omega ),& \mathrm{in}\mathcal{D},\end{array}$$

(14)

$$\begin{array}{cc}\hfill u(\mathit{x},\omega )=0,& \mathrm{on}\partial \mathcal{D},\end{array}$$

(15)

where $(\Omega ,\mathcal{F},\mathcal{P})$ is a probability space. We consider a square physical domain $\mathcal{D}=[-1,1]\times [-1,1]\subset {\mathbb{R}}^2$, and $u(\mathit{x},\omega )$ in (14)–(15) denotes the concentration of a contaminant at the point $\mathit{x}\in \mathcal{D}$. Let $f(\mathit{x},\omega )$ denote the field of contaminant source. As f is usually strictly positive following the setting in [13,46,47,48], we set the prior distribution of $f(\mathit{x},\omega )$ to a log-normal random field, that is, $f(\mathit{x},\omega )=\exp (a(\mathit{x},\omega ))$ where $a(\mathit{x},\omega )$ is a normal random field. In this study, the experimental goal is to infer the underlying contaminant field f given several observations ${\{{\mathit{x}}_i,{\mathit{y}}_i\}}_{i=1}^K$, where design variable ${\mathit{x}}_i$ denotes i-th sensor placement, the response is the corresponding numerical PDE solution $u({\mathit{x}}_i)$ with additional noise, that is, ${\mathit{y}}_i=u({\mathit{x}}_i)+{ϵ}_i$, ${ϵ}_i\sim \mathcal{N}(0,{\sigma }^2)$, and K is the number of sensors. In this test problem, the hyperparameters of DLBMC are set to $l_{\mathrm{in}}=0.02,{({\sigma }_f)}_{\mathrm{in}}=0.01,$ and $l_{\mathrm{out}}=0.005,{({\sigma }_f)}_{\mathrm{out}}=0.005.$

We parameterize the permeability field $\log [f(\mathit{x},\omega )]$ by a truncated Karhunen-Loève (KL) expansion. Consider the random field $a(\mathit{x},\omega )=\log [f(\mathit{x},\omega )]$ with mean function $a_0(\mathit{x})$, standard deviation $\sigma$ and covariance function $C(\mathit{x},\mathit{y})$,

$$C(\mathit{x},\mathit{y})=\sigma \exp \left(-\frac{|x_1-y_1|}{c}-\frac{|x_2-y_2|}{c}\right),$$

(16)

where c is the correlation length. Then the truncated KL expansion of f is expressed as

$$f(\mathit{x},\omega )\approx \exp \left(a_0(\mathit{x})+\sum _{n=0}^M\sqrt{{\lambda }_n}{\xi }_n{a}_n(\mathit{x})\right),$$

(17)

where $a_n(\mathit{x})$ and ${\lambda }_n$ are the eigenfunctions and eigenvalues of (16) and ${\{{\xi }_n\}}_{n=1}^M$ are uncorrelated random variables. The prior of ${\{{\xi }_n\}}_{n=1}^M$ are set to be independent standard normal distributions, ${\xi }_n\sim \mathcal{N}(0,1)$ for $n=1,\dots ,M$.

In the numerical experiment, we set $a_0(x)=1$, $c=2$ and $\sigma =1$. Fixing the hyperparameters in the truncated KL expansion, the response depends on the random variables ${\{{\xi }_i\}}_{i=1}^M$. Therefore we define the parameter of interest as $\mathbf{\theta }:=[{\xi }_1,\dots ,{\xi }_M]$, and rewrite the governing diffusion equation as,

$$\begin{array}{cc}\hfill -{\nabla }^{2}u(\mathit{x},\mathbf{\theta })=f(\mathit{x},\mathbf{\theta }),& \mathrm{in}\mathcal{D}\times \Gamma ,\end{array}$$

(18)

$$\begin{array}{cc}\hfill u(\mathit{x},\mathbf{\theta })=0,& \mathrm{on}\partial \mathcal{D}\times \Gamma .\end{array}$$

(19)

We use the bilinear finite element method (FEM) to discretize the diffusion equation over a $17\times 17$ square grid and let the standard deviation of noise be 1% of the mean observed value. Supposing K sensors are placed over the design space, generally, we can perform a batch design, and write the following altered forward model

$$\left[\begin{array}{c}{\mathit{y}}_1\\ ⋮\\ {\mathit{y}}_i\\ ⋮\\ {\mathit{y}}_K\end{array}\right]=\left[\begin{array}{c}\mathit{G}({\mathit{d}}_1,\mathbf{\theta })\\ ⋮\\ \mathit{G}({\mathit{d}}_i,\mathbf{\theta })\\ ⋮\\ \mathit{G}({\mathit{d}}_K,\mathbf{\theta })\end{array}\right]+\left[\begin{array}{c}{\mathbf{ϵ}}_1\\ ⋮\\ {\mathbf{ϵ}}_i\\ ⋮\\ {\mathbf{ϵ}}_K\end{array}\right]=\mathcal{G}({\mathit{d}}_{1:K},\mathbf{\theta })+\mathbf{ϵ},$$

where the subscript i denotes the i-th design variable for $i=1,\dots ,K$. Directly maximing over the EIG with altered forward model can give optimal design in the context of batch design.

Let $f_{\mathrm{truth}}$ denote the underlying true permeability field. Suppose we have collected data on K sensors, and then we perform MCMC to get samples ${\{{\mathbf{\theta }}^{(i)}\}}_{i=1}^{N_{\mathrm{iter}}}$ of the posterior distribution using Algorithm A1. In this test problem, we set $N_{\mathrm{iter}}=4000$. Two useful statistics can be obtained from the samples—the maximum a posterior (MAP) estimate ${\mathbf{\theta }}_{\mathrm{MAP}}$ and the mean estimate ${\mathbf{\theta }}_{\mathrm{MEAN}}$. Let $f_{\mathrm{MAP}}$ and $f_{\mathrm{MEAN}}$ be the source fields generated by ${\mathbf{\theta }}_{\mathrm{MAP}}$ and ${\mathbf{\theta }}_{\mathrm{MEAN}}$ respectively. To test the accuracy of the inversion, we introduce the following relative errors

$$\begin{array}{cc}\hfill & E_{\mathrm{MAP}}=\frac{\parallel f_{\mathrm{MAP}}-f_{\mathrm{truth}}{\parallel }_2}{\parallel f_{\mathrm{truth}}{\parallel }_2},\hfill \\ \hfill & E_{\mathrm{MEAN}}=\frac{\parallel f_{\mathrm{MEAN}}-f_{\mathrm{truth}}{\parallel }_2}{\parallel f_{\mathrm{truth}}{\parallel }_2},\hfill \end{array}$$

where $f_{\mathrm{truth}},f_{\mathrm{MAP}}$ and $f_{\mathrm{MEAN}}$ are discretized over the FEM grids for computational simplicity and ${\parallel ·\parallel }_2$ denotes the $l^2$ vector norm.

Noting that performing grid search over ${(17\times 17)}^K$ grid is computationally expensive, we utilize Bayesian optimization method to efficiently find the optimal designs within a few iterations. Three cases of K are considered in the following, which are $K=1,2,3$.

First, for $K=1$, we first perform Bayesian optimization over a $17\times 17$ grid. The performance of Bayesian optimization is shown in Figure 5. In Figure 5(Left), we can see that, Bayesian optimization can find the maximum of the EIG within a few iterations. The optimal design found by Bayesian optimization is $[0,0]$, which is expected due to the symmetry of the forward model. In this case, as the number of gird points (289 points) is not too large, we can verify that the optimal design is $[0,0]$ through grid search. Figure 5(Right) shows that the average cumulative regret is convergent to zero.

For comparison, we randomly generate 20 different design points and use the MH-MCMC algorithm to generate 4000 samples of the posterior distribution. Figure 6 shows the locations of the optimal design and the 20 random designs. Figure 7a,b show the relative errors of MAP and mean estimates of the source field (averaged over 20 trails) versus the values of EIG, where it is clear that as the value of EIG becomes larger, the errors of both MAP and mean estimates reduce. In addition, the optimal design point gives large EIG values and relatively small errors. Although the errors associated with the optimal design point are typically smaller than the errors associated with the random design points. They are still large, and the estimated source fields are not accurate enough. Therefore, we next consider more design points.

For the multiple design cases ($K=2,3$), given the fact that the computational cost of batch design increases exponentially as K increases, we perform Bayesian optimization over a uniform $9\times 9$ coarse grid. For $K\ge 2$, it can be seen that the optimal solution is not unique due to the symmetry of the forward model, for example, $[{\mathit{x}}_1;{\mathit{x}}_2]$ and $[-{\mathit{x}}_1;-{\mathit{x}}_2]$ share the same EIG value. After performing Bayesian optimization several times with BO budget $t_{\max }=100$, the sets of optimal designs for $K=2$ case are shown in Figure 8, where each line connecting blue circle and red circle represents a pair of optimal design. The numerical result shows that, compared with a pair of two design points that are symmetric with respect to $[0;0]$, a pair of slightly skewed design points can provide more information.

Again, we randomly generate 20 sets of design points, and compare them with the optimal design (we choose $[0.5,0.5;-0.5,-0.25]$). Figure 9 shows the errors of MAP and mean estimates (averaged over 20 trails) of the source field versus the values of EIG, where it is clear that the optimal design leads to the largest EIG value and the smallest error. Besides, the comparison of true source field and the fields generated by MAP and mean estimates are presented in Figure 10. It can be seen that the estimated source field associated with the optimal design matches the true source field well.

For $K=3$, let the BO budget $t_{\max }=100$, the set of optimal design that found by Algorithm 2 is $[0.75,0.25;0.5,-0.25;-0.75,-0.25]$. Figure 11 shows the values of EIG and the relative errors in MAP and mean estimates (averaged over 20 trails) for the optimal design and twenty random designs, where it can be seen that the optimal design has the largest EIG value and the smallest errors which are consistent with the results for $K=1,2$. Figure 12 shows that the estimated source fields generated by the MAP and the mean estimates match the true source field well.

To further quantify the performance of optimal designs, we compute the ratio of $E_{\mathrm{MAP}}$ of random designs and $E_{\mathrm{MAP}}$ of optimal designs (denoted as $E_{\mathrm{MAP}}^{(\mathrm{random})}$ and $E_{\mathrm{MAP}}^{(\mathrm{opt})}$ respectively) for $K=1,2,3$ cases. Figure 13 presents the histograms of ratio of relative errors (i.e., $E_{\mathrm{MAP}}^{(\mathrm{random})}/E_{\mathrm{MAP}}^{(\mathrm{opt})}$), where the green lines are the corresponding kernel smoothing function estimates. We can see that, in all three cases, with high probability, the optimal design can give better posterior performance than the random designs. Especially, from Figure 13c, the error of random designs can be ten times greater than that of the optimal design.

5. Conclusions and Discussion

Efficiently using a small number of samples to reduce the cost of computing the expected information gain (EIG) is a fundamental concept to solve the challenging Bayesian optimal experimental design problem. Based on the Bayesian Monte Carlo (BMC) method, a novel double-loop Bayesian Monte Carlo (DLBMC) estimator is proposed for evaluating the EIG in this work. To result in an efficient overall optimization procedure to find the maximizer of the EIG, a Bayesian optimization (BO) procedure for EIG is developed. In addition, our analysis gives explicit expressions of the mean estimate of the BMC estimator and the bounds of its variance for the uniform and the normal distributions. Detailed numerical studies show that our DLBMC method can provide accurate mean estimates with small variances, and the overall BO procedure leads to optimal designs which give efficient Bayesian inference results.

As our novel DLBMC estimator for EIG is based on Gaussian process, it is currently limited to low-dimensional problems where the number of design variables is not large. In this work, the classical BO and MCMC approaches are used, and it is not straight forward to apply them for high-dimensional problems. For high-dimensional Bayesian optimal design problems with a large number of design variables, a possible solution is to conduct a sequential design procedure, and apply DLBMC at each step in the sequential procedure. Conducting a systematic DLBMC based on the sequential design will be the focus of our future work.