MCMC Method of Inverse Problems Using a Neural Network—Application in GPR Crosshole Full Waveform Inversion: A Numerical Simulation Study

Abstract

Ground-penetrating radar (GPR) crosshole tomography is widely applied to subsurface media images. However, the inadequacies of ray methods may limit the resolution of crosshole radar images, since the ray method is a type of high-frequency approximation. To solve this problem, the full waveform method is introduced for GPR inversion. However, full waveform inversion is computationally expensive. In this paper, we introduce a trained neural network that can be evaluated very quickly to replace a computationally intensive forward model. Additionally, the forward error of the trained neural network can be statistically analyzed. We demonstrate a methodology for a full waveform inversion of crosshole ground-penetrating radar data using the Markov chain Monte Carlo (MCMC) method. An accurate forward model based on Maxwell’s equations is replaced by a quickly trained neural network. This method achieves a high computation efficiency, which is four orders of magnitude faster than the accurate forward model. The inversion result of the synthetic waveform data shows a good performance of the trained neural network, which greatly improves the calculation efficiency.

1. Introduction

Ground-penetrating radar (GPR) crosshole tomography is widely used to map subsurface electrical properties, e.g., electrical conductivity and dielectric permittivity, which are important in investigating environments [1,2,3,4]. A GPR crosshole experiment includes transmitting radar antennas and receiving antennas that are distributed in the opposite borehole. Transmitting antennas (20 MHz to 1 GHz) generate high-frequency electromagnetic energy to acquire crosshole radar data.

Inversion refers to using physical phenomena observed on the earth’s surface to infer the spatial changes of the physical structure of the underground medium. To solve the inversion problem, a number of inversion methods have evolved over the past two decades, including deterministic inversion algorithms and probabilistic inversion [5]. Compared with deterministic inversion algorithms [6,7,8], probabilistic inversion methods do not need an accurate a priori model and can use any a priori known information, regardless of its complexity. The result of the probabilistic inversion is the a posteriori probability density [9]. The model variability of posterior samples can reflect the information contained in the posterior probability density, which can solve more sophisticated questions [10]. To solve the highly nonlinear inversion problem, Mosegaard et al. proposed the extended Metropolis algorithm to sample the a posteriori probability density [11]. Instead of an explicit mathematical expression of a priori information, the extended Metropolis algorithm only needs a “black box” algorithm to sample the a priori probability. A sequential Gibbs sampler is proposed to serve as a black box algorithm to sample a priori information defined by any geostatistical algorithm [12,13]. In this way, sampling the a posteriori probability density becomes very flexible [14,15,16,17].

Ray tomographic methods are generally based on ray theory, which uses only a small portion of signal information, limiting the resolution of crosshole radar images [18,19]. To solve this problem, a full waveform method is introduced for GPR inversion. Ernst successfully used full waveform data from a GPR crosshole to invert electrical conductivity and dielectric permittivity [20]. Klotzsche et al. characterized a gravel aquifer by the full waveform crosshole ground-penetrating radar data [21]. Cordua obtained the first a posteriori probability density of full waveform GPR crosshole data [22]. Hansen implemented an efficient Monte Carlo sampling method using a neural network to simulate travel time to replace the ray methods [23]. However, the problem of the huge computational costs of full waveform inversion has not been solved.

In this paper, to obtain fast full waveform inversion, we introduce the trained neural network forward model into the MCMC method of GPR crosshole data. The inverse problem refers to obtaining dielectric permittivity through full waveform GPR crosshole data. We replace a numerical complex forward model based on the 2D FDTD solution of Maxwell’s equations with a two-layer feedforward network. Taking advantage of the fast mapping of a two-layer feedforward network, this MCMC inverts four orders of magnitude faster than the 2D FDTD solution of Maxwell’s equations.

This paper is organized as follows: First, we introduce probability inversion using the MCMC method, including its theories and formulas. Second, we present the forward method based on the 2D FDTD solution of Maxwell’s equations. Then, we briefly introduce the two-layer feedforward network. Finally, we use the synthetic FWI data from a ground-penetrating radar crosshole to analyze the error and efficiency of the trained neural network.

2. Methods

2.1. Probabilistically Formulated Inversion

For geophysical inverse problems, the subsurface can be described by a set of parameters $m$, and the observed data can be represented by a dataset $d$. Hence, the forward relation between $m$ and $d$ is as follows [5]:

$$d=f\left(m\right)$$

(1)

where the function $f$ is a mapping operator solved by a physical relation. The corresponding inverse problem can be expressed as follows:

$$m=f^{-1}\left(d\right)$$

(2)

In this formula, the inverse operator $f^{-1}$ is difficult to obtain because it is non-trivial or does not exist. Furthermore, the forward relation is an approximation to the correct physical law. In this paper, relative dielectric permittivity refers to the subsurface parameters $m$ that we want to invert, and the observed data are the waveform data of the GPR crosshole.

For probabilistic inversion, the corresponding inversion formula is as follows [5]:

$${\sigma }_M\left(m\right)=k{\rho }_M\left(m\right)L\left(m\right)$$

(3)

where ${\rho }_M\left(m\right)$ is the a priori probability density, which represents the a priori information of the model parameters. The solution of the inverse problem is the a posteriori probability distribution ${\sigma }_M\left(m\right)$. $k$ is a normalization factor, and $L\left(m\right)$ is the likelihood function. $L\left(m\right)$ is a probabilistic measure that describes how well the modeled data explain the observed data. Its general formula is given by the following:

$$L\left(m\right)=\int dd\frac{{\rho }_D\left(g\left(m\right)\right)\theta \left(d|m\right)}{{\mu }_D\left(d\right)}$$

(4)

where ${\rho }_D\left(g\left(m\right)\right)$ represents measurement uncertainties that are mainly related to the recording instrument. $\theta (d|m)$ represents the model error caused by the imperfect forward method defective parameterization. ${\mu }_D\left(d\right)$ describes the homogeneous state of information, which ensures parameterization is invariant when the coordinate system changes. In most cases, ${\mu }_D\left(d\right)$ can be assumed to be constant.

2.2. Extended Metropolis Algorithm

For non-linear and non-Gaussian problems, obtaining the mathematical formula of the a priori probability density is difficult. The extended Metropolis algorithm can solve this difficulty using sequential Gibbs sampling to directly sample ${\rho }_M\left(m\right)$. To perform the extended Metropolis algorithm, we used a sequential Gibbs sampler as a part of the extended Metropolis algorithm to directly sample ${\rho }_M\left(m\right)$. The extended Metropolis algorithm is performed as follows [14]:

• Start in the current model, $m_{\mathrm{cur}}$, which constitutes the a priori probability density. Using the sequential Gibbs sampler to sample $m_{\mathrm{cur}}$, a new model candidate $m_{\mathrm{pro}}$ is generated.
• Calculate the probability of accepting the proposed model using the ratio of likelihood function: $P_{acc}=min(1,\frac{L{m}_{\mathrm{pro}}}{L{m}_{\mathrm{cur}}})$
• If $m_{\mathrm{pro}}$ is accepted, use $m_{\mathrm{pro}}$ instead of $m_{\mathrm{cur}}$. Consequently, the proposed model takes the place of the current model: $m_{\mathrm{cur}}$ = $m_{\mathrm{pro}}$. Otherwise, the random walker staying at a location in $m_{\mathrm{cur}}$ and $m_{\mathrm{cur}}$ is counted again.

2.3. Forward Model Based on Waveform

For the forward model, several methods can be selected to simulate the GPR signal’s wavefield. In this paper, we chose the 2D FDTD solution of Maxwell’s equations in transverse electric mode. This FDTD method simulates the vertical component of the electrical field as the observed data. For electromagnetic wave propagation, the transverse electric or TE modes of Maxwell’s equations in the (x, z) plane of Cartesian coordinates are as follows [24]:

$$\frac{\partial E_x}{\partial t}=\frac{1}{\epsilon }\left(-\frac{\partial H_y}{\partial z}-\sigma E_x\right)$$

(5)

$$\frac{\partial E_z}{\partial t}=\frac{1}{\epsilon }\left(\frac{\partial H_y}{\partial x}-\sigma E_x\right)$$

(6)

$$\frac{\partial H_y}{\partial t}=\frac{1}{\mu }\left(\frac{\partial E_z}{\partial x}-\frac{\partial E_x}{\partial z}\right)$$

(7)

where $E_x$ and $E_z$ represent the horizontal and vertical components of the electric field, respectively; $\epsilon$ is the dielectric permittivity; $\sigma$ is the electrical conductivity; $H_y$ is the magnetic field perpendicular to the propagation plane; and $\mu$ represents the magnetic permeability, which is equivalent to the free-space permeability assumed to be constant in the following.

We used the FDTD techniques [24,25] based on the staggered grid finite difference operator with second-order accuracy in space and time to solve Equations (5)–(7). A generalized perfectly matched layer (GPML) surrounding the edges of the FDTD grid absorbs the artificial reflections at the edges of the model space. Near the surface, electromagnetic signals are mainly affected by dielectric permittivity and electrical conductivity. Permittivity primarily controls the phases, and conductivity primarily controls the amplitudes of the observed signals. In this paper, we consider only the influence of the dielectric permittivity. For all examples presented in this paper, we inverted the permittivity while keeping the conductivities fixed [8].

2.4. Solving the forward Problem Using a Trained Neural Network

Trained neural networks are methods that infer a non-linear mapping operator from one data set to another data set. For the inversion problem, neural networks can be applied to simulate the forward operator and the inverse operator [26,27,28,29]. In this paper, we used neural networks to replace the computationally expensive forward model in Equations (5)–(7). The neural networks can evaluate the forward problem quickly. However, this also leads to some modeling errors. We used a Gaussian probability distribution to quantify the modeling error $\theta (d|m)$ caused by the trained neural network. In this paper, we used a two-layer feedforward neural network containing a hidden layer [30]. It should be noted that we can use any regression-type network. The main reason we chose the two-layer feedforward network is that, once trained, the mapping of the network on a new model is potentially very fast and meets our requirement for efficiency. The mathematical expression of this network is as follows:

$$d_k=h_1\left({\displaystyle \sum }_j^{\mathrm{NH}}{w}_{jk}^2{h}_2\left({\displaystyle \sum }_i^{\mathrm{NM}}{w}_{ij}^1{m}_i\right)\right)$$

$d_k=\left[d_1,d_2,\dots ,d_{\mathrm{ND}}\right]$ represents each of the observed data $d$, and ND is the number of observed data parameters. $m_i=\left[m_1,m_2,\dots ,m_{\mathrm{ND}}\right]$ represents each of the model parameters. NM represents the number of parameters in the model.

NH represents the number of hidden layers. $h_1$ and $h_2$ are two sigmoidal types of activation functions. $w_{ij}^1$ refers to the weights of the units in the first layer, and $w_{jk}^2$ refers to the weights of the units in the second layer. If we have a sufficient training set and enough units of the hidden layer, we can replace any continuous mapping. Figure 1 shows the one hidden-layer network architecture that we used.

For probabilistically formulated inverse problems, a great quantity of training data can be easily generated from the a priori distribution ${\rho }_M\left(\mathbf{m}\right)$. The accuracy of the network is highly dependent on the size of the training data set NT. We can create an arbitrarily large sample of the a priori M and simulate the observational data D using the forward model. If we want to train the network needs NT times, the M and D should consist of NT samples.

For all models in M, the corresponding simulated observational data are simulated by the exact, but expensive, forward model based on the FDTD solution of Maxwell’s equations, $f_{\mathrm{FWI}}$. The neural network training result, $f_{\mathrm{nn}}$, provides an approximation of $f_{\mathrm{FWI}}$. In other words, using $f_{\mathrm{nn}}$ leads to a modeling error. To evaluate the training effect, we need the model error. To estimate the forward model error between $f_{\mathrm{FWI}}$ and $f_{\mathrm{nn}}$, we can analyze Nr samples from the a priori model.

First, we use the a priori distribution to generate Nr realizations in $M*=\left[m_1,m_2,\dots ,m_{\mathrm{Nr}}\right]$. We use both $f_{\mathrm{FWI}}$ and $f_{\mathrm{nn}}$ to compute the observed data set $D_{\mathrm{FWI}}$ and $D_{\mathrm{nn}}$ of every model in $M*$. Second, we obtain a sample of the modeling error $\theta (d|m)$ using $D_{\mathsf{\theta }} = D_{\mathrm{FWI}} - D_{\mathrm{nn}}$ and use the method of Hansen et al. to obtain a Gaussian model of modeling error [31].

3. Experimental Validation

For the synthetic example, we simulated a reference model $m$ of size 5.2 × 12 m (26 × 60 = 1560 pixels of size 0.2 m × 0.2 m). This model is created by a multivariate Gaussian probability distribution, with the mean of the relative dielectric permittivity ${\epsilon }_r$ being 4.0 and the variance being 0.75. In this reference model, the electrical conductivity is assumed to be a constant value of 3 mS/m. There is a relatively high dielectric permittivity structure located at a depth of about 4 m, which is the main feature of our inversion.

To obtain the observed data, we designed a recording geometry of the ground-penetrating radar (GPR) cross borehole. Figure 2 shows a recording geometry that contains four transmitters, and each borehole contains two (the transmitters are represented by red crosses, one at 3 m and one at 9 m). The receivers are represented by black dots at equally spaced intervals of 1.5 m in the two boreholes. Due to the effects of wave guiding [32], the travel path is not at the center of the antenna, but at the tips of the antenna [33]. Data where the angle between a transmitter–receiver and the horizontal is larger than 45° are omitted. This decision was made with a similar reasoning to the parameters inferred in Cordua using the same geometrical setup [22].

The FDTD algorithm is used to calculate a full waveform synthetic dataset. The FDTD model for solving Maxwell’s equations uses a regular grid consisting of square elements of 0.1 m × 0.1 m, and the number of boundary elements is 40. In the synthetic data, we assumed that the source pulse is known, which was a ricker wavelet, and that the central frequency is 100 MHz. For the field data, it should be noted that the source wavelet needs to be estimated. Some methods can estimate the source wavelet [34,35].

The recorded synthetic observational data are the vertical components of the electrical field. They contain a total of 20 waveform traces. We added Gaussian-distributed data uncertainties to the waveform data. In Figure 3, we present five waveform traces generated by a transmitter at a depth of 3 m in the left borehole. The signal-to-noise ratios from the top are 8, 15, 20, 30, and 40, respectively. The blue dotted curves represent noise-free waveforms, and the red curves represent noisy waveforms. As can be seen in Figure 3, the matching of the waveforms from top to bottom gradually becomes better between the noise-free waveforms and the noisy waveforms.

For this reference model and observed data, we set the size of the input layer NM as 1560. The output layer ND was 460. The number of hidden units NH was 100. The accuracy of the network was very heavily affected by the size of the training data set NT. We generated four training data sets whose training sizes were NT = [1000, 5000, 10,000, 20,000] to analyze the training effect of the neural network. The corresponding four forward models refer to $f_{1000}$, $f_{5000}$, $f_{\mathrm{10,000}}$, and $f_{\mathrm{20,000}}$. Before the inversion, we needed to obtain NT models as unconditional realizations of the a priori distribution and the observed data computed by the forward model $f_{\mathrm{FWI}}$ described previously. Figure 4 shows five different sampling models generated from the a priori probability density.

The extended Metropolis algorithm was used to obtain the a posteriori probability density. Figure 5 shows four groups of inversion results based on the four neural network forward models. The synthetic GPR shows that the trained neural network can obtain the main features of the inverse models, even when using the neural network forward model whose training size is 5000. Clearly, the inversion models contain features that appear to be consistent with those in the reference model. The relatively high dielectric permittivity structures are located at a depth of about 4 m. This proves the effectiveness of the MCMC inversion based on the trained neural network. To test the training effect, we used the observed waveform created by the FDTD method to compare the simulated waveform of the four neural networks. We selected trace 1 and trace 5 of the top left borehole, corresponding to the minimum signal-to-noise ratios trace and the maximum signal-to-noise ratios trace, as Figure 6 shows. There are some obvious distortion points in the simulated waveforms created by $f_{1000}$ and $f_{5000}$. The simulated waveforms created by $f_{\mathrm{20,000}}$ are more similar to the observed waveforms.

To analyze the forward modeling error of the neural network-based forward, we created a 1D distribution of the modeling error, which was created by 6000 realizations, in Figure 7. Clearly, the magnitude of modeling error using the trained neural network decreases as training times increases. Figure 8 shows the covariance model of four forward methods based on the trained neural network. $f_{1000}$ leads to a more correlated covariance model compared with $f_{5000}$, $f_{\mathrm{10,000}}$, and $f_{\mathrm{20,000}}$.

Table 1. Computation time analysis of different forward models.

Forward Model	${\mathit{f}}_{\mathrm{FWI}}$	${\mathit{f}}_{1000}$	${\mathit{f}}_{5000}$	${\mathit{f}}_{\mathrm{10,000}}$	${\mathit{f}}_{\mathrm{20,000}}$
Calculation Time (s)	32,164	3.6994	3.1000	3.3001	3.1995

lists the computation time of the forward model. We tested the forward model 10,000 times in the extended metropolis algorithm to record the computation time. For the four forward models based on a neural network, the running time of the extended Metropolis algorithms are similar, and all the step sizes tend to be stable after 3000 iterations. Table 1 shows that, compared with the full waveform forward model $f_{\mathrm{FWI}}$, the forward model based on our trained neural network is more than four orders of magnitude faster. This is a significant improvement in computation efficiency.

4. Conclusions

This paper replaces a computationally complex full waveform forward model with a fast forward model based on a trained neural network, which, when combined with the MCMC method, solves the inversion problem. Although the neural network of the minimum training time, 1000, cannot reflect the correct features of the reference model, the neural network whose training size is 5000 contains features that are consistent with the reference model. There are some obvious distortion points in the simulated waveform created by the neural network with less training times. When the training times reach 10,000, the distortion disappears clearly in the simulated waveform. The error analysis shows that the inversion effect becomes better when the size of the training times is increased. Compared with the full waveform forward model based on the 2D FDTD solution of Maxwell’s equations, our trained neural network is more than four orders of magnitude faster. The results demonstrate that trained neural networks can be used to describe a full waveform forward model, which is an obvious advantage in application. In the future, we will research how to apply this method to some complex models and try to apply it to field data.