DL-AWI: Adaptive Full Waveform Inversion Using a Deep Twin Neural Network

Abstract

Full waveform inversion (FWI) iteratively improves the accuracy of the model by minimizing the discrepancies between the predicted and the observed data. However, FWI commonly suffers from cycle skipping when the initial model is poor, leading to an erroneous result. To mitigate this problem, we propose deep-learning-backed adaptive waveform inversion (DL-AWI), which introduces a deep twin neural network to precondition the waveforms and compare the ratio of two signals with a zero-lag spike, thereby enhancing the stability of the inversion process. DL-AWI can project the synthetic and observed signals into an extended latent space via several convolutional neural networks (CNNs) with shared weights, which can accelerate the data matching. Compared with classic FWI methods, the proposed DL-AWI provides a wider space for model updates, significantly decreasing the risk of being trapped in local minima. We use synthetic and field examples to validate its efficiency in subsurface model inversion, and the results show that DL-AWI is robust even when a poor initial model is provided.

1. Introduction

Full waveform inversion (FWI) is playing an increasingly important role in the field of seismic exploration due to its high-resolution superiority of subsurface model reconstruction [1,2,3,4]. Stopin et al. [5] conducted FWI based on wide-azimuth data for improved velocity, which helps imaging of complex geological structures. Wang et al. [6] incorporated generative models into the FWI process to improve the resolution of the final results. Taufik et al. [7] developed a learned regularization via diffusion model for elastic FWI, reaching improved data fitting and model accuracy. Moreover, Silva et al. [8] introduced a receiver-extension scheme for time-lapse FWI to address non-repeatable issues (e.g., noise), thereby providing more accurate time-lapse models.

However, although FWI, as well as its recent developments, has provided satisfactory results, FWI is highly nonlinear, and the commonly used L2-norm-based objective function may mislead the inversion toward a local minimum when a good initial model is not available [9,10,11,12,13], which is related to the cycle skipping phenomenon. Numerous scholars have proposed various schemes to address this issue. First, low-frequency components benefit FWI as they are less sensitive to errors associated with the initial model [14]. So, Fang et al. [15] used a CNN to map high-frequency signals to their low-frequency version and recovered the missing low-frequency information for FWI. Wang et al. [16] put forward a method to predict the missing low-frequency components based on a dense convolutional network used to mitigate the non-convexity of FWI. Second, many scholars have dedicated themselves to developing good initial models through traveltime tomography, migration velocity analysis, or incorporating prior information, such as well log data [17,18,19,20]. Nevertheless, creating a robust initial model is a demanding task, and it will inevitably require extra computational cost. Third, several researchers derived more convex objective functions as alternatives to alleviate cycle skipping. Warner and GUasch [21] proposed adaptive waveform inversion (AWI) by penalizing the deconvolution of the predictions and the observations to be a zero-lag spike, and Li et al. [22] extended AWI to multiparameter inversion. Additionally, Yang et al. [23] applied the concept of optimal transport and the quadratic Wasserstein metric to quantify data differences during FWI, thereby improving robustness and noise resistance. Next, Wang et al. [13] combined Wasserstein-1 and Fourier metrics to create a high-resolution velocity model, which prevented the inversion process from falling into local minima. Moreover, Li and Alkhalifah [24] formulated a matching-filter-based FWI to extend the search space for better convexity and convergence properties of the objective function. Fourth, as an extended version of FWI, envelope inversion (EI) [25,26] and wavefield reconstruction inversion (WRI) [27,28] are able to stabilize the inversion process when the initial model is not located within the convergence region of the global minimum.

Recently, with the fast development of computing power, deep learning (DL) is widely used in seismic denoising [29,30], interpolation [31,32], inversion [33,34], earthquake monitoring [35], and so on. Attracted by the strong capability of DL in feature extraction and nonlinear mapping, many scholars have proposed to integrate DL into FWI to enhance the quality of the recovered models [36,37]. For instance, Yang et al. [38] utilized a specific U-Net to project shot gathers directly to their corresponding velocity model, and the learned network produced promising results at a low cost. Sun and Alkhalifah [39] proposed an improved machine learning (ML)-based optimization scheme for FWI and validated its effectiveness. Dhara and Sen [40] developed a physics-guided FWI that can map shot gathers to reliable velocity model directly, avoiding the requirement of a good initial model in classic FWI. Additionally, Alali and Alkhalifah [41] conducted multi-scale FWI assisted by a Unet to better recover the salt body in FWI, and Ren et al. [42] incorporated a self-supervised method for seismic forward prospecting in tunnels, which can improve the final image quality and guide underground engineering excavation. Furthermore, numerous scholars have considered incorporating DL to formulate a new FWI misfit function for improved accuracy and robustness [43]. Yang and Ma [44] integrated generative adversarial networks (GANs) into FWI in an unsupervised manner and obtained promising inversion results. Saad et al. [45,46] proposed SiameseFWI, an extended full waveform inversion framework based on a Siamese network that enhances data fitting and improves both final accuracy and robustness. Nevertheless, although SiameseFWI can achieve a better result when the initial model is poor, it completely depends on the network to implement enhanced data matching and requires more iterations.

In this paper, we propose an improved ML-based misfit by introducing a deep neural network to precondition the predicted and the observed data for better data fitting, namely DL-AWI. DL-AWI is nonstationary and incorporates a deep twin network with several CNN layers to extract the non-stationary data characteristics, and utilizes local matching to address the complexity of amplitude and phase variations in seismic data. However, a global and stationary matching filter used in classic AWI cannot cope with the varying amplitude and phase difference of seismic data [47]. Compared to classic AWI, the network can balance the energy difference during waveform matching in an extended latent space, improving the convergence rate. Furthermore, compared to SiameseFWI, the proposed DL-AWI has significantly improved initial model tolerance and provides a wider convergence region. DL-AWI can converge to the global minimum more efficiently and adapt to some extreme conditions (e.g., a simple initial model with constant layers).

In detail, the deep neural network mainly contains two CNN branches to extract representative characterizations of input data progressively to construct a latent space for data comparison. The predicted and observed data will first be mapped onto an extended latent space to highlight the preferred feature maps, and then deconvolution will be conducted between the estimated feature maps, trace by trace. Next, we use the Euclidean loss function to measure the misfit between the deconvolution result and a zero-lag spike, which can be exploited to update the network and velocity model simultaneously until the process reaches the predefined maximum number of iterations. More importantly, we integrate the data mapping and model update processes into a flexible automatic differentiation (AD) framework, thereby avoiding the derivation of an adjoint source in the latent space. Compared with classic FWI, classic AWI, and SiameseFWI, DL-AWI can significantly reduce the dependence on the initial model and provide reliable results more efficiently when cycle skipping occurs. The remainder of this paper is organized as follows. In the second section, we will first review the theory of classic AWI and then introduce the mathematical formulation of the proposed DL-AWI. Next, we will use synthetic and field data examples to validate the superiority of DL-AWI.

2. Methodology

2.1. Adaptive Waveform Inversion

In this section, we begin by reviewing the reverse formulation of adaptive waveform inversion (AWI) [21], which is developed to mitigate cycle skipping. Classic FWI aims to minimize the data residual between the synthesized data $\mathbf{p}(v,t)$ and the observed data $\mathbf{d}\left(t\right)$ in the least square loss, and this widely used misfit function for acoustic FWI can be formulated as

$${\mathbf{J}}_{\mathrm{FWI}}={\displaystyle \frac{1}{2}}{∥\mathbf{p}(v,t)-\mathbf{d}\left(t\right)∥}_2^2,$$

(1)

where $v_{\mathrm{curr}}$ corresponds to the current velocity model. Similar to Equation (1), the objective function of AWI has the following form:

$${\mathbf{J}}_{\mathrm{AWI}}={\displaystyle \frac{1}{2}}{\displaystyle \frac{{∥\mathbf{T}\mathbf{w}\left(t\right)∥}_2^2}{{∥\mathbf{w}\left(t\right)∥}_2^2}},$$

(2)

$$\mathbf{D}\left(t\right)\mathbf{w}\left(t\right)=\mathbf{p}(v,t),$$

(3)

where $\mathbf{D}\left(t\right)$ is a Toeplitz matrix constructed by $\mathbf{d}\left(t\right)$ and $\mathbf{T}$ is a weight function that monotonically increases away from zero time lag and is represented here by a matrix. In other words, this optimization problem gradually transforms the deconvolution result $\mathbf{w}\left(t\right)$ to a zero-lag delta-like function.

Notably, governed by the acoustic wave equation, $\mathbf{p}(v,t)$ is the signals of the wavefield $u\left(\mathbf{x},t,v\right)$ located at the receivers. Alternatively, we can rewrite Equation (3) as

$$\mathbf{w}\left(t\right)=argmin{\displaystyle \frac{1}{2}}{∥\mathbf{D}\left(t\right)\mathbf{w}\left(t\right)-\mathbf{p}(v,t)∥}_2^2,$$

(4)

$$\mathbf{A}\left(v\right)u\left(\mathbf{x},t,v\right)=s\left(t\right),$$

(5)

where operator $\mathbf{R}$ represents the process of extracting the wavefield at each receiver location, $s\left(t\right)$ is the source wavelet, and $\mathbf{A}\left(v\right)=\left({\displaystyle \frac{\partial }{t^2}}-v^2{\nabla }^2\right)$ is the wavefield propagation operator. According to the theory of the adjoint state method [48], the gradient of Equation (2) with respect to velocity has the following expression:

$${\nabla }_v{\mathbf{J}}_{\mathrm{AWI}}=\sum _t{u}_{\mathrm{for}}\left(\mathbf{x},t,v\right){\displaystyle \frac{\partial \mathbf{A}\left(v\right)}{\partial v}}{u}_{\mathrm{bac}}\left(\mathbf{x},t,v\right),$$

(6)

$$\mathbf{A}\left(v\right)u_{\mathrm{for}}\left(\mathbf{x},t,v\right)=s\left(t\right),$$

(7)

$$\mathbf{A}\left(v\right)u_{\mathrm{bac}}\left(\mathbf{x},t,v\right)=a\left(t\right),$$

(8)

$$a\left(t\right)={\displaystyle \frac{\partial {\mathbf{J}}_{\mathrm{AWI}}}{\partial \mathbf{p}\left(v,t\right)}}=\mathbf{D}\left(t\right){\left({\mathbf{D}}^T\left(t\right)\mathbf{D}\left(t\right)\right)}^{-1}\left({\displaystyle \frac{{\mathbf{T}}^2-2{\mathbf{J}}_{\mathrm{AWI}}\mathbf{I}}{{\mathbf{w}}^T\left(t\right)\mathbf{w}\left(t\right)}}\right)\mathbf{w}\left(t\right),$$

(9)

where $\mathbf{I}$ is an identity matrix and $u_{\mathrm{for}}\left(\mathbf{x},t,v_{\mathrm{curr}}\right)$ and $u_{\mathrm{bac}}\left(\mathbf{x},t,v_{\mathrm{curr}}\right)$ are the wavefield generated by source wavelet $s\left(t\right)$ and adjoint source wavelet $a\left(t\right)$, respectively [21]. Then, the model update process is summarized as

$$v_{\mathrm{k}+1}=v_{\mathrm{k}}-\beta {\nabla }_v{\mathbf{J}}_{\mathrm{AWI}},$$

(10)

where k is the iteration and $\beta$ stands for step length.

2.2. Deep-Learning-Assisted AWI

Although AWI performs well when the initial model is poor, AWI does not consider the local nature of the seismic data, which may lead to suboptimal inversion results [47]. From Equation (9), we can find that AWI does not, by nature, address the discrepancy between simulated and real data. In view of this, we propose a deep-learning-based AWI (DL-AWI). DL-AWI can first learn to extract feature maps from the predicted and observed data using a CNN, thereby highlighting more local information for better data fitting. Furthermore, DL-AWI is embedded into a flexible AD framework, which not only simplifies and facilitates the calculation of the adjoint source for each iteration [45,49] but also provides the opportunity to filter features that cannot be matched between simulated and observed data.

Equations (2) and (3) show that the main concept of AWI is to update $\mathbf{w}\left(t\right)$ toward a zero-lag delta function $\delta \left(t\right)$. Consequently, we can derive an equivalent objective function of AWI as

$${\mathbf{J}}_{\mathrm{AWI}}\left(v\right)={\displaystyle \frac{1}{2}}{∥\mathbf{w}(v,t)-\delta \left(t\right)∥}_2^2.$$

(11)

Based on Equation (11), we can formulate a modified objective function ${\widehat{\mathbf{J}}}_{\mathrm{AWI}}$ in an extended latent space via CNN feature extraction, where

$${\widehat{\mathbf{J}}}_{\mathrm{AWI}}(v,\theta )={\displaystyle \frac{1}{2}}{∥\widehat{\mathbf{w}}(v,\theta ,t)-\delta \left(t\right)∥}_2^2,$$

(12)

$$\widehat{\mathbf{w}}(v,\theta ,t)=argmin{\displaystyle \frac{1}{2}}{∥\mathrm{Toep}\left(\mathbf{E}\left(\mathbf{d}\left(t\right),\theta \right)\right)\widehat{\mathbf{w}}\left(t\right)-\mathbf{E}\left(\mathbf{p}(v,t),\theta \right)∥}_2^2,$$

(13)

where $\mathrm{Toep}(•)$ aims to construct a Toeplitz matrix based on a given vector, $\mathbf{E}(•)$ represents the CNN-based feature extraction operator, $\theta$ is the network parameters, and $\widehat{\mathbf{w}}(v,\theta ,t)$ is the matching filter. Compared with Equation (3), we can find that Equation (13) aims to first transfer data into an extended latent space and match the extracted features trace by trace instead of data itself via a filter $\widehat{\mathbf{w}}\left(t\right)$. The CNN will focus more on local information due to the small size of the convolution kernel, resulting in improved data fitting. Notably, based on Equations (12) and (13), the velocity and network parameter $\theta$ are automatically updated via the AD framework. Due to the mathematical equivalence between AD and the adjoint state method [50], we can derive the adjoint source of DL-AWI $\widehat{a}\left(t\right)$ used for velocity update with the following form for intuitive illustration:

$$\widehat{a}\left(t\right)={\displaystyle \frac{\partial {\widehat{\mathbf{J}}}_{\mathrm{AWI}}}{\partial \widehat{\mathbf{w}}\left(t\right)}}{\displaystyle \frac{\partial \widehat{\mathbf{w}}\left(t\right)}{\partial \mathbf{E}(\mathbf{p},\theta )}}{\displaystyle \frac{\partial \mathbf{E}(\mathbf{p},\theta )}{\partial \mathbf{p}}}={\left({\mathbf{G}}^T\mathbf{G}\right)}^{-1}{\mathbf{G}}^T\left(\widehat{\mathbf{w}}(v,\theta ,t)-\delta \left(t\right)\right){\mathbf{E}}_{\mathbf{p}},$$

(14)

where $\mathbf{G}=\mathrm{Toep}\left(\mathbf{E}\left(\mathbf{d}\left(t\right),\theta \right)\right)$ and ${\mathbf{E}}_{\mathbf{p}}$ is the derivative of $\mathbf{E}(\mathbf{p},\theta )$ with respect to $\mathbf{p}$. Note that the extended adjoint source $\widehat{a}\left(t\right)$ used for the velocity update is also related to the neurons. The updated neurons help to balance the amplitude and phase difference and accelerate the convergence rate of the inversion process. Compared to Equation (9), Equation (14) can focus more on local waveform difference with the help of CNN kernels, which can more effectively deal with the non-stationary amplitude and phase variation. Classic AWI applies matching-filter normalization to the adjoint source, which may lead to the risk of ignoring non-stationary amplitude variations during inversion.

In detail, Figure 1 shows the workflow of the inversion process. First, the synthesized data $\mathbf{p}(v_{\mathrm{curr}},t)$ and observed data $\mathbf{d}(v_{\mathrm{real}},t)$ are input into the CNN blocks with shared weights to create feature maps ${\mathbf{p}}_{\mathrm{FM}}$ and ${\mathbf{d}}_{\mathrm{FM}}$ trace by trace in a latent space. Subsequently, a deconvolution-based loss function is used to update the velocity and network parameters simultaneously. Finally, using the updated velocity, we generate new predicted data and input the new predicted data and observed data into CNN blocks to obtain the refined feature maps. All steps are performed based on PyTorch 2.9.0 and the deepwave toolbox 0.0.20 [51].

Figure 2 displays the detailed architecture of the CNN blocks. The architecture is relatively simple. We do not use any up- or down-sampling layers, so the extracted feature maps have the same size as the original input data. Reading from top to bottom, the number of extracted feature maps of each CNN layer is 1, 2, 2, 4, 4, 2, 1, and 1, respectively, and the convolution kernel size in each CNN layer is set to 3 × 3. Given an input vector $\mathbf{s}\in {\mathrm{\Omega }}^{N_b\times N_t\times N_x}$, the input will first go through the main branch with eight consecutive CNN layers connected by addition operators, where $N_b$, $N_t$, and $N_x$ represent the batch size, the number of time samples, and the number of traces. Furthermore, eight additional CNN layers are connected to the output of each CNN layer in the main branch via skip connections. The summation of the output of each CNN layer in the main branch and the output of the corresponding skip connection will be fed into the next CNN layer. Mathematically, the output of the first seven CNN layers ${\mathbf{y}}_{\mathrm{i}}$ (e.g., $i\in 1-7$) can be expressed as

$${\mathbf{y}}_{\mathrm{i}}=\sigma \left({\mathbf{W}}_{\mathrm{i}}^{\mathrm{m}}{\mathbf{y}}_{\mathrm{i}-1}\right)+\sigma \left({\mathbf{W}}_{\mathrm{i}}^{\mathrm{k}}\mathbf{s}\right),$$

(15)

where ${\mathbf{W}}_{\mathrm{i}}^{\mathrm{m}}$ and ${\mathbf{W}}_{\mathrm{i}}^{\mathrm{k}}$ are the weight matrix of the i-th CNN layers in the main branch and the corresponding skip connection. $\sigma$ is the LeakyReLU activation function. Specifically, ${\mathbf{y}}_1=\sigma \left({\mathbf{W}}_1^{\mathrm{m}}\mathbf{s}\right)+\sigma \left({\mathbf{W}}_1^{\mathrm{k}}\mathbf{s}\right)$ and the final output of the CNN block ${\mathbf{y}}_8$ can be written as

$${\mathbf{y}}_8=\gamma \left({\mathbf{W}}_8^{\mathrm{m}}{\mathbf{y}}_7\right)+\gamma \left({\mathbf{W}}_8^{\mathrm{k}}\mathbf{s}\right)+\mathbf{s},$$

(16)

where $\gamma$ stands for the linear activation function. Equation (16) represents an improved data-fitting process, where the first two terms beginning with $\gamma$ are the data-driven shaping constraints. Note that Equations (15) and (16) follow a ResNet-like residual learning paradigm and can mitigate the gradient vanishing problem. Meanwhile, the extra shaping constraints can counteract those components that are challenging to match after fine-tuning, leading to model updates and final inversion results.

3. Results

In this section, to demonstrate the superiority of the proposed DL-AWI, we will apply the proposed DL-AWI to two synthetic examples and a field example and compare it with the classic FWI method, classic AWI method [21], and the SiameseFWI [45], which are the state-of-the-art FWI algorithms. Specifically, SiameseFWI adopts Euclidean distance to build the misfit function. Classic AWI and DL-AWI use a deconvolution-based misfit function for inversion. All the experiments are conducted using an NVIDIA RTX 6000 GPU, and we use the Adam method to optimize the network and velocity model.

3.1. Marmousi Model

We first apply the proposed DL-AWI to the Marmousi model to evaluate the performance. The size of the velocity model is 141 × 681 with a sampling interval of 25 m, and we uniformly ignite 136 shots at the surface using a 5 Hz Ricker wavelet. The effective bandwidth range is approximately 2 to 8 Hz. Figure 3a,b show the real and initial models used for inversion, where we can see that the initial model is poor, which can result in cycle skipping if FWI is performed directly. During the inversion process, the number of iterations is set to 60, and the learning rates for the velocity update and network update are 5 and 0.0001, respectively.

Figure 4 displays the inverted results using classic FWI, classic AWI, SiameseFWI, and DL-AWI. From a poor initial model, we find that the results in Figure 4a,c show more artifacts due to the effect of cycle skipping (e.g., highlighted by the red arrows), which demonstrates that classic FWI and SiameseFWI fail to provide reliable results. Figure 4b,d are the results of classic AWI and DL-AWI, where we can see that, compared with the real model, the inverted results are promising, indicating the robustness of classic AWI and DL-AWI. However, it is noticeable that the result of DL-AWI is cleaner than that of classic AWI. The result of DL-AWI displays more details (e.g., highlighted by the red arrows). Figure 5 compares the shot gathers generated using the real model, the initial model, and the inverted model. Note that from Figure 5b, we can see that the diving waves are mismatched at the far offset, indicating that the initial model is poor. Figure 5c–f show the interleaved shot generated by the real model and the inverted results using different methods. Obviously, at the far offset, we can see that the generated shot in Figure 5f shows an improved continuity and similarity (e.g., marked by red arrow), demonstrating that the inverted result of DL-AWI is more reliable. Figure 5g–j show the final data residual corresponding to different benchmarks, where we can see that DL-AWI can lead to a lower mean squared error (MSE), which means that DL-AWI can converge faster under the same conditions. To provide an intuitive comparison, we extract two vertical profiles from Figure 4 at distances of 4000 m and 10,000 m, and the results are displayed in Figure 6. It is apparent that although the accuracy of DL-AWI decreases at greater depths compared with the ground truth (magenta line), its performance (black line) remains visually superior to that of the other benchmark methods. We use the green arrows to highlight the obvious improvements, demonstrating the superiority of the proposed DL-AWI.

Moreover, we also use the signal-to-noise ratio (SNR) and structure similarity index measure (SSIM) to quantify the inverted results. Figure 7 depicts the SNR and SSIM variation curves of the Marmousi model with an increasing number of iterations. In detail, from Figure 7a, we can find that the increasing rate of SSIM curves of classic FWI and SiameseFWI will decrease gradually, which means that it is challenging for classic FWI and SiameseFWI to update the current model toward the real model. As for the SSIM curves of classic AWI and DL-AWI, we observe that they increase as expected, and DL-AWI accelerates model updates for the same number of iterations, validating its improved efficiency. Figure 7b is the SNR variation curve. We find that the SNR variations corresponding to classic FWI and SiameseFWI are small because, when trapped in local minima, they cannot progressively correct the gradient error. Furthermore, we observe that the curves for classic AWI and DL-AWI increase. However, with the help of the deep twin network, DL-AWI can benefit from data fitting during data matching after fine-tuning, providing improved results.

3.2. Overthrust Model

We then use the SEG/EAGE overthrust model to compare the efficacy of the proposed DL-AWI with benchmarks. We uniformly generate 80 shots at the surface with a 5 Hz Ricker wavelet because a 5 Hz wavelet contributes to the robustness of the inversion process, and we will discuss the choice of wavelet in the discussion section. The model size is 94 × 401 with an interval of 30 m, and the other parameters are unchanged. Figure 8a shows the real model, and Figure 8b is the initial model used in the experiment. The poor initial model will cause cycle skipping. Figure 9 shows the inverted results using different methods. Comparing classic FWI, classic AWI, and SiameseFWI with DL-AWI, it is obvious that the result of DL-AWI shows improved accuracy and resolution. Due to the poor initial model, although classic FWI and SiameseFWI cannot recover the real model, we find that SiameseFWI can suppress some artifacts during the inversion process to some extent with the help of the deep neural network. Figure 9b,d show the inverted results of classic AWI and DL-AWI, respectively. It is apparent that the result shown in Figure 9d is more accurate, especially at the edge and deep positions (e.g., highlighted by the black arrows). The result of AWI shows obviously abnormal lower values at the bottom left corner, and such a phenomenon may be caused by the complexity of globally matching using AWI. As for DL-AWI, it will match the mapped predictions and the observations in an extended latent space, where more attention will be paid to local signal features. Figure 10 displays the generated shots using the real model, the initial model, and the inverted model corresponding to different benchmarks. From Figure 10b, we can see that the shot waveform at the far offset is mismatched, indicating that the initial model is poor. Figure 10c–f compare the real shot with the predicted shot created using the inversion results of different methods. In particular, we can find that the shot in Figure 10f shows improved similarity. Figure 10g,j depict the final data residual corresponding to classic FWI, classic AWI, SiameseFWI, and DL-AWI, respectively. It is obvious that the data residual shown in Figure 10j is the lowest, indicating that DL-AWI provides a more accurate inversion result.

Figure 11 shows the extracted vertical profiles at different positions from Figure 9 for further comparison. Compared with the real model, we can easily find that the result of DL-AWI is superior in terms of accuracy in both Figure 11a,b. Classic FWI and SiameseFWI are trapped in the local minima, and we can see that the inverted velocity models (cyan and red lines) are far away from the real model (magenta line). Additionally, although classic AWI performs better than classic FWI and SiameseFWI, it results in more errors at a depth of 2200 m than DL-AWI as shown in Figure 11b, which demonstrates that DL-AWI is more effective and robust under the same circumstances. Figure 12 shows the SSIM and SNR curves corresponding to the second synthetic example. SSIM considers changes in local structure information, and SNR mainly focuses on the amplitude fidelity. Both of them can evaluate the accuracy of the inverted results from different perspectives. Obviously, although the SSIMs corresponding to classic FWI and SiameseFWI increase, the final results are not accurate due to the cycle skipping problem. In contrast, classic AWI and DL-AWI are both effective for recovering the velocity model and, notably, DL-AWI exhibits a higher convergence rate than classic AWI. As for the SNR curves shown in Figure 12b, we can see that classic FWI and SiameseFWI lead to inferior results with lower SNRs. Compared to classic AWI, DL-AWI offers a better velocity model, and more iterations are required for classic AWI to achieve the same performance. Both the SNR and SSIM validate the great efficiency and robustness of DL-AWI.

3.3. Field Data

In this section, we use a field example from the South China Sea to evaluate the effectiveness of DL-AWI. The field data comprises 50 shots with an interval of approximately 100 m. Each shot gather contains 360 traces, and the offset ranges from 0.0875 km to 4.5875 km. Figure 13 shows the original shot gathers, the initial velocity model, and the corresponding source wavelet. Then, we apply a bandpass filter to the original dataset to extract the signals ranging from 5 to 10 Hz for inversion. During the experiment, we also keep the learning rates for velocity update and network update unchanged and perform inversion for 15 iterations.

Figure 14 shows the inverted velocity model using classic FWI, classic AWI, SiameseFWI, and DL-AWI, respectively. Compared with the initial model, the background velocity values shown in Figure 14b,d decrease. The velocity values in Figure 14a,c increase slightly. To evaluate the accuracy of the inverted velocity models in Figure 14, we generate synthetic diving waves using the inverted models and compare them with the real diving waves. Figure 15 shows the results of the diving wave comparison. Figure 15a compares the real and synthetic diving wave after classic FWI, where we can see an obvious mismatch at the far offset (e.g., highlighted by the red ellipse). Due to the effect of cycle skipping, classic FWI cannot provide an accurate result. Figure 15b–d display the real and synthetic diving waves after classic AWI, SiameseFWI, and DL-AWI. By careful observation, we can see that the time difference between real and synthetic diving waves is completely corrected by DL-AWI, demonstrating its efficacy and robustness. Although classic AWI and DL-AWI can mitigate the mismatch, DL-AWI is more efficient with the same number of iterations, thereby reducing the computational cost of the inversion process.

Next, we conduct Kirchhoff prestack depth migration (PSDM) using the inverted velocity models to evaluate the contribution of the velocity model to the image. Figure 16 displays the migration profiles corresponding to velocity models obtained by different methods. Comparing Figure 16d with Figure 16a–c, it is obvious that the velocity obtained by DL-AWI can lead to a better visual quality with improved resolution (e.g., highlighted by the red arrows). Notably, compared with Figure 16a, although we can see that the image quality improves in Figure 16b,c at the right corner, the ability of classic AWI and SiameseFWI to alleviate cycle skipping is inferior and less efficient than that of DL-AWI. Figure 17 shows the extracted offset domain common image gather (ODCIG) at different locations. It is intuitive that the events in Figure 17d are almost flattened, validating the reliability of the inverted velocity using DL-AWI. Compared with Figure 17a, we find that the events in Figure 17b,c are still curving, which demonstrates the suboptimal efficacies of classic AWI and SiameseFWI when dealing with cycle skipping.

4. Discussion

In this section, we discuss the robustness of the proposed DL-AWI and provide some caveats for its practical applications. All the experiments are based on the Overthrust model shown in Figure 8.

4.1. Dependence on Initial Models

First of all, we analyze the sensitivity of DL-AWI to initial models. We create a fairly poor initial model for inversion, as shown in Figure 18a, which is a simple layered constant-velocity model with velocities of 3500 m/s and 5000 m/s. We conduct DL-AWI for 60 iterations and keep the other parameters unchanged. Figure 18b shows the final inversion result based on Figure 18a. Surprisingly, the inversion result is satisfactory. To quantify the inverted result, the SNR and SSIM corresponding to Figure 18b are 19.33 dB and 0.66, respectively. Note that although the accuracy is relatively low at some bottom positions, the inversion validates that DL-AWI can significantly mitigate the strong dependence of FWI on a good initial model and demonstrates the potential to obtain reliable results under extreme conditions.

4.2. The Influence of Low-Frequency Components and Bandwidth

Moreover, we will analyze the influences of low-frequency components and frequency bandwidth on the proposed DL-AWI because they are important factors. To evaluate the influence of low-frequency components, we first filtered the components below 3 Hz in the wavelet, and the corresponding waveform and spectrum are displayed in Figure 19a,b. Based on the initial model shown in Figure 8b, Figure 19c shows the final inversion result inverted using a filtered 5 Hz Ricker wavelet, where we can see that DL-AWI is robust and can also provide reliable results when the low-frequency components are missing. Then we use an 8 Hz wavelet and filter those components below 3 Hz. The corresponding wavelet and spectrum are shown in Figure 19d,e. Figure 19f displays the final inversion result using the filtered 8 Hz wavelet, where we can see some obvious artifacts (e.g., marked by the black arrow) due to the poor initial model. In reality, we suggest using a multiscale scheme to conduct inversion. We can begin with low frequencies (e.g., 3 Hz) and gradually add higher-frequency components to improve accuracy and resolution. At the early inversion stage, incorporating more high-frequency components may yield inferior results. Moreover, Figure 20 shows the spectrum of the inverted velocity shown in Figure 9c,d. The inversion results are obtained using a 5 Hz wavelet with and without low-frequency components. It is clear that the spectrum matches with each other, indicating that the proposed DL-AWI can recover low-wavenumber components for the velocity model and is less dependent on the low-frequency components in the raw seismic data.

4.3. Noise Level Influence

Additionally, the noise level can also influence the final performance of DL-AWI. Based on the Overthrust model, we generate noisy data with different noise levels (e.g., 5 dB and 2 dB) and conduct inversion to assess the effects of noise based on Figure 8b. Figure 21 shows the noisy data and the corresponding inversion results, where the final model SNR and SSIM of the inverted results are 24.01 dB, 23.32 dB, 0.55, and 0.52, respectively. Intuitively, compared to the reference model shown in Figure 9d, we can recognize that although the accuracy and resolution of the final inversion result will degrade with increasing noise level (for example, marked by black arrows), the proposed DL-AWI is also able to suppress artifacts related to the poor initial model.

4.4. Computational Cost Analysis

In addition, we compare the computational costs of classic FWI, classic AWI, SiameseFWI, and DL-AWI. Based on the inversion results shown in Figure 9, the corresponding computational times are 321 s, 377 s, 319 s, and 324 s, respectively. Notably, compared with classic FWI and AWI, the proposed DL-AWI incurs only a small additional computational burden. Currently, all the experiments in the manuscript are based on 2D data. Regarding 3D FWI, we expect the computational cost to increase, and we will make addressing this issue a major focus of our future research.

4.5. Ablation Tests and Architecture Analysis

Next, we will perform ablation experiments to evaluate the architecture of the deep neural network. In our first ablation experiment, we remove all the skip connections in the CNN blocks and conduct the inversion process with the other parameters unchanged. Figure 22a shows the final inverted result, where we can see that the inverted result is less accurate and contains more artifacts. The final SNR and SSIM of Figure 22a are 18.3 dB and 0.21, respectively. We believe that skip connections are helpful and necessary for DL-AWI because the network weights are randomly initialized, which cannot guarantee that the network will build a wide latent space that includes the global convergence region. The additional skip connections can provide sufficient original information to mitigate instability in the early stages of the inversion process [32]. Furthermore, we also assess the influence of the number of CNN layers used for inversion. Based on Figure 2, we change the number of CNN layers to 5 and 10 and reconduct the inversion. Figure 22b,c are the inverted results, and the corresponding SNR and SSIM are 19.2 dB and 0.32 and 24.6 dB and 0.64, respectively. Obviously, fewer CNN layers may prevent the network from extracting the most relevant signals for data matching, leading to suboptimal results. Moreover, compared with the result shown in Figure 9d, increasing the number of CNN layers does not yield visible improvements. In most cases, eight CNN layers are suitable in terms of accuracy and computational cost. Regarding the architectural design, more complex models (e.g., Transformer-based architectures) could potentially achieve better global data fitting. However, in practice, it is not necessary to completely eliminate the mismatch between the predicted and observed data. Instead, it suffices to construct a simple, compact filter to correct phase differences exceeding half a cycle, which is sufficient to ensure the stability of the inversion. Although a Transformer-based model may achieve comparable performance, FWI is already computationally expensive, and adopting such architectures would further increase the computational burden.

4.6. Interpretation of the Inversion Mechanism

The principle of the deep twin network in the DL-AWI algorithm is to extract the most distinct components in the synthetic and observed data for misfit calculation so that the iterative inversion process will only focus on the most dominant wave components (e.g., direct and refracted waves) in early iterations. Later on, as the gradient-based inversion process (left part in Figure 1) recovers more velocity details, the synthetic data will contain more complicated wave components (e.g., diving and reflected waves). The deep twin network will further extract the more complicated wave components for misfit calculation [45]. This strategy is analogous to the multi-scale inversion scheme, which has been demonstrated to overcome cycle-skipping issues, albeit to a limited extent, in a completely data-driven, neural-network-guided manner. This approach can also be understood as a concatenated traveltime tomography and full waveform inversion framework, all through the inclusion of convolutional neural networks. By including more complex wave components across epochs, the proposed DL-AWI gradually recovers the lowest-wavenumber velocity model and then the higher-wavenumber velocity model components. Similar ideas have also been explored in the global seismology community by designing the time windows to extract the most distinct components over the iterations [52]. However, here, we leverage CNNs for adaptively choosing these “time windows” as well as correcting their time shifts and amplitude errors.

In detail, Figure 23 shows the original predicted and observed data and their extended version in an interleaved manner. Figure 23a–d show the original observed and predicted data with an increasing number of iterations, where we can see obvious waveform differences in terms of energy and phase. Specifically, Figure 23e–g show the extended observed and predicted data in the latent domain with the increasing number of iterations. Unlike the original data, we can see that the amplitude discrepancy decreases after network mapping. The network provides a robust environment for data matching by balancing the amplitude difference between the predicted and the observed data, facilitating the process of data matching. More importantly, the balanced data can effectively contribute to the accelerated convergence rate of the deconvolution-based objective function. During the whole inversion process, we can see that DL-AWI can deal with amplitude and phase mismatches simultaneously. Classic AWI and SiameseFWI solely rely on either network mapping or a more convex objective function to escape from local minima, leading to suboptimal performance and efficiency on waveform fitting.

The two network branches share the same weight for the purpose of only extracting similar patterns from both synthetic and observed data, instead of paying special attention to one component and ignoring others for both network branches. The network training process (right-hand side of Figure 1) aims to produce optimally fitted waveforms that highlight only the gradually simulated wave components during normal FWI. In this sense, it is comparable to the widely used matching filtering approach in FWI [10,24] and the dynamic time warping (DTW)-based approaches [53,54], but it is significantly different from all these existing methods by leveraging CNNs to match not only the time difference caused by velocity errors but also the amplitude difference caused by visco-elastic or physically unexplainable effects. The proposed method is built on top of the recently proposed SiameseFWI [45] approach, which leverages similar ideas for matching synthetic and observed waveforms. However, DL-AWI extends the SiameseFWI idea by substituting the common absolute-error-based objective function with the more adaptive deconvolution-based objective function [21].

5. Conclusions

In this paper, we propose a deep-learning-based framework for adaptive waveform inversion, referred to as DL-AWI. The proposed DL-AWI incorporates a deep twin network with shared weights to map predictions and observations into an extended latent space, thereby mitigating the nonlinearity of FWI when a poor initial model is provided. Through iterations, the twin network recovers more nonlinear components and helps to precondition the data for misfit calculation. DL-AWI can use deep-learning-based regularization to counteract signals that are difficult to match, leading to improved data fitting and inversion results. Moreover, we integrate the entire DL-AWI process into an AD framework to make it more flexible and automatic, thereby significantly simplifying the complex mathematical derivations in FWI. Compared with classic FWI, classic AWI, and SiameseFWI, both synthetic and field examples demonstrate that DL-AWI can not only achieve superior performance when cycle skipping occurs but also offer better results with improved accuracy and efficiency.