Automatic Horizon Picking Using Multiple Seismic Attributes and Markov Decision Process

Abstract

Picking the reflection horizon is an important step in velocity inversion and seismic interpretation. Manual picking is time-consuming and no longer suitable for current large-scale seismic data processing. Automatic algorithms using different seismic attributes such as instantaneous phase or dip attributes have been proposed. However, the computed attributes are usually inaccurate near discontinuities. The waveforms in the horizontal direction often change dramatically, which makes it difficult to track a horizon using the similarity of attributes. In this paper, we propose a novel method for automatic horizon picking using multiple seismic attributes and the Markov decision process (MDP). For the design of the MDP model, the decision time and state are defined as the horizontal and vertical spatial position on a seismic image, respectively. The reward function is defined in multi-dimensional feature attribute space. Multiple attributes can highlight different aspects of a seismic image and therefore overcome the limitations of the single-attribute MDP through the cross-constraint of multiple attributes. The optimal decision is made by searching the largest state value function in the reward function space. By considering cumulative reward, the lateral continuity of a seismic image can be effectively considered, and the impacts of abnormal waveform changes or bad traces in local areas for automatic horizon picking can be effectively avoided. An effective implementation scheme is designed for picking multiple reflection horizons. The proposed method has been successfully tested on both synthetic and field data.

1. Introduction

In exploration seismology, the reflection horizons in imaging volumes play an important role in velocity inversion and seismic interpretation. For velocity inversion, the reasonable application of the reflection horizon can effectively improve the convergence and obtain a geologically plausible velocity model. For seismic interpretation, the main reflection horizon can determine the relationship between geological layers and improve the reliability and accuracy of seismic interpretation. Therefore, automatically and efficiently picking the horizons from a complex seismic image is very important.

There are two main methods for tracking horizons. The first method is to identify and extract the horizon position using various signal or image processing methods based on various seismic attributes. For example, we can use the amplitude attribute [1], phase attribute [2,3,4,5,6], waveform attribute [7,8,9], and slope or dip attribute [10,11,12,13,14,15,16,17] to track the horizon. Furthermore, multiple attributes can be combined to constrain the picking results [18,19,20,21,22,23], such as using the instantaneous phase and dip attributes [18,22] and using the envelope, phase attributes, and texture analysis [23]. In addition, there are also various tracking algorithms for horizon picking. Admasu and Tönnie [24] used the correlation of horizons across faults in 3D seismic data, and Zou et al. [9] tracked the horizon based on the waveform similarity. Bondár [25] used the edge detection algorithm to detect seismic horizons. Aurnhammer and Tonnies [26] used a genetic algorithm for automated horizon correlation across faults in seismic images. Many global methods [27,28,29] or the flattening method [12,13,14,30,31,32,33,34] are also widely used for automated horizon picking.

The second method is mainly based on the Machine Learning (ML) algorithm to intelligently track the horizon; it mainly includes supervised learning methods with artificial neural networks [35,36,37,38,39], convolutional neural networks [40,41,42,43,44,45], and unsupervised learning methods with self-organizing networks [46] and convolutional auto-encoding networks [47].

In the first method, conventional horizon-picking methods depend heavily on the seismic attribute. When the attribute changes in the horizontal direction are relatively dramatic, it is difficult to track the correct horizon position. With the help of big data and ML methods, we can learn the structural information of the seismic image according to the data’s statistical characteristics. However, supervised learning methods need a lot of training data and labels, and the results obtained using unsupervised network algorithms are often unreliable.

If the problem of automatic horizon picking is regarded as an optimal decision-making process, a kind of ML method named reinforcement learning, the core algorithm of which is based on the Markov decision process (MDP), can be used to replace manual picking by imitating human behavior. The MDP based on the Markov chain can perceive the state change in the system environment and simulate humans to make a decision. Wald [48] adopted the MDP to solve the sequential analysis problem. Shapley [49] introduced a Markov decision process when studying random games. Bellman [50] put forward the discrete-time MDP (Bellman equation) for dynamic programming, which established a complete methodology of MDP. With the improvement of the MDP theory and method [51,52,53], the MDP method has been widely used in sequential decision making [54,55,56], dynamic programming [53,57], and reinforcement learning [58,59]. In the exploration seismology, Ma et al. [60] introduced reinforcement learning for selecting the first arrival. Luo et al. [61] designed a Markov decision process for first break picking.

In this paper, we formulate the horizon picking of seismic images as a Markov decision process (MDP) with multiple seismic attributes. The paper is organized as follows: Firstly, this paper briefly reviews the basic concepts and algorithms of the MDP. Then, the finite-stage MDP is designed for automatic horizon picking. For the MDP, the decision time is defined as the horizontal spatial position, and the state is defined as the vertical spatial position on the seismic image. The reward function is defined in multi-dimensional feature attribute space including the waveform attribute, instantaneous phase attribute, envelope attribute, and extremum attribute. When the seed point is determined, we can perceive the state change and take some actions to obtain reward value, where the selected action depends on the dip attribute. Therefore, we can make the optimal decision for horizon picking through constant trial and error and interaction with the environment. Moreover, for picking multiple reflection horizons, we design an effective implementation scheme using multiple seismic attributes and MDP. Finally, synthetic and field data are used to verify the effectiveness of the proposed method.

2. Review of the Markov Decision Process

2.1. 5-Tuple <$s,a,\mu ,p,r$>

The Markov decision process (MDP) mainly includes the following elements: state ($s$), action ($a$), policy ($\mu$), transition probability ($p$), and reward function ($r$).

--State: The state $s_i^t$ is defined as one of the system characteristics at the decision-making time $t$. If a state does not change with time, the state becomes $s_i$, called a stable state. State space $S$ is defined as a finite set $S=\left\{s_1,s_2,\dots ,s_M\right\}$, where $M$ is the size of state space.

--Action: The action $a$ is the response of a system to the state $s$. For a certain state $s_i$, the set of all possible actions can be expressed as $A_{s_i}=\left\{a_1\left(s_i\right),a_2\left(s_i\right),\dots ,a_n\left(s_i\right)\right\}$. The action space of a system is defined as the union composed of action sets under all states $A=U_{s_i\in s}=\left\{A_{s_1},A_{s_i},\dots ,A_{s_M}\right\}=\left\{a_1,a_2,\dots ,a_N\right\}$, where $N$ is the size of action space.

--Policy: The mapping process from state to action is called policy $\mu :s\to a\left(s\right)$. The policy depends on the whole decision process, and the sequence formed by the mapping of state to action at all times is called a policy $\pi$:

$$\pi =\left({\mu }_{i,k}^0,{\mu }_{i,k}^1,{\mu }_{i,k}^2,\dots ,{\mu }_{i,k}^{t-1},{\mu }_{i,k}^t,\dots ,{\mu }_{i,k}^{T-1},{\mu }_{i,k}^T\right),$$

(1)

where ${\mu }_{i,k}^t=\mu \left(s_i^t,a_k^t\right)$ is the probability of mapping state $s_i^t$ to action $a_k^t$. The policy space is defined as a finite set $\Pi =\left\{{\pi }_1,{\pi }_2,\dots ,{\pi }_L\right\}$, where $L$ is the size of the policy space.

--Transition probability: When performing an action $a_k^t\in A$ from one state $s_i^t\in S$ at the time $t$, the system makes a transition to a new state $s_j^{t+1}\in S$, where the transition is based on a probability distribution $p\left(s_j^{t+1}|s_i^t,a_k^t\right):s\times a\times s\to \left[0,1\right]$. The transition probability of the system from the state $s_i^t$ to the state $s_j^{t+1}$ is:

$$p\left(s_j^{t+1}|s_i^t\right)={\displaystyle {\sum }_{a_k^t}\mu \left(s_i^t,a_k^t\right)p\left(s_j^{t+1}|s_i^t,a_k^t\right)}.$$

(2)

According to Markov property, the state of a system at the time $t+1$ only depends on the state at the time $t$ and is independent of the previous state:

$$p\left(s^{t+1}|s^1,s^2,\dots ,s^t\right)=p\left(s^{t+1}|s^t\right).$$

(3)

--Reward Function: The reward function $r\left(s_i^t,a_k^t\right)$ is defined as $R:s\times a\to ℝ$,

$$r\left(s_i^t,a_k^t\right)={\displaystyle {\sum }_{s_j^{t+1}}p\left(s_j^{t+1}|s_i^t,a_k^t\right)r\left(s_j^{t+1}|s_i^t,a_k^t\right)},$$

(4)

where $r\left(s_j^{t+1}|s_i^t,a_k^t\right)$ is the reward function when the system executes action $a_k^t$ under the state $s_i^t$ at the time $t$, then transfers to the state $s_j^{t+1}$. The reward function is only related to the current state and action and has nothing to do with the previous state and action.

2.2. The Markov Decision Process

By integrating the above elements, we can define the MDP model, which can be viewed in Figure 1. This process can be completely described as: at the time $t$, the system is in the state $s_i$, and the decision-maker executes action $a_k$ and generates reward value $r\left(s_i^t,a_k^t\right)$ and then transfers to the next state $s_j^{t+1}$ with probability function $p\left(s_j^{t+1}|s_i^t,a_k^t\right)$. The model forms the complete MDP time series at all times, which can be expressed as:

$$\tau =\left(s^0,a^0,r^1;\dots ;s^{t-1},a^{t-1},r^t;s^t,a^t,r^{t+1};\dots ;s^T,a^T,r^{T+1}\right).$$

(5)

Taking a decision (with an action) according to policy ${\mu }_{i,k}^t$ from the state $s_i^t$ at the time $t$, the system can obtain a reward sequence at the subsequent time,

$$\left\{R^{t+1},R^{t+2},\dots ,R^T\right\},$$

(6)

where $R^{t+1}=r\left(s_i^t,a_k^t\right)$ is the immediate reward function. The overall expectation from the immediate reward function $R^{t+1}$ to the future reward function sequence $R^T$ with the policy $\pi$ is defined as value function $v_{\pi }\left(s^t\right)$:

$$v_{\pi }\left(s^t\right)=E_{\pi }\left[{\displaystyle \sum _{k=0}^T{\gamma }^k}{R}^{t+k+1}|s^t\right], \forall s^t\in S 0<\gamma <1,$$

(7)

where $E_{\pi }$ is the expectation and $\gamma$ is the discount factor, and $0<\gamma <1$, which means that the further the reward is from the decision-making time, the smaller the impact on the value function. Based on the Markov property, Formula (7) can be written as:

$$v_{\pi }\left(s^t\right)={\displaystyle \sum _{a^t\in A}\mu \left(s^t,a^t\right)\left(r\left(s^t,a^t\right)+\gamma {\displaystyle \sum _{s^{t+1}\in S}p\left(s^{t+1}|s^t,a^t\right)v_{\pi }\left(s^{t+1}\right)}\right)}.$$

(8)

This equation is the so-called Bellman equation [50] or the state transition equation. Based on the Bellman equation, the state value function at the current time can be calculated by the state value function of the next decision time.

The value function is used to evaluate how “good” a decision-maker is in a given state [59]. The goal of the MDP model is to find the best policy that can maximize the value function. Therefore, an optimal policy ${\pi }_{\ast }$ is defined as $v_{{\pi }_{\ast }}\left(s^t\right)\ge v_{\pi }\left(s^t\right)$ for all $s^t\in S$ and all policies $\pi \in \Pi$. The optimal solution $v_{\ast }\left(s^t\right)=\underset{\pi }{max} v_{\pi }\left(s^t\right)$ satisfies the following Bellman optimality equation:

$$v_{\ast }\left(s^t\right)=\underset{a^t\in A}{max}\left\{r\left(s^t,a^t\right)+\gamma {\displaystyle \sum _{s^{t+1}\in S}{p}_{\pi }\left(s^{t+1}|s^t\right)v_{\ast }\left(s^{t+1}\right)}\right\}.$$

(9)

Once the MDP model is determined, an appropriate and effective solution method can be selected to obtain the optimal policy, for example, the value iteration method [50], the policy iteration method [51], the Q learning algorithm [58], or the SARSA [62].

3. Formulation of Horizon Picking with MDP

For the problem of automatic horizon picking on seismic images, we can regard the seed point of one reflection horizon as the initial state and take some action policies to track the horizon. Based on the reward function of the current state (vertical position) and action, we can find an optimal policy to transfer to a new state at the next decision time (horizontal position). The state and transition probability distribution only depend on the current position and have nothing to do with the state and action of the previous position. Therefore, we regard the automatic horizon picking on the seismic image as an intelligent decision-making problem in a random environment which can be modeled as a discrete MDP with finite stages.

The horizon-picking model with finite-stage MDP is defined as follows:

(1) Decision time

The decision time $t\in T=\left\{1,2,\dots ,nx\right\}$ is defined as the horizontal spatial position where the vertical and horizontal grid number is $nz$ and $nx$, respectively, for a 2D seismic image.

According to the position of the seed point, the decision-making time can be divided into three categories:

(a) The starting time is at the left endpoint $t_0=1$, and the ending time is at the right endpoint $t_n=nx$. The decision time sequence is $T=\left\{1,2,\dots ,nx-1,nx\right\}$.

(b) The starting time is at the right endpoint $t_0=nx$, and the ending time is at the left endpoint $t_n=1$. The decision time sequence is $T=\left\{nx,nx-1,\dots ,2,1\right\}$.

(c) The starting time is at the middle position $t_0=i$. The decision time is divided into two parts: one is to track to the right endpoint $t_n=nx$(consistent with mode a); the second is to track to the left endpoint $t_n=1$(consistent with mode b).

(2) State

The state is defined as the vertical spatial position on a seismic image. The set of states is represented as $S=\left[1,\dots ,nz\right]$. A seismic image often contains multiple horizons. When we pick one of them, the rest are irrelevant. The farther away an image is from the current axis in the vertical direction, the less influence it has on the decision. Moreover, we reduce the state space to $S=\left[z_0^t-dz,\dots ,z_0^t+dz\right]$, where $dz$ is the width. $z_0^t$ is the middle position of the state space, which is determined by the conventional picking method. Particularly, $z_0^0$ is the seed point. The reduction in state space can not only weaken the impact of another horizon, but also improve computational efficiency.

(3) Action

The action is defined as a movement from a lateral position $x_i$ to a lateral position $x_{i+1}$, $a\left(s\right):x_i\to x_{i+1}$.

(4) Policy

We adopt the stochastic policy to realize the mapping from state to action under the constraint of the dip attribute

$${\mu }^t\left(s,a\right)=e^{-\frac{{(a-a_0)}^2}{{\sigma }^2}}, a_0=s+p\raisebox{1ex}{$dz$}\!\left/ \!\raisebox{-1ex}{$dx$}\right.,$$

(10)

where $p$ is the dip at the position of the state $s$, and $dx$ and $dz$ are the sampling intervals in the lateral and vertical directions, respectively. $\sigma$ is the variance in the Gaussian probability distribution. Figure 2 shows the mapping probability diagram from state to action. Following the dip direction, the probability of action is high. Staying away from the dip direction, the probability of action slowly becomes smaller. When the state position changes, the corresponding action probability density function also changes.

(5) Transition probability

For state $s_i^t\in S$ and action $a_k^t\in A$ at time $t$, the state $s_j^{t+1}\in S$ at time $t+1$ is relatively independent. The transition probability is defined as

$$p\left(s_j^{t+1}|s_i^t,a_k^t\right)=1.$$

(11)

Therefore, the transition probability of the system from the state $s_i^t$ to the state $s_j^{t+1}$ is:

$$p\left(s_j^{t+1}|s_i^t\right)={\displaystyle {\sum }_{a_k^t}\mu \left(s_i^t,a_k^t\right)}.$$

(12)

(6) Reward function

In the MDP, the optimal state is found according to the value function. However, the value function depends on the reward function. The reward function is the most important part of the MDP, which implicitly defines the learning objective [59]. In this paper, we use multiple seismic attributes of the seismic image to define the reward function.

Based on Formula (4) and (11), the relationship between the reward function $r\left(s_i^t,s_j^{t+1}\right)$ and $r^t\left(s_j^{t+1}|s_i^t,a_k^t\right)$ is as follows:

$$r\left(s_i^t,s_j^{t+1}\right)=r\left(s_j^{t+1}|s_i^t,a_k^t\right).$$

(13)

Therefore, the reward function is defined as

$$r\left(s_i^t,s_j^{t+1}\right)={\displaystyle \sum _{f\in F}^{}{\alpha }_f{r}_f\left(s_i^t,s_j^{t+1}\right)} s.t. {\displaystyle \sum _{f\in F}^{}{\alpha }_f}=1,$$

(14)

$$r_f\left(s_i^t,s_j^{t+1}\right)=R_f\times F\left(\mathsf{\Delta }f\right),$$

(15)

where $\mathsf{\Delta }f={‖f\left(s_i\right)-f\left(s_j\right)‖}^2$ is the distance between the state $s_i$ and $s_j$ for the attribute $f$. The operator $F\left(•\right)$ is a measure of attribute difference. $R_f$ is the maximum reward value, $r_f\left(s_i^t,s_j^{t+1}\right)$ is the reward value, and ${\alpha }_f$ is the weight coefficient of the attribute $f$. The weighting coefficient is determined by the Fuzzy C-Means (FCM) clustering method, where the result is obtained by using the conventional horizon-picking method with a single attribute.

In this paper, the following attributes are selected to calculate the reward function:

(a) The waveform similarity for $r_{waveform}\left(s_i^t,s_j^{t+1}\right)$:

$$\begin{array}{l}{r}_{waveform}\left(s_i^t,s_j^{t+1}\right)=R_{waveform}\times \overline{\gamma }\left(s_i^t,s_j^{t+1}\right)\\ \left\{\begin{array}{l}\overline{\gamma }=\frac{{\displaystyle {\sum }_{k=1}^n{\gamma }_k}}{n}\\ {\gamma }_k=\frac{〈s^t\left(z\right),s^{t+1}\left(z\right)〉}{\sqrt{〈s^t\left(z\right),s^t\left(z\right)〉}\sqrt{〈s^{t+1}\left(z\right),s^{t+1}\left(z\right)〉}}\\ 〈s^t\left(z\right),s^{t+1}\left(z\right)〉={\displaystyle {\int }_{-\mathsf{\Delta }z-k}^{\mathsf{\Delta }z+k}{s}^t\left(z\right)s^{t+1}\left(z\right)dz}\end{array}\right.\end{array}$$

(16)

where $\mathsf{\Delta }z$ is the size of the local space window, and ${\gamma }_k$ is the similarity coefficient.

(b) The extremum attribute for $r_{et}\left(s_i^t,s_j^{t+1}\right)$:

$$r_{et}\left(s_i^t,s_j^{t+1}\right)=\left\{\begin{array}{l}1 , if f\left(s_i^t\right) is \mathrm{extremum}\\ 0 , else\end{array}\right.$$

(17)

where the attribute $f$ includes the waveform attribute, instantaneous phase attribute, and envelope attribute.

The instantaneous attributes of the seismic image can be obtained by applying a Hilbert transform to generate complex signals. For a 2-D seismic image $I\left(z,x\right)$, the $I\left(z,x\right)$ is treated like a real part of the complex signals, and the imaginary part $\tilde{I}\left(z,x\right)$ can be obtained after the Hilbert transformation. The complex signals $\widehat{I}\left(z,x\right)$ are:

$$\widehat{I}\left(z,x\right)=I\left(z,x\right)+i\tilde{I}\left(z,x\right).$$

(18)

Then, the envelope $e\left(z,x\right)$ and the instantaneous phase $\theta \left(z,x\right)$ can be obtained by

$$e\left(z,x\right)=\sqrt{I^2\left(z,x\right)+{\tilde{I}}^2\left(z,x\right)}.$$

(19)

$$\theta \left(z,x\right)=arctan\left(\frac{\tilde{I}\left(z,x\right)}{I\left(z,x\right)}\right).$$

(20)

However, the range of the instantaneous phase solved by Equation (20) has discontinuity. Based on the relationship between the seismic image and its transformation result:

$$\left\{\begin{array}{l}I\left(z,x\right)=e\left(z,x\right)cos\left(\theta \left(z,x\right)\right)\\ \tilde{I}\left(z,x\right)=e\left(z,x\right)sin\left(\theta \left(z,x\right)\right)\end{array}\right..$$

(21)

The instantaneous phase can also be represented as

$$cos\left(\theta \left(z,x\right)\right)=\frac{I\left(z,x\right)}{e\left(z,x\right)}.$$

(22)

Furthermore, we use the least squares inverse division [63] to avoid the instability caused by zero division:

$$cos\left(\theta \left(z,x\right)\right)=\arg min{‖I\left(z,x\right)-cos\left(\theta \left(z,x\right)\right)e\left(z,x\right)‖}^2.$$

(23)

Finally, the instantaneous phase is calculated as

$$cos\left(\theta \left(z,x\right)\right)=\frac{I\left(z,x\right)e\left(z,x\right)}{e\left(z,x\right)e\left(z,x\right)+\epsilon },$$

(24)

where $\epsilon$ is a threshold coefficient.

Then, we use the structure tensor method to calculate the dip attribute. Considering the waveform oscillation of seismic image, the structure tensor is obtained using the envelope attribute:

$$G\left(z,x\right)=\nabla e\left(z,x\right)•\nabla e^t\left(z,x\right)=\left(\begin{array}{c}{e}_x^2\left(z,x\right) e_x\left(z,x\right)e_z\left(z,x\right)\\ e_x\left(z,x\right)e_z\left(z,x\right) e_z^2\left(z,x\right)\end{array}\right),$$

(25)

where $\nabla e$ represents the gradient of the envelope, and $e_x$ and $e_z$ are the directional derivatives in the $x$ direction and $z$ direction, respectively.

The eigenvalues and eigenvectors of the structure tensor matrix are obtained by eigenvalue decomposition of the structure tensor matrix:

$$\left\{\begin{array}{l}{\lambda }_1\left(z,x\right)=\frac{1}{2}\left(e_x^2\left(z,x\right)+e_z^2\left(z,x\right)\right)+\frac{1}{2}\sqrt{{\left(e_x^2\left(z,x\right)-e_z^2\left(z,x\right)\right)}^2+4{\left(e_x\left(z,x\right)e_z\left(z,x\right)\right)}^2}\\ {\lambda }_2\left(z,x\right)=\frac{1}{2}\left(e_x^2\left(z,x\right)+e_z^2\left(z,x\right)\right)-\frac{1}{2}\sqrt{{\left(e_x^2\left(z,x\right)-e_z^2\left(z,x\right)\right)}^2+4{\left(e_x\left(z,x\right)e_z\left(z,x\right)\right)}^2}\end{array}\right.$$

(26)

$$\left\{\begin{array}{l}{v}_1\left(z,x\right)=\left[\begin{array}{c}2e_x\left(z,x\right)e_z\left(z,x\right)\\ e_z^2\left(z,x\right)-e_x^2\left(z,x\right)+\sqrt{{(e_x^2\left(z,x\right)-e_z^2\left(z,x\right))}^2+4{(e_x\left(z,x\right)e_z\left(z,x\right))}^2}\end{array}\right]\\ v_2\left(z,x\right)=\left[\begin{array}{c}-e_z^2\left(z,x\right)-e_x^2\left(z,x\right)-\sqrt{{(e_x^2\left(z,x\right)-e_z^2\left(z,x\right))}^2+4{(e_x\left(z,x\right)e_z\left(z,x\right))}^2}\\ 2e_x\left(z,x\right)e_z\left(z,x\right)\end{array}\right]\end{array}\right.,$$

(27)

where ${\lambda }_1$ and ${\lambda }_2$ are the eigenvalues, and $v_1$ and $v_2$ are the corresponding eigenvectors, respectively. ${\lambda }_1$ represents the tensor energy of the first eigenvector, which is consistent with the direction of the gradient. The orientations of the eigenvector $v_1$ and $v_2$ are:

$$\left\{\begin{array}{l}{\phi }_1\left(z,x\right)=arctan\left(\frac{{\lambda }_1\left(z,x\right)-e_z^2\left(z,x\right)}{e_z\left(z,x\right)e_x\left(z,x\right)}\right)\\ {\phi }_2\left(z,x\right)=arctan\left(\frac{e_z\left(z,x\right)e_x\left(z,x\right)}{{\lambda }_2\left(z,x\right)-e_x^2\left(z,x\right)}\right)\end{array}\right.,$$

(28)

where ${\phi }_1$ is the gradient direction.

(7) Discount factor

Considering the impact on the subsequent limited-time steps, the discount factor is defined as

$$\gamma \left(k\right)=\left\{\begin{array}{l}{e}^{-\frac{k^2}{{\sigma }^2}} , if k<N_t\\ 0 , else\end{array}\right.,$$

(29)

where $N_t$ is the maximum time step, and $k=t-t_c$, $t_c$ is the current time. Figure 3 shows the discount factor for $N_t=10$. As the time step gets further away, the discount factor gets smaller, and the subsequent value function becomes less influential on the decision.

4. An Effective Scheme for Horizon Picking

We designed an effective implementation scheme for automatically picking multiple horizons using multiple seismic attributes and MDP, shown in Figure 4. The scheme mainly includes five parts as follows:

(1) Preprocessing the seismic image.

The seismic image contains not only multiple horizons but also various coherent and incoherent noises. The proper preprocessing method can highlight wavelet characteristics and suppress some noises. We use the structural smoothing filtering method to process the seismic image.

(2) Determining the position of the initial seed point

We proposed two methods to obtain the seed point to adapt the different applications. One method is to rely on manual interpretation, and the second method is to use edge detection and clustering to obtain seed points automatically. For the automated method, firstly, we use an edge detection method to calculate edge points with different seismic attributes, and then cluster these edge points together to obtain the corresponding center point location. Last, the initial seed points are determined by the constraints of horizon interval and maximum horizon number. The second method is mainly aimed at the process of automatically determining seed points, such as extracting structural information in inversion iteration. However, for complex imaging results, we prefer manual interpretation.

(3) Determining the size of state space

According to the position of seed points $z_0^0$, we use the conventional method (waveform, phase, and extremum attributes) to pick up the horizon. Then, the $z_0^t$ location is determined by superimposing these conventional results to form the initial state.

$$\begin{array}{l}{F}_i^c=\left\{\begin{array}{l}{F}_i^c if \left(F_i^c-z_0^0\right)<dz, i=\left\{w,p,e\right\}\\ z_0^0 else ,\end{array}\right.\\ z_0^t=\left(F_w^c+F_p^c+F_e^c\right)/3\end{array}$$

(30)

where $F_w^c,F_p^c,F_e^c$ are the results of the waveform, phase, and extremum attributes, respectively.

(4) MDP tracking process

After defining the MDP model with multiple seismic attributes, the optimal state (horizon position) can be obtained by the state value function. When the following conditions are met during MDP, the tracking will stop:

(a) When the decision time reaches the end point (the boundary of the profile), it automatically turns to the termination state and records the length of the horizon.

(b) When the long-term value function is less than the given threshold (it means that this may encounter a fault), the system will turn to the termination state and record the length of the horizon.

(5) Output the tracking result

If the length of the horizon is less than the given threshold length, the horizon will be discarded, and the MDP tracking process will be started with the new seed point. On the contrary, we will output the horizon as the final result.

5. Numerical Examples

In this section, we illustrate the effectiveness of the proposed automatic horizon-picking method in both synthetic and field data examples.

5.1. A Local Layered Model

In the first example, we use the migration result of a synthetic model to demonstrate the validity of the proposed method. Figure 5a shows the local seismic image under the condition of inaccurate migration velocity. A large number of migration artifacts appear in the profile, and the waveforms change rapidly along the horizons. The attributes of the envelope, instantaneous phase, and dip are shown in Figure 5b–d. As can be seen, the different attributes can highlight the partial characteristics of the seismic image. For example, when the envelope attribute occurs discontinuously (as shown in the black box in Figure 5b), the phase attribute has good continuity (as shown in the black box in Figure 5c). Multiple attributes can be used to supplement each other’s defects. However, not all attributes may be good. At this time, if we take a conventional method such as single-trace-picking-based similarity, the reflection horizon cannot be traced accurately. Although all attributes are not good in local areas, they have certain similarities in the horizontal neighborhood of the area. This means that we can use the attributes of a neighborhood of the area to determine the horizon using a multi-step prediction method such as the MDP method. Figure 5e shows the results of the conventional single trace cross-correlation picking method and the proposed method using the multiple attributes with MDP. The sharp changes in the waveform in the horizontal direction and the attribute differences in the region lead to the inability of the conventional picking method to correctly pursue the horizon position. The Markov decision process method based on multi-attribute and multi-trace rewards can effectively avoid local noise interference and successfully track the correct reflection horizon.

5.2. The North Sea F3 Field Data

The second example illustrates the application of the proposed method to a real 3-D seismic survey F3 acquired over the North Sea. The extracted slice is shown in Figure 6a, and the instantaneous phase attribute is shown in Figure 6d. We observe that there is an obvious reflection horizon shown as the red arrow, and there is also a horizon with weak amplitude and less continuity, within the depth range of 0.8 km~0.9 km and the horizon position 0~1 km. Figure 6b,e show the troughs of the seismic data and the instantaneous phase attribute. Figure 6c,f show the peaks of the seismic data and the instantaneous phase attribute. Whatever a trough or a peak is shown the seismic data and the instantaneous phase attribute, the horizon shows poor continuity, which is marked by circles shown in Figure 6b,c,e,f. We cannot pick the correct reflection horizon according to any of these attributes. Figure 6g,h show the tracked horizon (red curve) using our proposed method on the image and waveform of seismic data, respectively. The blue cross symbols in Figure 6g,h are the seed points obtained via manual interpretation. Note that the waveform is similar and easy to track within the left portion and the right portion of the seismic slice. The waveform changes rapidly within the middle portion of the seismic slice, which makes it difficult to obtain the correct horizon position via the conventional method. However, the tracked horizon using the proposed method exactly follows the seismic reflection event.

5.3. The Overthrust Model for Picking Multiple Horizons

The third example illustrates the application to the overthrust model for picking multiple horizons. Figure 7a shows the migration imaging result with the background velocity model. In this example, we use the automated method to obtain seed points. The maximum number of seed points is set to eight, the minimum interval of the horizon is 0.1 km, and the minimum length is 1/10 of the number of lateral points. After the edge detection and clustering with the constraints of horizon interval and maximum horizon number, eight seed points are automatically obtained, which are marked by stars shown in Figure 7b. The tracked horizons with red curves using the proposed method overlaid on the seismic image are shown in Figure 7b. Because of the inaccurate migration velocity, the reflection horizon in the migration seismic image is not focused and discontinuous, shown as the black arrows, which makes it difficult to pick the correct reflection horizon information by using the conventional method. The reflection structure extracted using the proposed method is consistent with the real structure, which shows the effectiveness of this method. Therefore, the desired reflection horizons can be selected effectively and applied to the process of velocity modeling and the seismic interpretation process.

5.4. The Field Data for Picking Multiple Horizons

We further test the effectiveness of the proposed method by using the migration image from a field dataset acquired on land. The tracked horizons with red circles using the proposed method overlaid on the local migration image are shown in Figure 8. The horizons have a small interval, and multiple waveforms are stacked together. Therefore, we cannot extract the complete wavelet waveform. This is very common in sandstone thin interbed reservoirs and is exacerbated when the result is included in internal multiples. Given the seed points, we can trace the right horizons from the superimposed waveform. The position of the extracted horizons is consistent with the initial state positions.

Finally, we apply the proposed method to field offshore data. Figure 9a shows the migration imaging result in the background velocity model from the original observed seismic data. Figure 9b shows the tracked horizons with a red curve overlaid on the migration image. The blue stars in Figure 9b are the positions of the seed points obtained via manual interpretation. As can be seen, the picking result of the proposed method is still very good, even when dealing with an actual complex seismic image.

6. Discussion

An appropriate data preprocessing method can effectively improve the accuracy of horizon picking. A seismic image contains not only multiple horizons but also various coherent and incoherent noises. For example, due to inaccurate migration velocity, serious migration artifacts will interfere with the continuity of the waveform. The migration operator is not amplitude-preserving, resulting in a large amplitude difference in the transverse direction of horizons. Therefore, various methods (such as a amplitude-preserving migration algorithm, edge-preserving smoothing, and a denoising method) can be applied to improve the imaging quality. For the problem that the imaging result is not focused due to an inaccurate migration velocity model, we can extract the common-image gathers (CIGs), then the flattening and denoising operators can be applied to the CIGs, and the focused imaging result can be obtained by optimal stacking [64].

The imaging results cannot be directly mapped to horizons. It is necessary to extract some attributes of a seismic image and highlight the difference between the horizon and noise, so as to detect and extract the horizon. The purpose of attribute extraction is to highlight the feature information and eliminate redundant and irrelevant information in a seismic image. However, the waveform in the horizontal direction changes dramatically, and one single attribute often cannot trace to the correct horizon. Multiple attributes can highlight different aspects of seismic image properties. It is necessary to use multiple attributes to make up for the shortcomings of a single attribute. Therefore, the reward function is defined according to the similarity of multiple attributes for adjacent traces. The more attributes there are, the more information is available. However, it should be noted that the calculation of attributes must be accurate. Only correct attributes can contribute to picking. It is better to leave a deficiency uncovered than to have it covered without discretion. Attributes should be as independent as possible and highlight the structural features and similarities of the horizon.

Regarding horizon picking with the MDP, the key is to determine the state space, action space, and reward function. Considering the relationship between state and action at different times, and the similarity and continuity between the near horizons and inline or crossline slice, the association search constraint can be applied with the MDP. Through association search, we can transfer learning to the adjacent horizons or slices. Different MDP-solving methods can also be considered. Various kinds of reinforcement learning based on the MDP algorithm have been developed rapidly in recent years. For example, the Deep Q Network algorithm [65] can effectively search large-scale state space.

Moreover, when picking a horizon with faults, the MDP model needs to be optimized. The presence of a fault causes the low similarity of seismic attributes and low reward value. One solution is to place seed points on both sides of the fault. The position of the seed points will affect the picking result. The results are not the same for different positions of the seed points. The numbers and positions of seed points are also key factors. The method of obtaining seed points requires flexible selection through practical problems. The role of seed points is not only to provide the initial state but also to effectively select and extract the corresponding horizons.

In addition, for 2-D cases, we can regard the single horizon-picking problem as finding one optimal curve under the constraint of multiple seismic attributes. In the case of 3-D problems, the optimal curve will be replaced by the optimal surface. Tracing the optimal surface is a more complex problem. In this paper, we just explain how to track the horizon for a 2-D imaging section; extension to a 3-D problem with the MDP model is the next endeavor.

7. Conclusions

We propose a horizon-picking method based on multiple seismic attributes and the Markov decision process. We regard tracing reflection events on the seismic image as an intelligent decision problem. With the interaction process between the picking target and the environment, we can perceive the change of state and take some action to obtain reward value, where the reward function is defined by multiple seismic attributes. Multiple attributes can highlight different aspects of a seismic image and can make up for the shortcomings of a single attribute through the cross-constraint of multiple attributes. Then, the reward function will affect the state of the environment. Therefore, we can obtain the optimal picking horizon via the MDP model. Numerical examples on both synthetic and field data show that, by considering the optimal MDP with multiple attributes, we can avoid the influence caused by waveform changes in a single trace or a local area and pick up the desired reflection horizon robustly. The result of horizon picking can be used not only in velocity modeling but also in seismic interpretation.