Convergence of Limiting Cases of Continuous-Time, Discrete-Space Jump Processes to Diffusion Processes for Bayesian Inference

Abstract

Jump-diffusion algorithms are applied to sampling from Bayesian posterior distributions. We consider a class of random sampling algorithms based on continuous-time jump processes. The semigroup theory of random processes lets us show that limiting cases of certain jump processes acting on discretized spaces converge to diffusion processes as the discretization is refined. One of these processes leads to the familiar Langevin diffusion equation; another leads to an entirely new diffusion equation.

1. Introduction

This paper develops Monte Carlo algorithms based on continuous-time jump processes for sampling from Bayesian posterior distributions. Section 3 relates some specific continuous-time jump processes to diffusion processes, thereby linking two of the main tools in the bag of tricks of Monte Carlo algorithm designers and offering insight.

1.1. Two Kinds of Convergence

This paper invokes the term convergence in two distinct ways. The first sense of convergence is the convergence of the marginal distribution of random processes over time to a stationary distribution, as defined on p. 238 of [1]. In Bayesian inference applications, we seek Markov processes whose stationary distribution is the desired Bayesian posterior under investigation. The “time” here refers not to the physical time over which we collect data, but an “algorithmic time” associated with the fictitious process we construct.

The second sense of convergence is the convergence of a sequence of different processes to some limiting process, with the convergence being taken over the entire process. This is a more complex issue. While the convergence described in the proceeding paragraph is over simple spaces, such as ${\Re }^n$, or perhaps groups like $SO\left(n\right)$, the convergence of entire processes deals with abstract infinite-dimensional spaces, and defining these notions properly requires considerable care.

The codex by Ethier and Kurtz [1] is one of the most extensive treatises available on the detailed characterization of Markov processes and their convergence via semigroup theory, and we extensively invoke its definitions and theorems. Semigroup theory is our Swiss army knife for the study of convergence, particularly the infinitesimal generators (defined on p. 8 of [1]) that characterize these semigroups (defined on p. 6 of [1]). A semigroup and its generator can be related via a differential equation according to Proposition 1.5(b) on p. 9 of [1]; the uniqueness of this characterization is addressed by Proposition 2.9 on p. 15 of [1].

To show that a chosen process converges to the stationary distribution, it suffices to show that the integral of the generator against the stationary distribution is zero (Proposition 9.2 on p. 239 of [1]). This sort of argument has been a staple of research into pattern-theoretic [2,3,4,5] jump-diffusion algorithms [6,7,8,9,10,11,12,13,14], beginning with Theorem 1 of [15] and the appendices of [16,17].

To show convergence in the second sense—convergence of entire processes—we show that their associated generators converge to the generator of the limiting process. This implies the convergence of semigroups (Theorem 6.1 on p. 28 of [1]), which, in turn, implies the convergence of processes (Theorem 2.5 on p. 167 of [1]) in the Skorohod topology, as defined on pp. 116–118 of [1] and pp. 293–295 of [18]; see also Theorem 7.8 on p. 131 of [1].

1.2. Algorithmic Context

Geman and Hwang [19], among others [20,21,22], proposed simulating Langevin diffusions defined by the stochastic differential equation (SDE)

$$dX\left(t\right)={\displaystyle \frac{1}{2}}{E}^{\prime }\left(X\left(t\right)\right)dt+dW\left(t\right),$$

(1)

where $W\left(t\right)$ is a Wiener process, to sample from densities of the form $\pi \left(x\right)=exp\left[E\right(x\left)\right]/Z$. Langevin diffusion is essentially a gradient ascent with an additional random term. To accommodate scenes of varying dimension, [15] combined diffusions with jump processes to move between subspaces.

Langevin diffusions must be discretized for computer implementation, and the discretized process only approximately maintains a detailed balance. Besag [23] suggested this can be readily remedied by performing a Metropolis–Hastings [24,25] acceptance/rejection step after making a discretized Langevin diffusion step; the resulting discrete-time Markov chain algorithm incorporates all the advantages of the Langevin diffusion approach, namely, the analogy with the gradient search (natural for continuous variables) and completely parallel site updating. Thus, diffusion techniques fit nicely within the Metropolis–Hastings umbrella. Roberts and Rosenthal [26] explore this approach in depth and deduce optimal step sizes.

The remaining sections offer two primary take-home messages in this vein:

• Metropolis–Hastings algorithms and other discrete-time Markov chain Monte Carlo algorithms have been the subject of much investigation in the statistical literature. In Section 2.2, we introduce a continuous-time analogue of a conditional Gibbs sampler [27], where the process jumps on exponential times, with a mean proportional to the inverse of the total probability contained in a neighborhood around the current state. Compared with discrete-time MCMC techniques, such continuous-time jump processes appear to have remained largely unexplored by the random sampling community, although they are well-known as models for physical phenomena: see, for instance, Chapters 4 and 5 of [28].
• If derivatives are difficult to compute, Metropolis–Hasting proposals can be made using a density centered around the old value, such as a Gaussian. The acceptance/rejection step ensures that we still achieve the target distribution. Most intriguingly, ref. [29] shows that certain continuous-time interpolations of some Metropolis sampling algorithms, such as the Gaussian proposals mentioned here, weakly converge to Langevin diffusions with the same stationary distribution. Thus, even if the gradient is not explicitly employed, it implicitly guides the inference in a limiting sense. The difficulty of computing derivatives in recognizing objects in visual aerial images for the U.K. Defence Evaluation and Research Agency led the authors of [30] to construct jump-diffusion algorithms using such an approach. We explore variations of this idea in Section 3, where algorithms that operate on discretized sample spaces are shown to converge to different kinds of diffusions as the discretization is refined.

2. Continuous-Time Jump Processes

2.1. Definitions

A continuous-time Markov jump process is characterized by a jump transition intensity $q(x,dy)$ (using the notation on p. 561 of [15], along the lines of [31]). The generator for a jump process is

$$Af(x)=-q(x)f(x)+\int q(x,dy)f(y),$$

(2)

where the jump intensity is defined as

$$q(x)=\int q(x,dy).$$

(3)

As described on p. 163 of [1] (and in accordance with Theorem 1 on p. 353 of [31]), a continuous-time jump process may be simulated by generating a discrete-time Markov chain, in which we wait a certain random amount of time at each stage in the chain, as follows. Beginning from a state $x_0$ at time $\tau =t_0$, a sample path of the process $X\left(\tau \right)$ is constructed as follows:

• Draw an exponentially distributed random variable $w_i$ with mean $1/q\left(x_i\right)$.
• Let $t_{i+1}=t_i+w_i$. $X\left(\tau \right)=x_i$ for $\tau \in [t_i,t_{i+1})$.
• Draw y from the transition distribution

  $$Q(x_i,dy)={\displaystyle \frac{q(x_i,dy)}{q\left(x_i\right)}},$$

  (4)

  and let $x_{i+1}=y$.
• Assign $i\leftarrow i+1$ and go to step 1.

2.2. A Conditional Gibbs Sampler Subordinated to a Markov Process

We now consider a random-sampling algorithm based on a continuous-time jump process analogous to Corollary 1 of Theorems 1 and 2 of [15]. In [15], the process diffuses between jumps. In this section, we keep the state constant between jumps.

Suppose we restrict the set of potential transitions to a neighborhood around the current state x, denoted as $\mathcal{N}\left(x\right)$. We require the neighborhood to have the following properties:

• Reversibility: $y\in \mathcal{N}\left(x\right)$ if and only if $x\in \mathcal{N}\left(y\right)$. Written in terms of indicator functions, we have $I_{\mathcal{N}\left(x\right)}\left(y\right)=I_{\mathcal{N}\left(y\right)}\left(x\right)$.
• Connectedness: For any x and y, there exists a finite sequence of states $z_1,\dots ,z_m$ such that $z_1\in \mathcal{N}\left(x\right),\dots ,y\in \mathcal{N}\left(z_m\right)$.

Now suppose we define our jump transition intensity according to the posterior density, restricted to the neighborhood

$$q(x,dy)=\pi \left(y\right)I_{\mathcal{N}\left(x\right)}\left(y\right)dy$$

(5)

and

$$q\left(x\right)={\int }_{\mathcal{N}\left(x\right)}\pi \left(y\right)dy.$$

(6)

Proposition 9.2 on p. 239 of [1] may be used to show that $\pi \left(x\right)dx$ is an invariant measure of the resulting jump process. Furthermore, the connectedness of the neighborhoods implies the irreducibility of the underlying chain, which, in turn, implies that the stationary measure is unique and that the process (starting from any initial distribution) converges in a total variation norm to the invariant measure (see pp. 73–74 of [32] for details and a further discussion of ergodicity).

Notice (from step 1 in the construction of jump processes given in the previous subsection) that if the process is at state x, the process will tend to jump away from x rapidly if there is a lot of probability mass contained in the neighborhood around x. If there is little probability in the surrounding neighborhood, the process will tend to spend more time loitering at x.

During implementation, there is no need for the computer to physically wait an amount of time $w_i$. When computing statistics from the chain, we can simply weight the samples according to the $w_i$ instances.

3. Limiting Cases of Discrete-State Continuous-Time Processes

We now present two theorems illustrating how continuous-time random processes defined on discretizations of ℜ with step size $ϵ$ converge to diffusions as the discretization is made finer (as in Section 5.7 of [33]). The first theorem explores a special case of the continuous-time jumping algorithm outlined in Section 2.2. The second is analogous to a Metropolis–Hastings acceptance/rejection scheme. In the second case, the limiting diffusion is a Langevin diffusion. In the first case, the limiting diffusion is a different sort of a diffusion; to our knowledge, it was not previously reported.

One advantage of the generator approach used in this section is that one can easily combine the small-step diffusions presented here with jumping processes that move between subspaces by adding the generators associated with the diffusion and jump processes. An elegant example of this approach is given in Section 5.4 of [32].

Section 5.5 of [32] discusses the the ergodic properties of diffusion processes, including the uniqueness of their invariant measures and their convergence to those invariant measures.

3.1. Conditional Gibbs Style

Theorem 1. 

Let π be a strictly positive, twice-differentiable probability density on ℜ with a Gibbs representation $\pi \left(x\right)=exp\left[E\right(x\left)\right]/Z$. Consider a family of jump processes $X_{ϵ}\left(t\right)$, indexed by $ϵ>0$, with a jump measure given by

$$q_{ϵ}(x,dy)={\displaystyle \frac{1}{{ϵ}^2}}\left\{\pi (x+ϵ)\delta (y-(x+ϵ\left)\right)+\pi (x-ϵ)\delta (y-(x-ϵ\left)\right)\right\}dy,$$

(7)

where δ is the Dirac delta function (this is analogous to Equation (20) of [15]). As $ϵ\to 0$, $X_{ϵ}$ converges in the Skorohod topology to a diffusion process X defined by the novel SDE

$$dX\left(t\right)=2E^{\prime }\left(X\left(t\right)\right)\pi \left(X\left(t\right)\right)dt+\sqrt{2\pi \left(X\right(t\left)\right)}dW\left(t\right).$$

(8)

Moreover, $\pi \left(x\right)dx$ is a stationary measure for X.

Remark 1. 

Notice that (8) is somewhat similar to a Langevin SDE, with some important distinctions. The process still drifts along the gradient of the posterior, but the multiplication by $\pi \left(X\right(t\left)\right)$ in the drift term implies that the gradient pulls harder when exploring regions of higher probability. Similarly, the Wiener process is more “turbulent” in regions of high probability. We have been unable to find (8) in the literature (such as the examples and problems in Chapter 7 of [34]), and therefore do not have a name for it.

Proof. 

Here, the jump intensity is

$$q_{ϵ}\left(x\right)=\int q(x,dy)={\displaystyle \frac{1}{{ϵ}^2}}\left\{\pi (x+ϵ)+\pi (x-ϵ)\right\}.$$

(9)

The generator (2) associated with index $ϵ$ is given by

$$\begin{array}{c}\hfill A_{ϵ}f\left(x\right)={\displaystyle \frac{1}{{ϵ}^2}}\left\{-\left[\pi \right(x+ϵ)+\pi (x-ϵ\left)\right]f\left(x\right)+\pi (x+ϵ)f(x+ϵ)+\pi (x-ϵ)f(x-ϵ)\right\}.\end{array}$$

Since we take the limits to form derivatives, we restrict the domain of f to be twice-differential bounded functions.

Expanding functions with $ϵ$ in their arguments in Taylor series to the second order, the limit of (10) as $ϵ\to 0$ is $A_{0}f\left(x\right)\stackrel{\mathrm{df}}{=}{lim}_{ϵ\to 0}{A}_{ϵ}f\left(x\right)=$

$$\begin{array}{cc}\hfill & \underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{{ϵ}^2}}\{-[\pi (x+ϵ)+\pi (x-ϵ)]f\left(x\right)+\pi (x+ϵ)f(x+ϵ)+\pi (x-ϵ)f(x-ϵ)\}\hfill \\ \hfill & =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{{ϵ}^2}}\{-[2\pi \left(x\right)+{ϵ}^2{\pi }^{″}\left(x\right)]f\left(x\right)+2f\left(x\right)\pi \left(x\right)+{ϵ}^2{(\pi ·f)}^{″}\left(x\right)\}\hfill \\ \hfill & =-{\pi }^{″}\left(x\right)f\left(x\right)+{\pi }^{″}\left(x\right)f\left(x\right)+2{\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)+\pi \left(x\right)f^{″}\left(x\right)\hfill \\ \hfill & =2{\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)+\pi \left(x\right)f^{″}\left(x\right),\hfill \end{array}$$

(10)

which is the generator of a diffusion with infinitesimal mean $2{\pi }^{\prime }\left(x\right)=2E^{\prime }\left(x\right)\pi \left(x\right)$ and infinitesimal variance $2\pi \left(x\right)$ (Equation (1.2) on p. 366 of [1]). This process can be implicitly defined by our newly discovered SDE

$$dX\left(t\right)=2E^{\prime }\left(X\left(t\right)\right)\pi \left(X\left(t\right)\right)dt+\sqrt{2\pi \left(X\right(t\left)\right)}dW\left(t\right),$$

(11)

where $W\left(t\right)$ is a standard Wiener process.

Theorem 6.1 on p. 28 of [1] shows that the convergence of generators implies the convergence of their associated semigroups. Combining this with Theorem 2.5 on p. 167 of [1] implies that $X_{ϵ}\to X$ in the Skorohod topology as $ϵ\to 0$.

By Proposition 9.2 on p. 239 of [1], $\pi$ is a stationary density of X if and only if $\int A_{0}f\left(x\right)\pi \left(x\right)dx=0$ for all f in the domain of the generator:

$$\int A_{0}f\left(x\right)\pi \left(x\right)dx=\int {\pi }^2\left(x\right)f^{″}\left(x\right)dx+2\int {\pi }^{\prime }\left(x\right)\pi \left(x\right)f^{\prime }\left(x\right)dx.$$

(12)

Applying integration by parts (let $u={\pi }^2\left(x\right)$ and $dv=f^{″}\left(x\right)dx$, so $du=2\pi \left(x\right){\pi }^{\prime }\left(x\right)dx$ and $v=f^{\prime }\left(x\right)$) to the first term and employing the fact that $\pi \left(x\right)$ and $f^{\prime }\left(x\right)$ must both go to zero at the extremes yields

$$\int A_{0}f\left(x\right)\pi \left(x\right)dx={\pi }^2\left(x\right)f^{\prime }{\left(x\right)|}_{-\infty }^{\infty }-2\int {\pi }^{\prime }\left(x\right)\pi \left(x\right)f^{\prime }\left(x\right)dx+2\int {\pi }^{\prime }\left(x\right)\pi \left(x\right)f^{\prime }\left(x\right)dx,$$

(13)

which is zero. □

3.2. Metropolis Style

This section is in the spirit of Corollary 2 of Theorems 1 and 2 of [15]. An informal recollection of the classic construction of Brownian motion in ℜ in terms of coin flips will be helpful for interpreting the next theorem. At each time step, if the coin comes up heads, our Brownian traveller takes a step to the left; if tails, a step to the right. Letting the time between steps and the size of the steps shrink yields Brownian motion. Now, suppose we place a density $\pi$ over the range of places the traveller may sojourn, and suppose that instead of automatically taking a step after each coin flip, the traveller first looks at the value of the density $\pi$ at the place they are asked to move and compares it with the value of $\pi$ where they are currently at. If the probability is higher, they take the step; if lower, they decide to accept or reject the step in the traditional Metropolis fashion. At each stage, the traveller waits for a random amount of time according to an exponential distribution with mean ${ϵ}^2$. This heuristic scenario is made precise in the next theorem, where we find that this alternate traveller’s journey, instead of converging to Brownian motion, remarkably converges to a Langevin diffusion with a stationary density $\pi$. For clarity, we make the Metropolis procedure explicit in our theorem statement by allowing the process to “jump” to the same state; we address this further in a remark after the proof.

Theorem 2. 

Let π be a strictly positive, twice-differentiable probability density on ℜ with a Gibbs representation $\pi \left(x\right)=exp\left[E\right(x\left)\right]/Z$. Consider a family of jump processes $X_{ϵ}\left(t\right)$, indexed by $ϵ>0$, with a jump measure given by $q_{ϵ}(x,dy)=$

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{{ϵ}^2}}\left\{{\displaystyle \frac{1}{2}}min\left({\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right)\delta (y-(x+ϵ))+{\displaystyle \frac{1}{2}}min\left({\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right)\delta (y-(x-ϵ))\right.\\ \hfill \left.+{\displaystyle \frac{1}{2}}\left[1-min\left({\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right)\right]\delta (y-x)+{\displaystyle \frac{1}{2}}\left[1-min\left({\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right)\right]\delta (y-x)\right\}dy.\end{array}$$

(14)

The first two terms of (14) correspond to Metropolis acceptances; the second two correspond to rejections. As $ϵ\to 0$, $X_{ϵ}$ converges in the Skorohod topology to a diffusion process X defined by the Langevin SDE

$$dX\left(t\right)={\displaystyle \frac{1}{2}}{E}^{\prime }\left(X\left(t\right)\right)dt+dW\left(t\right).$$

(15)

Moreover, $\pi \left(x\right)dx$ is a stationary measure for X.

Proof. 

Here, the jump intensity is

$$q_{ϵ}\left(x\right)=\int q(x,dy)={\displaystyle \frac{1}{{ϵ}^2}}.$$

The generator (2) associated with the index $ϵ$ is given by

$$\begin{array}{ccc}\hfill A_{ϵ}f\left(x\right)& =& {\displaystyle \frac{1}{2{ϵ}^2}}\left\{-\left[min\left({\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right)+min\left({\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right)\right]f\left(x\right)\right.\hfill \\ & & +min\left({\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right)f(x+ϵ)\left.+min\left({\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right)f(x-ϵ)\right\}.\hfill \end{array}$$

(16)

It will be easiest to consider separate cases. For $\pi (x+ϵ)>\pi \left(x\right)$ and $\pi (x-ϵ)<\pi \left(x\right)$, generator (16) simplifies to

$$A_{ϵ}f\left(x\right)={\displaystyle \frac{1}{2{ϵ}^2}}\left\{-\left[1+{\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}}\right]f\left(x\right)+f(x+ϵ)+{\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}}f(x-ϵ)\right\}.$$

(17)

Expanding functions with $ϵ$ in their arguments in Taylor series to the second order yields

$$\begin{array}{cc}\hfill A_{0}f\left(x\right)\stackrel{\mathrm{df}}{=}\underset{ϵ\to 0}{lim}{A}_{ϵ}f\left(x\right)& =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-\left[1+{\displaystyle \frac{\pi \left(x\right)-ϵ{\pi }^{\prime }\left(x\right)+({ϵ}^2/2){\pi }^{″}\left(x\right)}{\pi \left(x\right)}}\right]f\left(x\right)\right.\hfill \\ \hfill & +f\left(x\right)+ϵf^{\prime }\left(x\right)+({ϵ}^2/2)f^{″}\left(x\right)\hfill \\ \hfill & \left.+{\displaystyle \frac{\pi \left(x\right)f\left(x\right)-ϵ{(\pi ·f)}^{\prime }\left(x\right)+({ϵ}^2/2){(\pi ·f)}^{″}\left(x\right)}{\pi \left(x\right)}}\right\}\hfill \\ \hfill & =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-\left[1+{\displaystyle \frac{\pi \left(x\right)-ϵ{\pi }^{\prime }\left(x\right)+({ϵ}^2/2){\pi }^{″}\left(x\right)}{\pi \left(x\right)}}\right]f\left(x\right)\right.\hfill \\ \hfill & +f\left(x\right)+ϵf^{\prime }\left(x\right)+{\displaystyle \frac{{ϵ}^2}{2}}{f}^{″}\left(x\right)\hfill \\ \hfill & +{\displaystyle \frac{\pi \left(x\right)f\left(x\right)-ϵ[{\pi }^{\prime }\left(x\right)f\left(x\right)+\pi \left(x\right)f^{\prime }\left(x\right)]}{\pi \left(x\right)}}\hfill \\ \hfill & \left.+{ϵ}^2{\displaystyle \frac{{\pi }^{″}\left(x\right)f\left(x\right)+2{\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)+\pi \left(x\right)f^{″}\left(x\right)}{2\pi \left(x\right)}}\right\}\hfill \\ \hfill & =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-2f\left(x\right)+ϵ{\displaystyle \frac{{\pi }^{\prime }\left(x\right)f\left(x\right)}{\pi \left(x\right)}}-{ϵ}^2{\displaystyle \frac{{\pi }^{″}\left(x\right)f\left(x\right)}{F}}2\pi \left(x\right)\right.\hfill \\ \hfill & +2f\left(x\right)-ϵ{\displaystyle \frac{{\pi }^{\prime }\left(x\right)f\left(x\right)}{\pi \left(x\right)}}+{ϵ}^2{\displaystyle \frac{{\pi }^{″}\left(x\right)f\left(x\right)}{2\pi \left(x\right)}}\hfill \\ \hfill & \left.+{ϵ}^2{\displaystyle \frac{{\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)}{\pi \left(x\right)}}+{ϵ}^2{f}^{″}\left(x\right)\right\}\hfill \\ \hfill & ={\displaystyle \frac{{\pi }^{\prime }\left(x\right)}{2\pi \left(x\right)}}{f}^{\prime }\left(x\right)+{\displaystyle \frac{1}{2}}{f}^{″}\left(x\right).\hfill \end{array}$$

(18)

Working this through for the opposite case with $\pi (x+ϵ)<\pi \left(x\right)$ and $\pi (x-ϵ)>\pi \left(x\right)$ yields the same result.

Now consider the case where $\pi (x+ϵ)<\pi \left(x\right)$ and $\pi (x-ϵ)<\pi \left(x\right)$ holds as $ϵ\to 0$ so that ${\pi }^{\prime }\left(x\right)=0$. Here,

$$\begin{array}{cc}\hfill A_{0}f\left(x\right)\stackrel{\mathrm{df}}{=}\underset{ϵ\to 0}{lim}{A}_{ϵ}f\left(x\right)& =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-{\displaystyle \frac{2\pi \left(x\right)+{ϵ}^2{\pi }^{″}\left(x\right)}{\pi \left(x\right)}}f\left(x\right)+{\displaystyle \frac{2\pi \left(x\right)f\left(x\right)+{ϵ}^2{(\pi ·f)}^{″}\left(x\right)}{\pi \left(x\right)}}\right\}\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}\left\{-{\displaystyle \frac{{\pi }^{″}\left(x\right)}{\pi \left(x\right)}}f\left(x\right)+{\displaystyle \frac{{\pi }^{″}\left(x\right)f\left(x\right)}{\pi \left(x\right)}}+2{\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)+{\displaystyle \frac{\pi \left(x\right)f^{″}\left(x\right)}{\pi \left(x\right)}}\right\}\hfill \\ \hfill & ={\displaystyle \frac{f^{″}\left(x\right)}{2}}.\hfill \end{array}$$

(19)

Finally, consider the case where $\pi (x+ϵ)>\pi \left(x\right)$ and $\pi (x-ϵ)>\pi \left(x\right)$ hold as $ϵ\to 0$ such that ${\pi }^{\prime }\left(x\right)=0$. Now we have

$$\begin{array}{cc}\hfill A_{0}f\left(x\right)\stackrel{\mathrm{df}}{=}\underset{ϵ\to 0}{lim}{A}_{ϵ}f\left(x\right)& =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-2f\left(x\right)+f(x+ϵ)+f(x-ϵ)\right\}\hfill \\ \hfill & =\underset{ϵ\to 0}{lim}{\displaystyle \frac{1}{2{ϵ}^2}}\left\{-2f\left(x\right)+2f\left(x\right)+{ϵ}^2{f}^{″}\left(x\right)\right\}={\displaystyle \frac{f^{″}\left(x\right)}{2}}.\hfill \end{array}$$

(20)

Notice that (19) and (20) can both be written as (18), which is the generator of a diffusion process with infinitesimal mean ${\pi }^{\prime }\left(x\right)/\left[2\pi \left(x\right)\right]=E^{\prime }\left(x\right)/2$ and infinitesimal variance 1. This process can be implicitly defined by the SDE for the Langevin diffusion

$$dX\left(t\right)={\displaystyle \frac{1}{2}}{E}^{\prime }\left(X\left(t\right)\right)dt+dW\left(t\right),$$

(21)

where $W\left(t\right)$ is a standard Wiener process. As in the preceding theorem, applying Theorem 6.1 on p. 28 of [1], along with Theorem 2.5 on p. 167 of [1], reveals that $X_{ϵ}\to X$ in the Skorohod topology as $ϵ\to 0$.

Again, by Proposition 9.2 on p. 239 of [1], $\pi$ is a stationary density of X if and only if $\int A_{0}f\left(x\right)\pi \left(x\right)dx=0$ for all f in the domain of the generator:

$$\int A_{0}f\left(x\right)\pi \left(x\right)dx={\displaystyle \frac{1}{2}}\left\{\int \pi \left(x\right)f^{″}\left(x\right)dx+\int {\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)dx\right\}.$$

(22)

Applying integration by parts (let $u=\pi \left(x\right)$ and $dv=f^{″}\left(x\right)dx$, so $du={\pi }^{\prime }\left(x\right)dx$ and $v=f^{\prime }\left(x\right)$) to the first term and employing the fact that $\pi \left(x\right)$ and $f^{\prime }\left(x\right)$ must both go to zero at the extremes yields

$$\int A_{0}f\left(x\right)\pi \left(x\right)dx={\displaystyle \frac{1}{2}}\left\{\pi \left(x\right)f^{\prime }{\left(x\right)|}_{-\infty }^{\infty }-\int {\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)dx+\int {\pi }^{\prime }\left(x\right)f^{\prime }\left(x\right)dx\right\},$$

(23)

which is zero. □

Remark 2. 

Consider a jump transition intensity consisting of the first two terms of (14): $q_{ϵ}(x,dy)=$

$${\displaystyle \frac{1}{{ϵ}^2}}\left\{{\displaystyle \frac{1}{2}}min\left({\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right)\delta (y-(x+ϵ))+{\displaystyle \frac{1}{2}}min\left({\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right)\delta (y-(x-ϵ))\right\},$$

(24)

which is analogous to Equation (21) of [15] (except the “proposals” are chosen with equal probability instead of according to a prior). The corresponding jump intensity is

$$q_{ϵ}\left(x\right)={\displaystyle \frac{1}{2{ϵ}^2}}\left\{min\left[{\displaystyle \frac{\pi (x+ϵ)}{\pi \left(x\right)}},1\right]+min\left[{\displaystyle \frac{\pi (x-ϵ)}{\pi \left(x\right)}},1\right]\right\},$$

as would be used in the algorithm described in Section 2.1. The resulting generator $A_{ϵ}f\left(x\right)$ exactly matches (16), so it describes the same family of processes, though it suggests a different practical implementation.

4. Conclusions

Section 3 presents two new theorems that demonstrate that certain random sampling algorithms operating on a discretized space in which moves are restricted to neighboring points converge to diffusion algorithms as the discretization is refined to the continuum. A Metropolis–Hastings formulation leads to the Langevin SDE; in contrast, a Gibbs-style formulation leads to an entirely new SDE that, to our knowledge, had not previously appeared in the literature.

These theorems were formulated for one-dimensional processes; future work could consider extensions to multi-dimensional processes, including Cartesian products that involve rotation groups, as in [35]. A first step toward an extension to ${\Re }^n$ might be to add terms associated with increments and decrements of $ϵ$ in additional coordinates to (7) and (14).

The convergence results in this paper do not make any claims about the rates of convergence; future work on numerical simulations would yield insight along these lines by comparing different methods.