Arbitrary-Order Finite-Time Corrections for the Kramers–Moyal Operator

Abstract

With the aim of improving the reconstruction of stochastic evolution equations from empirical time-series data, we derive a full representation of the generator of the Kramers–Moyal operator via a power-series expansion of the exponential operator. This expansion is necessary for deriving the different terms in a stochastic differential equation. With the full representation of this operator, we are able to separate finite-time corrections of the power-series expansion of arbitrary order into terms with and without derivatives of the Kramers–Moyal coefficients. We arrive at a closed-form solution expressed through conditional moments, which can be extracted directly from time-series data with a finite sampling intervals. We provide all finite-time correction terms for parametric and non-parametric estimation of the Kramers–Moyal coefficients for discontinuous processes which can be easily implemented—employing Bell polynomials—in time-series analyses of stochastic processes. With exemplary cases of insufficiently sampled diffusion and jump-diffusion processes, we demonstrate the advantages of our arbitrary-order finite-time corrections and their impact in distinguishing diffusion and jump-diffusion processes strictly from time-series data.

1. Introduction

The reconstruction of stochastic evolution equations from time-series data in terms of the Langevin equation and the corresponding Fokker–Planck equation is often challenged by the inevitably finite temporal sampling of time-series data. Moreover, the Fokker–Planck equation is restricted to continuous stochastic processes, i.e., diffusion, and thus cannot adequately describe discontinuous transitions in time-series data. A more general description of continuous and discontinuous stochastic processes can be constructed using the Kramers–Moyal equation [1,2] given by

$$\frac{\partial }{\partial \tau }p(x,t+\tau |x^{\prime },t)=\sum _{n=1}^{\infty }{\left(-\frac{\partial }{\partial x}\right)}^n{D}_n\left(x\right)p(x,t+\tau |x^{\prime },t),$$

of which the Fokker–Planck equation is a particular case ($D_n\equiv 0$ for $n>2$). The Kramers–Moyal equation serves as a stepping stone to adequately describe time-series data with both diffusive and discontinuous characteristics, but it is nevertheless challenged by finite-time sampling in real-world data. Recent applications of the Kramers–Moyal equation include brain [3,4] and heart dynamics [5], stochastic harmonic oscillators [6], renewable-energy generation [7], solar irradiance [8], turbulence [9], nano-scale friction [10], and X-ray imaging [11,12].

Previous work has demonstrated that a finite sampling interval $\Delta t$ not only influences the first- and second-order Kramers–Moyal (KM) coefficients [13] but also causes non-vanishing, higher-order (>2) coefficients [4,14,15,16,17,18,19,20]. A more recent example is the jump-diffusion process discussed in reference [3]

$$\mathrm{d}{X}_t=a(X_t,t)\mathrm{d}t+b(X_t,t)\mathrm{d}W\left(t\right)+\xi (X_t,t)\mathrm{d}J\left(t\right),$$

(1)

where $a(X_t,t)$ is a drift function, $b(X_t,t)$ is the diffusion associated with an uncorrelated Brownian motion or Wiener process $W\left(t\right)$, $J\left(t\right)$ is a Poisson process with jump rate $\lambda$, independent of $W\left(t\right)$, and $\xi (X_t,t)$ is Gaussian-distributed $\mathcal{N}(0,s)$ with zero mean and variance s. For such jump-diffusion processes, additional influences of the finite temporal sampling need to be taken into account. As shown in reference [8], jump events produce terms of order $\mathcal{O}(\Delta t)$ in the KM coefficients of even orders and the jump rate and amplitude induce terms of order $\mathcal{O}(\Delta t^2)$ in all coefficients. These influences are heightened for bivariate jump-diffusion processes [21], since terms of order $\mathcal{O}(\Delta t^i),i\ge 3$ impact higher-order (≥4) coefficients [22].

Although most of the aforementioned studies reported on finite-time corrections for KM coefficients and/or conditional moments of various orders, we still lack an explicit arbitrary-order correction or a closed-form solution in which the conditional moments are represented as functions of the KM coefficients and vice versa. In this article, we derive a full expansion of the generator of the Kramers–Moyal operator in exponential form for one-dimensional Markovian processes. This is equivalent to van Kampen’s system-size expansion, which is taken over a finite time interval $\tau$ [23,24]. The derivations presented henceforth are generally applicable to Markovian diffusion as well as jump-diffusion processes.

On a more general level, our solution is an explicit approximate solution of the Kramers–Moyal equation [1,2], which generalises the Fokker–Planck equation for discontinuous processes [13,23,25]. Our approximation of the Kramers–Moyal operator can be taken as an arbitrary order. In particular, we focus on the solution of this partial differential equation by representing the Kramers–Moyal operator in an exponential form and equating the conditional moments with the KM coefficients after representing the exponential operator as a power series. This representation of the exponential operator can similarly be used in other problems with an equivalent formulation [26,27,28] or similar discontinuous stochastic processes with different jump distributions, e.g., the Gamma distribution [29,30].

2. Mathematical Background

The Fokker–Planck(–Kolmogorov) equation (Kolmogorov forward equation or Smoluchowski equation) for the conditional probability density $p(x,t+\tau |x^{\prime },t)$, that is well-known within the fields of physics and mathematics, yields the propagation in time and space of any diffusion (thus continuous) process, is given by [31]

$$\frac{\partial }{\partial \tau }p(x,t+\tau |x^{\prime },t)=\left[\frac{\partial }{\partial x}{D}_1(x,t)+\frac{{\partial }^2}{\partial x^2}{D}_2(x,t)\right]p(x,t+\tau |x^{\prime },t).$$

(2)

We restrict our investigation to stationary processes, hence $D_n(x,t)=D_n\left(x\right)$. Equation (2) describes the evolution of, for instance, a Brownian particle (for the case $D_1\left(x\right)=0$), which results in the known heat equation, or more complicated Markovian motions with drift. Here, one recognises the function $D_1\left(x\right)$, the first KM coefficient, commonly denoted as drift, and the function $D_2\left(x\right)$, the second KM coefficient, commonly denoted as diffusion or volatility. The Fokker–Planck equation is, nevertheless, only valid for continuous motions and thus cannot describe jump-diffusion processes as in the case in Equation (1) or other stochastic motions with discontinuous paths.

A more general equation—the so-called Kramers–Moyal equation—takes higher-order KM coefficients $D_n\left(x\right),n\in \mathbb{N}$ into account

$$\frac{\partial }{\partial \tau }p(x,t+\tau |x^{\prime },t)={\mathcal{L}}_{\mathrm{KM}}p(x,t+\tau |x^{\prime },t),$$

(3)

where ${\mathcal{L}}_{\mathrm{KM}}$ denotes the Kramers–Moyal operator defined as the power series [1,2]

$${\mathcal{L}}_{\mathrm{KM}}=\sum _{n=1}^{\infty }{\left(-\frac{\partial }{\partial x}\right)}^n{D}_n\left(x\right),$$

which we will subsequently solve for $\tau$ and an appropriate starting condition by exponentiating the Kramers–Moyal operator ${\mathcal{L}}_{\mathrm{KM}}$.

When examining a stochastic process in terms of time-series data, there is no direct access to the KM coefficients $D_n\left(x\right)$ but rather to the conditional moments of the data. The $n$th-order conditional moment $M_n(x,\tau )$ is given by

$$M_n(x^{\prime },\tau )=\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^{n}p(x,t+\tau |x^{\prime },t)\mathrm{d}x.$$

(4)

The KM coefficients $D_n\left(x\right)$ can be retrieved from the conditional moments $M_n(x,\tau )$ via

$$D_n\left(x\right)=\frac{1}{n!}\underset{\tau \to 0}{lim}\frac{M_n(x,\tau )}{\tau }.$$

When dealing with real-world data, we do not have access to infinite temporal resolution, meaning that the above limit $\tau \to 0$ is not possible. A best-case scenario is to analyse the smallest possible temporal differences. If the data are sampled at $\Delta t$ time steps, take

$$D_n\left(x\right)=\frac{1}{n!}\frac{1}{\Delta t}{M}_n(x,\Delta t).$$

In order to non-parametrically retrieve the conditional moments $M_n(x,\tau )$ from data, a set of histogram or Nadaraya–Watson estimators can be utilised (see Refs. [29,32] for details). Here, we will focus not on how to estimate the conditional moments but rather on how to derive a set of finite-time corrections to estimate the KM coefficients from conditional moments. These can be retrieved from data with software packages like kramersmoyal [33] or JumpDiff [34] in Python or Langevin [35] in R.

3. The Formal Solution of the Kramers–Moyal Equation and Its Approximations

First, we explicitly derive the corrective terms and subsequently link these to the results in reference [36], connecting them to the relation between statistical cumulants and moments [13].

Let us assume a well-defined initial state of the Kramers–Moyal equation be given by $\delta (x-x^{\prime })$. The formal solution of the time-dependent Kramers–Moyal equation (3) is given by

$$p(x,t+\tau |x^{\prime },t)=exp\left(\tau {\mathcal{L}}_{\mathrm{KM}}\right)\delta (x-x^{\prime })=\sum _{k=0}^{\infty }\frac{{\left(\tau {\mathcal{L}}_{\mathrm{KM}}\right)}^k}{k!}\delta (x-x^{\prime }),$$

(5)

where $p(x,t+\tau |x^{\prime },t)$ is a normalisable function, such that ${\int }_{-\infty }^{\infty }p(x,t+\tau |x^{\prime },t)\mathrm{d}x=1,\forall (t,\tau )$. We will now proceed to show the first-, second-, third-order, and arbitrary-order approximation to the solution of this partial differential equation with this particular initial condition.

3.1. The First- and Second-Order Approximations

The first-order approximation of the formal solution of Equation (5) is given by

$$p(x,t+\tau |x^{\prime },t)=exp\left(\tau {\mathcal{L}}_{\mathrm{KM}}\right)\delta (x-x^{\prime })=\left[1+\tau {\mathcal{L}}_{\mathrm{KM}}+\mathcal{O}\left({\tau }^2\right)\right]\delta (x-x^{\prime }),$$

yielding for the conditional moments $M_n(x^{\prime },\tau )$ in Equation (4)

$$\begin{array}{cc}\hfill M_n(x^{\prime },\tau )& \simeq M_n^{\left[1\right]}(x^{\prime },\tau )=\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\left[1+\tau {\mathcal{L}}_{\mathrm{KM}}\right]\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\delta (x-x^{\prime })\mathrm{d}x+\tau \underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\sum _{m=1}^{\infty }{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\left(x\right)\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =0+\tau \sum _{m=1}^{\infty }{(-1)}^m\underset{-\infty }{\overset{\infty }{\int }}{D}_m\left(x\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^m{(x-x^{\prime })}^n\right]\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =\tau \sum _{m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}{D}_m\left(x\right)\frac{n!}{(n-m)!}{(x-x^{\prime })}^{n-m}\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =\tau \sum _{m=1}^{\infty }{D}_m\left(x^{\prime }\right)\frac{n!}{(n-m)!}{\delta }_{n,m}\hfill \\ \hfill & =\tau (n!)D_n\left(x^{\prime }\right),\hfill \end{array}$$

where the large square brackets indicate that the derivation operation is limited to the terms within the brackets. The superscript $\left[1\right]$ indicates the order of approximation.

The second-order approximation is obtained in a similar fashion, now including the quadratic term from the exponential representation Equation (5), i.e.,

$$\begin{array}{cc}\hfill p(x,t+\tau |x^{\prime },t)& =\left[1+\tau {\mathcal{L}}_{\mathrm{KM}}+\frac{{\tau }^2}{2}{\mathcal{L}}_{\mathrm{KM}}{\mathcal{L}}_{\mathrm{KM}}+\mathcal{O}\left({\tau }^3\right)\right]\delta (x-x^{\prime }).\hfill \end{array}$$

To alleviate the notation, we refer to the KM coefficient without explicit state dependencies, i.e., $D_n$. The second-order approximation $M_n^{\left[2\right]}(x^{\prime },\tau )$ of the n-th conditional moment in Equation (4) reads

$$\begin{array}{cc}\hfill M_n^{\left[2\right]}(x^{\prime },\tau )& =\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\left[1+\tau {\mathcal{L}}_{\mathrm{KM}}+\frac{{\tau }^2}{2}{\mathcal{L}}_{\mathrm{KM}}{\mathcal{L}}_{\mathrm{KM}}\right]\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =M_n^{\left[1\right]}(x^{\prime },\tau )+\frac{{\tau }^2}{2}\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\sum _{p=1}^{\infty }{\left(-\frac{\partial }{\partial x}\right)}^p{D}_p\sum _{m=1}^{\infty }{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =M_n^{\left[1\right]}(x^{\prime },\tau )+\frac{{\tau }^2}{2}\sum _{p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n{\left(-\frac{\partial }{\partial x}\right)}^p{D}_p{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =M_n^{\left[1\right]}(x^{\prime },\tau )+\frac{{\tau }^2}{2}\sum _{p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}\frac{n!}{(n-p)!}{(x-x^{\prime })}^{n-p}{D}_p{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & =M_n^{\left[1\right])}(x^{\prime },\tau )+\frac{{\tau }^2}{2}\sum _{p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}\frac{n!(n-p)!{(x-x^{\prime })}^{n-p-m}}{(n-p)!(n-p-m)!}{D}_p{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & +\frac{{\tau }^2}{2}\sum _{p,m=1}^{\infty }\sum _{s=0}^{m-1}\underset{-\infty }{\overset{\infty }{\int }}\frac{n!{(x-x^{\prime })}^{n-p-s}}{(n-p-s)!}\left(\genfrac{}{}{0pt}{}{m}{s}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-s}{D}_p\right]D_m\delta (x-x^{\prime })\mathrm{d}x.\hfill \end{array}$$

The first integral is only non-vanishing if $n-p-m=0$ and the second integral is only non-vanishing if $n-p-s=0$, with $s<n$. Hence,

$$\begin{array}{cc}\hfill M_n^{\left[2\right]}(x^{\prime },\tau )& =M_n^{\left[1\right]}(x^{\prime },\tau )+\frac{{\tau }^2}{2}(n!)\sum _{m=1}^{n-1}{D}_{n-m}\left(x^{\prime }\right)D_m\left(x^{\prime }\right)\hfill \\ \hfill & +\frac{{\tau }^2}{2}(n!)\sum _{s=0}^{n-1}\sum _{m=s+1}^{\infty }\left(\genfrac{}{}{0pt}{}{m}{s}\right)\left[{\left(\frac{\partial }{\partial x^{\prime }}\right)}^{m-s}{D}_{n-s}\left(x^{\prime }\right)\right]D_m\left(x^{\prime }\right).\hfill \end{array}$$

Separating the terms between those with explicit derivatives of the KM coefficients and those without, it is immediately clear that the second-order approximation follows a structure given by the partial ordinary Bell polynomials ${\widehat{B}}_{n,m}$ [37]

$${\widehat{B}}_{n,m}(x_1,x_2,\dots ,x_{n-m+1})=\sum \frac{m!}{j_1!j_2!\cdots j_{n-m+1}!}{x}_1^{j_1}{x}_2^{j_2}\cdots x_{n-m+1}^{j_{n-m+1}}.$$

(6)

where the summation is taken over $j_1,\cdots ,j_{n-m+1}\in \{0,1,2,\dots ,n-m+1\}$ such that

$$\sum _{r=1}^{n-m+1}{j}_r=m\mathrm{and}\sum _{r=1}^{n-m+1}r{j}_r=n.$$

(7)

The first- and second-order approximations can be written with the help of the partial ordinary Bell polynomials with $m=1$ and $m=2$, respectively,

$$\begin{array}{cc}\hfill M_n^{\left[1\right]}(x^{\prime },\tau )& =(n!)\tau {\widehat{B}}_{n,1}\left(D_1,\dots ,D_n\right),\hfill \\ \hfill M_n^{\left[2\right]}(x^{\prime },\tau )& =(n!)\left[\tau {\widehat{B}}_{n,1}\left(D_1,\dots ,D_n\right)+\frac{{\tau }^2}{2}{\widehat{B}}_{n,2}\left(D_1,\dots ,D_{n-1}\right)+{\mathsf{\Phi }}_n^{\left[2\right]}\right],\hfill \end{array}$$

(8)

where ${\mathsf{\Phi }}_n^{\left[2\right]}$ incorporates all derivatives of the KM coefficients from the 2nd-order corrections, and is given by

$$\begin{array}{c}\hfill {\mathsf{\Phi }}_n^{\left[2\right]}=\frac{{\tau }^2}{2}\sum _{s=0}^{n-1}\sum _{m=s+1}^{\infty }\left(\genfrac{}{}{0pt}{}{m}{s}\right)\left[{\left(\frac{\partial }{\partial x^{\prime }}\right)}^{m-s}{D}_{n-s}\left(x^{\prime }\right)\right]D_m\left(x^{\prime }\right).\end{array}$$

To simplify the description, we introduce a short-hand notation and take the superscript $\left(m\right)$ in the KM coefficients: $D_p^{\left(m\right)}\left(x^{\prime }\right)={\left(\frac{\partial }{\partial x^{\prime }}\right)}^m{D}_p\left(x^{\prime }\right)$.

These results are in line with those reported for diffusion-type processes [16,17,18,19,38], where the Kramers–Moyal operator ${\mathcal{L}}_{\mathrm{KM}}={\mathcal{L}}_{\mathrm{FP}}$ reduces to the Fokker–Planck operator and we are solely left with the first two KM coefficients, as in Equation (2). In particular, applying the second-order approximation in Equation (8) to the two first KM coefficients results in

$$\begin{array}{cc}\hfill M_1^{\left[2\right]}& =\tau D_1+\frac{{\tau }^2}{2}\sum _{m=1}^{\infty }{D}_m{D}_1^{\left(m\right)},\hfill \\ \hfill M_2^{\left[2\right]}& =2\tau D_2+{\tau }^2{D}_1^2+{\tau }^2\left[\sum _{m=1}^{\infty }{D}_m{D}_2^{\left(m\right)}+\sum _{m=2}^{\infty }m{D}_m{D}_1^{(m-1)}\right],\hfill \end{array}$$

and truncating the sums at second order yields the expressions in reference [16].

3.2. The Third-Order Approximation

Before we introduce the general formalism for the arbitrary-order approximation, we explicitly derive the third-order approximation

$$p(x,t+\tau |x^{\prime },t)=\left[1+\tau {\mathcal{L}}_{\mathrm{KM}}+\frac{{\tau }^2}{2}{\mathcal{L}}_{\mathrm{KM}}{\mathcal{L}}_{\mathrm{KM}}+\frac{{\tau }^3}{6}{\mathcal{L}}_{\mathrm{KM}}{\mathcal{L}}_{\mathrm{KM}}{\mathcal{L}}_{\mathrm{KM}}+\mathcal{O}\left({\tau }^4\right)\right]\delta (x-x^{\prime }),$$

which leads to

$$\begin{array}{cc}\hfill M_n^{\left[3\right]}(x^{\prime },\tau )=& M_n^{\left[1\right]}(x^{\prime },\tau )+M_n^{\left[2\right]}(x^{\prime },\tau )\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{q,p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n{\left(-\frac{\partial }{\partial x}\right)}^q{D}_q{\left(-\frac{\partial }{\partial x}\right)}^p{D}_p{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill =& M_n^{\left[1\right]}(x^{\prime },\tau )+M_n^{\left[2\right]}(x^{\prime },\tau )\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{q,p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}\frac{n!}{(n-q-p-m)!}{(x-x^{\prime })}^{n-q-p-m}{D}_q{D}_p{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{q=1}^n\sum _{p,m=1}^{\infty }\sum _{s=0}^p\underset{-\infty }{\overset{\infty }{\int }}\frac{n!{(x-x^{\prime })}^{n-q-s}}{(n-q-s)!}\left(\genfrac{}{}{0pt}{}{p}{s}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p-s}{D}_q\right]\hfill \\ \hfill & \times D_p{\left(-\frac{\partial }{\partial x}\right)}^m{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill =& M_n^{\left[1\right]}(x^{\prime },\tau )+M_n^{\left[2\right]}(x^{\prime },\tau )\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{q,p,m=1}^{\infty }\underset{-\infty }{\overset{\infty }{\int }}\frac{n!}{(n-q-p-m)!}{(x-x^{\prime })}^{n-q-p-m}{D}_q{D}_p{D}_m\delta (x-x^{\prime })\mathrm{d}x\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{q=1}^n\sum _{p,m=1}^{\infty }\sum _{s=0}^p\sum _{k=0}^m\sum _{r=0}^{m-k}\underset{-\infty }{\overset{\infty }{\int }}\frac{n!{(x-x^{\prime })}^{n-q-s-k}}{(n-q-s-r)!}\left(\genfrac{}{}{0pt}{}{p}{s}\right)\left(\genfrac{}{}{0pt}{}{m}{k}\right)\left(\genfrac{}{}{0pt}{}{m-k}{r}\right)\hfill \\ \hfill & \times \left[{\left(-\frac{\partial }{\partial x}\right)}^{p-s+r}{D}_q\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-k-r}{D}_p\right]D_m\delta (x-x^{\prime })\mathrm{d}x.\hfill \end{array}$$

Notice that the first integral is only non-vanishing for the combination $q+p+m=n$, which can again be expressed via the partial ordinary Bell polynomial ${\widehat{B}}_{n,m}$, where $m=3$, for the third-order approximation. The second expression requires $q+s+k=n$ as well as $p+r\ne s\wedge m-r\ne k$. Separating these again into two expressions, one with and another without derivatives, we can express the third-order approximation as

$$\begin{array}{cc}\hfill M_n^{\left[3\right]}(x^{\prime },\tau )& =(n!)\left[\tau {\widehat{B}}_{n,1}\left(D_1,\dots ,D_n\right)+\frac{{\tau }^2}{2}{\widehat{B}}_{n,2}\left(D_1,\dots ,D_{n-1}\right)+{\mathsf{\Phi }}_n^{\left[2\right]}\right.\hfill \\ & \left.+\frac{{\tau }^3}{6}{\widehat{B}}_{n,3}\left(D_1,\dots ,D_{n-2}\right)+{\mathsf{\Phi }}_n^{\left[3\right]}\right],\hfill \end{array}$$

(9)

where ${\mathsf{\Phi }}_n^{\left[3\right]}$ incorporates all derivatives of the KM coefficients from the third-order corrections

$$\begin{array}{cc}\hfill {\mathsf{\Phi }}_1^{\left[3\right]}& =\frac{{\tau }^3}{6}\sum _{p,m=1}^{\infty }\sum _{r=0}^m\left(\genfrac{}{}{0pt}{}{m}{r}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p+r}{D}_1\left(x\right)\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-r}{D}_p\left(x\right)\right]D_m\left(x\right).\hfill \end{array}$$

(10)

Here, we compare our derivation to the derivation of third-order approximation in Gottschall and Peinke [16]. We note that our derivation takes the general form of the Kramers–Moyal operator, to which the Fokker–Planck operator is circumscribed. From Equation (9), we derive an identical expression for the Fokker–Planck operator reported in reference [16]. Since the Fokker–Planck operator is limited to second-order terms, i.e., $D_n\equiv 0$ for $n\ge 3$, the sum in Equation (10) can be express in full. For the first conditional moment $M_1^{\left[3\right]}$, we obtain the corrective terms ${\tilde{\mathsf{\Phi }}}_1^{\left[3\right]}$ given by

$$\begin{array}{cc}\hfill {\tilde{\mathsf{\Phi }}}_1^{\left[3\right]}& =\frac{{\tau }^3}{6}\sum _{p,m=1}^2\sum _{r=0}^m\left(\genfrac{}{}{0pt}{}{m}{r}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p+r}{D}_1\left(x\right)\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-r}{D}_p\left(x\right)\right]D_m\left(x\right).\hfill \\ \hfill & =\frac{{\tau }^3}{6}\left[D_1^{\left(1\right)}{D}_1^{\left(1\right)}{D}_1+D_1^{\left(2\right)}{D}_1{D}_1+3D_1^{\left(1\right)}{D}_1^{\left(2\right)}{D}_2+2D_1^{\left(3\right)}{D}_1{D}_2\right.\hfill \\ \hfill & \left.+D_1^{\left(2\right)}{D}_2^{\left(1\right)}{D}_1+D_1^{\left(2\right)}{D}_2^{\left(2\right)}{D}_2+2D_1^{\left(3\right)}{D}_2^{\left(1\right)}{D}_2+D_1^{\left(4\right)}{D}_1{D}_1\right],\hfill \end{array}$$

which is identical to Equation (A1) in the Appendix of reference [16]. Similarly, for the second conditional moment $M_2^{\left[3\right]}$, we obtain the corrective terms ${\tilde{\mathsf{\Phi }}}_2^{\left[3\right]}$

$$\begin{array}{cc}\hfill {\tilde{\mathsf{\Phi }}}_2^{\left[3\right]}& =\frac{{\tau }^3}{6}\sum _{p,m=1}^2\sum _{r=0}^{m}p\left(\genfrac{}{}{0pt}{}{m}{r}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p+r-1}{D}_1\left(x\right)\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-r}{D}_p\left(x\right)\right]D_m\left(x\right)\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{p,m=1}^2\sum _{r=0}^{m-1}m\left(\genfrac{}{}{0pt}{}{m-1}{r}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p+r}{D}_1\left(x\right)\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-r-1}{D}_p\left(x\right)\right]D_m\left(x\right)\hfill \\ \hfill & +\frac{{\tau }^3}{6}\sum _{p,m=1}^2\sum _{r=0}^m\left(\genfrac{}{}{0pt}{}{m}{r}\right)\left[{\left(-\frac{\partial }{\partial x}\right)}^{p+r}{D}_2\left(x\right)\right]\left[{\left(-\frac{\partial }{\partial x}\right)}^{m-r}{D}_p\left(x\right)\right]D_m\left(x\right)\hfill \\ \hfill & =\frac{{\tau }^3}{3}\left[3D_1{D}_1^{\left(1\right)}{D}_1+7D_1^{\left(2\right)}{D}_1{D}_2+4D_1^{\left(1\right)}{D}_1^{\left(1\right)}{D}_2+3D_1^{\left(1\right)}{D}_2^{\left(1\right)}{D}_1\right.\hfill \\ \hfill & +4D_1^{\left(1\right)}{D}_2^{\left(2\right)}{D}_2+7D_1^{\left(2\right)}{D}_2^{\left(1\right)}{D}_2+4D_1^{\left(3\right)}{D}_2{D}_2+D_2^{\left(2\right)}{D}_1{D}_1\hfill \\ \hfill & \left.+2D_2^{\left(3\right)}{D}_2{D}_1+D_2^{\left(2\right)}{D}_2^{\left(1\right)}{D}_1+D_2^{\left(2\right)}{D}_2^{\left(2\right)}{D}_2+2D_2^{\left(3\right)}{D}_2^{\left(1\right)}{D}_2+D_2^{\left(4\right)}{D}_2{D}_2\right]\hfill \end{array}$$

which is in agreement with Equation (A2) in the Appendix of reference [16]. A similar derivation can be found in Appendix B of reference [8], which also yields congruent findings for the first two conditional moments of jump-diffusion processes. However, no explicit expression for all terms is given in either publication.

As a simple rule of thumb, one can confer if the result is correct, as follows: the sum of the order of the KM coefficients subtracted by the derivation operation must equal n, the order of the conditional moment being calculated. In the notation used in this work, the sum of subscripts minus the sum of superscripts must equal the order n of the coefficient under investigation.

3.3. Arbitrary-Order Approximation

We now derive the arbitrary-order corrections of the Kramers–Moyal operator. This is done by induction from the previous derivations, whilst disregarding any emerging terms with derivatives of the KM coefficients

$$M_n^{\left[m\right]}(x^{\prime },\tau )=\underset{-\infty }{\overset{\infty }{\int }}{(x-x^{\prime })}^n\sum _{k=1}^m\frac{{\tau }^k}{k!}{\mathcal{L}}_{\mathrm{KM}}^k\delta (x-x^{\prime })\mathrm{d}x=n!\sum _{k=1}^m\left[\frac{{\tau }^k}{k!}\prod _{\sigma (k,n)}^m{D}_{\sigma (k,n)}+{\mathsf{\Phi }}_n^{\left[k\right]}\right],$$

with $\sigma (k,n)$ a partition of a set of $k\in \mathbb{N}$ obeying Equation (7). This, in turn, is the same as a collection of partial Bell polynomials, namely

$$M_n^{\left[m\right]}(x^{\prime },\tau )=(n!)\sum _{k=1}^m\left[\frac{{\tau }^k}{k!}{\widehat{B}}_{n,k}(D_1,D_2,\dots ,D_{n-k+1})+{\mathsf{\Phi }}_n^{\left[k\right]}\right],$$

where we combine terms with derivatives in ${\mathsf{\Phi }}_n^{\left[k\right]}$. If we disregard the derivative terms, the summation has an upper bound, namely $m\le n$. This is directly seen as the Bell polynomials are similarly bounded, and thus we arrive at

$$M_n(x^{\prime },\tau )=(n!)\sum _{k=1}^n\frac{{\tau }^k}{k!}{\widehat{B}}_{n,k}(D_1,D_2,\dots ,D_{n-k+1}).$$

(11)

neglecting the derivative terms $\mathsf{\Phi }$.

From the perspective of estimation, the aim is to determine the KM coefficients $D_n\left(x^{\prime }\right)$, however what we have expressed here is the relation of the conditional moments $M_n(x^{\prime },\tau )$. As we now have an explicit relation in terms of partial Bell polynomials, we will invert the relation and express the KM coefficients $D_n\left(x^{\prime }\right)$ as functions of the conditional moments $M_1(x^{\prime },\tau ),\dots ,M_n(x^{\prime },\tau )$.

Note that the first conditional moment $M_1(x^{\prime },\tau )$ is solely a function of the first KM coefficient $D_1\left(x^{\prime }\right)$. The second conditional moment $M_2(x^{\prime },\tau )$ is a function of the second KM coefficient $D_2\left(x^{\prime }\right)$, and by substitution, a function of the first conditional moment $M_1(x^{\prime },\tau )$, given by Equation (11). Subsequently $M_3(x^{\prime },\tau )$ is a function of $D_3\left(x^{\prime }\right)$, $M_2(x^{\prime },\tau )$, and $M_1(x^{\prime },\tau )$. Thus, by recursively substituting the $n-1$ KM coefficients by their expressions via the conditional moments, we obtain a relation of $D_n\left(x^{\prime }\right)$ as a function of the $M_n(x^{\prime },\tau ),M_{n-1}(x^{\prime },\tau ),\dots ,M_1(x^{\prime },\tau )$ conditional moments.

To this end, we rewrite Equation (11) in terms of the partial exponential Bell polynomials $B_{n,m}$

$$B_{n,m}(x_1,x_2,\dots ,x_{n-m+1})=\sum \frac{n!}{j_1!j_2!\cdots j_{n-m+1}!}{\left(\frac{x_1}{1!}\right)}^{j_1}{\left(\frac{x_2}{2!}\right)}^{j_2}\cdots {\left(\frac{x_{n-m+1}}{(n-m+1)!}\right)}^{j_{n-m+1}},$$

where the summation terms obey the constraints of the Bell polynomials given in Equation (7). This can be expressed through the partial ordinary Bell polynomials in Equation (6) as

$${\widehat{B}}_{n,m}(x_1,x_2,\dots ,x_{n-m+1})=\frac{m!}{n!}{B}_{n,m}(1!·x_1,2!·x_2,\dots ,(n-m+1)!·x_{n-m+1}).$$

Thus, Equation (11) reads

$$M_n(x^{\prime },\tau )=\sum _{k=1}^n{B}_{n,k}(1!\tau D_1,2!\tau D_2,\dots ,(n-k+1)!\tau D_{n-k+1}).$$

We can then utilise the reciprocal relations of the partial exponential Bell polynomials: for a set of variables $y_1,\cdots ,y_n$, defined as functions of n other variables $x_1,\cdots ,x_n$ given by

$$y_n=\sum _{k=1}^n{B}_{n,k}(x_1,x_2,\dots ,x_{n-k+1}),$$

(12)

the inverse relation holds

$$x_n=\sum _{k=1}^n{(-1)}^{k-1}(k-1)!B_{n,k}(y_1,y_2,\dots ,y_{n-k+1}).$$

(13)

With this, we can finally express any KM coefficients $D_n\left(x^{\prime }\right)$ from the nth-order power series expansion, neglecting the derivative terms $\mathsf{\Phi }$, as 

$$D_n(x^{\prime },\tau )=\frac{1}{n!}\frac{1}{\tau }\sum _{k=1}^n{(-1)}^{k-1}(k-1)!B_{n,k}(M_1,M_2,\dots ,M_{n-k+1}).$$

(14)

We note here that these relations are equivalent to the relation between cumulants and (non-central) moments of a probability distribution [13,36]. Let $\mathcal{M}\left(y\right)$ be the moment-generating function, such that

$$\mathcal{M}\left(y\right)=1+\sum _{n=1}^{\infty }\frac{{\mu }_n^{\prime }{y}^n}{n!}=\exp \left[\sum _{n=1}^{\infty }\frac{{\kappa }_n{y}^n}{n!}\right]=\exp \left[\mathcal{K}\left(y\right)\right],$$

with ${\mu }_n^{\prime }$ the (non-central) moments and $\mathcal{K}\left(x\right)$ the cumulant-generating function. For $n<4$, the cumulants ${\kappa }_n$ and the central moments are the same (e.g., the mean and variance). This is not the same for higher cumulants and moments. The relation between the cumulants ${\kappa }_n$ and the (non-central) moments ${\mu }_n^{\prime }$ is given by the reciprocal relation of the Bell polynomials, as in Equations (12) and (13). This is in line with our exponential representation of the Kramers–Moyal operator. Here, the KM coefficients are the cumulants (with the exception of the $\tau$ term).

4. Exemplary Cases with Constant Diffusion and Constant Jumps

Here, we present two illustrative examples: first, a constant diffusion process, the  Ornstein–Uhlenbeck process; secondly, we augment this process with jumps to obtain a jump-diffusion process. We implement the corrective terms derived thus far to show the impact of the finite-time corrections. This choice of parameters, i.e., constant diffusion and constant jumps, considerably simplifies Equation (1) to

$$\mathrm{d}{X}_t=-a{X}_t\mathrm{d}t+b\mathrm{d}W\left(t\right)+\xi \mathrm{d}J\left(t\right),$$

(15)

where $-a{X}_t$ is the state-dependent linear drift function, with $a>0$, also denoted mean-reverting strength, $b>0$ a constant diffusion, $W\left(t\right)$ a Brownian motion or Wiener process, $\xi$ a state-independent and normally distributed jump amplitude with zero mean and variance s, and $J\left(t\right)$ a Poisson process with jump rate $\lambda$. Note that the conventional Ornstein–Uhlenbeck process is recovered if we omit the jump process.

We have derived an expression for the conditional moments $M_n(x,\tau )$ as a function of the KM coefficients $D_n\left(x\right)$, given by Equation (11), which is valid for any Markovian diffusion or jump-diffision process. For our particular application to the Poissonian jump-diffusion process in Equation (1) we require at least the first six KM coefficients/first six moments. These are given by

$$\begin{array}{cc}\hfill M_1& =\tau D_1,\hfill \\ \hfill M_2& =2\tau D_2+{\tau }^2{D}_1^2,\hfill \\ \hfill M_3& =6\tau D_3+6{\tau }^2{D}_1{D}_2+{\tau }^3{D}_1^3,\hfill \\ \hfill M_4& =24\tau D_4+12{\tau }^2\left(2D_1{D}_3+D_2^2\right)+12{\tau }^3{D}_1^2{D}_2+{\tau }^4{D}_1^4,\hfill \\ \hfill M_5& =120\tau D_5+120{\tau }^2\left(D_1{D}_4+D_2{D}_3\right)+60{\tau }^3\left(D_1^2{D}_3+D_1{D}_2^2\right)+20{\tau }^4{D}_1^3{D}_2+{\tau }^5{D}_1^5,\hfill \\ \hfill M_6& =720\tau D_6+360{\tau }^2\left(2D_1{D}_5+2D_2{D}_4+D_3^2\right)+120{\tau }^3\left(3D_1^2{D}_4+6D_1{D}_2{D}_3+D_2^3\right)\hfill \\ \hfill & +60{\tau }^4\left(2D_1^3{D}_3+3D_1^2{D}_2^2\right)+30{\tau }^5{D}_1^4{D}_2+{\tau }^6{D}_1^6.\hfill \end{array}$$

We invert this expression explicitly using Equation (14) and report on the KM coefficients as functions of the conditional moments, which are given by

$$\begin{array}{cc}\hfill D_1& =\frac{1}{1!}\underset{\tau \to 0}{lim}\frac{1}{\tau }{M}_1,\hfill \\ \hfill D_2& =\frac{1}{2!}\underset{\tau \to 0}{lim}\frac{1}{\tau }\left[M_2-M_1^2\right],\hfill \\ \hfill D_3& =\frac{1}{3!}\underset{\tau \to 0}{lim}\frac{1}{\tau }\left[M_3-3M_1{M}_2+2M_1^3\right],\hfill \\ \hfill D_4& =\frac{1}{4!}\underset{\tau \to 0}{lim}\frac{1}{\tau }\left[M_4-4M_1{M}_3-3M_2^2+12M_1^2{M}_2-6M_1^4\right],\hfill \\ \hfill D_5& =\frac{1}{5!}\underset{\tau \to 0}{lim}\frac{1}{\tau }\left[M_5-5M_1{M}_4-10M_2{M}_3+30M_1{M}_2^2+20M_1^2{M}_3\right.\hfill \\ \hfill & \left.-60M_1^3{M}_2+24M_1^5\right],\hfill \\ \hfill D_6& =\frac{1}{6!}\underset{\tau \to 0}{lim}\frac{1}{\tau }\left[M_6-6M_1{M}_5-10M_3^2-15M_2{M}_4+30M_2^3+120M_1{M}_2{M}_3\right.\hfill \\ \hfill & \left.+30M_1^2{M}_4-270M_1^2{M}_2^2-120M_1^3{M}_3+360M_1^4{M}_2-120M_1^6\right].\hfill \end{array}$$

(16)

We again note that these expressions are valid for any case of diffusion and jump-diffusion processes. In the first case, where there are no jump terms in Equation (15), i.e., the Ornstein–Uhlenbeck process, we know that all KM coefficients $D_n\left(x\right)$ with $n\ge 3$ are zero. However, this is not the case when estimating the coefficients from time-series data, i.e., from one realisation of the stochastic process sampled at finite resolution. It is common to find that these terms do not vanish due to finite-time effects. In our second case with a jump-diffusion process, the KM coefficients $D_n\left(x\right)$ with $n\ge 3$ can be related directly to the jump parameters. These relations were derived in reference [3], and are given by

$$\begin{array}{cc}\hfill D_1\left(x\right)& =a\left(x\right),\hfill \\ \hfill D_2\left(x\right)& =\frac{1}{2}\left[b{\left(x\right)}^2+s\lambda \right],\hfill \\ \hfill D_{2n}\left(x\right)& =\frac{s^n\lambda }{2^n(n!)},\hfill \end{array}$$

(17)

where $\langle {\xi }^{2n}\rangle =\frac{(2n!)}{2^n(n!)}{\langle {\xi }^2\rangle }^n=\frac{(2n!)}{2^n(n!)}{s}^n$, for Gaussian distributions with zero mean and variance s.

We will now compare the derived theoretical corrections to KM coefficients estimated from numerically generated time-series data. In Figure 1 and Figure 2, we display the second-, fourth-, and sixth-order KM coefficients $D_2\left(x\right)$, $D_4\left(x\right)$, and $D_6\left(x\right)$ estimated with the first-order, second-order, and full-order approximations given by Equation (16) (or in general Equation (14)). The full-order approximations have the same order as the KM coefficients, i.e, second-, fourth-, and sixth-order approximation for $D_2\left(x\right)$, $D_4\left(x\right)$, and $D_6\left(x\right)$, respectively. For the data shown in Figure 1, we use a Euler–Maruyama scheme to numerically integrate an Ornstein–Uhlenbeck process Equation (15) (without the jump terms) with parameters: drift $a=1.0$ and diffusion $b=0.5$ ($\lambda =s=0.0$). We numerically integrate this process with a coarse time-step $\Delta t=0.1$ to deliberately emphasise the finite-time effects on the aforementioned KM coefficients. For example, the second-order KM coefficients $D_2\left(x\right)$ takes a quadratic form, despite the fact that the diffusion term is constant. The KM coefficients $D_4\left(x\right)$ and $D_6\left(x\right)$ are not truly zero, as would be expected for purely diffusive processes [39,40], due to the finite-time effects, but the full-order finite-time correction approximates the theoretical values with far greater detail.

For the data shown in Figure 2 we follow a similar approach, now augmenting the Ornstein–Uhlenbeck process with Poissonian jumps, i.e., as given in Equation (15). The parameters are as follows: drift $a=0.5$, diffusion $b=0.5$, jump amplitude with a Gaussian distribution with variance $s=0.75$ and zero mean, a Poissonian jump rate $\lambda =0.6$, and a time step $\Delta t=0.05$. For this process, we know the higher-order KM coefficients $D_4\left(x\right)$ and $D_6\left(x\right)$ reflect the presence of discontinuous paths, which, for our particular case of the Poissonian jump-Ornstein–Uhlenbeck process, we know the explicit inversion in Equation (17) (cf. reference [3]). For the chosen coarse time step, we notice that the estimations do not correspond exactly with the theoretical values, regardless of the order of finite-time correction chosen. This can likely be traced back to the limitations of the Kramers–Moyal equation to fully capture discontinuous stochastic processes (cf. reference [41]). Nevertheless, the higher-order finite-time corrections approximate the theoretical values with greater accuracy.

We note here that the parameter estimation from data heavily depends on the number of data points and the sampling rate of numerically simulated or real-world time-series data. Real-world time-series data can often be sampled at higher sampling rates, but not always in such a large number of datapoints. A closer inspection of the limitations of both the sampling rate and the number of data points in parameter estimation is necessary, but falls outside the scope of this publication. Moreover, it should be emphasised that, prior to any examination of time-series data within the purview of either the Fokker–Planck or the Kramers–Moyal equation, the Markov property of the data must be account for, i.e., a vanishing memory of the increments of the data. This can be examined, for example, via the Chapman–Kolmogorov equality [13].

Summarising our findings, we conclude that our proposed arbitrary-order finite-time corrections considerably help in differentiating one-dimensional purely diffusive processes and jump-diffusion processes, as these accurately show that higher-order KM coefficients $D_n\left(x\right)$, $n\ge 2$ vanish for purely diffusive processes. These arbitrary-order finite-time corrections should now also be considered for N-dimensional stochastic processes. A first examination of the second-order finite-time corrections for two-dimensional processes was recently addressed in reference [22]. Note that the one-dimensional second-order finite-time correction for these KM coefficients was recently addressed in another publication [34]. Here, it is extended to arbitrary order.

5. Implementation: Symbolic Calculations in Python

In this section, we implement the main results from above to compute the moments into available software packages, e.g., kramersmoyal [33] and JumpDiff [34] in Python or Langevin [35] in R, or any self-made parametric or non-parametric estimator. In order to facilitate numerical implementations of the higher-order corrections, we include a short Python script to obtain the non-derivative corrections to any desired order, with the desired truncation of the power-series expansion.

First, we present a Python code to numerically generate the conditional moments $M_n(x^{\prime },\tau )$ as functions of the KM coefficients $D_n\left(x^{\prime }\right)$, as given in Equation (14). Here the parameter n indicates the order of the KM coefficients/moments n and the parameter m the order of the correction, with m ≤ n. We utilise Python’s symbolic language library sympy [42].

To generate the KM coefficients $D_n\left(x^{\prime }\right)$ as function of the conditional moments, the following must be implemented:

6. Conclusions

We have presented a set of arbitrary-order finite-time corrections to the Kramers–Moyal operator, solved by exponentiating the Kramers–Moyal operator, equivalent to van Kampen’s system-size expansion. We expressed the exponential operator as a power series and worked out each element of the series, ultimately combining it in a series representation via the partial Bell polynomials. We obtained a closed form for the set of arbitrary-order finite-time corrections relating the conditional moments to the Kramers–Moyal coefficients. Moreover, by representing the arbitrary-order finite-time corrections with partial Bell polynomials, we derived a reciprocal relation for the conditional moments and the Kramers–Moyal coefficients. This provided a closed-form representation of the Kramers–Moyal coefficients via conditional moments, which is crucial for time-series data estimation. We included two illustrative cases of poorly sampled diffusion and jump-diffusion processes with constant diffusion and constant jumps, demonstrating the suitability of our corrections for a non-parametric estimation of higher-order Kramers–Moyal coefficients. Our corrections approximated the theoretical values with a high degree of accuracy and help to distinguish processes with and without jumps. We are confident that our arbitrary-order finite-time corrections contribute to an improved reconstruction of stochastic evolution equations from empirical time-series data.