State and Parameter Estimation from Observed Signal Increments

Abstract

The success of the ensemble Kalman filter has triggered a strong interest in expanding its scope beyond classical state estimation problems. In this paper, we focus on continuous-time data assimilation where the model and measurement errors are correlated and both states and parameters need to be identified. Such scenarios arise from noisy and partial observations of Lagrangian particles which move under a stochastic velocity field involving unknown parameters. We take an appropriate class of McKean–Vlasov equations as the starting point to derive ensemble Kalman–Bucy filter algorithms for combined state and parameter estimation. We demonstrate their performance through a series of increasingly complex multi-scale model systems.

1. Introduction

The research presented in this paper has been motivated by the state and parameter estimation problem for particles moving under a stochastic velocity field, with the measurements given by partial and noisy observations of their position increments. If the deterministic contributions to the velocity field are stationary, and the position increments of the moving particle are exactly observed, then one is led to a standard parameter estimation problem for stochastic differential equations (SDEs) [1,2]. In [3], this setting was extended to the case where the deterministic contributions to the velocity field themselves undergo a stochastic time evolution. Furthermore, while continuous-time observations of position increments are at the focus of the present study, the assimilation of discrete-time observations of particle positions has been investigated in [4,5] under a so-called Lagrangian data assimilation setting for atmospheric fluid dynamics.

The assumption of exactly and fully observed position increments is not always realistic and the case of partial and noisy observations is at the center of the present study. With access to partial and noisy observations of position increments leads to correlations between the measurement and model errors. The theoretical impact of such correlations on state and parameter estimation problems has been discussed, for example, in [6] in the context of linear systems, and in [7] for nonlinear systems. In particular, one finds that the appropriately adjusted data likelihood involves the gradient of log-densities, which is nontrivial from a computational perspective, and which prevents a straightforward application of standard Markov chain Monte Carlo (MCMC) or sequential Monte Carlo (SMC) methods [8].

In this paper, we instead follow an alternative Monte Carlo approach based on appropriately adjusted McKean–Vlasov filtering equations, an approach pioneered in [9] in the context of the standard state estimation problem for diffusion processes. McKean–Vlasov equations, first studied in [10], are a class of SDEs in which the right-hand side depends on the law of the process itself. We rely on a particular formulation of McKean–Vlasov filtering equations, the so-called feedback particle filters [11], utilising stochastic innovation processes [12].

Our proposed Monte Carlo formulation avoids the need for estimating log-densities, and can be implemented in a numerically robust manner relying on a generalised ensemble Kalman–Bucy filter approximation applied to an extended state space formulation [13]. The ensemble Kalman–Bucy filter [14,15] has been introduced previously as an extension of the popular ensemble Kalman filter [13,16,17] to continuous-time data assimilation under the assumption of uncorrelated measurement and model errors.

While the McKean-Vlasov formulation is essentially mathematically equivalent to the more conventional one based on the Kushner-Stratonovitch equation [7], these two approaches differ significantly in structure, suggesting different tools for their analysis as well as numerical approximations. More broadly speaking, the McKean–Vlasov approach to filtering is appealing since its Monte Carlo implementations completely avoid the need for resampling characteristic of standard SMC methods. Furthermore, a wide range of approximations are possible within the McKean–Vlasov framework with some of them, such as the ensemble Kalman–Bucy filter, applicable to high-dimensional problems. The McKean–Vlasov approach also arises naturally when analysing sequential Monte Carlo methods [18].

In Section 6, we apply the proposed algorithms to a series of state and parameter estimation problems of increasing complexity. First, we study the state and parameter estimation problem for an Ornstein–Uhlenbeck process [2]. Two further experiments investigate the behaviour of the filters for reduced model equations, with the data being collected from underlying multi-scale models. There we distinguish between the averaging and homogenisation scenarios [19]. Finally, we look at examples of nonparametric drift estimation [3] and parameter estimation for the stochastic heat equation [20].

We finally mention that SMC methods for correlated noise terms in discrete-time have been discussed, for example, in [21] and in the context of the ensemble Kalman filter in [22]. Similar ideas have also been pursued in a more applied context in [23].

2. Mathematical Problem Formulation

We consider the time evolution of a random state variable $X_t\in {\mathbb{R}}^{N_x}$ in $N_x$-dimensional state space, $N_x\ge 1$, as prescribed by an SDE of the form

$$\mathrm{d}{X}_t=f(X_t,a)\mathrm{d}t+G\mathrm{d}{W}_t,$$

(1)

for time $t\ge 0$, with the drift function $f:{\mathbb{R}}^{N_x}\times {\mathbb{R}}^{N_a}\to {\mathbb{R}}^{N_x}$ depending on $N_a\ge 0$ unknown parameters $a={(a^1,\dots ,a^{N_a})}^{\mathrm{T}}\in {\mathbb{R}}^{N_a}$. Model errors are represented through standard $N_w$-dimensional Brownian motion $W_t$, $N_w\ge 1$, and a matrix $G\in {\mathbb{R}}^{N_x\times N_w}$. We also introduce the associated model error covariance matrix $Q=G{G}^{\mathrm{T}}$. We will generally assume that the initial condition $X_0$ is fixed, that is, $X_0=x_0$ a.s. for given $x_0\in {\mathbb{R}}^{N_x}$. In terms of a more specific example, one can think of $X_t$ denoting the position of a particle at time $t\ge 0$ moving in $N_x=3$ dimensional space under the influence of a stochastic velocity field, with deterministic contributions given by f and stochastic perturbations by $G{W}_t$. In the case $G=0$, the SDE (1) reduces to an ordinary differential equation with given initial condition $x_0$.

We assume throughout this paper that (1) possesses unique, strong solutions for all parameter values a. See, for example, [2] (Section 3.3) for sufficient conditions on the drift function f. The distribution of $X_t$ is denoted by ${\pi }_t$, which we also abbreviate by ${\pi }_t=\mathrm{Law}(X_t)$. We use the same notation for measures and their Lebesgue densities, provided they exist.

Example 1.

A wide class of drift functions can be written in the form

$$f(x,a)=f_0(x)+B(x)a=f_0(x)+\sum _{i=1}^{N_a}{b}_i(x)a^i,$$

(2)

where $f_0:{\mathbb{R}}^{N_x}\to {\mathbb{R}}^{N_x}$ is a known drift function, the $b_i:{\mathbb{R}}^{N_x}\to {\mathbb{R}}^{N_x}$, $i=1,\dots ,N_a$, denote appropriate basis functions, and the vector $a={(a^1,\dots ,a^{N_a})}^{\mathrm{T}}\in {\mathbb{R}}^{N_a}$ contains the unknown parameters of the model. The family $\{b_i(x)\}$ of basis functions, which we collect in a matrix-valued function $B(x)=(b_1(x),b_2(x),\dots ,b_{N_a}(x))\in {\mathbb{R}}^{N_x\times N_a}$, could arise from a finite-dimensional truncation of some appropriate Hilbert space $\mathcal{H}$. See, for example, [24] for computational approaches to nonparametric drift estimation using a Galerkin approximation in $\mathcal{H}$, where the $b_i(x)$ become finite element basis functions. Furthermore, the expansion coefficients $\{a^i\}$ could be made time-dependent by letting them evolve according to some system of differential equations arising, for example, from the discretisation of an underlying partial differential equation with solutions in $\mathcal{H}$. See [3] for specific examples of such a setting. While the present paper focuses on stationary drift functions, i.e., the parameters $\{a^i\}$ are time-independent, the results from Section 3 and Section 5, respectively, can easily be extended to the non-stationary case where the parameters themselves satisfy given evolution equations.

Data and an observation model are required in order to perform state and parameter estimation for SDEs of the form (1). In this paper, we assume that we observe partial and noisy increments $\mathrm{d}{Y}_t$ of the signal $X_t$, given by

$$\mathrm{d}{Y}_t=H\mathrm{d}{X}_t+R^{1/2}\mathrm{d}{V}_t=Hf(X_t,a)\mathrm{d}t+HG\mathrm{d}{W}_t+R^{1/2}\mathrm{d}{V}_t,Y_0=X_0=x_0,$$

(3)

for t in the observation interval $[0,T]$, $T>0$, where $H\in {\mathbb{R}}^{N_y\times N_x}$ is a given linear operator, $V_t$ denotes standard $N_y$-dimensional Brownian motion with $N_y\ge 1$ and $R\in {\mathbb{R}}^{N_y\times N_y}$ is a covariance matrix. We introduce the observation map

$$h(x,a)=Hf(x,a)$$

(4)

for later use. Unless $HG=0$, it is clear that the model error $E_t^{\mathrm{m}}:=G{W}_t$ in (1) and the total observation error

$$E_t^{\mathrm{o}}:=HG{W}_t+R^{1/2}{V}_t$$

(5)

in (3) are correlated. The impact of correlations between the model and measurement errors on the state estimation problem have been discussed by [6,7]. Furthermore, such correlations require adjustments to sequential estimation methods [16,17,25] which are the main focus of this paper. We assume throughout this paper that the covariance matrix

$$C=HG{G}^{\mathrm{T}}{H}^{\mathrm{T}}+R=HQ{H}^{\mathrm{T}}+R$$

(6)

of the observation error (5) is invertible.

The special case $R=0$ and $H=I$ leads to a pure parameter estimation problem which has been extensively studied in the literature in the settings of maximum likelihood and Bayesian estimators [1,2]. In Section 3, we provide a reformulation of the Bayesian approach as McKean–Vlasov equations for the parameters, based on the results in [9,11].

If $R\ne 0$, then (1) and (3) lead to a combined state and parameter estimation problem with correlated noise terms. We will first discuss the impact of this correlation on the pure state estimation problem in Section 4 assuming that the parameters of the problem are known. Again, we will derive appropriate McKean–Vlasov equations in the state variables. Our key contribution is a formulation that avoids the need for log-density estimates, and can be put into an appropriately generalised ensemble Kalman–Bucy filter approximation framework [14,15]. We also formally demonstrate that the McKean–Vlasov filter equation reduces to $\mathrm{d}{X}_t=\mathrm{d}{Y}_t$ in the limit $R\to 0$ and $H=I$, a property that is less straightforward to demonstrate for filter formulations involving log-densities.

These McKean–Vlasov equations are generalised to the combined state and parameter estimation problem via an augmentation of state space [13] in Section 5. Given the results from Section 4, such an extension is rather straightforward.

The numerical experiments in Section 6 rely exclusively on the generalised ensemble Kalman–Bucy filter approximation to the McKean–Vlasov equations, which are easy to implement and yield robust and accurate numerical results.

3. Parameter Estimation from Noiseless Data

In this section, we treat the simpler Bayesian parameter estimation problem which arises from setting $R=0$ and $H=I$ in (3), i.e., $N_y=N_x$. This leads to $\mathrm{d}{X}_t=\mathrm{d}{Y}_t$ and, furthermore, $X_t=Y_t$ for all $t\in [0,T]$, provided $X_0=Y_0=x_0$ which we assume throughout this paper. The requirement that $C=Q$ is invertible requires that G has rank $N_x$; that is, $N_w\ge N_x$ in (1). The data likelihood

$$l_t(a)=exp\left({\int }_0^{t}f{(Y_s,a)}^{\mathrm{T}}{Q}^{-1}\mathrm{d}{Y}_s-\frac{1}{2}{\int }_0^{t}f{(Y_s,a)}^{\mathrm{T}}{Q}^{-1}f(Y_s,a)\mathrm{d}s\right)$$

(7)

thus follows from the observation model with additive Brownian noise in (3). Given a prior distribution ${\Pi }_0(a)$ for the parameters, the resulting posterior distribution at any time $t\in (0,T]$ is

$${\Pi }_t(a)=\frac{l_t(a){\Pi }_0(a)}{{\Pi }_0[l_t]}$$

(8)

according to Bayes’ theorem [7]. Here, we have introduced the shorthand

$${\Pi }_0[l_t]={\int }_{{\mathbb{R}}^{N_a}}{l}_t(a){\Pi }_0(a)\mathrm{d}a$$

(9)

for the expectation of $l_t$ with respect to ${\Pi }_0$. It is well-known that the posterior distributions ${\Pi }_t$ satisfy the stochastic partial differential equation

$$\mathrm{d}{\Pi }_t[\varphi ]={\left({\Pi }_t[\varphi h_t]-{\Pi }_t[\varphi ]{\Pi }_t[h_t]\right)}^{\mathrm{T}}{Q}^{-1}(\mathrm{d}{Y}_t-{\Pi }_t[h_t]\mathrm{d}t)$$

(10)

with the time-dependent observation map

$$h_t(a)=f(Y_t,a),$$

(11)

where $\varphi :{\mathbb{R}}^{N_a}\to \mathbb{R}$ is a compactly supported smooth test function, and ${\Pi }_t[\varphi ]$ again denoting the expectation of $\varphi$ with respect to ${\Pi }_t$. See [7] for a detailed discussion. Equation (10) is a special instance of the well-known Kushner–Stratonovitch equation from time-continuous filtering [7].

3.1. Feedback Particle Filter

We now state a McKean–Vlasov reformulation of the Kushner–Stratonovitch Equation (10) as a special instance of the feedback particle filter of [11,12]. The key idea is to formulate a stochastic differential equation in the parameters in which they are treated as time-dependent random variables. We introduce the notation ${\tilde{A}}_t$ for these, and require that the law of ${\tilde{A}}_t$ coincide with (8) for $t\in [0,T]$, i.e., with the solution to (10).

Lemma 1 (Feedback particle filter).

Consider the McKean–Vlasov equations

$$\mathrm{d}{\tilde{A}}_t=K_t({\tilde{A}}_t)\mathrm{d}{I}_t+{\Omega }_t({\tilde{A}}_t)\mathrm{d}t,$$

(12)

where the matrix-valued Kalman gain $K_t\in {\mathbb{R}}^{N_a\times N_y}$ satisfies

$$\nabla ·\left({\tilde{\Pi }}_t\left(K_{t}Q\right)\right)=-{\tilde{\Pi }}_t{\left(h_t-{\tilde{\Pi }}_t[h_t]\right)}^{\mathrm{T}},{\tilde{\Pi }}_t=\mathrm{Law}({\tilde{A}}_t).$$

(13)

The innovation process $I_t$ can be chosen to be given by either

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\frac{1}{2}\left(h_t({\tilde{A}}_t)+{\tilde{\Pi }}_t[h_t]\right)\mathrm{d}t,$$

(14)

or

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\left\{h_t({\tilde{A}}_t)\mathrm{d}t+G\mathrm{d}{W}_t\right\},$$

(15)

and

$${\Omega }_t^i=\frac{1}{2}\sum _{j=1}^{N_a}\sum _{k,l=1}^{N_y}{Q}^{kl}{K}_t^{jl}\left({\partial }_j{K}_t^{ik}\right),i=1,\dots ,N_a.$$

(16)

Then, the distribution ${\tilde{\Pi }}_t=Law({\tilde{A}}_t)$ coincides with the solution to (10), provided that the initial distributions agree. In other words, ${\tilde{\Pi }}_t={\Pi }_t$ for all $t\in [0,T]$.

Throughout this paper, we write (12) in the more compact Stratonovitch form

$$\mathrm{d}{\tilde{A}}_t=K_t({\tilde{A}}_t)\circ \mathrm{d}{I}_t,$$

(17)

where the Stratonovitch interpretation is to be applied only to ${\tilde{A}}_t$ in $K_t({\tilde{A}}_t)$, while the explicit time-dependence of $K_t$ remains in its Itô interpretation. It should be noted that the matrix-valued function $K_t$ is not uniquely defined by the PDE (13). Indeed, provided $K_t$ solves (13), $K_t+{\beta }_t$ is also a solution whenever $\nabla ·\left({\tilde{\Pi }}_t{\beta }_t\right)=0$. As discussed in [15], the minimiser over all suitable $K_t$ with respect to a kinetic energy-type functional is of the form

$$K_t=\nabla {\Psi }_t{Q}^{-1}$$

(18)

for a vector of potential functions ${\Psi }_t=({\psi }_t^1,\dots ,{\psi }_t^{N_x})$, ${\psi }_t^k:{\mathbb{R}}^{N_a}\to \mathbb{R}$. Inserting (18) into (13) leads to $N_x$ elliptic partial differential equations (often referred to as Poisson equations),

$$\nabla ·\left({\tilde{\Pi }}_t\nabla {\Psi }_t\right)=-{\tilde{\Pi }}_t{\left(h_t-{\tilde{\Pi }}_t[h_t]\right)}^{\mathrm{T}},{\tilde{\Pi }}_t[{\Psi }_t]=0,$$

(19)

understood component wise, where the centring condition ${\tilde{\Pi }}_t[{\Psi }_t]=0$ makes the solution unique under mild assumptions on ${\tilde{\Pi }}_t$ (see [26]). The numerical approximation of (19) in the context of the feedback particle filter has been discussed in [27]. Finally, (15) yields a particularly appealing formulation, since it is based on a direct comparison of $\mathrm{d}{Y}_t$ with a random realisation of the right hand side of the SDE (1), given a parameter value $a={\tilde{A}}_t(\omega )$ and a realisation of the noise term $\mathrm{d}{W}_t(\omega )$. This fact will be explored further in Section 4.

Remark 1.

For clarity, let us repeat Equations (13) and (18) in their index forms:

$$\sum _{i=1}^{N_a}\sum _{j=1}^{N_y}{\partial }_i\left({\tilde{\Pi }}_t\left(K_t^{ij}{Q}^{jk}\right)\right)=-{\tilde{\Pi }}_t\left(h_t^k-{\tilde{\Pi }}_t[h_t^k]\right),k=1,\dots ,N_y,$$

(20)

$$\sum _{j=1}^{N_y}{K}_t^{ij}(a)Q^{jk}={\partial }_i{\psi }_t^k(a),i=1,\dots ,N_a,k=1,\dots ,N_y.$$

(21)

3.2. Ensemble Kalman–Bucy Filter

Let us now assume that the initial distribution ${\Pi }_0$ is Gaussian, and that f is linear in the unknown parameters such as in (2). Then, the distributions ${\tilde{\Pi }}_t$ remain Gaussian for all times with mean ${\overline{a}}_t$ and covariance matrix $P_t^{aa}$. The elliptic PDE (13) is solved by the parameter-independent Kalman gain matrix

$$K_t=P_t^{aa}B{(Y_t)}^{\mathrm{T}}{Q}^{-1}$$

(22)

and one obtains the McKean–Vlasov formulation

$$\mathrm{d}{\tilde{A}}_t=P_t^{aa}B{(Y_t)}^{\mathrm{T}}{Q}^{-1}\mathrm{d}{I}_t$$

(23)

of the Kalman–Bucy filter, with the innovation process $I_t$ defined by either

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\left(f_0(Y_t)+\frac{1}{2}B(Y_t)({\tilde{A}}_t+{\overline{a}}_t)\right)\mathrm{d}t$$

(24)

or

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\left\{\left(f_0(Y_t)+B(Y_t){\tilde{A}}_t\right)\mathrm{d}t+G\mathrm{d}{W}_t\right\}.$$

(25)

Please note that the Stratonovitch formulation (17) reduces to the standard Itô interpretation, since $K_t$ no longer depends explicitly on ${\tilde{A}}_t$.

The McKean–Vlasov Equation (23) can be extended to nonlinear, non-Gaussian parameter estimation problems by generalising the parameter-independent Kalman gain matrix (22) to

$$K_t=P_t^{ah}{Q}^{-1},P_t^{ah}={\tilde{\Pi }}_t\left[(a-{\overline{a}}_t){(h_t(a)-{\tilde{\Pi }}_t[h_t])}^{\mathrm{T}}\right]={\tilde{\Pi }}_t\left[a{(h_t(a)-{\tilde{\Pi }}_t[h_t])}^{\mathrm{T}}\right]$$

(26)

Clearly, the gain (26) provides only an approximation to the solution of (13). However, such approximations have become popular in nonlinear state estimation in the form of the ensemble Kalman filter [16,17], and we will test its suitability for parameter estimation in Section 6.

Numerical implementations of the proposed McKean–Vlasov approaches rely on Monte–Carlo approximations. More specifically, given M samples ${\tilde{A}}_0^i$, $i=1,\dots ,M$, from the initial distribution ${\Pi }_0$, we introduce the interacting particle system

$$\mathrm{d}{\tilde{A}}_t^i=K_t^M({\tilde{A}}_t^i)\circ \mathrm{d}{I}_t^i,$$

(27)

where the innovation processes $I_t^i$ are defined by either

$$\mathrm{d}{I}_t^i=\mathrm{d}{Y}_t-\frac{1}{2}\left(h_t({\tilde{A}}_t^i)+{\overline{h}}_t^M\right)\mathrm{d}t,{\overline{h}}_t^M=\frac{1}{M}\sum _{i=1}^M{h}_t({\tilde{A}}_t^i),$$

(28)

or, alternatively,

$$\mathrm{d}{I}_t^i=\mathrm{d}{Y}_t-\left(h_t({\tilde{A}}_t^i)\mathrm{d}t+G\mathrm{d}{W}_t^i\right),$$

(29)

and $W_t^i$, $i=1,\dots ,M$, denote independent $N_w$-dimensional Brownian motions. For $K_t^M$, we will use the parameter-independent empirical Kalman gain approximation

$$K_t^M={\widehat{P}}_t^{ah}{Q}^{-1},{\widehat{P}}_t^{ah}=\frac{1}{M-1}\sum _{i=1}^M{\tilde{A}}_t^i{(h_t({\tilde{A}}_t^i)-{\overline{h}}_t^M)}^{\mathrm{T}},$$

(30)

in our numerical experiments, which leads to the so-called ensemble Kalman–Bucy filter [14,15]. Please note that ${\widehat{P}}_t^{ah}$ provides an unbiased estimator of $P_t^{ah}$.

Finally, a robust and efficient time-stepping procedure for approximating ${\tilde{A}}_{t_n}$, $t_n=n\Delta t$, is provided in [28,29,30]. Denoting the approximations at time $t_n$ by ${\tilde{A}}_n^i$, $i=1,\dots ,M$, we obtain

$${\tilde{A}}_{n+1}^i={\tilde{A}}_n^i+\Delta t{\widehat{P}}_n^{ah}{\left(Q+\Delta t{\widehat{P}}_n^{hh}\right)}^{-1}\Delta I_n^i$$

(31)

with step size $\Delta t>0$, empirical covariance matrices

$${\widehat{P}}_n^{ah}=\frac{1}{M-1}\sum _{i=1}^M{\tilde{A}}_n^i{(h_n({\tilde{A}}_n^i)-{\overline{h}}_n^M)}^{\mathrm{T}},{\widehat{P}}_n^{hh}=\frac{1}{M-1}\sum _{i=1}^M{h}_n({\tilde{A}}_n^i){(h_n({\tilde{A}}_n^i)-{\overline{h}}_n^M)}^{\mathrm{T}},$$

(32)

and innovation increments $\Delta I_n^i$ given by either

$$\Delta I_n^i=\Delta Y_n-\frac{1}{2}\left(h_n({\tilde{A}}_n^i)+{\overline{h}}_n^M\right)\Delta t,{\overline{h}}_n^M=\frac{1}{M}\sum _{i=1}^M{h}_n({\tilde{A}}_n^i),$$

(33)

or

$$\Delta I_n^i=\Delta Y_n-\left(h_n({\tilde{A}}_n^i)\Delta t+\Delta t^{1/2}G{\Xi }_n^i\right),{\Xi }_n^i\sim \mathrm{N}(0,I).$$

(34)

Here we have used the abbreviations $h_n(a)=f(Y_n,a)$, $Y_n=Y_{t_n}$, and $\Delta Y_n=Y_{t_{n+1}}-Y_{t_n}$.

While the feedback particle formulation (17) and its ensemble Kalman–Bucy filter approximation (31) are special cases of already available formulations, they provide the starting point for our novel McKean–Vlasov equations and their numerical approximation of the combined state and parameter estimation problem with correlated measurement and model errors, which we develop in the following two sections.

4. State Estimation for Noisy Data

We return to the observation Model (3) with $R\ne 0$ and general H. The pure state estimation problem is considered first; that is, $f(x,a)=f(x)$ in (1).

Using $E_t^{\mathrm{o}}$, given by (5), and $E_t^{\mathrm{c}}$ defined by

$$E_t^{\mathrm{c}}=G(I-G^{\mathrm{T}}{H}^{\mathrm{T}}{C}^{-1}HG)W_t-Q{H}^{\mathrm{T}}{C}^{-1}{R}^{1/2}{V}_t$$

(35)

with the total measurement error covariance matrix C given by (6), we find that

$$G{W}_t=E_t^{\mathrm{c}}+Q{H}^{\mathrm{T}}{C}^{-1}{E}_t^{\mathrm{o}},$$

(36)

and the covariations [2] satisfy

$${\langle E^{\mathrm{o}},E^{\mathrm{c}}\rangle }_t=0,{\langle E^{\mathrm{o}},E^{\mathrm{o}}\rangle }_t=Ct,{\langle E^{\mathrm{c}},E^{\mathrm{c}}\rangle }_t=G(I-G^{\mathrm{T}}{H}^{\mathrm{T}}{C}^{-1}HG)G^{\mathrm{T}}t.$$

(37)

These errors naturally suggest linear combinations of $W_t$ and $V_t$ in (1) and (3) that shift the correlation between measurement and model errors to the signal dynamics, yielding

$$\begin{array}{cc}\hfill \mathrm{d}{X}_t& =f(X_t)\mathrm{d}t+G{(I-G^{\mathrm{T}}{H}^{\mathrm{T}}{C}^{-1}HG)}^{1/2}\mathrm{d}{\widehat{W}}_t+Q{H}^{\mathrm{T}}{C}^{-1/2}\mathrm{d}{\widehat{V}}_t,\hfill \end{array}$$

(38a)

$$\begin{array}{cc}\hfill \mathrm{d}{Y}_t& =Hf(X_t)\mathrm{d}t+C^{1/2}\mathrm{d}{\widehat{V}}_t,\hfill \end{array}$$

(38b)

where ${\widehat{W}}_t$ and ${\widehat{V}}_t$ denote mutually independent standard Brownian motions of dimension $N_w$ and $N_y$, respectively. These equations correspond exactly to the correlated noise example from [7] (Section 3.8). Furthermore, $H=I$ and $R=0$ lead to $E_t^{\mathrm{c}}=0$, $Q{H}^{\mathrm{T}}{C}^{-1/2}=C^{1/2}$, and, hence, $\mathrm{d}{X}_t=\mathrm{d}{Y}_t$.

A straightforward application of the results from [7] (Section 3.8) yields the following statement:

Lemma 2 (Generalised Kushner–Stratonovich equation).

The conditional expectations ${\pi }_t[\varphi ]=\mathbb{E}[\varphi (X_t)|Y_{[0,t]}]$ satisfy

$$\begin{array}{cc}\hfill {\pi }_t[\varphi ]& ={\pi }_0[\varphi ]+{\int }_0^t{\pi }_s[\mathcal{L}\varphi ]\mathrm{d}s+{\int }_0^t{\pi }_s{\left[\varphi h+HQ\nabla \varphi -\varphi {\pi }_s[h]\right]}^{\mathrm{T}}{C}^{-1}\left(\mathrm{d}{Y}_s-{\pi }_s[h]\mathrm{d}s\right),\hfill \end{array}$$

(39)

where We use the notation $Q:\nabla \nabla \varphi ={\sum }_{i,j=1}^{N_x}{Q}^{ij}{\partial }_i{\partial }_j\varphi$.

$$\mathcal{L}=f·\nabla +\frac{1}{2}Q:\nabla \nabla$$

(40)

is the generator of (1), $h(x)=Hf(x)$ denotes the observation map, and ϕ is a compactly supported smooth function.

For the convenience of the reader, we present an independent derivation in Appendix A. We note that (39) also arises as the Kushner–Stratonovitch equations for an SDE Model (1) with observations $Y_t$ satisfying the observation model

$$\mathrm{d}{Y}_t=H\left(f(X_t)-Q\nabla log{\pi }_t(X_t)\right)\mathrm{d}t+C^{1/2}\mathrm{d}{\tilde{V}}_t,$$

(41)

where ${\tilde{V}}_t$ denotes $N_y$-dimensional Brownian motion independent of the Brownian motion $W_t$ in (1). Here we have used that ${\pi }_t\left[HQ\nabla {\pi }_t\right]=0$. This reinterpretation of our state estimation problem in terms of uncorrelated model and observation errors and modified observation map

$${\tilde{h}}_t(x)=H\left(f(x)-Q\nabla log{\pi }_t(x)\right)$$

(42)

allows one to apply available MCMC and SMC methods for continuous-time filtering and smoothing problems. See, for example, [16]. However, there are two major limitations of such an approach. First, it requires approximating the gradient of the log-density. Second, the modified observation Model (41) is not well-defined in the limit $R\to 0$ and $H=I$, since the density ${\pi }_t$ collapses to a Dirac delta function under the given initial condition $X_0=x_0$ a.s.

In order to circumvent these complications, we develop an alternative approach based on an appropriately modified feedback particle filter formulation in the following subsection.

4.1. Generalised Feedback Particle Filter Formulation

While it is clearly possible to apply the standard feedback particle filter formulations using (41), the following alternative formulation avoids the need for approximating the gradient of the log-density.

Lemma 3 (Feedback particle filter with correlated innovation).

Consider the McKean–Vlasov equation

$$\mathrm{d}{\tilde{X}}_t=f({\tilde{X}}_t)\mathrm{d}t+G\mathrm{d}{W}_t+K_t({\tilde{X}}_t)\circ \mathrm{d}{I}_t+{\Omega }_t({\tilde{X}}_t)\mathrm{d}t,$$

(43)

where the gain $K_t\in {\mathbb{R}}^{N_x\times N_y}$ solves

$$\nabla ·\left({\tilde{\pi }}_t\left(K_{t}C-Q{H}^{\mathrm{T}}\right)\right)=-{\tilde{\pi }}_t{\left(h-{\tilde{\pi }}_t[h]\right)}^{\mathrm{T}},{\tilde{\pi }}_t=Law({\tilde{X}}_t),$$

(44)

with observation map $h(x)=Hf(x)$. The function ${\Omega }_t$ is given by

$${\Omega }_t^i=-\frac{1}{2}\sum _{l=1}^{N_x}\sum _{j=1}^{N_y}{\partial }_l{K}_t^{ij}{(Q{H}^{\mathrm{T}})}^{lj},i=1,\dots ,N_x,$$

(45)

and the innovation process $I_t$ by

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\left(h({\tilde{X}}_t)\mathrm{d}t+HG\mathrm{d}{W}_t+R^{1/2}\mathrm{d}{U}_t\right).$$

(46)

Here, $W_t$ and $U_t$ denote mutually independent $N_x$-dimensional and $N_y$-dimensional Brownian motions, respectively. Then, ${\tilde{\pi }}_t=Law({\tilde{X}}_t)$ coincides with the solution to (39), provided that the initial distributions agree.

It should be stressed that $W_t$ in (43) and (46) denote the same Brownian motion, resulting in correlations between the innovation process and model noise.

Proof. 

In this proof the Einstein summation convention over repeated indices is employed, noting that (44) takes the form

$${\partial }_i\left({\tilde{\pi }}_t\left(K_t^{ij}{C}^{jk}-{(Q{H}^{\mathrm{T}})}^{ik}\right)\right)=-{\tilde{\pi }}_t\left(h^k-{\tilde{\pi }}_t[h^k]\right),k=1,\dots ,N_y.$$

(47)

We begin by writing (43) in its Itô-form,

$$\mathrm{d}{\tilde{X}}_t=f({\tilde{X}}_t)\mathrm{d}t+G\mathrm{d}{W}_t+K_t({\tilde{X}}_t)\mathrm{d}{I}_t+{\widehat{\Omega }}_t({\tilde{X}}_t)\mathrm{d}t,$$

(48)

where

$$\begin{array}{cc}\hfill {\widehat{\Omega }}_t^i& ={\Omega }_t^i+\frac{1}{2}\left\{-\left({\partial }_l{K}_t^{ij}\right){(Q{H}^{\mathrm{T}})}^{lj}+2\left({\partial }_l{K}_t^{ij}\right)K_t^{lk}{C}^{kj}\right\}\hfill \\ \hfill & =\left({\partial }_l{K}_t^{ij}\right)\left\{K_t^{lk}{C}^{kj}-{(Q{H}^{\mathrm{T}})}^{lj}\right\}\hfill \end{array}$$

(49)

Here, we have used that the covariation between $K_t$ and $I_t$ satisfies

$$\mathrm{d}{〈K^{ij},I^j〉}_t={\partial }_l{K}_t^{ij}\left(G^{lk}\mathrm{d}{〈W^k,I〉}_t+K_t^{lk}\mathrm{d}{〈I^k,I^j〉}_t\right).$$

(50)

Furthermore, ${\langle GW,I\rangle }_t=-Q{H}^{\mathrm{T}}t$ and ${\langle I,I\rangle }_t=2Ct$.

For a smooth compactly supported test function $\varphi$, Itô’s formula implies

$$\varphi ({\tilde{X}}_t)=\varphi ({\tilde{X}}_0)+{\int }_0^t{\partial }_i\varphi ({\tilde{X}}_s)\mathrm{d}{\tilde{X}}_s^i+\frac{1}{2}{\int }_0^t{\partial }_i{\partial }_j\varphi ({\tilde{X}}_s)\mathrm{d}{\langle {\tilde{X}}^i,{\tilde{X}}^j\rangle }_s,$$

(51)

where the covariation process is given by

$${\langle \tilde{X},\tilde{X}\rangle }_t=tQ-{\int }_0^t\left(K_{s}HQ+Q{H}^{\mathrm{T}}{K}_s^{\mathrm{T}}\right)\mathrm{d}s+2{\int }_0^t{K}_{s}C{K}_s^{\mathrm{T}}\mathrm{d}s.$$

(52)

Our aim is to show that ${\tilde{\pi }}_t[\varphi ]$ coincides with ${\pi }_t[\varphi ]$ as defined by the Kushner–Stratonovich Equation (39). To this end, we insert (48) and (52) into (51) and take the conditional expectation, arriving at

$$\begin{array}{cc}{\tilde{\pi }}_t[\varphi ]\hfill & ={\tilde{\pi }}_0[\varphi ]+{\int }_0^t{\tilde{\pi }}_s[\mathcal{L}\varphi ]\mathrm{d}s+{\int }_0^t{\tilde{\pi }}_s\left[({\partial }_i\varphi )K_s^{ij}\right]\mathrm{d}{Y}_s^j-{\int }_0^t{\tilde{\pi }}_s\left[({\partial }_i\varphi )K_s^{ij}{h}^j\right]\mathrm{d}s\hfill \\ \hfill & +{\int }_0^t{\tilde{\pi }}_s\left[({\partial }_i\varphi ){\widehat{\Omega }}_s^i\right]\mathrm{d}s+{\int }_0^t{\tilde{\pi }}_s\left[\left({\partial }_i{\partial }_j\varphi \right){\left(K_s(C{K}_s^{\mathrm{T}}-HQ)\right)}^{ij}\right]\mathrm{d}s,\hfill \end{array}$$

(53)

recalling that the generator $\mathcal{L}$ has been defined in (40). Under the assumption that $K_t$ satisfies (44), the two Equations (39) and (53) coincide. Indeed,

$${\tilde{\pi }}_s\left[({\partial }_i\varphi )(K_s^{ik}{C}^{kj}-{(Q{H}^{\mathrm{T}})}^{ij})\right]={\tilde{\pi }}_s\left[\varphi \left(h^j-{\tilde{\pi }}_s\left[h^j\right]\right)\right]$$

(54)

implies

$${\tilde{\pi }}_s[\nabla \varphi ·K_s]={\tilde{\pi }}_s{\left[\varphi h+HQ\nabla \varphi -\varphi {\tilde{\pi }}_s[h]\right]}^{\mathrm{T}}{C}^{-1},$$

(55)

and the $\mathrm{d}{Y}_s$-contributions agree. To verify the same for the $\mathrm{d}s$-contributions, we use (44) to obtain

$$\begin{array}{cc}{\tilde{\pi }}_s\left[({\partial }_i\varphi )K_s^{ij}(h^j-{\tilde{\pi }}_t[h^j])\right]\hfill & =-{\int }_{{\mathbb{R}}^{N_x}}({\partial }_i\varphi )K_s^{ij}{\partial }_l\left({\tilde{\pi }}_s\left(K_s^{ln}{C}^{nj}-{(Q{H}^{\mathrm{T}})}^{lj}\right)\right)\mathrm{d}x\hfill \\ \hfill & ={\tilde{\pi }}_s\left[({\partial }_i\varphi ){\widehat{\Omega }}_s^i\right]+{\tilde{\pi }}_s\left[\left({\partial }_i{\partial }_j\varphi \right){\left(K_s(C{K}_s^{\mathrm{T}}-K_{s}HQ)\right)}^{ij}\right].\hfill \end{array}$$

(56)

Finally, collecting terms in (53) and (56), and applying (55) to the remaining $\mathrm{d}s$-contribution, i.e., $-{\tilde{\pi }}_s[\nabla \varphi ·K_s]{\tilde{\pi }}_s[h]$, leads to the desired result. □

We note that the correlation between the innovation process $I_t$ and the model error $W_t$ leads to a correction term ${\Omega }_t$ in (43) which cannot be subsumed into a Stratonovitch correction, in contrast to the standard feedback particle filter formulation (17).

Remark 2.

Assuming that there exist potential functions ${\Psi }_t=({\psi }_t^1,\dots ,{\psi }_t^{N_y})$, ${\psi }_t^k:{\mathbb{R}}^{N_x}\to \mathbb{R}$, solving the Poisson equation(s) (19) (with ${\tilde{\Pi }}_t$ being replaced by ${\tilde{\pi }}_t$), (44) can be solved by requiring

$$K_t=(\nabla {\Psi }_t+Q{H}^{\mathrm{T}})C^{-1},$$

(57)

thus generalising (18).

Remark 3.

If we set $R=0$, $H=I$, and $K_t=Q{H}^{\mathrm{T}}{C}^{-1}=I$ in (43), then one obtains

$$\mathrm{d}{\tilde{X}}_t=\mathrm{d}{Y}_t$$

(58)

since ${\Omega }_t$ vanishes, and all other terms in (43) cancel each other out. If, furthermore, $Y_0={\tilde{X}}_0=x_0$ a.s., then ${\tilde{X}}_t=Y_t$ for all $t\in [0,T]$, which in turn justifies our assumption that the gain $K_t$ is independent of the state variable. Hence, the McKean–Vlasov formulation (43) reproduces the exact reference trajectory $Y_t$ in the case of no measurement errors and perfectly known initial conditions.

We develop a simplified version of the feedback particle filter formulation (43) for linear SDEs and Gaussian distributions in the following subsection, which will form the basis of the generalised ensemble Kalman–Bucy filter put forward in the follow-up Section 4.3.

4.2. Generalised Kalman–Bucy Filter

Let us assume that $f(x)=Fx$ with $F\in {\mathbb{R}}^{N_x\times N_x}$, i.e., Equations (1) and (3) take the form

$$\begin{array}{cc}\hfill \mathrm{d}{X}_t& =F{X}_t\mathrm{d}t+G\mathrm{d}{W}_t,\hfill \end{array}$$

(59a)

$$\begin{array}{cc}\hfill \mathrm{d}{Y}_t& =HF{X}_t\mathrm{d}t+HG\mathrm{d}{W}_t+R^{1/2}\mathrm{d}{V}_t,\hfill \end{array}$$

(59b)

with initial conditions drawn from a Gaussian distribution. In this case ${\pi }_t$ stays Gaussian for all $t>0$, i.e., ${\pi }_t\sim \mathrm{N}({\overline{x}}_t,P_t)$ with ${\overline{x}}_t\in {\mathbb{R}}^{N_x}$, $P_t\in {\mathbb{R}}^{N_x\times N_x}$. Equation (19) can be solved uniquely by ${\nabla }_x\Psi =P_t{F}^{\mathrm{T}}{H}^{\mathrm{T}}$, and thus the McKean–Vlasov equations for the feedback particle filter (43) reduce to

$$\mathrm{d}{\tilde{X}}_t=F{\tilde{X}}_t\mathrm{d}t+G\mathrm{d}{W}_t+\left(P_t{F}^{\mathrm{T}}{H}^{\mathrm{T}}+Q{H}^{\mathrm{T}}\right)C^{-1}\mathrm{d}{I}_t,$$

(60)

with the innovation process (46) leading to

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-HF{\tilde{X}}_t\mathrm{d}t-HG\mathrm{d}{W}_t-R^{1/2}\mathrm{d}{U}_t.$$

(61)

We take the expectation in (60) and (61) and end up with

$$\mathrm{d}{\overline{x}}_t=F{\overline{x}}_t\mathrm{d}t+\left(P_t{F}^{\mathrm{T}}+Q\right)H^{\mathrm{T}}{C}^{-1}\left(\mathrm{d}{Y}_t-HF{\overline{x}}_t\mathrm{d}t\right).$$

(62)

Defining $u_t:={\tilde{X}}_t-{\overline{x}}_t$, we see that

$$\mathrm{d}{u}_t=F{u}_t\mathrm{d}t+G\mathrm{d}{\tilde{W}}_t-\left(P_t{F}^{\mathrm{T}}+Q\right)H^{\mathrm{T}}{C}^{-1}\left(HF{u}_t\mathrm{d}t+HG\mathrm{d}{W}_t+R^{1/2}\mathrm{d}t\right).$$

(63)

Next we use

$$\mathrm{d}\left(u_t{u}_t^{\mathrm{T}}\right)=\mathrm{d}{u}_t{u}_t^{\mathrm{T}}+u_t\mathrm{d}{u}_t^{\mathrm{T}}+\mathrm{d}{\langle u,u^{\mathrm{T}}\rangle }_t$$

(64)

and $P_t=\mathbb{E}[u_t{u}_t^{\mathrm{T}}]$ to obtain, after some calculations,

$$\mathrm{d}{P}_t=(F{P}_t+P_t{F}^{\mathrm{T}})\mathrm{d}t-\left(P_t{F}^{\mathrm{T}}+Q\right)H^{\mathrm{T}}{C}^{-1}H\left(F{P}_t+Q\right)\mathrm{d}t+Q\mathrm{d}t.$$

(65)

Hence we have shown that our McKean–Vlasov formulation (60) agrees with the standard Kalman–Bucy filter equations for the mean and the covariance matrix in the correlated noise case [6].

4.3. Ensemble Kalman–Bucy Filter

The McKean–Vlasov Equation (60) for linear systems, along with Gaussian prior and posterior distributions, suggest approximating the feedback particle filter formulation (43) for nonlinear systems by

$$\mathrm{d}{\tilde{X}}_t=f({\tilde{X}}_t)\mathrm{d}t+G\mathrm{d}{W}_t+\left(P_t^{xh}+Q{H}^{\mathrm{T}}\right)C^{-1}\mathrm{d}{I}_t,$$

(66)

where the innovation process $I_t$ given by (46) as before. In other words, we approximate the gain matrix $K_t$ in (43) by the state independent term $\left(P_t^{xh}+Q{H}^{\mathrm{T}}\right)C^{-1}$ with the covariance matrix $P_t^{xh}$ defined by

$$P_t^{xh}={\tilde{\pi }}_t\left[(x-{\overline{x}}_t){(h(x)-{\tilde{\pi }}_t[h])}^{\mathrm{T}}\right]={\tilde{\pi }}_t\left[x{(h(x)-{\tilde{\pi }}_t[h])}^{\mathrm{T}}\right]$$

(67)

where ${\tilde{\pi }}_t$ denotes the law of ${\tilde{X}}_t$.

We can now generalise the ensemble Kalman–Bucy filter formulation (31) for the pure parameter estimation problem to the state estimation problem with correlated noise. We assume that M initial state values ${\tilde{X}}_0^i$ have been sampled from an initial distribution ${\pi }_0$ or, alternatively, $X_0^i=x_0$ for all $i=1,\dots ,M$ in case the initial condition is known exactly. These state values are then propagated under the time-stepping procedure

$${\tilde{X}}_{n+1}^i={\tilde{X}}_n^i+\Delta tf({\tilde{X}}_n^i)+\Delta t^{1/2}G{\Theta }_n^i+\left({\widehat{P}}_n^{xh}+Q{H}^{\mathrm{T}}\right){\left(C+\Delta t{\widehat{P}}_n^{hh}\right)}^{-1}\Delta I_n^i$$

(68)

with ${\Theta }_n^i\sim \mathrm{N}(0,I)$, step size $\Delta t>0$, empirical covariance matrices

$$\begin{array}{cc}\hfill {\widehat{P}}_n^{xh}& =\frac{1}{M-1}\sum _{i=1}^M{\tilde{X}}_n^i{(h({\tilde{X}}_n^i)-{\overline{h}}_n^M)}^{\mathrm{T}},{\overline{h}}_n^M=\frac{1}{M}\sum _{i=1}^{M}h({\tilde{X}}_n^i),\hfill \end{array}$$

(69a)

$$\begin{array}{cc}\hfill {\widehat{P}}_n^{hh}& =\frac{1}{M-1}\sum _{i=1}^{M}h({\tilde{X}}_n^i){(h({\tilde{X}}_n^i)-{\overline{h}}_n^M)}^{\mathrm{T}},\hfill \end{array}$$

(69b)

and innovation increments $\Delta I_n^i$ given by

$$\Delta I_n^i=\Delta Y_n-\Delta th({\tilde{X}}_n^i)-\Delta t^{1/2}HG{\Theta }_n^i-\Delta t^{1/2}{R}^{1/2}{\Xi }_n^i,{\Xi }_n^i\sim \mathrm{N}(0,I).$$

(70)

The McKean–Vlasov equations of this section form the basis for the methods proposed for the combined state and parameter estimation problem to be considered next.

5. Combined State and Parameter Estimation

We now return to the combined state and parameter estimation problem, and consider the augmented dynamics

$$\begin{array}{cc}\hfill \mathrm{d}{X}_t& =f(X_t,A_t)\mathrm{d}t+G\mathrm{d}{W}_t,\hfill \end{array}$$

(71a)

$$\begin{array}{cc}\hfill \mathrm{d}{A}_t& =0,\hfill \end{array}$$

(71b)

with observations (3) as before. The initial conditions satisfy $X_0=x_0$ a.s., and $A_0\sim {\Pi }_0$. Let us introduce the extended state space variable $Z_t={(X_t^{\mathrm{T}},A_t^{\mathrm{T}})}^{\mathrm{T}}$. In terms of $Z_t$, the Equations (3) and (71) take the form

$$\begin{array}{c}\hfill \mathrm{d}{Z}_t=\overline{f}(Z)\mathrm{d}t+\overline{G}\mathrm{d}{W}_t,\end{array}$$

(72a)

$$\begin{array}{c}\hfill \mathrm{d}{Y}_t=\overline{H}\mathrm{d}{Z}_t+R^{1/2}\mathrm{d}{V}_t,\end{array}$$

(72b)

with

$$\overline{f}(z)=\left(\begin{array}{c}f(x,a)\\ 0\end{array}\right),\overline{G}=\left(\begin{array}{cc}G& 0\\ 0& 0\end{array}\right),\overline{H}=\left(\begin{array}{cc}H& 0\end{array}\right).$$

(73)

Thus we end up with an augmented state estimation problem of the general structure considered in detail in Section 4 already. Below we provide details on some of the necessary modifications.

5.1. Feedback Particle Filter Formulation

The appropriately extended feedback particle filter Equation (43) leads to

$$\begin{array}{cc}\hfill \mathrm{d}{\tilde{X}}_t& =f(\tilde{X},{\tilde{A}}_t)\mathrm{d}t+G\mathrm{d}{W}_t+({\nabla }_x{\Psi }_t({\tilde{X}}_t,{\tilde{A}}_t)+Q{H}^{\mathrm{T}})C^{-1}\circ \mathrm{d}{I}_t+{\Omega }_t({\tilde{X}}_t,{\tilde{A}}_t),\hfill \end{array}$$

(74a)

$$\begin{array}{cc}\hfill \mathrm{d}{\tilde{A}}_t& ={\nabla }_a{\Psi }_t({\tilde{X}}_t,{\tilde{A}}_t)C^{-1}\circ \mathrm{d}{I}_t,\hfill \end{array}$$

(74b)

where (46) takes the form

$$\mathrm{d}{I}_t=\mathrm{d}{Y}_t-\left(h({\tilde{X}}_t,{\tilde{A}}_t)\mathrm{d}t+HG\mathrm{d}{W}_t+R^{1/2}\mathrm{d}{U}_t\right)$$

(75)

with observation map (4) and correction ${\Omega }_t$ given by (45), with Q replaced by $\overline{Q}=\overline{G}{\overline{G}}^T$ and H by $\overline{H}$. In the Poisson equation(s) (19), ${\tilde{\Pi }}_t$ is replaced by ${\tilde{\pi }}_t$ denoting the joint density of $({\tilde{X}}_t,{\tilde{A}}_t)$. We also stress that ${\Psi }_t$ becomes a function of x and a, and we distinguish between gradients with respect to x and a using the notation ${\nabla }_x$ and ${\nabla }_a$, respectively.

Numerical implementations of the extended feedback particle filter are demanding due to the need for solving the Poisson equation(s) (19). Instead, we again rely on the ensemble Kalman–Bucy filter approximation, which we describe next.

5.2. Ensemble Kalman–Bucy Filter

We approximate the joint density ${\tilde{\pi }}_t$ of ${\tilde{Z}}_t$ by an ensemble of particles

$${\tilde{Z}}_t^i=\left(\begin{array}{c}{\tilde{X}}_t^i\\ {\tilde{A}}_t^i\end{array}\right),$$

(76)

that is,

$${\tilde{\pi }}_t\approx \frac{1}{M}\sum _{i=1}^M{\delta }_{{\tilde{Z}}_t^i},$$

(77)

where ${\delta }_{z^{\prime }}$ denotes the Dirac delta function centred at $z^{\prime }$. The initial ensemble satisfies $X_0^i=x_0$ for all $i=1,\dots ,M$, and the initial parameter values $A_0^i$ are independent draws from the prior distribution ${\Pi }_0$.

At the same time, we make the approximation ${\tilde{Z}}_t\sim \mathrm{N}({\overline{z}}_t^M,{\widehat{P}}_t^{zz})$ when dealing with the Kalman gain of the feedback particle filter. Here the empirical mean ${\overline{z}}_t^M$ has components

$${\overline{x}}_t^M=\frac{1}{M}\sum _{i=1}^M{\tilde{X}}_t^i,{\overline{a}}_t^M=\frac{1}{M}\sum _{i=1}^M{\tilde{A}}_t^i,$$

(78)

and the joint empirical covariance matrix is given by

$${\widehat{P}}_t^{zz}=\frac{1}{M-1}\sum _{i=1}^M{\tilde{Z}}_t^i{({\tilde{Z}}_t-{\overline{z}}_t^M)}^{\mathrm{T}}=\left(\begin{array}{cc}{\widehat{P}}_t^{xx}& {\widehat{P}}_t^{xa}\\ {({\widehat{P}}_t^{xa})}^{\mathrm{T}}& {\widehat{P}}_t^{aa}\end{array}\right).$$

(79)

As in Section 4.3, the solution to (19) can be approximated by

$${\nabla }_x{\Psi }_t=P_t^{xh},{\nabla }_a{\Psi }_t=P_t^{ah},$$

(80)

where finally, the covariance matrices $P_t^{xh}$ and $P_t^{ah}$ are estimated by their empirical counterparts

$$\begin{array}{cc}\hfill {\widehat{P}}_t^{xh}& =\frac{1}{M-1}\sum _{i=1}^M{\tilde{X}}_t^i{(h({\tilde{X}}_t^i,{\tilde{A}}_t^i)-{\overline{h}}_t^M)}^{\mathrm{T}},\hfill \end{array}$$

(81a)

$$\begin{array}{cc}\hfill {\widehat{P}}_t^{ah}& =\frac{1}{M-1}\sum _{i=1}^M{\tilde{A}}_t^i{(h({\tilde{X}}_t^i,{\tilde{A}}_t^i)-{\overline{h}}_t^M)}^{\mathrm{T}},\hfill \end{array}$$

(81b)

with ${\overline{h}}_t^M$ defined by

$${\overline{h}}_t^M=\frac{1}{M}\sum _{i=1}^{M}h({\tilde{X}}_t^i,{\tilde{A}}_t^i).$$

(82)

Summing everything up, we obtain the following generalised ensemble Kalman–Bucy filter equations

$$\begin{array}{cc}\hfill \mathrm{d}{\tilde{X}}_t^i& =f({\tilde{X}}_t^i,{\tilde{A}}_t^i)\mathrm{d}t+G\mathrm{d}{W}_t^i+({\widehat{P}}_t^{xh}+Q{H}^{\mathrm{T}})C^{-1}\mathrm{d}{I}_t^i,\hfill \end{array}$$

(83a)

$$\begin{array}{cc}\hfill \mathrm{d}{\tilde{A}}_t^i& ={\widehat{P}}_t^{ah}{C}^{-1}\mathrm{d}{I}_t^i,\hfill \end{array}$$

(83b)

where the innovations are given by

$$\mathrm{d}{I}_t^i=\mathrm{d}{Y}_t-\left(h({\tilde{X}}_t^i,{\tilde{A}}_t^i)\mathrm{d}t+HG\mathrm{d}{W}_t^i+R^{1/2}\mathrm{d}{U}_t^i\right),$$

(84)

and $W_t^i$ and $U_t^i$ denote independent $N_x$-dimensional and $N_y$-dimensional Brownian motions, respectively, for $i=1,\dots ,M$.

The interacting particle Equation (83) can be time-stepped along the lines discussed in Section 4.3 for the pure state estimation formulation of the ensemble Kalman–Bucy filter.

6. Numerical Results

We now apply the generalised ensemble Kalman–Bucy filter formulation (83) with innovation (84) to five different model scenarios.

6.1. Parameter Estimation for the Ornstein–Uhlenbeck Process

Our first example is provided by the Ornstein–Uhlenbeck process

$$\mathrm{d}{X}_t=a{X}_t\mathrm{d}t+Q^{1/2}\mathrm{d}{W}_t$$

(85)

with unknown parameter $a\in \mathbb{R}$, and known initial condition $X_0=1/2$. We assume an observation model of the form (3) with $H=1$, and a measurement error taking values $R=0.01$, $R=0.0001$, and $R=0$. The model error variance is set to either $Q=0.5$ or $Q=0.005$. Except for the case $R=0$ a combined state and parameter estimation problem is to be solved. We implement the ensemble Kalman–Bucy filter (Section 5.2) with innovation (84), step size $\Delta t=0.005$, and ensemble size $M=1000$. The data is generated using the Euler–Maruyama method applied to (85), with $a=-1/2$ and integrated over a time-interval $[0,500]$ with the same step size. The prior distribution ${\Pi }_0$ for the parameter is Gaussian with mean $\overline{a}=-1/2$ and variance ${\sigma }_a^2=2$. The results can be found in Figure 1. We find that the ensemble Kalman–Bucy filter is able to successfully identify the unknown parameter under all tested experimental settings, except for the largest measurement error case where $R=0.01$. There, a small systematic offset of the estimated parameter value can be observed. One can also see that the variance in the parameter estimate monotonically decreases in time in all cases, while the variance in the state estimates approximately reaches a steady state.

6.2. Averaging

Consider the equations

$$\begin{array}{cc}\hfill \mathrm{d}{Y}_t& =\left(1-Z_t^2\right)Y_t\mathrm{d}t+Q^{1/2}\mathrm{d}{W}_t^y,\hfill \end{array}$$

(86a)

$$\begin{array}{cc}\hfill \mathrm{d}{Z}_t& =-\frac{\alpha }{ϵ}{Z}_t\mathrm{d}t+\sqrt{\frac{2\lambda }{ϵ}}\mathrm{d}{W}_t^z\hfill \end{array}$$

(86b)

from [19] for $\lambda ,\alpha ,\gamma ,ϵ>0$, and initial condition $Y_0=1/2$, $Z_0=0$. The reduced equations in the limit $ϵ\to 0$ are given by (85), with parameter value

$$a=1-\frac{\lambda }{\alpha }$$

(87)

and initial condition $X_0=1/2$. The reduced dynamics corresponds to a (stable) Ornstein–Uhlenbeck process for $\lambda /\alpha >1$. We wish to estimate the parameter a from observed increments

$$\Delta Y_n=Y_{n+1}-Y_n+\Delta t^{1/2}{R}^{1/2}{\Xi }_n,{\Xi }_n\sim \mathrm{N}(0,1),$$

(88)

where the sequence of ${\{Y_n\}}_{n\ge 0}$ is obtained by time-stepping (86) using the Euler–Maruyama method with a step size $\Delta t$. We set $\lambda =3$, $\alpha =2$ (so that $a=-1/2$), $Q=0.5$, and $ϵ\in \{0.1,0.01\}$ in our experiments. The measurement noise is set to $R=0.01$ or $R=0$ (pure parameter estimation).

We implement the ensemble Kalman–Bucy filter (83) with innovation (84), step size $\Delta t=ϵ/50$, and ensemble size $M=1000$ for the reduced Equation (87). The data is generated from an Euler–Maruyama discretization of (86) with the same step size. We also investigate the effect of subsampling the observations for $ϵ=0.01$ by solving (86) with step size $\Delta t=ϵ/50$ and storing only every tenth solution $Y_n$, while the reduced equations and the ensemble Kalman–Bucy filter equations are integrated with $\Delta t=ϵ/5$. The results are shown in Figure 2. Figure 3 shows the results for the same experiments repeated with a smaller ensemble size of $M=10$. We find that the smaller ensemble size leads to more noisy estimates for the variance in ${\tilde{X}}_n$ and a faster decay of the variance in ${\tilde{A}}_n$, but the estimated parameter values are equally well converged. Subsampling does not lead to significant changes in the estimated parameter values. This is in contrast to the example considered next.

We finally mention [31] for alternative approaches to sequential estimation in the context of averaging using however different assumptions on the data.

6.3. Homogenisation

In this example, the data is produced by integrating the multi-scale SDE

$$\begin{array}{cc}\hfill \mathrm{d}{Y}_t& =\left(\frac{\sqrt{\sigma /2}}{ϵ}{Z}_t+a{Y}_t\right)\mathrm{d}t,\hfill \end{array}$$

(89a)

$$\begin{array}{cc}\hfill \mathrm{d}{Z}_t& =-\frac{1}{{ϵ}^2}{Z}_t\mathrm{d}t+\frac{\sqrt{2}}{ϵ}\mathrm{d}{W}_t^z\hfill \end{array}$$

(89b)

with parameter values $ϵ=0.1$, $a=-1/2$, $\sigma =1/2$, and initial condition $Y_0=1/2$, $Z_0=0$. Here, $W_t^z$ denotes standard Brownian motion. The equations are discretised with step size $\Delta \tau ={ϵ}^2/50=0.0002$, and the resulting increments (88) are stored over a time interval $[0,500]$. See [32] for more details.

According to homogenisation theory, the reduced model is given by (85) with $Q=\sigma$, and we wish to estimate the parameter a from the data $\{\Delta Y_n\}$ produced according to (88). It is known that a standard maximum likelihood estimator (MLE) given by

$$a_{\mathrm{ML}}=\frac{{\sum }_n{Y}_{t_n}(Y_{t_{n+1}}-Y_{t_n})}{{\sum }_n{Y}_{t_n}^2\Delta \tau }$$

(90)

leads to $a_{\mathrm{ML}}=0$ in the limit $\Delta \tau \to 0$ and the observation interval $T\to \infty$. This MLE corresponds to $H=I$ and $R=0$ in our extended state space formulation of the problem. Subsampling can be achieved by choosing an appropriate time-step $\Delta t>\Delta \tau$ in the ensemble Kalman–Bucy filter equations and a corresponding subsampling of the data points $Y_n$ in (88). We used $\Delta t=50\Delta \tau =0.01$ and $\Delta t=500\Delta \tau =0.1$, respectively. The results can be found in Figure 4. It can be seen that only the larger subsampling leads to a correct estimate of the parameter a. This is in line with known results for the maximum likelihood estimator (90). See [32] and references therein.

6.4. Nonparametric Drift and State Estimation

We consider nonparametric drift estimation for one-dimensional SDEs over a periodic domain $[0,2\pi )$ in the setting considered from a theoretical perspective in [33]. There, a zero-mean Gaussian process prior $\mathcal{GP}(0,{\mathcal{D}}^{-1})$ is placed on the unknown drift function, with inverse covariance operator

$$\mathcal{D}:=\eta [{(-\Delta )}^p+\kappa I].$$

(91)

The integer parameter p sets the regularity of the process, whereas $\eta ,\kappa \in {\mathbb{R}}^{+}$ control its characteristic correlation length and stationary variance.

Spatial discretization of the problem is carried out by first defining a grid of $N_d$ evenly spaced points on the domain, at locations $x_i=i\Delta x$, $\Delta x=2\pi /N_d$. The drift function is projected onto compactly supported functions centred at these points, which are piecewise linear with

$$b_i(x_j)={\delta }_{ij}$$

(92)

and linear interpolation is used to define a drift function $f(x,a)$ for all $x\in [0,2\pi )$, that is, it is of the form (2) with $f_0(x)\equiv 0$. In this example, we set $N_d=200$. Sample realisations, as well as the reference drift $f^{*}$, can be found in Figure 5a.

Data is generated by integrating the SDE (1) with drift $f^{*}$ forward in time from initial condition $X_0=\pi$ and with noise level $Q=0.1$, using the Euler–Maruyama discretisation with step size $\Delta t=0.1$ over one million time-steps. The spatial distribution of the solutions $X_n$ is plotted in Figure 5b. The data is then given by

$$\Delta Y_n=X_{n+1}-X_n+\Delta t^{1/2}{R}^{1/2}{\Xi }_n$$

(93)

with $R=0.00001$. Data assimilation is performed using the time-discretised ensemble Kalman–Bucy filter Equation (83) with innovation (84), ensemble size $M=200$, and step size $\Delta t=0.1$.

The final estimate of the drift function (ensemble mean) and the ensemble of drift functions can be found in Figure 5c. Figure 5d displays the ensemble of state estimates and the value of the reference solution at the final time. We find that the ensemble Kalman–Bucy filter is able to successfully estimate the drift function and the model states. Further experiments reveal that the drift function can only be identified for sufficiently small measurement errors.

6.5. Spde Parameter Estimation

Consider the stochastic heat equation on the periodic domain $x\in [0,2\pi )$, given in conservative form by the stochastic partial differential equation (SPDE)

$$\begin{array}{c}\hfill \mathrm{d}u(x,t)=\nabla ·\left(\theta (x)\nabla u(x,t)\right)\mathrm{d}t+{\sigma }^{1/2}\mathrm{d}W(x,t),\end{array}$$

(94)

where $W(x,t)$ is space-time white noise. With constant $\theta (x)=\theta$, this SPDE reduces to

$$\begin{array}{c}\hfill \mathrm{d}u(x,t)=\theta \Delta u(x,t)\mathrm{d}t+{\sigma }^{1/2}\mathrm{d}W(x,t).\end{array}$$

(95)

In this example, we examine the estimation of $\theta$ from incremental measurements of a locally averaged quantity $q(x,t)$ that arises naturally in a standard finite volume discretisation of (95).

To discretise the system, one first defines $q_t^i=q(x_i,t)$ around $N_d=200$ grid points $x_i$ on a regular grid, separated by distances $\Delta x$, as

$$\begin{array}{c}\hfill q_t^i={\int }_{x_i-\Delta x/2}^{x_i+\Delta x/2}u(x,t)\mathrm{d}x.\end{array}$$

(96)

The conservative (drift) term in (94) reduces to

$$\begin{array}{c}\hfill {\int }_{x_i-\Delta x/2}^{x_i+\Delta x/2}\nabla ·\left(\theta (x)\nabla u(x,t)\right)\mathrm{d}x={\theta }_{i+1/2}\nabla u_t^{i+1/2}-{\theta }_{i-1/2}\nabla u_t^{i-1/2},\end{array}$$

(97)

where ${\theta }_{i\pm 1/2}\equiv \theta (x_i+\Delta x/2)$, etc. The following standard finite difference approximations

$$\begin{array}{c}\hfill \nabla u_t^{i+1/2}\simeq \frac{u_t^{i+1}-u_t^i}{\Delta x},u_t^i\simeq \Delta x^{-1}{q}_t^i\end{array}$$

(98)

yield the $N_d$-dimensional SDE

$$\begin{array}{cc}\hfill \mathrm{d}{q}_t^i& =\theta \left(\frac{q_t^{i+1}-2q_t^i+q_t^{i-1}}{\Delta x^2}\right)\mathrm{d}t+{\sigma }^{1/2}\Delta x^{1/2}\mathrm{d}{W}_t^i\hfill \end{array}$$

(99)

for constant $\theta$, where $W_t^i$ are independent one-dimensional Brownian motions in time.

Following recent results from [20] we consider the case of estimation of a constant $a=\theta$ value from measurements $\mathrm{d}{q}_t^{*}$ at a fixed location/index $j^{*}\in \{1,\dots ,N_d\}$. The data trajectory is thus given by

$$\begin{array}{c}\hfill \mathrm{d}{Y}_t=\mathrm{d}{q}_t^{*}+R^{1/2}\mathrm{d}{V}_t\end{array}$$

(100)

where $R^{1/2}$ is a scalar and $V_t$ is a standard Brownian motion in one dimension. We perform numerical experiments in which the initial state $q_0^i$ is set to zero for all indices i and the prior on the unknown parameter $a=\theta$ is uniform over the interval $[0.2,1.8]$.

The increment data is generated by first integrating (95) forward in time from the known initial condition $q_i(0)=0$ for all i. The equation is discretised in time using the Euler-Maruyama method. It is known that $\Delta t<\theta \Delta x^2/2$ is required for stability of the Euler–Maruyama discretisation; we use the much smaller time step $\Delta t=\Delta x^2/80$. The solution is sampled with this same time step, and increment measurements are approximated at time $t_n$ by setting the measurement noise level R to zero in (100), resulting in

$$\begin{array}{c}\hfill \Delta Y_n=q_{n+1}^{*}-q_n^{*}.\end{array}$$

(101)

Please note that the associated model error in (1) is given by $G={\sigma }^{1/2}\Delta x^{1/2}I$ and the matrix H in (3) projects the vector of state increments onto a single component with index $j^{*}=N_d/2$. Simulations are performed over the time-interval $[0,20]$. The results can be found in Figure 6a. We also compute the model evidence for a sequence of parameter values $\theta \in \{0.2,0.3,\dots ,1.8\}$ based on a standard Kalman–Bucy filter [6] for the associated linear state estimation problem. See Figure 6b. Both approaches agree with the reference value $\theta =1$.

6.6. Discussion

The results presented here demonstrate that the proposed methodology can be applied to a broad range of continuous-time state and parameter estimation problems with correlated measurement and model errors. Alternatively, one could have employed standard SMC or MCMC methods utilising the modified observation Model (41) as implied by the Kushner–Stratonovitch formulation (39) of the filtering problem. However, such implementations require the approximation of the additional $Q\nabla log{\pi }_t$ term which is nontrivial if only samples from ${\pi }_t$ are available. Furthermore, the limiting behaviour of such implementations in the limit $R\to 0$ and $H=I$ (pure parameter estimation problem) is unclear since ${\pi }_t$ degenerates into a Dirac delta distribution, potentially leading to numerical difficulties in this singular regime. The proposed generalised feedback particle filter formulation avoids these issues through the use of stochastic innovations which are correlated with the model noise. In other words, the distribution ${\pi }_t$ does not appear explicitly in the innovation process (46), and the correlated noise terms cancel each other out as discussed in Remark 3 for $R=0$ and $H=I$. The main computational challenge of the feedback particle filter approach is given by the need for finding the Kalman gain matrix (57). However, the constant gain ensemble Kalman–Bucy approximation

$$K_t\approx \left(P^{xh}+Q{H}^{\mathrm{T}}\right)C^{-1}$$

(102)

is easy to implement. In fact, the only differences with the standard ensemble Kalman–Bucy filter formulation of [14] are in the additional $Q{H}^{\mathrm{T}}$ term in the Kalman gain, and a correlation between the stochastic innovation process and the model error. While the ensemble Kalman–Bucy filter gave rather satisfactory results for the numerical experiments displayed in Section 6, strongly non-Gaussian distributions might require more accurate approximations to the Kalman gain matrix (57). In that case, one could rely on the particle-based diffusion map approximation considered in [27].

7. Conclusions

In this paper, we have derived McKean–Vlasov equations for combined state and parameter estimation from continuously observed state increments. An approximate and robust implementation of these McKean–Vlasov equations in the form of a generalised ensemble Kalman–Bucy filter has been provided and applied to a range of increasingly complex model systems. Future work will address the treatment of temporally-correlated measurement and model errors, as well as a rigorous analysis of these McKean–Vlasov equations in the contexts of multi-scale dynamics and nonparametric drift estimation.