Dynamics, Noise, Delays and the Gibbs and Conditional Entropy

Abstract

In this paper we review the concepts of the Gibbs and conditional entropies and examine their dynamic behaviour when the underlying dynamics are described by ordinary differential equations or stochastic differential equations. We then go on to introduce techniques for the analogous examination when the dynamics involve delays and noise. It is found that the effects of stochastic perturbations and/or delayed dynamics may be such that the approach of the entropies to equilibrium are not necessarily monotone and are dependent on system parameters.

1. Introduction

The inexorable passage of time both fascinates and terrifies humans: witness the amount of money spent yearly trying to cosmetically reverse, or slow down, the effects of the passage of time, to obtain an idea of just how concerning people find it. This one-way evolution of time is, of course, simply a reflection of the operation of the second law of thermodynamics. Were it not for the consequences of the second law there would presumably be far less panic about the passage of time. We are quite sure, however, there would still be a lingering curiosity about why time evolves in only one direction.

The development of thermodynamics was spurred on by developments in industrial processes during the early to mid-nineteenth century as people had to deal with the concepts of heat, work, and temperature in the pursuit of increased efficiency of steam engines. The initial work by Sadi Carnot led to the formulation of the laws of thermodynamics, due to the efforts of William Thompson (later Lord Kelvin) and Rudolf Clausius. These developments in thermodynamics, including the ‘laws of thermodynamics’, culminated in the development of statistical mechanics, which makes predictions of the collective macroscopic behaviour of particles from their microscopic behaviours.

The second law of thermodynamics states that in a closed (no transfer of matter or energy in or out) system the entropy of the system can only increase (or stay constant) as time progresses. We may characterize this as the strong form of the second law, while the weak form would simply specify that the entropy will approach zero in the limit of long times. The strong form clearly implies the weak form.

In the latter part of the nineteenth century, in the developing field of statistical mechanics, individuals started to seriously contemplate how to define and compute this quantity known as entropy. Three names of giants come to mind—Ludwig Boltzmann (Germany), Henri Poincaré (France), and J. Willard Gibbs (USA)—for their insights into the concept of entropy. For reasons that we will discuss later, Gibbs was the clear winner, and Section 2 will introduce the Gibbs entropy and a generalization of it. The remainder of the paper will deal with the issue of the temporal change in entropy, and this will be based on considerations from physics. Section 3 commences our consideration of the temporal evolution of entropy, as enshrined in the second law of thermodynamics, by reviewing entropy evolution when the underlying dynamics are defined by ordinary differential equations. We then continue in the same section to consider the added effects of noise on entropy evolution. This is a review of the material presented in [1,2].

Section 4 constitutes the bulk of this paper, adding the possibility of delays in dynamics to the mix of entropy evolution and the second law. In Section 4.1 we briefly review multidimensional stochastic functional differential equations, and in Section 4.2 we turn to a consideration of general linear equations with Gaussian noise, as it is only for such equations that we know what the density evolution is and what the entropies are. This examination of the effects of Gaussian noise in linear systems as a first approximation is only possible because of the well-developed theory of Gaussian processes with continuous sample paths. We derive various formulas for entropy dynamics, and show that the conditional entropy will approach zero but need not do so as a monotone function of time. Though linear dynamics driven by Gaussian noise is well motivated by analytical tractability and by its relevance to a broad class of stochastic processes, it is essential to emphasize to the reader that the closed-form entropy expressions we obtain here rely crucially on these assumptions. In particular, the Gaussian structure and linear evolution ensure that the state remains fully characterized by its first two moments, a property that generally fails in nonlinear and/or non-Gaussian systems. Consequently, the results derived here should not be interpreted as generic entropy laws applicable beyond the linear/Gaussian framework.

In Section 4.3 we give examples of non-monotone convergence to zero of the conditional entropy. First we consider a one-dimensional example as initially presented in [3], and next we present completely new material on a second-order differential delay equation perturbed by noise. Section 4.4 contains a derivation of analytical expressions for the variances of the stationary position and velocity variables from Section 4.3. Finally in Section 4.5 we introduce one possible approach to obtain a Fokker–Planck-type equation for stochastic delay differential equations, extending the material presented in [3,4] to delay equations with both discrete and distributed delays. We conclude with some brief remarks in Section 5.

2. Entropies: Gibbs and Conditional

In this paper we are concerned with the dynamic behaviour of entropies that are relevant for physical systems, and thus we confine our attention to the Gibbs entropy, first considered in Gibbs [5], and the conditional entropy which generalizes the Gibbs entropy. The latter has been the object of study by a number of different authors [6,7,8].

In his original work Gibbs assumed that he was dealing with a system operating in a phase space X which attains a steady-state density $f_{*}$. Given this steady-state density he defined the ‘index of probability’ as $\ln f_{*}\left(x\right)$ (ln is the natural logarithm) and identified the steady-state (equilibrium) entropy to be

$$H_G\left(f_{*}\right)=-{\int }_X{f}_{*}\left(x\right)\ln f_{*}\left(x\right) dx.$$

(1)

The equilibrium entropy definition, according to Gibbs, in (1) is the standard for equilibrium computations in thermodynamics and statistical mechanics and has led to the identification of $H_G\left(f_{*}\right)$ with the equilibrium thermodynamic entropy.

There is no particular reason to restrict the definition of the Gibbs entropy in (1) to equilibrium situations, and indeed we extend it to define the time-dependent Gibbs entropy of $f(x,t)$ by

$$H_G\left(f(·,t)\right)=-{\int }_{X}f(x,t)\ln f(x,t) dx.$$

(2)

In addition to generalizing the Gibbs entropy to time-dependent situations we can also generalize the Gibbs entropy to situations in which we have a non-uniform steady-state density $f_{*}$. This generalization has acquired the name of conditional entropy [9] and is given by

$$H_c\left(f(·,t)\right|f_{*})=-{\int }_{X}f(x,t)\ln {\displaystyle \frac{f(x,t)}{f_{*}\left(x\right)}}dx.$$

(3)

It should be noted that $-H_c\left(f(·,t)\right|f_{*})$ is also known as the Kullback–Leibler divergence (I-divergence or relative entropy) [10]. It measures the distance between the two densities and it satisfies $-H_c\left(f(·,t)\right|f_{*})\ge 0$.

But what about the Austrian giant Ludwig Boltzmann and his original entropy [11] (actually there were several different varieties of Boltzmann entropies and [12] has untangled and clarified their inter-relationships)? Problems with the Boltzmann entropy have been very elegantly and clearly analyzed in [13]. The Boltzmann entropy fails to give proper equilibrium entropies in several physical situations [14], while the Gibbs entropy does not suffer from this defect. Secondly, the Gibbs and Boltzmann entropies are only in agreement for extremely dilute gases in which the constituents do not interact. Finally, studies [2] have identified situations in which the Boltzmann entropy behaves non-physically but the Gibbs entropy does not. There is a very nice study of Boltzmann versus Gibbs entropy [15] for the interested reader.

To conclude this section, we recall the amusing quotation from Eddington [16] re-framing the issue of temporal unidirectionality in terms of the behaviour of entropy, saying

“The law that entropy always increases holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell’s equations—then so much the worse for Maxwell’s equations. If it is found to be contradicted by observation—well, these experimentalists do bungle things sometimes. But if your theory is found to be against the Second Law of Thermodynamics I can give you no hope; there is nothing for it to collapse in deepest humiliation.”

3. Behavior of Entropies for Dynamics Described by Differential Equations and Their Stochastic Perturbations

3.1. Density Evolution Equations

Consider the dynamics defined by the system of ordinary differential equations

$${\displaystyle \frac{d{x}_i}{dt}}=F_i\left(x\right),\hspace{1em}i=1,\dots ,d,$$

with initial conditions $x_i\left(0\right)=x_{i,0}$ from a subset of ${\mathbb{R}}^d$. We rewrite it in the vector form as

$$x^{\prime }\left(t\right)=F\left(x\left(t\right)\right),\hspace{1em}t>0,\hspace{1em}x\left(0\right)=x_0.$$

The probability density $f(x,t)$, if it exists, of the variable $x\left(t\right)$ satisfies the generalized Liouville equation

$${\displaystyle \frac{\partial f}{\partial t}}=-\sum _i{\displaystyle \frac{\partial \left(f{F}_i\right)}{\partial x_i}},$$

(4)

and if there exists a steady-state density $f_{*}$, then it is the stationary solution of Equation (4):

$$\sum _i{\displaystyle \frac{\partial \left(f_{*}{F}_i\right)}{\partial x_i}}=0.$$

We next turn our attention to the Itô-type stochastic differential equation

$$dx\left(t\right)=F\left(x\left(t\right)\right)dt+\Sigma \left(x\left(t\right)\right) dw\left(t\right),\hspace{1em}t>0,\hspace{1em}x\left(0\right)=x_0,$$

(5)

where $\Sigma \left(x\right)=\left[{\sigma }_{ij}\left(x\right)\right]$ is a $d\times n$ matrix and w is an n-dimensional standard Wiener process on an underlying probability space $(\Omega ,\mathcal{F},\mathbb{P})$. Recall that a standard Wiener process is a Gaussian process with continuous paths and stationary independent increments for which

$$\mathbb{E}(w\left(t\right)-w\left(s\right))=0,\hspace{1em}\mathbb{E}\left((w\left(t\right)-w\left(s\right)){(w\left(t\right)-w\left(s\right))}^T\right)=I|t-s|,\hspace{1em}t,s\ge 0,$$

where the superscript T denotes the transpose and I is the identity matrix. Here and later $\mathbb{E}$ is the expectation operator corresponding to the probability measure $\mathbb{P}$. We assume that both F and $\Sigma$ satisfy a global Lipschitz condition. It is known [17,18,19] that if $x\left(t\right)$ is the solution to the stochastic differential Equation (5) then the evolution of the density function $f(x,t)$ of $x\left(t\right)$ is described by the Fokker–Planck equation (Kolmogorov forward equation)

$${\displaystyle \frac{\partial f}{\partial t}}=-\sum _{i=1}^d{\displaystyle \frac{\partial \left[F_i\left(x\right)f\right]}{\partial x_i}}+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{\displaystyle \frac{{\partial }^2\left[a_{ij}\left(x\right)f\right]}{\partial x_i\partial x_j}}$$

(6)

where

$$a_{ij}\left(x\right)=\sum _{k=1}^n{\sigma }_{ik}\left(x\right){\sigma }_{jk}\left(x\right).$$

Moreover, if there exists a stationary density $f_{*}\left(x\right)$ then it is the solution of the stationary Fokker–Planck Equation (6):

$$-\sum _{i=1}^d{\displaystyle \frac{\partial \left[F_i\left(x\right)f\right]}{\partial x_i}}+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{\displaystyle \frac{{\partial }^2\left[a_{ij}\left(x\right)f\right]}{\partial x_i\partial x_j}}=0.$$

(7)

3.1.1. Gibbs and Conditional Entropies

Differentiating with respect to time the Gibbs entropy, as given by Equation (2), leads to

$${\displaystyle \frac{d{H}_G}{dt}}=\int f\left(\sum _i{\displaystyle \frac{\partial F_i\left(x\right)}{\partial x_i}}-{\displaystyle \frac{1}{2}}\sum _{i,j}{\displaystyle \frac{{\partial }^2{a}_{ij}\left(x\right)}{\partial x_i\partial x_j}}\right) dx+{\displaystyle \frac{1}{2}}\int {\displaystyle \frac{1}{f}}\sum _{i,j=1}^d{a}_{ij}\left(x\right){\displaystyle \frac{\partial f}{\partial x_i}}{\displaystyle \frac{\partial f}{\partial x_j}} dx.$$

Note that if $a_{ij}$ values do not depend on x then we obtain

$${\displaystyle \frac{d{H}_G}{dt}}=\int f\sum _i{\displaystyle \frac{\partial F_i\left(x\right)}{\partial x_i}} dx+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{a}_{ij}\int {\displaystyle \frac{1}{f}}{\displaystyle \frac{\partial f}{\partial x_i}}{\displaystyle \frac{\partial f}{\partial x_j}} dx.$$

(8)

In particular, the second term in the right-hand side of (8) is non-negative definite, but the first term is of indeterminant sign [20] (Equation (14)). Thus the temporal behavior of the Gibbs entropy in its approach to equilibrium is unclear.

Next, if we differentiate Equation (3) for the conditional entropy with respect to time, then we obtain

$${\displaystyle \frac{d{H}_c}{dt}}={\displaystyle \frac{1}{2}}\int \left({\displaystyle \frac{f_{*}^2}{f}}\right)\sum _{i,j=1}^d{a}_{ij}\left(x\right){\displaystyle \frac{\partial }{\partial x_i}}\left({\displaystyle \frac{f}{f_{*}}}\right){\displaystyle \frac{\partial }{\partial x_j}}\left({\displaystyle \frac{f}{f_{*}}}\right) dx,$$

where we used (6), integration by parts and Equation (7) for the stationary density $f_{*}$. Consequently,

$${\displaystyle \frac{d{H}_c}{dt}}\ge 0,$$

(9)

because the matrix $\left(a_{ij}\left(x\right)\right)$ is non-negative definite. Hence, we conclude that the conditional entropy is a monotone increasing function of time and satisfies the strong form of the second law. In particular, if all $a_{ij}$ values are equal to zero (no noise) then ${\displaystyle \frac{d{H}_c}{dt}}=0$, implying that in the noise-free case when f is the solution of the Liouville Equation (4), then

$$H_c\left(f(·,t)\right|f_{*})\equiv H_c\left(f_0\right|f_{*}).$$

(10)

This last result deserves some consideration since it is so important.

1.

All of the formulations of fundamental classical, relativistic, and quantum physics are in terms of ordinary or partial differential equations and without noise.

2.

Thus any temporal evolution of entropy in these systems would appear to be impossible.

3.

It could only occur that the entropy might change (see (9)) in the presence of noise.

This is astonishing and pinpoints exactly the conundrum that physics has always faced (Equation (10)) in terms of trying to find a dynamical foundation for the second law of thermodynamics (either weak or strong).

3.1.2. Convergence to Zero of the Conditional Entropy

Various conditions implying convergence to zero of the conditional entropy are known; see, e.g., Mackey and Tyran-Kamińska [1] for a review. Here we only indicate how one can obtain a lower bound on the conditional entropy $H_c\left(f(·,t)\right|f_{*})$ knowing that the probability density function $p(x,t|x_0)$ of $x\left(t\right)$ conditional on $x\left(0\right)=x_0$ exists. One identifies the transition probability $p(x,t|x_0)$ as a solution of the Fokker–Planck Equation (6) with the initial condition $\delta (x-x_0)$. Then we obtain

$$f(x,t)=\int p(x,t|x_0)f_0\left(x_0\right) d{x}_0,$$

where $f_0$ is the density of the initial condition $x_0$, leading to

$$0\ge H_c\left(f(·,t)\right|f_{*})\ge \int H_c\left(p(·,t|x_0)\right|f_{*})f_0\left(x_0\right)d{x}_0$$

(11)

by Jensen’s inequality (see [3]). Consequently, if ${lim}_{t\to \infty }{H}_c\left(p(·,t|x_0)\right|f_{*})=0$ for all $x_0$ then

$$\underset{t\to \infty }{lim}{H}_c\left(f(·,t)\right|f_{*})=0.$$

3.2. Effects of Noise

Specific examples for which one can determine the densities $f(·,t)$ and $f_{*}$ and the corresponding entropies are mainly known for linear differential equations with additive noise. Our first example is an Ornstein–Uhlenbeck process that was developed as a perturbation to the velocity v of a Brownian particle and is given by the Langevin equation

$${\displaystyle \frac{dv}{dt}}=-\gamma v+\sigma \xi \left(t\right),$$

where $\gamma >0,\sigma >0$ and $\xi \left(t\right)$ is white noise. In this case (5) reduces to

$$dv\left(t\right)=-\gamma v\left(t\right)dt+\sigma dw\left(t\right),$$

where $w\left(t\right)$ is a standard one-dimensional Wiener process. Another frequently used example is a noisy harmonic oscillator

$${\displaystyle \frac{d^2}{d{t}^2}}x\left(t\right)+\gamma {\displaystyle \frac{d}{dt}}x\left(t\right)+\beta x\left(t\right)=\sigma \xi \left(t\right)$$

(12)

with positive constant coefficients. If we introduce the variables

$$\mathbf{x}\left(t\right)=\left(\begin{array}{c}x\left(t\right)\\ v\left(t\right)\end{array}\right)\hspace{1em}\mathrm{with}\hspace{1em}v\left(t\right)=x^{\prime }\left(t\right),$$

then we obtain

$$d\mathbf{x}\left(t\right)=A\mathbf{x}\left(t\right)dt+\Sigma dw\left(t\right),$$

where

$$A=\left(\begin{array}{cc}0& 1\\ -\beta & -\gamma \end{array}\right)\hspace{1em}\mathrm{and}\hspace{1em}\Sigma =\left(\begin{array}{c}0\\ \sigma \end{array}\right).$$

Thus, in this section we consider the general multidimensional Ornstein–Uhlenbeck process

$$dx\left(t\right)=Ax\left(t\right)dt+\Sigma dw\left(t\right)$$

(13)

where A is a $d\times d$ matrix and $\Sigma$ is a $d\times n$ matrix. The solution to Equation (13) is

$$x\left(t\right)=e^{tA}x\left(0\right)+{\int }_0^t{e}^{(t-r)A}\Sigma dw\left(r\right)$$

(14)

where $e^{tA}={\sum }_{k=0}^{\infty }{\displaystyle \frac{t^k}{k!}}{A}^k$. Note that $X\left(t\right)=e^{tA}$ is the solution to $X^{\prime }\left(t\right)=AX\left(t\right)$ satisfying $X\left(0\right)=I$ and it is called the fundamental solution of the deterministic equation $x^{\prime }\left(t\right)=Ax\left(t\right)$. The properties of stochastic integrals imply that the process

$$\zeta \left(t\right)={\int }_0^{t}X(t-s)\Sigma dw\left(r\right),\hspace{1em}t\ge 0,$$

is Gaussian. Its mean vector is equal to 0 and its covariance function is given by

$$\mathbb{E}\left(\zeta \left(t\right)\zeta {(t+s)}^T\right)={\int }_0^t{e}^{(t-r)A}\Sigma {\Sigma }^T{e}^{(t+s-r)A^T}dr,$$

by Itô’s isometry. In particular, the distribution of $\zeta \left(t\right)$ is Gaussian with mean 0 and the variance–covariance matrix

$$R\left(t\right):=\mathbb{E}\left(\zeta \left(t\right)\zeta {\left(t\right)}^T\right)={\int }_0^t{e}^{rA}\Sigma {\Sigma }^T{e}^{r{A}^T}dr.$$

(15)

Observe that the matrix $R\left(t\right)$ is symmetric. Since $e^{rA}\Sigma {\Sigma }^T{e}^{r{A}^T}$ is non-negative definite for all $r\ge 0$, the matrix $R\left(t\right)$ is also nonnegative definite, but it might not be positive definite. In particular, if the Kalman rank condition holds [21,22], i.e., for the $d\times d$ matrix A and the $d\times n$ matrix $\Sigma$, the $d\times dn$ block matrix

$$\left[A\right|\Sigma ]:=[\Sigma A\Sigma \dots A^{d-1}\Sigma ]$$

is of full rank ($\mathrm{rank}\left[A\right|\Sigma ]=d$), then the matrix $R\left(t\right)$ is of full rank. Consequently, $R\left(t\right)$ is positive definite for all $t>0$, or equivalently $\det R\left(t\right)>0$.

Let $f_{m,Q}$ be a Gaussian density with mean vector m and positive definite covariance matrix Q, i.e.,

$$f_{m,Q}\left(x\right)={\displaystyle \frac{1}{{\left(2\pi \right)}^{d/2}{\left(\det Q\right)}^{1/2}}}{e}^{-{\displaystyle \frac{1}{2}}{(x-m)}^T{Q}^{-1}(x-m)},\hspace{1em}x\in {\mathbb{R}}^d,$$

(16)

where $Q^{-1}$ is the inverse of the matrix Q. Assume that the density $f_0$ of $x\left(0\right)$ is $f_{m,Q}$. Then $x\left(t\right)$ as given in (14) has a Gaussian distribution with mean vector $\mathbb{E}\left(x\left(t\right)\right)=e^{tA}m$ and the covariance matrix $\overline{R}\left(t\right)$ is of the form

$$\overline{R}\left(t\right)=e^{tA}Q{e}^{t{A}^T}+R\left(t\right),\hspace{1em}t>0.$$

(17)

Consequently, the distribution of $x\left(t\right)$ does not depend on t if and only if $m=0$ and there exists a covariance matrix $R_{*}$ such that

$$R_{*}=e^{tA}{R}_{*}{e}^{t{A}^T}+R\left(t\right),\hspace{1em}t>0.$$

(18)

Differentiating (18) with respect to t and then setting $t=0$ we obtain the following equation for $R_{*}$:

$$A{R}_{*}+R_{*}{A}^T+\Sigma {\Sigma }^T=0.$$

(19)

Thus the covariance matrix $R_{*}$ satisfies (19). Conversely, if $R_{*}$ is the solution of (19) then (18) holds as well. Note that if $R\left(t\right)$ is positive definite then $R_{*}$ is also positive definite by (18).

3.2.1. Existence of a Stationary Density

In what follows we assume that $\mathrm{rank}\left[A\right|\Sigma ]=d$. Then $x\left(t\right)$ with $x\left(0\right)=x_0$ has a Gaussian distribution with mean $e^{tA}{x}_0$ and a positive definite covariance $R\left(t\right)$ as in (15). Letting $R{\left(t\right)}^{-1}$ be the inverse matrix of $R\left(t\right)$, the density of $x\left(t\right)$ is given by

$$p(x,t|x_0)={\displaystyle \frac{1}{{\left(2\pi \right)}^{d/2}{\left(\det R\left(t\right)\right)}^{1/2}}}exp\{-{\displaystyle \frac{1}{2}}{(x-e^{tA}{x}_0)}^{T}R{\left(t\right)}^{-1}(x-e^{tA}{x}_0)\}.$$

A stationary density $f_{*}$ exists if and only if the matrix A has only eigenvalues with negative real parts. In that case, the stationary density $f_{*}$ has the form

$$f_{*}\left(x\right)={\displaystyle \frac{1}{{\left(2\pi \right)}^{d/2}{\left(\det R_{*}\right)}^{1/2}}}exp\left\{-{\displaystyle \frac{1}{2}}{x}^T{R}_{*}^{-1}x\right\},$$

(20)

with the positive definite matrix $R_{*}$ being the unique symmetric matrix satisfying (19) and given by

$$R_{*}={\int }_0^{\infty }{e}^{sA}\Sigma {\Sigma }^T{e}^{s{A}^T}ds.$$

Since $R\left(t\right)\to R_{*}$ and $e^{tA}\to 0$ as $t\to \infty$, we have

$$\underset{t\to \infty }{lim}p(x,t|x_0)=f_{*}\left(x\right).$$

Consequently, every stationary solution of the corresponding Fokker–Planck equation is a constant multiple of the stationary density $f_{*}$.

A stationary solution of (13) can be represented as

$$Z\left(s\right)={\int }_{-\infty }^s{e}^{(s-r)A}\Sigma dw\left(r\right),\hspace{1em}s\in \mathbb{R},$$

where we extend the Wiener process w to $\mathbb{R}$ taking its independent copy $\{\overline{w}\left(t\right):t\ge 0\}$ and setting $w\left(r\right)=-\overline{w}(-r)$ for $r<0$. Observe that

$$Z\left(t\right)=e^{tA}Z\left(0\right)+\zeta \left(t\right),\hspace{1em}t\ge 0,$$

implying that $Z\left(t\right)$ is a solution of (13). Since $\zeta \left(t\right)$ is a Gaussian distributed vector, we see that $Z\left(t\right)$ has a Gaussian distribution with mean 0 and variance–covariance matrix $R_{*}$. Consequently, $Z\left(t\right)$, $t\ge 0$, is a stationary solution of (13) and $f_{*}$ is the density of $Z\left(t\right)$.

3.2.2. Noise and the Temporal Behaviour of Gibbs and Conditional Entropies

For the Gaussian density $f_{m,Q}$ in (16), the Gibbs entropy is

$$H_G\left(f_{m,Q}\right)={\displaystyle \frac{1}{2}}(d+\ln \left[{\left(2\pi \right)}^d\det Q\right]),$$

(21)

while for two Gaussian densities $f_{m_1,Q_1}$ and $f_{m_2,Q_2}$ the conditional entropy is

$$\begin{array}{c}{H}_c\left(f_{m_1,Q_1}\right|f_{m_2,Q_2})={\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{\det Q_1}{\det Q_2}}+{\displaystyle \frac{1}{2}}\mathrm{tr}\left[(Q_1^{-1}-Q_2^{-1})Q_1\right]\hfill \\ \hfill -{\displaystyle \frac{1}{2}}\mathrm{tr}\left[Q_2^{-1}(m_1-m_2){(m_1-m_2)}^T\right].\hspace{1em}\end{array}$$

(22)

Consequently, if $f_0=f_{m,Q}$ with $m=\mathbb{E}\left(x\right(0\left)\right)$ and $Q=\mathbb{E}(x\left(0\right)-m){(x\left(0\right)-m)}^T$, then for the density $f(x,t)$ of $x\left(t\right)$ we obtain the following

$$H_G\left(f(·,t)\right)={\displaystyle \frac{1}{2}}(d+\ln \left[{\left(2\pi \right)}^d\det \overline{R}\left(t\right)\right]),$$

(23)

where $\overline{R}\left(t\right)$ is given by (17) and

$$H_c\left(f(·,t)\right|f_{*})={\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{\det \overline{R}\left(t\right)}{\det R_{*}}}+{\displaystyle \frac{1}{2}}\mathrm{tr}(I-R_{*}^{-1}\overline{R}\left(t\right))-{\displaystyle \frac{1}{2}}\mathrm{tr}\left(R_{*}^{-1}{e}^{tA}m{m}^T{e}^{t{A}^T}\right).$$

(24)

Note that if $Q=R_{*}$ then $\overline{R}\left(t\right)=R_{*}$ and Equation (24) reduces to

$$H_c\left(f(·,t)\right|f_{*})=-{\displaystyle \frac{1}{2}}\mathrm{tr}\left(R_{*}^{-1}{e}^{tA}m{m}^T{e}^{t{A}^T}\right)$$

which is always an increasing function of time when $m\ne 0$.

Observe that if $m=x\left(0\right)=:x_0$ and $Q=0$, then we recover $H_c\left(p(·,t|x_0)\right|f_{*})$ from Equation (24). Thus, for an arbitrary initial density $f_0$ with finite variance, we obtain from (11) the lower bound

$$H_c\left(f(·,t)\right|f_{*})\ge {\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{\det R\left(t\right)}{\det R_{*}}}+{\displaystyle \frac{1}{2}}\mathrm{tr}(I-R_{*}^{-1}R\left(t\right))-{\displaystyle \frac{1}{2}}\int \mathrm{tr}\left(R_{*}^{-1}{e}^{tA}{x}_0{x}_0^T{e}^{t{A}^T}\right)f_0\left(x_0\right)d{x}_0$$

for the conditional entropy. Since

$$\underset{t\to \infty }{lim}{e}^{tA}=0\hspace{1em}\mathrm{and}\hspace{1em}\underset{t\to \infty }{lim}R\left(t\right)=R_{*},$$

it is clear that ${lim}_{t\to \infty }{H}_c\left(f(·,t)\right|f_{*})=0$. Remember that we have ${\dot{H}}_c\left(f(·,t)\right|f_{*})\ge 0$ by (9) so we have the strong form of the second law obeyed in this case.

Thus, the conditional entropy $H_c$ for these stochastic systems increases monotonically to 0. To see what the temporal behavior of the Gibbs entropy is, consider first the one-dimensional Ornstein–Uhlenbeck process. If we start with a Gaussian density with mean $m_0$ and variance ${\sigma }_0^2$

$$f_0\left(v_0\right)={\displaystyle \frac{1}{\sqrt{2\pi {\sigma }_0^2}}}exp\left\{-{\displaystyle \frac{{(v_0-m_0)}^2}{2{\sigma }_0^2}}\right\},$$

then by Equations (17) and (15) we obtain

$$\overline{R}\left(t\right)={\sigma }_0^2{e}^{-2\gamma t}+{\int }_0^t{\sigma }^2{e}^{-2\gamma s}ds={\sigma }_0^2{e}^{-2\gamma t}+{\displaystyle \frac{{\sigma }^2}{2\gamma }}(1-e^{-2\gamma t})$$

and the stationary density is given by

$$f_{*}\left(v\right)={\displaystyle \frac{1}{\sqrt{2\pi {\sigma }_{*}^2}}}exp\left\{-{\displaystyle \frac{v^2}{2{\sigma }_{*}^2}}\right\}\hspace{1em}\mathrm{with} {\sigma }_{*}^2={\displaystyle \frac{{\sigma }^2}{2\gamma }}.$$

Thus, the Gibbs entropy is

$$H_G\left(f(·,t)\right)={\displaystyle \frac{1}{2}}+{\displaystyle \frac{1}{2}}\ln \left(2\pi \right)+{\displaystyle \frac{1}{2}}\ln ({\sigma }_{*}^2+({\sigma }_0^2-{\sigma }_{*}^2)e^{-2\gamma t}),$$

and it is either an increasing function of time when ${\sigma }_0^2>{\sigma }_{*}^2$ or a decreasing function of time when ${\sigma }_0^2<{\sigma }_{*}^2$, while the conditional entropy, given by

$$H_c\left(f(·,t)\right|f_{*})={\displaystyle \frac{1}{2}}\ln \left(1+\left({\displaystyle \frac{{\sigma }_0^2}{{\sigma }_{*}^2}}-1\right)e^{-2\gamma t}\right)-{\displaystyle \frac{1}{2}}\left({\displaystyle \frac{{\sigma }_0^2}{{\sigma }_{*}^2}}-1\right)e^{-2\gamma t}-{\displaystyle \frac{m_0^2}{2{\sigma }_{*}^2}}{e}^{-2\gamma t},$$

is a strictly increasing function of time for ${\sigma }_0^2\ne {\sigma }_{*}^2$ or $m_0\ne 0$.

The temporal behavior of the Gibbs entropy can have more complicated patterns; see Figure 1 and Figure 2 and [1,2] in the case of the noisy harmonic oscillator.

Figure 1. Entropy behaviour in the noisy harmonic oscillator (12). The left-hand panel shows the plot of the conditional entropy given by (24) and the right panel shows the plot of the Gibbs entropy given by (23). The parameters are $\gamma =1$, $\beta =1/4$, and ${\sigma }^2=1$ and the variance–covariance matrix Q is diagonal, with the initial variances of position and velocity being equal to 1 and 2 respectively, while the initial means are set to be equal to 0.

Figure 2. Entropy behaviour in the noisy harmonic oscillator (12). The left-hand panel shows the plot of the conditional entropy given by (24) and the right panel shows the plot of the Gibbs entropy given by (23). The parameters are $\gamma =1$, $\beta =20$, and ${\sigma }^2=40$ and the variance–covariance matrix Q is again diagonal, with the initial variances of position and velocity equal to 0.9 and 25 respectively, while the initial means are set equal to 0.

4. Adding Delays in the Dynamics

Though the subject of the dynamics of differential delay equations (DDEs) is well developed and relatively mature, things are not quite so clear when one wishes to examine the role of delays in the temporal evolution of entropy. This is because in studying entropy dynamics, the density in a system plays a central role, and there are great difficulties in the study of density evolution under the action of delayed dynamics. Thus we investigate the temporal behavior of entropies for linear systems with additive perturbations leading to Gaussian processes with densities being determined through a vector-valued mean function and a matrix-valued variance–covariance function extending the one-dimensional linear differential equations with a single delay in [3]. We show that the temporal behavior of the conditional entropy depends on the initial condition and is not necessarily a monotone function of time.

The monograph [23] treats in detail the techniques that have been tried unsuccessfully to examine density evolution in delayed systems, while [24] examines certain aspects of this problem from the perspective of microphysical systems. Historically, important approaches have been made by a number of different groups ([24,25,26,27,28,29,30,31,32]), but it is unfortunately the situation that foundational issues are still unresolved and a satisfactory and unified theory allowing one to examine the evolution of densities in (infinite dimensional) delayed systems is yet to be developed. We [3,4] have employed various techniques to circumvent the problems outlined in [23] for the single delay case. At the end of this section, we show how this approach can be extended to functional differential equations with both discrete and distributed delays. The presentation is somewhat technical, but unavoidable, because of the difficulties encountered due to the infinite dimensional nature of systems with delayed dynamics.

4.1. Stochastic Functional Differential Equations

DDEs or more generally functional differential equations (FDEs) are equations in which the rate of change of a system at the present time depends not only on its current state but also on its state at one or more times in the past. The evolution of the system cannot be determined from a single initial value, because the equation explicitly refers to past states. Instead, one must specify a history function, which prescribes the state of the system over a whole time interval in the past—typically from time $-\tau$ up to 0, where $\tau$ is the maximum delay. This history function acts as the ‘initial condition’ for the differential equation: once it is given, the equation can be used to compute the solution forward in time, which motivates working on the space of paths that are continuous functions on the interval $[-\tau ,0]$. In this path-space setting, the solution map, denoted below as $S_t$, provides a clean way to describe a dynamical system showing how the initial function evolves forward in time. Because delayed dynamics are inherently non-Markovian at the level of the observed state, we need to work on this space of functions to recover the Markov property when adding noise. In Section 4.5 we recall the associated extended generator that captures the infinitesimal time evolution of observables on this enlarged space. Once this framework is in place, we can project back from paths to the observed state variable, recovering evolution equations for its probability density. We refer the reader to [33,34,35] for a thorough introduction to functional differential equations and their stochastic perturbations.

Suppose $\tau >0$ is a given real number. For a continuous $x:[-\tau ,\infty )\to {\mathbb{R}}^d$ we introduce the notation $x_t\in C$, where

$$x_t\left(s\right)=x(t+s)\hspace{1em}\mathrm{for} s\in [-\tau ,0],t\ge 0,$$

and $C:=C([-\tau ,0],{\mathbb{R}}^d)$ is the space of continuous functions $\varphi :[-\tau ,0]\to {\mathbb{R}}^d$. Given a mapping $F:C\to {\mathbb{R}}^d$ we consider the functional differential equation

$$\left\{\begin{array}{c}{x}^{\prime }\left(t\right)=F\left(x_t\right),\hspace{1em}t\ge 0,\hfill \\ x_0=\varphi .\hfill \end{array}\right.$$

(25)

We assume that F satisfies a global Lipschitz condition so that a unique solution $x:[-\tau ,\infty )\to {\mathbb{R}}^d$ of (25) exists for each initial condition $x_0=\varphi \in C$, and it depends continuously on $(t,\varphi )$ and $x_t\in C$ for $t>0$. Then the solution map $S_t:C\to C$ given by

$$S_t\left(\varphi \right)\left(s\right)=x_t\left(s\right)\hspace{1em}\mathrm{for} s\in [-\tau ,0],$$

is well defined for all $t\ge 0$ and gives rise to a semi-dynamical system ${\left\{S_t\right\}}_{t\in {\mathbb{R}}^{+}}$ on the Banach space C equipped with the supremum norm [33]. In particular if

$$F\left(\varphi \right)=\mathcal{F}\left(\varphi \right(0),\varphi (-\tau \left)\right),\hspace{1em}\varphi \in C,$$

for some transformation $\mathcal{F}:{\mathbb{R}}^d\times {\mathbb{R}}^d\to {\mathbb{R}}^d$, then we recover the case of a single delay equation whose density evolution was studied in [3,4].

We next look at a noise-perturbed version of (25):

$$\left\{\begin{array}{c}dx\left(t\right)=F\left(x_t\right)dt+\Sigma \left(x_t\right)dw\left(t\right),\hspace{1em}t\ge 0,\hfill \\ x_0=\varphi ,\hfill \end{array}\right.$$

(26)

where ${\left\{w\left(t\right)\right\}}_{t\ge 0}$ is again an n-dimensional standard Wiener process and the stochastic perturbation has an amplitude (diffusion coefficient) $\Sigma$, potentially continuously dependent on $x_t$, with values in the space ${\mathbb{R}}^{n\times d}$ of $n\times d$ matrices. We assume that both F and $\Sigma$ satisfy a global Lipschitz condition, so that the stochastic functional differential Equation (26) (see [35]) has a pathwise solution x and that ${\left\{x_t\right\}}_{t\ge 0}$ is a Markov process with values in the space C, where again

$$x_t\left(s\right)=x(t+s),\hspace{1em}s\in [-\tau ,0],t\ge 0.$$

(27)

We end this section with a remark about how one can obtain a lower bound on the conditional entropy as in Section 3, but now for the densities defined by solutions of the stochastic functional differential equation considered in this section. If it happens that the solution $x\left(t\right)=x_t\left(0\right)$ of (26) with each deterministic initial condition $\varphi \in C$ has a probability density function, say $p(x,t|\varphi )$, then the function

$$f(x,t)={\int }_{C}p(x,t|\varphi ){\mu }_0\left(d\varphi \right),\hspace{1em}x\in {\mathbb{R}}^d,t\ge 0,$$

defines the density of $x\left(t\right)=x_t\left(0\right)$ when the initial condition $x_0$ has the distribution ${\mu }_0$, which is now a distribution on the function space C. Using Jensen’s inequality, we obtain [3]

$$H_c\left(f(·,t)\right|f_{*})\ge {\int }_C{H}_c\left(p(·,t|\varphi )\right|f_{*}){\mu }_0\left(d\varphi \right).$$

Consequently, if $H_c\left(p(·,t|\varphi )\right|f_{*})$ converges to zero as $t\to \infty$ for all $\varphi \in C$ then we conclude that

$$\underset{t\to \infty }{lim}{H}_c\left(f(·,t)\right|f_{*})=0.$$

4.2. A Linear Functional Differential Equation

A particular example of a linear functional differential equation is

$$x^{\prime }\left(t\right)=Ax\left(t\right)+Bx(t-\tau )+{\int }_{-\tau }^{0}A\left(\theta \right)x(t+\theta )d\theta ,$$

where A and B are $d\times d$ matrices and $A(·):[-\tau ,0]\to {\mathbb{R}}^{d\times d}$ is a continuous matrix-valued function. Once again, to proceed, we require some technical developments specific to linear delayed dynamics.

Suppose that $F:C\to {\mathbb{R}}^d$ is a continuous linear mapping. By the Riesz representation theorem, we can write

$$F\left(\varphi \right)=A\varphi \left(0\right)+{\int }_{-\tau }^{0}d\eta \left(\theta \right)\varphi \left(\theta \right),$$

where $\eta ={\left[{\eta }_{ij}\right]}_{i,j=1,\dots ,d}$ is a matrix-valued function of bounded variation and the integral is the Lebesgue–Stieltjes integral. Note that $F\left(\varphi \right)-A\varphi \left(0\right)$ is a d-dimensional vector with coordinates

$$\sum _{j=1}^d{\int }_{-\tau }^0{\varphi }_j\left(\theta \right)d{\eta }_{ij}\left(\theta \right),\hspace{1em}i=1,\dots ,d.$$

Let $S_t:C\to C$ be the solution map of the functional linear equation

$$\left\{\begin{array}{c}{\displaystyle x^{\prime }\left(t\right)=Ax\left(t\right)+{\int }_{-\tau }^{0}d\eta \left(\theta \right)x(t+\theta ),\hspace{1em}t\ge 0,}\hfill \\ x_0=\varphi .\hfill \end{array}\right.$$

(28)

In particular, we have

$$x\left(t\right)=S_t\varphi \left(0\right),\hspace{1em}t\ge 0.$$

The stability of the trivial solution $x\left(t\right)\equiv 0$ is determined by the roots of the characteristic (eigenvalue) equation

$$\det h\left(\lambda \right)=0,\hspace{1em}h\left(\lambda \right)=\lambda I-A-{\int }_{-\tau }^0{e}^{\theta \lambda }d\eta \left(\theta \right),$$

(29)

where $h:\mathbb{C}\to {\mathbb{C}}^{d\times d}$. If all solutions of (29) have negative real parts, then all solutions of (28) tend to 0 as $t\to \infty$.

The fundamental solution $X\left(t\right)$ of Equation (28) is the $d\times d$ matrix function satisfying

$$X^{\prime }\left(t\right)=AX\left(t\right)+{\int }_{-\tau }^{0}d\eta \left(\theta \right)X(t+\theta )$$

(30)

with

$$X\left(s\right)=0\hspace{1em}\mathrm{for} s\in [-\tau ,0)\hspace{1em}\mathrm{and}\hspace{1em}X\left(0\right)=I,$$

(31)

where I is the identity matrix. Let

$${\alpha }_0=sup\{\mathrm{Re}\left(\lambda \right):\det h\left(\lambda \right)=0\}.$$

(32)

As in the scalar case the fundamental matrix solution X has the following bound [33]: for each $\alpha >{\alpha }_0$ there exists a constant $C>0$ such that

$$\left|X\left(t\right)\right|\le C{e}^{\alpha t}\hspace{1em}\mathrm{for} \mathrm{all} t>0,$$

(33)

where $|·|$ is a norm of the $d\times d$ matrix.

Let $\Sigma$ be a $d\times n$ matrix and w be an n-dimensional standard Wiener process. Given an initial continuous function $\varphi :[-\tau ,0]\to {\mathbb{R}}^d$ we can write the solution of

$$\left\{\begin{array}{c}{\displaystyle dx\left(t\right)=(Ax\left(t\right)+{\int }_{-\tau }^{0}d\eta \left(\theta \right)x(t+\theta ))dt+\Sigma dw\left(t\right),\hspace{1em}t>0,}\hfill \\ x_0=\varphi ,\hfill \end{array}\right.$$

(34)

as

$$x\left(t\right)=S_t\varphi \left(0\right)+{\int }_0^{t}X(t-r)\Sigma dw\left(r\right),$$

where $S_t$ is the solution map and $X\left(t\right)$ is the fundamental matrix solution of the deterministic part (28). This corresponds to (14) in the case of $\tau =0$. The process

$$\zeta \left(t\right)={\int }_0^{t}X(t-r)\Sigma dw\left(r\right),\hspace{1em}t\ge 0,$$

is a Gaussian process with values in ${\mathbb{R}}^d$ that have zero mean vector and the matrix-valued covariance function of the form

$$\mathbb{E}\left(\zeta \left(t\right)\zeta {(t+s)}^T\right)={\int }_0^{t}X(t-r)\Sigma {\Sigma }^{T}X{(t+s-r)}^{T}dr,\hspace{1em}t,s\ge 0,$$

by Itô’s isometry. The variance of $\zeta \left(t\right)$ is thus given by

$$R\left(t\right):=\mathrm{Var}\left(\zeta \left(t\right)\right)=\mathbb{E}\left(\zeta \left(t\right)\zeta {\left(t\right)}^T\right)={\int }_0^{t}X\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^{T}dr,\hspace{1em}t\ge 0.$$

(35)

Note that $R\left(t\right)$ is as in (15) when $\tau =0$. Although the matrix $R\left(t\right)$ in (35) is symmetric and non-negative definite, it does not need be positive definite. Observe that $\zeta$ defines for each $t\ge 0$ the segment ${\zeta }_t$ by

$${\zeta }_t\left(s\right)=\left\{\begin{array}{cc}\zeta (t+s),\hfill & \mathrm{if} s+t\ge 0,s\in [-\tau ,0],\hfill \\ 0,\hfill & \mathrm{if} s+t<0,s\in [-\tau ,0].\hfill \end{array}\right.$$

Consequently, the process $x_t$ in (27) being defined by the solution of (34) has the following representation:

$$x_t=S_t\varphi +{\zeta }_t,\hspace{1em}{x}_0=\varphi .$$

(36)

Let ${\mu }_t$ be the distribution of $x_t$ as in (36). Now suppose that $x_0=\varphi$ is a Gaussian process. Then $x_t$ is a Gaussian process and $x\left(t\right)$ is an ${\mathbb{R}}^d$-valued Gaussian random vector. To characterize its distribution $\mu \left(t\right)$ it is enough to find its mean vector and the variance–covariance matrix. We have

$$\mathbb{E}\left(x\left(t\right)\right)=S_t{m}_0\left(0\right),$$

where $S_t{m}_0$ is the solution of (28) with the initial condition being the mean function

$$m_0\left(s\right)=\mathbb{E}\left(\varphi \left(s\right)\right),\hspace{1em}s\in [-\tau ,0],$$

and the variance of $x\left(t\right)$ is given by the equation

$$\overline{R}\left(t\right):=\mathrm{Var}\left(x\left(t\right)\right)=\mathrm{Var}\left(S_t\varphi \left(0\right)\right)+R\left(t\right),\hspace{1em}t>0.$$

(37)

In particular, if $x_0=\varphi$ is deterministic, so that the function $\varphi$ is fixed, then $x_t$ is also a Gaussian process and $\mathrm{Var}\left(S_t\varphi \left(0\right)\right)=0$. If, on the other hand, $S_t\varphi$ is an ${\mathbb{R}}^d$-valued Gaussian process with mean vector $S_t{m}_0$ then we can assume that $m_0\equiv 0$.

4.2.1. Existence of a Stationary Density

Suppose that ${\alpha }_0<0$, where ${\alpha }_0$ is defined in (32). Then $S_t\varphi \left(0\right)\to 0$ as $t\to \infty$. It follows from (33) that for some $\alpha <0$ and $C>0$

$$|{\int }_t^{\infty }X\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^{T}dr|\le C|\Sigma {\Sigma }^T|{\int }_t^{\infty }{e}^{2\alpha r}dr\to 0\hspace{1em}\mathrm{as} t\to \infty ,$$

implying that the following limit exists and is finite:

$$\underset{t\to \infty }{lim}R\left(t\right)={\int }_0^{\infty }X\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^{T}dr:=R_{*}.$$

(38)

If $R\left(t\right)$ is non-singular (equivalently, $R\left(t\right)$ is positive definite), then the density p of the distribution $\mu \left(t\right)$ of $x\left(t\right)$ is given by the Gaussian density

$$p(x,t|\varphi )={\displaystyle \frac{1}{{\left(2\pi \right)}^{d/2}{\left(\det R\left(t\right)\right)}^{1/2}}}{e}^{-{\displaystyle \frac{1}{2}}{(x-S_t\varphi \left(0\right))}^{T}R{\left(t\right)}^{-1}(x-S_t\varphi \left(0\right))},\hspace{1em}x\in {\mathbb{R}}^d,$$

(39)

where $R{\left(t\right)}^{-1}$ is the inverse matrix of the covariance matrix $R\left(t\right)$. If $R\left(t\right)$ is positive definite, we see that its limit $R_{*}$ is also positive definite and thus invertible. Then we have

$$\underset{t\to \infty }{lim}p(x,t|\varphi )={\displaystyle \frac{1}{{\left(2\pi \right)}^{d/2}{\left(\det R_{*}\right)}^{1/2}}}{e}^{-{\displaystyle \frac{1}{2}}{x}^T{R}_{*}^{-1}x}=f_{*}\left(x\right),$$

(40)

and $f_{*}$ is as in (20) with $R_{*}$ now given by (38). Equation (34) has a stationary solution of the form [35] (Corollary 4.2.1)

$$Z\left(s\right)={\int }_{-\infty }^{s}X(s-r)\Sigma dw\left(r\right),\hspace{1em}s\in \mathbb{R}$$

(41)

and the distribution of $Z\left(s\right)$ is Gaussian with density $f_{*}$ as in (40).

4.2.2. The Gibbs and Conditional Entropies

In this section, we assume that the matrix $R\left(t\right)$ is positive definite for all sufficiently large t. Let the initial condition $\varphi$ be a Gaussian process and let $f(·,t)$ be the Gaussian density of $x\left(t\right)$ with variance $\overline{R}\left(t\right)$ as in (37) and the mean vector $S_t{m}_0$ with $m_0$ being the mean vector of the initial condition $\varphi$. By (21) we have

$$H_G\left(f(·,t)\right)={\displaystyle \frac{1}{2}}\left\{d+\left[\ln {\left(2\pi \right)}^d\det \overline{R}\left(t\right))\right]\right\},$$

(42)

and thus

$${\displaystyle \frac{d{H}_G\left(f(·,t)\right)}{dt}}={\displaystyle \frac{1}{2\det \overline{R}\left(t\right)}}{\displaystyle \frac{d\det \overline{R}\left(t\right)}{dt}}.$$

By Jacobi’s formula we have

$${\displaystyle \frac{d\ln \det \overline{R}\left(t\right)}{dt}}=\mathrm{tr}\left(\overline{R}{\left(t\right)}^{-1}{\displaystyle \frac{d\overline{R}\left(t\right)}{dt}}\right),$$

implying that

$${\displaystyle \frac{d{H}_G\left(f(·,t)\right)}{dt}}={\displaystyle \frac{1}{2}}\mathrm{tr}\left(\overline{R}{\left(t\right)}^{-1}{\displaystyle \frac{d\overline{R}\left(t\right)}{dt}}\right).$$

If we take $f_{m_1,Q_1}$ as in (39) and $f_{m_2,Q_2}$ as in (40), then $Q_1=\overline{R}\left(t\right)$, $m_1=S_t{m}_0\left(0\right)$, and $Q_2=R_{*}$, $m_2=0$. Hence, by (22), we have

$$\begin{array}{c}{H}_c\left(f(·,t)\right|f_{*})={\displaystyle \frac{1}{2}}\ln \det \left(R_{*}^{-1}\overline{R}\left(t\right)\right)+{\displaystyle \frac{1}{2}}\mathrm{tr}[I-R_{*}^{-1}\overline{R}\left(t\right)]\hfill \\ \hfill -{\displaystyle \frac{1}{2}}\mathrm{tr}\left[R_{*}^{-1}{S}_t{m}_0\left(0\right){\left(S_t{m}_0\left(0\right)\right)}^T\right]\hspace{1em}\end{array}$$

(43)

and so the temporal derivative of the conditional entropy satisfies

$${\displaystyle \frac{d{H}_c\left(f(·,t)\right|f_{*})}{dt}}={\displaystyle \frac{1}{2}}\mathrm{tr}\left(\overline{R}{\left(t\right)}^{-1}{\displaystyle \frac{d\overline{R}\left(t\right)}{dt}}-R_{*}^{-1}{\displaystyle \frac{d\overline{R}\left(t\right)}{dt}}-R_{*}^{-1}{\displaystyle \frac{d{S}_t{m}_0\left(0\right){\left(S_t{m}_0\left(0\right)\right)}^T}{dt}}\right).$$

(44)

If we take as the initial condition $x_0=\varphi$ with $\varphi \left(s\right)=Z\left(s\right)+m_0\left(s\right)$ for $s\in [-\theta ,0]$, where Z is the Gaussian process, being the stationary solution of (34), and $m_0$ is a non-zero initial mean function, then $\overline{R}\left(t\right)=R_{*}$ and we obtain from (43) that

$$H_c\left(f(·,t)\right|f_{*})=-{\displaystyle \frac{1}{2}}\mathrm{tr}\left[R_{*}^{-1}{S}_t{m}_0\left(0\right){\left(S_t{m}_0\left(0\right)\right)}^T\right].$$

(45)

However, we cannot conclude that the conditional entropy $H_c$, given by the function $-{\displaystyle \frac{1}{2}}\mathrm{tr}\left[R_{*}^{-1}{S}_t{m}_0\left(0\right){\left(S_t{m}_0\left(0\right)\right)}^T\right]$, is strictly increasing although it converges to zero. See Section 4.3 for examples.

If, on the other hand, we take $m_0=\varphi$ with $\varphi$ fixed, so that the initial condition is deterministic, then $\overline{R}\left(t\right)=R\left(t\right)$. Hence, we recover $H_G\left(p(·,t|\varphi )\right)$ from Equation (42) and $H_c\left(p(·,t|\varphi )\right|f_{*})$ from (43). Since the matrix ${\displaystyle \frac{dR\left(t\right)}{dt}}$ is non-negative definite, we see that the matrix $R{\left(t\right)}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}$ is also nonnegative definite. Consequently, its trace is nonnegative, implying that

$${\displaystyle \frac{d{H}_G\left(p(·,t|\varphi )\right)}{dt}}\ge 0$$

(46)

so the Gibbs entropy of the density p is monotone increasing and thus the strong form of the second law holds. Since

$$R{\left(t\right)}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}-R_{*}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}=R_{*}^{-1}(R_{*}-R\left(t\right))R{\left(t\right)}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}$$

and the matrix $R_{*}-R\left(t\right)$, given by

$$R_{*}-R\left(t\right)={\int }_t^{\infty }X\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^{T}dr,$$

is non-negative definite, we conclude that

$$\mathrm{tr}\left(R{\left(t\right)}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}-R_{*}^{-1}{\displaystyle \frac{dR\left(t\right)}{dt}}\right)\ge 0.$$

However, as in the one-dimensional case we cannot claim that

$$-\mathrm{tr}\left(R_{*}^{-1}{\displaystyle \frac{d{S}_t\varphi \left(0\right){\left(S_t\varphi \left(0\right)\right)}^T}{dt}}\right)\ge 0$$

and once again the presence of the delay has opened the possibility of a non-monotone approach of the conditional entropy to zero and, thus, a shift to the weak form of the second law.

Finally, for the density $f(x,t)$ of $x_t\left(0\right)$ with initial condition $x_0$ chosen according to a distribution ${\mu }_0$ we obtain

$$H_c\left(f(·,t)\right|f_{*})\ge {\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{\det R\left(t\right)}{\det R_{*}}}+{\displaystyle \frac{1}{2}}\mathrm{tr}[I-R_{*}^{-1}R\left(t\right)]-{\displaystyle \frac{1}{2}}{\int }_C\mathrm{tr}\left[R_{*}^{-1}{S}_t\varphi \left(0\right){\left(S_t\varphi \left(0\right)\right)}^T\right]{\mu }_0\left(d\varphi \right).$$

Since $R\left(t\right)\to R_{*}$ and $S_t\varphi \left(0\right)\to 0$ as $t\to \infty$, we conclude that

$$\underset{t\to \infty }{lim}{H}_c\left(f(·,t)\right|f_{*})=0.$$

(47)

4.2.3. The Covariance of the Stationary Solution

To find $R_{*}$ let us look at the matrix-valued covariance of the process $\left\{Z\right(s):s\in \mathbb{R}\}$:

$$K\left(s\right)=\mathbb{E}\left(Z\left(t\right)Z{(s+t)}^T\right),\hspace{1em}s\in \mathbb{R}.$$

The covariance function is given by

$$K\left(s\right)={\int }_{-\infty }^{min\{t,s+t\}}X(t-u)\Sigma {\Sigma }^{T}X{(s+t-u)}^{T}du$$

by the Itô isometry. Changing the variables in the integral leads to

$$K\left(s\right)={\int }_0^{\infty }X\left(r\right)\Sigma {\Sigma }^{T}X{(r+s)}^{T}dr,\hspace{1em}s\ge 0,$$

and

$$K\left(s\right)={\int }_0^{\infty }X(r-s)\Sigma {\Sigma }^{T}X{\left(r\right)}^{T}dr,\hspace{1em}s<0.$$

Hence $K\left(s\right)=K{(-s)}^T$ for $s<0$. Note that

$$R_{*}=K\left(0\right)=K{\left(0\right)}^T.$$

Let us now derive a differential equation for the matrix $K\left(t\right)$ for $t>0$. We have

$$K^{\prime }\left(t\right)={\int }_0^{\infty }X\left(r\right)\Sigma {\Sigma }^T{X}^{\prime }{(r+t)}^{T}dr.$$

It follows from (30) that

$$X^{\prime }{(r+t)}^T=X{(r+t)}^T{A}^T+{\int }_{-\tau }^{0}X{(r+t+\theta )}^{T}d{\eta }^T\left(\theta \right),\hspace{1em}r,t>0,$$

implying that K satisfies the equation

$$K^{\prime }\left(t\right)=K\left(t\right)A^T+{\int }_{-\tau }^{0}K(t+\theta )d{\eta }^T\left(\theta \right),\hspace{1em}t>0.$$

(48)

Moreover, we have

$${\displaystyle \frac{d}{dr}}(X\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^T)=X^{\prime }\left(r\right)\Sigma {\Sigma }^{T}X{\left(r\right)}^T+X\left(r\right)\Sigma {\Sigma }^T{X}^{\prime }{\left(r\right)}^T$$

and

$${\int }_0^{\infty }X\left(r\right)\Sigma {\Sigma }^T{X}^{\prime }{\left(r\right)}^{T}dr=K^{\prime }\left(0\right),\hspace{1em}\mathrm{where} K^{\prime }\left(0\right)=\underset{s\downarrow 0}{lim}{K}^{\prime }\left(s\right).$$

Hence, we obtain

$$K^{\prime }{\left(0\right)}^T+K^{\prime }\left(0\right)=-\Sigma {\Sigma }^T\hspace{1em}\mathrm{with} K^{\prime }\left(0\right)=K\left(0\right)A^T+{\int }_{-\tau }^{0}K\left(\theta \right)d{\eta }^T\left(\theta \right).$$

(49)

Consequently, if we want to find $R_{*}=K\left(0\right)$, then we need to solve Equation (48) subject to condition (49).

In general, it is impossible to find a solution of (48), though in the one-dimensional case with a single discrete delay one was derived in [36]; see also [28]. Here we obtain a second-order differential equation for the covariance K of the stationary solution of equation

$$dx\left(t\right)=\left[Ax\right(t)+Bx(-\tau \left)\right]dt+\Sigma dw\left(t\right),$$

(50)

where B is a $d\times d$ matrix. It follows from (48) that the covariance $K\left(t\right)$ of the stationary solution satisfies

$$K^{\prime }\left(t\right)=K\left(t\right)A^T+K(t-\tau )B^T,\hspace{1em}t>0,$$

and $K\left(s\right)=K{(-s)}^T$ for $s\le 0$. For $t\in (0,\tau )$ we have $K(t-\tau )=K{(\tau -t)}^T$. Thus, we obtain

$$K^{\prime }\left(t\right)=K\left(t\right)A^T+K{(\tau -t)}^T{B}^T,\hspace{1em}t\in (0,\tau ).$$

(51)

Next, differentiating the function $U\left(t\right)=K{(\tau -t)}^T$ we get

$$U^{\prime }\left(t\right)={\displaystyle \frac{d}{dt}}\left[K{(\tau -t)}^T\right]=-AK{(\tau -t)}^T-BK\left(t\right)$$

implying that

$$K^{\prime \prime }\left(t\right)=K^{\prime }\left(t\right)A^T+U^{\prime }\left(t\right)B^T=K^{\prime }\left(t\right)A^T-AK{(\tau -t)}^T{B}^T-BK\left(t\right)B^T.$$

From Equation (51) we obtain

$$K{(\tau -t)}^T{B}^T=K^{\prime }\left(t\right)-K\left(t\right)A^T.$$

Consequently, the covariance satisfies for $t\in (0,\tau )$ the second-order matrix differential equation

$$K^{\prime \prime }\left(t\right)=K^{\prime }\left(t\right)A^T-A{K}^{\prime }\left(t\right)+AK\left(t\right)A^T-BK\left(t\right)B^T$$

(52)

with

$$K^{\prime }\left(0\right)+K^{\prime }{\left(0\right)}^T=-\Sigma {\Sigma }^T,\hspace{1em}{K}^{\prime }\left(0\right)=K\left(0\right)A^T+K{\left(\tau \right)}^T{B}^T.$$

(53)

Let us summarize symbols used in Section 4.2:

• F—a general linear continuous mapping on $C=C([-\tau ,0],{\mathbb{R}}^d)$, A, B – $d\times d$ matrices;
• $S_t$—the solution map of (28); $S_t\varphi \left(0\right)$ is the solution of (28);
• $X\left(t\right)$—the fundamental matrix solution (30) and (31);
• $x\left(t\right)$—the solution of (34); its variance is $\overline{R}\left(t\right)$;
• $\zeta \left(t\right)$—a Gaussian vector with zero mean and the variance equal to $R\left(t\right)$ as in (35);
• $R_{*}$—the limit of the variance $R\left(t\right)$ as defined in (38);
• $Z\left(s\right),s\in \mathbb{R}$—the Gaussian process as in (41) that is the stationary solution of (34);
• $K\left(s\right)$—the matrix-valued covariance function of the stationary process Z; $K\left(0\right)=R_{*}$.

4.3. Examples

4.3.1. A Linear Equation with Single Delay in One Dimension

Consider the single delay Equation (50) in the case of $d=1$, so that $A=a$ and $B=b$ are real constants

$$dx\left(t\right)=\left(ax\right(t)+bx(t-\tau \left)\right)dt+\sigma dw\left(t\right).$$

The entirety of the $(a\tau ,b\tau )$ plane inside the hatched wedge-shaped area in Figure 3 corresponds to parameter values $(a,b,\tau )$ such that this equation has a stationary solution and a stationary density.

Figure 3. A graphical representation of the stability region for the linear equation with single delay as determined by [37]. Modified from [38].

Equation (52) for the variance $K\left(t\right)$ of the stationary solution Z reduces to

$$K^{\prime \prime }\left(t\right)=(a^2-b^2)K\left(t\right),$$

with initial and boundary conditions

$$K^{\prime }\left(0\right)=-{\displaystyle \frac{{\sigma }^2}{2}},\hspace{1em}{K}^{\prime }\left(0\right)=aK\left(0\right)+bK\left(\tau \right).$$

It follows from [36] that if $a^2\ne b^2$ then the variance $K\left(0\right)=V_{*}$ is given by

$$V_{*}={\displaystyle \frac{{\sigma }^2}{2}}{\displaystyle \frac{bsinh\left(\sqrt{a^2-b^2}\tau \right)-\sqrt{a^2-b^2}}{\sqrt{a^2-b^2}(bcosh\left(\sqrt{a^2-b^2}\tau \right)+a)}}.$$

Note that if $\tau =0$, $b=0$ and $a=-\gamma$ we recover the variance ${\sigma }_{*}^2$ of the stationary Ornstein–Uhlenbeck process from Section 3.2. The conditional entropy as given in (45) reduces to

$$H_c\left(f(·,t)\right|f_{*})=-{\displaystyle \frac{1}{2}}{\displaystyle \frac{m^2\left(t\right)}{{\sigma }_{*}^2}},$$

(54)

where $m\left(t\right)=S_t{m}_0\left(0\right)$ is the solution of the equation $x^{\prime }\left(t\right)=ax\left(t\right)+bx(t-\tau )$ with the initial condition function $m_0\left(s\right)$, $s\in [-\tau ,0]$; see Figure 4.

Figure 4. The left-hand panel shows the temporal behaviour of the conditional entropy given by (54) for an initial condition $m_0\left(s\right)=s+1$ with $s\in [-1,0]$ and parameters $a=-1$, $b=-2$, $\tau =1$, and $\sigma =1$, while the right-hand panel shows the plot of the increasing conditional entropy given by (54) for $\tau =0$, $b=0$, and initial condition $m_0=1$.

4.3.2. A Second-Order Equation with One Delay

We now consider a particle of mass m, located at a position $x\left(t\right)$ and moving with velocity $v\left(t\right)$ and subject to a delayed restoring force $[-\beta x(t-\tau \left)\right]$ as well as a delayed frictional force $-\gamma v(t-\tau )$ and a fluctuating force $\sigma \xi \left(t\right)$, where $\beta >0$ and $\gamma >0$, while $\sigma \ge 0$ and $\xi \left(t\right)$ is the white noise. Thus, the dynamics of this noisy, delayed harmonic oscillator are given by

$$\left\{\begin{array}{c}{\displaystyle {\displaystyle \frac{d}{dt}}x\left(t\right)=v\left(t\right),}\hfill \\ m{\displaystyle \frac{d}{dt}}v\left(t\right)=-\gamma v(t-\tau )-\beta x(t-\tau )+\sigma \xi \left(t\right).\hfill \end{array}\right.$$

(55)

Note that the system (55) with $m=1$ in the absence of noise ($\sigma =0$) is equivalent to the second-order linear delayed equation

$${\displaystyle \frac{d^2}{d{t}^2}}x\left(t\right)=-\gamma {\displaystyle \frac{d}{dt}}x(t-\tau )-\beta x(t-\tau ).$$

(56)

This equation is a special case of the general second-order delay differential equation

$${\displaystyle \frac{d^2}{d{t}^2}}x\left(t\right)+a_1{\displaystyle \frac{d}{dt}}x\left(t\right)+a_{0}x\left(t\right)=b_{0}x(t-\tau )+b_1{\displaystyle \frac{d}{dt}}x(t-\tau ),$$

(57)

where $a_1,a_0,b_1,b_0$ are real constants. The stability of the trivial solution $x\left(t\right)\equiv 0$ of the deterministic Equation (57) is determined by the roots of the characteristic (eigenvalue) equation

$${\lambda }^2+a_1\lambda +a_0-b_1\lambda e^{-\lambda \tau }-b_0{e}^{-\lambda \tau }=0,$$

(58)

obtained with the ansatz $x\left(t\right)\sim e^{\lambda t}$. One can infer conditions for asymptotic stability from the literature [39,40,41,42]. In particular, if $b_0\ge a_0$ then the trivial solution is unstable. In what follows we thus assume that $a_0>b_0$ and that all roots of (58) have negative real parts, i.e.,

$$a_0>b_0\hspace{1em}\mathrm{and}\hspace{1em}{\alpha }_0:=sup\{\mathrm{Re}\left(\lambda \right):\lambda \mathrm{satisfies} \left(58\right)\}<0.$$

Note that if we take $\tau =0$ in (58), then we have ${\alpha }_0<0$ iff $a_0>b_0$ and $a_1>b_1$. See Section 4.4.2 for necessary and sufficient conditions for ${\alpha }_0<0$ when $\tau >0$. The stability region of parameters for Equation (56) is presented in Figure 5.

Figure 5. A graphical representation of the stability region of the steady state $x\equiv 0$ of (56). The entirety of the $(\gamma \tau ,\beta {\tau }^2)$ plane in the hatched banana-shaped area corresponds to parameter values $(\beta ,\gamma ,\tau )$ such that $x=0$ is stable. Modified from Kolmanovskii and Myshkis [34] (Chapter 4).

Now consider Equation (57) with noise

$${\displaystyle \frac{d^2}{d{t}^2}}x\left(t\right)+a_1{\displaystyle \frac{d}{dt}}x\left(t\right)+a_{0}x\left(t\right)=b_{0}x(t-\tau )+b_1{\displaystyle \frac{d}{dt}}x(t-\tau )+\sigma \xi \left(t\right),$$

and see [29,31,32] for particular examples. Introducing the variables

$$\mathbf{x}\left(t\right)=\left(\begin{array}{c}x\left(t\right)\\ v\left(t\right)\end{array}\right)\hspace{1em}\mathrm{with}\hspace{1em}v\left(t\right)=x^{\prime }\left(t\right),$$

then we obtain

$$d\mathbf{x}\left(t\right)=\left[A\mathbf{x}\right(t)+B\mathbf{x}(t-\tau \left)\right]dt+\Sigma dw\left(t\right),$$

(59)

where $w\left(t\right)$ is a standard one-dimensional Wiener process and

$$A=\left(\begin{array}{cc}0& 1\\ -a_0& -a_1\end{array}\right),\hspace{1em}B=\left(\begin{array}{cc}0& 0\\ b_0& b_1\end{array}\right),\hspace{1em}\Sigma =\left(\begin{array}{c}0\\ \sigma \end{array}\right).$$

We can define a stationary solution $\left\{Z\right(t):t\in \mathbb{R}\}$ for (59) as in the previous section. Let $K\left(t\right)={\left(k_{ij}\left(t\right)\right)}_{i,j=1,2}$ be its covariance matrix $t\ge 0$.

First, we show that

$$k_{12}\left(t\right)+k_{21}\left(t\right)=0\hspace{1em}\mathrm{for} \mathrm{all} t\ge 0.$$

(60)

Recall that

$$K\left(t\right)={\int }_0^{\infty }X\left(r\right)\Sigma {\Sigma }^{T}X{(r+t)}^{T}dr.$$

(61)

Since the fundamental solution $X\left(t\right)$ is a $2\times 2$ matrix, which we represent as $X\left(t\right)={\left(x_{ij}\left(t\right)\right)}_{i,j=1,2}$, and furthermore, it satisfies

$$X^{\prime }\left(t\right)=AX\left(t\right)+BX(t-\tau ),\hspace{1em}t>0,X\left(0\right)=I, X\left(s\right)=0\hspace{1em}\mathrm{for} s<0,$$

we see that

$$x_{12}^{\prime }\left(t\right)=x_{22}\left(t\right)\hspace{1em}\mathrm{for} t>0,\hspace{1em}{x}_{12}\left(s\right)=0\hspace{1em}\mathrm{for} s\le 0.$$

Thus, we obtain

$$X\left(r\right)\Sigma =\sigma \left(\begin{array}{c}{x}_{12}\left(r\right)\\ x_{12}^{\prime }\left(r\right)\end{array}\right)$$

and

$$X\left(r\right)\Sigma {\Sigma }^{T}X{(r+t)}^T={\sigma }^2\left(\begin{array}{cc}{x}_{12}\left(r\right)x_{12}(r+t)& x_{12}\left(r\right)x_{12}^{\prime }(r+t)\\ x_{12}^{\prime }\left(r\right)x_{12}(r+t)& x_{12}^{\prime }\left(r\right)x_{12}^{\prime }(r+t)\end{array}\right).$$

We have

$$k_{12}\left(t\right)={\int }_0^{\infty }{\sigma }^2{x}_{12}\left(r\right)x_{12}^{\prime }(r+t)dr.$$

Since $x_{12}\left(r\right)\to 0$ as $r\to \infty$ and $x_{12}\left(0\right)=0$, integrating by parts leads to

$$k_{12}\left(t\right)=-{\int }_0^{\infty }{x}_{12}^{\prime }\left(r\right)x_{12}(r+t)dr=-k_{21}\left(t\right).$$

Next it follows from (60) that the matrix $R_{*}=K\left(0\right)$ is diagonal, since $K{\left(0\right)}^T=K\left(0\right)$. Thus the stationary density $f_{*}$ as given by (40) will be of the form

$$f_{*}(x,v)={\displaystyle \frac{1}{2\pi \sqrt{{\sigma }_x^2{\sigma }_v^2}}}{e}^{-\left({\displaystyle \frac{x^2}{2{\sigma }_x^2}}+{\displaystyle \frac{v^2}{2{\sigma }_v^2}}\right)},\hspace{1em}x,v\in \mathbb{R},$$

where the variances

$${\sigma }_x^2=k_{11}\left(0\right)\hspace{1em}\mathrm{and}\hspace{1em}{\sigma }_v^2=k_{22}\left(0\right)$$

(62)

and $k_{11}\left(0\right)$, $k_{22}\left(0\right)$ are the diagonal elements of the matrix $K\left(0\right)$ as given by (85).

The Gibbs and conditional entropies behave as described in Section 4.2. Although the conditional entropy always converges to zero, it might not be an increasing function of time, as the following example shows.

We consider the initial condition ${\mathbf{x}}_0$ to be ${\mathbf{x}}_0\left(s\right)=Z\left(s\right)+m_0\left(s\right)$, $s\in [-\tau ,0]$, where Z is the stationary Gaussian process and $m_0$ is a non-zero mean vector function with $m_0\left(s\right)={(m_{0x}\left(s\right),m_{0x}^{\prime }\left(s\right))}^T$, $s\in [-\tau ,0]$. Then the conditional entropy is by (45), equal to

$$H_c\left(f(·,t)\right|f_{*})=-{\displaystyle \frac{1}{2}}\left({\displaystyle \frac{m_x^2\left(t\right)}{{\sigma }_x^2}}+{\displaystyle \frac{m_v^2\left(t\right)}{{\sigma }_v^2}}\right),$$

(63)

where $m_x\left(t\right)$ is the solution of (57) with initial condition $m_{0x}$, while $m_v\left(t\right)=m_x^{\prime }\left(t\right)$. Particular examples are given in Figure 6 and Figure 7, where we have $a_0=0$ and $a_1=0$ together with $b_0=-\beta$ and $b_1=-\gamma$ with the values of $(\beta ,\gamma ,\tau )$ in the hatched area of Figure 5.

Figure 6. The left-hand panel shows a plot of the conditional entropy given by (63) for initial condition $m_{0x}\left(s\right)=s+1$ for $s\in [-1,0]$ and parameters $a_0=a_1=0$, $b_0=-0.25$, $b_1=-1$, $\tau =1$, $\sigma =1$, while the right-hand panel shows the plot of the conditional entropy, given by (63), with $\tau =0$ for initial condition $m_x\left(0\right)=1$, $m_v\left(0\right)=1$.

Figure 7. This is a plot of the conditional entropy as given by (63) for initial condition $m_{0x}\left(s\right)=s^2+1$ for $s\in [-1,0]$ and parameters $a_0=a_1=0$, $b_0=-0.25$, $b_1=-1$, $\tau =1$, $\sigma =1$.

4.4. Determination of the Variances in (62)

This section is devoted to a derivation of the variances $k_{11}\left(0\right)$ and $k_{22}\left(0\right)$ as functions of parameters $a_1,a_0,b_1,b_0$ and the delay $\tau$ for the stationary solution of (59). Let $K\left(t\right)={\left(k_{ij}\left(t\right)\right)}_{i,j=1,2}$ be its covariance matrix function (61). To find the equation for $k_{ij}\left(t\right)$ for $t\in (0,\tau )$ we infer from (51) and (60) that

$$\begin{array}{c}\left(\begin{array}{cc}{k}_{11}^{\prime }\left(t\right)& -k_{21}^{\prime }\left(t\right)\\ k_{21}^{\prime }\left(t\right)& k_{22}^{\prime }\left(t\right)\end{array}\right)=\left(\begin{array}{cc}{k}_{11}\left(t\right)& -k_{21}\left(t\right)\\ k_{21}\left(t\right)& k_{22}\left(t\right)\end{array}\right)\left(\begin{array}{cc}0& -a_0\\ 1& -a_1\end{array}\right)\hfill \\ \hfill +\left(\begin{array}{cc}{k}_{11}(\tau -t)& k_{21}(\tau -t)\\ -k_{21}(\tau -t)& k_{22}(\tau -t)\end{array}\right)\left(\begin{array}{cc}0& b_0\\ 0& b_1\end{array}\right),\end{array}$$

implying that

$$k_{11}^{\prime }\left(t\right)=-k_{21}\left(t\right),\hspace{1em}{k}_{21}^{\prime }\left(t\right)=k_{22}\left(t\right)$$

(64)

and

$$k_{22}^{\prime }\left(t\right)=-a_0{k}_{21}\left(t\right)-a_1{k}_{22}\left(t\right)-b_0{k}_{21}(\tau -t)+b_1{k}_{22}(\tau -t).$$

(65)

Using (52) we arrive at

$$k_{22}^{\prime \prime }\left(t\right)=-2a_0{k}_{21}^{\prime }\left(t\right)+(a_0^2-b_0^2)k_{11}\left(t\right)+(a_1^2-b_1^2)k_{22}\left(t\right)$$

and applying (64) we obtain the following system of second-order differential equations:

$$\left\{\begin{array}{c}{k}_{11}^{\prime \prime }\left(t\right)=-k_{22}\left(t\right),\hfill \\ k_{22}^{\prime \prime }\left(t\right)=(a_0^2-b_0^2)k_{11}\left(t\right)+(a_1^2-b_1^2-2a_0)k_{22}\left(t\right).\hfill \end{array}\right.$$

(66)

The initial boundary conditions for this equation, based on (53), turn into

$$k_{11}^{\prime }\left(0\right)=0,\hspace{1em}{k}_{22}^{\prime }\left(0\right)=-{\displaystyle \frac{{\sigma }^2}{2}},$$

and

$$\left\{\begin{array}{c}{k}_{21}^{\prime }\left(0\right)=k_{22}\left(0\right),\hfill \\ -k_{21}^{\prime }\left(0\right)=-a_0{k}_{11}\left(0\right)+b_0{k}_{11}\left(\tau \right)+b_1{k}_{21}\left(\tau \right),\hfill \\ k_{22}^{\prime }\left(0\right)=-a_1{k}_{22}\left(0\right)-b_0{k}_{21}\left(\tau \right)+b_1{k}_{22}\left(\tau \right).\hfill \end{array}\right.$$

Combining them together with $k_{21}\left(\tau \right)=-k_{11}^{\prime }\left(\tau \right)$ by (64), we conclude that Equation (66) is supplemented with

$$\left\{\begin{array}{c}{a}_0{k}_{11}\left(0\right)-k_{22}\left(0\right)-b_0{k}_{11}\left(\tau \right)+b_1{k}_{11}^{\prime }\left(\tau \right)=0,\hfill \\ a_1{k}_{22}\left(0\right)-b_0{k}_{11}^{\prime }\left(\tau \right)-b_1{k}_{22}\left(\tau \right)={\displaystyle \frac{{\sigma }^2}{2}},\hfill \\ k_{11}^{\prime }\left(0\right)=0,\hspace{1em}{k}_{22}^{\prime }\left(0\right)=-{\displaystyle \frac{{\sigma }^2}{2}}.\hfill \end{array}\right.$$

(67)

Consequently, we may rewrite (66) as a second-order system of equations in the vector form

$$\left(\begin{array}{c}{k}_{11}^{\prime \prime }\left(t\right)\\ k_{22}^{\prime \prime }\left(t\right)\end{array}\right)=G\left(\begin{array}{c}{k}_{11}\left(t\right)\\ k_{22}\left(t\right)\end{array}\right)$$

with

$$G=\left(\begin{array}{cc}0& -1\\ a_0^2-b_0^2& a_1^2-b_1^2-2a_0\end{array}\right).$$

(68)

The general solution of this equation is

$$\left(\begin{array}{c}{k}_{11}\left(t\right)\\ k_{22}\left(t\right)\end{array}\right)=C\left(t\right)\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)+S\left(t\right)\left(\begin{array}{c}{k}_{11}^{\prime }\left(0\right)\\ k_{22}^{\prime }\left(0\right)\end{array}\right),$$

(69)

where

$$C\left(t\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{t^{2k}}{\left(2k\right)!}}{G}^k\hspace{1em}\mathrm{and}\hspace{1em}S\left(t\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{t^{2k+1}}{(2k+1)!}}{G}^k.$$

(70)

Since $C^{\prime }\left(t\right)=GS\left(t\right)$ and $S^{\prime }\left(t\right)=C\left(t\right)$, we also have

$$\left(\begin{array}{c}{k}_{11}^{\prime }\left(t\right)\\ k_{22}^{\prime }\left(t\right)\end{array}\right)=GS\left(t\right)\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)+C\left(t\right)\left(\begin{array}{c}{k}_{11}^{\prime }\left(0\right)\\ k_{22}^{\prime }\left(0\right)\end{array}\right).$$

(71)

We can rewrite (67) as the following equation:

$$G_1\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)+G_2\left(\begin{array}{c}{k}_{11}\left(\tau \right)\\ k_{22}\left(\tau \right)\end{array}\right)+G_3\left(\begin{array}{c}{k}_{11}^{\prime }\left(\tau \right)\\ k_{22}^{\prime }\left(\tau \right)\end{array}\right)=\left(\begin{array}{c}0\\ {\displaystyle \frac{{\sigma }^2}{2}}\end{array}\right),$$

(72)

where

$$G_1=\left(\begin{array}{cc}{a}_0& -1\\ 0& a_1\end{array}\right),\hspace{1em}{G}_2=\left(\begin{array}{cc}-b_0& 0\\ 0& b_1\end{array}\right),\hspace{1em}{G}_3=\left(\begin{array}{cc}{b}_1& 0\\ -b_0& 0\end{array}\right).$$

(73)

Using (69) and (71) for $t=\tau$, we rewrite condition (72) as

$$(G_1+G_{2}C\left(\tau \right)+G_{3}GS\left(\tau \right))\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)+(G_{2}S\left(\tau \right)+G_{3}C\left(\tau \right))\left(\begin{array}{c}0\\ -{\displaystyle \frac{{\sigma }^2}{2}}\end{array}\right)=\left(\begin{array}{c}0\\ {\displaystyle \frac{{\sigma }^2}{2}}\end{array}\right).$$

Consequently, we obtain

$$\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)={(G_1+G_{2}C\left(\tau \right)+G_{3}GS\left(\tau \right))}^{-1}(I+G_{2}S\left(\tau \right)+G_{3}C\left(\tau \right))\left(\begin{array}{c}0\\ {\displaystyle \frac{{\sigma }^2}{2}}\end{array}\right)$$

(74)

and

$$\left(\begin{array}{c}{k}_{11}\left(t\right)\\ k_{22}\left(t\right)\end{array}\right)=C\left(t\right)\left(\begin{array}{c}{k}_{11}\left(0\right)\\ k_{22}\left(0\right)\end{array}\right)+S\left(t\right)\left(\begin{array}{c}0\\ -{\displaystyle \frac{{\sigma }^2}{2}}\end{array}\right),\hspace{1em}t\in (0,\tau ).$$

Remark 1.

Our approach implies that the function $k_{22}$ satisfies, by (66), the following fourth-order differential equation

$$k_{22}^{\left(iv\right)}\left(t\right)-(a_1^2-b_1^2-2a_0)k_{22}^{\prime \prime }\left(t\right)+(a_0^2-b_0^2)k_{22}=0.$$

(75)

If we go back to (64) and (65), and if we take $m=1$, $a_1=a$, $a_0=k$, $b_1=-b$, $b_0=0$, $k_{21}\left(t\right)=g\left(t\right)$, and $k_{22}\left(t\right)=C\left(t\right)$, then we get the same equation as Equation (10) in [29], where it is postulated that a general solution of Equation (10) is of the form

$$C\left(t\right)=A_{c}cos\left(\omega (t-\tau /2)\right)+A_{s}sin\left(\omega (t-\tau /2)\right),$$

with constants $A_c$, $A_s$, ω derived separately. However, observe that no matter what the constants are, we obtain

$$C^{\prime \prime }\left(t\right)=-{\omega }^{2}C\left(t\right),$$

but $C\left(t\right)$ should be a general solution of (75). A similar approach was used in [31] (Appendix B) with Equation (B2), where $b_1=0$, $b_0=g/Q_0$, $a_1=1/Q_0$, $a_0=1$, and $k_{22}\left(t\right)={\varphi }_{vv}\left(t\right)$.

4.4.1. General Form of the Matrices $C\left(t\right)$ and $S\left(t\right)$ in (70)

To calculate the matrices $C\left(t\right)$ and $S\left(t\right)$, we find the eigenvalues of the matrix G. The characteristic equation $\det (zI-G)=0$ for the matrix G in (68) is given by

$$z^2-\mathrm{tr}\left(G\right)z+\det \left(G\right)=0,$$

where

$$\mathrm{tr}\left(G\right)=a_1^2-b_1^2-2a_0\hspace{1em}\mathrm{and}\hspace{1em}\det \left(G\right)=a_0^2-b_0^2.$$

(76)

Thus, we arrive at

$$z^2-(a_1^2-b_1^2-2a_0)z+(a_0^2-b_0^2)=0,$$

(77)

whose roots are

$$z_{\pm }={\displaystyle \frac{1}{2}}(a_1^2-b_1^2-2a_0\pm \sqrt{\Delta }),$$

(78)

where

$$\Delta ={(a_1^2-b_1^2-2a_0)}^2-4(a_0^2-b_0^2).$$

Suppose that $\Delta >0$, so that there are two different real eigenvalues. If $\det G<0$ then $z_{+}{z}_{-}<0$ while if $\det G>0$ then $z_{\pm }<0$ or $z_{\pm }>0$. The case of $\det G=0$ implies that one of the roots is equal to 0. If $\Delta <0$ then $z_{\pm }=\alpha e^{\pm i\beta }$, where $\alpha \ge 0$ and $\beta \in \mathbb{R}$, and we write $\sqrt{z_{\pm }}=\sqrt{\alpha }{e}^{\pm i\beta /2}$. Finally, if $\Delta =0$ and $\det G\ne 0$ then $z_{+}=z_{-}\ne 0$, while if $\det G=0$ then $z_{+}=z_{-}=0$.

We have

$$\begin{array}{c}\hfill C\left(t\right)=c_0\left(t\right)I+c_1\left(t\right)G,\hspace{1em}S\left(t\right)=s_0\left(t\right)I+s_1\left(t\right)G,\end{array}$$

(79)

for some functions $c_i$, $s_i$, $i=0,1$ that we now specify.

1.

If $\Delta \ne 0$ and $\det \left(G\right)\ne 0$ then

$$\begin{array}{c}\hfill c_0\left(t\right)=z_{+}{c}_{-}\left(t\right)-z_{-}{c}_{+}\left(t\right),\hspace{1em}{c}_1\left(t\right)=c_{+}\left(t\right)-c_{-}\left(t\right)\\ \hfill s_0\left(t\right)=z_{+}{s}_{-}\left(t\right)-z_{-}{s}_{+}\left(t\right),\hspace{1em}{s}_1\left(t\right)=s_{+}\left(t\right)-s_{-}\left(t\right)\end{array}$$

where

$$c_{\pm }\left(t\right)={\displaystyle \frac{cosh\left(t\sqrt{z_{\pm }}\right)}{z_{+}-z_{-}}},\hspace{1em}{s}_{\pm }\left(t\right)={\displaystyle \frac{sinh\left(t\sqrt{z_{\pm }}\right)}{(z_{+}-z_{-})\sqrt{z_{\pm }}}}.$$

2.

If $\Delta \ne 0$ and $\det \left(G\right)=0$ so that one of the roots is equal to zero, then $c_0$ and $c_1$ stay the same and

$$s_0\left(t\right)=0,\hspace{1em}{s}_1\left(t\right)={\displaystyle \frac{sinh\left(t\sqrt{z_{+}}\right)}{z_{+}\sqrt{z_{+}}}}.$$

3.

Now suppose that $\Delta =0$ and $\det \left(G\right)\ne 0$, so that $z_{+}=z_{-}\ne 0$. Then

$$c_0\left(t\right)=cosh\left(t\sqrt{z_{+}}\right)-{\displaystyle \frac{1}{2}}\sqrt{z_{+}}tsinh\left(t\sqrt{z_{+}}\right),\hspace{1em}{c}_1\left(t\right)={\displaystyle \frac{t}{2\sqrt{z_{+}}}}sinh\left(t\sqrt{z_{+}}\right)$$

and

$$\begin{array}{cc}\hfill s_0\left(t\right)& ={\displaystyle \frac{3}{2\sqrt{z_{+}}}}sinh\left(t\sqrt{z_{+}}\right)-{\displaystyle \frac{t}{2}}cosh\left(t\sqrt{z_{+}}\right),\hfill \\ \hfill s_1\left(t\right)& ={\displaystyle \frac{t}{2z_{+}}}cosh\left(t\sqrt{z_{+}}\right)-{\displaystyle \frac{1}{2z_{+}\sqrt{z_{+}}}}sinh\left(t\sqrt{z_{+}}\right).\hfill \end{array}$$

4.

Finally, when $\Delta =0$ and $\det \left(G\right)=0$, then $z_{+}=z_{-}=0$ and we have

$$c_0\left(t\right)=1,\hspace{1em}{c}_1\left(t\right)={\displaystyle \frac{1}{2}}{t}^2,\hspace{1em}{s}_0\left(t\right)=t,\hspace{1em}{s}_1\left(t\right)={\displaystyle \frac{1}{6}}{t}^3.$$

4.4.2. Stability Analysis of the Deterministic Delay Equation

To find the stability region for the second-order delay equation [39,40], substitute $\lambda =i\omega$ with $\omega >0$ into the characteristic Equation (58), separating it into real and imaginary parts:

$$a_0-{\omega }^2=b_{0}cos\left(\omega \tau \right)+b_1\omega sin\left(\omega \tau \right),\hspace{1em}{a}_1\omega =b_1\omega cos\left(\omega \tau \right)-b_{0}sin\left(\omega \tau \right),$$

and leading to a quartic equation for $\omega$:

$${\omega }^4+(a_1^2-b_1^2-2a_0){\omega }^2+(a_0^2-b_0^2)=0.$$

(80)

Now observe that if we substitute $z=-{\omega }^2$ into (77) then we obtain Equation (80). Consequently, solutions of (80) satisfy ${\omega }^2=-z_{\pm },$ where $z_{\pm }$ are as in (78). Thus, we can find a positive solution of (80) if and only if $z_{+}<0$ or $z_{-}<0$.

Here we classify the existence of these positive solutions in terms of $\det \left(G\right)$ and $\mathrm{tr}\left(G\right)$ as given in (76). Let m be the number of positive solutions of (80). Then

• $m=0$ iff $\det \left(G\right)>0$ and $\mathrm{tr}\left(G\right)>-2\sqrt{\det \left(G\right)}$ or $\det \left(G\right)=0$ and $\mathrm{tr}\left(G\right)\ge 0$;
• $m=1$ iff $\det \left(G\right)<0$ or $\det \left(G\right)=0$ and $\mathrm{tr}\left(G\right)<0$ or $\det \left(G\right)>0$ and $\mathrm{tr}\left(G\right)=-2\sqrt{\det \left(G\right)}$;
• $m=2$ iff $\det \left(G\right)>0$ and $\mathrm{tr}\left(G\right)<-2\sqrt{\det \left(G\right)}$.

Note that $\det \left(G\right)>0$ iff $|a_0|>|b_0|$.

Recall that we assume $a_0>b_0$. Then ${\alpha }_0<0$ if and only if one of the following holds (see [39,40,41,42]):

1.

$a_1>b_1$, $m=0$ and $\tau \ge 0$;

2.

$a_1>b_1$, $m=1$ and $\tau \in [0,{\tau }_0^{+})$ for some ${\tau }_0^{+}>0$;

3.

$a_1>b_1$, $m=2$ and

$$\tau \in [0,{\tau }_0^{+})\cup \bigcup _{j=1}^k({\tau }_{j-1}^{-},{\tau }_j^{+}),$$

for some k and a sequence of delays ${\tau }_j^{\pm }$;

4.

$a_1\le b_1$, $m=2$ and

$$\tau \in \bigcup _{j=0}^k({\tau }_j^{-},{\tau }_j^{+}).$$

4.4.3. Formulas for Variances

We need to find the inverse of the matrix $H:=G_1+G_{2}C\left(\tau \right)+G_{3}GS\left(\tau \right)$ and the second column of the matrix $I+G_{2}S\left(\tau \right)+G_{3}C\left(\tau \right)$. To this end, observe that

$$G_1+G_{2}C\left(\tau \right)+G_{3}GS\left(\tau \right)=G_1+c_0\left(\tau \right)G_2+c_1\left(\tau \right)G_{2}G+s_0\left(\tau \right)G_{3}G+s_1\left(\tau \right)G_3{G}^2,$$

where $c_i=c_i\left(\tau \right)$, $s_i=s_i\left(\tau \right)$, $i=0,1$ are from (79) and the matrices $G_1,G_2,G_3$ are as in (73). We have

$$G_{2}G=\left(\begin{array}{cc}-b_0& 0\\ 0& b_1\end{array}\right)\left(\begin{array}{cc}0& -1\\ \det \left(G\right)& \mathrm{tr}\left(G\right)\end{array}\right)=\left(\begin{array}{cc}0& b_0\\ b_1\det \left(G\right)& b_1\mathrm{tr}\left(G\right)\end{array}\right)$$

and

$$G_{3}G=\left(\begin{array}{cc}{b}_1& 0\\ -b_0& 0\end{array}\right)G=\left(\begin{array}{cc}0& -b_1\\ 0& b_0\end{array}\right),\hspace{1em}{G}_3{G}^2=\left(\begin{array}{cc}-b_1\det \left(G\right)& -b_1\mathrm{tr}\left(G\right)\\ b_0\det \left(G\right)& b_0\mathrm{tr}\left(G\right)\end{array}\right),$$

implying that the elements of the matrix $G_1+G_{2}C\left(\tau \right)+G_{3}GS\left(\tau \right)={\left(h_{ij}\left(t\right)\right)}_{i,j=1,2}$ are given by

$$\begin{array}{ccc}\hfill h_{11}& =a_0-b_0{c}_0-b_1\det \left(G\right)s_1,\hspace{1em}\hfill & \hfill h_{12}=-1+b_0{c}_1-b_1{q}_1,\\ \hfill h_{21}& =\det \left(G\right)[b_1{c}_1+b_0{s}_1],\hspace{1em}\hfill & \hfill h_{22}=a_1+b_1{q}_0+b_0{q}_1,\end{array}$$

(81)

where

$$q_0=c_0+\mathrm{tr}\left(G\right)c_1,\hspace{1em}{q}_1=s_0+\mathrm{tr}\left(G\right)s_1.$$

(82)

The determinant of H is

$$\begin{array}{c}\det \left(H\right)=(a_0-b_0{c}_0-b_1\det \left(G\right)s_1)(a_1+b_1{q}_0+b_0{q}_1)\hfill \\ \hfill -\det \left(G\right)(b_1{c}_1+b_0{s}_1)(-1+b_0{c}_1-b_1{q}_1)\end{array}$$

which reduces to

$$\begin{array}{c}\det \left(H\right)=a_0{a}_1+a_0(b_1{q}_0+b_0{q}_1)-a_1(b_0{c}_0+b_1\det \left(G\right)s_1)+\det \left(G\right)(b_0{s}_1+b_1{c}_1)\hfill \\ \hfill -(b_0{c}_0+b_1\det \left(G\right)s_1)(b_1{q}_0+b_0{q}_1)-\det \left(G\right)(b_1{c}_1+b_0{s}_1)(b_0{c}_1-b_1{q}_1).\end{array}$$

(83)

Next, we find the vector

$$\left(\begin{array}{c}{h}_1\\ h_2\end{array}\right)=(I+G_{2}S\left(\tau \right)+G_{3}C\left(\tau \right))\left(\begin{array}{c}0\\ 1\end{array}\right).$$

We have

$$\begin{array}{c}I+G_{2}S\left(\tau \right)+G_{3}C\left(\tau \right)=I+s_0{G}_2+s_1{G}_{2}G+c_0{G}_3+c_1{G}_{3}G\hfill \\ \hfill =\left(\begin{array}{cc}1-b_0{s}_0+b_1{c}_0& s_1{b}_0-b_1{c}_1\\ b_1\det \left(G\right)s_1-b_0{c}_0& 1+b_1{s}_0+b_1\mathrm{tr}\left(G\right)s_1+b_0{c}_0\end{array}\right)\end{array}$$

implying that

$$h_1=b_0{s}_1-b_1{c}_1,\hspace{1em}{h}_2=1+b_0{c}_0+b_1{q}_1.$$

(84)

Consequently, the diagonal matrix $K\left(0\right)$ has on its diagonal by (74) the following values

$$k_{11}\left(0\right)={\displaystyle \frac{{\sigma }^2}{2\det \left(H\right)}}[h_{22}{h}_1-h_{12}{h}_2],\hspace{1em}{k}_{22}\left(0\right)={\displaystyle \frac{{\sigma }^2}{2\det \left(H\right)}}[h_{11}{h}_2-h_{21}{h}_1],$$

(85)

where $\det \left(H\right)$ is as in (83), $h_{ij}$ as in (81) with $q_i$ in (82), and $h_i$ as in (84). We have

$$\begin{array}{c}{h}_{22}{h}_1-h_{12}{h}_2=(a_1+b_1{q}_0+b_0{q}_1)(b_0{s}_1-b_1{c}_1)-(-1+b_0{c}_1-b_1{q}_1)(1+b_0{c}_0+b_1{q}_1)\hfill \\ \hfill =1+a_1(b_0{s}_1-b_1{c}_1)+b_0(c_0-c_1)+b_0{b}_1(c_0{q}_1-q_0{s}_1)+b_1^2{q}_0{c}_1+b_0^2{q}_1{s}_1-b_1^1{q}_1^2\end{array}$$

and

$$\begin{array}{c}{h}_{11}{h}_2-h_{21}{h}_1=(a_0-b_0{c}_0-b_1\det \left(G\right)s_1)(1+b_0{c}_0+b_1{q}_1)\hfill \\ \hfill -\det \left(G\right)(b_1{c}_1+b_0{s}_1)(b_0{s}_1-b_1{c}_1).\end{array}$$

Note that if $\tau =0$, $b_0=0,b_1=0$, then we have $\det \left(H\right)=a_0{a}_1$ and

$$k_{11}\left(0\right)={\displaystyle \frac{{\sigma }^2}{2a_0{a}_1}},\hspace{1em}{k}_{22}\left(0\right)={\displaystyle \frac{{\sigma }^2}{2a_1}},$$

that are the variances of the stationary solution of the noisy harmonic oscillator from Section 3.2 with $\beta =a_0$ and $\gamma =a_1$.

4.5. Is There a Fokker–Planck-like Equation for Differential Delay Equations?

We start with a few preliminaries with the ultimate goal of deriving a Fokker–Planck-like equation for the evolution of a ‘density’ under the action of noisy, delayed dynamics. The process ${\left\{x_t\right\}}_{t\ge 0}$ as in (27) defines a transition semigroup ${\left\{T^t\right\}}_{t\ge 0}$ by

$$T^t\psi \left(\varphi \right)=\mathbb{E}\left(\psi \left(x_t\right)\right|x_0=\varphi ),\hspace{1em}t\ge 0,\varphi \in C,\psi \in B\left(C\right),$$

on the space $B\left(C\right)$ of bounded Borel measurable functions $\psi :C\to \mathbb{R}$. For each $t\ge 0$ we define the Borel probability measure ${\mu }_t$ on the space C as the distribution of $x_t$, so that

$$⟨\psi ,{\mu }_t⟩=⟨T^t\psi ,{\mu }_0⟩,\hspace{1em}t\ge 0,\psi \in B\left(C\right),$$

where we use the customary scalar product notation

$$⟨\psi ,\mu ⟩={\int }_C\psi \left(\varphi \right)\mu \left(d\varphi \right).$$

Then ${\left\{{\mu }_t\right\}}_{t\ge 0}$ is said to solve the equation

$${\displaystyle \frac{\partial }{\partial t}}{\mu }_t={\mathcal{L}}^{*}{\mu }_t,$$

(86)

where ${\mathcal{L}}^{*}$ is a formal adjoint of the so-called extended generator$\mathcal{L}$ of the Markov process. The linear operator $\mathcal{L}:\mathcal{D}\to B\left(C\right)$ with domain $\mathcal{D}\subset B\left(C\right)$ is such that

$${\int }_0^t⟨\left|\mathcal{L}\psi \right|,{\mu }_r⟩dr<\infty$$

and

$$⟨\psi ,{\mu }_t⟩=⟨\psi ,{\mu }_0⟩+{\int }_0^t⟨\mathcal{L}\psi ,{\mu }_r⟩dr$$

for all $t>0$ and $\psi \in \mathcal{D}$.

Now, instead of dealing with the abstract Equation (86) we proceed as in [3,4,43]. We introduce a differential operator $L:C_c^2\left({\mathbb{R}}^d\right)\to B\left(C\right)$ defined by

$$Lg\left(\varphi \right)=\sum _{i=1}^d{F}_i\left(\varphi \right){\displaystyle \frac{\partial g}{\partial x_i}}\left(\varphi \left(0\right)\right)+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{(\Sigma {\Sigma }^T)}_{ij}\left(\varphi \right){\displaystyle \frac{{\partial }^{2}g}{\partial x_i\partial x_j}}\left(\varphi \left(0\right)\right),\hspace{1em}\varphi \in C,$$

for $g\in C_c^2\left({\mathbb{R}}^d\right)$, where $C_c^2\left({\mathbb{R}}^d\right)$ is the space of functions with compact support that possess continuous derivatives up to second order. Here, ${\left[{(\Sigma {\Sigma }^T)}_{ij}\right]}_{i,j=1,\dots ,d}$ is the matrix $\Sigma {\Sigma }^T$, where ${\Sigma }^T$ is the transpose of the matrix $\Sigma$. Consider the distribution $\mu \left(t\right)$ of $x\left(t\right)=x_t\left(0\right)$, $t\ge 0$, defined on $\mathcal{B}\left({\mathbb{R}}^d\right)$, the $\sigma$-algebra of Borel subsets of ${\mathbb{R}}^d$, by

$$\mu \left(t\right)\left(B\right)={\mu }_t\{\varphi \in C:\varphi \left(0\right)\in B\},\hspace{1em}B\in \mathcal{B}\left({\mathbb{R}}^d\right).$$

Then ${\left\{{\mu }_t\right\}}_{t\ge 0}$ is said to solve the equation

$${\displaystyle \frac{\partial }{\partial t}}\mu \left(t\right)=L^{*}{\mu }_t$$

(87)

if

$${\int }_0^t⟨\left|Lg\right|,{\mu }_r⟩dr<\infty$$

and

$${\int }_{{\mathbb{R}}^d}g\left(x\right)\mu \left(t\right)\left(dx\right)={\int }_{{\mathbb{R}}^d}g\left(x\right)\mu \left(0\right)\left(dx\right)+{\int }_0^t⟨Lg,{\mu }_r⟩dr$$

(88)

for all $t>0$ and $g\in C_c^2\left({\mathbb{R}}^d\right)$. Note that we can represent the Borel measure $\mu \left(t\right)$ as $\mu \left(t\right)={\mu }_t\circ {\pi }_0^{-1}$, where the projection map ${\pi }_0:C\to {\mathbb{R}}^d$ is given by ${\pi }_0\left(\varphi \right)=\varphi \left(0\right)$, $\varphi \in C$. Thus, Equation (87) can be compared with (86) as it can be rewritten as

$${\displaystyle \frac{\partial }{\partial t}}{\mu }_t\circ {\pi }_0^{-1}=L^{*}{\mu }_t.$$

Suppose that the drift coefficient F and the diffusion coefficient $\Sigma$ in (26) are those with discrete delays

$$F\left(\varphi \right)=\mathcal{F}(\varphi \left({\tau }_0\right),\dots ,\varphi \left({\tau }_k\right)),\hspace{1em}\Sigma \left(\varphi \right)=\sigma (\varphi \left({\tau }_0\right),\dots ,\varphi \left({\tau }_k\right)),$$

where $k\in \mathbb{N}$, $\mathcal{F}$ and $\sigma$ are given transformations on ${\mathbb{R}}^{d(k+1)}$ and

$$0={\tau }_0<\dots <{\tau }_k=\tau .$$

We introduce the projection mapping ${\pi }_{{\tau }_0,\dots ,{\tau }_k}:C\to {\mathbb{R}}^{d(k+1)}$

$${\pi }_{{\tau }_0,\dots ,{\tau }_k}\left(\varphi \right)=(\varphi (-{\tau }_0),\dots ,\varphi (-{\tau }_k)),\hspace{1em}\varphi \in C,$$

and the measure

$$\nu \left(t\right)\left(B\right)={\mu }_t\circ {\pi }_{{\tau }_0,\dots ,{\tau }_k}^{-1}\left(B\right)={\mu }_t\{\varphi \in C:{\pi }_{{\tau }_0,\dots ,{\tau }_k}\left(\varphi \right)\in B\},\hspace{1em}t\ge 0,B\in \mathcal{B}\left({\mathbb{R}}^{d(k+1)}\right).$$

Observe that $\nu \left(t\right)$ is the distribution of $(x(t-{\tau }_0),\dots ,x(t-{\tau }_k))$. By using the change-of-variables formula we obtain

$$⟨Lg,{\mu }_t⟩={\int }_{{\mathbb{R}}^d}{\int }_{{\mathbb{R}}^{dk}}\left(\sum _{i=1}^d{\mathcal{F}}_i(x,y){\displaystyle \frac{\partial g}{\partial x_i}}\left(x\right)+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{a}_{ij}(x,y){\displaystyle \frac{{\partial }^{2}g}{\partial x_i\partial x_j}}\left(x\right)\right)\nu \left(t\right)(dx,y),$$

where $a_{ij}={\left(\sigma {\sigma }^T\right)}_{ij}$. Next, if $f_{\nu }$ is the density of the measure $\nu \left(t\right)$, assuming that it exists, with respect to the Lebesgue measure on ${\mathbb{R}}^{d(1+k)}$, i.e., $\nu \left(t\right)(dx,dy)=f_{\nu }(x,y,t)dxdy$, then f, the density of $x\left(t\right)$, exists and it is the marginal density obtained from $f_{\nu }$ by

$$f(x,t)={\int }_{{\mathbb{R}}^{dk}}{f}_{\nu }(x,y,t)dy.$$

Consequently, Equation (88) rewritten as

$$\begin{array}{cc}\hfill \hspace{1em}& {\int }_{{\mathbb{R}}^d}g\left(x\right)f(x,t)dx={\int }_{{\mathbb{R}}^d}g\left(x\right)f(x,0)dx\hfill \\ \hfill \hspace{1em}& +{\int }_0^t{\int }_{{\mathbb{R}}^d}{\int }_{{\mathbb{R}}^{dk}}\left(\sum _{i=1}^d{\mathcal{F}}_i(x,y){\displaystyle \frac{\partial g}{\partial x_i}}\left(x\right)+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{a}_{ij}(x,y){\displaystyle \frac{{\partial }^{2}g}{\partial x_i\partial x_j}}\left(x\right)\right)f_{\nu }(x,y,r)dxdydr,\hfill \end{array}$$

is the weak form of the Fokker–Planck-type evolution equation

$${\displaystyle \frac{\partial }{\partial t}}f(x,t)=-\sum _{i=1}^d{\displaystyle \frac{\partial }{\partial x_i}}{\int }_{{\mathbb{R}}^{dk}}{\mathcal{F}}_i(x,y)f_{\nu }(x,y,t)dy+{\displaystyle \frac{1}{2}}\sum _{i,j=1}^d{\displaystyle \frac{{\partial }^2}{\partial x_i\partial x_j}}{\int }_{{\mathbb{R}}^d}{a}_{ij}(x,y)f_{\nu }(x,y,t)dy.$$

This is the central and desired result of this rather long and technical section.

5. Conclusions

It is clear from the above considerations that the dynamic origin of the (weak or strong) second law of thermodynamics is far from obvious. If the dynamics are given by typical (time-reversal-invariant) ordinary differential equations the behaviour of the Gibbs entropy is unclear (8) while in the same circumstance the conditional entropy is constant (10). The presence of noise does not alter the indeterminate behaviour of the Gibbs entropy but does induce a monotonic approach of the conditional entropy to its equilibrium value of zero (9).

The inclusion of temporal delays in the dynamics does not clarify things, but does increase the computational complexity of the situation. The Gibbs entropy might be monotone increasing (46) (strong form of the second law), whereas the conditional entropy may display a non-monotone approach (44) to equilibrium (47) (weak form of the second law).

We have emphasized the somewhat primitive state of the development of the underlying theory of the evolution of densities in infinite dimensional (delayed) dynamical systems, and why correction of this must necessarily precede a satisfactory understanding of the effects of delayed dynamics on entropy behaviours. There is much to be done!

Why Bother?

One might wonder, given all of the technical problems involved in the treatment of density evolution in delayed systems, why is one bothering? The reason is straightforward. Biological processes are rife with delays modifying the dynamics, and models in many fields of the biological sciences contain significant delays at a variety of levels, e.g., ranging from gene regulation [44,45], through immunology [46], up to the cellular–clinical treatment level [47,48,49,50]. If we are ever to build a successful non-equilibrium thermodynamics for biological processes it will be essential to solve the problems pointed out in this study.