Information Geometry Control under the Laplace Assumption ^†

Abstract

By combining information science and differential geometry, information geometry provides a geometric method to measure the differences in the time evolution of the statistical states in a stochastic process. Specifically, the so-called information length (the time integral of the information rate) describes the total amount of statistical changes that a time-varying probability distribution takes through time. In this work, we outline how the application of information geometry may permit us to create energetically efficient and organised behaviour artificially. Specifically, we demonstrate how nonlinear stochastic systems can be analysed by utilising the Laplace assumption to speed up the numerical computation of the information rate of stochastic dynamics. Then, we explore a modern control engineering protocol to obtain the minimum statistical variability while analysing its effects on the closed-loop system’s stochastic thermodynamics.

1. Introduction

Stochastic systems are ubiquitous and include a large set of complex systems, such as the time evolution of molecular motors [1], the stock market [2], decision making [3], population dynamics [4] or engineering systems with parameter uncertainties [5]. The description of stochastic dynamics commonly involves the calculation of time-varying probability density functions (PDFs) governed by a Fokker–Planck equation and its corresponding stochastic differential equation [1]. Since a time-varying PDF describes all the possible trajectories the stochastic system can take in time, such a formalism has been advantageously applied in emergent fields as stochastic thermodynamics [6] or inference control [7].

As time-varying PDFs contain enormous dynamical information, defining a metric of the path’s length that a stochastic system takes through time can bring benefits when, for example, designing “efficient” systems (for instance, see [8]). In this regard, the field of information geometry has brought to light a true metric of the differences in the time evolution of the statistical states in a stochastic process [9,10]. Specifically, the concept of information length (IL) [11,12], given by the time integral of the stochastic dynamics information rate, describes the total amount of statistical changes that a time-varying probability distribution takes through time. The previous works showed that IL provides a link between stochastic processes, complexity and geometry [12]. Additionally, IL has been applied to the quantification of hysteresis in forward–backward processes [8,13], correlation and self-regulation among different players [13], phase transitions [14], and prediction of sudden events [15]. It is worth noting that in nonlinear stochastic systems, IL is generally difficult to obtain because the analytical/numerical solution of the Fokker–Planck equation and the execution of stochastic simulations are usually complicated and computationally costly. Hence, analytical simplifications are advantageous to ease IL’s calculation while increasing the possibility of applying IL to broader practical scenarios [16].

Even though the information rate may seem purely like a statistical quantity, its meaning can be understood in relation to the thermodynamics [11,12]. This result is of great advantage as we may use it to quantify the effects, for example, that a minimum statistical variability (constant information rate) control could have on the system’s energetic behaviour. Note that the idea of thermodynamic informed control systems is not something new. For instance, since the beginning of the 21st century, various works have proposed the consideration of entropy-informed control protocols to generate “intelligent/efficient” systems (for further details, see [17] and the references therein). Yet, describing the effects of an information geometry-informed control protocol over the system’s stochastic thermodynamics is to be carried out.

In this regard, we consider the application of the so-called Laplace assumption (Gaussian approximation of the system’s time-varying PDF) to the computation of IL and the information rate for a set of nonlinear stochastic differential equations. By using this assumption, we derive the values of the entropy rate, entropy production and entropy flow and their relation to the information rate. Then, we formulate an optimisation problem for the minimum information variability control and study the closed-loop stochastic dynamics and thermodynamics in a numerical example. Thus, creating a connection between information geometry, stochastic thermodynamics and control engineering (Figure 1).

To help readers, in the following, we summarise our notations. $\mathbb{R}$ is the set of real numbers; $\mathbf{x}\in {\mathbb{R}}^n$ represents a column vector $\mathbf{x}$ of real numbers of dimension n; $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ represents a real matrix of dimension $n\times n$ (bold-face letters are used to represent vectors and matrices); $\mathrm{Tr}\left(\mathbf{A}\right)$ corresponds to the trace of the matrix $\mathbf{A}$; $\left|\mathbf{A}\right|$, $\mathbf{vec}\left(\mathbf{A}\right)$, ${\mathbf{A}}^{\top }$ and ${\mathbf{A}}^{-1}$ are the determinant, vectorisation, transpose and inverse of matrix $\mathbf{A}$, respectively. The value ${\mathbf{I}}_n$ denotes the identity matrix of order n. Newton’s notation is used for the partial derivative with respect to the variable t (i.e. $\frac{\partial y}{\partial t}=\dot{y}$). Finally, the average of a random vector $\mathbf{\zeta }$ is denoted by $\mathbf{\mu }:=\langle \mathbf{x}\rangle$, the angular brackets representing the average.

2. Model

Throughout this work, the following set of nonlinear Langevin equations is considered

$$\dot{\mathbf{x}}=f(\mathbf{x},u)+\xi .$$

(1)

Here, $f:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is a function taking as input a vector $\mathbf{x}\in {\mathbb{R}}^n$ and a bounded smooth time-dependent deterministic function $u\left(t\right)\in \mathbb{R}$ to output a vector $f\left(\mathbf{x}\right)\in {\mathbb{R}}^n$ with elements $f_i$ ($i=1,2,...n$); $\xi \in {\mathbb{R}}^n$ is a Gaussian stochastic noise given by an n dimensional vector of $\delta$-correlated Gaussian noises ${\xi }_i$ ($i=1,2,...n$), with the following statistical property

$$\langle {\xi }_i\left(t\right)\rangle =0,\langle {\xi }_i\left(t\right){\xi }_j\left(t_1\right)\rangle =2D_{ij}\left(t\right)\delta (t-t_1),D_{ij}\left(t\right)=D_{ji}\left(t\right),\forall i,j=1,\cdots ,n.$$

(2)

The Fokker–Planck equation of (1) is

$$\dot{p}=-\nabla ·J=-\nabla ·fp+\nabla ·\mathbf{D}\nabla p=-\sum _{i=1}^n{\partial }_{x_i}\left(f_{i}p\right)+\sum _{i,j=1}^n\left({\partial }_{x_i}{D}_{ij}{\partial }_{x_j}\right)p,$$

(3)

where $J=[f_{i}p-{\mathbf{D}}_i\nabla p,\cdots ,f_{n}p-{\mathbf{D}}_n\nabla p]$.

2.1. The Laplace Assumption

The Laplace assumption allows us to describe the solution of (3) through a fixed multivariable Gaussian distribution given by [4]

$$p(\mathbf{x};t)=\frac{1}{\sqrt{|2\pi \mathbf{\Sigma }|}}{e}^{\frac{1}{2}Q(\mathbf{x};t)},$$

(4)

where $Q(\mathbf{x};t)=-\frac{1}{2}{\left(\mathbf{x}-\mu \left(t\right)\right)}^{\top }{\mathbf{\Sigma }}^{-1}\left(t\right)\left(\mathbf{x}-\mu \left(t\right)\right)$, and $\mu \left(t\right)\in {\mathbb{R}}^n$ and $\mathbf{\Sigma }\left(t\right)\in {\mathbb{R}}^{n\times n}$ are the mean and covariance value of the random variable $\mathbf{x}$. The value of the mean $\mu \left(t\right)$ and covariance matrix $\mathbf{\Sigma }\left(t\right)$ can be obtained from the following result.

Proposition 1

(The Laplace assumption). Under the Laplace assumption, the dynamics of the mean μ and covariance Σ at any time t of a nonlinear stochastic differential system (1) are governed by the following differential equations

$$\begin{array}{ccc}\hfill \dot{\mu }& =& {\left[f_1(\mu ,u)+\frac{1}{2}\mathrm{Tr}\left(\mathbf{\Sigma }{\mathbf{H}}_{f_1}\right),f_2(\mu ,u)+\frac{1}{2}\mathrm{Tr}\left(\mathbf{\Sigma }{\mathbf{H}}_{f_2}\right),\cdots ,f_n(\mu ,u)+\frac{1}{2}\mathrm{Tr}\left(\mathbf{\Sigma }{\mathbf{H}}_{f_n}\right)\right]}^{\top }\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \dot{\mathbf{\Sigma }}& =& {\mathbf{J}}_f\mathbf{\Sigma }+\mathbf{\Sigma }{\mathbf{J}}_f^{\top }+\mathbf{D}+{\mathbf{D}}^{\top }\hfill \end{array}$$

(6)

where ${\mathbf{H}}_{f_i}$ is the Hessian matrix of the function $f_i(\mathbf{x},u)$, and ${\mathbf{J}}_f$ is the Jacobian of the function $f(\mathbf{x},u)$.

Proof. 

See Appendix A. □

Note that when $f_i(\mathbf{x};t)$ in (1) is a linear function defined as

$$f_i(\mathbf{x};t):={\mathbf{A}}_i\mathbf{x}\left(t\right)+{\mathbf{B}}_i\mathbf{u}\left(t\right)=\sum _{j=1}^n{a}_{ij}{x}_j\left(t\right)+\sum _j^p{b}_{ij}{u}_j\left(t\right),$$

(7)

i.e., we consider a set of particles driven by a harmonic potential (${\mathbf{A}}_i\mathbf{x}\left(t\right)$) and a deterministic force (${\mathbf{B}}_i\mathbf{u}\left(t\right)$, the value ${\mathbf{H}}_{f_i}=0$, meaning that the mean value $\mu$ is not affected by the covariance matrix $\mathbf{\Sigma }$.

Limits of the Laplace Assumption

Since the Laplace assumption does not always hold, we first check on its limitation by considering the following cubic stochastic differential equation

$$\dot{x}\left(t\right)=-\gamma x{\left(t\right)}^3+u\left(t\right)+\xi \left(t\right),$$

(8)

where $\langle \xi \rangle =0$, $\langle \xi \left(t\right)\xi \left(t^{\prime }\right)\rangle =2D\delta (t-t^{\prime })$ and $\gamma \in {\mathbb{R}}^{+}$. Then, we denote $q(x,t)$ that is based on the Laplace Assumption, which take the following Gaussian form

$$q(x,t)=\frac{1}{\sqrt{2\pi \Sigma }}{e}^{\frac{1}{2}\frac{{(x-\mu )}^2}{\Sigma }},$$

(9)

where $\mu$ and $\Sigma$ are determined by the solution of

$$\begin{array}{ccc}\hfill \dot{\mu }& =& -\gamma {\mu }^3+u-3\gamma \mu \Sigma ,\hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill \dot{\Sigma }& =& -6\gamma \Sigma {\mu }^2+2D.\hfill \end{array}$$

(11)

To obtain the real system PDF $\tilde{p}(x,t)$ of system (8), we use stochastic simulations and kernel density estimators (for further details see [18]). Now, to highlight the limits of the Gaussian approximation $q(x,t)$, we apply the Kullback divergence (KL) $D_{KL}$ or relative entropy between the estimated $\tilde{p}$ and the Gaussian approximation q of the time-varying system (8) PDFs defined as

$$D_{KL}(\tilde{p}\left|\right|q)={\int }_{\mathbb{R}}p(x;t)log\left(\frac{p(x;t)}{q(x;t)}\right)\mathrm{d}x.$$

(12)

Figure 2 shows the KL divergence trough time between $\tilde{p}$ and q when changing the parameters $\gamma$ and D in Equation (8). The result shows that a valid LA requires a small damping (slow behaviour) and a wider noise amplitude in comparison with the initial value of $\Sigma$.

3. Stochastic Thermodynamics

Stochastic thermodynamics uses stochastic calculus to draw a connection between the “micro/mesoscopic stochastic dynamics” and the “macroscopic thermodynamics” [1,6]. In physical terms, this means that stochastic thermodynamics describes the interaction of a micro/mesoscopic system with one or multiple reservoirs (for instance, the dynamics of a Brownian particle suspended in a fluid in thermodynamic equilibrium described by a Langevin/Fokker Planck equation).

3.1. Entropy Rate

Given a time-varying multivariable PDF $p(\mathbf{x};t)$, we can calculate the entropy rate, a fundamental concept of stochastic thermodynamics, as follows [20]

$$\dot{S}\left(t\right)=\frac{d}{dt}S\left(t\right)=-{\int }_{{\mathbb{R}}^n}\dot{p}(\mathbf{x};t)\ln \left(p(\mathbf{x};t)\right)\mathrm{d}\mathbf{x}=\Pi -\Phi .$$

(13)

Proposition 2.

Under the Laplace assumption, the value of entropy rate $\dot{S}$, entropy production Π and entropy flow Φ is given by

$$\begin{array}{ccc}\hfill \dot{S}& =& \mathrm{Tr}\left({\mathbf{\Sigma }}^{-1}\mathbf{D}\right)+\mathrm{Tr}\left({\mathbf{J}}_f\right)=\frac{1}{2}\mathrm{Tr}\left({\mathbf{\Sigma }}^{-1}\dot{\mathbf{\Sigma }}\right),\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill \Pi & =& \mathrm{Tr}\left({\mathbf{\Sigma }}^{-1}\mathbf{D}\right)+\mathrm{Tr}\left(f{(\mu ,u)}^{\top }{\mathbf{D}}^{-1}f(\mu ,u)\right)+\mathrm{Tr}\left({\mathbf{J}}_f{\mathbf{D}}^{-1}{\mathbf{J}}_f^{\top }\mathbf{\Sigma }\right)+2\mathrm{Tr}\left({\mathbf{J}}_f\right),\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill \Phi & =& \mathrm{Tr}\left(f{(\mu ,u)}^{\top }{\mathbf{D}}^{-1}f(\mu ,u)\right)+\mathrm{Tr}\left({\mathbf{J}}_f{\mathbf{D}}^{-1}{\mathbf{J}}_f^{\top }\mathbf{\Sigma }\right)+\mathrm{Tr}\left({\mathbf{J}}_f\right).\hfill \end{array}$$

(16)

Proof. 

See Appendix B. □

3.2. Example

To illustrate the application of Proposition 2, consider the following Langevin form of the Duffing equation

$$\begin{array}{cc}\hfill \dot{x}\left(t\right)& =v\left(t\right)+{\xi }_1\left(t\right)\hfill \\ \hfill \dot{v}\left(t\right)& =-\delta v\left(t\right)-\alpha x\left(t\right)-\beta x{\left(t\right)}^3+\gamma cos\left(\omega t\right)+{\xi }_2\left(t\right)\hfill \end{array},$$

(17)

where $x\left(t\right)$ is the displacement at time t, $v\left(t\right)=\dot{x}\left(t\right)$ is the first derivative of x with respect to time, i.e., velocity, $\xi$ is a delta correlated noise, and the values $\delta ,\alpha ,\beta ,\gamma$ and $\omega$ are given constants.

Figure 3 shows a simulation of (17) using the deterministic equations of the mean vector $\mu ={[\langle x\rangle ,\langle v\rangle ]}^{\top }$ and covariance matrix $\mathbf{\Sigma }$ as described by Proposition 1. Specifically, Figure 3a includes the time evolution of the random variables x and v with its phase portrait, and the time evolution of ${\Sigma }_{11},{\Sigma }_{12}$ and ${\Sigma }_{22}$. Figure 3b shows the time evolution of the system’s stochastic thermodynamics including the entropy rate $\dot{S}$, the entropy production $\Pi$ and the entropy flow $\Phi$. In all subplots, time is scaled by the factor $T=2\pi /\omega$.

More importantly, Figure 3 shows that via Propositions 1 and 2 it is possible to describe the thermodynamics of any nonlinear stochastic system at every instant of time. Hence, as will be discussed in Section 5, Propositions 1 and 2 allow us to perceive the effects of a control protocol on the closed-loop system thermodynamics.

4. Information Length and Information Rate

For a time-varying multivariable PDF $p(\mathbf{x};t)$, we define its IL $\mathcal{L}$ as [15,16]

$$\mathcal{L}\left(t\right)={\int }_0^t\left(\sqrt{{\int }_{{\mathbb{R}}^n}p(\mathbf{x};\tau ){\left[{\partial }_{\tau }lnp(\mathbf{x};\tau )\right]}^2\mathrm{d}\mathbf{x}}\right)\mathrm{d}\tau ={\int }_0^t\Gamma \left(\tau \right)\mathrm{d}\tau ,$$

(18)

where $\Gamma$ is called the information rate. The value of ${\Gamma }^2$ can be understood as the Fisher information where the time is the control parameter [12]. Since $\Gamma$ gives the rate of change of $p(\mathbf{x};t)$, its time integral $\mathcal{L}$ quantifies the amount of statistical changes that the system goes through in time from the initial PDF $p(\mathbf{x};0)$ to a final PDF $p(\mathbf{x};t)$ [16].

Under the Laplace assumption, i.e., when $p(\mathbf{x};t)$ is a Gaussian PDF, the value of information rate $\Gamma$ of the joint PDF takes the compact form [11,16]

$${\Gamma }^2={\dot{\mu }}^{\top }{\mathbf{\Sigma }}^{-1}\dot{\mu }+\frac{1}{2}\mathrm{Tr}\left({\left({\mathbf{\Sigma }}^{-1}\dot{\mathbf{\Sigma }}\right)}^2\right),$$

(19)

where the time derivatives of $\dot{\mathbf{\mu }}$ and $\dot{\mathbf{\Sigma }}$ are given by Equations (5) and (6), respectively.

Relation with Stochastic Thermodynamics

Considering a fully decoupled nonlinear stochastic system and using the Laplace assumption, the value of the information rate ${\Gamma }^2$ is related to the entropy production $\Pi$ and the entropy flow $\dot{S}$ as follows

$${\Gamma }^2=\sum _{i=1}^n\frac{D_{ii}}{{\Sigma }_{ii}}{\Pi }_i+\sum _{i=1}^n{\dot{S}}_i^2+\frac{1}{2}\sum _{i=1}^n{H}_{f_i}({\dot{\mu }}_i+f_i({\mu }_i,u)),$$

(20)

where ${\Pi }_i$ and ${\dot{S}}_i$ are the entropy production and entropy rate from the marginal PDF $p(x_i,t)$ of $x_i$. $H_{f_1}=\frac{{\partial }^2}{\partial x_i^2}{f}_i({\mu }_i,t)$. If $f_i$ describes a harmonic potential (7), then $H_{f_i}=0$ and (20) lead to the expression

$${\Gamma }^2=\sum _i^n\frac{D_{ii}}{{\Sigma }_{ii}}{\Pi }_i+\sum _i^n{\dot{S}}_i^2.$$

(21)

Note that Equation (21) gives us a case where a minimum information length $\mathcal{L}$ would produce both a minimum entropy production/rate and a minimum statistical variability behaviour.

5. Minimum Variability Control

To impose a minimum statistical variability when going from an initial to a desired state (for instance, see [21]), we propose the optimisation problem with the following cost function

$$\mathbf{c}=arg\underset{\tilde{\mathbf{c}}}{min}\left(J={\int }_0^{t_f}\left({(\Gamma \left(t\right)-\Gamma \left(0\right))}^2+{(\mathbf{Y}\left(t\right)-{\mathbf{Y}}_d)}^{\top }\mathbf{Q}(\mathbf{Y}\left(t\right)-{\mathbf{Y}}_d)+\tilde{\mathbf{c}}{\left(t\right)}^{\top }\mathbf{R}\tilde{\mathbf{c}}\left(t\right)\right)\mathrm{d}t\right)$$

(22)

where $\mathbf{Y}\left(t\right):={[\mu \left(t\right),\mathbf{vec}(\mathbf{\Sigma }\left(t\right))]}^{\top },{\mathbf{Y}}_d:={[{\mu }_d,\mathbf{vec}\left({\mathbf{\Sigma }}_d\right)]}^{\top },\widetilde{\mathbf{c}\left(t\right)}:={[u\left(t\right),\mathbf{vec}\left(\mathbf{D}\left(t\right)\right)]}^{\top },\mathbf{Q}\in {\mathbb{R}}^{n+n\times n}$, and $\mathbf{R}\in {\mathbb{R}}^{1+n\times n}$. The solution $\mathbf{c}\left(t\right)$ corresponds to the control vector that allows us to obtain the minimum statistical variability. In Equation (22), the term ${(\Gamma \left(t\right)-\Gamma \left(0\right))}^2$ keeps the information variability constant. The term involving $\mathbf{Q}$ drives the system to reach a given PDF defined by ${\mathbf{Y}}_d$. The term containing $\mathbf{R}$ in the right-hand side of (22) regularises the control action $\mathbf{c}$ to avoid abrupt changes in the inputs. Note that the values of $\Sigma ,\mu$ and $\Gamma$ can be easily computed for any nonlinear stochastic process through Proposition (1) or the Laplace assumption. A control that comes after solving (22) would be called an information length quadratic regulator (IL-QR).

5.1. Model Predictive Control

A solution to the proposed optimisation problem (22) can be obtained by one of the most popular optimisation-based control techniques currently available—the so-called model-predictive-control (MPC) scheme [22]. Generally, MPC is an online optimisation algorithm for constrained control problems whose benefits have been recognized in applications to robotics [23], solar energy [24] or bioengineering [25]. Furthermore, MPC has the advantage of being easily implemented owing to packages such as CasADi [26] or the Hybrid Toolbox [27].

Figure 4 briefly details the working principle of the MPC’s optimiser in the form of a block diagram. The MPC method consists of utilising a prediction model to solve the optimisation problem in a finite horizon. Then, the optimal solution is applied to the system in real-time. Finally, the system’s output is fed back to the MPC algorithm to start the optimisation procedure again. In this work, we ease the prediction and simulation of the stochastic process by employing the Laplace assumption.

5.2. Example

We now present an example of the application of the MPC method to obtain the minimum variability behaviour of a stochastic system. Figure 5 shows the IL-QR applied to the cubic stochastic process given by Equation (8), where the control vector and the state vector are given by $\mathbf{c}={[u,D]}^{\top }$ and $\mathbf{Y}={[\mu ,\Sigma ]}^{\top }$, respectively. In the simulation, the initial state is $\mathbf{Y}\left(0\right)={[2+5/6,1/(2\times 30)]}^{\top }$, while the desired state is ${\mathbf{Y}}_d\left(t\right)={[2+1/30,1/(2\times 3)]}^{\top }$. Additionally, we consider the parameters $\gamma =0.1$, $T_s=1\times {10}^{-3}$, $N=5$, $I_L=1\times {10}^3$, $\mathbf{R}=1\times {10}^{-5}{\mathbf{I}}_2$, $Q_{12}=Q_{21}=0,Q_{12}=1\times {10}^2$ and $Q_{22}=8\times {10}^2$. Here, $T_s$ is the integration time step, and N is the number of future time steps considered in the prediction model. The value of $\Gamma \left(0\right)$ is imposed via the initial conditions and Equation (19).

In Figure 5a, we show the time evolution of the mean $\mu$, the inverse temperature $\beta =\frac{1}{2\Sigma }$, the input force u, the noise amplitude D, the information rate ${\Gamma }^2$ and the information length $\mathcal{L}$. We also show the PDF time evolution of the simulation model computed via the Laplace approximation (q) or via stochastic simulations ($\tilde{p}$) and the corresponding $KL$-divergence (12) between them. In the subplot of $\mu$ and $\beta$, the legend LA and SS stand for the Laplace assumption and stochastic simulations, respectively. Interestingly, we can see from this that the Laplace approximation works fine when used as a prediction model in the MPC method. The controls have a chattering effect (oscillations having a finite frequency and amplitude) similar to the one encountered when implementing other control methods, such as the sliding mode control [28], when trying to keep the system in the desired state ${\mathbf{Y}}_d$.

Figure 5b demonstrates the effects of controls (22) on the closed-loop system stochastic thermodynamics. The results show that at the desired state ${\mathbf{Y}}_d$ the value of $\dot{S}$ oscillates around zero with a small amplitude. This means $\Phi =-\Pi$ holds at some instants of time when $\mathbf{Y}$ reaches ${\mathbf{Y}}_d$. In other words, all the energy is exchanged with the system’s environment when the control keeps $\mathbf{Y}$ on the nonequilibrium state ${\mathbf{Y}}_d$.

6. Conclusions

In this work, we developed a new control MPC method to derive the evolution of a system with a minimum information variability in systems governed by nonlinear stochastic differential equations. Specifically, we identified the limitations of the Laplace assumption and utilised it to reduce the computational cost of calculating the time-varying PDFs and to develop a prediction model in the MPC algorithm. We also derived the relations that permit us to analyse the controller effects on the close-loop system’s thermodynamics.

In future work, we aim to apply our results for maximising the free-energy (minimum entropy production) [12] and for the analysis of the closed-loop stochastic thermodynamics in higher-order systems.