Lagrangian Function on the Finite State Space Statistical Bundle

Abstract

The statistical bundle is the set of couples ($Q,W$) of a probability density Q and a random variable W such that ${𝔼}_Q\left[W\right]=0$. On a finite state space, we assume Q to be a probability density with respect to the uniform probability and give an affine atlas of charts such that the resulting manifold is a model for Information Geometry. Velocity and acceleration of a one-dimensional statistical model are computed in this set up. The Euler–Lagrange equations are derived from the Lagrange action integral. An example Lagrangian using minus the entropy as potential energy is briefly discussed.

1. Introduction

The set-up of classical Lagrangian Mechanics is a finite-dimensional Riemannian manifold. For example, see the monographs by V.I. Arnold ([1], Chapters III–IV), R. Abraham and J.E. Mardsen ([2], Chapter 3), J.E. Marsden and T.S. Ratiu ([3], Chapter 7). Classical Information geometry, as it was first defined in the monograph by S.-I. Amari and H. Nagaoka [4], views parametric statistical models as a manifold endowed with a dually-flat connection. In a recent paper, M. Leok and J. Zhang [5] have pointed out the natural relation between these two topics and have given a wide overview of the mathematical structures involved.

In the present paper, we take up the same research program with two further qualifications. First, we assume a non-parametric approach by considering the full set of positive probability functions on a finite set, as it was done, for example, in our review paper [6]. The discussion is restricted here to a finite state space to avoid difficult technical problems. Second, we consider a specific expression of the tangent space of the statistical manifold, which is a Hilbert bundle that we call a statistical bundle. Our aim is to emphasize the basic statistical intuition of the geometric quantities involved. Because of that, we chose to systematically use the language of non-parametric differential geometry as it is developed in the monography of S. Lang [7].

Herein, we use our version of Information Geometry; see the review paper [6]. Preliminary versions of this paper have been presented at the SigmaPhy2017 Conference held in Corfu, Greece, 10–14 July 2017, and at a seminar held at Collegio Carlo Alberto, Moncalieri, on 5 September 2017. In these early versions, we did not refer to Leok and Zhang’s work, which we were unaware of at that time.

In Section 2, we review the definition and properties of the statistical bundle, and of the affine atlas that endows it with both a manifold structure and a natural family of transports between the fibers. In Section 3, we develop the formalism of the tangent space of the statistical bundle and derive the expression of the velocity and the acceleration of a one-dimensional statistical model in the given affine atlas. The derivation of the Euler–Lagrange equations, together with a relevant example, is discussed in Section 4.

2. Statistical Bundle

We consider a finite sample space $\mathsf{\Omega }$, with $\#\mathsf{\Omega }=N$. The probability simplex is $\Delta (\mathsf{\Omega })$, and ${\Delta }^{\circ }(\mathsf{\Omega })$ is its interior. The uniform probability on $\mathsf{\Omega }$ is denoted as $\mu$, $\mu (x)=\frac{1}{N}$, $x\in \mathsf{\Omega }$. The maximal exponential family $\epsilon \left(\mu \right)$ is the set of all strictly positive probability densities of $(\mathsf{\Omega },\mu )$. The expected value of $f:\mathsf{\Omega }\to \mathbb{R}$ with respect to the density $P\in \epsilon \left(\mu \right)$ is denoted ${𝔼}_P\left[f\right]={𝔼}_{\mu }\left[fP\right]=\frac{1}{N}{\sum }_{x\in \mathsf{\Omega }}f(x)P(x)$.

In [6,8,9], we made the case for the statistical bundle being the key structure of Information Geometry. The statistical bundle with base $\mathsf{\Omega }$ is

$$S\epsilon \left(\mu \right)=\left\{(Q,V)|Q\in \epsilon \left(\mu \right),{𝔼}_Q\left[V\right]=0\right\}.$$

The statistical bundle is a semi-algebraic subset of ${\mathbb{R}}^{2N}$; i.e., it is defined by algebraic equations and strict inequalities. It is trivially a real manifold. At each $Q\in \epsilon \left(\mu \right)$, the fiber $S_Q\epsilon \left(\mu \right)$ is endowed with the scalar product

$$(V_1,V_2)\mapsto {⟨V_1,V_2⟩}_Q={𝔼}_Q\left[V_1{V}_2\right]={Cov}_Q\left(V_1,V_2\right).$$

To this structure we add a special affine atlas of charts in order to show a structure of affine manifold, which is of interest in the statistical applications. The exponential atlas of the statistical manifold $S\epsilon \left(\mu \right)$ is the collection of charts given for each $P\in \epsilon \left(\mu \right)$ by

$$s_P:S\epsilon \left(\mu \right)\ni (Q,V)\mapsto (s_P(Q),{}^{\mathrm{e}}{\mathbb{U}}_Q^{P}V)\in S_P\epsilon \left(\mu \right)\times S_P\epsilon \left(\mu \right),$$

(1)

where (with a slight abuse of notation)

$$s_P(Q)=log\frac{Q}{P}-{𝔼}_P\left[log\frac{Q}{P}\right],{}^{\mathrm{e}}{\mathbb{U}}_Q^{P}V=V-{𝔼}_P\left[V\right].$$

(2)

As $s_P(P,V)=(0,V)$, we say that $s_P$ is the chart centered at P. If $s_P(Q)=U$, it is easy to derive the exponential form of Q as a density with respect to P; namely, $Q={\mathrm{e}}^{U-{𝔼}_P\left[log\frac{Q}{P}\right]}·P$. As ${𝔼}_{\mu }\left[Q\right]=1$, then $1={𝔼}_p\left[{\mathrm{e}}^{U-{𝔼}_P\left[log\frac{P}{Q}\right]}\right]={𝔼}_p\left[{\mathrm{e}}^U\right]{\mathrm{e}}^{-{𝔼}_P\left[log\frac{P}{Q}\right]}$, so that the cumulant function $K_P$ is defined on $S_P\epsilon \left(\mu \right)$ by

$$K_p(U)=log{𝔼}_P\left[{\mathrm{e}}^U\right]={𝔼}_P\left[log\frac{P}{Q}\right]=D\left(P\parallel Q\right);$$

that is, $K_P(V)$ is the expression in the chart at P of Kullback–Leibler divergence of $Q\mapsto D\left(P\parallel Q\right)$, and we can write

$$Q={\mathrm{e}}^{U-K_P(U)}\times P={\mathrm{e}}_P(U).$$

The patch centered at P is

$$s_P^{-1}=e_P:{(S_P\epsilon \left(\mu \right))}^2\ni (U,W)\mapsto ({\mathrm{e}}_P(U),{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}W)\in S\epsilon \left(\mu \right).$$

In statistical terms, the random variable $log\left(Q/P\right)$ is the relative point-wise information about Q relative to the reference P, while $s_P(Q)$ is the deviation from its mean value at P. The expression of the other divergence in the chart centered at P is

$$D\left(Q\parallel P\right)={𝔼}_Q\left[log\frac{Q}{P}\right]={𝔼}_Q\left[U-K_P(U)\right]={𝔼}_Q\left[U\right]-K_P(U).$$

The equation above shows that the two divergences are convex conjugate functions in the proper charts; see [10].

The transition maps of the exponential atlas in Equations (1) and (2) are

$$\begin{array}{c}\hfill s_{P_2}\circ e_{P_1}(U,W)=s_{P_2}\left({\mathrm{e}}_{P_1}(U),{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_{1}P(U)}W\right)=s_{P_2}\left({\mathrm{e}}^{U-K_{P_1}(U)}\times P_1,W-{𝔼}_{{\mathrm{e}}_{P_1}(U)}\left[W\right]\right)=\\ \hfill \left(U-K_{P_1}(U)+log\frac{P_1}{P_2}-{𝔼}_{P_2}\left[U-K_{P_1}(U)+log\frac{P_1}{P_2}\right],W-{𝔼}_{{\mathrm{e}}_{P_1}(U)}\left[W\right]-{𝔼}_{P_2}\left[W-{𝔼}_{{\mathrm{e}}_{P_1}(U)}\left[W\right]\right]\right)=\\ \hfill \left({}^{\mathrm{e}}{\mathbb{U}}_{P_1}^{P_2}U+s_{P_2}(P_1),{}^{\mathrm{e}}{\mathbb{U}}_{P_1}^{P_2}W\right),\end{array}$$

so that the exponential atlas is indeed affine. Notice that the linear part is ${}^{\mathrm{e}}{\mathbb{U}}_{P_1}^{P_2}$.

3. The Tangent Space of the Statistical Bundle

Let us compute the expression of the velocity at time t of a smooth curve $t\mapsto \gamma (t)=(Q(t),W(t))\in S\epsilon \left(\mu \right)$ in the chart centered at P. The expression of the curve is

$${\gamma }_P(t)=s_P(\gamma (t))=\left(s_P(Q(t)),{}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^{P}W(t)\right),$$

and hence we have, by denoting the derivative in ${\mathbb{R}}^N$ by the dot,

$$\frac{d}{dt}{s}_P(Q(t))=\frac{d}{dt}\left(log\frac{Q(t)}{P}-{𝔼}_P\left[log\frac{Q(t)}{P}\right]\right)=\frac{\dot{Q}(t)}{Q(t)}-{𝔼}_P\left[\frac{\dot{Q}(t)}{Q(t)}\right]={}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^P\frac{\dot{Q}(t)}{Q(t)},$$

(3)

and

$$\frac{d}{dt}{}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^{P}W(t)=\frac{d}{dt}\left(W(t)-{𝔼}_P\left[W(t)\right]\right)=\dot{W}(t)-{𝔼}_P\left[\dot{W}(t)\right]={}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^P\left(\dot{W}(t)-{𝔼}_{Q(t)}\left[\dot{W}(t)\right]\right).$$

(4)

If we define the velocity of $t\mapsto Q(t)={\mathrm{e}}^{U(t)-K_p(U(t))}\times P$ to be

$$\stackrel{\star }{Q}(t)=\frac{\dot{Q}(t)}{Q(t)}=\frac{d}{dt}logQ(t)=\dot{U}(t)-d{K}_P(U(t))[\dot{U}(t)]\in S_{Q(t)}\epsilon \left(\mu \right),$$

then $t\mapsto (Q(t),\stackrel{\star }{Q}(t))$ is a curve in the statistical bundle whose expression in the chart centered at P is $t\mapsto (U(t),\dot{U}(t))$. The velocity as defined above is nothing else as the score function of the one-dimensional statistical model; see e.g., the textbook by B. Efron and T. Haste (Section 4.2, [11]). The variance of the score function (i.e., the squared norm of $\stackrel{\star }{Q}(t)$ in $S_{Q(t)}\epsilon \left(\mu \right)$) is classically known as Fisher information at t.

We define the second statistical bundle to be

$$S^2\epsilon \left(\mu \right)=\left\{(Q,W,X,Y)|(Q,W)\in S\epsilon \left(\mu \right),X,Y\in S_Q\epsilon \left(\mu \right)\right\},$$

with charts

$$s_P(Q,V,X,Y)=\left(s_P(Q,V),{}^{\mathrm{e}}{\mathbb{U}}_Q^{P}X,{}^{\mathrm{e}}{\mathbb{U}}_Q^{P}Y\right),$$

we can identify the second bundle with the tangent space of the first bundle as follows.

For each curve $t\mapsto \gamma (t)=(Q(t),W(t))$ in the statistical bundle, define its velocity at t to be

$$\stackrel{\star }{\gamma }(t)=\left(Q(t),W(t),\stackrel{\star }{Q}(t),\dot{W}(t)-{𝔼}_{Q(t)}\left[\dot{W}(t)\right]\right),$$

because $t\mapsto \stackrel{\star }{\gamma }(t)$ is a curve in the second statistical bundle, and its expression in the chart at P has the last two components equal to the values given in Equations (3) and (4).

In particular, consider the a curve $t\mapsto \chi (t)=(Q(t),\stackrel{\star }{Q}(t))$. The velocity is

$$\stackrel{\star }{\chi }(t)=\left(Q(t),\stackrel{\star }{Q}(t),\stackrel{\star }{Q}(t),\stackrel{**}{Q}(t)\right),$$

where the acceleration $\stackrel{**}{Q}(t)$ is

$$\stackrel{**}{Q}(t)=\frac{d}{dt}\frac{\dot{Q}(t)}{Q(t)}-{𝔼}_{Q(t)}\left[\frac{d}{dt}\frac{\dot{Q}(t)}{Q(t)}\right]=\frac{\ddot{Q}(t)}{Q(t)}-\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)$$

(5)

It should be noted that the acceleration has been defined without explicitly mentioning the relevant connection. In fact, the connection here is implicitly defined by the transports ${}^{\mathrm{e}}{\mathbb{U}}_P^Q$, which is unusual in Differential Geometry, but is quite natural from the probabilistic point of view; see P. Gibilisco and G. Pistone [12]. We shall see below that the non-parametric approach to Information Geometry allows the definition of a dual transport, hence a dual connection as it was in [4]. Because of that, we could have defined other types of acceleration together with the one we have defined. Namely, we could consider an exponential acceleration ${}^e{D}^{2}Q(t)=\stackrel{**}{Q}(t)$, a mixture acceleration ${}^m{D}^{2}Q(t)=\ddot{Q}(t)/Q(t)$, and a Riemannian acceleration

$${}^0{D}^{2}Q(t)=\frac{1}{2}\left({}^e{D}^{2}Q(t)+{}^m{D}^{2}Q(t)\right)=\frac{\ddot{Q}(t)}{Q(t)}-\frac{1}{2}\left({\left(\frac{\dot{Q}(t)}{Q(t)}\right)}^2-{𝔼}_{Q(t)}\left[{\left(\frac{\dot{Q}(t)}{Q(t)}\right)}^2\right]\right),$$

(6)

each acceleration being associated with a specific connection; see the review paper [6]. We do not further discuss the different second-order geometries associated with the statistical bundle in this paper.

Example 1 (Boltzmann–Gibbs).

Let us compare the formalism we have introduced above with standard computations in Statistical Physics. The Boltzmann–Gibbs distribution gives to point $x\in \mathsf{\Omega }$ the probability ${\mathrm{e}}^{-(1/\theta )H(x)}/Z(\theta )$, with $Z(\theta )={\sum }_{x\in \mathsf{\Omega }}{\mathrm{e}}^{-(1/\theta )H(x)}$ and $\theta >0$, see Landau and Lifshitz ([13], Chapter 3). As a curve in $\epsilon \left(\mu \right)$, it is $Q(\theta )=N{\mathrm{e}}^{-(1/\theta )H}/Z(\theta )$ because of the reference to the uniform probability. The velocity defined above becomes in this case $\stackrel{\star }{Q}(\theta )={\theta }^{-2}(H-{𝔼}_{\theta }\left[H\right])$, while the acceleration of Equation (5) is $\stackrel{**}{Q}(\theta )=-{\theta }^{-3}(H-{𝔼}_{\theta }\left[H\right])$. Notice that we have the equation $\theta \stackrel{**}{Q}(\theta )+\stackrel{\star }{Q}(\theta )=0$.

Following the original construction of Amari’s Information Geometry [4], we have defined on the statistical bundle a manifold structure which is both an affine and a Riemannian manifold. The base manifold $\epsilon \left(\mu \right)$ is actually a Hessian manifold with respect to any of the convex functions $K_p(U)=log{𝔼}_p\left[{\mathrm{e}}^U\right]$, $U\in S_p\epsilon \left(\mu \right)$ (see [14]). Many computations are actually performed using the Hessian structure. The following equations are easily checked and frequently used:

$$\begin{array}{cc}\hfill {𝔼}_{{\mathrm{e}}_P(U)}\left[H\right]& =d{K}_P(U)[H];\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}H& =H-d{K}_P(U)[H];\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill d^2{K}_P(U)[H_1,H_2]& ={⟨{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_1,{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_2⟩}_{{\mathrm{e}}_P(U)};\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill d^3{K}_p(U)[H_1,H_2,H_3]& ={𝔼}_{{\mathrm{e}}_P(U)}\left[{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_1\times {}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_2\times {}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_3\right].\hfill \end{array}$$

(10)

We have defined a centering operation that can be thought of as a transport among fibers,

$${}^{\mathrm{e}}{\mathbb{U}}_P^Q:S_p\epsilon \left(\mu \right)\to S_q\epsilon \left(\mu \right),$$

whose adjoint is ${}^{\mathrm{m}}{\mathbb{U}}_q^{p}V=\frac{q}{p}V$. In fact, is the adjoint of ${}^{\mathrm{e}}{\mathbb{U}}_p^q$,

$${⟨{}^{\mathrm{e}}{\mathbb{U}}_P^{Q}U,V⟩}_Q={𝔼}_Q\left[(U-{𝔼}_Q\left[U\right])V\right]={𝔼}_Q\left[UV\right]={𝔼}_P\left[U\left(\frac{Q}{P}V\right)\right]={⟨U,{}^{\mathrm{m}}{\mathbb{U}}_Q^{P}V⟩}_P$$

Moreover, iff $U,V\in S_P\epsilon \left(\mu \right)$, then

$${⟨{}^{\mathrm{e}}{\mathbb{U}}_P^{Q}U,{}^{\mathrm{m}}{\mathbb{U}}_P^{Q}V⟩}_Q={⟨{}^{\mathrm{e}}{\mathbb{U}}_Q^P{}^{\mathrm{e}}{\mathbb{U}}_P^{Q}U,V⟩}_P={⟨U,V⟩}_P.$$

Example 2 (Entropy flow).

This example is taken from [8]. In the scalar field $\epsilon \left(Q\right)=-{𝔼}_Q\left[logQ\right]$, there is no dependence on the fiber. If $t\mapsto Q(t)={\mathrm{e}}^{V(t)-K_P(V(t))}·P$ is a smooth curve in $\epsilon \left(\mu \right)$ expressed in the chart centered at P, then we can write

$$\begin{array}{cc}\hfill \mathscr{H}\left(Q(t)\right)=-{𝔼}_{Q(t)}\left[V(t)-K_P(V(t))+logP\right]=& \\ & K_P(V(t))-{𝔼}_{Q(t)}\left[V(t)+logP+\mathscr{H}\left(P\right)\right]+\mathscr{H}\left(P\right)=\hfill \\ & K_P(V(t))-d{K}_P(V(t))[V(t)+logP+\mathscr{H}\left(P\right)]+\mathscr{H}\left(P\right),\hfill \end{array}$$

(11)

where the argument of the last expectation belongs to the fiber $S_P\epsilon \left(\mu \right)$ and we have expressed the expected value as a derivative by using Equation (7).

Again using Equations (7) and (9), we compute the derivative of the entropy along the given curve as

$$\begin{array}{cc}\hfill \frac{d}{dt}\mathscr{H}\left(Q(t)\right)& =\frac{d}{dt}{K}_P(V(t))-\frac{d}{dt}d{K}_P(V(t))[V(t)+logP+\mathscr{H}\left(P\right)]=\hfill \\ & d{K}_P(V(t))[\dot{V}(t)]-d^2{K}_P(V(t))[V(t)+logP+\mathscr{H}\left(P\right),\dot{V}(t)]-d{K}_P(V(t))[\dot{V}(t)]=\hfill \\ & -{𝔼}_{Q(t)}\left[{}^{\mathrm{e}}{\mathbb{U}}_P^{Q(t)}(V(t)+logP){}^{\mathrm{e}}{\mathbb{U}}_P^{Q(t)}\dot{V}(t)\right].\hfill \end{array}$$

We use now the equations

$$V(t)+logP=logQ(t)+K_P(V(t)),{}^{\mathrm{e}}{\mathbb{U}}_P^{Q(t)}\left(logQ(t)+K_P(V(t))\right)=logQ(t)+\mathscr{H}\left(Q(t)\right),$$

and ${}^{\mathrm{e}}{\mathbb{U}}_P^{Q(t)}\dot{V}(t)=\stackrel{\star }{Q}(t)$ to obtain

$$\frac{d}{dt}\mathscr{H}\left(Q(t)\right)=-{⟨logQ(t)+\mathscr{H}\left(Q(t)\right),\stackrel{\star }{Q}(t)⟩}_{Q(t)}.$$

We have identified the gradient of the entropy in the statistical bundle,

$$grad\mathscr{H}\left(Q\right)=-(logQ+\mathscr{H}\left(Q\right)).$$

(12)

Notice that the previous computation could have been done using the exponential family $Q(t)={\mathrm{e}}_P(tV)$. See the computation of the gradient flow in [8].

In the next section, we extend the computation illustrated in the example above to scalar fields on the statistical bundle.

4. Lagrangian Function

A Lagrangian function is a smooth scalar field on the statistical bundle

$$L:S\epsilon \left(\mu \right)\ni (Q,W)\mapsto L(Q,W)\in \mathbb{R}.$$

At each fixed density $Q\in \epsilon \left(\mu \right)$, the partial mapping

$$S_Q\epsilon \left(\mu \right)\ni W\mapsto L(Q,W)$$

(13)

is defined on the vector space $S_q\epsilon \left(\mu \right)$; hence, we can use the ordinary derivative, which in this case is called the fiber derivative,

$$d_{2}L(Q,W)[H_2]={\left.\frac{d}{dt}L(Q,W+t{H}_2)\right|}_{t=0},H_2\in S_Q\epsilon \left(\mu \right).$$

(14)

Example 3 (Running Example 1).

If

$$L(Q,W)=\frac{1}{2}{⟨W,W⟩}_Q+\kappa \mathscr{H}\left(Q\right),\kappa \ge o,$$

(15)

then $d_{2}L(Q,W)[H_2]={⟨W,H_2⟩}_Q$. The example is suggested by the form of the classical Lagrangian function in mechanics, where the first term is the kinetic energy and $-\kappa \epsilon \left(Q\right)$ is the potential energy.

As the statistical bundle $S\epsilon \left(\mu \right)$ is non-trivial, the computation of the partial derivative of the Lagrangian with respect to the first variable requires some care. We want to compute the expression of the total derivative in a chart of the affine atlas defined in Equations (1) and (2).

Let $t\mapsto \gamma (t)=(Q(t),W(t))$ be a smooth curve in the statistical bundle. In the chart centered at P, we have

$$Q(t)={\mathrm{e}}^{U(t)-K_P(U(t))}\times P={\mathrm{e}}_P(U(t)),W(t)={}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U(t))}V(t),$$

with $t\mapsto {\gamma }_P(t)=(U(t),V(t))$ being a smooth curve in ${(S_P\epsilon \left(\mu \right))}^2$. Let us compute the velocity of variation of the Lagrangian L along the curve $\gamma$.

$$\frac{d}{dt}L(\gamma (t))=\frac{d}{dt}L(Q(t),W(t))=\frac{d}{dt}L({\mathrm{e}}_P(U(t)),{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U(t))}V(t))=\frac{d}{dt}{L}_P(U(t),V(t)),$$

with $L_P(U,V)=L({\mathrm{e}}_P(U),{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}V)$. It follows that

$$\frac{d}{dt}L(Q(t),W(t))=d_1{L}_P(U(t),V(t))[\dot{U}(t)]+d_2{L}_P(U(t),V(t))[\dot{V}(t)].$$

(16)

If we write $Q={\mathrm{e}}_P(U)$ and $W={}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}V$, then we have

$$d_2{L}_P(U,V)[H_2]={\left.\frac{d}{dt}{L}_P(U,V+t{H}_2)\right|}_{t=0}={\left.\frac{d}{dt}L(Q,W+t{}^{\mathrm{e}}{\mathbb{U}}_P^Q{H}_2)\right|}_{t=0}=d_{2}L(Q,W)[{}^{\mathrm{e}}{\mathbb{U}}_P^Q{H}_2],$$

(17)

where $d_{2}L$ is the fiber derivative of L. As $\dot{U}(t)={}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^P\stackrel{\star }{Q}(t)$ and ${}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U(t))}\dot{V}(t)=\stackrel{\star }{W}(t)$, it follows from Equations (16) and (17) that

$$\frac{d}{dt}L(Q(t),W(t))=d_1{L}_P(U(t),V(t))[{}^{\mathrm{e}}{\mathbb{U}}_{Q(t)}^P\stackrel{\star }{Q}(t)]+d_{2}L(Q(t),W(t))[\stackrel{\star }{W}(t)].$$

In the equation above, the first term on the RHS does not depend on P because the LHS and the second term of the RHS do not depend on P. Hence, we define the first partial derivative of the Lagrangian function to be

$$d_1(Q,W)[H_1]=d_1{L}_P(U,V)[{}^{\mathrm{e}}{\mathbb{U}}_{{\mathrm{e}}_P(U)}^P{H}_1],H_1\in S_Q\epsilon \left(\mu \right),$$

(18)

so that the derivative of L along $\gamma$ becomes

$$\frac{d}{dt}L(Q(t),W(t))=d_{1}L(Q(t),W(t))[\stackrel{\star }{Q}(t)]+d_{2}L(Q(t),W(t))[\stackrel{\star }{W}(t)].$$

(19)

In particular, if $W(t)=\stackrel{\star }{Q}(t)$, then

$$\frac{d}{dt}L(Q(t),\stackrel{\star }{Q}(t))=d_{1}L(Q(t),\stackrel{\star }{Q}(t))[\stackrel{\star }{Q}(t)]+d_{2}L(Q(t),\stackrel{\star }{Q}(t))[\stackrel{**}{Q}(t)],$$

see Equation (5).

Example 4 (Running Example 2).

With the Lagrangian of Equation (15), we have

$$\begin{array}{c}\hfill L_P(U,V)=\frac{1}{2}{⟨{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}V,{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}V⟩}_{{\mathrm{e}}_P(U)}-\kappa {𝔼}_{{\mathrm{e}}_P(U)}\left[U-K_P(U)+logP\right]=\\ \hfill \frac{1}{2}{d}^2{K}_P(U)[V,V]+\kappa \left(K_P(U)-d{K}_P(U)[U+logP+\mathscr{H}\left(P\right)]+\mathscr{H}\left(P\right)\right),\end{array}$$

see Equations (9) and (11). The first partial derivative is

$$\begin{array}{cc}\hfill d_1{L}_P(U,V)[H_1]=& \\ & \frac{1}{2}{d}^3{K}_P(U)[V,V,H_1]+\kappa \left(d{K}_P(U)[H_1]-d^2{K}_P(U)[U+logP+\mathscr{H}\left(P\right),H_1]-d{K}_P(U)[H_1]\right)=\hfill \\ & \frac{1}{2}{d}^3{K}_P(U)[V,V,H_1]-\kappa d^2{K}_P(U)[U+logP+\mathscr{H}\left(P\right),H_1]=\hfill \\ & \frac{1}{2}{𝔼}_Q\left[W^2{}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_1\right]-\kappa {𝔼}_Q\left[(logQ+\mathscr{H}\left(Q\right)){}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_1\right]=\hfill \\ & {𝔼}_Q\left[\left(\frac{1}{2}\left(W^2-{𝔼}_Q\left[W^2\right]\right)-\kappa (logQ+\mathscr{H}\left(Q\right))\right){}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}{H}_1\right],\hfill \end{array}$$

where we have used Equations (9) and (10) together with ${}^{\mathrm{e}}{\mathbb{U}}_P^{{\mathrm{e}}_P(U)}(U+logP+\mathscr{H}\left(P\right))=logQ+\mathscr{H}\left(Q\right)$.

We have found that

$$d_{1}L(Q,W)[H_1]={⟨\frac{1}{2}\left(W^2-{𝔼}_Q\left[W^2\right]\right)-\kappa (logQ+\mathscr{H}\left(Q\right)),H_1⟩}_Q,H_1\in S_Q\epsilon \left(\mu \right),$$

(20)

and also

$$d_{1}L(Q(t),\stackrel{\star }{Q}(t))[\stackrel{\star }{Q}(t)]={⟨\frac{1}{2}\left({\stackrel{\star }{Q}(t)}^2-{𝔼}_Q\left[{\stackrel{\star }{Q}(t)}^2\right]\right)-\kappa (logQ+\mathscr{H}\left(Q\right)),\stackrel{\star }{Q}(t)⟩}_Q.$$

Using the fiber derivative computed in the first part of the running example, we find

$$\frac{d}{dt}L(Q(t),\stackrel{\star }{Q}(t))={⟨\frac{1}{2}\left({\stackrel{\star }{Q}(t)}^2-{𝔼}_Q\left[{\stackrel{\star }{Q}(t)}^2\right]\right)-\kappa (logQ+\mathscr{H}\left(Q\right)),\stackrel{\star }{Q}(t)⟩}_Q+{⟨\stackrel{\star }{Q}(t),\stackrel{**}{Q}(t)⟩}_Q.$$

Notice that Equation (12) shows that one of the terms in the equations above is $grad\mathscr{H}\left(Q\right)$.

5. Action Integral

If $[0,1]\ni t\mapsto Q(t)$ is a smooth curve in the exponential manifold, then the action integral

$$A(Q)={\int }_0^{1}L(Q(t),\stackrel{\star }{Q}(t))dt$$

is well defined. We consider the expression of Q in the chart centered at P, $Q(t)={\mathrm{e}}^{U(t)-K_P(U(t))}\times P$.

Given $\phi \in C^1([0,1])$ with $\phi (0)=\phi (1)=0$, for each $\delta \in \mathbb{R}$ and $H\in S_P\epsilon \left(\mu \right)$, we define the perturbed curve

$$Q_{\delta }(t)={\mathrm{e}}^{(U(t)+\delta \phi (t)H)-K_P(U(t)+\delta \phi (t)H)}\times P.$$

We have $Q_{\delta }(0)=Q(0)$, $Q_{\delta }(1)=Q(1)$, and

$${\stackrel{\star }{Q}}_{\delta }(t)=\dot{U}(t)+\delta \dot{\phi }(t)H-{𝔼}_{Q_{\delta }(t)}\left[(\dot{U}(t)+\delta \dot{\phi }(t))H\right],$$

whose expression in the chart centered at P is $\dot{U}(t)+\delta \dot{\phi }(t)H$.

Let us consider the variation in $\delta$ of the action integral. We apply Equation (19) applied to the smooth curve in $S\epsilon \left(\mu \right)$ given by

$$\delta \mapsto (Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t)),$$

where t is fixed. As

$$\frac{d}{d\delta }log{Q}_{\delta }(t)=\frac{d}{d\delta }\left(U(t)+\delta \phi (t)H\right)-{𝔼}_{Q_{\delta }(t)}\left[\frac{d}{d\delta }\left(U(t)+\delta \phi (t)H\right)\right]=\phi (t)(H-{𝔼}_{Q_{\delta }(t)}\left[H\right])$$

and

$${}^{\mathrm{e}}{\mathbb{U}}_P^{Q_{\delta }(t)}\frac{d}{d\delta }(\dot{U}(t)+\delta \dot{\phi }(t)H)=\dot{\phi }(t)(H-{𝔼}_{Q_{\delta }(t)}\left[H\right]),$$

we obtain

$$\begin{array}{c}\hfill \frac{d}{d\delta }A(Q_{\delta })={\int }_0^1\frac{d}{d\delta }L(Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t))dt=\\ \hfill {\int }_0^1\left(\phi (t)d_{1}L(Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t))[H-{𝔼}_{Q_{\delta }(t)}\left[H\right]]+\dot{\phi }(t)d_{2}L(Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t))[H-{𝔼}_{Q_{\delta }(t)}\left[H\right]]\right)dt=\\ \hfill {\int }_0^1\phi (t)\left(d_{1}L(Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t))[H-{𝔼}_{Q_{\delta }(t)}\left[H\right]]-\frac{d}{dt}{d}_{2}L(Q_{\delta }(t),{\stackrel{\star }{Q}}_{\delta }(t))[H-{𝔼}_{Q_{\delta }(t)}\left[H\right]]\right)dt.\end{array}$$

If $t\mapsto Q(t)$ is a critical curve of the action integral, then ${\left.\frac{d}{d\delta }A(Q_{\delta })\right|}_{\delta =0}=0$; hence, for all $\phi$ and H, we have

$${\int }_0^1\phi (t)\left(d_{1}L(Q(t),\stackrel{\star }{Q}(t))[H-{𝔼}_{Q(t)}\left[H\right]]-\frac{d}{dt}{d}_{2}L(Q(t),\stackrel{\star }{Q}(t))[H-{𝔼}_{Q(t)}\left[H\right]]\right)dt=0.$$

(21)

This in turn implies that for each $t\in [0,1]$ and $H\in S_{Q(t)}\epsilon \left(\mu \right)$, the Euler–Lagrange equation holds:

$$d_{1}L(Q(t),\stackrel{\star }{Q}(t))[H]-\frac{d}{dt}{d}_{2}L(Q(t),\stackrel{\star }{Q}(t))[H]=0.$$

(22)

Example 5 (Running Example 3).

For the Lagrangian of Equation (15), we can use Equation (20) in the form

$$\begin{array}{c}{d}_{1}L(Q(t),\stackrel{\star }{Q}(t))[H-{𝔼}_{Q(t)}\left[H\right]]=\hfill \\ \hfill {⟨\frac{1}{2}\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)-\kappa (log\left(Q(t)\right)+\mathscr{H}\left(Q(t)\right)),H-{𝔼}_{Q(t)}\left[H\right]⟩}_{Q(t)},\end{array}$$

with $H\in S_P\epsilon \left(\mu \right)$. For the other term, we have

$$d_{2}L(Q(t),\stackrel{\star }{Q}(t))[H-{𝔼}_{Q(t)}\left[H\right]]={⟨\stackrel{\star }{Q}(t),H-{𝔼}_{Q(t)}\left[H\right]⟩}_{Q(t)}=d^2{K}_P(U(t))[\dot{U}(t),H],$$

whose derivative is

$$\begin{array}{c}\hfill \frac{d}{dt}{d}^2{K}_P(U(t))[\dot{U}(t),HR]=d^3{K}_P(U(t))[\dot{U}(t),\dot{U}(t),H]+d^2{K}_P(U(t))[\ddot{U}(t),H]=\\ \hfill {𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2(H-{𝔼}_{Q(t)}\left[H\right])\right]+{𝔼}_{Q(t)}\left[\stackrel{**}{Q}(t)(H-{𝔼}_{Q(t)}\left[H\right])\right]=\\ \hfill {𝔼}_{Q(t)}\left[\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)(H-{𝔼}_{Q(t)}\left[H\right])\right]+{𝔼}_{Q(t)}\left[\stackrel{**}{Q}(t)(H-{𝔼}_{Q(t)}\left[H\right])\right].\end{array}$$

Dropping the generic H, the Euler–Lagrange equation becomes

$$\stackrel{**}{Q}(t)+\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)=\frac{1}{2}\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)-\kappa (log\left(Q(t)\right)+\mathscr{H}\left(Q(t)\right));$$

that is,

$$\stackrel{**}{Q}(t)+\frac{1}{2}\left(\stackrel{\star }{Q}{(t)}^2-{𝔼}_{Q(t)}\left[\stackrel{\star }{Q}{(t)}^2\right]\right)=-\kappa (log\left(Q(t)\right)+\mathscr{H}\left(Q(t)\right)).$$

The equation above has been derived using the exponential affine geometry of the statistical bundle and involves $\stackrel{**}{Q}(t)$. However, by using Equations (5), (6), and (12), we find the equivalent form

$${}^0{D}^{2}Q(t)=\kappa grad\mathscr{H}\left(Q(t)\right).$$

6. Discussion

We have shown that the research program consisting of applying concepts taken from Classical Mechanics to Statistics makes sense, even if no practical application has been produced in this paper. Some simple examples have been discussed in order to show clearly that the language from classical mechanics is indeed suggestive when applied to typical concepts in Statistics such as Fisher score and statistical entropy. The derivation of the Euler–Lagrange equations is classically done in the set-up of the Riemannian geometry, while here we have used the affine structure of Information Geometry. The present provisional results prompt a generalization to non-finite sample spaces and the development of applications. Finally, the related Hamiltonian formalism remains to be investigated.