Hamilton–Jacobi–Bellman Equations in Stochastic Geometric Mechanics ^†

Abstract

This paper summarises a new framework of Stochastic Geometric Mechanics that attributes a fundamental role to Hamilton–Jacobi–Bellman (HJB) equations. These are associated with geometric versions of probabilistic Lagrangian and Hamiltonian mechanics. Our method uses tools of the “second-order differential geometry”, due to L. Schwartz and P.-A. Meyer, which may be interpreted as a probabilistic counterpart of the canonical quantization procedure for geometric structures of classical mechanics. The inspiration for our results comes from what is called “Schrödinger’s problem” in Stochastic Optimal Transport theory, as well as from the hydrodynamical interpretation of quantum mechanics. Our general framework, however, should also be relevant in Machine Learning and other fields where HJB equations play a key role.

1. Hamilton–Jacobi–Bellman Equations

Hamilton–Jacobi–Bellman (HJB) equations are a fundamental tool of Optimal Control theory, more precisely, of “Dynamical Programming”, and were created in the 1950s by R. Bellman and collaborators for the needs of aerospace engineering. Although problems of classical calculus of variations can be solved using it, the impact of the HJB equations never stopped extending far beyond their original motivations. In stochastic Optimal Control [1], they also allow control of Markovian diffusion processes in the form of nonlinear partial differential equations of second-order (in space) for a scalar field S on ${\mathbb{R}}^n$:

$$\frac{\partial S}{\partial t}-H\left(x,-\nabla S,-{\nabla }^{2}S,t\right)=0,$$

(1)

where H is called a second-order (SO) Hamiltonian, analogous to the Hamilton–Jacobi equation of classical mechanics. In Equation (1), the presence of a Hessian operator in H is due to the infinitesimal generator of underlying diffusion processes as a consequence of Itô’s correction. On the other hand, HJB equations have become essential in recent developments of the mathematics of, for instance, deep learning [2] and geometric studies of hydrodynamical interpretation of quantum mechanics [3].

Here, we are not going to consider difficulties associated with the fact that solutions of HJB (the “value functions”) are generally too irregular to be interpreted in a classical sense, or on those resulting from the practical need to solve very high-dimensional versions of such PDEs. Instead, we shall summarize a recent work answering the following natural questions about Equation (1):

If Equation (1) is a kind of deformation of the classical Hamilton–Jacobi equation, what are the relevant stochastic Lagrangian and Hamiltonian mechanics? Additionally, what are the latent geometrical structures?

Our guide to achieve these goals is a program of stochastic deformation of classical mechanics founded on an old idea of E. Schrödinger (often called these days, “Schrödinger’s problem” [4,5]). In substance, this is a statistical physics analogue of quantum mechanics, regarded as a stochastic deformation of classical Optimal Transport. The associated solution processes are called Bernstein’s reciprocal processes [6,7] and enjoy a special version of time-reversibility despite the fact that they are generally inhomogeneous. This aspect of the theory will not be elaborated here.

Instead of traditional tools of stochastic analysis on manifolds, founded by Itô, Malliavin, etc., we shall adapt a less familiar approach, due to L. Schwartz and P.-A. Meyer, called stochastic (or second-order) differential geometry [8]. This way to deform classical geometric structures into others, compatible with the stochastic nature of Brownian randomness, can be regarded as a probabilistic counterpart of the quantization procedure.

2. Second-Order Differential Geometry

The first question to ask about Equation (1) is: In what sense is the “Hamiltonian”, say H, a natural deformation of a Hamiltonian $H_0$ in classical mechanics? The first step [8] is to define second-order versions of tangent and cotangent spaces of a smooth manifold M.

A second-order (SO) tangent vector A at a given point $q\in M$ by

$$A=A^i\frac{\partial }{\partial x^i}{|}_q+A^{jk}\frac{{\partial }^2}{\partial x^j\partial x^k}{|}_q$$

(2)

for coefficients $A^i$, $A^{jk}$ such that $\left(A^{jk}\right)$ forms a symmetric $(2,0)$-tensor and the expression on the right-hand side is invariant under changes of coordinates. In general, $\left(A^i\right)$ is not a vector, which can be seen from changes of coordinates. The second-order tangent space ${\mathcal{T}}_q^{S}M$ to M at q is the set of all SO tangent vectors at q. The second-order tangent bundle is then ${\mathcal{T}}^{S}M={\cup }_{q\in M}{\mathcal{T}}_q^{S}M$. Clearly, $TM\subset {\mathcal{T}}^{S}M$ as a subbundle. A smooth field of second-order tangent vectors, i.e, a smooth section of ${\mathcal{T}}^{S}M$, is called a second-order vector field.

According to Schwartz and Meyer, any geometric statement for such a second-order tangent vector has a probabilistic content. To see this, we consider the following Itô stochastic differential equations (SDEs) on ${\mathbb{R}}^d$:

$$d{X}^i\left(t\right)=b^i(t,X\left(t\right))dt+{\sigma }_r^i(t,X\left(t\right))d{B}^r\left(t\right).$$

(3)

Its associated generator is given by $A^X=b^i\frac{\partial }{\partial x^i}+\frac{1}{2}{\sum }_{r=1}^N{\sigma }_r^i{\sigma }_r^j\frac{{\partial }^2}{\partial x^i\partial x^j}$, which is a typical example of SO vector fields. In general, the generators of diffusion processes are called a second-order elliptic vector field due to the positive semi-definiteness of the coefficients of second-order derivatives. For a diffusion process X, defined on a probability space $(\Omega ,\mathcal{G},\mathbf{P})$ and adapted to a nondecreasing filtration $\left\{{\mathcal{P}}_t\right\}$, the coefficients of its generator can be characterized by

$$\begin{array}{c}{\left(DX\right)}^i\left(t\right)={\displaystyle \underset{ϵ\to 0^{+}}{lim}}\mathbf{E}\left[\frac{X^i(t+ϵ)-X^i\left(t\right)}{ϵ}|{\mathcal{P}}_t\right],\\ {\left(QX\right)}^{jk}\left(t\right)={\displaystyle \underset{ϵ\to 0^{+}}{lim}}\mathbf{E}\left[\frac{(X^j(t+ϵ)-X^j\left(t\right))(X^k(t+ϵ)-X^k\left(t\right))}{ϵ}|{\mathcal{P}}_t\right].\end{array}$$

This last relation encapsulates the second-order statistical information about all trajectories $t\mapsto X\left(t\right)$. The pair $(DX,QX)$ is a process taking values in ${\mathcal{T}}^{S}M$ and called the mean derivatives of X. When X has differentiable trajectories, $DX$ reduces to a classical time derivative and $QX$ to 0. For the Itô SDE (3) on ${\mathbb{R}}^d$, its mean derivatives are given by $DX\left(t\right)=b(t,X(t\left)\right)$ and $QX\left(t\right)=\left(\sigma {\sigma }^T\right)(t,X\left(t\right))$. However, for a general diffusion X valued on a manifold M, $DX$ does not transform as a vector field, which can be verified through applying Itô’s formula for changes of coordinates. In order to overcome this problem, we equip M with a linear connection ∇ and use it to compensate as a correction term resulting from Itô’s formula. That is, we define the following ∇-dependent mean derivative, in terms of Christoffel’s symbols,

$${\left(D_{\nabla }X\right)}^i={\left(DX\right)}^i+\frac{1}{2}{\Gamma }_{jk}^i{\left(QX\right)}^{jk}.$$

(4)

Then, $D_{\nabla }X$ does transform as a vector field.

Inspired by mean derivatives, we denote the canonical coordinates on ${\mathcal{T}}^{S}M$ by $(x,Dx,Qx)$ and define their action on A in (2) as follows:

$$x^i\left(A\right)=x^i\left(q\right),D^{i}x\left(A\right)=A^i,Q^{jk}x\left(A\right)=2A^{jk}.$$

The objects dual to SO tangent vectors are second-order cotangent vectors, whose general form is:

$$\alpha ={\alpha }_i{d}^2{x}^i{{|}_q+\frac{1}{2}{\alpha }_{jk}d{x}^j·d{x}^k|}_q,$$

(5)

where $\left({\alpha }_i\right)$ forms a covector and ${\alpha }_{jk}$ is symmetric in $j,k$. The pairing of the above $\alpha$ with SO tangent vector A in (2) is given by

$$⟨\alpha ,A⟩={\alpha }_i{A}^i+{\alpha }_{jk}{A}^{jk}.$$

The SO cotangent bundle, i.e., the set of all SO cotangent vectors on M, is represented by ${\mathcal{T}}^{S^{*}}M$. The canonical coordinates on it are denoted by $(x,p,o)$ and are defined, when acting on $\alpha$ in (5), as follows:

$$x^i\left(\alpha \right)=x^i\left(q\right),p_i\left(\alpha \right)={\alpha }_i,o_{jk}\left(\alpha \right)={\alpha }_{jk}.$$

(6)

There are two basic examples of second-order forms, say, $d^{2}f$ and $df·dg$, where f and g are given smooth functions on M. They are defined as follows: for $A\in {\mathcal{T}}^{S}M$,

$$⟨d^{2}f,A⟩:=Af,⟨df·dg,A⟩:=A\left(fg\right)-fAg-gAf=:{\Gamma }_A(f,g),$$

where d is the classical exterior differential; the operator $d^2$ is called a second-order differential; the dot operator · is called a symmetric product; and ${\Gamma }_A$ is usually called a “carré du champ” operator. By construction, the restriction of any SO form to $T^{*}M$, the classical cotangent fibre bundle, is a classical form.

3. Stochastic Hamiltonian Mechanics

In classical mechanics, the canonical symplectic structure on the cotangent bundle plays a substantial role in Hamiltonian mechanics. The symplectic 1-form is given by $p_{i}d{x}^i$, also known as Poincaré’s relative integral invariant [9]. Now, the second-order version of the Poincaré 1-form [10] is given, according to (5) and (6), by

$$p_i{d}^2{x}^i+\frac{1}{2}{o}_{jk}d{x}^j·d{x}^k,$$

as a second-order form on the phase space ${\mathcal{T}}^{S^{*}}M$. Analogous to the classical symplectic 2-form $d{x}^i\wedge d{p}_i$, one obtains the second-order version involving an extra set of coordinates $o_{jk}$:

$$\Omega =-d^2\left(p_i{d}^2{x}^i+\frac{1}{2}{o}_{jk}d{x}^j·d{x}^k\right)=d^2{x}^i\wedge d^2{p}_i+\frac{1}{2}d{x}^j·d{x}^k\wedge d^2{o}_{jk}.$$

Associated with a SO Hamiltonian function $H\in C^{\infty }\left({\mathcal{T}}^{S^{*}}M\right)$, the SO Hamiltonian vector field $A_H$ on ${\mathcal{T}}^{S^{*}}M$ is defined by

$$\Omega (A_H,B)=d^{2}H\left(B\right),\forall B\in {\mathcal{T}}^S{\mathcal{T}}^{S*}M,$$

namely, in local coordinates,

$$\begin{array}{cc}\hfill A_H=& \frac{\partial H}{\partial p_i}\frac{\partial }{\partial x^i}-\frac{\partial H}{\partial x^i}\frac{\partial }{\partial p_i}+\frac{\partial H}{\partial o_{jk}}\frac{{\partial }^2}{\partial x^j\partial x^k}-\left(\frac{{\partial }^{2}H}{\partial x^j\partial x^k}+C_{jk}\right)\frac{\partial }{\partial o_{jk}}\hfill \\ \hfill & +A_{jk}\frac{{\partial }^2}{\partial p_j\partial p_k}+A_{ijkl}\frac{{\partial }^2}{\partial o_{ij}\partial o_{kl}}+A_k^j\frac{{\partial }^2}{\partial x^j\partial p_k}+A_{kl}^j\frac{{\partial }^2}{\partial x^j\partial o_{kl}}+A_{jkl}\frac{{\partial }^2}{\partial p_j\partial o_{kl}},\hfill \end{array}$$

where the coefficients $C_{jk},A_{jk},A_{ijkl},A_k^j,A_{kl}^j,A_{jkl}$ are smooth functions satisfying

$$C_{jk}\frac{\partial H}{\partial o_{jk}}=A_{jk}\frac{{\partial }^{2}H}{\partial p_j\partial p_k}+A_{ijkl}\frac{{\partial }^{2}H}{\partial o_{ij}\partial o_{kl}}+A_k^j\frac{{\partial }^{2}H}{\partial x^j\partial p_k}+A_{kl}^j\frac{{\partial }^{2}H}{\partial x^j\partial o_{kl}}+A_{jkl}\frac{{\partial }^{2}H}{\partial p_j\partial o_{kl}}.$$

The stochastic Hamilton equations associated with $A_H$ are given, in local coordinates, by

$$\left\{\begin{array}{cc}\hfill D^{i}x& =\frac{\partial H}{\partial p_i},D_{i}p=-\frac{\partial H}{\partial x^i},\hfill \\ \hfill Q^{jk}x& =2\frac{\partial H}{\partial o_{jk}},D_{jk}o=-\left(\frac{{\partial }^{2}H}{\partial x^j\partial x^k}+C_{jk}\right),\hfill \\ \hfill C_{ij}\frac{\partial H}{\partial o_{ij}}& =\frac{1}{2}{Q}_{jk}p\frac{{\partial }^{2}H}{\partial p_j\partial p_k}+\frac{1}{2}{Q}_{ijkl}o\frac{{\partial }^{2}H}{\partial o_{ij}\partial o_{kl}}+Q_k^j(x,p)\frac{{\partial }^{2}H}{\partial x^j\partial p_k}\hfill \\ \hfill & +Q_{kl}^j(x,o)\frac{{\partial }^{2}H}{\partial x^j\partial o_{kl}}+Q_{jkl}(p,o)\frac{{\partial }^{2}H}{\partial p_j\partial o_{kl}}.\hfill \end{array}\right.$$

(7)

The solution is of the form $\left(X\right(t),p(t,X\left(t\right)),o(t,X\left(t\right)\left)\right)$ for X as an M-valued process and $(p,o)$ as a time-dependent SO form.

Notice that the last three equations describe fundamental second-order additions to deterministic Hamiltonian equations. However, the mean derivatives D are the only regularization needed in the first two equations. Qualitatively, since p and o are functions of $X\left(t\right)$, the last two equations can be simplified by applying

$$D=\frac{\partial }{\partial t}+\frac{\partial H}{\partial p_j}\frac{\partial }{\partial x^j}+\frac{\partial H}{\partial o_{jk}}\frac{{\partial }^2}{\partial x^j\partial x^k}$$

to the second and fourth equations, assuming that (the distribution of) $X\left(t\right)$ has full support for all t. It follows that

$$o_{ij}(t,x)=\frac{\partial p_i}{\partial x^j}(t,x)=\frac{\partial p_j}{\partial x^i}(t,x),$$

(8)

the second equality being the Maxwell relations for thermodynamics [11]. We refer to (8) as an integrability condition of (7).

Similar to classical mechanics, when the SO Hamiltonian H depends explicitly on time, one extends the phase space to be ${\mathcal{T}}^{S^{*}}M\times \mathbb{R}$ and endows it with the second-order analogue of the Poincaré-Cartan form:

$$p_i{d}^2{x}^i+\frac{1}{2}{o}_{jk}d{x}^j·d{x}^k-Hdt.$$

(9)

Canonical transformations are changes of coordinates in the extended phase space ${\mathcal{T}}^{S*}M\times \mathbb{R}$ from $(x,p,o,t)$ to $(y,P,O,t)$ that leave the stochastic Hamilton Equation (7) invariant, or equivalently, leave the canonical form (9) invariant up to an exact second-order differential:

$$\left(P_i{d}^2{y}^i+\frac{1}{2}{O}_{jk}d{y}^j·d{y}^k-Kdt\right)=\left(p_i{d}^2{x}^i+\frac{1}{2}{o}_{jk}d{x}^j·d{x}^k-Hdt\right)+{\mathit{d}}^{2}G,$$

(10)

where $K\in C^{\infty }({\mathcal{T}}^{S^{*}}M\times \mathbb{R})$ is the new SO Hamiltonian after transformation, and ${\mathit{d}}^2$ is the total differential of first-order in time and second-order in space. This implies that the generating function of the canonical transformation $G(x,y,t)$ satisfies

$$\left\{\begin{array}{c}{p}_i=-\frac{\partial G}{\partial x^i},o_{jk}\frac{\partial x^k}{\partial y^l}=-\frac{{\partial }^{2}G}{\partial x^j\partial x^k}\frac{\partial x^k}{\partial y^l}-\frac{{\partial }^{2}G}{\partial x^j\partial y^l},P_i=\frac{\partial G}{\partial y^i},O_{jk}=\frac{{\partial }^{2}G}{\partial y^j\partial y^k}+\frac{{\partial }^{2}G}{\partial y^j\partial x^l}\frac{\partial x^l}{\partial y^k},\hfill \\ H-K=\frac{\partial G}{\partial t}.\hfill \end{array}\right.$$

(11)

The Hamilton–Jacobi–Bellman (HJB) equation can be introduced by formally letting the new Hamiltonian K vanish. In this case, we write the generating function G as S. Using (11), we can write the HJB equation, as the general version of (1) on manifolds:

$$\frac{\partial S}{\partial t}-H\left(x^i,-\frac{\partial S}{\partial x^i},-\frac{{\partial }^{2}S}{\partial x^j\partial x^k},t\right)=0.$$

(12)

4. Stochastic Hamiltonian and Lagrangian Mechanics on Riemannian Manifolds

If $(M,g)$ is a Riemannian manifold with Levi–Civita connection ∇, one can produce a class of SO Hamiltonians $H=H_{\hslash }$ by deforming a classical one $H_0\in C^{\infty }\left(T^{*}M\right)$ in a canonical way; that is,

$$H_{\hslash }(x,p,o)=H_0(x,p)+\frac{\hslash }{2}{g}^{ij}\left(x\right)\left(o_{ij}-{\Gamma }_{ij}^k\left(x\right)p_k\right),$$

where ℏ is a positive constant (our deformation parameter). Then, system (7) reduces to the following stochastic Hamilton equations on $T^{*}M$:

$$D_{\nabla }X={\nabla }_p{H}_0,\frac{\overline{\mathbf{D}}}{dt}p=-d_x{H}_0,$$

(13)

subject to ${\left(QX\right)}^{ij}\left(t\right)=\hslash g^{ij}\left(X\left(t\right)\right)$, where $\frac{\overline{\mathbf{D}}}{dt}=\frac{\partial }{\partial t}+{\nabla }_{D_{\nabla }X}+\frac{1}{2}{\Delta }_{\mathrm{LD}}$ is the damped mean covariant derivative with respect to X, and ${\Delta }_{\mathrm{LD}}$ is the Laplace-de Rham operator on forms.

Such a Hamiltonian formulation can also been transformed into a Lagrangian formulation by the Legendre transform. Recall that the Legendre transform is a change of variables $T^{*}M\to TM,(x,p)\mapsto (x,\dot{x})$ given by ${\dot{x}}^i=\frac{\partial H_0}{\partial p_i}$. If the Legendre transform is a diffeomorphism (in which case, H is called hyperregular), a Lagrangian function can be produced from H; that is,

$$L_0(x,\dot{x})=p_i{\dot{x}}^i-H_0(x,p).$$

(14)

In this way, the stochastic Hamilton equations (13) are equivalent to the stochastic Euler–Lagrange equation,

$$\frac{\overline{\mathbf{D}}}{dt}\left(d_{\dot{x}}{L}_0\left(X\left(t\right),D_{\nabla }X\left(t\right)\right)\right)=d_x{L}_0\left(X\left(t\right),D_{\nabla }X\left(t\right)\right),$$

which results from the stochastic Hamilton’s stationary-action principle $\delta \mathcal{S}=0$ for the following action functional:

$$\mathcal{S}:=\mathbf{E}{\int }_0^T{L}_0\left(X\left(t\right),D_{\nabla }X\left(t\right)\right)dt,$$

(15)

where the variation $\delta$ is taken over all diffusions X on M over time interval $[0,T]$, satisfying ${\left(QX\right)}^{ij}\left(t\right)=\hslash g^{ij}\left(X\left(t\right)\right)$, and with given endpoint marginal distributions $\mathrm{Law}\left(X\left(0\right)\right)={\mu }_0$ and $\mathrm{Law}\left(X\left(T\right)\right)={\mu }_T$.

Consider a time-dependent Hamiltonian $H_0$. On Riemannian manifolds, canonical transformations of the last section can also be reduced to the tangent bundle. First, we observe that by the Legendre transform (14), the definition (4) of $D_{\nabla }$, and the integrability condition (8), the action functional (15) can be rewritten as

$$\mathcal{S}=\mathbf{E}{\int }_0^T\left(p_i{\left(DX\right)}^i+\frac{1}{2}\frac{\partial p_j}{\partial x^k}{\left(QX\right)}^{jk}-H_{\hslash }\right)dt=\mathbf{E}{\int }_0^T\left(p_i\circ d{x}^i-H_{\hslash }dt\right),$$

where $\circ d$ denotes the Stratonovich stochastic differential. Now, we make a change of coordinates on $T^{*}M\times \mathbb{R}$ from $(x^i,p_i,t)$ to $(y^i,P_i,t)$ and denote the SO Hamiltonian by $K_{\hslash }$ and its classical part by $K_0$.

As in the previous section, the general condition for a transformation to be canonical is to preserve the form of a stochastic Hamilton system (13). This is equivalent to preserving the form of the stochastic stationary++action principle of (15). It follows that

$$\delta \mathbf{E}{\int }_0^T\left(p_i\circ d{x}^i-H_{\hslash }dt\right)=\delta \mathbf{E}{\int }_0^T\left(P_i\circ d{y}^i-K_{\hslash }dt\right)=0.$$

Since the underlying process X has zero variation at the endpoints, both equalities will be satisfied if the integrands are related by the following SDE:

$$P_i\circ d{y}^i-K_{\hslash }dt=p_i\circ d{x}^i-H_{\hslash }dt+dG.$$

(16)

In contrast with classical theory of canonical transformations and also (10), which are described by equations for forms, Equation (16) is understood as a stochastic differential equation. However, as in classical theory [9], here we can also have all four types of generating functions for (16) that are related to each other through classical Legendre transforms. Indeed, canonical transformations here are processed on cotangent bundles, which means they are a special case of (10) where the canonical transformations on SO cotangent bundles are induced by classical ones. We take the type-one generating function $G=G_1(x,y,t)$. Using Itô’s formula, $dG=\frac{\partial G_1}{\partial t}dt+\frac{\partial G_1}{\partial x^i}\circ d{x}^i+\frac{\partial G_1}{\partial y^i}\circ d{y}^i$, and vanishing the coefficients of every (stochastic) differential, $\circ dx$, $\circ dy$, and $dt$ in (16), we get

$$p_i=-\frac{\partial G_1}{\partial x^i},P_i=\frac{\partial G_1}{\partial y^i},H_{\hslash }-K_{\hslash }=\frac{\partial G_1}{\partial t},$$

which partially recovers (11). By requiring the new Hamiltonian $K_0$ to be identically zero and writing $G_1$ as S, the last equation turns into the following Hamilton–Jacobi–Bellman equations:

$$\frac{\partial S}{\partial t}(x,y,t)-H_0\left(x^i,-\frac{\partial S}{\partial x^i}(x,y,t),t\right)+\frac{\hslash }{2}{\Delta }_{x}S(x,y,t)+\frac{\hslash }{2}{\Delta }_{y}S(x,y,t)=0,$$

(17)

where $(x,y)$ are regarded as coordinates on the product manifold of the two manifolds before and after transformation and are equipped with the direct-sum Riemannian metric and Levi–Civita connection. Clearly, Equation (17) can be interpreted as ℏ-deformation of the classical Hamilton–Jacobi equation:

$$\frac{\partial S}{\partial t}-H_0\left(x^i,-\frac{\partial S}{\partial x^i},t\right)=0.$$

(18)

Type-two generating functions are also useful. Let $G=G_2(x,P,t)+y^i{P}_i$. In the same way as type one, we can get

$$p_i=-\frac{\partial G_2}{\partial x^i},y^i=-\frac{\partial G_2}{\partial P_i},H_{\hslash }-K_{\hslash }=\frac{\partial G_2}{\partial t}.$$

As an example, we consider the Hamiltonian $H_0(x,p,t)=-\frac{1}{2}{g}^{ij}\left(x\right)p_i{p}_j-g^{ij}\left(x\right)p_i\frac{\partial S}{\partial x^j}$$(x,t)-V\left(x\right)$. We take $G_2(x,P,t)=S(x,t)-x^i{P}_i$. Then $p_i=P_i-\frac{\partial S}{\partial x^i}$, $y=x$ and $H_{\hslash }-K_{\hslash }=\frac{\partial S}{\partial t}$. Thus, the new Hamiltonian is $K_0(y,P,t)=-\frac{1}{2}{g}^{ij}\left(y\right)P_i{P}_j+\frac{1}{2}{|\nabla S(y,t)|}^2-V\left(y\right)-\frac{\hslash }{2}\Delta S(y,t)-\frac{\partial S}{\partial t}(y,t)$. To make $K_0$ be the standard form $K_0(y,P,t)=-\frac{1}{2}{g}^{ij}\left(y\right)P_i{P}_j$, we only need to assume that S and V solve the following HJB equation:

$$\frac{\partial S}{\partial t}-\left(\frac{1}{2}{|\nabla S|}^2-V\right)+\frac{\hslash }{2}\Delta S=0.$$

(19)

A key observation is that $H_{\hslash }$ is a $\left\{{\mathcal{P}}_t\right\}$-martingale but $H_0$ is not. A stochastic Noether’s theorem in [10] shows that such a martingale is always associated with symmetry of an HJB equation.

5. Relations with Stochastic Deformation and Schrödinger’s Problem

The last observation had already been made long ago in the research program of stochastic deformation (cf. [12] and references therein) from a completely different perspective, namely the analogy between Schrödinger’s problem and quantum mechanics.

We are going to specialize our analysis to the HJB Equation (12) on Euclidean space $M={\mathbb{R}}^n$ with the SO Hamiltonian given by the ℏ-deformation of the classical Hamiltonian $H_0(x,p)=\frac{1}{2}{\left|p\right|}^2-V\left(x\right)$:

$$H_{\hslash }(x,p,o)=\frac{1}{2}{\left|p\right|}^2-V\left(x\right)+\frac{\hslash }{2}\mathbf{tr}\left(o\right)$$

(20)

namely, Equation (19) for a given final boundary condition $S(x,T)=S^{*}\left(x\right)$, where $V:{\mathbb{R}}^n\to \mathbb{R}$ is a bounded (for simplicity) scalar potential. Notice the opposite sign of the potential with respect to classical Hamiltonians of such elementary systems. This is expected when well-defined measures are associated with (18), as in the (“Euclidean”) quantization procedure. The left-hand side of the second equation of (7) means $(\frac{\partial }{\partial t}+p·\nabla +\frac{\hslash }{2}\Delta )p_i$. Let us introduce a positive solution $\eta$ of the retrograde (or backward) heat equation:

$$\hslash \frac{\partial \eta }{\partial t}=-\frac{{\hslash }^2}{2}\Delta \eta +V\eta ,$$

(21)

with nonnegative final boundary condition $\eta (T,·)={\eta }_T$. Now define $S=-\hslash log\eta$, solving HJB Equation (19). If $p=-\nabla S=\hslash \nabla log\eta$, take ∇ of Equation (19) and use the integrability condition (8). The result agrees with our second equation of (7). Therefore, the first and third ones characterize a Bernstein’s reciprocal diffusion X:

$$\left\{\begin{array}{cc}\hfill {\left(DX\right)}^i\left(t\right)& =\frac{\partial log\eta }{\partial x^i}(t,X\left(t\right)),\hfill \\ \hfill {\left(QX\right)}^{jk}\left(t\right)& =\hslash {\delta }^{jk}\left(X\left(t\right)\right).\hfill \end{array}\right.$$

On the other hand, the Lagrangian associated with $H_0$ is $L_0(x,\dot{x})=\frac{1}{2}{\left|\dot{x}\right|}^2+V\left(x\right)$. The Benamou–Brenier formula for Schrödinger’s problem, from the Optimal Transport perspective [4], shows that minimizing the action functional (15) is equivalent to minimizing the following relative entropy:

$$H(\mathbf{P}\mid \mathbf{R})=\left\{\begin{array}{cc}{\int }_{{\mathcal{C}}_0^T}log\left(\frac{d\mathbf{P}}{d\mathbf{R}}\right)d\mathbf{P},\hfill & \mathbf{P}\ll \mathbf{R}\hfill \\ +\infty ,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

over all probability measures $\mathbf{P}$ on the path space $C([0,T],{\mathbb{R}}^n)$, such that ${\mu }_0,{\mu }_T$ are the initial and final time marginal distributions of $\mathbf{P}$, i.e., ${\mathbf{P}}_0={\mu }_0$ and ${\mathbf{P}}_T={\mu }_T$. Here, $\mathbf{R}$, called the reference measure, is the distribution of a reversible diffusion (in Kolmogorov’s sense) with generator $\frac{{\hslash }^2}{2}\Delta -V$.

As explained in [12], the quantum “expectation” in state ${\psi }_t\in L^2\left({\mathbb{R}}^n\right)$ of the Hamiltonian operator $\mathcal{H}=-\frac{{\hslash }^2}{2}\Delta +V$, the quantization of $H_0$ in Equation (18), is

$${⟨\mathcal{H}⟩}_{{\psi }_t}=\int {\overline{\psi }}_t\left(x\right)\mathcal{H}{\psi }_t\left(x\right)dx=\int \left({\overline{\psi }}_t{\psi }_t\right)\left(x\right)\frac{\mathcal{H}{\psi }_t}{{\psi }_t}\left(x\right)dx,$$

where ${\psi }_t$ is in the domain ${\mathcal{D}}_{\mathcal{H}}$ of $\mathcal{H}$ dense in $L^2$, and ${\overline{\psi }}_t{\psi }_t={\left|{\psi }_t\right|}^2$ is interpreted as a (Born) probability density. Now consider $\eta (x,t)={\eta }_t\left(x\right)$ solving the retrograde heat Equation (21), i.e., after a change of the variable $t\to -it$ to the Schrödinger equation of $\mathcal{H}$. Using ${\hslash }^2\frac{\Delta \eta }{\eta }={(\hslash \nabla log\eta )}^2+\hslash \nabla ·(\hslash \nabla log\eta )$, the random variable playing the role of $\mathcal{H}$ should be (minus of):

$$\frac{1}{2}{\left|p\right|}^2-V+\frac{\hslash }{2}\nabla ·p,$$

namely, our $H_{\hslash }$ in (20). This is why Schwartz–Meyer second-order differential geometry can be regarded as a kind of (Euclidean) quantization method. We have only summarized here the forward geometric part of our construction. The role of $\overline{\psi }$ is played by positive solutions ${\eta }_t^{*}$ of a Cauchy problem for the usual heat equation (with the same $\mathcal{H}$, which is self-adjoint). Then, any well-defined expectation with respect to Bernstein’s reciprocal diffusion X is computed using the fundamental aspect of Schrödinger’s analogy:

$$\mathbf{P}(X\left(t\right)\in U)={\int }_U{\eta }_t^{*}\left(x\right){\eta }_t\left(x\right)dx,\mathrm{for}\mathrm{a}\mathrm{Borel}\mathrm{set}U.$$

(22)

Associated with ${\eta }_t^{*}$, there is a dual formulation of our results involving a nonincreasing filtration $\left\{{\mathcal{F}}_t\right\}$. In particular, there is another (Cauchy) problem of HJB, adjoint to Equation (19):

$$\frac{\partial S^{*}}{\partial t}+\frac{1}{2}{|\nabla S^{*}|}^2-V-\frac{\hslash }{2}\Delta S^{*}=0.$$

In classical mechanics, it is known (but often forgotten) that the coexistence of two adjoint Hamilton–Jacobi equations, in a given Hamiltonian system, is closely related with the regularity of the trajectories. Our two adjoint HJB equations play the same role for the trajectories $t\mapsto X\left(t\right)$ of Bernstein’s reciprocal processes solving Schrödinger’s problem, cf. [12].

The analogy of the $L^2$ complex conjugate is Schrödinger’s version of time-reversal involved in (22). Consequently, although typically time inhomogeneous, the resulting diffusions are invariant under this time-reversal.

The SO Poincaré-Cartan form allows formulation of a global stochastic Euler–Lagrange equation compatible with our Hamiltonian ones and then a global Noether’s theorem, which is a more general perspective than Schrödinger’s original problem [10]. All these results are founded on HJB equations, which are regarded as SO deformations of classical Hamilton–Jacobi equations.