An Optimal Control Perspective on Classical and Quantum Physical Systems

Abstract

This paper analyzes classical and quantum physical systems from an optimal control perspective. Specifically, we explore whether their associated dynamics can correspond to an open- or closed-loop feedback evolution of a control problem. Firstly, for the classical regime, when it is viewed in terms of the theory of canonical transformations, we find that a closed-loop feedback problem can describe it. Secondly, for a quantum physical system, if one realizes that the Heisenberg commutation relations themselves can be considered constraints in a non-commutative space, then the momentum must depend on the position of any generic wave function. That implies the existence of a closed-loop strategy for the quantum case. Thus, closed-loop feedback is a natural phenomenon in the physical world. By way of completeness, we briefly review control theory and the classical mechanics of constrained systems and analyze some examples at the classical and quantum levels.

1. Introduction

Over the last decades, there has been increasing interest among physicists in applying their concepts and methods to economics, as seen in some classical texts [1,2,3,4,5,6,7,8] or exciting articles about quantum mechanics in finance [9,10,11,12,13,14,15,16]. In this paper, we want to do the exact opposite, that is, we want to apply ideas from the optimal control theory (usually used in finance and economics [17,18,19,20,21]) to interpret the classical and quantum physical world. Indeed, open- and closed-loop control problems are not in the toolbox of mathematical methods in physics; nevertheless, dynamic optimization with its optimal control theory is the cornerstone of modern economic analysis.

There is little in the literature on the possible relationships between optimal control theory and classical and quantum mechanics, despite the fact that they share certain similar mathematical structures. For example, the Pontryagin equations resemble the Hamilton equations, and the Bellman equation is analogous to the Hamilton–Jacobi Equation, which corresponds to the classical limit of quantum theory. Recently, some studies have appeared on how a control problem associated with an economic model can be interpreted as a second-class constrained physical system [22,23,24,25,26]. In [22], it is found that at the classical level, the constrained dynamics given by Dirac’s brackets are the same as those provided by the Pontryagin equations [27]. The quantization of this second-class constrained system (with a right-order operator scheme) ended with a Schrödinger equation, which is just the Hamilton–Jacobi–Bellman equation used in optimal control theory and economics [28]. It is surprising to find that after the quantization of the Pontryagin theorem, Bellman’s maximum principle is obtained.

Thus, if an economic feedback system (characterized by an optimal control theory model) can be seen as a physical system, one can naturally ask the inverse question: Can the physical systems be seen from an optimal control perspective? Or, more specifically, do there exist open-loop and closed-loop strategies in physical systems? If the answer is affirmative, then where are they? Moreover, how and why do these feedback problems appear? This paper gives some clues and answers to all of these questions.

From a control theory perspective, open-loop strategies for the Lagrangian multiplier characterize the Pontryagin theory, while the Hamilton–Jacobi–Bellman equation has intrinsically closed-loop strategies. Since Pontryagin’s approach is equivalent to a classical mechanical model and Bellman’s theory is a quantum mechanical one, that suggests that open-loop strategies can be related to classical mechanics and closed-loop feedbacks with quantum mechanics.

This article shows that classical mechanics can be represented as an open-loop feedback problem in terms of Hamilton’s equations and a closed-loop strategies problem when one uses the canonical transformations approach. Indeed, both schemes are equivalent due to the invariance of the classical Hamiltonian equations by canonical transformations.

In the same way, from the perspective of quantum physics, the Heisenberg canonical commutation relations can be seen as a constraint in the non-commutative phase-space; this necessarily implies the existence of a specific relation between the momentum and the position, which is equivalent to a closed-loop feedback. Thus, open- and closed-loop feedback are also natural phenomena in our physical world.

This work has two principal objectives: The first is to give an optimal control perspective of the classical and quantum systems and to understand how and why such feedback appears. The second one is to show the physicist and optimal control community how the optimal control theory and physical systems (with its classical and quantum mechanical description of the world) are related. For that, it is necessary to have a common language and give the principal ideas behind each theory. The second objective will be fulfilled with the lecture in Section 2 and Section 3 of this paper. After that, we progress to complete the first objective.

It should be noted that given the length of our article, we have not addressed interesting recent topics of stability in control theory (see, for example, [29,30]) and quantum calculus (see, for example, [31,32]). Thus, the main purpose of the paper is to provide a general discussion of the relationship between classical mechanics and quantum mechanics with optimal control theory.

The paper is structured as follows: Section 2 reviews, for the non-specialist reader, some central ideas in control theory, such as the Pontryagin/Bellman approaches and the concepts of closed- and open-loop strategies. Section 3 establishes the basic ideas of classical mechanics in Lagrangian and Hamiltonian forms. Also, the so-called constrained systems in physics are defined and given as an example of this class of system, the optimal control theory itself, at classical and quantum levels.

In Section 4, we start with the paper’s primary objective, so Section 4.1 gives the first clues to interpret classical mechanics as a closed-loop system in control theory. Section 4.2 briefly reviews canonical transformations, which is the standard way to obtain the Hamilton–Jacobi Equation in physics from a purely classical perspective (non-quantum). In Section 4.3, we consider quantum mechanics and discuss why quantum mechanical systems must be interpreted naturally as closed-loop ones. Section 5 gives several examples of the presence of closed-loop strategies in physics at the classical and quantum levels. Lastly, in Section 6, the conclusions of this work are presented.

2. Key Ideas in Control Theory

This section reviews some critical concepts in control theory, such as the Pontryagin approach with its open-loop strategies and the Bellman approach with its closed-loop strategies, that we will use later to interpret the physical systems from an optimal control point of view.

2.1. Dynamic Optimization: The Pontryagin Approach

A generic one-dimensional system will be considered to emphasize the fundamental ideas and keep the equations simple. Generalizations to higher dimensions are straightforward. Consider a standard economic optimal control problem [17,18] for which one must optimize functional

$$A[x,u]={\int }_{t_0}^{t_1}F(x,u,t)dt,$$

(1)

where x represents a state variable (for example, the production rate of a specific good), and u denotes the control variable (such as the cost associated with a firm marketing). The state variable x must obey the market dynamics defined by the differential equation

$$\dot{x}=f(x,u,t)x\left(t_0\right)=x_0.$$

(2)

and the optimal control problem consists of determining the production path $x=x\left(t\right)$ and the control path $u=u\left(t\right)$, which maximise or minimise the cost function. To achieve this goal, one must apply the Lagrange multiplier method, so one considers instead an improved functional A defined over the extended space $(x,u,\lambda )$, which is given by:

$$A[x,u,\lambda ]={\int }_{t_0}^{t_1}F(x,u,t)-\lambda (\dot{x}-f(x,u,t))dt.$$

(3)

To obtain the optimal solution, the integrand of (3) can be interpreted as the Lagrangian:

$$L(x,u,\lambda ,\dot{x},\dot{u},\dot{\lambda })=F(x,u,t)-\lambda (\dot{x}-f(x,u,t)).$$

(4)

so the extremal curves then satisfy the Euler–Lagrange equations:

$$\frac{\partial L}{\partial \lambda }-\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{\lambda }}\right)=0,\frac{\partial L}{\partial x}-\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{x}}\right)=0,\frac{\partial L}{\partial u}-\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{u}}\right)=0.$$

(5)

These equations are also written in the form

$$\dot{x}=\frac{\partial H}{\partial \lambda },\dot{\lambda }=-\frac{\partial H}{\partial x},\frac{\partial H}{\partial u}=0,$$

(6)

with the Hamiltonian H defined by

$$H=H(x,u,\lambda ,t)=F(x,u,t)+\lambda f(x,u,t).$$

(7)

Equation (6) are the Pontryagin equations, which can be obtained through the Pontryagin maximum principle, and these solve the optimization problem.

2.2. Open-Loop and Closed-Loop Strategies

Note that the action (3) can be written in a Hamiltonian form as

$$A[x,u,\lambda ]={\int }_{t_0}^{t_1}-\lambda \dot{x}+H(x,u,\lambda ,t)dt.$$

(8)

Now, the Pontryagin equations can also be obtained by optimizing the action (8) for its three variables x, u and $\lambda$. If $\delta x,\delta u$, and $\delta \lambda$ denote the corresponding functional variations, by expanding the Hamiltonian in a Taylor series around the optimal solution $x^{*}\left(t\right)$, $u^{*}\left(t\right)$, ${\lambda }^{*}\left(t\right)$ as

$$x\left(t\right)=x^{*}\left(t\right)+\delta x\left(t\right),u\left(t\right)=u^{*}\left(t\right)+\delta u\left(t\right)\lambda \left(t\right)={\lambda }^{*}\left(t\right)+\delta \lambda \left(t\right),$$

(9)

one obtains, after standard manipulations (that is, (1) by keeping only the first-order terms, (2) making an integration by parts and (3) noting that the initial point is fixed, i.e., $\delta x\left(t_0\right)=0$).

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[\left(\frac{\partial H}{\partial \lambda }-\dot{x}\right)\delta \lambda +\left(\frac{\partial H}{\partial x}+\dot{\lambda }\right)\delta x+\frac{\partial H}{\partial u}\delta u\right]dt-\lambda \left(t_1\right)\delta x\left(t_1\right).}$$

(10)

where quantities inside the square bracket in the integral are evaluated at the optimal solution $x^{*}$, $u^{*}$, ${\lambda }^{*}$. To maximize or minimize the action $\delta A$ must vanish, that is, the action must be stationary under arbitrary functional variations

$$\delta A=0.$$

(11)

Now, in optimal control theory, there are two classes of control strategies:

• Open-loop strategies that depend only on time: $u=u\left(t\right)$;
• Closed-loop strategies that depend on the state variable x and time: $u=u(x,t)$ [33].

In the case of an open-loop strategy, $x\left(t\right),u\left(t\right)$, and $\lambda \left(t\right)$ are independent variables, so $\delta x,\delta u$, and $\delta \lambda$ are linearly independent. Hence, Equation (11) implies that coefficients that multiplies the functional variations in (10) must be vanish, and one obtains then the Pontryagin Equation (6) and the transversality condition $\lambda \left(t_1\right)=0$ for the optimal solution

$$x=x^{*}\left(t\right),\lambda ={\lambda }^{*}\left(t\right),u=u^{*}\left(t\right).$$

(12)

What happens, however, with closed-loop strategies? In this case, due to the relations between $u=u(x,t)$ and x, the variations are related by $\delta u=\frac{\partial u}{\partial x}\delta x$. Substituting this into (10) yields

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[\left(\frac{\partial H}{\partial \lambda }-\dot{x}\right)\delta \lambda +\left(\frac{\partial H}{\partial x}+\frac{\partial H}{\partial u}\frac{\partial u}{\partial x}+\dot{\lambda }\right)\delta x\right]dt-\lambda \left(t_1\right)\delta x\left(t_1\right).}$$

(13)

If $\lambda$ and x are independent, the optimization of the functional (11) implies

$$\frac{\partial H}{\partial \lambda }-\dot{x}=0,\frac{\partial H}{\partial x}+\frac{\partial H}{\partial u}\frac{\partial u}{\partial x}+\dot{\lambda }=0,$$

(14)

plus the transversality condition. Note that the optimization process does not give the equation associated to the optimal control in (6). Then, $u=u(x,t)$ must be given explicitly in terms of x, in another case, one has three unknowns but only two equations of motion. Thus, the variational problem is not well-defined for an arbitrary closed-loop strategy $u=u(x,t)$ because the equations of motion can not determine its structure. It must be given from the beginning.

Now, note that equation for the control u in (6) is not a differential equation, but is algebraic:

$$\frac{\partial H}{\partial u}=\frac{\partial F(x,u,t)}{\partial u}+\lambda \frac{\partial f(x,u,t)}{\partial u}=0.$$

(15)

From the above equation $u^{*}$ can be solved (in principle) as

$$u=u^{*}(x,\lambda ,t),$$

(16)

Thus, the optimization problem implies that the optimal open-loop $u^{*}$ strategy (12) is a closed-loop one. How can one understand this contradiction? Could the optimization problem be inconsistent?

To give an answer, consider the following arbitrary (non-necessarily optimal) closed-loop strategy $u=u(x,\lambda ,t)$, so $\delta u=\frac{\partial u}{\partial x}\delta x+\frac{\partial u}{\partial \lambda }\delta \lambda$. After substituting this into (10)

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[\left({\displaystyle \frac{\partial H}{\partial \lambda }+{\displaystyle \frac{\partial H}{\partial u}\frac{\partial u}{\partial \lambda }-\dot{x}}}\right)\delta \lambda +\left(\frac{\partial H}{\partial x}+\frac{\partial H}{\partial u}\frac{\partial u}{\partial x}+\dot{\lambda }\right)\delta x\right]dt-\lambda \left(t_1\right)\delta x\left(t_1\right).}$$

(17)

If $x\left(t\right)$ and $\lambda \left(t\right)$ are independent, then (11) gives the equations

$$\frac{\partial H}{\partial \lambda }+\frac{\partial H}{\partial u}\frac{\partial u}{\partial \lambda }-\dot{x}=0,\frac{\partial H}{\partial x}+\frac{\partial H}{\partial u}\frac{\partial u}{\partial x}+\dot{\lambda }=0.$$

(18)

Clearly the above equations are not equivalent to the Pontryagin open-loop equations. but if one choose for u its optimal value $u^{*}$ defined by $\frac{\partial H}{\partial u}=0$, then Equation (18) becomes

$$\frac{\partial H(x,u^{*}(x,\lambda ,t),\lambda ,t)}{\partial \lambda }-\dot{x}=0,\frac{\partial H(x,u^{*}(x,\lambda ,t),\lambda ,t)}{\partial x}+\dot{\lambda }=0,$$

(19)

which are the Pontryagin equations. The solutions of (19) give optimal paths for the state variable $x=x^{*}\left(t\right)$ and Lagrangian multipliers $\lambda ={\lambda }^{*}\left(t\right)$ respectively. Then, the optimal control open-loop strategy $u^{*}\left(t\right)$ in (12) is obtained from the optimal closed-loop strategy (16) as

$$u=u^{*}\left(t\right)=u^{*}(x^{*}\left(t\right),{\lambda }^{*}\left(t\right),t)$$

(20)

In this way, the optimal closed-loop strategy $u=u^{*}(x,\lambda ,t)$ evaluated over the optimal $x^{*}\left(t\right)$ and ${\lambda }^{*}\left(t\right)$ trajectories is equivalent to the optimal open-loop $u=u^{*}\left(t\right)$ one. So, from now on, we will not distinguish between them in the case of Pontryagin’s theory.

Lastly, consider the case in which $\lambda$ and x are not independent but $\lambda =\lambda (x,t)$, then $\delta \lambda =\frac{\partial \lambda }{\partial x}\delta x$. Replacing it into (17), choosing the optimal control $u=u^{*}$, considering the transversality condition $\lambda \left(t_1\right)=0$ and the fact that $\dot{\lambda }={\displaystyle \frac{\partial \lambda }{\partial x}\dot{x}+\frac{\partial \lambda }{\partial t}}$, one obtains

$$\delta A={\int }_{t_0}^{t_1}\left[\left(\frac{\partial H}{\partial \lambda }\frac{\partial \lambda }{\partial x}-\dot{x}\frac{\partial \lambda }{\partial x}\right)+\left(\frac{\partial H}{\partial x}+\frac{\partial \lambda }{\partial x}\dot{x}+\frac{\partial \lambda }{\partial t}\right)\right]\delta xdt.$$

(21)

From $H=F(x,u^{*},t)+\lambda f(x,u^{*},t)$, it follows that

$$\frac{\partial H}{\partial \lambda }=f(x,u^{*},t)\frac{\partial H}{\partial x}=\frac{\partial F(x,u^{*},t)}{\partial x}+\lambda \frac{\partial f(x,u^{*},t)}{\partial x},$$

and using these equations in (21) finally gives

$$\delta A={\int }_{t_0}^{t_1}\left[\left(f\frac{\partial \lambda }{\partial x}+\frac{\partial F}{\partial x}+\lambda \frac{\partial f}{\partial x}\right)+\frac{\partial \lambda }{\partial t}\right]\delta xdt$$

or

$$\delta A={\int }_{t_0}^{t_1}\left[\frac{d{H}^{*}}{dx}+\frac{\partial \lambda }{\partial t}\right]\delta xdt.$$

where

$$H^{*}=H^{*}(x,t)=H(x,u^{*}(x,\lambda (x,t),t),\lambda (x,t),t)$$

(22)

is the effective Hamiltonian in terms of state variable x. Thus, the optimization process (11) implies that $\lambda =\lambda (x,t)$ satisfies the optimal consistency condition

$$\frac{d{H}^{*}(x,t)}{dx}+\frac{\partial \lambda }{\partial t}=0,$$

(23)

If ${\lambda }^{*}(x,t)$ satisfies (23), the optimal state variable $x\left(t\right)$ is obtained from Equation (2) as

$$\dot{x}=f(x,u^{*}(x,{\lambda }^{*}(x,t),t),t).$$

(24)

Thus, there exist three types of strategies in optimal control theory: open-loop ($x=x\left(t\right),\lambda =\lambda \left(t\right),u=u^{*}\left(t\right)$), inert closed-loop ($x=x\left(t\right),\lambda =\lambda \left(t\right),u=u^{*}(x\left(t\right),\lambda \left(t\right),t)$) and $\lambda$ closed-loop ($x=x\left(t\right),\lambda =\lambda (x\left(t\right),t),u=u^{*}(x\left(t\right),\lambda (x\left(t\right),t),t)$) strategies. The first two are entirely equivalent because these give the same dynamical equations.

2.3. Dynamic Optimization: Bellman theory

A different approach to the optimal control problem is given by the dynamic programming theory developed by Richard Bellman [28] in the 1950s. In this case, the fundamental variable is not the state variable but is the optimal value of the action defined by

$$J(x_0,t_0)=\underset{u}{max}\left({\int }_{t_0}^{t}F(x,u,t)dt\right),$$

(25)

where x satisfies (2). Bellman’s dynamic programming principle implies that $J(x,t)$ satisfies a non-linear partial differential equation called the Hamilton–Jacobi–Bellman equation [17] given by

$$\underset{u}{max}\left(F(x,u,t)+\frac{\partial J(x,t)}{\partial x}f(x,u,t)\right)=-\frac{\partial J(x,t)}{\partial t}.$$

(26)

Note that the Lagrangian multiplier $\lambda$ of the Pontryagin approach can be identified with $\frac{\partial J(x,t)}{\partial x}$. Thus, from the Pontryagin perspective, Bellman’s theory is a model with a closed-loop $\lambda$ strategy $\lambda (x,t)=\frac{\partial J(x,t)}{\partial x}$. Maximizing the left side of (26) for the control variable gives

$$u^{*}=u^{*}(x,t)=u^{*}(x,\frac{\partial J(x,t)}{\partial x},t)=u^{*}(x,\lambda (x,t),t)$$

so the Hamilton–Jacobi–Bellman equation becomes

$$F(x,u^{*},t)+\frac{\partial J(x,t)}{\partial x}f(x,u^{*},t)=-\frac{\partial J(x,t)}{\partial t}.$$

(27)

Differentiating (27) with respect to x, one obtains

$$\begin{array}{c}{\displaystyle \frac{\partial F(x,u^{*},t)}{\partial x}+{\displaystyle \frac{{\partial }^{2}J(x,t)}{\partial x^2}f(x,u^{*},t)+\frac{\partial J(x,t)}{\partial x}\frac{\partial f(x,u^{*},t)}{\partial x}}}\hfill \\ \\ +({\displaystyle \frac{\partial F(x,u^{*},t)}{\partial u^{*}}+{\displaystyle \frac{\partial J(x,t)}{\partial x}\frac{\partial f(x,u^{*},t)}{\partial u^{*}})\frac{d{u}^{*}(x,t)}{dx}}}\hfill \\ \\ =-{\displaystyle \frac{{\partial }^{2}J(x,t)}{\partial x\partial t}.}\hfill \end{array}$$

Noting that

$$\frac{\partial H}{\partial u}=\frac{\partial F(x,u^{*},t)}{\partial u^{*}}+\frac{\partial J(x,t)}{\partial x}\frac{\partial f(x,u^{*},t)}{\partial u^{*}}=0,$$

and replacing $\frac{\partial J(x,t)}{\partial x}$ by $\lambda (x,t)$, one arrives to

$${\displaystyle \frac{\partial F(x,u^{*},t)}{\partial x}+\frac{\partial \lambda (x,t)}{\partial x}f(x,u^{*},t)+\lambda \frac{\partial f(x,u^{*},t)}{\partial x}=-{\displaystyle \frac{\partial \lambda (x,t)}{\partial t},}}$$

or

$$\frac{d{H}^{*}(x,t)}{dx}+\frac{\partial \lambda (x,t)}{\partial t}=0.$$

The last equation is the same equation (23). Thus, the previous analysis implies that the consistency condition (23) is just the derivative of the Hamilton–Jacobi–Bellman equation. Equation (23) is then according to (27) equal to

$$\frac{d}{dx}\left(F(x,u^{*},t)+\frac{\partial J(x,t)}{\partial x}f(x,u^{*},t)+\frac{\partial J(x,t)}{\partial t}\right)=0.$$

(28)

By integrating the last equation one obtains

$$F(x,u^{*},t)+\frac{\partial J(x,t)}{\partial x}f(x,u^{*},t)+\frac{\partial J(x,t)}{\partial t}=g\left(t\right),$$

where $g\left(t\right)$ is a time-dependent arbitrary function. Then, the Pontryagin approach gives for a closed-loop ${\lambda }^{*}(x,t)=\frac{\partial J(x,t)}{\partial x}$ a non-homogeneous Hamilton–Jacobi–Bellman equation, whereas Bellman’s maximum principle gives instead a homogeneous Hamilton–Jacobi–Bellman equation.

3. Classical Mechanics

In this section we review the Lagrangian and Hamiltonian approaches to classical mechanics and the analysis of constrained systems in physics.

3.1. Lagrangian Mechanics

It is well-known in classical mechanics that for a conservative system, that is, the one for which the force is the gradient of a potential energy $U\left(\overrightarrow{r}\right)$

$$\overrightarrow{F}=-\nabla U\left(\overrightarrow{r}\right),$$

(29)

the Newton equations

$$\overrightarrow{F}=m\overrightarrow{a}$$

(30)

can be obtained through an optimization process similar to an optimization dynamic economic problem. For simplicity, in the case of a one-dimensional non-relativistic system given by the x coordinate, the objective functional is the Action functional $A[x,\dot{x}]$ defined by

$$A[x,\dot{x}]={\int }_{t_0}^{t_1}L(x,\dot{x})dt$$

(31)

where $L(x,\dot{x})=\frac{1}{2}m{\dot{x}}^2-U\left(x\right)$ is the Lagrangian function. The action (31) optimization for x gives the well-known Euler–Lagrange equation

$$\frac{\partial L}{\partial x}-\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{x}}\right)=0$$

(32)

that, of course, is the Newton equation for the x variable:

$$\frac{\partial L}{\partial \dot{x}}=m\dot{x},\frac{d}{dt}\left(\frac{\partial L}{\partial \dot{x}}\right)=\ddot{x}=a,\frac{\partial L}{\partial x}=-\frac{\partial U}{\partial x}=F_x$$

(33)

so Equation (32) is just $F_x=m{a}_x$. Note that the Lagrangian approach gives a second-order differential equation for the coordinate x.

3.2. Hamiltonian Mechanics

Instead of using the space $(x,\dot{x})$ associated with the Lagrangian formalism, one can take the phase-space $(x,p_x)$, where the momentum variable $p_x$ is defined by

$$p_x=\frac{\partial L(x,\dot{x})}{\partial \dot{x}}.$$

(34)

The last equation can be used to solve $\dot{x}$ in terms of $p_x$ as $\dot{x}=\dot{x}\left(p_x\right)$. In this phase-space $(x,p_x)$, the objective functional is the Hamiltonian Action defined by

$$A[x,p_x]={\int }_{t_0}^{t_1}{p}_x\dot{x}-H(x,p_x)dt,$$

(35)

where the Hamiltonian function $H(x,p_x)$ is constructed from the Lagrangian $L(x,\dot{x})$ as

$$H(x,p_x)=p_x\dot{x}\left(p_x\right)-L(x,\dot{x}\left(p_x\right)).$$

(36)

The equations of motion in the phase-space $(x,p_x)$ are obtained by the optimization of the Action functional (35) for x and $p_x$ as independent variables. In this case, by the same standard techniques used in (10) one has that

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[\left(\frac{\partial H}{\partial p_x}-\dot{x}\right)\delta p_x+\left(\frac{\partial H}{\partial x}+\dot{p_x}\right)\delta x\right]dt+\left[p_x\left(t_2\right)\delta x\left(t_2\right)-p_x\left(t_1\right)\delta x\left(t_1\right)\right]}$$

(37)

To obtain the optimal phase-space trajectory one need that

$$\delta A=0,$$

(38)

so all the terms in $\delta x$ and $\delta p_x$ must vanish. For this, it is supposed that:

• The end points of $x\left(t\right)$ are fixed, that is, $\delta x\left(t_1\right)=0$ and $\delta x\left(t_0\right)=0$;
• $\delta x$ and $\delta p_x$ are linearly independent.

Therefore, the Hamiltonian equations of motion are obtained from (38) as:

$$\dot{x}=\frac{\partial H(x,p_x)}{\partial p_x},{\dot{p}}_x=-\frac{\partial H(x,p_x)}{\partial x}.$$

(39)

One must recall that Hamilton equations are first-order differential equations, one for each independent x and $p_x$ variable. Both first-order equations are equivalent to the single second-order Euler–Lagrange Equation (32).

Now, by introducing the Poisson brackets between phase-space functions $A(x,p_x)$ and $B(x,p_x)$ according to

$$\{A,B\}=\left(\frac{\partial A}{\partial x}\frac{\partial B}{\partial p_x}-\frac{\partial B}{\partial x}\frac{\partial A}{\partial p_x}\right)$$

(40)

the Hamilton equations of motion read as

$$\dot{x}=\{x,H\}{\dot{p}}_x=\{p_x,H\}.$$

(41)

Note that the dynamical evolution of any variable $\theta$ defined on phase-space will given by Hamiltonian-type equation

$$\dot{\theta }=\{\theta ,H\}$$

(42)

3.3. Constrained Systems in Hamiltonian Mechanics

In the usual cases, the velocities $\dot{x}$ can be solved in terms of $p_x$ from Equation (34). However, there are situations (the most interesting ones) where this cannot be carried out. For example, for the Lagrangian

$$L(x,\dot{x})=a\dot{x}+b.$$

(43)

one has that

$$p_x=\frac{\partial L}{\partial \dot{x}}=a.$$

(44)

The class of systems for which the velocities cannot be solved regarding the momentum are called constrained systems. In this case, the Lagrangian and Hamiltonian theories can be different. Paul Maurice Dirac, a British physicist and Nobel winner, developed a methodology for studying general constrained systems in the phase-space. This method is now called Dirac’s method [34,35,36,37]. One must recall that constrained systems are ubiquitous in physics because all important physical theories, such as Electromagnetism, Einstein’s General Relativity, and the Gauge theories of particle physics, are examples of this category.

In the following subsection, Dirac’s method will be used to understand control theory from a physicist’s point of view.

3.4. Optimal Control Theory as an Example of a Constrained System

Recently, studies have shown how a generic control problem can be reinterpreted as a second-class constrained physical system [22,23,24,25,26]. In [22], it is found that at the classical level, the optimal control problem (1) and (2) when writing in terms of the state variable x, control variable u and the Lagrangian multiplier $\lambda$ as (see Equations (7) and (8))

$$A[x,u,\lambda ]={\int }_{t_0}^{t_1}-\lambda \dot{x}+H(x,u,\lambda ,t)dt={\int }_{t_0}^{t_1}-\lambda \dot{x}+F(x,u,t)+\lambda f(x,u,t)dt$$

(45)

can be mapped to the Hamiltonian Action of a fictitious particle. In fact, by comparing Equation (45) with (35), one notes that the Lagrangian multiplier $\lambda$ can be identified with momentum $p_x$, so the functional (45) can be written as

$$A[x,u,p_x]=-{\int }_{t_0}^{t_1}{p}_x\dot{x}-\left(F(x,u,t)+p_{x}f(x,u,t)\right)dt$$

(46)

The above action is just a Hamiltonian action modulo a minus sign, so that one can use the action

$$A[x,u,p_x]={\int }_{t_0}^{t_1}{p}_x\dot{x}-\left(F(x,u,t)+p_{x}f(x,u,t)\right)dt$$

(47)

The Lagrangian associated with Equation (47) is

$$L=p_x\dot{x}-\left(F(x,u,t)+p_{x}f(x,u,t)\right)$$

(48)

One must note that L does not depend on $\dot{u}$, so its momentum $p_u$ associated with the u coordinate becomes

$$p_u=\frac{\partial L}{\partial \dot{u}}=0$$

(49)

Equation (49) is then a constraint in the phase-space $(x,u,p_x,p_u)$ (one cannot solve $\dot{u}$ in terms of $p_u$ from the above equation), and the theory of optimal control thus becomes a constrained system. We denote the constraint in (49) as

$${\mathsf{\Phi }}_1=p_u=0$$

(50)

These constraints are called primary constraints according to Dirac’s method. The corresponding Hamiltonian in the phase-space $(x,u,p_x,p_u)$ is then

$$H\left(x,u,p_x,p_u\right)=p_x\dot{x}+p_u\dot{u}-L=p_x\dot{x}+p_u\dot{u}+\left(F(x,u,t)+p_{x}f(x,u,t)\right)-p_x\dot{x}$$

(51)

so

$$H\left(x,u,p_x,p_u,t\right)={\mathsf{\Phi }}_1\dot{u}+F(x,u,t)+p_{x}f(x,u,t)$$

(52)

or

$$H\left(x,u,p_x,p_u,t\right)={\mathsf{\Phi }}_1\dot{u}+H_0\left(x,u,p_x,p_u,t\right)$$

(53)

with

$$H_0\left(x,u,p_x,p_u,t\right)=F(x,u,t)+p_{x}f(x,u,t).$$

(54)

On the constraint surface ${\mathsf{\Phi }}_1=0$, the Hamiltonian is then

$$H\left(x,u,p_x,p_u,t\right)=H_0\left(x,u,p_x,p_u,t\right)$$

(55)

Thus, $H_0$ is the energy of the system. The constraint (49) would be incorporated in the model by the Lagrange multipliers method, so one should consider instead a modified Hamiltonian defined by

$$\tilde{H}\left(x,u,p_x,p_u,t\right)=H_0+{\mu }_1{\mathsf{\Phi }}_1=H_0+{\mu }_1{p}_u$$

(56)

where ${\mu }_1$ is a Lagrange multiplier. Now, one can impose that the constraint ${\mathsf{\Phi }}_1$ be preserved in time with the modified Hamiltonian (56), so

$${\dot{\mathsf{\Phi }}}_1=\left\{{\mathsf{\Phi }}_1,\tilde{H}\right\}=0,$$

(57)

where $\{A,B\}$ denotes the Poison bracket in the $\left(x,u,p_x,p_u\right)$ phase-space

$$\{A,B\}=\left(\frac{\partial A}{\partial x}\frac{\partial B}{\partial p_x}-\frac{\partial B}{\partial x}\frac{\partial A}{\partial p_x}\right)+\left(\frac{\partial A}{\partial u}\frac{\partial B}{\partial p_u}-\frac{\partial B}{\partial u}\frac{\partial A}{\partial p_u}\right)$$

(58)

Equation (57) yields

$${\dot{\mathsf{\Phi }}}_1=\left\{p_u,\tilde{H}\right\}=-\frac{\partial \tilde{H}}{\partial u}=-\frac{\partial H_0}{\partial u}=\frac{\partial F(x,u,t)}{\partial u}+p_x\frac{\partial f(x,u,t)}{\partial u}=0.$$

(59)

From the last equation, one cannot solve any velocity $\dot{x}$ or $\dot{u}$ in terms of momentum $p_x$ or $p_u$. Thus, (57) is a new secondary constraint and is just the optimal law for the control variable. Then, from a phase-space analysis, the third Pontryagin equation in (6) is just the secondary constraint

$${\mathsf{\Phi }}_2=\frac{\partial H_0}{\partial u}=\frac{\partial F(x,u,t)}{\partial u}+p_x\frac{\partial f(x,u,t)}{\partial u}=0.$$

(60)

To add the effect of the secondary constraint, one again uses the Lagrange multipliers method and considers a new extended Hamiltonian

$${\tilde{H}}_2\left(x,u,p_x,p_u\right)=H_0+{\mu }_1{\mathsf{\Phi }}_1+{\mu }_2{\mathsf{\Phi }}_2.$$

(61)

According to Dirac’s method, one must impose time preservation for both constraints ${\mathsf{\Phi }}_1$ and ${\mathsf{\Phi }}_2$ with the extended Hamiltonian ${\tilde{H}}_2$:

$$\begin{array}{cc}\hfill & {\dot{\mathsf{\Phi }}}_1=\left\{{\mathsf{\Phi }}_1,{\tilde{H}}_2\right\}=0,\hfill \\ \hfill & {\dot{\mathsf{\Phi }}}_2=\left\{{\mathsf{\Phi }}_2,{\tilde{H}}_2\right\}=0,\hfill \end{array}$$

(62)

that is

$$\begin{array}{cc}& \left\{{\mathsf{\Phi }}_1,H_0\right\}+{\mu }_2\left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}=0\hfill \\ & \left\{{\mathsf{\Phi }}_2,H_0\right\}+{\mu }_1\left\{{\mathsf{\Phi }}_2,{\mathsf{\Phi }}_1\right\}=0,\hfill \end{array}$$

The above system can be written as

$$\left(\begin{array}{c}\left\{{\mathsf{\Phi }}_1,H_0\right\}\hfill \\ \left\{{\mathsf{\Phi }}_2,H_0\right\}\hfill \end{array}\right)+\left(\begin{array}{cc}0& \left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}\\ -\left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}& 0\end{array}\right)\left(\begin{array}{c}{\mu }_1\hfill \\ {\mu }_2\hfill \end{array}\right)=\left(\begin{array}{c}0\hfill \\ 0\hfill \end{array}\right).$$

The matrix

$$\mathsf{\Delta }=\left(\begin{array}{cc}0& \left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}\\ -\left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}& 0\end{array}\right)$$

is called Dirac matrix. If the Dirac matrix is invertible, the above equation system can be solved for ${\mu }_1$ and ${\mu }_2$, and in this situation, no new constraint appears in the theory.

Now

$$\left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}=\left\{p_u,{\mathsf{\Phi }}_2\right\}=\frac{\partial {\mathsf{\Phi }}_2}{\partial u}=\frac{{\partial }^2{H}_0}{\partial u^2}$$

so

$$\mathsf{\Delta }=\left(\begin{array}{cc}0& \frac{{\partial }^2{H}_0}{\partial u^2}\\ -\frac{{\partial }^2{H}_0}{\partial u^2}& 0\end{array}\right).$$

If

$$\begin{array}{c}\hfill \frac{{\partial }^2{H}_0}{\partial u^2}\ne 0\end{array}$$

(63)

then $\mathsf{\Delta }$ matrix is invertible, and no new constraint appears. In this case, the constraint set $\left\{{\mathsf{\Phi }}_1,{\mathsf{\Phi }}_2\right\}$ is termed a second-class one (see [34,35,36,37] for details). Note that for a large class of economic systems, such as the linear quadratic ones, the condition (63) is fulfilled.

In summary, from a phase-space optics, the optimal control theory corresponds to a second-class constrained system, defined by the Hamiltonian $H_0=F(x,u,t)+p_{x}f(x,u,t)$ and the two second-class constraints ${\mathsf{\Phi }}_1=p_u=0$ and ${\mathsf{\Phi }}_2=\frac{\partial H_0}{\partial u}=\frac{\partial F(x,u,t)}{\partial u}+p_x\frac{\partial f(x,u,t)}{\partial u}=0$.

Now, one can go further by trying to quantize this classical system by imposing canonical commutations relations for the phase-space variables $(x,u,p_x,p_u)$:

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \mathbb{I}[\widehat{u},{\widehat{p}}_u]=i\hslash \mathbb{I}$$

(64)

and where any other commutator is zero. Thus, the positions x, u and the momentum variables $p_x$, $p_u$ become operators, which can be represented in a function space (Hilbert space) as

$$\begin{array}{cc}\hfill \widehat{x}=x& {\widehat{p}}_x=-i\hslash \frac{\partial }{\partial x}\hfill \\ \hfill \widehat{u}=u& {\widehat{p}}_u=-i\hslash \frac{\partial }{\partial u}\hfill \end{array}$$

(65)

The time-dependent Schrödinger equation associated to the Hamiltonian $H_0$ is

$${\widehat{H}}_0\left(\widehat{x},\widehat{u},{\widehat{p}}_x,{\widehat{p}}_u\right)\mathsf{\Psi }(x,u,t)=i\hslash \frac{\partial }{\partial t}\mathsf{\Psi }(x,u,t)$$

or

$$\left(F(x,u,t)-i\hslash f(x,u,t)\frac{\partial }{\partial x}\right)\mathsf{\Psi }(x,u,t)=i\hslash \frac{\partial \mathsf{\Psi }(x,u,t)}{\partial t},$$

(66)

Note that for the quantization process, a right-side operator order was chosen for the momentum operator.

Following Dirac, the only physically admissible solutions ${\mathsf{\Psi }}_P$ of the Schrödinger Equation (66) satisfy the constraints ${\mathsf{\Phi }}_1=p_u=0$ and ${\mathsf{\Phi }}_2=\frac{\partial H_0}{\partial u}=\frac{\partial F(x,u,t)}{\partial u}+p_x\frac{\partial f(x,u,t)}{\partial u}=0$ at quantum level, that is, as operator equations acting on the wave function ${\mathsf{\Psi }}_P$

$$\begin{array}{cc}& {\widehat{\mathsf{\Phi }}}_1{\mathsf{\Psi }}_P=0,\hfill \\ & {\widehat{\mathsf{\Phi }}}_2{\mathsf{\Psi }}_P=0.\hfill \end{array}$$

or explicitly

$$-i\hslash \frac{\partial }{\partial u}{\mathsf{\Psi }}_P=0$$

(67)

$$\left(\frac{\partial F(x,u,t)}{\partial u}-i\hslash \frac{\partial f(x,u,t)}{\partial u}\frac{\partial }{\partial x}\right){\mathsf{\Psi }}_P=0$$

(68)

and also the Schrödinger equation

$$\left(F(x,u,t)-i\hslash f(x,u,t)\frac{\partial }{\partial x}\right){\mathsf{\Psi }}_P=i\hslash \frac{\partial }{\partial t}{\mathsf{\Psi }}_P.$$

(69)

Equation (67) implies that ${\mathsf{\Psi }}_P$ does not depend on u, so ${\mathsf{\Psi }}_P={\mathsf{\Psi }}_P(x,t)$. Now, if one write the wave function in the form

$${\mathsf{\Psi }}_P(x,t)=e^{\frac{i}{\hslash }S(x,t)}$$

the quantum Equations (68) and (69) becomes the following equations for $S(x,t)$

$$\frac{\partial F(x,u,t)}{\partial u}+\frac{\partial f(x,u,t)}{\partial u}\frac{\partial S(x,t)}{\partial x}=0,$$

(70)

$$F(x,u,t)+f(x,u,t)\frac{\partial S(x,t)}{\partial x}=-\frac{\partial S(x,t)}{\partial t}$$

(71)

Note that the above two equations are ℏ-independent, due to the linear character of the Hamiltonian operator $H_0$ and the quantum constraint (68), in the momentum operator ${\widehat{p}}_x=-i\hslash \frac{\partial }{\partial x}$. Also, Equations (70) and (71) are equivalent to a single Hamilton–Jacobi–Bellman Equation (26)

$$\underset{u}{max}\left(F(x,u,t)+f(x,u,t)\frac{\partial S(x,t)}{\partial x}\right)=-\frac{\partial S(x,t)}{\partial t},$$

if the $S(x,t)$ is identified with the optimal value function $J(x,t)$ [28]. Thus, the Bellman approach to the optimal control problem is the right quantization of the Pontryagin theory. For more details about this topic, the elimination of the second-class constraints at the classical level, different quantization schemes, and issues related to the quantization process, see [22,25].

4. Open/Closed-Loop Strategies and Physics

The past two sections deal with control theory and the physicist’s view of it as a second-class constrained system in the phase-space. So after this digression, we enter the principal objective of this paper, that is, to invert the optics of the problem: instead of seeing the control theory as a physical system, one can consider the classical and quantum physical systems from the point of view of control theory. We will start with an analysis of the classical theory.

4.1. Open/Closed-Loop Strategies and Classical Mechanics

From an economic or dynamic optimization point of view, the problem of optimizing the Action (35) is analogous to an optimal control problem but without the control variable u. As seen in the previous section, the solutions to the control problem are given by the Pontryagin Equation (6) plus the transversality condition. Note that the first two Pontryagin equations in (6) are precisely the Hamiltonian equations of motion (39) if one identifies the Lagrangian multiplier $\lambda$ with the canonical momentum $p_x$. Because Pontryagin theory has open-loop strategies ($x\left(t\right)$ and $\lambda \left(t\right))$ are independent), the Hamiltonian theory can then be considered as an open-loop model, similar to Pontryagin’s theory.

One may ask: can closed-loop strategies occur in physical systems as in economic systems?

As is shown in this paper, the answer is affirmative, and they appear naturally in the context of canonical transformations. In order to give a first clue to the answer to the above question, suppose that one imposes a constraint over the phase-space of the form

$$\mathsf{\Phi }(x,p_x,t)=0.$$

(72)

This constraint represents at each time a surface in the phase-space (a line in our bi-dimensional case) where the system can evolve (see Figure 1).

Actually, one can write p in terms of x from (72) by solving the constraint:

$$\begin{array}{c}\hfill p_x=p_x(x,t).\end{array}$$

(73)

Using the analogy of the momentum p with the Lagrangian multiplier $\lambda$, Equation (73) corresponds to the closed-loop $\lambda$ strategy from an economic point of view. Thus, x and p are not independent in this case, and their variations are related by $\delta p_x=\frac{\partial p_x}{\partial x}\delta x$. By replacing in (37) and using the fact that the endpoints of x are fixed, one arrives at

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[-\frac{\partial p_x}{\partial t}-\frac{\partial H}{\partial x}-\frac{\partial H}{\partial p_x}\frac{\partial p_x}{\partial x}\right]\delta xdt.}$$

(74)

By defining the reduced Hamiltonian $H^{*}(x,t)=H(x,p_x(x,t),t)$, Equation (74) can be written as

$$\delta A={\displaystyle {\int }_{t_0}^{t_1}\left[-\frac{\partial p_x}{\partial t}-\frac{\partial H^{*}}{\partial x}\right]\delta xdt.}$$

(75)

The optimization of the action then gives

$$-\frac{\partial p_x}{\partial t}-\frac{\partial H^{*}}{\partial x}=0.$$

(76)

This last equation is a consistency condition that the closed-loop strategy (73) must satisfy, to give an extremal of the action. That is, if $p_x=p_x(x,t)$ satisfies (76), then $p_x=p_x(x,t)$ is an optimal closed-loop strategy.

Now, if the closed-loop momentum strategy $p_x=p_x(x,t)$ is just the derivative of some function $S(x,t)$, such as

$$\begin{array}{c}\hfill p_x(x,t)=\frac{\partial S(x,t)}{\partial x},\end{array}$$

(77)

condition (76) gives

$$\frac{\partial }{\partial x}\left[-\frac{\partial S(x,t)}{\partial t}-H^{*}(x,t)\right]=0.$$

(78)

So, one obtains, by integration, that

$$-\frac{\partial S(x,t)}{\partial t}-H(x,\frac{\partial S(x,t)}{\partial x})=g\left(t\right)$$

(79)

for some function g of time. The above equation is just an inhomogeneous Hamilton–Jacobi Equation. Thus, the derivative of the Hamilton–Jacobi Equation can be seen as the consistency condition to give the Action an extremal in the closed-loop $p_x$ strategy case. Also, this small analysis implies that closed-loop momentum strategies are closely related to the Hamilton–Jacobi Equation.

Now, it is well known that the Hamilton–Jacobi Equation appears in classical mechanics in the context of the canonical transformations, but how and where do the closed-loop $p_x=p_x(x,t)$ strategies appear there? In the next section, a short review of canonical transformations will be given, and it will elucidate how the closed-loop strategies appear in that context.

4.2. Canonical Transformations and Closed-Loop Strategies

The Hamilton–Jacobi Equation can be considered as a bridge between nature’s classical and quantum descriptions. The Hamilton–Jacobi Equation is obtained from the quantum sector, taking the limit $\hslash \to 0$ of a particular form of the Schrödinger equation called the quantum Hamilton–Jacobi Equation (see Section 5 for details). Instead, from the pure classical mechanical description, the Hamilton–Jacobi Equation can be obtained using the canonical transformation framework. Canonical transformations are coordinate transformations in the Hamiltonian mechanics’ phase-space $(x,p_x)$. Thus, two observers are present in this framework, each associated with a coordinate system. The first one is the standard Cartesian system. But, the second one is special: it is constructed with the trajectories that are solutions of the same Hamiltonian equations, i.e., the solutions of the Newton equations. Thus, one hopes from the point of view of the second observer that the solution of the equation of motion is described by constant values.

To understand how the canonical transformations method works, consider a general phase-space coordinate transformation of the form

$$Q=Q(x,p_x)P=P(x,p_x).$$

(80)

The transformation is called canonical if the Hamiltonian equations of motions are invariant under (80), that is, if

$$\dot{Q}=\frac{\partial \tilde{H}}{\partial P},\dot{P}=-\frac{\partial \tilde{H}}{\partial Q},$$

(81)

where

$$\tilde{H}(P,Q,t)=H(x,p_x,t)+\frac{\partial F(x,Q,t)}{\partial t}.$$

(82)

The function F is called the generator of the canonical transformation, and the coordinate transformation (80) can be reconstructed from F through Equations [38,39]

$$p_x=\frac{\partial F(x,Q,t)}{\partial x},-P=\frac{\partial F(x,Q,t)}{\partial Q}.$$

(83)

One must note at this point that the Hamiltonian Equation (39) refers to a unique coordinate system (in this case, a Cartesian coordinate system). So, the Hamilton equations in (39) are “single observer” equations. Instead, the canonical transformation brings a new second observer into the problem, because one has two different coordinate systems: the initial Cartesian $(x,p_x)$ and the second one $(Q,P)$. So, the theory of canonical transformations is a “two observers” view of classical mechanics. This characteristic induces the closed-loop $p_x$-strategies from a purely classical point of view (closed-loop $p_x$-strategies can also be induced from quantum mechanics to the classical realm, as we shall see later).

The Hamilton–Jacobi theory relies on considerable freedom in choosing $F(x,Q,t)$. This theory does not work directly with F, but with its Legendre transformation S defined by

$$S(x,P,t)=F(x,Q,t)+PQ$$

(84)

In this case, the canonical transformation is reconstructed via the equations

$$p_x=\frac{\partial S(x,P,t)}{\partial x}$$

(85)

$$Q=\frac{\partial S(x,P,t)}{\partial P}$$

(86)

and the respective Hamiltonians are related by

$$\tilde{H}(P,Q,t)=H(x,p_x,t)+\frac{\partial S(x,P,t)}{\partial t}$$

(87)

Note again that one has huge freedom in choosing S. The Hamilton–Jacobi theory corresponds to the choice of S that makes the second-observer Hamiltonian $\tilde{H}(Q,P)$ equal zero:

$$\tilde{H}(P,Q,t)=0.$$

(88)

So, the equations of motion for the second observer are

$$\dot{Q}=0,\dot{P}=0.$$

(89)

Thus, for the second observer, the dynamical variable remains constant in time:

$$Q\left(t\right)=Q_0,P\left(t\right)=P_0.$$

(90)

Note that this is possible only if the second coordinate system $(P,Q)$ corresponds to the trajectories of the solution of the Hamilton equations. Only in this way can the solutions of the equations of motion have constant values. However, what does the first observer see? First, due to Equation (90), the coordinate transformations (80) give

$$Q_0=Q(x,p,t),P_0=P(x,p,t),$$

(91)

but each of these equations defines constant $(Q,P)$ coordinate lines. These are constraints over the phase-space $(p_x,x)$ of the first observer, from which one can generate two different closed-loop $p_x$-strategies according to Equation (72) (See Figure 2).

Thus, the “two observers” perspective of classical mechanics, through the method of canonical transformation, is responsible for the generation of the closed-loop $p_x$-strategies. From (91), it is not clear if the $p_x$ closed-loops strategies thus generated satisfy the consistency condition (76) or if they satisfy the second condition (77) to obtain a Hamilton–Jacobi Equation as in (79) for S. Instead of using (91), one can see these constant coordinate lines in terms of Equations (85) and (86). These equations are equivalent to (91), because the canonical transformation can be reconstructed from (85) and (86).

A constant $P_0$-line in (91) is equivalent (from (85)) to

$$p_x=\frac{\partial S(x,P_0,t)}{\partial x},$$

(92)

thus, the constant $P_0$-line in (91) satisfies (77). From (87) and (88), $S(x,t)$ satisfies the Hamilton–Jacobi Equation as in (79) with $g\left(t\right)=0$. This implies that a closed-loop strategy generated by (92) satisfies the consistency relation (76), so (92) defines a true optimal closed-loop $p_x$ strategy.

For the $Q_0$ constant coordinate line in (91), one has, however, due to (86), that

$$Q_0=\frac{\partial S(x,P,t)}{\partial P},$$

(93)

but, from this equation, one cannot obtain $p_x$ in terms of x, so this line does not generate a $p_x$ closed-loop strategy at all. This is due to the structure of Equations (85) and (86). In order to reconstruct the canonical transformation (80), it is necessary to invert the system (85) and (86). Note that only from (85) can the momentum P be written in terms of the first-observer variables as $P=P(x,p_x,t)$. But, from (86) alone, one cannot solve Q in terms of the $x,p_x$. The other Equation (85) is needed to do that. Thus, a constant $Q_0$-line alone can not generate a true closed-loop $p_x=p_x(x,t)$ strategy.

In this way, closed-loop $p_x$ strategies appear in classical mechanics due to the two observers’ interpretation of the canonical transformation theory. These closed-loop strategies are inert, similarly to how the optimal $u^{*}$ closed-loop ones are inert in control theory. That is because both closed-loop approaches $p_x$ and $u^{*}$ give the same dynamical equation of the open-loop case. For the closed-loop $p_x$ case, the open-loop dynamics (analogous to those given by Pontryagin’s equations) are provided by the Hamiltonian equations of motion of the first observer in the $(x,p_x)$ phase-space.

4.3. Quantum Mechanics and Closed-Loop Strategies

In this section, the origins of the Hamilton–Jacobi–Bellman equation that appears in the limit $\hslash \to 0$ in the quantum phenomena will be explained as a consequence of the emergence of closed-loop $p_x$-strategies in the quantum world.

Consider the Schrödinger equation for a non-relativistic particle of mass m:

$$-\frac{{\hslash }^2}{2m}\frac{{\partial }^2\mathsf{\Psi }(x,t)}{\partial x^2}+U\left(x\right)\mathsf{\Psi }(x,t)=i\hslash \frac{\partial \mathsf{\Psi }(x,t)}{\partial t}.$$

(94)

Writing the wave function in the form

$$\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)},$$

(95)

and by substituting (95) into the Schrödinger equation, the following equation, called the quantum Hamilton–Jacobi Equation for $S(x,t)$, is obtained:

$$\frac{1}{2m}{\left(\frac{\partial S(x,t)}{\partial x}\right)}^2+U\left(x\right)-\frac{i\hslash }{2m}\frac{{\partial }^{2}S(x,t)}{\partial x^2}=-\frac{\partial S(x,t)}{\partial t}.$$

(96)

Note that this equation is completely equivalent to Schrödinger’s equation, but here, the classical and quantum realms can be clearly identified. In fact, by taking the limit $\hslash \to 0$ in (96), one obtains

$$\frac{1}{2m}{\left(\frac{\partial S(x,t)}{\partial x}\right)}^2+U\left(x\right)=-\frac{\partial S(x,t)}{\partial t}.$$

(97)

Equation (97) is just the classical Hamilton–Jacobi Equation

$$H(x,\frac{\partial S(x,t)}{\partial x})=-\frac{\partial S(x,t)}{\partial t},$$

(98)

associated with the classical Hamiltonian function of the non-relativistic particle

$$H(x,p_x)=\frac{p_x^2}{2m}+U\left(x\right),$$

(99)

where one must identify $p_x$ with the derivative of S

$$p_x=p_x(x,t)=\frac{\partial S(x,t)}{\partial x}.$$

(100)

to make contact with the classical Hamiltonian theory. And, it is precisely this identification which generates the closed-loop $p_x$ strategy through (100). Note that it is induced from the quantum realm to the classical world, in the limit $\hslash \to 0$, of the quantum Hamilton–Jacobi Equation. The identification in (100) thus comes from a pure quantum description. Consider the momentum operator

$${\widehat{p}}_x=-i\hslash \frac{\partial }{\partial x}.$$

(101)

This operator is characterized by its eigenfunctions and eigenvalues:

$${\widehat{p}}_x{\mathsf{\Phi }}_{p_x}\left(x\right)=p_x{\mathsf{\Phi }}_{p_x}\left(x\right).$$

(102)

where the solution of this equation gives

$${\mathsf{\Phi }}_{p_x}\left(x\right)=e^{\frac{i}{\hslash }{p}_{x}x}.$$

(103)

In this context, $p_x$ and x are independent variables, and the eigenfunction (103) corresponds to states with well-defined values of the momentum.

Note now that if one applies the momentum operator to a generic wave function $\mathsf{\Psi }$, which is a solution of the Schrödinger equation (written in the “momentum form” (95)), one obtains

$${\widehat{p}}_x\mathsf{\Psi }(x,t)=\frac{\partial S(x,t)}{\partial x}\mathsf{\Psi }(x,t).$$

(104)

By looking at the wave function as a vector with a continuous index x, the above equation implies that (locally at each point x) the momentum operator is diagonal so that any wave function can be seen as an eigenstate of the momentum operator with momentum eigenvalue $\frac{\partial S(x,t)}{\partial x}$. Thus, one must identify the momentum eigenvalue $p_x$ in this quantum state with the derivative of the S function through (100). This identification generates the closed-loop $p_x$-strategies directly in the quantum world.

On the other hand, the same Heisenberg canonical commutation relations

$$[\widehat{x},{\widehat{P}}_x]=\widehat{x}{\widehat{P}}_x-{\widehat{P}}_x\widehat{x}=i\hslash \widehat{I}.$$

(105)

can be seen as a constraint in the non-commutative phase-space $(\widehat{x},{\widehat{P}}_x)$. Thus, from (105) one could “solve” the momentum operator ${\widehat{P}}_x$ in terms of the $\widehat{x}$ operator. This necessarily implies the existence of a certain relation between ${\widehat{P}}_x$ and $\widehat{x}$ or between their eigenvalues. The representation of the canonical operator as a differential operator acting on a function space or Hilbert space as

$$\widehat{x}\to x,{\widehat{p}}_x\to -i\hslash \frac{\partial }{\partial x}$$

(106)

is equivalent to solving the constraint (105), because on any wave function $\mathsf{\Psi }(x,t)$, Equation (105) is satisfied identically. The memory of the quantum constraint (105) is then transferred locally to the momentum eigenvalue, according to (104). In a sense, the representation of the wave function as $\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)}$ locally diagonalizes the momentum operator ${\widehat{p}}_x$ over any quantum state, and (96) is just the Schrödinger equation in this diagonal basis. Note that all of this is a kinematic effect created by the Heisenberg commutation relation (105); the dynamical effects appear when the explicit form of $S(x,t)$ is needed, and for that, one must solve the full quantum Hamilton–Jacobi Equation (96) explicitly. Note that quantum mechanics, as in (105), can be viewed as a constrained system in a non-commutative space, so one would apply a generalization of Dirac’s method [34,35,36,37] to non-commutative spaces [40] to study quantum mechanical systems.

We can say that closed-loop $p_x$-strategies correspond to a pure quantum phenomenon and are a consequence of Heisenberg’s uncertainty principle. In an arbitrary quantum state, momentum and position cannot be independent; they are related through the non-commutative character of the position and momentum operators defined by the canonical commutation relation (105). A less-defined position state would emerge in a more-defined momentum state. Thus, these two variables must depend on one another in some way. Relation (100) is tantamount to a conversation between them. Only in a pure-momentum state, as given in (103), the link disappears and position and momentum become independent variables.

In fact, in a pure-momentum state, $p_x(x,t)=p_x^0$ is constant, that is, all of the eigenvalues are the same, so Equation (100) gives

$$\begin{array}{c}\hfill S(x,t)=p_x^{0}x+\varphi \left(t\right)\end{array}$$

(107)

as a solution, where $\varphi \left(t\right)$ is some function of time. Thus, the wave function is

$$\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)}=e^{\frac{i}{\hslash }(p_x^{0}x+\varphi \left(t\right))}=e^{\frac{i}{\hslash }\varphi \left(t\right)}{\mathsf{\Phi }}_{p_x^0}\left(x\right),$$

(108)

which is the same momentum eigenstate amplified by a temporal arbitrary phase. Then, the linear character of $S(x,t)$ in terms of x implies that $p_x$ and x are independent variables, and no closed-loop $p_x$ strategy exists in this case. The same can be said for a pure-position eigenstate.

Thus, closed-loop $p_x$-strategies are an inherent part of the quantum mechanical world and permeate the classical world in the limit $\hslash \to 0$ through the Hamilton–Jacobi Equation.

5. Some Examples

In the following Section 5.1 and Section 5.2, we analyze some common textbook examples from closed-loop strategies’ point of view. Section 5.3 gives a quantum mechanical example, and Section 5.4 and Section 5.5 give some non-canonical quantum examples that illustrate the dependence of closed-loop strategy on the explicit form of the canonical commutations relations, that is, the dependence on the quantum constraint. Finally, in Section 5.5 we give a quantum control example.

5.1. The Stationary Case

The quantum and classical Hamilton–Jacobi Equations (96) and (97) are non-stationary equations, that is, they depend explicitly on time. In quantum mechanics, stationary states play a fundamental role. They are defined by

$$\mathsf{\Psi }(x,t)=e^{-\frac{i}{\hslash }Et}\mathsf{\Phi }\left(x\right).$$

(109)

By substituting this into the time-dependent Schrödinger Equation (94), the time-independent or stationary Schrödinger equation is obtained:

$$-\frac{{\hslash }^2}{2m}\frac{{\partial }^2\mathsf{\Psi }\left(x\right)}{\partial x^2}+U\left(x\right)\mathsf{\Psi }\left(x\right)=E\mathsf{\Psi }\left(x\right).$$

(110)

Now, by writing

$$\mathsf{\Psi }\left(x\right)=e^{\frac{i}{\hslash }W\left(x\right)}$$

(111)

and substituting into (110), the stationary quantum Hamilton–Jacobi Equation holds:

$$\frac{1}{2m}{\left(\frac{\partial W\left(x\right)}{\partial x}\right)}^2+U\left(x\right)-\frac{i\hslash }{2m}\frac{{\partial }^{2}W\left(x\right)}{\partial x^2}=E.$$

(112)

Taking again the classical limit $\hslash \to 0$ in (112), the stationary classical Hamilton–Jacobi Equation appears:

$$\frac{1}{2m}{\left(\frac{\partial W\left(x\right)}{\partial x}\right)}^2+U\left(x\right)=E.$$

(113)

Due to (109) and (111), we have $\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)}=e^{\frac{i}{\hslash }(W\left(x\right)-Et)}$, which implies that

$$S(x,t)=W\left(x\right)-Et$$

(114)

for the stationary case. In this case, the closed-loop $p_x$-strategies are given by

$$p_x=p_x(x,t)=\frac{\partial W\left(x\right)}{\partial x}.$$

5.2. The Non-Stationary Case

But what about the non-stationary closed-loop $p_x$-strategies in the classical limit? In order to analyze this case, consider the example of a free particle, that is, $U\left(x\right)=0$. The non-stationary classical Hamilton–Jacobi Equation is now

$$\frac{1}{2m}{\left(\frac{\partial S(x,t)}{\partial x}\right)}^2=-\frac{\partial S(x,t)}{\partial t}.$$

(115)

One can find a solution of the form $S(x,t)=\frac{1}{2}a\left(t\right)x^2$, so by substituting in (115), one obtains $a\left(t\right)=\frac{-1}{(P_0-\frac{t}{m})}$, so $S(x,t)=\frac{1}{2}\frac{-x^2}{(P_0-\frac{t}{m})}$, and the corresponding closed-loop $p_x$ strategy is

$$p_x(x,t)=\frac{\partial S(x,t)}{\partial x}=\frac{-x}{(P_0-\frac{t}{m})}.$$

(116)

One can evaluate $x\left(t\right)$ using Equation (93)

$$Q_0=\frac{\partial S(x,P_0,t)}{\partial P_0},$$

(117)

where the integration constant $P_0$ must be identified with the constant momentum for the second observer in the coordinate system $(Q,P)$. Thus

$$Q_0=\frac{1}{2}\frac{x^2}{{(P_0-\frac{t}{m})}^2},$$

(118)

from which $x\left(t\right)$ is computed as

$$x\left(t\right)=\sqrt{2{\left(P_0-\frac{t}{m}\right)}^2{Q}_0}=\sqrt{2Q_0}\left(P_0-\frac{t}{m}\right).$$

(119)

The associated open-loop $p_x\left(t\right)$ strategy is found by $p_x\left(t\right)=p_x(x\left(t\right),t)$, similarly to Equation (20). Thus, by substituting (119) into (116):

$$p_x\left(t\right)=\frac{-x\left(t\right)}{(P_0-\frac{t}{m})}=\frac{-\sqrt{2Q_0}(P_0-\frac{t}{m})}{(P_0-\frac{t}{m})}=-\sqrt{2Q_0}=p_0,$$

(120)

so

$$x\left(t\right)=-p_0\left(P_0-\frac{t}{m}\right)=p_0\frac{t}{m}+x_0,$$

(121)

where $x_0=-P_0{p}_0$. These last equations are the solutions for the motion of a free particle of course!

Now from the Hamiltonian equations, one obtains the open-loop dynamics for the free particle:

$$\dot{x}=\frac{p_x}{m},{\dot{p}}_x=-\frac{\partial U\left(x\right)}{\partial x}=0,$$

(122)

so the open-loops dynamics are

$$x\left(t\right)=\frac{p_0}{m}t+x_0,p_x\left(t\right)=p_0.$$

(123)

Then, the open-loop $p_x\left(t\right)$ strategy coming from the Hamiltonian equations of motion is equivalent to the non-stationary closed-loop $p_x(x,t)$ strategy coming from the non-stationary classical Hamilton–Jacobi Equation. This equivalence is valid because the classical Hamilton–Jacobi Equation approach corresponds to a “two-observer” point of view of classical mechanics. The $S(x,t)$ function is just the generator of the canonical transformation, which leaves the Hamiltonian equations invariant. Thus, there are two schemes:

• The Hamiltonian “one observer” approach with its open-loop $p_x\left(t\right)$-strategies;
• The Hamilton–Jacobi “two observer” approach with its closed-loop $p_x(x,t)$-strategies.

Closed-loop strategies $p_x(x,t)$ coming from the Hamilton–Jacobi Equation are similar to the inert optimal closed-loop strategies $u^{*}(x,t)$ of Pontryagin’s approach (in optimal control theory) in the sense that they are equivalent to the open-loop ones.

Note that this equivalence is not generally valid for control theory when analyzed in the phase-space, as conducted in Section 3.4. As is shown in [37,41], for second-class systems, the description of a mechanical system in terms of canonical transformations (together with the Hamilton–Jacobi Equation for the S function) can be inconsistent. It is due that the second-class systems violate the Carathéodory’s integrability conditions. By using the methodology developed in [41], in [26], there is proof that there exists a class of optimal controls problems, called regular subclass, in which the functions $F(x,u,t)$ and $f(x,u,t)$ in (1) and (2) have the form

$$\begin{array}{cc}& F(x,u,t)=F_0\left(x\right)+F_{1}u+\frac{1}{2}{F}_2{u}^2,\hfill \\ & f(x,u,t)=f_0\left(x\right)+f_{1}u,\hfill \end{array}$$

(where $F_1,F_2$ and $f_1$ are constants and $F_0\left(x\right)$ and $f_0\left(x\right)$ are arbitrary continuous functions), such that for this subclass:

(i)

The Hamilton–Jacobi Equation is well-defined;

(ii)

The solution to the Hamilton–Jacobi Equation gives the same dynamics as the Pontryagin equations.

For more general forms of the $F(x,u,t)$ and $f(x,u,t)$ functions, it is not clear whether the problem is consistent or if the dynamics of the Hamilton–Jacobi–Bellman are equivalent to the Pontryagin equations in the phase-space.

5.3. The Pure Quantum Limit and Closed-Loop Strategies

In the previous section, the classical limit $\hslash \to 0$ was taken, and its characteristics were explored in terms of the closed-loop strategies. In this section, the inverse limit will be taken, that is, $\hslash >>1$, and the consequences of a higher non-commutative system of quantum variables will be explored.

Consider the non-stationary quantum Hamilton–Jacobi Equation (96). Taking the limit $\hslash >>1$ and supposing that the time derivative of the S function has a higher value,

$$-\frac{i\hslash }{2m}\frac{{\partial }^{2}S(x,t)}{\partial x^2}=-\frac{\partial S(x,t)}{\partial t}.$$

(124)

or, what is the same,

$$-\frac{{\hslash }^2}{2m}\frac{{\partial }^{2}S(x,t)}{\partial x^2}=i\hslash \frac{\partial S(x,t)}{\partial t}.$$

(125)

But, this is the free-particle Schrödinger equation for $S(x,t)$. In this way, one can write

$$S(x,t)=e^{\frac{i}{\hslash }T(x,t)}$$

(126)

where $T(x,t)$ satisfies the quantum Hamilton–Jacobi Equation

$$\frac{1}{2m}{\left(\frac{\partial T(x,t)}{\partial x}\right)}^2-\frac{i\hslash }{2m}\frac{{\partial }^{2}T(x,t)}{\partial x^2}=-\frac{\partial T(x,t)}{\partial t}$$

(127)

and the wave function is given by

$$\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)}=e^{\frac{i}{\hslash }{e}^{\frac{i}{\hslash }T(x,t)}}.$$

(128)

The corresponding closed-loop $p_x$ strategy is

$$p_x=\frac{\partial S(x,t)}{\partial x}=\frac{i}{\hslash }\frac{\partial T(x,t)}{\partial x}S(x,t)$$

(129)

Note that $\frac{\partial T(x,t)}{\partial x}$ can be interpreted as a closed-loop $p_T$ strategy for the quantum Hamilton–Jacobi Equation (127). Thus, denoting the closed-loop $p_x$ strategy for (124) by $p_S(x,t)$, then both strategies are related by

$$p_S(x,t)=\frac{i}{\hslash }S(x,t)p_T(x,t)$$

(130)

Again, if the time derivative of T has a higher value, and as $\hslash >>1$, the quantum Hamilton–Jacobi for $T(x,t)$ (127) is in this limit again a Schrödinger equation,

$$-\frac{{\hslash }^2}{2m}\frac{{\partial }^{2}T(x,t)}{\partial x^2}=i\hslash \frac{\partial T(x,t)}{\partial t}.$$

(131)

Hence, we can write

$$T(x,t)=e^{\frac{i}{\hslash }U(x,t)}$$

(132)

where $U(x,t)$ satisfies

$$\frac{1}{2m}{\left(\frac{\partial U(x,t)}{\partial x}\right)}^2-\frac{i\hslash }{2m}\frac{{\partial }^{2}U(x,t)}{\partial x^2}=-\frac{\partial U(x,t)}{\partial t}.$$

(133)

and the wave function is

$$\mathsf{\Psi }(x,t)=e^{\frac{i}{\hslash }S(x,t)}=e^{\frac{i}{\hslash }{e}^{\frac{i}{\hslash }{e}^{\frac{i}{\hslash }U(x,t)}}}$$

(134)

Putting $p_U=\frac{\partial U(x,t)}{\partial x}$ the closed-loop strategy associated to the quantum Hamilton–Jacobi Equation for U (133), the corresponding closed-loop $p_S$ strategy is then in this case

$$p_S(x,t)=\frac{i}{\hslash }{p}_U(x,t)S(x,t)T(x,t).$$

(135)

Since one can keep iterating this procedure to infinity, quantum mechanical systems can admit multistage closed-loop strategies, and they are connected in a strongly non-linear way, as in (134).

5.4. A Non-Canonical Example

It is well known that quantum gravity effects, modelled by string theory, loop quantum gravity, or black hole physics, predict the existence of a Generalized Uncertainty Principle (GUP), which can change the usual canonical commutation relations [42,43,44,45,46,47,48] and its implications for entanglement and the Hamilton–Jacobi Equation [49,50,51,52,53]. In this case, the canonical commutation relations

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \mathbb{I}$$

(136)

are replaced by a more general one of the type

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \left(\mathbb{I}+F(\widehat{x},{\widehat{p}}_x)\right)$$

(137)

for some function $F(x,p_x)$. The form of the quantum algebra (137) guarantees that the system has a classical limit when ℏ goes to zero. Now, if one sees the commutation relations as a constraint in a noncommutative space, then if one “solves” ${\widehat{p}}_x$ in terms of $\widehat{x}$ from (137), then an explicit form of ${\widehat{p}}_x$ in terms of $\widehat{x}$ would depend on the function F. Thus, the expression (100) for $p_x(x,t)$ would depend on F. To explicitly show that dependence, we consider two simple toy examples of GUP algebras below.

5.4.1. A GUP Algebra Depending on $\widehat{x}$

As an example this dependence on GUP, consider the following commutation relations

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \left(I+\alpha \widehat{x}\right)$$

(138)

for which $F(x,p)=\alpha x$, and where $\alpha$ is a constant. An operator representation of the above commutation relations is

$$\widehat{x}=x$$

(139)

$${\widehat{p}}_x=-i\hslash \left(\frac{\partial }{\partial x}+\alpha x\frac{\partial }{\partial x}\right)=-i\hslash (1+\alpha x)\frac{\partial }{\partial x}$$

(140)

The momentum eigenstates are, in this case,

$$\begin{array}{cc}& {\widehat{p}}_x{\mathsf{\Phi }}_{p_x}\left(x\right)=p_x{\mathsf{\Phi }}_{p_x}\left(x\right)\end{array}$$

(141)

$$\begin{array}{cc}& -i\hslash (1+\alpha x)\frac{\partial {\mathsf{\Phi }}_{p_x}}{\partial x}=p_x{\mathsf{\Phi }}_{p_x}\end{array}$$

(142)

so

$${\mathsf{\Phi }}_{p_x}\left(x\right)=C{(1+\alpha x)}^{\frac{i{p}_x}{\alpha \hslash }}=C{e}^{\frac{i{p}_x}{\hslash }ln[{(1+\alpha x)}^{\frac{1}{\alpha }}]}.$$

(143)

Consider now the usual non-relativist classical Hamiltonian (99). The quantization of this Hamiltonian by the rule (138), implies the following Schrödinger equation

$$\frac{-{\hslash }^2}{2m}(1+\alpha x)\frac{\partial }{\partial x}\left[(1+\alpha x)\frac{\partial }{\partial x}\right]\mathsf{\Psi }(x,t)+V\left(x\right)\mathsf{\Psi }(x,t)=i\hslash \frac{\partial \mathsf{\Psi }(x,t)}{\partial t}$$

(144)

or

$$\frac{-{\hslash }^2}{2m}\left[\alpha (1+\alpha x)\frac{\partial \mathsf{\Psi }(x,t)}{\partial x}+{(1+\alpha x)}^2\frac{{\partial }^2\mathsf{\Psi }(x,t)}{\partial x^2}\right]+V\left(x\right)\mathsf{\Psi }(x,t)=i\hslash \frac{\partial \mathsf{\Psi }(x,t)}{\partial t}$$

(145)

The corresponding Hamilton–Jacobi Equation is obtained from (95), by using the fact that

$$\begin{array}{cc}& \frac{\partial \mathsf{\Psi }(x,t)}{\partial x}=e^{\frac{i}{\hslash }S}\left(\frac{\partial S}{\partial x}\right)\frac{i}{\hslash }\end{array}$$

(146)

$$\begin{array}{cc}& \frac{{\partial }^2\mathsf{\Psi }(x,t)}{\partial x^2}=e^{\frac{i}{\hslash }S}{\left(\frac{\partial S}{\partial x}\right)}^2{\left(\frac{i}{\hslash }\right)}^2+\frac{i}{\hslash }{e}^{\frac{i}{\hslash }S}\left(\frac{{\partial }^{2}S}{\partial x^2}\right)\end{array}$$

(147)

so (145) becomes

$$\begin{array}{cc}& \frac{-{\hslash }^2}{2m}\left[\alpha (1+\alpha x)e^{\frac{i}{\hslash }S}\left(\frac{\partial S}{\partial x}\right)\frac{i}{\hslash }+{(1+\alpha x)}^2\left(e^{\frac{i}{\hslash }S}\frac{i^2}{{\hslash }^2}{\left(\frac{\partial S}{\partial x}\right)}^2+x^2\frac{i}{\hslash }{e}^{\frac{i}{\hslash }S}\frac{{\partial }^{2}S}{\partial x^2}\right)\right]\end{array}$$

(148)

$$\begin{array}{cc}& +V\left(x\right)e^{\frac{i}{\hslash }S}=i\hslash e^{\frac{i}{\hslash }S}\frac{i}{\hslash }\frac{\partial S}{\partial t}\end{array}$$

(149)

or

$$\frac{{(1+\alpha x)}^2}{2m}{\left(\frac{\partial S}{\partial x}\right)}^2+V\left(x\right)-\frac{i\hslash }{2m}\left[(1+\alpha x)\frac{\partial S}{\partial x}+{(1+\alpha x)}^2\frac{{\partial }^{2}S}{\partial x^2}\right]=-\frac{\partial S}{\partial t}$$

(150)

By taking the classical limit $\hslash \to 0$, one arrives to the modified Hamilton–Jacobi equation

$$\frac{{(1+\alpha x)}^2}{2m}{\left(\frac{\partial S}{\partial x}\right)}^2+V\left(x\right)=-\frac{\partial S}{\partial t}.$$

(151)

Now, one must identify $p_x(x,t)$, not with the derivative of S as in (100), but instead with

$$p_x(x,t)=(1+\alpha x)\frac{\partial S(x,t)}{\partial x}.$$

(152)

Thus, the closed-loop $p_x$ strategy, depends on the form of the GUP. In fact, the momentum operator (140) acting on (95) gives

$${\widehat{p}}_x\mathsf{\Psi }(x,t)=-i\hslash (1+\alpha x)\frac{\partial }{\partial x}{e}^{\frac{i}{\hslash }S(x,t)}=(1+\alpha x)\frac{\partial S}{\partial x}\mathsf{\Psi }(x,t)$$

(153)

so from the above equation, one can identify again the local momentum $p_x(x,t)$ again with $(1+\alpha x)\frac{\partial S}{\partial x}$.

If, instead of the commuting relation (138), one can consider the generalization

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \left(I+\alpha {\widehat{x}}^n\right)$$

(154)

for $n=1,2,3\dots$, the momentum operator has the representation

$${\widehat{p}}_x=-i\hslash (1+\alpha x^n)\frac{\partial }{\partial x}$$

(155)

so when acting on the wave function (95) it gives

$${\widehat{p}}_x\mathsf{\Psi }(x,t)=-i\hslash (1+\alpha x^n)\frac{\partial }{\partial x}{e}^{\frac{i}{\hslash }S(x,t)}=(1+\alpha x^n)\frac{\partial S}{\partial x}\mathsf{\Psi }(x,t)$$

(156)

then the corresponding closed-loop $p_x(x,t)$ strategy is in this case

$$p_x(x,t)=(1+\alpha x^n)\frac{\partial S(x,t)}{\partial x}.$$

(157)

5.4.2. A GUP Algebra Depending on the Momentum ${\widehat{p}}_x$

Consider now the following GUP commutation relations depending on the momentum

$$[\widehat{x},{\widehat{p}}_x]=i\hslash \left(\mathbb{I}+\beta {\widehat{p}}_x\right)$$

(158)

These commutation relations can be represented by the differential operators

$$\widehat{x}=x$$

(159)

$${\widehat{p}}_x=\frac{1}{\beta }\left(e^{\beta {\widehat{p}}_0}-\mathbb{I}\right)$$

(160)

where ${\widehat{p}}_0=-i\hslash \frac{\partial }{\partial x}$ is the standard momentum operator for $\beta =0$ case.

The action of the momentum operator (160) on the wave function (95) is

$${\widehat{p}}_x\mathsf{\Psi }(x,t)=\frac{1}{\beta }\left(e^{\beta {\widehat{p}}_0}-\mathbb{I}\right)e^{\frac{i}{\hslash }S(x,t)}$$

(161)

By expanding the exponential operator $e^{\beta {\widehat{p}}_0}$, one can show that

$$\begin{array}{cc}& \frac{1}{\beta }\left(e^{\beta {\widehat{p}}_0}-\mathbb{I}\right)e^{\frac{i}{\hslash }S(x,t)}=\frac{1}{\beta }\left(e^{\beta \frac{\hslash }{i}\frac{\partial }{\partial x}}-\mathbb{I}\right)e^{\frac{i}{\hslash }S(x,t)}=\end{array}$$

(162)

$$\begin{array}{cc}& \left[\frac{1}{\beta }\left(e^{\beta \left(\frac{\partial S}{\partial x}\right)}-1\right)+\frac{i}{\beta \hslash }\left(e^{\left(\frac{\beta \hslash }{i}\right)\frac{\partial }{\partial x}}-\mathbb{I}-\frac{\beta \hslash }{i}\frac{\partial }{\partial x}\right)S(x,t)+\frac{\hslash }{i}\mathsf{\Theta }(x,t)\right]e^{\frac{i}{\hslash }S(x,t)}\end{array}$$

(163)

where the $\mathsf{\Theta }$ function contains products between powers of $\left(\frac{\partial S}{\partial x}\right)$ and high-order derivatives of $S(x,t)$. For example, the first terms of $\mathsf{\Theta }$ are

$$\begin{array}{cc}\hfill \mathsf{\Theta }(x,t)=& \frac{1}{2}{\beta }^2\left(\frac{\partial S}{\partial x}\right)\frac{{\partial }^{2}S}{\partial x^2}+\frac{1}{4}{\beta }^3{\left(\frac{\partial S}{\partial x}\right)}^2\frac{{\partial }^{2}S}{\partial x^2}+\end{array}$$

(164)

$$\begin{array}{cc}& \frac{1}{8}\left(\frac{\hslash }{i}\right){\beta }^3{\left(\frac{{\partial }^{2}S}{\partial x^2}\right)}^2+\frac{1}{6}\left(\frac{\hslash }{i}\right){\beta }^3\left(\frac{\partial S}{\partial x}\right)\left(\frac{{\partial }^{3}S}{\partial x^3}\right)+\dots \end{array}$$

(165)

so

$${\widehat{p}}_x\mathsf{\Psi }(x,t)=p(x,t)\mathsf{\Psi }(x,t)$$

(166)

where the local quantum momentum $p(x,t)$ is

$$p(x,t)=\frac{1}{\beta }\left(e^{\beta \left(\frac{\partial S}{\partial x}\right)}-1\right)+\frac{i}{\beta \hslash }\left(e^{\left(\frac{\beta \hslash }{i}\right)\frac{\partial }{\partial x}}-\mathbb{I}-\frac{\beta \hslash }{i}\frac{\partial }{\partial x}\right)S(x,t)+\frac{\hslash }{i}\mathsf{\Theta }(x,t)$$

(167)

Note that $p(x,t)$ is a complex number in this case. In the classical limit $\hslash \to 0$, the local quantum momentum (167) becomes

$$p(x,t)=\frac{1}{\beta }\left(e^{\beta \left(\frac{\partial S}{\partial x}\right)}-1\right)$$

(168)

because the operator

$$\frac{i}{\beta \hslash }\left(e^{\left(\frac{\beta \hslash }{i}\right)\frac{\partial }{\partial x}}-\mathbb{I}-\frac{\beta \hslash }{i}\frac{\partial }{\partial x}\right)$$

(169)

goes to zero when $\hslash \to 0$. The classical local momentum given by (168) could also be obtained from the corresponding Hamilton–Jacobi Equation. The Schrödinger equation associated with the Hamiltonian function (99) is in this case

$$\frac{1}{2m{\beta }^2}\left(e^{\beta {\widehat{p}}_0}-\mathbb{I}\right)\left(e^{\beta {\widehat{p}}_0}-\mathbb{I}\right)\mathsf{\Psi }+V\left(x\right)\mathsf{\Psi }=i\hslash \frac{\partial \mathsf{\Psi }}{\partial t}$$

(170)

or

$$\frac{1}{2m{\beta }^2}\left(e^{2\beta {\widehat{p}}_0}-2e^{\beta {\widehat{p}}_0}+\mathbb{I}\right)e^{\frac{i}{\hslash }S(x,t)}+V\left(x\right)e^{\frac{i}{\hslash }S(x,t)}=-\frac{\partial S(x,t)}{\partial t}{e}^{\frac{i}{\hslash }S(x,t)}$$

(171)

Due to the fact that in the classical limit $\hslash \to 0$

$$e^{\beta {\widehat{p}}_0}{e}^{\frac{i}{\hslash }S(x,t)}=e^{\beta \frac{\hslash }{i}\frac{\partial }{\partial x}}{e}^{\frac{i}{\hslash }S(x,t)}=e^{\beta \left(\frac{\partial S}{\partial x}\right)}{e}^{\frac{i}{\hslash }S(x,t)}$$

(172)

Equation (171) becomes in this classical limit

$$\frac{1}{2m{\beta }^2}\left(e^{2\beta \left(\frac{\partial S}{\partial x}\right)}-2e^{\beta \left(\frac{\partial S}{\partial x}\right)}+1\right)e^{\frac{i}{\hslash }S(x,t)}+V\left(x\right)e^{\frac{i}{\hslash }S(x,t)}=-\frac{\partial S(x,t)}{\partial t}{e}^{\frac{i}{\hslash }S(x,t)}$$

(173)

that is

$$\frac{1}{2m{\beta }^2}{\left(e^{\beta \left(\frac{\partial S}{\partial x}\right)}-1\right)}^2+V\left(x\right)=-\frac{\partial S(x,t)}{\partial t}$$

(174)

From the above equation, one can identify the corresponding classical closed-loop strategy $p(x,t)$ as the same as that given in (168). Thus, we see that the quantum commutation relations (canonical or GUP) imply the existence of closed-loop strategies for the local momentum, because these quantum commutation relations are constraints in a non-commutative space.

5.5. A Quantum Control Example

At last, we consider a simple example of quantum control [54,55] defined by a single spin particle $\overrightarrow{S}=\frac{1}{2}({\sigma }_x,{\sigma }_y,{\sigma }_z)$, being ${\sigma }_x,{\sigma }_y$, and ${\sigma }_z$, which are the so-called Pauli matrices. The Schrödinger equation for this particle in a magnetic field $\overrightarrow{B}=(B_x,B_y,B_z)$ is [54]

$$\frac{\partial \psi }{\partial t}=\frac{1}{2}\left[{\sigma }_z{u}_z+{\sigma }_x{u}_x\left(t\right)+{\sigma }_y{u}_y\left(t\right)\right]\psi ,$$

(175)

where the components $B_x,B_y$ are considered as time-dependent control variables $u_x\left(t\right),u_y\left(t\right)$, and the $B_z$ component is a fixed parameter $u_z$.

By separating the two component spinor

$$\psi ={\psi }_R+i{\psi }_I=\begin{array}{c}\hfill \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)\end{array}+i\begin{array}{c}\hfill \left(\begin{array}{c}{x}_3\\ x_4\end{array}\right)\end{array}$$

in its real and imaginary parts and doing the same with the Pauli matrices, the Schrödinger (175) can be writing as a four vector equation of the form [54]

$$\frac{d\overrightarrow{x}}{dt}=\frac{1}{2}\left[H_x{u}_x\left(t\right)+H_y{u}_y\left(t\right)+H_z{u}_z\right]\overrightarrow{x}.$$

(176)

where

$$\begin{array}{cc}& H_x=\left(\begin{array}{cccc}0& 0& 0& -1\\ 0& 0& -1& 0\\ 0& 1& 0& 0\\ 1& 0& 0& 0\end{array}\right)\hfill \\ & H_y=\left(\begin{array}{cccc}0& -1& 0& 0\\ 1& 0& 0& 0\\ 0& 0& 0& -1\\ 0& 0& 1& 0\end{array}\right)\hfill \\ & H_z=\left(\begin{array}{cccc}0& 0& -1& 0\\ 0& 0& 0& 1\\ 1& 0& 0& 0\\ 0& -1& 0& 0\end{array}\right)\hfill \end{array}$$

and

$$\begin{array}{c}\hfill \overrightarrow{x}=\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\\ x_4\end{array}\right)\end{array}$$

The quantum control problems consist thus in to minimize the functional [54]

$$J\left(u_x,u_y\right)=\mathsf{\Phi }\left(\overrightarrow{x}\left(T\right)\right)+k{\int }_0^T\left(u_x^2\left(t\right)+u_y^2\left(t\right)\right)dt$$

(177)

subject to the Schrödinger Equation (176).

5.5.1. Open-Loop Pontryagin Dynamics

Introducing a Lagrange multiplier for each scalar equation in the system (176), i.e., the vector

$$\begin{array}{c}\hfill \overrightarrow{\lambda }=\left(\begin{array}{c}{\lambda }_1\\ {\lambda }_2\\ {\lambda }_3\\ {\lambda }_4\end{array}\right)\end{array}$$

the Pontryagin Hamiltonian h is

$$h=k\left(u_x^2\left(t\right)+u_y^2\left(t\right)\right)+\frac{1}{2}{\overrightarrow{\lambda }}^t\left[H_x{u}_x\left(t\right)+H_y{u}_y\left(t\right)+H_z{u}_z\right]\overrightarrow{x}$$

(178)

The corresponding Pontryagin equations are for the state variables (spinor components)

$$\dot{\overrightarrow{x}}=\frac{1}{2}\left[H_x{u}_x\left(t\right)+H_y{u}_y\left(t\right)+H_z{u}_z\right]\overrightarrow{x}=\frac{1}{2}H\overrightarrow{x}$$

(179)

and for the Lagrange multipliers

$$\dot{\overrightarrow{\lambda }}=\frac{1}{2}\left[H_x{u}_x\left(t\right)+H_y{u}_y\left(t\right)+H_z{u}_z\right]\overrightarrow{\lambda }=\frac{1}{2}H\overrightarrow{\lambda }$$

(180)

where

$$H=\left[\begin{array}{cccc}0& -u_y& -u_z& -u_x\\ u_y& 0& -u_x& u_z\\ u_z& u_x& 0& -u_y\\ u_x& -u_z& u_y& 0\end{array}\right]$$

(181)

The equation for the controls gives

$$u_x=-\frac{1}{4k}{\overrightarrow{\lambda }}^t{H}_x\overrightarrow{x}=-\frac{-{\lambda }_1\left(t\right)x_4\left(t\right)-{\lambda }_2\left(t\right)x_3\left(t\right)+{\lambda }_3\left(t\right)x_2\left(t\right)+{\lambda }_4\left(t\right)x_1\left(t\right)}{4k}$$

(182)

$$u_y=-\frac{1}{4k}{\overrightarrow{\lambda }}^t{H}_y\overrightarrow{x}=-\frac{-{\lambda }_1\left(t\right)x_2\left(t\right)+{\lambda }_2\left(t\right)x_1\left(t\right)-{\lambda }_3\left(t\right)x_4\left(t\right)+{\lambda }_4\left(t\right)x_3\left(t\right)}{4k}$$

(183)

By replacing (182) and (183) in (181) one has that

$$\begin{array}{c}H[\overrightarrow{x},\overrightarrow{\lambda }]=\left[\begin{array}{cc}0& -\frac{{\lambda }_1\left(t\right)x_2\left(t\right)-{\lambda }_2\left(t\right)x_1\left(t\right)+{\lambda }_3\left(t\right)x_4\left(t\right)-{\lambda }_4\left(t\right)x_3\left(t\right)}{4k}\\ \\ -\frac{-{\lambda }_1\left(t\right)x_2\left(t\right)+{\lambda }_2\left(t\right)x_1\left(t\right)-{\lambda }_3\left(t\right)x_4\left(t\right)+{\lambda }_4\left(t\right)x_3\left(t\right)}{4k}& 0\\ \\ u_z& -\frac{-{\lambda }_1\left(t\right)x_4\left(t\right)-{\lambda }_2\left(t\right)x_3\left(t\right)+{\lambda }_3\left(t\right)x_2\left(t\right)+{\lambda }_4\left(t\right)x_1\left(t\right)}{4k}\\ \\ -\frac{-{\lambda }_1\left(t\right)x_4\left(t\right)-{\lambda }_2\left(t\right)x_3\left(t\right)+{\lambda }_3\left(t\right)x_2\left(t\right)+{\lambda }_4\left(t\right)x_1\left(t\right)}{4k}& -u_z\end{array}\right.\\ \\ \left.\begin{array}{cc}-u_z& -\frac{{\lambda }_1\left(t\right)x_4\left(t\right)+{\lambda }_2\left(t\right)x_3\left(t\right)-{\lambda }_3\left(t\right)x_2\left(t\right)-{\lambda }_4\left(t\right)x_1\left(t\right)}{4k}\\ \\ -\frac{{\lambda }_1\left(t\right)x_4\left(t\right)+{\lambda }_2\left(t\right)x_3\left(t\right)-{\lambda }_3\left(t\right)x_2\left(t\right)-{\lambda }_4\left(t\right)x_1\left(t\right)}{4k}& u_z\\ \\ 0& -\frac{{\lambda }_1\left(t\right)x_2\left(t\right)-{\lambda }_2\left(t\right)x_1\left(t\right)+{\lambda }_3\left(t\right)x_4\left(t\right)-{\lambda }_4\left(t\right)x_3\left(t\right)}{4k}\\ \\ -\frac{-{\lambda }_1\left(t\right)x_2\left(t\right)+{\lambda }_2\left(t\right)x_1\left(t\right)-{\lambda }_3\left(t\right)x_4\left(t\right)+{\lambda }_4\left(t\right)x_3\left(t\right)}{4k}& 0\end{array}\right]\end{array}$$

(184)

The Hamiltonian matrix (184) gives for equations (179) and (180) a Ricatti-type system of equations for both $\overrightarrow{x}$ and $\overrightarrow{\lambda }$. One must integrate these equations with the initial conditions $\overrightarrow{x}\left(0\right)={\overrightarrow{x}}_0$ for the state variables, and the final conditions (the transversality condition)

$$\overrightarrow{\lambda }\left(T\right)=(\frac{\partial \mathsf{\Phi }}{\partial x_1},\frac{\partial \mathsf{\Phi }}{\partial x_2},\frac{\partial \mathsf{\Phi }}{\partial x_3},\frac{\partial \mathsf{\Phi }}{\partial x_4})$$

for the Lagrangian multipliers. Note that due to the fact that the Hamiltonian matrix H in (184) is antisymmetric, one has that

$$\frac{d{||\overrightarrow{x}||}^2}{dt}=\frac{d}{dt}({\overrightarrow{x}}^t·\overrightarrow{x})=2{\overrightarrow{x}}^t·\frac{d\overrightarrow{x}}{dt}={\overrightarrow{x}}^{t}H\overrightarrow{x}=\sum _{i,j}{x}_i{H}_{ij}{x}_j=0$$

(185)

and the same is true for $\overrightarrow{\lambda }$

$$\frac{d{||\overrightarrow{\lambda }||}^2}{dt}=\frac{d}{dt}({\overrightarrow{\lambda }}^t·\overrightarrow{\lambda })=2{\overrightarrow{\lambda }}^t·\frac{d\overrightarrow{\lambda }}{dt}={\overrightarrow{\lambda }}^{t}H\overrightarrow{\lambda }=\sum _{i,j}{\lambda }_i{H}_{ij}{\lambda }_j=0.$$

(186)

The above two equations imply that the norm of both vectors $\overrightarrow{x}$ and $\overrightarrow{\lambda }$ remain constant during the dynamical evolution. Also,

$$\frac{d}{dt}{\overrightarrow{\lambda }}^t·\overrightarrow{x}=\frac{d{\overrightarrow{\lambda }}^t}{dt}·\overrightarrow{x}+{\overrightarrow{\lambda }}^t·\frac{d\overrightarrow{x}}{dt}=\frac{1}{2}{\left(H\overrightarrow{\lambda }\right)}^t\overrightarrow{x}+\frac{1}{2}{\overrightarrow{\lambda }}^{t}H\overrightarrow{x}=\frac{1}{2}\left({\overrightarrow{\lambda }}^t{H}^t\overrightarrow{x}+{\overrightarrow{\lambda }}^{t}H\overrightarrow{x}\right)=\frac{1}{2}\left(-{\overrightarrow{\lambda }}^{t}H\overrightarrow{x}+{\overrightarrow{\lambda }}^{t}H\overrightarrow{x}\right)=0$$

(187)

so

$$\frac{d}{dt}\left(||\overrightarrow{\lambda }||||\overrightarrow{x}||cos\left(\alpha \right)\right)=0$$

(188)

and due to the fact that the norms of $\overrightarrow{x}$ and $\overrightarrow{\lambda }$ are constant, the last equation implies that

$$\frac{d}{dt}cos\left(\alpha \right)=0$$

(189)

so, the angle $\alpha$ between $\overrightarrow{x}$ and $\overrightarrow{\lambda }$ remains constant. The vector system $\overrightarrow{x}\left(t\right),\overrightarrow{\lambda }\left(t\right)$ evolves in time as a rigid structure with no relative motion. In this way, one can always decompose $\overrightarrow{\lambda }\left(t\right)$ in the form

$$\overrightarrow{\lambda }\left(t\right)=a\overrightarrow{x}\left(t\right)+b{\overrightarrow{x}}_{\perp }\left(t\right)$$

(190)

where a and b are constant, and ${\overrightarrow{x}}_{\perp }$ is an orthogonal vector to $\overrightarrow{x}$. Let ${\overrightarrow{x}}_{\perp }=(a_1,a_2,a_3,a_4)$, due to the fact that

$$\overrightarrow{x}·{\overrightarrow{x}}_{\perp }=x_1\left(t\right)a_1\left(t\right)+x_2\left(t\right)a_2\left(t\right)+x_3\left(t\right)a_3\left(t\right)+x_4\left(t\right)a_4\left(t\right)=0$$

(191)

we can choose ${\overrightarrow{x}}_{\perp }$ as

$$\begin{array}{c}\hfill {\overrightarrow{x}}_{\perp }=\left(\begin{array}{c}{a}_1\left(t\right)\\ a_2\left(t\right)\\ a_3\left(t\right)\\ -\frac{x_1\left(t\right)a_1\left(t\right)+x_2\left(t\right)a_2\left(t\right)+x_3\left(t\right)a_3\left(t\right)}{x_4\left(t\right)}\end{array}\right)\end{array}$$

(192)

By replacing (190) in (179) and (180), one obtains

$$\dot{\overrightarrow{x}}=\frac{1}{2}H[\overrightarrow{x},a\overrightarrow{x}+b{\overrightarrow{x}}_{\perp }]\overrightarrow{x}$$

(193)

$${\dot{\overrightarrow{x}}}_{\perp }=\frac{1}{2}H[\overrightarrow{x},a\overrightarrow{x}+b{\overrightarrow{x}}_{\perp }]{\overrightarrow{x}}_{\perp }.$$

(194)

In this case, the controls (182) and (183) read as

$$u_x=-\frac{1}{4k}{(a\overrightarrow{x}+b{\overrightarrow{x}}_{\perp })}^t{H}_x\overrightarrow{x}=-\frac{b}{4k}{\overrightarrow{x}}_{\perp }^t{H}_x\overrightarrow{x}$$

(195)

$$u_y=-\frac{1}{4k}{(a\overrightarrow{x}+b{\overrightarrow{x}}_{\perp })}^t{H}_y\overrightarrow{x}=-\frac{b}{4k}{\overrightarrow{x}}_{\perp }^t{H}_y\overrightarrow{x}$$

(196)

or explicitly

$$u_x=-\frac{b\left(-a_1\left(t\right)x_1^2\left(t\right)-a_1\left(t\right)x_4^2\left(t\right)-a_2\left(t\right)x_1\left(t\right)x_2\left(t\right)-a_2\left(t\right)x_3\left(t\right)x_4\left(t\right)-a_3\left(t\right)x_1\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)x_4\left(t\right)\right)}{4k{x}_4\left(t\right)}$$

(197)

$$u_y=-\frac{b\left(-a_1\left(t\right)x_1\left(t\right)x_3\left(t\right)-a_1\left(t\right)x_2\left(t\right)x_4\left(t\right)+a_2\left(t\right)x_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_2\left(t\right)x_3\left(t\right)-a_3\left(t\right)x_3^2\left(t\right)-a_3\left(t\right)x_4^2\left(t\right)\right)}{4k{x}_4\left(t\right)}$$

(198)

One can consider first the case $b=0$, so $\overrightarrow{\lambda }=a\overrightarrow{x}$. In this case, the controls (195) and (196) read as

$$u_x=-\frac{1}{4k}a{\overrightarrow{x}}^t{H}_x\overrightarrow{x}=0$$

(199)

$$u_y=-\frac{1}{4k}a{\overrightarrow{x}}^t{H}_y\overrightarrow{x}=0$$

(200)

so

$$H=\left[\begin{array}{cccc}0& 0& -u_z& 0\\ 0& 0& 0& u_z\\ u_z& 0& 0& 0\\ 0& -u_z& 0& 0\end{array}\right]$$

(201)

and the equation of motion for $\overrightarrow{x}$ becomes

$$\left(\begin{array}{c}\frac{d{x}_1}{dt}\\ \frac{d{x}_2}{dt}\\ \frac{d{x}_3}{dt}\\ \frac{d{x}_4}{dt}\end{array}\right)=\frac{1}{2}\left[\begin{array}{cccc}0& 0& -u_z& 0\\ 0& 0& 0& u_z\\ u_z& 0& 0& 0\\ 0& -u_z& 0& 0\end{array}\right]\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\\ x_4\end{array}\right)$$

(202)

or

$$\left[\begin{array}{c}\frac{d}{dt}{x}_1\left(t\right)\\ \frac{d}{dt}{x}_2\left(t\right)\\ \frac{d}{dt}{x}_3\left(t\right)\\ \frac{d}{dt}{x}_4\left(t\right)\end{array}\right]=\frac{1}{2}\left[\begin{array}{c}-u_z{x}_3\left(t\right)\\ u_z{x}_4\left(t\right)\\ u_z{x}_1\left(t\right)\\ -u_z{x}_2\left(t\right)\end{array}\right]$$

(203)

so

$$\begin{array}{cc}{x}_1\left(t\right)=& A_{1}sin(\frac{u_z}{2}t+{\varphi }_1)\\ x_2\left(t\right)=& A_{2}sin(\frac{u_z}{2}t+{\varphi }_2)\\ x_3\left(t\right)=& -A_{1}cos(\frac{u_z}{2}t+{\varphi }_1)\\ x_4\left(t\right)=& -A_{2}cos(\frac{u_z}{2}t+{\varphi }_2)\end{array}$$

(204)

which is the usual sinusoidal solution.

The systems for ${\overrightarrow{x}}_{\perp }$ are

$$\begin{array}{c}\left[\begin{array}{c}\frac{d}{dt}{a}_1\left(t\right)\\ \\ \frac{d}{dt}{a}_2\left(t\right)\\ \\ \frac{d}{dt}{a}_3\left(t\right)\\ \\ -\frac{\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)\frac{d}{dt}{x}_4\left(t\right)}{x_4^2\left(t\right)}+\\ \frac{-a_1\left(t\right)\frac{d}{dt}{x}_1\left(t\right)-a_2\left(t\right)\frac{d}{dt}{x}_2\left(t\right)-a_3\left(t\right)\frac{d}{dt}{x}_3\left(t\right)}{x_4\left(t\right)}+\\ \frac{-x_1\left(t\right)\frac{d}{dt}{a}_1\left(t\right)-x_2\left(t\right)\frac{d}{dt}{a}_2\left(t\right)-x_3\left(t\right)\frac{d}{dt}{a}_3\left(t\right)}{x_4\left(t\right)}\end{array}\right]=\\ \\ \frac{1}{2}\left[\begin{array}{c}-u_z{a}_3\left(t\right)\\ \\ \frac{u_z\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)}{x_4\left(t\right)}\\ \\ u_z{a}_1\left(t\right)\\ \\ -u_z{a}_2\left(t\right)\end{array}\right]\end{array}$$

(205)

Note that there are three variables and four equations, so the system for ${\overrightarrow{x}}_{\perp }$ can be inconsistent. For the particular case $b=0$, if one replaces the derivatives of $a_1,a_2,a_3$ from the three first equations in the fourth one, it obtains the identity

$$\frac{-u_z{a}_2\left(t\right)}{2}=\frac{-u_z{a}_2\left(t\right)}{2}$$

thus, there is no contradiction this case.

Consider now the case $a=0$, so $\overrightarrow{\lambda }=b{\overrightarrow{x}}_{\perp }$. The systems for $\overrightarrow{x}$ are

$$\begin{array}{cc}& \frac{d}{dt}{x}_1\left(t\right)=\hfill \\ & +\frac{b}{8k}\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)x_2\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{1}{2}{u}_z{x}_3\left(t\right)\hfill \end{array}$$

$$\begin{array}{cc}& \frac{d}{dt}{x}_2\left(t\right)=\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)x_3\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{b}{8k}\frac{b\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)x_1\left(t\right)}{x_4\left(t\right)}\hfill \\ & +\frac{1}{2}{u}_z{x}_4\left(t\right)\hfill \end{array}$$

$$\begin{array}{cc}& \frac{d}{dt}{x}_3\left(t\right)=\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)x_2\left(t\right)}{x_4\left(t\right)}\hfill \\ & +\frac{b}{8k}\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)\hfill \\ & +\frac{1}{2}{u}_z{x}_1\left(t\right)\hfill \end{array}$$

$$\begin{array}{cc}& \frac{d}{dt}{x}_4\left(t\right)=\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)x_1\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)x_3\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{1}{2}{u}_z{x}_2\left(t\right)\hfill \end{array}$$

whereas the systems for ${\overrightarrow{x}}_{\perp }$ are

$$\begin{array}{cc}& \frac{d}{dt}{a}_1\left(t\right)=\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)\right)}{x_4^2\left(t\right)}\hfill \\ & +\frac{b}{8k}\frac{\left(\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)}{x_4^2\left(t\right)}\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)a_2\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{1}{2}{u}_z{a}_3\left(t\right)\hfill \end{array}$$

$$\begin{array}{cc}& \frac{d}{dt}{a}_2\left(t\right)=\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)a_3\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)a_1\left(t\right)}{x_4\left(t\right)}\hfill \\ & +\frac{1}{2}\frac{u_z\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)}{x_4\left(t\right)}\hfill \end{array}$$

$$\begin{array}{cc}& \frac{d}{dt}{a}_3\left(t\right)=\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)a_2\left(t\right)}{x_4\left(t\right)}\hfill \\ & +\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)\right)}{x_4^2\left(t\right)}\hfill \\ & +\frac{b}{8k}\frac{\left(\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)}{x_4^2\left(t\right)}\hfill \\ & +\frac{1}{2}{u}_z{a}_1\left(t\right)\hfill \end{array}$$

The last equation is

$$\begin{array}{cc}& -\frac{\left(-a_1\left(t\right)x_1\left(t\right)-a_2\left(t\right)x_2\left(t\right)-a_3\left(t\right)x_3\left(t\right)\right)\frac{d}{dt}{x}_4\left(t\right)}{x_4^2\left(t\right)}+\hfill \\ & \frac{-a_1\left(t\right)\frac{d}{dt}{x}_1\left(t\right)-a_2\left(t\right)\frac{d}{dt}{x}_2\left(t\right)-a_3\left(t\right)\frac{d}{dt}{x}_3\left(t\right)-x_1\left(t\right)\frac{d}{dt}{a}_1\left(t\right)-x_2\left(t\right)\frac{d}{dt}{a}_2\left(t\right)-x_3\left(t\right)\frac{d}{dt}{a}_3\left(t\right)}{x_4\left(t\right)}\hfill \\ & =-\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_1\left(t\right)+\left(-a_1\left(t\right)x_4\left(t\right)-a_2\left(t\right)x_3\left(t\right)+a_3\left(t\right)x_2\left(t\right)\right)x_4\left(t\right)\right)a_1\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{b}{8k}\frac{\left(-\left(a_1\left(t\right)x_1\left(t\right)+a_2\left(t\right)x_2\left(t\right)+a_3\left(t\right)x_3\left(t\right)\right)x_3\left(t\right)+\left(-a_1\left(t\right)x_2\left(t\right)+a_2\left(t\right)x_1\left(t\right)-a_3\left(t\right)x_4\left(t\right)\right)x_4\left(t\right)\right)a_3\left(t\right)}{x_4\left(t\right)}\hfill \\ & -\frac{1}{2}{u}_z{a}_2\left(t\right)\hfill \end{array}$$

In this case, if one replaces the derivatives of $a_1,a_2,a_3$ in the fourth equation, one does not obtain an identity, but instead a contradiction, unless that $b=0$. The same can be said for the general case $\overrightarrow{\lambda }\left(t\right)=a\overrightarrow{x}\left(t\right)+b{\overrightarrow{x}}_{\perp }\left(t\right)$.

5.5.2. Closed-Loop Bellman Dynamics

Consider now the closed-loop feedbacks case, so one must write the Hamilton–Jacobi–Bellman equation for the optimization problem (177)

$$\underset{u_x,u_y}{max}\left\{k\left(u_x^2+u_y^2\right)+\frac{1}{2}\sum _{i=1}^4\sum _{j=1}^4\frac{\partial S}{\partial x_i}{H}_{ij}\left(\overrightarrow{u}\right)x_j\right\}=-\frac{\partial S}{\partial t}$$

(206)

where $H_{ij}\left(\overrightarrow{u}\right)$ are the matrix elements of the Hamiltonian matrix in (181). The maximization, with respect to the control, gives

$$\begin{array}{cc}\hfill & 2k{u}_x+\frac{1}{2}\sum _j\frac{\partial S}{\partial x_i}{\left(H_x\right)}_{ij}{x}_j=0\hfill \\ \hfill & 2k{u}_y+\frac{1}{2}\sum _j\frac{\partial S}{\partial x^i}{\left(H_y\right)}_{ij}{x}_j=0\hfill \end{array}$$

(207)

or

$$\begin{array}{cc}\hfill & u_x=-\frac{1}{4k}{(\nabla S)}^t{H}_x\overrightarrow{x}=-\frac{x_1\frac{\partial S}{\partial x_4}+x_2\frac{\partial S}{\partial x_3}-x_3\frac{\partial S}{\partial x_2}-x_4\frac{\partial S}{\partial x_1}}{4k}\hfill \\ \hfill & u_y=-\frac{1}{4k}{(\nabla S)}^t{H}_y\overrightarrow{x}=-\frac{x_1\frac{\partial S}{\partial x_2}-x_2\frac{\partial S}{\partial x_1}+x_3\frac{\partial S}{\partial x_4}-x_4\frac{\partial S}{\partial x_3}}{4k}\hfill \end{array}$$

(208)

The HJB equation becomes

$$\begin{array}{cc}\hfill \frac{1}{16k}(& 8k{u}_z{x}_1\frac{\partial S}{\partial x_3}-8k{u}_z{x}_2\frac{\partial S}{\partial x_4}-8k{u}_z{x}_3\frac{\partial S}{\partial x_1}+8k{u}_z{x}_4\frac{\partial S}{\partial x_2}\hfill \\ \hfill & -x_1^2{\left(\frac{\partial S}{\partial x_2}\right)}^2-x_1^2{\left(\frac{\partial S}{\partial x_4}\right)}^2+2x_1{x}_2\frac{\partial S}{\partial x_1}\frac{\partial S}{\partial x_2}\hfill \\ \hfill & -2x_1{x}_2\frac{\partial S}{\partial x_3}\frac{\partial S}{\partial x_4}+2x_1{x}_4\frac{\partial S}{\partial x_1}\frac{\partial S}{\partial x_4}\hfill \\ \hfill & +2x_1{x}_4\frac{\partial S}{\partial x_2}\frac{\partial S}{\partial x_3}-x_2^2{\left(\frac{\partial S}{\partial x_1}\right)}^2-x_2^2{\left(\frac{\partial S}{\partial x_3}\right)}^2\hfill \\ \hfill & +2x_2{x}_3\frac{\partial S}{\partial x_1}\frac{\partial S}{\partial x_4}+2x_2{x}_3\frac{\partial S}{\partial x_2}\frac{\partial S}{\partial x_3}-x_3^2{\left(\frac{\partial S}{\partial x_2}\right)}^2\hfill \\ \hfill & -x_3^2{\left(\frac{\partial S}{\partial x_4}\right)}^2-2x_3{x}_4\frac{\partial S}{\partial x_1}\frac{\partial S}{\partial x_2}+2x_3{x}_4\frac{\partial S}{\partial x_3}\frac{\partial S}{\partial x_4}\hfill \\ \hfill & -x_4^2{\left(\frac{\partial S}{\partial x_1}\right)}^2-x_4^2{\left(\frac{\partial S}{\partial x_3}\right)}^2)=-\frac{\partial S}{\partial t}\hfill \end{array}$$

(209)

Because the Lagrangian multiplier is $\overrightarrow{\lambda }=\nabla S$, then by Equation (190), one could try to write

$$\nabla S=a\overrightarrow{x}+b{\overrightarrow{x}}_{\perp }$$

(210)

or

$$\begin{array}{cc}\hfill \frac{\partial S}{\partial x_1}=& a{x}_1+b{a}_1\left(\overrightarrow{x}\right)\hfill \\ \hfill \frac{\partial S}{\partial x_2}=& a{x}_2+b{a}_2\left(\overrightarrow{x}\right)\hfill \\ \hfill \frac{\partial S}{\partial x_3}=& a{x}_3+b{a}_3\left(\overrightarrow{x}\right)\hfill \\ \hfill \frac{\partial S}{\partial x_4}=& a{x}_4-b\left(\frac{x_1{a}_1\left(\overrightarrow{x}\right)+x_2{a}_2\left(\overrightarrow{x}\right)+x_3{a}_3\left(\overrightarrow{x}\right)}{x_4}\right)\hfill \end{array}$$

(211)

for some functions $a_1$, $a_2$, $a_3$. In order to integrate S, one can choose $a_1\left(\overrightarrow{x}\right)=a_2\left(\overrightarrow{x}\right)=a_3\left(\overrightarrow{x}\right)=F\left(x_4\right)$, so

$$\begin{array}{cc}\hfill \frac{\partial S}{\partial x_1}=& a{x}_1+bF\left(x_4\right)\hfill \\ \hfill \frac{\partial S}{\partial x_2}=& a{x}_2+bF\left(x_4\right)\hfill \\ \hfill \frac{\partial S}{\partial x_3}=& a{x}_3+bF\left(x_4\right)\hfill \\ \hfill \frac{\partial S}{\partial x_4}=& a{x}_4-b\frac{F\left(x_4\right)}{x_4}(x_1+x_2+x_3)\hfill \end{array}$$

(212)

By integrating S from the three first equations, one obtains

$$S=\frac{1}{2}a(x_1^2+x_2^2+x_3^2+x_4^2)+bF\left(x_4\right)(x_1+x_2+x_3).$$

By replacing S in the fourth equations, one has

$$a{x}_4+b\frac{dF\left(x_4\right)}{d{x}_4}(x_1+x_2+x_3)=a{x}_4-b\frac{F\left(x_4\right)}{x_4}(x_1+x_2+x_3)$$

(213)

or

$$\frac{dF\left(x_4\right)}{d{x}_4}=-\frac{F\left(x_4\right)}{x_4}$$

(214)

so

$$F\left(x_4\right)=\frac{1}{x_4}$$

(215)

Thus, the S function has the structure

$$S=\frac{1}{2}a(x_1^2+x_2^2+x_3^2+x_4^2)+b\frac{(x_1+x_2+x_3)}{x_4}$$

(216)

Now, one must verify that the S function above is the solution of the HJB equation. By replacing (216) in (209), one obtains

$$\begin{array}{cc}\hfill \frac{b}{16k{x}_4^4}(& -b{x}_1^4-2b{x}_1^3{x}_2-2b{x}_1^3{x}_3-b{x}_1^2{x}_2^2-2b{x}_1^2{x}_2{x}_3+2b{x}_1^2{x}_2{x}_4-2b{x}_1^2{x}_3^2-3b{x}_1^2{x}_4^2+\hfill \\ & 2b{x}_1{x}_2^2{x}_4-2b{x}_1{x}_2{x}_3^2-2b{x}_1{x}_3^3-4b{x}_1{x}_3{x}_4^2+2b{x}_1{x}_4^3-b{x}_2^2{x}_3^2-2b{x}_2^2{x}_3{x}_4\hfill \\ & -2b{x}_2^2{x}_4^2-2b{x}_2{x}_3^3-2b{x}_2{x}_3^2{x}_4-b{x}_3^4-3b{x}_3^2{x}_4^2-2b{x}_3{x}_4^3-2b{x}_4^4+4k{u}_z{x}_1{x}_2{x}_4^2\hfill \\ & +4k{u}_z{x}_1{x}_4^3+4k{u}_z{x}_2^2{x}_4^2+4k{u}_z{x}_2{x}_3{x}_4^2-4k{u}_z{x}_3{x}_4^3+4k{u}_z{x}_4^4)=0\hfill \end{array}$$

(217)

so the only possibility is $b=0$, so there is no ${\overrightarrow{x}}_{\perp }$ component in the solution. Thus, finally,

$$S=\frac{1}{2}a(x_1^2+x_2^2+x_3^2+x_4^2)$$

(218)

and the controls (208) give $u_x=0$, $u_y=0$, from which the equations of motion for the $\overrightarrow{x}$ vector is again given by the system (203). Thus, open- and closed-loop strategies give the same answer for the dynamics, because this example belongs to the regular subclass [26] of optimal control problems mentioned at the end of the Section 5.2.

6. Conclusions

In this article, we developed an optimal control perspective on the dynamical behaviour of classical and quantum physical systems. The most crucial element of this view is the presence of feedback characterized by open- or closed-loop strategies in the system.

Thus, in quantum theory, the closed-loop strategies appear naturally due to thinking that Heisenberg’s commutation relations are a constraint in a non-commutative phase-space. Hence, this implies a relation between any quantum state’s momentum and particle position.

By taking the classical limit $\hslash \to 0$ in the full quantum Hamilton–Jacobi Equation, one arrives at the closed-loop dynamics associated with the classical Hamilton–Jacobi theory. The non-commutative character of quantum theory (generated by quantum constraint) is transferred to the classical theory through the closed-loop $p_x=\frac{\partial S(x,t)}{\partial x}$ strategy. Since $S(x,t)$ satisfies the classical Hamilton–Jacobi Equation, the dynamics generated by $S(x,t)$ (under the properties of canonical transformations, whose generator is just $S(x,t)$) are completely equivalent to those open-loop dynamics dictated by the Hamiltonian equations of motion.

From a purely classical point of view, these closed-loop strategies can be explained by the canonical transformation theory’s “two observers” character. If the solutions of the equations of motion are constant for the second observer, then their solutions look like constraints for the first one. That necessarily relates the momentum of the particle with its position for the first observer, generating, in this way, the closed-loop $p_x=\frac{\partial S(x,t)}{\partial x}$ strategy.