Invariant-Based Inverse Engineering for Balanced Displacement of a Cartpole System

Abstract

Adiabaticity is a key concept in physics, but its applications in mechanical and control engineering remain underexplored. Adiabatic invariants ensure robust dynamics under slow changes, but they impose impractical time limitations. Shortcuts to Adiabaticity (STA) overcome these limitations by enabling fast operations with minimal final excitations. In this work, we set a STA strategy based on dynamical invariants and inverse engineering to design the trajectory of a cartpole, a system characterized by its instability and repulsive potential. The trajectories found guarantee a balanced transport of the cartpole within the small oscillations regime. The results are compared to numerical simulations with the exact non-linear model to set the working domain of the designed protocol.

1. Introducton

Underactuated systems, characterized by having fewer control inputs than degrees of freedom, are prevalent in a multitude of applications, ranging from robotics and aerospace to biomechanics [1,2]. These systems present significant control challenges due to inherent nonlinearities and limited actuation capabilities. Consequently, the development of effective control strategies for these systems, which are crucial for many economically relevant areas, has been an active field of research over the last decades.

The cartpole system has become a widely recognized benchmark for evaluating control strategies for underactuated systems. This inverted pendulum setup, consisting of a pole hinged to a cart moving along a horizontal track (see Figure 1), has been intensively studied in control theory because of its direct applications in various areas such as personal transport devices [3] and robotics [4].

Control strategies based on PID control have been combined with genetic algorithms [5] and with LQR control [6,7]. PD controllers have also been applied to cartpoles moving along variable slope surfaces [8]. Novel sliding mode control approaches have been specifically designed for the cartpole [9,10] and intelligent control strategies; approaches based in fuzzy logic [11,12] or artificial neural networks [13] have been used.

Moreover, control strategies are a usual benchmark for reinforcement learning algorithms and neouromorphic computing [14]. Recently, these approaches have allowed to extend the cartpole system to the quantum realm, where classical machine learning techniques have been used to balance the system [15]. Here, we take the same path but in the opposite direction. We apply control techniques originally developed for quantum systems to the classical cartpole system.

The concept of adiabaticity is ubiquitous in physics, but it is not fully exploited in control engineering. Adiabatic theorems set the existence of approximate adiabatic invariants, such as the action integral in classical mechanics, when the control parameters vary slowly enough in time [16]. “Shortcuts To Adiabaticity” (STA) is a set of control methods developed to reach the same results of an adiabatic protocol in short times [17,18]. Adiabaticity is often used to drive systems in a robust manner. An example is a load hanging from an overhead crane. If the motion is slow enough, the energy of the pendulum is an adiabatic invariant and stays constant. In particular, if the load starts at the minimum energy configuration, this state is preserved throughout the operation [19,20,21]. STA accelerate the process and still achieve the same result, avoiding issues related to long operation times such as accumulation of random and/or uncontrollable perturbations.

STA methods have been succesfully applied to a variety of control operations with quantum systems: quantum computation [22,23,24,25], cooling [26,27], quantum transport [28,29], quantum state preparation [30,31,32,33], manipulation of cold atoms [34,35,36,37,38,39], or control of polyatomic molecules [40]. Perhaps surprisingly, because of the differing orders of magnitude involved, the dynamics of the cartpole are closely related to those of a microscopic particle transport in moving traps [28,41]. In both domains, the linearized models imply the displacement of an inverted harmonic oscillator.

Cartpole control tasks usually correspond to either swing-up operations, where the pole moves from a downwards vertical position to the unstable equilibruim point at the vertical upwards position, or to balance operations, where the goal is to mantain the system at the upwards position. Here, besides aiming for balance, we consider a specific dynamical task, the displacement of the cart such that the cartpole starts in the upwards vertical positions at certain location and finishes at a different position in the same balanced configuration. The design of the control operation is far from trivial since we deal with a repulsive potential with unstable equilibrium [42]. Our STA approach is presented here without feedback, but it may be combined with feedback control techniques.

The article is organized as follows. The physical model and Hamiltonian of the system are set in Section 2, both in exact form and in the small oscillation regime. In Section 3, dynamical invariants are identified and the STA protocol is designed. Numerical results are presented in Section 4 and finally, in Section 5, we end with the conclusions and discuss some open questions.

2. Physical Model and Basic Equations

The physical model and relevant parameters are shown in Figure 1. The model assumes several conditions and idealizations: (i) the mass of the wires and friction are neglected; (ii) point masses; (iii) constant wire lengths l; (iv) the cartpole position is treated as a control parameter rather than a dynamical variable. This last assumption is a common simplification [43], often used to facilitate control design, but it requires a well-designed controller to ensure accurate implementation. A more fundamental approach, where the cartpole position is treated as a dynamical variable, is also be possible and would allow for a more detailed analysis of energy consumption of STA protocols as explored in [19].

In terms of the generalized angle $\theta$, the position of the load is given by its cartesian coordinates

$$\begin{array}{c}\hfill x\left(t\right)=x_0\left(t\right)+l\sin \theta \left(t\right);y\left(t\right)=l\cos \theta \left(t\right)\end{array}$$

(1)

where the cartpole’s position $x_0\left(t\right)$ is the control parameter to be engineered. The kinetic (T) and potential (V) are written as

$$\begin{array}{ccc}\hfill T& =& {\displaystyle \frac{1}{2}}m\left({\dot{x}}^2+{\dot{y}}^2\right);V=mgy,\hfill \end{array}$$

(2)

where dots represent time derivatives and where we drop the explicit time dependence from the variables for simplicity. The Lagrangian of the system can then be written as $\mathcal{L}=T-V$, from where the equation of motion of the load can be easily derived from Euler–Lagrange equations ${\displaystyle \frac{d}{dt}}{\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }\dot{\theta }}}-{\displaystyle \frac{\mathsf{\partial }\mathcal{L}}{\mathsf{\partial }\theta }}=0$,

$$\begin{array}{c}\hfill l\ddot{\theta }-g\sin \theta =-{\ddot{x}}_0\cos \theta .\end{array}$$

(3)

Using now the horizontal displacement of the mass $q=l\sin \theta$ as the new variable and assuming small oscillations, the dynamics of the system are described, to the first order in $\theta$, by the linear equation

$$\begin{array}{c}\hfill \ddot{q}-{\omega }^{2}q=-{\ddot{x}}_0\end{array}$$

(4)

which corresponds to a forced inverted harmonic oscillator with natural frequency ${\omega }^2=g/l$. It is easy to show that this equation of motion may also be derived from the Hamiltonian

$$H={\displaystyle \frac{p^2}{2m}}-{\displaystyle \frac{1}{2}}m{\omega }^2{q}^2+m{\ddot{x}}_{0}q,$$

(5)

where in this small oscillation regime $p=m\dot{q}$. This approximation linearizes the dynamical equations of motion of the system. Results found with exact and approximate dynamics are compared later to check the validity of the approximation and its limits.

3. Shortcuts to Adiabaticity

3.1. Dynamical Invariant

A dynamical invariant of a Hamiltonian system remains constant during the time evolution [44]. If I is an invariant of H, the following equation is satisfied:

$$\begin{array}{ccc}\hfill {\displaystyle \frac{dI}{dt}}& =& {\mathsf{\partial }}_{t}I+\{I,H\}=0,\hfill \end{array}$$

(6)

where $\{I,H\}$ refers to the Poisson bracket. Quadratic Hamiltonians with a linear in position term form the so-called Lewis–Leach family of Hamiltonians whose quadratic invariants are explicitly known [45]. The invariant I of the Hamiltonian of our particular system needs some minor modifications from the Lewis–Leach invariants [18,45] due to the repulsive behaviour of the potential involved. In particular, let us take the following expression for I:

$$\begin{array}{ccc}\hfill I& =& {\displaystyle \frac{1}{2m}}{\left(p-m\dot{\alpha }\right)}^2-{\displaystyle \frac{1}{2}}m{\omega }^2{\left(q-\alpha \right)}^2,\hfill \end{array}$$

(7)

whose time derivative

$$\begin{array}{ccc}\hfill {\displaystyle \frac{dI}{dt}}& =& {\displaystyle \frac{\mathsf{\partial }I}{\mathsf{\partial }t}}+{\displaystyle \frac{\mathsf{\partial }H}{\mathsf{\partial }p}}{\displaystyle \frac{\mathsf{\partial }I}{\mathsf{\partial }q}}-{\displaystyle \frac{\mathsf{\partial }H}{\mathsf{\partial }q}}{\displaystyle \frac{\mathsf{\partial }I}{\mathsf{\partial }p}}=\left(m\dot{\alpha }-p\right)\left(\ddot{\alpha }-{\omega }^2\alpha +{\ddot{x}}_0\right)\hfill \end{array}$$

(8)

is identically zero as long as the function $\alpha \left(t\right)$ satisfies the auxiliary equation

$$\begin{array}{c}\hfill \ddot{\alpha }-{\omega }^2\alpha =-{\ddot{x}}_0.\end{array}$$

(9)

Therefore, if $\alpha \left(t\right)$ satisfies the above equation, I, as defined in Equation (7), is a dynamical invariant of the Hamiltonian (5) and remains constant during the time evolution of the system. This auxiliary equation (9) is the Newton equation of motion for a forced harmonic (and inverted) oscillator. This $\alpha$ function may be regarded as an auxiliary function satisfying the same Newton equation (4). However, we impose to $\alpha$ some extra boundary conditions that guarantee zero final excitations as we see further.

3.2. Shortcut to Adiabaticity

Let us now impose the following boundary conditions (BC) for the auxiliar equation just derived:

$$\begin{array}{c}\hfill \alpha \left(t_b\right)=\dot{\alpha }\left(t_b\right)=\ddot{\alpha }\left(t_b\right)=0.\end{array}$$

(10)

where $t_b=0,t_f$ stands for boundary times. These boundary conditions guarantee that the invariant I in Equation (7) coincides with the Hamiltonian H at initial and final times. Moreover, at boundary times, the imposed BCs also imply that $\ddot{x}\left(t_b\right)=0$ so that the Hamiltonian represents the total mechanical energy of the system. Therefore, if the above BCs are satisfied, the invariant I, Hamiltonian H, and total mechanical energy E coincide at boundary times, $I\left(t_b\right)=H\left(t_b\right)=E\left(t_b\right)$. Then, if a fast finite-time process is designed so that the auxiliary function $\alpha$ satisfies boundary conditions in (10), the energy at initial and final times coincides regardless of the initial conditions since

$$\begin{array}{c}\hfill E\left(0\right)=H\left(0\right)=I\left(0\right)=I\left(t_f\right)=H_q\left(t_f\right)=E\left(t_f\right).\end{array}$$

(11)

One must be careful when considering the equality of energy at the initial and final times, given the repulsive nature of the potential. The proposed shortcut guarantees that H (and therefore the total energy) remains the same at the boundary times, independent of the initial conditions. However, due to the instability introduced by the repulsive potential, the presented shortcut is physically meaningful only if the pendulum starts in equilibrium, i.e., at rest in the vertical position. These null initial conditions are assumed throughout the rest of the work.

In the following, we show how to construct the cartpole trajectory $x_0\left(t\right)$ by means of an inverse engineering approach so that the desired conditions in (10) are satisfied.

3.3. Inverse Engineering

The inverse engineering strategy is relatively simple. From (9) and integrating twice with respect to time, we write the cartpole trajectory as

$$x_0\left(t\right)={\int }_0^t{\int }_0^{{\tau }_2}\left[{\omega }^2\alpha \left({\tau }_1\right)-\ddot{\alpha }\left({\tau }_1\right)\right]d{\tau }_{1}d{\tau }_2.$$

(12)

This function should also obey some physical boundary conditions. In particular, we must impose at final time that

$$\begin{array}{c}\hfill x_0\left(t_f\right)=d; {\dot{x}}_0\left(t_f\right)=0\end{array}$$

(13)

i.e., the transported distance and the final time smooth operation. There are, in summary, eight boundary conditions to be satisfied: the six conditions for $\alpha \left(t\right)$ given in (10), plus these final times conditions (13). Note that the conditions at initial times $x_0\left(0\right)={\dot{x}}_0\left(0\right)=0$ do not have to be imposed since they are automatically satisfied by construction.

We adopt a seventh-degree polynomial ansatz for the auxiliary function $\alpha$:

$$\begin{array}{c}\hfill \alpha \left(t\right)=\sum _{k=0}^7{a}_k{\tau }^k\end{array}$$

(14)

where $\tau =t/t_f$. The eight free parameters $a_k$ are determined by imposing the corresponding boundary conditions. This ansatz is one possible choice among many functional forms. However, we select this polynomial ansatz for its simplicity, favorable mathematical properties (such as smoothness and ease of differentiation), and its flexibility in efficiently satisfying boundary conditions.

Paramateres $a_0,\dots ,a_5$ are obtained imposing (10), while the remaining $a_6,a_7$ are obtained imposing (13). This leads to the following cartpole trajectory:

$$\begin{array}{ccc}\hfill x_0\left(t\right)& =& d\left[126{\tau }^5-420{\tau }^6+540{\tau }^7-315{\tau }^8+70{\tau }^9\right.\hfill \\ & +& \left.{\displaystyle \frac{1}{{\omega }^2{t}_f^2}}\left(-2520{\tau }^3+12600{\tau }^4-22680{\tau }^5+17640{\tau }^6-5040{\tau }^7\right)\right].\hfill \end{array}$$

(15)

Some cartpole trajectories and velocities for different parameters are shown in Figure 2.

For short times, the velocities and accelerations involved in the process are quite large, as expected, and there are several segments of braking and acceleration. On the other hand, for slow processes ($\omega t_f\gg 1$), the second term in (15) vanishes and the trajectory $x_0\left(t\right)$ tends asymptotically to

$$x_{\infty }\left(t\right)=d\left(126{\tau }^5-420{\tau }^6+540{\tau }^7-315{\tau }^8+70{\tau }^9\right).$$

(16)

In this regime, there is only one acceleration time segment up to $t_f/2$ (inflexion point of $x_{\infty }$) and a subsequent braking segment. The maximal velocity reached by the cartpole in ths asymptotic scenario is ${\dot{x}}_{\infty }(t_f/2)={\displaystyle \frac{315}{128}}\left({\displaystyle \frac{d}{t_f}}\right)$.

In spite of the high order of the polynomial ansatz, the force required to generate such a trajectory is perfectly feasible. It is also noteworthy that, within this STA scheme, the complete trajectory of the cart from initial to final process time is set beforehand. We do not need any feedback sensors such as encoders or inertial measurement units to adjust the trajectory accordingly during the process time. This significantly facilitates the actual implementation of the trajectory in the control system, where actuators like motors or servos convert control signals into physical movements of the cart, indirectly influencing the pole dynamics [10,46,47].

4. Numerical Results

Once the trajectory for the cartpole is designed, the dynamical equation of motion can be integrated either numerically in its exact form (3) or analytically in its approximated, harmonic linearized version (4). See some results in Figure 3.

It is important to underline that the designed shortcut protocol works as long as the small oscillation regime holds, i.e., as long as the swing angle $\theta$ is small throughout the transport process. As previously commented, we only consider the case where the cartpole is initially in equilibrium (null initial conditions). In this situation, a perfect shortcut should lead to the same final configuration (pendulum in equilibrium). This is indeed what happens if the linear equation of motion is integrated (red dashed lines in Figure 3). In the exact case, however, deviations from the target final configuration are observed (blue solid lines in Figure 3). For large transport distances d, short pendulum lengths l, or short process times $t_f$, anharmonic effects become more and more important, deviating from the ideal result and limiting the validity of our shortcut protocol. We study these non-linear effects in detail in the following sub-section.

Non-Linear Effects

Let us start by looking at the contributions that the different physical parameters offer to the the non-linear effects in dynamical Equation (3) with the cartpole trajectory given by (15). The role of the transport distance d and pendulum length l is clear. Larger distances d imply larger accelerations, so an increase in anharmonic effects is expected, and a larger pendulum implies a flatter potential, so the non-linear effects decrease as l increases.

The effect of the transport total time $t_f$ is not that clear at first sight. This quantity enters the differential equation as a parameter via the cartpole’s acceleration term on the right hand side of (3). It is then clear that longer processes imply lower accelerations, so that non-linear effects decrease as $t_f$ increases. But, on the other hand, $t_f$ also enters the dynamical equation as the final integration limit. Therefore, long processes also imply an increase in the anharmonic effects since for longer integration times these effects start to accumulate, leading to undesired final time excitations.

This is more clearly visualized if we write (3) using a dimensionless time $\tau =t/t_f$ as a parameter. The equation of motion (that should be now integrated from $\tau =0$ to $\tau =1$) takes the dimensionless form

$$\begin{array}{ccc}\hfill \ddot{\theta }& =& {\displaystyle \frac{g{t}_f^2}{l}}\sin \theta -{\displaystyle \frac{d}{l}}{a}_1\cos \theta -{\displaystyle \frac{d}{g{t}_f^2}}{a}_2\cos \theta ,\hfill \end{array}$$

(17)

where dots now represent derivatives with respect to $\tau$ and $a_{1,2}$ are different terms of a dimensionless acceleration

$$\begin{array}{ccc}\hfill a_1\left(\tau \right)& =& 5040{\tau }^7-17640{\tau }^6+22680{\tau }^5-12600{\tau }^4+2520{\tau }^3\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill a_2\left(\tau \right)& =& -211680{\tau }^5+529200{\tau }^4-453600{\tau }^3+151200{\tau }^2-15120\tau .\hfill \end{array}$$

(19)

The effect of each parameter is now clear: larger d or smaller l imply larger non-linear effects, whereas the effect of $t_f$ is twofold, so a compromise in this magnitude is needed.

To quantify the excitation at final time in a way that is easy to understand and visualize, we measure the final energy $\Delta E$ in terms of a fictitious angle $\Theta$. This angle is defined as the final angle when the final energy is considered (artificially) to be purely potential. This angle is a way to measure the final excitation energy in terms of a more visual quantity. Taking the pendulum vertical position as the zero of the potential energy and considering null initial conditions (pendulum initially in equlibrium), the initial energy is $E\left(0\right)=0$, so the energy difference is given by

$$\begin{array}{c}\hfill \Delta E=E\left(t_f\right)-E\left(0\right)={\displaystyle \frac{1}{2}}m{l}^2\dot{\theta }{\left(t_f\right)}^2+2mgl{\sin }^2{\displaystyle \frac{\theta \left(t_f\right)}{2}},\end{array}$$

(20)

where the values of $\theta$ and $\dot{\theta }$ at $t=t_f$ are obtained by numerically integrating (3). This final energy has contributions from both kinetic and potential energies. One can artificially consider this final energy to be purely potential by defining the ficticious angle $\Theta$ by the relation $\Delta E=2mgl{\sin }^2{\displaystyle \frac{\Theta }{2}}$ so that

$$\begin{array}{c}\hfill \Theta =2\arcsin \left[\sqrt{{\displaystyle \frac{l\dot{\theta }{\left(t_f\right)}^2}{4g}}+{\sin }^2{\displaystyle \frac{\theta \left(t_f\right)}{2}}}\right].\end{array}$$

(21)

Some results of this quantity are shown in Figure 4 in different scenarios. As previously commented, for a given configuration of transport distance d and pendulum length l, excessively short or long processes lead to undesired final excitations.

One can also numerically obtain the optimal value of each process time $t_f$ so that the resultant ficticious angle $\Theta$ is minimal for a given d-l configuration (minima of each curve in Figure 4); see some results and comments in Figure 5.

5. Conclusions

In this work, an invariant-based inverse engineering STA method is applied to design the balanced displacement of a cartpole system. The cartpole is a mechanical system characterized by its instability and repulsive potential.

First, we set the physical model and basic equations. The equation of motion derived from the Euler–Lagrange equations corresponds to a forced inverted harmonic oscillator and may also be derived from the Hamiltonian presented in Equation 5. Then, we find an invariant of motion for such a Hamiltonian. This requires some minor modifications of Lewis–Leach dynamical invariants due to the repulsive potential characterizing our system. An inverse engineering strategy is then followed in order to design the desired trajectory of the cart. It is remarkable that no feedback control is required whatsoever.

The designed transport protocol minimizes final energy excitations. Numerical simulations using the exact non-linear model allow us to check the parameter interval, or regime of validity, where our protocol works accurately. We also quantify the final excitations due its inherent non-linear behavior. These simulations validate the effectiveness of the chosen approach as long as the parameters keep within the proper regime, demonstrating its potential for practical applications in robotics and automation.

The results highlight the advantages of STA methods in overcoming the limitations of adiabatic processes, which, while robust, impose impractically long operation times. The inverse engineering approach allows us to generate trajectories that achieve fast and minimal excitation transport, expanding the applicability of STA beyond quantum and microscopic systems to classical mechanical engineering problems.

One of the key features of the STA approach we follow is its flexibility. It should be underlined that we do not really optimize the trajectory. We inverse engineer the trajectory by means of a polinomial ansatz, but of course this choice, and therefore the solutions to the inverse problem, are not unique, leaving room for further optimization based on specific performance criteria, such as robustness to different kindd of perturbations and/or energy cost and efficiency.

Although this work focuses on non-feedback control, hybrid control strategies combining STA with some feedback mechanism is an interesting open line for future research. Future work could also explore the extension of this protocol to more complex systems such as dynamical systems with higher degrees of freedom (double or triple pendulums). These extensions would further improve the robustness of the method to real-world industrial and robotic applications.