Active Inference and Functional Parametrisation: Differential Flatness and Smooth Random Realisation

Abstract

This paper is a first attempt to marry constructive nonlinear control theory techniques with active inference. Specifically, we are interested in the relationship between differential flatness and the design of generative models for use in control settings. We place specific emphasis on the pathwise properties of differentially flat systems that inherit from their definition in terms of successive temporal derivatives and relate this to the use of generalised coordinates of motion in formulating continuous-time generative models in active inference. To illustrate the basic concepts, we appeal to the example of oculomotor control.

1. Introduction

Active inference is one, if not the most, promising formal framework for computational neuroscience, with many applications in a number of areas (see, e.g., [1] and the references therein). The recent developments foregrounding pathwise formulations and Bayesian mechanics—developed in [2,3] among others—furnish a principled and natural setting to address many aspects of perception, planning, and control. The central idea that we will appeal to in this paper is that (natural or artificial) creatures make use of implicit generative world models both to draw inferences about the world and to control their sensed world. It is the latter that is more salient for our current purposes. In fact, one could regard the inferential (i.e., perceptual) role of generative models as merely a method of attaining tight bounds on marginal likelihoods—the quantity of interest when optimising sensory data through action.

We here consider some conditions whose fulfilment may help to construct generative models well suited for controlling one’s environment. Crucially, conditions are in direct agreement with constructive frameworks developed in nonlinear control (see, e.g., [4,5,6]). We thus try to show how this framework, and in particular the differential flatness structural property, can be leveraged in applications of active inference to control problems.

Many insightful works have already been published in this general area, both on the links and differences between active inference and classical control schemes and on deploying active inference for control (see, e.g., [7,8,9,10,11,12,13,14] just to name a few).

At first sight, differential flatness and active inference seem quite distant frameworks. The first aims to reduce a trajectory tracking error to zero, while the second minimises surprise or variational free energy; the first is inherently deterministic, and the second naturally deals with stochastic fluctuations. We shall see that the trajectory tracking error is indeed a form of surprise and that the very definition of differential flatness can be adapted to smooth random fluctuations, particularly apt for neuroscientific applications. There are two main approaches to minimise surprise or discrepancy:

○

The first is to envision the problem as an optimisation procedure, as in active inference (but also in optimal control or model predictive control); in this respect, the goal is to fulfil an optimisation criterion, leading to the minimisation of target discrepancies.

○

The second is to derive a deterministic control (action) scheme for the goal and estimate the fluctuations online to actively compensate for them.

The first route is that taken by active inference, while the second underwrites the differential flatness approach. We shall nevertheless see that the two frameworks are not as different as one may think. Indeed, the mode dynamics enforced by free energy minimisation—when those dynamics enjoy the differential flatness property—yield a functional parametrisation linking the two schemes intimately. Moreover, this parametrisation appears to be a most fruitful and promising tool to study nonlinear dynamics in neuroscience and biology.

More broadly, here, we try to foreground some potentially useful features of our framework, which draws from dynamical control system structural properties in a typical pathwise and physically preserving formulation. In particular, we shall see how the most striking feature of differential flatness—namely, differential parametrisation—can be of use within the active inference framework. More specifically, this differential parametrisation induces, among other relations, an invertible mapping from the sensation to action.

The problem we will be interested in in Section 5 is one in which we have a generative model that describes how (we believe) some system will evolve over time as a function of some control variables. Given a desired state or path for this system, our interest is in understanding how the control variables would have to be set in order to realise that desired configuration. More specifically, we are interested in the properties a generative model should possess in order to infer the control variables that realise some goal.

This paper is structured as follows. We first provide a brief introduction to the idea of a generative model for control and introduce some of the key definitions we will appeal to subsequently. Several of these definitions turn out to rely upon differentiable fluctuations, which leads us to a consideration of the interpretation of stochastic fluctuations that we will commit to (i.e., the use of random periodic functions). Having outlined the basic structure of the sorts of generative models we are interested in, we consider the way in which the principles that underwrite active inference constrain choices we might make when designing these models and equipping them with goals. We use the forms of the key objective functions from active inference (variational and expected free energy) to consider some of the generic properties of good generative models and consider whether these properties can be motivated by appealing to the notion of differential flatness. Following these theoretical considerations, we demonstrate the way these ideas work in practice through a worked example based upon oculomotor control. We conclude with a discussion of the relationship between some of these ideas and active inference, with a particular focus on the notion of generalised coordinates of motion, which inherit from similar ideas to that of differential flatness but turn out to play a very different role.

2. Preliminary Notions: Generative Model, Action, State, and Fluctuation Choice

2.1. Generative Models

As outlined above, we are interested in control problems that are formulated in terms of generative models. As an example to make this clear, which will be unpacked in much greater detail later on, consider how we might go about controlling the positions of our eyes such that we achieve a specific fixation point on a surface or track a moving point. To decide how to move our eyes, our brains might employ a model in which action variables ($\mathit{u}$), like the contraction of extra-ocular muscles, might influence the dynamics of states ($\mathit{x}$), such as the angle of our gaze or fixation point, and in which these states return some observable (e.g., visual) data ($\mathit{y}$). We now precisely outline the notion of a generative model.

Definition 1 (Generative model).

A generative model is a set of stochastic differential equations of the following form:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u},\mathit{\theta })+{\mathit{\zeta }}_{\mathit{x}}\hfill \end{array}$$

(1a)

$$\begin{array}{cc}\hfill \mathit{y}& =h(\mathit{x},\mathit{\theta })+{\mathit{\zeta }}_{\mathit{y}}\hfill \end{array}$$

(1b)

where $\mathit{x}\left(t\right)\in {\mathbb{R}}^n$ is a vector representing the state of the system (e.g., oculomotor apparatus) to be controlled (see Definition for a precise definition), $\mathit{u}\left(t\right)\in {\mathbb{R}}^m$ is the action vector (an external control input), the signal or instruction we might send to that system, θ is a vector of the agent’s model parameters, $\mathit{y}\left(t\right)\in {\mathbb{R}}^p$ is the output vector, the agent’s sampling of the sensor signal, and ${\mathit{\zeta }}_{\mathit{x}}\left(t\right)\in {\mathbb{R}}^n$ and ${\mathit{\zeta }}_{\mathit{y}}\left(t\right)\in {\mathbb{R}}^p$ are the state and output fluctuations, considered, in the Stratonovich sense, as smooth random functions (see Section 2.3 below). The functions f and h are here supposed to be meromorphic in their arguments, and we suppose that $\mathit{x},\mathit{y}$ belong to a functional space $\mathcal{F}$ such that $\mathcal{F}⫋C^0\left(\mathbb{R}\right)$, with $C^0\left(\mathbb{R}\right)$ being the space of continuous functions in time.

It is often convenient, in applications of active inference, to write a generative model in terms of generalised coordinates of motion. These coordinates are the coefficients of a Taylor series expansion of variables around the current time. In doing so, we replace a single nonlinear stochastic differential equation (for x) $\stackrel{.}{x}=f(x,u,\theta )+{\zeta }_x$ with a set of locally linearised equations for each order of motion: ${\stackrel{.}{x}}^{\left(n\right)}=f^{\left(n\right)}(x^{\left(n\right)},u,\theta )+{\zeta }_x^{\left(n\right)}$, where the superscript in brackets indicates the order of motion (i.e., the term in the Taylor series to which the coefficient belongs or the order of the temporal derivative one must take to get to these variables from our initial variable).

The motivation for this formulation is twofold. First, it allows one to express noise processes of varying smoothness in terms of the covariance between the fluctuations assigned to each order of motion. Second, through applications of the chain rule, it allows one to determine the gradients of generalised coordinates of y with respect to u via the gradients of the generalised coordinates of x. This will be important later, when we look to the way in which descent of variational free energy gradients—by changing u—can be understood as analogous to the enaction of spinal reflex arcs that bring proprioceptive data in line with anticipated setpoints.

The need to find mappings from sensation to action is one of the key ideas common to active inference—using generalised coordinates of motion—and differential flatness as applied to control. It is interesting that both approach this problem by appealing to successive derivatives of the variables in a generative model.

Definition 2 (Mean generative model). 

Given a generative model as above, the mean, or deterministic, generative model is given by the following set of differential equations:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u},\mathit{\theta })\hfill \end{array}$$

(2a)

$$\begin{array}{cc}\hfill \mathit{y}& =h(\mathit{x},\mathit{\theta })\hfill \end{array}$$

(2b)

It thus corresponds to the deterministic part of the corresponding stochastic differential equation system.

2.2. Action, Output, and State

We here give precise definitions for the notions of action, or control input, output, state, and realisation, which will be useful in Section 6.2. These were given in a differential algebraic setting in [4]; see also [15] for an elementary treatment, analogous to the one below (although stated in a deterministic setting). Let us note that the subsequent definitions, especially the differential flatness definitions, are made in the presence of fluctuations. This is not to say that we lose the deterministic character of this notion, since these perturbations may well be deterministic functions. They may also be stochastic, if they are sufficiently smooth, i.e., differentiable up a certain order.

Note that the action $\mathit{u}\left(t\right)$, or the control input, are functions enabling us to act on the system in order to fulfil a specified goal. The dynamics equations thus form an undetermined system of differential equations, since the control functions $\mathit{u}\left(t\right)$ are not a priori determined. Once the control variables are fixed (i.e., substituted with known functions of time), the  system (1) becomes determined (i.e.,  can be solved or integrated).

An output $\mathit{y}\left(t\right)$ is generated by the model. These outputs or observations may represent signals coming from the senses, in case the agent is a living being, or from the sensors in the case of an artificial system. More precisely, we have the following definition.

Definition 3 (action (control input) and output). 

Consider a model with variables $\mathit{z}\left(t\right)=(z_1\left(t\right),\dots ,z_r\left(t\right))$, fluctuations $\mathit{\zeta }\left(t\right)=({\zeta }_1\left(t\right),\dots ,{\zeta }_s\left(t\right))$, and equations

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\rho }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_{\mathit{\zeta }}\right)})& =0\hfill \end{array}$$

(3)

A control input, or action, of the model is an m-tuple (with $m<r$) of variables $\mathit{u}\left(t\right)=(u_1\left(t\right),\dots ,$$u_m\left(t\right))$ with the following properties.

• Endogenous character.
  The components of $\mathit{u}$ can be (locally and generically) expressed as a function of $\mathit{z}$ and its derivatives:

  $$\begin{array}{cc}\hfill u_i& ={\psi }_{ui}(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left(r_{ui}\right)})\hfill \end{array}$$

  (4)
• Differential independence.
  There does not exist any nontrivial differential relation of the form

  $$\begin{array}{cc}\hfill R_u(\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\beta }_u\right)})& =0\hfill \end{array}$$

  (5)
• Any system variable is influenced by the input.
  Every $z_i$, $i=1,\dots ,r$ satisfies

  $$\begin{array}{cc}\hfill E_i(z_i,{\stackrel{.}{z}}_i,\dots ,z_i^{\left({\delta }_{zi}\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\delta }_{ui}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\delta }_{\zeta i}\right)})& =0\hfill \end{array}$$

  (6)

An output $\mathit{y}\left(t\right)=(y_1\left(t\right),\dots ,y_p\left(t\right))$ (with $p<r$) is a p-tuple, where the $y_i$s are functions of the model’s variables:

$$\begin{array}{cc}\hfill y_i& =h_{y_i}(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\nu }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\nu }_{\zeta }\right)})\hfill \end{array}$$

(7)

Remark 1 (Local genericity). 

Generally, in (4), we have to deal with implicit functions:

$$\begin{array}{cc}\hfill \mathsf{\Psi }(\mathit{u},\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left(r_u\right)})& =0\hfill \end{array}$$

(8)

And, locally, in the neighbourhood of generic regular points (i.e., points where the Jacobian of Ψ with respect to $\mathit{u}$ is regular), we can solve (8) for $\mathit{u}$ using the implicit function theorem. This will be termed locally and generically here and in the forthcoming definitions.

The state variables represent the instantaneous memory of the system: once the control (action) variables have been determined, the knowledge of the state variables (at time t) enables the prediction of the future state (at time $t+dt$). A complementary formulation is the following: the state of a dynamical system is a set of physical quantities, the specification of which (in the absence of external excitation) completely determines the evolution of the system. More precisely, we have the following definition.

Definition 4 (state). 

Consider a model with variables  $\mathit{z}\left(t\right)=(z_1\left(t\right),\dots ,z_r\left(t\right))$, input $\mathit{u}\left(t\right)=(u_1\left(t\right),\dots ,$$u_m\left(t\right))$, fluctuations $\mathit{\zeta }\left(t\right)=({\zeta }_1\left(t\right),\dots ,{\zeta }_s\left(t\right))$, and equations

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\rho }_z\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\rho }_u\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_{\zeta }\right)})& =0\hfill \end{array}$$

(9)

A state of the model is an n-tuple (with $n⩽r$) of variables $\mathit{x}\left(t\right)=(x_1\left(t\right),\dots ,x_n\left(t\right))$ with the following properties.

• Endogenous character.
  The components of $\mathit{x}$ can be (locally and generically) expressed as a function of $\mathit{z}$ and its derivatives:

  $$\begin{array}{cc}\hfill x_i& ={\psi }_{xi}(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left(r_{xi}\right)})\hfill \end{array}$$

  (10)
• Independence with respect to the input.
  There does not exist any nontrivial differential relation of the form

  $$\begin{array}{cc}\hfill R_x(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\beta }_x\right)})& =0\hfill \end{array}$$

  (11)
• Representation property.
  Every variable $\check{x}$ that is a function of $\mathit{z}$ and of its time derivatives can be expressed through $\mathit{x}$, $\mathit{u}$ and its derivatives. In other words, there exists (locally and generically) a representation of the form

  $$\begin{array}{cc}\hfill \check{x}& ={\varphi }_{\check{x}}(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\beta }_{\check{x}}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\beta }_{\zeta i}\right)})\hfill \end{array}$$

  (12)

By virtue of the above representation property, for all $i=1,\dots ,n$, the time derivatives ${\stackrel{.}{x}}_i$ are expressed through $\mathit{x}$, $\mathit{u}$ and its time derivatives. This can be otherwise stated as follows.

Definition 5 (State representation).

Consider a model with variables $\mathit{z}\left(t\right)=(z_1\left(t\right),\dots ,z_r\left(t\right))$, input $\mathit{u}\left(t\right)=(u_1\left(t\right),\dots ,$$u_m\left(t\right))$, fluctuations $\mathit{\zeta }\left(t\right)$$=({\zeta }_1\left(t\right),$$\dots ,$${\zeta }_s\left(t\right))$, equations

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\rho }_z\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\rho }_u\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_{\zeta }\right)})& =0\hfill \end{array}$$

(13)

and state $\mathit{x}$. Then, there exists a so-called state representation of the form

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_i& ={\varphi }_{x_i}(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\beta }_{x_i}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\mathit{\dots },{\mathit{\zeta }}^{\left({\beta }_{\zeta i}\right)})\hfill \end{array}$$

(14)

When the right-hand side of (14) does not depend on the derivatives of the input $\mathit{u}$ (i.e., when ${\beta }_{x_i}=0$ for all $i=1,\dots ,n$), the state representation is called classical.

A realisation of a model consists of a state and a state representation for this model, as the following definition states.

Definition 6 (Realisation).

Consider a model with variables $\mathit{z}\left(t\right)=(z_1\left(t\right),\dots ,z_r\left(t\right))$, fluctuations $\mathit{\zeta }\left(t\right)=({\zeta }_1\left(t\right),\dots ,{\zeta }_s\left(t\right))$, and equations

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\rho }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_{\zeta }\right)})& =0\hfill \end{array}$$

(15)

where $\mathit{z}$ comprises an input $\mathit{u}$ and output $\mathit{y}$: $\{u_1,\dots ,u_m,y_1,\dots ,y_p\}\subset \{z_1,\dots ,z_r\}$. A realisation of this model consists in the existence of a state $\mathit{x}$ and a state representation of the form (14) for the model.

2.3. Fluctuation Choice

An interesting point about the definitions above is that they depend upon there being differentiable (i.e., smooth or analytic) fluctuations. This means we need to think carefully about what we mean by fluctuations—an important topic generally in the study of stochastic dynamical systems. Several choices can be made for the fluctuations ${\zeta }_{*}$, $\ast \in \{\mathit{x},\mathit{y}\}$ appearing in the above generative models. These include the following:

○

Stochastic processes in the Itô sense.

○

Stochastic processes in the Stratonovich sense.

○

Nonstandard infinitesimals (see, e.g., [16,17,18]);

○

Stochastic processes with Hölder continuous sample paths, yielding radom ODEs (RODEs) (see, e.g., [19]);

○

Rough paths (see, e.g., [20]);

○

Random Fourier series (RFS) (see, e.g., [21,22] for metric and convergence properties, with extensions on Riemannian manifolds [23] and locally compact groups [24]; see also [25] for an engineer’s view).

Let us choose the latter because they may furnish a convenient ${\mathcal{C}}^{\infty }$ form of fluctuations, and the solution of random ODEs can be shown to converge to solutions of a Stratonovich stochastic differential equation (see, e.g., [25], Theorem 5.1). We may thus consider so-called periodic smooth random functions:

$$\begin{array}{cc}\hfill \zeta \left(t\right)& =a_0+\sum _{j=1}^r\left[a_{j}cos\left({\displaystyle \frac{2\pi j}{L}}t\right)+b_{j}sin\left({\displaystyle \frac{2\pi j}{L}}t\right)\right],r=\lfloor L/\lambda \rfloor \hfill \end{array}$$

(16)

where $\lambda ,L>0$, each $a_j$ and $b_j$ is an independent sample from $\mathcal{N}(0,1/(2r+1\left)\right)$ (where $\mathcal{N}(\mu ,V)$ denotes the real normal distribution of mean $\mu$ and variance V), and $\lfloor ·\rfloor$ is the floor function. This type of function is L-periodic, entire, and $(2\pi /\lambda )$-band-limited.

Then, multiplying (16) by $\sqrt{2/\lambda }$ and taking the variance to be $1/\left(\right(2r+1\left)\lambda \right)$ instead of $1/(2r+1)$, we obtain the notion of a big periodic smooth random function. Note that since $r\approx L/\lambda$, we have $2/\left(\right(2r+1\left)\lambda \right)\approx 1/L$. Thus, in the big normalisation, the random coefficients of the sum $\zeta$ have variances essentially independent of $\lambda$ as $\lambda \to 0$. Then, as $\lambda \to 0$, indefinite integrals of big smooth random functions converge with probability 1 to standard Brownian paths (see, e.g., Theorem 4.3 of [25] or Theorem 2, p. 236 of [21]). Regarding smooth random functions, the reader may also consult the very interesting works of R.J. Adler and J.E. Taylor, [26,27,28], with insightful chapters on geometry and smooth random manifolds, among others. Interestingly, this technology underwrites generative models in classical brain imaging analysis, namely, statistical parametric mapping, based upon random field theory and topological inference [29,30]. The previous convergence can also be related to the Wong–Zakai theorems (see, e.g., [31]) used in the insightful paper [32], Subsection 1.3.2.

Smooth random functions may not be an apt choice at atomic scales, where a particle’s movements are highly erratic. However, they become particularly appropriate at the cell and mesoscopic scales and most probably appropriate at macroscopic scales at which many fluctuations are generated by dynamical systems that evolve over a timescale faster than that considered for a given control problem (see [1] and the seminal observations of Stratonovich: “a certain care must be taken in replacing an actual process by Markov process, since Markov processes have many special features, and, in particular, differ from the processes encountered in radio engineering by their lack of smoothness… Any random process actually encountered in radio engineering is analytic, and all its derivatives are finite with probability one” ([33], pp. 122–124)).

Remark 2 (Wavelet random series). 

One may be tempted, in the spirit of the above, to consider wavelet random series, since wavelet expansions are better behaved than their Fourier counterparts. However, recent works by C. Esser, S. Jaffard, and B. Vede [34] (see also [35]) suggest that caution is in order; in contrast to Fourier series, the randomisation of almost every continuous function gives an almost surely nowhere locally bounded function.

We sill refer to the notion of a tube around a nominal function, i.e., for $f\left(t\right)+\zeta \left(t\right)$, where f is a deterministic function and $\zeta$ a periodic smooth random function. We shall be concerned with estimating the probability of leaving a tube of width $2\lambda$ (for $\lambda >0$). For instance, one has the following estimate (see [36]):

$$\begin{array}{c}\hfill P\left(\underset{t\in [0,T]}{sup}\zeta \left(t\right)⩾\lambda \right)⩽\left({\displaystyle \frac{\sigma }{\sqrt{2\pi }\lambda }}+{\displaystyle \frac{T\sqrt{{\lambda }_2}}{2\pi \sigma }}\right)exp\left({\displaystyle \frac{-{\lambda }^2}{2{\sigma }^2}}\right)\end{array}$$

with $\sigma =\sqrt{E\left(\right|\zeta \left(t\right){|}^2)}$, ${\lambda }_2=E\left(\right|\stackrel{.}{\zeta }\left(t\right){|}^2)$. The reader may consult [36] for generalisations of this bound inequality. The notable paper by [37] also contains results for the volume of tubes using Gaussian–Minkowski functionals.

Example 1 (Smooth random functions). 

The following two plots are examples of smooth random functions with varying wavelength λ. More precisely, the left plot is a smooth random function (a sum of sines and cosines of the form (16)) with parameters $L=8$, $\lambda =2$; recall that the sum has $r+1$ terms, with  $r=\lfloor L/\lambda \rfloor$. Thus, when $L=8$, $\lambda =2$, we obtain $r=4$ terms in the sum. The right plot is an analogous sum with $r=\lfloor 8/0.1\rfloor =80$ terms (See Figure 1).

In order to offer a more concrete intuition as to the notions of state, control input, and output, let us consider a simple, although generic, example.

Example 2 (Simple generic example; model). 

Consider the following simple example with scalar action (control) and sensor output:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =x_2+{\zeta }_{x_1}\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f(x_1,x_2)+u+{\zeta }_{x_2}\hfill \end{array}$$

(17b)

$$\begin{array}{cc}\hfill y& =x_1+{\zeta }_y\hfill \end{array}$$

(17c)

where f is a smooth function: $f\in C^{\infty }\left({\mathbb{R}}^2\right)$. The fluctuations are taken as smooth random Gaussian functions:

$$\begin{array}{cccc}\hfill {\zeta }_{x_1}\left(t\right)& =\sum _{k⩾0}{a}_{k_1}{\varphi }_k\left(t\right),\hfill & \hfill a_{k_1}& =\mathcal{N}(0,{\sigma }_x)\hfill \end{array}$$

(18a)

$$\begin{array}{cccc}\hfill {\zeta }_{x_2}\left(t\right)& =\sum _{k⩾0}{a}_{k_2}{\varphi }_k\left(t\right),\hfill & \hfill a_{k_2}& =\mathcal{N}(0,{\sigma }_x)\hfill \end{array}$$

(18b)

$$\begin{array}{cccc}\hfill {\zeta }_y\left(t\right)& =\sum _{k⩾0}{c}_k{\varphi }_k\left(t\right),\hfill & \hfill c_k& =\mathcal{N}(0,{\sigma }_y)\hfill \end{array}$$

(18c)

$$\begin{array}{cccc}\hfill {\varphi }_k\left(t\right)& =cos\left({\displaystyle \frac{2\pi j}{L}}t\right),\hfill & \hfill r& =\lfloor L/\nu \rfloor \hfill \end{array}$$

(18d)

with $\nu ,L>0$. Here (2) are the equations of a generative model, in the sense of Definition 1, $\mathit{x}=(x_1,x_2)$ is the state of the generative model, in the sense of Definition 4, u is the input, and y is an output, in the sense of Definition 3.

This type of model includes all models predicated upon Newton’s law of motion, such as

$$\begin{array}{cc}\hfill M\stackrel{..}{y}& =F(y,\stackrel{.}{y})+v\hfill \end{array}$$

(19)

where M is a mass, F is a model for internal forces depending on the position y and velocity $\stackrel{.}{y}$, and v is an external force.

A more general case would be the following:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =f_1(x_1,x_2)+{\zeta }_{x_1}\hfill \end{array}$$

(20a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f_2(x_1,x_2,x_3)+{\zeta }_{x_2}\hfill \end{array}$$

(20b)

$$⋮$$

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_n& =f(x_1,\dots ,x_n)+u+{\zeta }_{x_n}\hfill \end{array}$$

(20c)

$$\begin{array}{cc}\hfill y& =x_1+{\zeta }_y\hfill \end{array}$$

(20d)

with $f_i$s being smooth functions of their arguments, invertible with respect to the final function. All the computations made for the simple example (17) can be readily extended to the above.

3. Free Energy, Flatness, and Conceptual Similarities

3.1. Free and Expected Free Energy

One can understand both active inference- and differential flatness-informed control schemes as identifying mappings from sensation to action from models that detail the influence of action on sensation. Flatness rests upon there being an invertible mapping between action and sensation such that a desired sensory trajectory uniquely determines the actions that generate it. Active inference involves the selection of actions that bring sensations in line with the mode of a marginal density of sensory data implied by a generative model. This is mediated by reflexive actions determined by sensory data. The marginal density that identifies desired sensory trajectories is often specified in terms of an expected free energy—whose role is to determine prior plausibilities of alternative action sequences based upon their capacity to minimise the Kullback–Lieber divergence (also known as risk) between desired and anticipated sensory trajectories.

Consistent with control-theoretic formulations of the sort outlined above, active inference can be formulated as optimising a functional of a model that relates controllable variables to some observable outcomes. Specifically, it depends upon optimisation (minimisation) of a variational free energy that acts as an upper bound on the surprise or negative log marginal likelihood of those observations. Variational free energy can be formulated in several ways to quantify the performance of a system engaging in active inference in terms of energies (i.e., surprise) and divergences (i.e., relative entropies):

$$\begin{array}{l}\mathcal{F}\left(\mathit{u}\right(t),\mathit{y}(t\left)\right)=\hfill \\ \hfill & \underset{\mathrm{Energy}}{\underbrace{-\int q(\mathit{x}(t)|\mathit{u}(t))lnp(\mathit{x}(t),\mathit{y}(t)|\mathit{u}(t))d\mathit{x}(t)}}+\underset{\mathrm{Neg}\text{-}\mathrm{entropy}}{\underbrace{\int q(\mathit{x}(t)|\mathit{u}(t))lnq(\mathit{x}(t)|\mathit{u}(t))d\mathit{x}(t)}}\hfill \end{array}$$

(21a)

$$\begin{array}{cc}\hfill & =\underset{\mathrm{Divergence}}{\underbrace{\int q(\mathit{x}(t)|\mathit{u}(t))ln{\displaystyle \frac{q(\mathit{x}(t)|\mathit{u}(t))}{p(\mathit{x}(t)|\mathit{y}(t),\mathit{u}(t))}}d\mathit{x}(t)}}+\underset{\mathrm{Surprise}}{\underbrace{\left(-lnp(\mathit{y}(t)|\mathit{u}(t))\right)}}\hfill \end{array}$$

(21b)

$$\begin{array}{cc}\hfill & =\underset{\mathrm{Complexity}}{\underbrace{\int q(\mathit{x}(t)|\mathit{u}(t))ln{\displaystyle \frac{q(\mathit{x}(t)|\mathit{u}(t))}{p(\mathit{x}(t)|\mathit{u}(t))}}d\mathit{x}(t)}}-\underset{\mathrm{Accuracy}}{\underbrace{\int q(\mathit{x}(t)|\mathit{u}(t))lnp(\mathit{y}(t)|\mathit{x}(t),\mathit{u}(t))d\mathit{x}(t)}}\hfill \end{array}$$

(21c)

The above presents variational free energy $\mathcal{F}$ as a functional of two probability distributions (for discrete states) or densities (for continuous states). The density labelled p is that associated with our generative model, while q represents a density variously referred to as a recognition density, an approximate posterior density, or a variational density. For the purposes of this paper, the variational density is assumed to have already been optimised such that $q\left(\mathit{x}\right(t\left)\right|\mathit{u}\left(t\right))=p(\mathit{x}\left(t\right)\left|\mathit{y}\right(t),\mathit{u}(t\left)\right)$. Each formulation of the free energy depends upon different factorisations of the generative model. When expressed as a joint density, the minimisation of free energy can be seen as a constrained maximum entropy problem. On factorising into conditionals and marginal likelihoods, the free energy is seen to be an upper bound on surprise —the negative log marginal likelihood, Bayesian model evidence, or improbability of an observation under a given generative model. Finally, factorising a generative model into priors and likelihoods gives us a balance between complexity—how far we must move from prior beliefs to explain observations — and the accuracy with which we can account for sensory inputs.

Further to this, one can formulate expected free energies that highlight other forms of discrepancy that are especially relevant for the optimisation of control. When formulated explicitly in a pathwise setting, we have

$$\begin{array}{cc}\hfill \mathcal{G}\left(\mathit{u}\right(t\left)\right)& =-\underset{\mathrm{Mutual}\mathrm{information}}{\underbrace{\int \int q(\mathit{x}\left(t\right),\mathit{y}\left(t\right)|\mathit{u}\left(t\right))ln{\displaystyle \frac{q\left(\mathit{x}\right(t),\mathit{y}(t)|\mathit{u}\left(t\right))}{p\left(\mathit{x}\right(t),\mathit{y}(t)|\mathit{u}\left(t\right))}}d\mathit{x}\left(t\right)d\mathit{y}\left(t\right)}}-\hfill \end{array}$$

$$\begin{array}{cc}\hfill & \underset{\mathrm{Expected}\mathrm{value}}{\underbrace{\int q(\mathit{y}\left(t\right)|\mathit{u}\left(t\right))lnp(\mathit{y}\left(t\right)|{\mathit{y}}_r\left(t\right))d\mathit{y}\left(t\right)}}\hfill \\ \hfill & =\underset{\mathrm{Risk}}{\underbrace{\int q(\mathit{y}\left(t\right)|\mathit{u}\left(t\right))ln{\displaystyle \frac{q\left(\mathit{y}\right(t\left)\right|\mathit{u}\left(t\right))}{p(\mathit{y}\left(t\right)|{\mathit{y}}_r\left(t\right))}}d\mathit{y}\left(t\right)}}+\hfill \end{array}$$

(22a)

$$\begin{array}{cc}\hfill & \underset{\mathrm{Ambiguity}}{\underbrace{\left(-\int \int q(\mathit{y}(t),\mathit{x}(t)|\mathit{u}(t))lnp(\mathit{y}(t)|\mathit{x}(t),\mathit{u}(t))d\mathit{x}(t)d\mathit{y}(t)\right)}}\hfill \end{array}$$

(22b)

The expected free energy is typically used for planning, where we might integrate this quantity along future paths and assign paths of control states higher probabilities for lower expected free energies. Of particular relevance for the discussion that follows is the idea that, by placing priors over the path of observations we wish to obtain (here indicated by conditioning upon a goal ${\mathit{y}}_r$), optimisation of expected free energy involves determining the set of control paths that would realise these outcomes.

We can now identify at least three different aspects of optimality, namely surprise, inadequacy, and discrepancy:

• The discrepancy in observation (inaccuracy) in gathering information from the world. This discrepancy is implicit in the (negative) accuracy term of the variational free energy, which under Gaussian assumptions will reduce to the square of a prediction error: ${\epsilon }_{obs}=\mathit{y}-\widehat{\mathit{y}}$, where $\widehat{\mathit{y}}$ is the predicted observation under beliefs about $\mathit{x}$. This is the (sensory) prediction error that features in predictive coding or Bayesian filtering, linear quadratic control, and model predictive control.
• The discrepancy in action in the way the agent acts in the world. This discrepancy is between the goal ${\mathit{y}}_r\left(t\right)$, a predefined trajectory to be followed, and the actual position $\mathit{y}\left(t\right)$: ${\mathit{\epsilon }}_{act}=\mathit{y}-{\mathit{y}}_r$. This is implicit in the risk term of the expected free energy, which quantifies the divergence between the distribution anticipated under a set of control states and the distribution anticipated given the goal.
• The discrepancy in modelling in the way the agent represents the world internally. This discrepancy would normally be quantified using the marginal likelihood of sensory observations under the model and is reflected in the surprise term of the variational free energy. When free energy is minimal with respect to q, it becomes a tight bound on this discrepancy. For this reason, variational free energy is often used as a tractable method of approximating Bayes factors to compare alternative models in statistical inference.

The free energies above ($\mathcal{F}$ and $\mathcal{G}$) can be seen as so-called global Lyapunov functions (see, e.g., [38]) that capture these aspects seen in optimal control (see, e.g., [39,40]). In what follows, we shall mostly be interested in $\mathcal{G}$, and specifically the risk term, leaving the other aspects to future works. More precisely, we are interested in whether the notion of differential flatness coheres with the selection of generative models that optimise $\mathcal{F}$ and $\mathcal{G}$. The derivation of trajectory tracking action laws on differentially flat models will be seen to minimise the above risk in $\mathcal{G}$ (see Section 4.6).

Table 1. Notation table.

Variable	Meaning
$\mathit{x}$	Hidden state vector
$\mathit{y}$	Output vector (e.g., sensor data)
$\mathit{u}$	Action vector
${\zeta }_{\mathit{x}}$	Fluctuations in hidden state dynamics
${\zeta }_{\mathit{y}}$	Fluctuations in sensor data
$\mathcal{F}$	Variational free energy
$\mathcal{G}$	Expected free energy

summarises the notations used so far, where bold symbols are vectors.

3.2. Differential Flatness

3.2.1. Controllability

A ubiquitous notion—when one wishes to steer a system—is the global controllability, as stated in the following definition (see, e.g., [41]).

Definition 7.

A system such as (1), but idealised such that we assume the fluctuations are infinitely precise,

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f_m(\mathit{x},\mathit{u})\hfill \end{array}$$

(23a)

$$\begin{array}{cc}\hfill \mathit{y}& =h_m\left(\mathit{x}\right)\hfill \end{array}$$

(23b)

is said to be globally controllable if, for any time instants $t_0$ and $t_1$ and initial and final states ${\mathit{x}}_0$ and ${\mathit{x}}_1$, there exists an action (a control law) $\mathit{u}$ in a space of admissible controls steering the system from $\mathit{x}\left(t_0\right)={\mathit{x}}_0$ to $\mathit{x}\left(t_1\right)={\mathit{x}}_1$.

The reader may note that this definition is purely descriptive: it does not contain any constructive procedure for steering the system; it is not possible to infer, from reading the definition alone, the form of the control law to be applied to go from ${\mathit{x}}_0$ to ${\mathit{x}}_1$. This is a definition of the existence-of-a-solution kind, not of a solution construction to the given problem.

The reader may note that this definition is purely descriptive, in that it does not contain any constructive procedure for steering the system. Moreover it is pointwise in spirit, rather than pathwise. Indeed, this definition does not say anything about the path that links the initial to the final state. This path can be implausible and still fulfil the controllability requirements. We shall see in the next subsection a stronger and, in our sense, more useful property, namely, differential flatness.

3.2.2. Motivation Through Observation and Action

The premise of active inference is that an agent seeks to minimise the surprise or divergence between its beliefs or expectations about the surrounding environment and the actual state it experiences. This minimisation can, in principle, be enforced through effective information gathering and action or be engrained in the very structure of the agent’s model (such as being refined through learning and evolution).

We will examine a case where the requisite efficiency is encoded in the agent’s model structure itself. More precisely the imperatives for perception and action are directly fulfilled through the following:

(Odsf)

Observation discrepancy structural fulfilment. The state $\mathit{x}$ can be recovered through what the agent is able to know directly (i.e., without any inference, reflection, or computation), that is $\mathit{y}$, $\mathit{u}$, and their time derivatives. This amounts to the system being constructively observable.

(Adsf)

Action discrepancy structural fulfilment. The link between the goal ${\mathit{y}}_{\mu r}$ and the action $\mathit{u}$—required to reach that goal—is direct, in that the action is given as a function of the goal and its time derivatives. This amounts to the system being left-invertible.

A system such as (23) is said to be left-invertible with respect to $\mathit{z}$ if  the action $\mathit{u}$ is a function of $\mathit{z}$ and its derivatives (see Property 5 of [42]). Thus, for a dynamical system to fulfil both (Odsf) and (Adsf), one needs to have a function $\mathit{\omega }$ such that both the state $\mathit{x}$ and the action $\mathit{u}$ can be expressed in terms of $\mathit{\omega }$ and its time derivatives. This corresponds to differential flatness [5,43,44], a property shared by a great number of practical dynamical systems (see, e.g., [45] and the references therein).

3.2.3. Motivation Through Direct and Inverse Views

The fundamental property of flat systems is that all their solutions can be functionally parameterised by a finite number of functions and their time derivatives. Although flatness is a relatively recent notion, introduced in control theory in the 1990s [5,43,44], it actually has a long history: a similar notion— of systems of undetermined differential equations integrable without integration—dates back to Hilbert [46] and Cartan [47]. Indeed, the control system $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})$ can be seen as an underdetermined differential system consisting of n equations ${\stackrel{.}{x}}_i=f_i(x,u)$, for $i=1,\dots ,n$, and $n+m$ variables (n states and m action variables). The difference between the number of variables and the number of equations gives the degrees of freedom number of the system. It follows that m functions can be chosen freely. In the context of control systems, one usually chooses freely the input $\mathit{u}\left(t\right)=(u_1\left(t\right),\dots ,u_m\left(t\right))$ and then integrates in order to compute the state $\mathit{x}\left(t\right)$. But is this the only way to achieve this?

In order to answer this question, consider the following simple single-input system:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =x_2\hfill \end{array}$$

(24a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =u\hfill \end{array}$$

(24b)

We can freely choose $u\left(t\right)$, then integrate it once to compute $x_2\left(t\right)$, and then integrate it a second time to compute $x_1\left(t\right)$. Let us now choose freely $x_1\left(t\right)$, differentiate it once to get $x_2\left(t\right)$, and then differentiate $x_2\left(t\right)$ to obtain $u\left(t\right)$. It follows that the system with Equation (24) admits two functional parameterisations: one via the input, for which we have to integrate twice (which is, in the general case of $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})$, difficult and, sometimes, impossible analytically), and one via the state $x_1$, for which we have to differentiate twice (which is always possible in a straightforward way). Hence, there are underdetermined differential systems (also known as control systems) that are solvable without integration, namely, differentially flat systems.

3.2.4. Formal Definition

Let us consider the central definition of this subsection, stated for systems with specified parameters, for ease of reading (see, e.g., [15,42,45]; see also the Supplementary Material, File AIandFP-FlatnessAndSRR-HMTPKF-2026-SupplMaterial-v1.pdf, Section D and [48] for a Python library, version 0.10.2).

Definition 8 (Differential flatness).

Consider a model with variables $\mathit{z}\left(t\right)=(z_1\left(t\right),\dots ,$ $z_r\left(t\right))\in$${\mathbb{R}}^r$, fluctuations $\mathit{\zeta }\left(t\right)=({\zeta }_1\left(t\right),\dots ,$${\zeta }_s\left(t\right))\in {\mathbb{R}}^s$, and equations

$$\begin{array}{cc}\hfill \mathsf{\Phi }(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\rho }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_{\zeta }\right)})& =0\hfill \end{array}$$

(25)

The model is called differentially flat if there exists an m-tuple of variables, $\mathit{\omega }\left(t\right)=({\omega }_1\left(t\right),$$\dots ,$${\omega }_m\left(t\right))$, named flat outputs, with the following three properties.

• Endogenous character. We have (locally and generically)

  $$\begin{array}{cc}\hfill \mathit{\omega }& =h(\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\eta }_z\right)}){\eta }_z\in \mathbb{N}\hfill \end{array}$$

  (26)
  In other words, the components of the flat output are combinations of the system’s variables.
• Functional parameterisation.
  The system’s variables $\mathit{z}$ can be (locally and generically) expressed through the flat output and a finite number of its derivatives:

  $$\begin{array}{cc}\hfill \mathit{z}& =A_z(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{({\alpha }_{\zeta })})\hfill \end{array}$$

  (27)

  with ${\alpha }_z,{\alpha }_{\zeta }$ integers, such that the system’s equations

  $$\begin{array}{cc}\hfill \mathsf{\Phi }(A_z{\stackrel{.}{A}}_z,\dots ,A_z^{({\rho }_z)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{({\rho }_{\zeta })})& =0\hfill \end{array}$$

  are identically verified.
• Differential independence. The components of the flat output are differentially independent; i.e., any differential relation

  $$\begin{array}{cc}\hfill \mathsf{\Xi }(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left(\beta \right)})& =0\hfill \end{array}$$

  is necessarily trivial: $\mathsf{\Xi }\equiv 0$.

Remark 3 (Differential flatness–state-space form case). 

In case the model has the following form:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u})+\mathit{\zeta }\hfill \end{array}$$

(28)

with state $\mathit{x}\left(t\right)\in {\mathbb{R}}^n$, action $\mathit{u}\left(t\right)\in {\mathbb{R}}^m$, and fluctuation $\mathit{\zeta }\left(t\right)\in {\mathbb{R}}^n$, the functional parametrisation (27) becomes

$$\begin{array}{cc}\hfill \mathit{x}& =A(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_x\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_x\right)})\hfill \end{array}$$

(29a)

$$\begin{array}{cc}\hfill \mathit{u}& =B(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_u\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_u\right)})\hfill \end{array}$$

(29b)

Remark 4.

Note that the conventional definition of differential flatness is made for determinsitic (mean) models:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u})\hfill \end{array}$$

(30)

with state $\mathit{x}\left(t\right)\in {\mathbb{R}}^n$ and action $\mathit{u}\left(t\right)\in {\mathbb{R}}^m$, in which case the functional parametrisation (27) becomes

$$\begin{array}{cc}\hfill \mathit{x}& =A(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_x\right)})\hfill \end{array}$$

(31a)

$$\begin{array}{cc}\hfill \mathit{u}& =B(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_u\right)})\hfill \end{array}$$

(31b)

The fluctuations considered above in (29a) and (29b) are not necessarily supposed to be known functions of time, but rather are seen as generic, sufficiently differentiable functions. The relations (29) are valid whatever the fluctuations ζ may be at this structural level. See Section 3.2.5 below for a discussion on this functional parametrisation.

Remark 5 (Local genericity).

As in Remark 1, the relations of endogenous character in (26) and of functional parameterisation in (27), naturally yield implicit functional relations:

$$\begin{array}{cc}\hfill H(\mathit{\omega },\mathit{z},\stackrel{.}{\mathit{z}},\dots ,{\mathit{z}}^{\left({\eta }_z\right)})& =0\hfill \end{array}$$

(32a)

$$\begin{array}{cc}\hfill {\mathsf{\Lambda }}_z(\mathit{z},\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_z\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_{\mathit{\zeta }}\right)})& =0\hfill \end{array}$$

(32b)

And, locally, in the neighbourhood of generic regular points (i.e., points where the Jacobian of H with respect to ω and ${\mathsf{\Lambda }}_z$ with respect to $\mathit{z}$ is regular), we can solve (32a) for ω and (32b) for $\mathit{z}$ using the implicit function theorem.

Let us note that the differential flatness definition is made in the presence of fluctuations. The latter may be deterministic or smooth stochastic functions (i.e., differentiable up a certain order). We will consider relations furnishing, in particular, the action as a functional of the sensory and fluctuation paths. This enables one to study precisely and quantitatively the influence that each perturbation function may exert on the action. Moreover the relation yielding the action—respectively, the state—as a function of sensor and fluctuation paths is generic, in the sense that it is valid for any (sufficiently smooth) sensory and fluctuation paths. In this sense, the notion of differential flatness is agnostic with respect to the stochasticity of the generative model. The latter may be deterministic (as is the case for mean generative models), subject to deterministic, but unkown, perturbations or subject to stochastic (and sufficiently smooth) fluctuations. The previous definition readily affords the following characterisation:

Proposition 1.

A differentially flat system in state-space form is a system like (28), i.e., $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})$, which is observable and left invertible with respect to ω.

To summarise what has been said so far, in order to express what we think we know about the system (our beliefs) and what we wish for the system (our expectations), we require a model structure that expresses first the agent system’s constructive observability (what we know or believe) and second the agent system’s left invertibility (what we expect and where the action $\mathit{u}$ is expressed in terms of the flat output goal, ${\mathit{\omega }}_r$). The latter amounts to differential flatness, which is no more than the generator character of the flat output, i.e., the functional parameterisation and the differential polynomial independence of the flat output components. The last item enables one to choose the various components ${\omega }_{ir}\left(t\right)$ of ${\mathit{\omega }}_r\left(t\right)$ independently of each other. Both properties are reminiscent of the basis notion in vector spaces (i.e., the minimally generating and maximally independent characters); indeed, in a differential algebraic setting, the notion of flat output corresponds to a differential transcendence basis (see [5]).

An interesting point of contact with active inference is that optimisation of expected free energy functionals implies a high degree of mutual information between different components of a model (specifically, between states and observations). This implies precise mappings from actions, via states, to  observations. Crucially, that same mutual information means that we would expect observations to be highly informative about states and, possibly, actions. The potential to recover subsets of variables from others in such models is heuristically compatible with the differential flatness concept. Furthermore, the need for differential independence has an interesting link with the interpretation of variational free energy as an objective function for constrained maximum entropy inference—where in the absence of the ‘energy’ constraints, the best configuration is that with maximum entropy without mutual constraints between the components of a system.

Remark 6 (Tracking an arbitrary output). 

Suppose that the agent’s model is differentially flat. The agent’s goal ${\mathit{y}}_r$ may coincide with the flat output goal: ${\mathit{\omega }}_r={\mathit{y}}_r$. If this is not the case, consider the expression of the agent’s output $\mathit{y}$ as a function of the flat output ω and its derivatives:

$$\begin{array}{cc}\hfill \mathit{y}& =A_y(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\alpha }_y\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_{\mathit{\zeta }}\right)})\hfill \end{array}$$

(33)

and view this expression as a differential equation in ω:

$$\begin{array}{cc}\hfill {\mathit{\omega }}^{\left({\alpha }_y\right)}& ={\tilde{A}}_y(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{({\alpha }_y-1)},\mathit{y},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_{\mathit{\zeta }}\right)})\hfill \end{array}$$

(34)

Substituting $\mathit{y}$ by the goal ${\mathit{y}}_r$ yields the flat output goal ${\mathit{\omega }}_r$ as a solution of

$$\begin{array}{cc}\hfill {\mathit{\omega }}_r^{\left({\alpha }_y\right)}& ={\tilde{A}}_y({\mathit{\omega }}_r,{\stackrel{.}{\mathit{\omega }}}_r,\dots ,{\mathit{\omega }}_r^{({\alpha }_y-1)},{\mathit{y}}_r,\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\alpha }_{\mathit{\zeta }}\right)})\hfill \end{array}$$

(35)

3.2.5. Functional Parameterisation

The functional parameterisation property is an essential, if not by far the most essential feature, of differential flatness. Indeed, the original model

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u})+{\mathit{\zeta }}_{\mathit{x}}\hfill \end{array}$$

(36a)

$$\begin{array}{cc}\hfill \mathit{y}& =h\left(\mathit{x}\right)+{\mathit{\zeta }}_{\mathit{y}}\hfill \end{array}$$

(36b)

is totally equivalent to its functional parametric form:

$$\begin{array}{cc}\hfill \mathit{x}& =A_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_x\right)},\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_x\right)})\hfill \end{array}$$

(37a)

$$\begin{array}{cc}\hfill \mathit{u}& =B_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_u\right)},\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_u\right)})\hfill \end{array}$$

(37b)

$$\begin{array}{cc}\hfill \mathit{\zeta }& =({\mathit{\zeta }}_{\mathit{x}},{\mathit{\zeta }}_{\mathit{y}})\hfill \end{array}$$

(37c)

And, crucially, the functional parametric form (37) is quasi-static, whereas the original model (36) has a dynamical form. Hence, when one wishes to deal with (36), one is naturally tempted to seek the solution of the ODE system (36a), and most often, an analytical solution cannot be found. But when dealing with the functional parametric form (37), all system variables, i.e., $\mathit{x}$ and $\mathit{u}$ here, are parametrised by the function $\mathit{\omega }$, since once we know the function $\mathit{\omega }$, then we readily know the functions $\mathit{x}$ and $\mathit{u}$ through (37) (assuming, in a structural step such as this one, that the fluctuations $\mathit{\zeta }$ are known or measured).

Note that this parametrisation is a pathwise one and that it carries over the whole fluctuation; it can thus be envisioned either as a tube around the mean (see, e.g., [27,28,36]) or as a function sheaf (see, e.g., [49,50]; see also [51] for an elementary introduction and [52] for more complete while still accessible references).

Thus, the action, the control $\mathit{u}$, and the state $\mathit{x}$ are envisioned as functionals of the flat output and of the fluctuations, i.e., as functions over spaces of functions:

$$\begin{array}{cc}\mathit{x}:{\mathcal{F}}^m\hfill & \times {\mathcal{F}}^{n+p}⟶{\mathcal{F}}^n\hfill \\ \hfill (& \mathit{\omega },\mathit{\zeta })⟶A_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_x\right)},\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_x\right)})\hfill \\ \hfill \mathit{u}:{\mathcal{F}}^m& \times {\mathcal{F}}^{n+p}⟶{\mathcal{F}}^m\hfill \end{array}$$

(38a)

$$\begin{array}{cc}\hfill (& \mathit{\omega },\mathit{\zeta })⟶B_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_u\right)},\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\rho }_u\right)})\hfill \\ \hfill \mathit{\zeta }& =({\mathit{\zeta }}_{\mathit{x}},{\mathit{\zeta }}_{\mathit{y}}),\mathcal{F}\subseteq C^{\rho }\left(\mathbb{R}\right),\rho =\max ({\rho }_x,{\rho }_u)\hfill \end{array}$$

(38b)

And it is the study of these functionals that may be of great interest to the active inference community.

3.3. Conceptual Similarities

We can now see—with a simple concrete example—that both differential flatness and free energy minimisation enforce an inverse mapping from the sensed output to the action, namely, the control input.

Example 3 (Simple generic example; similarities). 

Recall the following simple example with scalar action (control) and sensor output:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =x_2+{\zeta }_{x_1}\hfill \end{array}$$

(39a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f(x_1,x_2)+u+{\zeta }_{x_2}\hfill \end{array}$$

(39b)

$$\begin{array}{cc}\hfill y& =x_1+{\zeta }_y\hfill \end{array}$$

(39c)

This model is differentially flat with flat output y. Indeed, the functional parameterisation is obtained as follows. Equation (39c) yields

$$\begin{array}{cc}\hfill x_1& =y-{\zeta }_y\hfill \end{array}$$

(40)

Equation (39a) yields

$$\begin{array}{cc}\hfill x_2& ={\stackrel{.}{x}}_1-{\zeta }_{x_1}=\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y-{\zeta }_{x_1}\hfill \end{array}$$

(41)

Then, using Equation (39b),

$$\begin{array}{cc}\hfill u& =\stackrel{..}{y}-f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y-{\zeta }_{x_1})-{\stackrel{..}{\zeta }}_y-{\stackrel{.}{\zeta }}_{x_1}-{\zeta }_{x_2}\hfill \end{array}$$

(42)

Thus, the functional parametrisation associated with the differential flatness property is

$$\begin{array}{cc}\hfill x_1& =y-{\zeta }_y\hfill \end{array}$$

(43a)

$$\begin{array}{cc}\hfill x_2& =\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y-{\zeta }_{x_1}\hfill \end{array}$$

(43b)

$$\begin{array}{cc}\hfill u& =\stackrel{..}{y}-f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y-{\zeta }_{x_1})-{\stackrel{..}{\zeta }}_y-{\stackrel{.}{\zeta }}_{x_1}-{\zeta }_{x_2}\hfill \end{array}$$

(43c)

We see in the last equation the inverse mapping from the sensory output $y\left(y\right)$ to the control input $u\left(t\right)$. Note that this relation embeds all the influences of fluctuations on the action, since it is a functional in y ${\zeta }_{x_1}$, ${\zeta }_{x_2}$, and ${\zeta }_y$.

In the case of a deterministic system,

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =x_2\hfill \end{array}$$

(44a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f(x_1,x_2)+u\hfill \end{array}$$

(44b)

$$\begin{array}{cc}\hfill y& =x_1\hfill \end{array}$$

(44c)

The parametrisation becomes

$$\begin{array}{cc}\hfill x_1& =y\hfill \end{array}$$

(45a)

$$\begin{array}{cc}\hfill x_2& =\stackrel{.}{y}\hfill \end{array}$$

(45b)

$$\begin{array}{cc}\hfill u& =\stackrel{..}{y}-f(y,\stackrel{.}{y})\hfill \end{array}$$

(45c)

Note that (44) is dynamically equivalent to (45).

Then, minimising the free energy $\mathcal{F}$ enforces the dynamical mode (assuming it is possible to set all terms of this free energy to zero, at least approximately), i.e., the dynamical model obtained without fluctuations:

$$\begin{array}{cc}\hfill {\stackrel{.}{\mu }}_{x_1}& ={\mu }_{x_2}\hfill \end{array}$$

(46a)

$$\begin{array}{cc}\hfill {\stackrel{.}{\mu }}_{x_2}& =f({\mu }_{x_1},{\mu }_{x_2})+u\hfill \end{array}$$

(46b)

$$\begin{array}{cc}\hfill y& ={\mu }_{x_1}\hfill \end{array}$$

(46c)

where ${\mu }_{\ast }$, $\ast \in \{x_1,x_2,u\}$, is the mode of ∗ under the recognition density. By virtue of the previous equivalence, (46) is dynamically equivalent to

$$\begin{array}{cc}\hfill {\mu }_{x_1}& =y\hfill \end{array}$$

(47a)

$$\begin{array}{cc}\hfill {\mu }_{x_2}& ={\stackrel{.}{\mu }}_y\hfill \end{array}$$

(47b)

$$\begin{array}{cc}\hfill u& =\stackrel{..}{y}-f(y,\stackrel{.}{y})\hfill \end{array}$$

(47c)

We now see that the minimisation of free energy enforces an (at least approximate) inverse mapping from the sensed output to the action. There is a further relationship that we will return to later, predicated upon generalised coordinates of motion, which we have not addressed here and which relates to the final equation above. As the action influences the rate of change of the states, which then influence the sensory data, the gradient of current sensory data with respect to the action is zero. This can be seen explicitly by noting that there is no term in the free energy in which sensory data and the mode of the action jointly appear. However, as highlighted here, there can be a non-zero gradient associated with temporal derivatives of the sensory data (i.e., higher orders of generalised coordinates of motion). This is essential for the reflexive formulation of action under active inference.

4. References, Flatness-Based Trajectory Tracking, and Perceptual and Active Inferences

4.1. Equivalence to Linearity

4.1.1. Differential Flatness Characterisation

The class of differentially flat systems is—despite the fact that it occurs quite frequently in practice—the simplest nonlinear class with respect to the feedback equivalence classes. Indeed, we have the following.

Proposition 2.

A system is flat if, and only if, it is linearisable by endogenous feedback and a change of coordinates.

A dynamic feedback is called endogenous if it does not include any external dynamics. More precisely, the following holds.

Definition 9.

Consider the dynamics $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})+\mathit{\zeta }$. The feedback

$$\begin{array}{cc}\hfill \mathit{u}& =\phi (\mathit{x},\mathit{\xi },\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_u\right)},\mathit{v})\hfill \end{array}$$

(48a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}& =\psi (\mathit{x},\mathit{\xi },\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_v\right)},\mathit{v})\hfill \end{array}$$

(48b)

(where $\mathit{v}$ is the new input) is called a dynamic endogenous feedback if the original dynamics $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})+\mathit{\zeta }$ is equivalent to the transformed one

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\phi (\mathit{x},\mathit{\xi },\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_u\right)},\mathit{v}))\hfill \end{array}$$

(49a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}& =\psi (\mathit{x},\mathit{\xi },\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_v\right)},\mathit{v})\hfill \end{array}$$

(49b)

Two systems are called equivalent if there exists an invertible transformation that exchanges their trajectories.

Crucially the preceding linearisation is not local but global, and the category of flat systems is not very far from the one of linear systems, since they are equivalent through endogenous feedback and coordinate change.

4.1.2. Dynamical Extension Algorithm

This procedure enables one to determine if an m-tuple $\mathit{\omega }=({\omega }_1,\dots ,{\omega }_m)$ is a flat output or not and to obtain a linearizing feedback.

Phase I—Weak Brunovský Index Gathering

(1)

Differentiate ${\omega }_1$ until a combination of control inputs appears; denote by ${\kappa }_1$ the number of successive differentiations:

$$\begin{array}{cc}\hfill {\omega }_1^{\left({\kappa }_1\right)}& ={\gamma }_1(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_1-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta 1}\right)})\hfill \end{array}$$

with ${\sigma }_{\mathit{u}1}⩽{\kappa }_1-1$ and ${\sigma }_{\zeta 1}⩽{\kappa }_1$.

(2)

Differentiate ${\omega }_2$ until a combination of control inputs (independent of the previous ones) appears; denote by ${\kappa }_2$ the number of successive differentiations:

$$\begin{array}{cc}\hfill {\mathit{\omega }}_2^{\left({\kappa }_2\right)}& ={\gamma }_2(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_2-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}2}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta 2}\right)})\hfill \end{array}$$

with ${\sigma }_{\mathit{u}2}⩽{\kappa }_2-1$ and ${\sigma }_{\zeta 2}⩽{\kappa }_2$.

⋮  

(m)

Differentiate ${\omega }_m$ until a combination of control inputs (independent of the previous ones) appears; denote by ${\kappa }_m$ the number of successive differentiations:

$$\begin{array}{cc}\hfill {\mathit{\omega }}_m^{\left({\kappa }_m\right)}& ={\gamma }_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_m-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\mathit{\zeta }m}\right)})\hfill \end{array}$$

with ${\sigma }_{\mathit{u}m}⩽{\kappa }_m-1$ and ${\sigma }_{\zeta m}⩽{\kappa }_m$.

Phase II—Flatness Character Determination

Then, if ${\kappa }_1+\cdots +{\kappa }_m=n$ (n being the state dimension), the system admits $\mathit{\omega }$ as a flat output. If not, $\mathit{\omega }$ is not a flat output.

Phase III—Linearizing Feedback

In case $\mathit{\omega }$ is a flat output, the linearizing feedback is given by

$$\begin{array}{cc}\hfill {\gamma }_1(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_1-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta 1}\right)})& =v_1\hfill \end{array}$$

(50a)

$$\begin{array}{cc}\hfill {\gamma }_2(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_2-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}2}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta 2}\right)})& =v_2\hfill \end{array}$$

(50b)

$$⋮$$

$$\begin{array}{cc}\hfill {\gamma }_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\kappa }_m-1\right)},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta m}\right)})& =v_m\hfill \end{array}$$

(50c)

where $v_1,\dots ,v_m$ are the new action (new control input) variables. Since in Phase I, the ${\gamma }_i$s ($i=1,\dots ,m$) are functionally independent, the latter Equations (50) is invertible in $u_1,$ $\dots ,u_m$:

$$\begin{array}{cc}\hfill u_1=& {\tilde{\gamma }}_1(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\sigma }_{{\tilde{\kappa }}_1}\right)},\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta 1}\right)})\hfill \end{array}$$

(51a)

$$\begin{array}{cc}\hfill u_2=& {\tilde{\gamma }}_2(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\sigma }_{{\tilde{\kappa }}_2}\right)},\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}2}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta 2}\right)})\hfill \end{array}$$

(51b)

$$⋮$$

$$\begin{array}{cc}\hfill u_m=& {\tilde{\gamma }}_m(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\sigma }_{{\tilde{\kappa }}_m}\right)},\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta m}\right)})\hfill \end{array}$$

(51c)

with $\mathit{v}=(v_1,\dots ,v_m)$.

The original dynamics $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})+\mathit{\zeta }$ is then transformed, via the linearising endogenous feedback (50), to a linear dynamics of the form

$$\begin{array}{cc}\hfill {\omega }_1^{\left({\kappa }_1\right)}& =v_1\hfill \end{array}$$

(52a)

$$⋮$$

$$\begin{array}{cc}\hfill {\omega }_m^{\left({\kappa }_m\right)}& =v_m\hfill \end{array}$$

(52b)

with the new input $\mathit{v}=(v_1,\dots ,v_m)$.

Thus, the agent’s model (36) is equivalent, through the feedback (50), to the so-called flat output dynamics:

$$\begin{array}{cc}\hfill {\omega }_1^{\left({\kappa }_1\right)}& ={\gamma }_1(\mathit{\omega },\dots ,{\mathit{\omega }}^{\left({\kappa }_1-1\right)},\mathit{u},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}1}\right)},\mathit{\zeta },\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta 1}\right)})\hfill \end{array}$$

(53a)

$$⋮$$

$$\begin{array}{cc}\hfill {\omega }_m^{\left({\kappa }_m\right)}& ={\gamma }_m(\mathit{\omega },\dots ,{\mathit{\omega }}^{\left({\kappa }_m-1\right)},\mathit{u},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}m}\right)},\mathit{\zeta },\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\zeta m}\right)})\hfill \end{array}$$

(53b)

Thus, the original model (36) with n equations has been reduced, exactly, i.e., without any approximation, to the flat output dynamics (53) with m equations, where m is, in most practical cases, significantly smaller than n.

4.2. Differential Flatness and Controllability

A natural question a reader may ask is as follows: when is the differential flatness property verifiable? In other words, what are the checkable conditions ensuring flatness for a given system? The answer, for general nonlinear systems, is still an open problem. There are some conditions for restricted system classes or feedback equivalence classes. For instance, conditions are known for single-input systems and for static state feedback equivalence (see the Supplementary Material, File AIandFP-FlatnessAndSRR-HMTPKF-2026-SupplMaterial-v1.pdf, Section C).

There are some simple classes that are trivially flat. In the case of single-input systems, the class of systems that are, up to a feedback equivalence, in a cascade form like the following,

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =f_1(x_1,x_2)\hfill \end{array}$$

(54a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f_2(x_1,x_2,x_3)\hfill \end{array}$$

(54b)

$$⋮$$

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_n& =f_n(x_1,\dots ,x_n,u)\hfill \end{array}$$

(54c)

with the functions $f_i$ being invertible in their last argument, are differentially flat, with $x_1$ as a flat output. The systems that are, up to a feedback equivalence, in a form with a finite number m of blocks like (54) are also differentially flat. We shall nevertheless see that differential flatness is a very strong property, as the following proposition suggests.

Proposition 3.

A differentially flat system is globally controllable.

Proof. 

The global controllability property (see Definition 7) amounts to the following: for any initial and final states ${\mathit{x}}_0$ and ${\mathit{x}}_1$ of ${\mathbb{R}}^n$, and any initial and final times $t_0$ and $t_1$, there exists a control law $u\in {\mathcal{F}}_u$, in  the so-called space of admissible controls ${\mathcal{F}}_u$, driving the model from the initial state $\mathit{x}\left(t_0\right)={\mathit{x}}_0$ to the final state $\mathit{x}\left(t_1\right)={\mathit{x}}_1$. Consider now a flat system with equations

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\mathit{u})\hfill \\ \hfill \mathit{y}& =h\left(\mathit{x}\right)\hfill \end{array}$$

with flat output $\mathit{\omega }$. We thus have, by virtue of the flatness property, the following functional parametrisation:

$$\begin{array}{cc}\hfill \mathit{x}& =A(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_x\right)})\hfill \\ \hfill \mathit{u}& =B(\mathit{\omega },\stackrel{.}{\mathit{\omega }},\dots ,{\mathit{\omega }}^{\left({\rho }_u\right)})\hfill \end{array}$$

Consider two arbitrary states ${\mathit{x}}_0$ and ${\mathit{x}}_1$ in ${\mathbb{R}}^n$, and $t_0,t_1>0$. Let ${\mathit{\omega }}_r\left(t\right)$ be a polynomial such that

$$\begin{array}{cc}\hfill \mathit{x}\left(t_0\right)& =A({\mathit{\omega }}_r\left(t_0\right),{\stackrel{.}{\mathit{\omega }}}_r\left(t_0\right),\dots ,{\mathit{\omega }}_r^{\left({\rho }_x\right)}\left(t_0\right))={\mathit{x}}_0\hfill \\ \hfill \mathit{x}\left(t_1\right)& =A({\mathit{\omega }}_r\left(t_1\right),{\stackrel{.}{\mathit{\omega }}}_r\left(t_1\right),\dots ,{\mathit{\omega }}_r^{\left({\rho }_x\right)}\left(t_1\right))={\mathit{x}}_1\hfill \end{array}$$

The control steering the system from ${\mathit{\omega }}_0$ to ${\mathit{\omega }}_1$ is

$$\begin{array}{cc}\hfill {\mathit{u}}_r& =B({\mathit{\omega }}_r,{\stackrel{.}{\mathit{\omega }}}_r,\dots ,{\mathit{\omega }}_r^{\left({\rho }_u\right)})\hfill \end{array}$$

Thus the system is globally controllable. □

Example 4 (Oculomotor dynamics and flatness).

As a simple example, consider a model of oculomotor control, a problem that has been addressed extensively through active inference: see, e.g., [53,54,55,56,57]. Here, we will consider a simplified kinematics and dynamics of eye movement. Consider a human agent whose eyes are located at a fixed distance d from a vertical visual scene. Let us concentrate of the movement of a single eye and denote the pupil’s center by E. The eye tracks a point P with coordinates $\mathrm{X},\mathrm{Y}$ in the visual scene’s fixed reference frame. The agent’s head is supposed to be fixed. Let $P_0$, with coordinates ${\mathrm{X}}_0,{\mathrm{Y}}_0$, be the orthogonal projection of E on the visual scene plane. We shall first describe the generative model and then demonstrate that this is differentially flat. We then consider how it might be linearised as outlined above.

According to Listing’s law [58], two angles characterise the movement of the pupil: ψ, the orientation, or yaw angle, and ϕ, the elevation, or pitch angle. Let us note that this model is extremely similar to the one of a two-degree-of-freedom gimbal system (see, e.g., [59]). We then have

$$\begin{array}{cc}\hfill \mathrm{X}& ={\mathrm{X}}_0+dtan\psi +{\zeta }_{\mathrm{X}}\hfill \end{array}$$

(55a)

$$\begin{array}{cc}\hfill \mathrm{Y}& ={\mathrm{Y}}_0+dtan\varphi +{\zeta }_{\mathrm{Y}}\hfill \end{array}$$

(55b)

Let us set

$$\begin{array}{cc}\hfill v_{\psi }& =\stackrel{.}{\psi },v_{\varphi }=\stackrel{.}{\varphi }\hfill \end{array}$$

The differentiation of the expressions ofXandXin (55) with respect to time yields (See Figure 2)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathrm{X}}& =d{v}_{\psi }(1+{tan}^2\psi )+{\stackrel{.}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(56a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathrm{Y}}& =d{v}_{\varphi }(1+{tan}^2\varphi )+{\stackrel{.}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(56b)

Then, using the expressions of X and Y in (55), we get expressions of $1+{tan}^2\psi$ and $1+{tan}^2\varphi$ as functions of X and Y:

$$\begin{array}{cc}\hfill d(1+{tan}^2\psi )& ={\displaystyle \frac{d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}{d}}\hspace{1em}\hspace{1em}d(1+{tan}^2\varphi )={\displaystyle \frac{d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}{d}}\hfill \end{array}$$

Thus, the relations expressing $\stackrel{.}{\mathrm{X}}$ and $\stackrel{.}{\mathrm{Y}}$ in (56) become

$$\begin{array}{cc}\hfill \stackrel{.}{\mathrm{X}}& =v_{\psi }{\displaystyle \frac{d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(57a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathrm{Y}}& =v_{\varphi }{\displaystyle \frac{d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(57b)

The relationship between $v_{\psi },v_{\varphi }$ and the torques exerted by the eye muscles can be written as follows:

$$\begin{array}{cc}\hfill 2I_e{\stackrel{.}{v}}_{\psi }& =u_{\psi }\hfill \end{array}$$

(58a)

$$\begin{array}{cc}\hfill I_e{\stackrel{.}{v}}_{\varphi }& =u_{\varphi }\hfill \end{array}$$

(58b)

with $I_e$ being the rotational inertia of the eye, supposed to be spherical (see the Supplementary Material, Section A, File AIandFP-FlatnessAndSRR-HMTPKF-2026-SupplMaterial-v1.pdf, for the reason why the inertia is doubled in the yaw equation above). Let us set

$$\begin{array}{cc}\hfill \mathit{x}& =(x_{\mathrm{X}},x_{\mathrm{Y}},x_{\psi },x_{\varphi }),x_{\mathrm{X}}=\mathrm{X},x_{\mathrm{Y}}=\mathrm{Y},x_{v_{\psi }}=\psi ,x_{v_{\varphi }}=\varphi \hfill \end{array}$$

The full generative model will then be written as follows:

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{X}}=x_{v_{\psi }}{\displaystyle \frac{d^2+{(x_{\mathrm{X}}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(59a)

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{Y}}=x_{v_{\varphi }}{\displaystyle \frac{d^2+{(x_{\mathrm{Y}}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(59b)

$$\begin{array}{cc}\hfill 2I_e& {\stackrel{.}{x}}_{v_{\psi }}=u_{\psi }+{\zeta }_{\psi }\hfill \end{array}$$

(59c)

$$\begin{array}{cc}\hfill I_e& {\stackrel{.}{x}}_{v_{\varphi }}=u_{\varphi }+{\zeta }_{\varphi }\hfill \end{array}$$

(59d)

where ${\zeta }_{*}\left(t\right)$, $\ast \in \{x,y,\psi ,\varphi \}$, are fluctuations, which may be due to various alterations in human vision and the eye inertia $I_e=2m_e{r}_e^2/5$, with $m_e$ and $r_e$ being the eye’s mass and radius.

This model is differentially flat, with $\omega =(x,y)$ as a flat output. Indeed, one has the following:

• From the first two lines of (59), we get the expressions of $x_{v_{\psi }}$ and $x_{v_{\varphi }}$:

  $$\begin{array}{cc}\hfill x_{v_{\psi }}& ={\displaystyle \frac{d({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d^2+{(x_{\mathrm{X}}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}}\hfill \end{array}$$

  (60a)

  $$\begin{array}{cc}\hfill x_{v_{\varphi }}& ={\displaystyle \frac{d({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d^2+{(x_{\mathrm{X}}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}}\hfill \end{array}$$

  (60b)
• From the final two lines of (59), we get the expressions of the action, the control inputs $u_{\psi }$ and $u_{\varphi }$:

  $$\begin{array}{cc}\hfill u_{\psi }& =2I_e{\displaystyle \frac{d({\stackrel{..}{x}}_{\mathrm{X}}-{\stackrel{..}{\zeta }}_{\mathrm{X}})\left(d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2\right)-2d(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}}){({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}^2}{{\left(d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2\right)}^2}}-{\zeta }_{\psi }\hfill \end{array}$$

  (61a)

  $$\begin{array}{cc}\hfill u_{\varphi }& =I_e{\displaystyle \frac{d({\stackrel{..}{x}}_{\mathrm{Y}}-{\stackrel{..}{\zeta }}_{\mathrm{Y}})\left(d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\right)-2d(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}}){({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}^2}{{\left(d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\right)}^2}}-{\zeta }_{\varphi }\hfill \end{array}$$

  (61b)

Note that the previous parametrisation is a pathwise one and that it carries over all of the fluctuations. Let us consider the reference trajectories $x_{\mathrm{X}r}\left(t\right)$ and $x_{\mathrm{Y}r}\left(t\right)$ for x and y, depicted in Figure 3. More precisely, the plotted trajectory has the following form:

$$\begin{array}{ccc}\hfill x_{\mathrm{X}r}\left(t\right)& ={\displaystyle \frac{x_h-x_l}{2}}\left(1+tanh\left(\gamma (t-t_{raise})\right)\right)+x_l,t\hfill & \hfill \in [0,T],\\ \hfill T& =80 \mathrm{s},x_l=1,x_h=4,\gamma =4,t_{raise}=20 \mathrm{s}\hfill \end{array}$$

(62)

where the simulation is performed on $[0,80 \mathrm{s}]$, $\gamma$ is the stiffness of the trajectory, and $t_{raise}$ is the time where $x_{\mathrm{X}r}$ has reached half of the maximum. The minimum is ${lim}_{t\to -\infty }{x}_{\mathrm{X}r}\left(t\right)=x_l=1$; the maximum is ${lim}_{t\to +\infty }{x}_{\mathrm{X}r}\left(t\right)=x_h=4$.

The open-loop components of the states $v_{\psi r}$ and $v_{\varphi r}$, corresponding to Equation (60), are shown in Figure 4, together with the influence of the fluctuations ${\zeta }_{\mathrm{X}}$, ${\zeta }_{\mathrm{Y}}$ and their first derivatives. These fluctuations were chosen to be

$$\begin{array}{cc}\hfill {\zeta }_{*}& =\alpha \zeta \ast \in \{x,y,\psi ,\varphi \}\hfill \end{array}$$

(63)

with ζ being of the form (16), $L=8$, $\lambda =2$, and $\alpha =2\times {10}^{-6}$. Recall the form (16):

$$\begin{array}{cc}\hfill \zeta \left(t\right)& =a_0+\sum _{j=1}^r\left[a_{j}cos\left({\displaystyle \frac{2\pi j}{L}}t\right)+b_{j}sin\left({\displaystyle \frac{2\pi j}{L}}t\right)\right],r=\lfloor L/\lambda \rfloor \hfill \end{array}$$

(64)

where $\lambda ,L>0$, each $a_j$ and $b_j$ is an independent sample from $\mathcal{N}(0,1/(2r+1\left)\right)$ (where $\mathcal{N}(\mu ,V)$ denotes the real normal distribution of mean $\mu$ and variance V), and $\lfloor ·\rfloor$ is the floor function. Since $L=8$ and $\lambda =2$, we have $r=4$.

The corresponding fluctuations ${\zeta }_{\mathrm{X}}$ and ${\zeta }_{\mathrm{Y}}$ are depicted in Figure 5.

The open-loop action components $v_{\psi r}$ and $v_{\varphi r}$, corresponding to Equation (61), are described in Figure 6. Although the fluctuations in Figure 5 are extremely small, their influence is already noticeable, although quite modest, in the $u_{\varphi r}$ plot.

If one considers $\alpha ={10}^{-4}$, which forces ${\zeta }_{\mathrm{X}}$ and ${\zeta }_{\mathrm{Y}}$ to be still very small, the corresponding plots of $v_{\psi r}$ and $v_{\varphi r}$ change radically, as depicted in Figure 7.

Consider again Example 4 and its generative model:

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{X}}=x_{v_{\psi }}{\displaystyle \frac{d^2+{(x_{\mathrm{X}}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(65a)

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{Y}}=x_{v_{\varphi }}{\displaystyle \frac{d^2+{(x_{\mathrm{Y}}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(65b)

$$\begin{array}{cc}\hfill 2I_e& {\stackrel{.}{x}}_{v_{\psi }}=u_{\psi }+{\zeta }_{\psi }\hfill \end{array}$$

(65c)

$$\begin{array}{cc}\hfill I_e& {\stackrel{.}{x}}_{v_{\varphi }}=u_{\varphi }+{\zeta }_{\varphi }\hfill \end{array}$$

(65d)

To exactly linearise the model (65), one first differentiates (65c) and (65d), so that the action (control input) $u_{\psi }$, $u_{\varphi }$ can appear:

$$\begin{array}{cc}\hfill {\stackrel{..}{x}}_{\mathrm{X}}& ={\displaystyle \frac{1}{2d{I}_e}}\left(d^2+{(x_{\mathrm{X}}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2\right)(u_{\psi }+{\zeta }_{\psi })+{\displaystyle \frac{2x_{v_{\psi }}}{d}}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})+{\stackrel{..}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(66a)

$$\begin{array}{cc}\hfill {\stackrel{..}{x}}_{\mathrm{Y}}& ={\displaystyle \frac{1}{d{I}_e}}\left(d^2+{(x_{\mathrm{Y}}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\right)(u_{\varphi }+{\zeta }_{\varphi })+{\displaystyle \frac{2v_{\varphi }}{d}}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(66b)

or, in matrix form,

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{\stackrel{..}{x}}_{\mathrm{X}}\\ {\stackrel{..}{x}}_{\mathrm{Y}}\end{array}\right)& ={\displaystyle \frac{1}{2d{I}_e}}\left(\begin{array}{cc}{\delta }_{\mathrm{X}}& 0\\ 0& 2{\delta }_{\mathrm{Y}}\end{array}\right)\left(\begin{array}{c}{u}_{\psi }+{\zeta }_{\psi }\\ u_{\varphi }+{\zeta }_{\varphi }\end{array}\right)+{\displaystyle \frac{2}{d}}\left(\begin{array}{c}{x}_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})\\ x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})\end{array}\right)+\left(\begin{array}{c}{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)\hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill {\delta }_{\mathrm{X}}& =d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2{\delta }_{\mathrm{Y}}=d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\hfill \end{array}$$

(68)

To exactly linearise the dynamics (65), we shall equate ${({\stackrel{..}{x}}_{\mathrm{X}},{\stackrel{..}{x}}_{\mathrm{Y}})}^T$ to ${(v_{\mathrm{X}},v_{\mathrm{Y}})}^T$, where $v_{\mathrm{X}}\left(t\right)$, $v_{\mathrm{Y}}\left(t\right)$ are new action functions. Hence, we obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{2d{I}_e}}\left(\begin{array}{cc}{\delta }_{\mathrm{X}}& 0\\ 0& 2{\delta }_{\mathrm{Y}}\end{array}\right)\left(\begin{array}{c}{u}_{\psi }+{\zeta }_{\psi }\\ u_{\varphi }+{\zeta }_{\varphi }\end{array}\right)+\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)& =\left(\begin{array}{c}{v}_{\mathrm{X}}\\ v_{\mathrm{Y}}\end{array}\right)\hfill \end{array}$$

(69)

Then, the nonlinear dynamics (65) is transformed into

$$\begin{array}{cc}\hfill {\stackrel{..}{x}}_{\mathrm{X}}& =v_{\mathrm{X}}\hfill \\ \hfill {\stackrel{..}{x}}_{\mathrm{Y}}& =v_{\mathrm{Y}}\hfill \end{array}$$

4.3. Trajectory Design and Planning

In order to perform tracking of a predefined trajectory, one has first to design this trajectory, the goal of the subsequent action. In some cases, this trajectory will be obvious to design, e.g., for an eye to move along a line, or in a circular motion, or for an arm to grab an object when no obstacle is present. In other circumstances, the task of planning a trajectory may be highly complex, especially in cases where both the agent and some obstacles are moving. The literature on planning is huge, and it has been studied extensively in fields like robotics, where it plays a crucial role (see, e.g., [60]).

4.4. Synthesis Law Computations: Tracking Controller

4.4.1. General Action Tracking Law

There are numerous ways to achieve trajectory tracking with stability, i.e., ensure that the discrepancy in action ${\epsilon }_{act}$ tends to zero asymptotically, and the corresponding literature is huge (see, e.g., [41] for a classic reference). For our purposes, i.e., the fulfilment of the inference guidelines, all of the laws described in [41] are inappropriate. This is because they do not rely on pathwise properties, and more importantly they do not rely on a basis-like property. In contrast, the flatness-based framework is inherently pathwise while embedding the physics of the agent’s model.

Considering a differentially flat model $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})$, we here wish to derive an action law (a controller) able to follow any reference trajectory $t\mapsto {\mathit{\omega }}_r\left(t\right)$. In order to compensate for model mismatch and poorly known initial conditions, one has to complement the open loop (obtained through flatness) with a closed-loop corrective term depending on the error $\mathit{\omega }\left(t\right)-{\mathit{\omega }}_r\left(t\right)$.

Knowing that the dynamics is flat, with flat output $\mathit{\omega }$, it can be transformed via the linearising endogenous feedback (50) to a linear dynamics of the form

$$\begin{array}{cc}\hfill {\omega }_1^{\left({\kappa }_1\right)}& =v_1\hfill \end{array}$$

(70a)

$$⋮$$

$$\begin{array}{cc}\hfill {\omega }_m^{\left({\kappa }_m\right)}& =v_m\hfill \end{array}$$

(70b)

with the new input $\mathit{v}=(v_1,\dots ,v_m)$. Then, the elementary tracking feedback law

$$\begin{array}{cccc}\hfill v_i& ={\omega }_{ir}^{\left({\kappa }_i\right)}-\sum _{j=0}^{{\kappa }_i-1}{\lambda }_{\omega ij}({\omega }_i^{\left(j\right)}-{\omega }_{ir}^{\left(j\right)}),\hfill & \hfill & i=1,\dots ,m\hfill \\ \hfill & ={\omega }_{ir}^{\left({\kappa }_i\right)}-\sum _{j=0}^{{\kappa }_i-1}{\lambda }_{\omega ij}{e}_{\omega i}^{\left(j\right)},\hfill & \hfill & \mathrm{with}{e}_{\omega i}={\omega }_i-{\omega }_{ir}\hfill \end{array}$$

(71)

with appropriately chosen ${\lambda }_{zij}$ gains renders the error dynamics

$$\begin{array}{cc}\hfill e_{\omega i}^{\left({\kappa }_i\right)}+\sum _{j=0}^{{\kappa }_i-1}{\lambda }_{\omega ij}{e}_{\omega i}^{\left(j\right)}& =0,i=1,\dots ,m\hfill \end{array}$$

(72)

asymptotically stable. In vector form, the preceding law (71) is expressed as follows:

$$\begin{array}{cc}\hfill v_i& ={\omega }_r^{\left({\kappa }_i\right)}-{\lambda }_{\omega i}{\left({\mathit{e}}_{\omega i}^{\langle {\kappa }_i-1\rangle }\right)}^T\hfill \end{array}$$

(73)

where ${\mathit{e}}_{\omega i}^{\langle {\kappa }_i-1\rangle }=({\mathit{e}}_{\omega i},{\stackrel{.}{\mathit{e}}}_{\omega i},\dots ,{\mathit{e}}_{\omega i}^{({\kappa }_i-1)})$ and ${\lambda }_{\omega i}=({\lambda }_{0i},{\lambda }_{1i},\dots ,{\lambda }_{({\kappa }_i-1)i})$. Finally, the tracking action (tracking feedback control) law is expressed as follows:

$$\begin{array}{cc}\hfill u_1& ={\tilde{\gamma }}_1(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta 1}\right)})\hfill \end{array}$$

(74a)

$$\begin{array}{cc}\hfill u_2& ={\tilde{\gamma }}_2(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}2}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta 2}\right)})\hfill \end{array}$$

(74b)

$$⋮$$

$$\begin{array}{cc}\hfill u_m& ={\tilde{\gamma }}_m(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta m}\right)})\hfill \end{array}$$

(74c)

$$\begin{array}{cc}\hfill \mathit{v}& =(v_1,\dots ,v_m)\hfill \end{array}$$

(74d)

$$\begin{array}{cc}\hfill v_i& ={\mathit{\omega }}_r^{\left({\kappa }_i\right)}-{\mathit{\lambda }}_{zi}{\left({\mathit{e}}_{zi}^{\langle {\kappa }_i-1\rangle }\right)}^T,i=1,\dots ,m\hfill \end{array}$$

(74e)

which ensures the tracking of $\mathit{\omega }$ to ${\mathit{\omega }}_r$, with stability through driving the discrepancy ${\epsilon }_{act}=\mathit{\omega }-{\mathit{\omega }}_r$ to zero.

Remark 7 (Open-loop and model-free).

The preceding action law is named a linearising feedback controller due to the fact that the flat output dynamics is exactly linearised in a first step. Another possibility—most probably more fruitful—is to use one of the following control laws. A first possible choice is an open-loop controller, i.e., an action law obtained through the use of (29b), p. 13, in which ω is substituted with ${\mathit{\omega }}_r$, a reference trajectory, with this open-loop law being supplemented by a model-free controller, in the spirit of [61]. Other possible choices include (74) supplemented by a model-free controller, an ADRC (active disturbance rejection control) one [62,63], or a sliding mode control one [64,65].

Example 5 (Simple generic example; tracking).

Consider again the following simple example with scalar action (control) and sensor output:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1& =x_2+{\zeta }_{x_1}\hfill \end{array}$$

(75a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2& =f(x_1,x_2)+u+{\zeta }_{x_2}\hfill \end{array}$$

(75b)

$$\begin{array}{cc}\hfill y& =x_1+{\zeta }_y\hfill \end{array}$$

(75c)

In order to elaborate on the tracking action laws, we have to obtain the sensor output y dynamics, which is obtained by differentiating (75a), thus obtaining ${\stackrel{..}{x}}_1$:

$$\begin{array}{cc}\hfill {\stackrel{..}{x}}_1& =f(x_1,{\stackrel{.}{x}}_1)+u+{\stackrel{.}{\zeta }}_{x_1}+{\zeta }_{x_2}\hfill \end{array}$$

(76)

and using (75c):

$$\begin{array}{cc}\hfill \stackrel{..}{y}& =f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y)+u+{\stackrel{..}{\zeta }}_y+{\stackrel{.}{\zeta }}_{x_1}+{\zeta }_{x_2}\hfill \end{array}$$

(77)

Setting ${ϵ}_y=y-y_r$, we have $y=y_r+{ϵ}_y$. Then, the y-dynamics (77) becomes

$$\begin{array}{cc}\hfill {\stackrel{..}{ϵ}}_y& =-{\stackrel{..}{y}}_r+f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y)+u+{\stackrel{..}{\zeta }}_y+{\stackrel{.}{\zeta }}_{x_1}+{\zeta }_{x_2}\hfill \end{array}$$

(78)

The trajectory tracking goal is

$$\begin{array}{cc}\hfill \underset{t\to \infty }{lim}{ϵ}_y& =\underset{t\to \infty }{lim}(y-y_r)=0\hfill \end{array}$$

(79)

When expressing this goal in a dynamical setting, this renders ${ϵ}_y$ the solution of a target differential equation:

$$\begin{array}{cc}\hfill {\stackrel{..}{ϵ}}_y+{\lambda }_{y,1}{\stackrel{.}{ϵ}}_y+{\lambda }_{y,2}{ϵ}_y& =0\hfill \end{array}$$

(80)

with ${\lambda }_{y,1}, {\lambda }_{y,2}\in \mathbb{R}$ such that all solutions of (80) are exponentially decreasing functions of time.

In order to transform (78) into (80), we set

$$\begin{array}{cc}\hfill -{\stackrel{..}{y}}_r+f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y)+u+{\stackrel{..}{\zeta }}_y+{\stackrel{.}{\zeta }}_{x_1}+{\zeta }_{x_2}& =-{\lambda }_{y,1}{\stackrel{.}{ϵ}}_y-{\lambda }_{y,2}{ϵ}_y\hfill \end{array}$$

(81)

Hence, the control law ensuring the tracking of $y_r\left(t\right)$ with stability is given by

$$\begin{array}{cc}\hfill u& ={\stackrel{..}{y}}_r-f(y-{\zeta }_y,\stackrel{.}{y}-{\stackrel{.}{\zeta }}_y)-{\stackrel{..}{\zeta }}_y-{\stackrel{.}{\zeta }}_{x_1}-{\zeta }_{x_2}-{\lambda }_{y,1}{\stackrel{.}{ϵ}}_y-{\lambda }_{y,2}{ϵ}_y\hfill \end{array}$$

(82)

This control law assumes the full knowledge of the fluctuations ${\zeta }_{x_1}$, ${\zeta }_{x_2}$, and ${\zeta }_y$. When this knowledge is not available, we have to derive a control law based on a deterministic generative model (which is all that the agent knows), estimate the fluctuations, and compensate for them (see, e.g., [66] for the so-called model-free control or [62,65] for other schemes).

4.4.2. Oculomotor Example

Example 6 (Oculomotor tracking).

Consider again Example 4 and its generative model:

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{X}}=x_{v_{\psi }}{\displaystyle \frac{d^2+{(x_{\mathrm{X}}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{X}}\hfill \end{array}$$

(83a)

$$\begin{array}{cc}\hfill & {\stackrel{.}{x}}_{\mathrm{Y}}=x_{v_{\varphi }}{\displaystyle \frac{d^2+{(x_{\mathrm{Y}}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2}{d}}+{\stackrel{.}{\zeta }}_{\mathrm{Y}}\hfill \end{array}$$

(83b)

$$\begin{array}{cc}\hfill 2I_e& {\stackrel{.}{x}}_{v_{\psi }}=u_{\psi }+{\zeta }_{\psi }\hfill \end{array}$$

(83c)

$$\begin{array}{cc}\hfill I_e& {\stackrel{.}{x}}_{v_{\varphi }}=u_{\varphi }+{\zeta }_{\varphi }\hfill \end{array}$$

(83d)

and suppose that one wishes to track a reference trajectory $(x_{\mathrm{X}r}\left(t\right),x_{\mathrm{Y}r}\left(t\right))$ for $(\mathrm{X},\mathrm{Y})$. We have seen in Example 4 that the following action feedback:

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{2d{I}_e}}& \left(\begin{array}{cc}{\delta }_{\mathrm{X}}& 0\\ 0& 2{\delta }_{\mathrm{Y}}\end{array}\right)\left(\begin{array}{c}{u}_{\psi }+{\zeta }_{\psi }\\ u_{\varphi }+{\zeta }_{\varphi }\end{array}\right)+\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)=\left(\begin{array}{c}{v}_{\mathrm{X}}\\ v_{\mathrm{Y}}\end{array}\right)\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill & {\delta }_{\mathrm{X}}=d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2{\delta }_{\mathrm{Y}}=d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\hfill \end{array}$$

(85)

transforms the nonlinear dynamics (83) into the following linear dynamics:

$$\begin{array}{cc}\hfill {\stackrel{..}{x}}_{\mathrm{X}}& =v_{\mathrm{X}}\hfill \\ \hfill {\stackrel{..}{x}}_{\mathrm{Y}}& =v_{\mathrm{Y}}\hfill \end{array}$$

The trajectory tracking can then be enforced through the following tracking feedback law in $v_{\mathrm{X}}$, $v_{\mathrm{Y}}$:

$$\begin{array}{cc}\hfill v_{\mathrm{X}}& ={\stackrel{..}{x}}_{\mathrm{X}r}-{\lambda }_{\mathrm{X}1}{\stackrel{.}{\epsilon }}_{\mathrm{X}}-{\lambda }_{\mathrm{X}0}{\epsilon }_{\mathrm{X}}\hfill \end{array}$$

(86a)

$$\begin{array}{cc}\hfill v_{\mathrm{Y}}& ={\stackrel{..}{x}}_{\mathrm{Y}r}-{\lambda }_{\mathrm{Y}1}{\stackrel{.}{\epsilon }}_y-{\lambda }_{\mathrm{Y}0}{\epsilon }_y\hfill \end{array}$$

(86b)

$$\begin{array}{cc}\hfill {\epsilon }_{\mathrm{X}}& =x_{\mathrm{X}}-x_{\mathrm{X}r}{\epsilon }_y=x_{\mathrm{Y}}-x_{\mathrm{Y}r}\hfill \end{array}$$

(86c)

where ${\lambda }_{\mathrm{X}0}$, ${\lambda }_{\mathrm{X}1}$, ${\lambda }_{\mathrm{Y}0}$, and ${\lambda }_{\mathrm{Y}1}$ are suitable constants (the action or feedback control gains) ensuring the stability of the closed-loop error dynamics:

$$\begin{array}{cc}\hfill & {\stackrel{..}{\epsilon }}_{act}+{\lambda }_1{\stackrel{.}{\epsilon }}_{act}+{\lambda }_0{\epsilon }_{act}=0\hfill \\ \hfill & {\epsilon }_{act}=\left(\begin{array}{c}{\epsilon }_{\mathrm{X}}\\ {\epsilon }_y\end{array}\right){\lambda }_0=\left(\begin{array}{c}{\lambda }_{\mathrm{X}0}\\ {\lambda }_{\mathrm{Y}0}\end{array}\right){\lambda }_1=\left(\begin{array}{c}{\lambda }_{\mathrm{X}1}\\ {\lambda }_{\mathrm{Y}1}\end{array}\right)\hfill \end{array}$$

Here, it is sufficient to take all ${\lambda }_{*}>0$, $\ast \in \{{\mathrm{X}}_0,{\mathrm{Y}}_0,{\mathrm{X}}_1,{\mathrm{Y}}_1\}$. Note that the preceding laws $v_{\mathrm{X}}$ and $v_{\mathrm{Y}}$ in (86) can be replaced by other ones, such as, for instance, model-free control ones (see, e.g., [66]). Finally, to express the original control law $\mathit{u}=(u_{\psi },u_{\varphi })$, we must invert (84), yielding $u_{\psi }$ and $u_{\varphi }$ as functions of $v_{\mathrm{X}}$ and $v_{\mathrm{Y}}$:

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{u}_{\psi }\\ u_{\varphi }\end{array}\right)& ={\displaystyle \frac{d{I}_e}{{\delta }_{\mathrm{X}}{\delta }_{\mathrm{Y}}}}\left(\begin{array}{cc}2{\delta }_{\mathrm{Y}}& 0\\ 0& {\delta }_{\mathrm{X}}\end{array}\right)\left[-\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)+\left(\begin{array}{c}{v}_{\mathrm{X}}\\ v_{\mathrm{Y}}\end{array}\right)\right]-\left(\begin{array}{c}{\zeta }_{\psi }\\ {\zeta }_{\varphi }\end{array}\right)\hfill \end{array}$$

(87)

Then substituting (86a) into (87) yields the final action tracking controller:

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{u}_{\psi }\\ u_{\varphi }\end{array}\right)& ={\displaystyle \frac{d{I}_e}{{\delta }_{\mathrm{X}}{\delta }_{\mathrm{Y}}}}\left(\begin{array}{cc}2{\delta }_{\mathrm{Y}}& 0\\ 0& {\delta }_{\mathrm{X}}\end{array}\right)\left[-\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)+\left(\begin{array}{c}{\stackrel{..}{x}}_{\mathrm{X}r}-{\lambda }_{\mathrm{X}1}{\stackrel{.}{\epsilon }}_{\mathrm{X}}-{\lambda }_{\mathrm{X}0}{\epsilon }_{\mathrm{X}}\\ {\stackrel{..}{x}}_{\mathrm{Y}r}-{\lambda }_{\mathrm{Y}1}{\stackrel{.}{\epsilon }}_y-{\lambda }_{\mathrm{Y}0}{\epsilon }_y\end{array}\right)\right]-\left(\begin{array}{c}{\zeta }_{\psi }\\ {\zeta }_{\varphi }\end{array}\right)\hfill \end{array}$$

(88)

which ensures the tracking of $(x,y)$ to $(x_{\mathrm{X}r},x_{\mathrm{Y}r})$ with stability, i.e., the fulfilment of (IG-Perfo) through driving the discrepancy ${\mathit{\epsilon }}_{act}={({\epsilon }_{\mathrm{X}},{\epsilon }_y)}^T$ to zero. The quantity sensed by the agent is the position $\mathrm{X},\mathrm{Y}$ of the point P in the visual scene.

Remark 8 (A more realistic example).

Note that we could have considered a more realistic example than (83) by including, as in [53], a delay ${\tau }_s$ in sensing and an advance ${\tau }_a$ in acting. In order to achieve predictions, the agent uses a form of open-loop control. A possible choice could be the use of reference signals, using the functional parametrisation offered by the flatness property. In other words, using (61) after substitution of $(x,y)$ with $(x_{\mathrm{X}r},x_{\mathrm{Y}r})$, we get the open-loop action law:

$$\begin{array}{cc}\hfill u_{\psi r}& ={\displaystyle \frac{1}{I_e}}{\displaystyle \frac{d{\stackrel{..}{x}}_{\mathrm{X}r}\left(d^2+{(x_{\mathrm{X}r}-x_0)}^2\right)-2d(x_{\mathrm{X}r}-x_0){\stackrel{.}{x}}_{\mathrm{X}r}^2}{{\left(d^2+{(x_{\mathrm{X}r}-x_0)}^2\right)}^2}}\hfill \end{array}$$

(89a)

$$\begin{array}{cc}\hfill u_{\varphi r}& ={\displaystyle \frac{1}{I_e}}{\displaystyle \frac{d{\stackrel{..}{x}}_{\mathrm{Y}r}\left(d^2+{(x_{\mathrm{Y}r}-y_0)}^2\right)-2d(x_{\mathrm{Y}r}-y_0){\stackrel{.}{x}}_{\mathrm{Y}r}^2}{{\left(d^2+{(x_{\mathrm{Y}r}-y_0)}^2\right)}^2}}\hfill \end{array}$$

(89b)

Another (heavier) choice would be to simulate the generative model to get a prediction $(\mathrm{X}(t+{\tau }_s+{\tau }_a),\mathrm{Y}(t+{\tau }_s+{\tau }_a))$ for $\left(\mathrm{X}\right(t),\mathrm{Y}(t\left)\right)$.

4.4.3. Simulations

The following plots refer to the simulation of the occulomotor model in (83), with the following parameters:

$$I_e=4.032\times {10}^{-7} \mathrm{kg}.{\mathrm{m}}^2,d=10 \mathrm{m},{\mathrm{X}}_0=0 \mathrm{m},{\mathrm{Y}}_0=0 \mathrm{m}$$

(90)

The fluctuations ${\zeta }_{\mathrm{X}},{\zeta }_{\mathrm{Y}},{\zeta }_{\psi },{\zeta }_{\varphi }$ are all smooth random functions (i.e., sums of cosine and sine with coefficients that are random variables with a normal distribution). More precisely, they are of the form depicted in Equation (16):

$$\begin{array}{cc}\hfill {\zeta }_{*}\left(t\right)& =\alpha \left(a_0+\sum _{j=1}^r\left[a_{*,j}cos\left({\displaystyle \frac{2\pi j}{L}}t\right)+b_{*,j}sin\left({\displaystyle \frac{2\pi j}{L}}t\right)\right]\right),r=\lfloor L/\lambda \rfloor \hfill \end{array}$$

(91)

$$\begin{array}{c}\hfill \ast \in \left\{\mathrm{X},\mathrm{Y},\psi ,\varphi \right\}\end{array}$$

(92)

where $\lambda ,L>0$, each $a_{*,j}$ and $b_{*,j}$ is an independent sample from $\mathcal{N}(0,1/(2r+1\left)\right)$ (where $\mathcal{N}(\mu ,V)$ denotes the real normal distribution of mean $\mu$ and variance V), and $\lfloor ·\rfloor$ is the floor function. Here, $L=8,\lambda =2,\alpha ={10}^{-5}$; hence $r=4$. Let us consider a smooth eye tracking of a point along a regular curve, here a quatrefoil, shown in Figure 8 with the reference curve in red and the actual tracking (i.e., the eye movement) in blue. The trajectory type has been chosen because of its smoothness. Its precise time parametrisation is

$$\begin{array}{cc}\hfill \mathrm{X}\left(t\right)& =2a{sin}^2\left(t\right)cos\left(t\right)\hfill \end{array}$$

(93a)

$$\begin{array}{cc}\hfill \mathrm{Y}\left(t\right)& =2a{cos}^2\left(t\right)sin\left(t\right)\hfill \end{array}$$

(93b)

$$\begin{array}{cc}\hfill a& =2\hfill \end{array}$$

(93c)

The corresponding references $x_{\mathrm{X}r}\left(t\right)$, $x_{\mathrm{Y}r}\left(t\right)$ and the actual values $x_{\mathrm{X}}\left(t\right)$, $x_{\mathrm{Y}}\left(t\right)$ are plotted in Figure 9.

The corresponding action tracking laws $u_{\psi }$ and $u_{\varphi }$ are shown in Figure 10. As a last example, let us consider the hypocycloid in Figure 11. We chose this type of trajectory because it may be seen as typical of saccadic eye movements. Its precise time parametrisation is

$$\begin{array}{cc}\hfill \mathrm{X}\left(t\right)& =(a-b)cos\left(t\right)+bcos\left({\displaystyle \frac{(a-b)t}{b}}\right)\hfill \end{array}$$

(94a)

$$\begin{array}{cc}\hfill \mathrm{Y}\left(t\right)& =(a-b)sin\left(t\right)-bsin\left({\displaystyle \frac{(a-b)t}{b}}\right)\hfill \end{array}$$

(94b)

$$\begin{array}{cc}\hfill a& =2,b={\displaystyle \frac{6}{5}}\hfill \end{array}$$

(94c)

4.5. Active Inference

The principle of active inference can then be stated as extremisation of free energy via both action and perception, under a prior belief that action will extremise expected free energy (see, e.g., Figure 7 of [2]):

$$\begin{array}{cc}\hfill {\mathit{\mu }}_x^{*}& ={\mathrm{argmin}}_{{\mathit{\mu }}_x}\mathcal{F}({\mathit{\mu }}_x,{\mathit{\mu }}_u,\mathit{y}\left(\mathit{u}\right))\hfill \end{array}$$

(95a)

$$\begin{array}{cc}\hfill {\mathit{u}}^{*}& ={\mathrm{argmin}}_{\mathit{u}}\mathcal{F}({\mathit{\mu }}_x^{*},{\mathit{\mu }}_u,\mathit{y}\left(\mathit{u}\right))\hfill \end{array}$$

(95b)

$$\begin{array}{cc}\hfill \mathcal{F}(\mathit{x},\mathit{y})& ={\mathbb{E}}_{q,x}\left(\ln {\displaystyle \frac{q(\mathit{x}\mid \mathit{u})}{p(\mathit{x}\mid \mathit{u})}}\right)-{\mathbb{E}}_{q,x}\left(\ln p(\mathit{y}\mid \mathit{x},\mathit{u})\right)\hfill \end{array}$$

(95c)

$$\begin{array}{cc}\hfill {\mathit{\mu }}_u& ={\mathrm{argmin}}_{\mathit{u}}\mathcal{G}\left(\mathit{u}\right)\hfill \end{array}$$

(95d)

$$\begin{array}{cc}\hfill \mathcal{G}\left(\mathit{u}\right)& ={\mathbb{E}}_{q,y}\left(\ln {\displaystyle \frac{q(\mathit{y}\mid \mathit{u})}{p(\mathit{y}\mid {\mathit{y}}_r)}}\right)-{\mathbb{E}}_{q,x,y}\left(\ln p(\mathit{y}\mid \mathit{x},\mathit{u})\right)\hfill \end{array}$$

(95e)

with ${\mu }_{\mathit{x}}$ being the agent’s estimate of the hidden state $\mathit{x}$, and $\mathcal{F},\mathcal{G}$ as in (21c) (complexity/accuracy) and (22b) (risk/ambiguity), written in a compact form.

$$\begin{array}{cc}\hfill {\mathbb{E}}_{q,\xi }\left(\varphi \right)& =\int q\left(\xi \right(t)\mid \mathit{u}(t\left)\right)\varphi \left(t\right)d\xi \left(t\right)\hfill \\ \hfill {\mathbb{E}}_{q,\xi ,\chi }\left(\varphi \right)& =\int \int q\left(\xi \right(t),\chi (t)\mid \mathit{u}(t\left)\right)\varphi \left(t\right)d\xi \left(t\right)d\chi \left(t\right)\hfill \end{array}$$

The above extremisations can be formulated as the solutions to a gradient descent.

4.6. Link with Flatness-Based Tracking

The imperative for active inference is the minimisation of surprise, namely, the discrepancy between expectations, or beliefs, and actual values. In the light of the previous decomposition of variational free energy in Section 3.1, we conclude by foregrounding the links between active inference and nonlinear control. Consider the agent’s dynamics $\stackrel{.}{\mathit{x}}=f(\mathit{x},\mathit{u})+{\mathit{\zeta }}_{\mathit{x}}$. The dynamic feedback (see (48))

$$\begin{array}{cc}\hfill \mathit{u}& =\phi (\mathit{x},\mathit{\xi },{\mathit{\zeta }}_{\mathit{x}},{\stackrel{.}{\mathit{\zeta }}}_{\mathit{x}},\dots ,{\mathit{\zeta }}_{\mathit{x}}^{\left({\sigma }_u\right)},\mathit{v})\hfill \end{array}$$

(96a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}& =\psi (\mathit{x},\mathit{\xi },{\mathit{\zeta }}_{\mathit{x}},{\stackrel{.}{\mathit{\zeta }}}_{\mathit{x}},\dots ,{\mathit{\zeta }}_{\mathit{x}}^{\left({\sigma }_{\xi }\right)},\mathit{v})\hfill \end{array}$$

(96b)

$$\begin{array}{cc}\hfill \mathit{v}& =\mathsf{\Gamma }({\mathit{\epsilon }}_{act},{\stackrel{.}{\mathit{\epsilon }}}_{act},\dots ,{\mathit{\epsilon }}_{act}^{\left({\sigma }_v\right)})\hfill \end{array}$$

(96c)

(where $\mathit{v}$ is the new input) transforms the agent’s dynamics to

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}& =f(\mathit{x},\phi (\mathit{x},\mathit{\xi },{\mathit{\zeta }}_{\mathit{x}},{\stackrel{.}{\mathit{\zeta }}}_{\mathit{x}},\dots ,{\mathit{\zeta }}^{\left({\sigma }_u\right)},\mathit{v}))\hfill \end{array}$$

(97a)

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}& =\psi (\mathit{x},\mathit{\xi },{\mathit{\zeta }}_{\mathit{x}},{\stackrel{.}{\mathit{\zeta }}}_{\mathit{x}},\dots ,{\mathit{\zeta }}_{\mathit{x}}^{\left({\sigma }_{\xi }\right)},\mathit{v})\hfill \end{array}$$

(97b)

$$\begin{array}{cc}\hfill \mathit{v}& =\mathsf{\Gamma }({\mathit{\epsilon }}_{act},{\stackrel{.}{\mathit{\epsilon }}}_{act},\dots ,{\mathit{\epsilon }}_{act}^{\left({\sigma }_v\right)})\hfill \end{array}$$

(97c)

The above Equation (97) then yield the following action surprise dynamics, or action discrepancy dynamics:

$$\begin{array}{cc}\hfill \mathsf{\Xi }({\mathit{\epsilon }}_{act},{\stackrel{.}{\mathit{\epsilon }}}_{act},\dots ,{\mathit{\epsilon }}_{act}^{\left({\sigma }_{{\epsilon }_{act}}\right)})& =0\hfill \end{array}$$

(98)

This surprise dynamics will not, in general (i.e., for any feedback law (96)), imply the action discrepancy’s limit to be zero, i.e., a minimisation of the free energy. Let us now consider the tracking action (tracking feedback control) law stemming from (74):

$$\begin{array}{cc}\hfill u_1& ={\tilde{\gamma }}_1(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\mathit{\zeta }1}\right)})\hfill \end{array}$$

(99a)

$$\begin{array}{cc}\hfill u_2& ={\tilde{\gamma }}_2(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}2}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\mathit{\zeta }2}\right)})\hfill \\ \hfill & ⋮\hfill \end{array}$$

(99b)

$$\begin{array}{cc}\hfill u_m& ={\tilde{\gamma }}_m(\mathit{\omega },\mathit{v},\stackrel{.}{\mathit{v}},\dots ,{\mathit{v}}^{\left({\sigma }_{\mathit{v}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\tilde{\sigma }}_{\zeta m}\right)})\hfill \end{array}$$

(99c)

$$\begin{array}{cc}\hfill \mathit{v}& =(v_1,\dots ,v_m)\hfill \end{array}$$

(99d)

$$\begin{array}{cc}\hfill v_i& ={\mathit{\omega }}_r^{\left({\kappa }_i\right)}-{\mathit{\lambda }}_{zi}{\left({\mathit{e}}_{zi}^{\langle {\kappa }_i-1\rangle }\right)}^T,i=1,\dots ,m\hfill \end{array}$$

(99e)

which yields a special form of (98):

$$\begin{array}{cc}\hfill {\mathit{\epsilon }}_{act}^{\left(n\right)}-\mathsf{\Lambda }(0,{\mathit{\epsilon }}_{act},{\stackrel{.}{\mathit{\epsilon }}}_{act},\dots {\mathit{\epsilon }}_{act}^{(n-1)})& =0\hfill \end{array}$$

(100)

where $\mathsf{\Lambda }$ is a linear function of its arguments such that (100) admits solely exponentially stable solutions. The tracking action (tracking feedback control) law (99) thus ensures the tracking of $\mathit{\omega }$ to ${\mathit{\omega }}_r$, with stability through driving the discrepancy ${\mathit{\epsilon }}_{act}=\mathit{\omega }-{\mathit{\omega }}_r$ to zero, minimising the risk in the expected free energy $\mathcal{G}$.

Example 7 (Oculomotor tracking control law).

Consider again the oculomotor Example 4. The tracking action (tracking feedback control) law (99) takes the form of (88):

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{u}_{\psi }\\ u_{\varphi }\end{array}\right)& ={\displaystyle \frac{d{I}_e}{{\delta }_{\mathrm{X}}{\delta }_{\mathrm{Y}}}}\left(\begin{array}{cc}2{\delta }_{\mathrm{Y}}& 0\\ 0& {\delta }_{\mathrm{X}}\end{array}\right)\left[-\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)+\left(\begin{array}{c}{\stackrel{..}{x}}_{\mathrm{X}r}-{\lambda }_{\mathrm{X}1}{\stackrel{.}{\epsilon }}_{\mathrm{X}}-{\lambda }_{\mathrm{X}0}{\epsilon }_{\mathrm{X}}\\ {\stackrel{..}{x}}_{\mathrm{Y}r}-{\lambda }_{\mathrm{Y}1}{\stackrel{.}{\epsilon }}_y-{\lambda }_{\mathrm{Y}0}{\epsilon }_y\end{array}\right)\right]-\left(\begin{array}{c}{\zeta }_{\psi }\\ {\zeta }_{\varphi }\end{array}\right)\hfill \end{array}$$

(101)

5. Prediction as a Link Between Active Inference and Differential Flatness

5.1. Delays and $\delta$-Flatness

In real systems stemming from neuroscience or physiology, delays are present, both in sensing and acting. This has not yet been taken into account in the previous sections, although it remains of utmost importance. Indeed, there are fundamental differences between a delay-free model and one including delays; the most striking difference is the infinite-dimensional character of delay systems. To be more precise, consider a delay-differential equation of the following form:

$$\begin{array}{cc}\hfill x\left(t\right)& =f\left(x\right(t-\tau \left)\right)\hfill \end{array}$$

(102)

If we want to integrate such an equation, we do not need to know a pointwise initial condition such as $x\left(0\right)$ but have to know a whole function on a time interval $\mathsf{x}:[-\tau ,0)\to \mathbb{R}$. Another view on this is that there is not just one operator acting on the variables—time differentiation—but two: time differentiation and the time delay. Therefore, the structural properties (controllability and observability) of such systems become more complex, even for linear delay systems (see, e.g., [67]). The notion of flatness has been extended to the case of delay systems (see [68,69]). We shall informally recall the definition of the so-called $\delta$-flatness (a special case of the $\pi$-flatness where the delays do not only occur on the sensor outputs and the action variables but also on the hidden state) and then expose two different kinds of predictors, which are needed for tracking. We denote a delay operator of amplitude $\tau$ by ${\delta }_{\tau }$:

$$\begin{array}{cc}\hfill \left({\delta }_{\tau }x\right)\left(t\right)& =x(t-\tau )\hfill \end{array}$$

(103)

In informal terms, a $\delta$-flat system is a system of delay differential equations that is differentially flat when one allows for delays. Let us see how this notion unfolds in a concrete example.

Example 8.

Consider Example 2 with a delay in the action:

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_1\left(t\right)& =x_2\left(t\right)+{\zeta }_{x_1}\left(t\right)\hfill \end{array}$$

(104a)

$$\begin{array}{cc}\hfill {\stackrel{.}{x}}_2\left(t\right)& =f(x_1\left(t\right),x_2\left(t\right))+u(t-\tau )+{\zeta }_{x_2}\left(t\right)\hfill \end{array}$$

(104b)

$$\begin{array}{cc}\hfill y& =x_1\left(t\right)+{\zeta }_y\left(t\right)\hfill \end{array}$$

(104c)

with $\tau >0,\tau \in \mathbb{R}$. This model is easily seen to be δ-flat, with y as a δ-flat output. Indeed, we have

$$\begin{array}{cc}{x}_1\left(t\right)\hfill & =y\left(t\right)-{\zeta }_y\left(t\right)\hfill \end{array}$$

(105a)

$$\begin{array}{cc}\hfill x_2\left(t\right)& =\stackrel{.}{y}\left(t\right)-{\stackrel{.}{\zeta }}_y\left(t\right)-{\zeta }_{x_1}\left(t\right)\hfill \\ \hfill u\left(t\right)& =\stackrel{..}{y}(t+\tau )-f(y(t+\tau )-{\zeta }_y(t+\tau ),\stackrel{.}{y}(t+\tau )-{\stackrel{.}{\zeta }}_y(t+\tau )-{\zeta }_{x_1}(t+\tau ))-\hfill \end{array}$$

(105b)

$$\begin{array}{cc}\hfill & {\stackrel{..}{\zeta }}_y(t+\tau )-{\stackrel{.}{\zeta }}_{x_1}(t+\tau )-{\zeta }_{x_2}(t+\tau )\hfill \end{array}$$

(105c)

5.2. Trajectory Tracking and Predictors

When one wishes to perform trajectory tracking, as in (99), the presence of delays in the sensor output and/or in the action control will induce the necessity to predict some or all of the hidden states or sensor output. Consider this under the previous example.

Example 9.

The tracking control scheme (82) is now transformed into

$$\begin{array}{cc}\hfill u\left(t\right)& ={\stackrel{..}{y}}_r(t+\tau )-f\left(\mathsf{y}\left(t\right)-{\mathsf{z}}_{\mathsf{y}}\left(t\right),\stackrel{.}{\mathsf{y}}\left(t\right)-{\stackrel{.}{\mathsf{z}}}_{\mathsf{y}}\left(t\right)\right)-\hfill \end{array}$$

$$\begin{array}{cc}\hfill & {\stackrel{..}{\mathsf{z}}}_{\mathsf{y}}\left(t\right)-{\stackrel{.}{\mathsf{z}}}_{{\mathsf{x}}_1}\left(t\right)-{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{1}}}\left(t\right)-{\lambda }_{y,1}{\stackrel{.}{\mathsf{e}}}_{\mathsf{y}}\left(t\right)-{\lambda }_{y,2}{\mathsf{e}}_{\mathsf{y}}\left(t\right)\hfill \end{array}$$

(106a)

$$\begin{array}{cc}\hfill \mathsf{y}\left(t\right)=y(t+\tau ),& {\mathsf{e}}_y\left(t\right)={ϵ}_y(t+\tau )\hfill \end{array}$$

(106b)

$$\begin{array}{cc}\hfill {\mathsf{z}}_{\mathsf{y}}\left(t\right)& ={\zeta }_y(t+\tau ),{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{1}}}\left(t\right)={\zeta }_{x_1}(t+\tau ),{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{2}}}\left(t\right)={\zeta }_{x_2}(t+\tau )\hfill \end{array}$$

(106c)

We now review two kinds of predictors, which may be used in tracking control schemes. Consider the following type of mean generative model with delayed action and a linear relation between the sensory output and hidden state:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{x}}\left(t\right)& =f\left(\mathit{x}\right(t),\mathit{u}(t-\tau \left)\right)\hfill \end{array}$$

(107a)

$$\begin{array}{cc}\hfill \mathit{y}\left(t\right)& =C\mathit{x}\left(t\right)\hfill \end{array}$$

(107b)

with $C\in {\mathbb{R}}^{n\times p}$. This model can be transformed into the advanced generative model

$$\begin{array}{cc}\hfill \stackrel{.}{\mathsf{x}}\left(t\right)& =f\left(\mathsf{x}\right(t),\mathit{u}(t\left)\right)\hfill \end{array}$$

(108a)

$$\begin{array}{cc}\hfill \mathsf{y}\left(t\right)& =C\mathsf{x}\left(t\right)\hfill \end{array}$$

(108b)

$$\begin{array}{cc}\hfill \mathsf{x}\left(t\right)& =\mathit{x}(t+\tau ),\mathsf{y}\left(t\right)=\mathit{y}(t+\tau )\hfill \end{array}$$

(108c)

The first scheme is a so-called delayed observer (see [70]):

$$\begin{array}{cc}\hfill \stackrel{.}{\widehat{\mathsf{x}}}\left(t\right)& =f(\widehat{\mathsf{x}}\left(t\right),\mathit{u}\left(t\right))+L(\mathit{x}\left(t\right)-\widehat{\mathsf{x}}(t-\tau ))\hfill \end{array}$$

(109)

with $L\in {\mathbb{R}}^{n\times n}$ being the predictor gain matrix. The simulation of Equation (109) yields an estimate $\widehat{\mathsf{x}}\left(t\right)$ of $\mathit{x}(t+\tau )$. It is proven [70] that the so-called prediction observer error $\mathit{x}\left(t\right)-\widehat{\mathsf{x}}(t-\tau )$ tends to zero as t tends to infinity.

The second scheme is an integral form one (see [71,72,73]):

$$\begin{array}{cc}\hfill \mathit{u}\left(t\right)& =\kappa \left(\mathsf{x}\left(t\right)\right)\hfill \end{array}$$

(110a)

$$\begin{array}{cc}\hfill \mathsf{x}\left(t\right)& ={\int }_{t-\tau }^{t}f(\mathsf{x}\left(\xi \right),\mathit{u}\left(\xi \right))d\xi +\mathit{x}\left(t\right)\hfill \end{array}$$

(110b)

where $\kappa$ is such that $\stackrel{.}{\mathit{x}}=f(\mathit{x},\kappa \left(\mathit{x}\right))$ is globally asymptotically stable at $\mathit{x}=0$; i.e., for every trajectory $\mathit{x}\left(t\right)$, we have $\mathit{x}\left(t\right)\to 0$ as $t\to \infty$. Equation (110b) comes from the following simple observation:

$$\begin{array}{cc}\hfill {\int }_t^{t+\tau }\stackrel{.}{z}\left(\xi \right)d\xi & =z(t+\tau )-z\left(t\right)\hfill \end{array}$$

(111)

The initial condition for the integral Equation (110b) for $\mathsf{x}\left(t\right)$ is defined by

$$\begin{array}{cc}\hfill \mathsf{x}\left(\sigma \right)& ={\int }_{-\tau }^{\sigma }f(\mathsf{x}\left(\xi \right),\mathit{u}\left(\xi \right))d\xi +\mathit{x}\left(0\right),\sigma \in [-\tau ,0]\hfill \end{array}$$

(112)

The predictor state $\mathsf{x}\left(t\right)$ is given by the implicit relation (110b), which can be solved using various approximation strategies for the integral on the right-hand side. Then, the predictive control law of the simple example shall be implemented as follows.

Example 10.

The tracking control scheme (106a) is now transformed into

$$\begin{array}{cc}\hfill u\left(t\right)& ={\stackrel{..}{y}}_r(t+\tau )-f\left(\mathsf{y}\left(t\right)-{\mathsf{z}}_{\mathsf{y}}\left(t\right),\stackrel{.}{\mathsf{y}}\left(t\right)-{\stackrel{.}{\mathsf{z}}}_{\mathsf{y}}\left(t\right)\right)-\hfill \end{array}$$

$$\begin{array}{cc}\hfill & {\stackrel{..}{\mathsf{z}}}_{\mathsf{y}}\left(t\right)-{\stackrel{.}{\mathsf{z}}}_{{\mathsf{x}}_1}\left(t\right)-{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{1}}}\left(t\right)-{\lambda }_{y,1}{\stackrel{.}{\mathsf{e}}}_{\mathsf{y}}\left(t\right)-{\lambda }_{y,2}{\mathsf{e}}_{\mathsf{y}}\left(t\right)\hfill \end{array}$$

(113a)

$$\begin{array}{cc}\hfill \mathsf{y}\left(t\right)& ={\int }_{t-\tau }^{t}f(\mathsf{y}\left(\xi \right),u\left(\xi \right))d\xi +y\left(t\right),{\mathsf{e}}_y\left(t\right)={ϵ}_y(t+\tau )\hfill \end{array}$$

(113b)

$$\begin{array}{cc}\hfill {\mathsf{z}}_{\mathsf{y}}\left(t\right)& ={\zeta }_y(t+\tau ),{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{1}}}\left(t\right)={\zeta }_{x_1}(t+\tau ),{\mathsf{z}}_{{\mathsf{x}}_{\mathsf{2}}}\left(t\right)={\zeta }_{x_2}(t+\tau )\hfill \end{array}$$

(113c)

5.3. Generalised Coordinates

In the active inference framework, there is a crucial distinction between what is called motion of the mean compared to mean of the motion. To unpack this distinction we first need the notion of generalised coordinates, which is obtained through a linearised differentiation. Generally a differentiation operation ∂ in algebra is defined as one such that, for any variables (here time functions) ${\omega }_1$ and ${\omega }_2$, the chain rule is fulfilled:

$$\begin{array}{cc}\hfill \partial \left({\omega }_1{\omega }_2\right)& =\partial {\omega }_1{\omega }_2+{\omega }_1\partial {\omega }_2\hfill \end{array}$$

(114)

A linearised (or first-order) differentiation, $d_1$, here denoted as ′, is an operator following the above chain rule such that, for any time functions $z\left(t\right)$ and $\zeta \left(t\right)$,

$$\begin{array}{cc}\hfill z^i{\zeta }^{\left(j\right)}& =0\hfill \end{array}$$

(115)

whenever $j>0$, $i+j>1$ and denoting by ${\zeta }^{\left(j\right)}$ the jth iterated application of the ′ operator to $\zeta$. This type of differentiation is linked to first-order stochastic realisation problems, as briefly discussed below.

Higher-order differentiations can also be considered. A k-th-order differentiation $d_k$ is an operator following the above chain rule such that, for any time functions $z\left(t\right)$ and $\zeta \left(t\right)$,

$$\begin{array}{cc}\hfill z^i{d}_k^j{\zeta }^{\left(j\right)}& =0\hfill \end{array}$$

(116)

whenever $j>0$, $i+j>k$. This type of differentiation is linked to higher-order stochastic realisation problems, although these may induce quite involved computations.

In other words, suppose the solution $\mathit{\xi }\left(t\right)$ of

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}\left(t\right)& =F\left(\mathit{\xi }\right(t\left)\right)+\mathit{\eta }\left(t\right)\hfill \end{array}$$

(117)

can be expressed through a series expansion:

$$\begin{array}{cc}\hfill \mathit{\xi }\left(t\right)& =\sum _{i=0}^{\infty }{\displaystyle \frac{\mathit{\xi }\left(0\right)}{i!}}{t}^i\hfill \end{array}$$

(118)

e.g., it could be an analytic function of time or a Gevrey series (i.e., belonging to a class of functions in between the analytic and the full $C^{\infty }$ one—see, e.g., [74,75]; see also [76] for extensions). Then, the various derivatives at the time origin can be recovered as follows:

$$\begin{array}{cc}\hfill \stackrel{.}{\mathit{\xi }}\left(0\right)& =F\left(\mathit{\xi }\right(0\left)\right)+\eta \left(0\right)\hfill \end{array}$$

(119a)

$$\begin{array}{cc}\hfill \stackrel{..}{\mathit{\xi }}\left(0\right)& ={\displaystyle \frac{d}{dt}}F\left(\mathit{\xi }\left(0\right)\right)+\stackrel{.}{\eta }\left(0\right)\hfill \end{array}$$

(119b)

$$\begin{array}{cc}\hfill \stackrel{...}{\mathit{\xi }}\left(0\right)& ={\displaystyle \frac{d^2}{d{t}^2}}F\left(\mathit{\xi }\left(0\right)\right)+\stackrel{..}{\eta }\left(0\right)\hfill \\ \hfill ⋮& \hfill \end{array}$$

(119c)

assuming the fluctuations $\eta \left(t\right)$ are sufficiently smooth and

$$\begin{array}{cc}\hfill {\displaystyle \frac{d}{dt}}F\left(\mathit{\xi }\left(0\right)\right)& =\nabla F\left(\mathit{\xi }\left(0\right)\right)\stackrel{.}{\mathit{\xi }}\left(0\right)\hfill \end{array}$$

(120a)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d^2}{d{t}^2}}F\left(\mathit{\xi }\left(0\right)\right)& =\nabla F\left(\mathit{\xi }\left(0\right)\right)\stackrel{..}{\mathit{\xi }}\left(0\right)+{\stackrel{.}{\mathit{\xi }}}^T\left(0\right){\nabla }^{2}F\left(\mathit{\xi }\left(0\right)\right)\stackrel{.}{\mathit{\xi }}\left(0\right)\hfill \\ \hfill ⋮& \hfill \end{array}$$

(120b)

The so-called local linear approximation, in terms of [32], amounts to neglecting all higher-order terms in Equation (120), i.e., considering

$$\begin{array}{cc}\hfill {\displaystyle \frac{d^k}{d{t}^k}}F\left(\mathit{\xi }\left(0\right)\right)& =\nabla F\left(\mathit{\xi }\left(0\right)\right){\mathit{\xi }}^{\left(\mathit{k}\right)}\left(0\right)\hfill \end{array}$$

(121)

Such an approximation, when valid, enables one to solve the so-called stochastic realisation problem quite easily (recall that a realisation is a differential equation involving a hidden state obtained from an input/action–output/sensor differential equation—see, e.g., Definition 6; see, e.g., [77,78] on the stochastic realisation problem). This approximation is also justified when studying generalised Bayesian filtering under the Laplace approximation (see, e.g., [32], Subsections 3.3.4 and 3.3.5).

The generalised coordinates may be seen as a coordinate frame that moves with the current point. In this view, it is linked with one-form transformations one can apply to nonlinear systems (see, e.g., [79,80]); the latter are more general than the endogenous dynamical feedbacks used here. It is also linked with the Cartan moving frame method (see, e.g., [81]).

6. Conclusions, Limitations, and Future Directions

We have considered the utility of differential flatness through the lens of active inference (see, e.g., [1]). This utility has been detailed in terms of control as inference. Specifically, one might conclude that if the generative models that underwrite active inference or control as inference can be limited to the class of differentially flat models, we obtain an extremely efficient control-theoretic scheme. This work therefore enables a control-theoretic perspective on active inference as control as inference by focusing on action trajectories and the minimisation of various discrepancies (afforded by variational and expected free energy). From the perspective of active inference, this work is a primer on differential flatness and its particular relevance to the kinds of generative models one might consider and be committed to. Crucially, this paper is the first (provisional) attempt to consider expected free energy in the setting of continuous state-space models.

In addition to their roles in developing systems for control, one could consider other applications of the frameworks outlined in this paper. One area is the field of computational psychiatry (see, e.g., [82,83,84]), where one can develop generative models of decision-making tasks—solved with active inference—and use these to understand the computational mechanisms of psychopathology. Such models are often formulated in terms of the selection among discrete alternatives. However, as noted by our reviewers, differential flatness pertains to continuous, differentiable state spaces. While it is true that many applications of computational psychiatry focus on discrete probabilities, there are important areas of psychiatry that depend upon continuous variables of the sort addressed in differential flatness accounts. These include the altered motor dynamics associated with catatonia in psychotic disorders (see, e.g., [85]) and altered smooth pursuit eye movements in schizophrenia (see, e.g., [86,87]).

There are several related notions that could not be addressed here due to the lack of space. These include Liouvillian aspects (see [88,89,90] related to automatic control and [91,92] for the examination of this property in the setting of hypothalamic–pituitary–adrenal axis models and Wilson–Cowan population networks), and robust tracking with model-free control (see [66]). Liouvillian aspects, in particular, offer an opportunity to extend the notion of flatness when a model is not differentially flat. Future work will consider these and other issues related to energy transmission and controllability (see, e.g., [93,94] for a formulation of the corresponding problem) and variational function transmission. These deal with characterising a model in terms of salient features and understanding how reparameterisations might transform such features.

We discuss the following points, emphasising limitations of the current treatment and sketching some future directions.

• Generalised coordinates, where some limitations—due to its approximate character—may be avoided through smooth random realisations.
• Extensions of flatness: Liouvillian characters to deal with cases where the model is not differentially flat.
• Observers and algebraic estimators to estimate the hidden state from sensor output.
• Robust control law synthesis to cope with uncertainty under the generative model, including fluctuations.
• Constraint fulfilment, where constraints are imposed on the hidden state, the action, and their time derivatives.
• Feature transmission: How a feature of interest; e.g., the energy ($L^2$ norm), the slope or curvature, etc., is transformed through functional parametrisation.

6.1. Generalised Coordinate Limitations

The generalised coordinates are not appropriate for deriving tracking feedback action in the sense being considered here. Indeed, the dynamical extension algorithm needs to make the action appear through time differentiation, which will not be the case unless the action was already present in the state-space dynamics. More precisely, for the components of the flat output whose Brunovksỳ index ${\kappa }_i$ is strictly superior to 2, the generalised coordinates are not appropriate, since successive application of $d_1$ will not make the action appear. For the components whose index ${\kappa }_i=1$ or 2, the generalised coordinates will be sufficient.

Let us illustrate this through three simple examples.

• First consider the longitudinal motion of a car, used, for instance, in ACCs (Automatic Cruise Controllers):

  $$\begin{array}{cc}\hfill M{\stackrel{.}{V}}_x& =-F\left(V_x\right)+{\displaystyle \frac{1}{r}}u\hfill \end{array}$$

  (122)

  with M being the car’s mass, $V_x\left(t\right)$ the longitudinal speed of the vehicle’s centre of gravity, $-F\left(V_x\right)$ the force due to the wind and to friction of the tyres on the ground, r the mean wheel radius, and u the traction engine propulsion torque, taken as the action. This system is trivially flat, with flat output $V_x$, and the preceding equation is rewritten as

  $$\begin{array}{cc}\hfill M{V}_x^{\prime }& =-F(V_x,{\stackrel{.}{V}}_x)+{\displaystyle \frac{1}{r}}u\hfill \end{array}$$
  The action is already present in the dynamics equation, $\kappa =1$, and there is no need to differentiate more. The generalised coordinate dynamics will lead to the same action tracking controller as the original one.
• Second, consider the oculomotor example again. Then one supplementary differentiation of $x_{\mathrm{X}}$ (resp. $x_{\mathrm{Y}}$) will be sufficient. Both of these will involve $f(\mathit{x},\mathit{u})$ and its partial derivatives with respect to $\mathit{x}$ and $\mathit{u}$ in the flat output dynamics (see (69)):

  $$\begin{array}{cc}\hfill \left(\begin{array}{c}{x}_{\mathrm{X}}\\ x_{\mathrm{Y}}\end{array}\right)& ={\displaystyle \frac{1}{2d{I}_e}}\left(\begin{array}{cc}{\delta }_{\mathrm{X}}& 0\\ 0& 2{\delta }_{\mathrm{Y}}\end{array}\right)\left(\begin{array}{c}{u}_{\psi }+{\zeta }_{\psi }\\ u_{\varphi }+{\zeta }_{\varphi }\end{array}\right)+\left(\begin{array}{c}{\displaystyle \frac{2x_{v_{\psi }}({\stackrel{.}{x}}_{\mathrm{X}}-{\stackrel{.}{\zeta }}_{\mathrm{X}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{X}}\\ {\displaystyle \frac{2x_{v_{\varphi }}({\stackrel{.}{x}}_{\mathrm{Y}}-{\stackrel{.}{\zeta }}_{\mathrm{Y}})}{d}}+{\stackrel{..}{\zeta }}_{\mathrm{Y}}\end{array}\right)=\left(\begin{array}{c}{v}_{\mathrm{X}}\\ v_{\mathrm{Y}}\end{array}\right)\hfill \end{array}$$

  (123a)

  $$\begin{array}{cc}\hfill {\delta }_{\mathrm{X}}& =d^2+{(\mathrm{X}-{\mathrm{X}}_0-{\zeta }_{\mathrm{X}})}^2{\delta }_{\mathrm{Y}}=d^2+{(\mathrm{Y}-{\mathrm{Y}}_0-{\zeta }_{\mathrm{Y}})}^2\hfill \end{array}$$

  (123b)
• Third, consider one of the most popular model for diabetes, namely the Bergman minimal model (see, e.g., [95,96]):

  $$\begin{array}{cc}\hfill \stackrel{.}{G}& =-k_1(G-G_b)-XG+D=f_G(G,X,D)\hfill \end{array}$$

  (124a)

  $$\begin{array}{cc}\hfill \stackrel{.}{X}& =-k_{2}X+k_3(I-I_b)=f_X(X,I)\hfill \end{array}$$

  (124b)

  $$\begin{array}{cc}\hfill \stackrel{.}{I}& =-k_4(I-I_b)+u=f_I(I,u)\hfill \end{array}$$

  (124c)

  $$\begin{array}{cc}\hfill D\left(t\right)& ={\displaystyle \frac{1}{2}}\left(1+tanh\left(\gamma (t-t_{bol})\right)\right)B{e}^{-d(t-t_{bol})}\hfill \end{array}$$

  (124d)

  where $G\left(t\right)$ is the concentration of blood glucose; $X\left(t\right)$ is the concentration of insulin in the tissue fluid; $I\left(t\right)$ is the concentration of insulin in the blood; $G_b$ and $I_b$ are the basal concentrations of glucose and insulin; and $D\left(t\right)$ is the glycaemic influence of a meal, seen as a fluctuation. In addition, $k_1$, $k_2$, and $k_3$ are positive-valued parameters that control the rates of appearance and disappearance of glucose and insulin; $t_{bol}$ is the meal intake time (bolus intake).
  We wish to control the glucose concentration $G\left(t\right)$ in order to track a desired trajectory $G_r\left(t\right)$ known in advance. To do so, we apply the dynamical extension algorithm, and we have to differentiate Equation (124a) two times in order for the action u to appear. We get

  $$\begin{array}{cc}\hfill \stackrel{..}{G}& =-(k_1+X)\stackrel{.}{G}-G\stackrel{.}{X}+\stackrel{.}{D}\hfill \end{array}$$

  and

  $$\begin{array}{cc}\hfill \stackrel{...}{G}& =-2\stackrel{.}{G}\stackrel{.}{X}-(k_1+X)\stackrel{..}{G}-G\stackrel{..}{X}+\stackrel{..}{D}\hfill \end{array}$$

  and the control will appear in $\stackrel{.}{I}$, present in $\stackrel{..}{X}$. The generalised coordinates unfold as follows. The preceding model is rewritten, within the differentiation $d_1$, as follows:

  $$\begin{array}{cc}\hfill G^{\prime }& =-k_1(G-G_b)-XG+D=f_G(G,X,D)\hfill \end{array}$$

  (125a)

  $$\begin{array}{cc}\hfill X^{\prime }& =-k_{2}X+k_3(I-I_b)=f_X(X,I)\hfill \end{array}$$

  (125b)

  $$\begin{array}{cc}\hfill I^{\prime }& =-k_4(I-I_b)+u=f_I(I,u)\hfill \end{array}$$

  (125c)

  $$\begin{array}{cc}\hfill D\left(t\right)& ={\displaystyle \frac{1}{2}}\left(1+tanh\left(\gamma (t-t_{bol})\right)\right)B{e}^{-d(t-t_{bol})}\hfill \end{array}$$

  (125d)
  The successive application of $d_1$ yields

  $$\begin{array}{cc}\hfill G^{\prime }& ={\partial }_G{f}_G{G}^{\prime }+{\partial }_X{f}_G{X}^{\prime }+{\partial }_D{f}_G{D}^{\prime }=-(k_1+X)G^{\prime }-G{X}^{\prime }+D^{\prime }\hfill \end{array}$$

  and

  $$\begin{array}{cc}\hfill G^{″}& =-(k_1+X)G^{″}-G{X}^{″}+D^{″}\hfill \end{array}$$
  Thus, the term $-2X^{\prime }{G}^{\prime }$ is missing when compared to its counterpart with time differentiation. And, if one attempted to use this in order to produce an action tracking feedback, the resulting error dynamics would be of the form

  $$\begin{array}{cc}\hfill {\epsilon }_G^{\prime \prime \prime }& =-{\lambda }_{G2}{\epsilon }_G^{″}-{\lambda }_{G1}{\epsilon }_G^{\prime }-{\lambda }_{G0}{\epsilon }_G-2X^{\prime }{G}^{\prime }\hfill \end{array}$$

  (126)

  with ${\epsilon }_G=G-G_r$ and $G_r\left(t\right)$ being a reference glucose trajectory. Although the ${\lambda }_{Gi}$ are such that the solutions of

  $$\begin{array}{cc}\hfill {\epsilon }_G^{\prime \prime \prime }+{\lambda }_{G2}{\epsilon }_G^{\prime \prime }+{\lambda }_{G1}{\epsilon }_G^{\prime }+{\lambda }_{G0}{\epsilon }_G& =0\hfill \end{array}$$

  are exponentially decreasing, the solutions of (126) may not tend to zero, since it is steered by $-2X^{\prime }{G}^{\prime }$.

6.2. Smooth Random Realisation

The use of this type of differentiation is also linked to the stochastic realisation problem (see, e.g., [10], 4.(c).(i), p. 15), which may be quite complex in the general case. In contrast to this, the differential flatness property yields a weak Brunovský canonical form through the flat output (see [97], Subsection 4.1 and Definition 4.3). This canonical form yields the so-called flat output dynamics, which readily gives a smooth random realisation, as the following proposition states.

Proposition 4 (Smooth random realisation).

Consider a differentially flat model, with flat output $\omega =({\omega }_1,\dots ,{\omega }_m)$. Then, there exists integers ${\kappa }_1,\dots ,{\kappa }_m$ such that $({\omega }_1,{\stackrel{.}{\omega }}_1,\dots ,{\omega }_1^{({\kappa }_1-1)},{\omega }_2,$$\dots ,$${\omega }_m^{({\kappa }_m-1)})$ is a state. This state, through the flat output dynamics, yields a smooth random realisation of the model.

Proof. 

The existence of the integers ${\kappa }_1,\dots ,{\kappa }_m$ such that $({\omega }_1,$${\stackrel{.}{\omega }}_1,$$\dots {\omega }_1^{({\kappa }_1-1)},{\omega }_2,\dots ,{\omega }_m^{({\kappa }_m-1)})$ is a state is ensured by the dynamical extension algorithm. To see this, and to obtain the associated realisation, recall the so-called flat output dynamics (Equation (53) above), which can be written as the following state-space representation:

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_1& ={\mathrm{x}}_2\hfill \end{array}$$

(127a)

$$⋮$$

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{{\kappa }_1-1}& ={\mathrm{x}}_{{\kappa }_1}\hfill \end{array}$$

(127b)

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{{\kappa }_1}& ={\gamma }_1(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\mathit{\zeta }1}\right)})\hfill \end{array}$$

(127c)

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{{\kappa }_1+1}& ={\mathrm{x}}_{{\kappa }_1+2}\hfill \end{array}$$

(127d)

$$⋮$$

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{{\kappa }_1+{\kappa }_2-1}& ={\mathrm{x}}_{{\kappa }_1+{\kappa }_2}\hfill \end{array}$$

(127e)

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{{\kappa }_1+{\kappa }_2}& ={\gamma }_2(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}1}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\mathit{\zeta }2}\right)})\hfill \end{array}$$

(127f)

$$⋮$$

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_{n-1}& ={\mathrm{x}}_n\hfill \end{array}$$

(127g)

$$\begin{array}{cc}\hfill {\stackrel{.}{\mathrm{x}}}_n& ={\gamma }_m(\mathit{x},\mathit{u},\stackrel{.}{\mathit{u}},\dots ,{\mathit{u}}^{\left({\sigma }_{\mathit{u}m}\right)},\mathit{\zeta },\stackrel{.}{\mathit{\zeta }},\dots ,{\mathit{\zeta }}^{\left({\sigma }_{\mathit{\zeta }m}\right)})\hfill \end{array}$$

(127h)

with $\mathit{x}=({\mathrm{x}}_1,\dots ,{\mathrm{x}}_n)$, since ${\kappa }_1+\dots +{\kappa }_m=n$, and

$$\begin{array}{c}\hfill \begin{array}{ccccccccc}\hfill {\mathrm{x}}_1& \hfill =& \hfill {\omega }_1,& {\mathrm{x}}_2& =& {\stackrel{.}{\omega }}_1,\dots ,& {\mathrm{x}}_{{\kappa }_1}& =& {\omega }_1^{({\kappa }_1-1)}\\ \hfill {\mathrm{x}}_{{\kappa }_1+1}& \hfill =& \hfill {\omega }_2,& {\mathrm{x}}_{{\kappa }_1+2}& =& {\stackrel{.}{\omega }}_2,\dots ,& {\mathrm{x}}_{{\kappa }_1+{\kappa }_2}& =& z^{({\kappa }_2-1)}\\ \hfill & \hfill ⋮& \hfill & & & & & & \\ \hfill {\mathrm{x}}_{{\kappa }_1+\dots +{\kappa }_{m-1}+1}& \hfill =& \hfill {\omega }_m,& \dots & & \dots ,& {\mathrm{x}}_n& =& {\omega }_m^{({\kappa }_m-1)}.\end{array}\end{array}$$

Then $\mathit{x}$ is a state that yields a smooth random realisation of the original system, based on the differential flatness property. □

The present framework can be seen as an adequate proposal for the future direction in Subsection 4.1,, paragraph “Stochastic control via generalized coordinates” of [32]. It also embeds, in a rather simple fashion, non-stationary smooth random signals (see Remark 4.1.1 in [32]).

6.3. Other Future Directions

We now briefly consider future directions that could not be unpacked in this paper due to lack of space.

6.3.1. Extensions of Flatness: Liouvillian Characters

Although many practical system models are differentially flat, some, among specific classes, are not; this is especially the case for many biological and neuroscience population models. Fortunately, another analogous property is shared by a much wider class, the one of so-called Liouvillian systems. Liouvillian systems can be seen as an extension of flat systems [88,89,98]. The most striking property of the latter is that all state and control variables of the system can be directly expressed—without integration of any differential equation—in terms of the flat output and a finite number of its time derivatives. So-called Liouvillian systems share a similar property, but in order to derive the trajectories of a Liouvillian system, we also need integration of a few differential equations whose solutions are known analytically. It follows that flatness-based control approaches can be extended up to solving a finite number of differential equations.

6.3.2. Observers and Algebraic Estimators

We have here dealt with control law synthesis, but we did not touch on the equally important subject of observer or estimator synthesis, i.e., the procedures aimed at estimating the hidden state from sensor measurements. Two main paths are available: The first—of so-called observers—amounts to a simulation of the mean generative model, driven by the error $\mathit{y}-g\left(\mathit{x}\right)$ between the sensor output and its observation model, a function of the state (see, e.g., [99,100]). The second, more direct, approach is to directly make use of the so-called constructive observability (see, e.g., [4,101]), where the state is a function of sensor output, action, and their derivatives: $\mathit{x}={\psi }_{\mathit{x}}(\mathit{y},\mathit{u},\stackrel{.}{\mathit{y}},\stackrel{.}{\mathit{u}},\dots ,{\mathit{y}}^{\left({\rho }_{\mathit{y}}\right)},{\mathit{u}}^{\left({\rho }_{\mathit{u}}\right)})$. This last procedure requires the estimation of sensor output derivatives (see [6,102]).

6.3.3. Robust Control Law Synthesis

When the mean generative model is a crude approximation to the real system, the cumulative effect of the fluctuations needs to be compensated for in the control scheme. So-called robust control laws aim to fulfil a control goal, such as trajectory tracking, despite the fluctuations, or perturbations, and unmodelled dynamics (i.e., model mismatch). These can be achieved via the so-called model-free control scheme, where the cumulative effects of fluctuations are estimated online and compensated for on the fly (see, e.g., [66]). The recent HEOL scheme (see [103]) achieves the same goal through slightly different techniques. While model-free control is often associated with flatness-based feedback tracking control laws synthesised on a nominal, or mean generative, model, HEOL one uses an open-loop flatness-based scheme and the tangent, or variational system, associated with the simplified flat system, i.e., the linearised system around a reference trajectory of the simplified flat system. Other instances include, among others, sliding mode control and active disturbance rejection control (see, e.g., [62,64,65]).

6.3.4. Constraint Fulfilment

In real-world applications, dynamical systems are always subject to constraints: on the state (for example the configuration space of a robot is not the whole space) and on the action (e.g., muscles have finite power). These can be handled in the present framework through optimisation-based planning of the flat output trajectories (see, e.g., [104,105,106]). A promising framework is the one of model-free predictive control (MFPC, see [107]), mixing the popular predictive control (see, e.g., [108,109]) and model-free control, cited above.

6.3.5. Feature Transmission

Another interesting characteristic—of the functional parametrisation associated with differential flatness—is one of feature transmission. This includes the transmission of geometric features: how the curvature of the flat output trajectory is related to the curvature of the action; in other words, deducing the action’s curvature as a function of the flat output curvature and its time derivatives from the relation yielding the action as a function of the flat output and its derivatives.