Dimensionless Policies Based on the Buckingham π Theorem: Is This a Good Way to Generalize Numerical Results?

Abstract

The answer to the question posed in the title is yes if the context (the list of variables defining the motion control problem) is dimensionally similar. This article explores the use of the Buckingham $\pi$ theorem as a tool to encode the control policies of physical systems into a more generic form of knowledge that can be reused in various situations. This approach can be interpreted as enforcing invariance to the scaling of the fundamental units in an algorithm learning a control policy. First, we show, by restating the solution to a motion control problem using dimensionless variables, that (1) the policy mapping involves a reduced number of parameters and (2) control policies generated numerically for a specific system can be transferred exactly to a subset of dimensionally similar systems by scaling the input and output variables appropriately. Those two generic theoretical results are then demonstrated, with numerically generated optimal controllers, for the classic motion control problem of swinging up a torque-limited inverted pendulum and positioning a vehicle in slippery conditions. We also discuss the concept of regime, a region in the space of context variables, that can help to relax the similarity condition. Furthermore, we discuss how applying dimensional scaling of the input and output of a context-specific black-box policy is equivalent to substituting new system parameters in an analytical equation under some conditions, using a linear quadratic regulator (LQR) and a computed torque controller as examples. It remains to be seen how practical this approach can be to generalize policies for more complex high-dimensional problems, but the early results show that it is a promising transfer learning tool for numerical approaches like dynamic programming and reinforcement learning.

1. Introduction

To solve challenging motion control problems in robotics (locomotion, manipulation, vehicle control, etc.), many approaches now include a type of mathematical optimization that has no closed-form solution and that is solved numerically, either online (trajectory optimization [1], model predictive control [2], etc.) or offline (reinforcement learning [3]). Numerical tools, however, have a major drawback compared to simpler analytical approaches: the parameters of the problem do not appear explicitly in the solutions, which makes it much harder to generalize and reuse the results. Analytical solutions to control problems have the useful property of allowing the solution to be adjusted to different system parameters by simply substituting the new values in the equation. For instance, an analytical feedback law solution to a robot motion control problem can be transferred to a similar system by adjusting the values of the parameters (lengths, masses, etc.) in the equation. However, with a reinforcement learning solution, we would have to re-conduct all the training, implying (generally) multiple hours of data collection and/or computation. It would be a great asset to have the ability to adjust black-box numerical solutions with respect to some problem parameters.

In this paper, we explore the concept of dimensionless policies, a more generic form of knowledge conceptually illustrated in Figure 1, as an approach to generalize numerical solutions to motion control problems. First, in Section 2, we use dimensional analysis (i.e., the Buckingham $\pi$ theorem [4]) to show that motion control problems with dimensionally similar context variables must share the same feedback law solution when expressed in a dimensionless form, and discuss the implications. Two main generic theoretical results, relevant for any physically meaningful control policies, are presented as Theorems 1 and 2. Then, in Section 3, we present two case studies with numerical results. Optimal feedback laws computed with a dynamic programming algorithm are used to demonstrate the theoretical results and their relevance for (1) the classical motion control problem of swinging up an inverted pendulum in Section 3.1 and (2) a car motion control problem in Section 3.2. Furthermore, in Section 4, we illustrate—with two examples—how the proposed dimensional scaling is equivalent to changing parameters in an analytical solution.

A very promising application of the concept of dimensionless policies is to empower reinforcement learning schemes, for which data efficiency is critical [5]. For instance, it would be interesting to use the data of all the vehicles on the road, even if they are of varying dimensions and dynamic characteristics, to learn appropriate maneuvers in situations that occur very rarely. This idea of reusing data or results in a different context is usually called transfer learning [6] and has received a great deal of research attention, mostly targeted at applying a learned policy to new tasks. The more specific idea of transferring policies and data between systems/robots has also been explored, with schemes based on modular blocks [7], invariant features [8], a dynamic map [9], a vector representation of each robot hardware [10], or using tools from adaptive control [11] and robust control [12]. Dimensionless numbers and dimensional analysis comprise a technique based on the idea that some relationships should not depend on units that can be used for analyzing many physical problems [4,13,14]. The most well-known application in the field of fluid mechanics is the idea of matching ratios (i.e., Reynolds, Prandtl, or Mach numbers) to allow for the generalization of experimental results between systems of various scales. The recent success of machine learning and data-driven schemes highlights the question of generalizing results, and there is renewed interest in using dimensional analysis in the context of learning [15,16,17]. In this paper, we present an initial exploration of how dimensional analysis can be applied specifically to help generalize policy solutions for motion control problems involving physically meaningful variables like force, length, mass, and time.

2. Dimensionless Policies

In the following section, we develop the concept of dimensionless policies based on the Buckingham $\pi$ theorem and present generic theoretical results that are relevant for any type of physically meaningful control policy.

2.1. Context Variables in the Policy Mapping

Here, we call a feedback law mapping f, specific to a given system, from a vector space representing the state x of the dynamic system to a vector space representing the control inputs u of the system:

$$u=f\left(x\right)$$

(1)

Under some assumptions (fully observable systems, additive costs, and infinite time horizon), the optimal feedback law is guaranteed to be in this state feedback form [18]. We will only consider motion control problems that lead to such time-independent feedback laws in the following analysis. To consider the question of how this system-specific feedback law can be transferred to a different context, it is useful to think about higher dimensional mapping $\pi$, which is herein referred to as a policy, also having a vector of variables c describing the context as an additional input argument as illustrated in Figure 2.

Definition 1. 

A policy is defined as the solution to a motion control problem in the form of a function computing the control inputs u from the system states x and context parameters c as follows:

$$\begin{array}{cc}\hfill u=& \pi \left(x,c\right)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill \mathit{with}u\in {\mathbf{R}}^k& x\in {\mathbf{R}}^{n}c\in {\mathbf{R}}^m\hfill \end{array}$$

(3)

where k is the dimension of the control input vector, n is the dimension of the dynamic system state vector, and k is the dimension of the vector of context parameters.

The context c is a vector of relevant parameters defining the motion control problem, i.e., parameters that affect the feedback law solution. The policy $\pi$ is thus a mapping consisting of the feedback law solutions for all possible contexts. In Section 3.1, a case study is conducted by considering the optimal feedback law for swinging up a torque-limited inverted pendulum. For this example, the context variables are the pendulum mass m, the gravitational constant g, and the length l, as well as what we call task parameters: a weight parameter in the cost function q and a constraint ${\tau }_{max}$ on the maximum input torque. For a given pendulum state, the optimal torque is also a function of the context variables; i.e., the solution is different if the pendulum is heavier or more torque-limited.

Definition 2. 

A feedback law with a subscript letter a is defined as the solution to a motion problem for a specific situation defined by an instance of context variables $c_a$, as follows:

$$f_a\left(x\right)=\pi \left(x,c=c_a\right)\forall x$$

(4)

The feedback law $f_a$ thus represents a slice of the global policy when the context variables are fixed at $c_a$ values as illustrated in Figure 3.

The goal of generalizing a feedback law to a different context can thus be formulated into the following question: if a feedback law $f_a$ is known for a context described by variables $c_a$, can this knowledge help us to deduce the policy solution in a different context, namely $c_b$?

$$\pi \left(x,c=c_a\right)=f_a\left(x\right)\Rightarrow \pi \left(x,c=c_b\right)=?$$

(5)

Using the Buckingham $\pi$ theorem [4], we will show that, if the context is dimensionally similar, then both feedback laws must be equal up to scaling factors (Theorem 2).

2.2. Buckingham $\pi$ Theorem

The Buckingham $\pi$ theorem [4] is a tool based on dimensional analysis [13,14] that enables restating a relationship involving multiple physically meaningful dimensional variables using fewer dimensionless variables:

$$x_1=f(x_2,\dots ,x_n)\Rightarrow {\Pi }_1=f({\Pi }_2,\dots ,{\Pi }_p)$$

(6)

If d fundamental dimensions are involved in the n dimensional variables (for instance, time [T], length [L], and mass [M]), then the number of required dimensionless variables, often called $\Pi$ groups, is $p\ge n-d$. In most situations, the number of variables in the relationship can be reduced directly by the number of fundamental dimensions involved, and $p=n-d$. The Buckingham $\pi$ theorem provides a methodology to generate the $\Pi$ groups; however, the choice of $\Pi$ groups is not unique. The approach is to select (arbitrarily) d variables involving the d fundamental dimensions independently, called the repeated variables. Then, the $\Pi$ groups are generated by multiplying all the other variables by the repeated variables exponentiated by rational exponents selected to make the group dimensionless. Assuming $x_1$, … $x_d$, where the selected repeated variables, the $\Pi$ groups are

$${\Pi }_i=x_{d+i}\underset{\mathrm{Repeated}\mathrm{variables}}{\underbrace{x_1^{e_{1i}}{x}_2^{e_{2i}}\dots x_d^{e_{di}}}}i=\{1,\dots ,p\}$$

(7)

Finding the correct exponents to make all groups dimensionless can be formulated as solving a linear system of d equations. We refer to previous literature for more details on the theorem, and here use it specifically on the defined concept of policy map.

2.3. Dimensional Analysis of the Policy Mapping

If a policy is physically meaningful (for example, a policy that computes a force based on position and velocity, but not a policy for playing chess), we can use the Buckingham $\pi$ theorem to simplify the policy in dimensionless form.

Theorem 1. 

If a policy is physically meaningful and all its variables involve d fundamental dimensions that are independently present in the context variables c, then the policy can be restated in a dimensionless form as follows:

$$\begin{array}{cc}\hfill u=\pi (x,c)& \Rightarrow u^{\ast }={\pi }^{\ast }(x^{\ast },c^{\ast })\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill u\in {\mathbf{R}}^{k}x\in {\mathbf{R}}^{n}c\in {\mathbf{R}}^m& u^{\ast }\in {\mathbf{R}}^k{x}^{\ast }\in {\mathbf{R}}^n{c}^{\ast }\in {\mathbf{R}}^{(m-d)}\hfill \end{array}$$

(9)

where the dimensionless variables can be related to dimensional variables using transformation matrices that depend only on the context variables as follows:

$$\begin{array}{cc}\hfill u^{\ast }& =\left[T_u\left(c\right)\right]u\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill x^{\ast }& =\left[T_x\left(c\right)\right]x\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill c^{\ast }& =\left[T_c\left(c\right)\right]c\hfill \end{array}$$

(12)

Furthermore, the transformation matrices can be used to relate the dimensional and dimensionless policy as follows:

$$\begin{array}{c}\hfill \pi (x,c)=T_u^{-1}\left(c\right){\pi }^{\ast }\left(T_x\left(c\right)x,T_c\left(c\right)c\right)\end{array}$$

(13)

Proof. 

For a system with k control inputs, we can treat the policy as k mappings from states and context variables to each scalar control input $u_j$:

$$u_j={\pi }_j\left(x_1,\dots ,x_n,c_1,\dots \dots ,c_m\right)$$

(14)

where Equation (14) is the jth line of the policy in vector form, as described by Equation (2). Then, if the state vector is defined by n variables, and the context is defined by m (system and task) parameters, then each mapping ${\pi }_j$ is a relation between $1+n+m$ variables. Under the assumption that the policy involves physically meaningful variables, and that it is invariant under an arbitrary scaling of any fundamental dimensions—i.e., independent of a system of units—then we can apply the Buckingham $\pi$ theorem [4] to conclude that, if d dimensions are involved in all the variables, then Equation (14) can be restated as an equivalent relationship between p dimensionless $\Pi$ groups where $p\ge 1+n+m-d$. Assuming that d dimensions are involved in the m context variables, and that we are in the typical scenario where maximum reduction is possible ($p=1+n+m-d$), then we can select d context variables $\{c_1,c_2,\dots ,c_d\}$ as the basis (the repeated variables) to scale all other variables into dimensionless $\Pi$ groups. We denote dimensionless $\Pi$ group as the base variables with an asterisk (*) as follows:

$$\begin{array}{cc}\hfill u_j^{\ast }& =u_j{\left[c_1\right]}^{e_{1j}^u}{\left[c_2\right]}^{e_{2j}^u}\dots {\left[c_d\right]}^{e_{dj}^u}j=\{1,\dots ,k\}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill x_i^{\ast }& =x_i{\left[c_1\right]}^{e_{1i}^x}{\left[c_2\right]}^{e_{2i}^x}\dots {\left[c_d\right]}^{e_{di}^x}i=\{1,\dots ,n\}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill c_i^{\ast }& =c_i{\left[c_1\right]}^{e_{1i}^c}{\left[c_2\right]}^{e_{2i}^c}\dots {\left[c_d\right]}^{e_{di}^c}i=\{d+1,\dots ,m\}\hfill \end{array}$$

(17)

where exponents $e_{ij}$ are rational numbers selected to make all equations dimensionless. We can then define transformation matrices and write Equations (15)–(17) in a vector form where the repeated variables are grouped into matrices defined as the following:

$$\begin{array}{cc}\hfill \underset{u^{\ast }}{\underbrace{\left[\begin{array}{c}{u}_1^{\ast }\\ ⋮\\ u_k^{\ast }\end{array}\right]}}& =\underset{T_u\left(c\right)}{\underbrace{\left[\begin{array}{ccc}\left({\left[c_1\right]}^{e_{11}^u}{\left[c_2\right]}^{e_{21}^u}\dots {\left[c_d\right]}^{e_{d1}^u}\right)& 0& 0\\ 0& \ddots & 0\\ 0& 0& \left({\left[c_1\right]}^{e_{1k}^u}{\left[c_2\right]}^{e_{2k}^u}\dots {\left[c_d\right]}^{e_{dk}^u}\right)\end{array}\right]}}\underset{u}{\underbrace{\left[\begin{array}{c}{u}_1\\ ⋮\\ u_k\end{array}\right]}}\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill \underset{x^{\ast }}{\underbrace{\left[\begin{array}{c}{x}_1^{\ast }\\ ⋮\\ x_n^{\ast }\end{array}\right]}}& =\underset{T_x\left(c\right)}{\underbrace{\left[\begin{array}{ccc}\left({\left[c_1\right]}^{e_{11}^x}{\left[c_2\right]}^{e_{21}^x}\dots {\left[c_d\right]}^{e_{d1}^x}\right)& 0& 0\\ 0& \ddots & 0\\ 0& 0& \left({\left[c_1\right]}^{e_{1n}^x}{\left[c_2\right]}^{e_{2n}^x}\dots {\left[c_d\right]}^{e_{dn}^x}\right)\end{array}\right]}}\underset{x}{\underbrace{\left[\begin{array}{c}{x}_1\\ ⋮\\ x_n\end{array}\right]}}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \underset{c^{\ast }}{\underbrace{\left[\begin{array}{c}{c}_{d+1}^{\ast }\\ ⋮\\ c_m^{\ast }\end{array}\right]}}& =\underset{T_c\left(c\right)}{\underbrace{\left[\begin{array}{cccccc}0& \dots & 0& \left({\left[c_1\right]}^{e_{1(d+1)}^u}{\left[c_2\right]}^{e_{2(d+1)}^u}\dots {\left[c_d\right]}^{e_{d(d+1)}^u}\right)& 0& 0\\ 0& \dots & 0& 0& \ddots & 0\\ 0& \dots & 0& 0& 0& \left({\left[c_1\right]}^{e_{1m}^u}{\left[c_2\right]}^{e_{2m}^u}\dots {\left[c_d\right]}^{e_{dm}^u}\right)\end{array}\right]}}\underset{c}{\underbrace{\left[\begin{array}{c}{c}_1\\ ⋮\\ c_m\end{array}\right]}}\hfill \end{array}$$

(20)

which correspond to Equations (10)–(12). Matrices $T_u$ and $T_x$ are square diagonal matrices and Equations (10) and (11) are thus inversibles (unless a repeated variable is equal to zero) and can be used to go back and forth between dimensional and dimensionless states and input variables. Matrix $T_c$ consists of a block of d columns of zeros, followed by a diagonal block of dimensions $(m-d)\times (m-d)$, and Equation (12) is not inversible. For a given context c, there is only one dimensionless context $c^{\ast }$; however, a given dimensionless context $c^{\ast }$ may correspond to multiple dimensional contexts c.

Then, the Buckingham $\pi$ theorem indicates that the relationship described by Equation (14) can be restated in a relationship between the $\Pi$ groups involving d less variables, which, based on the selected repeated variable, corresponds to

$$u_j^{\ast }={\pi }_j^{\ast }\left(x_1^{\ast },\dots ,x_n^{\ast },c_{d+1}^{\ast },\dots ,c_m^{\ast }\right)$$

(21)

By applying the same procedure to all control inputs, we can then assemble all k mappings back into a vector form, as follows:

$$\underset{\mathrm{Dimensionless}\mathrm{feedback}\mathrm{law}{f}^{\ast }}{\underbrace{\left[\begin{array}{c}{u}_1^{\ast }\\ ⋮\\ u_k^{\ast }\end{array}\right]={\pi }^{\ast }(\left[\begin{array}{c}{x}_1^{\ast }\\ ⋮\\ x_n^{\ast }\end{array}\right]}},\underset{\mathrm{context}{c}^{\ast }}{\underbrace{\left[\begin{array}{c}{c}_{d+1}^{\ast }\\ ⋮\\ c_m^{\ast }\end{array}\right]}})$$

(22)

corresponding to Equation (8). Finally, based on the defined transformations in Equations (10)–(12), we can relate the dimensional policy to the dimensionless version as follows:

$$\begin{array}{c}\hfill \pi (x,c)=\underset{u}{\underbrace{T_u^{-1}\left(c\right)\underset{u^{\ast }}{\underbrace{{\pi }^{\ast }\left(\underset{x^{\ast }}{\underbrace{T_x\left(c\right)x}},\underset{c^{\ast }}{\underbrace{T_c\left(c\right)c}}\right)}}}}\end{array}$$

(23)

which corresponds to Equation (13). □

2.4. Transferring Feedback Laws between Similar Systems

Based on the dimensional analysis, we can demonstrate that any feedback law can be generalized to a different context under the condition of dimensional similarity. In this section, we show that a feedback law can be transferred exactly to another motion control problem by scaling the input and output of the function based on matrices that can be computed using the dimensional analysis. The salient feature of this result is that the conditions are very generic; even a black-box discontinuous non-linear policy (such as those obtained using deep-reinforcement learning algorithms) can be transferred this way. The limitation is that the condition for an exact transfer is having equal dimensionless context variables $c^{\ast }$.

First, it is useful to define dimensionless feedback laws that correspond to specific cases of the dimensionless policy, as we defined for the dimensional mapping.

Definition 3. 

We denote a dimensionless feedback law $f_a^{\ast }$, the global dimensionless policy for a specific instance of context variables $c_a$, as follows:

$$f_a^{\ast }\left(x^{\ast }\right)={\pi }^{\ast }(x^{\ast },c^{\ast }=c_a^{\ast })\forall x^{\ast }$$

(24)

where $c_a^{\ast }$ is the dimensionless version of the context variables instance $c_a$, and equal to

$$c_a^{\ast }=T_c\left(c_a\right)c_a$$

(25)

Lemma 1. 

Two feedback laws, which are solutions to the same motion control problem for two instances of context variables, will be equal in dimensionless form if they share the same dimensionless context:

$$f_b^{\ast }\left(x^{\ast }\right)=f_a^{\ast }\left(x^{\ast }\right)\forall x^{\ast }if{c}_a^{\ast }=c_b^{\ast }$$

(26)

Proof. 

This follows from the definition:

$$\begin{array}{cc}\hfill f_a^{\ast }\left(x^{\ast }\right)& ={\pi }^{\ast }(x^{\ast },c^{\ast }=c_a^{\ast })\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill f_a^{\ast }\left(x^{\ast }\right)& ={\pi }^{\ast }(x^{\ast },c^{\ast }=c_b^{\ast })\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill f_a^{\ast }\left(x^{\ast }\right)& =f_b^{\ast }\left(x^{\ast }\right)\hfill \end{array}$$

(29)

□

Lemma 2. 

In a specific context described by variables $c_a$, a dimensional feedback law can be restated in dimensionless form, and vice versa, by scaling the input and the output using the defined transformation matrices $T_x\left(c_a\right)$ and $T_u\left(c_a\right)$ as follows:

$$\begin{array}{cc}\hfill f_a\left(x\right)& =T_u^{-1}\left(c_a\right)\underset{u^{\ast }}{\underbrace{f_a^{\ast }\left(\underset{x^{\ast }}{\underbrace{T_x\left(c_a\right)x}}\right)}}\forall x\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill f_a^{\ast }\left(x^{\ast }\right)& =T_u\left(c_a\right)\underset{u}{\underbrace{f_a\left(\underset{x}{\underbrace{T_x^{-1}\left(c_a\right)x^{\ast }}}\right)}}\forall x^{\ast }\hfill \end{array}$$

(31)

Proof. 

Starting from Equation (13) and substituting $c^{\ast }$ with a specific instance $c_a^{\ast }$, then substituting policy maps on each side with feedback laws $f_a$ and $f_a^{\ast }$ based on the definition, we obtain Equation (30):

$$\begin{array}{cc}\hfill \pi (x,c_a)& =T_u^{-1}\left(c_a\right){\pi }^{\ast }\left(T_x\left(c_a\right)x,T_c\left(c_a\right)c_a\right)\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill f_a\left(x\right)& =T_u^{-1}\left(c_a\right)f_a^{\ast }\left(T_x\left(c_a\right)x\right)\hfill \end{array}$$

(33)

Then, starting from the right side of Equation (31) and substituting the function $f_a$ with Equation (30), the matrices are reduced to identity matrices and we obtain Equation (31):

$$\begin{array}{cc}\hfill T_u\left(c_a\right)f_a\left(T_x^{-1}\left(c_a\right)x^{\ast }\right)& =T_u\left(c_a\right)T_u^{-1}\left(c_a\right)f_a^{\ast }\left(T_x^{-1}\left(c_a\right)T_x\left(c_a\right)x^{\ast }\right)\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill T_u\left(c_a\right)f_a\left(T_x^{-1}\left(c_a\right)x^{\ast }\right)& =f_a^{\ast }\left(x^{\ast }\right)\hfill \end{array}$$

(35)

□

Theorem 2. 

If a feedback law $f_a$ is known—for instance, as the result of a numerical algorithm—and this is the solution to a motion control problem with context variables $c_a$, we can compute the solution $f_b$ to the same motion control problem for different context variables $c_b$ by scaling the input and output of $f_a$ as follows:

$$\begin{array}{cc}\hfill f_b\left(x\right)& =\left[T_u^{-1}\left(c_b\right)T_u\left(c_a\right)\right]f_a\left(\left[T_x^{-1}\left(c_a\right)T_x\left(c_b\right)\right]x\right)\forall x\hfill \end{array}$$

(36)

if contexts $c_a$ and $c_b$ are dimensionally similar, i.e., if the following condition is true:

$$T_c\left(c_b\right)c_b=T_c\left(c_a\right)c_a$$

(37)

Proof. 

First, $f_b$ can be written based on its dimensionless form $f_b^{\ast }$ in a context $c_b$ using Equation (30) from Lemma 2. Also, based on Lemma 1, under the similarity condition—i.e., $c_b^{\ast }=c_a^{\ast }$ or equivalently $T\left(c_b\right)c_b=T\left(c_a\right)c_a$— we have that $f_b^{\ast }$ is equal to $f_a^{\ast }$. Finally, $f_a^{\ast }$ can be written based on its dimensional form $f_a$ in a context $c_a$, using Equation (31) from Lemma 2, as follows:

$$\begin{array}{cc}\hfill f_b\left(x\right)& =T_u^{-1}\left(c_b\right)f_b^{\ast }\left(T_x\left(c_b\right)x\right)\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill f_b\left(x\right)& =T_u^{-1}\left(c_b\right)f_a^{\ast }\left(T_x\left(c_b\right)x\right)\mathrm{if}{c}_b^{\ast }=c_a^{\ast }\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill f_b\left(x\right)& =\left[T_u^{-1}\left(c_b\right)T_u\left(c_a\right)\right]f_a\left(\left[T_x^{-1}\left(c_a\right)T_x\left(c_b\right)\right]x\right)\hfill \end{array}$$

(40)

□

The idea is summarized in Figure 4. To transfer a feedback law, we must first extract the dimensionless form, a more generic form of knowledge, and then scale it back to the new context.

2.5. Dimensionally Similar Contexts

Equation (36) can be used to scale a policy for an exact transfer of policy solutions regarding context c sharing the same dimensionless context $c^{\ast }$, a condition that is referred to as dimensionally similar. Equation (12) is a mapping from an m dimensional space to an $m-d$ dimensional space, and its inverse has multiple solutions. A given dimensionless context $c^{\ast }$ corresponds to a subset of all possible values of dimensional context c. As illustrated in Figure 5 and Figure 6 with low-dimensional examples ($m=2$ and $d=1$), the subsets of context c leading to the same $c^{\ast }$ can be linear if $c^{\ast }$ is just a ratio of two variables of the same dimensions, or a non-linear curve if $c^{\ast }$ involves exponents leading to a more complex polynomial relationship. In general, when the context c involves many dimensions, it is important to note that the similarity condition means meeting multiple conditions (one for each element of the vector $c^{\ast }$) in a higher dimensional space, as illustrated in Figure 7 for the pendulum swing-up example that is studied in the next section. To some degree, this dimensionally similar context condition is a technique to regroup the motion control problems that are the same up to scaling factors. Therefore, it is also logical that their solutions should be equivalent up to scaling factors.

2.6. Summary of the Theoretical Results

The dimensional analysis leads us to the following relevant theoretical results, which are very generic since no assumptions on the form of the policy function are necessary:

• The global problem of learning $\pi (x,c)$, i.e., the feedback policies for all possible contexts, is simplified in a dimensionless form ${\pi }^{\ast }(x^{\ast },c^{\ast })$ because we can remove d input dimensions from the unknown mapping (typically, d would be 2 or 3 for controlling a physical system involving time, force, and length); see Theorem 1.
• The feedback law solutions of dimensionally similar subset of contexts share the exact same solution when restated in a dimensionless form; see Lemma 1.
• A feedback law, which is a solution to a motion control problem in a context, can be transferred exactly to another context, under a condition of dimensional similarity, by appropriately scaling its inputs and outputs; see Theorem 2.

Just for illustration purposes, let us imagine we have a policy for a spherical submarine where the context is defined by a velocity, a viscosity, and a radius. In dimensionless form, we would find that the context can be described by a single variable, the Reynolds number, and that (1) learning the policy will be easier in dimensionless form because it is a function of a lesser number of variables and (2) if we know the feedback law solution for a specific context of velocity, viscosity, and radius, then we can actually re-use it for multiple versions of the same motion control problem sharing the same Reynolds number.

3. Case Studies with Numerical Results

In this section, we use numerically generated optimal policy solutions for two motion control problems as examples illustrating the salient features of the presented theoretical results of Section 2 and the potential for transfer learning.

3.1. Optimal Pendulum Swing-Up Task

The first numerical example is the classical pendulum swing-up task. This example illustrates that an optimal feedback law in the form of a table look-up generated for a pendulum of a given mass and length can be transferred to a pendulum of a different mass and length if the motion control problem is dimensionally similar. The example is also used to introduce the concept of regime for motion control problems.

3.1.1. Motion Control Problem

The motion control problem is defined here as finding a feedback law for controlling the dynamic system described by the following differential equation:

$$m{l}^2\ddot{\theta }-mglsin\theta =\tau$$

(41)

which minimizes the infinite horizon quadratic cost function provided by

$$J={\int }_0^{\infty }\left(q^2{\theta }^2+{\tau }^2\right)dt$$

(42)

subject to input constraints provided by

$$-{\tau }_{max}\le \tau \le {\tau }_{max}$$

(43)

Note that, here, (1) the cost function parameter q has a power of two to allow its value to be in units of torque; (2) it was chosen not to penalize high velocity values for simplicity; (3) the weight multiplying the torque is set to one without a loss of generality as only the relative values of weights impact the optimal solution; and (4) all parameters are time-independent constants. Thus, assuming that there are no hidden variables and that Equations (41)–(43) fully describe the problem, the solution—i.e., the optimal policy for all contexts—involves the variables listed in

Table 1. Pendulum swing-up optimal policy variables.

Variable	Description	Units	Dimensions
Control inputs
$\tau$	Actuator torque	Nm	[$M{L}^2{T}^{-2}$]
State variables
$\theta$	Joint angle	rad	[]
$\dot{\theta }$	Joint angular velocity	rad/s	[$T^{-1}$]
System parameters
m	Pendulum mass	kg	[M]
g	Gravity	m/s2	[$L{T}^{-2}$]
l	Pendulum lenght	m	[L]
Task parameters
q	Weight parameter	Nm	[$M{L}^2{T}^{-2}$]
${\tau }_{max}$	Maximum torque	Nm	[$M{L}^2{T}^{-2}$]

, and should be of the form provided by

$$\underset{\mathrm{inputs}}{\underbrace{\tau }}=\pi \left(\underset{\mathrm{states}}{\underbrace{\theta ,\dot{\theta }}},\underset{\mathrm{Context}c}{\underbrace{\underset{\mathrm{system}\mathrm{parameters}}{\underbrace{m,g,l}},\underset{\mathrm{task}\mathrm{parameters}}{\underbrace{q,{\tau }_{max}}}}}\right)$$

(44)

It is interesting to note that, while there are three system parameters, m, g, and l, they only appear independently in two groups in the dynamic equation. We can thus consider only two system parameters. For convenience, we selected $mgl$, corresponding to the maximum static gravitational torque (i.e., when the pendulum is horizontal), and the natural frequency $\omega =\sqrt{\frac{g}{l}}$, as listed in

Table 2. Pendulum reduced system parameters.

Variable	Description	Units	Dimensions
$mgl$	Maximum gravitational torque	Nm	[$M{L}^2{T}^{-2}$]
$\omega =\sqrt{\frac{g}{l}}$	Natural frequency	s−1	[$T^{-1}$]

.

3.1.2. Dimensional Analysis

Here, we have one control input, two states, two system parameters, and two task parameters, for a total of $1+(n=2)+(m=4)=7$ variables involved. In those variables, only $d=2$ independent dimensions ($M{L}^2{T}^{-2}$ and $T^{-1}$) are present. Using $c_1=mgl$ and $c_2=\omega$ as the repeated variables leads to the following dimensionless groups:

$$\begin{array}{cc}\hfill {\Pi }_1& ={\tau }^{\ast }=\frac{\tau }{mgl}\frac{\left[M{L}^2{T}^{-2}\right]}{\left[M\right]\left[L{T}^{-2}\right]\left[L\right]}\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill {\Pi }_2& ={\theta }^{\ast }=\theta \left[\right]\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\Pi }_3& ={\dot{\theta }}^{\ast }=\frac{\dot{\theta }}{\omega }\frac{\left[T^{-1}\right]}{\left[T^{-1}\right]}\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill {\Pi }_4& ={\tau }_{max}^{\ast }=\frac{{\tau }_{max}}{mgl}\frac{\left[M{L}^2{T}^{-2}\right]}{\left[M\right]\left[L{T}^{-2}\right]\left[L\right]}\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\Pi }_5& =q^{\ast }=\frac{q}{mgl}\frac{\left[M{L}^2{T}^{-2}\right]}{\left[M\right]\left[L{T}^{-2}\right]\left[L\right]}\hfill \end{array}$$

(49)

All three torque variables ($\tau$, q, and ${\tau }_{max}$) are scaled by the maximum gravitational torque, and the pendulum velocity variable is scaled by the natural pendulum frequency. The transformation matrices are thus

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =\underset{T_u}{\underbrace{\left[1/mgl\right]}}\tau \hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{\theta }^{\ast }\\ {\dot{\theta }}^{\ast }\end{array}\right]& =\underset{T_x}{\underbrace{\left[\begin{array}{cc}1& 0\\ 0& 1/\omega \end{array}\right]}}\left[\begin{array}{c}\theta \\ \dot{\theta }\end{array}\right]\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill \underset{c^{\ast }}{\underbrace{\left[\begin{array}{c}{q}^{\ast }\\ {\tau }_{max}^{\ast }\end{array}\right]}}& =\underset{T_c}{\underbrace{\left[\begin{array}{cccc}0& 0& 1/mgl& 0\\ 0& 0& 0& 1/mgl\end{array}\right]}}\underset{c}{\underbrace{\left[\begin{array}{c}mgl\\ \omega \\ q\\ {\tau }_{max}\end{array}\right]}}\hfill \end{array}$$

(52)

By applying the Buckingham $\pi$ theorem [4], Equation (44) can be restated as a relationship between the five dimensionless $\Pi$ groups:

$${\tau }^{\ast }={\pi }^{\ast }\left(\theta ,{\dot{\theta }}^{\ast },q^{\ast },{\tau }_{max}^{\ast }\right)$$

(53)

According to the results of Section 2, for dimensionally similar swing-up contexts (meaning those with equal $q^{\ast }$ and ${\tau }_{max}^{\ast }$ ratios), the optimal feedback laws should be equivalent in their dimensionless forms. In other words, the optimal policy $f_a$, found in the specific context $c_a=[m_a,l_a,g_a,q_a,{\tau }_{max,a}]$, and the optimal policy $f_b$, in a second context, $c_b=[m_b,l_b,g_b,q_b,{\tau }_{max,b}]$, are equal when restated in dimensionless form: $f_a^{\ast }=f_b^{\ast }$ if $q_a^{\ast }=q_b^{\ast }$ and ${\tau }_{max,a}^{\ast }={\tau }_{max,b}^{\ast }$. Furthermore, $f_b$ can be obtained from $f_a$ or vice versa using the scaling formula provided by Equation (36) if this condition is met. However, if $q_a^{\ast }\ne q_b^{\ast }$ or ${\tau }_{max,a}^{\ast }\ne {\tau }_{max,b}^{\ast }$, then $f_a$ cannot provide us with information on $f_b$ without additional assumptions. Figure 7 illustrates that, for the pendulum swing-up problem, the similarity condition can be represented as a line in a three-dimensional space created by three dimensional context variables. Each condition of equal values of $q^{\ast }$ and ${\tau }_{max}^{\ast }$ is a plane in this space, and the intersection of the two planes is the subset of context meeting the two conditions. Also, it is interesting to note that the fourth context variable $\omega$ is not an additional axis here because it is not involved in Equation (52).

3.1.3. Numerical Results

Here, we use a numerical algorithm (methodological details are presented in Section 3.1.5) to compute numerical solutions to the motion control problem defined by Equations (41)–(43). The algorithm computes feedback laws in the form of look-up tables, based on a discretized grid of the state space. The optimal (up to discretization errors) feedback laws are computed for nine instances of context variables, which are listed in

Table 3. Pendulum swing-up problem context variables.

	m	g	l	q	${\mathit{\tau }}_{\mathit{max}}$
Problems with ${\tau }_{max}^{\ast }=0.5$ and $q^{\ast }=0.1$
Context $c_a$:	1.0	10.0	1.0	1.0	5.0
Context $c_b$:	1.0	10.0	2.0	2.0	10.0
Context $c_c$:	2.0	10.0	1.0	2.0	10.0
Problems with ${\tau }_{max}^{\ast }=1.0$ and $q^{\ast }=0.05$
Context $c_d$:	1.0	10.0	1.0	0.5	10.0
Context $c_e$:	1.0	10.0	2.0	1.0	20.0
Context $c_f$:	2.0	10.0	1.0	1.0	20.0
Problems with ${\tau }_{max}^{\ast }=1.0$ and $q^{\ast }=10$
Context $c_g$:	1.0	10.0	1.0	100.0	10.0
Context $c_h$:	1.0	10.0	2.0	200.0	20.0
Context $c_i$:	2.0	10.0	1.0	200.0	20.0

. In those nine contexts, there are three subsets of three dimensionally similar contexts. Also, each subset includes the same three pendulums: a regular pendulum, one that is two times longer, and one that is twice as heavy (as illustrated in Figure 1). Contexts $c_a$, $c_b$, and $c_c$ describe a task where the torque is limited to half the maximum gravitational torque. Contexts $c_d$, $c_e$, and $c_f$ describe a task where the application of large torques is highly penalized by the cost function. Contexts $c_g$, $c_h$, and $c_i$ describe a task where position errors are highly penalized by the cost function.

Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16 illustrate that, for each subset with equal dimensionless context, the dimensional feedback laws generated look numerically very similar. They are similar up to the scaling of their axis, if we neglect slight differences due to discretization errors. Furthermore, the figures also illustrate that the dimensionless version of the feedback laws ($f^{\ast }$), computed using Equation (31), is equal within each dimensionally similar subset. These were the expected results predicted by the dimensional analysis presented in Section 2.

In terms of how this can be applied in a practical scenario, we see that, if we compute the feedback law provided in Figure 8a, we can obtain the feedback law provided in Figure 9a directly by scaling the original policy with Equation (36) using the appropriate context variables, without having to recompute. In some sense, Equation (36) provides us with the ability to adjust the feedback law spontaneously to conform with new system parameters $mgl$ or $\omega$, as would be the case with an analytical solution, even when working with black-box mapping, resulting in the form of a table look-up. However, the equivalence of the scaled solution is only guaranteed within a dimensionally similar context subset, which is the main limitation of this approach. The feedback law provided in Figure 8a cannot be scaled into the feedback law provided in Figure 14a, for instance, since ${\tau }_{max}^{\ast }$ and $q^{\ast }$ are not equals. It is also interesting to note that trajectory solutions from the same starting point—and computed cost-to-go functions (not illustrated)—are also all equivalent, up to scaling factors, within similar subgroups. Hence, optimal trajectories and cost-to-go solutions could also be shared and transferred between similar systems using the same technique that we demonstrate here for feedback laws.

3.1.4. Regimes of Solutions

In some situations, changing a context variable will not have any effect on the optimal policy. For instance, for a torque-limited optimal pendulum swing-up problem, augmenting ${\tau }_{max}$ or q while keeping the other value fixed will have little effect above a given threshold. For instance, if we look at the solutions for contexts $c_d$, $c_e$, and $c_f$, using a large amount of torque is so highly penalized by the cost function that the saturation limit does not have much impact on the solution (except for edge cases on the boundary). Thus, we would expect that augmenting ${\tau }_{max}$ would not change the solution. Figure 17 and Figure 18 show a slice (to allow for visualization) of the dimensionless optimal policy solution for various contexts. Figure 17 illustrates the results of changing ${\tau }_{max}^{\ast }$ while keeping $q^{\ast }$ fixed. We can see that, when ${\tau }_{max}^{\ast }<0.3$, the policy is almost always on the min–max allowable torque values; this behavior is often called bang–bang. At the other extreme, when ${\tau }_{max}^{\ast }>2.5$, the policy solution is continuous and almost never affected by the saturation. Figure 18 illustrates the results of changing $q^{\ast }$ while keeping ${\tau }_{max}^{\ast }$ fixed. We can see that, when $q^{\ast }<0.1$, the optimal policy solution does not reach min–max saturation, while, when $q^{\ast }>1.0$, the policy is almost always on the min–max allowable values.

We can see that, for extreme context values, two types of behavior occur, illustrated as regions in the dimensionless context space in Figure 19. Those regions are best characterized by a ratio of $q^{\ast }$ and ${\tau }_{max}^{\ast }$, a new dimensionless value that we define as the ratio of the maximum torque saturation ${\tau }_{max}$ over the weight parameter in the cost function q:

$$\begin{array}{c}\hfill R^{\ast }=\frac{{\tau }_{max}^{\ast }}{q^{\ast }}=\frac{{\tau }_{max}}{q}\end{array}$$

(54)

When the value of $R^{\ast }\approx 1$, the policy solution is partially continuous and reaches the min–max value in some other region of the state space; this is a behavior we call the transition regime. When the value of $R^{\ast }\ll 1$, the constraint on torque drives the solution to exhibit bang–bang behavior. In this region (that we approximate here, based on our sensitivity analysis, as $R^{\ast }\le 0.1$), the global policy is only a function of ${\tau }_{max}^{\ast }$:

$$\begin{array}{cc}\hfill {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q^{\ast },{\tau }_{max}^{\ast })& \approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },{\tau }_{max}^{\ast })\mathrm{if}{R}^{\ast }\ll 1\hfill \end{array}$$

(55)

i.e., the value of $q^{\ast }$ does not affect the solution. On the other hand, when the value of $R^{\ast }\gg 1$, the policy is unconstrained. In this region (that we approximate here, based on our sensitivity analysis, as $R^{\ast }\ge 10$), the global policy is only a function of $q^{\ast }$ since the constraint is so far away:

$$\begin{array}{cc}\hfill {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q^{\ast },{\tau }_{max}^{\ast })& \approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q^{\ast })\mathrm{if}{R}^{\ast }\gg 1\hfill \end{array}$$

(56)

The concept of regime is often leveraged in fluid mechanics. It allows us to generalize results between situations where the relevant dimensionless numbers do not match exactly. For instance, when the Mach number is small ($Ma<0.3$), we can generally assume an incompressible regime where various speeds of sound would not change the behavior much. Here, for the purpose of transferring policy solutions between contexts, this means that the condition of having the same exact dimensionless context variables can be relaxed with an inequality that corresponds to a regime. For instance, if we have two contexts in the unconstrained regime, it is sufficient to match only $q^{\ast }$ to create equivalent dimensionless policies.

Proposition 1. 

If it is assumed that Equation (56) holds, the condition of having equivalent dimensionless feedback laws is relaxed to an inequality for one of the context variables as follows:

$$\begin{array}{cc}\hfill & f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\approx f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\hfill \end{array}$$

(57)

$$\begin{array}{cc}& \mathit{if}{q}_a^{\ast }=q_b^{\ast }\mathit{and}{R}_a^{\ast }\gg 1\mathit{and}{R}_b^{\ast }\gg 1\hfill \end{array}$$

(58)

Proof. 

First, if $R_a^{\ast }\gg 1$ and $R_b^{\ast }\gg 1$, then from Equation (56) we can approximate the policy not to be a function of ${\tau }_{max}^{\ast }$:

$$\begin{array}{cc}\hfill f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })& ={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_a^{\ast },{\tau }_{max,a}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_a^{\ast })\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })& ={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_b^{\ast },{\tau }_{max,b}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_b^{\ast })\hfill \end{array}$$

(60)

Hence, if $q_a^{\ast }=q_b^{\ast }$, we have the following:

$$\begin{array}{c}\hfill f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_a^{\ast })={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_b^{\ast })\approx f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\end{array}$$

(61)

□

Also, for two contexts in a bang–bang regime, it is sufficient to match only ${\tau }_{max}^{\ast }$ to have equivalent dimensionless policies.

Proposition 2. 

If it is assumed that Equation (55) holds, the condition of having equivalent dimensionless feedback laws is relaxed to an inequality for one of the context variables as follows:

$$\begin{array}{cc}\hfill & f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\approx f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\hfill \end{array}$$

(62)

$$\begin{array}{cc}& \mathit{if}{\tau }_{max,a}^{\ast }={\tau }_{max,b}^{\ast }\mathit{and}{R}_a^{\ast }\ll 1\mathit{and}{R}_b^{\ast }\ll 1\hfill \end{array}$$

(63)

Proof. 

First, if $R_a^{\ast }\ll 1$ and $R_b^{\ast }\ll 1$, then, from Equation (55), we can approximate the policy not to be a function of $q^{\ast }$:

$$\begin{array}{cc}\hfill f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })& ={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_a^{\ast },{\tau }_{max,a}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },{\tau }_{max,a}^{\ast })\hfill \end{array}$$

(64)

$$\begin{array}{cc}\hfill f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })& ={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },q_b^{\ast },{\tau }_{max,b}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },{\tau }_{max,b}^{\ast })\hfill \end{array}$$

(65)

Hence, if ${\tau }_{max,a}^{\ast }={\tau }_{max,a}^{\ast }$, we have the following:

$$\begin{array}{c}\hfill f_a^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\approx {\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },{\tau }_{max,a}^{\ast })={\pi }^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast },{\tau }_{max,b}^{\ast })\approx f_b^{\ast }({\theta }^{\ast },{\dot{\theta }}^{\ast })\end{array}$$

(66)

□

From another point of view, assuming one of those regimes means that we could have removed one variable from the context at the start of the dimensional analysis. All in all, the impact of identifying such regimes is that we can increase the size of the context subset to which the dimensionless version of the policy should be equivalent, leading to a potentially larger pool of systems that can share a learned policy and numerical results.

3.1.5. Methodology

We obtained the optimal feedback law presented in this section using a basic dynamic programming algorithm [18] on a discretized version of the continuous system. The approach is almost equivalent to the value iteration algorithm [5], which is sometimes referred to as model-based reinforcement learning, with the exception that, here, the total number of iteration steps was fixed (corresponding to a very long time horizon approximating an infinite horizon) instead of the iteration being stopped after reaching a convergence criterion. This approach was chosen to enable the collection of consistent results across all contexts, leading to a wide range of order-of-magnitude cost-to-go solutions. The time step was set to 0.025 s, the state space was discretized into an even 501 × 501 grid, and the continuous torque input was discretized into 101 discrete control options. Special out-of-bounds and on-target termination states were included to guarantee convergence [18]. Also, using dynamic programming made the setting of additional parameters to define the domain necessary. Although those parameters should not affect the optimal policy far away from the boundaries, dimensionless versions of those parameters were kept fixed in all the experiments as follows:

$$\begin{array}{cc}\hfill {\theta }_{max}^{\ast }& ={\theta }_{max}=2\pi \hfill \end{array}$$

(67)

$$\begin{array}{cc}\hfill {\dot{\theta }}_{max}^{\ast }& =\frac{{\dot{\theta }}_{max}}{\omega }=\pi \hfill \end{array}$$

(68)

$$\begin{array}{cc}\hfill t_f^{\ast }& =t_f\omega =20\times 2\pi \hfill \end{array}$$

(69)

where ${\theta }_{max}$ is the range of angles for which the optimal policy is solved, set to one full revolution; ${\dot{\theta }}_{max}$ is the range of angular velocity for which the optimal policy is solved; and $t_f$ is the time horizon, set to 20 periods of the pendulum using the natural frequency. The source code is available online at the following link: https://github.com/alx87grd/DimensionlessPolicies (accessed on 25 February 2024), and this Google Colab page allows users to reproduce the results: https://colab.research.google.com/drive/1kf3apyHlf5t7XzJ3uVM8mgDsneVK_63r?usp=sharing (accessed on 25 February 2024).

3.2. Optimal Motion for a Longitudinal Car on a Slippery Surface

The second numerical example is a simplified car positioning task. We use this example to illustrate that an optimal feedback law in the form of a table look-up generated for a car of a given size can be transferred to a car of a different size if the motion control problem is dimensionally similar, as illustrated at Figure 20. The example includes state constraints and a different type of non-linearity (i.e., not similar to the pendulum swing-up) to illustrate how generic the developed dimensionless polices concepts are.

3.2.1. Motion Control Problem

The motion control problem is defined here as finding a feedback law to control the dynamic system, as described by the following differential equation:

$$\ddot{x}=\frac{\mu \left(s\right)g{x}_c}{l+\mu \left(s\right)y_c}\mathrm{with}\mu \left(s\right)=\frac{f_x}{f_n^f}=\frac{2}{1+e^{-70s}}-1$$

(70)

where $\mu \left(s\right)$ is the ratio of vertical to horizontal forces on the front wheel, that is, a non-linear function of the front wheel slip s. The above equations represent a simple dynamic model of the longitudinal motion of a car, assuming that the controller can impose the wheel slip of the front wheel and that suspensions are infinitely rigid (but that weight transfer is included). Interestingly, it is already standard practice to model the ground–tire interaction with an empirical curve $\mu \left(s\right)$ relating two dimensionless variables.

The objective is to minimize the infinite horizon quadratic cost function provided by the following:

$$J={\int }_0^{\infty }\left(q^{-2}{x}^2+s^2\right)dt$$

(71)

subject to the constraints of keeping ground reaction forces positive, as provided by the following:

$$\begin{array}{cc}\hfill 0\le & f_n^f=g{x}_c-\ddot{x}{y}_c\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill 0\le & f_n^r=g(l-x_c)+\ddot{x}{y}_c\hfill \end{array}$$

(73)

where the weight transfer potentially limits the allowable motions. Note that the cost function parameter q in this problem has a power of minus two to have a value with units of length, and all parameters are time-independent constants. The solution to this problem, i.e., the optimal policy for all contexts, involves the variables listed in

Table 4. Longitudinal car optimal policy variables.

Variable	Description	Units	Dimensions
Control inputs
s	Wheel slip	-	[]
State variables
x	Car position	m	[L]
$\dot{x}$	Car velocity	m/s	[$L{T}^{-1}$]
System parameters
g	Gravity	m/s2	[$L{T}^{-2}$]
l	Length (wheel base)	m	[L]
$x_c$	center of gravity (CG) horizontal position	m	[L]
$y_c$	center of gravity (CG) vertical position	m	[L]
Task parameters
q	Weight parameter	m	[L]

and should be of the form provided by

$$\underset{\mathrm{input}}{\underbrace{s}}=\pi \left(\underset{\mathrm{states}}{\underbrace{x,\dot{x}}},\underset{\mathrm{Context}c}{\underbrace{x_c,y_c,g,l,q}}\right)$$

(74)

3.2.2. Dimensional Analysis

Here, we have one control input, two states, and five context parameters, for a total of $1+(n=2)+(m=5)=8$ variables. Of those variables, only $d=2$ independent dimensions (length $\left[L\right]$ and time $\left[T\right]$) are present. Using $c_1=g$ and $c_2=l$ as the repeated variables leads to the following dimensionless groups:

$$\begin{array}{cc}\hfill {\Pi }_1& =s^{\ast }=s\left[\right]\hfill \end{array}$$

(75)

$$\begin{array}{cc}\hfill {\Pi }_2& =x^{\ast }=\frac{x}{l}\frac{\left[L\right]}{\left[L\right]}\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill {\Pi }_3& ={\dot{x}}^{\ast }=\frac{\dot{x}}{\sqrt{gl}}\frac{\left[L{T}^{-1}\right]}{{\left[L{T}^{-2}\right]}^{1/2}{\left[L\right]}^{1/2}}\hfill \end{array}$$

(77)

$$\begin{array}{cc}\hfill {\Pi }_4& =x_c^{\ast }=\frac{x_c}{l}\frac{\left[L\right]}{\left[L\right]}\hfill \end{array}$$

(78)

$$\begin{array}{cc}\hfill {\Pi }_5& =y_c^{\ast }=\frac{y_c}{l}\frac{\left[L\right]}{\left[L\right]}\hfill \end{array}$$

(79)

$$\begin{array}{cc}\hfill {\Pi }_6& =q^{\ast }=\frac{q}{l}\frac{\left[L\right]}{\left[L\right]}\hfill \end{array}$$

(80)

All three length variables are scaled by the wheel base, and the velocity variable is scaled using a combination of the wheel base and gravity. The transformation matrices are then as follows:

$$\begin{array}{cc}\hfill s\ast & =\underset{T_u}{\underbrace{\left[1\right]}}s\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{x}^{\ast }\\ {\dot{x}}^{\ast }\end{array}\right]& =\underset{T_x}{\underbrace{\left[\begin{array}{cc}\frac{1}{l}& 0\\ 0& \frac{1}{\sqrt{gl}}\end{array}\right]}}\left[\begin{array}{c}x\\ \dot{x}\end{array}\right]\hfill \end{array}$$

(82)

$$\begin{array}{cc}\hfill \underset{c^{\ast }}{\underbrace{\left[\begin{array}{c}{x}_c^{\ast }\\ y_c^{\ast }\\ q^{\ast }\end{array}\right]}}& =\underset{T_c}{\underbrace{\left[\begin{array}{ccccc}0& 0& 1/l& 0& 0\\ 0& 0& 0& 1/l& 0\\ 0& 0& 0& 0& 1/l\end{array}\right]}}\underset{c}{\underbrace{\left[\begin{array}{c}g\\ l\\ x_c\\ y_c\\ q\end{array}\right]}}\hfill \end{array}$$

(83)

By applying the Buckingham $\pi$ theorem [4], Equation (74) can be restated as a relationship between the six dimensionless $\Pi$ groups as follows:

$$s^{\ast }={\pi }^{\ast }\left(x^{\ast },{\dot{x}}^{\ast },x_c^{\ast },y_c^{\ast },q^{\ast }\right)$$

(84)

3.2.3. Numerical Results

Here, as in the pendulum example, numerical solutions to the motion control problem are computed for the nine instances of context variables listed in

Table 5. Car problem parameters.

	l	g	${\mathit{x}}_{\mathit{c}}$	${\mathit{y}}_{\mathit{c}}$	q
Problems with $x_c^{\ast }=0.5$, $y_c^{\ast }=0.5$, and $q^{\ast }=20$
Context $c_a$:	2.0	9.8	1.0	1.0	40
Context $c_b$:	1.0	9.8	0.5	0.5	20
Context $c_c$:	3.0	9.8	1.5	1.5	60
Problems with $x_c^{\ast }=0.5$, $y_c^{\ast }=1.5$, and $q^{\ast }=10$
Context $c_d$:	2.0	9.8	1.0	3.0	20
Context $c_e$:	1.0	9.8	0.5	1.5	10
Context $c_f$:	3.0	9.8	1.5	4.5	30
Problems with $x_c^{\ast }=0.5$, $y_c^{\ast }=0.1$, and $q^{\ast }=2$
Context $c_g$:	2.0	9.8	1.0	0.2	4
Context $c_h$:	1.0	9.8	0.5	0.1	2
Context $c_i$:	3.0	9.8	1.5	0.3	6

. In those nine contexts, there are three subsets of three dimensionally similar contexts. Contexts $c_a$, $c_b$, and $c_c$ describe situations where the CG horizontal position is at half the wheel base; contexts $c_d$, $c_e$, and $c_f$ describe situations in which the the CG is very high (and hence the cars are very limited by the weight transfer); and contexts $c_h$, $c_i$, and $c_j$ describe situations in which position errors are highly penalized by the cost function plus cars with a very low CG relative to the wheel base.

Figure 21, Figure 22, Figure 23, Figure 24, Figure 25, Figure 26, Figure 27, Figure 28 and Figure 29 illustrate that, for each subset with an equal dimensionless context, solutions are equal within each dimensionally similar subset when scaled into the dimensionless form. This was, again, the expected result predicted by the dimensional analysis presented in Section 2. In terms of how to use this in a practical scenario, this exemplifies how various cars (which are different but share the same ratios) could share a braking policy, for instance.

3.2.4. Methodology

The same methodology as the pendulum example (see Section 3.1.5) was used for the car motion control problem. The time step was set to 0.025 s, the state space was discretized into an even 501 × 501 grid, and the continuous slip input was discretized into 101 discrete control options. Additional domain parameters were set as follows:

$$\begin{array}{cc}\hfill x_{max}& =10l\hfill \end{array}$$

(85)

$$\begin{array}{cc}\hfill {\dot{x}}_{max}& =\frac{2}{\sqrt{gl}}\hfill \end{array}$$

(86)

$$\begin{array}{cc}\hfill t_{max}& =10\frac{x_{max}}{v_{max}}\hfill \end{array}$$

(87)

The source code is available online at the following link: https://github.com/alx87grd/DimensionlessPolicies (accessed on 25 February 2024), and this Google Colab page allows users to reproduce the results: https://colab.research.google.com/drive/1-CSiLKiNLqq9JC3EFLqjR1fRdICI7e7M?usp=share_link (accessed on 25 February 2024).

4. Case Studies with Closed-Form Parametric Policies

To better understand the concept of a dimensionless policy, in this section, two examples based on well-known closed-form solutions to classical motion control problems are presented to illustrate how using Theorem 2 can be equivalent to substituting new system parameters in an analytical solution.

4.1. Dimensionless Linear Quadratic Regulator

The first example is based on the linear quadratic regulator (LQR) solution [19] for the linearized pendulum that allows for a closed-form analytical solution of optimal policy. This allows us to compare the method of transferring the policy with the proposed scaling law of Equation (36) to the method of transferring the policy by substituting the new system parameters in the analytical solution.

Here, we consider a simplified version of the pendulum swing-up problem (see Section 3.1), and a linearized version of the equation of motion is used as follows:

$$m{l}^2\ddot{\theta }-mgl\theta =\tau$$

(88)

The same infinite horizon quadratic cost function is used, as follows:

$$J={\int }_0^{\infty }\left(q^2{\theta }^2+{\tau }^2\right)dt$$

(89)

However, no constraints on the torque are included in this problem. All parameters are also assumed to be time-independent constants. The same variables are used in this problem definition as before, except that the torque limit ${\tau }_{max}$ variable is absent. The global policy solution should then have the following form:

$$\underset{\mathrm{inputs}}{\underbrace{\tau }}={\pi }_{lqr}\left(\underset{\mathrm{states}}{\underbrace{\theta ,\dot{\theta }}},\underset{\mathrm{context}c}{\underbrace{\underset{\mathrm{system}\mathrm{parameters}}{\underbrace{m,g,l}},\underset{\mathrm{task}\mathrm{parameters}}{\underbrace{q}}}}\right)$$

(90)

We can thus select the same dimensionless $\Pi$ groups as in Section 3.1.2 and conclude that Equation (90) can be restated under the following dimensionless form:

$${\tau }^{\ast }={\pi }_{lqr}^{\ast }\left({\theta }^{\ast },{\dot{\theta }}^{\ast },q^{\ast }\right)$$

(91)

Proposition 3. 

For this motion control problem, defined by Equations (88) and (89), an analytical solution exists and the optimal policy is provided by

$$\begin{array}{cc}\hfill \tau & =\left[mgl+\sqrt{{\left(mgl\right)}^2+q^2}\right]\theta +\left[\sqrt{2m{l}^2(mgl+\sqrt{{\left(mgl\right)}^2+q^2}}\right)]\dot{\theta }\hfill \end{array}$$

(92)

Proof. 

See Appendix A. □

Applying Equation (31) to this feedback law leads to the dimensionless form, using $G=mgl$ and $H=m{l}^2$ for shortness, as follows:

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =f^{\ast }\left(x^{\ast }\right)=\left[T_u\left(c\right)\right]f\left(\left[T_x^{-1}\left(c\right)\right]x^{\ast }\right)=\frac{1}{G}f({\theta }^{\ast },\omega {\dot{\theta }}^{\ast })\hfill \end{array}$$

(93)

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =\left[\frac{1}{G}\right]\left[G+\sqrt{G^2+q^2}\right]{\theta }^{\ast }+\left[\frac{1}{G}\right]\left[\sqrt{2H\left(G+\sqrt{G^2+q^2}\right)}\right]\left[\omega {\dot{\theta }}^{\ast }\right]\hfill \end{array}$$

(94)

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =\left[1+\sqrt{\frac{G^2+q^2}{G^2}}\right]{\theta }^{\ast }+\left[\sqrt{\frac{2H{\omega }^2}{G}\frac{G+\sqrt{G^2+q^2}}{G}}\right]{\dot{\theta }}^{\ast }\hfill \end{array}$$

(95)

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =\left[1+\sqrt{1+{\left(q^{\ast }\right)}^2}\right]{\theta }^{\ast }+\left[\sqrt{2}\sqrt{1+\sqrt{1+{\left(q^{\ast }\right)}^2}}\right]{\dot{\theta }}^{\ast }\hfill \end{array}$$

(96)

The dimensionless policy is only a function of the dimensionless states and the dimensionless cost parameter $q^{\ast }$, as predicted by Equation (91) based on the dimensional analysis. It is interesting to note that Equation (96) represents the core generic solution to the LQR problem and is independent of unit and scale.

We can also use this analytical policy solution to demonstrate Theorem 2, i.e., show that scaling the policy with Equation (36) is equivalent to substituting new context variables when the contexts are dimensionally similar.

Proposition 4. 

Suppose that we have two context instances, labeled a and b, and that we use the global policy solution of Equation (92) to obtain two versions of context-specific feedback laws:

$$\begin{array}{cc}\hfill f_a(\theta ,\dot{\theta })& =\left[G_a+\sqrt{G_a^2+q_a^2}\right]\theta +\left[\sqrt{2H_a(G_a+\sqrt{G_a^2+q_a^2})}\right]\dot{\theta }\hfill \end{array}$$

(97)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[G_b+\sqrt{G_b^2+q_b^2}\right]\theta +\left[\sqrt{2H_b(G_b+\sqrt{G_b^2+q_b^2})}\right]\dot{\theta }\hfill \end{array}$$

(98)

where

$$\begin{array}{c}\hfill G_a=m_a{g}_a{l}_a{H}_a=m_a{l}_a^2\end{array}$$

(99)

$$\begin{array}{c}\hfill G_b=m_b{g}_b{l}_b{H}_b=m_b{l}_b^2\end{array}$$

(100)

Based on Theorem 2, if $q_a^{\ast }=q_b^{\ast }$, we can obtain $f_b$ directly by scaling $f_a$ based on Equation (36) as follows:

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[\frac{G_b}{G_a}\right]f_a\left(\theta ,\left[\frac{{\omega }_a}{{\omega }_b}\right]\dot{\theta }\right)\hfill \end{array}$$

(101)

where

$$\begin{array}{c}\hfill {\omega }_a=\sqrt{G_a/H_a}{\omega }_b=\sqrt{G_b/H_b}\end{array}$$

(102)

Proof. 

If we substitute $f_a$ in Equation (101) by the analytical solution provided by Equation (97) and then distribute the multiplying scaling factors, we obtain

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[\frac{G_b}{G_a}\right]\left(\left[G_a+\sqrt{G_a^2+q_a^2}\right]\theta +\left[\sqrt{2H_a(G_a+\sqrt{G_a^2+q_a^2})}\right]\left[\frac{{\omega }_a}{{\omega }_b}\right]\dot{\theta }\right)\hfill \end{array}$$

(103)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =G_b\left[1+\sqrt{1+{\left(q_a^{\ast }\right)}^2}\right]\theta +G_b\left[\sqrt{2}\sqrt{1+\sqrt{1+{\left(q_a^{\ast }\right)}^2}}\right]\frac{\dot{\theta }}{{\omega }_b}\hfill \end{array}$$

(104)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[G_b+\sqrt{G_b^2+{\left(G_b{q}_a^{\ast }\right)}^2}\right]\theta +\left[\sqrt{2H_b}\sqrt{G_b+\sqrt{G_b^2+{\left(G_b{q}_a^{\ast }\right)}^2}}\right]\dot{\theta }\hfill \end{array}$$

(105)

which is equivalent to Equation (98) when

$$\begin{array}{c}\hfill G_b{q}_a^{\ast }=q_b\mathrm{or}\mathrm{equivalently}{q}_a^{\ast }=q_b^{\ast }\end{array}$$

(106)

which is the condition of having equal dimensionless contexts ($c_a^{\ast }=c_b^{\ast }$) for this motion control problem. □

This example illustrates that applying the scaling of Equation (36) based on the dimensional analysis framework is equivalent to changing the context variables in an analytical solution when the dimensionless context variables are equal.

4.2. Dimensionless Computed Torque

The second example is again based on the pendulum but using the computed torque control technique [20]. This also allows us to compare the method of transferring the policy with the proposed scaling law of Equation (36) to the method of transferring the policy by substituting the new system parameters in the analytical solution. This example is not based on a quadratic cost function, as opposed to previous examples, to illustrate the flexibility of the proposed schemes.

Here, we present a second analytical example. A computed torque feedback law is a model-based policy (assuming that there are no torque limits) that is the solution to the motion control problem of making a mechanical system that converges on a desired trajectory, with a specified second-order exponential time profile defined by the following equation:

$$0=({\ddot{\theta }}_d-\ddot{\theta })+2{\omega }_d\zeta ({\dot{\theta }}_d-\dot{\theta })+{\omega }_d^2(\theta -\theta )$$

(107)

For the specific case of the pendulum swing-up problem, we assume that all parameters are time-independent constants and that our desired trajectory is simply the upright position (${\ddot{\theta }}_d={\dot{\theta }}_d={\theta }_d=0$), leaving only two parameters to define the tasks: ${\omega }_d$ and $\zeta$. Then, the computed torque policy takes the following form:

$$\begin{array}{cc}\hfill \underset{\mathrm{input}}{\underbrace{\tau }}& ={\pi }_{ct}\left(\underset{\mathrm{states}}{\underbrace{\theta ,\dot{\theta }}},\underset{\mathrm{context}c}{\underbrace{\underset{\mathrm{system}\mathrm{parameters}}{\underbrace{m,g,l}},\underset{\mathrm{task}\mathrm{parameters}}{\underbrace{{\omega }_d,\zeta }}}}\right)\hfill \end{array}$$

(108)

and the analytical solution is as follows:

$$\begin{array}{cc}\hfill \tau & =mglsin\theta -2m{l}^2{\omega }_d\zeta \dot{\theta }-m{l}^2{\omega }_d^2\theta \hfill \end{array}$$

(109)

Here, the context includes the system parameters and two variables characterizing the convergence speed. Note that the task parameters directly define the desired behavior, as opposed to the previous examples where they were defining the behavior indirectly through a cost function. The states, control inputs, and system parameters are the same as before; only the task parameters differ, and their dimensions are presented in

Table 6. Computed torque task variables.

Variable	Description	Units	Dimensions
Task parameters
${\omega }_d$	Desired closed-loop frequency	s−1	[$T^{-1}$]
$\zeta$	Desired closed-loop damping	-	[-]

.

Here, seven variables and only $p=2$ independent dimensions ($M{L}^2{T}^{-2}$ and $T^{-1}$) are involved. Thus, five dimensionless groups can be formed, as follows:

$$1+(n=2)+(m=4)-(p=2)=5$$

(110)

Using $mgl$ and $\omega$, the system parameters, as the repeating variables leads to the following dimensionless groups:

$$\begin{array}{cc}\hfill {\Pi }_1& ={\tau }^{\ast }=\frac{\tau }{mgl}\frac{\left[M{L}^2{T}^{-2}\right]}{\left[M\right]\left[L{T}^{-2}\right]\left[L\right]}\hfill \end{array}$$

(111)

$$\begin{array}{cc}\hfill {\Pi }_2& ={\theta }^{\ast }=\theta [-]\hfill \end{array}$$

(112)

$$\begin{array}{cc}\hfill {\Pi }_3& ={\dot{\theta }}^{\ast }=\frac{\dot{\theta }}{\omega }\frac{\left[T^{-1}\right]}{\left[T^{-1}\right]}\hfill \end{array}$$

(113)

$$\begin{array}{cc}\hfill {\Pi }_4& ={\omega }_d^{\ast }=\frac{{\omega }_d}{\omega }\frac{\left[T^{-1}\right]}{\left[T^{-1}\right]}\hfill \end{array}$$

(114)

$$\begin{array}{cc}\hfill {\Pi }_5& ={\zeta }^{\ast }=\zeta [-]\hfill \end{array}$$

(115)

Then, applying the Buckingham $\pi$ theorem tells us that the computed torque policy can be restated as the following relationship between the dimensionless variables:

$${\tau }^{\ast }={\pi }_{ct}^{\ast }\left(\theta ,{\dot{\theta }}^{\ast },{\omega }_d^{\ast },{\zeta }^{\ast }\right)$$

(116)

Here, we can confirm directly (since we have an analytical solution) that applying Equation (31) to the computed torque feedback law provided by Equation (109) leads to the following dimensionless form:

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =\left[\frac{1}{mgl}\right]\left(mglsin\theta -2m{l}^2{\omega }_d\zeta \left(\omega {\dot{\theta }}^{\ast }\right)-m{l}^2{\omega }_d^2\theta \right)\hfill \end{array}$$

(117)

$$\begin{array}{cc}\hfill {\tau }^{\ast }& =sin{\theta }^{\ast }-2{\omega }_d^{\ast }{\zeta }^{\ast }{\dot{\theta }}^{\ast }-{\left({\omega }_d^{\ast }\right)}^2{\theta }^{\ast }\hfill \end{array}$$

(118)

thereby confirming the structure predicted by Equation (116) based on the dimensional analysis.

We can, again, use this example to demonstrate Theorem 2 and show that, when the dimensionless context is equal, scaling a policy using Equation (36) is equivalent to substituting new values of the system parameters into the analytical equation.

Proposition 5. 

Suppose that we have two context instances, labeled a and b, and that we use the global policy solution of Equation (109) to obtain two versions of context-specific feedback laws:

$$\begin{array}{cc}\hfill f_a(\theta ,\dot{\theta })& =G_{a}sin\theta -2H_a{\omega }_{d,a}{\zeta }_a\dot{\theta }-H_a{\omega }_{d,a}^2\theta \hfill \end{array}$$

(119)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =G_{b}sin\theta -2H_b{\omega }_{d,a}{\zeta }_b\dot{\theta }-H_b{\omega }_{d,a}^2\theta \hfill \end{array}$$

(120)

Based on Theorem 2, if ${\omega }_{d,a}^{\ast }={\omega }_{d,b}^{\ast }$ and ${\zeta }_a^{\ast }={\zeta }_b^{\ast }$, we can obtain $f_b$ directly by scaling $f_a$ based on Equation (36) as follows:

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[\frac{G_b}{G_a}\right]f_a\left(\theta ,\left[\frac{{\omega }_a}{{\omega }_b}\right]\dot{\theta }\right)\hfill \end{array}$$

(121)

Proof. 

If we substitute $f_a$ in Equation (121) by the analytical solution provided by Equation (119) and then distribute the multiplying scaling factors, we obtain

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =\left[\frac{G_b}{G_a}\right][G_{a}sin\theta -2H_a{\omega }_{d,a}{\zeta }_a\left[\frac{{\omega }_a}{{\omega }_b}\right]\dot{\theta }-H_a{\omega }_{d,a}^2\theta ]\hfill \end{array}$$

(122)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =G_b\left[sin\theta -2\frac{{\omega }_{d,a}}{{\omega }_a}{\zeta }_a\frac{\dot{\theta }}{{\omega }_b}-{\left(\frac{{\omega }_{d,a}}{{\omega }_a}\right)}^2\theta \right]\hfill \end{array}$$

(123)

$$\begin{array}{cc}\hfill f_b(\theta ,\dot{\theta })& =G_{b}sin\theta -2H_b\left(\frac{{\omega }_b}{{\omega }_a}{\omega }_{d,a}\right){\zeta }_a\dot{\theta }-H_b{\left(\frac{{\omega }_b}{{\omega }_a}{\omega }_{d,a}\right)}^2\theta \hfill \end{array}$$

(124)

which is exactly equivalent to Equation (120) (i.e., equivalent to substituting the a instance of the context variables to the b instance) if

$$\begin{array}{c}\hfill \frac{{\omega }_b}{{\omega }_a}{\omega }_{d,a}={\omega }_{d,b}\mathrm{and}{\zeta }_a={\zeta }_b\end{array}$$

(125)

which is the dimensional similarity condition ($c_a^{\ast }=c_b^{\ast }$) for this motion control problem:

$$\begin{array}{c}\hfill \frac{{\omega }_{d,a}}{{\omega }_a}={\omega }_a^{\ast }={\omega }_b^{\ast }=\frac{{\omega }_{d,b}}{{\omega }_b}\mathrm{and}{\zeta }_a^{\ast }={\zeta }_b^{\ast }\end{array}$$

(126)

□

5. Conclusions

The dimensional analysis of physically meaningful control policies, leveraging the Buckingham $\pi$ theorem, leads to two interesting theoretical results. (1) In dimensionless form, the solution to a motion control problem involves a reduced number of parameters. (2) It is possible to exactly transfer a feedback law between similar systems without any approximation, simply by scaling the input and output of any type of control law appropriately, including via numerically generated black-box mapping. However, the main practical limitation of this approach is that, if the condition of dimensional similarity ($c_a^{\ast }=c_b^{\ast }$) is not met exactly, then there are no theoretical guarantees regarding whether a policy is transferable without additional assumptions, such as with the discussed concept of regimes of behavior. Also, we demonstrated how those results can be used to exactly transfer even discontinuous black-box policies between similar systems using two simple examples of dynamical systems and numerically generated optimal feedback laws. An interesting direction for further exploration would be investigating how good an approximation is when a feedback law is transferred from a context that is not exactly the same but close. Also, it would be interesting to test the concept of dimensionless policies to empower a reinforcement learning scheme that could collect data from various, but dimensionally similar, systems to accelerate the learning process.