Minirobots Moving at Different Partial Speeds

: In this paper, we present the mathematical point of view of our research group regarding the multi-robot systems evolving in a multi-temporal way. We solve the minimum multi-time volume problem as optimal control problem for a group of planar micro-robots moving in the same direction at different partial speeds. We are motivated to solve this problem because a similar minimum-time optimal control problem is now in vogue for micro-scale and nano-scale robotic systems. Applying the (weak and strong) multi-time maximum principle, we obtain necessary conditions for optimality and that are used to guess a candidate control policy. The complexity of ﬁnding this policy for arbitrary initial conditions is dominated by the computation of a planar convex hull. We pointed this idea by applying the technique of multi-time Hamilton-Jacobi-Bellman PDE. Our results can be extended to consider obstacle avoidance by explicit parameterization of all possible optimal control policies.


Introduction
Our multi-time model extends the single-time case formulated and solved by T. Bretl [1,2] (see also, [3][4][5][6]). We refer to a microrobotic system consisting of n planar robots which evolve in multi-temporal sense. The control of this system is hard, at least from an algorithmic point of view. We solve the problem via a multi-time maximum optimal control problem and via the technique of multi-time Hamilton-Jacobi-Bellman PDE (see, [7][8][9][10][11][12][13]). The problem of multi-temporal evolution has many pitfalls due to the correlation between the dimension of state variables and that of evolution variables.
The microrobotic systems are intended for a wide range of applications that include microfabrication, minimally invasive medical diagnosis and treatment, adaptive optics, regenerative electronics, and biosensing for environmental monitoring and toxin detection [2].
The term "multi-time" was used for the first time by Dirac (1932) [14] to introduce "multi-time wave function" as candidate for relativistic many-particle quantum mechanics.
Section 2 formulates a multi-time optimal control problem for a system of many robots that move at different partial speeds, but that must all move in the same partial direction. Section 3 shows how we can solve the problem via the weak multi-time maximum principle. Here, the solution of the adjoint PDEs system is obtained by geometrical techniques. As it is too complicated to continue with this method, Section 4 solves the problem by the strong multi-time maximum principle. Section 5 gives a geometrical solution of our problem. Section 6 proves that the multi-time dynamic programming

Many Robots That Move at Different Partial Speeds
The evolutive multivariate parameter t = (t 1 , . . . , t m ) ∈ R m + is called multi-time. A multi-temporal evolution is conceived as follows: It is considered a generic hyper-parallelepiped Ω 0T ∈ R m + determined by the diagonal opposite points 0, T ∈ R m + . An evolution in Ω 0T is determined by the partial order in R m + and by a positive sense of movement. A C 1 curve γ : [0, 1] → Ω 0T , t α = t α (τ), τ ∈ [0, 1], joining the points γ(0) = 0 and γ(1) = T, is called marker of evolution in Ω 0T if dt α dτ ≥ 0 (increasing curve). The simplest marker of evolution is the main diagonal t α = T α τ, τ ∈ [0, 1] that joins the points 0 and T. Now let us consider a C 1 function ϕ : Ω 0T → R. The evolution in ϕ(Ω 0T ) means that the image of the function ϕ runs from the point ϕ(0) to the point ϕ(T). The graph (t, ϕ(t)) can be more suggestive, being a hypersurface in Ω 0T × R, running from the point (0, ϕ(0)) to the point (T, ϕ(T)). The normal vector field to this hypersurface is (∇ϕ, −1). The marker of evolution in Ω 0T induces a marker of evolution in the image ϕ(Ω 0T ), if ∇ϕ, dt dτ ≥ 0 (acute angle), and more suggestive, a marker of evolution on the hypersurface (t, ϕ(t)).
To study the multi-temporal evolution of micro-scale and nano-scale robotic systems we must create a controlled m-flow evolution, an elapsed volume functional and a minimum type problem. We underline that the initial positions of the robots are given, and the goal is to bring them to the origin, minimizing the elapsed time volume. The solution (x(t), y(t)) of a controlled completely integrable system takes the place of evolutionary function ϕ(t).
If we leave the multi-time T free, then for n planar robots the following problem of multi-time optimal control appears: Let (x, y) = (x 1 , y 1 , . . . , x n , y n ) ∈ (R 2 ) n be the state variables (one pair (x i , y i ) means one robot) and u ∈ R, v = (v i α ) ∈ R mn , α = 1, . . . , m; i = 1, . . . , n be the controls (inputs). The main goal is to find The previous controlled PDEs can be written If rank(v i α (t)) = m ≤ n, then the 2m vector fields X i For each robot (x i , y i ), it appears the square of speed Consequently, the group of n robots move in a planar workspace at different (although bounded) speeds, but that must all move in the same partial direction fixed by the unit vector (cos u(t), sin u(t)).
In fact, the speeds δ αβ v i α v i β , i = 1, . . . , n and the direction (cos u(t), sin u(t)) are the only physically observable measures.
The complete integrability conditions of this PDE system are It follows the piecewise general solution , v(t)) constitutes an admissible m-mapping if it has the following properties: . Please note that the second property implies that (x(t), y(t)) is differentiable almost everywhere as a function of multi-time t, satisfying the previous PDE system for almost all t ∈ Ω 0T .
(ii) If the previous PDE system is not completely integrable, we can formulate and solve a similar problem using the nonholonomic evolution dx Because of periodicity of sine and cosine, we can take u ∈ [−π, π]. Also, we can restrict u ∈ [0, π) without loss of generality.

Solution via Weak Multi-Time Maximum Principle
According the weak multi-time maximum principle [7] (coming from variational calculus techniques), along any optimal sheet (x * , y * , u * , v * , p * , q * ), we must have Due to the fact that this Hamiltonian is a linear function with respect to v, its extremum point cannot be interior. Moreover, we have

Solving the Adjoint PDEs System
Since this Hamiltonian has no dependence on the state vector variables (x, y), the adjoint PDEs are of divergence form ∂p * α i ∂t α = 0, ∂q * α i ∂t α = 0, i = 1 . . . , n. To find the general solution of this adjoint divergence PDEs system, we recall some facts from differential geometry [15] about closed and exact forms.
An r-form ω is called closed if dω = 0. We say that ω is exact if there exists an (r − 1)-form η such that dη = ω.
To characterize situations in which closed forms are also exact, we call a famous.
Theorem 1 (The Poincaré Lemma). Let U be a contractible domain in R n . If ω is a closed r-form, then there exists an (r − 1)-form η such that dη = ω. In other words, all closed differential r-forms on contractible domains are exact.
In particular, if ω is a closed r-form on R n , then it is exact. The m-form (volume form) ω = dt 1 ∧ · · · ∧ dt m and the vector fields ∂ ∂t α produce (see the inner derivative) the (m − 1)-forms ω α = ∂ ∂t α ω and the (m − 2)-forms ω βα = ∂ ∂t β ω α . These satisfy Now, the Lagrange multipliers p, q are the m-forms As solutions of the adjoint PDEs, they are closed, i.e., It follows that the solution of the adjoint system is On the other hand, the strong multi-time maximum principle actually shows that the particular solution p β i (t) = const, q β i (t) = const is sufficient to obtain the complete solution of our problem.

Solution via Strong Multi-Time Maximum Principle
According the strong multi-time maximum principle [11] (coming from m-needle techniques), along any optimal sheet (x * , y * , u * , v * , p * , q * ), we must have

Solving the Adjoint PDEs System
Since the control Hamiltonian tensor field has no dependence on the state (x, y), the adjoint PDEs reduce to

Finding the Maximum with Respect to v
To prove the existence of a bang-bang control v, we use the following steps.

Lemma 1.
The maximum of the Hamiltonian H(x, y, u, v, p, q) with respect to the control v is Proof. The inputs v i α belong to the control set V = [−1, 1] mn ⊂ R mn . The maximum of the linear function v → H exists since each control variable v i α belongs to the interval [−1, 1]; for maximum, the control must be at a vertex of ∂V (see, linear optimization, simplex method).
are the switching functions, then each optimal control v * i α must be the function , and hence v i α is undetermined. Otherwise, the function Q α i (t) vanishes only for one value of u(t). Then, the singular control is ruled out and the remaining possibilities are bang-bang controls. This optimal control is discontinuous since each component jumps from a minimum to a maximum and vice versa, in response to each change in the sign of each switching function. The form of the optimal Hamiltonian follows.

Finding the Maximum with Respect to u
Although we follow the path of finding the maximum with respect to v and then those with respect to u, it is useful to keep in mind the reverse procedure. This facilitates the understanding of some formulas in the following text.
According Formula (13), along any multi-time optimal sheet v * , the Hamiltonian is a function only of the heading angle u. As continuous function it has a maximum on the compact interval [0, π].
Since H(0) = H(π), the same maximum value is on the interval [0, π). We shall show that at least one and at most mn values of u maximize the Hamiltonian. We conclude that the input u(t) is piecewise constant and takes on at most mn values along any multi-time optimal sheet.
To simplify, we use the mn functions φ α If u 0 is a maximum point, then we should have h (u 0 −) > 0 and h (u 0 +) < 0, i.e., −p 1 1 sin u 0 + q 1 1 cos u 0 + A (u 0 ) > 0, Consequently On the other hand, p 1 1 cos u 0 + q 1 1 sin u 0 = 0 or ctan u 0 = − Each function φ α i (u), which is not identically zero, has exactly one zero in the interval [0, π). Totally, we have a set A consisting of at most mn zeros in [0, π). Lemma 4. On an interval fixed by two consecutive zeros in A, the Hamiltonian (13) has the properties: (i) it is a C ∞ function, (ii) it is a concave function, (iii) the derivative dH du has at most one zero.
Proof. (i) The function u → H(u) + 1 is a sum of absolute values of smooth functions and consequently it is piecewise smooth. (ii) Since Lemma 5. The maximum of the Hamiltonian H(x, y, u, v, p, q) with respect to the control u, for an optimal value v * , is Proof. The maximum of the Hamiltonian H(x, y, u, v, p, q) with respect to the control v is given in Lemma 1. On the other hand, the maximum of the function It follows the maximum of the Hamiltonian.
Proof. Suppose w is a maximum value function and is the generating vector field. The multi-time Hamilton-Jacobi-Bellman PDE (feedback law) is [11] ∂w α ∂t α + max u∈U;v∈V ∂w α ∂x cos u + ∂w α ∂y sin u v α − 1 = 0.
On the other hand, the evolution PDEs and the Lagrangian L = −1 do not depend on the variable t. Then, the generating vector field is independent on t. Consequently, the statement is true.

Guess Solution for Maximum with Respect to u
In our target problem, the adjoint variables (co-states p, q) have no conditions on the boundary and so they may not be initially specified. However, giving extrema points u, we can calculate the optimal co-states p, q. Indeed, for k ≤ n, and the sequence we can define where α = 1, . . . , m. The function (13) becomes This expression demonstrates the following properties: (i) sign (p i cos u j + q i sin u j ) = −1 for i ≤ j 1 for i > j.
(iii) The points u j , j = 1, . . . , k, are the only maximum points for the function H(u). Consequently, max u H(u) = 0.

Finding the Optimal Evolution
The optimal control has a piecewise form In this way, we have transformed the foregoing problem from an infinite-dimensional one, in which we are required to specify the functions u : [0, T] → [0, π) and v i α : [0, T] → [−1, 1], for α = 1, . . . , m; i = 1, . . . , n, into a finite-dimensional problem, in which we are required only to specify a double sequence of mn values of v. Then the optimal evolution is a piecewise solution to the Pfaff equations dx i (t) = v * i α dt α cos u * , dy i (t) = v * i α dt α sin u * . The general solution is These formulas generate a piecewise general solution, splitting the domain Ω 0T into sub-domains depending on the optimal values u * . As example, for a single optimal u * and the boundary conditions we obtain the optimal evolution

Geometrical Solution
Suppose the set of robots {z 1 , . . . , z n ; −z 1 , . . . , −z n } determine, in this order, a convex polygon in R 2 . We select the point z j . Applying the Bretl theory [1], the point z j can attend the origin in n steps P 1 , . . . , P n defined by with the convention z n+1 = −z 1 . The spend time for each step P i is t i = 1 2 ||z i+1 − z i ||, i = 1, . . . , n. For the point z j , the connection between our point of view and the theory of Bretl [1] is If T 1 = ∑ n i=1 t 1 i and T 2 = ∑ n j=1 t 2 j , we must solve the first problem: To solve this problem, we use the Lagrange function From the equations of critical points, we find Since |v kα | = 1, it follows and hence T 1 = T 2 (square). On the other hand, the product T 1 T 2 depends on t. According Bretl, for min (T 1 T 2 ), we have Running the point z j , the relations (19) are changed into Fixing the index i, we obtain a system of n linear equations with two unknowns t 1 i , t 2 i . If the rank of the system is two, one obtains the uni-temporal case of Bretl, i.e., either t 1 i = 0 or t 2 i = 0. For significant two-time case, the rank must be one, and we can take the repartition t 1

Multi-Time Hamilton-Jacobi-Bellman PDE
To solve the problem formulated in Section 2, let us use the idea that the multi-time dynamic programming method permits the design of multi-time optimal controls.

Remark 2.
For m-volume multi-time optimal problems, the maximum value function w does not depend on the multi-time t.
6.1. One Optimal Value of the Control u 6.1.1. Case α = 1, 2, i = 1 Omitting the index "star", the constraints (boundary value problem) rewrite in the form Generally, y x = tan u and the relation x = v α (t α − T α ) cos u connects linearly the differences t 1 − T 1 and t 2 − T 2 . We need to find we find the critical point condition Let us correlate this result with the Hamilton-Jacobi-Bellman PDE. Since w((x, y)) does not depend on t, the generating vector (w 1 ((x, y)), w 2 ((x, y))) does not depend on t.
Suppose w is a maximum value function and is the generating vector field. The two-time Hamilton-Jacobi-Bellman PDE (feedback law) is [11] ∂w α ∂t α + max u∈U;v∈V The maximum with respect v is obtained for v α = sign ∂w 1 ∂x cos u + ∂w 1 ∂y sin u .
It follows the PDE max u ∂w 1 ∂x cos u + ∂w 1 ∂y sin u + ∂w 2 ∂x cos u + ∂w 2 ∂y sin u − 1 = 0. Taking the value max u is 1 for tan u = y x . Consequently, is a generating vector field. In this case, using the total derivative operator D, we have For v 1 = v 2 = −1, one obtains a single PDE On the other hand, according [11], From the evolution (17), it follows The equality confirms the previous results.
Omitting the index "star", the constraints (boundary value problem) rewrite in the form Since y i = (tan u)x i , i = 1, 2, to find the maximum value max u(·),v(·) J, we need to solve the problem: The two-time Hamilton-Jacobi-Bellman PDE (feedback law) is [11] max u∈U;v∈V This PDE can be rewritten in the form Using a single optimal control u * (t) = const, the previous two-time Hamilton-Jacobi-Bellman PDE reduces to We obtain an eikonal PDE with the unknown functions w 1 (x, y), w 2 (x, y). This PDE is equivalent to the system Consequently, for ∑ 2 α,i=1 v * i α = 2, a solution (w 1 , w 2 ) of the Hamilton-Jacobi-Bellman PDE is obtained from w 1 ((x 1 , y 1 ), (x 2 , y 2 )) = for (x 1 , y 1 ) ∈ R 2 , (x 2 , y 2 ) ∈ R 2 . The solution obtained via the strong multi-time maximum principle is recovered by the conditions In this case, the complexity of finding an optimal policy (for arbitrary initial conditions) is dominated by the computation of a planar convex hull.
Case det v = 0. In this case, we need to solve the problem The result is similar to those when α = 1, 2, i = 1.
Show that: (ii) the function u(x) = dist(x; ∂Ω) solves the eikonal PDE in the viscosity sense.
By application of the (weak and strong) multi-time maximum principle, we obtain necessary conditions for optimality and use them to guess a candidate control policy. By the multi-time Hamilton-Jacobi-Bellman PDE, we verify that our guess is optimal. The complexity of finding this policy for arbitrary initial conditions is only quasilinear in the number of robots, and in fact is dominated by the computation of a planar convex hull.
In our minds the previous theory can be extended to the situation of three-dimensional robots, using the versor of unit sphere, which we will do in a future paper. We tested the theory of multi-time optimal control in relevant applications: multi-time control strategies for skilled movements [13], optimal control of electromagnetic energy [16], multi-time optimal control for quantum systems [10] etc.