In this paper, we present the mathematical point of view of our research group regarding the multi-robot systems evolving in a multi-temporal way. We solve the minimum multi-time volume problem as optimal control problem for a group of planar micro-robots moving in the same direction at different partial speeds. We are motivated to solve this problem because a similar minimum-time optimal control problem is now in vogue for micro-scale and nano-scale robotic systems. Applying the (weak and strong) multi-time maximum principle, we obtain necessary conditions for optimality and that are used to guess a candidate control policy. The complexity of finding this policy for arbitrary initial conditions is dominated by the computation of a planar convex hull. We pointed this idea by applying the technique of multi-time Hamilton-Jacobi-Bellman PDE. Our results can be extended to consider obstacle avoidance by explicit parameterization of all possible optimal control policies.
Our multi-time model extends the single-time case formulated and solved by T. Bretl [1,2] (see also, [3,4,5,6]). We refer to a microrobotic system consisting of n planar robots which evolve in multi-temporal sense. The control of this system is hard, at least from an algorithmic point of view. We solve the problem via a multi-time maximum optimal control problem and via the technique of multi-time Hamilton-Jacobi-Bellman PDE (see, [7,8,9,10,11,12,13]). The problem of multi-temporal evolution has many pitfalls due to the correlation between the dimension of state variables and that of evolution variables.
The microrobotic systems are intended for a wide range of applications that include microfabrication, minimally invasive medical diagnosis and treatment, adaptive optics, regenerative electronics, and biosensing for environmental monitoring and toxin detection .
The term “multi-time” was used for the first time by Dirac (1932)  to introduce “multi-time wave function” as candidate for relativistic many-particle quantum mechanics.
Section 2 formulates a multi-time optimal control problem for a system of many robots that move at different partial speeds, but that must all move in the same partial direction. Section 3 shows how we can solve the problem via the weak multi-time maximum principle. Here, the solution of the adjoint PDEs system is obtained by geometrical techniques. As it is too complicated to continue with this method, Section 4 solves the problem by the strong multi-time maximum principle. Section 5 gives a geometrical solution of our problem. Section 6 proves that the multi-time dynamic programming method permits the design of multi-time optimal controls for the problem in Section 3. Section 7 refers to originality of the subject and to the possibility of further research.
We consistently use mathematical language from multi-temporal dynamical systems and differential geometry. Particularly the Einstein convention of summation, and a short dictionary for notations in differential geometry (∧ = exterior product or wedge product of two differential forms, = Kronecker symbols, ⌟ = interior product or inner derivative) are used throughout. The tensor fields are written also via their components etc.
2. Many Robots That Move at Different Partial Speeds
The evolutive multivariate parameter is called multi-time. A multi-temporal evolution is conceived as follows: It is considered a generic hyper-parallelepiped determined by the diagonal opposite points . An evolution in is determined by the partial order in and by a positive sense of movement. A curve , joining the points and , is called marker of evolution in if (increasing curve). The simplest marker of evolution is the main diagonal that joins the points 0 and T.
Now let us consider a function . The evolution in means that the image of the function runs from the point to the point . The graph can be more suggestive, being a hypersurface in , running from the point to the point . The normal vector field to this hypersurface is . The marker of evolution in induces a marker of evolution in the image , if (acute angle), and more suggestive, a marker of evolution on the hypersurface .
To study the multi-temporal evolution of micro-scale and nano-scale robotic systems we must create a controlled m-flow evolution, an elapsed volume functional and a minimum type problem. We underline that the initial positions of the robots are given, and the goal is to bring them to the origin, minimizing the elapsed time volume. The solution of a controlled completely integrable system takes the place of evolutionary function .
If we leave the multi-time T free, then for n planar robots the following problem of multi-time optimal control appears: Let be the state variables (one pair means one robot) and , be the controls (inputs). The main goal is to find
The previous controlled PDEs can be written
If , then the vector fields , , , are linearly independent.
For each robot , it appears the square of speed
Consequently, the group of n robots move in a planar workspace at different (although bounded) speeds, but that must all move in the same partial direction fixed by the unit vector . In fact, the speeds and the direction are the only physically observable measures.
The complete integrability conditions of this PDE system are
It follows the piecewise general solution
(i) The quadruple constitutes an admissible m-mapping if it has the following properties: (1) is a measurable function from to ; (2) for , , (path independent curvilinear integrals), (3) (compact set). Please note that the second property implies that is differentiable almost everywhere as a function of multi-time t, satisfying the previous PDE system for almost all .
(ii) If the previous PDE system is not completely integrable, we can formulate and solve a similar problem using the nonholonomic evolution .
Because of periodicity of sine and cosine, we can take . Also, we can restrict without loss of generality.
We solve the foregoing problem using the multi-time maximum principle (see, [7,8,9,10,11,12,13]). We introduce the Lagrange multipliers , the Hamiltonian
and its anti-trace
called the control Hamiltonian tensor field.
3. Solution via Weak Multi-Time Maximum Principle
According the weak multi-time maximum principle  (coming from variational calculus techniques), along any optimal sheet
we must have
Due to the fact that this Hamiltonian is a linear function with respect to v, its extremum point cannot be interior. Moreover, we have
Solving the Adjoint PDEs System
Since this Hamiltonian has no dependence on the state vector variables , the adjoint PDEs are of divergence form
To find the general solution of this adjoint divergence PDEs system, we recall some facts from differential geometry  about closed and exact forms.
An r-form is called closed if . We say that is exact if there exists an -form such that .
To characterize situations in which closed forms are also exact, we call a famous.
(The Poincaré Lemma). Let U be a contractible domain in . If ω is a closed r-form, then there exists an -form η such that . In other words, all closed differential r-forms on contractible domains are exact.
In particular, if is a closed r-form on , then it is exact.
The m-form (volume form) and the vector fields produce (see the inner derivative) the -forms and the -forms . These satisfy
Now, the Lagrange multipliers are the m-forms
As solutions of the adjoint PDEs, they are closed, i.e.,
According the Poincaré Lemma, there exist two -forms
It follows that the solution of the adjoint system is
On the other hand, the strong multi-time maximum principle actually shows that the particular solution , is sufficient to obtain the complete solution of our problem.
4. Solution via Strong Multi-Time Maximum Principle
According the strong multi-time maximum principle  (coming from m-needle techniques), along any optimal sheet
we must have
Also, the function is constant.
Since is a linear function with respect to v, the Hamiltonian has no interior extremum point. Also, we have
4.1. Solving the Adjoint PDEs System
Since the control Hamiltonian tensor field has no dependence on the state , the adjoint PDEs reduce to
with the piecewise constant solution
4.2. Finding the Maximum with Respect to v
To prove the existence of a bang-bang control v, we use the following steps.
The maximum of the Hamiltonian with respect to the control v is
The inputs belong to the control set . The maximum of the linear function exists since each control variable belongs to the interval ; for maximum, the control must be at a vertex of (see, linear optimization, simplex method). If are the switching functions, then each optimal control must be the function
If , then , and hence is undetermined. Otherwise, the function vanishes only for one value of . Then, the singular control is ruled out and the remaining possibilities are bang-bang controls. This optimal control is discontinuous since each component jumps from a minimum to a maximum and vice versa, in response to each change in the sign of each switching function. The form of the optimal Hamiltonian follows. □
4.3. Finding the Maximum with Respect to u
Although we follow the path of finding the maximum with respect to v and then those with respect to u, it is useful to keep in mind the reverse procedure. This facilitates the understanding of some formulas in the following text.
According Formula (13), along any multi-time optimal sheet , the Hamiltonian is a function only of the heading angle u. As continuous function it has a maximum on the compact interval . Since , the same maximum value is on the interval . We shall show that at least one and at most values of u maximize the Hamiltonian. We conclude that the input is piecewise constant and takes on at most values along any multi-time optimal sheet.
To simplify, we use the functions .
(i) The equality is true if and only if each term of the sum
(ii) A zero of one of the functions , with , is not a maximum point of .
Let , with , for example, . Then the function ,
has the derivative
If is a maximum point, then we should have and , i.e.,
On the other hand, or , whence which is a contradiction. □
If in an open interval I, then the first two derivatives of the function are
Each function , which is not identically zero, has exactly one zero in the interval . Totally, we have a set A consisting of at most zeros in .
On an interval fixed by two consecutive zeros in A, the Hamiltonian (13) has the properties: (i) it is a function, (ii) it is a concave function, (iii) the derivative has at most one zero.
(i) The function is a sum of absolute values of smooth functions and consequently it is piecewise smooth. (ii) Since
(iii) It is almost obvious. □
The maximum of the Hamiltonian with respect to the control u, for an optimal value , is
The maximum of the Hamiltonian with respect to the control v is given in Lemma 1. On the other hand, the maximum of the function
with respect to u, is
It follows the maximum of the Hamiltonian. □
For any t, the maximum value is .
Suppose w is a maximum value function and
is the generating vector field. The multi-time Hamilton-Jacobi-Bellman PDE (feedback law) is 
On the other hand, the evolution PDEs and the Lagrangian do not depend on the variable t. Then, the generating vector field is independent on t. The multi-time Hamilton-Jacobi-Bellman PDE becomes
Consequently, the statement is true. □
Guess Solution for Maximum with Respect to u
In our target problem, the adjoint variables (co-states ) have no conditions on the boundary and so they may not be initially specified. However, giving extrema points u, we can calculate the optimal co-states . Indeed, for , and the sequence
we can define
where . The function (13) becomes
This expression demonstrates the following properties:
(iii) The points , , are the only maximum points for the function . Consequently, .
4.4. Finding the Optimal Evolution
The optimal control has a piecewise form
In this way, we have transformed the foregoing problem from an infinite-dimensional one, in which we are required to specify the functions and , for , into a finite-dimensional problem, in which we are required only to specify a double sequence of values of v. Then the optimal evolution is a piecewise solution to the Pfaff equations
The general solution is
These formulas generate a piecewise general solution, splitting the domain into sub-domains depending on the optimal values . As example, for a single optimal and the boundary conditions
we obtain the optimal evolution
5. Geometrical Solution
Suppose the set of robots determine, in this order, a convex polygon in . We select the point . Applying the Bretl theory , the point can attend the origin in n steps defined by
with the convention The spend time for each step is , .
For the point , the connection between our point of view and the theory of Bretl  is
If and , we must solve the first problem:
To solve this problem, we use the Lagrange function
From the equations of critical points, we find
Since , it follows
and hence (square). On the other hand, the product depends on t. According Bretl, for , we have
Running the point , the relations (19) are changed into
Fixing the index i, we obtain a system of n linear equations with two unknowns . If the rank of the system is two, one obtains the uni-temporal case of Bretl, i.e., either or . For significant two-time case, the rank must be one, and we can take the repartition , . It follows (square)
6. Multi-Time Hamilton-Jacobi-Bellman PDE
To solve the problem formulated in Section 2, let us use the idea that the multi-time dynamic programming method permits the design of multi-time optimal controls.
To simplify, let us accept . Also, to use the multi-time maximum principle, we replace the initial multiple integral functional by
(equivalent minimum area). Let us consider the set , where . Since
we transform the maximum problem in Section 1 into similar problems: find
were is selected to have a minimum norm.
For m-volume multi-time optimal problems, the maximum value function w does not depend on the multi-time t.
6.1. One Optimal Value of the Control u
6.1.1. Case ,
Omitting the index “star”, the constraints (boundary value problem) rewrite in the form
Generally, and the relation connects linearly the differences and . We need to find
we find the critical point condition
Because , it follows
Let us correlate this result with the Hamilton-Jacobi-Bellman PDE. Since does not depend on t, the generating vector does not depend on t.
Suppose w is a maximum value function and
is the generating vector field. The two-time Hamilton-Jacobi-Bellman PDE (feedback law) is 
The maximum with respect v is obtained for
It follows the PDE
the value is 1 for . Consequently,
is a generating vector field.
In this case, using the total derivative operator D, we have
Omitting the index “star”, the constraints (boundary value problem) rewrite in the form
Since , , to find the maximum value , we need to solve the problem:
Case If , then we find
The two-time Hamilton-Jacobi-Bellman PDE (feedback law) is 
This PDE can be rewritten in the form
since each optimal control is
Using a single optimal control , the previous two-time Hamilton-Jacobi-Bellman PDE reduces to
We obtain an eikonal PDE
with the unknown functions . This PDE is equivalent to the system
Consequently, for , a solution of the Hamilton-Jacobi-Bellman PDE is obtained from
for . The solution obtained via the strong multi-time maximum principle is recovered by the conditions
In this case, the complexity of finding an optimal policy (for arbitrary initial conditions) is dominated by the computation of a planar convex hull.
Case . In this case, we need to solve the problem
The result is similar to those when , .
Consider the eikonal PDE
(ii) the function solves the eikonal PDE in the viscosity sense.
6.2. Two Optimal Values of the Control u
Let us consider the partitions , , and the rectangles . We order the optimal values in an increasing sequence and we set for the multi-time set . For finding the optimal evolution it is enough to consider the diagonal rectangles . The points are determined by and are connected to .
To simplify, in , let us consider two diagonal rectangles
the first corresponding to the optimal value , and the second to . Denoting
the optimal evolution splits as:
We need to solve the problem of finding the maximum cost on , then on and to add them.
Maximum on . Using the Lagrangian function
on , where
Maximum on . Since the constraints have the form the result is similar to those in “Case: One optimal value of the control”. Hence
on . It follows
6.3. Viscosity Solution
The Hamilton-Jacobi-Bellman PDE has smooth solution on , respectively on . Since at the point we have a discontinuity of the partial derivatives, we must refer to the PDE system (18) and to its viscosity solutions. The basic idea is to replace the differentials at a point where it does not exist (for example because of a kink in ) with the differentials of a smooth function touching the graph of , from above for the subsolution condition and from below for the supersolution one, at the point .
(i) A continuous function is said to be a viscosity subsolution of the PDE system (18) if, for any point and for any smooth function such that each function , has a maximum point at , we have
(ii) A continuous function is said to be a viscosity supersolution of (16) if, for any point and for any smooth function ψ such that each function , has a minimum point at , we have
(iii) A continuous function is said to be a viscosity solution of the PDE system (16) if it is a viscosity subsolution and supersolution.
The viscosity solution of the PDE system is where Q is a quarter of the perimeter of the parallelogram i.e.,
Our work is the first which introduces and studies the theory of minirobots moving at different partial speeds (in a multi-temporal sense), but that must all move in the same partial direction. We are motivated to solve this problem because constraints of previous sort must be common in micro-scale and nano-scale robotic systems appearing in applied fields mentioned above. To understand a multi-temporal evolution we must think the dependence on multi-time either as an immersion, or as a diffeomorphism, or as a submersion, and that the partial order in induces a partial order on the image of such a function.
The phenomenon described by us takes place in spaces with at least four dimensions. That is why graphic representations lose their meaning.
By application of the (weak and strong) multi-time maximum principle, we obtain necessary conditions for optimality and use them to guess a candidate control policy. By the multi-time Hamilton-Jacobi-Bellman PDE, we verify that our guess is optimal. The complexity of finding this policy for arbitrary initial conditions is only quasilinear in the number of robots, and in fact is dominated by the computation of a planar convex hull.
In our minds the previous theory can be extended to the situation of three-dimensional robots, using the versor of unit sphere, which we will do in a future paper. We tested the theory of multi-time optimal control in relevant applications: multi-time control strategies for skilled movements , optimal control of electromagnetic energy , multi-time optimal control for quantum systems  etc.
The contributions of both authors are equal. The main results and illustrative examples were developed together. All authors have read and agreed to the published version of the manuscript.
This research received funding from Balkan Society of Geometers, Bucharest, Romania.
Thanks to referees for pertinent remarks.
Conflicts of Interest
The authors declare no conflict of interest.
Bretl, T. Minimum-time optimal control of many robots that move in the same direction at different speeds. IEEE Trans. Robbot.2012, 28, 351–363. [Google Scholar] [CrossRef]
DeVon, D.A.; Bretl, T. Control of many robots moving in the same direction with different speeds: A decoupling approach. In Proceedings of the 2009 American Control Conference, St. Louis, MO, USA, 10–12 June 2009. [Google Scholar]
Becker, A.; Onyuksel, C.; Bretl, T.; McLurkin, J. Controlling many differential-drive robots with uniform control inputs. Int. J. Robot. Res.2014, 33, 1626–1644. [Google Scholar] [CrossRef]
Bien, Z.; Lee, J. A minimum-time trajectory planning method for two robots. IEEE Trans. Robot. Autom.1992, 8, 414–418. [Google Scholar] [CrossRef]
Bloch, A.M.; Baillieul, J.; Crouch, P.E.; Marsden, J.E. Nonholonomic Mechanics and Control; Springer: Berlin/Heidelberg, Germany; New York, NY, USA, 2003. [Google Scholar]
Mauder, M. Time-Optimal Control of the Bi-Steerable Robot. Ph.D. Thesis, Fakultät für Mathematik und Informatik der Julius-Maximilians-Universität, Würzburg, Germany, 2012. [Google Scholar]
Udrişte, C. Multitime controllability, observability and bang-bang principle. J. Optim. Theory Appl.2008, 39, 141–157. [Google Scholar] [CrossRef]
Udrişte, C.; Ţevy, I. Multitime dynamic programming for curvilinear integral actions. J. Optim. Theory Appl.2010, 146, 189–207. [Google Scholar] [CrossRef]
Udrişte, C. Equivalence of multitime optimal control problems. Balk. J. Geom. Appl.2010, 15, 155–162. [Google Scholar]
Udrişte, C. Multitime optimal control for quantum systems. In Proceedings of the Third International Conference on Lie-Admissible Treatments of Irreversible Processes (ICLATIP-3), Kathmandu University, Dhulikhel, Nepal, 3–7 January 2011. [Google Scholar]
Udrişte, C.; Ţevy, I. Multitime dynamic programming for multiple integral actions. J. Glob. Optim.2011, 51, 345–360. [Google Scholar] [CrossRef]
Udrişte, C.; Bejenaru, A. Multitime optimal control with area integral costs on boundary. Balk. J. Geom. Appl.2011, 16, 138–154. [Google Scholar]
Iliuţă, M.; Udrişte, C.; Ţevy, I. Multitime control strategies for skilled movements. Balk. J. Geom. Appl.2013, 18, 31–46. [Google Scholar]