Coordinated Vision-Based Tracking by Multiple Unmanned Vehicles

Abstract

We address the problem of coordinated vision-based tracking of a moving target using multiple unmanned vehicles that exchange information over a supporting time-varying network. The objective of this work is to formulate decentralized control algorithms that enable multiple vehicles to follow the target while coordinating their phase separation. A typical scenario involves multiple unmanned aerial vehicles that are required to monitor a moving ground object (target tracking) while maintaining a desired inter-vehicle separation (coordination). To solve the vision-based tracking problem, the yaw rate of each vehicle is used as the control input, while the speeds of the vehicles are adjusted to achieve coordination. It is assumed that the vehicles are equipped with an internal autopilot, which is able to track yaw rate and speed commands. The performance of the coordinated vision-based tracking algorithm is evaluated as a function of the target’s velocity, tracking performance of the onboard autopilot, and the quality of service of the communication network.

1. Introduction

Unmanned Vehicles (UxVs)—including air, ground, and marine robots—play a crucial role today in many complex missions, especially in hazardous situations, where the presence of human operators must be avoided. Numerous problems related to motion control of autonomous vehicles have been studied in recent years. One focus area of research delineated in the literature is the vision-based motion control UxVs. A variety of approaches based on visual feedback have been proposed in [1,2,3,4,5,6,7,8,9,10,11,12,13]. Typical applications considered in the literature include autonomous landing [10], aerial refueling [11], and vision-based target tracking (VBT) [7], where the objective is to control the motion of the UxVs relative to a static or moving target.

In spite of relevant theoretical and technological advances, the problem of vision-based motion control is still very challenging. In fact, the control system has to deal with poor, and insufficient, accuracy of the acquired data. Moreover, in the situation where only passive sensor data are available about the target, the control task must be defined using the information obtained in the camera frame only. Finally, the control engineer must take into account that processing image or video in general requires high computational resources, and that the information on the target’s and vehicle’s position and velocity might be unreliable or not available at all.

Another challenge to be addressed is the problem of vision-based motion control of multiple cooperative UxVs. Relevant work in the area of cooperative control of autonomous systems includes [14,15,16,17,18,19,20] (and the references therein). In these cases, one needs to establish communication between the vehicles. This compels the control designer to take into consideration the fact that the amount of information that can be exchanged could be limited and may vary over time. Cooperative motion control strategies must be developed that yield robust performance even in the presence of temporary loss of communication links and switching communication topologies.

Motivated by these challenges, we describe a solution to the Coordinated Vision-Based Tracking (CVBT) problem. The objective of this work is to enable a set of n vehicles, with dynamics described by the kinematic car model (i.e., Dubins’ car), to orbit around (track and monitor) a target while coordinating their phase separation with each other. Figure 1 depicts the control framework adopted in this paper. The work focuses on two main parts. First, we derive a control algorithm for the yaw rate to enable the vehicles involved in the mission to track a (static or moving) target and orbit around it (VBT). Second, we formulate control laws for the speed of each vehicle so that they maintain a predefined space separation while orbiting around the target (coordination).

At the VBT level, we show that VBT is achieved by using only vehicle-to-target relative line-of-sight (LOS) angles, which can be obtained from the visual measurement. This reduces the computational load required to acquire more complex information, such as the target’s position and velocity. The solution to the VBT problem presented in this paper builds on previous papers [21,22], where the authors demonstrate that the proposed control algorithm guarantees exponential convergence and ultimate boundedness of the VBT error in the case of static and moving targets. One fundamental assumption made in [21] is that the velocity of the unmanned vehicle is constant. In [22], such an assumption is relaxed and convergence of the VBT error to zero is guaranteed as long as the speed of the vehicle is lower and upper bounded by some constants. However, the work in [22] assumes that the UxV is equipped with an ideal inner-loop autopilot (AP), which guarantees that the difference between commanded and actual angular rates is always identical to zero. In the present paper, we address a more realistic case, where the AP can only ensure that such a difference is bounded. It is shown that the VBT algorithm is robust to non-perfect tracking of the VBT input commands. Performance of the outer-loop VBT controller is derived as a function of the performance of the inner-loop AP.

At the coordination level, motivated by [14], where the authors propose a solution to the cooperative path-following problem, we propose a control law for the speeds of the UxVs and show that coordination can be achieved by allowing the vehicles to exchange minimum information (the LOS angle only) between each other over a communication network. Performance of the coordination algorithm is analyzed, while taking into account non-ideal communication scenarios (such as failures, drop-outs, and switching topology). We further extend the coordination result by considering non-perfect tracking of the speed commands. We show that the controller’s performance is guaranteed as long as the difference between the commanded and actual speed of the vehicles remains bounded.

The main features of this paper are as follows: (i) VBT is achieved by using only a UxV-target-relative (LOS) angle; (ii) performance of the overall control architecture is guaranteed against communication failures among the UxVs and non-ideal tracking of the reference control commands by the autopilot.

The paper is organized as follows: Section 2 formulates the CVBT problem; it describes the kinematics of the systems of interest and introduces a set of assumptions and limitations on the onboard autopilot, as well as on the supporting communication network. Section 3 presents a control algorithm that solves the problem at hand. Section 4 illustrates the simulation results. Finally, conclusions are summarized in Section 5. Theoretical proofs of the results are reported in the Appendix A, Appendix B, Appendix C, Appendix D and Appendix E.

2. Problem Formulation

In what follows, we first consider a single UxV and characterize the dynamics of the kinematic errors, describing the VBT approach that was previously presented in [21]. Second, following a decentralized coordination strategy similar to the one presented in [23], we consider the case of multiple UxVs by introducing a set of appropriate coordination variables, along with assumptions and limitations on a given communication network.

Table 1. Symbols and definitions.

Symbol	Definition
UxV	Unmanned (ground, aerial, underwater, surface) vehicle
$\mathcal{I}$	Inertial frame
$V_g\left(t\right)$, ${\overrightarrow{v}}_g\left(t\right)$	Speed and velocity of the UxV at time t
$V_{c,i}\left(t\right)$, ${\dot{\psi }}_{c,i}\left(t\right)$	Speed and angular rate commands for UxV i at time t
$\psi \left(t\right)$	Heading angle of the UxV at time t
$V_t\left(t\right)$, ${\psi }_t\left(t\right)$	Speed and heading angle of the UxV at time t
$\rho \left(t\right)$	Horizontal distance between UxV and target at time t
$\lambda \left(t\right)$	Line of sight phase angle at time t
$\widehat{\eta }\left(t\right)$	Vision-based angle tracking error at time t
${\rho }_d$	Desired range
${\rho }_e\left(t\right)$	Vision-based distance tracking error at time t
$x\left(t\right)$	Vision-based tracking error vector at time t
$G_i\left(t\right)$	Neighborhood of vehicle i at time t
$\Lambda \left(t\right)$	Communication graph
$L\left(t\right)$	Laplacian of the graph at time t
$T,\mu$	Parameters representing the quality of the communication network
$\zeta \left(t\right)$	Coordination error vector at time t
$h_{ij}\left(t\right)$	Heterogeneity parameter between nodes i and j at time t

lists the symbols and their definitions for clarity.

2.1. Vision-Based Tracking

Consider the horizontal projection of the UxV and target-relative kinematics in the inertial frame $\mathcal{I}$ illustrated in Figure 2. Let ${\overrightarrow{v}}_g\left(t\right)\in {\mathbb{R}}^3$ and $\psi \left(t\right)\in \mathbb{R}$ be the UxV’s velocity vector and heading angle, respectively. Let $V_g\left(t\right)\in \mathbb{R}$ be the magnitude of ${\overrightarrow{v}}_g\left(t\right)$. Let $V_t\left(t\right)$ be the speed of the target and ${\psi }_t\left(t\right)$ be its heading angle. Let $\rho \left(t\right)\in \mathbb{R}$ be the horizontal distance between the UxV and the target. Let $\lambda \left(t\right)\in \mathbb{R}$ be the LOS phase angle, let ${\overrightarrow{\lambda }}_g\left(t\right)\in {\mathbb{R}}^3$ denote the LOS vector, and let ${\overrightarrow{\lambda }}_p\left(t\right)\in {\mathbb{R}}^3$ be the vector perpendicular to ${\overrightarrow{\lambda }}_g\left(t\right)$. Finally, let $\eta \left(t\right)\in \mathbb{R}$ denote the angle between ${\overrightarrow{\lambda }}_p\left(t\right)$ and ${\overrightarrow{v}}_g\left(t\right)$, which can be written as

$$\eta \left(t\right)=\psi \left(t\right)-(\lambda \left(t\right)-\frac{\pi }{2}).$$

(1)

Since the rotational speed of the LOS can be expressed as

$$\dot{\lambda }\left(t\right)=\frac{V_g\left(t\right)cos\left(\eta \left(t\right)\right)}{\rho \left(t\right)}-\frac{V_t\left(t\right)cos({\psi }_t\left(t\right)-(\psi \left(t\right)-\eta \left(t\right)))}{\rho \left(t\right)},$$

(2)

one can compute the derivative of Equation (1):

$$\dot{\eta }\left(t\right)=-\frac{V_g\left(t\right)cos\eta \left(t\right)-V_t\left(t\right)cos({\psi }_t\left(t\right)-\psi \left(t\right)+\eta \left(t\right))}{\rho \left(t\right)}+\dot{\psi }\left(t\right).$$

Next, define $\widehat{\eta }\left(t\right)\triangleq \eta \left(t\right)/{\rho }_d$.

In what follows, $\widehat{\eta }\left(t\right)$ plays the role of guidance tracking error. Obviously, if $\widehat{\eta }\left(t\right)=0$, then $\eta \left(t\right)=0$, and it follows from Equation (1) that the angle between the UxV velocity vector and the LOS vector equals $\frac{\pi }{2}$ rad. This results in the UxV orbiting around the target, which constitutes part of the control objective. In fact, to accomplish VBT, the control law must also enable the UxV to orbit along a desired circle. This can be achieved when driving the horizontal range $\rho \left(t\right)$ to a desired constant value, ${\rho }_d\in [{\rho }_{dmin},{\rho }_{dmax}]\subset \mathbb{R}$. By projecting the horizontal components of the UxV velocity vector onto the LOS, one can express the dynamics of $\rho \left(t\right)$ as follows:

$$\dot{\rho }\left(t\right)=-V_g\left(t\right)sin\eta \left(t\right)+V_t\left(t\right)sin({\psi }_t\left(t\right)-\psi \left(t\right)+\eta \left(t\right)).$$

Finally, by defining ${\rho }_e\left(t\right)\triangleq 1/\rho \left(t\right)-1/{\rho }_d$, we can write the dynamics of the VBT error states

$$\left\{\begin{array}{c}\begin{array}{cc}\hfill \dot{\widehat{\eta }}\left(t\right)=-\frac{V_g\left(t\right)cos\left(\widehat{\eta }\left(t\right){\rho }_d\right)}{{\rho }_d}& \left({\rho }_e\left(t\right)+\frac{1}{{\rho }_d}\right)+\frac{\dot{\psi }\left(t\right)}{{\rho }_d}\hfill \\ & +\frac{V_t\left(t\right)cos(\widehat{\eta }\left(t\right){\rho }_d-{\psi }_r\left(t\right))\left({\rho }_e\left(t\right)+\frac{1}{{\rho }_d}\right)}{{\rho }_d}\hfill \end{array}\hfill \\ \begin{array}{cc}\hfill {\dot{\rho }}_e\left(t\right)={\left({\rho }_e\left(t\right)+\frac{1}{{\rho }_d}\right)}^2& V_g\left(t\right)sin\left(\eta \left(t\right)\right)\hfill \\ & -{\left({\rho }_e\left(t\right)+\frac{1}{{\rho }_d}\right)}^2{V}_t\left(t\right)sin(\widehat{\eta }\left(t\right){\rho }_d-{\psi }_r\left(t\right)),\hfill \end{array}\hfill \end{array}\right.$$

(3)

where ${\psi }_r\left(t\right)\triangleq \psi \left(t\right)-{\psi }_t\left(t\right)$. Before providing a formal statement of the problem at hand, we give a set of assumptions that must be satisfied:

Assumption 1.

The speed of the UxV is bounded as follows:

$$0<V_{g,min}\le V_g\left(t\right)\le V_{g,max}.$$

Assumption 2.

The speed of the target to be tracked satisfies

$$0\le V_{t,min}\le V_t\left(t\right)\le V_{t,max}.$$

Assumption 3.

The UxV is equipped with an autopilot capable of following speed and yaw rate commands, $V_c\left(t\right)$ and ${\dot{\psi }}_c\left(t\right)$. The tracking performance of the autopilot satisfies

$$|V_c\left(t\right)-V_g\left(t\right)|<{\delta }_V,|{\dot{\psi }}_c\left(t\right)-\dot{\psi }\left(t\right)|<{\delta }_{\psi },$$

for all $t\ge 0$ and for some ${\delta }_V,{\delta }_{\psi }>0$.

Finally, the VBT control problem can be defined as follows:

Problem 1.

Vision-Based Tracking (VBT): Consider a UxV and a target with speeds satisfying Assumptions 1 and 2. Assume that the UxV is equipped with an autopilot satisfying Assumption 3. Then, the objective of VBT is to define a control law for the commanded angular velocity of the UxV, ${\dot{\psi }}_c\left(t\right)$, such that the error vector $x\left(t\right)\triangleq {[\widehat{\eta }\left(t\right),{\rho }_e\left(t\right)]}^{\top }$, with dynamics in Equation (3), converges to a neighborhood of zero.

Remark 1.

The control algorithm utilizes feedback from image processing software (centroid position in camera frame) and an on-board inertial measurement unit, typical for modern autopilots. Hence, only the line of sight (LOS) to the target is assumed to be known, justifying the use of the term “vision-based tracking”. The crucial aspect of the approach is achieving convergence of the horizontal range to a desired value ${\rho }_d$, without knowledge of the actual range $\rho \left(t\right)$, target position, or velocity.

2.2. Coordination

Consider now a set of n UxVs and let $V_{g,i}\left(t\right)$, ${\psi }_i\left(t\right)$, and ${\lambda }_i\left(t\right)$ be the speed, the heading angle, and the LOS angle of the i-th vehicle ($i\in \{1,\dots ,n\}$), respectively. Furthermore, let ${\widehat{\eta }}_i\left(t\right)$ and ${\rho }_{e,i}\left(t\right)$ be its VBT errors with the dynamics described in Equation (3). The objective of coordination is to drive the difference between the UxVs LOS phase angles to a desired value, in order to maintain a desired phase separation. Thus, we consider the i-th and j-th vehicles to be coordinating at time t if their LOS angles satisfy the condition

$${\lambda }_i\left(t\right)-{\lambda }_j\left(t\right)={\lambda }_d,$$

where ${\lambda }_d\in \mathbb{R}$ is the desired separation in terms of LOS angle. Finally, without loss of generality, it is convenient to set ${\lambda }_d=0$; when a specific phase shift is desired, ${\lambda }_d$ can be assigned a desired constant value. Therefore, the coordination problem for multiple UxVs reduces to achieving ${\lambda }_i\left(t\right)-{\lambda }_j\left(t\right)=0,$ $\forall i,j\in \{1,\dots ,n\},i\ne j,\forall t>0$.

To guarantee coordination, the states ${\lambda }_1\left(t\right),\dots ,{\lambda }_n\left(t\right)$ have to be exchanged between the UxVs over a communication network. In this paper, we use the notation and tools from algebraic graph theory. Introduction to the material and further relevant reading can be found in [24]. Let $L\left(t\right)$ be the Laplacian matrix of the communication graph $\Gamma \left(t\right)$. Let $Q\in {\mathbb{R}}^{(n-1)\times n}$ be such that $Q{1}_n=0$ and $Q{Q}^{\top }=I_{n-1}$, with $1_n=[1,1,\dots ,1]\in {\mathbb{R}}^n$. Let $G_i\left(t\right)$ denote the set of vehicles positioned in a neighborhood of the i-th UxV. Now, we can define a set of assumptions that the communication network must satisfy [25]:

Assumption 4.

The i-th UxV can only communicate with the neighboring set $G_i\left(t\right)$.

Assumption 5.

The communication between two vehicles is bidirectional, with no time delays.

Assumption 6.

The connectivity of the communication graph $\Gamma \left(t\right)$ satisfies

$$\frac{1}{n}\frac{1}{T}{\int }_t^{t+T}\overline{L}\left(\tau \right)d\tau \ge \mu I_{n-1}\forall t\ge 0,$$

where $\overline{L}\left(t\right)\triangleq QL\left(t\right)Q^{\top }$ has the same spectrum of $L\left(t\right)$ without the eigenvalue ${\lambda }_1\left(L\left(t\right)\right)=0$, and the parameters $T>0,\mu \in (0,1]$ describe the quality of service of the communication network.

It is worth noting that Assumption 6 stipulates that the communication graph $\Gamma \left(t\right)$ is only required to be connected in an integral sense over the time range T rather than pointwise in time. In fact, the graph may be disconnected during some interval of time or may even fail to be connected at all times. In this sense, it is general enough to capture packet dropouts, loss of communication, switching topologies, etc. Small values of T and large values of $\mu$ indicate better quality of the communication network.

Finally, the coordination problem can be defined as follows:

Problem 2.

Coordination: Consider a set of n UxVs with speed satisfying Assumption 1 and a target with speed satisfying Assumption 2. Assume that the vehicles are equipped with an autopilot satisfying Assumption 3. Finally, suppose that the UxVs are supported by a communication network satisfying Assumptions 4–6. Then, the objective is to design feedback control laws for the commanded speeds $V_{c,i}\left(t\right),\forall i\in \{1,\dots ,n\}$ such that the difference between the LOS angles of the n UxVs converge to a neighborhood of zero, i.e.,

$${\lambda }_i\left(t\right)-{\lambda }_j\left(t\right)\to 0,\forall i,j\in \{1,\dots ,n\},i\ne j,\forall t>0,$$

and such that the speeds of the vehicles converge to predefined constant values, i.e.,

$$V_{g,i}\left(t\right)\to V_{d,i},\forall i\in \{1,\dots ,n\},\forall t>0.$$

3. Main Result

In this section, we first present the outer-loop nonlinear controller that uses the angular velocity ${\dot{\psi }}_{c,i}\left(t\right)$ to solve Problem 1. Then, we formulate a control law for the speed commands $V_{c,i}\left(t\right),\forall i\in \{1,\dots ,n\}$, which solves Problem 2.

3.1. Vision-Based Tracking

Recall from Section 2 that the main objective of the VBT control algorithm is to drive the states $(\widehat{\eta }\left(t\right),{\rho }_e\left(t\right))$ to a neighborhood of zero. To this end, we propose the following guidance law [21]:

$${\dot{\psi }}_c\left(t\right)=\frac{V_g\left(t\right)}{{\rho }_d}cos\left(\eta \left(t\right)\right)-k_{\eta }\eta \left(t\right).$$

(4)

Then, the solution to the VBT problem for a single UxV can be summarized in the following Lemma:

Lemma 1.

Vision-Based Tracking (VBT): Consider a UxV with speed $V_g\left(t\right)$ satisfying Assumption 1 and a moving target with speed satisfying Assumption 2. Let the commanded angular rate ${\dot{\psi }}_c\left(t\right)$ be governed by Equation (4) and assume that the vehicle is equipped with an autopilot satisfying Assumption 3. Moreover, assume that at time $t=0$ the error vector satisfies

$$x\left(0\right)={[\widehat{\eta }\left(0\right),{\rho }_e\left(0\right)]}^{\top }\in {\Omega }_v\triangleq \{x:x^{\top }Px\le c^2\},$$

(5)

with $c=\frac{c_{\rho }}{{\rho }_d}\sqrt{{\lambda }_{min}\left(P\right)}$; $c_{\rho }$ satisfies

$$c_{\rho }\sqrt{\frac{c_{\rho }^2}{4}+{(2+c_{\rho }+c_{\rho }/6)}^2}<\frac{{\rho }_d}{V_{g,max}}\frac{({\lambda }_{min}\left(W\right)-2{\lambda }_v{\lambda }_{max}\left(P\right))}{2{\lambda }_{max}\left(P\right)},$$

(6)

with P and W being positive definite matrices, $0<{\lambda }_v<\frac{{\lambda }_{min}\left(W\right)}{2{\lambda }_{max}\left(P\right)},$ and

$$k_{\eta }>max\left(\frac{2V_g/{\rho }_d}{\sqrt{4V_g/V_{g,min}-1}},1\right).$$

Then, if the velocity of the target $V_t\left(t\right)$ and the angular speed tracking performance of the autopilot (i.e., ${\delta }_{\psi }$) satisfy

$$\frac{V_{t,max}(1+c_{\rho })\sqrt{1+{(1+c_{\rho })}^2}}{{\lambda }_v{\rho }_d^2}+\frac{{\delta }_{\psi }}{2{\lambda }_v{\rho }_d}<\frac{c_{\rho }}{{\rho }_d},$$

the VBT error $x\left(t\right)$, with the dynamics described in Equation (3), is ultimately bounded with bound $\left|\right|x\left(t\right)\left|\right|\le \frac{c_{\rho }}{{\rho }_d}.$

Corollary 1.

Under the assumptions stated in Lemma 1, if the velocity of the target is $V_t=0,\forall t\ge 0$ and the autopilot has ideal performance of (i.e., ${\delta }_{\psi }={\delta }_V=0$), the VBT error converges exponentially to zero as follows:

$$\left|\right|x\left(t\right)\left|\right|\le \sqrt{\frac{{\lambda }_{max}\left(P\right)}{{\lambda }_{min}\left(P\right)}}\left|\right|x\left(0\right)\left|\right|e^{-{\lambda }_{v}t}.$$

(7)

Proof .

The proofs of Lemma 1 and Corollary 1 are given in Appendix A and Appendix B, respectively.

3.2. Coordinated Vision-Based Tracking

To drive the LOS angles of the UxVs to the same increasing value, the speed of the vehicles is adjusted based on the coordination information exchanged over the communication network. Since, by assumption, each UxV can only communicate with a neighboring set of vehicles, denoted as $G_i\left(t\right)$, we let the commanded velocity $V_{c,i}\left(t\right)$ for the i-th vehicle be governed by

$$V_{c,i}\left(t\right)=\frac{u_{\mathrm{coord},i}\left(t\right){\rho }_{d,i}}{cos\left({\eta }_i\left(t\right)\right)},$$

(8)

where

$$\begin{array}{cc}\hfill u_{\mathrm{coord},1}\left(t\right)& =-a\sum _{j\in G_1\left(t\right)}({\lambda }_1\left(t\right)-{\lambda }_j\left(t\right))+\frac{V_{d,1}}{{\rho }_{d,1}},\hfill \end{array}$$

(9a)

$$\begin{array}{cc}\hfill u_{\mathrm{coord},i}\left(t\right)& =-a\sum _{j\in G_i\left(t\right)}({\lambda }_i\left(t\right)-{\lambda }_j\left(t\right))+{\chi }_{I,i}\left(t\right),i=2,...,n,\hfill \end{array}$$

(9b)

$$\begin{array}{cc}\hfill {\dot{\chi }}_{I,i}\left(t\right)& =-b\sum _{j\in G_i\left(t\right)}({\lambda }_i\left(t\right)-{\lambda }_j\left(t\right)),{\chi }_{I,i}\left(0\right)=\frac{V_{d,i}}{{\rho }_{d,i}},i=2,...,n,\hfill \end{array}$$

(9c)

with vehicle 1 elected as leader of the fleet, and $a,b>0$. Note that $V_{d,1},V_{d,2},\dots ,V_{d,n}$ are constant desired speeds that satisfy $\frac{V_{d,1}}{{\rho }_{d,1}}=\frac{V_{d,2}}{{\rho }_{d,2}}=\dots =\frac{V_{d,n}}{{\rho }_{d,n}},$ in order to maintain the same turn rate around the target. Moreover, the following inequalities must hold to guarantee feasibility of the mission:

$$0<V_{g,min}/\delta <V_{d,min}\le V_{d,j}\le V_{d,max}<\delta V_{g,max},$$

$\forall j\in \{1,\dots ,n\}$, where $0<\delta <1$.

Remark 2.

Parameter δ guarantees that a margin is left between the minimum (and maximum) UxV’s velocity and the desired speed, therefore ensuring the existence of a solution for the coordination problem. In fact, assume that the desired speed for the i-th UxV is $V_{d,i}=V_{g,min}$. It follows that, if the vehicle is required to slow down in order to coordinate with the rest of the fleet, it would not be possible since no margin is left between the desired and minimum feasible velocities.

Next, we define the coordination error vector as

$$\zeta \left(t\right)\triangleq {[{\zeta }_1^{\top }\left(t\right),{\zeta }_2^{\top }\left(t\right)]}^{\top },$$

(10)

where

$${\zeta }_1\left(t\right)=Q\lambda \left(t\right),{\zeta }_2\left(t\right)={\chi }_I\left(t\right)-\frac{V_d}{{\rho }_d}{1}_n,$$

$$\lambda \left(t\right)\triangleq {[{\lambda }_1\left(t\right),...,{\lambda }_n\left(t\right)]}^{\top }\in {\mathbb{R}}^n,$$

$${\chi }_I\left(t\right)\triangleq {[{\chi }_{I,2}\left(t\right),...,{\chi }_{I,n}\left(t\right)]}^{\top }\in {\mathbb{R}}^{n-1},$$

where $1_n={[1,1,\dots ,1]}^{\top }\in {\mathbb{R}}^n$ and

$$\frac{V_d}{{\rho }_d}\triangleq \frac{V_{d,1}}{{\rho }_{d,1}}=...=\frac{V_{d,n}}{{\rho }_{d,n}}.$$

Note that, from the properties of Q, if at time t we have ${\zeta }_1\left(t\right)=0$, then $\lambda \left(t\right)\in \mathrm{span}\left\{1_n\right\}$, which implies that ${\lambda }_i\left(t\right)={\lambda }_j\left(t\right)$ for all $i,j\in \{1,\dots ,n\}$. Moreover, if at the same time ${\zeta }_2\left(t\right)=0$, then the commanded speed of each vehicle is equal to its desired constant speed, which is

$$V_{c,i}=V_{d,i},\forall i\in \{1,\dots ,n\}.$$

Before stating the main result of the paper, let us define $c_v,c_c>0$ as $c_v\triangleq \frac{c_{\rho }}{{\rho }_{d,max}}\sqrt{{\lambda }_{min}\left(P\right)},$ and

$$c_c\triangleq min\left(\frac{V_{g,max}-V_{d,max}/c_{\eta }}{2{\kappa }_1{\rho }_{d,max}(an+1)}-\frac{{\kappa }_2}{{\kappa }_1}\frac{c_{\rho }}{{\rho }_{d,max}},\frac{V_{d,min}-V_{g,min}}{2{\kappa }_1{\rho }_{d,max}(an+1)}-\frac{{\kappa }_2}{{\kappa }_1}\frac{c_{\rho }}{{\rho }_{d,max}}\right),$$

where P is a positive definite matrix, $c_{\rho }$ satisfies Equation (6), and

$$c_{\rho }<min\left(\frac{1}{2kn((1+a)n+1+a){\rho }_k^2},\frac{V_{g,max}-V_{d,max}/c_{\eta }}{2{\kappa }_2(an+1)},\frac{V_{d,min}-V_{g,min}}{2{\kappa }_2(an+1)}\right),$$

where $k>2$; $0<c_{\eta }\le 1$; and some positive ${\rho }_k$, ${\kappa }_1$ and ${\kappa }_2$.

Finally, the main result is stated in the following Theorem:

Theorem 1.

Coordinated Vision-Based Tracking (CVBT): Consider a set of n UxVs and a target with speed satisfying Assumption 2. Let the commanded angular rate of the UxVs be governed by Equation (4), and let the commanded speed be governed by Equation (8). Assume that the UxVs are equipped with autopilots satisfying Assumption 3. Suppose that the UxVs communicate over a network satisfying Assumptions 4–6. If at time $t=0$ the VBT error and the coordination error satisfy

$$x_i\left(0\right)\in {\Omega }_v\triangleq \{x_i:x_i^{\top }P{x}_i\le c_v^2\},\forall i\in \{1,\dots ,n\},$$

(11)

and

$$\zeta \left(0\right)\in {\Omega }_c\triangleq \{\zeta :|\left|\zeta \right||\le c_c\},$$

(12)

respectively, then we have the following:

(i)

The velocities of the UxVs satisfy Assumption 1;

(ii)

The VBT error $x\left(t\right)$, with dynamics described in Equation (3), is ultimately bounded with bound

$$\left|\right|x\left(t\right)\left|\right|\le \frac{c_{\rho }}{{\rho }_d},$$

(13)

if the velocity of the target $V_t\left(t\right)$ and the angular rate tracking performance of the autopilot (i.e., ${\delta }_{\psi }$) satisfy

$$\frac{V_{t,max}(1+c_{\rho })\sqrt{1+{(1+c_{\rho })}^2}}{{\lambda }_v{\rho }_d^2}+\frac{{\delta }_{\psi }}{2{\lambda }_v{\rho }_d}<\frac{c_{\rho }}{{\rho }_d};$$

(14)

(iii)

The coordination error vector $\zeta \left(t\right)$ defined in Equation (10) satisfies the following bound:

$$\left|\right|\zeta \left(t\right)\left|\right|\le {\kappa }_1^{*}\left|\right|\zeta \left(0\right)\left|\right|e^{-{\lambda }_c(1-\theta )t}+{\kappa }_2^{*}\underset{\tau \in [0,t]}{sup}\left|\right|u_{xv}\left(\tau \right)\left|\right|,$$

(15)

where $u_{xv}\left(t\right)=3max\left(\frac{V_{d,max}}{{\rho }_{d,min}}{\rho }_{d,max}\left|\right|x\left|\right|,\frac{V_{t,max}}{{\rho }_{min}},\frac{{\delta }_V}{{\rho }_{min}}\right),$ and ${\kappa }_1,{\kappa }_2$ are positive constants.

Corollary 2.

Under the same assumptions stated in Theorem 1, if the velocity of the target is $V_t=0,\forall t\ge 0$, and the autopilot has ideal performance (i.e., ${\delta }_{\psi }={\delta }_V=0$), we have the following:

(i)

The velocities of the UxVs satisfy Assumption 1;

(ii)

The VBT error $x\left(t\right)$, with dynamics described in Equation (3), converges exponentially to zero (see Equation (7));

(iii)

The coordination error vector $\zeta \left(t\right)$ defined in Equation (10) converges exponentially to zero as follows:

$$\begin{array}{c}\hfill \left|\right|\zeta \left(t\right)\left|\right|\le {\overline{\kappa }}_1\left|\right|\zeta \left(0\right)\left|\right|e^{-{\lambda }_{c}t}+{\overline{\kappa }}_2\left|\right|x\left(0\right)\left|\right|e^{-\frac{{\lambda }_v+{\lambda }_c}{2}t},\end{array}$$

where ${\overline{\kappa }}_1,{\overline{\kappa }}_2$ are positive constants, with rate of convergence ${\lambda }_c=\frac{\overline{\gamma }}{(1+kn){\rho }_k}$, with $\overline{\gamma }\ge \frac{an\mu }{T{(1+anT)}^2}$, and

$${\rho }_k\ge max\left(\frac{2k\overline{\gamma }}{1+kn},1,\left(1+\frac{1}{4}{\overline{\gamma }}^2+\frac{{(kn+1)}^2}{{\overline{\gamma }}^2{k}^2{n}^3}\right)\right),k>2.$$

(16)

Proof .

The proofs of Theorem 1 and Corollary 2 are given in Appendix C and Appendix D, respectively.

Remark 3.

The maximum guaranteed rate of convergence can be obtained by letting $a=\frac{1}{nT}$, which is

$$\underset{a>0}{max}{\lambda }_c=\frac{\mu }{4T^2(1+kn){\rho }_k}.$$

Moreover, it is worth noticing that if one modifies the control laws given in Equation (9) as follows:

$$\begin{array}{cc}\hfill u_{\mathrm{coord},i}\left(t\right)& =-a\sum _{j\in G_1\left(t\right)}({\lambda }_i\left(t\right)-{\lambda }_j\left(t\right))+\frac{V_{d,i}}{{\rho }_{d,i}},\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill {\dot{\chi }}_{I,i}\left(t\right)& =-b\sum _{j\in G_i\left(t\right)}({\lambda }_i\left(t\right)-{\lambda }_j\left(t\right)),{\chi }_{I,i}\left(0\right)=\frac{V_{d,i}}{{\rho }_{d,i}},i=1,...,n,\hfill \end{array}$$

(17b)

i.e., all the UxVs involved in the mission are leaders; then, one can show [23] that the rate of convergence reduces to

$${\lambda }_c=\frac{\mu }{4T^2(1+k){\rho }_k}.$$

This implies that the rate of convergence depends on the quality of the communication network, and it scales with the number of vehicles.

4. Simulation Results

This section presents the simulation results. In the proposed scenario, six UxVs are asked to rotate around a target along two orbits of different radius, while maintaining a phase separation of $\frac{\pi }{3}\mathrm{rad}$ from each other. In particular, UxV1—elected as the leader—UxV3, and UxV5 are required to rotate on an orbit of radius $100\mathrm{m}$, while UxV2, UxV4, and UxV6 rotate on an orbit of radius $150\mathrm{m}$. The desired speeds at which the UxVs are asked to converge are $V_{d,1}=V_{d,3}=V_{d,5}=25\mathrm{m}/\mathrm{s}$ and $V_{d,2}=V_{d,4}=V_{d,6}=37.5\mathrm{m}/\mathrm{s}$.

We assume that the vehicles can only communicate with each other when the distance between them is less than $250\mathrm{m}$. All-to-all communication is never achieved throughout the mission. In Figure 3, the black lines connecting vehicle i and vehicle j indicate that the two UxVs are communicating. For clarity of presentation, these lines are shown only at four different time instances ($t=0\mathrm{s},180\mathrm{s},300\mathrm{s},600\mathrm{s}$).

To illustrate the performance of the CVBT controller in the presence of non-ideal tracking of reference commands, we implemented a proportional controller as follows

$$\begin{array}{cc}\hfill {\dot{V}}_g\left(t\right)& =k_V(V_c(t-t_d)-V_g\left(t\right)),\hfill \\ \hfill \ddot{\psi }\left(t\right)& =k_{\psi }({\dot{\psi }}_c(t-t_d)-\dot{\psi }\left(t\right)),\hfill \end{array}$$

with gains $k_V=0.2,k_{\psi }=0.05$, and time delay $t_d=0.1\mathrm{s}$. The time delay is intentionally introduced to simulate the effects of real-world delays that can impact the performance of an unmanned vehicle’s autopilot system.

The control gains of the CVBT controller have been chosen as follows: $k_{\eta }=2,a=0.1,$$b=0.05$. The imposed speed limits are $V_{g,min}=17\mathrm{m}/\mathrm{s}$ and $V_{g,max}=32\mathrm{m}/\mathrm{s}$.

Mission

At $t=0\mathrm{s}$, the target is positioned at $[-400;-400]\mathrm{m}$ with velocity $V_t\left(t\right)=0$, and the UxVs are positioned in a straight-line formation near the target, as depicted in Figure 3. When the mission begins, the UxVs start approaching the desired orbits with different speed profiles. Figure 4 and Figure 5 show the convergence to a neighborhood of zero of the VBT error. Figure 6 and Figure 7 illustrate the coordination error and the actual speeds. The slow rate of convergence depends on the facts that the UxVs start the mission far away from each other and each vehicle is able to communicate with only few of its neighbors (see Figure 3, black lines).

At $t=200\mathrm{s}$, the target starts moving in the northeast direction at a speed of $V_t\left(t\right)=5\mathrm{m}/\mathrm{s}$. As it can be seen in Figure 7, the speeds of the vehicles oscillate around the desired values. The reason is that the trajectory of the UxVs at this time is not a circular orbit, as it is in the case of a stationary target (this can be noted from the trajectory of UxV1 depicted in Figure 3), and following this trajectory requires oscillatory (linear and angular) velocity commands. Finally, Figure 4, Figure 5 and Figure 6 show the oscillatory behavior of the VBT and the coordination errors, which, however, remain bounded as expected.

At $t\approx 380\mathrm{s}$, the target reaches position $[200;200]\mathrm{m}$ and stops (i.e., $V_t\left(t\right)=0\mathrm{m}/\mathrm{s}$). Again, the speeds converge to the desired values, while the VBT and the coordination error converge close to zero.

A video of these results and the Matlab code used to generate the plots can be found at the following link: https://github.com/venanziocichella/Cooperative-Vision-Based-Tracking (accessed on 2 March 2023).

5. Conclusions

In this paper, a solution for the coordinated vision-based tracking problem is given. The proposed control laws enable n unmanned vehicles (UxVs) to move around a target along n desired orbits, while coordinating with each other (i.e., maintaining a desired phase shift). The yaw rate is used to solve the vision-based tracking problem. The coordination between the UxVs is achieved by adjusting the speed of each vehicle. Non-perfect tracking performance of an on-board autopilot is considered. First, a solution to the vision-based tracking problem is given considering the case of a single UxV. Second, we provide a solution to the coordination problem. The simulation results validate the theoretical findings.