Particle Swarm Optimization for Target Encirclement by a UAV Formation ^†

Abstract

This paper presents an idea of using particle swarm optimization (PSO) to tune the control system of a decentralized unmanned aerial vehicle (UAV) formation. Simulations were run on a consensus-based decentralized UAV formation. Vector field guidance was used to control the formation. A fitness function is proposed that is based not only on the error of distance to the circular path, but also on the relative inter-UAV distance error. To demonstrate the effectiveness of the proposed method, the obtained results of such tuning are compared to those obtainable by the conventional trial and error method.

1. Introduction

Use of autonomous robots in groups is a promising area of research in today’s mobile robotics, and it receives much attention. Decentralized control of autonomous robots is one of the more complex yet effective approaches. Thus, many papers cover decentralized control applications in ground-based robots  [1,2,3] and autonomous unmanned aerial vehicles (UAVs) alike [4,5]. Most papers cover decentralized control of rotary-wing UAV groups, mainly quadcopter formations [6,7].

Currently, control of decentralized swarms of autonomous robots [8,9,10] and unmanned aerial vehicles (UAVs) [11,12] is a promising area of research. However, beside control and trajectory-planning algorithms, these formations require optimizing their transient trajectories. Since autonomous robot formations, including UAV formations, are complex nonlinear interconnected systems, soft computing could be effective for such optimization.

Control optimization research focuses on evolutionary algorithms [13] including genetic algorithms [14,15] and particle swarm optimization [16,17]. Some papers show implementation of particle swarm optimization for single-quadcopter controllers [18,19,20], UAV formations [21,22,23], UAV trajectory optimization [24], UAV movement planning [25] and UAV formations [26] in an uncertain environment. However, swarm optimization for vector field-controlled UAV formations remains under-researched. Thus, the goal hereof is to test the feasibility of applying particle swarm optimization to a decentralized consensus-based UAV formation controlled via vector field guidance.

2. Preliminary Remarks and Statement of Problem

The assumptions here are the same as in [12,27]: no wind, inter-UAV communication enabled and sufficiently accurate computation of the relative inter-UAV distance in the group.

The dynamics of a decentralized UAV formation can be tested via full models as well as via high-level models that approximate the movement of the formation provided that the UAVs are equipped with fine-tuned onboard autopilots. Full models are preferable, e.g., for final testing of control algorithms or for testing formation-wide stability. High-level models, also referred to in the literature as guidance models, are more suitable for simulating spatial movement planning algorithms as well as for trajectory optimization. However, full models are still useful when the trial and error method is used to find the initial values for the formation controller coefficients that are further to be used for trajectory optimization.

UAV formation trajectory optimization is a subtask of cooperative target tracking. This task is sometimes referred to as collective circumnavigation or target encirclement. The idea is to maintain a certain preset distance not only between the UAVs (through specified angular values) but also to the target encirclement orbit, which is a moving path. Formal statement of the problem can be found in Section 4. We covered a similar problem in [28], where we used a genetic algorithm to solve it.

3. Consensus-Based UAV Formation Control Algorithm

This paper uses a decentralized neighbor-to-neighbor interaction topology, where UAVs only receive data from the neighboring aircraft. The topology can be defined as a graph referred to as the “interaction graph” hereinafter. Paper [12] shows a mathematical model of interaction in a consensus-based UAV formation. The same model is described later in this section. Let $𝒩$ be the set of all UAV agents.

Let ${\mathbf{e}}_{\theta }\in {\mathbb{R}}^{N\times 1}$ be this vector, where ${\mathbb{R}}^{N\times 1}$ is a space of $N\times 1$-dimensional matrices with components from $\mathbb{R}$. To find this vector, use some elements of the vector of all possible relative phase shift angle errors ${\overline{\mathbf{e}}}_{\theta }=\left({\widehat{e}}_{i,j}\right)\in {\mathbb{R}}^{N\left(N-1\right)\times 1}$, where ${\widehat{e}}_{i,j}$ is the value of error for the directly interacting ith and jth agents. The choice is dictated by the interaction architecture; in this research, the control action vector is set as such for open-chain interaction in the same manner as described in [12,27]:

$${\mathbf{e}}_{\theta }=\left[\begin{array}{c}{e}_1\\ ⋮\\ e_k\\ ⋮\\ e_N\end{array}\right]=\left[\begin{array}{c}{\widehat{e}}_{12}\\ ⋮\\ -{\widehat{e}}_{k-1,k}+{\widehat{e}}_{k,k+1}\\ ⋮\\ -{\widehat{e}}_{N-1,N}\end{array}\right]={\widehat{\mathbf{M}}}_{\theta }{\stackrel{⌣}{\mathbf{e}}}_{\theta }+\mathbf{D},$$

(1)

where $\mathbf{D}=-{\mathbf{M}}_{\theta }{\mathbf{H}}_{\theta }^{-1}{\left({\left({\mathbf{P}}_{\theta }^d\right)}^{\mathrm{T}},{\widehat{P}}_{\theta }\right)}^{\mathrm{T}}$ is a system control vector in the space of relative distances (an $\left(N-1\right)$-dimensional space generated by the interaction graph incidence matrix columns), and ${\mathbf{H}}_{\theta }$ is a matrix that specifies the agents for agent-to-agent distance measurements, defined as follows:

$${\mathbf{H}}_{\theta }=\left[\begin{array}{c}{\mathbf{q}}_1\\ {\mathbf{q}}_2\\ ⋮\\ {\mathbf{q}}_N\end{array}\right],{\mathbf{q}}_i={\left[\begin{array}{c}⋮\\ \begin{array}{c}\begin{array}{c}1\\ ⋮\end{array}\\ -1\end{array}\\ ⋮\end{array}\right]}^{\mathrm{T}},i<N,{\mathbf{q}}_N={\left[\begin{array}{c}1\\ 1\\ ⋮\\ 1\end{array}\right]}^{\mathrm{T}},$$

where ${\mathbf{H}}_{\theta }\in {\mathbb{R}}^{N\times N},{\mathbf{q}}_i\in {\mathbb{R}}^{1\times N}$ and the positions of “1” and “−1” in ${\mathbf{q}}_i$ are determined according to the structure of the interaction graph.

${\mathbf{P}}_{\theta }^d\in {\mathbb{R}}^{\left(N-1\right)\times 1}$ is the vector of the desired inter-UAV phase shift angles and ${\widehat{P}}_{\theta }={\displaystyle \sum _{k=1}^N}{\phi }_k$ is the total of the current UAV phase angles in an inertial coordinate system;

${\stackrel{⌣}{\mathbf{e}}}_{\theta }$ = $({{\stackrel{⌣}{\mathbf{e}}}_{i,i+1})}_{\overline{i=1,N-1}}\in {\mathbb{R}}^{(N-1)\times 1}$ is the vector of current phase shift angles for directly co-engaged agents, calculated by the triple scalar product, e.g., when the final movement is directed clockwise, the following applies:

If $\mathbf{n}·\left({\mathbf{d}}_i\times {\mathbf{d}}_{i+1}\right)\ge 0$, then ${\stackrel{⌣}{e}}_{i,i+1}$ = β = arccos $\frac{({\mathbf{d}}_i,{\mathbf{d}}_{i+1})}{\Vert {\mathbf{d}}_i\Vert \Vert {\mathbf{d}}_{i+1}\Vert }$ and ${\stackrel{⌣}{e}}_{i,i+1}=2\pi -\beta$ in other cases, where d_k,k ∈ $𝒩$ is the vector of aircraft-to-moving-target distance at a given time, n = (0, 0, 1)^T;

${\mathbf{M}}_{\theta }\in {\mathbb{R}}^{N\times N}$ is an interaction matrix that in cases of decentralized neighbor–neighbor interactions as herein is as follows:

$${\mathbf{M}}_{\theta }=\left[\begin{array}{ccccc}-1& 1& 0& \cdots & 0\\ 1& -2& \ddots & \ddots & ⋮\\ 0& \ddots & \ddots & 1& 0\\ ⋮& \ddots & 1& -2& 1\\ 0& \cdots & 0& 1& -1\end{array}\right];$$

${\widehat{\mathbf{M}}}_{\theta }\in {\mathbb{R}}^{N\times \left(N-1\right)}$ is a matrix derived from the matrix ${\mathbf{M}}_{\theta }{\mathbf{H}}_{\theta }^{-1}$ by removing the Nth column.

For collective target encirclement, this paper uses the same control laws as in [12,27]. For control based on angular errors, it uses the speeds of the UAVs in the formation. The following control command vector ${\mathbf{v}}^c$ is set for UAV speeds:

$${\mathbf{v}}^c={\left[\begin{array}{cccc}{v}_1& v_2& \dots & v_N\end{array}\right]}^{\mathrm{T}}=v{\mathbf{1}}_N+\mathbf{L},$$

(2)

where ${\mathbf{1}}_N={\left[\begin{array}{cccc}1& 1& \dots & 1\end{array}\right]}^{\mathrm{T}}\in {\mathbb{R}}^{N\times 1}$ and the vector $\mathbf{L}={\left(v_f\left(2/\phantom{2\pi }\pi \right)arctan\left\{k_{\theta }{e}_i+k_{\dot{\theta }}{\dot{e}}_i\right\}\right)}_{i=\overline{1,N}}\in {\mathbb{R}}^{N\times 1}$ is found given (1), $k_{\theta }$ is the positive tuning coefficient, $k_{\dot{\theta }}$ is the positive tuning coefficient for the derivative signal, $v_f$ is the maximum norm of the additional velocity vector that is to be adjusted for the constraints of the real-UAV dynamics, and v is the ultimate linear cruise speed of the UAVs provided that the target is stationary.

Path error-based control relies on the heading angles of the UAVs in the formation. The following control law from [12,27] is applied to the heading-angle command vector ${\mathrm{Ø}}^c$ with slightly modified coefficients:

$${\mathrm{Ø}}^c={\left({\phi }_i+\lambda \left[\frac{\pi }{2}+arctan\left(k_o^i\left(d_i-\rho \right)+k_{\dot{o}}^i{\dot{d}}_i\right)\right]\right)}_{i=\overline{1,N}}\in {\mathbb{R}}^{N\times 1},$$

(3)

where $d_i$ is the ith UAV-to-target distance, ${\dot{d}}_i$ is the corresponding derivative signal, $k_o^i$ is the tuning coefficient for the distance-to-circular-path signal for the ith UAV, $k_{\dot{o}}^i$ is the tuning coefficient for the distance-to-circular-path derivative signal for the ith UAV, $\rho$ is the radius of the circular path that the UAV follows whilst encircling the target, ${\phi }_i$ is the phase angle of circumvolution around the target for the ith UAV.

4. Implementation of Particle Swarm Optimization

The standard particleswarm solver from MATLAB2015b with default parameters was used for particle swarm optimization. A four-UAV formation was tested for this paper, hence eight tuning parameters. These parameters are coefficients in the control law (3):

$$\mathbf{K}=\left[\begin{array}{ccccc}{k}_o^1& k_{\dot{o}}^1& \dots & k_o^4& k_{\dot{o}}^4\end{array}\right]\in {\mathbb{R}}^{1\times 8}.$$

We also added upper and lower bounds as follows:

$$\left[\begin{array}{ccccc}0.1& 0.1& \dots & 0.1& 0.1\end{array}\right]\le \mathbf{K}\le \left[\begin{array}{ccccc}30& 30& \dots & 30& 30\end{array}\right]$$

These constraints were chosen in order to preserve the UAV formation stability. Stability can be lost if the control law coefficients go beyond certain limits in the absence of adaptive control.

The initial guess for the vector $\mathbf{K}$ was as follows:

$${\mathbf{K}}_{initial}=\left[\begin{array}{ccccc}1& 2& \dots & 1& 2\end{array}\right]\in {\mathbb{R}}^{1\times 8}.$$

For the fitness function, we chose the following:

$$F_{fitness}\stackrel{\Delta }{=}{\int }_0^{t_n}t\sum _{i=1,\dots ,4}\left(\left|d_i-\rho \right|+\left|e_i\right|\right)dt,$$

(4)

where $t_n$ is the particle swarm optimization time; the remaining parameters are defined in Equations (2) and (3). Thus, this solution optimizes not only for the error of each UAV’s distance to the ultimate orbit of target encirclement but also for the relative neighbor-to-neighbor distance errors.

The formal statement of the goal would be as follows:

$$\mathrm{minimize}{F}_{fitness}\stackrel{\Delta }{=}{\int }_0^{t_n}t\sum _{i=1,\dots ,4}\left(\left|d_i-\rho \right|+\left|e_i\right|\right)dt,$$

$$\mathrm{s}.\mathrm{t}.\left[\begin{array}{ccccc}0.1& 0.1& \dots & 0.1& 0.1\end{array}\right]\le \mathbf{K}\le \left[\begin{array}{ccccc}30& 30& \dots & 30& 30\end{array}\right]$$

5. Simulation Results

5.1. Simulation Parameters

For simulation, we ran a high-level UAV model from [29]. The formation consisted of four UAVs of the same type. To make an initial guess, we also ran full UAV models of this formation. This allowed us to find, by trial and error, a controller coefficient that would keep the entire formation system stable. Control laws (2) and (3) were used in the simulation. The simulation parameters are shown in

Table 1. Simulation parameters.

Parameter	Symbol	Values
Initial UAV heading vector (rad)	$Ø\left(0\right)$	${\left[\begin{array}{cccc}0& 0& 0& 0\end{array}\right]}^{\mathrm{T}}$
Initial UAV speed vector (m/s)	$\mathbf{v}\left(0\right)$	${\left[\begin{array}{cccc}13& 13& 13& 13\end{array}\right]}^{\mathrm{T}}$
Circular path radius (m)	$\rho$	200
Initial target coordinates (m)	$\mathbf{c}={\left[\begin{array}{ccc}{c}_e\left(0\right)& c_n\left(0\right)& h\left(0\right)\end{array}\right]}^{\mathrm{T}}$	${\left[\begin{array}{ccc}500& 500& 100\end{array}\right]}^{\mathrm{T}}$
Ultimate cruise speed of the formation in case of a stationary target (m/s)	v	13
Target speed (m/s)	$v_{\mathrm{target}}$	2
Target course angle (rad)	${\chi }_{\mathrm{target}}$	$\pi /\phantom{\pi 4}4$
Vector of initial UAV coordinates in the ICS (m)	$\begin{array}{c}{\mathbf{p}}_1\left(0\right),\hfill \\ {\mathbf{p}}_2\left(0\right),\hfill \\ {\mathbf{p}}_3\left(0\right),\hfill \\ {\mathbf{p}}_4\left(0\right)\hfill \end{array}$	$\begin{array}{c}{\left[\begin{array}{ccc}180& 910& -100\end{array}\right]}^{\mathbf{T}},\hfill \\ {\left[\begin{array}{ccc}245& 247& -100\end{array}\right]}^{\mathbf{T}},\hfill \\ {\left[\begin{array}{ccc}710& 135& -100\end{array}\right]}^{\mathbf{T}},\hfill \\ {\left[\begin{array}{ccc}915& 760& -100\end{array}\right]}^{\mathbf{T}}\hfill \end{array}$
Desired inter-UAV phase shift angles $\left({}^{\circ }\right)$	${\mathbf{P}}_{\theta }^d$	${\left[\begin{array}{ccc}270& 260& 290\end{array}\right]}^{\mathrm{T}}$
Initial tuning coefficients	$k_{\theta },k_{\dot{\theta }},k_o^i,k_{\dot{o}}^i$	35, 15, 1, 2

.

5.2. Simulation Results and Discussion

Figure 1 shows how the fitness function changed during optimization. Apparently, the function’s value stopped declining drastically after 25 iterations. However, a significantly increasing number of iterations would be required to further reduce the value.

Running the optimization algorithm returned the following $\mathbf{K}$ = ${\mathbf{K}}_{opt}$ values that were used in the simulation:

$${\mathbf{K}}_{opt}={\left[\begin{array}{cccccccc}4.7426,& 25.1807,& 2.6214,& 23.8265,& 20.6034,& 29.4922,& 18.7241,& 26.8984\end{array}\right]}^{\mathrm{T}}$$

Figure 2 shows UAV formation angular errors before and after optimization. As can be seen in the graphs, optimization enabled the formation to reach the pre-specified relative angular positions somewhat faster. Figure 3 shows how path errors changed in the UAV formation. As can be seen from the graphs, UAV1 and UAV2 showed the most drastic changes. Apparently, transient trajectories before and after optimization are different (Figure 4). Even though the trajectories look similar in the figure, they are still different. That is especially noticeable in the trajectories for UAV1 and UAV2.

Notably, although the controller coefficients were tuned only for the control law (3), the fitness function included the total angular error of the formation (4). The reason for this was that control by path errors is tied to control by angular errors in a decentralized UAV formation system. This connection can be seen, among other things, in the simulation results in Figure 2.

6. Conclusions

The paper demonstrates a successful use of particle swarm optimization for target encirclement and tracking by a UAV formation. The formation itself was a vector fieldcontrolled decentralized formation. Simulations showed a reduction in the proposed fitness function as well as a change in the pattern of transient trajectories. A close connection was found between optimizing the path error controller optimization and the quality of transient trajectories for angular errors in the UAV formation.