Evolutionary Task Allocation and Cooperative Control of Unmanned Aerial Vehicles in Air Combat Applications

Abstract

Autonomous air combat of unmanned aerial vehicles (UAVs) is a typical and difficult problem that must be studied in the field of military intelligence. The UAVs are required to work together to accomplish different tasks in the air combat. In this note, both task allocation and cooperative control of UAVs are studied for air combat applications. A two-layer framework of the air combat is established containing a task allocation upper layer and a cooperative control lower layer. In the task allocation upper layer, an evolutionary task allocation strategy is proposed for UAVs to allocate tasks by considering their cooperative behavior. After the task allocation, the UAVs, which are allocated for the same task, form the same group. In the cooperative control lower layer, a cooperative control approach is proposed for groups of UAVs to maintain a desired formation to accomplish the corresponding task. The evolutionary stable state and stabilization of the UAVs are achieved. Finally, both Software-in-the-Loop and Model-in-the-Loop simulations are conducted based on the RflySim platform to demonstrate the effectiveness of the proposed approach and further display the multi-UAV air combat vision realistically.

1. Introduction

In the era of information war, more and more unmanned combat equipment has been widely used in the modern battlefield [1]. Note that a single unmanned combat equipment only can perform some simple and limited tasks. While the efficiency of the whole combat system can be improved by multiple unmanned combat equipment with complementary capacity and action coordination. Therefore, the form of unmanned combat is gradually developing from a single platform to a “swarm” of multiple platforms [2]. Unmanned aerial vehicles (UAVs) have become a new type of combat platform that countries are competing to develop [3,4]. Unmanned aerial vehicle (UAV) swarms are regarded as one of the main forms of intelligent combat in the future battlefield, and the effective collaboration among UAVs is the key to affect the combat effectiveness. In this paper, we study the task allocation and cooperative control strategy of multiple UAVs under limited information and resources in the air combat vision.

2. Related Work

In the recent years, UAV swarms have received unprecedented attention due to their better economy, adaptability, survivability, and task diversity [5,6,7]. In the near future, UAV swarms will affect or even change the rules of combat. With the deepening and development of applications of UAVs, researchers hope that UAVs can complete more complex and diverse tasks [8,9]. However, a single UAV often has limited load capacity, which further limits its operation ability.

How to reasonably allocate tasks under the condition of limited information and resources, so that UAVs can efficiently cooperate to complete tasks, is the primary problem to improve air combat effectiveness [10,11,12]. A human-to-robot task allocation problem has been studied for multi-robot networks by minimum execution time-based selection [13]. A joint optimization scheme has been proposed for heterogeneous multi-UAV networks by considering both task and resource layers [14]. A survey has been given to summarize the dynamic task allocation strategies in recent years for improving the autonomy of UAV swarms [15]. A bid-based grouping strategy has been given to improve the communication efficiency of UAV swarms [16].

Note that cooperation, which is one of the natural laws of evolution, plays a very important role in the progress and development of human society and the evolution of biological groups. Evolutionary game theory, which is a combination of traditional game theory and biological evolution theory, can be used to design a strategy to promote cooperation among individuals [17]. By evolutionary game theory, groups adapt to the external environment through learning, imitation, trial and error, and further reach the evolutionary stable state under the conditions of incomplete rationality, asymmetric information, and deviation from the environment and expectations [18]. It can also be said that evolutionary game theory uses cooperation to replace malignant competition to allocate the limited resources and further to maximize the overall combat effectiveness. Many researchers use evolutionary game theory to study cooperation problems, and have made many important achievements, which are widely used in solving practical problems [19,20,21]. The evolutionary task allocation has been studies in uncertain environments [22], Internet of Things [23], cloud manufacturing [24], bandwidth allocation [25], and resource-constrained heterogeneous environments [26].

It is noted that the cooperative control of UAVs has been one of the hot topics due to its various applications in practice [27]. A cooperative occupation model has been proposed for unmanned combat aerial vehicles by considering both weapon engagement zone and combat geometry [28]. By tuning parameters, a modified potential field method has been given for formation control of multiple robots [29]. A mixed-initiative formation control approach has been studied for UAVs in a leader–follower framework [30]. Note that most researchers consider the task allocation problem and cooperative control separately. There is a lot of space to explore the algorithms applicable for the air combat and further establish the operational architecture combining the two aspects.

In this paper, both task allocation and cooperative control of UAVs are studied for air combat applications, which is established by a two-layer framework containing a task allocation upper layer and a cooperative control lower layer. First, in the task allocation upper layer, an evolutionary game-based task allocation strategy is proposed for UAVs in the sense of cooperation to allocate the tasks. Then, in the cooperative control lower layer, a cooperative control approach is proposed for a group of UAVs, which are allocated for the same task, to maintain a desired formation to accomplish the corresponding task. Main contributions of the paper are concluded as follows.

• A two-layer framework is established for the air combat of UAVs, which contains a task allocation upper layer and a cooperative control lower layer. The two-layer framework can also be extended and applied to a complex air combat with multiple operational layers directly.
• In the upper layer, the evolutionary task allocation method is proposed for UAVs to allocate the tasks by considering their cooperative behavior. In the lower layer, a cooperative control approach is proposed for UAVs to perform desired formations to accomplish the corresponding tasks.
• Both Software-in-the-Loop (SIL) and Model-in-the-Loop (MIL) simulations are conducted based on the RflySim platform to demonstrate the effectiveness of the proposed approach and further display the multi-UAV air combat vision realistically.

3. Problem Formulation

Considering the complexity of the air combat in practice, N friendly UAVs, which are called UAVs for short in the following text, work together to accomplish M tasks such as searching, tracking, circular navigation, attacking, pursuit, and evasion tasks of an enemy UAV. The model of UAV i, $i=1,2,\cdots$, N, can be expressed as a mass point, i.e.,

$$\begin{array}{c}\hfill {\dot{p}}_i\left(t\right)=v_i\left(t\right)\end{array}$$

(1)

where $p_i\left(t\right)\in \mathbf{R}$ is the position, $v_i\in \mathbf{R}$ is the velocity of UAV i. The model of the enemy UAV is also expressed as a mass point, i.e.,

$$\begin{array}{c}\hfill {\dot{p}}_e\left(t\right)=v_e\left(t\right),\end{array}$$

(2)

where $p_e\left(t\right)\in \mathbf{R}$ is the position, $v_e\in \mathbf{R}$ is the velocity of the enemy UAV.

The UAVs allocate the tasks by considering their cooperative behavior first. Then, the UAVs allocated for same tasks form the same groups. Finally, a cooperative formation control law is designed to drive UAVs in the same group to complete the task together. Therefore, the air combat structure can be established by a two-layer framework containing a task allocation upper layer and a cooperative control lower layer, which is shown in Figure 1.

In the task allocation upper layer, the UAVs, which are allocated for task r, form group r, $r=1,2,\cdots$, M. Then, the proportion of UAVs which performs task r is ${\delta }_r\left(t\right)=N_r\left(t\right)/n$, where $N_r\left(t\right)$ is the number of UAVs in group r. To avoid the malignant competition of allocating the limited operational resources, an evolutionary game is used to find an evolutionary stable state solution of each group and further maximize the overall combat effectiveness cooperatively. Therefore, the task allocation problem in the upper layer can be described as follows.

Problem 1 In the task allocation upper layer, design the utility of allocated task r, $r=1,2,\cdots$, M, as

$$\begin{array}{ccc}\hfill & & {\xi }_r\left(t\right)={\mathbf{f}}_r(P_r,C_r),\hfill \end{array}$$

(3)

where ${\mathbf{f}}_r(\ast )$ is the function, $P_r$ and $C_r$ are the total profit and cost of task r, respectively. If the utility function (3) can make the replicator dynamic function

$$\begin{array}{ccc}\hfill & & {\dot{\delta }}_r\left(t\right)={\delta }_r\left(t\right)({\xi }_r\left(t\right)-\overline{\xi }\left(t\right)),\hfill \end{array}$$

(4)

where $\overline{\xi }\left(t\right)$ is the average utility of UAVs in group r, achieve the evolutionary stable state, i.e., ${\dot{\delta }}_r\left(t\right)=0$, then the task allocation problem is solved by the evolutionary game.

In the cooperative control lower layer, the UAVs in group r cooperatively perform a desired formation to accomplish task r. According to the requirements of different tasks, the UAVs perform a tracking formation or a circular formation to accomplish the corresponding tasks. The tracking formation means that the UAVs keep desired distances from the enemy UAV and simultaneously maintain a desired spacing between two neighboring UAVs. Compared with the tracking formation, the circular formation has an additional requirement that the UASs also need to rotate around the enemy UAV. Therefore, the cooperative control problem in the lower layer can be described as follows.

Problem 2 In the cooperative control lower layer, considering the UAVs (1) in group r, $r=1,2,\cdots ,M$, design the following cooperative control law

$$\begin{array}{ccc}\hfill & & v_i\left(t\right)=\mathbf{u}(p_i\left(t\right)-p_j\left(t\right),p_i\left(t\right)-p_e\left(t\right),v_j\left(t\right),v_e\left(t\right)),i,j=1,2,\cdots ,N_r\left(t\right)\hfill \end{array}$$

(5)

where $\mathbf{u}(·)$ is a function, such that the UAVs move with the desired formations. Then, the cooperative control problem is solved.

4. Main Results

4.1. Evolutionary Game for Task Allocation

The evolutionary game is adopted for UAVs to allocate the tasks in the upper layer. Note that the UAVs communicate with each other to adjust their decisions. All UAVs allocate the tasks randomly at the beginning. If a UAV finds that a certain UAV has a higher utility than itself, then it has a probability to change its choice. Typically, UAV swarms perform two types of tasks in the air combat applications, i.e., reconnaissance task and attack task. In this work, we consider the two tasks with complementary functions. The total numbers of UAVs which perform the reconnaissance task 1 and the attack task 2 are expressed by $N_1\left(t\right)$ and $N_2\left(t\right)$, respectively. The UAVs, which are allocated for the same task, share the profit of performing the task equally, i.e., $P_r/N_r\left(t\right)$, $r=1,2$.

To solve Problem 1, the utility of each UAV can be calculated by a difference value between the allocated profit and its cost. According to (3), the utility of each UAV performs task r is

$$\begin{array}{ccc}\hfill & & {\xi }_r\left(t\right)={\lambda }_{1}log\left(\frac{\epsilon P_r}{N_r\left(t\right)}\right)-{\lambda }_2{C}_r,\hfill \end{array}$$

(6)

where ${\lambda }_1$ and $\epsilon$ are the predefined parameters, ${\lambda }_2$ is the equivalent benefit per unit cost contributing to the whole utility.

It is obvious that the total number of UAVs can be calculated as $N=N_r\left(t\right)+N_{r^{\prime }}\left(t\right)$, where $r,r^{\prime }=1,2$, $r\ne r^{\prime }$. The proportion of UAVs which performs task r is ${\delta }_r\left(t\right)$. Then, one obtains that ${\delta }_r\left(t\right)+{\delta }_{r^{\prime }}\left(t\right)=1$.

According to [31,32], the replicator dynamic function for the UAVs which perform task r is shown in (4), where the average utility of all UAVs can be calculated by

$$\begin{array}{cc}\hfill \overline{\xi }\left(t\right)& =(N_r\left(t\right){\xi }_r\left(t\right)+N_{r^{\prime }}\left(t\right){\xi }_{r^{\prime }}\left(t\right))/N\hfill \\ \hfill & ={\delta }_r{\xi }_r\left(t\right)+{\delta }_{r^{\prime }}\left(t\right){\xi }_{r^{\prime }}\left(t\right)\hfill \\ \hfill & ={\delta }_r{\xi }_r\left(t\right)+(1-{\delta }_r){\xi }_{r^{\prime }}\left(t\right)\hfill \end{array}$$

Based on the replicator dynamic function (4), the proportion of the UAVs evolves over time until an evolutionary equilibrium is achieved, i.e., ${\dot{\delta }}_r\left(t\right)=0$. It is indicated from ${\dot{\delta }}_r\left(t\right)=0$ that

$$\begin{array}{ccc}\hfill & & {\xi }_r\left(t\right)={\xi }_{r^{\prime }}\left(t\right).\hfill \end{array}$$

(7)

When the evolutionary equilibrium is achieved, the proportion of UAVs will not change because the utility of each UAV equals to the average utility of all UAVs. According to Equations (6) and (7), one has that

$$\begin{array}{ccc}\hfill & & {\lambda }_{1}log\left(\frac{\epsilon P_r}{N_r\left(t\right)}\right)-{\lambda }_2{C}_r={\lambda }_{1}log\left(\frac{\epsilon P_{r^{\prime }}}{N_{r^{\prime }}\left(t\right)}\right)-{\lambda }_2{C}_{r^{\prime }}.\hfill \end{array}$$

(8)

Due to ${\delta }_{r^{\prime }}\left(t\right)=1-{\delta }_r$, Equation (8) yields that

$$\begin{array}{ccc}\hfill & & {\lambda }_{1}log\left(\frac{\epsilon P_r}{{\delta }_r\left(t\right)N_r}\right)-{\lambda }_2{C}_r={\lambda }_{1}log\left(\frac{\epsilon P_{r^{\prime }}}{(1-{\delta }_r\left(t\right))N_r}\right)-{\lambda }_2{C}_{r^{\prime }}.\hfill \end{array}$$

(9)

The evolutionary equilibrium is calculated from Equation (9) as

$$\begin{array}{ccc}\hfill & & N_r^{*}=\frac{P_{r}N}{P_{r^{\prime }}·e^{\lambda (C_r-C_{r^{\prime }})}+P_r},\hfill \end{array}$$

(10)

where $\lambda ={\lambda }_2/{\lambda }_1$. Moreover, stability of the evolutionary equilibrium can be analyzed by evaluating a Jacobian matrix of the replicator dynamic function (4) as in [32].

The evolution process of UAVs is given below.

(1) For all UAVs, task r is randomly chosen (i.e., $r\in \{1,2\}$).

(2) loop

(3) UAV i computes its utility ${\xi }_r^i\left(t\right)$ from the allocated task profit and cost by using (6). The utility of UAV i is sent to a centralized controller. (${\xi }_r^i\left(t\right)$ is the utility of UAV i allocated task r.)

(4) The centralized controller computes average utility $\overline{\xi }\left(t\right)={\sum }_{r=1}^2{\sum }_{i=1}^{N_r}{\xi }_r^i\left(t\right)/n$ for the UAVs and broadcasts it back to the UAVs.

(5) if ${\xi }_r^i\left(t\right)<\overline{\xi }\left(t\right)$ then

(6) if $rand\left(\right)<(\overline{\xi }\left(t\right)-{\xi }_r^i\left(t\right))/\overline{\xi }\left(t\right)$ then

(7) UAV i is selected to perform task $r^{\prime }$, where $r^{\prime }\ne r$ and ${\xi }_{r^{\prime }}\left(t\right)>{\xi }_r\left(t\right)$.

(8) end if

(9) end if

(10) end loop for all UAVs

Remark 1.

According to the detailed evolution process, the allocation result will change when step (6) is satisfied. It means that when the UAV finds a higher utility than itself, that is, its utility is lower than the average utility, the allocation result has probability to change. The larger the difference between its own utility and the average utility, the higher is the probability that the allocation result will change.

4.2. Cooperative Control for Formation Achievement

In the cooperative control lower layer, after the task allocation, the UAVs form different groups and cooperatively perform desired formations to accomplish the corresponding tasks. As shown in Figure 1, both the tracking and circular formations are considered according to the requirements of the reconnaissance task 1 and attacking task 2, respectively.

The neighbor topology of the UAVs in group r is defined according to the positions of UAVs $p_1,p_2,\cdots ,p_{N_r}$ in the clockwise order, which is depicted in Figure 2.

As shown in Figure 2, the pre-neighbor UAV $i^{-}$ and the next-neighbor UAV $i^{+}$ of UAV i are the one previous and next to it, respectively. The neighbor topology for all UAVs in the same group can be described as an undirected ring graph. The set of UAV i’s two neighbors is denoted by ${\mathcal{N}}_i=\{i^{+},i^{-}\}$. Note that two UAVs will crash into each other if they are at the same place, which is not considered in this work. UAV i can sense the relative position information from itself to the enemy UAV and its neighboring UAVs. The sensing information of UAV i is also shown in Figure 2.

Note that the relative position between UAV i and the enemy UAV is expressed as

$$\begin{array}{c}\hfill z_i\left(t\right)=p_i\left(t\right)-p_e\left(t\right).\end{array}$$

(11)

The distance between each UAV and the enemy UAV is $D_i=\parallel z_i\left(t\right)\parallel$. Let ${\beta }_i\left(t\right)\in [0,2\pi )$, ${\beta }_i\left(t\right)=\arctan 2(z_i^y\left(t\right),z_i^x\left(t\right))$ be the angular of vector $z_i\left(t\right)={[z_i^x\left(t\right),z_i^y\left(t\right)]}^T$ in the polar coordinate. The angular spacing ${\tilde{\beta }}_i\left(t\right)$ can be calculated by

$$\begin{array}{c}\hfill {\tilde{\beta }}_i\left(t\right)={\beta }_{i^{+}}\left(t\right)-{\beta }_i\left(t\right)+ϵ\end{array}$$

(12)

with

$$\begin{array}{c}\hfill ϵ=\left\{\begin{array}{c}0,{\beta }_{i^{+}}\left(t\right)-{\beta }_i\left(t\right)\ge 0\\ 2\pi ,{\beta }_{i^{+}}\left(t\right)-{\beta }_i\left(t\right)<0\end{array}\right.\end{array}$$

The control objectives consist of two parts. The first objective is to let the UAVs maintain desired distances to the enemy UAV, i.e.,

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}{D}_i\left(t\right)={\overline{D}}_i\end{array}$$

The second objective is to let the UAVs maintain desired angular spacing to the neighboring UAVs, i.e.,

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}{\tilde{\beta }}_i\left(t\right)={\overline{\beta }}_i\end{array}$$

To solve Problem 2, design the following controller for UAV i, i.e.,

$$\begin{array}{c}\hfill v_i\left(t\right)=v_{i1}\left(t\right)+v_{i2}\left(t\right)+v_e\left(t\right)\end{array}$$

(13)

In controller (13), an orthogonal pair ${\mathbf{b}}_i\left(t\right)={[cos{\beta }_i\left(t\right),sin{\beta }_i\left(t\right)]}^T$ and ${\mathbf{c}}_i\left(t\right)=[-sin{\beta }_i\left(t\right),$$cos{\beta }_i{\left(t\right)]}^T$ is adopted to simultaneously achieve the distance and spacing objectives of the formation as in [33].

The first term is designed to let UAV i eventually attain the desired distance ${\overline{D}}_i$ from itself to the enemy UAV, i.e.,

$$\begin{array}{c}\hfill v_{i1}\left(t\right)=-{\kappa }_1{D}_i\left(t\right)(D_i\left(t\right)-{\overline{D}}_i){\mathbf{b}}_i\left(t\right)\end{array}$$

(14)

The second term is designed to achieve the spacing adjustment objective ${\overline{\beta }}_i$ for UAV i by adjusting angular distances among their neighbors, i.e.,

$$\begin{array}{c}\hfill v_{i2}\left(t\right)={\kappa }_2{D}_i\left(t\right)z_i\left(t\right){\mathbf{c}}_i\left(t\right)\end{array}$$

(15)

with

$$\begin{array}{c}\hfill z_i\left(t\right)=\frac{{\overline{\beta }}_{i^{-}}}{{\overline{\beta }}_i+{\overline{\beta }}_{i^{-}}}{\tilde{\beta }}_i\left(t\right)-\frac{{\overline{\beta }}_i}{{\overline{\beta }}_i+{\overline{\beta }}_{i^{-}}}{\tilde{\beta }}_{i^{-}}\left(t\right)+\phi ,\end{array}$$

${\kappa }_1$ and ${\kappa }_2$ are positive constants, $\phi$ is a nonnegative constant.

The last term is designed to compensate for the velocity of enemy UAV.

The UAVs perform a tracking formation when $\phi =0$, which represents that the UAVs are tracking the enemy UAV. The UAVs perform a circular formation when $\phi \ne 0$, which represents that the UAVs are tracking the enemy UAV and simultaneously circling around the enemy UAV with circular velocity $\phi$. The stability of the closed-loop system can be analyzed by the same method as in [33].

Remark 2.

In the cooperative lower layer, by onboard sensors, the UAVs calculate the control inputs through measuring the relative positions to the enemy UAV and to the neighboring UAVs. There is no further communication in the cooperative lower layer, which partly saves computational time. In the task allocation upper layer, there are some extended evolutionary task allocation methods such as the Markov decision evolutionary game-based learning algorithm [20] and the stochastic-tuning-based evolutionary algorithm [22] considering more complex combat environments and the efficiency of the task allocation problem, which can be further studied. The combined consideration of the two layers establishes an extendable ground framework and is applied to a complex air combat. The computational time is beyond the scope of this context and can be studied in the future.

Remark 3.

Some excellent works consider the cooperative task assignment problems [11] and the path planning problems [9], and both of them [12]. In this work, we design a two-layer framework framework from the perspective of control as [27]. The evolutionary task allocation method is proposed for UAVs to allocate the tasks by considering their cooperative behavior. In addition, the cooperative control approach is proposed for UAVs to perform different desired formations to accomplish the corresponding tasks. The combination of the approaches provides a two-layer framework which characterizes real scenarios and is applicable to the air combat of UAVs.

Remark 4.

The two-layer framework in Figure 1 can also be extended and applied to a complex air combat with multiple operational layers directly. Note that UAVs in the same group accomplish the same task after the task allocation. If the task requires to be divided into several subtasks due to the complexity of the air combat, then we can extend the same task allocation strategy for the UAVs directly. In addition, our proposed cooperative control law (13) is also applicable for the UAVs to accomplish the corresponding subtasks. We take the circular formation as an example. If we give different desired distances between UAVs and the enemy UAV, the UAVs moving along different circular orbits can accomplish the corresponding different subtasks.

5. Simulation

Note that the RflySim platform is based on Pixhawk/PX4, MATLAB/Simulink and Python for UAV education and research. In order to realistically show multi-UAV air combat vision, we conduct some SIL and MIL simulations based on the RflySim platform. We work with the RflySim platform on Windows 10, Intel(R) Core i5, CPU 1.60 GHz 2.11 GHz, NVIDIA GetForce MX250.

Simulation I We conduct an SIL simulation through MATLAB/Simulink PX4 for UAVs. There are 4 UAVs working together to achieve reconnaissance task 1 and attack task 2.

In the task allocation upper layer, the profits and the costs of the two tasks are given by $P_1=1$, $P_2=3$, $C_1=1$, and $C_2=1.6$, respectively. Other parameters are given as ${\lambda }_1=3$, ${\lambda }_2=0.5$ and $\epsilon =1$. The initial properties of allocated tasks 1 and 2 are randomly set as ${\delta }_1=0.6$ and ${\delta }_2=0.4$, respectively. Based on the utility function (6) and replicator dynamic function (4), evolutions of the UAVs’ decision are depicted in Figure 3. It can be seen from Figure 3 that the properties of allocated tasks 1 and 2 converge to ${\delta }_1=0.2571$ and ${\delta }_2=0.7429$ finally. It indicates that there is $N_1\approx 1$ UAV performing the reconnaissance task 1. In addition, there are $N_2\approx 3$ UAVs performing the attack task 2. Then, the diagram of UAVs can be depicted in Figure 4 intuitively.

In the cooperative control lower layer, the UAVs adopt the cooperative control law (13) with (14) and (15) to perform a desired formation to accomplish the task. The initial positions of all the UAVs in both QGroundControl and Rflysim 3D are shown in Figure 5. The desired distance and angular spacing are ${\overline{D}}_i=10$ and ${\overline{\beta }}_i=2\pi /3$, $i=2,3,4$, respectively.

The SIL simulation results through MATLAB/Simulink PX4 are shown in Figure 6, Figure 7 and Figure 8. In Figure 6, the circular formation trajectories of UAVs are displayed by QGroundControl and RflySim 3D. To clearly show the effectiveness of the cooperative control law, we draw the trajectories, distance between each attack UAV and reconnaissance UAV $D_i\left(t\right)$, the angular spacing between two attacking UAVs ${\tilde{\beta }}_i$ in Figure 7 and Figure 8, respectively. The distance between the reconnaissance UAV and each attack UAV converges to the desired one. The angular spacing between two neighboring attack UAVs also converges to the desired one.

Simulation II We conduct an MIL simulation for a simplified UAV model through Python to show the air combat vision due to hardware limitations. Consider $N=10$ UAVs working together to achieve reconnaissance task 1 and attack task 2.

In the task allocation upper layer, the profits and the costs of the two tasks are given by $P_1=4$, $P_2=3$, $C_1=1$, $C_2=1.6$. Other parameters are the same as in Simulation I. Evolutions of the UAVs’ decision are depicted in Figure 9. It can be seen from Figure 9 that the properties of allocated tasks 1 and 2 converge to ${\delta }_1=0.5885$, ${\delta }_2=0.4114$. It indicates that there are $N_1\approx 6$ UAVs performing the reconnaissance task 1, $N_2\approx 4$ UAVs performing the attack task 2. Then, the diagram of UAVs can be depicted in Figure 10 intuitively.

In the cooperative control lower layer, UAVs 1–6 perform a circular formation and UAVs 7–10 perform a tracking formation to accomplish the reconnaissance task 1 and attack task 2, respectively. The desired distance and angular spacing are ${\overline{D}}_i=2$, $i=1,2,\cdots$, 4, ${\overline{D}}_j=3$, $j=5,6,\cdots$, 10 and ${\overline{\beta }}_k=\pi /5$, $k=1,2,\cdots$, 10, respectively. The MIL simulation result of the simplified UAV model through python is shown in Figure 11. As noted in Remark 4, for the circular formation, the UAVs in different orbits can perform different subtasks of the attacking task.

Simulation III To show the robustness of the proposed strategy, we conduct another MIL simulation in the air combat vision when the enemy UAVs are moving in an irregular curve. The corresponding strategies in the two layers are the same as those in Simulation II. The trajectories of UAVs are shown in Figure 12.

6. Discussion

In this paper, both task allocation and cooperative control of UAVs have been studied and established by a two-layer framework for air combat applications. In the task allocation upper layer, a task allocation strategy has been given by evolutionary theory for UAVs to allocate different types of tasks. In the cooperative control layer, a cooperative control approach has been proposed, by which the UAVs can perform a tracking formation and a circular formation to accomplish the corresponding reconnaissance task and attacking task, respectively. Finally, SIL simulation results through MATLAB/Simulink PX4 show that the evolutionary stable state and the stabilization control can be achieved for 4 UAVs. In addition, MIL simulation results realistically display the multi-UAV air combat vision for 10 UAVs, where 4 UAVs are tracking one enemy UAV to accomplish the reconnaissance task and 6 UAVs rotate around the other enemy UAV to accomplish the attack task. In the future, we will display the multi-UAV air combat vision by SIL simulation through MATLAB/Simulink PX4 and Pixhawk.