Co-Evolutionary Algorithm-Based Multi-Unmanned Aerial Vehicle Cooperative Path Planning

Abstract

Multi-UAV cooperative path planning is a key technology to carry out multi-UAV tasks, and its research has important practical significance. A multi-UAV cooperative path is a combination of single-UAV paths, so the idea of problem decomposition is effective to deal with multi-UAV cooperative path planning. With this analysis, a multi-UAV cooperative path planning algorithm based on co-evolution optimization was proposed in this paper. Firstly, by analyzing the meaning of multi-UAV cooperative flight, the optimization model of multi-UAV cooperative path planning was given. Secondly, we designed the cost function of multiple UAVs with the penalty function method to deal with multiple constraints and designed two information-sharing strategies to deal with the combination path search between multiple UAVs. The two information-sharing strategies were called the optimal individual selection strategy and the mixed selection strategy. The new cooperative path planning algorithm was presented by combining the above designation and co-evolution algorithm. Finally, the proposed algorithm is applied to a rendezvous task in complex environments and compared with two evolutionary algorithms. The experimental results show that the proposed algorithm can effectively cope with the multi-UAV cooperative path planning problem in complex environments.

1. Introduction

In both civilian fields and military fields, there is increasing attention on the application of unmanned aerial vehicles (UAVs), and the application scenarios of UAVs are also increasing. However, in many cases, it is difficult for a single UAV to complete a certain task, and multiple UAVs are required to work together. For example, in the combat tasks of positioning, interference, and attack, multiple UAVs need to cooperate with each other to complete the task. In search and rescue missions, multiple UAVs need to cooperate to complete searching, positioning, and transportation.

With the increasing demand for multi-UAV cooperative applications, cooperative path planning technology has gradually developed [1,2]. Compared with path planning for a single UAV, multi-UAV cooperative path planning is much more complex [3]. The first problem facing multi-UAV cooperative path planning is the dramatically large search space, which grows exponentially with the number of aircraft. The path of a single aircraft is represented as $Path=(p_1,L,p_l)$, and $p_i(i=1,\cdots ,l)$ is a path point. If the search space of a single path is $O(a)$, then the search space of N aircrafts is $O(a^N)$, which poses a huge challenge to finding the algorithm’s solution. The second problem faced by multi-UAV cooperative path planning is to seek the least costly path while maintaining synergy between the paths. Path-to-path coordination means that aircraft cannot collide during flight, and agents must reach the target point at the same time or in sequence, as required. Therefore, spatial and temporal cooperative relationships between each UAV path should be maintained at the same time, which increases the number of constraints and brings new difficulties to finding a solution for the algorithm.

To deal with the problem of the sharp increase in search space, we use the co-evolution algorithm [4]. (Here, co-evolution algorithm coordination refers to the multiple sub-populations in the algorithm that need to share information to find a path, and path coordination means that the aircraft cannot collide during flight and they reach the target point at the specified time, which is different). The co-evolution algorithm decomposes complex problems into several sub-problems and searches them separately by various populations, realizes the evaluation through the exchange of information between the populations, and then guides the algorithm to search for the optimal solution [5,6,7,8,9]. A complete solution is a multi-UAV path planning task in a combination of all aircraft paths, which can be regarded as a sub-problem for single-UAV path planning, and multi-UAV cooperative path planning is a combination of these sub-problems. The complexity can be reduced by decomposing the problem into sub-problems and then combining them, so the application of co-evolutionary algorithms to solve the multi-UAV cooperative path planning problem is an effective method. Most of the existing research methods have adopted the idea of co-evolution to deal with multi-machine cooperative path planning [10,11,12].

For the coordination relationship of the multi-UAV path, temporal and spatial cooperative relationships between UAVs are the main considerations, and are added as constraints to the optimization model of multi-UAV cooperative path planning. A spatial cooperative relationship is reflected in the fact that the distance between each aircraft should be greater than a safe distance to prevent collision [1]. The time coordination is reflected in the time constraint to reach the target point. To deal with the time cooperative relationship between multiple UAVs, we can adjust the path length and speed allocation. Speed allocation is to assign different flight speeds to UAVs with different path lengths, to maximize the possibility of meeting the time constraint for multiple UAVs to reach the target point. Adjusting the path length can be categorized into two methods: the additional maneuver method and the path cooperative planning method [13]. The additional maneuvering method means that some UAVs with short path lengths adopt strategies such as flying around [14] and circling [15] to lengthen their paths to ensure that the path lengths between multiple UAVs are approximately the same. In this paper, the path length is adjusted in the second way, which considers the length of the direct path to meet the requirements of time coordination with a fixed aircraft speed in the algorithm solution.

For the solution of multi-UAV cooperative path planning problems, different research methods will be produced according to different missions [1,16], and this paper mainly discusses the flight mission in which all aircraft reach the same goal. Researchers have proposed many methods for this type of problem. Lu Jiangsong solved the multi-machine cooperative path planning problem based on multi-ant colony cooperative search by introducing the co-evolution mechanism into the ant colony algorithm [17]. The idea of co-evolution is used in the algorithm, but the algorithm reduces the dimension of the three-dimensional space path planning to two-dimensional path planning by simplifying the processing. Huang [18] applied the ant colony algorithm and k-degree smoothing method to generate multi-machine path curves while considering time coordination. Sun [19] considered the multi-machine cooperative path planning of spatial coordination and solved it by using the artificial potential field method. Shao [20] propose a distributed particle swarm optimization algorithm to solve the multi-machine cooperative path planning problem. Zhang [21] used the pigeon herd algorithm to solve the multi-aircraft cooperative path planning problem, considering the communication range, spatial cooperative constraints, and time cooperative constraints of the aircraft, but did not adopt the idea of cooperative search. Ma Peijun [22] studied the multi-machine cooperative path problem based on the sparse A* algorithm, by imposing collision avoidance constraints in the process of node expansion to achieve path space cooperative constraints without discussing time cooperative constraints. The paper realizes collision avoidance constraints by ordering the paths of each machine, for which is difficult to find the optimal solution.

Although many methods have been proposed [23,24,25,26,27], there is still much work to be performed to reduce the amount of computation and improve efficiency. In this paper, a multi-UAV cooperative path planning algorithm based on a co-evolution algorithm (CCEA-CPP) is proposed. The algorithm uses the penalty function to deal with various constraints of the aircraft, including cooperative constraints. Representative individuals of the sub-population are used to share information, and the cooperative search and combination solution is used to reduce the complexity of the algorithm and improve the efficiency of multi-UAV path search. The proposed algorithm is applied to a rendezvous task in complex environments and compared with two evolutionary algorithms. The results show that the performance of the proposed algorithm is effective.

The main contributions of our work are as follows:

(1)

Considering the collaborative relationship between multiple unmanned aerial vehicles, the constrained optimization model is established for the multi-UAV collaborative path planning problem.

(2)

A co-evolutionary optimization framework is provided to tackle the multi-UAV cooperative path planning problem. The co-evolutionary algorithm reduces the complexity of the multi-UAV collaborative path planning problem through decomposition.

(3)

Two information-sharing strategies are designed with the global optimal and local optimal information. The two strategies guide the search direction of the population to the potential optimal solutions with the purpose of quickly finding the optimum solution and improving the efficiency of the proposed algorithm.

The arrangement of this article is as follows: in Section 2, we establish a multi-constraint objective optimization model for collaborative path planning of multiple unmanned aerial vehicles and analyze the objective function and flight constraints in the model; in Section 3, we design a multi-machine collaborative path planning algorithm based on collaborative evolution, and give the algorithm framework and detailed steps; in Section 4, we evaluate the performance of the algorithm through simulation experiments; the conclusion and outlook of this article are presented in Section 5.

2. Cooperative Path Planning Problem Analysis and Modeling

The multi-UAV cooperative path planning problem can be regarded as a constraint optimization problem, which minimizes the path length cost and threat cost of the multi-UAV path under the premise of satisfying various flight constraints of aircraft and cooperative constraints between aircraft. So, it can be modeled [1] as follows:

$$\min f(x) s.t.\left\{\begin{array}{c}{g}_i(x)\le 0,i=1,2,\cdots ,n\\ h_j(x)=0,j=1,2,\cdots ,p\end{array}\right.$$

(1)

where $f(x)$ is the optimized objective function, which refers to path length and threat cost; $g(x)$ and $h(x)$ are, respectively, the inequality constraints and the equality constraints of the problem, which refer to flight constraints and coordination constraints. For the multi-UAV cooperative path planning problem, the form of the solution can be expressed as $x=(pat{h}_1,pat{h}_2,\cdots ,pat{h}_N)$, where $pat{h}_i$ is the path of the ith UAV and $N$ is the number of UAVs.

2.1. Objective Function

We first give the single-UAV path cost, and then give the multi-UAV path cost, which is the optimized objective function. The path cost of a single UAV will take different forms depending on the focus of the problem under consideration [1,23]. This paper mainly considers two important factors, path length cost and path threat cost, so the single-UAV path cost expression is constructed as follows:

$$C_i={\displaystyle \sum _{j=1}^n(lengt{h}_{ij}+threa{t}_{ij})} (i=1,\cdots ,N)$$

(2)

where $C_i(i=1,\cdots ,N)$ is the single-UAV path cost of the ith UAV; n indicates that the planning path is composed of n path segments connected; $lengt{h}_{ij}$ is the length of the jth segment path in the planning path $pat{h}_i$; $threa{t}_{ij}$ is the cost of enemy threats to the jth segment of the planned path. The calculation of the threat cost adopts a more general calculation method, which will not be repeated here. Please refer to references [1,23,25].

The optimization objective function for multi-UAV cooperative path planning problems can be expressed as follows:

$$f(x)={\displaystyle \sum _{i=1}^N{C}_i}$$

(3)

where $C_i$ is the single-UAV path cost of the ith UAV and $N$ is the number of UAVs.

2.2. Constraints

A.

UAV flight constraints

As an object moving at high velocity in three-dimensional space, the constraints faced by UAVs mainly come from the physical characteristic constraints and task requirements. This article mainly considers the following types of UAV flight constraints:

(1)

Minimum turning radius constraint:

$$R_i>R_{min}(i=1,2,\cdots ,n)$$

(4)

where $R_i$ is the turning radius of the planned path from the ith segment to the i+1 segment; $R_{min}$ is the minimum turning radius of a UAV.

(2)

Minimum path segment length constraint:

$$dS>l_{\min }$$

(5)

where $dS$ is the unit step size for path planning; $l_{\min }$ is the minimum path segment length.

(3)

Maximum path slope angle constraint

$$\frac{{\mathrm{b}}_i}{\left|a_i\right|}\le \tan {\theta }_{\max }(i=1,2,\cdots ,n)$$

(6)

where $a_i={(x_i-x_{i-1},y_i-y_{i-1})}^T$ is the horizontal projection vector of the ith segment path; $\left|a_i\right|$ is the module for $a_i$; $b_i=\left|z_i-z_{i-1}\right|$ is the vertical projection vector of the ith segment path; ${\theta }_{\max }$ is the maximum path slope angle.

(4)

Minimum ground clearance constraint:

$${\mathrm{z}}_i\ge H_{\min }(i=1,2,\cdots ,n)$$

(7)

where ${\mathrm{z}}_i$ is the height value of the ith planning point; $H_{\min }$ is the minimum ground clearance.

B.

UAV cooperative constraints

Multi-UAV paths’ cooperative relationship refers to the inability of UAV to collide during flight, and the UAVs must reach the target point simultaneously or in sequence as required. Therefore, they are added as constraints to the optimization model.

The time cooperative relationship can be modeled as an inequality constraint as follows:

$$\left|T_1\cap T_2\cap \cdots \cap T_N\right|>0$$

(8)

where $T_i$ is the estimated arrival time window of the ith UAV. Multi-UAV time coordination means that the estimated arrival time window of multi-UAV cannot have an empty set.

The spatial cooperative relationship between multi-UAV can be modeled as the following equation constraints:

$$collision\_check(pat{h}_1,pat{h}_2,\cdots ,pat{h}_N)=0$$

(9)

where $collision\_check()$ is the path collision detection function. Multi-UAV spatial cooperative relationship means that there cannot be cross collisions between the paths of multi-UAV.

3. Co-Evolutionary Algorithm-Based Multi-UAV Cooperative Path Planning Algorithm

Multi-UAV cooperative path planning is a complex constrained optimization problem with huge search space. The co-evolutionary algorithm reduces the complexity of multi-UAV path planning by decomposing it into single UAV path planning, and then combining the ideas to solve it. It is an effective method, and many researchers have also adopted this idea. In this paper, the CCEA-CPP algorithm is proposed based on the co-evolutionary algorithm. In the algorithm, we have improved the processing of constraints, evaluation of fitness, and communication methods to complete the cooperative path planning of UAVs.

3.1. Overall Algorithm Framework Design

Co-evolutionary algorithms deal with large-scale and complex optimization problems. The basic idea is to use a distributed computing framework to build a multi-population parallel evolution, and to achieve cooperative evolution through information interaction between populations [23]. Since the sub-population evolution generally adopts mature specific evolutionary algorithms, co-evolution focuses more on the type of information interaction between populations and the design of exchange mode.

The framework of the co-evolutionary algorithm-based multi-UAV cooperative path planning algorithm (CCEA-CPP) is shown in Figure 1. In the algorithm, each sub-population is responsible for planning the path of a single UAV. Before evaluating the comprehensive cost of an individual, the representative individual and the current population are combined from other populations to form a complete multi-UAV cooperative path. Afterward, the evaluation individual selects the best solution as the individual of the next generation sub-population. The selection of representative individuals from other populations involves the way information is exchanged, and the evaluation of individual comprehensive costs involves the processing of constraint functions. How to exchange information and how to evaluate individuals is the key to improving algorithmic efficiency. These two parts are described in detail below.

3.2. Constraint Processing and Comprehensive Cost of Cooperative Paths

Multi-UAV cooperative path planning is a constrained optimization problem with many constraints. The penalty function method is an effective method for dealing with constraint problems. The main idea of the penalty function method is to use the constraint function in the process of solving constrained optimization problems and add a penalty term to the objective function as a new objective function to guide the algorithm search, transforming the original constrained optimization problem into an unconstrained optimization problem.

In summary, for a multi-UAV cooperative path planning problem, a constraint violation degree function corresponding to flight constraints, time and space cooperative constraints is constructed. Then, the constraint violation degree function is added as a penalty term to the objective function to form a new objective function as follows:

$$C_{coo}={\displaystyle \sum _{i=1}^N{C}_i}+{\omega }_p\cdot (vi{o}_{time}+vi{o}_{space})+{\omega }_c\cdot vi{o}_{con}$$

(10)

where $C_{coo}$ is the new objective function used to represent the comprehensive cost of multi-UAV cooperative paths for the entire UAV fleet; $C_i$ is the single-UAV path cost of the ith UAV; $vi{o}_{time}$ is the time cooperative violation in the UAV group; $vi{o}_{space}$ is the spatial cooperative violation in the UAV group; $vi{o}_{con}$ is the flight constraint violation for UAV path planning; ${\omega }_p$ is the penalty factor, which is used to punish solutions that do not meet the time and spatial cooperative constraints, representing the penalty system for flight constraint violations.

(1)

Time cooperative violation

Time cooperative constraint refers to the ability of each UAV to depart from different starting points simultaneously, fly along a planned cooperative path, and arrive at the target point on time. For the ith UAV, if its planned path length is $L_i$, it can be assigned different flight speeds through speed adjustment to reach the target point at any time within the $\left[L_i/V_{max},L_i/V_{min}\right]$ time interval.

In order to ensure the time coordination of the UAV group during task execution, speed adjustment can be used to ensure that each UAV reaches the target airspace simultaneously. However, the premise is that the cooperative path length of each UAV is not significantly different, and it can meet the expected arrival time window of each UAV at the intersection.

For a UAV group composed of $N$ UAVs, the time cooperative violation can be constructed by calculating the length difference between the corresponding paths of UAVs and the sum of absolute values to measure the overall time synergy of the UAV group. Based on the above analysis, the construction time cooperative violation function is as follows:

$$vi{o}_{time}={\displaystyle \sum _{i=1}^{N-1}{\displaystyle \sum _{j=i+1}^N\left|L_i-L_j\right|}}$$

(11)

where $vi{o}_{time}$ is the time cooperative violation in the UAV group, which is the absolute sum of the length differences between different paths in the UAV group; $L_i$ is the length of the planned path for the ith UAV.

(2)

Spatial cooperative violation

Spatial cooperative relationship refers to the fact that the flight distance between any two UAVs in a group is always greater than the safe distance to prevent collisions during flight. For a group of UAVs, a spatial cooperative violation can be constructed by detecting path point collisions in pairs corresponding to the UAVs, and recording the total number of collisions to measure the satisfaction of the cooperative path solution with spatial cooperative constraints. Based on the above analysis, the construction of the spatial cooperative violation function is as follows:

$$vi{o}_{space}={\displaystyle \sum _{i=1}^{N-1}{\displaystyle \sum _{j=i+1}^{N}nu{m}_{ij}}}$$

(12)

where $vi{o}_{space}$ is the spatial cooperative violation of cooperative path for the UAV group; $nu{m}_{ij}$ is the number of collisions between the ith UAV path and the jth UAV path generated by path point collision detection.

(3)

UAV flight constraint violation

The minimum path segment length constraint, maximum path slope angle constraint, and minimum ground clearance constraint in the flight constraints of UAVs have been considered during the construction of the map, so the path constructed on this map must meet these constraints. The turning radius of a UAV can be approximated by the angle formed by the connection of the path segments. The violation of UAV flight constraints mainly considers the minimum turning radius constraint of UAVs. When the angle formed by the connection of the path segments in the current moment is less than the limit caused by the minimum turning radius constraint, punishment is imposed. $vi{o}_{con}$ represents the flight constraint violation of the drone’s planned path, defined as follows:

$$vi{o}_{con}={\displaystyle \sum _{i=1}^N{n}_i}$$

(13)

where $n_i$ represents the number of times the trajectory of the ith UAV violated the minimum turning radius constraint.

3.3. Information Sharing Methods Design

In the cooperative co-evolutionary algorithm, the main problem is decomposed into multiple subproblems, each sub-population is responsible for solving only one subproblem, and the evolutionary individuals in the sub-population only correspond to partial solutions of the problem. Therefore, for the evaluation of the evolutionary individuals in the sub-population, it is necessary to select the representative individuals in other sub-populations to combine and build the complete solution of the original problem, and then evaluate the evolutionary individuals in the sub-population based on the complete solution. The combination of complete solutions of sub-populations is shown in Figure 2.

So, how do we choose representative individuals of other sub-populations? The paper provides two selection methods.

(1)

Best individual selection method (recorded as Bs-): Select the individual with the best overall cost ($C_{coo}$) for each sub-population as the representative individual for information exchange, which has the advantage of retaining the optimal combination.

(2)

Mass selection method (recorded as Hs-): Select the individual with the best comprehensive cost ($C_{coo}$) for each sub-population and the individual with the best single machine path cost ($C_i$) for each sub-population as representative individuals. Compared with Bs -, the selection of representative individuals has been added to enhance the search for the smallest neighborhood of a single UAV path.

The different methods of individual selection determine the different ways of information sharing and also affect the search range of the combined solution, which is a committed step of co-evolution. This paper tries two different selection methods: the optimal individual selection method searches along the neighborhood with the lowest cost of the comprehensive path, and the hybrid selection method searches along the neighborhood with the lowest cost of the comprehensive path and only considers the smallest cost of the single path without considering constraints, which hopefully will accelerate the convergence of the algorithm and improve its performance.

3.4. Co-Evolutionary Algorithm-Based Multi-UAV Cooperative Path Planning Algorithm (CCEA-CPP)

Combining the above design with the co-evolutionary algorithm, we present a co-evolutionary algorithm-based multi-UAV cooperative path planning algorithm (CCEA-CPP).

The following is the pseudo-code of the algorithm (Algorithm 1):

Algorithm 1. Pseudo-code of co-evolutionary algorithm-based multi-UAV cooperative path planning algorithm (CCEA-CPP).

Begin          $t=0;$          For each path planning subproblem $i;$          $Po{p}_i(t)=Generate\_initial\_subPop(D_i);$//Generate initial sub-populations of paths for each UAV          $b_i=Generate\_initial\_representative(D_i);$          End          $B=(b_1,\cdots ,b_N)$;//individual          While termination criteria not satisfied//If the termination condition is not met, loop continuously          For each path planning subproblem          $Q_i(t)=Recombin(Po{p}_i(t));$          $Q_i(t)=Mutation(Q_i(t));$          $C{Q}_i(t)=Get\_combination(Q_i(t),B);$//Obtaining combinatorial solutions          $F_i(t)=Evaluate(C{Q}_i(t));$//According to Equations (10)–(13), calculate the constraint violation and comprehensive cost          $Po{p}_i(t+1)=Select(Po{p}_i(t)\cup Q_i(t),F_i(t));$          //Selecting the next-generation population          $B=Update(B,Po{p}_i(t+1));$ //select representative individuals with Bs- or Hs- methods and update B          End          $t=t+1;$          End          End

While representing individual $B$ updates in the algorithm, the optimal path is also preserved. The algorithm CCEA-CPP updated with Bs- method is denoted as Bs-CCEA-CPP, while the algorithm CCEA-CPP updated with Hs- method is denoted as Hs-CCEA-CPP. The other operators in the algorithm, such as crossover, mutation, etc., are consistent with the path planning of a single UAV, which will not be repeated here [28].

The co-evolutionary algorithm-based multi-UAV cooperative path planning algorithm decomposes multi-UAV path planning into single UAV path planning, and then searches multi-UAV paths through path information sharing. This search method is conducive to preserving the excellent path combinations obtained, avoiding the destruction of the excellent parts of the entire multi-UAV path by crossing and mutation, reducing the complexity of the problem, and improving search efficiency.

4. Simulation Studies

The proposed algorithm, CCEA-CPP, is tested in a battlefield threat environment and compared with two evolutionary algorithms.

4.1. Environmental and Parametric

To make the simulation environment based on path planning closer to the real environment, this paper selected a 500 × 500 km digital elevation model as the environmental model for path planning. The size of the digital elevation model is 1000 × 1000 and data points exist. Based on the environmental model, we set various threats to simulate the real battlefield environment. The parameters of the environment are shown in

Table 1. Parameters related to various threats.

Type of Threat		Attributes
	Radar Number	Coordinates	Radius
Air defense	radar 0	(300, 400, 20)	80 km
radar	radar 1	(700, 600, 25)	70 km
Surface-to-air	Missile Number	Coordinates	Radius
missile	missile 0	(400, 750, 20)	60 km
Anti-aircraft	Artillery Number	Coordinates	Radius
artillery	artillery 0	(600, 180, 25)	30 km
	artillery 1	(750, 320, 20)	30 km
No-fly zone	p1 = (520, 320);	Vertex Coordinates	p2 = (620, 340);
	p3 = (600, 430);		p4 = (520, 430);

. The battlefield threat environment spatial model is shown in Figure 3.

In Figure 3, the two irregular surfaces are air defense radars; the upper right part of the figure is surface-to-air missile; the lower two figures are anti-aircraft artillery, and the diamond area is a No-fly zone. The positions of these threats or obstacles are randomly generated, so this environment can be extended to other forms of environment. In this paper, we focus on the rendezvous task. Various parameter settings in the algorithm are shown in

Table 2. The settings of various parameters in the algorithm.

Parameter	Value
Minimum path segment length constraints	50
Maximum path slope angle constraint	10
Minimum ground clearance constraint	10
Minimum turning radius constraint	85
Penalty factor ${\omega }_c$	0.2
Penalty factor ${\omega }_p$	1

.

4.2. Algorithm Analysis

(1)

Analysis of simulation results for five-UAV cooperative path planning

The results of the five-UAV cooperative path planning for the above scenario are shown in Figure 4.

In Figure 4, the paths of UAV D0, UAV D1, UAV D2, UAV D3, and UAV D4 are shown from left to right. The paths planned by the Bs-CCEA-CPP and Hs-CCEA-CPP algorithms for UAV D0, UAV D1, and UAV D2 all meet the requirements of spatial coordination while crossing narrow safety zones near the threat area to reach the target point. The paths of UAV D3 and UAV D4 both undergo a certain degree of detour to maintain temporal coordination with other UAVs while meeting the requirements of spatial coordination. The paths of both algorithms can converge to the vicinity of the optimal solution, finding a path set that ensures time and spatial cooperative relationship between UAVs for the UAV group.

The convergence curve of the five-UAV path comprehensive costs corresponding to path planning in the above scenario is shown in Figure 5.

From Figure 5, it can be seen that as the algebra increases, the comprehensive cost of the paths planned by the Bs-CCEA-CPP and Hs-CCEA-CPP algorithms gradually decrease, and eventually stabilize and converge to the minimum value.

Time-coordinated breach variation curves corresponding to the path planning of the above scenario are shown in Figure 6.

From Figure 6, it can be seen that as the algebra increases, the time-coordinated breach between the planned paths of the two algorithms decreases and eventually converges to a stable value. The time-coordinated breach is the absolute sum of the length differences between different paths in the UAV group. From the experimental results, it can be seen that although the cooperative violation has not reached zero, the length difference between different paths is already very small, which can meet the non-empty time window, which is a combination track that meets time-coordinated constraint and can be found. The Hs-CCEA-CPP algorithm can find track sets with smaller time violations faster than the Bs-CCEA-CPP algorithm.

In the path planning of the above scenario, the path length, cooperative arrival time window, and spatial cooperative relationship of the five UAVs are described in

Table 3. Table of coordinated arrival parameters of the five UAVs.

Algorithm Type	Flight Track Length/km	Cooperative Arrival Time Window/h	Spatial Cooperative Relationship
Hs-CCEA-CPP	D0: 499.18 D1: 499.09 D2: 479.18 D3: 479.81 D4: 484.20	(1.597, 2.496)	meet
Hs-CCEA-CPP	D0: 501.14 D1: 501.01 D2: 492.15 D3: 498.88 D4: 490.20	(1.593, 2.451)	meet

.

From Table 3, it can be seen that the planned path lengths of the five UAVs planned by the two algorithms are the same, and the cooperative arrival time window can be obtained through speed allocation. At the same time, the path of the five UAVs meets the requirements of a spatial cooperative relationship.

(2)

Simulation verification of algorithm convergence stability

The CCEA-CPP algorithm independently runs 30 times in five-UAV cooperative penetration scenarios for the problem of cooperative path planning. Then, the optimal trajectory comprehensive cost for each planning process is recorded for statistical analysis, which is shown in

Table 4. Convergence stability analysis table of the convergence numerical analysis algorithm for the cost of the path after 30 runs.

Algorithm Type	Maximum Value	Minimum Value	Average Value	Variance
Bs-CCEA-CPP	21.6830	5.1230	11.2546	14.2146
Hs-CCEA-CPP	12.5050	3.5160	5.3216	10.4141

.

From Table 4, it can be seen that for the scenario of five-UAV cooperative penetration tasks, the final convergence values of the two algorithms are mostly near the optimal or sub-optimal solution. Compared to the Bs-CCEA-CPP algorithm, the Hs-CCEA-CPP algorithm has a smaller comprehensive cost of obtaining the optimal path, while the stability of the Hs-CCEA-CPP algorithm is also stronger.

(3)

Analysis of cooperative constraints

For spatial cooperative constraints, the CCEA-CPP algorithm was used to calculate the spatial cooperative violations of the optimal path corresponding to 30 independent runs in the scene, as shown in

Table 5. Spatial coordinated analysis table.

Algorithm Type	Non-Zero Occurrences of Spatial Cooperative Violations	Success Rate of Spatial Collaboration
Bs-CCEA-CPP	2	93.3%
Hs-CCEA-CPP	0	100%

.

From Table 5, it can be seen that the two algorithms independently run 30 times to obtain the optimal path of five UAVs. The Bs-CCEA-CPP algorithm corresponds to two non-zero spatial cooperative violations, with a spatial cooperative success rate of 93.3%. The Hs-CCEA-CPP algorithm corresponds to zero spatial cooperative violations, with a spatial cooperative success rate of 100%. From this, it can be seen that in most cases, both algorithms can achieve non-crossing and collision-free flight paths, but the Hs-CCEA-CPP algorithm performs better.

For time cooperative constraints, analysis can be conducted based on whether the estimated arrival time windows of each UAV intersect. Firstly, the optimal path length of the five UAVs obtained by independently running the algorithm 30 times is calculated. Assuming that the UAV flight speed is between 200 km/h and 300 km/h, the estimated arrival time window can be calculated based on the UAV path length. By taking the intersection of the estimated arrival time windows of each UAV, the cooperative arrival time window of the UAV group can be obtained. The statistical results are shown in

Table 6. Time-coordinated analysis table.

Algorithm Type	Maximum Time Window Length	Minimum Time Window Length	Number of Occurrences of Empty Sets	Success Rate of Time Collaboration
Bs-CCEA-CPP	0.7551	0.5570	0	100%
Hs-CCEA-CPP	0.7545	0.5970	0	100%

.

From Table 6, it can be seen that the optimal path of the five UAVs obtained by running the two algorithms independently 30 times can be calculated to obtain a non-empty set of five UAVs with cooperative arrival time windows, meeting the requirements of time coordinated, which means five UAVs can fly to the target point at the same time through speed distribution.

(4)

Compared experiments

In order to test the performance of the proposed algorithm, two algorithms are used to compare with the proposed algorithm. These two algorithms are GA and co-evolutionary algorithm based on random representation (Ra-CCEA). GA evolves a population to search the optimal cooperation path without decomposing the problem. Ra-CCEA is a type of coevolutionary algorithm. The difference between Ra-CCEA and the proposed algorithm is that Ra-CCEA use the random method to select the representative individuals for information sharing. Figure 7 shows the convergence curves of the four algorithms, and

Table 7. Convergence stability analysis of the comprehensive cost of the optimal path over 30 runs for four algorithms.

Algorithm Type	Maximum Value	Minimum Value	Average Value	Variance
GA	178.1610	59.6750	151.6780	550.0395
Ra-CCEA	37.1960	8.4930	25.3129	57.8910
Bs-CCEA-CPP	21.6830	5.1230	11.2546	14.2146
Hs-CCEA-CPP	12.5050	3.5160	5.3216	10.4141

shows the stability analysis of the four algorithms.

From Figure 7 and Table 7, we can see that the proposed algorithm shows good performance in terms of convergence speed, stability, and the optimal solution. The proposed algorithm adopts a co-evolutionary algorithm to reduce the complexity of the cooperation path planning problem, and utilizes the heuristic information of global and local optima for information sharing to improve the convergence speed and quality of the proposed algorithm. Although the Ra-CCEA algorithm also adopts the co-evolutionary algorithm, it adopts a random method for information sharing, which is weaker than the proposed algorithm. The GA algorithm directly searches the entire collaborative paths’ space, which is a huge search space, so the GA performs worse than the proposed algorithm.

5. Conclusions

The multi-UAV cooperative path planning problem is a complex optimization problem, and the idea of decomposition can effectively reduce the complexity. This paper proposes a multi-UAV cooperative path planning algorithm based on the co-evolution algorithm under the guidance of this idea. Firstly, the multi-UAV cooperative path planning problem is analyzed and modeled, and the time cooperative relationship and spatial cooperative relationship between UAVs are added to the optimization model as constraints. Secondly, the constraint processing method and information sharing method are designed, and a new algorithm is proposed. The design of information-sharing methods in co-evolution algorithms is a key point, and we have designed two different ways of sharing information. One way is to share the least synthetically expensive individual information in the subgroup, focusing on the big picture. Another way is to share the individual information with the least comprehensive cost and the individual information with the least cost of a single track in the sub-population, which aims to enhance the search for the optimal track neighborhood without considering constraints. For the algorithm proposed in this paper, we design a five-UAV cooperative penetration mission scenario for simulation verification. Through simulation verification, the algorithms designed in this paper can effectively cope with the path planning problem of multi-UAVs in complex environments and find a safe track set that meets the time and space constraints for UAV swarms, which has certain reference significance for enhancing the cooperative operation and viability of UAV swarms in complex environments.

This paper mainly focuses on the problem of collaborative trajectory planning in static scenarios. The static collaborative trajectory planning method can plan safe and flying collaborative trajectories for unmanned aerial vehicle groups before the implementation of multi-aircraft collaborative penetration tasks, but it cannot handle dynamic threats that may arise during the multi-aircraft collaborative penetration process. However, in reality, threats in the environment are often highly dynamic. Therefore, in the future, our research will be able to conduct dynamic collaborative trajectory planning for sudden threats and re-plan collaborative trajectories that meet the collaborative relationship. This will enable the drone group to continue to maintain the temporal and spatial collaborative relationship between each drone while avoiding sudden threats.