Multi-Conﬂict-Based Optimal Algorithm for Multi-UAV Cooperative Path Planning

: Multi-UAV cooperative path planning can improve the efﬁciency of task completion. To deal with the space and time conﬂicts of multi-UAVs in complex environments, a multi-collision-based multi-UAV cooperative path planning algorithm, multi-conﬂict-based search (MCBS), is proposed. First, the ﬂight and cooperative constraints of UAV are analyzed, and a three-dimensional environment model is established that incorporates geographical information. Then, hierarchical optimization is used to design collaborative algorithms. In the low-level path design, UAV ﬂight constraints are combined with a sparse A* algorithm, and by improving the cost function, the search space is reduced, and the search time is shortened. In high-level cooperation, the priorities of different conﬂicts are set, heuristic information is introduced to guide the constraint tree to grow in the direction of satisfying the constraints, and the optimal path set is searched by the best priority search algorithm to reduce the convergence time. Finally, the planning results of the proposed algorithm, the traditional CBS algorithm, and the sparse A* algorithm for different UAV tasks are compared, and the inﬂuence of the optimization parameters on the calculation results is discussed. The simulation results show that the proposed algorithm can solve cooperative conﬂict between UAVs, improve the efﬁciency of path searches, and quickly ﬁnd the optimal safe cooperative path that satisﬁes ﬂight and cooperative constraints.


Introduction
In recent years, as the flight environment and mission of UAVs have become more complex, a single UAV can no longer meet the mission requirements. Multi-UAV cooperative systems have attracted much attention from researchers due to their advantages such as scalability and robustness [1]. Multi-purpose UAVs are widely used in both military and civilian applications such as search and rescue, detection, and surprise defense [2][3][4]. Avoiding various threats in the environment is a key problem in achieving cooperative operations with multi-UAVs [5,6], which is usually defined as finding the optimal set of paths that satisfy various constraints of multi-UAVs [7,8].
Many multi-UAV path planning algorithms have been recently proposed by research scholars, and can be divided into two categories: reactive collision avoidance methods and active cooperative path generation methods. In reactive collision avoidance methods [9][10][11], any UAV that detects a risk of collision with another UAV performs a local cooperative replanning algorithm to avoid collision. Many algorithms based on reactive collision avoidance methods have been proposed, such as optimal reciprocal collision avoidance (ORCA) [12] and distributed reactive collision avoidance (DRCA) [13,14]. Reactive collision avoidance methods have been widely used in practice by virtue of their efficiency [15]. However, such methods only consider local cooperative collision avoidance, and thus tend to fall into a local optimum. In addition, such algorithms are difficult to deal with path planning problems that require satisfying both time and space cooperative constraints on UAVs. Among the active cooperative path generation the constraint tree and designed a heuristic function to guide the algorithm search. The proposed algorithm has been proven effective in simulation experiments.
The main contributions of our work are as follows: (1) Considering the constraint information of UAVs, the complex environment of multiple UAVs was modeled. (2) A sparse A * algorithm was designed to meet the flight constraints of UAVs, which reduces the search space and shortens the search time. (3) The collaborative conflict between multiple UAVs was defined; the priority of different collaborative conflicts is set; and the heuristic information was designed to guide the constraint tree to grow in the direction of conflict resolution to improve the algorithm's convergence time.
The arrangement of this article is as follows: in Section 2, we establish a constrained optimization model for multi-UAV cooperative path planning, analyze the model's flight and cooperative constraints, and then model the battlefield environment; in Section 3, we present in detail the multi-UAV cooperative path planning algorithm based on CBS; in Section 4, we evaluate the performance of the algorithm through simulation experiments; the conclusion is given in Section 5.

Problem Description
The problem studied in this paper is the cooperative path planning problem when a multi-UAV performs tasks in a complex environment. The multi-UAV cooperative path planning algorithm not only needs to find the path with the smallest cost for each aircraft, but also needs to satisfy certain constraints, such as those imposed by the physical characteristics of UAVs, environment threats and obstacles, and the cooperative constraints between the UAVs. Therefore, the multi-UAV cooperative path planning problem can be regarded as a constrained optimization problem that can be described as the following formula: minJ(P/E) s.t. g(P) ≤ 0 h(P) = 0 (1) where E is the environment for path planning; g(P) and h(P) are inequality constraints that comprehensively consider UAV flight and cooperative constraints; P is a set of paths that satisfy the constraints, which can be expressed as P = {p 1 , p 2 , · · · , p N }.

Path Cost Analysis
The total path cost J is the sum of all UAV path costs: where p i is the path of the ith UAV; N is the number of UAVs; J(P/E) is the total path cost of the path set P in the current environment E. The path cost J i of the ith UAV is composed of two parts: the path length cost J length (p i ) and the path threat cost J length (p i ):

Path Length Cost Function
When planning the path of the ith UAV, the path p i is the set of waypoints where the ith UAV is located at different times. When evaluating the length cost of the UAV path p i , the path length cost function of the nth waypoint v i n is as follows: j l (v n i ) = (x n − x n−1 ) 2 + (y n − y n−1 ) 2 + (z n − z n−1 ) 2 (4) where (x n , y n , z n ) is the 3D coordinates of the ith waypoint in the path. The path length cost is the sum of all waypoint length costs except for the starting point. The calculation formula is as follows: where l i is the number of waypoints in the path.

Path Threat Cost Function
The path threat cost mainly comes from five aspects: terrain, air defense radar, surfaceto-air missile, anti-aircraft artillery and no-fly zones. The path threat cost is the sum of the threat costs of all waypoints in the path. The path threat cost function is as follows: where the threat cost j t of the nth waypoint v i n is the weighted sum of five threat costs, as shown in the following formula: j t = k g · U g + k r · U r + k m · U m + k a · U a + k n · U n (7) where k g , k r , k m , k a , and k n are weighting factors. The path threat cost is affected by the environment E, so it is crucial to model environmental threats accurately.
(1) Terrain threat model and cost analysis To make the simulation environment based on path planning closer to the real environment, this paper selected a 500 × 500 km digital elevation model as the environmental model for path planning, as shown in Figure 1. The digital elevation model is located roughly between 107 • -119 • E and 34 • -35 • N. The size of the digital elevation model is 1000 × 1000 and 10 6 data points exist. For waypoint v i n , the terrain threat cost is calculated as follows: (8) where z n is the height of the waypoint v i n ; h g is the ground height; and H min is the minimum height above ground.
(2) Air defense radar model and analysis When modeling the radar, we took into account the radar blind spot created by terrain obscuring. The calculation steps of radar terrain masking a blind area are as follows: Calculate the radar ray equation Assuming that the height of the radar station is h r , the schematic diagram of the radar detecting the terrain obscuring blind area in a certain azimuth is as shown in Figure 2. For waypoint n i v , the terrain threat cost is calculated as follows: where n z is the height of the waypoint n i v ; g h is the ground height; and min H is the minimum height above ground.
(2) Air defense radar model and analysis When modeling the radar, we took into account the radar blind spot created by terrain obscuring. The calculation steps of radar terrain masking a blind area are as follows: Calculate the radar ray equation Assuming that the height of the radar station is r h , the schematic diagram of the radar detecting the terrain obscuring blind area in a certain azimuth is as shown in Figure  2.  Based on known polar coordinates P (θ,k) of different radar detection azimuths and digital elevation models h (θ,k) , the radar detection ray L 1 equation is: (b) Calculate the height of the radar blind spot Expand outward in turn from P (θ,0) . If the terrain height h (θ,k) is below the radar detection ray, there is a blind spot. If the terrain height h (θ,k) is above the radar detection ray, update ray, and continue expanding outward to calculate the blind spot height. If the terrain height at P (θ,2) is below the radar detection ray and there is a blind spot, its height at this point can be represented by the dotted line in Figure 2, and the height value is L 1 (P (θ,2) ) − h (θ,2) . The threat range of the radar in the final environment is shown in Figure 3, and the radar detection area is above the curved surface.
When the UAV flies over the radar detection area above the radar detection blind spot surface, it will face the danger of being detected by the enemy. We represent the radar threat cost of waypoint v i n using the radar detection probability: where P r is the point set of the radar detection coverage area; R r is the radar detection radius; and d is the horizontal distance between the waypoint v i n and the radar center point. detection ray, there is a blind spot. If the terrain height ( , ) k h θ is above the radar detection ray, update ray, and continue expanding outward to calculate the blind spot height. If the terrain height at ( ,2) P θ is below the radar detection ray and there is a blind spot, its height at this point can be represented by the dotted line in Figure 2, and the height value is . The threat range of the radar in the final environment is shown in Figure   3, and the radar detection area is above the curved surface. When the UAV flies over the radar detection area above the radar detection blind spot surface, it will face the danger of being detected by the enemy. We represent the radar threat cost of waypoint n i v using the radar detection probability: where r P is the point set of the radar detection coverage area; r R is the radar detection radius; and d is the horizontal distance between the waypoint n i v and the radar center point.
(3) Surface-to-air missile threat model and analysis For general surface-to-air missiles, the interception coverage area can be approximated by a hemisphere. The path planning space we studied was mainly aimed at an airspace of 6000 m and below for the flight of midair UAVs, so the surface-to-air missile interception coverage area at this height is represented as the remaining hemisphere with the top part removed. The threat range of the surface-to-air missile in the environment is shown in Figure 4. (3) Surface-to-air missile threat model and analysis For general surface-to-air missiles, the interception coverage area can be approximated by a hemisphere. The path planning space we studied was mainly aimed at an airspace of 6000 m and below for the flight of midair UAVs, so the surface-to-air missile interception coverage area at this height is represented as the remaining hemisphere with the top part removed. The threat range of the surface-to-air missile in the environment is shown in Figure 4.  The threat cost of the surface-to-air missile to the target within the attack range can be expressed by the following formula: where m r is the distance between the missile launcher and the attack target; max m R is the maximum attack distance of the missile; min m R is the minimum attack distance of the missile; and m P is the point set of the surface-to-air missile interception coverage area.
(4) Anti-aircraft artillery threat model and analysis Similar to the surface-to-air missile, the threat space of anti-aircraft artillery can also be represented as the remaining hemisphere with the upper part removed. The threat range of anti-aircraft artillery in the environment is shown in Figure 5. The threat cost of the surface-to-air missile to the target within the attack range can be expressed by the following formula: where r m is the distance between the missile launcher and the attack target; R mmax is the maximum attack distance of the missile; R mmin is the minimum attack distance of the missile; and P m is the point set of the surface-to-air missile interception coverage area.
(4) Anti-aircraft artillery threat model and analysis Similar to the surface-to-air missile, the threat space of anti-aircraft artillery can also be represented as the remaining hemisphere with the upper part removed. The threat range of anti-aircraft artillery in the environment is shown in Figure 5. is the minimum attack distance of the missile; and m P is the point set of the surface-to-air missile interception coverage area.
(4) Anti-aircraft artillery threat model and analysis Similar to the surface-to-air missile, the threat space of anti-aircraft artillery can also be represented as the remaining hemisphere with the upper part removed. The threat range of anti-aircraft artillery in the environment is shown in Figure 5. Within the scope of the anti-aircraft artillery, the closer the target is to the firing point of the artillery, the greater the probability of its being shot down; the farther the target is from the firing point, the smaller the probability, and the probability is zero when it exceeds the artillery range. The air defense artillery threat cost of the path waypoint n i v at Within the scope of the anti-aircraft artillery, the closer the target is to the firing point of the artillery, the greater the probability of its being shot down; the farther the target is from the firing point, the smaller the probability, and the probability is zero when it exceeds the artillery range. The air defense artillery threat cost of the path waypoint v i n at a certain point can be represented by U a , which can be determined using the following formula: where (x 0 , y 0 , z 0 ) is the position coordinate of the anti-aircraft artillery; (x n , y n , z n ) is the waypoint coordinate; R a is the maximum firing radius of the anti-aircraft artillery; and P a is the effective point set of the anti-aircraft artillery. Besides avoiding entering enemy threat areas, UAVs also need to avoid no-fly areas set by meteorological, political or other criteria. Generally speaking, their design is mostly quadrilateral, so we use quadrangular prisms to represent the no-fly zone for UAVs. The threat range is shown in Figure 6. ( ) Besides avoiding entering enemy threat areas, UAVs also need to avoid no-fly areas set by meteorological, political or other criteria. Generally speaking, their design is mostly quadrilateral, so we use quadrangular prisms to represent the no-fly zone for UAVs. The threat range is shown in Figure 6.

Constraint Analysis
The different sources of constraints that UAVs must satisfy in flight can be divided into two categories: UAV flight constraints derived from its physical characteristics, and For waypoint v i n , define P n as the point set in the no-fly zone space, and the calculation method of the no-fly zone threat cost is as follows:

Constraint Analysis
The different sources of constraints that UAVs must satisfy in flight can be divided into two categories: UAV flight constraints derived from its physical characteristics, and the cooperative constraints between UAVs.

UAV Flight Constraint Analysis
For the multi-UAV path planning problem studied in this paper, we select and derive a set of flight constraints suitable for this paper based on the representative constraints in the literature [29,30] to consider the key performance constraints and complex environmental requirements in UAV path planning. We mainly consider the following four UAV flight constraints: (1) Minimum turning radius Due to maneuvering constraints, the UAV turning radius needs to be larger than the minimum. Therefore, the planned path needs to impose a minimum turning radius constraint to be flyable, and this can be expressed as: where r i is the turning radius of the planned path in the first path segment, and r min is the minimum turning radius of the UAV.
(2) Minimum path segment length The minimum path segment length is the minimum distance that the UAV must fly before changing its attitude. The planned path unit step size we set needed to meet the minimum path segment length constraint, which can be expressed as dS > dS min (15) where dS is the path planning unit step size, and dS min is the minimum path length.
(3) Maximum path slope angle Since the UAV is constrained by engine performance and flight safety, the vertical ascent and dive angles of the path are also limited. Therefore, within a unit path step, the planned flight path of the UAV must meet the maximum path slope angle constraint. Assuming that the horizontal projection of the ith path is expressed as a i = (x i − x i−1 , y i − y i−1 ) T , the difference in the vertical direction is b i = |z i − z i−1 |, and the maximum path slope angle is assumed to be θ max , then the maximum path slope angle constraint can be expressed as: (4) Minimum ground height To avoid collision with terrain threats such as mountains, it is necessary to set the minimum height above the ground, so that the flying height of the UAV is greater than the minimum height above the ground. The minimum ground height constraint can be expressed as: where z i is the height value of the ith path planning point, and H min is the minimum height above the ground.

Constraint Analysis
In addition to satisfying the flight constraints of a single aircraft, multi-UAV path planning also needs to meet the cooperative constraints that require UAVs to cooperate to complete the task within a specified time without any space conflicts. According to the task requirements, UAVs need to keep consistent or fly in a certain order. Therefore, the cooperative constraints between UAVs are divided into space and time cooperative constraints.
(1) Time cooperative constraint analysis A time cooperative constraint requires that each UAV flies from different starting points according to the planned cooperative path and reaches the target point according to a certain time sequence or the same time interval. Assuming that the planned track length of the ith UAV is L i and the speed range is [V min , V max ], the time range T i for the UAV to reach the target point is To enable all N UAVs to reach the target point at the same time through speed allocation, it is necessary that the intersection of the expected arrival time windows T i of UAVs not be empty, that is, the cooperative arrival time window of the UAV group should be T = T 1 ∩ T 2 ∩ . . . ∩ T N . Although the time cooperation between UAVs can be achieved by adjusting speed, the adjustable range is limited. Therefore, as shown in Figure 7, we mainly considered adjusting the arrival time by adjusting the UAV path length.  (2) Space cooperative constraint analysis During multi-UAV cooperative path planning, in addition to the time cooperation of UAVs, it is also necessary to consider the space cooperation of different UAVs. As shown in Figure 8, the space cooperative constraint is that the distance between any two UAVs in flight at the same time is always greater than the safety distance safe D . Suppose the distance between the ith UAV and jth UAV at the time t is ( ) ij D t . If all N UAVs are required to maintain a safe distance, the following formula must be satisfied for any time t :

Multi-Conflict-Based Optimal Algorithm
Compared with single-UAV path planning, multi-UAV path planning has much greater complexity because it needs to consider not only the optimal path of the single-UAV but also the multi-UAV cooperative path problem. We noticed that the CBS algo- (2) Space cooperative constraint analysis During multi-UAV cooperative path planning, in addition to the time cooperation of UAVs, it is also necessary to consider the space cooperation of different UAVs. As shown in Figure 8, the space cooperative constraint is that the distance between any two UAVs in flight at the same time is always greater than the safety distance D sa f e . Suppose the distance between the ith UAV and jth UAV at the time t is D ij (t). If all N UAVs are required to maintain a safe distance, the following formula must be satisfied for any time t: Drones 2023, 7, x FOR PEER REVIEW 10 of 26 limited. Therefore, as shown in Figure 7, we mainly considered adjusting the arrival time by adjusting the UAV path length. (2) Space cooperative constraint analysis During multi-UAV cooperative path planning, in addition to the time cooperation of UAVs, it is also necessary to consider the space cooperation of different UAVs. As shown in Figure 8, the space cooperative constraint is that the distance between any two UAVs in flight at the same time is always greater than the safety distance safe D . Suppose the distance between the ith UAV and jth UAV at the time t is ( ) ij D t . If all N UAVs are required to maintain a safe distance, the following formula must be satisfied for any time t : Figure 8. Space cooperative path.

Multi-Conflict-Based Optimal Algorithm
Compared with single-UAV path planning, multi-UAV path planning has much greater complexity because it needs to consider not only the optimal path of the single-UAV but also the multi-UAV cooperative path problem. We noticed that the CBS algo-

Multi-Conflict-Based Optimal Algorithm
Compared with single-UAV path planning, multi-UAV path planning has much greater complexity because it needs to consider not only the optimal path of the single-UAV but also the multi-UAV cooperative path problem. We noticed that the CBS algorithm solved the space cooperative problem of planar multi-agent path planning excellently, so we tried to solve the multi-UAV path planning problem using the CBS algorithm; however, new problems were encountered. For example, the low-level A* algorithm had difficulty carrying out 3D space path planning; the UAV had many constraints; and the path conflict between UAVs was lacking. The specific definition and coordination problems included both space and time conflicts. In response to these problems, we made a new algorithm called multi-conflict-based search (MCBS).

Conflict Based Search
Conflict-based search (CBS) is a multi-agent path planning algorithm chiefly designed to create a constraint set for each agent according to the conflicts between the paths of agents and break down the multi-agent path planning problem into a large number of constrained combinations of single-agent path planning problems. The CBS algorithm is divided into two levels. The high level is responsible for discovering the conflicts between paths of agents and adding constraints; the low level is responsible for planning paths that meet the constraints of the single agent.
The pseudo-code for CBS is shown in Algorithm 1. The high level uses the best-first search algorithm. The algorithm will generate a root tree node Root with empty constraints set at first. It consists of four parts: the set of paths, the solution Root.solution, the cost of the node Root.cost, and the constraints set Root.constraints. The search tree then begins to expand. All the tree nodes that have not been selected yet are put into the OPEN cbs table and then the node with the lowest growth cost is selected. Each tree node contains a set of constraints to avoid path conflicts and a set of paths that meet the constraints. The cost of the node is the sum of the lengths of the paths. When expanding the constraint tree node, the algorithm will first check the conflict between paths, that is, whether multiple agents are occupying the same vertex or the same edge at the same time. If there is no conflict, the path is the target path, and the algorithm terminates. Otherwise, if there is a conflict between the ith agent a i and the jth agent a j at vertex v, CBS will generate two child nodes, with the current constraint tree node as the parent. Each child node inherits a path set and constraint information from its parent, which resolves the conflict by adding new constraints c s to the child nodes and then replanning the path of the agent through the low-level A* algorithm. By the growth of the constraint tree, the CBS algorithm analyzes the two solutions to a conflict to ensure the completeness of the algorithm and the optimality of the algorithm through the best-first search at both high and low levels.

Algorithm Design of MCBS
Corresponding to the two-level design of the CBS algorithm, MCBS divides the constraints of multi-UAV path planning into two categories: constraints that must be satisfied in single aircraft planning, such as physical constraints in flight, and the other is space and time constraints between UAVs. We dealt with these two kinds of constraints in the low level and the high level of the algorithm, respectively, to find the optimal solution to satisfy the constraints. For low-level path planning, we designed a sparse A* algorithm based on the flight constraints of a UAV, which solved the problem of the A* algorithm adopted by the CBS algorithm having difficulty dealing with UAV path planning in 3D space. For high-level design, the CBS algorithm only considered the space conflict, whereas MCBS considers both space and time conflicts and sets the priority of conflict resolution. The constraint tree always grows first because of the space conflict, and when there is no space conflict it grows according to the time conflict. The constraint tree will continue to grow until it finds a path combination solution that has neither a time nor space conflict. When searching for the optimal path combination solution in the constraint tree, the search is carried out according to the best priority of the path cost. However, in extreme cases, the algorithm will traverse all path combinations, making the convergence of the algorithm slow. Therefore, we designed a heuristic function to make the algorithm search in the direction of fewer conflicts.

High Level Search
The purpose of the high level is to search different path combinations to find the optimal one through the growth of the constraint tree. We defined the path conflict between UAVs and improved the growth mode of the constraint tree and designed a conflict resolution framework to solve the time and space cooperative problems, as well as a new heuristic function to guide the algorithm search.
(1) Conflict detection and constraint generation Unlike the grid environment, the path planning space in the 3D environment is larger, so the conflict is more difficult to define. We divided the conflict in the path planning problem of a multi-UAV into two categories: space and time.
(a) Space conflict When UAVs fly, it is necessary to maintain a safe distance between UAVs at the same time. The two space conflicts we considered were point and edge. Point conflict is shown in Figure 9. Conflict occurs when two aircraft are at the same path point at the same time or when the distance between two points at the same time is less than the safe distance. As shown in Figure 10, the edge is formed by connecting the front and rear path points. When the distance between the points on the edges of two aircraft at the same time is less than the safe distance, edge conflict occurs. ristic function to guide the algorithm search.
(1) Conflict detection and constraint generation Unlike the grid environment, the path planning space in the 3D environment is larger, so the conflict is more difficult to define. We divided the conflict in the path planning problem of a multi-UAV into two categories: space and time.
(a) Space conflict When UAVs fly, it is necessary to maintain a safe distance between UAVs at the same time. The two space conflicts we considered were point and edge. Point conflict is shown in Figure 9. Conflict occurs when two aircraft are at the same path point at the same time or when the distance between two points at the same time is less than the safe distance. As shown in Figure 10, the edge is formed by connecting the front and rear path points. When the distance between the points on the edges of two aircraft at the same time is less than the safe distance, edge conflict occurs.  Due to mission constraints, multi-UAVs need to reach a target point at the same time or in a certain order. When the speed is given, the UAV arrival time is determined by the path length, so the time can be expressed by the number of waypoints in the UAV path. When the difference between the number of path nodes in the path set is greater than the maximum time difference required for time cooperation, a time conflict occurs.
This conflict can be generated as ( , ) x tar a T , which means that the difference between the path waypoints of UAV x a and the maximum waypoints in the path set is greater than the target time difference tar T . The time constraint t c can be expressed as (2) The constraint tree A high-level search uses the best-first search in the nodes of the constraint tree. The growth of the constraint tree represents the increase in the searchable path combination solutions. Compared with CBS, MCBS needs to deal with two conflicts because its constraint tree designs different growth methods for different conflicts.
Each tree node in the constraint tree includes three parts: constraint set, path set, and tree node cost. However, compared with CBS, MCBS subdivides the constraint set into a time set Time C and space set Space C according to the two conflicts. Space conflict can be generated as (a i , a j , t, D sa f e ), which means that the mutual distance between UAV a i and UAV a j at the path point at time t was less than the safe distance D sa f e . Space conflict c s can be generated as (a i , v i , t), where v i is the waypoint where UAV a i is in the event of a conflict, which means that UAV a i is forbidden to be at the waypoint v i at time t.

(b)
Time conflict Due to mission constraints, multi-UAVs need to reach a target point at the same time or in a certain order. When the speed is given, the UAV arrival time is determined by the path length, so the time can be expressed by the number of waypoints in the UAV path. When the difference between the number of path nodes in the path set is greater than the maximum time difference required for time cooperation, a time conflict occurs.
This conflict can be generated as (a x , T tar ), which means that the difference between the path waypoints of UAV a x and the maximum waypoints in the path set is greater than the target time difference T tar . The time constraint c t can be expressed as (a x , v t , t, dS), and v t is the path point of the UAV a x at the time t. The time constraint means that UAV a x is prohibited from occupying the sphere with path point v t as its center and radius dS at time t, making this sphere a time-limited area.
(2) The constraint tree A high-level search uses the best-first search in the nodes of the constraint tree. The growth of the constraint tree represents the increase in the searchable path combination solutions. Compared with CBS, MCBS needs to deal with two conflicts because its constraint tree designs different growth methods for different conflicts.
Each tree node in the constraint tree includes three parts: constraint set, path set, and tree node cost. However, compared with CBS, MCBS subdivides the constraint set into a time set C Time and space set C Space according to the two conflicts.
(3) Create tree nodes When the constraint tree generates the initial tree node, it does not consider space and time cooperation with other UAVs; instead, it plans a path set that only meets the flight constraints of UAVs as the initial tree node. Each time the constraint tree grows, the tree node with the lowest cost is selected as the current node. When the conflicts of the current node are different, the growth methods of the constraint tree are different. The growth process is shown in Figure 11. If the current node has a time conflict, then the current node is taken as the parent node to generate child nodes. As shown in Figure 11a, the number of child nodes is equal to the number of waypoints l in the path of the UAV with time conflict. Then, a new time constraint c ti is added to the ith child node's time set C Time to solve the conflict. If the current node has a space conflict, the growth mode of the constraint tree is the same as that of the CBS. As shown in Figure 11b, the first conflict in the path set is selected, and then the current node is taken as the parent node to generate two child nodes and add new space constraint c s for each child node. The child nodes inherit all the information from the parent and update their own path set through a low-level algorithm to provide a feasible solution to the conflict. If the current node has both time and space conflict, the constraint tree of MCBS defines which has priority. It always grows according to the space conflict first. When the current node has no space conflict, it grows according to the time conflict. to solve the conflict. If the current node has a space conflict, the growth mode of the constraint tree is the same as that of the CBS. As shown in Figure 11b, the first conflict in the path set is selected, and then the current node is taken as the parent node to generate two child nodes and add new space constraint s c for each child node. The child nodes inherit all the information from the parent and update their own path set through a low-level algorithm to provide a feasible solution to the conflict. If the current node has both time and space conflict, the constraint tree of MCBS defines which has priority. It always grows according to the space conflict first. When the current node has no space conflict, it grows according to the time conflict.
(a) (b) Figure 11. The growth methods of the constraint tree. (a) Tree grows due to time conflict; (b) tree grows due to space conflict.
(4) Tree node cost At the high level, the algorithm will maintain an open set, select the tree node with the lowest path cost in the open set as the parent node for the next growth, then put the new growth node into the open set and remove the parent node. If the path cost is calculated according to formula (2), the algorithm will traverse all tree nodes with conflicting connections in extreme cases, which will lead to a huge search space. To reduce both the search space and the time to find the optimal solution, we added the number of space conflicts of paths and the time cooperative violation of paths as heuristic information to the path cost and designed a new cost function to guide the growth of the constraint tree. The new node cost function is defined as follows:  (4) Tree node cost At the high level, the algorithm will maintain an open set, select the tree node with the lowest path cost in the open set as the parent node for the next growth, then put the new growth node into the open set and remove the parent node. If the path cost is calculated according to Formula (2), the algorithm will traverse all tree nodes with conflicting connections in extreme cases, which will lead to a huge search space. To reduce both the search space and the time to find the optimal solution, we added the number of space conflicts of paths and the time cooperative violation of paths as heuristic information to the path cost and designed a new cost function to guide the growth of the constraint tree. The new node cost function is defined as follows: where n s is the number of space cooperative conflicts between UAVs; l max is the maximum number of waypoints in the path set; l i is the number of waypoints in the path of the ith UAV; ω s and ω t are space and time cooperative penalty factors; and J is the current total path cost. When the algorithm finds out the target path set, the number of space conflicts between UAVs is equal to zero, and the difference in the number of voyages in the path set n s is 0. At this time, F = J.

(5) Conflict resolution
The flow chart of the conflict resolution framework is shown in Figure 12. The conflict resolution framework can be divided into space conflict and time conflict resolution frameworks, each with different conflict resolution methods. The process is as follows: Time conflict is not caused by a fixed path point or an edge but by a certain path in the path set. As shown in Figure 13, the time conflict can be solved by using the time For the current tree node, the space conflicts in the path sets are detected first. When space conflict (a i , a j , t, D sa f e ) occurs, two child nodes are grown with the current tree node as the parent node, and the child nodes inherit the original constraint set and add space constraints (a i , v i , t) and (a j , v j , t), respectively. Thus, the two solutions to the current space conflict are discussed through the child nodes.
Time conflict is not caused by a fixed path point or an edge but by a certain path in the path set. As shown in Figure 13, the time conflict can be solved by using the time constraint set to make the low-level algorithm bypass the point where the cost is too small and prolong the path length, which is too short. When there is no space conflict in the current node, we look for a time conflict. When the time conflict (a x , T tar ) occurs at the current node, it is used as the parent node to grow l x child nodes, and each child node randomly selects path point v t at a time t in the path of UAV a x as the time constraint (a x , v t , t, dS). Then, the time constraint is added to the original time constraint set of child nodes. l x is the number of waypoints in the path of UAV a x , which means that all solutions to the current time conflict are discussed and analyzed through subsequent subnodes to ensure the completeness of the algorithm.

Low Level Search
The algorithm sets the constraint set for each UAV according to the conflict at the high level and finds a path satisfying the constraint set for the UAV at the low level. We adopted the sparse A* algorithm at the low level and, based on it, we eliminated invalid points by combining the flight constraints of the UAV, thus reducing the search space, shortening the search time, and meeting the physical characteristics of UAV, which can be directly applied to its flight.
(1) Waypoint cost function When the sparse A* algorithm was expanded, the cost function of the nth waypoint could be determined according to the following formula: where ( ) The heuristic information ( ) (2) Waypoint extension Successor waypoints can be divided into upper, horizontal, and lower layers according to the different heights. Each layer contains three subsequent waypoints at the same altitude. When the waypoint is expanded, subsequent waypoints on the horizontal plane are calculated according to the current yaw angle and the minimum turning radius of the

Low Level Search
The algorithm sets the constraint set for each UAV according to the conflict at the high level and finds a path satisfying the constraint set for the UAV at the low level. We adopted the sparse A* algorithm at the low level and, based on it, we eliminated invalid points by combining the flight constraints of the UAV, thus reducing the search space, shortening the search time, and meeting the physical characteristics of UAV, which can be directly applied to its flight.
(1) Waypoint cost function When the sparse A* algorithm was expanded, the cost function of the nth waypoint could be determined according to the following formula: where f l v i is the cumulative length cost; f t v i is the path threat cost; and h(v n ) is heuristic information. As shown in Formula (21), the cumulative length cost f l v i is the sum of the distances from the starting point to the current waypoint, and the cumulative threat cost f t v i is the sum of the threat costs of all waypoints.
The heuristic information h(v n ) is the Manhattan distance from the current waypoint to the target waypoint. When the waypoint is identical to the target waypoint, h(v n ) is zero, and the waypoint cost f t v i is equal to the path cost J i of the UAV.
(2) Waypoint extension Successor waypoints can be divided into upper, horizontal, and lower layers according to the different heights. Each layer contains three subsequent waypoints at the same altitude. When the waypoint is expanded, subsequent waypoints on the horizontal plane are calculated according to the current yaw angle and the minimum turning radius of the UAV; then the maximum climb and maximum dive altitudes that correspond to the aircraft are calculated based on the maximum path slope angle. Finally, the successor waypoints of the upper and lower layers are calculated.
The expansion of the waypoint in the horizontal layer is shown in Figure 14. On the horizontal plane, each waypoint expands three subsequent waypoints at one time. S i j indicates that the parent waypoint of the current waypoint S j is S i . The expansion of the waypoint in the horizontal layer is shown in Figure 14. On the horizontal plane, each waypoint expands three subsequent waypoints at one time. i j S indicates that the parent waypoint of the current waypoint j S is i S . In Figure 14, dS is the unit step size of the waypoint and min r is the minimum turning radius of the UAV. The coordinates of the next generation extended waypoints

Algorithm Procedure
The multi-UAV cooperative path planning problem is essentially a constrained optimization problem. The MCBS algorithm adopts a two-level design; the low level is used to find the optimal path of UAV, and the high level is used to find the optimal combination path of a multi-UAV. MCBS divides the UAV constraints into flight and cooperative constraints. Flight constraints are resolved at the low level, and collaborative constraints at the high level.
The high level solves the time and space conflicts separately by setting different priorities for the UAV constraints. Corresponding to lines 8 to 19, the algorithm first solves the space conflict through the space constraint set. According to lines 20 to 30, when the current node has no space conflicts, time conflicts are resolved through the time constraint set. Lines 31 to 33 state that when there are no space or time conflicts at the current node, a solution that satisfies the constraints is obtained. To obtain the solution that satisfies the constraints as the optimal solution, the best-first algorithm was carried out in the constraint tree to ensure the optimality of the algorithm, which corresponds to line 26. In Figure 14, dS is the unit step size of the waypoint and r min is the minimum turning radius of the UAV. The coordinates of the next generation extended waypoints (S n+1 n , S n+2 n , S n+3 n ) can be calculated as

Algorithm Procedure
The multi-UAV cooperative path planning problem is essentially a constrained optimization problem. The MCBS algorithm adopts a two-level design; the low level is used to find the optimal path of UAV, and the high level is used to find the optimal combination path of a multi-UAV. MCBS divides the UAV constraints into flight and cooperative constraints. Flight constraints are resolved at the low level, and collaborative constraints at the high level.
The high level solves the time and space conflicts separately by setting different priorities for the UAV constraints. Corresponding to lines 8 to 19, the algorithm first solves the space conflict through the space constraint set. According to lines 20 to 30, when the current node has no space conflicts, time conflicts are resolved through the time constraint set. Lines 31 to 33 state that when there are no space or time conflicts at the current node, a solution that satisfies the constraints is obtained. To obtain the solution that satisfies the constraints as the optimal solution, the best-first algorithm was carried out in the constraint tree to ensure the optimality of the algorithm, which corresponds to line 26.
The lower level of the algorithm combines the flight constraints of the UAV to limit the waypoint selection so that the final path meets the physical characteristics of the UAV. At the high level, when resolving a space conflict, the constraint tree generates two child nodes, and when resolving a time conflict, the constraint tree generates child nodes with the same number of waypoints in the path where the time conflict occurred. In this way, all solutions to different conflicts are analyzed and evaluated in detail, ensuring the completeness of the algorithm. At the high and low levels, we introduce heuristic information to accelerate convergence and ensure that the results were optimal. The algorithm pseudocode is shown in Algorithm 2.

Algorithm 2 Pseudocode of proposed algorithms.
Input: number of UAV, starting points, target points, D sa f e , T tar . Output: optimal solution that satisfies flight constraints and cooperative constraints. Root.time_constraints ← ∅ ; Root.space_constraints ← ∅ ; Root.solution ← Find path for UAV through low level search; Root.cost ← cost(Root.solution) ; insert Root into OPEN cbs ; while OPEN cbs = ∅ find the tree node N with the minimal cost in OPEN cbs ; if N.solution has space conflicts, find the space conflict (a i , a j , t, D sa f e ) which occurs first in N.solution; remove N from OPEN cbs ; for a x in a i , a j create a new space constraint set c s = (a x , v x , t); P ← CreatNode() ; P.time_constraints ← N.time_constraints ; P.space_constraints ← N.space_constraints + c s P.solution ← N.solution P.solution(a x ) ← Find path for UAV a x through low level search; P.cost ← cost(P.solution) ; insert P into OPEN cbs ; if N.solution does not have space conflicts but has time conflicts (a x , T tar ), remove N from OPEN cbs ; for i =1 : l x Create a new time constraint set c ti = (a x , v t , t, dS); P ← CreatNode() ; P.time_constraints ← N.time_constraints + c ti ; P.space_constraints ← N.space_constraints ; P.solution ← N.solution ; P.solution(a x ) ← Find path for UAV a x through low level search; P.cost ← cost(P.solution) ; insert P into OPEN cbs ; if N.solution has neither space conflicts nor time conflicts, break; then, return N.solution.

Simulation Studies
We simulated and analyzed MCBS in a complex environment. It mainly verified the superiority of a heuristic search at the high-level and the applicability of solving temporal and space cooperation problems in multi-UAV cooperative path planning.

Environmental and Parametric
In order to be close to the real battlefield environment, we set the parameters of various threats, as shown in Table 1, based on the real environment space in a certain western region as the battlefield setting area. The cooperate constraints imposed on UAVs are affected by the type of mission. We selected the most common allocation and rendezvous task for the multi-UAV. The allocation task was to require the UAV to avoid various threats and reach the airspace near the target point from an initial point, which required high space cooperation. The rendezvous tasks required all UAVs to arrive at the same target point at the same time, which required higher space and time coordination. The parameter settings of the algorithm are shown in Table 2.

Algorithm Comparative Analysis
(1) Allocation task Figures 15 and 16 show the multi-UAV cooperative path planning results of the MCBS algorithm when the number of UAVs performing the allocation tasks was different. As can be seen in Figures 15 and 16, the MCBS algorithm is able to find a safe path for each UAV.
However, it is difficult to see from Figures 15 and 16 whether the set of paths satisfies the cooperation constraints. Therefore, we set the velocity of all UAVs to 600 km/h and calculate the shortest distance between UAVs and the time tolerance of the path set. The time tolerance of the ith UAV is calculated as follows, where l i is the length of the path of the ith UAV; is the median of all UAV path lengths; and v i is the velocity of the ith UAV. If the time tolerance of all UAVs is less than 1.5 min, the time cooperative constraints are met. Figure 17 shows shortest distance results between UAVs when five and ten UAVs performed allocation tasks by different algorithms. Figure 18 shows time tolerance results of different algorithms when five and ten UAVs performed allocation tasks by different algorithms. However, it is difficult to see from Figures 15 and 16 whether the set of paths satisfies the cooperation constraints. Therefore, we set the velocity of all UAVs to 600 km/h and calculate the shortest distance between UAVs and the time tolerance of the path set. The time tolerance of the ith UAV is calculated as follows, where i l is the length of the path of the ith UAV; is the median of all UAV path lengths; and i v is the velocity of the ith UAV. If the time tolerance of all UAVs is less than 1.5 min, the time cooperative constraints are met. Figure 17 shows shortest distance results between UAVs when five and ten UAVs performed allocation tasks by different algorithms. Figure 18 shows time tolerance results of different algorithms when five and ten UAVs performed allocation tasks by different algorithms. However, it is difficult to see from Figures 15 and 16 whether the set of paths satisfies the cooperation constraints. Therefore, we set the velocity of all UAVs to 600 km/h and calculate the shortest distance between UAVs and the time tolerance of the path set. The time tolerance of the ith UAV is calculated as follows, where i l is the length of the path of the ith UAV; is the median of all UAV path lengths; and i v is the velocity of the ith UAV. If the time tolerance of all UAVs is less than 1.5 min, the time cooperative constraints are met. Figure 17 shows shortest distance results between UAVs when five and ten UAVs performed allocation tasks by different algorithms. Figure 18 shows time tolerance results of different algorithms when five and ten UAVs performed allocation tasks by different algorithms.  To further verify that the paths satisfy the flight constraints and to evaluate the set of paths planned by different algorithms, we present the constraints satisfaction results for different algorithms in Table3. From Table 3, it can be seen that the paths planned by all three algorithms can satisfy the flight constraints of UAVs. However, the results of sparse A* cannot handle the cooperative constraints, and CBS handled the space constraints well, but not the time constraints. Only the set of paths planned by the MCBS algorithm can meet all the constraints.  Figure 21 shows the path length and shortest distance results at the same moment, planned by different algorithms when five and ten UAVs performed a rendezvous task. Figure 22 shows the time tolerance results of different algorithms when five and ten UAVs performed rendezvous tasks by different algorithms. Table 4 presents the constraints meet results for different algorithms. As can be seen from Figures 19 and 20, MCBS is capable of planning safe paths for UAVs performing the rendezvous task. From Figures 21 and 22 and Table 4, MCBS can solve the time and space cooperative constraints and found the optimal path set for multi-UAVs to fly safely and meet the flight constraints. To further verify that the paths satisfy the flight constraints and to evaluate the set of paths planned by different algorithms, we present the constraints satisfaction results for different algorithms in Table 3. From Table 3, it can be seen that the paths planned by all three algorithms can satisfy the flight constraints of UAVs. However, the results of sparse A* cannot handle the cooperative constraints, and CBS handled the space constraints well, but not the time constraints. Only the set of paths planned by the MCBS algorithm can meet all the constraints.  Figure 21 shows the path length and shortest distance results at the same moment, planned by different algorithms when five and ten UAVs performed a rendezvous task. Figure 22 shows the time tolerance results of different algorithms when five and ten UAVs performed rendezvous tasks by different algorithms. Table 4 presents the constraints meet results for different algorithms. As can be seen from Figures 19 and 20, MCBS is capable of planning safe paths for UAVs performing the rendezvous task. From Figures 21 and 22 and Table 4, MCBS can solve the time and space cooperative constraints and found the optimal path set for multi-UAVs to fly safely and meet the flight constraints.      . The runtime related positively to the number of conflicts it needed to deal with. When the number of UAVs was fewer than 15, the number of conflicts was relatively small, and the optimal solution to the problem was found within 5 min. When the time  Based on the above environment, we selected different starting and ending points many times to test the runtime of our algorithm under different parameters. The number of UAVs varied from 3 to 23; the maximum time difference T tar from 0 to 2 min; and safe distance D sa f e from 0 to 7.5 km in increments of 2.5 km. The test results are shown in Figure 23.

Penalty Factor Parameter Analysis
Previously, we introduced heuristic information including penalty factors in the cost function of tree nodes. To verify the effectiveness of this scheme, we analyzed the impact of the penalty factor on the success rate of the algorithm when performing different tasks based on the environment in 4.1. We stipulated that the algorithm failed when it could not give a valid solution within 5 min. The simulation results are shown in Figure 24.
The size of the penalty factor influenced the importance of space and time cooperation when the constraint tree grew. As it did, we prioritized resolving space conflicts. Then, on the basis that the space conflicts have been resolved, the constraint tree was made to grow in the direction of resolving time conflicts without generating a new space conflict. Therefore, as can be seen from Figure 24, when ω t = 5 and ω s = 10 such as that ω s × ∆n s > ω t × ∆ n ∑ i=1 (l max − l i ) > ∆J, the algorithm was the most efficient.

Task Parameter Analysis
Based on the above environment, we selected different starting and ending points many times to test the runtime of our algorithm under different parameters. The number of UAVs varied from 3 to 23; the maximum time difference tar T from 0 to 2 min; and safe . The runtime related positively to the number of conflicts it needed to deal with. When the number of UAVs was fewer than 15, the number of conflicts was relatively small, and the optimal solution to the problem was found within 5 min. When the time and space cooperative conditions became more stringent or the number of UAVs increased, the number of collaborative conflicts increased rapidly, as did the time used by the MCBS.

Penalty Factor Parameter Analysis
Previously, we introduced heuristic information including penalty factors in the cost function of tree nodes. To verify the effectiveness of this scheme, we analyzed the impact of the penalty factor on the success rate of the algorithm when performing different tasks based on the environment in 4.1. We stipulated that the algorithm failed when it could not give a valid solution within 5 min. The simulation results are shown in Figure 24. The size of the penalty factor influenced the importance of space and time cooperation when the constraint tree grew. As it did, we prioritized resolving space conflicts. Then, on the basis that the space conflicts have been resolved, the constraint tree was made to grow in the direction of resolving time conflicts without generating a new space conflict. Therefore, as can be seen from Figure 24

Conclusions
In this study, a cooperative path planning algorithm for UAVs based on multi-collision detection was proposed. A complex mission environment model was established based on UAV flight and cooperative constraints, and a spatial and temporal collaborative solution framework was designed for multi-UAV collaborative problems. A three-dimensional environment path planning method based on an improved sparse A* was designed for a single aircraft. For multi-UAV cooperation, the collaborative conflict between multiple UAVs was defined; the priority of different collaborative conflicts was set; the growth mode of the constraint tree was changed; and a new heuristic function was designed to guide the search to reduce the convergence time of the algorithm. A comparison was made among the planning results of this algorithm, the traditional CBS, and the sparse A * algorithm for different UAV group tasks in a complex mission environment. The simulation

Conclusions
In this study, a cooperative path planning algorithm for UAVs based on multi-collision detection was proposed. A complex mission environment model was established based on UAV flight and cooperative constraints, and a spatial and temporal collaborative solution framework was designed for multi-UAV collaborative problems. A three-dimensional environment path planning method based on an improved sparse A* was designed for a single aircraft. For multi-UAV cooperation, the collaborative conflict between multiple UAVs was defined; the priority of different collaborative conflicts was set; the growth mode of the constraint tree was changed; and a new heuristic function was designed to guide the search to reduce the convergence time of the algorithm. A comparison was made among the planning results of this algorithm, the traditional CBS, and the sparse A * algorithm for different UAV group tasks in a complex mission environment. The simulation results showed that the proposed algorithm could deal with the coupling problem of time and space cooperation in complex environments and find the optimal safe cooperative path that satisfied the flight and cooperation constraints for multiple UAVs.