1. Introduction
Unmanned aerial vehicle (UAV) light shows (UAVLS) are an emerging technology for festivals and company promotions. Compared with traditional fireworks, UAVLS are environmentally friendly and more flexible to present animations. With the rapid advancement of UAVs and their control technologies, successful UAVLS cases can be found, manufactured by various companies such as Intel and Ehang [
1,
2]. As a result, studies of swarm control, communication, planning, and software and hardware design are becoming increasingly popular for more intelligent, energyefficient, and beautiful light shows.
There are many ways to control a group of UAVs, e.g., using a brain–computer interface (BCI) [
3] or a ground control station (GCS). In most UAVLS systems, a ground control station, e.g., a highperformance computer or mobile device, is responsible for managing and controlling the UAVs [
4]. Depending on the degree of intervention from the GCS, UAVLS systems can be operated under three modes, i.e., manintheloop, semiautonomous, and autonomous [
5,
6]. In the manintheloop mode, UAVs are completely controlled by the GCS during the light show. In the semiautonomous mode, the onboard flight controller, which maintains the flight stability and avoids collisions, cooperates with the GCS. In the autonomous mode, UAVs can autonomously make decisions and plan their paths. In the context of a light show, there is a necessary tradeoff between the cost of hundreds or even thousands of UAVs and the control performance. Indeed, the success of the autonomous mode heavily relies on powerful onboard computing units and sensors, which are extremely expensive for a largescale UAV group. As a result, the semiautonomous mode is considered in this paper as an affordable and competitive solution. In such mode, the GCS sends waypoint information to UAVs and the onboard flight controller tracks the waypoint and then sends back the status of the UAVs. Since UAVLS systems usually perform in fixed and limited airspace, the increasing demand of communications among UAVs and the GCS in the semiautonomous mode would not result in significant expense growth.
One of the key steps in the process of a UAVLS is formation transformation, where UAVs move from an initial formation to a target one in order to switch light graphics in the air. To do so, the task assignment problem (TAP) must first be tackled, which in essence states that each node in the target formation should be assigned to a UAV in order to satisfy a predefined criteria, e.g., minimum total flight distance. Then, the trajectory planning problem must be considered to generate smooth and collisionfree trajectories for safety concerns. Studies on these problems have been reported by many researchers. In [
7,
8,
9], path planning and obstacle avoidance algorithms for a single UAV have been reviewed. In [
10,
11], the particle swarm optimization (PSO) algorithm has been employed to solve the TAP without collision avoidance. In [
12], a discrete wolf pack algorithm is proposed to solve the multiUAV task assignment with fast convergence speed. In [
13], Duan et al. present a multiUAV formation reconfiguration algorithm based on the hybrid PSO and genetic algorithm (HPSOGA) to achieve formation transformation while satisfying various formation constraints. In [
14,
15], a distributed cooperative control method is proposed to realize the desired formation without task assignment and collision avoidance. In [
16], a UAVsLS system is realized to achieve precise formation and coordination control using multiUAV teaming technologies. However, in the aforementioned works, the TAP and the trajectory planning problem are decoupled, such that the TAP generates straightline trajectories from the initial node to the target one. Yet in practice, UAVs often deviate from the straightline trajectories due to mutual collision avoidance and external disturbances, leading to a mismatch between the actual task execution cost and the desired one.
In this paper, we present a UAVLS system including a dynamic collisionfree formation transformation algorithm, a software package that consists of an animation design interface (ADI) and a realtime monitoring application, and a complete hardware design and realization. The proposed formation transformation algorithm dynamically solves the TAP in a conditionaltriggering way and constructs repulsion field around obstacles to avoid intercollisions of UAVs. Compared with the traditional static task assignment methods, the TAP is not only solved offline to obtain an initial task assignment with the minimum total flight distance of all UAVs, but is also dynamically solved online to consider the impact of realtime trajectory deviation due to collision avoidance. The collision avoidance between UAVs is set as a trigger condition to reduce the frequency of dynamic task assignment, and the TAP is only solved within a group of local UAVs to avoid a high computational cost of solving a global TAP. In addition, formation drawing and 3D animation simulations are provided in the ADI to facilitate the animation design process. During the light show, the monitoring and control of UAVs is conducted through the realtime monitoring application. The UAVLS hardware consists of subsystems of decisionmaking, realtime kinematic (RTK) global positioning system (GPS), wireless communication, and UAV platforms. The UAV platform is made up of a number of homogeneous quadrotors without onboard collision avoidance sensors for economic reasons. Finally, outdoor experiments using six quadrotors are performed. Details of implementations of centimeterlevel positioning, communication and computation are given. Results show that the developed UAVLS system can successfully complete a light show, and the proposed algorithm is better than the traditional static task assignment one in terms of scalability, efficiency, and performance.
The remainder of this paper is organized as follows. In
Section 2, the formation transformation algorithm is introduced.
Section 3 introduces the software package developed for the UAVsLS.
Section 4 provides details of the selfdeveloped quadrotor prototype and the UAVsLS system. The simulations of proposed formation transformation algorithm and experiments of UAVLS are presented in
Section 5. Finally,
Section 6 gives concluding remarks.
2. Formation Transformation in UAVsLS
In the process of UAVsLS, one of the key steps is the formation transformation between an initial formation and a target one. Since the volume of UAVs cannot be ignored, collisionfree trajectories must be planned for safety concerns. In general, the formation transformation can be decomposed into two subproblems:
TAP: for each UAV, a match between the initial formation and the target one is generated to satisfy certain criteria;
Trajectory planning problem: for each UAV, a collisionfree trajectory is planned based on the match from the TAP.
An example of formation transformation is shown in
Figure 1, where five UAVs transform from a wedge formation to a diamond one. The TAP step is to assign, for each node (i.e., UAV) of the wedge formation, a target node in the diamond formation. The second step is to plan a collisionfree trajectory connecting the initial node and the target one.
2.1. Task Assignment Problem
In general, given N initial nodes and N target ones, there are in total $N!$ possible matches among them. The purpose of task assignment is to find the optimal assignment minimizing a certain criteria.
Define the position of the
ith UAV in the initial formation and the
jth waypoint in the target formation as
The initial and the target formation of all UAVs can be stored in vectors as
Define the assignment map
$\phi :\{1,2,3,\dots ,N\}\to \{1,2,3,\dots ,N\}$, which specifies the target position corresponding to each UAV with
where
${\phi}_{i}=j(i=1,\dots ,N,j=1,\dots ,N)$ means the
jth target position is assigned to the
ith UAV. The TAP is to obtain the optimal match
${\phi}^{*}$ by solving the following optimization problem
where
${\mathbf{t}}_{{\phi}_{i}}$ is the target position assigned to the
ith UAV according to
${\phi}_{i}$.
Since the number of UAVs and target nodes are equal, problem (
6) is linear and can be solved by the Hungarian algorithm within bounded time [
17]. The pseudo code of the Hungarian algorithm is presented in Algorithm 1.
Algorithm 1 The Hungarian algorithm for the TAP 
Input: initial formation $\mathbf{X}$, target formation $\mathbf{T}$ 
 1:
for Each UAV do  2:
for Each target position do  3:
$C(i,j)=\Vert {\mathbf{x}}_{\mathbf{i}}{\mathbf{t}}_{\mathbf{j}}\Vert $  4:
end for  5:
end for  6:
Subtract the minimum value in each row from all the values of its row and get ${C}^{{}^{\prime}}$  7:
Subtract the minimum value in each column from all the values of its column and get ${C}^{{}^{\u2033}}$  8:
for Each row and column do  9:
Repeat Draw lines through appropriate rows and rows and columns  10:
Until All the zero entries of the cost matrix are covered. The minimum number of such lines is m  11:
end for  12:
if$m<N$then  13:
Determine the smallest entry not covered by any line. Subtract this entry from each uncovered row, and then add it to each covered column. Return to Step 8  14:
end if  15:
if$m=N$then  16:
Obtain the optimal assignment matrix ${\phi}^{*}$  17:
end if

Note that the optimal match obtained from the Hungarian algorithm generates straight lines connecting the initial position
${\mathbf{x}}_{\mathbf{i}}$ and the optimal target position
${\widehat{\mathbf{t}}}_{\mathbf{i}}$ without crossovers [
11]. However, since the volume of UAVs cannot be ignored, collisionfree trajectory planning is necessary which would result in trajectory deviations from the straight lines.
2.2. CollisionFree Trajectory Planning
In this paper, we assume that there is no obstacle in the air and the UAVs only need to avoid interUAV collisions. Artificial potential field (APF) has been applied to obstacle avoidance and path planning for decades. In APF, a potential field is constructed where each UAV is attracted by the target and repelled by the obstacles. The attractive, repulsive and the resultant potential field
${U}_{att},{U}_{rep},U$ of the
ith UAV can be written as:
where
$\eta $ and
${k}_{p}$ are attractive and repulsive force gain coefficients, respectively;
${\chi}_{i}$ is the actual position of the
ith UAV;
${\rho}_{o}$ is the minimum distance under which the repulsive potential field does not make effect;
${\rho}_{i}$ is the actual distance to an obstacle of the
ith UAV. The corresponding attractive, repulsive, and the resultant forces
${F}_{att,i},{F}_{rep,i},{F}_{i}$ are:
where
${n}_{i}$ is the number of obstacles detected within a radius of
${\rho}_{o}$ of the
ith UAV at time
t.
2.3. A Dynamic Formation Transformation Algorithm
2.3.1. Graph Theory in Formation Transformation
The communication topology between UAVs during formation transformation can be represented by an undirected graph $G=\{V,E\}$, where $V=\{{v}_{1},{v}_{2},\dots ,{v}_{N}\}$ is the set of UAVs and $E\subseteq V\times V$ is the set of edges representing the adjacent relationship among UAVs, i.e., edges ${e}_{ij}=(i,j)$ and ${e}_{ji}=(j,i)$ exist if the distance between the ith UAV and the jth UAV is less than ${\rho}_{0}$. The adjacent matrix $A\triangleq {\left[{a}_{ij}\right]}_{N\times N}$ contains elements ${a}_{ij}=1$, if there is an edge ${e}_{ij}$. A path between the ith UAV and the jth UAV is a sequence of edges $(i,{j}_{1}),({j}_{2},{j}_{3}),\dots ,({j}_{l},j)$. The graph G is said to be connected if there is a path for each pair of UAVs. A connected component is a subset of the graph which is associated with the smallest partition of the set of UAVs, and each partition is connected. Note that a disconnected graph has more than one component.
2.3.2. Dynamic Formation Transformation Algorithm
The Hungarian algorithm minimizes the total flight distance of all UAVs. One of the limitations of the Hungarian algorithm is that the minimum flight distance is obtained by statically considering straight lines between the initial and the target formations. However, UAVs deviate from such straightline trajectory when avoiding collisions, thus may result in longer flight distance if the task assignment result remains the same. To address this issue, we propose a dynamic formation transformation algorithm by solving the TAP in a conditionaltriggering manner during realtime trajectory planning using APF, as shown in
Figure 2. To avoid heavy computational cost, a global TAP is first solved offline and local TAPs are then formulated and solved online based on latest formation information.
Specifically, for each formation transformation (in general, a UAVsLS animation contains a series of formation transformations), a global TAP in the form of (
6) is firstly formulated offline, seeking a match between the initial and the target formation coordinates of all UAVs, which includes
where
${\mathbf{X}}_{g}$ represents the current formation position coordinates and
${\mathbf{T}}_{g}$ represents the target formation position coordinates.
${N}_{g}$ is the total number of UAVs. The optimal match
${\phi}_{g}^{*}$ is obtained by using Algorithm 1 to solve the global TAP. A global APF is constructed by attracting UAVs to move along its negative gradient direction towards the target formation. During online operation, local repulsion fields are generated by the UAVs within a component.
When UAVs are moving in the air, the members of each component and the number of components may vary. The target position appointed by the current TAP may thus be affected in a component due to deviation from the original straight trajectory to avoid collisions. Therefore, a local TAP based on a conditionaltriggering manner must be formulated and solved. In particular, the local TAP is triggered when there is a component. The connected components of undirected graph can be obtained by the adjacent matrix
${A}_{N\times N}$ [
18,
19]. An improved version of depthfirst search has been proposed to solve the connected components in [
19], and the method is employed to obtain the connected components of undirected graph. An illustrative example is given in
Figure 3, where three components are formulated for six UAVs according to the adjacent relationship given in
Table 1.
Table 2 presents the solutions of the connected components.
Given a component
$k,\phantom{\rule{0.166667em}{0ex}}k=1,2,\dots ,m$ and its UAVs, we define the local TAP matching objectives
${\mathbf{X}}_{l}^{k}$ and
${\mathbf{T}}_{l}^{k}$ as:
where
${\mathbf{X}}_{l}^{k}$ and
${\mathbf{T}}_{l}^{k}$ are the current and assigned target position of the
kth component. The local TAP solves the matching relationship between the current positions and target positions of the neighboring UAVs at discrete sampling instants while they are moving. Minimum flight distance can be maintained by using Algorithm 1 to solve the local TAP.
Consequently, a UAVLS formation transformation (UAVsLSFT) method is developed by combining the Hungarian algorithm and the APF dynamically, as summarized in Algorithm 2, which is simple enough for realtime computations.
Algorithm 2 Dynamic UAVLS formation transformation algorithm 
Input: initial formation $\mathbf{X}$, target formation $\mathbf{T}$ 1:
Obtain the global optimal assignment by Algorithm 1  2:
while at each sampling instant do  3:
if $\rho <{\rho}_{0}$ then  4:
Construct the adjacent matrix A  5:
Compute the connected components by [ 19]  6:
Obtain the local optimal assignment by Algorithm 1  7:
for each UAV do  8:
Compute the resultant force by ( 12)  9:
Obtain the next waypoint  10:
end for  11:
else  12:
for each UAV do  13:
Compute the resultant force by ( 12)  14:
Obtain the next waypoint  15:
end for  16:
end if  17:
if all the UAVs have reached the target position then  18:
Stop  19:
end if  20:
end while

5. Simulation and Experiment Results
In the section, we first present the comparison and analysis between the proposed dynamic formation transformation method and another two methods. Then outdoors experiments using six quadrotors are performed.
5.1. Simulation
Three methods are compared in the simulation. Method 1 is the traditional method which separates the task assignment and trajectory planning, i.e., task assignment is performed offline only before the formation transformation. Method 2* is the proposed dynamic formation transformation algorithm where a global TAP is solved offline once and local TAPs are formulated and solved in connected components online. Method 3 sets as a benchmark where global TAPs are solved online once collision avoidance movement occurs.
The simulations consider a threedimensional environment with the length, width, and height of $50\times 50\times 50$ m. The initial and target states in the formation are randomly generated. The attractive gain coefficient $\eta $ is set to 0.8, and the repulsive gain coefficient ${k}_{p}$ is set to 0.3. The influence radius of repulsion point ${\rho}_{o}$ is set to $2.3$ m. The sampling time is 20 ms. A UAV is considered to have arrived at its target when the distance between them is less than 0.01 m.
The simulation platform is MATLAB2020a running on a PC with AMD 3970X. It should be noted that only a single CPU core is utilized. The simulation is repeated 100 times in which the initial and target positions of the UAVs are randomly generated. For a fair comparison, the initial and target positions are the same for three methods in each test. The average computation time for each iteration, the average flight distance of the UAVs arrived at the target position, the maximum distance moved among the UAVs, the time taken for all UAVs to reach their targets and the number of the UAVs reached the destination are shown in
Table 3.
It can be seen that Method 1 has the shortest average computation time because the TAP is solved only once. The proposed Method 2* has a slightly longer computation time than Method 1 but it is far less than that of Method 3. However, the number of arrived UAVs for Method 2* and Method 3 is almost the same, and is larger than Method 1. The average flight distance and the maximum flight distance of Method 2* is slightly longer than Method 3 by only solving local TAPs in connected components, while Method 1 has the longest average flight distance and maximum fight distance since it solves a global TAP once offline without guaranteeing the optimality of the TAP during collision avoidance. Similarly, Method 1 has the longest arrival time, Method 2* and Method 3 are obviously lower than Method 1 which benefit from the TAP solution during the formation transformation. The targets of part of UAVs are updated after solving the TAP, and the flight distance is shortened so that all drones may transform the desired formation in a faster way. In conclusion, the proposed Method 2* achieves a good tradeoff between performance and efficiency. Method 2* is better than the traditional static Method 1 in terms of flight distance, arrival time and arrived UAVs, and is also computationally efficient when compared to the global optimal Method 3.
Remark 1. The initial and target positions of the UAVs are uniformly randomly generated in the simulation. While most UAVs are allocated to nearby targets with a straightline trajectory, a small number of UAVs need to avoid intercollisions hence deviate from the straightline trajectories. Since the flight distance of the three methods is averaged in Table 3, the distance differences among the three methods seem small. However, the benefit of dynamically solving TAP cannot be ignored, especially in denser and more compact spaces, and for reducing the arriving time of UAVs. Remark 2. Due to the randomness of the initial and target position, the distance between some target points are less than the safe distance. This makes it impossible for the UAVs to arrive at the destination. Moreover, a few UAVs were trapped into extreme point due to the repulsive force of multiple UAVs, which also causes the inability to reach the target position. Therefore, all three methods cannot guarantee that all UAVs arrive at their target positions. To solve this issue, the minimum distance between generated target positions can be configured in the formation transformation to increase the number of UAVs arriving at the target positions. In addition, using Method 2* and Method 3 can reduce the influence of extreme points on the number of arrived UAVs, which is an advantage of our proposed method.
Remark 3. In the simulation, conservative attractive and repulsion coefficients are selected in a compact space with a large number of UAVs generated randomly, for finding a balance between sampling time and smooth trajectories. As a result, the UAVs take a relatively long time (about 60s) to arrive at their targets. Nevertheless, the attraction and repulsion coefficients can be tuned according to UAV specifications and the flight space to reduce the arriving time.
To further understand how the number of UAVs affects computational time, we set the number of UAVs as a single variable, the average computation time for each iteration is shown in
Figure 10. Compared with Method 3, the proposed method has a significant computational cost advantage, and the advantage becomes more obvious as the number of UAVs increases.
Next, two case studies are given to better demonstrate the advantages of the proposed method in fight distance and the number of arrived UAVs. The trajectory of using Method 3 is not shown since it is indistinguishable from that of the Method 2*. In these case studies, the attractive gain coefficient is $\eta =0.8$; the repulsive gain coefficient ${k}_{p}$ is $0.3$, and the other parameters are kept the same.
5.1.1. Case A
In Case A, three UAVs realize formation transformation using Method 1 and Method 2*. The initial positions of the three UAVs are
${\mathbf{x}}_{1}=[1.2;3.9;6]$,
${\mathbf{x}}_{2}=[1.2;2.9;6]$, and
${\mathbf{x}}_{3}=[1;6;6]$. The positions of the target formation are
${\mathbf{t}}_{1}=[0;0;6]$,
${\mathbf{t}}_{2}=[1;2;6]$, and
${\mathbf{t}}_{3}=[1;2;6]$. The trajectories of UAVs are displayed in
Figure 11 and the distance between the UAVs is given in
Figure 12.
It can be seen from
Figure 11 that all UAVs arrived at their offline assigned targets using Method 1, despite the significant deviations from the planned straightline trajectory. Using Method 2*, local TAPs are solved online such that UAV 1 and UAV 3 are assigned to different targets while avoiding collisions. The flight distances of the three UAVs are shown in
Table 4. Compared with Method 1, the average flight distance and the arrival time of Method 2* are reduced up to
$11\%$ and
$15\%$, respectively. Both methods can maintain the safe distance among UAVs during the formation transformation.
5.1.2. Case B
In Case B, four UAVs are simulated to construct a desired formation, where the initial positions are
${\mathbf{x}}_{1}=[1.2;3.2;6]$,
${\mathbf{x}}_{2}=[1.2;2.9;6]$,
${\mathbf{x}}_{3}=[0.8;8;6]$ and
${\mathbf{x}}_{4}=[1.2;6;6]$. The positions of the target formation are
${\mathbf{t}}_{1}=[1;2;6]$,
${\mathbf{t}}_{2}=[1;2;6]$,
${\mathbf{t}}_{3}=[0;0;6]$, and
${\mathbf{t}}_{4}=[0;2;6]$. The trajectories of the four UAVs and the distance between the UAVs are shown in
Figure 13 and
Figure 14, respectively.
Using Method 1, UAV 4 stops at a local minimum and is unable to arrive at the target, as shown in
Figure 13a. In
Figure 13b, local TAPs are triggered on and solved during collisionavoidance. All four UAVs have arrived at the target. The flight distance of the UAVs is shown in
Table 5. It can be seen that using Method 2* increases the flight distance in this case, yet the number of UAVs that can arrive at the target position is increased and the arrival time is reduced up to
$34\%$. The proposed dynamic task assignment and trajectory planning algorithm successfully helps UAVs escape from the local minimum point.
5.2. Experiment
In this work, an experiment with UAVLS is carried out consisting of 6 quadrotors transformed from a random initial formation to a desired formation. A laptop equipped with i77700hq CPU and 8GB RAM is used as the GCS for animation design, simulation, and realtime monitoring and control.
Realtime measurement data are sent from UAVs to the GCS at a frequency of 10 HZ and the collisionfree reference trajectories are planned by the proposed dynamic formation transformation algorithm, which is sent back from the GCS to the UAVs at a frequency of 5HZ. The attractive gain coefficient $\eta $ is set to 0.8, and the repulsive gain coefficient ${k}_{p}$ is set to 0.3. The influence radius ${\rho}_{o}$ of other UAVs is set to 2 m.
In this experiment, the multithreaded programming feature of C# is used. The GCS computer has 4 cores and 8 threads, in which 6 threads are enabled, each of which computes reference trajectories for a quadrotor in parallel. The initial and target formation positions of the quadrotors are given in
Table 6.
Onsite photos of the UAVLS at times 0, 14, 22, 40 s are shown in
Figure 15, where (a) shows all quadrotors are steadily waiting at the initial formation; (b) shows all quadrotors started to avoid collisions and the local TAPs are solved in two connected components, which are formed by the 1st, 2nd, 3rd UAV and the 4th, 5th, 6th UAV, respectively; (c) shows the 1st, 3rd, 4th and 5th UAV are moving to the target waypoints; and (d) all quadrotors have arrived at their targets.
Figure 16 presents the actual trajectories of the 6 quadrotors on the X–Y plane during the experiment.
Figure 17 illustrates the minimum distance of each quadrotor from the others during the formation transformation. It can be concluded that the proposed task assignment and trajectory planning algorithm are able to dynamically plan the trajectories of UAVs when they deviate from the initial trajectory due to collision avoidance. The safe distance is always maintained during the experiment.
5.3. Discussion
In
Section 5.1, comparative simulation experiments were carried out among the three methods. Compared with Method 1 which decouples task assignment and trajectory planning, the proposed Method 2* has advantages in terms of shorter flight distance, shorter arrival time and more arrived number of UAVs. The disadvantage compared with Method 1 is the increased computational time. Compared with the Method 3 which solves a global TAP once collision avoidance movement occurs, the proposed Method 2* has slight disadvantages in flight distance, arrival time, and number of arrived UAVs. Yet, the proposed Method 2* has obvious advantages over Method 3 in terms of computational time and the scalability, as shown in
Figure 10. In
Section 5.2, an experiment with UAVLS successfully verified the effectiveness of the proposed method in practice.
Through the above analysis, the proposed method achieves a good tradeoff between performance and efficiency which is desired for most multiUAV systems with a number of controlled UAVs and limited computing resources.
6. Conclusions and Future Work
In this paper, a UAVLS system is developed including a dynamic task assignment and trajectory planning algorithm, a software toolkit and hardware design and realization. In particular, the dynamic formation transformation algorithm works in a conditionaltriggering manner where local TAPs of connected components are constructed and solved online to achieve minimum flight distance and to reduce the impact of collision avoidance on task assignment. To build such a UAVLS system, a wireless transmission subsystem was established to transfer data based on the MAVLink protocol between the UAVs and the GCS, and the RTKGPS technique was utilized to provide centimeterlevel positioning accuracy for UAV. Simulations and experiments were conducted to verify the proposed UAVLS system.
In future work, efforts will be devoted to cooperative formation transformation techniques. Flight distance balancing issues will also be considered, i.e., the difference between the flight distance of each UAV is minimized to avoid battery energy imbalance.