A Two-Stage Distributed Task Assignment Algorithm Based on Contract Net Protocol for Multi-UAV Cooperative Reconnaissance Task Reassignment in Dynamic Environments

Multi-UAV systems have been widely used in reconnaissance, disaster relief, communication, and other fields. However, many dynamic events can cause a partial failure of the original mission during the mission execution process, in which case task reassignment should be carried out. How to reassign resources and tasks in multi-dynamic, multi-target, and multi-constraint events becomes a core issue in the enhancement of combat efficiency. This paper establishes a model of multi-UAV cooperative reconnaissance task reassignment that comprehensively considers various dynamic factors such as UAV performance differences, size of target areas, and time window constraints. Then, a two-stage distributed task assignment algorithm (TS-DTA) is presented to achieve multi-task reassignment in dynamic environments. Finally, this paper verifies the effectiveness of the TS-DTA algorithm through simulation experiments and analyzes its performance through comparative experiments. The experimental results show that the TS-DTA algorithm can efficiently solve the task reassignment problem in dynamic environments while effectively reducing the communication burden of UAV formations.


Introduction
Compared with manned aircrafts, UAVs have the advantages of low cost, strong concealment, and being unmanned [1][2][3].They are widely used in reconnaissance, disaster relief, communication, and other fields [4][5][6][7].However, with the continuous expansion of task scales, in many cases, a single UAV cannot complete its tasks efficiently [8].Multi-UAV systems have higher efficiency and stronger robustness than single UAVs [9,10].In order to improve the efficiency and success rate of reconnaissance tasks, multi-UAVs often perform them cooperatively, especially in the field of military reconnaissance.Task assignment is a key issue for multi-UAVs to cooperate efficiently [11].
Before a reconnaissance mission begins, multi-UAV systems need to assign tasks according to static prior information and obtain the initial mission plan.In recent years, many researchers have established models and proposed solutions to such task assignment problems [12][13][14][15][16].However, there are many uncertainties in the actual task execution process, such as target movement, drone damage, and many other dynamic events [17].When these dynamic events occur, multi-UAV systems need to adjust the initial mission plan to adapt to these changes, a process called task reassignment [18].
Many researchers have established relevant models of task reassignment in dynamic environments.Yang et al. [17] established a task reassignment model for multi-UAVs performing search and rescue (SAR) missions, considering five dynamic events: finding new tasks, canceling original tasks, updating task coordinates, updating task deadlines, and updating task durations.However, the model only involved dynamic changes of targets and did not consider damage to UAVs during the mission.In the context of multi-UAV cooperative task assignment in SAR missions, Chen et al. [19] considered time sensitivity and dynamic environments and studied the problem of multi-UAV task reassignment after the emergence of a new target.However, their model did not consider the simultaneous occurrence of multiple targets.Oh et al. [20], in the context of suppression of enemy air defense (SEAD) missions, considered the problem of task assignment in the case of pop-up threats and UAV loss.However, the performance differences of UAVs were not considered in the model.Zhang et al. [21] studied the dynamic task assignment problem in the context of multi-UAVs attacking multiple ground targets cooperatively and considered the emergence of new targets and sudden UAV failure.However, the time window constraint of targets and movement of targets were not considered in the model.
Due to the nature of armed conflict, multi-UAVs are often in a dynamic environment when performing a cooperative reconnaissance mission.Existing multi-UAV task reassignment models in dynamic environments are not directly used for the modeling of multi-dynamic, multi-target, and multi-constraint events.In order to better describe the problem of multi-UAV cooperative reconnaissance task reassignment and discuss the literature on the above subject, this paper establishes a model of multi-UAV cooperative reconnaissance task reassignment in dynamic environments (DE-MCRTR) based on our previous work [22].The DE-MCRTR model addresses task reassignment problems in dynamic events, including UAV damage, new target occurrence, location change of the target, as well as time window change of the target, while comprehensively considering various factors such as UAV performance differences, size of target areas, and time window constraints.
Multi-UAV task assignment is an NP-hard problem [23,24].Commonly used task assignment algorithms can be divided into two categories [5,19]: one comprises centralized algorithms and the other comprises distributed algorithms.Centralized algorithms have the ability of global optimization [24] but their computational demand is high [17].So, the solution speed of centralized algorithms is low, which causes their slow responsiveness to dynamic changes [25].Although distributed algorithms cannot obtain global optimal solutions [24], their computational complexity is relatively small and their stability is higher [26].Since task reassignment in dynamic environments is carried out in the process of mission execution, higher requirements are placed on the speed of reassignment.Thus, researchers usually use distributed task assignment algorithms to solve the problem of task reassignment in dynamic environments.The most common are consensus-based auction algorithms and market-based algorithms [27].Among them, the most widely used are consensus-based bundle algorithms (CBBAs) [28] and the contract net protocol (CNP) [29].
Aiming at the problem of UAV swarms performing cooperative reconnaissance-and-attack tasks on multiple targets in complex and uncertain combat scenarios, Qin et al. [30] proposed a cooperator determination mechanism and a selection mechanism of sequential tasks, and extended the contract net protocol to design an event-triggered dynamic task allocation strategy.This strategy adopted the selection mechanism of sequential tasks and assigned tasks one by one.When the number of tasks was large, it was difficult to ensure efficiency.Zitouni et al. [31] proposed a distributed multi-UAV task assignment algorithm by combining a consensus-based bundle algorithm and an ant colony algorithm.However, the algorithm was only applicable if some original tasks were cancelled.Using the state feedback, gradient descent, and primal-dual methods, Deng et al. [32] designed a distributed algorithm for high-order agents to perform resource allocation tasks autonomously.Zhang et al. [33] introduced a sales contract, exchange contract, and replacement contract into the original contract net protocol and proposed an algorithm based on a hybrid contract net protocol to achieve dynamic task reassignment.This method had a low proportion of feasible solution combinations in the solving process; so, the solving efficiency was low when the number of targets is large.Yang et al. [17] proposed a partial reassignment algorithm based on the PI algorithm to realize online task reassignment after the target dynamic changes during mission execution.The algorithm was used for a single task reas-Sensors 2023, 23, 7980 3 of 26 signment situation, and it was not suitable for multiple tasks.Gao et al. [34] designed a task reassignment algorithm based on the contract net protocol to solve the task reassignment problem when new targets appear or UAV damage occurs during task execution.However, with the expansion of the scale of the problem, the number of UAV communications would be multiplied, resulting in poor reliability of the system.
There are multiple targets that need to be assigned in the DE-MCRTR model, and some targets contain time window constraints.In order to realize rapid reassignment of multiple targets, reduce the communication burden of multi-UAV formation, and ensure the quality of task assignment to a certain extent, this paper proposes a two-stage distributed task assignment algorithm based on contract net protocol (TS-DTA).The goal of the first stage and the second stage of the TS-DTA algorithm is to assign the targets without a time window and the targets with a time window, respectively.Based on the original contract net protocol, the bidding strategy with bidding benchmark is introduced in the first stage to reduce the communication burden of UAV formation, the bidding strategy based on route distance is proposed in the first stage to realize synchronous assignment of multiple tasks, and the cyclic bidding strategy based on task timing is proposed in the second stage to achieve efficient assignment of targets with time window constraints.
The rest of this paper is organized as follows: In Section 2, the background of multi-UAV cooperative reconnaissance task reassignment problem is introduced, and the task reassignment model is established.In Section 3, the TS-DTA algorithm is introduced in detail.In Section 4, a series of simulation experiments is conducted.In Section 5, the conclusions and prospects are given.

Task Reassignment Model
This section first introduces the background of the task reassignment problem of multi-UAV cooperative reconnaissance, and then establishes the task reassignment model according to dynamic events.

Background
The reconnaissance in this paper refers to using the photoelectric equipment and radar carried by UAVs to obtain real-time video and image information of targets so as to provide information support for decision-making for the command center.After the initial mission plan is obtained based on static prior information, UAVs start from their ship platform to each target to perform reconnaissance.The circular reconnaissance route is adopted by UAVs when performing reconnaissance.The radius of circular route is dependent on the danger degree and the size of each target.After completing all tasks, UAVs return to the ship platform.However, in the actual mission execution process, the confrontation between the enemy and ourselves would lead to many dynamic events, such as damage to the UAV, movement of targets, and so on.These dynamic events may partially invalidate the original mission plan.In order to ensure the completion of all tasks, it is necessary to reasonably assign unfinished tasks to currently dispatched UAVs.Combined with the actual mission, this paper focuses on the following four kinds of dynamic events.
(1) UAV damage During the mission, a drone is suddenly hit by enemy air defense forces, making it impossible to complete the remaining tasks.
(2) New target occurrence During the reconnaissance mission, early warning aircraft or ships may find some new targets that need to be reconnoitered.
(3) Location change of target In order to enable UAV formation to quickly capture the moving targets, the provider of prior information needs to update the location of those targets in time.
(4) Time window change of target In order to realize an effective cooperation with other combat forces, UAVs need to reconnoiter some targets within a given time window.When the time window of some targets are advanced or pushed back, the original plan may no longer meet the requirements of the new time window.

DE-MCRTR Model
Based on the previous work [22], this article introduces dynamic events and establishes a multi-UAV cooperative reconnaissance task reassignment model in a dynamic environment.The specific model is as follows: 2.2.1.Basic Information of the Model (1) After the reconnaissance mission begins, each UAV starts from the ship platform at the same time, and the departure time is t = 0.
(2) UAVs that obtain tasks in the initial task assignment are called dispatched UAVs.After the dynamic event occurs, all dispatched UAVs participate in the task reassignment.
(3) Each task is completed by a single UAV.
(4) Every UAV can obtain its own location in real time.

UAV Model
The UAVs are initially carried by the ship platforms.The number of ship platforms is n s , the ship platform set is S = {S 0 , S 1 , . . . ,S n s −1 }, and the coordinate of S i is X S i , Y S i .The types and numbers of UAVs carried by each platform are known.The number of UAVs is n u , the UAV set is U = {U 0 , U 1 , . . . ,U n u −1 }, the cruising speed of U i is v i , the maximum range of U i is mr i , and the initial coordinate is the coordinate of the ship platform.Plan = {P 0 , P 1 , . . . ,P n u −1 } is the task scheme of UAV formation, which will be updated during task execution.The number of dispatched UAVs is When dynamic events occur, the coordinate of U i is X U i , Y U i ; the leftover maximum range of U i is mrl i ; the list of finished tasks of U i is ; the list of leftover tasks of U i is and n i f , n i l are the number of finished tasks and leftover tasks of U i , respectively.

Target Model
The initial number of targets is n t , the target set is T = {T 0 , T 1 , . . . ,T n t −1 }, and the approximate coordinate of T j is X T j , Y T j .The reconnaissance time window for T j with time window constraint is TW j = tws j , twe j .The radius of the circular reconnaissance route for UAV to reconnaissance target T j is ar j .If n new new targets appear, the new target set is , and the radius of circular reconnaissance route is ar T Meanwhile, these new targets need to be added to the original target set T, and the updated target set is T = {T 0 , T 1 , . . . ,T n t −1 , T n t , . . . ,T n t +n new −1 }.If the location of target T j is changed, its coor- dinates need to be changed to the new location X n T j , Y n T j .If the time window of target T j is changed, its time window needs to be changed to the new time window tws n j , twe n j .

Decision Variable
Referring to our previous work [22], define re ij as the decision variable, where i = 0, 1, . . ., n u − 1 and j = 0, 1, . . ., n t − 1.The value of re ij is 0 or 1; re ij = 1 indicates that U i conducts reconnaissance on target T j and vice versa.

Constraint Condition
In engineering fields, constraints occur naturally, such as resource limitations, which the agents are subject to [35].The purpose of task reassignment is to reasonably assign tasks that (2) Leftover maximum range constraint After U i receives the new mission plan P i n , it starts from the current location X U i , Y U i and flies to each target point in turn to perform reconnaissance.The remaining flight distance of U i is recorded as f d i ; suppose , f d i is expressed as Equation ( 2): where represents the sum length of the circular reconnaissance route of targets, ∑ represents the sum distances of transferring between targets, and L T n i i ,S i U represents the flight distance of the UAV from the last target to its ship platform.
The leftover maximum range needs to be updated in real time.According to the discussion in Section 2.2.2, the list of finished tasks of U i is and the mrl i of U i is expressed as Equation (3): where L S i U ,TF 1 i represents the distance from the platform to the first finished target, and represents the distance from the last finished target to the current location of U i .If In order to ensure that U i can successfully complete the mission, it is required that the remaining flight distance of U i cannot be greater than its remaining maximum range: (3) Time window constraint Suppose that the time when a dynamic event happens is t d .According to Plan new , each UAV can obtain the time period for reconnaissance of its targets.Take U i as an example, supposing . The start time of reconnaissance for T j i is ts j i , and the end time of reconnaissance for T j i is te j i .Then, ts j i and te j i are expressed as Equations ( 5) and (6): In order to intuitively evaluate the pros and cons of the final mission plan, we establish the cost function of UAV formation by referring to the previous literatures [22,36].
where CT represents the mission completion time, which is the moment that the UAV complete the last task; AT represents the average flight time-that is, the average time spent by each UAV from departure to return to the platform (except damaged UAVs); and α, β ∈ [0, 1] are proportional coefficients satisfying α + β = 1, based on the discussions in literatures [12,37], which are α = β = 0.5 here.Plans that do not satisfy the constraints cannot be used as the final mission plan, considering that a good mission plan should enable UAV formation to complete the task with less consumption.Referring to [36], γ × 10 4 is used to penalize the plans that do not satisfy the constraints in Section 2.2.5.γ is the constraint factor, and the value of γ is 0 or 1: if the mission plan does not meet the constraints, γ = 1, so that f c increases by 10 4 ; otherwise, γ = 0.
(2) Cost of single UAV Due to the distributed architecture used in online task reassignment, each UAV cannot obtain the overall task mission plan.When updating its mission scheme, a single UAV cannot measure the merits of its scheme through the cost of UAV formation.Therefore, this paper introduces the single cost function to measure the cost of a single UAV.The cost of U i depends on its task scheme, denoted as c i , which is calculated by Equation ( 9): where ct i represents the time from now to U i completing all tasks according to its scheme P i ; rt i represents the time from now to U i returning back to platform; ω 1 , ω 2 are proportional coefficients, ω 1 = ω 2 = 0.5 here.γ i is the constraint factor: if P i is satisfied with the constraints, γ i = 0; otherwise, γ i = 1.

Two-Stage Distributed Task Assignment Algorithm Based on CNP
The advantages of the TS-DTA algorithm mainly include the following three aspects.The first is to achieve the assignment of multiple targets.The two-stage task assignment strategy can avoid the conflicts of multi-targets in the task allocation process, ensuring smooth task allocation.Secondly, tasks can be assigned efficiently.The algorithm is designed based on CNP.The original CNP is improved such that the TS-DTA algorithm can achieve rapid reassignment and ensure the quality of the solution to a certain extent.Thirdly, the communication burden of UAV formation is relatively small.By introducing the bidding benchmark and the bidding strategy based on route distance, the communication times of UAV formation during task assignment are reduced.

Tasks to Be Assigned
Task reassignment is carried out when some tasks cannot be completed due to dynamic events.When executing task reassignment, partial reassignment can be carried out on the basis of the previous mission plan, which can save computing resources and improve the speed of calculation [17,33].Therefore, the TS-DTA algorithm proposed in this paper adopts the partial reassignment strategy-that is, only part of the targets that need to be reassigned are assigned and these targets are reasonably added to the existing task scheme of dispatched UAVs U dispatch .
According to the description in Section 2.1, the targets directly related to dynamic events need to be reassigned, which should be added to the list to be assigned TL un and removed from the current scheme of their corresponding UAVs.Removing some targets may lead to the reconnaissance time of subsequent targets violating the time window constraints, which will interfere with task reassignment.Therefore, it is necessary to remove the subsequent targets that do not meet the constraints from the current task scheme and add them to TL un .These targets are called the additional targets of task reassignment.For example, the current task scheme of U i is . When TL 2 i needs to be reassigned, we remove TL 2 i from TL i ; so, TL i = TL 1 i , TL 3 i , . . ., TL n i l i , supposing that TL 4 i , TL 6 i contain time window constraints.According to the new task scheme, the reconnaissance time of TL 4 i , TL 6 i will be earlier than the beginning of the time window.Then, it is necessary to remove TL 4 i , TL 6 i from TL i to TL un .Based on the above discussion, the sources of TL un include the leftover targets of damaged UAVs, new targets, the targets whose location changed, the targets whose time window changed, and additional targets.After a dynamic event occurs, the above targets need to be added to TL un .The additional targets are confirmed by UAV formation during the task reassignment process.

The Manager UAV and Contractor UAV
When the contract net protocol is used for task assignment, UAVs obtain a conflict-free solution by negotiating with each other.During this period, a manager UAV U m needs to be specified as the manager in the negotiation process.The other UAVs of U dispatch are contractor UAVs, which are denoted as CU.CU bids for tasks issued by U m during the assignment process.In addition, in the algorithm proposed in this paper, the manager UAV U m itself also participates in the task bidding.
From the perspective of balancing the communication distance between UAVs, this paper takes the UAV closest to the center of dispatched UAV formation as the manager and the remaining UAVs as the contractor.During the mission, U m obtains the location and task execution process of each UAV in real time by communicating with CU and switches the manager's identity in time according to the relative location between UAVs so as to dynamically update U m and CU.

The Framework of TS-DTA
Based on the previous discussion, some of the targets in TL un may contain time window constraints.If the targets with time window constraints are assigned first, the previously assigned targets may violate time window constraints when other targets are inserted later.These targets need to be assigned again, thus affecting the smooth progress of task assignment.Therefore, this paper adopts the two-stage assignment strategy.Targets without time window constraints are assigned in the first assignment stage.On this basis, the second assignment stage is carried out to assign targets with time window constraints.
After manager UAV U m obtains the list of tasks to be assigned TL un , it is first divided into two sub-lists according to whether the target contains a time window constraint: the task list without time window constraints is FTL un ; the task list with time window constraints is WTL un .At the same time, U m removes every target in TL un from the existing scheme P i of the corresponding UAV U i to obtain the current mission plan, Plan = {P 1 , P 2 , . . . ,P nu }.In particular, for a damaged UAV U b , its task list P b needs to be emptied.The specific task assignment architecture is shown in Figure 1.
the task list without time window constraints is  ; the task list with constraints is  .At the same time,  removes every target in  ing scheme  of the corresponding UAV  to obtain the current missio  ,  , … ,  .In particular, for a damaged UAV  , its task list  needs The specific task assignment architecture is shown in Figure 1.The objective of the first assignment stage is to assign the targe .  needs to determine the number of targets in  first.It n posed in two cases: If there is only one target  in  , the bidding stra ding benchmark is used for task assignment, as detailed in Section 3.4   2 targets in  , the bidding strategy based on route distance i assignment, as detailed in Section 3.4.3.In addition,  needs to add add to  in the first assignment stage; the specific process is described in and 3.4.3.
The objective of the second assignment stage is to assign the targe  .In the second stage, the cyclic bidding strategy based on task tim task assignment, as detailed in Section 3.5.The objective of the first assignment stage is to assign the targets in FTL un to U dispatch .U m needs to determine the number of targets in FTL un first.It needs to be disposed in two cases: If there is only one target T x in FTL un , the bidding strategy with bidding benchmark is used for task assignment, as detailed in Section 3.4.2.If there are n (n ≥ 2) targets in FTL un , the bidding strategy based on route distance is used for task assignment, as detailed in Section 3.4.3.In addition, U m needs to add additional targets to WTL un in the first assignment stage; the specific process is described in Sections 3.4.2and 3.4.3.
The objective of the second assignment stage is to assign the targets in WTL un to U dispatch .In the second stage, the cyclic bidding strategy based on task timing is used for task assignment, as detailed in Section 3.5.Before we begin to introduce the strategies in the first assignment stage, we need to introduce the single-target insertion method.This method is designed to obtain the optimal new scheme BP j i and the minimum cost increment mdc j i for U i to insert T j into the current scheme P i .There may be several targets in P i already; thus, there are different positions of inserting T j into P i .The method is to traverse each insertable position of P i and measure the quality of insertion scheme by comparing the cost of U i .After the new target is inserted into the current scheme of U i , we can obtain a new task scheme of U i .The pseudo-code of this method is shown in Algorithm 1.

Algorithm 1 Single-target Insertion Method
U i calculates the cos t c i according to P i by Equation ( 9) 3 For k in range (0, n) do 4 CoP i = copy (P i ) 5 U i insert T j into CoP i at the position k 6 U i calculates the new cos t according to CoP i by Equation (9), denoted as End for Based on the above discussion, before T j is inserted into P i , P i satisfies the constraint conditions.If mdc j i ≥ 10 4 , it means that the task allocation scheme does not satisfy the constraints after target T j is inserted into P i .

Bidding Strategy with Bidding Benchmark
The contract net protocol enables UAVs to negotiate with each other by simulating the bidding process of the market mechanism.One round of bidding includes four stages: initialization, task bidding, winning bids, and signing contracts [38].The manager and each contractor need to communicate three times in every round of bidding.In this way, with the increase in the number of UAVs, the communication burden of U m is heavier, which reduces the stability of the UAV system.Therefore, this paper proposes a bidding strategy with a bidding benchmark, which requires U m to publish the bidding benchmark at the same time when publishing bidding information.Before bidding, every U i in CU compares its bid with the bidding benchmark.If its bid does not meet the bidding benchmark, U i directly abandons the bidding.In this strategy, the number of communications between UAVs can be reduced.The specific steps are as follows: (1) U m verifies whether P m meet constraints or not.If P m does not meet constraints, U m removes the targets with time window from P m into WTL un .Then, U m finds the best scheme BP m and the minimum cost increment mdc m by inserting T x into P m by single-target insertion method, and the mdc m acts as bidding benchmark.At last, U m sends T x , mdc m , and Plan to every UAV of CU.
(2) After U i (U i ∈ CU) receives the bidding information sent by U m , U i first verifies whether P i meets the constraints or not.If P i does not meet the constraints, U i removes the targets with time window from P i into WT i , which will be sent to U m later.Then, U i find the best scheme BP i and the minimum cost increment mdc i of inserting T x into P i by single-target insertion method.At last, U i compares mdc i with mdc m .If mdc i is smaller than mdc m , and mdc i is smaller than 10 4 , then U i sends BP i , mdc i , and WT i to U m ; otherwise, U i sends WT i to U m and gives up the bidding.
(3) After U m receives the bidding information sent by CU, U m puts the additional targets in all received WT i into WTL un .If all U i in CU give up the bidding and mdc m is smaller than 10 4 , T x is assigned to U m .If all U i in CU give up the bidding and mdc m is bigger than 10 4 , T x fails to be assigned.Otherwise, U m compares all received mdc and finds the U best with the min mdc, assigns T x to U best , updates the task scheme of U best (P best = BP best ), sends the losing signal to those U i that participated in the bidding (except U best ), and sends the winning signal to U best .
According to the bidding strategy with bidding benchmarks, when the contractor UAV in CU finds that its bid exceeds the bidding benchmark, it directly abandons the bidding, and the U m no longer needs to transmit the information of bidding failure to it.The pseudo-code is shown in Algorithm 2. The 'Parallel' in the pseudo-code indicates that the calculation of each UAV is concurrent.This is because the UAV formation adopts a distributed architecture; so, calculation can be carried out independently based on the computing resources of each UAV.Therefore, the calculation of each UAV is in no order and they do not interfere with each other.When there are multiple targets to be allocated, there will be repeated communications between U m and CU, if targets are tendered one by one.As the number of targets and UAVs increases, the communication burden will be heavy and the stability will be poor.In order to better complete synchronous assignment of multiple targets, this paper proposes a bidding strategy based on route distance.In this strategy, U m assigns bids according to the relative distance between the remaining waypoints of U dispatch (including U m ) and the targets to be allocated.The specific steps are as follows: (1) U m sends FTL un , Plan to every UAV of CU first, then verifies whether P m meet the constraints or not.If P m does not meet constraints, U m removes the targets with time window from P m into WTL un .Finally, the distance from each route point contained in P m to each target point in FTL un is calculated, and the distance matrix DT m is obtained.
(2) After receiving the bidding information sent by U m , U i (U i ∈ CU) first verifies whether P i meets the constraints or not.If P i does not meet constraints, U i removes the targets with time window from P i into WT i , which will be sent to U m later.Then, U i calculates the distance of every route point in P i to each target in FTL un and obtains the distance matrix DT i .Finally, the DT i and WT i are sent to the manager UAV U m .
(3) After U m receives the bidding information sent by CU, U m puts the additional targets in all received WT i into WTL un .Then, all the distance matrices (including DT m ) are processed to obtain the task assignment scheme, Assign = {A 1 , A 2 , . . . ,A n }.Assign is a two-dimensional list, in which the elements in sub-list A j are the serial numbers of UAVs in U dispatch .The serial numbers are arranged according to the minimum route distance from each UAV to the target T j in ascending order.
(4) U m assigns targets to UAVs according to Assign.At the kth assignment (first set k = 1), U m directly assigns each target to the kth UAV in the corresponding sub-list of Assign for bidding.Thus, U m obtains the assigned task list Then, U m sends the AL i k to each corresponding UAV U i and processes the task list assigned to itself.
(5) After U i U i ∈ U dispatch receives AL i k , for every task T j in AL i k , the single-target insertion method is adopted to find the optimal scheme BP j i and the minimum cost increment mdc j i of inserting T j into P i .If mdc j i is smaller than 10 4 , update the task scheme of U i (P i = BP j i ), remove T j from AL i k , and try to insert the next target in AL i k .Otherwise, try to insert the next target in AL i k directly.After all the targets in AL i k are traversed once, the remaining AL i k is sent back to U m .(6) After obtaining feedback from all U dispatch , U m merges the targets in all AL i k into a new FTL un .Then, it is judged whether FTL un is an empty set: if it is empty, the assignment of FTL un is completed; if not empty, let k = k + 1, and then repeat steps (3)- (5).Continue this process until FTL un is an empty set, meaning the task assignment is successful; otherwise, if k > n d (n d is the number of dispatched UAVs), it means the remaining targets in FTL un failed to be assigned.
According to Formula (9), for targets without a time window constraint, the main factor affecting the assignment result is distance.From the perspective of reducing flight distance of UAVs, the target should be assigned to the UAV with the smallest flight distance added by completing the target.Therefore, the bidding strategy based on route distance is proposed.The manager U m directly assigns UAVs to bid for targets, which ensures the quality of the task scheme while reducing communication burden.The pseudo-code is shown in Algorithm 3. Tasks in FTL un failed to be assigned

The Second Assignment Stage
The second assignment stage mainly relies on the cyclic bidding strategy based on task timing to complete the task assignment since targets in WTL un are constrained by time windows, and the end times of time windows are different.The end time of the time window of T j is denoted as twe j .It is a better choice to first assign the targets with an earlier end time of time window.Otherwise, some subsequent targets may need to be inserted ahead of the targets assigned previously.This insertion would change the mission process.As a result, the reconnaissance time of previously assigned tasks may not satisfy its time window constraint.In order to ensure that the task assignment can be carried out more smoothly, the targets in WTL un are firstly sorted according to the end time of time window, and then allocated one by one.At the same time, after the post-order targets are assigned, the pre-order targets that fail to be assigned could be successfully allocated.In order to enable more targets to be successfully assigned, we use a cyclic bidding method.In addition, in order to reduce the communication burden and improve the efficiency of assignment, similar to Section 3.4.1, the cyclic bidding strategy introduces a bidding benchmark when assigning each target in WTL un .The specific process is as follows: (1) U m finds the best scheme BP k m and the minimum cost increment mdc k m of inserting the kth (first set k = 1) target in WTL un into P m by single-target insertion method.The mdc k m acts as bidding benchmark.Then, U m sends T k w , mdc k m , and Plan to every UAV of CU.
(2) After receiving the bidding information sent by U m , U i (U i ∈ CU) finds the best scheme BP k i and the minimum cost increment mdc k i of inserting T k w into P m by single-target insertion method.Then, U i compares mdc k i with mdc k m .If mdc k i is smaller than mdc k m , and mdc k i is smaller than 10 4 , then U i sends BP k i and mdc k i to U m ; otherwise, U i sends a give up signal to U m and gives up the bidding.
(3) After U m receives the bidding information sent by CU, U m compares all received mdc and finds the U best with the min mdc; then, it assigns T k w to U best , updates the task scheme of U best (P best = BP best ), sends losing signal to those U i that participated in the bidding (except U best ), and sends winning signal to U best .Then, T k w is removed from WTL un .If all U i in CU give up the bidding and mdc k m is smaller than 10 4 , T k w is assigned to U m .(4) Repeat steps (1)-(3) (increase k by 1 each time) until all targets in WTL un complete a traversal.
(5) Determine whether WTL un is an empty set: if yes, the task allocation is complete; otherwise, it is judged whether the target in WTL un is consistent with the target after the last traversal.If not, let k = 1 and repeat steps (1)-(4); if yes, it means that the existing tasks in WTL un can no longer be successfully assigned, the second assignment stage is terminated, and targets in WTL un failed to be assigned.
The pseudo-code of this strategy is shown in Algorithm 4.

Performance Analysis of TS-DTA
In order to comprehensively analyze the performance of the TS-DTA algorithm, a series of experiments are conducted based on the DE-MCRTR model.All algorithms are implemented in Python 3.9, the IDE is PyCharm, the computer CPU is an AMD Ryzen 5-5600H, and the CPU clock speed is 3.30 GHz.The distance unit in the text is kilometer (km) and the time unit is second (s).In order to simplify the expression, the units are omitted in the following text.

Background Information
In order to verify the effectiveness of the TS-DTA algorithm in the DE-MCRTR model, we conducted a set of simulation experiments.Referring to existing literature [12,22,37], this article assumes that there are four ship platforms, each equipped with two UAVs.The cruising speeds and maximum ranges of different types of UAVs are different.The ship platforms are distributed in the area of [0, 100] × [0, 100]; we randomly initialize the location of the four platforms.The initialization information and related parameters are shown in Table 1.The targets are distributed in the area of [100, 300] × [100, 300].The radius of the circular routes of targets are initialized randomly within [1, 3].We assume that there are twenty targets, of which five targets contain time window constraints.Similarly, this article randomly initializes the locations of the twenty targets within the given range and randomly selects five targets to set the time window constraints.The initialization information of targets is shown in Table 2. Before the mission starts, the offline task assignment algorithm is first used to obtain the initial mission plan: Plan = [[2, 13, 5, 4], [6,15,12], [1,14], [7,16,17,18], [], [], [8,19,9], [0, 10,11,3]].The Plan is a two-dimensional list, where the sub-list represents the scheme of each UAV.The numbers in the sub-list represent the targets' number.The same is true later.According to the initial mission plan, a total of six UAVs are dispatched to carry out the reconnaissance mission:  Before the mission starts, the offline task assignment algorithm is first used to obtain the initial mission plan:  = [[2, 13, 5, 4], [6,15,12], [1,14], [7,16,17,18], [], [], [8,19,9], [0, 10,11,3]].The  is a two-dimensional list, where the sub-list represents the scheme of each UAV.The numbers in the sub-list represent the targets' number.The same is true later.According to the initial mission plan, a total of six UAVs are dispatched to carry out the reconnaissance mission:   ,  ,  ,  ,  ,  ,  6.The initial situation and the initial mission plan are shown in Figure 2a

Task Reassignment in the Case of UAV Damage
To verify the effectiveness of the TS-DTA algorithm in the case of UAV damage, test 1 is conducted.The moment when the drone departs from the ship platform is t = 0. Assuming t = 1200,  is attacked by the enemy and loses the ability to execute tasks.In the simulation experiment, the location and task execution status of each UAV is calculated based on the initial mission plan.By this way, the finished task list of UAV formation is  2 , , 1 , 7, 16 , , , 8 , 0, 10 , the leftover task list of UAV formation is  13, 5, 4 , 6, 15, 12 , 14 , 17, 18 , , , 19, 9 , 11, 3 , and  6, 15, 12 is the leftover task list of  .The tasks in  need to be assigned to  .So, the list of tasks to be assigned is  6, 15, 12 , and   ,  ,  ,  ,  .The current battlefield situation is shown in Figure 3a.

Task Reassignment in the Case of UAV Damage
To verify the effectiveness of the TS-DTA algorithm in the case of UAV damage, test 1 is conducted.The moment when the drone departs from the ship platform is t = 0. Assuming t = 1200, U 1 is attacked by the enemy and loses the ability to execute tasks.In the simulation experiment, the location and task execution status of each UAV is calculated based on the initial mission plan.By this way, the finished task list of UAV formation is TF = [ [2], [], [1], [7,16], [], [], [8], [0, 10]], the leftover task list of UAV formation is TL = [[13, 5, 4], [6, 15, 12], [14], [17,18], [], [], [19,9], [11,3]], and TL 1 = [6, 15, 12] is the leftover task list of U 1 .The tasks in TL 1 need to be assigned to U dispatch .So, the list of tasks to be assigned is TL un = [6, 15, 12], and U dispatch = [U 0 , U 2 , U 3 , U 6 , U 7 ].The current battlefield situation is shown in Figure 3a.In Figure 3, the green circular icons represent the targets that have been reconnoitered, the blue circular icons represent the targets that have not been finished, the gray circular icons represent the targets that have changed, and the solid lines of different colors represent the remaining flight routes of UAVs according to the current mission plan.The UAVs that have not been given a task are still located on ship platforms; so, the icons of these UAVs and ship platforms overlap.
Run the TS-DTA algorithm to assign tasks in  .The new mission plan is  [ [13,5,4], [], [14,6], [17,18,12,15], [], [], [19,9], [11,3].The cost of multi-UAV formation is  2051.28.So,  is assigned to  , and  ,  are assigned to  .According to  , the remaining routes of UAVs are shown in Figure 3b.The reconnaissance schedules of targets are shown in Figure 4.The green icons represent the reconnaissance time of the targets that have been reconnoitered, and the blue icons represent the reconnaissance time of the targets that will be reconnoitered.The same is true later.From Figure 3b, we can find that  will be reconnoitered by  and  ,  will be reconnoitered by  .Combining Table 2 and Figure 4,  can ensure that all targets with time windows are reconnoitered within the time window.
According to  ,  is reconnoitered after  in the scheme of  .In Figure 5, there are two possible routes for  to finish  and  .One is the shorter route, shown by the green dotted line, where the start and end times of the  reconnaissance of  In Figure 3, the green circular icons represent the targets that have been reconnoitered, the blue circular icons represent the targets that have not been finished, the gray circular icons represent the targets that have changed, and the solid lines of different colors represent the remaining flight routes of UAVs according to the current mission plan.The UAVs that have not been given a task are still located on ship platforms; so, the icons of these UAVs and ship platforms overlap.
Run the TS-DTA algorithm to assign tasks in TL un .The new mission plan is Plan new = [ [13,5,4], [], [14,6], [17,18,12,15], [], [], [19,9], [11,3]  In Figure 3, the green circular icons represent the targets that have been reconnoi tered, the blue circular icons represent the targets that have not been finished, the gray circular icons represent the targets that have changed, and the solid lines of differen colors represent the remaining flight routes of UAVs according to the current mission plan.The UAVs that have not been given a task are still located on ship platforms; so, th icons of these UAVs and ship platforms overlap.
Run the TS-DTA algorithm to assign tasks in  .The new mission plan i  [ [13,5,4], [], [14,6], [17,18,12,15], [], [], [19,9], [11,3].The cost of multi-UAV formation is  2051.28.So,  is assigned to  , and  ,  are assigned to  .Ac cording to  , the remaining routes of UAVs are shown in Figure 3b.The recon naissance schedules of targets are shown in Figure 4.The green icons represent the re connaissance time of the targets that have been reconnoitered, and the blue icons repre sent the reconnaissance time of the targets that will be reconnoitered.The same is tru later.From Figure 3b, we can find that  will be reconnoitered by  and  ,  will b reconnoitered by  .Combining Table 2 and Figure 4,  can ensure that all target with time windows are reconnoitered within the time window.
According to  ,  is reconnoitered after  in the scheme of  .In Figure 5 there are two possible routes for  to finish  and  .One is the shorter route, shown by the green dotted line, where the start and end times of the  reconnaissance of  From Figure 3b, we can find that T 6 will be reconnoitered by U 2 and T 12 , T 15 will be reconnoitered by U 3 .Combining Table 2 and Figure 4, Plan new can ensure that all targets with time windows are reconnoitered within the time window.
According to Plan new , T 6 is reconnoitered after T 14 in the scheme of U 2 .In Figure 5, there are two possible routes for U 2 to finish T 6 and T 14 .One is the shorter route, shown by the green dotted line, where the start and end times of the U 2 reconnaissance of T 14 are marked in green font in the figure.Another is the actual route, shown by the brown solid line, where the start and end times of the U 2 reconnaissance of T 14 are marked in brown font in the figure.ts j i represents the U i start time to reconnoiter T j ; te j i represents the U i end time reconnoitering T j .From Figure 5, we can find that in the shorter route T 6 is reconnoitered before T 14 .However, if the shorter route is adopted, ts    the  end time reconnoitering  .From Figure 5, we can find that in the shorter route  is reconnoitered before  .However, if the shorter route is adopted,  2 14 1624.95and  2 14 1701.89,which obviously contradicts  1000, 1600 .So, the shorter route (green dotted line) is abandoned.The list of tasks to be allocated is TL un = [20, 21, 22].The UAVs participating in task reassignment are [1], [7,16], [], [], [8], [0, 10]], and the leftover task list of UAV formation is TL = [ [13, 5, 4], [6,15,12], [14], [17,18], [], [], [19,9], [11,3]].
mation is  2 , , and the leftover task list of UAV formation is  13, Run the TS-DTA algorithm to obtain the new task plan:  [ [13,5,4], [6,20,15,12], [14], [17,18], [], [], [19,21,9], [11,22,3]].The cost of multi-UAV formation is  2166.98.According to  ,  is assigned to  ,  is assigned to  , and  is assigned to drone  .The result is shown in Figure 6b.The reconnaissance schedules of targets are shown in Figure 7. Analysis shows that  ,  are assigned through bidding strategy based on route distance.So,  and  are assigned to  and  , respectively, which are judged from the route distance.It is reasonable to assign to  and  from the perspective of flight distance.However,  needs to be completed within the time window of 1500, 1800 , which conflicts with the time window of  that belongs to  .So,  is finally assigned to  .It is reasonable to assign to U 1 and U 2 from the perspective of flight distance.However, T 20 needs to be completed within the time window of [1500, 1800], which conflicts with the time window of T 14 that belongs to U 1 .So, T 20 is finally assigned to U 2 .

Task Reassignment in the Case of Changing the Location of Targets
To verify the effectiveness of the TS-DTA algorithm in the case of changing the location of targets, test 3 is conducted.It is assumed that at time t = 1100, the early warning aircraft detects that four targets have moved to new locations as follows: T 4 moves from [184.89, 215.84] to [157.00, 198.00],T 9 moves from [109.69, 185.35] to [105.00, 210.00],T 14 moves from [193.87, 108.35] to [194.00, 132.00], and T 17 moves from [260.90, 226.18] to [250.00, 256.00].The list of tasks to be allocated is TL un = [4, 9, 14, 17].Among them, T 4 and T 14 contain time window constraints, and T 9 and T 17 do not contain time window constraints.The current battlefield situation is shown in Figure 8a.The lines in Figure 8a represent the original planned routes of UAVs.Because the locations of  ,  ,  , and  are changed, UAVs can no longer finish the reconnaissance of these moved targets according to the original planned route.So, we need to reassign these moved targets and update the flight routes according to  to ensure The lines in Figure 8a represent the original planned routes of UAVs.Because the locations of T 4 , T 9 , T 14 , and T 17 are changed, UAVs can no longer finish the reconnaissance of these moved targets according to the original planned route.So, we need to reassign these moved targets and update the flight routes according to Plan new to ensure that all tasks can be completed.Now, the finished task list of UAV formation is TF = [[], [], [ ], [7], [], [], [], [0]], and the leftover task list of UAV formation is TL = [ [2,13,5], [6,15,12], [1], [16,18], [], [], [8,19], [10,11,3]].The UAVs participating in the assignment are U dispatch = [U 0 , U 1 , U 2 , U 3 , U 6 , U 7 ].
Run the TS-DTA algorithm to obtain the new task plan: Plan new = [[2, 4, 13, 5], [18,6,15,12], [1,14], [16], [], [], [8,19,9], [10,11,17,3]]; T 4 , T 9 , T 14 , T 17 are assigned to U 0 , U 6 , U 2 , U 7 , respectively.The cost of multi-UAV formation is f c = 2440.83.The remaining routes are shown in Figure 8b, and the reconnaissance schedules of targets are shown in Figure 9.Although T 9 and T 14 have moved, the range of motion is not large; thus, they are assigned to the original UAV and the task orders are not changed.As for T 4 , the time window constraint cannot be satisfied due to the change in location.After reassignment, the task execution order of U 0 is adjusted to ensure that T 4 is reconnoitered within its time window.The lines in Figure 8a represent the original planned routes of UAVs.Because the locations of  ,  ,  , and  are changed, UAVs can no longer finish the reconnaissance of these moved targets according to the original planned route.So, we need to reassign these moved targets and update the flight routes according to  to ensure that all tasks can be completed.Now, the finished task list of UAV formation is  , , , 7 , , , , 0 , and the leftover task list of UAV formation is  [ [2,13,5], [6,15,12], [1], [16,18], [], [], [8,19], [10,11,3]].The UAVs participating in the assignment are   ,  ,  ,  ,  ,  .
Run the TS-DTA algorithm to obtain the new task plan:  [ [2,4,13,5], [18,6,15,12], [1,14], [16], [], [], [8,19,9], [10,11,17,3]];  ,  ,  ,  are assigned to  ,  ,  ,  , respectively.The cost of multi-UAV formation is  2440.83.The remaining routes are shown in Figure 8b, and the reconnaissance schedules of targets are shown in Figure 9.Although  and  have moved, the range of motion is not large; thus, they are assigned to the original UAV and the task orders are not changed.As for  , the time window constraint cannot be satisfied due to the change in location.After reassignment, the task execution order of  is adjusted to ensure that  is reconnoitered within its time window.For U 3 , because the location of T 17 is changed, the algorithm first removes T 17 from the task list of U 3 , which leads to the advance in the reconnaissance time of T 18 .As a result, the time window of T 18 cannot be satisfied.So, T 18 is the additional target, which should be added to TL un .Finally, T 18 is assigned to U 0 , and T 17 is assigned to U 7 , which is nearer to T 17 .According to Plan new , U 0 first reconnoiters T 18 and then performs subsequent tasks.By this way, the completion time of subsequent tasks are delayed; however, the time window constraints of all targets are satisfied.
Run the TS-DTA algorithm to obtain a new mission plan:  [ [13,4,5], [6,15,12], [14], [18,17], [], [], [19,9], [11,3]].The cost of multi-UAV formation is  1958.39.The results are shown in Figures 10b and 11.According to the initial plan, the task scheme of  is  2, 13, 5, 4 and the task scheme of  is  7, 16, 17, 18 .So, in the initial plan,  belongs to  and  belongs to  .It can be seen from  that both  and  are assigned to the UAV they originally belonged to.However, compared with the initial plan, the orders of tasks in  and  have changed.According to  ,  reconnoiters  and then reconnoiters  , and  reconnoiters  and then reconnoiters  .This is because the time windows of  and  have advanced.Run the TS-DTA algorithm to obtain a new mission plan: Plan new = [ [13,4,5], [6,15,12], [14], [18,17], [], [], [19,9], [11,3]].The cost of multi-UAV formation is f c = 1958.39.The results are shown in Figures 10a and 11.According to the initial plan, the task scheme of U 0 is P 0 = [2, 13, 5, 4] and the task scheme of U 3 is P 3 = [7, 16, 17, 18].So, in the initial plan, T 4 belongs to U 0 and T 18 belongs to U 3 .It can be seen from Plan new that both T 4 and T 18 are assigned to the UAV they originally belonged to.However, compared with the initial plan, the orders of tasks in P 0 and P 3 have changed.According to Plan new , U 0 reconnoiters T 4 and then reconnoiters T 5 , and U 3 reconnoiters T 18 and then reconnoiters T 17 .This is because the time windows of T 4 and T 18 have advanced.

Analysis of Communication Simplification Effect
The communication times mean the total times that UAVs communicate with each other during the process of task assignment.This paper reduces the communication times of UAV formation by introducing a bidding benchmark and assignment strateg based on route distance into the TS-DTA algorithm.Gao et al. [34] proposed a CNP based algorithm to solve the task reassignment problem, in which targets are allocated one by one.In order to analyze the simplification effect of the TS-DTA algorithm on th communication process, we carried out some comparative experiments.Since the CNP based algorithm [34] is only applicable to the two cases of UAV damage and new targe occurrence, we only consider the above two dynamic events when setting up compara tive experiments.
In the experiment, we randomly generate some targets that need to be reconnoi

Comparative Analysis 4.2.1. Analysis of Communication Simplification Effect
The communication times mean the total times that UAVs communicate with each other during the process of task assignment.This paper reduces the communication times of UAV formation by introducing a bidding benchmark and assignment strategy based on route distance into the TS-DTA algorithm.Gao et al. [34] proposed a CNP-based algorithm to solve the task reassignment problem, in which targets are allocated one by one.In order to analyze the simplification effect of the TS-DTA algorithm on the communication process, we carried out some comparative experiments.Since the CNP-based algorithm [34] is only applicable to the two cases of UAV damage and new target occurrence, we only consider the above two dynamic events when setting up comparative experiments.
In the experiment, we randomly generate some targets that need to be reconnoitered in the range of [100, 300] × [100, 300] as new targets.The specific data are shown in Table 3.We conducted the following three sets of tests: UAV damage, new target appearance, and new target appearance at the same time as UAV damage.Experiments are based on the initial mission plan described in Section 4.1.1,and different dynamic events are introduced for comparative experiments.
The dynamic events are introduced at time t = 1000 in the first set of experiments, recorded as test 5.The leftover task list of UAV formation is TL = [[2, 13, 5, 4], [6,15,12], [1,14], [16,17,18], [], [], [8,19,9], [10,11,3]].By changing the serial number and numbers of damaged UAVs, comparative experiments are carried out.The experimental results are shown in Table 4.In the table, n d represents the number of UAVs in U dispatch and n un represents the number of targets to be allocated.It can be seen from the results that under various experimental conditions, the communication times of the CNP-based algorithm are greater than those of the TS-DTA algorithm.It is noted that the communication times of the TS-DTA algorithm are not stable even if n d and n un are constant, which is mainly caused by three reasons: First, the bidding benchmark is introduced in the algorithm and the number of contractors that meet the bidding benchmark in the bidding process is not certain.Second, the number of assignments in the bidding strategy based on the route distance is not stable.Third, the number of cycles of the cyclic bidding strategy based on task timing is not stable.
The dynamic events are introduced at time t = 1200 in the second set of experiments, recorded as test 6.The leftover task list of UAV formation is TL = [ [13,5,4], [6,15,12], [14], [17,18], [], [], [19,9], [11,3]].By introducing different numbers of new targets for comparative analysis, the experimental results are shown in Table 5.It can be seen from the results that as the number of new targets increases, the communication times of the CNP-based algorithm increases linearly.However, the communication times of the TS-DTA algorithm are significantly less than those of the CNP-based algorithm, and as the number of targets increases, the advantage is more obvious.The dynamic events are introduced at time t = 1200 in the third set of experiments, recorded as test 7.A comparative analysis was performed by changing the number of damaged UAVs and the number of new targets.The experimental results are shown in Table 6.It can be seen from the results that the number of communication times of the TS-DTA algorithm is significantly lower than that of the CNP-based algorithm in various cases.In order to analyze the solution speed and solution quality of the TS-DTA algorithm, we conducted a set of comparative experiments.In addition to the CNP-based algorithm mentioned in Section 4.2.1, we add two centralized task assignment algorithms for comparison, including RPSO [39] and IEPPSO [22].The solution speed and solution quality of RPSO and IEPPSO are related to the population size of particles.In order to analyze the performance of the algorithm more comprehensively, the task assignment effects of RPSO and IEPPSO algorithms with population sizes of 100, 300, and 500 are compared in each experiment.In the experiment, the number of iterations of the centralized algorithm is set to 200.
In the experiment, the number of targets to be reassigned is set to 2, 4, 6, 8, and 10, respectively.The performance is analyzed by comparing the CPU running time (t cpu ) and the overall cost of UAV formation ( f c) in various cases.Due to the randomness of the results of RPSO and IEPPSO algorithms, in order to ensure the objectivity of experiments, the RPSO and IEPPSO algorithms are run 20 times in each test, and the average of the 20 results are taken as the final result.The results are shown in Table 7.According to the results of Table 7, the paper draws a line chart of CPU running time changing with the number of targets to be allocated, as shown in Figure 12, and draws a line chart of overall cost changing with the number of targets to be allocated, as shown in Figure 13.From Figure 12, we can intuitively see that the CPU running time of the TS-DTA a gorithm and the CNP-based algorithm is the shortest, followed by the RPSO algorithm and that of the IEPPSO algorithm is the longest.As the population size increases, th time consumption of the RPSO and IEPPSO algorithms increase.Based on the exper mental results in Section 4.2.1, the CNP-based algorithm requires a large amount o communication for task assignment.In the experiment, we did not consider the time cos of communication, and a large amount of communication will lead to a slower solutio speed in actual task assignment.
From Figure 13, we can see that as the number of targets increases, the solving abi ity of the RPSO algorithm becomes weaker and the proportion of solutions that do no meet the constraints becomes higher, resulting in an overall higher cost value.The solv ing ability of the IEPPSO algorithm is relatively stable, and when the population size i 500, the solution of IEPPSO algorithm is optimal in various situations.However, com bined with Figure 12, we find that the CPU running time of the IEPPSO algorithm signif icantly increases with the increase in the number of targets, which reflects that th IEPPSO algorithm improves the quality of the solution by increasing the computationa load.So, the IEPPSO algorithm is not applicable in dynamic environments with high re quirements for solving speed.Besides, the quality of solutions of the TS-DTA algorithm From Figure 12, we can intuitively see that the CPU running time of the TS-DTA algorithm and the CNP-based algorithm is the shortest, followed by the RPSO algorithm, and that of the IEPPSO algorithm is the longest.As the population size increases, the time consumption of the RPSO and IEPPSO algorithms increase.Based on the experimental results in Section 4.2.1, the CNP-based algorithm requires a large amount of communication for task assignment.In the experiment, we did not consider the time cost of communication, and a large amount of communication will lead to a slower solution speed in actual task assignment.
From Figure 13, we can see that as the number of targets increases, the solving ability of the RPSO algorithm becomes weaker and the proportion of solutions that do not meet the constraints becomes higher, resulting in an overall higher cost value.The solving ability of the IEPPSO algorithm is relatively stable, and when the population size is 500, the solution of IEPPSO algorithm is optimal in various situations.However, combined with Figure 12, we find that the CPU running time of the IEPPSO algorithm significantly increases with the increase in the number of targets, which reflects that the IEPPSO algorithm improves the quality of the solution by increasing the computational load.So, the IEPPSO algorithm is not applicable in dynamic environments with high requirements for solving speed.Besides, the quality of solutions of the TS-DTA algorithm and the CNP-based algorithm are similar, and there is not much difference compared with the IEPPSO algorithm with a population size of 500.
Finally, considering the solving speed, communication burden, and solution quality, the TS-DTA algorithm is more suitable for solving the task reassignment problem in dynamic environments.

Conclusions and Future Work
In this paper, a model of multi-UAV cooperative reconnaissance task reassignment is established to comprehensively consider many dynamic events including UAV damage, new target occurrence, location change of the target, and time window change of the target.Then, a two-stage distributed task assignment algorithm (TS-DTA) based on the improved contract net protocol is presented to realize the rapid reassignment of multiple targets, reduce the communication burden of multi-UAV formation, and ensure the quality of task assignment to a certain extent.Finally, the experimental results show that the proposed TS-DTA algorithm can efficiently solve the task reassignment problem in dynamic environments while effectively reducing the communication burden of UAV formation.
Our future works will focus mainly on the two following aspects.Firstly, the TS-DTA algorithm will be further optimized to improve efficiency.Secondly, the task assignment
The First Assignment Stage 3.4.1.Single-Target Insertion Method .3.Bidding Strategy Based on Route Distance n d = 6.The initial situation and the initial mission plan are shown in Figure 2a,b.The numbers next to the targets icon indicate targets' number, the same is true later.

Figure 3 .
Figure 3. (a) The battlefield situation of test 1.(b) Result of task reassignment in test 1.

Figure 3 .
Figure 3. (a) The battlefield situation of test 1.(b) Result of task reassignment in test 1.
. The cost of multi-UAV formation is f c = 2051.28.So, T 6 is assigned to U 2 , and T 12 , T 15 are assigned to U 3 .According to Plan new , the remaining routes of UAVs are shown in Figure3b.The reconnaissance schedules of targets are shown in Figure4.The green icons represent the reconnaissance time of the targets that have been reconnoitered, and the blue icons represent the reconnaissance time of the targets that will be reconnoitered.The same is true later.

Figure 3 .
Figure 3. (a) The battlefield situation of test 1.(b) Result of task reassignment in test 1.

14 2 =
1624.95 and te 14 2 = 1701.89,which obviously contradicts TW 14 = [1000, 1600].So, the shorter route (green dotted line) is abandoned.Sensors 2023, 23, x FOR PEER REVIEW 17 of 27 are marked in green font in the figure.Another is the actual route, shown by the brown solid line, where the start and end times of the  reconnaissance of  are marked in brown font in the figure.   represents the  start time to reconnoiter  ;    representsthe  end time reconnoitering  .From Figure5, we can find that in the shorter route  is reconnoitered before  .However, if the shorter route is adopted, , which obviously contradicts  1000, 1600 .So, the shorter route (green dotted line) is abandoned.

Figure 5 .
Figure 5. Routes comparison.4.1.3.Task Reassignment in the Case of New Target Occurrence To verify the effectiveness of the TS-DTA algorithm in the case of finding new targets, test 2 is conducted.Suppose that at t = 1300, the early warning aircraft detects three new targets:  ,  , and  .The location of  is  [217.00,153.00], the radius of the annular route  = 1.50, and the reconnaissance time window is  1500, 1800 .The location of  is  [130.00,206.00], and the radius of the annular route  = 2.30, without a reconnaissance time window.The location of  is  [241.00,263.00], and the radius of annular route  = 1.90, without a reconnaissance time window.According to Section 2.2.3, new targets need to be added into the original target set .The original target number is 20, numbered from 0 to 19.So,  is numbered 20, denoted as  .Similarly,  and  are numbered 21 and 22, which are denoted as  and  , respectively.The current battlefield situation is shown in Figure 6a.

Figure 6 .
Figure 6.(a) The battlefield situation of test 2. (b) Result of task reassignment in test 2. The list of tasks to be allocated is  20, 21, 22 .The UAVs participating in task reassignment are   ,  ,  ,  ,  ,  .The finished task list of UAV for-

Figure 5 .
Figure 5. Routes comparison.4.1.3.Task Reassignment in the Case of New Target Occurrence To verify the effectiveness of the TS-DTA algorithm in the case of finding new targets, test 2 is conducted.Suppose that at t = 1300, the early warning aircraft detects three new targets: T 1 n , T 2 n , and T 3 n .The location of T 1 n is LOC T 1 n = [217.00,153.00], the radius of the annular route ar T 1 n = 1.50, and the reconnaissance time window is TW T 1 n = [1500, 1800].The location of T 2 n is LOC T 2 n = [130.00,206.00], and the radius of the annular route ar T 2 n = 2.30, without a reconnaissance time window.The location of T 3 n is LOC T 3 n = [241.00,263.00], and the radius of annular route ar T 2 n = 1.90, without a reconnaissance time window.According to Section 2.2.3, new targets need to be added into the original target set T. The original target number is 20, numbered from 0 to 19.So, T 1 n is numbered 20, denoted as T 20 .Similarly, T 2 n and T 3 n are numbered 21 and 22, which are denoted as T 21 and T 22 , respectively.The current battlefield situation is shown in Figure 6a.

Figure 5 .
Figure 5. Routes comparison.4.1.3.Task Reassignment in the Case of New Target Occurrence To verify the effectiveness of the TS-DTA algorithm in the case of finding new targets, test 2 is conducted.Suppose that at t = 1300, the early warning aircraft detects three new targets:  ,  , and  .The location of  is  [217.00,153.00], the radius of the annular route  = 1.50, and the reconnaissance time window is  1500, 1800 .The location of  is  [130.00,206.00], and the radius of the annular route  = 2.30, without a reconnaissance time window.The location of  is  [241.00,263.00], and the radius of annular route  = 1.90, without a reconnaissance time window.According to Section 2.2.3, new targets need to be added into the original target set .The original target number is 20, numbered from 0 to 19.So,  is numbered 20, denoted as  .Similarly,  and  are numbered 21 and 22, which are denoted as  and  , respectively.The current battlefield situation is shown in Figure 6a.

Figure 6 .
Figure 6.(a) The battlefield situation of test 2. (b) Result of task reassignment in test 2. The list of tasks to be allocated is  20, 21, 22 .The UAVs participating in task reassignment are   ,  ,  ,  ,  ,  .The finished task list of UAV for-

Figure 6 .
Figure 6.(a) The battlefield situation of test 2. (b) Result of task reassignment in test 2.

4. 1
.4.Task Reassignment in the Case of Changing the Location of Targets To verify the effectiveness of the TS-DTA algorithm in the case of changing the location of targets, test 3 is conducted.It is assumed that at time t = 1100, the early warning aircraft detects that four targets have moved to new locations as follows:  moves from 184.89, 215.84 to [157.00, 198.00],  moves from 109.69, 185.35 to [105.00, 210.00],  moves from 193.87, 108.35 to [194.00, 132.00], and  moves from 260.90, 226.18 to [250.00, 256.00].The list of tasks to be allocated is  4, 9, 14, 17 .Among them,  and  contain time window constraints, and  and  do not contain time window constraints.The current battlefield situation is shown in Figure 8a.

Figure 8 .
Figure 8.(a) The battlefield situation of test 3.(b) Result of task reassignment in test 3.

Figure 8 .
Figure 8.(a) The battlefield situation of test 3.(b) Result of task reassignment in test 3.

Figure 8 .
Figure 8.(a) The battlefield situation of test 3.(b) Result of task reassignment in test 3.

Figure 10 .
Figure 10.(a) The battlefield situation of test 4; (b) Result of task reassignment in test 4.
not been finished when a dynamic event occurs under the premise of satisfying various constraints and to obtain a new mission plan, denoted as Plan new = P 0 n , P 1 n , . . ., P n new −1 have

Algorithm 2
Bidding Strategy with Bidding Benchmark Input : T x , WTL un , Plan, U m , CU Output : updated Plan, WTL un 1 If P m does not meet the constraints in Section 2.2.5 2 U m removes the targets with time window from P m into WTL un 3 compares all received mdc and finds the U best with the min mdc 26 P best = BP best , U m sends losingsignal to those U i participated in the bidding (except U best ) and sends winningsignal to U best 27 End if m get BP m , mdc m of inserting T x into P m by single-target insertion method 5 U m sends T x , mdc m , Plan to every UAV of CU 6 For U i in CU do (Parallel) 7 If P i does not meet the constraints in Section 2.2.5 8 U i removes the targets with time window from P i into WT i 9 End if 10 U i get BP i , mdc i of inserting T x into P i by single-target insertion method 11 If mdc i < mdc m and mdc i < 10 4 12 U i sends BP i , mdc i , WT i to U m 13 Else 14 U i sends WT i to U m and give up the bidding 15 End if 16 End for 17 U m puts the targets in all received WT i into WTL un 18 If all U i in CU give up the

Algorithm 3
Bidding Strategy Based on Route Distance Input : FTL un , WTL un , Plan, U m , CU, U dispatch , n d Output : updated Plan, WTL un 1 U m sends FTL un , Plan to every UAV of CU 2 If P m does not meet the constraints in Section 2.2.5 3 U m removes the targets with time window from P m into WTL un 4 End if 5 U m calculates the distance of every route point to each target in FTL un , gets DT m 6 For U i in CU do (Parallel) 7 If P i does not meet the constraints in Section 2.2.5 8 U i removes the targets with time window from P i into WT i 9 End if 10 U i calculates the distance of every route point to each target in FTL un , gets DT i 11 U i sends DT i , WT i to U m 12 End for 13 U m puts the targets in all received WT i into WTL un 14 U m processes all DT i , gets Assign un = ∅ and k ≤ n d do 17 For j in range 1 to n f tl do 18 Assign FTL un [j] to Assign[j][k] 19 End for 20 U m gets the assigned task list ASTL k and sends AL i k to U i AL m k handled by U m ) 21 For U i in U dispatch do (Parallel) 22 For T j in AL i k do 23 U i get BP j i , mdc i of inserting T j into P i by single-target insertion method While WTL un = ∅ or CoWTL un = WTL un do 3 CoWTL un = copy (WTL un ) compares all received mdc and finds the U best with the min mdc 23 P best = BP best , U m sends losingsignal to those U i participated in the bidding (except U best ) and sends winningsignal to U best Algorithm 4 Cyclic Bidding Strategy Based on Task Timing Input : WTL un , Plan, U m , CU Output : updated Plan 1 CoWTL un = copy (WTL un ) 2

Table 1 .
Initialization information of platforms and UAVs.

Table 2 .
Initialization information of targets.

Table 3 .
Data of new targets.

Table 4 .
Statistics of communication times in test 5.

Table 5 .
Statistics of communication times in test 6.

Table 6 .
Statistics of communication times in test 7.

Table 7 .
The CPU running time and the overall cost of UAV formation.