Distributed Task Allocation for a Multi-UAV System with Time Window Constraints

Abstract

This paper develops a distributed method, namely, distributed allocation with time windows (DATW) for managing the multi-UAV task assignment problem (MTAP) with complex time window constraints. By aiming directly to minimize the average task completion time, the pro-posed DATW intends to achieve a conflict-free result that allocates all tasks within the validity time windows. Based on the decentralized PI (Performance Impact) framework, the proposed algorithm addresses the MTAP in a three-phase task assignment strategy, which includes task inclusion, conflict resolution, and task reallocation. The newly introduced task allocation phase achieves a noteworthy increase in an average number of allocated tasks. Unlike the traditional PI methods, the start time of each task is broadcasted among agents via communication typology, and the significance value of each task is directly related to its validity time window, such that the vast majority of tasks are able to be assigned properly without imposing any extra communication burdens. In the obtained conflict-free allocation solution by DATW, each task is allocated to a proper UAV with all given constraints satisfied. Finally, the simulation results demonstrate the effectiveness and superiority of the proposed DATW. Compared with existing (CBBA-based) solutions, results show up to an 18% increase in success rate (SR) using the proposed method.

1. Introduction

Currently, multiagent systems (e.g., unmanned aerial vehicle systems) have drawn increasing attention from researchers worldwide [1,2,3,4,5,6]. In comparison to a single agent, multiagent systems can achieve more complicated mission objectives and increase execution efficiency through cooperation. In this paper, we focus on the application of UAVs in multiagent systems due to the advantages of high flexibility, easy assembly, and low-cost consumption [7,8]. In particular, UAVs have been widely used in a variety of industries, including search and rescue, military reconnaissance, and traffic transportation. To date, there is a growing consensus that the multi-UAV task assignment problem (MTAP) has become one of the most challenging problems of multiagent systems.

In general, the MTAP is a combinatorial optimization problem with complex constraints that aims to coordinate a group of UAVs to accomplish various tasks while optimizing some given overall objectives [9]. Moreover, the MATP is proven to be a nondeterministic polynomial hard (NP-hard) problem [10], and as a result, the scale of the problem has a direct impact on the complexity of the solving process.

Early studies of the MATP focused on centralized approaches, where each UAV communicates situational awareness (SA) to a centralized server that generates a collaborative plan for the entire UAV swarm. Initially, the problem is modeled as mixed-integer linear programming (MILP) [11], which includes multiple processor resource allocation (MPRA) [12], the multiple traveling salesman problem (MTSP) [13], and the vehicle routing problem (VRP) [14]. By embedding complicated constraints into the models, the MILP algorithms have advantages in describing the mission; however, the tree search method and exhaustive enumerations in MILP have resulted in an explosion in the computational demands for large-scale systems. Then, some heuristic algorithms [15,16,17,18], namely, swarm intelligence algorithms (SIAs), are proposed by searching a certain range of solution space in an acceptable time to obtain feasible solutions for MATP. Common heuristic methods include the genetic algorithm (GA) [15], ant colony algorithm [16], particle swarm optimization algorithm [17], wolf swarm algorithm [18], etc. Although these SIAs have a relatively straightforward structure, their stability is weak and a single point of failure can bring down the entire system [19].

Recently, some distributed allocation methods have been proposed, which are flexible and robust, consuming less communication and computational resources than centralized approaches. Most of these distributed approaches (e.g., contract network protocol) [20,21,22,23] are developed on the basis of market auction mechanisms. The UAVs place bids on tasks, and the highest bid wins the assignment. Choi et al., in [24], proposed the classical consensus-based bundle algorithm (CBBA) and introduced a consensus procedure to achieve agreement on the winning bid values through a consensus routine instead of SA. A series of modifications to the classical CBBA were made in [25,26] to expand the function and application of such a method. Then, another distributed approach called performance impact (PI) was proposed by Whitbrook et al. [27] and Zhao et al. [28], which aims to directly optimize the overall objective. Such methods introduce a key concept called significance to assess each task’s contribution to the local time cost of an UAV. In comparison to CBBA, the PI method is developed on the basis of the heuristic optimization principle and can solve more time-sensitive MTAP while achieving allocation with lower time costs across tasks.

In realistic situations, the existence of various constraints and uncertainties will cause significant growth in the complexity of MATP [29]. To this end, most studies on MTAP describe this problem in terms of ideal mathematical models. For example, [30] proposes a “closed-loop CBBA” that considers the return of UAVs to the take-off base after the mission is completed, but ignores existing constraints during the allocation. Ponda et al., in [26], proposed an extension algorithm of CBBA to handle MTAP with time window constraints and vehicle fuel cost constraints; however, the allocated tasks in results notably decrease when time window constraints between tasks are more complex. In addition, PI-MaxAss was proposed as an extension of PI by Turner et al. [31], which aims to maximize the number of task allocations under strict time constraints. This article discusses task deadlines under a search and rescue scenario, while the time window constraints between tasks are ignored.

In this paper, common constraints such as time windows and UAV capabilities are combined to build a complex MTAP model. We propose an algorithm, namely, distributed allocation with time windows (DATW), which aims to achieve conflict-free allocation with a minimum average task completion time. The proposed DATW embeds time window constraints into the PI-based framework and iterates between three phases: the task inclusion phase, conflict resolution phase, and task reallocation phase. The first phase selects the optimal task within validity time windows for each UAV. The second reaches a consensus based on the significance value over all UAVs. The last allocates the tasks that remain unassigned after the former two phases due to time window constraints. Note that the time window constraints in this paper are “hard constraints”, i.e., each task must start within its time window constraints; otherwise, the task is not allowed to be assigned. The key contributions of this paper are summarized as follows:

• The time window constraints are first embedded into the PI-based framework. That is, the significance value of each task is directly related to its validity time window. In the obtained allocation, the vast majority of tasks are assigned by achieving a minimum average task completion time.
• Compared with the PI method in [22], a new task reallocation phase that only considers the marginal significance value is proposed. The number of allocated tasks is effectively improved by the aid of this third phase, where simulation results show that the average rate of feasible tasks has exceeded 70%.
• The experiments reveal that the proposed DATW assigns more tasks within validity time windows than the existing E-CCBA method in [26]. In addition, the DATW has facilitated an 18% increase in success rate when assigning all tasks for UAVs.

This paper is organized as follows. Section 2 presents the description of MTAP and depicts the mathematical model of the distributed allocation with time window constraints. Section 3 proposes the PI-based distributed method DATW to solve the studied task allocation problem. Section 4 presents numerical simulations to demonstrate the merits of DATW. Finally, Section 5 concludes the paper.

2. Problem Description

This paper studies the multi-UAV task assignment problem (MTAP) with time window constraints. Let N_t = {t₁, t₂, …, t_m} be the set of m tasks and N_u = {u₁, u₂, …, u_n} be a set of n UAVs. A list of key symbols used hereafter is provided in

Table 1. Symbol Definitions.

Symbol	Definitions
Nt	Set of m tasks
Nu	Set of n UAVs
Dj	The duration required to perform task tj
loc(tj)	The spatial location of task tj, loc(tj) = (xj, yj, zj)
loc(ui)	The initial position of UAV ui, loc(ui) = (xi, yi, zi)
vi	The flight velocity of UAV ui
G	The communication topology between UAVs
Pi	An ordered sequence of tasks for UAV ui
Φ(p, Pi)	The task which locates at the p-th position in the sequence Pi
Tj(Pi)	The time at which ui starts to execute tj
$ℱ$(Pi)	The sum of the completion times for all tasks in Pi, i.e., the local time cost of ui
Li	The execution capacity of ui
[Tj_start, Tj_end ]	The execution time window of task tj
Zi	A task list stored on agent ui that keeps track of which task is assigned to which UAV
Qi	A list stored on ui that records the significance value for each task
Ti	A list stored on ui that records the start time for each task
si	A list of timestamps for ui
Pi⊖tj	Pi with task tj removed
qij(Pi⊖tj)	The significance value of task tj for ui
Pi⨁k tj	Pi with task tj inserted into the kth position
qij*(Pi⨁tj)	The marginal significance value of task tj for ui

. The properties of each t_j ∈ N_t are denoted by a tuple <loc(t_j), D_j>, where loc(t_j) = (x_j, y_j, z_j) is the coordinate of the task in the 3-D spatial plane and D_j is the duration required to perform task t_j. Meanwhile, an UAV u_i ∈ N_u is denoted by <loc(u_i), v_i>, where loc(u_i) = (x_i, y_i, z_i) is the initial position of u_i, and v_i is the flight velocity. UAVs communicate through a local communication topology denoted by a symmetric adjacency matrix G, where G_ih = 1 means u_i and u_h are capable of communicating with each other, while G_ih = 0 otherwise. We use P_i = <t_i¹, t_i², …, t_i^|Pi|> to denote an ordered sequence of tasks assigned to u_i; UAV u_i starts from its initial coordinate loc(u_i) and performs tasks in P_i sequentially. Once UAV u_i completes task t_i^j in P_i, it immediately flies to the location of the next task t_i^j+1. The assigned task sequences for all UAVs constitute a task allocation P = [P₁, P₂, …, P_n]^T of MTAP under consideration.

On the basis of the above-mentioned information, the general working mechanism of the multi-UAV systems with time window is illustrated in Figure 1. Herein, the initial information board inputs all the relevant information of tasks and the underlying time window constraints which should be satisfied by all UAVs. Specifically, for each UAV u_i, its reason system generates an allocation plan P_i consisted of tasks assigned to u_i and their ordered sequence. Whenever a u_i includes a new task, it updates the task’s time information and transmits this information to another UAV u_j with G_ij = 1 via communication link. This reasoning system continuously updates the allocation plan according to newly received information. Based on such a dynamic reasoning system, all UAVs work in parallel and interact with each other. Thus, one key problem of our paper is to design an effective reasoning system to achieve a conflict-free allocation by satisfying all time window constraints.

For the sake of convenience, we use the notation Φ(P_i, g) = t_j to denote the fact that task t_j is the g-th (g > 1) task in P_i, i.e., t_j = t_i^g. If task t_k is t_j’s preorder task, Φ(P_i, g−1) = t_k. Then, the time parameter T_j(P_i), when u_i starts to execute t_j, is calculated as follows:

$$T_j\left({\mathit{P}}_i\right)=\{\begin{array}{c}{T}_k\left({\mathit{P}}_i\right)+D_k+\frac{dis\left(loc\left(t_k\right),loc\left(t_j\right)\right)}{v_i}, g>1\\ \frac{dis\left(loc\left(u_i\right),loc\left( t_j\right)\right)}{v_i}, g=1\end{array}$$

(1)

where dis(•) represents the distance between two different locations. Moreover, the completion time F_j(P_i) for task t_j is calculated as follows:

$$F_j\left({\mathit{P}}_i\right)=T_j\left({\mathit{P}}_i\right)+D_j$$

(2)

In the rest of this paper, let $ℱ$(P_i) = ${\sum }_{t_k\in {\mathit{P}}_i}{F}_k\left({\mathit{P}}_i\right)$ be the sum of the completion time for all tasks in P_i; herein, $ℱ$(P_i) is called the local time cost of UAV u_i.

The practical MTAP is subject to the constraints related to UAVs and tasks:

• Conflict-freeness: each task is only assigned to one UAV.
• Capability constraints: each UAV can execute at most L_i tasks at a time. That is,

$$\left|{\mathit{P}}_i\right|\le L_i \left(1\le i\le n\right)$$

(3)

3.

Time window constraints: each task must be started during the interval of a time window. Let T_{j_start} be the earliest start time of task t_j, and T_{j_end} be the latest start time so that the validity interval for t_j is given as

$$T_j\left({\mathit{P}}_i\right)\in \left[T_{j\_start} ,T_{j\_end}\right]$$

(4)

4.

Power consumption constraints: the remaining fuel mass Fr_i for each UAV must be more than a threshold Δ. Let Fo_i be the initial fuel mass of UAV u_i, and u_i consumes the fuel at a nominal rate vm_i, such that the power consumption constraint is given as follows

$$F{r}_i=F{o}_i-v{m}_i\times ℱ\left(P_i\right) \ge \Delta$$

(5)

This paper aims to find the optimal task allocation for MATP with the goals of satisfying the above-mentioned constraints and minimizing the global time cost, which is the sum of all local time costs across all UAVs. The sought MTAP can be modeled mathematically as follows:

$$J=min\left\{{\sum }_{i=1}^nℱ\left({\mathit{P}}_i\right)\right\}$$

(6)

$$s.t. \begin{array}{c}\bigcup _{i=1}^n{\mathit{P}}_i=N_t,{\mathit{P}}_i\bigcap {\mathit{P}}_j=\varnothing ,i\ne j\end{array}$$

(7)

$$\left|{\mathit{P}}_i\right|\le L_i, \forall u_i\in N_u$$

(8)

$$T_{j\_start}\le T_j\left({\mathit{P}}_i\right)\le T_{j\_end}, \forall t_j\in {\mathit{P}}_i,\forall u_i\in N_u$$

(9)

$$F{r}_i=F{o}_i-v{m}_i\times ℱ\left(P_i\right) \ge \Delta , \forall u_i\in N_u$$

(10)

where objective (6) minimizes the global time cost; (7) ensures that the allocation includes all tasks and each task is assigned to only one UAV, indicating conflict-freeness; (8) ensures that the number of tasks assigned to each UAV u_i does not exceed the execution capacity L_i; (9) guarantees the satisfaction of the time window constraint for all tasks; (10) ensures that each UAV satisfies the power consumption constraints.

3. Distributed Allocation with Time Window Constraints

This section proposes a heuristic method, namely, distributed allocation with time windows (DATW), for solving the MATP under consideration. The DATW algorithm is run concurrently on all UAVs that transmit information to each other via local communication. The proposed DATW algorithm consists of three phases: task inclusion, conflict resolution, and task reallocation. The first two phases are performed iteratively until an initial allocation is agreed upon by all UAVs, although some tasks may remain unassigned. Then, the last phase aims to maximize the allocation of these unassigned tasks to UAVs under the time window constraints.

Before introducing the DATW algorithm, some message or information stored on each UAV u_i is defined as follows.

• Task list:Z_i = [Z_i1, …, Z_im]^T is a vector that keeps track of which task is assigned to which UAV. The entry Z_ij = k if u_i thinks that task t_j is assigned to u_k, while Z_ij is infinity, i.e., Z_ij → ∞, if u_i deems t_j to be unassigned;
• Task significance list: Q_i = [Q_i1, …, Q_im]^T is a vector recording the significance values of all tasks; the concept of the significance of tasks will be given in Section 3.1. If Z_ij → ∞, we set Q_ij → ∞;
• Task start time list: T_i = [T_i1,…, T_im]^T is a vector recording the begin-execute time for each task. When Z_ij = k, T_ij reflects the time when UAV u_k starts to execute task t_j. Note that if Z_ij → ∞, T_ij → ∞;
• Timestamp list: s_i = [s_i1, …, s_in]^T records the timestamp when u_i obtains the latest message from each of the other UAVs. Once a message is passed, the timestamp s_ih is calculated as follows:

  $$s_{ih}=\{\begin{array}{c}{\tau }_r, G_{ih}=1\\ ma{x}_{\alpha :G_{i\alpha }=1}{s}_{\alpha h}-1, G_{ih}\ne 1\end{array}$$

  (11)

  where τ_r represents the time when u_i receives the information.

3.1. Task Inclusion Phase

In the first phase, each UAV u_i selects the optimal task that can minimize u_i’s local time cost and inserts this task into a proper position of list P_i so that the time window constraints are satisfied. The concepts of “significance” and “marginal significance” are given to describe how tasks affect the local time cost of an UAV.

• Significance

Consider a task t_j ∈ P_i; let P_i⊖t_j represents the remaining task sequence after removing t_j from P_i. Then, the significance value of t_j with regard to u_i is formulated as follows:

$$q_{ij}\left({\mathit{P}}_i\ominus t_j\right)=\left[ℱ\left({\mathit{P}}_i\right)-ℱ\left({\mathit{P}}_i\ominus t_j\right)\right]\times \left[T_j\left({\mathit{P}}_i\right)-T_{j\_start}\right]$$

(12)

By definition, q_ij(P_i⊖t_j) is composed of two parts: $ℱ$(P_i) − $ℱ$(P_i⊝t_j) and T_j(P_i) − T_{j_start}. The first part equals the decrease in u_i’s local time cost by removing t_j from P_i, directly reflecting the contribution of t_j to the local time cost of u_i. Meanwhile, the second is the interval between the earliest start time allowed for t_j to start under the time window constraints and the time u_i starts to execute t_j. That is, agent u_i gives preference to tasks that are performed immediately upon meeting the time window constraints. In this case, both T_j(P_i) and F_j(P_i) for task t_j are further decreased, and thus, the contribution of t_j to the local time cost of u_i is logically decreased. Note that if t_j ∉ P_i, let q_ij(P_i⊖t_j) →∞.

2.

Marginal significance

Considering a task t_j ∉ P_i, the local time cost of UAV u_i is increased after inserting t_j into P_i. Then, the marginal significance, denoted by q_ij*(P_i⨁t_j), is required to reflect the minimum increase in u_i’s local time cost by adding t_j into P_i, which is formulated as follows:

$$q_{ij}^{*}\left({\mathit{P}}_i\oplus t_j\right)=mi{n}_{k=1}^{\left|{\mathit{P}}_i\right|+1}\left[ℱ\left({\mathit{P}}_i{\oplus }_k{t}_j\right)-ℱ\left({\mathit{P}}_i\right)\right]\times \left[T_j\left({\mathit{P}}_i\right)-T_{j\_start}\right]$$

(13)

where P_i⨁_k t_j represents the task sequence after inserting t_j into the kth position of P_i. Note that q_ij*(P_i⨁t_j) → ∞ if t_j ∈ P_i. Moreover, if t_j ∈ P_i, Z_ij= i and Q_ij= q_ij(P_i⊖t_j).

In this task inclusion phase, a task t_j is allowed to be assigned to the UAV u_i if the following conditions hold:

(a). Z_ij ≠ i and q_ij*(P_i⨁t_j) < Q_ij;

(b). Let p = argq_ij*(P_i⨁t_j); after inserting t_j into the p-th position of P_i, all tasks in the sequence P_i⨁_pt_j satisfy the underlying time window constraints;

(c). Fo_i – vm_i × $ℱ$(P_i⨁t_j) ≥ Δ.

The above condition (a) ensures that t_j is unassigned to u_i and requires that the marginal significance value q_ij*(P_i⨁t_j) for t_j is strictly less than its significance Q_ij stored on u_i, such that the global time cost might be decreased by assigning t_j to u_i. Condition (b) guarantees the satisfaction of time window constraints after inserting t_j into P_i, while condition (c) ensures the satisfaction of power consumption constraints.

Let ℘(P_i) = {t_j ∈ N_t | Conditions (a), (b), and (c) both hold for t_j and u_i} be the set of tasks that satisfy the above conditions. This phase chooses the task t_k ∈ ℘(P_i) expressed in (14) and inserts it into the position p = argq_ik*(P_i⨁t_k) of list P_i.

$$t_k=argma{x}_{t_j\in \wp \left({\mathit{P}}_i\right)}(Q_{ij}-q_{ij}^{*}\left({\mathit{P}}_i\oplus t_j\right))$$

(14)

The corresponding information related to task t_k is updated as Z_ik = i, Q_ik = q_ik*(P_i⨁t_k), and the start time for all tasks in P_i is recalculated.

The above task addition process is repeated until ℘(P_i) = ∅ or |P_i| = L_t, which indicates that no task can be included in P_i. Since the inclusion of new tasks may change the significance of other existing tasks in P_i, the list Q_i is required to update accordingly (12) after inclusion. The whole procedure is depicted in Algorithm 1.

Algorithm 1:ui execution task adding process at iteration λ.

Initialization: Pi(λ) = Pi(λ − 1), Zi(λ) = Zi(λ − 1), Qi(λ) = Qi(λ − 1), Ti(λ) = Ti(λ − 1).

1: while |Pi| ≤ Li do

2:  For each task tj ∈ Pi compute the marginal significance value qij*(Pi⨁tj);

3:  Compute ℘(Pi) ← {tj ∈ Nt | Conditions a), b) and c) both hold for tj and ui};

4:  if ℘(Pi) ≠ ∅ then

5:    tk ← $argma{x}_{t_j\in \wp \left({\mathit{P}}_i\right)}$(Qij − qij*(Pi⨁tj));

6:    p ← argqij*(Pi⨁tk);

7:    Insert task tk into p − th of Pi;

8:    Zik← i;

9:    Qik ← qij*(Pi⨁tj);

10:   Update Ti;

11:  end if

12:  else

13:    break;

14:  end if

15: end while

16: Update Qi;

3.2. Conflict Resolution Phase

In the previous task inclusion phase, a task may be assigned to more than one UAV, as Algorithm 1 runs concurrently on all UAVs, leading to a conflict. Thus, the second phase is conducted to obtain a conflict-free allocation via local communication topology. In particular, the conflict resolution phase is composed of two stages: (i) consensus, where UAVs communicate with each other for consensus, and (ii) task removal, where UAVs determine whether to remove a task from its current task list. These two stages are repeated alternately on each UAV until the global consensus is achieved across all UAVs, i.e., Q_i = Q_h, ∀u_i, u_h ∈ N_u, and u_i ≠ u_h.

3.2.1. Consensus Stage

To search for the smallest significance value for every task, in this stage, each UAV u_i transmits the relevant information, including Z_i, Q_i, T_i, and s_i, to its neighbor UAV u_j through local communications.

Suppose that UAV u_i receives a message from u_h where G_ih = 1. It determines whether the received message is the latest for task t_j based on lists Z_i and s_i, and then, three possible actions are taken on task t_j:

• Update: Z_ij = Z_hj, Q_ij = Q_hj, T_ij = T_hj;
• leave: Z_ij = Z_ij, Q_ij = Q_ij, T_ij = T_hj;
• reset: Z_ij → ∞, Q_ij → ∞, T_ij → ∞.

Specifically, agents select actions according to the decision rules given in

Table 2. UAVs communication rule.

Zhj (uh is Sender)	Zij (ui is Receiver)	ui’s Action
h	i	if Qhj < Qij: update
	h	update
	α∉{h,i}	if shα > siα or Qhj < Qij: update
	∞	update
i	i	leave
	h	reset
	α∉{h,i}	if shα > siα: reset
	∞	leave
α∉{h,i}	i	if shα > siα and Qhj < Qij: update
	h	if shα > siα: update
		other case: reset
	α	if shα > siα: update
	β ∉ {h,I,m}	if shα > siα and shβ > siβ: update
		if shα > siα and Qhj < Qij: update
		if shβ > siβ and siα > shα: reset
	∞	if shα > siα: update
∞	i	leave
	h	update
	α∉{h,i}	if shα > siα: update

. The first two columns of Table 2 record the executor of task t_j believed by sender u_h and receiver u_i, respectively. The third column outlines the action taken on t_j with respect to receiver u_i. Note that once a piece of message is passed, timestamp g_i is updated according to Formula (11) to obtain the latest time information.

3.2.2. Task Removal Stage

After the consensus stage, each UAV u_i checks the current task sequence P_i and removes tasks in two sets A = {t_j ∈ P_i | Z_ij ≠ i } and B = {t_j ∈ P_i | T_j(P_i) <T_{j_start} or T_j(P_i) >T_{j_end}}. Herein, A denotes the set of tasks in P_i whose corresponding information Z_ij has been changed after the communication, while B represents the set of tasks in P_i violating the time window constraints.

First, a task in A can be removed from P_i and A, according to the following criterion:

$$ma{x}_{t_j\in A}\left(\overline{q_{ij}}-Q_{ij}\right)>0$$

(15)

where $\overline{q_{ij}}$ = $ℱ$(P_i) − $ℱ$(P_i⊝t_j) represents the significance of t_j computed from the current task sequence P_i, while Q_ij denotes t_j’s significance obtained from the consensus stage.

We release a task t_k from P_i and A such that t_k = $argma{x}_{t_j\in A}\left(\overline{q_{ij}}-Q_{ij}\right)$. This removal process is continued until (15) is not satisfied or A is empty. When this process is terminated but A ≠ ∅, each remaining task t_k ∈ A should be returned to agent u_i by setting Z_ik = u_i.

Next, each UAV u_i checks its sequence P_i and releases the tasks in B sequentially. Specifically, once there is a task t_j ∈ P_i that does not meet the time window constraint, t_j is removed from the sequence P_i immediately, and the related information is updated as Z_ij → ∞, Q_ij → ∞, T_ij → ∞. Note that each time u_i removes a task from P_i, the start time for other tasks in P_i is updated. Thus, we need to update set B after each removal. Such a removal process is repeated alternately until all tasks in all P_i satisfy the time window constraint. At the end of the task removal stage, the relevant information Q_ij and T_ij for each t_j ∈ P_i is updated correspondingly.

In the conflict resolution phase, the algorithm iterates over the consensus stage and task removal stage until a global consensus is achieved across all UAVs. To this end, we obtain a conflict-free allocation where tasks are assigned by meeting the time window constraints. The whole procedure is illustrated by Algorithm 2.

Algorithm 2: Conflict resolution phase of UAV ui at iteration λ.

Initialization:Pi(λ) = Pi(λ − 1), Zi(λ) = Zi(λ − 1), Qi(λ) = Qi(λ − 1), Ti(λ) = Ti(λ − 1), set si to zero vector.

1: Send Qi, Ti,Zi and si to uh where Gih=1;

2: Receive Qh,Th,Zh and sh from uh where Ghi=1;

3: Update Qi, Ti,Zi and si according to the rules in Table 2;

4: A←{tj ∈ Pi |Zij ≠ i, tj ∈ Pi};

5: while $ma{x}_{t_j\in A}\left(\overline{q_{ij}}-Q_{ij}\right)$ > 0 do

6:    tk ← $argma{x}_{t_j\in A}\left(\overline{q_{ij}}-Q_{ij}\right)$;

7:    Remove task tk from Pi and A;

8:  end while

9: if A≠∅ then

10:    let Zij = i, ∀tj ∈A;

11: end if

12: B←{tj ∈ Pi |Tj(Pi) <Tj_start or Tj(Pi) >Tj_end};

13: while B≠∅ do

14:    Remove task tk∈B from Pi;

15:    Set Zij, Qij, Tij as infinity values

16:    Update B;

17: end while

3.3. Task Reallocation Phase

After completing iterations between the above phases, the obtained allocation generally covers all tasks. However, in some cases, a few tasks may remain unassigned due to the strict time window constraints. Let Ca_i be the set of these unassigned tasks for each UAV u_i, i.e., Ca_i ={t_k | Z_ik→∞}. This phase intends to reassign tasks in Ca_i to some UAVs to increase the number of allocated tasks. That is, for each UAV u_i, only tasks in Ca_i can be added to or removed from the current list P_i, while assigned task t_j ∉ Ca_i is not allowed to be moved. This phase is composed of two stages: secondary inclusion and secondary conflict resolution.

3.3.1. Secondary Inclusion

We define a vector M_i = [m_i1, m_i2, …, m_i|Cai|]^T for all tasks in Ca_i with regards to UAV u_i, where the entry m_ij = 1 if a task t_j ∈ Ca_i has been added to P_i but removed from the same sequence for violating time window constraint, and m_ij = 0 otherwise. By definition, a task t_j ∈ Ca_i can be assigned to UAV u_i according to the following conditions:

(d). Z_ij ≠ i and m_ij = 0;

(e). let p = argq_ij*(P_i⨁t_j); after inserting t_j into the p-th position of P_i, all tasks in the sequence P_i⨁_pt_j satisfy the underlying time window constraint.

(f). Fo_i – vm_i × $ℱ$(P_i⨁t_j) ≥ Δ.

Condition (d) ensures that tasks are not repeatedly added or removed by the same UAV, condition (e) ensures that all tasks in P_i⨁t_j satisfy the time window constraints after inserting t_j into P_i, and condition (f) ensures the satisfaction of power consumption constraints.

Let ℜ(P_i, Ca_i) = {t_j ∈ Ca_i | Conditions (d), (e) and (f) both hold for t_j and u_i} denote the set of tasks that satisfies the above conditions. To minimize the significance value, each UAV u_i chooses the task t_k∈ℜ(P_i, Ca_i), according to (16), and inserts it into the position p = argq_ik*(P_i⨁t_k) of list P_i.

$$t_k=argmi{n}_{t_j\in \Re \left({\mathit{P}}_i, C{a}_i\right)}{q}_{ij}^{*}({\mathit{P}}_i\oplus t_j)$$

(16)

Different from criterion (14), (16) requires that the UAV only selects out the unassigned task t_k with the minimum marginal significance value. The above process is repeated until no task can be included in P_i, i.e., ℜ(P_i, Ca_i) = ∅ |P_i| = L_i.

3.3.2. Secondary Conflict Resolution

To obtain a conflict-free allocation, the secondary conflict resolution stage iterates between the consensus process and the task removal process until global consensus is achieved across all UAVs.

In the consensus process, the heuristic consensus rule proposed in Section 3.2.1 is used to obtain a conflict-free allocation. Note that each UAV only exchanges the information related to tasks in Ca_i with its neighboring UAVs through local communications. According to the decision rules given in Table 2, once u_i receives a message, it updates, resets, or leaves the stored information Z_ij, Q_ij, T_ij, and s_ij for each task t_j ∈ Ca_i.

Then, in the task removal process, each UAV u_i checks its current task sequence P_i and removes tasks belonging to two sets A′={t_j ∈ P_i | Z_ij ≠ i} and B′={t_j ∈ P_i | T_j(P_i)<T_{j_start} or T_j(P_i)>T_{j_end}} from P_i. Herein, A′ denotes the set of tasks whose information Z_ij is changed after the consensus process, while B′ represents the set of tasks violating the time window constraints. The tasks in A′ and B′ are sequentially removed from P_i using the same method as in Section 3.2.2. In addition, each time a task t_j ∈ B′ is removed, we update its related information as Z_ij → ∞, Q_ij → ∞, T_ij → ∞, and we also set m_ij = 1 such that task t_j is prevented from repeatedly assigning to P_i, according to condition c). These requirements not only avoid invalid task inclusion and removal, but also improve the performance of the entire system.

The secondary inclusion and secondary conflict resolution are repeated alternately until no actions can be made for a period of time. That is, the task reallocation phase has already converged, and an optimized global conflict-free task assignment that satisfies all time window constraints is obtained.

3.4. Convergence Analysis

The first two phases of the DATW algorithm alternate until an initial allocation is obtained, and then, the last phase is performed to reallocate these unassigned tasks. In the ideal case, the system deems to be converged when no changes can be made in the first two phases. However, due to the strict time window constraints, some tasks may be repeatedly included and removed by the same UAV, and thus, an infinite cycle arises. To avoid such reiteration and reallocate those tasks in the last phase, iterations between the first two phases are terminated, provided that the allocations obtained after the second phase remain the same for a specified period.

Moreover, each UAV always aims to decrease the global time cost at each iteration. Specifically, the significance value of a task is highly related to its current contribution to the local time cost of its assigned UAV. In the first two phases, each UAV u_i exchanges the significance of all tasks with its neighbor UAVs, and then, based on these significance values, u_i tries to decrease the global time cost ${\sum }_{n=1}^nℱ\left(\mathit{P}i\right)$ by recursively adding or removing tasks from its task sequence. The last phase reassigns tasks in Ca_i, which collects all unassigned tasks after the first two phases, to agents. In particular, each u_i selects unassigned tasks in Ca_i with the minimum significance value in each iteration of the last phase. Note that the initial significance value for each task is set to infinity and is continuously updated as the significance of a task is highly correlated with the current allocation. Consequently, the DATW algorithm converges when the task significance list of each UAV is not changed for a specified period of time.

Formally, the proposed DATW running on each UAV can be expressed in detail as follows.

Step 1: Performing the task inclusion phase, according to Algorithm 1.

Step 2: Running Algorithm 2. The conflict resolution phase is repeated until global consensus is achieved over all UAVs.

Step 3: Step 1 and Step 2 are repeated until the allocation obtained after Step 2 remains the same for a specified period of time.

Step 4: Carrying out the secondary inclusion stage to assign tasks in Ca_i to agents.

Step 5: Carrying out the secondary conflict resolution stage, performing the consensus process and the task removal process alternately, until the global consensus is achieved over all UAVs.

Step 6: Step 4 and Step 5 are repeated until no changes have been made in the task significance list Q_i for a specified period of time, and then, the algorithm is completed.

Moreover, we obtain the working architecture of the DATW by integrating the above 6steps through the process in Figure 2.

4. Simulation

In this section, numerical simulations are conducted to assess the effectiveness of the proposed DATW algorithm. The proposed method is first verified across search and rescue scenarios with different settings generated randomly. Then, the proposed method is compared with an extension of the CBBA (E-CBBA) method in [26] in terms of various parameter settings.

4.1. Scenario and Simulation Setup

The performance of the DATW method is evaluated based on the search and rescue scenarios proposed in [28]. Suppose that a number of survivors are discovered in a specific area, and a group of UAVs are required to perform rescue tasks on these survivors. The survivors are classified as A-type, B-type, and C-type according to their health condition. All survivors are considered to have equal priority but must be rescued within the different time window constraints summarized in

Table 3. Time Window Constraints [T_{j_strart}, T_{j_end}] of rescue missions for A, B, and C survivors.

Survivors’ Type	Range of Values for the Earliest Start Time Tj_strart of a Rescue Mission	Range of Values for the Latest Start Time Tj_end of Rescue Missions
A	[0 s, 20 s]	[100 s, 120 s]
B	[50 s, 200 s]	[300 s, 400 s]

. It is assumed that one-quarter of the survivors are A-type, one-quarter are B-type and the remaining ones are C-type. The goal is to minimize the average time cost across all rescue tasks and to rescue as many survivors as possible upon the validity time interval. Moreover, in such a scenario, all UAVs and survivors are randomly initialized in a 10,000 × 10,000 × 1000 m 3-D spatial plane. The UAVs’ speeds are assumed to always be 50 m/s, and each UAV takes 30 s to rescue a survivor but executes at most five tasks at a time. Moreover, the UAVs’ initial fuel masses are set as 400kg, and each UAV consumes the fuel at 0.05kg/s.

To better describe the problem size of various scenarios, we introduce a new parameter, namely, the task-to-UAV ratio (TUR), represented by δ ≡ m/n, where m (resp. n) is the number of tasks (resp. UAVs).

Table 4. Parameter Size for Each Instance Type.

Instance Type	UAV Num	TUR
Small	3	δ = 1, 2, 3
	4
Medium	9
	10
Large	15
	16

shows the parameters of all testing scenarios where the UAV number is denoted by n ∈ {3, 4, 9, 10, 15, 16} and the corresponding TUR is set as δ ∈ {1, 2, 3}. Hence, there are 18 = 6 × 3 combinations of scenarios to consider. The time windows for all these combinations are randomly generated, according to Table 3.

4.2. Simulation Steps

In this paper, all experiments are performed on an Intel(R) Core (TM) [email protected] GHz PC, and the simulation program is executed with Python under the Windows 10 operating system. Four basic libraries in Python are employed to simulate the entire process: NumPy, Pandas, Matplotlib, and Time. Specifically, the NumPy is used to handle the calculations of a matrix; the Pandas is used to export the generated data in the simulation and perform further analysis; the Matplotlib is used for drawing two-dimensional diagrams, while the Time is added for addressing problems with time information.

The simulation program consists of a clock module, a data generation module, a task assignment module, and an output module. The clock module, which uses functions from the Time library, is the first component of the simulation. The clock module keeps track of the start and end time of each simulation, which aids in evaluating the computational complexity of the whole algorithm. Meanwhile, the clock module also records the timestamp s_i when each UAV u_i obtains the latest message from the other UAVs. Then, the data generation module is introduced and is based on the functions in the NumPy library. In particular, this module randomly generates the initial scenario for each simulation, according to the settings in Section 4.1. To this end, the task assignment module imports the above-mentioned data and obtains the proper allocation for each UAV. Ultimately, the output module is employed based on the functions in the Pandas and the Matplotlib, such that the visual output of the allocation results is achieved.

4.3. Simulation Results

4.3.1. Validation Experiments

A series of validation experiments are conducted to evaluate the performance of DATW; there are four performance parameters to consider:

• G: the average completion time of all tasks, which is the objective of the studied MTPA and can be obtained from (6);
• Λ: the number of iterations, which is determined by the last time an allocation change is made;
• П: the number of communications, which depends on the passed messages summed over all UAVs during the consensus phases in Section 3.2 and the secondary conflict resolution in Section 3.3.2.;
• Ψ: the percentage of feasible tasks that are allocated to UAVs and the given time window constraints are satisfied.

The above parameter G reflects the objective of MATP, Λ determines the computation time of the whole algorithm, П implies the communication cost of performing DATW, and Ψ illustrates the effectiveness of DATW in handling the time window constraints.

In this section, all experiments are conducted based on the scenarios summarized in Table 4. For each combination in Table 4, 30 instances are created under randomly generated initialization settings. A mesh communication topology is used such that each UAV can communicate with every other UAV. The simulation results are presented in

Table 5. Result of the DATW Method in Search and Rescue Scenario.

UAV Num	Task Num	TUR	Ave. Completion Time (G)	Ave. Iteration Times (Λ)	Ave. Communication Num (П)	Percentage of Feasible Tasks (Ψ)
3	3	1	103.88	1.27	5.83	100.0%
	6	2	149.05	2.47	9.07	93.8%
	9	3	149.02	3.13	11.03	74.1%
4	4	1	128.01	2.2	13.5	96.7%
	8	2	143.66	2.87	16.53	92.5%
	12	3	146.02	3.67	16.43	75.6%
9	9	1	110.37	3.03	42.4	99.3%
	18	2	132.89	5.1	67.73	96.7%
	27	3	139.16	6.4	65.47	84.7%
10	10	1	104.42	3.67	58.83	99.3%
	20	2	129.91	5.8	86.9	96.0%
	30	3	139.31	6.3	72.33	85.8%
15	15	1	100.87	4.8	114.23	98.4%
	30	2	123.82	6.83	147.37	97.0%
	45	3	135.28	8.07	142.27	87.7%
16	16	1	100.64	4.87	130.07	99.8%
	32	2	124.27	6.8	156.5	96.4%
	48	3	135.42	7.93	160.3	86.6%

.

According to Table 5, the average rate of feasible tasks exceeds 70% for each combination, which means that DATW is capable of finding a solution that allocates most tasks to UAVs properly. Moreover, we observe that the parameter TUR directly affects the average completion time and the number of iterations. In particular, the average completion time G logically rises with TUR as there are more tasks required to be performed. Additionally, as TUR becomes larger, each agent has increasingly available task options, such that the algorithm must iterate more times to select the valuable allocation. Hence, parameter Λ is enlarged. Moreover, the parameter П increases with an increasing UAV number since more communications are required to reach consensus when the number of UAVs is increasing. In addition, П appears to take its minimum value at δ = 1 when the number of UAVs is not changed. The reasons seem simple: the number of tasks is relatively small with δ = 1, and hence, the allocated tasks are much easier to satisfy with the time window constraints. In this case, DATW may achieve its optimal allocation before the task reallocation phase.

4.3.2. Comparisons on the Task Reallocation Phase

Task reallocation, the third phase of DATW, aims to allocate the tasks to UAVs that are unassigned after the first two phases. To validate the effectiveness of this reallocation phase, we carry out our experiments to compare and analyze the allocations before and after this phase. Herein, each scenario combination in Table 4 is simulated 50 times under random initialization settings.

From Figure 3, when the TUR is set as δ ∈ {1, 2, 3}, the line graphs depict the average number of allocated tasks before and after the task reallocation phase. The bar graphs depict the average increase in the number of allocated tasks during that phase. Notably, the task reallocation phase can significantly improve the total number of allocated tasks from the solution achieved by the first two phases with various UAV numbers and TURs. Comparing (a), (b), and (c) in Figure 3, for combinations with the same UAV numbers, more tasks are allocated during the third phase as the value of TUR increases. This is because when the value of TUR grows, there are more tasks involved, such that the time window constraints between tasks are more likely to overlap. As this situation leads to greater difficulties in assigning tasks, the task reallocation phase is necessary for obtaining a well-performed allocation. Moreover, we can observe that the number of tasks allocated through the third phase logically increases with the number of UAVs for each value of TUR. In the best case, the average increase in the number of allocated tasks is up to 8.04 when allocating 45 tasks to 15 UAVs.

4.3.3. E-CBBA Versus DATW

Since the proposed DATW and the E-CBBA algorithm in [26] both aim to allocate tasks within validity time windows, we conduct a range of comparisons to determine the merit of the assignments achieved. Herein, we assume that the obtained allocation deems to be a success if each task is assigned to a proper UAV and complies with time window constraints. Then, based on this assumption, we introduce a new parameter, namely, the success rate (SR), to assess the performance of these two algorithms. All comparisons are conducted based on the search and rescue scenarios, where the UAV number is varied as a parameter from two to thirty and TUR is set as δ = 2. Each combination of these scenarios is simulated 50 times under random initialization settings, such that the success rate of a combination is calculated as follows:

$$SR=N_{sa}/50$$

(17)

where N_sa denotes the number of successful allocations over all simulations and 50 is the total number of simulations.

Figure 4 compares the success rate of the DATW and the E-CBBA with different numbers of UAVs. The line graphs show the success rate of the two algorithms, while the bar graphs show the increase in success rate achieved by the DATW. Obviously, it demonstrates that the DATW consistently outperforms the E-CBBA algorithm in terms of success rate. Specifically, the average success rate for DATW is 52.7%, while the average success rate for E-CBBA is 34.7%, which means that the proposed DATW achieves an 18% (=52.7% − 34.7%) increase in the average success rate. Notably, in the combination of 22 UAVs and 44 tasks, all tasks are successfully allocated within validity time windows 32 times by DATW, but only 10 times by E-CBBA.

4.3.4. Topology Comparison

The dynamic environment with moving UAVs incurs various changes in communication typologies between UAVs, and thus, the proposed algorithm is required to perform effectively in different communication topologies. Figure 5 depicts four basic communication topologies with nondirected graphs: a fully connected mesh topology, a row topology, a circular topology, and a star topology. Specifically, we aim to analyze how these different typologies affect the performance of DATW based on a search and rescue scenario and assess the DATW with respect to the number of communications (П) and the percentage of feasible tasks (Ψ). All experiments are conducted on a scenario combination with 16 UAVs and 32 tasks, where the combination is simulated 50 times under random initialization settings.

The scatterplot in Figure 6 shows the result of over 50 simulations, which compares the PI, E-CBBA, and DATW in terms of parameters П and Ψ. Obviously, the parameter Ψ is independent of the specific topology, while the parameter П appears to vary according to the type of topology. In particular, the circular topology and the star topology have a similar degree of communication, such that П and Ψ are similar across topologies for PI, E-CBBA, and DATW. Moreover, for each algorithm, the parameter П decreases notably with fully connected mesh topology and increases with the row topology. Such an increase in П is due to information requiring multiple communications among UAVs to reach a consensus when the network is not fully connected. Moreover, the PI algorithm cannot effectively handle the task allocation problem under various time window constraints, while the proposed DATW based on PI always achieves well-performing allocations. Additionally, we observe that under each communication topology, the distributions of the parameter П are always similar, in terms of the DATW and PI algorithms. That is, the DATW achieves superior allocations without imposing any extra communication burdens and allocates the most tasks to UAVs within time window constraints.

5. Conclusions

In this paper, a distributed task allocation algorithm (DATW) is proposed for the MTAP problem with time window constraints. This algorithm embeds the time window constraints into the PI-based decentralized framework for the first time and aims to minimize the average completion time of all tasks. As an extension of the PI algorithm, the DATW method uses a three-phase architecture, including the task inclusion phase, conflict resolution phase, and task reallocation phase. Herein, the average number of allocated tasks exceeds 70% by the aid of the newly introduced task reallocation phase. Another novel idea is to correlate the significance value of tasks with the time window constraints, such that each task can be completed as early as possible during its validity interval. Moreover, the start time of each task is broadcasted among agents via communication typology to allocate tasks in a validity time interval without imposing any extra communication burdens. To this end, a conflict-free allocation with a minimum average task completion time can be obtained by the proposed DATW algorithm, where the vast majority of tasks are assigned within the validity time window. A series of simulations are conducted under a search and rescue scenario. The experimental results confirm that the DATW is effective and superior in increasing the total number of allocated tasks as well as minimizing the average completion time of the tasks. Moreover, compared with existing (CBBA-based) solutions, results show up to an 18% increase in success rate (SR) using the DATW. Thus, the DATW can successfully enable a team of UAVs to perform complex missions under various time window constraints.

Future work will focus on facilitating real-time task allocation in dynamic environments, which indicates integrating the presented DATW into the asynchronous framework and addressing the MTAP with latency in task computation and communication.