A Survey of Recent Extended Variants of the Traveling Salesman and Vehicle Routing Problems for Unmanned Aerial Vehicles

Abstract

The use of Unmanned Aerial Vehicles (UAVs) is rapidly growing in popularity. Initially introduced for military purposes, over the past few years, UAVs and related technologies have successfully transitioned to a whole new range of civilian applications such as delivery, logistics, surveillance, entertainment, and so forth. They have opened new possibilities such as allowing operation in otherwise difficult or hazardous areas, for instance. For all applications, one foremost concern is the selection of the paths and trajectories of UAVs, and at the same time, UAVs control comes with many challenges, as they have limited energy, limited load capacity and are vulnerable to difficult weather conditions. Generally, efficiently operating a drone can be mathematically formalized as a path optimization problem under some constraints. This shares some commonalities with similar problems that have been extensively studied in the context of urban vehicles and it is only natural that the recent literature has extended the latter to fit aerial vehicle constraints. The knowledge of such problems, their formulation, the resolution methods proposed—through the variants induced specifically by UAVs features—are of interest for practitioners for any UAV application. Hence, in this study, we propose a review of existing literature devoted to such UAV path optimization problems, focusing specifically on the sub-class of problems that consider the mobility on a macroscopic scale. These are related to the two existing general classic ones—the Traveling Salesman Problem and the Vehicle Routing Problem. We analyze the recent literature that adapted the problems to the UAV context, provide an extensive classification and taxonomy of their problems and their formulation and also give a synthetic overview of the resolution techniques, performance metrics and obtained numerical results.

1. Introduction and Motivation

Drones or Unmanned Aerial Vehicles (UAVs) are remotely operated aircraft without on-board human pilots. They were first used for military purposes partly with the intent of keeping pilots at a safe distance from dangerous missions and recently they have found a wide variety of different purposes and usages in civil applications, which has been a driving force in the emergence of the technology and its quick expansion. For delivery and transportation services, UAVs can be used to deliver parcels to customers safely [1,2] or to transportdiagnostic samples and medical laboratory items between health care facilities [3]. Moreover, drones are being increasingly deployed in emergency and disaster management situations [4]. Their cameras allow a view of disaster scenes to be obtained promptly, and they can collect critical data that is unavailable to response teams on the ground. Drones are also able to provide indications to firefighters about the direction of the propagation of fire, allowing a sharp reduction of their response time [5]. Another interesting application for UAVs is area coverage for either monitoring, surveillance, mapping, or network connectivity, where a UAV can become an airborne base station or data mule [6]. For agriculture, remote sensing provides a more accurate view of the fields and of all irrigation systems, thanks to sophisticated cameras and sensors (thermal cameras, hyperspectral cameras, light detection and ranging scanners, etc.) mounted on UAVs. Drones can also be useful for identifying diseased plants early and to establish an inventory of crops or to assess the soil salinity of arable land [7]. For industrial warehouses, UAVs are also useful for conveniently establishing inventories through RFID [8].

In most of the applications, at least two issues have to be addressed simultaneously. The first one is how to implement the targeted application properly, for example, with proper drone features, proper sensors, proper delivery mechanisms. The other one is related to the mobility of the drone: a UAV should be present at the target location(s) in the shortest possible time with the proper positioning (altitude, angle, etc.), and then remain on site for a proper amount of time. To achieve this, in the absence of direct control from a human operator, the UAV would follow an automatically computed and established path. The computation of an efficient, or even optimal, collision-free trajectory of a UAV is a central issue since it is generally non-straightforward problem due to potential constraints such as obstacles, time or energy cost, delay constraints, air turbulence. For this purpose, different techniques have been developed but it should be noted that to a large extent, the same family of techniques can be adapted to different applications. In the next subsection, we start with an overview of the general problem of UAV trajectory computation and existing reviews.

1.1. Overview of the Trajectory Computation Problems and Existing Surveys

The same general idea of computing an optimized vehicle itinerary is encountered in the related literature, expressed under different terms such as vehicle routing, trajectory optimization, path planning, motion planning and so forth. The differences in the terms correspond to actual differences in meaning, with different assumptions and models—described below—partly depending on their different focus on the three following important points to consider when determining a UAV trajectory. First point, the trajectory should be feasible, that is, the planned path should satisfy dynamic constraints and should be enforced by its control system. Second point, the safety of the drone should be ensured, that is, the UAV should successfully prevent collisions with obstacles or with the ground during its flight. Third point, the UAV path should be efficient and the optimal path (or at least one efficient path) is obtained by introducing a cost function and minimizing it within an acceptable calculation time.

When focusing on the first point, adopting the most detailed physical model and often operating at smaller scales, the concept and the terms of trajectory optimization are used. They refer to an optimal control problem that consists in identifying a trajectory that minimizes some measure of performance such as flight time or fuel consumption, while satisfying a set of constraints on the kinematics (e.g., position, velocity and acceleration) and the dynamics (e.g., forces and moments) of the vehicle, as a function of time [9,10]. Trajectory optimization takes into account the dynamic constraints of the aircraft and produces a solution as time-indexed states and controls such as positions, speeds and accelerations. When reducing the level of detail of the physical model, for example, ignoring vehicle dynamics but still taking into account position and geometry of UAVs and objects of the environment, the focus is on the second point and the problem becomes path planning or motion planning, where the main objective is to avoid obstacles. Occasionally, variants of path planning may conserve some vehicle dynamics, for instance, motion or wind constraints.

Finally, considering a higher level, more macroscopic view (focus on the third point), one can find the class of routing problems, where the question is to organize vehicles to visit specific places. The Traveling Salesman Problem (TSP) and the Vehicle Routing Problem (VRP) are classic prominent problems in operational research and combinatorial optimization. The TSP focuses on computing the shortest possible route that visits all customers and returns to the starting position. Whereas, in the VRP, routes are assigned to a set of vehicles that must serve a set of customers such that the total cost of the operation is minimized. The Task Assignment is a sub-problem of the Routing Problem. Instead of assigning goods to deliver to UAVs, it consists in finding the optimal assignment of tasks to perform at specified places to UAVs, subject to mission constraints. The UAV Task Assignment Problem shares some characteristics with the VRP but differs by allowing, for example, multiple visits and subtours. Unlike path planning and trajectory optimization problems, the UAV Task Assignment Problem does not consider the kinematics and dynamics of UAVs.

For each form of problem (trajectory optimization or routing) and sub-problems, there exists a large body of literature in the context of UAVs and accordingly, there also exists a number of recent surveys of which we give an overview in the rest of this section. We start with one remark inspired by Reference [9], namely, in many cases, formulations of UAV routing problems ignore flight dynamics while studies on the optimization of the trajectory of drones often ignore any higher-level aspects of routing. The study in Reference [9] proposed a unified mathematical formulation as the most general form of optimization problem—the UAV Routing and Trajectory Optimization Problem (UAVRTOP), where a fleet of UAVs has to visit a set of positions assuming both generic kinematics and dynamics constraints. Wind conditions, collision avoidance between UAVs and obstacles can also be incorporated into the model.

Considering path planning solutions, among the recent reviews, we can cite the study in Reference [11], which discussed several techniques introduced to achieve UAV path detection, planning and obstacle avoidance for real-time communicative environments. We can also cite Reference [12], which proposed a comparative study of existing UAV path-planning algorithms with both heuristic and non-heuristic methodologies including Potential Field, Genetic, Floyd-Warshall, Mixed Integer Linear Programming (MILP) and so forth. The review in Reference [13] was interested in Coverage Path Planning problems (CPP) with UAV. The authors surveyed existing studies addressing simple geometric flight patterns for the CPP problem. The study in Reference [14] analyzed a large number of UAV path planning algorithms in 3D environments. The authors proposed a 3D path planning algorithm taxonomy based on five categories and discussed the characteristics of each one. In general, in path planning solutions, geometric constraints are introduced and paths can be searched by techniques such as tree search or iterative path computation algorithms on graphs (e.g., Dijkstra, Floyd-Warshall, etc.) but widely different methods have been proposed, such as UAV path planning through computational intelligence, surveyed in Reference [15].

On the part of routing, although numerous research papers focused on routing optimization problems and associated methods to compute UAV trajectories, only a few surveys that analyzed these studies have been proposed. Among these, in Reference [16], the authors proposed a survey on optimization problems for UAV civil applications but with a broader scope than routing, as they targeted all existing UAV optimization issues in general. The authors dedicated a section to discuss the existing literature based on UAV routing related to the Vehicle Routing Problem and the Traveling Salesman Problem. In that section, they synthesized the related articles through a table titled “routing for a set of locations”, identifying the general methodology of each article (mathematical formulation, closed-form solution, control theory, game theory, empirical results, …), the number of drones in the problem, the characteristics of the drones and the general class of application (transport, agriculture, infrastructure & construction, entertainment & media, general, other). They also included an informative overview and discussion of the different features, variants and constraints explicitly introduced for drones.

As mentioned, another interesting study that summarizes the literature about joint routing and trajectory optimization for UAVs is proposed in Reference [9]. The authors introduced a compact taxonomy with the purpose of classifying differing formulations of the optimization problems expressing both routing and trajectory objectives and constraints simultaneously and related them to their proposed unified problem formulation UAVRTOP. Their taxonomy highlights key features in both types of problems, enumerates the assumptions and is used to synthesize in a table the reviewed literature, by identifying: the number of UAVs and their characteristics, their mobility model; the constraints and properties of the waypoints to visit; the environment in terms of geometry; the number of depots; the time constraints, the general approach to solve the problem and the motivating application.

1.2. Motivation and Organization of This Review

The purpose of this survey is to provide pointers for researchers and practitioners in the UAV field that are interested in getting insights on how to compute efficient drone trajectories between points of interest when looking at problems from a macroscopic scale. This can be useful for instance for applications that involve mostly long-range considerations such as communications, problems such as refueling or for applications that are genuinely about transportation (of parcels or data) between remote points. Thus, in our study, we focus specifically only on routing optimization problems and on their formulation and their resolution. Our survey is different from the cited studies since we selected the most recent papers that proposed only extended variants of the Traveling Salesman Problem (TSP) and Vehicle Routing Problem (VRP) for UAVs.

For other problems than routing, for instance for problems of drone control, collision avoidance and, in general, problems of drone mobility at small time scales, there already exist reviews, such as the ones addressing path planning [11,13,14], presented in the previous section and a large literature on drone control.

For our topic of routing problems applied to UAVs, the two existing reviews [9,16] have different scopes and our survey is complementary to them. Reference [16] is informative in terms of comprehensive insights and considerations related to the applications or drone characteristics. For routing problems where trajectory optimization, path planning or collision avoidance are also required, Reference [9] is also an excellent reference. Both surveys do address routing aspects, but to our knowledge, there is a lack of a source entirely devoted to this class of problems. This is the primary objective of the present survey. This exclusive focus arguably allows the classification of the problems and the methods in categories more finely and the inclusion of information about the reported performance of resolution methods.

Since UAVs applications often cover interdisciplinary problems, one of the objectives is also to guide readers that are not entirely familiar with the mathematical formulation and the application of methods that originate from the transport optimization literature. Through the surveyed works, we give a flavor of the type of problems that can be formulated and of the manner in which they can be solved. Moreover, this has the added benefit of providing instructional examples on how classical routing problems can be modified to incorporate UAV constraints or can be adapted to UAV applications.

Then another important part of the survey is the description of resolution methods that have been applied by each article to obtain solutions. Keeping in mind that the surveyed problems are often computationally difficult to solve, the TSP and the VRP being NP-hard, the choice was made to collect and report information on the performance of the resolution algorithms in terms of the quality of the solution or/and the computation time (in Table 2) and correspondingly, we report information on the size of the problems, the metrics, objective functions and simulation parameters.

To organize the description of the works and for convenience of the reader, we introduced structured classifications in terms of problem tackled and application domains and but also highlighted their adopted approach for solving the problem as illustrated in Table 1. This classification is summarized through a proposed taxonomy.

This survey is organized as follows: In Section 2 and Section 3, we provide a detailed review of articles focusing on UAVs path optimization problem derived from the Traveling Salesman Problem and Vehicle Routing Problem, respectively. In Section 4, we propose a taxonomy for routing problems. We present an analysis of the reviewed papers and a synthetic summary table in Section 5. Finally, we conclude in Section 6.

2. Traveling Salesman Problem TSP

The Traveling Salesman Problem (TSP) is a combinatorial optimization problem. It determines, for a list of cities, the shortest possible route that goes through each city once and returns to the origin city. The challenge of the problem is that the traveling salesman wants to minimize the total distance traveled.

TSP is dedicated to applications like transportation services, goods distribution and delivery, planning, logistics.

Developments in drone technology give rise to new variants of TSP. In the following subsections, we review recently published studies that proposed extended variants of TSP for UAV’s trajectory optimization. Some of them focused on the parcel delivery by truck and drone, other studies were proposed in the context of DTN where UAVs are in charge of carrying Data. UAVs are also deployed for monitoring, surveillance and filming.

2.1. Delivery-Transportation

Parcel delivery trucks can be assisted by one or several drones. Providing a delivery truck with a drone gives many benefits. The main advantages of a drone are its speed, its independence of road and its low costs. However, the load capacity and traveling range of UAVs are limited, which is the reason behind the use of the drone for last-mile delivery.

2.1.1. Delivery by 1-Truck 1-Drone

In the literature, the truck drone distribution concept gives rise to a new optimization problem called the Traveling Salesman Problem with Drone (TSP-D) [17]. The authors of Reference [17] modeled the TSP-D problem as a novel integer program and were able to solve instances of reasonable size optimally. They also developed several route-first cluster-second heuristics for large instances based on local search and dynamic programming. This solution is able to find the best assignment of truck and drone deliveries for any given delivery sequence. A theoretical analysis of the developed heuristics is proposed in this paper by providing a worst-case approximation guarantee. Experimental comparison is performed to compare heuristic solutions to exact solutions of the TSP-D. A numerical study is also proposed to assess the performance of a drone delivery system in various customer densities, geographic distributions and drone speeds. Experimentation showed that substantial time savings are possible with this concept in comparison to truck-only delivery.

The TSP-D Problem introduced in Reference [17] was extended in Reference [18] with the aim of maximizing the drone coverage and usage in parcel delivery. However, in Reference [18] a truck can deliver and pick a drone up not only at a node, that is, the depot or at a customer location but also along a routing arc (en-route). To solve the problem with en-route operations, the authors developed a greedy heuristic, based on the Greedy Randomized Adaptive Search Procedure GRASP [19] and modified it to take into account the waiting time in the cost function. In this study, the proposed heuristic was tested on benchmark instances and the results have shown the benefits introduced with the en-route approach in terms of energy-saving and reducing waiting time.

Motivated by a scenario in which a Drone works in collaboration with a traditional delivery truck to distribute parcels, the authors of Reference [20] introduced two mixed integer linear programming formulations for two delivery-by-drone problems based on the Traveling Salesman Problem. The first one is called Flying Sidekick Traveling Salesman Problem (FSTSP). It addresses the challenge of determining optimal customer assignments for a UAV working in tandem with a delivery truck. Based on FSTSP, a drone travels along with a truck, they start from a common Distribution Center (DC) and must return to the same DC at the end of the tour. At any customer, the drone might be launched from the truck, starting a sortie. In this case, it will start the delivery to a customer and then make a rendezvous with the vehicle at a subsequent customer. The objective of FSTSP is to minimize the time required to serve all customers and return both vehicles to the depot. The second optimization problem proposed in Reference [20], called the Parallel Drone Scheduling TSP (PDSTSP), will be presented in Section 2.1.2.

The study in Reference [19] introduces a new variant of TSP-D following the hypotheses of the FSTSP proposed in Reference [20]. In FSTSP, the objective is to minimize the delivery completion time, or in other words, the time coming back to the depot, of both truck and drone. In the new variant called min-cost TSP-D, the objective is to minimize the total operational cost of the system, including two distinct parts. The first part is the transportation cost of truck and drone, while the second part relates to the wait time a vehicle has to wait for the other whenever the drone is launched. The authors of Reference [19] proposed a MILP formulation for the problem that is an extended version of the model proposed in Reference [20]. Then, they proposed two algorithms to solve their problem. The first algorithm (TSP-LS) was adapted from the approach proposed for FSTSP [20], in which an optimal TSP solution is converted to a feasible TSP-D solution by local searches. The second algorithm, a Greedy Randomized Adaptive Search Procedure (GRASP), is based on a new splitting procedure to generate a TSP-D solution from TSP tour. Once a TSP-D solution has been generated, it is then improved by local search operators. Performance evaluation of various instances with different sizes and characteristics of both objective functions was proposed. The results showed that GRASP outperforms TSP-LS in terms of solution quality under acceptable running time. However, TSP-LS provides a lower quality of solution in limited computation time.

The authors of Reference [21] introduced a new Hybrid Genetic Algorithm (HGA) with adaptive diversity control to effectively solve the TSP-D under both min-cost and min-time objectives. HGA is a combination of the genetic algorithm and local search technique together with a population management, diversity control and a penalization mechanism to balance the search between feasible and unfeasible search spaces. This method has been used to solve many variants of VRP efficiently. They also presented problem-tailored components such as local searches, crossover, restore method and penalty mechanisms to improve the performance of the algorithm significantly. Different computational experiments show the improvements in terms of solution quality under both objectives and different instance sets and the importance of the new proposed elements.

The authors of Reference [22] proposed a hybrid heuristic called Hybrid General Variable Neighborhood Search (HGVNS) to solve the Flying Sidekick Traveling Salesman Problem (FSTSP) introduced in Reference [20]. The initial solution is generated from the optimal TSP solution obtained by a TSP solver. Next, an implementation of the General Variable Neighborhood Search (GVNS) is used to obtain the delivery routes of the truck and the drone. Computational experiments show the potential of the algorithm to improve the delivery time significantly. This study demonstrates that collaborative work of truck and drone can drastically decrease delivery times by up to $67.79\%$.

2.1.2. Delivery by 1-Truck and m-Drones

The Parallel Drone Scheduling TSP (PDSTSP) is the second optimization problem proposed in Reference [20] and it is associated with devising optimal truck and UAV assignments in the case of a DC located in close proximity to customers. In PDSTSP, a single depot exists, from which a single delivery truck and a fleet of one or more identical UAVs must depart and return. The truck serves customers along a TSP route, while the UAVs serve customers directly from the DC. Unlike the FSTSP, there is no synchronization between a UAV and a truck in the PDSTSP. The objective of PDSTSP is to minimize the latest time that a vehicle returns to the depot, such that each customer is served exactly once. Two simple heuristic methods for solving the FSTSP and PDSTSP, respectively, were introduced. The authors noted that only small size instances could be solved up to optimality due to the NP-hard nature of the proposed problems. They performed numerical analyses that demonstrated the effectiveness of the proposed heuristics and studied the trade-offs between using drones with faster flight speeds and greater endurance.

The authors of Reference [23] proposed solving the problem of PDSTSP, introduced in Reference [20], with an original iterative two-step heuristic, composed of: a coding step that transforms a solution into a customer sequence and a decoding step that decomposes the customer sequence into a tour for the vehicle and series of trips for the drone(s). The authors conducted experiments on benchmark instances from the literature and they confirmed the efficiency of their approach.

The study in Reference [24] proposes an extension of the TSP-D problem in which a truck travels with m $(m>1)$ drones (called TSP-mD) instead of one drone in TSP-D. The authors adapted the greedy randomized adaptive search procedure (GRASP) proposed in Reference [19] and proposed an Adaptive Large Neighborhood Search (ALNS) heuristic to the resolution of this problem. Experimental results on different instances showed that the GRASP could yield a more efficient solution when combining a truck with more than one drone. Moreover, the ALNS is more efficient than the GRASP in this context.

2.1.3. Delivery by n-Trucks and m-Drones

The multiple Traveling Salesman Problem with Drones (mTSPD) is an extended variant of the Multiple Traveling Salesman Problem (mTSP) with the implementation of drones in the operations. In the mTSP, the m salesmen must visit n nodes, forming m tours in total, one per salesperson. There are two main constraints in the mTSP problem. The first constraint requires that all salesmen must depart and return to the starting node (depot) at the end of the trip, no matter which tour they choose. The second constraint states that every salesman must travel to a specific set of customers between the first and the last node, given that each node can only be visited once by the assigned salesman except the starting node. The typical objective of the mTSP is to find the total shortest tour that each salesman must travel from the depot to visit the assigned set of cities and back to the depot.

The authors in Reference [25] propose a new routing model, the Multiple Traveling Salesman Problem with Drones (mTSPD), which implements both trucks and drones in the last-mile delivery. The model is a variation of the classic TSP problem and the extension of the previous FSTSP [20] model. They generalize the FSTSP to the mTSPD in which multiple drones and multiple trucks perform deliveries. A mixed integer programming (MIP) formulation to solve mTSPD is presented with the objective of minimizing the arrival time of both trucks and drones at the depot after completing the deliveries. The authors also proposed a new heuristic called Adaptive Insertion algorithm (ADI) to solve mTSPD. ADI consists of two phases: building the mTSP solutions and applying removal and insertion operators to the initial mTSP solution to construct the mTSPD solution. Three mTSP construction heuristics were proposed: Genetic Algorithm, Combined K-means/Nearest Neighbor and Random Cluster/Tour. The results revealed that in small size generated problems, both solvers and GA-ADI reached an optimal solution but GA-ADI solved the problem significantly faster than the CPLEX Optimizer [26] which is designed to solve integer programming problems. In addition, by comparing the results of the mTSPD with other existing last-mile delivery models (TSP/FSTSP), the delivery time can be significantly improved by using multiple drones in route planning.

In large-size instances, the authors conducted another set of experiments in which they use GA-ADI heuristic. The obtained solutions are compared with the optimal solution from the Adapted FSTSP heuristic (allowing that one drone per truck and the drone has to return to the original truck it is launched from). The experimental results show that the proposed model using multiple drones and trucks along with the heuristic provides shorter delivery completion time than merely using trucks alone, multiple trucks a single truck-drone in the operations.

The authors of Reference [27] extended the problem PDSTSP proposed in Reference [20] by considering two different types of drone tasks: drop and pickup. Once a drone achieves a drop, it can either return to the depot to deliver the next parcels or fly directly to another customer for pickup. Integrated scheduling of multiple depots hosting a fleet of trucks and a fleet of drones is further studied to achieve operational excellence. This problem is uniquely modeled as an unrelated parallel machine scheduling (PMS) with sequence-dependent setup (for travel-distance), precedence-relationship (for drop-pickup) and re-entrant (for multi-visit and time-window), which gave a framework to consider those operational challenges adequately. A constraint programming approach is proposed and tested with problem instances of multiple trucks, drones and depot, and a hundred-customers distributed across an 8-mile square region. The objective of the proposed approach is to minimize the maximum time needed to satisfy all tasks.

2.2. Logistics: Path Planning

The authors of Reference [28] proposed two parallel optimization algorithms to solve the traveling salesman problem (TSP) for unmanned aerial vehicle (UAV) path planning. The first one is based on the genetic algorithm and it is called the Improved Genetic Algorithm (IGA) and the second one is a hybrid algorithm, called the Particle-Swarm-Optimization-based Ant Colony Optimization algorithm (PSO-ACO). IGA utilizes a real−coded method and effective selection operation, crossing operation, mutation operation to improve the global searching ability of UAV path planning and simultaneously guarantees its convergence speed. PSO-ACO mixes the idea of PSO and ACO to overcome the early maturity of ACO, which is beneficial to obtain a globally optimal solution for UAV path planning. Ants in PSO-ACO system have the particle characteristics that can update the locally optimal solution and globally optimal solution after completing every single traversal. Experimental results and analyses demonstrate that the proposed IGA and PSO-ACO algorithms can obtain more reasonable and effective solutions for the problem of UAV path planning.

2.3. Delay Tolerant Networks

Delay Tolerant Networks (DTNs) are sparse networks where complete direct end-to-end communication between source and destination can seldom be established. Routing mechanisms in DTN rely on the mobility of the nodes to connect disconnected nodes, carrying messages around the network to overcome path disconnection. The proactive DTN approach consists of introducing dedicated nodes whose only purpose is to establish communication between ordinary nodes, and relieve them from energy-consuming work, such as message routing and forwarding.

Unmanned Aerial Vehicles (UAVs) have many potential applications in wireless communication systems, like providing cost-effective wireless connectivity for devices without infrastructure coverage due to, for example, heavy shading by urban or mountainous terrain or damage to the communication infrastructure due to natural disasters.

The study in Reference [29] addresses the use of Unmanned Aerial Vehicles (UAVs) as DTN relays and introduces a proactive scheme called Deadline Triggered Pigeon with Travelling Salesman Problem with Deadlines (DTP-TSP-D). In the proposed DTN, ground nodes can only communicate by flying UAVs with the capacity of carrying messages from one location to another. Each UAV either belongs to one node or a cluster of nodes and its role is to hover over their home-ground node (or ferrying around their home cluster) until they are triggered to deliver messages directed to other ground nodes. The triggering criterium is based on the deadlines of the messages present in the UAVs buffer, evaluating its ability to deliver all of them on time. It uses a developed TSP Genetic Algorithm to compute the route that achieves the most (timely) deliveries. The performance of DTP-TSP-D has been compared to dedicated node protocols, namely SIRA (SIngle Route Algorithm) and MRT-Grid (Multiple route algorithm). The performance metrics used were the delivery ratio and the average delay. The results show that DTP-TSP-D achieves higher delivery rates than its competitors while keeping a consistent average delay.

2.4. Intelligence, Surveillance and Reconnaissance (ISR)

The Fuel Constrained UAV Routing Problem [30] (FCURP) arises in Intelligence, Surveillance and Reconnaissance (ISR) where one or several UAVs with limited payload and fuel constraints are used to gather information about a set of potential targets. FCURP is a generalization of the Asymmetric Traveling Salesman Problem (ATSP), where for every pair of cities i and j, the distance from i to j is different from the one from j to i. In FCURP there are multiple depots and the unique vehicle is allowed to refuel at any depot. The objective of the problem is to find a UAV path where each target is visited at least once, the fuel constraint for UAV is never violated along the path and the total fuel consumed by the UAV is a minimum.

Approximation algorithm and fast heuristics were developed to solve a generalization of the single-vehicle routing problem with fuel constraints. A mixed-integer, linear programming formulation was also proposed to find optimal solutions to the problem.

2.5. Monitoring, Tracking, Filming

The Close Enough Traveling Salesman Problem (CETSP) is a variant of TSP. In the CETSP, the salesman must visit a specific region containing the vertex rather than the vertex itself.

In several applications such as monitoring, tracking, filming and so forth, UAVs do not have to reach the target location. Hence, the UAV path, as well as its energy, could be optimized. When the UAV is equipped with a sensor, it can operate successfully from a certain distance to the target. Another example occurs in special military operations, where UAV may drop its cargo as close as possible to the target locations.

The authors of Reference [31] proposed an exact algorithm, based on Branch-and-Bound and Second-Order Cone Programming (SOCP) to solve the CETSP. They studied the feasibility of embedding path algorithm in a UAV in order to compute a trajectory that visits all points with the highest resolution of images. In this context, TSP and CETSP have been studied and both approaches presented satisfactory results.

3. Vehicle Routing Problem VRP

The Vehicle Routing Problem (VRP) is a combinatorial optimization and integer programming problem. It generalizes the traveling salesman problem. The VRP focuses on optimizing a set of routes for a fleet of vehicles to traverse in order to deliver to a given set of customers. The objective of the VRP is to minimize the total routes cost.

VRP is usually used in the context of delivering goods located at a central depot to customers who have placed orders for such goods.

Many studies based on VRP have been published for terrestrial applications. With the emerging of UAVs new variants of the vehicle routing problem have been designed to satisfy the UAVs characteristics.

3.1. Delivery-Transportation

In this section, we present research studies from the literature that focused on Truck-Drone delivery and on delivery by Drone only based on the Vehicle Routing Problem.

3.1.1. Delivery by Truck(s) and Drone(s)

As for the traveling salesman problem, new variants of the routing vehicle problem have been proposed to deal with the delivery with drones for the last-mile.

The authors of Reference [32] introduced the Vehicle Routing Problem with Drones (VRPD) where a fleet of trucks equipped with drones deliver packages to customers. Drones can start the delivery and then the trucks can pick them up at the depot or at any of the customer locations. The objective of VRPD is to minimize the time required to serve all customers and return the fleet to the depot. The authors in this article studied the maximum saving time due to the use of the drones. They derived a number of worst-case results and concluded that the number of drones per truck and the speed of the drones relative to the speed of the truck have an impact on the worst-case results.

The proposed study in Reference [32] was extended in Reference [33]. It was shown that drones allow the truck to parallelize tasks and are able to take advantage of crow-fly distances.

The authors of Reference [34] proposed two heuristics to solve the VRPD. The first heuristic is called Two-Phase Heuristic (TPH). It is a two-stage approach that starts by creating efficient VRP tours and then creating VRPD solutions through the insertion of drones. The second heuristic is called Single-Phase Heuristic (SPH) and it focuses on creating a good VRPD solution right from scratch. The authors carried out numerical experiments on large-scale TSP instances to assess the performance of both heuristics. Numerical results showed that the TPH provides better results than the SPH in most cases. A conclusion from these studies is that good VRPD solutions can be constructed by using a two-stage heuristic, starting from good VRP solutions.

A new type of delivery called Same-Day Delivery (SDD) is recently rising and are provided by companies such as Amazon, FedEx, UPS and so forth. Due to these new delivery services, customers are able to order goods online and receive them the same day. In SDD, customer requests may occur while the vehicles are already on the road. In order to serve new customers, the vehicles need to return to the depot to pick up the ordered goods before they can serve the new customer requests. This leads to dynamic vehicle routing problems with depot returns. Work on these problems is based on urban vehicles to offer the SDD. The authors of Reference [35] analyzed how drones can be combined with conventional delivery vehicles to improve same-day delivery performance. They proposed a dynamic vehicle routing problem called Same-Day Delivery routing Problem with Heterogeneous Fleets of drones and vehicles(SDDPHF). In this problem, customers order goods over the course of the day. These goods can be delivered either by a UAV or by an urban vehicle within a delivery deadline. Some assumptions and constraints are considered in this study, such as: drones are faster but have a limited capacity as well as charging times. Also, vehicle capacities are unlimited but vehicles are slow due to urban traffic. To decide whether an order is delivered by a drone or by a vehicle, the authors introduced a policy function approximation based on geographical division. A computational study revealed that the combination of drone and vehicle fleets might reduce routing costs significantly.

3.1.2. Delivery by Drone(s)

Planning the routes for drones is challenging due to multiple operational characteristics, including multi-trip operations, recharge planning, as well as calculating energy consumption. In this context, the study in Reference [36] introduced a Multi-Trip Drone Routing Problem (MTDRP) with time window which explicitly considers the influence of payload and distance on flight duration. The authors proposed two modeling schemes and developed exact algorithms based on Branch-and-cut for their formulations.

A sizing problem related to civilian drone delivery activity in an urban area is presented in Reference [37]. The authors proposed a new formulation for a drone delivery problem, Capacitated Vehicle Routing Problem with Time Windows (CVRPTW): multiple parcels to carry for each drone with added the batteryâ™s energy constraint. The proposed model is based on three different objectives joined together: the minimization of the distance, the number of drones and the number of batteries used. This model is based on a defined battery strategy, which is: $100\%$ of battery capacity for each mission performed by a drone. According to this strategy, the drone operator tries to minimize the three objectives in order to reduce as much as possible the cost. The authors divided their study into three cases in order to focus on each objective separately. This approach helps to show the impact of each objective. Based on this study, the authors proposed a decision-making tool for the design of a drone fleet in the case of forecast deliveries over a time horizon under operational constraints.

One of the major constraints to the use of drones in delivery services is the power source, which is usually limited. To deal with this limitation, charging points could be considered in order to enhance vehicles covering range. With the classical VRP, vehicle autonomy is not considered. An extension of VRP to cope with the energy constraint is proposed in Reference [38]. This new variant is called the Green Routing Problem (GRP). The Green Vehicle Routing problem GVRP is proposed for alternative fuel vehicles that have constrained energy. This problem seeks to find for one tour for each vehicle, that starts and ends at the depot, visiting a subset of vertices including not only customers but also alternative fueling stations, such that the total distance traveled is minimized.

In addition to the energy constraints, there is another problem called time-dependent routing problem where UAVs should be able to change, adapt, modify and optimize their routes in real-time. This problem could be solved by proposing several individual objective functions. The authors of Reference [39] proposed a multi-objective Green UAV Routing Problem (GUAVRP), which minimizes seven objective functions: total traveled distance; UAVs maximum speed; number of vehicles used; makespans of the last collected and delivered package; average time spent with each package; and maximize batteries load at the end of the schedule. Furthermore, the proposed model considers the drones’ operational requirements, such as: maximum weight they are able to carry, minimum battery Depth-of-Discharge; UAV maximum speed and so forth. In the proposed approach, a dynamic scenario is considered, in which new orders may arrive at any moment. Then, drones en route can be considered after finishing their current trip. The proposed model considers UAV autonomy and the possibility of charging them during their routes. The authors in this paper used the Multi-Objective Smart Pool Search (MOSPOOLS) Matheuristic, in order to obtain a set of non-dominated solutions for the proposed routing problem.

The GVRP was originally designed for trucks with high carrying capacity. Unlike these terrestrial vehicles, drones have small carrying capacities and have limited flight times, meaning that in general, they can only carry a small number of packages per route. In the GVRP problem, the multiple trips to the depot are not allowed. Then to satisfy all demands while decreasing the maximum traveling time, a high number of drones is needed. Therefore, the authors of Reference [40] proposed two multi-trip VRPs (MTVRP) for drone delivery that compensates for each drone its limited carrying capacity by reusing drones when possible in order to deal with the GVRP limitation for UAVs and also to take into account the effect of battery and payload weight on energy consumption. The first contribution minimizes the costs subject to a delivery time limit, while the second one minimizes the overall delivery time subject to a budget constraint. The authors proposed an energy consumption model for multi-rotor drones demonstrating that energy consumption varies approximately linearly with payload and battery weight. This consumption model is implemented as Mixed Integer Linear Programs (MIPLs) and solved based on Simulated Annealing heuristic. Simulation results proved the benefit of reusing drones and optimizing battery size in VRPs.

3.2. Monitoring, Surveillance

Small drones generally have fuel constraints. Therefore, they must make one or more refueling stops during surveillance and data collection missions. The study in Reference [41] addressed an essential aspect of refueling in the context of routing of multiple small UAVs to complete a surveillance or data collection mission. The authors in this work modeled a multiple-UAV routing problem as a two-stage stochastic optimization problem with uncertainty associated with the fuel consumption of each UAV. The proposed two-stage Fuel-Constrained Multiple-UAV Routing Problem (FCMURP) works as follows: In the first stage, the FCMURP aims to compute a route for each UAV that starts and ends at the depot where all the UAVs are initially stationed, such that each target is visited at least once by some UAV and no UAV runs out of fuel as it traverses its route. If a UAV does not have enough fuel, it can make a refueling stop at any depot. This fuel restriction is imposed in the first stage using a nominal value for the fuel consumed by any UAV as it travels between a pair of targets/depots. In the second stage, The FCMURP aims to satisfy the UAV fuel-constraint by adding additional refueling trips to the first-stage routes. In the first stage, the objective of the FCMURP is to minimize the cumulative traveling distances of all UAVs while in the second-stage its goal is to minimize the expected travel distance for the additional refueling trips. The Sample Average Approximation (SAA) is used to obtain statistical estimates of the lower and upper bounds for the optimal solution of the two-stage stochastic program. However, the SAA run time can be prohibitive for medium- and large-scale test instances. Then, the authors proposed a tabu search heuristic to find sub-optimal solutions for the two-stage stochastic model. Based on extensive computational experiments, the authors proved the benefits of the two-stage model over the deterministic model; they also confirmed the advantage of the heuristics considered to obtain solutions of high quality.

3.3. Logistic: Task Assignment

Thanks to logistics systems and task assignment, some strategies have significantly reduced the operation cost of logistics in enterprise and improved the transport efficiency.

The authors of Reference [42] proposed the task assignment model for UAV based on the Vehicle Routing Problems with Time Windows (VRPTW).

This model consists of assigning a swarm of drones to serve customers within predefined time frames and taking into account multi-constraints such as weight coefficients, the constraints of the UAV and so forth. Then an improved Particle Swarm Optimization (PSO) algorithm is used to solve the task assignment problem with multiple constraints. The authors compared the improved PSO with the Genetic Algorithm. The simulation results show that the proposed PSO is well suited to solve the problem of task assignment for UAVs.

The study in Reference [43] addresses the problem of locating and routing of small UAVs to maximize the total score collected from visited interest points by flight routes of UAVs. The authors formulated the problem as an integer linear program (ILP) and also developed a novel ant colony optimization metaheuristic approach. The authors mentioned that the proposed metaheuristic could find the best known or close to the best-known solution in a short time.

The wind has a significant impact on the control of fixed-wing UAVs. During its flight, the UAV’s heading angle and ground speeds may change due to the wind speeds and direction. In Reference [44], in order to optimize the task allocation and path planning of fixed-wing UAV, the steady wind environment was introduced into the optimization model and a Variable-Speed Dubins path Vehicle Routing Problem (VS-DP-VRP) model was established considering the dynamic constraints of UAV. The optimization objective of this model is to minimize the time required by UAVs to complete all of the tasks. Considering that the VS-DP-VRP is still an NP-hard problem, the authors choose the Genetic Algorithm (GA) to solve this problem by designing a crossover operator and mutation operator. Based on the implemented metaheuristic, simulation results showed that an effective UAV task allocation and path planning solution under steady wind could be provided.

3.4. Wildfire

Wildfires are a class of dangerous disasters that are very difficult to fight, especially fires in mountainous terrain when using traditional fire-fighting equipment. Unmanned aerial vehicles (UAVs) are increasingly used in forest firefighting. Given the suddenness of forest fires, the adaptive and dynamic assignment of fire-fighting tasks for drones is of great importance and the current assignment of fire-fighting tasks cannot solve this problem.

The study in Reference [45], proposed an adaptive and dynamic multiple task assignment method for UAVs. The authors in this paper proposed a mathematical formulation for the adaptive and dynamic firefighting task assignment optimization problem. They also proposed an assignment algorithm based on the Particle Swarm Optimization (PSO), to solve their problem and conducted experiments to confirm the effectiveness of this algorithm.

4. Taxonomy of Routing Problems

In this section, we provide a taxonomy in order to classify the recent existing UAV’s optimization problems found in the literature, by their routing problems, their adopted approaches, and their application domains, and also guide the reader to identify the most suitable routing problem and approach for his application. We first start by giving a comprehensive description of the attributes considered in this taxonomy. Then, we apply the proposed taxonomy to the recent studies analyzed in our survey, as illustrated in

Table 1. Characteristics of the reviewed papers

Ref.	Vehicles					Routing Problem									Application					Approach
						TSP				VRP
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26	27	28
[17]			×			×									×												×
[18]			×			×									×												×
[20]			×	×		×									×												×
[19]			×			×									×												×
[21]			×			×									×					×							×
[22]			×			×									×												×
[27]				×		×									×													×
[23]				×		×									×												×
[24]				×		×									×												×
[25]					×		×								×					×							×
[28]	×					×												×		×	×	×
[29]	×					×										×				×
[30]	×						×									×											×
[31]	×								×								×									×
[32]					×					×					×												×
[33]					×					×					×												×
[34]					×					×					×												×
[36]		×												×			×									×
[35]					×					×					×												×
[37]		×										×					×									×
[38]		×											×				×									×
[39]		×											×				×								×
[40]		×											×	×	×									×
[41]		×											×				×						×				×
[42]		×									×							×		×	×
[43]		×								×								×					×
[44]		×								×								×		×
[45]		×								×									×		×

.

Available Vehicles

• 1- UAV = 1; Only one UAV is used
• 2- UAV = m; A fleet of UAVs is deployed
• 3- UAV = 1, Truck = 1; One UAV works in tandem with one truck
• 4- UAV = m, Truck = 1; A fleet of UAVs working in tandem with one truck
• 5- UAV = m, Truck = n; A fleet of UAVs working in tandem with many trucks

Routing Problems

All the existing UAV’s routing problems published during the last few years are either extended variants of the Traveling Salesman Problem or the Vehicle Routing Problem.

• Extended variants of the Traveling Salesman Problem

  –

  6- TSP-D: Traveling Salesman Problem with Drone. Given a number of target positions, one depot, one Drone and a cost metric, the objective of TSP-D is to determine a UAV’s tour that visits each target only once and returns to the depot such that the total tour cost (e.g., distance traveled or completion time) is minimized.

  –

  7- mTSP-D: multiple Traveling Salesman Problem with Drones is a generalization of TSP-D in which more than one Drone is used. The objective of mTSP-D is to determine a set of m tours such that the total cost of all tours is minimized and each target is visited by only one Drone. When UAVs are working in tandem with trucks in delivery applications, the corresponding routing problem is called mTSP-D only if a fleet of trucks and a fleet of Drones are considered.

  –

  8- ATSP-D: Asymmetric Traveling Salesman Problem with Drone is also a generalization of TSP-D in which the bi-direction cost between a pair of targets may not be identical like in TSP-D.

  –

  9- CETSP-D: Close Enough Traveling Salesman Problem with Drone is a generalization of TSP-D where each target is identified by its neighborhood. Then, the objective CETSP-D is to determine a UAV’s tour that visits the neighborhood of each target only once and returns to the depot such that the total tour cost is minimized.

• Extended variants of the Vehicle Routing Problem

  –

  10- VRP-D: Vehicle Routing Problem with Drone. VRP-D aims to determine a set of routes for a fleet of Drones that visit a set of targets such that the total routes cost is minimized.

  –

  11- VRPTW-D: Vehicle Routing Problem with Time Window with Drone is a generalization of VRP-D where the service at each target starts at a given time called time window. This time window is specified at each target and is defined by the earliest and latest time of the service.

  –

  12- CVRPTW-D: Capacitated Vehicle Routing Problem with Time Window with Drone is a generalization of VRPTW-D where each Drone has some capacity (e.g., payload capacity) and the total demand served by each Drone does not exceed its capacity.

  –

  13- GVRP-D: Green Vehicle Routing Problem with Drone. GVRP-D is a variant of VRP-D that aims to reduce the number of Vehicles used by allowing the Drone to refuel its energy during its tour.

  –

  14- MTVRP-D: Multi-Trip Vehicle Routing Problem is a variant of the GVRP-D that aims to cope with the limited payload capacity of UAVs. The GVRP-D cope with the limited energy of Drones; however, these flying vehicles might not be able to carry all the load to satisfy all customers on their routes. Then, they can return to the depot and perform multiple trips.

UAV’s Applications

The routing problems for UAVs path optimization have been adopted in numerous applications such as:

• 15- Transportation and delivery: such as parcel delivery
• 16- Communication: UAVs can be used as flying relay points to re-establish connectivity or data mules that pick up Data from one site and deliver it to another.
• 17- Surveillance, monitoring, tracking: UAVs equipped with sensors can be used for area monitoring
• 18- Logistic processes: UAVs can be used for stock inventory in warehouses.
• 19- Disaster management: UAVs can be used in forest fire-fighting or for locating and tracking people or animals.

Approaches to solving the routing problems

• 20- GA: Genetic algorithm
• 21- PSO: Particle Swarm Optimization Algorithm
• 22- ACO: Ant Colony Optimization Algorithm
• 23- Tabu Algorithm
• 24- Simulated Annealing algorithm
• 25- Smart Pool Search Algorithm
• 26- Exact Algorithm such as:

  –

  Branch and Bound

  –

  Branch and Cut

• 27- Local Search and dynamic programming such as:

  –

  GRASP: Greedy Randomized Adaptive Search Procedure

  –

  GVNS: General Variable Neighborhood Search

  –

  ALNS: Adaptive Large Neighborhood Search

  –

  Route First Cluster Second heuristic

  –

  Two Steps Approach

• 28- Constraint programming

5. Analysis of the Reviewed Papers

In this survey, we reviewed the most recent studies (i.e., papers that have been published for at most four years). These articles introduced extended models of TSP or VRP optimization problems and some corresponding heuristics to solve them. Those models have been adopted in further published papers while adding or relaxing some constraints to improve their corresponding objective function (e.g., completion time, traveling distance, energy consumption, etc.). To validate the designed models, authors proceed by considering exact algorithms and heuristics using instances from the well known TSPLIB (i.e., Traveling Salesman Problem Library) while others proposed their own test instances. Many of them have been adopted in subsequent studies to compare heuristics performance with each other. This type of comparative studies is essential and impactful in order to establish the advantages and drawbacks of each heuristic. However, it is difficult to select the best one since they are based on different approaches and the authors used different metrics for performance evaluation. For these reasons, we chose to present the different reviewed papers in a detailed table (see

Table 2. Detailed description of the reviewed articles.

Year	Ref.	Scope	Approach	Simulation Parameters	Metrics	Simulation Results
2016	[17]	• Improve Truck and Drone schedule for Parcel Delivery • RP: TSP-D	• IP • First-cluster second procedure • Minimum Spanning tree	• Instance from [46] • 30 instances • 100 km $\times 100$ km square region • 1 depot • 10, 12, 100 nodes • 1 UAV • 1 Truck	• Saving time compared to delivery by truck only • The number of drone nodes • The number of truck nodes • The travel distance of the truck and the drone • The waiting time of both the drone and the truck	• An optimal solution to the TSP is a $(1+\alpha )$—approximation to the TSP-D. • The benefits of combining a truck and a drone are time savings, on average between a factor $1.4$ and 2. • Using the exact partitioning outperforms the greedy heuristic. • The heuristics that use the exact partitioning (TSP-ep) show consistent savings of about $30\%$ and the greedy heuristic (TSP-gp) of approximately $20\%$. • The greedy approaches (TSP-gp-swap and TSPgp-all) are extremely fast and solve all the instances with 100 nodes within a few seconds.
2017	[18]	• Maximize Drone coverage in Parcel Delivery • RP: TSP-D	• IP (same in [17]) • The Greedy Randomized Adaptive Search Procedure (GRASP) including en route drone operations	• Benchmark instances by [46] • 15 km $\times 15$ km, 30 km×30 km, 50 km ×50 km square regions • 1 depot • 10, 20 and 50 nodes • 1 UAV • 1 Truck • UAV endurance= 30 min • UAV speeds = 40 or 60 km/h • The truck speed = 40 km/h • The en route operation cost = 1 min	• The percentage of savings over the TSP solution • The percentage of battery savings related to the remaining endurance for each operation • The waiting time	• With regards to the savings over TSP solution, the en-route operations seem to be less useful when the drone is faster than the truck. When truck and drone have the same speed, the obtained result is improved due to the en-route operation. • Battery savings and waiting times are significantly improved. • The average increase of $10\%$ in battery savings using en-route operations.
2015	[20]	• Find the optimal customer assignments for a UAV working in tandem with delivery truck • PR: TSP-D	• IP • Savings • Nearest neighbor • Sweep	• 72 instances • 8-miles square region • 10 nodes • 1 depot • 1 UAV • 1 Truck • UAV endurance = 20–40 min • UAV speed = 15,25,30 mile/h • Truck speed 25 miles/h	• Running time • Solution quality: minimizing delivery time.	• The IP approach outperformed all other heuristics in terms of solution quality. However, it is not scalable. • Savings heuristic performed well in terms of running time (fraction of second) • Nearest and Sweep are not competitive in terms of solution quality.
2015	[20]	•Minimize the latest time that a UAV or a truck returns to the depot •RP: TSP-D	•MILP •Savings heuristic •Nearest neighbor •PMS subproblem: binary integer programming formulation to obtain optimal PMS solutions and the popular longest processing time (LPT) first heuristic	•720 instances •8-mile square region •$10,20$ nodes •$80e{}_{1}90\%$ of nodes are UAV-eligible according to weight •1 Depot •$1,2,3$ UAVs •1 Truck •UAV-Truck speed = 25 miles/h •UAV endurance = 30 min	•Running time •Solution quality •Delivery time •Speed versus endurance	•The solutions obtained via the LPT first heuristic were often identical in quality to those obtained when the IP formulation was used to solve the PMS subproblems optimally. •The LPT first heuristic is generally faster than the IP approach to solving the PMS subproblems. •The LPT first heuristic appears to provide near-optimal solutions to the PMS subproblems. •Delivery-by-drone systems may be made more efficient by utilizing faster UAVs, even at the expense of reduced flight endurance.
2018	[19]	•Minimize the delivery completion time •RP: TSP-D	•TSP-LS (Local Searches) •GRASP	•65 instances •100 km×100 km, 500 km× 500 km, 1000 km×1000 km •1 depot •$10,50,100$ nodes •UAV-Truck speed = 40 Km/h •UAV endurance = 20 min •$80\%$ of customers can be served by UAV	• min-time: completion time (time of delivery, achievement) • min-cost: drone time tour + truck time tour + waiting time	•GRASP outperforms TSP-LS in terms of solution quality. •The running time is slower with GRASP.
2018	[21]	•Minimize the latest time that a UAV or a truck returns to the depot •RP: TSP-D	•Hybrid Genetic Algorithm (HGA)	•Instance sets from [19,20] •72 instances of 10 nodes from [20] •60 instances of 50 and 100 nodes from [19] •1 Depot •1 UAV •1 Truck •UAV-truck speed = 40 Km/h •UAV endurance = 20–40 min	• min-time: completion time (time of delivery, achievement) • min-cost: drone time tour + truck time tour + waiting time	•For min-cost TSP-D, the average objective values of solutions of HGA are even better than those of the best solutions of GRASP found in most instances. •HGA can improve existing best-known solutions by $6.16\%$ and $15.10\%$ on average for 50 and 100 nodes instances, respectively. •HGA performs better in large instances (i.e., 100 node instances). However, for two instances, GRASP performs better. •Regarding run time, HGA is $1.5$ to 2 times slower than GRASP due to its more complex design. This result is acceptable since it still can deliver significantly better results in less than 1 min for 50-node instances and less than 5 min for 100-node instances. •For min-time TSP-D, HGA can also improve the existing best-known solutions found by GRASP on all instances but not as significantly as in min-cost TSP-D. •HGA performs approximately $1.5$ times slower than GRASP but can still deliver better solutions in less than 1 min and 5 min for 50 and 100 node instances, respectively. •For min-time TSP-D, HGA can also improve the existing best-known solutions found by GRASP on all instances but not as significantly as in min-cost TSP-D. •HGA performs approximately $1.5$ times slower than GRASP but can still deliver better solutions in less than 1 min and 5 min for 50 and 100 node instances, respectively.
2018	[27]	•Minimize the time needed to satisfy all delivery requests •RP: TSP-D	•Constraint Programming (CP) •Variable orderings heuristics (VOH)	•120 instances •8-mile square region •20, 50 and 100 nodes •[1–2] Depots •[1–3] Trucks •[1–2] UAVs •UAV is 30 percent faster than the truck •UAV endurance= 14 miles	•Minimize the maximum completion time •Run time	•The computational study demonstrates the merit of both streamlining drop-pickup tasks and integrating multiple-depot. •CP is a promising technology for the UAVs scheduling problem. •CP proves optimality for larger test instances.
2018	[22]	•Minimize the delivery time •RP: TSP-D	•MIP •Hybrid General Variable Neighborhood Search (HGVNS)	•Instances from [47]: •32 km × 32 km square region •1 Depot •1 UAV •1 truck •UAV speed = 80.47 km/h •Truck speed = 56.32 km/h •UAV endurance = 24 min •The percentage of feasible drone customers is $80\%$ •Instance from [17] •TSPLIB instances: •25 instances •[51–200] nodes •$85\%$ and $90\%$ of nodes eligible to UAV delivery	•Solution quality •Completion time •Run time •Gap between the average solution and the best-known solution found in the literature	•Collaborative work of truck and drone can drastically decrease delivery times up to $67.79\%$ •In the instances introduced in [47], HGVNS improved the solution of the majority instances, achieving an improvement up to $24.84\%$. •In the instance proposed in [17], the best improvement occurred in one instance with 75 nodes (also called customers) where the drone traveled twice as fast as the truck and the least improvements were observed for instances where both vehicles presented the same speed. •In the instances based on the TSPLIB instances, the best solution found in these instances shows an improvement of $45.48\%$ when over the optimal TSP tour value.
				•$10\%$ to $15\%$ are truck-only nodes. •UAV speed = 40 km/h •Truck speed = 40 km/h •UAV endurance = 40 min
2018	[23]	•Minimize the delivery completion time •RP: TSP-D	•MILP •An iterative two-step heuristic •Single-start two-step •Multi-start two-step	•Instance of [20]: •10, 20 nodes •UAV speed = 25 miles/h •UAV endurance = 30 min •1 Depot •1 Truck •1,2,3 UAVs •Instance TSPLIB: •UAV speed 1, 2, 3, 4 or 5 times faster than truck •1,2,3,4,5,6 UAVs	•Completion time •Running time •Percentage of UAV-eligible nodes •UAV speed •Number of UAVs •Solution quality	•With regards to the instance from [20]: •The average completion time value decreases when the number of drones increases. •The proposed scheme is optimal with a single UAV while it is not when using several UAVs. •With regards to TSPLIB instance: •Quick convergence towards good-quality solutions.
2019	[25]	•Minimize the delivery time •RP: mTSP-D	•MILP •Adaptive Insertion Heuristic (ADI) •ADI-GA •ADI-Kmeans-NN •ADI-Random	•185 test instances •2,25,50 nodes •1000 unit×1000 unit square region •1 Depot •n UAVs •m Trucks •UAV speed is 1.5 times faster than the truck speed •large instance: TSPLIB from [48]	•Minimizes the arrival time of drones and trucks at the depot •Run time	•In small size instance, GA-ADI reached the optimal solution significantly faster than the CPLEX Optimizer. •By comparing the results of the mTSPD with TSP and FSTSP, the delivery time can be significantly improved by using multiple drones in the route planning. •The total time to complete the delivery by using one truck and multiple drones is $39\%$ faster than the total delivery time using the truck alone and around $18.80\%$ faster than the total delivery time using one truck and one drone. •In large instances, the effect of drones becomes less significant once the number of trucks increases. •Allowing multiple drones in delivery operations can significantly accelerate deliveries. •The runtime increases once the number of trucks increases in different instances. •The runtime of the ADI heuristic rises quickly as the number of nodes increases. •The use of multiple drones and trucks along with ADI provides shorter delivery completion time than simply using trucks alone, multiple trucks and a single truck-drone in operation.
2017	[28]	•Minimize the UAV tour time •RP: TSP-D	•Improved Genetic Algorithm (IGA) •Particle-Swarm Optimization based Ant Colony Optimization algorithm (PSO-ACO) •Ant Colony Optimization (ACO)	•10,20,30,40,50 nodes •$10\times 10$ square region •1 UAV	•Shortest path •Optimal solution	•With the increasing of the number of nodes, IGA and PSO-ACO can obtain shorter paths compared with the contrast ACO method. •IGA and PSO-ACO algorithms can obtain more reasonable and effective solutions for the problem of UAV path planning.
2018	[29]	•Ensure network communication in a DTN context •RP: TSP-D	•Deadline Triggered Pigeon with Travelling Salesman Problem Deadlines (DTP-TSP-D) •Single Route Algorithm (SIRA) •Multiple route algorithm (MRT-Grid)	•$100\times 100$ square region •25 nodes •5,15 UAVs •UAV speed= 1 unit of distance per unit of time	•Delivery ratio •Average delay	•The DTP-TSPD outperforms the other schemes in terms of delivery ratio, although it does not always present the best average delay. •DTP-TSP-D average delay is not significantly affected by using more UAVs in the network.
2014	[30]	•Data gathering from target location with UAV energy refueling •RP: ATSP-D	•MIP •Approximation algorithm (Approx) •Improvement heuristic	•50 instances •5,10,15,20,25 nodes •5000 unit×5000 unit square region •5 Depots •1 UAV = Fixed-wing •Maximum fuel capacity = 4500 units •UAV speed constant •UAV Minimum turning radius = 100 units •UAV turning angle [0, 2$\pi$]	•Solution quality	•The average quality of the solutions produced by the improvement heuristic is much superior compared to the average quality of the solutions found by the approximation algorithm. •Using the feasible solution produced by the heuristic as a starting point, the CPLEX Optimizer was able to improve the quality of the solution further. •For instances with 25 targets and 5 depots, the CPLEX Optimizer was further able to improve the average solution quality of the instances to $1.39\%$. •The proposed algorithms can be effectively used in conjunction with standard optimization software like the CPLEX Optimizer in order to obtain high-quality solutions for the FCURP.
2018	[34]	•Minimize the delivery time •RP: VRP-D	•Two-Phase Heuristic (TPH) •Single-Phase Heuristic (SPH)	•TSPLIB instance: •1 Depot •3 Trucks •1 UAV per Truck	•Solution quality	•The TPH provides better results than the SPH in most cases and there are a few cases where the SPH produces better or competitive results. •Good VRPD solutions can be constructed by using a two-stage heuristic, starting from good VRP solutions.
2018	[35]	•Minimize the delivery time •RP: VRP-D	•Policy Function Approximation (PFA): •Policy 1: ${\pi }^{Vehicles}$ •Policy 2: ${\pi }^{PFA}$ •Policy 3: ${\pi }^{Drone}$	•300, 400, 500 and 800 expected orders •Depot ≥ 1 •[1–20] UAVs •[1–5] Trucks •UAV speed = 40Km •Truck speed = 20Km	•Solution quality (served/requests)	•PFA is effective both in a geography in which customers are more tightly clustered around the depot as well as one in which the customer distribution is more heterogeneous. •Comparisons to other threshold-type benchmark policies reveal that the districting by the proposed PFA is highly beneficial in significantly increasing the expected number of services by the fleets. •Using the PFA, a combination of drones and vehicles may reduce operational resources required to serve the majority of customers.
2018	[37]	CVRPTW	•Mathematic model	•Time window [0.1, 0.5] h •Number of node [5,10] •UAV: md4-1000 model •UAV payload = 1.2 kg •UAV endurance = 70 min •UAV speed= 13 m/s	•The fleet size •The traveled distance •The energy consumption •Impact of the Time Window	•Delivering in short time windows requires an important number of batteries. •The number of batteries needed may exceed the fleetâ™s size. •Three problem classes are proposed: 1-minimizing the distance traveled 2-minimizing the number of UAVs and 3-minimizing the number of batteries used. •Based on the three proposed cases, we can evaluate the fleet and battery sizes and the necessary energy consumption to deliver a defined set of customers during a defined time window.
2017	[39]	•Minimize the delivery time with UAV refueling •RP: GVRP-D	•MILP •Multi Objective Smart Pool Search (MOSPOOLS) Matheuristic	•Lower layer: •1 km×1 km square region •10 nodes •Upper layer •5 km×5 km square region •25 nodes •Common parameters: •UAV capacity in [0.3 5] kg •UAV speed = [20 120] km •3 inter layer/supporting points •Depot = 5 recharging points	•Minimizes seven objective functions: •Total traveled distance •UAVs maximum speed •Number of used UAVs •Makespans of the last collected and delivered package •Average time spent with each package •Maximize batteries load at the end of the schedule.	•By minimizing the last collect, the model minimized the last delivered with high correlation between both objectives. •If the schedule finishes with more charged batteries, then the solutions run higher distances. •Using more drones resulted in schedules that finish earlier.
2017	[40]	•Minimize the delivery cost while reusing UAVs •RP: MTVRP-D and GVRP-D	•MILP •Simulated Annealing (SA) heuristic	•100 random instances •Small area 0.25 km×0.25 km •[6 8] nodesâ“delivery locations •Large area 1 km × 1 km •1 depot •UAV capacity = 3 kg •UAV speed = 6 m/s •[125 500] nodes •UAV: Multirotor helicopter	•Energy consumption •Delivery time •Run time	•Minimize cost in dollars or delivery time while considering battery weight, payload weight and drone reuse •Increasing battery and payload weight can noticeably increase energy consumption, which in turn reduces flight time. •To minimize the cost or the delivery time for UAVs deliveries, the payload weight, battery weight and flight time should be considered. •The energy consumption increases at an approximately linear rate with battery and payload weight. •UAVs consume approximately the same power regardless of being in hover or flying at a constant speed. •SA implementations find near-optimal solutions to small instances of the DDPs and provide consistent results for larger instances. •Reusing UAVs to perform multiple trips is an essential strategy for reducing costs. •Optimizing battery weight is an effective strategy for reducing the total cost and overall delivery time.
2018	[41]	•Minimize the traveling distance while performing refueling •RP: GVRP-D	•Two-stage stochastic optimization problem •Two-stage stochastic program •Sample average approximation (SAA) •Tabu search-based heuristic	•100 unit × 100 unit square region •5 Instances of 10 nodes •120 instances of 20,30 nodes •1 Depot •4 refueling sites •UAVs number in 3,4,5 •UAV capacity in 2.25$\lambda$, 2.5$\lambda$, 2.75$\lambda$, 3$\lambda$	•Minimum travel costs for UAV routes •Fuel consumption •Run time	•The solutions obtained with the heuristic are computationally efficient and not very different from the solutions for deterministic models and that they are also very robust in the face of uncertainty.
2016	[43]	•Optimize UAV task assignment •RP: VRP-D	•MILP •Novel Ant Colony Optimization metaheuristic approach (ACO)	•60 test instances. •$200\times 200$ NM2 (square nautical miles of less than 100 nodes •$300\times 300$ NM2 for larger instances. •UAV speed = 60 knots •UAV number [6,12]	•Maximize the total score of covered interest points	•Most instances are found difficult to solve optimally. •Ant colony optimization metaheuristic can provide the best-known or close to the best-known solution in a short time.
2018	[44]	•Optimize UAV task allocation and path planning •RP: VRP-D	•GA based optimization algorithm	•Airspeed = 10 ms •Minimum turning radius = 200 m •3 targets •1,2 UAV s	•Minimizing the time required for UAVs to visit all the targets and return to the starting point	•The allocation of tasks and the planning of the route are integrated and optimized under the influence of the wind, the result of which is the best possible solution in the wind. •The actual flight time will be reduced by $35.06\%$ in the presence of wind.

) including the approach adopted by each of them, the instance description and vehicle characteristics, the evaluated metrics and the conclusions drawn by the authors. This table is a good guide for the readers and researchers to select, for example, the suitable UAV optimization problem or the proposed approach depending on their available instances.

In this survey, we first proposed a classification of the reviewed papers based on whether they are based on the Traveling Salesman Problem or the Vehicle Routing Problem. Then, we introduced a second classification based on the application domain. According to this classification, we observed that the same applications appear in both TSP and VRP optimization problems. Hence, one cannot decide if a given application is automatically linked to one of the specific families of optimization problems. We also noticed that about $48\%$ of the cited papers are proposed for transportation and delivery applications. It shows the importance of the use of UAVs in such applications and the reason behind the interest brought by current companies [1,2] to invest in UAV projects. The well known Traveling Salesman Problem and Vehicle Routing Problem were originally designed for terrestrial vehicles. Those optimization problems cannot be directly applied to UAVs since these aerial vehicles have different characteristics from terrestrial vehicles (e.g., aerial vehicles, limited energy, limited capacity, etc.). That is why extended variants of TSP and VRP were proposed for UAVs path optimization. Most of them do not consider UAV kinematics and flight dynamics. It is evident that including these constraints in the optimization problem can only increase the complexity of solving the problem. However, these constraints respect the UAVs characteristics and would have a significant impact on the resulting UAV paths. The authors of Reference [9] included the UAV kinematics and flight dynamics in their problem model, in, perhaps, one of the most general formulations, taking into account trajectory optimization and routing. However, in this formulation, it has not been shown how to solve the problem in practice, neither by an optimization programming solver nor by a meta-heuristic. The UAVs kinematics and system dynamics constraints are more considered in studies focusing on the Trajectory Optimization Problem, which is the second category of path optimization not discussed in this survey.

6. Conclusions and Future Directions

In this survey, we proposed a review of recent existing literature devoted to UAV path optimization problems, focusing on the well-known routing problems: the Traveling Salesman Problem and the Vehicle Routing Problem which were originally designed to optimize the trajectory of urban vehicles. Researchers have extended the TSP and VRP problems to the UAV context in order to meet the requirements of new emerging UAV’s applications. We identified the preexisting survey literature on related topics and then focused on a recent selection of articles that adapted the two classical routing problems to UAV environments. We analyzed them in detail and identified their commonalities and differences, in terms of their application domains and of UAV specific constraints such as energy refueling or payload capacity. We provided an extensive classification through a taxonomy based on: type and number of vehicles, type (and subclass) of routing problem (TSP or VRP), application and resolution techniques. The taxonomy was used to give a synthetic overview of surveyed material through one table, with the aim of helping the readers identifying articles relevant for their purpose. A second detailed table collects extended information on the articles and acts as a summary. It adds the scope of the original articles and also focuses on reporting the provided performance information (metrics, size and set-up experimental problems considered, and experimental results).

The area of UAV in general and UAV routing optimization in particular is a very active field. Present and future research work are expected to expand to new UAV applications, to better combine TSP or VRP formulations to new UAV constraints and to introduce more efficient heuristics, with better solutions, with larger possible instance size or lower computation time. New algorithms for classical TSP or VRP such as machine learning techniques [49,50,51,52] are also a source of new solutions that could be advantageously transposed to UAV problems. For instance, the work in Reference [49] proposed a framework able to solve several combinatorial optimization problems using reinforcement learning. The proposed framework was applied to solve the Capacited Vehicle Routing Problem and performance evaluation showed its efficiency compared to heuristics since it is able to solve larges instances in a very competitive time. An interesting study in Reference [50] proposed a Neural Combinatorial Optimization approach to tackle combinatorial optimization problems with reinforcement learning and neural networks. This approach is also able to solve the traveling salesman problem with close to optimal solution in large instances. The Deep Optimization (DO) [51] is another worth mentioning approach. It is designed to solve combinatorial optimization problems using Deep Neural Networks (DNNs). DO was applied to solve the Traveling Salesman Problem and performance evaluation showed its reliability compared to the heuristic search. In the same context, the work in Reference [52] proposed heuristics using the Convolutional Neural Network (CNN), which is a class of DNN, to solve the traveling salesman problem. Combining combinatorial optimization problems and machine learning techniques to solve vehicle routing problems is valuable and showed very relevant solutions in the studies previously mentioned. This hybrid approach is an interesting and promising line of work to solve UAVs routing problems. We expect that such approach or similar one will tackle several additional factors affecting the UAV’s trajectory such as endurance and energy constraints, collision avoidance, weather conditions, payload and so forth.