Acceleration-Aware Path Planning with Waypoints

Abstract

In this article we demonstrate that acceleration and deceleration of direction-turning drones at waypoints have a significant influence to path planning which is important to be considered for time-critical applications, such as drone-supported search and rescue. We present a new path planning approach that takes acceleration and deceleration into account. It follows a local gradient ascend strategy which locally minimizes turns while maximizing search probability accumulation. Our approach outperforms classic coverage-based path planning algorithms, such as spiral- and grid-search, as well as potential field methods that consider search probability distributions. We apply this method in the context of autonomous search and rescue drones and in combination with a novel synthetic aperture imaging technique, called Airborne Optical Sectioning (AOS), which removes occlusion of vegetation and forest in real-time.

1. Introduction

Autonomous UAVs are becoming more and more adept at handling complex tasks and are thus used in various civil and commercial applications [1]. However, autonomous and adaptive path planning still poses serious challenges due to numerous constraints, such as limited energy, speed, and payload. An additional constraint in search and rescue scenarios can be considered of locating the target as fast as possible. This type of problem is well studied in literature and is termed as a minimum time search problem (MTS) [2,3,4,5,6,7,8,9,10,11,12,13,14]. The most prominent objective in these approaches is to optimize the expected time of target detection [3,4,5,6]; however, other alternative approaches involve optimizing the probability of target detection [7,8,9,15], minimizing its counterpart, i.e., probability of non-detection [10,11] or maximizing the information gain [12,13,16]. Various sub-optimal and heuristics-based algorithms such as gradient-based approaches [7,10,11,12,15], cross-entropy optimization [2,5], Bayesian optimization algorithms [4], ant colony optimization [6], or genetic algorithms [3] have been proposed to address the NP-hard complex problem [13]. These approaches can also be differentiated based on the considered UAV dynamics models, where they either do not consider velocity at all [2,4,5,6,7,8,9,15], or only consider simple linear velocity models [3,10,11] but not acceleration or deceleration.

In addition to efficient path planning, effective imaging is also essential for wilderness search and rescue operations. Large depth of field of conventional cameras (resultant of having a narrow aperture) often project sharply the entire occlusion volumes (such as forests) into the images captured. Objects of interest (people in search and rescue scenario) at a particular distance often remain occluded by the occluders (such as forests). Airborne optical sectioning (AOS) is a wide synthetic-aperture aerial imaging technique that applies camera drones for the real-time removal of occlusion caused by vegetation, such as forests [15,17,18,19,20,21,22,23,24,25]. It has been demonstrated as a capable and effective tool in various applications (such as archaeology [17], wildlife observation [21], and search and rescue [15,24]). AOS’ efficiency concerning the occlusion density, occluder sizes, number of integrated samples, and size of the synthetic aperture has been explained by employing a randomly distributed statistical model [19,25]. By computationally integrating individual images captured over a large scan area (possibly hundreds to thousands of square meters) with narrow aperture camera optics, AOS generates integral images of an extremely shallow depth of field below an occluding volume (cf. Figure 1b). These images enable optical slicing through dense occlusion (caused by leaves, branches, and bushes) and reveal focused targets in each slice (such as artefacts, objects, wildlife, or persons) which would remain occluded for regular cameras (cf. Figure 1c). A fully autonomous and classification-driven UAV (cf. Figure 1a) has been developed and deployed for carrying out wilderness search and rescue operations [15]. The system presented in [15] comprises of two essential and independent modules (AOS based imaging and classification module and an adaptive path planning module). Thermal images are acquired in a 1D sampling pattern and integrated for achieving an occlusion free view of the ground/target. A pre-trained deep learning network achieves an average precision of 86% for detecting person in integral images. In [24] we have already demonstrated how classification of partially occluded persons in forests using aerial thermal images is significantly more effective when AOS is used to integrate single images before classification rather than combined classification results of single images. However, certain environmental conditions (warm background temperature, precipitation, fog, etc.) can affect the performance of the thermal imaging system and thus limiting the efficiency of AOS.

In our previous work [15], a potential-field [14] based adaptive path planning was applied that was driven by confidences from a deep-learning person classifier collected during the flight. The probability map used by the path-planning algorithm was continuously updated with classification confidence values of potential findings. However, acceleration and deceleration were also not considered, which makes path-planning highly unrealistic in practice—especially for rotor-based drone systems that navigate through waypoints. In this work, we focus on achieving a linear velocity over the scanned region while considering the UAV’s acceleration and deceleration to plan its trajectories. Achieving constant velocity over the scanned region is essential for uniform sampling in AOS [15]. We also propose a gradient-based approach that maximizes the probability of target detection with the above mentioned limitations. This proposed path planning algorithm can directly replace our old potential-field based method while utilizing the other aspects of AOS (e.g., integral imaging, person classification) for target detection as it is described in [15]. Our trajectory planning, which takes a dynamic UAV model with acceleration and deceleration into account, is described in Section 2.1 whereas Section 2.2 describes our proposed algorithm to maximize the probability of target detection. Section 3 presents simulation results for various representative probability distributions, and in Section 4 we discuss limitations of our approach and the potential of future UAV models and flight controllers with more dynamic maneuver capabilities for path planning.

2. Materials and Methods

Most commercial drones (especially rotor-based drones) only support piecewise linear waypoint flights and are still not able to fly continuously at a high uniform speed. They need to stop, or slow down (due to limitations of maneuvering and tracking speed capability), before changing directions. Not considering the changes in acceleration and deceleration leads to an unrealistic path planning, as illustrated in Section 2.1. In Section 2.2, we present a new path planning approach that considers acceleration/deceleration and outperforms our previous potential field based method used in [15], as well as classic coverage-based path planning algorithms (i.e., spiral search or grid search).

2.1. Acceleration-Driven Trajectory Planning

As in [15], we want to assume the search region to be discretized into a uniform grid of 30 m × 30 m cells, and that cells are sampled horizontally, vertically, or diagonally at constant velocity to ensure full coverage of the cell within the drones’ field-of-view and uniform sampling. Similar to our work in [15], we utilize calibrated cameras and GPS information to project the digital elevation model of the terrain within each cell. The drone samples multiple single images while crossing each cell and combines them to AOS integral images. Flight speed within a cell must be constant to ensure uniform sampling for AOS, since the person classifier used (YOLOv4-tiny network architecture [26]) was trained with uniformly sampled image data. A non-uniform sampling which differs from the training data would significantly reduce classification rate. Person classification is carried out for each integral image. Each cell is associated with the probability of a person being found within it (cf. Figure 2). The probability maps are initially defined by the rescue team (a uniform probability map is assumed in its absence), and is adapted during flight based on confidence scores of the person classifier. The quality of the initial probability map accounts for how much area has to be scanned until the person is potentially found. In the worst case, the whole area has to be covered. A discussion on adaptive sampling and person classification is out of the scope of this article, and is independent of the presented path-planning approach. The interested reader is referred to [15] for more details. Thus far, constant flight speeds were assumed for the entire path through search area, which does not hold in practice because of finitely fast accelerations and decelerations at the turning waypoints.

Our new trajectory planner ensures that each scanned cell is crossed exactly with the desired and constant scanning speed (see Figure 3a). This, however, introduces additional linear trajectories (for acceleration and deceleration shown in Figure 3b) at both ends (entrance and exit) of the scanned cell. Before entering it, the drone must be decelerated to the desired scan speed and after leaving it the drone can be accelerated again to the drone’s maximum velocity (to progress quicker to the next scan cell, see Figure 3c). We utilize a kinematics based linear motion model to generate these piecewise linear path segments. The distance between the entrance and exit edges of the scanned cell and the auxiliary waypoints at which deceleration starts and acceleration is finished are computed as follows: $d=\left(v_2^2-v_1^2\right)/\left(2a\right)$, where $v_1,v_2$ are the velocities before and after acceleration/deceleration, and $a$ the rate of acceleration/deceleration (positive/negative).

Note, that if neighboring cells are scanned in the same flight direction, the drone is not accelerated or decelerated, but continues at constant scan speed. Thus, the auxiliary deceleration and acceleration trajectories are only needed to bridge distances between non-neighboring cells fast, or if flight direction must be changed. They are flown at the maximum speed that the drone supports without overshooting into the scan cell too fast.

Figure 2 illustrates, for the same probability map as used in [15], the difference in potential-field-based adaptive path-planning when acceleration/deceleration is ignored (Figure 2a), as it was the case in [15], and when it is considered (Figure 2b). With our drone prototype (a 4.5 kg MikroKopter OktoXL 6S12, measured average acceleration/deceleration of 1.4 m/s², a scan speed of 5 m/s, and a maximum flight speed of 10 m/s) and for a 6.3 ha search area, a flight path of 1291 m and a total of 225 s flight time was determined until the target person is found for constant flying speed (i.e., ignoring acceleration/deceleration). In practice, however, this does not hold because the acceleration/deceleration trajectories are ignored. Considering them as explained above results in a 1489 m long flight path and in a flight time of 380 s instead (an increase in required flight time by a factor of 1.7).

We conclude that for waypoint-based path-planning acceleration driven trajectory planning leads to more realistic results and in more accurate estimates of drones’ limited energy and flight time. However, it should be noted that the drones´ energy consumption does not only depend on flight time and acceleration/deceleration but also on many other varying factors like wind, type of drone, gross weight, etc. Our trajectory computes an ideal path without considering these unpredictable outside forces like wind, etc. The flight controllers available on commercial rotor based drones are capable to correct any drifts that are caused by such forces.

When only minimizing flight time or flight distance while ignoring local detection probabilities but considering acceleration/deceleration, classic coverage-based path planning algorithms (i.e., spiral search or grid search) are fastest in traversing the entire search region fully. This is primarily due to a minimum number of direction changes caused by these search techniques, which also minimizes acceleration/deceleration trajectories (see also examples in Appendix A). However, classification probabilities cannot be ignored if the overall goal is to find a person as fast and as reliable as possible. Therefore, we utilize the integral (area) under the sequentially accumulated probability ($p\left(t\right)$) curve w.r.t time (APT) for evaluating the efficiency of different path planning algorithms in Section 3. The upper limit of the integral is set by the path traversal time of the fastest algorithm (t_min).

$$\mathrm{APT}={{\displaystyle \int }}^{t_{min}}p\left(t\right)dt.$$

(1)

The following section presents a new path-planning approach that considers acceleration/deceleration, and that outperforms not only our previous potential-field based method, but also spiral search and grid search for AOS-supported search and rescue applications.

2.2. Radial Gradient Accent (RGA)

Observing the need to minimize turns while utilizing an acceleration driven trajectory planner, we propose a new gradient-based method to maximize the probability of target detection while minimizing flight time by reducing turns (cf. Figure 4).

To decide for the next cell to be scanned from the drone’s current position, a unique set of directions (called radials) is determined in such a way that each radial will have a unique direction and that all radials (originating all from the current cell’s center) together will cross the centers of all unvisited cells. Each unvisited cell is then assigned to its corresponding radial if its center is crossed by it. How many radials exist depends on the resolution of the grid and the number of remaining unvisited cells. For each radial (i.e., the cells assigned to it), we now compute the required trajectories and resulting flight time, as explained in Section 2.1, and select the radial with the highest gradient of accumulated probability w.r.t time (APT). Note, that the APT is similar to the unnormalized cumulative distribution function and the area under this function is the inverse of the Expected Time (ET) of target detection as explained in [2]. The first unvisited cell along this radial is sampled next. Then we iteratively repeat the above process to decide for the next cells to be scanned until all cells are visited or the target is found (i.e., classification with high confidence confirmed by the rescue team, as in [15]).

Overall, this approach follows a local APT gradient ascend strategy with the assumption that after the last cell was scanned, the best local choice for the next scan direction is the one which contributes the highest APT gradient because it maximizes probability accumulation while reducing turns (and with that, flight time). Note that, as explained in Section 2.1, unvisited cell segments are scanned at constant scan speed, but are approached at maximum flight speed. This requires deceleration and acceleration before and after scanning.

3. Results

In this section, we evaluate our Radial Gradient Accent (RGA) method by comparing it against grid search and spiral search which, as fastest coverage-based path planning algorithms, set the speed bars for full grid coverage but do not consider probabilities; and the potential field algorithm in [15] (extended to acceleration/deceleration, as discussed in Section 2.1 and shown in Figure 2a). We apply the probability maps shown in Figure 5 as representative examples for various scenarios during search and rescue operations conducted in the field. Note, however, that the behavior of a lost person depends on many factors (including psychology, physiology, age, gender, etc.) and is not considered here while generating the representative probability maps. For evaluation, only simulations have been carried out to compare the different path planning algorithms under identical conditions. An exact comparison in the field would not be possible, since our online-path planning depends on classification scores which vary with many factors, such as local occlusion, lighting, and wind. Thus, carrying out two search flights under exactly identical conditions is not possible.

Figure 6 and Figure 7 illustrate the performance of the potential field algorithm and our Radial Gradient Accent approach without and with considering acceleration/deceleration on the scattered-smooth probability map (Figure 5b). Results for the other probability maps can be found in Appendix A. Appendix B shows the results of our algorithm for a uniform probability map (assumed in cases where the initial probability map is unreliable or unavailable). The resulting path is more or less uniform spiral like depending on the initial starting position.

Comparing the results in Figure 6 and Figure 7 reveals visually already that RGA requires much less turns (i.e., accelerations/decelerations) and samples more uniformly than the potential field algorithm. A quantitative comparison for all probability maps shown in Figure 5 is presented in Figure 8.

The APT plots in Figure 8 show that although spiral- and grid search always cover the full search region fastest (planned trajectories are presented in Appendix A), they do not maximize detection reliability over time. The latter corresponds to the integral of the accumulated probability w.r.t time plots up to the shortest possible full coverage time.

Table 1. Quantitative comparison (total flight time and distance, APT score) between considered path planning methods and probability maps. While spiral search covers the full search region fastest without considering probabilities, RGA outperforms all other methods in APT score (integral of the accumulated probability w.r.t time). Note that, depending on the chosen starting point, grid and spiral search lead to slightly different results, and grid search might even be faster than spiral search. However, both methods always result in a much lower APT score than RGA since probabilities are not considered.

Probability Map	Method	Time (s)	Distance (m)	APT Score
scattered	grid	2355.98	9945.46	28,523.70
	spiral	2144.08	9518.96	32,121.20
	potential field	3612.76	17,030.36	39,247.74
	RGA	3026.44	14,433.31	42,834.65
scattered-smooth	grid	2355.98	9945.46	26,463.08
	spiral	2144.08	9518.96	28,934.02
	potential field	3704.78	16,799.93	28,814.11
	RGA	2644.96	12,483.69	34,513.97
exponential	grid	2355.98	9945.46	62,702.47
	spiral	2144.08	9518.96	77,296.31
	potential field	3173.29	15,238.73	84,015.36
	RGA	2614.88	12,766.08	91,526.18
multiple-patches	grid	2355.98	9945.46	26,909.31
	spiral	2144.08	9518.96	32,415.52
	potential field	2678.48	12,225.33	44,628.51
	RGA	2551.18	11,624.64	47,666.90
large-patch	grid	2355.98	9945.46	50,121.14
	spiral	2144.08	9518.96	57,197.52
	potential field	3546.04	16,542.41	62,949.78
	RGA	2852.51	13,679.18	70,344.42
small-patch	grid	2355.98	9945.46	24,251.82
	spiral	2144.08	9518.96	29,707.75
	potential field	3158.86	14,456.42	34,517.50
	RGA	2691.61	12,677.97	38,068.94

presents a quantitative summary.

From the results presented in Figure 8 and Table 1 it can be seen that our RGA approach always outperforms spiral- and grid-search, as well as potential field in all cases as its APT score (which is the accumulated probability w.r.t time integral) is significantly higher (28%, 50%, 11% respectively). Note, that APT combines accumulated detection probability over search time.

4. Discussion and Conclusions

This article demonstrates that considering acceleration and deceleration matters for realistic path planning—especially when drones are applied, where velocity is by far not linear over the flight path. Acceleration and deceleration for waypoint sampling has a significant share of total flight time. Taking this into account is important especially for time-critical applications, such as search and rescue. Furthermore, we presented a new path planning approach, Radial Gradient Accent (RGA), which considers acceleration/deceleration. It follows a local gradient ascend strategy that locally minimizes turns while maximizing probability accumulation. In case of the 16 × 16 search grid resolutions that were chosen for our evaluations, RGA requires about 2.36 ms on a laptop equipped with Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz and 8GB of RAM. Considering all radials per iteration, however, might be computationally too intensive for higher resolution grids and lower-performance on-board processors. Furthermore, many of the radials intersect only one cell at its center. Achieving a better radial coverage by considering also cells in the vicinity of the radials might lead to performance improvements. RGA is a greedy choice approach that strongly depends on the quality of local decisions. More efficient heuristics than the max. APT gradient might achieve better overall results (i.e., higher APT scores). Our algorithm could also be adapted and applied to drone swarms for conducting faster search and rescue operations. Additional constraints, such as collision and obstacle avoidance also needs to be considered. All of this has to be explored in the future. Currently, we require uniform sampling in AOS as we apply a uniformly distributed occlusion model to represent forest [19]. However, considering more complex, non-uniform occlusion model (e.g., if sparse and densely occluded regions can be measured during flight) will benefit from non-uniform sampling. This will be investigated in future.

The problem of acceleration and deceleration is caused mainly because today’s flight controllers used in commercial drones follow waypoints in a piecewise linear fashion. Sharper turns at waypoints require major acceleration and deceleration—especially for rotor-based drones. The influence of acceleration and deceleration can be significantly reduced if future flight controllers (especially in combination with agile wing-based drones and fast tracking) allow polynomial instead of piece-wise linear flight paths. Figure 9 illustrates this effect for the assumption that the drone is physically able to fly continuously at constant velocity (without acceleration/deceleration). Instead of flying from waypoint to waypoint, we simulate continuous heading changes (at differential time steps of 1 s) towards the highest probability gradient (at +/−20 deg limits for smooth heading changes at realistic velocities). As shown in Figure 9 for the scattered-smooth probability map, a constant-velocity-based path-planning (continuous gradient) clearly outperforms (in APT integral and sampling uniformity) an acceleration-aware path-planning, such as RGA, that is applied for classical waypoint navigation which requires acceleration and deceleration. Thus, improvements in aerodynamics and flight-control open entirely new doors for efficient UAV-supported search and rescue missions.