A Hierarchical Heuristic Architecture for Unmanned Aerial Vehicle Coverage Search with Optical Camera in Curve-Shape Area

Abstract

This paper focuses on the problem of dynamic target search in a curve-shaped area by an unmanned aerial vehicle (UAV) with an optical camera. Our objective is to generate an optimal path for UAVs to obtain the maximum detection reward by a camera in the shortest possible time, while satisfying the constraints of maneuverability and obstacle avoidance. First, based on prior qualitative information, the original target probability map for the curve-shaped area is modeled by Parzen windows with 1-dimensional Gaussian kernels, and then several high-value curve segments are extracted by density-based spatial clustering of applications with noise (DBSCAN). Then, given an example that a target floats down river at a speed conforming to beta distribution, the downstream boundary of each curve segment in the future time is expanded and predicted by the mean speed. The rolling self-organizing map (RSOM) neural network is utilized to determine the coverage sequence of curve segments dynamically. On this basis, the whole path of UAVs is a successive combination of the coverage paths and the transferring paths, which are planned by the Dubins method with modified guidance vector field (MGVF) for obstacle avoidance and communication connectivity. Finally, the good performance of our method is verified on a real river map through simulation. Compared with the full sweeping method, our method can improve the efficiency by approximately 31.5%. The feasibility is also verified through a real experiment, where our method can improve the efficiency by approximately 16.3%.

1. Introduction

Over the past few decades, unmanned aerial vehicles (UAVs) have been increasingly used in many applications, including mapping [1], surveillance [2], and tracking [3]. Compared with ground vehicles or surface vehicles, UAVs have better maneuverability, wider field of view (FOV), and less restriction by terrain [4], and thus they are very suitable for target search missions. This paper particularly focuses on using UAVs to search a dynamic target located in a curve-shaped area, e.g., river, coastline, road, etc. Assuming that a person falls into a flowing river, their probability of survival will decrease rapidly over time. In addition, because the person flows downstream by the water current, the dynamically changing information of the person will make the search and rescue mission more difficult. Therefore, UAVs need to cover and detect the area of interest as quickly as possible. The dynamic target search problem here can be formulated as optimizing the searching path of UAVs, which aims to obtain the maximum detection reward in the shortest possible time under the constraints of UAV maneuverability and obstacle avoidance. The work in [5] proposes two methods, namely Equivalent Area Iteration (EAI) and Probability Estimation Diffusion (PED), for situations where environmental information is relatively accurate with certain errors, respectively. However, there is no specific quantification of the relatively accurate range of environmental information. It should be noted that, although the target search problem also needs to plan the UAV path, it is quite different from the traditional path planning problem between the starting point and destination for autonomous navigation. In [6], a new DAALO method based on natural-inspired ALO is proposed to solve the problem of UAV route planning. The DAALO method can plan flight paths faster and smoother, but it is only applicable to single aircrafts and has not been extended to multi-UAVs.

Researchers have already presented various methods for searching targets located in a 2D wide area. For static target search, the work in [7] proposes a two-step search and investigation algorithm for multi-UAV systems for searching forest fires. For dynamic target search, the work in [8] proposes a search improvement algorithm based on the theory of dynamical system traversal for searching drift areas on the ocean surface. Although the proposed method improves the search efficiency by an order of magnitude, this improvement is achieved by abandoning the optimal utilization of fuel and resources. Without any priori information, UAVs can use a geometry method or a completely random way to perform search missions. In [9], the researchers propose a progressively spiral-out-based method for UAV searching for ground moving targets, but the method itself requires a sensor detection probability of 100%. In [10], an online distributed algorithm is proposed for UAV search of targets, taking into account energy loss, but the accuracy of the algorithm will decrease as the search area expands. The work in [11] proposes a lost target search algorithm based on recursive regional clustering and target trajectory prediction in marine environments. However, the efficiency of these patterns will be low if some prior information is already known in a complex search problem. In this case, graph-based heuristic methods (such as biologically inspired swarm intelligence [12], the greedy method [13,14], receding-horizon approach, coverage control method [15], mixed-integer linear programming [16], and reinforcement learning) are widely used, relying on various graphs (such as a target probability map, information density map, pheromone map). For example, based on the target probability map of the search area, our previous work [14] models the Glasius bio-inspired neural network to solve the target search problem and introduces the Gaussian mixture model (GMM)-based global guidance mechanism to improve the searching efficiency. The different flight heights of UAVs can also affect the resolution of camera sensors, so the work in [17] proposes a method for determining the optimal height of UAVs by considering their power consumption, high-value areas, and image resolution. After determining the altitude, the motion trajectory of the UAV only considers a 2D wide space.

As mentioned above, the existing searching methods mainly focus on target search mission in a 2D wide space, where UAVs can cover the entire region completely or detect the area of interest hierarchically according to prior information. This paper, however, emphasizes a dynamic target located in a curve-shaped area, which can be modeled as abstract curves subjected to geographical constraints resulting in the limitation of the UAV’s coverage motion. Hence, the existing 2D searching methods cannot be utilized for this mission directly [18]. In fact, this mission is similar to the road-network search path planning problem to some extent [19], but the existing methods mainly guide UAVs to cover the areas of interest without any quantified heuristic architecture [20,21]. Inspired by the hierarchical heuristic architecture for target searches in a 2D wide area, our previous work has achieved the coverage search of stationary targets in a river by extracting and sorting some high-value curve segments [20]. Although the scenario is simple, this proposed scheme has shown its huge potential and high efficiency in searching targets located in a curve-shaped area.

Based on the above analysis, a three-layer heuristic framework is used to solve the dynamic target search problem in a curve-shaped area. First, based on prior information, the curve-shaped area should be modeled and several high-value curve segments can be extracted. Second, the optimal visiting sequence of segments for UAVs should be determined. Third, the whole path of the UAV is planned by combining transferring paths and coverage paths. For the first step, the traditional distance-based clustering methods, such as GMM or K-means [21], require some knowledge in advance of the number of clusters and can only handle regularly shaped clusters. Density-based spatial clustering of applications with noise (DBSCAN) is a density-based clustering method widely used in 2D clustering problems, and it does not require any settings of the number of clusters and can discover arbitrarily shaped clusters [22]. For the second step, it leads to a variant of the well-known traveling salesman problem [23], and many methods have been proposed for this, e.g., approximation insertion, generalized solution [24], self-organizing map (SOM) neural network [25]. But, these methods may not be suitable for the target search problem in a dynamically changing area. For the third step, the constraints of UAV maneuverability and obstacle avoidance need to be considered at the same time in a complex scenario.

There has been a lot of research on the autonomous control of UAVs in regular cases [26,27]. This paper aims to solve the problem of dynamic target searching in a curve-shaped area, which can be modeled as a path planning problem for UAVs to obtain the maximum detection reward. The main contributions are as follows.

(1)

Based on the original target probability map generated by Parzen windows with 1D Gaussian kernels, several high-value curve segments can be extracted by DBSCAN.

(2)

Considering the dynamic characteristics of drifting targets, the boundary of each curve segment is predicted, and the coverage sequence of curve segments is dynamically determined by the rolling SOM (RSOM) neural network.

(3)

The whole path of UAVs is a successive combination of coverage paths and transferring paths planned by the Dubins method with modified guidance vector field (MGVF) satisfying the constraints of maneuverability and obstacle avoidance.

The paper is organized as follows. Section 2 gives the problem formulation. Section 3 proposes the three-layer hierarchical heuristic architecture. Section 4 shows the experimental results. Section 5 draws the conclusion.

2. Problem Formulation

This paper assumes that an UAV equipped with a gimbaled camera performs the search task and takes a water falling event as an example that a person falls into a river and waits to be rescued. The real position of the person is undetermined. As time passes, the area of interest will gradually expand since the person may be carried downstream by the current, while the person’s probability of survival will decrease rapidly. Therefore, the UAV must detect the area of interest as quickly as possible to find the person.

2.1. Simplified UAV Model

Suppose that the UAV flies at a fixed altitude, and its kinematic model can be described by

$$\left\{\begin{array}{l}\dot{x}=v\cos \phi \\ \dot{y}=v\sin \phi \\ \dot{\phi }=\omega \end{array}\right.$$

(1)

where ${(x,y)}^{\mathrm{T}}$ is the inertial position, $v$ the speed, $\phi$ the heading angle, and $\omega$ the turning rate with constraint $\left|\omega \right|\le {\omega }_{\max }$. In order to ensure the confidence of target detection and decrease the cruising time, the flying speed when detecting the target (i.e., the UAV is directly covering the curve-shaped area) is $v_{\mathrm{cov}}=v_{\min }$, and in other cases it is the maximum cruising speed $v_{\mathrm{cru}}=v_{\max }$. Here, the change between $v_{\mathrm{cov}}$ and $v_{\mathrm{cru}}$ is ignored. In addition, this paper assumes that the UAV’s maneuverability can satisfy the geographical curvature of the curve-shaped area, and thus only the maneuverability during cruising is considered with the minimum turning radius $R_{\min }=v_{\mathrm{cru}}/{\omega }_{\max }$.

2.2. Environment Model

This paper takes a portion of the Weihe river in Weinan City, Shaanxi Province, China, as an example of a curve-shaped area, as shown in Figure 1. Usually the detection range of the camera on a UAV is larger than the width of river, so the river can be modeled as a curve with the appropriate resolution. From Figure 1b, the whole curve can be divided into $M=L/L_s$ cells uniformly with respect to the forward distance $s$, where $L$ is the total length of the curve and $L_s$ is the cell length. Thus, the 2D position $x_m={\left(x_m,y_m\right)}^{\mathrm{T}}$ of cell $m$ can be reduced to 1D expression by $s_m=m\cdot L_s$. This paper utilizes the target probability map as the basic information for target searching, which will be described in Section 3. Each cell has a probability $p\left(x_m\right)\in \left[0,1\right]$ determined by prior information and it represents the detection payoff of this cell. Additionally, if only one target exists, the volume of the whole probability map is equal to 1:

$${\displaystyle \sum _{m=1}^{M}p\left(x_m\right)=1}$$

(2)

We define $z_m^{1:t}=\left\{z_m^1,\dots z_m^t\right\}$ as the historical detection events of cell $m$ at time $t$. Using Bayes’ rule, the target probability $p\left(x_m\left|z_m^{1:t}\right.\right)$ after detection can be updated by

$$p\left(x_m\left|z_m^{1:t}\right.\right)=\lambda \cdot p\left(x_m\left|z_m^{1:t-1}\right.\right)\cdot \left(1-g(z_m^t|x_m)\right)$$

(3)

where $p\left(x_m\left|z_m^{1:t-1}\right.\right)$ at the initial time $t=1$ is equal to the prior value of $p\left(x_m\right)$, $\lambda =1/{\displaystyle {\sum }_{m=1}^M\left(p\left(x_m\left|z_m^{1:t-1}\right.\right)\cdot \left(1-g(z_m^t|x_m)\right)\right)}$ is a normalization parameter, and $g(z_m^t|x_m)$ is the detection confidence related to the observation ability of the camera. This paper assumes the sensor performance to be perfect, i.e., when the UAV completes the coverage of cell $m$ at time $t$, a new observation occurs with $g(z_m^t|x_m)=1$ and $z_m^t=1$, and otherwise $g(z_m^t|x_m)=0$ with $z_m^t=0$.

In addition, since the target drifts downstream by the water current in the river, the prior target probability map will be changed as well. This paper uses the beta distribution to describe the probability of drifting speed:

$$f\left(v_{\mathrm{t}}\right)=\frac{c}{v_{\mathrm{tmax}}}{\left(\frac{v_{\mathrm{t}}}{v_{\mathrm{tmax}}}\right)}^{a-1}{\left(1-\frac{v_{\mathrm{t}}}{v_{\mathrm{tmax}}}\right)}^{b-1}$$

(4)

where $v_{\mathrm{t}}\in \left[0,v_{\mathrm{tmax}}\right]$ denotes the drifting speed of the target, and $a,b,c=\left(a+b-1\right)!/\left(\left(a-1\right)!\left(b-1\right)!\right)$ are parameters of beta distribution. Based on the Chapman–Kolmogorov equation, the target probability can further be predicted by

$$p\left(x_m\left|z_m^{1:t}\right.\right)={\displaystyle \sum _{l=1}^M\left(p\left(x_l\left|z_l^{1:t}\right.\right)\cdot {\overline{f}}_l\left(v_{\mathrm{t}}\right)\right)}$$

(5)

where the value of $v_{\mathrm{t}}$ is determined by the transition from any cell $l$ to the current cell $m$, and ${\overline{f}}_l\left(v_{\mathrm{t}}\right)$ is an integration of $f\left(v_{\mathrm{t}}\right)$ in the relevant cell. This prediction model has good convergence and low complexity.

2.3. Problem Description

The single conditional probability of non-detection for UAVs can be obtained by the volume of multiplying the target probability and the sensor non-detection confidence:

$${\overline{D}}^t={\displaystyle \sum _{m=1}^M\left(p(x_m|z_m^{1:t-1})\cdot \left(1-g(z_m^t|x_m)\right)\right)}$$

(6)

Thus, the probability that the target has been found so far, i.e., the cumulative detection reward, is given by

$$D^{1:t}=1-{\displaystyle \prod _{i=1}^t{\overline{D}}^i}$$

(7)

It should be noted that ${\overline{D}}^t<1$ and $D^{1:t}\to 1$ with $t\to \infty$. Next, we define $\zeta$ as a candidate path. Hence, the problem of dynamic target search can be formulated as finding the optimal path ${\zeta }^{\ast }$ attached with the maximum searching efficiency (i.e., the maximum detection reward in the shortest possible time) under the constraints of UAV maneuverability and obstacle avoidance:

$$\begin{array}{l}{\zeta }^{\ast }=\arg \underset{\zeta }{\max }\left(D^{1:t}(\zeta )/t\right),\\ \mathrm{s}.\mathrm{t}.\begin{array}{c}\zeta \cap S_F=\varnothing \end{array},R\ge R_{\min }\end{array}$$

(8)

where $S_F$ denotes the obstacle region, and $R$ is the turning radius of the UAV.

3. Three-Layer Hierarchical Heuristic Architecture

This paper utilizes a three-layer heuristic architecture to solve the dynamic target search problem in a curve-shaped area, as shown in Figure 2. First, the original target probability map for the curve-shaped area is generated by the Parzen windows method, and on this basis, some high-value curve segments are extracted by DBSCAN preliminarily. Second, given an example of a drifting target along the river, the downstream boundaries of curve segments in the future time can be updated and predicted by the mean speed of beta distribution, and the coverage sequence of these high-value curve segments is determined by RSOM dynamically. Third, the transferring path between curve segments (or the transferring path between the starting point and curve segment) is planned by the Dubins method with MGVF, and thus the whole path is obtained by connecting the transferring paths and the coverage paths (i.e., the curve segments) successively.

3.1. Preliminary Processing of Curve-Shape Area

3.1.1. Generation of Original Target Probability Map

In most cases, there usually exists prior qualitative information about the target to be searched, which may be obtained from historical incidents, reports from commons, inference from experts, etc. Suppose there are J discrete data and each one $Q=\left\{\left(s_j,{\gamma }_j,{\lambda }_j\right)\right\},j\in \left\{1,\dots J\right\}$ has three attributes, i.e., the position $s_j$, the reliability ${\gamma }_j$, and the accuracy ${\lambda }_j$. As seen in Section 2.2, the position $s_j$ can be reduced to a 1D expression with respect to forward distance. The reliability ${\gamma }_j\in \left[0,1\right]$ denotes the confidence of this prior information. The inverse of accuracy ${\lambda }_j$ denotes the possible region around $s_j$. Thus, a standard 1D Gaussian function $G\left(s_j,1/{\lambda }_j\right)$ with reliability ${\gamma }_j$ can be used to describe this prior information.

Given that the prior information matches the Gaussian distribution, it is appropriate to utilize Parzen windows with Gaussian kernels to build the original target probability map. Because the amount of the prior qualitative data is usually insufficient, each item should first be expanded to $E_j=E_{\max }{\gamma }_j$ samples from its corresponding Gaussian function. Then, the original target probability map can be generated by summing the Gaussian kernels of all the samples:

$$p\left(s\right)=\frac{1}{E}{\displaystyle \sum _{e=1}^E{G}_e}\left(s_e,\sigma \right)$$

(9)

where $E={\displaystyle {\sum }_{j=1}^J{E}_j}$ denotes the number of samples after expansion, $s_e$ the position of each sample, $\sigma$ the standard deviation for each Gaussian kernel calculated by

$$\sigma =0.5E^{-1/5}\min \left({\sigma }_{\mathrm{std}},IQR/1.34\right)$$

(10)

where $IQR$ denotes the difference between the 75th and 25th percentile of samples, and ${\sigma }_{\mathrm{std}}$ the standard deviation of samples. In addition, the continuous form of (9) should be discretized into $p\left(x_m\right)$ for ease of processing.

3.1.2. Extraction of High-Value Curve Segments by DBSCAN

DBSCAN is a density-based clustering algorithm [24] that is suitable for extracting high-value curve segments from the original target probability map. First, we choose N points ($N\ge M$) to form the training samples. To reflect the prior information objectively, the number of points placed in the center of each cell is

$$N_m=\left[p\left(x_m\right)\cdot N\right]$$

(11)

where the symbol $[ ]$ denotes the rounding operation. Additionally, the number of points in some cells needs to be fine-tuned to satisfy ${\displaystyle {\sum }_{m=1}^M{N}_m=N}$.

In DBSCAN, we define $R_{\eta }$ as the neighborhood radius for each cell in the 1D space, which is an integer multiple of $L_s$. We define $minPts$ as the minimum number of points in the neighborhood. The algorithm works as follows. First, an unmarked cell $m$ from $M$ cells is selected randomly. If there exists at least $minPts$ points in the neighborhood of cell $m$, this cell is marked as a core one. Then, all the cells in the neighborhood of cell $m$ are added to the same high-value segment. These steps are repeated until all the cells have been visited. Those cells which are not core cells but fall within the neighborhood of a core cell are considered as boundary cells, since they are not sufficient to form a segment but can be assigned to another segment. Figure 3 illustrates the principle of the DBSCAN algorithm in a 1D space, where the red core cells and the blue boundary cells form a cluster. Finally, each cluster corresponds to an extracted high-value curve segment.

3.2. Coverage Sequence of Curve Segments

3.2.1. Prediction of Downstream Boundary by Beta Distribution

The geographical restriction for the target makes only the coverage motion of the UAV along the curve be valuable. Thus, the UAV should cover and detect the high-value curve segments in an optimal order to obtain the maximum detection reward in the shortest time. However, the target probability map is dynamically changing by (4) and (5), and consequently the extracted high-value curve segments are changing as well. Although the DBSCAN method could be used repeatedly to update the curve segments at each moment, it is too time consuming. Here, this paper takes an example of a drifting target at the speed conforming to beta distribution in a river and utilizes its dynamic characteristics to update and predict the boundaries of the original high-value curve segments directly.

As described in (4), although the speed of a dynamic target is uncertain at each moment, its mean speed tends to be constant:

$$v_{\mathrm{mean}}=\frac{a{v}_{\mathrm{tmax}}}{a+b}$$

(12)

Considering the dynamic characteristics of a drifting target, the upstream boundary of the curve segment is unchanged, but the downstream boundary will extend a distance of mean speed multiplied by time.

3.2.2. Sorting Curve Segments by RSOM

Unlike most neural networks, the SOM neural network only contains an input layer and output layer [28]. In this paper, each boundary point of the high-value segment denotes a neuron of the input layer. The starting point of the UAV is also an input neuron. Additionally, to ensure the region constraints, i.e., the upstream and downstream boundary points of the same segment are continuously accessed by the UAV, we add some virtual points evenly distributed in each curve segment as neurons of the input layer. The number of virtual points inserted in each segment is

$${\zeta }_k=\left[\zeta \cdot L_k/{\displaystyle {\sum }_{k=1}^K{L}_k}\right]$$

(13)

where $\zeta$ is a constant denoting the total number of virtual points, $L_k$ denotes the length of a curve segment, and $K$ is the number of curve segments. Each neuron of the input layer is denoted by its coordinates $P_k={\left(x_k,y_k\right)}^{\mathrm{T}},k\in \{1,\dots ,\zeta +2K+1\}$. The spatial relationship of these $\zeta +2K+1$ points is the pattern that the neural network needs to learn, and finally a circular topological structure of neurons $c_h,h\in \left\{1,\dots ,H\right\}$ in the output layer will be obtained. It is noticed that $H$ is usually much larger than $\zeta +2K+1$, and the initial positions of output neurons are set randomly. Because the upstream/downstream boundary points of the same curve segment can be seen as adjacent with the help of virtual points, and each output neuron $c_h$ gets close to any one input point $P_k$ gradually, the final circular topological structure with neuron connections can reflect the coverage sequence of curve segments, as well as the entry/exit point for each curve segment. Figure 4 illustrates the structure of the SOM, where each neuron of the output layer is connected with all the neurons of the input layer.

In this paper, three evaluation factors including the transferring time, the coverage time, and the reward are used in the winner selection step of the SOM framework. The shortest transferring path between the input neuron and output neuron under the constraint of UAV maneuverability can be obtained using the Dubins method, where the heading of input/output neuron is determined by the geographical restriction. Then, the transferring time can be calculated by

$$T_{P_k\to c_h}=\frac{d_{P_k\to c_h}}{v_{\mathrm{cru}}}$$

(14)

where $d_{P_k\to c_h}$ denotes the length of Dubins path. As the UAV should cover the corresponding curve segment after arriving at the boundary point, the coverage time can be calculated by

$$T_h=\frac{L_h}{v_{\mathrm{cov}}}$$

(15)

and $L_h$ denotes the length of the curve segment which $c_h$ is closest to. And the reward of the curve segment is the sum of the target probability of cells contained in this segment:

$$R_h={\displaystyle {\sum }_{x_m\in S_h}p\left(x_m\right)}$$

(16)

where $S_h$ denotes the curve segment. Therefore, for any input neuron $P_k$, the winner neuron of the output layer can be selected by

$$c_{\mathrm{win}}=\arg \underset{h\in \left\{1,\dots ,H\right\}}{\max }\frac{R_h}{T_{P_k\to c_h}+T_h}$$

(17)

Then, the neighbor neurons of $c_{\mathrm{win}}$ can be updated by

$$c_h^{\prime }=c_h+{\delta }^t\cdot \exp \left(-d_h^2/{\sigma }_c^{t2}\right)\cdot \left(c_{\mathrm{win}}-c_h\right)$$

(18)

where $d_h$ is the distance between the winner neuron and the neighbor neuron, ${\delta }^t$ is a learning factor, and ${\sigma }_c^t$ is the radius of neighborhood decayed over iteration:

$$\begin{array}{l}{\delta }^t={\gamma }_{\delta }\cdot {\delta }^{t-1}\\ {\sigma }_c^t={\gamma }_{{\sigma }_{\mathrm{c}}}\cdot {\sigma }_c^{t-1}\end{array}$$

(19)

where ${\gamma }_{\delta }$ and ${\gamma }_{{\sigma }_c}$ denote the decay factors. The above process of winner selection and weight update repeats until reaching the maximum number of iterations or the threshold of neighborhood radius.

The traditional SOM is more suitable for a static scenario assuming all the curve segments remain unchanged. However, the boundaries of curve segments in this paper will expand slowly in real time, so a rolling strategy is introduced into the SOM. The proposed RSOM algorithm consists of four main steps. First, the boundary points of the remaining curve segments are updated and predicted. Then, for the remaining curve segments, we generate some virtual points according to the length of segment. Then, the optimal coverage sequence for these segments is planned by the iterative process of winner selection and weight update in the SOM. Finally, the UAV only covers the first one of these sequenced segments and updates the current state. The above steps repeat until all the curve segments are covered.

3.3. UAV Path Planning

3.3.1. UAV Path in Free Environment

Based on the sequenced curve segments, the whole path of the UAV is actually a successive combination of transferring paths and coverage paths. The curve segment can be used as the coverage path directly. The Dubins path can be taken as the transferring path between the starting point and the first curve segment, or that between two curve segments (i.e., the boundary points with specified heading angle under geographical restriction). Figure 5 illustrates a scenario including three high-value curve segments $S_1$, $S_2$ and $S_3$. Obviously, the transferring paths (red) and the coverage paths (blue) can combine the whole path $c=\{S_{0\to 1},S_1,S_{1\to 2},S_2,S_{2\to 3},S_3\}$ in sequence.

3.3.2. UAV Path in Obstacle Environment

In some complicated cases, there may be some obstacles that bring risk for the UAV. When obstacles exist on the transferring path, for example, the UAV cannot avoid them along the original Dubins path. Thus, this paper utilizes the MGVF method to achieve the Dubins path tracking and obstacle avoidance simultaneously, assuming that the UAV is equipped with lidar sensors. First, to achieve Dubins path tracking in a free environment, the original velocity of the UAV can be obtained by

$$v=v_{\mathrm{cru}}\cdot \frac{\left(d_D/d_0\cdot \delta +\tau \right)}{\left|d_D/d_0\cdot \delta +\tau \right|}$$

(20)

where $d_D$ denotes the vertical distance between the current UAV position and the Dubins path, $d_0$ is a constant, $\delta ={\left(-\partial \zeta /\partial x,\partial \zeta /\partial y\right)}^{\mathrm{T}}$ is the normal vector pointing towards the path, and $\tau ={\left(-\partial \zeta /\partial y,\partial \zeta /\partial x\right)}^{\mathrm{T}}$ is the tangent vector along the path.

Then, the original velocity needs to be adjusted when the UAV detects some obstacles. Considering the distance and orientation of the point cloud from lidar, the threat level for each point on the obstacle is calculated by

$$\mathsf{\Gamma }=\frac{1+\cos \theta }{2d_{\mathrm{obs}}}$$

(21)

where $d_{\mathrm{obs}}$ denotes the distance between the UAV’s position and the lidar’s sampling point, and $\theta$ is the angle between the UAV’s heading and the lidar’s ray direction. Then, we select the obstacle point $P_{\mathrm{o}}={\left(x_{\mathrm{o}},y_{\mathrm{o}}\right)}^{\mathrm{T}}$ with the highest threat level to define the normal vector $\mathit{n}$ and tangent vector $\mathit{t}$ by

$$\mathit{n}=\frac{1}{\sqrt{{\left(x_{\mathrm{o}}-x\right)}^2+{\left(y_{\mathrm{o}}-y\right)}^2}}\left[\begin{array}{l}{x}_{\mathrm{o}}-x\\ y_o-y\end{array}\right]$$

(22)

$$\mathit{t}=\frac{1}{\sqrt{{\left(x_{\mathrm{o}}-x\right)}^2+{\left(y_{\mathrm{o}}-y\right)}^2}}\left[\begin{array}{l}y-y_{\mathrm{o}}\\ x_{\mathrm{o}}-x\end{array}\right]$$

(23)

Thus, the correction matrix is defined as

$$\overline{\mathit{M}}=\mathit{I}-\frac{\mathit{n}{\mathit{n}}^{\mathrm{T}}}{{\left(e^{d_{\mathrm{obs}}}\right)}^{\frac{1}{\rho }}{\left|\mathit{n}\right|}^2}+\frac{\mathit{t}{\mathit{n}}^{\mathrm{T}}}{{\left(e^{d_{\mathrm{obs}}}\right)}^{\frac{1}{\epsilon }}\left|\mathit{t}\right|\left|\mathit{n}\right|}$$

(24)

where $I$ denotes the unit matrix, and $\rho$ and $\epsilon$ are constants. After that, the desired velocity of the UAV can be corrected by

$$\overline{\mathit{v}}=v_{\mathrm{cru}}\frac{\overline{\mathit{M}}\mathit{v}}{\left|\overline{\mathit{M}}\mathit{v}\right|}$$

(25)

Using the MGVF method, both the stability of the Dubins path tracking and the safety of obstacle avoidance can be ensured.

3.4. Constraint of UAV Communication Network

The above solution framework is also suitable for multi-UAVs, once the subregions are allocated to corresponding UAVs [29]. In this case, the connectivity of the multi-UAV communication network should be ensured to achieve the continuous information transmission. Here, we quantify the communication connectivity between UAVs by the successful transmission probability. First, we suppose that UAV i sends a message to UAV j under the sending power $P_i$ and the transmitter output noise with average power ${\sigma }_j^2$, so the received signal-to-noise ratio (SNR) of the signal can be defined as

$${\mathsf{\Gamma }}_{ij}=\frac{P_i{G}_{ij}}{{\sigma }_j^2}$$

(26)

The channel gain is

$$G_{ij}=\frac{C_{ij}{\left|h_{ij}\right|}^2}{D_{ij}^{\alpha }}$$

(27)

where $C_{ij}$ is related to the antenna gain and shading, $h_{ij}$ is a factor of multipath decay, $D_{ij}$ is the distance between two UAVs, and $\alpha$ is the propagation loss factor. To ensure successful information transmission with acceptable packet loss, the SNR should be higher than the minimum level of link quality $\gamma$. Thus, if $h_{ij}$ conforms to a standard Gaussian distribution, the probability of successful transmission can be expressed as

$$P_r^{ij}(\mathsf{\Gamma }\ge \gamma )=\exp \left(-\frac{{\sigma }_j^2\gamma D_{ij}^{\alpha }}{C_{ij}{P}_i}\right)$$

(28)

If $P_r^{ij}$ is larger than a certain threshold ${\delta }_P$, the two UAVs are connected. The above constraint should always be satisfied in order to form a communication network with strong connectivity. Therefore, if the communication constraint is not satisfied, one UAV should fly straight to the other one, i.e., the original velocity of the UAV in (20) should be redefined:

$$v=\left\{\begin{array}{cc}{v}_{\mathrm{cru}}\cdot \frac{\left(d_D/d_0\cdot \delta +\tau \right)}{\left|d_D/d_0\cdot \delta +\tau \right|},& P_r^{ij}>{\delta }_P\\ v_{\mathrm{cru}}\cdot \frac{\left(d_D/d_0\cdot \delta +\tau +\lambda ({\delta }_P-P_r^{ij})\cdot \eta \right)}{\left|d_D/d_0\cdot \delta +\tau +\lambda ({\delta }_P-P_r^{ij})\cdot \eta \right|},& P_r^{ij}\le {\delta }_P\end{array}\right.$$

(29)

where $\eta$ denotes the normal vector of the UAV pointing towards the other one, and $\lambda$ is a constant. It should be noted that even if only a single UAV is involved for a target search, we still need to consider the constraint of the communication network between the UAV and base station.

4. Experiment

The effectiveness of the proposed method is verified via simulation in the river map of Figure 1 by MATLAB R2022a on a computer with 64-b Intel Core i7-11800H CPU with 2.3 GHz frequency. Unless otherwise specified, the parameter values are as follows: cell length $L_s=10 \mathrm{m}$, minimum turning radius of UAV $R_{\min }=150 \mathrm{m}$, cruising speed of UAV $v_{\mathrm{cru}}=80 \mathrm{m}/\mathrm{s}$, detecting speed of UAV $v_{\mathrm{cov}}=20 \mathrm{m}/\mathrm{s}$, maximum speed of target $v_{\mathrm{tmax}}=3 \mathrm{m}/\mathrm{s}$, $a=3$, $b=2$, maximum number of expanded samples $E_{\max }=400$, number of training samples in DBCSAN $N=5000$, neighborhood radius in DBSCAN $R_{\eta }=20 \mathrm{m}$, and minimum number of neighbor points in DBSCAN $minPts=10$, and the parameters of SOM and MGVF can be chosen in a relatively broad range. Additionally, a simple real-world flight experiment is conducted on a quadrotor UAV to verify the feasibility of our method.

4.1. Coverage Search by DBSCAN-RSOM

Figure 6 gives the original target probability map of a dynamic target in the Weihe river generated by the Parzen windows method based on some prior qualitative information about the target. Obviously, there are several high-value clusters conforming to Gaussian distribution in this map. On this basis, six high-value segments can be extracted by DBSCAN accurately, as shown in Figure 7. Additionally, the expanded downstream boundary of each high-value segment at $t=1500 \mathrm{s}$ is shown in Figure 8, considering the beta distribution-based speed of the target drifting along the river (from left to right). It is obvious that the dynamically changing scenario cannot be ignored, since the length of each expansion (green) will be long. We suppose that the UAV starts from 3000 m, 0 m with a heading angle $90°$ and performs the coverage search task. Figure 9 illustrates the planned path by RSOM and MGVF, where the green areas denote the obstacles, the blue circles denote the original upstream/downstream boundary points of segments, and the yellow circles mean the expanded downstream boundary points. The RSOM method can determine the optimal coverage sequence (3,2,1,4,5,6) and the corresponding entry point (1,1,0,1,0,0, where 0 and 1 denote upstream/downstream boundary point, respectively) of these dynamically changing segments. Additionally, the transferring path by MGVF can avoid both regular and irregular obstacles.

4.2. Comparison of Different Methods

To verify the advantages of the DBSCAN-RSOM, some other methods are utilized as well for comparison. It should be noticed that the obstacle avoidance is not considered when using these methods. The first is the full sweeping method, i.e., the UAV flies along the river and covers the whole river after a long time, as shown in Figure 10a. The second is the GMM-RSOM, i.e., the architecture is similar to our method but the high-value curve segments are extracted by the GMM. However, the clustering result by the GMM is heavily dependent on the given number of clusters. Suppose that the preset number of clusters is 4, for example, it will result in some low-probability areas being included in the extracted segments. As shown in Figure 10b, the UAV will waste some time on detecting these low-value areas. The third is the DBCSAN-DW, where the dynamic window (DW) strategy is utilized to sort the segments extracted by the DBCSAN. The principle of the DW is similar to the well-known model predictive control method, and the optimal sequence for the next three segments is computed but only the first one is covered by the UAV in a rolling way. For the DBCSAN-SOM, the coverage sequence of segments is only computed once by the traditional SOM method, where the dynamic characteristic of the target is ignored. The planned paths using different methods can be seen in Figure 10. In addition, Figure 11 shows the curve of cumulative detection reward of each method. The RSOM is a global optimization strategy and, inversely, the DW emphasizes a local myopia, so the reward of the DBCSAN-RSOM is less than the DBCSAN-DW in the early stage but the DBCSAN-RSOM will complete the search task in the shortest time. The performance of each method is also provided in

Table 1. Performance of different methods.

Method	Final Reward	Flight Time (s)	Computation Time (s)
Full sweeping	1	923	0.91
GMM-RSOM	0.99	671	2.33
DBSCAN-DW	0.99	694	2.28
DBSCAN-SOM	0.99	674	3.38
DBSCAN-RSOM	0.99	620	4.03

. Obviously, the hierarchical heuristic architecture (i.e., the latter four methods) has a much higher efficiency (less flight time) than the full sweeping method, which can improve the efficiency by approximately 31.5%. And the DBCSAN-RSOM can obtain almost all the detection rewards (0.99) in the shortest time (620 s). Furthermore, although the computation time of our method is longer than others, it is still acceptable for the time-sensitive mission in this paper.

We also ran 20 tests in various random scenarios to verify the robustness of our method. In each test, 10~30 discrete qualitative data with a random position, reliability, and accuracy were set. Because the final reward of each method was almost the same, we only took the flight time as the comparison index. As the flight time by the full sweeping method was much larger than others, Figure 12 mainly displays the statistical results of four other hierarchical methods. Overall, the flight time by our method is the shortest in most cases, showing its high search efficiency in various scenarios.

4.3. Real-World Flight Experiment

To verify the feasibility of the DBSCAN-RSOM, we utilize a quadrotor UAV to perform a simple real-world flight experiment on a $80 \mathrm{m}\times 60 \mathrm{m}$ field, where a virtual target is assumed to exist in the curve-shaped road and a series of prior data points are set artificially. With a wheelbase of 450 mm, the quadcopter is equipped with a Raspberry Pi 4B on-board computer, monocular and binocular cameras, GPS and IMU, etc., as shown in Figure 13. Three high-value curve segments with true coordinates of boundary points are extracted by the DBSCAN. Assuming that the virtual target moves at 0.1 m/s, the cruising speed and the detecting speed of the quadrotor is 1 m/s and 0.5 m/s, respectively, and the path of quadrotor is then planned by the RSOM. Figure 14 shows the real flight path, and obviously the quadcopter completes the coverage of high-value segments. As seen in

Table 2. Performance of different methods in real-world flight experiment.

Method	Final Reward	Flight Time (s)
Full Sweeping	1	92
DBSCAN-DW	0.99	85
DBSCAN-RSOM	0.99	77

, our method can improve the efficiency by approximately 16.3%, compared with the full sweeping method.

5. Conclusions

The dynamic target search mission in a curve-shaped area is formulated as an optimal UAV path planning problem, which aims to obtain the maximum detection reward in the shortest flight time under various constraints. To improve the mission efficiency, this paper proposes a three-layer hierarchical heuristic architecture, including the extraction of high-value curve segments by the DBSCAN, the sorting of high-value curve segments by the RSOM, and path planning by Dubins with MGVF. The effectiveness, robustness, and feasibility of our method is verified through simulations and a real flight experiment. Compared with the full sweeping method, our method can improve the efficiency by approximately 31.5% in simulations and 16.3% in real experiments, respectively. The proposed hierarchical heuristic architecture is not limited to curve-shaped areas, and it can be generalized to 2D or even 3D wide cases (e.g., sea surface, farmland, hilly region, etc.).