Emergent Search of UAV Swarm Guided by the Target Probability Map

Abstract

In the cooperative searching scenario, most traditional methods are based on the top–down mechanisms. These mechanisms are usually offline and centralized. The characteristics limit the adaptability of unmanned aerial vehicle (UAV) swarm to the complex mission environments, such as those with inaccurate information of the targets and grids. In order to improve the searching ability of UAV swarm, a novel searching method named emergent search of UAV swarm guided by the target probability map (ESUSTPM) is proposed. ESUSTPM is based on local rules to organize and guide UAV agents to achieve the flocking state, search the mission area and detect the hidden targets concurrently. In ESUSTPM, local rules contain the flocking rules and the guiding rules. The flocking rules are the interactions between the agents, which are designed by a novel constructed function based on two exponential functions in this paper. The new constructed function can better maintain the relatively stable distances between the agents and realize the smooth transition of the positions at the given centers. The local guiding rules based on the target probability information of the nearby grids are firstly designed to realize the multi-function of the swarm, including full area coverage, target detection and reduction in environmental uncertainty (EU). Finally, the simulations verify that ESUSTPM can achieve the full coverage of the mission area while taking into account the target search. The statistical results also indicate that the searching efficiency of the proposed ESUSTPM is higher than the traditional searching algorithms based on the division and allocation of the area or the heuristic algorithms.

1. Introduction

In recent years, unmanned aerial vehicle (UAV) swarm has been increasingly used in military and civilian environments to perform some typical tasks, including traffic monitoring, disease surveillance, target tracking, area search and so on. Cooperative search is an important task of UAV swarm, and the improvement of the cooperative reconnaissance efficiency and intelligence is a research hotspot at home and abroad [1,2,3,4].

Many scholars have conducted extensive research. In traditional cooperative searching algorithms, region segmentation and assignment are an important strategy. Considering that UAVs often fly into the region from a unilateral side, unilateral region segmentation for the equal area method of the arbitrary search region is proposed. Each area grid is assigned to one UAV agent for scanning search [5]. The Voronoi polygon topology is usually used as a tool for region division. Based on the central Voronoi configuration, reference [6] realizes the optimal searching coverage of an unknown area and the optimal deployment of the searching cost consumption. The nonconvex environment is not conducive to UAV swarm for search execution; reference [7] proposes a transforming method to transform the cooperative searching problem in the nonconvex region into an equivalent one in the convex region.

The early classical cooperative searching algorithms focus on the division and distribution of the searched area. In the searching process, the coordination between UAV agents in different grids is weak.

The mission environment is usually complex and, due to a lack of prior information, a set of mechanisms, such as the digital pheromone map (DPM) and the target probability map (TPM) are constructed for determining the flight directions [8,9]. The DPM is adopted to solve the swarm coordination problem, and the TPM accounts for the target guidance. The two information maps are updated in the moving process of UAVs, and the coordinates of UAV agents are recalculated by taking the cognitive information as the decision function.

The cooperative searching algorithms based on the two information maps, including DPM and TPM, can avoid collision and repeated search as much as possible. However, factors such as resource consumption and path optimality are not considered in these cooperative searching strategies. Therefore, the heuristic algorithms attract the attention of many researchers. Considering that multiple targets are usually randomly distributed in the mission area, the cooperative search is regarded as the traveling salesman problem (TSP) in some literature works. In the TSP, resource consumption and path optimization are optimized during optimization iteration. In reference [10], an improved cooperative searching algorithm based on the non-dominated sorted genetic algorithm-II is put forward to solve a multi-objective optimization problem. The objective function combines the detection profit, energy consumption and flight distance criteria for obtaining the optimized searching routes of unmanned aerial vehicles (UAVs). In reference [11], an improved harmony searching algorithm, taking maximum total reconnaissance as the fitness function, is adopted to search lots of targets in a given area under the condition of resource constraints. Similarly, a novel cooperative patrol task algorithm based on the improved push forward insertion heuristic (PFIH) algorithm is presented in reference [12] to calculate the feasible patrolling routes for maximizing the patrol profit under the limited resources conditions.

UAVs often execute the cooperative search in an unknown environment with no prior information. In order to improve the searching efficiency and reduce the uncertainty of the mission area, many cooperative searching models based on the bio-heuristic algorithms, including the pigeon-inspired optimization (PIO), particle swarm optimization (PSO), bean optimization (BO), multiple ant colony optimization (MACO) and co-evolution genetic algorithm (CEGA), are built, as shown in references [13,14,15,16,17,18,19]. In these heuristic algorithms, UAVs are analogized as some particles or gene segments. The intelligent cooperative searching behaviors of UAV swarm under many constraints are generated by the interaction between the particles and the cross variation of gene segments for achieving the feasible rewarding function, such as the coverage rate, targets discovering profits, cooperation profits and so on. The designed algorithms can be adopted to obtain the optimal results for realizing the overlaying search and targets discovery of the region effectively. The reinforcement learning (RL) scheme is used to study the cooperative search of multi targets in the unknown sea areas by UAVs in reference [20]. In order to ensure searching efficiency, a reward and punishment function is built to generate a multi-UAV cooperative search path. In view of the trial-and-error mechanism of RL, this mechanism increases the computing burden. Therefore, it is not suitable for large-scale scenarios.

The cooperative searching algorithms based on the heuristic and RL mechanisms usually draw the conclusion by performing multi-objective iteration. The process often takes a long time to deal with the dynamic path planning task. With the increase in the agent numbers and the environmental complexity, the computational burden of UAV onboard the computer will increase. Therefore, a decentralized searching algorithm with real-time computation and sufficient intelligence becomes positively significant.

With the rapid development of UAVs and sensors, robotic swarms can conduct some complex mission or specific tasks by interacting with each other. In a decentralized swarm system, all agents can receive information from the other agents and the environment to decide on what to do next. Renolds et al. in reference [21] propose typical local rules of swarm behavior. It is named the Boids model, which follows three local rules: collision avoidance, velocity matching and flocking centering. The Boids model can realize the flocking state and velocity alignment.

In most collective behaviors, it is hypothesized that each agent in the group tends to align its direction of motion with those of its neighbors. Vicsek et al. in reference [22] bring the nearest neighbor principle into the Boids model. This model considers the change of the moving direction of the monomer. Therefore, the monomer can finally achieve the same moving direction, but this model cannot guarantee collision avoidance. Murakami et al. in [23] suggest that the collective polarization arises from interactions other than the explicit alignment rule. Thus, a new lattice-based computational model that is based on mutual anticipation and asynchronous updating is proposed. The simulation shows that this model can realize densely collective motion with high polarity.

In order to overcome the drawbacks of recent papers on motion control for swarm, including the use of unbounded forces for collision avoidance, lack of scalability, irregular fragmentation and collapse, Reza O.S. in reference [24] presents a theoretical and computational framework for the design and analysis of scalable flocking algorithms in a multi-dimension space in the presence or lack of obstacles. This analysis sheds light on the cooperation and emergence of collective behavior in complex organizations.

A bottom-up flocking model is suitable to improve the survivability of swarm in complex environment with obstacles. Zhu C.C. et al. in [25] propose a control strategy of self-organization obstacle avoidance based on the conformal module and obstacle avoidance module. The conformal module is used to realize the directional consistency. In terms of avoidance module, the principle of earth deflection is used to increase the rotational force effect for collision between agents and obstacles to reach the mission point.

Constrained by the agent dynamic, communication delay, perturbations or the presence of barriers, the numerous existing flocking models cannot perform a sufficiently good collective behavior for full coverage or path generation from one point to another point. In order to make swarm better complete tasks and enhance environmental adaptability, the rule-based model and evolution algorithm are combined to generate the optimized control model for UAV swarm [26,27,28,29,30]. The corresponding research results verify that this strategy can generate multi-objective optimization conclusions.

As for the area coverage of UAV swarm, Oleg A. Yakimenko et al. propose four local rules to achieve full coverage of the mission area and good communication connectivity [31,32]; then, the evolution algorithm is used to generate the optimized solution. The paper realizes the best coverage strategy, but environment uncertainty and target search are not considered.

The current research points mainly focus on the implementation of flocking rules and the single task with multiple constraints in the numerous existing flocking models. Considering the economy and efficiency, a multi-function swarm is needed. The tasks of the UAV swarm cooperative search are full area coverage, target detection and reduction in environmental uncertainty. The combination of these requirements makes the cooperative search of UAV swarm in a top–down manner a challenging problem. Conversely, the requirements are relatively easy to be realized in an emergent mode.

For improving the adaptability of UAV swarm to the environment and the various complex requirements, ESUSTPM is proposed. ESUSTPM is a novel searching method, and swarm searching behavior is shaped in a down–top manner. The down–top manner means that the high-level swarm behaviors are generated by the agent-to-agent interactions. The generation mode of the swarm searching behaviors determines that swarm shows high intelligence, high self-organization and adaptability in the emergent process. These characteristics improve the flexibility and survivability of UAV swarm in the cooperative search of the complex mission area.

As the proposed ESUSTPM is based on local rules, including the flocking and guiding rules, it enables UAV agents to realize the decentralized and online path planning. The main contributions of this study are as follows:

• A new flocking rule based on a novel constructed function, which can realize smooth transition of the UAV states at the center points, is proposed. The constructed function is Equation (9), and it contains two exponential functions shown in Equations (7) and (8). Equation (8) is a typical flocking rule that was proposed in references [20,21]. The slope of the corresponding curve is so high at the center point that it is difficult to avoid state oscillation of the UAV. Therefore, a novel adjusting function in Equation (7) is proposed to adjust the curve slope of Equation (8) at the center point. Thus, the proposed curve proposed in Equation (9) can transit smoothly at the center point. This design realizes a smooth transition of the UAV states at the center points in theory. Finally, a more stable space between agents can be maintained.
• A new guiding rule based on TPM is proposed. The guiding mechanism always guides the UAV to grids with the highest target existing probability. During access to the grids, TPM can be calculated and updated according to Equations (3)–(5). Therefore, the multi-functions of UAV swarm, including full area coverage, target detection and reduction in environmental uncertainty, can be realized.
• The fault tolerance to prior information on the TPM is realized. In the ESUSTPM, the swarm can calculate and update the target existing probability (TEP) of each grid in the process of accessing the grids. If the target does exist in the visited grid, the TEP increases. Conversely, the TEP decreases. Thus, the error information can be overcome by adopting this mechanism.

The contents of the article are organized as follows. In Section 2, the mathematical models of ESUSTPM, including the environmental model, the UAV motion model and the target existing probability map, are built. In Section 3, the local rules of ESUSTPM are introduced. In Section 4, the steps of the corresponding searching algorithm are shown. To verify the proposed method, simulations are conducted in Section 5. Finally, the summary and expectations are pointed out in Section 6.

2. Mathematical Model of Cooperative Search

2.1. Environment Model

The task region is usually designed as a large rectangle, and some targets are assumed to be randomly distributed in the mission area. In order to search and detect the targets conveniently, the task region is divided into $L_m\times L_n$ grids shown in Figure 1, and the grid set $E=\left\{\left(m,n\right)|m=1,2,\cdots ,L_m,n=1,2,\cdots ,L_n\right\}$ represents the whole mission environment.

UAV agents adopt the sensors to detect the environment to find the targets and guide the next step. The complex degree of the environment has an impact on the sensor parameters, including detection and false alarm possibility. The two parameters of the UAV sensor are set as 0.3 and 0.1, respectively.

2.2. UAV Movement Model

In order to conduct research on swarm behaviors conveniently, the following assumptions about UAV swarm are made:

• The agents of UAV swarm are assumed as the mass points with the maximum velocity.
• There is no leader in the UAV swarm. The monomers decide the next moving directions and velocities separately according to the information from the neighbors and the nearby environmental grids.
• The detection and communication range of UAV is limited.

The assumptions about the environments are as follows:

• When the monomers reach the boundaries of the mission area, the velocities are reversed.
• The monomers cannot collide with other monomers.

According to the above assumptions, the kinematic model of UAV with the maximum velocity is modeled. The UAV is controlled by a simple proportional controller. The inner-loop controller updates the objective velocity $\stackrel{\rightharpoonup }{mvel}$, and the outer-loop controller minimizes the position norm $\Vert \stackrel{\rightharpoonup }{P}-\stackrel{\rightharpoonup }{P_{sp}}\Vert$, where $\stackrel{\rightharpoonup }{P_{sp}}$ is the objective position, and $\stackrel{\rightharpoonup }{P_{}}$ is the current position of UAV agent. In the emergent searching process, the agents can communicate with their neighbors to calculate an accurate direction and velocity of each agent.

2.3. Target Information Model

In the cooperative searching process, UAV swarm usually faces complex unknown environments. Therefore, the goals of the UAV swarm are to perform the overlaying search of the whole mission region as soon as possible for the full area coverage, target detection and reduction in environmental uncertainty. Due to a lack of accurate prior information, the targets cannot be located accurately at the initial time.

Every grid $\left(m,n\right)$ in the mission area owns its target existing probability $P_{mn}$ in the TPM. The target existing probability represents the uncertainty of the region grid, and it can be updated in the cooperative searching process. The mixed two-dimensional Gaussian distribution function (MTDGDF) is suitable for describing the state uncertainty of the targets. Assuming that the initial coordination of a target is $\left(x,y\right)$, the two-dimensional Gaussian distribution of the target’s existing probability density is shown in Equation (1).

$$f\left(x,y\right)=\frac{1}{2\pi {\sigma }_1{\sigma }_2\sqrt{1-{\rho }^2}}·\exp \left(-\frac{1}{2\left(1-{\rho }^2\right)}\left\{\frac{{\left(x-{\mu }_1\right)}^2}{{\sigma }_1{}^2}-\frac{2\rho \left(x-{\mu }_1\right)\left(y-{\mu }_2\right)}{{\sigma }_1{\sigma }_2}-\frac{{\left(y-{\mu }_2\right)}^2}{{\sigma }_2{}^2}\right\}\right),$$

(1)

where ${\mu }_1$ and ${\mu }_2$ are the mean values of $x$ and $y$, respectively. ${\sigma }_1$ and ${\sigma }_2$ are the standard deviation of $x$ and $y$, respectively. $\rho$ is the covariance of $x$ and $y$, $\rho =\mathrm{cov}\left(x,y\right)$.

There may be multiple targets in the mission area, and the target existing probability density distribution of each target is different. Thus, the target existing probability density in the mission area is defined as

$$f(x,y)={\displaystyle \sum _{i=1}^N{\alpha }_i{f}_i(x,y)},$$

(2)

where ${\alpha }_i$ is the scale factor presenting the important degree of the target $i$, $f_i(x,y)$ is the probability density function of the target $i$, and $N$ is the target number.

The target existing probability in grid $\left(m,n\right)$ is $P_{mn}={\displaystyle \iint sf(·)dxdy}$, where s represents the area corresponding to the $f(·)$. Assuming that the existing probability distribution of the targets is independent in the x and y directions, thus ${\sigma }_1={\sigma }_2={\sigma }_0$, ${\sigma }_0$ is a constant.

2.4. The Updating Mechanism of the Target Existing Probability Map

The target existing probability map represents the possibility that the target exists in a specific grid, and it is the guidance of the searching behavior of the UAV agent. This probability map can be updated based on the feedback of the airborne sensors in the emergent searching process. When UAV arrives at the corresponding grid, the sensor on UAV obtains the target existing probability of the current grid and eight neighboring grids. However, the sensor can only sense whether there is a goal in the current grid. Then, the onboard computer calculates and updates the target existing probability of the current grid according to Equations (3)–(5). The updating mechanism is set as follows.

• The grid $(m,n)$ is not searched by UAV agents:

  $$P_{mn}(t)=P_{mn}(t-1),$$

  (3)
• The grid $(m,n)$ is searched by UAV, and the target exists in this grid:

  $$P_{mn}(t)=\frac{P_d{P}_{mn}(t-1)}{(P_d-P_f)P_{mn}(t-1)+P_f},$$

  (4)
• The grid $(m,n)$ is searched by UAV, and the target does not exist in this grid:

  $$P_{mn}(t)=\frac{(1-P_d)P_{mn}\left(t-1\right)}{1-P_f},$$

  (5)

  where $P_d$ and $P_f$ represent the detection probability and the false alarm probability, respectively. According to the above equations, the existing probability distribution of the targets can be adopted to guide UAV agent to move the grid with the maximum probability in each step and find the target when the probability exceeds the threshold $\theta$.

3. The Local Rules of ESUSTPM

3.1. The Local Rules

The missions of the UAV swarm are to achieve area coverage, target search and reduction in environmental uncertainty as much as possible before the deadline in the cooperative search. In addition, swarm must keep the flocking state with the given space for communication connectivity as far as possible in the process of task execution. Thus, the agents can communicate with other agents in the same communication net built by the agents within the given distance. Then, each agent adopts the three closest neighbors chosen by the distance and TPM to calculate the velocity vector of the monomer itself in the onboard computer. The local rules of ESUSTPM contain two parts: the local rules between UAVs and the local rules between UAV agents and the nearby environmental grids. The velocity vector of each UAV in the swarm can be calculated in the interaction with the neighbors and the nearby grids. The details of the local rules are listed as follows:

• Direction and distance to the first closest neighbor, $\overrightarrow{F_1}$;
• Direction and distance to the second closest neighbor, $\overrightarrow{F_2}$;
• Direction and distance to the third closest neighbor, $\overrightarrow{F_3}$;
• Direction to the nearby grid with the highest target existing probability, $\overrightarrow{F_4}$.

Figure 2 is the sketch map of the local rules in the ESUSTPM, and the black square denotes the nearby grid with the highest target existing probability around each UAV.

Generally, eight surrounding squares are distributed around each UAV in the swarm. The local rules between UAV agents in the swarm require the velocities and positions of the three closest neighbors. The information of the target existing probability in the eight grids is calculated and updated in the emergent process; then, the grids with the highest target probability are adopted as the guide of the UAV movement.

3.2. The Velocity Vector Calculation of UAV Agent

The objective velocity vector of each agent is calculated in Equation (6), and the agent keeps track of the objective velocity vector in the searching mission:

$$\stackrel{\rightharpoonup }{mvel}=\frac{1}{4}{\displaystyle \sum _{i=1}^4\frac{\stackrel{\rightharpoonup }{F_i}}{\Vert F_i\Vert }{\beta }_i},$$

(6)

where $\stackrel{\rightharpoonup }{mvel}$ is the objective velocity vector of the agent calculated in each step; ${\beta }_i$ is the scale factor.

3.3. The Parameters of the Local Rules

In order to maintain a stable communication connectivity and collision avoidance, each agent of UAV swarm should keep a given distance from the other agent as far as possible. Thus, the design of the local rules should consider the distance and direction factors concurrently. The relationship between the distance and $\stackrel{\rightharpoonup }{F}$ is reflected by the coefficient ${\beta }_i$ in Equation (9). Equation (9) demonstrates that the interaction strength scale varies with the distance between agents. ${\beta }_i$ is a constructed function based on two exponential functions shown in Equations (7) and (8).

$$g_i(d_i)=t_i\left(1-\exp (\frac{-{(d_i-c_i)}^2}{{\delta }_i{{}^{\prime }}^2})\right),$$

(7)

$$a_i(d_i)=k_i(d_i-c_i)\exp (\frac{-{(d_i-c_i)}^2}{{\delta }_i{}^2}),$$

(8)

$${\beta }_i=a_i(d_i)g_i(d_i),$$

(9)

where $a_i(d_i)$ was designed to avoid collision between agents and keep a given distance for the connectivity maintenance [20,21]. $k_i$ is the scale factor of $a_i(d_i)$, and the parameters ${\delta }_i$ allow for the adjusting range of the $a_i(d_i)$. The curve of the $a_i(d_i)$ is shown in Figure 3a. The curve slope at the center point $c_i$ is so high that the UAV agent tends to oscillate at the center point $c_i$. In order to avoid the state oscillation of the agents at $c_i$, the adjusting function $g_i(d_i)$ shown in Figure 3a is firstly designed in Equation (7) to adjust the curve slope of $a_i(d_i)$. $t_i$ and ${\delta }_i{}^{\prime }$ in $g_i(d_i)$ present the factor of the adjusting strength and the adjusting range of the $g_i(d_i)$. $c_i$ in Equations (7) and (8) is the given distance that the agents should maintain.

According to Equation (9), the curve of ${\beta }_i$ shown in Figure 3c is smooth at the center point $c_i$. This operation ensures that UAV states change smoothly at the center $c_i$, and the agents will be subject to resistance gradually when leaving the center $c_i$.

3.4. The Updating Positions in ESUSTPM

The monomer position is calculated and updated after acquiring information from the three closest neighbors and the nearby grids. The updating equations of each UAV agent are listed as Equations (10) and (11):

$$\stackrel{\rightharpoonup }{P_i}\left(t+1\right)={\stackrel{\rightharpoonup }{P}}_i\left(t\right)+\mathsf{\Delta }\stackrel{\rightharpoonup }{P_i}\left(t\right),$$

(10)

$$\mathsf{\Delta }\stackrel{\rightharpoonup }{P_i}\left(t\right)={\stackrel{\rightharpoonup }{mvel}}_i\left(t\right)·\mathsf{\Delta }t,$$

(11)

where ${\stackrel{\rightharpoonup }{mvel}}_i\left(t\right)$ is the calculated objective velocity of the monomer $i$ at time $t$. $P_i\left(t\right)$ is the position of the monomer $i$ at time $t$. $\mathsf{\Delta }{P}_i\left(t\right)$ is the displacement increment of the monomer $i$.

4. The Steps of ESUSTPM

When UAV swarm executes the ESUSTPM algorithm, the environment information, the target information and the information of the nearby neighbors determine the profit of each UAV in the next step. Then, UAV agents calculate the next moving velocities. Thus, UAV swarm can achieve full coverage of the mission area, target search and reduction in environmental uncertainty as far as possible. The block diagram of ESUSTPM is shown in Figure 4.

The steps of the proposed ESUSTPM algorithm are listed as follows:

• Build and initialize the environmental map, target existing information map, UAV coordinates, target coordinates. The target coordinates are unknown to the UAV agents at the initial time.
• Adopt Equations (6)–(9) to calculate the next velocity $\stackrel{\rightharpoonup }{mvel}$ of each UAV agent; then, the UAV agent updates self-speed and position.
• Update the target existing probability of the mission area; then, each agent performs the same operations in Equations (2) and (3) until UAV swarm covers the entire area or finds all targets.

If there is no target in the environment, UAV agents will achieve full coverage of the mission area by the TEP guidance. Conversely, UAV agents will repeatedly visit the nearby grid with the highest TEP until the targets are finally identified. After the targets are found, the agents will continue to execute the coverage of the remaining area until the environmental uncertainty falls below the given threshold.

In order to introduce the flow of the ESUSTPM algorithm guided by the target probability map clearly, the corresponding pseudocode of ESUSTPM is shown in Algorithm 1.

Algorithm 1. Emergent search guided by the TPM

//Initialization

Create and initialize environmental information map, target existing probability map $TP{M}_{10\times 10}$, target coordinates, UAV coordinates $P$ and UAV number M.

Initialize the parameters $c_i$, $k_i$, ${\sigma }_i$, $t_i$, ${\delta }_i{}^{\prime }$, time interval $\mathsf{\Delta }t$ and the simulation time cost $T$.

//Main loop

For $t$ = 1: $\mathsf{\Delta }t$:$T$

For $i$ = 1:1:M

Calculate the velocity $v_i$ of the agent $i$ according to Equation (6).

$P_{i+1}=P_i+v_i·\mathsf{\Delta }t$

m’, n’ = map(int, $P_{i+1}$) //Round the abscissa and ordinate of the monomer

if (m’, n’) equals the coordinate of the grid that the target exists in

{

Update the TPM[m’][n’] according to Equation (4)

}elseif (m’, n’) equals the coordinate of the grid that the target does not exist in

{

Update the TPM[m’][n’] according to Equation (5)

}elseif the grid cannot be visited

{

According to Equation (3), the corresponding TPM of the grid cannot be updated

}

If the TEP of a grid is more than the given value, the target is discovered.

End

If all targets are found or the environmental uncertainty is lower than the given value, the algorithm stops.

End

5. Simulation Experiment

To verify the feasibility and superiority of the proposed ESUSTPM method, three simulation scenarios are carried out in a rectangular area with a certain number of UAVs and targets.

Four searching algorithms, including the classical random searching algorithm, the mower searching algorithm, the searching algorithm based on Boids model with guidance and the searching algorithm based on Boids model without guidance, are used for comparative experiments. Three criteria, including the visited grid number, the environmental uncertainty and corresponding statistical metrics, are used as the metrics to verify the effectiveness of the ESUSTPM algorithm.

The simulation environment is written in Python, and the animation module in Matplotlib library is used to simulate the movement of UAV agents, the communication connectivity and the target existing probability. According to the proposed novel flocking rule, the agents repel each other when the distance between them is less than the safe distance. Therefore, the collision between agents can be avoided by the proposed local rule. When the agent meets the boundary, the velocity is reversed. Each agent is equipped with a radio for communication with other agents. For these experiments, the agents are assumed to have a downward-facing camera, and this setting can capture the below and eight neighboring grids for finding the objects of interest. This simulation does not emulate internal/external camera geometry and instead simply assumes that the area of interest can be divided into cells, and each agent can move freely and continuously across the entire area of interest. Each cell is smaller than the area covered by the camera at any given time. The mission area is evenly divided as $10\times 10$ grids in the simulation of this paper. A total of 10 UAVs start from the coordinates (100, 100), (100, 200), (100, 300), (100, 400), (100, 500), (200, 100), (200, 200), (200, 300), (200, 400), (200, 500) to execute the cooperative searching mission separately. The maximum velocity of UAV agents is $10 \mathrm{m}/\mathrm{s}$. UAV swarm does not know the target positions at the initial time. UAVs should find the targets and cover the area in the emergent searching process as far as possible. The initial target probability of each grid shown in Figure 5 is set as 0.5, and it indicates that the environment is unknown to UAV swarm at the initial time. The simulation interval is $10 \mathrm{ms}$. The total simulation time cycle is 200. When the UAV and the target are in the same cell at a certain time, the target is considered to be detected by the airborne sensors. Then, the onboard computer updates the target existing probability of the corresponding grid and determines the target existence of the grid by comparison with the given threshold. Figure 5 is the experimental scenario of the single UAV swarm at the initial time. The red points are UAV agents, and the black lines represent communication connectivity. When the distance between agents is less than the given value, communication connectivity is built. The blue color of the grid shown in Figure 5 represents the target existing probability of each grid, and the color degree denotes that the mission area is unknown to UAV agents.

5.1. Unknown Environment with No Target

5.1.1. Cooperative Search Based on the Proposed ESUSTPM Model

According to the ESUSTPM mechanism, the agents tend to move to the grids with the highest target existing probability. If the sensor on the UAV cannot find the objects of interest, the TEP of the grid decreases. The parameters of the ESUSTPM algorithm are listed in

Table 1. Parameters of the ESUSTPM algorithm.

Searching Method	${\mathit{k}}_{\mathit{i}}$	${\mathit{t}}_{\mathit{i}}$	${\mathit{c}}_{\mathit{i}}$	${\mathit{\delta }}_{\mathit{i}}$	${\mathit{\delta }}_{\mathit{i}}{}^{\prime }$
ESUSTPM	[−80,−80,−80,−80]	[5,5,5,5]	[120,120,120,120]	[40,40,40,40]	[10,10,10,10]

.

Figure 6 is the simulation result at the 200th time cycle, and the color degree of each grid represents the target existing probability. Compared to Figure 5, the color of most grids becomes lighter in the emergent searching process. It indicates that most grids are covered, and the environmental uncertainty decreases significantly. The experimental results verify the effectiveness of the ESUSTPM algorithm in the unknown environment with no target.

5.1.2. Cooperative Search Based on the Classical Random Searching Algorithm

In the classical random searching experiment, the directions of the UAV agents are random, and the step size is constant in each cycle. In each step, the monomers can avoid collision when the distances between the agents or the distances between the agent and boundary are less than the set safe value.

In order to simulate this searching behavior, a random direction is generated by Equation (12):

$$\stackrel{\rightharpoonup }{dir}=\frac{rnd\left(d,1\right)}{\Vert rnd\left(d,1\right)\Vert },$$

(12)

where $d$ is the dimension of the space, and it is 2 in this paper. $rnd(·)$ represents a function that produces a random number in [−1,1]. $\Vert ·\Vert$ indicates the norm.

The position of the agent $i$ is updated in Equation (13):

$${\stackrel{\rightharpoonup }{P}}_i(t+1)={\stackrel{\rightharpoonup }{P}}_i(t)+\stackrel{\rightharpoonup }{dir}·S·\mathsf{\Delta }t,$$

(13)

where $S$ is the constant step, and it equals 10 in this paper. $\mathsf{\Delta }t$ is the simulation interval.

The experimental result at the 200th time cycle is shown in Figure 7, and most grids are still blue. Compared with the proposed ESUSTPM algorithm, the cooperative searching algorithm based on the classical random algorithm cannot complete the full coverage of the mission area within 200 time cycles, and the reduction in the environmental uncertainty is also not sufficient.

5.1.3. Cooperative Search Based on the Mower Search

The mower searching algorithm is a classical searching algorithm [33], and UAV agents sweep the mission area back and forth at a certain speed. Figure 8 shows the mower searching result at the 200th time cycle. The color of each grid becomes lighter at the 200th time cycle, and this indicates that all grids have been visited, and the TEP decreases.

In order to simulate this searching behavior clearly, the position of the UAV agent $i$ is updated by the following Equation (14):

$${\stackrel{\rightharpoonup }{P}}_i(t+1)={\stackrel{\rightharpoonup }{P}}_i(t)+\stackrel{\rightharpoonup }{b}·l·\mathsf{\Delta }t,$$

(14)

where $\stackrel{\rightharpoonup }{b}$ is the unit direction. This direction will reverse when the UAV meets the boundary. $l$ is a certain step, and it equals 10 in this paper. $\mathsf{\Delta }t$ is the simulation interval.

5.1.4. Cooperative Search Based on the Boids Model without Guidance

The Boids model consists of three rules, including collision avoidance, velocity matching and flocking centering. Collision avoidance demonstrates that the monomer attempts to avoid collision with the nearby neighbors. Velocity matching means that each swarm monomer attempts to match the velocity with the nearby neighbors. The flock centering indicates that the agent attempts to stay close to the nearby neighbors. The monomers can realize a distributed synergy effect of UAV swarm by adopting the three principles to interact with each other. In the cooperative search based on the Boids model without guidance, the local rules only contain the above three rules for maintaining the flocking state and searching the grids. As the guiding rules are not used to guide the movement of UAV agents, the searching efficiency of this cooperative searching method will not be high in theory.

Figure 9 is the simulation result, which indicates that the searching algorithm based on the Boids model without guidance is executed after 200 time cycles. The given safe distance between the agents is 120 m. As the monomer cannot acquire the environmental guidance in the searching process and the interactions between agents further restrict and disturb the movement of the nearby monomers, the swarm can only partially cover the mission area.

5.1.5. Cooperative Search Based on the Boids Model with Guidance

The Boids principles and the proposed guiding mechanism are adopted to perform the full coverage mission. The searching time is 200 time cycles. The given safe distance between the agents is also 120 m. Figure 10 shows the experimental result of the searching algorithm based on the Boids model with guidance. The color of the partial grids is still blue, and this indicates that these grids are not visited within 200 time cycles. It proves that the efficiency of the searching algorithm based on the Boids model with guidance is lower than the proposed ESUSTPM algorithm in this paper.

Figure 11 shows the relationship between the grids visited number and time cycles of the five searching algorithms. The red line, purple line, green line, orange line and the indigo line represent the proposed ESUSTPM algorithm, the mower searching algorithm, the searching algorithm based on the Boids model with guidance, the searching algorithm based on the Boids model without guidance, the classical random searching algorithm, respectively. It shows that the completion time of the proposed searching algorithm is almost equal to the mower searching algorithm, but the other searching algorithms cannot achieve the area coverage search within the given time. Generally, the mower searching algorithm has the highest coverage efficiency. Therefore, the conclusion is drawn that the ESUSTPM algorithm is efficient.

Figure 12 shows the change of the environmental uncertainty derived by the five searching algorithms during 200 time cycles. The purple line represents the result of the mower searching algorithm, and the red line is the result of the proposed ESUSTPM algorithm. During the stipulated experiment time, the environmental uncertainty of the proposed ESUSTPM algorithm decreases more than with the other searching algorithms.

In all, combining the results of Figure 11 and Figure 12, the proposed method is most effective in terms of area coverage and the reduction in environmental uncertainty at the same time.

5.2. Unknown Environment with Static Target

In the unknown searching scenario with some targets, the coordinates of three targets are (400, 500), (400, 700) and (700, 500). The initial positions of UAV agents are distributed in the lower left corner. At the initial time, the UAV swarm does not know whether there is a target in the mission area. Therefore, the initial target probabilities of all grids are set as 0.5, and the color of each grid is blue in Figure 5. In ESUSTPM, the UAV agent visits the nearby grid with the highest target probability. If the airborne sensors cannot detect the targets in the visited grid, the target existing probability decreases, and the color of the grid becomes lighter. Conversely, the target existing probability increases when the target is detected by the airborne sensors in the corresponding grid. Meanwhile, the blue color of the grid becomes deeper. When the probability value in a grid is higher than the given value, the onboard computer confirms that the target exists in this grid, and the corresponding color becomes red. The simulation cycles are also 200 time cycles in this experiment.

5.2.1. Cooperative Target Search Based on the Proposed ESUSTPM Model

In target searching mission, the parameters of the local rules in ESUSTPM are the same as the parameters in Table 1. Within the 200 time cycles, the number of the red grids increases from Figure 13, Figure 14 and Figure 15, and the color of the other grids also becomes lighter, as shown in Figure 15. This experimental result proves that the proposed ESUSTPM algorithm is feasible, and ESUSTPM can not only find the targets but also cover 94% of the area.

5.2.2. Cooperative Target Search Based on the Classical Random Searching Algorithm

Figure 16 shows the result of the target search based on the classical random searching algorithm. Within 200 time cycles, the searching method based on the classical random searching algorithm cannot cover most of the grids and find a target. The result shows that this algorithm is not suitable for the cooperative target search.

5.2.3. Cooperative Target Search Based on the Mower Searching Algorithm

The cooperative search based on the mower searching algorithm is executed during the 200 time cycles. Figure 17 is the corresponding result at the final cycle. As the target existing probability of each grid is less than the given value, no target is found within the specified time cycles, and the environmental uncertainty of all grids does not drop too much.

5.2.4. Cooperative Target Search Based on the Boids Model without Guidance

The target searching algorithm based on the Boids model without guidance is executed, and the results in Figure 18 show that one target is found, and most grids are not covered within 200 time cycles. The experimental result indicates that the algorithm based on the Boids model without guidance is not suitable to conduct the target searching mission.

5.2.5. Cooperative Target Search Based on the Boids Model with Guidance

The target searching algorithm based on the Boids model with guidance is executed. The result in Figure 19 shows that two targets are found, and nearly half of the grids are covered within the given time cycles. The experimental result indicates that the Boids model guided by the TEP information can be adopted to conduct the searching mission, but the searching effect is slightly inferior to the proposed ESUSTPM algorithm in this paper.

Figure 20 shows the relationship between the grid visited number and time cycles. The red line, purple line, green line, orange line and indigo line represent the proposed searching algorithm, the mower searching algorithm, the searching algorithm based on the Boids model with guidance, the searching algorithm based on the Boids model without guidance, the classical random searching algorithm, respectively. The grid numbers covered by the mower searching algorithm and the proposed searching algorithm are close within the specified time cycles. However, the other searching algorithms cannot cover most of the grids in the specified time cycles.

Figure 21 shows the change of the environmental uncertainty derived by the five target searching algorithms within 200 time cycles. The purple line represents the result of the mower searching algorithm. The red line is the result of the proposed ESUSTPM algorithm. Within the given simulation time, the environmental uncertainty obtained by the proposed searching algorithm decreases more than with the other searching algorithms.

Figure 22 is the relationship between the number of targets found and time cycles. Within the 200 time cycle, only the proposed ESUSTPM algorithm can find all targets. The results verify that the efficiency of the proposed ESUSTPM is higher than the other searching algorithms.

5.3. Environment with Partial Prior Information of Static Targets

Three targets are assumed to exist at (400, 500), (400, 700) and (700, 500). Therefore, the initial distribution of the target existing probability density in the mission area is shown in Figure 23 according to Equation (2).

5.3.1. The Mission Area with Accurate Prior Information

As the initial target existing probability map is known before executing the target search, UAV swarm based on the ESUSTPM algorithm tends to move to the area with the high probability information for detecting targets. Figure 24 shows that all targets are found, and the environmental uncertainty is also reduced after 200 time cycles.

Figure 25 shows that the ESUSTPM algorithm without prior information takes more time to find three targets than ESUSTPM with prior information. The results prove that prior information is helpful to improve the searching efficiency.

Figure 26 is the distribution of the statistical time cycles to find three targets. The yellow boxplots are the target searching cycle distribution of UAV swarm in the environment with prior information, and the green boxplots are the target searching cycle distribution of UAV swarm in the unknown environment without prior information. The results indicate that accurate prior information can improve the searching efficiency.

5.3.2. The Mission Area with Inaccurate Prior Information

According to prior information, three targets may be distributed in the area with high probability, as shown in Figure 23. The target coordinates actually exist in (400, 500), (400, 700), but the target in (700, 500) is not true. Prior information is not accurate enough, but the UAV swarm cannot know this situation at the initial time. Figure 27a shows the corresponding results of the proposed ESUSTPM algorithm. The red grids indicate that the targets in (400, 500) and (400, 700) are found. Concurrently, the color of the grid in (700, 500) becomes white in the emergent searching process, and the algorithm identifies that the target does not exist in this grid.

In Figure 27b, the color of grid (300, 200) becomes red in the emergent searching process, but the color of grids (400, 500), (400, 700), (700, 500) becomes lighter. The target is considered unlikely to exist in the grid (300, 200), but the target is detected and does exist in this grid.

The results in Figure 27 indicate that the proposed ESUSTPM algorithm can discover the hidden targets, and it has fault tolerance to wrong prior information.

For intuitively displaying the pros and cons of the above searching methods,

Table 2. Comparison of the above five searching methods.

Searching Methods	Grid with Target			Grid without Target
	Visited Grid Number	Environmental Uncertainty	Target Found Number	Visited Grid Number	Environmental Uncertainty
The classical random searching algorithm	26	43.92	0	29	42.76
The mower searching algorithm	100	52.34	0	100	52.34
The searching algorithm based on Boids model without guidance	53	42.86	1	40	43.93
The searching algorithm based on Boids model with guidance	75	42.11	2	82	43.53
The proposed algorithm	89	40.23	3	95	40.24

lists the comparative results of three metrics, including the visited grid number, environmental uncertainty and target found number at the 200th cycle. The results in Table 2 indicate that the proposed algorithm can minimize the environmental uncertainty within the 200 time cycles. Furthermore, the efficiency of the proposed algorithm is also the highest among these searching methods within 200 time cycles.

5.4. Statistical Analysis and Summary

To verify the robustness of the proposed ESUSTPM algorithm, five simulations with different indices, including the time cost of full coverage, environmental uncertainty and time cost of target search, are executed 10 times.

Figure 28 is the full coverage time distribution of the five searching methods. The boxplots show the simulation time cycle distribution of the full area coverage based on the classical random algorithm, the mower searching algorithm, the searching algorithm based on the Boids model without guidance, the searching algorithm based on the Boids model with guidance and the proposed ESUSTPM algorithm, from the left to the right side. The statistical results verify that the searching algorithms with a guidance mechanism are better than the algorithms without guidance. The time consumption of the proposed ESUSTPM algorithm is very close to the mower searching algorithm, and the time distribution of the proposed ESUSTPM algorithm is relatively concentrated. The results shown in Figure 28 prove that the proposed ESUSTPM method has a good efficiency in the searching mission of the area coverage.

The boxplots in Figure 29 are the distributions of the environmental uncertainty in the area coverage searching scenario at 200 time cycles. The median value of the proposed ESUSTPM algorithm is the lowest, and the environment uncertainty distribution is concentrated. This indicates the effectiveness of the proposed ESUSTPM algorithm in the index of environmental uncertainty.

Figure 30 shows the statistical results of the time cycles distribution from one found target to three found targets. The classical random searching algorithm and the searching algorithm based on the Boids model without guidance need to spend more time searching three targets. Among the five searching algorithms, the proposed ESUSTPM algorithm is the most efficient in finding each target in the searching process.

The above statistical results show that the proposed algorithm is more efficient in the reduction of environmental uncertainty and target search mission. In addition, the proposed ESUSTPM algorithm is robust to the accurate degree of prior information. In all, the proposed ESUSTPM algorithm is suitable to be applied in a cooperative searching scenario of the UAV swarm.

6. Conclusions

The conventional heuristic cooperative searching algorithms have a certain amount of calculation burden. In pace with the increase in the searching complexity, the amount of calculation becomes heavier. Therefore, an intelligent algorithm with fast calculation and highest searching efficiency is needed. Therefore, a novel cooperative search method, named the emergent search of UAV swarm, guided by the target probability map, is proposed in this paper. ESUSTPM is a new searching algorithm whereby each monomer decides its next velocity by acquiring the information of some closest neighbors and the surrounding environment. The exponential constructed function is designed and optimized to keep the given distance between the agents for obstacle avoidance and communication connectivity. The TPM is firstly proposed to guide the monomer to search the grid with the highest target existing probability emergently. The simulation results show that the controllers designed by the proposed searching algorithm can guide the UAV monomers to cover the mission area, find the hidden targets and decrease the environmental uncertainty as far as possible. In addition, the statistical results of ESUSTPM show that the searching efficiency also outperforms the conventional searching algorithms.

Emergence is a new concept, and its mechanism is suitable to be adopted to execute the complex area searching mission. It is also worth being extended to other scenarios in the future.