Multi-UAV Cooperative Anti-Submarine Search Based on a Rule-Driven MAC Scheme

Abstract

In order to enhance the anti-submarine capability of multi-unmanned aerial vehicles (multi-UAVs) in the unknown sea environment and improve the search efficiency, in this paper, we propose a rule-inspired-multi-ant colony (RI-MAC)-based UAV cooperative search algorithm. First, a special sea area anti-submarine search model is established, including an association rule-driven target probability map (TPM) model, a UAV kinematics model, and a sensor model. The novel model has the characteristics of rule linkage, which effectively improves the accuracy of target detection probability in unknown environments. Secondly, according to the established search model, a multi-objective utility function based on association rules is derived. In order to solve the problem of multi-objective optimization, an RI-MAC algorithm based on association rules is proposed, and a pheromone update method using threat avoidance is designed to optimize the search path of multi-UAVs. Finally, a simulation experiment is conducted to verify the effectiveness and superiority of the proposed search algorithm.

1. Introduction

In the past decade, UAVs have been widely used in surveillance, search, target tracking, damage assessment, and other fields. Multi-UAV cooperative anti-submarine research has received increasing attention from academics and the national defense industry [1,2,3]. Because UAV cooperative anti-submarine technology has the characteristics of good concealment and strong maneuvering search ability, it can realize reconnaissance coverage of the mission area to discover the target and obtain information. Therefore, a considerable amount of research on multi-UAV cooperative search submarine technology has been carried out [4,5,6].

Collaborative search is a probabilistic detection task, which requires updating of the probability of dynamic targets according to the observations of multi-UAVs in order to describe the position of the target in a probabilistic way and help UAVs to make increasingly accurate decisions to find the target. To achieve the probability update of dynamic targets, it is necessary to establish a search environment model. To date, significant research has been conducted in this field; among the important topics is to establish an environmental grid search map [4,7,8,9,10], which can be divided into a target probability map, digital pheromone map, return rate map, etc. For example, in [7], a search map method based on a target probability map was proposed, which can update the target existence probability in real time based on a collaborative search framework and effectively improve search efficiency. A UAV collaborative search method based on a digital pheromone model was designed in [8]; the search environment was modeled by abstracting the concept of digital pheromones, which was used to coordinate a swarm of UAVs into biological swarms, which can adapt to dynamic changes in the environment. In [9], a digital pheromone map was extended and combined with a genetic algorithm to solve the coordination problem of multi-UAVs with respect to the area coverage problem. In [10], an integrated target probability map and an certainty map were proposed based on the concept of a “territorial awareness information map”, which can efficiently coordinate the tracks of multi-UAVs in the face of clustered targets. In [4], a method of dynamic asset allocation was designed in an uncertain task environment, whereby each asset is allocated to a specific area, and, with the movement of the target, the assets are periodically reallocated, which optimizes the search space and improves the search efficiency. However, the establishment of the above search grid probability does not consider the correlation between the targets. In particular, in anti-submarine missions, the target submarine cooperates with other warships or submarines in a certain tactical formation [11]; that is, the submarine maintains a certain position in relation to the surrounding warships to form a protective formation. Therefore, it is necessary to study the correlation between target submarines and ships, as well as the formation characteristics of mutual cover, and to establish an association rule-driven target probability map model. This remains an open and challenging problem and is the research object of this paper.

On the other hand, extensive research has also been carried out on search algorithms for multi-UAV dynamic targets. Intelligent algorithms have received increasing attention as a means solving path-planning problems in complex dynamic environments. Intelligent algorithms [12,13,14,15] include particle swarm algorithms, immune genetic algorithms, bee swarm algorithms, neural network algorithms, etc. For example, in [12], the search problem was transformed into an optimization problem of detecting the target probability by using the Bayesian theorem, and a motion-coding particle swarm optimization algorithm was proposed to generate the motion path of UAVs. Considering the constraints of flight distance, initial heading, and terminal heading, an immune genetic algorithm was proposed to improve the target search efficiency of UAVs in uncertain environments [13]. A new bee colony algorithm was proposed in [14], which applied an improved search equation to generate candidate solutions to improve the search ability of the bee colony algorithm. In [15], an improved ant colony algorithm was proposed for multi-UAV cooperative search planning, and a new state transition rule and pheromone update rule were designed to ensure that the UAVs could return to the initial point under the constraint conditions as required after completing a task. However, the proposed methods and technologies are not suitable for the anti-submarine search problem that we are interested in. In particular, anti-submarine search remains an open and challenging problem due to the frequent use of quiet state in submarines, as well as the low detection rate and high false-alarm rate of onboard sensors. Related works are summarized in

Table 1. Related research.

Reference	Application	Search Map/Algorithm	Deficiency
Polycarpou [7]	Basic problem of search for stationary targets	Target probability map	Unexpanded moving targets
Parunak [8]	Stationary targets with attack	Digital pheromone map	Does not consider evading attacks
Paradzik [9]	Moving target in search area	Extension of digital pheromone map	Target motion characteristics not considered
Yue [10]	Moving formation targets in search area	Target probability map and certainty map	Aggressiveness of targets not considered
Mishra [4]	Moving target search problem with multiple searchers	Dynamic asset allocation	Not suitable for searching for submarine targets
Manh [12]	Retrieval of lost moving target problem	Motion-coding particle swarm optimization algorithm	Relevance characteristics of the target are not considered
Zhou [13]	Target search problem in uncertain environment	Immune genetic algorithm	The movement characteristics and danger of the target are not considered
Gao [14]	Path optimization problem of UAVs	New bee colony algorithm	Does not take into account the aggressiveness and relevance of the target
Zhen [15]	Cooperative search-attack mission planning problem for multi-UAVs	Improved ant colony algorithm	Correlation with aggressive target movement not considered

.

Inspired by the above research, in this paper, we propose an association rule-driven cooperative anti-submarine method for multi-UAVs, which fully considers the cooperative characteristics between the target submarine and the warship and can still find the target quickly and effectively when the target submarine and the warship have attack abilities. The main contributions of this paper are as follows: First, the association rule database and the association rule function applicable to submarine targets are constructed. On this basis, an association rule-driven probability map model is established. Secondly, an improved rule-heuristic multi-ant colony algorithm is proposed. The association rules are used as heuristic information to design the state transition rules of an ant colony, and the attack threat of a submarine is considered in pheromone updating. Simulation results show that the proposed controller achieves improved search performance.

The rest of this paper is organized as follows: In Section 1, we establish the environment model, the association rule-driven probabilistic graph model, the association rule-driven probability map update rule, the UAV model, and the association rule-based multi-objective efficiency function. In Section 2, we propose an improved rule-inspired multi-ant colony algorithm. In Section 3, we present simulation results, demonstrating the effectiveness and advantages of the proposed method. In Section 4, we present our conclusions.

2. Materials and Methods

2.1. Search Environment Modeling

In this section, we will establish an anti-submarine sea area model for multi-UAV swarms, considering the existence of an unknown number of targets (including submarines, ships, aircraft carriers, etc.) in the sea area, as shown in Figure 1. $N_v$ UAVs with detection sensors (visible light, magnetic detectors, etc.) are used to search for relevant targets within the mission area. During the search process, UAVs share information in the sea area through wireless communication. In the interest of ensuring safety, the goal is to complete the search task in the shortest time possible and at the lowest cost. The area is divided into grids, and the grid set describing the search area is defined as:

$$E=\left\{(m,n)|m=1,2,\cdots ,L_x,n=1,2,\cdots L_y\right\}$$

(1)

where $E$ represents the grid set; $(m,n)$ represents the grid coordinates; and $L_x$ and $L_y$ represent the number grids along the $x$ and $y$ axes, respectively. With this task area, an association rule-driven target probability map model, an association rule-driven probability map update rule, and a UAV motion model will be established in sequence.

2.2. Building an Association Rule-Driven TPM Model

In the mission area, the targets may exist individually or in formations depending on their tasks [11], and the target formation has different distribution rules under different task scenarios. Accordingly, the association rule base is established as follows:

$$S=\{S_1,S_2,\dots ,S_i,\dots ,S_n\}$$

(2)

where $S$ denotes the association rule base, and $S_i$ is the distribution rules of the targets in one of the tasks.

Based on (2), the association rule function for multiple targets is defined as:

$$c_k(S)={\left[\begin{array}{llllll}{\epsilon }_1{f}_k(S_1)\hfill & {\epsilon }_2{f}_k(S_2)\hfill & \dots \hfill & {\epsilon }_i{f}_k(S_i)\hfill & \dots \hfill & {\epsilon }_n{f}_k(S_n)\hfill \end{array}\right]}^{\mathrm{T}}$$

(3)

where $c_k(S)$ is the association rule for multiple targets at time $k$, $f_k(S_i)$ is the target distribution function under association rule $S_i$ at time $k$, and ${\epsilon }_i\in \left\{0,1\right\},(i=1,2,\dots ,n)$ represents the binary weight and satisfies ${\epsilon }_1+{\epsilon }_2+\cdots +{\epsilon }_i+\cdots +{\epsilon }_n=1$. When ${\epsilon }_i=1$, corresponding to ${\epsilon }_1={\epsilon }_2=,\dots ,={\epsilon }_{i-1}={\epsilon }_{i+1}={\epsilon }_n=0$, the current association rule function selection is $f_k(S_i)$.

After some targets are found, the position coordinates of other targets are updated according to the association rule function, $c_k(S)$. Due to the uncertainty of the movement of the target, the target position can only be obtained with a certain probability based on the association rule function. Therefore, the task area grid is defined as three forms: (1) high-probability area, $hp$; (2) low-probability area, $lp$; and (3) uncertain-probability area, $mp$.

The search process uses association rule $S_i$ to describe changes in high and low TPMs, as shown in the following steps.

Step 1. In the initial stage, according to the prior information, distribute the high and low TPMs, as shown in Figure 2a;

Step 2. If UAVs find targets, according to association rule $S_i$, the TPM in the sea area is updated in real time. For example, after finding target A, according to the characteristics of submarine-guardship formation [11], the TPM in the surrounding grid of A increases; the update process is shown in Figure 2b;

Step 3. If UAVs find the second target B, according to association rule $S_i$, the probability of some grids on the target connection line and the vertical line will be upgraded to the high-probability region, hp; the corresponding adjustment TPM is shown in Figure 2d;

Step 4. UAVs continue to search according to the rule base until all targets in the area are found, and the search task ends.

Then, define a matrix with multiple elements to represent the initial information of the search environment as:

$$\psi ={[\begin{array}{ccc}{p}_{mn}& c_0(S)& f_g\end{array}]}_{L_x\times L_y}$$

(4)

where $p_{mn}\in [0,1]$ is the prior probability of grid $(m,n)$; $c_0(S)$ represents the association rule initialization function; and $f_g$ is the flag bit, which is used to identify whether there is a target in the grid. The value of $f_g$ can be expressed as:

$$f_g=\{\begin{array}{l}\begin{array}{ccc}1& ,& \mathrm{target}\end{array}\\ \begin{array}{ccc}0& ,& \begin{array}{cc}\mathrm{No}& \mathrm{target}\end{array}\end{array}\end{array}$$

(5)

Based on (3)–(5), the update matrix of the TPM driven by association rules is formed as:

$$\left[\begin{array}{ccc}\left[\begin{array}{ccc}{S}_i& P_s& f_g\end{array}\right]& \cdots & \left[\begin{array}{ccc}{S}_i& P_s& f_g\end{array}\right]\\ ⋮& \ddots & ⋮\\ \left[\begin{array}{ccc}{S}_i& P_s& f_g\end{array}\right]& \cdots & \left[\begin{array}{ccc}{S}_i& P_s& f_g\end{array}\right]\end{array}\right]$$

(6)

where $\left[\begin{array}{ccc}{S}_i& P_s& f_g\end{array}\right]$ represents the information contained in a grid, $S_i$ is the association rules for the grid, and $P_s\in \{hp,lp,mp\}$ denotes the probability characteristics of the grid under association rule $S_i$.

2.3. Association Rule-Driven Probabilistic Map Update Rules

Here, based on the Bayesian probability update rule [16] and considering the association characteristics of the target, the update of an association rule-driven TPM is given as follows:

• Grid $(m,n)$ without access:

  $$p_{mn}(k+1)={\tau }_1{p}_{mn}\left(k\right)$$

  (7)

  where $P_{mn}(k)$ and $p_{mn}(k+1)$ represent the probability of grid $(m,n)$ at time k and $k+1$, respectively; and ${\tau }_1\in [0,1]$ is the dynamic information factor.
• Grid $(m,n)$ has access, and $f_g=1$:

  $$p_{mn}\left(k+1\right)=\frac{p_d{p}_{mn}(k)}{p_f(1-p_{mn}(k))+p_d{p}_{mn}(k)}$$

  (8)

  where $p_d\in [0,1]$ and $p_f\in [0,1]$ represent the detection probability and false-alarm probability of the magnetic detector, respectively; if the target in the grid $(m,n)$ is detected, the TPM of other grids is updated based on $S_i$ as follows (here, take grid $(o,p)((o,p)\ne (m,n))$ as an example):
  • High-probability area, $hp$:

    $$p_{op}(k+1)=\frac{1}{s_{hp}}{\displaystyle \sum _{i=1}^{s_{hp}}{\tau }_{hp}{p}_{mn}\left(k\right)}$$

    (9)

    where $p_{op}(k+1)$ represents the probability of the grid $(o,p)$ at time $k+1$; ${\tau }_{hp}$ is the probability enhancement factor; and $s_{hp}$ is the number of grids that conform to the high- probability area, $hp$.
  • Low-probability area, $lp$:

    $$p_{op}(k+1)=\frac{1}{s_{lp}}{\displaystyle \sum _{i=1}^{s_{lp}}{\tau }_{lp}{p}_{mn}\left(k\right)}$$

    (10)

    where ${\tau }_{lp}$ is the probability reduction factor; and $s_{lp}$ is the number of grids that meet the low-probability area, $lp$.
  • Uncertain probability area, $mp$:

    $$p_{op}(k+1)={\tau }_2{p}_{mn}\left(k\right)$$

    (11)

    where ${\tau }_2\in [0,1]$ is the dynamic information factor.
• Grid $(m,n)$ has access, and $f_g=0$:

  $$p_{mn}(k+1)=\frac{(1-p_d)p_{mn}(k)}{(1-p_f)(1-p_{mn}(k))+(1-p_d)p_{mn}(k)}$$

  (12)

If the target in grid $(m,n)$ is detected and based on association rule $S_i$, the probability of other grids is updated as (8)–(10).

2.4. UAV Modeling

2.4.1. Kinematic Model

Assuming that the UAV flies at a constant speed in the mission area, $U_i$ $i=\{1,2,3,\dots ,N\}$ represents the i-th UAV platform, and the motion model for $U_i$ is expressed as:

$$\begin{array}{l}{\dot{x}}_i=v_i\cos {\theta }_i\cos {\phi }_i\\ {\dot{y}}_i=v_i\cos {\theta }_i\sin {\phi }_i\\ {\dot{z}}_i=v_i\sin {\theta }_i\\ {\dot{\phi }}_i={\eta }_i{\phi }_{\max }\\ {\dot{\theta }}_i={\eta }_i{\theta }_{\max }\end{array}$$

(13)

where ${[x_i,y_i,z_i]}^{\mathrm{T}}$ represents the coordinates in the mission area; $v_i$, ${\phi }_i$, and ${\theta }_i$ are the velocity, yaw angle, and pitch angle of $U_i$, respectively; ${\eta }_i\in [-1,1]$ is the angular rate of $U_i$, ${\phi }_{\max }$; and ${\theta }_{\max }$ is the maximum yaw and pitch angle.

The state equation of UAV can be expressed as:

$$s_i\left(k+1\right)=s_i(k)+f(s_i(k),u_i(k),\mathsf{\Delta }t)$$

(14)

where $s_i(k)={\left[\begin{array}{ccc}{\tilde{s}}_i(k)& {\phi }_i(k)& {\theta }_i(k)\end{array}\right]}^{\mathrm{T}}$ is the state of $U_i$ at time $k$, ${\tilde{s}}_i(k)={[\begin{array}{ccc}{x}_i& y_i& z_i\end{array}]}^{\mathrm{T}}$ is the coordinate of $U_i$ at time $k$, $u_i(k)={[\begin{array}{ccc}{v}_i(k)& {\phi }_i(k)& {\theta }_i(k)\end{array}]}^{\mathrm{T}}$ represents the control input of $U_i$, ${\phi }_i(k)$ represents the heading angle at time $k$, ${\theta }_i(k)$ is the pitch angle at time $k$, $f(s_i(k),u_i(k),\mathsf{\Delta }t)=\left[\begin{array}{l}{v}_i(k)\cos ({\phi }_i(k)+{\eta }_i(k)\mathsf{\Delta }t)\mathsf{\Delta }t\\ v_i(k)\sin ({\phi }_i(k)+{\eta }_i(k)\mathsf{\Delta }t)\mathsf{\Delta }t\\ {\phi }_i(k)+{\eta }_i(k)\mathsf{\Delta }t\\ {\theta }_i(k)+{\eta }_i(k)\mathsf{\Delta }t\end{array}\right]$ is the state change at time $\mathsf{\Delta }t$, and ${\eta }_i(k)$ is the angular rate at time $k$.

The process of UAV motion planning satisfies the limited constraints, such as UAV operation, connectivity maintainence, and collision avoidance. The constraint equations are given as follows:

$$\{\begin{array}{l}{s}_i\left(k+t+1\right)=s_i(k+t)+f(s_i(k+t),u_i(k+t),\mathsf{\Delta }t)\\ b\le \Vert {\tilde{s}}_i(k+1)-{\tilde{s}}_n(k+1)\Vert \le d_c\\ v_{\min }\le v_i\le v_{\max }\\ {\phi }_{\min }\le {\phi }_i\le {\phi }_{\max }\\ {\theta }_{\min }\le {\theta }_i\le {\theta }_{\max }\end{array}$$

(15)

where $d$ represents the maximum communication range between UAVs; $b$ represents the minimum safe distance for collision avoidance between UAVs; and ${\phi }_{\min }$ and ${\theta }_{\min }$ correspond to the minimum angular velocities of the yaw and pitch angles, respectively.

2.4.2. Magnetic Detector Model

The field of view of the magnetic detector is an important factor for measuring the instantaneous search area of the UAV [17]. The airborne sensor detection model is shown in Figure 3. In the relative coordinate system, the detection width is established as follows:

$$d_u=2·h·\frac{\tan {\gamma }_u}{\sin {\alpha }_u}$$

(16)

where $h$ is the flight height of the UAV, ${\alpha }_u$ is the installation angle of the magnetic detector, and ${\gamma }_u$ is the horizontal field of view of the magnetic detector.

2.5. Association Rule-Based Multi-Objective Utility Function

The process of multi-UAV cooperative anti-submarine search is expected to complete the search task with the least fuel and the minimum search time and cost. Therefore, the utility functions consist of target discovery revenue, $J_p$; execution cost, $E_c$; collision cost, $C_c$; and threat avoidance cost, $W_c$, under the following association rules.

Target discovery revenue, $J_p$, is defined as the possibility of the i-th UAV finding targets based on association rules:

$$\begin{array}{cc}\hfill J_{pi}(k)=& {\displaystyle \sum _{(m,n)\in R_i^n}[(p_d-p_f)p_{mn}(k)+p_f}]\hfill \\ & +{\displaystyle \sum _{s=1}^{s_{hp}}[{\tau }_{hp}(p_d-p_f)p_{mn}\left(k\right)+p_f]}\hfill \\ & +{\displaystyle \sum _{s=1}^{s_{lp}}[{\tau }_{lp}(p_d-p_f)p_{mn}\left(k\right)+p_f]}\end{array}$$

(17)

Execution cost, $E_c$, is defined as time and fuel consumed by the i-th UAV to fly to the next grid according to the path planned by rule $S_i$:

$$E_{ci}(k)=\left|\right|s_i(k)-s_i(k+1)\left|\right|/v_i(k)$$

(18)

where $s_i(k)$ is the state of the UAV at time $k$, and $v_i(k)$ is the speed of the UAV at time $k$.

Collision cost, $C_c$: In order to avoid the overlapping of UAV waypoints during the search process, the collision cost set can be defined as:

$$C_{ci}(k)=c\overline{L_{hp+lp}^{i,j}(k)},i\ne j$$

(19)

where $\overline{L_{hp+lp}^{i,j}(k)}$ is the path by which the i-th UAV searches for high- and low-probability grids at time $k$, the j-th UAV chooses a path different from that of the i-th UAV, and $c$ is the collision coefficient.

Threat avoidance cost, $W_c$. The threat avoidance cost can be defined as:

$$W_{ci}(k)=w\Vert L_{safe}(k)-L(k)+\kappa R_T\Vert$$

(20)

where $w$ is the evasion threat cost coefficient; $L\left(k\right)$ is the distance between the UAV and the threat center at time k; $L_{safe}(k)$ is the safe distance at time $k$; and $L_{safe}(k)\ge L(k)-\kappa R_T$, where $R_T$ is the threat radius, and $\kappa >1$ is the random coefficient of the radius.

Furthermore, the utility function of the i-th UAV cooperative search is defined as:

$$J_i(k)={\omega }_1{J}_{pi}(k)-{\omega }_2{E}_{ci}(k)-{\omega }_3{C}_{ci}(k)-{\omega }_4{W}_{ci}(k)$$

(21)

where $J_i(k)$ is the utility function of the i-th UAV, and $0\le {\omega }_i<1,(i=1,2,3,4)$ represents the weighting factors.

Remark 1.

The abovementioned benefits and costs have different dimensions and need to be normalized separately and then summed.

3. RI-MAC Algorithm Design

In this section, an RI-MAC algorithm for collaborative path optimization for multi-UAVs is proposed, in which the ants are divided into $N_v$ populations, and each ant sub-group corresponds to one UAV. The ant colony, $UA=\{U{A}_i,i=1,2,\dots \dots N_v\}$, is defined as the number of ant populations corresponding to the drone swarm, and each ant sub-colony is defined as $U{A}_i=\{An{t}_t,t=1,2,\dots \dots M\}$, where M is the number of ants in each and sub-colony. Through the cooperation between the sub-groups, a search path that satisfies the association rule base, $S$, is constructed for the UAV group, and each ant in each sub-group has an independent computing unit, which selects the route according to the designed state transition rules and interacts with other ants to deal with its local optimization problem. In the following sections, we will discuss pheromone initialization, state transition, and pheromone update and analyze how RI-MAC performs cooperative path optimization for multi-UAV cooperative search.

3.1. Pheromone Initialization

Because the initial search stage uses the initial information, $\psi$, as in (4), the following pheromone initialization function is proposed according to the prior probability, $p_{mn}$, as:

$${\tau }_{i0}=\frac{1}{s}{\displaystyle \sum _{i=1}^s{\tau }_0{p}_{mn}}$$

(22)

where $s$ is the grid that initially conforms to the association rules, and ${\tau }_0$ is a constant.

Remark 2.

It is worth noting that the algorithm proposed in this paper effectively combines the association rules of the initial pheromone value and the initial target to initialize the environmental prior information, which considerably improves the problem of completely random initial information of the traditional meta-heuristic ant colony algorithm [15,18].

3.2. Rule-Inspired Path-Selection Strategy

In this subsection, the association rules are used as heuristic information to drive each ant to select the next grid. In the t-th iteration, the state transition rules of the ants of population l from grid $(m,n)$ to grid $(o,p)$ are designed as follows:

$$p_{mnop}^l(t)=\{\begin{array}{l}\begin{array}{cc}\frac{{\tau }_{op}^{\alpha }(t){\eta }_{op}^{\beta }(t){\phi }_{op}^{-\gamma }(t){\lambda }_{mnop}^{\sigma }(t)}{{\displaystyle \sum _{op\in U_K}{\tau }_{op}^{\alpha }(t){\eta }_{op}^{\beta }(t)}{\phi }_{op}^{-\gamma }(t){\lambda }_{mmop}^{\sigma }(t)},& op\in U_K\end{array}\\ \begin{array}{cc}0,& otherwise\end{array}\end{array}$$

(23)

where $U_K$ represents the selectable grid driven by association rules, ${\phi }_{op}(t)$ represents the value of the residual pheromone of other ant sub-groups at grid $(o,p)$, and ${\lambda }_{mnop}(t)$ implies that there is a target at grid $(m,n)$. Then, according to association rule $S_i$ and the target probability value existing at grid $(o,p)$, $\alpha$ represents the importance of the pheromone in grid selection, $\beta$ represents the relative importance of search revenue, $\gamma$ represents the inhibitory effect of other population pheromones; the larger $\gamma$ is, the stronger the relative inhibitory effect. Through this inhibition, UAVs of other groups can avoid repeated searches of the already searched grids of this group. $\sigma$ represents the enhancement factor for the rule-driven search of grid $(m,n)$, and ${\eta }_{op}(t)$ represents heuristic information, which can be defined as:

$${\eta }_{op}(t)=\frac{s_{hp}(t)+s_{lp}(t)}{L_x\times L_y}$$

(24)

where $s_{hp}(t)$ and $s_{lp}(t)$ are the number of high- and low-probability grids that conform to the association rules after the t-th iteration, respectively.

Remark 3.

It should be noted that the rule-inspired path selection strategy (23) takes association rules as heuristic information and to design the heuristic function (24). This selection strategy not only depends on the pheromone concentration of the next grid to select the path but also selects the path according to the association rules presented by the target and the grid that conforms to the association rules.

3.3. Pheromone Update Rules considering Threat Avoidance

Because this research involves a situation of avoiding threats, two pheromone update methods should be considered. One involves judging a situation in which the next grid is not threated according to (20), and the other is a situation in which the next grid is threatened.

• Pheromone update method 1 (the next grid is not threatened)

$${\tau }_{op}(t+1)=(1-\rho (t)){\tau }_{op}(t)+\mathsf{\Delta }{\tau }^{\sigma }(o,p)$$

(25)

where ${\tau }_{op}(t+1)$ and ${\tau }_{op}(t)$ are the values of the pheromone in grid $(o,p)$ before and after the update, respectively; and $\rho (t)$ represents the pheromone volatilization coefficient. The traditional pheromone volatilization coefficient is a constant value. If the setting value is too high, it is easy to fall into the local optimal solution, which affects the global search ability of the algorithm. If the setting value is too low, the search randomness is enhanced, and the convergence speed is slow. In order to improve the convergence accuracy of the algorithm, $\rho (t)$ was designed as:

$$\rho \left(t+1\right)=\frac{T_{\max }}{(T_{\max }+s_{hp}(t)+s_{lp}(t))e^{1-\rho \left(t\right)}}$$

(26)

where $T_{\max }$ is the maximum iteration number of the algorithm. It can be seen that with the search, the grid that conforms to the association rules constantly changes, and the pheromone volatilization coefficient also changes. In the initial stage, $\rho (t)$ is large. An ant colony algorithm can quickly search for a preferable path and improve the global search ability. With progressive iterations, the grid conforming to the association rules gradually increases, and $\rho (t)$ gradually decreases. At this time, the search space can be expanded to avoid falling into local convergence.

$\mathsf{\Delta }{\tau }^{\sigma }(o,p)$ is the pheromone update value, designed as follows:

$$\mathsf{\Delta }{\tau }^{\sigma }(o,p)={\displaystyle \sum _{l=1}\mathsf{\Delta }{\tau }_l^{\sigma }(o,p)}$$

(27)

where $\mathsf{\Delta }{\tau }_l^{\sigma }(o,p)$ is the pheromone left by ant $l$ in the sub-group on grid $(o,p)$, which is defined as:

$$\mathsf{\Delta }{\tau }_l^{\sigma }(o,p)=\{\begin{array}{l}\frac{rQ}{L_{s_{hp}+s_{lp}}{\delta }_1{J}_l},{\epsilon }_i=1\\ \frac{rQ}{L_0{\delta }_0{J}_l},{\epsilon }_i=0\end{array}$$

(28)

where $L_{s_{hp}+s_{lp}}$ represents the path of grids $s_{hp}$ and $s_{lp}$ traversed by the ants in path optimization, $L_0$ represents the path that ants walk before they find the target, $Q$ is the pheromone update constant, $r$ is the constant, $J_l$ is the utility function of ant $l$ obtained according to (21), and ${\delta }_1,{\delta }_2$ is the weight.

Remark 4.

In order to improve the accuracy of the MAC [18] and avoid falling into the local optimum, the volatility coefficient, $\rho (t)$, is based on the rule information of the grid. The pheromone update method of each grid is sub-divided into two cases: ${\epsilon }_i=0$and${\epsilon }_i=1$. When ${\epsilon }_i=0$, there is no applicable rule information in the initial stage, and the pheromone is updated to the path information traveled by the ants. When ${\epsilon }_i=1$, the pheromone value is re-updated according to the grid that the ants traveled that conforms to the rules.

2.

Pheromone update method 2 (the next grid is threatened);

Each time the pheromone is updated, the threat level of the next grid needs to be judged. If the grid threat level is high, the pheromone concentration of the next grid needs to be weakened:

$${\tau }_{op}(t+1)={\tau }_{op}(t)-\mathsf{\Delta }{\tau }_{op}(t)$$

(29)

where $\mathsf{\Delta }{\tau }_{op}(t)$ is the decrement of the pheromone within the grid $(o,p)$, defined as follows:

$$\mathsf{\Delta }{\tau }_{op}(t)=\mathsf{\Delta }{\tau }_0{L}_{safe}(t){\tau }_{op}(t)$$

(30)

where $\mathsf{\Delta }{\tau }_o$ is the pheromone decay coefficient, and $L_{safe}(t)$ is the safe distance of the UAV from the threat at the t-th iteration.

Remark 5.

Update method 2 considers a situation of threats in the grid. According to the level of threat, the pheromone concentration of the threat grid is relatively weakened. This update method can help UAV avoid threats and re-plan the flight path.

4. Simulation Experiment and Analysis

In order to verify the effectiveness of the proposed algorithm, a simulation environment for multi-UAV collaborative search is established in the MATLAB environment. The $20\mathrm{km}\times 20\mathrm{km}$ search mission area is divided into $40\times 40$ grids, and the initial coordinates, speeds, and headings of the four UAVs are shown in

Table 2. UAV information.

UAV Label	$\mathbf{Starting}\mathbf{Coordinate}\left(\mathbf{km}\right)$	Starting Relative Angle (°)	$\mathbf{Velocity}(\mathbf{km}/\mathbf{h})$
1	(0, 0.5)	0	100
2	(0, 0.6)	0	100
3	(0, 0.7)	180	100
4	(0, 0.8)	180	100

, assuming that the detection range of the UAVs’ magnetic detectors is a $1\times 1$ unit grid; the detection probability and the false alarm probability are $p_d=0.89$ and $p_f=0.05$, respectively; and the following three scenarios are considered:

• Scenario 1: multi-UAV search without association rules;
• Scenario 2: multi-UAV search using association rules;
• Scenario 3: comparison of the number of targets searched with or without association rules.

4.1. Scenario 1: Multi-UAV Search without Association Rules

In modern operations, ships perform various combat missions and adopt various battle formations. For example, a ship formation can consist of destroyers, frigates, and nuclear submarines, which are usually used in enemy attack and air defense. In this experiment, multi-UAVs are mainly for the search of underwater targets (nuclear submarines). The prior information on the targets is shown in Figure 4. Targets on water (warships) are in a high-probability area, so the search for underwater targets (nuclear submarines) is based on the location of warships. The initial position and movement speed of the water target are shown in

Table 3. Initial target information.

Target Label	$\mathbf{Starting}\mathbf{Coordinate}\left(\mathbf{km}\right)$	$\mathbf{Velocity}(\mathbf{km}/\mathbf{h})$
1	(6, 8)	3
2	(14, 13)	3

, and the movement direction of all targets is unknown.

In scenario 1, four algorithms—RI-MAC, MAC [15], ABC [19], and PSO [12]—are used to search. Figure 5 shows the trajectories of search targets solved by multi-UAVs using four algorithms without considering association rules. After UAV searches for water targets (marked in light blue in the figure). The probability is updated according to the Bayesian probability update formula until all underwater targets (marked in dark blue in the figure) are searched and the search task ends.

Table 4. Target-discovery benefit of four search algorithms without association rules.

Step	20	40	60	80	100	120	140	160	180
RI-MAC	0.0369	0.2161	0.7922	1.3821	1.6164	2.3892	2.9799	3.3142	3.8741
MAC	0.0396	0.2170	0.7699	1.1899	1.4871	2.1189	2.6971	3.2457	3.7614
ABC	0.0368	0.2156	0.7964	0.9843	1.4532	1.9761	2.6713	3.2241	3.6659
PSO	0.0197	0.1335	0.6395	0.9476	1.3984	1.7113	2.3712	2.7366	3.3854

shows the target-discovery benefits of the four search algorithms with no association rules. The Monte Carlo method was used for 500 experiments, with the average number of targets found under different iteration steps. In the initial stage, the differences in the number of targets searched by the four search algorithms are small. Comparing the target-discovery gains of the four algorithms, the MAC algorithm has the highest target-discovery gains at the 20th and 40th times, whereas the RI-MAC algorithm has slightly lower target-discovery gains than the MAC algorithm. Because the RI-MAC algorithm uses a grid that conforms to the association rules as the heuristic information, when the association rules are not used, the heuristic information is invalid, and there are many parameters to be initialized in the initial stage compared with the MAC algorithm, resulting in a slow convergence speed. However, with an increase in the number of iterations, the sea area information mastered by the UAVs continues to increase, and the advantages of the RI-MAC algorithm in the pheromone update method improve the performance of the algorithm. These results can be further verified by the convergence curves shown in Figure 5. According to the results of target-discovery revenue, the PSO algorithm requires the most iteration steps, with the worst revenue, and the target-discover revenue of ABC is slightly lower than that of RI-MAC and MAC.

4.2. Scenario 2: Multi-UAVs Search Using Association Rules

The target information setting of scenario 2 is the same as that of scenario 1, and association rules are also used to search under four search algorithms.

Figure 6 shows the trajectory diagram of multi-UAVs searching for targets based on association rules. If a UAV first finds an underwater target, the applicable association rules, $S_i$, are applied according to the type of underwater target. Then, the grid where the target is located is used as the center, the surrounding grid is refined, and the search radius is gradually expanded. If a related underwater target is searched, then the current searched target matches rule $S_{i+1}$, and the UAV searches for the above-water target according to rule $S_{i+1}$, as well as the connection line of the underwater target and the mid-perpendicular line, and searches for all the underwater targets to complete the search task. The UAV uses two association rules to perform the search task. The RI-MAC algorithm used in this paper has a smaller search trajectory and a better trajectory optimization effect than that of the other three algorithms.

Table 5. Target-discovery benefit of four search algorithms with association rules.

Step	20	40	60	80	100	120	140	160	180
RI-MAC	0.1246	1.2171	1.3252	2.2891	2.9764	3.7862	3.8999	3.9012	3.9799
MAC	0.1245	0.6786	1.1649	1.4461	1.9875	2.9309	3.6971	3.8457	3.9566
ABC	0.1132	0.6385	0.9695	1.3456	1.7984	2.7036	3.6362	3.7121	3.8791
PSO	0.1214	0.4356	0.7964	0.9843	1.4532	2.2761	2.6713	3.3241	3.6659

shows the target-discovery benefits of the four search algorithms under association rules. The Monte Carlo method was used for 500 experiments, and the average number of targets found under different iteration steps is presented. It can be seen from Table 4 that in the 20th iteration, the gap between the four algorithms is small; as the number of iterations increases, especially after some target information is mastered, in the process of searching for other targets, with additional association rules, the RI-MAC algorithm has more obvious advantages, and relative to the other three algorithms, it finds the target faster.

The proposed algorithm locates one more target than MAC, about 1.2 more targets than ABC, and about 1.5 more targets than PSO. With 140 iterations, the proposed algorithm can basically complete the search task, whereas the other three algorithms require more iterations. According to comparative analysis, the algorithm proposed in this paper finds the target faster than the other tested algorithms, with greater benefits, and it is more suitable for solving the nuclear submarine search problem.

In Figure 7, the search performance of the four algorithms is further compared. With 180 iterations and with an increasing number of iteration steps, the RI-MAC algorithm used in this paper achieves improved search performance relative to the other three algorithms. In addition, the proposed algorithm requires fewer iteration steps. The highest performance achieved by the RI-MAC, MAC, ABC, and PSO algorithms is 38.6, 36.2, 29.6, and 35.7, respectively. The average performance of the RI-MAC, MAC, ABC, and PSO algorithms is 26.6, 22.2, 22.5, and 19.2, respectively. It is clear that among the tested algorithms, RI-MAC is most suitable for searching for targets with association rules.

Table 6. Fitness values of four algorithms under two scenarios.

Scenario	RI-MAC	MAC	ABC	PSO
1	0.3618 ± 0.0016	0.3529 ± 0.0016	0.3526 ± 0.0129	0.2569 ± 0.0316
2	0.5412 ± 0.0034	0.4612 ± 0.0031	0.4319 ± 0.0346	0.3915 ± 0.0279

shows the fitness values corresponding to the optimal solutions of the four algorithms in under two scenarios. Each data set contains the mean and standard deviation of the probability of finding the target. It can be seen from the table that the RI-MAC algorithm used in this paper achieves the best results in scenario 1. The fitness values of the heuristic algorithms, MAC and ABC, are similar, and the advantages are not obvious. However, in scenario 2, the RI-MAC algorithm has a higher degree of distinction from the other three algorithms, with higher applicable values and smaller deviations, indicating that the algorithm proposed in this paper achieves superior performance.

In Figure 8, the convergence of the four search algorithms is further compared. In scenario 1, the fitness of the RI-MAC, MAC, and ABC algorithms is better than that of the PSO algorithm. However, in scenario 2, the RI-MAC algorithm achieves a good convergence performance, which is significantly higher than that of the other three algorithms.

4.3. Scenario 3: Comparison of the Number of Targets Searched with or without Association Rules

Table 3 and Table 4 show the target-discovery benefits of UAVs when using the RI-MAC algorithm proposed in this paper. In order to compare the target-benefit gap between the two search scenarios more intuitively, 20 targets are placed under the search task area. Two search methods are used to search, with the number of iterations set to 200, using the Monte Carlo method for 150 experiments. The target-discovery revenue statistics are shown in Figure 9. It can be seen from the figure that with an increase in the number of iterations, the number of search targets with and without association rules does not differ significantly. However as the number of targets searched increases, the group using association rules increases faster. Comparing 30 and 40 iterations, the number of targets searched by a set of association rules increases sharply from 1.912 to 6.876 compared with the group without association rules, which increased from 2.156 to 5.139 (an increase of 66.4%). Similarly, between the 60th and 80th, iterations, an increase of 69.5% is observed, and between the 80th to 100th iterations, an increase of 73% is observed. The group with association rules is able to complete the search task after 120 iterations, whereas the group without association rules requires 170 or more iterations to complete the search task. This experiment confirms that the use of association rules enables a rapid search for targets in the task area, saving search time.

5. Discussion

In this paper, a rule-driven multi-ant colony search algorithm for anti-submarine applications is proposed, and a probabilistic map model of unknown sea area targets based on prior information and association rules is established. The TPM is updated in real time according to the association rules matched by different types of targets. The effectiveness of the algorithm is verified by simulation analysis.

The next step is to use reinforcement learning technology to supplement the target association rule base so that it is more in line with the actual application scenario in order to obtain a more effective search method.