Stealth UAV Path Planning Based on DDQN Against Multi-Radar Detection

Abstract

Considering the dynamic RCS characteristics of stealthy UAVs, we proposed a stealthy UAV path planning algorithm based on the Double Deep Q-Network (DDQN). By introducing the reinforcement learning model that can interact with the environment, the stealth UAV adjusts the path planning strategy through the rewards obtained from the environment to design the optimal path in real-time. Specifically, by considering the effect of RCS from different angles on the detection probability of the air defense radar, the stealth UAV realizes the iterative optimization of the path planning scheme to improve the reliability of the penetration path. Under the guidance of a goal-directed composite reward function proposed, the convergence speed of the stealth UAV path planning algorithm is improved. The simulation results show that the stealth UAV can reach the target position with the optimal path while avoiding the threat zone.

1. Introduction

The problem of UAV penetration into the radar air defense area can be modeled as a problem of flight path planning under path constraints, fire threat constraints, and flight time constraints. The purpose of the UAV is to plan a reasonable flight path to complete the task efficiently and ensure its safety [1]. The path planning of UAV penetration can be divided into static and dynamic path planning according to the dynamic change in the fire threat area [2]. Static path planning means that the space environment faced by the UAV is unchanged, and the flight path planning is completed before the UAV flies, and the waypoints and routes are loaded. Dynamic path planning means that the space environment faced by the UAV is dynamically changing, and the UAV can realize flight and design by perceiving the environment and adjusting its state [3,4]. The problem considered in this paper belongs to dynamic path planning.

In recent years, more and more algorithms have been proposed to solve the path planning problem. The commonly used algorithms include the graph search algorithm [5], operations research linear programming algorithm [6,7], intelligent optimization algorithm [8,9], and reinforcement learning algorithm [10,11]. The graph search algorithm is one of the classical algorithms of graph theory that can effectively solve the shortest path problem. However, as the environment becomes complicated and the number of nodes increases, the solving efficiency decreases [12]. The linear programming algorithm is a mathematical method that obtains the extreme value of the objective function under linear constraints. It is widely used in military and engineering technology fields and has the characteristics of simple and efficient calculation. However, in complex space environments, its constraints are not necessarily linear [13]. In UAV path planning, the genetic algorithm [14], particle swarm algorithm [15,16], ant colony algorithm [17], and hybrid algorithm [18,19] are widely used, but there are also problems with an unclear logical relationship. Reinforcement learning is an algorithm framework that interacts with the environment to obtain feedback and optimize decisions based on it. The core of reinforcement learning is the Markov decision process. In recent years, researchers have mainly used the Q-learning algorithm and the Deep Q-Network (DQN) algorithm for path planning [20,21], but the Q-learning algorithm is prone to overfitting, leading to local optimization, and the DQN algorithm has the problem of overestimating the reward function [22,23,24]. In order to solve the above two problems, the DDQN algorithm was introduced in this paper for the path planning of the stealth UAV. Compared with Dueling DQN, DDQN reduces the Q value estimation bias and has better initial performance. Compared with PPO, it has the characteristics of offline learning, is more suitable for a discrete action space, and has low dependence on the environment model. Compared with the DDPG algorithm, the DDQN algorithm is more suitable for discrete action output and has higher sample efficiency, while the DDPG algorithm is suitable for continuous action output. Compared with the MTCS algorithm, the DDQN algorithm has better online optimization ability and less dependence on the environment model, while the MTCS algorithm is based on model search, has high dependence on the model, and has weak online optimization ability [25].

It is different from the traditional assumption that the RCS of the UAV is constant. The RCS of the stealth UAV from various angles is different, and the angle between the stealth UAV and the radar constantly changes during the flight, resulting in a dynamic change in the detection probability of the radar and the range of the anti-aircraft kill zone. Using the characteristics of the anti-aircraft kill zone, the stealthy UAV penetration into the multi-radar anti-aircraft zone with optimal path planning reaches the mission destination. Compared with existing conventional UAV path planning algorithms based on reinforcement learning [26], the main innovation of this paper lies in the proposed stealth UAV path planning algorithm. The RCS characteristics of the stealthy UAV designed by our team are significantly different from those of conventional UAVs. At present, most of the research on stealthy UAV path planning is based on the traditional A* algorithm. The RCS value of stealth UAVs varies with the angle of the fuselage. Multiple radars can detect the aircraft from various angles, posing a serious threat to stealth UAVs. To address this issue, reference [27] proposed an A* algorithm based on the trajectory planning of stealth UAVs. Considering the RCS characteristics of stealth UAVs, reference [28] proposed an improved A* algorithm, a sparse A* algorithm, and a dynamic A* algorithm, taking into account the feasibility of the trajectory through the radar detection probability, and completed the trajectory planning under three different scenarios. In view of the shortcomings of traditional algorithms in addressing the issue of stealth penetration, and fully considering the requirements of rapidity and safety when planning the flight path, an improved A* algorithm was designed in reference [29]. Considering the survivability and penetration capability under the deployment of single-base and dual-base radars, reference [30] proposed an improved A-Star algorithm based on the multi-step search method to achieve the path planning of stealth UAVs in complex scenarios. To balance the average RCS and peak RCS, reference [31] introduced a detection probability model and a penetration efficiency model and used a gradient-free optimization algorithm based on the genetic algorithm to maximize efficiency. However, the A* algorithm has the characteristics of large computational complexity and being unsuitable for dynamic environments. To address this problem, the continuous turning angle is discretized, the algorithm structure is simplified, and the convergence speed of the reinforcement learning algorithm is improved. Finally, the feasibility of the proposed algorithm was verified by comparing it with the existing algorithms.

The rest of this paper is organized as follows. The DDQN algorithm is introduced in Section 2, and the mapping relationship between UAV action and reinforcement learning elements is given. In Section 3, the environment model of this paper is presented, and the stealth UAV penetration algorithm based on DDQN is proposed. In Section 4, the effectiveness of the proposed algorithm is verified by simulation. Finally, Section 5 summarizes this paper.

2. DDQN Algorithm

This paper uses the improved DDQN algorithm under the reinforcement learning framework. After obtaining environmental information, the stealth UAV can avoid the air defense zone composed of multiple radars through real-time planning and maneuvering. A stealth UAV is an agent that cruises through the airspace at a constant speed, and the airspace environment is an anti-aircraft kill zone composed of multiple radars. The state information of the agent $\mathbf{s}$ is its position in a two-dimensional space, the action $a$ is the corner direction of the stealth UAV at the path point, and the reward function $R$ is designed as a function related to the distance from the stealth UAV to the target point and the safety state.

In the reinforcement learning framework, the state information the stealth UAV perceives is its position in the airspace. According to the state information, the stealth UAV makes decisions according to the reinforcement learning model, and the action is the steering angle of the UAV at the path point. After completing the action, the stealth UAV constantly cruised to the following path point and updated its state information. In this framework, the stealth UAV must estimate the probability of being detected by each radar according to its state before taking action. The radar detection probability is related to the reward function, and the action taken aims to maximize the reward function. The real-time decision-making framework of the stealth UAV is shown in Figure 1.

The state space perceived by stealth UAVs is its position in a two-dimensional space. It is not conducive to training because of many states. Therefore, this paper uses the powerful fitting ability of neural networks to fit the Q function through neural networks. The input of the Q-network is the position information $\mathbf{s}=(x,y)$ of the stealth UAV, and the output is the angle $a$ of the stealth UAV. In DDQN, the training and target Q-networks have the same structure, with three hidden layers. The hidden layer is the fully connected layer of 128 nodes, and the activation function adopts ReLU. The Q-network architecture is shown in Figure 2.

In the DQN algorithm, the Q-network loss function is

$${\theta }^{\ast }=\arg \underset{\theta }{\min }{\displaystyle \frac{1}{2}}{[Q_{\theta }(\mathbf{s},a)-(R+\gamma \underset{w^{\prime }}{\max }{Q}_{\theta }({\mathbf{s}}^{\prime },a^{\prime }))]}^2$$

(1)

In order to solve the problem of constantly changing targets and overestimation of the Q value when the network is updated, the DDQN algorithm proposes to use two networks to estimate, that is, one network selects the action with the largest value, and the other network calculates the value. In this case, the loss function is

$${\theta }^{\ast }=\arg \underset{\theta }{\min }{\displaystyle \frac{1}{2}}{[Q_{\theta }(s,a)-(R+\gamma Q_{{\theta }^{-}}(s^{\prime },\arg \underset{a^{\prime }}{\max }{Q}_{\theta }(s^{\prime },a^{\prime })))]}^2$$

(2)

The state is the position of the stealth UAV. In order to better determine the relative relationship between the various states of the stealth UAV, the initial states of the stealth UAV are set as the origin. The action is the discrete value of the steering angle of the stealth UAV at the path point. The angle is discretized into five fixed values $a\in \mathsf{\Omega }=\{0,{45}^{°},{90}^{°},-{45}^{°},-{90}^{°}\}$ to benefit network training and reduce the amount of data.

The reward function needs to be set to solve the sparse reward problem in stealth UAVs’ path selection. If the target position is set as a positive reward, the stealth UAV will obtain a negative reward for a long time, making searching for the target position difficult. Therefore, this paper sets the reward function as the sum of three sub-reward functions. The three reward functions are the arrival reward $R_a$, the detection penalty $R_b$, and $R_c$.

$$R_a=\left\{\begin{array}{cc}{r}_1& \mathbf{s}={\mathbf{s}}_{end}\\ 0& \mathbf{s}\ne {\mathbf{s}}_{end}\end{array}\right.$$

(3)

$$R_b=\left\{\begin{array}{cc}0& P_d<0.3\\ r_{2,0}& 0.3<P_d<0.5\\ r_{2,1}& 0.5<P_d<0.6\\ r_{2,2}& 0.6<P_d<0.7\\ r_{2,3}& 0.7<P_d<0.8\\ r_{2,4}& 0.8<P_d<0.9\\ r_{2,5}& 0.9<P_d<1\end{array}\right.$$

(4)

$$R_c=r_3$$

(5)

where Formula (1) represents the positive reward obtained by the stealth UAV for reaching the target position. The positive reward is $r_1=1$, and the positive reward is 0 if the stealth UAV does not reach the target position. Formula (2) represents the relationship between the negative reward obtained by the stealth UAV and the detection probability. When the detection probability is less than 0.5, the reward is 0. $r_{2,4}$ is a maximum value when the detection probability $P_d>0.8$ is detected by radar and destroyed by anti-aircraft fire. When the detection probability $P_d>0.9$, the path ends. At this time, $r_{2,5}$ is a maximum value, the stealth UAV has been destroyed by anti-aircraft fire, and the rest of the detection probability corresponds to the minimum value of the corresponding level. Equation (3) represents the negative reward obtained when the stealth UAV flies one step, usually set to $r_3=-1$. The total reward is set to

$$R=R_a+R_b+R_c$$

(6)

The relationship between the detection penalty and detection probability is shown in Figure 3. It can be seen that when the radar detection probability is greater than 0.8, the UAV will be greatly penalized. The purpose of the penalty value is to make the stealthy UAV avoid choosing this path node as much as possible. In this paper, the existing penalty value is set based on references [26,32] and multiple tests.

The environment will give feedback according to each action of the stealth UAV. If the action is valuable, such as approaching the target and avoiding the threat area, the environment will reward the stealth UAV. If the action is not valuable, such as moving away from the target and entering the threat area, the environment will penalize the stealth UAV.

When the stealth UAV reaches the target location or is shot down by anti-aircraft fire, a round is over. At the end of each round, the sum of the cumulative reward functions of the round is obtained, and the purpose of the stealth UAV is to obtain the maximum sum of the cumulative reward functions per round. The sum of the cumulative reward functions returned can be expressed as

$$G_t=R_t+\gamma R_{t+1}+{\gamma }^2{R}_{t+2}+\cdots ={\displaystyle \sum _{k=0}^{\infty }{\gamma }^k{R}_{t+k}}$$

(7)

where $\gamma$ means the discount factor.

In order to solve the problem of local optimality in path planning, the design of the reward function should be further clarified. When the anti-aircraft fire shoots down the stealth UAV, the penalty the stealth UAV will obtain is a maximum value, the sum of the discount of the cumulative reward will become small, and the subsequent stealth UAV will not choose this track. When the stealth UAV executes action selection at the path point, the stealth UAV will obtain a penalty value. If the stealth UAV does not fly to the target position by the shortest path, the sum of the discount of the cumulative reward will become smaller, and the subsequent stealth UAV will not choose this track. The local optimum problem in path planning can be solved by integrating the above two constraints.

3. Environmental Model and Algorithm

3.1. Environmental Model

The environment is set to a square closed airspace of $100 \mathrm{km} \times 100 \mathrm{km}$, the flight starting point of the stealth UAV is $(0,0)$, the target endpoint is $(100 \mathrm{km},100 \mathrm{km})$, and the stealth UAV cannot fly out of the set airspace. In order to enable the stealth UAV to find the shortest path to the target location quickly, the pre-training environment is set as no air defense fire network in the airspace, and the stealth UAV can freely pass to the target point. After the pre-training, the environmental model is transformed into four air defense radars in the airspace, whose positions are, respectively, $(30 \mathrm{km},30 \mathrm{km})$, $(30 \mathrm{km},70 \mathrm{km})$, $(70 \mathrm{km},40 \mathrm{km})$, and $(70 \mathrm{km},80 \mathrm{km})$. The detection probability of radar can be expressed as [30]

$$P_d\approx 0.5\times erfc(\sqrt{-\ln P_{fa}}-\sqrt{SNR+0.5})$$

(8)

where $P_{fa}$ represents the probability of false alarm, $SNR$ represents the signal-to-noise ratio (SNR) received by the radar, $erfc(·)$ is the complementary error function, and the expansion form is

$$erfc(z)=1-{\displaystyle \frac{2}{\sqrt{\pi }}}{\displaystyle {\int }_0^z{e}^{-v^2}dv}$$

(9)

According to Formula (5), the detection probability is related to the set false alarm probability and the signal-to-noise ratio of radar receiving stealthy UAV echo signal. $SNR$ can be expressed as

$$SNR={\displaystyle \frac{P_t{G}^2{\lambda }^2\sigma }{{(4\pi )}^3{R}^{4}k{T}_{e}B{F}_{n}L}}$$

(10)

where $P_t$ is the peak transmit power, $G$ is the antenna gain, $\lambda$ is the signal wavelength, $\sigma$ is the radar cross-section area, $R$ is the detection distance, $k$ is the Boltzmann constant, $T_e$ is the effective noise temperature, and $B$ is the radar operating bandwidth. $F_n$ is the noise factor. Let $\sigma =0.1$ and other parameters be consistent with Section 4; stealth UAVs fly flat in the sky. The spatial detection probability of radar is shown in Figure 4.

Figure 4 shows that when the RCS is a constant value, the detection probability distribution of radar is a concentric circle with itself as the origin, reducing the model’s complexity. However, this is different from the actual situation. During the flight of the UAV, the relative angle with the radar changes, its RCS will also change dynamically, and the detection probability of the radar for the UAV will change accordingly. Only by combining the dynamic characteristics of the RCS can the path planning be more realistic. When multiple radars are deployed independently, the stealth UAV will be shot down if the probability of being detected by any radar is greater than a given threshold. By setting this scenario, the complexity of stealth UAV penetration is increased.

3.2. Algorithm Design

The DDQN algorithm directly estimates the action value function based on the time-difference algorithm. The time-difference algorithm uses the current reward plus the value estimate of the following state as the value of the current state.

$$Q({\mathbf{s}}_t,a_t)=Q({\mathbf{s}}_t,a_t)+\alpha [R_t+\gamma \underset{a}{\max }Q({\mathbf{s}}_{t+1},a)-Q({\mathbf{s}}_t,a_t)]$$

(11)

where $\alpha$ represents the step size, and $R_t+\gamma \underset{a}{\max }Q({\mathbf{s}}_{t+1},a)-Q({\mathbf{s}}_t,a_t)$ represents the sequential difference error. In order to balance exploration and utilization, the $\epsilon$-greedy strategy is adopted in each action selection. An action is randomly selected from the action space with the probability of $\epsilon$, and the action with the maximum action value function is selected with the probability of $1-\epsilon$. The $\epsilon$-greedy strategy can be expressed as

$$\pi (a|\mathbf{s})=\left\{\begin{array}{cc}{\displaystyle \frac{\epsilon }{\left|\Omega \right|}}+1-\epsilon & a=\underset{a^{\prime }}{\arg \max }Q(\mathbf{s},a^{\prime })\\ {\displaystyle \frac{\epsilon }{\left|\Omega \right|}}& else\end{array}\right.$$

(12)

The stealth UAV uses the $\epsilon$-greedy strategy to interact with the environment and puts the five-element array $(\mathbf{s},a,R,{\mathbf{s}}^{\prime },d)$ obtained from the interaction into the experience playback pool, where $d$ indicates the state of the stealth UAV, $d=True$ indicates that the stealth UAV can continue to sail, and $d=False$ indicates the end of the voyage. There are two situations at the end of the path: one is that the stealth UAV reaches the target point and obtains a reward, and the other is that the stealth UAV is shot down by anti-aircraft fire and obtains a punishment. The Q-network is trained by randomly sampling a certain number of five-element arrays. In order to solve the problem of updating the network parameters, the target $R_t+\gamma \underset{a}{\max }Q({\mathbf{s}}_{t+1},a)$ contains the network output. We introduce the target network $Q_{{\theta }^{-}}$, and the target network is updated after a fixed $N_{train}$ round. At the same time, to solve the problem of error accumulation caused by maximum operation in the target network, the training network $Q_{\theta }$ selects the maximum action, and the target network $Q_{{\theta }^{-}}$ calculates the action value. In this case, the loss function of the network is given in Equation (2).

In summary, the path planning algorithm of a stealth UAV based on DDQN is shown in Algorithm 1.

Algorithm 1. Initialize the training network $Q_{\theta }$ and experience playback pool R, set $Q_{{\theta }^{-}}=Q_{\theta }$, and enter hyperparameters

For number of rounds $e=1\to E$ do

Get initial states $\mathbf{s}$

While $d=True$ do

According to the current network $Q_{\theta }(\mathbf{s},a)$, choose action $a_t$ with $\epsilon$-greedy strategy

Perform action $a_t$, obtain reward $R_t$, states change to ${\mathbf{s}}_{t+1}$, update $d$

Store $({\mathbf{s}}_t,a_t,R_t,{\mathbf{s}}_{t+1},d)$ in the experience playback pool

if the amount of data in R is greater than $N_{\min }$ do

Random sample $N$ groups of data ${({\mathbf{s}}_i,a_i,R_i,{\mathbf{s}}_{i+1},d)}_{i=1,\cdots ,N}$ from R

Calculate TD error target $y_i=R_i+\gamma Q_{{\theta }^{-}}({\mathbf{s}}_{i+1},\arg \underset{a^{\prime }}{\max }{Q}_{\theta }({\mathbf{s}}_{i+1},a^{\prime }))$

Minimize the loss function $L={\displaystyle \frac{1}{N}}{\displaystyle \sum _i{(y_i-Q_{\theta }({\mathbf{s}}_i,a_i))}^2}$ and update the training network $Q_{\theta }$

When the number of training network updates reaches $N_{train}$, set ${\theta }^{-}\leftarrow \theta$

end

end

end for

4. Simulation Results

Stealth UAVs realize state-to-action mapping based on the proposed path planning algorithm, and the settings of each hyperparameter of the algorithm are shown in

Table 1. Algorithm hyperparameter settings.

Parameter Name	Parameter Value	Parameter Name	Parameter Value
Learning rate	$lr=2 \times { 10}^{-3}$	Total rounds	$E=800$
Discount factor	$\gamma =0.98$	Greed probability	$\epsilon =0.01$
Target network update interval	$N_{train}=10$	Maximum storage in R	$N_{\max }=\mathrm{10,000}$
Minimum number of samples in R	$N_{\min }=1000$	Sampling batch value	$N=64$

. In terms of specific parameter settings, this paper learned the parameter settings from the literature [25] and fine-tuned the values of each parameter after multiple simulation tests. The values of each parameter are shown in Table 1. The learning rate $lr$ can effectively balance convergence speed and algorithm stability. The discount factor $\gamma$ represents the weight of future rewards, a discount factor close to 0 indicates more attention to near-term rewards, and a discount factor close to 1 indicates more attention to long-term rewards. The greedy probability $\epsilon$ can effectively balance exploration and utilization.

In the pre-training environment, there is no air defense fire unit. Through pre-training, the stealth UAV can find the nearest flight path to the target location, provide prior experience for the subsequent stealth UAV rapid penetration air defense fire unit, and reduce the training time of later training. When there is no anti-aircraft fire unit, the flight path of the stealth UAV is shown in Figure 5.

From the observation of Figure 5, we can find no anti-aircraft fire in the airspace. The stealth UAV will fly directly from the starting point to the target position according to the shortest path, verifying the effectiveness of the proposed algorithm in the absence of anti-aircraft fire. Further, the optimization rate of the algorithm can be obtained by observing the relationship between the return of the stealth UAV and the number of turns. The relationship between the return obtained by the stealth UAV and the number of rounds is shown in Figure 6.

Figure 6 shows that when there is no threat of anti-aircraft fire, the return obtained by stealth drones can converge at a speedy rate. The return fluctuates slightly after convergence because the stealth drone needs to balance the relationship between exploration and utilization, and it will randomly select actions with a probability of $\epsilon$. It helps to find the optimal decision again if the environment changes.

Then, the environment model is changed to include four air defense radars in the airspace. Before the simulation experiment, the parameters of the combat environment between the stealth UAV and the radars are first set. The parameters are shown in

Table 2. Radar counter environment parameters.

Parameter	Parameter Value	Parameter	Parameter Value
$G$	$20 \mathrm{dB}$	$P_t$	$30 \mathrm{MW}$
$f_0$	$9 \mathrm{GHz}$	$c$	${3\times 10}^8 \mathrm{m}/\mathbf{s}$
$\lambda$	$c/f_0$	$k$	$1{.38\times 10}^{-23} \mathrm{J}/\mathrm{K}$
$T_e$	$290 \mathrm{K}$	$B$	$100 \mathrm{MHz}$

.

At this time, the radar airspace detection probability is shown in Figure 7 for the RCS of the stealth UAV.

From the observation of Figure 7, it can be seen that the probability of the stealth UAV being detected by radar is not only related to the distance but is also closely related to its RCS characteristics because the RCS of stealth UAVs from different angles is different. Based on the model obtained in pre-training, the flight path planning of the stealth UAV is trained again.

In order to verify the path planning algorithm of a stealth UAV based on a dynamic RCS and DDQN algorithms, the path based on the A* algorithm [31] and the path based on a fixed RCS and DDQN algorithms are presented simultaneously in the simulation. The simulation results are shown in Figure 8.

As can be seen from Figure 6, all three algorithms can effectively avoid the anti-aircraft kill zone and reach the target position. The path length based on the DDQN algorithm with a fixed RCS is $178.91 \mathrm{km}$, the path length based on the A* algorithm is $167.20 \mathrm{km}$, and the path length based on the DDQN algorithm with a dynamic RCS is $173.05 \mathrm{km}$. Although the path based on the DDQN algorithm with a fixed RCS can reach the destination, its path is long and not optimal because it fails to use the RCS characteristics fully. Although the path based on the A* algorithm can reach the destination with a shorter path, there are also problems, such as the path of the A* algorithm taking the diagonal as the reference, lacking the global perception ability, and passing through the threatened area several times. The path of the A* algorithm has many times of turning. Compared with the above two cases, the path based on the DDQN algorithm with a dynamic RCS can effectively overcome the shortcomings of the above two cases and effectively balance the relationship between the path length and fire threat. The stealth UAV can effectively avoid the anti-aircraft kill zone by interacting with the environment and flying to the target location with the shortest path. Finally, the relationship between the return of the stealth drone and the number of rounds is shown in Figure 9.

Figure 9 shows that when four air defense radars are deployed, the return obtained by stealth drones can still converge at a speedy rate. The fluctuation of return is related to the value of the $\epsilon$. When $\epsilon =1$, the stealth UAV randomly selects actions at each step to explore the entire airspace; when $\epsilon =0$, the stealth UAV uses the current information to select the action with the greatest reward; when $\epsilon \in (0,1)$, the stealth UAV will balance the relationship between exploration and utilization; in this paper, through many experiments, $\epsilon =0.01$ is set, and the experimental results verify the effectiveness of the proposed algorithm. In summary, this section analyzes the influence of each hyperparameter’s setting on the algorithm’s performance and highlights the proposed algorithm’s advantages through a comparison of path length and convergence speed.

5. Conclusions

The UAV flight path planning technology has developed rapidly, but the penetration flight path based on the dynamic RCS of the stealth UAV has not been studied. This paper presents an effective route planning method for the stealth UAV. The method considers the detection probability of multiple radars and the constraints of RCS characteristics of the stealth UAV, discretizes the path space, and guides the stealth UAV to the target position efficiently through the proposed new reward function. The simulation results show that compared with the A* algorithm and the DDQN algorithm with a fixed RCS, the proposed DDQN-based path planning algorithm for stealth UAV penetration in multi-radar air defense zones has good results in completeness, effectiveness, and adaptability. In short, this paper has achieved good experimental results in 2D path planning, but there is no research on 3D path planning and measured verification for a stealth UAV. The subsequent plan is to further study the 3D path planning according to the 3D RCS characteristics of the stealth UAV, and on this basis, the actual measurement verification is carried out to make the experimental results closer to the real battlefield environment.