SCPL-TD3: An Intelligent Evasion Strategy for High-Speed UAVs in Coordinated Pursuit-Evasion

Highlights

What are the main findings?

• Proposes the SCPL-TD3 strategy to enable effective evasion for High-Speed UAVs against coordinated pursuers.
• Analyzes and classifies the impact of pursuer spacing on evasion difficulty levels.

What is the implication of the main finding?

• Achieves superior evasion rates while minimizing resource costs, thereby preserving operational capability for subsequent missions.
• Provides a foundational framework and a critical decision-making metric for assessing evasion difficulty and optimizing vehicle trajectory in complex pursuit-evasion scenarios.

Abstract

The rapid advancement of kinetic pursuit technologies has significantly increased the difficulty of evasion for high-speed UAVs (HSUAVs), particularly in scenarios where two collaboratively operating pursuers approach from the same direction with optimized initial space intervals. This paper begins by deriving an optimal initial space interval to enhance cooperative pursuit effectiveness and introduces an evasion difficulty classification framework, thereby providing a structured approach for evaluating and optimizing evasion strategies. Based on this, an intelligent maneuver evasion strategy using semantic classification progressive learning with twin delayed deep deterministic policy gradient (SCPL-TD3) is proposed to address the challenging scenarios identified through the analysis. Training efficiency is enhanced by the proposed SCPL-TD3 algorithm through the employment of progressive learning to dynamically adjust training complexity and the integration of semantic classification to guide the learning process via meaningful state-action pattern recognition. Built upon the twin delayed deep deterministic policy gradient framework, the algorithm further enhances both stability and efficiency in complex environments. A specially designed reward function is incorporated to balance evasion performance with mission constraints, ensuring the fulfillment of HSUAV’s operational objectives. Simulation results demonstrate that the proposed approach significantly improves training stability and evasion effectiveness, achieving a 97.04% success rate and a 7.10–14.85% improvement in decision-making speed.

1. Introduction

High-speed unmanned aerial vehicles (HSUAVs) possess distinct characteristics, including high flight speeds and exceptional evasion capabilities [1,2,3]. Therefore, HSUAVs are increasingly utilized for critical missions, including reconnaissance, and rapid mobility [4]. However, advancements in pursuit technologies—particularly the development of sophisticated pursuers with optimized guidance systems—have significantly increased the complexity of evasive maneuvers for HSUAVs [5,6]. This challenge becomes particularly pronounced when two pursuers approach from the same direction with precisely coordinated spatial and temporal intervals [7,8]. The evolving threat landscape underscores the need for the development of intelligent and adaptive evasion strategies for HSUAVs.

Conventional evasive maneuvers for HSUAVs primarily rely on predefined maneuver patterns [9,10]. However, these approaches are highly predictable, making them susceptible to tracking by pursuers while lacking the flexibility required for dynamic threat environments. In response, recent research has explored active evasion strategies. These methods can be broadly divided into two categories: threat-avoidance trajectory optimization—exemplified by the Gaussian pseudospectral method [11,12], convex optimization [13,14,15], and predictor–corrector methods [16,17,18]—and guidance laws based on modern control theory [19,20]. However, threat-avoidance trajectory optimization is highly conservative due to its reliance on complete spatial evasion. This approach often necessitates extensive trajectory deviations, leading to increased travel distance and excessive energy consumption, imposing significant operational constraints on HSUAVs.

Modern control-based evasion guidance laws address spatiotemporal engagement dynamics between HSUAVs and pursuers, primarily using optimal control [21] or differential game theory [22]. Yu et al. [23] developed optimal control-based evasion laws maintaining minimum interceptor distance, and Liu et al. [24] validated optimal evasion via 6-DOF models. However, these methods are only effective when target maneuver patterns are known, yielding unilateral optimal solutions that significantly limit practical applicability. In reality, engagements are inherently dynamic, characterized by continuous interactions between opposing forces in a two-sided game [25]. Differential game theory provides a framework for bilateral optimization [26], as demonstrated in Wang et al.’s [27] midcourse evasion strategy. However, such approaches often require significant simplifications due to computational constraints. Moreover, both methods rely heavily on precise environmental and dynamic modeling, which limits their effectiveness in highly uncertain or unpredictable real-world scenarios.

Deep reinforcement learning (DRL) combines deep learning and reinforcement learning to enable model-free decision-making in complex environments [28,29]. However, its application to HSUAV evasion, especially under coordinated pursuit, remains underexplored. Existing research on flight vehicle evasion has yielded a series of significant findings [30,31]. Li et al. [32] proposed an evasion strategy based on virtual targets and the contextual Markov decision process. To enhance adaptability and learning efficiency, Yan et al. [33] developed an unmanned aerial vehicle evasion guidance law using deep reinforcement learning in an approximate head-on pursuit–evasion (PE) scenario. These studies primarily focus on one-on-one PE scenarios, limiting their applicability to coordinated pursuit engagements. Furthermore, since reinforcement learning is an unguided, experience-free learning paradigm, deep reinforcement learning-based evasion strategies often suffer from prolonged training times and slow convergence in complex adversarial environments.

The literature on HSUAV evasion strategies presents several critical limitations:

• Most existing studies focus primarily on one-on-one PE scenarios [30,31,32,33,34], leaving a critical void in strategies against coordinated pursuit, which involves higher-dimensional state spaces and complex interaction dynamics;
• Existing methodologies primarily emphasize the successful execution of evasion maneuvers while neglecting the extent of deviation from the intended trajectory [23,24,25,26,27]. This oversight reduces the probability of successfully reaching the target zone post-evasion, thereby limiting mission effectiveness;
• Classical methods suffer from computational intractability in high-dimensional state spaces and reliance on idealized assumptions, while existing DRL approaches lack sample efficiency and exhibit slow convergence as well as training instability.

To address these challenges, this study focuses on developing an intelligent evasion strategy for HSUAVS in the coordinated PE scenario where two pursuers approach from the same direction with optimized initial space intervals. The primary contributions of this study are as follows:

• A semantic classification progressive learning framework, combined with the Twin Delayed Deep Deterministic Policy Gradient (SCPL-TD3) algorithm, is introduced to improve agent training in complex PE environments. Unlike conventional fixed training paradigms, SCPL-TD3 dynamically adjusts training complexity and leverages semantic classification to prioritize critical state-action patterns, thereby providing targeted guidance during policy learning. Furthermore, by building upon the twin delayed deep deterministic policy gradient framework, the proposed method not only accelerates convergence but also significantly improves training stability and sample efficiency in dynamic and high-dimensional environments;
• This work presents an analysis of a coordinated PE scenario involving two pursuers, from which an optimal initial spatial interval is derived to enhance the effectiveness of coordinated pursuit maneuvers. Moreover, an evasion difficulty classification framework is introduced, which systematically categorizes two-pursuer PE scenarios into distinct levels according to their spatial and dynamic constraints. This framework provides a structured method for evaluating and optimizing evasion strategies under cooperative pursuer conditions;
• A reward function is proposed that integrates energy consumption constraints and critical miss distance into a unified optimization objective. This multi-objective design not only minimizes evasion cost but also maintains sufficient operational margins, thereby ensuring the reliable accomplishment of subsequent mission objectives. By effectively balancing immediate evasion demands and overall mission requirements, the proposed reward mechanism significantly enhances the practicality and robustness of the learned evasion strategies.

A comparison of the aforementioned methods is presented in

Table 1. Comparison of HSUAV Evasion Strategies.

Method Category	Advantages	Limitations	Key Distinctions from Proposed Method
Predefined Maneuvers	Simple implementation, computationally efficient	Predictable, inflexible in dynamic environments	SCPL-TD3 uses adaptive learning rather than fixed patterns
Trajectory Optimization	Provides safe trajectories by avoiding threat zones	Overly conservative, high-energy consumption, large deviations	SCPL-TD3 balances evasion with mission constraints
Modern Control-Based Guidance Laws	Rigorous theoretical foundation, optimal solutions under ideal conditions	Requires perfect target knowledge, limited to unilateral solutions	SCPL-TD3 is model-free and computationally efficient for high-dimensional states
Existing DRL Methods	model-free approach, potential for complex environments	Slow convergence, unstable training, limited to one-on-one scenarios	SCPL-TD3 accelerates convergence and handles coordinated pursuit

in order to more clearly illustrate their respective advantages and limitations.

2. Problem Formulation and Vehicle Modeling

This paper investigates the scenario in which a HSUAV is subject to pursuit by two cooperative pursuers. A central focus is the derivation of the optimal initial space interval to enhance cooperative pursuit effectiveness, alongside the establishment of an evasion difficulty classification framework that provides a formal basis for quantifying and evaluating evasion algorithm performance. Building upon this formulation, a reinforcement learning algorithm—SCPL-TD3—is designed to address critical challenges including real-time decision-making, sample efficiency, and training stability in adversarial settings. The proposed algorithm is subsequently evaluated under the scenarios defined by the classification framework.

This section formulates multiple cooperative PE engagements, provides rigorous definitions of key concepts, and introduces the vehicle and engagement dynamics models, thereby laying the theoretical foundation for subsequent algorithm development and performance analysis.

2.1. Coordinated PE Scenario Construction

In a one-on-two confrontation scenario, the initial spatial separation between the two pursuers plays a critical role in determining their ability to sustain a continuous threat. This section classifies these scenarios into two difficulty levels—easy and hard—based on the initial interval and presents the corresponding evasion strategies. To systematically characterize the complexity of one-on-two confrontation scenarios, the following definitions are introduced:

Definition 1.

The Single Pursuer Equivalence Condition (SPEC) is defined as the state in which the space interval $\Delta X\left(t\right)$ between the two pursuers satisfies $\Delta X\left(t\right)<A_1$, where $A_1$ is a predefined threshold. Under this condition, the pursuers are considered sufficiently close to be treated as a single entity for evasion analysis and strategy design. This concept is illustrated in Figure 1.

Definition 2.

The Non-Continuous Threat Condition (NCTC) is defined as the state in which the space interval between the two pursuers $\Delta X\left(t\right)>A_2$, where $A_2$ is a predefined threshold such that $A_2>A_1$. When this condition is met, the distance between the pursuers is sufficiently large to prevent them from forming a continuous threat. Under these circumstances, the HSUAV can successfully evade using a one-on-one evasion strategy, as illustrated in Figure 2.

Definition 3.

The Coordinated Threat Condition (CTC) is defined as the state in which the space interval $\Delta X\left(t\right)$ between the two pursuers satisfies $\Delta X\left(t\right)\in \left[A_1,A_2\right]$ where $A_1$ and $A_2$ are predefined thresholds with $A_2>A_1$. Under this condition, the two pursuers maintain a persistent threat, preventing the HSUAV from escaping either simultaneously or sequentially (as illustrated in Figure 3). In such cases, a specialized evasion strategy must be designed to counteract the coordinated posture.

In this study, the one-on-two confrontation scenario is classified into two levels of difficulty based on the space interval $\Delta X$. The SPEC and NCTC are categorized as simple scenarios, whereas the CTC represents a difficult scenario requiring advanced evasion techniques. The method for determining the space interval range in difficult scenarios is detailed in Section 3.1. This paper primarily focuses on the challenges posed by the difficult scenario and proposes targeted strategies to enhance evasion performance under such conditions.

For clarity, the confrontation scenarios are designated as follows: the confrontation scenario under the SPEC is referred to as Scenario 1; the confrontation scenario under the NCTC is referred to as Scenario 2; and the confrontation scenario under the CTC is referred to as Scenario 3.

2.2. Vehicle Modeling

To accurately capture the dynamic characteristics of the vehicle during flight, the kinematic and dynamic models of the HSUAV and interceptors are defined as follows:

$$\left\{\begin{array}{l}{\displaystyle \frac{d{V}_i}{dt}}=g\left(n_{ix}-\sin {\theta }_i\right)\\ {\displaystyle \frac{d{\theta }_i}{dt}}={\displaystyle \frac{g}{V_i}}\left(n_{iy}-\sin {\theta }_i\right)\\ {\displaystyle \frac{d{\psi }_{Vi}}{dt}}=-{\displaystyle \frac{g}{V_i\cos {\theta }_i}}{n}_{iz}\end{array}\right.$$

(1)

$$\left\{\begin{array}{l}{\displaystyle \frac{d{x}_i}{dt}}=V_i\cos {\theta }_i\cos {\psi }_{Vi}\\ {\displaystyle \frac{d{y}_i}{dt}}=V_i\sin {\theta }_i\\ {\displaystyle \frac{d{z}_i}{dt}}=-V_i\cos {\theta }_i\sin {\psi }_{Vi}\end{array}\right.$$

(2)

where $i=H,I_{\varsigma }$ and $H$ represent HSUAV, and $I_{\varsigma }$ represents the $\varsigma \mathrm{th}$ pursuer ($\varsigma =1,2$); $V_i$ represents the speed of the vehicle; $g$ represents the acceleration of gravity; ${\theta }_i$ denotes the ballistic inclination angle; ${\psi }_{Vi}$ denotes the ballistic deflection angle; $n_{ix}$, $n_{iy}$, $n_{iz}$ denote the overload components along the three axes in the ballistic coordinate system, whereas $x$, $y$, $z$ represent the displacement components in the geographical coordinate system.

To enhance the realism of the confrontation scenario, the autopilot characteristics are incorporated into the modeling process. In accordance with Assumption 1 and for computational efficiency, the autopilot model is approximated as a first-order dynamical system. The relationship between the actual overload and the commanded overload is expressed as follows:

$${\displaystyle \frac{n_i\left(s\right)}{n_{ic}\left(s\right)}}={\displaystyle \frac{1}{1+{\tau }_{i}s}}$$

(3)

where $n_i$ represents the actual aircraft overload; $n_{ic}$ represents the aircraft overload command; ${\tau }_i$ is the time constant characterizing the first-order system response.

The pursuers utilize an Augmented Proportional Navigation (APN) guidance law, incorporating both longitudinal and lateral overload commands, given by:

$$\left\{\begin{array}{l}{n}_{I_{\varsigma }y}={\displaystyle \frac{N{V}_{H{I}_{\varsigma }}{\dot{\lambda }}_{H{I}_{\varsigma }P}}{g}}+\cos {\theta }_{I_{\varsigma }}+{\displaystyle \frac{1}{2}}{n}_{Hy}\\ n_{I_{\varsigma }z}=-{\displaystyle \frac{N{V}_{H{I}_{\varsigma }}{\dot{\lambda }}_{H{I}_{\varsigma }Y}\cos {\theta }_{I_{\varsigma }}}{g}}+{\displaystyle \frac{1}{2}}{n}_{Hz}\end{array}\right.$$

(4)

where $n_{I_{\varsigma }y}$ and $n_{I_{\varsigma }z}$ denote the longitudinal and lateral overload commands of the $\varsigma \mathrm{th}\left(\varsigma =1,2\right)$ pursuer. The parameter $V_{H{I}_{\varsigma }}$ represents the relative velocity between the HSUAV and the pursuer, while $N\in \left[3,5\right]$ is the navigation factor.

During the cruise phase, the HSUAV’s maneuvering flight consists of longitudinal and lateral maneuvers, which may occur independently or simultaneously. In this study, lateral maneuvers are prioritized, because lateral maneuvers help mitigate the influence of speed and altitude variations on attitude angle and control system performance. Therefore, the confrontation model is formulated in the lateral plane for simplification.

When two pursuers are present, the schematic representation of the confrontation in the lateral plane, along with relevant angle definitions, is depicted in Figure 4. Without loss of generality, the initial line-of-sight direction of $H-I_{\varsigma }$ is aligned with the X-axis, while the Z-axis remains perpendicular to the X-axis within the fixed-altitude plane.

Based on the geometric relationships in Figure 4, and considering the first-order autopilot model, the relative kinematic equation is expressed as follows:

$$\left\{\begin{array}{l}{\dot{r}}_{H{I}_{\varsigma }}=-V_H\cos \left({\psi }_{VH}-{\lambda }_{H{I}_{\varsigma }}\right)+V_{I_{\varsigma }}\cos \left({\psi }_{V{I}_{\varsigma }}+{\lambda }_{H{I}_{\varsigma }}\right)\\ {\dot{\lambda }}_{H{I}_{\varsigma }}=\left(V_H\sin \left({\psi }_{VH}-{\lambda }_{H{I}_{\varsigma }}\right)-V_{I_{\varsigma }}\sin \left({\psi }_{V{I}_{\varsigma }}+{\lambda }_{H{I}_{\varsigma }}\right)\right)/r_{H{I}_{\varsigma }}\\ {\ddot{r}}_{H{I}_{\varsigma }}=n_{H}g\sin \left({\psi }_{VH}-{\lambda }_{H{I}_{\varsigma }}\right)+n_{I_{\varsigma }}g\sin \left({\psi }_{V{I}_{\varsigma }}+{\lambda }_{H{I}_{\varsigma }}\right)+r_{H{I}_{\varsigma }}{\dot{\lambda }}_{H{I}_{\varsigma }}^2\\ {\ddot{\lambda }}_{H{I}_{\varsigma }}=\left(n_{H}g\cos \left({\psi }_{VH}-{\lambda }_{H{I}_{\varsigma }}\right)+n_{I_{\varsigma }}g\cos \left({\psi }_{V{I}_{\varsigma }}+{\lambda }_{H{I}_{\varsigma }}\right)-2{\dot{r}}_{H{I}_{\varsigma }}{\dot{\lambda }}_{H{I}_{\varsigma }}\right)/r_{H{I}_{\varsigma }}\\ {\dot{\psi }}_{VH}=n_{H}g/V_H\\ {\dot{\psi }}_{V{I}_{\varsigma }}=-n_{I_{\varsigma }}g/V_{I_{\varsigma }}\end{array}\right.$$

(5)

For the terminal guidance problem, it is assumed that the confrontation occurs near the initial triangular collision region, where the line-of-sight angle ${\lambda }_{H{I}_{\varsigma }}\left(\varsigma =1,2\right)$ is small with minimal variation. Here, it is assumed that the kinematic confrontation model can be linearized along the X-axis (Assumption 2). Defining the state variable as $X_{H{I}_{\varsigma }}={\left[z_{H{I}_{\varsigma }},{\dot{z}}_{H{I}_{\varsigma }},n_H,n_{I_{\varsigma }}\right]}^T$, where $z_{H{I}_{\varsigma }}=z_H-z_{I_{\varsigma }}$, ${\dot{z}}_{H{I}_{\varsigma }}$ is the is the derivative of $z_{H{I}_{\varsigma }}$. $n_H$ and $n_{I_{\varsigma }}$ are the normal overloads of HSUAV and pursuers, respectively. The linearized confrontation model is given by:

$${\dot{X}}_{H{I}_{\varsigma }}=A{X}_{H{I}_{\varsigma }}+B_H{n}_{Hc}+B_{I_{\varsigma }}{n}_{I_{\varsigma }c}$$

(6)

where

$$A=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& \mathrm{g}\cos {\psi }_{VH0}& \mathrm{g}\cos {\psi }_{V{I}_{\varsigma }0}\\ 0& 0& -{\displaystyle \frac{1}{{\tau }_H}}& 0\\ 0& 0& 0& -{\displaystyle \frac{1}{{\tau }_{I_{\varsigma }}}}\end{array}\right],B_H=\left[\begin{array}{c}0\\ 0\\ {\displaystyle \frac{1}{{\tau }_H}}\\ 0\end{array}\right],B_{I_{\varsigma }}=\left[\begin{array}{c}0\\ 0\\ 0\\ {\displaystyle \frac{1}{{\tau }_{I_{\varsigma }}}}\end{array}\right]$$

(7)

Assumption 1

[23]. This study assumes that both the HSUAV and the pursuers have access to real-time state information of the other, enabled by their respective sensor systems. Specifically, the HSUAV is equipped with a radar warning receiver, while the pursuers employ active radar or infrared seekers.

Assumption 2

[35]. The autopilot dynamics of both the HSUAV and the pursuers are approximated by a first-order system. This is a standard and widely adopted simplification in guidance and control literature for high-performance vehicles. While real-world autopilots are higher-order systems, their dominant closed-loop behavior in the pursuit-evasion context is often effectively captured by a first-order lag, prioritizing computational tractability for large-scale Monte Carlo simulations without significantly sacrificing the fidelity of the engagement realism.

Assumption 3

[36]. The kinematic confrontation model is linearized along the X-axis. This simplification is justified by the endgame scenario characteristics, wherein the extremely high relative velocity (exceeding Mach 10) results in a very short engagement time, and the limited maneuverability (with accelerations commonly bounded within 80 m/s²) ensures that the ballistic declination angles of both adversaries remain nearly constant. This approach, prevalent in terminal-phase guidance studies, is essential for deriving analytical solutions and facilitates the tractable design and analysis of the proposed intelligent evasion strategy.

3. Methods

3.1. Initial Common Space Interval Generation

Due to speed disadvantages relative to HSUAV, pursuers must adopt an approximately head-on posture to maximize effectiveness. This posture is adopted in the present study. This section employs Monte Carlo simulations to determine the common space interval that enables pursuers to achieve a coordinated pursuit posture. Three typical maneuvering patterns of HSUAV are considered, providing a basis for designing an optimal evasion strategy.

A hybrid framework combining offline training and online application is developed to determine the space interval between interceptors.

The offline training phase consists of three key steps. First, three representative maneuver commands for HSUAV are selected: (1) constant-value maneuver (HSUAV maintains a constant maximum available overload). (2) Bang-Bang maneuver (HSUAV initially applies full positive overload for the first interceptor, followed by full negative overload for the second interceptor). (3) HSUAV follows maneuvering strategies documented in the Ref. [9]. Second, Monte Carlo simulations are conducted to explore space intervals, identifying those that facilitate coordinated posture. Finally, neural networks are trained to approximate the common space intervals, leading to the development of a space interval estimation network.

The online application phase utilizes the trained estimation network to determine the appropriate initial interval between the two pursuers in real time. Given a new confrontation scenario, the initial interval between the first pursuer and HSUAV is input into the network, which then estimates the interval required to establish a coordinated pursuit posture. The algorithmic process is illustrated in Figure 5.

3.2. SCPL-TD3 Algorithm

To address the complexities of coordinated PE game and reduce training difficulty while improving algorithm convergence under adverse conditions, this section introduces targeted enhancements to the TD3 algorithm to develop an effective evasion strategy for HSUAV.

Building on this objective, the semantic classification progressive learning with twin delayed deep deterministic policy gradient (SCPL-TD3) algorithm is proposed. This algorithm is designed to enhance learning efficiency, stability, and adaptability, thereby ensuring robust performance in high-dimensional and adversarial environments. To establish a foundation for the proposed improvements, an overview of the traditional TD3 algorithm is first provided, outlining its key mechanisms and inherent limitations.

The TD3 algorithm is an optimization-based extension of DDPG. During training, the agent generates information $\left(s_t,a_t,s_{t+1},r_t\right)$ at each step and stores it as a tuple in the experience replay buffer (ERB) $D$. The actor and critic networks update their parameters by randomly selecting small batches of data from this buffer. The critic network is trained by minimizing the following loss function $L\left({\mathit{\theta }}^Q\right)$:

$$\left\{\begin{array}{l}L\left({\mathit{\theta }}^{Q_i}\right)={\mathrm{E}}_{s_t,a_t,r_t,s_{t+1}}\left[{\left(y_i-Q_i\left(s_i,a_i | {\mathit{\theta }}^{Q_i}\right)\right)}^2\right]\\ y_i=r_i+\gamma \underset{i=1,2}{\min }{Q}_i^{\prime }\left(s_{t+1},\tilde{a}\right)\\ \tilde{a}\leftarrow {\pi }_{\varphi }^T\left(s_{t+1}\right)+ϵ,ϵ\sim clip(\mathcal{N}(0,\tilde{\sigma }),-c,c)\end{array}\right.$$

(8)

where $\tilde{\alpha }$ represents the action space augmented with Gaussian white noise to encourage exploration and prevent the policy from converging to a local optimum, ${\mathit{\theta }}^{Q_i}$ is the parameter of the critic network, ${\mathit{\theta }}^{Q_i^{\prime }}$ is the parameter of the critic target network, $i=1,2$, $y_i$ is the target value, and $c$ defines the noise truncation range.

The gradient update for the actor network parameters is expressed as follows:

$${\nabla }_{{\mathit{\theta }}^{\mu }}J\left({\mathit{\theta }}^{\mu }\right)\approx {\displaystyle \frac{1}{N}}{\displaystyle \sum _i{\nabla }_a{Q}_i\left(s,a | {\mathit{\theta }}^{Q_i}\right)}{|}_{s=s_i,a=\mu \left(s_i\right)}{\nabla }_{{\mathit{\theta }}^{\mu }}\mu \left(s | {\mathit{\theta }}^{\mu }\right){|}_{s_i}$$

(9)

where ${\mathit{\theta }}^{\mu }$ denotes the parameters of the actor network; ${\nabla }_a{Q}_i\left(s,a | {\mathit{\theta }}^{Q_i}\right)$ represents the gradient of the critic network $Q_i$ with respect to action $a$; ${\nabla }_{{\mathit{\theta }}^{\mu }}\mu \left(s | {\mathit{\theta }}^{\mu }\right)$ corresponds to the gradient of the actor network with respect to its parameter ${\mathit{\theta }}^{\mu }$.

In the TD3 framework, deep neural networks parameterized by ${\mathit{\theta }}^{\mu }$, ${\mathit{\theta }}^Q$, and ${\mathit{\theta }}^{Q^{\prime }}$ are employed to represent the actor network, critic network, and critic target network, respectively. The optimization process in TD3 is centered on three key modifications: (1) Mitigating overestimation bias in the critic network within the DDPG framework through a strategy similar to the dual Q-network approach. (2) Delaying the actor network update, a technique that improves algorithmic stability. (3) Introducing noise into the actor target network to enhance resilience against perturbations. These modifications collectively define the TD3 algorithm, as illustrated in Figure 6.

While these enhancements improve performance in many continuous control tasks, they are not inherently designed to address the challenges of cooperative interception scenarios, such as rapid adaptation to dynamic threats and efficient exploration of high-dimensional state spaces. To overcome these limitations, further improvements to the TD3 algorithm are proposed in this study.

In cooperative PE scenarios, HSUAVs encounter significant challenges, including high-dimensional state spaces, dynamic adversarial interactions, and the need for rapid adaptation. To address these challenges, progressive learning is integrated into the training framework. Initially, the HSUAV undergoes training in a basic evasion scenario to acquire fundamental strategies such as attitude adjustment and overload control. Once proficiency is achieved, the training advances to a more complex cooperative PE scenario, where the HSUAV must navigate intricate interactions with multiple pursuers. This progressive learning approach not only accelerates the acquisition of evasion tactics but also significantly enhances adaptability to diverse and dynamic confrontation scenarios, providing a robust and scalable solution for real-world deployment.

To further improve algorithmic efficiency and mitigate the limitations associated with random experience selection, the SCPL-TD3 algorithm incorporates an importance- based classification mechanism to optimize experience replay. Specifically, SCPL-TD3 employs three ERBs to categorize and store experience samples based on their relative importance, prioritizing high-importance samples to accelerate learning. The classification methodology is structured as follows:

At the initialization stage, the importance of all samples in the three ERBs is set to zero. When a new experience sample is generated, the first step is to determine whether the agent successfully evades one or two interceptors. If the evasion attempt fails, the sample is assigned an importance value of −1 and stored in the corresponding experience buffer ${\Xi }_{low}$. Conversely, the average immediate reward value of all successful experience samples is updated. The next step involves comparing the sample’s immediate reward value to this updated average. If the sample’s immediate reward exceeds the average of successful experiences, it is assigned an importance value of 1 and placed in the corresponding experience buffer ${\Xi }_{high}$. Otherwise, the sample is assigned an importance value of 0 and stored in the designated buffer ${\Xi }_{medium}$.

During training, a greater proportion of high-importance experiences should be utilized, particularly those stored in the high and medium importance ERBs ${\Xi }_{high}$ and ${\Xi }_{medium}$. Accordingly, experience replay is conducted proportionally, as described by the following sampling method:

$$\left\{\begin{array}{l}{N}_1=\wp \left(e^{-\varpi }\right)M\\ N_2=\mathcal{l}M\\ N_3=M-N_1-N_2\end{array}\right.$$

(10)

where $N_1$, $N_2$, and $N_3$ represent the quantities sampled from the high, medium, and low importance ERBs, respectively; $M$ denotes the total number of samples; $\mathcal{l}\in \left(0,1\right)$ represents the medium- importance sample sampling rate; $\wp \left(\cdot \right)$ is the precision function, retaining three decimal places; and $\varpi$ denotes the total number of samples. The first term indicates that as the total sample size increases, the proportion of samples drawn from ${\Xi }_{low}$ decreases.

Based on these considerations, the framework for constructing the HSUAV evasion model using the SCPL-TD3 algorithm is presented in Figure 7.

In the figure, Subtask 1 corresponds to Scenario 2, while Subtask 2 corresponds to Scenario 3. The term $a_t={\aleph }_{Gaussian}\left({\mu }_t,{\sigma }^2\right)$ represents the action space augmented with Gaussian white noise, and $B_{sample}$ denotes the sampled information.

The selection of Scenario 2 as Subtask 1 is based on the following rationale: In Scenario 1, the spatial separation between the two pursuers is sufficiently small, allowing the HSUAV to evade them simultaneously. This characteristic makes Scenario 1 less suitable for designing evasion strategies applicable to coordinated PE scenarios. Scenario 2, which presents a more challenging and realistic context, is therefore selected as Subtask 1 to support the development of effective evasion strategies.

Remark 1.

The training process employs a progressive learning strategy, structured into two distinct phases. Initially, the model is trained on simple scenarios from sub-task 1 until full convergence is achieved. The parameters obtained from this phase are then saved and used to initialize the model for the subsequent phase (sub-task 2), with this process advancing sequentially. The escalation in complexity is primarily manifested through gradual increases in data dimensionality and task difficulty. Transition between phases is determined based on the model’s convergence on the current sub-task, specifically when performance on the validation set meets a predefined threshold. This ensures that knowledge from the preceding phase is thoroughly assimilated before proceeding to more complex scenarios. This approach not only enhances the stability of the training process but also facilitates efficient knowledge transfer in challenging settings.

3.3. Design of State Space and Motion Space

The HSUAV perceives its current state and makes decisions based on available state information. However, the high-dimensional nature of the state space complicates the extraction of relevant information and reduces computational efficiency. Additionally, insufficient state information decreases the distinguishability between states, impairing decision-making accuracy and increasing the likelihood of errors. To address these challenges, a normalized state space is formulated based on the relative motion equation, integrating relative motion parameters, HSUAV state variables, and pursuer position data, as expressed in the following equation:

$${\mathit{s}}_{state}={\left[r_{H{I}_1}/r_{H{I}_{1}0},r_{H{I}_2}/r_{H{I}_{2}0},{\lambda }_{H{I}_1},{\lambda }_{H{I}_2},{\sigma }_1{\dot{\lambda }}_{H{I}_1},{\sigma }_2{\dot{\lambda }}_{H{I}_2},V_H/V_{I_1},V_H/V_{I_2}\right]}^T$$

(11)

where: $r_{H{I}_1}$ and $r_{H{I}_2}$ denote the relative distances between the HSUAV and the two pursuers, while $r_{H{I}_{1}0}$ and $r_{H{I}_{2}0}$ represent their initial values. The parameters ${\lambda }_{H{I}_1}$ and ${\lambda }_{H{I}_2}$ correspond to the line-of-sight angles between the HSUAV and the pursuers. The HSUAV’s flight speed is given by $V_H$, while $V_{I_1}$ and $V_{I_2}$ denote the flight speeds of the two pursuers. The normalization coefficient ${\sigma }_i\left(i=1,2\right)$ ensures that the magnitudes of selected state variables remain comparable, thereby facilitating algorithm convergence.

The objective of deep reinforcement learning is to determine the optimal action for each state. Based on guidance requirements, the selected action is defined as the HSUAV’s normal overload $n_H$. Accordingly, the following condition must be satisfied:

$$n_H\in \left[-n_{Hmax},n_{Hmax}\right]$$

(12)

where $n_{Hmax}$ represents the overload limit for HSUAV.

3.4. Reward Function Design Considering the Evasion Cost

In HSUAV evasion scenarios, evasion cost is a critical factor in strategy design. Two primary aspects are considered: (1) the HSUAV’s lateral maneuver distance and (2) its energy consumption. The lateral maneuver distance directly influences the feasibility of subsequent missions; excessive deviation may hinder mission completion. Meanwhile, energy consumption affects the ability to execute future evasion.

The design of the reward function plays a crucial role in determining the efficiency and convergence of reinforcement learning algorithms, as it directly shapes the agent’s learning behavior and performance. In this study, a comprehensive two-part reward function is proposed, consisting of process rewards ($R_1$, $R_2$, and $R_3$) and terminal rewards ($R_4$ and $R_5$). The process reward, which quantifies the effectiveness of evasion, is defined as follows:

$$R_1=-{\beta }_1\left({\displaystyle \frac{1}{r_{H{I}_1}}}-{\displaystyle \frac{1}{r_{H{I}_{1}0}}}\right)+{\beta }_2\left|{\dot{\lambda }}_{H{I}_1}\right|-{\beta }_3\left({\displaystyle \frac{1}{r_{H{I}_2}}}-{\displaystyle \frac{1}{r_{H{I}_{2}0}}}\right)+{\beta }_4\left|{\dot{\lambda }}_{H{I}_2}\right| , v_{H{I}_1}<0 \mathrm{o}\mathrm{r} v_{H{I}_2}<0$$

(13)

where ${\beta }_1$, ${\beta }_2$, ${\beta }_3$, and ${\beta }_4$ are weighting factors. As shown in Equation (13) R₁ consists of four components. The first component $-{\beta }_1\left({\displaystyle \frac{1}{r_{H{I}_1}}}-{\displaystyle \frac{1}{r_{H{I}_{1}0}}}\right)$ and third component $-{\beta }_3\left({\displaystyle \frac{1}{r_{H{I}_2}}}-{\displaystyle \frac{1}{r_{H{I}_{2}0}}}\right)$ components are derived from potential energy functions, where the penalty (i.e., repulsive force) intensifies as the HSUAV approaches an pursuer. The second component ${\beta }_2\left|{\dot{\lambda }}_{H{I}_1}\right|$ and fourth component ${\beta }_4\left|{\dot{\lambda }}_{H{I}_2}\right|$ incentivize maneuvering by promoting an increased rate of change in the line-of-sight angle.

The survival reward $R_2$ is designed to encourage prolonged survival, assigning a higher reward for extended evasion time. It is expressed as:

$$R_2={\beta }_{5}t$$

(14)

where ${\beta }_5$ is a weighting factor.

The reward function $R_3$ is formulated in Equation (15) to discourage the HSUAV from sustaining high overloads over extended periods, thereby minimizing energy consumption and enhancing operational efficiency:

$$R_3=-{\beta }_6{n}_{H\max }t , n_{H\max }-\left|n_{Hc}\right|\le \delta$$

(15)

where ${\beta }_6$ is the weighting factor. $\delta$ is a constant set to 0.1 in this study.

In addition to process rewards, a terminal reward function is introduced to evaluate the final outcome of the HSUAV’s evasion strategy. This function consists of two components, $R_4$ and $R_5$, whose design rationale and formulations are detailed below.

The first component, $R_4$, represents the final reward for successfully evading the first pursuer. It incentivizes the HSUAV based on the favorability of the terminal state following successful evasion while imposing a penalty of −100 in the event of failure. This component ensures that successful evasion is prioritized and that the HSUAV optimizes its terminal state. The formulation of $R_4$ is expressed as follows:

$$R_4=\left\{\begin{array}{l}{e}^{{\displaystyle \frac{1}{{\left|r_{H{I}_1}\left(t_{f_1}\right)-z_c\right|}^2}}} , v_{H{I}_1}\ge 0 \& r_{H{I}_1}\left(t_{f_1}\right)\ge 1\\ -100 , v_{H{I}_1}\ge 0 \& r_{H{I}_1}\left(t_{f_1}\right)<1\end{array}\right.$$

(16)

where $t_{f_1}$ denotes the moment of intersection between the HSUAV and the first pursuer, $z_c$ is defined as the maximum distance that satisfies the direct collision condition, serving as the lower limit of the miss distance.

The second component, $R_5$, represents the final reward for evading the second pursuer. Similar to $R_4$, it rewards the HSUAV based on the favorability of the terminal state upon successful evasion and imposes a penalty of −100 in case of failure. $R_5$ is formulated as follows:

$$R_5=\left\{\begin{array}{l}{e}^{{\displaystyle \frac{1}{{\left|r_{H{I}_2}\left(t_{f_2}\right)-z_c\right|}^2}}} , v_{H{I}_2}\ge 0 \& r_{H{I}_2}\left(t_{f_2}\right)\ge 1\\ -100 , v_{H{I}_2}\ge 0 \& r_{H{I}_2}\left(t_{f_2}\right)<1\end{array}\right.$$

(17)

where $t_{f_2}$ denotes the moment of intersection between the HSUAV and the second pursuer.

The total reward is computed as follows:

$$R=R_1+R_2+R_3+R_4+R_5$$

(18)

For the reward function weight design, a two-stage approach was adopted. The initial weights were determined by the relative importance of each objective, prioritizing survival, then energy consumption, and finally the rewards related to evasion maneuvers, and were subsequently refined through empirical tuning in preliminary simulations to ensure a balanced contribution from each term.

3.5. Network Structure Design

Both the policy network and the value function network are implemented using a fully connected neural network comprising three hidden layers. The ReLU function is adopted as the activation function for the hidden layers.

The proposed network structure is summarized in

Table 2. Overall network structure in the simulation.

Type of the Network	Policy Network	Actor Network
Input layer	8	8
Hidden layer 1	256	256
Hidden layer 2	256	256
Hidden layer 3	256	256
Hidden layer 4	128	256
Hidden layer 5	64	128
Output layer	1	1

. The parameters of the policy and value function networks are optimized using the Adam optimizer.

Remark 2.

This study introduces a space interval calculation approach based on a combined “offline training + online application” framework, allowing real-time generation of initial space intervals for the two pursuers. This approach enhances the robustness and generalizability of the strategy in highly adversarial scenarios.

Remark 3.

Given the complexity of coordinated PE scenarios, this study divides the problem into two subtasks: simple PE scenarios and difficult PE scenarios. The proposed SCPL-TD3 algorithm adopts a progressive learning approach, where the HSUAV first masters evasion strategies for simple scenarios before advancing to more complex ones. This enables the HSUAV to gradually acquire new strategies from relatively simple to increasingly challenging conditions, achieving adaptive evasion across diverse confrontation scenarios. Additionally, to address the inefficiency caused by random experience sampling, SCPL-TD3 designs three prioritized experience replay buffers to classify and store experience samples based on their importance, significantly improving training efficiency.

Remark 4.

The introduction of the critical miss distance concept in the design of $R_4$ and $R_5$ strengthens the HSUAV’s evasion strategy. By incentivizing the HSUAV to evade both pursuers with the smallest possible miss distance, the proposed reward structure minimizes lateral maneuvering distance while enhancing the HSUAV’s capability to execute subsequent missions. This distance, the proposed reward structure minimizes lateral maneuvering distance while enhancing the HSUAV’s capability to execute subsequent missions. This optimization ensures efficient energy utilization and improves overall mission effectiveness in complex engagement scenarios. Additionally, the combined effects of $R_3$, $R_4$, and $R_5$ effectively reduce evasion costs, supporting both energy efficiency and operational performance.

Remark 5.

Compatibility with Ground Intervention. Although the proposed SCPL-TD3 algorithm is designed for autonomous on-board decision-making, it remains compatible with ground-based guidance updates. The policy can incorporate high-level commands—such as revised waypoints or new mission objectives—received via communication links. This feature enhances operational flexibility, though real-world implementation would need to account for communication delays and reliability, especially in contested environments.

4. Simulation and Analysis

This section presents a comprehensive simulation analysis of the HSUAV’s one-versus-two engagement scenario, focusing on three critical aspects: algorithm effectiveness, robustness, and real-time performance.

The simulations were conducted using MATLAB 2022b on a hardware platform equipped with an Intel Core i5-12500H CPU (3 GHz), an RTX 3060 GPU with 16 GB of memory, and a 1 TB SSD. The detailed simulation parameters are provided in

Table 3. Simulation parameters.

Variable	Value
Initial coordinate value of HSUAV $\left(x_{H0},y_{H0},z_{H0}\right)/\mathrm{km}$	$\left(0,25,0\right)$
Initial coordinate value of the typical $\mathrm{situation} 1 \left(x_{I_{1,2}0},y_{I_{1,2}0},z_{I_{1,2}0}\right)/\mathrm{km}$	$\left(4,25,0\right),\left(5.5,25,0\right)$
Initial coordinate value of the typical $\mathrm{situation} 2 \left(x_{I_{1,2}0},y_{I_{1,2}0},z_{I_{1,2}0}\right)/\mathrm{km}$	$\left(4,25,0\right),\left(4.5,25,0\right)$
Initial coordinate value of the typical $\mathrm{situation} 3 \left(x_{I_{1,2}0},y_{I_{1,2}0},z_{I_{1,2}0}\right)/\mathrm{km}$	$\left(4,25,0\right),\left(8,25,0\right)$
$\mathrm{Space} \mathrm{interval} \mathrm{range} \left[A_1,A_2\right]/\mathrm{km}$	$\left[0.75,2\right]$
Mach number $V_H,V_{I_{1,2}}/\mathrm{Ma}$	6, 4, 4
Initial value of the ballistic deviation angle ${\psi }_{VH0},{\psi }_{V{I}_{1,2}0}{/}^{\circ }$	0, 180, 180
Time constants of autopilot ${\tau }_H,{\tau }_{I_{1,2}}/\mathrm{s}$	0.5
Maximum lateral overload $n_{Hmax},n_{I_{1,2}max}$	2, 6, 6
Miss distance threshold $z_c/\mathrm{m}$	1
Sampling time/ms	0.1
Navigation coefficient $N$	4
Size of the ERB	${10}^6$
Small batch sample size	256
Learning rate of actor network and critic network	$1\times {10}^{-3},1\times {10}^{-4}$
Initial exploration rate	0.3
Attenuation factor	$6\times {10}^{-5}$
Discount factor $\gamma$	0.99

. Both pursuers were guided by the APN law. The maximum available overload ratio between the HSUAV and the pursuer was set to 1:3, reflecting the pursuer’s superior maneuverability. This substantial disparity in overload capacity resulted in a lower instantaneous maneuvering capability for the HSUAV, thereby increasing the evasion difficulty.

4.1. Effectiveness Validation

4.1.1. Training Efficiency and Performance

A comprehensive simulation framework was established to evaluate the effectiveness and sample efficiency of the proposed SCPL-TD3 algorithm, with a comparative analysis against baseline methods TD3 and PPO. The training for all algorithms was conducted over a maximum of 6000 episodes on the complex Subtask 2.

As illustrated in Figure 8, the proposed SCPL-TD3 algorithm demonstrates clear superiority over the baseline methods, a result attributable to its progressive learning strategy. By leveraging knowledge transferred from Subtask 1, SCPL-TD3 achieved convergence approximately 1400 iterations faster than TD3 (around 2800 versus 4200 iterations) and exhibited significantly smaller post-convergence reward fluctuations, highlighting its improved sample efficiency and training stability. Furthermore, the PPO algorithm suffered from premature convergence to a local optimum. In contrast, SCPL-TD3 continued effective exploration, achieving a final reward value substantially higher than PPO, which underscores its enhanced capability in finding superior policies.

The trained agent was subsequently evaluated through simulation tests in both relatively simple tasks (Scenarios 1 and 2) and a more challenging task (Scenario 3).

4.1.2. Simple Evasion Scenario

A comparative analysis in both Scenario 1 and Scenario 2 is conducted against the baseline method M1 from Ref. [9], which is based on classical game-theoretic approaches, to further validate the practical effectiveness of the proposed intelligent evasion strategy.

The simulation results for Scenario 1 are presented in Figure 9 and Figure 10, corresponding to the proposed SCPL-TD3 strategy and the baseline method M1, respectively. As shown in Figure 9a,b and Figure 10a, both methods achieve successful penetration with miss distances greater than 1 m. However, a comparative analysis of the two-dimensional evasion trajectories in Figure 9a and Figure 10a reveals that the maximum lateral maneuvering distance of the proposed strategy is 0.0072 km, significantly smaller than that of M1 (0.0139 km), indicating a substantial improvement in trajectory optimality. Furthermore, by comparing the overload command curves in Figure 9c with the trajectory result in Figure 10b, it is observed that the SCPL-TD3 strategy avoids the prolonged use of maximum overload maneuvers employed by M1, thereby reducing energy consumption. This is attributed to the intelligent maneuver sequence adopted by the HSUAV based on the proposed strategy: a short-duration positive overload maneuver followed swiftly by a negative overload maneuver. This tactic compresses the pursuers’ reaction time, facilitating a successful escape, as evidenced by the synchronized variation between the overload command in Figure 9c and the ballistic deflection angle in Figure 9d.

The results for Scenario 2, illustrated in Figure 11 and Figure 12, further validate the advantages of the SCPL-TD3 strategy. Both methods again achieve successful evasion, as confirmed by the miss distances exceeding 1 m in Figure 11a,b and Figure 12a. The proposed strategy again demonstrates a significantly smaller maximum lateral maneuvering distance (0.0186 km) compared to M1 (0.0451 km), as evident from the trajectories in Figure 11a and Figure 12a. Moreover, analysis of the overload commands in Figure 11c and Figure 12b indicates that SCPL-TD3 avoids the energy-inefficient, sustained maximum overload maneuvers characteristic of M1. The overload and ballistic deflection angle curves in Figure 11c,d reveal well-timed directional switches in maneuvers, thereby effectively disrupting the pursuers’ tracking and facilitating a successful evasion.

In summary, while both strategies achieve successful evasion against two pursuers in the simple scenarios, the proposed SCPL-TD3 strategy exhibits superior performance by optimizing the flight trajectory through minimized lateral maneuvering distance and reduced energy consumption. This enhanced efficiency significantly lowers the overall evasion cost, underscoring the practical advantage of the SCPL-TD3 algorithm.

4.1.3. Difficult Evasion Scenario

A more comprehensive comparative analysis was conducted in the challenging Scenario 3 to rigorously evaluate the generalization capability and strategic superiority of the proposed SCPL-TD3 strategy. The baseline methods for comparison included the constant-value maneuver, the Bang-Bang maneuver, and the method M1 from Ref. [9]. The simulation results are presented in Figure 13, Figure 14, Figure 15 and Figure 16.

As evidenced by the miss distances greater than 1 m for both pursuers in Figure 13a,b, the HSUAV employing the proposed SCPL-TD3 strategy successfully evaded capture in this demanding scenario. In stark contrast, the other three strategies all failed, as indicated by their miss distances falling below the 1 m threshold in Figure 14a, Figure 15a, and Figure 16a, respectively.

A detailed analysis of the overload commands reveals the underlying reasons for these outcomes. The constant-value maneuver, maintaining a constant positive maximum overload as shown in Figure 14b, successfully evaded Pursuer 1, while also resulting in the largest lateral displacement among all strategies and ultimately failing to evade Pursuer 2. The Bang-Bang maneuver, depicted in Figure 15b, switched the sign of its overload after evading Pursuer 1, which reduced the lateral maneuvering distance compared to the constant-value strategy. However, this switch occurred too late, missing the critical window to effectively respond to Pursuer 2 and ultimately resulting in capture. As shown in Figure 16b, the M1 method evaded Pursuer 1 but then failed against Pursuer 2 despite adjustments to the overload’s magnitude and direction, surviving only 0.2 s longer than the Bang-Bang maneuver.

In contrast, the proposed method demonstrated intelligent and anticipatory decision-making. The overload command and ballistic deflection angle curves in Figure 13c,d show that the HSUAV initially executed a negative overload maneuver for approximately 0.3 s, followed by a zero-overload phase until 0.4 s, before initiating a gradual positive maneuver. This sequence was designed to compress the reaction time of the pursuers strategically. By considering the threat from both pursuers in an integrated manner from the outset, the strategy successfully created a sufficient evasion window against Pursuer 2 while handling Pursuer 1, thereby achieving successful evasion against the coordinated pair.

This comparative analysis under challenging conditions conclusively demonstrates that the SCPL-TD3 strategy is uniquely capable of handling complex, multi-threat scenarios effectively, attributable to its robustness, energy efficiency, and trajectory optimization that collectively enhance mission survivability and success.

4.2. Robustness Verification

To rigorously evaluate the robustness and generalizability of the proposed intelligent evasion strategy, extensive Monte Carlo simulations were conducted under the three engagement scenarios defined previously in Section 2.1. The initial position of pursuer 1 was set to $\left(x_{I_{1}0},y_{I_{1}0},z_{I_{1}0}\right)=\left(4,25,0\right) \mathrm{km}$.

A total of 5000 Monte Carlo simulations were conducted to validate the robustness of the proposed strategy. The simulations encompassed three distinct scenarios, categorized by increasing difficulty: simple cases (Scenarios 1 and 2) and a difficult case (Scenario 3). Specifically, Scenario 1 (1000 runs) was defined with $\Delta X\in \left[500,750\right)$ m, while Scenario 2 (1000 runs) with $\Delta X\in \left(2000,3000\right]$ m. Scenario 3 (3000 runs) was specifically designed to evaluate the policy’s generalization capability by introducing an untrained initial LOS angle perturbation, uniformly distributed between −10° and 10°, in addition to the $\Delta X\in \left[750,2000\right]$ m.

The corresponding simulation results are presented in Figure 17. A joint analysis of Figure 17a,c confirms the exceptional reliability of the strategy under nominal conditions (Scenarios 1 and 2). The scatter plot in Figure 17a demonstrates that for all 2000 runs in these scenarios, the resulting miss distances are consistently above the 1-m. This visual evidence of perfect success is further corroborated by the corresponding boxplots in Figure 17c. The boxes are compact and located high on the axis, indicating that the strategy produces near-optimal and highly predictable evasion outcomes in both Scenarios 1 and 2 head-on engagements, achieving a 100% success rate.

To evaluate the generalization capability, we further introduced untrained LOS angle perturbations into the setup of Scenario 3. The results are shown in Figure 17b–d. As shown in Figure 17b, the strategy maintains a high evasion success rate of 95.07% even in the presence of untrained LOS angle perturbations. Figure 17c further quantifies the performance distribution: the median miss distance is 1.62 m, with an interquartile range (IQR) of 1.37–1.90 m. Although variability increases compared to nominal scenarios, the results remain within an effective range. Moreover, Figure 17d shows a high median cumulative reward of 1466.99 with a concentrated distribution, indicating that the policy consistently makes high-quality decisions even when confronted with unseen state configurations, highlighting its strong generalization capability and intelligent adaptability.

In summary, the proposed method achieves an aggregate success rate of 97.04% across the entire set of tests, including a deliberately challenging scenario designed to test generalization. This result demonstrates the method’s strong robustness and generalization capability.

4.3. Real-Time Performance Validation

Ensuring real-time performance is a critical requirement for the practical implementation of algorithms in engineering applications. To assess this aspect, the parameter settings in this section were kept identical to those in Section 4.2. A total of 5000 Monte Carlo simulations were conducted to compare the real-time performance of the SCPL-TD3 algorithm with that of the algorithm presented in the Ref. [9]. The simulation results are provided in Figure 18.

The results indicate that the SCPL-TD3 algorithm exhibits significantly improved real-time performance compared to the algorithm from the Ref. [9]. Specifically, the average command generation time is reduced from 0.0169 ms to 0.0157 ms, representing a 7.10% improvement. Furthermore, the optimal command generation time is reduced from 0.0101 ms to 0.0086 ms, corresponding to a 14.85% improvement. This acceleration in command generation delivers critical tactical advantages for HSUAVs operating in real-time confrontation scenarios. This enhancement translates directly into two key system-level benefits:

Firstly, it secures a decisive decision-making advantage. Given the extreme closing velocities (exceeding Mach 10) that define the engagement envelope, the achieved reduction in computational latency allows the evasion strategy to be executed earlier. This head-start is paramount, enabling the HSUAV to initiate and commit to optimal maneuvers that accumulate vital lateral displacement, thereby dramatically increasing the probability of successful penetration against advanced threats.

Secondly, the improved computational efficiency alleviates the burden on the onboard processing unit. The freed computational resources can be reallocated to other critical functions—such as high-fidelity sensor processing, resilient communication links, or more complex predictive models—enhancing overall system capability and robustness without the need for hardware upgrades.

Consequently, within the operational context of high-speed autonomous systems where milliseconds determine mission outcomes, this advancement in processing speed is not just an incremental improvement but a necessary enabler for practical and effective intelligent decision-making.

The proposed SCPL-TD3 algorithm has been rigorously validated across multiple critical dimensions, demonstrating significant advantages:

(1)

The proposed SCPL-TD3 algorithm demonstrates superior training efficiency, converging significantly faster than TD3 and achieving higher performance than PPO by effectively avoiding premature convergence.

(2)

In evasion effectiveness tests, SCPL-TD3 achieves successful evasion in complex scenarios where all baseline methods fail, while its energy-saving trajectory optimization directly enhances mission success probability by preserving critical kinetic energy for follow-on tasks.

(3)

Comprehensive Monte Carlo simulations validate the algorithm’s robustness, showing a 97.04% success rate under significant initial condition perturbations.

(4)

The real-time performance validation confirms the method’s practical deployability, generating intelligent evasion commands within stringent computational constraints.

5. Conclusions and Future Work

This study addresses the limitations of existing evasion strategies by proposing a comprehensive framework for HSUAV maneuvering in coordinated PE scenarios. An optimal interval range for initial spatial separation is derived, and the evasion difficulty classification scheme is introduced to categorize evasion scenarios, facilitating effective strategy optimization. A reward function that integrates energy constraints and critical miss distance is designed to minimize evasion cost while maintaining an operational margin. Additionally, the proposed SCPL-TD3 algorithm dynamically adjusts training complexity and employs semantic patterns to prioritize critical state-action relationships, significantly enhancing learning efficiency and stability. Simulation results demonstrate that the SCPL-TD3 algorithm achieves efficient convergence and robust performance, with a 97.04% success rate and a 7.10–14.85% improvement in decision-making speed, confirming its effectiveness in complex pursuit-evasion scenarios.

Despite these achievements, this study has several limitations. The evaluation is confined to a two-pursuer configuration, leaving the performance in scenarios with more pursuers or heterogeneous adversary teams unexamined. Furthermore, practical factors such as environmental disturbances and communication delays were not incorporated into the current model. Future research will focus on the following aspects: (1) extending the approach to support cooperative evasion among multiple HSUAVs under communication constraints; (2) incorporating robust perception and control strategies to accommodate sensor noise and state uncertainty; (3) validating the algorithm through hardware-in-the-loop simulation to verify performance under realistic conditions.