A Deep Reinforcement Learning Method with a Low Intercept Probability in a Netted Synthetic Aperture Radar

Abstract

A deep reinforcement learning (DRL)-based power allocation method is proposed to achieve a low probability of intercept (LPI) in a netted synthetic aperture radar (SAR). To provide a physically meaningful and intuitive assessment of a netted radar for LPI performance, a netted circular equivalent vulnerable radius (NCEVR) is proposed and adopted. For SAR detection performance, the resolution, signal-to-noise ratio in a single pulse, and signal-to-noise ratio in SAR imaging are integrated at the task level. The LPI performance is achieved by minimizing NCEVR within the constraints of SAR detection performance. The powers in multiple moments are optimized using the DRL proximal policy optimization algorithm with the designed reward and observation. A DRL-based solver is provided for LPI radar, which handles problems that are difficult to optimize using traditional methods. The effectiveness is verified by simulations.

1. Introduction

A synthetic aperture radar (SAR) is an all-weather, high-resolution imaging system, which is widely used in military reconnaissance applications [1,2]. With the increasing complexity of the battlefield environment and the development of electronic technology, the survival of a radar in combat is seriously threatened [3]. To ensure survivability, it is necessary to develop a SAR with low probability of intercept (LPI). The LPI radar technology has been greatly developed since it was proposed [4]. A netted radar system consisting of several multiple input multiple output (MIMO) radars has more robust detection and LPI performance [5,6,7,8,9,10,11,12]. In this paper, a power allocation optimization method of a netted MIMO radar is proposed, aiming to implement LPI in SARs.

The resource of the radar is optimized by LPI techniques, resulting in good detection performance and LPI performance. Currently, the performance metric of LPI for a netted radar is still an interesting issue. Existing LPI metrics for a netted radar mainly include interception factor, interception probability, and radiated power. The interception factor for a netted radar is proposed in [13], which equates the netted radar to a single radar without considering the distribution of the radars, thus calculating the equivalent interception factor. The equivalent interception factor is minimized under the condition of the mutual information for target parameter estimation, achieving LPI by adjusting the powers of the netted radar. The interception factor of the netted radar is also used as the LPI metric in [14], which is minimized by optimizing the power allocation and target assignment under the constraint of a minimum mean square error. As touching the interception probability for a netted radar, several papers achieve LPI by minimizing it [6,15,16,17]. For instance, Ref. [6] minimizes the interception probability of the radar system by optimizing the revisit interval, dwell time, and transmit power under the constraint of target tracking performance. Similarly, multiple approaches are proposed in target search and target tracking scenarios [15,16,17], which minimize the interception probability for a netted radar under the constraints of detection performance, thus enhancing the LPI performance. However, the interception probability for a netted radar is a function of the interceptor, which usually requires the known parameters and locations of the interceptors. It is not easy to obtain in practice. The radiated power for a netted radar is used as the LPI performance metric in several papers [18,19,20,21,22]. The system’s radiated energy is considered an objective function to be minimized by optimizing the node selection, dwell time, bandwidth, and power while satisfying the constraint of target tracking accuracy [18,19]. The LPI performance of the radar system is enhanced while maintaining a certain level of target tracking performance. The radiated power of the radar system is considered a constraint in [20,21,22]. The target tracking performance is improved by optimizing the node selection, dwell time, power, and bandwidth, which realizes the LPI performance by the certain radar system power. The probability of being intercepted by an enemy reconnaissance aircraft reduces with a reduction in the radiated power due to the reduction in the propagation strength of the radar signal in space. But the LPI performance is considered indirectly; it is difficult for radiated power to achieve a quantitative and intuitive assessment of LPI. Circular equivalent vulnerable radius (CEVR) is proposed as an LPI metric for a single radar, which is calculated by the easily intercepted area [23]. It is an intuitive LPI metric, but the complexity of the calculation results in few applications. The LPI metric for a netted radar is still an interesting concern. Hence, the CEVR is extended to netted radars in this paper.

Existing LPI methods can be classified into two kinds: maximizing the current detection performance while satisfying the condition of the current LPI performance, and minimizing the current LPI performance while satisfying the condition of the current detection performance. The long-term performance at the task level has to be considered in some scenarios. In SAR, the result is imaged based on data acquired over the entire aperture time. It is necessary to consider the detection performance over the mission period as a whole. Similarly, for LPI performance, one should only consider the effect of the current action on the current situation, which does not optimize the performance over the entire task period. It can not guarantee the performance of the whole task. Therefore, it is necessary to consider the long-term performance of the whole task to ensure the overall optimization of the entire mission for LPI in SAR. Due to scheduling during the entire task in multiple moments, traditional optimization methods may no longer be applicable, which poses a challenge to the solver.

With the development of artificial intelligence technology, deep reinforcement learning (DRL) has become a novel solver for radar resource scheduling in recent years [24]. The optimal action policy is optimized using a data-driven approach, which utilizes the accumulated interaction data between the agent and the environment [25]. DRL not only considers the scheduling for the current moment but also the scheduling over the entire task period for multiple moments. At the same time, it has the characteristic of adaptability, which avoids the complex solution process of traditional optimization methods. In the single-target tracking scenario, a DRL-based resource allocation method is proposed to achieve better long-term and short-term tracking performance, which optimizes node selection and power allocation using a deep deterministic policy gradient (DDPG) method [26]. For multi-target tracking, the DRL method is also adopted to schedule the beam selection and transmit power, resulting in better multi-target tracking performance [27,28]. In the air combat scenario, to ensure the survival and the detection performance of aircraft, a DQN-based reinforcement learning method is employed to achieve real-time scheduling of sensor resources in a dynamic environment, which considers the threat level of the target, the accumulated power radiation, and the acquisition of detection information [29]. In the multi-target detection scenario, a domain knowledge-assisted DRL framework was proposed to achieve better target detection probability by allocating the power of netted MIMO radar, which utilizes the generated timely reward and final task reward to train the policy network [30]. In the integration of target detection and communication scenario, DRL is used to optimize massive MIMO systems to improve the probability of detecting weak targets in strong clutter environments by controlling the number of targets to be focused [31]. In the LPI radar scenario, Ref. [32] considered the transmit signal power and the cumulative signal-to-noise ratio (SNR) intercepted by the electronic intelligence (ELINT) system by optimizing the transmit signal power and the modulation strategy and realizing the LPI in SAR imaging using the reinforcement learning Q-learning algorithm. The LPI performance is considered by the long-term cumulative SNR intercepted by ELINT, and the SAR imaging performance is considered by the current power at each moment without long-term performance. Reinforcement learning-based resource scheduling methods have been applied in multiple scenarios; however, the research in SAR with LPI is not sufficient.

Deep reinforcement learning-based scheduling methods achieve better long-term performance through data-driven approaches, and they can be applied to problems that are difficult to optimize for traditional methods. In this paper, a reinforcement learning-based resource scheduling method is proposed to reduce the probability of a netted MIMO radar being intercepted in SAR detection. For the LPI performance, based on CEVR [23], we propose and adopt the netted circular equivalent vulnerable radius (NCEVR) to provide a physically meaningful and intuitive assessment of a netted radar. For the SAR detection performance, multiple factors of the resolution, single-pulse SNR, and SNR in SAR imaging are considered at the level of the entire long-term imaging task. The LPI performance is achieved by optimizing the power at multiple moments in the entire synthetic aperture time by minimizing the cumulative NCEVR while maintaining the SAR detection performance. The innovation of this paper is as follows: 1. A reinforcement learning-based LPI method for a netted radar is proposed in SAR detection, which is achieved by the joint control of power at multiple moments in aperture time considered at the entire task level. 2. A netted radar LPI metric, NCEVR, is proposed by considering the interactions of multiple radars in areas susceptible to interception. It achieves a physically meaningful and intuitive assessment of LPI performance for any radar distribution and number. 3. Due to the complex computation of NCEVR and the joint scheduling on multiple moments for the performance, it is difficult to solve for traditional methods. This paper provides a novel solver for the intractable problems in LPI radar scenarios.

The rest of this paper is organized as follows. In Section 2, the performance characterizations of SAR imaging and LPI are introduced, and the optimization model of NCEVR in netted radar SAR imaging is described. In Section 3, the observation, action, reward, and the proposed algorithm are designed, and the applicability of the algorithm to the optimization model is described. The proposed method is tested, and the simulations are presented in Section 4. The conclusions are presented in Section 5.

2. Problem Formulation

A netted MIMO radar is considered, which is used to achieve LPI for SAR in spotlight mode. In this section, the performance characterizations of low interception and SAR detection are presented separately. And the optimization model of the netted LPI for a SAR is established.

2.1. Performance Characterization of SAR Detection

In this subsection, the performance characterization of a SAR is introduced, which includes the single-pulse SNR, the SNR in SAR imaging, and the range and azimuth resolution.

2.1.1. Single-Pulse SNR

For a MIMO radar, according to the radar equation [33], the single-pulse SNR can be described as follows:

$$SN{R}_{\mathrm{P}m,n,t}={\displaystyle \frac{P_{\mathrm{T}m,t}{G}_{\mathrm{T}}{G}_{\mathrm{R}}{\lambda }^2\sigma G_{\mathrm{RP}}}{{\left(4\pi \right)}^3{R}_{m,n,t}^{4}K{T}_{\mathrm{R}}{F}_{\mathrm{R}}{B}_{\mathrm{R}}}},$$

(1)

where m, n, and t are the number of radar, target, and time, respectively; for example, $SN{R}_{\mathrm{P}m,n,t}$ is the SNR for a single pulse for radar m to target n at time t. $P_{\mathrm{T}m,t}$ is the transmitting pulse power of the radar m at time t. $G_{\mathrm{T}}$ and $G_{\mathrm{R}}$ are the gains of the transmitting and receiving antennas for all radars. $\lambda$ is the signal wavelength. $\sigma$ is the radar cross-section (RCS) of the target, which is a reflectivity number that normalizes per unit area [34]. $G_{\mathrm{RP}}$ is the pulse compression gain attributed to the range processing. $R_{m,n,t}$ is the distance between the radar m and the target n at time t. K is the Boltzmann constant. $T_{\mathrm{R}}$, $F_{\mathrm{R}}$, and $B_{\mathrm{R}}$ represent absolute temperature, noise figure, and bandwidth, respectively, for all radar receivers. The single-pulse SNR is a function of transmit power and distance when the radar system and the target are fixed.

2.1.2. SNR in SAR Imaging

A SAR is a high-resolution imaging radar technology that utilizes the echoes reflected by the target during the aperture time. The SNR in SAR imaging is a key performance measure that is critical for obtaining high-quality images [34,35]. The imaging process can be regarded as a two-dimensional matched filtering operation in both range and azimuth dimensions [36]. A good SNR improves signal coherence and suppresses noise, leading to the features of the targets not being lost in the noise, and thus making detection more accurate and reliable [37]. Conversely, a low SNR leads to blurry shapes of the structures and mixed pixels after filtering [38,39,40]. Thus, the SNR in SAR imaging is a fundamental performance metric that critically determines image quality. For the single-pulse SNR fixed at each moment, the SNR in SAR imaging of target n for radar m can be described as [34]:

$$\begin{array}{cc}\hfill SN{R}_{m,n}& =SN{R}_{\mathrm{P}m,n,{\mathrm{t}}_0}{G}_{\mathrm{A}}\hfill \\ \hfill & =SN{R}_{\mathrm{P}m,n,{\mathrm{t}}_0}{N}_0\hfill \end{array}$$

(2)

where $SN{R}_{\mathrm{P}m,n,{\mathrm{t}}_0}$ is used to represent the SNR over the entire measurement line. $G_{\mathrm{A}}$ is the azimuth process gain due to $N_0$ coherent pulse integrals. For a more general scenario, when the single-pulse SNR varies, $SN{R}_{m,n}$ is modified as follows:

$$SN{R}_{m,n}=\sum _{t=1}^{T}SN{R}_{\mathrm{P}m,n,t}{N}_t$$

(3)

where T is the overall number of time steps for SAR imaging. $N_t$ is the pulse number at step t. For the netted radar, the integrated SNR after coherent integration of M radars can be expressed as follows:

$$SN{R}_n=\sum _{m=1}^{M}SN{R}_{m,n}$$

(4)

To ensure the SNR requirements are met, the system can be characterized as follows:

$$SN{R}_n\ge SN{R}_{\mathrm{th}}$$

(5)

where $SN{R}_{\mathrm{th}}$ is the threshold in SAR imaging.

2.1.3. Range and Azimuth Resolution

The resolution of SAR is an important parameter of imaging capability, which affects the clarity and detail of SAR imaging. The resolution is usually divided into range resolution and azimuth resolution. The range resolution ${\rho }_{\mathrm{R}}$ is a function with respect to the radar bandwidth $B_{\mathrm{R}}$, which can be described as follows [33]:

$${\rho }_{\mathrm{R}}={\displaystyle \frac{c}{2B_{\mathrm{R}}}}$$

(6)

where c is the speed of light. The azimuth resolution of SAR in spotlight mode can be described as follows [41]:

$${\rho }_{\mathrm{A}}={\displaystyle \frac{\lambda }{4sin(\Delta \theta /2)cos{\theta }_c}}.$$

(7)

where ${\rho }_{\mathrm{A}}$ is the azimuth resolution. $\Delta \theta$ is the beam rotation angle during synthetic aperture time. ${\theta }_c$ is the beam center oblique angle. For a given target, the azimuth resolution depends on the beam angle of the start time and the end time.

In order to obtain clear images, the resolution is usually required to be better than the threshold, which can be described as follows:

$${\rho }_{\mathrm{R}}\le {\rho }_{\mathrm{Rth}}$$

(8)

$${\rho }_{\mathrm{A}}\le {\rho }_{\mathrm{Ath}}$$

(9)

Usually, the PRF of the SAR should be greater than a threshold to avoid aliasing in the azimuthal direction. The PRF can be obtained as follows [34]:

$$f_{\mathrm{P}}={\displaystyle \frac{2kv}{D}}.$$

(10)

where k is a constant factor; typically, $k\ge 1.5$. v is the platform velocity in the horizontal and orthogonal to the target direction. D is the physical aperture dimension of the antenna.

2.2. Performance Characterization of LPI Radar

With the increasing complex electromagnetic threats, netted radar is being used in low-intercept radar systems to achieve better performance, while at the same time, the characterization of LPI in netted radars is still a topic of interest. In this paper, a new low-intercept performance metric called NCEVR is proposed for netted radars based on CEVR [23]. It achieves the evaluation of LPI performance for any radar distribution and number, which can be described as follows:

$$\begin{array}{c}NCEV{R}_t=\left(Area\left[\left({\displaystyle \frac{P_{\mathrm{I}1,t}}{N_{\mathrm{I}}}}>SN{R}_{\mathrm{Ith}}\right)\right.\right.\mathrm{or}\hfill \\ {\left.\left.\left({\displaystyle \frac{P_{\mathrm{I}2,t}}{N_{\mathrm{I}}}}>SN{R}_{\mathrm{Ith}}\right)\mathrm{or}\cdots \mathrm{or}\left({\displaystyle \frac{P_{\mathrm{I}M,t}}{N_{\mathrm{I}}}}>SN{R}_{\mathrm{Ith}}\right)\right]/\pi \right)}^{{\displaystyle \frac{1}{2}}}\hfill \end{array}$$

(11)

where $NCEV{R}_t$ is the netted circular equivalent vulnerable radius at time t. $P_{\mathrm{I}m,t}$ is the power of radar m received by the interceptor at time t. $N_{\mathrm{I}}$ is the noise power of the interceptor. $SN{R}_{\mathrm{Ith}}$ is the required SNR of the interceptor satisfying a certain detection probability $p_{\mathrm{D}}$ and a certain false alert probability $p_{\mathrm{F}}$. Function $Area$ represents the area that satisfies the condition. It can be seen that the netted radar vulnerability area is calculated, and then the equivalent radius is calculated based on the area. Specifically, according to the radar equation, $P_{\mathrm{I}m,t}$ can be described as follows:

$$P_{\mathrm{I}m,t}={\displaystyle \frac{P_{\mathrm{A}m,t}}{\left(4\pi \right)R_{\mathrm{I}n}^2}}={\displaystyle \frac{P_{\mathrm{T}m,t}{T}_{\mathrm{E}}{f}_{\mathrm{P}}}{\left(4\pi \right)R_{\mathrm{I}n}^2}}$$

(12)

where $P_{\mathrm{A}m,t}=P_{\mathrm{T}m,t}{T}_{\mathrm{E}}{f}_{\mathrm{P}}$ is the average transmitting power of radar m at time t. $T_{\mathrm{E}}$ is the effective pulse width, and $f_{\mathrm{P}}$ is the pulse repetition frequency. $R_{\mathrm{I}m}$ is the distance between radar m and the interceptor. $N_{\mathrm{I}}$ can be described as follows:

$$N_{\mathrm{I}}=K{T}_{\mathrm{I}}{F}_{\mathrm{I}}{B}_{\mathrm{I}}$$

(13)

where $T_{\mathrm{I}}$, $F_{\mathrm{I}}$, and $B_{\mathrm{I}}$ are the absolute temperature, the noise figure, and the bandwidth of the interceptor receiver, respectively. The relationship between $SN{R}_{\mathrm{Ith}}$, $p_{\mathrm{D}}$, and $p_{\mathrm{F}}$ can be described as follows [16]:

$$p_{\mathrm{D}}={\displaystyle \frac{1}{2}}erfc\left(\sqrt{-ln{p}_{\mathrm{F}}}-\sqrt{SN{R}_{\mathrm{Ith}}+0.5}\right)$$

(14)

It should be noted that NCEVR is not a simple summation of CEVR but considers multiple radars in the vulnerability areas. It takes into account the interactions of multiple radars on the vulnerability areas, providing a physically meaningful and intuitive evaluation of LPI for a netted radar. However, the need to identify and compute the vulnerability areas when calculating NCEVR also poses significant challenges for traditional optimization methods.

2.3. Optimization Model of NCEVR in Netted Radar SAR Imaging

The coherent combining echoes from multiple pulses increased the SNR in the SAR image, but it also increased the probability of being intercepted. Therefore, it is necessary to consider low interception performance and SAR detection performance comprehensively. An optimization method is used to obtain the transmit power at each moment, minimizing the probability of interception at a certain SAR detection performance. In our paper, only signals with a single-pulse SNR greater than the threshold are considered to be echo signals, and the indicator function $I_{m,n,t}$ can be expressed as follows:

$$I_{m,n,t}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}SN{R}_{\mathrm{P}m,n,t}\ge SN{R}_{\mathrm{Pth}}\hfill \\ 0,\hfill & \mathrm{else}\hfill \end{array}\right.,$$

(15)

where $SN{R}_{\mathrm{Pth}}$ is the single-pulse SNR threshold. According to (3), (4), and (15), the SNR in SAR imaging for target n until aperture time T can be modified as follows:

$$SN{R}_n=\sum _{m=1}^M\sum _{t=1}^{T}SN{R}_{\mathrm{P}m,n,t}{I}_{m,n,t}{N}_t$$

(16)

In the absence of the azimuthal direction aliasing, the resolution is obtained from the beam rotation angle and the beam center oblique angle at a certain PRF during the synthetic aperture time. However, in order to minimize the $NCEV{R}_t$, the proposed LPI netted radar system adjusts the transmit strategy at each moment, which can not guarantee the continuity of the aperture time. For example, the radar only transmits to the target at the start position and the end position, whereas it does not transmit in the span. It causes a very high sidelobe due to the lack of acquisition at the positions between the start and end. Therefore, we calculate the resolution using the beam rotation angle and the beam center oblique angle during continuous acquisition, thus satisfying the resolution without azimuthal aliasing. For a known radar, the specific process is shown in Algorithm 1. In summary, the data that satisfy the single pulse SNR are used for SAR imaging. Then, by judging whether the data acquisition is consecutive or not, the start and end positions of consecutive acquisitions are updated, and the corresponding resolution is calculated and recorded, respectively. Finally, the resolution of each target during the whole task time is calculated.

Algorithm 1 Calculate the azimuth resolution of LPI netted radar for SAR

Input: The positions $Po{s}_{m,t}$ of the M radars in time T. The positions of the N targets. The single-pulse signal-to-noise ratio $SN{R}_{\mathrm{p}m,n,t}$ from M radars to N targets in time T. Output: Azimuth resolution ${\rho }_{\mathrm{A}n}$ of each target for the whole task time.   1: Initialize the record of azimuth resolution ${\mathbf{Rec}}_{m,n,t}$ with element ∞.   2: for $t=1toT$do   3:      for $m=1toM$ do   4:          for $n=1toN$ do   5:               if $(SN{R}_{\mathrm{P}m,n,t}\ge SN{R}_{\mathrm{Pth}}\mathrm{and}SN{R}_{\mathrm{P}m,n,t+1}\le SN{R}_{\mathrm{Pth}})\mathrm{or}t=1$ then   6:                   Obtain the start position during continuous acquisition $Po{s}_{\mathrm{start}}=Po{s}_{m,t}$. Obtain the end position during continuous acquisition $Po{s}_{\mathrm{end}}=Po{s}_{m,t+1}$.   7:                   Calculate the the beam rotation angle $\Delta \theta$ and beam center oblique angle ${\theta }_c$. Calculate the azimuth resolution ${\rho }_{\mathrm{A}m,n,t}$ from radar m to target n at time t, and store in ${\mathbf{Rec}}_{m,n,t}$.   8:               end if   9:               if $SN{R}_{\mathrm{P}m,n,t}\ge SN{R}_{\mathrm{Pth}}\mathrm{and}SN{R}_{\mathrm{P}m,n,t+1}\ge SN{R}_{\mathrm{Pth}}$ then 10:                   Update the end position during continuous acquisition $Po{s}_{\mathrm{end}}=Po{s}_{m,t+1}$. 11:                   Calculate the the beam rotation angle $\Delta \theta$ and beam center oblique angle ${\theta }_c$. Calculate the azimuth resolution ${\rho }_{\mathrm{A}m,n,t}$ from radar m to target n at time t, and store in ${\mathbf{Rec}}_{m,n,t}$. 12:               end if 13:          end for 14:      end for 15: end for 16: Calculate the azimuth resolution of target n at time t; ${\rho }_{\mathrm{A}n,t}={min}_m{\mathbf{Rec}}_{m,n,t}$. Calculate the azimuth resolution of target n for the whole task time; ${\rho }_{\mathrm{A}n}={min}_t{\rho }_{\mathrm{A}n,t}={min}_{m,t}{\mathbf{Rec}}_{m,n,t}$.

In this paper, the cumulative NCEVR in the whole task time is minimized under the conditions of a single-pulse SNR, an SNR in SAR imaging, and the range and azimuth resolution, thus achieving LPI for the netted radar. It should be noted that the single-pulse SNR is included in the calculation of $SN{R}_{\mathrm{I}n}$ in (16) and the calculation of ${\rho }_{\mathrm{A}n}$ in Algorithm 1. The optimization model can be described as follows:

$$\begin{array}{cc}\hfill \underset{P_{\mathrm{T}m,t}}{minimize}& \sum _{t=1}^{T}NCEV{R}_t\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& SN{R}_n\ge SN{R}_{\mathrm{th}},\forall n\hfill \\ \hfill & {\rho }_{\mathrm{A}n}\le {\rho }_{\mathrm{Ath}},\forall n\hfill \\ \hfill & {\rho }_{\mathrm{R}n}\le {\rho }_{\mathrm{Rth}},\forall n\hfill \\ \hfill & P_{\mathrm{A}m,t}\le P_{\max },\forall m,t\hfill \\ \hfill & P_{\mathrm{A}m,t}\ge P_{\min },\forall m,t\hfill \end{array},$$

(17)

where $P_{\max }$ and $P_{\min }$ are the upper and lower limits of the average transmitting power $P_{\mathrm{A}m,t}$, respectively.

3. Data-Driven Resource Allocation Policy Method

In this paper, a deep reinforcement learning method, proximal policy optimization (PPO) [42] is adopted to solve the optimization model. The proposed specific algorithm based on PPO and the applicability to the optimization model are described separately.

3.1. Algorithm

The observation, action, and reward of the agent are presented separately. For a certain SAR, the observation at time step t is defined as follows:

$${\mathbf{obs}}_t=\left[{\mathbf{Pos}}_t,{\mathbf{CR}}_{\mathrm{A}t},{\mathbf{CR}}_{\mathrm{E}t}\right]$$

(18)

where ${\mathbf{Pos}}_t$ is the M radar positions at time t; specifically,

$${\mathbf{Pos}}_t=\left[Po{s}_{1,t},\cdots ,Po{s}_{M,t}\right]$$

(19)

${\mathbf{CR}}_{\mathrm{A}t}$ is the completion rate of the azimuth resolution for N targets at time t; specifically,

$${\mathbf{CR}}_{\mathrm{A}t}=\left[min\left({\displaystyle \frac{{\rho }_{\mathrm{Ath}}}{{\rho }_{\mathrm{A}1,t}}},1\right),\dots ,min\left({\displaystyle \frac{{\rho }_{\mathrm{Ath}}}{{\rho }_{\mathrm{A}N,t}}},1\right)\right]$$

(20)

The highest completion rate is 1. ${\rho }_{\mathrm{A}n,t}$ can be obtained as shown in line 16 of Algorithm 1. ${\mathbf{CR}}_{\mathrm{S}t}$ is the completion rate of the SNR in imaging for N targets at time t; specifically,

$${\mathbf{CR}}_{\mathrm{S}t}=\left[min\left({\displaystyle \frac{SN{R}_{1,t}}{SN{R}_{\mathrm{th}}}},1\right),\cdots ,min\left({\displaystyle \frac{SN{R}_{N,t}}{SN{R}_{\mathrm{th}}}},1\right)\right]$$

(21)

Element $SN{R}_{n,t}$ is the SNR in SAR imaging of target n until time t for the netted radar. Similar to the SNR until aperture time T as shown in (16), $SN{R}_{n,t}$ can be described as follows:

$$SN{R}_{n,t}=\sum _{m=1}^M\sum _{i=1}^{t}SN{R}_{\mathrm{P}m,n,i}{I}_{m,n,i}{N}_i$$

(22)

The observation contains information on the positions and the completion rates. The radar adjusts the transmitting power according to the observation.

The action at time step t is defined as follows:

$${\mathbf{act}}_t=\left[P_{\mathrm{T}1,t},\dots ,P_{\mathrm{T}M,t}\right]$$

(23)

It is the optimization variables that control the transmitting pulse power of M radars to achieve an LPI netted radar in SAR detection.

The reward is the core component of deep reinforcement learning, which is used to guide the agent to achieve the optimal strategy. In order to achieve a better strategy, the total reward $r_t$ includes the primary reward $r_{\mathrm{P}t}$ and auxiliary reward $r_{\mathrm{A}t}$. In our method, it can be described as follows:

$$r_t=r_{\mathrm{P}t}+r_{\mathrm{A}t}$$

(24)

Specifically, $r_{\mathrm{P}t}$ can be described as follows:

$$r_{\mathrm{P}t}=-{\alpha }_{1}NCEV{R}_t+{\alpha }_2{r}_{\mathrm{cond}1t}+{\alpha }_3{r}_{\mathrm{cond}2t}$$

(25)

where ${\alpha }_n$ is the coefficient constant of the corresponding reward. $-NCEV{R}_t$ is the reward to realize the LPI performance by maximizing the cumulative rewards in reinforcement learning, which corresponds to ${\sum }_{t=1}^{T}NCEV{R}_t$. $r_{\mathrm{cond}1t}$ is the reward for the azimuth resolution constraint, which is defined as follows:

$$r_{\mathrm{cond}1t}=\left\{\begin{array}{cc}5,\hfill & \mathrm{if}t=\mathrm{T}\mathrm{and}\forall n,{\rho }_{\mathrm{A}n}\le {\rho }_{\mathrm{Ath}}\hfill \\ -5,\hfill & \mathrm{if}t=\mathrm{T}\mathrm{and}\exists n,{\rho }_{\mathrm{A}n}>{\rho }_{\mathrm{Ath}}\hfill \\ 0,\hfill & \mathrm{else}\hfill \end{array}\right.$$

(26)

$r_{\mathrm{cond}2t}$ is the reward for the SNR in SAR imaging, which is defined as follows:

$$r_{\mathrm{cond}2t}=\left\{\begin{array}{cc}5,\hfill & \mathrm{if}t=\mathrm{T}\mathrm{and}\forall n,SN{R}_n\ge SN{R}_{\mathrm{th}}\hfill \\ -5,\hfill & \mathrm{if}t=\mathrm{T}\mathrm{and}\exists n,SN{R}_n<SN{R}_{\mathrm{th}}\hfill \\ 0,\hfill & \mathrm{else}\hfill \end{array}\right.$$

(27)

To maximize returns, the strategy tends to satisfy the constraints with training.

For the strategy converging better, the auxiliary reward $r_{\mathrm{A}t}$ is considered, which can be defined as follows:

$$r_{\mathrm{A}t}={\alpha }_4{\Delta }_{\mathrm{PA}t}+{\alpha }_5{\Delta }_{\mathrm{PS}t}+{\alpha }_6{\Delta }_{\mathrm{NA}t}+{\alpha }_7{\Delta }_{\mathrm{NS}t}$$

(28)

where ${\Delta }_{\mathrm{PA}t}$ and ${\Delta }_{\mathrm{PS}t}$ are the improving completion rate of the azimuth resolution and SNR in imaging for the targets at time t. ${\Delta }_{\mathrm{NA}t}$ and ${\Delta }_{\mathrm{NS}t}$ are the improved satisfied number of the azimuth resolution and SNR in imaging for the targets at time t. Specifically, the improving completion rate of the azimuth resolution ${\Delta }_{\mathrm{PA}t}$ is defined as follows:

$${\Delta }_{\mathrm{PA}t}=\sum _{n=1}^{N}C{R}_{\mathrm{A}t,n}-\sum _{n=1}^{N}C{R}_{\mathrm{A}t-1,n}$$

(29)

where $C{R}_{\mathrm{A}t,n}$ is the $n$-th element of ${\mathbf{CR}}_{\mathrm{A}t}$, which is the completion rate of the azimuth resolution for the $n$-th target at time t. ${\sum }_{i=n}^{N}C{R}_{\mathrm{A}t,n}$ is the total completion rate of the azimuth resolution at time t. Thus, ${\Delta }_{\mathrm{PA}t}$ is the improving completion rate compared to the previous moment. Similar to ${\mathbf{CR}}_{\mathrm{A}t}$, the improving completion rate of the SNR in imaging ${\Delta }_{\mathrm{PS}t}$ is defined as follows:

$${\Delta }_{\mathrm{PS}t}=\sum _{n=1}^{N}C{R}_{\mathrm{S}t,n}-\sum _{n=1}^{N}C{R}_{\mathrm{S}t-1,n}$$

(30)

where $C{R}_{\mathrm{S}t,n}$ is the $n$-th element of ${\mathbf{CR}}_{\mathrm{S}t}$. The improving satisfied number of the azimuth resolution ${\Delta }_{\mathrm{NA}t}$ is defined as follows:

$${\Delta }_{\mathrm{NA}t}=\sum _{n=1}^N{1}_{\{C{R}_{\mathrm{A}t,n}=1\}}-\sum _{n=1}^N{1}_{\{C{R}_{\mathrm{A}t-1,n}=1\}}$$

(31)

${\sum }_{i=n}^N{1}_{\{C{R}_{\mathrm{A}t,n}=1\}}$ is the total satisfied number of the azimuth resolution at time t, where the completion rate $C{R}_{\mathrm{A}t,n}=1$. Thus, ${\Delta }_{\mathrm{NA}t}$ is the improvement satisfied number compared to the previous moment. Similar to ${\Delta }_{\mathrm{NA}t}$, the improvement satisfied number of the SNR in imaging ${\Delta }_{\mathrm{NS}t}$ is defined as follows:

$${\Delta }_{\mathrm{NS}t}=\sum _{n=1}^N{1}_{\{C{R}_{\mathrm{S}t,n}=1\}}-\sum _{n=1}^N{1}_{\{C{R}_{\mathrm{S}t-1,n}=1\}}$$

(32)

In our paper, the proximal policy optimization (PPO) method is adopted to realize the LPI netted radar for SAR imaging. The algorithm is shown in Algorithm 2. The policy network $\pi$ is used for controlling radar action. Value network V is used for estimating the cumulative reward. $N_{\mathrm{train}}$ is the training epoch. $N_{\mathrm{update}}$ is the number of stochastic gradient descent (SGD) iterations in each epoch. After training, the radar action in time T can be obtained, realizing LPI in SAR imaging. The optimization results are obtained utilizing the interaction of the agent with the environment through a data-driven approach. The agent learns the optimal strategy through a trial-and-error process. Compared with traditional optimization methods, it can effectively handle complex models and avoid the process that is difficult to solve.

Algorithm 2 LPI netted radar for SAR imaging in PPO

1: Initialize the policy network $\pi$ with parameter $\psi$, initialize the value network V with parameter $\varphi$, and initialize the hyperparameter of PPO.   2: for $n_{\mathrm{train}}=1to{N}_{\mathrm{train}}$do   3:     Initialize the replay buffer D. Initialize the positions of the N targets and the positions $Po{s}_{m,t}$ of the M radars in time T.   4:     for $ttoT$ do   5:         Calculate the observation ${\mathbf{obs}}_t$ according to (18). Calculate action ${\mathbf{act}}_t=\pi \left({\mathbf{obs}}_t\mid \psi \right)$. Calculate the reward $r_t$ according to (24).   6:         if $t<T$ then   7:            Calculate ${\mathbf{obs}}_{t+1}$.   8:         else   9:            Go to step 3. 10:         end if 11:         Store (${\mathbf{obs}}_t$,${\mathbf{act}}_t$,$r_t$,${\mathbf{obs}}_{t+1}$) in replay buffer D. 12:     end for 13:     for $n_{\mathrm{update}}=1to{N}_{\mathrm{update}}$ do 14:         Sample mini-batch of transitions from D. Update parameters $\psi$ and parameters $\varphi$ based on PPO. 15:     end for 16: end for

3.2. Applicability to the Optimization Model

The original optimization problem as shown in (17) can be relaxed to the following unconstrained problem by the Lagrangian relaxation method [43], which can be described as follows:

$$\begin{array}{cc}\hfill \underset{P_{\mathrm{A}m,t}}{minimize}\left[\sum _{t=1}^{T}NCEV{R}_t\right.& +\sum _{n=1}^N{\mu }_n\left(SN{R}_{\mathrm{th}}-SN{R}_n\right)\hfill \\ \hfill & \left.+\sum _{n=1}^N{\beta }_n\left({\rho }_{\mathrm{A}n}-{\rho }_{\mathrm{Ath}}\right)\right]\hfill \end{array}$$

(33)

where ${\mu }_n$ and ${\beta }_n$ are the Lagrange multipliers corresponding to inequality constraints.

Based on the DRL theory, the agent learns the optimal strategy by maximizing the objective function. The objective function $J\left({\pi }_{\theta }\right)$ is the expectation of cumulative discount reward, which can be defined as [44]:

$$J\left({\pi }_{\psi }\right)=E_{\tau \sim {\rho }_{\psi }\left(\tau \right)}\left[\sum _{t=0}^T{\gamma }^t{r}_t\right]$$

(34)

where $\psi$ is the parameters of the policy network $\pi$. $\tau$ is the trajectory observed by the agent. ${\rho }_{\psi }$ is the probability distribution of the trajectory. $\gamma \in [0,1]$ is the reward discount factor, which is used to denote the importance of the current reward for the future. The total reward $r_t$ can be divided into the primary reward $r_{\mathrm{P}t}$ and the auxiliary reward $r_{\mathrm{A}t}$. The potential-based reward shaping is used to design the auxiliary reward $r_{\mathrm{A}t}$, which can accelerate the convergence of the algorithm under the optimal policy invariance [45]. Thus, the objective function of the optimal policy can be simplified to

$$J\left({\pi }_{\psi }\right)=E_{\tau \sim {\rho }_{\psi }\left(\tau \right)}\left[\sum _{t=0}^T{\gamma }^t{r}_{\mathrm{P}t}\right]$$

(35)

According to (25) and (35) can be described as follows:

$$\begin{array}{cc}\hfill J\left({\pi }_{\psi }\right)=E_{\tau \sim {\rho }_{\psi }\left(\tau \right)}[& \sum _{t=0}^T\left(-{\gamma }^t{\alpha }_{1}NCEV{R}_t\right.\hfill \\ \hfill & \left.\left.+{\gamma }^t{\alpha }_2{r}_{\mathrm{cond}1t}+{\gamma }^t{\alpha }_3{r}_{\mathrm{cond}2t}\right)\right]\hfill \\ \hfill =E_{\tau \sim {\rho }_{\psi }\left(\tau \right)}[& (\sum _{t=0}^T-{\gamma }^t{\alpha }_{1}NCEV{R}_t)\hfill \\ \hfill & \left.+{\gamma }^T{\alpha }_2{r}_{\mathrm{cond}1t}+{\gamma }^T{\alpha }_3{r}_{\mathrm{cond}2t}\right]\hfill \end{array}$$

(36)

When $\gamma =1$,maximizing $J\left({\pi }_{\psi }\right)$ is equivalent to minimizing the original problem with Lagrangian relaxation as shown in (33). ${\mu }_n$, ${\beta }_n$, ${\alpha }_n$, and the value of $r_{\mathrm{cond}1t}$ and $r_{\mathrm{cond}2t}$ are coefficients that correspond to each other. For example, (33) can correspond to ${\alpha }_1=1$ in (36). When $SN{R}_1<SN{R}_{\mathrm{th}}$, $r_{\mathrm{cond}2t}=-5{\alpha }_3$; this corresponds to ${\mu }_1$ being equal to a value such that ${\mu }_1\left(SN{R}_{\mathrm{th}}-SN{R}_1\right)=5{\alpha }_3$ and ${\mu }_{n\ne 1}=0$. In summary, the optimal solution of the optimization model as shown in (17) can be solved by the proposed method. However, the algorithm is not easy to converge when $\gamma =1$ [46], which leads to bad results. In practice, $\gamma$ can be set to converge to 1, thus obtaining a suboptimal solution to the original optimization model.

4. Simulation

In this section, the proposed method’s performance is verified by evaluating the simulations performed. The optimization model in (17) can be solved not only by the proposed algorithm based on PPO but also by other intelligent optimization algorithms. The particle swarm optimization algorithm [47] is used for comparison, which optimizes the transmitting power of the netted radar at all times, referred to as PSO-A. Furthermore, a conventional situation with fixed transmitting power across all times is considered and optimized using particle swarm optimization, referred to as PSO-S. The simulation setting and simulation results are introduced in turn.

4.1. Simulation Setting

As shown in Figure 1, an LPI netted radar consisting of 4 movement radars was considered, which realized SAR imaging with a high resolution. The total time of the task was 198 s with 99 time steps. The initial positions of the radar were [10 km, 10 km], [10 km, 18 km], [10 km, 22 km], and [10 km, 30 km], respectively, and the velocities were the same [100 m/s, 0 m/s]. The positions of the 10 targets were [20 km, 0 km], [25 km, 5 km], [15 km, 12 km], [20 km, 14 km], [25 km, 16 km], [20 km, 26 km], [12 km, 40 km], [15 km, 40 km], [27 km, 40 km], and [28 km, 40 km], respectively. The RCS of each target was 1. For the radars, the transmitting antenna gain $G_{\mathrm{T}}$ was 25 dB, and the receiving antenna gain $G_{\mathrm{R}}$ was 25 dB. The signal wavelength $\lambda$ was 0.03 m. The pulse repetition frequency $f_{\mathrm{P}}$ was 2000 Hz, and the pulse width $T_{\mathrm{E}}$ was about 2.1 × ${10}^{-6}$ s. The pulse compression gain $G_{\mathrm{RP}}$ was 30 dB. The absolute temperature of the receiver $T_{\mathrm{r}}$ was 290 K, and the noise figure $F_{\mathrm{r}}$ was 3 dB. The single-pulse SNR threshold $SN{R}_{\mathrm{Pth}}$ was set to −30 dB. The SNR threshold in SAR imaging $SN{R}_{\mathrm{th}}$ was set to 40 dB. The azimuth resolution threshold ${\rho }_{\mathrm{Ath}}$ and range resolution threshold ${\rho }_{\mathrm{Rth}}$ were both 0.1 m. In order to satify the range resolution ${\rho }_{\mathrm{R}}\le 0.1$ m, the bandwidth $B_{\mathrm{R}}$ was set to 1.5 GHz according to (6). The upper limit $P_{\max }$ and lower limit $P_{\min }$ of the average transmitting power were set to 1 and 0, respectively. Regarding the interceptor, the receiver temperature, noise figure, and bandwidth were $T_{\mathrm{I}}$ = 290 K, $F_{\mathrm{I}}$ = 3 dB, and $B_{\mathrm{I}}$ = 1.5 GHz. The required SNR of the interceptor $SN{R}_{\mathrm{Ith}}$ was set to 15 dB, whose detection probability $p_{\mathrm{D}}$ was about 0.99, with the false alert probability $p_{\mathrm{F}}$ = 1 × ${10}^{-7}$, according to (14).

In this study, the policy network $\psi$ and the value network V were constructed as fully connected neural networks with two hidden layers, each containing 256 neurons. The coefficient constants of the reward were set to ${\alpha }_1=2\times {10}^{-5}$, ${\alpha }_2=1$, ${\alpha }_3=1$, ${\alpha }_4=2$, ${\alpha }_5=2$, ${\alpha }_6=0.2$, and ${\alpha }_7=0.2$. The reward discount factor $\gamma$ was set to 0.99. The replay buffer size $D_m$ was 10,000. The learning rate decreased gradually from $1\times {10}^{-5}$ to $1\times {10}^{-7}$. The training epoch $N_{\mathrm{train}}$ was 800, and the number of SGD iterations $N_{\mathrm{update}}$ in each epoch was 30. As shown in the reward in Figure 2, the algorithm gradually converged with the network training.

4.2. Simulation Results

In scenario one, the average transmitting power $P_{\mathrm{A}m,t}$ of 4 radars was obtained after the policy network $\psi$ was trained, as shown in Figure 3. The results were obtained by minimizing the total NCEVR in task time subject to constraints on detection performance. It can be seen that, compared with Radar 2 and Radar 3, the average transmitting power of Radar 1 and Radar 4 is higher, which is because the distance of the target at the top (Target 1) and the bottom (Targets 7, 8, 9, 10) in scenario one is farther. In particular, Radar 4 has to take on more SAR imaging and transmit more power since multiple long-range targets are at the top.

The easily intercepted area corresponding to the transmitting power is shown in Figure 4. Due to space limitations, only a few moments are shown. It should be noted that the easily intercepted area is not the sum of the respective radars’ areas, which should take into account the radar’s positional distribution due to the overlap. The easily intercepted areas for PPO and PSO-A are presented in the first and second rows of Figure 4, respectively. While both methods optimize the transmitting power across all time steps, their computational architectures differ fundamentally, with PPO’s deep reinforcement learning framework contrasting with PSO’s intelligent optimization algorithms. The easily intercepted area for PSO-S is presented in the third row of Figure 4, resulting from the fixed transmitting power configuration in the conventional scenario, where transmitting powers are optimized using PSO. Since each radar’s transmitting power remains fixed, the easily intercepted area remains constant across all time steps.

The proposed PPO method enhances LPI performance while maintaining imaging performance. To verify its LPI performance, we compare the results of PPO, PSO-A, and PSO-S, where NCEVR computed from the easily intercepted area for a netted radar is evaluated at different time, as shown in Figure 5.

For the proposed PPO-based method, the mid-mission NCEVR is larger than that of PSO-A and PSO-S, but the NCEVR of PPO is smaller at most of the other moments. The objective function is to optimize the cumulative NCEVR over the entire task time; the results are shown in Figure 6, scenario one. The average NCEVR in the task time for PPO is 13.77, which is less than 14.85 for PSO-A and 14.50 for PSO-S. It should be noted that the PSO-S method can be viewed as a special case of the more general PSO-A approach. In theory, if both algorithms converge to their global optima, PSO-A should achieve performance that is either superior or at least equivalent to that of PSO-S. However, the long-term power allocation problem, which involves optimizing transmit power for multiple radars over time, poses a significant challenge for PSO-A. Due to the high dimensionality and non-convexity of the problem, PSO-A tends to fall into local optima. Therefore, a reinforcement learning-based method, PPO, was adopted to solve the long-term optimization problem in this study. By leveraging its ability to explore and exploit high-dimensional state-action spaces effectively, the proposed PPO avoids local optima and achieves better LPI performance.

To verify the imaging performance, the SNR in SAR imaging and the azimuth resolution of 10 targets are shown in Figure 7, where the red dashed line is the threshold. The SNR in the SAR imaging of the targets $SN{R}_n$ is greater than the threshold, and the azimuth resolution of the targets ${\rho }_{\mathrm{A}n}$ is less than the threshold, satisfying the constraints. As for the range resolution ${\rho }_{\mathrm{R}}$, it is ensured by the radar bandwidth $B_{\mathrm{R}}$ in (6).

To further verify the SAR imaging performance, the back projection (BP) algorithm was adopted [48]. In the BP algorithm, the target signal energy distributed across multiple echoes is concentrated at its true position through phase compensation and coherent integration, thereby reconstructing the image. Meanwhile, the noises in the echoes do not accumulate due to incoherence. Therefore, the imaging performance is represented by the sum of the SNR of multiple radars at multiple times, that is, the sum SNR after coherent accumulation in (16). The mean square error (MSE) of 10 targets in scenario one after the BP algorithm imaging is shown in Figure 8. Targets with a higher SNR generally exhibit a lower MSE.

Due to space limitations, part of the imaging results are shown in Figure 9, corresponding to the maximum MSE = 0.0043 (target 8) and minimum MSE = 0.0022 (target 6) in Figure 8 for the proposed method. As shown in Figure 9a, the ground truths of all targets are identical, featuring a 0.3 m × 0.3 m square with a 0.1 m × 0.1 m hollow centered at its midpoint. Target 8, which produced the worst imaging result (highest MSE) under the proposed PPO method, PPO, PSO-A, and PSO-S, achieved similar imaging results as shown in the second row of Figure 9, meeting the required SAR imaging performance. Target 6, which achieved the minimum MSE among all targets under the proposed PPO method, exhibited imaging results with significantly reduced noise as shown in Figure 9e. In general, the proposed PPO method achieved satisfactory imaging of the targets while providing better LPI performance compared with PSO-A and PSO-S.

The proposed PPO method’s LPI performance improvement compared with PSO-A and PSO-S is related to the scenarios. A specific case in scenario two is considered as shown in Figure 10a.

The transmitting power of the PPO method is shown in Figure 11a. Radar 1 continuously emits power toward the targets to ensure LPI and imaging performance, as it operates at a shorter distance to the targets. Since the transmitting power of PPO is similar to the fixed power of PSO-S in conventional scenarios, PPO achieves an average NCEVR of 10.58, which is comparable to PSO-S’s 11.12 as shown in Figure 6, scenario two.

As for the imaging results, due to space limitations, the targets with maximum MSE = 0.0043 for PPO are shown in the first row of Figure 12. The results are intentionally chosen to reflect the worst imaging performance of PPO. Nevertheless, PPO achieves satisfactory imaging results compared to PSO-A and PSO-S.

In scenario three of Figure 10b, the transmitting power of the PPO method is presented in Figure 11b. Since the five targets are located at the starting position of Radar 1, it tends to transmit power at this time. Meanwhile, since three of the five targets [12 km, 3 km], [14 km, 3 km], and [16 km, 3 km] are relatively far away from Radar 1, targets [16 km, 13 km] and [18 km, 15 km] are also adequately detected during the scanning of these three targets, eliminating the need for other radars to operate. When they move away from these targets, Radar 1 stops transmitting, ensuring that the NCEVR is minimized for the entire task. In this scenario, the conventional fixed transmitting power approach in PSO-S is no longer suitable. As a result, PPO achieves an average NCEVR of 5.88, significantly lower than PSO-S’s 9.05 as shown in Figure 6, scenario three. And PSO-A converged to a local optimum solution, resulting in 14.25. As shown in the second row of Figure 12, the target with maximum MSE = 0.0037 for PPO is selected to show its worst-case imaging performance. The results show that PPO achieves the best LPI performance with similar imaging performance.

In scenario four of Figure 10c, a randomly generated scenario is also considered to verify the proposed PPO-based method. The transmitting power is shown in Figure 11c. The imaging results of the target with maximum MSE=0.0047 for PPO are shown in the third row of Figure 12. Within SAR imaging performance constraints, PPO minimizes LPI through optimal transmit power allocation over the whole mission. As shown in Figure 6 scenario four, PPO achieves an average NCEVR of 6.30, which is less than 14.18 for PSO-A and 11.75 for PSO-S. In summary, the proposed PPO algorithm explores the optimal action using a deep reinforcement learning method. It is illustrated and validated in different scenarios, which achieves better performance compared with PSO-A and PSO-S.

5. Conclusions

In this paper, a PPO-based reinforcement learning method is proposed to achieve LPI for netted radar in SAR imaging. For the LPI performance, this paper proposes NCEVR based on CEVR, which provides a physically meaningful characterization of the LPI performance for netted radar. For SAR imaging performance, the single-pulse SNR, the SNR in SAR imaging, and the resolution ratio are considered and verified using the BP algorithm. It should be noted that the LPI performance and SAR imaging performance in this paper are the comprehensive results of radar resource scheduling at different moments during the entire time. A PPO-based method is employed to optimize the cumulative NCEVR for the whole mission, with the constraints of the SAR imaging performance. The proposed method was verified through simulation experiments, and its LPI performance was better than that of the comparison algorithm.