PC-YOLO: Moving Target Detection in Video SAR via YOLO on Principal Components

Highlights

What are the main findings?

• We reduced the heavy level of clutter in ViSAR by the principal component decomposition learning technique. So the SNR of SAR image can be improved. It played an important role for the subsequent task of moving target detection. A hierarchical learning mechanism composed of the self-attention, the spatial attention, and the channel attention were presented. Much more effective features for the task of target detection can be learned.
• A cross-scale feature fusion strategy composed of the temporal fusion and the spatial fusion were presented within the look once detection framework. So the moving targets with mutable scales can be located accurately. Multiple rounds of experiments were performed. The results demonstrated the advantage of proposed method in comparison to the classical methods, as well as the state-of-the-art detection algorithms.

What are the implications of the main findings?

• A new clutter removal strategy specific for the task of moving target detection. Different from the preceding works, we regarded the complicated imaging scenarios as a typically combinatorial scene, where the moving targets contributed to the sparse components, while the background clutter attributed to the low-rank components. So far as we known, the assumption for ViSAR was first presented in this paper.
• A new detection framework for the extraction of shadow. Different from the generic method, the hierarchical attention mechanism and the cross-scale feature fusion were presented. So the target with various scales can be located accurately.

Abstract

Video synthetic aperture radar could provide more valuable information than static images. However, it suffers from several difficulties, such as strong clutter, low signal-to-noise ratio, and variable target scale. The task of moving target detection is therefore difficult to achieve. To solve these problems, this paper proposes a model and data co-driven learning method called look once on principal components (PC-YOLO). Unlike preceding works, we regarded the imaging scenario as a combination of low-rank and sparse scenes in theory. The former models the global, slowly varying background information, while the latter expresses the localized anomalies. These were then separated using the principal component decomposition technique to reduce the clutter while simultaneously enhancing the moving targets. The resulting principal components were then handled by an improved version of the look once framework. Since the moving targets featured various scales and weak scattering coefficients, the hierarchical attention mechanism and the cross-scale feature fusion strategy were introduced to further improve the detection performance. Finally, multiple rounds of experiments were performed to verify the proposed method, with the results proving that it could achieve more than 30% improvement in mAP compared to classical methods.

1. Introduction

Synthetic aperture radar (SAR) sensors have been widely applied across various civilian and military fields. Two-dimensional (2-D) high-resolution SAR images can be generated by pulse compression and coherent accumulation techniques along the range and the azimuth, respectively. Phase history data can be jointly compressed and focused using imaging algorithms, such as the range Doppler algorithm, the polar format algorithm, the chirp scaling algorithm, and the $\omega$-K algorithm [1,2]. Given the high resolution of SAR images, valuable information can be extracted by image interpretation for target detection and recognition [3].

In realistic scenarios, both static and moving targets are widely available. Compared with static targets, the scattering characteristics of moving targets differ significantly as they are displaced over time and have varying spatial locations. The motion phenomenology causes the target imagery to be defocused due to spatially variant phase errors [4]. It is therefore much more difficult for SAR sensors to discern, especially in complex, cluttered environments. A set of exemplars is shown in Figure 1. The current solutions for moving target detection can be summarized from two perspectives: SAR images and video SAR (ViSAR).

1.1. Moving Target Detection via SAR Images

Early studies typically achieved the task of moving target detection using both single-channel and multi-channel SAR images. The single-channel data were collected using radar with a single antenna and provided very limited information. The moving target detection performance was therefore poor, especially for slow-moving targets on the ground. For the multi-channel SAR system, a series of radar antenna arrays was deployed on airborne or spaceborne platforms to allow significant amounts of higher-dimensional information to be collected by the phase center antenna array. The detection performance was improved accordingly. Typical multi-channel methods for moving target detection include the Displaced Phase Center Antenna (DPCA) [5], the Along-Track Interferometry (ATI) [6,7], and the Space-Time Adaptive Processing (STAP) [8] methods.

• The DPCA technique leverages the identical scattering characteristics of stationary clutter across channels. By subtracting the channel echo signals, stationary clutter can be filtered out, whereas moving targets, due to their distinct motion characteristics, are not canceled by this subtraction, enabling their detection.
• The ATI technique operates on the principle that stationary targets have a consistent echo phase between channels, whereas moving targets exhibit a phase variation. By comparing this inter-channel phase information, moving targets can be detected, and these phase data can also be used to estimate the target’s radial velocity.

Additionally, the multi-channel radar echo data can also be processed by jointly applying DPCA and ATI to achieve enhanced performance [9,10]. However, DPCA and ATI impose strict constraints on the configuration of the radar antenna. Detection performance deteriorates in strong clutter environments. Another technique, STAP, was then proposed [11], which employs adaptive algorithms to estimate the statistical properties of clutter in the joint space–time domain. A two-dimensional filter that is precisely matched to the clutter spectrum can then be formed accordingly. It is capable of suppressing the clutter while preserving the signal of the moving target. Although STAP could reduce the clutter its detection performance for the moving target is poor.

Recent studies therefore paid more attention to advanced signal processing tricks, such as principal component analysis. Yang et al. proposed a fast RPCA-based (robust principal component analysis) detection method for multichannel SAR under a strong clutter background [12]. They applied RPCA in the range-Doppler domain to improve STAP performance. The experiments proved that the presented method was even effective for channel imbalance or platform motion error. Guo et al. presented a moving target detection method based on RPCA for SAR systems [13]. An atomic norm-based optimization program was constructed to transform the data sparsity requirement into a moving target sparsity requirement. Li et al. developed an along-track interferometry GoDec approach for GMTI in multichannel SAR systems under a strong clutter background [14]. The proposed method was separated into predetection and postdetection. The former reduced the number of missing targets using an efficient ATI-RPCA, while the latter reduced the number of false alarms using a magnitude and phase processing technique. L. Ramos et al. presented two ground scene estimation methods for SAR images [15], where the RPCA via principal component pursuit and tensor RPCA via the tensor nuclear norm were employed to produce ground scene estimation results. On this basis, a combination of RPCA and STAP techniques was proposed for moving target detection in Ref. [16].

1.2. Moving Target Detection via Video SAR

Unlike in SAR images, targets can be observed continuously using video SAR technology, which makes so much more information available for subsequent tasks, such as target detection. Popular methods for target detection in video SAR include signal processing tricks and shadow discernment.

• Signal processing tricks. Many kinds of signal processing techniques have been proposed to achieve target detection in video SAR. These methods identify moving targets through the statistical variations in radar echo amplitude. Since the scattering signatures of the targets had changed significantly, the resulting amplitude, phase, and frequency of the received echoes were altered accordingly. So, a natural idea was to extract motion information by comparing the amplitude, phase, and frequency of radar echoes, enabling the achievement of the task of target detection and tracking. Fan et al. proposed an echo-driven motion compensation framework to improve THz-ViSAR images [17]. Airborne motion errors were modeled as both radial and vibration errors and were further estimated from the echoes. Yan et al. derived the mathematical model of moving target echoes in a video SAR system [18]. Typical system parameters for motion trajectory estimation can be simulated. Li et al. developed a curvilinear moving target refocusing method for ViSAR images [19]. The localized phase-gradient autofocus trick was used to compensate for Doppler chirp-rate inconsistencies, while additional spatial-domain information was used for geometric deformation correction. Gou et al. proposed a circular-track video SAR detection method using sub-aperture segmentation [20]. This method achieved a high detection probability in high-frame-rate scenes. However, methods based on signal processing are sensitive to ground clutter and multipath interference, which reduce accuracy in complex environments.
• Shadow discernment. Moving targets exhibited a Doppler shift in video SAR. Energy leaking across the range cells caused the images to be defocused. This phenomenon resulted in a deviation along the azimuth direction and a certain angular spread, preventing the target’s true position from fully reflecting the radar waves and forming distinct shadow features. These characteristics can be used to approximate the true position and state information of targets. Another idea was to detect targets from their shadows. Early methods include the inter-frame difference method and the background difference method. Luo et al. applied image binary operations after differencing the obtained background model [21]. Morphological processing was then imposed on the resulting binary image to extract moving target shadows and suppress false alarms. Zhang et al. combined the background difference model with the three-frame difference method for shadow extraction [22]. Zhang et al. proposed extracting the target by subtracting three sequential frames [23]. The extracted images were then handled using the mathematical morphology technique. Wang et al. performed several rounds of simulation experiments on low Radar Cross Section (RCS) moving target signals [24]. They were then transferred into the real Ka-band airborne video SAR images. The detection method was effective even for the stealth targets. He et al. introduced a ViSAR generation method based on fast factorization back-projection to address the problem of speed limitations [25]. It was applicable to both video SAR generation and moving target detection. Yin et al. presented a decomposition framework based on a low-rank sparse decomposition model [26]. It was combined with trajectory area extraction to achieve fast and accurate detection in specific regions.

Likewise, recent studies have gradually relied on the learning technique for target detection. Zhang et al. proposed a moving target tracking approach with shadow detection and tracking [27]. The CNN tracking classification was applied to potential moving target candidates extracted from a sequence of temporal and spatial sub-aperture SAR images. Ding et al. presented a framework for shadow-aided moving target detection with the combination of a faster region-based convolutional neural network (Faster-RCNN) and bi-directional long-short-term memory (Bi-LSTM) [28]. The former was used to detect shadows in a single frame, while the latter was employed to suppress missing alarms. Yan et al. developed a deep framework composed of a 3-D convolutional encoding path, a 2-D convolutional decoding path, and a bridge path to capture the target’s shadow information [29]. The spatiotemporal features and high-level semantics of raw continuous images can be exploited accordingly. Yang et al. proposed a fast multiscale feature extraction module embedded with triplet attention to tackle the SAR shadow tracking problem [30]. Wen et al. combined shadow detection in SAR images and Doppler energy detection in the range-Doppler spectrum using a dual faster region-based convolutional neural network [31].

1.3. Our Solution

Although many approaches devoted to moving target detection have been presented in prior work, they still suffer from some problems. Their typical difficulties can be summarized as follows.

• Manual operations for feature extraction. Traditional methods, such as the inter-frame difference and the background difference, relied on many manual engineering operations, such as frame registration, de-speckling, and the estimation of background. These intermediate operations had a significant impact on termination performance.
• Strong clutter environments. Unlike common vision images, the frame of ViSAR was formed by coherent imaging, while both the diffuse reflection and the specular reflection were available. Therefore, the signal-to-noise ratio (SNR) of the SAR image was much lower. Low-SNR environments make shadow extraction much more difficult, and how to address the clutter remained an open problem.
• Variable target scales. In radar images, shadows form because electromagnetic echoes are occluded by the target itself. So the scale of the shadow was dependent on the size of the target, the incident angle, and the resolution. These factors make the scale of shadows mutable in ViSAR. Another problem was how to extract shadows with various scales, especially for those that covered few resolution cells.

To address these problems, this paper proposes a model and data co-driven cross-learning method. Unlike prior work, we regarded the imaging scenario in ViSAR as a combination of low-rank and sparse scenes. Specifically, the moving targets (local anomalous patterns) contributed to the sparse scene, while the global, slowly varying background (clutter) formed the low-rank scene. A sparse reconstruction and low-rank decomposition technique was then introduced to separate them accurately. The task of moving target detection in ViSAR can then be decoupled as two sub-tasks: clutter suppression via an empirical model, and shadow extraction via a data-driven model. The first sub-task was achieved using the sparse and low-rank decomposition technique. The resulting components were then delivered into the look once neural network. Since the moving targets had varying scales and weak scattering coefficients, a hierarchical attention learning mechanism and cross-scale feature fusion strategy were proposed. The shadow of moving targets can then be extracted accurately. To verify the proposed method, multiple rounds of experiments were performed. The results proved that the proposed method could achieve more than 30% improvement in mAP compared to classical methods.

2. The Signal Model

The primary operational modes for video SAR were stripmap and spotlight. In comparison, spotlight mode can continuously observe a fixed area to obtain richer and more precise information. In this section, the video SAR mechanism was demonstrated using the spotlight imaging mode.

The spatial model for spotlight SAR imaging is illustrated in Figure 2. Suppose the aircraft flew along the positive direction of the X axis at a velocity of v, with a flight altitude of h. The scene center point was denoted as O, and the size of the swath was $W_a\times W_r$. The notation used in the following sections is summarized in

Table 1. The notation for the ViSAR signal model.

$r_c$	the shortest slant range from scene center to aircraft
$t_s,t_e$	the start, terminate time of observation
$t_c$	the center time of the full aperture
T	the synthetic aperture duration
$\Delta \theta$	the rotation angle of the aircraft relative to the scene center
${\theta }_c$	the angle between flight direction and line-of-sight at $t_c$

.

Suppose the radar transmitted the linear frequency modulation signal (LFM),

$$s_t\left(t\right)=\sigma \left(t\right)·rect\left({\displaystyle \frac{t}{T_r}}\right)exp(j2\pi f_{c}t+\pi K{t}^2),$$

(1)

where $f_c$ is the carrier frequency, K is the chirp rate, $T_r$ is the pulse repetition time, and $\sigma$ is the scattering coefficients.

2.1. Range Compression

Given the time delay from the scene $\tau$, the received signal was

$$s_r\left(t\right)=\sigma \left(t\right)·rect\left({\displaystyle \frac{t}{T_r}}\right)exp(j2\pi f_c(t-\tau )+\pi K{(t-\tau )}^2).$$

(2)

Range compression was achieved using match filtering $s_c\left(t\right)=s_r\left(t\right)\ast h\left(t\right)$, in which the reference functions were the conjugate transpose of the transmitted signal $h\left(t\right)=s_t^{*}\left(t\right)$. The range resolution was determined by the bandwidth of the transmitted signal ${\rho }_r={\displaystyle \frac{c}{2B}}$, where c is the speed of light.

For different operational modes, when using pulse-modulated transmitted signals, the range resolution depends on the pulse interval. The setting of the pulse interval is also constrained by the detection range. Therefore, SAR typically employs signals with a large time-bandwidth product combined with pulse compression technology to achieve high range resolution while meeting certain transmission power requirements.

2.2. Azimuth Resolution

According to the spatial model of spotlight imaging, the instantaneous Doppler frequency of the echo signal at time $t_s$ was

$$f_{as}={\displaystyle \frac{2vcos\left({\theta }_c-\Delta \theta /2\right)}{\lambda }}.$$

(3)

Likewise, the instantaneous Doppler frequency of the received signal at time $t_e$ was

$$f_{ae}={\displaystyle \frac{2vcos\left({\theta }_c+\Delta \theta /2\right)}{\lambda }}.$$

(4)

So, the Doppler bandwidth of the echo signal over the entire observation time T was

$$B_a=\left|f_{as}-f_{ae}\right|={\displaystyle \frac{4v}{\lambda }}·sin(\Delta \theta /2)sin\left({\theta }_c\right).$$

(5)

The azimuth resolution can be obtained as

$${\rho }_a={\displaystyle \frac{vsin{\theta }_c}{B_a}}={\displaystyle \frac{\lambda }{4sin\left(\Delta \theta /2\right)}}\approx {\displaystyle \frac{\lambda }{2\Delta \theta }}.$$

(6)

This was dependent on the wavelength $\lambda$ and the rotation angle $\theta$ relative to the scene center, and independent of the size of the radar antenna D. However, long spotlight durations would lead to large squint angles, in which range–azimuth coupling and range cell migration were produced, thereby deteriorating the focusing quality. Thus, the circular trajectory was typically used for ViSAR.

2.3. The Generation of ViSAR

In the small-angle spotlight mode, the synthetic aperture length can be approximately expressed as $L=r_c\times \Delta \theta$. Thus, the synthetic aperture time can be obtained as

$$T={\displaystyle \frac{r_c\times \Delta \theta }{v}}.$$

(7)

The frame rate of SAR imaging can then be defined as

$$F={\displaystyle \frac{1}{T}}={\displaystyle \frac{v}{r_c\times \Delta \theta }}={\displaystyle \frac{2v{\rho }_a}{\lambda r_c}}={\displaystyle \frac{2v{\rho }_a{f}_c}{c{r}_c}}.$$

(8)

So the SAR imaging frame rate is related to platform velocity v, azimuth resolution ${\rho }_a$, the operating frequency, and the slant range $r_c$. ${\rho }_a$ and $r_c$ were predetermined for the radar systems [32]. Likewise, the velocity of platform v can be tuned in a limited manner. Furthermore, an increase in platform velocity would lead to the Doppler spectrum aliasing and the clutter spectrum broadening. A reasonable idea for boosting the ViSAR frame rate was to increase the carrier frequency of the radar.

2.4. The Formation of Shadow

According to the ViSAR imaging mechanism, it is easy to come to the conclusion that the formation of a shadow for a stationary target and a moving target is significantly different. For the stationary target, shadows are accompanied by the target, and the concept of shadows here is similar to the “shadow” in optical images. If the radar beam shines on a target of a certain height at a specific angle, the front side will produce a reflected signal, and therefore obtain the scattering characteristics in the image, while the back side will be occluded by itself. The energy information was therefore lost. Then the dark areas in the SAR image can be formed.

Conversely, when the target is moved with a velocity along a certain direction, the formation of the shadow is significantly different. If the movement along the range direction is available, the overlay area will shift from its true position. Contrarily, if there is movement along the azimuth, the imaging result of the target will be defocused. In these circumstances, the shadow of the target would be highlighted further. It is therefore convenient to detect the target using its shadow.

3. The Proposed Method

One of the major advantages of ViSAR was its ability to monitor targets continuously. Much more information could be provided. However, it also suffered from some issues, such as low SNR, a heavy level of clutter, and the mutable scale of the shadows. In this paper, a model and data co-driven learning method, look once on principal component (PC-YOLO), is proposed. We decoupled the detection of a moving target into two subtasks: clutter removal and shadow extraction. The former was achieved using a sparse, low-rank constraint decomposition model, while the latter was achieved using an improved detection network.

3.1. Method Overview and Motivation

Inspired by previous work, this paper proposes target detection via look once on principal components. We first suppose that the background (composed of clutter and speckle) of the frame in video SAR is usually strongly correlated. Thus, all of them could span a low-rank linear subspace. Conversely, the foreground (composed of targets) exhibited different motion modes and texture features compared to the background. This can be regarded as a small amount of gross error that deviates from the low-rank subspace. The stationary and moving components can be separated using low-rank and sparsity constraints. The task of moving target detection can then be decoupled as clutter removal and shadow extraction.

• Clutter removal. To suppress the clutter, the imaging scenarios were viewed as the combination of low-rank and sparse components. The former models the components with strong correlations, while the latter represents the motion modes. Each ViSAR frame was then decomposed into a sparse component and a low-rank component. The background clutter was suppressed, while the targets were enhanced.
• Shadow extraction. To extract shadows of various sizes, a new detection algorithm was proposed, where the cross-scale feature fusion strategy and the hierarchical attention mechanism were deployed. The sequential representations of the multi-scale features generated from the backbone were formed to obtain the features at different levels of detail. They were further combined to enhance the perception ability of multi-scale targets so that the detection performance could be improved accordingly.

The overall implementation of the proposed method is shown in Figure 3. The video SAR was first rearranged into the frame sequences. Each frame was normalized to ensure scattering consistency. It was then decomposed into the low-rank and sparse components ${\mathcal{P}}_L,{\mathcal{P}}_S$. The former models the stationary components, such as the clutter and the speckle. In contrast, the latter highlighted the motion modes compared to the background. This decomposition effectively suppressed the background interference and enhanced the saliency of motion-induced shadows. The results were fed into an improved target detection network that integrates a hierarchical attention mechanism and multi-scale feature fusion for shadow extraction.

3.2. The Sparse and Low-Rank Decomposition Model

In complex imaging or sensing environments, the raw ViSAR data are usually contaminated by random noise, clutter, and interference, which make the detection of moving targets difficult. To overcome these problems, we regarded the imaging scenarios as the combination of sparse and low-rank components. The sparse and low-rank decomposition technique was then introduced to jointly model the low-rank and sparse structures. An accurate separation between the structured background and the sparse components can be achieved. The former expressed the global, slowly varying background information, while the latter showed the sparse component that represents localized anomalies, such as moving objects or salient targets. Suppose the observed ViSAR data were $D\in {\mathbb{R}}^{m\times n}$; we aim to solve the following decomposition,

$$D={\mathcal{P}}_L+{\mathcal{P}}_S,$$

(9)

where ${\mathcal{P}}_L,{\mathcal{P}}_S$ denotes the low-rank and sparse components. The underlying structure of the observed data can be recovered. The Equation (9) can be solved using a convex optimization that minimizes the combination of the nuclear norm and the ${\ell }_1$-norm,

$$\underset{{\mathcal{P}}_L,{\mathcal{P}}_S}{min}\parallel {\mathcal{P}}_L{\parallel }_{*}+\lambda {\parallel {\mathcal{P}}_S\parallel }_1\mathrm{s}.\mathrm{t}.D={\mathcal{P}}_L+{\mathcal{P}}_S,$$

(10)

where $\parallel {\mathcal{P}}_L{\parallel }_{*}$ denotes the nuclear norm to enforce low-rankness, $\parallel {\mathcal{P}}_S{\parallel }_1$ represents the ${\ell }_1$-norm to promote sparsity, and $\lambda$ is a trade-off parameter. The formulation can be efficiently solved by the augmented Lagrange multiplier or the Alternating Direction Method of Multipliers (ADMM). Likewise, the principal component pursuit problem can also be relaxed as

$$\begin{array}{cc}\hfill \underset{{\mathcal{P}}_L,{\mathcal{P}}_S}{min}& \parallel D-{\mathcal{P}}_L-{\mathcal{P}}_S{\parallel }_F^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \mathrm{rank}\left({\mathcal{P}}_L\right)\le r,\hfill \\ \hfill & \mathrm{card}\left({\mathcal{P}}_S\right)\le k,\hfill \end{array}$$

(11)

where r denotes the rank maximum of ${\mathcal{P}}_L$, and k represents the number of nonzero elements in ${\mathcal{P}}_S$. Stochastic optimization can be used to approximate the decomposition under these constraints and is central to robust principal component analysis learning. Through low-rank and sparse decomposition, the heavy level of background clutter can be suppressed, while the moving targets can be highlighted, conversely. In the context of ViSAR, ${\mathcal{P}}_L$ captures the stable terrain or scene background, whereas ${\mathcal{P}}_S$ enhances the dynamic or anomalous components such as moving objects or shadow regions. So, a high signal-to-noise representation that significantly benefits subsequent processing tasks can be generated.

In video SAR imagery, consecutive frames exhibit strong temporal correlations and a dominant stationary background, whereas moving targets (shadows) primarily exhibit subtle intensity anomalies. By applying Equation (9) to a sequence of video SAR frames, the low-rank component ${\mathcal{P}}_L$ models the stable background clutter and correlated speckle noise, whereas the sparse component ${\mathcal{P}}_S$ isolates the transient or non-stationary features associated with moving targets and their shadows. The decomposition operations not only suppress the background clutter and the coherent speckle noise, but also enhance the saliency and discriminability of small shadow targets.

3.3. The Look Once Detection Network

According to the presented decomposition model, ViSAR can be decomposed into sparse and low-rank components. The extracted sparse component was employed to achieve the subsequent sub-task, moving target detection. In this paper, it was realized via shadow extraction [33,34]. An improved version of the look once algorithm, incorporating cross-scale feature fusion and a hierarchical attention mechanism, was presented. The overall framework for shadow extraction is shown in Figure 4.

In SAR video imagery, the shadows of moving targets are typically small in size and dim in intensity. Their features closely resemble the background clutter of SAR images. Due to the low contrast and small size of shadow targets, generic methods often fail to detect them, especially in complex backgrounds, where shadows overlap with clutter and noise. To solve this problem, two types of improvements are presented in this section: the hierarchical attention mechanism and the cross-scale feature fusion.

3.3.1. Hierarchical Attention Mechanism

The hierarchical attention mechanism includes self-attention, channel attention, and spatial attention. The self-attention decoupled the channel by a multi-head first.

Self-attention. Take the input feature map $\mathbf{F}\in {\mathbb{R}}^{C\times H\times W}$, for example; the self-attention learning includes four steps.

Step 1. Get the query, the key, and the value vectors using the feature mapping,

$$\mathbf{Q},\mathbf{K},\mathbf{V}={\varphi }_s\left({\phi }_{cov}^{1\times 1}\left(\mathbf{F}\right)\right),$$

(12)

where ${\varphi }_s$ denotes the feature split, and ${\phi }_{cov}^{1\times 1}$ is the 1 × 1 convolution.

Step 2. Calculate the similarity measurement between the query and the key, $\mathbf{S}={\displaystyle \frac{\mathbf{Q}·{\mathbf{K}}^T}{\sqrt{d_k}}}$, where $\sqrt{d_k}$ is the scale factor.

Step 3. Normalize the similarity measurement using softmax activation, $A=softmax\left(S\right)$.

Step 4. Obtain the outputs of self-attention using the weighted summation, ${\mathbf{F}}_a=AV$.

Spatial attention. Given the self-attention features ${\mathbf{F}}_a\in {\mathbb{R}}^{C\times H\times W}$, the spatial attention mechanism partition the self-attention features into several non-overlapping sub-features along the spatial, resulting the top-left (${\mathbf{F}}_a^{\left(s_1\right)}$), top-right (${\mathbf{F}}_a^{\left(s_2\right)}$), bottom-left (${\mathbf{F}}_a^{\left(s_3\right)}$), and bottom-right (${\mathbf{F}}_a^{\left(s_4\right)}$) of the original feature, respectively. Then two types of attention were applied to the sub-features, local attention and global attention,

$$\left\{\begin{array}{cc}{\mathbf{F}}_s^{\left(l\right)}=\oplus \left({\kappa }_c({\mathbf{F}}_a^{\left(s_i\right)}\otimes {\mathbf{F}}_a^{\left(s_i\right)})\right)\hfill & \mathit{Local}\hfill \\ {\mathbf{F}}_s^{\left(g\right)}={\kappa }_c\left({\mathbf{F}}_a\right)\otimes {\mathbf{F}}_a\hfill & \mathit{Global},\hfill \end{array}\right.$$

(13)

where ${\kappa }_c$ is the channel mechanism, ⊕ is the tensor concatenation operation, and ⊗ is the element-wise multiplication. The resulting features were further combined together, ${\mathbf{F}}_s={\mathbf{F}}_s^{\left(l\right)}+{\mathbf{F}}_s^{\left(g\right)}$.

Channel attention. Similar operations can be applied to the original features, forming the channel attention mechanism. The only difference was that the self-attention features were decomposed along the channel dimension, rather than the spatial dimension. The resulting features were noted as ${\mathbf{F}}_c={\mathbf{F}}_c^{\left(l\right)}+{\mathbf{F}}_c^{\left(g\right)}$.

3.3.2. Cross-Scale Feature Fusion

Although the generic YOLO algorithm performs well on vision images, it has limitations when applied to small targets, particularly shadows. To address the challenge of scale variation in video-based SAR, a cross-scale feature fusion strategy was presented.

Spatial fusion. The spatial fusion refers to summing or concatenating the multi-level features, such as ${\mathcal{F}}_1,{\mathcal{F}}_2,{\mathcal{F}}_3,{\mathcal{F}}_4$. The workflow of cross-scale feature fusion was illustrated in Figure 5.

Suppose the poorest feature was ${\mathbf{F}}_4$; it was first up-sampled to be aligned with the finer scale of features ${\mathbf{F}}_3$,

$${\mathbf{F}}_{43}={\varphi }_{conv}^{1\times 1}(\oplus ({\varphi }_{\uparrow }^2\left({\varphi }_{conv}^{1\times 1}\left({\mathbf{F}}_4\right)\right),{\mathbf{F}}_3)).$$

(14)

The obtained tensor was further combined with the down-sampled version of ${\mathbf{F}}_2$,

$${\mathbf{F}}_{23}={\varphi }_{conv}^{1\times 1}(\oplus ({\varphi }_{\uparrow }\left({\mathbf{F}}_{43}\right),{\mathbf{F}}_2,{\varphi }_{\downarrow }\left({\mathbf{F}}_1\right))),$$

(15)

where ${\varphi }_{\uparrow },{\varphi }_{\downarrow }$ denote the up-sampling and down-sampling operations along the spatial dimensions.

Temporal fusion. Temporal fusion focuses on integrating shallow and deep features across multiple scales. It captures structural and semantic differences induced by scale changes and performs fine-grained fusion across hierarchical feature representations, enabling more accurate detection of small shadow targets under varying spatial resolutions and temporal dynamics. Given the feature maps $\mathbf{F}$, the series of reconstructed features was expressed as

$${\mathbf{F}}_r=\mathbf{F}\ast h\left(\sigma \right)=\mathbf{F}\ast {\displaystyle \frac{1}{2\pi {\sigma }^2}}exp\left(-{\displaystyle \frac{u^2+v^2}{2{\sigma }^2}}\right),$$

(16)

where $h\left(\sigma \right)$ was the Gaussian filter with the standard deviation $\sigma$, and $(u,v)$ denotes the spatial index. This series of features was then fused in a similar way to the spatial fusion. The only difference was that the down-sampling and up-sampling operations were not implemented.

4. Experiments and Discussion

To verify the proposed method, multiple rounds of experiments were performed on the measured ViSAR. The experimental setting was first delineated, followed by comparative studies.

4.1. Experimental Settings

We first delineate the experimental setup, including the imaging, algorithm, and evaluation metrics.

4.1.1. The Imaging Setting

The video SAR dataset used in this paper was publicly released by the Sandia National Laboratory [35]. The imaging parameters available for the radar system are shown in

Table 2. The imaging parameters of the SNL radar to collect the video SAR data.

Parameter	Value
Mode	Spotlight
Center Frequency	16.7 GHz (Ku band)
Wavelength	1.8 cm
Incidence Angle	65°
Platform Height	2 km
Platform Speed	245 km/h
Cross-Range Resolution	0.1 m
Total Rotation Angle	200°

. The FPS (frame-per-second) of the ViSAR is 30, and the resolution of each frame of the SAR image is 720 × 660. To test the presented algorithm, almost 900 continuous scenes of SAR images were obtained from the ViSAR. They were randomly partitioned into the training and test sets with a ratio of 8:2.

4.1.2. The Algorithm Setting

All of the experiments were performed on a computational platform with the NVIDIA RTX 3060 GPU. During the training process, the model was trained by the Adam optimizer with an initial learning rate of 1 × 10⁻³ and a minimum learning rate of 1 × 10⁻⁵. A cosine annealing learning rate schedule was adopted to gradually reduce the learning rate and stabilize the training process. For the inference phase, the nonmaximum suppression (NMS) was used to suppress duplicate detection boxes with an IoU threshold of 0.50.

4.1.3. The Evaluation Metrics

In the evaluation framework, the positive samples were defined as the moving targets (shadow), while the negative ones were expressed as the others, such as natural and man-made clutter. They were false alarms in the task of target detection. So all of the inference combinations are summarized in

Table 3. The prediction matrix for the task of target detection.

Truth	Prediction				$\mathit{R}=\frac{{\mathit{N}}_{\mathit{t}\mathit{p}}}{{\mathit{N}}_{\mathit{t}\mathit{p}}+{\mathit{N}}_{\mathit{f}\mathit{n}}}$
		Yes	No	Total
	Yes	TP	FN	Positive (T)	Missing alarm
	No	FP	TN	Negative (T)
	Total	$P={\displaystyle \frac{N_{tp}}{N_{tp}+N_{fp}}}\to$ False alarm			$F_1={\displaystyle \frac{P·R}{P+R}}$

, where $TP$, $FP$, and $FN$ denote the numbers of true positives, false positives, and false negatives, respectively.

Typical metrics for measuring detection performance include precision, recall, F1 score, and mean average precision (mAP). They were defined as follows:

$$\left.\begin{array}{c}\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}\hfill \\ \mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}\hfill \end{array}\right\}\Rightarrow F1={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}.$$

(17)

We could therefore conclude that recall reflects the missing alarms, while precision delineates the false alarms.

The mAP can be explained as the average precision across the classes of the target,

$$mAP={\displaystyle \frac{1}{N}}\sum _{i=1}^{N}A{P}_i,$$

(18)

where $A{P}_i$ represents the average precision of the i-th category, and N is the total number of categories.

4.2. The Ablation Study

In this paper, the problem of moving target detection was solved using a model and data co-driven learning method (PC-YOLO). The former refers to the principal component decomposition technique for producing low-rank and sparse components from the ViSAR so that the heavy level of clutter can be suppressed. On this basis, a detection framework comprising a hierarchical attention mechanism and cross-scale feature fusion was presented, which enables the moving target (shadow) with mutable scales to be located accurately. To further test the performance of the presented techniques, some ablation experiments were performed. The results are shown in

Table 4. Ablation study.

Method	Det	PCs	Fusion	Precision	Recall	F1	mAP
Raw	✓	-	-	0.3967	0.4451	0.4195	0.3562
Det (PCs)	✓	✓	-	0.7587	0.5163	0.6145	0.7239
Det (F)	✓	-	✓	0.8328	0.6972	0.7590	0.7504
Our (PCs + F)	✓	✓	✓	0.8280	0.7156	0.7590	0.7652

.

• “Raw” refers to the performance of the original YOLO algorithm. The detection performance was very poor, with an mAP of 0.3562, due to the high level of clutter. The problem of moving-target detection in ViSAR could not be solved by applying generic methods directly.
• The “Det (PCs)” means applying the generic detection algorithm to the principal component. We found that the performance was clearly improved. The results prove that the principal component decomposition can suppress the complex background clutter effectively and enhance the saliency of motion-induced shadows simultaneously, and hence is advantageous to the following feature extraction and detection. Although the mAP was improved from 0.3562 to 0.7239, the recall rate, 0.5163, is not satisfactory. The results demonstrate that many false alarms are present.
• The “Det (F)” denotes the improved target detection algorithm. It improves the mAP from 0.3562 to 0.7504 using a combination of a hierarchical attention mechanism and cross-scale feature fusion. Thus, shadows at various scales can be located.

The proposed method, the improved detection algorithm on the principal components, achieves the best performance. This is because the heavy level of clutter has been reduced by the principal component decomposition, and the targets of multiple scales have been located by the improved detection technique. The mAP for the proposed method is 1.48%, 4.13%, 40.90% better than Det (F), Det (PCs), and Raw. Similar improvements can be obtained for Precision, Recall, and F1-score. In brief, the ablation results validate the complementary advantages of the proposed PC-YOLO framework for moving-target detection.

4.3. The Verification of Principal Component Decomposition

This paper employed the principal component decomposition technique to achieve the task of clutter suppression. The resulting sparse components were used for the subsequent task of target detection, while the low-rank components were ignored. The effectiveness of principal component decomposition therefore needs to be tested.

4.3.1. The Visualization of Principal Components

We first visualize a set of experiments for the decomposition of the principal component. The methods to be compared include background difference, mean difference, contrast enhancement, principal component analysis, and sparse and low-rank decomposition. The results are displayed in Figure 6.

• “Raw” refers to the original image in the video SAR. We found that the heavy clutter is widespread in the imaging plane due to the coherent imaging mechanism. The low-SNR image makes target observations much more difficult.
• The “bkDIFF” is the background difference method. It is typically used to reduce clutter. Here, the current frame of ViSAR is subtracted from the previous one to cancel the clutter and noise. As can be seen, the results of the difference are much poorer due to the change of view.
• “bkMEAN” subtracts the average background from the current ViSAR frame so that the clutter cancellation can be achieved. It is also widely used in previous work. The results are not satisfactory. The moving targets are mixed with the background clutter.
• “PCA-L” and “PCA-S” represent the image reconstruction via the 95% eigenvalue energy and the corresponding residual. It prioritizes the directions where the data vary the most because more variation usually indicates more useful information.

In this paper, we decompose the image of ViSAR as the low-rank and sparse components by constraining their ranks so that the heavy level of clutter can be suppressed. As can be seen from the final two rows in Figure 6, the clutter has been removed, while the targets have simultaneously been enhanced. On the other hand, it can be observed that the “PC-LR” reflects the global, slowly varying background information (such as clutter and noise), and the “PC-SP” expresses the localized mutable information (such as moving objects, or salient targets). These results also corroborate the analysis in Section 3.2.

4.3.2. Quantitative Experiments

We further verify the effectiveness of the clutter suppression technique from the perspective of target detection. Several rounds of experiments were performed. The methods commonly used in prior work were employed as the baseline. The results are shown in

Table 5. Comparative experiments for clutter suppression.

Method	Precision	Recall	F1	mAP
bkMEAN	0.0937	0.0567	0.0706	0.0436
bkDIFF	0.0831	0.0736	0.0781	0.0352
PCA	0.7863	0.5586	0.6532	0.6622
Proposed	0.8280	0.7156	0.7677	0.7652

.

The results of the quantitative experiments are consistent with the visualizations. While the classical methods, “bkMEAN” and “bkDIFF” are commonly used in previous work [36], their performance is much poorer. This is because the imaging scenes of ViSAR differ from those of multichannel SAR systems. The imaging views are changed dynamically. Therefore, the consecutive frames must be registered before the difference operations. These problems can be circumvented using principal component decomposition. For example, the results of PCA are much better than those of classical methods. The introduction of principal analysis improved the mAP metric to 66.22%. It achieves nearly a 60% improvement compared to classical tricks. On this basis, the presented method further introduces the low-rank and sparse constraints. The mAP metric has been improved to 76.52% and is 10.30% better than that of the classical principal component analysis technique.

4.4. The Comparative Studies

In this section, the overall detection performance of the proposed method is verified by several rounds of comparative studies. Some state-of-the-art target detection algorithms are used as baselines. These methods include YOLO5, YOLO7, and Faster-RCNN. Two versions of the detection method, “L” and “X”, are considered. The former refers to the slim network, while the latter refers to the full network.

4.4.1. Quantitative Experiments

The results of the quantitative experiments on moving target detection are shown in

Table 6. Comparative Experiment Based on SNL Video-SAR.

Method	Precision	Recall	F1	mAP
Faster-RCNN	0.3219	0.2917	0.3617	0.2846
YOLO5L	0.4367	0.3951	0.4149	0.3770
YOLO5X	0.5458	0.4951	0.5192	0.4432
YOLO7L	0.4936	0.3676	0.4214	0.4130
YOLO7X	0.7987	0.5586	0.6574	0.7172
PC-YOLO	0.8280	0.7156	0.7677	0.7652
$\uparrow \to$	3.02%	13.86%	14.00%	4.80%

. The proposed method is compared with the baseline algorithms. The two-stage detection method, Faster R-CNN, demonstrates poorer performance, with a mAP measurement of 0.2846. This proves that the generic method is not suitable for the task of target detection in heavy clutter environments. The accuracy obtained by the two versions of the slim network, YOLO5L and YOLO7L, is poorer than that of the full network, because the feature learning ability has been limited by the trimmed network. For methods with the full network configured, YOLO5X and YOLO7X, the recall rates are poor. The results demonstrate that many more false alarms are present in their detection results.

The proposed method, PC-YOLO, achieves the best scores across all metrics. Precision, Recall, F1-score, and mAP are 3.02%, 13.86%, 14.00%, and 4.80% higher than those of the nearest competitors, respectively. This is because the clutter has been suppressed, while the moving targets have simultaneously been enhanced.

4.4.2. The Visualization of Detection Results

We then show the visual detection performance obtained by the methods under study. The results are displayed in Figure 7. We find that both false-positive predictions and false-negative inferences are present in these maps.

• In Figure 7b,c, many false alarms denoted by the green rectangle are produced.
• In Figure 7d,e,g, some moving targets (shadow) have been missed.
• Although all of the targets have been located in Figure 7f,h, the confidence values for (f) are much lower than for (g).

In traditional background-based methods, numerous false alarms are produced in cluttered cells. They are represented by the green rectangle. The results prove that these methods cannot achieve the task of detecting a moving target in complex environments. For the deep learning strategies—Faster-RCNN, YOLO5, and YOLO7—some missing targets are produced. They fail to detect many targets with low SNR. In contrast, the proposed method (PC-YOLO) achieves the most accurate detection results, indicating its strong detection capability in highly cluttered environments.

To verify the proposed method further, ViSAR frames at different angles were considered. The set of visualization results is shown in Figure 8. As can be seen, all of the moving targets in different scenarios can be located accurately by the proposed method. This is because both the empirical model and the historical data were unified in the proposed framework. An explainable learning architecture can be formed accordingly.

5. Conclusions

This paper presents a mode- and data co-driven learning method (PC-YOLO) to address moving-target detection in video SAR. It has been decoupled into two sub-tasks: the clutter removal and the shadow extraction. The former is achieved through low-rank and sparse decomposition, while the latter is solved using the hierarchical attention mechanism and cross-scale feature fusion. Multiple rounds of experiments are performed, from which some conclusions can be drawn.

• It is not effective to apply the generic methods to video SAR data for moving target detection.
• The clutter removal plays an important role in the proposed method, because the clutter in the SAR image makes the detection task difficult.

In the future, an intriguing question is how to combine clutter suppression and shadow extraction into a unified framework so that clutter removal and target detection can be boosted synchronously. On the other hand, how to exploit information across frames remains to be studied.