Refined Deformable-DETR for SAR Target Detection and Radio Signal Detection

Abstract

SAR target detection and signal detection are critical tasks in electromagnetic signal processing, with wide-ranging applications in remote sensing and communication monitoring. However, these tasks are challenged by complex backgrounds, multi-scale target variations, and the limited integration of domain-specific priors into existing deep learning models. To address these challenges, we propose Refined Deformable-DETR, a novel Transformer-based method designed to enhance detection performance in SAR and signal processing scenarios. Our approach integrates three key components, including the half-window filter (HWF) to leverage SAR and signal priors, the multi-scale adapter to ensure robust multi-level feature representation, and auxiliary feature extractors to enhance feature learning. Together, these innovations significantly enhance detection precision and robustness. The Refined Deformable-DETR achieves a mAP of 0.682 on the HRSID dataset and 0.540 on the spectrograms dataset, demonstrating remarkable performance compared to other methods.

1. Introduction

Radar, particularly Synthetic Aperture Radar (SAR), is a tool that detects the distance, velocity, and altitude of objects by emitting electromagnetic waves, offering advantages over traditional optical sensors [1,2]. In recent years, with the advancement of SAR imaging technology and the improvement of image resolution, SAR target detection has become a hot topic, being part of SAR image post-processing. The goal of SAR target detection is to extract the orientation and location of targets from complex scenes quickly and effectively. This is crucial for Synthetic Aperture Radar automatic target recognition, where the accuracy of detection determines the success of subsequent tasks. This study is applicable to not only SAR target detection but also signal detection, aiming to enhance the ability to identify signals and targets in complex electromagnetic environments through deep learning techniques. Signal detection and recognition in dynamic electromagnetic environments is a critical field of study within electronic engineering, driven by the exponential growth of wireless technology usage across sectors. In a wireless monitoring system, signal detection is fundamental and important, serving as the basis for signal recognition, direction, and spectral analysis. However, the electromagnetic environment is complex, presenting significant challenges to detection. Signal detection can be divided into time and frequency domain detection. Additionally, deep learning has demonstrated superior performance in signal detection, enabling automatic feature extraction, improving detection accuracy. Wireless radio signal classification and modulation recognition are crucial issues in intelligent signal processing, holding significance for managing electromagnetic spectrum resources and ensuring reliability and security of communication systems. Researchers have proposed various methods for wireless radio signal identification to improve classification and modulation recognition accuracy.

In recent years, deep learning has been extensively applied across various industries, demonstrating superior performance relative to traditional methods. Many researchers globally have incorporated deep learning techniques into signal detection. O’Shea pioneered using convolutional neural networks (CNN) [3] and deep neural networks (DNN) in radio communications for feature learning and signal recognition [4]. Deep learning-based signal recognition methods typically utilize the spectro-temporal image obtained after short-time Fourier transformation (STFT) [5] as input and output the spectro-temporal information. Prasad et al. [5] used a reduced version of Faster-RCNN [6] for radio signal detection to reduce network size. Another study [7] applied YOLO object detectors to spectrum images to discover signals including Zigbee, Bluetooth, and Wi-Fi from the Internet of Things (IoT). Rahman et al. proposes a Bi-LSTM-based channel estimation and signal detection method, demonstrating its effectiveness in RIS-supported multi-user SISO OFDM systems [8]. Compared to traditional detection algorithms requiring manual feature extraction, deep learning methods leverage convolutional neural networks with superior feature extraction capabilities, enabling richer feature collection.

Despite the remarkable performance of Transformer-based models in the field of image processing and the potential they exhibit in SAR target detection and electromagnetic signal detection, they have not yet incorporated prior knowledge from signal processing. In this study, we introduce the latest deep learning models to enhance the accuracy and efficiency of signal modulation identification. These new models not only demonstrate superior performance in image processing and object recognition, particularly outperforming traditional methods in terms of processing speed and accuracy, but also overcome the high misclassification rates and slow response times of traditional models in specific scenarios through innovative algorithm optimization. Our key innovations are as follows.

• Half-window filtering technology: By proposing the half-window filter (HWF), which leverages prior knowledge from SAR signal processing, the feature extraction process is enhanced, significantly improving the recognition performance for targets of different scales. While Transformer-based models have demonstrated strong performance in image processing, SAR target detection and electromagnetic signal detection, they have not yet fully integrated SAR signal processing priors, which this approach addresses.
• Auxiliary feature extractors: During training, auxiliary feature extractors are introduced to provide additional supervision signals, enhancing the model’s encoding and decoding capabilities. Traditional object detection algorithms have shown poor performance on SAR target detection tasks. The introduction of auxiliary feature extractors significantly improves this by providing more robust feature learning and enhancing overall recognition accuracy.
• Multi-scale adapter: The multi-scale adapter dynamically constructs feature pyramids through upsampling and downsampling operations, enhancing multi-scale feature alignment and significantly improving detection accuracy for targets of varying sizes and scales.

2. Related Work

2.1. Detection Transformer

Following the success of convolutional neural networks (CNNs) and deep neural networks (DNNs), the Transformer architecture [9] has garnered widespread attention for its exceptional sequence modeling capabilities. It has been successfully applied to various visual tasks, including object detection. Transformer-based models, such as DETR [10] and its variants, not only offer new perspectives and approaches for object detection but also demonstrate great potential in signal detection and recognition. Despite these advances, existing detection models still face challenges, such as detection accuracy, model convergence speed, and the recognition of small or partially occluded objects in complex scenarios.

The primary objective of this study is to introduce improved attention mechanisms and propose a novel object detection model aimed at enhancing performance in dynamic and complex environments. Specifically, the model seeks to improve detection of small or partially occluded objects or signals. Detailed technical analyses and comparisons will be provided in subsequent sections to establish a solid theoretical and technical foundation for this work.

Detection Transformer (DETR) [10] treats object detection as a direct set prediction problem. By leveraging the Transformer encoder–decoder architecture and a global loss function, DETR infers object relationships and global image context through a set of learned object queries. This approach avoids many hand-designed components, offering a more end-to-end solution. On the COCO object detection dataset, DETR achieves accuracy and runtime performance comparable to Faster R-CNN [11].

Originally designed for natural language processing (NLP) tasks, the Transformer’s unique self-attention mechanism is well-suited to handling sequential data such as images and sound signals. In signal detection tasks, Transformers excel at capturing global characteristics of signals, enabling more accurate feature extraction and classification. DETR brings the Transformer’s self-attention mechanism to object detection, simplifying the traditional detection pipeline by directly predicting both object category and location, thus eliminating the need for non-maximum suppression (NMS) [12].

Further advancements, such as Deformable-DETR [13], optimize this architecture with a variable attention mechanism, improving adaptability to diverse object sizes and shapes. Conditional DETR [14] accelerates convergence and enhances detection efficiency through conditioned decoding. Additionally, DAB-DETR (Deformable Attention-Based DETR) [15] increases flexibility and accuracy in handling complex scenes by introducing deformable attention modules. This is especially valuable in complex electromagnetic environments, where such advancements significantly improve signal classification and recognition performance.

2.2. SAR Target Detection

In the field of SAR (Synthetic Aperture Radar) target detection, researchers have been striving to overcome challenges posed by environmental noise, sensor errors, and variations in target scale and shape. Early SAR target detection methods primarily relied on feature-based techniques, such as the extraction of texture, edge, and shape features. These methods were often combined with traditional machine learning classifiers, such as support vector machines (SVM) and random forests, for target recognition [16]. However, these approaches often required complex feature engineering and were limited in handling complex backgrounds and varying targets.

In recent years, the rise of deep learning has significantly improved the effectiveness of SAR target detection. Convolutional neural networks (CNNs) have become the most commonly used tool because they can automatically learn effective spatial features from raw SAR images, eliminating the need for manual feature extraction required by traditional methods [17]. Advanced object detection algorithms, such as Faster R-CNN and YOLO, have been successfully applied to SAR images, showing higher detection accuracy and speed [18,19]. Additionally, generative adversarial metworks (GANs) have been applied in SAR target detection for data augmentation and noise reduction, thereby improving the robustness and generalization ability of the models [20].

Despite the significant progress made by deep learning methods in SAR target detection, there are still many challenges. In particular, improving model performance with limited data remains a crucial issue. Many methods perform poorly in few-shot learning and when labeled data is scarce. As a result, effective data augmentation and the use of unsupervised learning methods have become hot topics in current research [21]. Furthermore, the environmental variability and diversity of targets in SAR images require detection models to have greater adaptability, and techniques such as transfer learning and domain adaptation are progressively being applied [22].

2.3. Filtering Technology

Filtering technology has widespread applications in image processing [23]. Traditional filtering methods like Gaussian [24], mean [25], and median filtering reduce image noise through local processing, but lack adaptability. Modern filter techniques incorporate deep learning methods like the one proposed in the literature, using fast discrete filters based on Gaussian curvature, mean curvature, and total variation (TV) [26] regularization to reduce variational energy. These methods effectively reduce noise and preserve features via pixel-level filters. In deep learning, ResNet [27] has achieved significant success in tasks like image classification and feature extraction through its residual module. Research combining filter technology and deep learning suggests adaptive filter layers can enhance feature extraction, improving model performance.

2.4. Auxiliary Supervision

Auxiliary supervision enhances model training by introducing extra supervisory signals, which improve feature learning and overall detection and recognition performance. In self-supervised learning, auxiliary supervision combined with self-supervised tasks enhances feature learning efficiency [28]; in generative adversarial networks, auxiliary supervision improves image generation quality through conditional generation [29]; in reinforcement learning, auxiliary tasks accelerate policy learning and model convergence [30,31]. For instance, Zong et al. [32] used auxiliary supervision signals through collaborative hybrid assignments training (CHAT) to improve the DETR model’s detection efficiency and accuracy. Zhang, Junbo and Tian [33] introduced the SUGAR algorithm, using two complementary heterogeneous networks and additional supervision signals during training to strengthen feature learning. The auxiliary feature extractor generates extra classification and regression losses, optimizing the feature learning process and improving the encoding and decoding capabilities of the model, thus enhancing recognition performance.

3. Method

As shown in Figure 1, our model processes input images to produce final outputs. The image is initially processed by the main network and Transformer encoder, generating feature maps. These feature maps are then processed by the multi-scale adapter to create multi-scale features. Further processing by auxiliary feature extractors generates classification and regression predictions. By comparing these predictions with real labels, we calculate classification and regression losses. The total loss optimizes model parameters, enhancing feature learning and improving performance.

3.1. Half-Window Filter

The half-window filter (HWF) is a learnable operator that dynamically adapts to SAR image characteristics through end-to-end training. Unlike traditional fixed convolution kernels, its projection weights are optimized via backpropagation to balance edge preservation and noise suppression. HWF is designed to reduce the variational energy while maintaining image edges. Its basic concept is to use the pixel local solutions of the regularization operator as the projection operation, and to reduce the regularization energy by approximating the image as a constant, minimum or expandable surface within a local window.

Given the input image I, after passing through the $\mathrm{HWF}$ and $\mathrm{CNN}$, we obtain the feature map F.

$$F=Encoder\left(CNN\right(HWF\left(I\right)\left)\right)$$

(1)

As shown in Equation (1), after passing the input image I through the $\mathrm{HWF}$ and $\mathrm{CNN}$, we obtain the feature map F.

The HWF works through two main steps. First, in the $3\times 3$ neighborhood of each pixel, the image is approximated locally as a constant, minimal surface, or developable surface through local regression. Second, among all possible local approximations, the one that results in the smallest change in the central pixel intensity is selected, thereby minimizing the increase in data fitting energy.

The following equations describe the implementation of the projection operator ${\mathcal{P}}_m$ used in the HWF. Let $U(i,j)$ represent the value of the input function at the pixel (or index) $(i,j)$. The surrounding values can be defined as follows: $U_{i,j}=U(i,j),U_{i-1,j}=U(i-1,j),U_{i+1,j}=U(i+1,j),U_{i,j-1}=U(i,j-1),U_{i,j+1}=U(i,j+1),U_{i-1,j-1}=U(i-1,j-1),U_{i+1,j-1}=U(i+1,j-1),U_{i-1,j+1}=U(i-1,j+1),U_{i+1,j+1}=U(i+1,j+1)$.

The projection operator ${\mathcal{P}}_m$ is applied as follows. The coefficients in (2)–(5) are derived based on a local regression approach within a $3\times 3$ window, aiming to approximate the image as a constant, minimal, or developable surface. The weighting factors ${\displaystyle \frac{5}{16}},{\displaystyle \frac{5}{8}},{\displaystyle \frac{1}{8}}$ are chosen to minimize the local variation energy while preserving edge structures. The computed differences $d_1,d_2,d_3,d_4$ correspond to directional variations, and the smallest one ($d_m$) is used to adjust the central pixel value, ensuring minimal distortion while effectively suppressing noise.

$$d_1={\displaystyle \frac{5}{16}}(U_{i-1,j}+U_{i+1,j})+{\displaystyle \frac{5}{8}}{U}_{i,j+1}-{\displaystyle \frac{1}{8}}(U_{i-1,j+1}+U_{i+1,j+1})-U_{i,j}$$

(2)

$$d_2={\displaystyle \frac{5}{16}}(U_{i-1,j}+U_{i+1,j})+{\displaystyle \frac{5}{8}}{U}_{i,j-1}-{\displaystyle \frac{1}{8}}(U_{i-1,j-1}+U_{i+1,j-1})-U_{i,j}$$

(3)

$$d_3={\displaystyle \frac{5}{16}}(U_{i,j-1}+U_{i,j+1})+{\displaystyle \frac{5}{8}}{U}_{i-1,j}-{\displaystyle \frac{1}{8}}(U_{i-1,j-1}+U_{i-1,j+1})-U_{i,j}$$

(4)

$$d_4={\displaystyle \frac{5}{16}}(U_{i,j-1}+U_{i,j+1})+{\displaystyle \frac{5}{8}}{U}_{i+1,j}-{\displaystyle \frac{1}{8}}(U_{i+1,j-1}+U_{i+1,j+1})-U_{i,j}$$

(5)

$$d_m=\min \left(|d_1|,|d_2|,|d_3|,|d_4|\right)$$

(6)

$${\widehat{U}}_{i,j}=U_{i,j}+d_m$$

(7)

where $d_1$, $d_2$, $d_3$, and $d_4$ represent the computed differences based on the surrounding values of $U(i,j)$, and $d_m$ is the minimum absolute difference. The final result, $\widehat{U}(i,j)$, is obtained by adding $d_m$ to the original value $U(i,j)$, which adjusts the value based on the neighboring pixel differences.

In our model, the HWF serves as a layer within the CNN and is trained via backpropagation using standard error. This method harnesses deep learning techniques for image processing and feature extraction while significantly reducing variational energy, thereby enhancing model performance and training efficiency. This approach effectively supplants traditional convolution kernels, utilizing deep learning methodologies to achieve superior image processing and feature extraction.

3.2. Multi-Scale Adapter

In object detection tasks, the multi-scale adapter, a specialized tool, is strategically utilized to transform single-scale or multi-scale features. These features are meticulously processed and output by the encoder into a multi-scale feature pyramid. The adapter’s functionality significantly enhances the ability of auxiliary feature extractors to meet their specific needs. This process results in a more effective and efficient object detection system.

As shown in Figure 2, For the single-scale encoder output, the feature pyramid is first constructed through bilinear interpolation and convolution operations. Let the feature map output by the encoder be F. We construct the feature pyramid $\{F_1,···,F_J\}$ through the following steps.

In the process of upsampling the feature map, bilinear interpolation is applied. For each feature map $F_i$, the upsampling operation can be expressed as Equation (8).

$$F_{i+1}=BilinearInterpolate\left(F_i\right)$$

(8)

After upsampling, the feature map is further processed with a $3\times 3$ convolution. The convolution operation is defined as Equation (9).

$$F_{i+1}=Con{v}_{3\times 3}\left(F_{i+1}\right)$$

(9)

As shown in Equation (8), the feature map is upsampled using bilinear interpolation. Then, as described in Equation (9), the upsampled feature map undergoes a convolution operation.

For the multi-scale encoder, we only downsample the coarsest feature in the multi-scale encoder features to build the feature pyramid. For the multi-scale encoder output, assuming the coarsest feature map is $F_{coarse}$, we construct the feature pyramid through downsampling operations:

Start with the coarsest feature map $F_{coarse}$. The initial feature map is given.

$$F_1=F_{coarse}$$

(10)

Next, apply a $3\times 3$ convolution with stride 2 to the feature map to downsample it. This downsampling operation can be expressed.

$$F_{i+1}=Con{v}_{3\times 3,stride=2}\left(F_i\right)$$

(11)

Repeat this downsampling step as necessary to construct the feature pyramid.

As shown in Equation (10), we start with the coarsest feature map $F_{coarse}$. Then, as described in Equation (11), a downsampling operation is applied to the feature map.

The multi-scale adapter is crucial for object detection tasks, as it transforms single-scale or multi-scale features into a multi-scale feature pyramid. This enhances the ability of auxiliary feature extractors to meet their needs, improving the effectiveness and efficiency of the object detection system. The adapter achieves this through bilinear interpolation and convolution operations or downsampling the coarsest feature map, constructing multi-level feature representations. In summary, the multi-scale adapter greatly facilitates performance optimization and training convergence of object detection models.

3.3. Auxiliary Feature Extractor

In modern object detection frameworks, feature extraction and accurate prediction play crucial roles in achieving high performance. Auxiliary feature extractors are introduced to provide additional supervision signals during the training phase, thereby enhancing the learning of features. This section explains the process and significance of using auxiliary feature extractors in our model.

Given the input image I, after passing through the Backbone (including the HWF and convolutional neural network (CNN)) and the Transformer encoder, we obtain the feature map F.

$$F=Encoder\left(CNN\right(HWF\left(I\right)\left)\right)$$

(12)

Multi-scale adapter processes the feature map F, generating multi-scale features $F_{multi}$.

$$F_{multi}=MultiScaleAdapter\left(F\right)$$

(13)

Each auxiliary feature extractor processes the multi-scale features, generating classification predictions $C_k$ and regression predictions $R_k$.

$$C_k,R_k=A_k\left(F_{multi}\right)$$

(14)

where k denotes the k-th auxiliary feature extractor.

The use of auxiliary feature extractors enhances feature learning by providing additional supervision signals during training. Unlike traditional data augmentation, which increases data diversity through geometric and color transformations, auxiliary feature extractors generate extra labels and features to improve the model’s encoding and decoding abilities. This method enhances feature-level diversity and information content, leading to improved model performance and generalization.

We use three different auxiliary feature extractors in our experiments: ATSS, YOLOv3 and Faster-RCNN. ATSS (Adaptive Training Sample Selection) [34] bridges the gap between anchor-based and anchor-free detection methods, offering adaptability and robustness in various scenarios. YOLOv3 [35] is a single-stage detector that predicts multiple object bounding boxes and categories in a single forward pass, excelling in computational efficiency. Faster-RCNN [11] is a region-based convolutional neural network that combines a region proposal network (RPN) with a Fast R-CNN detector. It provides higher detection accuracy but requires more computational resources.

The selection of ATSS, YOLOv3, and Faster-RCNN as auxiliary feature extractors is driven by their complementary methodological frameworks in object detection research. ATSS represents state-of-the-art anchor-free detection with adaptive training sample selection, YOLOv3 exemplifies efficient single-stage detection through its real-time optimization paradigm, while Faster-RCNN stands as the seminal two-stage detector with precise region proposal mechanisms. This combination ensures comprehensive feature supervision by spanning: global semantic modeling (ATSS) versus local discriminative analysis (Faster-RCNN), and computational efficiency (YOLOv3) versus high-precision localization capabilities. Their synergistic integration addresses the intrinsic challenges of SAR targets, which demand both context-aware interpretation and multi-scale sensitivity.

3.4. Loss Function and Optimization

In deep learning, the loss function and optimization are central to the training process. The loss function quantifies the discrepancy between the model’s predictions and the ground truth, while optimization methods are used to adjust the model parameters to minimize this discrepancy. In this section, we discuss the loss functions used in our model, the total loss function, and the optimization process.

The loss function in our model consists of two main components: classification loss and regression loss. These losses are calculated separately for each auxiliary feature extractor. The classification loss measures the difference between the predicted class probabilities and the true class labels, while the regression loss measures the difference between the predicted bounding box coordinates and the ground-truth coordinates.

In object detection, the classification loss $L_{cls}^k$ measures the difference between the predicted class and the ground truth class label, typically calculated using cross-entropy loss. The formula is as Equation (15).

$$L_{cls}^k={\mathcal{L}}_{cls}(C_k,G_{cls})=-\sum _i{G}_{cls}^i\log \left(C_k^i\right)$$

(15)

where $C_k$ is the predicted class output from the k-th auxiliary feature extractor, and $G_{cls}$ is the ground truth class label.

The regression loss $L_{reg}^k$ measures the difference between the predicted bounding box and the ground truth bounding box, typically computed using smooth L1 loss. The formula is as Equation (16).

$$L_{reg}^k={\mathcal{L}}_{reg}(R_k,G_{reg})=\sum _i{\mathbb{1}}_{\{i\in matched\}}{\mathcal{L}}_{smooth\_L1}(R_k^i,G_{reg}^i)$$

(16)

where $R_k$ is the regression output (e.g., bounding box coordinates) from the k-th auxiliary feature extractor, $G_{reg}$ is the ground truth bounding box coordinates, and ${\mathbb{1}}_{\{i\in matched\}}$ is an indicator function that checks whether the predicted and ground truth boxes are matched.

The classification loss $L_{cls}^k$ for the k-th auxiliary feature extractor is calculated as Equation (17).

$$L_{cls}^k={\mathcal{L}}_{cls}(C_k,G_{cls})$$

(17)

Similarly, the regression loss $L_{reg}^k$ is computed as Equation (18).

$$L_{reg}^k={\mathcal{L}}_{reg}(R_k,G_{reg})$$

(18)

where $C_k$ and $R_k$ are the predicted class and bounding box coordinates, and $G_{cls}$ and $G_{reg}$ are the ground truth class labels and bounding box coordinates.

The total loss function $L_{total}$ is the weighted sum of the classification and regression losses from all auxiliary feature extractors. It is formulated as Equation (19).

$$L_{total}=\sum _{k=1}^K(\alpha L_{cls}^k+\beta L_{reg}^k)$$

(19)

Here, $\alpha$ and $\beta$ are the weighting coefficients, which balance the contributions of the classification and regression losses. The coefficients $\alpha$ and $\beta$ are empirically set to 0.8 and 0.2, respectively, prioritizing localization stability while maintaining classification accuracy.

To minimize the total loss function $L_{total}$, we use an optimization algorithm to update the model parameters during training. We employ the Adam optimizer, which combines the benefits of both adaptive learning rates and momentum, making it effective for training deep networks. The optimization process aims to minimize the total loss function by adjusting the model’s parameters iteratively.

The design of the loss function significantly impacts the optimization process. By balancing the classification and regression losses appropriately, we ensure that the model is not biased toward one task over the other. The loss function guides the optimization process toward finding the best model parameters that minimize the prediction errors for both classification and bounding box regression tasks. Furthermore, the auxiliary feature extractors help regularize the model by providing additional supervision signals, which enhances the optimization process and prevents overfitting.

In summary, the loss function and optimization play a crucial role in training our model, ensuring that it effectively learns to perform both classification and regression tasks while generalizing well to unseen data.

4. Results

4.1. Dataset and Experimental Settings

In this study, we utilized the HRSiD (high-resolution Synthetic Aperture Radar ship detection) [36] dataset, which is specifically designed for ship detection tasks using high-resolution Synthetic Aperture Radar (SAR) images. The HRSiD dataset contains a large number of high-resolution SAR images captured under various environmental conditions, aimed at evaluating the performance of ship detection algorithms in complex backgrounds. The dataset includes data from multiple voyages, covering a range of weather conditions, sea states, and different types of ships. Each image is meticulously annotated with detailed information, such as ship categories and locations, providing comprehensive support for model training and performance evaluation. In this research, we used the HRSiD dataset for both model training and testing, with the goal of validating the effectiveness and robustness of our proposed method in ship detection tasks.

To better illustrate the characteristics of the HRSiD dataset, we present two example images. Figure 3a shows a high-resolution SAR image of a ship under calm sea conditions, while Figure 3b depicts a ship in a more complex maritime environment with rough sea states. These images highlight the challenges of ship detection in SAR imagery, including varying ship types, sea states, and environmental conditions. The HRSID dataset contains 120 high-resolution SAR images (1 m spatial resolution, HH polarization) with diverse maritime scenarios, including open oceans, coastal areas, and harbor regions. Each image captures varying ship densities (5–32 vessels) and clutter interference levels (calm seas to stormy conditions), reflecting real-world operational challenges in SAR-based maritime surveillance.

Our experiments also use two-dimensional spectrograms generated from radio waves through a short-time Fourier transform (STFT). These serve as primary input for our model, with the horizontal axis representing signal frequency and the vertical axis denoting time. The signal is visually depicted as a rectangular block within the spectrogram for clarity. We also utilize the spectral color mapping scheme to enhance diagram efficacy. Specific parameter settings tailored for our dataset are as shown in

Table 1. Dataset parameters.

Parameter	Value
Total sample	12,500
Sampling rate	1024 kHz
Bandwidth	[10, 20, 50, 100, 150] Hz
Signal Power	[0.1, 0.125, 0.25, 0.5, 1, 2, 4, 8, 10, 100]
Modulation methods	BPSK, QPSK, 8PSK, OQPSK, 16PSK, 16QAM, 64QAM, 256QAM, 2FSK, 4FSK
Noise Power	1
Size of spectrograms	(512, 512)
Ratio of dataset	train:validation:test = 8:1:1

.

In our experiments, the 10 modulation modes are considered separately. The signal can be divided into two types: unaliased and aliased, as shown in Figure 4a. Figure 4a is a simple pattern of radio with no time and frequency aliasing, while Figure 4b is a complex pattern with time or frequency aliasing.

We conducted extensive trials to compare our model with other successful detectors. All experiments were carried out on a GeForce RTX 3070 GPU. We trained all detectors using the SGD optimizer, with a momentum of 0.937 and a weight decay of 0.00001. In our experiments, we adopted ATSS [34] as auxiliary feature extractors. We set the number of learnable object queries to 300 and the hyperparameters $\{{\lambda }_1,{\lambda }_2\}$ to the default values of $\{1.0,2.0\}$.

4.2. Comparative Results on HRSiD Dataset

Table 2. Comparison experiments on HRSID.

Method	mAP50	mAP
Faster RCNN	0.720	0.465
RetinaNet	0.789	0.536
YOLOv6n	0.882	0.628
YOLOv7-tiny	0.854	0.572
YOLOv8n	0.911	0.669
YOLOv11n	0.897	0.658
Yue et al. [37] *	0.911	0.665
CPoints-Net [38] *	0.905	-
FEPS-Net [39] *	0.907	0.657
BL-Net [40] *	0.867	-
FBUA-Net [41] *	0.903	-
${\mathrm{CS}}^3$Net [42] *	0.912	0.660
Refined Deformable-DETR	0.902	0.682

* Denotes data obtained from their papers.

presents the results of the comparative experiments conducted on the HRSiD dataset, evaluating different methods using ${\mathrm{mAP}}_{50}$ and mAP (${\mathrm{mAP}}_{50:95}$). The results demonstrate that our proposed method achieves a mAP of 0.682, surpassing all other methods, including the previous state-of-the-art ${\mathrm{CS}}^3$Net (0.660). Additionally, our method achieves competitive performance in terms of ${\mathrm{mAP}}_{50}$, reaching 0.902, which is comparable to ${\mathrm{CS}}^3$Net (0.912) and Yue et al. [37] (0.911).

4.3. Comparative Results on Spectrogram Dataset

As shown in

Table 3. Comparison of different methods on spectrograms dataset.

Method	Backbone	AP	AP50	AP75	APS	APM	APL
CenterNet	Resnet18	0.169	0.299	0.160	0.122	0.301	0.417
Faster-RCNN	Resnet50	0.139	0.164	0.156	0.037	0.417	0.551
YOLOv3	Darknet	0.004	0.011	0.003	0.001	0.012	0.102
CornerNet	HourglassNet	0.207	0.315	0.217	0.162	0.335	0.421
DETR	Resnet18	0.266	0.513	0.238	0.170	0.503	0.780
DETR	Resnet50	0.086	0.178	0.076	0.051	0.201	0.770
Deformable-DETR	Resnet50	0.257	0.405	0.273	0.225	0.372	0.997
DAB-DETR	Resnet50	0.211	0.391	0.195	0.170	0.360	0.898
Conditional-DETR	Resnet50	0.112	0.223	0.093	0.078	0.221	0.788
Refined Deformable-DETR	Resnet50	0.540	0.804	0.586	0.462	0.791	0.986

, the proposed Refined Deformable-DETR outperforms a range of DETR-based detectors and other popular object detection frameworks on the spectrogram dataset. It achieves the best performance across all evaluation metrics: AP = 0.540, AP₅₀ = 0.804, and AP₇₅ = 0.586.

To further analyze the model’s performance across different modulation schemes, we present the confusion matrix in Figure 5. From the confusion matrix, we observe that the model performs exceptionally well on certain classes, such as GMSK and 64QAM, with classification accuracies of 66% and 89%, respectively. This indicates that these modulation schemes have distinct characteristics, making them easier for the model to classify correctly.

4.4. Ablation Study

To assess the contribution of the HWF module and auxiliary feature extractors, we conducted ablation experiments on both the HRSiD and spectrogram datasets. As shown in

Table 4. Comparison of auxiliary extractors on HRSID.

Auxiliary Feature Extractor	HWF	AP	AP50	Inference Time (ms)
YOLOv3 based AFE	enabled	0.682	0.902	45.3
	disabled	0.619	0.887	38.7
Faster-RCNN based AFE	enabled	0.602	0.858	72.1
	disabled	0.587	0.822	66.5
ATSS based AFE	enabled	0.677	0.882	58.9
	disabled	0.652	0.837	50.4

, enabling HWF consistently improves performance. For instance, with YOLOv3 as the extractor, AP improves from 0.619 to 0.682 and ${\mathrm{AP}}_{50}$ from 0.887 to 0.902.

Similarly, results on the spectrogram dataset (

Table 5. Comparison of auxiliary extractors on spectrograms dataset.

Auxiliary Feature Extractor	HWF	AP	AP50	Inference Time (ms)
YOLOv3 based AFE	enabled	0.540	0.804	41.2
	disabled	0.525	0.794	35.6
Faster-RCNN based AFE	enabled	0.527	0.782	69.8
	disabled	0.499	0.664	63.4
ATSS based AFE	enabled	0.533	0.771	55.7
	disabled	0.510	0.752	48.3

) show consistent improvements when HWF is enabled. Notably, ATSS achieves the best performance with AP = 0.533 and ${\mathrm{AP}}_{50}$ = 0.771 when using HWF.

The inference time analysis shows that while enabling HWF introduces a slight computational overhead (e.g., 5–7 ms increase), the performance gains are substantial. This highlights the utility of HWF in precision-critical tasks.

5. Discussion

The extensive experimental results confirm the superior performance and practical potential of the proposed Refined Deformable-DETR model. This section elaborates on the comparative advantages, robustness analysis, ablation insights, and remaining challenges, based on both quantitative metrics and qualitative observations.

5.1. Performance Comparison with State-of-the-Art

Across both SAR (HRSiD) and spectrogram-based detection tasks, our method achieves the best overall mAP scores (0.682 on HRSiD and 0.540 on spectrograms), outperforming not only classical detectors (e.g., Faster R-CNN, YOLO variants) but also Transformer-based baselines (DETR, Deformable-DETR, DAB-DETR, Conditional-DETR). In particular, although ${\mathrm{CS}}^3$Net and Yue et al. [37,42] report high ${\mathrm{mAP}}_{50}$ values, their mAP (under stricter IoU thresholds) falls short, indicating that our model achieves better localization precision under challenging conditions.

5.2. Robustness and Generalization

The results indicate that Refined Deformable-DETR generalizes well across domains—from maritime SAR scenes to complex signal spectrograms. Its resilience to spatial deviations, as demonstrated by the strong high-IoU performance and confusion matrix analysis, makes it particularly suitable for scenarios requiring accurate region localization, such as ship detection in cluttered sea conditions or fine-grained signal demodulation in spectrum monitoring.

5.3. Effectiveness of the HWF Module

The ablation studies (Table 4 and Table 5) clearly demonstrate the contribution of the proposed Half-Window Filtering (HWF) module. Across all backbone extractors (YOLOv3, ATSS, Faster R-CNN), HWF leads to consistent improvements in both AP and ${\mathrm{AP}}_{50}$, at the cost of only minor increases in inference time. These results suggest that HWF effectively enhances temporal-spectral attention and can be considered a lightweight yet powerful plug-in module.

5.4. Remaining Limitations and Future Work

Despite strong results, challenges remain. As revealed in the confusion matrix (Figure 5), certain modulation types—especially those with overlapping spectral patterns—remain difficult to classify. Future work may involve leveraging more expressive feature hierarchies, integrating contrastive learning signals, or applying cross-modal regularization. Moreover, deploying the system in real-time or edge settings will require further model compression and latency optimization.

6. Conclusions

We proposed Refined Deformable-DETR for SAR target detection and signal modulation recognition. Experimental results demonstrate the outstanding performance of Refined Deformable-DETR. By proposing the half-window filtering technique and the auxiliary feature extractor, Refined Deformable-DETR enhances detection performance, particularly for small, medium, and large targets. In the context of SAR target detection, the model effectively handles complex background environments, improving detection accuracy in cluttered or noisy scenarios. Moreover, the model shows robust performance in signal modulation recognition, further validating its effectiveness in electromagnetic environments. Future work will focus on optimizing the training process to enhance efficiency and exploring additional application scenarios to achieve more efficient and accurate signal detection and recognition, particularly in SAR-based tasks.