IR-SAM2: Target Enhancement with SAM2 for Infrared Small Target Detection

Highlights

What are the main findings?

• IR-SAM2 introduces a novel target enhancement framework for infrared small target detection (IRSTD) by integrating a Contrast Query Generator (CQG) and a Radial High-Pass Modulator (RHPM), effectively incorporating frequency domain context to suppress low-frequency background clutter and enhance small target saliency.
• IR-SAM2 achieves state-of-the-art performance on IRSTD-1k (IoU: 69.75%, $P_d$: 96.60%) and NUDT-SIRST (IoU: 94.18%, $P_d$: 98.94%), while striking an optimal balance between $P_d$ and $F_a$ on NUAA-SIRST, demonstrating superior robustness in complex backgrounds.

What are the implications of the main findings?

• This work validates the feasibility of adapting vision foundation models (SAM2) to specialized infrared tasks by bridging the domain gap through spatio-frequency learning, offering a new paradigm for high-precision segmentation of extremely small targets with sparse visual cues.
• The proposed target-centric CSA loss and frequency driven optimization provide a robust and transferable solution for real-world infrared monitoring systems, such as maritime surveillance and airspace early warning, where distinguishing dim targets from heavy low-frequency interference is a critical requirement.

Abstract

Foundation models such as the Segment Anything Model (SAM) have substantially advanced promptable object segmentation in remote sensing. However, extending these capabilities to infrared small target detection (IRSTD) remains highly challenging in the presence of severe background clutter and extremely low target visibility. In this paper, we propose IR-SAM2, an effective target enhancement framework for mask-level infrared small target segmentation in the IRSTD setting. Specifically, IR-SAM2 equips the SAM2 decoder with a dedicated frequency branch, facilitating simultaneous spatio-frequency learning and deep spatio-frequency fusion, while preserving SAM2’s pre-trained knowledge. Moreover, we introduce a target-centric loss to better guide the model in distinguishing small targets from complex backgrounds. Extensive experiments show that IR-SAM2 achieves highly competitive performance on the IRSTD-1k and NUDT-SIRST benchmarks, while striking an optimal balance between detection probability and false alarm rate on NUAA-SIRST. The results further demonstrate the effectiveness of spatio-frequency cues for complex-scene infrared small target segmentation. The source codes have been made publicly available to support reproducibility.

1. Introduction

As an important technology, infrared target detection (IRTD) [1] has been widely used in military and civilian fields, such as monitoring sea areas [2,3], infrared ship detection [4,5,6], and implementing early warning in airspace [7,8]. In this paper, we focus on mask-level infrared small target segmentation, often referred to as infrared small target detection (IRSTD). As illustrated in Figure 1, infrared small targets typically occupy only a few to several dozen pixels, which poses significant challenges for precise target-background separation. Furthermore, dominant low-frequency background components markedly diminish target contrast and visibility, rendering the decoder highly susceptible to background-induced activations and false positives in heavily cluttered environments. These impediments often lead to elevated false alarm rates or missed detections.

Meanwhile, vision foundation models have promoted artificial intelligence by leveraging massive datasets and Transformer-based architecture (e.g., LLMs) to achieve remarkable performance across various tasks. In computer vision, the Segment Anything Model [10] has emerged as a prominent foundation model, especially SAM2 [11], which excels in segmentation with its fine-grained contour capture and impressive zero-shot capabilities, requiring only sparse prompts like points or bounding boxes. This makes SAM2 suitable for downstream segmentation tasks with precise boundary detection, better generalization, and reduced annotation dependency.

In the context of IRSTD, due to the limited training samples, several previous works have explored different strategies for adapting SAM on IRSTD benchmarks. Typically, IRSAM [12] incorporates a Perona–Malik diffusion (PMD)-based block into SAM to better capture essential structural features while suppressing noise. Subsequently, SAM-SPL [13] introduced a unified self-prompt learning (SPL) framework to adapt SAM visual priors to IRSTD. However, due to the huge domain gap between the infrared and natural images, the adaptation of the fundamental model for IRSTD is constrained.

We observe that existing SAM-based adaptations primarily focus on spatial domain prompt engineering or structural tuning. Although these strategies are effective for natural images with rich textures and clear boundaries, they may still be insufficient to address the unique challenges of IRSTD. Infrared small targets typically occupy only a few to several dozen pixels and lack distinct structural semantics, manifesting merely as weak thermal radiation submerged in severe background clutter. Consequently, SAM’s native spatial attention mechanisms are easily overwhelmed by the dominant low-frequency energy of the background, leading to false activations and poor localization of dim targets. To bridge this fundamental domain gap, we argue that the target enhancement must go beyond the spatial domain and leverage the inherent physical characteristics of infrared imaging. From a frequency domain perspective, the distinction between targets and complex backgrounds becomes highly discriminative: large-scale background clutter is predominantly concentrated in low-frequency components, whereas small targets and local details inherently appear as high-frequency transients. By decoupling the feature representation into the frequency domain, we can explicitly isolate and suppress the overpowering low-frequency background interference before it misguides the Transformer decoder.

Motivated by this, we propose IR-SAM2, a SAM2-based framework that injects frequency domain cues into the decoder to enhance foreground target awareness in cluttered infrared scenes. Through a learnable high-pass filter module, IR-SAM2 can suppress low-frequency components that usually correspond to background noise. Moreover, IR-SAM2 introduces a target-centric CSA loss to guide the whole model to achieve robust perception across varying sizes and shapes. In this way, IR-SAM2 captures the small targets more accurately. Equipped with both feature-level and supervision-level enhancement strategies, our IR-SAM2 demonstrates superior performance across multiple IR datasets, including IRSTD-1k [14], NUAA-SIRST [15], and NUDT-SIRST [9].

Our main contributions are as follows:

• We augment SAM-based IRSTD frameworks with frequency domain context, which is helpful for foreground target enhancement. Specifically, we propose a filter module embedded within the critical paths of feature extraction and mask generation. By leveraging adaptive frequency domain filtering, this module suppresses low-frequency background clutter. Unlike existing frequency domain methods that rely on static thresholds or are limited to single-channel rectangular cutoffs, our module introduces multi-channel, radial high-pass filtering to accommodate isotropic target characteristics.
• We propose a target-centric CSA loss that shifts from conventional global image-level metrics to localized supervision centered on individual targets. By adaptively modulating loss contributions based on each target’s specific scale and local contrast, this approach ensures a more balanced optimization across all instances. This mechanism significantly improves instance-level separation in IR-SAM2, even in scenarios with dense target distributions.
• Extensive experiments across IRSTD benchmarks verify that our method generates highly discriminative features that capture instance-level semantics and morphological details by combining frequency domain refinement with localized adaptive supervision.

2. Materials and Methods

2.1. Related Work

In this section, we first survey the historical and current progress within the IRSTD field. After that, we introduce recent innovations in SAM adaptations and loss functions specific to IRSTD, which provide the theoretical foundation for our proposed methodology.

2.1.1. Infrared Small Target Detection

Existing infrared small target detection (IRSTD) methods are generally categorized into traditional model-driven approaches and data-driven deep learning approaches. Early model-driven methods primarily leveraged hand-crafted features and mathematical priors. For instance, filter-based approaches [16,17,18] apply spatial or frequency domain kernels to suppress background clutter. Meanwhile, methods inspired by the human visual system [19,20] enhance target saliency by measuring local contrast. Furthermore, optimization-based frameworks, such as low-rank and sparse decomposition [21,22,23,24,25], model an infrared image as the superposition of a low-rank background component and a sparse target component. Although traditional models offer strong interpretability and computational efficiency, their performance heavily depends on rigid prior assumptions, making them struggle to adapt to dynamic scenes.

With the advancement of deep learning, data-driven methods have become mainstream, shifting the focus toward designing sophisticated network architectures for discriminative feature extraction. For instance, ISNet [14] introduces Taylor finite differences to capture target shape features, DNANet [9] employs densely nested interaction modules for multi-level feature fusion, and ACMNet [15] proposes asymmetric contextual modulation. However, Convolutional Neural Networks often suffer from limited receptive fields and down-sampling operations, restricting the model’s ability to capture global context and low-frequency background interference. Liu et al. [26] pioneered the application of Transformers to IRSTD by modeling the long-range dependencies of convolutional features. Building upon this, Wu et al. [27] introduced a multi-scale fusion paradigm, proposing the Multi-scale Vision Transformer Model (MVTM). However, in scenarios where target sparsity is less pronounced or the background is severely disrupted by heavy clutter, these methods may exhibit elevated false alarm rates and excessive target suppression.

Recently, some methods based on deep unfolding have also emerged, such as RPCANet [28] and DRPCA-Net [29]. While DRPCA-Net overcomes the limitations of fixed parameters in conventional deep unfolding networks by introducing a hypernetwork-driven dynamic parameter generation mechanism and enhances the interpretability of infrared small target detection, its performance remains constrained by the strict consistency requirements of model-driven approaches regarding low-rank and sparsity priors. Nevertheless, these operations are primarily confined to the spatial domain and remain less sensitive to the inherent frequency characteristics of infrared images.

While some approaches like FDA-IRSTD [30], FSGPNet [31] and HLSR-Net [32] attempt to integrate frequency domain analysis by separating high and low frequencies, they often rely on fixed thresholds and therefore lack adaptability to varying low-frequency interference across scenes. To overcome the constraints of static frequency processing, HDNet [33] introduced the dynamic high-pass filtering (DHPF) module, which adaptively computes and removes a specified proportion of low-frequency energy. HDNet is limited to single-channel processing and employs a rectangular frequency cutoff, which may be suboptimal for capturing the isotropic characteristics typical of infrared small targets.

2.1.2. Segment Anything Model

The Segment Anything Model (SAM) [10] introduced a prompt-based segmentation framework that demonstrates strong zero-shot generalization under diverse prompt types. However, SAM’s practical application is often hindered by high computational overhead and limited efficiency in parallel processing. To address these efficiency bottlenecks, SAM2 [11] decouples the image encoder and mask decoder to enable batched prompt processing. Building on this foundation, several specialized variants have been developed: Dual-SAM [34] integrates dual encoders with multi-level coupled prompting to improve performance in specific scenarios, while MAS-SAM [35] employs encoder adapters and multi-scale feature extractors to strengthen representation capabilities.

Regarding IRSTD-specific tasks, IRSAM [12] attempts to redesign the SAM architecture; however, it focuses primarily on structural modifications, but its utilization of SAM’s general visual priors for IRSTD-specific contextual understanding remains limited. SAM-SPL [13] leverages pre-trained general knowledge from SAM to facilitate task-specific contextual understanding. However, SAM-based adaptations may still exhibit limited generalization and struggle to segment fine-grained morphological structures, which are critical for infrared small targets. While prompt learning offers a promising approach for efficient knowledge transfer by modulating backbone behavior via learnable vectors, its application in IRSTD remains challenging. Specifically, generating task-aware prompts that can simultaneously supplement low-level target details and coordinate with high-level semantic guidance remains an open problem in the field.

2.1.3. Loss Function in IRSTD

The design of the loss function plays a pivotal role in an IRSTD task. Conventional loss functions, such as BCE loss [36], IoU loss [37], and Dice loss [38], are susceptible to extreme class imbalance and exhibit limited sensitivity to minute targets. To mitigate scale inconsistency, the scale and location sensitive (SLS) loss [39] incorporates scale and location-sensitive terms; however, it fundamentally remains a global image-level objective. Consequently, SLS lacks the ability to explicitly guide the model to focus on the local regions surrounding each target or to capture the distinctive local contrast characteristics essential for infrared small target detection.

Recently, SD loss [40] and TDA loss [41] dynamically adjust weights for targets of different scales. The SD loss introduces a dynamic weighting coefficient that adaptively adjusts according to the target’s pixel size; meanwhile, TDA loss employs an image-patch-based mechanism and an adaptive weighting strategy that adjusts the contribution of each target. These loss-driven refinements provide a potent strategy for enhancing overall IRSTD performance without increasing the model’s computational complexity during inference.

2.2. Methodology

In this section, we first present the overall pipeline of our method in Section 2.2.1. Section 2.2.2 and Section 2.2.3 then detail the CQG and RHPM modules, respectively. Finally, Section 2.2.4 describes the model optimization.

2.2.1. Overall Architecture

As illustrated in Figure 2, the overall pipeline consists of four main stages. First, given the infrared image, multi-scale features F are extracted by a Hiera-based image encoder. Subsequently, the features are processed through a $1\times 1$ convolution to serve as the input $F_{in}$ for the following stages. Second, the Contrast Query Generator (CQG) employs a filtering module to transform the multi-scale feature maps $F_{in}$ into high-pass spatial features $F_{\lambda }$, and subsequently performs saliency sampling on this map to obtain dynamic queries $Q_{final}$. These queries then serve as the input for the two-branch Transformer decoder to localize targets and produce deep features $F_{deep}$. Third, these features are forwarded to a multi-scale decoder, where each scale uses independent convolutions with non-shared parameters to capture scale-specific mapping patterns. After spatial resizing and channel reduction, the decoder produces a set of multi-scale predicted masks $Mas{k}_i$ for subsequent stages. Fourth, a cascaded sequence of Radial High-Pass Modulator (RHPM) modules is integrated across the mask generation stages to progressively suppress low-frequency clutter and produce the final high-resolution segmentation mask $P_{final}$.

2.2.2. Contrast Query Generator

The performance of transformer-based decoders is heavily dependent on the quality of object queries. For infrared small target detection, directly using randomly initialized queries often leads to slow convergence in the early stages, especially for weak infrared targets. To mitigate this issue, we propose a Contrast Query Generator (CQG) that converts high-frequency salient responses into location-aware query tokens, facilitating more accurate target localization.

To extract frequency domain information, the CQG first utilizes a filtering module to suppress low-frequency components from the input multi-scale features $F_{in}\in {\mathbb{R}}^{H\times W\times C}$, yielding high-pass spatial features $F_{\lambda }\in {\mathbb{R}}^{H\times W\times C}$. These features are then compressed into a 2D energy map $S\in {\mathbb{R}}^{H\times W}$ via channel-wise averaging:

$$S(x,y)={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}{F}_{\lambda }^{\left(c\right)}(x,y).$$

(1)

Consequently, high-energy regions in $S(x,y)$ indicate potential target locations.

To capture all potential targets, we then flatten S into a vector and select the Top-K points with the largest values as the candidate set $P_{cand}$:

$$P_{cand}=\{(p_k,s_k){\}}_{k=1}^K,$$

(2)

where $p_k=(x_k,y_k)$ represents the normalized spatial coordinate (e.g., scaled to [0,1]) of the k-th point, and $s_k$ denotes the corresponding saliency intensity.

For each candidate point $p_k$, its spatial coordinates are mapped via sine and cosine functions to generate a positional embedding vector $PE\left(p_k\right)\in {\mathbb{R}}^D$, where D is the channel dimension of $F_{in}$, thereby facilitating subsequent two-branch interaction.

Unlike traditional Top-K queries that directly use multiple tokens, we leverage the saliency values $s_k$ to compute normalized aggregation weights with the Softmax function:

$${\alpha }_k={\displaystyle \frac{\exp \left(s_k\right)}{{\sum }_{k=1}^K\exp \left(s_k\right)}}.$$

(3)

Subsequently, the position embeddings of all candidate points are weighted and summed to obtain the aggregated position features $T_{agg}$:

$$T_{agg}={\displaystyle \sum _{k=1}^K}{\alpha }_k·PE\left(p_k\right).$$

(4)

This weighted aggregation can be interpreted as estimating the centroid of the salient high-frequency energy, allowing the query to focus on the most likely target region. Finally, $T_{agg}$ is fused with the learnable content query $Q_{task}$ via a linear projection, yielding the final query vector $Q_{final}$:

$$Q_{final}=Q_{task}+\mathrm{MLP}\left(\mathrm{LayerNorm}\left(T_{agg}\right)\right).$$

(5)

After that, $Q_{final}$, which encodes both task-related semantic context and a positional prior that indicates likely target locations, is fed into the Two-branch Transformer as the decoder input:

$$\begin{array}{cc}\hfill & Q_{final}^{\prime }=\mathrm{MLP}\left(\mathrm{CA}(\mathrm{SA}\left(Q_{final}\right),F_{in},F_{in})\right),\hfill \\ \hfill & F_{in}^{\prime }=\mathrm{CA}(F_{in},Q_{final}^{\prime },Q_{final}^{\prime }),\hfill \end{array}$$

(6)

where SA$(·)$, CA$(·)$, and MLP$(·)$ represent the self-attention, cross-attention, and multilayer perceptron modules, respectively.

After several iterations, the refined features $F_{in}^{\prime }$ undergo spatial upsampling to reconstruct fine-grained resolution. Specifically, a hypernetwork MLP is employed to transform the updated query $Q_{final}^{\prime }$ into a set of channel-wise scaling weights, which serve as modulation factors. By applying these factors to $F_{in}^{\prime }$ through channel-wise scaling, we effectively infuse the dense features with target-aware guidance. Finally, the modulated features pass through a projection layer for channel reduction and smoothing, yielding the deep features $F_{deep}$:

$$F_{deep}=\mathrm{Proj}\left({\mathrm{MLP}}_{hyper}\left(Q_{final}^{\prime }\right)\otimes \mathrm{Up}\left(F_{in}^{\prime }\right)\right)\in {\mathbb{R}}^{\left(2H\right)\times \left(2W\right)\times {\displaystyle \frac{C}{4}}},$$

(7)

where ${\mathrm{MLP}}_{hyper}$ denotes the hypernetwork MLP, $\mathrm{Up}(·)$ represents the spatial upsampling operation, and ⊗ denotes the element-wise multiplication along the channel dimension.

To ensure precise target perception, we adopt a resolution recovery strategy that captures fine-grained features by establishing skip-convs between the two stages. Specifically, the framework utilizes Up-Blocks to progressively restore the spatial resolution of $F_{deep}$, integrating feature feedback from skip-convs at each stage. This process generates a series of multi-scale predicted masks $\{Mas{k}_0,Mas{k}_1,Mas{k}_2\}$, which serve as the input for subsequent processes.

2.2.3. Radial High-Pass Modulator

Small targets in infrared images usually appear as high-frequency transients, while large-scale backgrounds, such as cloud layers and sea surfaces, are mainly concentrated in low-frequency components. To better separate weak target responses from dominant low-frequency clutter, we introduce the RHPM module, which performs adaptive radial high-pass filtering in the frequency domain. It adaptively determines the cutoff radius according to the energy distribution of the feature map, enabling scene-dependent suppression of low-frequency interference.

As shown in Figure 3, the cascaded RHPM takes the input image $IRI\in {\mathbb{R}}^{H\times W\times 1}$ and the multi-scale masks $\{Mas{k}_0,Mas{k}_1,Mas{k}_2\}$ in Section 2.2.2 as inputs.

Specifically, the mask from the deepest layer, denoted as $Mas{k}_2\in {\mathbb{R}}^{H\times W\times 1}$, is utilized for initial spatial filtering to highlight potential targets. The resulting enhanced features are then projected into the frequency domain through a two-dimensional Fast Fourier Transform (FFT):

$$F_{f,2}=\mathcal{F}(IRI\otimes Mas{k}_2+IRI),$$

(8)

where $\mathcal{F}(·)$ represents the FFT operation and $F_{f,2}\in {\mathbb{R}}^{H\times W\times 1}$ represents the frequency feature map.

The process of high-pass filtering is illustrated in Figure 4.

To evaluate the frequency distribution, we calculate the square of the frequency feature map as the power spectrum $E(u,v)$:

$$E(u,v)={|F_{f,2}(u,v)|}^2,$$

(9)

where $F_{f,2}(u,v)$ is the amplitude value at pixel $(u,v)$. Finally, the total spectral energy $E_{total}$ is obtained by aggregating the energy across all frequency components:

$$E_{total}={\displaystyle \sum _{u=0}^{H-1}}{\displaystyle \sum _{v=0}^{W-1}}E(u,v).$$

(10)

To achieve adaptive background suppression, we introduce stage-specific hyperparameters $\Lambda =[{\lambda }_0,{\lambda }_1,{\lambda }_2]$; then, we optimize the cutoff radius $r_{cut}$ to satisfy the condition that the low-frequency energy ${\mathcal{E}}_{low}\left(r\right)$ accounts for at least ${\lambda }_2$ of the total energy $E_{total}$. The optimization objective function in the first RHPM stage (corresponding to ${\lambda }_2$) is defined as:

$$\begin{array}{cc}\hfill r_{cut}=arg\underset{r}{\min }& r\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& 1\le r\le {\displaystyle \frac{\min (H,W)}{2}},\hfill \\ \hfill & {\displaystyle \frac{{\mathcal{E}}_{low}\left(r\right)}{E_{total}}}\ge {\lambda }_2.\hfill \end{array}$$

(11)

where ${\mathcal{E}}_{low}\left(r\right)$ denotes the cumulative energy within a radius r centered at the spectral origin $(c_u,c_v)$. We set the low-frequency components within the cutoff region to zero while retaining the high-frequency components. Based on this, we construct a dynamic filtering mask $M(u,v)$ to eliminate low-frequency components while preserving high-frequency details:

$$M(u,v)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}{(u-c_u)}^2+{(v-c_v)}^2<r_{cut}^2\hfill \\ 1,\hfill & \mathrm{otherwise}\hfill \end{array}\right..$$

(12)

Considering that the amount of low-frequency background diminishes as the decoder transitions to shallower layers, we adaptively decrease the energy filtering rate ${\lambda }_i$ across stages with setting $\Lambda =[{\lambda }_0,{\lambda }_1,{\lambda }_2]=[0.1,0.2,0.4]$ for the respective levels, where the indices $\{0,1,2\}$ represent the layers from shallow to deep. The sensitivity analysis of the hyperparameter is provided in Section 3.3.4.

The enhanced high-frequency features are then reconstructed via the Inverse Fast Fourier Transform (iFFT):

$$X_{{\lambda }_2}=|{\mathcal{F}}^{-1}(F_{f,2}\otimes M)|,$$

(13)

where ${\mathcal{F}}^{-1}(·)$ denotes the iFFT. The subsequent stages follow a similar procedure to yield the final output $X_{{\lambda }_0}$, the detailed workflow of which is illustrated in Figure 3.

Finally, the spatial prediction P is refined using the frequency domain features $X_{{\lambda }_0}$ generated by the third-stage RHPM to yield the final output $P_{final}$:

$$\begin{array}{cc}\hfill & P={\mathrm{Conv}}_{1\times 1}\left(\mathrm{Concat}[Mas{k}_0,Mas{k}_1,Mas{k}_2]\right),\hfill \\ \hfill & P_{final}=P\otimes \sigma \left(X_{{\lambda }_0}\right)+P\in {\mathbb{R}}^{H\times W\times 1}\hfill \end{array}$$

(14)

where P is obtained by cascading the multi-scale prediction maps from the intermediate decoder stage.

2.2.4. Model Optimization

To address extreme foreground–background imbalance and weak local contrast in infrared imagery, we design a Contrast and Shape-Aware Adaptive (CSA) loss, denoted as ${\mathcal{L}}_{CSA}$. We adopt the Scale and Location Sensitive (SLS) loss as the primary supervision, and use the Target-Driven Adaptive (TDA) loss as auxiliary supervision to improve robustness under challenging conditions.

Primary Supervision. The SLS loss optimizes IoU while incorporating polar coordinate constraints to refine the target’s geometric center. It is defined as a sum of a scale-sensitive loss ${\mathcal{L}}_S$ and a location-sensitive loss ${\mathcal{L}}_L$:

$${\mathcal{L}}_{SLS}={\mathcal{L}}_S+{\mathcal{L}}_L,$$

(15)

where ${\mathcal{L}}_S$ is defined as

$${\mathcal{L}}_S=1-\beta \ast {\displaystyle \frac{I+ϵ}{U+ϵ}},$$

(16)

where I and U denote the intersection and union areas between the prediction and the ground truth, respectively. The weighting factor $\beta$ is adaptively calculated based on the total pixel deviation to smooth the loss landscape and facilitate more stable optimization. Meanwhile, ${\mathcal{L}}_L$ is defined as:

$${\mathcal{L}}_L={\displaystyle \sum _{i=1}^N}(1-{\rho }_i+{\theta }_i),$$

(17)

where ${\rho }_i$ represents the radial distance consistency between the predicted and true centroid, while ${\theta }_i$ measures the deviation in azimuth angle of the predicted centroid relative to the true centroid. This polar coordinate system supervision mechanism can effectively correct the deformation of the target, making the predicted mask better aligned with the ground truth in terms of shape.

Auxiliary Supervision. Inspired by the TDA loss [41], we introduce an auxiliary loss ${\mathcal{L}}_{aux}$. Unlike conventional global image-level supervision, ${\mathcal{L}}_{aux}$ operates on local patches centered around target regions, employing a patch-based supervision strategy coupled with an adaptive weighting mechanism.

To implement hard sample mining, we assign higher weights to targets with smaller scales or lower contrasts. For the t-th connected component (target), we extract its pixel area $s_t$ and local contrast $c_t$. An adaptive importance index $p_t$ is defined by comparing these attributes against the mean values $s_{mean}$ and $c_{mean}$ calculated from the entire dataset:

$$p_t=1+\sigma \left(-{\displaystyle \frac{s_t}{s_{\mathrm{mean}}}}\right)+\sigma \left(-{\displaystyle \frac{c_t}{c_{\mathrm{mean}}}}\right),$$

(18)

where $\sigma (·)$ denotes the sigmoid function. This formulation ensures that $p_t$ increases as the target size decreases or the local contrast decreases, effectively forcing the model to focus on the most challenging samples.

Global prediction maps often suffer from severe class imbalance, where the overwhelming majority of background gradients can dilute the sparse positive gradients from extremely small targets. To mitigate this, we limit the supervision to local patches that contain the target and its nearby surroundings.

For a target with centroid $(C_x,C_y)$ and bounding box dimensions $(w,h)$, the patch is defined as

$$Patchbox=[C_x-{\displaystyle \frac{w}{2}}-d,C_y-{\displaystyle \frac{h}{2}}-d,C_x+{\displaystyle \frac{w}{2}}+d,C_y+{\displaystyle \frac{h}{2}}+d],$$

(19)

where $d\in [2,5]$ is a random dilation factor providing surrounding context.

Based on this $Patchbox$, we crop the corresponding regions from the global predicted mask and the ground truth, denoted as ${\widehat{y}}_t$ and $y_t$. To eliminate the influence of target size and ensure scale invariance during loss calculation, both ${\widehat{y}}_t$ and $y_t$ are uniformly rescaled to a fixed resolution of $48\times 48$. Within this normalized local region, we supervise the model using a weighted soft IoU loss ${\mathcal{L}}_t$. Specifically, ${\mathcal{L}}_t$ incorporates a focal-like modulating factor to adaptively adjust the gradient contribution based on target difficulty, thereby prioritizing the optimization of hard-to-segment targets:

$${\mathcal{L}}_t=-(1-I_t^{p_t})·\log (I_t+\epsilon ),$$

(20)

where $I_t$ denotes the IoU between the rescaled patches, and $p_t$ serves as an importance weight to emphasize hard targets. The final auxiliary loss is defined as the average loss across all N targets in the image:

$${\mathcal{L}}_{aux}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{t=1}^N}{\mathcal{L}}_t.$$

(21)

This local focus encourages the model to capture fine-grained morphology and edge features while reducing potential false alarms.

Contrast and Shape-Aware Adaptive Loss. Finally, the Contrast and Shape-Aware Adaptive (CSA) loss function integrates both global and local supervision mentioned above:

$${\mathcal{L}}_{CSA}={\mathcal{L}}_{SLS}+w_{aux}·{\mathcal{L}}_{aux},$$

(22)

where $w_{aux}$ is a factor. This hybrid strategy improves localization capability while boosting target-level recall for faint targets in complex backgrounds.

3. Results

3.1. Experimental Setup

Datasets. We evaluate our approach on three public IRSTD benchmarks: IRSTD-1k [14], NUAA-SIRST [15], and NUDT-SIRST [9], with sample sizes of 1001, 427, and 1327, respectively. Following prior works [42], the NUAA-SIRST and NUDT-SIRST datasets are partitioned into training and testing sets with a 1:1 ratio. For IRSTD-1k, we adopt a training-to-testing split ratio of 4:1.

Evaluation Metrics. We evaluate our model using standard metrics categorized into two dimensions: pixel-level segmentation quality, measured by intersection over union (IoU), and target-level detection performance, represented by probability of detection ($P_d$) and false alarm rate ($F_a$). Additionally, Receiver Operating Characteristic (ROC) curves are generated to provide a comprehensive comparison between our method and current state-of-the-art solutions.

The IoU is defined as

$$\mathrm{IoU}={\displaystyle \frac{A_{inter}}{A_{union}}}={\displaystyle \frac{{\sum }_{i=1}^{N}TP\left[i\right]}{{\sum }_{i=1}^N(T\left[i\right]+P\left[i\right]-TP\left[i\right])}},$$

(23)

where $A_{inter}$ and $A_{union}$ denote the cumulative areas of intersection and union across N samples, respectively. The variables $T\left[i\right]$, $P\left[i\right]$, and $TP\left[i\right]$ denote the numbers of ground-truth, predicted, and true positive pixels for the i-th sample, respectively.

The probability of detection is defined as

$$P_d={\displaystyle \frac{N_{detect}}{N_{total}}},$$

(24)

where $N_{detect}$ is the count of accurately predicted targets and $N_{total}$ is the total number of all targets.

The false alarm rate is defined as

$$F_a={\displaystyle \frac{P_{false}}{P_{total}}},$$

(25)

where $P_{false}$ is the number of falsely predicted target pixels and $P_{total}$ is the total number of pixels in the image.

The ROC curves are generated by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) across a range of decision thresholds.

Implementation Details. Following prior works [42], input images are resized to $256\times 256$. The model is optimized using the adaptive Nesterov momentum algorithm (Adan) [43] for 300 epochs, with a batch size of 12. We set the initial learning rate at 0.01 and employ a CosineAnnealingLR scheduler to facilitate a smooth decay to $1\times {10}^{-6}$. We adopt the first three stages of the SAM2 (Hiera–Tiny) image encoder as the backbone feature extractor. All experiments are implemented within the PyTorch framework and accelerated by an NVIDIA GeForce RTX 4090 GPU.

3.2. Performance Comparison

3.2.1. Compared Methods

To validate the effectiveness of the proposed method on infrared small target segmentation across different sensing platforms, we compare IR-SAM2 with a wide range of state-of-the-art IRSTD methods:

• Model-driven methods: top-hat [44], max–median [16], IPI [21], RIPT [22], NRAM [45] and PSTNN [23].
• Deep learning methods: ACMNet [15], ALCNet [46], ISNet [14], RDIAN [42], MDvsFA [47], DNANet [9], AGPCNet [48], UIUNet [49], MSHNet [39], SCTransNet [50], PConv [40], MMLNet [51], L2SKNet [52], SAM-SPL [13], HDNet [33] and MTUNet [27].
• Deep unfolding-based methods: RPCANet [28] and DRPCA-Net [29].

3.2.2. Quantitative Comparison

Table 1. Performance comparison of different models. Blue, orange, and green indicate the best, second-best, and third-best performance, respectively.

Method	Publish	IRSTD-1k			NUDT-SIRST			NUAA-SIRST
		$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$(10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
Traditional Methods
Top-hat [44]	OE’96	5.59	69.36	702.36	22.26	91.32	77	1.51	79.74	16456
Max–median [16]	SPIE’99	7.00	65.21	59.73	4.20	58.41	36.89	6.02	84.34	774.3
IPI [21]	TIP’13	27.92	81.37	16.18	28.63	74.49	41.23	1.09	87.05	30467
RIPT [22]	JSTARS’17	14.11	77.55	28.31	29.17	91.85	344.3	16.79	69.76	59.33
NRAM [45]	RS’18	9.882	72.48	24.73	12.08	72.58	84.77	6.93	56.40	19.27
PSTNN [23]	RS’19	24.57	71.99	35.26	27.72	66.13	44.17	30.30	72.80	48.99
Deep Learning Methods
MDvsFA [47]	ICCV’19	49.50	82.11	80.33	75.14	90.47	25.34	60.30	89.35	56.35
ACMNet [15]	WACV’21	60.33	93.27	68.49	68.48	96.26	10.27	69.44	92.02	22.71
ALCNet [46]	TGRS’21	63.52	85.86	12.96	85.03	97.89	18.20	73.74	97.25	26.79
DNANet [9]	TIP’22	65.71	91.84	17.61	79.98	96.93	12.78	74.81	93.53	38.27
ISNet [14]	CVPR’22	64.73	87.21	15.03	87.77	95.56	9.42	70.49	95.06	67.98
UIUNet [49]	TIP’22	65.37	89.23	21.52	90.52	98.83	8.34	77.53	92.40	9.33
MTUNet [27]	TGRS’23	65.24	89.56	9.51	84.02	97.99	7.49	74.85	99.08	7.09
AGPCNet [48]	TAES’23	64.92	89.56	18.56	85.35	97.35	6.78	–	–	–
RDIAN [42]	TGRS’23	64.37	92.26	18.2	81.06	98.36	14.09	70.74	95.06	48.16
MSHNet [39]	CVPR’24	67.16	93.88	15.03	80.55	97.99	11.77	73.5	97.25	31.05
SCTransNet [50]	TGRS’24	68.03	93.27	10.74	94.09	98.62	4.29	77.5	96.95	13.92
L2SKNet [52]	TGRS’25	67.81	90.24	17.46	93.58	97.57	5.33	73.43	98.17	20.82
PConv [40]	AAAI’25	67.45	92.20	10.70	–	–	–	–	–	–
MMLNet [51]	TGRS’25	67.21	94.28	14.00	81.81	98.43	11.77	78.71	98.88	25.71
SAM-SPL [13]	TGRS’25	64.57	89.80	8.35	94.63	97.24	2.55	73.8	97.25	1.95
HDNet [33]	TGRS’25	66.66	93.88	17.46	85.17	98.52	2.78	77.05	100	12.95
Deep Unfolding-Based Methods
RPCANet [28]	CVPR’24	63.21	88.31	43.9	89.31	97.14	28.7	65.08	93.58	10.85
DRPCA-Net [29]	TGRS’25	64.14	92.09	17.92	94.16	98.41	2.55	–	–	–
Our Proposed Method
Ours	–	69.75	96.60	9.03	94.18	98.94	2.97	75.16	99.08	4.79

summarizes the quantitative comparison between our IR-SAM2 and 24 state-of-the-art (SOTA) methods across three benchmark datasets. Overall, IR-SAM2 demonstrates superior performance in terms of IoU, $P_d$, and $F_a$ across all three datasets. Specifically, on the IRSTD-1k [14], IR-SAM2 obtains the best IoU (69.75%) and $P_d$ (96.60%), showing that the combination of frequency-aware enhancement and target-centric supervision is particularly effective in cluttered scenes. Despite not attaining the peak IoU on NUAA-SIRST [15], IR-SAM2 offers a more favorable $P_d/F_a$ balance than leading methods such as UIUNet, SCTransNet, and MMLNet, highlighting its robust discriminative capability against complex clutter.

On NUDT-SIRST [9], although SAM-SPL achieves slightly better IoU and $F_a$, IR-SAM2 still attains the highest $P_d$ (98.94%), suggesting that our method is more sensitive to target responses but may occasionally suppress extremely weak targets during frequency filtering. Nonetheless, IR-SAM2 consistently surpasses other filtering-based models, such as HDNet, on both the IRSTD-1k and NUDT-SIRST datasets. This validates the effectiveness of adapting the SAM2 [11] framework for infrared small target segmentation.

We also presented ROC curves for several advanced methods on the IRSTD-1k dataset. As illustrated in Figure 5, where a curve positioned closer to the top-left corner indicates better performance, IR-SAM2 consistently maintains the uppermost position. Notably, our method attains a higher TPR at significantly lower FPR levels, reflecting its exceptional sensitivity and ability to suppress false alarms in challenging scenarios.

3.2.3. Qualitative Comparison

Figure 6 provides a qualitative comparison between our IR-SAM2 and six representative state-of-the-art methods. We select representative samples from the IRSTD-1k [14] and NUDT-SIRST [9] datasets to visually illustrate the segmentation performance. In the visualization, ground truth targets, false alarms, and missed detections are highlighted with red, yellow, and blue boxes, respectively.

The inherent difficulty of IRSTD is clearly illustrated in the raw infrared images shown in Figure 6. As observed in the first, fifth, and sixth rows, the targets are extremely minute and often occupy only a few pixels, causing them to blend into the background and making them exceptionally difficult to distinguish from their surroundings. Furthermore, the irregular morphologies of the targets (as seen in the second, third, and fourth rows) pose a significant challenge to the multi-scale perception capabilities of existing models.

In contrast, our proposed IR-SAM2 achieves the strongest overall qualitative performance, showing more accurate localization and fewer false alarms across targets of different scales. Whether dealing with typical small targets or those with relatively larger dimensions, our method accurately localizes visually blurred targets within complex backgrounds while effectively suppressing false alarms. Although the visualizations indicate that IR-SAM2 still produces a few false positives, similar behavior can also be observed in other methods to varying degrees.

The qualitative results in Figure 6 show that competing methods often either miss dim targets or generate spurious responses in structurally complex backgrounds. This is particularly evident in cluttered scenes, where background structures exhibit intensity patterns similar to target responses. In contrast, IR-SAM2 produces more compact and accurate predictions around the target region, with fewer spurious activations in the surrounding background. These observations suggest that the proposed frequency-aware filtering helps suppress low-frequency clutter while preserving target-related high-frequency cues.

3.3. Ablation Study

In this section, we conduct comprehensive ablation studies to validate the effectiveness of our proposed modules on the three public IRSTD benchmarks (IRSTD-1k [14], NUDT-SIRST [9], and NUAA-SIRST [15]).

3.3.1. Importance of Each Component

We validate the effectiveness of our proposed components by comparing four network configurations: (i) a baseline model that retains the same SAM2 encoder–decoder backbone without RHPM and CQG; (ii) baseline + CQG, which adds the CQG module to the decoder; (iii) baseline + RHPM, which inserts the RHPM module along the mask generation path; and (iv) the full IR-SAM2 model.

Table 2. Ablation study of different components on three datasets. The symbol “–” indicates that the corresponding component is not used, while “✓” indicates that the component is used. The best results are highlighted in bold.

Components		IRSTD-1k			NUDT-SIRST			NUAA-SIRST
RHPM	CQG	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
–	–	65.43	93.53	17.15	91.47	98.41	3.84	73.68	99.08	6.56
✓	–	67.20	88.77	8.50	87.36	98.94	3.24	72.03	100	32.65
–	✓	66.84	92.51	7.51	86.22	98.41	4.59	73.65	97.24	21.29
✓	✓	69.75	96.60	9.03	94.18	98.94	2.97	75.16	99.08	4.79

reports the corresponding quantitative comparisons.

Compared with the baseline model, the integration of either the RHPM or CQG module yields consistent performance gains in most metrics. Specifically, on the IRSTD-1k dataset, the inclusion of the RHPM module increases the IoU from 65.43% to 67.20% while significantly reducing the false alarm rate from $17.15\times {10}^{-6}$ to $8.50\times {10}^{-6}$. This demonstrates the efficacy of dynamic high-pass filtering in suppressing background clutter. Similarly, the CQG module independently enhances the IoU to 66.84%, validating its capability in refining target representation through contextual guidance. The full IR-SAM2 model, which combines both components, achieves the overall best performance across all three datasets. Notably, on the IRSTD-1k dataset, IR-SAM2 reaches an IoU of 69.75% and a $P_d$ of 96.60%, outperforming the baseline by 4.32% and 3.07%, respectively. Furthermore, on the NUAA-SIRST dataset, the full model achieves the lowest false alarm rate ($4.79\times {10}^{-6}$) among all configurations. These results indicate that RHPM and CQG are not only individually effective but also highly complementary when combined. Specifically, RHPM primarily contributes to clutter suppression, as reflected by the reduction in $F_a$, whereas CQG mainly improves target localization by providing more informative decoder queries. Their combination leads to the best overall performance across all three datasets.

3.3.2. Different Loss Functions

To validate the effectiveness of the proposed CSA loss, we conducted comparative experiments against three commonly used loss functions in IRSTD, including BCE loss [36], IoU loss [37], TDA loss [41], and SLS loss [39]. The quantitative results are summarized in

Table 3. Performance comparison ofdifferent functions. The best results are highlighted in bold.

Method	IRSTD-1k			NUDT-SIRST			NUAA-SIRST
	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
BCE loss [36]	63.44	84.69	4.55	94.52	98.94	1.84	72.77	98.16	17.92
IoU loss [37]	63.15	93.54	36.97	90.65	98.20	3.77	74.12	97.25	9.94
TDA loss [41]	65.28	95.93	25.81	88.98	98.73	7.63	71.40	98.16	21.11
SLS loss [39]	67.64	90.14	7.52	85.68	98.73	8.16	73.62	98.24	14.58
CSA loss(Ours)	69.75	96.60	9.03	94.18	98.94	2.97	75.16	99.08	4.79

.

Overall, the proposed CSA loss performs better in the majority of evaluation metrics and datasets compared to the baseline loss functions. While the BCE loss shows a marginal advantage on the NUDT-SIRST [9] dataset for specific metrics, the CSA loss consistently maintains more robust segmentation results across the broader benchmarks.

For a further qualitative assessment, we visualized the segmentation results of our method alongside several representative baselines in Figure 7.

It can be observed that models trained with conventional loss functions generally suffer from missed targets when encountering targets with small scales and low local contrast. Although employing the TDA loss alone demonstrates some capability in capturing the features of these dim and small targets, it is highly susceptible to high-frequency noise and background clutter. This vulnerability inevitably drives up the $F_a$. Moreover, as illustrated in Figure 7, the TDA loss exhibits a weak regression capability for target shapes and boundaries. Consequently, its fundamental pixel-level segmentation performance remains insufficient, directly leading to relatively lower IoU scores. Therefore, the TDA loss is primarily suitable only as an auxiliary loss rather than a standalone objective. In contrast, the model integrated with CSA loss successfully identifies even smaller and more blurred targets in multi-target scenarios. These observations suggest that CSA loss improves learning on hard targets by adaptively emphasizing low-contrast and small-scale instances in local regions, a capability that is difficult to achieve with purely global objectives such as BCE or IoU loss.

We also evaluated the sensitivity of the weight $w_{aux}$ within the CSA loss on the IRSTD-1k [14] and NUDT-SIRST [9] datasets. By varying $w_{aux}$ across values of 0.1, 0.2, 0.3, and 0.5, we analyzed the response of evaluation metrics (see

Table 4. Parameter sensitivity analysis of $w_{aux}$ on IRSTD-1k and NUDT-SIRST datasets. The best results are highlighted in bold.

Setting	IRSTD-1k			NUDT-SIRST
${\mathit{w}}_{\mathit{aux}}$	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
0.5	65.95	90.81	10.24	90.33	97.24	2.18
0.3	67.46	93.93	7.06	90.84	98.51	11.23
0.2	69.75	96.60	9.03	94.18	98.94	2.97
0.1	68.91	95.59	8.35	92.77	98.94	4.25

).

It was observed that an excessively large weight (e.g., $w_{aux}=0.5$) causes the model to over-prioritize the auxiliary task, thereby compromising the learning efficacy of the primary branch. Conversely, an overly small weight (e.g., $w_{aux}=0.1$) fails to provide sufficient supervision signals to facilitate effective target refinement. The optimal balance is achieved at $w_{aux}=0.2$, where the model attains the highest $P_d$ while maintaining a consistently low $F_a$ across both datasets.

3.3.3. Ablation Study of the CQG Module

To investigate the impact of the number of candidate points K in the CQG on model performance, a parameter sensitivity analysis is conducted on the IRSTD-1k and NUDT-SIRST datasets. The quantitative evaluation results for different values of K ($K\in \{1,5,10,20\}$) are summarized in

Table 5. Parameter sensitivity analysis of K in the CQG on IRSTD-1k and NUDT-SIRST datasets. The best results are highlighted in bold.

Setting	IRSTD-1k			NUDT-SIRST
$\mathit{K}$	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
1	67.38	95.91	14.12	91.95	98.73	8.09
5	69.30	96.41	5.62	93.98	98.62	2.29
10	69.75	96.60	9.03	94.18	98.94	2.97
20	68.19	95.79	10.93	93.29	98.94	4.61

.

As shown in the table, when $K=1$, the model exhibits the lowest segmentation accuracy (IoUs of 67.38% and 91.95%) and the highest false alarm rates ($F_a$ up to $14.12\times {10}^{-6}$ and $8.09\times {10}^{-6}$) on both datasets. Mechanistically, selecting only a single point with the highest energy makes the model highly susceptible to interference from isolated high-frequency noise or heavy background clutter. Furthermore, a single point cannot effectively cover the holistic features of dim and small targets that possess irregular morphologies or span multiple pixels, thereby resulting in severe deviations in position estimation.

As K increases to 10, all performance metrics of the model reach their optimum, with the IoUs improving to 69.75% and 94.18% and $P_d$ reaching 96.60% and 98.94% on IRSTD-1k and NUDT-SIRST, respectively. This validates the rationality of the CQG module design: an appropriate number of candidate points can sufficiently cover the salient regions of potential targets. By leveraging the saliency intensity $s_k$ to compute the Softmax-normalized aggregation weights ${\alpha }_k$ and performing a weighted summation over the positional encodings, the model can more robustly estimate the centroid of the high-frequency energy. This enables the final query vector $Q_{final}$ to precisely focus on the most likely target regions, effectively enhancing the detection rate and segmentation accuracy under complex backgrounds.

However, when K is further increased to 20, although the $F_a$ decreases due to broader contextual aggregation, both the IoU and $P_d$ exhibit a slight degradation. This performance decline is attributed to the fact that an excessively large K inevitably introduces background edges surrounding the target or non-target high-frequency noise into the candidate set $P_{cand}$. During the weighted aggregation computation, the inclusion of these redundant points dilutes the attention weights of the true targets, causing a shift in the position prior estimation and subsequently weakening the model’s capability for precise perception and localization of extremely small targets.

In summary, $K=10$ achieves the optimal balance between target feature coverage and noise exclusion. Therefore, in all subsequent experiments in this paper, the default value of K is set to 10.

3.3.4. Ablation Study of the RHPM Module

To investigate the influence of energy filtering rate across layers from shallow to deep, denoted as $\Lambda =[{\lambda }_0,{\lambda }_1,{\lambda }_2]$, we conducted parameter sensitivity experiments on the IRSTD-1k and NUDT-SIRST datasets. As summarized in

Table 6. Sensitivity analysis of energy filtering rate $\Lambda$ on IRSTD-1k and NUDT-SIRST datasets. The best results are highlighted in bold.

Settings	IRSTD-1k			NUDT-SIRST
Λ = [λ0, λ1, λ2]	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)	$\mathbf{IoU}$ (%)	${\mathit{P}}_{\mathit{d}}$ (%)	${\mathit{F}}_{\mathit{a}}$ (10−6)
$[0.1,0.1,0.1]$	67.17	95.91	14.12	92.06	98.73	7.14
$[0.2,0.2,0.2]$	68.50	96.26	9.50	93.47	98.51	1.37
$[0.4,0.4,0.4]$	65.81	97.27	11.69	89.04	98.09	8.06
$[0.1,0.2,0.4]$	69.75	96.60	9.03	94.18	98.94	2.97

, we first evaluated equal rate configurations where ${\lambda }_0={\lambda }_1={\lambda }_2$. Increasing the energy filtering rate from 0.1 to 0.2 improved the IoU from 67.17% to 68.50% on IRSTD-1k and from 92.06% to 93.47% on NUDT-SIRST. These gains suggest that moderate filtering effectively suppresses background clutter while accentuating target features. However, further increasing the rate to 0.4 led to significant performance degradation. Specifically, the IoU on NUDT-SIRST plummeted to 89.04%, indicating that excessive filtering may over-smooth the morphological details of minute targets and compromise localization precision.

Notably, the non-uniform configuration $\Lambda =[0.1,0.2,0.4]$ yielded the best overall performance by progressively intensifying filtering from deep to shallow layers. On IRSTD-1k [14], this setting achieved a peak IoU of 69.75% and a minimum $F_a$ of $9.03\times {10}^{-6}$. Similarly, the IoU and $P_d$ on NUDT-SIRST [9] reached 94.18% and 98.94%, respectively. This improvement is primarily attributed to the adaptive filtering requirements during the decoding process. In the initial stage, where deep features are utilized for filtering, the input $IRI$ contains an abundance of low-frequency background clutter, necessitating a more aggressive energy filtering rate of 0.4 to ensure effective suppression. As the decoder progressively transitions to shallower layers, the amount of low-frequency background within the predicted maps gradually diminishes. Consequently, a lower filtering rate (0.1) is sufficient for these subsequent stages, allowing the model to maintain target-related feature gain while avoiding the over-smoothing of fine details.

3.4. Computational Efficiency

To evaluate the architectural complexity and computational efficiency of our proposed IR-SAM2, we employ Params, FLOPs, and FPS as quantitative metrics. As summarized in

Table 7. Comparison ofmodel complexity and efficiency in terms of Params, FLOPs, and FPS between our IR-SAM2 and recent advanced models. ↑ indicates that higher values are better, while ↓ indicates that lower values are better. “–” indicates that the corresponding data are not available.

Method	Type	Year	IoU (%) ↑	Params (M) ↓	FLOPs (G) ↓	FPS (f/s) ↑
ALCNet [46]	CNN	2021	63.52	0.51	6.09	–
DNANet [9]	CNN	2022	65.71	4.69	14.26	68.96
UIUNet [49]	CNN	2023	65.37	50.54	54.42	9.87
MTUNet [27]	Transformer	2023	65.24	12.75	34.43	10.87
AGPCNet [48]	CNN	2023	64.92	12.36	43.18	–
RPCANet [28]	CNN	2024	63.21	0.68	44.57	2.80
SCTransNet [50]	Transformer	2024	68.03	11.19	67.40	34.36
MMLNet [51]	CNN	2025	67.8	3.58	20.41	38.17
SAM-SPL [13]	SAM	2025	64.57	25.48	30.46	56.18
IR-SAM2 (Ours)	SAM	–	69.75	25.55	30.47	54.35

, our model is compared against several state-of-the-art (SOTA) methods from the past five years on the IRSTD-1k [14] dataset, encompassing both representative CNN-based and Transformer-based architectures.

Notably, compared to the recent SAM-based method SAM-SPL, our IR-SAM2 achieves a significant performance gain of 5.18% in IoU while maintaining a nearly identical computational overhead (approx. 25.5 M Params and 30.5 G FLOPs) and a comparable inference speed (54.35 vs. 56.18 FPS). Furthermore, when compared to SCTransNet, which holds the second-best IoU, our model not only achieves higher accuracy (69.75% vs. 68.03%) but also delivers a much higher frame rate (54.35 vs. 34.36 FPS) with less than half the FLOPs (30.47 G vs. 67.40 G). Although some lightweight CNNs like DNANet offer higher FPS, their segmentation accuracy falls drastically behind.

To provide a more intuitive and comprehensive evaluation, the trade-off among segmentation accuracy, computational cost, and model scale is illustrated in Figure 8.

Specifically, the overall detection performance is quantified by the IoU on the IRSTD-1k dataset, while the area of each circle corresponds to the parameter size of the respective method. As can be observed from the chart, IR-SAM2 achieves superior overall performance without introducing excessive computational overhead, thereby showcasing high computational efficiency and validating its effectiveness for practical infrared small target detection tasks.

4. Discussion

Although IR-SAM2 demonstrates highly competitive performance across multiple public IRSTD benchmarks, the framework still has several notable limitations in extreme scenarios and under specific hardware constraints.

First, in extremely low signal-to-noise ratio (SNR) scenarios, the frequency domain filtering module (RHPM) may inadvertently suppress exceedingly faint targets. This partially explains why IR-SAM2 achieves a relatively lower IoU on the NUAA-SIRST [15] dataset compared to its performance on IRSTD-1k [14] and NUDT-SIRST [9]. By examining the typical failure cases illustrated in Figure 9, the underlying causes of this phenomenon can be clearly observed.

As shown in the first two rows of Figure 9, when the radiation intensity of small targets is extremely low and highly blended with complex backgrounds, their high-frequency characteristics become barely distinguishable. During the adaptive suppression of low-frequency background energy by RHPM, these weak target responses are prone to being erased alongside background clutter, leading to missed detections.

As illustrated in the last row of Figure 9, for infrared targets with blurred edges, the intensity transition from the center to the periphery is gradual. These blurred edges often exhibit relatively low-frequency characteristics in the frequency domain. When performing high-pass filtering, the RHPM module tends to misclassify these low-gradient blurred edges as background and remove them. Consequently, the final predicted mask retains only the brightest core region of the target. This “over-suppression” phenomenon results in a predicted mask area that is significantly smaller than the ground truth, directly contributing to the degradation of the IoU metric in certain scenarios.

Second, the design of CQG may be insufficient for dense or multiple small targets. Based on the principles detailed in Section 2.2.2, CQG extracts the Top-K high-frequency salient points and employs Softmax weights to perform global weighted aggregation of their positional encodings, thereby estimating the high-frequency energy centroid of potential target regions. However, when multiple spatially dispersed, discrete small targets are present in the image, this global weighted aggregation approach may weaken the positional awareness of each individual point within the calculated query features. Consequently, it fails to provide precise instance-level positional guidance for the Transformer decoder. This single-query aggregation mechanism inherently limits the model’s capability for efficient separation and precise localization of dense multiple targets.

Furthermore, regarding computational efficiency, although our method holds an advantage over other Transformer-based models, the architecture built upon the SAM2 backbone still introduces relatively high computational complexity (as summarized in Table 7, with approximately 25.55 M Params and 30.47 G FLOPs). In airborne or spaceborne infrared monitoring edge devices, such computational overhead may hinder true real-time deployment. To address the aforementioned limitations, future work will focus on the following directions: (1) developing more perception-aware adaptive filtering strategies that effectively suppress low-frequency clutter while better preserving the complete morphological structure of targets with blurred edges; (2) exploring multi-query generation or local-clustering-based contextual aggregation mechanisms for dense multi-target scenarios to replace the current global centroid estimation, thereby enhancing instance-level perception capabilities; (3) further investigating model compression and knowledge distillation techniques to advance the lightweight design of the framework; and conducting systematic cross-dataset evaluation, such as training on one IRSTD benchmark and testing on another, to further assess and improve its cross-domain generalization capability, thereby meeting the practical deployment requirements of real-world monitoring systems.

5. Conclusions

In this work, we presented IR-SAM2, a SAM2-based framework for infrared small target segmentation. The proposed method introduces the Radial High-Pass Modulator (RHPM) module to incorporate frequency domain priors for better target–background discrimination, and the Contrast Query Generator (CQG) to convert high-frequency salient responses into location-aware query tokens for precise localization. Furthermore, a target-centric Contrast and Shape-Aware Adaptive (CSA) loss is designed to address the limitations of global objectives under extreme foreground–background imbalance. Extensive ablation studies comprehensively verify the effectiveness and complementary nature of each proposed component, enabling our unified pipeline to achieve competitive performance on multiple public IRSTD benchmarks.