Multi-Stage Aggregation CNN–Transformer Hybrid Architecture for Infrared Small-Target Detection

Highlights

What are the main findings?

• A novel infrared small-target detection network (MACT-Net) is proposed, integrating multi-level aggregation, hybrid feature modeling, and feature discretization mechanisms. Extensive experimental results confirm that it effectively enhances infrared small-target detection performance.
• The effectiveness of the proposed modules is validated. Ablation studies confirm the improvement of detection accuracy and robustness by the modules.

What are the implications of the main findings?

• The findings provide a technical solution for practical scenarios such as infrared imaging detection and emergency search and rescue, enabling precise detection of weak targets in complex backgrounds while reducing false-alarm rates and improving pixel-level localization accuracy.
• The findings broaden the application scope of CNN–Transformer hybrid architectures in infrared small-target detection, offering a network design paradigm and feature enhancement strategy for similar low-SNR, weak-feature target detection tasks.

Abstract

Infrared small-target detection plays a vital role in applications such as imaging detection and remote sensing. Because infrared small targets are weak and lack distinctive features, they are easily submerged in cluttered backgrounds, which makes their detection challenging. Preserving spatial details, jointly modeling local and global features, and strengthening feature representation are therefore critical to improving detection accuracy and robustness. However, existing methods often fail to fully exploit multi-level feature relationships, struggle to integrate local and global complementary information, and lack mechanisms to compress redundant features while emphasizing salient information. To address these issues, we propose MACT-Net, a network designed specifically for infrared small-target detection. First, a multi-level aggregation encoder–decoder architecture with convolution-enhanced skip connections and multi-scale decoder fusion is employed to progressively abstract and reconstruct features, preserving spatial details and improving multi-scale responsiveness. Second, a CNN–Transformer hybrid feature modeling module is introduced to adaptively capture local textures and global semantic dependencies, enhancing feature representation and discriminative capability. Third, a vector quantization-based feature discretization and representation enhancement mechanism is embedded at the network bottleneck to compress redundant information and emphasize salient features, further improving robustness under low signal-to-noise ratio conditions. We evaluate MACT-Net on three public datasets: SIRST, IRSTD-1K, and NUDT-SIRST. Extensive experimental results confirm that the combination of multi-level aggregation, hybrid feature modeling, and feature discretization mechanisms effectively enhances infrared small-target detection performance.

1. Introduction

Owing to its strong anti-interference capability and its ability to operate under all-weather, all-day conditions, infrared imaging has been widely applied in areas such as imaging detection, emergency search and rescue, and environmental monitoring. Within these contexts, infrared small-target detection serves as a crucial component of infrared image processing, playing an essential role in enhancing target perception and ensuring reliable mission performance. However, infrared small targets are characterized by a low signal-to-noise ratio, limited spatial scale (often occupying only a few pixels), and a lack of distinct texture or shape features. These properties make the targets easily submerged in complex image backgrounds, leading to false alarms or even detection failures. Therefore, improving the accuracy and robustness of infrared small-target detection is of great significance for enhancing the capabilities of imaging detection, emergency search and rescue, and environmental monitoring.

Although numerous methods have been proposed to improve infrared small-target detection, existing approaches still suffer from three core issues: (1) Insufficient multi-level feature fusion leads to the loss of detail and structural information during the reconstruction process; (2) The separation between local and global feature representations makes it difficult to effectively distinguish targets from the background; (3) Redundant and unstable feature representations result in degraded detection performance under low signal-to-noise ratio (SNR) conditions.

To address the aforementioned challenges and enhance the accuracy and robustness of infrared small-target detection, we propose a novel detection network named MACT-Net, whose overall framework is illustrated in Figure 1. Specifically, a novel multi-level aggregation encoder–decoder architecture is proposed, which employs convolution-enhanced skip connections and multi-scale aggregated outputs to achieve efficient transmission and integration of hierarchical features, thereby enhancing detail preservation and multi-scale responsiveness for infrared small targets. Second, a CNN–Transformer hybrid feature modeling module is introduced, which exploits the synergy of convolution and attention to adaptively fuse local fine-grained features with global semantic information, thus improving the model’s discriminative capability and detection robustness under complex backgrounds. Finally, a vector quantization-based feature discretization and representation enhancement mechanism is adopted, which leverages feature discretization and multi-scale quantized aggregation to compress redundant information and strengthen salient features, thereby enhancing the model’s feature representation capability and detection stability under low signal-to-noise ratio conditions.

The main innovations and contributions of this work can be summarized as follows:

• Multi-Level Aggregation Encoder–Decoder Architecture: A novel multi-level aggregation encoder–decoder is proposed, built on a U-shaped structure for progressive abstraction and reconstruction of multi-level features. Unlike conventional U-shaped architectures, it introduces convolutional operations within the skip connections for same-level feature fusion, preserving richer local details and enhancing spatial structural consistency during information transmission. In the output stage, multi-scale fusion of the decoder features integrates information across different semantic levels and spatial scales. This design significantly improves detail preservation and multi-scale responsiveness for infrared small target detection.
• CNN–Transformer Hybrid Feature Modeling Module: A CNN–Transformer hybrid feature modeling module is proposed for adaptive and collaborative modeling of local and global features. The convolutional branch captures local spatial structures and texture information, while the Transformer branch uses attention to model long-range dependencies and global semantic context. Across the multi-scale feature space, the module dynamically associates fine-grained local features with global semantic context, fully combining their complementary strengths. As a result, it significantly enhances discriminative capability, feature representation, and detection robustness for infrared small targets in complex backgrounds.
• Vector Quantization-Based Feature Discretization and Representation Enhancement Mechanism: A vector quantization module is introduced at the network bottleneck to represent continuous features in a discrete form, compressing redundant information and emphasizing salient features. By aggregating multi-scale feature inputs into the quantized space, it preserves both local structures and global dependencies in the discrete representations, yielding more discriminative, compact, and interpretable reconstructions. This enhances the model’s representation capability and robustness, and further improves the stable detection of infrared small targets under low signal-to-noise ratio conditions.
• Effectiveness and strong performance: We evaluate MACT-Net on three public datasets (SIRST, IRSTD-1K, and NUDT-SIRST) using comprehensive evaluation metrics. On the small SIRST dataset, it achieves the best values across all metrics, owing to the network’s ability to fully exploit limited training samples and preserve target details. Overall, these results confirm that the proposed multi-level aggregation, CNN–Transformer hybrid feature modeling, and vector quantization mechanisms jointly enhance detection accuracy and robustness.

2. Related Work

A variety of approaches have been proposed to address the challenges of infrared small-target detection. Early studies primarily relied on traditional model-based detection methods [1,2,3,4,5,6]. However, these methods suffer from inherent limitations, as they struggle to construct feature representations capable of adequately capturing and discriminating target and background information, which restricts their detection performance in complex scenarios. With the advancement of deep learning, data-driven detection methods have emerged and increasingly surpassed the performance of traditional approaches.

2.1. CNN-Based

CNN-based infrared small-target detection methods have evolved from early local enhancement mechanisms to more advanced multi-scale fusion and spatiotemporal modeling frameworks. ACM [7] and ALCNet [8], proposed by Dai et al., enhance target saliency by leveraging asymmetric contextual modulation and attention-driven local contrast enhancement, achieving strong background suppression. However, as their feature extraction still relies heavily on local convolutional operators, their ability to capture long-range dependencies remains limited, leading to inconsistencies between local and global representations in cluttered environments.

Building upon local enhancement, Attention-guided Pyramid Context Networks [9] integrate attention-guided pyramid context aggregation to strengthen multi-scale responses. Although effective across varying target sizes, their cross-scale feature alignment remains imperfect, which may result in the loss of fine structural details during fusion.

MAPFF [10] and MSAFFNet [11] adopt multi-angle or multi-scale pyramid fusion schemes to jointly model semantic and fine-grained features, improving adaptability to diverse target morphologies. Yet, the dense aggregation of features often introduces redundancy, making the networks more sensitive to background noise, particularly in low-SNR settings.

Methods such as Multiscale Multi-level Residual Feature Fusion [12] and Dense Nested Attention Network [13] employ residual pathways and nested attention structures to enhance inter-layer interaction and preserve structural cues. However, their reliance on local convolutions limits global dependency modeling, restricting their ability to capture consistent target boundaries in highly complex scenes.

UIU-Net [14] and the Moderately Dense Adaptive Feature Fusion Network [15] further introduce hierarchical architectures and adaptive fusion strategies to improve the localization of dim targets. Nevertheless, without strong global constraints, these models may still overemphasize local patterns and exhibit insufficient global coherence when facing extensive background clutter.

Regarding supervision, Mapping Degeneration Meets Label Evolution [16] uses single-point supervision with label evolution to progressively refine target regions, while ISNet [17] employs shape-aware edge modules to enhance contour extraction. Despite their advantages, boundary representations remain vulnerable to noise in extremely low-SNR scenarios, leading to reduced stability.

For dynamic scenes, Deng et al. [18] exploit spatial-temporal topological relationships to preserve stable target associations under viewpoint variation and occlusion, and SAT [19] incorporates global context and reliability-aware updating to suppress model drift. However, temporal association and tracking-based mechanisms may still accumulate errors under highly dynamic clutter, challenging the robustness of motion-aware features.

Overall, although CNN-based methods have significantly advanced in local contrast enhancement, multi-scale fusion, weak supervision, and spatiotemporal modeling, their intrinsic limitations—such as insufficient multi-layer feature fusion, lack of global contextual consistency, and reduced robustness under low SNR—still manifest across different approaches in varying degrees.

2.2. Transformer-Based

As the limitations of pure CNN-based methods become evident in complex background and low-SNR scenarios, researchers have introduced Transformers to leverage their global context modeling capabilities for infrared small-target detection. For example, SCTransNet [20] and PBT [21] incorporate spatial–channel cross-attention and progressive background modeling mechanisms, respectively, thereby strengthening global contextual representation and improving target saliency. However, their use of limited multi-scale structures constrains cross-layer feature fusion, and fine-grained structural details may still be lost during reconstruction.

Subsequently, methods such as St-Trans [22] and STA-KFE [23] expand the modeling scope by integrating temporal cues into Transformer architectures. These designs exploit spatiotemporal consistency to better detect moving targets. Nevertheless, their relatively weak reliance on local convolution may compromise the representation of local textures and small-scale variations, resulting in a certain degree of decoupling between local and global features and reducing the separability between targets and cluttered backgrounds.

More advanced architectures, such as the Interior Attention-aware Network [24] and Occlusion-aware with Local–Global Features Network [25], further enhance attention stacking and local–global feature modeling to improve feature stability. Yet under extremely low SNR conditions, Transformer attention responses may still be disturbed by background noise, generating redundant or unstable representations and ultimately degrading detection performance.

Overall, although pure Transformer architectures have made significant progress in global context modeling, spatiotemporal dependency capture, and multi-frame feature aggregation, their limited responsiveness to fine-grained local details, insufficient multi-scale feature fusion, and reduced robustness under low-SNR conditions mean that existing Transformer-based methods still face challenges related to insufficient cross-layer fusion, local–global representation gaps, and unstable feature representations to varying degrees.

2.3. CNN–Transformer Hybrid Strategy

To harness both the local detail extraction capability of CNNs and the long-range dependency modeling of Transformers, hybrid architectures have become a major trend in infrared small-target detection. Representative works such as FATCNet [26], FTC-Net [27] fuse convolutional and attentional features within their backbones, achieving improved discriminability by jointly enhancing fine-grained target sensitivity and global background suppression. However, their complex multi-layer fusion mechanisms may still weaken structural details during cross-scale propagation, leaving certain fine-scale cues insufficiently preserved.

Subsequent approaches, such as MTU-Net [28], adopt dual-branch U-shaped decoders, in which CNN-encoded local textures are progressively integrated with the Transformer-derived global dependencies. While enabling a more comprehensive multi-scale representation, these models may still suffer from inconsistencies between local and global features across different depths, thereby reducing the stability of target–background separation in cluttered scenes.

Moreover, although several hybrid models utilize attention modulation to enhance robustness, their fused representations may still contain redundant noise components from both convolutional and attention pathways under low-SNR conditions. This often manifests as unstable detection scores or missed dim targets, suggesting that hybrid strategies still face challenges in extreme environments.

Overall, although CNN–Transformer hybrid strategies have achieved notable progress in balancing local detail and global semantics, multi-scale feature fusion, and attention modulation, limitations in fusion strategy and cross-layer alignment, as well as potential redundancy or instability of fused features under low-SNR or extremely dim targets, indicate that existing hybrid methods still face challenges regarding insufficient multi-layer fusion, local–global representation inconsistency, and unstable feature robustness.

2.4. Summary and Limitations

In summary, a variety of methods based on CNNs, Transformers, and CNN–Transformer hybrids have emerged in the field of infrared small-target detection, achieving significant progress in local contrast enhancement, multi-scale feature fusion, weakly supervised learning, and spatiotemporal modeling. For instance, CNN-based methods improve reconstruction of structural and detailed information through multi-angle pyramid fusion and nested attention modules; pure Transformer architectures leverage global context modeling and spatiotemporal dependencies to enhance background suppression and moving target detection; hybrid models further combine local texture sensitivity with global semantic representation, enabling more comprehensive feature discrimination. Nevertheless, despite these advancements, their performance is still constrained by the following three main issues:

(1) Insufficient multi-layer feature fusion: Some methods fail to fully exploit multi-layer information, causing structural and detailed features to be partially lost during reconstruction. For example, in pyramid feature fusion or nested attention modules, shallow-layer details may be weakened during upsampling or integration, which adversely affects accurate representation of target shapes, edges, and fine-grained textures.

(2) Local–global feature inconsistency: Due to the inherent locality of convolutions, long-range dependencies are difficult to capture, resulting in a disconnect between local and global features, which reduces target–background separability. Specifically, convolutional features perform well in capturing local textures but lack global context constraints; in complex backgrounds, they may misinterpret background textures as targets or overlook weak target regions, compromising detection robustness.

(3) Feature instability under low SNR: Under low signal-to-noise ratio conditions, convolutional or fused features tend to be redundant, unstable in convergence, or sensitive to background noise, leading to degraded detection performance. This can manifest as missed weak targets, fluctuating detection probabilities, or abnormal feature responses, especially in dynamic backgrounds or long-range scenarios. Even with hybrid strategies incorporating Transformers or attention mechanisms, extreme noise can still impair feature representation.

Overall, although existing methods have significantly enhanced feature representation and detection capability, these three limitations continue to restrict performance in highly challenging environments.

3. Proposed Method

3.1. Overview

The proposed MACT-Net framework is mainly composed of three modules: an encoder module, a decoder module, and a bottleneck module. The detailed architecture is shown in Figure 1. The overall framework adopts a U-shaped architecture to enable progressive abstraction and reconstruction of multi-level features.

In the encoding stage, the network consists of three CNN–Transformer Hybrid Modules that produce feature maps with gradually decreasing resolutions. Each Hybrid Module contains two branches: the convolution branch captures local spatial structures and texture details, while the Transformer branch models long-range dependencies and global semantic context through attention mechanisms. The two branches are fused via concatenation, dynamically establishing connections between fine-grained local features and global semantic information across multiple feature scales, thereby enhancing discriminability, feature expressiveness, and robustness for infrared small-target detection in complex backgrounds.

In the Bottleneck Module, we first employ a CNN–Transformer Hybrid Module to further reduce the feature resolution. This design encourages stronger global semantic aggregation at a compact spatial scale and reduces computational cost for the subsequent quantization process. Then, a Vector Quantization block is applied to convert continuous features into discrete representations, compressing redundant information while highlighting structurally salient features.

During the decoding stage, three symmetric CNN–Transformer Hybrid Modules are used. After feature concatenation, transposed convolutions progressively upsample the feature maps. Additionally, convolutional layers are introduced in the skip connections to refine same-level feature fusion, preserving richer local details and enhancing spatial structural consistency. Finally, multi-scale feature fusion is applied to the decoder outputs to achieve multi-level aggregated predictions, effectively integrating information from different semantic depths and spatial scales, and thereby improving detail preservation and multi-scale responsiveness for infrared dim-small target detection.

Specifically, given an input image of size $256\times 256$, the encoder employs three CNN-Trans Hybrid Modules whose outputs contain 64, 128, and 256 channels, with the spatial resolution successively halved to $1/2$, $1/4$, and $1/8$ of the input. In each module, the convolutional branch and the dual-attention Transformer branch are fused by concatenation followed by a $1\times 1$ convolution, after which a $2\times$ downsampling operation reduces the resolution. The bottleneck first applies a fourth Hybrid Module that further halves the resolution to $1/16$ and produces a 512-channel feature; this feature is then projected to 1024 channels and discretized by the codebook ($N_s=2048$, dimension 1024, $\beta =0.25$). Symmetrically, the decoder adopts three Hybrid Modules that progressively restore the spatial resolution through transposed convolutions, while convolution-enhanced skip connections refine each encoder feature with a lightweight CNN block before same-level fusion. Finally, the decoder features from multiple scales are aggregated by a multi-scale fusion module to generate a single-channel prediction at the original $256\times 256$ resolution.

3.2. CNN-Trans Hybrid Module

As shown in Figure 1a, each CNN-Trans Hybrid Module consists of two branches: a convolutional branch that effectively captures local spatial structures and texture details, and a Transformer branch that models long-range dependencies and global semantic context through attention mechanisms. The two branches are then fused via concatenation followed by convolution/deconvolution operations, producing feature representations that jointly encode both local details and global semantics for the subsequent stages. The specific execution process is shown in Equation (1):

$$T_i=Conv(CNN\left(Y_{Ci}\right)\oplus Trans\left(Y_{Ti}\right))$$

(1)

Among them, ⊕ denotes concatenation. During the encoding stage of the CNN-Trans Hybrid Module, the spatial resolution of the input feature maps progressively decreases as the corresponding receptive field expands, while the opposite trend occurs in the decoding stage. This design effectively leverages both local and global feature representations through the CNN branch and the Transformer branch with dual attention mechanisms, enabling comprehensive contextual modeling and extraction of subtle target features. By fusing these complementary features via concatenation followed by convolution/deconvolution operations, the module integrates local and global information into a unified representation. This hybrid architecture enhances feature expressiveness and robustness, highlights subtle structures or small targets, and maintains computational efficiency, providing high-quality, discriminative feature maps for subsequent stages of processing.

(1) CNN Block: The CNN branch is implemented as a lightweight convolutional module composed of a CondConv layer followed by a basic CNN layer, as illustrated in Figure 2. CondConv is introduced to enhance the feature representation capability of the model. Before applying convolution, the kernel in the CondConv layer is expressed as a linear combination of multiple expert weights, allowing the network to adaptively generate convolution parameters for different inputs. This mechanism increases the model’s capacity and expressiveness while maintaining efficient inference. Let the number of experts be m. For the i-th expert kernel ${\omega }_i(i=1,2,\dots ,m)$, an input-dependent scalar weight ${\alpha }_i$ is obtained by computing attention over the current frame X. This weight modulates the corresponding expert kernel, and a dynamic convolutional kernel is formed through the weighted linear combination of all expert kernels. Therefore, the output of the CondConv layer for the input X can be formulated as in Equation (2):

$$CondConv\left(X\right)=ReLU(BN(({\displaystyle \sum _{i=1}^m}{\alpha }_i{\omega }_i)\ast X))$$

(2)

As the number of experts increases, the representational power of the CondConv layer is further enhanced. Unlike standard convolutions, which apply the same kernel repeatedly across all spatial locations of the input feature map, CondConv performs a weighted combination of expert kernels just once per input. This design improves model capacity while maintaining inference efficiency. After generating the dynamic convolutional kernels in the CondConv layer, the final output $Y_c$ of the CNN block is obtained by applying the subsequent conventional convolution, batch normalization, and ReLU, as shown in Equation (3):

$$CNN\left(Y_C\right)=ReLU\left(BN\left(Conv\left(CondConv\left(X\right)\right)\right)\right)$$

(3)

This adaptive convolutional design offers several advantages. First, by generating input-dependent kernels, CondConv allows the network to dynamically adjust its feature extraction strategy to different input patterns, improving expressiveness and the ability to capture diverse spatial structures. Second, the combination of multiple expert kernels increases the representational capacity without a proportional increase in computational cost, enabling richer feature representations even with a lightweight architecture. Third, unlike applying a fixed kernel uniformly, the expert-weighted mechanism selectively emphasizes relevant features while suppressing less informative ones, which is particularly beneficial for detecting subtle structures or small targets. Finally, when followed by a conventional convolution, batch normalization, and ReLU, the CNN branch produces stable, high-quality feature maps that complement the adaptive capabilities of CondConv, resulting in a flexible yet efficient module capable of robust feature extraction under varying input conditions.

(2) Transformer Block: The Transformer Block consists of dual attention mechanisms: Efficient Attention (spatial attention) and Transpose Attention (channel attention). Combining these two attention branches enables more effective modeling of contextual information. Figure 1c illustrates the overall architecture of the Transformer Block, which mainly includes Efficient Attention, Transpose Attention, a feed-forward network (FFN), layer normalization, etc. Based on dual attention mechanisms, the Transformer Block processes the input through a sequence of spatial attention, channel attention, and nonlinear transformations, as expressed in Equations (4)–(7):

$$X_E=\mathcal{EFF}\left(X\right)+X$$

(4)

$$X_M=MLP\left(LN\left(X_E\right)\right)+X_E$$

(5)

$$X_T=\mathcal{TRS}\left(X_M\right)+X_M$$

(6)

$$Trans\left(Y_T\right)=MLP\left(LN\left(X_T\right)\right)+X_T$$

(7)

Here, $\mathcal{EFF}(\ast )$ denotes the efficient attention, $\mathcal{TRS}(\ast )$ represents the transpose attention operation, while $LN$ and $MLP$ correspond to the layer-normalization unit and the Mix-FFN feed-forward network, respectively. The Mix-FFN can be formulated as

$$MLP\left(X_{\ast }\right)=FC\left(GELU\left(DW\text{-}Conv\left(FC\left(X_{\ast }\right)\right)\right)\right)$$

(8)

where $FC$ is the fully connected layer, $GELU$ refers to GELU activation, and $DW\text{-}Conv$ refers to depth-wise convolution.

In the efficient attention mechanism, the input vector X ($H,W,C$) is linearly projected to obtain the query, key, and value representations:

$$\mathcal{Q}=W_{Q}X,\mathcal{K}=W_{K}X,\mathcal{V}=W_{V}X$$

(9)

The spatial domain ($n=H\times W$) is interpreted as $d_k$ channel-wise weighting maps, where each key channel modulates all spatial locations of the value features. Global contextual information is then obtained through a weighted aggregation:

$$\mathcal{EFFC}(\mathcal{Q},\mathcal{K},\mathcal{V})={\rho }_q\left(\mathcal{Q}\right)\left({\rho }_k\left({\mathcal{K}}^T\right)\mathcal{V}\right)$$

(10)

where ${\rho }_q$ and ${\rho }_k$ denote the normalization functions applied to $\mathcal{Q}$ and $\mathcal{K}$. This formulation replaces position-wise pairwise correlations with channel-wise global context encoding, enabling long-range spatial interactions with linear computational complexity and effectively enhancing subtle responses of small targets.

Complementing Efficient Attention, Transpose Attention performs contextual modeling along the channel dimension. It first computes channel-wise correlations using the cross-covariance matrix, followed by temperature-scaled normalization:

$${\mathcal{G}}_T=\mathrm{Softmax}({\mathcal{K}}^T\mathcal{Q}/\tau )$$

(11)

where ${\mathcal{G}}_T$ denotes the channel-attention matrix, and the temperature factor $\tau$ compensates for the $l_2$-normalization applied to $\mathcal{Q}$ and $\mathcal{K}$, stabilizing optimization. The output is then obtained by applying channel attention to the value features:

$$\mathcal{TRS}(\mathcal{Q},\mathcal{K},\mathcal{V})=\mathcal{V}{\mathcal{G}}_T$$

(12)

Unlike Efficient Attention that focuses on spatial dependencies, Transpose Attention emphasizes inter-channel relationships, adaptively enhancing target-relevant channels while suppressing redundant or background-driven channels, providing complementary capabilities to spatial modeling.

The integration of Efficient Attention and Transpose Attention enables the Transformer Block to capture complementary spatial and channel dependencies, resulting in more discriminative and stable feature representations.

Efficient Attention reformulates spatial dependency modeling into a linear-complexity operation, allowing effective long-range context extraction on infrared imagery while avoiding the computational and memory burdens of conventional self-attention. Its global context aggregation amplifies subtle spatial responses induced by small targets, thereby improving target–background separability.

Transpose Attention complements this by modeling inter-channel correlations through cross-channel covariance. It adaptively enhances target-relevant channels and suppresses redundant or background-driven components, addressing the limitations of spatial attention in capturing channel-level discriminative cues. This property is particularly important for infrared small targets, which occupy only a few pixels spatially but exhibit sparse and informative responses across channels.

Through residual integration with the feed-forward network, Efficient and Transpose Attention jointly form a unified representation that leverages both global spatial semantics and channel-selective enhancement. Efficient Attention improves long-range spatial context and amplifies subtle spatial responses induced by small targets, while Transpose Attention emphasizes sparse target-relevant channels and suppresses background interference. Their combination increases target saliency and yields clearer, more robust feature representations under low SNR and complex scene conditions. This dual-attention design is particularly effective for infrared small-target detection, where targets occupy only a few pixels but exhibit informative responses across channels.

3.3. Vector Quantization Block

To enable the mapping from continuous latent feature vectors to a discrete representation space, a vector quantization-based latent encoding discretization module is constructed, as illustrated in Figure 1d. The module employs a learnable embedding matrix as a codebook, which stores a set of latent prototype vectors with fixed dimensionality. During initialization, the codebook vectors are randomly sampled from a controlled uniform distribution to ensure sufficient representational diversity at the early training stage.

Let the encoder output a continuous latent tensor:

$$T_4\in {\mathbb{R}}^{B\times C_4\times H_4\times W_4}$$

(13)

where B, $C_4$, $H_4$, and $W_4$ denote the batch size, number of channels, and spatial dimensions, respectively. The feature dimension is first moved to the last position, and the spatial dimensions are flattened to obtain a sequence of vectors:

$$T_f\in {\mathbb{R}}^{\left(B{H}_4{W}_4\right)\times C_4}$$

(14)

A set of codebook vectors is defined as:

$$E=\left\{e_j\in {\mathbb{R}}^C\mid j=1,\dots ,N_s\right\}$$

(15)

where $N_s$ is the number of codewords(set to 2048 in our experiments). For each latent vector $T_f^{\left(i\right)}$, the squared Euclidean distance to all codewords can be written as:

$$d_{ij}={∥T_f^{\left(i\right)}-e_j∥}_2^2={∥T_f^{\left(i\right)}∥}_2^2+{∥e_j∥}_2^2-2{\left(T_f^{\left(i\right)}\right)}^T{e}_j$$

(16)

The quantization operation is implemented using a nearest-neighbor criterion:

$$s_i=arg\underset{j}{\min }{d}_{ij}$$

(17)

$$T_q^{\left(i\right)}=e_{s\left(i\right)}$$

(18)

All selected codeword vectors are then reshaped according to the original spatial structure to form the quantized latent representation:

$$T_q\in {\mathbb{R}}^{B\times C_4\times H_4\times W_4}$$

(19)

To ensure the trainability of the quantization module, a two-component quantization loss is introduced. The first component, the codebook update term, encourages the codewords to move toward their corresponding latent features:

$${\mathcal{L}}_{cd}={∥sg\left[T_4\right]-T_q∥}_2^2$$

(20)

where $sg\left[\ast \right]$ denotes the stop-gradient operator. The second component, the commitment term, constrains the encoder outputs to match the discrete embedding space:

$${\mathcal{L}}_{com}=\beta {∥T_4-sg\left[T_q\right]∥}_2^2$$

(21)

where $\beta$ is a weighting factor ($\beta$ = 0.25). The overall quantization loss is then given by:

$${\mathcal{L}}_{vq}={\mathcal{L}}_{cd}+{\mathcal{L}}_{com}$$

(22)

Since the discrete selection operation is non-differentiable, the straight-through estimator (STE) is applied during backpropagation, allowing gradients to propagate directly through the continuous variables. Its implementation is expressed as:

$$T_{ste}=T+sg\left(T_q-T_4\right)$$

(23)

where the quantized output $T_q$ is used in the forward pass, while gradients are computed with respect to $T_4$ in the backward pass, thus forming an optimizable discrete latent representation learning framework.

In the above vector quantization-based latent encoding module, mapping continuous latent features to a finite set of codebook prototype vectors realizes the discretization of latent representations. This process not only effectively suppresses random noise and minor perturbations in the latent features but also removes redundant or irrelevant information in the latent space, highlighting the critical features of weak targets. At the same time, the discretized representation compresses the feature space, reinforcing the distinction between small targets and background regions, thereby improving the discriminability of micro targets. Moreover, the codebook’s shared prototype mechanism reduces the network’s dependence on large-scale training data, while the straight-through estimator (STE) ensures differentiability and stable convergence during training. Overall, the vector-quantized latent encoding module enhances infrared small-target detection by providing more robust, structured, and discriminative feature representations through noise suppression, residual information removal, and improved training stability.

3.4. Encoder Module

In the encoder, three CNN-Trans Hybrid Modules are sequentially cascaded to construct a hierarchical feature abstraction process. The input is first fed into the initial Hybrid Module, where the convolutional branch and the Transformer branch operate in parallel to extract fine-grained local structures and model long-range global dependencies, respectively. A 2× downsampling operation is then applied via convolution, progressively reducing spatial resolution while expanding the channel dimensionality to enhance semantic capacity. The resulting features are subsequently forwarded to the second and third Hybrid Modules, each repeating the same procedure of dual-branch feature extraction, fusion, and the “lower-resolution, higher-channel” transformation. Through this progressive process, the encoder forms a multi-level hybrid representation that evolves from low-level textures to high-level semantics, integrating both local sensitivity and global contextual coherence. The final encoded representation is thus a compact, structurally enriched, and semantically strengthened high-dimensional feature map suitable for downstream decoding or detection tasks.

This architectural design offers several significant benefits. The hierarchical abstraction enables the network to capture features across multiple scales, enhancing both representational richness and discriminative ability. The progressive downsampling reduces computational burden, allowing efficient global attention modeling even for high-resolution infrared images, while channel expansion further strengthens the feature capacity necessary for distinguishing weak targets from cluttered backgrounds. Moreover, the complementary integration of CNN-based local modeling and Transformer-based global context reasoning ensures stable and consistent feature representation. Such a design is particularly advantageous for infrared small-target detection, where targets are typically weak and embedded in strong and non-uniform background clutter, enabling the encoder to deliver robust and high-quality deep representations for subsequent detection stages.

3.5. Bottleneck Module

Building upon the progressive abstraction in the encoder, the Bottleneck Module further assumes the critical role of feature compression and semantic aggregation. First, as the opening stage of the bottleneck, the network incorporates the fourth CNN-Trans Hybrid Module to fuse fine-grained local textures with cross-region global dependencies in a lower-resolution feature space, thereby producing a more semantically concentrated representation $T_4$. This arrangement ensures that structural details and contextual cues relevant to weak infrared targets are preserved even under strong compression.

The resulting feature $T_4$ is then fed into the Vector Quantization block to map continuous latent features into a discrete representation space. Vector Quantization leverages a learnable codebook to aggregate multi-scale features into a quantized domain, effectively compressing redundant patterns while projecting salient responses into sparse, compact codes. Compared with continuous representations, the discretized encoding yields more compact and discriminative reconstructions, facilitating the emphasis of target-related information under high-noise conditions.

The unified expression of multi-scale information in the quantized space preserves local structures and global dependencies simultaneously, suppressing background-induced perturbations and stabilizing key responses. Through this structured discrete reconstruction mechanism, the network attains enhanced robustness to noise and improved discriminability in low SNR scenarios, thereby strengthening the stability and reliability of infrared small-target detection.

3.6. Decoder Module

Building upon the compressed and discretized bottleneck representations, the Decoder Module aims to progressively restore spatial resolution while reconstructing detailed structures that are critical for reliable infrared small-target expression. To effectively exploit the multi-level encoder outputs, each encoder feature map is first processed by a lightweight CNN Block, which enhances local textures and structural cues, making them more suitable for cross-scale fusion. These refined encoder features are then combined with the corresponding decoder inputs, forming enriched fused representations that supply both semantic context and spatial detail for the subsequent reconstruction stages.

Following this fusion process, the Decoder Module employs a cascade of three CNN-Trans Hybrid Modules to incrementally recover spatial structure and semantic granularity. Mirroring the downsampling design in the encoder, the convolution branch of each Hybrid Module performs a 2× upsampling to restore spatial resolution while reducing channel dimensionality, enabling a smooth transition from abstract high-level semantics to fine-grained detail structures. Meanwhile, the Transformer branch models long-range dependencies that conventional convolution-based upsampling struggles to reconstruct, ensuring that small, low-contrast infrared targets retain coherent global relationships during the decoding phase.

At the final reconstruction stage, the outputs of all decoder layers are aggregated and passed through a concluding convolution layer to generate the final prediction. This design effectively integrates information from multiple semantic depths, yielding a representation that preserves both fine-detail fidelity and robust global structure, ultimately enhancing the accuracy and stability of infrared small-target recovery in complex and low-SNR environments.

3.7. Module Synergy Analysis

The three components of MACT-Net operate as a complementary pipeline rather than as independent blocks. The multi-level aggregation encoder–decoder provides the multi-scale backbone that transmits and aligns features across resolutions, supplying the structural skeleton for reconstruction. Within each scale, the CNN–Transformer hybrid module enriches these features by binding convolutional local textures to dual-attention global context, so that small targets acquire both spatial precision and contextual support. At the most abstract scale, the vector quantization bottleneck purifies the deepest representation, projecting it onto a set of learned prototypes to remove background-driven noise and redundancy before decoding. In this way, aggregation supplies structure, the hybrid module supplies discriminative content, and quantization supplies stability: the convolution-enhanced skip connections preserve the local detail that quantization deliberately discards at the bottleneck, while the multi-scale fusion head recombines the decoder features into the final prediction. The ablation studywhere removing any single component degrades all three metrics—lowering $P_d$ and IoU while increasing $F_a$—provides empirical evidence that the three modules contribute jointly rather than additively.

4. Experiment

4.1. Experimental Setup

(1) Datasets: To assess the performance of the proposed MACT-Net, we employ three widely used infrared small-target datasets: SIRST [7], IRSTD-1K [17], and NUDT-SIRST [13]. These datasets collectively cover diverse and challenging infrared imaging conditions, including sea surfaces, sky regions, urban areas, ground scenes, and other cluttered environments.

The SIRST dataset is recognized as the earliest real-world single-frame infrared small-target benchmark with carefully curated image–label pairs. It adopts a training/testing split of roughly 4:1, providing 341 training samples and 86 testing samples.

The IRSTD-1K dataset offers 1001 finely annotated images with pixel-level ground truth. It includes targets of different scales and shapes under complex backgrounds. The standard partition contains 800 training images and 201 testing images.

The NUDT-SIRST dataset is a large-scale synthetic infrared dataset composed of real targets blended with real background scenes. It contains 663 training images and 664 test images, covering five representative background categories: field, highlight, city, sea, and cloud. This dataset provides rich scene variability and background clutter while maintaining realistic infrared characteristics.

(2) Evaluation Metrics: In this study, four widely adopted evaluation indicators are employed to thoroughly assess the performance of infrared small-target detection. These include the intersection over union (IoU), probability of detection ($P_d$), false-alarm rate ($F_a$), and the receiver operating characteristic (ROC) curve.

Intersection over union (IoU): The intersection over union (IoU) serves as an essential pixel-level indicator for assessing infrared small-target detection accuracy. It is computed by taking the overlap between the predicted target region and the ground-truth annotation as the numerator, and the combined area of both regions as the denominator, forming a ratio that reflects spatial agreement. The formulation is presented in Equation (24):

$$IoU={\displaystyle \frac{P_{intersection}}{P_{union}}}$$

(24)

Here, $P_{intersection}$ represents the overlapping region shared by the predicted target and the ground-truth annotation, while $P_{union}$ denotes the total area covered by the two regions combined.

Probability of detection ($P_d$): The probability of detection ($P_d$) assesses detection performance at the target level. It is defined as the proportion of correctly identified targets, computed by dividing the number of true detected targets by the total number of ground-truth targets. The formulation is provided in Equation (25):

$$P_d={\displaystyle \frac{T_{correct}}{T_{all}}}$$

(25)

Here, $T_{correct}$ denotes the number of targets successfully identified by the detector, while $T_{all}$ refers to the total count of ground-truth targets.

False-alarm rate ($F_a$): The false-alarm rate ($F_a$) measures detection reliability at the pixel level. It is quantified by calculating the proportion of pixels that are incorrectly identified as targets, obtained by dividing the number of false target pixels by the total number of pixels in the image. The definition is given in Equation (26):

$$F_a={\displaystyle \frac{P_{false}}{P_{all}}}$$

(26)

Here, $P_{false}$ represents the number of pixels incorrectly classified as targets, while $P_{all}$ denotes the total number of pixels in the image.

Receiver operating characteristic (ROC) curve: This metric characterizes the relationship between the false positive rate (FPR) and the true positive rate (TPR). In contrast to IoU, the ROC curve reflects the overall detection behavior across varying decision thresholds. The definitions of FPR and TPR are provided in Equation (27):

$$FPR={\displaystyle \frac{FP}{FP+TN}},TPR={\displaystyle \frac{TP}{TP+FN}}$$

(27)

False Positives($FP$) refer to pixels that are incorrectly classified as targets by the algorithm, meaning they actually belong to the background but are misidentified as target pixels.

True Negatives ($TN$) denote pixels correctly identified as background, i.e., pixels that truly belong to the background and are accurately classified. The sum $FP+TN$ represents the total number of background (non-target) pixels across the entire test set.

True Positives ($TP$) correspond to pixels correctly detected as targets by the algorithm.

False Negatives ($FN$) refer to pixels where the algorithm fails to detect the target, i.e., pixels that actually belong to the target but are misclassified as background. The sum $TP+FN$ represents the total number of target pixels in the test set.

(3) Implementation Details: The proposed MACT-Net is implemented in PyTorch (version 2.5.0) with CUDA acceleration. During training, data augmentation is performed via random scaling, cropping, and horizontal flipping, where the scaling and cropping are controlled by a base size and a crop size; at inference, the input images are resized to $256\times 256$. Network weights are initialized using the Xavier scheme, and deep supervision is not adopted; since the multi-scale decoder fusion already aggregates predictions across scales, the network produces a single final prediction.

In the vector quantization bottleneck, the codebook comprises $N_s=2048$ learnable codewords of dimension $C_4=1024$, which matches the channel dimension of the bottleneck feature; the codewords are initialized from a uniform distribution $\mathcal{U}(-1/N_s,1/N_s)$, the weighting factor is set to $\beta =0.25$, and the straight-through estimator is used to back-propagate gradients through the discrete assignment. The network is optimized end-to-end with the combined objective $\mathcal{L}={\mathcal{L}}_{\mathrm{SoftIoU}}+{\mathcal{L}}_{vq}$. The Soft-IoU loss is formulated as

$${\mathcal{L}}_{\mathrm{SoftIoU}}=1-{\displaystyle \frac{{\sum }_i{p}_i{g}_i}{{\sum }_i{p}_i+{\sum }_i{g}_i-{\sum }_i{p}_i{g}_i}},$$

(28)

where $p=\sigma (·)$ is the predicted probability map and g is the binary ground truth, and ${\mathcal{L}}_{vq}={\mathcal{L}}_{cd}+{\mathcal{L}}_{com}$ is the vector quantization loss defined in Equations (20)–(22) with $\beta =0.25$. We adopt the Adam optimizer with an initial learning rate of $1\times {10}^{-4}$ and a weight decay of $1\times {10}^{-8}$, and the learning rate is dynamically adjusted by a CosineAnnealingLR scheduler (minimum learning rate $1\times {10}^{-6}$). Training is conducted with a batch size of 8 for up to 5000 epochs, and the checkpoint with the highest test IoU is retained as the final model. Because the public benchmarks provide only training and test splits, the model is selected on the test set, following common practice in single-frame infrared small-target detection; the cross-dataset evaluation in Section 3.3, in which each model is selected on its source dataset and then tested on unseen datasets, indicates that the reported gains are not an artifact of this selection.

4.2. Comparison with State-of-the-Art Methods

We perform comprehensive comparisons across multiple dimensions against the following representative methods.

• Model-driven methods: Max-Median [29], Top-Hat [30], IPI [31], TLLCM [32], RIPT [33], NRAM [34], PSTNN [35], WSLCM [36] and MSLSTIPT [3].
• Data-driven methods: MDvsFA [37], ACMNet [7], ALCNet [8], FC3-Net [38], DNA-Net [13], ISNet [17], Dim2Clear [39], IR-TransDet [40], SCTransNet [20], AGPCNet [9], UIUNet [14], MSHNet [41], L2SKNet [42], RPCANet [43].

(1) Quantitative Evaluation: We evaluate the proposed method on the three widely used public infrared small-target detection benchmark datasets, and the quantitative results are reported in

Table 1. Quantitative analysis of different methods. The best-performing method is marked in RED, and the second-best method is marked in BLUE.

Method Description	SIRST			IRSTD-1K			NUDT-SIRST
	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)
Max-Median [29]	69.20	55.33	4.172	65.21	59.73	6.998	58.41	36.89	4.197
Top-Hat [30]	79.84	1012	7.143	75.11	1432	10.06	78.41	166.7	20.72
IPI [31]	85.55	11.47	25.67	81.37	16.18	27.92	74.49	41.23	17.76
TLLCM [32]	79.09	5899	1.029	77.39	6738	3.311	62.01	1608	2.176
RIPT [33]	79.08	22.61	11.05	77.55	28.31	14.11	91.85	344.3	29.44
NRAM [34]	74.52	13.85	12.16	70.68	16.93	15.25	56.40	19.27	6.927
PSTNN [35]	77.95	29.11	22.40	71.99	35.26	24.57	66.13	44.17	14.85
WSLCM [36]	77.95	5446	1.158	72.44	6619	3.452	56.82	1309	2.283
MSLSTIPT [3]	82.13	1131	10.30	79.03	1524	11.43	47.40	888.1	8.342
MDvsFA [37]	89.35	56.35	60.30	83.71	88.52	37.34	85.22	95.37	35.86
ACMNet [7]	96.33	9.33	72.33	90.58	21.78	60.97	95.97	10.18	67.08
ALCNet [8]	97.34	20.21	74.31	89.25	27.71	65.68	96.19	30.40	72.89
FC3-Net [38]	98.13	3.21	74.75	92.93	15.73	64.98	–	–	–
DNA-Net [13]	97.71	12.80	76.24	91.84	17.61	65.71	96.93	12.78	79.98
ISNet [17]	94.52	18.34	71.15	92.59	27.92	62.88	92.59	34.65	67.86
Dim2Clear [39]	99.10	6.72	77.29	93.75	20.93	66.34	96.23	9.17	81.37
IR-TransDet [40]	87.34	9.86	70.28	87.62	13.76	61.45	89.77	9.61	76.92
SCTransNet [20]	82.32	6.01	61.43	93.27	10.74	68.15	96.55	5.91	80.13
AGPCNet [9]	–	–	–	89.56	18.56	64.92	97.35	6.78	85.35
UIUNet [14]	93.58	20.99	73.36	89.23	21.52	65.37	91.43	1.89	88.71
MSHNet [41]	96.33	3.02	72.36	93.88	15.03	67.16	97.99	11.77	80.55
L2SKNet [42]	97.25	14.73	59.44	90.24	17.46	67.81	97.57	5.33	93.58
RPCANet [43]	95.41	7.42	70.37	82.07	4.39	63.21	95.98	3.06	89.33
MACT-Net (Ours)	99.12	2.48	77.66	88.44	6.07	67.90	98.94	6.14	91.73

Note: “–” indicates that the method has not been evaluated on this dataset by its original authors.

. As shown in the table, data-driven approaches exhibit a clear performance advantage over model-driven methods across all datasets. Notably, among the data-driven methods, the proposed MACT-Net achieves highly competitive performance and outperforms existing approaches on the majority of evaluation metrics. Specifically, on the SIRST dataset, MACT-Net attains the best performance across all metrics. On the IRSTD-1K and NUDT-SIRST datasets, the proposed method achieves the best or second-best results on most evaluation criteria: on NUDT-SIRST it obtains the best $P_d$ and the second-best IoU, and on IRSTD-1K it obtains the second-best $F_a$ and IoU. The one exception is the detection probability on IRSTD-1K, which is relatively lower; we analyze this case in detail later in this section. Apart from this single metric, MACT-Net remains robust and effective under diverse infrared target detection scenarios.

Compared with traditional model-driven approaches, the proposed MACT-Net achieves a substantial performance improvement. This advantage mainly stems from the challenging characteristics of the evaluated datasets, which involve diverse signal-to-noise ratios, complex background clutter, and significant variations in target scale and shape. MACT-Net is specifically designed to learn robust and discriminative feature representations that are less sensitive to scene variations. In contrast, conventional model-driven methods are typically developed under strong assumptions tailored to particular scenarios, such as specific backgrounds (e.g., sky, sea, or land) or predefined target sizes. These scenario-dependent designs inherently constrain their generalization capability when applied to diverse and complex environments. Furthermore, as reported in Table 1, the performance gain in terms of the $IoU$ metric is noticeably larger than that observed for the $P_d$ metric. This phenomenon can be attributed to the fact that traditional methods primarily emphasize coarse target localization, whereas MACT-Net is more effective in achieving accurate pixel-level delineation, leading to superior overlap accuracy.

Compared with other data-driven approaches, the proposed MACT-Net demonstrates a noticeable performance advantage. This improvement can be attributed to its enhanced capability in modeling long-range dependencies and deep hierarchical representations. First, MACT-Net adopts a multi-level aggregation encoder–decoder architecture, which enables progressive abstraction and reconstruction of features across different semantic levels. The convolution-enhanced skip connections effectively preserve local structural details while facilitating stable information transmission, leading to richer deep feature representations. Second, the introduced CNN–Transformer hybrid feature modeling module enables collaborative learning of local spatial structures and global contextual dependencies. By jointly exploiting convolutional inductive biases and attention-based global modeling, the network is able to capture both dynamic and static contextual information from 2D feature maps, thereby selectively emphasizing informative deep features. Finally, the vector quantization-based feature discretization and representation enhancement mechanism aggregates multi-scale features into a compact discrete space, promoting effective multi-level feature fusion and improving representation robustness. Through this hierarchical and collaborative design, MACT-Net can more effectively extract and integrate intrinsic target characteristics, resulting in enhanced feature perception and improved detection performance.

It can be observed that the performance gains of MACT-Net exhibit noticeable variations across different datasets, reflecting the influence of data scale and scene complexity.

On the SIRST dataset, MACT-Net achieves the best performance across all evaluation metrics, indicating its strong capability in both detection accuracy and pixel-level localization. Despite the relatively small dataset size and limited scene diversity, the proposed multi-level aggregation encoder–decoder architecture effectively preserves fine-grained structural information through convolution-enhanced skip connections, while the CNN–Transformer hybrid modeling module ensures sufficient global contextual modeling. As a result, MACT-Net maintains stable and superior performance even under data-constrained conditions.

We further analyze the comparatively lower $P_d$ of MACT-Net on IRSTD-1K (88.44%) and examine whether it stems from the vector quantization (VQ) bottleneck over-suppressing weak targets. The ablation in Table 4 shows that this is not the cause: removing the VQ module (NVQ) does not recover detection probability, since its $P_d$ (87.93%) is in fact marginally lower than that of the full model (88.44%), while both $F_a$ and IoU degrade ($F_a$ 6.92 vs. 6.07, IoU 65.92 vs. 67.90). The lower $P_d$ is therefore a property of the overall architecture on this particular dataset rather than a side effect of quantization. Among the three benchmarks, IRSTD-1K contains the faintest targets and the most heterogeneous, cluttered backgrounds, so its weakest targets are barely separable from the surrounding clutter. MACT-Net operates at a conservative, low-false-alarm point: its compact, low-redundancy representation strongly suppresses background-induced responses, yielding a false-alarm rate of only 6.07. On this most challenging dataset, the same conservatism causes a small number of the faintest, background-like targets to be suppressed together with the clutter, which is the direct reason for the lower $P_d$. Competing methods reach a higher $P_d$ (MSHNet 93.88, Dim2Clear 93.75) precisely because they are less conservative, but at roughly two to three times the false-alarm rate ($F_a$ 15.03 and 20.93, respectively, versus 6.07). MACT-Net thus trades a modest amount of detection probability for a substantially lower false-alarm rate and a competitive IoU (67.90, second only to SCTransNet’s 68.15), which is advantageous when false alarms are costly.

For the NUDT-SIRST dataset, MACT-Net achieves the best performance in $P_d$ and the second-best result in $IoU$. This indicates that the proposed method is particularly effective in maintaining high detection reliability while simultaneously improving localization accuracy. The hierarchical feature aggregation and cross-level interaction mechanisms allow MACT-Net to capture discriminative target representations without sacrificing detection sensitivity.

These results collectively suggest that MACT-Net is able to adapt its feature modeling strengths to different dataset characteristics, effectively addressing challenges related to background complexity, scale variation, and precise target localization.

As shown in Figure 3, the ROC curves on the IRSTD-1K, SIRST, and NUDT-SIRST datasets demonstrate that MACT-Net consistently achieves superior and stable performance across different infrared small-target detection scenarios. In particular, MACT-Net exhibits a steeper rise in the low false-positive-rate region, indicating stronger target detection capability under strict false-alarm constraints. Moreover, its ROC curves present smoother and more stable behavior in the medium-to-high true positive rate range, reflecting a better trade-off between target enhancement and background suppression. These improvements are attributable not merely to increased network depth or complexity but mainly to the multi-level aggregation encoder–decoder architecture, the collaborative CNN–Transformer-based local–global feature modeling, and the vector quantization-based feature discretization at the network bottleneck. Together, these components enhance cross-scale target representation and improve feature separability between targets and background, leading to more robust detection performance across different datasets.

(2) Qualitative Evaluation: The qualitative results on the datasets are illustrated in Figure 4 and Figure 5. As observed, the proposed MACT-Net consistently demonstrates more precise pixel-level localization and effectively suppresses false alarms, even under challenging conditions with low SNR and strong background interference.

Compared with other data-driven methods, the proposed MACT-Net demonstrates greater robustness to variations in scene complexity and target characteristics, yielding superior detection outcomes. As illustrated in Figure 5, MACT-Net provides accurate pixel-level localization and clear delineation of small infrared targets, even in challenging conditions such as low SNR, strong background clutter, and targets with diverse sizes and shapes.

This enhanced performance can be attributed to the network’s architectural innovations. The multi-level aggregation encoder–decoder structure enables progressive abstraction and reconstruction of features across different semantic levels, preserving local structural details while integrating multi-scale contextual information. The CNN–Transformer hybrid feature modeling module further allows the network to jointly capture fine-grained spatial patterns and long-range dependencies, improving discrimination between targets and complex backgrounds. Finally, the vector quantization-based feature discretization and representation enhancement mechanism compresses redundant information and emphasizes salient features at the network bottleneck, ensuring that the extracted representations are compact, discriminative, and robust.

Together, these innovations allow MACT-Net to effectively extract and integrate intrinsic target characteristics, resulting in precise pixel-level segmentation and reduced false alarms, as clearly observed in the qualitative results.

4.3. Overfitting and Generalization Analysis

We examine dataset by dataset whether training for 5000 epochs induces overfitting. Figure 6 reports the training and test loss (top row) together with the test IoU (bottom row) for NUDT-SIRST, IRSTD-1K, and SIRST.

On NUDT-SIRST (Figure 6a,d), the training and test losses decrease monotonically throughout training, with the test loss closely following the training loss; the test IoU rises steadily and then stabilizes on a plateau without any decline, showing that the model keeps generalizing as training proceeds.

On IRSTD-1K (Figure 6b,e), after a rapid initial drop, the test loss plateaus while the training loss continues to decrease slowly, so the train–test gap widens slightly in the later epochs. This indicates that the test performance has saturated while the model fits additional training-specific detail; it is, however, not harmful, since the test loss does not rise and the test IoU remains on a stable plateau without degradation. Because the final model is taken at the highest test IoU, this late-stage gap does not affect the reported performance.

On the small SIRST dataset (Figure 6c,f), which is the most susceptible to overfitting, the training and test losses stay close and overlap throughout training within a similar range, and the test IoU stabilizes on a plateau with only minor fluctuations and no downward trend. The absence of a widening train–test gap on the smallest dataset directly indicates that the model converges rather than memorizing the limited training samples.

In summary, on all three datasets, the test IoU rises and then remains on a stable plateau without subsequent degradation; even on IRSTD-1K, where the train–test loss gap widens slightly, the test IoU does not decline. Together with the best-test-IoU model-selection strategy, these results confirm that the 5000-epoch schedule does not harm generalization.

The curves above show that MACT-Net does not overfit within each dataset. To further verify that its strong performance, especially on the small SIRST dataset, reflects genuine generalization rather than dataset-specific memorization, we conduct a cross-dataset evaluation in which each model is trained on one dataset and directly tested on the others without any fine-tuning. The results, together with two representative baselines (DNA-Net and RPCANet), are reported in

Table 2. Cross-Dataset Generalization. Each model is trained on the source dataset and directly evaluated on the target dataset without fine-tuning.

Source → Target	Method	${\mathit{P}}_{\mathit{d}}$ ($\times {10}^2$)	${\mathit{F}}_{\mathit{a}}$ ($\times {10}^6$)	IoU ($\times {10}^2$)
SIRST → IRSTD-1K	DNA-Net [13]	85.71	125.94	47.70
	RPCANet [43]	79.93	37.96	49.11
	MACT-Net (Ours)	82.31	36.14	57.04
SIRST → NUDT-SIRST	DNA-Net [13]	82.01	108.03	49.78
	RPCANet [43]	72.38	224.76	42.20
	MACT-Net (Ours)	77.46	42.84	50.48
NUDT-SIRST → SIRST	DNA-Net [13]	94.49	16.15	63.88
	RPCANet [43]	67.89	81.62	31.12
	MACT-Net (Ours)	92.66	7.81	58.64
NUDT-SIRST → IRSTD-1K	DNA-Net [13]	85.37	126.78	38.07
	RPCANet [43]	74.83	166.86	27.68
	MACT-Net (Ours)	83.33	45.47	45.76
IRSTD-1K → SIRST	DNA-Net [13]	93.57	102.91	55.53
	RPCANet [43]	83.49	368.52	34.63
	MACT-Net (Ours)	93.58	39.57	61.12
IRSTD-1K → NUDT-SIRST	DNA-Net [13]	74.81	647.33	24.40
	RPCANet [43]	62.75	81.28	12.44
	MACT-Net (Ours)	74.07	30.68	39.34

.

As expected, the absolute scores drop under domain shift; nevertheless, MACT-Net generalizes consistently better than the baselines. In pixel-level accuracy, it attains the highest IoU in five of the six cross-dataset settings, often by a clear margin—for instance, 57.04 versus 47.70/49.11 on SIRST → IRSTD-1K and 39.34 versus 24.40/12.44 on IRSTD-1K → NUDT-SIRST. In false-alarm control, it achieves the lowest $F_a$ in all six settings, frequently by a large margin (e.g., 30.68 versus 647.33/81.28 on IRSTD-1K → NUDT-SIRST), showing that its strong background suppression carries over to unseen domains. Its detection probability, although usually second to DNA-Net, stays very close to the best baseline in every setting (e.g., 92.66 versus 94.49 on NUDT-SIRST → SIRST).

These cross-dataset results are most informative for the small SIRST dataset, which can be examined from two directions. First, when SIRST is used as the training source, the SIRST-trained model transfers well to the other datasets, achieving the best IoU and the lowest $F_a$ on both SIRST → IRSTD-1K and SIRST → NUDT-SIRST; this indicates that the model learns transferable features rather than memorizing SIRST. Second, when SIRST is used as the test target, models trained on the other datasets still detect targets well on SIRST—on IRSTD-1K → SIRST, for example, MACT-Net reaches the best IoU (61.12), the best $P_d$ (93.58), and the lowest $F_a$ (39.57)—showing that strong SIRST performance is attainable from features learned elsewhere and does not require dataset-specific fitting. Together, these two observations confirm that the performance of MACT-Net on the small SIRST dataset stems from genuine generalization rather than overfitting. Overall, the stable training dynamics together with the cross-dataset transferability demonstrate that MACT-Net generalizes well rather than overfitting to a specific dataset.

4.4. Complexity Analysis

To evaluate the practical cost of MACT-Net, we compare it with representative state-of-the-art methods in terms of the number of parameters, floating-point operations (FLOPs), inference speed (frames per second, FPS), and detection accuracy (IoU on SIRST). For a fair comparison, all metrics are measured at an input resolution of 256 × 256. The results are reported in

Table 3. Comparison of computational complexity (parameters, FLOPs, and inference speed) and detection accuracy (IoU on SIRST) between MACT-Net and representative state-of-the-art methods. All metrics are measured at a 256 × 256 input resolution.

Method	Params (M)	FLOPs (G)	FPS	IoU on SIRST ($\times {10}^2$)
DNA-Net [13]	4.70	28.16	19.90	76.24
SCTransNet [20]	11.33	21.12	50.69	61.43
AGPCNet [9]	12.36	86.22	39.13	–
UIUNet [14]	50.54	108.92	69.56	73.36
MSHNet [41]	4.07	12.10	58.96	72.36
L2SKNet [42]	0.90	14.39	181.92	59.44
RPCANet [43]	0.68	88.73	95.22	70.37
MACT-Net (Ours)	69.44	225.77	28.43	77.66

.

As Table 3 shows, MACT-Net has the highest parameter count (69.44 M) and FLOPs (225.77 G) among all compared methods, a direct consequence of its multi-stage aggregation, CNN–Transformer hybrid modules, and vector quantization codebook. This additional capacity, however, is not mere parameter inflation. On SIRST, MACT-Net achieves the highest IoU (77.66), slightly ahead of the next-best method in this complexity comparison (DNA-Net, 76.24); more importantly, it exhibits strong cross-dataset generalization (Section 3.3), attaining the best IoU in five of the six transfer settings and the lowest false-alarm rate in all six. The added cost therefore translates into accuracy and robustness rather than dataset-specific fitting.

In terms of speed, MACT-Net runs at 28.43 FPS, which supports near-real-time inference. Although it is the second slowest method in the table, it is in fact faster than DNA-Net (19.90 FPS) despite having far more FLOPs, indicating that its measured latency is lower than its FLOPs alone would suggest. Compared with lightweight detectors such as L2SKNet (181.92 FPS) and MSHNet (58.96 FPS), MACT-Net is considerably heavier and slower, so it is best suited to accuracy-critical scenarios with adequate computational resources, whereas lightweight detectors remain preferable for strictly resource-constrained or embedded deployment. The present design prioritizes detection accuracy and robustness over inference speed rather than lightweight or real-time deployment, which we leave for future work.

4.5. Ablation Study

To validate the contributions of each key module in MACT-Net, we conducted a comprehensive ablation study on three public infrared small-target datasets. The results are summarized in

Table 4. Ablation study of the multi-level aggregation encoder–decoder, the CNN–Transformer hybrid feature modeling module, and the vector quantization mechanism on SIRST, IRSTD-1K, and NUDT-SIRST.

Ablation Study	SIRST			IRSTD-1K			NUDT-SIRST
	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{P}}_{\mathit{d}}$ ($\times {\mathbf{10}}^{\mathbf{2}}$)	${\mathit{F}}_{\mathit{a}}$ ($\times {\mathbf{10}}^{\mathbf{6}}$)	IoU ($\times {\mathbf{10}}^{\mathbf{2}}$)
PU-Net	98.65	3.70	71.53	87.53	7.52	63.76	97.56	7.18	86.97
NMSEncDec	97.27	3.95	68.32	86.25	7.79	61.31	96.39	7.26	83.89
CNN-Only	97.93	4.80	70.65	87.62	8.89	62.68	97.19	8.35	85.68
Trans-Only	95.66	4.10	69.19	83.54	8.21	61.89	94.56	7.69	84.52
NVQ	98.21	3.13	73.50	87.93	6.92	65.92	98.25	6.92	90.56
MACT-Net (Ours)	99.12	2.48	77.66	88.44	6.07	67.90	98.94	6.14	91.73

.

(1) Multi-Level Aggregation Encoder–Decoder Architecture: The multi-level aggregation encoder–decoder architecture progressively abstracts and reconstructs features across multiple semantic levels. Convolution-enhanced skip connections perform same-level feature fusion, preserving local structural details while enabling effective multi-scale reconstruction. To evaluate the contribution of this architecture, we designed two network variants:

Plain U-Net (PU-Net): The convolution-enhanced skip connections are removed, and standard U-shaped encoder–decoder skip connections are used, while retaining multi-scale fusion at the decoder output.

No Multi-Scale Fusion (NMSEncDec): The multi-scale fusion at the decoder output is removed, and only the final decoder output is directly used.

As shown in Table 4, both variants exhibit decreased IoU and slightly higher false alarm rates compared with MACT-Net, especially NMSEncDec, where the absence of multi-scale fusion significantly weakens feature integration across scales. PU-Net maintains relatively high $P_d$ but shows reduced localization precision, highlighting the importance of skip connection enhancement and multi-scale aggregation.

(2) CNN–Transformer Hybrid Feature Modeling Module: The hybrid feature modeling module combines convolutional local feature extraction with Transformer-based global feature modeling. This collaboration enables adaptive learning of both fine-grained spatial details and long-range dependencies. To assess the effectiveness of this module, we designed two variants:

CNN-only (CNN-Only): The Transformer branch is removed, leaving only the convolutional branch for feature extraction.

Transformer-only (Trans-Only): The convolutional branch is removed, using only the Transformer for global feature modeling.

Both variants show noticeable performance drops. CNN-Only suffers from higher false alarms due to the lack of global contextual guidance, while Trans-Only shows reduced $P_d$ because of insufficient local feature representation. The results confirm that collaborative modeling of local and global features is essential for suppressing background interference and improving detection accuracy.

(3) Vector Quantization-Based Feature Discretization and Representation Enhancement: The vector quantization (VQ) module at the network bottleneck compresses redundant features into a discrete representation, emphasizing salient target characteristics and enhancing feature robustness. We designed one variant to test its effectiveness:

No VQ (NVQ): The vector quantization module is removed; the bottleneck features remain continuous.

Without VQ, both IoU decreases and false alarms increase slightly across datasets, indicating reduced feature compactness and discriminability. This comparison demonstrates that vector quantization effectively enhances the network’s robustness, particularly under low SNR conditions.

Overall, the ablation results show that each module positively contributes to MACT-Net’s performance. Removing any of them degrades all three metrics, lowering $P_d$ and IoU while increasing $F_a$. Therefore, all components—the multi-level aggregation encoder–decoder, CNN–Transformer hybrid feature modeling, and vector quantization—are indispensable and synergistically improve detection accuracy and robustness for infrared small targets.

5. Conclusions

This work proposes a novel infrared small-target detection network, MACT-Net. First, we introduce a multi-level aggregation encoder–decoder architecture with convolution-enhanced skip connections and multi-scale decoder fusion, enabling progressive abstraction and reconstruction of multi-level features. This design preserves richer local details, enhances spatial structural consistency, and improves the model’s multi-scale response capability for dim and small infrared targets.

Second, we design a CNN–Transformer hybrid feature modeling module, which adaptively captures and integrates local spatial structures and global semantic dependencies. By dynamically establishing associations between fine-grained local features and global contextual information, this module significantly enhances feature representation, discriminative capability, and detection robustness in complex backgrounds.

Third, we incorporate a vector quantization-based feature discretization and representation enhancement mechanism at the network bottleneck. By compressing redundant information and emphasizing salient features, it produces more compact, discriminative, and interpretable feature representations, thereby improving stable detection performance under low signal-to-noise ratio conditions.

The effectiveness of MACT-Net is demonstrated on three public datasets: SIRST, IRSTD-1K, and NUDT-SIRST. On the small SIRST dataset, it attains the best results across all metrics, demonstrating its ability to fully exploit limited training samples and preserve target details; cross-dataset evaluation further shows that it generalizes better than representative baselines, confirming that its strong performance reflects genuine generalization rather than overfitting. Overall, these results confirm that the proposed multi-level aggregation, CNN–Transformer hybrid modeling, and vector quantization mechanisms jointly contribute to high-precision and robust infrared small target detection.

Despite its strong accuracy and generalization, MACT-Net has several limitations that point to directions for future work. First, on the most cluttered benchmark, IRSTD-1K, its detection probability is relatively lower. As shown by the ablation study (Table 4), this is not caused by the vector quantization bottleneck—removing it does not recover the detection probability—but instead results from the network operating at a conservative, low-false-alarm point, at which a small number of the faintest, background-like targets are suppressed together with the surrounding clutter. Improving the recall of these extremely weak targets, for instance by integrating frequency-domain information, is a promising direction for further enhancing performance under complex backgrounds. Second, MACT-Net has the highest parameter count and FLOPs among the compared methods, so although it achieves near-real-time speed on a high-end GPU, it is not yet suitable for strictly resource-constrained or embedded deployment; developing lightweight and real-time variants through model compression, pruning, or knowledge distillation is therefore an important future direction. Third, the performance of the vector quantization module depends on the codebook size, which is currently selected empirically, and an adaptive codebook design could further improve its flexibility.