Dual-Domain Prior-Driven Deep Network for Infrared Small-Target Detection

Abstract

In recent years, data-driven deep networks have demonstrated remarkable detection performance for infrared small targets. However, continuously increasing the depth of neural networks to enhance performance has proven impractical. Consequently, the integration of prior physical knowledge related to infrared small targets within deep neural networks has become crucial. It aims to improve the models’ awareness of inherent physical characteristics. In this paper, we propose a novel dual-domain prior-driven deep network (DPDNet) for infrared small-target detection. Our method integrates the advantages of both data-driven and model-driven methods by leveraging the prior physical characteristics as the driving force. Initially, we utilize the sparse characteristics of infrared small targets to boost their saliency at the input level of the network. Subsequently, a high-frequency feature extraction module, seamlessly integrated into the network’s backbone, is employed to excavate feature information. DPDNet simultaneously emphasizes the prior sparse characteristics of infrared small targets in the spatial domain and their prior high-frequency characteristics in the frequency domain. Compared with previous CNN-based methods, our method achieves superior performance while utilizing fewer convolutional layers. It has a performance of 78.64% IoU, 95.56 Pd, and 2.15 × 10⁻⁶ Fa on the SIRST dataset.

1. Introduction

Infrared detection systems offer numerous advantages, including strong concealment, a wide detection range, and resistance to environmental interference. These systems find widespread applications on mobile platforms such as satellites, airplanes, and drones. Due to the long imaging distance, the target in the image plane only occupies a few or dozens of pixels, which leads to the lack of intrinsic features (such as the color, shape, and texture) [1]. Furthermore, the presence of complex backgrounds introduces significant interference, which causes the precise detection of the small infrared targets to become difficult [2].

After years of development, the field of infrared small-target detection has made significant progress. On the one hand, scholars’ understanding of specific domain knowledge has been deepening continuously. It began with early filtering-based methods [3,4] that simply modeled infrared small targets as protrusions in smooth backgrounds. Subsequently, more advanced methods [5,6] began to take into account the image edge structures and the targets with high local contrast. In recent times, some methods [7,8] have been achieved based on low-rank sparse recovery. The performance of model-driven detection methods has gradually improved.

On the other hand, with the rapid development of deep neural networks in the field of computer vision, scholars have transferred deep network models that were originally developed for visible-light image object detection to infrared image processing tasks. Liu et al. [9] first proposed a CNN-based method. They designed a multilayer perceptron (MLP) network with five layers for infrared small-target detection. Hou et al. [10] treated small targets in infrared images as a special type of sharp noise and established a mapping network between feature maps and small-target likelihood in the image. They then applied a threshold to the likelihood map to segment the real targets. Dai et al. [11] proposed an attentional local contrast network (ALCNet) that fused detailed features with deep semantic features to enhance the effect of infrared small-target detection. Deep learning offers the capability to construct end-to-end systems that autonomously extract multilevel features while learning the target detection task. This breakthrough overcomes the restrictions of manually engineered features and classifiers, leading to substantial advancements in detection performance.

However, both model-driven and data-driven approaches still suffer from pressing challenges:

• Purely model-driven methods face challenges in achieving precise modeling because of their heavy dependence on scholars’ expertise and experience. These methods are typically simplified forms of real data, which limits their ability to address complex real-world scenarios. Consequently, they encounter difficulties in terms of their detection performance and robustness.
• While current data-driven methods have demonstrated favorable results, achieving precise detection of infrared small targets is still a difficult task. The main challenges arise from the limited proportion of pixels occupied by targets, the serious imbalance between the foreground and background, and the weak semantic connections between the targets and the environment [12]. Since data-driven approaches fail to leverage domain-specific knowledge effectively, simply deepening the neural network has shown minimal impact on improving detection performance.
• Most deep networks for infrared small-target detection rely on supervised learning, which utilizes labeled data. However, the simulation of infrared data is not perfect, and measured data lack samples in areas of infrared target detection and recognition [13]. Even if a batch of high-quality labeled samples is amassed, the training models are still sensitive to the variations of backgrounds, potentially leading to poor generalization performance on new datasets [14].

To tackle the aforementioned challenges, we propose a novel dual-domain prior-driven deep network (DPDNet) for infrared small-target detection, which integrates advantages of both the model-driven and data-driven methods. We embed reliable physical priors into different network levels by leveraging the sparsity and the high-frequency characteristics in the spatial and frequency domains, respectively. This integration enriches the limited annotated data, reduces the network complexity, and leads to better feature representations. Therefore, the proposed method improves the detection performance for infrared small targets. Additionally, the interpretability and reliability of the CNN are also enhanced through this augmentation.

Our proposed method focuses on existing domain knowledge to guide the model at both the input and the inner levels of the network. Given that the development of recovering the low-rank and sparse matrices is well established, the infrared patch-image model [15,16,17] based on unsupervised learning offers a reliable theoretical basis for the sparse-characteristic-driven module. In addition, targets in infrared imagery exhibit higher radiation intensity than the background, which causes distinct high-frequency characteristics in the frequency domain. To capture the specific high-frequency characteristics of small targets, a hypercomplex infrared Fourier transform approach is used as the high-frequency-characteristic-driven module. The above two characteristics enable the network to exploit more feature information of infrared small targets.

We conducted some visible analysis to illustrate the impact on embedding physical characteristics in different network levels. A 3D visualization result was used to analyze the effect on the input data after introducing sparse characteristics. Additionally, we employed class activation mapping to visualize the inner features of the network, which can be used to analyze the relationship between the activated features and the high-frequency-characteristic-driven module.

The contributions are summarized as follows:

• We propose a novel dual-domain prior-driven deep network for infrared small-target detection, which integrates the data-driven methods with the model-driven approaches.
• We guide supervised data-driven models by proposing prior-driven modules that embed domain knowledge at both the input and the inner levels of the network.
• We analyze the effectiveness and reliability of the prior-driven modules in guiding learning and enhancing the expression capability of target features.

The rest of this research is organized as follows: In Section 2, we introduce the background about the interpretable physical information and the prior domain knowledge used for our infrared small-target detection. Section 3 introduces each module and the specific structure of the DPD network in detail. In Section 4, we validate the detection performance of our method and compare it with other state-of-the-art methods. In addition, an ablation experiment is presented to validate the effectiveness of each module in the proposed network. Section 5 presents a visualization analysis of the impact on different network levels after incorporating physical priors. Finally, Section 6 concludes the paper.

2. Background

2.1. Informed Machine Learning

The complexity of contemporary machine learning models poses challenges for human comprehension of the precise decision-making process during inference [18]. By harnessing the domain knowledge accumulated by scholars, it is possible to strengthen models’ interpretability and robustness while reducing data requirements [19]. The fusion of domain knowledge and data-driven approaches is mainly manifested by knowledge embedding, which constructs deep network models with physical prior knowledge. On the one hand, embedding domain knowledge into the models allows the powerful fitting capability to describe complex mapping relationships among high-dimensional variables, thereby improving the model’s accuracy. On the other hand, incorporating prior knowledge ensures that the predicted results correspond to fundamental physical mechanisms and common sense.

2.2. Sparse Characteristic of Infrared Small Targets

An infrared image $f_F$ can be represented as a linear combination of background image $f_B$, target image $f_T$, and noise image $f_N$:

$$f_F=f_B+f_T+f_N$$

(1)

The input image can be represented as a block tensor $F$ by using a sliding window with a certain step size to traverse the image from left to right and from top to bottom. Stack the obtained image blocks into a three-dimensional cube; then, Equation $\left(1\right)$ can be transformed into

$$\mathcal{F}=\mathcal{B}+\mathcal{T}+\mathcal{N}$$

(2)

where $\mathcal{B},\mathcal{T},\mathcal{N}\in R^{I\times J\times P}$ denote the background block tensor, target block tensor, and noise block tensor, respectively [7]. $I$ and $J$ are the patch height and width, respectively, and $P$ is the patch number.

Since the target occupies a small proportion of the entire infrared image, it can be regarded as a sparse matrix. In contrast, the background pixels display correlation, even if they cross distant positions in the image, which is denoted as nonlocal self-correlation. According to this characteristic, we can consider the background image as a low-rank matrix. Therefore, target detection can be transformed into an optimization problem of the low-rank and sparse matrices’ recovery.

2.3. High-Frequency Characteristics of Infrared Small Targets

The high-intensity saliency of the specific frequency domain in infrared small targets makes the use of a dual-domain feature extraction method highly effective. This approach aims to capture the high-frequency features present in the image’s frequency domain. These features are fused with spatial-domain features to form a comprehensive representation that combines spatial and frequency characteristics.

In the development of infrared image feature extraction, traditional methods [20,21,22] mainly rely on techniques such as histogram equalization, filtering, and gray-level stretching. Nevertheless, these methods have limitations of concurrently incorporating both spatial- and frequency-domain information, which restricts the effectiveness of their practical application. With the application of wavelet transform and local contrast enhancement [16,23,24], it becomes feasible to effectively extract spatial- and frequency-domain information from infrared images. In addition, the recent development of deep learning techniques has brought new ideas for dual-domain feature extraction from infrared images. Utilizing deep learning models [25,26,27] enables the automatic extraction of dual-domain information from infrared images.

3. Methods

The workflow of the DPDNet is shown in Figure 1. At the input level, we utilize the sparse characteristics in the spatial domain as the driving module. This module utilizes the preliminary targets and background imagery recovered from the sparse and low-rank matrices to enhance the original image $x$, resulting in a more salient target in $\overline{x}$. Considering the inherent temporal physical characteristics present in infrared images, we incorporate the high-frequency physical characteristics of infrared images as the driving module at the inner level of the network. Repeating the extraction and fusion of dual-domain features makes more efficient use of frequency-domain physical characteristics. Inspired by Dense Nested Net [28], we embed the high-frequency-characteristic-driven module into a multilayer complex dense network to construct the primary detection module. The skip connections are specifically utilized to fuse low-level and high-level features in the detection module.

3.1. Sparse-Characteristic-Driven Module

To incorporate prior spatial-domain physical knowledge into the training data, the sparse-characteristic-driven module operates on the neural network before training. Background image blocks in infrared images mostly come from one or more low-rank subspaces. In addition, due to the lack of nonlocal self-similarity, small targets constitute sparse components. Through the decomposition of the infrared image into the low-rank and sparse matrices, it becomes possible to distinguish background information from target information. The target features of the original image $x$ can be enhanced and represented as follows:

$${\overline{x}}_{background}=fold\left(x\right)<T_{down}$$

(3)

$${\overline{x}}_{target}=fold\left(x\right)>T_{up}$$

(4)

$$\overline{x}=w_{0}x\oplus \left(w_1{\overline{x}}_{background}+w_2{\overline{x}}_{target}\right)$$

(5)

where $fold\left(x\right)$ denotes the low-rank and sparse decomposition applied to the original image, ${\overline{x}}_{background}$ and ${\overline{x}}_{target}$ are obtained after adaptive dual-threshold segmentation ($T_{down}$ and $T_{up}$ are thresholds), $+$ is the pixel-level image fusion, and $\oplus$ is the channel-level image fusion.

The data processed by the sparse-characteristic-driven module show richer target information, which improves the feature quality in the subsequent feature extraction processes. Figure 2 shows the resulting images $\overline{x}$ after the sparse-characteristic-driven module. The targets in $\overline{x}$ are more discernible to the naked eye.

3.2. High-Frequency Characteristic Extraction Module

Inspired by the successful application of the hypercomplex Fourier transform (HFT) in RGB color image processing tasks [29], we propose a hypercomplex infrared Fourier transform (HIFT) to extract high-frequency characteristics. The HIFT is a development based on the Fourier transform. The HIFT extends the real-valued signals to the hypercomplex signals and applies them to infrared images to extract features in the frequency domain. The hypercomplex input is specified to be a quaternion formula, which can be regarded as an extension of the concept of complex numbers. The complex quaternion tuple that represents the hypercomplex signal is denoted as $f\left(m,n\right)$:

$$f\left(m,n\right)={\omega }_0{f}_0+{\omega }_1{f}_{1}i+{\omega }_2{f}_{2}j+{\omega }_3{f}_{3}k$$

(6)

where $m$ and $n$ are discrete spatial variables, $i,j,k$ are imaginary units, ${\omega }_0,{\omega }_1,{\omega }_2,{\omega }_3$ are weights, $f_0,f_1,f_2,f_3$ are feature matrices, and $f_0$ is the motion feature. Hence, $f_0=0$ for single-frame static input images. $f_1$ is the gray value of the image:

$$f_1=Gray\left(x, y\right)$$

(7)

$f_2$ is the infrared radiation [30]:

$$f_2={\Phi }_{min}+\frac{\left(g/255-r\right)\times \left({\Phi }_{max}-{\Phi }_{min}\right)}{1-r}$$

(8)

where $g$ denotes the gray level of each pixel of an image; $r\in \left[0,1\right]$ is a constant, which typically depends on the parameters of the environment and the infrared sensor. We replace $r$ with the normalized grayscale image value. The maximum and minimum radiation intensities of the target are denoted as ${\varphi }_{max}$ and ${\varphi }_{min}$, respectively, and $f_3$ is the local thermal variation of the image, which can be calculated as follows:

$$f_3=\frac{L_{in}\left(i,j\right)}{n_{in}}-\frac{L_{out}\left(i,j\right)}{n_{out}}$$

(9)

where

$$L_{in}\left(i,j\right)={\displaystyle \sum }_{\left(k,l\right)\in N_{in}\left(i,j\right)}\left|g\left(k,l\right)-{\mu }_{in}\left(i,j\right)\right|$$

(10)

$$L_{out}\left(i,j\right)={\displaystyle \sum }_{\left(k,l\right)\in N_{out}\left(i,j\right)}\left|g\left(k,l\right)-{\mu }_{out}\left(i,j\right)\right|$$

(11)

$${\mu }_{in}\left(i,j\right)=\frac{1}{n_{in}}{\displaystyle \sum }_{\left(k,l\right)\in N_{in}\left(i,j\right)}g\left(k,l\right)$$

(12)

$${\mu }_{out}\left(i,j\right)=\frac{1}{n_{out}}{\displaystyle \sum }_{\left(k,l\right)\in N_{out}\left(i,j\right)}g\left(k,l\right)$$

(13)

where $L_{in}\left(i,j\right)$ is the rectangular neighborhood of pixel $i,j$, and its length and width are the largest length and width of the targets in the dataset. $L_{out}\left(i,j\right)$ is a larger rectangular neighborhood of pixel $i,j$ than $L_{in}\left(i,j\right)$, except for those pixels in $L_{in}\left(i,j\right)$. $g\left(k,l\right)$ is the intensity of the pixel in the neighborhood of the $k$-th row and the $l$-th column, and $\mu$ is the mean intensity of the neighborhood. $n_{in}$ and $n_{out}$ are the pixel numbers of $L_{in}\left(i,j\right)$ and $L_{out}\left(i,j\right)$, respectively.

After defining the feature matrices, the discrete HIFT formula is

$${\mathcal{F}}_{\mathcal{H}}\left[s,t\right]=\frac{1}{\sqrt{MN}}{\displaystyle \sum }_{m=0}^{M-1}{\displaystyle \sum }_{n=0}^{N-1}{e}^{\omega 2\pi \left(ms/M+nt/N\right)}f\left(m,n\right)$$

(14)

where $\omega$ is a unit pure quaternion, and ${\omega }^2=-1$. $M,N$ denote the height and width of the image, respectively.

The inverse transformation of HIFT is defined as follows:

$$f\left(m,n\right)=\frac{1}{\sqrt{MN}}{\displaystyle \sum }_{s=0}^{M-1}{\displaystyle \sum }_{t=0}^{N-1}{e}^{\omega 2\pi \left(ms/M+nt/N\right)}{\mathcal{F}}_{\mathcal{H}}\left[s,t\right]$$

(15)

The HIFT possesses the capability to capture both local frequency and phase information in infrared images. Therefore, it addresses the limitation of the traditional Fourier transform, which can only capture global frequency information. The HIFT enables a more accurate representation of local signal features and enhances the robustness of the network when dealing with nonstationary signals in infrared small-target detection tasks. The effectiveness of the high-frequency characteristic extraction module in processing infrared images is shown in Figure 3. This module is mainly applied in the detection module, which will be further introduced in Section 3.3.

3.3. Detection Module

The detection module consists of four main structures: an encoder, skipping connections, a high-frequency-characteristic-driven module, and a decoder, as shown in Figure 4.

Encoder: In the encoder part of the network, we build upon the traditional U-Net [31] architecture. This consists of multiple convolutional and pooling layers for feature extraction from the input image while progressively reducing the size of the feature map. Inspired by the Dense Nested Net [28] approach, we adopt a nested network structure consisting of two U-Net subnetworks with varying depths. This design enables the module to capture high-level features while preserving the representation of small targets within the deep network.

Skip connections: The skip connections are introduced to establish connections between the encoder and the decoder. These structures facilitate the fusion of high-level and low-level features, enlarging the receptive field of the feature map and improving the localization of small targets. Within the skip connection paths, we create multiple sub-nodes. All of these intermediate nodes exhibit dense connectivity, forming a nested-shaped network structure. As illustrated in Figure 5, each node has the capacity to receive features from surrounding nodes. In particular, nonadjacent nodes conduct skip connections in the same feature layer. Consequently, the representation of small targets is still preserved in the deeper neural network layers, yielding improved outcomes.

High-frequency-characteristic-driven module: In each feature extraction and fusion pathway, we apply a high-frequency-characteristic-driven module, as illustrated in Figure 6. This module consists of two parts (the frequency-domain part and the spatial-domain part). The high-frequency characteristic extraction module, as detailed in Section 3.2, is designed to extract frequency-domain features that propagate among different nodes.

$$f_T^{out}={\mathcal{F}}_{\mathcal{H}}\left(f^{in}\right)$$

(16)

Inspired by CBAM [32], the attention mechanism, which includes both the channel attention and the spatial attention, is used to extract features in the spatial domain. The channel attention dynamically adjusts the feature weights in the channel dimension, allowing the network to focus on more relevant and informative channels. The spatial attention dynamically adjusts the feature weights in the spatial dimension, enabling the network to focus on important regions within the feature maps.

$$f_S^{out}=\mathcal{A}\left(f^{in}\right)$$

(17)

In Figure 6, we integrate dual-domain features:

$$f_D=f_T^{out}\oplus f_S^{out}$$

(18)

where $f^{in}$ denotes the input feature, $f_T^{out}$ is the frequency-domain portion of the output feature, and $f_S^{out}$ is the spatial portion of the output feature. $\mathcal{A}\left(·\right)$ indicates the attention extraction of spatial features. The fused dual-domain features are denoted as $f_D$, and $\oplus$ denotes the channel-level image fusion.

This module enables the network to acquire inherent features in the dual domain and dynamically adjusts the feature weights of different convolutional layers based on the input data characteristics. Figure 7 illustrates the feature map of each sub-node when employing the high-frequency-characteristic-driven module. It can be seen that the features of small targets are fully preserved at both high and low levels. This validates the effectiveness of the module in capturing and retaining crucial information related to small targets.

Decoder: The decoder has the opposite structure to the encoder, which gradually increases the size of the feature maps by employing multiple upsampling operations. After each upsampling operation, the decoder also performs feature fusion with the corresponding high-frequency-characteristic-driven module to strengthen the feature information.

4. Results

4.1. Dataset

We used the SIRST dataset [33] to validate our network. The SIRST dataset is a single-frame infrared small-target dataset containing 427 infrared images from various scenes, with over 10% of the images containing multiple targets. Many of the infrared small targets in SIRST are extremely dim and hidden in complex backgrounds, with approximately 55% of the target areas in the dataset being less than 0.02%. Moreover, only 35% of the objects in the dataset contain pixels in the brightest part of the image. Representative images from the SIRST dataset are shown in Figure 8.

4.2. Implementation Details

We utilized the dataset described in Section 4.1 for training. All images of different sizes were first cropped to a resolution of 256 × 256 and normalized. The U-Net paradigm with ResNet [34] was chosen as our deep feature extraction backbone. The number of downsampling layers was chosen as four ($i=4$) in our network. Our network was trained using the Soft-IoU loss function and optimized with the Adam method [35]. CosineAnnealingLR was used as the learning rate decay strategy. We initialized the model’s weights and biases using the Xavier method [36]. The batch size was set to four, and the learning rate was set as 0.001. All experiments were implemented in PyTorch on a computer equipped with an NVidia GeForce 3090Ti GPU.

4.3. Evaluation Metrics

Pioneering CNN-based works [11,32,34] mainly use pixel-level evaluation metrics like IoU, precision, and recall values. However, the overall target localization is the most important criteria for SIRST detection [28]. To better evaluate the proposed network, we used the probability of detection and the false alarm rate to evaluate localization ability, and we used IoU to evaluate shape description ability.

Intersection over union (IoU): IoU is a pixel-level evaluation metric for detection accuracy, which is calculated by the ratio of the intersection area and the union area between the predicted values and the ground-truth labels. The calculation formula is expressed as follows:

$$IoU=\frac{{{\displaystyle \sum }}_i^{N}TP\left[i\right]}{{{\displaystyle \sum }}_i^N\left(T\left[i\right]+P\left[i\right]-TP\left[i\right]\right)}$$

(19)

where $N$ is the number of targets in the test set, $TP$ denotes the true positive prediction, $T$ denotes the small-target region in the ground-truth label, and $P$ denotes the small-target region predicted by the model.

The ROC curve is used to show the trade-off between the true positive rate and the false positive rate at various classification thresholds.

$$\mathrm{TPR}=\frac{TP}{TP+FN}$$

(20)

$$\mathrm{FPR}=\frac{FP}{FP+TN}$$

(21)

where $FN$ denotes the number of false negative predictions, $FP$ stands for the number of false positive predictions, and $TN$ represents the number of true negative predictions.

Probability of detection and false alarm rate (Pd and Fa): These are object-level evaluation metrics. Pd measures the ratio of correctly predicted targets to the total number of targets, while Fa measures the ratio of incorrectly predicted pixels to all pixels in the image. Pd and Fa are defined as follows:

$$P{d}_{}=\frac{T_{correct}}{T_{all}}$$

(22)

$$F{a}_{}=\frac{P_{false}}{P_{all}}$$

(23)

where $T_{correct}$ is the number of correctly detected targets, and $T_{all}$ is the number of all targets. $P_{false}$ is the number of false predicted pixels, and $P_{all}$ is the total number of image pixels. If the centroid derivation of the target is larger than a maximum allowed derivation, we consider those pixels as falsely predicted ones. We set the maximum centroid derivation as three in this paper.

4.4. Effect Verification

To validate the performance of the proposed method, we compared it with pure model-driven methods (Top-Hat [4], WSLCM [37], and IPI [3]) and previous CNN-based methods (MDvsFA-cGAN [38], ACM [33], ALCNet [11], and DNANet [28]). The results are shown in

Table 1. The comparison results of different methods in terms of IoU, Pd, and Fa metrics on the SIRST dataset. For IoU and Pd, larger values indicate higher performance. For Fa, smaller values indicate higher performance.

Method	$\mathit{I}\mathit{o}\mathit{U}\left(\times {10}^{-2}\right)$	$\mathit{P}{\mathit{d}}_{}\left(\times {10}^{-2}\right)$	$\mathit{F}{\mathit{a}}_{}\left(\times {10}^{-6}\right)$
		(Tr = 50%)
Filtering-Based: Top-Hat [4]	7.143	79.84	1012
Local-Contrast-Based: WSLCM [8]	1.158	77.95	5446
Local-Rank-Based: IPI [3]	25.67	85.55	11.47
CNN-Based: MDvsFA-cGAN [38]	60.3	89.35	56.35
CNN-Based: ACM [33]	70.33	93.91	3.728
CNN-Based: ALCNet [11]	73.33	96.57	30.47
CNN-Based: DNANet [28]	76.24	97.71	12.8
DPDNet (Ours)	78.64	95.56	2.15

.

For all compared traditional methods (Top-Hat [4], WSLCM [37], and IPI [3]), we first obtained prediction values and then suppressed noise by setting a threshold. The adaptive thresholds for traditional methods were calculated using Equation (24). For the CNN-based approaches (MDvsFA-cGAN [38], ACM [33], ALCNet [11], and DNANet [28]), we followed their original papers and employ their fixed thresholds.

$$T=max\left\{Max\left(P\right)\times 0.7,\sigma \left(P\right)\times 0.5+Avg\left(P\right)\right\}$$

(24)

where $Max\left(P\right)$ denotes the maximum value of the output, $\sigma \left(P\right)$ denotes the standard deviation of the output, and $Avg\left(P\right)$ denotes the average value of the output.

Compared with the traditional methods, our method achieved significant improvements. This is because the traditional methods are usually designed for specific scenarios (e.g., specific target size and cluttered background), and manually selected parameters limit their generalization performance.

Similarly, our method outperformed the previous best CNN-based method by 2.4% in the IoU, performed well in the evaluation of the Fa, and maintained a high level of the Pd. In addition, as shown in Figure 9, the proposed DPDNet achieved a higher true positive rate at the same false positive rate. This is because we embedded physical modules in the deep network based on physical prior knowledge. These physical modules guide the extraction of target features and reduce the network’s dependence on structural complexity.

Interestingly, it can be seen in

Table 2. The performance comparison between the previous SOTA data-driven methods with more convolutional layers and the DPDNet with fewer layers.

Method	$\mathit{I}\mathit{o}\mathit{U}\left(\times {10}^{-2}\right)$	$\mathit{P}{\mathit{d}}_{}\left(\times {10}^{-2}\right)$	$\mathit{F}{\mathit{a}}_{}\left(\times {10}^{-6}\right)$	#Params(M)
		(Tr = 50%)
DNANet-ResNet10 [28]	76.24	97.71	12.8	2.61
DNANet-ResNet18 [28]	77.47	98.48	2.35	4.7
DNANet-ResNet34 [28]	77.54	98.1	2.51	8.79
DPDNet-ResNet10 (Ours)	78.64	95.56	2.15	1.81

that increasing the depth of the feature extraction network by stacking convolutional blocks improved the detection performance to some extent. However, our method using a shallow network still achieved a higher value of the IoU than SOTA DNANet-ResNet34 [28]. This merit can be attributed to the combination of data-driven and model-driven approaches in our method, which enables the network to obtain more physical information about the target features during the feature extraction. This proves that embedding knowledge into the deep network can compensate for the limitations of reducing model complexity. Thanks to the utilization of a shallower neural network architecture, the proposed DPDNet has significantly fewer parameters compared with previous CNN-based methods.

4.5. Ablation Study

To demonstrate the effectiveness of each module in our proposed method, we conducted ablation studies. We used ResNet10 and Dense-Net as shared modules and evaluated the impacts on the sparse-characteristic-driven module and the high-frequency-characteristic-driven module separately, as shown in

Table 3. The verification results for the sparse-characteristic-driven module and the high-frequency-characteristic-driven module.

Shared Module	Sparse-Characteristic-Driven Module	High-Frequency-Characteristic-Driven Module	$\mathit{I}\mathit{o}\mathit{U}\left(\times {10}^{-2}\right)$	$\mathit{P}{\mathit{d}}_{}\left(\times {10}^{-2}\right)$	$\mathit{F}{\mathit{a}}_{}\left(\times {10}^{-6}\right)$
				(Tr = 50%)
ResNet10 and Dense-Net			96.24	97.71	12.8
	✓		78.18	94.44	5.09
		✓	78.53	95.19	6.23
	✓	✓	78.64	95.56	2.15

. The results show that the integration of both the sparse-characteristic-driven module and the high-frequency-characteristic-driven module led to a significant improvement in the IoU compared with the baseline. Moreover, the value of Fa suffered a remarkable decrease when simultaneously using the above two modules.

Table 4. The verification results of adding the high-frequency-characteristic-driven module in different feature layers.

Method Description	$\mathit{I}\mathit{o}\mathit{U}\left(\times {10}^{-2}\right)$	$\mathit{P}{\mathit{d}}_{}\left(\times {10}^{-2}\right)$	$\mathit{F}{\mathit{a}}_{}\left(\times {10}^{-6}\right)$	#Params(M)
		(Tr = 50%)
Layer 0	77.08	94.81	7.02	2.6
Layer 0, 1	78.04	94.81	4.8	2.5
Layer 0, 1, 2	78.2	96.3	0.72	2.27
Layer 0, 1, 2, 3	78.64	95.56	2.15	1.84

shows the effectiveness of integrating the high-frequency-characteristic-driven module in different feature layers. Figure 10 shows the definition of the feature layers. As more modules are added in deeper layers, the value of IoU gradually improves, while the value of Fa decreases significantly, and the value of Pd remains at a high level. In addition, gradually adding the high-frequency-characteristic-driven module can reduce the number of network parameters, effectively saving computational resources.

5. Discussion

In this section, we focus on the physical interpretability of DPDNet. First, the sparse-characteristic-driven module enhances the contrast between the targets and the background information, which helps the network to capture stronger physical features during training. Second, the dual-domain prior-driven module preserves the physical interpretability of targets in different feature layers. This promotes the acquisition of meaningful outcomes during the training process. According to above two aspects, our approach has higher physical interpretability and is more reliable than other deep neural networks.

5.1. Physics Explanation of $\overline{x}$

Detecting small targets in infrared images is a challenging task due to the unclear representation of the targets and the poor contrast between targets and the background [39]. Figure 11 shows the 3D visualization results of the original images and $\overline{x}$. It shows that the values of the target pixels in $\overline{x}$ are more prominent compared with the background information. Consequently, in the subsequent feature extraction processes, the target information in $\overline{x}$ is better preserved.

In Section 3.2, we described the equation of $\overline{x}$, where $w_0$, $w_1$, and $w_2$ are weight coefficients. Theoretically, increasing $w_2$ could make the target representation more prominent in $\overline{x}$. However, we found that setting $w_2$ too large led to overfitting of the network. Therefore, we chose $w_0=\frac{1}{2}, w_1=\frac{0.5}{4}, w_2=\frac{1.5}{4}$ to balance the target representation and the overfitting issues.

5.2. Physical Interpretability Discussion

In deep learning models, understanding the internal workings is a great challenge due to their high complexity and black-box nature. The activation map provides visual demonstrations of the regions in the feature maps that guide detection. The first row of Figure 12 shows that the feature maps propagate in the mesh nodes of DPDNet. The second row shows the corresponding activation maps of these nodes. It can be observed that the target regions are well activated and utilized during the feature propagation, which effectively explains the high performance on the values of IoU, Pd, and Fa achieved by our network.

6. Conclusions

In this paper, we propose a novel dual-domain prior-driven deep network for infrared small-target detection. DPDNet consists of three key modules: the sparse-characteristic-driven module, the high-frequency-characteristic-driven module, and the main detection module. The sparse-characteristic-driven module utilizes spatial-domain prior physical knowledge to guide the learning and decision-making process. The high-frequency-characteristic-driven module incorporates dual-domain knowledge into the feature extraction layer to enhance the representation of small targets. The main detection module efficiently exploits the intrinsic information of small targets by repeated fusion and enhancement. Our research demonstrates the significant impact on the detection performance and the interpretability of infrared small targets by embedding physical knowledge into deep learning. The experimental results on the SIRST dataset show that DPDNet achieves superior performance compared with previous SOTA CNN-based methods. Moreover, our proposed method is a lightweight network. By decreasing the depth of the convolutional layer, the network has only 1.84 million parameters.