Local Convergence Index-Based Infrared Small Target Detection against Complex Scenes

Abstract

Infrared small target detection (ISTD) plays a crucial role in precision guidance, anti-missile interception, and military early-warning systems. Existing approaches suffer from high false alarm rates and low detection rates when detecting dim and small targets in complex scenes. A robust scheme for automatically detecting infrared small targets is proposed to address this problem. First, a gradient weighting technique with high sensitivity was used for extracting target candidates. Second, a new collection of features based on local convergence index (LCI) filters with a strong representation of dim or arbitrarily shaped targets was extracted for each candidate. Finally, the collective set of features was inputted to a random undersampling boosting classifier (RUSBoost) to discriminate the real targets from false-alarm candidates. Extensive experiments on public datasets NUDT-SIRST and NUAA-SIRST showed that the proposed method achieved competitive performance with state-of-the-art (SOTA) algorithms. It is also important to note that the average processing time was as low as 0.07 s per frame with low time consumption, which is beneficial for practical applications.

1. Introduction

Infrared small target detection is one of the key technologies for infrared search and tracking systems and it is extensively used in a variety of applications such as aerospace, navigation, and early-warning systems [1,2].

In the context of small target detection in infrared images, a complex scene is a scene with a large number of objects and background features, such as buildings, clouds, shadows, noise, etc., that can make it difficult for algorithms to distinguish between the true targets and their surrounding environment and clutter. Additionally, a complex scene may involve multiple targets with varying sizes and intensities. Conversely, some datasets used in earlier research on infrared small target detection were considered as containing simple scenes due to their relatively clean backgrounds with minimal interference, as shown in Figure 1.

Due to the limitations of remote infrared sensing imaging, some useful features (e.g., texture features) are difficult to extract due to the small target size, complex background, clutter, and noise, which can lead to high false alarm rates and low detection rates of detection algorithms [3]. Therefore, IR small target detection algorithms that are function robustly with complex scenes are the focus of IR search-and-tracking research.

Generally, infrared small target detection methods can be classified into non-deep learning and deep learning methods.

Deep learning methods, especially convolutional neural networks (CNNs), can automatically learn high-level features from raw data, eliminating the need for feature engineering. Therefore, they are particularly suitable for classifying targets with complex color, texture, and shape features in visible light images. Deep learning has been introduced into the field of infrared small target detection, achieving promising results [4,5,6,7]. However, deep learning usually requires a large amount of annotated data and computational resources to achieve satisfactory performance.

Non-deep learning methods rely on prior knowledge. These methods [8,9,10,11,12,13] require significant expertise and effort in mathematical modeling or feature engineering, making it challenging to achieve optimal results. For the task of infrared small target detection, owing to the lack of color, texture, and shape features of small targets in infrared images and the fact that their shapes are all similar to a 2D Gaussian distribution, it is feasible to design handcrafted features for description.

Owing to the fact that there is a shortage of real data for small targets in practical applications, there are large-sized images in practical applications that make it difficult to train and apply deep learning; moreover, infrared target detection is a risk-sensitive task which requires the algorithm to be interpretable. We therefore propose a novel method for detecting infrared small targets using local convergence index (LCI) filters [14] and a random undersampling boosting classifier (RUSBoost) [15]. First, target candidates are extracted using a multi-scale, multi-directional, gradient weighting technique, which helps to detect dim, small targets with weak boundaries. Second, features are extracted for each candidate based on each candidate’s intensity, geometry, and LCI response (LCI filters are based on gradient convergence rather than intensity, which can effectively represent low-contrast objects in a densely cluttered background). Third, the feature set is fed into the boosting classifier to distinguish between real targets and false-alarm candidates. We determined that RUSBoost was an appropriate classifier, since our dataset was skewed, with a small number of true target candidates and a large number of false-alarm candidates.

The major contributions of this research can be summarized as follows:

• A novel detection scheme for infrared small targets based on an LCI filter is proposed, with a high detection rate and a low false alarm rate that outperforms SOTA techniques; the scheme also has low time consumption and is beneficial for practical applications against complex scenes.
• In the coarse detection stage of candidate regions, the sensitivity and accuracy of infrared small target detection are significantly improved by introducing a multi-scale and multi-directional gradient weighting strategy.
• To solve the imbalance problem between true targets and false-alarm sources for infrared small target detection, we used RUSBoost as a classifier, which combines undersampling and ensemble learning. For larger sample sizes, undersampling can fully reflect its advantages and improve the operational efficiency while balancing the dataset.

The remaining part of the work is structured as follows: Section 2 describes the presented methodology, Section 3 goes into detail about the experiments used to demonstrate the effectiveness of the approach, and Section 4 provides the conclusion.

2. Methodology

2.1. Related Works

Infrared small target detection algorithms can be broadly classified into data-driven deep neural networks or model-driven algorithms that do not learn. Typically, deep neural networks consider infrared small target detection as a semantic segmentation task. Li et al. [4] modified a U-shaped structured network for infrared small target detection using a densely nested interactive module and obtained good target detection results. Hou et al. [5] proposed a robust infrared small target detection network in which a handcrafted feature extraction structure was designed based on the local saliency of small infrared targets and embedded in a convolutional neural network. Zhang et al. [6] proposed an attention-guided pyramidal context network, which constructs a framework combining contextual pyramidal modules and attention mechanisms to integrate contextual information. Dai et al. [7] introduced a unique deep network for infrared small target detection that integrates labeled data and domain knowledge with discriminative networks and standard model-driven approaches. Although these methods perform better than model-driven algorithms, a higher detection rate depends on more and better-labeled samples. In addition, these methods have a low generalization capability due to various dense clutter in practical applications.

Non-learning, model-driven approaches depend heavily on prior knowledge in infrared scenes and mainly include background estimation-based, target saliency-based, and sparse representation-based approaches. Background estimation-based methods—including top-hat operations, median filtering, maximum filtering, and morphological reconstruction—may incorrectly detect edge clutter as targets in complex scenes. Target saliency-based approaches, such as the local contrast metric (LCM) [8] and the multi-scale patch-based contrast measure (MPCM) [9], have a reduced detection rate when the target is dim and small. For sparse representation-based algorithms, the detection task is modeled as a convex optimization problem by exploiting the low-rank nature of the background image and the sparseness of the target image. Gao et al. [10] proposed an infrared patch image (IPI) model that can separate small target images from background images by reconstructing low-rank and sparse matrices. Based on Gao’s study, the sparse prior information, and non-local autocorrelation prior information of the target, researchers developed the column-weighted IPI model [11], the nonnegative infrared patch image model (NIPPS) [12], and the tensor kernel parametric partial sum model (PSTNN) [13]. For conventional infrared images, these approaches provide reliable small-target detection capabilities. For infrared images with dim objects or complicated backgrounds, however, high-intensity clutter may readily compromise detection accuracy. In addition, the image decomposition and iterative optimization activities of the above methods are time-consuming and difficult to use for real-time target detection.

2.2. Overall Framework

To detect small targets in complex scenes, a fast algorithm with high sensitivity can be used initially to extract dim target candidates. Then, a multi-feature model can be built, and a classifier can be applied to further reduce false-alarm candidates. This reliable scheme has been used for small object detection in medical images, such as fundus lesion detection [16,17,18] and lung nodule detection [19]. Figure 2 shows the overall framework of the proposed algorithm, which includes candidate extraction using multi-directional and multi-scale Gaussian derivative kernels, multi-feature modeling using a local convergence index filter, and candidate classification by training a RUSBoost classifier.

2.3. Candidate Extraction

Since infrared small targets usually exhibit stronger intensity in the local context [20], edge features will exist even for dim targets. The edges of targets are very different from those of other targets in terms of background clutter: edges of clutter are usually disconnected or long, while edges of targets are short and encircling. Therefore, edge detection algorithms can be used to extract candidates of infrared small targets. Compared with other techniques, multi-directional and multi-scale Gaussian derivative kernels can easily remove background noise and enhance the gradient boundaries of small targets [21].

The directional derivatives of the Gaussian kernel in different directions were obtained as Gaussian derivative kernels for different scales and orientations, which were calculated using Equation (1):

$$GD(x,y,{\sigma }_G,{\theta }_G)=-\frac{G(x,y,{\sigma }_G)}{{\sigma }_G^2}(x\cos {\theta }_G+y\sin {\theta }_G),$$

(1)

where $G(x,y,{\sigma }_G)=(1/\sqrt{2\pi {\sigma }_G^2}{)\mathrm{e}}^{-(x^2+y^2)/2{\sigma }_G^2}$ is the Gaussian kernel and ${\sigma }_G\in \{1,2,3\}$ px is its corresponding standard deviation, which is used to regulate the scale of the kernel. ${\theta }_G\in \{0,\pi /6,\pi /3,\pi /2\}$ is the rotation angle.

Then, the gradient significance values for different scales and different orientations were obtained by the convolution of the original infrared image ($I_O$) with the rotated Gaussian derivative kernel:

$$\begin{array}{l}{I}_M(x,y,{\sigma }_G,{\theta }_G)=[(\omega ({\theta }_G)\ast GD(x,y,{\sigma }_G,{\theta }_G)\otimes {I_O(x,y))}^2+\\ (\omega ({\theta }_G+\pi /2)\ast GD(x,y,{\sigma }_G,{\theta }_G+\pi /2)\otimes {I_O(x,y))}^2{]}^{1/2}\end{array},$$

(2)

where $\otimes$ is the convolution operator, and $\omega ({\theta }_G)$ is the normalization factor of the Gaussian derivative kernel $GD(x,y,{\sigma }_G,{\theta }_G)$. Samples of gradient significance images for different scales and orientations are shown in Figure 3.

In $I_M$, a pixel with a smaller gradient magnitude means its corresponding area is smooth, and a pixel with a higher value is the edge of the image. First, we normalized $I_M$ to obtain $I_{NM}$. Then, the gradient-weighted image was obtained by:

$$I_W(x,y,{\sigma }_G,{\theta }_G)=\frac{1-I_{NM}{(x,y,{\sigma }_G,{\theta }_G)}^2}{1+I_{NM}{(x,y,{\sigma }_G,{\theta }_G)}^2},$$

(3)

The final gradient-weighted image $I_{WOS}$ was obtained by summing $I_W$ of all orientations and scales as follows, and adaptive threshold processing was applied to $I_{WOS}$ to obtain the binary candidate image $I_{CAN}$.

$$I_{WOS}(x,y,{\sigma }_G)={\displaystyle \sum _{{\sigma }_G=1}^3{\displaystyle \sum _{{\theta }_G=0}^{\pi /2}{I}_W(x,y,{\sigma }_G,{\theta }_G)}}$$

(4)

Then, a set of infrared small targets ($C_{k\in \{1,\cdots ,N_C\}}$, where $N_C$ is the total number of components) was extracted from each component by analyzing the connected components in each candidate image. To further remove false-alarm candidates with significant differences from the small targets (e.g., large cirrus clouds and long edges), the aspect ratio of the area of the object and the minimum bounding rectangle were calculated, thus reducing the false positive rate, complexity, and time consumption for the next step. In our experiments, if the area of the object and the aspect ratio of the minimum bounding rectangle were less than 400 px and 4, respectively, the object was included in the final selection of candidate set $M_{k\in \{1,\cdots ,N_s\}}$. The parameters were adjusted to provide the maximum detection rate.

2.4. Multi-Feature Modeling

In this step, we constructed a multi-feature model for each candidate, in which we extracted three types of features, namely intensity-based features, geometry-based features, and LCI-based features [22].

Table 1. Description of extracted features in multi-feature modeling.

Type	Name	Description
Intensity-based features	$I_{mean},I_{std},I_{ske}$	The mean, standard deviation and skewness
	$I_{ent},I_{ene}$	The entropy and energy
	$I_{con}$	The contrast
Geometry-based features	$S_{rec}$	Ratio of the target area to the area of the enclosing matrix
	$S_{cir}$	Ratio of the target area to the square of the perimeter of the outer contour
	$S_{sol}$	Ratio of the target area to the convex area
	$S_{ecc}$	Ratio of distance between the foci and the major axis length of the ellipse with a same second moment as the region
LCI-based features	$F_{ARF},R_{ARF}$	ARF filter responses on and its radius average
	$F_{SBF},\overline{R_{SBF}}$	SBF filter responses on and its radius average
	$F_{DRF},\overline{R_{DRF}}$	DRF filter responses on and its radius average

describes the features we used in the proposed method.

2.4.1. Intensity-Based Features

Intensity features are integral to feature extraction in many image classification methods [23,24,25]. We used a candidate region extraction algorithm based on edge extraction, so some candidate regions were edges of complex backgrounds with intensity features that were often quite different from those of bright true targets. The features we used included mean ($I_{mean}$), standard deviation ($I_{std}$), entropy ($I_{ent}$), energy ($I_{ene}$), skewness ($I_{ske}$), and contrast ($I_{con}$), which were extracted from the target in the center of the candidate areas.

2.4.2. Geometry-Based Features

Geometric features are also significant features that can be used to characterize the overall shape of a target [26]. Real targets tend to have smoother edges and more rounded geometry than false targets [20,27]. We can extract geometric properties to describe the shape of the candidate, mainly including the following:

• Rectangularity ($S_{rec}$): the ratio of the target area to the area of the enclosing matrix.
• Roundness ($S_{cir}$): the ratio of the target area to the square of the outer contour perimeter.
• Solidity ($S_{sol}$): the ratio of the target area to the convex area.
• Eccentricity ($S_{ecc}$): the ratio of the distance between the focal point and the long axis length of the ellipse with a same second moment as the region.

2.4.3. LCI-Based Features

Local convergence index (LCI) filter features are obtained mainly by gradient convergence without intensity information, and thus can characterize small, low-contrast, infrared targets hidden in a cluttered background [14]. The LCI filter measures the convergence degree of the surrounding pixels to the central pixel of interest by means of the gradient map of a local region (support region)., as shown in Figure 4. Given a two-dimensional input image, $P(x,y)$ is a pixel of interest, $Q(k,l)$ is a pixel in the support region, and the support region is a circular area of radius $r$. The angle $\theta (k,l)$ is the orientation of the gradient vector $g(k,l)$ for line $\overline{PQ}$. $\cos \theta (k,l)$ calculates the convergence index of the gradient vector at $(k,l)$. The final convergence index (CI) is defined as the average of the convergence index at all pixels as follows:

$$CI(x,y)=\frac{1}{M}{\displaystyle \sum _{(k,l)\in S}\cos \theta (k,l)},$$

(5)

where $M$ is the number of points in the support region $S$, and $\theta (k,l)$ is the gradient vector of points $(k,l)$. The output of CI is between −1 and +1. If all gradient vectors point to the pixel of interest, CI will get a maximum of +1, which can be used to describe a small target in the shape of a bottom cap. For a small infrared target with a gray level higher than the background, its value tends to −1.

In practical calculations, CI of all pixels in $S$ is usually not calculated. As shown in Figure 4b, the points are obtained by sampling in N directions with the pixel of interest as the starting point. The angle $\theta$ in the support region line $L_i$ is calculated as follows:

$$\begin{array}{l}\theta (x,y,{\alpha }_i,m)={\alpha }_i-\varphi (x,y,{\alpha }_i,m)\\ \varphi (x,y,{\alpha }_i,m)={\tan }^{-1}\left(\frac{\frac{\partial }{\partial x}I(x+m\sin ({\alpha }_i),y+m\cos ({\alpha }_i))}{\frac{\partial }{\partial x}I(x+m\sin ({\alpha }_i),y+m\cos ({\alpha }_i))}\right)\end{array},$$

(6)

where $\varphi (x,y,{\alpha }_i,m)$ is the angle related to the gradient vector, and its coordinates lie in the ${\alpha }_i$ direction of the pixel of interest with the radial coordinate $m$.

Different CI filters are formed according to different support regions and search methods, among which the adaptive ring filter (ARF) [28] and the sliding band filter (SBF) [29] are the most typical CI filters.

The support region of an ARF is a ring band, and the radius can be adaptively changed, as shown in Figure 5a. The response of an ARF is given by:

$$F_{\mathrm{ARF}}(x,y)=\underset{0\le r\le R_{\max }}{\max }\frac{1}{N\times d}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))}},$$

(7)

where $N$ denotes the number of support region lines as described above, $R_{\max }$ is the radius of the support region, and $d$ is the width of the band. The value $m$ on different support lines is the same, so the support region is a circle (T) or a ring, which facilitates the extraction of convergent features for small circular targets.

We take the center coordinate of each candidate as the point of interest for the ARF. In addition to the filtering results, the radius that causes the maximum response is calculated and used as an LCI feature:

$$R_{\mathrm{ARF}}(x,y)=\underset{0\le r\le R_{\max }}{\max }\frac{1}{N\times d}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))}},$$

(8)

where $R_{\mathrm{ARF}}$ is the radius that causes the maximum response, corresponding to the highest convergence of the point of interest $(x,y)$.

Since the shape of a small target is not necessarily circular, the SBF allows a more flexible characterization of target convergence properties. As shown in Figure 5b, the SBF’s support region is a band of fixed width with different radii in each direction, and the response is obtained by:

$$F_{\mathrm{SBF}}(x,y)=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left[\underset{R_{\min }\le r\le R_{\max }}{\max }{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))}\right]},$$

(9)

where $R_{\min }$ and $R_{\max }$ are the inner and outer sliding band limits, respectively.

Similarly, we use SBF response at the center of the candidate region as one of the local convergence index features. Considering that the SBF has different radii on different support lines, the average radius can be used as the second feature:

$$R_{\mathrm{SBF}}(x,y,i)=\underset{R_{\min }\le r\le R_{\max }}{\arg \max }{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))},$$

(10)

$$\overline{R_{\mathrm{SBF}}}(x,y)=\frac{1}{N}{\displaystyle \sum _{i=1}^N{R}_{SBF}(x,y,i)},$$

(11)

where different support lines have different $R_{\mathrm{SBF}}(x,y,i)$, and $\overline{R_{\mathrm{SBF}}}(x,y)$ is the average radius.

Although the above two methods have a certain ability to describe small infrared targets, they still have drawbacks: the search area is a fixed circular region, while the actual data of small infrared targets may have different scales and shapes, making it difficult to find a definite support region for CI index calculation. Setting a larger region will be time-consuming and introduce more errors. The search region should ideally be more adaptive according to the shape of the target. Therefore, we propose a new CI filter called the Deformable Region Filter (DRF). The DRF has an adaptive support region, which is related to the target shape, as shown in Figure 5c.

The gradient method is used to obtain the candidate regions of small infrared targets. Thus, the white pixel region in the binary candidate image $I_{CAN}$, with sufficient gradient information, can be used as the search region. The search region can be described as follows:

$$SR=\{(x,y)|I_{CE}(x,y)=1\},$$

(12)

where $I_{CE}$ is the result of dilating $I_{CAN}$ using 3 × 3 structural elements to obtain a larger search field. The average of all coordinates in $SR$ is calculated as the coordinate of the pixel of interest. Subsequently, this coordinate is used as the origin of the coordinate system.

Then the points on the support region line $I_i$ can be obtained by:

$$L_i=\{(x,y)|x=[m\cos {\alpha }_i],y=[m\sin {\alpha }_i],m\ge 1\},$$

(13)

where ${\alpha }_i=2\pi (i-1)/N,i\in \{1,\cdots ,N\},[x]$ is for rounding $x$. Theoretically, a larger $N$ means a smaller angle, which allows for a more accurate description of the convergence of a target in multiple directions. Since small targets in this study occupied fewer than 200 pixels, the value of $N$ was set to 36.

The set of coordinates of the support region in a certain direction can be expressed as:

$$SR{L}_i=SR\cap L_i.$$

(14)

Considering the special case where $SR{L}_i$ is an empty set, we complement it using the adjacent support line $SR{L}_{i-1}$, which can be obtained as follows:

$$\begin{array}{l}SR{L}_i=\{(x,y)|x=[m\cos {\alpha }_i],y=[m\sin {\alpha }_i],\\ R_{\min }(i-1)\le m\le R_{\max }(i-1)\}\end{array},$$

(15)

where $R_{\min }(i-1)$ and $R_{\max }(i-1)$ correspond to the maximum and minimum distances from the point on the line E to the pixel of interest, respectively.

The final response of the DRF is obtained by:

$$F_{\mathrm{DRF}}(x,y)=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left[\underset{R_{\min }(i)\le r\le R_{\max }(i)}{\max }{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))}\right]}$$

(16)

and the average radius can be obtained by:

$$R_{\mathrm{DRF}}(x,y,i)=\underset{R_{\min }(i)\le r\le R_{\max }(i)}{\max }{\displaystyle \sum _{m=r}^{r+d}\cos (\theta ({\alpha }_i,m))}$$

(17)

$$\overline{R_{\mathrm{DRF}}}(x,y)=\frac{1}{N}{\displaystyle \sum _{i=1}^N{R}_{\mathrm{DRF}}(x,y,i)}.$$

(18)

We conducted ARF, SBF and DRF filtering, respectively, on normalized images, and used the obtained filtering response and estimated radius as LCI features for subsequent classification. To maximize the LCI response of the infrared small target, we reversed the obtained gradient.

The complete feature set F consisted of 16 features, including 6 intensity-based features, 4 geometry-based descriptors, and 6 LCI-based features, as detailed in

Table 2. Main characteristics of the datasets we used.

Datasets	Image Type	Background	NI	Target Type
NUAA-SIRST	Real	Cloud/city/sea	427	Point/spot/extended
NUDT-SIRST	Synthetic	Cloud/City/Sea/Field/Highlight	1327	Point/spot/extended

. All features were extracted from each candidate in set M and normalized before input to the classifier.

2.5. Classifier and Detector

In many infrared small target detection algorithms, there is a problem of detecting more false targets than real targets, which forms uneven categories. Therefore, we used RUSBoost [30] in the classification step to distinguish between real targets and false-alarm candidates.

RUSBoost uses an adaptive boosting classifier (AdaBoost) [31] to train the sample set and random undersampling (RUS) [15] technology to solve the problem of class imbalance.

The main idea of the Adaboost algorithm is to train multiple weak classifiers through different sample sets and conduct joint evaluation on the final samples; that is, combine multiple weak classifiers into a strong classifier. During the training process of each weak classifier, the weight of the samples changes adaptively. The weight of the wrongly predicted samples increases in the next round of training, while the weight of the correctly predicted samples decreases [31].

Before training, the category distribution of the dataset can be changed by using RUS technology [15]. The class imbalance problem can be solved by randomly deleting instances from most classes to get a specific sample distribution. In particular, in each iteration of the AdaBoost algorithm, RUSBoost randomly undersamples a subset of the majority class, causing the weak learner to be trained using a balanced dataset. Each weak learner generates a prediction of the new input data weighted by the learner’s stage values. The sum of all weighted predictions is assigned as the final classification result.

In the context of infrared small target classification, RUSBoost was a suitable classifier, as we were dealing with skewed datasets with a few true target candidates and mostly false-alarm candidates.

The details of the RUSBoost classifier are given in Algorithm 1 [22]. This study used decision trees as weak learners in the RUSBoost algorithm. During training, the number of trees was set to 90, and the learning rate was 0.03. Validation experiments were performed using five-fold cross-validation on dataset NUAA-SIRST [32] and dataset NUDT-SIRST [33].

Algorithm 1. RUSBoost Classifier [22]
Input: Set $D$ of training samples $(F_1,L_1),\cdots ,(F_m,L_m)$ where $m$ is the number of training samples, $F_i\in F$ and $l_i\in L=\{-1,1\}$ Output: RUSBoost classifier $H(x)$
1:	$W_1(i)=\frac{1}{m}$ for all $i$
2:	for$t$ = 1:$T$ do
3:	$D\leftarrow$random undersampling (RUS) training set $D$
4:	${\tilde{W}}_i\leftarrow$ extract weights for the subset $\tilde{D}$
5:	Call Weaklearn with subset $\tilde{D}$ and weights ${\tilde{W}}_t$ to get weak classifier $h_t$:
6:	$h_t=\arg \min {\displaystyle {\sum }_1^{\left|{\tilde{W}}_t\right|}\tilde{W}(i)[l_i\ne h_j(F_i)]},h_j\in H$
7:	Pseudo-loss calculation for $D$ and $W_t$:
8:	${\epsilon }_t={\displaystyle {\sum }_1^m{W}_t(i)[l_i\ne h_j(F_i)]}$
9:	Weight update parameter:
10:	${\alpha }_t=\frac{1}{2}\ln \frac{1-e^t}{e^t}$
11:	Update weights and normalization:
12:	$W_{t+1}(i)=W_t(i)\exp (-{\alpha }_t{l}_i{h}_t(F_i))/{\displaystyle {\sum }_{i=1}^m{W}_{t+1}(i)}$
13:	end for
14:	Output and final classifier:
15:	$H(x)={\displaystyle {\sum }_{t=1}^T{\alpha }_t{h}_t(x)/{\displaystyle {\sum }_{j=1}^T{\alpha }_t}}$

3. Experiment Set

In this section, the evaluation metrics are introduced first. Then the candidate extraction, feature extraction, candidate classification, and final algorithm are evaluated and compared.

3.1. Evaluation Metrics

In the candidate extraction step, recall ($Re$) and false positives per image (FPI) were used to assess its performance and were calculated as follows:

$$Re=\frac{T_{correct}}{T_{ALL}},FPI=\frac{N}{N_1},$$

(19)

where $T_{ALL}$ is the number of extracted candidates and $T_{correct}$ is the number of candidates with true target. $N_1$ is the number of true images in a dataset and $N$ is the number of candidates with false target.

Then, we used receiver operating characteristics (ROC) [34] and the area under ROC curve (AUC) to evaluate the performance of classification and the significance of each feature.

The final result evaluation metrics for infrared small target detection algorithms mainly included pixel-level and target-level evaluations. Semantic segmentation network-based approaches mainly use pixel-level evaluations, such as intersection over union (IoU), accuracy, and recall values. These metrics are primarily concerned with evaluating the target’s shape and do not apply to small targets that lack shape information and texture. Traditional algorithms mainly use target-level measures such as detection rate and false alarm rate, which are more important criteria for single-frame detection. If the center of a candidate region was more than three pixels away from the nearest target, we considered it an incorrectly predicted one.

Intersection over union (IoU): IoU is a pixel-level evaluation metric. It focuses on the shape of the measured object and is calculated by the area of the intersection divided by the area of the union of predicted and labeled areas.

$$IoU=\frac{A_I}{A_U},$$

(20)

where $A_I$ and $A_U$ are the interaction areas and all areas.

Recall ($Re$): $Re$ is a target-level evaluation metric calculated as Equation (19). False alarm rate ($F_a$): $F_a$ is another target-level evaluation metric. It quantifies the ratio of incorrectly predicted pixels to the total number of pixels.

$$F_a=\frac{P_{false}}{P_{ALL}},$$

(21)

where $P_{false}$ and $P_{ALL}$ are the number of incorrectly predicted pixels and total pixels, respectively.

3.2. Datasets

We used the NUAA-SIRST dataset and the NUDT-SIRST dataset to evaluate our proposed algorithm, and their information is shown in Table 2.

4. Results

4.1. Candidate Extraction Evaluation

Candidate extraction is a crucial step in the algorithm, which requires high recall and low FPI. Higher recall at this stage ensures higher recall for the final algorithm, and lower FPI reduces the time consumption and false alarm rate $F_a$ of the final algorithm.

Table 3. Candidate extraction results.

Datasets	Recall	FPI
NUAA-SIRST	0.9834	2.86
NUDT-SIRST	0.9665	1.86

shows the recall and FPI of the proposed infrared small target candidate extraction technique.

The proposed method achieved a sensitivity of 0.9834 on the NUAA-SIRST dataset, with a corresponding FPI of 2.86. On the NUDT-SIRST dataset, the sensitivity and FPI values were 0.9665 and 1.86, respectively. These results demonstrate that our candidate extraction algorithm had high sensitivity and generated a low average number of negative samples per image. Figure 6 displays the original images and their corresponding gradient magnitude images.

The gradient magnitude was obtained as a core step of candidate extraction and was calculated using Equation (2). As shown in the figure, the proposed method exhibited a strong response to dim targets with bright and complex backgrounds, facilitating the easy extraction of targets later.

4.2. Candidate Classification Evaluation

In the process of multi-feature modeling, we extracted intensity features (IF) to describe the grayscale statistics of the target; geometric features (GF) to describe the shape of the target; and LCI features (LCI) to describe the gradient convergence of the target, which included the ARF feature (ARF), SBF feature (SBF), and the DRF feature (DRF) proposed by us. The ROC curve and AUC value were used to measure the classification performance of different features, and the results are shown in

Table 4. Candidate classification results.

Feature Subset	NUAA-SIRST	NUDT-SIRST
IF	0.94412	0.9747
GF	0.938	0.96678
IF + GF	0.97083	0.99342
IF + GF + ARF	0.97425	0.9953
IF + GF + SBF	0.97559	0.99541
IF + GF + DRF	0.98762	0.99573
All	0.98882	0.99661

and Figure 7. On the real dataset NUAA-SIRST, the AUC value was 0.97491 when using only intensity and geometric features for classification, which increased to 0.98679 after adding LCI features. Weak targets were easy to miss when using only intensity characteristics and geometric shapes of targets; however, adding LCI features that measure gradient but are not affected by intensity enabled correct identification of such targets. Therefore, the performance of the model was improved with the introduction of LCI features. At the same time, the addition of our proposed DRF feature also improved the model performance, with an AUC value of 0.98639.

Figure 7b shows the results on the synthetic dataset NUDT-SIRST. Excellent results were obtained using only the geometry and intensity information of the target, with an AUC value of 0.99342. This was possible because the same object was used for many different backgrounds in synthetic images. The LCI features did not significantly improve the model, and its AUC value was 0.99661.

In terms of time consumption, the extraction time for intensity features, geometric features, and LCI features in each candidate region was 0.001 s, 0.001 s, and 0.01 s, respectively.

4.3. Detection Results

Table 5. IoU, $Re$, and $F_a$ values achieved on the NUDT-SIRST and NUAA-SIRST datasets.

Method	NUDT-SIRST			NUAA-SIRST
	IoU	$\mathit{R}\mathit{e}$	${\mathit{F}}_{\mathit{a}}$	IoU	$\mathit{R}\mathit{e}$	${\mathit{F}}_{\mathit{a}}$
	(×10−2)	(×10−2)	(×10−6)	(×10−2)	(×10−2)	(×10−6)
Top-hat [35]	20.72	78.41	166.7	7.143	79.84	1012
Max–median [36]	4.197	58.41	36.89	4.172	69.2	55.33
WSLCM [37]	2.283	56.82	1309	1.158	77.95	5446
TLLCM [38]	2.176	62.01	1608	1.029	79.09	5899
IPI [11]	17.76	74.49	41.23	25.67	85.55	11.47
NRAM [39]	6.927	56.4	19.27	12.16	74.52	13.85
RIPT [40]	29.44	91.85	344.3	11.05	79.08	22.61
PSTNN [13]	14.85	66.13	44.17	22.4	77.95	29.11
MSLSTIPT [41]	8.342	47.4	888.1	10.3	82.13	1131
MDvsFA-cGAN [33]	75.14	90.47	25.34	60.3	89.35	56.35
ACM [32]	67.08	95.67	10.18	70.33	93.91	3.728
ALCNet [7]	81.4	96.51	9.261	73.33	96.57	30.47
Proposed	49.64	90.64	9.833	44.35	90.6	9.06

shows IoU, $Re$, and $F_a$ values achieved on NUDT-SIRST and NUAA-SIRST datasets by our method compared with different SOTA methods: Filter-based methods include top-hat [35] and max–median [36]; local contrast-based methods include WSLCM [37] and TLLCM [38]; local rank-based methods include IPI [10], NRAM [39], RIPT [40], PSTNN [13], and MSLSTIPT [41]; CNN-based methods include MDvsFA-cGAN [33], ACM [32], and ALCNet [7]. Since our candidate extraction algorithm adopted the edge extraction method, we successively performed the morphological closing operation and corrosion operation on the candidate mask to eliminate holes and excessive edges when calculating IoU. Regarding IoU value at the pixel level, our algorithm was not better than CNN networks on pixel segmentation but it was better than traditional algorithms, such as filter-based, local contrast-based, and local rank-based methods. CNN-based algorithms, such as MDvsFA-cGAN, ACM, and ALCNet, achieve higher $Re$ than other traditional algorithms due to their data-driven approach, which aims to minimize loss or maximize accuracy. On target-level evaluation metrics, the values of $Re$ in datasets NUDT-SIRST and NUAA-SIRST were 0.9064 and 0.9060, respectively, which were lower than those of ACM and ALCNet, but higher than those of most traditional algorithms. Although the detection rate of our algorithm was lower than that of the CNN-based algorithms, our framework can be applied to images with different resolutions and requires fewer images to train the model. Additionally, our algorithm appeared to be more effective in eliminating false alarms, with the values of $F_a$ in the two datasets being $9.833\times {10}^{-6}$ and $9.06\times {10}^{-6}$, respectively, which were much lower than in other traditional algorithms, and the average $F_a$ of the two datasets was lower than that of ALCNet. In the candidate extraction step, we extracted all targets that met the gradient threshold through gradient weight technology, which contained a large number of true targets, which explains why the detection rate of our algorithm was higher than most traditional algorithms. However, less consideration was given to the problem of false alarm removal. Therefore, in the next step, we further applied multi-feature modeling and classification to remove false alarms.

For better visualization, Figure 8 and Figure 9 show qualitative evaluations on two datasets of different methods, including top-hat [35], LCM [8], MPCM [9], IPI [10], RIPT [40], NRAM [39], PSTNN [13], RTRC [42], TMESNN [43] and our proposed method. Classic infrared target detection algorithms, such as top-hat and LCM, cannot avoid the occurrence of false alarms in various scenarios. Models based on low-rank decomposition, such as IPI, RIPT, and NRAM, perform well in simple scenarios but slightly worse in complex ones. For example, the image containing a complex background in line 5 shows a large number of false alarms, and PSTNN and TMESNN even missed the weak target. Our approach generated accurate target localization and segmentation with a very low false alarm rate compared with traditional methods.

The proposed algorithm was also evaluated for time consumption. It consisted of three processes: candidate extraction, multi-feature modeling, and feature classification.

Table 6. Time consumption of each step of the process.

Step	Time (s)
Candidate extraction	0.02–0.06
Multi-feature modeling	0.01–0.24
Classifier and detector	0.04

shows the running time of each step of the process on NUAA-SIRST. In the candidate extraction step, multiple convolution operations were first used to find the target candidate region, which consumed 0.02 s. Then, image blocks were extracted for subsequent feature extraction, which took approximately 0.002 s to extract an image block. Therefore, the running time of this step was mainly determined by the number of extracted candidates. Considering that the number of candidates ranged from 0 to 20, the running time of this step was roughly between 0.02 s and 0.06 s. Meanwhile, feature extraction expended 0.012 s of time for each candidate, so the running time of the multi-feature modeling step ranged from 0 s to 0.24 s. The average discriminant time of the final classifier was 0.04 s. In other words, the algorithm took longer to run for complex backgrounds, such as buildings, because more candidates could be extracted and classified. For relatively simple backgrounds, such as the sky, the time consumption was as low as 0.07 s, which shows the practical application potential of this algorithm. We also made a comparison with other algorithms in terms of time consumption, and the results are shown in

Table 7. Processing time for each image in the NUAA-SIRST dataset by different algorithms.

Method	Time (s)
Top-hat	0.1056
LCM	7.3676
MPCM	0.1261
IPI	5.4185
NRAM	0.1507
PSTNN	0.1687
RTRC	0.7576
TMESNN	0.1393
proposed	0.1652

. Although our algorithm was not the fastest, compared with PSTNN and NRAM with similar speed, it had greater advantages in detection rate and false alarm elimination.

5. Conclusions

In this paper, a novel approach for infrared small target detection was proposed by combining gradient weighting with LCI features, and small infrared targets were accurately identified using RUSBoost. The problem that detection algorithms are prone to false and missing detections under complex scenes was explicitly addressed by this method. LCI-based descriptors were shown to characterize low-contrast objects well, since the LCI filters were gradient convergence-based. Extensive experiments on NUDT-SIRST and NUAA-SIRST datasets demonstrated that the performance of the proposed method was competitive with SOTA but with lower time consumption. The proposed method has limitations in its single-frame nature since it is specifically designed for single-frame infrared target detection applications. In applications that allow for multiple frames, multi-frame-based algorithms can obtain more temporal information from the image sequences, which can enable better suppression of noise similar to targets in the images. Our future works will focus on improving the detection of infrared small targets in high-frequency regions, including skyline and horizontal plane lines. To achieve this goal, we plan to explore an integrated learning framework that combines deep features with physical characteristics. We believe that this approach could effectively enhance the detection performance of infrared dim small targets and provide more reliable solutions for real-world applications.