Infrared Small-Target Detection Using Multiscale Local Average Gray Difference Measure

: In infrared (IR) guidance and target tracking systems, dim target intensity and complex background clutter are some of the typical challenges, especially for the accurate detection of small objects. In this article, we propose a novel IR target detection method based on new local contrast measures. First, the local average gray difference measure (LAGDM) is presented to accentuate the difference between a small object and its local background. Then, an LAGDM map is generated to effectively enhance targets and suppress background clutter. Finally, we use an adaptive segmentation method to separate the object from the background. Experimental results on multiple sequences show that the proposed small-target detection method can effectively improve the signal-to-clutter ratio (SCR) of the image, and it exhibits robust performance against cloudy sky, sea sky, and mountain forest backgrounds.


Introduction
The infrared search and tracking (IRST) system has the advantages of strong antiinterference and high resolution, and it is widely used in military, aerospace, and industrial testing fields [1,2]. Accurate detection of IR-weak targets enables the accurate guidance of weapon systems and the early warning of dangerous targets, and it is one of the key technologies of IRST systems [3,4]. Small targets occupy few pixels in the image; therefore, there is a lack of information on shape, size, and texture, and their images are disturbed by complex backgrounds, such as cloudy sky, sea sky and sea clutter. Because of this, small-target detection is a challenging task [5,6]. Therefore, the accurate detection of IRweak targets has become a popular research topic in recent years and has considerable significance for military and other practical applications.
For the small-object detection method of single-frame images, typically, the spatial information of the image is used to obtain the position of a small target. Several contemporary methods focus on enhancing targets and suppressing the background. Based on the Gaussian model for spatial distribution of the IR small objects, the Laplacian of Gaussian (LoG) [7] and difference of Gaussian (DoG) methods [8] are proposed to enhance low signalto-clutter ratio (SCR) targets. However, they are sensitive to background noise, especially background edges. In contrast, some methods focus on background features; first, the original image and the estimated image background are used to determine the difference; then, threshold segmentation methods, such as top-hat [9], max/median [10], and PQFT [11], are used to achieve weak target detection. These methods are computationally simple and exhibit fast detection, but they are sensitive to edge information. Some transform domain filtering-based methods have demonstrated good background suppression performance, such as high-pass filtering and wavelet transform filtering methods; however, they are computationally complex.
In some methods, detection operators are designed, and the target extracted, according to the differences in the grayscale, energy, and structure of the target and the background. Inspired by the human visual contrast mechanism, Chen et al. proposed a local contrast measure (LCM) to highlight small targets with grayscale values higher than the surrounding area [12]. In this method, the entire image is traversed pixel by pixel; as a result, the realtime performance is poor. Han et al. improved the LCM method based on the visual attention shift mechanism [13], reducing the false alarm rate (FAR) to some extent; however, it still did not solve the problem of over-enhancement of background highlights. Wei et al. proposed a small-target detection method based on patch differences [14]; it uses a parallel method to improve the computing speed, but in the case of complex background clutter, the detection performance is not adequate. Deng et al. proposed a weighted local difference measure (WLDM) to distinguish between the real target and the background interference object [15]. Han et al. used the difference in the grayscale of the target and the background to reduce the influence of highlight background noise [16]. Xia et al. proposed a new contrast factor (local energy factor, LEF) for small-target detection [17]. Deng et al. proposed the average absolute gray difference (AAGD) to suppress background noise and enhance weak small targets. This method is also sensitive to high-intensity edges [18,19]. This literature review shows that the HVS-based small-target detection method has attracted considerable attention, but it is necessary to improve its real-time performance and adaptability under complex backgrounds.
In this paper, a new IR small-target detection algorithm is proposed using multiscale relative average gray difference measures. The method considered detection as well as realtime performance, and it demonstrated effective performance in the presence of complex background clutter. The main contributions of this study are as follows: (1) An average gray difference metric is proposed that can effectively reflect the prominence of a target on a background; (2) An LAGDM-based multiscale small-target detection algorithm is designed; (3) The calculation method of the LAGDM map is optimized, and adaptive threshold segmentation is used to realize the rapid detection of small targets. Experimental results show that this method has the advantages of high accuracy and fast speed.
The rest of this paper is organized as follows. Section 2 describes the calculation principle of the difference measurement factor LAGDM and discusses its feasibility for target detection. Then, detailed calculation steps of the proposed detection algorithm are given. Section 3 presents comparative experiments to verify the proposed method. Conclusions and perspectives are presented in Section 4.

Methodology
This study introduces a new scheme for IR small-target detection that uses a new difference measure factor LAGDM to highlight targets on complex backgrounds. The calculation principle of LAGDM is shown in Figure 1a; it then uses adaptive threshold segmentation to extract real targets. Furthermore, the calculation process of the detection algorithm is optimized.

Local Average Gray Difference Measure
The energies and structures of small targets and their surrounding regions show subtle to considerable differences. Because of this, a large number of HSV-based IR smalltarget detection algorithms have been proposed, and these are robust and accurate. In this section, we propose a new local difference measure. First, we define the target and its local background area as shown in Figure 1b. T represents the target area, and B represents the local background area.
Relative local contrast has better results in small-target detection applications. Therefore, we use the smallest pixel of the local background as a benchmark to construct the relative gray index, T E Δ , of the target area, and it is defined as follows: where m ax T is the maximum grayscale value of the target area, and m in B is the minimum grayscale value of the background area. To avoid the influence of large surrounding area selection on target detection, we define the weighted average relative gray index of the background area as follows, to weaken the influence of pixels in long-distance areas: where N is the number of pixels of the local background, and k B is the grayscale value of the local background area. k d is the distance from a pixel, with a value of k B , to the center of the target. G is a Gaussian function with standard deviation, s td G , and where 0 E is a positive constant to prevent the denominator from being 0. The LAGDM can reflect the extent of protrusion of the target on a background; for a more prominent target, the value of LAGDM is larger. For detecting small and weak IR targets, different types of interference can be encountered, such as high brightness and sea sky, among other complex backgrounds. We select several groups of challenging IR small target images to analyze the effectiveness and reliability of LAGDM, as shown in Figure 2.

Local Average Gray Difference Measure
The energies and structures of small targets and their surrounding regions show subtle to considerable differences. Because of this, a large number of HSV-based IR small-target detection algorithms have been proposed, and these are robust and accurate. In this section, we propose a new local difference measure. First, we define the target and its local background area as shown in Figure 1b. T represents the target area, and B represents the local background area.
Relative local contrast has better results in small-target detection applications. Therefore, we use the smallest pixel of the local background as a benchmark to construct the relative gray index, ∆E T , of the target area, and it is defined as follows: where T max is the maximum grayscale value of the target area, and B min is the minimum grayscale value of the background area. To avoid the influence of large surrounding area selection on target detection, we define the weighted average relative gray index of the background area as follows, to weaken the influence of pixels in long-distance areas: where N is the number of pixels of the local background, and B k is the grayscale value of the local background area. d k is the distance from a pixel, with a value of B k , to the center of the target. G is a Gaussian function with standard deviation, G std , and based on the relative gray index of the target and background, we propose a simple and effective metric: where E 0 is a positive constant to prevent the denominator from being 0. The LAGDM can reflect the extent of protrusion of the target on a background; for a more prominent target, the value of LAGDM is larger. For detecting small and weak IR targets, different types of interference can be encountered, such as high brightness and sea sky, among other complex backgrounds. We select several groups of challenging IR small target images to analyze the effectiveness and reliability of LAGDM, as shown in Figure 2. For a typical scene shown in Figure 2, the LAGDM values of the target and background clutter are calculated. According to the histogram on the right, the LAGDM value of the target is significantly greater than the LAGDM value of the background. Whether there is sea background clutter or a high-brightness area, it can be distinguished from the target region using LAGDM values. To some extent, the LAGDM enhances the difference between the target and the background, which is also beneficial for small-target detection.

Multiscale LAGDM Map
To suppress the high brightness noise in the background, the average value of the central area is substituted for its maximum value, and a new target relative gray index is constructed as follows: where mean T is the mean of the center sub-patch. The center sub-patch is slid pixel by pixel from left to right and from top to bottom on the original image, as shown in Figure  3b, and the LAGDM value of the center block at each position is calculated using Equation (3). Ideally, the size of the center sub-patch should be consistent with the target size. However, in practical applications, the sizes of small targets may vary with time, making prediction difficult. Therefore, we use center sub-patches with different sizes to traverse an image, calculate the corresponding LAGDM values, and obtain multiple different LAGDM-based saliency maps. At this time, w and h are unknown, and we assume w = h=l, where l is the size of the center sub-patch. The calculation principle of the LAGDM-based saliency map is shown in Figure 3.  For a typical scene shown in Figure 2, the LAGDM values of the target and background clutter are calculated. According to the histogram on the right, the LAGDM value of the target is significantly greater than the LAGDM value of the background. Whether there is sea background clutter or a high-brightness area, it can be distinguished from the target region using LAGDM values. To some extent, the LAGDM enhances the difference between the target and the background, which is also beneficial for small-target detection.

Multiscale LAGDM Map
To suppress the high brightness noise in the background, the average value of the central area is substituted for its maximum value, and a new target relative gray index is constructed as follows: where T mean is the mean of the center sub-patch. The center sub-patch is slid pixel by pixel from left to right and from top to bottom on the original image, as shown in Figure 3b, and the LAGDM value of the center block at each position is calculated using Equation (3). Ideally, the size of the center sub-patch should be consistent with the target size. However, in practical applications, the sizes of small targets may vary with time, making prediction difficult. Therefore, we use center sub-patches with different sizes to traverse an image, calculate the corresponding LAGDM values, and obtain multiple different LAGDM-based saliency maps. At this time, w and h are unknown, and we assume w = h = l, where l is the size of the center sub-patch. The calculation principle of the LAGDM-based saliency map is shown in Figure 3. For a typical scene shown in Figure 2, the LAGDM values of the target and background clutter are calculated. According to the histogram on the right, the LAGDM value of the target is significantly greater than the LAGDM value of the background. Whether there is sea background clutter or a high-brightness area, it can be distinguished from the target region using LAGDM values. To some extent, the LAGDM enhances the difference between the target and the background, which is also beneficial for small-target detection.

Multiscale LAGDM Map
To suppress the high brightness noise in the background, the average value of the central area is substituted for its maximum value, and a new target relative gray index is constructed as follows: where mean T is the mean of the center sub-patch. The center sub-patch is slid pixel by pixel from left to right and from top to bottom on the original image, as shown in Figure  3b, and the LAGDM value of the center block at each position is calculated using Equation (3). Ideally, the size of the center sub-patch should be consistent with the target size. However, in practical applications, the sizes of small targets may vary with time, making prediction difficult. Therefore, we use center sub-patches with different sizes to traverse an image, calculate the corresponding LAGDM values, and obtain multiple different LAGDM-based saliency maps. At this time, w and h are unknown, and we assume w = h=l, where l is the size of the center sub-patch. The calculation principle of the LAGDM-based saliency map is shown in Figure 3.    for j = 1: cols do 5.
Compute LAGDM (i, j, l) according to Equation ( Algorithm 1 has a large number of "for loops", which increases the computational complexity of the algorithm and is time-consuming. Accordingly, we transform Equation (2) as follows: where N = 8 × l 2 , and l × l is the size of a sliding patch. The fast calculation of the LAGDM map can be completed by designing three filters, f 1 , f 2 , and f 3 , which are defined as follows: ones(l, l) ones(l, l) ones(l, l) ones(l, l) zeros(l, l) ones(l, l) ones(l, l) ones(l, l) ones(l, l) where ones(l, l) is a full 1-matrix of l × l, and zeros(l, l) is a full 0-matrix of l × l. G l is a Gaussian matrix of size 3l × 3l and standard deviation l/2. Based on the typical functions "imfilter" and "ordfilt2" in Matlab, we present the fast calculation process of the LAGDM map, as shown in Algorithm 2.

LAGDM-Based Small-Target Detection
Using the proposed local average gray difference measurement, the contrast between the small target and the background is considerably enhanced. To make the target more prominent on the background, we use the average value of grayscale difference between the target and the background to improve the LAGDM map: where I is a unit matrix with the same size as matrix M. Once the LAGDM map is calculated, the target is identified using the adaptive threshold segmentation method. In this study, the segmentation threshold is defined as follows: where µ M and σ M are the mean and standard deviation of the LAGDM map, respectively. κ th is an empirical constant, and it changes as the IR scene changes. Our experiments show that the optimal range of κ th is [5,20].

Experiments
Several sets of comparative experiments are carried out to examine the performance of the proposed algorithm. First, we introduce the evaluation index of the small-target detection algorithm. Then, we use multiple infrared image sequences of different scenes to compare the proposed algorithm with contemporary algorithms. All simulations and experiments are performed on a PC with 16 GB memory and a 2.70 GHz Intel i7 processor, and all codes are completed on matlab2020b.

Evaluation Metrics
The detection performance of IR small targets is mainly determined by two aspects, namely, target enhancement and background suppression. For a single-frame image, typically, the signal-to-clutter ratio gain (SCRG) and background suppression factor (BSF) reflect the performance of an algorithm [20]. Large values of SCRG and BSF imply better algorithm performance. They are defined as follows: where σ in and σ out denote the standard deviation of the full image before and after processing, respectively. SCR out and SCR in are the signal-to-clutter ratios of the original image and the processed image, respectively. µ t is the average of the target region, and µ b and σ b are the average and standard deviation of the neighboring region, respectively. The size of the surrounding area is 3w × 3h. However, in practical applications, the local background of the processed image may be 0, and the effective SCR cannot be calculated. For such situations, we use a new metric, CG = |µ t − µ b |, instead of SCR [20]. For IR image sequences, we evaluate the detection performance using the probability of detection (PD) and the FAR [21].

Enhancement Performance
To demonstrate the applicability of the proposed method, it is compared with MPCM [14], PQFT, and ILCM [15]. Figure 4 shows the results of the proposed algorithm on a sample image. Small targets can be clearly identified from the processed image. The highlight areas and edges in the background are filtered, visually verifying the effectiveness of the proposed algorithm. The SCRG and BSF of each image are determined as shown in Figure 5. Statistical results of the proposed algorithm are optimal in most of the images, indicating that the proposed method achieves better target enhancement as well as background inhibition.
the number of false detections FAR the number of sequence frames

Enhancement Performance
To demonstrate the applicability of the proposed method, it is compared with MPCM [14], PQFT, and ILCM [15]. Figure 4 shows the results of the proposed algorithm on a sample image. Small targets can be clearly identified from the processed image. The highlight areas and edges in the background are filtered, visually verifying the effectiveness of the proposed algorithm. The SCRG and BSF of each image are determined as shown in Figure 5. Statistical results of the proposed algorithm are optimal in most of the images, indicating that the proposed method achieves better target enhancement as well as background inhibition.

Detection Performance
Four sets of sequences (Seq 1, Seq 2, Seq 3 and Seq 4) are used to comprehensively assess the detection performance of the algorithm, and the benchmark algorithms are MPCM, PQFT, ILCM, TLLCM [22] and ADMD [23]. These sequences contain 48, 49, 201, and 67 images, respectively. When the target detection position is within 5 pixels of the labeling position, the real target is detected. The sample detection results of different algorithms are presented in Figure 6, which demonstrates the better target enhancement performance of the proposed algorithm.

Detection Performance
Four sets of sequences (Seq 1, Seq 2, Seq 3 and Seq 4) are used to comprehensively assess the detection performance of the algorithm, and the benchmark algorithms are MPCM, PQFT, ILCM, TLLCM [22] and ADMD [23]. These sequences contain 48, 49, 201, and 67 images, respectively. When the target detection position is within 5 pixels of the labeling position, the real target is detected. The sample detection results of different algorithms are presented in Figure 6, which demonstrates the better target enhancement performance of the proposed algorithm.

Detection Performance
Four sets of sequences (Seq 1, Seq 2, Seq 3 and Seq 4) are used to comprehensively assess the detection performance of the algorithm, and the benchmark algorithms are MPCM, PQFT, ILCM, TLLCM [22] and ADMD [23]. These sequences contain 48, 49, 201, and 67 images, respectively. When the target detection position is within 5 pixels of the labeling position, the real target is detected. The sample detection results of different algorithms are presented in Figure 6, which demonstrates the better target enhancement performance of the proposed algorithm.  Figure 7 shows the receiver operating characteristic (ROC) [21] curve obtained by testing all methods on the image set. Compared with the baseline method, the proposed method showed better detection performance. Therefore, we can conclude that the proposed algorithm can be applied for small-target detection on different types of backgrounds, and it demonstrates good background suppression and target enhancement per-  Figure 7 shows the receiver operating characteristic (ROC) [21] curve obtained by testing all methods on the image set. Compared with the baseline method, the proposed method showed better detection performance. Therefore, we can conclude that the proposed algorithm can be applied for small-target detection on different types of backgrounds, and it demonstrates good background suppression and target enhancement performance.  Figure 7 shows the receiver operating characteristic (ROC) [21] curve obtained by testing all methods on the image set. Compared with the baseline method, the proposed method showed better detection performance. Therefore, we can conclude that the proposed algorithm can be applied for small-target detection on different types of backgrounds, and it demonstrates good background suppression and target enhancement performance.

Computational Complexity
We measured the runtime of the proposed detection algorithm for the different images in Figures 4 and 5, as shown in Table 1. Table 1 shows that the speed of the algorithm is simultaneously related to the size of the image and the scale of the sliding patch, and it is very time-saving. The real-time nature of the algorithm makes it a good application prospect for engineering practice.

Conclusions
This study proposes a small-target detection algorithm based on LAGDM. This method highlights the target and suppresses the background by measuring the relative gray difference between the target and its local region. The computational speed is opti-

Probability of detection (Pd)
Probability of detection (Pd) Figure 7. ROC curves of the proposed method and baseline methods in Seq 1-4.

Computational Complexity
We measured the runtime of the proposed detection algorithm for the different images in Figures 4 and 5, as shown in Table 1.  Table 1 shows that the speed of the algorithm is simultaneously related to the size of the image and the scale of the sliding patch, and it is very time-saving. The real-time nature of the algorithm makes it a good application prospect for engineering practice.

Conclusions
This study proposes a small-target detection algorithm based on LAGDM. This method highlights the target and suppresses the background by measuring the relative gray difference between the target and its local region. The computational speed is optimized on the basis of convolutional filtering. The effectiveness and robustness of the proposed algorithm are demonstrated in different scenarios. The proposed method has the advantages of fast detection speed and high detection accuracy in most cases; however, on images with discrete noise, detection performance needs to be improved. This will be the focus of our future research.

Conflicts of Interest:
There are no conflicts of interest declared by the authors.