Multiframe Infrared Small Target Detection via Novel Low-Rank Approximation and Robust CUR Decomposition

Highlights

What are the main findings?

• We refine the existing low-rank matrix decomposition approaches, allowing the low-rank component of multi-frame infrared images to be more accurately approximated via an abundance matrix.
• Traditional multi-frame infrared small-target detection methods based on low-rank and sparse decomposition suffer from high computational complexity and extremely time-consuming optimization. By introducing column-row (CUR) decomposition into the iterative optimization process, the proposed method significantly reduces the overall computational cost of the algorithm.

What are the implications of the main findings?

• The proposed low-rank approximation enables more accurate recovery of background components in multi-frame infrared images, effectively suppressing background clutter and reducing false detections. As a result, it improves the detection accuracy of low-rank sparse-based infrared small-target detection methods.
• The effective integration of the improved low-rank approximation with robust CUR decomposition enables the proposed method to rapidly and accurately detect infrared small targets in multi-frame infrared imagery.

Abstract

Low-rank sparse decomposition models have become the mainstream optimization framework for multiframe infrared small target detection. Existing low-rank matrix decomposition approximations typically pre-decompose infrared videos into the product of two low-rank matrices to capture the background’s low-rank characteristics. However, such approximations are not optimal and often result in suboptimal background recovery. To achieve more accurate low-rank recovery, we exploit the intrinsic relationship between low-rank matrices and their generalized inverse matrices, thereby improving conventional decomposition approximations. Moreover, to address the high computational cost of applying low-rank and sparse decomposition models to multi-frame infrared videos, we introduce a robust column-row (CUR) decomposition to accelerate the iterative process, thereby significantly improving computational efficiency. The experimental results show that the proposed method achieves fast detection of small targets in infrared videos while maintaining competitive detection performance.

1. Introduction

Infrared small target detection (ISTD) plays a crucial role in locating and identifying targets of interest under low-illumination and complex environmental conditions, and has been widely applied in civil, military, and industrial fields [1,2,3]. Despite significant progress in recent years, multi-frame infrared small target detection still faces substantial challenges. Specifically, small targets are easily submerged in complex background clutter, making reliable detection difficult. Moreover, the high data dimensionality inherent in multi-frame infrared imagery leads to increased computational burden and slow detection speed [4,5]. Consequently, achieving accurate and efficient separation of background sequences and small target sequences from multi-frame infrared images remains a critical and challenging problem.

Existing infrared small target detection methods can be broadly categorized into single-frame-based approaches and multi-frame-based approaches. Single-frame-based methods usually formulate the ISTD problem as a subspace-based or tensor-based optimization model. One of the most classical optimization-based methods is the Infrared Patch-Image (IPI) model, which transforms the single-frame infrared small target detection problem into a matrix-based low-rank and sparse optimization problem by employing a local patch construction strategy [6]. Wang et al. [7] proposed an adaptive robust principal component analysis (RPCA) approach, effectively separating small infrared targets from the background. Dai et al. [8] converted a single infrared image into a three-dimensional tensor and adopted a tensor-based RPCA model to improve small target detection performance. However, single-frame methods utilize only spatial structural information while neglecting the temporal characteristics of target motion. In contrast, multi-frame detection methods can jointly exploit both spatial and temporal features from infrared video sequences, leading to significantly improved detection performance. Therefore, this work primarily focuses on improving both the accuracy and efficiency of small target detection in multi-frame infrared video.

In the field of infrared small target detection, accurate and efficient background modeling is crucial for separating sparse targets from complex scenes. Although tensor-based models have been widely applied to multi-dimensional data, they often suffer from high computational cost, excessive memory consumption, and over-smoothing of small targets due to strict low-rank assumptions. In contrast, matrix models incorporating CUR decomposition achieve low-rank approximation by directly selecting representative rows and columns from the original data, thereby preserving the intrinsic background structure while maintaining target sparsity [9,10]. Furthermore, CUR-based models offer advantages in computational efficiency, memory usage, and robustness to complex or dynamic backgrounds, making them particularly suitable for real-time infrared video small target detection applications [11,12].

Model-driven multi-frame infrared small target detection methods mainly rely on low-rank and sparse decomposition of matrices or tensors. These methods are based on the observation that background sequences in multi-frame infrared images exhibit low-rank characteristics, whereas small targets are typically sparse, enabling effective separation of the two components. To cope with noise interference, tensor robust principal component analysis (TRPCA)-based methods and their variants have attracted increasing research interest in recent years [13,14,15]. Sun et al. [16] introduced total variation (TV) regularization into the tensor decomposition model to capture local smoothness in both spatial and temporal dimensions. Zhang et al. [17] incorporated edge-aware priors into a spatio-temporal tensor framework to better distinguish background structures from moving small targets. Despite their effectiveness, tensor-based models often involve complex regularization terms and high-order tensor operations, which significantly increase computational burden. As a result, they are often computationally intensive and suffer from low detection efficiency. To approximate low-rank background components, many methods rely on nuclear norm minimization or manually designed regularizers, which typically involve expensive singular value decomposition (SVD) during optimization [18,19,20]. To mitigate this issue, RCTVW [21] employed low-rank matrix factorization (LRMF) to avoid SVD computations, thereby improving computational efficiency. By applying TV regularization to the reduced-size matrices obtained via matrix decomposition, RCTVW significantly reduced complexity and improved runtime. Inspired by this, we also adopt low-rank matrix decomposition to approximate the low-rank background. Furthermore, we improve existing low-rank decomposition approximation methods by analyzing the relationship between the low-rank matrix and its generalized inverse, allowing for more accurate background recovery. The background restoration results for infrared videos obtained using the improved low-rank approximation and the conventional low-rank matrix factorization–based approximation are presented in Figure 1. It can be observed that, during the iterative optimization process, the improved low-rank approximation consistently maintains more accurate background restoration performance. Finally, to further accelerate the optimization process, we adopt the robust CUR decomposition method proposed in [22], which significantly reduces the computation time of the overall algorithm. The main contributions are summarized as follows:

Figure 1. Background restoration results using the improved low-rank approximation and the conventional LRMF approximation. The red dotted line indicates the position of the minor target.

• We propose an improved low-rank approximation method that more accurately recovers background components in infrared videos, thereby improving the accuracy of small target detection;
• The effective integration of the improved low-rank approximation and robust CUR decomposition enables the proposed method to rapidly and accurately recover low-rank backgrounds in infrared videos while simultaneously detecting sparse small targets;
• Extensive experimental results show that the proposed method achieves fast and accurate small target detection in multiframe infrared images.

2. Related Work

2.1. Traditional Low Rank Approximation

Low rank approximation mainly approximates the low-rank component of the observation matrix $\mathit{Y}$ by imposing constraints on the two submatrices $\mathit{U}$ and $\mathit{V}$. Different constraints form SVD or PCA approximation method [23] and nonnegative matrix factorization (NMF) approximation method [24].

$$\begin{array}{l}\underset{\mathit{U},\mathit{V}}{\min }{‖\mathit{Y}-\mathit{U}{\mathit{V}}^T‖}_F\hspace{1em}\mathit{s}.\mathit{t}.\hspace{1em}\mathit{U}={\mathit{V}}^T\hspace{1em}(\mathit{S}\mathit{V}\mathit{D}/\mathit{P}\mathit{C}\mathit{A})\\ \underset{\mathit{U},\mathit{V}}{\min }{‖\mathit{Y}-\mathit{U}\mathit{V}‖}_F\hspace{1em}\mathit{s}.\mathit{t}.\hspace{1em}\mathit{U}\ge 0,\mathit{V}\ge 0\hspace{1em}(\mathit{N}\mathit{M}\mathit{F})\end{array}$$

(1)

Low-rank approximation has been widely used in the fields of large language model [25], background subtraction [26] and Infrared Small Target Detection [27,28].

2.2. Robust CUR Decomposition

Literature [22] proposed and demonstrated the feasibility of robust CUR decomposition. Specifically, given the observation matrix $\mathit{Y}=\mathit{X}+\mathit{S}$, where $\mathit{X}$ is the low-rank component and $\mathit{S}$ denotes the noise component, a subset of columns and rows is uniformly sampled from $\mathit{Y}$, forming the column submatrix $\mathit{C}=\mathit{Y}(:,\mathcal{J})$ and the row submatrix $\mathit{R}=\mathit{Y}(\mathcal{I},:)$, respectively. Let $\mathcal{I}$ and $\mathcal{J}$ denote the row and column index sets, respectively. The cardinalities $\left|\mathit{I}\right|$ and $\left|\mathit{J}\right|$ correspond to the numbers of sampled rows and sampled columns. Robust principal component analysis (RPCA) is then applied to $\mathit{C}$ and $\mathit{R}$ separately to obtain their low-rank approximations, denoted as matrices $\widehat{\mathit{C}}$ and $\widehat{\mathit{R}}$. The CUR approximation of the entire low-rank matrix $\mathit{X}$ is then constructed as $\mathit{X}=\widehat{\mathit{C}}{(\widehat{\mathit{C}}(\mathcal{I},:))}^{†}\widehat{\mathit{R}}$. This robust CUR decomposition has been successfully applied to problems such as random noise attenuation [29] and infrared small target detection [9].

3. Method Description

3.1. Improved Low Rank Decomposition Approximation

Consider an optimization problem as follows:

$$\underset{\mathit{X},\mathit{S}}{\min }{\displaystyle \frac{1}{2}}{‖(\mathit{Y}-\mathit{S})-\mathit{X}‖}_{\mathit{F}}^2+{‖\mathit{X}‖}_{\ast }+\lambda {‖\mathit{S}‖}_1$$

(2)

where $\mathit{Y}\in {ℝ}^{\mathit{m}\times \mathit{n}}$ is the original observation matrix, $\mathit{X}\in {ℝ}^{\mathit{m}\times \mathit{n}}$ is the low-rank approximation matrix of $\mathit{Y}$. $\mathit{S}$ is the sparse component. $\left|\right|\cdot |{|}_{\ast }$ is the nuclear norm of a matrix. $\left|\right|\cdot |{|}_1$ is the L1-norm. $\left|\right|\cdot |{|}_{\mathit{F}}$ is the frobenius norm.

Literature [30] has proven that the upper bound of the nuclear norm is:

$${‖\mathit{X}‖}_{\ast }=\underset{\mathit{U},\mathit{V}}{\inf }\left\{{\displaystyle \frac{1}{2}}{‖\mathit{U}‖}_{\mathit{F}}^2+{\displaystyle \frac{1}{2}}{‖\mathit{V}‖}_{\mathit{F}}^2:\mathit{X}=\mathit{U}\mathit{V}\right\}$$

(3)

Here, $\mathit{U}\in {ℝ}^{\mathit{m}\times \mathit{r}}$ and $\mathit{V}\in {ℝ}^{\mathit{r}\times \mathit{n}}$ represent the low-rank decomposition factors of $\mathit{X}$. Thus, the optimization problem can be reformulated as:

$$\underset{\mathit{U},\mathit{V},\mathit{S}}{\min }{\displaystyle \frac{1}{2}}{‖(\mathit{Y}-\mathit{S})-\mathit{U}\mathit{V}‖}_{\mathit{F}}^2+{\displaystyle \frac{1}{2}}\left({‖\mathit{U}‖}_{\mathit{F}}^2+{‖\mathit{V}‖}_{\mathit{F}}^2\right)+\lambda {‖\mathit{S}‖}_1$$

(4)

First, we initialize $\mathit{Y}-{\mathit{S}}_0$ using singular value decomposition (SVD) to obtain the initial factor matrices ${\mathit{U}}_0$ and ${\mathit{V}}_0$, and then employ the gradient descent algorithm to solve the optimization problem (4).

$${\mathit{S}}_{\mathit{k}+1}={\mathcal{S}}_{\lambda }\left(\mathit{Y}-{\mathit{U}}_{\mathit{k}}{\mathit{V}}_{\mathit{k}}\right)$$

(5)

$${\mathit{U}}_{\mathit{k}+1}={\mathit{U}}_{\mathit{k}}-\mu \left({\mathit{U}}_{\mathit{k}}{\mathit{V}}_{\mathit{k}}-(\mathit{Y}-{\mathit{S}}_{\mathit{k}+1})\right){\mathit{U}}_{\mathit{k}}{\left({\mathit{V}}_{\mathit{k}}{\mathit{V}}_{\mathit{k}}^{\mathit{T}}\right)}^{-1}$$

(6)

$${\mathit{V}}_{\mathit{k}+1}={\mathit{V}}_{\mathit{k}}-\mu {\left({\mathit{U}}_{\mathit{k}}{\mathit{V}}_{\mathit{k}}-(\mathit{Y}-{\mathit{S}}_{\mathit{k}+1})\right)}^{\mathit{T}}{\mathit{U}}_{\mathit{k}}{\left({\mathit{U}}_{\mathit{k}}^{\mathit{T}}{\mathit{U}}_{\mathit{k}}\right)}^{-1}$$

(7)

Therefore, we obtain a low-rank approximation matrix $\mathit{X}$ for $\mathit{Y}-\mathit{S}$. However, $\mathit{X}$ is only an approximate low-rank representation of $\mathit{Y}-\mathit{S}$ and is not guaranteed to be optimal. When computing the optimal rank $\mathit{r}$ approximation of $\mathit{Y}-\mathit{S}$, as stated in [31], if

$$\begin{array}{l}\arg \min {‖\left(\mathit{Y}-\mathit{S}\right)-\mathit{X}‖}_{\mathit{F}}^2\\ \mathit{r}\mathit{a}\mathit{n}\mathit{k}\left(\mathit{X}\right)\le \mathit{r}\end{array}$$

(8)

the optimal low-rank approximation of $\mathit{Y}-\mathit{S}$ is given by ${\mathit{X}}^{\ast }={\mathit{U}}_{[:,:\mathit{r}]}{\Sigma }_{[:\mathit{r},:\mathit{r}]}{\mathit{V}}_{[:\mathit{r},:]}$, where $\mathit{Y}-\mathit{S}=\mathit{U}\Sigma \mathit{V}$ represents the singular value decomposition of $\mathit{Y}-\mathit{S}$. However, optimization problem (4) introduces an additional constraint that $\mathit{X}$ must be expressed as a product of two low-rank matrices, i.e., $\mathit{X}=\mathit{U}\mathit{V}$, which is incorporated into optimization problem (8). Our objective is to construct an optimal low-rank approximation of $\mathit{Y}-\mathit{S}$ using factor matrices $\mathit{U}$ and $\mathit{V}$. By analyzing the relationship between the low-rank matrix $\mathit{X}$ and its pseudoinverse, as discussed in [32], we deduce that the optimal low-rank approximation of $\mathit{Y}-\mathit{S}$ under subspace constraints can be expressed as ${\mathit{X}}^{\ast }=\mathit{X}{\mathit{X}}^{†}\left(\mathit{Y}-\mathit{S}\right)$.

Proof: 

Let the column space of $\mathit{X}$ be denoted as $\mathit{R}(\mathit{X})=\left\{\mathit{x}\in {\mathit{C}}^{\mathit{m}}:\mathit{x}=\mathit{X}(\mathit{y}-\mathit{s}),\mathit{y}-\mathit{s}\in {\mathit{C}}^{\mathit{n}}\right\}$, and let $\mathit{X}{\mathit{X}}^{†}$ represent the orthogonal projection operator onto the subspace $\mathit{R}\left(\mathit{X}\right)$. According to [33,34], the orthogonal projection provides the minimal approximation error. As illustrated in Figure 2, if $\mathit{X}{\mathit{X}}^{†}(\mathit{y}-\mathit{s})$ is chosen as the new low-rank approximation of $\mathit{y}-\mathit{s}$, the following condition must hold: $‖(\mathit{y}-\mathit{s})-\mathit{X}{\mathit{X}}^{†}(\mathit{y}-\mathit{s})‖\le ‖(\mathit{y}-\mathit{s})-\mathit{x}‖$. Finally, we obtain the optimal low-rank approximation vector for $\mathit{y}-\mathit{s}$ in $\mathit{R}(\mathit{X})$. Therefore, assume that $\mathit{X}$ is a low-rank approximation of $\mathit{Y}-\mathit{S}$, and ${\mathit{X}}^{\ast }=\mathit{X}{\mathit{X}}^{†}(\mathit{Y}-\mathit{S})$ is another low-rank approximation of $\mathit{Y}-\mathit{S}$. Let $\mathit{Q}={\mathit{X}}^{†}(\mathit{Y}-\mathit{S})$, when $\mathit{Q}=\mathit{I}$, obviously ${‖(\mathit{Y}-\mathit{S})-{\mathit{X}}^{\ast }‖}_{\mathit{F}}^2={‖(\mathit{Y}-\mathit{S})-\mathit{X}‖}_{\mathit{F}}^2$. When $\mathit{Q}\ne \mathit{I}$, since $\mathit{X}{\mathit{X}}^{†}$ is an orthogonal projection operator, according to the minimum orthogonal projection distance, we have ${‖(\mathit{Y}-\mathit{S})-{\mathit{X}}^{\ast }‖}_{\mathit{F}}^2={‖(\mathit{Y}-\mathit{S})-\mathit{X}{\mathit{X}}^{†}(\mathit{Y}-\mathit{S})‖}_{\mathit{F}}^2<{‖(\mathit{Y}-\mathit{S})-\mathit{X}‖}_{\mathit{F}}^2$. In summary, ${\mathit{X}}^{\ast }=\mathit{X}{\mathit{X}}^{†}\left(\mathit{Y}-\mathit{S}\right)$ is the optimal low-rank approximation of $\mathit{Y}-\mathit{S}$.□

Figure 2. Illustration of the subspace projection.

According to reference [35], the pseudoinverse of a low-rank matrix $\mathit{X}$ can be expressed in terms of its low-rank factor matrices $\mathit{U}$ and $\mathit{V}$ as follows:

$${\mathit{X}}^{†}={\mathit{V}}^{\mathit{T}}{({\mathit{U}}^{\mathit{T}}\mathit{X}{\mathit{V}}^{\mathit{T}})}^{-1}{\mathit{U}}^{\mathit{T}}={\mathit{V}}^{\mathit{T}}{\left(\mathit{V}{\mathit{V}}^{\mathit{T}}\right)}^{-1}{\left({\mathit{U}}^{\mathit{T}}\mathit{U}\right)}^{-1}{\mathit{U}}^{\mathit{T}}$$

(9)

Therefore, the optimal low-rank approximation of matrix $\mathit{Y}-\mathit{S}$ can be expressed as:

$${\mathit{X}}^{\ast }=\mathit{X}{\mathit{X}}^{†}\left(\mathit{Y}-\mathit{S}\right)=\mathit{U}{\left({\mathit{U}}^{\mathit{T}}\mathit{U}\right)}^{-1}{\mathit{U}}^{\mathit{T}}\left(\mathit{Y}-\mathit{S}\right)$$

(10)

Since $\mathit{X}=\mathit{U}\mathit{V}$, it follows from the above equation that the optimal low-rank approximation can be expressed solely in terms of the matrices $\mathit{U}$ and $\mathit{Y}-\mathit{S}$.

3.2. Infrared Small Target Detection Model

Let $\mathcal{Y}\in {ℝ}^{\mathit{m}\times \mathit{n}\times \mathit{t}}$ denote the video tensor. By vectorizing each video frame and stacking them along the temporal dimension, we obtain a matrix $\mathit{Y}\in {ℝ}^{\mathit{m}\mathit{n}\times \mathit{t}}$. Given the inherently low-rank nature of the video background and the sparsity of small moving objects, the task of small object detection in multiframe infrared images can be formulated as the following optimization problem:

$$\underset{\mathit{U},\mathit{V},\mathit{S}}{\min }{\displaystyle \frac{1}{2}}{‖\mathit{Y}-\mathit{U}\mathit{V}-\mathit{S}‖}_{\mathit{F}}^2+\lambda {‖\mathit{S}‖}_{\mathit{w},1}$$

(11)

$${||\mathit{S}||}_{\mathit{w},1}={\displaystyle {\sum }_{{\mathit{n}}_{\mathit{i}}}\log \left(\left|{\mathit{S}}_{{\mathit{n}}_{\mathit{i}}}\right|+\epsilon \right)}{\mathit{n}}_{\mathit{i}}\epsilon$$

(12)

where $\mathit{Y},\mathit{S}\in {ℝ}^{\mathit{m}\mathit{n}\times \mathit{t}}$, $\mathit{U}\in {ℝ}^{\mathit{m}\times \mathit{r}},\mathit{V}\in {ℝ}^{\mathit{r}\times \mathit{n}}$. $\mathit{S}$ is the sparse component. $\mathit{m}$ and $\mathit{n}$ are the dimensions of a video frame, and $\mathit{t}$ is the number of video frames. $\left|\right|\cdot |{|}_{\mathit{w},1}$ refers to the reweighted ${\mathit{l}}_1$ norm, which is an improvement on the ${\mathit{l}}_1$ norm and can better highlight sparse small targets. $\epsilon$ is a constant. ${\mathit{n}}_{\mathit{i}}$ is the number of all the entries in $\mathit{S}$, and $\mathit{w}=\left(1/\left(\mathit{S}+\epsilon \right)\right)$.

3.3. Model Solving

To solve optimization problem (11), we propose a novel optimization algorithm based on an improved low-rank approximation combined with robust CUR decomposition. This approach significantly reduces the computational complexity of the iterative process and accelerates the convergence of the model. The data flow diagram of the entire algorithm is illustrated in Figure 3. Specifically, we denote the column submatrix of matrix $\mathit{Y}$ as $\mathit{Y}={\mathit{Y}}_{\mathcal{C}}$, and the row submatrix as $\mathit{Y}={\mathit{Y}}_{\mathcal{R}}$. First, uniform random sampling is performed on the input matrix $\mathit{Y}$ to obtain the column submatrix ${\mathit{Y}}_{\mathcal{C}}=\mathit{Y}(:,\mathcal{J})$ and the row submatrix ${\mathit{Y}}_{\mathcal{R}}=\mathit{Y}(\mathcal{I},:)$. Subsequently, alternating iterations of low-rank and sparse component decomposition are carried out on the column submatrix ${\mathit{Y}}_{\mathcal{C}}$ and the row submatrix ${\mathit{Y}}_{\mathcal{R}}$, respectively, resulting in the optimal low-rank component ${\mathit{X}}_{\mathcal{C}}$ and sparse component ${\mathit{S}}_{\mathcal{C}}$ for the column submatrix, and the optimal low-rank component ${\mathit{X}}_{\mathcal{R}}$ and sparse component ${\mathit{S}}_{\mathcal{R}}$ for the row submatrix. Finally, the optimal low-rank component of matrix $\mathit{Y}$ is obtained as $\mathit{X}={\mathit{X}}_{\mathcal{C}}{\left({\mathit{X}}_{\mathcal{C}}\left(\mathcal{I},:\right)\right)}^{†}{\mathit{X}}_{\mathcal{R}}$.

Figure 3. Flowchart of the proposed algorithm.

The process described above at the n-th iteration can be expressed as follows:

$${\mathit{X}}_{\mathcal{C}}=\underset{{\mathit{X}}_{\mathcal{C}}}{\min }{\displaystyle \frac{1}{2}}{‖\mathit{Y}(:,\mathcal{J})-{\mathit{U}}_{\mathcal{C}}{\mathit{V}}_{\mathcal{C}}-{\mathit{S}}_{\mathcal{C}}‖}_{\mathit{F}}^2$$

(13)

$${\mathit{X}}_{\mathcal{R}}=\underset{{\mathit{X}}_{\mathcal{R}}}{\min }{\displaystyle \frac{1}{2}}{‖\mathit{Y}(\mathcal{I},:)-{\mathit{U}}_{\mathcal{R}}{\mathit{V}}_{\mathcal{R}}-{\mathit{S}}_{\mathcal{R}}‖}_{\mathit{F}}^2$$

(14)

$${\mathit{S}}_{\mathcal{C}}=\underset{{\mathit{S}}_{\mathcal{C}}}{\min }{\displaystyle \frac{1}{2}}{‖\mathit{Y}(:,\mathcal{J})-{\mathit{X}}_{\mathcal{C}}-{\mathit{S}}_{\mathcal{C}}‖}_{\mathit{F}}^2+\lambda {‖{\mathit{S}}_{\mathcal{C}}‖}_{\mathit{w},1}$$

(15)

$${\mathit{S}}_{\mathcal{R}}=\underset{{\mathit{S}}_{\mathcal{R}}}{\min }{\displaystyle \frac{1}{2}}{‖\mathit{Y}(\mathcal{I},:)-{\mathit{X}}_{\mathcal{R}}-{\mathit{S}}_{\mathcal{R}}‖}_{\mathit{F}}^2+\lambda {‖{\mathit{S}}_{\mathcal{R}}‖}_{\mathit{w},1}$$

(16)

(1)

Update the low-rank component $\mathit{X}$:

As discussed in Section 3.1, since the low-rank components depend solely on the basis matrix $\mathit{U}$ and the observation matrix $\mathit{Y}$, the explicit computation of the projection operator $\mathit{X}{\mathit{X}}^{†}$ is avoided. This significantly reduces the computational burden of the optimization process.

$${\mathit{X}}_{\mathcal{C}}={\mathit{U}}_{\mathcal{C}}{\left({\mathit{U}}_{\mathcal{C}}{\mathit{U}}_{\mathcal{C}}^{\mathit{T}}\right)}^{-1}{\mathit{U}}_{\mathcal{C}}^{\mathit{T}}\left({\mathit{Y}}_{\mathcal{C}}-{\mathit{S}}_{\mathcal{C}}\right)$$

(17)

$${\mathit{X}}_{\mathcal{R}}={\mathit{U}}_{\mathcal{R}}{\left({\mathit{U}}_{\mathcal{R}}{\mathit{U}}_{\mathcal{R}}^{\mathit{T}}\right)}^{-1}{\mathit{U}}_{\mathcal{R}}^{\mathit{T}}\left(\mathit{Y}(\mathcal{I},:)-{\mathit{S}}_{\mathcal{R}}\right)$$

(18)

(2)

Update sparse component $\mathit{S}$:

$${\mathit{S}}_{\mathcal{C}}={\mathcal{S}}_{\mathit{w},\lambda }\left(\mathit{Y}(:,\mathcal{J})-{\mathit{X}}_{\mathcal{C}}\right)$$

(19)

$${\mathit{S}}_{\mathcal{R}}={\mathcal{S}}_{\mathit{w},\lambda }\left(\mathit{Y}(\mathcal{I},:)-{\mathit{X}}_{\mathcal{R}}\right){\mathcal{S}}_{\mathit{w},\lambda }\left(\cdot \right)$$

(20)

where the proximal operator ${\mathcal{S}}_{\mathit{w},\lambda }\left(\cdot \right)$ employs the reweighted soft thresholding oper-ator proposed in [36].

Finally, we update the low-rank and sparse components of the entire input matrix as follows:

$$\mathit{X}={\mathit{X}}_{\mathcal{C}}{\left({\mathit{X}}_{\mathcal{C}}\left(\mathcal{I},:\right)\right)}^{†}{\mathit{X}}_{\mathcal{R}}$$

(21)

$$\mathit{S}=\mathrm{\Gamma }\left(\mathit{Y}-\mathit{X}\right)$$

(22)

The complete solution procedure is summarized in Algorithm 1.

Algorithm 1: Iterative Optimization for Minimizing (11)

1: Input: $\mathit{Y}$: input video; $r$: rank; maximum iterations $K$; $\mathcal{I},\mathcal{J}$: sampling number of rows and columns.

2: Initialize: ${\mathit{S}}_0={\mathcal{S}}_{\lambda }\left(\mathit{Y}\right)$, $\left[L_0,\sum ,{\mathit{R}}_0\right]=\mathit{S}\mathit{V}{\mathit{D}}_{\mathit{r}}\left(\mathit{Y}-{\mathit{S}}_0\right)$,${\mathit{U}}_0=L_0\sqrt{\sum }$,${\mathit{V}}_0=\sqrt{\sum }{\mathit{R}}_0$. Uniformly sample row indices $\mathcal{I}$ and column indices $\mathcal{J}$.

3: while $k\le K$, or stopping criteria not fulfilled do

4:  Optimizing ${\mathit{X}}_{\mathcal{C}}$ with (17);

5:  Optimizing ${\mathit{X}}_{\mathcal{R}}$ with (18);

6:  Optimizing ${\mathit{S}}_{\mathcal{C}}$ with (19);

7:  Optimizing ${\mathit{S}}_{\mathcal{R}}$ with (20);

8:  Optimizing $\mathit{X}$ with (21);

9:  Optimizing $\mathit{S}$ with (22);

10:   $k\leftarrow k+1$

11: end while

Output: $\mathit{X},\mathit{S}$

4. Experimental Results and Discussion

To evaluate the effectiveness of the proposed method, we conducted comparative experiments on two publicly available infrared video datasets: Drone Detection [37] and NUDT-MIRSDT [38]. The competing methods included IPI [6], FGLR-MCP [39], GST [35], ECA-STT [17], NRAM [40], and SRWS [41]. Considering the limitations of traditional 2D receiver operating characteristic (2D ROC) curves in representing detection performance, we adopted 3D receiver operating characteristic (3D ROC) curves and their derived eight quantitative metrics as evaluation criteria. The comparative experiment is mainly divided into four parts: visual comparison, numerical result comparison, running time, and ablation study.

4.1. Visualization Comparisons

To visually compare the detection performance of different methods, Figure 3 and Figure 4 present qualitative visualizations of the proposed method alongside other competing approaches. The five selected infrared video sequences encompass representative challenging scenarios, including forests, clouds, and urban building environments. In these visualizations, accurately detected targets are marked with red solid lines, while false detections are indicated by blue dashed lines. The corresponding detection method is annotated in each image for clarity.

Figure 4. Detection results on three sequences from the Drone Detection video dataset.

As shown in Figure 4, the proposed method is compared with six other approaches in terms of small target detection performance on the Drone Detection infrared video dataset. Three representative challenging scenarios with complex jungle and cloud backgrounds are selected, where the scenes contain numerous background structures that resemble target components. In these environments, competing methods exhibit varying degrees of false positives and false negatives, especially when the jungle background is heavily cluttered with noise, which severely impairs the accurate detection of small infrared targets. Specifically, IPI is capable of enhancing target signals; however, it fails to suppress background noise and clutter, resulting in residual noise and a generally grayish background in the detection outputs. ECA-STT tends to misclassify background interference as targets, leading to a high incidence of false alarms, indicating its limited ability to suppress background clutter. GST, NRAM, and SRWS demonstrate better performance in suppressing most background interference; however, they also tend to miss certain small targets, resulting in false negatives. In contrast, both FGLR-MCP and the proposed method exhibit superior performance by effectively suppressing background interference while accurately detecting sparse small targets, achieving a better balance between false positives and false negatives.

Figure 5 presents the infrared small target detection results of the proposed method in comparison with other methods on the NUDT-MIRSDT dataset. In Sequence 4, bright reflections from building rooftops and other non-target background structures exhibit characteristics similar to actual infrared small targets, posing significant challenges for accurate detection. Sequence 5 contains a substantial amount of background noise, where true targets are obscured by clutter, making it extremely difficult to distinguish small targets from background interference. These two sequences are therefore well-suited for evaluating the robustness of infrared small target detection methods in suppressing background interference while highlighting true targets. Experimental results demonstrate that in Sequence 4, methods such as IPI, GST, ECA-STT, NRAM, and SRWS tend to misclassify building rooftops—which resemble small targets—as actual targets, resulting in false positives. In contrast, both FGLR-MCP and the proposed method successfully detect the true small target while avoiding the misclassification of background interference. Notably, the proposed method preserves more complete target details compared to FGLR-MCP. In Sequence 5, where the background noise is particularly severe, IPI, GST, ECA-STT, NRAM, and SRWS fail to suppress the interference, mistakenly identifying background clutter as targets. The proposed method, however, significantly reduces background false alarms and achieves the most accurate detection performance among all methods.

Figure 5. Detection results on two sequences from the NUDT-MIRSDT video dataset.

4.2. Quantitative Comparison Results with Other Methods

We first plotted the 3D ROC curves for the proposed method and the comparative methods on five infrared image sequences, as shown in Figure 6. In the context of infrared small target detection, a $(P_D,P_F)$ curve that approaches the upper-left corner of the coordinate axes indicates superior overall detection performance. Similarly, a $(P_D,\tau )$ curve closer to the top-right corner signifies stronger target detection capability, while a $(P_F,\tau )$ curve approaching the bottom-left corner reflects better background interference suppression. As illustrated in Figure 6, the $(P_F,\tau )$ curve of IPI remains distant from the bottom-left corner, indicating poor background suppression capability. In contrast, the $(P_F,\tau )$ curves of GST and NRAM are positioned near the bottom-left corner, demonstrating their effectiveness in mitigating background noise. However, their $(P_D,\tau )$ curves are noticeably far from the top-right corner, revealing suboptimal object detection performance. ECA-STT, FGLR-MCP, and the proposed method exhibit $(P_D,\tau )$ curves that are clustered near the top-right corner, signifying excellent object detection capabilities. Notably, compared to ECA-STT and FGLR-MCP, the proposed method’s $(P_F,\tau )$ curve is positioned closer to the bottom-left corner, indicating superior background suppression performance. Consequently, the proposed method’s $(P_D,P_F)$ curve lies near the upper-left corner of the coordinate axes, reflecting its overall outstanding detection performance. These results demonstrate that the proposed method achieves a favorable balance of robust background suppression and strong target detection capabilities, outperforming the comparison methods.

Figure 6. The 3DROC curves and its corresponding 2DROC curves for the different methods.

For the 3D ROC evaluation, the average AUC values over all infrared image sequences are reported in Table 1. In Table 1, the best results are highlighted in bold, while the second-best results are underlined. The single-frame-based methods, such as IPI, NRAM, and SRWS, achieve lower AUC_TD and AUC_BS values than the multi-frame-based methods. This observation indicates that single-frame approaches generally exhibit limited detection performance when applied to multi-frame infrared sequences. Among the multi-frame-based methods, FGLR-MCP attains the highest AUC_TD values on multiple sequences; however, its AUC_BS values are relatively moderate. As a result, the overall performance of FGLR-MCP is suboptimal. In contrast, the proposed method outperforms the compared methods on most AUC metrics, demonstrating its superior capability in terms of target detectability and background suppression.

Table 1. 3D ROC-derived AUC metric.

	Methods	AUC(D,F)	${\mathbf{AUC}}_{(\mathbf{D},\mathit{\tau }}$)	${\mathbf{AUC}}_{(\mathbf{F},\mathit{\tau }}$)	AUCTD	AUCBS	AUCSNPR	AUCTD-BS	AUCODP
Seq.1	IPI	0.9987	0.7617	0.3562	1.7604	0.6425	2.1385	0.4055	1.4042
	FGLR-MCP	1.0000	0.8775	0.0024	1.8775	0.9976	368.3908	0.8752	1.8752
	GST	0.9326	0.0963	0.0063	1.0288	0.9263	15.3728	0.0899	1.0226
	ECA-STT	0.9999	0.8506	0.0280	1.8505	0.9719	30.3574	0.8226	1.8225
	NRAM	0.6805	0.2665	0.0024	0.9470	0.6784	125.4999	0.2644	0.9449
	SRWS	0.9999	0.5230	0.0021	1.5230	0.9980	262.9373	0.5210	1.5210
	Proposed	1.0000	0.8658	0.0023	1.8658	0.9977	375.9842	0.8635	1.8635
Seq.2	IPI	0.7487	0.6843	0.4381	1.4330	0.3106	1.5620	0.2462	0.9949
	FGLR-MCP	1.0000	0.8765	0.0353	1.8765	0.9647	24.8225	0.8412	1.8412
	GST	0.9838	0.0980	0.0065	1.0819	0.9773	15.1391	0.0916	1.0754
	ECA-STT	0.9999	0.8078	0.0090	1.8078	0.9909	89.6249	0.7988	1.7987
	NRAM	0.7471	0.1069	0.0032	0.8540	0.7439	33.0221	0.1036	0.8507
	SRWS	0.4993	0.0020	0.0022	0.5012	0.4970	0.8733	−0.0003	0.4990
	Proposed	1.0000	0.8896	0.0021	1.8896	0.9979	416.4358	0.8874	1.8874
Seq.3	IPI	0.9989	0.8412	0.2668	1.8401	0.7321	3.1532	0.5744	1.5733
	FGLR-MCP	1.0000	0.8695	0.0024	1.8695	0.9976	360.5366	0.8671	1.8671
	GST	0.9966	0.3580	0.0096	1.3546	0.9870	37.2852	0.3484	1.3450
	ECA-STT	0.9999	0.8067	0.0094	1.8067	0.9906	85.9654	0.7973	1.7973
	NRAM	0.5663	0.0031	0.0035	0.5694	0.5628	0.8781	−0.0004	0.5659
	SRWS	1.0000	0.7247	0.0198	1.7246	0.9802	36.5940	0.7048	1.7048
	Proposed	1.0000	0.7958	0.0021	1.7958	0.9979	387.7901	0.7937	1.7937
Seq.4	IPI	0.9466	0.5015	0.3869	1.4481	0.5597	1.2963	0.1146	1.0613
	FGLR-MCP	0.9875	0.7081	0.5094	1.6959	0.4781	1.3900	0.1987	1.1862
	GST	0.8536	0.0401	0.0048	0.8937	0.8488	8.4085	0.0353	0.8889
	ECA-STT	0.6717	0.2379	0.0041	0.9096	0.6676	57.6524	0.2338	0.9055
	NRAM	0.6946	0.2185	0.0050	0.9131	0.6895	43.3089	0.2135	0.9081
	SRWS	0.9198	0.0906	0.0028	1.0105	0.9171	32.6913	0.0879	1.0077
	Proposed	1.0000	0.6876	0.0569	1.6876	0.9431	12.0926	0.6308	1.6308
Seq.5	IPI	0.6234	0.1560	0.1279	0.7794	0.4956	1.2196	0.0281	0.6515
	FGLR-MCP	0.8750	0.6703	0.0976	1.5453	0.7774	6.8703	0.5727	1.4477
	GST	0.7929	0.1072	0.0068	0.9000	0.7861	15.7977	0.1004	0.8932
	ECA-STT	0.9228	0.4379	0.0056	1.3607	0.9171	78.0696	0.4323	1.3551
	NRAM	0.6065	0.0683	0.0090	0.6748	0.5975	7.5874	0.0593	0.6658
	SRWS	0.9145	0.2608	0.0178	1.1753	0.8967	14.6772	0.2430	1.1575
	Proposed	1.0000	0.5276	0.0176	1.5276	0.9824	29.9000	0.5100	1.5100

4.3. Running Time

The computational efficiency of a detection algorithm is just as important as its detection performance. Therefore, we compared the computational efficiency of the proposed method with several existing methods across five video sequences. The frame sizes are 128 × 128 for Sequences 1, 2, and 3; 379 × 246 for Sequence 4; and 328 × 220 for Sequence 5. The time required by each method to process a single frame is presented in Table 2. It can be observed that as frame size increases, the processing time of other methods grows significantly. In contrast, the proposed method demonstrates a more modest increase in processing time, owing to the row and column sampling strategy applied to the input matrix. Methods based on low-rank sparse decomposition typically suffer from slow inference due to iterative optimization processes involving SVD computations. In comparison, both FGLR-MCP and the proposed method achieve significantly faster detection speeds, primarily by avoiding such SVD operations. Specifically, the proposed method benefits from the use of a robust CUR decomposition in the optimization process, further enhancing its speed. As shown in Table 3, the proposed method achieves the highest FPS among all methods, indicating its ability to process more video frames within the same period. Thus, the proposed method provides valuable insight into improving the computational efficiency of low-rank sparse decomposition-based approaches.

Table 2. The runtime per frame for all methods in Sequence 1–5.

Methods	Seq 1	Seq 2	Seq 3	Seq 4	Seq 5
IPI	0.7806	0.5201	0.6329	36.6403	13.8224
FGLR-MCP	0.0248	0.0257	0.0296	0.1639	0.1247
GST	0.2355	0.1743	0.2316	0.3064	0.2706
ECA-STT	2.4590	2.3477	2.4618	4.0009	3.4819
NRAM	0.3712	0.3772	0.4354	7.8124	5.5857
SRWS	0.1833	0.1628	0.1768	5.3317	5.4786
Proposed	0.0177	0.0162	0.0135	0.0435	0.0423

Table 3. The FPS values of different methods on sequences 1–5.

Methods	Seq 1	Seq 2	Seq 3	Seq 4	Seq 5
IPI	1.2811	1.9227	1.5800	0.0273	0.0723
FGLR-MCP	40.3226	38.9105	33.7838	6.1013	8.0192
GST	4.2463	5.7372	4.3178	3.2637	3.6955
ECA-STT	0.4067	0.4259	0.4062	0.2499	0.2872
NRAM	2.6940	2.6511	2.2967	0.1280	0.1790
SRWS	5.4555	6.1425	5.6561	0.1876	0.1825
Proposed	56.4972	61.7284	74.0741	22.9885	23.6407

4.4. Ablation Study

To verify the effectiveness of each component in the proposed method, ablation experiments were conducted on infrared video sequence 1. The ablation study comparatively evaluates the infrared small target detection performance of matrix factorization-based low-rank approximation, the proposed low-rank approximation method, the conventional L1 norm, and the reweighted L1 norm. The experimental results are presented in Table 4. In the ablation configurations, “LRMF” denotes the low-rank approximation based on nonnegative matrix factorization, while “NLRA” represents the proposed low-rank approximation method. The “L1” configuration applies the conventional L1 norm for small target detection, whereas “RL1” adopts the reweighted L1 norm. As shown in Table 4, the reweighted L1 norm is more effective than the conventional L1 norm in highlighting small targets while suppressing sparse non-target points. In addition, the improved low-rank approximation method achieves more accurate background reconstruction than the nonnegative matrix factorization-based low-rank approximation and significantly enhances the suppression of background interference.

Table 4. Detection results of different configurations.

LRMF	NLRA	L1	RL1	$\mathit{A}\mathit{U}{\mathit{C}}_{(\mathit{D},\mathit{F})}$	$\mathit{A}\mathit{U}{\mathit{C}}_{(\mathit{D},\tau )}$	$\mathit{A}\mathit{U}{\mathit{C}}_{(\mathit{F},\tau )}$	$\mathit{A}\mathit{U}{\mathit{C}}_{\mathit{T}\mathit{D}}$	$\mathit{A}\mathit{U}{\mathit{C}}_{\mathit{B}\mathit{S}}$	$\mathit{A}\mathit{U}{\mathit{C}}_{\mathit{S}\mathit{N}\mathit{P}\mathit{R}}$	$\mathit{A}\mathit{U}{\mathit{C}}_{\mathit{T}\mathit{D}\mathit{B}\mathit{S}}$	$\mathit{A}\mathit{U}{\mathit{C}}_{\mathit{O}\mathit{D}\mathit{P}}$
✓		✓		0.8863	0.3170	0.0025	1.2032	0.8838	127.5293	0.3145	1.2008
	✓	✓		0.9999	0.5154	0.0024	1.5154	0.9975	209.7335	0.5129	1.5129
✓			✓	0.9999	0.5294	0.0023	1.5294	0.9977	234.5522	0.5271	1.5271
	✓		✓	1.0000	0.8658	0.0023	1.8658	0.9977	375.9842	0.8635	1.8635

Note: "✓" indicates the selected configuration.

4.5. Parameter Analyses

We follow the row and column sampling strategy described in [42]. Accordingly, for the index $\mathit{I}\subseteq [\mathit{m}\mathit{n}],\mathit{J}\subseteq [\mathit{t}]$, our method sets the number of sampled rows to $\left|\mathit{I}\right|={\mathit{c}}_{\mathit{I}}\mathit{r}\log (\mathit{m}\mathit{n})$ and the number of sampled columns to $\left|\mathit{J}\right|={\mathit{c}}_{\mathit{J}}\mathit{r}\log (\mathit{t})$. Based on the singular value distribution of the infrared video shown in Figure 7, the rank is set to $\mathit{r}=10$. Constants ${\mathit{c}}_{\mathit{I}}\in [3,5]$, ${\mathit{c}}_{\mathit{J}}\in [0.1,0.5]$. Therefore, we set ${\mathit{c}}_{\mathit{I}}=4$ and ${\mathit{c}}_{\mathit{J}}=0.3$. Additionally, the regularization coefficient $\lambda$ controls the balance between background and target, and we set $\lambda$ to 0.15 in our experiments.

Figure 7. Singular value distribution curves of infrared image tensor along each mode.

4.6. Complexity Analyses

By employing robust CUR decomposition, the proposed algorithm achieves lower computational complexity than traditional nonnegative matrix factorization-based approximation methods. In conventional NMF-based methods, the computational cost required to update the low-rank component $\mathit{X}$ in each iteration is $o(\mathit{r}\mathit{m}\mathit{n}\mathit{t})$, while updating the sparse component $\mathit{S}$ requires $o(\mathit{m}\mathit{n}\mathit{t})$. In contrast, for the proposed method, the computational cost of updating $\mathit{X}$ in each iteration is $o(\mathit{r}|\mathit{I}|\mathit{t})+o(\mathit{r}\mathit{m}\mathit{n}|\mathit{J}|)$, and the cost of updating $\mathit{S}$ is $o(|\mathit{I}|\mathit{t})+o(\mathit{m}\mathit{n}|\mathit{J}|)$. Since $\left|\mathit{I}\right|$ and $\left|\mathit{J}\right|$ are much smaller than $\mathit{m}\mathit{n}$ and $\mathit{t}$, i.e., $\left|\mathit{I}\right|,\left|\mathit{J}\right|\ll \min (\mathit{m}\mathit{n},\mathit{t})$, the proposed method significantly reduces the computational cost during the iterative optimization process.

4.7. Convergence Analyses

To validate the effectiveness of the proposed low-rank approximation method, we conducted a comparative experiment on video sequence 3. Specifically, we defined the approximation error obtained by traditional low-rank matrix decomposition as ${\mathit{X}}_{\mathit{e}}$, and the approximation error of the proposed method as ${\mathit{X}}_{\mathit{e}}^{\ast }$, where the errors are computed as follows:

$${\mathit{X}}_{\mathit{e}}={‖\mathit{U}\mathit{V}-{\mathit{X}}_{\mathit{G}\mathit{T}}‖}_{\mathit{F}}^2/{‖{\mathit{X}}_{\mathit{G}\mathit{T}}‖}_{\mathit{F}}^2$$

(23)

$${\mathit{X}}_{\mathit{e}}^{\ast }={‖\mathit{U}{\left({\mathit{U}}^{\mathit{T}}\mathit{U}\right)}^{-1}{\mathit{U}}^{\mathit{T}}\left(\mathit{Y}-\mathit{S}\right)-{\mathit{X}}_{\mathit{G}\mathit{T}}‖}_{\mathit{F}}^2/{‖{\mathit{X}}_{\mathit{G}\mathit{T}}‖}_{\mathit{F}}^2$$

(24)

Here, ${\mathit{X}}_{\mathit{G}\mathit{T}}$ denotes the ground-truth background of the infrared video.

The relationship between the iterative error and the number of iterations for both traditional low-rank matrix decomposition and the proposed optimal low-rank approximation is illustrated in Figure 8. As shown in Figure 8, the error curve of the proposed low-rank approximation consistently lies below that of the traditional low-rank matrix decomposition method. This indicates that, during the iterative process, our method remains closer to the ground-truth low-rank background component at each iteration. Consequently, the proposed approach facilitates a reduction in the number of iterations, accelerates convergence, and achieves more accurate background reconstruction.

Figure 8. Error curve of the algorithm.

5. Discussion

Although the proposed method is effective for the fast detection of small targets in high-dimensional infrared video sequences, it still has certain limitations. Specifically, the detection performance of the proposed method becomes less satisfactory in scenarios where the infrared video background changes rapidly. This is mainly because rapidly varying backgrounds often violate the low-rank assumption of the background model. In future work, we will focus on developing more accurate models to better handle rapidly changing background scenes.

6. Conclusions

In this paper, we combine an improved low-rank approximation with robust CUR decomposition to achieve fast and accurate small object detection in infrared videos. The improved low-rank approximation enables more accurate recovery of low-rank backgrounds, thereby enhancing the accuracy of small object detection in multiframe infrared images. Robust CUR decomposition efficiently handles high-dimensional video sequences, improving the speed of detection. Additionally, we employ the reweighted ${\mathit{l}}_1$ norm to further distinguish between sparse background noise and small objects, thereby reducing false positives. As a result, the proposed method outperforms other comparative methods in terms of both efficiency and accuracy for multiframe infrared small object detection. Furthermore, as the dimensionality of video data increases, the proposed method requires less runtime to process videos compared to other methods, making it highly suitable for infrared small object detection in high-dimensional videos.