UIMM-Tracker: IMM-Based with Uncertainty Detection for Video Satellite Infrared Small-Target Tracking

Abstract

Infrared video satellites have the characteristics of wide-area long-duration surveillance, enabling continuous operation day and night compared to visible light imaging methods. Therefore, they are widely used for continuous monitoring and tracking of important targets. However, energy attenuation caused by long-distance radiation transmission reduces imaging contrast and leads to the loss of edge contours and texture details, posing significant challenges to target tracking algorithm design. This paper proposes an infrared small-target tracking method, the UIMM-Tracker, based on the tracking-by-detection (TbD) paradigm. First, detection uncertainty is measured and injected into the multi-model observation noise, transferring the distribution knowledge of the detection process to the tracking process. Second, a dynamic modulation mechanism is introduced into the Markov transition process of multi-model fusion, enabling the tracking model to autonomously adapt to targets with varying maneuvering states. Additionally, detection uncertainty is incorporated into the data association method, and a distance cost matrix between trajectories and detections is constructed based on scale and energy invariance assumptions, improving tracking accuracy. Finally, the proposed method achieves average performance scores of 68.5%, 45.6%, 56.2%, and 0.41 in IDF1, MOTA, HOTA, and precision metrics, respectively, across 20 challenging sequences, outperforming classical methods and demonstrating its effectiveness.

1. Introduction

With the continuous development of new moving targets such as missiles, unmanned aerial vehicles (UAVs), and aircraft [1], the real-time tracking of targets to form continuous trajectories has become a primary requirement. Existing ground-based and air-based imaging methods can only operate within local spatio-temporal ranges [2], unable to meet the task requirements for long-duration continuous tracking. Space-based remote-sensing satellites, characterized by wide field of view and high timeliness, have emerged as the main direction for future moving target detection [3,4,5]. However, targets in remote-sensing images exhibit sparse characteristics [6], with most areas occupied by scene clutter. The targets share similar color and texture characteristics with cloud clutter and scene structural clutter [7], rendering visible light imaging unsuitable for target tracking tasks. Infrared imaging, compared to visible light, possesses powerful night imaging capabilities [8] and is widely used in characteristic measurement [9], disaster prediction [10], target detection [11], and target tracking [12] fields. Therefore, utilizing space-based infrared video satellites for tracking moving targets is a feasible technical approach, offering advantages such as sustainability, high robustness, and low interference. However, infrared small-target tracking still faces the following technical challenges: (1) Uncertainty in detection results: due to the complexity of space-based infrared scenes and the low energy and lack of morphological and textural features of small targets, there is a deviation between detection results and ground truth [13], introducing inherent errors caused by the detection algorithm that are difficult for tracking algorithms to eliminate (as shown in Figure 1a). (2) Trajectory interference: the spatial resolution of space-based infrared remote-sensing satellites is relatively low, causing targets and clutter that are close in spatial distance to couple, making it difficult to effectively distinguish target trajectories (as shown in Figure 1b). (3) Association difficulties: infrared small targets occupy only a few to tens of pixels in remote-sensing images, rendering traditional intersection over union (IoU) matching methods unsuitable for extremely small-scale targets (as shown in Figure 1c).

To address the aforementioned issues, numerous scholars have conducted corresponding research, the focus of which can be broadly classified into traditional methods [12] and deep-learning methods [14]. Traditional methods take detection results as input and combine data association algorithms to label targets with the same identity. These methods mainly include tracking based on target assignment (the Hungarian algorithm [15], linear assignment [16], and bipartite graph matching [17]), tracking based on probabilistic models (JPDA [18] and MHT [19]), tracking based on filters (the Kalman filter [20] and particle filter [21]), tracking based on graphical models (Bayesian estimation [22], random finite sets [23], minimum cost flow [24], and conditional random fields [25]), tracking based on trajectory modeling (global data association [26]), and tracking based on distributed frameworks (IMM [27]). Tracking based on target assignment has the advantages of low computational complexity and strong real-time performance, but its performance highly depends on the accuracy of the detector, and association errors are prone to occur in scenarios with target occlusion or a low signal-to-noise ratio. Tracking based on probabilistic models enhances tracking robustness through multi-hypothesis modeling but faces the problem of exponential growth in computational complexity, making it difficult to meet real-time requirements. Tracking based on filters effectively handles target motion uncertainty through state prediction but lacks adaptability to nonlinear/non-Gaussian noise. Tracking based on graphical models can achieve a global optimal association, but the modeling complexity is high, and parameter sensitivity is strong. Tracking based on trajectory modeling is suitable for dense target scenarios but requires a large amount of historical trajectory data for support. Tracking based on distributed frameworks addresses target maneuverability through multi-model interaction but has a strong dependence on prior motion models.

Traditional methods primarily address three components: target feature extraction, state prediction, and data association. Feature extraction aims to leverage the morphological and structural uniqueness of targets to extract their appearance features, which need to differentiate from other targets’ features and remain temporally invariant. For example, Wang et al. filter low-quality feature embeddings through a trajectory selection strategy and transition feature states based on an adaptive exponential moving average method [28]. State prediction utilizes the motion characteristics of targets to construct a state transition model, enabling the acquisition of target states such as position and velocity at future time steps. State prediction can be used in conjunction with appearance information, such as in optical flow methods [29], or independently, such as using LSTM to mine historical information to predict the current state [30]. In recent years, some novel state prediction methods have emerged, such as utilizing the Mamba architecture to predict target positions [31]. Data association algorithms address the matching problem between trajectories and detections, primarily implemented through cost computation based on appearance [32] or motion [33] features, combined with classical methods like the Hungarian algorithm for target association. Appearance and motion features are obtained during the feature extraction phase. Deep-learning methods rely heavily on extensive data annotation. Currently, remote-sensing satellite infrared small-target tracking datasets are limited, and factors such as satellite orbit, attitude, and payload design parameters directly impact image quality, making it challenging to adapt training to all scenarios.

Traditional methods and deep-learning approaches exhibit distinct advantages and limitations in space-based infrared small-target tracking. Due to the long imaging distances inherent to space-based systems, moving targets manifest as small-scale objects with weak radiation signatures on focal plane arrays. Deep-learning methods typically require learning target appearance features for inter-frame data association. However, the non-unique appearance characteristics of dim, small infrared targets combined with spatial downsampling and pooling operations during feature extraction may discard critical target details, leading to feature extraction failures and subsequent erroneous associations. Moreover, the scarcity of existing space-based infrared small-target tracking datasets imposes significant constraints on deep-learning approaches, which depend heavily on extensive annotated data. Under limited training samples, these algorithms demonstrate restricted generalization capabilities when confronted with complex operational scenarios. In contrast, traditional methods employ predefined prior constraints to govern target prediction and association processes, effectively circumventing global error propagation while remaining applicable to data-scarce environments. Nevertheless, current conventional approaches neglect the critical influence of detection uncertainty, which directly impacts measurement accuracy in observation models and consequently affects target state updates. The unaccounted measurement errors ultimately degrade both association reliability and localization precision.

To address the aforementioned issues and meet the requirements of infrared remote-sensing sparse-data small-target tracking applications, we propose the UIMM-Tracker based on the accuracy of state estimation for complex maneuvering targets within the IMM framework. This method obtains target uncertainty measurements during the non-maximum suppression processing stage of detection results. Unlike the classical IMM method, it utilizes these uncertainty measurements in the target state update and data association processes, transferring the detector’s empirical distribution into the tracking process to enhance tracking accuracy. Moreover, to address the mismatch of the given prior Markov transition matrix for target state switching in traditional IMM methods, we specifically design a dynamic Markov transition matrix construction method. This method updates model transition probabilities by combining historical dynamic information and current static information. Finally, in the data association phase, we leverage the advantages of intersection over union (IoU) and normalized Wasserstein distance (NWD) to associate targets of different scales, transferring uncertainty into the cost calculation process. To address the ambiguity attributes of the cost matrix caused by uncertain values, we take classical scale invariance and energy invariance assumptions as examples to handle ambiguously matched tracks and targets, verifying the importance of measuring detection uncertainty distribution and the necessity of disambiguating the cost matrix, thereby achieving high-precision target tracking.

Overall, the main contributions of the paper are as follows:

(1) Addressing the challenge of eliminating tracking errors, we inject detection uncertainty measurements into the observation noise of the IMM, which is used to propagate the empirical distribution of the detector during the observation prediction and state update processes, thereby obtaining more accurate state estimation results.

(2) A dynamically adaptive Markov transition matrix is proposed, enabling the IMM framework to adjust current model mixing weights based on the current likelihood probabilities of the models and the rate of change in model probabilities. This achieves precise matching between the target’s motion state and motion models, solving the problem of target trajectory crosstalk.

(3) To address the challenge of associating tracks with detection results in remote-sensing infrared images, this paper incorporates detection uncertainty measurements into the cost calculation process of data association. Using scale and energy invariance assumptions as examples, further constraints are applied to ambiguous matches between tracks and detections, thereby reducing incorrect associations.

2. Related Works

2.1. Small-Target Tracking in Video Satellites

Small-target tracking in video satellites is a pivotal technology in the field of remote sensing, aimed at continuously locating and associating targets (such as aircraft, ships, etc.) that are small in size (typically less than 0.1% of the image area), have low signal-to-noise ratios, and exhibit complex motion patterns from satellite video sequences. It primarily addresses issues such as the lack of apparent target features due to resolution limitations, strong background interference (e.g., clouds, surface textures), and nonlinear motion (rapid movement of targets from a satellite’s perspective). Most existing methods embed strategies, modules, or mechanisms within the classic multi-target tracking framework to address the inherent challenges of small-target tracking applications in remote sensing, thereby meeting practical application requirements. Specific improvements are categorized into three types: enhanced traditional approaches, domain-adaptive deep-learning methods, and hybrid intelligent frameworks. In enhanced traditional approaches, Hua et al. [34] introduced a perspective-aware UKF that models target motion using angular measurements rather than target coordinates in the image plane, thereby reducing drift errors during long-term tracking. In domain-adaptive deep-learning methods, HB-YOLO [35] employs an improved HorNet to replace conventional convolutions for higher-order spatial interactions and replaces ELANs with the BoTNet attention mechanism to enhance small-target features. MCTracker [36] combines inter-frame motion correlation with multi-scale cascaded feature enhancement to improve the feature fusion representation capabilities of multiple modules. To address the issue of data annotation, scholars have also used frameworks similar to MetaEarth [37] to generate synthetic satellite videos with physics-based noise models (e.g., atmospheric turbulence and jitter effects) for pretraining. In hybrid intelligent frameworks, Revach et al. [38] embedded a structured linear Gaussian state-space model and a dedicated recurrent neural network module into the Kalman filter’s process, enabling the model to learn from data to achieve target state estimation in nonlinear environments with low complexity.

However, the aforementioned methods lack a description of uncertainty in target detection results, making it impossible to further guide the generation of high-precision tracking results. Additionally, corresponding solutions are absent for issues such as target trajectory crosstalk and variable-scale association. Based on the shortcomings of existing methods, this paper proposes the UIMM-Tracker framework for small-target tracking in infrared remote-sensing satellite imagery.

2.2. Infrared Small-Target Tracking

To address the challenges posed by the small size, low signal-to-noise ratio (SNR), weak texture features, and frequent interference from complex backgrounds in infrared small-target tracking, researchers have proposed various approaches, including traditional methods and deep-learning-based techniques. Early infrared small-target tracking primarily relied on classical filtering methods, such as Kalman filtering and particle filtering. These methods achieve target state prediction and update by constructing motion and observation models. However, their performance degrades when dealing with complex background interference or target occlusion. For instance, [39] introduced an improved particle filtering algorithm that enhances robustness to target motion to some extent but still exhibits limitations in dynamic scenarios. In recent years, the rapid development of deep-learning techniques has provided new solutions for infrared small-target tracking. Shan et al. [40] proposed the OSTrack model [41], based on a single-stream deep-learning framework, which integrates feature extraction and target tracking processes, significantly improving the accuracy and robustness of infrared small-target tracking. By analyzing the motion characteristics of infrared small targets, [42] designed the IRSDT framework, which combines deep learning with traditional methods to enhance tracking performance under complex backgrounds. Siamese networks [43] are classical methods in target tracking. Considering the weak texture and small size of infrared small targets, Qian et al. [44] proposed the SiamIST algorithm, which improves the SiamRPN network structure to enhance precise localization of small targets during tracking. To alleviate the issue of insufficient data, data augmentation techniques are widely applied. Zhang et al. [45] utilized generative adversarial networks (GANs) to generate synthetic infrared small-target data, effectively expanding the training dataset. Additionally, the fusion of multimodal data has become a research hotspot. For example, [46] proposed a joint tracking framework that integrates infrared and visible light information, significantly enhancing the robustness of tracking algorithms.

This paper addresses the weak and small characteristics of remote-sensing infrared targets and their complex maneuvering states by designing the UIMM-Tracker, which integrates detection uncertainty measurement, IMM, a dynamic Markov transition matrix, and uncertainty-based association cost calculation, enabling precise tracking and high-accuracy localization of targets.

2.3. Application of Uncertainty Measurement

Uncertainty has been widely applied across various fields. For example, in autonomous driving, uncertainty is used to adjust decision-making strategies, preventing accidents caused by incorrect detections [47]. In object detection, uncertainty information can dynamically adjust the confidence threshold of detectors, reducing false positives and missed detections, and adapting to different application scenarios [48]. During data annotation, uncertainty helps select the most informative samples for labeling, reducing annotation costs and improving model performance [49]. In model evaluation and improvement, analyzing uncertainty distribution can reveal weaknesses in specific scenarios or categories, guiding data augmentation or model optimization [50]. In human–machine collaboration scenarios, such as medical image analysis or security monitoring, uncertainty serves as a cue to help human experts focus on high-risk areas or detection results [51]. On resource-constrained edge devices, uncertainty information can dynamically adjust computation paths, such as skipping high-confidence detection boxes and allocating resources to process high-uncertainty regions [52]. In anomaly detection and scene understanding, uncertainty analysis enables the detection of anomalous objects or unfamiliar scenes, enhancing the model’s adaptability to unknown environments [53]. This paper introduces detection uncertainty into the task of remote-sensing infrared small-target tracking, improving target matching accuracy and reducing tracking drift.

3. Method

In this section, we provide a detailed explanation of the components and implementation principles of the proposed UIMM-Tracker. Section 3.1 introduces the overall framework of the tracker, while Section 3.2,Section 3.3,Section 3.4 and Section 3.5 describe the complete process, from detection uncertainty measurement to the tracker’s output at a single time step. Section 3.6 explains the construction method of the dynamic Markov transition matrix, which serves as input for target state estimation in the next time step. Section 3.7 introduces the trajectory completion method, which is used to restore missing target information.

3.1. Overview

The overall framework of the UIMM-Tracker is shown in Figure 2. The tracker consists of four steps: uncertainty measurement of detection results, state prediction (input interaction and filtering), data association, and model update and data fusion. The results of uncertainty measurement serve as inputs for data association and model state updates, transferring the detector’s uncertainty empirical distribution into the target tracking process. Finally, the Markov transition probability matrix is dynamically adjusted by combining current and historical information, enabling the model to adapt to complex target motion states. The overall workflow of the algorithm is shown in Algorithm 1.

Algorithm 1 UIMM-Tracker execution process
Input: Image sequence $\mathcal{ℐ}$1~$\mathcal{ℐ}$n, Detection result $\mathrm{d}=[x^n,y^n,w^n,h^n,s^n{]}_{n=1}^{\mathrm{N}}$, Detection uncertainty covariance $R_0=\mathrm{diag}{({\sigma }_x^2,{\sigma }_y^2,{\sigma }_w^2,{\sigma }_h^2)}_0$
Output: Trajectory $T$, The mean of the updated ${\widehat{x}}_{k|k}^j$, updated state covariance ${\widehat{p}}_{k|k}^j$, fused state mean $x_k$, fused state covariance $p_k$, Markov transition matrix $M_k$, model probability $u_k$
Initial: CT model control variable $y_1\sim y_t={[a_x,a_y,\dot{\omega }]}^{\mathrm{T}}$, Markov transition matrix $M_0=\left[0.8,0.1,0.1;0.1,0.8,0.8;0.1,0.1,0.8\right]$, model probability $u_0=\left[1,0,0;0,1,0;0,0,1\right]$, target initial state mean ${\widehat{x}}_{0|0}^1$$~{\widehat{x}}_{0|0}^N$, Initial state covariance ${\widehat{p}}_{0|0}^1$$~{\widehat{p}}_{0|0}^N$= $\mathrm{diag}(R_0,0.3,0.3,3)$.
1   for k in 1~t do
2	Uncertainty of Measurement Detection Result d: $R_k=\mathrm{diag}{({\sigma }_x^2,{\sigma }_y^2,{\sigma }_w^2,{\sigma }_h^2)}_k$;
3	Input Interaction and Filtering;
4	Calculate the mixed state mean ${\widehat{x}}_{k-1|k-1}^{oj}$ and mixed covariance ${\widehat{p}}_{k-1|k-1}^{oj}$ for model j based on the updated state mean ${\widehat{x}}_{k-1|k-1}^i{|}_{i=1}^N$ and updated state covariance ${\widehat{p}}_{k-1|k-1}^i{|}_{i=1}^N$ of all models;                                         #(1)~(4)
5	Obtain the sigma points of the UKF based on ${\widehat{x}}_{k-1|k-1}^{oj}$ and ${\widehat{p}}_{k-1|k-1}^{oj}$;                               #(5)
6	Obtain the predicted mean ${\widehat{x}}_{k|k-1}^{oj}$ and predicted covariance ${\widehat{p}}_{k|k-1}^{oj}$;                                  #(6)~(8)
7	Calculate the observation predicted mean ${\widehat{z}}_{k|k-1}^{oj}$ and observation predicted covariance ${\widehat{s}}_{k|k-1}^{oj}$;                 #(9)~(11)
8	Data Association;
9	Combine NWD and IoU distance to calculate the cost $C_{pq}$ of trajectory $T_p$ and detection $d_q$;                 #(12)~(15)
10	Iterate over p and q to obtain the cost matrix $C={\left\{C_{pq}\right\}}_{p=1,q=1}^{p=P,q=Q}$;
11	Calculate the scale cost $U_{pqs}^s$ for ambiguous matches of trajectory $T_p$ and detection $d_{q\mathrm{s}}$;                      #(16)
12	Calculate the scale cost $U_{pqs}^e$ for ambiguous matches of trajectory $T_p$ and detection $d_{q\mathrm{s}}$;                   #(17)~(18)
13	Combine the scale cost and energy cost to obtain the additional cost matrix U;                        #(19)
14	Integrate the cost matrix and apply Hungarian matching to obtain the association result ${\mathsf{Ƶ}}_{k|k-1}^{oj}$;
15	Model Probability Update;
16	Calculate the cross-covariance $Co{v}_{k|k-1}^j$ between the predicted mean ${\widehat{x}}_{k|k-1}^{oj}$ and the association result ${\mathsf{Ƶ}}_{k|k-1}^{oj}$;            #(20)
17	Calculate the Kalman gain $K_k^j$;                                              #(21)
18	Calculate the updated state mean ${\widehat{x}}_{k|k}^j$ and updated state covariance ${\widehat{p}}_{k|k}^j$;                         #(22)~(23)
19	Data Fusion;
20	Calculate the likelihood probability ${\Lambda }_k^j$ using ${\mathsf{Ƶ}}_{k|k-1}^{oj}$, ${\widehat{z}}_{k|k-1}^{oj}$ and ${\widehat{s}}_{k|k-1}^{oj}$;                              #(24)
21	Calculate the model probability $u_k^j$ using ${\Lambda }_k^j$, Markov transition probability $M_{k-1}^{ij}$, and the model probability $u_{k-1}^i$;        #(25)
22	Calculate the fused state mean $x_k$ and fused covariance $p_k$ using $u_k^j$, ${\widehat{x}}_{k|k}^j$, and ${\widehat{p}}_{k|k}^j$;                  #(26)~(27)
23	Dynamic Markov Transition Matrix;
24	Calculate the state prediction error ${\widetilde{ep}}_k^j$ using the observation predicted mean ${\widehat{z}}_{k|k-1}^{oj}$ and the association result ${\mathsf{Ƶ}}_{k|k-1}^{oj}$;
25	Calculate the normalized likelihood probability ${\tilde{\Lambda }}_k^j$;
26	Calculate the rate of change of model probability ${\rho }_k^j$;
27	Calculate the Markov transition probability matrix $M_k$;                             #(28)~(29)
28	Filling Discontinuous Trajectories;
29	For discontinuous trajectories from time t1 to t2, use the fused state mean $x_{t_2}$ at time t2 and the predicted mean ${\widehat{x}}_{t_2|t_2-1}$ to correct the prediction error.                                                     #(30)
30  end

3.2. Uncertainty Measurement of Detection Results

The results of object detection consist of a series of bounding boxes and confidence scores. For a given object, the detection result is described as $(x^n,y^n,w^n,h^n,s^n)$, where $n=1,2,\dots ,N$ and N represent the number of detection boxes. $(x^n,y^n)$ denotes the center position of the n-th detection box, $(w^n,h^n)$ represents its width and height, and $s^n\in [0,1]$ is the confidence score of the n-th detection box. After applying non-maximum suppression (NMS), the optimal bounding box is retained. In conventional tracking methods, the de-redundant bounding box is used as input to the tracking algorithm, and association methods are employed to link bounding boxes belonging to the same object, ultimately producing the tracking results. This process treats redundant detection results as irrelevant information and models the observation noise at time step k as a random distribution with a mean of 0 and a root mean square of $R_k$, making it difficult to eliminate detection biases during the tracking process.

We model the detection results as a spatial distribution with a standard deviation σ. σ is calculated by measuring the variance of each detection result relative to the bounding box retained after non-maximum suppression (NMS) and is propagated to the tracker to quantify the observation noise of the IMM tracker, thereby characterizing the distribution range of the observations. As shown in Figure 3a, the modeled ${\left\{R_k=\mathrm{diag}{({\sigma }_x^2,{\sigma }_y^2,{\sigma }_w^2,{\sigma }_h^2)}_k\right\}}_{k=1}^N$, where $\mathrm{diag}(•)$ represents a diagonal matrix.

3.3. Input Interaction and Filtering

The target state mean of the IMM filter output at time step k is defined as $x_k={[x_k,y_k,w_k,h_k]}^{\mathrm{T}}$. Considering the CV model, CA model, and CT model, the target state space is represented as ${[x_k,y_k,w_k,h_k,{\dot{x}}_k,{\dot{y}}_k,{\theta }_k]}^{\mathrm{T}}$, where ${\theta }_k$ denotes the target’s rotation angle at time step k.

The conditional probability matrix of model transitions is defined as $\Pi ={\left\{{\pi }_{ij}\right\}}_{i,j=1}^N$, where ${\pi }_{ij}=P(M_k^j\left|M_{k-1}^i\right.)$ represents the probability of transitioning from model i to model j. $M^t$ denotes different motion models, with $t=1,2,\dots N$, where N is the number of models. The ${\widehat{x}}_{k-1|k-1}^i$ and ${\widehat{p}}_{k-1|k-1}^i$ represent the updated state mean and updated covariance output by model i at time k − 1, respectively. The mixed initial state and corresponding covariance are calculated based on the model transition probability $\Pi$:

$${\widehat{x}}_{k-1|k-1}^{oj}={\displaystyle \sum _{i=1}^N{\pi }_{ij}{\widehat{x}}_{k-1|k-1}^i}$$

(1)

$${\widehat{p}}_{k-1|k-1}^{oj}={\displaystyle \sum _{i=1}^N{\pi }_{ij}\left[{\widehat{p}}_{k-1|k-1}^i+({\widehat{x}}_{k-1|k-1}^{oj}-{\widehat{x}}_{k-1|k-1}^i){({\widehat{x}}_{k-1|k-1}^{oj}-{\widehat{x}}_{k-1|k-1}^i)}^{\mathrm{T}}\right]}$$

(2)

where ${\widehat{x}}_{k-1|k-1}^{oj}$ and ${\widehat{p}}_{k-1|k-1}^{oj}$ represent the mixed initial state variable and mixed initial covariance of the j-th model after mixing, respectively. The calculation of ${\pi }_{ij}$ is as follows:

$${\pi }_{ij}={\displaystyle \frac{1}{{\overline{\varsigma }}^j}}{M}_{k-1}^{ij}{u}_{k-1}^i$$

(3)

$${\overline{\varsigma }}^j={\displaystyle \sum _{i=1}^N{M}_{k-1}^{ij}{u}_{k-1}^i}$$

(4)

where $M_{k-1}^{ij}$ represents the Markov transition probability from model i to model j at time k − 1, and $u_{k-1}^i$ denotes the probability of model i at time k − 1. The calculation of ${\overline{\varsigma }}^j$ is for normalization. $M_{k-1}^{ij}$ adapts based on the target state to fit the motion model, enabling accurate tracking of complex maneuvering targets. The detailed design process is described in Section 3.6.

For each model j, the UKF is used for state prediction. The UKF predicts the distribution of data by propagating sigma points through nonlinear transformations, replacing the complex linearization process. The sigma points are sampled as follows:

$${\mathbf{𝒳}}_{k-1}^{oj}={\widehat{x}}_{k-1|k-1}^{oj}\pm \sqrt{(n+\lambda ){\widehat{p}}_{k-1|k-1}^{oj}}$$

(5)

where ${\mathbf{𝒳}}_{k-1}^{oj}$ represents the sigma point matrix, n denotes the dimension of the state, and λ is a parameter of the UKF, defined as $\lambda ={\alpha }^2(n+\kappa )-n$, where α is the scaling parameter and κ is set to 3 − n.

Each sigma point is propagated through the state transition function ${\mathrm{f}}^j(·)$ for state prediction:

$${\mathbf{𝒳}}_{k|k-1}^{oj}={\mathrm{f}}^j({\mathbf{𝒳}}_{k-1}^{oj})$$

(6)

The predicted state mean ${\widehat{x}}_{k|k-1}^{oj}$ and the predicted covariance ${\widehat{p}}_{k|k-1}^{oj}$ are obtained as follows:

$${\widehat{x}}_{k|k-1}^{oj}={\displaystyle \sum _{q=1}^{2n+1}{W}_q^{(m)}{\mathbf{𝒳}}_{k|k-1,q}^{oj}}$$

(7)

$${\widehat{p}}_{k|k-1}^{oj}={\displaystyle \sum _{q=1}^{2n+1}{W}_q^{(c)}({\mathbf{𝒳}}_{k|k-1,q}^{oj}-{\widehat{x}}_{k|k-1}^{oj}){({\mathbf{𝒳}}_{k|k-1,q}^{oj}-{\widehat{x}}_{k|k-1}^{oj})}^{\mathrm{T}}}+Q^j$$

(8)

where $W_q^{(m)}$ and $W_q^{(c)}$ represent the weights for the calculation of the state mean and covariance in the UKF process, respectively.

The sigma points after state prediction are propagated as follows through the observation function $\mathrm{h}(•)$:

$${\mathit{\gamma }}_{k|k-1}^{oj}=\mathrm{h}({\mathbf{𝒳}}_{k|k-1}^{oj})$$

(9)

The measurement prediction mean ${\widehat{z}}_{k|k-1}^{oj}$ is used for subsequent data association, which is described in Section 3.4:

$${\widehat{z}}_{k|k-1}^{oj}={\displaystyle \sum _{q=1}^{2n+1}{W}_q^{(m)}}({\mathit{\gamma }}_{k|k-1,q}^{oj})$$

(10)

The measurement prediction covariance ${\widehat{s}}_{k|k-1}^{oj}$ is used in the likelihood probability calculation process described in Section 3.5. The specific calculation is as follows:

$${\widehat{s}}_{k|k-1}^{oj}={\displaystyle \sum _{q=1}^{2n+1}{W}_q^{(c)}}({\mathit{\gamma }}_{k|k-1,q}^{oj}-{\widehat{z}}_{k|k-1}^{oj})({\mathit{\gamma }}_{k|k-1,q}^{oj}-{\widehat{z}}_{k|k-1}^{oj}{)}^{\mathrm{T}}+R_k$$

(11)

where R_k represents the covariance of the detection results at time k, obtained in Section 3.2.

3.4. Data Association

To account for the association cost calculation of targets with different scales, inspired by [54], we adopt a scale-aware joint NWD-IOU distance to get the cost C_pq between the trajectory $T_p$ (derived from the measurement prediction mean ${\widehat{z}}_{k|k-1}^{oj}$) and the detection $d_q$ as follows:

$$C_{pq}={\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}}\times \mathrm{IoU}+(1-{\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}})\times \mathrm{NWD}(C)$$

(12)

where A is the area enclosed by the detection box $d_q$, and C is a constant balancing the IoU and NWD distances.

Due to the uncertainty of detection results, the IoU and NWD costs of $T_p$ and $d_q$ are extended from a single value to a range. The corresponding cost $C_{pq}={\overline{C}}_{pq}\pm \Delta C_{pq}$, according to Equation (11), is calculated as follows:

$$\Delta C_{pq}={\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}}\times \Delta \mathrm{IoU}+(1-{\displaystyle \frac{1}{1+e^{-(\sqrt{A}/C-1)}}})\times \Delta \mathrm{NWD}(C)$$

(13)

Since the area fluctuation range of small-scale targets is limited to the pixel level, its impact on the above equation is negligible. A can thus be treated as a constant, which is calculated after applying NMS to the detection results. Therefore, the factors ultimately affecting the cost range are $\Delta \mathrm{IoU}$ and $\Delta \mathrm{NWD}$. In the estimation of $\Delta \mathrm{IoU}$, the calculation of IoU is more sensitive to the uncertainty in the intersection region. Thus, $\Delta \mathrm{IoU}$ is represented by the uncertainty of the intersection region, i.e., $\Delta \mathrm{IoU}/\mathrm{IoU}=\Delta {\mathrm{A}}_{\cap }/{\mathrm{A}}_{\cap }$, as shown in Figure 3b. For bounding boxes $B_1=(x_1,y_1,h_1,w_1)$ and $B_2=(x_2,y_2,h_2,w_2)$ with inaccuracies $\Delta B_1=(\Delta x_1,\Delta y_1,\Delta h_1,\Delta w_1)$ and $\Delta B_2=(\Delta x_2,\Delta y_2,\Delta h_2,\Delta w_2)$, respectively, the relationship between $\Delta \mathrm{IoU}$ and IoU is derived based on the principle of linear error propagation as follows:

$${\displaystyle \frac{\Delta \mathrm{IoU}}{\overline{\mathrm{IoU}}}}={\displaystyle \frac{\Delta {\mathrm{A}}_{\cap }}{{\mathrm{A}}_{\cap }}}={\displaystyle \frac{\Delta x_2-\Delta x_1+\frac{1}{2}(\Delta w_2+\Delta w_1)}{x_2-x_1+\frac{1}{2}(w_2+w_1)}}+{\displaystyle \frac{\Delta y_2-\Delta y_1+\frac{1}{2}(\Delta h_2+\Delta h_1)}{y_2-y_1+\frac{1}{2}(h_2+h_1)}}$$

(14)

The uncertainty $\Delta r$, $r\in \{x_1,y_1,w_1,h_1,x_2,y_2,w_2,h_2\}$, represents the standard deviation of the corresponding measurement, which is obtained in Section 3.2 by measuring the standard deviation of each detection result. Based on the above calculations, the range of $\mathrm{IoU}$ values is ${\overline{\mathrm{IoU}}}_{pq}\pm \Delta {\mathrm{IoU}}_{pq}$.

In the calculation of $\Delta \mathrm{NWD}$, since the computation of the center point Wasserstein distance $d_c$ and the scale Wasserstein distance $d_s$ are nonlinear processes, we simplify the operation by using a first-order Taylor expansion to approximate the derivative process and neglect higher-order terms. The relationship between $\Delta \mathrm{NWD}$ and NWD is derived as follows:

$$\left\{\begin{array}{l}{\displaystyle \frac{\Delta \mathrm{NWD}}{\overline{\mathrm{NWD}}}}={\displaystyle \frac{\Delta d_c+\Delta d_s}{d_c+d_s}}-{\displaystyle \frac{(w_1\Delta h_1+h_1\Delta w_1+w_2\Delta h_2+h_2\Delta w_2)}{2(w_1{h}_1+w_2{h}_2)}}\\ \Delta d_c={\displaystyle \frac{(x_2-x_1)(\Delta x_2-\Delta x_1)+(y_2-y_1)(\Delta y_2-\Delta y_1)}{d_c}}\\ \Delta d_s={\displaystyle \frac{(w_2-w_1)(\Delta w_2-\Delta w_1)+(h_2-h_1)(\Delta h_2-\Delta h_1)}{2d_s}}\\ d_c=\sqrt{{(x_2-x_1)}^2+{(y_2-y_1)}^2}\\ d_s=\sqrt{((w_2-w_1)/2{)}^2+((h_2-h_1)/2{)}^2}\end{array}\right.$$

(15)

The uncertainty $\Delta r$, $r\in \{x_1,y_1,w_1,h_1,x_2,y_2,w_2,h_2\}$, is obtained in the same manner as in the $\Delta \mathrm{IoU}$ calculation process. Based on the above calculations, the range of NWD values is ${\overline{\mathrm{NWD}}}_{pq}\pm \Delta {\mathrm{NWD}}_{pq}$. Finally, the cost range between trajectory $T_p$ and detection $d_q$ is $C_{pq}\in [{\overline{C}}_{pq}-\Delta C_{pq},{\overline{C}}_{pq}+\Delta C_{pq}]$.

In the association process, the cost matrix $C={\left\{C_{pq}\right\}}_{p=1,q=1}^{p=P,q=Q}$ is solved to obtain the optimal assignment between trajectories and detections, where P represents the number of trajectories at the current time step, and Q represents the number of detections. However, with the introduction of uncertain measurements, each element in the cost matrix expands from a single value to an interval range, disrupting the one-to-one matching relationship between trajectories and detections (as shown in Figure 3c) and introducing ambiguous matching cases. To correctly associate trajectories with detections $d_q$, an additional cost matrix $U={\left\{U_{p{q}_s}\right\}}_{q_s=1}^{q_s=Q_s}$ (where Q_s represents the number of ambiguously matched detections) is introduced to replace the ambiguous elements in the original cost matrix C. Finally, the Hungarian algorithm is applied to obtain the association results between trajectories and detections.

We introduce the following two classic, simple prior assumptions of space-based infrared small-target imaging to design the additional cost matrix U:

(1) The target’s scale remains nearly consistent across consecutive frames, meaning targets with significant scale differences should have lower matching degrees;

(2) The target’s energy remains nearly consistent across consecutive frames, meaning targets with significant energy differences should have lower matching degrees.

Based on the above two assumptions, $U$ consists of two components: the shape constraint matrix $U^s$ and the energy constraint matrix $U^e$. The calculation of $U^s$ is as follows:

$$\left\{\begin{array}{l}{\widetilde{AT}}_p=(A{T}_p-\overline{AT})/{\sigma }_T\\ {\widetilde{Ad}}_{q_s}=(A{d}_{q_s}-\overline{Ad})/{\sigma }_d\\ U_{p{q}_s}^s=e^{-{\displaystyle \frac{1}{2}}{({\widetilde{AT}}_p-{\widetilde{Ad}}_{q_s})}^2}\end{array}\right.$$

(16)

where $A{T}_p$, $\overline{AT}$, and ${\sigma }_T$ represent the bounding box area of trajectory p, the mean area of the trajectory, and the standard deviation of the trajectory area, respectively. Thus, ${\widetilde{AT}}_p$ is the normalized bounding box area with a mean of 0 and a standard deviation of 1. Similarly, ${\widetilde{Ad}}_{q_s}$ is the normalized result for the ambiguously matched detection. $U_{p{q}_s}^s$ is the distance metric function between the two, normalized to [0, 1], where a closer match between them results in $U_{p{q}_s}^s$ approaching 1.

The calculation of $U^e$ differs slightly from that of $U^s$. In infrared scenarios, targets may move across different backgrounds, and the energy of a target point can be considered as the sum of the target energy and the background energy. Therefore, directly using the grayscale value of the target point for calculation introduces background interference. To address this issue, we use the signal-to-clutter ratio (SCR) to eliminate the influence of background energy. For detection $d_{q_s}$, the grayscale value of the target point is $E_{q_s}$, the mean background grayscale is ${\overline{B}}_{q_s}$, and the background standard deviation is ${\sigma }_{q_s}^B$. The signal-to-clutter ratio $SC{R}_{q_s}$ is calculated as follows:

$$S{d}_{q_s}={\displaystyle \frac{\left|E_{q_s}-{\overline{B}}_{q_s}\right|}{{\sigma }_{q_s}^B}}$$

(17)

The calculation of $U^e$ is as follows:

$$\left\{\begin{array}{l}{\widetilde{ST}}_p=(S{T}_p-\overline{ST})/{\sigma }_{ST}\\ {\widetilde{Sd}}_{q_s}=(S{d}_{q_s}-\overline{Sd})/{\sigma }_{Sd}\\ U_{p{q}_s}^e=e^{-{\displaystyle \frac{1}{2}}{({\widetilde{ST}}_p-{\widetilde{Sd}}_{q_s})}^2}\end{array}\right.$$

(18)

where $S{T}_p$, $\overline{ST}$, and ${\sigma }_{ST}$ represent the SCR of trajectory p, the mean SCR of the trajectory, and the standard deviation of the trajectory SCR, respectively. The resulting ${\widetilde{ST}}_p$ is the normalized SCR of trajectory p. Similarly, ${\widetilde{Sd}}_{q_s}$ is the normalized SCR of detection q_s. $U_{{pq}_s}^e$ measures the distance between the two and is normalized to [0, 1]. Therefore, the additional cost matrix U is

$$U_{p{q}_s}=\lambda \cdot U_{p{q}_s}^s+(1-\lambda )\cdot U_{p{q}_s}^e$$

(19)

where λ is the coefficient used to balance the shape constraint and energy constraint.

During the matching process between the measurement prediction ${\widehat{z}}_{k|k-1}^{oj}$ of the IMM and the detections, the association results generated by each model are defined as ${\mathsf{Ƶ}}_{k|k-1}^{oj}$ and are used for subsequent model state updates.

3.5. Model Probability Update and Data Fusion

Using the predicted sigma points ${\mathbf{𝒳}}_{k|k-1}^{oj}$ (Equation (6)), the predicted state mean ${\widehat{x}}_{k|k-1}^{oj}$(Equation (7)), the associated detection results ${\mathsf{Ƶ}}_{k|k-1}^{oj}$, and the measurement prediction mean ${\widehat{z}}_{k|k-1}^{oj}$ (Equation (10)), the cross-covariance matrix between ${\widehat{x}}_{k|k-1}^{oj}$ and ${\mathsf{Ƶ}}_{k|k-1}^{oj}$ is calculated as follows:

$$Co{v}_{k|k-1}^j={\displaystyle \sum _{q=1}^{2n+1}{W}_q^{(c)}({\mathbf{𝒳}}_{k|k-1}^{oj}-{\widehat{x}}_{k|k-1}^{oj})}{({\mathsf{Ƶ}}_{k|k-1}^{oj}-{\widehat{z}}_{k|k-1}^{oj})}^{\mathrm{T}}$$

(20)

The Kalman gain is calculated using the cross-covariance matrix and the observation covariance. According to Section 3.2, the observation covariance $R_k$ of the measurement is obtained. Therefore, the Kalman gain $K_k^j$ is

$$K_k^j=Co{v}_{k|k-1}^j{(R_k)}^{-1}$$

(21)

The updated state mean and covariance are

$${\widehat{x}}_{k|k}^j={\widehat{x}}_{k|k-1}^{oj}+K_k^j({\mathsf{Ƶ}}_{k|k-1}^{oj}-{\widehat{z}}_{k|k-1}^{oj})$$

(22)

$${\widehat{p}}_{k|k}^j={\widehat{p}}_{k|k-1}^{oj}-K_k^j{R}_k{(K_k^j)}^{\mathrm{T}}$$

(23)

After obtaining the updated state ${\widehat{x}}_{k|k}^j$ and updated covariance ${\widehat{p}}_{k|k}^j$ for each model, they are used as inputs for the IMM tracking algorithm in the next time step. Simultaneously, the associated detection results ${\mathsf{Ƶ}}_{k|k-1}^{oj}$ is used to calculate the likelihood probability of each model, which represents the degree of matching between the model’s measurement prediction mean and the actual observation. The model likelihood probability is calculated based on Bayesian theory, and the specific calculation is as follows:

$${\Lambda }_k^j=P(z_k\left|M_k^j\right.)={\displaystyle \frac{1}{\sqrt{\left|2\pi {\widehat{s}}_{k|k-1}^{oj}\right|}}}\exp \left(-{\displaystyle \frac{1}{2}}{({\mathsf{Ƶ}}_{k|k-1}^{oj}-{\widehat{z}}_{k|k-1}^{oj})}^{\mathrm{T}}{({\widehat{s}}_{k|k-1}^{oj})}^{-1}({\mathsf{Ƶ}}_{k|k-1}^{oj}-{\widehat{z}}_{k|k-1}^{oj})\right)$$

(24)

where ${\widehat{s}}_{k|k-1}^{oj}$ represents the measurement prediction covariance (Equation (11)), and ${\widehat{z}}_{k|k-1}^{oj}$ denotes the measurement prediction mean (Equation (10)).

The probability $u_k^j$ of model j is calculated by combining the likelihood probability ${\Lambda }_k^j$ and the Markov transition probability $M_{k-1}^{ij}$. The calculation is as follows:

$$u_k^j={\displaystyle \frac{{\Lambda }_k^j{\displaystyle \sum _{i=1}^N{M}_{k-1}^{ij}{u}_{k-1}^i}}{{\displaystyle \sum _{j=1}^N({\Lambda }_k^j{\displaystyle \sum _{i=1}^N{M}_{k-1}^{ij}{u}_{k-1}^i})}}}$$

(25)

Based on $u_k^j$, combined with the updated state mean results ${\widehat{x}}_{k|k}^j$ and updated covariance ${\widehat{p}}_{k|k}^j$ of all models, the IMM fused output for the fused target state mean and fused covariance is

$$x_k={\displaystyle \sum _{j=1}^N{u}_k^j{\widehat{x}}_{k|k}^j}$$

(26)

$$p_k={\displaystyle \sum _{j=1}^N{u}_k^j\left[{\widehat{p}}_{k|k}^j+({\widehat{x}}_{k|k}^j-x_k){({\widehat{x}}_{k|k}^j-x_k)}^{\mathrm{T}}\right]}$$

(27)

3.6. Dynamic Markov Transition Matrix

The fused target state is directly influenced by the Markov transition matrix $M_{k-1}^{ij}$, which is predefined in traditional IMM methods. However, for complex maneuvering targets, the predefined matrix cannot adapt to the rapid switching of motion models required for frequent state transitions. Therefore, we design a dynamic Markov transition matrix that autonomously updates by comprehensively considering model prediction errors, likelihood probability calculations, and the changes in model probabilities over two consecutive time steps. This approach improves the accuracy of the IMM algorithm.

Specifically, the normalized state prediction error of model j at time k − 1 is ${\widetilde{ep}}_k^j=‖{\widehat{z}}_{k|k-1}^{oj}-{\mathcal{Ƶ}}_{k|k-1}^{oj}‖/{\displaystyle {\sum }_{j=1}^N‖{\widehat{z}}_{k|k-1}^{oj}-{\mathcal{Ƶ}}_{k|k-1}^{oj}‖}$, the normalized likelihood probability at time k is ${\tilde{\Lambda }}_k^j={\Lambda }_k^j/{\displaystyle {\sum }_{j=1}^N{\Lambda }_k^j}$, and the model probabilities at times k − 1 and k are $u_{k-1}^j$ and $u_k^j$, respectively. The rate of change in model probability at time k is denoted as ${\rho }_k^j=\exp (u_k^j-u_{k-1}^j)$. ${\widetilde{ep}}_k^j\in [0,1]$, where a smaller ${\widetilde{ep}}_k^j$ indicates that the predicted measurement ${\widehat{z}}_{k|k-1}^{oj}$ is closer to the detected measurement ${\mathcal{Ƶ}}_{k|k-1}^{oj}$, suggesting a higher degree of consistency between the model and the actual situation, and thus the model transition probability should be higher. ${\tilde{\Lambda }}_k^j\in [0,1]$, where a larger ${\tilde{\Lambda }}_k^j$ indicates a higher likelihood probability for model j, thus the model transition probability should also be higher. ${\rho }_k^j\in [1/e,e]$, representing the change in the model’s ability to match the actual motion over two consecutive time steps, ${\rho }_k^j>1$ indicates an increased consistency between model j and the target motion, and the model transition probability should increase accordingly. Based on the above analysis, the Markov transition probability at time k, ${\widehat{M}}_k^{ij}$, is estimated as follows:

$${\widehat{M}}_k^{ij}=(1-{\widetilde{ep}}_k^j)\cdot (1+{\tilde{\Lambda }}_k^j)\cdot {\rho }_k^j\cdot M_{k-1}^{ij}$$

(28)

By normalizing ${\widehat{M}}_k^{ij}$, the sum of the elements in each row of the Markov transition matrix is equal to 1, such that

$$M_k^{ij}={\displaystyle \frac{{\widehat{M}}_k^{ij}}{{\displaystyle {\sum }_{j=1}^N{\widehat{M}}_k^{ij}}}}$$

(29)

The dynamic adjustment of the Markov transition matrix occurs at each time step’s output and serves as the input for the next time step, replacing the fixed prior values used in traditional IMM. This enables the algorithm to dynamically adjust the transition probabilities between models based on the actual state, thereby adapting to complex maneuvering targets.

3.7. Method for Filling Discontinuous Trajectories

When detection loss causes a break in the trajectory, continuous prediction of the trajectory is required, and real-time matching with detection results is performed based on the predictions. Assume the trajectory $T=[x_{t_s},x_{t_s+1},\dots ,{\widehat{x}}_{t_1|t_1-1},\dots {\widehat{x}}_{t_2-1|t_2-2},x_{t_2}]$, where $x_t$ represents the state mean at time t, and ${\widehat{x}}_{t|t-1}$ represents the state prediction mean at time t. The trajectory breaks at time t₁, and is re-associated at time t₂. Therefore, the state mean values from t₁ to t₂ in the trajectory sequence $T$ are replaced by predicted mean values. After re-association at time t₂, a new state mean $x_{t_2}$ is formed. Using linear error propagation theory, the error between the state mean $x_{t_2}$ and the prediction mean ${\widehat{x}}_{t_2|t_2-1}$ is used to correct the prediction errors from t₁ to t₂, forming a state mean to fill the broken trajectory as follows:

$$x_t={\widehat{x}}_{t|t-1}+{\displaystyle \frac{t-t_1}{t_2-t_1}}(x_{t_2}-{\widehat{x}}_{t_2|t_2-1}), t_1\le t\le t_2$$

(30)

4. Experiments and Results

In this section, we present the necessary preparations for the experiments in Section 4.1, and the experimental results in Section 4.2 and Section 4.3. The experimental preparations include dataset preparation, an introduction to comparison methods, the construction of the evaluation metric system, and implementation details of the tracker, such as parameter initialization and hyperparameter settings. The experimental results comprise comparative results with various methods and ablation study results of key components of the UIMM-Tracker.

4.1. Experimental Setting

4.1.1. Datasets

We select 20 sequences from the large-scale infrared video target tracking dataset IRSatVideo-LEO [55], which include various background types and targets with different motion states to validate the performance of the UIMM-Tracker. Examples from the dataset are shown in Figure 4, consisting of a total of 8627 frames with 96 targets. The total number of frames in which targets appear is 37,278. The sequences feature diverse maneuvering types, and the backgrounds include farmland, mountains, land, oceans, and clouds.

4.1.2. Comparison Methods

We use traditional and deep-learning methods as comparison methods to illustrate the advantages of the UIMM-Tracker. Traditional methods include the VB-EOT-SN [56], PMB-EOT-BP [57], TrPMBM [58], TPMBM [59], Gaussian CD-PMBM [60], and MEM-EKF [61] algorithms, while the deep-learning methods include ByteTrack [62], CMTrack [63], AdapTrack [64], Deep-EIoU [65], and BoostTrack [66].

4.1.3. Evaluation Metrics

The evaluation metrics are categorized into tracking metrics, association metrics, and localization accuracy metrics. Tracking metrics, including MOTA [67] and HOTA [68], assess the overall performance of the algorithm. Association metrics, such as AssA [68], IDF1 [69], and ID Switch (IDs) [70], evaluate the data association capability in target tracking algorithms. The localization accuracy metric is the mean Euclidean distance (precision) between the center of the tracked bounding box and the center of the ground truth trajectory, used to assess the state estimation capability of the tracking algorithm when incorporating detection uncertainty distribution and multi-model strategies.

4.1.4. Implementation Details

Given that the target state is denoted as, and considering the CV, CA, and CT models comprehensively, the state prediction process is expressed as $x_k=\mathrm{f}{x}_{k-1}+\mathrm{g} y_k+\mathrm{\Gamma }({\theta }_{k-1},{\dot{\theta }}_k)\Delta t$, where $\mathrm{g}$ represents the input control matrix, $\mathrm{\Gamma }$ is the nonlinear coupling matrix of angle and velocity, $y_k={[a_x,a_y,\dot{\omega }]}^{\mathrm{T}}$ is the input control vector, and $\Delta t$ is the time interval of a single recursion. Therefore, the state transition matrix used is

$$\mathrm{f}=\left[\begin{array}{ccccccc}1& 0& 0& 0& \Delta t& 0& 0\\ 0& 1& 0& 0& 0& \Delta t& 0\\ 0& 0& 1& 0& 0& 0& 0\\ 0& 0& 0& 1& 0& 0& 0\\ 0& 0& 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 0& 1& 0\\ 0& 0& 0& 0& 0& 0& 1\end{array}\right]$$

(31)

The input control matrix is

$$\mathrm{g}=\left[\begin{array}{ccc}\left(1/2\right)\Delta t^2& 0& 0\\ 0& \left(1/2\right)\Delta t^2& 0\\ 0& 0& 0\\ 0& 0& 0\\ \Delta t& 0& 0\\ 0& \Delta t& 0\\ 0& 0& \Delta t\end{array}\right]$$

(32)

The nonlinear coupling matrix is

$$\mathrm{\Gamma }=\left[\begin{array}{c}-\left({\dot{x}}_{k-1}/\dot{\theta }\right)\sin \left({\theta }_{k-1}\right)+\left({\dot{y}}_{k-1}/\dot{\theta }\right)\cos \left({\theta }_{k-1}\right)\\ \left({\dot{x}}_{k-1}/\dot{\theta }\right)\cos \left({\theta }_{k-1}\right)+\left({\dot{y}}_{k-1}/\dot{\theta }\right)\sin \left({\theta }_{k-1}\right)\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}\right]$$

(33)

During the tracking process, hyperparameters are divided into two components: tracking initial values and model hyperparameters. (1) Tracking initial values $y_k$ is set to ${[0.5,0.5,5]}^{\mathrm{T}}$. The initial value of the Markov transition matrix is $M_0=\left[0.8,0.1,0.1;0.1,0.8,0.8;0.1,0.1,0.8\right]$. The initial value of the probability of the model is $u_0=\left[1,0,0;0,1,0;0,0,1\right]$. The initial state of the target is the combination of the detection result $d$ and the remaining state variables ${[d,0,0,0]}^{\mathrm{T}}$. The initial state covariance of the target is the combination of the detection uncertainty covariance $R_0$ obtained in Section 3.2 and the covariance of additional parameters $\mathrm{diag}(R_0,0.3,0.3,3)$. (2) Model hyperparameters: The value of C in Equation (12) is 4, and the background region radius for calculating the signal-to-clutter ratio in Equation (17) is three times the target radius. The parameter α in the sigma point generation process is set to 0.1. In the cost calculation, the coefficient λ balancing $U^s$ and $U^e$ is set to 0.7. Due to its integration of visual feature extraction, motion feature extraction, and motion affinity modules, LASNet [71] is suitable for cross-space and motion-scale target detection tasks, making it adaptable for space-based infrared moving target observation scenarios. Therefore, we select it as the detector, which provides bounding box coordinates (x, y, width, height) and confidence scores for each detected target. Before being utilized by the tracker, these candidate detections are filtered using a confidence threshold of 0.5 to eliminate false positives. The detector achieves an mAP₅₀ of 76.09%, a precision of 95.74%, a recall of 78.63%, and an F1 score of 87.12%, demonstrating the high quality of the detection results. The filtered data is then passed to the UIMM-Tracker as the observation input for the IMM framework. All experiments are conducted on a computer equipped with a 12th Gen Intel(R) Core(TM) i9-12,900 K CPU.

4.2. Comparison with Various Methods

4.2.1. Quantitative Analysis

To comprehensively compare the overall performance and applicability of various methods, we summarize the average values of all metrics across all sequences in

Table 1. The average MOTA (%), HOTA (%), AssA (%), IDF1 (%), IDs (count), and precision (pixels) metrics of different algorithms on each sequence are compared. Red indicates the best value, while blue indicates the second-best. ↑ and ↓ indicate that higher and lower values of the metric are better, respectively.

Method	MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↓
VB-EOT-SN [56]	37.2	46.1	50.0	62.1	51	0.82
PMB-EOT-BP [57]	41.8	54.3	52.5	63.2	44	0.90
TrPMBM [58]	40.5	52.7	52.6	61.6	39	0.93
TPMBM [59]	45.3	55.8	56.1	67.2	38	1.19
Gaussian CD-PMBM [60]	44.7	53.9	55.1	67.3	39	1.31
MEM-EKF [61]	35.0	44.5	50.4	59.7	68	0.74
ByteTrack [62]	45.0	55.2	54.9	66.3	41	0.88
CMTrack [63]	44.4	52.9	53.8	63.6	40	0.85
AdapTrack [64]	46.1	54.7	56.2	67.9	42	1.55
Deep-EIoU [65]	43.8	54.6	54.7	65.1	43	0.92
BoostTrack [66]	27.2	33.1	32.6	41.9	84	1.48
UIMM-Tracker	45.6	56.2	55.7	68.5	35	0.41

and report the MOTA and precision metrics of different methods on sequences 1 to 10 (detailed descriptions of the sequences are provided in

Table 2. The target and background description for visual comparison and refined metric evaluation.

Seq	Frames	Size	Target Number	Target Condition	Background Condition
(1)	273	1024 × 1024	5	Regular motion High speed Small scale	Cloud–Sea background Complex structure Gradually changing background
(2)	225		4	Regular motion Moderate speed Weak energy	Striped background Slow movement Complex structure
(3)	157		6	Trajectory crossing Varying motion speeds Small scale	Fragmented cloud background High noise Rotating background
(4)	445		2	Continuous acceleration Weak energy Small scale	Irregular cloud background High contrast High noise
(5)	586		4	High maneuverability Varying motion speeds Weak energy	Farmland background Inconsistent light and shadow Weak noise
(6)	378		4	Trajectory crossing Varying motion speeds Varying scales	Cloud–Sea background High contrast High noise
(7)	687		7	Move across backgrounds Varying motion states Moderate speed	Land–Sea–Cloud background Complex terrain Inconsistent light and shadow
(8)	499		7	Significant differences in speeds High maneuverability Presence of dense regions	Urban river background High-temperature false alarm source High contrast
(9)	708		2	Slow motion Fluctuating energy levels Continuously changing scales	Mountainous background Uneven illumination Clutter from structures such as ridges
(10)	420		5	Significant differences in motion states High maneuverability Presence of sudden speed changes	Land–Sea background High noise Uneven illumination

) in

Table 3. The MOTA (%) and precision (pixels) metrics of different algorithms are compared across 10 challenging datasets. Red indicates the best value, while blue indicates the second-best.

Method	MOTA ↑|Precision ↓
	Seq1	Seq2	Seq3	Seq4	Seq5	Seq6	Seq7	Seq8	Seq9	Seq10
VB-EOT-SN [56]	37.1|1.05	36.2|0.82	39.2|0.69	41.5|0.73	42.4|0.81	32.8|0.89	40.9|0.77	30.6|0.89	41.4|0.94	30.9|0.91
PMB-EOT-BP [57]	44.1|0.69	42.1|1.05	46.9|0.98	38.9|0.91	46.5|0.92	37.6|0.79	45.3|0.87	36.9|1.06	38.3|0.92	38.5|0.71
TrPMBM [58]	43.4|0.93	37.8|0.84	40.3|0.92	38.8|0.96	44.5|0.97	37.9|0.90	41.4|0.89	37.3|1.05	38.8|0.78	42.7|0.97
TPMBM [59]	48.7|1.38	50.3|1.22	41.8|0.93	40.0|0.97	49.7|1.44	47.6|0.82	45.1|1.15	39.9|0.92	46.2|1.50	44.8|1.26
Gaussian CD-PMBM [60]	49.8|1.31	44.2|1.30	44.4|1.42	44.7|1.22	48.1|1.13	44.3|1.17	42.7|1.28	40.1|1.49	45.6|1.48	45.1|1.51
MEM-EKF [61]	35.6|0.86	36.4|0.63	35.8|0.64	32.1|0.68	40.2|0.69	34.8|0.72	31.9|0.70	38.7|0.83	32.3|0.82	33.2|0.72
ByteTrack [62]	43.9|0.82	41.6|0.79	46.6|0.87	42.8|0.90	48.4|0.92	47.0|0.91	47.7|0.78	42.1|1.04	45.3|0.79	44.6|0.99
CMTrack [63]	41.7|0.89	43.5|0.91	45.0|0.82	43.3|0.94	47.1|0.75	44.8|0.89	47.0|0.69	41.8|1.05	44.5|0.81	45.3|0.75
AdapTrack [64]	43.1|1.54	43.7|1.70	46.0|1.86	44.8|1.58	49.6|1.25	48.3|1.38	49.2|1.24	45.1|1.83	46.0|1.60	45.2|1.52
Deep-EIoU [65]	42.0|0.99	44.5|1.12	43.0|0.92	42.6|0.93	45.4|0.97	44.1|0.87	45.2|0.74	42.0|0.99	44.3|0.81	44.9|0.88
BoostTrack [66]	26.6|1.67	26.7|1.48	27.3|1.61	23.6|1.54	27.4|1.54	30.9|1.47	27.1|1.24	25.5|1.69	27.3|1.32	29.6|1.25
UIMM-Tracker	40.5|0.50	44.8|0.48	46.6|0.43	44.1|0.45	51.2|0.46	48.4|0.34	48.9|0.32	42.4|0.56	46.1|0.33	45.7|0.35

. As observed in Table 1, the UIMM-Tracker achieves the best performance in tracking, association, and localization. Compared to the second-ranked traditional algorithm TPMBM, the UIMM-Tracker improves MOTA and HOTA by 0.3% and 0.4%, respectively, while reducing IDs and precision by 3 and 0.78, respectively. Compared to the optimal deep-learning method AdapTrack, although the UIMM-Tracker shows a 0.5% decrease in MOTA, it achieves a 1.5% improvement in HOTA and reduces IDs and precision by 7 and 1.1, respectively. These improvements benefit from the adaptability of the IMM motion model, the autonomous adjustment capability of the dynamic Markov transition matrix, and the accurate representation of observation noise through the measurement of detection uncertainty.

According to the evaluation results in Table 2 for different sequences, the UIMM-Tracker achieves the best performance on sequences with varying target motion states (Seq3, Seq5, Seq6, Seq7, Seq8, and Seq10). For sequences with highly maneuvering targets (Seq5, Seq8, and Seq10), the proposed method also performs best, while the second-best methods, including AdapTrack, TPMBM, and MEM-EKF, vary across sequences. This indicates that other methods are limited to specific scenarios, whereas the proposed method demonstrates stronger generalization ability due to the integration of detection uncertainty measurement, the IMM model, association cost calculation with a joint strategy, and the dynamic Markov transition matrix. It satisfies application requirements under diverse conditions, including different backgrounds, target motion states, and target scales. Notably, the proposed method achieves the highest localization accuracy across all sequences. This is attributed to the use of multiple models and the dynamic Markov transition matrix, which aligns the state prediction process more closely with the actual target motion model, improving prediction accuracy. Moreover, accurate measurement of detection uncertainty ensures a precise representation of the true distribution of observations, enhancing the accuracy of the state update process.

To comprehensively compare the performance differences between our algorithm and the comparison methods, we use significance tests to assess the statistical significance of the experimental results, employing t-tests to analyze mean differences among various methods. According to Table 2, the means of the UIMM-Tracker in MOTA and precision metrics are significantly higher than those of the comparison methods. The visualization of p-value statistics is shown in Figure 5, indicating that the p-values for both metrics between our method and others are close to zero, demonstrating statistical significance in algorithm differences. Additionally, to ensure the robustness of the results, we conducted an ANOVA analysis to verify the interactions among multiple metrics. As shown in Figure 6, the analysis results further support the conclusions of the t-test, confirming that the differences between our method and the comparison methods are statistically significant. This confirms that the impact of the experimental treatment on metrics is not due to random factors but directly reflects differences in tracker performance. Therefore, the UIMM-Tracker method demonstrates significant superiority.

4.2.2. Qualitative Analysis

To intuitively demonstrate the advantages of the UIMM-Tracker in scenario adaptability and tracking performance, we visualize the tracking results of different trackers on the 10 sequences listed in Table 2 in Section 4.2.1, as shown in Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16.

The results indicate that the UIMM-Tracker adapts well to various complex environments, target motion states, and ID switches. For example, in Figure 7, VB-EOT-SN and MEM-EKF produce more mismatches, while the UIMM-Tracker generates fewer erroneous trajectories. In Figure 8, due to a large number of false positives caused by the striped background during detection coupled with background motion, other methods produce more incorrect trajectories, whereas UIMM-Tracker keeps these errors at a lower level. In Figure 9, the motion of fragmented clouds and temporal brightness variations lead methods like BoostTrack to incorrectly associate cloud motion with targets, resulting in numerous ID switches. However, UIMM-Tracker robustly tracks the targets. In Figure 10, strong noise interferes with detection results, causing discontinuities in the detected points of trajectories. Consequently, methods like MEM-EKF produce more ID switches, while UIMM-Tracker achieves the best tracking performance due to its multi-model configuration. In Figure 11, methods like TPMBM generate continuous erroneous trajectory segments, whereas UIMM-Tracker’s unique association strategy effectively suppresses clutter-induced trajectories. In Figure 12, due to trajectory interactions and limitations in detector capability, VB-EOT-SN produces errors in tracked target trajectories, while UIMM-Tracker adapts well to such situations, achieving the best tracking results. In Figure 13, the high contrast of the sea–land boundary and occasional satellite motion result in numerous erroneous trajectories. The proposed method demonstrates the best clutter trajectory suppression capability. Due to the presence of highly maneuvering targets, TPMBM fails to track the target during its initial maneuver, causing ID switches, whereas UIMM-Tracker maintains stable tracking. In Figure 14, the motion states of each target vary, and methods using a single-motion model lead to frequent ID switches. The proposed method achieves continuous target tracking over long time spans. In Figure 15, trajectory intersections cause MEM-EKF to associate two targets as one, while UIMM-Tracker effectively utilizes prior state estimates of each target to prevent trajectory interference. In Figure 16, due to targets being submerged in a noisy background, MEM-EKF and ByteTrack struggle to accurately match targets, resulting in erroneous associations and ID switches. UIMM-Tracker consistently tracks targets stably while maintaining fewer erroneous trajectories.

In summary, due to the unique design of the UIMM-Tracker in model probability transitions and data association, it adapts to complex environmental backgrounds and various target motion states while maintaining low error associations, ultimately achieving the best overall performance compared to other methods.

The analysis of single targets with both simple and complex maneuvering states evaluates the time-varying precision of different methods, as shown in Figure 17. From Figure 17a,b, it is evident that all methods successfully track simple trajectories. Although MEM-EKF converges faster, its precision is lower compared to methods like PMP-EOT-BP. The TrPMBM method ranks average among all methods in both convergence speed and precision. UIMM-Tracker achieves the best precision and the fastest convergence, owing to the IMM model with a dynamic Markov transition matrix, which autonomously updates the current estimate based on the target’s historical motion states. Instead of predicting the target state using a single-motion equation, it describes target motion as a weighted combination of multiple basic motions, significantly enhancing algorithm performance.

From Figure 17c,d, it is observed that Gaussian CD-PMBM and TPMBM initially succeed in tracking the target but fail during the first strong maneuver, with their tracking trajectories gradually deviating from the ground truth. Compared to other methods that successfully track the target, UIMM-Tracker achieves the best tracking precision. Whether before the target maneuver or after the first and second maneuvers, UIMM-Tracker converges the fastest, fully demonstrating the effectiveness of the proposed method.

4.2.3. Efficiency Comparison

Table 4. Performance and efficiency comparison of different algorithms. Bold indicates the best value, while the second-best is underlined.

Methods	MOTA ↑	HOTA ↑	Precision ↓	Time (s) ↓
VB-EOT-SN [56]	37.2	46.1	0.82	2.65
PMB-EOT-BP [57]	41.8	54.3	0.90	9.07
TrPMBM [58]	40.5	52.7	0.93	18.63
TPMBM [59]	45.3	55.8	1.19	17.29
Gaussian CD-PMBM [60]	44.7	53.9	1.31	17.95
MEM-EKF [61]	35.0	44.5	0.74	1.71
ByteTrack [62]	45.0	55.2	0.88	1.42
CMTrack [63]	44.4	52.9	0.85	2.47
AdapTrack [64]	46.1	54.7	1.55	1.51
Deep-EIoU [65]	43.8	54.6	0.92	0.93
BoostTrack [66]	27.2	33.1	1.48	4.18
UIMM-Tracker	45.6	56.2	0.41	2.81

presents the time required by different algorithms to process one frame. For a fair comparison, deep-learning methods perform inference on the CPU. Deep-learning methods exhibit faster runtime compared to traditional methods but require significant time investment during the training phase. Compared to traditional methods, UIMM-Tracker achieves a processing time of 2.81 s per frame, demonstrating higher overall efficiency. MEM-EKF and VB-EOT-SN rank as the top two in speed, but their performance is suboptimal. Our method processes each frame only 0.16 s slower than VB-EOT-SN yet delivers significantly better performance. Additionally, compared to the second-ranked TPMBM method in terms of performance, our method offers a substantial advantage in processing time, highlighting its ability to effectively balance efficiency and speed.

The computational complexity of the UIMM-Tracker depends on the number of targets $N_t$, the number of motion models $M$, the state dimension $n_x$, the measurement (detection result) dimension $n_z$, and the number of sigma points $2n_x+1$. During the input data interaction phase, each target requires the calculation of the model transition probability matrix $M\times M$ times. Cholesky decomposition is used to generate sigma points for covariance calculation with a complexity of $O(n_x^3)$. Therefore, the computational complexity for $N_t$ targets in the data interaction phase is $O(M^2\cdot N_t\cdot n_x^3)$. In the state prediction phase, each sigma point requires $n_x\times n_x$ calculations to obtain the predicted value, resulting in a computational complexity of $O(2(n_x+1)\cdot n_x^2)$. The computational complexity for covariance prediction is $O(n_x^3)$. Therefore, the total complexity for $M$ models and $N_t$ targets is $O(M\cdot N_t\cdot n_x^3)$. The data association phase uses the Hungarian algorithm, with a computational complexity of $O(N_t^3)$. In the measurement update phase, the cross-covariance matrix requires $n_z\times n_z$ calculations for each sigma point, with a complexity of $O(2(n_x+1)\cdot n_z^2)$. The complexity for the inversion of the measurement covariance matrix to compute the Kalman gain is $O(n_z^3)$. Therefore, the total complexity for $M$ models and $N_t$ targets is $O(M\cdot N_t\cdot (n_x^3+2(n_x+1)\cdot n_z^2))$. In the model probability update and fusion phase, the likelihood functions for each model and the updated model probabilities are calculated. The likelihood function calculation involves the inversion of the covariance matrix, with a computational complexity of $O(M\cdot N_t\cdot n_z^3)$. Each target requires $M$ calculations to update model probabilities with a complexity of $O(M\cdot N_t)$. Therefore, the total complexity for this phase is $O(M\cdot N_t\cdot n_z^3)+O(M\cdot N_t)$. The dynamic adjustment of Markov transition probabilities is proportional to the matrix size, requiring $M\times M$ calculations, resulting in a complexity of $O(M^2)$. Considering all the aforementioned processes, the overall computational complexity is approximately $M\cdot N_t\cdot O(n_x^3+n_z^3)$.

4.3. The Ablation Study

4.3.1. Impact of Detection Uncertainty

To evaluate the impact of detection uncertainty on the UIMM-Tracker, we design the following three experiments to verify the algorithm’s performance:

(1) Without $R_k$, it is simply set to 0. In this case, the association process does not consider the impact of uncertainty, which is used to evaluate the effect of observation noise on target tracking performance.

(2) $R_k$ from prior, where $R_k=\mathrm{diag}(0.09,0.09,0.16,0.16)$. In this case, the association process does not consider the impact of uncertainty, which is used to evaluate the effect of given observation noise on target tracking performance.

(3) $R_k$ from detections, where $R_k$ is measured from detection results and used as input to the tracking algorithm, which is used to evaluate the performance of the target tracking algorithm under accurate observation noise.

The experimental results on the impact of detection uncertainty on the performance of target tracking algorithms are shown in

Table 5. The impact of different $R_k$ acquisition methods on the performance of UIMM-Tracker is analyzed. The optimal values are highlighted in bold.

The Method for Obtaining ${\mathbf{R}}_{\mathit{k}}$	MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↓
Without $R_k$	40.4	50.7	49.8	58.1	52	0.84
$R_k$ from prior	45.0	55.1	55.4	67.8	38	0.52
$R_k$ from detections	45.6	56.2	55.7	68.5	35	0.41

. The following conclusions can be drawn:

(1) When the observation noise $R_k$ is not provided, the algorithm exhibits the worst performance, with MOTA, HOTA, AssA, and IDF1 metrics decreasing by 5.2%, 5.5%, 5.9%, and 10.4%, respectively, while IDs and precision increase by 17 and 0.43, respectively. The algorithm’s performance is severely degraded at the tracking and association levels, and significant errors are introduced at the localization level. This is because observation systems are inherently non-ideal, and observation noise must be included in the measurement and prediction process. Without $R_k$, the observed values deviate significantly from the true values, disrupting the data association process and causing association errors. This leads to degraded performance in target tracking and association, resulting in incorrect state updates, erroneous state estimation, and reduced target localization accuracy.

(2) When the observation noise is provided a priori, the algorithm’s performance is slightly lower compared to when the noise is obtained through measurement. This is due to the inaccuracy of the given noise. Although the provided noise partially reflects the distribution range of the observations, it does not represent the true distribution but rather an approximation based on human experience. As a result, significant deviations still exist between the observed and actual values, leading to data association errors. Additionally, under such conditions, the cost function in the association algorithm does not account for the distribution range of the cost values or additional cost computation methods, further impacting the performance of target association.

(3) The observation noise $R_k$ obtained through measurement based on detection results achieves the best performance. Compared to the observation noise provided a priori, the algorithm improves by 0.6%, 1.1%, 0.3%, and 0.7% in MOTA, HOTA, AssA, and IDF1 metrics, respectively, while IDs and precision decrease by 3 and 0.11, respectively. This is because the detection results serve as the observations for the multi-object tracking system, inherently incorporating the imaging system’s intrinsic errors and the detection algorithm’s localization errors, which are ultimately presented in the form of multiple bounding boxes. The covariance measurements obtained during the non-maximum suppression process effectively represent the fusion of all these errors, providing clear and accurate observation noise inputs for the multi-object tracking algorithm. This minimizes association errors to the greatest extent, supplies accurate observations for state updates, and ultimately ensures the accuracy of target localization.

4.3.2. The Impact of the Markov Transition Matrix

To evaluate the impact of the Markov transition matrix on the UIMM-Tracker, we design the following five experiments to verify the algorithm’s performance:

(1) A priori transition matrix $M_k=\left[0.8,0.1,0.1;0.1,0.8,0.8;0.1,0.1,0.8\right]$ is used to evaluate the target tracking performance of the IMM algorithm under fixed model transition probabilities.

(2) A dynamic Markov transition matrix is employed without considering the dynamic modulation effect of the state measurement error ${\widetilde{ep}}_k^j$ on $M_k$ to evaluate the impact of modulation of ${\widetilde{ep}}_k^j$ on target tracking performance.

(3) A dynamic Markov transition matrix is employed without considering the dynamic modulation effect of the model likelihood ${\tilde{\Lambda }}_k^j$ on $M_k$ to evaluate the impact of modulation of ${\tilde{\Lambda }}_k^j$ on target tracking performance.

(4) A dynamic Markov transition matrix is employed without considering the dynamic modulation effect of the model probability rate of change ${\rho }_k^j$ on $M_k$ to evaluate the impact of modulation of ${\rho }_k^j$ on target tracking performance.

(5) A dynamic Markov transition matrix is employed comprehensively considering the dynamic modulation effects of ${\widetilde{ep}}_k^j$, ${\tilde{\Lambda }}_k^j$ and ${\rho }_k^j$ on $M_k$ to demonstrate the performance advantages of the algorithm.

The experimental results of different Markov transition matrices are shown in

Table 6. The impact of different Markov transition matrix construction methods on the performance of the UIMM-Tracker. Bold values indicate optimal results.

Markov Transition Matrix Construction Methods			MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↑
${\widetilde{\mathit{e}\mathit{p}}}_{\mathit{k}}^{\mathit{j}}$	${\tilde{\mathbf{\Lambda }}}_{\mathit{k}}^{\mathit{j}}$	${\mathit{\rho }}_{\mathit{k}}^{\mathit{j}}$
✗	✗	✗	35.2	47.5	51.4	62.2	59	0.67
✗	✓	✓	44.1	53.4	55.9	68.1	42	0.45
✓	✗	✓	44.6	53.5	55.3	68.2	43	0.44
✓	✓	✗	40.3	51.2	53.6	65.4	50	0.55
✓	✓	✓	45.6	56.2	55.7	68.5	35	0.41

, leading to the following conclusions:

(1) The algorithm performs the worst under the condition of a given Markov transition matrix, with MOTA, HOTA, AssA, and IDF1 decreasing by 10.4%, 8.7%, 4.3%, and 6.3%, respectively, while IDs and precision increase by 24 and 0.26, respectively. This is because the algorithm cannot adaptively track target trajectory changes in maneuvering states based on historical trajectory information, leading to inaccuracies during the multi-model fusion process and resulting in significant state estimation errors. These errors severely impact the processes of association, tracking, and localization, causing a comprehensive decline in all performance metrics.

(2) During the dynamic adjustment of the Markov transition matrix, temporal features incorporating historical information outperform spatial features that only consider current measurements. The tracking results without considering ${\rho }_k^j$ are worse than those without considering ${\widetilde{ep}}_k^j$ or ${\tilde{\Lambda }}_k^j$. Compared to the results without considering ${\widetilde{ep}}_k^j$, the MOTA, HOTA, AssA, and IDF1 metrics decrease by 3.8%, 2.2%, 2.3%, and 2.7%, respectively, while IDs and precision increase by 8 and 0.10, respectively. Compared to the results without considering ${\tilde{\Lambda }}_k^j$, the MOTA, HOTA, AssA, and IDF1 metrics decrease by 4.3%, 2.3%, 1.7%, and 2.8%, respectively, while IDs and precision increase by 7 and 0.11, respectively. This is because the model probability rate of change ${\rho }_k^j$ represents the degree of variation in the matching between the target motion state and each model, capturing the temporal trend of the target motion model and its affinity with each motion model. This allows for a more comprehensive adjustment of the Markov transition matrix, thereby influencing the processes of target association, tracking, and localization.

(3) The algorithm that comprehensively considers ${\widetilde{ep}}_k^j$, ${\tilde{\Lambda }}_k^j$ and ${\rho }_k^j$ achieves the best tracking performance. Compared to the second-best algorithm, which does not consider ${\widetilde{ep}}_k^j$, it improves MOTA and HOTA by 1.5% and 2.8%, respectively, while reducing IDs and precision by 7 and 0.04, respectively. This fully demonstrates the importance of measurement error, model likelihood probability, and the rate of change in model matching for multi-model data fusion. By incorporating dynamic modulation with multiple factors, the Markov transition matrix better facilitates the matching process between the target and the model under changing motion states. It ensures accuracy while avoiding overly frequent or unreasonable model switches, resulting in a more natural and smoother model transition.

4.3.3. The Impact of Association Methods

To evaluate the impact of the cost calculation method on the UIMM-Tracker, we design the following four experiments to verify the algorithm’s performance:

(1) Excluding uncertainty and not introducing the additional cost $U$ calculation to verify the importance of incorporating uncertainty in the target data association process.

(2) Considering the uncertainty of detection results but excluding scale constraints to verify the impact of the scale consistency assumption on target data association.

(3) Considering the uncertainty of detection results but excluding energy constraints to verify the impact of the energy consistency assumption on target data association.

(4) Considering the uncertainty of detection results and comprehensively incorporating both scale and energy constraints to verify the impact of multiple hypothesis conditions on target data association.

The experimental results of different cost calculation methods are shown in

Table 7. The impact of different cost calculation methods on the performance of the UIMM-Tracker. Bold values indicate optimal results.

Cost Calculation Methods			MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↑
Uncertainty	$U^{\mathit{s}}$	$U^{\mathit{e}}$
✗	--	--	42.5	51.3	50.5	63.7	49	0.64
✓	✗	✓	44.7	54.4	54.8	67.2	41	0.46
✓	✓	✗	45.4	56.5	54.3	67.0	39	0.44
✓	✓	✓	45.6	56.2	55.7	68.5	35	0.41

, leading to the following conclusions:

(1) When uncertainty is not considered, the algorithm achieves the worst tracking performance, with MOTA, HOTA, AssA, and IDF1 decreasing by 3.1%, 4.9%, 5.2%, and 4.8%, respectively, while IDs and precision increase by 14 and 0.23, respectively. This is because, without considering uncertainty, the IoU and NWD hybrid distance is a single value, failing to capture the distribution characteristics of the cost. During Hungarian matching, trajectories and detections that should have uncertain matches are uniquely assigned, resulting in errors in data association and ultimately affecting the algorithm’s performance in association, tracking, and localization.

(2) When considering certainty, the shape consistency constraint of the target is more important than the energy consistency constraint. Compared to the results without $U^s$, the results without $U^e$ show a decrease of 0.7% and 2.1% in MOTA and HOTA, respectively, and a decrease of 0.02 in precision. This is because the scale characteristic in cost calculation represents an absolute property of the target, while the energy characteristic, described by the signal-to-clutter ratio, is a relative property between the target and the background. Since the absolute energy property of the target cannot be obtained directly from the image, the scale consistency of the target imposes a stronger constraint.

(3) By jointly considering shape and scale constraints, the algorithm achieves optimal performance. This is because the cost calculation process does not rely solely on the absolute scale invariance assumption or the relative energy invariance assumption but instead leverages the fusion of both. This allows the algorithm to adapt to scenarios that partially deviate from a single assumption, resulting in better performance compared to cases where $U^s$ or $U^e$ is not considered.

4.3.4. The Impact of Multi-Model

To evaluate the impact of different motion models on the UIMM-Tracker, we design the following four experiments to verify the algorithm’s performance:

(1) CV model only: The CV model is used exclusively to predict and update the target state, adapting to uniform linear motion, to evaluate the algorithm’s performance.

(2) CA model only: The CA model is used exclusively to predict and update the target state, adapting to accelerated linear motion, to evaluate the algorithm’s performance.

(3) CT model only: The CT model is used exclusively to predict and update the target state, adapting to turning motion, to evaluate the algorithm’s performance.

(4) Complete IMM: The CV, CA, and CT models are jointly used to update the target state and perform probabilistic model fusion, meeting the requirements of tracking targets with complex motion states such as accelerated linear motion and turning, thereby verifying the performance advantages of the multi-model tracking algorithm.

The algorithm’s performance under different motion models is shown in

Table 8. The impact of different motion models on the performance of the UIMM-Tracker. Bold indicates optimal values.

Motion Model			MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↑
CV	CA	CT
✓	✗	✗	39.4	48.7	46.2	58.8	65	0.75
✗	✓	✗	44.2	54.5	53.6	65.1	40	0.49
✗	✗	✓	26.3	34.8	39.2	41.4	116	1.66
✓	✓	✓	45.6	56.2	55.7	68.5	35	0.41

. Since the targets in the scene predominantly exhibit accelerated linear motion, the method considering only the CT model performs the worst. Compared to the method integrating all models, it shows decreases of 19.3%, 21.4%, 16.5%, and 27.1% in MOTA, HOTA, AssA, and IDF1, respectively, while IDs and precision increase by 81 and 1.25, respectively. Similarly, the CA model outperforms the CT model as it accounts for the influence of acceleration. However, it still shows decreases of 1.4%, 1.7%, 2.1%, and 3.4% in MOTA, HOTA, AssA, and IDF1, respectively, with increases of 5 and 0.08 in IDs and precision, respectively. The method integrating all motion models achieves the best performance, as it not only adapts to most accelerated linear motion scenarios but also incorporates the CT model to maintain robustness when the target executes turning maneuvers.

The visualization results of tracking performance for targets under different motion states using various motion models are shown in Figure 18. From Figure 18a,b, it is evident that all models successfully track targets with simple motion states. However, the IMM model converges faster and achieves the highest precision, highlighting the importance of the hybrid model. From Figure 18c,d, it can be observed that all models successfully track highly maneuvering targets. Nevertheless, in terms of convergence speed and localization accuracy, the IMM model demonstrates the best performance, indicating its adaptability for tracking highly maneuvering small infrared targets in remote-sensing images.

4.3.5. Comparison of Tracking Performance Under Different Detection Qualities

As an algorithm based on the TbD paradigm, the UIMM-Tracker’s tracking performance is constrained by detection capabilities. To assess the potential impact of detector performance on tracking algorithm performance, we compare tracking performance using detection results from the RDIAN [72], DNANet [73], LMAFormer [11], SSTNet [74], and LASNet [71] detectors as inputs. As shown in

Table 9. Comparison of UIMM-Tracker algorithm performance under different detection inputs. Bold indicates optimal values.

Detector	Precision (%)	Recall (%)	MOTA ↑	HOTA ↑	AssA ↑	IDF1 ↑	IDs ↓	Precision ↓
RDIAN [72]	79.98	65.83	38.4	46.5	49.2	64.8	46	0.41
DNANet [73]	80.84	71.97	42.7	55.8	52.4	67.0	45	0.42
LMAFormer [11]	81.06	71.29	42.6	55.1	52.3	66.9	38	0.40
SSTNet [74]	83.95	69.51	43.3	53.6	52.8	67.2	36	0.43
LASNet [71]	85.32	73.64	45.6	56.2	55.7	68.5	35	0.41

, experimental results indicate that detector performance directly affects target tracking performance. Higher detection performance enhances tracker capabilities, with metrics such as MOTA, AssA, IDF1, and IDs influenced by both precision and recall. The HOTA metric is mainly affected by recall. Due to the small pixel count occupied by space-based infrared targets, detector positioning accuracy is at the pixel level, and the UIMM-Tracker incorporates detection uncertainty measurements, resulting in consistent precision metrics across different detectors.

4.3.6. Analysis of Model Hyperparameters

The hyperparameters of the UIMM-Tracker have a direct impact on tracking results, primarily involving the computation of the cost matrix, including the balance parameter C for IoU and NWD costs, the ratio r of background to target radius for calculating the signal-to-clutter ratio, and the weight λ for balancing target scale and energy cost. We conduct separate analyses of each hyperparameter to evaluate the effects of different settings on tracking and state estimation accuracy using the metrics MOTA and precision.

(1) Hyperparameter C for balancing IoU and NWD costs: As shown in Figure 19a, when C ranges from 1 to 5 with a step size of 1, the MOTA and precision metrics are optimal at C = 4. According to Equation (12), when $C=\sqrt{A}$, the weights of IoU and NWD are equally distributed, where A represents the area of the detection box. Given that the average target scale in the dataset is 4 × 4, the best balance between IoU and NWD is achieved at C = 4. When the target scale is smaller than 4 × 4, $-(\sqrt{A}/C-1)>0$, giving more weight to the NWD cost. Conversely, when the target scale is larger than 4 × 4, $-(\sqrt{A}/C-1)<0$, causing the cost matrix to be dominated by the IoU measure.

(2) The ratio r of background to target radius in the signal-to-noise ratio calculation: As shown in Figure 19b, when r ranges from 1 to 5 with a step size of 1, the MOTA and precision metrics are optimal at r = 3. This is because each pixel in infrared sensor imaging contains thermal noise. If the background area is too small, the clutter fluctuation ${\sigma }_{q_s}^B$ in Equation (17) mainly reflects thermal noise, failing to accurately characterize the energy difference between the target and background. Conversely, if the value is too large, the calculation of ${\sigma }_{q_s}^B$ introduces additional scene information, leading to inaccurate results.

(3) The balance weight λ for target scale and energy cost: As shown in Figure 19c, when λ ranges from 0.1 to 0.9 with a step size of 0.2, the MOTA and precision metrics are optimal at λ = 0.7. This is because the target scale measures the intrinsic properties of the target, independent of flight speed, maneuvering state, or background, thus offering greater continuity and consistency. In contrast, the energy cost, measured by the signal-to-clutter ratio (SCR), is highly dependent on the background, showing weaker continuity and consistency as the target moves across different backgrounds. Therefore, a value of λ = 0.7 allows the cost calculation to rely more on the scale consistency assumption while also considering energy consistency, thereby enhancing the accuracy of target association.

5. Conclusions

To address challenges such as trajectory interference, association difficulties, and localization inaccuracies in small infrared target tracking under complex remote-sensing backgrounds, this paper introduces detection uncertainty measurements and a dynamic Markov transition matrix into the IMM framework, proposing a method named UIMM-Tracker. The method calculates the covariance of target bounding boxes during the non-maximum suppression (NMS) process as a quantitative description of detection uncertainty. This uncertainty is injected into the observation noise of the IMM to accurately represent the actual distribution of observations, thereby improving the precision of target state estimation. Furthermore, detection uncertainty measurements are incorporated into the data association process. A weighted value of IoU and NWD is adopted as a distance metric, extending the single distance metric between trajectories and detections into a distribution range. Based on this, an auxiliary distance metric is constructed under the assumption of temporal scale and energy invariance of the target, thereby eliminating ambiguous matches between trajectories and detections and enhancing the robustness and accuracy of the data association process. Additionally, this paper proposes a dynamic modulation method for the Markov transition matrix, based on the historical model probability variation rate, current model likelihood probability, and state prediction error. This allows the algorithm to autonomously adjust the weighting of different motion models according to the target’s state changes, enabling adaptation to complex and maneuvering targets and ultimately improving the accuracy of target state prediction. Finally, the proposed algorithm is validated on 20 publicly available complex remote-sensing infrared image sequences. Compared to various classical methods, our approach achieves state-of-the-art performance while maintaining a good balance between accuracy and efficiency.

However, this method still has certain limitations. The assumptions of scale and energy invariance are used as auxiliary means in the cost computation process. When the current cost values are ambiguous, additional assumptions are introduced to resolve the ambiguity. However, the scale of a target is influenced by its inherent size as well as its relative position to the infrared satellite, leading to fluctuations in the target’s size on the image plane within a certain range. The direct assumption of scale invariance cannot fully capture these fluctuations. Similarly, the signal-to-clutter ratio (SCR) of the target also fluctuates with changes in its motion state. The energy invariance assumption simplifies and approximates the modeling of the relative energy relationships between frames, introducing a certain degree of error. Therefore, precise modeling of target scale and energy variations will be a key focus of future research.

In the future, we will focus on studying the interaction and relationships between trajectories and exploring behavior transfer among dense targets to achieve high-precision trajectory determination and reduce interference. Additionally, considering the widespread cloud distribution in space-based infrared imaging scenarios, target trajectories may become discontinuous when passing through cloud layers. To address this, we design a motion-feature-driven appearance re-identification network based on the spatiotemporal characteristics of infrared small targets. This network aims to connect disordered trajectories under cloud-penetrating conditions, enabling the continuous and robust generation of target trajectories.