Highlights
What are the main findings?
- We propose an improved end-to-end multi-object tracking framework based on MOTRv2, incorporating geometric prior injection via sine–cosine encoding, width–height-modulated deformable attention for scale-adaptive feature aggregation, and a motion-direction-consistency-based trajectory re-association mechanism.
- Our method achieves state-of-the-art performance on multiple challenging datasets (52.4% HOTA and 72.1% MOTA on UAVSwarm, 48.1% HOTA on BEE24, and 42.3% HOTA on the VTMOT infrared dataset), with significantly reduced identity switches (only 51 on UAVSwarm).
What is the implication of the main findings?
- The proposed framework effectively addresses the key challenges in high-density, highsimilarity group tracking scenarios (such as UAV swarms and biological collectives) by leveraging query propagation and geometric priors, offering a robust solution that surpasses traditional approaches relying on explicit motion estimation (e.g., Kalman filtering) and appearance feature extraction (e.g., Re-ID models).
- The synergistic effect of our proposed modules—specifically the integration of scalemodulated attention with motion-consistency-based re-association—underscores the value of combining geometric priors and motion continuity. This approach substantially enhances tracking stability, particularly in scenarios where traditional appearance-based methods often struggle.
Abstract
The query propagation paradigm provides a unified theoretical framework for end-to-end multi-object tracking, yet it still faces challenges in complex scenarios involving multi-scale variations, dense interactions, and trajectory fragmentation, including insufficient query initialization quality, imprecise feature alignment, and difficult identity recovery. Building upon MOTRv2, this paper proposes three core improvements. First, we design a geometric prior injection strategy based on sine–cosine encoding, which explicitly encodes target location and scale information into detection queries, providing high-quality initialization for tracking queries. Second, we propose a width–height-modulated deformable attention mechanism that dynamically adjusts the sampling range of deformable convolution according to target size, enabling fine-grained feature matching for multi-scale targets. Third, we construct a motion-direction-consistency-based trajectory re-association module that leverages motion continuity to efficiently recover lost trajectories without introducing additional appearance models. Furthermore, we introduce a progressive joint training strategy that optimizes detection and tracking modules in stages, effectively mitigating gradient competition in multi-task learning. Extensive quantitative and qualitative experiments on the BEE24, UAVSwarm, and VTMOT infrared datasets validate the effectiveness of the proposed method. On the UAVSwarm dataset, our method achieves state-of-the-art performance with 52.4% HOTA, 72.1% MOTA, and only 51 identity switches. Ablation studies further reveal the synergistic enhancement mechanism among the proposed modules.
1. Introduction
Multi-target tracking (MOT), as one of the core tasks in the field of computer vision, aims to continuously localize multiple targets of interest in video sequences and associate their trajectories across frames. This process can be formulated as follows: given a video sequence , the objective of MOT is to output a set of trajectories , where each trajectory contains the bounding boxes and the unique identity identifier of the target at each time step [1].
Currently, the mainstream technical paradigm in the field of multi-object tracking widely adopts the Tracking-by-Detection architecture. In this paradigm, the association module models the appearance or motion features of targets across sequential frames to achieve data association between trajectories and detections. Specifically, the extraction of appearance features usually relies on re-identification (Re-ID) models [2], while motion features are modeled via Kalman filters [3] or other motion predictors. Based on these features, the system constructs a cross-frame similarity measurement matrix among multiple targets, thereby transforming the trajectory association problem into a bipartite graph matching problem. Optimization methods, such as the Hungarian algorithm, are then employed to solve the optimal matching between M trajectories and N detection boxes in each frame. Representative works following this paradigm, such as DeepSORT [4], ByteTrack [5], and CenterTrack [6], have achieved notable success on multiple public multi-object tracking datasets (e.g., MOT17 and DanceTrack), demonstrating their effectiveness and robustness in general scenarios.
However, when such methods are applied to specific high-density, high-similarity group scenarios, their performance declines significantly due to the following inherent challenges: (1) High appearance similarity—individuals in the cluster share nearly identical visual features, greatly reducing the discriminative power of Re-ID models. (2) Complex motion patterns—group movement is highly nonlinear and non-stationary, making linear motion models such as the Kalman filter inadequate for accurate prediction. (3) Frequent dynamic interactions—dense crossing flights, transient occlusions, and rapid positional changes among individuals increase ambiguity in data association. (4) Severe deformation and scale variation—targets undergo notable shape and scale changes across frames due to fast viewpoint shifts, further complicating feature matching, as shown in Figure 1.
Figure 1.
Challenges in multi-object tracking: (a) Swarms with highly similar individual appearance features. (b) Fixed-wing UAVs exhibiting complex nonlinear motion. (c) Rotary-wing UAVs with small target sizes and frequent crossing occlusions. (d) Helicopter clusters undergoing severe scale and viewpoint variations. In these scenarios, targets with different identities are annotated with bounding boxes of different colors.
To address the trajectory association dilemma in scenarios such as UAV swarms, which stems from high visual similarity and complex dynamic motion, an association mechanism that is capable of adaptively learning discriminative representations while possessing temporal reasoning ability is required. Traditional approaches relying on handcrafted features and association rules struggle to adapt to such dynamic environments. Inspired by recent advances in end-to-end multi-object tracking, this paper proposes an improved solution based on the MOTRv2 framework. The core strengths of this framework are threefold: first, it implicitly learns spatiotemporal joint representations of targets through a Transformer encoder–decoder architecture instead of relying on separate Re-ID or motion models; second, its tracking-query mechanism enables direct propagation of identity information across frames, transforming the association problem into a continuous optimization of queries; third, the attention mechanism allows the model to capture inter-target interactions, which is essential for understanding group motion patterns.
Based on the aforementioned challenges, this paper presents an improved end-to-end multi-object tracking method built upon the MOTRv2 framework [7]. Aiming at key challenges in group tracking, the method incorporates four core improvements: (1) a temporal query propagation mechanism that integrates detection geometric priors into tracking queries via sine–cosine encoding, enhancing initial localization and identity continuity; (2) a width–height-modulated deformable attention mechanism that achieves scale-adaptive feature aggregation, thereby improving matching accuracy for multi-scale targets; (3) a motion-direction-consistency-based trajectory re-association strategy, which leverages motion continuity to assist identity recovery, significantly reducing identity switches under complex interactions; (4) a progressive joint training strategy that optimizes detection and tracking modules in stages, mitigating task conflict and improving training stability and final performance.
The experimental results demonstrate that the proposed method achieves significant performance gains across several high-density, high-similarity group tracking datasets. On the BEE24 dataset, HOTA reaches 48.6% with only 1124 identity switches; on the UAVSwarm dataset, HOTA improves to 52.4%, MOTA reaches 72.1%, and identity switches are reduced to 51, surpassing all the existing state-of-the-art methods. Additionally, experiments on an infrared dataset verify the method’s strong generalization capability. Ablation studies further confirm the effectiveness and synergy of each proposed module, offering a robust and efficient solution for tracking group targets with high appearance similarity and complex motion patterns.
2. Related Works
Multi-object tracking methods can be categorized into three main paradigms based on their technical frameworks, each with distinct characteristics that reflect the evolution of the field.
Tracking-by-Detection: The Tracking-by-Detection paradigm follows a two-stage processing flow of “detect first, associate later” and represents the most mature and widely adopted tracking system. Its core idea is decoupling the tracking task into two independent sub-problems: first, object detection models localize targets of interest in each frame; then, data association algorithms link these discrete detection boxes into continuous trajectory sequences. This paradigm has produced a series of seminal works: the SORT [8] algorithm combines Kalman filtering with the Hungarian algorithm for motion prediction and data association, respectively, laying the foundation for efficient online tracking; DeepSORT [4] enhances SORT by introducing deep appearance descriptors, significantly improving re-identification capability in complex scenes and becoming a widely used benchmark in industry; ByteTrack [5] proposes a simple yet effective association strategy that leverages low-confidence detections to enhance robustness against occluded targets; OC-SORT [9] refines motion estimation through observation-centric modeling, improving adaptability to non-linear motion and camera jitter. The main strength of this paradigm lies in its modular design, which allows independent optimization of detection and association while maintaining relatively low overall computational complexity. StrongSORT [10] comprehensively upgrades the DeepSORT framework from multiple dimensions, including detector, appearance feature extractor, motion compensation, adaptive Kalman filtering, feature updating strategy, matching cost design, and matching algorithm. UniTrack [11] unifies detection and tracking into a shared representation space, enabling plug-and-play generalization without task-specific fine-tuning. MixSort-Byte [12] combines MixSort with ByteTrack’s strategy of handling high- and low-confidence detections separately to improve association robustness while maintaining real-time speed. PID-MOT [13] relies on motion prediction and appearance feature fusion, employing a dynamic matching strategy for reliable association. Hybrid-SORT [14] augments the SORT pipeline with both IoU and re-identification features in a hybrid matching scheme, significantly reducing identity switches. TOPICTrack [15] follows the detection-based tracking paradigm and adaptively selects either appearance or motion features for association depending on the scene context, enhancing flexibility in complex scenarios. However, its inherent limitation stems from the separation of detection and association, which can lead to inconsistent optimization objectives, propagation and amplification of detection errors, and difficulty in maintaining identity consistency under severe occlusion or drastic appearance changes.
Joint Detection and Tracking: To overcome the limitations of the two-stage paradigm, the joint detection and tracking paradigm emerged, whose core concept is to simultaneously accomplish object detection and temporal association within a unified framework through shared feature representation and co-optimization. This approach aims to reduce error propagation between intermediate representations and jointly improve detection accuracy and tracking stability. Representative works include: Tracktor [16], which innovatively leverages the bounding-box regression head of existing detectors for direct target propagation without an explicit association module; CenterTrack [6], which builds upon a center-point detection framework to jointly predict object locations and their displacements across adjacent frames, integrating detection and motion estimation; FairMOT [17], which integrates detection and re-identification branches in parallel on an anchor-free detection architecture and achieves balanced multi-task learning through an equitable loss design; TransCenter [18], which introduces the Transformer architecture into a dense-prediction tracking framework and models interactions between detection and tracking via cross-attention mechanisms. MM-Tracker [19] formulates cross-frame association as an end-to-end motion field prediction problem, thereby transforming the association task into motion field estimation. This paradigm improves computational efficiency through shared backbone networks and has the potential to mitigate error accumulation. TraDeS [20] leverages tracking cues to assist detection by propagating target features from past frames, enabling joint optimization of detection and association, especially in low-quality videos. CTrackerv1 [21] integrates convolutional networks with recurrent structures for end-to-end joint detection and tracking, enhancing feature stability through temporal information for dense drone-view scenes. UAVS-MOT [22] is specifically designed for drone-based videos, addressing challenges such as large viewpoint variations and drastic scale changes by combining motion prediction with appearance features. Nevertheless, due to inherent conflicts between the optimization objectives of detection and tracking, its performance heavily depends on carefully designed multi-task learning strategies and loss-balancing mechanisms, and its generalization ability across different scenarios is often limited.
Query-Propagation-Based Tracking: Inspired by the DETR (Detection Transformer) architecture, query-propagation-based tracking represents the cutting edge of end-to-end multi-object tracking. Its core idea is to represent each tracked target as a learnable query vector that evolves over time and is directly decoded into the target’s state trajectory, thereby realizing truly end-to-end tracking. Pioneering works in this paradigm include: TrackFormer [23], which first applied the Transformer query mechanism to multi-object tracking, implicitly performing data association through cross-frame query propagation; MOTR [24], which explicitly introduced the concept of “track queries” and constructed a complete end-to-end tracking framework without any post-processing association steps; MOTRv2 [7], which significantly boosted tracking performance by integrating a stronger detector and proposing tracking-aware query initialization strategies; GTR [25], which introduced a global association Transformer to explicitly model spatiotemporal relations among targets using graph neural networks; and MeMOTR [26], which combined memory-augmentation mechanisms to further improve handling of long-term occlusions and target reappearance. The key strengths of this paradigm lie in its architectural unity and theoretical completeness: through the self-attention mechanism of Transformers, it achieves global context modeling and avoids complex post-processing pipelines and handcrafted association rules. Although MixFormer [27] is a single-object tracking framework, its approach of encoding the target to be tracked as a query vector and interacting with global image features to achieve robust tracking is conceptually similar to the query propagation mechanism. However, it still faces notable challenges, including high training complexity, convergence difficulties, and substantial computational overhead, which impose high demands on computing resources in practical deployment.
3. Method
3.1. Overall Framework
This study proposes an improved end-to-end multi-object tracking framework, whose overall architecture is illustrated in Figure 2. The framework retains the query propagation mechanism of the MOTR series but introduces important modular restructuring. Specifically, the system consists of three core components: a multi-scale feature detection network (DetectNet) based on our prior work for generating detection proposals, an encoder for feature extraction, and a query interaction system responsible for temporal association. For each frame in a video sequence, DetectNet first produces initial detection proposals; then, the encoder extracts image features; finally, the detection queries (Detect Q) of the current frame interact with the tracking queries (Track Q) from the previous frame in the decoder together with the image features to jointly predict target states. Notably, the framework introduces a lost-query bank that temporarily stores queries of targets lost due to occlusion or departure from the field of view, enabling rapid identity recovery when the targets reappear. Compared with the unified encoder–decoder design of MOTR and the detector-integration scheme of MOTRv2, the proposed framework maintains the advantages of end-to-end learning while offering better modularity and adaptability to complex scenarios.
Figure 2.
This is a schematic diagram of the overall algorithm framework. The small black boxes represent detection queries generated by the detection network, while the small colored boxes represent the tracking queries of each trajectory in a single frame, with different colors corresponding to targets of different identities; the red arrows point to the specific targets they represent. The orange arrows indicate the propagation of tracking queries across consecutive frames.
3.2. Scale-Modulated Inter-Frame Query Propagation Mechanism
Unlike MOTRv2, we integrate the target’s position, size, category, confidence, and other parameters into the initial queries by fusing them via sine–cosine encoding with learnable content vectors. When the query vectors interact with the global image features, the target scale is used to modulate the computation of cross-attention, thereby enabling the model to achieve scale-aware generalization, as illustrated in Figure 3.
Figure 3.
Flowchart of the scale-modulated inter-frame query propagation mechanism. The black are generated by the detection network, while the red markings represent the tracking queries propagated across frames and their corresponding tracking results.
For each frame in the video sequence, a set of detection proposals is first generated by the detection network, where each detection proposal includes the target’s location, size, confidence score, and category information:
where denote the normalized coordinates of the bounding-box center, are the normalized width and height, is the confidence score, and is the target category label. To encode these geometric and semantic cues into query vectors, we first apply sine–cosine encoding to the position and size information:
where is the sine–cosine position encoding, is the sine–cosine size encoding, and is the confidence and category encoding following a similar principle. The encoded geometric features are then combined with a learnable shared query vector to generate the detection query (Detect Q):
The detection query carries the detection prior information of the current frame, providing high-quality initial localization for each trajectory. For the first frame of the sequence, the input queries to the decoder consist solely of detection queries. For subsequent frames, the input queries are formed by concatenating the detection queries with the tracking queries from the previous frame,
where denotes the updated tracking queries from the previous frame, which are responsible for maintaining identity continuity of existing targets; is responsible for discovering newly emerging targets; represents the number of targets present in the previous frame; and is the number of detection proposals provided by the detection network for the current frame. This dual-branch design achieves an organic unification of detection and tracking. The decoder core employs target-scale-modulated deformable attention for query–image feature interaction. For each query , its corresponding anchor-box geometric information is explicitly used to modulate the attention computation. Before calculating cross-attention, the query vector is fused with the geometrically encoded features,
where is a learnable projection matrix. The geometrically enhanced query representation enables it to carry positional prior information in the feature space. In traditional deformable attention, the sampling offsets are learned based on query content but ignore target scale. This mechanism introduces a width–height modulation factor to adaptively adjust the sampling range according to scale,
where and are learnable parameters, and is a small constant to prevent numerical overflow. The modulated sampling positions are given by
where ⊙ denotes element-wise multiplication. This design enables queries of large-scale targets to attend to wider receptive fields, while queries of small-scale targets focus on local fine-grained features, achieving scale-adaptive feature aggregation. The width–height-modulated deformable attention is computed as follows:
The attention weights are computed from the similarity between the query and the features at the modulated sampling points, where , and are projection matrices, and d is the feature dimension.
After being processed by multiple decoder layers, the output hidden states are fed into two separate multi-layer perceptron heads to predict bounding-box offsets and confidence scores, respectively:
For predictions derived from tracking queries , denotes the offset relative to the previous frame’s position; for predictions derived from detection queries , represents the correction relative to the detection proposal. The final bounding box is given by
Based on the predicted confidence score , the output queries are divided into three groups: active tracking queries represent successfully tracked targets; lost queries are moved into the lost-query bank as illustrated in Figure 4; and invalid queries are discarded directly. The active tracking queries are updated with their geometric information to form the tracking-query input for the next frame:
Figure 4.
When targets undergo crossing occlusions, temporary target loss may occur. For example, the target marked in cyan blue in the 48th frame of our provided example disappears and reappears in the 66th frame. During this period, its Track Q is not discarded directly but is instead cached in the lost-query bank. When the target reappears, a re-association strategy is employed to recover its original identity rather than treating it as a new target.
Compared with traditional query propagation methods, the proposed mechanism offers three main advantages: By injecting geometric and semantic information of detection proposals directly into the query vectors via sine–cosine encoding, it provides high-quality initialization for tracking. The width–height modulation mechanism dynamically adjusts the attention range according to target size, thereby improving feature-matching accuracy for multi-scale targets. The separation of detection queries and tracking queries at the input level, together with the split-stream mechanism, clearly delineates the functional boundaries between new-target discovery and trajectory maintenance. While preserving the simplicity of end-to-end tracking, this mechanism significantly enhances tracking stability in complex scenarios, demonstrating superior performance, particularly in scenes such as UAV swarms that involve frequent scale variations and intensive target interactions.
3.3. Re-Association Strategy Based on Motion Consistency
To address the identity-switch problem caused by trajectory crossing and occlusion, we propose a motion-direction-consistency-based re-association mechanism. This mechanism relies on a key physical principle: the motion trend of a target remains continuous during short-term occlusion. Even if a target is occluded for several frames, its motion direction after reappearance should be consistent with its direction before occlusion, as illustrated in Figure 5. This property provides a reliable basis for distinguishing between visually similar targets that follow different motion trajectories.
Figure 5.
Schematic diagram of intersecting trajectories from different identities. Different colors represent different IDs. When the three trajectories intersect, the red target remains visible in the image, while the blue and green targets are temporarily lost. Upon reappearance, their motion directions exhibit a certain consistency with the directions before they were lost.
When the tracking confidence falls within the intermediate range (), the query is moved into the lost-query bank for temporary storage. Each lost query records its last known position, motion-direction vector, and timestamp of loss. When a new frame contains an unmatched high-confidence detection, the system computes a comprehensive matching cost between this detection and each query in the lost-query bank:
where is a weighting coefficient; is the IoU (Intersection over Union) cost, which measures the spatial overlap between the last known position of the lost query and the position of the new detection; is the motion-direction consistency cost between the average motion direction of the i-th query before loss and the motion vector of the j-th detection between the current frame and the previous frame. It is calculated as follows:
where · denotes the vector dot product, represents the vector magnitude, and is a small positive constant to prevent division by zero. When the two motion directions are exactly the same, the cost equals 0; when they are completely opposite, the cost equals 2. Based on the comprehensive matching-cost matrix , we employ the Hungarian algorithm to obtain the optimal matching. For a successfully matched query–detection pair , the following conditions must be satisfied: the overall matching cost , the motion-direction consistency cost , and the maximum lost duration of the trajectory does not exceed the recovery window, i.e., . The detailed trajectory re-association matching algorithm is presented in the pseudo-code, as shown in Table 1.
Table 1.
Lost-trajectory re-association algorithm.
3.4. Progressive Joint Training Strategy
To address the optimization conflict between detection and tracking tasks during joint training in an end-to-end multi-object tracking framework, we propose a progressive joint training strategy. This strategy decomposes the training process into three sequential stages: detection backbone pre-training → tracking-association module training → end-to-end joint fine-tuning. By focusing on different optimization objectives at each stage, it ensures an optimal balance between convergence stability and final performance.
In the first stage, we pre-train the detection network separately on the adopted datasets to establish strong target localization capability while freezing the parameters of the tracking module. In the second stage, with the detection network weights fixed, we train the tracking-association module specifically on the tracking datasets, enabling it to learn cross-frame identity continuity without interference from detection noise. In the third stage, all parameters are unfrozen for end-to-end joint fine-tuning, employing a dynamic loss-weight adjustment mechanism to promote cooperative optimization between the detection and tracking tasks.
Compared with conventional end-to-end training methods, the progressive strategy offers three main advantages: significantly improved training stability (smooth loss curves without severe oscillation), superior final tracking performance (2–3 percentage point gain in HOTA and over 30% reduction in ID switches), and enhanced training efficiency. This strategy is particularly suitable for tracking tasks with limited data or complex scenarios, demonstrating excellent generalization capability in challenging environments such as UAV swarms.
3.5. Discussion on Key Module Differences and Necessity for Dense Swarms
Unlike general-purpose multi-object trackers, our framework is specifically designed for dense swarm scenarios where targets share highly similar appearances, undergo frequent crossing occlusions, and exhibit drastic scale variations. To address these challenges, we introduce three key structural modifications to the query-propagation paradigm, each differing from existing methods in both design and necessity.
First, geometric prior injection via sine–cosine encoding differs from baseline models (e.g., MOTRv2) that rely on learnable query vectors. Instead, we explicitly encode target location and scale from detection proposals into queries using high-frequency sine–cosine functions. In dense swarms, where targets are spatially close and visually nearly identical, pure learnable queries struggle to distinguish individuals. Sine–cosine encoding transforms absolute coordinates into spatially periodic features, providing high-quality initial localization that forces queries to focus on distinct spatial structures from the very first frame.
Second, width–height-modulated deformable attention (WHMDA) improves upon standard deformable attention, which learns sampling offsets solely based on query content and ignores the physical scale of the target. Our WHMDA introduces a modulation factor that dynamically scales the sampling range according to the target’s normalized width and height. In drone swarms, fast viewpoint shifts cause severe scale changes. WHMDA enables the model to concentrate on local fine-grained features for small distant drones while expanding to broader contextual regions for large nearby drones. This scale-adaptive aggregation is essential for maintaining feature discriminability when drones overlap or vary dramatically in size.
Third, re-association strategy based on motion consistency addresses a common weakness of end-to-end trackers—the lack of an explicit recovery mechanism for lost targets—which leads to identity switches after occlusions. Our strategy introduces a lost-query bank and a matching cost that prioritizes motion-direction consistency over simple spatial overlap. In swarms, drones frequently cross paths, causing transient occlusions. Because appearance features are too similar to rely on re-identification models, the continuity of motion trends becomes the most reliable cue. By measuring the consistency between pre-loss and post-reappearance motion vectors, our framework successfully recovers identities even under complex non-stationary group movements.
Together, these three modifications form a coherent set of solutions tailored to the unique challenges of dense swarm tracking.
4. Experiments and Results
4.1. Datasets and Evaluation Metrics
To comprehensively evaluate the performance of the proposed multi-object tracking algorithm, this paper selects three challenging public datasets—BEE24 [15], UAVSwarm [28], and the infrared sequences from the VTMOT [29] dataset—and conducts extensive comparative and ablation experiments. These datasets cover diverse scenarios, including target-scale variations, dense target interactions, and dynamic backgrounds, offering strong representativeness and breadth of evaluation. Among them, VTMOT includes multiple object categories (e.g., pedestrians and vehicles) with complex backgrounds and partial occlusions, making it suitable for assessing the robustness and generalization ability of algorithms in generic scenarios. BEE24 focuses on tracking dense biological targets (bees), where targets exhibit high appearance similarity and frequently intersecting trajectories, imposing stringent requirements on the identity-preserving capability of tracking algorithms. UAVSwarm is based on a UAV aerial perspective and concentrates on tracking UAV swarms. It features dynamic backgrounds, camera motion, and drastic target-scale variations, making it appropriate for evaluating algorithm performance on mobile platforms. Furthermore, we merge the aircraft-related video sequences from the GMOT40 [30] dataset with the sequences from UAVSwarm to enhance data diversity. Specifically, GMOT-40 is a relatively small dataset consisting of only 40 video sequences. To enhance data diversity, we selected four sequences from GMOT-40 that share similar characteristics with drone swarm scenes—such as distant airborne small targets—and incorporated them into the training set of the UAVSwarm dataset.
To assess algorithm performance from multiple dimensions, such as detection accuracy, association consistency, and identity-preserving capability, we adopt a comprehensive set of metrics: HOTA (higher-order tracking accuracy), MOTA (multi-object tracking accuracy), DetA (detection accuracy), AssA (association accuracy), AssR (association recall), and IDF1 (identity-based F1 score). Together, these metrics form a complete evaluation framework that systematically reflects the overall performance of a tracking algorithm in terms of target localization, trajectory continuity, and identity stability.
4.2. Experimental Setup
All experiments were conducted on a workstation equipped with two NVIDIA GeForce RTX 3090 GPUs (24 GB memory each). The operating system was Ubuntu 20.04 LTS, with PyTorch 2.0.1 as the deep learning framework (CUDA 11.8, cuDNN 8.6.0). Our method adopts MOTRv2 as the baseline framework, and the detection network (DetectNet) follows the multi-scale attention architecture described in our prior work [31]. Input images are uniformly resized to pixels to balance computational cost and small-target detection accuracy. The Transformer decoder consists of 6 layers, each with 8 attention heads and a hidden dimension of 256, and the number of sampling points K in deformable attention is set to 8. The classification loss employs focal loss, while the regression loss combines L1 loss and GIoU loss. Based on grid search on the validation sets, the following thresholds were determined: high-confidence threshold , low-confidence threshold , re-association matching cost threshold , motion-direction consistency threshold , recovery window frames, and direction consistency weight . The lost-query bank is implemented as an FIFO queue with a maximum capacity of 256 queries, which was found sufficient to handle temporary target disappearances in all evaluated datasets.
We adopt the progressive joint training strategy described in Section 3.4, which consists of three stages. In Stage 1, the detection network is pre-trained on a combined detection dataset built from the VisDrone-detection subset and the IRMT-UAV dataset, using SGD with an initial learning rate of 0.01, momentum 0.937, weight decay 0.0005, and a cosine annealing schedule; the batch size is 64 and training runs for 300 epochs. In Stage 2, the detection network weights are frozen, and the Transformer-based association module is trained on the tracking datasets (BEE24, UAVSwarm, and VTMOT infrared sequences). A random-interval sampling strategy is used: from each training video, starting at a random initial frame, one frame is sampled every random interval (maximum interval of 10), yielding 15 frames per training sample. The AdamW optimizer is employed with an initial learning rate of , weight decay , and a batch size of 2 (each containing 8 consecutive frames); training lasts for 150 epochs. After epoch 10, if the loss continues to decrease, the learning rate is sharply reduced to one-tenth of the initial value. In Stage 3, all parameters are unfrozen, and the entire model is fine-tuned end-to-end for 50 additional epochs with a learning rate reduced to , while other settings remain the same as Stage 2. Mixed-precision training and gradient checkpointing are enabled to accelerate computation and support longer temporal sequences. On BEE24 and UAVSwarm, we follow the official dataset splits; for the infrared sequences of VTMOT, we randomly split the data into training and test sets with a ratio of 7:3. All metrics are computed using the TrackEval toolkit to ensure reproducibility, and each experiment is repeated three times, with the average results reported. For fair comparison, all competing methods are either trained using their recommended configurations or evaluated with official pre-trained models under the same data splits.
4.3. State-of-the-Art Comparison on BEE24
On the BEE24 dataset, we conducted comprehensive comparisons between the proposed method and other state-of-the-art algorithms. The core metric, HOTA, reached 48.1%, significantly outperforming other advanced methods (e.g., 46.3% for TOPICTrack and 42.7% for OC-SORT), as shown in Table 2. This demonstrates that the proposed method achieves a breakthrough in balancing detection and association, attaining higher-order tracking accuracy. Our approach exhibits the following advantages, beginning with superior performance in both detection and association. The proposed method achieves the highest scores in both MOTA (67.9%) and IDF1 (59.7%), indicating that the algorithm ensures high detection quality (low missed and false detections) while also attaining optimal association performance. Compared to the second-best method, TOPICTrack, it improves by 1.8% and 1.7%, respectively. Second, it demonstrates outstanding robustness in association. Compared to ByteTrack, which relies on motion features (AssA 38.3%; AssR 52.7%), and TOPICTrack, which combines motion and appearance features through weighted fusion (AssA 40.1%; AssR 57.2%), our proposed method maintains ID stability and effectively reduces trajectory fragmentation (AssA 42.5%; AssR 61.4%) in complex scenarios (e.g., occlusion and intersections). Third, it demonstrates optimal control of identity switches. Our method achieves the lowest number of ID switches, only 1124, which is far lower than other methods, proving the effectiveness of our optimization for long-term tracking continuity.
Table 2.
A comparative analysis between the proposed algorithm and other state-of-the-art methods on the BEE24 dataset.
4.4. State-of-the-Art Comparison on UAV Swarm
Our method achieves comprehensive leading tracking performance on the UAVSwarm dataset, significantly outperforming the existing state-of-the-art methods across all the evaluation metrics, as shown in Table 3. This fully validates its superiority in complex aerial swarm scenarios. As shown in the table, our proposed method attains an overall tracking accuracy (HOTA) of 52.4%, substantially surpassing the previous best methods, UAVS-MOT (45.1%) and MOTRv2 (42.3%). This result directly demonstrates that our framework achieves a substantive breakthrough in balancing detection and association, enabling higher-order tracking quality.
Table 3.
A comparative analysis between the proposed algorithm and other state-of-the-art methods on the UAVSwarm dataset.
Simultaneously, our method exhibits particularly outstanding performance in detection-related metrics: MOTA reaches 72.1%, the highest among all the methods, while both the false positive count (FP, 898) and false negative count (FN, 2900) are reduced to the lowest levels, representing decreases of approximately 40% and 38%, respectively, compared to the second-best method. This demonstrates that our detection module achieves high recall and precision in dense dynamic drone scenarios.
Regarding identity consistency, our IDF1 score reaches 67.1%, outperforming UAVS-MOT (65.2%), and the number of identity switches (IDs) is only 51, significantly lower than other methods (e.g., 134 for MOTRv2 and 221 for ByteTrack). This indicates that our association mechanism can not only effectively handle challenges such as frequent interactions among targets and appearance similarities but also maintain stable identity assignments over extended periods, substantially reducing ID confusion.
In summary, compared with existing SOTA methods, our approach demonstrates superior comprehensive performance on the UAVSwarm dataset, particularly excelling in reducing false positives, false negatives, and controlling identity switches. It provides an effective solution for robust and reliable multi-object tracking in dense drone swarm scenarios.
4.5. Generalization on Infrared Tracking Datasets
On the infrared sequences of the Visible–Thermal Multi-Object Tracking (VTMOT) dataset, the proposed algorithm demonstrates excellent cross-modal generalization capability. As shown in Table 4, our method significantly outperforms existing state-of-the-art approaches across all key metrics in infrared tracking scenarios: the overall tracking accuracy (HOTA) reaches 42.3%, representing a 2.7-percentage-point improvement over the second-best method; the detection accuracy (DetA) achieves 41.6%, leading the second-best method by 3.5 percentage points, which reflects the algorithm’s robust representational capacity for infrared target characteristics. Meanwhile, the association accuracy (AssA) attains 43.1%, comparable to the performance of motion-optimized ByteTrack-series methods, indicating that our association mechanism maintains stable trajectory associations even in low-texture, low-contrast infrared environments. Notably, our method achieves the highest MOTA (43.4%) and IDF1 (48.1%) scores among all the compared methods on the infrared sequences, further validating the superior detection–association balance and identity–consistency preservation of our framework in cross-modal scenarios.
Table 4.
A comparative analysis between the proposed algorithm and other state-of-the-art methods on the infrared sequences of the VTMOT dataset.
4.6. Ablation Study
4.6.1. Study on the Impact of the Detection Network
To deeply investigate the impact of the object detection module on the performance of multi-object tracking systems, this paper designs a set of ablation experiments that systematically compare the tracking performance under three conditions: without incorporating detection results (i.e., using only randomly initialized queries), employing mainstream detection networks (YOLOX [33] and CenterNet [34]) to generate detection queries, and utilizing the multi-scale object detection network (MSFENet [31]) from our previous work to generate queries. MSFENet is a multi-scale feature enhancement network designed for detecting multi-scale targets. Its core components include: (1) the iAFF module, which employs multi-scale channel attention to fuse cross-scale features, effectively preserving both large-scale global targets and small-scale local details; (2) the OAAM module, which strengthens channel relationships in partially occluded regions to improve tolerance for local feature loss; (3) the dynamic head, which unifies scale-aware, spatial-aware, and task-aware attention mechanisms to adaptively represent targets across varying scales; and (4) the NWD loss, which replaces conventional IoU-based losses to address the sensitivity of bounding-box regression for small targets. Together, these modules enable MSFENet to achieve robust detection performance across a wide range of target scales in complex infrared scenes with strong clutter, scale variation, and partial occlusion.
In the absence of an additional detection network—namely when using randomly initialized queries—the tracker achieves the lowest scores across all the metrics, indicating that effective detection queries are crucial for initializing object trajectories. Compared with random queries, introducing detection networks such as YOLOX and CenterNet significantly improves all the evaluation metrics. Among these, CenterNet, owing to its center-point-based representation, demonstrates advantages in object localization, achieving a higher DetA (detection accuracy) than YOLOX. Furthermore, by incorporating the detection network specifically designed in our previous work to address the challenges of multi-scale and small-object density in drone scenarios, the proposed method attains the best performance across all the metrics. Notably, it achieves substantial improvements over the second-best approach in HOTA (+9.9%), which reflects overall tracking accuracy, DetA (+13.4%), and AssA (+7.2%), which denote data association accuracy. These results validate the effectiveness of our method in generating high-quality detection queries, as shown in Table 5.
Table 5.
Ablation study on the impact of different detection query generation methods on multi-object tracking performance.
To intuitively demonstrate the performance differences among various detection strategies in complex scenarios, we selected representative frames from the same test sequence for visual analysis, as shown in Figure 6. Red arrows in the figure indicate missed drone targets.
Figure 6.
We selected a video sequence from the UAVSwarm dataset to illustrate the impact of detection boxes generated by different detection algorithms on multi-object tracking performance. The first row corresponds to frame 1, and the second row corresponds to frame 79. The tracked targets are all long-range small-rotor UAVs, which appear as small objects in the image. Red arrows in the figure indicate missed drone targets. The results demonstrate that, when no pre-trained detection network is used to generate detection queries, the missed detection rate is the most severe. When employing general-purpose detection algorithms, partial missed detections still occur. In contrast, our previously proposed small-object detection method based on multi-scale attention mechanism is able to track all targets successfully.
When using randomly initialized queries, the lack of effective target priors results in the most severe missed detections, making it impossible to reliably initialize multiple targets. The detection scheme employing YOLOX exhibits partial missed detections for small-scale targets in the scene (e.g., drones numbered 7 and 8). When adopting the CenterNet-based detection scheme, confusion tends to occur in scenarios where multiple targets (e.g., drones numbered 4 and 5) are spatially close or densely distributed. Due to its center-point-based detection characteristic, this approach is prone to misassignment, leading to brief tracking loss or missed detection of certain targets (e.g., drone number 4). In contrast, benefiting from its multi-scale feature fusion design, MSFENet maintains the most stable detection and tracking performance throughout the entire sequence, achieving the lowest missed detection rate and the best trajectory continuity.
The experimental results demonstrate that the quality of detection queries is a critical bottleneck for multi-object tracking performance. Randomly initialized queries fail to provide effective starting points for tracking, while general-purpose detection networks exhibit limitations under specific challenges, such as small targets and dense target distribution. The proposed MSFENet, specifically designed for multi-drone tracking scenarios, enhances multi-scale object detection capability and generates more robust and accurate detection queries. This significantly improves the performance of subsequent tracking modules in terms of detection accuracy, data association, and overall tracking precision, effectively alleviating missed detections and tracking loss.
4.6.2. Study on the Impact of Target Position Prior Encoding Methods
To investigate effective encoding of target position priors in query-based multi-object tracking, this experiment compares three query initialization strategies, with the key difference lying in how detection anchors are encoded. Quantitative results are shown in Table 6.
Table 6.
Ablation study on the impact of target position prior encoding methods.
Learnable queries without anchors yield the lowest performance (HOTA = 20.3%), indicating that lacking positional guidance severely hinders target localization and association. Introducing anchors via linear projection significantly boosts all the metrics, especially DetA (from 18.7% to 31.9%), confirming that raw position priors help queries to focus on target regions. However, AssA improvement remains limited (22.1% → 27.6%), suggesting linear fusion inadequately captures spatial relationships for distinguishing nearby targets and preserving long-term identities.
Anchor sine–cosine encoding, which applies high-frequency sinusoidal functions to anchor coordinates, achieves optimal performance across all the metrics, particularly AssA (61.3%), far exceeding the other strategies. HOTA (51.9%) and IDF1 (66.7%) are also substantially improved. This demonstrates that sine–cosine encoding transforms absolute positions into spatially periodic features, enabling the model to understand both target locations and their relative spatial structures. Such enriched priors enhance query discriminability during data association, ensuring stable tracking under dense and crossing scenarios.
These results confirm that effectively encoding detection anchors into initial queries is crucial in query-based frameworks. Compared to learnable queries or linear projection, sine–cosine encoding delivers structured spatial representations that significantly boost sensitivity to target positions and comprehension of spatial relations. This leads to balanced gains in both DetA and AssA, and substantial improvements in overall tracking accuracy (HOTA, MOTA, and IDF1).
4.6.3. Study on the Effectiveness of the Re-Association Strategy
To validate the trajectory re-association module and assess different similarity metrics, we conduct systematic ablation experiments. This module recovers trajectories interrupted by occlusion or detection failures during association. All the re-association strategies improve performance over the baseline, confirming the necessity of explicitly handling trajectory fragmentation, as shown in Table 7.
Table 7.
Ablation study on different similarity measurement strategies for trajectory re-association module.
Using only yields clear gains, especially in MOTA (+1.2%) and AssR (+3.7%), indicating that spatial overlap is effective for short-term recovery. Using only motion-direction consistency produces larger improvements, particularly in MOTA (+4.7%) and AssR (+6.8%), demonstrating that motion continuity is more discriminative than static position for mid-term to long-term recovery under complex motion.
Fusing both and motion-direction consistency achieves optimal results across all the metrics. HOTA rises from 40.2% to 47.1% and IDF1 from 55.3% to 59.1%, with substantial gains in AssA and AssR. This highlights the complementarity of spatial and motion cues: enables precise local matching, while motion consistency ensures physical plausibility. Their combination effectively reduces ambiguous associations.
On the BEE24 dataset, we select a typical scenario involving the crossing and occlusion of two individuals to visually analyze the effectiveness of different trajectory re-association strategies. As shown in Figure 7, the center positions of each target are marked, and different colors are used to distinguish individuals with different identity IDs. The experimental results show that, when only is used, the identity of target No. 41 undergoes one false switch after being temporarily lost due to occlusion and subsequently re-detected. Without any re-identification mechanism, identity switches occur more frequently, with two misidentifications observed, yielding the worst tracking performance. In contrast, the strategy relying solely on motion-direction consistency performs comparably to the IoU–direction-weighted fusion strategy, both of which achieve accurate and stable identity recognition for crossing individuals.
Figure 7.
Comparison of the effectiveness of different re-association strategies on identity recognition for two cross-occluded targets. Figure (a) illustrates the re-association effect of trajectory fragments after the crossing of targets No. 41 and No. 43 under the weighted averaging strategy combining and . With this strategy, the identities of two individuals with highly similar appearances are accurately recognized and maintained after crossing and separation. Figures (b), (c), and (d), respectively, present the recognition performance under three control conditions: without any additional re-identification mechanism, using only for association, and relying solely on . The trajectories in the figure represent the center positions of targets in historical frames, with different colors indicating identity switches.
This ablation study confirms that the proposed trajectory re-association module is a critical component for enhancing tracking robustness. Among the evaluated strategies, motion-direction consistency proves to be a more effective association cue than spatial overlap alone, while their fusion yields the greatest overall efficacy. By comprehensively leveraging both spatial geometric and motion information of targets, this module substantially strengthens the algorithm’s capability to recover trajectories after interruptions, thereby holistically improving tracking accuracy (HOTA; MOTA), identity preservation capability (IDF1), and association precision and recall (AssA; AssR).
4.6.4. Study on the Effectiveness of Scale-Modulated Cross-Attention and Progressive Joint Training
To validate the effectiveness of the proposed width–height-modulated deformable attention and progressive joint training strategy in improving multi-object tracking performance, we conducted systematic ablation experiments on the VTMOT infrared dataset. Four experimental configurations were adopted: baseline, which uses standard deformable attention with conventional training; Configuration A, which employs standard deformable attention but adopts the progressive joint training strategy; Configuration B, which applies width–height-modulated deformable attention with conventional training; and the full model, which integrates both width–height-modulated deformable attention and progressive joint training. The comprehensive analysis results are shown in Table 8.
Table 8.
Study on the effectiveness of scale-modulated cross-attention and progressive joint training. DA denotes deformable attention without width–height modulation; WHMDA denotes width–height-modulated deformable attention; PJT refers to the progressive joint training strategy.
A comparison of the performance metrics between the baseline and Configuration A demonstrates that the progressive joint training strategy effectively coordinates the optimization process of detection and tracking tasks, thereby improving association performance. With the attention mechanism held constant, the introduction of the progressive joint training strategy yields a 1.8% improvement in MOTA (37.1% → 38.9%), a 0.9% improvement in IDF1 (40.4% → 41.3%), and a 2.2% improvement in AssR (37.2% → 39.4%).
The comparison between the baseline model and Configuration B indicates that the width–height-modulated deformable attention mechanism enables the model to more accurately capture key deformation features of targets by dynamically adjusting the spatial distribution of attention regions. Under conventional training, the incorporation of this mechanism leads to substantial gains: HOTA increases by 5.8% (35.1% → 40.9%), MOTA by 4.1% (37.1% → 41.2%), and IDF1 by 4.9% (40.4% → 45.3%).
The full model achieves the best performance across all the evaluation metrics. Compared with the baseline model (row 1), it attains improvements of 7.2% in HOTA (35.1% → 42.3%), 6.0% in MOTA (37.1% → 43.1%), and 7.7% in IDF1 (40.4 → 48.1). Notably, association accuracy (AssA) and association recall (AssR) are enhanced by 7.8% (32.9% → 40.7%) and 12.5% (37.2% → 49.7%), respectively. These results indicate that the two proposed components exhibit a significant synergistic enhancement effect in improving target association performance.
As illustrated in Figure 8, we selected two representative scenarios from the VTMOT dataset to conduct a comparative visualization analysis of feature heatmaps generated by different cross-attention mechanisms. The experimental results demonstrate that, compared with conventional attention mechanisms, the width–height-modulated attention mechanism adaptively adjusts the attention focus according to target size: for small-scale targets, attention is more concentrated on local neighborhoods, enhancing the extraction of fine-grained features; for large-scale targets, it expands the focus to broader regions, improving the integration of contextual information. This scale-adaptive attention allocation strategy effectively enhances target spatial localization accuracy and identity discrimination robustness under complex scenes, thereby providing a superior representation foundation for feature matching and identity preservation in multi-object tracking.
Figure 8.
Comparison of heatmaps generated during inference by two different attention mechanisms. The first column shows two representative scenes with ground-truth annotations from the VT-MOT dataset. The second column presents the heatmaps produced by the width–height-modulated deformable attention, and the third column displays those generated by the standard deformable attention without width–height modulation.
5. Discussion
We propose an improved end-to-end multi-object tracking framework based on query propagation, targeting the challenges of identity confusion and trajectory fragmentation in high-density, high-similarity group tracking scenarios, such as UAV swarms and biological collectives. Unlike the prevailing Tracking-by-Detection paradigm, our approach does not explicitly model motion or appearance features for cross-frame association. Instead, we leverage the intrinsic strength of Transformers in global context modeling by abstracting each target as a learnable tracking-query vector, which evolves frame by frame as the target state changes. This formulation transforms data association into a continuous optimization process of queries, thereby circumventing the systematic biases introduced by handcrafted association rules and decoupled optimization of detection and association modules in conventional methods.
Compared with query-propagation-based approaches such as MOTRv2, our framework exhibits three major distinctions. First, we introduce a geometric prior injection mechanism based on sine–cosine encoding during detection query generation, enabling the query to explicitly perceive target location and scale information, thereby improving initialization quality. Second, to address the difficulty of multi-scale feature alignment, we propose a width–height-modulated deformable attention mechanism, which adaptively adjusts the sampling range according to the target size, enhancing fine-grained feature extraction for small targets and contextual information integration for large targets. Third, to recover identities after trajectory fragmentation, we design a motion-direction-consistency-based re-association strategy, which leverages motion continuity to achieve efficient identity recall without relying on appearance features.
Nevertheless, the proposed method still has certain limitations. The effectiveness of our re-association strategy heavily depends on the continuity of the target’s motion trend during the lost period. The strategy assumes that the motion direction before and after occlusion remains consistent. If the target undergoes severe nonlinear maneuvers during brief occlusion (e.g., sharp turns or backtracking), the matching cost constructed based on historical motion direction may become invalid, leading to incorrect identity recovery or identity switches. This limitation is particularly pronounced in highly dynamic scenarios, such as UAV swarm confrontations or emergency obstacle avoidance in bird flocks. Future work will explore the integration of lightweight motion prediction models or short-term trajectory interpolation mechanisms to enhance the robustness of the re-association module against nonlinear motion. Moreover, the proposed model has approximately 101 million parameters and runs at around 20 FPS on a single NVIDIA RTX 3090 Ti GPU. Compared to several lightweight tracking methods, our approach exhibits relatively lower real-time performance. Nevertheless, extending the proposed framework to online lightweight deployment scenarios without significantly increasing computational overhead constitutes another important direction for subsequent research. Improving computational efficiency while maintaining tracking performance will be one of our key focuses in future work.
6. Conclusions
We present an improved end-to-end multi-object tracking framework tailored for high-density, high-similarity group tracking tasks, such as UAV swarms. Centered on query propagation, the method integrates detection geometric priors via sine–cosine encoding, introduces width–height-modulated deformable attention for scale-adaptive feature aggregation, devises a motion-direction-consistency-driven trajectory re-association mechanism, and adopts a progressive joint training strategy to mitigate optimization conflicts between detection and tracking. Extensive experiments on the BEE24, UAVSwarm, and VTMOT infrared datasets demonstrate that the proposed method significantly outperforms state-of-the-art approaches across key metrics, including HOTA, MOTA, IDF1, AssA, and AssR, with particularly notable advantages in identity-switch control. Ablation studies further validate the effectiveness and synergistic contributions of each module. This work provides an efficient solution for tracking group targets characterized by high appearance similarity, complex motion, and dense interactions without requiring explicit appearance modeling, and it holds promising potential for a wide range of applications.
Author Contributions
Conceptualization, W.D., Z.L. and J.R.; Methodology, S.Z., W.D. and J.R.; Software, S.Z.; Validation, S.Z.; Formal analysis, S.Z.; Investigation, S.Z.; Data curation, W.D.; Writing—original draft, S.Z.; Writing—review & editing, S.Z.; Supervision, Z.L. and J.R.; Project administration, Z.L. and J.R.; Funding acquisition, Z.L. and J.R. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by National Key R&D Program of China [2024YFB3614400].
Data Availability Statement
All the datasets used in this paper are publicly available, and their sources have been indicated in the text. The code is available upon request to the corresponding author with a declaration of intended use.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Guan, Z.; Wang, Z.; Zhang, G.; Li, L.; Zhang, M.; Shi, Z.; Jiang, N. Multi-object tracking review: Retrospective and emerging trend. Artif. Intell. Rev. 2025, 58, 235. [Google Scholar] [CrossRef] [Scilit]
- Asperti, A.; Fiorilla, S.; Nardi, S.; Orsini, L. A review of recent techniques for person re-identification. arXiv 2025, arXiv:2509.22690. [Google Scholar] [CrossRef] [Scilit]
- Kalman, R.E. A new Approach to Linear Filtering and Prediction Problems. J. Basic Eng. 1960, 82, 35–45. [Google Scholar] [CrossRef] [Scilit]
- Veeramani, B.; Raymond, J.W.; Chanda, P. DeepSort: Deep convolutional networks for sorting haploid maize seeds. BMC Bioinform. 2018, 19, 289. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Zhang, Y.; Sun, P.; Jiang, Y.; Yu, D.; Weng, F.; Yuan, Z.; Luo, P.; Liu, W.; Wang, X. Bytetrack: Multi-object tracking by associating every detection box. In Computer Vision—ECCV 2022; European Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2022; pp. 1–21. [Google Scholar]
- Zhou, X.; Koltun, V.; Krähenbühl, P. Tracking objects as points. In Computer Vision—ECCV 2020; European Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2020; pp. 474–490. [Google Scholar]
- Zhang, Y.; Wang, T.; Zhang, X. Motrv2: Bootstrapping end-to-end multi-object tracking by pretrained object detectors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; pp. 22056–22065. [Google Scholar]
- Bewley, A.; Ge, Z.; Ott, L.; Ramos, F.; Upcroft, B. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing (ICIP); IEEE: New York, NY, USA, 2016; pp. 3464–3468. [Google Scholar]
- Cao, J.; Pang, J.; Weng, X.; Khirodkar, R.; Kitani, K. Observation-centric sort: Rethinking sort for robust multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; pp. 9686–9696. [Google Scholar]
- Du, Y.; Zhao, Z.; Song, Y.; Zhao, Y.; Su, F.; Gong, T.; Meng, H. Strongsort: Make deepsort great again. IEEE Trans. Multimed. 2023, 25, 8725–8737. [Google Scholar] [CrossRef] [Scilit]
- Qin, Z.; Wang, L.; Zhou, S.; Fu, P.; Hua, G.; Tang, W. Towards generalizable multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 16–22 June 2024; pp. 18995–19004. [Google Scholar]
- Cui, Y.; Zeng, C.; Zhao, X.; Yang, Y.; Wu, G.; Wang, L. Sportsmot: A large multi-object tracking dataset in multiple sports scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023; pp. 9921–9931. [Google Scholar]
- Lv, W.; Zhang, N.; Zhang, J.; Zeng, D. One-shot multiple object tracking with robust ID preservation. IEEE Trans. Circuits Syst. Video Technol. 2023, 34, 4473–4488. [Google Scholar] [CrossRef] [Scilit]
- Yang, M.; Han, G.; Yan, B.; Zhang, W.; Qi, J.; Lu, H.; Wang, D. Hybrid-sort: Weak cues matter for online multi-object tracking. In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 26–27 February 2024; Volume 38, pp. 6504–6512. [Google Scholar]
- Cao, X.; Zheng, Y.; Yao, Y.; Qin, H.; Cao, X.; Guo, S. TOPIC: A parallel association paradigm for multi-object tracking under complex motions and diverse scenes. IEEE Trans. Image Process. 2025, 34, 743–758. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Sridhar, V.H.; Roche, D.G.; Gingins, S. Tracktor: Image-based automated tracking of animal movement and behaviour. Methods Ecol. Evol. 2019, 10, 815–820. [Google Scholar] [CrossRef] [Scilit]
- Zhang, Y.; Wang, C.; Wang, X.; Zeng, W.; Liu, W. Fairmot: On the fairness of detection and re-identification in multiple object tracking. Int. J. Comput. Vis. 2021, 129, 3069–3087. [Google Scholar] [CrossRef] [Scilit]
- Xu, Y.; Ban, Y.; Delorme, G.; Gan, C.; Rus, D.; Alameda-Pineda, X. TransCenter: Transformers with dense representations for multiple-object tracking. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 45, 7820–7835. [Google Scholar] [CrossRef] [Scilit]
- Yao, M.; Peng, J.; He, Q.; Peng, B.; Chen, H.; Chi, M.; Liu, C.; Benediktsson, J.A. MM-Tracker: Motion Mamba with Margin Loss for UAV-platform Multiple Object Tracking. arXiv 2024, arXiv:2407.10485. [Google Scholar] [CrossRef] [Scilit]
- Wu, J.; Cao, J.; Song, L.; Wang, Y.; Yang, M.; Yuan, J. Track to detect and segment: An online multi-object tracker. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 12352–12361. [Google Scholar]
- Peng, J.; Wang, C.; Wan, F.; Wu, Y.; Wang, Y.; Tai, Y.; Wang, C.; Li, J.; Huang, F.; Fu, Y. Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking. In Computer Vision—ECCV 2020; European Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2020; pp. 145–161. [Google Scholar]
- Chuanyun, W.; Yang, S.; Linlin, W.; Tian, W.; Jingjing, W.; Qian, G. Multi-object continuous robust tracking algorithm for anti-UAV swarm. Acta Aeronaut. Astronaut. Sin. 2024, 45, 329017. [Google Scholar]
- Meinhardt, T.; Kirillov, A.; Leal-Taixe, L.; Feichtenhofer, C. Trackformer: Multi-object tracking with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 8844–8854. [Google Scholar]
- Zeng, F.; Dong, B.; Zhang, Y.; Wang, T.; Zhang, X.; Wei, Y. Motr: End-to-end multiple-object tracking with transformer. In Computer Vision—ECCV 2022; European Conference on Computer Vision; Springer: Berlin/Heidelberg, Germany, 2022; pp. 659–675. [Google Scholar]
- Xiong, L.; Su, L.; Zeng, S.; Li, X.; Wang, T.; Zhao, F. Generalized spatial–temporal regression graph convolutional transformer for traffic forecasting. Complex Intell. Syst. 2024, 10, 7943–7964. [Google Scholar] [CrossRef] [Scilit]
- Gao, R.; Wang, L. MeMOTR: Long-term memory-augmented transformer for multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 1–6 October 2023; pp. 9901–9910. [Google Scholar]
- Cui, Y.; Jiang, C.; Wang, L.; Wu, G. Mixformer: End-to-end tracking with iterative mixed attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 13608–13618. [Google Scholar]
- Wang, C.; Su, Y.; Wang, J.; Wang, T.; Gao, Q. UAVSwarm dataset: An unmanned aerial vehicle swarm dataset for multiple object tracking. Remote Sens. 2022, 14, 2601. [Google Scholar] [CrossRef] [Scilit]
- Zhu, Y.; Wang, Q.; Li, C.; Tang, J.; Gu, C.; Huang, Z. Visible–thermal multiple object tracking: Large-scale video dataset and progressive fusion approach. Pattern Recognit. 2025, 161, 111330. [Google Scholar] [CrossRef] [Scilit]
- Bai, H.; Cheng, W.; Chu, P.; Liu, J.; Zhang, K.; Ling, H. Gmot-40: A benchmark for generic multiple object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 6719–6728. [Google Scholar]
- Zhang, S.; Du, W.; Liu, Y.; Zhou, N.; Li, Z. Feature Enhancement Network for Infrared Small Target Detection in Complex Backgrounds Based on Multi-Scale Attention Mechanism. Appl. Sci. 2025, 15, 4966. [Google Scholar] [CrossRef] [Scilit]
- Sun, P.; Cao, J.; Jiang, Y.; Zhang, R.; Xie, E.; Yuan, Z.; Wang, C.; Luo, P. Transtrack: Multiple object tracking with transformer. arXiv 2020, arXiv:2012.15460. [Google Scholar]
- Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. Yolox: Exceeding yolo series in 2021. arXiv 2021, arXiv:2107.08430. [Google Scholar] [CrossRef] [Scilit]
- Duan, K.; Bai, S.; Xie, L.; Qi, H.; Huang, Q.; Tian, Q. Centernet: Keypoint triplets for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 6569–6578. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.







