Multi-Object Tracking Method Based on Domain Adaptation and Camera Motion Compensation

Abstract

Although recent multi-object tracking (MOT) methods have shown impressive performance, MOT remains challenging due to two key issues: the poor generalization of ReID in MOT tasks and motion estimation errors caused by camera movement. To address these, we propose DCTrack, a novel tracker with two core modules: Domain Adaptation-based Batch Instance Normalization (DA-BIN) and camera motion compensation mapped to the Ground Plane (CMC-GP). DA-BIN enhances appearance modeling through domain adaptation using a combination of batch and instance normalization layers, allowing DCTrack to simulate unsuccessful generalization scenarios and improving ReID performance in MOT. CMC-GP uses a Kalman Filter-based method to map object motion estimation to the ground plane and applies the same compensation parameters across the video sequence, enhancing robustness to camera motion. Experimental results show that DCTrack effectively improves ReID generalization in MOT and performs well in scenes with camera motion, achieving a HOTA of 67.3% and an IDF1 of 65.8% on DanceTrack.

1. Introduction

Multi-Object Tracking (MOT) has emerged as a cornerstone technology in computer vision, enabling critical applications from autonomous driving and surveillance systems to sports analytics and human–computer interaction [1,2,3,4]. As the demand for real-time, robust tracking in diverse and challenging environments continues to grow, the field faces fundamental challenges that existing solutions struggle to address effectively.

The predominant Tracking-by-Detection (TBD) paradigm [5] decomposes MOT into object detection and cross-frame association, relying heavily on appearance features from re-identification (ReID) models [6,7] and motion features from prediction models [8]. While conceptually elegant, this approach encounters two critical bottlenecks in real-world deployment: domain adaptation failures and camera motion interference.

The Domain Gap Challenge: ReID models, trained on specific datasets with controlled conditions, exhibit significant performance degradation when deployed in diverse MOT scenarios. This degradation stems from the fundamental mismatch between ReID’s assumption of consistent cross-camera statistics and MOT’s reality of dynamic, multi-sequence environments with varying illumination, viewpoints, and scene characteristics. Consequently, even state-of-the-art (SOTA) ReID performance fails to translate into effective MOT performance, particularly in crowded scenes with frequent occlusions.

The Camera Motion Problem: Traditional motion models operate in 2D image coordinates, where camera movement introduces pseudo-motion artifacts that severely compromise tracking accuracy. Existing camera motion compensation (CMC) methods [9] attempt post hoc corrections through expensive frame-by-frame processing, creating computational bottlenecks that conflict with real-time requirements. This fundamental mismatch between 2D image plane modeling and 3D ground plane reality represents a persistent challenge in the field.

Current solutions typically address these challenges independently through additional datasets, complex training strategies, or computationally intensive compensation methods. However, these approaches often sacrifice either accuracy for efficiency or vice versa, failing to provide the robust, real-time performance required for practical deployment.

In this work, we introduce DCTrack (domain adaptation and camera motion compensation track), a novel MOT framework that fundamentally addresses both challenges through principled domain adaptation and geometric motion modeling. Our approach makes two key contributions.

First, we propose the Domain Adaptation-based Batch Instance Normalization (DA-BIN) module, which transcends traditional normalization approaches by actively simulating failure scenarios during training. Unlike existing methods that passively adapt to domain shifts, DA-BIN employs a learnable balancing mechanism between batch normalization (BN) and instance normalization (IN), coupled with multi-objective optimization that deliberately explores overfitting and underfitting boundaries. This approach enhances ReID generalization without requiring additional datasets or complex training strategies.

Second, we introduce the camera motion compensation on ground plane (CMC-GP) module, which revolutionizes motion modeling by projecting Kalman Filter (KF) from image coordinates to ground plane coordinates. This paradigm shift eliminates camera-induced pseudo-motion at its source while modeling spatial motion correlations through non-diagonal noise covariance matrices. Unlike traditional frame-by-frame compensation methods, CMC-GP achieves superior accuracy with minimal computational overhead.

As demonstrated in Figure 1, DCTrack achieves exceptional performance on the challenging DanceTrack benchmark [10], reaching 67.3 HOTA and 65.8 IDF1 while using only ReID training data without additional MOT dataset augmentation. This performance represents a significant advancement in balancing accuracy and efficiency for real-world MOT applications.

Our main contributions are as follows:

• We propose the DA-BIN module, a novel domain adaptation framework that dynamically balances BN and IN through learnable parameters. By actively simulating failure scenarios during training, DA-BIN prevents source domain overfitting and significantly enhances ReID generalization in MOT tasks without requiring additional training data.
• We introduce the CMC-GP module, a ground plane-based camera motion compensation approach that projects KF from image coordinates to ground coordinates. This paradigm shift fundamentally eliminates camera-induced pseudo-motion while modeling spatial correlations through non-diagonal noise covariance matrices, achieving superior accuracy with minimal computational overhead.
• We present DCTrack, integrating DA-BIN and CMC-GP modules within the TBD framework, which achieves SOTA performance across DanceTrack, MOT20 [11], and MOT17 [12] benchmarks, demonstrating significant improvements in both accuracy and efficiency for real-world MOT applications.

The rest of this paper is arranged as follows: Section 2 examines the relevant studies on appearance modeling and motion estimation for MOT. Section 3 presents the detailed methodology of DCTrack, including the DA-BIN and CMC-GP modules. Section 4 provides comprehensive experimental evaluation and analysis. Section 5 concludes this paper with a discussion of limitations and future directions.

2. Related Works

2.1. Appearance Models and Domain Adaptation in MOT

To improve long-term association, current MOT methods can be divided into two major categories depending on their use of appearance features. Traditional two-stage methods [13,14] employ independent ReID models to extract deep appearance features, which effectively reduce identity switches (IDSWs) in occlusion scenarios but suffer from high computational overhead. Joint Detection and Tracking (JDT) methods [15,16,17,18,19,20] address this efficiency issue by optimizing detection and embedding through shared backbone networks, yet require complex association strategies and large-scale joint training on both detection and ReID datasets.

However, both paradigms face a fundamental challenge: domain shift in appearance models. ReID models, designed for cross-camera person retrieval, exhibit significant performance degradation when applied to MOT tasks due to statistical distribution differences between training (source) and testing (target) domains. This problem is exacerbated in MOT scenarios where datasets consist of multiple sub-sequences with varying statistical properties, creating not only source-target domain gaps but also intra-target domain variations.

Current domain adaptation approaches in ReID primarily rely on BN-based strategies [21] that adjust batch statistics to align with cross-domain distributions. These methods fall into two categories: (1) online statistical estimation for unlabeled target data [22], and (2) progressive feature transfer via pseudo-labeling or combined statistics calculation [23]. While effective in traditional ReID settings, these approaches have critical limitations when applied to MOT:

• Data dependency: Require substantial additional datasets and complex training strategies with limited performance gains;
• Single normalization bias: Using BN or IN alone leads to overfitting or underfitting in diverse MOT scenarios;
• Passive adaptation: Cannot actively explore domain boundaries or simulate failure scenarios.

To address these limitations, our DA-BIN module introduces a novel domain adaptation framework specifically designed for MOT. Unlike existing methods that passively adapt to domain shifts, DA-BIN actively simulates failure scenarios during training to enhance generalization. By dynamically balancing BN and IN through learnable parameters and employing three complementary loss functions, DA-BIN achieves superior cross-domain adaptation without requiring additional data or complex network architectures. This approach significantly improves cross-domain ID matching accuracy while maintaining model efficiency, making it particularly suitable for the TBD paradigm where lightweight and effective appearance models are crucial.

2.2. Motion Models and Camera Motion Compensation in MOT

Motion prediction is fundamental to MOT, where objects’ future positions are estimated based on previous states through Bayesian estimation frameworks. Despite rapid advances in object detection, many SOTA end-to-end MOT models still underperform compared to classical motion model-based approaches, highlighting the continued importance of effective motion modeling.

Current Motion Models and Their Limitations: The KF remains the dominant motion model due to its recursive predict–update cycle and computational efficiency. Representative methods include SORT [24] with linear motion assumptions, ByteTrack [25] that balances detection quality and tracking confidence, and OC-SORT [2] that enhances robustness in nonlinear scenarios. Alternative approaches include optical flow-based models [26], LSTM-based sequential models [27], transformer-based long-term dependency models [28], and graph-based implicit motion estimation [20]. However, these methods share critical limitations:

• Image plane constraint: Traditional motion models operate in 2D image coordinates, failing to capture true 3D object motion on the ground plane;
• Camera motion interference: Camera movement introduces pseudo-motion artifacts that severely degrade motion prediction accuracy;
• Computational complexity: Advanced models (LSTM, Transformer, graph-based) require extensive computational resources and labeled data, compromising real-time performance.

Camera Motion Compensation Challenges: To address camera-induced motion artifacts, existing camera motion compensation (CMC) methods [29,30,31] employ techniques such as ORB feature matching [32] or Enhanced Correlation Coefficient (ECC) maximization [33]. These approaches fall into two categories: (1) offline methods that pre-compute affine transformations but lack adaptability, and (2) online methods that provide better adaptation but impose significant computational overhead. Recent CMC frameworks [34] leverage temporal–spatial structures but require pre-provided background models, limiting generalization to new scenes. Critically, current CMC techniques struggle with high-frame-rate, high-resolution videos due to computational constraints, hindering real-time MOT applications.

Fundamental Problem: Existing motion models suffer from a fundamental mismatch between the 2D image plane representation and the 3D ground plane reality of object motion. This discrepancy is exacerbated by camera motion, which introduces complex nonlinear distortions that cannot be effectively handled by traditional image-based KF or computationally expensive compensation methods.

Our proposed CMC-GP module addresses these limitations through a paradigm shift: projecting motion estimation from image plane to ground plane. This approach offers several key advantages: (1) Ground-truth motion modeling: Eliminates camera-induced pseudo-motion by operating in the actual motion space. (2) Efficient compensation: Achieves camera motion compensation through geometric projection rather than expensive feature matching. (3) Correlation modeling: Captures inherent spatial correlations in ground plane motion through non-diagonal noise covariance matrices. And (4) real-time performance: Maintains computational efficiency suitable for real-time applications. Unlike existing methods that treat camera motion as a post-processing correction, CMC-GP fundamentally reformulates the motion estimation problem in the appropriate coordinate system, providing both theoretical elegance and practical effectiveness.

3. Method

In this section, we introduce two key improvements of DCTrack that address fundamental challenges in multi-object tracking: the DA-BIN module for cross-domain generalization and the CMC-GP module for camera motion compensation.

The DA-BIN module tackles the domain shift problem in ReID models by dynamically balancing BN and IN through a learnable parameter $\theta$. By simulating failure scenarios during training with three complementary loss functions, DA-BIN enhances model generalization to unknown target domains without requiring additional data.

The CMC-GP module addresses camera motion interference by projecting traditional image plane KF to the ground plane. This approach eliminates pseudo-motion artifacts caused by camera movement and models spatial motion correlations through a non-diagonal noise covariance matrix, significantly improving tracking accuracy in dynamic camera scenarios.

DCTrack follows the TBD paradigm. The complete pipeline of DCTrack is shown in Figure 2 and Algorithm 1. The tracking process operates as follows: For each frame, detections are first separated by confidence scores, then DA-BIN extracts appearance features for high-confidence detections while CMC-GP predicts track locations on the ground plane. Two-stage association is performed using IoU and embedding distances, followed by track updates with non-diagonal noise modeling and initialization of new tracks with ground plane KF.

Algorithm 1 Pseudo-code of DCTrack
1: Input: A video sequence V; object detector $Det$; appearance feature extractor $Enc$; high detection score threshold $\tau$; new track score threshold $\eta$; camera parameters A
2: Output: Tracks $\mathcal{T}$ of the video
3: Initialization: $\mathcal{T}\leftarrow \varnothing$
4: for frame $f_k$ in V do
5:	▹ Handle new detections
6:       ${\mathcal{D}}_k\leftarrow Det\left(f_k\right)$
7:       ${\mathcal{D}}_{high}\leftarrow \varnothing$
8:       ${\mathcal{D}}_{low}\leftarrow \varnothing$
9:       ${\mathcal{F}}_{high}\leftarrow \varnothing$
10:       for d in ${\mathcal{D}}_k$ do
11:           if $d.score>\tau$ then
12:	▹ Store high scores detections
13:             ${\mathcal{D}}_{high}\leftarrow {\mathcal{D}}_{high}\cup \left\{d\right\}$
14:	▹ Extract appearance features using DA-BIN
15:             ${\mathcal{F}}_{high}\leftarrow {\mathcal{F}}_{high}-DA\text{-}BIN(d.box)$	▹ Equation (1): hybrid BN-IN
16:           else
17:	▹ Store low scores detections
18:             ${\mathcal{D}}_{low}\leftarrow {\mathcal{D}}_{low}\cup \left\{d\right\}$
19:           end if
20:       end for
21:	▹ Find warp matrix from k-1 to k
22:       $A_{k-1}^k\leftarrow findAffineMotion(f_{k-1},f_k)$
23:	▹ Predict new locations of tracks using CMC-GP
24:       for t in $\mathcal{T}$ do
25:           $t\leftarrow KalmanFilter\left(t\right)$	▹ Standard prediction
26:           $t\leftarrow MotionCompensation(t,A_{k-1}^k,A)$	▹ CMC-GP Equations (11)–(15)
27:       end for
28:	▹ First association
29:       $C_{iou}\leftarrow IOUDist({\mathcal{T}}_{boxes},{\mathcal{D}}_{high})$
30:       $C_{emb}\leftarrow FusionDist({\mathcal{T}}_{features},{\mathcal{F}}_{high},C_{iou})$
31:       $C_{high}\leftarrow min(C_{iou},C_{emb})$
32:       Linear assignment by Hungarian’s alg. with $C_{high}$
33:       ${\mathcal{D}}_{remain}\leftarrow$ remaining object boxes from ${\mathcal{D}}_{high}$
34:       ${\mathcal{T}}_{remain}\leftarrow$ remaining tracks from $\mathcal{T}$
35:	▹ Second association
36:       $C_{low}\leftarrow IOUDist({\mathcal{T}}_{remain},{\mathcal{D}}_{low})$
37:       Linear assignment by Hungarian’s alg. with $C_{low}$
38:       ${\mathcal{T}}_{re-remain}\leftarrow$ remaining tracks from ${\mathcal{T}}_{remain}$
39:	▹ Update matched tracks using CMC-GP
40:       Update matched tracklets Kalman filter	▹ with non-diagonal ${\mathbf{R}}_t$ Equation (14)
41:       Update tracklets appearance features	▹ using DA-BIN features Equation (1)
42:	▹ Delete unmatched tracks
43:       $\mathcal{T}\leftarrow \mathcal{T}\setminus {\mathcal{T}}_{re-remain}$
44:	▹ Initialize new tracks with CMC-GP
45:       for d in ${\mathcal{D}}_{remain}$ do
46:           if $d.score>\eta$ then
47:             $\mathcal{T}\leftarrow \mathcal{T}\cup \left\{d\right\}$	▹ Initialize on ground plane Equation (11)
48:           end if
49:       end for
50:	▹ Cascaded offline post-processing
51:       $\mathcal{T}\leftarrow LinearInterpolation\left(\mathcal{T}\right)$
52: end for
53: Return:  $\mathcal{T}$
In red are the key modules of our method.

3.1. DA-BIN:Appearance Enhancement Model Based on Domain Adaptation

The core problem faced by ReID models in MOT tasks is domain shift: statistical distribution differences exist between source domain and target domain. The multi-subsequence nature of MOT datasets exacerbates this issue, as not only do source and target domains have inconsistent distributions, but different subsequences within the target domain also exhibit varying statistical properties.

To tackle this core problem, this paper proposes the DA-BIN module with three key contributions: (1) designing a BN-IN hybrid module with learnable balancing parameters that dynamically balances batch statistics and instance statistics; (2) innovatively introducing failure scenario simulation training to actively explore overfitting/underfitting boundaries; and (3) proposing intra-domain scattering loss to enhance source domain distribution coverage through feature dispersion. This method significantly improves cross-domain generalization without requiring additional data.

The DA-BIN module is formulated as follows:

$$y=\theta ({\alpha }_B·{\widehat{X}}_B+{\beta }_B)+(1-\theta )({\alpha }_I·{\widehat{X}}_I+{\beta }_I)$$

(1)

where y represents the BN-and-IN hybrid module output, $\theta \in [0,1]$ is the learnable balance parameter, ${\alpha }_B,{\beta }_B$ are the affine parameters for the BN layer, and ${\alpha }_I,{\beta }_I$ serve the same purpose for the IN layer. ${\widehat{X}}_B$ is the batch-normalized features, and ${\widehat{X}}_I$ is the instance-normalized features.

Dynamic Adjustment of Balance Parameter $\theta$: The balance parameter $\theta$ is updated through standard backpropagation with specific gradient flow designed to explore domain adaptation boundaries. The gradient of $\theta$ concerning the total loss is determined by calculating

$${\displaystyle \frac{\partial L}{\partial \theta }}={\displaystyle \frac{\partial L}{\partial y}}·{\displaystyle \frac{\partial y}{\partial \theta }}={\displaystyle \frac{\partial L}{\partial y}}·[({\alpha }_B·{\widehat{X}}_B+{\beta }_B)-({\alpha }_I·{\widehat{X}}_I+{\beta }_I)]$$

(2)

The parameter update follows the following:

$${\theta }_{t+1}={\theta }_t-\eta {\displaystyle \frac{\partial L}{\partial \theta }}$$

(3)

where $\eta$ is the learning rate. The key insight is that different loss components drive $\theta$ in different directions:

• $L_c$ encourages $\theta$ toward values that maximize classification accuracy;
• $L_t$ with hard negatives pushes $\theta$ toward IN ($\theta \to 0$) to simulate overfitting scenarios;
• $L_s$ promotes $\theta$ toward BN ($\theta \to 1$) when domain diversity is needed.

To ensure $\theta$ remains in $[0,1]$, we apply sigmoid activation: $\theta =\sigma \left({\theta }_{\mathrm{raw}}\right)$, where ${\theta }_{\mathrm{raw}}$ is the unconstrained parameter.

However, balancing BN and IN through learnable parameters can lead to over-adaptation to the source domain style. To solve this, we use three loss functions to update the model while simulating unsuccessful scenarios during training.

First, we use cross-entropy loss $L_c$ for target identity learning:

$$L_c(X_B;\theta ,\varphi )={\displaystyle \frac{1}{N_B}}\sum _{i=1}^{N_B}{l}_c(g_{\varphi }\left(f_{\theta }\left(x_i\right)\right),y_i)$$

(4)

where $X_B$ represents the mini-batch samples, $\theta$ represents the balance parameter, $\varphi$ represents the classifier parameters, $N_B$ is the batch size, $x_i$ is the i-th sample, $y_i$ is its label, $f_{\theta }(·)$ is the feature extractor, $g_{\varphi }(·)$ is the classifier, and $l_c(·)$ is the cross-entropy function.

Second, we employ triplet loss to simulate generalization failures by inducing the balancing parameter towards IN (examining overfitting) and towards BN (exploring underfitting):

$$L_t(X_B;\theta )={\displaystyle \frac{1}{N_B}}\sum _{i=1}^{N_B}{[e(f_i^a,f_i^p)-e(f_i^a,f_i^n)+m]}_{+}$$

(5)

Failure Scenario Simulation Mechanism: The triplet loss is designed to deliberately create challenging scenarios:

• When hard negative mining selects very similar inter-class samples, the gradient ${\displaystyle \frac{\partial L_t}{\partial \theta }}$ tends to be positive, pushing $\theta$ toward 0 (more IN), simulating overfitting to instance-specific features;
• When positive samples are diverse within the same identity, the gradient tends to be negative, pushing $\theta$ toward 1 (more BN), exploring potential underfitting to batch statistics;
• This creates a dynamic exploration of the $\theta$ space, preventing the model from settling into suboptimal local minima.

Where $L_t(·)$ is the triplet loss function, $f_i^a=f_{\theta }\left(x_i^a\right)$ is the anchor feature vector, $f_i^p$ and $f_i^n$ are the hardest positive and negative features in the mini-batch, $e(·)$ is the Euclidean distance, m is the margin parameter, and ${\left[z\right]}_{+}=max(z,0)$.

Third, to enhance intra-domain diversity and extend the source domain feature distribution, we introduce the intra-domain scattering loss:

$$L_s(X_s;\theta )={\displaystyle \frac{1}{N_s}}\sum _{p=1}^{P_s}\sum _{i=1}^{N_s^p}(1-c(f_i^p,\overline{f^p}))$$

(6)

where $\overline{f^p}$ is the mean feature vector of domain p in the mini-batch, $P_s$ is the number of source domains, $N_s^p$ is the number of samples in domain p, $N_s$ is the total number of samples, and $c(·)$ represents cosine similarity.

The total loss function combines all three components:

$$L(X_B;\theta ,\varphi )={\lambda }_1{L}_c(X_B;\theta ,\varphi )+{\lambda }_2{L}_t(X_B;\theta )+{\lambda }_3{L}_s(X_s;\theta )$$

(7)

where ${\lambda }_1,{\lambda }_2,$ and ${\lambda }_3$ are weighting coefficients to balance the three loss functions.

Training Dynamics and Convergence: During training, $\theta$ exhibits the following behavior patterns:

• Early stage: $\theta$ oscillates significantly as the model explores different BN-IN combinations;
• Middle stage: $\theta$ gradually stabilizes but maintains sensitivity to hard samples;
• Late stage: $\theta$ converges to domain-specific optimal values while retaining adaptability.

The convergence pattern indicates successful boundary exploration without overfitting to any single normalization strategy.

Through this multi-objective optimization, we induce parameter bias via three loss functions. By simulating unsuccessful target ReID scenarios during parameter updates, we extend the source domain’s feature distribution and enhance the model’s generalization to unknown target domains.

3.2. CMC-GP: Camera Motion Compensation Model on Ground Plane

Traditional KF-based tracking operates in the image plane, where camera motion introduces pseudo-motion that degrades tracking accuracy. To address this fundamental limitation, this paper proposes the CMC-GP module with key innovations: (1) projecting KF from image plane to ground plane to eliminate camera-induced pseudo-motion interference; (2) designing non-diagonal measurement noise covariance matrix to model spatial correlations in ground motion; (3) integrating camera parameters through Jacobian transformation for accurate noise propagation. This method significantly improves tracking precision in moving camera scenarios.

In traditional image plane tracking, the KF uses the following formulation:

$$\begin{array}{cc}\hfill {\mathit{x}}_t^{img}& ={[x_t,y_t,a_t,h_t,{\dot{x}}_t,{\dot{y}}_t,{\dot{a}}_t,{\dot{h}}_t]}^T\hfill \\ \hfill {\mathit{z}}_t^{img}& ={[z_t\left(x\right),z_t\left(y\right),z_t\left(a\right),z_t\left(h\right)]}^T\hfill \\ \hfill {\mathit{Q}}_t^{img}& =\mathrm{diag}\left({\left({\sigma }_p{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_p{\widehat{h}}_{t-1}\right)}^2,\right.\hfill \\ \hfill & {\left({\sigma }_p{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_p{\widehat{h}}_{t-1}\right)}^2,\hfill \\ \hfill & {\left({\sigma }_v{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_v{\widehat{h}}_{t-1}\right)}^2,\hfill \\ \hfill & \left.{\left({\sigma }_v{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_v{\widehat{h}}_{t-1}\right)}^2\right)\hfill \\ \hfill {\mathit{R}}_t^{img}& =\mathrm{diag}\left({\left({\sigma }_m{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_m{\widehat{h}}_{t-1}\right)}^2,\right.\hfill \\ \hfill & \left.{\left({\sigma }_m{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_m{\widehat{h}}_{t-1}\right)}^2\right)\hfill \end{array}$$

(8)

where $(x_t,y_t)$ are the horizontal and vertical coordinates of the BBox in the image plane, $a_t$ is the aspect ratio (width-to-height) of the BBox, $h_t$ is the height of the BBox, and ${\dot{x}}_t,{\dot{y}}_t,{\dot{a}}_t,$ and ${\dot{h}}_t$ represent the corresponding rate of change. $({\sigma }_p,{\sigma }_v)$ are the process noise factors, and ${\sigma }_m$ is the measurement noise factor.

To minimize motion estimation deviations, we estimate object motion directly on the ground plane. Based on the linear camera model [35], the mapping relationship between the ground-truth coordinates $(e_t,f_t)$ and the image plane coordinates $(x_t,y_t)$ can be expressed as

$$\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\mathbf{A}\left[\begin{array}{c}e\\ f\\ 1\end{array}\right]$$

(9)

where $\mathbf{A}={\left[a_{ij}\right]}_{3\times 3}$ denotes the projection matrix, which is the product of the camera’s internal and external parameters:

$$\mathbf{A}=\mathbf{K}·[\mathbf{R}\mid \mathit{t}]$$

(10)

where $\mathbf{K}$ is the camera intrinsic matrix, $\mathbf{R}$ is the rotation matrix, and $\mathit{t}$ is the translation vector.

Ground Plane Projection: Our method transforms the traditional 8-dimensional image state into a simplified 4-dimensional ground state by mapping from the image plane to the ground plane. This projection eliminates camera motion artifacts while preserving essential motion dynamics.

The transformed state vector, measurement vector, and covariance matrices for ground plane tracking are

$${\mathbf{x}}_t={[e_t,f_t,{\dot{e}}_t,{\dot{f}}_t]}^T$$

(11)

$${\mathbf{z}}_t={[e_t\left(x\right),f_t\left(y\right)]}^T$$

(12)

$${\mathbf{Q}}_t=\left[\begin{array}{cccc}{\displaystyle \frac{\Delta t^4}{4}}& 0& {\displaystyle \frac{\Delta t^3}{2}}& 0\\ 0& {\displaystyle \frac{\Delta t^4}{4}}& 0& {\displaystyle \frac{\Delta t^3}{2}}\\ {\displaystyle \frac{\Delta t^3}{2}}& 0& \Delta t^2& 0\\ 0& {\displaystyle \frac{\Delta t^3}{2}}& 0& \Delta t^2\end{array}\right]·\mathrm{diag}({\sigma }_p^2,{\sigma }_p^2,{\sigma }_v^2,{\sigma }_v^2)$$

(13)

Non-diagonal Noise Modeling: The key innovation lies in the non-diagonal measurement noise covariance matrix ${\mathbf{R}}_t$, which captures the inherent correlations between lateral and longitudinal movements on the ground plane:

$$\begin{array}{cc}\hfill {\mathbf{R}}_t& ={\mathbf{J}}_t·\mathrm{diag}({\left({\sigma }_m{\widehat{a}}_{t-1}\right)}^2,{\left({\sigma }_m{\widehat{h}}_{t-1}\right)}^2)·{\mathbf{J}}_t^T\hfill \end{array}$$

(14)

where ${\mathbf{J}}_t$ is the Jacobian matrix describing the sensitivity of image coordinates to ground plane coordinates:

$${\mathbf{J}}_t=\left[\begin{array}{cc}{\displaystyle \frac{\partial x_t}{\partial e_t}}& {\displaystyle \frac{\partial x_t}{\partial f_t}}\\ {\displaystyle \frac{\partial y_t}{\partial e_t}}& {\displaystyle \frac{\partial y_t}{\partial f_t}}\end{array}\right]=\left[\begin{array}{cc}{\displaystyle \frac{a_{11}-a_{31}{e}_t}{a_{31}{e}_t+a_{33}}}& {\displaystyle \frac{a_{12}-a_{32}{e}_t}{a_{31}{e}_t+a_{33}}}\\ {\displaystyle \frac{a_{21}-a_{31}{f}_t}{a_{31}{f}_t+a_{33}}}& {\displaystyle \frac{a_{22}-a_{32}{f}_t}{a_{31}{f}_t+a_{33}}}\end{array}\right]$$

(15)

where $e_t$ and $f_t$ are the ground plane coordinates, and ${\dot{e}}_t,{\dot{f}}_t$ are their velocities.

Correlation Modeling Significance: The non-diagonal matrix ${\mathbf{R}}_t$ shows that measurement noise in different dimensions is correlated. Object motion patterns on the ground plane are often intrinsically linked, as lateral and longitudinal movements can be affected by common disturbances like camera rotation or translation. By reflecting this correlation in ${\mathbf{R}}_t$, the KF can more accurately represent the statistical properties of measurement noise. This allows for a better handling of uncertainties during data association. Our CMC-GP module leverages the non-diagonal properties of ${\mathbf{R}}_t$ for more precise object association. Specifically, by considering the correlation of measurement noise, the KF can more accurately estimate an object’s position and velocity on the ground plane, improving trajectory prediction. This is crucial when the camera is in motion, as it can introduce extra noise and uncertainties.

4. Experiments

4.1. Implementation Details

Our ReID model employed ResNet50 [36] backbone with an additional fully connected layer, trained exclusively on Market1501 [37] for 60 epochs without MOT dataset enhancement. We used SGD optimizer (momentum = 0.9) with learning rate reduction at epochs 20 and 40, achieving 86.3% rank-1 accuracy on Market1501. All experiments used a YOLOX [38] detector with identical configurations across all datasets. Lost tracks were retained for 30 frames for potential re-association. Experiments were run on Intel i9-10900X CPU and NVIDIA RTX3090 GPU with Ubuntu 20.04, CUDA 11.6, and PyTorch 1.13.1.

4.2. Datasets

We evaluated on three challenging MOT benchmarks:

DanceTrack: Features similar appearances and irregular movements in dance scenarios, emphasizing data association over detection. Contains 100 videos (40 train, 25 validation, 35 test) with 105,834 total frames.

MOT20: Focuses on extremely dense crowds (up to 246 people per frame), testing tracking robustness in frequent occlusion scenarios. Contains 8 videos with 13,410 frames total (4 train, 4 test). Training follows the same split strategy as DanceTrack.

MOT17: A standard MOT benchmark with 28 sequences (14 train, 14 test) containing 11,235 frames. Each sequence provides three detector variants (DPM, FRCNN, SDP). We apply consistent training/validation splits across all variants.

4.3. Evaluation Metrics

We employed three standard MOT metrics:

HOTA [39]: Higher-Order Tracking Accuracy balancing detection accuracy (DetA), association accuracy (AssA), and localization precision. Primary evaluation metric.

MOTA [40]: Multiple-Object Tracking Accuracy measuring detection errors (FP, FN) and IDSW. Higher values indicate better performance.

IDF1 [41]: Identity F-Score evaluating the proportion of correctly identified detections, emphasizing tracking continuity and ReID accuracy.

4.4. Benchmark Results

DCTrack achieves SOTA performance across all three datasets, demonstrating superior accuracy while maintaining real-time efficiency. Despite incorporating ReID computation, our method remains practical for real-world MOT applications. The best results are highlighted in bold, with ↑/↓ indicating performance direction.

The DanceTrack is characterized by irregular object motions and similar appearances, which can cause identity confusion errors in appearance-based tracking methods and challenge motion-based methods to handle motion changes. As shown in

Table 1. Comparison of our method with other SOTA algorithms on DanceTrack. The best results are shown in bold.

Tracker	HOTA↑/%	MOTA↑/%	IDF1↑/%	DetA↑%	AssA↑%
FairMOT [42]	39.7	82.2	40.8	66.7	23.8
CenterTrack [43]	41.8	86.8	35.7	78.1	22.6
TransTrack [44]	45.5	88.4	45.2	75.9	27.5
DeepSORT [45]	45.6	87.8	47.9	71.0	29.7
QDTrack [46]	45.7	83.0	44.8	72.1	29.2
ByteTrack [25]	47.3	89.5	52.5	71.6	31.4
SORT [24]	47.9	91.8	50.8	72.0	31.2
OC-SORT [2]	55.7	92.0	54.6	81.7	38.3
C-BIoU Tracker [47]	60.6	91.6	61.6	81.3	45.4
Hybrid-SORT [48]	62.2	91.6	63.0	-	-
Ours	67.3	91.2	65.8	83.7	47.6

with DanceTrack, DCTrack achieves a 5.1% improvement in the HOTA metric over Hybrid-SORT, mainly due to the synergy of the DA-BIN and CMC-GP modules. The DA-BIN module enhances the generalization of ReID models across domains by simulating domain shift scenarios. On DanceTrack, where objects have similar appearances and complex motion patterns, models are prone to domain differences. DA-BIN combines BN and IN with a learnable balance parameter, enabling models to better adapt to varying appearance distributions and reducing misassociation due to similar object appearances. This improves identity association accuracy. The CMC-GP module addresses motion estimation deviations from camera movements by performing camera motion compensation on the ground plane. On DanceTrack, camera movements can significantly change object projection positions on the image plane, increasing motion estimation difficulty. CMC-GP uses a KF to simulate object motion on the ground plane and employs unified compensation parameters for more accurate object position prediction. This reduces trajectory prediction errors caused by camera movements and boosts the model’s robustness in handling irregularly moving objects.

As shown in

Table 2. Comparison of our method with other SOTA algorithms on MOT20. The best results are shown in bold.

Tracker	HOTA↑/%	MOTA↑/%	IDF1↑/%	FP↓	FN↓	IDs↓	FPS↑
GSDT [49]	53.6	67.1	67.5	31,913	135,409	3131	0.9
FairMOT	54.6	61.8	67.3	103,440	88,910	5243	13.2
OC-SORT	55.2	61.7	67.9	5700	192,000	508	29
RelationTrack [18]	56.5	67.2	70.5	61,134	104,597	4243	2.7
MOTRv2 [50]	61.0	76.2	73.1	-	-	-	-
ByteTrack	61.3	77.8	75.2	26,249	87,594	1223	17.5
StrongSORT [38]	62.6	73.8	77.0	16,632	117,920	770	1.4
MotionTrack [28]	62.8	78.0	76.5	28,629	84,152	1165	-
BoTSORT [6]	63.3	77.8	77.5	24,638	88,863	1257	2.4
SMILETTRACK [39]	63.4	78.2	77.5	24,554	85,548	1318	7.2
Ours	65.9	79.1	80.1	27,252	79,263	486	15.6

with the MOT20 dataset, DCTrack achieves a 2.5% increase in HOTA metric compared to SMILEtrack. MOT20 is known for its extremely high target density and frequent occlusions. The DA-BIN module enhances to distinguish different targets in high-density scenarios by improving its adaptation to varying appearance features across domains. The CMC-GP module reduces trajectory crossing and identity confusion errors caused by dense target motion through more accurate motion estimation. Consequently, the tracking performance is enhanced.

For MOT17, DCTrack also surpasses SMILEtrack in key metrics. As shown in

Table 3. Comparison of our method with other SOTA algorithms on MOT17. The best results are shown in bold.

Tracker	HOTA↑/%	MOTA↑/%	IDF1↑/%	FP↓	FN↓	IDs↓	FPS↑
OC-SORT	52.9	59.4	65.7	6600	222,000	801	29.0
TransTrack	54.1	75.2	63.5	50,157	86,442	3603	10.0
FairMOT	59.3	73.7	72.3	27,507	117,477	3303	25.9
RelationTrack	61.0	73.8	74.7	27,999	118,623	1374	-
MOTRv2	62.0	78.6	75.0	-	-	-	-
ByteTrack	63.1	80.3	77.3	25,491	83,721	2196	29.6
StrongSORT	64.4	79.6	79.5	27,876	86,205	1194	7.1
BoTSORT	65.0	80.5	80.2	22,521	86,037	1212	4.5
MotionTrack	65.1	81.1	80.1	23,802	81,660	1140	-
SMILEtrack	65.3	81.1	80.5	22,963	79,428	1246	5.6
Ours	67.8	81.9	83.9	24,061	77,483	637	22.1

with MOT17, object motion patterns are highly linear, yet occlusions and interactions between objects still occur. The DA-BIN module enhances the model’s generalization ability, enabling it to better handle appearance changes across different scenarios. The CMC-GP module ensures trajectory continuity and stability in long-sequence tracking through effective camera motion compensation, reducing ID switches and fragmented trajectories.

4.5. Ablation Studies

To validate the impact of the DA-BIN module and the CMC-GP module on tracking performance and quantify the contributions of each module, we conducted comprehensive ablation experiments on DanceTrack, MOT20, and MOT17, as shown in

Table 4. Each component of our method is ablated and compared on DanceTrack, MOT20, and MOT17. DA-BIN = Domain Adaptation-based Batch Instance Normalization. CMC-GP = camera motion compensation on ground plane. The best results are shown in bold.

Datasets	Method	DA-BIN	CMC-GP	HOTA↑/%	MOTA↑/%	IDF1↑/%
DanceTrack	A	✕	✕	60.2	89.5	59.5
	B	✕	✓	61.1	90.3	60.6
	C	✓	✕	65.9	90.8	63.2
	D	✓	✓	67.3	91.2	65.8
MOT20	A	✕	✕	61.3	77.8	75.2
	B	✕	✓	63.5	78.3	76.8
	C	✓	✕	64.2	78.9	77.4
	D	✓	✓	65.9	79.1	80.1
MOT17	A	✕	✕	63.1	80.3	77.3
	B	✕	✓	65.5	80.7	79.4
	C	✓	✕	66.2	81.2	81.6
	D	✓	✓	67.8	81.9	83.9

.

We designed four experimental configurations:

• Method A: Baseline method without proposed modules;
• Method B: Baseline + CMC-GP module only;
• Method C: Baseline + DA-BIN module only;
• Method D: Baseline + both DA-BIN and CMC-GP modules (DCTrack).

DA-BIN Module Analysis: Comparing Method A with Method C reveals significant performance improvements: on DanceTrack, there is a +5.7 HOTA, +1.3 MOTA, and +3.7 IDF1 boost; on MOT20, a +2.9 HOTA, +1.1 MOTA, and +2.2 IDF1 improvement; and on MOT17, a +3.1 HOTA, +0.9 MOTA, and +4.3 IDF1 gain. The results highlight the module’s positive impact on tracking performance. Notably, on DanceTrack, irregular motion and dense occlusions create many low-scoring boxes in crowded scenes, making the use of limited appearance features critical. Our method achieves a remarkable +5.7 boost in HOTA. The experimental results demonstrate that the DA-BIN module enhances the appearance model’s generalization ability on MOT datasets, thereby improving the tracker’s overall performance.

CMC-GP Module Analysis: Comparing Method A with Method B reveals the following performance improvements: on DanceTrack, there are gains of +0.9 HOTA, +0.8 MOTA, and +1.1 IDF1; on MOT20, improvements of +2.2 HOTA, +0.5 MOTA, and +1.6 IDF1 are seen; and on MOT17, enhancements of +2.4 HOTA, +0.4 MOTA, and +2.1 IDF1 are observed. These results demonstrate our CMC-GP module’s positive impact on tracking performance. In particular, on MOT17, sequences 05, 10, 11, and 13 involve significant camera motion. Our proposed method achieves a +2.4 boost in HOTA, indicating that the CMC-GP module effectively addresses motion estimation deviations, thereby improving tracker accuracy and stability. Although DanceTrack sequences do not exhibit significant viewpoint changes, our CMC-GP module still leads to performance improvements, showing its effectiveness even with minor camera jitter.

Combined Module Analysis: Finally, we analyzed the joint impact of the DA-BIN and CMC-GP. By comparing Method A with Method D, we observe improvements across all three metrics on the three validation datasets. Specifically, the HOTA metric shows significant improvements of +7.1 on DanceTrack, +5.6 on MOT20, and +4.7 on MOT17. The ablation studies demonstrate that DCTrack can significantly enhance MOT results through the complementary effects of both modules.

As shown in

Table 5. Comparison between our method and CMC method on DanceTrack, MOT20, and MOT17. The best results are shown in bold.

Method	CMC	CMC-GP	HOTA↑/%	FPS↑
DanceTrack
Baseline	✕	✕	60.2	18.7
Baseline + CMC	✓	✕	62.9	15.9
DCTrack	✕	✓	67.3	17.3
DCTrack+	✓	✓	66.7	14.2
MOT20
Baseline	✕	✕	61.3	17.5
Baseline + CMC	✓	✕	63.2	14.7
DCTrack	✕	✓	65.9	15.6
DCTrack+	✓	✓	66.3	12.8
MOT17
Baseline	✕	✕	63.1	29.6
Baseline + CMC	✓	✕	65.3	20.7
DCTrack	✕	✓	67.8	22.1
DCTrack+	✓	✓	68.2	18.3

, we explored the impact of CMC on DCTrack across three datasets. Our CMC-GP outperforms traditional CMC in all cases, with better efficiency due to using unified compensation parameters instead of per-frame compensation, which reduces computational overhead. When applied to DCTrack, CMC-GP consistently boosted performance on MOT20 and MOT17. However, on DanceTrack, traditional CMC caused a performance drop due to minimal viewpoint changes between frames, leading to incorrect trajectory matches from slight object movements. In contrast, CMC-GP showed greater robustness to minor object movements, demonstrating the effectiveness of our ground plane-based approach over conventional frame-by-frame compensation methods.

Analysis of the Balancing Parameter $\theta$: A key parameter in the DA-BIN module for balancing BN and IN weights, $\theta$ is dynamically adjusted via a loss function with three components: cross-entropy loss $L_c$ for target identity learning, triplet loss $L_t$ for similarity learning, and intra-domain scattering loss $L_s$ for enhancing domain diversity. Early in training, $\theta$ is driven by $L_c$ due to the model’s poor adaptation to the source domain, gradually shifting toward IN. As training progresses and the model adapts to the source style, $L_t$ and $L_s$ become more influential. $L_t$ improves feature discrimination by similarity learning, pushing $\theta$ toward BN. $L_s$ increases intra-domain diversity, preventing overfitting to the source style and dynamically adjusting $\theta$ between BN and IN.

4.6. Visualization Results

We conducted a qualitative analysis to demonstrate the effectiveness of our proposed modules. Figure 3 shows frames 189–192 from the dancetrack0008 dataset under severe occlusion conditions, and Figure 4 shows frames 287–290 from the same dataset under dense occlusion conditions. In both figures, group (a) represents the baseline algorithm and group (b) represents our DCTrack algorithm.

Severe Occlusion Analysis: As shown in the 190th frame of Figure 3, severe occlusion occurs between tracking targets ID 1 and 8, as well as ID 2 and 3, with targets 3 and 8 having visibility of less than 0.2. Our method, utilizing the DA-BIN module, fully exploits appearance features for severely occluded targets, allowing the tracker to continue matching trajectories accurately. In contrast, the baseline algorithm fails to handle severe occlusion effectively, causing targets 3 and 8 to lose matching opportunities. The results show that our DCTrack greatly boosts the tracker’s capacity to manage severe occlusion by optimally utilizing limited appearance features.

These visualization results validate the complementary effects of DA-BIN and CMC-GP modules: DA-BIN enhances appearance-based matching under severe occlusion through improved cross-domain generalization, while CMC-GP maintains accurate motion prediction in camera motion scenarios, together providing robust tracking performance in challenging conditions.

Dense Occlusion Analysis: In the densely occluded scenario shown in the 288th frame of Figure 4, frequent object movement, interaction, and occlusion generate numerous low-confidence detections. The CMC-GP module reduces motion estimation errors caused by camera movement, while the DA-BIN module recovers the correct IDs of targets, boosting accurate tracking of low-confidence detections affected by camera motion. The baseline algorithm, which matches all trajectories simultaneously without properly handling dense occlusion, causes incorrect tracking of targets 4 and 6 and loses target 7. The results show that our proposed MOT method, combining domain adaptation and camera motion compensation, increases target matching likelihood in dense occlusion scenarios and reduces identity switches caused by occlusion.

In Figure 5, we analyze the distribution ratio of the balancing parameter, based on experiments conducted on DanceTrack. All balancing parameters are initialized to 1, favoring BN. It can be observed that $\theta$ retains BN characteristics, with only a few shifting toward IN. This shift improves performance, proving that eliminating irrelevant information on source domain style enhances generalization. Specifically, DA-BIN shows that in middle and shallow layers, shifting slightly toward IN effectively reduces overfitting and boosts generalization.

In Figure 6, the cross-entropy loss function $L_c$ creates a positive gradient, pushing $\theta$ toward IN. The triplet loss function $L_t$ generates a negative gradient, pulling $\theta$ toward BN. The intra-domain scattering loss function $L_s$ enables dynamic adjustment of $\theta$ between BN and IN. We demonstrate that training via the total loss function $L(X_B;\theta ,\varphi )$ guides $\theta$ movement in specific directions. This means that simulating unsuccessful generalization scenarios in advance can enhance the tracker’s overall performance, as shown in Table 4.

Limitations and Applicability: While CMC-GP significantly improves tracking accuracy for ground-based scenarios, it relies on the ground plane assumption. The method is designed for typical MOT applications where objects move on relatively flat surfaces, such as pedestrian and vehicle tracking in urban environments. For scenarios involving aerial targets (e.g., UAVs) or significant height changes (e.g., staircases), traditional image plane tracking would be more appropriate. Current MOT benchmarks (MOT17, MOT20, DanceTrack) inherently satisfy this assumption, validating our approach for mainstream tracking applications.

5. Conclusions

This work addresses two fundamental challenges in multi-object tracking: domain adaptation failures in appearance models and camera motion interference in motion estimation. Our DCTrack framework introduces paradigmatic solutions through two key innovations. First, the DA-BIN module revolutionizes domain adaptation by actively simulating failure scenarios during training. Through dynamic balancing of BN and IN with multi-objective optimization, DA-BIN achieves superior cross-domain generalization without additional training data. And second, the CMC-GP module transforms motion estimation by projecting KF from image coordinates to ground plane coordinates. This geometric reformulation eliminates camera-induced pseudo-motion while modeling spatial correlations through non-diagonal noise covariance matrices. Results show SOTA performance across three benchmarks (DanceTrack: 67.3% HOTA, MOT20: 66.9% HOTA, MOT17: 67.8% HOTA) while maintaining real-time capability using only standard ReID training data. Our approach establishes new paradigms for domain adaptation and motion compensation with broader implications for computer vision applications involving cross-domain deployment and camera motion.