Search Region-Guided Adaptive Template Update for Robust Multi-Modal UAV Tracking

Highlights

What are the main findings?

• We propose GTUTrack, a search region-guided adaptive template update framework for robust multi-modal UAV tracking, which jointly performs adaptive template selection, thresholding, and memory management.
• GTUTrack achieves state-of-the-art performance on VTUAV and also shows strong generalization on RGBT210, RGBT234, and LasHeR under challenging conditions such as occlusion, illumination variation, and scale change.

What are the implications of the main findings?

• The results show that template update in multi-modal UAV tracking should be conditioned on the current search region rather than relying on fixed update intervals and manually designed thresholds.
• The proposed framework provides an effective solution for robust target tracking in UAV remote sensing and other multi-modal surveillance scenarios with complex appearance variation and modality discrepancy.

Abstract

Existing multi-modal UAV tracking methods typically rely on fixed-interval dynamic template update strategies to capture diverse target appearances, together with predefined thresholds to select high-quality search regions for template update. However, due to the irregular motion of targets and the complexity of real-world scenarios, such passive update mechanisms suffer from notable limitations. Fixed sampling intervals often fail to adequately capture appearance variations, while fixed threshold-based selection is insufficient to accommodate diverse imaging conditions, leading to ineffective updates or the introduction of noisy templates, thereby degrading tracking robustness and accuracy. To address these issues, we propose a search region-guided adaptive dynamic template update framework for robust multi-modal UAV tracking, aiming to improve both scene adaptability and target matching capability. Specifically, we design a Guided Template Selection Transformer, which dynamically matches templates conditioned on the current search region, enabling the tracker to autonomously select the most suitable template for the target’s current state. Furthermore, we introduce a Dynamic Threshold Module that adaptively adjusts template selection criteria according to different tracking scenarios, ensuring the reliability and contextual relevance of candidate templates. In addition, we develop a Dynamic Template Memory Module to maintain an ordered repository of target templates under different target states, providing a structured and high-quality template pool for the proposed selection mechanism. Extensive experiments on a standard multi-modal UAV tracking benchmark demonstrate that the proposed method significantly outperforms existing approaches, effectively overcoming the limitations of conventional fixed update strategies. Moreover, the proposed approach exhibits strong generalization capability across three additional multi-modal tracking datasets from typical surveillance scenarios.

1. Introduction

Vision-based remote sensing image analysis is currently booming, attracting numerous researchers to engage in relevant studies, such as the analysis of SAR images. Xue et al. [1] propose a lightweight modality compensation network to transfer multimodal knowledge to single-SAR models via knowledge distillation, solving the problems of missing AIS data and high computational cost to balance ship detection accuracy and real-time performance. Ai et al. [2] present MKSFF-CNN, which employs multi-scale parallel convolutions to extract diverse features and fuse them reasonably, improving feature integrity and achieving superior SAR target classification performance verified on the MSTAR dataset. Ai et al. [3] propose a SAR multi-target ship detection algorithm based on MSRIHL-CNN, which adopts TCS-JCFAR for efficient prescreening and fuses MSRI-HL low-level features with CNN high-level features for discrimination, achieving excellent performance verified on Gaofen-3 SAR images. Multi-modal UAV tracking [4] has attracted increasing attention in recent years due to its wide applications in aerial surveillance, disaster response, intelligent transportation systems, border patrol, and public safety monitoring. Compared with conventional ground-view tracking, UAV-based tracking [5,6,7] provides a broader field of view and more flexible observation perspectives, making it particularly suitable for large-scale dynamic scene perception. By leveraging complementary information from multiple modalities, such as RGB and thermal infrared [8,9], multi-modal UAV tracking is expected to achieve more robust performance under challenging conditions, including illumination variation, partial occlusion, background clutter, and adverse weather. In particular, RGB data provide rich texture and structural cues under favorable lighting conditions, while thermal infrared data offer stronger target saliency in low-light or visually degraded environments. The integration of these modalities therefore provides an effective way to improve target perception in complex UAV tracking scenarios.

Despite these advantages, robust multi-modal UAV tracking remains highly challenging. In practical UAV scenarios, targets often exhibit rapid motion, abrupt scale variation, significant viewpoint changes, and frequent occlusion due to camera motion, altitude changes, or scene complexity. Moreover, UAV platforms typically capture targets at relatively low resolution, making the target appearance more ambiguous and unstable than in conventional surveillance settings. These characteristics make it difficult for a tracker to maintain a reliable and discriminative target representation over time. As a result, effectively modeling dynamic target appearance variations remains a critical challenge in multi-modal UAV tracking.

To alleviate this issue, dynamic template updating has been widely adopted as a key component in modern multi-modal UAV trackers [9]. The underlying motivation is intuitive: as the target appearance evolves over time, the tracker should continuously update its target representation by incorporating newly observed appearances under varying poses, scales, and illumination conditions. In this way, a dynamic template pool can be maintained to cover target appearance variations throughout the tracking process. Dynamic template update is particularly important for UAV tracking, where target states can change rapidly across frames and the mismatch between outdated templates and the current target appearance may easily lead to degraded matching quality or even tracking drift. Therefore, a reliable template update mechanism is essential for preserving robust target representation and improving long-term tracking stability.

However, due to the intrinsic complexity of multi-modal UAV tracking scenarios and the stochastic nature of target motion, existing template update strategies remain largely heuristic and exhibit limited adaptability. Specifically, current approaches [9,10,11,12] typically rely on fixed update intervals and predefined thresholds for template selection, which suffer from several inherent limitations. First, the irregular motion patterns and nonlinear appearance changes of targets make fixed update intervals fundamentally suboptimal. They may introduce redundant samples when the target state is stable, leading to unnecessary computation and noise accumulation, while failing to capture critical appearance variations under abrupt motion or scale changes, resulting in insufficient coverage of the target appearance distribution and potential tracking drift. Second, the significant variation in imaging quality across different scenarios [13,14] further limits the effectiveness of fixed threshold-based selection. For example, the thermal modality often dominates under low-light conditions, whereas both modalities may degrade in adverse environments such as fog or smoke. In UAV tracking, this issue is often more pronounced because aerial viewpoints, altitude variation, and camera motion can further amplify image blur, background interference, and modality inconsistency. Such variability makes predefined thresholds difficult to generalize: overly strict thresholds may discard informative samples, while loose thresholds tend to introduce low-quality templates, undermining the reliability of the template pool. Furthermore, existing template memory mechanisms typically lack structured management [9,15], leading to redundancy and disorder within the template pool. This not only hinders efficient template retrieval and matching but also degrades tracking performance. In many cases, newly collected templates may contain overlapping or low-quality target appearances, while historically reliable templates are not explicitly preserved and organized. As a result, the template pool may gradually accumulate noisy or redundant templates, weakening its ability to provide stable target representations for subsequent matching. Meanwhile, the inherent modality discrepancy between RGB and thermal infrared data introduces additional challenges in feature alignment, making template selection more susceptible to cross-modal noise and further reducing tracking accuracy.

From this perspective, the key difficulty of multi-modal UAV tracking is not merely determining how to update templates more frequently, but how to update them more appropriately. An effective template update strategy should answer three closely related questions: when to update, which template should be selected, and how the selected templates should be stored and managed over time. Existing passive strategies based on fixed rules often fail to address these questions in a unified and adaptive manner. This observation motivates us to rethink template update from the perspective of search-region-guided decision making, where the current search region serves as an active cue for evaluating template relevance, estimating update reliability, and organizing template memory.

To address the above challenges, we propose a search region-guided adaptive template update framework for robust multi-modal UAV tracking, which establishes a fully adaptive pipeline for template selection, update, and memory management, as shown in Figure 1. Instead of relying on passive and predefined update rules, the proposed framework actively leverages the current search region to guide template-related decisions. In this way, template update becomes a context-aware process that is dynamically adjusted according to the current target state and scene characteristics.

Specifically, we design a Guided Template Selection Transformer (GTST) that formulates template selection as a search region-conditioned attention matching process. This enables the tracker to dynamically select the most relevant template according to the current target state, achieving precise alignment between the template and the search region. Unlike the CSIL [16] framework designed based on the center-to-surrounding interaction structure, the GTST proposed in this paper is built on the Vision Transformer architecture, focusing on the multi-modal UAV tracking task and improving tracking robustness through an adaptive template selection mechanism. Furthermore, we introduce a Dynamic Threshold Module (DTM) to adaptively adjust template selection criteria by modeling scene characteristics and tracking confidence, thereby overcoming the limitations of fixed thresholds. In addition, we develop a Dynamic Template Memory Module (DTMM) to maintain an ordered and structured template pool, supporting efficient template storage, update, and replacement while avoiding redundancy and disorder. The three components work collaboratively: GTST improves template–target matching, DTM ensures reliable candidate template generation, and DTMM provides a structured memory basis for robust adaptive template update.

Extensive experiments on a standard multi-modal UAV tracking benchmark, i.e., VTUAV [4], demonstrate that the proposed method consistently outperforms state-of-the-art approaches. These results verify the effectiveness of the proposed framework in UAV-specific scenarios characterized by small targets, fast motion, and viewpoint variation. Moreover, our method generalizes well to other multi-modal tracking datasets from diverse surveillance scenarios, including RGBT210 [17], RGBT234 [18], and LasHeR [19], validating its effectiveness and adaptability.

The contributions of this paper are summarized as follows:

• We propose a search region-guided adaptive template update framework that enables fully adaptive template selection, update, and memory management for multi-modal UAV tracking.
• We design a Guided Template Selection Transformer (GTST) to dynamically match templates conditioned on the current search region, improving target representation and matching accuracy.
• We introduce a Dynamic Threshold Module (DTM) to adaptively control template selection across diverse scenarios, enhancing robustness.
• We develop a Dynamic Template Memory Module (DTMM) to construct a structured and high-quality template pool, alleviating redundancy and improving efficiency.
• Extensive experiments on four benchmark datasets demonstrate that our tracker achieves state-of-the-art performance.

2. Method

In this section, we present the proposed multi-modal UAV tracking framework, termed GTUTrack, which incorporates a search region-guided adaptive template update mechanism. We first introduce the overall architecture and data flow of GTUTrack. We then describe the proposed Guided Template Selection Transformer, Dynamic Threshold Module, and Dynamic Template Memory Module in detail. Finally, we present the loss functions used for model optimization.

2.1. Overall Framework

The overall architecture of GTUTrack is illustrated in Figure 2. The model takes as input an initial template pair, a dynamic template pair, and a search region pair. The overall tracking architecture follows the baseline tracker OSTrack [20]. During tracking, the Dynamic Threshold Module (DTM) adaptively determines the confidence threshold for template update according to the current tracking scenario. When the tracking confidence score of the current search region exceeds the corresponding update threshold, the search region is cropped into template format and stored in the Dynamic Template Memory Module (DTMM), which consists of Historical Template Memory, Candidate Template Memory, and Fixed Template Memory.

Specifically, the Historical Template Memory stores template pairs that have been selected as dynamic templates so as to accumulate reliable historical target appearance variations. The Candidate Template Memory stores candidate templates obtained by cropping reliable search region pairs. The Fixed Template Memory preserves the unique initial template pair as a stable fallback throughout tracking. Subsequently, the Guided Template Selection Transformer (GTST) takes the templates stored in the DTMM together with the current search region pair as input, and selects the most suitable dynamic template pair from the Historical and Candidate Template Memories for object tracking.

2.2. Guided Template Selection Transformer

The Guided Template Selection Transformer is built upon a Vision Transformer architecture and is formulated as a template selection module, as illustrated in Figure 2. Its goal is to select the template that yields the best tracking performance for the current search region, thereby improving the robustness of multi-modal UAV tracking. By learning the matching relationship between candidate templates and the current search region, the module establishes a mapping from input features to template quality, enabling adaptive template selection.

The training process of the GTST is shown in Figure 3. Given a search region pair $S=\{I_x^r,I_x^t\}$ containing both visible and thermal modalities, and a candidate template set $T=\{T_1,T_2,\dots ,T_K\}$, where $T_i=\{I_z^r,I_z^t\}$ denotes the i-th multi-modal template pair, each template–search pair independently calculates its tracking performance, each template–search pair $(T_i,S)$ is fed into the tracker to obtain the corresponding Intersection over Union (IoU) score $O_i=IoU(T_i,S)$. This score directly reflects the tracking quality of template $T_i$ on the current search region. The IoU scores of all candidate templates are organized into a vector $O={[O_1,O_2,\dots ,O_K]}^{\top }$, and the optimal template index is determined as:

$$i^{*}=\underset{i\in \{1,\dots ,K\}}{arg\; max}{O}_i.$$

(1)

The optimal index is then converted into a one-hot label vector $y\in {\{0,1\}}^K$, in which the entry corresponding to the optimal template is set to 1 and all others are set to 0. Notably, the number of candidate templates remains fixed throughout training and inference, since the dimension of the classification head is predefined and cannot adapt to variable template pool sizes.

The GTST adopts a standard Vision Transformer architecture. In practical implementation, we traverse all candidates in a loop and process each template–search pair separately. For each candidate template $T_i$ and search region S, their multi-modal data are concatenated to form an input sample $X_i=\mathrm{Concat}(T_i,S)$, Each sample $X_i$ is independently sent into the network for self-attention and feed-forward feature extraction following the standard image classification design, and all candidate samples $\{X_1,X_2,\dots ,X_K\}$ are processed in a batch. The model maps each input sample $X_i$ to a classification logit vector $z_i=f_{\theta }\left(X_i\right)$, where $f_{\theta }$ denotes the Transformer classifier parameterized by $\theta$. All K independent scalar logits are assembled into a unified logit vector, and the assembled logits are then passed through a Softmax function to obtain the classification probabilities:

$${\widehat{\mathbf{y}}}_i=\mathrm{Softmax}\left({\mathbf{z}}_i\right)={\left[{\displaystyle \frac{e^{z_{i,1}}}{{\sum }_{k=1}^K{e}^{z_{i,k}}}},\dots ,{\displaystyle \frac{e^{z_{i,K}}}{{\sum }_{k=1}^K{e}^{z_{i,k}}}}\right]}^{\top }.$$

(2)

This probability distribution reflects the model’s confidence that a candidate template $T_i$ is the optimal template for the current search region. During inference, the template with the highest predicted confidence $T_{\widehat{i}}$ is selected and fed into the tracker for subsequent target localization, enabling adaptive template selection during tracking.

2.3. Dynamic Threshold Module

The Dynamic Threshold Module is a key component of GTUTrack, designed to generate adaptive thresholds for template update under different tracking scenarios. This module addresses the limitations of fixed thresholds, which often fail to generalize across complex tracking conditions and may either suppress effective updates or introduce excessive noisy templates.

The prediction head of the tracker produces a confidence response map $F_{score\_map}\in {\mathbb{R}}^{H\times W}$ for the input search region, where H and W denote the spatial dimensions of the feature map. Each element in this map represents the confidence that the corresponding location belongs to the target, and the maximum response is taken as the core tracking confidence score of the current frame. This score reflects the reliability of matching the current template to the target. Since different tracking sequences may exhibit substantially different scene characteristics, including target appearance complexity, background interference, and illumination variation, a fixed threshold is often inappropriate. An overly high threshold may prevent valid template updates in low-confidence but reliable sequences, whereas an overly low threshold may introduce noisy templates and cause tracking drift.

To address this issue, we adopt a robust statistical strategy based on the truncated mean to estimate a personalized template update threshold for each tracking sequence. By removing extreme values, this strategy reduces the influence of outlier frames and improves both robustness and scene adaptability.

For a given sequence, the dynamic threshold is computed from the confidence scores of the first n frames. Specifically, the confidence scores of the first n frames are collected into a set $S=\{s_1,s_2,\dots ,s_n\}$, where $s_t$ denotes the highest confidence score of the t-th frame. We then remove the largest m and smallest m elements from S and retain the remaining $n-2m$ valid scores in the set $S_{\mathrm{valid}}$. The dynamic threshold $\theta$ for template update is computed as the mean of the valid scores:

$$\theta ={\displaystyle \frac{1}{|{\mathcal{S}}_{\mathrm{valid}}|}}\sum _{s\in {\mathcal{S}}_{\mathrm{valid}}}s.$$

(3)

In our implementation, we set n = 20 and m = 2. We further set a fixed initial baseline threshold of 0.7. If the statistically calculated dynamic threshold $\theta$ is higher than 0.7, we adopt $\theta$ as the final update threshold; otherwise, we keep the initial threshold 0.7 as the standard. Importantly, the threshold is only calculated once using the early-stage frame information, and it will not be further updated during the subsequent whole tracking process. The resulting final threshold $\theta$ is used as the criterion for template update in the current sequence. When the core tracking confidence score $F_s$ of a subsequent frame exceeds $\theta$, the corresponding search region is cropped into a candidate template and stored in memory. Otherwise, template update is suspended to avoid introducing unreliable templates.

This design is motivated by two considerations. First, the truncated mean is a classical robust statistical estimator with strong resistance to outliers, such as abnormally high scores caused by occasional mismatches or low scores caused by temporary occlusion, thus providing a more reliable estimate of the baseline confidence level of the sequence. Second, the first n frames usually capture the initial scene characteristics of the sequence, and the threshold estimated from this stage can better reflect the intrinsic difficulty of the sequence. For example, low-texture targets tend to yield lower baseline confidence, whereas high-contrast targets generally produce higher-confidence responses. In this way, the proposed DTM effectively overcomes the poor generalization of fixed thresholds.

Notably, the DTM only requires lightweight statistical computation on the first n frames and introduces neither additional trainable parameters nor extra optimization objectives. It therefore achieves a favorable balance between computational efficiency and scene adaptability. For sequences with simple backgrounds and stable target motion, the threshold is automatically increased to avoid redundant updates. In contrast, for challenging sequences with cluttered backgrounds and overall lower confidence, the threshold is automatically reduced to preserve valid template updates. This fully automatic and personalized threshold generation strategy provides a reliable foundation for adaptive template updates.

2.4. Dynamic Template Memory Module

The Dynamic Template Memory Module is the core component responsible for storing and managing the lifecycle of templates in GTUTrack. It is implemented using a double-ended queue (Deque), which supports efficient insertion and deletion operations with $O\left(1\right)$ time complexity. This design enables efficient template storage, update, and retrieval while satisfying the real-time requirements of UAV tracking. To control memory usage and maintain retrieval efficiency, the module adopts a fixed-capacity queue design. The DTMM consists of three memory components, each serving a distinct role in template storage and update.

(1) Fixed Template Memory

The Fixed Template Memory provides the fundamental reference throughout the entire tracking process. Its role is to store the initial template pair $T_{\mathrm{init}}=\{T_{\mathrm{init}}^{rgb},T_{\mathrm{init}}^{tir}\}$, which contains both visible and thermal modalities. This memory has a fixed capacity of one and remains unchanged during tracking, i.e., no update, replacement, or eviction is performed.

The main advantage of this design is that it provides a stable tracking anchor. Since the initial template preserves the original target appearance, it can serve as a reliable fallback when dynamic templates become unreliable due to occlusion, abrupt appearance variation, or severe tracking drift. In such cases, the fixed template helps prevent catastrophic tracking failure by maintaining a consistent reference for target localization.

(2) Candidate Template Memory

The Candidate Template Memory serves as a dynamic pool for accumulating valid candidate templates during tracking. After each frame, if the core tracking confidence score $F_s$ exceeds the dynamic threshold $\theta$, the current search region is regarded as reliable and is assumed to contain valid target information. The search region is then cropped according to the predicted target center and resized into template format, forming a candidate template $T_{ct}=\{T_{ct}^{rgb},T_{ct}^{tir}\}$, which is appended to the tail of the candidate memory.

To maintain both efficiency and relevance, the Candidate Template Memory adopts a first-in-first-out (FIFO) strategy with a fixed capacity. When the memory is full, the oldest template is removed from the head of the queue. As a result, the memory always maintains a set of recent candidate templates $T_{ct}=\{T_{ct1},T_{ct2},\dots ,T_{ctN}\}$, where N denotes the memory capacity. This design allows the module to continuously capture target appearance variations, including pose, scale, and viewpoint changes, thereby providing up-to-date candidates for adaptive template selection and alleviating the adverse effect of outdated templates.

(3) Historical Template Memory

The Historical Template Memory acts as a repository of high-quality templates. Templates stored in this memory are those selected as optimal by the Guided Template Selection Transformer and therefore exhibit strong matching reliability and robustness.

During tracking, if the GTST selects an optimal template $T_{\mathrm{best},t}$ from the candidate memory, this template is further inserted into the historical memory. Similar to the candidate memory, the historical memory is implemented as a fixed-capacity FIFO queue, maintaining a template set $T_{ht}=\{T_{ht1},T_{ht2},\dots ,T_{htM}\}$, where M denotes the memory capacity.

The main advantage of the Historical Template Memory lies in its ability to provide reliable fallback templates. Since all stored templates have been validated through previous matching performance, they can robustly represent the target appearance across different stages of tracking. Even when the candidate memory is contaminated by low-quality templates caused by sudden scene changes, the historical memory can still provide reliable alternatives, thereby reducing the risk of failure due to erroneous template selection.

Moreover, the interaction between the Candidate Template Memory and the Historical Template Memory forms a positive feedback mechanism. The candidate memory captures recent appearance variations of the target, while high-quality templates selected through matching are preserved in the historical memory. This design ensures both adaptability and stability during tracking.

During inference, all components of DTMM are initialized with the initial template. Specifically, the capacity of the fixed template memory is set to 1, the candidate template memory is 5, and the historical template memory is 2, reaching a total capacity of 8, which is kept consistent with the dimension setting of GTST. In the first 20 frames, whether dynamic templates are inserted into the candidate template memory is judged according to the initial threshold defined in DTM. Additionally, duplicate templates will not be repeatedly stored in the historical template memory.

Overall, the collaboration of the three memory components enables the DTMM to balance adaptability and reliability. The fixed memory provides a stable reference, the candidate memory captures real-time appearance variations, and the historical memory preserves reliable high-quality templates. Together, they form a structured and hierarchical template memory system that supports robust and accurate template selection.

2.5. Loss Function

Given the backbone-extracted search region tokens from both modalities, we first perform element-wise fusion between the RGB and thermal tokens, and then reshape the fused tokens into a 2D spatial feature map. This fused representation effectively integrates complementary cross-modal information for subsequent prediction.

The resulting feature map is then fed into a series of $Conv$-$BN$-$ReLU$ layers to produce three prediction branches, including the target classification score map, local offset prediction, and normalized bounding box size prediction.

Following OSTrack [20], the tracking head is optimized using both classification and regression losses. The overall tracking loss is formulated as:

$${\mathcal{L}}_{tracking}={\mathcal{L}}_{cls}+{\lambda }_{giou}{\mathcal{L}}_{giou}+{\lambda }_{L_1}{\mathcal{L}}_1,$$

(4)

where ${\mathcal{L}}_{cls}$ denotes the weighted Focal Loss for classification, ${\mathcal{L}}_{giou}$ denotes the Generalized IoU loss, and ${\mathcal{L}}_1$ denotes the $L_1$ loss. The balancing coefficients are set to ${\lambda }_{giou}=2$ and ${\lambda }_{L_1}=5$.

For the Guided Template Selection Transformer, which is formulated as a template classification task, we adopt the Binary Cross-Entropy (BCE) loss for optimization.

Specifically, the input to the selector is defined as the concatenated representation of a candidate template and the current search region, denoted as ${\mathbf{X}}_i=\mathrm{Concat}(T_i,\mathcal{S})$, where $T_i$ is the i-th candidate template and $\mathcal{S}$ is the current search region. The model outputs a probability ${\widehat{p}}_i\in [0,1]$ indicating the likelihood that $T_i$ is the optimal template. The ground-truth label $p_i$ is defined using one-hot encoding, where the optimal template is assigned 1 and all others are assigned 0.

The selector loss is formulated as:

$$L_{\mathrm{selector}}=-{\displaystyle \frac{1}{K}}{\displaystyle \sum _{i=1}^K}\left[(K-1)·p_{i}log\left({\widehat{p}}_i\right)+1·(1-p_i)log(1-{\widehat{p}}_i)\right],$$

(5)

where K is the number of candidate templates, $p_i$ is the ground-truth label, and ${\widehat{p}}_i$ is the predicted probability. Specifically, the positive sample is assigned a weight of $K-1$, while each negative sample is assigned a weight of 1. In our experiments, we set K = 8.

The BCE loss encourages the selector to assign higher confidence to the optimal template while suppressing non-optimal templates. In particular, it penalizes low predicted probabilities for positive samples and high predicted probabilities for negative samples, thereby enabling the model to learn discriminative matching features between candidate templates and the current search region.

To jointly optimize target localization and adaptive template selection, we combine the tracking loss and the selector loss into a unified multi-task objective. The overall loss function of GTUTrack is defined as:

$${\mathcal{L}}_{total}={\mathcal{L}}_{tracking}+{\mathcal{L}}_{selector},$$

(6)

By jointly optimizing these two objectives, the proposed framework simultaneously enhances tracking accuracy and template selection reliability. The tracking loss ensures precise target localization and bounding box regression, while the selector loss improves the ability of the Guided Template Selection Transformer to identify the most suitable template for the current search region. As a result, the tracking network and the template selector can be trained in an end-to-end manner, allowing the tracking network and the template selector to mutually reinforce each other during optimization.

2.6. Differences from Existing Methods

Unlike existing RGBT tracking methods [8,9,21], which mainly regard multimodal feature fusion as their core contribution and focus on cross-modal alignment and interaction, we pay attention to the adaptive optimization of template resources. Motivated by advanced template updating schemes [22], we move beyond the fusion-dominated paradigm and explore a more effective template updating strategy. In contrast to most passive template updating methods [23,24], which rely on fixed rules and refresh templates mechanically according to preset thresholds, we propose an active template updating strategy. Specifically, the search region dynamically perceives target-state changes and autonomously selects the most suitable dynamic templates during online tracking. In this way, we can effectively filter out invalid interference templates and enhance the discriminability of target templates in complex scenarios.

3. Experiments

In this section, we first introduce the experimental settings and implementation details of the proposed GTUTrack. We then compare GTUTrack with a broad range of state-of-the-art multi-modal tracking methods on multiple benchmarks. Finally, extensive ablation studies and qualitative analyses are conducted to validate the effectiveness of each component and to further investigate the behavior of the proposed search region-guided adaptive template update framework.

3.1. Experimental Settings

3.1.1. Evaluation Datasets

To comprehensively evaluate the effectiveness and generalization capability of GTUTrack, we conduct experiments on four public multi-modal tracking benchmarks, including one UAV-oriented benchmark and three surveillance-oriented benchmarks.

VTUAV [4] is collected in unmanned aerial vehicle (UAV) scenarios. It focuses on UAV-specific tracking challenges, such as small target size, fast motion, viewpoint variation, scale change, and cluttered backgrounds. The dataset contains 500 aligned RGBT (RGB–Thermal) video pairs with approximately 1.7 million frames in total. It is divided into a training set and a test set, each containing 250 sequences. Moreover, VTUAV is further divided into long-term and short-term tracking subsets. Since our work focuses on short-term tracking, all training and evaluation are conducted on the short-term subset. VTUAV serves as the primary benchmark in our experiments because it directly corresponds to the target task of multi-modal UAV tracking.

RGBT210 [17] is a widely used RGBT tracking benchmark consisting of 210 video pairs with approximately 104,700 frames. It is divided into 12 subsets according to different attributes, enabling detailed analysis of tracker performance under diverse conditions. However, the alignment between RGB and thermal image pairs in this dataset is not sufficiently accurate, making it more challenging for methods that rely heavily on precise cross-modal correspondence.

RGBT234 [18] contains 234 RGBT video pairs with approximately 116,700 frames. Compared with RGBT210, this dataset provides more accurate alignment between RGB and thermal modalities as well as more precise target annotations. It is therefore suitable for evaluating the robustness of multi-modal tracking methods under relatively well-aligned surveillance scenarios.

LasHeR [19] is currently the largest RGBT tracking dataset for surveillance scenarios. It contains 1224 aligned RGBT video sequences with approximately 734,800 frames, including 979 training sequences and 245 test sequences. The dataset covers a wide range of challenging attributes and can be divided into 19 subsets, including occlusion, illumination variation, low resolution, deformation, and background clutter. Due to its large scale and rich challenge diversity, LasHeR provides a comprehensive benchmark for assessing the robustness and generalization ability of tracking methods.

Overall, these four datasets provide complementary evaluation perspectives. VTUAV is used to verify the effectiveness of the proposed method in UAV-specific scenarios, while RGBT210, RGBT234, and LasHeR are adopted to assess the generalization capability of GTUTrack in representative surveillance scenarios.

3.1.2. Evaluation Metrics

Following previous methods [8], we adopt precision rate (PR), success rate (SR), and normalized precision rate (NPR) as the quantitative evaluation metrics under the one-pass evaluation (OPE) protocol.

PR measures the percentage of frames in which the Euclidean distance between the center of the predicted bounding box and that of the ground-truth bounding box is smaller than a threshold $\tau$. It mainly evaluates the localization accuracy of the tracker. A higher PR indicates that the tracker can more accurately estimate the target center across frames.

SR measures the percentage of frames in which the intersection over union (IoU) between the predicted bounding box and the ground-truth bounding box is larger than a threshold $\delta$. It mainly evaluates the ability of the tracker to estimate target scale and spatial extent. By varying $\delta$, we obtain the success curve, and the representative SR score is computed as the area under the curve.

NPR is a normalized version of PR that alleviates the influence of image resolution and target size. By varying the normalization threshold, a normalized precision curve is obtained, and the area under the curve within $[0, 0.5]$ is used as the representative NPR score. This metric is especially useful when the target size varies substantially across different sequences and datasets.

3.2. Implementation Details

GTUTrack is implemented based on the OSTrack framework, and the overall tracking architecture follows the standard template-search matching pipeline. During tracking, the initial template pair is fixed as the anchor reference, while dynamic templates are updated online through the proposed search region-guided adaptive template update strategy.

For the Guided Template Selection Transformer (GTST), we use a batch size of 32 for training and adopt the AdamW optimizer with an initial learning rate of $1\times {10}^{-4}$. A cosine annealing schedule is employed to ensure stable convergence. The selector is trained to estimate the relative quality of candidate templates with respect to the current search region, so that it can provide reliable template selection during inference.

To alleviate the severe class imbalance in template selection, where only one template is optimal and the remaining $K-1$ templates are non-optimal, we assign adaptive weights to different samples during training. Specifically, the positive sample (i.e., the optimal template) is assigned a weight of $K-1$, while each negative sample is assigned a weight of 1. This strategy balances their contributions during optimization and improves the discriminative ability of the selector.

For the Dynamic Threshold Module (DTM), the update threshold of each sequence is estimated from the confidence statistics of the first several frames using the truncated mean strategy. This design introduces no additional trainable parameters and incurs negligible computational overhead. For the Dynamic Template Memory Module (DTMM), the fixed, candidate, and historical template memories are maintained online during tracking to support adaptive template selection and update. All experiments are conducted under the same evaluation protocol as the compared methods to ensure fair comparison.

3.3. Quantitative Comparison

We first evaluate GTUTrack on VTUAV, a standard benchmark for multi-modal UAV tracking, to verify its effectiveness in UAV-specific scenarios. We then further evaluate the proposed method on three additional multi-modal tracking benchmarks collected in typical surveillance scenarios, including RGBT210, RGBT234, and LasHeR, in order to validate its generalization capability. The overall comparison results are summarized in

Table 1. Comparison of GTUTrack with advanced trackers on four public datasets. The highest and second-best results are highlighted in bold and underline respectively.

Alorithms	Publication	RGBT210		RGBT234		LasHeR			VTUAV
		PR	SR	PR	SR	PR	NPR	SR	PR	SR
CAT [25]	ECCV 20	79.2	53.3	80.4	56.1	45.0	39.5	31.4	-	-
CMPP [26]	CVPR 20	-	-	82.3	57.5	-	-	-	-	-
ADRNet [27]	IJCV 21	-	-	80.7	57.0	-	-	-	62.2	46.6
MANet++ [28]	TIP 21	-	-	80.0	55.4	46.7	40.4	31.4	-	-
APFNet [29]	AAAI 2022	-	-	82.7	57.9	50.0	-	36.2	-	-
TFNet [30]	TCSVT 22	77.7	52.9	80.6	56.0	-	-	-	-	-
DMCNet [31]	TNNLS 22	79.7	55.5	83.9	59.3	49.0	43.1	35.5	-	-
MFGNet [32]	TMM 22	74.9	49.4	78.3	53.5	-	-	-	-	-
MIRNet [33]	ICME 22	-	-	81.6	58.9	-	-	-	-	-
DRGCNet [34]	SENS J 23	-	-	82.5	58.1	48.3	42.3	33.8	-	-
CAT++ [35]	TIP 24	82.2	56.1	84.0	59.2	50.9	44.4	35.6	-	-
DSiamMFT [36]	SPIC 20	64.2	43.6	-	-	-	-	-	-	-
SiamCDA [37]	TCSVT 21	-	-	76.0	56.9	-	-	-	-	-
HMFT [4]	CVPR 22	78.6	53.5	78.8	56.8	-	-	-	75.8	62.7
SiamMLAA [38]	TMM 23	77.9	56.7	79.5	58.4	53.8	-	43.1	-	-
QAT [39]	ACM MM 23	86.8	61.9	88.4	64.3	64.2	59.6	50.1	80.1	66.7
MTNet [40]	ICME 23	-	-	85.0	61.9	60.8	-	47.4	-	-
ViPT [13]	CVPR 23	-	-	83.5	61.7	65.1	-	52.5	-	-
TBSI [8]	CVPR 23	85.3	62.5	87.1	63.7	69.2	65.7	55.6	-	-
MACFT [41]	Sensors 23	-	-	85.7	62.2	65.3	-	51.4	80.1	66.8
OneTracker [42]	CVPR 24	-	-	85.7	64.2	67.2	-	53.8	-	-
BAT [14]	AAAI 2024	-	-	86.8	64.1	70.2	-	56.3	-	-
TATrack [9]	AAAI 24	85.3	61.8	87.2	64.4	70.2	66.7	56.1	-	-
Un-Track [43]	CVPR 24	-	-	84.2	62.5	66.7	-	53.6	-	-
SDSTrack [44]	CVPR 24	-	-	84.8	62.5	66.5	-	53.1	-	-
USTrack [45]	IJCAI 24	-	-	87.4	65.8	-	-	-	86.9	74.4
SHT [46]	TIM 25	86.4	63.7	88.1	65.6	70.1	66.1	56.1	-	-
TUFNet [47]	KBS 25	86.7	61.9	88.2	64.1	70.8	-	55.7	-	-
CAFormer	AAAI 25	85.6	63.2	88.3	66.4	70.0	66.1	55.6	88.6	76.2
CGATrack [48]	IFFUS 25	87.8	64.3	89.0	66.6	72.2	68.3	57.5	89.0	76.6
Ours	-	90.9	66.3	92.0	68.8	75.0	71.1	59.3	91.4	78.4

, where the best, second-best, and third-best results are highlighted in bold, underline, and italic, respectively.

A clear observation from Table 1 is that GTUTrack consistently achieves the best performance across all four datasets. Compared with recent strong baselines, including CAFormer, CGATrack, and TATrack, the proposed method yields consistent gains in both localization and overlap-based metrics. These results indicate that the proposed search region-guided adaptive template update mechanism improves not only target localization accuracy, but also the stability of template matching under complex appearance variation.

(1) Evaluation on VTUAV.

We first evaluate GTUTrack on VTUAV, which is specifically designed for multi-modal UAV tracking and contains challenges such as small target size, fast motion, significant viewpoint variation, and cluttered backgrounds. As shown in Table 1, GTUTrack achieves 91.4% in PR and 78.4% in SR, significantly outperforming all competing methods. Compared with CGATrack, GTUTrack improves PR and SR by 2.4% and 1.8%, respectively. Compared with CAFormer, the gains further increase to 2.8% in PR and 2.2% in SR. These improvements are particularly meaningful on VTUAV, where the target appearance often changes abruptly due to UAV motion and viewpoint variation.

The strong performance on VTUAV demonstrates that GTUTrack is well suited to multi-modal UAV tracking. On the one hand, GTST can adaptively select the template that best matches the current search region, which is crucial when the target undergoes a rapid appearance change. On the other hand, DTM prevents unreliable search regions from being updated into the template pool, while DTMM preserves both stable and high-quality historical target representations. As a result, the tracker can better maintain robust target representation throughout the UAV tracking process.

(2) Generalization on Surveillance Benchmarks.

To further validate the generalization capability of GTUTrack beyond UAV scenarios, we additionally evaluate it on three widely used RGBT tracking benchmarks from typical surveillance scenarios, namely RGBT210, RGBT234, and LasHeR.

On RGBT210, GTUTrack achieves the best performance on both PR and SR, reaching 90.9% and 66.3%, respectively. Compared with the second-best method CGATrack, GTUTrack improves PR and SR by 3.1% and 2.0%, respectively. It also surpasses CAFormer by 5.3% and 3.1% in PR and SR, respectively, and outperforms QAT by 4.1% in PR and 4.4% in SR. Since RGBT210 contains imperfectly aligned RGB and thermal pairs, these gains suggest that the proposed adaptive template selection mechanism can effectively alleviate the impact of cross-modal inconsistency and noisy template updates.

On RGBT234, GTUTrack achieves 92.0% in PR and 68.8% in SR, consistently outperforming all competing methods. Compared with CGATrack, GTUTrack achieves gains of 3.0% in PR and 2.2% in SR. Compared with CAFormer and USTrack, GTUTrack also shows clear improvements. Since RGBT234 provides better cross-modal alignment and more accurate annotations, the results on this dataset further verify that the proposed framework can effectively exploit multi-modal complementary information while maintaining accurate and adaptive template update.

On LasHeR, a large-scale and highly challenging benchmark, GTUTrack achieves the best performance across all three metrics, reaching 75.0% in PR, 71.1% in NPR, and 59.3% in SR. Compared with CGATrack, GTUTrack achieves improvements of 2.8%, 2.8%, and 1.8% in PR, NPR, and SR, respectively. Compared with TATrack and BAT, GTUTrack also yields consistent gains. Because LasHeR contains diverse challenges such as occlusion, low resolution, deformation, and background clutter, these results indicate that the proposed adaptive template memory and selection mechanism can effectively handle substantial appearance variation over long tracking sequences.

Overall, the quantitative results on VTUAV verify the effectiveness of GTUTrack for multi-modal UAV tracking, while the consistent improvements on RGBT210, RGBT234, and LasHeR demonstrate its strong generalization capability across typical surveillance scenarios. More importantly, the superiority of GTUTrack across both UAV and surveillance benchmarks suggests that search region-guided adaptive template update is a generally effective solution for improving multi-modal tracking robustness under dynamic scene variation. Meanwhile, as shown in the

Table 2. Comprehensive comparison between GTUTrack with advanced trackers. The highest and second-best results are highlighted in bold and underline respectively.

Algorithms	Publication	LasHeR			VTUAV		Params	FLOPs	FPS
		PR	NPR	SR	PR	SR	(M)	(G)
ViPT [13]	CVPR 23	65.1	-	52.5	-	-	92.96	21.80	70.7
TBSI [8]	CVPR 23	69.2	65.7	55.6	-	-	201.98	82.52	36.2
MACFT [41]	Sensors 23	65.3	-	51.4	80.1	66.8	-	-	33.3
OneTracker [42]	CVPR 24	67.2	-	53.8	-	-	94.92	-	-
BAT [14]	AAAI 2024	70.2	-	56.3	-	-	92.44	56.68	67.2
TATrack [9]	AAAI 24	70.2	66.7	56.1	-	-	-	-	26.1
Un-Track [43]	CVPR 24	66.7	-	53.6	-	-	98.72	-	-
SDSTrack [44]	CVPR 24	66.5	-	53.1	-	-	102.18	108.40	21
USTrack [45]	IJCAI 24	-	-	-	86.9	74.4	-	-	84.2
SHT [46]	TIM 25	70.1	66.1	56.1	-	-	-	-	28
TUFNet [47]	KBS 25	70.8	-	55.7	-	-	97.35	-	27
CAFormer	AAAI 25	70.0	66.1	55.6	88.6	76.2	97.50	42.91	83.6
CGATrack [48]	IFFUS 25	72.2	68.3	57.5	89.0	76.6	95.96	58.71	88.8
Ours	-	75.0	71.1	59.3	91.4	78.4	182.52	76.36	62

, GTUTrack maintains favorable efficiency with 182.52M parameters, 76.36G FLOPs and 62 FPS, offering a good balance between accuracy and speed for real-time UAV tracking.

3.4. Ablation Study

(1) Component Analysis.

We conduct incremental ablation experiments to evaluate the contribution of each component in GTUTrack. As shown in

Table 3. Ablation study on components of the GTUTrack. The highest results are highlighted in bold.

#	Methods	LasHeR			RGBT234		VTUAV		FPS
		PR	NPR	SR	PR	SR	PR	SR
1	OSTrack+RGBT	67.8	64.3	54.0	86.4	64.5	85.3	73.2	96.4
2	+ GTST	72.5	68.5	57.5	88.8	66.2	88.9	76.5	66.3
3	+ DTM	73.3	69.1	58.2	89.8	67.0	89.8	77.6	64.1
4	+ HTM	74.4	70.8	59.0	91.5	68.4	90.7	78.1	62.6
5	+ CTM	75.0	71.1	59.3	92.0	68.8	91.4	78.4	62

, the baseline tracker (OSTrack+RGBT) achieves 67.8%/64.3%/54.0% in PR/NPR/SR on LasHeR and 86.4%/64.5% in PR/SR on RGBT234. After introducing GTST, the performance increases to 72.5%/68.5%/57.5% on LasHeR and 88.8%/66.2% on RGBT234. This corresponds to gains of 4.7% in PR and 3.5% in NPR on LasHeR, as well as 2.4% in PR on RGBT234, demonstrating that search region-guided adaptive template selection plays a dominant role in improving tracking performance.

After further incorporating the Dynamic Threshold Module (DTM), the performance improves to 73.3%/69.1%/58.2% on LasHeR and 89.8%/67.0% on RGBT234. These gains indicate that adaptive thresholding can effectively filter unreliable candidate templates and reduce the risk of noisy template updates. Compared with fixed update strategies, DTM provides sequence-specific update criteria, which are better suited to varying scene conditions.

When the Historical Template Memory (HTM) is added, the performance further increases to 74.4%/70.8%/59.0% on LasHeR and 91.5%/68.4% on RGBT234. This result shows that preserving high-quality historical templates is beneficial for maintaining stable target representation, especially when recent candidate templates are affected by temporary noise or local appearance degradation.

Finally, by introducing the Candidate Template Memory (CTM), the complete GTUTrack achieves the best performance across all metrics, i.e., 75.0%/71.1%/59.3% on LasHeR and 92.0%/68.8% on RGBT234. This final improvement demonstrates that recent candidate templates and reliable historical templates are complementary. Together, they provide both adaptability to current appearance variation and stability against noisy updates. Overall, the ablation results confirm that GTST, DTM, and DTMM each make a positive contribution, and their combination yields the strongest performance.

(2) Threshold Analysis.

We further analyze the impact of different template update thresholds. As shown in

Table 4. Ablation study on template update threshold. The highest results are highlighted in bold.

#	Thresholds	LasHeR			RGBT234		VTUAV
		PR	NPR	SR	PR	SR	PR	SR
1	0.70	72.8	68.3	58.0	89.4	66.8	89.1	76.4
2	0.75	73.3	69.0	58.3	89.8	66.9	89.5	76.6
3	0.80	72.6	68.2	57.8	89.1	66.7	89.0	76.4
4	0.85	72.0	67.9	57.6	88.9	66.5	88.9	76.3
5	$DT{M}_{i=null}$	70.7	67.0	55.7	87.0	65.1	87.6	75.3
6	$DT{M}_{i=0.65}$	73.6	69.7	58.3	91.5	68.5	89.6	76.8
7	$DT{M}_{i=0.70}$	75.0	71.1	59.3	92.0	68.8	91.4	78.4
8	$DT{M}_{i=0.75}$	74.6	70.5	58.9	91.5	68.4	91.0	77.8

, fixed thresholds lead to noticeable performance fluctuations. On LasHeR, the best fixed-threshold result is obtained at 0.75, with 73.3% PR, 69.0% NPR, and 58.3% SR. However, when the threshold is increased to 0.80 and 0.85, the performance drops consistently. A similar trend can be observed on RGBT234, where the best fixed-threshold setting still underperforms the proposed adaptive strategy.

In contrast, the proposed DTM consistently outperforms all fixed-threshold settings. Compared with the best fixed threshold (0.75), DTM improves PR/NPR/SR by 1.7%/2.1%/1.0% on LasHeR and improves PR/SR by 2.2%/1.9% on RGBT234. These results demonstrate that adaptive thresholding can better balance template quality and update frequency. Instead of relying on a globally fixed rule, DTM adjusts the update criterion according to the confidence statistics of each sequence, thereby producing more reliable candidate templates and improving overall tracking robustness.

We further perform ablation analysis on the initial threshold of the DTM module. As shown in Experiment 5, tracking performance declines significantly without adopting an initial threshold, where the template update threshold is entirely determined by the confidence scores of the first 20 frames within each tracking sequence. This phenomenon mainly arises because the calculated threshold tends to be unreliable when the early frames suffer from complex tracking difficulties. By comparing Experiments 6, 7 and 8, we verify that setting the initial threshold to 0.75 yields the optimal configuration for template updating.

The threshold analysis also provides further evidence for the necessity of adaptive template update. A fixed threshold that works reasonably well on one dataset or sequence may not generalize to others with different confidence distributions. By contrast, the proposed DTM offers a lightweight yet effective way to improve update reliability without introducing additional learnable parameters.

(3) Memory Capacity Analysis.

We conduct ablation experiments to investigate how different template memory configurations affect tracking performance on three benchmarks, namely LasHeR, RGBT234, and VTUAV. The triplet in the “Methods” column denotes the capacities of the historical template memory, candidate template memory, and fixed template memory, respectively. As shown in

Table 5. Ablation study on dynamic template memory capacity. The highest results are highlighted in bold.

#	Methods	LasHeR			RGBT234		VTUAV
		PR	NPR	SR	PR	SR	PR	SR
1	[0, 7, 1]	71.6	67.8	56.7	89.9	66.7	88.6	76.2
2	[1, 6, 1]	72.4	68.6	57.1	90.1	66.9	88.9	77.3
3	[2, 5, 1]	75.0	71.1	59.3	92.0	68.8	91.4	78.4
4	[3, 4, 1]	74.6	70.7	59.0	91.6	68.5	90.6	77.8

, the configuration $[2, 5, 1]$ achieves the best overall performance, obtaining the highest scores across all datasets. Reducing the historical memory to 0 or 1, as in $[0, 7, 1]$ and $[1, 6, 1]$, leads to a noticeable performance drop, indicating that a small but effective historical template pool is important for modeling long-term target appearance variations. In contrast, increasing the historical memory to 3 while reducing the candidate memory to 4 ($[3, 4, 1]$) slightly degrades performance, as a smaller candidate pool limits the model’s ability to adapt to recent appearance changes. These results suggest that balancing historical and candidate memories is crucial. Therefore, we adopt the setting of 2 historical templates, 5 candidate templates, and 1 fixed template in our full model.

(4) Analysis of Threshold Initialization Parameters.

We conduct ablation experiments in

Table 6. Ablation study on dynamic threshold initialization parameters. The highest results are highlighted in bold.

#	Methods	LasHeR			RGBT234		VTUAV
		PR	NPR	SR	PR	SR	PR	SR
1	[10, 0]	72.6	68.9	57.2	91.4	68.3	89.5	76.5
2	[10, 1]	73.2	69.4	58.1	90.7	67.7	89.9	77.1
3	[10, 2]	73.1	69.3	58.1	90.7	67.8	89.6	77.0
4	[20, 0]	74.9	71.0	59.4	91.6	68.6	91.5	78.4
5	[20, 2]	75.0	71.1	59.3	92.0	68.8	91.4	78.4
6	[20, 4]	74.1	70.3	58.8	92.2	69.2	91.6	78.3

to explore the influence of hyperparameters n and m in the dynamic threshold module. Here, the two values in the “Methods” column follow the format $[n,m]$, where n denotes the number of initial frames adopted to calculate the adaptive threshold, and m represents the number of maximum and minimum confidence scores eliminated to eliminate outliers.

Experimental results show that inappropriate combinations of n and m will degrade tracking accuracy. When n is too small, the early frame information is insufficient, making the estimated threshold unable to reflect the real scene difficulty. When n is excessively large, redundant frames introduce more interference and increase computation cost. Meanwhile, a too small m cannot effectively filter abnormal confidence values caused by occlusion and background clutter, while an overlarge m will discard valid feature information.

The optimal performance is achieved when n = 20 and m = 2. This setting can stably exclude outlier scores, accurately fit the baseline confidence level of the tracking sequence, and generate reliable adaptive update thresholds. It successfully balances scene adaptability and computational simplicity, so we adopt this parameter combination in all subsequent experiments.

(5) Perturbation Analysis.

We evaluate the robustness of our tracker against spatial misalignment between RGB and thermal modalities through perturbation experiments. As shown in

Table 7. Ablation study on Perturbation. The highest results are highlighted in bold.

#	Deviation		LasHeR			VTUAV
	Translational	Rotational	PR	NPR	SR	PR	SR
1	-	-	75.0	71.1	59.3	91.4	78.4
2	$\pm 10$ px	-	72.2	68.2	57.3	88.4	75.8
3	-	$\pm 5°$	70.9	67.3	56.2	86.8	73.6
4	$\pm 10$ px	$\pm 5°$	67.5	63.9	53.8	80.1	67.1

, ours method (row 1) corresponds to perfectly aligned data, yielding the highest performance on both LasHeR and VTUAV. When introducing translational offsets of $\pm 10$ pixels (row 2) or rotational deviations of $\pm 5°$ (row 3), the tracking metrics gradually degrade, and performance drops further under combined translation and rotation perturbations (row 4). This is expected, since our model is trained on well-aligned multi-modal data; any spatial misalignment breaks the learned cross-modal correspondence and thus weakens the tracking ability.

3.5. Attribute Analysis

We evaluate the attribute-based tracking performance of GTUTrack on the challenging RGBT234 dataset. The radar chart reports the overall performance across 12 typical attribute subsets, including background clutter (BC), scale variation (SC), partial occlusion (PO), thermal crossover (TC), no occlusion (NO), motion blur (MB), low resolution (LR), low illumination (LI), heavy occlusion (HO), fast motion (FM), deformation (DEF), and camera motion (CM). As shown in Figure 4, GTUTrack consistently achieves the best performance across all attribute scenarios compared with other state-of-the-art trackers, including BAT, CGATrack, SDSTrack, ViPT, and TBSI. The polygon corresponding to GTUTrack forms the outermost boundary in the radar chart, indicating superior robustness under diverse challenging conditions.

GTUTrack also exhibits strong adaptability to both RGB-degraded and TIR-degraded scenarios. When RGB information is severely impaired, such as in low illumination (LI), partial occlusion (PO), and fast motion (FM), GTUTrack can effectively exploit complementary thermal information to maintain stable tracking. Meanwhile, when thermal information becomes unreliable, such as under thermal crossover (TC) and background clutter (BC), GTUTrack remains robust by leveraging discriminative RGB features. Moreover, GTUTrack shows clear advantages in handling non-rigid objects (NO), camera motion (CM), and scale variation (SC), demonstrating its ability to cope with complex geometric variations and motion patterns. Overall, the attribute-based evaluation confirms that GTUTrack is robust to a wide range of adverse conditions, and its cross-modal adaptive mechanism enables consistent performance gains over existing trackers.

3.6. Qualitative Analysis

We visualize the tracking results on representative sequences from VTUAV and LasHeR datasets, covering several challenging scenarios, including occlusion (OCC), small targets (ST), low illumination (LI), and high reflection (HR). Figure 5 shows that GTUTrack consistently produces more accurate and stable tracking results than competing methods in these challenging situations.

In bus_14, the target undergoes extreme illumination variation together with noticeable shape change during tracking. Under such conditions, several competing methods gradually drift away from the target or fail to maintain stable localization. In contrast, GTUTrack consistently remains locked onto the target throughout the sequence. This result demonstrates that the proposed method can effectively adapt to severe appearance variation and maintain reliable target representation even when the visual characteristics of the target change significantly.

In rightdarksingleman, GTUTrack successfully adapts to illumination changes and achieves consistent target localization throughout the sequence. Compared with competing methods that gradually deviate from the target under changing brightness conditions, GTUTrack remains more stable due to its ability to update templates adaptively according to the current search region.

Overall, these qualitative results demonstrate that GTUTrack exhibits superior robustness compared with existing multi-modal tracking methods, particularly in dynamic and challenging scenarios involving severe appearance variation, environmental interference, and modality degradation. The observations are also consistent with the quantitative and ablation results, further validating the effectiveness of the proposed search region-guided adaptive template update framework.

4. Discussion

The superior performance of GTUTrack can be mainly attributed to the proposed search region-guided adaptive template update mechanism. Unlike conventional methods that rely on fixed update intervals and manually designed thresholds, GTUTrack actively leverages the current search region to guide template selection and update decisions. This allows the tracker to better adapt to irregular target motion and complex scene variation.

The quantitative comparison results show that GTUTrack consistently improves performance on both UAV and surveillance benchmarks. This suggests that the proposed framework is not limited to a specific scenario, but instead captures a more general principle for multi-modal tracking: the template update should be conditioned on the current search context rather than predefined rules. In UAV tracking, this is particularly important because targets often undergo abrupt appearance variation caused by scale change, viewpoint transition, and background motion. In surveillance scenarios, the same mechanism also improves robustness against occlusion, low resolution, and modality inconsistency.

The ablation and comparison results further show that the three proposed components play complementary roles. GTST improves the matching quality between templates and the current search region, DTM provides adaptive and reliable update criteria under different scenarios, and DTMM maintains a structured template pool that balances adaptability and stability. Their collaboration enables GTUTrack to effectively avoid noisy template updates while preserving useful historical target appearances.

Nevertheless, the proposed method still has room for improvement. In extremely challenging cases, such as severe cross-modal degradation, long-term full occlusion, or dramatic target deformation, the quality of both the current search region and candidate templates may deteriorate simultaneously, which may limit the benefit of adaptive template selection. In addition, although the proposed template memory mechanism is lightweight, further efficiency optimization may still be desirable for resource-constrained UAV platforms. Future work will investigate stronger cross-modal interaction mechanisms, more robust template quality estimation strategies, and more efficient online memory management to further improve performance in such scenarios.

In future work, we will explore the weakly supervised paradigm inspired by ITER [49] to alleviate the reliance on expensive pixel-level annotations. Moreover, we will incorporate the online spectral compensation strategy proposed in OSICN [50] to promote more effective representation and fusion of spectral–spatial information.

5. Conclusions

In this paper, we propose a search region-guided adaptive template update framework for robust multi-modal UAV tracking. The proposed method enables adaptive template selection conditioned on the current search region, allowing the tracker to dynamically match templates with the most relevant target appearance. Specifically, we introduce a Guided Template Selection Transformer (GTST) to perform template quality evaluation and adaptive selection in a learnable manner. We further design a Dynamic Threshold Module (DTM) to adaptively determine template update criteria under different tracking scenarios, ensuring reliable candidate template generation. To improve stability and robustness, we develop a Dynamic Template Memory Module (DTMM), which includes a Fixed Template Memory that preserves the initial target appearance as a stable anchor, a Candidate Template Memory that captures recent appearance variations, and a Historical Template Memory that retains high-quality templates to prevent performance degradation caused by noisy updates. Extensive experiments on multiple benchmarks demonstrate that the proposed method achieves superior performance under significant target appearance variations. These results verify the effectiveness and generalization capability of GTUTrack across diverse and challenging surveillance scenarios.