Content Feature Contrastive Disentangling for Video Corpus Moment Retrieval ^†

Abstract

Video Corpus Moment Retrieval (VCMR) retrieves relevant untrimmed videos from a corpus and localizes target moments using natural language queries. While existing models achieve impressive accuracy, they suffer from a critical vulnerability: a disproportionate reliance on intrinsic dataset biases, particularly spurious dependencies on temporal locations. This over-reliance severely hinders their generalization capabilities. To address this, we propose a debiased inference framework rooted in a Structural Causal Model (SCM), implemented via a novel approach termed Content Disentangling with Contrastive Learning (CDCL). CDCL applies causal intervention by explicitly decoupling confounded video features into task-relevant content representations and bias-inducing confounders. Furthermore, we integrate a contrastive disentangling mechanism to fundamentally enhance the representational purity of the isolated content features. Experiments on the constructed TVR-OOD benchmark demonstrate consistent improvements for two representative late-fusion baselines. At VCMR Recall@10 with IoU = 0.5, CDCL improves XML from 10.60% to 12.32% and ReLoCLNet from 13.57% to 14.23%. These results demonstrate the effectiveness of combining content–location disentanglement with query-guided contrastive learning under the evaluated temporal distribution shift.

1. Introduction

Nowadays, as online video content increases sharply, there is an emergent demand for retrieving target moments from a large corpus of videos. Therefore, it has become a prominent trend for researchers to focus on the Video Corpus Moment Retrieval (VCMR) task. As a foundation for VCMR, Video Moment Retrieval (VMR) is a well-studied topic that aims to localize a specific moment within a single, pre-given video according to a textual query [1]. However, its prerequisite of a known target video limits its ability to handle realistic, multi-video scenarios. To bridge this gap, VCMR takes one more step to retrieve certain video moments from a large corpus of untrimmed, unsegmented videos given a natural language query [2], as shown in Figure 1.

Figure 1. In the VCMR task, multiple moments are retrieved from a large corpus of videos as candidates. Only the correct moment from the right video is considered to be a positive result.

Among the most widely studied methods, the core objective is to project video and query features into a common embedding space for cross-modal feature matching. Based on when features from different modalities interact, existing methods are primarily categorized into early fusion and late fusion strategies. Early fusion approaches [3,4] enforce deep cross-modal interactions at the initial stages to capture fine-grained correlations, but this requires joint encoding for every query-candidate pair, creating a severe computational bottleneck. In contrast, late-fusion strategies [5,6,7,8] process videos and texts independently, allowing corpus-side features to be pre-computed and reused across queries. This property makes late fusion well suited to corpus-scale retrieval, and we therefore adopt it as the base framework of this study.

Although existing studies present promising experimental results, we argue that these results do not truly reflect the models’ ability to perform multimodal semantic understanding. Recent studies on the VMR task have found that many state-of-the-art models suffer from studying implicit distribution biases in commonly used datasets [9,10,11]. Since VMR serves as a crucial subtask in the VCMR task, we believe that VCMR models also inevitably inherit various dataset biases. While Yoon et al. [8] attempted to eliminate the specific dependency on possible keywords in queries, another widespread and highly impactful bias—temporal location bias—has not yet been addressed. This bias arises from the annotation process of large-scale datasets: ground-truth clips are statistically far more likely to be located near the beginning or the end of the raw videos. This deeply rooted data bias creates a strong but unwanted tendency, misleading models to generate results preferentially from these fixed areas based on absolute locations, rather than relying on a genuine comprehension of cross-modal semantics.

To systematically tackle this problem, three crucial challenges must be conquered: (1) Establishing a fair evaluation environment to compare models’ adaptability to the dataset bias without interference from other factors; (2) Understanding precisely how the dataset bias affects the reasoning process and devising methods to mitigate its influence; (3) Developing a robust model architecture to explicitly eliminate the spurious dependency on temporal location bias present in existing methods.

To provide a fair evaluation of the models’ adaptability towards dataset bias, we conduct Out-Of-Distribution (OOD) testing [12] for existing VCMR models. While collecting entirely new datasets is a straightforward option, the financial and temporal costs are exceedingly high. Moreover, the presence of multiple confounding factors makes it difficult to isolate a model’s true generalizability, let alone evaluate its adaptability to specific biases of interest. In contrast, OOD testing centers on a sophisticated methodology for re-splitting existing datasets. By isolating the data distribution bias, it offers an equitable and finely-tuned assessment of model robustness.

To rigorously analyze how the temporal bias affects the inference process, we construct a Structural Causal Model (SCM) [13] to elucidate the causal relationships between variables in the widely recognized late fusion models. SCM is a theoretical methodology that abstracts a model’s reasoning process into causal paths, allowing for adjustments through causal intervention. Specifically, we formally model the reasoning path and reveal an implicit causal relationship associated with the absolute location confounder L, a critical factor that has been largely ignored by existing studies.

To practically address the spurious dependency caused by this dataset bias, we propose a computational framework to conduct backdoor adjustment on the causal paths with the do-operation [14]. Designed according to the established SCM, this framework implements specific operations to achieve causal intervention. Specifically, we explicitly decouple the raw, confounded video features into latent location representations and purified video content features, which are expected to contain no location information. By approximating the expectation of the location confounder (${\mathbb{E}}_{\mathsf{\ell }}$), we infer with deconfounded representations instead of the originally entangled features.

Separating content features from location confounders is only the first step; accurate retrieval also depends on the discriminability of the resulting content representation. We therefore develop Content Disentangling with Contrastive Learning (CDCL), which applies query-guided contrastive supervision to the disentangled content features. The retrieval evaluations show that this refinement improves the reported VCMR and VMR metrics under the studied temporal distribution shift.

The main contributions of this paper can be summarized as follows:

• We introduce a dataset re-splitting method to establish an OOD evaluation environment, objectively exposing the vulnerabilities of existing VCMR models to temporal location bias.
• We construct an SCM to thoroughly analyze the causal mechanisms underlying the bias and provide a robust debiasing solution based on causal intervention and backdoor adjustment.
• We propose a novel CDCL method that not only explicitly decouples location confounders but also integrates contrastive disentangling to highly purify the essential content features.
• Extensive experiments demonstrate the effectiveness of CDCL in alleviating spurious dependencies on location bias, achieving advanced generalizability and accuracy among existing works.

2. Related Work

2.1. Video Corpus Moment Retrieval

As a crucial task bridging natural language processing and video understanding, the core challenge of Video Corpus Moment Retrieval (VCMR) lies in balancing retrieval precision with computational efficiency. This fundamental trade-off has catalyzed two primary technical paradigms: early fusion, characterized by deep multi-modal interactions, and late fusion, which prioritizes retrieval efficiency.

The early fusion paradigm aims to maximize retrieval accuracy by enforcing deep, fine-grained cross-modal interactions during the encoding phase. For instance, Zhang et al. [3] proposed a hierarchical multi-modal encoder that simultaneously models coarse-grained video clips and fine-grained frames to learn multi-granular video representations. Hou et al. [15] further advanced this by introducing a context-query-aware ranking model, where query information permeates the entire video representation construction process to achieve query-guided adaptive feature fusion. From the perspective of knowledge calibration, Chen et al. [4] designed a cross-modal calibration model that first purifies intra-modal information and subsequently employs query-guided alignment to refine joint representations. To address the heavy reliance on precise temporal annotations, Yoon et al. [16] pioneered a weakly supervised framework incorporating text-guided candidate generation and hard negative mining. While these early fusion methods yield high precision, they mandate independent, computationally intensive joint encoding for every query-candidate pair, rendering them impractical for large-scale, real-world retrieval scenarios.

To circumvent the prohibitive computational overhead of early fusion models, the late fusion paradigm was developed. Adopting a decoupled two-stream architecture, this paradigm encodes video and text modalities independently.This allows corpus-side features to be pre-computed and indexed offline, avoiding query-conditioned re-encoding of every candidate video while retaining corpus-dependent retrieval scoring. A milestone in this direction was the TVR benchmark and its efficient baseline model, XML (Cross-modal Moment Localization), introduced by Lei et al. [5]. Subsequent research has largely focused on mitigating the inherent cross-modal alignment deficiencies of late fusion. For example, Zhang et al. [6] integrated video- and frame-level contrastive learning tasks into their ReLoCLnet to enhance semantic consistency from both global and local dimensions. To enrich the video representations intrinsically, Kim et al. [17] developed a semantic correlated network that deeply aligns visual and subtitle modalities during the initial encoding phase. Addressing the bottleneck of annotation dependency, Jung et al. [18] proposed an automatic pseudo-query generation network to guide model learning. Other studies have explored more granular architectural designs. Hou et al. [19] tackled “partial relevance” issues via a robust two-stage framework, while Zhang et al. [20] utilized deformable attention and query-guided mechanisms to construct more tightly coupled representations. Under weakly supervised settings, Pan et al. [21] introduced frame-based auxiliary alignment and weighted contrastive learning to alleviate insufficient alignment. Owing to its outstanding efficiency and scalability, the late fusion architecture has become the prevailing solution for massive video retrieval. Consequently, our work specifically targets this pipeline, aiming to perform bias elimination and feature quality optimization within late fusion models.

2.2. Bias Elimination

The robustness and generalization capabilities of machine learning models are heavily challenged by implicit biases inherent in training datasets. These biases frequently induce models to learn spurious statistical correlations, adopting a “shortcut learning” strategy. Although this phenomenon often yields inflated performance during in-distribution testing, it severely compromises the models’ efficacy in real-world, out-of-distribution applications.

Within the domain of single Video Moment Retrieval (VMR), extensive efforts have been dedicated to mitigating shortcut learning caused by dataset biases. Yoon et al. [7] explicitly identified evaluation biases in existing benchmarks and designed a multi-way deconfounding framework that utilizes causal intervention and fine-grained feature enhancement to bolster cross-modal alignment. Approaching the problem from a causal inference perspective, Nan et al. [9] employed backdoor adjustment combined with dual contrastive learning to eliminate confounders. Similarly, Yang et al. [10] utilized backdoor adjustment, focusing specifically on decoupling video content from temporal location features to alleviate positional bias. To tackle biases stemming from imbalanced temporal annotations, Wang et al. [22] proposed a model-agnostic disentangled feature mixing framework utilizing novel augmentation strategies to generate diverse training samples. Meanwhile, Lan et al. [23] progressively broke data biases by constructing hierarchical negative samples via cross-video shuffling and employing curriculum learning. Adopting an adversarial strategy, Qi et al. [24] introduced a dynamic bias generator that continuously creates “bias-conflict” samples, forcing the primary model to abandon reliance on uni-modal shortcuts through a minimax game.

The broader VCMR task encounters similar dataset challenges, and recent studies have begun to expose these vulnerabilities. Yoon et al. [8] argued that biases should not be uniformly eliminated but selectively utilized or suppressed based on the query semantics, proposing a selective query-guided debiasing network to this end. To tackle models’ reliance on spurious correlations between text queries and specific visual scenes, they introduced a counterfactual debiasing framework. This approach constructs parallel counterfactual networks to capture shortcuts and forces the main model to learn genuine cross-modal correspondences by contrasting fact with counterfact.

However, despite these valuable explorations, a critical specific bias—positional bias (where ground-truth moments exhibit a systematically uneven distribution along the temporal dimension)—has not yet received adequate attention in the VCMR domain. Currently, there is a distinct lack of systematic, quantitative analysis regarding the prevalence of this bias in mainstream VCMR benchmarks and its specific impact on model behavior. Therefore, this work is dedicated to addressing positional bias in VCMR, explicitly analyzing its underlying causal impact to drive the community toward establishing more robust and equitable model evaluation frameworks.

3. Problem Formulation

3.1. Task Formulation

While the qualitative objectives of Video Corpus Moment Retrieval (VCMR) have been introduced, it is imperative to formalize the task mathematically to establish a rigorous foundation for our proposed framework. The core challenge of VCMR lies in its inherent multi-modal nature, requiring the effective fusion of visual information (e.g., characters, scenes, and actions) with synchronized textual information (e.g., subtitles or dialogues) to achieve comprehensive semantic understanding.

Formally, let $\mathcal{V}={\left\{v_i\right\}}_{i=1}^n$ denote an extensive video corpus consisting of n untrimmed videos, and let $q_j$ represent a given natural language query. Each video $v_i\in \mathcal{V}$ can be represented as a sequence of l continuous clips, denoted as $v_i=[c_1^i,c_2^i,\dots ,c_l^i]$. Under this multi-modal setting, each clip $c_k^i$ encapsulates paired information: the visual frame sequences at that specific timestamp and the synchronously aligned subtitle text. Given the query $q_j$ and the corpus $\mathcal{V}$, the objective of the VCMR task is to accurately retrieve the optimal target moment $v_i[t_{st}:t_{ed}]=[c_{t_{st}}^i,\dots ,c_{t_{ed}}^i]$ that best matches the semantic intent of $q_j$, where $t_{st}$ and $t_{ed}$ denote the start and end clip indices within the target video $v_i$, respectively.

3.2. Out-of-Distribution Evaluation Paradigm

To rigorously evaluate a model’s performance on the formulated VCMR task, it is critical to re-examine the evaluation paradigm itself. Standard evaluation protocols inherently rely on the Independent and Identically Distributed (IID) assumption, presuming that both training and testing datasets are sampled from an identical, fixed joint probability distribution. Under this assumption, standard test metrics are considered valid proxies for a model’s generalizability. However, in real-world scenarios, this assumption is often overly idealistic. The dataset construction process inevitably introduces systematic biases—such as the temporal location of moments acting as a hidden confounder. Consequently, models often overfit the biased joint distribution rather than learning the essential, true conditional probability between the target moments and the multi-modal semantics.

To break the bottleneck of the IID assumption and provide a more stringent measurement of true generalization capabilities, we formalize an Out-of-Distribution (OOD) testing framework. Specifically, let the training data source be defined by the distribution $P_{train}(X,Y)$, where the input X comprises the given videos and queries, and the label Y represents the target temporal moments. While standard IID test sets are sampled from this same $P_{train}$, our OOD evaluation introduces a distinctly shifted test distribution $P_{ood}(X,Y)$, such that $P_{ood}\ne P_{train}$. This distribution shift is primarily driven by changes in the marginal distribution of specific attributes (i.e., temporal locations), while the underlying, semantics-based conditional relationships mandated by the task remain constant.

Within this framework, after a VCMR model $f_{\theta }$ is trained on $P_{train}$, its robustness can be quantified by comparing its performance across different distributions. We define the model’s IID performance as the expected evaluation score under the training distribution, denoted as $S_{iid}$, and its OOD performance as the expected score under the shifted distribution, denoted as $S_{ood}$. The objective of introducing this paradigm is to investigate the implicit performance gap (where $S_{iid}>S_{ood}$) prevalent in existing models. Furthermore, this formalization establishes a systematic methodology for constructing high-quality OOD test benchmarks from existing datasets, thereby providing the community with a more reliable and rigorous standard for evaluating true multi-modal semantic comprehension.

4. Methodology

To tackle the challenges and implicit biases outlined above, we propose a novel framework named Content Disentangling with Contrastive Learning (CDCL). The holistic architecture of this study is structurally divided into three core components: (1) Implementation of OOD Testing for VCMR: We instantiate the theoretical OOD evaluation paradigm onto standard VCMR datasets, conducting rigorous generalizability assessments on existing state-of-the-art methods to expose their inherent generalization deficiencies; (2) SCM-based Debiased Inference Framework: We employ a Structural Causal Model (SCM) to fundamentally analyze the spurious reasoning paths caused by temporal location biases, and subsequently design an interventional strategy to eliminate these biased pathways; (3) Query-guided Content Contrastive Disentangling: We concretely implement the modified causal model architecture. By estimating the expectation of the confounder, this component achieves structural decoupling while simultaneously utilizing contrastive learning objectives to highly purify the quality of the disentangled video content features.

4.1. Implementation of Out-of-Distribution Testing on VCMR

To fulfill the objective of constructing an OOD testing benchmark for the VCMR task, this section details the concrete methodology for systematically isolating a subset that strictly conforms to the OOD definition from an existing dataset. The core of this methodology lies in precisely quantifying and leveraging the latent distribution biases inherent in the data.

We select the widely used TV Retrieval (TVR) dataset [5] as our primary research object. Because the distribution of temporal location is a common and critical attribute in video datasets [1], our initial step involves conducting a quantitative modeling of its temporal location bias. By merging the training, validation, and testing splits of the original TVR dataset into a single set, we employ Kernel Density Estimation (KDE) with Gaussian kernels to compute the joint probability density of the normalized start and end temporal locations for all ground-truth moments. Let ${\mathit{z}}_i={(x_i,y_i)}^{\top }$ denote the normalized start and end locations of the i-th annotated moment. We apply the bivariate Gaussian kernel density estimator implemented by scipy.stats.gaussian_kde:

$$\widehat{f}\left(\mathit{z}\right)={\displaystyle \frac{1}{{2\pi n\left|\mathit{H}\right|}^{1/2}}}{\displaystyle \sum _{i=1}^n}\exp \left[-{\displaystyle \frac{1}{2}}{(\mathit{z}-{\mathit{z}}_i)}^{\top }{\mathit{H}}^{-1}(\mathit{z}-{\mathit{z}}_i)\right],$$

(1)

where n is the number of annotated moments and $\mathit{H}=h^2\widehat{\mathbf{\Sigma }}$ is the kernel covariance matrix. Here, $\widehat{\mathbf{\Sigma }}$ is the sample covariance matrix of the normalized start–end pairs, and the default Scott factor is

$$h=n^{-1/(d+4)}=n^{-1/6},d=2.$$

Thus, the covariance matrix accounts for the individual scales of the two coordinates and their statistical dependence. The estimator is evaluated at the observed start–end pairs to obtain the density scores used in the subsequent video-level ranking. No additional boundary correction is applied.

As intuitively validated by the KDE heatmap in Figure 2a, the distribution of ground-truth moments exhibits a severe long-tail trend with a significant concentration towards the beginning of the videos (the bottom-left region). Specifically, our quantitative analysis reveals that over 32% of ground-truth moments have the majority of their temporal length compressed within the first 20% of their respective videos. This massive accumulation forms an explicit “learning shortcut”. This density function $\widehat{f}(x,y)$ provides the crucial mathematical foundation for subsequently identifying and separating IID from OOD samples.

Figure 2. Bivariate Gaussian KDE visualizations of the normalized start–end locations in the merged TVR data and the resulting video-level splits. The darker colors in the figure represent a higher probability range for the samples.

However, implementing this OOD theory in the VCMR task introduces a unique structural challenge: datasets like TVR are organized at the video granularity to narrow the retrieval space. To ensure evaluation fairness and prevent data pollution caused by video content leakage, a strict constraint must be enforced: all queries attached to the same video must be grouped into the same split. Let ${\mathcal{Q}}_v$ denote the set of all query–moment instances associated with video v. The split assignment is defined on v and inherited by every instance in ${\mathcal{Q}}_v$, producing pairwise video-disjoint training, validation, and test subsets. This grouping prevents identical visual and subtitle streams from occurring in both model fitting and evaluation, so the measured performance reflects temporal-location shift rather than source-content overlap. The temporal-frequency statistics are therefore aggregated with the video as the atomic splitting unit.

To overcome this, we approximate a video-level “frequency score” ${\mathcal{S}}_v$ for each video v in the dataset using Equation (2):

$${\mathcal{S}}_v={\displaystyle \frac{1}{N_q}}{\displaystyle \sum _{i=1}^{N_q}}\widehat{f}(q_i^{st},q_i^{ed})$$

(2)

where $N_q$ is the total number of queries attached to video v, and $\widehat{f}(q_i^{st},q_i^{ed})$ is the KDE value of the ground-truth moment corresponding to the i-th query. A higher ${\mathcal{S}}_v$ indicates that the video’s content heavily conforms to the mainstream biased patterns. Based on this metric, we rank the videos in ascending order of ${\mathcal{S}}_v$ and assign the lowest-scoring $20\%$ of videos, together with all of their associated queries, to Test-OOD. The remaining videos are partitioned at the video level into the training set ($70\%$), Test-IID ($5\%$), and validation set ($5\%$). Consequently, no video or associated query is shared by different subsets.

Figure 2b–d visualizes the temporal-location distributions of the resulting Train, Test-OOD, and Val sets. Because Test-OOD is formed from videos at the low-density end of the video-level ranking, its annotated moments exhibit a clear temporal distribution shift from those in Train and Val. The construction targets temporal patterns that are less frequently represented in the pooled annotations; it does not impose a uniform distribution, and local density modes may remain.

Table 1 summarizes the performance of XML and ReLoCLNet under the original TVR protocol and the reconstructed TVR-OOD protocol. The TVR entries are the published results on the original split, whereas the TVR-OOD entries are obtained by retraining the corresponding architectures on the reconstructed training partition and evaluating them on Test-OOD. For example, VCMR R@10 at IoU $=0.5$ changes from $16.14\%$ to $10.60\%$ for XML and from $21.18\%$ to $13.57\%$ for ReLoCLNet. The lower results on Test-OOD indicate that low-frequency temporal-location patterns constitute a more challenging evaluation distribution for both architectures.

Table 1. Performance of XML and ReLoCLNet on TVR and TVR-OOD.

Method			XML		ReLoCLNet
Dataset			TVR	TVR-OOD	TVR	TVR-OOD
VCMR	Recall@k, IOU = 0.5	$R1$	5.60	2.93	7.97	4.30
		$R5$	12.06	7.34	16.29	9.79
		$R10$	16.14	10.60	21.18	13.57
		$R100$	36.66	27.62	43.15	32.15
	Recall@k, IOU = 0.7	$R1$	2.46	1.58	4.10	2.35
		$R5$	6.66	4.19	9.99	5.98
		$R10$	9.25	6.17	13.88	8.66
		$R100$	23.36	17.68	31.23	22.35
VMR	Recall@1, IOU = $\mu$	$\mu =0.3$	48.32	43.41	49.41	42.35
		$\mu =0.5$	30.75	24.90	31.24	25.07
		$\mu =0.7$	13.41	11.71	14.84	11.84

4.2. The Structural Causal Model for VCMR

To rigorously diagnose the reasoning mechanisms of existing late-fusion models and isolate the source of dataset biases, we formulate the VCMR task through the lens of a Structural Causal Model (SCM) [13]. By representing variable dependencies as a Directed Acyclic Graph (DAG), the SCM provides a formal theoretical framework to identify structural flaws and design an effective causal intervention strategy.

Figure 3a illustrates the causal graph abstraction for conventional late-fusion architectures. Solid directed edges denote explicit algorithmic computations, whereas dashed edges signify implicit, confounding correlations learned by the network. The structural assignments within this graph are defined as follows:

Figure 3. The structural causal models before and after causal intervention. $Q_V$: query feature for generating VR scores. $Q_M$: query feature for generating ML scores. L: the location confounder that implicitly leads to spurious dependencies. V: video feature. $VR$: video retrieval (VR) score. $ML$: moment localization (ML) score. Y: the selected candidates for the VCMR task together with their confidence scores.

In this causal abstraction, $Q_V$, $Q_M$, and V are observed network representations, L denotes the absolute temporal location associated with the valid video clips, and Y is the final ranked VCMR output. For branch $r\in \{VR,ML\}$, let $Q_r=Q_V$ for $r=VR$ and $Q_r=Q_M$ for $r=ML$. We denote the joint branch input by $T_r=(Q_r,V)$ and the corresponding task outcome by $O_r\in \{VR,ML\}$. The variable L is the adjustment variable on the location-related backdoor path. In the network implementation, it is represented by the learned location feature ℓ, which is supervised by a fixed absolute positional encoding.

${\mathbf{Q}}_{\mathbf{V}}\to {\mathbf{Q}}_{\mathbf{M}}$. The base text query representation $Q_V$ is linearly projected into a distinct feature space $Q_M$, which is exclusively optimized for moment-level boundary localization.

$\{{\mathbf{Q}}_{\mathbf{V}},\mathbf{V}\}\to \mathbf{VR}$. The Video Retrieval (VR) score is computed by maximizing the cross-modal similarity between the global query feature $Q_V$ and the unrefined, sequential video clip features V.

$\{{\mathbf{Q}}_{\mathbf{M}},\mathbf{V}\}\to \mathbf{ML}$. Similarly, the transformed query $Q_M$ is matched against the video features V. Subsequent convolutional operations are applied to these matching similarities to generate boundary confidence probabilities, collectively denoted as Moment Localization (ML) scores.

$\{\mathbf{VR},\mathbf{ML}\}\to \mathbf{Y}$. The ultimate prediction Y is derived by aggregating the global retrieval and local boundary scores. Formally, given the probabilities $t_i^{st}$ and $t_j^{ed}$ for the start and end clips (constrained by $2\le j-i\le 16$), the video-level semantic similarity $s^{vr}$, and a scaling hyperparameter $\alpha$, the joint candidate confidence is formulated as:

$$s^{vcmr}(\mathit{v},v_i^{st},v_j^{ed}|\mathit{q})=t_i^{st}·t_j^{ed}·\exp \left(\alpha s^{vr}\right)$$

(3)

Candidates across the entire corpus are subsequently ranked based on $s^{vcmr}$ to yield the final retrieval output.

$\mathbf{ML}\leftarrow \mathbf{L}\to \mathbf{V}$ and $\mathbf{VR}\leftarrow \mathbf{L}\to \mathbf{V}$. These dashed pathways highlight the critical causal vulnerability in existing architectures. Since the raw video feature V intrinsically encapsulates absolute temporal location semantics L, the network inadvertently exploits L as a statistical confounder. Instead of optimizing for genuine multi-modal semantic alignment, models are driven to capture spurious correlations, artificially elevating the confidence of moments solely based on their occurrence at typical temporal locations. This confounding effect dictates the backdoor path that severely degrades generalization on out-of-distribution data.

To adjust the location-related backdoor paths, we apply the backdoor criterion separately to the retrieval and localization branches. In the SCM of Figure 3, L precedes $T_r$, is not a descendant of $T_r$, and blocks the modeled backdoor path between the context representation and $O_r$. Under these assumptions, the interventional distribution for branch r is identified as

$$P(O_r\mid do(T_r=t))=\sum _{\mathsf{\ell }\in \mathcal{L}}P(O_r\mid T_r=t,L=\mathsf{\ell })P(L=\mathsf{\ell }),r\in \{VR,ML\}.$$

(4)

The empirical temporal-location prior is highly imbalanced in the training annotations. We therefore evaluate the conditional outcome under a location-balanced reference distribution $\pi$ rather than propagating this empirical frequency into the adjusted representation:

$$P_{\pi }(O_r\mid do(T_r=t))={\displaystyle \sum _{j=1}^{N_l}}P(O_r\mid T_r=t,L={\mathsf{\ell }}_j)\pi \left({\mathsf{\ell }}_j\right),\pi \left({\mathsf{\ell }}_j\right)={\displaystyle \frac{1}{N_l}}.$$

(5)

With the query treated as an observed context, adjustment over the location confounder gives

$$P(Y\mid do(C=c),Q=q)=\sum _{\mathsf{\ell }}P(Y\mid C=c,Q=q,L=\mathsf{\ell })P(L=\mathsf{\ell }\mid Q=q).$$

(6)

For a prescribed target distribution $\pi \left(L\right)$, the corresponding standardized causal response is

$$P_{\pi }(Y\mid do(C=c),Q=q)=\sum _{\mathsf{\ell }}P(Y\mid C=c,Q=q,L=\mathsf{\ell })\pi \left(\mathsf{\ell }\right).$$

(7)

CDCL adopts the location-balanced target $\pi \left({\mathsf{\ell }}_j\right)=1/N_l$ over the $N_l$ valid temporal positions. This uniform distribution is not required by the backdoor criterion; it defines the target population over which the conditional causal responses are standardized. Let $s_j$ denote the content-conditioned compatibility score assigned to location state ${\mathsf{\ell }}_j$. The finite-sample expectation is parameterized as

$${\alpha }_j={\displaystyle \frac{\pi \left({\mathsf{\ell }}_j\right)\exp \left(s_j\right)}{{\sum }_{k=1}^{N_l}\pi \left({\mathsf{\ell }}_k\right)\exp \left(s_k\right)}},\overline{\mathsf{\ell }}={\displaystyle \sum _{j=1}^{N_l}}{\alpha }_j{\mathsf{\ell }}_j.$$

(8)

Because $\pi \left({\mathsf{\ell }}_j\right)$ is uniform, the normalized weights reduce to the softmax attention coefficients. The attention block therefore provides a representation-level approximation of the location-standardized expectation without reintroducing the empirical temporal-frequency distribution as an additional prior.

4.3. Query-Guided Content Contrastive Disentangling

To materialize the backdoor adjustment and yield debiased predictions, we propose the Content Disentangling with Contrastive Learning (CDCL) framework. As illustrated in Figure 4, the core processing pipeline of CDCL transitions from the standard entangled encoding to a disentangling-intervention paradigm. Structurally, the proposed method consists of three interrelated components: the fundamental Disentangling Backbone, a novel Targeted Semantic Refinement mechanism utilizing contrastive learning, and the Confounder Expectation Modeling module for final causal intervention.

Figure 4. An overview of the video processing part of the proposed CDCL, which generates the content part and location part from video features and conducts inference with intervened causal paths.

4.3.1. Primitive Confounder Extracting

To The foundational objective of the Disentangling Backbone is to decompose the raw, confounded video feature $\mathbf{v}$ into a primitive semantic content representation ${\mathit{c}}^v$ and a location confounder representation ${\mathsf{\ell }}^v$. This decoupling is achieved through two parallel Feed-Forward Neural Networks (FFNs), denoted as $g_c$ and $g_l$, respectively (i.e., ${\mathit{c}}^v=g_c\left(\mathbf{v}\right)$ and ${\mathsf{\ell }}^v=g_l\left(\mathbf{v}\right)$). Both mappings are implemented as position-wise residual FFNs that preserve the hidden dimension d. For an input $\mathit{x}\in {\mathbb{R}}^d$, the mapping is defined as $g\left(\mathit{x}\right)=\mathrm{LayerNorm}\left({\mathit{W}}_2\mathrm{ReLU}\left({\mathit{W}}_1\mathit{x}\right)+{\mathit{W}}_3\mathit{x}\right)$, where ${\mathit{W}}_1,{\mathit{W}}_2,{\mathit{W}}_3\in {\mathbb{R}}^{d\times d}$. The first two projections form the nonlinear transformation path, while ${\mathit{W}}_3$ provides a learnable residual projection. No dropout is applied within these FFNs. To enforce the efficacy and purity of this separation, we impose two types of regularization constraints during optimization: Reconstruction and Independence.

Reconstruction Constraint. We constrain ${\mathsf{\ell }}^v$ to closely approximate a static, non-learnable absolute positional embedding $\mathit{p}$ (identical to the standard Transformer absolute positional encodings [25]) via an $L_2$ norm loss ${\mathcal{L}}_{recon}^{\mathsf{\ell }}$. Concurrently, to ensure that ${\mathit{c}}^v$ retains fidelity to the original visual semantics, we employ a Kullback–Leibler (KL) divergence loss ${\mathcal{L}}_{recon}^c$ to penalize the distributional divergence between the content feature and the original entangled feature. The formulations are defined as follows:

$${\mathcal{L}}_{recon}^{\mathsf{\ell }}={\parallel {\mathsf{\ell }}^v-\mathit{p}\parallel }_2$$

(9)

$${\mathcal{L}}_{recon}^c=\mathrm{KL}(\widehat{{\mathit{c}}^v}\left|\right|\widehat{\mathbf{v}})=\sum _i\widehat{{\mathit{c}}_i^v}\log \left({\displaystyle \frac{\widehat{{\mathit{c}}_i^v}}{\widehat{{\mathbf{v}}_i}}}\right)$$

(10)

where $\widehat{{\mathit{c}}^v}$ and $\widehat{\mathbf{v}}$ represent the features activated by the Softmax function along the temporal dimension.

Independence Constraint. To reduce information overlap between ${\mathit{c}}^v$ and ${\mathsf{\ell }}^v$, we penalize their absolute cosine similarity, thereby encouraging the two representations to occupy approximately orthogonal subspaces:

$${\mathcal{L}}_{indep}=\left|{\displaystyle \frac{{\mathit{c}}^v·{\mathsf{\ell }}^v}{\parallel {\mathit{c}}^v\parallel ·\parallel {\mathsf{\ell }}^v\parallel }}\right|$$

(11)

The disentanglement objective jointly enforces location identifiability, content sufficiency, and cross-branch independence. Specifically, ${\mathcal{L}}_{recon}^{\mathsf{\ell }}$ anchors the location representation to temporal positions, ${\mathcal{L}}_{recon}^c$ preserves task-relevant semantics in the content representation, and ${\mathcal{L}}_{indep}$ suppresses redundant information shared by the two branches.

4.3.2. Targeted Semantic Refinement via Contrastive Learning

While the primitive decoupling ensures the structural separation of location and content, the resulting content representation ${\mathit{c}}^v$ remains purely query-agnostic. It merely reconstructs the visual semantics without differentiating between the query-relevant target clips and the irrelevant background noise. To overcome this limitation and further purify the content feature, we introduce a Query-guided Targeted Contrastive mechanism (conceptually abstracted in Figure 5).

Figure 5. Query-guided targeted contrastive learning. Content features inside the annotated interval form the positive set $\mathcal{P}$, while valid features outside the interval form the same-video background set $\mathcal{N}$. The resulting training objective directly supervises the disentangled content branch.

Let ${\mathit{c}}_{bi}^m\in {\mathbb{R}}^d$ denote the disentangled content feature of the i-th valid clip in sample b, where $m\in \{v,s\}$ denotes the visual or subtitle modality, and let ${\mathit{q}}_b^m\in {\mathbb{R}}^d$ denote the corresponding modularized query representation. Their matching score is

$$s_{bi}^m={\left({\mathit{c}}_{bi}^m\right)}^{\mathsf{T}}{\mathit{q}}_b^m.$$

(12)

Let $a_{bi}\in \{0,1\}$ be the valid-clip mask. The positive and background masks are

$$p_{bi}=a_{bi}\mathbb{I}[t_{st}\le i\le t_{ed}],n_{bi}=a_{bi}(1-p_{bi}).$$

(13)

Thus, clips within the annotated interval are positive samples, while valid clips outside it provide same-video background supervision. No additional hard-negative mining or temporal buffer is applied. The targeted contrastive term is formulated using the Jensen–Shannon objective [26]. For each modality, query–content pairs within the annotated interval and those from the valid same-video background constitute the two sample sets evaluated by $s_{bi}^m$. Their contributions are averaged separately over the valid clips of each sample, accommodating variations in sequence length and in the number of available background clips. The corresponding positive and negative score functions are defined as

$${\varphi }^{+}\left(s\right)=\log 2-\mathrm{softplus}(-s),{\varphi }^{-}\left(s\right)=\mathrm{softplus}(-s)+s-\log 2.$$

(14)

The modality-specific targeted contrastive loss is

$${\mathcal{L}}_{\mathrm{targetCL}}^m={\displaystyle \frac{1}{B}}{\displaystyle \sum _{b=1}^B}\left[{\displaystyle \frac{{\sum }_{i=1}^{N_l}{n}_{bi}{\varphi }^{-}\left(s_{bi}^m\right)}{{\sum }_{i=1}^{N_l}{n}_{bi}+ϵ}}-{\displaystyle \frac{{\sum }_{i=1}^{N_l}{p}_{bi}{\varphi }^{+}\left(s_{bi}^m\right)}{{\sum }_{i=1}^{N_l}{p}_{bi}+ϵ}}\right].$$

(15)

The two modality-specific terms are averaged as

$${\mathcal{L}}_{\mathrm{targetCL}}={\displaystyle \frac{1}{2}}\left({\mathcal{L}}_{\mathrm{targetCL}}^v+{\mathcal{L}}_{\mathrm{targetCL}}^s\right).$$

(16)

This objective learns compatibility with the annotated occurrence rather than asserting that related semantic content cannot recur elsewhere in the video. In ReLoCLNet, the original FrameCL objective remains unchanged and operates on the intermediate features preceding the final context-encoder layer. By contrast, ${\mathcal{L}}_{\mathrm{targetCL}}$ supervises the content representations produced by the proposed content–location decomposition. It is used only during training and updates the content branch through back-propagation.

Let ${\mathcal{L}}_{\mathrm{base}}$ denote the unchanged objective of the selected VCMR backbone. For ReLoCLNet, it comprises the original FrameCL, VideoCL, start/end boundary, negative-context, and negative-query terms:

$${\mathcal{L}}_{\mathrm{base}}^{\mathrm{ReLo}}={\lambda }_{\mathrm{FCL}}{\mathcal{L}}_{\mathrm{FCL}}+{\lambda }_{\mathrm{VCL}}{\mathcal{L}}_{\mathrm{VCL}}+{\lambda }_{\mathrm{st}\_\mathrm{ed}}{\mathcal{L}}_{\mathrm{st}\_\mathrm{ed}}+{\lambda }_{\mathrm{neg}\_\mathrm{ctx}}{\mathcal{L}}_{\mathrm{neg}\_\mathrm{ctx}}+{\lambda }_{\mathrm{neg}\_\mathrm{q}}{\mathcal{L}}_{\mathrm{neg}\_\mathrm{q}}.$$

(17)

After integrating CDCL, the complete objective is

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_{\mathrm{base}}+{\lambda }_{\mathrm{TCL}}{\mathcal{L}}_{\mathrm{targetCL}}+{\lambda }_{\mathrm{recon}}{\mathcal{L}}_{\mathrm{recon}}^{\mathsf{\ell }}+{\lambda }_{\mathrm{ind}}{\mathcal{L}}_{\mathrm{indep}}+{\lambda }_{\mathrm{reg}}{\mathcal{L}}_{\mathrm{recon}}^c.$$

(18)

The CDCL-specific coefficients are $({\lambda }_{\mathrm{TCL}},{\lambda }_{\mathrm{recon}},{\lambda }_{\mathrm{ind}},{\lambda }_{\mathrm{reg}})=(0.03,1\times {10}^{-3},1\times {10}^{-3},1\times {10}^{-4})$. These coefficients are selected using the validation set, while the baseline-specific loss settings remain unchanged.

4.3.3. Confounder Expectation Modeling for Debiased Inference

Having obtained the content representation ${\mathit{c}}^m$ and location representation ${\mathsf{\ell }}^m$ for modality m, CDCL constructs task-specific context features for the retrieval and boundary-prediction branches. Let ${\mathit{c}}^m,{\mathsf{\ell }}^m\in {\mathbb{R}}^{B\times N_l\times d}$, where B is the batch size, $N_l$ is the number of valid clips in the current video, and d is the hidden dimension. For task stream $r\in \{vr,st,ed\}$ and attention head $h\in \{1,\dots ,H\}$, the projected query, key, and value tensors are

$${\mathit{Q}}_{r,h}^m={\mathit{c}}^m{\mathit{W}}_{r,h}^{m,Q},{\mathit{K}}_{r,h}^m={\mathsf{\ell }}^m{\mathit{W}}_{r,h}^{m,K},{\mathit{U}}_{r,h}^m={\mathsf{\ell }}^m{\mathit{W}}_{r,h}^{m,V}.$$

(19)

Here, ${\mathit{Q}}_{r,h}^m,{\mathit{K}}_{r,h}^m,{\mathit{U}}_{r,h}^m\in {\mathbb{R}}^{B\times N_l\times d_h}$, $H=4$, and $d_h=d/H$. The masked attention coefficients and aggregated location representation are

$$\begin{array}{cc}\hfill {\mathit{A}}_{r,h}^m& ={\mathrm{Softmax}}_j\left({\displaystyle \frac{{\mathit{Q}}_{r,h}^m{\left({\mathit{K}}_{r,h}^m\right)}^{\mathsf{T}}}{\sqrt{d_h}}}+{\mathit{M}}^m\right),\hfill \\ \hfill {\mathit{e}}_r^m& ={\mathrm{Concat}}_{h=1}^H\left({\mathit{A}}_{r,h}^m{\mathit{U}}_{r,h}^m\right){\mathit{W}}_r^{m,O},\hfill \end{array}$$

(20)

where ${\mathit{M}}^m$ masks padded key positions. The softmax is applied once over the $N_l$ valid key positions. No empirical KDE-frequency weight or second normalization is applied: the attention coefficients model content-conditioned compatibility among the valid positions rather than reinstating the empirical temporal-location prior. The modality-specific projection $g_d^m$ is implemented using the same d-to-d residual FFN architecture as the content and location mappings, with independent parameters for the visual and subtitle modalities and no dropout. The task-specific context representation is

$${\mathit{z}}_r^m={\mathit{c}}^m+g_d^m\left({\mathsf{\ell }}^m\right)+{\mathit{e}}_r^m,r\in \{vr,st,ed\}.$$

(21)

Here, $vr$, $st$, and $ed$ denote the video-retrieval, start-boundary, and end-boundary streams, respectively. The resulting representations are passed to the unchanged query-matching and prediction operations of the selected backbone.

5. Experiments

To systematically validate the effectiveness and generalizability of the proposed Content Disentangling with Contrastive Learning (CDCL) framework in the Video Corpus Moment Retrieval (VCMR) task, we design and conduct comprehensive evaluations. This section is structured as follows: First, we detail the benchmark datasets, evaluation metrics, and implementation specifics to ensure reproducibility. Second, we integrate CDCL into two mainstream late-fusion baselines and rigorously compare their performance on the out-of-distribution (OOD) test set to quantify the advantages of our debiasing strategy. Third, we conduct an ablation study to isolate the contribution of the targeted contrastive learning module. Finally, we provide a qualitative case study to intuitively demonstrate how CDCL rectifies erroneous reasoning and achieves precise moment localization.

5.1. Datasets and Evaluation Metrics

Benchmark Dataset. All experiments are conducted on the widely recognized and challenging benchmark for this domain: the TV Retrieval (TVR) dataset [5]. It comprises approximately 21.8 K videos extracted from six TV shows, with an average length of $76.2$ s. These videos are paired with high-quality subtitles, providing a robust foundation for multi-modal understanding. The dataset provides over 100 K natural language queries (averaging $13.4$ words), each intricately describing a specific visual event corresponding to a manually annotated ground-truth moment (averaging $9.1$ s).

OOD Benchmark Construction. A core challenge for VCMR models is their generalizability when encountering data that violates the Independent and Identically Distributed (IID) assumption. To strictly evaluate the debiasing capability of our CDCL framework, we reorganize the TVR dataset into an OOD benchmark. Exactly following the theoretical KDE-based frequency scoring paradigm established in Section 4.1, we merge all available TVR data and re-split them into a training set ($70\%$), a test-ood set ($20\%$) exhibiting a drastically shifted temporal distribution, a test-iid set ($5\%$), and a validation set ($5\%$). The performance on this test-ood set serves as the primary indicator of a model’s true semantic comprehension capabilities.

Evaluation Metrics. Following standard protocols [5,6], we evaluate the models using metrics tailored for both VCMR and its subtask, Video Moment Retrieval (VMR). For VMR (where the target video is known), we report “Recall@1, IoU = $\mu$”. The Intersection over Union (IoU) measures the temporal overlap between the predicted moment $M_p$ and the ground-truth $M_{gt}$, formulated as $\mathrm{IoU}=(M_p\cap M_{gt})/(M_p\cup M_{gt})$. This metric denotes the percentage of queries where the top-1 predicted moment possesses an IoU $\ge \mu$ (we report $\mu \in \{0.3,0.5,0.7\}$). For the full VCMR task, we report “Recall@k, IoU = $\mu$” (with $k\in \{1,5,10,100\}$), which measures the percentage of queries that successfully retrieve the correct video and accurately localize the moment (IoU $\ge \mu$) within the top-k candidates across the entire corpus.

5.2. Implementation Details

To ensure a fair comparison, we utilize the publicly available features provided by Lei et al. [5]. Visually, videos are segmented into $1.5$-s clips and encoded via pre-trained I3D (1024-d) and ResNet-152 (2048-d) networks to capture dynamic motion and static appearance, respectively. Textually, both queries and subtitles are encoded using a fine-tuned 12-layer RoBERTa-base model, yielding 768-d embeddings. The publicly available I3D, ResNet-152, and RoBERTa representations are used as fixed input features. The subsequent context encoders inherited from the selected baseline and the added CDCL layers are optimized during training.

Our framework is implemented in PyTorch v2.2.2 and executed on a single NVIDIA GeForce RTX 3090 GPU. The network is optimized using the AdamW optimizer with an initial learning rate of $1\times {10}^{-4}$ and a weight decay of $1\times {10}^{-2}$. We apply a linear learning rate warm-up strategy for the first $10\%$ of iterations, followed by linear decay. The models are trained for 20 epochs with a batch size of 32. For checkpoint selection, each backbone is evaluated on the validation set after every epoch. The selected checkpoint is used to compute all final test metrics. Baseline-specific hyperparameters follow their original configurations.

Computational Complexity. Let $N_l$ denote the number of valid clips and d the hidden dimension. Each position-wise d-to-d FFN introduced by CDCL requires $O\left(N_l{d}^2\right)$ operations, while the four-head location-expectation attention requires $O\left(N_l^{2}d\right)$ operations. The number of additional trainable parameters is $O\left(d^2\right)$. The resulting corpus representations are query-independent and can be pre-computed, and ${\mathcal{L}}_{\mathrm{targetCL}}$ is used only during training. Online retrieval therefore uses the scoring procedure of the selected late-fusion backbone.

5.3. Main Results on OOD Generalization

To quantitatively demonstrate the efficacy of our debiasing strategy, we integrate the CDCL as a plug-and-play module into two highly influential and representative late-fusion architectures: XML [5] and ReLoCLNet [6]. Table 2 presents a comprehensive performance comparison on the test-ood set.

Table 2. Performance Comparison of SOTA Baselines with and without the proposed CDCL on the TVR-OOD Dataset.

Baseline Model			XML		ReLoCLNet
w/o or w/ CDCL			w/o	w/	w/o	w/
VCMR	Recall@k, IOU = 0.5	$R1$	2.93	3.63	4.30	4.49
		$R5$	7.34	8.65	9.79	10.24
		$R10$	10.60	12.32	13.57	14.23
		$R100$	27.62	30.35	32.15	33.04
	Recall@k, IOU = 0.7	$R1$	1.58	1.97	2.35	2.41
		$R5$	4.19	4.95	5.98	6.41
		$R10$	6.17	7.30	8.66	9.22
		$R100$	17.68	19.42	22.35	23.36
VMR	Recall@1, IOU = $\mu$	$\mu =0.3$	43.41	47.57	42.35	45.91
		$\mu =0.5$	24.90	28.41	25.07	28.15
		$\mu =0.7$	11.71	13.76	11.84	13.84

As shown in Table 2, CDCL obtains higher scores for every reported metric on both evaluated backbones under the TVR-OOD protocol. For XML, VCMR R@10 at IoU $=0.5$ increases from $10.60\%$ to $12.32\%$, while VMR R@1 at IoU $=0.5$ increases from $24.90\%$ to $28.41\%$. ReLoCLNet also obtains higher scores across the reported metrics, including an increase in VCMR R@10 at IoU $=0.5$ from $13.57\%$ to $14.23\%$. The observed improvements across retrieval and localization metrics support the combination of content–location disentanglement and query-guided contrastive refinement under the evaluated temporal distribution shift.

5.4. Ablation Study on Contrastive Disentangling

To dissect the contribution of individual components within our framework—particularly the effectiveness of the Targeted Contrastive Learning (TCL) mechanism—we conduct a detailed ablation study on the test-ood set. For this analysis, we utilize the XML architecture as our foundational baseline. As reported in Table 3, we systematically evaluate the performance variations when specific modules are incrementally integrated.

Table 3. Ablation Study on the Test-OOD dataset using XML as the baseline. “Disentangling” denotes the causal decoupling module, and “TCL” denotes the Targeted Contrastive Learning.

Architecture	VCMR (IoU = 0.5)		VCMR (IoU = 0.7)		VMR (R@1)
	R@1	R@10	R@1	R@10	$\mathit{\mu }\mathbf{=}\mathbf{0.5}$	$\mathit{\mu }\mathbf{=}\mathbf{0.7}$
Baseline (Vanilla)	2.93	10.60	1.58	6.17	24.90	11.71
+Disentangling (w/o TCL)	3.41	11.75	1.68	6.76	27.67	13.01
+Full CDCL (w/ TCL)	3.63	12.32	1.97	7.30	28.41	13.76

Table 3 shows the contributions of the two CDCL components on the XML backbone. Adding the disentangling module increases VMR R@1 at IoU $=0.5$ from $24.90\%$ to $27.67\%$ and VCMR R@10 at IoU $=0.5$ from $10.60\%$ to $11.75\%$, with increases also observed for the other reported metrics. These results are consistent with the intended role of separating content and temporal-location representations.

Adding TCL to the disentangling variant produces further improvements across the reported metrics. In particular, VCMR R@1 at IoU $=0.7$ increases from $1.68\%$ to $1.97\%$, and VCMR R@10 at IoU $=0.5$ increases from $11.75\%$ to $12.32\%$. Because TCL supervises the decomposed content features using the query and annotated interval, these results support its role as a complementary content-refinement objective. Overall, the ablation results show that both components contribute to the performance of the complete CDCL model under the evaluated protocol.

5.5. Qualitative Analysis

Beyond quantitative metrics, we provide a qualitative case study to intuitively visualize how CDCL rectifies erroneous reasoning. Figure 6 illustrates a challenging scenario where the query is: “Ross enters the apartment of a woman who is holding flowers and closes the door behind him.” The actual event occurs in the middle-to-latter segment of the video $13.21\mathrm{s}\sim 17.17\mathrm{s}$).

Figure 6. Qualitative comparison for one query. The vanilla baseline predicts a moment at the beginning of the video, whereas the CDCL prediction overlaps the annotated target interval more closely.

The vanilla baseline predicts a moment beginning at 0 s, whereas the annotated event occurs at $13.21$–$17.17$ s. The disentangling-only variant moves the prediction to 15–18 s, and the complete CDCL model predicts $13.5$–18 s. In this example, the progression from the baseline to the two CDCL variants is consistent with reduced reliance on the beginning-of-video location pattern and closer use of the query-relevant content.

6. Limitations

This study evaluates temporal-location robustness within the TVR corpus. The results obtained under the original TVR and reconstructed TVR-OOD protocols provide a contextual comparison rather than a matched same-checkpoint IID–OOD comparison, because the corresponding architectures are trained on different data partitions. The experiments therefore characterize performance under the reconstructed Test-OOD protocol but do not establish transfer across independently collected corpus-level VCMR datasets. Evaluation on external benchmarks supporting both corpus retrieval and temporal localization would provide complementary evidence of cross-corpus generalization.

TVR-OOD operationalizes temporal-location shift by ranking videos according to KDE-based temporal-frequency scores and constructing video-disjoint splits. The benchmark therefore targets a specific shift in the distribution of moment locations. Alternative density estimators, split criteria, or naturally occurring temporal shifts may expose different bias patterns and provide complementary robustness settings.

CDCL is instantiated on XML and ReLoCLNet, two representative late-fusion VCMR architectures, and the reported conclusions are confined to this architectural setting. Extending the content–location intervention to early-fusion or end-to-end video–language models requires architecture-specific treatment: decomposition may be performed before cross-modal interaction, whereas post-fusion intervention requires the location confounder to be reformulated over query-conditioned joint representations. The effectiveness of such formulations remains to be investigated.

7. Conclusions

This study provides a comprehensive investigation into the systemic vulnerabilities of video corpus moment retrieval models, specifically regarding the degradation of generalization performance under distribution shifts. We have demonstrated that the impressive accuracy reported by existing methodologies often stems from an over-reliance on spurious temporal location dependencies rather than a genuine comprehension of multimodal semantics. By exploiting these dataset-specific shortcuts, models fail to maintain robust performance when the underlying temporal distributions vary.

To address this challenge, we introduced a computational framework for content feature contrastive disentangling, which leverages structural causal modeling to execute formal interventions via backdoor adjustment. This approach facilitates the explicit decoupling of confounded video features into bias-agnostic content representations and isolated temporal location confounders. A critical contribution of this work is the integration of a targeted contrastive learning mechanism, which ensures that the disentangled content features retain high discriminative capacity. By supervising the decoupling process with a query-guided contrastive objective, the framework effectively reinforces temporal localization precision without reintroducing the original positional biases.

Extensive evaluations on the proposed out-of-distribution benchmark demonstrate that the integration of this framework into representative late-fusion architectures yields consistent performance gains. The results substantiate our core thesis that the synergy between causal intervention and targeted feature refinement is essential for achieving an optimal balance between generalization robustness and retrieval accuracy. Beyond providing a robust solution for temporal debiasing, this work establishes a rigorous evaluation paradigm that encourages the community to prioritize semantic-driven reasoning over bias-driven prediction.

This article is a revised and expanded version of a paper entitled “Resolving Spurious Temporal Location Dependency for Video Corpus Moment Retrieval” [27], which was presented at IEEE International Conference on Systems, Man, and Cybernetics 2024, Kuching, 6–10 October 2024.