Scene-Conditioned General Gaze Following in Static Images with Uncertainty-Preserving Geometry

Abstract

Given a static image and a selected person, general gaze following predicts the person’s likely gaze target. Ambiguous head cues can leave several geometrically plausible targets. We develop a scene-conditioned localization framework that retains directional uncertainty instead of collapsing it into one gaze direction. Building on a fixed, previously published geometry-guided reference, Distribution-Guided Query Interaction (DGQI) refines direction–distance candidates using scene evidence, the Distribution-Conditioned Localization Decoder (DCLD) adjusts the reference heatmap through residual prediction, and Selective Teacher-Margin Ranking Preservation (STMRP) protects selected correct spatial rankings during training. On the GazeFollow official test set, the final model reaches a pooled/global AUC of 0.9190, an original-resolution per-image AUC of 0.9342, and a normalized distance error of 0.1289 compared to the published baseline values of 0.9066, 0.9214, and 0.1423, respectively. Within-reference controls distinguish module effects from cross-generation changes in the scene backbone. DCLD yields the clearest localization gains; DGQI contributes a smaller discrimination gain; and STMRP improves per-image AUC at a localization cost. This separation has revealed that discrimination and peak localization need not improve together. Zero-shot experiments on VideoAttentionTarget and a coverage-limited ChildPlay subset expose further domain sensitivity. These results motivate protocol-aware, multi-metric evaluation of gaze predictions.

1. Introduction

Human gaze is a compact but highly informative signal for attention and social interaction. Observing where another person looks can orient an observer’s attention, support shared or joint attention, and provide cues to intentions and interactional goals. Psychological accounts therefore place gaze perception and gaze following within broader systems of social cognition and coordinated attention [1]. Eye-movement research likewise shows that facial and gaze cues influence overt orienting and saccadic behavior, while recent work using naturalistic scenes demonstrates that gaze-following effects persist beyond simplified laboratory displays [2,3]. Joint attention is also central to developmental research, where eye tracking is increasingly used to quantify gaze-following and gaze-shifting behaviors in children, including autistic populations [4,5]. These findings motivate computational approaches that infer another person’s attentional focus in natural scenes, while also emphasizing that such systems are tools for visual-attention and behavior understanding rather than direct measurements of an observer’s oculomotor state or clinical diagnostic systems. Recent reviews further emphasize that visual attention not only responds to social cues but also shapes social behavior and interpersonal coordination [6]. Empirical eye-tracking comparisons of joint-attention bids also show systematic differences in visual-attention allocation across developmental groups and cue conditions [7].

In general gaze following, the input is a scene photograph and a designated person, and the output is a spatial prediction of where that person is likely looking. Unlike gaze-direction estimation from visible eyes or a face, this task must connect incomplete person cues to possible targets in the scene. A person may be viewed from behind, the eyes may be hidden, or several nearby objects may lie along a plausible gaze direction. GazeFollow established a large benchmark for this image-based task [8], and subsequent methods combined directional cues with scene saliency and spatial target information [9]. Predicted gaze targets can also help interpret human–object interactions [10] and have been investigated for understanding attention in operating-room scenes [11]. These uses depend on knowing both where the model predicts attention and when that prediction may be unreliable.

Recent gaze-following research has substantially strengthened the representation of both scenes and targets. Depth- and geometry-aware formulations constrain the search space using explicit spatial cues [12], while patch-level distribution prediction relaxes overly rigid single-pixel supervision and better accommodates annotation variation [13]. Object-aware and semantic models treat attended regions as meaningful target entities rather than only heatmap peaks [14,15]. Transformer-based methods further model person–scene interactions through controlled tokens or learned interaction features, and large-scale pretrained encoders have raised the performance ceiling with strong general visual representations [16,17,18]. Together, these advances show that contextual, semantic, and relational information is essential for accurate gaze-target inference.

However, stronger scene representations do not by themselves resolve a distinct ambiguity between directional evidence and final target selection. Head-derived evidence can support several neighboring directions, and even after these directions are projected into the image, multiple objects or spatial regions may remain geometrically plausible. For example, two objects at different distances may align with a similarly plausible gaze direction, making directional uncertainty distinct from target identity. Collapsing the person cue into a single vector or a single spatial prior can remove alternatives before scene evidence has an opportunity to distinguish among them. Conversely, representing a broader target-space distribution can capture output ambiguity without necessarily preserving the directional structure that generated those alternatives. This distinction is central to the present study: uncertainty preservation asks which directions or regions remain plausible, whereas target commitment asks which of those plausible hypotheses should ultimately receive the strongest scene-conditioned support.

The uncertainty-aware CDDL–PGGM framework addressed the first part of this problem by representing gaze direction as a circular probability distribution and projecting retained directional hypotheses into a channel-wise spatial geometry volume [19]. This formulation avoids prematurely reducing uncertain head evidence to one deterministic direction and provides an interpretable uncertainty-preserving direction-to-region representation. Its geometry is nevertheless deliberately permissive: it identifies plausible spatial regions but does not explicitly determine which object or position along a plausible direction should receive the final commitment. This leads to the central research question of the present study: how can uncertainty-preserving gaze geometry be converted into scene-conditioned target commitment without discarding its geometric structure or destabilizing spatial relations that are already reliable? This is an engineering design question; the controlled experiments characterize the behaviors and trade-offs of the proposed computational model instead of testing a hypothesis about the stages of human gaze behavior.

We address this question with an uncertainty-preserving geometry framework for scene-conditioned target commitment, built on an established geometry-guided reference. In practical terms, the model first retains plausible gaze regions, then uses scene evidence to compare candidate targets and refine the predicted location. The reference converts the inherited direction distribution into a =1188full-18 probabilistic geometry volume and integrates it with a GCViT-T/FPN scene representation. The Distribution-Guided Query Interaction (DGQI) then factorizes the retained geometry into structured direction–distance candidates and lets them query scene tokens under a geometry-biased attention prior. DGQI redistributes support among plausible candidates and returns both spatial candidate maps and a posterior-weighted contextual representation. The Distribution-Conditioned Localization Decoder (DCLD) uses these outputs to predict a residual correction of the reference heatmap, treating target localization as controlled refinement rather than complete re-prediction. During training, Selective Teacher-Margin Ranking Preservation (STMRP) protects teacher-correct non-local target-versus-competitor rankings while leaving the local target neighborhood editable. STMRP is training-only and adds no inference branch.

The framework separates directional uncertainty preservation, scene-conditioned candidate disambiguation, and final localization. We evaluate these mechanisms on GazeFollow through official test comparisons, paired component and mechanism analyses, and stratification by annotation ambiguity and reference difficulty. Zero-shot transfer to VideoAttentionTarget and an available ChildPlay subset, together with efficiency and qualitative analyses, further reveal their distinct discrimination–localization trade-offs.

The main contributions of this study are summarized as follows:

• We formulate gaze-target prediction after uncertainty-preserving direction-to-region modeling as a scene-conditioned target-commitment problem over structured, uncertainty-preserving gaze geometry. DGQI constructs 216 structured direction–distance candidates from the retained gaze geometry and refines them through geometry-biased cross-attention, allowing scene semantics to redistribute candidate support without discarding the geometric origin of the hypotheses.
• We introduce DCLD, a distribution-conditioned residual localization decoder that combines scene features, coarse and refined candidate maps, and the DGQI context representation to predict a correction of the reference heatmap. Its residual formulation and neutral initialization refine an established localization state instead of replacing it with an independent predictor.
• We propose STMRP, a training-only selective ranking objective that preserves teacher-correct non-local target margins without distilling teacher-wrong relations or constraining the local target neighborhood. The objective preserves reliable global ordering while keeping local target localization editable, and it introduces no additional inference parameters or branch.
• We conduct a controlled, multi-level evaluation that separates the effects of DCLD, DGQI, and STMRP; analyzes their internal behavior with paired statistics and mechanism diagnostics; examines performance across annotation ambiguity and reference difficulty; evaluates zero-shot transfer to VideoAttentionTarget and a coverage-limited ChildPlay subset; and quantifies the computational cost. The results reveal complementary component roles and explicit discrimination–localization trade-offs rather than a uniformly monotonic ablation chain.

2. Related Work

2.1. Social Attention, Joint Attention, and Computational Gaze Following

Behavioral studies place gaze following within social and joint attention [1], while showing that its cueing effects depend on context, reliability, and their interaction with domain-general orienting [20,21]. Recent experiments using complex scenes are particularly relevant to computational gaze following because they show that observed gaze can guide attention under naturalistic visual conditions, where ocular information competes with richer scene content [3]. Recent cueing experiments further suggest that gaze cues can be more resistant than arrows to reinterpretation, indicating qualitative differences beyond average cueing magnitude [22].

Joint-attention research provides a complementary perspective on why target localization matters. Eye-tracking studies quantify whether and how observers follow social cues toward a shared object or region, and recent reviews emphasize substantial variation in task design, stimulus context, and gaze metrics [4,23]. More recent work has also moved toward automated and ecologically oriented joint-attention measurement [5]. Although computational gaze following does not measure joint attention directly, these studies share a need to connect person-conditioned directional cues with meaningful targets in complex scenes.

Within computer vision, the GazeFollow benchmark formalized third-person gaze-target inference from a scene and a selected person [8]. Later work integrated scene saliency, out-of-frame reasoning, and video modeling [9,24], while gaze-following outputs have also been used as intermediate behavioral cues for downstream human–object interaction prediction [10]. Related recent work has even used gaze-following labels as weak supervision for unconstrained 3D gaze estimation, illustrating a growing connection between gaze-target reasoning and broader gaze modeling [25]. These developments motivate gaze following as both a visual localization problem and a component of human-centered behavior understanding.

2.2. Geometry, Context, and Target-Space Structure

A recurring challenge in gaze following is that head orientation or an estimated gaze direction rarely identifies a unique target. Early methods combined person-specific gaze cues with scene saliency, whereas later approaches incorporated explicit field-of-view, depth, geometric, and contextual guidance [9,12]. Three-dimensional scene reasoning and auxiliary geometric cues can narrow the target search, but their localization quality remains coupled to the reliability of the upstream direction or auxiliary modality. Recent multi-view gaze-target estimation similarly exploits cross-view head and scene information to reduce face occlusion and target ambiguity [26].

A second line of work changes the structure of the target representation itself. Patch-Level Gaze Distribution Prediction models a distribution over spatial patches and explicitly addresses annotation variation and in-/out-of-frame prediction [13]. Object-aware gaze-target detection instead associates people with candidate objects or heads, providing a more semantic representation of the attended entity [14]. Recent semantic gaze-target formulations continue this transition from point localization toward identifying what is being attended to, while multi-person methods exploit social relations between people and targets [15,16,27]. Together, these approaches show that target prediction benefits from structuring the candidate space rather than treating the final heatmap as an isolated regression output. Transformer-based gaze-object prediction likewise models long-range head–object relations and directly predicts the attended object category and location [28]. Recent context-spatial integration work further combines gaze-target localization with social-gaze prediction, reinforcing the complementary roles of spatial consistency and contextual evidence [29].

The CDDL–PGGM framework addressed a different but related source of ambiguity: uncertainty in the direction-to-region mapping itself. Instead of collapsing head-derived evidence to one direction, it represented gaze as a circular distribution and projected multiple retained hypotheses into a channel-wise spatial probability volume [19]. This representation preserves multiple plausible directional regions, but it does not explicitly resolve competition among positions that remain compatible with those regions. The present study therefore starts from uncertainty-preserving geometry and focuses on the subsequent target-selection stage.

2.3. Scene Interaction and Modern Visual Encoders

Transformer-based gaze-following methods have shifted attention from manually designed fusion pipelines toward stronger scene representation and explicit person–scene interactions. Sharingan encodes person-specific information into controlled tokens and processes multiple people jointly, improving its efficiency and relational modeling [16]. ViTGaze exploits interaction information embedded in pretrained Vision Transformer self-attention and demonstrates that strong encoders can support gaze following with lightweight decoding [17]. These studies establish long-range interactions and learned feature correlations as important alternatives to increasingly complex task-specific decoders. The present study instead examines how uncertainty-preserving gaze geometry can structure the candidate space used for such scene interactions.

Large, pretrained encoders have also improved gaze following: Gaze-LLE combines a frozen DINOv2 encoder with a lightweight person-specific decoder [18]. Recent methods extend the problem through temporal and social contexts [15,27], audio–visual conversation cues [30], language-model reasoning [31], head-centered spatial representations [32], and unified vision–language gaze-target interpretation [33]. These developments illustrate the value of richer semantics and additional modalities; our emphasis instead lies on scene interactions constrained by uncertainty-preserving geometry.

The present study is complementary to these encoder- and modality-oriented advances. We do not claim a new Transformer backbone or a foundation-model alternative. GCViT-T provides a shared scene representation for the within-family component comparisons; its scale is comparable to ResNet-50, but its representation is not assumed to be equivalent. The methodological focus is instead on how uncertainty-preserving gaze geometry structures downstream interactions. DGQI constructs direction–distance queries from the probabilistic geometry volume and injects candidate-specific geometry into cross-attention as an explicit prior. The interaction therefore evaluates scene semantics within a spatial hypothesis space that remains tied to the retained gaze distribution.

2.4. Uncertainty Preservation, Ranking Supervision, and Target Commitment

Uncertainty is intrinsic to third-person gaze inference because head appearance, occlusion, annotation dispersion, and multiple nearby targets can all make the attended location ambiguous. The CDDL–PGGM formulation treated this uncertainty at the directional-representation level, preserving neighboring hypotheses before heatmap predictions [19]. The present study preserves that principle but moves the modeling emphasis downstream. Directional uncertainty is first converted into structured candidate support; scene-conditioned interaction then redistributes probability among plausible direction–distance candidates. Within the proposed computational model, uncertainty preservation and scene-conditioned target commitment are complementary operations rather than competing strategies.

A related methodological theme is ranking-based supervision. Ranking distillation was introduced to transfer relative preference structure from a teacher instead of matching only pointwise scores [34], and more recent work has examined how ranking information can be benchmarked, decomposed, or transferred under different learning settings [35,36]. Ranking formulations have also been used in visual-attention problems to encode relative salience among competing instances [37]. These studies support the general value of relative ordering as supervision, although their tasks and teacher–student assumptions differ from gaze following. Recent logit-distillation work based on Kendall’s τ further supports relative-order supervision as a complement to direct logit matching [38].

STMRP is positioned in this broader ranking-supervision context but is designed specifically for gaze heatmaps. It does not distill the complete teacher distribution, force numerical agreement with teacher logits, or preserve teacher-wrong target relations. Instead, it protects only teacher-correct non-local target-versus-competitor margins while leaving the local target neighborhood editable for residual localization. Together with DGQI and DCLD, STMRP enables a controlled decomposition within the proposed architecture: geometry structures the candidate space, scene interactions redistribute its support, residual decoding refines localization, and selective ranking preservation constrains learning. This combination distinguishes the present work from approaches centered primarily on stronger visual encoders, additional modalities, semantic labeling, or temporal/social reasoning.

3. Methods

3.1. Overview of the Geometry-Guided Target-Commitment Framework

Given a scene image and a selected person, the model produces a spatial heatmap of the likely gaze target. The present study extends uncertainty-aware CDDL–PGGM direction-to-region modeling by using uncertainty-preserving gaze geometry as the basis for scene-conditioned target commitment. The inherited Circular Direction Distribution Learning (CDDL) branch preserves directional uncertainty, and Probabilistic Gaze Geometry Modeling (PGGM) converts the resulting direction distribution into a structured spatial prior [19]. Instead of replacing this representation, the proposed framework uses it as a stable geometric basis for resolving which target should receive final commitment when multiple scene locations remain geometrically plausible. The overall geometry-guided target-commitment framework is shown in Figure 1.

Figure 1. Geometry-guided target-commitment framework. (a) Overall framework. The frozen CDDL–PGGM branch preserves directional uncertainty and constructs the full-18 geometry volume, which is fused with RGB input to establish the GCViT-T/FPN geometry-guided reference. DGQI and DCLD are optimized in the second stage on top of the frozen reference, whereas STMRP is used only during training. (b) DGQI forms 216 structured direction–distance candidates from 18 direction groups and 12 radial bins, combines candidate queries with scene tokens under a geometry-biased cross-attention prior, and produces the coarse and refined candidate maps C₀ and C together with the context vector g. (c) DCLD conditions the spatial scene feature using g, projects the coarse and refined candidate maps, and combines these signals with H₀ to predict the residual correction ΔH; the response-change visualizations are interpretation aids rather than additional inference outputs. (d) STMRP uses the detached reference prediction H₀ as a teacher and the final prediction H as a student, standardizes them using statistics from the non-local region, excludes the 3 × 3 target neighborhood from ranking preservation, and penalizes only the reductions in teacher-valid non-local margins. Schematic elements are labeled explicitly; unlabeled embedded response maps show outputs from the representative sample used in the figure. The sample image is from the GazeFollow dataset [8].

Figure 1a summarizes the overall architecture. A gaze-subject crop and normalized eye position are jointly processed by the frozen CDDL branch to obtain the 72-bin directional distribution, which is grouped into the 18-direction posterior used by PGGM. PGGM combines this grouped posterior with the eye position to construct the full-18 probabilistic geometry volume. The full-18 volume is concatenated with the RGB scene at the input level and encoded by a GCViT-T/FPN reference network, yielding a 56 × 56 scene feature and a geometry-guided reference heatmap. In parallel, the full-18 geometry and eye position define structured direction–distance candidates for the proposed target-commitment pathway. The detailed motivation and construction of the reference representation are given in Section 3.2.

Distribution-Guided Query Interaction (DGQI) lets the geometry-conditioned candidates interact with scene tokens under an explicit geometric attention prior, producing a refined candidate distribution and a posterior-weighted contextual representation. Distribution-Conditioned Localization Decoder (DCLD) combines these outputs with the reference scene feature and heatmap to predict a residual spatial correction. Geometry therefore narrows the hypothesis space, DGQI resolves scene-conditioned support among plausible candidates, and DCLD converts the resulting evidence into final spatial localization.

Selective Teacher-Margin Ranking Preservation (STMRP) complements the two inference-time modules only during learning. The frozen geometry-guided reference acts as a detached teacher, and STMRP selectively preserves teacher-correct non-local ranking relations while leaving the local target neighborhood editable. Consequently, the BCE-only and STMRP-regularized variants share exactly the same DGQI + DCLD inference architecture. Figure 1b–d expand on the DGQI, DCLD, and STMRP frameworks, and make their distinct information paths explicit.

3.2. Uncertainty-Preserving Geometry-Guided Reference Representation

The proposed target-commitment pathway is built on a geometry-guided reference representation that preserves the directional uncertainty established by the CDDL–PGGM framework [19]. Given a gaze-subject crop and normalized eye location e = (e_x,e_y), the frozen Circular Direction Distribution Learning (CDDL) branch produces 72 directional logits. After softmax normalization, every four adjacent 5° bins are aggregated into one of J = 18 direction groups, yielding the grouped posterior p = {p_j} in Equation (1), where $𝒢$_j denotes the set of four 5° bins belonging to direction group j, and 𝓁 denotes the 72-dimensional directional-logit vector. The group centers are uniformly spaced by 20°; in terms of image coordinates, 0° points upward, and the positive angular direction is clockwise. The grouped posterior is retained explicitly rather than collapsed to a single gaze direction, as the subsequent target-commitment stages operate on multiple plausible directional hypotheses.

$$p_j=\sum _{k\in {\mathcal{G}}_j}\mathrm{softmax}{\left(\mathcal{l}\right)}_k, j=1,\dots ,18.$$

(1)

The grouped posterior is converted into a spatial probabilistic geometry volume using the inherited PGGM construction. For a pixel location x on the 224 × 224 scene grid, we let v(x) denote the displacement vector from the eye position to x and d_j be the unit vector of direction group j. The forward projection r_j(x) = v(x) ⋅ d_j determines whether the pixel lies in front of the gaze origin, whereas the perpendicular displacement c_j(x) is the absolute two-dimensional cross-product magnitude between v(x) and d_j. A cone-like directional field F_j(x) is formed with the distance-dependent width σ_j(x) = max(10 + 0.15 r_j(x),1)), and responses behind the gaze origin are suppressed, as given by Equation (2). Here, 1 [·] denotes the indicator function.

$$F_j\left(x\right)=\exp \left[-{\displaystyle \frac{c_j{\left(x\right)}^2}{2{\sigma }_j{\left(x\right)}^2}}\right]\hspace{0.17em}1\left[r_j\left(x\right)>0\right].$$

(2)

In Equation (3), the resulting full-18 volume V ∈ ℝ^18×224×224 (V = {V_j(x)}) represents 18 probability-weighted forward gaze fields rather than a single deterministic ray. It is concatenated with the RGB scene image I ∈ ℝ^3×224×224 to form a 21-channel early-fusion input [I;V]. This design carries the uncertainty-preserving geometry into scene representation by learning from the first visual encoding stage, so that appearance features are formed under an explicit spatial gaze prior.

V_j(x) = p_jF_j(x).

(3)

Compared with the ResNet-50/FPN [39] heatmap network used in the CDDL–PGGM framework, the present reference adopts a GCViT-T/FPN [40] scene encoder. This change is motivated by the target-commitment problem rather than by a generic replacement of a CNN with a Transformer. Gaze-target localization often requires relating a gaze subject to candidate regions that are spatially distant while retaining local visual detail for distinguishing nearby competing targets. GCViT combines hierarchical local self-attention with global-context interaction, providing both short- and long-range spatial representations that can be integrated naturally with an FPN and subsequently queried by DGQI. The Tiny variant was selected as a capacity-controlled compromise: the official GCViT-T configuration contains approximately 28 M parameters and 4.7 GFLOPs at 224 × 224 input resolution, keeping its backbone capacity in a range comparable to ResNet-50 while avoiding the substantially larger GCViT-S and GCViT-B variants. This choice avoids a substantial increase in backbone size while providing local–global scene interactions; however, similar parameter counts do not imply equivalent representations or isolate the backbone contribution.

To retain ImageNet-pretrained GCViT-T weights after expanding the input stem from three to 21 channels, the original RGB kernels are preserved for the first three channels, whereas each of the 18 additional geometry channels is initialized from the channel-wise mean of the pretrained RGB kernels. The four hierarchical GCViT feature levels are then integrated by a feature pyramid network (FPN) [41]. Each level is projected to 256 channels and combined through the standard top-down pathway with lateral connections. The finest pyramid output P₂ is used as the reference scene representation F_s ∈ ℝ^256×56×56. A lightweight heatmap decoder maps F_s through 256 → 128 → 64 → 1 channels to obtain the reference heatmap logits H₀ ∈ ℝ^1×56×56. The final reference uses input-level geometry fusion only; the full-18 volume is not projected and re-injected immediately before the heatmap decoder. The corresponding architecture controls are reported in Section 4.3.2 and Supplementary Table S1.

Once this reference has been established, its complete state is fixed for target-commitment learning. Its forward pass exposes the grouped directional posterior p, full-18 volume V, scene feature F_s, and reference heatmap logits H₀; the normalized eye location e is retained as an additional geometric input. DGQI uses p, V, F_s, and e to construct and refine structured direction–distance candidates, whereas DCLD receives F_s and H₀, together with the DGQI outputs, to predict a residual localization correction. Fixing the reference during this stage preserves the inherited uncertainty-aware geometry and the established scene representation, thereby isolating the learning capacity of the proposed target-commitment mechanisms within the GCViT-based reference family.

3.3. Distribution-Guided Query Interaction

The full-18 representation constrains plausible gaze regions, but it does not by itself determine which location along a plausible direction corresponds to the attended target. Distribution-Guided Query Interaction (DGQI) addresses this remaining ambiguity by converting the continuous probabilistic geometry into a structured direction–distance candidate space and allowing each candidate to query scene content under an explicit geometric prior. Figure 1b illustrates this process. Importantly, DGQI does not generate the final gaze heatmap; it refines the candidate distribution and produces a scene-conditioned context representation that is subsequently consumed by DCLD.

Equations (4)–(6) define the radial basis, the direction–distance candidate support, and the corresponding coarse priors and normalized spatial masks. For numerical stability, the candidate mass used for normalization is lower-bounded by a small positive constant.

We first construct R = 12 radial bases around the normalized eye location e. On the 56 × 56 prediction grid, the normalized radial distance from the eye to a spatial location x is ρ(x;e). The radial centers are evenly distributed as μ_m = (m + 0.5)/R for m = 0, …, 11, and each radial basis uses a fixed width σ_r = 1/24. The resulting Gaussian basis is:

$$R_m\left(x;e\right)=\exp \left[-{\displaystyle \frac{{\left(\rho \left(x;e\right)-{\mu }_m\right)}^2}{2{\sigma }_r^2}}\right].$$

(4)

The full-18 volume from Section 3.2 is area-resized to 56 × 56 and combined with the radial bases. For direction j and radial bin m, the unnormalized candidate support is:

W_j,m(x) = V_j(x)R_m(x;e).

(5)

Its total mass $a_{j,m}=\sum _x{W}_{j,m}\left(x\right)$ defines the coarse candidate prior ${\pi }_{j,m}^0$, while normalization within the candidate yields the spatial mask M_j,m(x):

$${\pi }_{j,m}^0={\displaystyle \frac{a_{j,m}}{\sum _{u,v}{a}_{u,v}}}, M_{j,m}\left(x\right)={\displaystyle \frac{W_{j,m}\left(x\right)}{a_{j,m}}}.$$

(6)

With 18 direction groups and 12 radial bins, this produces 216 structured candidates. Because V_j already contains the directional posterior p_j and suppresses locations behind the gaze origin, the candidate mass combines directional confidence, forward gaze geometry, and radial compatibility. For interactions with scene tokens, each normalized candidate mask is adaptively pooled to 14 × 14 and renormalized over the resulting 196 positions, giving a candidate-specific geometric attention prior G_q,t.

Equations (7)–(9) define the directional-uncertainty descriptor, the candidate query, and the geometry-biased cross-attention weights.

Each candidate is represented by a 128-dimensional query vector that encodes its geometric identity and the current uncertainty state. The normalized entropy of the 18-group directional posterior is:

$${\mathcal{H}}_{\mathrm{dir}}=-{\displaystyle \frac{{\displaystyle {\sum }_{j=1}^{18}}{p}_j\log p_j}{\log 18}}.$$

(7)

For candidate q = (j,m), the query vector is formed by adding a learned direction embedding, a learned radial embedding, an encoded probability descriptor [p_j, logp_j, $\mathscr{H}$_dir], and an encoded eye-position vector, followed by layer normalization:

q_j,m = LN [E_dir(j) + E_rad(m) + ϕ_p (p_j, logp_j, $\mathscr{H}$_dir) + ϕ_e(e)],

(8)

where E_dir(j) and E_rad(m) denote learned 128-dimensional direction and radial embeddings, respectively. The mappings ϕ_p: ℝ³ → ℝ¹²⁸ and ϕ_e: ℝ² → ℝ¹²⁸ encode the probability descriptor [p_j, logp_j, $\mathscr{H}$_dir] and the normalized eye position, respectively, while LN(⋅) denotes the layer normalization. The directional posterior therefore enters DGQI in two distinct roles. Through the full-18 volume, it determines the coarse spatial candidate mass, whereas through the query descriptor, it informs the interaction module about the probability and uncertainty states associated with each direction. These two uses are complementary: one defines where a candidate is geometrically supported while the other conditions how that candidate interprets the scene’s evidence.

The frozen scene feature F_s ∈ ℝ^256×56×56 is adaptively pooled to 14 × 14, flattened into 196 scene tokens, linearly projected from 256 to 128 dimensions, and augmented with a fixed two-dimensional sine/cosine positional encoding. Layer normalization then yields scene tokens s_t ∈ ℝ¹²⁸, t = 1, …, 196. This spatial tokenization provides a compact scene representation while retaining the positional structure required for candidate-specific interaction.

DGQI uses four-head cross-attention with an embedding dimension of 128, giving a per-head dimension d_h = 32. We let $Q_q^{\left(h\right)}$ and $K_t^{\left(h\right)}$ denote the query and key projections for candidate q and scene token t in attention head h, respectively; G_q,t is the candidate-specific geometric prior, ε > 0 is a numerical stabilizer, and softmax_t denotes normalization over the scene-token index t. Following scaled dot-product attention [42], the content affinity between candidate q and scene token t is augmented by the logarithm of the corresponding geometric prior. For head h, the attention weight $A_{q,t}^{\left(h\right)}$ is:

$$A_{q,t}^{\left(h\right)}={\mathrm{softmax}}_t\left[{\displaystyle \frac{Q_q^{\left(h\right)}\cdot K_t^{\left(h\right)}}{\sqrt{d_h}}}+\log \left(G_{q,t}+\epsilon \right)\right].$$

(9)

The geometric term G_q,t acts as an attention prior rather than a hard mask: scene content can redistribute attention within the candidate support, but locations with little geometric support receive correspondingly low prior probability. The attended values from all heads are concatenated, projected, and added back to the query through a residual connection; a second residual feed-forward block then produces the interacted candidate representation z_q ∈ ℝ¹²⁸. This design makes scene semantics responsible for evaluating candidate compatibility while preserving uncertainty-aware gaze geometry as the spatial constraint on that evaluation.

Equations (10)–(12) define the refined candidate posterior, its coarse and refined spatial projections, and the posterior-weighted context vector.

A learned scalar scorer maps each interacted representation z_q to a semantic score s_q. Instead of replacing the geometric prior with an unconstrained softmax, DGQI performs multiplicative semantic reweighting of the coarse candidate prior:

$$c_q={\displaystyle \frac{{\pi }_q^0\exp \left(s_q\right)}{{\displaystyle {\sum }_{k=1}^{216}}{\pi }_k^0\exp \left(s_k\right)}},$$

(10)

where c_q denotes the refined posterior probability of candidate q, and $c={\left\{c_q\right\}}_{q=1}^{216}$ denotes the complete refined candidate posterior. The scorer output layer is initialized to zero. Consequently, s_q = 0 for all candidates at initialization, and the refined posterior c exactly reduces to the coarse prior π⁰. This neutral initialization preserves the reference candidate distribution before learning and allows DGQI to acquire scene-conditioned refinements without introducing an arbitrary initial shift.

The coarse and refined candidate maps C₀(x) and C(x) are projected into the 56 × 56 spatial domain using the normalized candidate masks:

$$C_0\left(x\right)={\displaystyle \sum _{q=1}^{216}}{\pi }_q^0{M}_q\left(x\right), C\left(x\right)={\displaystyle \sum _{q=1}^{216}}{c}_q{M}_q\left(x\right).$$

(11)

In parallel, the interacting candidate representations are aggregated into a posterior-weighted context vector:

$$g={\displaystyle \sum _{q=1}^{216}}{c}_q{z}_q.$$

(12)

DGQI therefore supplies DCLD with two complementary forms of information: C and C₀ retain spatially resolved candidate evidence, whereas g ∈ ℝ¹²⁸ summarizes scene-conditioned candidate semantics in a global vector. The candidate maps are intermediate representations rather than final gaze predictions; DCLD determines how they should influence the final heatmap.

For the mechanism diagnostics reported in Section 4.4, we additionally characterize the refined candidate posterior without using these quantities as training losses. Its normalized entropy $\mathscr{H}$_cand and divergence from the coarse prior D_KL (c‖π⁰) can be expressed as Equations (13) and (14):

$${\mathcal{H}}_{\mathrm{cand}}=-{\displaystyle \frac{{\displaystyle {\sum }_{q=1}^{216}}{c}_q\log c_q}{\log 216}},$$

(13)

$$D_{\mathrm{KL}}\left(c\parallel {\pi }^0\right)={\displaystyle \sum _{q=1}^{216}}{c}_q\log {\displaystyle \frac{c_q}{{\pi }_q^0}}.$$

(14)

Thus, the KL direction used throughout this study is refined posterior relative to coarse prior. Top-1 probability and top-5 probability mass are computed directly from c. These diagnostics quantify how strongly DGQI redistributes candidate probability mass, whereas final prediction quality is evaluated only after the refined representation is decoded by DCLD.

3.4. Distribution-Conditioned Localization Decoder

DGQI refines which direction–distance candidates are supported by the scene, but the candidate representation is not itself the final gaze prediction. The Distribution-Conditioned Localization Decoder (DCLD) converts this candidate evidence into a spatial correction of the geometry-guided reference heatmap. As illustrated in Figure 1c, DCLD receives the frozen scene feature F_s, the reference heatmap logits H₀, the refined and coarse candidate maps C and C₀, and the posterior-weighted DGQI context vector g. The design deliberately preserves both spatial candidate evidence and global candidate context, allowing the decoder to determine how scene content and distributional refinement should modify the existing localization state instead of replacing it with an independent prediction.

Equations (15)–(18) describe the feature conditioning, candidate projection, decoder-input fusion, and residual heatmap update of DCLD.

The frozen scene feature F_s ∈ ℝ^256×56×56 is first projected to 128 channels by a 1 × 1 convolution followed by GroupNorm and GELU, producing F_p ∈ ℝ^128×56×56. The context vector g ∈ ℝ¹²⁸ is layer-normalized and mapped by a two-layer multilayer perceptron to 256 values, which are split into feature-wise scale and shift vectors γ, δ∈ℝ¹²⁸. Following feature-wise linear modulation (FiLM) [43], the projected scene feature is conditioned as:

F_cond = F_p ⊙ (1 + γ) + δ,        F_cond ∈ ℝ^128×56×56,

(15)

where ⊙ denotes element-wise multiplication, and γ and δ are broadcast over the 56 × 56 spatial grid. This pathway lets the posterior-weighted DGQI context globally modulate which channels of the scene representation should be emphasized or suppressed without collapsing the spatial organization of F_s.

In parallel, the refined and coarse candidate maps are concatenated channel-wise and projected from two channels to 16 channels by a 1 × 1 convolution before being followed by GELU:

F_cand = ψ_cand ([C;C₀]),        F_cand ∈ ℝ^16×56×56.

(16)

where ψ_cand denotes the 1 × 1 candidate-projection convolution followed by GELU, and [C;C₀] denotes channel-wise concatenation. Retaining both C and C₀ allows the decoder to access not only the refined spatial support but also the geometry-derived reference distribution from which that support was obtained. The decoder can therefore learn from the relationship between the coarse and refined candidate fields instead of receiving only their difference or an already collapsed scalar summary.

The conditioned scene features, encoded candidate maps, and current reference heatmap states are then concatenated:

X = [F_cond; F_cand; H₀],        X ∈ ℝ^145×56×56.

(17)

Thus, X is the 145-channel fused input to the refinement decoder. A lightweight refinement decoder maps X through two 3 × 3 convolutional blocks with channel dimensions 145 → 128 → 64; each block uses GroupNorm and GELU. A final 1 × 1 convolution predicts a one-channel correction field ΔH ∈ ℝ^1×56×56. The final heatmap logits H are obtained by residual addition:

H = H₀ + ΔH.

(18)

This residual formulation is central to DCLD. The decoder is not required to reconstruct the entire gaze heatmap from scratch; instead, it learns where the already established geometry-guided response should be strengthened, suppressed, or spatially shifted. The reference heatmap H₀ therefore remains an explicit prediction state throughout localization, while DGQI-derived information conditions the correction applied to that state. This design is consistent with the target-commitment objective: the inherited geometry narrows the hypothesis space, DGQI refines candidate support, and DCLD translates that evidence into the final spatial adjustment.

DCLD is initialized to preserve the geometry-guided reference exactly. The final FiLM layer is zero-initialized, so γ = δ = 0 at initialization, and the final correction convolution is also zero-initialized, giving ΔH = 0. Consequently, before learning, F_cond = F_p and H = H₀. This neutral initialization makes the trainable target-commitment pathway start from the established reference behavior and learn only data-supported departures from it.

The controlled DCLD-only configuration used in Section 4 preserves the same decoder architecture while removing learned DGQI information. Specifically, the context vector is fixed to zero, and the refined candidate input is replaced by the static coarse candidate map, so that g = 0 and C = C₀. DCLD therefore still receives the frozen scene representation, reference heatmap, and static geometry but no learned query–scene candidate refinement. This matched construction isolates the downstream localization capacity of DCLD from the upstream candidate-disambiguation contribution of DGQI.

3.5. Selective Teacher-Margin Ranking Preservation

A localization correction can improve the target heatmap while also altering spatial rankings that the frozen reference already gets right. DCLD and DGQI introduce trainable corrections on top of a frozen geometry-guided reference. While this added flexibility is necessary for target commitment, unconstrained optimization can also disturb non-local spatial rankings that the reference already predicts correctly. Selective Teacher-Margin Ranking Preservation (STMRP) addresses this issue as a training-only objective. As illustrated in Figure 1d, the frozen reference heatmap H₀ serves as a detached teacher, whereas the final DGQI + DCLD heatmap H is the student. STMRP does not force the student to reproduce the teacher heatmap. Instead, it selectively preserves only those non-local target-versus-competitor orderings for which the teacher already assigns a higher standardized logit to the ground-truth target.

For each training sample, the normalized gaze point is mapped to a ground-truth cell i^∗ on the 56 × 56 prediction grid using the same 56 × 56 coordinate convention used throughout training and evaluation: each normalized coordinate is multiplied by the corresponding heatmap size, rounded to the nearest integer, and clipped to the valid index range. We let $𝒩$(i^∗) denote the local neighborhood consisting of all valid cells within one pixel of i^∗ in both horizontal and vertical directions. This forms a 3 × 3 region for interior targets. STMRP excludes $𝒩$(i^∗) from ranking preservation so that the student remains free to reshape the local peak and improve fine-grained localization.

To focus the loss on relative ordering rather than absolute logit scale or bias, teacher and student logits are standardized independently within each sample using statistics computed only over the non-local set Ω = $𝒳$∖$𝒩$(i^∗), where $𝒳$ denotes all 56 × 56 heatmap cells. We let T = H₀ denote the detached teacher logits and S = H the student logits. For Y ∈ {T,S}, we let μ_Y and σ_Y denote the mean and standard deviation computed over the non-local set Ω, and |Ω| denotes the number of cells in that set. These statistics are defined in Equation (19):

$${\mu }_Y={\displaystyle \frac{1}{\left|\Omega \right|}}\sum _{i\in \Omega }Y\left(i\right), {\sigma }_Y=\sqrt{{\displaystyle \frac{1}{\left|\Omega \right|}}\sum _{i\in \Omega }{\left(Y\left(i\right)-{\mu }_Y\right)}^2}.$$

(19)

Equation (20) defines the standardized logit $\tilde{Y}\left(i\right)$, where ε > 0 is a small numerical constant that prevents division by zero:

$$\tilde{Y}\left(i\right)={\displaystyle \frac{Y\left(i\right)-{\mu }_Y}{\max \left({\sigma }_Y,\epsilon \right)}}.$$

(20)

Because the normalization statistics are estimated from non-local cells only, edits within the 3 × 3 target neighborhood do not indirectly change the ranking reference through the mean or variance. The standardization also makes STMRP insensitive to ordinary positive affine rescaling of the heatmap logits, so that the loss primarily concerns relative spatial margins rather than logit magnitude.

For each non-local competitor cell i ∈ Ω, Equation (21) defines the teacher and student target-versus-competitor margins m_T(i) and m_S(i), where $\tilde{T}$ and $\tilde{S}$ are the standardized teacher and student logits from Equation (20).

$$m_T\left(i\right)=\tilde{T}\left(i^{\star }\right)-\tilde{T}\left(i\right), m_S\left(i\right)=\tilde{S}\left(i^{\star }\right)-\tilde{S}\left(i\right).$$

(21)

Equations (22)–(24) define the protected pair set, the selective teacher-margin ranking loss, and the final training objective.

A pair is protected only when the teacher already ranks the ground-truth cell above the competitor. The protected set is therefore defined as:

$𝒫$ = {i ∈ Ω:m_T(i) > 0}.

(22)

Teacher-wrong pairs, for which m_T(i) ≤ 0, are ignored rather than distilled. For protected pairs, the student is penalized only when it reduces the teacher’s correct margin:

$$L_{\mathrm{STMRP}}={\displaystyle \frac{1}{\left|\mathcal{P}\right|}}\sum _{i\in \mathcal{P}}\mathrm{ReLU}\left[m_T\left(i\right)-m_S\left(i\right)\right],$$

(23)

where ReLU(·)=max(·,0). If the student preserves or enlarges a teacher-correct margin, that pair contributes zero loss. Thus, STMRP is asymmetric: it preserves reliable ordering but does not require numerical agreement with the teacher and does not prevent the student from becoming more confident where the teacher is already correct. When no protected pair is present, the ranking term is defined as zero.

The final training objective combines the heatmap binary cross-entropy loss with STMRP using the fixed coefficient 0.10:

L = L_BCE + 0.10 L_STMRP.

(24)

STMRP introduces no additional inference branch or parameterized module, so the BCE-only and STMRP-regularized models use exactly the same DGQI + DCLD architecture at test time. The 3 × 3 exclusion keeps the local target region editable for residual localization, whereas selective non-local margin preservation discourages the new pathway from sacrificing reliable global ranking structure. This distinction motivates the separate discrimination and localization analyses in Section 4.3, Section 4.4 and Section 4.5. With the neutral initialization described in Section 3.3 and Section 3.4, the student starts from H₀, so L_STMRP = 0 initially; the ranking constraint becomes active only as the trainable target-commitment pathway departs from the frozen reference.

3.6. Staged Training and Inference of the Target-Commitment Pathway

Training proceeds in two functional stages. The inherited CDDL branch remains fixed throughout. First, the GCViT-T/FPN geometry-guided reference and its heatmap decoder are optimized under heatmap supervision while using the fixed CDDL–PGGM directional representation. Once this reference has been established, its complete state is fixed and used as the common basis for target-commitment learning. In the second stage, only DGQI and DCLD are optimized on top of the frozen reference. This staged design separates the establishment of an uncertainty-preserving direction-to-region representation from the subsequent learning of candidate disambiguation and residual target localization.

For each target-commitment training sample, the frozen reference is evaluated to obtain p, V, F_s, and H₀. DGQI constructs the 216 structured direction–distance candidates, performs geometry-biased query–scene interactions, and outputs the refined candidate posterior together with C, C₀, and g. DCLD uses these quantities with F_s and H₀ to predict ΔH and forms the final logits H = H₀ + ΔH. The BCE-only path optimizes DGQI and DCLD with the heatmap binary cross-entropy loss. In the STMRP-regularized path, the architecture and trainable parameters are unchanged; the detached reference logits H₀ are additionally used as the teacher in the ranking-preservation term defined in Section 3.5.

The BCE-only and STMRP-regularized target-commitment models therefore differ only in the training objective, not in their inference architecture. They are initialized from the same reference-preserving state: the DGQI semantic scorer initially leaves the coarse candidate prior unchanged, and DCLD initially produces zero residual correction. Consequently, both variants start from the same geometry-guided prediction before their trainable pathway adapts to the data. This controlled initialization helps attribute subsequent changes to learned target commitment rather than to an arbitrary perturbation of the reference output.

At inference, only a single forward path is required. The frozen reference first produces the uncertainty-aware geometry and scene representation; DGQI refines the structured candidate distribution from scene evidence; and DCLD converts the resulting candidate information into the residual correction that yields H. STMRP is not evaluated, no teacher–student comparison is performed, and no additional training-only branch is retained. The proposed model therefore incurs inference cost only from DGQI and DCLD. The computational effect of these modules is quantified separately in Section 4.7.

The same frozen geometry-guided reference is also used to construct the controlled component variants evaluated in Section 4.3, enabling the contributions of DCLD, DGQI, and STMRP to be isolated without changing the reference representation. Data splits, optimization hyperparameters, model-selection rules, full-data retraining, and evaluation protocols are reported in Section 4.1.

4. Experiments and Results

4.1. Experimental Setup

We evaluate the proposed framework on GazeFollow as the primary in-domain benchmark and further examine its zero-shot transfer to VideoAttentionTarget (VAT) and ChildPlay. Model development and final benchmark evaluation were kept strictly separated: architectural choices, training objectives, and training durations were determined from the official GazeFollow training data, whereas the official test set was reserved for final evaluation. The same finalized GazeFollow-trained models were subsequently applied to the external datasets without target-domain adaptation.

4.1.1. Datasets and Evaluation Scope

GazeFollow is the primary benchmark used in this study [8]. Its official training annotations contain 125,557 person instances. For development, the training data were partitioned by image identity into 112,988 training instances and 12,569 development-validation instances, preventing people from the same source image from appearing in both subsets. The GazeFollow target coordinates were manually estimated by annotators asked to identify where the depicted person appeared to be looking; they are not eye-tracker recordings of that person’s fixations. The official test set contains 4782 person instances, with 10 human gaze-target annotations collected for each instance, and was used only for final in-domain evaluation.

Cross-dataset generalization was assessed on two video-domain benchmarks. VAT extends gaze following to video person-frames and provides annotations for spatial gaze targets and in-/out-of-frame behavior [24,44]. Because the present framework predicts only an in-frame spatial heatmap, the zero-shot VAT evaluation is restricted to all 20,655 in-frame person-frames in the official test split; out-of-frame average precision is outside the scope of the present model.

ChildPlay contains child-centered social interactions with scene statistics that differ substantially from GazeFollow [45]. The locally available subset covers 25 of the 45 official test clips and contains 14,332 inside-visible person-frames from 12 videos. These results are therefore treated as coverage-limited supplementary evidence rather than as a complete-benchmark comparison. Neither VAT nor ChildPlay was used for training, fine-tuning, architecture selection, or training-duration selection.

The photographs reproduced in this paper are drawn exclusively from the GazeFollow benchmark [8], which incorporates images from several pre-existing datasets; no ChildPlay [45] video frames are reproduced. Reuse of these photographs remains subject to the applicable terms and rights of their respective original image sources.

4.1.2. Evaluation Metrics

For person instance i, we let G_i denote the set of M_i valid normalized gaze annotations g_ij; therefore, ${\overline{g}}_i$ denotes their arithmetic mean, e_i the normalized eye position of the gaze subject, H_i the predicted heatmap, and ${\widehat{g}}_i$ the normalized location of the maximum heatmap response. Two AUC constructions and four localization measures are reported. Because ROC-AUC depends on the construction of gaze-target annotation maps and on score aggregation, the two AUC protocols are explicitly distinguished rather than treated as interchangeable [46].

Pooled/global AUC (5 × 5) is retained for direct continuity with the CDDL–PGGM evaluation protocol [19]. Each predicted heatmap is resized to 5 × 5, the valid human gaze-target annotations are mapped to a binary 5 × 5 gaze-target annotation map, and the prediction and annotation-map arrays from the complete test set are concatenated before a single ROC-AUC is computed. Original-resolution per-image AUC instead resizes each predicted heatmap to the original image resolution, computes ROC-AUC separately for each person instance using all valid annotations, and averages the resulting per-instance values [17,18].

Equation (25) defines Dist_i and MDist_i. Dist_i is the Euclidean distance from the predicted peak to the mean human annotation ${\overline{g}}_i$, whereas MDist_i is the minimum Euclidean distance from the predicted peak to any valid annotation g_ij. Both are computed in normalized image coordinates. Dist therefore measures the deviation from the annotation centroid, whereas MDist reflects the agreement with at least one plausible human annotation.

$${\mathrm{Dist}}_i={∥{\widehat{g}}_i-{\overline{g}}_i∥}_2, {\mathrm{MDist}}_i=\underset{1\le j\le M_i}{\min }{∥{\widehat{g}}_i-g_{\mathit{ij}}∥}_2.$$

(25)

Equation (26) defines the three gaze vectors used for angular evaluation, where ${\widehat{v}}_i$ is the predicted gaze vector from the gaze-subject eye position to the predicted target, ${\overline{v}}_i$ is the vector from the eye position to the mean human annotation, and v_ij is the vector from the eye position to the j-th valid annotation.

$${\widehat{v}}_i={\widehat{g}}_i-e_i, {\overline{v}}_i={\overline{g}}_i-e_i, v_{\mathit{ij}}=g_{\mathit{ij}}-e_i.$$

(26)

Equation (27) defines Ang_i and MAng_i. Ang_i is the angular error, in degrees, between the predicted gaze vector and the direction toward the mean annotation, whereas MAng_i is the minimum angular error over the valid individual annotations. The cosine argument is clipped to [−1, 1] for numerical stability, and zero-length annotation vectors are excluded from the minimum-angle calculation. MDist and MAng are minimized independently and therefore need not correspond to the same annotation. A higher AUC is better, whereas lower Dist, MDist, Ang, and MAng indicate better localization.

$$\begin{array}{c}{\mathrm{Ang}}_i={\displaystyle \frac{180}{\pi }}\arccos \left({\displaystyle \frac{{\widehat{v}}_i\cdot {\overline{v}}_i}{\parallel {\widehat{v}}_i{\parallel }_2\parallel {\overline{v}}_i{\parallel }_2}}\right),\\ {\mathrm{MAng}}_i={\displaystyle \frac{180}{\pi }}\underset{1\le j\le M_i}{\min }\arccos \left({\displaystyle \frac{{\widehat{v}}_i\cdot v_{\mathit{ij}}}{\parallel {\widehat{v}}_i{\parallel }_2\parallel v_{\mathit{ij}}{\parallel }_2}}\right).\end{array}$$

(27)

VAT and ChildPlay are evaluated with their dataset-specific in-frame conventions rather than with either of GazeFollow AUC constructions. For VAT, we report the benchmark-specific per-person-frame tolerant-support AUC, normalized distance, short-side-normalized distance, and angular error. The coverage-limited ChildPlay subset is evaluated using its corresponding benchmark-specific AUC and the same three localization measures. These external AUC values are interpreted only within their respective datasets.

4.1.3. Training and Evaluation Protocol

The Circular Direction Distribution Learning (CDDL) branch and the full-18 probabilistic gaze-geometry representation were inherited from CDDL–PGGM and kept fixed throughout the present target-localization study [19]. The geometry-guided reference retains this 21-channel RGB + full-18 image-level early-fusion representation, but replaces the ResNet-50/FPN heatmap backbone with a GCViT-T/FPN scene encoder. The final reference uses only the input-level geometry fusion; the full-18 representation is not injected again immediately before the heatmap decoder.

All newly introduced localization mechanisms were developed using the image-disjoint training and development-validation subsets. For development-stage selection, we computed the mean per-instance ROC-AUC from each native 56 × 56 predicted heatmap using the single annotated gaze-target location for that development instance as the positive target. We refer to this quantity as development-validation AUC; it differs from both official test AUC protocols defined in Section 4.1.2 and was used as the primary selection criterion, with lower Dist and then lower Ang serving as the exact-tie criteria. After the architecture and training settings had been fixed, the geometry-guided reference, DCLD with static geometry, DGQI + DCLD with BCE-only training, and the proposed DGQI + DCLD with STMRP-regularized training were trained on all 125,557 official GazeFollow training instances for two, seven, five, and six epochs, respectively. The training duration of each configuration was selected separately using the same development-validation criterion before full-data retraining. Thus, the official test component comparisons share the prescribed data and evaluation protocols, but their final training durations are not epoch-matched; the observed differences should be interpreted as effects of the respective selected configurations rather than as epoch-isolated component effects.

The first three configurations use binary cross-entropy (BCE) heatmap supervision. The proposed configuration has the same DGQI + DCLD inference architecture as its BCE-only counterpart and differs only in the training objective, which augments BCE with Selective Teacher-Margin Ranking Preservation (STMRP). The resulting four models were evaluated once on the official GazeFollow test set using the same metric implementation. VAT and ChildPlay were then evaluated in a strict zero-shot setting using these finalized GazeFollow-trained models.

4.1.4. Implementation Details

The framework was implemented in PyTorch 2.12.0 and trained on a single NVIDIA GeForce RTX 5090 D V2 GPU with 24 GB of memory. Scene images and gaze-subject crops were resized to 224 × 224 pixels, and gaze-target prediction was represented by a 56 × 56 heatmap. The geometry-guided reference uses GCViT-T (GCViT-Tiny) as the hierarchical scene backbone [40], followed by an FPN that produces a 256-channel feature map at 56 × 56 resolution.

To preserve the full-18 early-fusion strategy while using ImageNet-pretrained GCViT-T weights, the input stem was expanded from three to 21 channels. The pretrained RGB kernels were retained for the first three channels, and each of the 18 additional PGGM channels was initialized with the channel-wise mean of the pretrained RGB kernels. The geometry-guided reference was optimized with AdamW using learning rates of 8 × 10⁻⁵ for the scene backbone and 4 × 10⁻⁴ for the heatmap decoder, with weight decay 1 × 10⁻⁴.

For DCLD with static geometry and the DGQI + DCLD configurations, the established geometry-guided reference was kept fixed while the newly introduced trainable localization modules were optimized with AdamW at 4 × 10⁻⁴, using the same weight decay and gradient-norm clipping at 5.0. No learning-rate scheduler was applied to these newly introduced modules. The training batch size was 64, the development-validation batch size was 128, and six data-loading workers were used.

Training used bfloat16 mixed precision; FP16 was not used. Final GazeFollow inference used FP32 computation without autocast. For STMRP-regularized training, the objective is L = L_BCE + 0.10L_STMRP. A 3 × 3 neighborhood centered on the ground-truth heatmap cell is excluded from the protected non-local ranking pairs, allowing local target refinement while preserving the intended non-local ranking constraint. The BCE-only and STMRP-regularized DGQI + DCLD models share the same initialization of all trainable DGQI/DCLD parameters.

4.1.5. Statistical Analysis

Paired analyses were performed on the same GazeFollow test instances whenever a per-instance metric was available. Pooled/global AUC is a single statistic computed after concatenating the complete test set and is therefore not used for paired person-instance inference. For original-resolution per-image AUC, Dist, MDist, Ang, and MAng, all six pairwise contrasts among the four finalized configurations were specified before the official test analysis. Each paired effect is accompanied by a nonparametric bootstrap 95% confidence interval based on 2000 resamples and a two-sided Wilcoxon signed-rank test; p-values are Holm-adjusted within each metric family. Sample-level analyses additionally report the proportions of improved, unchanged, and degraded instances. For each principal paired contrast, we also report the paired standardized mean difference d_z, calculated as the mean signed per-instance benefit divided by the sample standard deviation of signed benefits. The benefit direction is defined as the candidate minus the reference for original-resolution per-image AUC and the reference minus the candidate for localization errors; a positive value therefore denotes improvement. Differences with absolute magnitude at most 10⁻¹² are counted as unchanged. These standardized effects are not computed for the pooled/global AUC, which is a dataset-level statistic.

For the video-domain evaluations, resampling respects the temporal grouping of repeated frames. VAT confidence intervals are estimated with 10,000 cluster-bootstrap replicates at the (show, clip, track) level over 256 tracks, and the corresponding rank-based comparisons are performed on track means. ChildPlay confidence intervals are obtained from 10,000 clip-level bootstrap replicates on the coverage-limited subset, with rank-based comparisons performed on clip means. Holm correction is applied to the predefined metric families. When bootstrap intervals and rank-based tests provide different indications, both are reported rather than reduced to a single binary significance label.

Three-seed development experiments are summarized descriptively by their mean and sample standard deviation. Given the small number of seeds, these results are not interpreted as standalone inferential significance tests.

4.2. GazeFollow Benchmark Performance

On GazeFollow, the proposed framework improves the published CDDL–PGGM baseline in both discrimination and localization, reaching a pooled/global AUC of 0.9190 and an original-resolution per-image AUC of 0.9342 while reducing all four localization errors. Table 1 organizes the evidence into four explicitly labeled blocks: historical reference values, a protocol-matched pooled/global comparison, contemporary per-instance discrimination results, and methods using additional modalities or training resources. The One Human entry in Table 1a is retained for historical context; because its annotation aggregation and AUC construction are not established as identical to those used in the controlled comparison, it is not interpreted as a protocol-matched human–model performance gap. Methods that use additional modalities or substantially different training resources are separated so that the strongest direct claims remain tied to sufficiently compatible evaluation settings [19].

Table 1. Protocol-aware comparison with representative gaze-following methods on GazeFollow: (a) historical benchmark baselines and human reference; (b) strict conventional pooled/global comparison; (c) modern per-image or per-person-instance AUC; and (d) methods using additional cues, modalities, or training resources.

(a)
Method		AUC ↑	Dist ↓		MDist ↓		Ang ↓	MAng * ↓
Random [8]		0.504	0.484		0.391		69.0°	–
Center [8]		0.633	0.313		0.230		49.0°	–
Fixed bias [8]		0.674	0.306		0.219		48.0°	–
Judd et al. [47]		0.711	0.337		0.250		54.0°	–
SalGAN [48]		0.848	0.238		0.192		36.7°	22.4°
Recasens et al. [8]		0.878	0.190		0.113		24.0°	–
Recasens et al. (ResNet) [49] *		0.881	0.175		0.101		22.5°	11.6°
One Human [8] *		0.924	0.096		0.040		11.0°	–
(b)
Method		AUC ↑	Dist ↓		MDist ↓		Ang ↓	MAng * ↓
Lian et al. [49]		0.9060	0.1450		0.0810		17.60°	8.80°
CDDL–PGGM baseline [19]		0.9066	0.1423		0.0801		16.90°	8.23°
DGQI + DCLD + STMRP (ours)		0.9190	0.1289		0.0676		15.99°	7.27°
(c)
Method	Reported AUC ↑	Verified Evaluator Family				Representation/Model Class
Chong et al. [9]	0.921	Original-resolution per-image mean				Scene–head heatmap model
PDP [13]	0.934	Original-resolution per-image mean				Patch-level target distribution
Sharingan [16]	0.944	Per-image mean; output resolution				Multi-person Transformer
ViTGaze [17]	0.949	Original-resolution per-image mean				Transformer interaction features
Gaze-LLE, ViT-L [18]	0.958	Original-resolution per-person-instance mean				Large frozen visual encoder
CDDL–PGGM baseline [19]	0.9214	Original-resolution per-image mean				ResNet-50 direction-to-region model
Ours	0.9342	Original-resolution per-image mean				Geometry-guided DGQI–DCLD framework
(d)
Method		Reported AUC ↑		Main Comparability Difference
DAM [12]		0.922		Monocular depth and explicit three-dimensional gaze cues
ESCNet [50]		0.928		Three-dimensional geometry and scene-understanding modules
Gupta et al., image-only [51]		0.933		Additional gaze-direction pretraining on Gaze360
Gupta et al., multimodal [51]		0.943		RGB, depth, pose, and additional pretraining
Head–local–global coordination [52]		0.939		Local/global coordination, depth, and multiple gaze-pretraining datasets

* In Table 1a, Recasens et al. (ResNet) denotes the ResNet-50-based reimplementation reported by Lian et al. One Human is retained as a human-level reference. The exact spatial resolution and cross-sample aggregation of the historical Recasens AUC implementation cannot be reconstructed from the original paper alone. Table 1b is used for the direct conventional pooled/global comparison. Values in Table 1c,d provide broader research positioning under their reported evaluator families. Bold identifies the proposed model in Table 1b.

4.2.1. Protocol-Matched Comparison with the CDDL–PGGM Baseline and Lian et al.

Under the strict conventional pooled/global comparison in Table 1b, the proposed DGQI + DCLD + STMRP framework achieves an AUC of 0.9190 compared to 0.9066 for the published CDDL–PGGM baseline [19] and 0.9060 for Lian et al. [49]. Relative to the CDDL–PGGM baseline, Dist decreases from 0.1423 to 0.1289, MDist from 0.0801 to 0.0676, Ang from 16.90° to 15.99°, and MAng from 8.23° to 7.27°. Original-resolution per-image AUC also increases from 0.9214 to 0.9342. These matched protocol gains demonstrate that the present framework advances the earlier direction-to-region formulation in both discrimination and target localization.

The CDDL–PGGM comparator [19] is from our previously published study in the Journal of Eye Movement Research. Its benchmark values are taken from that publication rather than recomputed using the evaluation implementation of the present work; comparisons are reported only under the evaluator constructions identified in Table 1. At the reported numerical precision, the updated geometry-guided reference already accounts for approximately 91.1% of the total cross-generation pooled/global AUC gain; 98.4% of the original-resolution per-image AUC gain; and 94.0%, 93.6%, 95.6%, and 94.8% of the improvements in Dist, MDist, Ang, and Mang, respectively. These descriptive shares compare the published CDDL–PGGM system, the updated reference, and the final configuration; they do not isolate the causal effect of GCViT-T, since the cross-generation systems also differ in representation and training circumstances. The incremental roles of DCLD, DGQI, and STMRP are therefore evaluated primarily through the within-reference comparisons in Section 4.3.

The historical benchmark block provides additional context for the scale of these gains. The proposed model substantially outperforms the non-learned spatial baselines, generic saliency predictors, and the original Recasens gaze-following model, while the One Human row provides a human-level reference [8,47,48]. Because this is a cross-generation framework comparison, component-level attribution is addressed separately by the controlled analysis in Section 4.3.

4.2.2. Comparison with Contemporary Gaze-Following Methods

Table 1c shows that the proposed method reaches an original-resolution per-image AUC of 0.9342. This clearly improves on the published CDDL–PGGM results of 0.9214 and 0.921 reported by Chong et al. [9], and is essentially at the 0.934 level reported by PDP [13]. PDP reports two GazeFollow configurations: without depth, its reported AUC, average distance, and minimum distance are 0.928, 0.131, and 0.072; with depth, the corresponding values are 0.934, 0.123, and 0.065. For comparison, the present RGB-and-head model yields original-resolution per-image AUC of 0.9342, Dist of 0.1289, and MDist of 0.0676. The depth-free PDP comparison favors the present model on these reported metrics, whereas the depth-enabled PDP reports lower localization errors despite a similar AUC. As these values originate from independently reported evaluations, the comparison is descriptive rather than a controlled modality ablation.

More recent Transformer and foundation-encoder approaches report stronger absolute per-image benchmark values. Sharingan, ViTGaze, and Gaze-LLE [16,17,18] report AUC values of 0.944, 0.949, and 0.958, respectively, under their corresponding evaluator families. These results show that stronger semantic interaction modeling and large-scale visual pretraining can further improve absolute benchmark performance. The present framework substantially advances the CDDL–PGGM formulation but remains below the strongest reported values from these recent Transformer and large, pretrained approaches; its primary contribution is therefore the controlled study of how uncertainty-preserving geometry can support downstream target commitment rather than a claim of state-of-the-art benchmark performance.

Table 1d separately summarizes methods that use depth, three-dimensional scene cues, poses, additional gaze-direction pretraining, or multiple gaze-pretraining datasets [12,50,51,52]. Taken together, Table 1 shows a clear progression from the published CDDL–PGGM baseline to the present DGQI–DCLD framework and positions the resulting performance within contemporary gaze-following systems while keeping resource and protocol differences visible.

4.3. Controlled Component Analysis

The controlled analysis separates changes in inference architecture from changes in the training objective and reveals a clear division of labor among the proposed mechanisms. As defined in Section 3, the geometry-guided reference retains the fixed CDDL–PGGM full-18 representation, uses 21-channel image-level early fusion with a GCViT-T/FPN scene encoder, and does not re-inject geometry immediately before decoding. Table 2 shows the controlled component analysis on the official GazeFollow test set. Table 2a compares the four finalized configurations, whereas Table 2b reports the principal paired effects for the controlled transitions.

Table 2. Controlled component analysis on the official GazeFollow test set: (a) final configurations and metric-wise performances; and (b) paired effect estimates for the principal controlled contrasts.

(a)
Configuration	DCLD	DGQI	Training Objective	Pooled/Global AUC (5 × 5) ↑	Per-Image AUC ↑	Dist ↓	MDist ↓	Ang ↓	MAng ↓
Geometry-guided reference	–	–	BCE	0.9179	0.9340	0.1297	0.0684	16.03°	7.32°
DCLD with static geometry	✓	–	BCE	0.9197	0.9327	0.1269	0.0667	15.62°	7.00°
DGQI + DCLD	✓	✓	BCE	0.9197	0.9332	0.1268	0.0666	15.52°	7.03°
Proposed DGQI + DCLD	✓	✓	BCE + 0.10 STMRP	0.9190	0.9342	0.1289	0.0676	15.99°	7.27°
(b)
Controlled Contrast		Metric	Mean Benefit	Paired dz	95% Bootstrap CI		Holm-Adjusted p	Interpretation
DCLD with static geometry vs. reference		Dist	+0.002757	+0.0620	[+0.001536, +0.004032]		1.96 × 10−13	Localization benefit
DGQI + DCLD vs. DCLD with static geometry		Per-image AUC	+0.000515	+0.0864	[+0.000348, +0.000675]		1.31 × 10−9	Supported discrimination benefit
DGQI + DCLD vs. DCLD with static geometry		Dist	+0.000061	+0.0017	[−0.000950, +0.001092]		0.129	No statistically supported Dist change
STMRP-regularized vs. BCE-only DGQI + DCLD		Per-image AUC	+0.001010	+0.1055	[+0.000741, +0.001280]		2.31 × 10−45	Discrimination benefit
STMRP-regularized vs. BCE-only DGQI + DCLD		Dist	−0.002071	−0.0441	[−0.003471, −0.000711]		1.32 × 10−6	Localization trade-off

Note: Best values metric-wise in Table 2a are in bold. Pooled/global AUC (5 × 5) and original-resolution per-image AUC are different evaluation constructions. The four configurations used development-selected, configuration-specific training durations of two, seven, five, and six epochs; these official test contrasts are therefore not epoch-matched. In Table 2b, positive benefit denotes an AUC increase or an error reduction, depending on the metric. DCLD and DGQI are inference-time modules, while STMRP is a training-only objective added to BCE and does not change the DGQI + DCLD inference architecture.

4.3.1. Official Test Effects of DCLD, DGQI, and STMRP

The first controlled change adds the Distribution-Conditioned Localization Decoder (DCLD) while keeping the candidate evidence static. In this configuration, the learned DGQI query–scene interaction is absent: the decoder receives the fixed reference scene feature and heatmap together with a static coarse geometry map, while the context vector is neutral. This design isolates the downstream localization role of DCLD from candidate refinement.

As shown in Table 2a, DCLD with static geometry provides the clearest localization improvement over the geometry-guided reference. Dist decreases from 0.1297 to 0.1269, MDist from 0.0684 to 0.0667, Ang from 16.03° to 15.62°, and MAng from 7.32° to 7.00°. Pooled/global AUC also increases from 0.9179 to 0.9197, whereas original-resolution per-image AUC decreases from 0.9340 to 0.9327. The paired Dist effect in Table 2b corresponds to an error reduction of +0.002757 (95% bootstrap CI: [+0.001536, +0.004032]; Holm-adjusted p = 1.96 × 10⁻¹³). These results identify DCLD primarily as the localization-improving component of the framework.

The second change activates Distribution-Guided Query Interaction (DGQI) while retaining the same DCLD. DGQI replaces the static candidate input with a learned query–scene refinement of the geometry-biased candidate distribution and supplies the corresponding context representation to DCLD. Under this controlled comparison, original-resolution per-image AUC increases from 0.9327 to 0.9332. Table 2b reports a paired mean gain of +0.000515 (95% bootstrap CI: [+0.000348, +0.000675]; Holm-adjusted p = 1.31 × 10⁻⁹). The Dist effect is only +0.000061 in the favorable error-reduction direction, with a confidence interval crossing zero and an adjusted p = 0.129. DGQI therefore adds a statistically supported discrimination benefit without a statistically supported change in Dist. This incremental effect identifies the contribution of DGQI when coupled with DCLD; the present study does not establish the effect of DGQI as an independent localization pathway.

The final controlled change modifies the training objective while keeping the DGQI + DCLD inference architecture unchanged. Replacing BCE-only training with BCE + 0.10 STMRP increases original-resolution per-image AUC from 0.9332 to 0.9342, with a paired gain of +0.001010 (95% bootstrap CI: [+0.000741, +0.001280]; Holm-adjusted p = 2.31 × 10⁻⁴⁵). Dist increases from 0.1268 to 0.1289, corresponding to an error-reduction effect of −0.002071 (95% bootstrap CI: [−0.003471, −0.000711]; Holm-adjusted p = 1.32 × 10⁻⁶). MDist, Ang, MAng, and pooled/global AUC show the same non-monotonic tendency. Therefore, STMRP strengthens ranking-oriented discrimination at a measurable localization cost. The standardized paired effects are small despite low adjusted p-values, and the benefits are not uniform across instances. For example, STMRP improves original-resolution per-image AUC for 61.33% of the paired instances but reduces Dist for only 24.61%; full improved, unchanged, and degraded proportions for the principal contrasts are reported in Supplementary Table S2. The controlled results therefore establish metric-specific tendencies rather than broad, uniformly large gains.

4.3.2. Development Robustness and Architecture Control

The direction of the STMRP effect was consistent across three development seeds. Relative to BCE-only DGQI + DCLD, development-validation AUC increased for all three seeds, with a mean change of +0.000737 (sample SD = 0.000175), whereas Dist increased for all three seeds, with a mean change of +0.001224 (sample SD = 0.000334); Ang showed a mixed direction. With only three seeds, these results are treated as descriptive robustness evidence rather than as a standalone significance test. Seed-wise results are reported in Supplementary Table S3. An additional epoch-matched development comparison held the training duration at six epochs for both BCE-only and STMRP-regularized DGQI + DCLD across three random seeds. STMRP increased development-validation AUC by +0.001113, +0.001097, and +0.000869, respectively, while increasing Dist by +0.001551, +0.001368, and +0.001495. This consistent discrimination–localization trade-off at a fixed epoch (Supplementary Table S4) argues against an explanation based solely on different selected training durations, although it does not remove the unequal-epoch limitation of the four official test configurations.

A separate development-stage architecture control examined whether the reference model required a second injection of the full-18 geometry immediately before the heatmap decoder. The early-only and dual-injection variants produced nearly identical numerical validation results: early-only minus dual was −0.000067 for development-validation AUC, −0.000026 for Dist, and +0.028° for Ang. Supplementary Table S1 reports the full control. The final geometry-guided reference therefore retains the cleaner early-fusion pathway without decoder-side geometry re-injection; these small differences are reported as numerical control evidence and are not interpreted as statistical equivalence.

Development-stage sensitivity analyses further examined the radial candidate count, the STMRP loss weight, and the protected target neighborhood (Supplementary Tables S5–S7). Among the tested single-seed settings, 12 radial bins yielded the highest development-validation AUC; positive STMRP weights increased AUC relative to BCE-only training, with localization trade-offs; and varying the protected neighborhood between 1 × 1 and 5 × 5 produced small numerical changes. The weight 0.05 gave a slightly higher AUC and lower Dist than the original 0.10, so these comparisons do not establish a unique optimum. A separate ResNet-50/FPN bridge control did not reproduce the DCLD localization gain observed with the GCViT-T/FPN reference under the prespecified development-selection rule (Supplementary Table S8). These results delimit the robustness and representation dependence of the tested designs without changing the official test configurations.

4.4. Mechanism Analysis

As Section 4.3 has established the metric-wise effects of the controlled configurations, this section examines how these effects arise at distinct stages of the prediction pipeline. DCLD performs downstream correction of the geometry-guided response, DGQI refines the intermediate candidate representation and supplies scene-conditioned context to DCLD, and STMRP shapes selected non-local ranking relations during training. Table 3 summarizes the population-level diagnostics with the development-validation subset, while the accompanying official test cases provide spatial illustrations of the corresponding mechanisms.

Table 3. Development-stage mechanism diagnostics on the GazeFollow development-validation subset: (a) DCLD downstream-response diagnostics; (b) DGQI candidate-refinement diagnostics; and (c) STMRP selective-ranking diagnostics.

(a)
Diagnostic	Value
Learned DGQI context	Disabled (context vector = 0)
Candidate evidence	Static coarse geometry
Mean absolute decoder correction	0.2194
Mean maximum absolute decoder correction	1.3393
(b)
Diagnostic	Value
Coarse expected candidate-to-target distance	0.3393
DGQI-refined expected candidate-to-target distance	0.2527
Relative reduction	25.52%
Toward-target fraction	93.83%
Normalized candidate entropy	0.6331
Top-1 candidate probability	0.1120
Top-5 candidate probability mass	0.4127
KL divergence from coarse candidate distribution	0.4170
(c)
Diagnostic	Value
STMRP weight	0.10
Editable local neighborhood	3 × 3 (radius = 1)
Ranking loss	0.001823
Protected fraction	93.25%
Active-violation fraction	11.76%
Mean teacher margin	2.0555
Mean student margin	2.1186

Note: Table 3 reports explanatory development-stage diagnostics. The magnitude of a decoder correction is not itself a prediction-quality metric. The DGQI toward-target fraction compares refined and coarse expected candidate-to-target distances for each person instance. The STMRP protected fraction and active-violation fraction quantify different properties of the ranking objective and are not complementary proportions.

4.4.1. DCLD: Downstream Localization Correction

DCLD operates after the geometry-guided reference has produced scene features and an initial heatmap. Its input combines the reference scene representation, the current heatmap state, spatial candidate information, and an optional context vector; the decoder then predicts a residual correction that is added to the reference logits. The controlled configuration with static geometry isolates this decoding role because the learned DGQI is removed: both candidate inputs are supplied by the static coarse geometry, and the context vector is fixed to zero.

Table 3a confirms that DCLD remains active in the absence of learned DGQI information. At the selected development configuration, the mean absolute decoder correction is 0.2194 and the mean maximum absolute correction is 1.3393 despite the zero-context vector. These non-zero corrections confirm that the decoder can reshape the reference heatmap using scene features and static geometry alone, consistent with the localization gains observed in Section 4.3. A representative spatial correction is shown in Figure 2.

Figure 2. Example of downstream response correction produced by DCLD under static geometry. The panels show the input with ground-truth gaze information, the geometry-guided reference response, the DCLD response without learned DGQI, and the signed DCLD-induced response change. The gaze subject is marked in red, a green marker identifies the displayed target or prediction, and a yellow arrow links the gaze subject to that location. In the Input + GT column, the green marker denotes the mean of all valid human gaze annotations. Warm colors in the response-change panel denote an increased response while cool colors denote a decreased response. Prediction overlays are normalized for visualization and should not be compared by absolute color magnitude across panels. The input photograph is from the GazeFollow dataset [8].

Figure 2 illustrates targeted downstream redistribution rather than uniform amplification. Relative to the geometry-guided reference, the DCLD response places greater emphasis near the annotated target and shifts the peak closer to the human target. The signed response-change panel shows simultaneous enhancement and suppression across different regions, providing a spatial counterpart to the localization improvement measured in Table 2b.

4.4.2. DGQI: Candidate-Evidence Refinement

DGQI acts upstream of DCLD. Geometry-biased candidate queries interact with scene tokens, reweight the coarse candidate distribution, and produce both a refined candidate map and a posterior-weighted context representation. To measure this step independently of the final decoded heatmap, we compare the expected normalized candidate-to-target distance before and after DGQI refinement.

Table 3b shows a strong representation-level effect. The expected candidate-to-target distance decreases from 0.3393 for the coarse geometry-biased distribution to 0.2527 after DGQI refinement, a relative reduction of 25.52%. The refined representation is closer to the annotated target for 93.83% of the 12,569 development-validation instances. At the same time, it remains distributed: normalized candidate entropy is 0.6331, the top-one candidate probability is 0.1120, the top-five probability mass is 0.4127, and the KL divergence from the coarse distribution is 0.4170. DGQI therefore reallocates probability mass among plausible candidates rather than collapsing the geometry prior to a single point. Figure 3 provides a spatial example of this redistribution and the subsequent decoding.

Figure 3. Example of DGQI candidate refinement and subsequent decoding. From left to right, the panels show the input with ground-truth gaze information, the coarse geometry-biased candidate distribution, the DGQI-refined candidate distribution, the signed refinement change, and the final DGQI + DCLD heatmap. The gaze subject is marked in red, a green marker identifies the displayed target or prediction, and a yellow arrow links the gaze subject to that location. In the Input + GT columns, the green marker denotes the mean of all valid human gaze annotations. Warm colors in the change panel indicate increased candidate support while cool colors indicate decreased support. The candidate distributions are intermediate representations and should not be interpreted as final gaze predictions. The input photograph is from the GazeFollow dataset [8].

Figure 3 shows the intended sequence from candidate refinement to learned decoding. The coarse candidate distribution provides broad geometry-constrained support; DGQI increases relative support near the annotated target and suppresses part of the competing mass; the signed refinement map makes this redistribution explicit; and the final DGQI + DCLD heatmap shows how the refined representation is consumed by the decoder.

Representation-level refinement does not imply a proportional gain in the final heatmap. Additional sample-level coupling and context/correction diagnostics are reported in Supplementary Note S1; these analyses support interpreting DGQI candidate refinement separately from the magnitude of the final metric change. Although the candidate-level analysis characterizes DGQI refinement, its effect on the final gaze-target prediction is evaluated only together with DCLD; the independent contribution of DGQI is not identified by the present configuration set.

4.4.3. STMRP: Selective Non-Local Ranking Preservation

STMRP addresses a different stage of the pipeline and is active only during training. The geometry-guided reference heatmap acts as a detached teacher. For non-local locations, the loss protects only ranking relations in which the teacher places the ground-truth cell above a competing pixel; teacher-wrong relations are not distilled, and the 3 × 3 neighborhood centered on the ground-truth cell remains editable. Logits are standardized per image over the non-local region before the ranking margin is evaluated. The objective therefore preserves selected global ordering without forcing the student to reproduce teacher errors or the teacher’s local target shape.

Table 3c summarizes the selected development-stage behavior. The protected fraction is 93.25%, whereas the active-violation fraction is 11.76%; these quantities describe different properties of the objective. The ranking loss is 0.001823, with mean standardized teacher and student margins of 2.0555 and 2.1186, respectively. Together with the three-seed results reported in Section 4.3.2, these diagnostics show that STMRP applies selective non-local ranking pressure rather than global logit matching. A representative response change is shown in Figure 4.

Figure 4. Example of response reshaping induced by STMRP-regularized training. The panels show the input with ground-truth gaze information, the final DGQI + DCLD heatmap after BCE-only training, the final heatmap from the same inference architecture trained with BCE + STMRP, and the signed STMRP-induced response change. The gaze subject is marked in red, a green marker identifies the displayed target or prediction, and a yellow arrow links the gaze subject to that location. In the Input + GT columns, the green marker denotes the mean of all valid human gaze annotations. Warm colors denote an increased response while cool colors denote a decreased response. The comparison isolates a training-objective effect; STMRP introduces no additional inference branch. The input photograph is from the GazeFollow dataset [8].

Figure 4 compares the same DGQI + DCLD inference architecture after BCE-only and STMRP-regularized training. The signed response-change panel contains both positive and negative changes, demonstrating that STMRP reshapes the spatial ranking of the final response without adding an inference-time branch. In this selected trade-off case, the altered response does not improve peak localization, matching the official test pattern in which STMRP increases original-resolution per-image AUC while giving back part of the BCE-only localization advantage.

Taken together, the three mechanism visualizations establish a coherent division of roles. DCLD acts at the final decoding stage, DGQI acts upstream by redistributing candidate evidence and providing scene-conditioned context, and STMRP changes how the same DGQI + DCLD architecture is trained. This stage-wise interpretation explains why the strongest gains appear in different metrics rather than forming a uniformly monotonic ablation chain.

4.5. Performance Under Annotation Ambiguity and Reference Difficulty

Aggregate benchmark scores can conceal heterogeneous sample-level effects. The main text therefore focuses on two strata that are most directly related to target commitment: human-annotation ambiguity and geometry-guided reference difficulty. Sample-level benefit coverage and radial–lateral error decomposition are retained as supplementary diagnostics in Supplementary Figure S1 and Supplementary Figure S2, respectively. All analyses use the frozen predictions from the 4782 official GazeFollow test instances and are descriptive rather than model-selection procedures.

Figure 5 reports the stratified component effects by annotation ambiguity and geometry-guided reference difficulty. Annotation ambiguity was quantified by the dispersion of the valid human gaze annotations around their mean and divided into equal-size low-, medium-, and high-dispersion tertiles (1594 instances each). This statistic describes between-annotator dispersion in visually inferred gaze-target locations and not measured changes in the gaze subject’s eye movements. As shown in Figure 5a,b, DCLD retains a positive Dist reduction in all three strata (+0.0018, +0.0036, and +0.0029 from low to high ambiguity), whereas its per-image AUC effect is negative throughout, with its most negative effect in the high-ambiguity group (−0.0023). Thus, the localization advantage of DCLD extends to ambiguous scenes even when its discrimination effect follows a different direction.

Figure 5. Stratified component effects by annotation ambiguity and geometry-guided reference difficulty. (a) Per-image AUC effects across low-, medium-, and high-annotation-dispersion tertiles. (b) Dist effects across the same annotation-dispersion tertiles. (c) Per-image AUC effects across Q1 (easy) to Q4 (hard) reference Dist quartiles. (d) Dist effects across the same reference difficulty quartiles. Error bars denote 95% bootstrap confidence intervals.

DGQI shows the opposite tendency for per-image discrimination. Its mean AUC effect is nearly zero in the low-ambiguity tertile (+0.00006; confidence interval crossing zero), increases to +0.00060 in the medium tertile, and reaches +0.00088 in the high-ambiguity tertile. Its Dist effect does not follow a consistent ambiguity-dependent direction. STMRP yields a positive per-image AUC effect in all three ambiguity strata (+0.00077, +0.00053, and +0.00173), with the largest gain in the high-ambiguity group, whereas the corresponding Dist effect is negative on average in all three groups. Annotation ambiguity therefore modulates the discrimination-oriented effects of DGQI and STMRP more strongly than their peak-distance effects.

We next stratified samples by the geometry-guided reference Dist into four approximately equal-frequency difficulty groups (Q1 and Q4: 1196 instances each; Q2 and Q3: 1195 instances each), as shown in Figure 5c,d. The DCLD Dist effect changes from a negative error-reduction effect with the easiest quartile (−0.0044) to a large positive effect with the hardest quartile (+0.0126), indicating that its localization correction is concentrated on difficult reference cases. DGQI per-image AUC changes from slightly negative effects with the two easier quartiles to +0.00222 with the hardest quartile. STMRP per-image AUC is positive in every quartile and rises to +0.00236 with the hardest group, while its Dist effect changes from +0.00333 with the easiest quartile to −0.01293 with the hardest quartile. The hardest samples therefore expose both the largest discrimination gains and the largest localization trade-off.

These strata show that the three mechanisms respond systematically to target ambiguity and reference difficulty rather than contributing uniformly across all samples. Supplementary analyses further show that DGQI and STMRP have broader per-image AUC benefit coverage (Supplementary Figure S1), whereas the localization gain of DCLD is driven primarily by radial-error correction and DGQI/STMRP exhibit opposing radial–lateral changes (Supplementary Figure S2). Together, these findings help explain the non-monotonic, metric-specific pattern in Table 2a,b.

4.6. Cross-Dataset Generalization

To assess transfer beyond the training domain, the four finalized GazeFollow-trained configurations were evaluated without adaptation on VideoAttentionTarget (VAT) and ChildPlay. VAT provides the primary external evaluation using all 20,655 in-frame person-frames in the official test split. ChildPlay is reported as coverage-limited supplementary evidence because the locally available data cover 25 of the 45 official test clips. Neither external dataset contributed to training or model selection. Table 4 reports the zero-shot generalization on VAT.

Table 4. Zero-shot generalization on VideoAttentionTarget: (a) finalized-model performance on all 20,655 in-frame person-frames; and (b) cluster-aware effect estimates for the principal controlled contrasts.

(a)
Configuration	VAT AUC ↑		Dist ↓	Short-Side Dist ↓		Ang ↓
Geometry-guided reference	0.8859		0.1573	0.2266		19.08°
DCLD with static geometry	0.8924		0.1568	0.2249		19.09°
DGQI + DCLD (BCE)	0.8897		0.1562	0.2225		19.17°
Proposed DGQI + DCLD (BCE + STMRP)	0.8860		0.1565	0.2253		19.03°
(b)
Controlled Contrast	Metric	Mean Benefit	Cluster-Bootstrap 95% CI	Holm-Adjusted p	Interpretation
DCLD with static geometry vs. reference	VAT AUC	+0.006442	[+0.005138, +0.007806]	3.52 × 10−23	Supported transferable AUC gain
DGQI + DCLD vs. DCLD with static geometry	VAT AUC	−0.002621	[−0.003397, −0.001877]	4.70 × 10−10	External AUC decrease
STMRP-regularized vs. BCE-only DGQI + DCLD	VAT AUC	−0.003730	[−0.004907, −0.002557]	1.93 × 10−17	External AUC decrease
Proposed vs. reference	VAT AUC	+0.000091	[−0.000036, +0.000216]	3.24 × 10−4	Near-zero mean effect; inferential summaries disagree
Proposed vs. reference	Dist	+0.000797	[+0.000119, +0.001564]	3.24 × 10−4	Small localization benefit
Proposed vs. reference	Short-side Dist	+0.001326	[+0.000113, +0.002679]	1.93 × 10−3	Small localization benefit

Note: Best values metric-wise in Table 4a are in bold. All models were finalized before VAT evaluation. VAT AUC is the benchmark-specific per-person-frame tolerant-support AUC and is not interchangeable with either of GazeFollow AUC constructions. Positive benefits in Table 4b denotes a higher AUC or lower localization error. Cluster bootstrap and track-level rank tests summarize different estimands; when they disagree, both are retained.

4.6.1. Zero-Shot Transfer to VideoAttentionTarget

Table 4a reports the zero-shot VAT performance using the benchmark-specific per-person-frame tolerant-support AUC, benchmark distance, short-side-normalized distance, and angular error [24]. Because the present model predicts only an in-frame spatial heatmap, the evaluation is restricted to in-frame person-frames and does not include the out-of-frame average-precision task.

DCLD with static geometry shows the clearest transferable discrimination effect. VAT AUC increases from 0.8859 for the geometry-guided reference to 0.8924. The cluster-aware contrast in Table 4b gives a mean benefit of +0.006442 (95% cluster-bootstrap CI: [+0.005138, +0.007806]; Holm-adjusted p = 3.52 × 10⁻²³), with the candidate configuration improving 79.3% of individual person-frames. DCLD therefore provides a clear component-level AUC gain that transfers from GazeFollow to VAT without target-domain adaptation.

DGQI and STMRP are more domain-sensitive at the final-output level. Adding DGQI to DCLD reduces VAT AUC by 0.002621 on average (95% CI: [−0.003397, −0.001877]), and STMRP-regularized training further reduces VAT AUC by 0.003730 relative to BCE-only DGQI + DCLD (95% CI: [−0.004907, −0.002557]). Section 4.6.3 shows that substantial DGQI candidate refinement nevertheless persists on the external data, separating transfer of the intermediate mechanism from transfer of the final AUC gain.

At the end-to-end level, the mean VAT AUC effect of the proposed model relative to the reference is near zero (0.8860 versus 0.8859; mean difference = +0.000091). The cluster-bootstrap interval crosses zero, whereas the rank-based test is statistically supported; these inferential summaries are therefore reported as disagreement between estimands rather than as an equivalence or single-significance claim. In contrast, the proposed model obtains small, supported reductions in benchmark distance and short-side-normalized distance, with favorable error-reduction effects of +0.000797 and +0.001326, respectively. Angular error is also numerically lower (19.03° versus 19.08°), but the available inferential summaries do not support a strong angular-gain claim.

Recent large, pretrained approaches, including the frozen DINOv2-based Gaze-LLE variants, report stronger absolute zero-shot VAT performance [18]. Here, the external experiment serves a complementary purpose: it tests whether the controlled component effects remain visible when the GazeFollow-trained models are transferred without adaptation. Under this setting, DCLD provides the clearest transferable AUC benefit.

4.6.2. Coverage-Limited Zero-Shot Transfer to ChildPlay

ChildPlay [45] provides a socially distinct video domain with scene statistics that differ from GazeFollow. Because the locally available data cover only 25 of the 45 official test clips, the complete numerical results and clip-cluster inferences are reported in Supplementary Table S9 and kept separate from complete-benchmark rankings. Across the 14,332 inside-visible person-frames, DCLD again provides the most stable component-level AUC gain: AUC increases from 0.9493 to 0.9512, with a clip-bootstrap mean benefit of +0.001881 (95% CI: [+0.000746, +0.002498]; Holm-adjusted p = 0.0184).

DGQI does not provide a stable additional AUC gain on this subset, and the mean AUC effect of STMRP relative to BCE-only DGQI + DCLD is negative; the complete model remains close to the geometry-guided reference (mean AUC difference +0.000304). Because the clip-level estimands disagree for some contrasts, Supplementary Table S9 retains the full bootstrap and rank-based summaries. The main-text conclusion is therefore deliberately conservative: DCLD shows the most stable transferable final-output AUC effect, whereas DGQI and STMRP are more domain-sensitive.

4.6.3. Cross-Domain Component Behavior

The two external datasets reveal a consistent component-level pattern. DCLD is the most transferable source of final-output discrimination, with a supported AUC gain over the geometry-guided reference on both full in-frame VAT and the coverage-limited ChildPlay subset. DGQI remains mechanistically active under domain shift: on VAT, the expected candidate-to-target distance decreases from 0.3510 to 0.2610 for the BCE model and 88.18% of instances move toward the target; on ChildPlay, the corresponding distance decreases from 0.3195 to 0.2139 and 96.90% of instances move toward the target. The persistence of this candidate refinement alongside domain-sensitive final AUC effects shows that intermediate mechanism transfer and end-to-end metric transfer are distinct.

STMRP is similarly domain-sensitive. Its clearest GazeFollow advantage is the increase in original-resolution per-image AUC, whereas neither VAT nor ChildPlay shows an external AUC advantage over BCE-only DGQI + DCLD. This pattern is consistent with STMRP shaping a selective ranking objective whose benefit depends on how the target-domain response structure aligns with the reference ranking relations.

At the broader framework level, the proposed model advances the published CDDL–PGGM framework beyond GazeFollow under the same external evaluation lineage [19]. On VAT, AUC increases from 0.8823 to 0.8860, whereas benchmark distance, short-side distance, and angular error decrease from 0.1574 to 0.1565, 0.2264 to 0.2253, and 19.48° to 19.03°, respectively. On the same coverage-limited ChildPlay subset, AUC increases from 0.9362 to 0.9496, and all three localization measures also improve numerically. These cross-domain gains show that the present framework extends the published direction-to-region formulation beyond the in-domain benchmark, even though the incremental effects of DGQI and STMRP remain domain sensitive.

4.7. Computational Efficiency

The computational cost of the final configurations was measured on the same NVIDIA GeForce RTX 5090 D V2 GPU using a 224 × 224 FP32 synthetic-input inference protocol that isolates model-side computation from data and metric overhead. The main text reports batch-1 latency, batch-64 throughput, parameter count, and peak allocated GPU memory; the warm-up schedule, repeated measurement blocks, and excluded overheads are documented in Supplementary Method S1. Table 5 summarizes the results.

Table 5. Computational efficiency of the final configurations.

Configuration	Parameters (M)	Batch-1 Latency (ms/Sample) ↓	Batch-64 Throughput (Samples/s) ↑	Peak Allocated Memory, Batch-64 (MiB) ↓
Geometry-guided reference	55.470	31.95 (0.26)	738.07 (2.48)	2463.13
DCLD with static geometry	55.794	32.70 (0.33)	692.81 (2.23)	2466.87
DGQI + DCLD (BCE)	55.982	34.61 (0.27)	680.59 (1.52)	2467.58
Proposed DGQI + DCLD (BCE + STMRP)	55.982	34.78 (0.12)	681.00 (1.98)	2467.58

Note: Latency and throughput are reported as means (sample SD) over five measurement blocks after dedicated warm-up. Peak memory is CUDA maximum allocated memory during batch-64 forward inference. Full measurement details are provided in Supplementary Method S1. STMRP is training-only; the last two rows share the same inference architecture.

Table 5 shows that the proposed localization mechanisms add very little model size or peak memory, with the main cost appearing in runtime. Adding DCLD increases the parameter count from 55.470 M to 55.794 M (+0.58%) and raises batch-1 latency from 31.95 to 32.70 ms/sample (+2.36%). Batch-64 throughput decreases from 738.07 to 692.81 samples/s (−6.13%), whereas peak allocated memory increases by only 3.74 MiB (+0.15%).

Activating DGQI on top of DCLD increases the total parameter count to 55.982 M, only +0.92% over the reference. Batch-1 latency reaches 34.61 ms/sample (+8.34%), batch-64 throughput decreases to 680.59 samples/s (−7.79%), and peak memory remains nearly unchanged at 2467.58 MiB (+0.18%). The learned candidate-refinement pathway therefore adds measurable computation while preserving a compact parameter and memory footprint.

The BCE-only and STMRP-regularized DGQI + DCLD configurations have exactly the same inference architecture and parameter count. Their identical peak memories and nearly identical timings confirm that STMRP introduces no inference-time branch or parameter overhead; the 34.61 versus 34.78 ms/sample latency difference is consistent with measurement variability. Overall, the proposed model increases the parameter count by less than 1% and peak allocated memory by approximately 0.18% relative to the reference, with an approximately 9% increase in batch-1 latency and an approximately 8% decrease in batch-64 throughput.

Because the CDDL–PGGM baseline used a different ResNet-50/FPN heatmap network [19], Table 5 is used to isolate the incremental cost of DCLD and DGQI within the current GCViT-T/FPN architecture family rather than to compare cross-generation efficiency.

4.8. Qualitative Analysis and Failure Cases

The aggregate and stratified analyses characterize component-level behavior across the GazeFollow test set, while qualitative examples reveal how the final spatial response appears in individual scenes. We visualize five preselected official test instances using predictions from the finalized proposed model. The first three scenes are the same quantitatively selected cases used for the mechanism visualizations in Section 4.4; the remaining two represent successful target commitment under high annotation ambiguity and a representative failure. This reuse links the mechanism analysis to the final model behavior without changing the sample set for presentation.

4.8.1. Representative Final Predictions

Figure 6 presents each scene in three complementary views as input + GT, proposed prediction, and a human-annotation heatmap. Following the visualization convention used in CDDL–PGGM [19], the gaze subject is marked in red, a green marker identifies the displayed target or prediction, and a yellow arrow links the gaze subject to that location. In the input and human-annotation columns, the green marker denotes the mean of all valid human gaze annotations; in the proposed-prediction column, it denotes the predicted heatmap peak. Predicted heatmaps are overlaid continuously without hard response thresholding, preserving the full spatial response.

Figure 6. Representative final gaze-target predictions on GazeFollow. Rows (a–c) reuse the preselected scenes examined mechanistically in Section 4.4 and show only the final proposed output, row (d) is a successful case with high human-annotation ambiguity, and row (e) is a representative remaining failure. From left to right, each row shows the input with the mean human gaze annotation, the final proposed prediction, and a human-annotation heatmap constructed by averaging equal-weight Gaussian kernels centered at all valid human annotations. Red markers denote gaze subjects. Green markers denote the mean human annotation in the input and human-annotation columns and the predicted heatmap peak in the proposed-prediction column. Yellow arrows connect the gaze subject to the displayed location. Heatmaps are continuous overlays without hard thresholding and are independently normalized within each example; color magnitude should therefore not be compared across examples. The human-annotation heatmap is used for qualitative visualization only and is not an official benchmark ground-truth map. The input photographs are from the GazeFollow dataset [8].

The human-annotation heatmap in the third column visualizes annotation dispersion. Each valid gaze annotation is represented on the visualization grid using the same 21 × 21 Gaussian kernel (σ = 3) used for qualitative display in CDDL–PGGM, and the resulting maps are averaged with equal weight. Predicted and human-annotation heatmaps are independently normalized within each example, so color intensity represents within-panel spatial structure rather than an absolute confidence value that can be compared across examples.

Figure 6a–c revisit the three mechanism-analysis scenes using only the final proposed output, thereby linking the intermediate behaviors in Section 4.4 to the resulting end-to-end target commitment. Across these cases, the final heatmaps remain concentrated around the dominant human-annotation regions while preserving scene-specific response structure from the complete DGQI–DCLD pathway; the corresponding DCLD correction, DGQI candidate redistribution, and STMRP training effect are analyzed separately in Section 4.4.

4.8.2. Annotation Ambiguity and Remaining Failure Modes

Figure 6d,e expose contrasting outcomes under difficult visual evidence. In Figure 6d, the human-annotation heatmap is more spatially dispersed and contains weaker secondary support away from the dominant annotation region. The proposed prediction nevertheless concentrates on the principal annotated region. This example shows that the final model can form a decisive target commitment even when human annotations retain multiple plausible spatial modes, consistent with the stronger discrimination-oriented effects observed in higher-ambiguity strata in Section 4.5.

Figure 6e, by contrast, shows a clear remaining target-selection failure. The proposed heatmap forms a structured response on the broadly leftward side of the gaze subject, but its peak is displaced from the region in which the human annotations concentrate. The example highlights a central remaining challenge: geometry can restrict the plausible search region, yet correct semantic target selection is still difficult when multiple scene regions remain compatible with the estimated gaze direction.

Taken together, the qualitative results complete the evidence chain established in Section 4. DCLD, DGQI, and STMRP show distinct quantitative and mechanistic roles; the stratified analysis identifies where those roles are most visible; and the final examples show the resulting spatial behaviors in representative scenes. Successful cases remain concentrated and interpretable under both compact and ambiguous annotations, while the failure case exposes unresolved semantic target selection when several scene regions remain geometrically plausible.

5. Discussion

The experiments characterize scene-conditioned target commitment within an uncertainty-preserving geometry framework and show that its three mechanisms have distinct, partly opposing effects rather than uniformly additive gains. Relative to the published CDDL–PGGM baseline, the proposed model improves both retained GazeFollow AUC constructions and all four localization errors, while the controlled analysis assigns different functions to DCLD, DGQI, and STMRP. This distinction is important, because gaze following is simultaneously a distribution-ranking problem and a target-localization problem: a method must organize plausible spatial evidence while also committing to a target location. The results in Section 4 indicate that these objectives can be improved by different mechanisms and need not move monotonically together.

5.1. Stage-Wise Interpretation of Target Commitment

The CDDL–PGGM formulation focused on preserving directional uncertainty and projecting multiple plausible gaze hypotheses into image space [19]. The present study starts from that uncertainty-preserving probabilistic geometry and asks the next question: once a plausible directional region has been identified, how should the model decide where within that region to commit? Within the proposed architecture, the controlled comparisons distinguish downstream localization, scene-conditioned candidate refinement, and ranking-aware training as mechanisms with different and partly opposing effects.

DCLD provides the clearest evidence for the first role. On the official GazeFollow test set, adding DCLD with static geometry reduces Dist, MDist, Ang, and MAng relative to the geometry-guided reference, and the radial–lateral decomposition in Supplementary Figure S2 shows that most of the Dist gain comes from reduced radial error. The effect also grows on difficult reference cases, changing from a negative Dist effect in the easiest quartile to a +0.0126 error reduction in the hardest quartile. This pattern is consistent with a central limitation of direction-to-region modeling: estimating the correct angular sector does not by itself determine how far along that sector the attended target lies. DCLD appears to compensate for this residual distance uncertainty by learning a scene-conditioned correction after the geometric prior has constrained the search space.

DGQI addresses a complementary problem. Its strongest evidence is not a large shift in final peak distance, but a substantial reorganization of intermediate candidate evidence: expected candidate-to-target distance falls by 25.52%, and 93.83% of development-validation instances move toward the annotated target after refinement. The effect becomes more visible in discrimination-oriented metrics for ambiguous and difficult samples. This is compatible with the idea that several scene regions can remain geometrically plausible even after directional uncertainty has been modeled. Query–scene interactions can then use contextual evidence to redistribute probability among those candidates before DCLD performs the final spatial correction. Recent gaze-following systems likewise emphasize richer interaction modeling or large-scale visual encoders as a route to stronger scene understanding [16,17,18]. Our results add a geometry-conditioned perspective: within this architecture, scene interaction evaluates candidate alternatives after gaze geometry has structured their spatial support.

STMRP acts at a third level. It changes no inference-time representations, yet it increases the original-resolution per-image AUC and broadens the fraction of samples with a positive AUC change. The mechanism diagnostics show that the objective protects selected non-local teacher-consistent rankings while leaving the local target neighborhood editable. The resulting behavior suggests that target commitment is influenced not only by which features are available at inference, but also by how the model is trained to preserve or reshape the relative ordering of spatial responses. At the same time, the associated localization cost shows that global ranking preservation should not be interpreted as a universally beneficial constraint. Its value depends on the metric and on the structure of the target response that is being preserved. In the zero-shot VAT evaluation, STMRP did not provide an additional AUC advantage over the otherwise matched BCE-only configuration.

5.2. Complementary Discrimination and Localization Objectives

A second major finding is the persistent separation between discrimination-oriented AUC measures and peak-based localization errors. This is visible at several levels. DCLD improves localization while slightly reducing the original-resolution per-image AUC; DGQI improves per-image AUC with almost no net Dist change; and STMRP produces the clearest per-image AUC gain while giving back part of the localization advantage. The sample-level benefit-coverage analysis in Supplementary Figure S1 shows the same pattern: STMRP improves per-image AUC on 61.3% of test instances but improves Dist on only 24.6%, whereas many predictions retain an unchanged discrete peak.

These results are not contradictory. On the GazeFollow test set, multiple annotators estimate each depicted person’s gaze target; a predicted heatmap can improve its ranking across these spatially dispersed annotations without moving its maximum toward their centroid. Conversely, a peak can move closer to the mean target while the surrounding heatmap ranking becomes less favorable. This distinction is closely related to the broader observation that saliency-style AUC metrics depend on target construction and aggregation [46]. It also explains why the classical GazeFollow literature has historically combined AUC with distance-based measures rather than treating a single score as sufficient [8,49].

The practical implication is that gaze-following objectives should be evaluated according to the behavior they are intended to improve. If the goal is precise peak localization, Dist/MDist and angular errors remain essential. If the goal is to rank plausible target regions across an ambiguous scene, per-image AUC contains complementary information. The present dual-AUC reporting further shows that even two AUC constructions should not be assumed as interchangeable: pooled/global AUC emphasizes a test-set-level ranking after aggregation, whereas original-resolution per-image AUC evaluates each person instance independently. For this reason, the strongest interpretation of the proposed framework is metric-specific rather than based on a single scalar notion of superiority. For gaze-behavior research that uses predicted target locations, reporting both discrimination and localization measures can help distinguish a plausible target ranking from an accurate spatial estimate.

The same distinction also clarifies the DGQI mechanism results. On the development-validation subset, candidate-distance improvement is nearly uncorrelated with the net development-validation AUC gain from adding DGQI, despite clear population-level refinement of the candidate representation (Supplementary Note S1). Thus, an improved intermediate representation is neither equivalent to nor redundant with a better final heatmap. The nonlinear decoder can amplify, suppress, or redirect intermediate changes depending on scene context. Mechanism evaluation in gaze following should therefore examine both internal representation changes and final prediction behavior rather than using either one as a proxy for the other.

5.3. Generalization, Efficiency, and Practical Trade-Offs

The zero-shot experiments provide a third perspective on the proposed mechanisms. DCLD is the most transferable component: it yields a supported AUC gain over the geometry-guided reference on both complete in-frame VAT and the coverage-limited ChildPlay subset (Supplementary Table S9). This pattern supports the relative transferability of DCLD among the tested components, while the coverage-limited ChildPlay evaluation and differences between the two domains constrain broader generalization claims. In contrast, DGQI and STMRP are more domain-sensitive at the final-output level.

DGQI continues to shift candidate support toward annotated targets on both external datasets, even when its final AUC effect is unfavorable. This distinction indicates that downstream decoding may be sensitive to domain-specific scene appearance and candidate semantics. STMRP likewise depends on whether ranking relations learned on GazeFollow remain informative in the target domain.

At the full-framework level, the proposed model nevertheless improves numerically over the published CDDL–PGGM baseline on both external evaluations under the same evaluation lineage. On VAT, AUC rises from 0.8823 to 0.8860 and all three reported localization measures improve; on the same coverage-limited ChildPlay subset, AUC increases from 0.9362 to 0.9496 while the localization measures also improve numerically. These cross-generation improvements support the development of the overall framework, but they also incorporate reference-level changes and therefore do not isolate the effects of DGQI, DCLD, or STMRP. The corresponding within-family comparisons reveal the domain sensitivity of individual components [24,45].

The computational results indicate that this progression does not require a large increase in model size. Relative to the geometry-guided reference, the proposed configuration increases the parameter count by less than 1% and peak allocated memory by approximately 0.18%. The main cost appears in runtime, with roughly a 9% increase in batch-1 latency and an 8% reduction in batch-64 throughput. STMRP adds no inference-time parameters or branch. This profile is favorable for research settings where memory footprint and model size are constrained, although the runtime increase remains relevant for latency-sensitive applications. Stronger contemporary systems based on large frozen encoders achieve higher absolute benchmark scores [18], so future gains will need to balance richer scene semantics against computational cost rather than assuming that larger visual encoders are free improvements. Deployment suitability should be assessed under task-specific hardware and latency requirements.

5.4. Limitations and Future Directions

Several limitations remain. First, the present model predicts only an in-frame spatial heatmap and therefore does not address out-of-frame gaze targets. This restricts direct comparison with methods that jointly perform in-/out-of-frame classification and spatial localization, particularly on VAT. Extending the framework with an explicit out-of-frame decision branch would provide a more complete task formulation and enable evaluation under the full video benchmark protocol.

Second, the inherited CDDL branch and full-18 probabilistic geometry representation remain fixed during target-commitment learning. This design is useful for isolating the contribution of the downstream mechanisms, but it also prevents upstream correction when the directional prior is inaccurate. Joint calibration of directional uncertainty and target commitment may therefore provide additional gains, especially when head appearance is ambiguous or the true gaze direction lies near a direction-group boundary. Such an extension should preserve the uncertainty-aware character of the representation rather than collapsing the directional distribution prematurely.

Third, the end-to-end comparison with the published CDDL–PGGM framework is not a pure component ablation because the scene backbone also changes from ResNet-50/FPN to GCViT-T/FPN. The GCViT-T variant was selected as a capacity-controlled replacement with a comparable parameter scale, but the framework-level gains relative to this cross-generation baseline cannot be attributed exclusively to DGQI, DCLD, and STMRP. For this reason, component attribution in the present study is based on the controlled comparisons within the common GCViT-T/FPN reference family, where the reference representation is held fixed. A development-stage control attached the same static-geometry DCLD to a frozen ResNet-50/FPN reference. Under the predefined development-validation AUC selection rule, the selected model showed a small AUC increase (+0.000081), whereas Dist (+0.000245) and Ang (+0.063°) worsened slightly (Supplementary Table S8). Thus, the localization benefit observed with GCViT-T/FPN was not reproduced in this single-seed ResNet-50/FPN setting.

Fourth, the remaining qualitative failure indicates that geometry alone cannot determine semantic target identity when several objects occupy a plausible gaze region. Object-centric candidate representations, explicit person–object and object–object relations, and stronger semantic scene reasoning may help distinguish which geometrically compatible region is actually attended [14]. The present results suggest a natural extension in which probabilistic gaze geometry first restricts the hypothesis space, semantic and relational reasoning disambiguates target identity, and distance-aware localization determines the final spatial commitment.

Fifth, the external evaluation has two scope constraints. The ChildPlay analysis covers only 25 of the 45 official test clips and should therefore be interpreted as coverage-limited supplementary evidence rather than as a complete-benchmark claim. In addition, while VAT and ChildPlay are video datasets, they are evaluated frame-by-frame because the present study intentionally focuses on single-frame gaze following. Temporal consistency, motion cues, and cross-frame target persistence are consequently not modeled. Incorporating temporal evidence is a complementary direction that may be particularly useful when individual frames remain visually ambiguous.

Finally, the domain sensitivity of DGQI and STMRP motivates more adaptive calibration. Development sensitivity controls (Supplementary Tables S5–S7) support R = 12 among the tested radial grids, show a slightly more favorable AUC–Dist trade-off at λ = 0.05 than at the fixed λ = 0.10, and show limited variation across the tested target neighborhoods. These single-seed results are descriptive rather than evidence of a unique optimum. Future refinements could condition candidate interactions on scene uncertainty and weight ranking constraints by teacher reliability or target-domain statistics. Stronger pretrained encoders may improve object-relation modeling, but their capacity and pretraining effects should remain distinct from the contributions of the proposed modules [16,17,18].

Overall, the limitations reinforce the central interpretation of the present study. Uncertainty-preserving probabilistic gaze geometry is effective for structuring and narrowing plausible spatial hypotheses, but scene-conditioned target commitment remains sensitive to scene semantics, upstream directional quality, training-time ranking behavior, and domain shift. The stage-wise separation among geometry, candidate interaction, residual localization, and ranking preservation provides a controlled basis for addressing these remaining sources of uncertainty in future gaze-following models.

For researchers studying social attention and joint attention, these results also clarify how image-based gaze predictions should be interpreted. The intermediate candidate representation reveals how the model redistributes support among plausible locations, whereas the divergence between discrimination-oriented AUC and peak-based localization errors shows that candidate ranking and spatial accuracy are complementary outcomes. Because GazeFollow targets are annotated judgments rather than eye-tracker recordings, these predictions support computational analysis of likely attentional targets but do not measure fixation timing or oculomotor dynamics. Applying them to behavioral or developmental research therefore requires task-appropriate validation beyond the present image-based benchmarks.

6. Conclusions

This study extends uncertainty-aware gaze following from probabilistic direction-to-region modeling to scene-conditioned gaze-target localization in static images. Building on the inherited CDDL–PGGM representation, the proposed framework preserves multiple plausible geometric hypotheses while using scene evidence to refine candidate support and the final spatial prediction. DGQI updates direction–distance candidates through scene interaction, DCLD applies residual corrections to a geometry-guided reference heatmap, and STMRP selectively preserves correct spatial rankings during training without introducing an inference-time branch. Together, these components provide a structured and experimentally testable approach to scene-conditioned target selection.

On the GazeFollow official test set, the final model improves pooled/global AUC from 0.9066 to 0.9190, original-resolution per-image AUC from 0.9214 to 0.9342, and Dist from 0.1423 to 0.1289 relative to the published CDDL–PGGM system; MDist, Ang, and MAng also improve. These cross-generation gains include changes to the geometry-guided reference network and should not be attributed exclusively to the new modules. The controlled comparisons within a shared GCViT-T/FPN reference instead identify their distinct roles: DCLD provides the clearest localization improvement, DGQI contributes scene-conditioned candidate refinement and a smaller gain in per-image discrimination, and STMRP further improves discrimination while partly sacrificing localization accuracy. The divergence between the ranking-based AUC and peak-based localization measures is therefore an important evaluation finding, not merely a side effect of architectural design.

Zero-shot evaluation on VideoAttentionTarget and the available ChildPlay subset further shows that the component benefits are not uniformly preserved across domains; the stronger transfer of DCLD contrasts with the more domain-sensitive behaviors of DGQI and STMRP. The added components require fewer than 1% additional parameters relative to the common reference, with a measurable but moderate runtime overhead. Overall, the results demonstrate the value of retaining geometric alternatives until scene evidence can discriminate among candidate targets, while showing why prediction quality should be assessed using both distribution-ranking and localization measures. Future work can extend this framework with stronger object relation reasoning, out-of-frame decisions, and temporal evidence without conflating those extensions with the contributions measured here.