Foundation Model-Based One-Shot Anatomical Landmark Detection with Mamba and Graph Refinement

Abstract

Accurate anatomical landmark detection is important for orthodontic analysis, surgical planning, and morphometric measurement, but fully supervised methods usually require large expert-annotated datasets. This work studies a one-shot setting, where only a single annotated template image is used for training. We propose a foundation-model-based landmark detection framework using a frozen DINO Vision Transformer (ViT) backbone. The proposed framework integrates three complementary components: a Multi-Layer Multi-Facet (MLMF) module that adaptively fuses key and value features from multiple ViT layers through global source-wise reweighting; a Mamba-Based Long-Range Context Aggregation (MLCA) module that injects global anatomical context into fused patch descriptors with linear complexity; and a Topology-Constrained Graph Refinement (TCGR) module that refines the predicted landmark configuration using anatomical graph constraints. Experiments on the Cephalometric dataset and the Hand X-ray dataset demonstrate that the proposed method achieves strong performance. Overall, the results show that jointly exploiting multi-source foundation-model representations, efficient long-range context aggregation, and topology-aware refinement improves annotation-efficient anatomical landmark detection.

1. Introduction

The precise localization of anatomical landmarks in medical images is a fundamental prerequisite for a broad range of clinical and research workflows. In orthodontics and maxillofacial surgery, cephalometric landmark detection on lateral skull radiographs is of particular clinical significance. Accurate identification of key skeletal reference points, such as the sella turcica, nasion, anterior nasal spine, and menton, defines craniofacial skeletal relationships that directly inform treatment planning, surgical simulation, and longitudinal monitoring of craniofacial growth [1]. Beyond craniofacial analysis, landmark localization is equally indispensable in orthopedic imaging, where bone landmarks inform implant sizing, joint alignment correction, and deformity quantification [2], and in neuroimaging, where cortical and subcortical reference points underpin atlas-based segmentation, inter-subject registration, and population-level morphometric studies [3]. Accurate and reproducible landmark localization therefore constitutes a shared cornerstone across these domains, connecting clinical decision-making to the downstream computational pipelines of modern medical imaging.

A fundamental constraint shared by all fully supervised landmark detection paradigms is their dependence on large, meticulously expert-annotated training datasets. Expert annotation of anatomical landmarks is inherently labor-intensive and domain-specific, demanding substantial radiological expertise and exhibiting non-trivial inter- and intra-observer variability [1]. The annotation burden increases with the number of landmarks, image volume, and anatomical complexity. As a result, adapting a supervised model to a new anatomical region, imaging modality, or acquisition protocol often requires complete re-annotation and retraining, which is practically infeasible in many data-scarce clinical settings. One-shot learning addresses this bottleneck by localizing landmarks from a single annotated template image and generalizing to unannotated query images through feature correspondence [4]. In this formulation, a descriptor is extracted for each template landmark and the corresponding query position can be identified by similarity search in the query feature map. Recent methods extend this basic matching paradigm with learned feature adaptation, coarse-to-fine localization, spatial consistency filtering, or structural refinement. These developments motivate a closer examination of which representation, context, and structural cues remain underused in template-based landmark detection pipelines.

The past decade has witnessed substantial progress in deep learning for anatomical landmark detection. Convolutional Neural Network (CNN) architectures employing heatmap regression [2,5] or direct coordinate regression [6] have achieved strong accuracy on established benchmarks, with spatial configuration modules, multi-scale feature pyramids, and attention mechanisms further improving performance for spatially ambiguous or highly variable landmarks. More recently, Transformer-based architectures [7] have demonstrated the capacity to model long-range anatomical context via global self-attention, yielding competitive results across multiple anatomical regions and imaging modalities. In parallel, the emergence of Vision Foundation Models (VFMs)—particularly DINO [8] and DINOv2 [9], which train Vision Transformers (ViTs) [10] via self-supervised knowledge distillation—has fundamentally raised the quality ceiling for visual representations available to downstream tasks without any task-specific supervision. DINO ViT features exhibit emergent spatial correspondence properties wherein matching image patches across semantically related images produce highly similar feature vectors [8,11]—a property that directly benefits dense matching and landmark localization. Prior medical landmark detection frameworks have leveraged frozen DINO ViT backbones and demonstrated substantial improvements over traditional hand-crafted and CNN-based descriptors [4].

However, a critical limitation pervades existing approaches: they extract features from only a single intermediate layer and a single attention facet (typically the key projection) of the DINO ViT [12], fundamentally underutilizing the rich internal representational hierarchy of the backbone. Deep ViTs encode different types of information across network depths. Shallow layers capture local texture and edge patterns needed to distinguish anatomically adjacent structures, intermediate layers combine local appearance with regional structural context, and deep layers encode holistic semantic and geometric configurations of the anatomical assembly [8,11]. Meanwhile, the key and value projections within each Transformer block provide complementary information: keys encode geometric attention-routing patterns, whereas values carry the semantic content aggregated into output representations [13]. Furthermore, while ViT self-attention captures global context within the backbone, the fused feature map used for downstream matching is processed independently at each spatial position, entirely disregarding the long-range spatial dependencies among patch tokens that span the full anatomical field—dependencies crucial for resolving landmark ambiguities in structurally repetitive or low-contrast regions. Beyond feature extraction, two additional limitations persist: coarse-to-fine matching pipelines in prior work share feature parameters across the global and local branches despite their fundamentally different spatial scale and contextual requirements, and landmark coordinates are predicted independently for each point, entirely ignoring the strong topological constraints imposed by skeletal geometry and tissue connectivity.

To address these limitations, we propose an integration-oriented anatomical landmark detection framework. We do not claim DINO feature matching, Mamba-based sequence modeling, or graph refinement as isolated new techniques. Instead, the main contribution of this work lies in adapting and combining these complementary ideas for the single-template setting. In this setting, frozen DINO features provide strong visual correspondences, the Multi-Layer Multi-Facet (MLMF) module exploits the internal multi-layer and multi-facet representation hierarchy of the foundation model, the Mamba-Based Long-Range Context Aggregation (MLCA) module contextualizes fused patch descriptors through efficient Mamba-based long-range aggregation, and the Topology-Constrained Graph Refinement (TCGR) module refines the final landmark configuration with anatomical topology constraints.

The main contributions of this work are summarized as follows:

• We propose an anatomical landmark detection framework that uses a single labeled template image without requiring additional unlabeled training images.
• We design MLMF to adaptively fuse multi-layer and multi-facet DINO features, allowing the model to exploit complementary key and value projections from different ViT depths.
• We introduce MLCA to inject long-range anatomical context into fused patch descriptors using efficient Mamba-based bidirectional sequence modeling.
• We incorporate TCGR to refine independently matched landmarks through topology-constrained graph reasoning.
• We conduct extensive experiments, ablation studies, foundation-model comparisons, statistical testing, graph-design sensitivity analysis, and qualitative visualization to validate the effectiveness and limitations of the proposed framework.

2. Related Work

This section reviews the main research directions related to the proposed framework. We first summarize anatomical landmark detection methods, especially annotation-efficient landmark localization. We then review vision foundation models and their use as dense visual descriptors, followed by state space models for long-range context aggregation and graph-based structural reasoning for landmark refinement.

2.1. Anatomical Landmark Detection

Classical approaches to anatomical landmark detection, including active shape models [14] and constrained local models, encode statistical shape variation with handcrafted appearance features. Deep CNN architectures subsequently transformed the field: heatmap regression [2,5], coordinate regression [6], and multi-task frameworks [3] all achieved strong benchmark performance, while Transformer-based models [7] further introduced global self-attention for long-range anatomical context. All fully supervised methods, however, require large expert-annotated corpora, limiting their applicability in data-scarce settings.

One-shot methods address this constraint by localizing landmarks from a single annotated template. A feature descriptor is extracted at each template landmark position and matched to the query image via nearest-neighbor search in the query feature map. Siamese networks with metric learning objectives [15,16] are commonly adopted for descriptor training. Coarse-to-fine pipelines first identify a candidate region globally, then refine the prediction locally, while template augmentation—affine transforms, intensity jitter, and elastic deformations—mitigates overfitting to the single exemplar [4].

2.2. Vision Foundation Models

Vision Foundation Models, including DINO [8], DINOv2 [9], the Segment Anything Model (SAM) [17], and Masked Autoencoders (MAE) [18], have established new paradigms for visual representation learning, demonstrating remarkable generalization across diverse downstream tasks without task-specific fine-tuning. DINO trains Vision Transformers via a student-teacher self-distillation objective, producing spatially coherent, semantically organized representations in which explicit object localization emerges in the final-layer self-attention maps without any segmentation supervision [8]. The internal representations of DINO ViTs exhibit pronounced depth-dependent structure: shallow layers encode local texture and edge patterns, while deep layers encode high-level semantic and structural abstractions [11]. Within each Transformer block, the key and value projections carry functionally distinct information—keys encode geometric attention routing patterns while values carry the aggregated semantic content of attended tokens [13]—a complementarity that has been exploited in dense matching [19] and multi-layer feature stacking for keypoint localization on natural images [20]. Efficient and feature-guided representation learning has also been explored in broader medical and visual computing contexts, including efficient endoscopic image enhancement, multimodal feature-guided visual generation, and VFM-feature repacking for efficient diffusion transformers [21,22,23]. In particular, RePack then Refine observes that raw VFM features can be high-dimensional and redundant, and introduces a compact feature repacking stage followed by a refinement stage [23]. Although its task setting differs from anatomical landmark detection, its motivation is related to ours: both works aim to make frozen VFM representations more suitable for downstream task-specific pipelines. These studies further support the importance of compact and task-adaptive feature representation.

Recent studies have started to adapt visual foundation models to annotation-efficient medical landmark detection [24,25]. Foundation Model-Enabled One-Shot Detection of Anatomical Landmarks (FM-OSD) is the most directly related work [12], as it formulates anatomical landmark detection as a foundation-model-enabled one-shot matching problem and uses only a single labeled template image without additional unlabeled data. It employs a frozen visual foundation model as the feature extractor, introduces global and local feature decoders to increase the spatial resolution of extracted features, and uses bidirectional matching to improve robustness when similarity maps are scattered.

2.3. State Space Models and Mamba in Visual Representation Learning

Structured State Space Models (SSMs) [26] offer linear-complexity sequence modeling as an alternative to the quadratic self-attention of Transformers. Mamba [27] introduced input-dependent selective state transitions with a hardware-efficient scan, enabling content-aware long-range propagation while suppressing irrelevant tokens. VMamba [28] and Vision Mamba [29] adapted these ideas to 2D feature maps via cross-scan and bidirectional traversal strategies, achieving competitive results on recognition and dense prediction tasks. In medical imaging, SegMamba [30] and VM-UNet [31] incorporated Mamba blocks into hierarchical encoders to propagate long-range anatomical context across large-field-of-view scans with low memory overhead. We build on this line of work by inserting a Mamba block after the MLMF fusion step, contextualizing each fused patch descriptor with global anatomical scene information at $\mathcal{O}\left(L\right)$ cost before matching.

2.4. Graph-Based Structural Reasoning for Landmarks

Graph convolutional networks (GCNs) and Graph Attention Networks (GATs) [32] have been applied effectively to structured keypoint prediction tasks. In human pose estimation, graph-based models propagate structural context between joint nodes to correct individual estimates, with semantic graph convolutional networks [33] and skeleton-aware attention models [34] demonstrating significant improvements over independent joint estimation. In craniofacial and dental analysis, graph neural networks have been employed to model inter-landmark dependencies for joint detection [35], and in hand and bone age analysis, GCNs exploit the structured arrangement of phalanges and carpal bones to enforce physiologically plausible spatial configurations [36]. The predominant paradigm integrates graph reasoning as an end-to-end component jointly trained with the detection backbone, requiring gradient backpropagation through both the graph module and the feature extractor; this tight coupling limits applicability to settings where the full pipeline is differentiable and re-trainable.

3. Methodology

3.1. Problem Formulation

We address anatomical landmark detection in a setting where a single template image $I_t\in {\mathbb{R}}^{H\times W\times 3}$ is annotated with N landmark coordinates ${\left\{p_i^t\right\}}_{i=1}^N$, and the goal is to predict the corresponding set ${\left\{p_i^q\right\}}_{i=1}^N$ in an unannotated query image $I_q\in {\mathbb{R}}^{H\times W\times 3}$. No additional labeled or unlabeled target-domain images are used for training or adaptation. Following established protocols [4], the single template image is offline-augmented to generate optimization instances derived solely from that template prior to deployment. At inference time, each input pair consists of the annotated template image and an unannotated query image.

The proposed framework is built upon a frozen DINO ViT-S/8 backbone, whose weights remain fixed throughout training and inference. For each input image, we extract key and value projection features from multiple ViT layers, yielding six complementary feature sources from local texture to global structural semantics. MLMF first projects and adaptively weights these sources to produce a compact descriptor map. MLCA then serializes the fused map into forward and backward token sequences, processes them with a shared Mamba block, and concatenates the bidirectional outputs to obtain context-enriched features. The global decoder performs full-image matching to obtain coarse landmark positions, while the local decoder refines them within cropped neighborhoods using an independent MLMF–MLCA pipeline. Finally, TCGR jointly refines all predicted coordinates with anatomical topology constraints. An overview of the full framework is shown in Figure 1.

3.2. Multi-Layer Multi-Facet Adaptive Fusion (MLMF)

For each input image, the frozen DINO ViT-S/8 backbone produces patch-level features from multiple Transformer layers and attention facets. We extract key and value projections from Layers 5, 8, and 11, forming six feature sources in Equation (1):

$${\left\{{\mathbf{F}}_i\right\}}_{i=1}^6,{\mathbf{F}}_i\in {\mathbb{R}}^{B\times C_{\mathrm{in}}\times H_p\times W_p},C_{\mathrm{in}}=384.$$

(1)

Here, $H_p\times W_p$ denotes the patch grid. These features provide complementary information across network depth and attention facets. A direct channel-wise concatenation would produce a $6C_{\mathrm{in}}=2304$-channel feature map at each spatial location, which is memory inefficient and may mix redundant or conflicting cues.

To address this, MLMF first maps each source into a shared compact space. Each ${\mathbf{F}}_i$ is independently projected by a $1\times 1$ convolution, followed by batch normalization and a rectified linear unit (ReLU), as in Equation (2):

$${\widehat{\mathbf{F}}}_i={\varphi }_i\left({\mathbf{F}}_i\right),{\widehat{\mathbf{F}}}_i\in {\mathbb{R}}^{B\times C_{\mathrm{out}}\times H_p\times W_p},C_{\mathrm{out}}=256.$$

(2)

Each projected feature map is then globally average-pooled to obtain a source descriptor ${\overline{\mathbf{f}}}_i=\mathrm{GAP}\left({\widehat{\mathbf{F}}}_i\right)$, where GAP denotes global average pooling. The six descriptors are concatenated and passed through a two-layer multi-layer perceptron (MLP) with weights ${\mathbf{W}}_1$ and ${\mathbf{W}}_2$. This operation is a global source-wise reweighting mechanism, similar in spirit to squeeze-and-excitation style feature calibration. It produces one scalar importance weight for each DINO feature source, rather than token-level query-key-value attention. The fusion weights are computed in Equation (3):

$$\mathbf{w}=\mathrm{Softmax}\left(\mathbf{s}\odot {\mathbf{W}}_2\sigma \left({\mathbf{W}}_1[{\overline{\mathbf{f}}}_1;\dots ;{\overline{\mathbf{f}}}_6]\right)\right),$$

(3)

where $\sigma$ denotes ReLU, $\mathbf{s}\in {\mathbb{R}}^6$ is a learnable per-source scale vector initialized to ones. Specifically, ${\mathbf{W}}_1\in {\mathbb{R}}^{{\displaystyle \frac{6C_{\mathrm{out}}}{r}}\times 6C_{\mathrm{out}}}$ maps the concatenated source descriptor to a bottleneck hidden representation, and ${\mathbf{W}}_2\in {\mathbb{R}}^{6\times {\displaystyle \frac{6C_{\mathrm{out}}}{r}}}$ maps it to six source logits before Softmax normalization. The final fused descriptor map is defined by Equation (4):

$${\mathbf{F}}_{\mathrm{fused}}={\displaystyle \sum _{i=1}^6}{w}_i{\widehat{\mathbf{F}}}_i.$$

(4)

This design keeps fusion lightweight, preserves the contribution of each source, and allows the network to adaptively select useful DINO feature groups for landmark matching. Since the weights are computed at the source level, MLMF should be interpreted as adaptive source reweighting rather than spatial self-attention.

3.3. Mamba-Based Long-Range Context Aggregation (MLCA)

Although the MLMF module produces spatially rich descriptor maps by integrating multi-layer, multi-facet ViT features, each fused patch token is computed independently without explicit interaction across the spatial extent of the descriptor map. For anatomical landmark matching, this is a limiting factor: landmarks embedded in structurally ambiguous or low-contrast regions—such as the sella turcica or the condylar process in lateral cephalographs—can only be reliably localized if their local descriptor is contextualized by the broader anatomical configuration of the image. Standard self-attention would resolve this at $\mathcal{O}\left(L^2\right)$ cost (where $L=H_p\times W_p$ is the number of patch tokens), which is prohibitive for high-resolution scans. The MLCA module addresses this by applying a bidirectional selective state space model (the Mamba architecture [27]) to the serialized patch-token sequence, propagating global context at $\mathcal{O}\left(L\right)$ complexity while retaining the hardware-efficient scan of the original Mamba formulation. The architecture of the MLCA module is illustrated in Figure 2.

The fused map ${\mathbf{F}}_{\mathrm{fused}}\in {\mathbb{R}}^{B\times C_{\mathrm{out}}\times H_p\times W_p}$ is reshaped into a token sequence of length $L=H_p\times W_p$. We denote the forward row-major sequence as ${\mathbf{Z}}^f=[{\mathbf{z}}_1,\dots ,{\mathbf{z}}_L]$, where ${\mathbf{z}}_t\in {\mathbb{R}}^{C_{\mathrm{out}}}$. The backward sequence ${\mathbf{Z}}^b$ is obtained by reversing the same tokens. The two sequences are scanned in forward and backward directions to capture bidirectional spatial dependencies.

Each directional sequence is processed by a shared Mamba block with the selective SSM recurrence in Equation (5):

$${\mathbf{h}}_t^{\left(d\right)}={\overline{\mathbf{A}}}_t^{\left(d\right)}{\mathbf{h}}_{t-1}^{\left(d\right)}+{\overline{\mathbf{B}}}_t^{\left(d\right)}{\mathbf{z}}_t^{\left(d\right)},{\mathbf{y}}_t^{\left(d\right)}={\mathbf{C}}_t^{\left(d\right)}{\mathbf{h}}_t^{\left(d\right)},d\in \{f,b\}.$$

(5)

Here, $d=f$ and $d=b$ denote the forward and backward scan directions, respectively. ${\mathbf{z}}_t^{\left(d\right)}$, ${\mathbf{h}}_t^{\left(d\right)}$, and ${\mathbf{y}}_t^{\left(d\right)}$ are the input token, hidden state, and output token at step t under direction d. The state-transition matrices ${\overline{\mathbf{A}}}_t^{\left(d\right)}$, ${\overline{\mathbf{B}}}_t^{\left(d\right)}$, and ${\mathbf{C}}_t^{\left(d\right)}$ are input-dependent, enabling the model to selectively retain anatomically relevant tokens while suppressing uninformative background patches. The backward output is reversed back to the original order and concatenated with the forward output.

Let ${\mathbf{Z}}^{\mathrm{out}}\in {\mathbb{R}}^{B\times C_{\mathrm{out}}\times H_p\times W_p}$ denote the concatenated bidirectional output after projection back to $C_{\mathrm{out}}$ channels and reshaping to the patch grid. This output is injected into the original fused map via the learnable gated residual in Equation (6):

$${\mathbf{F}}_{\mathrm{ctx}}={\mathbf{F}}_{\mathrm{fused}}+\gamma \odot {\mathbf{Z}}^{\mathrm{out}},$$

(6)

where $\gamma \in {\mathbb{R}}^{C_{\mathrm{out}}}$ is initialized to zero, ensuring MLCA starts as an identity and gradually learns to inject contextual corrections. This initialization improves training stability because the randomly initialized Mamba branch does not perturb the fused DINO descriptors at the beginning of training. The model can therefore first rely on the frozen foundation-model features and then progressively learn useful long-range contextual updates. Layer normalization is applied before the Mamba block and after the residual addition.

The MLCA module is not inherently restricted to a fixed input resolution. For an input image resized to a patch grid of $H_p\times W_p$, MLCA operates on a sequence of length $L=H_p{W}_p$. Since the selective scan in Mamba has linear complexity in L, the same module can process different sequence lengths as long as the input resolution is compatible with the ViT patch size. In this work, we use a fixed input resolution for batching efficiency and fair comparison, but the formulation itself remains adaptive to other image sizes.

3.4. Two-Stage Global-to-Local Coarse-to-Fine Matching

The framework uses two independent branches, a global branch and a local branch, each with its own MLMF–MLCA pipeline. Both branches take the context-enhanced feature ${\mathbf{F}}_{\mathrm{ctx}}$ from MLCA as input.

During training, only the raw template image or its augmented views is processed by the global and local branches under landmark heatmap supervision. The frozen ViT backbone is not updated, while the MLMF, MLCA, and upsampling heads are optimized. Both branches are trained with heatmap mean squared error (MSE) loss [2,5,7]. Template augmentation (jitter, scaling, rotation, flipping, elastic deformation) is applied only to the single annotated template.

During inference, the annotated template image and the query image are first processed by the global branch. We denote the global descriptor maps of the template and query images as ${\mathbf{D}}_t^g$ and ${\mathbf{D}}_q^g$, respectively. For the ith landmark, the template descriptor is sampled from the annotated template location:

$${\mathbf{d}}_i^{t,g}={\mathbf{D}}_t^g\left[{\mathbf{p}}_i^t\right].$$

It is then matched against all candidate positions $\mathbf{p}\in \Omega$ in the query descriptor map, where $\Omega$ denotes the global search space. The coarse prediction is obtained by the cosine-similarity matching in Equation (7):

$${\widehat{\mathbf{p}}}_i^{\mathrm{coarse}}=\arg \underset{\mathbf{p}\in \Omega }{\max }\mathrm{sim}\left({\mathbf{d}}_i^{t,g},{\mathbf{D}}_q^g\left[\mathbf{p}\right]\right),$$

(7)

where $\mathrm{sim}(·,·)$ denotes cosine similarity. In implementation, top-K spatial consistency filtering is applied to reduce isolated false matches. For each landmark i, we first extract the top K local maxima from the global similarity map after non-maximum suppression, forming a candidate set ${\mathcal{C}}_i={\left\{{\mathbf{c}}_{i,m}\right\}}_{m=1}^K$. Let ${\widehat{\mathbf{p}}}_j^0$ denote the raw top-1 coarse prediction of landmark j, and let ${\mathcal{N}}_i$ denote the anatomical neighbors of landmark i in the TCGR graph. Each candidate is re-scored by combining its cosine similarity with the spatial consistency term in Equation (8):

$$S\left({\mathbf{c}}_{i,m}\right)=\mathrm{sim}\left({\mathbf{d}}_i^{t,g},{\mathbf{D}}_q^g\left[{\mathbf{c}}_{i,m}\right]\right)-\eta {\displaystyle \frac{1}{|{\mathcal{N}}_i|}}\sum _{j\in {\mathcal{N}}_i}{\displaystyle \frac{\left|\parallel {\mathbf{c}}_{i,m}-{\widehat{\mathbf{p}}}_j^0{\parallel }_2-{\parallel {\mathbf{p}}_i^t-{\mathbf{p}}_j^t\parallel }_2\right|}{\parallel {\mathbf{p}}_i^t-{\mathbf{p}}_j^t{\parallel }_2+ϵ}},$$

(8)

where $\eta$ controls the strength of the spatial penalty and $ϵ$ is a small constant for numerical stability. The final coarse prediction is selected as in Equation (9):

$${\widehat{\mathbf{p}}}_i^{\mathrm{coarse}}=\arg \underset{{\mathbf{c}}_{i,m}\in {\mathcal{C}}_i}{\max }S\left({\mathbf{c}}_{i,m}\right).$$

(9)

Unless otherwise specified, we set $K=10$, $\eta =0.1$, and $ϵ={10}^{-6}$. This filtering is used only in the global branch before local refinement. The local branch refines the coarse prediction within a cropped neighborhood. For each coarse prediction, fixed-size local patches are cropped from the template and query images and processed by an independent local MLMF–MLCA pipeline. The local upsampling head produces local descriptor maps, and we denote the local query descriptor map as ${\mathbf{D}}_q^l$. The corresponding local template descriptor is denoted as ${\mathbf{d}}_i^{t,l}$. Instead of searching the whole image again, the local branch searches only within a neighborhood $\mathcal{N}$ around the coarse location, as shown in Equation (10):

$${\widehat{\mathbf{p}}}_i^{\mathrm{fine}}={\widehat{\mathbf{p}}}_i^{\mathrm{coarse}}+\arg \underset{\delta \in \mathcal{N}}{\max }\mathrm{sim}\left({\mathbf{d}}_i^{t,l},{\mathbf{D}}_q^l[{\widehat{\mathbf{p}}}_i^{\mathrm{coarse}}+\delta ]\right),$$

(10)

where $\delta$ is a local offset candidate. This coarse-to-fine design keeps the global search robust while allowing the local branch to recover fine spatial details around each coarse prediction.

3.5. Topology-Constrained Graph Refinement (TCGR)

Per-landmark matching does not guarantee anatomically consistent configurations: outlier predictions may violate expected bone lengths or joint angles. TCGR addresses this as a graph reasoning problem, where each predicted landmark is a node and edges encode anatomical adjacency. For the Cephalometric dataset ($N=19$), we use a sparse anatomical graph covering the cranial base, maxillary arch, mandibular body, and soft tissue profile; for the Hand dataset ($N=37$), a fully connected graph captures dense inter-landmark relationships across all finger chains and the carpus.

Given the fine predictions from Equation (10), we denote the matched landmark set as $\widehat{\mathbf{P}}=[{\widehat{\mathbf{p}}}_1^{\mathrm{fine}},\dots ,{\widehat{\mathbf{p}}}_N^{\mathrm{fine}}]\in {\mathbb{R}}^{N\times 2}$. Let $\mathbf{A}\in {\{0,1\}}^{N\times N}$ be the adjacency matrix of the anatomical graph. Each node is initialized with its normalized predicted coordinate and a matching confidence cue when available. An MLP projects the node features to dimension $d_h=128$, followed by three GAT layers [32] with residual connections and layer normalization (LN) [37], as in Equation (11):

$${\mathbf{H}}^{\left(l\right)}=\mathrm{LN}\left({\mathbf{H}}^{(l-1)}+{\mathrm{GAT}}_l\left({\mathbf{H}}^{(l-1)},\mathbf{A}\right)\right),$$

(11)

where ${\mathbf{H}}^{\left(l\right)}\in {\mathbb{R}}^{N\times d_h}$ denotes the node representation after the lth graph attention layer.

An output MLP decodes the final node representations into coordinate residuals $\Delta \widehat{\mathbf{P}}\in {\mathbb{R}}^{N\times 2}$. The refined landmark coordinates are obtained by Equation (12):

$${\mathbf{P}}^{\ast }=\widehat{\mathbf{P}}+\alpha \Delta \widehat{\mathbf{P}},$$

(12)

where $\alpha$ is a learnable refinement scale initialized to $0.1$.

The coordinate loss is defined as the mean squared coordinate error in Equation (13):

$${\mathcal{L}}_{\mathrm{coord}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{∥{\mathbf{p}}_i^{\ast }-{\mathbf{p}}_i^{\mathrm{gt}}∥}_2^2,$$

(13)

where ${\mathbf{p}}_i^{\ast }$ and ${\mathbf{p}}_i^{\mathrm{gt}}$ denote the refined prediction and the target coordinate of the ith landmark, respectively.

The topology loss is defined over the graph edge set $\mathcal{E}$. It penalizes the normalized discrepancy between predicted and target inter-landmark distances, as shown in Equation (14):

$${\mathcal{L}}_{\mathrm{topo}}={\displaystyle \frac{1}{\left|\mathcal{E}\right|}}\sum _{(i,j)\in \mathcal{E}}{\left({\displaystyle \frac{{∥{\mathbf{p}}_i^{\ast }-{\mathbf{p}}_j^{\ast }∥}_2-{∥{\mathbf{p}}_i^{\mathrm{gt}}-{\mathbf{p}}_j^{\mathrm{gt}}∥}_2}{{∥{\mathbf{p}}_i^{\mathrm{gt}}-{\mathbf{p}}_j^{\mathrm{gt}}∥}_2+ϵ}}\right)}^2,$$

(14)

where $ϵ={10}^{-6}$ is used for numerical stability.

TCGR is trained with the coordinate and topology losses combined in Equation (15):

$${\mathcal{L}}_{\mathrm{TCGR}}={\mathcal{L}}_{\mathrm{coord}}+\lambda {\mathcal{L}}_{\mathrm{topo}},\lambda =0.1.$$

(15)

4. Experiments

4.1. Datasets

We evaluate our method on two publicly available X-ray anatomical landmark datasets. The first is the Head X-ray dataset from the IEEE International Symposium on Biomedical Imaging (ISBI) 2015 Cephalometric Landmark Challenge [1], which is a standard benchmark for cephalometric landmark detection. It contains 400 lateral skull radiographs with high-resolution image data and associated pixel-spacing information. Each image is annotated with $N=19$ anatomical landmarks by two experienced radiologists, covering key craniofacial structures such as the cranial base, maxilla, mandible, teeth, and soft-tissue profile. Following the previous protocol [12], we use the same annotated template selected in prior work and evaluate on the 250 official testing images. The template image is fixed across all experiments and ablation studies, and it is not selected based on validation or test performance. The remaining images in the official training split are not used as additional unlabeled data by our method.

The second dataset is the Hand X-ray dataset [2], which is also widely used in anatomical landmark detection [12]. It contains 909 hand radiographs annotated with $N=37$ landmarks, covering finger joints, fingertips, bone boundaries, and the wrist region. Following the same evaluation protocol, we use the same annotated template selected in prior work, and the remaining 300 images are used as query/test images. The same template is used for all Hand X-ray experiments and ablation studies. Distances on the Hand X-ray dataset are normalized by assuming that the wrist width is 50 mm, following prior work [2].

For both datasets, the ground-truth annotations of the query/test images are used only for computing evaluation metrics. They are not used for training the global branch, local branch, MLMF, MLCA, TCGR, model selection, or hyperparameter tuning. Thus, although the benchmark datasets provide annotations for evaluation, the proposed framework itself receives only a single annotated template image during training.

4.2. Implementation Details

The DINO ViT-S/8 backbone ($n_h=6$ attention heads, 12 Transformer layers, embedding dimension 384) is loaded with official pre-trained weights and frozen throughout all training stages. Input images are resized before being fed to the ViT, yielding a patch-token sequence of length L, on which MLCA operates at $\mathcal{O}\left(L\right)$ cost. Feature extraction employs forward hooks registered on the multi-head self-attention key and value projection outputs of Layers 5, 8, and 11, yielding six feature sources each of dimensionality 384 at this patch resolution. The MLMF module uses output dimension $C_{\mathrm{out}}=256$ with MLP reduction ratio $r=4$. The MLCA module applies a bidirectional Mamba block with $C_{\mathrm{out}}=256$ channels, SSM state dimension $N_{\mathrm{state}}=16$, and a bidirectional (forward/backward) scan strategy; it is inserted immediately after the MLMF weighted aggregation within each branch. The global upsampling head applies $\times 2$ bilinear upsampling followed by a $3\times 3$ convolutional layer (256 channels). The local upsampling head applies two transposed convolutional layers (each $\times 2$ upsampling, 256 and 128 channels respectively) followed by a $5\times 5$ convolutional refinement layer.

Global branch training uses the Adam optimizer [38] with an initial learning rate of ${10}^{-4}$, a cosine annealing schedule, 5000 training iterations, and a batch size of 8. The local branch uses the same optimizer and learning-rate schedule and is trained for 3000 iterations. For TCGR training, we use the same template-derived augmentation protocol as the global and local branches. Specifically, the target landmark coordinates are obtained by applying the known augmentation transformations to the single template annotation. No ground-truth coordinates from the official query/test images are used for TCGR optimization. TCGR is trained with Adam using a learning rate of ${10}^{-4}$ for 3000 iterations, with the refinement scale $\alpha$ initialized to 0.1 and the topology loss weight set to $\lambda =0.1$. Since this setting does not provide an additional labeled validation set, $\lambda =0.1$ is chosen heuristically using only template-derived augmented views and is kept fixed across all datasets and experiments.

The total number of trainable parameters across the MLMF modules, MLCA modules, upsampling heads, and TCGR is approximately 4.2 M, compared with the 21.1 M parameters of the frozen DINO ViT-S/8 backbone. More specifically, the global branch contains approximately 1.85 M trainable parameters, the local branch contains approximately 2.18 M trainable parameters, and TCGR contains approximately 0.17 M trainable parameters. This parameter breakdown shows that the dual-branch design introduces a moderate trainable overhead while keeping the foundation-model backbone frozen.

For the top-K spatial consistency filtering used in the global branch, we set $K=10$, the spatial penalty weight to $\eta =0.1$, and the non-maximum suppression window size to $5\times 5$. These values are fixed for all experiments.

Following prior work [1,4], we report two complementary metrics. The Mean Radial Error (MRE) in millimeters is the primary metric, computed as the average Euclidean distance between predicted and ground-truth landmark positions across all landmarks and all images. The Success Detection Rate (SDR) reports the percentage of predictions falling within a specified radius of the ground truth.

4.3. Comparison with State of the Art

Table 1. Comparison with state-of-the-art methods on both datasets. All methods operate in an annotation-efficient setting unless noted. Mean Radial Error (MRE) is reported in mm; Success Detection Rate (SDR) is reported in %. ↓ indicates lower is better and ↑ indicates higher is better. Bold: best performance. underline: second best performance. L/U = Labeled/Unlabeled images.

Method	Cephalometric (Head) Dataset						Hand X-Ray Dataset
	L/U	MRE ↓	SDR@2 mm ↑	SDR@2.5 mm ↑	SDR@3 mm ↑	SDR@4 mm↑	L/U	MRE ↓	SDR@2 mm ↑	SDR@4 mm ↑
CC2D	1/149	2.36	51.81	63.13	73.66	86.25	1/608	2.65	51.19	82.56
SAEM	1/149	2.58	54.34	64.51	70.82	80.76	1/608	1.69	76.61	92.52
EGTNLR	1/149	2.27	49.45	63.07	74.70	88.91	1/608	1.81	64.62	95.03
UOD	3/985	2.43	51.14	62.37	74.40	86.49	3/985	2.52	53.37	84.27
FM-OSD	1/0	1.82	67.35	77.92	84.59	91.92	1/0	1.41	86.66	96.66
Ours	1/0	1.72	71.31	80.14	86.42	94.08	1/0	1.65	80.81	95.93

presents a quantitative comparison against representative detection methods on both the ISBI 2015 Cephalometric and Hand X-ray datasets. Among the compared methods, CC2D [4], SAEM [39], EGTNLR [40], and UOD [41] all use one to three labeled images together with tens to hundreds of unlabeled images, whereas our method uses a single labeled template and no unlabeled images at training time.

Table 1 shows that the proposed framework is particularly effective on the Cephalometric dataset, where it achieves the best performance across all reported metrics while using only one labeled template and no unlabeled images. This result is notable because several competing methods rely on additional unlabeled images to learn or refine their representations. The comparison suggests that, for lateral skull radiographs, richer multi-layer and multi-facet foundation-model descriptors can compensate for the lack of extra target-domain data, and that adding long-range context and anatomical graph refinement helps convert these descriptors into more stable landmark configurations.

The Hand X-ray results show a different but still informative pattern. The proposed method clearly improves over conventional baselines, but FM-OSD remains the strongest method on this dataset. Hand radiographs contain denser landmark layouts, repeated finger structures, and stronger local similarity between neighboring joints, making identity preservation more challenging than in the Cephalometric setting. The competitive but not dominant performance therefore suggests that our representation-context-topology integration transfers to hand landmarks, while also indicating that anatomical regions with dense and repetitive structures may require more specialized topology modeling or landmark-specific local refinement.

4.4. Ablation Study

To systematically validate each proposed component, we conduct a unified ablation study on the Cephalometric dataset. Starting from the single-layer single-facet baseline (Layer 9 key only, no MLCA, no TCGR), we incrementally add each module and examine key MLMF and MLCA design variants. We repeat the ablation runs with three different random seeds and report the mean in

Table 2. Unified ablation study on the Cephalometric dataset. Rows are grouped by the factor being varied. The top block traces component-wise additions from the baseline to the full method; subsequent blocks examine internal design choices of Multi-Layer Multi-Facet (MLMF), Mamba-Based Long-Range Context Aggregation (MLCA), and Topology-Constrained Graph Refinement (TCGR). Mean Radial Error (MRE) is reported in mm; Success Detection Rate (SDR) is reported in %. # denotes row number. ↓ indicates lower is better and ↑ indicates higher is better. ΔMRE relative to the full method (row 4) within each block. Bold: best per metric.

#	Configuration	MRE ↓	ΔMRE	SDR@2 mm ↑	SDR@3 mm ↑	SDR@4 mm ↑
Component-wise addition (each row adds one module to the previous)
1	Baseline: Layer-9 key only, no MLCA, no TCGR	1.82	+0.10	67.40	82.31	91.89
2	+MLMF (Layers 5/8/11, key + value, adaptive fusion)	1.78	+0.06	68.93	83.46	92.58
3	+MLCA (2-direction scan, $N_{\mathrm{state}}=16$, residual gate)	1.76	+0.04	69.53	84.17	93.09
4	+TCGR (3 GAT layers + topology loss)—Full Method	1.72	—	71.31	86.42	94.08
MLMF design variants (MLCA + TCGR fixed at full-method settings)
5	Uniform fusion (equal weights, no learned $\mathbf{w}$)	1.73	+0.01	70.87	85.94	93.71
6	Key facet only (value projections removed)	1.75	+0.03	70.34	85.29	93.34
7	Single Layer {8} only (multi-layer disabled)	1.77	+0.05	69.91	84.73	92.87
MLCA design variants (MLMF + TCGR fixed at full-method settings)
8	No MLCA (MLMF output used directly)	1.76	+0.04	69.18	83.84	92.76
9	1-direction scan (left-to-right only)	1.74	+0.02	70.48	85.63	93.51
10	No residual gate ($\gamma$ fixed to 1)	1.74	+0.02	70.21	85.40	93.31
TCGR design variants (MLMF + MLCA fixed at full-method settings)
11	No TCGR (matching output used directly)	1.76	+0.04	69.67	84.39	93.22
12	1 GAT layer	1.74	+0.02	70.19	85.33	93.29
13	2 GAT layers	1.73	+0.01	70.84	85.89	93.67
14	No topology loss ($\lambda =0$)	1.75	+0.03	69.95	85.15	93.12
15	GCN mean aggregation (no attention)	1.73	+0.01	70.96	86.08	93.64

. In addition, we perform paired Wilcoxon signed-rank tests using case-wise MRE values between the full method and the strongest ablation variants. The full method shows statistically significant improvement ($p<0.05$) over the single-layer single-facet baseline, the no-MLCA variant, and the no-TCGR variant. These results indicate that the improvement is consistent across test cases rather than being caused by random variation. All results are reported in Table 2.

Overall trends: The ablation results show that the proposed components improve the baseline in complementary ways. Starting from the single-layer single-facet baseline, the full method reduces MRE from 1.82 mm to 1.72 mm. MLMF improves the feature representation by combining multi-layer and multi-facet ViT cues, MLCA adds long-range spatial context before matching, and TCGR refines the final landmark configuration with anatomical topology. We first discuss the step-by-step contribution of the three modules, and then analyze the main design choices inside MLMF, MLCA, and TCGR.

Component-wise contributions: Rows 1–4 show that each module steadily improves the baseline. MLMF brings the first clear gain by replacing the single-layer single-facet feature map with a richer multi-source descriptor. Based on this stronger representation, MLCA further improves matching by allowing each patch descriptor to access long-range spatial context. TCGR then acts at the coordinate level, correcting configuration-level outliers that remain after descriptor matching. These trends suggest that the three modules address different sources of error rather than simply duplicating the same function.

Multi-Layer Multi-Facet (MLMF) design analysis: Rows 5–7 further examine the design of MLMF. Using only a single ViT layer causes the largest degradation in this group, indicating that multi-layer feature extraction is the most important part of MLMF. Removing value features also reduces performance, which supports the use of both key and value projections rather than relying only on attention-routing cues from keys. Replacing adaptive fusion with uniform fusion has a smaller effect on the average error, but the learned weights still provide more stable results across the reported SDR thresholds. This suggests that adaptive fusion is useful, while the main benefit of MLMF comes from exploiting complementary ViT layers and facets.

Mamba-Based Long-Range Context Aggregation (MLCA) design analysis: Rows 8–10 show that MLCA contributes information that is not captured by MLMF alone. Even when the multi-source descriptor is kept, removing MLCA weakens the result, which indicates that long-range context remains necessary after feature fusion. The single-direction variant is less effective than the bidirectional scan, suggesting that anatomical context benefits from information propagation in both traversal directions. The residual gate is also important: learning the gate lets the model introduce contextual updates gradually rather than forcing the Mamba branch to overwrite the fused descriptor. These results show that both bidirectional context modeling and controlled residual injection are needed for stable MLCA performance.

Topology-Constrained Graph Refinement (TCGR) design analysis: Rows 11–15 evaluate the graph refinement stage. Removing TCGR increases MRE by 0.04 mm, showing that topology-aware coordinate refinement is still useful after coarse-to-fine matching. Increasing the GAT depth gives a consistent trend toward better accuracy, suggesting that multi-hop message passing helps capture anatomical relations among landmarks. The topology loss is also important because explicit inter-landmark distance regularization encourages anatomically plausible predictions. The mean-aggregation GCN variant remains competitive in SDR, but the attention-based design gives better localization accuracy, indicating that adaptive neighbor weighting is beneficial when refining strong initial predictions.

Table 3. Comparison of Vision Foundation Model backbones on the Cephalometric and Hand X-ray datasets. All backbones are frozen; only Multi-Layer Multi-Facet (MLMF), Mamba-Based Long-Range Context Aggregation (MLCA), and Topology-Constrained Graph Refinement (TCGR) parameters are trained. Mean Radial Error (MRE) is reported in mm; Success Detection Rate (SDR) is reported in %. ↓ indicates lower is better and ↑ indicates higher is better. Bold: best per metric.

Backbone	Cephalometric (Head) Dataset					Hand X-Ray Dataset
	MRE ↓	SDR@2 mm ↑	SDR@2.5 mm ↑	SDR@3 mm ↑	SDR@4 mm ↑	MRE ↓	SDR@2 mm ↑	SDR@4 mm ↑
SAM ViT-B	1.83	67.52	77.86	84.13	91.94	1.93	76.34	92.17
DINO ViT-S/8 (ours)	1.72	71.31	80.14	86.42	94.08	1.65	80.81	95.93
DINO ViT-B/8	1.69	72.18	81.07	87.33	94.61	1.61	80.27	96.34

reveals two consistent trends across both datasets. First, pre-training objective alignment is decisive: DINO ViT-B/8 and DINO ViT-S/8 both outperform SAM ViT-B by a substantial margin (Head MRE gap: 0.14 mm and 0.11 mm, respectively, computed from the reported two-decimal values), despite SAM ViT-B having comparable parameter counts. SAM ViT-B is pre-trained with a mask-prediction objective on SA-1B, producing representations optimized for region-level segmentation rather than point-level correspondence, which is directly at odds with the cosine-similarity nearest-neighbor matching at the core of our framework.

Second, within the DINO family, scaling the backbone from ViT-S/8 to ViT-B/8 yields consistent improvements on the Cephalometric dataset and a mixed but competitive pattern on the Hand X-ray dataset (Hand MRE: 1.65 mm → 1.61 mm; SDR@2 mm: 80.81% → 80.27%; SDR@4 mm: 95.93% → 96.34%). Larger embedding dimension (768 vs. 384) and more attention heads (12 vs. 6) can provide richer patch-level representations, but the benefit is threshold-dependent and comes at the cost of a $\sim 4\times$ increase in backbone parameters (86 M vs. 21 M). These results make DINO ViT-S/8 a parameter-efficient default, while DINO ViT-B/8 is useful when the modest MRE and high-threshold SDR gains justify the added computational cost.

To examine whether TCGR is overly dependent on a manually defined anatomical graph, we further conduct a graph-design sensitivity analysis, as shown in

Table 4. Sensitivity analysis of Topology-Constrained Graph Refinement (TCGR) under different graph designs on the Cephalometric dataset. The results evaluate whether the graph refinement module is overly dependent on a manually defined anatomical graph. Mean Radial Error (MRE) is reported in mm and Success Detection Rate (SDR) is reported in %. ↓ indicates lower is better and ↑ indicates higher is better. Bold: best per metric.

Graph Design	MRE ↓	SDR@2 mm ↑	SDR@3 mm ↑	SDR@4 mm ↑
No TCGR	1.76	69.67	84.39	93.22
Sparse random graph with matched edge number	1.75	70.02	84.91	93.37
k-nearest-neighbor graph ($k=3$)	1.74	70.42	85.57	93.58
Fully connected graph	1.73	70.88	85.94	93.71
Default anatomy-guided graph	1.72	71.31	86.42	94.08

. We compare the default anatomy-guided graph with several alternative graph structures, including a fully connected graph, a k-nearest-neighbor graph constructed from template landmark coordinates, and a sparse random graph with the same number of edges as the default graph. The no-TCGR variant is also included as a reference.

The results show that TCGR consistently improves over the no-graph-refinement baseline under most graph designs, indicating that graph-based coordinate refinement is generally beneficial. The default anatomy-guided graph achieves the best or near-best performance, suggesting that anatomical prior knowledge provides useful structural guidance. Meanwhile, the competitive performance of the alternative graph designs indicates that TCGR is not restricted to a single manually specified graph. This supports the transferability of the graph refinement idea, although applying it to new anatomical regions may still require redefining or estimating landmark relations.

4.5. Qualitative Comparison

Figure 3 provides a qualitative comparison of all five evaluated methods on representative Cephalometric and Hand X-ray images. The progression from CC2D to our proposed method shows a consistent reduction in the spread of predicted landmarks around ground-truth positions, with the most pronounced improvement in anatomically challenging regions such as the mandibular condyle and carpal bones.

5. Discussion

The experiments highlight a central message: effective anatomical landmark detection in this setting requires more than strong visual features alone. The model must also organize these features with spatial context and constrain the final predictions with anatomical structure. From this perspective, the proposed framework can be viewed as a progressive refinement process, moving from foundation-model descriptors to context-aware matching and finally to topology-consistent landmark configurations. We discuss the framework from three aspects: richer use of frozen foundation-model features, context-aware descriptor representation, and cooperative refinement toward anatomically consistent predictions.

Depth of foundation model feature exploitation: The results show that the benefit of DINO ViT is not limited to using it as a fixed feature extractor. A stronger strategy is to use its internal hierarchy more fully. Shallow and deep layers provide different levels of information, from local edge patterns to global anatomical structure, while key and value features offer complementary cues for matching. This explains why MLMF improves the baseline and why single-layer or key-only variants are less effective. In this sense, MLMF acts as a lightweight feature router: it selects useful cues from the frozen backbone without fine-tuning the foundation model itself. This is important when the model must adapt to a new anatomical target from very limited supervision.

Long-range context via selective state space modeling: Building on richer ViT features, the next question is how to make each local descriptor aware of the whole anatomical field. The ablation results indicate that feature fusion alone is not sufficient; long-range context still brings additional improvement when MLMF and TCGR are already used. MLCA addresses this need by propagating information along the serialized patch sequence with a linear-complexity Mamba block. Compared with standard self-attention, this gives a more efficient way to model global structure in high-resolution X-ray images. The bidirectional scan and residual gate further make the context update more stable, allowing global information to help matching without overwhelming the local descriptor.

Complementarity of Multi-Layer Multi-Facet (MLMF), Mamba-Based Long-Range Context Aggregation (MLCA), and Topology-Constrained Graph Refinement (TCGR): Together, the three modules form a coarse-to-fine correction pipeline at different levels. MLMF improves the descriptor before matching, MLCA enriches the descriptor with global spatial context, and TCGR corrects the final landmark configuration after matching. This layered design helps explain why removing any one module leads to a performance drop. The modules are not interchangeable: MLMF mainly addresses feature richness, MLCA addresses spatial awareness, and TCGR addresses anatomical consistency. Their combination is therefore more effective than improving only one stage of the pipeline. This also suggests that landmark detection benefits from jointly considering representation, context, and topology, rather than relying only on nearest-neighbor feature matching.

Limitations and future work: Although the proposed framework shows strong performance, several limitations remain. The first limitation lies in the topology refinement stage. TCGR uses a manually defined anatomical graph to encode prior structural relationships among landmarks. This design is effective for well-studied anatomical regions such as the head and hand, where landmark connectivity can be specified from prior knowledge. However, for new anatomical targets, the graph may be harder to define and may not fully reflect subject-specific anatomical variation. A possible future direction is to replace the fixed graph with a learnable graph structure [42], so that inter-landmark relationships can be adapted from data rather than fully specified by hand. Another limitation is the input resolution. The original Cephalometric images have high resolution with associated pixel-spacing information, whereas our ViT input is resized to a lower working resolution. This downsampling is necessary to keep frozen ViT feature extraction and patch-token processing computationally feasible. The predicted coordinates are mapped back to the original image coordinate system before MRE computation, but some fine-grained local image details may still be lost during resizing. The proposed global-to-local design partly mitigates this issue by using a local refinement stage around coarse predictions. Nevertheless, high-resolution or multi-scale feature extraction may further improve sub-millimeter localization accuracy and will be explored in future work. Another limitation is the fixed scan order used in MLCA. The current implementation serializes 2D patch tokens in a row-major order before applying the bidirectional Mamba scan. This strategy is simple and efficient, but it does not explicitly preserve all spatial relationships in the original image plane. More spatially aware scan strategies, such as 2D-aware linearization [29] or learned scan orders, may further improve long-range context modeling. Finally, our experiments focus on 2D X-ray images. Extending the framework to 3D modalities such as CT or MRI would require changes to tokenization, context aggregation, and graph construction. In addition, DINO ViT is pre-trained on natural RGB images. Although it transfers well to X-ray images in our experiments, its behavior on other medical modalities, such as ultrasound or fluorescence microscopy, still needs further study. Future work will explore modality-adaptive foundation models and Mamba-based context modules for broader medical landmark detection tasks.

6. Conclusions

We presented an anatomical landmark detection framework that combines frozen foundation-model features, long-range context modeling, and topology-aware graph refinement. The proposed MLMF module exploits multi-layer key and value features from DINO ViT to build richer landmark descriptors, while MLCA introduces global anatomical context through bidirectional Mamba-based sequence modeling. A dual-branch global-to-local matching strategy further improves localization precision, and TCGR refines the predicted landmark set under anatomical topology constraints.

Experiments on the Cephalometric and Hand X-ray benchmarks demonstrate the effectiveness of the proposed framework. On the Cephalometric dataset, the method achieves an MRE of 1.72 mm and an SDR@2 mm of 71.31%; on the Hand X-ray dataset, it achieves an MRE of 1.65 mm and SDRs@2/4 mm of 80.81%/95.93%. Ablation studies further show that MLMF, MLCA, and TCGR contribute complementary improvements, indicating that representation richness, long-range spatial context, and structural consistency are all important for robust landmark detection. These findings also provide a useful basis for future studies on more general, data-efficient medical landmark detection frameworks.