GazeMap: Dual-Pathway CNN Approach for Diagnosing Alzheimer’s Disease from Gaze and Head Movements

Abstract

Alzheimer’s disease (AD) is a progressive neurodegenerative disorder that impairs cognitive function, making early detection crucial for timely intervention. This study proposes a novel AD detection framework integrating gaze and head movement analysis via a dual-pathway convolutional neural network (CNN). Unlike conventional methods relying on linguistic, speech, or neuroimaging data, our approach leverages non-invasive video-based tracking, offering a more accessible and cost-effective solution to early AD detection. To enhance feature representation, we introduce GazeMap, a novel transformation converting 1D gaze and head pose time-series data into 2D spatial representations, effectively capturing both short- and long-term temporal interactions while mitigating missing or noisy data. The dual-pathway CNN processes gaze and head movement features separately before fusing them to improve diagnostic accuracy. We validated our framework using a clinical dataset (112 participants) from Konkuk University Hospital and an out-of-distribution dataset from senior centers and nursing homes. Our method achieved 91.09% accuracy on in-distribution data collected under controlled clinical settings, and 83.33% on out-of-distribution data from real-world scenarios, outperforming several time-series baseline models. Model performance was validated through cross-validation on in-distribution data and tested on an independent out-of-distribution dataset. Additionally, our gaze-saliency maps provide interpretable visualizations, revealing distinct AD-related gaze patterns.

1. Introduction

The human gaze is a critical biomarker that offers insights into an individual’s intentions and emotional states. Recently, gaze tracking has been widely applied in human–computer interaction [1,2] and healthcare [3,4,5]. Notably, gaze patterns have emerged as promising non-invasive indicators for diagnosing neurodegenerative conditions, including depression [6], stress [7], and Alzheimer’s disease (AD) [3].

Beyond clinical applications, video-oculography has long been integral to neurophysiological research, offering detailed insights into brain function across various neurodegenerative diseases [8]. Particularly in AD, eye movement abnormalities reflect underlying cortical and subcortical neuropathology. Quantitative oculomotor assessments have revealed impairments such as increased saccade latency and reduced accuracy, both of which are indicative of cognitive decline [9,10,11,12,13]. Smooth pursuit tasks further differentiate AD patients from healthy controls through diminished tracking precision, reflecting disruptions in visuospatial processing and attention mechanisms [14]. Recent developments employing advanced gaze tracking technologies have demonstrated high diagnostic accuracy, underscoring the potential of oculomotor metrics as reliable biomarkers for early detection and progression monitoring of AD [15].

Recent advances in computer vision have enabled the non-invasive fine-grained quantification of subtle behavioral cues using standard video recordings. These methods leveraging deep learning effectively analyze visual saliency, eye movements, and induced nystagmus from smartphone videos, demonstrating diagnostic capabilities comparable to specialized equipment [16,17,18]. These innovations significantly enhance the accessibility and scalability of cognitive assessments, providing promising tools for early diagnosis and continuous monitoring of neurological disorders.

As cognitive impairment progresses, disruptions in gaze control become more pronounced, often accompanied by subtle changes in head movement [19]. While healthy individuals exhibit relatively independent eye–head motions, AD patients show tighter coupling and increased vertical co-rotation, reflecting underlying motor control impairments [20]. Integrating gaze and head dynamics has been shown to improve classification performance in cognitive and motor disorders, yielding high diagnostic accuracy [21].

Despite growing interest in gaze-based assessments for AD detection [3,22,23], head movements remain underexplored. However, a combined analysis of gaze and head dynamics offers a more comprehensive view of cognitive status, capturing subtle motor control deficits that gaze alone may miss. Two major challenges must be addressed to translate these insights into practical diagnostic tools.

The first challenge is ensuring accurate gaze and head tracking while minimizing costs. Traditional gaze-tracking systems rely on infrared technology to detect the pupil size and position [24,25,26]. Although these systems offer high precision, they are expensive and are usually restricted to controlled clinical or research environments, limiting their practical applicability. Recent advancements in deep learning have enabled accurate gaze and head tracking using standard camera setups, thereby significantly reducing costs and broadening the possibilities for large-scale or continuous monitoring. This study leverages these advances to develop a more accessible and scalable approach that combines gaze and head movement data for AD detection.

The second challenge involves accurately analyzing diverse behavioral patterns derived from continuous gaze and head movement data. This includes detecting short-term events, such as brief saccades, rapid eye shifts, or minor head adjustments, as well as long-term patterns, such as sustained fixation or head orientation. Although these behaviors can indicate cognitive impairment, real-world data often contain artifacts (e.g., missed frames, blinks, and extreme head angles) that introduce noise or result in missing values. Therefore, it is essential to adopt a representation that effectively captures both short-term nuances and broader long-term trends while remaining robust to data loss and noise.

To overcome these challenges, we propose a novel AD detection framework that integrates gaze and head movement analysis, emphasizing both short- and long-term temporal dynamics. Our key innovation, GazeMap, transforms one-dimensional (1D) time-series data into structured two-dimensional (2D) images, enabling the extraction of rich temporal patterns while mitigating data loss and noise. This transformation allows convolutional neural networks (CNNs) to effectively classify cognitive function as normal or abnormal.

To further enhance diagnostic accuracy, we introduce a dual-pathway CNN architecture that simultaneously analyzes gaze and head motion, capturing their complex interactions. This approach improves precision by leveraging complementary movement patterns associated with cognitive impairment. Evaluated on a clinical dataset of 112 participants, our framework achieved 91.09% accuracy, outperforming gaze-only methods by 6.31%. By integrating gaze and head motion analysis with GazeMap, our method provides a scalable, non-invasive, and accessible tool for early AD detection, offering a promising solution for identifying individuals at risk of cognitive decline.

Our contributions are summarized as follows:

• We propose a novel dual-pathway framework that integrates gaze and head movement for AD detection, offering a more comprehensive and accurate assessment of cognitive health than gaze-only methods.
• We introduce GazeMap, a novel transformation method that converts 1D gaze and head pose sequences into structured 2D spatial representations, facilitating robust feature extraction and improving model interpretability.
• We validate the proposed framework through comprehensive experiments on both clinical and real-world datasets, demonstrating robust generalization and clinically meaningful performance in diverse, non-controlled environments, beyond conventional benchmark comparisons.

2. Related Work

Early and accurate detection of AD is crucial for timely intervention and management. Traditional diagnostic methods, such as magnetic resonance imaging and positron emission tomography, provide high diagnostic accuracy but are limited by their high cost, reliance on specialized equipment, and the need for expert operators. These challenges have driven research toward identifying more accessible biomarkers, including cognitive and behavioral indicators. Among these, gaze analysis has gained prominence as a cost-effective and non-invasive alternative for early AD detection.

Automatic AD Detection on Gaze Tracking

Gaze tracking is an emerging tool for AD detection because subtle changes in gaze behavior can provide critical insights into cognitive decline [27,28]. Although gaze tracking does not directly measure brain activity, it offers valuable insight into how brain function manifests in behavior, making it an informative biomarker for cognitive assessment [3]. Studies have explored various gaze-related measures, such as pupil dilation, fixation, and saccades. However, many early studies [20,22,23] focused on the correlations between gaze patterns and cognitive functions, with limited emphasis on directly validating gaze tracking as a diagnostic tool in clinical settings.

Recent research has leveraged deep learning to evaluate gaze patterns for AD detection. These studies primarily focused on analyzing gaze sequences. Recurrent neural networks (RNNs) such as long short-term memory (LSTM) and gated recurrent units (GRUs) have been employed to analyze temporal gaze patterns [29,30]. Another approach involves generating gaze heatmaps that visually represent the spatial gaze distribution and are analyzed using a CNN [31,32]. However, although these heatmaps efficiently model the statistical characteristics of a patient’s visual movements based on their spatial distribution, they fail to capture dynamic temporal interactions. Consequently, their ability to fully represent and illuminate user cognitive processes is limited.

To address these limitations, we propose a feature-map representation that simultaneously captures subtle gaze shifts and broader gaze patterns. By integrating temporal and spatial information into a single representation, our approach facilitates the detection of both gaze behaviors and visual trajectories.

3. Methods

Our goal is to enhance diagnostic accuracy by analyzing gaze and head movement patterns extracted from video streams. As illustrated in Figure 1, our framework consists of three key stages: (1) time-series feature extraction—extracting gaze and head pose sequences from video data; (2) temporal-to-spatial transformation—converting 1D time-series features into 2D spatial representations; (3) CNN-based classification—using a dual-pathway CNN to detect AD-related patterns from the transformed visual features.

This approach enables precise gaze and head-pose tracking without requiring specialized equipment while enhancing robustness against missing or noisy data.

3.1. Time-Series Feature Extraction

Figure 2 summarizes the feature extraction process. For each frame t in the input video, we first detect the facial region using RetinaFace [33], which precisely isolates the face from the background. The detected face is then cropped and resized to $224\times 224$, forming a 2D facial image sequence $\left\{f_t\right\}$. This sequence is processed using two separate pretrained networks, L2CS-Net [34] for gaze estimation and 6DRepNet [35] for head pose estimation, to obtain the 1D gaze sequence $\left\{g_t\right\}$ and 1D head pose sequence $\left\{h_t\right\}$. Both L2CS-Net and 6DRepNet are optimized for robust performance under various conditions to ensure accurate feature extraction.

As illustrated in Figure 3, gaze angles commonly include pitch (vertical eye movement) and yaw (horizontal eye movement), whereas head pose angles incorporate pitch, yaw, and roll. To facilitate the CNN-based processing of gaze cues, we convert gaze angles into a gaze-saliency map, inspired by optical flow visualization [36]. Similar to how optical flow visualization transforms motion vectors into color-coded images—capturing both the direction and magnitude of motion in a form suitable for convolutional processing—we encode gaze direction and magnitude into an image-like representation. We represent gaze direction as the hue component in the hue, saturation, and value (HSV) color space and magnitude as the saturation component and set the value component to 1. Subsequently, we transform this HSV map into an RGB image to obtain the final gaze 1D sequence $\left\{g_t^{\prime }\right\}$. This transformation provides an intuitive, image-like depiction of the gaze direction and magnitude, thus enhancing its suitability for CNN-based analyses. The HSV components are computed using the following equations:

$$Hu{e}_t={\displaystyle \frac{\mathbf{arctan}(g_t^{pitch},g_t^{yaw})+\pi }{2\pi }}$$

(1)

$$Saturatio{n}_t={\displaystyle \frac{\sqrt{{\left(g_t^{pitch}\right)}^2+{\left(g_t^{yaw}\right)}^2}}{M}}$$

(2)

$$Valu{e}_t=1$$

(3)

where $g_t^{pitch}$ and $g_t^{yaw}$, respectively, indicate the pitch and yaw angles in frame t, $\mathbf{arctan}(·,·)$ is the arctangent function, and M is a normalization constant corresponding to the maximum gaze magnitude.

3.2. Temporal-to-Spatial Transformation

Raw gaze and head pose data, though informative, often contain missing or noisy segments owing to blinking or abrupt head movements. A simple 1D analysis may struggle with these inconsistencies. To address this, we convert the 1D time-series data into a 2D spatial image sequence, as shown in Figure 4. We perform row-wise mapping, where time steps are sequentially arranged from top to bottom in an image-like format. This image-based representation allows CNNs to better model complex temporal dependencies while mitigating data loss.

3.3. CNN-Based Classification

We hypothesize that combining gaze and head pose features leads to more accurate AD detection. Inspired by the SlowFast architecture [37], we design a dual-pathway CNN where the main pathway extracts deep gaze features and the sub-pathway processes head pose features. The architecture details of the dual-pathway CNN are summarized in

Table 1. The architecture details of the dual-pathway CNN. The orange colors mark fewer channels for the sub-pathway. The backbone is ResNet50.

Stage	Main Pathway	Sub-Pathway	Output Sizes 1
Input	-	-	Main: $3\times {44}^2$
			Sub: $3\times {44}^2$
conv1	$7^2,64$	$7^2$, 8	Main: $64\times {22}^2$
	Stride $2^2$	Stride $2^2$	Sub: 8 $\times {22}^2$
cov2_x	$\left[\begin{array}{c}{1}^2,64\\ 3^2,64\\ 1^2,256\end{array}\right]\times 3$	$\left[\begin{array}{c}{1}^2,8\\ 3^2,8\\ 1^2,32\end{array}\right]\times 3$	Main: $64\times {22}^2$
			Sub: $8\times {22}^2$
conv3_x	$\left[\begin{array}{c}{1}^2,128\\ 3^2,128\\ 1^2,512\end{array}\right]\times 4$	$\left[\begin{array}{c}{1}^2,16\\ 3^2,16\\ 1^2,64\end{array}\right]\times 4$	Main: $128\times {11}^2$
			Sub: $16\times {11}^2$
conv4_x	$\left[\begin{array}{c}{1}^2,256\\ 3^2,256\\ 1^2,1024\end{array}\right]\times 6$	$\left[\begin{array}{c}{1}^2,32\\ 3^2,32\\ 1^2,128\end{array}\right]\times 6$	Main: $256\times 6^2$
			Sub: $32\times 6^2$
conv5_x	$\left[\begin{array}{c}{1}^2,512\\ 3^2,512\\ 1^2,2048\end{array}\right]\times 3$	$\left[\begin{array}{c}{1}^2,64\\ 3^2,64\\ 1^2,256\end{array}\right]\times 3$	Main: $512\times 6^2$
			Sub: $64\times 6^2$
global average pooling, metadata concatenate, fc layer			2 (HC, AD)

¹ Output sizes: $\mathrm{channel}\times \mathrm{height}\times \mathrm{width}$.

.

In our model, the main pathway employed the ResNet architecture [38] for gaze-related cues. Meanwhile, the sub-pathway processes head pose cues at a reduced channel capacity, set to $\alpha =1/8$ of the main pathway. This configuration ensures a lightweight but effective architecture for integrating head pose information. Then, to integrate gaze and head pose representations, we explore three fusion strategies—addition, multiplication, and concatenation. If addition or multiplication is used, the channel dimensions are matched by adding a $1\times 1$ convolutional layer to the sub-pathway output.

Finally, we apply global average pooling and concatenate participant metadata (age, gender, education level) to form a comprehensive diagnostic representation, which is processed through a fully connected layer for AD classification. This dual-pathway design effectively captures both gaze- and head-pose-driven cues, improving AD detection.

4. Data Collection from Gaze Tracking Task

To validate the effectiveness of GazeMap, we designed a structured gaze-tracking task and collected data from both Alzheimer’s disease (AD) patients and healthy control (HC) participants. Our task involved displaying a series of visual targets on a screen and instructing participants to track them with their gaze. During this process, both gaze movements and head motions were recorded, providing rich temporal data for analysis. To ensure a robust and meaningful task design, we conducted a thorough review of existing gaze-tracking studies, refining our approach to maximize diagnostic relevance. This task standardization and dataset collection represent another contribution, enabling a systematic evaluation of AD-related gaze and head movement abnormalities through GazeMap’s advanced spatiotemporal analysis. The following sections provide a detailed explanation of the gaze-tracking task design and the characteristics of the collected dataset.

4.1. Gaze Tracking Task Design

To assess participants’ visual attention and working memory, we designed a structured gaze-tracking task based on their eye movement patterns. This design was inspired by the King–Devick (KD) test [39], which has been shown to differentiate saccadic eye movements in individuals with AD from HC participants [40]. Participants were instructed to follow a red dot as it moved across a screen for 60 s, with position updates every 0.5 s. The task was divided into three phases: a 3-second preparation phase, where the dot remained centered on the screen; a 40-second regular phase, with predictable motion to assess sustained attention; and a 20-second random phase, where the dot moved unpredictably to evaluate flexibility in gaze control.

Figure 5 outlines this procedure. Figure 6 illustrates a 2.5-second interval (43.5–46.0 s) during the random phase. The red dot’s movement over time is visualized through sequential frames and summarized in a single image. The figure compares gaze behaviors between an HC participant and an AD participant using both the face detection and gaze detection models (see Section 3.1). Green bounding boxes mark the detected faces, and red arrows indicate gaze directions. During the 2.5-second interval (75 samples at 30 fps), the HC participant consistently tracks the moving dot, whereas the AD participant exhibits scattered and unstable gaze trajectories, reflecting impaired visual attention and motor control.

4.2. Collected Data Description

AD diagnosis was performed by board-certified psychiatrists through comprehensive clinical assessments, including Mini-Mental State Examination (MMSE) [41], neuropsychological testing, activities of daily living evaluations (the Korean Instrumental activities of daily living) [42], behavioral assessments (Caregiver-Administered Neuropsychiatric Inventory) [43], and dementia severity ratings (CDRs) [44]. Participants with depressive symptoms, impaired mobility, neurological disorders, or major psychiatric illnesses based on the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) [45] were excluded. The study protocol was approved by the Institutional Review Board of Konkuk University Medical Center (No. 2021-08-035).

We collected two datasets for gaze-based AD detection: an in-distribution dataset from a clinical setting and an out-of-distribution dataset from a real-world setting. The in-distribution dataset was acquired at the Konkuk University Hospital. This dataset includes 61 HC participants and 51 AD participants, all tested under controlled conditions using a desktop monitor. The out-of-distribution dataset was collected in senior centers and nursing homes. This dataset comprises 21 HC participants and 23 AD participants, recorded under diverse environmental conditions such as varying lighting, backgrounds, and participant states.

The in-distribution dataset was used for model training and validation, ensuring consistency in a controlled hospital environment. In contrast, the out-of-distribution dataset was collected under real-world conditions, introducing domain shifts that pose additional challenges for generalization.

Table 2. Demographic information of the research participants in the gaze tracking task. All variables except gender are expressed as ‘mean ± standard deviation’.

Variable	In-Distribution Dataset (n = 112)		Out-of-Distribution Dataset (n = 44)
	HC (n = 61)	AD (n = 51)	HC (n = 21)	AD (n = 23)
Age	65.9 ± 8.2	75.4 ± 7.9	62.9 ± 11.2	78.3 ± 9.8
Education 1	12.3 ± 3.1	7.8 ± 4.3	13.1 ± 3.7	9.3 ± 5.8
Gender 2	31/30	20/31	5/16	9/14
MMSE	28.4 ± 1.3	23.1 ± 3.9	28.7 ± 1.9	24.0 ± 5.3
Environment	Standardized setup		Diverse real-world setup

¹ Years of education; ² male/female.

presents participant demographics, including age, education, gender, and Mini-Mental State Examination (MMSE) scores. The MMSE [41] is a widely used neuropsychological tool for dementia screening and is based on a scale of 0–30, where higher values indicate stronger cognitive abilities. Both datasets were carefully designed to ensure a balanced distribution of disease status and maintain consistent gaze data collection through standardized equipment.

Both datasets were recorded using a Logitech C920s PRO HD webcam at 30 frames per second (fps). The resolution was down-sampled to 640 × 480 pixels for storage efficiency. The in-distribution and out-of-distribution data were collected using a 612.00 mm × 344.25 mm desktop and a 357 mm × 224 mm portable laptop, respectively. Figure 7 illustrates the recording conditions.

5. Experiments

We evaluate the effectiveness of using the participants’ gaze and head movement patterns for AD detection. We trained our dual-pathway CNN model on an in-distribution dataset and tested its performance on both the in-distribution and out-of-distribution datasets. The experimental setup is described in Section 5.1, and the experimental results are discussed in Section 5.2. Section 5.3 describes the ablation study, and the proposed features are evaluated in Section 5.4.

5.1. Experimental Settings

We applied three-fold cross-validation to the in-distribution dataset to evaluate the performance of our dual-pathway CNN. The primary evaluation metrics were accuracy, sensitivity, specificity, F1 score, and the receiver operating characteristic–area under the curve (ROC-AUC). All the metrics were averaged over three folds. We confirmed the generalizability of the proposed model by testing it on an out-of-distribution dataset.

5.1.1. Implementation Details

All experiments were implemented using PyTorch version 2.0.0. Each video sequence initially contained 1890 frames (63 s × 30 fps) that were zero-padded to 1936 frames for consistency. The 1D gaze sequence $\left\{g_t^{\prime }\right\}$ and 1D head pose sequence $\left\{h_t\right\}$ were each transformed into $44\times 44$ 2D representations, which were then z-normalized and fed into a dual-pathway CNN for classification. For optimization, we used stochastic gradient descent with a learning rate of 2 × 10⁻⁴ and weight decay of 1 × 10⁻⁴ to minimize the cross-entropy loss. A dropout rate of 0.2 was applied to the fully connected layer to prevent overfitting. The model was trained for 300 epochs with a batch size of 8. All the experiments were conducted on a single NVIDIA RTX 3090 GPU.

5.1.2. Baselines

We compared our dual-pathway CNN with several time-series classification baselines, including dynamic time warping support vector machine (DTW-SVM) [46], LSTM [47], GRU [48], temporal convolutional network (TCN) [49], and InceptionTime [50]. Each method is briefly described as follows:

• DTW-SVM: DTW measures similarity by allowing nonlinear alignments of time-series data. Here, DTW-derived distances for both gaze and head pose are used in an SVM classifier to distinguish AD participants and HC participants.
• LSTM and GRU: LSTM captures long-range dependencies through memory cells, whereas GRU provides a more computationally efficient structure. Both models were employed to classify AD using gaze and head pose sequences.
• TCN: TCN uses causal and dilated convolutions to model long-range temporal structures. Unlike RNN-based methods, it processes entire sequences in parallel to improve the training stability and efficiency.
• InceptionTime: InceptionTime is a deep CNN specifically designed for time-series classification. It applies multiple convolutional filters in parallel to capture temporal features at different scales.

5.2. Experimental Results

5.2.1. Main Results

Figure 8 compares the accuracy of all the methods on both the in-distribution and out-of-distribution datasets. Our proposed dual-pathway CNN achieves the highest accuracy in both settings, demonstrating its capability to capture AD-specific gaze and head-movement patterns while demonstrating robustness to real-world variability.

As shown in Figure 8a, the dual-pathway CNN attains 91.09% accuracy on the in-distribution dataset, significantly outperforming all baseline methods. Traditional approaches, such as DTW-SVM (61.62%), struggle to model complex temporal dependencies, while RNN-based models (LSTM: 81.25%, GRU: 85.15%) provide moderate improvements over 1D CNN-based methods (TCN: 79.49%, InceptionTime: 81.22%), likely due to their ability to capture fine-grained temporal variations.

Figure 8b presents the results on the out-of-distribution dataset. Overall accuracy declines due to domain shifts caused by greater variability. However, our dual-pathway CNN remains the most robust, achieving 83.33% accuracy. Baseline methods degrade more sharply, with DTW-SVM dropping to 46.42%, highlighting the challenges of AD detection in diverse, uncontrolled environments.

These findings emphasize that while standard 1D CNN architectures (e.g., TCN, InceptionTime) are promising, they fail to capture fine-grained temporal variations essential for AD detection. By transforming gaze and head pose data into a 2D structure, our dual-pathway CNN enables more stable feature learning, leading to superior classification performance in both clinical and real-world settings.

To provide a more detailed understanding of the classification behavior, we further present the confusion matrices of the dual-pathway CNN in Figure 9. On the in-distribution dataset, the model shows balanced and high classification performance, correctly identifying HC and AD participants with few misclassifications (HC: 57/61, AD: 45/51). On the out-of-distribution dataset, while a slight decrease in classification performance is observed due to increased variability, the model still maintains a strong ability to differentiate between classes. Specifically, only 1 AD case is misclassified as HC, demonstrating high sensitivity in detecting AD, which is critical in clinical applications. These results validate that the dual-pathway CNN not only achieves high overall accuracy but also exhibits stable class-wise performance under distributional shifts.

5.2.2. Robustness to Missing Data

To evaluate robustness against missing data, we randomly dropped feature sequences during inference and replaced them with zeros. Figure 10 illustrates that different models respond differently to missing data. The RNN-based approaches (LSTM and GRU) maintain relatively stable accuracy, demonstrating robustness to incomplete sequences. In contrast, 1D CNN-based models (TCN and InceptionTime) show a steeper decline, suggesting a stronger dependence on continuous data for capturing local feature patterns. Meanwhile, DTW-SVM produces inconsistent results, implying difficulty in effectively modeling partially missing signals.

The proposed dual-pathway CNN mitigates these challenges by transforming the 1D time-series data into a more robust 2D representation, thereby preserving critical information even when parts of the input are missing. As a result, it suffers a smaller performance drop than standard 1D CNN-based methods while maintaining competitive or superior accuracy compared with RNN-based models. These findings highlight the practicality of our method in real-world scenarios, where sensor noise and missing measurements are unavoidable.

5.3. Ablation Study

We conducted an ablation study to analyze the contributions of the different components in our model, including dual-pathway structure, ResNet backbone, and lateral connections.

5.3.1. Single- vs. Dual-Pathway Structure

Table 3. Comparison of accuracy for different feature combinations in the AD detection model, including single-pathway and dual-pathway architecture using gaze and head pose features. A check mark (✓) indicates the presence of the corresponding feature in the method.

Method	Gaze	Head Pose	Acc. (%)
Single-pathway
S1	✓	–	84.78
S2	–	✓	83.02
S3	✓	✓	83.93
Dual-pathway (Main pathway)
D1 (Head Pose)	✓	✓	80.05
D2 (Gaze)	✓	✓	91.09

Default settings are marked with a gray background. The best performance in each setting is highlighted in bold.

compares the single- and dual-pathway architectures using the gaze and head pose features. In a single-pathway setting, S1 (gaze-only) achieves 84.78% accuracy, while S2 (head pose-only) yields 83.02% accuracy. Combining both features in a single pathway (S3) results in an accuracy of 83.93%, which is slightly higher than S2 but lower than S1, indicating that simple stacking does not effectively leverage both modalities.

The dual-pathway architecture demonstrates a substantial performance improvement. When head pose features are used as the main pathway (D1), the accuracy drops to 80.05%, lower than any single-pathway model. In contrast, D2, which designates gaze features as the main pathway, achieves the highest accuracy of 91.09%, representing a 6.31% improvement over S1. These results highlight that gaze features are more informative for AD detection, and that modality prioritization in feature integration is critical to maximizing classification performance.

Figure 11 presents the precision–recall (PR) curves for each method. D2 yields the highest average precision (AP) of 0.914, outperforming all other configurations. Among the single-pathway models, S1 achieves an AP of 0.859, while S2 and S3 record AP values of 0.815 and 0.824, respectively. D1 shows the lowest AP at 0.800, consistent with its inferior accuracy. These findings further underscore the effectiveness of the dual-pathway structure with gaze feature prioritization, and demonstrate its robustness in balancing precision and recall, which is crucial for minimizing false positives and false negatives in clinical settings.

5.3.2. ResNet Backbone

Table 4. Performance comparison of ResNet backbones in the dual-pathway CNN model.

Backbone	Acc. (%)	Sens. (%)	Spec. (%)	F1 (%)	ROC-AUC
ResNet18	83.90	79.63	86.53	81.31	0.87
ResNet34	85.70	81.48	88.22	83.30	0.87
ResNet50	91.09	90.00	91.64	90.15	0.94

Default settings are marked with a gray background. The best performance in each setting is highlighted in bold. Metric: Acc.-Accuracy; Sens.-Sensitivity; Spec.-Specificity.

presents a performance comparison of the different ResNet backbones. Increasing the depth from ResNet18 to ResNet50 significantly improves the accuracy and F1 score, highlighting the advantages of deeper networks in capturing complex gaze and head pose patterns. However, increasing the depth of ResNet101 leads to performance degradation, possibly owing to overfitting. Based on these findings, ResNet50 was chosen as the optimal backbone to balance the accuracy.

5.3.3. Lateral Connection

Connection Type and Depth.

Table 5. Classification accuracy for different lateral connection methods and stages in our dual-pathway CNN model.

Lateral	1 → 5	2 → 5	3 → 5	4 → 5	5
Single Pathway (Gaze)	84.78
Addition	84.85	79.45	82.15	81.25	84.93
Multiplication	83.05	81.25	81.95	77.69	84.83
Concatenation	86.63	83.00	83.90	82.98	91.09

Default settings are marked with a gray background. The best performance in each setting is highlighted in bold.

shows how lateral connections (addition, multiplication, and concatenation) are performed at different convolutional layers. Early fusion (conv1, conv2_x) can disrupt low-level feature extraction and degrade overall performance, whereas connections in the final layer (conv5_x) tend to perform best. The concatenation at conv5_x achieves the highest accuracy of 91.09%.

Channel Capacity of Sub-Pathway. Finally, we vary the channel capacity of the sub-pathway (head pose) by modifying the ratio $\alpha$ with respect to the main pathway (Figure 12). The results indicate that a channel ratio of 1/8 achieves the highest accuracy (91.09%), confirming that a lightweight sub-pathway effectively enhances the feature representation of the main pathway. Moreover, reducing $\alpha$ below 1/2 consistently improves overall accuracy, whereas setting $\alpha$ to 1/2 or 1 produces no further gain, suggesting that head-pose information is more complementary than dominant in AD detection.

5.4. Effectiveness of Gaze-Saliency Map Features

We further examined the utility of our gaze-saliency map, which encodes gaze direction using hue and gaze magnitude using saturation, with a fixed intensity (value = 1). This encoding method enables an intuitive, image-like representation of gaze patterns: wide color variations indicate rapid and broad gaze changes, characteristic of healthy cognitive function. In contrast, uniform color patterns suggest slower or restricted movement, often seen in AD patients.

Figure 13 compares the saliency maps of HC and AD participants. HC participants (left) exhibit dynamic color shifts, reflecting diverse gaze transitions and frequent directional changes. However, AD participants (right) show more uniform and repetitive color patterns, indicating constrained gaze behavior. These results reinforce that healthy individuals display more flexible and dynamic gaze behavior, while AD patients exhibit limited and repetitive eye movements, aligning with cognitive decline.

The saliency map can provide an interpretable, diagnostic-rich representation of gaze behavior, further demonstrating its value as a marker for AD detection.

6. Discussion

This study proposed a novel dual-pathway framework that integrates gaze and head movements for AD detection. Our results demonstrate that gaze behavior, particularly when prioritized in the dual-pathway design, serves as a sensitive biomarker for cognitive impairment, achieving substantial improvements over single-modal or traditional 1D time-series models. To enhance feature representation, we introduced GazeMap, a transformation that converts 1D gaze and head pose time-series data into structured 2D spatial representations. This approach effectively captured both short- and long-term temporal interactions while mitigating the impact of missing or noisy data. Furthermore, gaze-saliency maps provided interpretable visualizations that revealed dynamic range differences between HC and AD participants, underscoring the potential for bridging deep learning models with clinical insight.

Traditional gaze heatmap approaches typically aggregate fixation points or gaze durations over spatial coordinates to produce static visual distributions, indicating where participants tend to focus. While effective for summarizing overall attention, these representations discard the temporal structure of gaze behavior, treating fixations as unordered spatial events. In contrast, GazeMap constructs a temporally ordered two-dimensional representation, where one axis corresponds to time and the other encodes spatial deviation. This design explicitly preserves the dynamic progression of gaze and head pose over time, enabling the model to capture fine-grained behavioral patterns associated with cognitive impairment. Rather than simply indicating where attention was concentrated, GazeMap reveals how gaze and head movements track, providing richer information for deep learning models to exploit.

Nevertheless, several limitations need to be acknowledged. Although the dataset employed in this study provided a solid foundation for initial validation, its size remains relatively modest for clinical translation. The participants were limited in number and demographic diversity, which may constrain the generalizability of our findings to broader populations. Moreover, the structured gaze tracking task, centered around a single moving target, while valuable for controlled experimentation, may not fully capture the complexity and variability of gaze behavior in naturalistic environments. Finally, while our gaze-saliency maps offer encouraging steps toward model interpretability, they currently lack grounding in clinically standardized quantitative metrics, which are necessary to translate qualitative insights into practical diagnostic tools.

Building on these insights, future work will focus on expanding and diversifying the dataset, both in terms of participant demographics and cognitive conditions, to better evaluate the specificity and robustness of gaze-based biomarkers. Moving beyond structured tasks, we aim to explore more naturalistic settings, such as free-viewing of complex scenes or real-world navigation, to capture richer gaze and head movement patterns. In parallel, deeper collaboration with clinical experts will be pursued to define interpretable, quantifiable metrics derived from gaze and head dynamics—such as fixation duration distributions, saccadic amplitudes, and head stabilization indices—that can directly inform diagnostic decision-making.

7. Conclusions

This paper introduced a novel gaze-based AD detection framework that integrates gaze and head pose analysis using a dual-pathway CNN. By transforming 1D time-series data into 2D representations, our method efficiently captured both short- and long-term temporal information while mitigating missing or noisy segments through spatial interpolation. Experimental results demonstrated that combining gaze and head pose in this two-pathway architecture substantially boosted the diagnostic accuracy to 91.09%, outperforming baseline approaches and offering a 6.31% gain over gaze-only methods. A key strength of our framework is its ability to visualize gaze features as saliency maps, enhancing clinical interpretability and bridging the gap between deep learning models and practical diagnostic applications. The proposed model provided a non-invasive and cost-effective alternative that can be deployed in real-world settings using standard webcams. For future work, our goal is to develop a scalable, interpretable, and accessible tool for early cognitive health monitoring and intervention.