The findings of the current research can be undertaken in two parts. The first set of findings consists of the observations from the eye-tracking experiment. In the subsequent part, we present the findings of forecasting models utilized with the data extracted from the eye-tracking experiment.
4.1. Observations in Eye-Tracking Experiment
Before extracting the data out of the eye-tracking for forecasting, the recordings were visually examined in the Tobii Pro Lab biometric research software to identify any significant findings or patterns. The below observations are extracted from eye-tracking recordings:
Observation 1. In the scenario provided, the participants demonstrate a greater tendency to attend to product reviews that align with their consumption motives, exhibiting longer fixations compared to other reviews. For instance, those who examine a utilitarian scenario tend to direct their attention more toward attribute-based parts of the product reviews about the nutritional value of the product. On the other hand, those who examine a hedonic scenario tend to direct their attention more toward experience-based parts
(such as feeling special, a great snack, perfect taste, etc.
) of the reviews. Examples can be observed in the heat maps given in
Figure 3 and
Figure 4 below.
Observation 2. Regardless of whether the participants make the decision with hedonic or utilitarian motives when they are purchasing for someone else (Scenario 2: utilitarian and for someone else; Scenario 4: hedonic and for someone else), in addition to the nutritional values, specific attention has been observed to keywords that could be attributed to hedonic motives such as
feeling special, handmade chocolate,
unique-original shape,
in school,
during sports,
friends as well as the appearance of the product. These can be observed in the heat maps presented in
Figure 5 and
Figure 6.
Observation 3. The protein content of the product was a notable keyword for almost all participants, regardless of the scenario they were given (see
Figure 3,
Figure 4,
Figure 5 and
Figure 6 above). On the other hand, the heat map-based visual analysis suggests that participants assigned to hedonic scenarios (Scenario 3 and Scenario 4) also tended to allocate visual attention to the appearance of the product on the packaging (see
Figure 7 for the heat map).
Observation 4. A holistic examination of the gaze data from all participants, across all scenarios and purchase purposes, revealed that participants in the 18–22 and 23–27 age groups exhibited longer total fixation durations compared to those in other age groups. A graph of the total durations of fixations for each participant is presented in
Figure 8 below.
To sum up the above observations, it is evident that participants, generally, examine product reviews in line with their relevant consumption motives and allocate their attention in line with their purchase purposes. Additionally, consumers making a purchase for someone else tend to put additional focus more on specific keywords in product reviews that align with hedonic motives. Moreover, observations derived from heat maps suggest that participants assigned to hedonic scenarios may also attend to product packaging. It should be noted that these packaging observations are derived from a qualitative heat map inspection and are distinct from the subsequent forecasting analysis, which focuses specifically on review-related AOIs. Finally, younger participants tend to have longer fixation durations.
4.2. Forecasting Phase
The forecasting analysis was conducted in Python (version 3.12.8). Following data exportation from the Tobii Pro Lab biometric research software, the data were imported from Excel files and processed in Python. The scripts employed in the analyses have been made publicly available on GitHub (v.3.5.12):
https://github.com/badslab-collab/consumption-motives (accessed on 30 May 2026).
The full dataset consists of 21 input variables and 1 output variable associated with 128 participants: fixation counts of 10 review-related Areas of Interest (AOIs), fixation durations of 10 review-related Areas of Interest (AOIs), and the assigned scenario (1 to 4) as inputs and purchasing decisions (0–1) as the output. As the forecasting phase focuses on review-based visual attention, product image and packaging-related gaze metrics were not included as predictive features in this dataset. We utilized three datasets with different features and six machine-learning methods—Artificial Neural Networks (ANN), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes and Logistic Regression—for each dataset to obtain purchasing decision forecasts. The target variable for all models is the purchasing decision and it is a binary variable (purchase: 1, do not purchase: 0). Before constructing the forecasting dataset, the eye-tracking data were screened based on predefined data-quality criteria. Participants with gaze sample acquisition rates below 80% and those whose calibration/validation accuracy error exceeded 0.9° were excluded from the dataset. No additional outlier removal or normalization procedure was applied after this eye-tracking data-quality screening.
The dataset, derived from our several-minute-long, in-person eye-tracking experiment involving 128 participants, represents a robust sample size within the context of conventional eye-tracking research [
81]. Nonetheless, in the domain of machine learning, substantially larger datasets are generally required to mitigate the risk of overfitting. To address this issue, studies using eye-tracking data, such as ours, typically employ validation methods, including k-fold cross-validation and Monte Carlo validation. K-fold cross-validation refers to partitioning a dataset into k disjoint subsets and iteratively using each subset for validation [
82]. Monte Carlo validation, on the other hand, involves splitting the dataset into two subsets at a predetermined ratio and repeatedly evaluating the model across a predefined number of iterations [
83]. Previous studies have shown that these approaches are effective in reducing overfitting when applying machine-learning techniques to small-scale eye-tracking datasets. For example, Wang et al. [
84] applied machine learning to a relatively small eye-tracking dataset derived from a controlled laboratory experiment by using a k-fold cross-validation procedure that protected their model against overfitting and ensured stable and reliable performance estimates for predictive models. To conduct a robust and comprehensive validation of our models, we applied both 10-fold cross-validation, which is deemed sufficient in the literature [
83], and Monte Carlo validation with 200 repetitions for all models except the ANN. In these models, the two procedures were integrated by performing a full 10-fold cross-validation within each Monte Carlo sampling iteration, thereby providing a multilayered assessment of model generalizability. However, we adopted a different validation strategy for the ANN model due to its statistical properties. Artificial neural networks are highly parameterized models and are sensitive to the size of the validation set. When k-fold cross-validation is applied to small datasets, increasing the number of folds can reduce the size of the validation sample, leading to unstable performance estimates. In contrast, Monte Carlo cross-validation does not suffer from this limitation, since it evaluates model performance through repeated random sampling while keeping the validation set size fixed, thereby providing more stable error estimates for artificial neural network models [
85]. Therefore, 10-fold cross-validation was not implemented for the ANN; instead, the model was evaluated exclusively through Monte Carlo validation with 200 repetitions, using repeated random sampling as the validation strategy.
The same train–test partitions were used across all machine-learning algorithms to ensure fair comparison, reproducibility, and controlled model evaluation. This design choice prevents performance differences from being driven by different random data splits and allows the observed differences to be attributed more directly to the algorithms themselves. Accordingly, the validation-specific majority-class baseline was identical across models within each dataset. In this study, this baseline was calculated as 53.85% and used as the naive benchmark for interpreting model accuracy.
Regarding model specification, hyperparameter optimization was not performed; instead, the models were implemented using standard or near-default parameter settings. This choice was made to avoid adding further model-selection complexity to a relatively limited experimental dataset, as hyperparameter tuning on small samples may increase the risk of overfitting during model selection [
86]. Therefore, standard parameter settings were preferred to support model generalizability.
In addition to hyperparameter settings, class distribution was also considered before model estimation. Class imbalance handling methods were not applied. The dataset included 78 positive and 50 negative purchase-decision observations, and this imbalance was not considered severe. Since undersampling could lead to information loss and oversampling could increase the risk of overfitting in a relatively limited dataset [
87], the original class distribution was preserved during the experiments. This full-sample class distribution was used as descriptive context, while the validation-specific majority-class baseline was used as the direct benchmark for interpreting model accuracy.
The forecasting results were presented and evaluated using three performance metrics: best accuracy, overall accuracy, and F1 scores. Given the observed full-sample class distribution of 78 positive and 50 negative purchase-decision observations, a majority-class classifier would achieve approximately 60.9% accuracy at the full-sample level. However, because the models were evaluated using fixed train–test partitions and repeated validation, the validation-specific majority-class baseline provides the appropriate benchmark for interpreting model accuracy. In this study, this validation-specific baseline was 53.85%. Therefore, the reported accuracy values are interpreted in relation to this benchmark rather than only in relation to the full-sample class distribution or a random-binary baseline. Accuracy is defined as the percentage of correct predictions, both true positives and true negatives, made by the model in relation to the total number of predictions made by the model [
88]. Best accuracy refers to the highest accuracy value obtained from a model throughout the repeated validation process. Overall accuracy, on the other hand, refers to the mean accuracy value obtained across the forecasting procedure. It has been demonstrated that the best accuracy metric can be deceptive when employed in isolation, as it presents a single value and overlooks the broader performance of the model. In particular, Valverde-Albacete and Peláez-Moreno [
89] demonstrate that classification accuracy can fail to capture crucial aspects of model performance, even when it is high, indicating that high accuracy does not necessarily imply better predictive capability. Therefore, it is employed as a form of supplementary information to facilitate the evaluation of overall accuracy, thereby addressing all the results obtained within the forecasting procedure. The F1 score is a metric used to measure the predictive power of a model, with higher scores indicating better predictive ability. F1 score is calculated as the harmonic mean of precision, defined as “the proportion of positive predictions that are correct”, and recall, which signifies the “percentage of actual positive cases that are correctly identified” by a particular model [
88]. The details of the datasets and model results are explained below:
Dataset 1. The initial forecasting results were obtained using 11 (fixation counts of 10 AOIs and scenario information) features and 1 target. The scenarios were incorporated into the models as categorical features. The output is the target, which represents the participant’s decision to purchase the product. After the models were trained, the participants’ decisions were forecasted using six algorithms. The accuracy values of the initial forecasts, representing the accurate forecasts out of 100%, are presented in
Table 2. In forecasting with the fixation count metrics of all participants for 10 AOIs, and information on scenarios (1–4), SVM has shown the best overall accuracy of 60%, and the second-highest overall accuracy (58%) occurred with Logistic Regression. The best accuracy metrics reached up to 92% with this dataset regarding the ANN, SVM, Naive Bayes, and Logistic Regression methods. Random Forest was found to be the worst performer (51%) in both accuracy metrics. These results indicate that SVM and Logistic Regression achieved the strongest classification performance among the algorithms evaluated for fixation-count-based features.
Dataset 2. Following the initial results, the fixation counts feature was replaced with another fixation metric, namely fixation durations. The data comprises 11 inputs (fixation durations of 10 AOIs and scenario information) and 1 target (purchase decision). The model training was repeated, and the accuracies given in
Table 3 were attained. These findings suggest that fixation duration alone did not provide a clear improvement over fixation-count-based features in the present forecasting setting.
The substitution of the fixation count feature with fixation duration resulted in little to no enhancement of the overall accuracy of the Naïve Bayes, ANN, and Random Forest methods, while the remaining methods showed a slight decrease when compared with the findings of the preceding analysis with Dataset 1. On the other hand, the best accuracy of the ANN, SVM, Naïve Bayes, and Logistic Regression models declined compared with Dataset 1, while the Random Forest model remained unchanged. The highest best accuracy (92%) was observed in KNN. SVM retained the highest overall accuracy in Dataset 2, at 59%, which was slightly lower than in Dataset 1. Similarly, the Random Forest algorithm exhibited the lowest overall accuracy, as previously observed.
Dataset 3. The final dataset comprises a full dataset of 21 inputs (fixation counts of 10 AOIs, fixation durations of 10 AOIs, and scenario information) and 1 output (purchase decision). As with the previous datasets, the categorical feature was taken as scenarios (1 to 4) assigned to participants. The training and testing procedures were repeated. The accuracy results are presented in
Table 4. Dataset 3 provided comparable overall accuracy to Dataset 1, with slight improvements for ANN, Random Forest, KNN, and Logistic Regression. However, Random Forest still yielded the lowest overall accuracy among the evaluated methods (see
Table 2,
Table 3 and
Table 4).
Table 5 summarizes the accuracy measures over three datasets, with the reported ranges showing the minimum and maximum values observed for each method. A consistent ranking pattern is observed in terms of overall accuracy, with Support Vector Machine (SVM) exhibiting the highest average accuracy across the datasets, achieving an overall accuracy of around 59–60%, while the Random Forest consistently yields the lowest results. Moreover, the differences in overall accuracy among the methods are relatively small; the best accuracy ranges largely overlap across methods, suggesting that this measure alone offers limited discriminatory value for forecasting performance.
To further address the interpretation of model accuracy, Support Vector Machines (SVM), which achieved the highest overall accuracy, and Logistic Regression, included as an interpretable benchmark model, were compared with the validation-specific majority-class baseline. The results are presented in
Table 6. The complete model-level comparison for all algorithms is provided in
Appendix A.3.
As shown in
Table 6, SVM consistently achieved the highest overall accuracy and exceeded the validation-specific majority-class baseline by 5.15 to 6.15 percentage points across the three datasets. Logistic Regression also outperformed the baseline in all datasets, with improvements ranging from 2.15 to 5.15 percentage points. These results indicate that SVM and Logistic Regression, particularly SVM, provided a modest improvement over the naive majority-class benchmark.
The forecasting results show that the three datasets, which integrate different review-related AOI-based features with scenario information, yielded modest but informative classification outcomes. Fixation counts with scenarios (Dataset 1) achieved one of the highest overall accuracy levels observed in the study and yielded results comparable to those of the combined feature set, suggesting that fixation counts may contain useful behavioral information for purchase-decision classification. When fixation durations were considered separately (Dataset 2), the outcomes indicated limited additional forecasting value. Combining fixation counts and durations with scenarios (Dataset 3) yielded similar performance, with a slight increase for Logistic Regression, but did not substantially surpass the best levels observed with fixation counts alone. These results suggest that, within the scope of this research, fixation counts play a central role in forecasting accuracy, while fixation durations may provide complementary information when combined with fixation counts.
Regarding the methods, Artificial Neural Networks, Logistic Regression, and Naive Bayes attained the highest accuracy values in some validation runs, reaching up to 92%. However, these peak values should be interpreted cautiously, as the overall accuracy provides a more stable indicator of model performance. In terms of overall accuracy, Support Vector Machines achieved the highest average performance across all datasets. Logistic Regression also achieved competitive results, ranking second across all datasets. Random Forest, on the other hand, consistently yielded lower performance across datasets, making it the least favorable approach in this context.
As an additional diagnostic check, confusion matrices for selected best-performing model runs are provided in
Appendix A.4. These matrices illustrate the models’ classification behavior for purchase and non-purchase decisions under selected validation runs.