4.3.1. Feature Effectiveness and Synergy in Multi-Feature Transitions
We compared our model under different feature configurations in the process coefficient. Specifically, four model variants are evaluated: MTF-c, which incorporates only category features; MTF-a, which integrates area features; MTF-pt, which emphasizes user check-in pattern features; and the full MTF-POI model, which combines all features (category, area, and pattern) into a unified framework.
The experimental results are presented in
Table 6. It can be observed that integrating multiple features yields better overall performance, as the combined model (MTF-POI) consistently outperforms the individual feature-based variants across all evaluation metrics.
The area-based variant (MTF-a) achieves the best performance among single-feature models, outperforming both the category-based (MTF-c) and pattern-based (MTF-pt) configurations across all evaluation metrics. This finding confirms that spatial correlation is a dominant factor influencing user mobility and that clustering POIs into representative areas enables the model to better capture users’ movement tendencies within geographically coherent regions.
More importantly, the combined model (MTF-POI) that integrates category, area, and pattern features yields the highest accuracy overall. Specifically, Recall@1 increases from 0.2084 (MTF-a) to 0.2480 for NYC and from 0.2300 to 0.2474 for TKY, indicating an additional gain of +3.96 and +1.74 percentage points, respectively. This improvement highlights a synergistic interaction among the three features: the area feature provides spatial stability, the category feature contributes semantic context, and the pattern feature refines temporal and behavioral consistency. Together, they enable the model to precisely predict the most probable next POI at the top-1 rank while maintaining steady gains in higher K metrics (Recall@3, Recall@5, and NDCG).
- (a)
Total Performance Score: TPS
The
in Equation (
21) was introduced to provide an integrated evaluation of model performance across multiple metrics and datasets. Specifically, the
reveals consistent performance trends across feature configurations. Among the evaluated variants, MTF-a achieves the highest TPS on both the NYC and TKY datasets, resulting in the largest overall TPS score. The ordering of TPS values (MTF-a > MTF-pt > MTF-c) remains stable across both datasets, demonstrating the robustness of feature importance rankings and reinforcing the generalizability of the proposed feature design. Overall, these results validate the effectiveness of incorporating richer contextual and temporal signals to enhance the next POI prediction performance.
- (b)
Normalized Performance Weight: NPW
The
, as defined in Equation (
22), quantifies the relative contribution of each feature by integrating its predictive performance across both datasets and all evaluation metrics. First, the score
is computed separately for each dataset by averaging the
scores over the evaluation cutoffs
k. This results in a unified performance score that reflects the overall effectiveness of feature
.
- (c)
The penalty factor
The weight
is then obtained by normalizing
with respect to the total score of all features, ensuring that the weights are comparable and collectively sum to one. Finally, the constraint
guarantees that the category, area, and pattern features together account for the full proportion of the performance contribution of the model, as Equations (
23) and (
24).
The model introduces a penalty factor to balance the contribution of different features. This design is consistent with feature weighting strategies, in which features are assigned different levels of importance based on their relevance to representing the underlying patterns [
23]. The value of
is determined empirically to achieve stable performance. Overall, the results in
Table 7 indicate that a moderate penalty factor (
) improves top-ranked recommendation quality while maintaining stable performance across other metrics. As Recall@1 represents the most stringent evaluation scenario—where the model must correctly predict the next POI as the top-ranked recommendation—we selected the configuration that yields the highest Recall@1 performance. However, the optimal value of
may vary across datasets and should be tuned according to their specific characteristics, such as data distribution and mobility patterns.
Figure 7 illustrates the effect of varying
on Recall@1 for both the NYC and TKY datasets. The results reveal that the optimal
differs across datasets: NYC achieves peak performance at
(Recall@1 = 0.2480), while TKY peaks at
(Recall@1 = 0.2474). This discrepancy is not unexpected and reflects fundamental differences in the mobility characteristics of the two cities.
Specifically, the optimal is governed by the degree of category transition noise in each dataset. NYC exhibits higher category diversity (236 active categories after preprocessing) with more dispersed visit distributions, producing noisier category transition signals that require stronger penalization () to prevent low-quality features from dominating the multi-feature score. TKY, while containing 225 categories, displays more concentrated activity in specific venue types driven by regular commuting behavior, resulting in slightly more informative category transitions that benefit from a milder penalty ().
These observations suggest that
should be interpreted as a dataset-sensitive hyperparameter rather than a universal constant. To reflect the cross-dataset generality of the framework, results in
Table 8 are reported using dataset-specific optimal values:
for NYC and
for TKY with NPW computation.
Feature synergy analysis shows the Total Performance Score for each feature variant and the normalized Performance weights for the NYC and TKY datasets. The ranking
indicates that the area feature consistently provides the highest contribution, while the normalized weights further confirm that
receives the largest weight across both datasets, as presented in
Figure 8.
4.3.2. Performance Comparison with Baselines
We compared our proposed framework, MTF-POI, with several widely used baseline methods for Next Point-of-Interest (Next-POI) recommendation. First, the evaluation is conducted exclusively on the Foursquare NYC and TKY datasets. This choice is deliberate for two reasons: (1) all baseline models compared in this study were originally benchmarked on these datasets, ensuring fair and reproducible comparison under identical experimental conditions; and (2) the proposed framework requires per-user sequential check-in trajectories that include co-located timestamps, GPS coordinates, and category labels to support area clustering (DBSCAN) and behavioral pattern detection (Isolation Forest).
Despite evaluating on a single platform, the use of two geographically and behaviorally distinct cities—New York City and Tokyo—provides a meaningful degree of cross-context validation. The two datasets exhibit contrasting mobility regimes: NYC is characterized by high venue diversity, spatially dispersed POI distributions, and irregular user schedules, while TKY displays concentrated commuting flows, higher behavioral regularity, and more predictable daily routines. This structural contrast ensures that findings are not specific to a single mobility pattern.
Results from the proposed methodology and baseline comparisons are presented in
Table 9. Experiments on two real-world datasets demonstrate that the proposed framework consistently achieves strong performance, particularly in top-1 accuracy (Recall@1) and ranking quality (NDCG). Notably, MTF-POI outperforms all baseline methods on the TKY dataset across all evaluation metrics. On the NYC dataset, while the proposed model achieves the best Recall@1 and NDCG scores, its performance on Recall@3 is slightly lower than AFNextPOI. These results suggest that the proposed framework is more effective in capturing precise next-location intent, especially for top-ranked recommendations. This behavior aligns with the design of the multi-feature transition modeling, which prioritizes precise next-step prediction over broader candidate ranking. Baseline results are partially taken from our previous work [
8].
Performance comparison between MTF-POI and baseline models on the NYC and TKY datasets is shown in
Figure 9. MTF-POI generally achieves the best performance across most metrics, particularly for Recall@1 and all NDCG metrics on both datasets. The bars corresponding to MTF-POI are highlighted with a hatch pattern, while the symbol ▾ indicates cases where MTF-POI performs slightly lower than AFNextPOI (NYC: R@3).
We assessed statistical significance using the Wilcoxon signed-rank test to determine whether differences in model performance are significant. Effect size is measured using Cliff’s
, which quantifies the magnitude of the difference in performance between two methods. Additionally, a 95% confidence interval (CI) is computed as Equation (
25).
where
denotes the sample mean,
s represents the standard deviation, and
n is the sample size.
Table 10 shows that, for the NYC dataset, MTF-POI achieves statistically significant improvements in both Recall@1 and NDCG@1 (
), indicating a stronger ability to accurately identify the correct next POI at the top recommendation position. This result suggests that the multi-feature modeling strategy effectively captures complex user mobility behaviors in dense urban environments such as New York City, where user movements are highly diverse and context-dependent. Although the improvements in Recall@3, NDCG@3, Recall@5, and NDCG@5 are not statistically significant, MTF-POI still maintains competitive, and often slightly higher, average performance, demonstrating stable recommendation quality across different ranking thresholds.
For the TKY dataset, MTF-POI shows statistically significant improvements in Recall@5 () and NDCG@5 (), indicating a clear advantage in ranking relevant POIs within the top-5 recommendation list.
This suggests that the proposed model is particularly effective in capturing broader candidate POIs within structured mobility patterns such as those commonly observed in Tokyo. The results imply that integrating heterogeneous mobility features enables MTF-POI to better model both short-term and contextual user preferences.
The overall statistical significance across metrics was further examined using Fisher’s combined probability test. The aggregated result for the NYC dataset yields , indicating that the overall performance difference between MTF-POI and AFNexPOI is not statistically significant across all metrics. In contrast, the TKY dataset shows a significant combined result (), suggesting that MTF-POI provides a statistically significant overall improvement over AFNexPOI.
4.3.3. Influence of Multifeature Feature Transition
The consistent superiority of MTF-POI across both datasets demonstrates that multi-feature transition awareness generalizes effectively under different spatial densities and urban structures. Hence, integrating all three correlated features not only enhances overall recall and ranking quality, but also strengthens the model’s adaptability to both routine and non-routine user behaviors.
The bar chart in
Figure 10 presents a comparison of Recall@1 performance across four model versions (MTF-c, MTF-a, MTF-pt, and MTF-POI) on two datasets (NYC and TKY). Each dataset has four groups of bars, representing the performance of the different version.
From the results, MTF-POI consistently achieves the highest Recall@1 in both datasets. Among the three versions (MTF-c, MTF-a, and MTF-pt), MTF-a consistently achieves the highest Recall@1 on both the NYC and TKY datasets. This indicates that the feature-enhanced version (MTF-a) performs better than the category-based (MTF-c) and pattern-based (MTF-pt) versions. By combining the three features and applying a weighted average for scoring, the performance is further improved compared to the MTF-POI version.
The contrasting results of MTF-pt between NYC and TKY reflect differences in mobility regularity and city structure. In the NYC dataset, MTF-pt performs slightly worse than MTF-c because the routine/non-routine pattern signal is not strongly correlated with the actual next-location transitions. NYC exhibits high mobility diversity, with users frequently visiting new places, changing routes dynamically, and having irregular schedules. As a result, binary pattern-type labels (routine vs. non-routine) provide little predictive power and may even introduce noise, leading MTF-pt to underperform regarding the simpler category-based feature.
In contrast, TKY displays much more structured and repeatable mobility behavior, driven by predictable commuting flows, concentrated activity zones, and higher regularity in daily routines. In such an environment, pattern-type information becomes highly valuable: whether a movement is routine or non-routine strongly influences the likelihood of the next POI. This makes MTF-pt substantially outperform MTF-c in the TKY dataset.
4.3.4. TPS Statistical Validation
Spearman’s rank correlation coefficient (
) was used to measure the monotonic association between TPS rankings and the individual evaluation metrics. As defined in Equation (
26), this coefficient quantifies the monotonic relationship between two ranked variables [
26].
where
denotes the difference between the ranks of the paired observations and
n is the number of samples.
Kendall’s rank correlation coefficient (
) was also used to assess the ordinal association between TPS rankings and individual evaluation metrics. As defined in Equation (
27), Kendall’s
measures the difference between the number of concordant and discordant pairs [
26,
27].
Figure 11 shows the rank correlation analysis between TPS and individual evaluation metrics across the NYC and TKY datasets. The first two heatmaps show the rank matrices of model variants across Recall, NDCG, and TPS metrics, with rank 1 indicating the best performance. The third heatmap shows the Spearman correlation between TPS and individual metrics, demonstrating moderate to perfect rank consistency.
The Spearman (
) and Kendall (
) correlations between the Total Performance Score (TPS) ranking and the individual evaluation metrics across feature configurations for both datasets are shown. Overall, these results indicate that TPS provides a consistent aggregated indicator of model performance across multiple evaluation metrics presented in
Table 11.
For the NYC dataset, moderate correlations are observed for Recall@1, Recall@3, and NDCG@1 (, ), indicating slight differences in ranking compared with TPS. In contrast, Recall@5, NDCG@3, and NDCG@5 exhibit perfect rank agreement (, ). For the TKY dataset, all metrics show perfect correlations with TPS, suggesting that the TPS ranking fully aligns with the rankings produced by the individual metrics.
4.3.5. Influence of Hidden Location Attraction
The MTF-c-AHLTP follows the same modeling process as MTF-c, where next-POI prediction is based on category transitions. The key difference is that MTF-c-AHLTP augments user trajectories with hidden locations inferred from category associations using the AHLTP [
15] approach prior to model training.
The comparison between MTF-c and MTF-c-AHLTP in
Figure 12 shows that incorporating hidden locations does not improve top-rank accuracy or ranking quality. While MTF-c-AHLTP slightly increases candidate coverage at higher K values, it introduces noise into category transitions, leading to lower Recall@1 and NDCG@K. These results suggest that, for large-scale next POI prediction, using only real check-in transitions is more effective than augmenting trajectories with inferred locations.
More importantly, this result indicates that addressing uncertain check-ins through feature transfer is more effective than explicitly inserting inferred locations into user trajectories. Rather than modifying the original trajectory structure, feature transfer enables the model to adaptively reweight or transform existing features based on contextual and behavioral information, thereby mitigating uncertainty without amplifying error propagation. In contrast, insertion of hidden location alters transition sequences by introducing inferred check-ins that may not reflect the user’s true short-term intent, leading to poor ranking performance.
These findings suggest that, for large-scale next-POI prediction, modeling uncertainty at the feature level provides a more robust solution than augmenting trajectories with inferred locations.