Improving Plausibility of Coordinate Predictions by Combining Adversarial Training with Transformer Models ^†

Abstract

Due to the significant potential of crowd flow prediction in the domains of commercial activities and public management, numerous researchers have commenced investing in pertinent investigations. The majority of existing studies employ recurrent neural networks, long short-term memory, and similar models to achieve their objectives. Despite the advancements in predictive modeling, the objective of many existing studies remains in the minimization of distance errors. This focus, however, introduces three notable limitations in prediction outcomes: (1) the predicted location may represent an average of multiple points rather than a distinct target, (2) the results may fail to reflect actual user behavior patterns, and (3) the predictions may lack geographic plausibility. To address these challenges, we developed a Transformer-based model integrated with adversarial network architecture. The Transformer component has shown considerable effectiveness in forecasting individual movement trajectories, while the discriminator within the adversarial framework guides the generator in refining outputs to better reflect user habits and spatial rationality.

1. Introduction

Next-location prediction focuses on estimating a user’s future position from their past movement data. In the increasingly connected digital world, accurate location prediction provides substantial benefits, aiding services such as personalized advertising based on location, analyzing consumer behavior in shopping districts, and estimating crowd flow in specific regions. These uses are vital not just for commercial strategies but also for public safety and disaster management.

Luca et al. [1] examined next-location prediction and crowd flow forecasting. Their results showed that next-location prediction depends on factors such as users’ past behavior, popular sites within the area, and geographic context, among which past behavior has the strongest and most regular influence. Traditional methods have used statistical models and machine learning. However, with the growing size and complexity of mobility data, deep learning has become the dominant approach. Despite this, the absence of standardized benchmarks and sensitivity to data completeness make it difficult to fairly compare model performance across different studies.

In deep learning, recurrent neural networks (RNNs), long short-term memory (LSTM), and Transformer models are commonly used for sequential prediction tasks. RNNs, however, tend to suffer from gradient explosion during long-term sequence modeling. LSTM has been developed to effectively address this problem, but its recursive structure results in higher computational costs, especially as data sizes increase. Transformer models provide a more scalable solution by using self-attention mechanisms, which improve training efficiency and prediction accuracy. Transformer models outperform LSTM and BiLSTM models in predicting user destinations [2].

Nonetheless, most Transformer-based models are optimized to minimize the distance error between predicted and actual coordinates. Although this approach can produce numerically accurate results, it presents three critical limitations in practical applications. First, predictions converge toward the average of multiple plausible destinations, resulting in outputs that fall within unrealistic intermediate zones. Second, the predicted locations fail to reflect habitual user trajectories; for example, a user who consistently follows a specific daily route might receive predictions that deviate from their actual path, thereby overlooking individual preferences and behavioral patterns. Third, even when distance errors are low, the predicted coordinates fall in geographically implausible areas, such as bodies of water or undeveloped land, rendering them unsuitable for real-world deployments.

To address these issues, we incorporated an adversarial network into a Transformer model to enhance the reality and geographic plausibility of predicted results. During training, a discriminator is introduced to evaluate whether the coordinates predicted by the Transformer resemble real-world locations. Rather than merely minimizing the distance error, the Transformer is trained to deceive the discriminator by producing predictions that are indistinguishable from real samples. Different from conventional generative adversarial networks, this approach does not rely on random noise for data generation. Instead, the Transformer serves as the predictive generator. The discriminator guides the Transformer to generate user-consistent and contextually appropriate predictions. As a result, the proposed model not only achieves low distance error but also overcomes the aforementioned limitations, thereby improving its practical applicability.

2. Related Work

The dataset utilized in this study was obtained from the 2024 HuMob competition, which was first held in 2023 and subsequently continued in 2024. The organizers provided a large-scale mobility dataset consisting of the trajectories of 100,000 users over 90 days. The objective of the competition was to predict users’ future coordinates, with evaluation based on two primary metrics: GeoBLEU [3] and dynamic time warping (DTW) [4], which assesses the spatial and structural consistency of the predicted sequences. In the 2023 competition, Solatorio et al. [5] proposed the GeoFormer model, which relied solely on the encoder module of the Transformer architecture and achieved strong performance in the final rankings.

In recent years, a growing body of research has applied Transformer models to personalized coordinate prediction. Xue et al. [6] proposed a Transformer framework capable of jointly predicting a user’s next point-of-interest (POI) and its corresponding coordinates. Qin et al. [7] employed a Transformer architecture to forecast the destination coordinates of taxi users. Mayemba et al. [8] reviewed the use of language models for human mobility prediction during infectious disease outbreaks, highlighting the effectiveness of Transformer-based methods in coordinate predictions and reporting preliminary studies incorporating large language models (LLMs). Due to its strength in modeling long-range dependencies, the Transformer model has become the dominant approach. However, most Transformer models are optimized solely to minimize distance-based errors, with performance typically evaluated using metrics such as mean square error and mean absolute error. While these measures capture numerical accuracy, they do not necessarily reflect the geographic plausibility of the predicted locations.

To address this limitation, researchers have introduced adversarial network architectures into deep learning models to enhance the realism of the predicted trajectories. Different from traditional generative adversarial networks, which take random noise as input, these approaches use only historical trajectory data as both input and output. The discriminator provides feedback to the main prediction model, guiding it to avoid producing overly averaged results and encouraging outputs that are more realistic and semantically coherent. For instance, Gao et al. [9] proposed the DeepTrip framework for travel itinerary recommendation, where an RNN-based model was trained using adversarial feedback to generate more human-like mobility sequences. Similarly, Jiang et al. [10] developed a Transformer-based route recommendation system enhanced with adversarial training, enabling the model to generate routes that better satisfy user constraints. Their experiments across multiple datasets demonstrated improved user satisfaction and the plausibility of the generated itineraries.

3. Datasets

We utilized the dataset provided by the 2024 HuMob Challenge (Human Mobility Prediction Challenge) to evaluate the proposed method. The competition offers user mobility data across four cities, where each coordinate corresponds to a 200 × 200 spatial grid. Each recorded coordinate indicates a user’s location at the time of application usage. The number of users per city is summarized in

Table 1. Number of users across cities.

City	A	B	C	D
Number of users	100,000	25,000	20,000	6000

. The dataset consists of five columns: UserID, day, time, x, and y, as shown in

Table 2. Trajectory dataset format.

UserID	Day	Time	X	Y
0	0	0	80	160
0	0	1	81	169
0	0	2	81	169
0	0	8	86	150
…
24999	74	47	999	999

. The data spans from day 1 to day 75. In this study, we used the data of City B, which includes trajectory records from 25,000 users. For full details on these datasets please refer to [11,12].

3.1. Scope of Dataset Utilization

According to the competition requirement, the records of the last 3000 users in the dataset were intentionally masked. The target columns for prediction in the challenge are marked with 999, and their ground truth values are not accessible. As a result, users whose data contained masked values were excluded from this study. After filtering, a total of 22,000 users from City B were retained for model evaluation.

3.2. User Behavior

At this stage, we analyzed whether users exhibited regular patterns in their mobility behavior. For each user, we identified their top five most frequently visited locations and compared the total visit count of these locations with that of all locations visited by the user, as shown in Equation (1). Based on this ratio, users were categorized into four groups, as shown in

Table 3. User grouping based on location regularity.

Group	Percentage	Number of Users
1	>80%	2131
2	50–80%	9981
3	20–50%	9330
4	<20%	558

.

$$Proportion={\displaystyle \frac{{\sum }_{i=1}^{5}Visit to Top i Locations}{Total Visits}}$$

(1)

The results indicate that most users exhibit consistent movement patterns: over 55% of users had more than half of their location records clustered in their top five most visited places. Users in Group 1 had over 80% of their records concentrated in these locations, showing highly regular behavior. Group 2 users had a lower concentration ratio than Group 1, possibly because they visited a greater number of locations frequently, but they still followed regular patterns. Group 3 users displayed some regularity, while Group 4 users showed highly irregular movement.

This classification suggests that most users in the dataset follow predictable mobility patterns, supporting the feasibility of forecasting their future locations based on historical trajectories.

4. Proposed Method

We developed a personalized coordinate prediction framework that integrates a Transformer encoder with adversarial training to enhance the plausibility of predicted coordinates. The training and prediction process is divided into two stages: offline processing and online processing. The offline processing stage includes data preprocessing, normalization, user clustering, and model training. In the online processing stage, the trained model is used to predict the coordinates of a given user in the test data, followed by performance evaluation (Figure 1 and Figure 2).

4.1. Data Preprocessing

The dataset used in this study was provided by the competition organizers and is therefore clean and free of outliers. However, according to the competition requirements, the data of 3000 users were intentionally masked. Since the true coordinate values for these users are unavailable, we excluded all records related to them during preprocessing. Subsequently, we performed user clustering, as we observed significant differences in usage patterns among users. For example, some users exhibited highly regular behavior, whose locations at the same time of day were nearly identical, while others showed highly irregular and random movement patterns. To prevent the model from being biased or underfitting due to mixed user behavior, we categorized users into four groups, as shown in Table 3.

4.2. Model Training

We adopted a deep learning model based on a Transformer encoder, combined with adversarial training, to improve the plausibility and accuracy of next-time-point coordinate prediction. Based on the user clustering results described in Section 4.1, we train a separate model for each user group. Each model consists of a Transformer model for coordinate prediction, and a discriminator to evaluate the plausibility of the predicted coordinates. The input to the model includes the day of the week (w_t), time of day (t_t), and the past 12 h coordinates (X_t). The overall input representation can be formulated as Equation (5).

$$w_t\in \left\{0,1,2,3,4,5,6\right\},$$

(2)

$$t_t\in \left\{0,1,\dots ,47\right\},$$

(3)

$$X_t=\left[(x_{t-48},y_{t-48}),\dots ,(x_{t-1},y_{t-1})\right],$$

(4)

$$Z_t=\left(w_t,t_t,X_t\right),$$

(5)

All input features are first passed through a linear projection layer and positional encoding, then fed into the Transformer encoder. The Transformer encoder consists of multiple stacked layers, where each layer comprises the following components: (1) multi-head self-attention, (2) feed-forward network, and (3) add and layer normalization. The overall architecture of the model is summarized in Equation (6).

$${\widehat{y}}_t=({\widehat{x}}_t,{\widehat{y}}_t)=\mathit{Linear}(\mathit{Pool}(\mathit{TransformerEncoder}(Z_t)\left)\right).$$

(6)

The discriminator determines whether the predicted coordinates generated by the Transformer are plausible by comparing them against real user data.

The discriminator distinguishes between real coordinates and those predicted by the Transformer, assessing whether the results are consistent with actual user behavior, as formulated in Equation (7). The loss functions used for training the model are defined in Equations (8) and (9).

$$D(X_t,y)=\sigma \left({\mathit{Linear}}_3\circ \mathit{ReLU}\circ {\mathit{Linear}}_2\circ \mathit{ReLU}\circ {\mathit{Linear}}_1\circ \right(\left[\mathit{flatten}(X_t\right),y]\left)\right),$$

(7)

$$L_{Discriminator}=-\mathit{log}D(X_t,y_t)-\mathit{log}(1-D(X_t,{\widehat{y}}_t)),$$

(8)

$$L_{TransformerEncoder}=D(X_t,{\widehat{y}}_t)+\lambda \cdot {‖y_t-{\widehat{y}}_t‖}_2^2$$

(9)

4.3. Evaluation

We used two evaluation metrics, GeoBLEU and DTW to assess the plausibility and accuracy of the predicted trajectories. These metrics are also used as the official evaluation criteria in the HuMob Challenge 2023 and 2024. GeoBLEU, proposed by Shimizu et al. [3] in 2022, is used to measure the similarity between the predicted and actual trajectories. Unlike traditional distance-based metrics, GeoBLEU considers both spatial closeness and semantic plausibility. The score ranges from zero to one, with values closer to one indicating higher similarity. DTW is widely used for comparing temporal sequences. It allows flexible alignment of time indices, making it suitable for evaluating human mobility trajectories. Lower DTW values indicate higher similarity between the predicted and actual sequences.

5. Result

To evaluate the feasibility of the proposed method, we built separate models for each user group and independently trained and assessed their prediction performance.

Table 4. Performance comparison between proposed method and baseline transformer model.

Group	Proposed Method		Baseline Transformer Model
	Average GeoBLEU	Average DTW	Average GeoBLEU	Average DTW
1	0.3206	2.6181	0.1535	3.7211
2	0.2596	3.6089	0.1181	4.3335
3	0.2137	4.9492	0.1071	5.1409
4	0.1498	6.5339	0.0872	5.8799
Total Average	0.2359	4.4275	0.1165	4.7689

compares the performance of the proposed method and a baseline model using only a Transformer encoder across different groups, using GeoBLEU and DTW as evaluation metrics. The developed method showed higher GeoBLEU scores than the baseline Transformer model in all four groups, indicating that the incorporation of adversarial training improves the plausibility of predicted trajectories compared to conventional models without such training. As for the DTW metric, the proposed method outperformed the baseline in three out of the four groups. In Group 4, however, the performance was worse, likely due to the lack of consistent behavioral patterns among users, which made accurate prediction particularly difficult.

On average, the developed method demonstrated better overall performance, suggesting that incorporating adversarial training into Transformer models can enhance both the plausibility and accuracy of coordinate predictions. The test dataset used in this study is different from the dataset used for evaluation in the HuMob competition. Since the target users and their feature distributions differ, the prediction results presented here are intended only for comparing model performance within this study and cannot be directly compared with the competition results.

6. Conclusions

We developed a personalized coordinate prediction method that integrates adversarial training with a Transformer-based architecture. Users are first grouped based on their behavioral patterns, and separate models are trained for each group. By incorporating a discriminator, the model encourages the Transformer to generate predictions with higher geographic plausibility and coordinate predictions. This effectively addresses the limitations of conventional models, which tend to produce overly averaged results that lack practical applicability due to reduced reality. Experimental results on the HuMob 2024 dataset demonstrate that the proposed method outperforms the Transformer-only model in both GeoBLEU and DTW. These findings confirm that integrating adversarial training enhances both the plausibility and accuracy of coordinate predictions. The application of other model architecture needs to be explored in combination with adversarial training, or incorporating additional features such as point-of-interest information to further improve predictive performance.