Dynamic Multi-Relation Learning with Multi-Scale Hypergraph Transformer for Multi-Modal Traffic Forecasting

Abstract

Accurate multi-modal traffic demand forecasting is key to optimizing intelligent transportation systems (ITSs). To overcome the shortcomings of existing methods in capturing dynamic high-order correlations between heterogeneous spatial units and decoupling intra- and inter-mode dependencies at multiple time scales, this paper proposes a Dynamic Multi-Relation Learning with Multi-Scale Hypergraph Transformer method (MST-Hype Trans). The model integrates three novel modules. Firstly, the Multi-Scale Temporal Hypergraph Convolutional Network (MSTHCN) achieves collaborative decoupling and captures periodic and cross-modal temporal interactions of transportation demand at multiple granularities, such as time, day, and week, by constructing a multi-scale temporal hypergraph. Secondly, the Dynamic Multi-Relationship Spatial Hypergraph Network (DMRSHN) innovatively integrates geographic proximity, passenger flow similarity, and transportation connectivity to construct structural hyperedges and combines KNN and K-means algorithms to generate dynamic hyperedges, thereby accurately modeling the high-order spatial correlations of dynamic evolution between heterogeneous nodes. Finally, the Conditional Meta Attention Gated Fusion Network (CMAGFN), as a lightweight meta network, introduces a gate control mechanism based on multi-head cross-attention. It can dynamically generate node features based on real-time traffic context and adaptively calibrate the fusion weights of multi-source information, achieving optimal prediction decisions for scene perception. Experiments on three real-world datasets (NYC-Taxi, -Bike, and -Subway) demonstrate that MST-Hyper Trans achieves an average reduction of 7.6% in RMSE and 9.2% in MAE across all modes compared to the strongest baseline, while maintaining interpretability of spatiotemporal interactions. This study not only provides good model interpretability but also offers a reliable solution for multi-modal traffic collaborative management.

1. Introduction

Traffic demand forecasting is a cornerstone of intelligent transportation systems (ITSs), playing a pivotal role in optimizing urban traffic management, scheduling resources, and providing real-time traveler information [1]. Accurate predictions empower authorities to mitigate congestion, reduce accidents, and enhance overall transportation efficiency. Early research primarily treated this task as a time series prediction problem, employing statistical methods like linear regression and ARIMA [2,3]. While effective for linear patterns, these approaches struggle to capture the complex, nonlinear, and spatiotemporal dependencies inherent in traffic data.

The advent of deep learning has revolutionized this field. Models such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), along with their variants (e.g., LSTMs, GRUs), demonstrated a superior capacity for modeling complex traffic modes [4]. Subsequently, graph neural networks (GNNs) [5,6] and spatiotemporal graph convolutional networks (STGCNs) [6] emerged as dominant paradigms, effectively capturing dependencies in non-Euclidean space defined by transportation networks. Despite these advancements, a significant limitation persisted: most studies focused on forecasting demand for a single transportation mode [7,8,9], overlooking the intricate interactions and spatiotemporal heterogeneity that characterize multi-modal urban systems. This narrow focus constrains the accuracy and generalizability of predictions in practical applications.

The availability of multi-source traffic data has recently spurred a shift towards collaborative forecasting frameworks. Leveraging data from multiple modes (e.g., taxi, bike, subway) enables not only improved accuracy for individual modes but also beneficial knowledge transfer from data-rich to data-sparse modes [10,11,12]. However, multi-modal forecasting faces two fundamental challenges rooted in data and model architecture.

(1)

Data heterogeneity and multi-scale spatiotemporal dynamics challenges.

The core challenge of multi-modal traffic prediction lies in its inherent data heterogeneity and multi-scale spatiotemporal dynamics. This heterogeneity is manifested in two dimensions: spatial and temporal. Spatially, data from modes such as subway and bike-sharing are organized by fixed stations, whereas taxi data are aggregated based on grids or administrative regions. This mismatch in spatial units, as visualized in Figure 1a, hinders the effective alignment and integration of multi-source data. Temporally, even under the same scale, data magnitudes vary drastically across modes (e.g., subway daily flow reaches millions while bike-sharing demand is merely thousands), as shown in Figure 1b, making temporal correlation capture challenging. Crucially, spatiotemporal dynamics exhibit complex multi-scale characteristics where node relationships shift across scales (e.g., Scale 1 vs. Scale 3 in Figure 2). However, existing mainstream temporal models, such as TCN [13] or multi-resolution attention-based methods, typically rely on fixed window sizes or linear convolutions, making it difficult to adaptively capture periodic and cross-modal interactions in traffic data.

(2)

Limitations of dynamic and high-order spatial correlation modeling.

At the model architecture level, a key challenge is designing a framework that simultaneously captures intra-mode spatiotemporal features and dynamic inter-mode interactions. Existing graph-based methods (e.g., ST-MRGNN [12]) often depend on pre-defined or static graph structures, which cannot adapt to evolving spatial dependencies triggered by traffic conditions. Moreover, these methods are limited to pairwise relationships and cannot represent higher-order associations (e.g., multiple bike stations serving a subway hub). While hypergraph neural networks (HGNNs) [14] model high-order correlations, they typically use static hypergraph constructions [14,15,16,17,18], lacking real-time dynamism.

To address these gaps, we propose the Dynamic Multi-Relation Learning with Multi-Scale Hypergraph Transformer (MST-Hyper Trans). Unlike the above methods, MST-Hyper Trans introduces a multi-scale temporal hypergraph structure that can dynamically group data based on traffic cycles (such as time, day, and week), and merge hyperedges through Sørensen–Dice coefficients to adaptively decouple intra/inter-modal dependencies. To overcome the limitations of capturing spatially heterogeneous features, we designed a Dynamic Multi-Relationship Space Hypergraph Network (DMRSHN). Unlike traditional graph models, DMRSHN innovatively integrates geographic proximity, passenger flow similarity, and traffic connectivity to construct structural hyperedges and combines KNN and K-means algorithms to generate dynamic hyperedges, thereby accurately modeling high-order spatial correlations of dynamic evolution between heterogeneous nodes. Unlike traditional paradigms of spatiotemporal feature extraction and static fusion, our model introduces a novel collaborative perception dynamic generation context fusion paradigm. In summary, the main contributions are as follows:

• Our framework leverages the Multi-scale Temporal Hypergraph Convolutional Network (MSTHCN) to extract multi-scale spatiotemporal features through collaborative temporal decoupling. Unlike existing multi-scale temporal methods (e.g., TCN [13]), which use uniform granularity and fixed convolutions, MSTHCN introduces a multi-scale temporal hypergraph structure that dynamically groups data based on traffic periodicity (e.g., hourly, daily, weekly scales) and merges hyperedges via the Sørensen–Dice coefficient. This enables adaptive capture of cross-modal temporal interactions and decouples intra-/inter-mode dependencies.

As summarized in

Table 1. Comparison of MSTHCN with existing multi-scale temporal models.

Aspect	Existing Methods (e.g., TCN [13])	MSTHCN (Our Work)	Substantive Improvement
Time scale handling	Fixed windows or linear decomposition	Multi-scale hypergraph with dynamic edge merging	Adapts to traffic periodicity via hyperedge construction (e.g., hour/day/week groups)
Cross-modal interaction	Limited, often post hoc fusion	Explicit modeling via hypergraph convolution and attention	Captures dynamic inter-mode dependencies in real-time
Adaptability	Static granularity	Attention-driven fusion of multi-scale features	Responds to traffic shifts via Sørensen–Dice similarity thresholding
Applicable scenario	General time series, struggles with heterogeneity	Tailored for multi-modal traffic data	Addresses data sparsity and magnitude variance

, MSTHCN substantially improves modeling capability by integrating hypergraph convolution and attention mechanisms, overcoming the rigidity of TCN-like approaches.

2.

We design a dynamic feature generation and fusion mechanism via the Conditional Meta Attention Gated Fusion Network (CMAGFN). Traditional multi-modal prediction models often adopt static fusion strategies and cannot adaptively adjust according to real-time traffic scenarios. We introduced the Conditional Meta Attention Gated Fusion Network (CMAGFN) for this purpose. As a lightweight meta network, it dynamically generates node features based on real-time context (e.g., time, weather) and calibrates the fusion weights of multi-source information through a gate control mechanism based on multi-head cross-attention, achieving scene-aware adaptive optimal prediction decisions and significantly improving the adaptability and interpretability of the model.

3.

We have validated the superiority, effectiveness, and practical value of MST-Hyper Trans through extensive experiments on three real-world datasets. The experimental results indicate that our model significantly outperforms a series of baseline models. More importantly, in-depth ablation research and case analysis have shown that our model can provide new insights into cross-modal spatiotemporal interaction mechanisms, particularly demonstrating excellent predictive performance in sparse data scenarios, providing key guidance for the practical deployment of intelligent transportation systems.

The remainder of this paper is organized as follows: Section 2 reviews related work. Section 3 formalizes the problem. Section 4 details the MST-Hyper Trans architecture. Section 5 present the experimental setup and results/discussion, respectively. Section 6 concludes the paper.

2. Related Works

2.1. Traffic Demand Prediction Based on Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are commonly employed in traffic demand forecasting because of their capacity to capture complex spatiotemporal patterns in traffic data. Ma et al. [19] introduced a hybrid architecture that combined CNNs and bidirectional long-term memory networks (BiLSTM) to predict passenger traffic in extensive subway networks. In 2021, Zhang et al. [20] proposed a CAS-CNN method featuring a channel attention mechanism and a split CNN to address data sparsity issues in short-term OD demand prediction for urban rail transit systems. Liang et al. [21] presented a Spatiotemporal Relational Network (STRN) for urban traffic prediction, utilizing CNN to extract local features from high-resolution urban traffic data, while incorporating global spatial dependencies and external factors through global relational modules and meta-learning modules, respectively. Qiao et al. [22] introduced a novel single-vehicle demand prediction model utilizing a Deep Convolutional Neural Network (DCNN) that incorporates diverse data sources such as weather and temporal information via a dynamic feature matrix. In the realm of multi-modal traffic demand prediction, Ye et al. [23] developed a sophisticated deep spatiotemporal neural network known as CoST-Net, which leverages convolutional layers and Long Short-Term Memory (LSTM) networks to capture spatiotemporal characteristics. Wang et al. [24] proposed the MultiST framework, employing Convolutional Neural Networks (CNNs) and gated recurrent units (GRUs) to extract spatiotemporal features across various traffic modes. Xu et al. [25] introduced the CMGAT framework by utilizing graph attention networks to model spatiotemporal dependencies among different traffic modes. Yang et al. [26] presented the Res-Transfer model, which integrates convolutional operations and attention mechanisms for feature extraction and inter-model correlation. However, CNN-based approaches require the transformation of the traffic network into a regular mesh network, thereby limiting their ability to capture spatial correlations within the traffic data.

2.2. Traffic Demand Prediction Based on Recurrent Neural Networks

Recurrent Neural Networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) and gated recurrent units (GRUs), have been pivotal in traffic demand forecasting. Lee and Ku [27] introduced a dual attention-based RNN model to predict bike-sharing demand. Wei et al. [28] integrated an attention mechanism with LSTM to capture key features in time series data and successfully predicted the passenger flow in the Hangzhou subway. Huang et al. [29] developed the MTLMetro model by utilizing LSTM within a multi-task learning framework to forecast subway demand. In the realm of multi-modal traffic demand forecasting, Zhang et al. [30] enhanced the short-term prediction accuracy of shared-bike usage by fusing historical shared-bike data with real-time public transport passenger data using LSTM. Huang et al. [10] employed a multi-modal dynamic graph convolution module and a gated loop unit to capture the dynamic effects of various traffic modes. Wang et al. [24] and Ye et al. [23] improved the multi-mode traffic demand prediction performance by extracting shared and unique spatiotemporal dependencies using technologies such as CNNs, GRUs, and deep spatiotemporal neural networks. In parallel to RNN approaches, researchers have also leveraged advanced architectures like Transformers for spatiotemporal modeling. For instance, Li et al. [31] proposed a Convolutional Transformer Network for vehicle trajectory prediction, which integrates convolutional layers for local spatial feature extraction with a Transformer encoder to capture long-range temporal dependencies. This work highlights the potential of hybrid architectures in complex urban traffic scenarios.

The emergence of deep learning technology has propelled temporal convolutional networks (TCNs) [13] to the forefront of research on traffic demand forecasting. Notably, Liu et al. [11] integrated a TCN and graph neural networks to model intricate spatiotemporal relationships among various traffic modes through multi-relationship graphs and introduced a novel approach to the collaborative prediction of multi-mode traffic demand. Similarly, Liang et al. [12] developed ST-MRGNN by leveraging TCN and a multi-graph neural network for the collaborative forecasting of multi-mode traffic demand. Building upon this, Liang et al. [32] enhanced multi-modal traffic demand forecasting by combining a TCN with a graph neural network. Furthermore, Zhang et al. [33] enhanced the performance of the joint prediction of multi-mode traffic spatiotemporal data by integrating a TCN, a graph sparse attention mechanism, and a bidirectional spatiotemporal convolutional network. Despite the notable achievements of Recurrent Neural Networks (RNNs) and their variants in traffic demand forecasting, the growing interest in TCN within this domain stems from its advantages of parallelism, flexibility, stability, and memory efficiency. However, TCN cannot adaptively group data based on the periodic flow in urban transportation systems for unified processing time scales, making it difficult to achieve the accurate prediction of peak hours for multiple transportation modes in cities.

2.3. Traffic Demand Prediction Based on Graph Convolutional Networks

Graph Neural Networks (GCNs) have transformed spatial feature extraction in traffic networks by modeling node relationships in non-Euclidean spaces. Several GCN variants have been introduced, such as STGCN [6], DCRNN [5], ASTGCN [34], and Graph WaveNet [35]. Moreover, Liu et al. [36] proposed a spatiotemporal attention graph convolutional network (ST-AGCN) for long-term public bicycle demand prediction. Gao et al. [37] developed a regularized spatiotemporal graph convolutional network (PMR-GCN) incorporating personalized enhanced graph convolutional networks and a multi-head self-attention mechanism to enhance accuracy and stability in subway passenger flow prediction. Wu et al. [38] proposed the TSHDNet, which employs a spatiotemporal heterogeneity decoupling gate to separate features from different traffic modes. Zhang et al. [39] combined Transformer networks, LSTM, and GCN to capture dependency relationships across multiple dimensions, enhancing subway passenger flow prediction performance. Additionally, Yang et al. [40] introduced DTG-MGCN, which leverages multi-graph convolution and dual time-domain gating branches to comprehensively consider nonlinear and spatial data correlations, thereby improving taxi demand prediction.

In studies focusing on collaborative forecasting of multi-mode traffic demand across heterogeneous spatial units, numerous researchers have employed graph neural networks to comprehend intricate spatial correlations among diverse traffic modes. Liu et al. [11] developed a self-learning heterogeneous traffic map alongside a space–time convolution module to autonomously discern spatial interdependencies among distinct traffic stations. They integrated multi-task learning networks to facilitate the collaborative prediction of multi-mode traffic demand. ST-MRGNN [12] introduced a multi-graph neural network to capture intricate spatiotemporal correlations among diverse traffic modes, enabling the collaborative prediction of heterogeneous traffic demand. In 2024, Li and Liu [41] devised a multi-modal traffic demand prediction model that leveraged federated learning. They introduced a fine-grained graph convolutional recurrent network (F-GCRN) to capture spatiotemporal correlations within traffic mode data. GSABT [33] established a flexible multi-mode traffic joint prediction framework by utilizing a graph sparse attention mechanism to extract spatial features, thereby enhancing the effectiveness of joint prediction.

However, a fundamental limitation persists across many of the graph-based methods mentioned above: they often rely on pre-defined or self-learning but essentially static graph structures, which struggle to capture the evolving spatial dependencies in dynamic traffic conditions. This issue is compounded when modeling associations between inherently heterogeneous spatial units (e.g., taxi regions vs. subway stations). Recent studies have thus begun to explore dynamic graph learning to address the first challenge. For example, Wei et al. [42] developed a dynamic graph convolutional network with an attention mechanism to adaptively learn the adjacency matrix.

Furthermore, the application of hypergraph neural networks has been extended to model higher-order correlations beyond pairwise relationships, offering a potential solution for complex heterogeneous interactions. The emergence of hypergraph learning techniques has, therefore, led to the adoption of hypergraph convolution methods for traffic demand prediction. Wang et al. [15] introduced a dynamic hypergraph convolutional network (DSTHGCN) to address the challenges posed by traditional graph neural networks in modeling intricate transportation networks. By constructing fundamental hypergraphs and advanced hyperedges, the DSTHGCN effectively captures the structural characteristics of subway networks and the evolving travel behaviors of passengers. Furthermore, Zhao et al. [16] presented a space–time hypergraph convolutional network (ALGHGCN) that incorporates a novel attention mechanism. By leveraging local and global hypergraph convolutions along with temporal attention mechanisms, the ALGHGCN adequately balances the local and global relationships in the data. In 2024, Ye et al. [17] developed a Dynamic Spatiotemporal Hypergraph Convolutional Network (DSTHGCN) that integrates collaborative convolution techniques from traffic flow graphs and hypergraphs to tackle the complexities associated with spatiotemporal modeling in traffic flow prediction. Additionally, Cao et al. [18] introduced the Spatial-Temporal Graph Hypergraph Convolutional Network (STGHCN), which combines hypergraph convolution with an attention mechanism to capture high-order spatial correlations across diverse traffic data types. Research on the application of hypergraph neural networks in multi-mode traffic demand collaborative forecasting is scarce. Wang et al. [43] introduced a multi-task hypergraph convolutional network (MT-HGCN) to enhance the prediction performance by leveraging multi-task learning and hypergraph convolutional networks to uncover internal correlations among various traffic mode data. Although hypergraph neural networks capture high-order correlations among multi-modal traffic nodes, there is still a need for further exploration into dynamically modeling heterogeneous traffic node graphs.

In summary, despite significant advances, the current literature on traffic demand forecasting exhibits three persistent gaps that this study aims to address. First, while graph-based methods and emerging dynamic hypergraph approaches (e.g., [15,42]) effectively model complex dependencies, they primarily focus on networks with homogeneous node types (e.g., only subway stations or only taxi zones), lacking a unified framework for collaborative forecasting across heterogeneous spatial units (e.g., regions, bike stations, subway stations). Second, existing methods often rely on static or semi-static graph structures, which are inadequate for capturing the high-order spatial correlations that evolve dynamically with traffic conditions. Third, and most critically, there is a lack of a dedicated mechanism that effectively decouples and models the intra- and inter-mode dependencies evolving over multiple time scales within a unified architecture for multi-modal systems.

Addressing this gap is crucial for developing practical intelligent transportation systems (ITSs), as urban mobility inherently involves interactions between multiple transportation modes operating in disparate spatial units. The inability to concurrently model dynamic, high-order correlations within and between these heterogeneous units limits the accuracy and generalizability of predictions in real-world scenarios.

3. Preliminaries

3.1. Notations and Definitions

3.1.1. Traffic Node

A traffic node refers to the spatial unit of a station within an urban transportation system, and can be categorized into two main types. The first type consists of stations with fixed locations, such as bus stops and subway stations, that can serve as nodes in a graph directly. The second type encompasses transportation modes without fixed stations, including those defined by order pick-up and drop-off points (such as bike-sharing services) and those defined by administrative regions (such as taxis). For bike-sharing services, this study employed a Density Peak Clustering (DPC) algorithm to cluster dispersed stations into virtual nodes. The regions themselves are treated as nodes for transportation modes defined by the administrative regions. In a multi-modal transportation system comprising $r$ types of transportation modes, the $i$-th mode has $n$ nodes, represented as the node set $M_i=\left\{M_{i_1}, M_{i_2}, \dots , M_{i_n}\right\}$, whereas the overall node set of the transportation system is denoted as $M=\left\{M_1, M_2, \dots , M_r\right\}$.

3.1.2. Traffic Demand and Node Features

In this study, the traffic demand of an urban traffic system is defined as the boarding demand $x_i^{\left(t\right)}$, which represents the number of vehicles traveling from node $i$ to other areas during time step $t$. For each traffic node, a time series $x_i^{t:t+T}$ is defined to indicate the boarding demand between times $t$ and $t+T$. The node features refer to the transportation demand at node $i$. For traffic mode $M_i$, in the demand study for time step $t+1$, the input node feature consists of the previous observations at time t, represented as $X_{M_i}^{\left(t\right)}=\{x_{M_{i_1}}^{\left(t\right)}, x_{M_{i_2}}^{\left(t\right)}, \dots , x_{M_{i_n}}^{\left(t\right)}\}$, where $M_{i_n}$ denotes node $n$ under traffic mode $M_i$, and $x_{M_{i_n}}^{\left(t\right)}$ is the traffic demand value at node $n$ for time step $t$ under traffic mode $M_i$.

3.1.3. Multi-Modal Transportation System

Urban transportation system $M$ consists of $r$ ($\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} r>1$) types of transportation modes, denoted as $M_i=\{M_1, M_2, \dots ,M_r\}$. Each transportation mode $M_i\left(i=1, 2, \dots , r\right)$ has $n_m$ nodes, which serve as the basic units for the bidirectional flow of vehicles. At time step $t$, the demand characteristics of node $i_j(j=1, 2, \dots , n_m)$ under mode $M_i$ are represented by vector $x_{M_{i_n}}^{\left(t\right)}$. Thus, the characteristics of all the nodes under mode $M_i$ at time step t are represented by $X_{M_i}^{\left(t\right)}=\{x_{M_{i_1}}^{\left(t\right)}, x_{M_{i_2}}^{\left(t\right)}, \dots , x_{M_{i_n}}^{\left(t\right)}\}$. The demand representation for all nodes across all transportation modes in the urban transportation system at time step t is given as $X^t=\{X_{M_1}^{\left(t\right)}, X_{M_2}^{\left(t\right)}, \dots , X_{M_r}^{\left(t\right)}\}$.

3.1.4. Heterogeneous Transportation Graph

A heterogeneous transportation graph is defined as $G=(V, E)$, where $V$ represents the set of nodes and E represents the set of edges. The neighborhood relationship is defined as $N\left(v\right)=\left\{v\in V\right| <u,v> \in E\}$, where $v\left(v\in V\right)$ denotes a node and $e= <u,v> \in E$ represents an undirected edge between node $v$ and node $u$.

3.2. Problem Definition

A traffic node refers to the spatial unit within an urban transportation system and can be categorized into two main types. The first type comprises stations with fixed locations, such as bus stops and subway stations. The second type encompasses transportation modes without fixed stations, including those defined by order pick-up and drop-off points (e.g., bike-sharing) and those defined by administrative regions (e.g., taxis).

The goal of the multi-modal traffic demand collaborative forecasting problem is to predict the future departure demands of various traffic modes for the next time interval. Specifically, we assumed that the multi-modal transportation system contained $r$ traffic modes ($\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} r>1$). The demand data for all the traffic modes are represented as $X^{1:T}=\left\{X_{M_i}^{1:T}\right|M_i=M_1, M_2, \dots , M_i\}$. Given a historical time window of length $L$, the historical demand data for the multi-modal transportation system can be expressed as $X^{T-L:T}=\left\{X_{M_i}^{T-L:T}\right|M_i=M_1, M_2, \dots , M_i\}$. We consider the intra-modal relationship graph $G_{in}$, inter-modal relationship graph $G_{inter}$, and external factor feature $C_E$. Our objective is to learn a mapping function $F\left(·\right)$ to jointly predict the demand for all traffic modes in the $T+1$ time interval, as follows:

$$X^{T+1}=F\left(X^{T-L:T},G_{in},G_{inter},C_E\right)$$

(1)

where $X^{T+1}=\left\{X_{M_i}^{T+1}\right|M_i=M_1,M_2,\dots ,M_i\}$ represents the demand for each traffic mode in the multi-modal transportation system during the next time interval, with $X^{T+1}\in R^N$. Here, $L$ denotes the length of the historical time interval used for the prediction.

In addition, we have compiled all the variables and their meanings in this article into

Table A1. Summary of notations used in this paper.

Notation	Description	Units
$M_i$	Set of nodes for the $i$-th transportation mode	count
$M$	Entire set of nodes for all transportation modes	count
$r$	Number of transportation modes in the system	count
$n$	Number of nodes in a transportation mode	count
$\nu$	Set of traffic nodes (stations or regions) in the urban transportation system.	count
${\nu }_m$	Set of nodes for the $m$-th traffic mode.	count
$\epsilon$	Set of edges representing relationships between nodes.	count
$G$	Heterogeneous transportation graph	dimensionless
$G_{M_i}$	Intra-modal relation graph for the $i$-th transportation mode	dimensionless
$G_{M_{ij}}$	Inter-modal relation graph between $M_i$ and $M_j$	dimensionless
$C_E$	External factor Feature	varies
$x_i^{\left(t\right)}$	Transportation demand at node $i$ during time step $t$	passengers/hour
$X_{M_i}^{\left(t\right)}$	Feature matrix of nodes for the $i$-th transportation mode at time t	passengers/hour
$A_{M_i}$	Weighted adjacency matrix for intra-modal relations	dimensionless
$A_{M_{ij}}$	Weighted adjacency matrix for inter-modal relations	dimensionless
$H^e$	Hyperedge set on hypergraph structure	count
$H^u$	Node set on hypergraph structure	count
$Z_{MSTHCN}$	Features captured by MSTHCN modules	dimensionless
$Z_{inter\_ST}$	Features captured by DMRSHN modules	dimensionless
$L$	Length of historical time steps used for prediction	hours (h)
$F(·)$	Mapping function to predict future demand	dimensionless
$x_i^{final}$	Predicted transportation demand at node $i$-th	passengers/hour
$X^{T+1}$	Predicted transportation demand at time $T+1$	passengers/hour
$d\left(v\right)$	Degree of vertex $\nu$ in a hypergraph	count
$\delta \left(e\right)$	Degree of hyperedge $e$	count
${\Delta }_{in}$	Intra-modal hypergraph Laplacian operator	dimensionless
${\Delta }_{inter}$	Inter-modal hypergraph Laplacian operator	dimensionless
$\Phi$	Orthogonal eigenvectors of the Laplacian matrix	dimensionless
$\Lambda$	Diagonal matrix of eigenvalues	dimensionless
$\Theta$	Trainable parameter matrix in hypergraph convolution	dimensionless
$\sigma$	Activation function (e.g., ReLU, Sigmoid)	dimensionless
g	The gating vector of CMAGFN module	dimensionless
$\omega$	Weight of hyperedge	dimensionless
DSC	Sørensen–Dice coefficient for hyperedge merging	dimensionless
MLP	Multi-layer perceptron	dimensionless
Softmax	Softmax activation function	dimensionless
ReLU	Rectified Linear Unit activation function	dimensionless
Concat	Concatenation operation	dimensionless
LayerNorm	Layer normalization	dimensionless
FeedForward	Feed-forward neural network	dimensionless
$[·\parallel ·]$	The splicing operation	dimensionless
$\gamma$	Weight of L2 regularization term	dimensionless
${ϵ}_m$	Loss weight for mode $m$	dimensionless
$k_{NN}$	Number of neighbors in KNN hyperedge construction	count
$k_c$	Number of clusters in K-means hyperedge construction	count
$k_s$	Number of neighbors in Structure hyperedge construction	count
$L_M$	Number of MDHGNN Layers	count
$\alpha$	Attention weight for fusing time and space features.	dimensionless
s	Node level attention scores based on different structural features	dimensionless
$\beta$	Attention weight for fusing features from different views	dimensionless

in Appendix A.

4. Methodology

4.1. Network Architecture

To bridge the gap in previous research, we propose the Multi-Scale Spatiotemporal Dynamic Hypergraph Transformer (MST-Hyper Trans). The novelty of our work lies in its integrated approach to dynamic multi-relation learning across spatial and temporal dimensions. Specifically, unlike prior works, (1) our model introduces a novel multi-scale temporal hypergraph (MSTHCN) to explicitly decouple and capture periodic and cross-modal temporal interactions at multiple granularities (e.g., hour, day, week). (2) We design a dynamic multi-relation spatial hypergraph network (DMRSHN) that integrates geographic proximity, passenger flow similarity, and transportation connectivity to construct both structural and dynamic hyperedges, enabling adaptive learning of high-order spatial correlations among heterogeneous nodes. (3) We introduce a Conditional Meta Attention Gated Fusion Network (CMAGFN) that dynamically generates and fuses features based on real-time context, moving beyond static fusion strategies. As shown in Figure 3, the model consists of three core modules, forming an end-to-end prediction process. This holistic framework represents a significant step towards accurate and interpretable collaborative forecasting for multi-modal urban transportation systems.

Firstly, the Multi-Scale Temporal Hyperedge Convolutional Network (MSTHCN) is responsible for handling the complexity of the temporal dimension. It effectively captures the periodic and trend patterns of transportation demand at different time granularities (such as hour, day, week) by constructing multi-scale temporal hypergraphs. The internal temporal attention mechanism can adaptively balance the contributions of different scale features, providing the model with powerful temporal representation capabilities. Secondly, the Dynamic Multi-Relationship Space Hypergraph Network (DMRSHN) focuses on modeling high-order dependencies between heterogeneous spatial units. This module integrates multiple semantic relationships, such as geographic proximity, passenger flow similarity, and transportation connectivity, to construct structural hyperedges, and generates dynamic hyperedges through KNN and K-means algorithms to capture spatial correlations that evolve with traffic conditions. The further introduction of a relationship-aware attention mechanism and a hierarchical hypergraph convolution structure achieve refined extraction and fusion of multi-relationship spatial features. Finally, the Conditional Meta Attention Gated Fusion Network (CMAGFN) serves as the decision core of the model, replacing the traditional static fusion approach. This lightweight meta network takes real-time context (such as time, weather, and real-time demand) as input and dynamically generates parameters. Through its core multi-head cross-attention gating mechanism, CMAGFN can adaptively calibrate the fusion weights of intra-mode features and cross-mode spatiotemporal features at the feature dimension level based on the current traffic scene, thereby achieving optimal prediction decisions.

MST-Hyper Trans has completed the complete prediction process from multi-scale temporal feature extraction, dynamic spatial relationship modeling, to context-aware fusion through the collaborative work of MSTHCN, DMRSHN, and CMAGFN modules, significantly improving the ability to accurately predict complex multi-modal transportation systems.

4.2. Multi-Scale Temporal Hypergraph Convolutional Network

To accurately capture the complex multi-scale temporal dependencies in multi-modal transportation demand, we propose the Multi-scale Time Series Hyperedge Convolutional Network (MSTHCN) module. This module significantly enhances the model’s ability to extract periodic, trending, and cross-modal temporal interaction features in transportation demand by combining hypergraph structure modeling with multi-scale temporal analysis. The MSTHCN method addresses the limitations of existing multi-scale temporal models (e.g., TCN) by leveraging multi-scale dynamic hyperedge merging to capture high-order temporal correlations and an attention mechanism for adaptive feature fusion, enabling real-time responsiveness to demand fluctuations in multi-modal traffic systems while mitigating overfitting. The overall structure of the MSTHCN core process includes three parts: multi-scale hyperedge construction, multi-scale temporal hypergraph convolution, and attention-driven fusion. Its structure is shown in Figure 4.

Inspired by multi-scale time series analysis methods, MSTHCN constructs hyperedges through grouping strategies (such as 1 h and 2 h granularity) and merges similar hyperedges (DSC > 0.8), which is more flexible than TCN’s fixed convolution kernel and can capture the periodicity of traffic demand (such as daily/weekly patterns). Specifically, for historical demand data $\left\{X^t\right|t=1, 2, \dots , T\}$, we grouped them by week and hour dimensions to generate four different time scale partitioning schemes: 1 h granularity (7 × 24 groups), 2 h granularity (7 × 12 groups), 3 h granularity (7 × 8 groups), and 4 h granularity (7 × 6 groups). Within each group, we retain the 40% data points with the highest and lowest demand values to form initial hyperedges in order to capture extreme traffic modes. Next, use the Sørensen–Dice coefficient (DSC) [44] to calculate the similarity between hyperedges:

$$DSC={\displaystyle \frac{2\left|H_a\cap H_b\right|}{\left|H_a\right|+\left|H_b\right|}}$$

(2)

When the DSC value exceeds the threshold of 0.8, merge similar hyperedges to obtain a simplified hyperedge set $\left\{H^{\left(q_k\right)}\right|k\in \left\{1, 2, 3, 4\right\}, q_k=1, \dots ,Q_k\}$, where $k$ represents the scale type, and $q_k$ is the hyperedge index. This design enables the model to adaptively capture multi-level time patterns from short-term fluctuations to long-term trends.

Multi-modal traffic data exhibits spatiotemporal heterogeneity. MSTHCN models high-order temporal correlations directly through hypergraph convolution, overcoming the limitation of traditional graph convolution that only deals with pairwise relationships. We use Spectral Hypergraph Convolutional Networks (HGNNs) [14] to separately process the temporal features within and between patterns. For the input signal $X_{in}^{\left(t\right)}$ within the pattern and the signal $X_{inter}^{\left(t\right)}$ between patterns, the hypergraph convolution operation is defined as:

$$X_{in}^{\left(t+1\right)}={\sigma }_{ReLU}\left({{\check{D}}_{v_{in}}}^{-1/2}{\tilde{H}}_{in}{\tilde{W}}_{in}{{\tilde{D}}_{e_{in}}}^{-1}{{\tilde{H}}_{in}}^T{{\tilde{D}}_{v_{in}}}^{-1/2}{X_{in}}^{\left(t\right)}{\Theta }_{in}\right)$$

(3)

$$X_{inter}^{\left(t+1\right)}={\sigma }_{ReLU}\left({{\check{D}}_{v_{inter}}}^{-1/2}{\tilde{H}}_{inter}{\tilde{W}}_{inter}{{\tilde{D}}_{e_{inter}}}^{-1}{{\tilde{H}}_{inter}}^T{{\tilde{D}}_{v_{inter}}}^{-1/2}{X_{inter}}^{\left(t\right)}{\Theta }_{inter}\right)$$

(4)

where ${\tilde{H}}_{in}$ and ${\tilde{H}}_{inter}$ are hypergraph correlation matrices with self-loops added, ${\tilde{D}}_v$ and ${\tilde{D}}_e$ are the degree matrices of nodes and hyperedges, respectively, and $\Theta$ is the learnable convolution kernel parameter. This process captures high-order temporal correlations between nodes through the hypergraph Laplacian operator, overcoming the limitation of ordinary graph convolution that can only model pairwise relationships.

To dynamically integrate temporal features within and between patterns, we introduce node-level attention weights to dynamically balance intra- and inter-mode features, while existing methods, such as multi-resolution attention, often rely on static weights. For node $i$, the attention weights of its intra- and inter-pattern features $x_{in}^i$ and $x_{inter}^i$ are calculated as follows:

$$a_{in}^i=softmax\left(W_{in}^i{X}_{in}^i+b_{in}^i\right)$$

(5)

$$a_{inter}^i=softmax\left(W_{inter}^i{X}_{inter}^i+b_{inter}^i\right)$$

(6)

where $W_{in}^i$, $W_{inter}^i$, $b_{in}^i$, and $b_{inter}^i$ are learnable parameters. The temporal characteristic output of the final node $i$ is:

$$Z_{MSTHCN}^i=a_{in}^i{X}_{in}^i+a_{inter}^i{X}_{inter}^i$$

(7)

This mechanism enables the model to adaptively adjust weights based on the current traffic context, highlighting dominant time patterns.

4.3. Dynamic Multi-Relation Spatial Hypergraph Network

Previous methods have had difficulty handling the complex heterogeneous spatial relationships between nodes in multi-modal transportation networks, as well as the characteristics of local and global relationships. The DMRSHN module captures complex high-order spatial dependencies between heterogeneous units (e.g., taxi regions and subway stations) by integrating domain knowledge via multi-relationship structured hyperedges, coupled with a relationship-aware attention mechanism and hierarchical hypergraph convolution for dynamic multi-scale spatial modeling. The DMRSHN module captures the complex high-order spatial relationships between multi-modal traffic nodes in the urban transportation system, as shown in Figure 5. The core of this network is to handle the dynamic evolution of multiple relationships (geographical proximity, passenger flow similarity, transportation connections), and to use hypergraph networks for high-order correlation modeling in the spatial domain, which can adaptively learn the spatial interactions between stations or regions under different transportation modes.

Firstly, the construction process of various structural hyperedges in the DMRSHN module’s multi-relationship structure hyperedge layer is introduced. The structural hyperedges mainly include geographical proximity hyperedges, passenger flow similarity hyperedges, and transportation connectivity hyperedges. The correlation matrix is mainly calculated between multi-mode transportation nodes, and then multiple hyperedges are obtained through threshold limitation. The calculation process of the relationship matrix is as follows:

(1)

Geographical Distance Embedding: For nodes $i$ and $j$, which are spatial units categorized as regions, we define their distance matrix as $A_G^r$. For nodes $i$ and $j$, which are spatial units categorized as sites, we define their distance matrix as $A_G^s$:

$$A_G^r=\{\begin{array}{cc}1& if i and j are adjacent\\ \exp \left(-{\left({\displaystyle \frac{d_{ij}}{{\sigma }_d}}\right)}^2\right)& if i and j are not adjacent and d_{ij}\le k_d^r\\ 0& if i and j are not adjacent and d_{ij}>k_d^r\end{array}$$

(8)

$$A_G^s=\{\begin{array}{cc}\exp \left(-{\left({\displaystyle \frac{d_{ij}}{{\sigma }_d}}\right)}^2\right)& if d_{ij}\le k_d^s\\ 0& if d_{ij}>k_d^s\end{array}$$

(9)

Here, $A_G^r$ represents the weight of the geographical proximity between regions $i$ and $j$, $d_{ij}$ is the distance between the geometric centers of regions $i$ and $j$, $k_d^r$ is the distance threshold, and ${\sigma }_d$ is the standard deviation of the distance. Similarly, $A_G^s$ denotes the weight of geographical proximity between sites $i$ and $j$, where $d_{ij}$ is the distance between sites $i$ and $j$, and $k_d^s$ is the distance threshold.

For node $i$ with spatial units defined as regions and node $j$ with spatial units defined as points, we define the distance matrix as $A_G^{rs}$. Specifically, when node $j$ is located within region $i$, we set $A_G^r$ = 1. When node $j$ is not within the region $i$, we define $d_{ij}$ as the shortest distance from node $j$ to region $i$, and the weight is calculated using the method described in Equation (9).

(2)

Passenger Flow Similarity Embedding: The historical demand sequences of all nodes are normalized, and the similarity between nodes is computed using correlation coefficients. The passenger flow similarity embedding matrix $A_P$ is defined as follows:

$$A_p=\{\begin{array}{cc}{\displaystyle \frac{Corr\left(p_i,p_j\right)}{{\sigma }_{p,i}{\sigma }_{p,j}}}& {\displaystyle \frac{Corr\left(p_i,p_j\right)}{{\sigma }_{p,i}{\sigma }_{p,j}}}\ge k_p\\ 0& {\displaystyle \frac{Corr\left(p_i,p_j\right)}{{\sigma }_{p,i}{\sigma }_{p,j}}}<k_p\end{array}$$

(10)

In this context, $A_p$ represents the weight of similarity in passenger flow variation patterns between nodes $i$ and $j$. The normalized historical demand sequences for nodes $i$ and $j$ are denoted as $p_i$ and $p_j$, respectively. The term $Corr(\ast )$ refers to the correlation coefficient used to calculate the relationship between two time series. Threshold $k_p$ is determined based on the sparsity of $A_p$, whereas ${\sigma }_{p,i}$ and ${\sigma }_{p,j}$ represent the standard deviations of $p_i$ and $p_j$, respectively.

(3)

Traffic Transfer Embedding: When embedding heterogeneous graphs, incorporating domain-specific knowledge can enhance the learning effectiveness of heterogeneous node features [45]. To achieve this, we constructed the traffic transfer embedding matrix $A_{Con}$ to capture the transfer relationships of multi-modal passenger traffic flow. For any two nodes $i$ and $j$, we define their traffic transfer embedding matrix $A_{Con}$:

$$A_{Con}^{i,j}=\{\begin{array}{cc}{\displaystyle \frac{{\sum }_{t=1}^T{C}_i^j}{{\sum }_{t=1}^T{\sum }_{j=1}^N{C}_i^j}}& {\displaystyle \frac{{\sum }_{t=1}^T{C}_i^j}{{\sum }_{t=1}^T{\sum }_{j=1}^N{C}_i^j}}\ge k_C and d_{ij}\le D\hfill \\ 0& {\displaystyle \frac{{\sum }_{t=1}^T{C}_i^j}{{\sum }_{t=1}^T{\sum }_{j=1}^N{C}_i^j}}<k_C or d_{ij}>D\hfill \end{array}$$

(11)

In this context, $A_{Con}^{i,j}$ represents the traffic connection relevance weight from node $i$ to node $j$, and $C_i^j$ denotes the number of passengers boarding at node $j$ within 15 min of alighting from node $i$. Variable $d_{ij}$ indicates the distance between two nodes, $D$ is the distance threshold, and $k_C$ serves as the threshold for controlling sparsity. Notably, $A_{Con}^{i,j}$ is typically not equal to $A_{Con}^{j,i}$, reflecting the directional nature of the transfer relationship.

For the three types of relationship matrices, we establish thresholds ${\delta }_G$, ${\delta }_P$, and ${\delta }_C$, and generate three types of hyperedges for each node based on these thresholds. Then, the similarity between any two super edges in the three super edge sets is calculated by using the Sørensen–Dice coefficient. Combine the super edges with a Sørensen–Dice coefficient less than 0.8 to obtain three relational super edge sets, which are defined as $H_G=\left\{h^{e_1},h^{e_2},\dots ,h^{e_G}\right\}, H_P=\left\{h^{e_1},h^{e_2},\dots ,h^{e_P}\right\}, H_{Con}=\{h^{e_1},h^{e_2},\dots ,h^{e_{Con}}\}$.

In the relationship-aware attention mechanism layer, we first perform the hypergraph convolution operation based on the initial hyperedge set corresponding to each relationship type (i.e., the geography, passenger flow, and connection hyperedge set before the merger), and obtain the preliminary feature representation of node $i$ under the three relationship types:

$$Z_i^G={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_G{\tilde{W}}_G{{\tilde{D}}_e}^{-1}{{\tilde{H}}_G}^T{{\tilde{D}}_v}^{-1/2}{X}_M{\Theta }_G\right)$$

(12)

$$Z_i^P={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_P{\tilde{W}}_P{{\tilde{D}}_e}^{-1}{{\tilde{H}}_P}^T{{\tilde{D}}_v}^{-1/2}{X}_M{\Theta }_P\right)$$

(13)

$$Z_i^C={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_{Con}{\tilde{W}}_{Con}{{\tilde{D}}_e}^{-1}{{\tilde{H}}_{Con}}^T{{\tilde{D}}_v}^{-1/2}{X}_M{\Theta }_{Con}\right)$$

(14)

where $H_G, H_P, \mathrm{and} H_{Con}$ are hyperedge incidence matrices of three relationship types respectively, $X_M$ is the input characteristics of nodes, and $\theta$ is a learnable parameter matrix.

Note that Equations (15)–(17) define the relationship-aware attention mechanism, which is proposed in our work to dynamically fuse spatial features from different relational hyperedges (geographic, passenger flow similarity, and connectivity). The attention score $s_i^k$ for node $i$ and relation type $K$ is computed by a single-layer MLP (Equation (15)) based on the concatenation of the current node feature and the relation-specific feature. The final attention weights $a_i^k$ are obtained by normalizing these scores using the Softmax function (Equation (16)), which are then used to compute the weighted sum of the relation-specific features to yield the final spatial representation $Z_i^{spatial}$ (Equation (17)). This design allows the model to adaptively prioritize the most relevant spatial relationship for each node under specific traffic contexts.

For node $i$ and relationship type $K \in \{G, P,C\}$, we splice its current node feature $X_i^M$ with the feature representation $Z_i^k$ under relationship type k, and calculate the attention score $s_i^k$ through a single-layer MLP and Leaky-Relu activation function:

$$s_i^k=LeakyReLU\left(w^T\left[x_i\parallel Z_i^k\right]\right)$$

(15)

where $[·\parallel ·]$ represents vector splicing operation and $w$ is a learnable attention vector.

Then use the Softmax function to normalize the attention scores of the three relationship types to obtain the normalized attention weight $a_i^k$:

$$a_i^k={\displaystyle \frac{\exp \left(s_i^k\right)}{{\sum }_{k^{\prime }\in \left\{G, P, C\right\}}\exp \left(s_i^{k^{\prime }}\right)}}$$

(16)

Finally, the spatial feature $Z_i^{spatial}$ of node $i$ after the fusion of the relationship-aware attention mechanism is the weighted sum of three relationship features:

$$Z_i^{spatial}=a_i^G{Z}_i^G+a_i^P{Z}_i^P+a_i^C{Z}_i^C$$

(17)

Through the relationship-awareness mechanism, the model can adaptively allocate different importance weights for geographical proximity, passenger flow similarity, and traffic connectivity according to the specific traffic context, so as to achieve a more refined modeling of heterogeneous spatial dependencies.

In order to overcome the shortcomings of traditional multi-scale convolution kernel parallel processing in feature hierarchy, we propose a hierarchical hypergraph convolution structure. The structure of hierarchical hypergraph convolution is shown in Figure 6. Through the serial stacking and residual connection of three layers of convolution, our hierarchical design achieves the gradual expansion of information perception range layer by layer, effectively simulating the progressive process of human cognition from local details to global patterns. Compared to simple parallel multi-scale convolution, this method can generate more hierarchical and rich feature representations, thus better serving multi-modal traffic prediction tasks. Equations (18)–(21) describe the hierarchical hypergraph convolution process, a novel design in our work for multi-scale spatial feature extraction. Inspired by the success of hierarchical convolution in CNNs for capturing features at different receptive fields [46], we apply hypergraph convolutional kernels of increasing effective sizes (e.g., 1 × 1, 3 × 3, 5 × 5) to the KNN and K-means hypergraphs. The outputs from different levels are fused through residual connections (Equations (19) and (20)), enabling the model to progressively capture local-to-global spatial dependencies. The final inter-modal spatial feature $Z_{inter\_ST}$ is obtained by concatenating and linearly transforming the features from different hypergraph types (Equation (21)).

The dynamic super edge generation layer samples KNN and K-means neighbor nodes based on the node structure characteristics generated by the relationship-awareness layer, and generates KNN super edge and K-means super edge, that is, the node set represented by $H_{NN}^e=\{u_1,u_2,\dots ,u_{NN}\}$, $H_{K-Means}^e=\{u_1,u_2,\dots ,u_{K-Means}\}$ and the node set represented by $H_{NN}^u=\{h_1,h_2,\dots ,h_{NN}\}$, $H_{K-Means}^u=\{h_1,h_2,\dots ,h_{K-Means}\}$.

After completing the construction of KNN hyperedges and K-means hyperedges in the dynamic hyperedge construction layer, this paper constructs a hierarchical hypergraph convolution layer. Multiple layers of different sized convolution kernels are applied to the KNN hypergraph and K-means hypergraph, respectively. Then, the feature representations obtained from different levels of convolution are fused. The calculation process for hierarchical convolution of KNN hyperedges and K-means hyperedges is as follows:

$$Z_{NN}^{\left(1\right)}={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_{NN}{\tilde{W}}_{NN}{{\tilde{D}}_{e,NN}}^{-1}{{\tilde{H}}_{NN}}^T{{\tilde{D}}_v}^{-1/2}{Z}^{spatial}{\Theta }_{NN}^{1\times 1}\right)$$

(18)

$$Z_{NN}^{\left(2\right)}={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_{NN}{\tilde{W}}_{NN}{{\tilde{D}}_{e,NN}}^{-1}{{\tilde{H}}_{NN}}^T{{\tilde{D}}_v}^{-1/2}{Z}^{spatial}{\Theta }_{NN}^{3\times 3}\right)+Z_{NN}^{\left(1\right)}$$

(19)

$$Z_{NN}={\sigma }_{ReLU}\left({{\check{D}}_v}^{-1/2}{\tilde{H}}_{NN}{\tilde{W}}_{NN}{{\tilde{D}}_{e,NN}}^{-1}{{\tilde{H}}_{NN}}^T{{\tilde{D}}_v}^{-1/2}{Z}^{spatial}{\Theta }_{NN}^{5\times 5}\right)+Z_{NN}^{\left(2\right)}$$

(20)

$$Z_{inter\_ST}=W_c\left[Z_{NN}\parallel Z_{K-Means}\parallel Z_{MSTHCN}\right]+b_c$$

(21)

where $[·\parallel ·]$ is the splicing operation and $W_c$ is the learnable weight matrix. $Z_{inter\_ST}$ is the spatial feature representation finally obtained by the DMRSHN module. $Z_{NN}$ and $Z_{K-Means}$ are the feature representations obtained by the hierarchical convolution of the KNN super edge and K-means hyperedge, respectively. The algorithmic principles of the DMRSHN module are presented in Algorithm A1 in Appendix B.

4.4. Conditional Meta Attention Gated Fusion Network

The CMAGFN module overcomes the static limitations of traditional fusion strategies by incorporating a gated fusion mechanism based on multi-head cross-attention, which dynamically calibrates feature weights for context-aware decision-making. As a lightweight meta network, CMAGFN can dynamically generate fusion weights based on real-time context, rather than relying on static parameters. It can dynamically calculate a fine-grained multidimensional gating vector (rather than a single scalar) based on real-time traffic context, thereby adapting the contribution weights of intra-mode features and cross-mode features at the feature dimension level, and achieving more accurate and interpretable feature fusion. Equations (22)–(30) formulate the core of the Conditional Meta Attention Gated Fusion Network (CMAGFN), a key contribution of this paper. The intra- and inter-modal features are first projected by lightweight MLPs (Equations (22) and (23)). The multi-head cross-attention mechanism (Equations (24)–(27)), drawing inspiration from the transformer architecture [47], is employed to compute the context-aware gating vector $g_i^{mh}$ (Equation (28)), where the query is the intra-modal feature, and the key/value are derived from the inter-modal features. This allows the model to dynamically assess the relevance of cross-modal information for each feature dimension. The final fused feature $x_i^{final}$ is a weighted sum of the intra- and inter-modal features under the control of this gating vector (Equation (29)). The entire process is designed to replace static fusion strategies, enabling fine-grained, adaptive feature integration.

The overall architecture is illustrated in Figure 7, which enables the dynamic optimization of the entire process, from information filtering to feature fusion, through an attention mechanism. The input of the CMAGFN layer is a context vector that comprehensively describes the current traffic status, ensuring that the generated dynamic weights can respond to real-time conditions. It mainly includes core environment context, intra-pattern requirement context, and cross-pattern context, where the core environment context includes timestamps (such as hour, day of the week) and weather conditions (such as sunny, rainy) encoded by embedding layer vectors $C_t$ and $C_w$, respectively. The intra-pattern requirement context $C_{intra}$ refers to the recent historical demand sequence of nodes in the target transportation mode (e.g., taxi), which is extracted using a lightweight temporal convolutional layer. The cross-modal context includes the pure historical demand sequence $C_{inter\_hist}$ of the relevant patterns, as well as the fused spatiotemporal features $Z_{inter\_ST}$ from the MSTHCN and DMRSHN modules. For each node, CMAGFN includes two parallel lightweight MLP paths that generate feature representations optimized for intra-pattern and cross-pattern information, respectively. The intra-model feature generator focuses on learning from the contextual requirements and core environment within the pattern, capturing the historical patterns of the pattern itself.

$$Z_i^{intra}={MLP}_{intra}\left(\left[C_t\parallel C_w\parallel C_{intra}\right]\right)$$

(22)

The cross-modal feature generator focuses on learning from all cross-modal contexts and core environmental contexts, extracting synergistic effect information provided by other traffic modes.

$$Z_i^{inter}={MLP}_{inter}\left(\right[C_t\parallel C_w\parallel C_{inter\_hist}\parallel Z_{inter\_ST}\left]\right)$$

(23)

where $Z_i^{intra},Z_i^{inter}\in R^d$, d is the feature dimension.

The core of the CMAGFN module is to use multi-head cross-attention to generate a dynamic multi-dimensional gating vector. The query Q is derived from the output $Z^{intra}$ of the intra-modal feature generator, which is obtained through linear transformation and represents the current node’s own state in the target pattern. The key K and value V are derived from the output $Z^{inter}$ of the cross-modal feature generator and obtained through linear transformation, representing the contextual information provided by other modals:

$$Q=Z_i^{intra}\times W^Q,K=Z_i^{inter}\times W^K,V=Z_i^{inter}\times W^V$$

(24)

where $W^Q$, $W^K$, and $W^V$ are learnable parameter matrices.

Subsequently, calculate the attention score and apply Softmax to obtain the weights:

$$Attention\left(Q,K,V\right)=Softmax\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(25)

where $d_k$ is the dimension of the key vector K, and the scaling factor ${\displaystyle \frac{1}{\sqrt{d_k}}}$ is used to prevent gradient vanishing due to excessive dot product results.

After computing the query (Q), key (K), and value (V), the CMAGFN module performs multi-head attention mechanism calculations by using multiple sets of different WQ, WK, and WV matrices to perform h parallel attention calculations as described above. Each head focuses on different aspects of the input information, and finally concatenates the outputs of each head:

$$MultiHead\left(Q,K,V\right)=Concat\left({head}_1,\dots ,{head}_h\right)W^O$$

(26)

$${head}_i=Attention\left(Q{W}_i^Q,K{W}_i^K,V{W}_i^V\right)$$

(27)

where $W^O$ is the output projection matrix.

The multi-head attention mechanism enables the model to evaluate the relationships between features from different subspaces, thereby gaining a more comprehensive understanding of cross-modal interactions. The CMAGFN module passes the output of multi-head attention through a linear layer and Sigmoid activation function to generate a multidimensional gating vector $g_i^{mh}$:

$$g_i^{mh}=\sigma \left(W_g·MultiHead\left(Q,K,V\right)+b_g\right)$$

(28)

The distribution of gate control vector $g_i^{mh}$ can directly reflect the degree of dependence of the model on intra- and cross-modal information in different contexts, providing insights for decision-making. Ultimately, the fusion feature $x_i^{final}$ of node i is the weighted sum of intra-pattern features and cross-pattern features under this gating vector:

$$x_i^{final}=g_i^{mh}⨀Z_i^{intra}+\left(1-g_i^{mh}\right)⨀Z_i^{inter}$$

(29)

where ⊙ represents the Hadamard product. This design allows the model to adopt different fusion strategies in different feature dimensions, achieving extremely refined fusion.

4.5. Multi-Task Learning Strategies

Based on the method for modeling task weights with homoscedastic uncertainty proposed by Kendall et al. [48], we set the loss weights for each task to be adaptively learned within the loss function (Equation (30)) by utilizing a multi-task learning strategy. The training objective of our model is to minimize the error between the predicted demand $\widehat{x}$ and the actual demand $x$ across all patterns and nodes. The loss function is defined as the mean squared error (MSE) between the predicted and actual values and is mathematically expressed as follows:

$$L\left(\theta \right)=\sum _{k =1}^r({\displaystyle \frac{1}{{\sigma }_k^2}}\sum _{i=1}^{N_k}\Vert {\widehat{x}}_{k,i}^{t+1}-x_{k,i}^{t+1}\Vert )+\gamma {\Vert \theta \Vert }_2^2$$

(30)

In this context, ${\widehat{x}}_{k,i}^{t+1}$ and $x_{k,i}^{t+1}$ represent the predicted and actual demand values for node $i$ of pattern $k$ at time step $t+1$, respectively. The noise variance for task $k$ is given by ${\sigma }_k^2=\mathrm{e}\mathrm{x}\mathrm{p}(\mathrm{l}\mathrm{o}\mathrm{g}\_vars[k\left]\right)$, and is a learnable parameter. Term ${\displaystyle \frac{1}{{\sigma }_k^2}}$ serves as the adaptive weight, whereas $log{\sigma }_k$ acts as a regularization term. Here, $N_k$ denotes the number of nodes in the $k$-th traffic mode, and $\gamma$ is the weight of the L2 regularization term. The training process for the proposed model is presented in Algorithm 1.

Algorithm 1: The training process of MST-Hyper Trans

Input: historical demand data $X^{T-L:T}=\{X_{M_i}^{T-L:T}, \forall M_i\}$, intra-modal multi-time scale hypergraph $G_{T\_intra}=(V_{intra}, E_{intra}, A_{intra})$ inter-modal multi-time scale hypergraph $G_{T\_inter}=(V_{inter}, E_{inter}, A_{inter})$, Weather data $C_w$, time data $C_t$, number of MDHGNN layers $L$, number of training epochs $E$

Output: prediction demand value $X^{T+1}=\{X_{M_i}^{T+1}, \forall M_i\}$

Initialize the parameters of MST-Hyper Trans;

Initialize its input dim, hidden dim and output dim;

Generate training loader;//Training loader format is [input length, number of vertex, input]

For $e$ in $\left\{1, 2,\dots E\right\}$ do

Initialize $H_{in}$ as $X^{T-L:T}$ for all modes, initialization of $H_{in,m}$ as $X_m^{T-L:T}$ for each mode $m\in \left\{\mathrm{1,2},\dots k\right\};$

For $m\in \left\{\mathrm{1,2},\dots k\right\}$ do

Compute $Z_{MSTHCN}$ from a MSTHCN layer given $H_{in}$, $H_{in,m}$, $G_{T\_intra}$,$G_{T\_inter}$;

Compute $Z_{inter\_ST}$ from DMRSHN layer given $Z_{MSTHCN}$;

Compute $H_{HA}$ from CMAGFN given $Z_{inter\_ST}$, $C_t$, $C_w$;

Generate predictions ${\widehat{X}}_m^{t+1}$ from $H_{HA}$ using a mode-specific output layer;

End

Calculate the loss function Equation (30) from prediction results $\left\{{\widehat{X}}_m^{t+1}, \forall m\in \left\{1, 2, \dots k\right\}\right\}$;

Update the parameters through error back propagation;

End

5. Experiments

5.1. Data Description

To validate the effectiveness of the proposed model, we conducted experiments on three real-world traffic datasets—NYC-Taxi, NYC-Bike, and NYC-Subway—collected from Manhattan, New York City, covering the period from 1 April to 30 June 2023.

The spatial units for each mode were defined differently. The taxi data were aggregated into irregular regions based on the Neighborhood Tabulation Areas (NTAs) defined by the New York City Department of City Planning. The bike-sharing and subway data were organized into virtual stations and physical stations, respectively. Each spatial unit across all datasets was assigned a unique numerical identifier. Key data fields include geographic coordinates (latitude and longitude) for trip start and end points, precise timestamps, and trip distances (for taxi) or passenger counts (for subway). Additionally, we integrated hourly weather data from the Central Park station, incorporating features such as temperature, humidity, precipitation, and snowfall after data cleaning.

A quantitative summary of the key statistics for these datasets is provided in

Table 2. Summary statistics of the NYC multi-modal traffic datasets.

Statistic	NYC-Taxi	NYC-Bike	NYC-Subway
Time Period	1 April 2023–30 June 2023	1 April 2023–30 June 2023	1 April 2023–30 June 2023
Total Records (Millions)	10.1 M	9.75 M	52.9 M
Spatial Units	63 regions	60 virtual stations	55 stations
Spatial Unit ID Range	0–62	123–177	63–122
Temporal Resolution	1 h	1 h	1 h
Key Features	Area ID, Pick-up/Drop-off Coordinates, Timestamp	Station ID, Start/End Coordinates, Timestamp	Station ID, Timestamp, Passenger Count
Data Source	TLC Trip Record Data	Citi Bike Trip Data	MTA Turnstile Data

offering a concise overview of their scale and structure.

In addition, we also conducted spatiotemporal feature analysis on three types of traffic data. Figure 8 shows the spatial distribution of demand for multiple modes of transportation in New York City. It can be seen that taxi demand (a) is concentrated in the downtown commercial area, bicycle demand (b) forms hotspots in residential areas and commuting hubs, while subway demand (c) is highly concentrated in major transportation hubs, which intuitively reflects the spatial heterogeneity of multi-modal data. From the time series perspective (Figure 9), all three patterns exhibit significant periodicity (such as morning and evening peaks) and volatility. The demand for taxis fluctuates sharply and has no clear pattern; the demand for bicycles exhibits a regular short-term peak; the subway has the largest demand volume and there is a continuous peak during the day. This collectively constitutes a complex, multi-scale temporal dynamic challenge in prediction tasks.

To construct the input for the model, the raw transaction data from each mode (taxi, bike, subway) were converted into a unified 4D tensor representation $X\in R^{B\times T\times N\times D}$, where B is the batch size, T is the historical time steps, $N$ is the number of spatial nodes, and $D$ is the feature dimension. This was achieved by aggregating records into hourly departure demand counts, mapping each data point to its respective spatial unit (63 regions for taxis, 60 virtual stations for bikes, 55 physical stations for subways), and normalizing the numerical features. The key distinction between the datasets lies in the dimension $N$, allowing the model to handle heterogeneous spatial units within a consistent architectural framework.

5.2. Baseline Models

To validate the effectiveness of the proposed model, we compared it with various baseline models, including multi-modal prediction methods for heterogeneous spatial units, and several unimodal prediction methods. Below is a detailed introduction to each baseline model and its parameter settings.

• HA (Historical Average): A simple time series forecasting method that predicts future demand by calculating the average of historical time steps.
• LR (Linear Regression): A classic statistical analysis method that models the linear relationship between demand and historical time series to forecast future demand.
• XGBoost (Extreme Gradient Boosting) [49]: A machine learning method based on a gradient boosting framework that predicts future traffic demand by constructing regression tree models.
• LSTM (Long Short-Term Memory) [50]: A deep learning approach that serves as a variant of RNNs. LSTM captures long-term dependencies in sequences through gating mechanisms, making it suitable for time series forecasting.
• STGCN (Spatiotemporal Graph Convolutional Network) [6]: A deep learning method based on graph neural networks that combines spectral graph convolution and temporal convolution layers to model spatial and temporal correlations.
• DCRNN (Diffusion Convolutional Recurrent Neural Network) [51]: A spatiotemporal graph convolutional network that utilizes an encoder–decoder structure, combining graph convolution networks and GRUs to capture spatiotemporal dependencies.
• Graph WaveNet [35]: A spatiotemporal graph learning method that performs node embedding through self-learning adjacency matrices, using temporal and graph convolutions to capture complex spatiotemporal dependencies.
• HGNN (Hypergraph Neural Network) [14]: A deep learning approach that handles complex data structures by modeling data through hypergraph structures and performing feature learning via hypergraph convolution.
• DHGNN (Dynamic Hypergraph Neural Network) [52]: A neural network method that models dynamically evolving hypergraph structures, adapts to data changes through dynamic updates of the hypergraph structure, and encodes high-order data relationships via vertex and hyperedge convolutions.
• CoGNN [11]: A demand collaborative prediction method for self-learning multi-modal heterogeneous graphs, which captures relationships between heterogeneous nodes through self-learning matrices and models temporal and spatial features separately using spatiotemporal convolutional networks.
• ST-MRGNN (Multi-Relation Spatiotemporal Graph Neural Network) [12]: A multi-modal demand collaborative prediction method for heterogeneous spatial units that captures intra- and inter-modal spatial relationships of multi-modal data through graph neural networks, employing temporal convolutional and graph convolutional layers to capture complex spatiotemporal dependencies.
• TSHDNet (Temporal-Spatial Heterogeneity Decoupling Network) [38]: A multi-modal traffic flow prediction method separates the temporal dynamics and spatial heterogeneity within and between modes through a gated attention mechanism that employs adaptive hyperedges to capture complex spatial dependencies.

5.3. Experiment Settings

The CPU used in this experiment was an Intel Xeon E5-2650 v4, and the GPU was an NVIDIA Tesla T4, which is based on NVIDIA’s Turing architecture and features 2560 CUDA cores, supporting various computation precisions. The operating system was Windows 11, the programming language was Python, and the deep learning framework was PyTorch 1.13.1.

To ensure data quality, a rigorous preprocessing pipeline was applied before modeling. Firstly, outliers in traffic order data (e.g., travel time, speed, trip distance) beyond the 99th percentile were filtered out. Subsequently, missing values were imputed using linear interpolation, and spatiotemporal consistency checks (e.g., valid timestamps and geographical coordinates within Manhattan) were performed. To eliminate dimensional discrepancies, all numerical features were normalized to the range [0, 1] using min-max scaling. The time granularity of all cleaned datasets was standardized to one hour, covering the period from 1 April to 30 June 2023, resulting in 2184 hourly records.

To maintain temporal integrity and prevent data leakage, the dataset was split sequentially by time into training (first 60%, April 1 to May 20), validation (next 20%, May 21 to June 10), and test (final 20%, June 11 to June 30) sets. This strategy ensures that the model is trained on historical data and evaluated on future data, mimicking real-world prediction scenarios. The validation set was utilized for model validation and hyperparameter tuning, while the test set was reserved for final evaluation. During training, we employed the Adam optimization algorithm with a cycle of 50 epochs and applied an early stopping strategy on the validation set to mitigate overfitting.

All models were evaluated on the test set using three commonly used metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Pearson Correlation Coefficient (PCC). These evaluation metrics are formally defined as follows:

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _i{\left(y_i-\widehat{y_i}\right)}^2}$$

(31)

$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}\left|\widehat{y_i}-y_i\right|$$

(32)

$$PCC={\displaystyle \frac{{\sum }_{l=1}^n\left(\widehat{y_i}-\overline{\widehat{y_i}}\right)\left(y_i-\overline{y_i}\right)}{\sqrt{{\sum }_{l=1}^n{\left(\widehat{y_i}-\overline{\widehat{y_i}}\right)}^2}\sqrt{{\sum }_{l=1}^n{\left(y_i-\overline{y_i}\right)}^2}}}$$

(33)

where $n$ denotes the total number of samples, $\widehat{y_i}$ represents the predicted value, and $y_i$ represents the actual value. The averages of the predicted and actual values are denoted by as $\overline{\widehat{y_i}}$ and $\overline{y_i}$, respectively.

We categorized the hyperparameters into model-specific and training hyperparameters. The model hyperparameters were pertained to the architecture of MST-Hyper Trans, such as the scales and structures of the hypergraphs (e.g., the number of neighbors for different hyperedge types, the introduction epochs of various hyperedges, and the K-means clustering settings). The training hyperparameters governed the learning process, including the historical observation length, learning rate, and regularization settings. To determine the optimal values, we employed a Bayesian optimizer for model hyperparameter search, the distribution of which is illustrated in Figure 10. Subsequently, a grid search was conducted on the training hyperparameters based on the fixed model architecture. The impact of key training hyperparameters on the model’s RMSE is visualized in Figure 11 and Figure 12. Furthermore, a sensitivity analysis on the critical hyperparameters for KNN and K-means hyperedges was conducted to justify their robustness (as detailed in Appendix C). The final, optimal set of all hyperparameters for our proposed MST-Hyper Trans model is comprehensively summarized in

Table 3. Hyperparameter settings of MST-Hyper Trans model.

Model	Hyperparameters	Taxi	Bike	Subway
MST-Hyper Trans	Initial Learning Rate	0.01	0.01	0.01
	Training Epochs	300	300	300
	Learning Rate Milestones	[10, 20, 30]	[10, 20, 30]	[10, 20, 30]
	Learning Rate Decay Factor	0.1	0.15	0.1
	MDHGNN Layers	1	1	1
	Units in Each Layer	256	256	512
	Batch Size	32	32	32
	Number of Epochs	30	30	30
	Dropout Rate	0.3	0.3	0.3
	History Length	6	6	6
	Neighbors for KNN Nodes	32	32	32
	Neighbors for Structured Nodes	8	8	8
	Neighbors for K-Means Nodes	4	4	4
	Epochs for KNN Hyperedge Introduction	0	0	0
	Epochs for Structural Hyperedge Introduction	5	5	5
	Epochs for K-Means Hyperedge Introduction	10	10	10
	K-Means Clustering Centers	6	6	6
	Central Nodes Associated With Nodes	1	1	1

.

5.4. Comparison of Model Performance Results

To evaluate the effectiveness of the proposed model, we conducted traffic demand prediction experiments on three traffic datasets from New York City. The experiments were divided into two parts: the first part compared single-modal demand prediction models, whereas the second part compared multi-modal demand collaborative prediction models.

Table 4. Performance comparison of different models on NYC-Taxi dataset.

Models	NYC-Taxi			NYC-Bike			NYC-Subway
	RMSE	MAE	PCC	RMSE	MAE	PCC	RMSE	MAE	PCC
HA	68.827	57.613	0.578	64.067	52.491	0.597	894.365	642.129	0.673
LR	46.698	39.914	0.776	43.251	38.566	0.764	701.992	401.122	0.714
XGBoost	35.175	32.509	0.805	32.789	27.172	0.796	643.747	349.754	0.834
LSTM	32.943	26.679	0.821	33.903	28.238	0.806	652.918	320.189	0.841
STGCN	31.461	24.270	0.857	28.746	23.091	0.843	587.276	310.107	0.863
DCRNN	29.769	21.166	0.891	23.057	21.683	0.849	562.248	306.779	0.877
Graph WaveNet	24.157	19.690	0.917	22.489	18.607	0.872	537.840	269.715	0.909
HGNN	22.367	17.358	0.931	20.168	15.255	0.906	473.269	247.332	0.927
DHGNN	19.968	11.107	0.948	17.384	11.787	0.924	390.902	203.880	0.954
MST-HyperTrans	16.512	9.629	0.969	14.766	9.032	0.946	352.808	177.979	0.975

presents the results of various single-modal prediction methods, while Tables 6–8 display the results of the multi-modal prediction methods under different dataset combinations. Each model was run 10 times, and the results were averaged. The best and second-best results are highlighted in bold and underlined, respectively.

As shown in the results of the single-mode experiments in Table 4, among all baseline models, the machine learning methods (HA and LR) exhibited poorer performance, whereas the performance of XGBoost in predicting the demand for shared bicycles and subways was comparable to that of LSTM. This indicates that traditional machine learning methods have limitations when handling complex spatiotemporal data. Among the single-mode deep learning methods, Graph WaveNet demonstrated superior predictive performance by capturing hidden spatial dependencies through an adaptive adjacency matrix. Both the HGNN and DHGNN achieved stable results by extracting high-order spatial dependencies through hyperedges. Our proposed MST-Hyper Trans model significantly outperformed all other baseline models across all evaluation metrics.

In the experimental results presented in

Table 5. Performance comparison of different models on NYC-Taxi and NYC-Subway datasets.

Models	NYC-Taxi			NYC-Subway
	RMSE	MAE	PCC	RMSE	MAE	PCC
STMRGNN	20.256	14.106	0.928	405.191	237.561	0.932
CoGNN	18.471	11.469	0.952	395.334	213.130	0.961
MST-Hyper Trans	16.952	10.352	0.968	362.110	195.786	0.970
Improved	8.22%	9.74%	1.68%	8.40%	8.14%	0.94%

,

Table 6. Performance comparison of different models on NYC-Taxi and NYC-Bike datasets.

Models	NYC-Taxi			NYC-Bike
	RMSE	MAE	PCC	RMSE	MAE	PCC
STMRGNN	21.002	13.276	0.936	17.847	13.790	0.921
CoGNN	19.632	11.229	0.950	17.575	11.620	0.923
MST-Hyper Trans	17.539	10.316	0.967	15.965	10.350	0.940
Improved	10.66%	8.13%	1.79%	9.16%	10.93%	1.84%

and

Table 7. Performance comparison of different models on NYC-Bike and NYC-Subway datasets.

Models	NYC-Subway			NYC-Bike
	RMSE	MAE	PCC	RMSE	MAE	PCC
STMRGNN	378.209	224.176	0.843	17.439	12.086	0.926
CoGNN	369.845	215.558	0.958	16.080	10.295	0.938
MST-Hyper Trans	343.762	193.058	0.971	14.292	9.368	0.953
Improved	7.05%	10.44%	1.36%	11.12%	9.00%	1.60%

, MST-Hyper Trans consistently demonstrated superior predictive performance compared to CoGNN. The performance of the ST-MRGNN was relatively poor, which may be attributed to its limitations in relying on a predefined graph structure to extract spatial dependencies. In contrast, CoGNN models potential spatial information both within and between modes using an adaptive matrix, resulting in more accurate predictions. These results indicate that MST-Hyper Trans has a significant advantage in collaborative prediction for multi-modal demand. Compared to ST-MRGNN and CoGNN, MST-HyperTrans not only considers the dynamic spatial correlations among heterogeneous nodes but also effectively captures complex spatiotemporal dependencies by modeling the relationships between different traffic modes. This capability significantly enhances the predictive accuracy of the model and improves the interpretability of the inter-modal relationships.

Table 8. Performance comparison of different models on NYC datasets.

Models	NYC-Taxi		NYC-Bike			NYC-Subway
	RMSE	MAE	PCC	RMSE	MAE	PCC	RMSE	MAE	PCC
TSHDNet	17.973	10.212	0.958	15.664	10.406	0.939	387.736	195.154	0.967
MST-Hyper Trans	16.512	9.629	0.969	14.766	9.032	0.946	352.808	177.979	0.975

presents a comparison of the prediction results between TSHDNet and MST-Hyper Trans across the three datasets for NYC. It is evident that MST-Hyper Trans exhibits superior predictive performance on all three datasets. This advantage may be attributed to MST-Hyper Trans’s approach of combining static and dynamic matrices for spatial dependency modeling in the transportation domain, as well as its ability to better learn the complex spatial dependencies among various traffic modes through hypergraph learning. In contrast, the predictive performance of the TSHDNet model is relatively weak, which may be due to the difficulty in capturing latent spatial information among diverse traffic modes, resulting in less accurate predictions. The results in the table indicate that MST-Hyper Trans demonstrates a more significant advantage over TSHDNet in multi-modal traffic prediction.

Figure 13 shows the time series comparison between the predicted and actual demand values for three transportation modes during a continuous 5-day testing period. It can be seen that our model (MST-Hyper Trans) not only has low overall error, but also accurately captures the daily cycle changes in each mode, the timing of peak occurrence and end, and the amplitude of peak values. Especially during the extreme peak days of subway demand, the model’s predicted curve closely matches the actual curve, demonstrating its ability to handle large-scale and highly fluctuating demand.

5.5. Computational Cost and Scalability Analysis

To thoroughly address the computational efficiency and scalability of the proposed MST-Hyper Trans framework, this section presents a comprehensive analysis of training/inference time, computational complexity, parameter scale, and scalability. The results demonstrate that our model achieves an optimal balance between prediction accuracy and computational overhead, confirming its practicality for real-world ITS deployments.

Firstly, there is a comparative analysis of training and inference efficiency. As summarized in

Table 9. Comparison of Training Time for Different Models.

Models	Training Time (s/Epoch)
	STMRGNN	CoGNN	TSHDNet	MST-HyperTrans
Taxi–Subway	4.931	4.798	5.079	4.796
Taxi–Bike	4.886	4.901	5.036	4.893
Subway–Bike	4.772	4.812	5.058	4.786
Collaboration of Three Modes	——	——	5.124	4.957

and

Table 10. Comparison of Inference Time between Different Models.

Models	Inference Time (s)
	STMRGNN	CoGNN	TSHDNet	MST-HyperTrans
Taxi–Subway	0.176	0.187	0.194	0.171
Taxi–Bike	0.175	0.190	0.191	0.178
Subway–Bike	0.186	0.193	0.195	0.182
Collaboration of Three Modes	——	——	0.201	0.186

, we compare the training time (per epoch) and inference time of MST-Hyper Trans against three representative baseline models (ST-MRGNN, CoGNN, and TSHDNet) on the NYC multi-modal dataset. The results indicate that MST-Hyper Trans achieves highly competitive efficiency: its training time per epoch is similar to ST-MRGNN and CoGNN, yet it significantly outperforms TSHDNet, which exhibits the longest training duration. More importantly, in terms of inference time, MST-Hyper Trans demonstrates superior efficiency. For instance, in the three-mode prediction task, our model achieves an inference time of 0.186 s, which is faster than TSHDNet (0.201 s) and comparable to ST-MRGNN. This efficiency is critical for real-time prediction scenarios, where a sub-second response is often required. Furthermore, when handling multiple traffic modes, the increase in both training and inference time for MST-Hyper Trans is more moderate than that of TSHDNet, highlighting its better scalability and lower computational overhead growth with increasing model complexity.

In addition, we analyze the theoretical computational complexity of MST-Hyper Trans and compare it with baseline models, as shown in

Table 11. Comparison of time and space complexity between MST-Hyper Trans and baseline models.

Models	Time Complexity	Space Complexity
LSTM	$O(N·D·T)$	$O(N·D·T)$
STGCN	$O(N^2+T·K)$	$O(N^2+T·D)$
DCRNN	$O(N^{2}K+N^{2}DT)$	$O(N^2+NDT)$
Graph WaveNet	$O(N^2+T·L)$	$O(N^2+T·D)$
HGNN	$O(N^2+NE)$	$O(NE+ND)$
DHGNN	$O(N^2+NE)$	$O(NE+ND)$
CoGNN	$O(N^2+T·R)$	$O(N^2+T·D)$
ST-MRGNN	$O(N^2+T·R)$	$O(N^2+T·D)$
TSHDNet	$O(N^2+N·D·T·L)$	$O(N·D·T·E+N·T·H)$
MST-Hyper Trans	$O(S·N·D·T+N·K/logN)$	$O(N·D·T+NE+ND+NK+C)$

. The overall time complexity of MST-Hyper Trans is $O(S·N·D·T+N·K/logN+N·D·H)$, and its space complexity is $O(N·D·T+NE+ND+NK+C+N^2)$, where N is the number of nodes, D is the feature dimension, T is the time step length, S denotes the number of temporal scales, and K represents the number of neighbors in KNN. Although the theoretical complexity of MST-Hyper Trans is higher than that of lightweight models (e.g., LSTM and STGCN), it is notably lower than TSHDNet. This complexity is well justified by the model’s superior performance in capturing complex multi-modal spatiotemporal dependencies, leading to higher prediction accuracy and stronger generalization. The empirical results in Table 10 and Table 11 confirm that the theoretical complexity does not translate to prohibitive computational costs in practice, validating the model’s efficiency.

Parameter count is a critical factor affecting model computational efficiency, training stability, and deployment feasibility. As summarized in

Table 12. Comparison of parameter count between MST-Hyper Trans and baseline models.

Models	Parameter Count
LSTM	Approximately 14 K–100 K
STGCN	Approximately 372 K
DCRNN	Approximately 264 K–372 K
Graph WaveNet	Approximately 1.65 M
HGNN	Approximately 211 K
DHGNN	Approximately 500 K–1 M
CoGNN	Approximately 2 K–10 K
ST-MRGNN	Approximately 800 K–1.5 M
TSHDNet	Approximately 300 K–800 K
MST-Hyper Trans	Approximately 377 K

, we compare the parameter sizes of MST-Hyper Trans and baseline models. The results reveal a polarized distribution among existing models: lightweight models (e.g., CoGNN, with <10 K parameters) incur low computational overhead but lack the capacity to capture complex multi-modal spatiotemporal dependencies, while high-capacity models (such as Graph WaveNet and ST-MRGNN (>800 K parameters)) achieve higher accuracy at the cost of significantly increased training time and overfitting risk.

MST-Hyper Trans strikes an optimal balance with a total of 377 K parameters, which is comparable to STGCN (372 K) but substantially lower than Graph WaveNet and ST-MRGNN. This efficiency is achieved through the collaborative design of three modular components. This modular and dynamically adaptive design enables MST-Hyper Trans to achieve the highest prediction accuracy while maintaining computational efficiency. In the three-mode collaborative prediction task, MST-Hyper Trans requires only 4.957 s per training epoch and 0.186 s per inference, outperforming TSHDNet (5.124 s/0.201 s) and ST-MRGNN. The parameter size of 377 K thus represents a sweet spot, providing sufficient model capacity to capture complex dependencies (evidenced by an RMSE reduction of 8–11% compared to ST-MRGNN) while avoiding the computational redundancy associated with over-parameterized models.

The experimental results demonstrate that MST-Hyper Trans is highly scalable and applicable to large-scale urban networks. The model’s sub-linear growth in inference time with increasing node counts (as shown in Table 12) ensures its feasibility for metropolitan-scale deployments. For instance, in real-time traffic management systems requiring frequent updates (e.g., every 5–15 min), the inference time of 0.186 s for three modes is practically viable. Furthermore, the modular architecture allows for targeted optimization or simplification based on resource constraints, enhancing its adaptability to edge computing environments.

5.6. Ablation Analysis

To evaluate the contributions of different components in the MST-Hyper Trans model, we designed eight variants for ablation experiments. Each variant assesses the impact of specific component removal or replacement in MST-Hyper Trans on the model performance. A detailed description of each variant is provided below:

Single-scale Temporal Hyperedge (-MSTHCN_S): This variant only contains single-scale temporal hyperedge features.

No Multi-Scale Temporal Hypergraph Convolutional Network (-MSTHCN): This variant removes the MSTHCN module.

No Relationship-aware Attention Mechanism (-RAAM): This variant removes the relationship-aware attention mechanism and directly merges multiple structural hyperedges.

No Hierarchical DHGNN (-HDHGNN): This variant removes the Hierarchical DHGNN layer.

No Dynamic Spatial Hyperedge (-DMRSHN_DR): This variant removes dynamic hyperedges from the Hierarchical DHGNN layer.

No Dynamic Multi-Relation Spatial Hypergraph Network (-DMRSHN): This variant removes the DMRSHN module.

No Attention Mechanism Gating Fusion (-CMAGFN_Gate): This variant removes the gated weighted fusion in the CMAGFN module and replaces it with concatenation.

No Conditional Meta Attention Gated Fusion Network (-CMAGFN): This variant removes the CMAGFN module.

We recorded the average performance of the MST-Hyper Trans model and its variants across the three datasets after ten independent runs, as summarized in

Table 13. Ablation analysis of different MST-Hyper Trans components.

Models	NYC-Taxi			NYC-Bike			NYC-Subway
	RMSE	MAE	PCC	RMSE	MAE	PCC	RMSE	MAE	PCC
MST-HyperTrans	16.512	9.629	0.969	14.766	9.032	0.946	352.808	177.979	0.975
-CMAGFN_Gate	18.545	11.206	0.963	16.254	10.117	0.936	389.841	201.757	0.962
-MSTHCN_S	17.852	10.975	0.962	15.895	9.975	0.939	379.978	189.806	0.963
-DMRSHN_DR	17.279	10.046	0.965	15.435	9.742	0.938	376.290	183.415	0.965
-DMRSHN	18.568	11.243	0.966	16.431	10.051	0.940	409.317	202.603	0.964
-HDHGNN	18.206	11.150	0.963	15.909	10.042	0.939	385.023	199.757	0.963
- RAAM	17.963	10.944	0.965	16.038	9.826	0.938	368.588	192.711	0.965
-MSTHCN	18.558	10.721	0.964	16.240	10.128	0.939	393.267	206.479	0.966
-CMAGFN	18.552	11.231	0.965	16.485	10.016	0.936	407.383	204.826	0.960

. Evaluated using the RMSE and MAE metrics, the results demonstrate that MST-Hyper Trans outperforms all its variants on each dataset, confirming the effectiveness of the model components. Specifically, the variant -DMRSHN, which excludes the Dynamic Multi-Relation Spatial Hypergraph Network (DMRSHN), shows the most significant decline in performance, exhibiting the highest RMSE and MAE values. Removing the DMRSHN module resulted in the most significant performance degradation, which confirms that dynamic multi-relational space modeling is the most critical step in multi-modal prediction tasks. In addition to DMRSHN, the variant that excludes the Conditional Meta Attention Gated Fusion Network (CMAGFN) module also significantly impacts performance, indicating that dynamically extracting various spatial relationships is crucial for modeling the complex dependencies among heterogeneous spatial units in a multi-modal context. To assess the stability of the model, we conducted a box plot analysis of the experimental results for MST-Hyper Trans and its eight variant models. Figure 14 shows the results for the NYC-Taxi, NYC-Bike, and NYC-Subway datasets. Box plots visually illustrate the data distribution characteristics through a five-number summary (maximum, minimum, median, and upper and lower quartiles). The results indicate that MST-Hyper Trans consistently exhibits lower prediction errors across the three datasets compared with all variant models, with smaller variance, suggesting higher stability and predictive accuracy.

5.7. Comparison of Multi-Traffic Modes and Single-Traffic Mode

To analyze the impact of inter-modal relationships on the predictive performance of multi-modal demand collaborative forecasting, we conducted multiple experiments on three transportation datasets from New York City using the MST-Hyper Trans model. The experimental design is as follows:

Group I: Individual predictions for taxi, subway, and bike-sharing modes.

Group II: Collaborative predictions for taxi and subway modes, with bike-sharing mode predicted separately.

Group III: Collaborative predictions for taxi and bike-sharing modes, with subway mode predicted separately.

Group IV: Collaborative predictions for subway and bike-sharing modes, with the taxi mode predicted separately.

Group V: Simultaneous collaborative predictions for taxi, subway, and bike-sharing modes, with separate output for each prediction.

Table 14. Performance comparison of different combinations in 56 areas of Manhattan NYC datasets.

Models	NYC-Taxi			NYC-Bike			NYC-Subway
	RMSE	MAE	PCC	RMSE	MAE	PCC	RMSE	MAE	PCC
I	19.431	12.073	0.967	17.242	11.601	0.938	379.115	218.944	0.974
II	16.952	10.352	0.968	17.242	11.601	0.938	362.110	195.786	0.970
III	17.539	10.316	0.967	15.965	10.350	0.940	379.115	218.944	0.974
IV	19.431	12.073	0.967	14.292	9.368	0.950	354.762	193.058	0.971
V	16.512	9.629	0.969	14.766	9.032	0.946	352.808	177.979	0.975

summarizes the model performance comparisons across different dataset combinations, with colored numbers indicating independent predictions for each mode within the corresponding combination. The best results for each column are highlighted in bold font. By comparing the results of different combinations, we found that the best performance was never observed in the combination in which all modes were predicted independently (Group I). This indicates that integrating data from multiple transportation modes into a single model can significantly enhance the model’s ability to learn from demand fluctuations, thereby improving prediction accuracy. From the distribution of the best results, it is evident that the majority of the optimal outcomes originated from the collaborative prediction of all three transportation modes in Group V, whereas some of the best results came from the combined predictions of bike-sharing and subway modes (Group III). This may be attributed to the low cost of bike-sharing and subway services, whereby the passenger demand exhibits similar temporal variations. Consequently, considering the correlations between the different transportation modes during collaborative forecasting is crucial for feature extraction and model learning.

5.8. Analysis of Cross-Modal Multi-Scale Spatiotemporal Relationships

To explore cross-modal multi-scale spatiotemporal dependencies, we analyzed the attention weights for the spatiotemporal dependency features captured by our method across various modes and within each mode. First, we visualized the spatiotemporal distribution of attention weights for inter-modal features at different time scales, as shown in Figure 15. The shade of the color reflects the magnitude of the attention weights. It is evident that the three traffic modes are most significantly influenced by cross-modal features at the 7 × 24 super edge (hourly) time scale, with most stations/regions exhibiting attention weights above 0.5, and many stations/regions exceeding 0.8. This indicates that cross-modal features effectively capture the spatiotemporal dependencies between traffic modes on an hourly time scale. In contrast, at the 7 × 6 super edge (four-hour) time scale, the influence of cross-modal features on nodes is significantly weakened, with node demand predictions relying more on intra-modal demand characteristics.

Subsequently, we present a heatmap of the RMSE obtained from the MST-Hyper Trans model operating at multi-spatial scales compared to a single spatial scale, as shown in Figure 16. Figure 16a illustrates the performance of our model at multi-spatial scales, while Figure 16b–d show the results of the model variants operating under three distinct single spatial scales. The color intensity in the figures reflects the range of the RMSE values under different spatial scale models. From the color distribution, it can be observed that the predictive accuracy of the model is higher under multi-spatial scales, whereas the performance of the model variants with three single convolution kernels is somewhat poorer. This indicates that the multi-spatial scale approach better captures the local and global spatial correlations among multi-modal traffic nodes, facilitating more accurate collaborative demand predictions for multiple modes.

To assess the improvement in capturing the complex spatiotemporal dependencies among various transportation modes at multiple spatiotemporal scales, we visualized the spatial distribution of the RMSE improvement ratios of the spatiotemporal multi-scale MST-Hyper Trans model compared to the single spatiotemporal scale MST-Hyper Trans model under different transportation modes. The RMSE improvement distribution for the three transportation modes is illustrated in Figure 17. The color intensity in the figure reflects the RMSE improvement ratios in different regions under spatiotemporal multi-scale modeling. From the color distribution, it can be observed that the prediction accuracy of nodes for all three transportation modes generally improved under multi-temporal scale modeling. The taxi mode exhibited the most significant enhancement in prediction accuracy, with many regions showing RMSE improvement ratios that exceeded 16%. Furthermore, in areas with lower subway network coverage, spatiotemporal multi-scale modeling notably improves the prediction accuracy for shared bicycles. This indicates that in regions with lower demand or weaker inter-mode dependencies, the extraction of multi-temporal scale features can better compensate for the shortcomings of single temporal scale modeling.

5.9. Analysis of Information Interaction Mechanism Between Multi-Modes

To explore the information interaction mechanisms among multiple transportation modes, we conducted a heatmap analysis of the attention gating vector changes in the cross-modal spatiotemporal features and intra-modal node features within the CMAGFN module across different time periods of the day. The results are shown in Figure 18. Throughout the day, the influence of cross-modal spatiotemporal features on the demand for multi-modal transportation varies with time. Notably, during the time periods of 00:00–06:00 and 18:00–24:00, the weight of the cross-modal spatiotemporal features is at its peak.

To further investigate the spatial characteristics of cross-modal spatiotemporal information interaction mechanisms, we performed a visual analysis of the attention gating of intra-modal node features for taxi nodes within the CMAGFN module in the Manhattan area, focusing on four key time periods throughout the day (Figure 19). Analysis combined with Points of Interest (POI) information in the Manhattan area reveals that during most parts of the day, the proportion of cross-modal spatiotemporal information is relatively high in the Upper Manhattan area. This was attributed to the presence of numerous luxury apartments and villas, resulting in sparse demand data across the three transportation modes, thereby highlighting the significance of cross-modal spatiotemporal information. In contrast, for the Financial District, during weekday daytime hours (e.g., 06:00–18:00), the traffic volume is substantial, leading to a relatively minor impact of cross-modal spatiotemporal information on transportation demand. However, at night, the traffic volume in the Financial District decreases significantly, which subsequently enhances the importance of cross-modal spatiotemporal information.

6. Conclusions

This study addresses the core challenges in multi-modal transportation demand forecasting, namely the coexistence of heterogeneous spatial units (e.g., taxi zones, bicycle stations, and subway stations) and the complex spatiotemporal dynamics inherent in urban traffic systems. To this end, the Multi-Scale Spatiotemporal Dynamic Hypergraph Convolutional Network (MST-Hyper Trans) is proposed. The model achieves accurate and dynamic modeling of high-order spatiotemporal dependencies through the integration of three core modules: the Multi-scale Temporal Hypergraph Convolutional Network (MSTHCN) for temporal pattern decoupling, the Dynamic Multi-Relationship Spatial Hypergraph Network (DMRSHN) for spatial correlation capture, and the Conditional Meta Attention Gated Fusion Network (CMAGFN) for adaptive feature fusion. Experimental results on multi-modal NYC data demonstrate that the proposed model achieves an average improvement of 9.07% in PCC and reduces RMSE by 7.6% compared to state-of-the-art benchmarks, validating its superior accuracy.

Unlike traditional methods reliant on static graphs, the MST-Hyper Trans framework incorporates geographic proximity, passenger flow similarity, and transportation connectivity to construct structural hyperedges, while employing KNN and K-means algorithms to generate dynamic hyperedges. This approach overcomes the limitations of ordinary graph models in capturing multi-node relationships and explicitly models the evolution of spatial dependencies with traffic conditions. By decoupling multi-time scale (e.g., hourly, daily, weekly) patterns via MSTHCN and hierarchically extracting local-to-global spatial features via DMRSHN, the model characterizes spatiotemporal traffic characteristics more effectively. The CMAGFN module further enhances this by leveraging a lightweight meta-network with multi-head cross-attention to dynamically calibrate fusion weights based on real-time context (e.g., time, weather, and demand), enabling context-aware decision-making and improving prediction accuracy and interpretability.

Experimental validation on real-world datasets from New York City demonstrates the superior predictive accuracy of MST-Hyper Trans. For instance, on the NYC-Taxi dataset, it attained an RMSE of 16.51 and an MAE of 9.63, which are 10.6% and 14.23% lower, respectively, than the CoGNN baseline. Similar improvements were observed across all modes. Ablation studies confirm the indispensable contributions of each module, and case analyses reveal physically meaningful cross-modal interactions (e.g., subway–bicycle demand connections during peak hours), offering actionable insights for resource allocation and scheduling. While the model exhibits superior performance, future work will focus on integrating richer external data (e.g., refined weather, social events, and road dynamics) to enhance scalability in complex scenarios.