STA-GCN: Spatial-Temporal Self-Attention Graph Convolutional Networks for Traffic-Flow Prediction

Abstract

As an important component of intelligent transportation-management systems, accurate traffic-parameter prediction can help traffic-management departments to conduct effective traffic management. Due to the nonlinearity, complexity, and dynamism of highway-traffic data, traffic-flow prediction is still a challenging issue. Currently, most spatial–temporal traffic-flow-prediction models adopt fixed-structure time convolutional and graph convolutional models, which lack the ability to capture the dynamic characteristics of traffic flow. To address this issue, this paper proposes a spatial–temporal prediction model that can capture the dynamic spatial–temporal characteristics of traffic flow, named the spatial–temporal self-attention graph convolutional network (STA-GCN). In terms of feature engineering, we used the time cosine decomposition and one-hot encoding methods to capture the periodicity and heterogeneity of traffic-flow changes. Additionally, in order to build the model, self-attention mechanisms were incorporated into the spatial–temporal convolution to capture the spatial–temporal dynamic characteristics of traffic flow. The experimental results indicate that the performance of the proposed model on two traffic-volume datasets is superior to those of several baseline models. In particular, in long-term prediction, the prediction error can be reduced by over 5%. Further, the interpretability and robustness of the prediction model are addressed by considering the spatial dynamic changes.

1. Introduction

Traffic congestion has become a serious problem due to the development of the economy and the dramatic increase in vehicles on the road, which has also resulted in environmental pollution and low traffic efficiency [1]. The timely prediction of traffic parameters, such as traffic volume, speed, and occupancy, is believed to effectively improve road capacity, alleviate traffic congestion, provide traffic-route information for urban travelers, and help traffic authorities to make better decisions. Traffic-flow prediction is still an important component within intelligent transportation systems (ITS).

In the past decade, extensive of studies on traffic-flow prediction have been conducted, which can be divided into two categories: model-driven and data-driven approaches. The model-driven methods were introduced to simulate traffic problems in terms of important parameters. However, the assumptions made for the model were usually not suitable for real-world traffic conditions [2]. On the other hand, through the development of traffic detectors and big-data-mining techniques, data-driven methods have gradually become a major research topic.

Generally, statistical models and classical machine-learning models were first used for traffic-flow prediction at a single traffic node. Statistical models, involving ARIMA and its variants [3,4], were restricted by the assumption of stationary time series, which led to poor performance in predicting non-linear changes in traffic flow. Consequently, traditional machine-learning and deep-learning algorithms, such as KNN [5], SVR [6], XGBOOST [7], and LSTM [8], were proposed to capture the complex non-linear temporal changes for a single traffic node or section. However, the daily periodicity of traffic-flow changes in the time dimension and the heterogeneity of traffic-flow changes on weekdays and weekends are always ignored when feature engineering is conducted. Moreover, to construct the prediction model in a complex traffic network, it is necessary to establish a prediction model for each station without considering the spatial correlation.

In order to capture the spatial correlation in a traffic-flow-prediction model, many scholars define the traffic network through a regular grid space in terms of CNN [9,10,11]. However, the topological information between irregular traffic network nodes is often ignored. With the emergence of graph neural networks (GNN), deep learning has been extended to non-Euclidean fields [12], and the subsequent graph convolutional neural network has become an effective model for demonstrating the spatial dependence of traffic networks [13]. Yu et al. [14] used graph convolutional neural networks and gated convolutional neural networks to construct the STGCN model, which achieved good prediction results in real traffic scenarios. However, the fixed spatial topological structure and a convolution structure that fused an adjacent time-stamp as prediction information could not effectively capture dynamic spatial–temporal characteristics. To solve this issue, Guo et al. [15] proposed a graph convolutional network (GCN) based on a spatial–temporal attention mechanism without considering the heterogeneity of different time periods and spatial–temporal dynamic features.

The integration of graph convolutional networks within traffic-flow-prediction models is a current research topic, which aims to capture the dynamic spatiotemporal patterns of traffic flow and consider the periodic heterogeneity of traffic flow in the temporal dimension. Therefore, to address concerns over the current spatial–temporal traffic-flow-prediction models, in this paper, we propose methods involving the periodic characteristics of traffic flow and dynamic spatial–temporal dependence. Specifically, the periodic characteristics of traffic flow in daily changes and the heterogeneity of weekdays and weekends were considered during feature encoding, which were utilized as the input variables for traffic-flow prediction. In addition, a self-attention mechanism was introduced to capture the dynamic dependence of traffic flow on both the temporal and spatial dimensions. Furthermore, the gated mechanism was utilized to selectively extract the important features to further improve the prediction performance.

2. Literature Review

As mentioned above traffic-flow prediction providing typical time-series prediction is one of the essential parts of ITS, which is still a challenging issue. Over the past decades, various models and techniques were employed in traffic-flow prediction, which can be roughly divided into model-driven and data-driven approaches [16,17].

The model-driven approaches mainly depend on mathematical statistics or historical observations in constructing parametric models, which include time-series models, Kalman filtering models, spectral analyses, etc. Many classic models are proposed for traffic-flow prediction, such as the autoregression moving-average model (ARMA) and autoregression integral moving-average model (ARIMA) [18,19]. However, parametric models, which are easily affected by external environmental factors, do not effectively deal with the non-linear issue of traffic. Compared to model-driven methods, data-driven methods mainly focus on the relationship between input and output rather than the model parameters. Example models include SVR, Kalman filtering, and artificial neural networks (ANNs) [20], which clearly illustrate non-linear mapping capability for prediction and weakness in learning the spatio-temporal characteristics of network traffic flow.

Recently, the data-driven models of traffic-flow prediction have shifted to deep-learning methods in terms of the temporal and spatial features. For instance, the graph convolutional network (GCN) extends convolutional operations from structured data to graph-structured data, mainly through spatial domain convolutions [21] and spectral domain convolutions [22]. Regarding spatial domain convolutions, Li et al. [23] modeled traffic flow as a diffusion process on a directed graph and proposed the diffusion convolutional recurrent neural network (DCRNN), which captures spatial correlations based on bidirectional random walks to predict traffic flow on large road networks. Song et al. [24] added spatial–temporal correlations between adjacent matrices of a graph and constructed a prediction framework using a combination of spatial domain convolutional layers and gated linear units. Wu et al. [25] argued that explicit graph structures are not sufficient to represent node relationships in real-world graphs and proposed the Graph WaveNet architecture based on node-embedding learning. The graph convolutional layer in this architecture extracts structural features between nodes through spatial domain convolutions. Regarding spectral domain convolutions, Bruna et al. [26] proposed a general graph convolutional framework based on the graph Laplacian matrix, and Defferrard et al. [27] optimized this method using Chebyshev polynomials to approximate feature decomposition. Diao et al. [28] designed a dynamic Laplacian matrix that incorporates tensor decomposition to better capture spatial–temporal changes, and constructed a corresponding dynamic spatial–temporal graph convolutional neural network (DGCNN).

On the other hand, the Transformer model [29], proposed in 2017, is a novel foundational model that holds equal importance to CNN and RNN. In recent years, the Transformer model has made significant progress in both NLP and CV fields, and its core algorithm, the self-attention mechanism, has been widely developed and applied. Velickovic et al. [30] used self-attention layers to process graph-structured data through neural networks and achieved state-of-the-art results. Cai et al. [31] used Transformer to capture temporal correlations in traffic flow and combined these with GNN, which can capture spatial dependencies, to build a spatial–temporal traffic-flow prediction model. Xu et al. [32] used a self-attention mechanism to simultaneously capture spatial and temporal dependencies of traffic-state changes, but when capturing spatial dependencies, the computational cost was excessively high due to the large number of spatial nodes. In real traffic scenarios, due to factors such as road characteristics and PIO types, although two nodes are adjacent, there may be differences in traffic state in the same time period, as shown in Figure 1.

Nodes 1, 2, and 3 are adjacent to node 0, but each node has different traffic trends and peak periods. The results indicate heterogeneity in traffic-state changes among adjacent traffic nodes during the same time period. Therefore, a fixed-graph structure is not sufficient to capture complex dynamic traffic-flow changes. In this study, a spatial self-attention mechanism was used to capture the dynamic changes in traffic flow in the spatial dimension. Subsequently, two different attention forms were used to capture the spatial–temporal dynamic features, namely a time self-attention mechanism, which considers the entire sequence for temporal dependency relationships, and a spatial attention mechanism, for road-network-traffic-node spatial characteristics.

3. Dataset and Methodology

3.1. Dataset

Two publicly available datasets from real world are utilized in this study. (1) PeMSD04, which contains traffic-flow data from the San Francisco Bay Area. The dataset includes 340 sensors, and the selected experimental period covers two months, from 1 January 2018 to 28 February 2018. (2) PeMSD08, a real-time traffic-flow-sensor data set, includes data from 295 sensor detectors from a two-month experiment period from 1 July 2016 to 31 August 2017. The datasets were collected by the Caltrans Performance Measurement System (PeMS) [33].

3.2. Symbols and Feature Encoding

In this section, we introduce some necessary symbols and definitions that are used in this article. Next, we provide some relevant knowledge about the theoretical traffic-flow-prediction model in this article.

Table 1. Symbols.

Notation	Description
$N$	Number of traffic nodes on the road network
$S$	Historical sequence length
$P$	Future sequence length
$d$	The dimensionality of the traffic-node attributes mapped by the input layer
$d^{\prime }$	The dimensionality of the node attributes after passing through a temporal gated convolutional layer
$d^{″}$	The dimensionality of the node attributes after passing through a graph convolutional layer
$\chi \in {\mathbb{R}}^{S\times N\times 4}$	The spatial–temporal information of the input
$f\in {\mathbb{R}}^{S\times d}$	The information from the self-attention layer of an input time for a single transportation node
$W^{\mathcal{T}}\in {\mathbb{R}}^{S\times S}$	Self-attention matrix for time
$f^{\mathcal{T}}\in {\mathbb{R}}^{S\times d}$	The information from a single traffic node after passing through self-attention for time
$F^{\mathcal{T}}\in {\mathbb{R}}^{N\times S\times d}$	The information from all traffic nodes after passing through self-attention for time
$F^c\in {\mathbb{R}}^{N\times S^{\prime }\times d^{\prime }}$	The information after passing through a temporal gated convolutional layer
$f^c\in {\mathbb{R}}^{N\times d^{\prime }}$	The information of the input space attention for a single time slice
$f^S\in {\mathbb{R}}^{N\times d^{\prime }}$	The information after passing through a spatial attention layer for a single time slice
$F^S\in {\mathbb{R}}^{S^{\prime }\times N\times d^{\prime }}$	Information from the spatial self-attention layer across all time series
$F^G\in {\mathbb{R}}^{S^{\prime }\times N\times d^{″}}$	Information from the spatial graph convolutional layer across all time series
$y\in {\mathbb{R}}^{P\times N\times 1}$	The final output of spatial–temporal prediction information

summarizes the main symbols.

In order to reflect the daily periodicity of traffic-flow changes in the training dataset, the method of cosine decomposition [34] was used to map the daily periodicity features to two dimensions separately, as shown in Equations (1) and (2). The heterogeneity of traffic conditions between working days and weekends was differentiated by using one-hot encoding, with weekends coded as 0 and working days coded as 1. As a result, the input data had 4 channels: traffic speed, time sine component, time cosine component, and heterogeneity components for working days and weekends.

$$X_s=\mathit{sin}(2\pi \times \frac{t}{T})$$

(1)

$$X_c=\mathit{cos}(2\pi \times \frac{t}{T})$$

(2)

The $X_s$ and $X_c$ are the sequences of cosine and sine components, respectively, for a time series. The $T$ is the length of the period, and $\left\{\mathrm{0,1},\cdots T-1\right\}$ is a set of integer sequences starting from 0.

3.3. Spatial–Temporal Self-Attention Graph Convolution Networks (STA-GCN)

In this section, the detailed description of the STA-GCN (spatial–temporal self-attention graph convolution network) model structure is provided. As shown in Figure 2, the model consists of an input layer, an output layer, and a spatial–temporal layer. The input layer is a fully connected neural network that maps the input spatial–temporal information $\chi \in {\mathbb{R}}^{S\times N\times 4}$, which is encoded with time, from 4 channels to a higher-dimensional channel. The spatial–temporal layer is composed of a temporal layer and a spatial layer, and we describe the model structures of these two layers in detail below.

3.3.1. Temporal Self-Attention

The self-attention mechanism is the core algorithm of the Transformer model, which can capture the dependency relationship between sequences through the query matrix Q, key matrix K, and value matrix V. Similarly, the dependency relationship between traffic-flow features at different time steps can be captured by the self-attention mechanism. The formula for calculating the temporal self-attention is shown in Equation (3).

$$A^{\mathcal{T}}=\mathit{softmax}\left(\frac{Q^{\mathcal{T}}{\left(K^{\mathcal{T}}\right)}^T}{\sqrt{d^{\mathcal{T}}}}\right)V^{\mathcal{T}}$$

(3)

The $Q^{\mathcal{T}}\in {\mathbb{R}}^{L_Q\times d^{\mathcal{T}}}$, $K^{\mathcal{T}}\in {\mathbb{R}}^{L_K\times d^{\mathcal{T}}}$, and $V^{\mathcal{T}}\in {\mathbb{R}}^{L_V\times d^{\mathcal{T}}}$ refer to the time-series query matrix, key matrix, and value matrix, respectively. These matrices are used in the self-attention mechanism to capture the dependency relationship between traffic-flow features at different time steps. The query matrix A contains the queries that are compared with the keys in the key matrix B, and the value matrix C contains the values corresponding to the keys. The softmax function performs a normalization operation on the temporal dependencies at each time step. The $\sqrt{d^{\mathcal{T}}}$ scaling factor $\sqrt{d^{\mathcal{T}}}$ is applied to prevent significant differences in the probability distribution of the matrix resulting from the dot product of points $Q^T$ and ${\left(K^T\right)}^T$ after passing through the softmax function. After obtaining the attention-probability matrix, a linear layer is applied to obtain $f^{\mathcal{T}}\in {\mathbb{R}}^{S\times d}$, which is then residually connected with $f\in {\mathbb{R}}^{S\times d}$ and normalized along the $S\times d$ dimension. Finally, the output is passed through a ReLu activation function.

Due to the different geographical locations of transportation nodes, different transportation nodes exhibit different traffic conditions during a certain time period. As shown in Figure 3, there is a significant difference in the traffic-speed changes of the five transportation nodes during the time period from 8:00 to 9:00, indicating the existence of heterogeneity in spatial traffic characteristics. Therefore, it is necessary to capture the time-varying dependence relationship for each transportation node. As shown in Figure 4, the spatial–temporal information from multiple transportation nodes is summed up after passing through a time-attention mechanism, forming an input $F^{\mathcal{T}}\in {\mathbb{R}}^{N\times S\times d}$ to the temporal gated convolution.

3.3.2. Temporal Gated Convolution

Although the RNN model is widely used for time-series prediction, the computational complexity of this recurrent neural network structure is relatively high. In contrast, CNNs have the advantages of fast training speed, simple structure, and no dependency constraints. Therefore, we introduced gated temporal convolution [35], and used an appropriate dilation rate for dilated convolution to obtain richer temporal dynamic correlations along the time axis. The structure of the gated temporal convolution is shown in Figure 4. After $F^{\mathcal{T}}\in {\mathbb{R}}^{N\times S\times d}$ is passed through a 1D convolution layer, it is divided into two parts, and $F^c\in {\mathbb{R}}^{N\times S\mathrm{\prime }\times d\mathrm{\prime }}$ is obtained through the GLU activation function, as shown in Equation (4).

$$F^c=\psi \left({\mathrm{\Phi }}_1 \mathrm{\ast } F^{\mathcal{T}}+F^{\mathcal{T}}\right)\odot \sigma \left({\mathrm{\Phi }}_2 \mathrm{\ast } F^{\mathcal{T}}+b_2\right)$$

(4)

where ${\mathrm{\Phi }}_1$ and ${\mathrm{\Phi }}_2$ are two one-dimensional convolution operations. The $\psi$ and $\sigma$ represent the tanh and sigmoid functions, respectively. The symbol $\odot$ represents the element-wise multiplication.

3.3.3. Spatial Self-Attention

Since the spatial–temporal information output from the time module has already undergone convolution on the time axis, each vector in the time dimension can represent the spatial–temporal information of a convolution-length-time segment. The dependency relationship between each traffic node for each time segment can also be captured through self-attention mechanism. The formula for computing spatial self-attention is shown in Equation (5).

$$A^S=\mathit{softmax}\left(\frac{Q^S{\left(K^S\right)}^S}{\sqrt{d^S}}\right)V^S$$

(5)

where $Q^S\in {\mathbb{R}}^{L_Q\times d^S}$, $K^S\in {\mathbb{R}}^{L_K\times d^S}$, and $V^S\in {\mathbb{R}}^{L_V\times d^S}$ refer to the time-series query matrix, key matrix, and value matrix, respectively. These matrices are used in the self-attention mechanism to capture the dependency relationship between traffic-flow features at different time steps. The query matrix A contains the queries that are compared with the keys in the key matrix B, and the value matrix C contains the values corresponding to the keys. The softmax function performs a normalization operation on the temporal dependencies at each time step. The $\sqrt{d^S}$ scaling factor $\sqrt{d^S}$ is applied to prevent significant differences in the probability distribution of the matrix resulting from the dot product of points $Q^S$ and ${\left(K^S\right)}^T$ after passing through the softmax function. After obtaining the attention-probability matrix, a linear layer is applied to obtain $f^S\in {\mathbb{R}}^{N\times d}$, which is then residually connected with $f\in {\mathbb{R}}^{N\times d}$ and normalized along the $N\times d$ dimension. Finally, the output is passed through a ReLu activation function.

3.3.4. Spatial Graph Convolution

The graph convolutional network (GCN) extends the convolution operation for structured data to graph-structured data. In order to fully utilize the topological relationships between various transportation nodes, spectral graph convolution is used to handle spatial dimension correlations. By analyzing the Laplacian matrix and its eigenvalues, the structural properties of the graph can be obtained. The Laplacian matrix of the graph is represented as $\mathbf{L}=\mathbf{D}-\mathbf{A}$, and in standard form as $\mathbf{L}={\mathbf{I}}_N-{\mathbf{D}}^{-\frac{1}{2}}\mathbf{A}{\mathbf{D}}^{-\frac{1}{2}}$, where $A$ is the adjacency matrix, ${\mathbf{I}}_N$ is the identity matrix, $D$ is the degree matrix, $L=U\mathbf{\Lambda }{U}^T$ is the eigenvalue decomposition of Laplacian matrix, $\mathbf{\Lambda }=\mathit{diag}\left(\left[{\lambda }_0,\dots ,{\lambda }_{N-1}\right]\right)\in R^{N\times N}$ is a diagonal matrix, and $U$ is the Fourier basis.

The traffic parameters at time $t$ and their information on graph $G$ are represented as $x={\mathbf{x}}_t^v$. The Fourier transform of this information is defined as $\widehat{x}={\mathbf{U}}^{T}x$. Graph convolution is a convolution operation that can be implemented by replacing the classic convolution operator with a diagonalizable linear operator $x$ in the Fourier domain. The formula for the convolution operation of $g_{\theta }$ on the graph $G$ is shown in Equation (6).

$$g_{\theta } \mathrm{\ast } Gx=g_{\theta }(\mathbf{L})x=g_{\theta }(\mathbf{U}\mathbf{\Lambda }{\mathbf{U}}^T)x=\mathbf{U}{g}_{\theta }(\mathbf{\Lambda }){\mathbf{U}}^{T}x$$

(6)

Due to the computational complexity caused by all eigenvalues and eigenvectors of the Laplacian matrix in the training process of spectral convolutional networks, this study uses graph convolutional networks based on Chebyshev polynomials to accelerate the solution of the feature matrix. The Chebyshev polynomial, as shown in Equation (7), is used for this purpose.

$$g_{\theta }(\mathbf{\Lambda })=\sum _{k=0}^{k-1}{\theta }_k{\mathbf{T}}_k(\tilde{\mathbf{\Lambda }})$$

(7)

where $\theta$ is the Chebyshev segment coefficient vector, $T_k(\tilde{\mathbf{\Lambda }})$ is the $k$-th order Chebyshev polynomial of $\tilde{\mathbf{\Lambda }}$, and in $\tilde{\mathbf{\Lambda }}=2\mathbf{\Lambda }/{\lambda N}_{max}$,${\lambda }_{max}$ is the largest eigenvalue.

For the $k$-th-order Chebyshev polynomial, there is a ${\mathbf{T}}_k=2x{\mathbf{T}}_{k-1}(x)-{\mathbf{T}}_{k-2}(x)$, which has a recurrence relation with the first two terms, $T_0(x)=1$ and $T_1(x)=x$. The convolution operation conducted after approximating the Chebyshev polynomial shown in Equation (8) is depicted in Figure 5.

$$g_{\theta } \mathrm{\ast } Gx=\mathbf{U}\left(\sum _{i=1}^K{\theta }_i{\mathbf{T}}_k(\tilde{\mathbf{\Lambda }})\right)U^{T}x\approx \sum _{i=1}^K{\theta }_i{\mathbf{T}}_k(\tilde{\mathbf{L}})x$$

(8)

4. Results and Discussion

4.1. Data Pre-Processing

For the two datasets, we collected traffic parameters into time windows of 5 min and normalized them. We used 60% of the data as the training set, 20% as the validation set, and 20% as the test set. We use the threshold Gaussian method to establish an adjacency matrix and calculated the weights A between each detector according to Equation (9).

$$w_{ij}=\left\{\begin{array}{cc}\mathit{exp}\left(-\frac{d_{ij}^2}{{\sigma }^2}\right),& i\ne j \mathrm{and} \mathit{exp}\left(-\frac{d_{ij}^2}{{\sigma }^2}\right)\ge ϵ\\ 0,& \mathrm{otherwise}. \end{array}\right.$$

(9)

where $d_{ij}$ represents the distance between detector $i$ and detector $j$. The ${\sigma }^2$ and $w$, setting 10 and 0.5 respectively, represent the sparsity and distribution.

4.2. Parameter Setting

In this study, the STA-GCN model was implemented using the PyTorch framework, with the following parameter settings during training: a batch size of 32, the Adam optimizer, RMSE as the loss function, and an initial learning rate of 0.001.

The output dimension of the input layer was set to 32, and a (3,1) kernel was chosen for the temporal convolution based on previous work.

The Chebyshev polynomial in the graph convolution layer was set to 3.

The number of temporal and spatial modules was set to 2, and the model used the previous 12 time windows to predict the next 12 time windows. To prevent overfitting, the model stopped training if the validation loss did not decrease within 10 epochs.

4.3. Baseline Models

We compared our model to other baseline models trained on the same computer, including:

ARIMA: a classic algorithm for time-series analysis and forecasting that uses an autoregressive integrated moving-average approach.

SVR: a machine-learning algorithm that can capture non-linear temporal features.

FNN: a feedforward neural network that uses multiple hidden layers to capture non-linear temporal changes.

GRU: a type of RNN model with gated recurrent units.

FC-LSTM: a recurrent neural network with fully connected LSTM hidden units.

STGCN: a spatial–temporal traffic-flow prediction model that combines gated temporal convolutional networks with graph convolutional networks.

ASTGCN: a spatial–temporal traffic-flow prediction model that incorporates attention mechanisms to capture temporal and spatial changes. The difference between our model and ASTGCN is that ASTGCN extracts both temporal and spatial features from attention mechanisms and inputs them into spatial–temporal convolutional layers, while our model uses self-attention mechanisms to capture spatial–temporal dynamic dependencies, which are input into graph convolutional networks. Therefore, the embedding of spatial–temporal attention mechanisms is different, and our model uses self-attention mechanisms, which can capture higher-dimensional spatial–temporal dynamic characteristics.

4.4. Experimental Results

We compared our model with seven baseline models regarding the performances of different prediction horizons in

Table 2. Performances of traffic-flow-prediction models.

Datasets	T	Metric	ARIMA	SVR	FNN	GRU	FC-LSTM	STGCN	ASTGCN	STA-GCN
PeMSD04	15	MAE	25.52	25.34	25.02	24.85	24.32	22.31	21.02	19.02
		RMSE	33.21	32.02	31.89	30.24	30.08	35.92	32.98	29.79
		MAPE	18.25%	18.02%	17.85	17.23	16.85	17.05%	15.21%	12.55%
	30	MAE	31.75	30.23	29.52	29.20	28.78	24.02	21.87	18.05
		RMSE	40.26	38.67	37.52	37.21	36.84	38.94	34.12	30.54
		MAPE	23.56%	21.23%	20.32	19.85	18.02	16.83%	15.24%	12.51%
	60	MAE	35.65	32.35	31.25	30.26	28.35	26.12	23.02	18.23
		RMSE	52.25	48.28	47.02	46.32	44.25	40.89	36.51	31.20
		MAPE	26.69%	23.78%	21.02	20.23	18.20	17.23%	16.95%	12.32%
PeMSD08	15	MAE	19.06	19.07	19.08	19.21	19.12	15.26	14.94	12.01
		RMSE	29.72	29.64	29.68	29.82	29.71	23.24	22.85	20.05
		MAPE	13.10%	12.98%	13.02%	13.45%	13.07%	10.19%	9.91%	7.21%
	30	MAE	23.12	21.51	21.05	20.85	20.13	15.52	15.04	12.30
		RMSE	35.53	32.25	31.25	31.01	30.65	23.88	23.23	21.45
		MAPE	16.21	14.62%	13.71%	13.69	13.54%	9.76%	9.60%	7.69%
	60	MAE	29.21	24.25	23.91	23.85	22.35	17.43	16.91	12.84
		RMSE	40.02	37.21	36.13	36.01	34.10	26.68	25.82	22.25
		MAPE	18.02%	15.03%	14.35%	14.24%	14.01%	11.74%	10.95%	7.83%

, and the AST-GCN showed the best performance on both datasets. Specifically, the traditional time-series prediction methods (ARIMA, SVR) performed well on short horizons (15 min), but struggled with longer horizons (30 min, 60 min), indicating that these methods have difficulty in handling complex nonlinear traffic-flow data. Among the deep-learning models, FC-LSTM, which considers spatial dependence, and the models containing GCN modules outperformed the traditional deep-learning models (FNN and GRU). Although FC-LSTM can capture spatial features, it does not utilize the topological structure of the road network and has a significant amount of data redundancy during training. Therefore, FC-LSTM is inferior to STGCN, ASTGCN, and STA-GCN. The prediction performances of ASTGCN and STA-GCN were better than those of STGCN, demonstrating the effectiveness of adding spatial–temporal attention mechanisms to capture dynamic spatial–temporal features. Furthermore, our STA-GCN outperformed ASTGCN, especially on 60-minute horizons, with the prediction errors were reduced by 4.63% and 3.12% on the PeMSD04 and PeMSD08 datasets, respectively. These results demonstrate the superiority of the hierarchical embedding of spatial–temporal attention and the superiority of spatial–temporal self-attention mechanisms.

The training loss function of the model was considered to be converged when the validation set did not decrease further after ten epochs of training. Specifically, the number of training epochs and the final RMSE metric of the three types of model were compared when they reached convergence, as shown in Figure 6. It was found that the STA-GCN model had a faster convergence rate and lower prediction error.

To further demonstrate that the predictive performance of STA-GCN is superior to that of STGCN and ASTGCN, we randomly selected one traffic node’s weekday and holiday prediction results from the PeMSD08 dataset for visualization. The visualization results for the weekdays are shown in Figure 7. During the time periods when the road-traffic volume suddenly increases (4:00–5:00) or decreases (17:30–18:30), STA-GCN can fit the trends in traffic-volume changes better than STGCN and ASTGCN. Similarly, the holiday prediction results are shown in Figure 8. During the time periods when the road-traffic volume suddenly increases (6:00–7:00) or decreases (19:00–20:00), STA-GCN can fit the trends in traffic-volume changes better than STGCN and ASTGCN.

To better study the roles of the spatial self-attention mechanisms in the models, in Figure 9, the top image displays the traffic-flow changes of four randomly selected traffic nodes (A, B, C, and D) over the course of one hour. The bottom 10 images show the self-attention-score matrices of the four traffic nodes over 10 time intervals. In the 0–2 time interval, A has a higher attention score on B because the traffic-flow trends of the two nodes show a negative correlation during this time. In the 2–4 time interval, the attention scores of A, B, and C on D are the highest because the traffic flows of B and C are similar to that of D, and the traffic-flow trends are also similar. Moreover, A has a similar trend to D, and their traffic-flow increases are almost the same, at 20 and 24 respectively. Therefore, our model not only performs well in prediction but also has good interpretability.

5. Conclusions

Traffic-flow prediction is an essential part of ITS research on improving traffic efficiency and safety. Both traffic managers and travelers can benefit from timely and accurate traffic-flow prediction. However, current spatial–temporal traffic-flow-prediction models rarely consider the periodicity and heterogeneity of traffic-flow changes from the network perspective.

To address this issue, in this paper, we proposed the spatial–temporal self-attention graph convolution network (STA-GCN) model, considering both periodic characteristics and the dynamic spatial–temporal dependence of network traffic flow. Specifically, Fourier decomposition was used to decompose the time series into periodic variables and one-hot encoding was used to distinguish the heterogeneity. Therefore, the proposed model incorporated spatial–temporal self-attention mechanisms into graph convolutional networks and time-gated convolutional networks to capture dynamic changes in traffic-flow characteristics.

Through a performance comparison with previous prediction models, the proposed model demonstrated a better performance on selected traffic-flow datasets, which indicates the effectiveness of STA-GCN in traffic-flow-prediction tasks. Furthermore, the visual analysis of the spatial-attention-prediction process was conducted to illustrate the interpretability of our model. Moreover, the proposed model is applicable to the prediction of other traffic parameters, including traffic speed and time occupancy, as well as passenger-flow predictions for subways and bus networks.

Although the STA-GCN model captures temporal–spatial traffic-flow patterns through the self-attention mechanism, the structure of the traffic-node network is fixed and unchanging. In future research, a promising method to improve the graph convolutional network would be to incorporate graph-theory-related approaches to enable the network to acquire dynamic characteristics. Furthermore, advanced neural network modules and more real-world datasets should be considered for integration into the prediction model.