STN-GCN: Spatial and Temporal Normalization Graph Convolutional Neural Networks for Traffic Flow Forecasting
Abstract
:1. Introduction
- This study proposes a spatial-temporal normalized graph convolutional neural network model. The model combines a time convolutional network (TCN) with spatial-temporal feature normalization in the time feature extraction, which can effectively remove noise and more fully extract the temporal feature;
- This study incorporates a transformer into the deep learning model and uses the spatial-transformer module to extract spatial features from the input data. The model connects the spatial-temporal features extracted after k spatial-temporal modules with residuals. The stacked data features are skip-connection through a fully connected layer to finally output the predicted values;
- The curriculum learning method is added to the training. Training is conducted in groups. It starts with simple samples and gradually accumulates up to the whole training sample, allowing achieving the best results in training more easily. The results of experiments on two real-world datasets show that the proposed model has an improvement in performance.
2. Background and Related Work
2.1. Graph Neural Networks
- Based on graph structure feature extraction [18,19,20], researchers began to combine graph theory and neural networks to propose a traffic flow forecasting model according to graph structure feature extraction. The graph attention networks (GAT) [21] model proposes a new graph neural network model, the graph attention network, which enables the network to adaptively assign different weights to different nodes through the introduction of an attention mechanism. In comparison to GCN, GAT can more effectively deal with sparse graph data, but it has slightly higher computational complexity. These models mainly use the adjacency matrix of the graph to represent the topology of the road network in order to extract the spatial relationships between different regions;
- Models based on graph convolutional neural networks [22,23,24] started to apply the ideas of convolutional neural networks (CNN) to graph neural networks and proposed traffic flow forecasting models based on graph convolutional neural networks (GCN). These models can adaptively learn the spatial dependencies between different regions and incorporate them into the forecasting model. However, this method has some problems with computational complexity and does not work well for sparse graphs;
- Models based on spatial-temporal graph [25,26] started to incorporate the temporal dimension into graph neural networks and proposed traffic flow forecasting models based on spatial-temporal graphs. With better predictive capabilities, these models can consider both spatial and temporal-series relationships between different regions.
2.2. Temporal Dependence
2.3. Spatial Correlation
3. Preparation
3.1. Definition of the Traffic Road Network Graph
3.2. Feature Matrix
4. Methodology
4.1. General Model Framework
4.2. Temporal Extraction Module (STT-BLOCK)
4.3. Spatial Extraction Module (ST-BLOCK)
4.3.1. Spatial Location Embedding Layer
4.3.2. Static Graph Convolution Layer
4.3.3. Dynamic Graph Convolution Layer
4.3.4. Gating Mechanism for Feature Fusion
5. Experiments
5.1. Experimental Setups
5.2. Data Description
5.3. Evaluation Indicators
5.4. Experimental Results
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Li, Y.; Yu, R.; Shahabi, C.; Liu, Y. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, International Conference on Learning Representations. In Proceedings of the ICLR 2018, Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Yao, H.X.; Wu, F.; Ke, J.T.; Tang, X.F.; Jia, Y.T.; Lu, S.Y.; Gong, P.H.; Ye, J.P.; Li, Z.H. Deep multi-view spatial-temporal network for taxi demand prediction. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), New Orleans, LA, USA, 2–7 February 2018; pp. 2588–2595. [Google Scholar]
- Xu, X.; Zhang, L.L.; Zhang, X.; Qi, K.; Gui, C.G. Enhanced-Historical Average for Long-Term Prediction. In Proceedings of the 2022 2nd International Conference on Computer, Control and Robotics (ICCCR), Shanghai, China, 18–20 March 2022. [Google Scholar]
- Javad, S.; Saeednia, N. Neuro-Fuzzy Modeling of Data Singular Spectrum Decomposition and Traffic Flow Prediction. Iran. J. Sci. Technol. Trans. Electr. Eng. 2020, 44, 519–535. [Google Scholar]
- Yang, H.Y.; Li, X.T.; Qiang, W.H.; Zhao, Y.H.; Zhang, W.; Tang, C. A network traffic forecasting method based on SA optimized ARIMA-BP neural network. Comput. Netw. 2021, 193, 108102. [Google Scholar] [CrossRef]
- Yang, Z.; Dong, R.X. Short-term traffic flow prediction model based on deep learning regression algorithm. Int. J. Comput. Sci. Math. 2021, 14, 155–166. [Google Scholar]
- Lu, Z.L.; Lv, W.F.; Xie, Z.P.; Du, B.W.; Huang, R.H. Leveraging Graph Neural Network with LSTM For Traffic Speed Prediction. In Proceedings of the 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innova, Leicester, UK, 19–23 August 2019. [Google Scholar]
- Guo, S.N.; Lin, Y.F.; Li, S.J.; Chen, Z.M.; Wan, H.Y. Deep spatial–temporal 3D convolutional neural networks for traffic data forecasting. IEEE Trans. Intell. Transp. Syst. 2019, 20, 3913–3926. [Google Scholar] [CrossRef]
- Yu, B.; Yin, H.T.; Zhu, Z.X. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, Stockholm, Sweden, 13–19 July 2018; pp. 3634–3640. [Google Scholar]
- Cai, L.; Janowicz, K.; Mai, G.C.; Yan, B.; Zhu, R. Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting. Trans. GIS 2020, 24, 736–755. [Google Scholar] [CrossRef]
- Zheng, C.; Fan, X.; Wang, C.; Qi, J. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 1234–1241. [Google Scholar]
- Wang, Y.; Jing, C.F.; Xu, S.S.; Guo, T. Attention based spatiotemporal graph attention networks for traffic flow forecasting. Inf. Sci. 2022, 607, 869–883. [Google Scholar] [CrossRef]
- Cui, Z.Y.; Henrickson, K.; Ke, R.; Wang, Y.H. Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting. IEEE Trans. Intell. Transp. Syst. 2019, 21, 4883–4894. [Google Scholar] [CrossRef] [Green Version]
- Shin, Y.Y.; Yoon, Y. Incorporating dynamicity of transportation network with multi-weight traffic graph convolutional network for traffic forecasting. IEEE Trans. Intell. Transp. Syst. 2020, 23, 2082–2092. [Google Scholar] [CrossRef]
- Gunawan, J.; Huang, C.Y. An Extensible Framework for Short-Term Holiday Load Forecasting Combining Dynamic Time Warping and LSTM Network. IEEE Access 2021, 9, 106885–106894. [Google Scholar] [CrossRef]
- Li, M.Z.; Zhu, Z.X. Spatial-temporal fusion graph neural networks for traffic flow forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtual, 19–21 May 2021; Volume 35, pp. 4189–4196. [Google Scholar]
- Wu, Z.H.; Pan, S.R.; Long, G.D.; Jiang, J.; Zhang, C.Q. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, 10–16 August 2019; pp. 1907–1913. [Google Scholar]
- Bai, L.; Yao, L.; Li, C.; Wang, X.Z.; Wang, C. Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Virtual, 6–12 December 2020. [Google Scholar]
- Li, D.; Lasenby, J. Spatiotemporal Attention-Based Graph Convolution Network for Segment-Level Traffic Prediction. IEEE Trans. Intell. Transp. Syst. 2021, 23, 8337–8345. [Google Scholar] [CrossRef]
- Li, Z.; Liu, H.; Zhang, Z.; Liu, T.; Xiong, N.N. Learning Knowledge Graph Embedding with Heterogeneous Relation Attention Networks. IEEE Trans. Neural Netw. Learn. Syst. 2022, 33, 3961–3973. [Google Scholar] [CrossRef] [PubMed]
- Kong, X.; Xing, W.; Wei, X.; Bao, P.; Lu, W. STGAT: Spatial-temporal graph attention networks for traffic flow forecasting. IEEE Access 2020, 8, 134363–134372. [Google Scholar] [CrossRef]
- Wu, H.; Zhou, H.; Zhao, J.; Xu, Y.; Ma, T.; Bian, Y. Deep Spatio-Temporal Residual Networks for Connected Urban Vehicular Traffic Prediction. In Proceedings of the 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), Victoria, BC, Canada, 18 November–16 December 2020; pp. 1–6. [Google Scholar] [CrossRef]
- Liang, Y.; Zhao, Z.; Sun, L. Dynamic spatiotemporal graph convolutional neural networks for traffic data imputation with complex missing patterns. arXiv 2021, arXiv:2109.08357. [Google Scholar]
- Zhang, Z.; Li, Z.; Liu, H.; Xiong, N.N. Multi-Scale Dynamic Convolutional Network for Knowledge Graph Embedding. IEEE Trans. Knowl. Data Eng. 2022, 34, 2335–2347. [Google Scholar] [CrossRef]
- Pan, Z.Y.; Zhang, W.T.; Liang, Y.X.; Zhang, W.N.; Yu, Y.; Zhang, J.B.; Zheng, Y. Spatio-Temporal Meta Learning for Urban Traffic Prediction. IEEE Trans. Knowl. Data Eng. 2022, 34, 1462–1476. [Google Scholar] [CrossRef]
- Xu, M.X.; Dai, W.R.; Liu, C.M.; Gao, X.; Lin, W.Y.; Qi, G.J.; Xiong, H.K. Spatial-Temporal Transformer Networks for Traffic Flow Forecasting. arXiv 2020, arXiv:2001.02908. [Google Scholar]
- Cao, D.F.; Wang, Y.J.; Duan, J.Y.; Zhang, C.; Zhu, X.; Huang, C.R.; Tong, Y.H.; Xu, B.X.; Bai, J.; Tong, J.; et al. Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting. arXiv 2020, arXiv:2103.07719. [Google Scholar]
- Li, J.H.; Yang, J.; Gao, L.; Wei, L.; Mao, F.Q. Dynamic Spatial-Temporal Graph Convolutional GRU Network for Traffic Forecasting. In Proceedings of the ICSCC 2021: 6th International Conference on Systems, Control and Communications, Chongqing, China, 15–17 October 2021; pp. 19–24. [Google Scholar]
- Lee, K.; Rhee, W. DDP-GCN: Multi-graph convolutional network for spatiotemporal traffic forecasting. Transp. Res. Part C Emerg. Technol. 2022, 134, 103466. [Google Scholar] [CrossRef]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30, Long Beach, CA, USA, 4–9 December 2017; pp. 5998–6008. [Google Scholar]
- Deng, J.L.; Chen, X.S.; Jiang, R.H.; Song, X.; Tsang, I.W. ST-Norm: Spatial and Temporal Normalization for Multi-Variate Time Series Forecasting. In Proceedings of the KDD’21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Singapore, 14–18 August 2021; ACM: New York, NY, USA, 2021. [Google Scholar]
- Yu, F.; Koltun, V. Multi-Scale Context Aggregation by Dilated Convolutions. In Proceedings of the ICLR 2016 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2–4 May 2016. [Google Scholar]
- Dauphin, Y.N.; Fan, A.; Auli, M.; Grangier, D. Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6–11 August 2017. [Google Scholar]
- Guo, S.N.; Lin, Y.F.; Feng, N.; Song, C.; Wan, H.Y. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the Ninth {AAAI} Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, HI, USA, 27 January–1 February 2019; pp. 922–929. [Google Scholar]
- Du, S.Y.; Liu, Y.Y.; Wang, X.J.; Chi, Y.T.; Zheng, N.N.; Guo, Y.C. Curriculum classification network based on margin balancing multi-loss and ensemble learning. Future Gener. Comput. Syst. 2023, 145, 150–163. [Google Scholar] [CrossRef]
- Xiong, N.; Vasilakos, A.V.; Wu, J.; Yang, Y.R.; Rindos, A.; Zhou, Y.; Song, W.-Z.; Pan, Y. A Self-tuning Failure Detection Scheme for Cloud Computing Service. In Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, Shanghai, China, 21–25 May 2012; pp. 668–679. [Google Scholar] [CrossRef]
Data | Nodes | Edges | Time Steps |
---|---|---|---|
METR-LA | 207 | 1515 | 34,272 |
PEMS-BAY | 325 | 2369 | 52,116 |
Data | Models | 15 min | 30 min | 60 min | ||||||
---|---|---|---|---|---|---|---|---|---|---|
MAE | RMSE | MAPE | MAE | RMSE | MAPE | MAE | RMSE | MAPE | ||
METR-LA | ARIMA | 3.98 | 8.21 | 9.60% | 5.15 | 10.45 | 12.07% | 6.90 | 13.23 | 17.40% |
DCRNN | 2.77 | 5.36 | 7.28% | 3.15 | 6.45 | 8.80% | 3.60 | 8.93 | 10.50% | |
ASTGCN | 2.70 | 5.24 | 6.89% | 2.71 | 7.18 | 7.89% | 3.64 | 7.65 | 10.62% | |
STGCN | 2.88 | 5.74 | 7.62% | 3.47 | 7.24 | 9.57% | 4.59 | 9.40 | 12.70% | |
GraphWaveNet | 2.85 | 6.30 | 6.90% | 3.47 | 6.22 | 9.57% | 3.53 | 7.37 | 10.01% | |
STN-GCN | 2.69 | 5.15 | 6.89% | 3.06 | 6.16 | 8.22% | 3.51 | 7.32 | 9.75% | |
PEMS-BAY | ARIMA | 1.62 | 3.30 | 3.50% | 2.33 | 4.76 | 5.40% | 3.38 | 6.50 | 8.30% |
DCRNN | 1.63 | 2.95 | 3.01% | 1.74 | 3.97 | 3.90% | 2.07 | 4.74 | 4.90% | |
ASTGCN | 1.48 | 3.01 | 3.02% | 1.75 | 3.85 | 4.15% | 2.21 | 5.32 | 5.26% | |
STGCN | 1.36 | 2.96 | 2.90% | 1.81 | 4.27 | 4.17% | 2.49 | 5.69 | 5.79% | |
GraphWaveNet | 1.30 | 2.74 | 2.80% | 1.63 | 3.70 | 3.67% | 1.95 | 4.52 | 4.63% | |
STN-GCN | 1.31 | 2.73 | 2.74% | 1.64 | 3.70 | 3.64% | 1.92 | 4.47 | 4.54% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, C.; Wang, L.; Wei, S.; Sun, Y.; Liu, B.; Yan, L. STN-GCN: Spatial and Temporal Normalization Graph Convolutional Neural Networks for Traffic Flow Forecasting. Electronics 2023, 12, 3158. https://doi.org/10.3390/electronics12143158
Wang C, Wang L, Wei S, Sun Y, Liu B, Yan L. STN-GCN: Spatial and Temporal Normalization Graph Convolutional Neural Networks for Traffic Flow Forecasting. Electronics. 2023; 12(14):3158. https://doi.org/10.3390/electronics12143158
Chicago/Turabian StyleWang, Chunzhi, Lu Wang, Siwei Wei, Yun Sun, Bowen Liu, and Lingyu Yan. 2023. "STN-GCN: Spatial and Temporal Normalization Graph Convolutional Neural Networks for Traffic Flow Forecasting" Electronics 12, no. 14: 3158. https://doi.org/10.3390/electronics12143158
APA StyleWang, C., Wang, L., Wei, S., Sun, Y., Liu, B., & Yan, L. (2023). STN-GCN: Spatial and Temporal Normalization Graph Convolutional Neural Networks for Traffic Flow Forecasting. Electronics, 12(14), 3158. https://doi.org/10.3390/electronics12143158