A Convolution Component-Based Method with Attention Mechanism for Travel-Time Prediction
Abstract
:1. Introduction
- To directly extract features in multiple links, we substitute the global-level representation for the traditional local receive fields in the input sequences.
- We proposed the convolution component-based method with attention mechanism for travel-time prediction. The interval times are considered as the aspects of attention mechanism.
- Based on the dataset provided by Highways England, the proposed method is trained and the experimental results show that the proposed method can achieve better predictive accuracy over the baseline methods.
2. Related Work
2.1. Traffic Prediction Methods
2.2. Convolution Neural Network
2.3. Attention Mechanism
3. Methodology
3.1. Neural Network Architecture
3.2. Input Sequence
3.3. Convolution
3.4. Attention Mechanism
3.5. Output
3.6. Model Training
3.7. Dataset and Task Definition
4. Experimental Setting
4.1. Evaluation Metrics
4.2. Hyperparameters
4.3. Baseline Methods
5. Experimental Results
5.1. Exploring the Spatiotemporal Relationship
5.2. Performance with Time Variation
5.3. Overall Performance
5.4. Performance for the Individual Links
6. Conclusions and Future Work
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Moria, U.; Mendiburub, A.; Álvarezc, M.; Lozanoa, J.A. A review of travel-time estimation and forecasting for advanced traveller information systems. Transportmetrica 2015, 11, 119–157. [Google Scholar] [CrossRef]
- Van Lint, J.W.C. Online learning solutions for freeway travel-time prediction. IEEE Trans. Intell. Transp. Syst. 2008, 9, 38–47. [Google Scholar] [CrossRef]
- Liu, H.; Zuylen, H.J.V.; Lint, H.V.; Salomons, M.; Liu, H. Predicting urban arterial travel-time with state-space neural networks and kalman filters. Transp. Res. Rec. J. Transp. Res. Board 2006, 1968. [Google Scholar] [CrossRef]
- Lint, H. Reliable Travel-Time Prediction For Freeways. Ph.D. Thesis, Delft University of Technology, Delft, The Nederland, 3 May 2004. [Google Scholar]
- Ma, X.; Tao, Z.; Wang, Y.; Yu, H.; Wang, Y. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transp. Res. Part C 2015, 54, 187–197. [Google Scholar] [CrossRef]
- Lv, Y.; Duan, Y.; Kang, W.; Li, Z.; Wang, F.Y. Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intell. Transp. Syst. 2015, 16, 865–873. [Google Scholar] [CrossRef]
- Polson, N.G.; Sokolov, V.O. Deep learning for short-term traffic flow prediction. Transp. Res. Part C Emerg. Technol. 2017, 79, 1–17. [Google Scholar] [CrossRef]
- Ma, X.; Dai, Z.; He, Z.; Ma, J.; Wang, Y.; Wang, Y. Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors 2017, 17, 818. [Google Scholar] [CrossRef] [PubMed]
- Ke, J.; Zheng, H.; Yang, H.; Chen, X. Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach. Transp. Res. Part C Emerg. Technol. 2017, 85, 591–608. [Google Scholar] [CrossRef]
- Wang, J.; Gu, Q.; Wu, J.; Liu, G.; Xiong, Z. Traffic speed prediction and congestion source exploration: A deep learning method. In Proceedings of the IEEE 16th International Conference on Data Mining, Barcelona, Spain, 12–15 December 2017; pp. 499–508. [Google Scholar]
- Lecun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef]
- Ran, X.; Shan, Z.; Fang, Y.; Lin, C. Travel-time prediction by providing constraints on a convolutional neural network. IEEE Access 2018, 6. [Google Scholar] [CrossRef]
- Mnih, V.; Heess, N.; Graves, A.; Kavukcuoglu, K. Recurrent models of visual attention. Adv. Neural Inf. Process. Syst. 2014, 3, 2204–2212. [Google Scholar]
- Luong, M.T.; Pham, H.; Manning, C.D. Effective Approaches to Attention-based Neural Machine Translation. arXiv 2015, arXiv:1508.04025. [Google Scholar]
- Chen, J.; Zhang, H.; He, X.; Liu, W.; Liu, W.; Chua, T.S. Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan, 7–11 August 2017; pp. 335–344. [Google Scholar]
- Ahmed, M.S.; Cook, A.R. Analysis of Freeway Traffic Time-Series Data by Using Box-Jenkins Techniques. Transp. Res. Rec. J. Transp. Res. Board 1979, 773, 1–9. [Google Scholar]
- Hamed, M.M.; Al-Masaeid, H.R.; Said, Z.M.B. Short-term prediction of traffic volume in urban arterials. J. Transp. Eng. 1995, 121, 249–254. [Google Scholar] [CrossRef]
- Williams, B.M.; Hoel, L.A. Modeling and forecasting vehicular traffic flow as a seasonal arima process: Theoretical basis and empirical results. J. Transp. Eng. 2003, 129, 664–672. [Google Scholar] [CrossRef]
- Kumar, S.V. Traffic Flow Prediction using Kalman Filtering. Procedia Eng. 2017, 187, 582–587. [Google Scholar] [CrossRef]
- Rice, J.; Van Zwet, E. A simple and effective method for predicting travel times on freeways. IEEE Trans. Intell. Transp. Syst. 2004, 5, 200–207. [Google Scholar] [CrossRef]
- Davis, G.A.; Nihan, N.L. Nonparametric regression and short-term freeway traffic forecasting. J. Transp. Eng. 1991, 117, 178–188. [Google Scholar] [CrossRef]
- Smith, B.L.; Williams, B.M.; Oswald, R.K. Comparison of parametric and nonparametric models for traffic flow forecasting. Transp. Res. Part C Emerg. Technol. 2002, 10, 303–321. [Google Scholar] [CrossRef]
- Chang, H.; Lee, Y.; Yoon, B.; Baek, S. Dynamic near-term traffic flow prediction: Systemoriented approach based on past experiences. IET Intell. Transp. Syst. 2012, 6, 292–305. [Google Scholar] [CrossRef]
- Drucker, H.; Burges, C.J.C.; Kaufman, L.; Smola, A.J.; Vapnik, V. Support vector regression machines. Adv. Neural Inf. Process. Syst. 1996, 28, 779–784. [Google Scholar]
- Wu, C.H.; Ho, J.M.; Lee, D.T. Travel-time prediction with support vector regression. IEEE Trans. Intell. Transp. Syst. 2004, 5, 276–281. [Google Scholar] [CrossRef]
- Castro-Neto, M.; Jeong, Y.S.; Jeong, M.K.; Han, L.D. Online-svr for short-term traffic flow prediction under typical and atypical traffic conditions. Expert Syst. Appl. Int. J. 2009, 36, 6164–6173. [Google Scholar] [CrossRef]
- Su, H.; Zhang, L.; Yu, S. Short-term traffic flow prediction based on incremental support vector regression. In Proceedings of the 3rd International Conference on Natural Computation, Haikou, China, 24–27 August 2007; pp. 640–645. [Google Scholar]
- Lint, H.V.; Hoogendoorn, S.P.; Zuylen, H.J.V. State space neural networks for freeway travel-time prediction. In Proceedings of the International Conference on Artificial Neural Networks (ICANN 2002), Madrid, Spain, 28–30 August 2002; pp. 1043–1048. [Google Scholar]
- Elman, J. Finding structure in time. Cogn. Sci. 1990, 14, 179–211. [Google Scholar] [CrossRef]
- Van Lint, H.; Van Hinsbergen, C. Short-term traffic and travel-time prediction models. Transp. Res. Circ. 2012, E-C168, 22–41. [Google Scholar]
- Zeng, X.; Zhang, Y. Development of recurrent neural network considering temporal-spatial input dynamics for freeway travel-time modeling. Comput.-Aided Civ. Infrastruct. Eng. 2013, 28, 359–371. [Google Scholar] [CrossRef]
- Zeng, D.; Liu, K.; Chen, Y.; Zhao, J. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 17–21 September 2015; pp. 1753–1762. [Google Scholar]
- Shi, X.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.C. Convolutional lstm network: A machine learning approach for precipitation nowcasting. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 7–12 December 2015; Volume 9199, pp. 802–810. [Google Scholar]
- Lecun, Y.; Boser, B.; Denker, J.S.; Henderson, D.; Howard, R.E.; Hubbard, W.; Jackel, L.D. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1989, 1, 541–551. [Google Scholar] [CrossRef]
- Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the 15th European Conference on Computer Vision, Munich, Germany, 8–14 September 2018. [Google Scholar]
- Wirth, R.; Hipp, J. CRISP-DM: Towards a standard process model for data mining. In Proceedings of the 4th International Conference on the Practical Application of Knowledge Discovery and Data Mining, Manchester, UK, 11–13 April 2000. [Google Scholar]
- Dean, J.; Corrado, G.S.; Monga, R.; Chen, K.; Devin, M.; Le, Q.V.; Mao, M.Z.; Ranzato, M.; Senior, A.; Tucker, P.; et al. Large scale distributed deep networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1223–1231. [Google Scholar]
- Vanhoucke, V.; Mao, M.Z. Improving the speed of neural networks on CPUs. Deep Learning & Unsupervised Feature Learning Workshop Nips. Available online: http://www.andrewsenior.com/papers/VanhouckeNIPS11.pdf (accessed on 1 May 2019).
- Highways England. Highways Agency Network Journey Time and Traffic Flow Data; Highways England: Guildford, UK, 2018. [Google Scholar]
- Fortmannroe, S. Understanding the Bias-Variance Tradeoff. Available online: http://scott.fortmann-roe.com/docs/BiasVariance.html (accessed on 1 May 2019).
Methods | Advantages and Disadvantages |
---|---|
SSNN | Use shared internal state (memory) to process sequences of inputs and exists the issue of back-propagated error decay through memory blocks [4]. |
LSTM NN | Can overcome the issue of back-propagated error decays through memory blocks [5]. |
AES | Spatial and temporal correlations are inherently considered [6]. |
CNN | Temporal correlations are inherently considered [12,32,33]. Based on the topology of the input data, three architectural ideas are proposed including local receptive fields, shared weights and spatial or temporal subsampling [34]. |
LinkRef | Date | TimePeriod (0–95) | AverageJT | LinkLength (km) |
---|---|---|---|---|
AL282 | 2015-03-01 | 0 | 642.56 | 15.64 |
AL282 | 2015-03-01 | 1 | 603.71 | 15.64 |
… | … | … | … | … |
AL292 | 2015-03-01 | 0 | 126.03 | 3.92 |
AL292 | 2015-03-01 | 1 | 130.52 | 3.92 |
… | … | … | … | … |
AL2274 | 2015-03-01 | 0 | 337.40 | 9.72 |
AL2274 | 2015-03-01 | 1 | 337.40 | 9.72 |
… | … | … | … | … |
AL286 | 2015-03-01 | 0 | 271.83 | 7.98 |
AL286 | 2015-03-01 | 1 | 272.69 | 7.98 |
… | … | … | … | … |
Hyperparameter | l | ||
---|---|---|---|
Value | 3 | 30 | 0.1 |
Models | 15-min on Training Set | 15-min on Test Set | Variance | ||||||
---|---|---|---|---|---|---|---|---|---|
RMSE | MAE | MAPE | RMSE | MAE | MAPE | ||||
k-NNR | uniform | 54.544 | 5.745 | 5.90 | 53.361 | 5.911 | 6.55 | 0.65 | |
distance | 33.512 | 4.308 | 3.34 | 53.361 | 5.911 | 6.55 | 3.21 | ||
SARIMA | 193.554 | 13.120 | 30.03 | 55.820 | 6.658 | 8.53 | 21.5 | ||
SVR | linear | 54.612 | 6.040 | 6.71 | 49.689 | 6.046 | 7.05 | 0.34 | |
rbf | 57.077 | 6.441 | 7.69 | 51.386 | 6.310 | 7.74 | 0.05 | ||
poly | 58.136 | 6.894 | 9.29 | 66.506 | 7.135 | 9.47 | 0.18 | ||
LR | 54.853 | 6.195 | 6.04 | 52.512 | 5.786 | 7.08 | 1.04 | ||
SSNN | 52.901 | 5.706 | 5.81 | 47.849 | 5.794 | 6.26 | 0.45 | ||
LSTM NN | 52.270 | 5.728 | 5.89 | 48.196 | 5.811 | 6.30 | 0.41 | ||
AEs | 60.041 | 6.117 | 6.70 | 48.326 | 6.028 | 6.81 | 0.11 | ||
CNN1 | 64.040 | 7.081 | 9.06 | 95.915 | 7.653 | 9.82 | 0.76 | ||
CNN2 | 51.772 | 5.775 | 6.02 | 53.829 | 5.804 | 6.32 | 0.30 | ||
The Proposed Method | 49.794 | 5.554 | 5.79 | 52.644 | 5.675 | 6.20 | 0.45 |
Models | 15-min on Training Set | 15-min on Test Set | Variance | ||||||
---|---|---|---|---|---|---|---|---|---|
RMSE | MAE | MAPE | RMSE | MAE | MAPE | ||||
k-NNR | uniform | 20.173 | 3.172 | 1.88 | 42.941 | 5.307 | 5.45 | 3.57 | |
distance | 44.663 | 5.235 | 5.14 | 42.941 | 5.307 | 5.45 | 0.31 | ||
SARIMA | 162.669 | 12.242 | 26.63 | 52.826 | 6.346 | 8.02 | 18.61 | ||
SVR | linear | 41.994 | 5.346 | 5.26 | 40.253 | 5.349 | 5.70 | 0.44 | |
rbf | 45.094 | 5.789 | 6.22 | 45.056 | 5.833 | 6.85 | 0.63 | ||
poly | 48.578 | 6.071 | 6.84 | 49.200 | 6.128 | 7.59 | 0.75 | ||
LR | 38.789 | 5.219 | 5.30 | 48.141 | 5.746 | 6.05 | 0.75 | ||
SSNN | 41.290 | 5.281 | 4.98 | 38.737 | 5.182 | 5.21 | 0.23 | ||
LSTM NN | 41.097 | 5.196 | 4.90 | 37.845 | 5.112 | 5.14 | 0.24 | ||
AEs | 47.404 | 5.563 | 5.58 | 41.638 | 5.615 | 6.13 | 0.55 | ||
CNN1 | 51.0347 | 5.90 | 6.29 | 45.772 | 5.968 | 6.87 | 0.58 | ||
CNN2 | 40.367 | 5.166 | 4.87 | 42.34 | 5.154 | 5.16 | 0.29 | ||
The Proposed Method | 39.278 | 5.081 | 4.79 | 41.19 | 5.024 | 5.02 | 0.23 |
LinkID | Kalman Filter | LR | CNN2 | LSTM | AES | CNN 1 | SSNN | The Proposed Method |
---|---|---|---|---|---|---|---|---|
AL344 | 13.89 | 5.83 | 6.09 | 5.68 | 10.98 | 18.42 | 6.03 | 5.46 |
AL3276 | 10.87 | 5.69 | 5.63 | 5.37 | 6.93 | 13.27 | 5.53 | 5.17 |
AL444 | 8.48 | 6.02 | 5.98 | 6.16 | 8.38 | 27.28 | 6.49 | 5.97 |
AL2878 | 16.03 | 10.34 | 7.96 | 8.41 | 14.1 | 12.83 | 8.93 | 7.66 |
AL2869 | 13.31 | 6.85 | 6.89 | 7.56 | 12.7 | 24.24 | 7.88 | 6.84 |
AL2871 | 13.39 | 8.58 | 6.64 | 6.79 | 10.64 | 11.3 | 7.48 | 6.03 |
AL2861 | 10.72 | 15.83 | 7.82 | 7.61 | 10.66 | 14.41 | 7.91 | 7.46 |
AL2853 | 8.43 | 5.08 | 4.73 | 5.32 | 7.3 | 15.03 | 5.17 | 4.56 |
AL2850 | 8.06 | 6.12 | 6.46 | 6.42 | 8.83 | 9.95 | 6.54 | 6.13 |
AL2852B | 5.31 | 2.02 | 1.82 | 1.8 | 2.11 | 4.78 | 1.94 | 1.75 |
AL2852A | 6.41 | 5.44 | 5.32 | 5.38 | 5.78 | 9.28 | 5.41 | 5.27 |
AL286 | 5.17 | 1.29 | 1.25 | 1.26 | 1.92 | 5.05 | 1.52 | 1.21 |
AL292 | 4.56 | 1.42 | 1.29 | 2.43 | 1.93 | 4.82 | 2.24 | 1.26 |
AL283 | 5.09 | 2.81 | 1.94 | 1.87 | 2.59 | 4.78 | 1.96 | 1.9 |
AL278 | 13.34 | 11.08 | 10.86 | 11.82 | 13.04 | 15.16 | 12.12 | 10.81 |
AL270 | 5.07 | 2.75 | 1.83 | 1.75 | 2.4 | 5.07 | 2.02 | 1.78 |
AL265 | 9.95 | 6.37 | 6.11 | 6.24 | 8.52 | 10.17 | 6.37 | 5.9 |
AL261A | 10.05 | 6.88 | 6.77 | 6.61 | 8.87 | 10.86 | 6.58 | 6.54 |
AL258A | 6.64 | 5.33 | 5.72 | 5.67 | 6.76 | 8.63 | 5.84 | 5.46 |
AL256 | 9.85 | 9.56 | 7.15 | 7.0 | 9.29 | 9.66 | 6.92 | 6.92 |
AL2282 | 8.05 | 6.27 | 6.22 | 6.1 | 7.77 | 9.54 | 6.37 | 6.17 |
AL248 | 8.44 | 5.39 | 5.65 | 5.64 | 7.11 | 9.31 | 5.6 | 5.56 |
AL241 | 8.89 | 6.41 | 6.19 | 6.08 | 6.87 | 7.84 | 6.55 | 5.96 |
AL242 | 7.59 | 9.35 | 6.95 | 6.84 | 7.42 | 10.14 | 6.95 | 6.77 |
AL236 | 8.57 | 7.66 | 7.16 | 7.36 | 8.1 | 10.71 | 7.74 | 7.05 |
AL2292 | 9.36 | 7.98 | 7.71 | 7.92 | 9.35 | 8.54 | 7.75 | 7.57 |
AL2295 | 7.30 | 5.46 | 5.1 | 5.38 | 6.37 | 6.83 | 5.31 | 4.65 |
AL237 | 6.84 | 6.29 | 6.32 | 6.41 | 7.1 | 6.84 | 6.6 | 6.08 |
AL243 | 7.94 | 10.4 | 5.55 | 6.26 | 6.68 | 12.46 | 5.77 | 5.5 |
AL240 | 7.83 | 6.74 | 6.09 | 6.03 | 6.86 | 8.78 | 6.49 | 5.97 |
AL254 | 9.85 | 8.53 | 7.72 | 7.57 | 9.34 | 9.93 | 7.93 | 7.48 |
AL257A | 7.07 | 6.5 | 6.2 | 6.08 | 7.16 | 7.63 | 6.52 | 6.08 |
AL267 | 7.50 | 6.42 | 5.02 | 5.36 | 6.73 | 7.55 | 5.15 | 4.96 |
AL272 | 3.95 | 1.95 | 1.41 | 1.72 | 1.72 | 3.33 | 1.64 | 1.33 |
AL279 | 19.26 | 16.75 | 14.89 | 17.12 | 19.79 | 26.38 | 17.0 | 14.04 |
AL284 | 5.88 | 6.29 | 6.17 | 6.1 | 6.92 | 8.86 | 6.54 | 6.06 |
AL291 | 4.86 | 2. | 1.26 | 1.29 | 2.01 | 3.9 | 1.49 | 1.20 |
AL288 | 6.24 | 2.43 | 1.71 | 1.74 | 3.01 | 5.02 | 1.83 | 1.64 |
AL2851A | 6.98 | 5.83 | 5.8 | 5.69 | 6.08 | 6.59 | 5.81 | 5.76 |
AL2851B | 9.49 | 6.11 | 5.93 | 6.25 | 8.41 | 10.55 | 6.56 | 5.88 |
AL2849 | 8.75 | 8.6 | 5.62 | 6.2 | 7.78 | 12.16 | 5.9 | 5.38 |
AL298 | 10.56 | 6.58 | 6.07 | 5.64 | 8.13 | 28.13 | 5.73 | 5.39 |
AL2862 | 7.27 | 6.43 | 6.19 | 6.14 | 7.24 | 7.75 | 6.84 | 5.7 |
AL343 | 11.26 | 9.21 | 5.2 | 6.44 | 11.15 | 9.31 | 6.11 | 5.03 |
AL2872 | 5.42 | 6.68 | 4.88 | 5.02 | 6.06 | 9.05 | 5.66 | 4.72 |
AL2870 | 6.58 | 5.9 | 5.82 | 5.95 | 7.05 | 9.96 | 6.17 | 5.75 |
AL2879 | 14.13 | 7.12 | 6.76 | 6.62 | 10.72 | 26.1 | 6.87 | 6.73 |
Mean | 8.73 | 6.61 | 5.70 | 5.88 | 7.59 | 11.03 | 6.038 | 5.50 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ran, X.; Shan, Z.; Fang, Y.; Lin, C. A Convolution Component-Based Method with Attention Mechanism for Travel-Time Prediction. Sensors 2019, 19, 2063. https://doi.org/10.3390/s19092063
Ran X, Shan Z, Fang Y, Lin C. A Convolution Component-Based Method with Attention Mechanism for Travel-Time Prediction. Sensors. 2019; 19(9):2063. https://doi.org/10.3390/s19092063
Chicago/Turabian StyleRan, Xiangdong, Zhiguang Shan, Yufei Fang, and Chuang Lin. 2019. "A Convolution Component-Based Method with Attention Mechanism for Travel-Time Prediction" Sensors 19, no. 9: 2063. https://doi.org/10.3390/s19092063
APA StyleRan, X., Shan, Z., Fang, Y., & Lin, C. (2019). A Convolution Component-Based Method with Attention Mechanism for Travel-Time Prediction. Sensors, 19(9), 2063. https://doi.org/10.3390/s19092063