Recursive Least Squares Based Refinement Network for Vehicle Trajectory Prediction
Abstract
:1. Introduction
- We propose an RLS-based framework with generated anchors that captures both local and global motion patterns of surrounding vehicles. This method manages to minimize the negative impact of error accumulation.
- Instead of artificial parameters, we propose a data-driven RLS method that allows the covariance matrix of RLS to be automatically updated by a learning-based method. Our prediction network based on the proposed RLS model can be more accurate and robust in various traffic scenes.
- Our model can be easily embedded into most rollout approaches. An ablation study has proved the effectiveness of our module which can be easily plugged into various networks to improve their performance.
2. Related Works
2.1. Deep Learning in Trajectory Prediction
2.2. Seq2seq Trajectory Prediction
2.3. Goal-Conditioned Methods
3. Recursive Least-Squares Based Refinement Network
3.1. Problem Definition
3.2. Anchor Generator
3.3. Trajectory Encoder
3.4. Trajectory Decoder
3.5. Recursive Least-Squares Module
4. Experiments
4.1. Quantitative Results and Analysis
4.1.1. Baselines
- S-LSTM [23]: An influential method based on a social pooling model.
- CS-LSTM [24]: A model using a convolutional social pooling structure to learn vehicle interactions.
- MHA-LSTM [5]: A model based on a multi-head attention mechanism for trajectory prediction.
- MATF GAN [47]: A model using convolutional neural network and generative adversarial network to learn social interaction.
- GRIP [48]: A model using graph networks to simulate interactions between multiple agents.
- MFP [22]: A state-of-the-art model which learns semantically latent variables for trajectory prediction.
4.1.2. Quantitative Evaluation
4.1.3. Ablation Study
4.2. Qualitative Results and Analysis
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Dixit, A.; Kumar, C.R.; Allam, Z. Safety and Risk Analysis of Autonomous Vehicles Using Computer Vision and Neural Networks. Vehicles 2021, 3, 595–617. [Google Scholar] [CrossRef]
- Lefèvre, S.; Vasquez, D.; Laugier, C. A survey on motion prediction and risk assessment for intelligent vehicles. Robomech J. 2014, 1, 1. [Google Scholar] [CrossRef]
- Paden, B.; Čáp, M.; Yong, S.Z.; Yershov, D.; Frazzoli, E. A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles. IEEE Trans. Intell. Veh. 2016, 1, 33–35. [Google Scholar] [CrossRef]
- Schmidhube, J. Deep learning in neural networks: An overview. Neural Netw. 2015, 61, 85–117. [Google Scholar] [CrossRef] [PubMed]
- Messaoud, K.; Yahiaoui, I.; Verroust-Blondet, A.; Nashashibi, F. Attention Based Vehicle Trajectory Prediction. IEEE Trans. Intell. Veh. 2021, 6, 175–185. [Google Scholar] [CrossRef]
- Kingma, D.P.; Welling, M. Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Representations, Banff, AB, Canada, 14–16 April 2013. [Google Scholar]
- Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. In Proceedings of the Neural Information Processing Systems, Montréal, QC, Canada, 8–11 December 2014. [Google Scholar]
- Gao, J.; Sun, C.; Zhao, H.; Shen, Y.; Anguelov, D.; Li, C.; Schmid, C. VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 11522–11530. [Google Scholar]
- Cho, K.; Ha, T.; Lee, G.; Oh, S. Deep Predictive Autonomous Driving Using Multi-Agent Joint Trajectory Prediction and Traffic Rules. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, The Venetian Macao, Macau, China, 3–8 November 2019; pp. 2076–2081. [Google Scholar]
- Graves, A. Generating Sequences With Recurrent Neural Networks. arXiv 2013, arXiv:1308.0850. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long short-term memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
- Jo, E.; Sunwoo, M.; Lee, M. Vehicle Trajectory Prediction Using Hierarchical Graph Neural Network for Considering Interaction among Multimodal Maneuvers. Sensors 2021, 21, 5354. [Google Scholar] [CrossRef] [PubMed]
- Ju, C.; Wang, Z.; Long, C.; Zhang, X.; Chang, D.E. Interaction-aware Kalman Neural Networks for Trajectory Prediction. In Proceedings of the IEEE Intelligent Vehicles Symposium, Las Vegas, NV, USA, 19 October–13 November 2020; pp. 1793–1800. [Google Scholar]
- Mercat, J.; Gilles, T.; El Zoghby, N.; Sandou, G.; Beauvois, D.; Gil, G.P. Mul-ti-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting. In Proceedings of the IEEE International Conference on Robotics and Automation, Paris, France, 31 May–1 August 2020; pp. 9638–9644. [Google Scholar]
- Vahidi, A.; Stefanopoulou, A.; Peng, H. Recursive least squares with forgetting for online estimation of vehicle mass and road grade: Theory and experiments. Veh. Syst. Dyn. 2005, 43, 31–35. [Google Scholar] [CrossRef]
- Punzo, V.; Borzacchiello, M.T.; Ciuffo, B. On the assessment of vehicle trajectory data accuracy and application to the Next Generation SIMulation (NGSIM) program data. Transp. Res. Part Emerg. Technol. 2011, 19, 1243–1262. [Google Scholar] [CrossRef]
- Zhang, W.; Sun, L.; Wang, D.; Shi, H.; Clausse, A.; Naumann, M.; Kümmerle, J.; Königshof, H.; Stiller, C.; de La Fortelle, A.; et al. INTERACTION Dataset: An International, Adversarial and Cooperative Motion Dataset in Interactive Driving Scenarios with Semantic Maps. arXiv 2019, arXiv:1910.03088. [Google Scholar]
- Cui, H.; Radosavljevic, V.; Chou, F.; Lin, T.; Nguyen, T.; Huang, T.; Schneider, J.G.; Djuric, N. Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks. In Proceedings of the International Conference on Robotics and Automation, Montreal, QC, Canada, 20–24 May 2019; pp. 2090–2096. [Google Scholar]
- Sun, L.; Zhan, W.; Tomizuka, M. Probabilistic Prediction of Interactive Driving Behavior via Hierarchical Inverse Reinforcement Learning. In Proceedings of the International Conference on Intelligent Transportation Systems, Maui, HI, USA, 4–7 November 2018; pp. 2111–2117. [Google Scholar]
- Rudenko, A.; Palmieri, L.; Herman, L.; Kitani, K.M.; Gavrila, D.M.; Arras, K.O. Human motion trajectory prediction: A survey. Int. J. Robot. Res. 2020, 39, 895–935. [Google Scholar] [CrossRef]
- Lee, N.; Choi, W.; Vernaza, P.; Choy, C.B.; Torr, P.H.; Chandraker, M. Desire: Distant future prediction in dynamic scenes with interacting agents. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 336–345. [Google Scholar]
- Tang, Y.C.; Salakhutdinov, R. Multiple futures prediction. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019; pp. 15424–15434. [Google Scholar]
- Alahi, A.; Goel, K.; Ramanathan, V.; Robicquet, A.; Li, F.F.; Savarese, S. Social LSTM: Human Trajectory Prediction in Crowd-ed Spaces. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 961–971. [Google Scholar]
- Deo, N.; Trivedi, M.M. Convolutional Social Pooling for Vehicle Trajectory Prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 18–23 June 2018; pp. 1549–1557. [Google Scholar]
- Gupta, A.; Johnson, J.; Li, F.F.; Savarese, S.; Alahi, A. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 2255–2264. [Google Scholar]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to sequence learning with neural networks. In Proceedings of the International Conference on Neural Information Processing Systems, Montréal, QC, Canada, 8–13 December 2014; pp. 3104–3112. [Google Scholar]
- Chung, J.; Gulcehre, C.; Cho, K.; Bengio, Y. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. In Proceedings of the Advances in Neural Information Processing Systems Workshop on Deep Learning, Montréal, QC, Canada, 8–13 December 2014. [Google Scholar]
- Ondruska, P.; Posner, I. Deep tracking: Seeing beyond seeing using recurrent neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA, 12–17 February 2016; pp. 3361–3367. [Google Scholar]
- Khosroshahi, A.; Ohn-Bar, E.; Trivedi, M.M. Surround vehicles trajectory analysis with recurrent neural networks. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems, Rio de Janeiro, Brazil, 1–4 November 2016; pp. 2267–2272. [Google Scholar]
- Kim, B.; Kang, C.M.; Lee, S.H.; Chae, H.; Kim, J.; Chung, C.C.; Choi, J.W. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems, Yokohama, Japan, 16–19 October 2017; pp. 399–404. [Google Scholar]
- Park, S.H.; Kim, B.; Kang, C.M.; Chung, C.C.; Choi, J.W. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture. In Proceedings of the IEEE Intelligent Vehicles Symposium, Changshu, China, 26–30 June 2018; pp. 1672–1678. [Google Scholar]
- Du, S.; Li, T.; Yang, Y.; Horng, S.J. Multivariate time series forecasting via attention-based encoder–decoder framework. Neural Comput. 2020, 388, 269–279. [Google Scholar] [CrossRef]
- Fang, L.; Jiang, Q.; Shi, J.; Zhou, B. Tpnet: Trajectory proposal network for motion prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 13–19 June 2020; pp. 6797–6806. [Google Scholar]
- Mangalam, K.; Girase, H.; Agarwal, S.; Lee, K.H.; Adeli, E.; Malik, J.; Gaidon, A. It Is Not the Journey However, the Destination: Endpoint Conditioned Trajectory Prediction. In Proceedings of the IEEE European Conference on Computer Vision, online-only, 23–28 August 2020; pp. 759–776. [Google Scholar]
- Dendorfer, P.; Osep, A.; Leal-Taixé, L. Goal-gan: Multimodal trajectory prediction based on goal position estimation. In Proceedings of the Asian Conference on Computer Vision, Kyoto, Japan, 30 November–4 January 2020; pp. 405–420. [Google Scholar]
- Zhao, H.; Wildes, R.P. Where are you heading? Dynamic Trajectory Prediction with Expert Goal Examples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 7629–7638. [Google Scholar]
- Zhao, H.; Gao, J.; Lan, T.; Sun, C.; Sapp, B.; Varadarajan, B.; Shen, Y.; Shen, Y.; Chai, Y.; Schmid, C.; et al. TNT: Target-driven Trajectory Prediction. In Proceedings of the Conference on Robot Learning, London, UK, 8–11 November 2021; pp. 895–904. [Google Scholar]
- Gu, J.; Sun, C.; Zhao, H. Densetnt: End-to-end trajectory prediction from dense goal sets. In Proceedings of the IEEE International Conference on Computer Vision, Montréal, QC, Canada, 10–17 October 2021; pp. 15303–15312. [Google Scholar]
- Casas, S.; Luo, W.; Urtasun, R. Intentnet: Learning to predict intention from raw sensor data. In Proceedings of the Conference on Robot Learning, Zurich, Switzerland, 29–31 October 2018; pp. 947–956. [Google Scholar]
- Feng, X.; Cen, Z.; Hu, J.; Zhang, Y. Vehicle trajectory prediction using intention-based conditional variational autoencoder. In Proceedings of the IEEE Intelligent Transportation Systems Conference, Auckland, New Zealand, 27–29 October 2019; pp. 3514–3519. [Google Scholar]
- Li, L.; Yao, J.; Wenliang, L.; He, T.; Xiao, T.; Yan, J.; Wipf, D.; Zhang, Z. GRIN: Generative Relation and Intention Network for Multi-agent Trajectory Prediction. In Proceedings of the Advances in Neural Information Processing Systems, online-only, 7–10 December 2021. [Google Scholar]
- Chai, Y.; Sapp, B.; Bansal, M.; Anguelov, D. MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction. In Proceedings of the Conference on Robot Learning, Osaka, Japan, 30 October–1 November 2019. [Google Scholar]
- Phan-Minh, T.; Grigore, E.C.; Boulton, F.A.; Beijbom, O.; Wolff, E.M. Covernet: Multimodal behavior prediction using trajectory sets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 14074–14083. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- Ma, X.; Karkus, P.; Hsu, D.; Lee, W.S. Particle Filter Recurrent Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 27 January–1 February 2019; pp. 5101–5108. [Google Scholar]
- Engel, Y.; Mannor, S.; Meir, R. The kernel recursive least-squares algorithm. IEEE Trans. Signal Process. 2004, 52, 2275–2285. [Google Scholar] [CrossRef]
- Zhao, T.; Xu, Y.; Monfort, M.; Choi, W.; Baker, C.; Zhao, Y.; Wang, Y.; Wu, Y.N. Multi-agent tensor fusion for contextual trajectory prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 12118–12126. [Google Scholar]
- Li, X.; Ying, X.; Chuah, M.C. GRIP: Graph-based Interaction-aware Trajectory Prediction. In Proceedings of the IEEE Intelligent Transportation Systems Conference, Auckland, New Zealand, 27–30 October 2019; pp. 3960–3966. [Google Scholar]
- Kalman, R.E. A New Approach to Linear Filtering and Prediction Problems. J. Basic Eng. 1960, 82, 35. [Google Scholar] [CrossRef]
- Lei, X.; Dali, D.; Hongpeng, Z.; Jianpu, W.; Zhuoran, Z. UCAV maneuvering trajectory prediction based on PSO-CNN. In Proceedings of the 2021 International Conference on Computer Engineering and Application, Guangzhou, China, 25–27 June 2021; pp. 54–58. [Google Scholar]
Time | S-LSTM | CS-LSTM | MHA-LSTM | MATF GAN | GRIP | MFP | RRN |
---|---|---|---|---|---|---|---|
1 s | 0.65 | 0.64 | 0.56 | 0.66 | 0.64 | 0.52 | 0.49 |
2 s | 1.31 | 1.27 | 1.22 | 1.34 | 1.13 | 1.11 | 1.09 |
3 s | 2.16 | 2.09 | 2.01 | 2.08 | 1.80 | 1.79 | 1.76 |
4 s | 3.25 | 3.10 | 3.00 | 2.97 | 2.62 | 2.59 | 2.54 |
5 s | 4.55 | 4.37 | 4.25 | 4.13 | 3.60 | 3.53 | 3.44 |
Time | S-LSTM | CS-LSTM | MHA-LSTM | MATF GAN | GRIP | MFP | RRN |
---|---|---|---|---|---|---|---|
1 s | 0.33 | 0.16 | 0.16 | 0.16 | 0.13 | 0.12 | 0.11 |
2 s | 0.76 | 0.72 | 0.69 | 0.67 | 0.56 | 0.55 | 0.52 |
3 s | 1.77 | 1.77 | 1.65 | 1.60 | 1.34 | 1.31 | 1.31 |
4 s | 3.42 | 3.22 | 3.96 | 3.72 | 2.45 | 2.42 | 2.38 |
5 s | 5.46 | 4.96 | 4.61 | 4.42 | 3.86 | 3.84 | 3.69 |
Time | Vanilla LSTM | Vanilla LSTM (+RLS) | Attention LSTM | Attention LSTM (+RLS) |
---|---|---|---|---|
1s | 0.65 | 0.64 | 0.56 | 0.56 |
2s | 1.58 | 1.56 | 1.21 | 1.20 |
3s | 2.77 | 2.78 | 1.96 | 1.94 |
4s | 4.26 | 4.27 | 2.86 | 2.78 |
5s | 6.11 | 6.01 | 4.01 | 3.89 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, S.; Xue, Q.; Shi, D.; Li, X.; Zhang, W. Recursive Least Squares Based Refinement Network for Vehicle Trajectory Prediction. Electronics 2022, 11, 1859. https://doi.org/10.3390/electronics11121859
Li S, Xue Q, Shi D, Li X, Zhang W. Recursive Least Squares Based Refinement Network for Vehicle Trajectory Prediction. Electronics. 2022; 11(12):1859. https://doi.org/10.3390/electronics11121859
Chicago/Turabian StyleLi, Shengyi, Qifan Xue, Dongfeng Shi, Xuanpeng Li, and Weigong Zhang. 2022. "Recursive Least Squares Based Refinement Network for Vehicle Trajectory Prediction" Electronics 11, no. 12: 1859. https://doi.org/10.3390/electronics11121859
APA StyleLi, S., Xue, Q., Shi, D., Li, X., & Zhang, W. (2022). Recursive Least Squares Based Refinement Network for Vehicle Trajectory Prediction. Electronics, 11(12), 1859. https://doi.org/10.3390/electronics11121859