A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles
Abstract
:1. Introduction
- A reinforcement-learning-based system is proposed in this paper to learn the driver behavior and realize the human-like control. Based on RL, the system dynamics are not required and can be learned directly from the interaction between drivers and the driving environment.
- By incorporating the controller into the learning system, the learned driving behavior can be converted to control commands for autonomous vehicles online, which realizes the personalized adaption for newly-involved drivers.
2. Proposed Personalized Behavior Learning System
2.1. Formulation of the Learning Module
2.2. Function Approximation Using ANN
2.3. Speed Control Module
3. Training Algorithm for PBLS
Algorithm 1: Pseudo-code for PBLS |
Initialization
|
4. Experiments with Constant Speed
4.1. Experimental Settings
4.2. Experimental Results
5. Experiments with Variant Speed
5.1. Driving Scene I
5.2. Driving Scene II
5.3. Driving Scene III
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Lefevre, S.; Carvalho, A.; Borrelli, F. A Learning-Based Framework for Velocity Control in Autonomous Driving. IEEE Trans. Autom. Sci. Eng. 2016, 13, 32–42. [Google Scholar] [CrossRef]
- Kocić, J.; Jovičić, N.; Drndarević, V.J.S. An End-to-End Deep Neural Network for Autonomous Driving Designed for Embedded Automotive Platforms. Sensors 2019, 19, 2064. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Y.; Chen, H.; Waslander, S.; Yang, T.; Zhang, S.; Xiong, G.; Liu, K.J.S. Toward a more complete, flexible, and safer speed planning for autonomous driving via convex optimization. Sensors 2018, 18, 2185. [Google Scholar] [CrossRef] [PubMed]
- González, D.; Pérez, J.; Milanés, V.; Nashashibi, F. A review of motion planning techniques for automated vehicles. IEEE Trans. Intell. Transp. Syst. 2016, 17, 1135–1145. [Google Scholar] [CrossRef]
- Naranjo, J.E.; Gonzalez, C.; Garcia, R.; De Pedro, T. Lane-change fuzzy control in autonomous vehicles for the overtaking maneuver. IEEE Trans. Intell. Transp. Syst. 2008, 9, 438–450. [Google Scholar] [CrossRef]
- Li, A.; Jiang, H.; Zhou, J.; Zhou, X. Learning Human-Like Trajectory Planning on Urban Two-Lane Curved Roads From Experienced Drivers. IEEE Access 2019, 7, 65828–65838. [Google Scholar] [CrossRef]
- Li, L.; Ota, K.; Dong, M. Humanlike Driving: Empirical Decision-Making System for Autonomous Vehicles. IEEE Trans. Veh. Technol. 2018, 67, 6814–6823. [Google Scholar] [CrossRef] [Green Version]
- Lu, C.; Hu, F.; Cao, D.; Gong, J.; Xing, Y.; Li, Z. Transfer Learning for Driver Model Adaptation in Lane-Changing Scenarios Using Manifold Alignment. IEEE Trans. Intell. Transp. Syst. 2019, 1–13. [Google Scholar] [CrossRef]
- Li, T.-H.S.; Chang, S.-J.; Chen, Y.-X. Implementation of human-like driving skills by autonomous fuzzy behavior control on an FPGA-based car-like mobile robot. Ind. Electron. IEEE Trans. 2003, 50, 867–880. [Google Scholar] [CrossRef]
- Okuda, H.; Ikami, N.; Suzuki, T.; Tazaki, Y.; Takeda, K. Modeling and Analysis of Driving Behavior Based on a Probability-Weighted ARX Model. IEEE Trans. Intell. Transp. Syst. 2013, 14, 98–112. [Google Scholar] [CrossRef]
- Lin, T.; Tseng, E.; Borrelli, F. Modeling driver behavior during complex maneuvers. In Proceedings of the American Control Conference (ACC), Washington, DC, USA, 17–19 June 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 6448–6453. [Google Scholar]
- Argall, B.D.; Chernova, S.; Veloso, M.; Browning, B. A survey of robot learning from demonstration. Robot. Auton. Syst. 2009, 57, 469–483. [Google Scholar] [CrossRef]
- Yin, X.; Chen, Q. Trajectory Generation With Spatio-Temporal Templates Learned from Demonstrations. IEEE Trans. Ind. Electron. 2017, 64, 3442–3451. [Google Scholar] [CrossRef]
- Zheng, J.; Suzuki, K.; Fujita, M. Car-following behavior with instantaneous driver–vehicle reaction delay: A neural-network-based methodology. Transp. Res. Part C Emerg. Technol. 2013, 36, 339–351. [Google Scholar] [CrossRef]
- Khodayari, A.; Ghaffari, A.; Kazemi, R.; Braunstingl, R. A modified car-following model based on a neural network model of the human driver effects. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2012, 42, 1440–1449. [Google Scholar] [CrossRef]
- Morton, J.; Wheeler, T.A.; Kochenderfer, M.J. Analysis of recurrent neural networks for probabilistic modeling of driver behavior. IEEE Trans. Intell. Transp. Syst. 2017, 18, 1289–1298. [Google Scholar] [CrossRef]
- Yang, S.; Wang, W.; Liu, C.; Deng, W. Scene Understanding in Deep Learning-Based End-to-End Controllers for Autonomous Vehicles. IEEE Trans. Syst. ManCybern. Syst. 2019, 49, 53–63. [Google Scholar] [CrossRef]
- García Cuenca, L.; Sanchez-Soriano, J.; Puertas, E.; Fernandez Andrés, J.; Aliane, N.J.S. Machine learning techniques for undertaking roundabouts in autonomous driving. Sensors 2019, 19, 2386. [Google Scholar] [CrossRef]
- Wiest, J.; Höffken, M.; Kreßel, U.; Dietmayer, K. Probabilistic trajectory prediction with gaussian mixture models. In Proceedings of the Intelligent Vehicles Symposium (IV), Madrid, Spain, 3–7 June 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 141–146. [Google Scholar]
- Lefèvre, S.; Carvalho, A.; Gao, Y.; Tseng, H.E.; Borrelli, F. Driver models for personalized driving assistance. Veh. Syst. Dyn. 2015, 53, 1705–1720. [Google Scholar] [CrossRef]
- Lv, C.; Xing, Y.; Lu, C.; Liu, Y.; Guo, H.; Gao, H.; Cao, D. Hybrid-Learning-Based Classification and Quantitative Inference of Driver Braking Intensity of an Electrified Vehicle. IEEE Trans. Veh. Technol. 2018, 67, 5718–5729. [Google Scholar] [CrossRef]
- Lu, C.; Wang, H.; Lv, C.; Gong, J.; Xi, J.; Cao, D. Learning Driver-Specific Behavior for Overtaking: A Combined Learning Framework. IEEE Trans. Veh. Technol. 2018, 67, 6788–6802. [Google Scholar] [CrossRef]
- Ng, L.; Clark, C.M.; Huissoon, J.P. Reinforcement learning of dynamic collaborative driving part I: Longitudinal adaptive control. Int. J. Veh. Inf. Commun. Syst. 2008, 1, 208–228. [Google Scholar] [CrossRef]
- Desjardins, C.; Chaib-draa, B. Cooperative adaptive cruise control: A reinforcement learning approach. IEEE Trans. Intell. Transp. Syst. 2011, 12, 1248–1260. [Google Scholar] [CrossRef]
- Ngai, D.C.K.; Yung, N.H.C. A multiple-goal reinforcement learning method for complex vehicle overtaking maneuvers. IEEE Trans. Intell. Transp. Syst. 2011, 12, 509–522. [Google Scholar] [CrossRef]
- Li, X.; Xu, X.; Zuo, L. Reinforcement learning based overtaking decision-making for highway autonomous driving. In Proceedings of the 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP), Wuhan, China, 26–28 November 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 336–342. [Google Scholar]
- Huang, Z.; Xu, X.; He, H.; Tan, J.; Sun, Z. Parameterized Batch Reinforcement Learning for Longitudinal Control of Autonomous Land Vehicles. IEEE Trans. Syst. Man Cybern. Syst. 2018, 49, 730–741. [Google Scholar] [CrossRef]
- Kim, I.-H.; Bong, J.-H.; Park, J.; Park, S. Prediction of driver’s intention of lane change by augmenting sensor information using machine learning techniques. Sensors 2017, 17, 1350. [Google Scholar] [CrossRef]
- Ten Hagen, S.; Kröse, B. Neural Q-learning. Neural Comput. Appl. 2003, 12, 81–88. [Google Scholar] [CrossRef]
- Priddy, K.L.; Keller, P.E. Artificial Neural Networks: An Introduction; SPIE Press: Bellingham, WA, USA, 2005; Volume 68. [Google Scholar]
- Li, Y.; Ang, K.H.; Chong, G.C. Patents, software, and hardware for PID control: An overview and analysis of the current art. IEEE Control Syst. 2006, 26, 42–54. [Google Scholar]
- Boyan, J.A. Technical update: Least-squares temporal difference learning. Mach Learn 2002, 49, 233–246. [Google Scholar] [CrossRef]
- Gnecco, G.; Sanguineti, M. The weight-decay technique in learning from data: An optimization point of view. Comput. Manag. Sci. 2009, 6, 53–79. [Google Scholar] [CrossRef]
- Masters, D.; Luschi, C. Revisiting small batch training for deep neural networks. arXiv 2018, arXiv:1804.07612. [Google Scholar]
- Marsden, G.; McDonald, M.; Brackstone, M. Towards an understanding of adaptive cruise control. Transp. Res. Part C Emerg. Technol. 2001, 9, 33–51. [Google Scholar] [CrossRef]
- Xu, Y.; Song, J.; Nechyba, M.C.; Yam, Y. Performance evaluation and optimization of human control strategy. Robot. Auton. Syst. 2002, 39, 19–36. [Google Scholar] [CrossRef]
- Wang, B.; Li, Z.; Gong, J.; Liu, Y.; Chen, H.; Lu, C. Learning and Generalizing Motion Primitives from Driving Data for Path-Tracking Applications. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; pp. 1191–1196. [Google Scholar]
Scenarios | ||||||||
---|---|---|---|---|---|---|---|---|
CS/10 [m·s−1] | 0.1 | 0.0005 | 15 | −15 | 80 | 0 | 4 | −4 |
CS/15 [m·s−1] | 0.1 | 0.0005 | 20 | −20 | 80 | 0 | 4 | −4 |
CS/22 [m·s−1] | 0.1 | 0.0005 | 25 | −25 | 80 | 0 | 6 | −6 |
VS/Scene I | 0.01 | 0.05 | 25 | −25 | 80 | 0 | 4 | −4 |
VS/Scene II | 0.01 | 0.5 | 25 | −25 | 80 | 0 | 8 | −8 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, C.; Gong, J.; Lv, C.; Chen, X.; Cao, D.; Chen, Y. A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles. Sensors 2019, 19, 3672. https://doi.org/10.3390/s19173672
Lu C, Gong J, Lv C, Chen X, Cao D, Chen Y. A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles. Sensors. 2019; 19(17):3672. https://doi.org/10.3390/s19173672
Chicago/Turabian StyleLu, Chao, Jianwei Gong, Chen Lv, Xin Chen, Dongpu Cao, and Yimin Chen. 2019. "A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles" Sensors 19, no. 17: 3672. https://doi.org/10.3390/s19173672
APA StyleLu, C., Gong, J., Lv, C., Chen, X., Cao, D., & Chen, Y. (2019). A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles. Sensors, 19(17), 3672. https://doi.org/10.3390/s19173672