Autonomous Vehicle Decision-Making with Policy Prediction for Handling a Round Intersection
Abstract
:1. Introduction
2. Related Work
3. Decision-Making and Planning for Handling Round Intersections
3.1. Markov Decision Processes
3.2. Application to Round Intersections
- is the class of objects in the decision-making problem. In this problem, we consider two classes of objects, which are the ego vehicle class for the ego vehicle and the other vehicle class for other vehicles in the environment.
- are the attributes of objects in different classes. The attributes of the ego vehicle class include the position information and motion information . The attributes of the other vehicle class include position information and velocity .
- is the domain of attributes. The domain of attributes in this paper is mainly on the poses of vehicles. For both the ego vehicle class and the other vehicle class, the positional domain is within the area of the round intersection, and the heading angle is within the range of .
- A is action set. The action is a combination of two finite set of actions, for linear acceleration and angular velocity. Each action is selected from the set and results in a pair .
4. Improving Decision-Making with Policy Prediction
5. Augmented Objective State and Policy-Based State Transition
6. Results and Discussion
- Planning time for each step: 1.0 s;
- Maximum search depth in the tree: 100;
- Exploration constant value: 2.0;
- Discount factor of the cumulative expected reward: 0.99.
- In the straight lane segments, the vehicle’s yaw rate is maintained at zero as it keeps driving straight.
- In the round intersection part, the vehicle drives along the road at the yaw rate of , where is the radius of the round intersection and is the vehicle’s linear velocity. This is a positive value when setting the counterclockwise direction as the positive direction.
- When the vehicle is entering/exiting the round intersection, the yaw rate is some negative value when the counterclockwise direction is taken as the positive direction. In the simulation, since the entering/exiting process only takes a very short time period, the yaw angle change is approximated with a yaw rate determined by the process of uncontrolled simulation vehicles entering the round intersection.
7. Conclusions and Recommendations
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Gkartzonikas, C.; Gkritza, K. What have we learned? A review of stated preference and choice studies on autonomous vehicles. Transp. Res. Part C 2019, 98, 323–337. [Google Scholar] [CrossRef]
- Rana, M.M.; Hossain, K. Connected and Autonomous Vehicles and Infrastructures: A Literature Review. Int. J. Pavement Res. Technol. 2023, 16, 264–284. [Google Scholar] [CrossRef]
- Reyes-Muñoz, A.; Guerrero-Ibáñez, J. Vulnerable Road Users and Connected Autonomous Vehicles Interaction: A Survey. Sensors 2022, 22, 4614. [Google Scholar] [CrossRef] [PubMed]
- Sadid, H.; Antoniou, C. Modelling and simulation of (connected) autonomous vehicles longitudinal driving behavior: A state-of-the-art. IET Intell. Transp. Syst. 2023, 17, 1051–1071. [Google Scholar] [CrossRef]
- Guvenc, L.; Guvenc, B.A.; Emirler, M.T. Connected and Autonomous Vehicles. In Internet of Things and Data Analytics Handbook; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2017; pp. 581–595. ISBN 978-1-119-17360-1. [Google Scholar]
- Claussmann, L.; Revilloud, M.; Gruyer, D.; Glaser, S. A Review of Motion Planning for Highway Autonomous Driving. IEEE Trans. Intell. Transp. Syst. 2020, 21, 1826–1848. [Google Scholar] [CrossRef]
- Kuwata, Y.; Teo, J.; Fiore, G.; Karaman, S.; Frazzoli, E.; How, J.P. Real-Time Motion Planning with Applications to Autonomous Urban Driving. IEEE Trans. Control Syst. Technol. 2009, 17, 1105–1118. [Google Scholar] [CrossRef]
- Yurtsever, E.; Lambert, J.; Carballo, A.; Takeda, K. A Survey of Autonomous Driving: Common Practices and Emerging Technologies. IEEE Access 2020, 8, 58443–58469. [Google Scholar] [CrossRef]
- Kuutti, S.; Fallah, S.; Katsaros, K.; Dianati, M.; Mccullough, F.; Mouzakitis, A. A Survey of the State-of-the-Art Localization Techniques and Their Potentials for Autonomous Vehicle Applications. IEEE Internet Things J. 2018, 5, 829–846. [Google Scholar] [CrossRef]
- Guvenç, L.; Guvenç, B.A.; Demirel, B.; Emirler, M.T. Control of Mechatronic Systems; IET Control, Robotics and Sensors Series; The Institute of Engineering and Technology: London, UK, 2017; ISBN 978-1-78561-145-2. [Google Scholar]
- Guvenc, L.; Aksun-Guvenc, B.; Zhu, S.; Gelbal, S.Y. Autonomous Road Vehicle Path Planning and Tracking Control; Book Series on Control Systems Theory and Application; Wiley/IEEE Press: New York, NY, USA, 2021; ISBN 978-1-119-74794-9. [Google Scholar]
- Li, S.; Shu, K.; Chen, C.; Cao, D. Planning and Decision-making for Connected Autonomous Vehicles at Road Intersections: A Review. Chin. J. Mech. Eng. 2021, 34, 133. [Google Scholar] [CrossRef]
- Schwarting, W.; Alonso-Mora, J.; Rus, D. Planning and Decision-Making for Autonomous Vehicles. Annu. Rev. Control Robot. Auton. Syst. 2018, 1, 187–210. [Google Scholar] [CrossRef]
- Gelbal, S.Y.; Aksun-Guvenc, B.; Guvenc, L. SmartShuttle: A Unified, Scalable and Replicable Approach to Connected and Automated Driving in a Smart City. In Proceedings of the Science of Smart City Operations and Platforms Engineering in partnership with Global City Teams Challenge (SCOPE-GCTC) Workshop, Pittsburgh, PA, USA, 18–21 April 2017. [Google Scholar]
- Al-Turki, M.; Ratrout, N.T.; Rahman, S.M.; Assi, K.J. Signalized Intersection Control in Mixed Autonomous and Regular Vehicles Traffic Environment—A Critical Review Focusing on Future Control. IEEE Access 2022, 10, 16942–16951. [Google Scholar] [CrossRef]
- Li, X.; Zhu, S.; Aksun-Guvenc, B.; Guvenc, L. Development and Evaluation of Path and Speed Profile Planning and Tracking Control for an Autonomous Shuttle Using a Realistic, Virtual Simulation Environment. J. Intell. Robot. Syst. 2021, 101, 42. [Google Scholar] [CrossRef]
- Savolainen, P.T.; Gates, T.J.; Gupta, N.; Megat-Johari, M.U.; Cai, Q.; Imosemi, S.; Ceifetz, A.; McArthur, A.; Hagel, E.C.; Smaglik, E.J. Evaluating the Performance and Safety Effectiveness of Roundabouts–An Update; Report Number: SPR-1725; Michigan Department of Transportation: Grand Rapids, MI, USA, 2023. [Google Scholar]
- Elvik, R. Road safety effects of roundabouts: A meta-analysis. Accid. Anal. Prev. 2017, 99, 364–371. [Google Scholar] [CrossRef] [PubMed]
- Wang, W.; Jiang, L.; Lin, S.; Fang, H.; Meng, Q. Imitation learning based decision-making for autonomous vehicle control at traffic roundabouts. Multimed. Tools Appl. 2022, 81, 39873–39889. [Google Scholar] [CrossRef]
- Monsalve, B.; Aliane, N.; Puertas, E.; Andrés, J.F. Think Aloud Protocol and Decision Tree for Driver Behavior Modeling at Roundabouts. IEEE Access 2023, 11, 41444–41454. [Google Scholar] [CrossRef]
- Sana, F.; Azad, N.L.; Raahemifar, K. Autonomous Vehicle Decision-Making and Control in Complex and Unconventional Scenarios—A Review. Machines 2023, 11, 676. [Google Scholar] [CrossRef]
- Muffert, M.; Pfeiffer, D.; Franke, U. A Stereo-Vision Based Object Tracking Approach at Roundabouts. IEEE Intell. Transp. Syst. Mag. 2013, 5, 22–32. [Google Scholar] [CrossRef]
- Okumura, B.; James, M.R.; Kanzawa, Y.; Derry, M.; Sakai, K.; Nishi, T.; Prokhorov, D. Challenges in Perception and Decision Making for Intelligent Automotive Vehicles: A Case Study. IEEE Trans. Intell. Veh. 2016, 1, 20–32. [Google Scholar] [CrossRef]
- Banjanovic-Mehmedovic, L.; Halilovic, E.; Bosankic, I.; Kantardzic, M.; Kasapovic, S. Autonomous Vehicle-to-Vehicle (V2V) Decision Making in Roundabout using Game Theory. Int. J. Adv. Comput. Sci. Appl. 2016, 7, 292. [Google Scholar] [CrossRef]
- Shen, M.; Sun, J.; Zhao, D. The Impact of Road Configuration in V2V-Based Cooperative Localization: Mathematical Analysis and Real-World Evaluation. IEEE Trans. Intell. Transp. Syst. 2018, 19, 3220–3229. [Google Scholar] [CrossRef]
- Fitzpatrick, C.D.; Abrams, D.S.; Tang, Y.; Knodler, M.A. Spatial and Temporal Analysis of Driver Gap Acceptance Behavior at Modern Roundabouts. Transp. Res. Rec. 2013, 2388, 14–20. [Google Scholar] [CrossRef]
- Wang, Z.; Liu, X.; Wu, Z. Design of Unsignalized Roundabouts Driving Policy of Autonomous Vehicles Using Deep Reinforcement Learning. World Electr. Veh. J. 2023, 14, 52. [Google Scholar] [CrossRef]
- Hang, P.; Huang, C.; Hu, Z.; Xing, Y.; Lv, C. Decision Making of Connected Automated Vehicles at an Unsignalized Roundabout Considering Personalized Driving Behaviours. IEEE Trans. Veh. Technol. 2021, 70, 4051–4064. [Google Scholar] [CrossRef]
- Farkas, Z.; Mihály, A.; Gáspár, P. Model Predictive Control Method for Autonomous Vehicles in Roundabouts. Machines 2023, 11, 75. [Google Scholar] [CrossRef]
- Cao, H.; Zoldy, M. MPC Tracking Controller Parameters Impacts in Roundabouts. Mathematics 2021, 9, 1394. [Google Scholar] [CrossRef]
- Ozcan, D.; Sonmez, U.; Guvenc, L. Optimisation of the Nonlinear Suspension Characteristics of a Light Commercial Vehicle. Int. J. Veh. Technol. 2013, 2013, 562424. [Google Scholar]
- Klos, M.J.; Sobota, A. Performance evaluation of roundabouts using a microscopic simulation model. Sci. J. Silesian Univ. Technology. Ser. Transp. 2019, 104, 57–67. [Google Scholar] [CrossRef]
- Arroju, R.; Gaddam, H.K.; Vanumu, L.D.; Rao, K.R. Comparative evaluation of roundabout capacities under heterogeneous traffic conditions. J. Mod. Transport. 2015, 23, 310–324. [Google Scholar] [CrossRef]
- Bagheri, M.; Bartin, B.; Ozbay, K. Implementing Artificial Neural Network-Based Gap Acceptance Models in the Simulation Model of a Traffic Circle in SUMO. Transp. Res. Rec. 2023; early access. [Google Scholar]
- El Ganaoui-Mourlan, O.; Camp, S.; Verhas, C.; Pollet, N.; Ortega, B.; Robic, B. Traffic Manager Development for a Roundabout Crossed by Autonomous and Connected Vehicles Using V2I Architecture. Sustainability 2023, 15, 9247. [Google Scholar] [CrossRef]
- Masi, S.; Xu, P.; Bonnifait, P. A Curvilinear Decision Method for Two-lane Roundabout Crossing and its Validation under Realistic Traffic Flow. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020; pp. 1290–1296. [Google Scholar] [CrossRef]
- Zainudin, H.; Koufos, K.; Lee, G.; Jiang, L.; Dianati, M. Impact analysis of cooperative perception on the performance of automated driving in unsignalized roundabouts. Front. Robot. AI 2023, 10, 1164950. [Google Scholar] [CrossRef]
- Somani, A.; Ye, N.; Hsu, D.; Lee, W.S. DESPOT: Online POMDP Planning with Regularization. In Proceedings of the Neural Information Processing Systems, Lake Tahoe, NV, USA, 5–10 December 2013; pp. 1772–1780. [Google Scholar]
- Wandzel, A.; Oh, Y.; Fishman, M.; Kumar, N.; Wong, L.S.; Tellex, S. Multi-object search using object-oriented POMDPs. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 7194–7200. [Google Scholar]
- Silver, D.; Veness, J. Monte-Carlo planning in large POMDPs. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, USA, 6–9 December 2010; pp. 2164–2172. [Google Scholar]
- Hubmann, C.; Schulz, J.; Becker, M.; Althoff, D.; Stiller, C. Automated Driving in Uncertain Environments: Planning with Interaction and Uncertain Maneuver Prediction. IEEE Trans. Intell. Veh. 2018, 3, 5–17. [Google Scholar] [CrossRef]
- Sriram, N.; Liu, B.; Pittaluga, F.; Chandraker, M. Smart: Simultaneous multi-agent recurrent trajectory prediction. In Proceedings of the European Conference on Computer Vision 2020, Glasgow, UK, 23–28 August 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 463–479. [Google Scholar]
- Tian, R.; Li, S.; Li, N.; Kolmanovsky, I.; Girard, A.; Yildiz, Y. Adaptive game-theoretic decision making for autonomous vehicle control at roundabouts. In Proceedings of the 2018 IEEE Conference on Decision and Control (CDC), Miami Beach, FL, USA, 17–19 December 2018; pp. 321–326. [Google Scholar]
- Lam, C.-P.; Yang, A.Y.; Driggs-Campbell, K.; Bajcsy, R.; Sastry, S.S. Improving human-in-the-loop decision making in multi-mode driver assistance systems using hidden mode stochastic hybrid systems. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September–3 October 2015; pp. 5776–5783. [Google Scholar]
- Nieuwenhuijze, M.R.I.; van Keulen, T.; Öncü, S.; Bonsen, B.; Nijmeijer, H. Cooperative Driving With a Heavy-Duty Truck in Mixed Traffic: Experimental Results. IEEE Trans. Intell. Transp. Syst. 2012, 13, 1026–1032. [Google Scholar] [CrossRef]
- Öncü, S.; Ploeg, J.; van de Wouw, N.; Nijmeijer, H. Cooperative Adaptive Cruise Control: Network-Aware Analysis of String Stability. IEEE Trans. Intell. Transp. Syst. 2014, 15, 1527–1537. [Google Scholar] [CrossRef]
- Feng, Y.; He, D.; Guan, Y. Composite Platoon Trajectory Planning Strategy for Intersection Throughput Maximization. IEEE Trans. Veh. Technol. 2019, 68, 6305–6319. [Google Scholar] [CrossRef]
- Aksun-Guvenc, B.; Guvenc, L.; Ozturk, E.S.; Yigit, T. Model Regulator Based Individual Wheel Braking Control. In Proceedings of the IEEE Conference on Control Applications, Istanbul, Turkey, 23–25 April 2003. [Google Scholar]
Desciption | Total Reward | Total Travel Time (s) | Emergency Brake |
---|---|---|---|
OOPOMDP with policy-based state transition | −527.9 | 21 | No emergency brake performed |
OOPOMDP-based decision-making | −593.674 | 23 | Emergency brake performed by another vehicle |
System driver | Not applicable | 23 | Not applicable |
Desciption | Total Reward | Total Travel Time (s) | Emergency Brake |
---|---|---|---|
OOPOMDP with policy-based state transition | −450 | 20.9 | No emergency brake performed |
OOPOMDP-based decision-making | −14,516.3 | 22 | Potential Collision |
System driver | Not applicable | 33.7 | Not applicable |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, X.; Guvenc, L.; Aksun-Guvenc, B. Autonomous Vehicle Decision-Making with Policy Prediction for Handling a Round Intersection. Electronics 2023, 12, 4670. https://doi.org/10.3390/electronics12224670
Li X, Guvenc L, Aksun-Guvenc B. Autonomous Vehicle Decision-Making with Policy Prediction for Handling a Round Intersection. Electronics. 2023; 12(22):4670. https://doi.org/10.3390/electronics12224670
Chicago/Turabian StyleLi, Xinchen, Levent Guvenc, and Bilin Aksun-Guvenc. 2023. "Autonomous Vehicle Decision-Making with Policy Prediction for Handling a Round Intersection" Electronics 12, no. 22: 4670. https://doi.org/10.3390/electronics12224670