# A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- a fixed number of players $N=\{1,2,3\cdots n\}$;
- a set of strategies ${S}_{1},{S}_{2},{S}_{3}\cdots {S}_{n}$ with respective to the number of players;
- specific outcomes or payoffs for any possible combination of the strategies $({s}_{1},{s}_{2},{s}_{3}\cdots {s}_{n})\in {S}_{1}\ast {S}_{2}\ast {S}_{3}\ast \cdots {S}_{n}$ (where ${s}_{i}$ is a current strategy of player i).

- an autonomous vehicle (EGO) (in white) that performs the lane-change maneuver and is controlled by the game theory-based strategy S;
- the following vehicle (FV) in green; and
- the leading vehicle in front of the FV, represented by LEAD (in red).

- When the traffic light is red, EGO enters the road from the right lane and stops behind the last vehicle in the queue.
- EGO needs to perform a lane change to turn left at the intersection. Due to the traffic stopped in the left lane, EGO interacts with FV by activating the turn signal to indicate its intention to merge in front.
- When the traffic light turns green, FV decides whether to wait and allow EGO to merge in front. This FV decision is made based on the perceived information related to the current EGO acceleration and previously conveyed information by the EGO vehicle regarding its merging intention.

- strategic layer, related to route planning;
- tactical layer, related to decision-making processes; and
- operational layer, linked to control tasks.

## 2. Related Work

## 3. The Proposed Decision-Making Model

#### 3.1. Game Formulation

- to change lanes, namely strategy $A1$, and
- not to change lanes, strategy $A2$.

- accepting EGO’s lane change or strategy $B1$
- rejecting EGO’s lane change or strategy $B2$

- outcome ${P}_{xy}$
- chosen strategy y

- outcome ${Q}_{xy}$
- chosen strategy x

#### 3.2. The Payoff and Penalty Functions

#### 3.2.1. Safety Payoff Function of FV

#### 3.2.2. Space Payoff Function of the FV

#### 3.2.3. Safety Payoff Function of EGO

#### 3.2.4. Penalty Function of the Vehicles

#### 3.2.5. The Outcome of the Vehicles

## 4. Data Acquisition and Use Case Definition for Model Validation

#### 4.1. 3DCoAutoSim Description

#### 4.2. SUMO Description

Algorithm 1 Trigger for Activating the EGO Vehicle |

input: Player class. $pv$; RoadID of the last intersection, $rID$;output: event t; |

#### 4.3. Experimental Setup

Algorithm 2 Proposed game theory-based decision-making model. |

input: Player vehicle class, $pv$; LEAD vehicle class, $lv$; EGO vehicle class, $ev$;output: Bool $res$;t1 = 0, t2 = 0; |

- Drive for 5 min to become familiarized with the simulation platform. No data were collected.
- Drive from the origin to destination, as described in the first paragraphs of Section 4.
- Drive from the origin to destination, as described but with a time limitation. This task aimed to add pressure and urgency and to motivate the participant to reject a lane-change request from the EGO side.

## 5. Results

#### 5.1. Game Solution Description

#### 5.2. Driving Experiment Prediction Results

## 6. Conclusions and Future Work

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Lai, F.; Carsten, O.; Tate, F. How much benefit does Intelligent Speed Adaptation deliver?—An analysis of its potential contribution to safety and environment. Accid. Anal. Prev.
**2012**, 48, 63–72. [Google Scholar] [CrossRef] - Ahmed, S.S.; Sonduru Pantangi, S.; Eker, U.; Fountas, G.; Still, S.E.; Anastasopoulos, P.C. Analysis of Safety Benefits and Security Concerns from the Use of Autonomous Vehicles: A Grouped Random Parameters Bivariate Probit Approach with Heterogeneity in Means. Anal. Methods Accid. Res.
**2020**, 28, 100134. [Google Scholar] [CrossRef] - Kauffmann, N.; Winkler, F.; Naujoks, F.; Vollrath, M. “What Makes a Cooperative Driver?” Identifying parameters of implicit and explicit forms of communication in a lane change scenario. Transp. Res. Part F Traffic Psychol. Behav.
**2018**, 58, 1031–1042. [Google Scholar] [CrossRef] - Olaverri-Monreal, C. Promoting trust in self-driving vehicles. Nat. Electron.
**2020**, 3, 292–294. [Google Scholar] [CrossRef] - Olaverri-Monreal, C. Road safety: Human factors aspects of intelligent vehicle technologies. In Smart Cities, Green Technologies, and Intelligent Transport Systems; Springer: London, UK, 2017; pp. 318–332. [Google Scholar]
- Hollander, Y.; Prashker, J.N. The applicability of non-cooperative game theory in transport analysis. Transportation
**2006**, 33, 481–496. [Google Scholar] [CrossRef] - Banjanovic-Mehmedovic, L.; Halilovic, E.; Bosankic, I.; Kantardzic, M.; Kasapovic, S. Autonomous Vehicle-to-Vehicle (V2V) Decision Making in Roundabout using Game Theory. Int. J. Adv. Comput. Sci. Appl.
**2016**, 7. [Google Scholar] [CrossRef] - Fox, C.W.; Camara, F.; Markkula, G.; Romano, R.A.; Madigan, R.; Merat, N. When should the chicken cross the road? Game theory for autonomous vehicle-human interactions. In Proceedings of the 4th International Conference on Vehicle Technology and Intelligent Transport Systems–VEHITS 2018, Madeira, Portugal, 16–18 March 2018; pp. 429–431. [Google Scholar] [CrossRef]
- Laurgeau, C. Intelligent Vehicle Potential and Benefits; Springer: London, UK, 2012; pp. 1537–1551. [Google Scholar] [CrossRef]
- Hang, P.; Lv, C.; Xing, Y.; Huang, C.; Hu, Z. Human-Like Decision Making for Autonomous Driving: A Noncooperative Game Theoretic Approach. IEEE Trans. Intell. Transp. Syst.
**2020**. [Google Scholar] [CrossRef] - Schwarting, W.; Pierson, A.; Alonso-Mora, J.; Karaman, S.; Rus, D. Social behavior for autonomous vehicles. Proc. Natl. Acad. Sci. USA
**2019**, 116, 2492–24978. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wang, M.; Hoogendoorn, S.P.; Daamen, W.; van Arem, B.; Happee, R. Game theoretic approach for predictive lane-changing and car-following control. Transp. Res. Part C Emerg. Technol.
**2015**, 58, 73–92. [Google Scholar] [CrossRef] - Ali, Y.; Zheng, Z.; Haque, M.M.; Wang, M. A game theory-based approach for modelling mandatory lane-changing behaviour in a connected environment. Transp. Res. Part C Emerg. Technol.
**2019**, 106, 220–242. [Google Scholar] [CrossRef] - Harsanyi, J.C. Games with Incomplete Information Played by “Bayesian” Players, I–III Part I. The Basic Model. Manag. Sci.
**1967**, 14, 159–182. [Google Scholar] [CrossRef] - Talebpour, A.; Mahmassani, H.S.; Hamdar, S.H. Modeling lane-changing behavior in a connected environment: A game theory approach. Transp. Res. Part C Emerg. Technol.
**2015**, 59, 216–232. [Google Scholar] [CrossRef] - Yu, H.; Tseng, H.E.; Langari, R. A human-like game theory-based controller for automatic lane changing. Transp. Res. Part C Emerg. Technol.
**2018**, 88, 140–158. [Google Scholar] [CrossRef] - Gipps, P.G. A model for the structure of lane-changing decisions. Transp. Res. Part B Methodol.
**1986**, 20, 403–414. [Google Scholar] [CrossRef] - Cao, P.; Hu, Y.; Miwa, T.; Wakita, Y.; Morikawa, T.; Liu, X. An optimal mandatory lane change decision model for autonomous vehicles in urban arterials. J. Intell. Transp. Syst. Technol. Plan. Oper.
**2017**, 21, 271–284. [Google Scholar] [CrossRef] - Cao, P.; Xu, Z.; Fan, Q.; Liu, X. Analysing driving efficiency of mandatory lane change decision for autonomous vehicles. IET Intell. Transp. Syst.
**2019**, 13, 506–514. [Google Scholar] [CrossRef] - Sun, D.; Elefteriadou, L. A driver behavior-based lane-changing model for urban arterial streets. Transp. Sci.
**2014**, 48, 184–205. [Google Scholar] [CrossRef] - Ji, A.; Levinson, D. A review of game theory models of lane changing. Transp. A Transp. Sci.
**2020**, 16, 1628–1647. [Google Scholar] [CrossRef] - Haas, J. A History of the Unity Game Engine—An Interactive Qualifying Project. 2014. Available online: https://web.wpi.edu/Pubs/E-project/Available/E-project-030614-143124/unrestricted/Haas_IQP_Final.pdf (accessed on 10 January 2021).
- Behrisch, M.; Bieker, L.; Erdmann, J.; Krajzewicz, D. SUMO–simulation of urban mobility: An overview. In Proceedings of the SIMUL 2011, The Third International Conference on Advances in System Simulation, Barcelona, Spain, 23–29 October 2011. [Google Scholar]
- Olaverri-Monreal, C.; Errea-Moreno, J.; Díaz-Álvarez, A. Implementation and Evaluation of a Traffic Light Assistance System Based on V2I Communication in a Simulation Framework. J. Adv. Transp.
**2018**, 2018. [Google Scholar] [CrossRef] [Green Version] - Olaverri-Monreal, C.; Errea-Moreno, J.; Díaz-Álvarez, A.; Biurrun-Quel, C.; Serrano-Arriezu, L.; Kuba, M. Connection of the SUMO Microscopic Traffic Simulator and the Unity 3D Game Engine to Evaluate V2X Communication-Based Systems. Sensors
**2018**, 18, 4399. [Google Scholar] [CrossRef] [Green Version] - Biurrun-Quel, C.; Serrano-Arriezu, L.; Olaverri-Monreal, C. Microscopic driver-centric simulator: Linking Unity3d and SUMO. In World Conference on Information Systems and Technologies; Springer: London, UK, 2017; pp. 851–860. [Google Scholar]
- Hussein, A.; Diaz-Alvarez, A.; Armingol, J.M.; Olaverri-Monreal, C. 3DCoAutoSim: Simulator for Cooperative ADAS and Automated Vehicles. In Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA, 4–7 November 2018; pp. 3014–3019. [Google Scholar] [CrossRef]
- Artal-Villa, L.; Hussein, A.; Olaverri-Monreal, C. Extension of the 3DCoAutoSim to simulate vehicle and pedestrian interaction based on SUMO and unity 3D. In IEEE Intelligent Vehicles Symposium; Institute of Electrical and Electronics Engineers Inc.: New York, NY, USA, 2019; Volume 2019, pp. 885–890. [Google Scholar] [CrossRef]
- Hussein, A.; García, F.; Olaverri-Monreal, C. Ros and unity based framework for intelligent vehicles control and simulation. In Proceedings of the 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, Spain, 12–14 September 2018; pp. 1–6. [Google Scholar]
- Liu, Y.; Novotny, G.; Smirnov, N.; Morales-Alvarez, W.; Olaverri-Monreal, C. Mobile Delivery Robots: Mixed Reality-Based Simulation Relying on ROS and Unity 3D. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020; pp. 15–20. [Google Scholar] [CrossRef]
- OpenStreetMap. Available online: https://www.openstreetmap.org/#map=7/47.714/13.349 (accessed on 14 October 2020).
- Netconvert-SUMO Documentation. Available online: https://sumo.dlr.de/docs/netconvert.html (accessed on 24 September 2020).
- Polyconvert-SUMO Documentation. Available online: https://sumo.dlr.de/docs/polyconvert.html (accessed on 8 October 2020).
- Tools/Trip-SUMO Documentation. Available online: https://sumo.dlr.de/docs/Tools/Trip.html (accessed on 7 June 2020).
- Validi, A.; Ludwig, T.; Hussein, A.; Olaverri-Monreal, C. Examining the Impact on Road Safety of Different Penetration Rates of Vehicle-to-Vehicle Communication and Adaptive Cruise Control. IEEE Intell. Transp. Syst. Mag.
**2018**, 10, 24–34. [Google Scholar] [CrossRef] - Geodatenviewer der Stadtvermessung Wien. Available online: https://www.wien.gv.at/ma41datenviewer/public/start.aspx (accessed on 6 November 2020).
- Blender.org. Available online: https://www.blender.org (accessed on 28 December 2020).

**Figure 1.**Illustration of the proposed lane-change scenario and the relevant players. The autonomous vehicle (EGO) (in white) performs a lane change to occupy the position in front of the following vehicle (FV) in green. The leading vehicle (LEAD, in red) is located in front of the FV.

**Figure 2.**Dynamic noncooperative game description in its extensive-form representation. EGO and FV denote the players; $A1$ and $A2$, and $B1$ and $B2$ denote their respective sets of strategies. Pxy and Qxy denote the outcome of the game, where x is the FV’s selected strategy and y is the strategy selected by EGO.

**Figure 3.**FV’s safety function represented by safety in the form of scored points and the headway or gap between vehicles in meters.

**Figure 4.**FV’s space function depending on the cooperation level regarding willingness to let EGO merge (scored points) and the headway or gap between FV and LEAD in meters.

**Figure 5.**Dependencies between the EGO’s safety payoff function and the position of EGO (${x}_{EGO}$). Note that ${x}_{LEAD}$ is the longitudinal position of LEAD, which determines the distance between FV and LEAD.

**Figure 6.**Penalty function, with ${a}_{a}=1.5\phantom{\rule{3.33333pt}{0ex}}{\mathrm{m}/\mathrm{s}}^{2}$, $V=0\phantom{\rule{3.33333pt}{0ex}}\mathrm{m}/\mathrm{s}$, $t=3\phantom{\rule{3.33333pt}{0ex}}\mathrm{s}$, ${V}_{a}=5\phantom{\rule{3.33333pt}{0ex}}\mathrm{m}/\mathrm{s}$, $wa=500$, and $wd=500$.

**Figure 7.**Model validation scenario: 1. The player’s vehicle (FV) approaches the target intersection. 2. The player’s vehicle enters the queue to wait for the traffic light to turn green. EGO is activated to move to the left lane; 3. EGO vehicle is located at the position defined in the proposed decision-making model.

**Figure 9.**The 3DCoAutoSim simulation platform during a driving test and a screenshot of the scenario setup and in-vehicle systems.

**Figure 10.**Trip origin and destination on the generated Simulation of Urban MObility (SUMO) road network to perform the model validation with a section of the generated SUMO simulation with different vehicle types at the destination.

**Figure 11.**SUMO simulation steps (adapted from [35]).

**Figure 12.**Dynamic noncooperative game description in its extensive-form representation. EGO and FV denote the players; ${a}_{0}$ and ${a}_{m}$ denote their respective sets of strategies. Pxy and Qxy denote the outcome of the game, where x is the FV’s selected strategy and y is the strategy selected by EGO.

**Table 1.**Dynamic noncooperative game description in its normal-form representation. EGO and FV denote the players; $A1$ and $A2$, and $B1$ and $B2$ denote their respective sets of strategies. Pxy and Qxy denote the outcome of the game, where x is the FV’s selected strategy and y is the strategy selected by EGO.

Action | EGO Vehicle | |
---|---|---|

A1 (Change Lane) | A2 (Do Not Change Lane) | |

The FV vehicle | ||

B1 (Accept) | $({P}_{11},{Q}_{11})$ | $({P}_{12},{Q}_{12})$ |

B2 (Decline) | $({P}_{21},{Q}_{21})$ | $({P}_{22},{Q}_{22})$ |

**Table 2.**Game description in its normal-form representation. EGO and FV denote the players. ${a}_{EGO}$ and ${a}_{FV}$ denote the sets of strategies for the EGO and FV, respectively. ${P}_{{a}_{FV},{a}_{EGO}}$ and ${Q}_{{a}_{FV},{a}_{EGO}}$ denote the outcome values of the game with respect to the players.

Action | EGO Vehicle | |
---|---|---|

$\mathbf{0}\le {\mathit{a}}_{\mathit{EGO}}\le \mathit{F}\left(\mathit{LEAD}\right)$ | ||

FV | ||

$0\le {a}_{FV}\le F\left(LEAD\right)$ | $({P}_{{a}_{FV},{a}_{EGO}},{Q}_{{a}_{FV},{a}_{EGO}})$ |

**Table 3.**Input values regarding the driving performance data used to predict cooperation for an EGO merging maneuver. ${a}_{a}$ and ${v}_{a}$ denote the acquired speed and acceleration values of Player prior to stopping at the last intersection, a and v are the acceleration and speed values at an instant of time right before being used to calculate the prediction, $gap$ is the existent gap between Player and LEAD, $action$ denotes the player’s decision, and $predicted$ is the model prediction value.

$\mathit{test}$ | ${\mathit{a}}_{\mathit{a}}$${\mathbf{m}/\mathbf{s}}^{2}$ | ${\mathit{v}}_{\mathit{a}}$ m/s | a m/s${}^{2}$ | v m/s | $\mathit{gap}$ m | $\mathit{action}$ | $\mathit{predicted}$ |
---|---|---|---|---|---|---|---|

1 | 0.84 | 8.61 | 0.01 | 0.07 | 7.07 | reject | accept |

2 | 1.48 | 12.01 | 0.04 | 0.95 | 3.50 | reject | reject |

3 | 0.80 | 6.84 | 0.00 | 0.00 | 7.72 | accept | accept |

4 | 1.17 | 8.98 | 0.02 | 0.56 | 5.31 | reject | reject |

5 | 1.34 | 9.65 | 0.00 | 0.00 | 7.07 | accept | accept |

6 | 1.42 | 11.53 | 0.02 | 0.27 | 2.92 | reject | reject |

7 | 1.22 | 10.62 | 0.03 | 0.34 | 1.36 | reject | reject |

8 | 2.11 | 12.89 | 0.00 | 0.21 | 3.11 | reject | reject |

9 | 1.08 | 10.33 | 0.02 | 0.17 | 5.17 | accept | accept |

10 | 1.44 | 12.03 | 0.03 | 0.59 | 3.25 | reject | reject |

11 | 1.03 | 7.79 | 0.2 | 0.27 | 4.76 | reject | reject |

12 | 1.03 | 8.77 | 0.00 | 0.01 | 4.41 | reject | accept |

13 | 1.16 | 8.41 | 0.00 | 0.00 | 3.79 | accept | accept |

14 | 1.80 | 11.29 | 0.03 | 0.48 | 1.93 | reject | reject |

15 | 1.24 | 10.81 | 0.00 | 0.00 | 7.53 | accept | accept |

16 | 2.08 | 14.28 | 0.02 | 1.14 | 5.21 | reject | reject |

n = 16 | Predicted: NO | Predicted: YES | |

Actual: NO | 9 | 2 | 11 |

Actual: YES | 0 | 5 | 2 |

9 | 7 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Smirnov, N.; Liu, Y.; Validi, A.; Morales-Alvarez, W.; Olaverri-Monreal, C.
A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios. *Sensors* **2021**, *21*, 1523.
https://doi.org/10.3390/s21041523

**AMA Style**

Smirnov N, Liu Y, Validi A, Morales-Alvarez W, Olaverri-Monreal C.
A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios. *Sensors*. 2021; 21(4):1523.
https://doi.org/10.3390/s21041523

**Chicago/Turabian Style**

Smirnov, Nikita, Yuzhou Liu, Aso Validi, Walter Morales-Alvarez, and Cristina Olaverri-Monreal.
2021. "A Game Theory-Based Approach for Modeling Autonomous Vehicle Behavior in Congested, Urban Lane-Changing Scenarios" *Sensors* 21, no. 4: 1523.
https://doi.org/10.3390/s21041523