# Successful Nash Equilibrium Agent for a Three-Player Imperfect-Information Game

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Three-Player Kuhn Poker

## 3. Nash Equilibrium-Based Agent

## 4. Experiments

## 5. Conclusions

## Author Contributions

## Acknowledgments

## Conflicts of Interest

## References

- Nash, J. Non-cooperative games. Ph.D. Thesis, Princeton University, Princeton, NJ, USA, 1950. [Google Scholar]
- Von Neumann, J. Zur Theorie der Gesellschaftsspiele. Math. Ann.
**1928**, 100, 295–320. [Google Scholar] [CrossRef] - Brown, N.; Sandholm, T. Safe and Nested Subgame Solving for Imperfect-Information Games. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, 4–9 December 2017. [Google Scholar]
- Moravčík, M.; Schmid, M.; Burch, N.; Lisý, V.; Morrill, D.; Bard, N.; Davis, T.; Waugh, K.; Johanson, M.; Bowling, M. DeepStack: Expert-level artificial intelligence in heads-up no-limit poker. Science
**2017**, 356, 508–513. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Koller, D.; Megiddo, N. The Complexity of Two-Person Zero-Sum Games in Extensive Form. Games Econ. Behav.
**1992**, 4, 528–552. [Google Scholar] [CrossRef] - Chen, X.; Deng, X. 3-Nash is PPAD-Complete; Report No. 134; Electronic Colloquium on Computational Complexity: Trier, Germany, 2005. [Google Scholar]
- Berg, K.; Sandholm, T. Exclusion Method for Finding Nash Equilibrium in Multiplayer Games. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
- Porter, R.; Nudelman, E.; Shoham, Y. Simple Search Methods for Finding a Nash Equilibrium. Games Econ. Behav.
**2008**, 63, 642–662. [Google Scholar] [CrossRef] - Govindan, S.; Wilson, R. A Global Newton Method to Compute Nash Equilibria. J. Econ. Theory
**2003**, 110, 65–86. [Google Scholar] [CrossRef] - Sandholm, T.; Gilpin, A.; Conitzer, V. Mixed-Integer Programming Methods for Finding Nash Equilibria. In Proceedings of the National Conference on Artificial Intelligence (AAAI), Pittsburgh, PA, USA, 9–13 July 2005; pp. 495–501. [Google Scholar]
- Lemke, C.; Howson, J. Equilibrium points of bimatrix games. J. Soc. Ind. Appl. Math.
**1964**, 12, 413–423. [Google Scholar] [CrossRef] - Ganzfried, S.; Sandholm, T. Computing an approximate jam/fold equilibrium for 3-player no-limit Texas hold’em tournaments. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Estoril, Portugal, 12–16 May 2008. [Google Scholar]
- Ganzfried, S.; Sandholm, T. Computing equilibria in multiplayer stochastic games of imperfect information. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), Pasadena, CA, USA, 11–17 July 2009. [Google Scholar]
- Johanson, M.; Zinkevich, M.; Bowling, M. Computing Robust Counter-Strategies. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), Vancouver, BC, Canada, 3–6 December 2007; pp. 1128–1135. [Google Scholar]
- Ganzfried, S.; Sandholm, T. Game theory-based opponent modeling in large imperfect-information games. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Taipei, Taiwan, 2–6 May 2011. [Google Scholar]
- Abou Risk, N.; Szafron, D. Using Counterfactual Regret Minimization to Create Competitive Multiplayer Poker Agents. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), Toronto, ON, Canada, 10–14 May 2010. [Google Scholar]
- Gibson, R. Regret Minimization in Games and the Development of Champion Multiplayer Computer Poker-Playing Agents. Ph.D. Thesis, University of Alberta, Edmonton, AB, Canada, 2014. [Google Scholar]
- Zinkevich, M.; Bowling, M.; Johanson, M.; Piccione, C. Regret Minimization in Games with Incomplete Information. In Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS), Vancouver, BC, USA, 3–6 December 2007. [Google Scholar]
- Bowling, M.; Burch, N.; Johanson, M.; Tammelin, O. Heads-up Limit Hold’em Poker is Solved. Science
**2015**, 347, 145–149. [Google Scholar] [CrossRef] [PubMed] - Annual Computer Poker Competition. Available online: http://www.computerpokercompetition.org/ (accessed on 13 May 2018).
- Szafron, D.; Gibson, R.; Sturtevant, N. A Parameterized Family of Equilibrium Profiles for Three-Player Kuhn Poker. In Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), St. Paul, MN, USA, 6–10 May 2013. [Google Scholar]

1. |

**Table 1.**Parameter values used for our first Nash Equilibrium agent (NE1). The strategies fall at the “lower bound” of the space of “robust” Nash equilibrium strategies that receive the best worst-case payoff assuming the other agents follow one of the strategies given by the computed infinite equilibrium family. The values are for the free parameters in the infinite family of Nash equilibrium strategies. To define these parameters, ${a}_{jk}$, ${b}_{jk}$ and ${c}_{jk}$ denote the action probabilities for players ${P}_{1}$, ${P}_{2}$ and ${P}_{3}$, respectively, when holding card j and taking an aggressive action (Bet (B) or Call (C)) in situation k, where the betting situations are defined in Table 2.

${\mathit{P}}_{1}$ | ${\mathit{P}}_{2}$ | ${\mathit{P}}_{3}$ |
---|---|---|

${a}_{11}=0$ | ${b}_{11}=0$ | ${c}_{11}=0$ |

${a}_{21}=0$ | ${b}_{21}=0$ | ${c}_{21}=\frac{1}{2}$ |

${a}_{22}=0$ | ${b}_{22}=0$ | ${c}_{22}=0$ |

${a}_{23}=0$ | ${b}_{23}=0$ | ${c}_{23}=0$ |

${a}_{31}=0$ | ${b}_{31}=0$ | ${c}_{31}=0$ |

${a}_{32}=0$ | ${b}_{32}=0$ | ${c}_{32}=0$ |

${a}_{33}=\frac{1}{2}$ | ${b}_{33}=\frac{1}{2}$ | ${c}_{33}=\frac{1}{2}$ |

${a}_{34}=0$ | ${b}_{34}=0$ | ${c}_{34}=0$ |

${a}_{41}=0$ | ${b}_{41}=0$ | ${c}_{41}=1$ |

**Table 2.**Betting situations in three-player Kuhn poker. For each player and situation, the sequence of capital letters denotes the history of the betting so far. ‘K’ stands for the action of “check” (which passes the turn to the next agent and does not put additional money in the pot); ‘F’ stands for “Fold” (give up on the hand and forfeit the pot); and ‘B’ stands for “Bet” (put additional money in the pot, which opponents are forced to match to remain in the hand). For example, for row ${P}_{1}$ Situation 1, the history of actions so far is that Player 1 has checked, Player 2 has checked and Player 3 has bet, and it is now Player 1’s turn.

Situation | ${\mathit{P}}_{1}$ | ${\mathit{P}}_{3}$ | ${\mathit{P}}_{3}$ |
---|---|---|---|

1 | – | K | KK |

2 | KKB | B | KB |

3 | KBF | KKBF | BF |

4 | KBC | KKBC | BC |

**Table 3.**Parameter values used for our second Nash equilibrium agent (NE2). The strategies fall at the “upper bound” of the space of “robust” Nash equilibrium strategies that receive the best worst-case payoff assuming the other agents follow one of the strategies given by the computed infinite equilibrium family. The values are for the free parameters in the infinite family of Nash equilibrium strategies. To define these parameters, ${a}_{jk}$, ${b}_{jk}$ and ${c}_{jk}$ denote the action probabilities for players ${P}_{1}$, ${P}_{2}$ and ${P}_{3}$, respectively, when holding card j and taking an aggressive action (Bet (B) or Call (C)) in situation k, where the betting situations are defined in Table 2.

${\mathit{P}}_{1}$ | ${\mathit{P}}_{2}$ | ${\mathit{P}}_{3}$ |
---|---|---|

${a}_{11}=0$ | ${b}_{11}=\frac{1}{4}$ | ${c}_{11}=0$ |

${a}_{21}=0$ | ${b}_{21}=\frac{1}{4}$ | ${c}_{21}=\frac{1}{2}$ |

${a}_{22}=0$ | ${b}_{22}=0$ | ${c}_{22}=0$ |

${a}_{23}=0$ | ${b}_{23}=0$ | ${c}_{23}=0$ |

${a}_{31}=0$ | ${b}_{31}=0$ | ${c}_{31}=0$ |

${a}_{32}=0$ | ${b}_{32}=1$ | ${c}_{32}=0$ |

${a}_{33}=\frac{1}{2}$ | ${b}_{33}=\frac{7}{8}$ | ${c}_{33}=0$ |

${a}_{34}=0$ | ${b}_{34}=0$ | ${c}_{34}=1$ |

${a}_{41}=0$ | ${b}_{41}=1$ | ${c}_{41}=1$ |

**Table 4.**Experiments using Nash agents against class project agents. The values reported are the total overall payoff of the agent in dollars (assuming an ante of $1 and a bet size of $1), divided by 100,000.

Nash | A1 | A2 | A3 | A4 | A5 | A6 | A7 | A8 | A9 | A10 |
---|---|---|---|---|---|---|---|---|---|---|

2.81 (NE1) | 2.24 | 1.17 | 2.54 | −1.66 | 2.32 | 1.74 | −1.34 | −9.54 | −3.47 | 1.42 |

2.81 (NE2) | 2.25 | 1.18 | 2.54 | −1.65 | 2.32 | 1.74 | −1.34 | −9.56 | −3.48 | 1.42 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ganzfried, S.; Nowak, A.; Pinales, J.
Successful Nash Equilibrium Agent for a Three-Player Imperfect-Information Game. *Games* **2018**, *9*, 33.
https://doi.org/10.3390/g9020033

**AMA Style**

Ganzfried S, Nowak A, Pinales J.
Successful Nash Equilibrium Agent for a Three-Player Imperfect-Information Game. *Games*. 2018; 9(2):33.
https://doi.org/10.3390/g9020033

**Chicago/Turabian Style**

Ganzfried, Sam, Austin Nowak, and Joannier Pinales.
2018. "Successful Nash Equilibrium Agent for a Three-Player Imperfect-Information Game" *Games* 9, no. 2: 33.
https://doi.org/10.3390/g9020033