# From Nash Equilibria to Chain Recurrent Sets: An Algorithmic Solution Concept for Game Theory

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

**Our results.**We characterize the set of chain recurrent points for replicator dynamics for two classic and well-studied classes of games, zero-sum games and potential games (as well as variants thereof). For zero-sum games with fully mixed (i.e., interior) Nash equilibria, the set of chain recurrent points of replicator dynamics is equal to the whole state space (i.e., $\mathcal{R}$ is all (randomized) strategy profiles). For potential games, $\mathcal{R}$ is minimal coinciding with the set of equilibria/fixed points of the (replicator) dynamics. We discuss how our results are robust as they straightforwardly extend to natural variants of the above games, i.e., affine/network zero-sum games and weighted potential games. Thus, an interesting high level picture emerges. In games with strongly misaligned incentives, $\mathcal{R}$ tends to be large implying unpredictability of the dynamics, whereas in games with strongly aligned incentives $\mathcal{R}$ tends to be small implying that learning/adaptation is easier in collaborative environments.

**Zero-sum games.**In zero-sum games, we show that, if a fully mixed Nash equilibrium exists (like in Matching Pennies), then the set of chain recurrent states is the whole state space (Theorem 6). This seems to fly in the face of the conventional game theoretic wisdom that in a zero-sum game with a unique maxmin/Nash equilibrium, all natural, self-interested dynamics will “solve” the game by converging swiftly to the maxmin. It is well known that convergence is restored if one focuses not on behavior, but on the time-average of the players’ utilities, strategies [7]; however, time averages are a poor way to capture the actual system behavior.

**(Weighted) potential games.**On the contrary, for potential games, it is already well known that replicators (and many other dynamics) converge to equilibria (e.g., [5,8]). As the name suggests, natural dynamics for these games have a Lyapunov/potential function that strictly decreases as long as we are not at equilibrium. As we show, unsurprisingly, this remains true for the more expansive class of weighted potential games. One could naturally come to hypothesize that, in such systems, known in dynamical systems theory as gradient-like, the chain recurrent set must be equal to the set of equilibria. However, this is not the case as one can construct artificial counterexamples where the limit points of each trajectories have a unique equilibrium but the whole statespace is chain recurrent.

## 2. Related Work

**Follow-up work:**In follow-up work to our conference version paper [22], Mertikopoulos, Piliouras and Papadimitriou [18] showed how to generalize the proofs of recurrent behavior for replicator dynamics in (network) zero-sum games by Piliouras and Shamma [23] to more general dynamics, i.e., the class of continuous time variants of follow the regularized leader (FTRL) dynamics. Bailey and Piliouras [24] studied the discrete-time version of FTRL dynamics in (network) zero-sum games (including Multiplicative Weights Update) and showed how they diverge away from equilibrium and towards the boundary. Piliouras and Schulman [25] show how to prove periodicity for replicator dynamics in the cases of non-bilinear zero-sum games. Specifically, they consider games that arise from the competition between two teams of agents where all the members in the team have identical interests but where the two teams have opposing interests (play a constant sum game). Periodicity for replicator dynamics has also been established for triangle-network zero-sum games [26]. Recurrence arguments for zero-sum games can also be adapted in the case of dynamically evolving games [27]. FTRL dynamics can be adapted so that they converge in bilinear [28] (or even more general non-convex-concave [29]) saddle problems. Such algorithms can be used for training algorithms for generative adversarial neural networks (GANs). Finally, Balduzzi et al. show how to explicitly “correct” the cycles observed in dynamics of zero-sum games by leveraging connections to Hamiltonian dynamics and use these insights to design different algorithms for training GANs [30].

## 3. Preliminaries

#### 3.1. Game Theory

#### 3.2. Replicator Dynamics

**Remark**

**1.**

**Replicator dynamics as the “fluid limit” of MWU.**The connection to Multiplicative Weight Updates (MWU) [6] is of particular interest and hence it is worth reviewing briefly here. MWU is an online learning dynamics where the decision maker keeps updating a weight vector which can be thought informally as reflecting the agent’s confidence that the corresponding actions will perform well in the future. In every time period, the weights are updated multiplicatively (as the name suggests) ${w}_{{s}_{i}}^{t+1}\leftarrow {w}_{{s}_{i}}^{t}{(1+\u03f5)}^{{u}^{t}({s}_{i})}$. In the next time period, the agent chooses each action with probability proportional to its weight. As long as the learning rate $\u03f5$ is a small constant (or even better decreasing with time, e.g., at a rate $1/\sqrt{t}$), then it is well known that MWU has good performance guarantees in online (or even adversarial) environments, i.e., low regret [34]. When all agents in a game apply MWU with learning parameter $\u03f5$ (MWU($\u03f5$)), this defines a deterministic map from the space of weights (or after normalization from the space of probability distributions over actions) to the space of probability distributions over actions. If we define this map from the space of mixed strategy profiles to itself as $f(\u03f5),$ then the replicator vector field corresponds to the coefficient of the first order term in the Taylor expansion of f as a function of $\u03f5$. In other words, the replicator vector field is equal to ${\frac{\partial f}{\partial \u03f5}|}_{\u03f5=0}$, a first order “smooth” approximation of the expected motion of the discrete time map MWU($\u03f5$). This argument was first exploited in [8] to study MWU dynamics in potential games.

#### 3.3. Topology of Dynamical Systems

**Definition**

**1.**

- (i)
- $\varphi (t,\xb7):X\to X$ is a homeomorphism for each $t\in \mathrm{I}\phantom{\rule{-0.2em}{0ex}}\mathbf{R}$.
- (ii)
- $\varphi (s+t,\mathbf{x})=\varphi (s,(\varphi (t,\mathbf{x})))$ for all $s,t\in \mathrm{I}\phantom{\rule{-0.2em}{0ex}}\mathbf{R}$ and all $x\in X$.

**Definition**

**2.**

**Definition**

**3.**

**Definition**

**4.**

#### 3.3.1. Liouville’s Formula

#### 3.3.2. Poincaré’s Recurrence Theorem

**Theorem**

**1**

**.**Let $(X,\Sigma ,\mu )$ be a finite measure space and let $f:X\to X$ be a measure-preserving transformation. Then, for any $E\in \Sigma $, the set of those points x of E such that ${f}^{n}(x)\notin E$ for all $n>0$ has zero measure. That is, almost every point of E returns to E. In fact, almost every point returns infinitely often. Namely,

**Corollary**

**1.**

#### 3.3.3. Homeomorphisms and Conjugacy of Flows

#### 3.4. The Fundamental Theorem of Dynamical Systems

**Definition**

**5.**

**Definition**

**6.**

**Definition**

**7.**

**Definition**

**8.**

**Definition**

**9.**

- (i)
- $\gamma ({\varphi}^{t}(x))$ is a strictly decreasing function of t for all $x\in X\setminus \mathcal{R}({\varphi}^{t})$,
- (ii)
- for all $x,y\in \mathcal{R}({\varphi}^{t})$ the points x, y are chain equivalent with respect to ${\varphi}^{t}$ if and only if $\gamma (x)=\gamma (y)$,
- (iii)
- $\gamma (\mathcal{R}({\varphi}^{t}))$ is nowhere dense.

#### Alternatively, Equivalent Formulations of Chain Equivalence

**Definition**

**10.**

**Definition**

**11.**

**Theorem**

**3**

**.**If ${\varphi}^{t}$ is a flow on a compact metric space $(X,d)$ and $x,y\in X$, then the following statements are equivalent:

- (i)
- The points x and y are chain equivalent with respect to ${\varphi}^{t}$.
- (ii)
- For every $\u03f5>0$ and $T>0,$ there exists an $(\u03f5,1)$-chain$$({x}_{0},\cdots ,{x}_{n};{t}_{0},\cdots ,{t}_{n-1})$$$${t}_{0}+\cdots +{t}_{n-1}\ge T$$$$({y}_{0},\cdots ,{y}_{m};{s}_{0},\cdots ,{s}_{m-1})$$$${s}_{0}+\cdots +{s}_{m-1}\ge T.$$
- (iii)
- For every $\u03f5>0$, there exists an $(\u03f5,1)$-chain from x to y and an $(\u03f5,1)$-chain from y to x.
- (iv)
- The points x and y are chain equivalent with respect to ${\varphi}^{1}$.

#### 3.5. Chain Components

**Definition**

**12.**

**Theorem**

**4**

**Definition**

**13.**

**Theorem**

**5**

**.**Every chain component of a flow on a compact metric space is closed, connected, and invariant with respect of the flow. Moreover,

- Every chain component of a flow on a metric space is chain transitive with respect to the flow.
- Every chain transitive set with respect to a flow on a metric space is a subset of a unique chain component of the flow.
- If A and B are chain transitive with respect to a flow on a metric space, $A\subset B$ and C is the unique chain component containing A, then $B\subset C$.

## 4. Chain Recurrent Sets for Zero-Sum Games

**Lemma**

**1**

**.**Let φ denote the flow of replicator dynamics when applied to a zero sum game with a fully mixed Nash equilibrium $q=({q}_{1},{q}_{2})$. Given any (interior) starting point $x(0)=({x}_{1}(0),{x}_{2}(0))$ then the sum of the KL-divergences between each agent’s mixed Nash equilibrium ${q}_{i}$ and his evolving strategy ${x}_{i}(t)$ is time invariant. Equivalently, ${D}_{\mathrm{KL}}({q}_{1}\parallel {x}_{1}(t))+{D}_{\mathrm{KL}}({q}_{2}\parallel {x}_{2}(t))={D}_{\mathrm{KL}}({q}_{1}\parallel {x}_{1}(0))+{D}_{\mathrm{KL}}({q}_{2}\parallel {x}_{2}(0))$ for all t.

**Lemma**

**2.**

**Theorem**

**6.**

**Proof.**

## 5. Chain Recurrent Sets for Weighted Potential Games

**Theorem**

**7**

**.**The chain recurrent set of a continuous (semi)flow on an arbitrary metric space is the same as the chain recurrent set of its time-one map.

**Theorem**

**8.**

**Proof.**

## 6. Discussion and Open Questions

**The structure of the Chain Recurrent Set (CRS) and the Chain Components (CCs).**A game may have many chain components (for example, the coordination games in Figure 3 has five). It is not hard to see that the chain components can be arranged as vertices of a directed acyclic graph, where directed edges signify possible transitions after an infinitesimal jump; for the coordination games in Figure 3, this directed acyclic graph (DAG) has two sinks (the pure Nash equilibria), two sources (the other two pure profiles), and a node of degree 4 (the mixed Nash equilibrium). Identifying this DAG is tantamount to analyzing the game, the generalization of finding its Nash equilibria. Understanding this fundamental structure in games of interest is an important challenge.**Price of Anarchy through Chain Recurrence.**We can define a natural distribution over the sink chain components (CCs) of a game, namely, assign to each sink CC the probability that a trajectory started at a (say, uniformly) random point of the state space will end up, perhaps after infinitesimal jumps, at the CC. This distribution, together with the CC’s expected utility, yield a new and productive definition of the average price of anarchy in a game, as well as a methodology for calculating it (see, for example, [46]).**Inside a Chain Component.**Equilibria and limit cycles are the simplest forms of a chain components, in the sense that no “jumps” are necessary for going from one state in the component to another. In Matching Pennies, in contrast, $O(\frac{1}{\u03f5}),$ many $\u03f5$-jumps are needed to reach the Nash equilibrium, starting from a pure strategy profile. What is the possible range of this form of complexity of a CC?**Complexity.**There are several intriguing complexity questions posed by this concept. What is the complexity of determining, given a game and two strategy profiles, whether they belong to the same chain component (CC)? What is the complexity of finding a point in a sink CC? What is the speed of convergence to a CC?**Multiplicative Weights Update and Discrete-time Dynamics through Chain Components.**We have explicitly discussed the connection between replicator dynamics and MWU. At the same time, it has recently been shown that for large step-sizes $\u03f5$ MWU($\u03f5$) can behave chaotically even in two agent two strategy coordination/potential games [17]. Is it possible to understand this chaotic behavior through the lens of CCs? Moreover, can we understand and predict the bifurcation from the convergent behavior of replicator dynamics (i.e., MWU($\u03f5$) with $\u03f5\to 0$) to the chaotic behavior of MWU($\u03f5$) with large step-size $\u03f5$?**Information Geometry, Social Welfare and the Fundamental Theorem of Dynamical Systems.**In the case of zero-sum games, each point of a given replicator trajectory lies at a fixed KL-divergence from the Nash equilibrium. Similar KL-divergence invariant properties also apply in the case of (network) coordination games [46]. It is not known whether any (information theoretic) invariant properties applies, e.g., to a general two person game for replicator dynamics. The fundamental theorem of dynamical systems shows the existence of a complete Lyapunov function that is invariant on the chain recurrence set (and hence on each chain component) but strictly decreases outside this set. Can we express this function for the replicator flow in a general two-person game as a combination of information theoretic properties (e.g., KL-divergences) and game theoretic properties (e.g., the sum of utilities of all agents)?

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A Information Theory

## Appendix B Missing Proofs of Section 4

**Lemma**

**A1.**

**Proof.**

**Lemma**

**A2.**

**Proof.**

## References

- Daskalakis, C.; Goldberg, P.W.; Papadimitriou, C.H. The Complexity of Computing a Nash Equilibrium; ACM Press: New York, NY, USA, 2006; pp. 71–78. [Google Scholar]
- Demichelis, S.; Ritzberger, K. From evolutionary to strategic stability. J. Econ. Theory
**2003**, 113, 51–75. [Google Scholar] [CrossRef] - Benaïm, M.; Hofbauer, J.; Sorin, S. Perturbations of set-valued dynamical systems, with applications to game theory. Dyn. Games Appl.
**2012**, 2, 195–205. [Google Scholar] [CrossRef] - Conley, C. Isolated invariant sets and the Morse index; CBMS Regional Conference Series in Mathematics, 38. Am. Math. Soc. Provid. RI
**1978**, 16. [Google Scholar] - Hofbauer, J.; Sigmund, K. Evolutionary Games and Population Dynamics; Cambridge University Press: Cambridge, UK, 1998. [Google Scholar]
- Arora, S.; Hazan, E.; Kale, S. The multiplicative weights update method: A meta algorithm and applications. Theory Comput.
**2012**, 8, 121–164. [Google Scholar] [CrossRef] - Fudenberg, D.; Levine, D.K. The Theory of Learning in Games; The MIT Press: Cambridge, MA, USA, 1998. [Google Scholar]
- Kleinberg, R.; Piliouras, G.; Tardos, É. Multiplicative Updates Outperform Generic No-Regret Learning in Congestion Games. In Proceedings of the Forty-First Annual ACM Symposium on Theory of Computing (STOC), Bethesda, MD, USA, 31 May–2 June 2009. [Google Scholar]
- Brown, G. Iterative Solutions of Games by Fictitious Play. In Activity Analysis of Production and Allocation; Koopmans, T.C., Ed.; Wiley: New York, NY, USA, 1951. [Google Scholar]
- Robinson, J. An Iterative Method of Solving a Game. Ann. Math.
**1951**, 54, 296–301. [Google Scholar] [CrossRef] - Shapley, L. Some topics in two-person games. Adv. Game Theory
**1964**, 52, 1–29. [Google Scholar] - Daskalakis, C.; Frongillo, R.; Papadimitriou, C.H.; Pierrakos, G.; Valiant, G. On learning algorithms for Nash equilibria. In Proceedings of the International Symposium on Algorithmic Game Theory, Athens, Greece, 18–20 October 2010; pp. 114–125. [Google Scholar]
- Kleinberg, R.; Ligett, K.; Piliouras, G.; Tardos, É. Beyond the Nash Equilibrium Barrier. In Proceedings of the Symposium on Innovations in Computer Science (ICS), Beijing, China, 7–9 January 2011; pp. 125–140. [Google Scholar]
- Ligett, K.; Piliouras, G. Beating the best Nash without regret. ACM SIGecom Exchang.
**2011**, 10, 23–26. [Google Scholar][Green Version] - Piliouras, G.; Nieto-Granda, C.; Christensen, H.I.; Shamma, J.S. Persistent Patterns: Multi-agent Learning Beyond Equilibrium and Utility. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems, Paris, France, 5–9 May 2014; pp. 181–188. [Google Scholar]
- Mehta, R.; Panageas, I.; Piliouras, G.; Yazdanbod, S. The computational complexity of genetic diversity. In LIPIcs-Leibniz International Proceedings in Informatics (ESA Conference); Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik: Wadern, Germany, 2016; Volume 57. [Google Scholar]
- Palaiopanos, G.; Panageas, I.; Piliouras, G. Multiplicative weights update with constant step-size in congestion games: Convergence, limit cycles and chaos. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2017; pp. 5872–5882. [Google Scholar]
- Mertikopoulos, P.; Papadimitriou, C.; Piliouras, G. Cycles in adversarial regularized learning. In Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms, New Orleans, LA, USA, 7–10 January 2018; pp. 2703–2717. [Google Scholar]
- Sandholm, W.H. Population Games and Evolutionary Dynamics; MIT Press: Cambridge, MA, USA, 2010. [Google Scholar]
- Börgers, T.; Sarin, R. Learning Through Reinforcement and Replicator Dynamics. J. Econ. Theory
**1997**, 77, 1–14. [Google Scholar] [CrossRef] - Chen, X.; Deng, X.; Teng, S.H. Settling the Complexity of Computing Two-player Nash Equilibria. JACM
**2009**, 56, 14. [Google Scholar] [CrossRef] - Papadimitriou, C.; Piliouras, G. From Nash equilibria to chain recurrent sets: Solution concepts and topology. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, Cambridge, MA, USA, 14–17 January 2016; pp. 227–235. [Google Scholar]
- Piliouras, G.; Shamma, J.S. Optimization despite chaos: Convex relaxations to complex limit sets via Poincaré recurrence. In Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, Portland, Oregon, USA, 5–7 January 2014; pp. 861–873. [Google Scholar]
- Bailey, J.P.; Piliouras, G. Multiplicative Weights Update in Zero-Sum Games. In Proceedings of the 2018 ACM Conference on Economics and Computation, Ithaca, NY, USA, 18–22 June 2018; pp. 321–338. [Google Scholar]
- Piliouras, G.; Schulman, L.J. Learning Dynamics and the Co-Evolution of Competing Sexual Species. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science (ITCS); ACM: New York, NY, USA, 2018. [Google Scholar]
- Nagarajan, S.G.; Mohamed, S.; Piliouras, G. Three Body Problems in Evolutionary Game Dynamics: Convergence, Periodicity and Limit Cycles. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, Stockholm, Sweden, 10–15 July 2018; pp. 685–693. [Google Scholar]
- Mai, T.; Mihail, M.; Panageas, I.; Ratcliff, W.; Vazirani, V.; Yunker, P. Cycles in Zero-Sum Differential Games and Biological Diversity. In Proceedings of the 2018 ACM Conference on Economics and Computation, Ithaca, NY, USA, 18–22 June 2018; pp. 339–350. [Google Scholar]
- Daskalakis, C.; Ilyas, A.; Syrgkanis, V.; Zeng, H. Training GANs with Optimism. In Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Mertikopoulos, P.; Zenati, H.; Lecouat, B.; Foo, C.S.; Chandrasekhar, V.; Piliouras, G. Mirror descent in saddle-point problems: Going the extra (gradient) mile. arXiv, 2018; arXiv:1807.02629. [Google Scholar]
- Balduzzi, D.; Racaniere, S.; Martens, J.; Foerster, J.; Tuyls, K.; Graepel, T. The Mechanics of n-Player Differentiable Games. In Proceedings of the International Conference on Machine Learning (ICML), Stockholm, Sweden, 10–15 July 2018. [Google Scholar]
- Taylor, P.D.; Jonker, L.B. Evolutionary stable strategies and game dynamics. Math. Biosci.
**1978**, 40, 145–156. [Google Scholar] [CrossRef] - Schuster, P.; Sigmund, K. Replicator dynamics. J. Theor. Biol.
**1983**, 100, 533–538. [Google Scholar] [CrossRef] - Losert, V.; Akin, E. Dynamics of Games and Genes: Discrete Versus Continuous Time. J. Math. Biol.
**1983**, 17, 241–251. [Google Scholar] [CrossRef] - Young, H.P. Strategic Learning and Its Limits; Oxford University Press: Oxford, UK, 2004. [Google Scholar]
- Karev, G.P. Principle of Minimum Discrimination Information and Replica Dynamics. Entropy
**2010**, 12, 1673–1695. [Google Scholar] [CrossRef][Green Version] - Weibull, J.W. Evolutionary Game Theory; MIT Press: Cambridge, MA, USA, 1997. [Google Scholar]
- Bhatia, N.P.; Szegö, G.P. Stability Theory of Dynamical Systems; Springer: Berlin, Germany, 1970. [Google Scholar]
- Alongi, J.M.; Nelson, G.S. Recurrence and Topology; American Mathematical Society: Providence, RI, USA, 2007; Volume 85. [Google Scholar]
- Poincaré, H. Sur le problème des trois corps et les équations de la dynamique. Acta Math.
**1890**, 13, 5–7. (In French) [Google Scholar] [CrossRef] - Barreira, L. Poincare recurrence: Old and new. In XIVth International Congress on Mathematical Physics; World Scientific: Singapore, 2006; pp. 415–422. [Google Scholar]
- Meiss, J. Differential Dynamical Systems; SIAM: Philadelphia, PA, USA, 2007. [Google Scholar]
- Hurley, M. Chain Recurrence, Semiflows, and Gradients. J. Dyn. Differ. Equat.
**1995**, 7, 437–456. [Google Scholar] [CrossRef] - Nagel, R. Unraveling in guessing games: An experimental study. Am. Econ. Rev.
**1995**, 85, 1313–1326. [Google Scholar] - Attanasi, G.; García-Gallego, A.; Georgantzís, N.; Montesano, A. Bargaining over strategies of non-cooperative games. Games
**2015**, 6, 273–298. [Google Scholar] [CrossRef][Green Version] - Bloomfield, R. Learning a mixed strategy equilibrium in the laboratory. J. Econ. Behav. Organ.
**1994**, 25, 411–436. [Google Scholar] [CrossRef] - Panageas, I.; Piliouras, G. Average Case Performance of Replicator Dynamics in Potential Games via Computing Regions of Attraction. In Proceedings of the 2016 ACM Conference on Economics and Computation, Maastricht, The Netherlands, 24–28 July 2016; pp. 703–720. [Google Scholar]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: New York, NY, USA, 1991. [Google Scholar]

**Figure 1.**Replicator trajectories in the Matching Pennies game (

**left**) and in a different zero-sum game with a fully mixed Nash equilibrium (

**right**). Each point encodes the probability assigned by the agents to their first strategy. In Matching Pennies at Nash equilibrium, each agent chooses the first strategy with probability $1/2$. In the second game at Nash equilibrium, each agent chooses the first strategy with probability $2/3$. All interior points lie on periodic trajectories.

**Figure 2.**Replicator trajectories of the first (resp. second) agent in the Rock–Paper–Scissors game. Each color corresponds to a different trajectory/initial condition. The three different initial conditions are not chosen to enforce any type of symmetric behavior for the two agents. In the format $({x}_{Rock},{x}_{Paper},{x}_{Scissors},{y}_{Rock},{y}_{Paper},{y}_{Scissors}),$ the initial conditions are the following: Black (0.5, 0.01, 0.49, 0.5, 0.25, 0.25), Blue (0.1, 0.2, 0.7, 0.8, 0.04, 0.16), and Purple (0.32, 0.3, 0.38, 0.33, 0.32, 0.35). As expected from our analysis, the trajectories for both agents are recurrent. Moreover, the purple trajectory whose initial condition is chosen to be "close" to the Nash equilibrium (in K-L divergence) stays close to it.

**Figure 3.**Replicator trajectories in a doubly symmetric partnership, coordination game (

**left**) and in a different coordination game with a fully mixed Nash equilibrium (

**right**). Each point encodes the probability assigned by the agents to their first strategy. In the first game at Nash equilibrium, each agent chooses the first strategy with probability $1/2$. In the second game at Nash equilibrium, each agent chooses the first strategy with probability $2/3$. All trajectories converge to equilibria.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Papadimitriou, C.; Piliouras, G. From Nash Equilibria to Chain Recurrent Sets: An Algorithmic Solution Concept for Game Theory. *Entropy* **2018**, *20*, 782.
https://doi.org/10.3390/e20100782

**AMA Style**

Papadimitriou C, Piliouras G. From Nash Equilibria to Chain Recurrent Sets: An Algorithmic Solution Concept for Game Theory. *Entropy*. 2018; 20(10):782.
https://doi.org/10.3390/e20100782

**Chicago/Turabian Style**

Papadimitriou, Christos, and Georgios Piliouras. 2018. "From Nash Equilibria to Chain Recurrent Sets: An Algorithmic Solution Concept for Game Theory" *Entropy* 20, no. 10: 782.
https://doi.org/10.3390/e20100782