Mean-Payoff Games with ω-Regular Specifications

Abstract

Multi-player mean-payoff games are a natural formalism for modelling the behaviour of concurrent and multi-agent systems with self-interested players. Players in such a game traverse a graph, while attempting to maximise a (mean-)payoff function that depends on the play generated. As with all games, the equilibria that could arise may have undesirable properties. However, as system designers, we typically wish to ensure that equilibria in such systems correspond to desirable system behaviours, for example, satisfying certain safety or liveness properties. One natural way to do this would be to specify such desirable properties using temporal logic. Unfortunately, the use of temporal logic specifications causes game theoretic verification problems to have very high computational complexity. To address this issue, we consider $\omega$-regular specifications. These offer a concise and intuitive way of specifying system behaviours with a comparatively low computational overhead. The main results of this work are characterisation and complexity bounds for the problem of determining if there are equilibria that satisfy a given $\omega$-regular specification in a multi-player mean-payoff game in a number of computationally relevant game-theoretic settings.

1. Introduction

Modelling concurrent and multi-agent systems such as games in which players interact by taking actions in pursuit of their preferences is an increasingly common approach in both formal verification and artificial intelligence [1,2,3]. One widely adopted semantic framework for modelling such systems is that of concurrent game structures [1]. Such structures capture the dynamics of a system—the actions that agents/players can perform, and the effects of these actions. On top of this framework, we can introduce additional structure to represent each player’s preferences over the possible paths of the system. Several approaches have been proposed for this purpose. One very natural method involves assigning a weight to every state of the game, and then considering each player’s mean-payoff over generated paths: a player prefers paths that maximise their mean-payoff [4,5,6]. These games are effective in modelling resource-bounded reactive systems, as well as any scenario with multiple agents and quantitative features. Under the assumption that each agent in the system is acting rationally, concepts from game theory offer a natural framework for understanding its possible behaviours [7]. This approach is (relatively) computationally tractable [5], expressive enough to capture applications of interest, and has been receiving increasing attention recently [8]. As such, equilibria for multi-player games with mean-payoff objectives have been studied, and the computation of Nash equilibria in such games shown to be NP-complete [5].

However, it is well-known in the game theory literature that equilibria may have undesirable properties. In the context of our setting, for example, an equilibrium may visit dangerous system states, or lead to a deadlock. Thus, one may also want to check if there exist equilibria which satisfy some additional desirable computational properties associated with the game. This decision problem—that is, determining whether a given formal specification is satisfied on some (or every) equilibrium of a given multi-agent system modelled as a multi-player game—is known as Rational Verification [9,10].

Previous approaches to rational verification have borrowed their methodology from temporal logic model checking, appealing to logics such as Linear Temporal Logic (LTL) [11] and Computation Tree Logic (CTL) [12]. However, since rational verification subsumes automated synthesis, the use of temporal logic specifications introduces high computational complexity [13]. To mitigate this problem, one might use fragments of temporal logic with lower complexity (e.g., GR(1) (generalised Reactivity(1)) formulae [14,15]), but in this work we adopt a different approach. Taking inspiration from automata theory, and in particular from [16], we consider system specifications given by a formal language for expressing ω-regular specifications, defined in terms of those states in the system that are visited infinitely often. With this approach, the complexity of the main game-theoretic decision problems is considerably lower than is the case with temporal logic specifications.

In this paper, we offer the following main contributions. We begin by introducing a language for $\omega$-regular specifications and demonstrate that they form a natural construct for representing properties of concurrent games. In Section 3, we study multi-player mean-payoff games with $\omega$-regular specifications in the non-cooperative setting [7], and consider the natural decision problems relating to these games and their Nash equilibria. Following this, in Section 4 we consider a cooperative solution concept derived from the core [7,17]. Finally, in Section 5 we look at reactive module games [18] as a way of succinctly representing systems, and investigate how the use of this representation affects our complexity results. We conclude with a discussion of related work in Section 6, before offering a glossary of terms, acronyms and notation used within the paper.

2. Models, Games, and Specifications

2.1. Games

A concurrent game structure [1] is a tuple,

$$M=(\mathrm{Ag},\mathrm{St},s^0,{\left({\mathrm{Ac}}_i\right)}_{i\in \mathrm{Ag}},\mathrm{tr}),$$

where,

• Ag and St are finite, non-empty sets of agents and system states, respectively, where $s^0\in \mathrm{St}$ is an initial state;
• For each $i\in \mathrm{Ag}$, ${\mathrm{Ac}}_i$ is a set of actions available to agent i;
• $\mathrm{tr}:\mathrm{St}\times {\mathrm{Ac}}_1\times \cdots \times {\mathrm{Ac}}_{\left|\mathrm{Ag}\right|}\to \mathrm{St}$ is a transition function.

We define the size of M to be $\left|\mathrm{St}\right|·{\left|\mathrm{Ac}\right|}^{\left|\mathrm{Ag}\right|}$.

Concurrent games are played as follows. The game begins in state $s^0$, and each player $i\in \mathrm{Ag}$ simultaneously picks an action ${\mathrm{ac}}_i^0\in {\mathrm{Ac}}_i$. The game then transitions to a new state, $s^1=\mathrm{tr}(s^0,{\mathrm{ac}}_1^0,\dots ,{\mathrm{ac}}_{\left|\mathrm{Ag}\right|}^0)$, and this process repeats. Thus, the nth state visited is $s^n=\mathrm{tr}(s^{n-1},{\mathrm{ac}}_1^{n-1},\dots ,{\mathrm{ac}}_{\left|\mathrm{Ag}\right|}^{n-1})$. Since the transition function is deterministic, a play of a game will be an infinite sequence of states, $\pi :\mathbb{N}\to \mathrm{St}$. We call such a sequence of states a path. Typically, we index paths with square brackets, i.e., the kth state visited in the path $\pi$ is denoted $\pi \left[k\right]$, and we also use slice notation to denote prefix, suffixes and fragments of paths. That is, we use $\pi [m..n]$ to mean $\pi \left[m\right]\pi [m+1]\dots \pi [n-1]$, $\pi [..n]$ for $\pi \left[0\right]\pi \left[1\right]\dots \pi [n-1]$ and $\pi [m..]$ for $\pi \left[m\right]\pi [m+1]\dots$. Now, consider a path $\pi$. We say that $\pi$ visits a state s if there is some $k\in \mathbb{N}$ such that $\pi \left[k\right]=s$. Since there are only finitely many states, some must be visited infinitely often. Furthermore, unless all states are visited infinitely often, there will also exist some set of states that are visited only finitely often. Thus, given a path $\pi$, we can define the following two sets, which one can use to define objectives over paths: $\mathrm{Inf}\left(\pi \right)=\{s\in \mathrm{St}\mid \pi \mathrm{visits}s\mathrm{infinitely}\mathrm{often}\}$ and its complement $\mathrm{Fin}\left(\pi \right)=\mathrm{St}\backslash \mathrm{Inf}\left(\pi \right)$.

2.2. Strategies

In order to describe how each player plays the game, we need to introduce the concept of a strategy. Mathematically, a strategy for a given player i can be understood as a function, ${\sigma }_i:{\mathrm{St}}^{+}\to {\mathrm{Ac}}_i$, which maps sequences, or histories, of states into a chosen action for player i. A strategy profile is a vector of strategies, $\overrightarrow{\sigma }=({\sigma }_1,\dots ,{\sigma }_{\left|\mathrm{Ag}\right|})$, one for each player. The set of strategies for a given player i is denoted by ${\mathsf{\Sigma }}_i$ and the set of strategy profiles is denoted by $\mathsf{\Sigma }$. If we have a strategy profile $\overrightarrow{\sigma }=({\sigma }_1,\dots ,{\sigma }_{\left|\mathrm{Ag}\right|})$, we use the notation ${\overrightarrow{\sigma }}_{-i}$ to denote the vector $({\sigma }_1,\dots ,{\sigma }_{i-1},{\sigma }_{i+1},\dots ,{\sigma }_{\left|\mathrm{Ag}\right|})$ and $({\overrightarrow{\sigma }}_{-i},{\sigma }_i^{\prime })$ to denote $({\sigma }_1,\dots ,{\sigma }_{i-1},{\sigma }_i^{\prime },{\sigma }_{i+1},\dots ,{\sigma }_{\left|\mathrm{Ag}\right|})$. Finally, we write ${\mathsf{\Sigma }}_{-i}$ as shorthand for ${\mathsf{\Sigma }}_1\times \cdots \times {\mathsf{\Sigma }}_{i-1}\times {\mathsf{\Sigma }}_{i+1}\times \cdots \times {\mathsf{\Sigma }}_{\left|\mathrm{Ag}\right|}$. All of these notations can also be generalised in the obvious way to coalitions of agents, $C\subseteq \mathrm{Ag}$. A strategy profile $\overrightarrow{\sigma }\in \mathsf{\Sigma }$ together with a state s will induce a unique path, which, with a small abuse of notation, we will denote by $\pi (\overrightarrow{\sigma },s):\mathbb{N}\to \mathrm{St}$, as well as an infinite sequence of actions $\overrightarrow{\mathrm{ac}}:\mathbb{N}\to \mathrm{Ac}$, with $\mathrm{Ac}={\mathrm{Ac}}_1\times \cdots \times {\mathrm{Ac}}_{\left|\mathrm{Ag}\right|}$. These paths are obtained in the following way. Starting from s, each player plays ${\mathrm{ac}}_i^0={\sigma }_i\left(s\right)$. This transforms the game into a new state, given by $s^1=\mathrm{tr}(s,{\mathrm{ac}}_1^0,\dots ,{\mathrm{ac}}_{\left|\mathrm{Ag}\right|}^0)$. Each player then plays ${\mathrm{ac}}_i^1={\sigma }_i\left(s{s}^1\right)$, and this process repeats infinitely often. Typically, we are interested in paths that begin in the game’s start state, $s^0$, and we write $\pi \left(\overrightarrow{\sigma }\right)$ as shorthand for the infinite path $\pi (\overrightarrow{\sigma },s^0)$.

When considering computational questions surrounding concurrent games, it is useful to work with a finite representation of strategies. We use two such representations: finite-memory strategies and memoryless strategies. A finite-memory strategy is a finite state machine with output: for player i, a finite-memory strategy ${\sigma }_i$ is a four-tuple, $(Q_i,q_i^0,{\delta }_i,{\tau }_i)$, where $Q_i$ is a finite, non-empty set of internal states with $q_i^0\in Q_i$ an initial state, ${\delta }_i:Q_i\times {\mathrm{Ac}}_1\times \cdots \times {\mathrm{Ac}}_{\left|\mathrm{Ag}\right|}\to Q_i$ is an internal transition function and ${\tau }_i:Q_i\to {\mathrm{Ac}}_i$ is an action function. This strategy operates by starting in the initial state, and for each state it is in, producing an action according to ${\tau }_i$, looking at what actions have been taken by everyone, and then moving to a new internal state as prescribed by $\delta$. Now, it is not hard to see that, if we have multiple finite-memory strategies playing against each other, then the play they generate will be periodic: the play must eventually revisit some configuration, and at this point, the game will start to repeat itself. More precisely, any play generated by a collection of finite-memory strategies will consist of a finite non-repeating sequence of states, followed by a finite sequence that repeats infinitely often. Because such a play will be periodic, we can write the path induced on the concurrent game structure as $\pi =\pi [..k]{\pi [k..m]}^{\omega }$, for some $k,m\in \mathbb{N}$ with $0\le k<m$.

Finally, a memoryless strategy is a strategy that depends only on the state the player is currently in. Thus, memoryless strategies can be expressed as functions ${\sigma }_i:\mathrm{St}\to {\mathrm{Ac}}_i$, which simply map states to actions; such functions can be directly implemented as lookup tables, in space $O\left(\right|\mathrm{St}\left|\right)$. Note that memoryless strategies can be encoded as finite-memory strategies, and that finite-memory strategies are a special case of arbitrary strategies ${\sigma }_i:{\mathrm{St}}^{+}\to {\mathrm{Ac}}_i$. Whilst we will work with finite-memory and memoryless strategies, we will use arbitrary strategies by default, unless otherwise stated.

2.3. Mean-Payoff Games

A two-player, mean-payoff game [4,6], is defined by a tuple:

$$G=(V_1,V_2,v^0,E,w).$$

Here $V_1$ and $V_2$ are disjoint, with $v^0\in V_1$. Additionally, we have: $E\subseteq (V_1\times V_2)\cup (V_2\times V_1)$ and w is a function with signature $w:E\to \mathbb{Z}$. There are two players, 1 and 2, and the vertices in $V_i$ should be thought of as those that player i controls—informally, the players take turns choosing edges to traverse, each of which is weighted. Player 1 is trying to maximise the average of the weights encountered, whilst player 2 is trying to minimise it. Formally, the game begins in state $v^0\in V_1$ and player 1 chooses an edge $(v^0,v^1)\in E\cap (V_1\times V_2)$. Then player 2 chooses an edge $(v^1,v^2)\in E\cap (V_2\times V_1)$ and this process repeats indefinitely. This induces a sequence of weights, $\overrightarrow{w}=w\left((v^0,v^1)\right),w\left((v^1,v^2)\right),\dots$, and player 1 (respectively, 2) chooses edges to try and maximise (respectively, minimise) the mean-payoff of $\overrightarrow{w}$, denoted $\mathsf{mp}\left(\overrightarrow{w}\right)$, where for $\beta \in {\mathbb{Z}}^{\omega }$, we have:

$$\mathsf{mp}\left(\beta \right)=\underset{n\to \infty }{lim\; inf}\frac{1}{n}\sum _{i=0}^{n-1}{\beta }_i.$$

There are two keys facts about two-player, mean-payoff games that we shall use without comment throughout. The first is that memoryless strategies suffice for both players to act optimally (i.e., achieve their maximum payoff) [4]. The second is that every game has a value (i.e., a payoff that player 1 can achieve regardless of what player 2 plays) and determining if a game’s value is equal to v lies in $\mathrm{NP}\cap \mathrm{co}-\mathrm{NP}$ [6]. In particular, given a game, determining its value can be seen as a problem that lies within TFNP [19].

Extending two-player, mean-payoff games to multiple players, a multi-player mean-payoff game [5] is given by a tuple,

$$G=(M,{\left(w_i\right)}_{i\in \mathrm{Ag}}),$$

where M is a concurrent game structure and for each agent $i\in \mathrm{Ag}$, $w_i:\mathrm{St}\to \mathbb{Z}$ is a weight function. In a multi-player mean-payoff game, a path $\pi =s^0{s}^1\dots$ induces an infinite sequence of weights for each player, $w_i\left(s^0\right)w_i\left(s^1\right)\dots$ We denote this sequence by $w_i\left(\pi \right)$. Under a given path, $\pi$, a player’s payoff is given by $\mathsf{mp}\left(w_i\left(\pi \right)\right)$. For notational convenience, we will write ${\mathsf{pay}}_i\left(\pi \right)$ for $\mathsf{mp}\left(w_i\left(\pi \right)\right)$. We can then define a preference relation over paths for each player as follows: $\pi {⪰}_i{\pi }^{\prime }\mathrm{if}\mathrm{and}\mathrm{only}\mathrm{if}{\mathsf{pay}}_i\left(\pi \right)\ge {\mathsf{pay}}_i\left({\pi }^{\prime }\right)$. We also write $\pi {\succ }_i{\pi }^{\prime }$ if $\pi {⪰}_i{\pi }^{\prime }$ and not ${\pi }^{\prime }{⪰}_i\pi$. Note that, since strategy profiles $\overrightarrow{\sigma }$ induce unique plays $\pi \left(\overrightarrow{\sigma }\right)$, we can lift preference relations from plays to strategy profiles, for example writing $\overrightarrow{{\sigma }_1}{⪰}_i\overrightarrow{{\sigma }_2}$ as a shorthand for $\pi \left(\overrightarrow{{\sigma }_1}\right){⪰}_i\pi \left(\overrightarrow{{\sigma }_2}\right)$.

In what follows, we refer to multi-player, mean-payoff games simply as mean-payoff games, and refer to two-player, mean-payoff games explicitly as such.

2.4. Solution Concepts

To analyse our games, we make use of solution concepts from both the non-cooperative and cooperative game theory literatures. With respect to non-cooperative solution concepts, a strategy profile $\overrightarrow{\sigma }$ is said to be a Nash equilibrium [20,21] if for all players i and strategies ${\sigma }_i^{\prime }$, we have $\overrightarrow{\sigma }{⪰}_i({\overrightarrow{\sigma }}_{-i},{\sigma }_i^{\prime })$. Informally, a Nash equilibrium is a strategy profile from which no player has any incentive to unilaterally deviate. In addition to Nash equilibrium, we also consider the cooperative solution concept known as the core [17,22]. While Nash equilibria are profiles that are resistant to unilateral deviations, the core consists of profiles that are resistant to those deviations by coalitions of agents, where every member of the coalition is better off, regardless of what the rest of the agents do. Formally, we say that a strategy profile, $\overrightarrow{\sigma }$, is in the core if for all coalitions $C\subseteq \mathrm{Ag}$, and strategy vectors ${\overrightarrow{\sigma }}_C^{\prime }$, then there is some complementary strategy vector ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$ such that $\overrightarrow{\sigma }{⪰}_i({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })$, for some $i\in C$. Given a game G, let $\mathrm{NE}\left(G\right)$ denote the set of Nash equilibrium strategy profiles of G, and let $\mathrm{CORE}\left(G\right)$ denote the set of strategy profiles in the core of G.

It is worth noting that if a strategy profile is not a Nash equilibrium, then at least one player can deviate and be better off, under the assumption that the remainder of the players do not change their actions. However, if a strategy profile is not in the core, then some coalition can deviate and become better off, regardless of what the other players do. Thus, the core should not be confused with the solution concept of strong Nash equilibrium: a strategy profile that is stable under multilateral deviations, assuming the remainder of the players ‘stay put’ [23,24]. We will not use strong Nash equilibria in this work, and only mention the concept in order to emphasise how strong Nash equilibria are different from the core.

2.5. $\omega$-Regular Specifications

In [16], Boolean combinations of atoms of the form $\mathrm{Inf}\left(F\right)$ are used to describe acceptance conditions of arbitrary $\omega$-automata. We use this approach to specify system properties for our games. Formally, the language of $\omega$-regular specifications, $\alpha$, is defined by the following grammar:

$$\alpha :=\mathrm{Inf}\left(F\right)\mid \neg \alpha \mid \alpha \wedge \alpha ,$$

where F ranges over subsets of St. For notational convenience, we write $\mathrm{Fin}\left(F\right)$ as shorthand for $\neg \mathrm{Inf}\left(F\right)$, $\mathrm{Inf}\left(\overline{F}\right)$ for $\mathrm{Inf}(\mathrm{St}\backslash F)$ and we define disjunction, $·\vee ·$, implication $·\to ·$ and bi-implication $·\leftrightarrow ·$ in the usual way. The size of a specification is simply the sum of the sizes of the sets within its atoms. We now talk about what it means for a path to model a specification. Let $\pi$ be a path, F be a subset of St and $\alpha ,\beta$ be arbitrary $\omega$-regular specifications. Then,

• $\pi \vDash \mathrm{Inf}\left(F\right)$, if $\mathrm{Inf}\left(\pi \right)\cap F\ne \varnothing$;
• $\pi \vDash \neg \alpha$, if it is not the case that $\pi \vDash \phi$;
• $\pi \vDash \alpha \wedge \beta$, if both $\pi \vDash \alpha$ and $\pi \vDash \beta$.

Note that we use Inf in two different, but interrelated senses. First, we use it as an operator over paths, as in $\mathrm{Inf}\left(\pi \right)$, to denote the set of states visited infinitely often in a path $\pi$, but we also use it as an operator over sets, as in $\mathrm{Inf}\left(F\right)$, as an atom in the specifications just defined. The semantics of the latter are defined in terms of the former. We will use these interchangeably: usage will be clear from the context. Using this notation, we can readily define conventional $\omega$-regular winning conditions, as follows:

Type	Associated Sets	$\mathbf{\omega }$-Regular Specification
Büchi	$F\subseteq \mathrm{St}$	$\mathrm{Inf}\left(F\right)$
Co-Büchi	$G\subseteq \mathrm{St}$	$\mathrm{Fin}\left(G\right)$
Gen. Büchi	${\left(F_k\right)}_{k\in K}\subseteq 2^{\mathrm{St}}$	${\bigwedge }_{k\in K}\mathrm{Inf}\left(F_k\right)$
Rabin	${(L_i,U_i)}_{i\in I}\subseteq 2^{\mathrm{St}}\times 2^{\mathrm{St}}$	${\bigvee }_{i\in I}\mathrm{Fin}\left(L_i\right)\wedge \mathrm{Inf}\left(U_i\right)$
Streett	${(L_j,U_j)}_{j\in J}\subseteq 2^{\mathrm{St}}\times 2^{\mathrm{St}}$	${\bigwedge }_{j\in J}\mathrm{Fin}\left(L_j\right)\vee \mathrm{Inf}\left(U_j\right)$
Muller	${\left(F_k\right)}_{k\in K}\subseteq 2^{\mathrm{St}}$	${\bigvee }_{k\in K}\mathrm{Inf}\left(F_k\right)\wedge \mathrm{Fin}\left(\overline{F_k}\right)$

Our $\omega$-regular specifications are equivalent to Emerson-Lei conditions [25], albeit with a different syntax. We can in fact represent all possible $\omega$-regular winning conditions—as another example, consider parity conditions. Suppose each state is labelled by a function, $\mu :\mathrm{St}\to \mathbb{N}$, with $\mu \left(s\right)\le m$ for some $m\in \mathbb{N}$, for all $s\in \mathrm{St}$. Given a path, $\pi$, the traditional parity condition is satisfied when $min\left\{\mu \right(s)\mid s\in \mathrm{Inf}(\pi \left)\right\}$ is odd. The sets of interest are defined by:

$$F_k=\{s\in \mathrm{St}\mid \mu \left(s\right)=k\}$$

Then, assuming m is odd (the formula for m even is similar), the parity condition can be expressed by the following specification:

$$\alpha =\mathrm{Fin}\left(F_0\right)\wedge \left(\mathrm{Inf}\left(F_1\right)\vee \left(\mathrm{Fin}\left(F_2\right)\wedge \left(\mathrm{Inf}\left(F_3\right)\vee \cdots \vee \left(\mathrm{Fin}\left(F_{m-1}\right)\wedge \mathrm{Inf}\left(F_m\right)\right)\right))\right)\right)$$

With $\omega$-regular specifications defined, we can talk about them in the context of games. Let $\overrightarrow{\sigma }$ be some strategy profile. Then, $\overrightarrow{\sigma }$ induces some path, $\pi \left(\overrightarrow{\sigma }\right)$, and given that $\omega$-regular specifications are defined on paths, we can talk about paths induced by strategies modelling specifications. However, we are not interested in whether the paths induced by arbitrary strategies model a given specification—it is more natural in the context of multi-player games to ask whether the paths induced by some or all of the equilibria of a game model a specification, both in the non-cooperative and in the cooperative contexts. In particular, we are interested in the Nash equilibria and the core, and whether the paths induced by strategy profiles that form an instance of these solution concepts model a specification.

Example 1.

Suppose we have four delivery robots in a warehouse (given by the coloured triangles in Figure 1), who want to pick up parcels at the pickup points (labelled by the bold Ps) and drop them off at the delivery points (labelled by the bold Ds). If a robot is not holding a parcel, and goes to a pickup point, it automatically gets given one. If it has a parcel, and goes to the delivery point, then it loses the parcel, and gains a payoff of 1. Furthermore, if two robots collide, by entering the same node at the same time, then they crash, and get a payoff of $-999$ at every future timestep.

Now, there are a number of Nash equilibria here (infinitely many, in fact), but it is easy to see that many of them exhibit undesirable properties. For instance, consider the strategy profile where red and pink go back and forth between the pickup and delivery points, and threaten to crash into, or deadlock, blue and yellow if they move from their starting positions. This is a Nash equilibrium, but is clearly not Pareto optimal—a socially undesirable outcome.

It is easy to identify the most socially desirable outcome—all four robots visiting the pickup and delivery points infinitely often, waiting for the others to pass when they reach bottleneck points. If we call the set containing the two states where robot i visits a pickup point $P_i$ and similarly label the set of delivery points $D_i$, we can express this condition concisely with the following ω-regular specification:

$$\underset{i\in \left[4\right]}{\bigwedge }\mathrm{Inf}\left(P_i\right)\wedge \mathrm{Inf}\left(D_i\right).$$

(1)

Thus, we can conclude that there exists some Nash equilibrium which models the above (Generalised Büchi) specification. However, we just did this by inspection. In practice, we would like to ask this question in a more principled way. As such, we will spend the rest of this paper exploring the natural decision problems associated with mean-payoff games with ω-regular specifications.

2.6. Mean-Payoff Games with $\omega$-Regular Specifications

Given that we have proposed $\omega$-regular specifications as an alternative to LTL [11] specifications, it is natural to ask how they compare. The connection between them is given by the following statement:

Proposition 1.

Let G be a game and let α be some ω-regular specification. Then there exists a set of atomic propositions, Φ, a labelling function $\lambda :\mathit{St}\to \mathcal{P}\left(\Phi \right)$, and an LTL formula φ such that, for all paths π, we have $\pi \vDash \alpha$ if and only if $\lambda \left(\pi \right)\vDash \phi$.

Proof. 

Without loss of generality, we may assume that $\alpha$ is written in conjunctive normal form, that is:

$$\alpha =\underset{i=1}{\overset{n}{\bigwedge }}\left(\underset{j=1}{\overset{m}{\bigvee }}{C}_{i,j}\right),$$

where each $C_{i,j}$ is an atom of the form $\mathrm{Inf}\left(F\right)$ or $\mathrm{Fin}\left(F\right)$ for some subset $F\subseteq \mathrm{St}$. We start by introducing a propositional variable $p_F$ for every subset $F\subset \mathrm{St}$. Then, for a given state $s\in \mathrm{St}$, we define:

$$\lambda \left(s\right)=\{p_F\in \mathrm{AP}\mid s\in F\}.$$

Then, we simply define:

$$\phi =\underset{i=1}{\overset{n}{\bigwedge }}\left(\underset{j=1}{\overset{m}{\bigvee }}{D}_{i,j}\right),$$

where $D_{i,j}=\mathbf{G}\mathbf{F}{p}_F$ if $C_{i,j}=\mathrm{Inf}\left(F\right)$ and $D_{i,j}=\mathbf{F}\mathbf{G}\neg p_F$ if $C_{i,j}=\mathrm{Fin}\left(F\right)$. We claim that for all paths $\pi$, we have $\pi \vDash \alpha$ if and only if $\lambda \left(\pi \right)\vDash \phi$.

First suppose that $\pi \vDash \alpha$. Thus, by definition, we have for all $1\le i\le n$ that $\pi \vDash {\bigvee }_{j=1}^m{C}_{i,j}$. This in turn implies that there exists some j such that $\pi \vDash C_{i,j}$. If $C_{i,j}=\mathrm{Inf}\left(F\right)$, then this implies that $\mathrm{Inf}\left(\pi \right)\cap F\ne \varnothing$. Take any $s\in \mathrm{Inf}\left(\pi \right)\cap F$. By definition, we have $p_F\in \lambda \left(s\right)$ and so we also have $\lambda \left(\pi \right)\vDash \mathbf{G}\mathbf{F}{p}_F$. However, by construction, this implies $\lambda \left(\pi \right)\vDash D_{i,j}$.

Similarly, if $C_{i,j}=\mathrm{Fin}\left(F\right)$, then we have $\mathrm{Inf}\left(\pi \right)\cap F=\varnothing$. Thus, for all $s\in \mathrm{Inf}\left(\pi \right)$, we have $p_F\notin \lambda \left(s\right)$ and so we have $\lambda \left(\pi \right)\vDash \mathbf{F}\mathbf{G}\neg p_F$. By construction, this implies that $\lambda \left(\pi \right)\vDash D_{i,j}$. Thus, for all i, there exists some j such that $\lambda \left(\pi \right)\vDash D_{i,j}$. This implies that $\lambda \left(\pi \right)\vDash \phi$.

Conversely, suppose that $\lambda \left(\pi \right)\vDash \phi$. So for all i, there exists a j such that $\lambda \left(\pi \right)\vDash D_{i,j}$. If $D_{i,j}=\mathbf{G}\mathbf{F}{p}_F$, then $\pi$ visits some state $s\in F$ infinitely often. Thus, $\mathrm{Inf}\left(\pi \right)\cap F\ne \varnothing$, so $\pi \vDash \mathrm{Inf}\left(F\right)$. Similarly, if $D_{i,j}=\mathbf{F}\mathbf{G}\neg p_F$, then $\pi \vDash \mathrm{Fin}\left(F\right)$. Either way, we have $\pi \vDash C_{i,j}$. So for all i, there exists some j such that $\pi \vDash C_{i,j}$, implying that $\pi \vDash \alpha$. □

Thus, $\omega$-regular specifications can be seen as being ‘isomorphic’ to a strict subset of LTL (it is straightforward to come up with LTL formulae that cannot be written as $\omega$-regular specifications—take for instance $\mathbf{G}\phi$, where $\phi$ is some non-trivial propositional formula). As such, we hope the restriction of the setting may yield some lower complexities when considering the analogous decision problems. That is, we will study a number of decision problems within the rational verification framework [10,17], where $\omega$-regular specifications replace LTL specifications in a very natural way.

Firstly, given a game, a strategy profile, and an $\omega$-regular specification, we can ask if the strategy profile is an equilibrium whose induced path models the specification. Secondly, given a game and an $\omega$-regular specification, we can ask if the specification is modelled by the path/paths induced by some/every strategy profile in the set of equilibria of the game. Each of these problems can be phrased in the context of a non-cooperative game or a cooperative game, depending on whether we let the set of equilibria be, respectively, the Nash equilibria or the core of the game. Formally, in the non-cooperative case, we have the following decision problems:

MEMBERSHIP:

Given: Game G, strategy profile $\overrightarrow{\sigma }$, and specification, $\alpha$.

Question: Is it the case that $\overrightarrow{\sigma }\in \mathrm{NE}\left(G\right)$ and $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$?

E-NASH:

Given: Game G and specification $\alpha$.

Question: Does there exist a $\overrightarrow{\sigma }\in \mathrm{NE}\left(G\right)$ such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$?

A natural dual to E-NASH is the A-NASH problem, which instead of asking if the specification holds in the path induced by some Nash equilibrium, asks if the specification holds in all equilibria:

A-NASH:

Given: Game G and specification $\alpha$.

Question: Is it the case that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$, for all $\overrightarrow{\sigma }\in \mathrm{NE}\left(G\right)$?

These decision problems were first studied in the context of iterated Boolean games [26], and are the ‘flagship‘ decision problems of rational verification [18].

In the cooperative setting, the analogous decision problems are defined by substituting $\mathrm{CORE}\left(G\right)$ for $\mathrm{NE}\left(G\right)$. We refer to these problems as MEMBERSHIP, E-CORE, and A-CORE, respectively, (with a small abuse of notation for the first problem: context will make it clear whether we are referring to the non-cooperative or cooperative problem). These variants were first studied in the setting of LTL games [17].

It is worth noting here one technical detail about representations. In the E-NASH problem, the quantifier asks if there exists a Nash equilibrium which models the specification. This quantification ranges over all possible Nash equilibria and the strategies may be arbitrary strategies. However, in the MEMBERSHIP problem, the strategy $\overrightarrow{\sigma }$ is part of the input, and thus, needs to be finitely representable. Therefore, when considering E-NASH (or A-NASH, or the corresponding problems for the core), we place no restrictions on the strategies, but when reasoning about MEMBERSHIP, we work exclusively with memoryless or finite-memory strategies.

Before we proceed to studying all these problems in detail, we note that even though some other types of games (for example, two-player, turn-based, zero-sum mean-payoff, or parity games) can be solved only using memoryless strategies, this is not the case in our setting:

Proposition 2.

There exist games G and ω-regular specifications α such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$ for some Nash equilibrium $\overrightarrow{\sigma }$, but for which there exists no memoryless Nash equilibrium $\overrightarrow{\sigma }$ such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$. The statement also holds true for the core.

Proof. 

Consider the game, $G=(M,{\left(w_i\right)}_{i\in \mathrm{Ag}})$, where $M=(\mathrm{Ag},\mathrm{St},s^0,{\left({\mathrm{Ac}}_i\right)}_{i\in \mathrm{Ag}},\mathrm{tr})$ is defined as follows: $\mathrm{Ag}=\{1,2\}$, $\mathrm{St}=\{\mathrm{mid},\mathrm{right},\mathrm{left}\}$, ${\mathrm{Ac}}_1={\mathrm{Ac}}_2=\{\mathbf{R},\mathbf{L}\}$, with the transition relation defined to be,

	Ac	$(\mathbf{R},\mathbf{R})$	$(\mathbf{R},\mathbf{L})$	$(\mathbf{L},\mathbf{R})$	$(\mathbf{L},\mathbf{L})$
St
mid		right	mid	mid	left
right		right	mid	mid	mid
left		mid	mid	mid	left

and the weight functions for the players defined as,

${\mathit{w}}_{\mathit{i}}\left(\mathit{s}\right)$	1	2
mid	0	0
right	1	0
left	0	1

Thus, the game looks like the following concurrent game structure (Figure 2):

In this game, each player decides whether they want to go left or right. When they both agree on what direction they want to go, they move in that direction. If they disagree, then they end up in the middle state. We now produce a finite memory strategy profile, $\overrightarrow{\sigma }$, and a specification $\alpha$ such that $\overrightarrow{\sigma }$ is a Nash equilibrium and $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$. The basic idea is that with finite memory strategies, the two players can alternate between the left and right state, each achieving a strictly positive payoff, threatening to punish one another if either deviates from this arrangement, whilst this is not possible in memoryless strategies, as in this case the players would have to take the same action each time in the middle state.

Consider the following strategy, ${\sigma }_1$, for player 1 (Figure 3):

Furthermore, the corresponding strategy, ${\sigma }_2$ for player 2 (Figure 4):

It is easy to verify that $\overrightarrow{\sigma }=({\sigma }_1,{\sigma }_2)$ is a Nash equilibrium. Each player gets a mean payoff of $1/4$ under $\overrightarrow{\sigma }$. Suppose player 1 deviates to another strategy, which does not match the sequence of actions as dictated by ${\sigma }_2$. Then player 2 will just start playing $\mathbf{L}$ forever, meaning that the game will never enter the right state again, implying player 1 gets a payoff of zero. By symmetry, we can conclude that this is a Nash equilibrium. Moreover, letting $\alpha$ be the $\omega$-regular specification $\alpha =\mathrm{Inf}\left(\right\{\mathrm{right}\left\}\right)\wedge \mathrm{Inf}\left(\right\{\mathrm{left}\left\}\right)$, we see that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$.

However, note that there cannot exist a memoryless Nash equilibrium, $\overrightarrow{\sigma }$, such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$. In a memoryless strategy, for a given state, both players must commit to an action and only play that given action vector in that state. If both players play $\mathbf{R}$ in the middle state, then they will never be able to reach the left state, and if they both play $\mathbf{L}$ in the middle state, they will never reach the right state. Additionally, if they disagree, then they will perpetually stay in the middle state. Thus, there cannot exist a memoryless strategy profile $\overrightarrow{\sigma }$, such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$. This example demonstrates that, in general, memoryless strategies are not powerful enough for our games: there may exist a Nash equilibrium which satisfies the specification, with no memoryless Nash equilibrium which satisfies it. Finally, note that $\overrightarrow{\sigma }$ is also in the core—individual deviations have been already accounted for, and clearly no group deviation containing both players strictly improves both pay-offs. □

3. Non-Cooperative Games

In the non-cooperative setting, MEMBERSHIP, E-NASH, and A-NASH are the relevant decision problems. In this section, we will show that MEMBERSHIP lies in P for memoryless strategies, while E-NASH is NP-complete and even remains NP-hard when restricted to memoryless strategies—thus, no worse than solving a multi-player mean-payoff game [5]. Because A-NASH is the dual problem of E-NASH, it follows that A-NASH is co-NP-complete. In order to obtain some of these results, we also provide a semantic characterisation of the paths associated with strategy profiles in the set of Nash equilibria that satisfy a given $\omega$-regular specification. We will first study the MEMBERSHIP problem, and then investigate E-NASH, providing an upper bound for arbitrary strategies and a lower bound for memoryless strategies—arguably, the simplest, yet still computationally important, model of strategies one may want to consider for multi-agent systems.

Theorem 1.

For memoryless strategies, MEMBERSHIPis in P.

Proof. 

We verify that a given strategy profile is a Nash equilibrium in the following way. We begin by calculating the payoff of each player in the strategy profile by ‘running’ the strategy and keeping note of the game states. When we encounter the same game state twice, we can simply take the average of the weights encountered at the states between the two occurrences, and that will be the payoff for a given player. By the pidgeonhole principle, it will take no more than $\left|\mathrm{St}\right|+1$ time steps to get to this point, and thus, this can be done in linear time.

Once we have each player’s payoff, we can determine if they have any incentive to deviate in the following way—for each player, look at the graph induced by the strategy profile excluding them. Formally, the graph induced by a partial strategy profile ${\overrightarrow{\sigma }}_{-i}$, denoted by $G\left[{\overrightarrow{\sigma }}_{-i}\right]=(V,E)$ is defined as follows: the set of nodes, V, simply consists of the set of states of the game, that is, $V=\mathrm{St}$ and the set of edges are simply the moves available to player i—that is, if $e=(s^1,s^2)\in E$, then there exists some ${\mathrm{ac}}_i^{\prime }\in {\mathrm{Ac}}_i$ such that $\mathrm{tr}(s^1,({\overrightarrow{\sigma }}_{-i}\left(s^1\right),{\mathrm{ac}}_i^{\prime }))=s^2$.

We can use Karp’s algorithm [27] to determine the maximum payoff that this player can achieve given the other players’ strategies. If this payoff is higher than the payoff originally achieved, then this means player i can successfully deviate from the given strategy profile and be better off, and so it is not a Nash equilibrium. If we do this for each player, and the maximum payoff each player can achieve to equal to their payoff in the given strategy profile, then we can conclude it is a Nash equilibrium, and moreover, we have determined this in polynomial time.

To determine if the strategy profile satisfies the specification, we run the strategy as before, and determine the periodic path which the game will end up in. This tells us which states will be finitely and infinitely visited, which in turn induces a valuation which will either model or not model the specification. Checking this can be done in polynomial time and thus, we can conclude that for memoryless strategies, MEMBERSHIP lies in P. □

The simplicity of the above algorithm may raises hopes that it might extend to finite-memory strategies. However, in this case, the configuration of the game is not just given by the current state—it is given by the current state, as well as the state that each of the player’s strategies are in. Thus, we might have to visit at least $\left|\mathrm{St}\right|·{\left|Q\right|}^{\left|\mathrm{Ag}\right|}+1$ (where Q is the smallest set of strategy states over the set of players) configurations until we discover a loop. Now whilst there is an exponential dependency on the number of players in the input to the problem, this bound on the number of configurations is not necessarily polynomial in the size of the input. More precisely, the size of the underlying concurrent game structure is $\left|\mathrm{St}\right|·{\left|\mathrm{Ac}\right|}^{\left|\mathrm{Ag}\right|}$ and so if $\left|Q\right|$ is larger that $\left|\mathrm{Ac}\right|$, the number of configurations will grow exponentially faster than the size of the input. Thus, we cannot use the above algorithm in the case of finite memory strategies to get a polynomial time upper bound.

We now consider the E-NASH problem. Instead of providing the full NP-completeness result here, we start by showing that the problem is NP-hard, even for memoryless strategies, and delay the proof of the upper bound until we develop a useful characterisation of Nash equilibrium in the context of $\omega$-regular specifications. For now, we have the following hardness result, obtained using a reduction from the Hamiltonian cycle problem [28,29]—a similar, but simpler, argument can be found in [5].

Proposition 3.

E-NASHis NP-hard, even for games with one player, constant weights, and memoryless strategies.

Proof. 

Let $G=(V,E)$ be a graph with $\left|V\right|=n$. We form a mean-payoff game G by letting $\mathrm{Ag}=\left\{1\right\}$ and $\mathrm{St}=V$. We pick the initial state arbitrary and label it $s^0$, and the actions for player 1 correspond to the edges of G. That is, we have ${\mathrm{Ac}}_1=E$ and we have $\mathrm{tr}(u,e)=v$ if and only if $e=(u,v)\in E$. Finally, we fix an integer constant $k\in \mathbb{Z}$ and let $w_1\left(s\right)=k$ for all $s\in \mathrm{St}$.

Let $F_s=\left\{s\right\}$ for each $s\in \mathrm{St}$ and let $\alpha$ be the following specification:

$$\alpha =\underset{s\in \mathrm{St}}{\bigwedge }\mathrm{Inf}\left(F_s\right).$$

We claim that G has a Hamiltonian cycle if and only if G has a memoryless Nash equilibrium $\overrightarrow{\sigma }$ such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$.

First suppose that G has a Hamiltonian cycle, $\pi =v^0{v}^1\dots v^{n-1}$. We define a memoryless strategy, ${\sigma }_1$, for player 1 by setting ${\sigma }_1\left(v^i\right)=v^{i+1}$, where the superscript is interpreted modulo n. As $\pi$ is a Hamiltonian cycle, it visits every node, so ${\sigma }_1$ is a well-defined, total function. By definition, we see that $\pi \left({\sigma }_1\right)=\pi$, so we have $\pi \left(\sigma \right)\vDash \alpha$. Additionally, as each state has the same payoff, this strategy is trivially a Nash equilibrium.

Now suppose that G has a memoryless Nash equilibrium ${\sigma }_1$ such that $\pi \left({\sigma }_1\right)\vDash \alpha$. Let $\pi \left({\sigma }_1\right)=\pi$. Since ${\sigma }_1$ is memoryless, $\pi$ must be of the form $\pi =\pi [..i]{\pi [i..j]}^{\omega }$ for integers i and j. Without loss of generality, assume that i and j are the smallest integers such that this holds. Moreover, since $\pi \vDash \alpha$, the path $\pi$ visits every state infinitely often so we must have that $\pi [i..j]$ contains every state, and by memorylessness, it must be a cycle. Thus, by definition, $\pi [i..j]$ is a Hamiltonian cycle. □

Theorem 1 and Proposition 3 together establish NP-completeness for multi-player mean-payoff games with $\omega$-regular specifications and memoryless strategies: one can non-deterministically guess a memoryless strategy for each player (which is simply a list of actions for each player, one for each state), and use MEMBERSHIP to verify that it is indeed a Nash equilibrium that models the specification. However, as we shall show later, the problem is also NP-complete in the general case. To prove this, we need to develop an alternative characterisation of Nash equilibria.

To do this, we need to introduce the notion of the punishment value in a multi-player mean-payoff game; cf., [30,31]. The punishment value, ${\mathrm{pun}}_i\left(s\right)$, of a player i in a given state s can be thought of as the worst value the other players can impose on a player at a given state. Concretely, if we regard the game G as a two player, zero-sum game, where player i plays against the coalition $\mathrm{Ag}\backslash \left\{i\right\}$, then the punishment value for player i is the smallest mean-payoff value that the rest of players in Ag can inflict on i from a given state. Formally, given a player i and a state $s\in \mathrm{St}$, we define the punishment value, ${\mathrm{pun}}_i\left(s\right)$ against player i at state s, as follows:

$${\mathrm{pun}}_i\left(s\right)=\underset{{\overrightarrow{\sigma }}_{-i}\in {\mathsf{\Sigma }}_{-i}}{min}\underset{{\tau }_i\in {\mathsf{\Sigma }}_i}{max}{\mathsf{pay}}_i\left(\pi (({\overrightarrow{\sigma }}_{-i},{\tau }_i),s)\right)$$

How efficiently can we calculate this value? As established in [5], we proceed in the following way: in a two player, turn-based, zero-sum, mean-payoff game, positional strategies suffice to achieve the punishment value [4]. Thus, we can non-deterministically guess a pair of positional strategies for each player (one for the coalition punishing the player, and one for the player themselves), use Karp’s algorithm [27] to find the maximum payoff for both the player and the coalition against their respective punishing strategies, and then verify that the two values coincide. With this established, we have the following lemma, which can be proved using techniques for mean-payoff games adapted from [5,15].

Lemma 1.

Let π be a path in G and let ${\left\{\overrightarrow{\mathit{ac}}\left[k\right]\right\}}_{k\in \mathbb{N}}$ be the path of associated action profiles. Then there is a Nash equilibrium, $\overrightarrow{\sigma }\in \mathit{NE}\left(G\right)$, such that $\pi =\pi \left(\overrightarrow{\sigma }\right)$ if and only if there exists some $\overrightarrow{z}\in {\mathbb{Q}}^{\mathit{Ag}}$, with $z_i\in \{{\mathit{pun}}_i\left(s\right)\mid s\in \mathit{St}\}$, such that:

• for each k, we have ${\mathit{pun}}_i\left(\mathit{tr}(\pi \left[k\right],({\overrightarrow{\mathit{ac}}\left[k\right]}_{-i},{\mathit{ac}}_i^{\prime }))\right)\le z_i$ for all $i\in \mathit{Ag}$ and ${\mathit{ac}}_i^{\prime }\in {\mathit{Ac}}_i$, and;
• for all players $i\in \mathit{Ag}$, we have $z_i\le {\mathit{pay}}_i\left(\pi \right)$.

Proof. 

First assume that we have some Nash equilibrium $\overrightarrow{\sigma }\in \mathrm{NE}\left(G\right)$ such that $\pi =\pi \left(\overrightarrow{\sigma }\right)$. Suppose there does not exist any $z\in {\mathbb{Q}}^{\mathrm{Ag}}$ with the desired properties. Furthermore, let us first suppose that for all $\overrightarrow{z}\in {\mathbb{Q}}^{\mathrm{Ag}}$, with $z_i\in \{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$, there exists some player $i\in \mathrm{Ag}$ such that $z_i>\mathsf{mp}\left(w_i\left(\pi \right)\right)$. However, this is true for all $z_i\in \{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$. So we have ${\mathrm{pun}}_i\left(s^0\right)>\mathsf{mp}\left(w_i\left(\pi \right)\right)$. This means that player i might as well play the positional strategy which ensures they achieve at least the punishment value. This in turn means that player i has some deviation, contradicting the fact that $\overrightarrow{\sigma }$ is a Nash equilibrium.

So instead, it must be the case that for all $\overrightarrow{z}\in {\mathbb{Q}}^{\mathrm{Ag}}$, with $z_i\in \{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$, there is some time step k, a player i and an action ${\mathrm{ac}}_i^{\prime }$ such that ${\mathrm{pun}}_i\left(\mathrm{tr}(\pi \left[k\right],({\overrightarrow{\mathrm{ac}}\left[k\right]}_{-i},{\mathrm{ac}}_i^{\prime }))\right)>z_i$. However, since this is true for all $z_i$, it is true for $z_i=max\{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$. Furthermore, since ${\mathrm{pun}}_i\left(\mathrm{tr}(\pi \left[k\right],({\overrightarrow{\mathrm{ac}}\left[k\right]}_{-i},{\mathrm{ac}}_i^{\prime }))\right)\in \{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$, this gives a contradiction—we cannot have ${\mathrm{pun}}_i\left(\mathrm{tr}(\pi \left[k\right],({\overrightarrow{\mathrm{ac}}\left[k\right]}_{-i},{\mathrm{ac}}_i^{\prime }))\right)>z_i$. Thus, it follows that the first part of the statement is true.

Now assume that there exists some $\overrightarrow{z}\in {\mathbb{Q}}^{\mathrm{Ag}}$ with the properties as prescribed in the statement of the lemma. We define a strategy profile $\overrightarrow{\sigma }$ in the following way. Each player follows $\pi$. If any player chooses to deviate from $\pi$, say at state $\pi \left[k\right]$, with an action ${\mathrm{ac}}_i^{\prime }$ then the remaining players play the punishing strategy which causes player i to have a payoff of at most ${\mathrm{pun}}_i\left(\mathrm{tr}(\pi \left[k\right],{\overrightarrow{\mathrm{ac}}\left[k\right]}_{-i},{\mathrm{ac}}_i^{\prime })\right)\le z_i\le {\mathsf{pay}}_i\left(\pi \right)$. Thus, no player has any incentive to deviate away from $\overrightarrow{\sigma }$ and so we have a Nash equilibrium with $\pi \left(\overrightarrow{\sigma }\right)=\pi$. □

With this lemma in mind, we define a graph, $G[\overrightarrow{z};F]=(V,E)$ as follows. We set $V=\mathrm{St}$ and include $e=(u,v)\in E$ if there exists some action profile $\overrightarrow{\mathrm{ac}}$ such that $v=\mathrm{tr}(u,\overrightarrow{\mathrm{ac}})$ with ${\mathrm{pun}}_i\left(\mathrm{tr}(u,({\overrightarrow{\mathrm{ac}}}_{-i},{\mathrm{ac}}_i^{\prime }))\right)\le z_i$ for all $i\in \mathrm{Ag}$ and ${\mathrm{ac}}_i^{\prime }\in {\mathrm{Ac}}_i$. Having done this, we then prune any components which cannot be reached from the start state and then remove all states and edges not contained in F, before reintroducing any states in F that may have been removed. Thus, given this definition and the preceding lemma, to determine if there exists a Nash equilibrium which satisfies an $\omega$-regular specification, $\alpha$, we calculate the punishment values, and guess a vector $\overrightarrow{z_s}\in {\mathrm{St}}^{\mathrm{Ag}}$, as well a set of states, F, which satisfy the specification. Letting $z_i={\mathrm{pun}}_i\left(z_s\right)$, we form the graph $G[\overrightarrow{z};F]$ and then check if there is some path $\pi$ in $G[\overrightarrow{z};F]$ with $z_i\le {\mathsf{pay}}_i\left(\pi \right)$ for each player i which visits every state infinitely often. Trivially, if this graph is not strongly connected, then no path can visit every state infinitely often. Thus, to determine if the above condition holds, we need one more piece of technical machinery, in the form of the following proposition:

Proposition 4.

Let $G=(V,E)$ be a strongly connected graph, let ${\left\{w_i\right\}}_{i\in \mathit{Ag}}$ be a set of weight functions, and let $\overrightarrow{z}\in {\mathbb{Q}}^{\mathit{Ag}}$. Then, we can determine if there is some path π with the properties,

• π visits every state infinitely often;
• $z_i\le {\mathit{pay}}_i\left(\pi \right)$ for each $i\in \mathit{Ag}$,

in polynomial time.

Conceptually, Proposition 4 is similar to Theorem 18 of [5], but with two key differences—firstly, we need to do additional work to determine if there is a path that visits every state infinitely often. Moreover, the argument of [5] is adapted so we have the corollary that if there is a Nash equilibrium that models the specification, then there is some finite state Nash equilibrium that also models the specification. This means that the construction in our proof can be used not only for verification, but also for synthesis.

For clarity of presentation, we shall split the above proposition into two constituent lemmas. To do this, we begin by defining a system of linear inequalities. We then go on to show that there is a path $\pi$ with the desired properties if and only if this system has a solution—one lemma for each direction. As the system of inequalities can be determined in polynomial time, this yields our result.

For a graph G, a set of weight functions ${\left\{w_i\right\}}_{i\in \mathrm{Ag}}$ be a set of weight functions, and a vector $\overrightarrow{z}\in {\mathbb{Q}}^{\mathrm{Ag}}$, we define a system of linear inequalities, $\ell (G,{\left\{w_i\right\}}_{i\in \mathrm{Ag}},\overrightarrow{z})$ as follows: for each agent $i\in \mathrm{Ag}$, and each edge $e\in E$, introduce a variable $x_{i,e}$, along with the following inequalities:

(i)

$x_{i,e}\ge 0$ for each agent $i\in \mathrm{Ag}$ and for each edge $e\in E$.

(ii)

${\sum }_{e\in E}{x}_{i,e}=1$ for each agent $i\in \mathrm{Ag}$.

(iii)

${\sum }_{e\in \mathrm{In}\left(v\right)}{x}_{i,e}={\sum }_{e\in \mathrm{Out}\left(v\right)}{x}_{i,e}$ for all $i\in \mathrm{Ag}$ and for each $v\in V$.

(iv)

$z_i\le {\sum }_{e\in E}{x}_{i,e}·w_i\left(e\right)$ for all $i\in \mathrm{Ag}$.

(v)

${\sum }_{e\in E}{x}_{i,e}·w_i\left(e\right)\le {\sum }_{e\in E}{x}_{j,e}·w_i\left(e\right)$ for all $i,j\in \mathrm{Ag}$.

It is worth briefly discussing what this system is actually encoding. Roughly speaking, the set ${\mathcal{C}}_i={\left\{x_{i,e}\right\}}_{e\in E}$ defines a cycle for each player that makes sure their payoff is greater than $z_i$. Each $x_{i,e}$ represents the proportion that a given edge is visited in the cycle. The idea is that we define a path by visiting each ${\mathcal{C}}_i$ an appropriate number of times, before travelling to the next cycle and visiting that repeatedly.

Conversely, if there exists some path with the stated properties, it will also define a solution to the system of inequalities. This has the corollary that if there is a Nash equilibrium in the game, then there is a finite state Nash equilibrium as well.

In what follows, for an edge $e=(u,v)$ and a weight function $w:V\to \mathbb{Z}$, we define $w\left(e\right):=w\left(u\right)$. We also extend weight functions to finite paths in the natural way, by summing along them.

Lemma 2.

Let $G=(V,E)$ be a strongly connected graph, let ${\left\{w_i\right\}}_{i\in \mathit{Ag}}$ be a set of weight functions, let $\overrightarrow{z}\in {\mathbb{Q}}^{\mathit{Ag}}$. Furthermore, suppose there exists some path π such that $z_i\le {\mathit{pay}}_i\left(\pi \right)$ for each $i\in \mathit{Ag}$, which visits every state infinitely often. Then $\ell (G,{\left\{w_i\right\}}_{i\in \mathit{Ag}},\overrightarrow{z})$ has a solution.

Proof. 

First suppose there exists some path $\pi$ with the stated properties. For each $n>0$ and $e\in E$, define $\lambda (n,e)$ to be the following quantity:

$$\lambda (n,e)=\frac{\#\{j<n\mid (\pi \left[j\right],\pi [j+1])\in E\}}{n}.$$

Informally, for a given edge e, $\lambda (n,e)$ gives us the proportion that e appears in the prefix $\pi [..n]$. Note that $0\le \lambda (n,e)\le 1$ for all $e\in E$ and all $n>0$. Additionally, for each n, it easy to see we have:

$$\sum _{e\in E}\lambda (n,e)=1.$$

(2)

By definition, for each $i\in \mathrm{Ag}$ we have:

$${\mathsf{pay}}_i\left(\pi \right)=\underset{n\to \infty }{lim\; inf}\frac{1}{n}\sum _{k=0}^{n-1}{w}_i\left(\pi \left[k\right]\right).$$

Thus, there must exist some subsequence of the natural numbers, ${\left\{n_t^i\right\}}_{t\in \mathbb{N}}$, such that:

$${\mathsf{pay}}_i\left(\pi \right)=\underset{t\to \infty }{lim}\frac{1}{n_t^i}\sum _{k=0}^{n_t^i-1}{w}_i\left(\pi \left[k\right]\right).$$

With this defined, we introduce a sequence of numbers, ${\left\{{\phi }_t^i\left(e\right)\right\}}_{t\in \mathbb{N}}$ for each edge $e\in E$, by defining ${\phi }_t^i\left(e\right)=\lambda (n_t^i,e)$. Now, as ${\left\{{\phi }_t^i\left(e\right)\right\}}_{t\in \mathbb{N}}$ is a bounded sequence, by the Bolzano-Weierstrass theorem, there must be a convergent subsequence for each edge e, ${\left\{{\psi }_t^i\left(e\right)\right\}}_{t\in \mathbb{N}}$. We define $x_{i,e}^{*}={lim}_{t\to \infty }{\psi }_t^i\left(e\right)$ and claim that these form a solution to the system of inequalities.

Since ${\phi }_t^i\left(e\right)=\lambda (n_t^i,e)$, it is clear that we have $0\le x_{i,e}^{*}\le 1$. Thus, Inequality (i) is satisfied. Additionally, by Equation (2), we can deduce that for each t, and for every $i\in Ag$, we have:

$$\sum _{e\in E}{\psi }_t^i\left(e\right)=1.$$

Taking limits, we can conclude that Inequality (ii) is satisfied. To establish (iii), by definition of $\lambda$, fix a $v\in V$. In a path, if we enter a node, we must exit it. Thus, we have:

$$-1\le \sum _{e\in \mathrm{In}\left(v\right)}n·\lambda (n,e)-\sum _{e\in \mathrm{Out}\left(v\right)}n·\lambda (n,e)\le 1,$$

implying,

$$-\frac{1}{n_t^i}\le \sum _{e\in \mathrm{In}\left(v\right)}{\phi }_t^i\left(e\right)-\sum _{e\in \mathrm{Out}\left(v\right)}{\phi }_t^i\left(e\right)\le \frac{1}{n_t^i}.$$

Taking the relevant subsequence and letting $t\to \infty$, we can deduce that Inequality (iii) is satisfied. To establish (iv), first note that for all $i\in \mathrm{Ag}$, we have,

$$\begin{array}{cc}\hfill {\mathsf{pay}}_i\left(\pi \right)& =\underset{n\to \infty }{lim\; inf}\frac{1}{n}\sum _{k=0}^{n-1}{w}_i\left(\pi \left[k\right]\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim}\frac{1}{n_t^i}\sum _{k=0}^{n_t^i-1}{w}_i\left(\pi \left[k\right]\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim}\sum _{e\in E}\lambda (n_t^i,e)·w_i\left(e\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim}\sum _{e\in E}{\phi }_t^i\left(e\right)·w_i\left(e\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim}\sum _{e\in E}{\psi }_t^i\left(e\right)·w_i\left(e\right)\hfill \\ \hfill & =\sum _{e\in E}{x}_{i,e}·w_i\left(e\right).\hfill \end{array}$$

The equality between lines 4 and 5 is valid, as we have established that the limit exists in line 2. Thus, since we have $z_i\le {\mathsf{pay}}_i\left(\pi \right)$, by assumption, we can conclude that Inequality (iv) is valid. In a similar fashion, note that for all $i,j\in \mathrm{Ag}$, we have,

$$\begin{array}{cc}\hfill {\mathsf{pay}}_i\left(\pi \right)& =\underset{n\to \infty }{lim\; inf}\frac{1}{n}\sum _{k=0}^{n-1}{w}_i\left(\pi \left[k\right]\right)\hfill \\ \hfill & \le \underset{t\to \infty }{lim\; inf}\frac{1}{n_t^j}\sum _{k=0}^{n_t^j-1}{w}_i\left(\pi \left[k\right]\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim\; inf}\sum _{e\in E}\lambda (n_t^j,e)·w_i\left(e\right)\hfill \\ \hfill & =\underset{t\to \infty }{lim\; inf}\sum _{e\in E}{\phi }_t^j\left(e\right)·w_i\left(e\right)\hfill \\ \hfill & \le \underset{t\to \infty }{lim}\sum _{e\in E}{\psi }_t^j\left(e\right)·w_i\left(e\right)\hfill \\ \hfill & =\sum _{e\in E}{x}_{j,e}·w_i\left(e\right).\hfill \end{array}$$

This, together with Inequality (iv), implies that Inequality (v) is satisfied.

Thus, putting all this together, we can conclude that if there exists some path with the stated properties, then there also exists a solution to $\ell (G,{\left\{w_i\right\}}_{i\in \mathrm{Ag}},\overrightarrow{z})$, the system of linear inequalities. □

We now show the other direction:

Lemma 3.

Let $G=(V,E)$ be a strongly connected graph, let ${\left\{w_i\right\}}_{i\in \mathit{Ag}}$ be a set of weight functions, let $\overrightarrow{z}\in {\mathbb{Q}}^{\mathit{Ag}}$. Suppose that $\ell (G,{\left\{w_i\right\}}_{i\in \mathit{Ag}},\overrightarrow{z})$ has a solution. Then there exists some path π such that $z_i\le {\mathit{pay}}_i\left(\pi \right)$ for each $i\in \mathit{Ag}$, which visits every state infinitely often.

Proof. 

Given that there is a solution to $\ell (G,{\left\{w_i\right\}}_{i\in \mathrm{Ag}},\overrightarrow{z})$, there must exist a solution that consists of rational numbers. Thus, letting $x_{i,e}^{*}$ be a rational solution, we can write $x_{i,e}^{*}=p_{i,e}/q$, for some $p_{i,e}\in \mathbb{N}$, and some appropriately chosen $q\in \mathbb{N}$.

Now, for each $i\in \mathit{Ag}$, we form a multigraph, $G_i=(V_i,E_i)$, which takes G and replaces each edge e by $p_{i,e}$ copies. Note that whilst G is strongly connected, some of the $p_{i,e}$ may be equal to 0, which would mean that $G_i$ is disconnected.

Now, by Inequality (iii), for each $v\in V$, we have:

$$\sum _{e\in \mathrm{In}\left(v\right)}{p}_{i,e}=\sum _{e\in \mathrm{Out}\left(v\right)}{p}_{i,e}.$$

Thus, the in-degree of each vertex in $V_i$ is equal to its out-degree. Thus, each strongly connected component of $G_i$ contains an Eulerian cycle (An Eulerian cycle is a path that starts and ends at the same node which visits every edge exactly once). Interpreting each of these Eulerian cycles as paths in G, we get a set of (not necessarily simple) cycles for each agent, $C_i^1,\dots C_i^{m_i}$, where $m_i\le \left|V\right|$.

Now for each agent $i\in \mathrm{Ag}$, and every $n\in \mathbb{N}$, we define a cycle ${\mathcal{C}}_i^n$ which starts at the first state of $C_i^1$, traces $C_i^1$n times, takes the shortest path from the start state of $C_i^1$ to $C_i^2$, traces $C_i^2$n times and repeats this process for all the cycles of agent i, until it has traced $C_i^{m_i}$n times. From here, it then takes the shortest path to the start state of $C_i^1$.

Let $M_i$ be the largest of the absolute values of the weights of agent i. That is, $M_i={max}_{v\in V}\left|w_i\left(v\right)\right|$. Then we have for all $i,j\in \mathrm{Ag}$,

$$n·\sum _{e\in E}{p}_{j,e}·w_i\left(e\right)-{\left|V\right|}^2·M_i\le w_i\left({\mathcal{C}}_j^n\right)\le n·\sum _{e\in E}{p}_{j,e}·w_i\left(e\right)+{\left|V\right|}^2·M_i,$$

and,

$$n·\sum _{e\in E}{p}_{j,e}\le |{\mathcal{C}}_j^n|\le n·\sum _{e\in E}{p}_{j,e}+{\left|V\right|}^2.$$

Putting these together, we get:

$$\begin{array}{cc}\hfill \frac{w_i\left({\mathcal{C}}_j^n\right)}{|{\mathcal{C}}_j^n|}& \hfill \ge \frac{n·{\sum }_{e\in E}{p}_{j,e}·w_i\left(e\right)-{\left|V\right|}^2·M_i}{n·{\sum }_{e\in E}{p}_{j,e}+{\left|V\right|}^2}\hfill \\ \hfill & \hfill \ge \frac{n·{\sum }_{e\in E}{p}_{i,e}·w_i\left(e\right)-{\left|V\right|}^2·M_i}{n·{\sum }_{e\in E}{p}_{j,e}+{\left|V\right|}^2}\hfill \\ \hfill & \hfill =\frac{{\sum }_{e\in E}{x}_{i,e}^{*}·w_i\left(e\right)-\frac{{\left|V\right|}^2·M_i}{n·q}}{{\sum }_{e\in E}{x}_{j,e}^{*}+\frac{{\left|V\right|}^2}{n·q}}\hfill \\ \hfill & \hfill \ge \frac{z_i-\frac{{\left|V\right|}^2·M_i}{n·q}}{1+\frac{{\left|V\right|}^2}{n·q}}.\hfill \end{array}$$

Taking limits, we see that:

$$\underset{n\to \infty }{lim}\frac{w_i\left({\mathcal{C}}_j^n\right)}{|{\mathcal{C}}_j^n|}\ge z_i.$$

Thus, for $n_i^{*}$ sufficiently large, we must also have, for all $j\in \mathrm{Ag}$:

$$\frac{w_i\left({\mathcal{C}}_j^{n_i^{*}}\right)}{|{\mathcal{C}}_j^{n_i^{*}}|}\ge z_i.$$

Moreover, if $n_i^{*}$ is large enough, the above equation will be true for all $n\ge n_i^{*}$. Assuming this is the case, we set $n^{*}={max}_{i\in \mathrm{Ag}}{n}_i^{*}$. Then, for all $i,j\in \mathrm{Ag}$, we have,

$$\frac{w_i\left({\mathcal{C}}_j^{n^{*}}\right)}{|{\mathcal{C}}_j^{n^{*}}|}\ge z_i.$$

We are now ready to define a path ${\pi }^N$, which will form the basis for our path of interest, $\pi$. The path starts by visiting ${\mathcal{C}}_1^{n^{*}}$N times, before taking the shortest path to the start of ${\mathcal{C}}_2^{n^{*}}$, and visiting that N times, and so on. Once ${\mathcal{C}}_{\left|\mathrm{Ag}\right|}^{n^{*}}$ has been visited N times, we then take the shortest path that visits any states that have not been yet visited, before returning to the start vertex of ${\mathcal{C}}_1^{n^{*}}$.

Let us look at each player’s payoff on ${\pi }^N$. We have:

$$\begin{array}{cc}\hfill \frac{w_i\left({\pi }^N\right)}{|{\pi }^N|}& \hfill \ge \frac{N·{\sum }_{j\in \mathrm{Ag}}{w}_i\left({\mathcal{C}}_j^{n^{*}}\right)-\left|V\right|·\left|\mathrm{Ag}\right|·M_i-{\left|V\right|}^2·M_i}{N·{\sum }_{j\in \mathrm{Ag}}|{\mathcal{C}}_j^{n^{*}}{|+|V|·|\mathrm{Ag}|+|V|}^2}\hfill \\ \hfill & \hfill \ge \frac{N·{\sum }_{j\in \mathrm{Ag}}{z}_i·|{\mathcal{C}}_j^{n^{*}}|-|V|·|\mathrm{Ag}|·M_i-{\left|V\right|}^2·M_i}{N·{\sum }_{j\in \mathrm{Ag}}|{\mathcal{C}}_j^{n^{*}}{|+|V|·|\mathrm{Ag}|+|V|}^2}\hfill \\ \hfill & \hfill =\frac{z_i·{\sum }_{j\in \mathrm{Ag}}\left|{\mathcal{C}}_j^{n^{*}}\right|-\frac{\left|V\right|·\left|\mathrm{Ag}\right|·M_i+{\left|V\right|}^2·M_i}{N}}{{\sum }_{j\in \mathrm{Ag}}\left|{\mathcal{C}}_j^{n^{*}}\right|+\frac{\left|V\right|·\left|\mathrm{Ag}\right|+{\left|V\right|}^2}{N}}.\hfill \end{array}$$

Taking limits as before, we can conclude that for $N^{*}$ sufficiently large, we have:

$$\frac{w_i\left({\pi }^{N^{*}}\right)}{|{\pi }^{N^{*}}|}\ge z_i.$$

We then simply define $\pi$ to be the path that repeatedly traverses ${\pi }^{N^{*}}$. It is easy to see that ${\mathsf{pay}}_i\left(\pi \right)\ge z_i$ and that $\pi$ visits every state infinitely often. □

We can now combine these two results to obtain our proof:

Proof. 

(Proof of Proposition 4) Given the statement of the proposition, we construct an algorithm which forms the linear program $\ell (G,{\left\{w_i\right\}}_{i\in \mathrm{Ag}},\overrightarrow{z})$ and see if it has a solution. As the linear program is of polynomial size, and as linear programs can be solved in polynomial time, we see that the overall algorithm can be done in polynomial time. By Lemma 3, we see that the algorithm is sound and then by Lemma 2, we see it is complete. □

From the propositions above, we can establish the complexity of E-NASH:

Theorem 2.

E-NASHis NP-complete.

Proof. 

For NP-hardness we have Proposition 3. For the upper bound, suppose we have an instance, $(G,\alpha )$, of the problem. Then we proceed as follows. We non-deterministically guess pairs of punishing strategy profiles, $({\zeta }_i,{\overrightarrow{\zeta }}_{-i})$ for each player $i\in \mathrm{Ag}$, a state $z_s$ for each player, and a set of states F. From these, we can easily check that the valuation induced by F satisfies the specification and we can also use Karp’s algorithm to compute the punishment values, ${\mathrm{pun}}_i\left(s\right)$, for each state $s\in \mathrm{St}$ and for each player $i\in \mathrm{Ag}$. Setting $z_i={\mathrm{pun}}_i\left(z_s\right)$, we invoke Lemma 1 and form the graph $G[\overrightarrow{z};F]$. If it is not strongly connected, then we reject. Otherwise, we use Proposition 4 to determine if the associated linear program has a solution. If it does, then we accept, otherwise we reject. □

Another benefit of splitting Proposition 4 is that it readily yields the following corollary:

Corollary 1.

Let G be a game and α an ω-regular specification. Suppose that G has some Nash equilibrium $\overrightarrow{\sigma }$ such that $\overrightarrow{\sigma }\vDash \alpha$. Then, G also has some finite-memory Nash equilibrium ${\overrightarrow{\sigma }}^{\prime }$ such that ${\overrightarrow{\sigma }}^{\prime }\vDash \alpha$.

Proof. 

Suppose G has some Nash equilibrium $\overrightarrow{\sigma }$ such that $\overrightarrow{\sigma }\vDash \alpha$. Furthermore, let $\pi =\pi \left(\overrightarrow{\sigma }\right)$. Then by Lemma 1, there exists some $\overrightarrow{z}\in {\mathbb{Q}}^{\mathrm{Ag}}$, with $z_i\in \{{\mathrm{pun}}_i\left(s\right)\mid s\in \mathrm{St}\}$, and a subset $F\subseteq \mathrm{St}$ such that in $G[\overrightarrow{z},F]$, $\pi$ visits every state infinite often and for each $i\in \mathrm{Ag}$, we have $z_i\le {\mathsf{pay}}_i\left(\pi \right)$. By Lemma 2, we see that the system of inequalities $\ell (G,{\left\{w_i\right\}}_{i\in \mathrm{Ag}},\overrightarrow{z})$ has a solution. However, then by Lemma 3, we see that there exists some periodic path ${\pi }^{\prime }$ such that in $G[\overrightarrow{z},F]$, again, we have that π′ visits every state infinite often and for each $i\in \mathrm{Ag}$, we have $z_i\le {\mathsf{pay}}_i\left({\pi }^{\prime }\right)$. Thus, applying Lemma 1 again, we see that G has some Nash equilibrium ${\overrightarrow{\sigma }}^{\prime }$ such that ${\overrightarrow{\sigma }}^{\prime }\vDash \alpha$ with ${\pi }^{\prime }=\pi \left({\overrightarrow{\sigma }}^{\prime }\right)$. Moreover, by the construction in Lemma 3, we see that we may assume that ${\overrightarrow{\sigma }}^{\prime }$ is a finite-memory strategy. □

4. Cooperative Games

We now move on to investigate cooperative solution concepts, and in particular, the core. We start by studying the relationship between Nash equilibrium and the core in our context. We first establish that they are indeed different:

Proposition 5.

Let G be a game with $\left|\mathit{Ag}\right|=1$. Then, $\mathit{NE}\left(G\right)=\mathit{CORE}\left(G\right)$. However, if $\left|\mathit{Ag}\right|\ge 2$, then there exist games G such that $\mathit{NE}\left(G\right)⊈\mathit{CORE}\left(G\right)$, and games G such that $\mathit{CORE}\left(G\right)⊈\mathit{NE}\left(G\right)$.

Thus, the two concepts do not coincide beyond the one-player case. In fact, there are two-player games in which the set of Nash equilibria is empty, while the core is not, which demonstrates that the core is not a refinement of Nash equilibrium. Nor does the other direction hold: Nash equilibrium is not a refinement of the core. The following two games (Figure 5 and Figure 6) serve as witnesses to these claims.

As we can see in the game in Figure 5, $G_1$ has a Nash equilibrium that is not in the core (and in which both players get a mean-payoff of 0—cf., Player 1 choosing $\mathbf{D}$ at the initial state while Player 2 chooses $\mathbf{D}$ at the middle state), since the coalition containing both players has a beneficial deviation in which both players get a mean-payoff of 1. On the other hand, in the game in Figure 6, $G_2$ has a non-empty core (consider every possible memoryless strategy) while it has an empty set of Nash equilibria. Moreover, in both games, the detailed strategies can be implemented without memory.

Regarding memory requirements, as with Nash equilibrium, it may be that, in general, memoryless strategies are not enough to implement all equilibria in the cooperative setting. Indeed, there are games with a non-empty core in which no strategy profile in the core can be implemented in memoryless strategies only. Take, for instance, the game shown in Figure 2, with the weights changed to $w_i\left(\mathrm{mid}\right)=1$, $w_1\left(\mathrm{right}\right)=6=w_2\left(\mathrm{left}\right)$. Only if the two players collaborate, and alternate between left and right, they will get their optimal mean-payoffs (of 2). Clearly, such an optimal payoff for both players cannot be obtained using only memoryless strategies.

Another important (game-theoretic) question about cooperative games is whether they have a non-empty core, a property that holds for games with LTL goals and specifications [17]. However, that is not the case for mean-payoff games, at least for games with $\left|\mathrm{Ag}\right|>2$.

Proposition 6.

In mean-payoff games, if $\left|\mathit{Ag}\right|\le 2$, then the core is non-empty. For $\left|\mathit{Ag}\right|>2$, there exist games with an empty core.

Proof. 

If $\left|\mathrm{Ag}\right|=1$, because of Proposition 5, the core coincides with the set of Nash equilibria in one-player games, which is always non-empty. For two-player games, let $\overrightarrow{\sigma }=({\sigma }_1,{\sigma }_2)$ be any strategy profile. If $\overrightarrow{\sigma }$ is not in the core, then either Player 1, or Player 2, or the coalition consisting of both players has a beneficial deviation. If the latter is true, then there is a strategy profile, ${\overrightarrow{\sigma }}^{\prime }=({\sigma }_1^{\prime },{\sigma }_2^{\prime })$ such that ${\overrightarrow{\sigma }}^{\prime }{\succ }_i\overrightarrow{\sigma }$ for both $i\in \{1,2\}$. We repeat this process until the coalition of both players does not have a beneficial deviation. This must eventually be the case as each player’s payoff is capped by their maximum weight, so either they both reach their corresponding maximum weight, or there comes a point when they cannot beneficially deviate together. At this point, we must either be in the core, or either player 1 or player 2 has a beneficial deviation. If player j ($j\in \{1,2\}$) has a beneficial deviation, say ${\sigma }_j$, then any strategy profile $({\sigma }_j,{\sigma }_i)$, with $i\ne j$, that maximises Player i’s mean-payoff is in the core. Thus, for every two-player game, there exists some strategy profile that lies in the core.

However, for three-player mean-payoff games, in general, the core of a game may be empty. Consider the following three-player game G, where each player has two actions, $\mathbf{H},\mathbf{T}$, and there are four states, $P,R,B,Y$. The states are weighted for each player as follows:

${\mathit{w}}_{\mathit{i}}\left(\mathit{s}\right)$	1	2	3
P	$-1$	$-1$	$-1$
R	2	1	0
B	0	2	1
Y	1	0	2

If the game is in any state other than P, then no matter what set of actions is taken, the game will remain in that state. Thus, we only specify the transitions for the state P:

Ac	St
$(\mathbf{H},\mathbf{H},\mathbf{H})$	R
$(\mathbf{H},\mathbf{H},\mathbf{T})$	R
$(\mathbf{H},\mathbf{T},\mathbf{H})$	B
$(\mathbf{H},\mathbf{T},\mathbf{T})$	P
$(\mathbf{T},\mathbf{H},\mathbf{H})$	P
$(\mathbf{T},\mathbf{H},\mathbf{T})$	Y
$(\mathbf{T},\mathbf{T},\mathbf{H})$	B
$(\mathbf{T},\mathbf{T},\mathbf{T})$	Y

Thus, the structure of this game looks like the following graph (Figure 7):

Note that strategies are characterised by the state that the game eventually ends up in. If the players stay in P forever, then they can all collectively change strategy to move to one of $R,B,Y$, and each get a better payoff. Now, if the game ends up in R, then players 2 and 3 can deviate by playing $(\mathbf{T},\mathbf{H})$, and no matter what player 1 plays, they will be in state B, and will be better off. However, similarly, if the game is in B, then players 1 and 3 can deviate by playing $(\mathbf{T},\mathbf{T})$ to enter state Y, in which they both will be better off, regardless of what player 2 does. Furthermore, finally, if in Y, then players 1 and 2 can deviate by playing $(\mathbf{H},\mathbf{H})$ to enter R and will be better off regardless of what player 3 plays. Thus, no strategy profile lies in the core. □

Before proceeding, it is worth reflecting on the definition of the core. We can redefine this solution concept in the language of ‘beneficial deviations’. That is, we say that given a game G, a strategy profile $\overrightarrow{\sigma }$, a beneficial deviation by a coalition C, is a strategy vector ${\overrightarrow{\sigma }}_C^{\prime }$ such that for all complementary strategy profiles ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$, we have $\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }){\succ }_i\pi \left(\overrightarrow{\sigma }\right)$ for all $i\in C$. We can then say that $\overrightarrow{\sigma }$ is a member of the core, if there exists no coalition C which has a beneficial deviation from $\overrightarrow{\sigma }$. Note this formulation is entirely identical to our earlier definition of the core.

From a computational perspective, there is an immediate concern here—given a potential beneficial deviation, how can we verify that it is preferable to the status quo under all possible counter-responses? Given that strategies can be arbitrary mathematical functions, how can we reason about that universal quantification effectively? Fortunately, as we show in the following lemma, we can restrict our attention to memoryless strategies when thinking about potential counter-responses to players’ deviations:

Lemma 4.

Let G be a game, $C\subseteq \mathit{Ag}$ be a coalition and $\overrightarrow{\sigma }$ be a strategy profile. Further suppose that ${\overrightarrow{\sigma }}_C^{\prime }$ is a strategy vector such that for all memoryless strategy vectors ${\overrightarrow{\sigma }}_{\mathit{Ag}\backslash C}^{\prime }$, we have:

$$\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathit{Ag}\backslash C}^{\prime }){\succ }_i\pi \left(\overrightarrow{\sigma }\right).$$

Then, for all strategy vectors, ${\overrightarrow{\sigma }}_{\mathit{Ag}\backslash C}^{\prime }$, not necessarily memoryless, we have:

$$\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathit{Ag}\backslash C}^{\prime }){\succ }_i\pi \left(\overrightarrow{\sigma }\right).$$

Before we prove this, we need to introduce an auxiliary concept of two-player, turn-based, zero-sum, multi-mean-payoff games [32] (we will just call these multi-mean-payoff games moving forward). Informally, these are similar to two-player, turn-based, zero-sum mean-payoff games, except player 1 has k weight functions associated with the edges, and they are trying to ensure the resulting k-vector of mean-payoffs is component-wise greater than a vector threshold. Formally, a multi-mean-payoff game is a 5-tuple, $G=(V_1,V_2,v^0,E,w,z^k)$, where $V_1,V_2$ are sets of states controlled by players 1 and 2, respectively, with $V=V_1\cup V_2$ the state space, $v^0\in V$ the start state, $E\subseteq V\times V$ a set of edges and $w:E\to {\mathbb{Z}}^k$ a weight function, assigning to each edge a vector of weights.

The game is played by starting in the start state, $s^0\in S_i$, and player i choosing an edge $(s^0,s^1)$, and traversing it to the next state. From this new state, $s^1\in S_j$, player j chooses an edge and so on, repeating this process forever. Paths are defined in the usual way and the payoff of a path $\pi$, $\mathsf{pay}\left(\pi \right)$, is simply the vector $(\mathsf{mp}\left(w_1\left(\pi \right)\right),\dots ,\mathsf{mp}\left(w_k\left(\pi \right)\right))$. Finally, $z^k\in {\mathbb{Q}}^k$ is a threshold vector and player 1 wins if the ${\mathsf{pay}}_i\left(\pi \right)\ge z_i$ for all $i\in \{1,\dots ,k\}$, and loses otherwise. An important question associated with these games is whether player 1 can force a win. As shown in [32], this problem is co-NP-complete. Whilst we do not need to utilise this result right now, this sets us up to prove Lemma 4:

Proof. 

(Proof of Lemma 4) Let ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}$ be an arbitrary strategy and let $i\in C$ be an arbitrary agent. Denote $\pi \left(\overrightarrow{\sigma }\right)$ by $\pi$ and $\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })$ by ${\pi }^{\prime }$. We aim to show that ${\pi }^{\prime }{\succ }_i\pi$. Suppose instead it is the case that $\pi {⪰}_i{\pi }^{\prime }$. Thus, we have $\pi \left(\overrightarrow{\sigma }\right){⪰}_i\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })$. Considering this as a two-player multi-mean-payoff game, where player 1’s strategy is fixed and encoded into the game structure (i.e., player 1 follows ${\overrightarrow{\sigma }}_C^{\prime }$, but has no say in the matter), and the payoff threshold is $\mathsf{mp}\left(\pi \left(\overrightarrow{\sigma }\right)\right)$, then ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$ is a winning strategy for player 2 in this game. Now, by [32,33], if player 2 has a winning strategy, then they have a memoryless winning strategy. Thus, there is a memoryless strategy ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{″}$ such that $\pi \left(\overrightarrow{\sigma }\right){⪰}_i\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{″})$. However, this contradicts the assumptions of the lemma, and thus we must have ${\pi }^{\prime }{\succ }_i\pi$ (In [32], their winning condition relates to whether a player’s payoff is greater than or equal to a given vector. One can adapt this argument to show it is also true for strict inequalities). □

We now look at some complexity bounds for mean-payoff games in the cooperative setting. Having introduced beneficial deviations, let us consider the following decision problem:

BENEFICIAL-DEVIATION (BEN-DEV):

Given: Game G and strategy profile $\overrightarrow{\sigma }$.

Question: Is there $C\subseteq \mathrm{Ag}$ and ${\overrightarrow{\sigma }}_C^{\prime }\in {\mathsf{\Sigma }}_C$ such that for all ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }\in {\mathsf{\Sigma }}_{\mathrm{Ag}\backslash C}$ and for all $i\in C$, we have:

$$\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }){\succ }_i\pi \left(\overrightarrow{\sigma }\right)?$$

Using this new problem, we can prove the following statement:

Proposition 7.

Let G be a game, $\overrightarrow{\sigma }$ a strategy profile, and α a specification. Then, $(G,\overrightarrow{\sigma },\alpha )\in \mathrm{M}\mathrm{E}\mathrm{M}\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{S}\mathrm{H}\mathrm{I}\mathrm{P}$ if and only if $(G,\overrightarrow{\sigma })\notin \mathrm{B}\mathrm{E}\mathrm{N}-\mathrm{D}\mathrm{E}\mathrm{V}$ and $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$.

Proof. 

Proof follows directly from definitions. □

The above proposition characterises the MEMBERSHIP problem for cooperative games in terms of beneficial deviations, and, in turn, provides a direct way to study its complexity. In the remainder of this section we concentrate on the memoryless case.

Theorem 3.

For memoryless strategies, BEN-DEV is NP-complete.

Proof. 

First correctly guess a deviating coalition C and a strategy profile ${\overrightarrow{\sigma }}_C^{\prime }$ for such a coalition of players. Then, use the following three-step algorithm. First, compute the mean-payoffs that players in C get on $\pi \left(\overrightarrow{\sigma }\right)$, that is, a set of values $z_j^{*}={\mathsf{pay}}_j\left(\pi \left(\overrightarrow{\sigma }\right)\right)$ for every $j\in C$—this can be done in polynomial time simply by ‘running’ the strategy profile $\overrightarrow{\sigma }$. Then compute the graph $G\left[{\overrightarrow{\sigma }}_C^{\prime }\right]$, which contains all possible behaviours (i.e., strategy profiles) for $\mathrm{Ag}\backslash C$ with respect to $\overrightarrow{\sigma }$—this construction is similar to the one used in the proof of Theorem 1, that is, the game when we fix ${\overrightarrow{\sigma }}_C^{\prime }$, and can be done in polynomial time. Finally, we ask whether every path $\pi$ in $G\left[{\overrightarrow{\sigma }}_C^{\prime }\right]$ satisfies ${\mathsf{pay}}_j\left(\pi \right)>z_j^{*}$, for every $j\in C$—for this step, we can use Karp’s algorithm to answer the question in polynomial time for every $j\in C$. If every path in $G\left[{\overrightarrow{\sigma }}_C^{\prime }\right]$ has this property, then we accept; otherwise, we reject.

For hardness, we use a small variation of the construction presented in [34]. Let $P=\{x_1,\dots ,x_n\}$ be a set of atomic propositions. From a Boolean formula $\phi ={\bigwedge }_{1\le c\le m}{C}_c$ (in conjunctive normal form) over P—where each $C_c=l_{c1}\vee l_{c2}\vee l_{c3}$, and each literal $l_{ck}=x_j$ or $\neg x_j$, with $1\le k\le 3$, for some $1\le j\le n$—we construct $M=(\mathrm{Ag},\mathrm{St},s^0,{\left({\mathrm{Ac}}_i\right)}_{i\in \mathrm{Ag}},\mathrm{tr})$, an m-player concurrent game structure defined as follows:

• $\mathrm{Ag}=\{1,\dots ,m\}$
• $\mathrm{St}=\left\{x_v\right|1\le v\le n\}\cup \left\{x_v^{\prime }\right|1\le v\le n\}\cup \{y_0,y_n,y^0,y^{*}\}$
• $s^0=y^0$
• ${\mathrm{Ac}}_i=\{t,f\}$, for every $i\in \mathrm{Ag}$, and $\mathrm{Ac}={\mathrm{Ac}}_1\times \cdots \times {\mathrm{Ac}}_m$
• For tr, refer to the figure below, such that $T=\left\{(t_1,\dots ,t_m)\right\}$ and $F=\mathrm{Ac}\backslash T$.

The concurrent game structure so generated is illustrated in Figure 8.

With M at hand, we build a mean-payoff game using the following weight function:

• $w_i\left(x_v\right)=1$ if $x_v$ is a literal in $C_i$ and $w_i\left(x_v\right)=0$ otherwise, for all $i\in \mathrm{Ag}$ and $1\le v\le n$
• $w_i\left(x_v^{\prime }\right)=1$ if $\neg x_v$ is a literal in $C_i$ and $w_i\left(x_v^{\prime }\right)=0$ otherwise, for all $i\in \mathrm{Ag}$ and $1\le v\le n$
• $w_i\left(y_0\right)=w_i\left(y_n\right)=w_i\left(y^0\right)=w_i\left(y^{*}\right)=0$, for all $i\in \mathrm{Ag}$

Then, we consider the game G over M and any strategy profile (in memoryless strategies) such that $\overrightarrow{\sigma }\left(s^0\right)=y^{*}$. For any of such strategy profiles the mean-payoff of every player is 0. However, if $\phi$ is satisfiable, then there is a path in M, from $y_0$ to $y_n$, such that in such a path, for every player, there is a state in which its payoff is not 0. Thus, the grand coalition Ag has an incentive to deviate since traversing that path infinitely often will give each player a mean-payoff strictly greater than 0. Observe two things. Firstly, that only if the grand coalition Ag agrees, the game can visit $y_0$ after $y^0$. Otherwise, the game will necessarily end up in $y^{*}$ forever after. Secondly, because we are considering memoryless strategies, the path from $y_0$ to $y_n$ followed at the beginning is the same path that will be followed thereafter, infinitely often. Then, we can conclude that there is a beneficial deviation (necessarily for Ag) if and only if $\phi$ is satisfiable, as otherwise at least one of the players in the game will not have an incentive to deviate (because its mean-payoff would continue to be 0). Then, formally, we can conclude that $(G,\sigma )\in \mathrm{BEN}-\mathrm{DEV}$ if and only if $\phi$ is satisfiable. □

From Theorem 3 follows that checking if no coalition of players has a beneficial deviation with respect to a given strategy profile is a co-NP problem. More importantly, it also follows that MEMBERSHIP is also co-NP-complete.

Theorem 4.

For memoryless strategies, MEMBERSHIP is co-NP-complete.

Proof. 

Recall that given a game G, a strategy profile $\overrightarrow{\sigma }$, and an $\omega$-regular specification $\alpha$, we have $(G,\overrightarrow{\sigma },\alpha )\in \mathrm{M}\mathrm{E}\mathrm{M}\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{S}\mathrm{H}\mathrm{I}\mathrm{P}$ if and only if $(G,\overrightarrow{\sigma })\notin \mathrm{B}\mathrm{E}\mathrm{N}-\mathrm{D}\mathrm{E}\mathrm{V}$ and $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$. Thus, we can solve MEMBERSHIP simply by first checking $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$, which can be done in polynomial time and we reject if that check fails. If $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$, then we ask $(G,\overrightarrow{\sigma })\in \mathrm{B}\mathrm{E}\mathrm{N}-\mathrm{D}\mathrm{E}\mathrm{V}$ and accept if that check fails, and reject otherwise. Finally, since BEN-DEV is NP-hard, it follows from the above procedure that MEMBERSHIP is co-NP-hard, which concludes the proof of the statement. □

BEN-DEV can also be used to solve E-CORE in this case.

Theorem 5.

For memoryless strategies, E-CORE is in ${\mathsf{\Sigma }}_2^{\mathrm{P}}$.

Proof. 

Given any instance $(G,\alpha )$, we guess a strategy profile $\overrightarrow{\sigma }$ and check that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$ and that $(G,\overrightarrow{\sigma },\alpha )$ is not an instance of BEN-DEV. While the former can be done in polynomial time, the latter can be solved in co-NP using an oracle for BEN-DEV. Thus, we have a procedure that runs in NP${}^{\mathrm{co}-\mathrm{NP}}$ = NP${}^{\mathrm{NP}}$ = ${\mathsf{\Sigma }}_2^{\mathrm{P}}$. □

From Theorem 5 follows that A-CORE is in ${\mathsf{\Pi }}_2^{\mathrm{P}}$ and, more importantly, that even checking that the core has any memoryless solutions (but not necessarily that it is empty) is also in ${\mathsf{\Sigma }}_2^{\mathrm{P}}$. This result sharply contrasts with that for Nash equilibrium where the same problem lies in NP. More importantly, the result also shows that the (complexity) dependence on the type of coalitional deviation is only weak, in the sense that different types of beneficial deviations may be considered within the same complexity class, as long as such deviations can be checked with an NP or co-NP oracle. For instance, in [17] other types of cooperative solution concepts are defined, which differ from the one in this paper (known in the cooperative game theory literature as $\alpha$-core [7]) simply in the type of beneficial deviation under consideration. Another concept introduced in [17] is that of ‘fulfilled coalition’, which informally characterises coalitions that have the strategic power (a joint strategy) to ensure a minimum given payoff no matter what the other players in the game do. Generalising to our setting, from qualitative to quantitative payoffs, we introduce the notion of a lower bound: let $C\subseteq \mathrm{Ag}$ be a coalition in a game G and let ${\overrightarrow{z}}_C\in {\mathbb{Q}}^C$. We say that ${\overrightarrow{z}}_C=(z_1,\dots ,z_i,\dots ,z_{\left|C\right|})$ is a lower bound for C if there is a joint strategy ${\overrightarrow{\sigma }}_C$ for C such that for all strategies ${\overrightarrow{\sigma }}_{-C}$ for $\mathrm{Ag}\backslash C$, we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{-C})\right)\ge z_i$, for every $i\in C$.

Based on the definition above, we can prove the following lemma, which characterises the core in terms of paths where (mean-)payoffs can be ensured collectively, no matter any adversarial behaviour.

Lemma 5.

Let π be a path in G. There is $\overrightarrow{\sigma }\in \mathit{CORE}\left(G\right)$ such that $\pi =\pi \left(\overrightarrow{\sigma }\right)$ if and only if for every coalition $C\subseteq \mathit{Ag}$ and lower bound ${\overrightarrow{z}}_C\in {\mathbb{Q}}^C$ for C, there is some $i\in C$ such that $z_i\le {\mathsf{pay}}_i\left(\pi \right)$.

Proof. 

To show the left-to-right direction, suppose that there exists a member of the core $\overrightarrow{\sigma }\in \mathrm{CORE}\left(G\right)$ with $\pi =\pi \left(\overrightarrow{\sigma }\right)$ and suppose further that there is some coalition $C\subseteq \mathrm{Ag}$ and lower bound ${\overrightarrow{z}}_C\in {\mathbb{Q}}^C$ for C, such that for every $i\in C$ we have $z_i>{\mathsf{pay}}_i\left(\pi \right)$. Because ${\overrightarrow{z}}_C$ is a lower bound for C, and $z_i>{\mathsf{pay}}_i\left(\pi \right)$, for every $i\in C$, then there is a joint strategy ${\overrightarrow{\sigma }}_C$ for C such that for all strategies ${\overrightarrow{\sigma }}_{-C}$ for $\mathrm{Ag}\backslash C$, we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{-C})\right)\ge z_i>{\mathsf{pay}}_i\left(\pi \right)$, for every $i\in C$. Then, it follows that $(G,\overrightarrow{\sigma })\in \mathrm{BEN}-\mathrm{DEV}$, which further implies that $\overrightarrow{\sigma }$ cannot be in the core of G—a contradiction to our initial hypothesis.

For the right-to-left direction, suppose that there is $\pi$ in G such that for every coalition $C\subseteq \mathrm{Ag}$ and lower bound ${\overrightarrow{z}}_C\in {\mathbb{Q}}^C$ for C, there is $i\in C$ such that $z_i\le {\mathsf{pay}}_i\left(\pi \right)$. We then simply let $\overrightarrow{\sigma }$ be any strategy profile such that $\pi =\pi \left(\overrightarrow{\sigma }\right)$. Now, let $C=\{j,\dots ,k\}\subseteq \mathrm{Ag}$ be any coalition and ${\overrightarrow{\sigma }}_C^{\prime }$ be any possible deviation of C from $\overrightarrow{\sigma }$. Either ${\overrightarrow{z^{\prime }}}_C=({\mathsf{pay}}_j\left(\pi ({\overrightarrow{\sigma }}_{-C},{\overrightarrow{\sigma }}_C^{\prime })\right),\dots ,$${\mathsf{pay}}_k\left(\pi ({\overrightarrow{\sigma }}_{-C},{\overrightarrow{\sigma }}_C^{\prime })\right))$ is a lower bound for C or it is not.

If we have the former, by hypothesis, we know that there is $i\in C$ such that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C},{\overrightarrow{\sigma }}_C^{\prime })\right)\le {\mathsf{pay}}_i\left(\pi \right)$. Therefore, i will not have an incentive to deviate along with $C\backslash \left\{i\right\}$ from $\overrightarrow{\sigma }$, and as a consequence coalition C will not be able to beneficially deviate from $\overrightarrow{\sigma }$.

If, on the other hand, ${\overrightarrow{z^{\prime }}}_C$ is not a lower bound for C, then, by the definition of lower bounds, we know that it is not the case that ${\overrightarrow{\sigma }}_C^{\prime }$ is a joint strategy for C such that for all strategies ${\overrightarrow{\sigma }}_{-C}^{\prime }$ for $\mathrm{Ag}\backslash C$, we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{-C}^{\prime })\right)\ge {\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C},{\overrightarrow{\sigma }}_C^{\prime })\right)$, for every $i\in C$. That is, there exists $i\in C$ and ${\overrightarrow{\sigma }}_{-C}^{\prime }$ for $\mathrm{Ag}\backslash C$ such that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{-C}^{\prime })\right)<{\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C},{\overrightarrow{\sigma }}_C^{\prime })\right)$. We will now choose ${\overrightarrow{\sigma }}_{-C}^{\prime }$ so that, in addition, ${\mathsf{pay}}_i\left(\pi \right)\ge {\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{-C}^{\prime })\right)$ for some i.

Let ${\overrightarrow{z^{″}}}_C=({\mathsf{pay}}_j\left(\pi ({\overrightarrow{\sigma }}_{-C}^j,{\overrightarrow{\sigma }}_C^{\prime })\right),\dots ,{\mathsf{pay}}_k\left(\pi ({\overrightarrow{\sigma }}_{-C}^k,{\overrightarrow{\sigma }}_C^{\prime })\right))$ where ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C}^i,{\overrightarrow{\sigma }}_C^{\prime })\right)$ is defined to be ${min}_{{\overrightarrow{\sigma }}_{-C}^{\prime }\in {\mathsf{\Sigma }}_{-C}}{\mathsf{pay}}_i\left(\pi \left(({\overrightarrow{\sigma }}_{-C}^{\prime },{\overrightarrow{\sigma }}_C^{\prime })\right)\right)$. That is, ${\overrightarrow{\sigma }}_{-C}^i$ is a strategy for $\mathrm{Ag}\backslash C$ which ensures the lowest mean-payoff for i assuming that C is playing the joint strategy ${\overrightarrow{\sigma }}_C^{\prime }$. By construction ${\overrightarrow{z^{″}}}_C$ is a lower bound for C— since each $z_i^{″}={\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C}^i,{\overrightarrow{\sigma }}_C^{\prime })\right)$ is the greatest mean-payoff value that i can ensure for itself when C is playing ${\overrightarrow{\sigma }}_C^{\prime }$, no matter what coalition $\mathrm{Ag}\backslash C$ does—and therefore, by hypothesis we know that for some $i\in C$ we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-C}^i,{\overrightarrow{\sigma }}_C^{\prime })\right)\le {\mathsf{pay}}_i\left(\pi \right)$. As a consequence, as before, i will not have an incentive to deviate along with $C\backslash \left\{i\right\}$ from $\overrightarrow{\sigma }$, and therefore coalition C will not be able to beneficially deviate from $\overrightarrow{\sigma }$. Because C and ${\overrightarrow{\sigma }}_C^{\prime }$ where arbitrarily chosen, we conclude that $\overrightarrow{\sigma }\in \mathrm{CORE}\left(G\right)$, proving the right-to-left direction and finishing the proof. □

With this lemma in mind, we want to determine if a given vector, ${\overrightarrow{z}}_C$, is in fact a lower bound and importantly, how efficiently we can do this. That is, to understand the following decision problem:

LOWER-BOUND:

Given: Game G, coalition $C\subseteq \mathrm{Ag}$, and vector ${\overrightarrow{z}}_C\in {\mathbb{Q}}^{\mathrm{Ag}}$.

Question: Is ${\overrightarrow{z}}_C$ is a lower bound for C in G?

Using the MULTI-MEAN-PAYOFF-THRESHOLD decision problem introduced earlier, we can prove the following proposition:

Theorem 6.

LOWER-BOUND is co-NP-complete.

Proof. 

First, we show that LOWER-BOUND lies in co-NP by reducing it to MULTI-MEAN-PAYOFF-THRESHOLD. Suppose we have an instance, $(G,C,{\overrightarrow{z}}_C)$, and we want to determine if it is in LOWER-BOUND. We can do this by forming the following two-player, multi-mean-payoff game, $G^{\prime }=(V_1,V_2,v^0,E,w^{\prime },z^k)$, where:

• $V_1=\mathrm{St}$;
• $V_2=\mathrm{St}\times {\mathrm{Ac}}_C$;
• $v^0=s^0$;
• $E=\{(s,(s,{\mathrm{ac}}_C))\mid s\in \mathrm{St},{\mathrm{ac}}_C\in {\mathrm{Ac}}_C\}$
  $\phantom{E}\cup \{((s,{\mathrm{ac}}_C),\mathrm{tr}(s,({\mathrm{ac}}_C,{\mathrm{ac}}_{\mathrm{Ag}\backslash C})))\mid {\mathrm{ac}}_{\mathrm{Ag}\backslash C}\in {\mathrm{Ac}}_{\mathrm{Ag}\backslash C}\}$;
• $w^{\prime }:E\to {\mathbb{Z}}^{\left|C\right|}$, with $w_i^{\prime }(s,(s,{\mathrm{ac}}_C))=w_i\left(s\right)$ and,
  $\phantom{w^{\prime }:E\to {\mathbb{Z}}^{\left|C\right|},\mathrm{with}}{w}_i^{\prime }((s,{\mathrm{ac}}_C),\mathrm{tr}(s,({\mathrm{ac}}_C,{\mathrm{ac}}_{\mathrm{Ag}\backslash C})))=w_i\left(s\right)$;
• $z^{\left|C\right|}={\overrightarrow{z}}_c$.

Informally, the two players of the game are C and $\mathrm{Ag}\backslash C$, the vector weight function is given by aggregating the weight functions of C and the threshold is ${\overrightarrow{z}}_C$. Now, if in this game, player 1 has a winning strategy, then there exists some strategy ${\overrightarrow{\sigma }}_C$ such that for all strategies of player 2, ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}$, we have that $\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})$ is a winning path for player 1. However, this means that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})\right)\ge z_i$ for all $i\in C$. However, it is easy to verify that this implies that ${\overrightarrow{z}}_C$ is a lower bound for C in G. Conversely, if player 1 has no winning strategy, then for all strategies, ${\overrightarrow{\sigma }}_C$, there exists some strategy ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}$ such that $\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})$ is not a winning path. This is turn implies that for some $j\in C$, we have that ${\mathsf{pay}}_j\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})\right)<z_j$, which means that ${\overrightarrow{z}}_C$ is not a lower bound for C in G. Additionally, note that this construction can be performed in polynomial time, giving us the co-NP upper bound. For the lower bound, we go the other way and reduce from MULTI-MEAN-PAYOFF-THRESHOLD.

For the lower bound, we reduce from MULTI-MEAN-PAYOFF-THRESHOLD. Suppose we would like to determine if an instance G is in MULTI-MEAN-PAYOFF-THRESHOLD. Then we form a concurrent mean-payoff game, $G^{\prime }$, with $k+1$ players, where the states of $G^{\prime }$ coincide exactly with the states of G. In this game, only the 1st and $(k+1)$th player have any influence on the strategic nature of the game. If the game is in a state in $V_1$, player one can decide which state to move into next. Otherwise, if the game is in a state within $V_2$, then the $(k+1)$th player makes a move. Note we only allow moves that agree with moves allowed within G.

Now, in $G^{\prime }$, the first k players have weight functions corresponding to the k weight functions of player 1 in G. The last player can have any arbitrary weight function. With this machinery in place, we ask if $z^k$ is a lower bound for $\{1,\dots ,k\}$. In a similar manner of reasoning to the above, it is easy to verify that G is an instance of MULTI-MEAN-PAYOFF-THRESHOLD if and only if $z^k$ is a lower bound for $\{1,\dots ,k\}$ in the constructed concurrent mean-payoff game. Moreover, this reduction can be done in polynomial time and we can conclude that LOWER-BOUND is co-NP-complete. □

We have not presented any bounds for the complexity of E-CORE in the general case. One possible reason for the upper bounds remaining elusive to us is due to the fact that whilst in a multi-mean-payoff game, player 2 can act optimally with memoryless strategies, player 1 may require infinite memory [32,33]. Given the close connection between the core in our concurrent, multi-agent setting and winning strategies in multi-mean-payoff games, this raises computational concerns for the E-CORE problem. Additionally, in [35], the authors study the Pareto frontier of multi-mean-payoff games, and provide a way of constructing a representation of it, but this procedure has an exponential time dependency. The same paper also establishes ${\mathsf{\Sigma }}_2^P$-completeness for the polyhedron value problem. Both of these problems appear to be intimately related to the core, and we hope we might be able to use these results to gain more insight into the E-CORE in the future.

With this having been said, we conclude this section by establishing a link between traditional non-transferable utility (NTU) games and our mean-payoff games—as NTU games are very well studied, and there is a wealth of results relating to core non-emptiness in this setting [36,37,38], we hope that some of these results could be utilised in order to understand the core of mean-payoff games.

Formally, an n-person game with NTU is a function, $V:\mathcal{P}\left(\mathrm{Ag}\right)\to {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$, such that,

• For all $C\subseteq \mathrm{Ag}$, $V\left(C\right)$ is a non-empty, proper, closed subset of ${\mathbb{R}}^{\left|\mathrm{Ag}\right|}$;
• For all $C\subseteq \mathrm{Ag}$, if we have $x\in V\left(C\right)$ and $y\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$ such that $y_i\le x_i$ for all $i\in C$, then we have $y\in V\left(C\right)$;
• We have that $V\left(\mathrm{Ag}\right)\backslash {\bigcup }_{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)$ is non-empty and bounded.

We begin by giving a translation from mean-payoff games to NTU games. Let G be a mean-payoff game; then we define an NTU game, $G_{\mathrm{NTU}}=V:\mathcal{P}\left(\mathrm{Ag}\right)\to {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$ as follows. If $C\subseteq \mathrm{Ag}$, then,

$$V\left(C\right)=\left\{\overrightarrow{z}\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}\mid \exists {\overrightarrow{\sigma }}_C\forall {\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}\forall i\in C,{\mathsf{pay}}_i\left(\pi \left({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}\right)\right)\ge z_i\right\}.$$

In words, $V\left(C\right)$ consists of the set of lower bounds that C can force. Note that for an outcome $x\in V\left(C\right)$, the components $x_i$ for $i\in \mathrm{Ag}\backslash C$ do not matter—they can be arbitrary real numbers.

Lemma 6.

Let G be a game, and let $G_{\mathrm{NTU}}$ be the NTU game associated with G. Then $G_{\mathit{NTU}}$ is well-defined.

Proof. 

We need to show that the three conditions in the definition of an NTU game hold for $G_{\mathrm{NTU}}$.

For condition (1), we see that $V\left(C\right)$ is always non-empty by noting a coalition can always force an outcome where they achieve at least their worst possible payoff each (the vector made up of each player’s lowest weight in the game). The fact that $V\left(C\right)$ is closed follows from Theorem 4 of [35]. We also see that $V\left(C\right)$ is a proper subset of ${\mathbb{R}}^{\left|\mathrm{Ag}\right|}$, as the members of C can do no better than achieve their maximum weights.

For condition (2), suppose we have $x\in V\left(C\right)$, and $y\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$ with $y_i\le x_i$ for all $i\in C$. If $x\in V\left(C\right)$, then there exists some ${\overrightarrow{\sigma }}_C$, such that for all ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}$, we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})\right)\ge x_i$ for all $i\in C$. However, this in turn implies that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C,{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C})\right)\ge y_i$ for all $i\in C$. Thus, by definition, we have $y\in V\left(C\right)$.

For condition (3), let $p_j$ be the punishment value of the player j in the game G. Informally, the punishment value of a player j can be thought of as the worst payoff that the other players can inflict on that player. Alternatively, we can view the punishment value for player j as the best payoff they can guarantee themselves, no matter what the remaining players do—in this way, we can see that the punishment value is a maximal lower bound for a player.

Consider the vector $p\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$, where the jth component of this vector is the punishment value for player j. Naturally, this vector lies in $V\left(\mathrm{Ag}\right)$. Additionally, we claim that is does not lie in $\mathrm{int}V\left(\right\{i\left\}\right)$ for any $i\in \mathrm{Ag}$. For a contradiction, suppose there existed some $j\in \mathrm{Ag}$ with $p\in \mathrm{int}V\left(\right\{j\left\}\right)$. So there exists some $ϵ>0$, such that for all $0\le r<ϵ$, there exists some strategy, ${\sigma }_j^r$, such that for all counterstrategies, ${\overrightarrow{\sigma }}_{-j}$, we have ${\mathsf{pay}}_j\left(\pi ({\sigma }_j^r,{\overrightarrow{\sigma }}_{-j})\right)\ge p_j+r$. However, this implies player j can achieve a better payoff than their punishment value—a contradiction. Thus, we see that the set $V\left(\mathrm{Ag}\right)\backslash {\bigcup }_{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)$ is non-empty.

Finally, to see that $V\left(\mathrm{Ag}\right)\backslash {\bigcup }_{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)$ is bounded, we claim that it is contained in a closed ball of radius M, where M is defined to be:

$$M=\underset{\begin{array}{c}i\in \mathrm{Ag}\\ s\in \mathrm{St}\end{array}}{max}\left|w_i\left(s\right)\right|.$$

We show that if $x\in V\left(\mathrm{Ag}\right)$, then we either have $x\in {\bigcup }_{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)$ or $x\in \overline{B(0,M)}$, i.e., the closed ball of radius M, centred at the origin.

If $x\in V\left(\mathrm{Ag}\right)$, then by definition, we must have $x_i\le M$ for all $i\in \mathrm{Ag}$. Now, there are two possibilities: if we have $x_i\ge -M$ for all $i\in \mathrm{Ag}$, then we have $x\in \overline{B(0,M)}$. So instead suppose there exists some $i\in \mathrm{Ag}$ such that $x_i<-M$. In this case, letting $ϵ$ be any positive number such that $x_i+ϵ\le -M$, any strategy ${\sigma }_i$ has the property that for all counter-strategies ${\overrightarrow{\sigma }}_{-i}$, we have ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_{-i},{\sigma }_i)\right)\ge x_i+ϵ$. Thus, we have $x\in \mathrm{int}V\left(\right\{i\left\}\right)$. This implies that,

$$V\left(\mathrm{Ag}\right)\subseteq \bigcup _{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)\cup \overline{B(0,M)},$$

which in turn implies,

$$V\left(\mathrm{Ag}\right)\backslash \bigcup _{i\in \mathrm{Ag}}\mathrm{int}V\left(\left\{i\right\}\right)\subseteq \overline{B(0,M)},$$

yielding the result. □

Given that we can translate mean-payoff games into well-defined NTU games, it is natural to ask whether we can use traditional cooperative game theory in order to understand the core in our setting. Thus, we introduce the (classic) definition of the core for NTU games. In an NTU game, we say that an element $x\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$ is in the core if $x\in V\left(\mathrm{Ag}\right)$, and there exists no $C\subseteq \mathrm{Ag}$ and no $y\in V\left(C\right)$ such that $x_i<y_i$ for all $i\in C$. In the following result, we show that the core of a mean-payoff game, and the core of its corresponding NTU game are intimately related:

Lemma 7.

Let G be a mean-payoff game. Let $G_{\mathit{NTU}}$ be the NTU game associated with G. Then the core of G is non-empty if and only if the core of $G_{\mathit{NTU}}$ is non-empty.

Proof. 

First suppose that G has a non-empty core. Thus, there exists some strategy profile $\overrightarrow{\sigma }$ such that for all coalitions C and for all strategy vectors ${\overrightarrow{\sigma }}_C^{\prime }$, there exists some ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$ such that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })\right)\le {\mathsf{pay}}_i\left(\pi \left(\overrightarrow{\sigma }\right)\right)$ for some $i\in C$. Let $x\in {\mathbb{R}}^{\mathrm{Ag}}$ be such that $x_i={\mathsf{pay}}_i\left(\overrightarrow{\sigma }\right)$ for all $i\in \mathrm{Ag}$. Then by definition, we have $x\in V\left(\mathrm{Ag}\right)$. We claim that x is in the core of $G_{\mathrm{NTU}}$. Suppose there is some $C\subseteq \mathrm{Ag}$ and a $y\in V\left(C\right)$ such that $x_i<y_i$ for all $i\in C$. Thus, there exists some ${\overrightarrow{\sigma }}_C^{\prime }$ such that for all ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$, such that ${\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })\right)\ge y_i>x_i={\mathsf{pay}}_i\left(\overrightarrow{\sigma }\right)$ for all $i\in C$. However, this implies that $\overrightarrow{\sigma }$ is not in the core of G, which is a contradiction. Thus, x is in the core of $G_{\mathrm{NTU}}$.

Conversely, suppose that $G_{\mathrm{NTU}}$ has an empty core. Thus, there exists some $x\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}$ such that $x\in V\left(\mathrm{Ag}\right)$, such that there exists no $C\subset \mathrm{Ag}$ and no $y\in V\left(C\right)$ with $x_i<y_i$ for all $i\in C$. Since $x\in V\left(\mathrm{Ag}\right)$, there exists some strategy $\overrightarrow{\sigma }$ such that ${\mathsf{pay}}_i\left(\pi \left(\overrightarrow{\sigma }\right)\right)\ge x_i$ for all $i\in \mathrm{Ag}$. We claim that $\overrightarrow{\sigma }$ is in the core of G. If it were not, then there would exist some coalition C, and some strategy vector ${\overrightarrow{\sigma }}_C^{\prime }$ such that for all strategy vectors ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$, we have $\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }){\succ }_i\pi \left(\overrightarrow{\sigma }\right)$ for all $i\in C$. We then define $y\in {\mathbb{R}}^{\mathrm{Ag}}$ by setting $y_i={min}_{{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }}{\mathsf{pay}}_i\left(\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })\right)$ for $i\in C$ and setting $y_i=0$ for $i\in \mathrm{Ag}\backslash C$. Then we have that $y\in V\left(C\right)$ by definition. Since for all ${\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime }$, we have that $\pi ({\overrightarrow{\sigma }}_C^{\prime },{\overrightarrow{\sigma }}_{\mathrm{Ag}\backslash C}^{\prime })$ is strictly preferred to $\pi \left(\overrightarrow{\sigma }\right)$ by all players in C, we must have that $y_i>x_i$ for all $i\in C$. However, this contradicts the fact that x is in the core of $G_{\mathrm{NTU}}$. Thus, we must have that $\overrightarrow{\sigma }$ is in the core of G. □

As stated previously, we have been unable to determine the complexity of E-CORE in the setting of mean-payoff games. However, given the above result, we suggest a route which may bear fruits in the future. In [36,37,38,39,40], the authors reason about the core of cooperative games (in both the transferable utility and non-transferable utility settings) by appealing to the notion of a balanced set. In [37], the authors generalise this by introducing the notion of $\pi$-balancedness. Let $\pi =\{{\pi }_C\in {\mathbb{R}}^{\left|\mathrm{Ag}\right|}\mid C\subseteq \mathrm{Ag}\}$ be a collection of vectors such that:

• For all $C\subseteq \mathrm{Ag}$, we have ${\pi }_C\ne \mathbf{0}$;
• For all $C\subseteq \mathrm{Ag}$, and for all $i\notin C$, we have ${\pi }_{C,i}=0$;
• For all $C\subseteq \mathrm{Ag}$, and for all $i\in C$, we have ${\pi }_{C,i}\ge 0$,

and let $\mathcal{C}\subseteq \mathcal{P}\left(\mathrm{Ag}\right)$ be a collection of coalitions. We say that $\mathcal{C}$ is $\pi$-balanced if there exist balancing weights, ${\lambda }_C>0$, for each $C\subseteq \mathrm{Ag}$ such that:

$$\sum _{C\in \mathcal{C}}{\lambda }_C{\pi }_C={\pi }_{\mathrm{Ag}}.$$

We then say that an NTU game, V, is $\pi$-balanced if whenever $\mathcal{C}$ is a $\pi$-balanced collection, we have:

$$\bigcap _{C\in \mathcal{C}}V\left(C\right)\subseteq V\left(\mathrm{Ag}\right).$$

In [37], the authors show that if there exists some $\pi$ such that V is $\pi$-balanced, then V has a non-empty core. The condition of $\pi$-balancedness translates readily over to the setting of mean-payoff games, and so we see that if such a game is $\pi$-balanced, then it has a non-empty core. This suggest a (sound, but not complete) algorithm for detecting if a mean-payoff game has a non-empty core; somehow guess a polynomial-sized $\pi$, use a linear program to calculate the corresponding balancing weights, and then use an co-NP oracle to verify there exists no $\pi$-balanced collection such that ${\bigcap }_{C\in \mathcal{C}}V\left(C\right)\subseteq V\left(\mathrm{Ag}\right)$. Obviously, this is not a rigorous argument, but is suggestive of what a possible solution may look like.

Additionally, whilst $\pi$-balancedness is a sufficient condition for core non-emptiness, it is not necessary. However, in [37], the authors strengthen the condition of $\pi$-balancedness in the setting of convex-valued NTU games, to obtain a necessary and sufficient result. Given in that mean-payoff games, the outcomes that a coalition can achieve can be expressed as a union of convex sets, this approach seems promising. However, we have been unable to yield any results via this route.

5. Weighted Reactive Module Games

One problem with concurrent game structures as we have worked with them so far is that they are extremely verbose. The transition function, $\mathrm{tr}:\mathrm{St}\times {\mathrm{Ac}}_1\times \cdots \times {\mathrm{Ac}}_{\left|\mathrm{Ag}\right|}\to \mathrm{St}$ is a total function, so it has size ${\left|\mathrm{Ac}\right|}^{\left|\mathrm{Ag}\right|}$. Thus, the size of the game scales exponentially with the number of the agents. In Example 1, the underlying concurrent game structure has a size of 429,981,696. If we are ever to have computational tools to support the decision problems described in this paper, then such “extensive” representations are not viable: we will require compact frameworks to represent games.

One natural framework we can use to induce succinctness is that of Reactive Modules [41]. Specifically, we modify the Reactive Module Games of [18] with weights on the guarded commands. We begin by walking through some preliminaries.

Reactive modules games do not use the full power of reactive modules, but instead use a subset of the reactive modules syntax, namely the simple reactive modules language (SRML) [42]. In SRML terms, agents are described by modules, which in turn consist of a set of variables controlled by the module, along with a set of guarded commands. Formally, given a set of propositional variables $\Phi$, a guarded command g is an expression of the form:

$$\phi ⇝x_1^{\prime }:={\psi }_1;\dots ;x_k^{\prime }:={\psi }_k,$$

where $\phi$ and each ${\psi }_i$ are propositional formulae over $\Phi$ and each $x_j$ also lies in $\Phi$. We call $\phi$ the guard of g and denote it by $\mathrm{guard}\left(g\right)$, and we call the variables (the $x_j$s) on the right-hand-side of g the controlled variables of g, denoted by $\mathrm{ctr}\left(g\right)$. The idea is that under a given valuation of a set of variables, $v\subseteq \Phi$, each module has a set of commands for which $\mathrm{guard}\left(g\right)$ is true (we say that they are enabled for execution). Each module can then choose one enabled command, g, and reassign the variables in $\mathrm{ctr}\left(g\right)$ according to the assignments given on the right hand side of g. For instance, if $\phi$ were true, then the above guarded command could be executed, setting each $x_j$ to the truth value of ${\psi }_j$ under v. Only if no g is enabled, a special guarded command $g^{\mathbf{skip}}$—which does not change the value of any controlled variable—is enabled for execution so that modules always have an action they can take.

To define the size of a guarded command, we first define the size of a propositional formula, $\phi$, denoted $\left|\phi \right|$, to be the number of logical connectives it contains. Then the size of a guarded command g, written $\left|g\right|$, is given by $\left|\mathrm{guard}\left(g\right)\right|+\left|\mathrm{ctr}\left(g\right)\right|$.

Given a set of propositional variables, $\Phi$, a simple reactive module, m, is a tuple $(\mathsf{\Psi },I,U)$, where:

• $\mathsf{\Psi }\subseteq \Phi$ is a set of propositional variables;
• I is a set of initialisation guarded commands, where for all $g\in I$, we have $\mathrm{guard}\left(g\right)=\top$ and $\mathrm{ctr}\left(g\right)\subseteq \mathsf{\Psi }$.
• U is a set of update guarded commands, where for all $g\in U$, $\mathrm{guard}\left(g\right)$ is a propositional formula over $\Phi$ and $\mathrm{ctr}\left(g\right)\subseteq \mathsf{\Psi }$.

After defining a simple reactive module m, we introduce an additional command, $g^{\mathbf{skip}}$, of the form,

$$\underset{g\in U}{\bigwedge }\neg \mathrm{guard}\left(g\right)⇝\varnothing$$

with $\mathrm{ctr}\left(g\right)=\mathsf{\Psi }$. The empty set on the right hand side of the guarded command means that no variables are changed. We introduce this extra guarded command so that at each stage, every module has some action they can take. However, this is not obligatory, and if we define a reactive module where we can prove at each step, there will be an available action, then introducing $g^{\mathbf{skip}}$ is not necessary. With this added, the size of a reactive module, $\left|m\right|$, is given by the sum of the sizes of its constituent guarded commands.

Given this, an SRML arena, A, is a tuple $A=(\mathrm{Ag},\Phi ,{\left\{m_i\right\}}_{i\in \mathrm{Ag}})$, where Ag is a finite, non-empty set of agents, $\Phi$ is a set of propositional variables and each $m_i$ is a simple reactive module $m_i=({\Phi }_i,I_i,U_i)$ such that ${\left\{{\Phi }_i\right\}}_{i\in \mathrm{Ag}}$ is a partition for $\Phi$. We define the size of an arena to be sum of the sizes of the modules within it.

With this syntactic machinery in place, we are finally ready to describe the semantics of SRML arenas. We give a brief, high-level description here—for details, please refer to [18,42].

With this syntactic machinery in place, we are finally ready to describe the semantics of SRML arenas. Let $v\subseteq \Phi$ be a valuation and let $i\in \mathrm{Ag}$ be an agent. We let ${\mathrm{enabled}}_i\left(v\right)$ denote the guarded commands available to agent i under the valuation v. Formally, we have:

$${\mathrm{enabled}}_i\left(v\right)=\{g\in U_i\mid vs.\vDash \mathrm{guard}\left(g\right)\}.$$

We then define $\mathrm{enabled}\left(v\right)={\mathrm{enabled}}_1\left(v\right)\times \cdots \times {\mathrm{enabled}}_{\left|\mathrm{Ag}\right|}\left(v\right)$. Given that each $U_i$ contains a command of the form $g_i^{\mathbf{skip}}$ as described previously, we can see that each ${\mathrm{enabled}}_i\left(v\right)$ is always non-empty.

We also define ${\mathrm{exec}}_i(g,v)$, which given a valuation v, is the valuation of ${\Phi }_i$ given by executing $g=\phi ⇝x_1^{\prime }:={\psi }_1;\dots ;x_k^{\prime }:={\psi }_k$. Specifically, we have:

$${\mathrm{exec}}_i(g,v)=(v_i\backslash \mathrm{ctr}\left(g\right))\cup \{x_j\mid vs.\vDash {\psi }_j\}.$$

Thus, upon executing g, agent i’s variables are reassigned according to the valuations of the propositional formulae on its right hand side. However, with multiple agents, there is some strategic interaction, and we wish to understand how the actions of all the agents affect the state of the system. As such, we define joint guarded commands, which are simply a selection of guarded commands, one for each agent. That is, we write $J=g_1\times \cdots \times g_{\left|\mathrm{Ag}\right|}$. Similarly to before, we set $\mathrm{exec}(J,v)={\mathrm{exec}}_1(g_1,v)\cup \dots \cup {\mathrm{exec}}_{\left|\mathrm{Ag}\right|}(g_{\left|\mathrm{Ag}\right|},v)$.

We are now ready to describe how arenas ‘play out’. Initially, each agent i picks a guarded command $g_i^0\in I_i$ and this sets each of their variables accordingly, inducing a valuation $v^0=\mathrm{exec}(J^0,\Phi )$. The agents then each pick a guarded command $g_i^1\in {\mathrm{enabled}}_i\left(v^0\right)$ and execute them, inducing another valuation, $v^1=\mathrm{exec}(J^1,v^0)$. They repeat this ad infinitum. As such, this induces a path of the game. However, unlike before, where we defined paths over states of the game, here, we define paths over joint guarded commands, $\pi :\mathbb{N}\to (I_1\cup U_1)\times \cdots \times (I_{\left|\mathrm{Ag}\right|}\cup U_{\left|\mathrm{Ag}\right|})$. Whilst, superficially, this may look like a departure from our previous convention, it is not. Given that these games are entirely deterministic, if we know the sequence of joint guarded commands that have been taken, we know infer sequence of states. Additionally, knowing the sequence of joint guarded commands provides us with more information that knowing the sequence of states—a state may have multiple joint guarded commands the lead to it. All of the techniques we developed before transfer readily to this new setting, so take it for granted that there is a straightforward link between the two approaches and will not comment on it further.

We can now define weighted reactive module games. A weighted reactive module game (WRMG), $G=(A,{\left\{w_i\right\}}_{i\in \mathrm{Ag}})$, is an SRML arena, $A=(\mathrm{Ag},\Phi ,{\left\{m_i\right\}}_{i\in \mathrm{Ag}})$, along with a set of weight functions, with $w_i:I_i\cup U_i\to \mathbb{Z}$. That is, each module has an assigned weight function that maps commands to integers. As before, a player’s payoff is given by the mean-payoff of the weights attached to a path.

Note that we are effectively assigning weights to transitions, rather than to states as we did before. This is not a huge conceptual shift, and moving between the two representations (weights on states and weights on transitions) is a relatively straightforward transformation.

Finally, we need to define $\omega$-regular specifications in the context of WRMGs. Sets of states are already conveniently parameterised by the propositional variables of $\Phi$, so we introduce specifications which are Boolean combinations of atoms of the form $\mathrm{Inf}\left(p\right)$ with $p\in \Phi$. Moreover, since if p is a proposition, then $\neg p$ is also a proposition, we use the shorthand $\mathrm{Inf}(\neg p)$, with the obvious interpretation, in place of the previous $\mathrm{Inf}\left(\overline{F}\right)$ notation. The semantics of these specifications are defined in a nearly identical way to before.

Before considering the decision problems relating to WRMGs, let us now walk through an example to demonstrate their conciseness and utility of WRMGs. We do this by revisiting Example 1.

Example 2.

In Example 1, the state of the game is entirely described by the position of each of the four robots, whether they are holding a parcel or not, and whether they have crashed. Thus, we define four reactive modules $m_1,\dots ,m_4$ with $m_i=({\Phi }_i,I_i,U_i)$ as follows—we set ${\Phi }_i=\{x_{i,1},\dots ,x_{i,12},p_i,d_i,c_i\}$, where the $x_i$ model which node the robot is in, numbered top-to-bottom, left-to-right with respect to the diagram, $p_i$ denotes if the robot is carrying a parcel or not, and $c_i$ denotes if the robot has crashed or not. With this defined, we can define one initialisation command for each robot:

$$\top ⇝x_{i,1}^{\prime }:=\perp ;\dots x_{i,n}^{\prime }:=\top ;\dots ;x_{i,12}^{\prime }:=\perp ;p_i^{\prime }:=\perp ;c_i^{\prime }:=\perp \left[0\right],$$

where n is appropriately set, given the starting position of the robot. Additionally, the $\left[0\right]$ at the end of the guarded command denotes the weight rewarded for performing that command. Then for each agent i and edge $(x_n,x_m)$ of the graph, we define a guarded command,

$$\neg c_i\wedge x_{i,n}⇝x_{i,n}^{\prime }:=\perp ;x_{i,m}^{\prime }:=\top \left[0\right].$$

We also model picking up and delivering a parcel, as well as crashing into another robot. We do this with the following commands:

$$\begin{array}{cc}\hfill \neg c_i\wedge \neg p_i\wedge (x_{i,1}\vee x_{i,2})⇝p_i\prime & \hfill :=\top \left[0\right],\hfill \\ \hfill \neg c_i\wedge p_i\wedge (x_{i,11}\vee x_{i,12})⇝p_i\prime & \hfill :=\perp \left[1\right],\hfill \\ \hfill \neg c_i\wedge x_{i,n}\wedge (x_{j,n}\vee x_{k,n}\vee x_{l_n})⇝c_i^{\prime }& \hfill :=\top [-999]\hfill \\ \hfill c_i⇝c_i^{\prime }& \hfill :=\top [-999]\hfill \end{array}$$

where i ranges over players; j, k and l range over the other players; and j ranges from 1 to 12. We also have the $g^{\mathit{skip}}$ command from before, so the robot can stay still on a node for a time step.

We can also rewrite the specification of Equation 1 in our new setting as follows:

$$\underset{i\in \left[4\right]}{\bigwedge }\mathit{Inf}\left(p_i\right)\wedge \mathit{Inf}(\neg p_i).$$

It is easy to see that this setup models the example from before, is exponentially more concise, requiring 52 guarded commands in total, and is natural to work with. Note that we could save even more space by encoding the robots positions in binary, at the expense of making our guarded commands slightly more complicated. Whilst this technique may be useful for larger systems, we give a unary encoding here for clarity. Moreover, our specification has not increased in size.

With WRMGs now adequately motivated, the main decision problem to consider then is the following:

WRMG-E-NASH:

Given: WRMG G, and $\omega$-regular specification $\alpha$.

Question: Does there exist a $\overrightarrow{\sigma }\in \mathrm{NE}\left(G\right)$ such that $\pi \left(\overrightarrow{\sigma }\right)\vDash \alpha$?

Theorem 7.

TheWRMG-E-NASHproblem lies in NEXPTIME and is EXPTIME-hard.

Proof. 

The idea is to ‘blow up’ the simple reactive module arena into a concurrent game structure, then apply the same techniques as before. Explicitly, given a WRMG, $G=(\mathrm{Ag},\Phi ,$${\left\{m_i\right\}}_{i\in \mathrm{Ag}},{\left\{w_i\right\}}_{i\in \mathrm{Ag}})$, we form a graph $\mathcal{G}=(V,E)$ as follows. We set $V=\mathcal{P}\left(\Phi \cup \left\{p_{\mathrm{init}}\right\}\right)$ and then introduce an edge, $e=(v,w)$ if there exists some joint guarded update command J such that $\mathrm{exec}(J,v)=w$. We also introduce edges $(\left\{p_{\mathrm{init}}\right\},v)$, if there exists some joint guarded initialisation command J such that $\mathrm{exec}(J,\varnothing )=v$. We then form a concurrent game structure, $G^{\prime }=({\mathrm{Ag}}^{\prime },\mathrm{St},s^0,{\left({\mathrm{Ac}}_i\right)}_{i\in \mathrm{Ag}},\mathrm{tr})$ in the natural way. We set ${\mathrm{Ag}}^{\prime }=\mathrm{Ag}$ and $\mathrm{St}=V$. We also set $s^0=\left\{p_{\mathrm{init}}\right\}$. The actions for each player, ${\mathrm{Ac}}_i$, consist of their respective guarded commands and the transition function corresponds to the edges of the graph. The weights of the guarded commands are attached to the weights of the transitions in the concurrent game structure (Whilst our analysis of E-NASH is couched in terms of weights on the states, in the proof of it, we modify the concurrent game structure so the weights exist on the transitions instead. Thus, this reduction does not need to introduce extra states to convert the weights on the transitions into weights on the states—we simply ‘cut the corner’ instead).

We also need to transform the $\omega$-regular specification on the weighted reactive module games, $\alpha$, to a specification on the concurrent game structure, ${\alpha }^{\prime }$. To do this, for each propositional variable p, define a subset $F_p\subseteq \mathrm{St}$ where $v\in F_p$ if $p\in v$. Then we simply replace every occurrence of $\mathrm{Inf}\left(p\right)$ in $\alpha$ with $\mathrm{Inf}\left(F_p\right)$.

It should be apparent that $(G^{\prime },{\alpha }^{\prime })$ exhibits the same behaviours and qualities as $(G,\alpha )$ and is exponential in size relative to $(G,\alpha )$. With this expansion complete, we can straightforwardly apply the NP algorithm of Theorem 2, immediately giving us the NEXPTIME upper bound.

To show hardness, we need to introduce another decision problem, PEEK-G4 [43]. A PEEK-G4 instance is a tuple $(X_1,X_2,X_3,\phi )$. Here, $X_1$ and $X_2$ are finite, disjoint, non-empty sets of propositional variables, $X_3\subseteq X_1\cup X_2$ gives the variables that are true in the initial state, and $\phi$ is a propositional formula over $X_1\cup X_2$. The game is played by starting in the configuration given by $X_3$. The players then alternate, either choosing to toggle a variable they control, or skipping a go. If on their turn, a player can make $\phi$ true, then they win and the game is over. The decision problem associated with this game is to determine if agent 2 has a winning strategy for a give tuple $(X_1,X_2,X_3,\phi )$.

We reduce an instance of PEEK-G4, $(X_1,X_2,X_3,\phi )$, to an instance of WRMG-E-NASH, $(\mathrm{Ag},\Phi ,{\left\{m_i\right\}}_{i\in \mathrm{Ag}},{\left\{w_i\right\}}_{i\in \mathrm{Ag}})$. We set

$$\mathrm{Ag}=\{0,1,2\}\mathrm{and}\Phi =(X_1\cup X_2)\cup \{{\mathsf{turn}}_{\mathrm{p}},{\mathsf{turn}}_{\mathrm{u}},{\mathsf{win}}_1,{\mathsf{win}}_2\}$$

and introduce three reactive modules over $\Phi$, $m_0$, $m_1$ and $m_2$. The modules $m_1$ and $m_2$ represent the two players, 1 and 2, whilst $m_0$ represents an umpire/officiator, who keeps track of whose turn it is and whether anyone has won the game yet.

The two players alternate, with the umpire enforcing this. That is, for $m_0$, we have ${\Phi }_0=\{{\mathsf{turn}}_{\mathrm{p}},{\mathsf{turn}}_{\mathrm{u}},{\mathsf{win}}_1,{\mathsf{win}}_2\}$, with the following single initialisation command:

$$\top ⇝{\mathsf{turn}}_{\mathrm{p}}^{\prime }:=\top ;{\mathsf{turn}}_{\mathrm{u}}^{\prime }:=\perp ;{\mathsf{win}}_1^{\prime }:=\perp ;{\mathsf{win}}_2^{\prime }:=\perp .$$

The ${\mathsf{turn}}_{\mathrm{p}}$ variable keeps track of which player’s turn it is, and the ${\mathsf{turn}}_{\mathrm{u}}$ variable determined if it is the umpire’s turn. Finally, the two $\mathsf{win}$ variables keep track of whether anyone has won yet. The update commands for $m_0$ are defined as follows:

$$\begin{array}{cc}\hfill \neg ({\mathsf{win}}_1\vee {\mathsf{win}}_2)\wedge \neg {\mathsf{turn}}_{\mathrm{u}}& ⇝{\mathsf{turn}}_{\mathrm{u}}^{\prime }:=\top ,\hfill \\ \hfill {\mathsf{turn}}_{\mathrm{u}}\wedge \neg \phi & ⇝{\mathsf{turn}}_{\mathrm{u}}^{\prime }:=\perp ;{\mathsf{turn}}_{\mathrm{p}}^{\prime }:=\neg {\mathsf{turn}}_{\mathrm{p}},\hfill \\ \hfill {\mathsf{turn}}_{\mathrm{u}}\wedge \phi \wedge {\mathsf{turn}}_{\mathrm{p}}& ⇝{\mathsf{win}}_1^{\prime }:=\top ,\hfill \\ \hfill {\mathsf{turn}}_{\mathrm{u}}\wedge \phi \wedge \neg {\mathsf{turn}}_{\mathrm{p}}& ⇝{\mathsf{win}}_2^{\prime }:=\top ,\hfill \\ \hfill {\mathsf{win}}_1\vee {\mathsf{win}}_2& ⇝\varnothing .\hfill \end{array}$$

There is a lot going on in the above set of commands, so let us break it down: the first two commands say that play alternates between the players and the umpire—one of the players makes a move, then play goes to the umpire, who checks for wins, then play passes to the other player, and so on. The third and fourth commands say that if a player managed to satisfy $\phi$ on their last turn, then they have won, and the umpire can declare them the winner. The final command says that the umpire will just stay put and keep their variables constant once one player has won.

The two player modules are symmetric, so we define $m_1$, with $m_2$ being formed in a similar way. For $m_1$, we have ${\Phi }_1=X_1\cup \left\{{\mathsf{win}}_2\right\}$, with a single initialisation command:

$$\top ⇝x_{i_1}^{\prime }:=\top ;\dots ;x_{i_n}^{\prime }:=\top ,$$

where ${\left\{x_{i_k}\right\}}_{1\le k\le n}=X_1\cap X_3$. This simply says that player 1 must set their variables as dictated by $X_3$ Then for every variable $x_i\in X_1$, we introduce an update guarded command of the following form,

$$\neg ({\mathsf{win}}_1\vee {\mathsf{win}}_2)\wedge {\mathsf{turn}}_{\mathrm{p}}\wedge \neg {\mathsf{turn}}_{\mathrm{u}}⇝x_i^{\prime }:=\neg x_i.$$

This says that if neither player has won yet, and it is player 1’s turn to play, then they can toggle one of the variables they control. Additionally, we need to allow a player to be able not to change any of their variables, and we do this with the following guarded command,

$$\neg ({\mathsf{win}}_1\vee {\mathsf{win}}_2)⇝\varnothing .$$

Note that player 1 can always play this command, even when it is not their turn. Finally, we need to introduce two more guarded commands that the players take when the game is over:

$$\begin{array}{cc}\hfill {\mathsf{win}}_1& ⇝\varnothing \hfill \\ \hfill {\mathsf{win}}_2& ⇝\varnothing .\hfill \end{array}$$

If someone has won the game, then both players ignore the turn system and have the lone option of following exactly one of the above two commands forever.

With this defined, we need to introduce weight functions for each of the three players. For the umpire, we give a constant weight function (so they are ambivalent between all outcomes) and for player 1 we give them a weight function $w_1$ such that,

$$\begin{array}{cc}\hfill w_1({\mathsf{win}}_1⇝\varnothing )& =1,\hfill \\ \hfill w_1({\mathsf{win}}_2⇝\varnothing )& =-1,\hfill \\ \hfill w_1\left(g\right)& =0,\mathrm{for}\mathrm{all}\mathrm{other}g.\hfill \end{array}$$

with player 2’s weight function being defined in the dual way. Finally, the $\omega$-regular specification is simply $\alpha =\mathrm{Inf}\left({\mathsf{win}}_2\right)$. We claim that player 2 has a winning strategy in PEEK-G4 if and only if there exists a Nash equilibrium in the weighted reactive module game that models the given specification. Moreover, it is patently clear that this construction can be performed in polynomial time.

First suppose that player 2 has a winning strategy in PEEK-G4. Then $m_2$ can play this strategy and $m_1$ can play any arbitrary strategy and eventually the game will end up in state ${\mathsf{win}}_2$. From here, each player only has one guarded command they can play, meaning player 2 will get a payoff of 1 and player 1 will get a payoff of $-1$. This is a Nash equilibrium. No matter what $m_1$ plays, they are unable to force a win against player 2’s winning strategy and so have no incentive to deviate. Additionally, player 2 is achieving their maximum payoff, and thus have no motivation to deviate. This strategy profile also models $\alpha$.

Conversely, suppose there exists some Nash equilibrium which models $\alpha$. This implies that the game eventually ends up in ${\mathsf{win}}_2$, and so player 2 has a payoff of 1 and player 1 a payoff of $-1$ in this equilibrium. The fact that this is an equilibrium implies that no matter what player 1 plays, they cannot increase their payoff, i.e. they always lose against the strategy player 2 is playing. This implies that player 2 has a winning strategy in PEEK-G4 and this concludes the proof. □

6. Related Work

6.1. Contributions of This Paper

This paper introduces $\omega$-regular specifications as a natural, expressive, computationally tractable way of reasoning qualitatively about concurrent games. In particular, we establish several results within the rational verification framework, the most important of which are the complexity bounds for the E-NASH, E-CORE and WRMG-E-NASH problems.

6.2. Mean-Payoff Games

Mean-payoff games are a useful tool in the analysis of computer systems. Most work has been devoted to the study of two-player zero-sum games, which can be solved in NP ∩ co-NP [6].

Beyond such games, two kinds of mean-payoff games have been studied: multi-player mean-payoff games, whose solution was studied with respect to Nash equilibria [5], and two-player multi-mean-payoff games [32], where the focus is on the computation of winning strategies for either player, a problem that can be solved in co-NP in the general case, and in NP for memoryless strategies.

6.3. Combined Qualitative and Quantitative Reasoning

Combining qualitative and quantitative reasoning has mainly been done by modifying players’ mean-payoff with some qualitative measure. In [44], the authors consider two-player, zero-sum games, where on each path of the game, every player is assigned a two-size tuple (parity goal, mean-payoff), where each player’s payoff is $-\infty$ if the parity goal is not met, and the mean-payoff otherwise. In a similar setting, [30,31] look at multi-player concurrent games with lexicographic preferences over (parity/LTL goal, mean-payoff) tuples and look at the decision problem of determining if there exists some finite state strict $ϵ$ Nash equilibrium. Additionally, [15] considered multi-player concurrent games where the players have mean-payoff goals, and the question is whether there is some Nash equilibrium which models some temporal specification.

6.4. $\omega$-Regular Specifications

Games with $\omega$-regular objectives have been studied mostly in the context of two-player games [8], where the goal of one of the players is to show that the $\omega$-regular objective holds in the system, while the goal of the other player is to show otherwise. Such games are usually used in the context of synthesis and model-checking of temporal logic specifications. These two-player zero-sum games are rather different from ours since in our games, the $\omega$-regular specification is not part of the goal of the players, but rather a property that an external system designer wishes to see satisfied. This changes completely the overall problem setup and explains why the drastic differences in complexity between traditional rgames with $\omega$-regular objectives—whose complexity can range from P (for instance, for Büchi games) to PSPACE (for instance, for Muller games)—and multi-player mean-payoff games with $\omega$-regular specifications, even for two-player zero-sum instances with constant weights.

6.5. On Rational Verification

The problem studied in this paper is called Rational Verification, which has been studied for different types of games and specification languages [10]. While rational verification with LTL goals is 2EXPTIME-complete, the problem can become considerably easier when considering simpler specification languages [15]. However, in the context of multi-player mean-payoff games, only a solution for generalised Büchi specification was known, using an encoding via $\mathrm{GR}\left(1\right)$ specifications, and only for Nash equilibrium. In this paper, we have extended such results to account for all $\omega$-regular specifications, and have provided results for cooperative games and succinct representations. With respect to the former, the only relevant related work is [17], where the core for concurrent game structures was introduced. Furthermore, regarding the latter, a comprehensive study using reactive modules games can be found in [18].

6.6. Future Work

There are multiple interesting avenues for future research. The two most obvious open problems are that of determining the complexity bounds for the E-CORE problem in the general case, as well as closing the complexity gap for the WRMG-E-NASH problem. Following this, there are other directions that appear to be fruitful, as well as interesting and worthwhile—for example, introducing both imperfect information and nondeterminism offers a closer approximation to real-world systems, as well as raising interesting mathematical questions. We are also interested in using $\omega$-regular specifications to understand $\omega$-regular games in a unified, principled way.