A Note on the Concept of Time in Extensive Games

Abstract

Using the concept of informational digraphs, we propose a “no redundant information sets” property that can characterize the exact class of extensive games which can be time structured. Our result can be applied to define time-dependent solution concepts like the Open-Loop and the Closed-Loop Nash Equilibrium in extensive games with imperfect information.

1. Introduction

A game in extensive form is said to be time structured if there exists a strictly increasing order preserving real-valued function on its information sets, where the order is described by some precedence relation. While a lack of time structure (e.g., the Absent-Minded Driver game in Piccione and Rubinstein (1997) [1] and Aumann, Hart and Perry (1997) [2]) is often associated with a violation of perfect recall, von Stengel and Forges (2008) [3] give an example of a two-player game with perfect recall that does not have a time structure. On the other hand, one can construct games that violate perfect recall but that can be time structured. In this note, we pin down the exact class of extensive games that can be time structured by proposing a “no-redundant information” sets property on the induced informational digraph of the game. As a result, time-dependent solution concepts like the Open-Loop and the Closed-Loop Nash Equilibrium in extensive games (e.g., Friedman (1990) [4]) can be defined for this class of games even under imperfect information.

2. Extensive Games

Let $\mathsf{\Omega }=((\mathcal{A},\mathcal{H}),(N,P),{\left(I_i\right)}_{i\in N},{\left({\ge }_i\right)}_{i\in N})$ be an Osborne–Rubinstein game in extensive form (OR) [5] that can be summarized as follows. $(\mathcal{A},\mathcal{H})$ is an action-sequence pair consisting of a set of actions$\mathcal{A}$ and a set of action histories$\mathcal{H}$, which are finite or infinite sequences of elements from $\mathcal{A}$. ${\left(a^k\right)}_{k=1}^L$ denotes a sequence in $\mathcal{H}$ where $L\in \{0,1,2,...\}\cup \{\infty \}$. Moreover, $(\mathcal{A},\mathcal{H})$ satisfies empty history (OR1), initial segment closed (OR2) and completeness (OR3) (see [6]). For each history $h\in \mathcal{H}$, let ${\mathcal{A}}_h$ denote the set of available actions at h. The set of histories $\mathcal{H}$ can be partitioned into the set of decision histories ${\mathcal{H}}^D$ (decision nodes of the games) and terminal histories${\mathcal{H}}^T=\mathcal{H}\backslash {\mathcal{H}}^D$ (terminal nodes of the game). $N=\{1,...,n\}$ is a finite set of players called the player set. The function $P:{\mathcal{H}}^D\to N$ assigns each decision history to a player. We use $P_i=\{h\in {\mathcal{H}}^D:P\left(h\right)=i\}$ to denote the set of decision histories where player i makes a decision. For each player $i\in N$, let ${\mathcal{I}}_i$ denote a partition of $P_i$ satisfying the property that ${\mathcal{A}}_h={\mathcal{A}}_{h^{\prime }}$ whenever h and $h^{\prime }$ are in the same member of the partition ${\mathcal{I}}_i$. ${\mathcal{I}}_i$ is called player i’s information partition and its elements are called the information sets of player i. Let $\mathcal{I}={\displaystyle \bigcup _{i\in N}}{\mathcal{I}}_i$ denote the set of all information sets of the game. For each player $i\in N$, ${\ge }_i$ is a complete and transitive binary relation over ${\mathcal{H}}^T$.

3. Informational Digraphs

Let $\mathsf{\Omega }$ be an OR game. If $h={\left(a^k\right)}_{k=1}^L\in \mathcal{H}$ and $h^{\prime }={\left(a^k\right)}_{k=1}^{L^{\prime }}\in \mathcal{H}$ for positive integer $L^{\prime }<L$, then we say that $h^{\prime }$ is a proper initial segment of h. We define binary relation ≺ on the set $\mathcal{I}$ as follows. For $I^{\prime }$, $I^{″}\in \mathcal{I}$, we say that $I^{\prime }\prec I^{″}$ if there exist $h^{\prime }\in I^{\prime }$ and $h^{″}\in I^{″}$ such that $h^{\prime }$ is a proper initial segment of $h^{″}$. While von Stengel and Forges (2008) [3] used this binary relation in the context of a two-player game satisfying perfect recall, our ordering can be applied to any general OR game. From the pair $\left(\mathcal{I},\prec \right)$, we define an informational digraph of $\mathsf{\Omega }$ by the directed graph $G=(V,E)$, where $V=\mathcal{I}$ and $E=$≺. A sequence of vertices $\alpha ={⟨I_i⟩}_{i=1}^k$, where $k\ge 2$ and $\left(I_i,I_{i+1}\right)\in E$ for each $i<n$ is called a walk from $I^{\prime }$ to $I^{″}$ if $I^{\prime }=I_1$ and $I^{″}=I_k$. When there is a walk from $I^{\prime }$ to $I^{″}$, we say that vertex $I^{″}$ is reachable from $I^{\prime }$ and we denote this reachability relation of digraph G by $R_G$ so that $I^{\prime }{R}_G{I}^{″}$ if $I^{″}$ is reachable from $I^{\prime }$. We call walk $\alpha$ from $I^{\prime }$ to $I^{″}$ the flow of information from information set $I^{\prime }$ to information set $I^{″}$ in game $\mathsf{\Omega }$. We say that G is asymmetric if for all $I^{\prime }$, $I^{″}\in \mathcal{I}$, $\left(I^{\prime },I^{″}\right)\in E$ implies $\left(I^{″},I^{\prime }\right)\notin E$. Note that asymmetry implies irreflexivity. We say that digraphs $G=(V,E)$ and $H=(W,F)$ have the same reachability relation if $V=W$ and $R_G=R_H$. We say that H is a transitive reduction of G if H is the digraph with the least number of edges that has the same reachability relation as G, that is, for all $H^{\prime }$ such that $R_G=R_{H^{\prime }}=R_H$ we have $\left|E\left(H\right)\right|\le \left|E\left(H^{\prime }\right)\right|$. ${⟨I_i⟩}_{i=1}^k$ is a maximal walk from $I^{\prime }$ to $I^{″}$ if it is not a proper subsequence of any walk from $I^{\prime }$ to $I^{″}$. Walk $\alpha$ is a proper subsequence of walk $\gamma$ if $\alpha$ is a subsequence of $\gamma$ while $\gamma$ is not a subsequence of $\alpha$.

We say that $\mathsf{\Omega }$ has an informational cycle if its informational digraph has some vertex $I^{\prime }$ and walk $\gamma ={⟨I_i⟩}_{i=1}^k$ from $I^{\prime }$ to itself satisfying the following: For all $m,n\in \left\{1,..,k\right\}$, $m\ne n$ imply $I^m\ne I^n$, except at $m=1$ and $n=k$ where $I_1=I_k=I^{\prime }$. Although the graph may have longer cycles with repeated vertices, the condition used in this definition is necessary for any kind of cycle to exist in the graph. We say that G is acyclic if it does not contain any informational cycle. We say that informational digraph $G=(V,E)$ does not contain redundant information if its transitive reduction is unique.

Lemma 1.

Let $G=(V,E)$ be an acyclic informational digraph. If G has a walk from $I^{\prime }$ to $I^{″}$ for some $I^{\prime },I^{″}\in V$, then there exists some maximal walk from $I^{\prime }$ to $I^{″}$.

Proof. 

Fix any $I^{\prime }$ and $I^{″}$ in V and let $\mathsf{\Theta }$ denote the set of all walks from $I^{\prime }$ to $I^{″}$. We define binary relation ⊲ on the set $\mathsf{\Theta }$ as follows. For $\alpha$, $\gamma \in \mathsf{\Theta }$, we say that $\alpha ⊲\gamma$ if $\alpha$ is a subsequence of $\gamma$. Clearly, ⊲ is a reflexive and transitive. Moreover, it is anti-symmetric since G is acyclical Note that if G were cyclical, there could exist infinite-length walks $\alpha$ and $\gamma$ such that $\alpha$ is a subsequence of $\gamma$ and $\gamma$ is a subsequence of $\alpha$, where $\alpha \ne \gamma$ and $\gamma$ is a cyclic permutation of $\alpha$.

Then, it follows that ⊲ is a partial order. Now, since G is acyclical and by the completeness of OR games (axiom OR3), every chain of $\left(\mathsf{\Theta },⊲\right)$ has an upper bound in $\mathsf{\Theta }$. Therefore, by Zorn’s Lemma, $\mathsf{\Theta }$ has a maximal element. Note that the maximal walk need not be unique. □

The following claims are direct consequences of Lemma 1 and the definition of transitive reduction.

Claim 1.

Let $G=(V,E)$ be an acyclic informational digraph and let $H=(V,F)$ be a transitive reduction of G. If α is a walk in $H$ from $I^{\prime }$ to $I^{″}$ for some $I^{\prime },I^{″}\in V$, then it is also a maximal walk from $I^{\prime }$ to $I^{″}$.

Claim 2.

Let $G=(V,E)$ be an acyclic informational digraph. Suppose further that $H=(V,F)$ and $H^{\prime }=(V,F^{\prime })$ are two transitive reductions of G. If α is a maximal walk from I to $I^{″}$ in H for some $I,I^{″}\in V$, then it is also a maximal walk from I to $I^{″}$ in $H^{\prime }$.

We say that OR game $\mathsf{\Omega }$ can be time structured if and only if there exists a real valued function f on the vertices of its informational digraph satisfying: $\left(u,v\right)\in E$ implies $f\left(u\right)<f\left(v\right)$. The following result is a direct consequence of the fact that acyclic digraphs admit a topological sorting (see Kahn (1962) [7]).

Theorem 1.

G is acyclic if there exists a real valued function f acting on its vertices satisfying: $\left(u,v\right)\in E$ implies $f\left(u\right)<f\left(v\right)$.

Theorem 2.

Let Ω be an OR game with informational digraph $G=\left(\mathcal{I},\prec \right)$. Then, Ω can be time structured if G is (i) asymmetric and (ii) does not contain redundant information.

Proof. 

By Theorem 1, it suffices to show that G is acyclic if (i) and (ii) hold.

(Only if). Suppose that G is acyclic, then (i) is trivially satisfied. To show (ii), we consider any two transitive reductions of G, say H and $H^{\prime }$. We claim that $E\left(H\right)=E\left(H^{\prime }\right)$. Indeed, suppose that there exist vertices x and y such that $\left(x,y\right)\in E\left(H\right)$. Then, by Claim 1 $⟨x,y⟩$ is a maximal walk from x to y in H, and by Claim 2 it is also a maximal walk from x to y in $H^{\prime }$. Hence, $\left(x,y\right)\in E\left(H^{\prime }\right)$.

(If) Suppose that (i) and (ii) are satisfied. (i) ensures that G cannot have cycles of length $<3$. Suppose by contradiction that G had some cycle $\gamma ={⟨I_i⟩}_{i=1}^k$, where $k\ge 4$. Then, there exists some cyclic permutation $\beta ={⟨L_j⟩}_{j=1}^k$ of $\gamma$ such that $I_1$ is mapped to $L_1$ while $I_2$ is not mapped to $L_2$ through the permutation. One can then construct graph $G^{\prime }$ such that (a) $R_G=R_{G^{\prime }}$ and (b) $G^{\prime }$ contains walk $\beta$, but not walk $\gamma$. As a result, there exists some transitive reduction of G denoted by A, which contains $\gamma$ and some other transitive reduction of G, denoted by B, which contains $\beta$. Since $A\ne B$, the digraph contains redundant information, a contradiction. □

Example 1.

We consider Figure 6 on page 1012 of [3]. Using the convention that $m.n$ denotes the nth information set of player m, we can construct informational digraph $\left(V,E\right)$, where $V=\left\{1.1,1.2,1.3,2.1,2.2,2.3\right\}$ and

$$E=\left\{\begin{array}{c}\left(2.1,1.1\right),\left(2.1,2.2\right),\left(2.1,1.2\right),\left(2.1,2.3\right),\left(2.1,1.3\right),\left(1.1,2.2\right),\\ \left(1.1,1.2\right),\left(1.1,1.3\right),\left(1.1,2.3\right),\left(2.2,1.2\right),\left(1.2,2.3\right),\left(2.3,1.3\right),\left(1.3,2.2\right)\end{array}\right\}$$

The above is a negative example of Theorem 2 as $⟨2.2,1.2,2.3,1.3,2.2⟩$ is an informational cycle. We can obtain a positive example of the theorem by removing edge $\left(1.3,2.2\right)$ from E. The digraph will then become acyclic and its transitive reduction is given by $\left(V,F\right)$, where $F=\left\{\left(2.1,1.1\right),\left(1.1,2.2\right),\left(2.2,1.2\right),\left(1.2,2.3\right),\left(2.3,1.3\right)\right\}$.

Final Remarks

Theorem 2 not only completely characterizes the class of games that have a time structure, but also provides a new perspective on issues related to perfect recall and absentmindedness. Moreover, it can be useful in defining strategies that depend explicitly on time (e.g., the Open-Loop Nash Equilibrium in repeated games with imperfect information and clock dependent strategies in stochastic games as in Hansen, Ibsen-Jensen and Neyman (2021) [8]).