Consistent Beliefs in Extensive Form Games

We introduce consistency of beliefs in the space of hierarchies of conditional beliefs (Battigalli and Siniscalchi) and use it to provide epistemic conditions for equilibria in finite multi-stage games with observed actions.


Introduction
Battigalli and Sinischalchi [1] constructed the space of hierarchies of conditional beliefs and used it to provide epistemic foundations for solution concepts in dynamic games.We consider the question of consistency of beliefs in the space of hierarchies of conditional beliefs.In the space of hierarchies of beliefs, Aumann [2], Aumann and Brandenburger [3] and Barelli [4], among others, have used consistency of beliefs to provide epistemic foundations for solution concepts in games in normal form.Here we provide an analogous analysis for multi-stage games with observable actions, in the corresponding space of hierarchies of conditional beliefs.In particular, we show that consistency of beliefs and extensive form rationality provide epistemic foundations for correlated subgame perfect equilibrium (correlated SPE), and these two conditions, plus a notion of constancy of conjectures, provide epistemic foundations for subgame perfect equilibrium (SPE). 1he following simple example helps understand the ideas involved.Consider the standard Battle of Sexes game, with the payoff matrix below, The story is that the players decide simultaneously where to meet (either at a football game, F , or an opera house, O), and each player would rather go to the same place as the other, but has a preference for one venue over the other.Let A i = {F, O} for i = 1, 2 and A = A 1 × A 2 .A correlated equilibrium for such a simultaneous move game is a Nash equilibrium of the game augmented by some payoff irrelevant state space, which is understood by both players.Consider, for instance, that each player chooses F if the weather is good, and chooses O otherwise (that is, they go to the outdoor event if the weather is good, and to the indoor event if the weather is not good).It is clear that such a strategy is a Nash equilibrium of the game augmented by the state space {good weather, weather not good}: if the other player uses the strategy, it is in the given player's interest use it as well (if the weather is good (not good), a given player knows that the other will go to the football game (opera house), and will do well to go there too).Let p be the probability of the weather being good.Then the pair of strategies above gives rise to the distribution of joint actions η ∈ ∆(A) given by η(F, F ) = p and η(O, O) = 1 − p, and it is without loss to focus directly on such distributions in describing a correlated equilibrium.It suffices that, for each a i ∈ A i , the expected payoff of a i given η(a i , •) ∈ ∆(A j ), j = i, is not smaller than the expected payoff of any other action a i , for i = 1, 2.
Now consider that the players play the game twice.That is, the players play the game once, observe its outcome, play it again, and get the sum of the payoffs obtained in each round.Let H = {∅}∪A denote the set of histories.The empty history represents the first round, and each of the four joint strategies in A represents a possible second round.Recall that a SPE is a Nash equilibrium of the entire game that induces Nash equilibria at each subgame.Analogously, a correlated SPE is a correlated equilibrium of the entire game that induces correlated equilibria at each subgame.It can be described as follows.Let η ∈ ∆(A) be a correlated equilibrium of the original Battle of the Sexes game, like the η described above.A correlated SPE is a list of probability distributions (ν h ) h∈H with ν h ∈ ∆(A) for each h ∈ H, where ν h a correlated equilibrium for the continuation game at history h ∈ A and ν ∅ a correlated equilibrium of the one shot game given by the first round outcome and the contingent second round outcome, given ν h with h ∈ A. That is, each of the four continuation games is simply the original Battle of the Sexes game played after the first round.So a correlated equilibrium for a continuation game is a probability distribution η ∈ ∆(A).In the first round, on the other hand, each joint action gives rise to a (potentially) different continuation strategy.So it is not a simple stage game as the games in the second round.But it can be viewed as an one-shot game, with payoffs given by the sum of what is obtained in the first round and of the conditional payoffs in the second round, given the correlated equilibria of the four potential continuation games.Then, for instance, ν h = η for all h ∈ H is a correlated SPE, because ν a (= η) is a correlated equilibrium of the continuation game after history h = a for each a ∈ A, and given the four continuation correlated moves (ν a ) a∈A , ν ∅ (= η) is a correlated equilibrium of the game with payoffs u i (a) + u i (η), where u i is player i's stage game payoff and u i (η) is the expected payoff given η (so, in particular, the expected payoff given ν ∅ is simply u i (η)+u i (η)).More complex correlated SPE involving different correlated continuation strategies can be constructed analogously.
Likewise, let η ∈ ∆(A 1 ) × ∆(A 2 ) be a joint distribution associated with a Nash equilibrium of the stage game.For instance, η( , which is the joint distribution associated with the Nash equilibrium of the original Battle of the Sexes game in non degenerate mixed strategies.Then a list (ν h ) h∈H with ν h = η for all h ∈ H is a SPE of the game, for the same reason as above.More complex SPE with different Nash equilibria of the continuation games can be constructed analogously.Now let's perform an epistemic analysis on the game, that is, an analysis of knowledge and beliefs of the players.In order to do so, we append a type structure (T 1 , T 2 , g 1 , g 2 ) with g i,h ∈ ∆(S × T j ) for each h ∈ H, where S = S 1 × 2 with S i = {F, O} H for i = 1, 2. The beliefs of a type t i , (g i,h (t i )) h∈H form a conditional probability system (CPS), (the formal definitions are provided below).A "state" for a player is a strategy-type pair (s i , t i ), describing the player's strategy choice and beliefs.Epistemic statements can now be stated in terms of the states of the players.For instance, let S i (h) be player i's set of strategies consistent with history h ∈ H. Let η = η(•|S j (h)) h∈H , where η(•|S j (h)) ∈ ∆(S j (h)) for each h ∈ H.We say that s i is a best response to η, written s i ∈ r i (η), if s i maximizes the expected utility with respect to η(•|S j (h)) for every history h consistent with s i .And we say that the strategy-type pair Statements like "rationality is common knowledge among the players" can be described by a type structure where in each state (s, t) ∈ S 1 × S 2 × T 1 × T 2 both players are rational.Note that a type of a player determines the conditional beliefs at every history, and rationality captures sequentially rational choices, after every history (given the conditional beliefs).
Assume that the beliefs of the players are consistent in the following sense.There is a CPS (µ h ) h∈H with µ h ∈ ∆(S(h) × T ) for each h ∈ H, such that g i,h (t i )(E × T j ) = µ h (E × T |t i ) for all E ⊂ S, t i ∈ T i and i = 1, 2. The idea is analogous to action-consistency in Barelli [4], which is a generalization of the standard common prior assumption.Because strategies are in principle verifiable entities, we can conceive of an outside observer offering bets on S, conditional on each history, where the payouts of the bets are measured in utils.The two players will be in a no-bets situation if there does not exist a bet that yields a sure gain to an outsider.In Barelli [4] it is shown that this is equivalent to consistency of beliefs, as defined above.Now, if consistency and rationality obtain at every (s, t) ∈ S × T ,2 then we can identify a correlated SPE (ν h ) h∈H from the CPS (µ h ) h∈H by putting ν h (a) = µ h ({s : s h = a} × T ), for all a ∈ A. Indeed, if it is the case that beliefs are consistent and the CPS (µ h ) h∈H satisfies µ h ({s : s h = (F, F )} × T ) = p and µ h ({s : s h = (O, O)} × T ) = 1 − p for every h ∈ H, then it is straightforward to verify that rationality is obtained at every state, and that we obtain the correlated SPE described above.Indeed, rationality implies that no player wants to deviate from the recommended action, as required in a correlated SPE, and (ν h ) h∈H is exactly the correlated SPE above.Other correlated SPE are analogously obtained as we vary the consistent CPS (µ h ) h∈H .The key observation here is that, under consistency, rationality ensures that the system of inequalities defining a correlated SPE is met.
Let S i (h) be player i's set of strategies consistent with history h ∈ H, and let H(s i ) be the set of histories consistent with s i .The relevant hypotheses for the players are thus the collection B = {S(h) : h ∈ H}.For a given player i, the hypotheses that are consistent with i's strategies are B i = {S i (h) : h ∈ H}.As in Battigalli and Siniscalchi [1], we simplify notation by writing ∆ B i (•) and ∆ B (•) as ∆ H (•).
In order to perform an epistemic analysis, we append a type structure to the game, describing the beliefs of the players.A type space is a tuple T = (T i , g i ) i∈I with g i : T i → ∆ H (S × T −i ) for each i ∈ I. Again, to simplify notation we write We say that s i is a best response to η, written s i ∈ r i (η), if for all h ∈ H(s i ) and s i ∈ S i (h), we have We then say that a strategy-type pair for all i ∈ I, all h ∈ H and all E ⊂ S. It then follows that g i,h (t i )(E × T −i ) = µ h (E × T |t i ) for all i ∈ I, all h ∈ H, all E ⊂ S and marg T i µ h -a.e.t i .Let supp µ = h∈H supp µ h denote the support of the consistent prior.As advanced above, consistency is founded on players being in a no-bets situation, that is, a situation where an outside observer cannot make a sure gain on the group of players by offering bets on the strategy choices of the players.Proposition 5.3 in Barelli [4] establishes the equivalence between consistency and no-bets, and the reader is referred to that paper for further details.
For the sake of comparison with the literature, consider a finite normal form game G = (A i , u i ) i∈I and a type space [2] showed that, when there is a common prior, common knowledge of rationality implies that players play a correlated equilibrium.Aumann and Brandenbuger [3] showed that common knowledge of rationality and of conjectures and the existence of a common prior are sufficient conditions for players to play a Nash equilibrium.These results were extended in Barelli [4] with the use of action-consistency in the place of common prior, rationality in the support of the action-consistent prior in the place of common knowledge of rationality and constancy of conjectures in the support of the action-consistent prior in the place of common knowledge of conjectures.
Note that the notion of consistency used here is much more demanding than using action-consistency in the normal form of the game.Consistency requires that players be at a no-bets situation after every history h ∈ H, whereas action-consistency allows for players to not be at a no-bets situation after histories that are not compatible with the strategy profiles in the support of the action-consistent prior.Players are required to be aware of a potential outsider at every counter factual that they envisage while choosing their strategies.

Results
For a given consistent prior µ, let ν = (ν h ) h∈H be given by ν h (a) = marg S µ h (s : s h = a) for each a ∈ A(h), so that ν h ∈ ∆(A(h)).We have: Proposition 1.Let G be a finite multi-stage game with observed actions, and let T be a type space associated with G. Assume that there exists a consistent prior µ ∈ ∆ H (S × T ) such that player i is rational at all (s, t) ∈ supp µ, for every i ∈ I. Then ν defined above is a correlated SPE.
Proof.By consistency, we have marg S −i g i,h (t i ) = marg S −i µ h (•|t i ) for every i ∈ I, h ∈ H and t i ∈ supp marg T i µ h .By rationality we then have for each i ∈ I and every h ∈ H(s i ) for every i ∈ I, s i , s i in S i (h) and h ∈ H(s i ).Now notice that the restriction of ν to a history h ∈ H, ν|h, is the behavioral representation of η h .By Kuhn's Theorem, the distribution over final outcomes induced by η h is the same as that induced by ν|h, so ν|h is a correlated equilibrium of the continuation game G(h) for every history h ∈ H, and we are done.
Let φ i,h (t i ) = marg S −i g i,h (t i ) denote the conjecture of type t i at h ∈ H.We have: Proposition 2. Let G be a finite multi-stage game with observed actions, and let T be a type space associated with G. Assume that there exists a consistent prior µ ∈ ∆ H (S × T ) such that (i) player i is rational at all (s, t) ∈ supp µ for every i ∈ I and (ii) φ i,h (t i ) = φ i,h (t i ) for every i ∈ I for every t i , t i ∈ supp marg T i µ h , for each h ∈ H. Then ν defined above is a SPE.
Proof.Fix h ∈ H and let φ i,h be player i's constant conjecture in the support of marg T i µ h .By consistency and rationality, we have for each s ∈ S(h) marg S µ h (s) = marg T i µ h (T i (s i ))φ i,h (s −i ) where T i (s i ) is as in Proposition 1. Hence marg S µ h = marg S i µ h ⊗ marg S −i µ h Now induction in the number of players shows that marg S µ h = ⊗ i∈I marg S i µ h , and a fortiori ν h ∈ × i∈I ∆(A i (h)), for all h ∈ H.The result then follows from Proposition 1.

Conclusion
The results in section 3 tell us the following: under consistency, rationality yields correlated SPE and adding constancy of conjectures to these two conditions yield SPE.These results are analogous to the results in Aumann [2], Aumann and Brandenburger [3] and Barelli [4].As in the latter, beliefs are required to be consistent only at events that are potentially observable by an outsider, who could in principle force beliefs to be consistent by offering bets on the observable events.Rationality and constancy of conjectures have to hold in the support of the consistent prior.Because rationality and/or constancy of conjectures are implied by (but do not imply) rationality and/or conjectures being commonly known among the players, we have that rationality need not be common knowledge for players to play a correlated SPE, and neither do conjectures have to be common knowledge for players to play a SPE.