Evolutionary Stable Strategies in Multistage Games

Abstract

Direct ESS has some disadvantages, which are seen even in the case of repeated games when the sequence of stage ESSs may not constitute the direct ESS in the repeated game. We present here the refinement of the ESS definition, which eliminates these disadvantages and represents the base for the definition of ESS in games in extensive form. The effectiveness of this approach for multistage n-person games is shown for metagame (this notion is used for the first time), in which under some relevant conditions, the existence of ESS is proved, and ESSs are constructed using threat strategies.

1. Introduction

Evolutionary games were first formulated in [1]. We shall follow [2] in the definition of evolutionary stable strategies (ESSs) [3,4,5,6] for symmetric bimatrix games; here, the definition of ESS (so-called “direct ESS”) applicable for extensive-form game with perfect recall (see [7]) is also purposed. This definition is based upon the concept of symmetry in extensive-form games introduced in [8]. As we saw earlier (see [9]), the classical definition of ESS proposed for normal two-person games cannot be applied to repeated and multistage games. In this paper, we propose a refinement of this definition, which can be considered an attempt to solve the problem. First, we present an example of a two-stage Hawk and Dove game, for which we try to explain the problem and show the effectiveness of the new refined definition. After, we propose the new ESS definition for general n-person games and specially for repeated and multistage games (metagames). In the last section, we present an algorithm for constructing ESS in general n-person multistage games (metagames) and prove the corresponding theorem. This result is illustrated by an example.

2. Definition of ESS for Two-Person Games

Following [2], the symmetric extensive-form 2-person game is a pair $(\Gamma ,T)$ where $\Gamma$ is an extensive-form game and T is a symmetry of $\Gamma$. If $b_1,b_2$ are the behavior strategies of player 1 in $(\Gamma ,T)$ and $b_1^T,b_2^T$ (behavior strategies of player 2) are the symmetric images of $b_1,b_2$, respectively, then the probability that the endpoint z is reached when $(b_1,b_2^T)$ is played is equal to the probability that $z^T$ is reached when $(b_2,b_1^T)$ is played. Therefore, the expected payoff of player 1 when $(b_1,b_2^T)$ is played is equal to player 2’s expected payoff when $(b_2,b_1^T)$ is played [10]:

$$E_1(b_1,b_2^T)=E_2(b_2,b_1^T),$$

(1)

Equation (1), restricted to pure strategies, defines the symmetric normal form of $(\Gamma ,T)$.

Definition 1.

Direct ESS in $(\Gamma ,T)$ is a behavior strategy $\overline{b}$ of player 1 that satisfies

$$E_1(\overline{b},{\overline{b}}^T)=\underset{b}{max}{E}_1(b,{\overline{b}}^T)$$

(2)

$$\mathit{and}\mathit{if}b\ne \overline{b}\mathit{and}{E}_1(b,{\overline{b}}^T)=E_1(\overline{b},{\overline{b}}^T),$$

$$\mathit{then}{E}_1(b,b^T)<E_1(\overline{b},b^T).$$

(3)

We try to purpose some refinement of this definition. Let $\mu (b^{\prime },b^{″})$ be the probability generated over the set of endpoints in the game if players choose behavior strategies $b^{\prime }$, $b^{″}$, respectively.

Definition 2.

The behavior strategy $\overline{b}$ is called ESS in $(\Gamma ,T)$ if $\overline{b}$ satisfies

$$E_1(\overline{b},{\overline{b}}^T)=\underset{b}{max}{E}_1(b,{\overline{b}}^T)$$

(4)

$$\mathit{and}\mathit{if}\mathit{for}{b}^{\prime }\mathit{such}\mathit{that}\mu (b^{\prime },{\overline{b}}^T)\ne \mu (\overline{b},{\overline{b}}^T)$$

$$\mathit{the}\mathit{payoff}{E}_1(b^{\prime },{\overline{b}}^T)=E_1(\overline{b},{\overline{b}}^T),$$

$$\mathit{then}{E}_1(b^{\prime },{b^{\prime }}^T)<E_1(\overline{b},{b^{\prime }}^T).$$

(5)

Note that in Definition 2, the important condition $\mu (b^{\prime },{\overline{b}}^T)\ne \mu (\overline{b},{\overline{b}}^T)$ is weak, and if we revert to biological interpretations of ESS, we have to take into account that the biological populations may not react to the changes of strategies in extensive-form games (remember that the strategy in an extensive game has a very complicated structure), and it is clear that “animals” cannot realize the deviation from it and may react to changes in probability measure on the final positions of the game (on the set of outcomes). Thus, deviations which do not affect measure $\mu$ on the endpoints cannot be taken into account when considering ESS.

Example 1.

We repeated the Hawk and Dove game [11]. This game is a two-person bimatrix game Γ with payoff matrices:

$$A=\begin{array}{cc}& \begin{array}{cc}H\hspace{1em}\hspace{1em}\hspace{1em}& D\end{array}\\ \begin{array}{c}H\\ D\end{array}& \left[\begin{array}{cc}\frac{1}{2}(V-C)& V\\ 0& \frac{1}{2}V\end{array}\right]\end{array}\hspace{1em}\hspace{1em}{A}^T=\begin{array}{cc}& \begin{array}{cc}H\hspace{1em}\hspace{1em}\hspace{1em}& D\end{array}\\ \begin{array}{c}H\\ D\end{array}& \left[\begin{array}{cc}\frac{1}{2}(V-C)& 0\\ V& \frac{1}{2}V\end{array}\right]\end{array}$$

If $V>C$, $(H,H)$ is ESS in Γ. Consider now a two-stage version of this game, which can be represented on Figure 1.

The strategy of player I (II) in this game is a rule, which defines the choice of one from two alternatives H or D in each information set of a player. Player I (II) has 5 information sets, and thus, each of them has 32 strategies, which can be represented as sequence $(H,H,D,H,D)$. Denote this strategy of player as $u(·)$.

Consider the strategy $u(·)=(H,H,H,H,H)$, which is composed from ESS (case V > C) in each stage game. It would be appropriate if this strategy is ESS in our two-stage game [12,13]. Unfortunately, it does not satisfy Definition 1, which was the reason to change in our paper this definition to Definition 2.

It can be easily seen that condition (2) holds since $\left(u\right(·),u(·\left)\right)$ is NE in the game Γ. However, there exist a strategy

$$v(·)=(H,H,D,D,D)$$

for which the payoff

$$E_1(v(·),u(·))=E_1(u(·),u(·)).$$

Since

$$E_1(v(·),u(·))=E_1(u(·),u(·))=V-C$$

and

$$E\left(v\right(·),v(·\left)\right)=E\left(u\right(·),v(·\left)\right)=V-C$$

this shows that the condition (3) is not satisfied.

However, according to Definition 2, the strategy $u(·)=(H,H,H,H,H)$ is ESS since the strategy $v(·)=(H,H,D,D,D)$ giving the same payoff against $u(·)$ as $u(·)$ itself is excluded from consideration because of condition (5) of Definition 2.

Remark: In our example, ESS is in pure strategies, and thus in definitions ((2)–(5)), the mathematical expectation of the payoff coincides with the payoff itself [14].

Suppose now that $\Gamma$ is the n-stage repeated bimatrix game. Let G be a stage symmetric bimatrix game. The strategies in G are alternatives in $\Gamma$. To each strategy i of player 1 in $\Gamma$, we correspond a strategy $T\left(i\right)=i$ of player 2 in G with the same index i. Each alternative $c\in C_i(i=1,2)$ in $\Gamma$ is a strategy (index) in some stage game G in $\Gamma$. The mapping $T\left(c\right)$ corresponds to the alternative c (strategy) of player 1 in stage game G, the alternative $T\left(c\right)=c$ (strategy) of player 2 in the same stage game (strategy with the same index). To each information set $u_1$ of player 1, mapping T corresponds the information set $u_2$ of player 2 in the same stage game (the bimatrix game can be represented as a game in extensive form with two moves and two successive information sets $u_1$ for player 1 and $u_2$ for player 2).

Theorem 1.

If $\overline{\beta }$ is a ESS in G, then the behavior strategy $\overline{b}$ prescribing the behavior $\overline{\beta }$ to the alternatives of each information set ($\overline{\beta }$ is ESS in stage game G) is ESS in $(\Gamma ,T)$.

3. Definition of ESS for n-Person Games

There are many different approaches to how the ESS should be extended to the n-person case. We shall follow the definition given in [15]. Suppose we have a game G in normal form:

$$G=<N;X_1,...,X_n;K_1,...,K_n>,$$

when $N=1,...,n$ is the set of players, $X_i=\left\{x_i\right\}$ is the set of strategies of player i, and $K_i(x_1,...,x_n)$ is the payoff function of player i. We suppose for simplicity that the sets $X_i,i=1,...,n$ are finite.

Note that the strategy profile $\overline{x}=({\overline{x}}_1,...,{\overline{x}}_n)$ is an ESS in G [16], if it is a strict Nash equilibrium, i.e., if

$$K_i(\overline{x}\left|\right|x_i)<K_i\left(\overline{x}\right)\mathrm{for}\mathrm{all}{x}_i\in X_i,i=1,...,n.$$

(6)

It is proved that condition (6) protects the strategy ${\overline{x}}_i$ against the invasion of a few mutants playing another strategy $y_i$.

It is also clear that (6) cannot be used to define ESS in multistage games since there is always a large number of strategies $y_i\in X_i$ such that for any strategy profile, $x=(x_1,...,x_n),K_i\left(x\right||y_i)=K_i\left(x\right)$.

Following the ideas of the previous section, try to refine the ESS concept specified in (6) in such a way that it could be useful also for n-person multistage games.

For this reason, we have to mention that (6) automatically excludes the mixed strategy profiles from consideration. Additionally, the refinement of this concept will act only with pure strategy profiles.

Denote by $U_i$ the strategy set of player i in $\Gamma$. $u_i\in U_i$ is the strategy of player i, and $H_i(u_1,...,u_n)$ is the payoff function of player i. Let $\Gamma$ be a multistage n-person game.

Definition 3.

The strategy profile $\overline{u}=({\overline{u}}_1,..,{\overline{u}}_n)$ in Γ is called ESS if

$$H_i(\overline{u}\left|\right|u_i)\le H_i\left(\overline{u}\right),u_i\in U_i,i=1,...,n$$

(7)

and if $H_i(\overline{u}\left|\right|u_i)=H_i\left(\overline{u}\right)$ for some $i\in N,u_i\in U_i$, then paths corresponding to $\left(\overline{u}\right|\left|u_i\right)$ and $\overline{u}$ necessarily coincide.

From Definition 3, it follows that strict inequality in (7) is valid for all those deviations, for which the resulting paths differ from that generated by the ESS strategy profile.

4. Existence of ESS in Multistage Repeated n-Person Games

Suppose that $\Gamma$ is a finite stage repeated n-person game with simultaneous n-person stage game G. Suppose that G has an ESS (strict Nash equilibrium) [17]. Denote ESS in G as $\overline{x}=({\overline{x}}_1,...,{\overline{x}}_n)$, and the payoff in G as $K_i({\overline{x}}_1,...,{\overline{x}}_n)$. Denote ${\overline{K}}_i=K_i({\overline{x}}_1,...,{\overline{x}}_n)$. Consider zero-sum games $G^i$ between player i as first player and subset $N\backslash \left\{i\right\}$ as second player with strategy sets

$$X_i,X_{N\backslash \left\{i\right\}}=\prod _{k\in N\backslash \left\{i\right\}}{X}_k$$

correspondingly and payoff of player i equal to $K_i(x_1,...,x_n)$. (The payoff of second player $N\backslash \left\{i\right\}$ equals − $K_i(x_1,...,x_n)$.) Denote by ${\mu }^{*}=({\mu }_i^{*},{\mu }_{N\backslash \left\{i\right\}}^{*})$ the corresponding mixed-strategy saddle point in $G^i$, and by ${\upsilon }_i$ the value of game $G^i$. Fix some n-tuple $\tilde{x}=({\tilde{x}}_1,...,{\tilde{x}}_n)$, ${\tilde{K}}_i=K_i\left(\tilde{x}\right)$, and consider

$$\widetilde{\tilde{K_i}}=\underset{x_i\in X_i}{max}{K}_i(\tilde{x}\left|\right|x_i).$$

Suppose that the following conditions hold

$${\tilde{K}}_i+{\overline{K}}_i>\widetilde{\tilde{K_i}}+{\upsilon }_i,{\tilde{K}}_i>{\overline{K}}_i,{\overline{K}}_i>{\upsilon }_i.$$

(8)

Theorem 2.

If there exists such an n-tuple of strategies $\tilde{x}=({\tilde{x}}_1,...,{\tilde{x}}_n)$ in G that (8) holds, then Γ has an ESS which is constructed as follows.

If l is a number of stages in Γ, then each player i has to play ${\tilde{x}}_i$ on the first $l-1$ stages and ${\overline{x}}_i$ (ESS in G) on the last stage l: in case on some stage $t<l$, player i first deviates for the first timefrom ${\tilde{x}}_i$, starting from the stage $t+1$ coalition of players, $(N\backslash \{i\left\}\right)$ chooses ${\mu }_{N\backslash \left\{i\right\}}^{*}$ from the mixed-strategy saddle point in $G^i$.

5. ESS for Metagames

Finite multistage game $\Gamma$, at each stage of which some n-person game G is played, is called metagame; the game realized at each stage depends on the players’ choices in previous games.

Over the strategy profiles x in the stage game G, the mapping $T_G\left(x\right)$ is defined, which corresponds to each stage game G and strategy profile x for the next stage game $G^1=T_G\left(x\right)$.

Suppose that in metagame $\Gamma$, on the first stage, the stage game $G^1$ is played. If in $G^1$, players choose strategy profile $x^1=(x_1^1,...,x_n^1)$, then on the second stage, the game $G^2=T_{G^1}\left(x^1\right)$ is played. If on stage k, players playing the stage game $G^k$ choose strategy profile $x^k=(x_1^k,...,x_n^k)$, on the next stage, the game $G^{k+1}=T_{G^k}\left(x^k\right)$ is played. The metagame ends on stage m. The payoff of player $i\in N$ in the metagame is equal to the sum of their payoffs in stage games. Denote by $K_i^l(x_1^l,..,x_n^l)$, the payoff of player $i\in N$ in stage game $G^l$, then the payoff of player $i\in N$ in metagame is equal to

$$H_i=\sum _{l=1}^m{K}_i^l(x_1^l,...,x_n^l),i\in N.$$

It is important that after each stage, players know all of the prehistory (prehistory—players’ choices before current stage of metagame).

The strategy $u_i$ of player $i\in N$ in $\Gamma$ is a mapping which corresponds to the choice of strategy in stage game G as a function of the strategy profiles of all players in stage games realized before the stage game G.

Suppose that stage game G has ESS (strict Nash equilibrium). Note that strategy profile $\overline{x}=({\overline{x}}_1,...,{\overline{x}}_n)$ is ESS in G, and $K_i({\overline{x}}_1,...,{\overline{x}}_n)={\overline{K}}_i$ is the payoff of player i in G under the strategy profile $\tilde{u}$.

Suppose that under strategy profile $\overline{u}=(u_1,...,u_i,...,u_n)$, the sequence of stage games $G^1,...,G^k,...,G^n$ is realized. This sequence of stage games we shall call path corresponding to n is the strategy profile $u=(u_1,...,u_i,...,u_n)$.

Now consider the stage game $G^k,k=1,...,l-1$. Note that the game $G^k$ depends also upon choices made by players in previous stage game $G^{k-1}$. This means that on stage k dependent on previous strategy choices, different games of type $G^k$ can be realized. For each stage game $G^k$, denote by $G_i^k$ the zero-sum game between player i as the first player and subset $N\backslash \left\{i\right\}$ as the second player with sets of strategies

$$X_i^k,X_{N\backslash \left\{i\right\}}^k=\prod _{m\in N\backslash \left\{i\right\}}^{}{X}_m^k$$

respectively, and the payoff of player i is given by $K_i^k(x_1^k,...,x_n^k)$. (Payoff of the second player $N\backslash \left\{i\right\}$ is given by − $K_i^k(x_1^k,..,x_n^k)$.)

Denote by $({\widehat{\eta }}_i^k,{\widehat{\eta }}_{N\backslash \left\{i\right\}}^k)$ the corresponding mixed-strategy profile in the saddle point of $G_i^k$ and by ${\upsilon }_i^k$ the value of $G_i^k$. Fix some strategy profile in $G^k$ as

$${\tilde{x}}^k=({\tilde{x}}_1^k,...,{\tilde{x}}_n^k),{\tilde{K}}_i^k=K_i^k\left({\tilde{x}}^k\right)$$

and suppose that

$${\tilde{K}}_i^k>{\overline{K}}_i^k,i=1,...,n.$$

Consider ${\tilde{\tilde{K}}}_i^k={max}_{x_i^k\in X_i^k}{K}_i^k({\tilde{x}}^k\left|\right|x_i^k)$.

Definition 4.

The strategy profile $u^{*}=(u_1^{*},...,u_n^{*})$ is ESS in the metagame if

$$H_i\left(u^{*}\right)\ge H_i(u^{*}\left|\right|u_i)$$

for all i and all $u_i$, and if $H_i\left(u^{*}\right|u_i)=H_i\left(u^{*}\right)$ for some $i\in N,u_i\in U_i$, then paths corresponding to $\left(u^{*}\right|u_i)$ and $u^{*}$ coincide.

This definition is common for definition of ESS for n-person games.

Generate strategy $u_i^{*}$ of player i in metagame $\Gamma$ as the following: in games $G^k,j=1,...,l-1$, players chooses strategies ${\tilde{x}}_i^k$, and at last stage in $G^k$—${\overline{x}}_i^k$. Then, strategy profile $u^{*}=(u_1^{*},...,u_n^{*})$ realizes a sequence of stage games $G^{1*},G^{2*},...,G^{l*}$ in metagame $\Gamma$, which we will call the optimal trajectory. Denote by ${\tilde{K}}_i^k$ the payoff of player i in ${\tilde{G}}_i^k$.

Suppose that player i deviates from $u_i^{*}$ at some stage $t<l$, then, beginning from stage $t+1$, players from $N\backslash \left\{i\right\}$ choose ${\widehat{\eta }}_{N\backslash \left\{i\right\}}^k,k=t-1,...,l$, see Figure 2. Define ${\overline{\overline{u}}}_i^k$, satisfying $H_i({\tilde{u}}^k\left|\right|{\overline{\overline{u}}}_i^k)\ge H_i\left({\tilde{u}}^k\right)$. After stage t, players $N\backslash \left\{i\right\}$ choose strategy ${\widehat{\eta }}_{N\backslash \left\{i\right\}}^l,l>t$, optimal in the zero-sum game $G_i^l$.

Denote $W_i={max}_{0\le k\le l}\left[{max}_{G^k}{\upsilon }_i^k\right]$. Suppose that

$$\sum _{k=t}^{l-1}{\tilde{K}}_i^k+{\overline{K}}_i^k>{\tilde{\tilde{K}}}_i^k+(l-t)W_i,t=1,...,l-1.$$

(9)

Theorem 3.

If there exist strategies ${\tilde{x}}_i^{k_j}$ in games $G^{k_j}$ such that (9) holds, then the strategy profile $u^{*}$, mentioned above, is ESS in metagame Γ.

Proof. 

The payoff of player i when the strategy profile $u^{*}$ is used is

$$H_i\left(u^{*}\right)=\sum _{k=1}^{l-1}{\tilde{K}}_i^k+{\overline{K}}_i^k=\sum _{k=1}^{l-1}{\tilde{K}}_i^k({\tilde{x}}_1^k,...,{\tilde{x}}_n^k)+{\overline{K}}_i^k({\overline{x}}_1^k,...,{\overline{x}}_n^k).$$

It is important to note that $x_i^k$ are pure strategies.

Suppose that player i deviates from $u_i^{*}$, and this happens at stage t of metagame $\Gamma$. Denote by $u_i$ this new strategy of player i. Then we obtain a new strategy profile $\left(u^{*}\right|\left|u_i\right)$ in $\Gamma$, which realizes the path, different from the optimal trajectory. Consider the payoff of player i under strategy profile $\left(u^{*}\right|\left|u_i\right)$, realizing the path different from the optimal trajectory. From (9), we obtain

$$\begin{array}{c}\hfill \begin{array}{c}\hfill H_i(u^{*}\left|\right|u_i)=\sum _{k=1}^{t-1}{\tilde{K}}_i^k+{\tilde{\tilde{K}}}_i^k+\sum _{k=t+1}^l{\upsilon }_i^k\le \sum _{k=1}^{t-1}{\tilde{K}}_i^k+{\tilde{\tilde{K}}}_i^k+\sum _{j=t+1}^l{W}_i\\ \hfill =\sum _{k=1}^{t-1}{\tilde{K}}_i^k+{\tilde{\tilde{K}}}_i^k+(l-t)W_i<\sum _{k=1}^{l-1}{\tilde{K}}_i^k+{\overline{K}}_i^k=H_i\left(u^{*}\right)\end{array}\end{array}$$

Thus, $u^{*}$ is ESS (see Definition 4).

The theorem is proved. □

Example 2.

Consider a metagame Γ, in which one of two possible games $G^{\prime }$ and $G^{″}$ is played on each stage. $G^{\prime }$ and $G^{″}$ are two-player games with strategy sets $X_1^{\prime }=(x_{11}^{\prime },x_{12}^{\prime },x_{13}^{\prime })$, $X_2^{\prime }=(x_{21}^{\prime },x_{22}^{\prime },x_{23}^{\prime })$ in $G^{\prime }$ of players I and II, and strategy sets $X_1^{″}=(x_{11}^{″},x_{12}^{″},x_{13}^{″})$, $X_2^{″}=(x_{21}^{″},x_{22}^{″},x_{23}^{″})$ in $G^{″}$ of player I and II, correspondingly. The payoffs in $G^{\prime }$ are defined as

Table 1. The payoffs in $G^{\prime }$.

	${\mathit{x}}_{21}^{\prime }$	${\mathit{x}}_{22}^{\prime }$	${\mathit{x}}_{23}^{\prime }$
	1	2	3
$x_{11}^{\prime }$ 1	(10, 10)	(0, 15)	(0, 0)
$x_{12}^{\prime }$ 2	(15, 0)	(6, 6)	(0, 0)
$x_{13}^{\prime }$ 3	(0, 0)	(0, 0)	(0, 0)

.

In $G^{″}$ as

Table 2. The payoffs in $G^{″}$.

	${\mathit{x}}_{21}^{″}$	${\mathit{x}}_{22}^{″}$	${\mathit{x}}_{23}^{″}$
	1	2	3
$x_{11}^{″}$ 1	(11, 11)	(0, 15)	(2, 2)
$x_{12}^{″}$ 2	(15, 0)	(6, 6)	(2, 2)
$x_{13}^{″}$ 3	(2, 2)	(2, 2)	(2, 2)

.

In both games, the Nash equilibrium is ${\overline{x}}^{\prime }=(x_{12}^{\prime },x_{22}^{\prime })$, ${\overline{x}}^{″}=(x_{12}^{″},x_{22}^{″})$ with payoffs

$${\overline{K}}_1^{\prime }(x_{12}^{\prime },x_{22}^{\prime })={\overline{K}}_2^{\prime }(x_{12}^{\prime },x_{22}^{\prime })={\overline{K}}_1^{\prime }(2,2)={\overline{K}}_2^{\prime }(2,2)=6$$

and

$${\overline{K}}_1^{″}(x_{12}^{″},x_{22}^{″})={\overline{K}}_2^{″}(x_{12}^{″},x_{22}^{″})={\overline{K}}_1^{″}(2,2)={\overline{K}}_2^{″}(2,2)=6.$$

Also we have

$$K_i^{\prime }(1,1)=10>6=K_i^{\prime }(2,2),i=1,2$$

and

$$K_i^{″}(1,1)=11>6=K_i^{″}(2,2),i=1,2.$$

Suppose ${\tilde{K}}_i^{\prime }=K_i^{\prime }(1,1)=10,i=1,2$. In both stage games, if player i deviates from $\tilde{x}=(1,1)=(x_{11}^{\prime },x_{21}^{\prime })$ (or $(x_{11}^{″},x_{21}^{″})$), they can obtain at most

$${\tilde{\tilde{K}}}_1^{\prime }=\underset{l}{max}{K}_1^{\prime }(\tilde{x}\left|\right|x_{1l})=\underset{l}{max}{K}_1^{\prime }(x_{11}^{\prime },x_{21}^{\prime }\left|\right|x_{1l})=K_1^{\prime }(2,1)=15.$$

Similarly, ${\tilde{\tilde{K}}}_2^{″}=15$. The metagame Γ proceeds as follows. On the first stage, players play the game $G^{\prime }$ ($G^1=G^{\prime }$) and if in $G^{\prime }$, they choose strategy profile (1, 1) or (1, 2), on the next stage, the game $G^{\prime }$ is repeated $(G^2=G^{\prime })$. In the other case (if strategy profiles (1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2), and (3, 3) are chosen), on the next stage, the game $G^{″}$ is played $(G^2=G^{″})$. If on stage k the game $G^{\prime }(G^k=G^{\prime })$ is played, the next stage game is defined as in the first stage. If on stage k, the game $G^{″}(G^k=G^{\prime \prime })$ is played, on the next stage, the game $G^{\prime }(G^{k+1}=G^{\prime })$ is played if in stage game $G^k$, the strategy profiles (1, 1) or (1, 2) are chosen. In other cases, on stage $k+1$, the game $G^{″}$ is played $(G^{k+1}=G^{″})$. The metagame ends on stage m.

In each case, when one of the players (player i) deviates from strategy profile $\tilde{x}$, the other player will choose strategy 2 on the next stages of the metagame. Hence, the payoff of the deviating player in all future stage games will be equal to 0. We see that the condition

$${\tilde{K}}_i(2,2)+{\overline{K}}_i(1,1)=6+10>{\tilde{\tilde{K}}}_i+v_i=15+0$$

is satisfied, and the strategy profile $u^{*}$ constructed above is strong NE and, thus, ESS.

6. Conclusions

In this paper, based on the definition of “direct ESS”, we try to provide a broader definition of ESS, from two-person games to multistage repeated n-person games. We propose the concept of the “meta-game”, in which the superiority of a broad ESS definition can be highlighted. We prove the existence of ESS under certain conditions and show an example.

The proposed refinement of ESS in the repeated games has the natural property that the repetition of ESS in the stage game will constitute an ESS in the whole game. This is not true if the classical ESS definition is used for repeated games.

Additionally, the ESS definition and its construction are proposed for general multistage n-person games (we call them metagames). It is determined that the use of ESS in each stage game of the metagame (note that this case is different from repeated games, since in the metagame, the stage games are different and depend upon the history of the game process) does not give the ESS in the whole game. In spite of that, we propose an algorithm of constructing the ESS in metagames using the ESS in stage games and threat strategies.