Abstract

The paper is devoted to inverse Stackelberg games with many players. We consider both static and differential games. The main assumption of the paper is the compactness of the strategy sets. We obtain the characterization of inverse Stackelberg solutions and under additional concavity conditions, establish the existence theorem.

1. Introduction

The paper is concerned with the inverse Stackelberg game, also known as the incentive problem. In ordinary Stackelberg games, one player (called a leader) announces his strategy while the other players (called followers) maximize their payoffs using this information. In the inverse Stackelberg games the leader announces the incentive strategy, i.e., the reaction to the followers’ strategies ([1,2,3,4,5] and reference therein). For dynamic cases, the reaction should be nonanticipative.

The inverse Stackelberg games appear in several models (see, for example, [6,7,8]). In games with many followers, it is often assumed that followers play a Nash game ([6,9,10]). If the strategy sets are normed space, then the incentive strategy can be constructed in the affine form (Ref. [11] for static games and Ref. [12] for differential games).

In this paper, we consider a case where the control spaces of the players are metric compacts. We consider both static and dynamic cases. Moreover, for the dynamic case, we apply punishment strategies. The concept of punishment strategies was first used for the analysis of Stackelberg games in the class of feedback strategies in Ref. [13]. The inverse Stackelberg solutions of two-person differential games were studied via punishment strategies in the paper by Kleimonov [14]. In that paper, the authors described the set of inverse Stackelberg solutions and derived the existence result. In particular, the set of inverse Stackelberg payoffs is equal to the set of feedback Stackelberg payoffs. Note that the incentive strategies considered in the paper by Kleimonov [14] use full memory, i.e., the leader plays with the nonanticipating strategies proposed in the papers by Elliot and Kalton [15] and Varaiya and Lin [16] for zero-sum differential games. The usage of the strategies depends only on the current follower’s control which decreases the payoffs.

In this paper, punishment strategies are applied to static inverse Stackelberg games and to differential inverse Stackelberg games with many followers. We obtain the characterization of the inverse Stackelberg solution and under additional concavity conditions, establish the existence theorem.

The paper is organized as follows. Section 2 is concerned with the static inverse Stackelberg game for a case with n followers. The differential game case is considered in Section 3. In Section 4, we prove the existence theorem for the inverse Stackelberg solution of a differential game.

2. Static Games

We denote the leader by 0. Further, we designate the followers by $1,\dots n$. Player i has a set of strategies ($P_i$) and a payoff function($J_i:P_0\times P_1\dots \times P_n\to \mathbb{R}$). We assume that the sets ($P_i$) are compact and the functions ($J_i$) are continuous.

The incentive strategy of the leader is a mapping:

$$\alpha :{\times }_{i=1}^n{P}_i\to P_0.$$

To define the inverse Stackelberg game, we specify the solution concept used by followers. We suppose that the followers play the Nash game. Let

$$P={\times }_{i=1}^n{P}_i.$$

An element ($u=(u_1,\dots ,u_n)$) of P is a profile of the followers’ strategies. If $u_i^{\prime }\in P_i$ then $(u_i^{\prime },u_{-i})$ is the profile of strategies $(u_1,\dots ,u_{i-1},u_i^{\prime },u_{i+1},\dots ,u_n)$. For simplification, we write $J_i(u_0,u)$ to denote $J_i(u_0,u_1,\dots ,u_n)$. Furthermore, $J_i(u_0,u_i^{\prime },u_{-i})\triangleq J_i(u_0,(u_i^{\prime },u_{-i}))$ is put. If $\alpha$ is an incentive strategy of the leader, u is a profile of strategies of the followers. Then, $J_i[\alpha ,u]\triangleq J_i(\alpha [u],u)$, $J_i[\alpha ,u_i^{\prime },u_{-i}]\triangleq J_i[\alpha ,(u_i^{\prime },u_{-i})]$ are denoted. Further, let $\mathcal{E}(\alpha )$ be a set of the followers’ Nash equilibria for a case where the leader uses the incentive strategy $\alpha$:

$$\mathcal{E}\left(\alpha \right)\triangleq \{u:J_i[\alpha ,u]\ge J_i[\alpha ,u_i^{\prime },u_{-i}]\mathrm{for}\mathrm{any}i=\overline{1,n}\mathrm{and}\mathrm{any}{u}_i^{\prime }\in P_i\}.$$

Definition 1.

The pair $({\alpha }^{\ast },u^{\ast })$ is an inverse Stackelberg solution in the game with one leader and n followers playing the Nash equilibrium if

(1) 

$u^{\ast }\in \mathcal{E}\left(\alpha \right).$

(2) 

$J_0[{\alpha }^{\ast },u^{\ast }]={max}_{\alpha }{max}_{u\in \mathcal{E}(\alpha )}{J}_0[\alpha ,u]$.

The structure of the inverse Stackelberg solution is given in the following statements. Denote

$$\mathcal{B}\triangleq \left\{(u_0^{♮},u^{♮}):\mathrm{for}\mathrm{any}i=\overline{1,n},J_i(u_0^{♮},u^{♮})\ge \underset{u_i}{max}\underset{u_0}{min}{J}_i(u_0,u_i,u_{-i}^{♮})\right\}.$$

Lemma 1.

The following properties hold true:

(1) 

If $u^{♮}\in \mathcal{E}\left(\alpha \right)$, then $(\alpha [u^{♮}],u^{♮})\in \mathcal{B}$;

(2) 

If the strategy of the leader ($u_0^{♮}$), and the profile of the followers’ strategies ($u^{♮}$) are $(u^{♮},u^{♮})\in \mathcal{B}$, then an incentive strategy of the leader α exists such that $u^{♮}\in \mathcal{E}\left(\alpha \right)$.

Proof. 

To use the first statement of the lemma, ${\widehat{u}}_i$ is picked to maximize

$$\underset{u_i\in P_i}{max}\underset{u_0\in P_0}{min}{J}_i(u_0,u_i,u_{-i}^{♮}).$$

Using the definition of the set $\mathcal{E}\left(\alpha \right)$, for $u_0^{♮}\triangleq \alpha [u^{♮}]$ and each $i=1,\dots ,n$, we have

$$\begin{array}{c}{J}_i(u_0^{♮},u^{♮})=J_i[\alpha ,u^{♮}]\ge J_i[\alpha ,{\widehat{u}}_i,u_{-i}^{♮}]=J_i(\alpha ({\widehat{u}}_i,u_{-i}^{♮}),{\widehat{u}}_i,u_{-i}^{♮})\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill \ge \underset{u_0\in P_0}{min}{J}_i(u_0,{\widehat{u}}_i,u_{-i}^{♮})=\underset{u_i\in P_i}{max}\underset{u_0\in P_0}{min}{J}_i(u_0,u_i,u_{-i}^{♮}).\end{array}$$

Thus, $(\alpha [u^{♮}],u^{♮})\in \mathcal{B}$.

Now, let us prove the second statement of the lemma.

For $u_i\in P_i$ let ${\beta }_i[u_i]\in \mathrm{Argmin}\{J_i(u_0,u_i,u_{-i}^{♮}):u_0\in P_0\}$. Further, an arbitrary $\overline{u}\in P$ is picked.

Put

$$\alpha [u_1,\dots ,u_n]\triangleq \left\{\begin{array}{cc}{u}_0^{♮},& u_i=u_i^{♮},i=1,\dots n,\\ {\beta }_i[u_i],& u_i\ne u_i^{♮},u_j=u_i^{♮},j\ne i,\\ \overline{u},& \mathrm{otherwise}.\end{array}\right.$$

First, notice that $\alpha [u^{♮}]=u_0^{♮}$. Further, if $u\in P$ is such that $u_i\ne u_i^{♮}$ for some i and, for all other j, $u_j=u_j^{♮}$, then

$$J_i(\alpha [u],u)=J_i({\beta }_i(u_i),u_i,u_{-i}^{♮})\le \underset{u_i\in P_i}{max}\underset{u_0\in P_0}{min}{J}_i(u_0,u_i,u_{-i}^{♮})\le J_i(u_0^{♮},u^{♮})=J_i[\alpha ,u^{♮}].$$

This proves the second statement of the lemma. ☐

Theorem 1.

(1) If $({\alpha }^{\ast },u^{\ast })$ is an inverse Stackelberg solution, then the profile of strategies $(u_0^{\ast },u_1^{\ast })$ with $u_0^{\ast }={\alpha }^{\ast }(u_1^{\ast })$ maximizes the value $J_0(u_0^{\ast },u_1^{\ast })$ over the set $\mathcal{B}$. (2) If the profile of strategies $(u_0^{\ast },u_1^{\ast })$ maximizes the value $J_0(u_0^{\ast },u_1^{\ast })$ over the set $\mathcal{B}$, then an incentive strategy (${\alpha }^{\ast }$) exists such that ${\alpha }^{\ast }[u_1^{\ast }]=u_0^{\ast }$, and $({\alpha }^{\ast },u_1^{\ast })$ is an inverse Stackelberg solution. (3) If the function $u_i^{\prime }\mapsto J_i(u_0,u_i^{\prime },u_{-i})$ is quasi-concave for all $u_0$, $u_{-i}$, and $i=1,\dots ,n$, then at least one inverse Stackelberg solution exists.

Proof. 

The proof of the first two statements directly follows from Lemma 1.

Let us prove the third statement of the theorem. Put

$$K_i(u_1,\dots ,u_n)\triangleq \underset{u_0\in P_0}{min}{J}_i(u_0,u_1,\dots ,u_i).$$

The functions $u_i^{\prime }\mapsto K_i(u_i^{\prime },u_{-i})$ are quasi-concave for all $u_{-i}$. Therefore, a profile of followers’ strategies ($u^{♮}$) exists such that all $u_i\in P_i$ $K_i(u^{♮})\ge K_i(u_i,u_{-i}^{♮})$. Hence, we any pair $(u_0,u^{♮})$ belongs to $\mathcal{B}$. Consequently, $\mathcal{B}$ is nonempty. Moreover, the set $\mathcal{B}$ is compact. This proves the existence of the pair $(u_0^{\ast },u^{\ast })$ maximizing $J_0$ over the set $\mathcal{B}$. The existence of inverse Stackelberg solution directly follows on from the second statement of the theorem. ☐

Example 1.

Consider a game with two followers. Let the set of strategies of the players be equal to $\{0,1\}$. In addition, let the followers’ rewards for $u_0=0$ be

a, b	0, 0
0, 0	b, a

where $a>b>0$. Further, let the followers’ rewards for $u_0=1$ be given by

0, 0	a, b
b, a	0, 0

Finally, we assume that the leader’s reward is equal to 1 when the followers outcome is $(0,0)$ and 0 in the opposite case. One can consider this game as a variant of the battle of sexes with the leader who can shift the roles of the players and win when there is no arrangement between the players.

It is easy to check that the set $\mathcal{B}$ is equal to the set of all strategies ${\{0,1\}}^3$. By maximizing the leader’s payoff over this set we get that the outcome of the players is $(1,0,0)$.

It is instructive to compare the result with the case where the leader declares his strategy first. Clearly, in this case, whatever the leader’s strategy is, the leader’s outcome is 0, whereas the flowers’ Nash equilibrium payoffs are $(a,b)$ and $(b,a)$.

3. Inverse Stackelberg Solution for Differential Games

As above we assume that player 0 is a leader when players $1,\dots ,n$ are followers. The dynamics of the system is given by the equation

$$\dot{x}=f(t,x,u_0,u_1,\dots ,u_n),t\in [0,T],x\in {\mathbb{R}}^d,x\left(0\right)=x_0,u_i\in P_i.$$

(1)

Player i wishes to maximize the payoff

$${\sigma }_i(x(T))+{\int }_0^T{g}_i(t,x,u_0,u_1,\dots ,u_n)dt.$$

The set

$${\mathcal{U}}_i=\{u_i:[0,T]\to P_{i}measurable\}$$

is the set of open-loop strategies of player i. As above, the n-tuple of open-loop strategies of followers ($u=(u_1,\dots ,u_n)$) is called the profile of strategies. To simplify notations, denote

$$f(t,x,u_0,u)\triangleq f(t,x,u_0,u_1,\dots ,u_n),g(t,x,u_0,u)\triangleq g(t,x,u_0,u_1,\dots ,u_n).$$

Further, put

$$\mathcal{U}={\times }_{i=1}^n{\mathcal{U}}_i.$$

If $u_0\in {\mathcal{U}}_0$, $u=(u_1,\dots ,u_n)\in \mathcal{U}$, $(t_{\ast },x_{\ast })\in [0,T]\times {\mathbb{R}}^d$, then denote by $x(·,t_{\ast },x_{\ast },u_0,u)$ the solution of initial value problem

$$\dot{x}\left(t\right)=f(t,x\left(t\right),u_0\left(t\right),u_1\left(t\right),\dots ,u_n\left(t\right)),x(t_{\ast })=x_{\ast }.$$

Put

$$z_i(t,t_{\ast },x_{\ast },u_0,u)={\int }_{t_{\ast }}^t{g}_i(t,x\left(t\right),u_0\left(t\right),u_1\left(t\right),\dots ,u_n\left(t\right))dt.$$

If $t_{\ast }=0$, $x_{\ast }=x_0$ we omit the arguments $t_{\ast }$ and $x_{\ast }$. Let $z(·,t_{\ast },x_{\ast },u_0,u)=(z_0(·,t_{\ast },x_{\ast },u_0,u),z_1(·,t_{\ast },x_{\ast },u_0,u),\dots ,z_n(·,t_{\ast },x_{\ast },u_0,u))$. We assume that the set of motions is closed, i.e., for all $(t_{\ast },x_{\ast })\in [0,T]\times {\mathbb{R}}^d$,

$$\begin{array}{c}\mathrm{cl}\{(x(·,t_{\ast },x_{\ast },u_0,u),z(·,t_{\ast },x_{\ast },u_0,u)):u_0\in {\mathcal{U}}_0,u\in \mathcal{U}\}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill =\{(x(·,t_{\ast },x_{\ast },u_0,u),z(·,t_{\ast },x_{\ast },u_0,u)):u_0\in {\mathcal{U}}_0,u\in \mathcal{U}\}.\end{array}$$

Here, $\mathrm{cl}$ stands for the closure in the space of continuous functions from $[0,T]$ to ${\mathbb{R}}^d$.

We assume that the followers use open-loop strategies ($u_i\in {\mathcal{U}}_i$) when the leader’s strategy is a nonanticipative strategy ($\alpha :\mathcal{U}\to {\mathcal{U}}_0$). The nonanticipation property means that $\alpha [u]\left(\tau \right)=\alpha [u^{\prime }]\left(\tau \right)$ for any u and $u^{\prime }$ coinciding on $[0,\tau ]$.

For $u_0\in {\mathcal{U}}_0$, $u\in \mathcal{U}$, $(t_{\ast },x_{\ast })$ define

$$J_i(t_{\ast },x_{\ast },u_0,u)\triangleq {\sigma }_i(x(T,t_{\ast },x_{\ast },u_0,u))+z_i(T,t_{\ast },x_{\ast },u_0,u).$$

Further, put

$$J_i[t_{\ast },x_{\ast },\alpha ,u]\triangleq J_i(t_{\ast },x_{\ast },\alpha \left(u\right),u).$$

We omit the arguments $t_{\ast }$ and $x_{\ast }$ if $t_{\ast }=0$, $x_{\ast }=x_0$.

We assume that the followers’ solution concept is Nash equilibrium. Let ${\mathcal{E}}_d\left(\alpha \right)$ denote the set of Nash equilibria in the case when the leader plays with the nonanticipating strategy $\alpha$:

$${\mathcal{E}}_d\left(\alpha \right)\triangleq \{u\in \mathcal{U}:J_i[\alpha ,u]\ge J_i[\alpha ,u_i^{\prime },u_{-i}]\mathrm{for}\mathrm{all}{u}_i^{\prime }\in {\mathcal{U}}_i\mathrm{and}\mathrm{any}i=\overline{1,n}\}.$$

Denote the set of nonanticpating strategies by ${\mathsf{\Gamma }}^{\ast }$.

Definition 2.

The pair consisting of a nonanticipative strategy of the leader (${\alpha }^{\ast }$) and $u^{\ast }\in \mathcal{U}$ is an inverse Stackelberg solution of the differential game if

(1) 

$u^{\ast }\in {\mathcal{E}}_d({\alpha }^{\ast })$

(2) 

$J_0[{\alpha }^{\ast },u^{\ast }]={max}_{\alpha }{max}_{u\in {\mathcal{E}}_d\left(\alpha \right)}{J}_0[\alpha ,u].$

The proposed definition is analogous to the definition of the inverse Stackelberg solution for static games. The characterization in the differential game case is close to the characterization in the static game case.

For a fixed profile of strategies of all players but the i-th $u_{-i}$, one can consider the zero-sum differential game of player 0 and player i. In this case, we assume that player 0 uses the nonaticipating strategies on $[t_{\ast },T]$ which are mappings (${\beta }_i:{\mathcal{U}}_i\to {\mathcal{U}}_0$) that satisfy the feasibility condition: if $u_i^{\prime }=u_i^{″}$ on $[t_{\ast },\tau ]$, then ${\beta }_i[u_i^{\prime }]={\beta }_i[u_i^{″}]$ on $[t_{\ast },\tau ]$. Denote the set of feasible mappings ${\beta }_i:{\mathcal{U}}_i\to {\mathcal{U}}_0$ by ${\mathsf{\Gamma }}_i[t_{\ast }]$. The lower value of this game is

$$V_i^{-}(t_{\ast },x_{\ast },u_{-i})\triangleq \underset{{\beta }_i\in {\mathsf{\Gamma }}_i[t_{\ast }]}{min}\underset{u_i^{\prime }\in {\mathcal{U}}_i}{max}{J}_i(t_{\ast },x_{\ast },{\beta }_i[u_i^{\prime }],u_i^{\prime },u_{-i}).$$

Let

$$\begin{array}{c}\mathcal{C}=\{(u_0,u)\in {\mathcal{U}}_0\times \mathcal{U}:\mathrm{for}\mathrm{any}i=\overline{1,n},t\in [0,T],\mathrm{and}x(·)=x(·,u_0,u)\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill J_i(t,x\left(t\right),u_0,u)\ge V_i^{-}(t,x\left(t\right),u_{-i})\}.\end{array}$$

Lemma 2.

Let α be an incentive strategy of the leader. If $u^{♮}\in {\mathcal{E}}_d\left(\alpha \right)$, then $(\alpha [u^{♮}],u^{♮})\in \mathcal{C}$.

Proof. 

Denote

$$u_0^{♮}\triangleq \alpha [u^{♮}].$$

We claim that

$$J_i[t,x^{♮}\left(t\right),u_0^{♮},u^{♮}]\ge J_i[t,x^{♮}\left(t\right),\alpha [u_i^{\prime },u_{-i}^{♮}],u_i^{\prime },u_{-i}^{♮}]$$

(2)

for any $u_i^{\prime }\in {\mathcal{U}}_i$, $u_0^{♮}=\alpha (u^{\ast })$, $x^{♮}=x(·,\alpha [u^{\ast }],u^{♮})$. Assume the converse. This means that, for some $u_i^{\prime }$ and $\tau$,

$$J_i[\tau ,x^{♮}(\tau ),u_0^{♮},u^{♮}]<J_i[\tau ,x^{♮}\left(\tau \right),\alpha [u_i^{\prime },u_{-i}^{♮}],u_i^{\prime },u_{-i}^{♮}].$$

(3)

Let us introduce the control ($u_i^{♭}$) by the following rule:

$$u_i^{♭}\triangleq \left\{\begin{array}{cc}{u}_i^{♮}\left(t\right),& t\in [0,\tau ]\\ u_i^{\prime }\left(t\right),& t\in [\tau ,T].\end{array}\right.$$

Further, denote

$$u_0^{♭}\triangleq \alpha [u_i^{♭},u_{-i}^{♮}],$$

$$x^{♭}(·)=x(·,u_0^{♭},(u_i^{♭},u_{-i}^{♮})).$$

We have

$$J_i[\alpha ,u_i^{♭},u_{-i}^{♮}]=\sigma (x^{♭}\left(T\right))+{\int }_0^{T}g(t,x^{♭}\left(t\right),u_0^{♭},(u_i^{♭},u_{-i}^{♮}))dt.$$

Since, for $t\in [0,\tau ]$,

$$u_i^{♭}\left(t\right)=u_i^{♮}\left(t\right),u_0^{♭}=u_0^{♮}\left(t\right)=\alpha [u^{♮}]\left(t\right),x^{♭}\left(t\right)=x^{♮}\left(t\right),$$

and, for $t\in [\tau ,T]$,

$$x^{♭}\left(t\right)=x(t,\tau ,x^{♮}\left(\tau \right),u_0^{♭},(u_i^{\prime },u_{-i}^{♮})),$$

Equation (3) implies the following inequality:

$$J_i[\alpha ,u_i^{♭},u_{-i}^{♮}]>{\int }_0^{\tau }{g}_i(t,x^{♮}\left(t\right),u_0^{♮},u^{♮})dt+J[\tau ,x^{♮}\left(\tau \right),\alpha ,u^{♮}]=J[\alpha ,u^{♮}].$$

This contradicts the assumption that $u^{♮}\in {\mathcal{E}}_d\left(\alpha \right)$.

The inequality (2) yields the inequality $J_i[t,x^{♮}\left(t\right),u_0^{♮},u^{♮}]\ge V_i^{-}(t,x^{♮}\left(t\right),u_{-i}^{♮})$. ☐

Lemma 3.

For any $(u_0^{♮},u^{♮})\in \mathcal{C}$, a nonanticipative strategy of the leader (α) exists so that $\alpha (u^{♮})=u_0^{♮}$ and $u^{♮}\in {\mathcal{E}}_d(\alpha )$.

Proof. 

Denote $x^{♮}(·)=x(·,u_0^{♮},u^{♮})$.

Pick $u\in \mathcal{U}$. Let $i_1,i_2,\dots ,i_n\in \overline{1,n}$, and let ${\tau }_{i_1},\dots ,{\tau }_{i_n}\in [0,T]$ satisfy the following properties

(1) 

$i_1,\dots ,i_n$ is a permutation of $1,\dots ,n$;

(2) 

${\tau }_{i_1}\le {\tau }_{i_2},\dots ,{\tau }_{i_n}$;

(3) 

for each k, $t_{i_k}$ is the greatest time such that $u_{i_k}=u_{i_k}^{♮}$ on $[0,{\tau }_{i_k}]$.

Let $y_{i_1}=x^{♮}({\tau }_{i_1})$. The mapping ${\beta }^{i_1}\in {\mathsf{\Gamma }}_{i_1}[{\tau }_{i_1}]$ exists such that

$$V_i({\tau }_{i_1},y_{i_1},u_{-i_i}^{♮})=\underset{u_{i_1}\in {\mathcal{U}}_{i_1}}{max}J({\tau }_{i_1},y_{i_1},{\beta }^{i_1}[u_{i_1}],u_{i_1},u_{-i_1}^{♮}].$$

Further, pick ${\overline{u}}_0\in \mathcal{U}$ arbitrarily.

Put

$$\alpha [u]\triangleq \left\{\begin{array}{cc}{u}_0^{♮},\hfill & t\in [0,{\tau }_{i_1}];\\ {\beta }^{i_1}[u_{i_1}],\hfill & t\in ({\tau }_{i_1},{\tau }_{i_2}];\\ {\overline{u}}_0,\hfill & t\in ({\tau }_{i_2},T].\end{array}\right\}$$

Notice that $\alpha [u^{♮}]=u_0^{♮}$. Now let $u=(u_i^{\prime },{♮}_{-i})$ Denote by $\tau$ the greatest time such that $u_i=u_i^{♮}$ on $[0,\tau ]$. In this case, $i_1=i$, ${\tau }_{i_1}=\tau$, ${\tau }_{i_k}=T$ for $k=2,\dots ,n$. By construction, we have

$$\begin{array}{c}{J}_i[\alpha ,u_i,u_{-i}^{♮}]={\int }_0^{\tau }{g}_i(t,x^{♮}\left(t\right),u_0^{♮}\left(t\right),u^{♮}\left(t\right))dt+J_i[\tau ,x^{♮}\left(t\right),\alpha ,u_i,u_{-i}^{♮}]\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ ={\int }_0^{\tau }{g}_i(t,x^{♮}\left(t\right),u_0^{♮}\left(t\right),u^{♮}\left(t\right))dt+V_i(\tau ,x^{♮}\left(\tau \right),u_{-i}^{♮})\\ \hfill \le {\int }_0^{\tau }{g}_i(t,x^{♮}\left(t\right),u_0^{♮}\left(t\right),u^{♮}\left(t\right))dt+J_i(\tau ,x^{♮}\left(\tau \right),u_0^{♮},u^{♮})=J_i[\alpha ,u^{♮}].\end{array}$$

 ☐

Theorem 2.

(1) If the pair $({\alpha }^{\ast },u^{\ast })$ is an inverse Stackelberg solution then $(u_0^{\ast },u^{\ast })\in \mathcal{C}$, and $(u_0^{\ast },u_1^{\ast })$ maximizes the value $J_0$ over the set $\mathcal{C}$ for $u_0^{\ast }={\alpha }^{\ast }[u^{\ast }]$. (2) Conversely, if the pair $(u_0^{\ast },u_1^{\ast })$ maximizes the value $J_0$ over the set $\mathcal{C}$, then an incentive strategy of the leader ${\alpha }^{\ast }$ exists such that ${\alpha }^{\ast }[u_1^{\ast }]=u_0^{\ast }$ and $({\alpha }^{\ast },u_1^{\ast })$ is an incentive Stackelberg solution.

The theorem directly follows from the Lemmas 2 and 3.

4. Existence of the Inverse Stackelberg Solution for Differential Game

In this section, we consider the differential game in the mixed strategies. This means that we replace the system (1) with the control system described by the following equation:

$$\dot{x}\left(t\right)={\int }_{P_0}{\int }_{P_1}\dots {\int }_{P_n}f(t,x\left(t\right),u_0,u_1,\dots ,u_n){\mu }_n(t,d{u}_n)\dots {\mu }_1(t,d{u}_1){\mu }_0(t,d{u}_0).$$

(4)

Here, ${\mu }_i(t,·)$ are probabilistic measures on $P_i$.

The relaxation means that we replace the control spaces $P_i$ with the control spaces $\mathrm{rpm}(P_i)$. Therefore, the open-loop strategy of the i-th player is a weakly measurable function: ${\mu }_i:[0,T]\to \mathrm{rpm}(P_i)$. This means that the mapping

$$t\mapsto {\int }_{P_i}\varphi (u_i){\mu }_i(t,d{u}_i)$$

is measurable for any continuous function ($\phi \in C(P_i)$). The set of open-loop strategies of the i-th player is denoted by $\mathcal{M}$.

Further, we use the following designations. Put

$$P\triangleq {\times }_{j=1}^n{P}_j,P_{-i}\triangleq {\times }_{j\ne i}{P}_j.$$

If $m_j\in \mathrm{rpm}(P_j)$, $j=1,\dots ,n$, then denote $m(du)=m_1(d{u}_1)\dots m_n(d{u}_n)$ with a slight abuse of notation. Further, for $\phi \in C(P)$,

$${\int }_P\phi \left(u\right)m\left(du\right)={\int }_{P_1}\dots {\int }_{P_n}\phi (u_1,\dots ,u_n)m_1(d{u}_1)\dots m_n(d{u}_n).$$

Analogously, we assume that $m_{-i}(d{u}_{-i})\triangleq {\times }_{j\ne i}{m}_j(d{u}_j)$. Thus,

$$\begin{array}{c}{\int }_{P_{-i}}\phi (u_{-i})m_{-i}(d{u}_{-i})={\int }_{P_1}\dots {\int }_{P_{i-1}}{\int }_{P_{i+1}}\dots {\int }_{P_n}\phi (u_1,\dots ,u_{i-1},u_{i+1},\dots ,u_n)\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill m_1(d{u}_1)\dots m_{i-1}(d{u}_{i-1})m_{i+1}(d{u}_{i+1})\dots m_n(d{u}_n).\end{array}$$

If $(t_{\ast },x_{\ast })\in [0,T]\times {\mathbb{R}}^d$, ${\mu }_0\in {\mathcal{M}}_0$, ${\mu }_1\in {\mathcal{M}}_1$,…, ${\mu }_n\in {\mathcal{M}}_n$, then we denote the solution of the initial value problem for equation (4) and the position $(t_{\ast },x_{\ast })$ by $x(·,t_{\ast },x_{\ast },{\mu }_0,{\mu }_1,\dots ,{\mu }_n)$.

As above, we call the n-tuple $\mu =({\mu }_1,\dots ,{\mu }_n)$ the profile of followers’ mixed strategies. Denote the set of followers’ strategies by $\mathcal{M}$. Put $x(·,t_{\ast },x_{\ast },{\mu }_0,\mu )=x(·,t_{\ast },x_{\ast },{\mu }_0,{\mu }_1,\dots ,{\mu }_n)$, $x(·,t_{\ast },x_{\ast },{\mu }_0,{\mu }_i^{\prime },{\mu }_{-i})=x(·,t_{\ast },x_{\ast },{\mu }_0,({\mu }_i^{\prime },{\mu }_{-i}))$.

For the given position $(t_{\ast },x_{\ast })\in [0,T]\times {\mathbb{R}}^d$ and measures ${\mu }_0\in {\mathcal{M}}_0$, $\mu \in \mathcal{M}$, the corresponding payoff of player i is equal to

$$J_i(t_{\ast },x_{\ast },{\mu }_0,\mu )={\sigma }_i(x(T,t_{\ast },x_{\ast },{\mu }_0,\mu ))+{\int }_{t_{\ast }}^T{\int }_{P_0}{\int }_P{g}_i(t,x(t,t_{\ast },x_{\ast },{\mu }_0,\mu ),u_0,u){\mu }_0(t,d{u}_0)\mu (t,du)dt.$$

As above, the mapping $\alpha :\mathcal{M}\to {\mathcal{M}}_0$ satisfying the condition of feasibility (the equality ${\mu }^{\prime }$ and ${\mu }^{\prime \prime }$ on $[0,\tau ]$ yields the equality $\alpha [{\mu }^{\prime }](t,·)=\alpha [{\mu }^{″}](t,·)$ on $[0,\tau ]$) is called the nonanticipative strategy. We denote the set of nonanticipating strategies by ${\mathsf{\Gamma }}^{\ast }$. Analogously, the set of mappings ${\beta }_i:{\mathcal{M}}_i\to {\mathcal{M}}_0$ satisfying the feasibility property on $[t_{\ast },T]$ is denoted by ${\mathsf{\Gamma }}_i[t_{\ast }]$.

Further, we use the nonanticipating strategies of player i. This is a mapping ${\gamma }_i:{\mathcal{M}}_0\to {\mathcal{M}}_i$ satisfying the feasibility property on $[t_{\ast },T]$: if ${\mu }_0^{\prime }={\mu }_0^{″}$ on $[t_{\ast },\tau ]$, then ${\gamma }_i[{\mu }_0^{\prime }]={\gamma }_i[{\mu }_0^{″}]$ on $[t_{\ast },\tau ]$. Let ${\mathcal{N}}_i$ stand for the set of nonanticipating strategies of player i on $[t_{\ast },T]$. By using these strategies, one can introduce the upper value function by the rule: if $(t_{\ast },x_{\ast })\in [0,T]\times {\mathbb{R}}^d$, ${\mu }_1\in {\mathcal{M}}_1$, …, ${\mu }_{i-1}\in {\mathcal{M}}_{i-1}$, ${\mu }_{i+1}\in {\mathcal{M}}_{i+1}$,…,${\mu }_n\in {\mathcal{M}}_n$, then

$$V^{+}(t_{\ast },x_{\ast },{\mu }_{-i})\triangleq \underset{{\beta }_i\in {\mathcal{N}}_i}{max}\underset{{\mu }_0\in {\mathcal{M}}_0}{min}{J}_i(t_{\ast },x_{\ast },{\mu }_0,{\gamma }_i[{\mu }_0],{\mu }_{-i}).$$

Generally,

$$V^{+}(t_{\ast },x_{\ast },{\mu }_{-i})\ge V^{-}(t_{\ast },x_{\ast },{\mu }_{-i}).$$

(5)

Theorem 3.

Assume that the following conditions hold true for each $i=\overline{1,n}$:

(1) 

$x\mapsto {\sigma }_i\left(x\right)$ is concave;

(2) 

$g_i(t,x,u_0,u)=g_i^0(t,x,u_{-i})+g_i^1(t,u_0,u_{-i})+g_i^2(t,u)$ and the function $x\mapsto g_i^0(t,x,u_{-i})$ is concave.

Then, an inverse Stackelberg solution exists in mixed strategies $({\alpha }^{\ast },{\mu }^{\ast })$.

Proof. 

Let us prove that the set $\mathcal{C}$ is nonempty.

Define the multivalued map $\mathcal{G}:{\mathcal{M}}_0\times \mathcal{M}⊸{\mathcal{M}}_0\times \mathcal{M}$ by the rule $({\mu }_0^{\prime },{\mu }^{\prime })\in \mathcal{G}({\mu }_0,\mu )$ if, for each $i=\overline{1,n}$,

$$J_i(t,x_i\left(t\right),{\mu }_0^{\prime },{\mu }_i^{\prime },{\mu }_{-i})\ge V_i^{-}(t,x_i\left(t\right),{\mu }_{-i}).$$

Here, $x_i(·)=x(·,{\mu }_0,{\mu }_i,{\mu }_{-i})$.

The assumption of the theorem implies that the set $\mathcal{G}({\mu }_0,\mu )$ is convex for all ${\mu }_0\in {\mathcal{M}}_0$, $\mu \in \mathcal{M}$. Moreover, $\mathcal{G}$ has a closed graph. Let us prove the nonemptiness of $\mathcal{G}({\mu }_0,\mu )$.

Put ${\mu }_0^{\prime }={\mu }_0$. From the Bellman principle, it follows that

$$\begin{array}{c}{V}_i^{+}(t_{\ast },x_{\ast },{\mu }_{-i})=\underset{{\gamma }_i\in {\mathcal{N}}_i}{max}\underset{{\nu }_0\in {\mathcal{M}}_0}{min}[V(t_{+},x(t_{+},t_{\ast },x_{\ast },{\nu }_0,{\gamma }_i[{\nu }_0],{\mu }_{-i}))\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ +{\int }_{t_{\ast }}^{t^{+}}{\int }_{P_0}{\int }_{P_i}{\int }_{P_{-i}}{g}_i(t,x(t_{+},t_{\ast },x_{\ast },{\nu }_0,{\gamma }_i({\nu }_0),{\mu }_{-i})),u_0,u_{-i},u_{-i})\\ \hfill {\mu }_{-i}(t,d{u}_{-i}){\gamma }_i[{\nu }_0](t,d{u}_i){\nu }_0(t,d{u}_0)dt].\end{array}$$

(6)

Let N be a natural number. Put $t_N^k=Tk/N$. Let ${\gamma }_{i,N}^k$ maximize the right-hand side at (6) for $t_{\ast }=t_N^k$, $t_{+}=t_N^{k+1}$, $x_{\ast }=y_{i,N}^{k-1}$. Here $y_{i,N}^k$ is defined inductively by the rule

$$y_{i,N}^0=x_0,y_{i,N}^k=x(t_{i,N}^k,t_{i,N}^{k-1},y_{i,N}^{k-1},{\mu }_0,{\gamma }_{i,N}^{k-1}[{\mu }_0],{\mu }_{-i}).$$

Put ${\tilde{\mu }}_{i,N}(t,·)={\gamma }_{i,N}^k[{\mu }_0](t,·)$ for $t\in [t_N^{k-1},t_N^k)$. Denote $x_{i,N}(·)=x(·,t_0,x_0,{\mu }_0,{\tilde{\mu }}_{i,N},{\mu }_{-i})$. Notice that $y_{i,N}^k=x_{i,N}(t_N^k)$. We have, for $k<l$, the inequality

$$\begin{array}{c}{V}_i^{+}(t_N^k,x_{i,N}(t_N^k),{\mu }_{-i})\le V_i^{+}(t_N^l,x_{i,N}(t_N^l),{\mu }_{-i})\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+{\int }_{t_N^k}^{t_N^l}{\int }_{P_0}{\int }_{P_i}{\int }_{P_{-i}}{g}_i(t,x_{i,N}\left(t\right),u_0,u_i,u_{-i}){\mu }_{-i}(t,d{u}_{-i}){\tilde{\mu }}_{i,N}(t,d{u}_i){\mu }_0(t,d{u}_0)dt.\end{array}$$

Note that $V_i^{+}(t_N^N,y_{i,N}^N,{\mu }_{-i})={\sigma }_i({\xi }_{i,N}^N)$.

Using the continuity of function $V_i^{+}$, we get

$$\begin{array}{c}{V}_i^{+}(t_{\ast },x_{i,N}(t_{\ast }),{\mu }_{-i})\le V_i^{+}(T,x_{i,N}\left(T\right),{\mu }_{-i})\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill +{\int }_{t_{\ast }}^T{\int }_{P_0}{\int }_{P_i}{\int }_{P_{-i}}{g}_i(t,x_{i,N}\left(t\right),u_0,u_i,u_{-i}){\mu }_{-i}(t,d{u}_{-i}){\tilde{\mu }}_{i,N}(t,d{u}_i){\mu }_0(t,d{u}_0)dt+{\delta }_N.\end{array}$$

(7)

Here, ${\delta }_N\to 0$, as $N\to \infty$.

The sequence $\{{\tilde{\mu }}_{i,N_r}\}$ converges to some ${\mu }_i^{\prime }\in {\mathcal{M}}_i$, as $r\to \infty$. Therefore, $x_{i,N_r}(·)=x(·,t_0,x_0,{\mu }_0,{\tilde{\mu }}_{i,N_r},{\mu }_{-i})$ tends to $x_i(·)=x(·,t_0,x_0,{\mu }_0,{\mu }_i^{\prime },{\mu }_{-i})$. This and inequalities (5), (7) yield the inequality for any $t_{\ast }\in [t_0,T]$:

$$\begin{array}{c}{V}_i^{-}(t_{\ast },x_i(t_{\ast }),{\mu }_{-i})\le V_i^{+}(t_{\ast },x_i(t_{\ast }),{\mu }_{-i})\le V_i^{+}(T,x_i(T),{\mu }_{-i})\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill \\ \hfill +{\int }_{t_{\ast }}^T{\int }_{P_0}{\int }_{P_i}{\int }_{P_{-i}}{g}_i(t,x_i\left(t\right),u_0,u_i,u_{-i}){\mu }_{-i}(t,d{u}_{-i}){\mu }_i^{\prime }(t,d{u}_i){\mu }_0(t,d{u}_0)dt.\end{array}$$

Put ${\mu }^{\prime }\triangleq ({\mu }_1^{\prime },\dots ,{\mu }_n^{\prime })$. We have $({\mu }_0,{\mu }^{\prime })\in \mathcal{G}({\mu }_0,\mu )$.

Since ${\mathcal{M}}_0\times \mathcal{M}$ is compact, and $\mathcal{G}$ is an upper semicontinuous multivalued map with nonempty convex compact values, $\mathcal{G}$ admits the fixed point $({\mu }_0^{\ast },{\mu }^{\ast })$. Obviously, it belongs to $\mathcal{C}$. The consequence of the theorem follows from this and theorem 2. ☐