Strong Time-Consistent Solution for Cooperative Differential Games with Network Structure

Abstract

One class of cooperative differential games on networks is considered. It is assumed that interaction on the network is possible not only between neighboring players, but also between players connected by paths. Various cooperative optimality principles and their properties for such games are investigated. The construction of the characteristic function is proposed, taking into account the network structure of the game and the ability of players to cut off connections. The conditions under which a strong time-consistent subcore is not empty are studied. The formula for explicit calculation of the Shapley value is derived. The results are illustrated by the example of one differential marketing game.

1. Introduction

One of the current directions of research in game theory is network games. Such games explore individuals’ interactions connected through a network and whose behavior depends on their neighbors’ behavior. An overview and structuring of the main trends in network games can be found in [1,2,3,4].

The theory of differential games on networks is commonly used when the evolution of decision-making occurs continuously in time. The papers investigating applied models of differential games on networks confirm the relevance of this topic. For example, a differential game of a duopoly with network externalities is examined in [5]. Differential games with network structure in marketing are considered in [6]. An application of such games in regional cooperation is developed in [7]. Adaptation of differential games to the problem of network traffic is proposed in [8]. Cooperative differential games on networks were first considered by L. Petrosyan [9], who introduced a new type of strategies. The possibility of cutting the links with neighboring players during the game is included in these strategies. When evaluating a coalition’s worth, the classical methods assume that players who are not in this coalition minimize the coalition’s payoff. The use of the new type of strategies has led to the possibility of measuring a coalition’s worth without considering the actions of players who are not members of this coalition. Based on this, a novel form for characteristic function—named as cooperative-trajectory characteristic function—is proposed in [10]. Moreover, it satisfies the convexity property. In [11], formulas for the Shapley value and some core imputations are obtained using the new characteristic function. In this paper, we generalize the approach used in [10,11] on a more interesting case when the payoff of a player depends not only on his neighbors’ actions but also on players’ actions with whom paths in the network connect the player. This case differs essentially from one considered in [10]. In such games, the convexity of the new characteristic function depends on the structure of the network on which the game is defined.

In this paper, we consider cooperative differential games on networks defined on a given time-interval. We suppose that payoffs are transferable. Under the cooperation, we understand that players choose their strategies to maximize the sum of payoffs and then define a rule (optimality principle) on how to allocate this joint payoff between them. There are different optimality principles such as core, NM-solution, the Shapley Value, and others in classical and differential game theory. However, a serious problem arises with time-consistency and, in the case of set-valued optimality principles, also with strong time-consistency, which very often is not satisfied (see [12,13]). This makes questionable the application of the optimality principles mentioned above.

Thus, our main contributions to the literature on differential network games are the following:

• A new characteristic function for the game with interactions through paths is defined.
• It is found that the properties of such a characteristic function depend on the game’s network structure.
• A condition under which the characteristic function is convex is obtained.
• Under this condition, we prove the non-emptiness of the strong time-consistent subcore and receive explicit forms for the Shapley value and the IDP of imputations from the strong time-consistent subcore.

The paper is structured as follows. The definition of the cooperative differential game on a network is given in Section 2. In Section 3, the definition of the characteristic function based on cooperative strategies used by players from a coalition is given. As mentioned above, the basic difference from previous approaches is that, when defining the value of the characteristic function for a given coalition, it is supposed that the left out players trying to minimize the payoff of the coalition are cutting connections with players from this coalition. Based on defined characteristic function, the core is constructed, and a strong time-consistent solution as a subset of the core is proposed in Section 4. The formula for the dynamic Shapley value is derived in Section 5. As an illustrative example, a differential marketing game on the network is investigated in Section 6. For this game, the characteristic function, the strong time-consistent subset of the core, and the dynamic Shapley value are computed in explicit form.

2. Problem Formulation

Consider a class of n-person differential games with prescribed duration of T. Let $N=\{1;2;\dots ;n\}$ be the set of players. The players are connected in a network system. A pair $(N,L)$ is called a network, where N is a set of nodes and $L\subset N\times N$ is a given set of links. The nodes are used to represent the players. If pair $(i,j)\in L$, there is a link connecting players $i\in N$ and $j\in N$. It is supposed that all links are undirected.

Denote the set of players directly connected to player i as $K\left(i\right)=\{j:(i;j)\in L\}$.

Denote by $K^m\left(i\right),$ where $m\ge 2$, the set of players connected with player $i\in N$ by a minimal path containing exactly m edges (only paths without cycles and loops are considered) and let $K^1\left(i\right)=K\left(i\right)\cup i$, for $i\in N$.

Every player $i\in N$ at any instant of time can cut the connection with any other players from $K\left(i\right)$.

The system dynamics is given by

$${\dot{x}}_i\left(\tau \right)=f_i(x_i\left(\tau \right);u_i\left(\tau \right)); x_i\left(t_0\right)=x_i^0;\mathrm{for}\tau \in [t_0;T]\mathrm{and}i\in N.$$

(1)

Here, $x_i\left(t\right)\subset R^m$ is the state variable of player $i\in N$ at time t and $u_i\left(t\right)\in U_i\subset R^k$ is the control variable of player $i\in N$. The function $f_i(x_i;u_i)$ is continuously differentiable in $x_i$ and $u_i$.

The payoff function of player i depends on his state variable, the state variables of players from the sets $K^m\left(i\right)$, $1\le m\le n-1$, and his control variable. In particular, the payoff of player i is given as

$$H_i(x^0;u_1,\dots ,u_n)=\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))d\tau .$$

(2)

The term $h_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))$ is the instantaneous gain that player i can obtain through interaction with player $j\in K^m\left(i\right)$ and $h_i^i(x_i\left(\tau \right);x_i\left(\tau \right);u_i\left(\tau \right))$ is the instantaneous gain that player i can obtain by itself. Assume that $\delta \in (0,1)$. The multiplier ${\delta }^{m-1}$ shows that, the farther players are in the network from player i, the less their behavior influences the payoff of this player. Suppose that functions $h_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))$, for $j\in K^m\left(i\right),j\ne i$ are non-negative. We denote by $x^0=(x_1^0;x_2^0;\dots ;x_n^0)$ the vector of initial conditions.

We say that we have the game $\Gamma (x^0,T-t_0)$ if the network $(N,L)$ is defined, the system dynamics (1) and the sets of feasible controls $U_i$, $i\in N$ are given, and the players’ payoffs are determined by (2). Each player, choosing a control variable $u_i$ from his set of feasible controls, steers his own state according to the differential Equation (1) and seeks to maximize his objective functional (2).

Suppose that players can cooperate in order to achieve the maximum joint payoff

$$\sum _{i\in N}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))d\tau .$$

(3)

subject to dynamics (1).

The optimal cooperative strategies of players $\overline{u}\left(t\right)=({\overline{u}}_1\left(t\right),\dots ,{\overline{u}}_n\left(t\right))$, for $t\in [t_0;T]$ are defined as follows

$$\overline{u}\left(t\right)=arg\underset{u_1,\dots ,u_n}{max}\sum _{i\in N}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))d\tau .$$

(4)

The trajectory corresponding to the optimal cooperative strategies $({\overline{u}}_1\left(t\right),\dots ,{\overline{u}}_n\left(t\right))$ is the optimal cooperative trajectory $\overline{x}\left(t\right)=({\overline{x}}_1\left(t\right);{\overline{x}}_2\left(t\right);\dots ;{\overline{x}}_n\left(t\right))$. The maximum joint payoff can be expressed as:

$$\sum _{i\in N}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j({\overline{x}}_i\left(\tau \right);{\overline{x}}_j\left(\tau \right);{\overline{u}}_i\left(\tau \right))d\tau =\underset{u_1,\dots ,u_n}{max}\left\{\sum _{i\in N}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j(x_i\left(\tau \right);x_j\left(\tau \right);u_i\left(\tau \right))d\tau \right\}$$

(5)

subject to dynamics

$${\dot{\overline{x}}}_i=f_i({\overline{x}}_i\left(\tau \right);{\overline{u}}_i\left(\tau \right)); {\overline{x}}_i\left(t_0\right)=x_i^0;\mathrm{for}\tau \in [t_0;T]\mathrm{and}i\in N.$$

(6)

To determine how to allocate the maximum total payoff among the players under an agreeable scheme, defining the characteristic function is necessary.

There are many approaches to define the characteristic function (see [14,15,16]).

In [9], for differential games on networks, it is supposed that the worth of the coalition $S\subset N$ does not take into account the actions of players from the coalition $N\backslash S$, since the worst thing they can do for the coalition S is to cut the connection with players from S. In [10], it is also proposed to find the value of the characteristic function of S on the cooperative trajectory when the players from S use cooperative strategies under the condition that the connections with players from $N\backslash S$ are cut off. The characteristic function constructed in this way is easier to compute and possesses some advantageous properties. In this paper, we apply this approach to the class of games under consideration.

Let $S\subset N$. A pair $(S,L_S)$ is called a subnet (subgraph) if it only has subset S of the set of vertices (players) of the original network and $L_S$ contains all links from L whose initial and final vertices are both within subset S. For player $i\in S$, denote by $K_S^m\left(i\right),$ where $m\ge 2$, the set of players connected with player i by a minimal path in $L_S$ containing exactly m edges and let $K_S^1\left(i\right)=K\left(i\right)\cap S\cup i$, for $i\in S$.

The worth of coalition S in the game is evaluated along the cooperative trajectory

$$V(S;x_0,T-t_0)=\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_S^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}{h}_i^j({\overline{x}}_i\left(\tau \right);{\overline{x}}_j\left(\tau \right);{\overline{u}}_i\left(\tau \right))d\tau ,$$

(7)

where ${\overline{x}}_i\left(t\right)$ and ${\overline{u}}_i\left(t\right)$ are the solutions obtained in (4) and (6).

Similarly, the cooperative-trajectory characteristic function of the subgame $\Gamma (\overline{x}\left(t\right),T-t)$ starting at time $t\in [t_0;T]$ can be evaluated as

$$V(S;\overline{x}\left(t\right),T-t)=\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_S^m\left(i\right)}\underset{t}{\overset{T}{\int }}{h}_i^j({\overline{x}}_i\left(\tau \right);{\overline{x}}_j\left(\tau \right);{\overline{u}}_i\left(\tau \right))d\tau .$$

(8)

3. Properties of the Characteristic Function

In references [10,11], the characteristic function is constructed in a similar way, but it was assumed that $h_i^j=0$ if the players i and j are not connected by an edge. It was shown that such characteristic function is convex. Characteristic function $V(S;x_0,T-t_0)$ is called convex (or supermodular) if for any coalitions $S_1,S_2\subseteq N$ the following condition holds: $V(S_1\cup S_2;x_0,T-t_0)+V(S_1\cap S_2;x_0,T-t_0)\ge V(S_1;x_0,T-t_0)+V(S_2;x_0,T-t_0)$. A game is called convex if its characteristic function is convex. However, in our case, additional restrictions on the network are required for the characteristic function to be convex.

Define functions $W(S;t)$ that can be interpreted as instantaneous values of the characteristic function according to the following rule:

$$W(S;t)=\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_S^m\left(i\right)}{h}_i^j({\overline{x}}_i\left(t\right);{\overline{x}}_j\left(t\right);{\overline{u}}_i\left(t\right)).$$

(9)

Proposition 1.

Let $S_1\subset N$, $S_2\subset S_1$. If there are no cycles in the network $(N,L)$, then the following inequality holds for each $i\in N\backslash S_1$ and each $t\in [t_0,T]$:

$$W(S_1\cup \left\{i\right\};t)-W(S_1;t)\ge W(S_2\cup \left\{i\right\};t)-W(S_2;t).$$

(10)

Proof. 

For simplicity, we denote

$$h_i^j({\overline{x}}_i\left(t\right);{\overline{x}}_j\left(t\right);{\overline{u}}_i\left(t\right))={\overline{h}}_i^j\left(t\right).$$

The absence of cycles in the network (tree or forest) means that there can be only one path between any two vertices. Suppose $i\notin S_1.$ If vertex i lies on the path between some vertices in the coalition $S_1\cup \left\{i\right\}$, then the path between these vertices does not exist in $(S_1,L_{S_1})$. Let $P_S^m\left(i\right)$ be the set of pairs of vertices $\{p,q\}$ such that $p\in S\backslash i$, $q\in S\backslash i$, the distance between them equals m, all the vertices of the path between p and q belong to S, and i lies on this path. Then,

$$\begin{array}{c}W(S_1\cup \left\{i\right\};t)-W(S_1;t)={\overline{h}}_i^i\left(t\right)+\hfill \\ \hfill +\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_{S_1\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_i^j\left(t\right)+{\overline{h}}_j^i\left(t\right))+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{p,q\}\in P_{S_1\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right)),\end{array}$$

(11)

$$\begin{array}{c}W(S_2\cup \left\{i\right\};t)-W(S_2;t)={\overline{h}}_i^i\left(t\right)+\hfill \\ \hfill +\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_{S_2\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_i^j\left(t\right)+{\overline{h}}_j^i\left(t\right))+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{p,q\}\in P_{S_2\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right)).\end{array}$$

(12)

Since $S_2\subset S_1$ and ${\overline{h}}_j^i\left(t\right)\ge 0$, we have

$$\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_{S_1\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_i^j\left(t\right)+{\overline{h}}_j^i\left(t\right))\ge \sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_{S_2\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_i^j\left(t\right)+{\overline{h}}_j^i\left(t\right)),$$

$$\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{p,q\}\in P_{S_1\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right))\ge \sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{p,q\}\in P_{S_2\cup \left\{i\right\}}^m\left(i\right)}({\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right)).$$

It follows that (10) is satisfied. □

Remark 1.

Note that the presence of a cycle in the network can lead to the violation of property (10).

Indeed, the presence of a cycle in the network allows several paths between two vertices. Let us assume that there are two paths with the length r between vertices $p\in S_2$ and $q\in S_2$. Suppose that all vertices from the first path belong to $S_2\cup \left\{i\right\}$, and vertex i lies on this path. Assume also that the second path contains vertices from $S_1\backslash S_2$ and vertex i does not lie on this path.

Note that there exists a path between p and q in $(S_2\cup \left\{i\right\},L_{S_2\cup \left\{i\right\}})$, but there is no path between these vertices in $(S_2,L_{S_2})$, since the first path goes through player i who is no longer in the coalition and the second path does not belong to $(S_2,L_{S_2})$. Then, there is a term ${\delta }^{r-1}({\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right))$ in the right-hand side of (12).

There are two paths with the length r between vertices p and q in $(S_1\cup \left\{i\right\},L_{S_1\cup \left\{i\right\}})$. One of them passes through vertex i and the other does not. Thus, there exists path between p and q in $(S_1,L_{S_1})$. This means that, on the right-hand side of (11), there is no term corresponding to the vertices p and q.

Thus, if ${\overline{h}}_q^p\left(t\right)+{\overline{h}}_p^q\left(t\right)$ is large enough, inequality (10) will not hold.

Corollary 1.

If there are no cycles in the network $(N,L)$, then the characteristic function defined in (7) and (8) is convex.

Proof. 

It is shown that, for each $\tau \in [t_0,T]$, $S_1\subset N$, $S_2\subset S_1$ and each $i\in N\backslash S_1$, the following inequality holds:

$$W(S_1\cup \left\{i\right\};t)-W(S_1;t)\ge W(S_2\cup \left\{i\right\};t)-W(S_2;t).$$

(13)

Integrating both sides of this inequality with respect to t, we have for each $t\in [t_0,T]$, each $S_1\subset N$, $S_2\subset S_1$, and each $i\in N\backslash S_1$

$$V(S_1\cup \left\{i\right\};\overline{x}\left(t\right),T-t)-V(S_1;\overline{x}\left(t\right),T-t)\ge V(S_2\cup \left\{i\right\};\overline{x}\left(t\right),T-t)-V(S_2;\overline{x}\left(t\right),T-t).$$

This means that the characteristic function defined in (7) and (8) is convex (see [17]). □

The absence of cycles guarantees the convexity of the characteristic function. However, even if there are cycles in the network, the characteristic function defined according to a given rule (7) and (8) may be convex for some parameter values. Therefore, the existing formula (7) for the characteristic function is relevant not only for networks without cycles.

4. Strong Time-Consistent Subcore

The next problem to define a cooperative solution is to determine an allocation rule to distribute among the players their maximum joint payoff. We again assume that there are no cycles in the network $(N,L)$.

The set of all imputations in the game $\Gamma (x_0,T-t_0)$, $L(x_0,T-t_0)$, is given by

$$\begin{array}{c}L(x_0,T-t_0)=\{\xi (x_0,T-t_0)=({\xi }_1(x_0,T-t_0),\dots ,{\xi }_n(x_0,T-t_0)):\hfill \\ \hfill \sum _{i\in N}{\xi }_i(x_0,T-t_0)=V(N;x_0,T-t_0),{\xi }_i(x_0,T-t_0)\ge V(\left\{i\right\};x_0,T-t_0),i\in N\}.\end{array}$$

(14)

Definition 1

([17]). The core $C(x_0,T-t_0)$ of the game $\Gamma (x_0,T-t_0)$ is the subset of the imputation set $L(x_0,T-t_0)$, such that

$$\begin{array}{c}\hfill C(x_0,T-t_0)=\{\xi (x_0,T-t_0)\in L(x_0,T-t_0):\sum _{i\in S}{\xi }_i(x_0,T-t_0)\ge V(S;x_0,T-t_0),S\subset N\}.\end{array}$$

(15)

Similarly, for every $t\in [t_0,T]$ denote by $L(\overline{x}\left(t\right),T-t)$ the set of all imputations and by $C(\overline{x}\left(t\right),$$T-t)$ the core in the subgame $\Gamma (\overline{x}\left(t\right),T-t)$ along the cooperative trajectory.

The convexity of the characteristic function (7) and (8), which is proved above, guarantees that the core is not empty for every $t\in [t_0,T]$ [17].

The time consistency of cooperative solutions is an important issue when considering dynamic games. Suppose the optimality principle chosen by the players contains a set of imputations. In that case, as the game evolves along the cooperative trajectory $\overline{x}\left(t\right)$, the players can deviate from the imputation chosen in the initial time for any other imputation in that optimality principle. Will it lead to the payment rule selected in the game corresponding to the initial optimality principle? This is true only if the cooperative solution is strong time-consistent. This property of cooperation solution was introduced by L. Petrosyan [13]. Let us show that there is a subset of the core in the class of games under consideration, which is strong time-consistent.

Definition 2

(see [18]). A function $\beta \left(t\right)=({\beta }_1\left(t\right),\dots ,{\beta }_n\left(t\right))$, $t\in [t_0,T]$ is the imputation distribution procedure (IDP) for imputation $\alpha \in L(x_0,T-t_0)$, if

$${\alpha }_i=\underset{t_0}{\overset{T}{\int }}{\beta }_i\left(\tau \right)d\tau ,i\in N.$$

Definition 3

(see [13]). An optimality principal $M(x_0,T-t_0)\subset L(x_0,T-t_0)$ is called strong time-consistent if

1. 

$M(\overline{x}\left(t\right),T-t)\ne \varnothing ,\forall t\in [t_0,T]$.

2. 

For each $\alpha \in M(x_0,T-t_0)$ there exists an IDP $\beta \left(\tau \right)=({\beta }_1\left(\tau \right),\dots ,{\beta }_n\left(\tau \right))$, $\tau \in [t_0,T]$, such that $\alpha =\underset{t_0}{\overset{T}{\int }}{\beta }_i\left(\tau \right)d\tau$, $i\in N$ and

$$M(x_0,T-t_0)\supset \underset{t_0}{\overset{t}{\int }}\beta \left(\tau \right)d\tau \oplus M(\overline{x}\left(t\right),T-t),\forall t\in [t_0,T].$$

(16)

For $a\in R^n$, $B\subset R^n$, the symbol ⊕ means the following: $a\oplus B=\{a+b:b\in B\}$.

In [19], a method for constructing a strong time-consistent subset of the core is proposed. Furthermore, conditions under which such a subset exists are obtained. In this paper, we implement this approach for the class of games under consideration, taking into account the peculiarities of the definition of the characteristic function.

Using the functions $W(S;t)$ introduced in (9), we define $B\left(t\right)$ for each $t\in [t_0,T]$ as the set of functions $\beta \left(t\right)=({\beta }_1\left(t\right),\dots ,{\beta }_n\left(t\right))$ such that

$$\begin{array}{c}{\displaystyle \sum _{i\in S}}{\beta }_i\left(t\right)\ge W(S;t)\forall S\subset N,\hfill \\ {\displaystyle \sum _{i\in N}}{\beta }_i\left(t\right)=W(N;t).\hfill \end{array}$$

(17)

Note that, if we consider $W(S;t):2^N\to R$ as a characteristic function of the stationary game, then $\beta \left(t\right)\in B\left(t\right)$ is an imputation in this game and $B\left(t\right)$ coincides with its core. Taking into account the fulfillment of property (10), we can conclude that, for each $t\in [t_0,T]$, the set $B\left(t\right)$ is not empty in the considered class of games since the characteristic function $W(S;t)$ in each moment t is convex [17].

To prove that $\beta \left(t\right)\in B\left(t\right)$ is an IDP in the initial differential game $\Gamma (x_0,T-t_0)$, let us consider $\overline{\beta }\left(t\right)\in B\left(t\right)$. Now, we define ${\overline{\xi }}_i(x_0,T-t_0)$ in the following way

$${\overline{\xi }}_i(x_0,T-t_0)=\underset{t_0}{\overset{T}{\int }}{\overline{\beta }}_i\left(\tau \right)d\tau ,i\in N.$$

(18)

According to (17),

$$\begin{array}{c}{\overline{\beta }}_i\left(t\right)\ge W(\left\{i\right\};t),\hfill \\ {\displaystyle \sum _{i\in N}}{\overline{\beta }}_i\left(t\right)=W(N;t).\hfill \end{array}$$

(19)

Then,

$${\overline{\xi }}_i(x_0,T-t_0)=\underset{t_0}{\overset{T}{\int }}{\overline{\beta }}_i\left(t\right)dt\ge \underset{t_0}{\overset{T}{\int }}W(\left\{i\right\};t)dt=V(\left\{i\right\};x_0,T-t_0),$$

and

$$\sum _{i\in N}{\overline{\xi }}_i(x_0,T-t_0)=\sum _{i\in N}\underset{t_0}{\overset{T}{\int }}{\overline{\beta }}_i\left(t\right)dt=\underset{t_0}{\overset{T}{\int }}W(N;t)dt=V(\left\{N\right\};x_0,T-t_0),$$

which proves that $\overline{\xi }(x_0,T-t_0)\in L(x_0,T-t_0)$. $\overline{\beta }\left(t\right)$ is an IDP for this imputation. Similarly, it can be proved that, for every $t\in [t_0,T]$ vector ${\overline{\xi }}_i(\overline{x}\left(t\right),T-t)$, defined by

$${\overline{\xi }}_i(\overline{x}\left(t\right),T-t)=\underset{t}{\overset{T}{\int }}{\overline{\beta }}_i\left(\tau \right)d\tau ,i\in N,$$

(20)

is an imputation in $\Gamma (\overline{x}\left(t\right),T-t)$.

Denote by $\overline{C}(\overline{x}\left(t\right),T-t)$ the set of all imputations $\overline{\xi }(\overline{x}\left(t\right),T-t)$, specified by Equation (20) for all $\overline{\beta }\left(t\right)\in B\left(t\right)$.

Proposition 2.

In the class of games under consideration, the sets $\overline{C}(\overline{x}\left(t\right),T-t)$ are non-empty for every $t\in [t_0,T]$ and $\overline{C}(\overline{x}\left(t\right),T-t)\subset C(\overline{x}\left(t\right),T-t)$.

Proof. 

The sets $B\left(t\right)$ are non-empty for each $t\in [t_0,T]$ because functions $W(S;t)$ are convex for each $t\in [t_0,T]$. Then, $\overline{C}(\overline{x}\left(t\right),T-t)$ are non-empty for each $t\in [t_0,T]$.

Now, we prove that $\overline{C}(\overline{x}\left(t\right),T-t)\subset C(\overline{x}\left(t\right),T-t)$.

Let $\overline{\xi }(\overline{x}\left(t\right),T-t)\in \overline{C}(\overline{x}\left(t\right),T-t)$. Then, for $\overline{\xi }(\overline{x}\left(t\right),T-t)$, there exists an IDP $\overline{\beta }\left(t\right)\in \overline{B}\left(t\right)$, such that

$$\begin{array}{c}{\displaystyle \sum _{i\in S}}{\overline{\beta }}_i\left(t\right)\ge W(S;t)\forall S\subset N,\hfill \\ {\displaystyle \sum _{i\in N}}{\overline{\beta }}_i\left(t\right)=W(N;t).\hfill \end{array}$$

(21)

Then,

$$\sum _{i\in S}{\overline{\xi }}_i(\overline{x}\left(t\right),T-t)=\underset{t}{\overset{T}{\int }}\sum _{i\in S}{\overline{\beta }}_i\left(\tau \right)d\tau \ge \underset{t}{\overset{T}{\int }}W(S;\tau )d\tau =V(S;\overline{x}\left(t\right),T-t)i\in N.$$

(22)

From (22), it follows that the imputation $\overline{\xi }(\overline{x}\left(t\right),T-t)$ belongs to the core $C(\overline{x}\left(t\right),T-t)$ for each $t\in [t_0,T]$. This means that $\overline{C}(\overline{x}\left(t\right),T-t)\subset C(\overline{x}\left(t\right),T-t)$ for each $t\in [t_0,T]$. □

Proposition 3.

$\overline{C}(\overline{x}\left(t_0\right),T-t_0)$ is a strong time-consistent optimality principle in the game $\Gamma (x_0,T-t_0)$.

Proof. 

The proof follows directly from Theorem 5.1 in [19]. □

Consider also a rule for constructing some IDP from the set $B\left(t\right)$.

Denote by $D^m\left(i\right)$ the set of pairs $\{k,l\}$, where $k\in N$, $l\in N$, the distance between k and l equals m, and vertex i belongs to the path between k and l (i can coincide with k or l). Let $D_S^m\left(i\right)$ be the set of pairs $\{k,l\}$, where $k\in S$, $l\in S$, with the distance between k and l equal to m, all vertices from this path belonging to S, and vertex i lying on the path between k and l.

For each pair of vertices $\{p,q\}$, $p,q\in N$, we denote by ${\Phi }_{p,q}$ the set of vectors ${\varphi }_{p,q}=({\varphi }_{p,q}^{{\gamma }_1},\dots ,{\varphi }_{p,q}^{{\gamma }_{m+1}}$), such that ${\displaystyle \sum _{j=1}^{m+1}}{\varphi }_{p,q}^{{\gamma }_j}=1$, $0\le {\varphi }_{p,q}^{{\gamma }_j}\le 1$, for each $j=\overline{1,m+1}$. Here, we assume the length of the path between p and q equals m (for each vertex ${\gamma }_j$, $j=\overline{1,m+1}$, belonging to the path between p and q, the coefficient ${\varphi }_{p,q}^{{\gamma }_j}$ is given).

Proposition 4.

In the class of games under consideration, the IDP defined by the Equation (23) belongs to the set $B\left(t\right)$ for any ${\varphi }_{p,q}\in {\Phi }_{p,q}$ for each pair of vertices $p\in N$, $q\in N$

$${\overline{\beta }}_i\left(t\right)={\overline{h}}_i^i\left(t\right)+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right)),i\in N.$$

(23)

Proof. 

We prove that the vector defined by Equation (23) satisfies conditions (17). Indeed,

$$\begin{array}{c}\sum _{i=1}^n{\overline{\beta }}_i\left(t\right)=\sum _{i=1}^n{\overline{h}}_i^i\left(t\right)+\sum _{i=1}^n\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\hfill \\ =\sum _{i=1}^n{\overline{h}}_i^i\left(t\right)+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in {\displaystyle \bigcup _{i\in N}}{D}^m\left(i\right)}({\varphi }_{k,l}^{{\gamma }_1}+\dots {\varphi }_{k,l}^{{\gamma }_{m+1}})({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\\ \hfill =\sum _{i=1}^n{\overline{h}}_i^i\left(t\right)+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in {\displaystyle \bigcup _{i\in N}}{D}^m\left(i\right)}({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))=W(N,t).\end{array}$$

(24)

$$\begin{array}{c}\sum _{i\in S}{\overline{\beta }}_i\left(t\right)=\sum _{i\in S}{\overline{h}}_i^i\left(t\right)+\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\hfill \\ =\sum _{i\in S}{\overline{h}}_i^i\left(t\right)+\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D_S^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\\ +\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)\backslash D_S^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\\ =\sum _{i\in S}{\overline{h}}_i^i\left(t\right)+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in {\displaystyle \bigcup _{i\in S}}{D}_S^m\left(i\right)}({\varphi }_{k,l}^{{\gamma }_1}+\dots +{\varphi }_{k,l}^{{\gamma }_{m+1}})({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\\ +\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)\backslash D_S^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\\ \hfill =W(S,t)+\sum _{i\in S}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D^m\left(i\right)\backslash D_S^m\left(i\right)}{\varphi }_{k,l}^i({\overline{h}}_l^k\left(t\right)+{\overline{h}}_k^l\left(t\right))\ge W(S,t).\end{array}$$

(25)

This concludes the proof. □

Denote the set of all IDP constructed with Equation (23) as $B_{\varphi }\left(t\right)$. Proposition 4 shows that $B_{\varphi }\left(t\right)\subset B\left(t\right)$ for each $t\in [t_0,T]$. Formula (23) allows finding IDP in explicit form.

5. The Shapley Value

Consider also the problem of constructing the Shapley value (see [20]) in differential games on networks. It turned out that using the characteristic function (7) in the considered class of games, the construction of the Shapley value does not require the calculation of the characteristic function of every $S\subset N$. Proposition 5 demonstrates this specific result.

Proposition 5.

If the characteristic function in the cooperative differential game on network (1) and (2) is defined under the rule (7) and the network $(N,L)$ has no cycles, then the Shapley value in this game has the following form

$$\begin{array}{c}S{h}_i(x_0,T-t_0)=V(\left\{i\right\};x_0,T-t_0)+\hfill \\ \hfill +\sum _{m=1}^{n-1}\frac{{\delta }^{m-1}}{m+1}\sum _{\{k,l\}\in D^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}(h_l^k({\overline{x}}_l\left(\tau \right);{\overline{x}}_k\left(\tau \right);{\overline{u}}_l\left(\tau \right))+h_k^l({\overline{x}}_k\left(\tau \right);{\overline{x}}_l\left(\tau \right);{\overline{u}}_k\left(\tau \right)))d\tau .\end{array}$$

(26)

Proof. 

The formula for the Shapley value [20] is as follows

$$S{h}_i(x_0,T-t_0)=\sum _{S:i\in S}\frac{(s-1)!(n-s)!}{n!}(V(S;x_0,T-t_0)-V(S\backslash i;x_0,T-t_0)).$$

Consider the term $V(S;x_0,T-t_0)-V(S\backslash i;x_0,T-t_0)$. This is the value that the coalition S loses when a player i leaves it. This value consists of

• $V(\left\{i\right\};x_0,T-t_0)=\underset{t_0}{\overset{T}{\int }}{h}_i^i({\overline{x}}_i\left(\tau \right);{\overline{x}}_i\left(\tau \right);{\overline{u}}_i\left(\tau \right))d\tau$ is the payoff of player i that he can obtain by itself.
• ${\displaystyle \sum _{m=1}^{n-1}}{\delta }^{m-1}{\displaystyle \sum _{j\in K_S^m\left(i\right)}}\underset{t_0}{\overset{T}{\int }}{h}_i^j({\overline{x}}_i\left(\tau \right);{\overline{x}}_j\left(\tau \right);{\overline{u}}_i\left(\tau \right))d\tau$ is the payoff that player i receives from interaction with players from $S\backslash i$.
• ${\displaystyle \sum _{m=1}^{n-1}}{\delta }^{m-1}{\displaystyle \sum _{j\in K_S^m\left(i\right)}}\underset{t_0}{\overset{T}{\int }}{h}_j^i({\overline{x}}_j\left(\tau \right);{\overline{x}}_i\left(\tau \right);{\overline{u}}_j\left(\tau \right))d\tau$ is the payoff that players from $S\backslash i$ receive from interaction with player i.
• ${\displaystyle \sum _{m=1}^{n-1}}{\delta }^{m-1}{\displaystyle \sum _{\{p,q\}}}\underset{t_0}{\overset{T}{\int }}(h_q^p({\overline{x}}_q\left(\tau \right);{\overline{x}}_p\left(\tau \right);{\overline{u}}_q\left(\tau \right))+h_p^q({\overline{x}}_p\left(\tau \right);{\overline{x}}_q\left(\tau \right);{\overline{u}}_p\left(\tau \right)))d\tau$ are the payoffs that players p and q receive from interaction with each other, where p and q are such vertices from $S\backslash i$ that there exists a path with the length m between them in S, and player i lies on this path (since there is no path in $S\backslash i$ between p and q, because player i leaves the set S).

Rewrite the values from the last three items into one summand using the definition of sets $D_S^m\left(i\right)$:

$$\begin{array}{c}\hfill S{h}_i=V(\left\{i\right\};x_0,T-t_0)+\sum _{S:i\in S}\frac{(s-1)!(n-s)!}{n!}\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{\{k,l\}\in D_S^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}({\overline{h}}_l^k\left(\tau \right)+{\overline{h}}_k^l\left(\tau \right))d\tau .\end{array}$$

(27)

Consider two fixed players k and l with the length of the path between them equal to m. These two vertices (players), together with all vertices (players) from the path between them, are included in $C_{n-m-1}^{s-m-1}$ coalitions of power s. Then,

$$\begin{array}{c}S{h}_i=V(\left\{i\right\};x_0,T-t_0)+\hfill \\ \hfill +\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{s\ge m+1}\frac{(s-1)!(n-s)!}{n!}{C}_{n-m-1}^{s-m-1}\sum _{\{k,l\}\in D^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}({\overline{h}}_l^k\left(\tau \right)+{\overline{h}}_k^l\left(\tau \right))d\tau ,\end{array}$$

(28)

where ${\displaystyle \sum _{s\ge m+1}}\frac{(s-1)!(n-s)!}{n!}{C}_{n-m-1}^{s-m-1}=\frac{1}{m+1}.$

Then,

$$\begin{array}{c}S{h}_i(x_0,T-t_0)=V(\left\{i\right\};x_0,T-t_0)+\hfill \\ \hfill +\sum _{m=1}^{n-1}\frac{{\delta }^{m-1}}{m+1}\sum _{\{k,l\}\in D^m\left(i\right)}\underset{t_0}{\overset{T}{\int }}(h_l^k({\overline{x}}_l\left(\tau \right);{\overline{x}}_k\left(\tau \right);{\overline{u}}_l\left(\tau \right))+h_k^l({\overline{x}}_k\left(\tau \right);{\overline{x}}_l\left(\tau \right);{\overline{u}}_k\left(\tau \right)))d\tau .\end{array}$$

(29)

This concludes the proof. □

Remark 2.

$Sh(x_0,T-t_0)$ belongs to the strong time-consistent subcore. This can be verified by choosing an IDP for it, constructed according to (23) with all coefficients ${\varphi }_{k,l}^i$ equal to $\frac{1}{m+1}$.

6. One Cooperative Differential Network Marketing Game

As an illustrative example, we consider a simple model of network marketing. There are n distributors (players) selling the same product or brand. Let $x_i$ denote the sales rate of player i. The sales rate dynamics of player i evolves according to the accumulation equation

$${\dot{x}}_i\left(t\right)=u_i\left(t\right),x_i\left(t_0\right)=x_i^0,$$

(30)

where $u_i\left(t\right)\ge 0$ is the promotional effort of distributor i, $x_i^0\ge 0$.

We assume that each player’s payoff depends on their own sales and those of other distributors. First, we consider the general case where a player’s payoff depends on all participants’ sales, taking into account the distance between players on the network. Then, we apply the obtained solution to a multi-level marketing model on a directed network. The directions of arcs are supposed to indicate the hierarchy between players. In this case, distributors can only benefit from down-line distributors.

If each agent extracts linear utility from his own sales and sales of other participants and the cost function has a quadratic form, then the objective functions are given by

$$H_i(x^0,u_1,\dots ,u_n)=\underset{t_0}{\overset{T}{\int }}(a_i{x}_i\left(\tau \right)-c_i{\left(u_i\left(\tau \right)\right)}^2+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}{b}^{i\left(j\right)}{x}_j\left(\tau \right))d\tau .$$

(31)

Here, $a_i{x}_i\left(\tau \right)$ is the instantaneous gain that player i can obtain by himself, ${\delta }^{m-1}{b}^{i\left(j\right)}{x}_j\left(\tau \right)$ is the instantaneous gain that player i obtains through interaction with player $j\in K^m\left(i\right)$, and $c_i{\left(u_i\left(\tau \right)\right)}^2$ is a cost function of agent i. Parameter $b^{i\left(j\right)}$ indicates the percentage of sales of player j that player i receives.

Suppose that the game is played in a cooperative scenario and players have the opportunity to cooperate in order to achieve maximum total payoff:

$$\underset{u_1,\dots ,u_n}{max}\sum _{i\in N}\left(\underset{t_0}{\overset{T}{\int }}\right(a_i{x}_i\left(\tau \right)-c_i{\left(u_i\left(\tau \right)\right)}^2+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}{b}^{i\left(j\right)}{x}_j\left(\tau \right)\left)d\tau \right).$$

(32)

To define the cooperative strategies, we use the Bellman dynamic programming technique. We denote by $V(N,x,T-t)$ the Bellman function in a subgame starting at the moment t from $x\left(t\right)$:

$$V(N,x,T-t)=\underset{u_1,\dots ,u_n}{max}\sum _{i\in N}\left(\underset{t}{\overset{T}{\int }}\right(a_i{x}_i\left(\tau \right)-c_i{\left(u_i\left(\tau \right)\right)}^2+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}{b}^{i\left(j\right)}{x}_j\left(\tau \right)\left)d\tau \right).$$

(33)

The Hamilton–Jacobi–Bellman (HJB) equation has the following expression:

$$-V_t(N,x,T-t)=\underset{u_1,\dots ,u_n}{max}\left[\sum _{i\in N}\right(a_i{x}_i\left(t\right)-c_i{\left(u_i\left(t\right)\right)}^2+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K^m\left(i\right)}{b}^{i\left(j\right)}{x}_j\left(t\right))+\sum _{i\in N}{V}_{x_i}(N,x,T-t)u_i\left(t\right)],$$

(34)

$$V(N,x(T),0)=0.$$

The solution of (34) is found in the form

$$V(N,x,T-t)=\sum _{i\in N}{A}_i\left(t\right)x_i+B\left(t\right).$$

(35)

The maximization problem in (34) yields a strategy for player i:

$$u_i=\frac{V_{x_i}(N,x,T-t)}{2c_i}=\frac{A_i\left(t\right)}{2c_i},i=\overline{1,n}.$$

(36)

Substituting it into (34), we have the following system of differential equations for $A_i\left(t\right)$ and B(t):

$$\begin{array}{c}{\dot{A}}_i\left(t\right)=-a_i-{\displaystyle \sum _{m=1}^{n-1}}{\delta }^{m-1}{\displaystyle \sum _{j\in K^m\left(i\right)}}{b}^{j\left(i\right)},A_i\left(T\right)=0.\hfill \\ \dot{B}\left(t\right)=-{\displaystyle \sum _{i\in N}}\frac{A_i^2\left(t\right)}{4c_i},B\left(T\right)=0.\hfill \end{array}$$

(37)

The solution of (37) is the following

$$\begin{array}{c}{A}_i\left(t\right)=(T-t)(a_i+b_i),i\in N,\hfill \\ B\left(t\right)={(T-t)}^3{\displaystyle \sum _{i\in N}}\frac{{(a_i+b_i)}^3}{12c_i},\hfill \end{array}$$

(38)

where $b_i={\displaystyle \sum _{m=1}^{n-1}}{\delta }^{m-1}{\displaystyle \sum _{j\in K^m\left(i\right)}}{b}^{j\left(i\right)}$.

Then, the optimal cooperative strategies have the form

$${\overline{u}}_i\left(t\right)=\frac{(T-t)(a_i+b_i)}{2c_i},i=\overline{1,n}.$$

(39)

The optimal cooperative trajectory is

$${\overline{x}}_i\left(t\right)=x_i^0+\frac{a_i+b_i}{4c_i}(2T-t-t_0)(t-t_0).$$

(40)

The value function for the cooperative joint payoff of all n players can be obtained as

$$V(N,x^0,T-t_0)=\sum _{i\in N}((T-t_0)(a_i+b_i)x_i^0+{(T-t_0)}^3\frac{{(a_i+b_i)}^3}{12c_i}).$$

(41)

According to (7), the characteristic function becomes

$$\begin{array}{c}V(S,x^0,T-t_0)=\sum _{i\in S}(a_i{x}_i^0(T-t_0)+\frac{a_i{(T-t_0)}^3(a_i+b_i)}{6c_i}-\hfill \\ \hfill -\frac{{(T-t_0)}^3{(a_i+b_i)}^2}{12c_i}+\sum _{m=1}^{n-1}{\delta }^{m-1}\sum _{j\in K_S^m\left(i\right)}{b}^{i\left(j\right)}(x_j^0(T-t_0)+\frac{{(T-t_0)}^3(a_j+b_j)}{6c_j})).\end{array}$$

(42)

Consider now the numeric example for the case of multi-level marketing game. This model assumes that the graph is directed. The direction of edge from player i to player j means that player i has recruited player j.

Figure 1 shows the structure of the network in the game. It is assumed that distributors can only benefit from down-line distributors. In the framework of the considered example of network marketing, this means that $b^{2\left(1\right)}=b^{3\left(1\right)}=b^{3\left(2\right)}=b^{4\left(1\right)}=b^{4\left(2\right)}=b^{4\left(3\right)}=b^{3\left(4\right)}=0$.

We assume the following values of the remaining parameters: $x_1^0=2$, $x_2^0=1$, $x_3^0=0$, $x_4^0=0$, $a_1=\frac{1}{2}$, $a_2=\frac{1}{3}$, $a_3=a_4=\frac{1}{4}$, $c_1=2$, $c_2=1$, $c_3=3$, $c_4=2$, $b^{1\left(2\right)}=b^{1\left(3\right)}=b^{1\left(4\right)}=b^{2\left(3\right)}=b^{2\left(4\right)}=\frac{1}{8}$, $T=10$, $t_0=0$, $\delta =\frac{1}{2}$.

According to Proposition 5, the Shapley value has the form

$$\begin{array}{c}S{h}_1=V(\left\{1\right\};x^0,T-t_0)+\frac{1}{2}\underset{t_0}{\overset{T}{\int }}{b}^{1\left(2\right)}{\overline{x}}_2\left(\tau \right)d\tau +\frac{1}{3}\delta \underset{t_0}{\overset{T}{\int }}(b^{1\left(3\right)}{\overline{x}}_3\left(\tau \right)+b^{1\left(4\right)}{\overline{x}}_4\left(\tau \right))d\tau ,\hfill \\ S{h}_2=V(\left\{2\right\};x_0,T-t_0)+\frac{1}{2}\underset{t_0}{\overset{T}{\int }}(b^{1\left(2\right)}{\overline{x}}_2\left(\tau \right)+b^{2\left(3\right)}{\overline{x}}_3\left(\tau \right)+b^{2\left(4\right)}{\overline{x}}_4\left(\tau \right))d\tau +\hfill \\ +\frac{1}{3}\delta \underset{t_0}{\overset{T}{\int }}(b^{1\left(3\right)}{\overline{x}}_3\left(\tau \right)+b^{1\left(4\right)}{\overline{x}}_4\left(\tau \right))d\tau ,\hfill \\ S{h}_3=V(\left\{3\right\};x_0,T-t_0)+\frac{1}{2}\underset{t_0}{\overset{T}{\int }}{b}^{2\left(3\right)}{\overline{x}}_3\left(\tau \right)d\tau +\frac{1}{3}\delta \underset{t_0}{\overset{T}{\int }}{b}^{1\left(3\right)}{\overline{x}}_3\left(\tau \right)d\tau ,\hfill \\ S{h}_4=V(\left\{4\right\};x_0,T-t_0)+\frac{1}{2}\underset{t_0}{\overset{T}{\int }}{b}^{2\left(4\right)}{\overline{x}}_4\left(\tau \right)d\tau +\frac{1}{3}\delta \underset{t_0}{\overset{T}{\int }}{b}^{1\left(4\right)}{\overline{x}}_4\left(\tau \right)d\tau .\hfill \end{array}$$

(43)

Thus, to obtain the Shapley value, it is not necessary to calculate all the values of the characteristic function; it is enough to find the following quantities:

$$\begin{array}{c}V(N,x^0,T-t_0)=55.798,\\ V(\left\{1\right\},x^0,T-t_0)=20.417,\\ V(\left\{2\right\},x^0,T-t_0)=11.291,\\ V(\left\{3\right\},x^0,T-t_0)=0.76,\\ V(\left\{4\right\},x^0,T-t_0)=1.139,\\ \underset{t_0}{\overset{T}{\int }}{b}^{1\left(2\right)}{\overline{x}}_2\left(\tau \right)\left(\tau \right)d\tau =10.799,\\ \underset{t_0}{\overset{T}{\int }}{b}^{1\left(3\right)}{\overline{x}}_3\left(\tau \right)d\tau =3.038,\\ \underset{t_0}{\overset{T}{\int }}{b}^{1\left(4\right)}{\overline{x}}_4\left(\tau \right)d\tau =4.557,\\ \underset{t_0}{\overset{T}{\int }}{b}^{2\left(3\right)}{\overline{x}}_3\left(\tau \right)d\tau =3.038,\\ \underset{t_0}{\overset{T}{\int }}{b}^{2\left(4\right)}{\overline{x}}_4\left(\tau \right)d\tau =4.557.\end{array}$$

Then,

$$Sh(x^0,T-t_0)=(27.082;21.753;2.785;4.178).$$

To construct IDP from $B_{\varphi }\left(t\right)$, we use a set of coefficients that satisfy the system:

$$\begin{array}{c}{\varphi }_{1,3}^1+{\varphi }_{1,3}^2+{\varphi }_{1,3}^3=1,\hfill \\ {\varphi }_{1,4}^1+{\varphi }_{1,4}^2+{\varphi }_{1,4}^4=1,\hfill \\ {\varphi }_{1,2}^1+{\varphi }_{1,2}^2=1,\hfill \\ {\varphi }_{2,3}^2+{\varphi }_{2,3}^3=1,\hfill \\ {\varphi }_{2,4}^2+{\varphi }_{2,4}^4=1,\hfill \\ {\varphi }_{i,j}^k\ge 0.\hfill \end{array}$$

(44)

Then, the formulas for IDP have the form:

$$\begin{array}{c}{\beta }_1\left(t\right)=a_1{\overline{x}}_1\left(t\right)-c_1{\left({\overline{u}}_1\left(t\right)\right)}^2+{\varphi }_{1,2}^1{b}^{1\left(2\right)}{\overline{x}}_2\left(t\right)+\delta ({\varphi }_{1,3}^1{b}^{1\left(3\right)}{\overline{x}}_3\left(t\right)+{\varphi }_{1,4}^1{b}^{1\left(4\right)}{\overline{x}}_4\left(t\right)),\hfill \\ {\beta }_2\left(t\right)=a_2{\overline{x}}_2\left(t\right)-c_2{\left({\overline{u}}_2\left(t\right)\right)}^2+{\varphi }_{1,2}^2{b}^{1\left(2\right)}{\overline{x}}_2\left(t\right)+{\varphi }_{2,3}^2{b}^{2\left(3\right)}{\overline{x}}_3\left(t\right)+{\varphi }_{2,4}^2{b}^{2\left(4\right)}{\overline{x}}_4\left(t\right)+\hfill \\ +\delta ({\varphi }_{1,3}^2{b}^{1\left(3\right)}{\overline{x}}_3\left(t\right)+{\varphi }_{1,4}^2{b}^{1\left(4\right)}{\overline{x}}_4\left(t\right)),\hfill \\ {\beta }_3\left(t\right)=a_3{\overline{x}}_3\left(t\right)-c_3{\left({\overline{u}}_3\left(t\right)\right)}^2+{\varphi }_{2,3}^3{b}^{2\left(3\right)}{\overline{x}}_3\left(t\right)+\delta {\varphi }_{1,3}^3{b}^{1\left(3\right)}{\overline{x}}_3\left(t\right),\hfill \\ {\beta }_4\left(t\right)=a_4{\overline{x}}_4\left(t\right)-c_4{\left({\overline{u}}_4\left(t\right)\right)}^2+{\varphi }_{2,4}^4{b}^{2\left(4\right)}{\overline{x}}_4\left(t\right)+\delta {\varphi }_{1,4}^4{b}^{1\left(4\right)}{\overline{x}}_4\left(t\right).\hfill \end{array}$$

(45)

Choosing ${\varphi }_{1,2}^1={\varphi }_{1,2}^2={\varphi }_{2,3}^2={\varphi }_{2,3}^3={\varphi }_{2,4}^2={\varphi }_{2,4}^4=\frac{1}{2}$, ${\varphi }_{1,3}^1={\varphi }_{1,3}^2={\varphi }_{1,3}^3={\varphi }_{1,4}^1={\varphi }_{1,4}^2={\varphi }_{1,4}^4=\frac{1}{3}$ we obtain the IDP for the Shapley Value.

Finally, we show the graphical representation of the obtained solutions. Expressing ${\xi }_4=V(N,x^0,T-t_0)-{\xi }_1-{\xi }_2-{\xi }_3$ and passing to the system of inequalities with three variables (15) to find the core, we plot the resulting area in Figure 2. The Shapley value (red color) and imputations from the strong time-consistent subcore, calculated according to (23) (blue color), are also indicated in Figure 2.

The example considered in Section 6 illustrates a possible application of the proposed model and shows how to distribute the joint payoff between the players connected by the network. This example also shows the possibility of applying the obtained solutions to the case of a directed graph.

7. Conclusions

One class of cooperative differential games on networks is investigated. It is assumed that each player’s payoff depends not only on his neighbors’ actions but also on players’ actions with whom paths in the network connect the player. A novel form for characteristic function in such games is proposed. It is shown that the convexity of this characteristic function not always takes place and essentially depends on the structure of the network on which the game is defined. It is proved that, if the network has no cycles, then the characteristic function is convex and the strong time-consistent subcore is not empty. The formula for explicit calculation of the Shapley value is derived. An algorithm for the construction of IDP, corresponding to imputations from the strong time-consistent subcore is given. An illustrative example demonstrating this result is considered.

Further research can be done on cooperative differential games on networks. It will be interesting to use the approach based on cooperative trajectory characteristic functions to differential games defined on general networks containing cycles. It seems that, in this case, the convexity of the corresponding characteristic function will not take place, but the existence of the core could be proved, and the dynamic Shapley value could be calculated in explicit form. It would also be interesting to obtain such values of $\delta$ for which the characteristic function turns out to be convex even in the presence of cycles in the network.