Time-Consistency of an Imputation in a Cooperative Hybrid Differential Game

Abstract

This work is aimed at studying the problem of maintaining the sustainability of a cooperative solution in an n-person hybrid differential game. Specifically, we consider a differential game whose payoff function is discounted with a discounting function that changes its structure with time. We solve the problem of time-inconsistency of the cooperative solution using a so-called imputation distribution procedure, which was adjusted for this general class of differential games. The obtained results are illustrated with a specific example of a differential game with random duration and a hybrid cumulative distribution function (CDF). We completely solved the presented example to demonstrate the application of the developed scheme in detail. All results were obtained in analytical form and illustrated by numerical simulations.

1. Introduction

This contribution aims at bringing together two different concepts: the notion of sustainable cooperation from game theory and the notion of hybrid optimal control from the theory of hybrid systems. We first give a brief overview of the related results.

The concept of sustainable cooperation constitutes one of the central ingredients of cooperative game theory. Indeed, in many works it has been shown that a cooperative agreement may turn out to be unstable in the sense that the players may decide to break up the agreement at some intermediate time instant. To overcome this problem, Petrosyan, in [1], introduced the imputation distribution procedure (IDP) that has proven to be a very useful tool in the field of cooperative games. In the original paper [1], a fixed and finite-duration differential game was considered. Later, in [2], the notion of the IDP was extended to the class of differential games with infinite duration and a discounting function of a rather general form. Since then, there have been a number of papers devoted to the analysis of optimal problems with different types of discounting functions and their extension to the class of differential games (see, e.g., [3,4,5], where this problem was considered in both deterministic and stochastic settings, and [6] for the very recent results.)

Although the class of hybrid control systems was introduced more than 20 years ago, most results on hybrid control were formulated within the control-related framework and did not address the game-theoretic problems. We will be mostly interested in the hybrid optimal control, the direction that was actively developed during the first decade of the 21st century. We mention the works [7,8] for an overview of the main results on the hybrid optimal control. Some more examples of optimal control and—to some extent—differential games with regime switching can be found in [9,10,11,12,13]. It was only recently that the theory of hybrid control was formally extended to game-theoretic problems. The first attempt was made in [14]; later, in [15], a special but rather general class of hybrid differential games was considered in detail.

In this paper, we consider the problem of the sustainability of a cooperative solution in an n-person hybrid differential game. Earlier, a differential game with a random time horizon and discontinuous distribution was studied in [16], where only one jump was considered, and in [17], where the method of parametrization for calculation of the optimal controls was used. In this work we extend and generalize the results of [17] and put them in the context of hybrid optimal control.

To solve the formulated hybrid optimal control problem we use the method of parametrization, which is a well-known approach for numerical solution of optimal control problems, see, e.g., [18], but is relatively rarely used in the context of differential games. See [15] for some suggestions on how to apply this method to differential games with a switching structure. In short, the whole optimal control problem is decomposed into a number of subproblems, whose initial or final states are parametrized by some variables. These problems are solved backwards by applying the Pontryagin maximum principle [19] to each interval. If the respective optimal control problems admit analytical solutions, these solutions can be further used to determine the optimal values of switching states.

The described approach was successfully applied to a particular differential game with random duration and a composite cumulative distribution function. We computed the optimal controls and cooperative solutions, and determined the imputation distribution procedure.

This paper is organized in the following way. In Section 2 we present the formulation of the problem and formally state all necessary results. In particular, we present a uniform description of the hybrid discounting function using the notion of the hybrid hazard rate and give an explicit formula for computing the IDP. In Section 3 we work out a particular numeric example aimed at illustrating the previously formulated theoretic results. The last Section of the paper presents a conclusion.

2. Problem Formulation

2.1. Differential Game

Consider a differential game involving n participants (players): ${\Gamma }^d(t_0,x_0)$ (where the superscript d refers to discounting). Suppose the set of players is $N=\{1,\dots ,n\}$. Assume that the game initiates at the moment $t_0$ with the initial state $x_0$.

The n-person differential game with prescribed duration $T-t_0$ in which the integral payoff of the player i can be represented in the following form.

• The dynamic constraint conditions for the game are given by

  $$\left\{\begin{array}{c}\dot{x}=g(x\left(t\right),u_1\left(t\right),...,u_n\left(t\right)),\hfill \\ x\in {\mathbb{R}}^h,x\left(t_0\right)=x_0.\hfill \end{array}\right.,$$

  (1)

  where (1) satisfy the standard requirements of existence and uniqueness. In particular, we assume that the function $g(x\left(t\right),u_1\left(t\right),\dots ,u_n\left(t\right))$ in (1) is continuously differentiable w.r.t. all its arguments;
• The controls $u_i\left(t\right)$ are assumed to be piecewise continuous functions on the interval $[t_0,T]$ that belong to the set of admissible control values $U_i$, which are consequently convex compact subsets of ${\mathbb{R}}^k$. The optimal controls are further assumed to be open-loop, i.e., they are defined as functions of t.

We will consider a differential game such that the payoff function changes its structure at specific time instants. Specifically, we will consider the situation in which the discounting function changes as one goes from one interval to another. Let $\sigma =\{T_0,\dots ,T_j,\dots ,T_r\}$ s.t. $t_0=T_0<T_1<\cdots <T_{r-1}<T_r=T$ be an ordered sequence of time instants at which the switches occur. Then the payoff of player i is defined as follows:

$$\begin{array}{c}{K}_i(x_0,\sigma ,u)=\sum _{j=0}^{r-1}\underset{T_j}{\overset{T_{j+1}}{\int }}{h}_i(x\left(t\right),u\left(t\right))L_j\left(t\right)dt,T_0=t_0,T_r=T,\end{array}$$

(2)

where $L_j\left(t\right)$ is the discounting function on the time interval $t\in [T_j;T_{j+1})$, and $h_i$ is the instantaneous payoff of the ith player. For the further analysis of problems with heterogeneous discounting see [3].

We assume the following conditions to be fulfilled for $L_j\left(t\right)$, $j=0,\dots r-1$:

• $L_0\left(T_0\right)=1$ and $L_{r-1}\left(T\right)=0$, i.e., the discounting function is equal to 1 at the initial time and 0 at the final time;
• $L_j\left(t\right)$, $j=0,\dots ,r-1$ are non-increasing and continuously differentiable a.e. functions on $[T_j;T_{j+1}]$;
• The discounting functions on the neighboring intervals agree at the switching points:

  $$L_j\left(T_{j+1}\right)=L_{j+1}\left(T_{j+1}\right),j=0,\dots r-2.$$

  (3)

An example of a discounting function is given in Figure 1. Note that this Figure contains not only the discounting function for the whole game, but also its restriction to a subgame as described in Section 2.2.

First, we present an approach to construct a composite discounting function for a given set of not necessarily coordinated functions. Let a set of functions $l_j\left(t\right)=1-{\varphi }_j\left(t\right)$, $j=0,\cdots ,r-1$ be given, where the functions ${\varphi }_j\left(t\right)$ satisfy the following conditions:

• ${\varphi }_0\left(T\right)=0$; ${\varphi }_{r-1}\left(T\right)=1$;
• ${\varphi }_j\left(t\right)$, $j=0,\dots ,r-1$ are non-decreasing and continuously differentiable a.e. on $[T_j;T_{j+1}]$.

We have chosen to use this specific form of individual discounting functions expressed in terms of ${\varphi }_j$ to ensure that our presentation will be compatible with the subsequent exposition. However, this choice is merely a convention and can be changed as long as the individual discounting functions satisfy the required properties.

We define a composite discounting function $L\left(t\right)$ on the base of the individual functions $l_j\left(t\right)=1-{\varphi }_j\left(t\right)$, $j=0,\cdots ,r-1$ while ensuring the property (3):

$$L\left(t\right)=\left\{\begin{array}{cc}{L}_0\left(t\right)=l_0\left(t\right)=1-{\varphi }_0\left(t\right),\hfill & t\in [T_0,T_1),\hfill \\ L_1\left(t\right)=l_1\left(t\right)\frac{L_0\left(T_1\right)}{l_1\left(T_1\right)}=(1-{\varphi }_1\left(t\right))\frac{L_0\left(T_1\right)}{1-{\varphi }_1\left(T_1\right)},\hfill & t\in [T_1,T_2),\hfill \\ \dots \hfill \\ L_{r-1}\left(t\right)=l_{r-1}\left(t\right)\frac{L_{r-2}\left(T_{r-1}\right)}{l_{r-1}\left(T_{r-1}\right)}=(1-{\varphi }_{r-1}\left(t\right))\frac{L_{r-2}\left(T_{r-1}\right)}{1-{\varphi }_{r-1}\left(T_{r-1}\right)},\hfill & t\in [T_{r-1},T],\hfill \end{array}\right.$$

(4)

We have previously assumed that ${\varphi }_j^{\prime }\left(t\right)$ exists a.e. for any $j=0,\dots ,r-1$ and $t\in [t_0,T]$. Let us define the new function ${\lambda }_{\sigma }\left(t\right)$, which is referred to as hazard rate in the reliability theory (see, e.g., [20]):

$${\lambda }_{\sigma }\left(t\right)=\left\{\begin{array}{cc}{\lambda }_0\left(t\right)=\frac{{\varphi }_0^{\prime }\left(t\right)}{1-{\varphi }_0\left(t\right)},\hfill & t\in [T_0,T_1),\hfill \\ {\lambda }_1\left(t\right)=\frac{{\varphi }_1^{\prime }\left(t\right)}{1-{\varphi }_1\left(t\right)},\hfill & t\in [T_1,T_2),\hfill \\ \dots \hfill \\ {\lambda }_{r-1}\left(t\right)=\frac{{\varphi }_{r-1}^{\prime }\left(t\right)}{1-{\varphi }_{r-1}\left(t\right)},\hfill & t\in [T_{r-1},T].\hfill \end{array}\right.$$

(5)

Let us consider the first interval $[T_0;T_1)$ and the hazard function ${\lambda }_0\left(t\right)$. Then we have

$$\begin{array}{c}{\lambda }_0\left(t\right)=\frac{{\varphi }_0^{\prime }\left(t\right)}{1-{\varphi }_0\left(t\right)}=-\frac{\frac{d}{dt}(1-{\varphi }_0\left(t\right))}{1-{\varphi }_0\left(t\right)}=-\frac{d}{dt}ln(1-{\varphi }_0\left(t\right)).\end{array}$$

(6)

By integrating both sides of (6) from $T_0$ to t we obtain:

$${\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau =-ln\frac{1-{\varphi }_0\left(t\right)}{1-{\varphi }_0\left(T_0\right)},$$

whence

$$e^{-{\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau }=\frac{1-{\varphi }_0\left(t\right)}{1-{\varphi }_0\left(T_0\right)},t\in [T_0,T_1).$$

(7)

Finally, we can express $1-{\varphi }_0\left(t\right)$ from (7) as

$$1-{\varphi }_0\left(t\right)=(1-{\varphi }_0\left(T_0\right))e^{-{\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau }=e^{-{\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau },t\in [T_0,T_1),$$

(8)

and the first component of the payoff function now can be represented as

$$\underset{T_0}{\overset{T_1}{\int }}{h}_i(x\left(t\right),u\left(t\right))(1-{\varphi }_0\left(t\right))dt=\underset{T_0}{\overset{T_1}{\int }}{h}_i(x\left(t\right),u\left(t\right))e^{-{\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau }dt.$$

Similarly to (6)–(8) we obtain the general formula

$$1-{\varphi }_j\left(t\right)=(1-{\varphi }_j\left(T_j\right))e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau },t\in [T_j,T_{j+1})).$$

(9)

Substituting (9) into (5) we obtain an exponential representation for the composite discounting function $L\left(t\right)$:

$$L\left(t\right)=\left\{\begin{array}{cc}{L}_0\left(t\right)=e^{-{\int }_{T_0}^t{\lambda }_0\left(\tau \right)d\tau },\hfill & t\in [T_0,T_1),\hfill \\ L_1\left(t\right)=e^{-{\int }_{T_0}^{T_1}{\lambda }_0\left(\tau \right)d\tau }·e^{-{\int }_{T_1}^t{\lambda }_1\left(\tau \right)d\tau },\hfill & t\in [T_1,T_2),\hfill \\ \dots \hfill \\ L_{r-1}\left(t\right)={\displaystyle \prod _{j=0}^{r-2}}{e}^{-{\int }_{T_j}^{T_{j+1}}{\lambda }_j\left(\tau \right)d\tau }·e^{-{\int }_{T_{r-1}}^t{\lambda }_{r-1}\left(\tau \right)d\tau },\hfill & t\in [T_{r-1},T],\hfill \end{array}\right.$$

(10)

Note that while ${\lambda }_{r-1}\left(t\right)$ is undefined at $t=T$, it can be shown that ${lim}_{t\to T}L\left(t\right)=0$.

Thus we formulate the following result.

Proposition 1.

The payoff of the player i on the interval $[T_j,T_{j+1})$, $j=0,\dots r-1$ can be represented in the following form:

$$\underset{T_j}{\overset{T_{j+1}}{\int }}{h}_i(x\left(t\right),u\left(t\right))L_j\left(t\right)dt=e^{-{\int }_{T_0}^{T_1}{\lambda }_0\left(\tau \right)d\tau }·e^{-{\int }_{T_1}^{T_2}{\lambda }_1\left(\tau \right)d\tau }·\dots e^{-{\int }_{T_{j-1}}^{T_j}{\lambda }_{j-1}\left(\tau \right)d\tau }\underset{T_j}{\overset{T_{j+1}}{\int }}{h}_i(x\left(t\right),u\left(t\right))e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau }dt,$$

(11)

and the payoff of the player i at the whole game (12) can be written as

$$\begin{array}{c}{K}_i(x_0,\sigma ,u)=\sum _{j=0}^{r-1}\left(\prod _{k=0}^{j-1}{e}^{-{\int }_{T_k}^{T_{k+1}}{\lambda }_k\left(\tau \right)d\tau }\right)·\underset{T_j}{\overset{T_{j+1}}{\int }}{h}_i(x\left(t\right),u\left(t\right))e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau }dt,T_0=t_0,T_r=T.\end{array}$$

(12)

Thus the problem was reduced to the problem with different discounting factors ${\lambda }_j\left(t\right)$ on the different time intervals $[T_j,T_{j+1})$ (cf. [6]).

Taking (10) into account we can also rewrite the payoff of player i (12) in a more concise way:

$$\begin{array}{c}{K}_i(x_0,\sigma ,u)=\underset{T_0}{\overset{T}{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt.\end{array}$$

(13)

2.2. Subgame

Let the game evolve and follow the trajectory $x^{\ast }\left(t\right)$. At any instantaneous time instant $\tau$ the players enter into a subgame ${\Gamma }^d(\tau ,x^{\ast }\left(\tau \right))$, which is considered to be a new game from the position $x^{\ast }\left(\tau \right)$ with duration $T-\tau$.

To this end, we have to redefine the payoff function of the player i (13) to the payoff in a subgame. First we take into account that the discounting function $\tilde{L}\left(t\right)$ for the game (which is a subgame) on time interval $[\tau ;T]$ should be normalized such that $\tilde{L}\left(\tau \right)=1$, $\tilde{L}\left(T\right)=0$ (see Figure 1).

Let the subgame start at $\tau \in [T_0,T]$. We define

$$\tilde{L}\left(t\right)=\frac{L\left(t\right)}{L\left(\tau \right)}.$$

(14)

Let $\tau \in [T_j;T_{j+1}]$, $j=0,\cdots ,r-1$, then for $t\in [\tau ;T_{j+1}]$ we have

$$\frac{L_j\left(t\right)}{L_j\left(\tau \right)}=e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau }·e^{{\int }_{T_j}^{\tau }{\lambda }_j\left(\tau \right)d\tau },$$

and for the reason that $t\ge \tau$ we have

$$\frac{L_j\left(t\right)}{L_j\left(\tau \right)}=e^{-{\int }_{\tau }^t{\lambda }_j\left(\tau \right)d\tau }.$$

Respectively, the discounting function in the whole subgame is defined as

$$\tilde{L}\left(t\right)=\left\{\begin{array}{cc}{e}^{-{\int }_{\tau }^t{\lambda }_j\left(\tau \right)d\tau },\hfill & t\in [\tau ,T_{j+1}),\hfill \\ e^{-{\int }_{\tau }^{T_j}{\lambda }_j\left(\tau \right)d\tau }·e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau },\hfill & t\in [T_{j+1},T_{j+2}),\hfill \\ \dots \hfill \\ e^{-{\int }_{\tau }^{T_j}{\lambda }_j\left(\tau \right)d\tau }·e^{-{\int }_{T_j}^{T_{j+1}}{\lambda }_j\left(\tau \right)d\tau }·\dots e^{-{\int }_{T_{r-1}}^t{\lambda }_{r-1}\left(\tau \right)d\tau },\hfill & t\in [T_{r-1},T].\hfill \end{array}\right.$$

(15)

We have the following form of the payoff of the player i in the subgame started at $\tau$:

$$K_i(x\left(\tau \right),\tau ,{\sigma }_j,u)=\underset{\tau }{\overset{T}{\int }}{h}_i(x\left(t\right),u\left(t\right))\tilde{L}\left(t\right)dt=\frac{1}{L\left(\tau \right)}\underset{\tau }{\overset{T}{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt,$$

where we used the notation ${\sigma }_j=\sigma \setminus \{T_0,\cdots ,T_{j-1}\}$.

Now we obtain:

$$\begin{array}{cc}\hfill K_i(x_0,T_0,\sigma ,u)=& \underset{T_0}{\overset{T}{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt\hfill \\ \hfill =& \underset{T_0}{\overset{\tau }{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt+L\left(\tau \right)·\frac{1}{L\left(\tau \right)}\underset{\tau }{\overset{T}{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt\hfill \\ \hfill =& \underset{T_0}{\overset{\tau }{\int }}{h}_i(x\left(t\right),u\left(t\right))L\left(t\right)dt+L\left(\tau \right)K_i(x\left(\tau \right),\tau ,{\sigma }_j,u).\hfill \end{array}$$

2.3. Cooperative Differential Game

Suppose that the game is played in a cooperative circumstance. In general, cooperation means that a group of participants agrees to cooperate in a form of coalition before starting the game.

Assume that all players agreed to maximize their total payoff, which we denote by $V(N,x_0,\sigma )$. Let $u^{\ast }\left(t\right)=\{u_1^{\ast }\left(t\right),\dots ,u_n^{\ast }\left(t\right)\}$ be the optimal control, s.t.

$$(u_1^{\ast },...,u_n^{\ast })=arg\underset{u\in U_1\times \cdots \times U_n}{max}\sum _{i=1}^n{K}_i(x_0,\sigma ,u)$$

and the corresponding trajectory $x^{\ast }\left(t\right)$ obtained from (1) is said to be the optimal trajectory.

We also have

$$\sum _{i=1}^n{K}_i(x_0,\sigma ,u^{\ast })=V(N,x_0,\sigma ).$$

As the standard in cooperative games, all players in the coalition unanimously agree on a distribution mechanism (cooperative agreement) to divide the total payoff $V(N,x_0,\sigma )$. It is probable that the solution of the current game loses it optimality at some instant based on the cooperative solution that was initially chosen, which means that the time-consistency for cooperative solution is not guaranteed. Since we are investigating a dynamic setting, it is necessary to define and determine an imputation distribution procedure [21,22,23,24], which is supposed to be in accordance with the payoff form.

Beforehand, we basically recall the notion of imputation: in an n-players cooperative game, an imputation is a distribution $\xi =({\xi }_1,\dots ,{\xi }_n)$ among players such that the sum of its coordinates is equivalent to the maximal payoff of the grand coalition and the component ${\xi }_i$ distributed to the i-th player is not less than what the player would acquire through a sole game. To be specific, suppose the set of players is N and the characteristic function [24] of the game is $v:2^N⟶\mathbb{R}$, then $\xi$ is an imputation if ${\xi }_1+\cdots +{\xi }_n=v\left(N\right)$ and ${\xi }_i\ge v\left(\left\{i\right\}\right)$ for all $i=1,\dots ,n$. The first property called efficiency makes sure that the imputation is a distribution method for the total gain among all players [25].

We adopt the definition of an imputation distribution procedure (IDP) first introduced in [21] for differential games with prescribed duration.

Definition 1.

Given an imputation $\xi =({\xi }_1,\dots ,{\xi }_n)\in {\mathbb{R}}_{+}^n$, such that for all $i=1,\dots ,n$ we have

$${\xi }_i=\underset{T_0}{\overset{T}{\int }}{\beta }_i\left(t\right)L\left(t\right)dt=\sum _{j=0}^{r-1}\underset{T_j}{\overset{T_{j+1}}{\int }}{\beta }_i\left(t\right)L_j\left(t\right)dt=\sum _{j=0}^{r-1}\prod _{k=0}^{j-1}{e}^{-{\int }_{T_k}^{T_{k+1}}{\lambda }_j\left(\tau \right)d\tau }\underset{T_j}{\overset{T_{j+1}}{\int }}{\beta }_i\left(t\right)e^{-{\int }_{T_j}^t{\lambda }_j\left(\tau \right)d\tau }dt,$$

(16)

then the vector function $\beta \left(t\right)=({\beta }_1\left(t\right),\dots ,{\beta }_n\left(t\right))\in {\mathbb{R}}_{+}^n$ is called an imputation distribution procedure (IDP) in the game ${\Gamma }^d(t_0,x_0)$.

Furthermore, we define the notion of time-consistency of an imputation.

Definition 2.

An imputation $\xi =({\xi }_1,\cdots ,{\xi }_n)\in {\mathbb{R}}^n$ in the game ${\Gamma }^d(t_0,x_0)$ is time-consistent if there exists an IDP $\beta \left(t\right)=({\beta }_1\left(t\right),\cdots ,{\beta }_n\left(t\right))\in {\mathbb{R}}^N$ such that for any $\vartheta \in [0,T]$ the vector

$${\xi }_i^{\vartheta }=\frac{1}{L\left(\vartheta \right)}\underset{\vartheta }{\overset{T}{\int }}{\beta }_i\left(t\right)L\left(t\right)dt=\underset{\vartheta }{\overset{T}{\int }}{\beta }_i\left(t\right)\tilde{L}\left(t\right)dt$$

(17)

belongs to the same cooperative agreement in the subgame ${\Gamma }^d(\vartheta ,x^{\ast })$, i.e., ${\xi }^{\vartheta }$ is an imputation in ${\Gamma }^d(\vartheta ,x^{\ast })$.

We now check the time consistency property in detail. Let $\vartheta \in [T_j;T_{j+1})$. Then

$${\xi }_i^{\vartheta }=\underset{\vartheta }{\overset{T_{j+1}}{\int }}{\beta }_i\left(t\right)e^{-{\int }_{\vartheta }^t{\lambda }_k\left(\tau \right)d\tau }dt+\sum _{k=j+1}^{r-1}\underset{T_k}{\overset{T_{k+1}}{\int }}{\beta }_i\left(t\right)e^{-{\int }_{\vartheta }^{T_{j+1}}{\lambda }_k\left(\tau \right)d\tau }\prod _{l=j+1}^{k-1}{e}^{-{\int }_{T_l}^{T_{l+1}}{\lambda }_l\left(\tau \right)d\tau }{e}^{-{\int }_{T_k}^t{\lambda }_k\left(\tau \right)d\tau }dt.$$

Then we obtain

$${\xi }_i=\sum _{k=0}^{j-1}\underset{T_k}{\overset{T_{k+1}}{\int }}{\beta }_i\left(t\right)L_k\left(t\right)dt+\underset{T_j}{\overset{\vartheta }{\int }}{\beta }_i\left(t\right)L_j\left(t\right)dt+e^{-{\int }_{T_0}^{T_1}{\lambda }_0\left(\tau \right)d\tau }·e^{-{\int }_{T_1}^{T_2}{\lambda }_1\left(\tau \right)d\tau }·\dots e^{-{\int }_{T_j}^{\vartheta }{\lambda }_j\left(\tau \right)d\tau }{\xi }_i^{\vartheta }.$$

By taking the derivative with respect to $\vartheta$, $\vartheta \in [T_j;T_{j+1})$ and noting that ${\xi }_i$ is a constant, we obtain

$${\beta }_i\left(\vartheta \right)L_j\left(\vartheta \right)+\prod _{k=0}^{j-1}{e}^{-{\int }_{T_k}^{T_{k+1}}{\lambda }_k\left(\tau \right)d\tau }·e^{-{\int }_{T_j}^{\vartheta }{\lambda }_j\left(\tau \right)d\tau }{\left({\xi }_i^{\vartheta }\right)}_{\vartheta }^{\prime }-{\lambda }_j\left(\vartheta \right)\prod _{k=0}^{j-1}{e}^{-{\int }_{T_k}^{T_{k+1}}{\lambda }_k\left(\tau \right)d\tau }·e^{-{\int }_{T_j}^{\vartheta }{\lambda }_j\left(\tau \right)d\tau }{\xi }_i^{\vartheta }=0.$$

Recall that $L_j\left(\vartheta \right)={\prod }_{k=0}^{j-1}{e}^{-{\int }_{T_k}^{T_{k+1}}{\lambda }_k\left(\tau \right)d\tau }·e^{-{\int }_{T_j}^{\vartheta }{\lambda }_j\left(\tau \right)d\tau }$. Canceling the respective terms we obtain

$${\beta }_i\left(\vartheta \right)+{\left({\xi }_i^{\vartheta }\right)}_{\vartheta }^{\prime }-{\lambda }_j\left(\vartheta \right){\xi }_i^{\vartheta }=0,$$

whence the final expression for ${\beta }_i\left(\vartheta \right)$ results:

$${\beta }_i\left(\vartheta \right)={\lambda }_j\left(\vartheta \right){\xi }_i^{\vartheta }-{\left({\xi }_i^{\vartheta }\right)}_{\vartheta }^{\prime },\vartheta \in [T_j;T_{j+1}),j=0,\dots ,r-1.$$

We now formally state this result.

Theorem 1.

Let the imputation ${\xi }_i^t$ of the game ${\Gamma }^d(t,x^{\ast }\left(t\right))$ be an absolutely continuous function of $t\in [0,T]$. If the IDP has the form

$${\beta }_i\left(\vartheta \right)={\lambda }_{\sigma }\left(\vartheta \right){\xi }_i^{\vartheta }-{\left({\xi }_i^{\vartheta }\right)}_{\vartheta }^{\prime },$$

(18)

for any $\vartheta \in [0,T]$ then ${\xi }_i$ is a time-consistent imputation in the game ${\Gamma }^d(0,x_0)$ with IDP given by (16).

Note that this formula has the same form [2,24] as the IDP computed for a problem with a single discounting function with the only difference that instead of $\lambda \left(t\right)$ we use the composite hazard rate function ${\lambda }_{\sigma }\left(t\right)$. Furthermore, if we consider a game with a prescribed duration and without discounting, we have ${\lambda }_{\sigma }\left(\vartheta \right)\equiv 0$, and (18) takes the standard form as, e.g., in [23].

3. Computation of IDP: A Numerical Example

3.1. Description of the Model

This Section will build upon the results presented in [17]. We will skip most results that were previously reported except for those that are necessary for the understanding of the current material.

Consider a model example describing the differential game of investment into the stock of knowledge. Assume that there are N individuals investing in a public stock of knowledge [26]. Let $x\left(t\right)$ be the stock of knowledge at time t and $u_i\left(t\right)$ be the ith agent’s investment in public knowledge at time t. The dynamics of the stock of knowledge is described by

$$\dot{x}\left(t\right)=\sum _{i=1}^N{u}_i\left(t\right),x\in \mathbb{R},u_i\in U\subseteq \mathbb{R},x\left(t_0\right)=x_0.$$

(19)

If each agent derives linear utility from the consumption of knowledge, the instantaneous payoff of the ith player is described by

$$h_i(x\left(t\right),u\left(t\right))=q_{i}x\left(t\right)-r_i{u}_i^2\left(t\right),q_i>0,r_i>0.$$

(20)

Further assume that the time instants are $\sigma =\{0,\overline{T}-\delta ,\overline{T}+\delta ,T_1,T_2,T_3\}$, where $\overline{T}>\delta$ and $\overline{T}+\delta <T_1<T_2<T_3$. We define the ${\varphi }_i\left(t\right)$ functions as follows:

$${\varphi }_k\left(t\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}k=0,t\in [0,\overline{T}-\delta ),\hfill \\ (1-p_1-p_2)\frac{t-\overline{T}+\delta }{2\delta },\hfill & \mathrm{if}k=1,t\in [\overline{T}-\delta ,\overline{T}+\delta ),\hfill \\ 1-p_1-p_2,\hfill & \mathrm{if}k=2,t\in [\overline{T}+\delta ,T_1),\hfill \\ 1-p_2,\hfill & \mathrm{if}k=3,t\in [T_1,T_2),\hfill \\ 1,\hfill & \mathrm{if}k=4,t\in [T_2,T_3].\hfill \end{array}\right.$$

(21)

Note that the conditions formulated in Section 2 hold, i.e., ${\varphi }_0\left(0\right)=0$, ${\varphi }_4\left(T_3\right)=1$, and all functions ${\varphi }_i\left(\tau \right)$ are continuously differentiable.

This choice can be interpreted as a problem with random duration in which the game ends at the random time instant with a known cumulative distribution function (21). For instance, this c.d.f. means that the game cannot stop before $\overline{T}-\delta$ but then it may stop with the probability given by the uniform distribution at the time interval $[\overline{T}-\delta ;\overline{T}+\delta ]$. Let us denote this game as ${\Gamma }^r(t_0,x_0)$, where the superscript r refers to random duration.

3.2. Optimal Solution

We consider the cooperative game. Assume that all players opt to cooperate and, hence, group their efforts to maximize the total payoff.

$$\sum _{i=1}^n{K}_i(x_0,\sigma ,u)\to \underset{u}{max}$$

The optimization problem can be tackled using state parametrization under four continuous intervals (see [17] for details):

$$\begin{array}{ccccc}\hfill & I_1:[0;\overline{T}-\delta ]\hfill & \hfill & \leftarrow \mathrm{both}\mathrm{sides}\mathrm{are}\mathrm{fixed}\hfill & \hfill \{x_0,x_1\}\\ \hfill & I_2:[\overline{T}-\delta ;\overline{T}+\delta ]\hfill & \hfill & \leftarrow \mathrm{both}\mathrm{sides}\mathrm{are}\mathrm{fixed}\hfill & \hfill \{x_1,x_2\}\\ \hfill & I_3:[\overline{T}+\delta ;T_1]\hfill & \hfill & \leftarrow \mathrm{both}\mathrm{sides}\mathrm{are}\mathrm{fixed}\hfill & \hfill \{x_2,x_3\}\\ \hfill & I_4:[T_1;T_2]\hfill & \hfill & \leftarrow \mathrm{only}\mathrm{one}\mathrm{side}\mathrm{is}\mathrm{fixed}\hfill & \hfill \left\{x_3\right\}\end{array}$$

Note that we do not consider the fifth interval $[T_2,T_3]$ because on this interval ${\varphi }_4\left(t\right)=0$ and, respectively, $l_4\left(t\right)=0$.

For the interval $I_1$, we obtain the following expressions for the optimal trajectory and control:

$$\begin{array}{c}\hfill x^{\ast }{\left(t\right)}_{I_1}=\frac{(x_1-x_0)t}{(\overline{T}-\delta )}+\frac{1}{4}\widehat{q}t(\overline{T}-\delta -t)\sum _{i=1}^n\frac{1}{r_i}+x_0,\\ \hfill u_i^{\ast }{\left(t\right)}_{I_1}=\frac{x_1-x_0}{r_i{\sum }_{i=1}^n\frac{1}{r_i}(\overline{T}-\delta )}+\frac{\widehat{q}(\overline{T}-\delta )}{4r_i}-\frac{\widehat{q}t}{2r_i}.\end{array}$$

(22)

For the interval $I_2$, we have the expressions for the optimal trajectory and control shown below:

$$\begin{array}{cc}\hfill x^{\ast }{\left(t\right)}_{I_2}& =\frac{\widehat{q}{\sum }_{i=1}^n\frac{1}{r_i}}{2{(1-p_1-p_2)}^2}\left({\delta }^2-\frac{\chi {\left(t\right)}^2}{4}\right)+\hfill \\ \hfill +& \frac{2(x_2-x_1)(1-p_1-p_2)-\widehat{q}{\sum }_{i=1}^n\frac{1}{r_i}{\delta }^2(1+p_1+p_2)}{2ln(p_1+p_2)(1-p_1-p_2)}ln\frac{\chi \left(t\right)}{2\delta }+x_1,\hfill \\ \hfill u_i^{\ast }{\left(t\right)}_{I_2}& =\frac{\widehat{q}\chi \left(t\right)}{4r_i(1-p_1-p_2)}-\frac{2(x_2-x_1)(1-p_1-p_2)-\widehat{q}{\delta }^2{\sum }_{i=1}^n\frac{1}{r_i}(1+p_1+p_2)}{2r_i{\sum }_{i=1}^n\frac{1}{r_i}\chi \left(t\right)ln(p_1+p_2)},\hfill \end{array}$$

(23)

where $\chi \left(t\right)=2\delta -(1-p_1-p_2)(t-\overline{T}+\delta ).$

For the interval $I_3$, we obtain:

$$\begin{array}{cc}\hfill x^{\ast }{\left(t\right)}_{I_3}& =\sum _{i=1}^n\frac{1}{r_i}\left[\frac{(x_3-x_2)(t-\overline{T}-\delta )}{{\sum }_{i=1}^n\frac{1}{r_i}(T_1-\overline{T}-\delta )}+\frac{\widehat{q}(T_1-\overline{T}-\delta )(t-\overline{T}-\delta )}{4}-\frac{\widehat{q}{(t-\overline{T}-\delta )}^2}{4}\right]+x_2,\hfill \\ \hfill u_i^{\ast }{\left(t\right)}_{I_3}& =\frac{(x_3-x_2)}{r_i{\sum }_{i=1}^n\frac{1}{r_i}(T_1-\overline{T}-\delta )}+\frac{\widehat{q}(T_1+\overline{T}+\delta -2t)}{4r_i}.\hfill \end{array}$$

(24)

For the interval $I_4$, we have:

$$\begin{array}{cc}\hfill x^{\ast }{\left(t\right)}_{I_4}& =-\widehat{q}\sum _{i=1}^n\frac{1}{4r_i}\left[(t-T_1)(t+T_1-2T_2)\right]+x_3,\hfill \\ \hfill u_i^{\ast }{\left(t\right)}_{I_4}& =\frac{\widehat{q}(T_2-t)}{2r_i}.\hfill \end{array}$$

(25)

The switching states $x_1,x_2,x_3$ are given below:

$$\begin{array}{cc}\hfill x_1^{\ast }=& \frac{\widehat{q}(\overline{T}-\delta )(p_1(T_1-\overline{T})+p_2(T_2-\overline{T})+\frac{\overline{T}+\delta }{2}){\sum }_{i=1}^n\frac{1}{r_i}}{2}+x_0,\hfill \\ \hfill x_2^{\ast }=& \frac{\widehat{q}{\delta }^2{\sum }_{i=1}^n\frac{1}{r_i}}{2{(1-p_1-p_2)}^2}({(p_1+p_2)}^2(2ln(p_1+p_2)-1)+1)-\hfill \\ & -\frac{\widehat{q}\delta {\sum }_{i=1}^n\frac{1}{r_i}ln(p_1+p_2)}{1-p_1-p_2}(p_1(T_1-\overline{T}-\delta )+p_2(T_2-\overline{T}-\delta ))+x_1,\hfill \\ \hfill x_3^{\ast }=& \frac{{\sum }_{i=1}^n\frac{1}{r_i}\widehat{q}(T_1-\overline{T}-\delta )}{2(p_1+p_2)}\left[\frac{(p_1+p_2)(T_1-\overline{T}-\delta )}{2}+p_2(T_2-T_1)\right]+x_2.\hfill \end{array}$$

(26)

Note that all optimal values of the switching states depend (either directly or indirectly) on the initial state $x_0$.

3.3. Optimal Solutions for Subgames

In this section we consider the optimal solutions in a subgame starting from some time $\vartheta \in [0,T_3]$. The formal definition of a subgame and the respective analysis of the related optimal control problems is presented in Section 2.2.

Subgame starting at $\vartheta \in [\overline{T}-\delta ,\overline{T}+\delta )$: Consider a subgame ${\Gamma }^r(\vartheta ,x^{\ast })$ such that $\vartheta \in [\overline{T}-\delta ,\overline{T}+\delta )$. The conditional c.d.f., which corresponds to the function $\tilde{L}\left(t\right)$ defined in Section 2.2, takes the following form:

$$F_{\vartheta }\left(t\right)=\left\{\begin{array}{cc}{\displaystyle \frac{\varphi \left(t\right)-\varphi \left(\vartheta \right)}{1-\varphi \left(\vartheta \right)}},\hfill & t\in [\vartheta ,\overline{T}+\delta )\hfill \\ {\displaystyle \frac{1-p_1-p_2-\varphi \left(\vartheta \right)}{1-\varphi \left(\vartheta \right)}},\hfill & t\in [\overline{T}+\delta ,T_1)\hfill \\ {\displaystyle \frac{1-p_2-\varphi \left(\vartheta \right)}{1-\varphi \left(\vartheta \right)}},\hfill & t\in [T_1,T_2),\hfill \\ 1,\hfill & t\ge T_2\hfill \end{array}\right..$$

The expected integral payoff for player i in this subgame is given by the following formula:

$$K_i(\vartheta ,x^{\ast },u_1,...,u_n)={\displaystyle \frac{1}{1-\varphi \left(\vartheta \right)}}\left[\underset{\vartheta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(t\right))h_i\left(t\right)dt+\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2)h_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt\right].$$

Subgame starting at $\vartheta \in [\overline{T}+\delta ,T_1]$: Consider a subgame ${\Gamma }^r(\vartheta ,x^{\ast })$ such that $\vartheta \in [\overline{T}+\delta ,T_1]$. The conditional c.d.f. takes the following form:

$$F_{\vartheta }\left(t\right)=\left\{\begin{array}{cc}0,\hfill & t\in [\vartheta ,T_1)\hfill \\ {\displaystyle \frac{p_1}{p_1+p_2}},\hfill & t\in [T_1,T_2)\hfill \\ 1,\hfill & t\ge T_2\hfill \end{array}\right..$$

The expected integral payoff for player i in this subgame is given by the following formula:

$$K_i(\vartheta ,x^{\ast },u_1,...,u_n)={\displaystyle \frac{1}{p_1+p_2}}\left[\underset{\vartheta }{\overset{T_1}{\int }}(p_1+p_2)h_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt\right].$$

Subgame starting at $\vartheta \in [T_1,T_2]$: Consider a subgame ${\Gamma }^r(\vartheta ,x^{\ast })$ such that $\vartheta \in [T_1,T_2]$. The conditional c.d.f. takes the following form:

$$F_{\vartheta }\left(t\right)=\left\{\begin{array}{cc}0,\hfill & t\in [\vartheta ,T_2)\hfill \\ 1,\hfill & t\ge T_2\hfill \end{array}\right..$$

The expected integral payoff for player i in this subgame is indicated in the following formula:

$$K_i(\vartheta ,x^{\ast },u_1,...,u_n)={\displaystyle \frac{1}{p_2}}\underset{\vartheta }{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt.$$

Finally, we reach the following general expression for the expected integral payoff of player i in the subgame ${\Gamma }^r(\vartheta ,x^{\ast })$, $\vartheta \in [t_0,T_2]$:

$$\begin{array}{cc}& K_i(\vartheta ,x^{\ast },u_1,...,u_n)=\hfill \\ \hfill & \left\{\begin{array}{cc}\underset{\vartheta }{\overset{\overline{T}-\delta }{\int }}{h}_i\left(\tau \right)d\tau +\underset{\overline{T}-\delta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(\tau \right))h_i\left(\tau \right)d\tau +\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2)h_i\left(\tau \right)d\tau +\underset{T_1}{\overset{T_2}{\int }}{p}_2{h}_i\left(\tau \right)d\tau ,\hfill & \mathrm{if}\vartheta \in [t_0;\overline{T}-\delta );\hfill \\ {\displaystyle \frac{1}{1-\varphi \left(\vartheta \right)}}\left[\underset{\vartheta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(t\right))h_i\left(t\right)dt+\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2)h_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt\right],\hfill & \mathrm{if}\vartheta \in [\overline{T}-\delta ,\overline{T}+\delta );\hfill \\ {\displaystyle \frac{1}{p_1+p_2}}\left[\underset{\overline{\vartheta }}{\overset{T_1}{\int }}(p_1+p_2)h_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt\right],\hfill & \mathrm{if}\vartheta \in [\overline{T}+\delta ,T_1);\hfill \\ {\displaystyle \frac{1}{p_2}}\underset{\overline{\overline{\vartheta }}}{\overset{T_2}{\int }}{p}_2{h}_i\left(t\right)dt,\hfill & \mathrm{if}\vartheta \in [T_1,T_2].\hfill \end{array}\right.\hfill \end{array}$$

(27)

3.4. Computation of the Imputation Distribution Procedure (IDP)

The definition of the imputation and the imputation distribution procedure were given in Section 2.3. Here we specify the previously given definitions for the considered differential game.

Specifically, we have that for a given imputation $\xi =({\xi }_1,\dots ,{\xi }_n)\in {\mathbb{R}}_{+}^n$ in a game ${\Gamma }^r(t_0,x_0)$, the imputation distribution procedure satisfies the following equation (compare to the equation in Definition 1):

$${\xi }_i=\underset{t_0}{\overset{\overline{T}-\delta }{\int }}{\beta }_i\left(\tau \right)d\tau +\underset{\overline{T}-\delta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(\tau \right)){\beta }_i\left(\tau \right)d\tau +\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2){\beta }_i\left(\tau \right)d\tau +\underset{T_1}{\overset{T_2}{\int }}{p}_2{\beta }_i\left(\tau \right)d\tau .$$

(28)

The next Definition formalizes the property of time-consistency for imputations.

Definition 3.

An imputation$\xi =({\xi }_1,\dots ,{\xi }_N)\in {\mathbb{R}}_{+}^N$in a game${\Gamma }^r(t_0,x^{\ast })$is time-consistent if there exists an IDP$\beta \left(t\right)=({\beta }_1\left(t\right),\dots ,{\beta }_n\left(t\right))\in {\mathbb{R}}_{+}^n$such that:

• For all $\vartheta \in [t_0,\overline{T}-\delta )$ the vector ${\xi }^{\vartheta }=({\xi }_1^{\vartheta },\dots ,{\xi }_n^{\vartheta })$, where

  $${\xi }_i^{\vartheta }=\underset{\vartheta }{\overset{\overline{T}-\delta }{\int }}{\beta }_i\left(\tau \right)d\tau +\underset{\overline{T}-\delta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(\tau \right)){\beta }_i\left(\tau \right)d\tau +\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2){\beta }_i\left(\tau \right)d\tau +\underset{T_1}{\overset{T_2}{\int }}{p}_2{\beta }_i\left(\tau \right)d\tau .$$

  for all $i=1,\dots ,n$, belongs to the same optimality principle in the subgame ${\Gamma }^r(\vartheta ,x^{\ast })$, i.e., ${\xi }^{\vartheta }$ is an imputation in ${\Gamma }^r(\vartheta ,x^{\ast })$;
• For all $\vartheta \in [\overline{T}-\delta ,\overline{T}+\delta )$ the vector ${\xi }^{\vartheta }=({\xi }_1^{\vartheta },\dots ,{\xi }_n^{\vartheta })$, where

  $${\xi }_i^{\vartheta }={\displaystyle \frac{1}{1-\varphi \left(\vartheta \right)}}\left[\underset{\vartheta }{\overset{\overline{T}+\delta }{\int }}(1-\varphi \left(t\right)){\beta }_i\left(t\right)dt+\underset{\overline{T}+\delta }{\overset{T_1}{\int }}(p_1+p_2){\beta }_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{\beta }_i\left(t\right)dt\right].$$

  for all $i=1,\dots ,n$, belongs to the same optimality principle in the subgame ${\Gamma }^r(\vartheta ,x^{\ast })$, i.e., ${\xi }^{\vartheta }$ is an imputation in ${\Gamma }^r(\vartheta ,x^{\ast })$;
• For all $\vartheta \in [\overline{T}+\delta ,T_1]$ the vector ${\xi }^{\vartheta }=\left({\xi }_1^{\vartheta },\dots ,{\xi }_n^{\vartheta }\right),$ where

  $${\xi }_i^{\vartheta }={\displaystyle \frac{1}{p_1+p_2}}\left[\underset{\vartheta }{\overset{T_1}{\int }}(p_1+p_2){\beta }_i\left(t\right)dt+\underset{T_1}{\overset{T_2}{\int }}{p}_2{\beta }_i\left(t\right)dt\right],$$

  for all $i=1,\dots ,n$, belongs to the same optimality principle in the subgame ${\Gamma }^r(\vartheta ,x^{\ast })$, i.e., ${\xi }^{\vartheta }$ is an imputation in ${\Gamma }^r(\vartheta ,x^{\ast })$;
• For all $\vartheta \in [T_1,T_2]$ the vector ${\xi }^{\vartheta }=\left({\xi }_1^{\vartheta },\dots ,{\xi }_n^{\vartheta }\right),$ where

  $${\xi }_i^{\vartheta }={\displaystyle \frac{1}{p_2}}\underset{\vartheta }{\overset{T_2}{\int }}{p}_2{\beta }_i\left(t\right)dt,$$

  for all $i=1,\dots ,n$, belongs to the same optimality principle in the subgame ${\Gamma }^r(\vartheta ,x^{\ast })$, i.e., ${\xi }^{\vartheta }$ is an imputation in ${\Gamma }^r(\vartheta ,x^{\ast })$.

Following the analysis presented in Section 2.3, we formulate the following formulas for determining the IDP.

Proposition 2.

If $\vartheta \in [t_0,\overline{T}-\delta )$, then for all $i=1,\dots ,n$, the i-th coordinate of the IDP is given by:

$${\beta }_i\left(\vartheta \right)=-{\left({\xi }_i^{\vartheta }\right)}^{\prime }.$$

(29)

If $\vartheta \in [\overline{T}-\delta ,\overline{T}+\delta )$, then for all $i=1,\dots ,n$, the i-th coordinate of the IDP is given by:

$${\beta }_i\left(\vartheta \right)={\displaystyle \frac{{\left(\varphi \left(\vartheta \right)\right)}^{\prime }}{1-\varphi \left(\vartheta \right)}}{\xi }_i^{\vartheta }-{\left({\xi }_i^{\vartheta }\right)}^{\prime }.$$

(30)

If $\vartheta \in [\overline{T}+\delta ,T_1)$, then for all $i=1,\dots ,n$, the i-th coordinate of the IDP is given by:

$${\beta }_i\left(\vartheta \right)=-{\left({\xi }_i^{\vartheta }\right)}^{\prime }.$$

(31)

If $\vartheta \in [T_1,T_2]$, then for all $i=1,\dots ,n$, the i-th coordinate of the IDP is given by:

$${\beta }_i\left(\vartheta \right)=-{\left({\xi }_i^{\vartheta }\right)}^{\prime }.$$

(32)

We conclude this part with the following proposition.

Proposition 3.

The imputation $\left\{{\xi }_i\right\}$ given by (28) is time-consistent if the respective IDPs are defined according to (29)–(32).

3.5. Numerical Illustration of the Computed IDP Numeric Example

To illustrate the obtained results we will consider the same parameter set as in [17]:

$$\begin{array}{cccc}\hfill & n=3\hfill & \hfill -& \mathrm{number}\mathrm{of}\mathrm{players}\hfill \\ \hfill & x_0=20\hfill & \hfill -& \mathrm{initial}\mathrm{number}\mathrm{of}\mathrm{funds}\hfill \\ \hfill & t_0=0\hfill & \hfill -& \mathrm{initial}\mathrm{time}\hfill \\ \hfill & \overline{T}=10,\delta =3,T_1=70,T_2=75\hfill & \hfill -& \mathrm{time}\mathrm{structure}\mathrm{parameters}\hfill \\ \hfill & q_1=1,q_2=3,q_3=6,\hfill & \hfill -& \mathrm{values}\mathrm{of}\mathrm{coefficients}\mathrm{of}\mathrm{payoffs}\hfill \\ \hfill & r_1=20,r_2=1,r_3=4\hfill & \\ \hfill & p_1=0.1,p_2=0.2\hfill & \hfill -& \mathrm{probabilities}\mathrm{to}\mathrm{stop}\mathrm{the}\mathrm{investment}\mathrm{at}{T}_1,T_2.\hfill \end{array}$$

Both the imputation and the imputation distribution procedure are illustrated in Figure 2. Note that the optimal solution undergoes a discontinuity at time $t=T_1$, where the cumulative probability function is discontinuous as well. One can observe that the imputation distribution procedure is positive during the whole interval of time except a short period of time between $t=11$ and $t=13$.

The resulting imputation and the imputation distribution function are computed in an egalitarian way in which the distribution for each player is taken in the form of the average of total payoff. Obviously, this approach can be extended to any further type of imputation.

4. Conclusions

The aim of this paper was not only to describe an approach of computing the IDP for a class of hybrid differential games, but also to present a worked-out example aimed at demonstrating the described procedure in full detail. The main points of the paper are as follows: (1) The differential game with hybrid discounting function can well describe a wide class of differential games, including the games with random horizon and a hybrid CDF; (2) the considered class of differential games can be described in a uniform way by using the notion of a hybrid hazard rate; (3) finally, it is possible to completely solve a problem of reasonable complexity. Our future work will be concentrated on extending the class of hybrid games.