Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

Abstract

The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order $\alpha \in (0,1)$ and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order $\alpha$ and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufficiently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.

1. Introduction

The paper follows the positional approach (see, e.g., [1,2,3]) and is concerned with the questions of how to characterize the value functional and construct optimal positional (feedback) strategies of the players in a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order $\alpha \in (0,1)$ and a given Bolza-type cost functional. For the basics of fractional calculus and the theory of fractional differential equations, the reader is referred to, e.g., [4,5,6]. Investigations of some other kinds of differential games for fractional-order systems can be found in, e.g., [7,8,9,10,11,12,13].

The differential game under consideration was previously studied in [14,15]. More precisely, in [14], based on a suitable approximation of the game by a differential game for a (first-order) time-delay system, it was proved that the original game has the value, and, moreover, optimal positional strategies of the players were obtained in the form of control procedures that use the approximating time-delay system as a guide (see, e.g., [1] (Section 8.2)). In [15], the characteristic u- and v-stability properties of the value functional were established, and optimal positional strategies of the players were built by a suitable modification of the method of extremal shift to accompanying points (see, e.g., [2] (Section 8)). It should be noted that the results obtained in [14,15] demonstrate a close relationship between positional differential games for fractional-order systems and that for time-delay systems.

The present paper continues these studies and proposes another way of constructing optimal positional strategies of the players, which is based on a different technique. Motivated by [16], the differential game under consideration is associated to the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant ($ci$-) derivatives of the order $\alpha$ and the natural boundary condition. Then, at a first step, a particular case is studied when it is additionally assumed that this Cauchy problem admits a sufficiently smooth (namely, $ci$-smooth of the order $\alpha$) solution. Using standard arguments (see, e.g., [3] (Section 11.5) and also [17] for time-delay systems), it is proved that this solution coincides with the value functional of the original differential game, and, furthermore, optimal positional strategies of the players can be obtained by applying the extremal aiming in the direction of the $ci$-gradient of the order $\alpha$ of this solution.

In a general non-smooth case, following, e.g., [3], it is shown that the value functional coincides with a generalized minimax solution of the given Cauchy problem [18,19]. The proof of this fact reduces to construction of optimal positional strategies of the players on the basis of this minimax solution. To this end, by analogy with, e.g., [20] and [3] (Section 12.2) (see also, e.g., [17,21] for time-delay systems), some “smoothing” transformation of the minimax solution is performed, which makes it possible to determine the desired extremal directions that can be used in the extremal aiming procedure instead of the $ci$-gradient of the order $\alpha$ (since it may fail to exist for the minimax solution). It should be emphasized that peculiarities inherent in fractional-order systems are handled with the help of a suitable choice of a Lyapunov–Krasovskii functional [19], which the transformation relies on.

The rest of paper is organized as follows. In Section 2, basic notation is given. The dynamical system of a fractional order and cost functional are described in Section 3. In Section 4, the value functional of the differential game is defined in terms of non-anticipative strategies of the players. In Section 5, positional strategies of the players are introduced. The Cauchy problem for the Hamilton–Jacobi–Bellman–Isaacs equation is formulated in Section 6. Section 7 deals with the case when the Cauchy problem has a smooth solution. In Section 8, the generalized minimax solution of this problem is considered, and its “smoothing” transformation is performed. Section 9 is devoted to the general non-smooth case. Concluding remarks are presented in Section 10.

2. Notation

Let $T>0$, $n\in \mathbb{N}$, and $\alpha \in (0,1)$ be fixed throughout the paper. By $\parallel ·\parallel$ and $\langle ·,·\rangle$, the Euclidean norm and inner product in ${\mathbb{R}}^n$ are denoted. Given $R\ge 0$, let $B\left(R\right)$ stand for the closed ball in ${\mathbb{R}}^n$ centered at the origin of radius R.

For every $t\in [0,T]$, following, e.g., [4] (Definition 2.3), let us consider the set $A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ of all functions $x:[0,t]\to {\mathbb{R}}^n$ that can be represented in the form

$$x\left(\tau \right)=x\left(0\right)+\frac{1}{\mathrm{\Gamma }\left(\alpha \right)}{\int }_0^{\tau }\frac{f\left(\xi \right)}{{(\tau -\xi )}^{1-\alpha }}\mathrm{d}\xi ,\tau \in [0,t],$$

for some (Lebesgue) measurable and essentially bounded function $f:[0,t]\to {\mathbb{R}}^n$. Here, the second term is the (left-sided) Riemann–Liouville fractional integral of the order $\alpha$ of the function $f(·)$ (see, e.g., [4] (Definition 2.1)), and $\mathrm{\Gamma }$ is the gamma function. The set $A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ is a linear space, and every function $x(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ is continuous (see, e.g., [4] (Remark 3.3)). Let $A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ be endowed with the norm

$${\parallel x(·)\parallel }_{[0,t]}=\underset{\tau \in [0,t]}{max}\parallel x\left(\tau \right)\parallel ,x(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n).$$

According to, e.g., [4] (Theorem 2.4), the (left-sided) Caputo fractional derivative of the order $\alpha$ of a function $x(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$, which is defined by (see, e.g., [5] (Section 2.4) and [6] (Chapter 3))

$${(}^C{D}^{\alpha }x)\left(\tau \right)=\frac{1}{\mathrm{\Gamma }(1-\alpha )}\frac{\mathrm{d}}{\mathrm{d}\tau }{\int }_0^{\tau }\frac{x\left(\xi \right)-x\left(0\right)}{{(\tau -\xi )}^{\alpha }}\mathrm{d}\xi ,$$

exists for almost every (a.e.) $\tau \in [0,t]$.

Finally, let us introduce the set G of all pairs $(t,w(·\left)\right)$ such that $t\in [0,T]$ and $w(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$. This set is endowed with the metric (see, e.g., [17] and also [16])

$$\begin{array}{c}dist\left((t,w(·)),(t^{\prime },w^{\prime }(·))\right)\hfill \\ =max\left\{{dist}^{\ast }\left((t,w(·)),(t^{\prime },w^{\prime }(·))\right),{dist}^{\ast }\left((t^{\prime },w^{\prime }(·)),(t,w(·))\right)\right\},\hfill \end{array}$$

where $(t,w(·\left)\right)$, $(t^{\prime },w^{\prime }(·))\in G$ and

$${dist}^{\ast }\left((t,w(·)),(t^{\prime },w^{\prime }(·))\right)=\underset{\tau \in [0,t]}{max}\underset{{\tau }^{\prime }\in [0,t^{\prime }]}{min}\sqrt{|\tau -{\tau }^{\prime }{|}^2+{\parallel w\left(\tau \right)-w^{\prime }\left({\tau }^{\prime }\right)\parallel }^2}.$$

Note that, by [16] (Proposition 8.2), the mappings $G\ni (t,w(·))\mapsto w(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ for a fixed $t\in [0,T]$ and $G\ni (t,w(·\left)\right)\mapsto t\in [0,T]$ are continuous, as well as, in view of the Arzelà–Ascoli theorem, the mapping $[0,T]\times A{C}^{\alpha }([0,T],{\mathbb{R}}^n)\ni (t,x(·))\mapsto (t,x_t(·))\in G$. Here, the symbol $x_t(·)$ means the restriction of the function $x(·)$ to the interval $[0,t]$, i.e.,

$$x_t\left(\tau \right)=x\left(\tau \right),\tau \in [0,t].$$

(1)

3. Dynamical System of Fractional Order and Cost Functional

The paper deals with a two-person zero-sum differential game in which the dynamical system is described by the fractional differential equation

$$\begin{array}{c}{(}^C{D}^{\alpha }x)\left(\tau \right)=f(\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right)),\\ \tau \in [0,T],x\left(\tau \right)\in {\mathbb{R}}^n,u\left(\tau \right)\in P,v\left(\tau \right)\in Q,\end{array}$$

(2)

where $\tau$ is time, $x\left(\tau \right)$ is the current state of the system, ${(}^C{D}^{\alpha }x)\left(\tau \right)$ is the Caputo derivative of the order $\alpha$, $u\left(\tau \right)$ and $v\left(\tau \right)$ are the current controls of the first and second players, respectively, $P\subset {\mathbb{R}}^{n_P}$ and $Q\subset {\mathbb{R}}^{n_Q}$ are compact sets, $n_P$, $n_Q\in \mathbb{N}$.

Assumption 1.

The function $f:[0,T]\times {\mathbb{R}}^n\times P\times Q\to {\mathbb{R}}^n$ is continuous and possesses the following two properties: for every $R\ge 0$, one can choose a number $\lambda >0$ such that

$$\begin{array}{c}\parallel f(\tau ,x,u,v)-f(\tau ,x^{\prime },u,v)\parallel \le \lambda \parallel x-x^{\prime }\parallel ,\\ \tau \in [0,T],x,x^{\prime }\in B\left(R\right),u\in P,v\in Q;\end{array}$$

and there exists a constant $c>0$ such that

$$\parallel f(\tau ,x,u,v)\parallel \le c(1+\parallel x\parallel ),\tau \in [0,T],x\in {\mathbb{R}}^n,u\in P,v\in Q.$$

(3)

As an initial data for system (2), we consider a pair $(t,w(·\left)\right)\in G$, which is called an initial position of the system (see, e.g., [14,15,16]). Here, t is an initial time, and the function $w(·)$ is treated as an initial history of a system’s motion. Thus, in view of notation (1), the initial condition takes the form $x_t(·)=w(·)$. By admissible (open-loop) controls of the first and second players on the time interval $[t,T]$, we mean any measurable functions $u:[t,T]\to P$ and $v:[t,T]\to Q$, respectively. Let $\mathcal{U}\left(t\right)$ and $\mathcal{V}\left(t\right)$ be the corresponding sets of all such controls. Denote

$$X(t,w(·))=\left\{x(·)\in A{C}^{\alpha }([0,T],{\mathbb{R}}^n):x_t(·)=w(·)\right\}.$$

(4)

According to, e.g., [14] (Proposition 2), from the initial position $(t,w(·\left)\right)$, every pair of controls $u(·)\in \mathcal{U}\left(t\right)$ and $v(·)\in \mathcal{V}\left(t\right)$ generate a unique motion of system (2), which is defined as a function $x(·)\in X(t,w(·\left)\right)$ that, together with $u(·)$ and $v(·)$, satisfies the fractional differential equation in (2) for a.e. $\tau \in [t,T]$.

For system (2), a differential game is studied in which the first player wants to minimize, and the second player wants to maximize the cost functional

$$\begin{array}{c}{\displaystyle J(t,w(·),u(·),v(·))=\sigma \left(x(·)\right)+{\int }_t^T\chi (\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\mathrm{d}\tau ,}\\ (t,w(·\left)\right)\in G,u(·)\in \mathcal{U}\left(t\right),v(·)\in \mathcal{V}\left(t\right),\end{array}$$

(5)

where $x(·)$ is the motion of system (2) generated from the initial position $(t,w(·\left)\right)$ by the players’ controls $u(·)$ and $v(·)$.

Assumption 2.

The mappings $\sigma :A{C}^{\alpha }([0,T],{\mathbb{R}}^n)\to \mathbb{R}$ and $\chi :[0,T]\times {\mathbb{R}}^n\times P\times Q\to \mathbb{R}$ are continuous, and, for every $R\ge 0$, one can find a number $\lambda >0$ such that

$$\begin{array}{c}|\chi (\tau ,x,u,v)-\chi (\tau ,x^{\prime },u,v)|\le \lambda \parallel x-x^{\prime }\parallel ,\\ \tau \in [0,T],x,x^{\prime }\in B\left(R\right),u\in P,v\in Q.\end{array}$$

4. Value Functional

The value functional of the differential game (2) and (5) is defined with the help of non-anticipative strategies of the players.

Let an initial position $(t,w(·\left)\right)\in G$ be fixed. By a non-anticipative strategy of the first player, we mean a mapping $\kappa :\mathcal{V}\left(t\right)\to \mathcal{U}\left(t\right)$ possessing the following property: for any $t^{\prime }\in [t,T]$ and any second player’s controls $v(·)$, $v^{\prime }(·)\in \mathcal{V}\left(t\right)$, if the equality $v\left(\tau \right)=v^{\prime }\left(\tau \right)$ holds for a.e. $\tau \in [t,t^{\prime }]$, then the corresponding first player’s controls $u(·)=\kappa \left(v\right(·\left)\right)$ and $u^{\prime }(·)=\kappa \left(v^{\prime }(·)\right)$ satisfy the equality $u\left(\tau \right)=u^{\prime }\left(\tau \right)$ for a.e. $\tau \in [t,t^{\prime }]$. Then, the lower value of the differential game is given by

$${\rho }^{-}(t,w(·))=\underset{\kappa }{inf}\underset{v(·)\in \mathcal{V}\left(t\right)}{sup}J\left(t,w(·),\kappa \left(v\right(·\left)\right),v(·)\right),$$

(6)

where the infimum is calculated over all first player’s non-anticipative strategies $\kappa$.

Similarly, a second player’s non-anticipative strategy is a mapping $\beta :\mathcal{U}\left(t\right)\to \mathcal{V}\left(t\right)$ such that, for every $t^{\prime }\in [t,T]$ and every $u(·)$, $u^{\prime }(·)\in \mathcal{U}\left(t\right)$, if $u\left(\tau \right)=u^{\prime }\left(\tau \right)$ for a.e. $\tau \in [t,t^{\prime }]$, then $v\left(\tau \right)=v^{\prime }\left(\tau \right)$ for a.e. $\tau \in [t,t^{\prime }]$, where $v(·)=\beta \left(u\right(·\left)\right)$ and $v^{\prime }(·)=\beta \left(u^{\prime }(·)\right)$. Respectively, the upper value of the differential game is given by

$${\rho }^{+}(t,w(·))=\underset{\beta }{sup}\underset{u(·)\in \mathcal{U}\left(t\right)}{inf}J\left(t,w(·),u(·),\beta \left(u\right(·\left)\right)\right),$$

(7)

where the supremum is calculated over all second player’s non-anticipative strategies $\beta$.

Let us suppose that, in addition to Assumptions 1 and 2, the following Isaacs’ condition, also known as the saddle point condition in a small game, is fulfilled.

Assumption 3.

The equality below is valid:

$$\begin{array}{c}{\displaystyle \underset{u\in P}{min}\underset{v\in Q}{max}\left(\langle s,f(\tau ,x,u,v)\rangle +r\chi (\tau ,x,u,v)\right)=\underset{v\in Q}{max}\underset{u\in P}{min}\left(\langle s,f(\tau ,x,u,v)\rangle +r\chi (\tau ,x,u,v)\right),}\\ \tau \in [0,T],x,s\in {\mathbb{R}}^n,r\in \mathbb{R}.\end{array}$$

Then, according to, e.g., [15] (Section 9), for every initial position $(t,w(·\left)\right)\in G$, the lower ${\rho }^{-}(t,w(·))$ and upper ${\rho }^{+}(t,w(·))$ game values coincide, which means that the differential game (2) and (5) has the value

$$\rho (t,w(·))={\rho }^{-}(t,w(·))={\rho }^{+}(t,w(·)).$$

(8)

Thus, relation (8) defines the game value functional $\rho :G\to \mathbb{R}$.

It should be noted that non-anticipative strategies, being a convenient tool in theoretical considerations, are rather difficult to implement. In this regard, the paper focuses on construction of optimal positional (feedback) strategies of the players, which are more acceptable from a practical point of view.

5. Positional Strategies

In accordance with [14,15], as a first player’s positional strategy in the differential game (2) and (5), we consider an arbitrary mapping $U:G\to P$. Let $(t,w(·\left)\right)\in G$ be an initial position, and let $\Delta$ be a partition of the time interval $[t,T]$, i.e.,

$$\Delta ={\left\{{\tau }_j\right\}}_{j\in \overline{1,k}},{\tau }_1=t,{\tau }_j<{\tau }_{j+1},j\in \overline{1,k-1},{\tau }_k=T,$$

where $k\in \mathbb{N}$. Here, and below, we use the notation $\overline{1,k}=\{j\in \mathbb{N}:j\le k\}$. The pair $(U,\Delta )$ is called a control law of the first player. From the initial position $(t,w(·\left)\right)$, this law together with a second player’s control $v(·)\in \mathcal{V}\left(t\right)$ uniquely generate the first player’s control $u(·)\in \mathcal{U}\left(t\right)$ (and, respectively, the corresponding motion $x(·)$ of system (2)) by the following step-by-step rule:

$$u\left(\tau \right)=U({\tau }_j,x_{{\tau }_j}(·)),\tau \in [{\tau }_j,{\tau }_{j+1}),j\in \overline{1,k-1},$$

(9)

and, formally, $u\left(T\right)=U(T,x(·\left)\right)$. In other words, at every time ${\tau }_j$, $j\in \overline{1,k-1}$, the first player measures the history $x_{{\tau }_j}(·)$ of the motion $x(·)$ on $[0,{\tau }_j]$ (see (1)), computes the value $u_j=U({\tau }_j,x_{{\tau }_j}(·))$, and then applies the constant control $u\left(\tau \right)=u_j$ until ${\tau }_{j+1}$, when a new measurement of the history is taken. Let us denote the corresponding value of cost functional (5) by $J(t,w(·),(U,\Delta ),v(·\left)\right)$.

Observe that, by construction, the mapping that assigns to each second player’s control $v(·)\in \mathcal{V}\left(t\right)$ the first player’s control $u(·)\in \mathcal{U}\left(t\right)$ formed by $(U,\Delta )$ is a non-anticipative strategy of the first player. Hence, in view of (6) and (8), we have

$$\underset{v(·)\in \mathcal{V}\left(t\right)}{sup}J\left(t,w(·),(U,\Delta ),v(·)\right)\ge \rho (t,w(·)).$$

(10)

In this connection, given a set $K\subset G$ and a number $\zeta >0$, a positional strategy of the first player U is called $\zeta$-optimal uniformly on K if the following property holds: there exists $\delta >0$ such that, for any initial position $(t,w(·\left)\right)\in K$ and any partition $\Delta$ of $[t,T]$ with the diameter $diam(\Delta )=max\{{\tau }_{j+1}-{\tau }_j:j\in \overline{1,k-1}\}\le \delta$, the inequality

$$\underset{v(·)\in \mathcal{V}\left(t\right)}{sup}J\left(t,w(·),(U,\Delta ),v(·)\right)\le \rho (t,w(·))+\zeta$$

takes place. Respectively, we say that the strategy U is optimal uniformly on the set K if it is $\zeta$-optimal uniformly on this set for every $\zeta >0$.

Similarly, any mapping $V:G\to Q$ is considered as a positional strategy of the second player. Given an initial position $(t,w(·\left)\right)\in G$, a partition $\Delta$ of the time interval $[t,T]$, and a first player’s control $u(·)\in \mathcal{U}\left(t\right)$, let $J(t,w(·),u(·),(V,\Delta \left)\right)$ be the value of cost functional (5) that corresponds to the second player’s control $v(·)\in \mathcal{V}\left(t\right)$ formed by the law $(V,\Delta )$ as follows:

$$v\left(\tau \right)=V({\tau }_j,x_{{\tau }_j}(·)),\tau \in [{\tau }_j,{\tau }_{j+1}),j\in \overline{1,k-1},$$

and, formally, $v\left(T\right)=V(T,x(·\left)\right)$. Due to (7) and (8), the estimate below is valid:

$$\underset{u(·)\in \mathcal{U}\left(t\right)}{inf}J\left(t,w(·),u(·),(V,\Delta )\right)\le \rho (t,w(·)).$$

(11)

Then, for a set $K\subset G$ and a number $\zeta >0$, a positional strategy of the second player V is called $\zeta$-optimal uniformly on K if there exists $\delta >0$ such that the inequality

$$\underset{u(·)\in \mathcal{U}\left(t\right)}{inf}J\left(t,w(·),u(·),(V,\Delta )\right)\ge \rho (t,w(·))-\zeta$$

is fulfilled for any $(t,w(·\left)\right)\in K$ and any partition $\Delta$ of $[t,T]$ with $diam(\Delta )\le \delta$. We say that V is optimal uniformly on K if it is $\zeta$-optimal uniformly on K for every $\zeta >0$.

6. Hamilton–Jacobi–Bellman–Isaacs Equation

Denote $G^0=\{(t,w(·))\in G:t<T\}$. According to [16], a functional $\phi :G\to \mathbb{R}$ is called coinvariantly ($ci$-) differentiable of the order $\alpha$ at a point $(t,w(·))\in G^0$ if there exist ${\partial }_t^{\alpha }\phi (t,w(·))\in \mathbb{R}$ and ${\nabla }^{\alpha }\phi (t,w(·))\in {\mathbb{R}}^n$ such that, for every function $x(·)\in X(t,w(·\left)\right)$ (see (4)), the following relation holds:

$$\begin{array}{c}\phi (\tau ,x_{\tau }(·))-\phi (t,w(·))={\partial }_t^{\alpha }\phi (t,w(·))(\tau -t)\hfill \\ {\displaystyle \phantom{\phi (\tau ,x_{\tau }(·))-\phi (t,w(·))}+{\int }_t^{\tau }\langle {\nabla }^{\alpha }\phi (t,w(·)),{(}^C{D}^{\alpha }x)\left(\xi \right)\rangle \mathrm{d}\xi +o(\tau -t),\tau \in (t,T),}\hfill \end{array}$$

where $x_{\tau }(·)$ is determined by (1), the function $(0,\infty )\ni \delta \mapsto o\left(\delta \right)\in \mathbb{R}$ may depend on t and $x(·)$, and $o\left(\delta \right)/\delta \to 0$ when $\delta \to +0$. The values ${\partial }_t^{\alpha }\phi (t,w(·))$ and ${\nabla }^{\alpha }\phi (t,w(·))$ are called, respectively, the $ci$-derivative in t and $ci$-gradient of the order $\alpha$ of $\phi$ at $(t,w(·\left)\right)$.

To the differential game (2) and (5), we associate the Hamiltonian (see Assumption 3)

$$\begin{array}{c}{\displaystyle H(t,x,s)=\underset{u\in P}{min}\underset{v\in Q}{max}\left(\langle s,f(t,x,u,v)\rangle +\chi (t,x,u,v)\right)}\hfill \\ {\displaystyle \phantom{H(t,x,s)}=\underset{v\in Q}{max}\underset{u\in P}{min}\left(\langle s,f(t,x,u,v)\rangle +\chi (t,x,u,v)\right),t\in [0,T],x,s\in {\mathbb{R}}^n,}\hfill \end{array}$$

(12)

and the Cauchy problem for the Hamilton–Jacobi–Bellman–Isaacs equation

$${\partial }_t^{\alpha }\phi (t,w(·))+H\left(t,w\left(t\right),{\nabla }^{\alpha }\phi (t,w(·))\right)=0,(t,w(·))\in G^0,$$

(13)

under the right-end boundary condition

$$\phi (T,w(·))=\sigma \left(w(·)\right),w(·)\in A{C}^{\alpha }([0,T],{\mathbb{R}}^n).$$

(14)

The unknown in problem (13) and (14) is a functional $\phi :G\to \mathbb{R}$.

The goal of the paper is to establish a relationship between the differential game (2) and (5) and the Cauchy problem (13) and (14) with a particular emphasis on construction of optimal and $\zeta$-optimal positional control strategies of the players. Let us first study this question under an additional supposition that the Cauchy problem (13) and (14) admits a sufficiently smooth solution.

7. Optimal Positional Strategies in the Smooth Case

Following [16], we say that a functional $\phi :G\to \mathbb{R}$ is $ci$-smooth of the order $\alpha$ if it is continuous, $ci$-differentiable of the order $\alpha$ at every point $(t,w(·))\in G^0$, and the mappings ${\partial }_t^{\alpha }\phi :G^0\to \mathbb{R}$ and ${\nabla }^{\alpha }\phi :G^0\to {\mathbb{R}}^n$ are continuous. If such a functional $\phi$ satisfies the Hamilton–Jacobi–Bellman–Isaacs equation (13) and boundary condition (14), then it is called a classical solution of the Cauchy problem (13) and (14).

Let us suppose that a classical solution $\phi :G\to \mathbb{R}$ of the Cauchy problem (13) and (14) exists. In this case, let us define players’ positional strategies $U^{\circ }$ and $V^{\circ }$ by applying the extremal aiming in the direction of the $ci$-gradient of the order $\alpha$ of this solution:

$$\begin{array}{c}{\displaystyle U^{\circ }(t,w(·))\in \underset{u\in P}{argmin}\underset{v\in Q}{max}\left(\langle {\nabla }^{\alpha }\phi (t,w(·)),f(t,w\left(t\right),u,v)\rangle +\chi (t,w\left(t\right),u,v)\right),}\\ {\displaystyle V^{\circ }(t,w(·))\in \underset{v\in Q}{argmax}\underset{u\in P}{min}\left(\langle {\nabla }^{\alpha }\phi (t,w(·)),f(t,w\left(t\right),u,v)\rangle +\chi (t,w\left(t\right),u,v)\right),}\\ (t,w(·))\in G^0,\end{array}$$

(15)

and, formally, let $U^{\circ }(T,w(·))\in P$ and $V^{\circ }(T,w(·))\in Q$ be chosen arbitrarily for any $w(·)\in A{C}^{\alpha }([0,T],{\mathbb{R}}^n)$.

Theorem 1.

Let Assumptions 1, 2, and 3 hold. In addition, suppose that the Cauchy problem (13) and (14) admits a classical solution $\phi :G\to \mathbb{R}$. Then, this solution φ coincides with the value functional $\rho :G\to \mathbb{R}$ of the differential game (2) and (5), i.e.,

$$\phi (t,w(·\left)\right)=\rho (t,w(·\left)\right),(t,w(·\left)\right)\in G,$$

(16)

and the players’ positional strategies $U^{\circ }$ and $V^{\circ }$ defined by this solution φ according to (15) are optimal uniformly on each compact set $K\subset G$.

The proof of the theorem follows the scheme from, e.g., [3] (Section 11.5) (see also [17] (Theorem 3.1) for time-delay systems), and, for convenience, it is divided into the two lemmas below, which are valid under the assumptions of the theorem.

Lemma 1.

For every compact set $K\subset G$ and every $\zeta >0$, there exists $\delta >0$ such that, for any initial position $(t,w(·\left)\right)\in K$, any partition Δ of $[t,T]$ with $diam(\Delta )\le \delta$, and any second player’s control $v(·)\in \mathcal{V}\left(t\right)$, the inequality below holds:

$$J\left(t,w(·),(U^{\circ },\Delta ),v(·)\right)\le \phi (t,w(·))+\zeta .$$

(17)

Before proceeding to the proof of the lemma, let us carry out some auxiliary constructions, which are also needed in the subsequent sections. Denote

$$\begin{array}{c}{X}_{\ast }(t,w(·))=\{x(·)\in A{C}^{\alpha }([0,T],{\mathbb{R}}^n):x(·)\in X(t,w(·)),\hfill \\ \phantom{X_{\ast }(t,w(·))=}\parallel {(}^C{D}^{\alpha }x)\left(\tau \right)\parallel \le c(1+\parallel x\left(\tau \right)\parallel )\mathrm{for}\mathrm{a}.\mathrm{e}.\tau \in [t,T]\},(t,w(·))\in G,\hfill \end{array}$$

(18)

where c is the constant from (3) and $X(t,w(·\left)\right)$ is the set from (4). Let a compact set $K\subset G$ be given. Consider the set

$$X_{\ast }=\left\{x(·)\in A{C}^{\alpha }([0,T],{\mathbb{R}}^n):x(·)\in X_{\ast }(t,w(·)),(t,w(·))\in K\right\}.$$

(19)

According to [22] (Theorem 2), compactness of the set K implies compactness of the set $X_{\ast }$ in $A{C}^{\alpha }([0,T],{\mathbb{R}}^n)$. In particular, by the Arzelà–Ascoli theorem, all functions $x(·)\in X_{\ast }$ are uniformly bounded and equicontinuous. Let a number $R_{\ast }>0$ be such that

$$\parallel x\left(\tau \right)\parallel \le R_{\ast },\tau \in [0,T],x(·)\in X_{\ast }.$$

(20)

Then, in view of (3), the following estimate is valid:

$$\parallel f(\tau ,x\left(\tau \right),u,v)\parallel \le c(1+R_{\ast }),\tau \in [0,T],x(·)\in X_{\ast },u\in P,v\in Q.$$

(21)

Furthermore, due to continuity of the functions f and $\chi$, let us take a number $R^{\ast }>0$ and a modulus of continuity $m^{\ast }(·)$ such that

$$\begin{array}{c}c(1+R_{\ast })\le R^{\ast },\parallel f(\tau ,x\left(\tau \right),u,v)-f({\tau }^{\prime },x\left({\tau }^{\prime }\right),u,v)\parallel \le m^{\ast }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \left|\chi (\tau ,x\left(\tau \right),u,v)\right|\le R^{\ast },|\chi (\tau ,x\left(\tau \right),u,v)-\chi ({\tau }^{\prime },x\left({\tau }^{\prime }\right),u,v)|\le m^{\ast }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \tau ,{\tau }^{\prime }\in [0,T],x(·)\in X_{\ast },u\in P,v\in Q.\end{array}$$

(22)

Proof of Lemma 1.

Let a compact set $K\subset G$ and $\zeta >0$ be fixed. Define the compact set $X_{\ast }\subset A{C}^{\alpha }([0,T],{\mathbb{R}}^n)$, number $R^{\ast }$, and modulus of continuity $m^{\ast }(·)$ as above. Since the functional $\phi$ is continuous, one can choose $\vartheta \in [0,T)$ from the condition

$$|\phi (T,x(·))-\phi (\tau ,x_{\tau }(·))|+(T-\tau )R^{\ast }\le \zeta /2,\tau \in [\vartheta ,T],x(·)\in X_{\ast }.$$

(23)

Then, owing to continuity of the mappings ${\partial }_t^{\alpha }\phi$ and ${\nabla }^{\alpha }\phi$, there are a number $R_{\vartheta }>0$ and a modulus of continuity $m_{\vartheta }(·)$ such that

$$\begin{array}{c}|{\partial }_t\phi (\tau ,x_{\tau }(·))-{\partial }_t\phi ({\tau }^{\prime },x_{{\tau }^{\prime }}(·))|\le m_{\vartheta }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \parallel {\nabla }^{\alpha }\phi (\tau ,x_{\tau }(·))\parallel \le R_{\vartheta },\parallel {\nabla }^{\alpha }\phi (\tau ,x_{\tau }(·))-{\nabla }^{\alpha }\phi ({\tau }^{\prime },x_{{\tau }^{\prime }}(·))\parallel \le m_{\vartheta }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \tau ,{\tau }^{\prime }\in [0,\vartheta ],x(·)\in X_{\ast }.\end{array}$$

(24)

Let us show that the conclusion of the lemma holds for $\delta >0$ satisfying the inequality

$$(1+R^{\ast })m_{\vartheta }\left(\delta \right)+(1+R_{\vartheta })m^{\ast }\left(\delta \right)\le \zeta /\left(2T\right).$$

(25)

Take an initial position $(t,w(·\left)\right)\in K$ and a partition $\Delta$ of $[t,T]$ with $diam(\Delta )\le \delta$. Consider a motion $x(·)$ of system (2) generated by the first player’s control law $(U^{\circ },\Delta )$ and a second player’s control $v(·)\in \mathcal{V}\left(t\right)$. Let $u(·)\in \mathcal{U}\left(t\right)$ be the corresponding first player’s control. Observe that the inclusion $x(·)\in X_{\ast }$ is fulfilled by (3), (18), and (19).

Let us introduce the function

$$\omega \left(\tau \right)=\phi (\tau ,x_{\tau }(·))+{\int }_t^{\tau }\chi (\xi ,x\left(\xi \right),u\left(\xi \right),v\left(\xi \right))\mathrm{d}\xi ,\tau \in [t,T].$$

Then, we have $\omega \left(t\right)=\phi (t,w(·\left)\right)$ and, in accordance with (5) and (14), we obtain $\omega \left(T\right)=J(t,w(·),(U^{\circ },\Delta ),v(·))$. Thus, the desired inequality (17) can be rewritten as follows:

$$\omega \left(T\right)-\omega \left(t\right)\le \zeta .$$

(26)

In the case when $t\in [\vartheta ,T]$, applying (22) and (23), we derive

$$\omega \left(T\right)-\omega \left(t\right)\le \phi (T,x(·))-\phi (t,x_t(·))+(T-t)R^{\ast }\le \zeta /2\le \zeta .$$

(27)

Now, let us suppose that $t<\vartheta$. Similarly to (27), we get $\omega \left(T\right)-\omega \left(\vartheta \right)\le \zeta /2$, and, therefore, in order to prove (26), it suffices to verify that

$$\omega \left(\theta \right)-\omega \left(t\right)\le \zeta /2.$$

(28)

Since the functional $\phi$ is $ci$-smooth of the order $\alpha$, based on [16] (Lemma 9.2), we conclude that the function $\omega (·)$ is Lipschitz continuous on $[t,\vartheta ]$ and

$$\begin{array}{c}\dot{\omega }\left(\tau \right)={\partial }_t^{\alpha }\phi (\tau ,x_{\tau }(·))+\langle {\nabla }^{\alpha }\phi (\tau ,x_{\tau }(·)),f(\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\rangle \hfill \\ \phantom{\dot{\omega }\left(\tau \right)}+\chi (\tau ,x(\tau ),u(\tau ),v(\tau \left)\right)\mathrm{for}\mathrm{a}.\mathrm{e}.\tau \in [t,\vartheta ].\hfill \end{array}$$

(29)

Here, and below, we use the notation $\dot{\omega }\left(\tau \right)=\mathrm{d}\omega \left(\tau \right)/\mathrm{d}\tau$. Consider $\tau \in [t,\vartheta )$ for which equality (29) is valid. Let $j\in \overline{1,k-1}$ be such that $\tau \in [{\tau }_j,{\tau }_{j+1})$. Then, estimates (21), (22), and (24) and the choice (25) of $\delta$ imply that

$$\begin{array}{c}\dot{\omega }\left(\tau \right)\le {\partial }_t^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·))+m_{\vartheta }(\tau -{\tau }_j)\hfill \\ \phantom{\dot{\omega }\left(\tau \right)}+\langle {\nabla }^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·)),f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\rangle +R^{\ast }{m}_{\vartheta }(\tau -{\tau }_j)+R_{\vartheta }{m}^{\ast }(\tau -{\tau }_j)\hfill \\ \phantom{\dot{\omega }\left(\tau \right)}+\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))+m^{\ast }(\tau -{\tau }_j)\hfill \\ \phantom{\dot{\omega }\left(\tau \right)}\le {\partial }_t^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·))+\langle {\nabla }^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·)),f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\rangle \hfill \\ \phantom{\dot{\omega }\left(\tau \right)}+\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))+\zeta /\left(2T\right).\hfill \end{array}$$

(30)

According to (9), we have $u\left(\tau \right)=U^{\circ }({\tau }_j,x_{{\tau }_j}(·))$. Hence, it follows from the definition (15) of the strategy $U^{\circ }$ and the definition (12) of the Hamiltonian H that

$$\begin{array}{c}\langle {\nabla }^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·)),f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\rangle +\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\hfill \\ {\displaystyle \le \underset{v\in Q}{max}\left(\langle {\nabla }^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·)),f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v)\rangle +\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v)\right)}\hfill \\ =H\left({\tau }_j,x\left({\tau }_j\right),{\nabla }^{\alpha }\phi ({\tau }_j,x_{{\tau }_j}(·))\right).\hfill \end{array}$$

(31)

Putting together relations (30) and (31) and taking into account that the functional $\phi$ satisfies Equation (13), we arrive at the inequality $\dot{\omega }\left(\tau \right)\le \zeta /\left(2T\right)$ for a.e. $\tau \in [t,\vartheta ]$, which in turn yields (28). The lemma is proved. □

Lemma 2.

For every compact set $K\subset G$ and every $\zeta >0$, there exists $\delta >0$ such that, for any initial position $(t,w(·\left)\right)\in K$, any partition Δ of $[t,T]$ with $diam(\Delta )\le \delta$, and any first player’s control $u(·)\in \mathcal{U}\left(t\right)$, the inequality below holds:

$$J\left(t,w(·),u(·),(V^{\circ },\Delta )\right)\ge \phi (t,w(·))-\zeta .$$

In view of relation (12), the proof of Lemma 2 repeats that of Lemma 1 with clear changes and, therefore, is omitted.

Proof of Theorem 1.

Given an initial position $(t,w(·\left)\right)\in G$, it follows from Lemma 1 and inequality (10) that $\rho (t,w(·\left)\right)\le \phi (t,w(·\left)\right)$. Similarly, due to Lemma 2 and inequality (11), we derive $\rho (t,w(·\left)\right)\ge \phi (t,w(·\left)\right)$. Hence, we obtain the desired equality (16). Taking this equality into account, we conclude that Lemmas 1 and 2 imply that the players’ positional strategies $U^{\circ }$ and $V^{\circ }$ are optimal uniformly on each compact set $K\subset G$. □

It should be noted that the additional supposition in Theorem 1 that the Cauchy problem (13) and (14) has a classical solution is rather difficult to verify, and, moreover, it is often not fulfilled. This fact motivates to consider generalized solutions of this problem.

8. Minimax Solution

Under Assumptions 1 and 2, the Hamiltonian H and the boundary functional $\sigma$ in the Cauchy problem (13) and (14) satisfy requirements $(H.1)$–$(H.3)$ and $\left(\sigma \right)$ from [19]. Hence, by [19] (Theorem 6.1), the problem (13) and (14) has a unique minimax solution, which is defined as a continuous functional $\phi :G\to \mathbb{R}$ that meets boundary condition (14) and possesses the following two properties:

(i) 

For any $(t,w(·))\in G^0$, any $\vartheta \in (t,T]$, any $s\in {\mathbb{R}}^n$, and any $\eta >0$, there exists a function $y(·)\in X_{\ast }(t,w(·))$ such that

$$\phi (\vartheta ,y_{\vartheta }(·))-{\int }_t^{\vartheta }\left(\langle s,{(}^C{D}^{\alpha }y)\left(\tau \right)\rangle -H(\tau ,y\left(\tau \right),s)\right)\mathrm{d}\tau \le \phi (t,w(·))+\eta ;$$

(ii) 

For any $(t,w(·))\in G^0$, any $\vartheta \in (t,T]$, any $s\in {\mathbb{R}}^n$, and any $\eta >0$, there exists a function $y(·)\in X_{\ast }(t,w(·))$ such that

$$\phi (\vartheta ,y_{\vartheta }(·))-{\int }_t^{\vartheta }\left(\langle s,{(}^C{D}^{\alpha }y)\left(\tau \right)\rangle -H(\tau ,y\left(\tau \right),s)\right)\mathrm{d}\tau \ge \phi (t,w(·))-\eta .$$

Here, $X_{\ast }(t,w(·))$ is the set from (18), and the function $y_{\vartheta }(·)$ is determined by $y(·)$ and $\vartheta$ according to (1).

Let us note that, in view of [19] (Section 4), if the Cauchy problem (13) and (14) admits a classical solution, then it coincides with the minimax solution of this problem. On the other hand, if the minimax solution is $ci$-differentiable of the order $\alpha$ at some point $(t,w(·))\in G^0$, then it satisfies equation (13) at this point, and, consequently, if the minimax solution turns out to be $ci$-smooth of the order $\alpha$, then it is a classical solution.

In a general case, the minimax solution $\phi$ of the Cauchy problem (13) and (14) is not $ci$-differentiable of the order $\alpha$ at some points $(t,w(·))\in G^0$. Therefore, the extremal aiming procedure (15) can not be directly applied in this case. In order to overcome this difficulty and obtain the desired positional strategies of the players, we follow, e.g., [20] and [3] (Section 12.2) (see also, e.g., [17,21] for time-delay systems) and perform some “smoothing” transformation of the minimax solution $\phi$.

This transformation relies on the Lyapunov–Krasovskii functional ${\nu }_{\epsilon }$ proposed in [19]. Namely, let a compact set $K\subset G$ be given, and let the corresponding compact set $X_{\ast }\subset A{C}^{\alpha }([0,T],{\mathbb{R}}^n)$ be defined by (19). Then, according to [19] (Section 5.1.4), one can choose ${\epsilon }_0>0$ and construct continuous mappings

$${\nu }_{\epsilon }:G\to \mathbb{R},p_{\epsilon }:G\to \mathbb{R},s_{\epsilon }:G\to {\mathbb{R}}^n,\epsilon \in (0,{\epsilon }_0],$$

(32)

such that the following properties are fulfilled:

(a) 

For any $\epsilon \in (0,{\epsilon }_0]$, the functional ${\nu }_{\epsilon }$ is nonnegative, and the estimate ${\nu }_{\epsilon }(t,w(·))\le \epsilon$ holds for every $t\in [0,T]$ when $w\left(\tau \right)=0$, $\tau \in [0,t]$;

(b) 

For any $\epsilon \in (0,{\epsilon }_0]$ and any $x(·)$, $x^{\prime }(·)\in X_{\ast }$, the function $\theta \left(\tau \right)={\nu }_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))$, $\tau \in [0,T]$, is Lipschitz continuous, and, for a.e. $\tau \in [0,T]$,

$$\dot{\theta }\left(\tau \right)\le p_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))+\langle s_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·)),{(}^C{D}^{\alpha }x)\left(\tau \right)-{(}^C{D}^{\alpha }{x}^{\prime })\left(\tau \right)\rangle ;$$

(c) 

If $\epsilon \in (0,{\epsilon }_0]$ and $x(·)$, $x^{\prime }(·)\in X_{\ast }$ satisfy the equality $x\left(0\right)=x^{\prime }\left(0\right)$, then

$$\begin{array}{c}{p}_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))+H\left(\tau ,x\left(\tau \right),s_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))\right)\hfill \\ \phantom{p_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))}-H\left(\tau ,x^{\prime }\left(\tau \right),s_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))\right)\le 0,\tau \in [0,T];\hfill \end{array}$$

(d) 

For any $M>0$ and any $\mu >0$, there is ${\epsilon }_{\ast }\in (0,{\epsilon }_0]$ such that, given $\epsilon \in (0,{\epsilon }_{\ast }]$ and $x(·)$, $x^{\prime }(·)\in X_{\ast }$ meeting the conditions $x\left(0\right)=x^{\prime }\left(0\right)$ and ${\nu }_{\epsilon }(T,x(·)-x^{\prime }(·))\le M$, the inequality $|\sigma \left(x(·)\right)-\sigma \left(x^{\prime }(·)\right)|\le \mu$ is valid.

Furthermore, consider the set

$$\begin{array}{c}{G}_{\ast }=\{(t,w(·))\in G:w(·)=x_t(·),\hfill \\ \phantom{G_{\ast }=\{(t,w(·))\in G:}x(·)\in X_{\ast }(t^{\prime },w^{\prime }(·)),(t^{\prime },w^{\prime }(·))\in K,t^{\prime }\le t\},\hfill \end{array}$$

(33)

where $X_{\ast }(t^{\prime },w^{\prime }(·))$ is the set from (18), and denote

$$\begin{array}{c}{Z}_{\ast }(t,w\left(0\right))=\left\{z(·)\in A{C}^{\alpha }([0,t],{\mathbb{R}}^n):(t,z(·))\in G_{\ast },z\left(0\right)=w\left(0\right)\right\},\\ (t,w(·))\in G_{\ast }.\end{array}$$

(34)

Observe that $K\subset G_{\ast }$, and, for any $(t,w(·))\in G_{\ast }$, any $x(·)\in X_{\ast }(t,w(·))$, and any $\tau \in [t,T]$, the inclusion $(\tau ,x_{\tau }(·))\in G_{\ast }$ is fulfilled. Moreover, since the set K is compact, it follows from [22] (Theorem 2) that the set $G_{\ast }$ is compact in G, which in turn implies compactness of the set $Z_{\ast }(t,w\left(0\right))$ in $A{C}^{\alpha }([0,t],{\mathbb{R}}^n)$ for every $(t,w(·))\in G_{\ast }$.

Now, given a number $\epsilon \in (0,{\epsilon }_0]$, by the minimax solution $\phi :G\to \mathbb{R}$ of the Cauchy problem (13) and (14), let us define the functional ${\phi }_{\epsilon }^{-}:G_{\ast }\to \mathbb{R}$ according to the equality

$${\phi }_{\epsilon }^{-}(t,w(·))=\underset{z(·)\in Z_{\ast }(t,w\left(0\right))}{min}\left(\phi (t,z(·))+{\nu }_{\epsilon }(t,w(·)-z(·))\right),(t,w(·))\in G_{\ast },$$

(35)

where the minimum is attained due to continuity of the functionals $\phi$ and ${\nu }_{\epsilon }$. After that, let us choose

$$z_{\epsilon }^{-}(·\mid t,w(·))\in \underset{z(·)\in Z_{\ast }(t,w\left(0\right))}{argmin}\left(\phi (t,z(·))+{\nu }_{\epsilon }(t,w(·)-z(·))\right),(t,w(·))\in G_{\ast },$$

(36)

and, taking the functional $s_{\epsilon }$ from (32), put

$$s_{\epsilon }^{-}(t,w(·))=s_{\epsilon }\left(t,w(·)-z_{\epsilon }^{-}(·\mid t,w(·))\right),(t,w(·))\in G_{\ast }.$$

(37)

The vector $s_{\epsilon }^{-}(t,w(·))$ serves as the desired extremal direction that is used in the next section instead of ${\nabla }^{\alpha }\phi (t,w(·))$ when constructing a positional strategy of the first player by applying the extremal aiming (15).

Similarly, from the point of view of the second player, let us define

$$s_{\epsilon }^{+}(t,w(·))=s_{\epsilon }\left(t,z_{\epsilon }^{+}(·\mid t,w(·))-w(·)\right),(t,w(·))\in G_{\ast },$$

where

$$z_{\epsilon }^{+}(·\mid t,w(·))\in \underset{z(·)\in Z_{\ast }(t,w\left(0\right))}{argmax}\left(\phi (t,z(·))-{\nu }_{\epsilon }(t,z(·)-w(·))\right).$$

9. Optimal Positional Strategies in a General Case

Let $\phi :G\to \mathbb{R}$ be the minimax solution of the Cauchy problem (13) and (14), and let $K\subset G$ be a compact set. In accordance with the previous section, let us take the corresponding number ${\epsilon }_0$ and set $G_{\ast }$, and, for every $\epsilon \in (0,{\epsilon }_0]$ and every $(t,w(·))\in G_{\ast }$, determine the extremal directions $s_{\epsilon }^{-}(t,w(·))$ and $s_{\epsilon }^{+}(t,w(·))$. Then, for any $\epsilon \in (0,{\epsilon }_0]$, taking (15) into account, let us define players’ positional strategies $U_{\epsilon }^{\circ }$ and $V_{\epsilon }^{\circ }$ as follows:

$$\begin{array}{c}{\displaystyle U_{\epsilon }^{\circ }(t,w(·))\in \underset{u\in P}{argmin}\underset{v\in Q}{max}\left(\langle s_{\epsilon }^{-}(t,w(·)),f(t,w\left(t\right),u,v)\rangle +\chi (t,w\left(t\right),u,v)\right),}\\ {\displaystyle V_{\epsilon }^{\circ }(t,w(·))\in \underset{v\in Q}{argmax}\underset{u\in P}{min}\left(\langle s_{\epsilon }^{+}(t,w(·)),f(t,w\left(t\right),u,v)\rangle +\chi (t,w\left(t\right),u,v)\right),}\\ (t,w(·))\in G_{\ast },\end{array}$$

(38)

and, formally, let us choose $U_{\epsilon }^{\circ }(t,w(·))\in P$ and $V_{\epsilon }^{\circ }(t,w(·))\in Q$ for $(t,w(·))\in G\setminus G_{\ast }$ arbitrarily.

Theorem 2.

Let Assumptions 1, 2, and 3 be fulfilled. Then, the minimax solution $\phi :G\to \mathbb{R}$ of the Cauchy problem (13) and (14) coincides with the value functional of the differential game (2) and (5), and, given a compact set $K\subset G$ and a number $\zeta >0$, one can find ${\epsilon }^{\ast }\in (0,{\epsilon }_0]$ such that, for any $\epsilon \in (0,{\epsilon }^{\ast }]$, the players’ positional strategies $U_{\epsilon }^{\circ }$ and $V_{\epsilon }^{\circ }$ defined by the minimax solution φ and the set K according to (38) are ζ-optimal uniformly on this set K.

The proof follows the scheme from, e.g., [20] and [3] (Theorem 12.3) (see also, e.g., [21] (Theorem 1) for time-delay systems). Let us establish two lemmas below, which are valid under the assumptions of the theorem.

Lemma 3.

For every compact set $K\subset G$ and every $\zeta >0$, there exist a number ${\epsilon }^{\ast }\in (0,{\epsilon }_0]$ and a function $(0,{\epsilon }^{\ast }]\ni \epsilon \mapsto {\delta }^{\ast }\left(\epsilon \right)\in (0,\infty )$ such that, for any $\epsilon \in (0,{\epsilon }^{\ast }]$, any initial position $(t,w(·\left)\right)\in K$, any partition Δ of $[t,T]$ with $diam(\Delta )\le {\delta }^{\ast }\left(\epsilon \right)$, and any second player’s control $v(·)\in \mathcal{V}\left(t\right)$, the inequality below holds:

$$J\left(t,w(·),(U_{\epsilon }^{\circ },\Delta ),v(·)\right)\le \phi (t,w(·))+\zeta .$$

(39)

Proof. 

By a fixed compact set $K\subset G$, define the compact set $X_{\ast }\subset A{C}^{\alpha }([0,T],{\mathbb{R}}^n)$, numbers $R_{\ast }$ and $R^{\ast }$, and modulus of continuity $m^{\ast }(·)$ by (19), (20), and (22). Since the functional $\sigma$ is continuous, one can find $M_{\ast }>0$ such that

$$\left|\sigma \left(x(·)\right)\right|\le M_{\ast },x(·)\in X_{\ast }.$$

(40)

Let $\zeta >0$ be fixed. By the numbers $M=2M_{\ast }+\zeta /3$ and $\mu =\zeta /3$, let us choose the number ${\epsilon }_{\ast }$ according to property $\left(d\right)$ (see Section 8) and then put ${\epsilon }^{\ast }=min\{\zeta /3,{\epsilon }_{\ast }\}$. Now, let $\epsilon \in (0,{\epsilon }^{\ast }]$ be given. Due to continuity of the mappings $p_{\epsilon }$ and $s_{\epsilon }$ from (32), there exist a number $R_{\epsilon }^{\ast }>0$ and a modulus of continuity $m_{\epsilon }^{\ast }(·)$ such that

$$\begin{array}{c}|p_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))-p_{\epsilon }({\tau }^{\prime },x_{{\tau }^{\prime }}(·)-x_{{\tau }^{\prime }}^{\prime }(·))|\le m_{\epsilon }^{\ast }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \parallel s_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))\parallel \le R_{\epsilon }^{\ast },\\ \parallel s_{\epsilon }(\tau ,x_{\tau }(·)-x_{\tau }^{\prime }(·))-s_{\epsilon }({\tau }^{\prime },x_{{\tau }^{\prime }}(·)-x_{{\tau }^{\prime }}^{\prime }(·))\parallel \le m_{\epsilon }^{\ast }\left(\right|\tau -{\tau }^{\prime }\left|\right),\\ \tau ,{\tau }^{\prime }\in [0,T],x(·),x^{\prime }(·)\in X_{\ast }.\end{array}$$

(41)

Finally, let us choose a number ${\delta }^{\ast }\left(\epsilon \right)>0$ satisfying the inequality

$$(2+2R_{\epsilon }^{\ast })m^{\ast }\left({\delta }^{\ast }\left(\epsilon \right)\right)+(1+2R^{\ast })m_{\epsilon }^{\ast }\left({\delta }^{\ast }\left(\epsilon \right)\right)\le \zeta /\left(6T\right).$$

(42)

Let us show that the conclusion of the lemma holds for the specified number ${\epsilon }^{\ast }$ and function ${\delta }^{\ast }(·)$.

Take arbitrarily $\epsilon \in (0,{\epsilon }^{\ast }]$, an initial position $(t,w(·\left)\right)\in K$, and a partition $\Delta$ of $[t,T]$ with $diam(\Delta )\le {\delta }_{\ast }\left(\epsilon \right)$. Consider a motion $x(·)$ of system (2) generated by the first player’s control law $(U_{\epsilon }^{\circ },\Delta )$ and a second player’s control $v(·)\in \mathcal{V}\left(t\right)$. Let $u(·)\in \mathcal{U}\left(t\right)$ be the corresponding first player’s control. Observe that, due to (3), (18), (19), and (33), the inclusions $x(·)\in X_{\ast }$ and $(\tau ,x_{\tau }(·))\in G_{\ast }$, $\tau \in [t,T]$, are fulfilled.

Following the lines of the proof of Lemma 1, let us introduce the function

$$\omega \left(\tau \right)={\phi }_{\epsilon }^{-}(\tau ,x_{\tau }(·))+{\int }_t^{\tau }\chi (\xi ,x\left(\xi \right),u\left(\xi \right),v\left(\xi \right))\mathrm{d}\xi ,\tau \in [t,T],$$

(43)

where ${\phi }_{\epsilon }^{-}$ is the functional from (35). Let us first estimate the value $\omega \left(t\right)$ from above. Since $w(·)\in Z_{\ast }(t,w\left(0\right))$ according to (34) and ${\nu }_{\epsilon }(t,w(·)-w(·))\le \epsilon \le {\epsilon }^{\ast }\le \zeta /3$ in view of property $\left(a\right)$ and the definition of ${\epsilon }^{\ast }$, we have

$$\omega \left(t\right)={\phi }_{\epsilon }^{-}(t,w(·))\le \phi (t,w(·))+{\nu }_{\epsilon }(t,w(·)-w(·))\le \phi (t,w(·))+\zeta /3.$$

(44)

Now, let us estimate the value $\omega \left(T\right)$ from below. Note that $x(·)\in Z_{\ast }(T,x\left(0\right))$. Hence, arguing by analogy with (44) and after that applying (14) and (40), we derive

$${\phi }_{\epsilon }^{-}(T,x(·))\le \sigma \left(x(·)\right)+{\nu }_{\epsilon }(T,x(·)-x(·))\le M_{\ast }+\zeta /3.$$

(45)

On the other hand, taking into account the choice (36) of the function $z_{\epsilon }^{-}(·\mid T,x(·))$ and the inclusion $z_{\epsilon }^{-}(·\mid T,x(·))\in X_{\ast }$, we obtain

$$\begin{array}{c}{\phi }_{\epsilon }^{-}(T,x(·))=\sigma \left(z_{\epsilon }^{-}(·\mid T,x(·))\right)+{\nu }_{\epsilon }\left(T,x(·)-z_{\epsilon }^{-}(·\mid T,x(·))\right)\hfill \\ \phantom{{\phi }_{\epsilon }^{-}(T,x(·))}\ge -M_{\ast }+{\nu }_{\epsilon }\left(T,x(·)-z_{\epsilon }^{-}(·\mid T,x(·))\right).\hfill \end{array}$$

(46)

It follows from (45) and (46) that

$${\nu }_{\epsilon }\left(T,x(·)-z_{\epsilon }^{-}(·\mid T,x(·))\right)\le 2M_{\ast }+\zeta /3.$$

Therefore, in accordance with the equality $z_{\epsilon }^{-}(0\mid T,x(·))=x\left(0\right)$, which is valid by the definition (34) of the set $Z_{\ast }(T,x\left(0\right))$, the choice of ${\epsilon }_{\ast }$ yields

$$\sigma \left(z_{\epsilon }^{-}(·\mid T,x(·))\right)\ge \sigma \left(x(·)\right)-\zeta /3.$$

Thus, based on (46), since the functional ${\nu }_{\epsilon }$ is nonnegative due to property $\left(a\right)$, we conclude that

$${\phi }_{\epsilon }^{-}(T,x(·))\ge \sigma \left(x(·)\right)-\zeta /3.$$

Consequently, in view of (5), we get

$$\omega \left(T\right)\ge J\left(t,w(·),(U_{\epsilon }^{\circ },\Delta ),v(·)\right)-\zeta /3.$$

(47)

Relations (44) and (47) imply that, in order to obtain the desired inequality (39), it suffices to verify that $\omega \left(T\right)-\omega \left(t\right)\le \zeta /3$. If $t=T$, this estimate obviously holds, so let us prove it in the case when $t<T$. To this end, let us show that, for every $j\in \overline{1,k-1}$, the inequality below is valid:

$$\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)\le \zeta ({\tau }_{j+1}-{\tau }_j)/\left(3T\right).$$

(48)

For brevity, let us denote (see (36) and (37))

$$z^{\ast }(·)=z_{\epsilon }^{-}(·\mid {\tau }_j,x_{{\tau }_j}(·)),s^{\ast }=s_{\epsilon }^{-}({\tau }_j,x_{{\tau }_j}(·))=s_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-z^{\ast }(·)).$$

By the definition (43) of the function $\omega (·)$, we have

$$\begin{array}{c}\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)={\phi }_{\epsilon }^{-}({\tau }_{j+1},x_{{\tau }_{j+1}}(·))-{\phi }_{\epsilon }^{-}({\tau }_j,x_{{\tau }_j}(·))\hfill \\ {\displaystyle \phantom{\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)}+{\int }_{{\tau }_j}^{{\tau }_{j+1}}\chi (\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\mathrm{d}\tau .}\hfill \end{array}$$

(49)

Furthermore, due to the definition (35) of the functional ${\phi }_{\epsilon }^{-}$, we get

$${\phi }_{\epsilon }^{-}({\tau }_j,x_{{\tau }_j}(·))=\phi ({\tau }_j,z^{\ast }(·))+{\nu }_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-z^{\ast }(·)).$$

(50)

Taking into account that the minimax solution $\phi$ possesses property $\left(i\right)$, let us choose a function $y^{\ast }(·)\in X_{\ast }({\tau }_j,z^{\ast }(·))$ from the condition

$$\begin{array}{c}{\displaystyle \phi ({\tau }_{j+1},y_{{\tau }_{j+1}}^{\ast }(·))-{\int }_{{\tau }_j}^{{\tau }_{j+1}}\left(\langle s^{\ast },{(}^C{D}^{\alpha }{y}^{\ast })\left(\tau \right)\rangle -H(\tau ,y^{\ast }\left(\tau \right),s^{\ast })\right)\mathrm{d}\tau }\hfill \\ \le \phi ({\tau }_j,z^{\ast }(·))+\zeta ({\tau }_{j+1}-{\tau }_j)/\left(6T\right).\hfill \end{array}$$

(51)

Since $({\tau }_j,z^{\ast }(·))\in G_{\ast }$ and $z^{\ast }\left(0\right)=x\left(0\right)$, we obtain that $y^{\ast }(·)\in X_{\ast }$, $({\tau }_{j+1},y_{{\tau }_{j+1}}^{\ast }(·))\in G_{\ast }$, and $y^{\ast }\left(0\right)=x\left(0\right)$. In particular, we derive $y_{{\tau }_{j+1}}^{\ast }(·)\in Z_{\ast }({\tau }_{j+1},x_{{\tau }_{j+1}}\left(0\right))$, and, therefore,

$${\phi }_{\epsilon }^{-}({\tau }_{j+1},x_{{\tau }_{j+1}}(·))\le \phi ({\tau }_{j+1},y_{{\tau }_{j+1}}^{\ast }(·))+{\nu }_{\epsilon }\left({\tau }_{j+1},x_{{\tau }_{j+1}}(·)-y_{{\tau }_{j+1}}^{\ast }(·)\right).$$

(52)

Thus, relations (49)–(52) imply that

$$\begin{array}{c}{\displaystyle \omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)\le {\int }_{{\tau }_j}^{{\tau }_{j+1}}\left(\langle s^{\ast },{(}^C{D}^{\alpha }{y}^{\ast })\left(\tau \right)\rangle -H(\tau ,y^{\ast }\left(\tau \right),s^{\ast })\right)\mathrm{d}\tau }\hfill \\ \phantom{\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)}+{\nu }_{\epsilon }\left({\tau }_{j+1},x_{{\tau }_{j+1}}(·)-y_{{\tau }_{j+1}}^{\ast }(·)\right)-{\nu }_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-z^{\ast }(·))\hfill \\ {\displaystyle \phantom{\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)}+{\int }_{{\tau }_j}^{{\tau }_{j+1}}\chi (\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\mathrm{d}\tau +\zeta ({\tau }_{j+1}-{\tau }_j)/\left(6T\right).}\hfill \end{array}$$

(53)

Let us estimate the values from the right-hand side of (53) from above. In view of the definition (12) of the Hamiltonian H and inequalities (22) and (41), we have

$$H(\tau ,y^{\ast }\left(\tau \right),s^{\ast })\le H({\tau }_j,y^{\ast }\left({\tau }_j\right),s^{\ast })+(1+R_{\epsilon }^{\ast })m^{\ast }(\tau -{\tau }_j),\tau \in [{\tau }_j,{\tau }_{j+1}].$$

(54)

Now, let us consider the auxiliary function

$$\theta \left(\tau \right)={\nu }_{\epsilon }(\tau ,x_{\tau }(·)-y_{\tau }^{\ast }(·))+{\int }_{{\tau }_j}^{\tau }\chi (\xi ,x\left(\xi \right),u\left(\xi \right),v\left(\xi \right))\mathrm{d}\xi ,\tau \in [{\tau }_j,{\tau }_{j+1}].$$

Observe that

$$\begin{array}{c}{\displaystyle {\nu }_{\epsilon }\left({\tau }_{j+1},x_{{\tau }_{j+1}}(·)-y_{{\tau }_{j+1}}^{\ast }(·)\right)-{\nu }_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-z^{\ast }(·))}\hfill \\ {\displaystyle +{\int }_{{\tau }_j}^{{\tau }_{j+1}}\chi (\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\mathrm{d}\tau =\theta \left({\tau }_{j+1}\right)-\theta \left({\tau }_j\right).}\hfill \end{array}$$

(55)

Moreover, property $\left(b\right)$ yields that the function $\theta (·)$ is Lipschitz continuous, and

$$\begin{array}{c}\dot{\theta }\left(\tau \right)\le p_{\epsilon }(\tau ,x_{\tau }(·)-y_{\tau }^{\ast }(·))\hfill \\ \phantom{\dot{\theta }\left(\tau \right)}+\langle s_{\epsilon }(\tau ,x_{\tau }(·)-y_{\tau }^{\ast }(·)),f(\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))-{(}^C{D}^{\alpha }{y}^{\ast })\left(\tau \right)\rangle \hfill \\ \phantom{\dot{\theta }\left(\tau \right)}+\chi (\tau ,x\left(\tau \right),u\left(\tau \right),v\left(\tau \right))\mathrm{for}\mathrm{a}.\mathrm{e}.\tau \in [{\tau }_j,{\tau }_{j+1}].\hfill \end{array}$$

Then, applying (18), (20)–(22), and (41), for a.e. $\tau \in [{\tau }_j,{\tau }_{j+1}]$, we get (see also (30))

$$\begin{array}{c}\dot{\theta }\left(\tau \right)\le p_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-y_{{\tau }_j}^{\ast }(·))+\langle s^{\ast },f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))-{(}^C{D}^{\alpha }{y}^{\ast })\left(\tau \right)\rangle \hfill \\ \phantom{\varkappa \left(\tau \right)}+\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))+(1+2R^{\ast })m_{\epsilon }^{\ast }(\tau -{\tau }_j)+(1+R_{\epsilon }^{\ast })m^{\ast }(\tau -{\tau }_j).\hfill \end{array}$$

(56)

According to (9), we have $u\left(\tau \right)=U_{\epsilon }^{\circ }({\tau }_j,x_{{\tau }_j}(·))$, $\tau \in [{\tau }_j,{\tau }_{j+1})$. Hence, if follows from the definition (38) of the strategy $U_{\epsilon }^{\circ }$ and the definition (12) of the Hamiltonian H that

$$\begin{array}{c}\langle s^{\ast },f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\rangle +\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v\left(\tau \right))\hfill \\ {\displaystyle \le \underset{v\in Q}{max}\left(\langle s^{\ast },f({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v)\rangle +\chi ({\tau }_j,x\left({\tau }_j\right),u\left(\tau \right),v)\right)}\hfill \\ =H({\tau }_j,x\left({\tau }_j\right),s^{\ast }),\tau \in [{\tau }_j,{\tau }_{j+1}).\hfill \end{array}$$

(57)

Finally, owing to property $\left(c\right)$, we obtain

$$p_{\epsilon }({\tau }_j,x_{{\tau }_j}(·)-y_{{\tau }_j}^{\ast }(·))+H({\tau }_j,x\left({\tau }_j\right),s^{\ast })-H({\tau }_j,y^{\ast }\left({\tau }_j\right),s^{\ast })\le 0.$$

(58)

Thus, putting together relations (53)–(58), we arrive at the inequality

$$\omega \left({\tau }_{j+1}\right)-\omega \left({\tau }_j\right)\le \left(\zeta /\left(6T\right)+(2+2R_{\epsilon }^{\ast })m^{\ast }\left({\delta }^{\ast }\left(\epsilon \right)\right)+(1+2R^{\ast })m_{\epsilon }^{\ast }\left({\delta }^{\ast }\left(\epsilon \right)\right)\right)({\tau }_{j+1}-{\tau }_j),$$

which yields (48) by virtue of the choice (42) of ${\delta }^{\ast }\left(\epsilon \right)$. The lemma is proved. □

Lemma 4.

For every compact set $K\subset G$ and every $\zeta >0$, there exist a number ${\epsilon }^{\ast }\in (0,{\epsilon }_0]$ and a function $(0,{\epsilon }^{\ast }]\ni \epsilon \mapsto {\delta }^{\ast }\left(\epsilon \right)\in (0,\infty )$ such that, for any $\epsilon \in (0,{\epsilon }^{\ast }]$, any initial position $(t,w(·\left)\right)\in K$, any partition Δ of $[t,T]$ with $diam(\Delta )\le {\delta }^{\ast }\left(\epsilon \right)$, and any first player’s control $u(·)\in \mathcal{U}\left(t\right)$, the inequality below holds:

$$J\left(t,w(·),u(·),(V_{\epsilon }^{\circ },\Delta )\right)\ge \phi (t,w(·))-\zeta .$$

The proof of Lemma 4 repeats that of Lemma 3 with clear changes in view of relation (12) and property $\left(ii\right)$ of the minimax solution $\phi$.

Thus, it remains to observe that Lemmas 3 and 4 imply Theorem 2 by essentially the same arguments as in the proof of Theorem 1.

Remark 1.

One can show that Theorems 1 and 2 are valid if, instead of Assumption 3, we assume only that relation (12) holds. In this case, the existence of the value of the differential game (2) and (5) does not follow from [15] but can be proved based on Lemmas 3 and 4 by the scheme from, e.g., [14] (Theorem 1).

10. Conclusions

In the paper, a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order $\alpha \in (0,1)$ and a Bolza-type cost functional has been studied. The Cauchy problem for the associated Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order $\alpha$ and the natural boundary condition has been considered. It has been proved that the value functional of the differential game coincides with the unique generalized minimax solution of the given Cauchy problem, and a new way of constructing optimal positional strategies of the players has been proposed. Moreover, a particular case has been investigated in detail when it is additionally assumed that the Cauchy problem admits a classical solution.

Despite the fact that the results obtained in the paper are of a theoretical nature, they may serve in the future as a necessary basis for constructing solutions of specific differential games for fractional-order systems.