Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

: The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α ∈ ( 0,1 ) and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufﬁciently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.


Introduction
The paper follows the positional approach (see, e.g., [1][2][3]) and is concerned with the questions of how to characterize the value functional and construct optimal positional (feedback) strategies of the players in a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α ∈ (0, 1) and a given Bolza-type cost functional. For the basics of fractional calculus and the theory of fractional differential equations, the reader is referred to, e.g., [4][5][6]. Investigations of some other kinds of differential games for fractional-order systems can be found in, e.g., [7][8][9][10][11][12][13].
The differential game under consideration was previously studied in [14,15]. More precisely, in [14], based on a suitable approximation of the game by a differential game for a (first-order) time-delay system, it was proved that the original game has the value, and, moreover, optimal positional strategies of the players were obtained in the form of control procedures that use the approximating time-delay system as a guide (see, e.g., [1] (Section 8.2)). In [15], the characteristic uand v-stability properties of the value functional were established, and optimal positional strategies of the players were built by a suitable modification of the method of extremal shift to accompanying points (see, e.g., [2] (Section 8)). It should be noted that the results obtained in [14,15] demonstrate a close relationship between positional differential games for fractional-order systems and that for time-delay systems.
The present paper continues these studies and proposes another way of constructing optimal positional strategies of the players, which is based on a different technique. Motivated by [16], the differential game under consideration is associated to the Cauchy problem for the corresponding Hamilton-Jacobi-Bellman-Isaacs equation with fractional coinvariant (ci-) derivatives of the order α and the natural boundary condition. Then, at a first step, a particular case is studied when it is additionally assumed that this Cauchy problem admits a sufficiently smooth (namely, ci-smooth of the order α) solution. Using standard arguments (see, e.g., [3] (Section 11.5) and also [17] for time-delay systems), it is proved that this solution coincides with the value functional of the original differential game, and, furthermore, optimal positional strategies of the players can be obtained by applying the extremal aiming in the direction of the ci-gradient of the order α of this solution.
In a general non-smooth case, following, e.g., [3], it is shown that the value functional coincides with a generalized minimax solution of the given Cauchy problem [18,19]. The proof of this fact reduces to construction of optimal positional strategies of the players on the basis of this minimax solution. To this end, by analogy with, e.g., [20] and [3] (Section 12.2) (see also, e.g., [17,21] for time-delay systems), some "smoothing" transformation of the minimax solution is performed, which makes it possible to determine the desired extremal directions that can be used in the extremal aiming procedure instead of the ci-gradient of the order α (since it may fail to exist for the minimax solution). It should be emphasized that peculiarities inherent in fractional-order systems are handled with the help of a suitable choice of a Lyapunov-Krasovskii functional [19], which the transformation relies on.
The rest of paper is organized as follows. In Section 2, basic notation is given. The dynamical system of a fractional order and cost functional are described in Section 3. In Section 4, the value functional of the differential game is defined in terms of non-anticipative strategies of the players. In Section 5, positional strategies of the players are introduced. The Cauchy problem for the Hamilton-Jacobi-Bellman-Isaacs equation is formulated in Section 6. Section 7 deals with the case when the Cauchy problem has a smooth solution. In Section 8, the generalized minimax solution of this problem is considered, and its "smoothing" transformation is performed. Section 9 is devoted to the general non-smooth case. Concluding remarks are presented in Section 10.

Notation
Let T > 0, n ∈ N, and α ∈ (0, 1) be fixed throughout the paper. By · and ·, · , the Euclidean norm and inner product in R n are denoted. Given R ≥ 0, let B(R) stand for the closed ball in R n centered at the origin of radius R.

Dynamical System of Fractional Order and Cost Functional
The paper deals with a two-person zero-sum differential game in which the dynamical system is described by the fractional differential equation where τ is time, x(τ) is the current state of the system, ( C D α x)(τ) is the Caputo derivative of the order α, u(τ) and v(τ) are the current controls of the first and second players, respectively, P ⊂ R n P and Q ⊂ R n Q are compact sets, n P , n Q ∈ N. Assumption 1. The function f : [0, T] × R n × P × Q → R n is continuous and possesses the following two properties: for every R ≥ 0, one can choose a number λ > 0 such that and there exists a constant c > 0 such that As an initial data for system (2), we consider a pair (t, w(·)) ∈ G, which is called an initial position of the system (see, e.g., [14][15][16]). Here, t is an initial time, and the function w(·) is treated as an initial history of a system's motion. Thus, in view of notation (1), the initial condition takes the form x t (·) = w(·). By admissible (open-loop) controls of the first and second players on the time interval [t, T], we mean any measurable functions u : [t, T] → P and v : [t, T] → Q, respectively. Let U (t) and V (t) be the corresponding sets of all such controls. Denote According to, e.g., [14] (Proposition 2), from the initial position (t, w(·)), every pair of controls u(·) ∈ U (t) and v(·) ∈ V (t) generate a unique motion of system (2), which is defined as a function x(·) ∈ X(t, w(·)) that, together with u(·) and v(·), satisfies the fractional differential equation in (2) for a.e. τ ∈ [t, T].
For system (2), a differential game is studied in which the first player wants to minimize, and the second player wants to maximize the cost functional where x(·) is the motion of system (2) generated from the initial position (t, w(·)) by the players' controls u(·) and v(·).

Value Functional
The value functional of the differential game (2) and (5) is defined with the help of non-anticipative strategies of the players.
Let an initial position (t, w(·)) ∈ G be fixed. By a non-anticipative strategy of the first player, we mean a mapping κ : V (t) → U (t) possessing the following property: for any t ∈ [t, T] and any second player's controls Then, the lower value of the differential game is given by where the infimum is calculated over all first player's non-anticipative strategies κ. Similarly, a second player's non-anticipative strategy is a mapping β : and v (·) = β(u (·)). Respectively, the upper value of the differential game is given by where the supremum is calculated over all second player's non-anticipative strategies β. Let us suppose that, in addition to Assumptions 1 and 2, the following Isaacs' condition, also known as the saddle point condition in a small game, is fulfilled.
It should be noted that non-anticipative strategies, being a convenient tool in theoretical considerations, are rather difficult to implement. In this regard, the paper focuses on construction of optimal positional (feedback) strategies of the players, which are more acceptable from a practical point of view.

Positional Strategies
In accordance with [14,15], as a first player's positional strategy in the differential game (2) and (5), we consider an arbitrary mapping U : G → P. Let (t, w(·)) ∈ G be an initial position, and let ∆ be a partition of the time interval [t, T], i.e., where k ∈ N. Here, and below, we use the notation 1, k = {j ∈ N : j ≤ k}. The pair (U, ∆) is called a control law of the first player. From the initial position (t, w(·)), this law together with a second player's control v(·) ∈ V (t) uniquely generate the first player's control u(·) ∈ U (t) (and, respectively, the corresponding motion x(·) of system (2)) by the following step-by-step rule: and, formally, u(T) = U(T, x(·)). In other words, at every time τ j , j ∈ 1, k − 1, the first player measures the history (1)), computes the value u j = U(τ j , x τ j (·)), and then applies the constant control u(τ) = u j until τ j+1 , when a new measurement of the history is taken. Let us denote the corresponding value of cost functional (5) by J(t, w(·), (U, ∆), v(·)). Observe that, by construction, the mapping that assigns to each second player's control v(·) ∈ V (t) the first player's control u(·) ∈ U (t) formed by (U, ∆) is a non-anticipative strategy of the first player. Hence, in view of (6) and (8), we have In this connection, given a set K ⊂ G and a number ζ > 0, a positional strategy of the first player U is called ζ-optimal uniformly on K if the following property holds: there exists δ > 0 such that, for any initial position (t, w(·)) ∈ K and any partition ∆ takes place. Respectively, we say that the strategy U is optimal uniformly on the set K if it is ζ-optimal uniformly on this set for every ζ > 0.
Then, for a set K ⊂ G and a number ζ > 0, a positional strategy of the second player V is called ζ-optimal uniformly on K if there exists δ > 0 such that the inequality inf u(·)∈U (t) J t, w(·), u(·), (V, ∆) ≥ ρ(t, w(·)) − ζ is fulfilled for any (t, w(·)) ∈ K and any partition ∆ of [t, T] with diam(∆) ≤ δ. We say that V is optimal uniformly on K if it is ζ-optimal uniformly on K for every ζ > 0.
The unknown in problem (13) and (14) is a functional ϕ : G → R. The goal of the paper is to establish a relationship between the differential game (2) and (5) and the Cauchy problem (13) and (14) with a particular emphasis on construction of optimal and ζ-optimal positional control strategies of the players. Let us first study this question under an additional supposition that the Cauchy problem (13) and (14) admits a sufficiently smooth solution.

Optimal Positional Strategies in the Smooth Case
Following [16], we say that a functional ϕ : G → R is ci-smooth of the order α if it is continuous, ci-differentiable of the order α at every point (t, w(·)) ∈ G 0 , and the mappings ∂ α t ϕ : G 0 → R and ∇ α ϕ : G 0 → R n are continuous. If such a functional ϕ satisfies the Hamilton-Jacobi-Bellman-Isaacs equation (13) and boundary condition (14), then it is called a classical solution of the Cauchy problem (13) and (14).
The proof of the theorem follows the scheme from, e.g., [3] (Section 11.5) (see also [17] (Theorem 3.1) for time-delay systems), and, for convenience, it is divided into the two lemmas below, which are valid under the assumptions of the theorem.

Lemma 1.
For every compact set K ⊂ G and every ζ > 0, there exists δ > 0 such that, for any initial position (t, w(·)) ∈ K, any partition ∆ of [t, T] with diam(∆) ≤ δ, and any second player's control v(·) ∈ V (t), the inequality below holds: Before proceeding to the proof of the lemma, let us carry out some auxiliary constructions, which are also needed in the subsequent sections. Denote where c is the constant from (3) and X(t, w(·)) is the set from (4). Let a compact set K ⊂ G be given. Consider the set X * = x(·) ∈ AC α ([0, T], R n ) : x(·) ∈ X * (t, w(·)), (t, w(·)) ∈ K .
According to [22] (Theorem 2), compactness of the set K implies compactness of the set X * in AC α ([0, T], R n ). In particular, by the Arzelà-Ascoli theorem, all functions x(·) ∈ X * are uniformly bounded and equicontinuous. Let a number R * > 0 be such that Then, in view of (3), the following estimate is valid: Furthermore, due to continuity of the functions f and χ, let us take a number R * > 0 and a modulus of continuity m * (·) such that Proof of Lemma 1. Let a compact set K ⊂ G and ζ > 0 be fixed. Define the compact set X * ⊂ AC α ([0, T], R n ), number R * , and modulus of continuity m * (·) as above. Since the functional ϕ is continuous, one can choose ϑ ∈ [0, T) from the condition Then, owing to continuity of the mappings ∂ α t ϕ and ∇ α ϕ, there are a number R ϑ > 0 and a modulus of continuity m ϑ (·) such that Let us show that the conclusion of the lemma holds for δ > 0 satisfying the inequality Take an initial position (t, w(·)) ∈ K and a partition ∆ of [t, T] with diam(∆) ≤ δ. Consider a motion x(·) of system (2) generated by the first player's control law (U • , ∆) and a second player's control v(·) ∈ V (t). Let u(·) ∈ U (t) be the corresponding first player's control. Observe that the inclusion x(·) ∈ X * is fulfilled by (3), (18), and (19).

Lemma 2.
For every compact set K ⊂ G and every ζ > 0, there exists δ > 0 such that, for any initial position (t, w(·)) ∈ K, any partition ∆ of [t, T] with diam(∆) ≤ δ, and any first player's control u(·) ∈ U (t), the inequality below holds: In view of relation (12), the proof of Lemma 2 repeats that of Lemma 1 with clear changes and, therefore, is omitted.
It should be noted that the additional supposition in Theorem 1 that the Cauchy problem (13) and (14) has a classical solution is rather difficult to verify, and, moreover, it is often not fulfilled. This fact motivates to consider generalized solutions of this problem.

Minimax Solution
Under Assumptions 1 and 2, the Hamiltonian H and the boundary functional σ in the Cauchy problem (13) and (14) satisfy requirements (H.1)-(H.3) and (σ) from [19]. Hence, by [19] (Theorem 6.1), the problem (13) and (14) has a unique minimax solution, which is defined as a continuous functional ϕ : G → R that meets boundary condition (14) and possesses the following two properties: (i) For any (t, w(·)) ∈ G 0 , any ϑ ∈ (t, T], any s ∈ R n , and any η > 0, there exists a function y(·) ∈ X * (t, w(·)) such that (ii) For any (t, w(·)) ∈ G 0 , any ϑ ∈ (t, T], any s ∈ R n , and any η > 0, there exists a function y(·) ∈ X * (t, w(·)) such that Here, X * (t, w(·)) is the set from (18), and the function y ϑ (·) is determined by y(·) and ϑ according to (1). Let us note that, in view of [19] (Section 4), if the Cauchy problem (13) and (14) admits a classical solution, then it coincides with the minimax solution of this problem. On the other hand, if the minimax solution is ci-differentiable of the order α at some point (t, w(·)) ∈ G 0 , then it satisfies equation (13) at this point, and, consequently, if the minimax solution turns out to be ci-smooth of the order α, then it is a classical solution.
In a general case, the minimax solution ϕ of the Cauchy problem (13) and (14) is not ci-differentiable of the order α at some points (t, w(·)) ∈ G 0 . Therefore, the extremal aiming procedure (15) can not be directly applied in this case. In order to overcome this difficulty and obtain the desired positional strategies of the players, we follow, e.g., [20] and [3] (Section 12.2) (see also, e.g., [17,21] for time-delay systems) and perform some "smoothing" transformation of the minimax solution ϕ.
The vector s − ε (t, w(·)) serves as the desired extremal direction that is used in the next section instead of ∇ α ϕ(t, w(·)) when constructing a positional strategy of the first player by applying the extremal aiming (15).

Conclusions
In the paper, a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α ∈ (0, 1) and a Bolza-type cost functional has been studied. The Cauchy problem for the associated Hamilton-Jacobi-Bellman-Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition has been considered. It has been proved that the value functional of the differential game coincides with the unique generalized minimax solution of the given Cauchy problem, and a new way of constructing optimal positional strategies of the players has been proposed. Moreover, a particular case has been investigated in detail when it is additionally assumed that the Cauchy problem admits a classical solution.
Despite the fact that the results obtained in the paper are of a theoretical nature, they may serve in the future as a necessary basis for constructing solutions of specific differential games for fractional-order systems.