An Analytic and Numerical Investigation of a Differential Game

: In this paper we present an appropriate singular, zero-sum, linear-quadratic differential game. One of the main features of this game is that the weight matrix of the minimizer’s control cost in the cost functional is singular. Due to this singularity, the game cannot be solved either by applying the Isaacs MinMax principle, or the Bellman–Isaacs equation approach. As an application, we introduced an interception differential game with an appropriate regularized cost functional and developed an appropriate dual representation. By developing the variational derivatives of this regularized cost functional, we apply Popov’s approximation method and show how the numerical results coincide with the dual representation.


Introduction
In this paper, we present a zero-sum differential game with linear dynamics and a quadratic cost functional. Such games appear in many areas of control theory, for example, robust controllability [1], pursuit-evasion [2][3][4][5], robust tracking [6] and robust investment [7], to name but a few.
The singularity of such a game is caused by the weight matrix of the minimizer's control cost, meaning that problem of minimization its variational Hamiltonian with respect to the minimizer control has either infinitely many solutions or no solutions. This makes the game challenging, since it can not be solved by some well-known approaches, such as the Isaacs min-max principle [8] and the Bellman-Isaacs equation [8][9][10][11].
Known techniques in the literature involve high order optimality conditions, but their practical usage is quite limited and not general enough [2,5]. Regularization approaches with additional assumption have been studied by several researchers, such as [12]. Other related results include [13][14][15].
In [16] the differential game is tackled by introducing a cost functional containing the minimizer's control cost. A regularization approach is then considered, yielding an auxiliary differential game with partial cheap control of the minimizer. While differential games with total cheap control of at least one of the players has been studied enough already-see, e.g., [1,6,[17][18][19][20]-partial cheap control of at least one of the players has been studied by only a few [16].
In the recent work of the authors [21], a saddle-point reformulation of the zero-sum singular differential game was studied, and two gradient methods were presented and analyzed. This work considering a slightly more general game than [16] and a pursuitevasion game illustrates the applicability of the numerical methods.
Following the above and as a continuing work of [21], the objectives of this work are as follows: Such regular cheap/partial cheap control games were analyzed in [12,16,22,23] using the study of the Riccati matrix differential equation for a finite-horizon game, and the Riccati matrix algebraic equation fora n infinite-horizon game.
In this work we study an interception game, which is a special regular differential game, in order to minimize with respect to u and maximize with respect to v the associated regularized cost functional. This then suggested exploring the equivalent saddle-point reformulation.
Consider the Hilbert space L 2 [0, t f ] and let Q ⊆ L 2 [0, t f ] and R ⊆ L 2 [0, t f ] be two closed, convex and bounded sets of admissible controls. Then, solving (1)-(4) is equivalent to solving the following min-max problem.
where J ε is continuous, convex-concave (convex in u and concave in v) and differentiable. See [24] for further details. A saddle-point reformulation of the min-max problem (7) is formulated as finding a for all v ∈ R and u ∈ Q. Another known relationship with the above assumptions on J ε is the following variational inequality reformulation. The saddle-point problem (8) where ·, · is an appropriate inner product and δJ ε /(δu) and δJ ε /(δv) are the variational derivatives of the functional J ε , as will be explained later (can be thought of as partial derivatives in the case of real functions).
These methods use gradients in each of their update rules. Thus we recall the variational/functional derivative definition next (see, for example, [33]).

Definition 3.
Consider an integral functional J(x(t)) of an argument x(t). The variational/functional derivative of J(x(t)) with respect to x(t), δJ δx : [0, T] → R n is defined as Recall that for a function f : R n → R, the gradient ∇ f is defined by Thus the variational derivative works similarly as such; that is, and with (9), we get Remark 1. In a more general setting, where the zero-sum differential cost functional is not differentiable, subdifferentials/subgradients are needed; see [24,25,34] and references therein.
Now we recall the Arrow-Hurwicz-Uzawa method [26] and Korpelevich's extragradient method [27], which we applied in our previous paper [21]. For simplicity we present the algorithms for solving variational inequalities, and clearly the translation to min-max and saddle-point problems can be easily derived.
Let H 1 , H 2 be two real Hilbert spaces and a bifunction F : H 1 × H 2 → R with partial derivatives ∇ u F and ∇ v F. Choose an arbitrary starting point (u 0 , v 0 ) ∈ Q × R and stepsize α > 0. Given the current iterate (u k , v k ), the Arrow-Hurwicz-Uzawa update rule is formulated as follows: where P Q and P R are the orthogonal projection operators for the sets Q and R, respectively. A related modification with weaker convergence assumptions is Korpelevich's method, in which additional mid-points computations are done, corresponding to gradients, and thus its known name is the extragradient method: Although convergence of the extragradient method is guaranteed under weaker assumptions than the Arrow-Hurwicz-Uzawa method, there is still the need to calculate two evaluations of ∇F = (∇ u F, ∇ v F) and two projections onto Q and R. One step in the direction of simplifying the extragradient method with respect to the double projections is Censor et al.'s [30,31] subgradient extragradient method. In this method, the second orthogonal projection onto R is replaced by an easy computed projection onto some constructible set T k .
For avoiding the extra evaluations of ∇F per each iteration, Popov [35] proposed the following modification introducing the so-called "leading" point: A standard assumption, which we also use, for the convergence of the above methods, is the so-called roundedness of the derivatives, which means that the functional derivatives of J ε (u k , v k ) and J ε (u k , v k ) with respect to u and v, respectively, are uniformly bounded; i.e., there is a constant M > 0 such that for all k ≥ 0.
Since the introduction of the above methods, many modifications and extensions have been offered using various techniques-inertial, hybrid, viscosity and more; see, e.g., [36,37] and the references therein.

Interception Game
In this section we consider a particular singular problem (1)-(4), namely, n = 2, r = 1, s = 1. The matrices of coefficients in (1) and (2) are with the scalars g, The system (1) subject to the data (15), (16) has the following form: The solution of (18) with initial position (17) has the following integral form: where is a fundamental matrix solution of the corresponding homogeneous system and Thus, the analytical solution (after some technical calculations in (19) with (20) and (22)) can be written as The system (18), with (17) is a linearized kinematic model of a planar engagement between two vehicles-an interceptor (pursuer) and a target (evader) where both vehicles are directly controlled by their lateral accelerations u(t) = −a p (t) and v(t) = a e (t), respectively. The coordinates of the state vector z(t) = (z 1 (t), z 2 (t)) are the relative lateral separation and the relative lateral velocity of the vehicles. The basic schematic view of the planar engagement geometry is shown in Figure 1, where:

1.
The x-axis of the coordinate system is aligned with the initial line of sight; 2.
The points (x p , y p ), (x e , y e ) are the current coordinates; 3.
The origin is collocated with the initial pursuer position; 4.
V p and V e are the velocities;

5.
a p and a e are the lateral accelerations; 6. ϕ p and ϕ e are the respective aspect angles between velocities vectors and reference line of sight; 7. y = y e − y p is the relative lateral separation normal to the initial sight of line; 8.
r is the current range between the vehicles; 9.
The line-of-sight angle λ is the angle between the current and initial lines of sight. More details of such an engagement can be found, for instance, in [38,39]. The behavior of each player in this singular game is evaluated by the following regularized cost functional.
The cost functional (25) has to be minimized by the pursuer u(t) and maximized by the evader v(t).
The game, consisting of the dynamics (1), with the data (15) and (16), initial condition (17) and cost functional (25) is called the interception differential game.
By combining the above data and definitions and substituting in the functional (25) we obtain: For the numerical validation of the saddle-point approximation of this functional, we next develop the appropriate variational derivatives. Theorem 1. The variational derivatives of (26) are given by: and δJ ε δv Proof. Our functional J ε has two dependent variables of t (u and v), and we have to find the variational derivative with respect to u and v. In the view of above arguments, we have and Equations (29) and (30) will render the functional derivatives δJ ε δu and δJ ε δv , respectively. Therefore, we start with the (29): The above expression together with (10) yield It follows that the variational derivative of J ε with respect to u is given by with some simplifications, we obtain (27), and in a similar way we derive (28); and the proof is complete.

Duality Representation
We consider the functional (26) with the constants f 1 , f 2 , g > 0 and a small parameter ε > 0, and present its variational derivatives.

Theorem 2.
The dual representation of the functional (26) is given by: where or equivalently in scalar form * = arg max and all the coefficients will be presented in the proof itself.
Proof. In the lines of [40], let us calculate the program maxi-min: Using [41] z T Rz = max where R is symmetric positive definite matrix. Note that in (25) where z = (z 1 , z 2 ) T , Then, where Consequently, Then (40) becomes: where In (42), the operations of maximum over ∈ E 2 and minimum over u(·) ∈ L 2 [0, t f ] commute. Therefore, The inner minimizer and maximizer in (46) are Substituting (47) and (48) into (43), Thus, Observe that the above is associated with the theory of symmetrical matrices [42]. Moreover, note that the studied interception differential game is solvable if the matrix G given by (54) is positive definite. Hence, the desired result has been obtained.

Numerical Validation
In this section we examine the numerical behavior of the double projection methods described in Section 2 (Arrow-Hurwicz-Uzawa [26], Korpelevich's extragradient [27] and Popov [35]). We show that under boundedness of the derivatives assumption (14) the results such as ε → 0 + coincide with the dual development of the previous section. We choose g = 4, f 1 , f 2 = 0.5, t f = 4 and present the results for ε = 0.1, 0.01, 0.001. Observe that the Tables 1-3 and Figures 2-6 show that for ε → 0 + , the values z 1 (4) and z 2 (4) go to zero, meaning that the relative lateral velocity z 2 and relative lateral separation z 1 of the vehicles tend to zero at the end of the game, and the objective of the interception control is achieved. Observe also that the graphs of the inner minimizer and maximizer in Figure 2 coincide with the results in Table 2. Moreover, the values of the cost functional (26) decrease to 0.

Conclusions
In this work we presented a singular, zero-sum, linear-quadratic differential game in which the weight matrix of the minimizer's control cost in the cost functional is singular. As an application we focused on an interception differential game and introduced a regularized cost functional; we examined its dual representation and validated it via numerical schemes for finding saddle points.
Author Contributions: All authors contributed equally to the following: conceptualization, methodology, formal analysis, writing-original draft preparation. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.

Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.

Data Availability Statement:
The study does not report any data.