Optimal Guidance Laws for a Hypersonic Multiplayer Pursuit-Evasion Game Based on a Differential Game Strategy

Abstract

The guidance problem of a confrontation between an interceptor, a hypersonic vehicle, and an active defender is investigated in this paper. As a hypersonic multiplayer pursuit-evasion game, the optimal guidance scheme for each adversary in the engagement is proposed on the basis of linear-quadratic differential game strategy. In this setting, the angle of attack is designed as the output of guidance laws, in order to match up with the nonlinear dynamics of adversaries. Analytical expressions of the guidance laws are obtained by solving the Riccati differential equation derived by the closed-loop system. Furthermore, the satisfaction of the saddle-point condition of the proposed guidance laws is proven mathematically according to the minimax principle. Finally, nonlinear numerical examples based on 3-DOF dynamics of hypersonic vehicles are presented, to validate the analytical analysis in this study. By comparing different guidance schemes, the effectiveness of the proposed guidance strategies is demonstrated. Players in the engagement could improve their performance in confrontation by employing the proposed optimal guidance approaches with appropriate weight parameters.

1. Introduction

In recent decades, the technology of hypersonic vehicles (HVs) developed rapidly and has drawn considerable attention among researchers. The generally accepted definition of hypersonic flight is a flight through the atmosphere between 20 km and 100 km at a speed above Mach 5. The advantage of complete controllability of the whole flight process indicates great potential in terms of the military (hypersonic weapon) and civil (hypersonic airliner) applications of HVs. Nevertheless, the disadvantages of HVs are obvious, one of which is easily detected by infrared detectors. Since violent friction with the atmosphere heats the vehicle surface during flight, it will generate intensive infrared radiation. Moreover, the maneuver of HVs relies on aerodynamic force only, which means that their overload and maneuverability are limited. As a result, HVs face serious threats of new interceptors with the development of endoatmosphere interception technology.

In order to reduce the risk of being intercepted, there are two methods to improve HVs’ ability of confrontation: developing guidance laws for one-on-one competition, or carrying defender vehicles and transforming the one-on-one confrontation into a multiplayer game. As for the former, the one-on-one scenario has been researched extensively. The classical guidance laws such as proportional navigation (PN), augmented proportional navigation (APN), and optimal guidance laws (OGLs) were proposed for pursuers [1,2,3,4]. From the perspective of evader, one-on-one pursuit-evasion games can be formulated as two types of problems: a one-side optimization problem or differential game problem. A key assumption of a one-side optimization problem is that the player could obtain maneuvering and guidance information of its rival [5,6,7]. This tight restriction was relaxed by introducing the multiple model adaptive estimator in Ref. [8]. Information sharing and missile staggering were exploited to reduce the dependency of prior information in one-side optimization problems [9]. On the other hand, the differential game approach makes no assumption on the rival’s maneuver but requires each adversary’s state information [10,11]. The results in Ref. [10] demonstrated that PN is actually an optimal intercept strategy. Air-to-air missile guidance laws based on optimal control and differential game strategy were derived in Ref. [11], where the guidance laws based on differential game strategy were proven to be less sensitive to errors in acceleration estimation. Other guidance laws using sliding-mode control, formation control, heuristic method, and artificial neural network were investigated in Refs. [12,13,14,15,16,17]. It is worth noting that HVs are prone to meet saturation problem and chattering phenomenon when using the aforementioned guidance laws, because of their limited overload and maneuverability.

Carrying an active defense vehicle is efficient to reduce the maneuverability requirement of a target in confrontation, as well as to alleviate the problem of control saturation. Other than the high requirement of maneuverability in one-on-one games, the number of adversaries covers the inferiority of maneuverability [18,19,20,21,22,23]. As indicated in Ref. [18], optimal cooperative evasion and pursuit strategies for the target pair and the pursuer were derived. It should be noted that cooperative differential strategies could reduce maneuverability requirements from the target pair but bring difficulties to parameter choice and induce complicated calculations. Shima [19] derived optimal cooperative strategies of concise forms for aircraft and its defending missile by using Pontryagin minimum principle. However, the laws are calculated by a signum function, which causes a chattering phenomenon in control signals. Shaferman and Shima [20] considered a novel scenario in which a team of cooperating interceptors pursue a high-value target, and a relative intercept angle index was introduced to improve the performance of interceptors. In Ref. [21], cooperative guidance laws for aircraft defense were performed in a nonlinear framework by using the sliding-mode control technique. In addition to the above studies, Qi et al. [22] discussed the infeasible and feasible region of initial zero-effort-miss distance in a multiplayer game and provided evasion-pursuit guidance laws for the attacker. Garcia et al. [23] exploited the multiplayer game in a three-dimensional case and derived optimal strategies from the perspective of geometry.

It can be seen that most of the above studies are based on ideal scenarios in which the response of adversaries is rapid, and the dynamics are assumed to be linear. These particular assumptions will cause potential problems in practical application, since the responding speed of HVs, whose overload is generated by aerodynamic force, is commonly low. Thus far, there are few studies focusing on hypersonic, pursuit-evasion games. Chen et al. [24] proposed a fractional calculus guidance algorithm based on nonlinear proportional and differential guidance (PDG) law for a hypersonic, one-on-one pursuit-evasion game. However, the adversaries in multiplayer games commonly have more than one objective, so the family of PID controllers is difficult to be utilized in a multiplayer game. To the best of the authors’ knowledge, research on guidance laws of hypersonic, multiplayer pursuit-evasion games has not been explored in the available literature.

In this paper, we consider a hypersonic multiplayer game in which an HV carrying an active defense vehicle is pursued by an interceptor. In order to match up with nonlinear dynamics, the output of the proposed strategy is set up as the angle of attack (AOA). The main contribution of this paper is proposing linear-quadratic optimal guidance laws (LQOGLs) for adversaries in the game by simultaneously considering energy cost, control saturation, and chattering phenomenon. The optimal guidance strategies are derived through solving the linear-quadratic differential game problem with the aid of the Riccati differential equation. In addition, the satisfaction of the saddle-point condition of the proposed guidance laws is proved analytically. Simulations based on nonlinear kinematics and dynamics are presented, to validate that each adversary can benefit most within its ability by employing the proposed strategies.

This paper is organized as follows: In Section 2, a description of the multiplayer scenario and mathematical model is presented. In Section 3, the linear-quadratic differential strategies are derived and analyzed. In Section 4, simulation analysis is presented. Finally, some conclusions are provided in Section 5.

2. Engagement Formulation

In this section, an engagement is considered in which an HV carrying an active defense vehicle is pursued by an interceptor. In this engagement, the HV plays as a maneuvering target (M), the HV interceptor plays as an interceptor (I), and the active defense vehicle plays as a defender (D). The defender is launched sometime during the end game to protect the HV by destroying the interceptor. The engagement is analyzed in a plane. The three-dimensional version of optimal guidance laws can be obtained by extending the optimal guidance laws in the plane to three-dimensional models [25,26] and, thus, will not be discussed here.

2.1. Problem Statement

A schematic view of the planar engagement geometry is shown in Figure 1, where X-O-Y is the Cartesian reference system. There are two collision triangles in the engagement. One is between the interceptor and the HV (I–M collision triangle), and the other is between the interceptor and the defender (I–D collision triangle). The altitude, velocity, flight path angle, and lift coefficient are represented by $h$, $V$, $\varphi$ and $Cl$, respectively. The distance between each adversary is represented by ${\rho }_{ID}$ and ${\rho }_{IM}$, while the angle between the light of sight (LOS) and $X$ axis is represented by $\lambda$.

The HV is required to evade the interceptor with the assistance of the defender. Conversely, the mission of the interceptor includes evading the defender and pursuing the HV. Therefore, the guidance laws for target pairs are designed to converge $h_{ID}$ to zero and to maximize $h_{IM}$, while the guidance laws designed for interceptors should converge $h_{IM}$ to zero and maximize $h_{ID}$.

2.2. Equations of Motion

Considering the I–M collision triangle, equations of motion can be given by

$$S1:\{\begin{array}{l}{\dot{\rho }}_{IM}=-V_{IM}^L\hfill \\ {\dot{\lambda }}_{IM}=\frac{V_{IM}^{\perp }}{{\rho }_{IM}}\hfill \\ {\dot{h}}_{IM}=V_I\sin {\varphi }_I-V_M\sin {\varphi }_M\hfill \\ {\dot{V}}_I=0\hfill \\ {\dot{V}}_M=0\hfill \end{array}$$

(1)

where $V_{IM}^L$ is the relative velocity along ${\mathrm{LOS}}_{IM}$, and $V_{IM}^{\perp }$ is the lateral speed orthogonal to ${\mathrm{LOS}}_{IM}$, which can be calculated by

$$V_{IM}^L=V_I\cos \left({\varphi }_I+{\lambda }_{IM}\right)+V_M\cos \left({\varphi }_M-{\lambda }_{IM}\right)$$

(2)

$$V_{IM}^{\perp }=V_I\sin \left({\varphi }_I+{\lambda }_{IM}\right)-V_M\sin \left({\varphi }_M-{\lambda }_{IM}\right)$$

(3)

Additionally, the relative motion of the interceptor and defender in the I–D collision triangle can also be described in a similar manner as Equation (1).

$$S2:\{\begin{array}{l}{\dot{\rho }}_{ID}=-V_{ID}^L\hfill \\ {\dot{\lambda }}_{ID}=\frac{V_{ID}^{\perp }}{{\rho }_{ID}}\hfill \\ {\dot{h}}_{ID}=V_I\sin {\varphi }_I-V_D\sin {\varphi }_D\hfill \\ {\dot{V}}_I=0\hfill \\ {\dot{V}}_D=0\hfill \end{array}$$

(4)

where

$$V_{ID}^L=V_I\cos \left({\varphi }_I+{\lambda }_{ID}\right)+V_D\cos \left({\varphi }_D-{\lambda }_{ID}\right)$$

(5)

$$V_{ID}^{\perp }=V_I\sin \left({\varphi }_I+{\lambda }_{ID}\right)-V_M\sin \left({\varphi }_D-{\lambda }_{ID}\right)$$

(6)

The flight path angles of each adversary can be defined as

$${\dot{\varphi }}_i=\frac{{\Gamma }_{i}C{l}_i-g\cos \varphi }{V_i},i=\left\{I,M,D\right\}$$

(7)

where $g$ is the gravitational constant, and ${\Gamma }_i$ is an operator defined as follows:

$${\Gamma }_i=\frac{q_i{S}_i}{m_i} , i=\left\{I,M,D\right\}$$

(8)

where $q_i$ is the dynamic pressure, $S_i$ is the reference cross-sectional area of aircraft, and $m_i$ is the mass of aircraft.

2.3. Linearized Equations of Motion

During the endgame, the adversaries can be considered as constant-speed mass points, since, in most cases, the acceleration generated by thrusters is not significant in the guided phase of the flight [6]. Therefore, the equations of motion can be linearized around the initial collision course according to small-perturbation theory, and the multiplayer game can be formulated as a fixed-time optimal control process.

$$\overline{\mathrm{S}}1:\{\begin{array}{l}{\dot{\rho }}_{IM}=-\left(V_I+V_M\right)\hfill \\ {\dot{\lambda }}_{IM}=0\hfill \\ {\ddot{h}}_{IM}={\Gamma }_{I}C{l}_I-{\Gamma }_{M}C{l}_M\hfill \\ {\dot{V}}_I=0\hfill \\ {\dot{V}}_M=0\hfill \end{array}$$

(9a)

$$\overline{\mathrm{S}}2:\{\begin{array}{l}{\dot{\rho }}_{ID}=-\left(V_I+V_D\right)\hfill \\ {\dot{\lambda }}_{ID}=0\hfill \\ {\ddot{h}}_{ID}={\Gamma }_{I}C{l}_I-{\Gamma }_{D}C{l}_D\hfill \\ {\dot{V}}_I=0\hfill \\ {\dot{V}}_D=0\hfill \end{array}$$

(9b)

As a consequence of linearized kinematics, the gravitational force is neglected [6]. Meanwhile, it is reasonable to assume that the dynamics of each agent can be represented by first-order equations as

$${\stackrel{.}{Cl}}_i=\frac{-\left(C{l}_i-u_i\right)}{{\tau }_i},i=\{I,M,D\}$$

(10)

where $u_i$ is the guidance command.

In this engagement, we were concerned more about the miss distances orthogonal to LOS. Thus, the state variables chosen to represent the engagement are given as

$$x_1=\left[\begin{array}{llll}{h}_{IM}\hfill & {\dot{h}}_{IM}\hfill & C{l}_I\hfill & C{l}_M\hfill \end{array}\right]$$

(11a)

$$x_2=\left[\begin{array}{llll}{h}_{ID}\hfill & {\dot{h}}_{ID}\hfill & C{l}_I\hfill & C{l}_D\hfill \end{array}\right]$$

(11b)

The state functions can be expressed as

$$\mathrm{L}1:\{\begin{array}{l}{\dot{x}}_{11}=x_{12}\hfill \\ {\dot{x}}_{12}={\Gamma }_I{x}_{13}-{\Gamma }_M{x}_{14}\hfill \\ {\dot{x}}_{13}=\frac{-\left(x_{13}-u_I\right)}{{\tau }_I}\hfill \\ {\dot{x}}_{14}=\frac{-\left(x_{14}-u_M\right)}{{\tau }_M}\hfill \end{array}$$

(12a)

$$\mathrm{L}2:\{\begin{array}{l}{\dot{x}}_{21}=x_{22}\hfill \\ {\dot{x}}_{22}={\Gamma }_I{x}_{23}-{\Gamma }_D{x}_{24}\hfill \\ {\dot{x}}_{23}=\frac{-\left(x_{23}-u_I\right)}{{\tau }_I}\hfill \\ {\dot{x}}_{24}=\frac{-\left(x_{24}-u_D\right)}{{\tau }_D}\hfill \end{array}$$

(12b)

Accordingly, the state-space representation of the pursuit-evasion game is obtained.

$${\dot{x}}_1=A_1{x}_1+B_1{\left[\begin{array}{ll}{u}_I\hfill & u_M\hfill \end{array}\right]}^T$$

(13a)

$${\dot{x}}_2=A_2{x}_2+B_2{\left[\begin{array}{ll}{u}_I\hfill & u_D\hfill \end{array}\right]}^T$$

(13b)

where

$$A_1=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& {\Gamma }_I& {\Gamma }_M\\ 0& 0& -\frac{1}{{\tau }_I}& 0\\ 0& 0& 0& -\frac{1}{{\tau }_M}\end{array}\right], B_1=\left[\begin{array}{cc}0& 0\\ 0& 0\\ \frac{1}{{\tau }_I}& 0\\ 0& \frac{1}{{\tau }_M}\end{array}\right]$$

(14a)

$$A_2=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& {\Gamma }_I& {\Gamma }_D\\ 0& 0& -\frac{1}{{\tau }_I}& 0\\ 0& 0& 0& -\frac{1}{{\tau }_D}\end{array}\right], B_2=\left[\begin{array}{cc}0& 0\\ 0& 0\\ \frac{1}{{\tau }_I}& 0\\ 0& \frac{1}{{\tau }_D}\end{array}\right]$$

(14b)

A zero-effort-miss (ZEM) method is introduced to reduce the complexity of the mathematical model, which is the missed distance if both vehicles in collision engagement would apply no control from the current time. The ZEM of interceptor and target is represented by $Z_1$, and that of interceptor and defender is represented by $Z_2$, which can be, respectively calculated as

$$Z_1(t)=L_1{\Phi }_1\left(t,t_{f1}\right)x_1(t)$$

(15a)

$$Z_2(t)=L_2{\Phi }_2\left(t,t_{f2}\right)x_2(t)$$

(15b)

where $t_{f1}$ and $t_{f2}$ represent interception time, $L_1$ and $L_2$ are constant vectors defined as

$$L_1=\left[\begin{array}{llll}1\hfill & 0\hfill & 0\hfill & 0\hfill \end{array}\right]$$

(16a)

$$L_2=\left[\begin{array}{llll}1\hfill & 0\hfill & 0\hfill & 0\hfill \end{array}\right]$$

(16b)

Additionally, ${\Phi }_i(·)$ is the transition matrix which can be calculated by

$${\Phi }_i(·)=L^{-1}{\left[{\left(sI-A_i\right)}^{-1}\right]}^T,i=\{1,2\}$$

(17)

where $L^{-1}[·]$ is the inverse Laplace transformation, and $I$ denotes the identity matrix. Associated with Equations (13)–(17), ${\Phi }_1$ and ${\Phi }_2$ can be calculated as follows:

$${\Phi }_1=\left[\begin{array}{cccc}1& t& {\Gamma }_I{\tau }_I^2\varphi \left({\chi }_{1I}\right)& -{\Gamma }_M{\tau }_M^2\varphi \left({\chi }_{1M}\right)\\ 0& 1& {\Gamma }_I{\tau }_I\psi \left({\chi }_{1I}\right)& -{\Gamma }_M{\tau }_M\psi \left({\chi }_{1M}\right)\\ 0& 0& e^{-{\chi }_{1I}}& 0\\ 0& 0& 0& e^{-{\chi }_{1M}}\end{array}\right]$$

(18a)

$${\Phi }_2=\left[\begin{array}{cccc}1& t& {\Gamma }_I{\tau }_I^2\varphi \left({\chi }_{2I}\right)& -{\Gamma }_D{\tau }_D^2\varphi \left({\chi }_{2M}\right)\\ 0& 1& {\Gamma }_I{\tau }_I\psi \left({\chi }_{2I}\right)& -{\Gamma }_D{\tau }_D^2\psi \left({\chi }_{2M}\right)\\ 0& 0& e^{-{\chi }_{21}}& 0\\ 0& 0& 0& e^{-{\gamma }_{2M}}\end{array}\right]$$

(18b)

where $\varphi (·)$, $\psi (·)$ and ${\chi }_{ij}$ are computed as

$$\varphi (\xi )=e^{-\xi }+\xi -1$$

(19)

$$\psi (\xi )=-e^{-\xi }+1$$

(20)

$${\chi }_{ij}=\frac{t_{fj}-t}{{\tau }_i}$$

(21)

As a result, $Z_1$ and $Z_2$ are calculated by

$$Z_1(t)=x_{11}+\left(t_{f1}-t\right)x_{12}+{\Gamma }_I{\tau }_I^2\varphi \left({\chi }_{1I}\right)x_{13}-{\Gamma }_M{\tau }_M^2\varphi \left({\chi }_{1M}\right)x_{14}$$

(22a)

$$Z_2(t)=x_{21}+\left(t_{f2}-t\right)x_{22}+{\Gamma }_I{\tau }_I^2\varphi \left({\chi }_{2I}\right)x_{23}-{\Gamma }_D{\tau }_D^2\varphi \left({\chi }_{2D}\right)x_{24}$$

(22b)

The derivative of $Z_1$ and $Z_2$ can be given as follows:

$${\dot{Z}}_1(t)={\Gamma }_I{\tau }_I{u}_I\psi \left({\chi }_{1I}\right)-{\Gamma }_M{\tau }_M{u}_M\psi \left({\chi }_{1M}\right)$$

(23a)

$${\dot{Z}}_2(t)={\Gamma }_I{\tau }_I{u}_I\psi \left({\chi }_{2I}\right)-{\Gamma }_D{\tau }_D{u}_D\psi \left({\chi }_{2D}\right)$$

(23b)

Therefore, the dynamic system corresponding to Equations (13) and (14) is transformed into

$$\dot{\overline{x}}(t)=G_I(t)u_I(t)-G_E(t)u_E(t)$$

(24)

where

$$\overline{x}(t)=\left[\begin{array}{c}{Z}_1(t)\\ Z_2(t)\end{array}\right], u_E(t)=\left[\begin{array}{c}{u}_M(t)\\ u_D(t)\end{array}\right], G_I=\left[\begin{array}{c}{\Lambda }_{I1}\\ {\Lambda }_{I2}\end{array}\right], G_E=\left[\begin{array}{cc}-{\Lambda }_{M1}& 0\\ 0& -{\Lambda }_{D2}\end{array}\right]$$

(25)

$${\Lambda }_{ij}={\Gamma }_i{\tau }_i\psi \left({\chi }_{ij}\right)$$

(26)

Remark 1.

According to Equations (22a) and (22b), $Z_i(t)$ is independent of guidance laws and only relies on current states. If the current state is determined, $Z_i(t)$ can be determined. It can be seen from Equations (23a) and (23b) that the derivative $Z_i(t)$ is state-independent. Corresponding to the new state space defined by Equations (24)–(26), an optimal control problem with a fixed terminal time in a continuous system is considered. The objective of the target pair is to design optimal guidance schemes that can converge $Z_2$ to zero as $t\to t_{f2}$ while keeping $Z_1$ as large as possible. Conversely, the control law of the interceptor is designed to make $Z_1$ converge to zero while maintaining $Z_2$ as large as possible.

2.4. Timeline

With the linearization assumption, the interception time is fixed and can be calculated by

$$t_{f1}=-\frac{{\rho }_{IM0}}{{\dot{\rho }}_{IM}}=\frac{{\rho }_{M0}}{\left(V_I+V_M\right)}$$

(27a)

$$t_{f2}=-\frac{{\rho }_{ID0}}{{\dot{\rho }}_{ID}}=\frac{{\rho }_{ID0}}{\left(V_I+V_D\right)}$$

(27b)

It is reasonable to assume that the engagement of the interceptor and the defender terminates before that of the interceptor and the target, and thus, $t_{f2}<t_{f1}$. The nonnegative time-to-go of the interceptor–defender engagement and the interceptor–target engagement can be, respectively, calculated as follows:

$$t_{go1}=\{\begin{array}{ll}{t}_{fIM}-t,\hfill & t<t_{fIM}\hfill \\ 0,\hfill & t\ge t_{fIM}\hfill \end{array}$$

(28a)

$$t_{go2}=\{\begin{array}{ll}{t}_{fID}-t,\hfill & t<t_{fID}\hfill \\ 0,\hfill & t\ge t_{fID}\hfill \end{array}$$

(28b)

3. Guidance Schemes

3.1. Cost Function

The quadratic cost function in this problem is chosen as follows:

$$J\left(u_I,u_E\right)=\frac{1}{2}{\overline{x}}^T\left(t_f\right)P\overline{x}\left(t_f\right)+\frac{1}{2}{\displaystyle {\int }_{t_0}^{t_f}\left[u_I^T(t)r_{11}{u}_I(t)-u_E^T(t)R_E{u}_E(t)\right]}dt$$

(29)

where

$$P=\left[\begin{array}{cc}{p}_{11}& 0\\ 0& -p_{22}\end{array}\right], R_E=\left[\begin{array}{cc}{r}_{22}& 0\\ 0& r_{33}\end{array}\right]$$

(30)

The weights $p_{11}$, $p_{22}$, $r_{11}$, $r_{22}$ and $r_{33}$ are nonnegative. Let $u_I^{*}$, $u_M^{*}$, and $u_D^{*}$ be optimal guidance laws for interceptor, target, and defender, respectively. Thus, the guidance laws are issued so as to meet the condition set as follows:

$$\underset{{}_{\left[u_M^{*}{u}_D^{*}\right]}}{\max }\underset{{}_{u_I^{*}}}{\min }J$$

(31)

3.2. Cost Function

Let $\lambda \left(t\right)$ be the Lagrange multiplier vector,

$$\lambda (t)=\left[\begin{array}{l}{\lambda }_1(t)\hfill \\ {\lambda }_2(t)\hfill \end{array}\right]$$

(32)

The corresponding Hamiltonian is given by

$$H=\frac{1}{2}\left[u_I^T(t)R_I{u}_I(t)-u_E^T(t)R_E{u}_E(t)\right]+{\lambda }^T\left[G_I(t)u_I(t)-G_E(t)u_E(t)\right]$$

(33)

The costate equations and transversality conditions are given by

$$\{\begin{array}{l}{\dot{\lambda }}_1=-\frac{\partial H}{\partial Z_1}=0\hfill \\ {\dot{\lambda }}_2=-\frac{\partial H}{\partial Z_2}=0\hfill \end{array}$$

(34)

$$\{\begin{array}{l}{\lambda }_1\left(t_{fMM}\right)=\frac{\partial J}{\partial Z_1\left(t_{fMM}\right)}=p_{11}{Z}_1\left(t_{fMM}\right)\hfill \\ {\lambda }_2\left(t_{fID}\right)=\frac{\partial J}{\partial Z_2\left(t_{fID}\right)}=-p_{22}{Z}_2\left(t_{fID}\right)\hfill \end{array}$$

(35)

As the Hamiltonian is second-order continuously differentiable with respect to $u_I$ and $u_E$, $u_I^{*}$ and $u_E^{*}$ satisfy

$$\{\begin{array}{l}\frac{\partial H}{\partial u_I}=u_I^T(t)R_I+{\lambda }^T(t)G_I(t)=0\hfill \\ \frac{\partial H}{\partial u_E}=u_E^T(t)R_E+{\lambda }^T(t)G_E(t)=0\hfill \end{array}$$

(36)

Therefore, the optimal guidance laws can be calculated by

$$\begin{array}{l}{u}_I^{*}(t)=-R_I^{-1}{G}_I^T(t)\lambda (t)\\ \left[\begin{array}{c}{u}_M^{*}(t)\\ u_D^{*}(t)\end{array}\right]=-R_E^{-1}{G}_E^T(t)\lambda (t)\end{array}$$

(37)

Considering Equation (35), the linear relationship between $\lambda \left(t_{fID}\right)$ and $Z\left(t_{fID}\right)$ can be obtained immediately. Thus, it is reasonable to assume that

$$\lambda (t)=K(t)\overline{x}(t)$$

(38)

where $K\left(t\right)$ is a square matrix of order two and satisfies

$$K\left(t_{fID}\right)=P$$

(39)

The derivative $\lambda \left(t\right)$ with respect to time can be calculated by

$$\begin{array}{cc}\dot{\lambda }(t)& =\stackrel{.}{K}(t)\overline{x}(t)+K(t)\dot{\overline{x}}(t)\hfill \\ & =\stackrel{.}{K}(t)\overline{x}(t)+K(t)\left[G_I(t)u_I(t)-G_E(t)u_E(t)\right]\end{array}$$

(40)

Substituting the optimal guidance laws (37) into Equation (40), we have

$$\begin{array}{l}\dot{\lambda }(t)=\stackrel{.}{K}(t)\overline{x}(t)+K(t)\left[G_I(t)u_I^{*}(t)-G_E(t)u_E^{*}(t)\right]\\ =\stackrel{.}{K}(t)\overline{x}(t)-K(t)\left[G_I(t)R_I^{-1}{G}_I^T(t)-G_E(t)R_E^{-1}{G}_E^T(t)\right]\lambda (t)\\ =\stackrel{.}{K}(t)\overline{x}(t)-K(t)\left[G_I(t)R_I^{-1}{G}_I^T(t)-G_E(t)R_E^{-1}{G}_E^T(t)\right]K(t)\overline{x}(t)\end{array}$$

(41)

According to Equations (34) and (41), since $\dot{\lambda }\left(t\right)=0$ is satisfied for any $\overline{x}\left(t\right)$, we have

$$\stackrel{.}{K}(t)-K(t)\left[G_I(t)R_I^{-1}{G}_I^T(t)-G_E(t)R_E^{-1}{G}_E^T(t)\right]K(t)=0$$

(42)

Equation (42) is a well-known Riccati differential equation. Considering the following equation:

$$\begin{array}{cc}\hfill \frac{d{K}^{-1}}{dt}& =-K^{-1}\frac{dK}{dt}{K}^{-1}\hfill \\ & =-K^{-1}\left[K\left(G_I{R}_I^{-1}{G}_I^T-G_E{R}_E^{-1}{G}_E^T\right)K\right]K^{-1}\hfill \\ & =-\left(G_I{R}_I^{-1}{G}_I^T-G_E{R}_E^{-1}{G}_E^T\right)\hfill \end{array}$$

(43)

Then, by integrating Equation (43) from $t$ to $t_{fID}$ and considering Equation (39), the analytical expression of $K\left(t\right)$ is derived as follows:

$$K(t)={\left[P^{-1}+{\displaystyle {\int }_t^{t_{\mathscr{H}}}\left[G_I(t)R_I^{-1}{G}_I^T(t)-G_E(t)R_E^{-1}{G}_E^T(t)\right]}dt\right]}^{-1}$$

(44)

Substituting Equation (44) into Equation (38), the LQOGL $u_I^{*}$, $u_D^{*}$ and $u_M^{*}$ can be calculated by

$$\begin{array}{l}{u}_I^{*}(t)=-R_I^{-1}{G}_I^T(t)K(t)\overline{x}(t)\\ \left[\begin{array}{c}{u}_M^{*}(t)\\ u_D^{*}(t)\end{array}\right]=-R_E^{-1}{G}_E^T(t)K(t)\overline{x}(t)\end{array}$$

(45)

Now, the solution of optimal guidance laws is presented completely. By using the interpolation method, the desired AOA can be obtained, and this completes the design of the guidance laws.

3.3. Proof of Saddle-Point Condition

The proposed guidance laws $u_I^{*}$, and $u_E^{*}$ are functions of state vector $\overline{x}\left(t\right)$, which form closing-loop feedback controls of $\overline{x}\left(t\right)$. It should be proven that the optimal guidance laws given by Equation (45) satisfy the saddle-point condition. Considering Equations (24) and (42), it can be derived that

$$\begin{array}{cc}\hfill \frac{1}{2}\frac{d}{dt}{\overline{x}}^{T}K\overline{x}& =\frac{1}{2}{\dot{\overline{x}}}^{T}K\overline{x}+\frac{1}{2}{\overline{x}}^T\dot{K}\overline{x}+\frac{1}{2}{\overline{x}}^{T}K\dot{\overline{x}}\hfill \\ & =\frac{1}{2}\left[u_I^T{\mathrm{G}}_I^T-{\mathrm{u}}_E^T{\mathrm{G}}_E^T\right]K\overline{x}+\frac{1}{2}{\overline{x}}^{T}K\left[G_I{u}_I-G_E{u}_E\right]\hfill \\ & +\frac{1}{2}{\overline{x}}^{T}K\left[G_I\left(t\right)r_{11}^{-1}{G}_I^T\left(t\right)-G_E\left(t\right)R_E^{-1}{G}_E^T\left(t\right)\right]K\overline{x}\hfill \\ & =\frac{1}{2}{\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]}^T{r}_{11}\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]\hfill \\ & -\frac{1}{2}{\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]}^T{R}_E\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]\hfill \\ & -\frac{1}{2}\left[u_I^T{r}_{11}{u}_I-u_E^T{R}_E{u}_E\right]\hfill \end{array}$$

(46)

Integrating Equation (46) from $t_0$ to $t_{fID}$ and taking Equation (29) into account, the cost function can be derived as follows:

$$\begin{array}{cc}J\left(u_I,u_E\right)& =\frac{1}{2}{\overline{x}}^T\left(t_{fID}\right)K(t_{fID})\overline{x}\left(t_{fID}\right)+\frac{1}{2}{\displaystyle {\int }_{t_0}^{t_{fID}}\left[u_I^T(t)r_{11}{u}_I(t)-u_E^T(t)R_E{u}_E(t)\right]dt}\hfill \\ & =\frac{1}{2}{\overline{x}}^T\left(t_0\right)K(t_0)\overline{x}\left(t_0\right)+{\displaystyle {\int }_{t_0}^{t_{fID}}\frac{1}{2}{\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]}^T{R}_I\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]dt}\hfill \\ & -{\displaystyle {\int }_{t_0}^{t_{fID}}\frac{1}{2}{\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]}^T{R}_E\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]dt}\hfill \end{array}$$

(47)

If $u_I\left(t\right)=u_I^{*}\left(t\right)=-r_{11}^{-1}{G}_I\left(t\right)K\left(t\right)\overline{x}\left(t\right)$, we have

$$J\left(u_I^{*},u_E\right)=\frac{1}{2}{\overline{x}}^T\left(t_0\right)K(t_0)\overline{x}\left(t_0\right)-{\displaystyle {\int }_{t_0}^{t_{fID}}\frac{1}{2}{\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]}^T{R}_E\left[u_E+R_E^{-1}{G}_{E}K\overline{x}\right]dt}$$

(48)

It is obvious that $u_E\left(t\right)=u_E^{*}\left(t\right)=-R_E^{-1}{G}_E\left(t\right)K\left(t\right)\overline{x}\left(t\right)$ yields the minimum of $J$, which means

$$J\left(u_I^{*},u_E^{*}\right)\le J\left(u_I^{*},u_E\right)$$

(49)

Similarly, if $u_E\left(t\right)=u_E^{*}\left(t\right)=-R_E^{-1}{G}_E\left(t\right)K\left(t\right)\overline{x}\left(t\right)$, we have

$$J\left(u_I,u_E^{*}\right)=\frac{1}{2}{\overline{x}}^T\left(t_0\right)K(t_0)\overline{x}\left(t_0\right)+{\displaystyle {\int }_{t_0}^{t_{fID}}\frac{1}{2}{\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]}^T{R}_I\left[u_I+r_{11}^{-1}{G}_{I}K\overline{x}\right]dt}$$

(50)

Thus, $u_I\left(t\right)=u_I^{*}\left(t\right)=-R_I^{-1}{G}_I\left(t\right)K\left(t\right)\overline{x}\left(t\right)$ yields the maximum of $J$, which means

$$J\left(u_I,u_E^{*}\right)\le J\left(u_I^{*},u_E^{*}\right)$$

(51)

Combined with two situations (49) and (51), $u_I^{*}$ and $u_E^{*}$ satisfy the saddle-point condition as follows:

$$J\left(u_I,u_E^{*}\right)\le J\left(u_I^{*},u_E^{*}\right)\le J\left(u_I^{*},u_E\right)$$

(52)

4. Simulation and Analysis

In this section, the performance of the proposed guidance algorithms is investigated through nonlinear numerical examples. A scenario, consisting of an HV as the target (M), an active defense vehicle as the defender (D), and an HV interceptor as the interceptor (I), is considered. The interceptor is assigned the task of capturing the HV and evading the defender. Perfect information for the adversaries’ guidance laws is assumed.

4.1. Simulation Setup

In the simulated scenarios, 3-DOF point mass planetary flight mechanics [27] are employed in each adversary. All players use rocket engines to achieve hypersonic speed; the target and the defender are launched by the same rocket, while each interceptor is launched by a separate small rocket. The target and the defender have higher speeds than the interceptor since the target has a higher range requirement and is launched by a more powerful rocket. Hence, the initial horizontal velocities of the three players are set as $V_D=V_M=3000 \mathrm{m}/\mathrm{s}$, and $V_I=2000 \mathrm{m}/\mathrm{s}$, respectively. The altitudes are set as $h_I=40.5 \mathrm{km}$, $h_M=40 \mathrm{km}$, and $h_D=40.1 \mathrm{km}$, respectively. The endgame starts when the horizontal distance between the target and the interceptor reaches 150 km. The defender is assumed to be launched 20 km in front of the target at the beginning of the scenario. During the endgame, all players are in the glide phase and perform maneuvers, mainly relying on aerodynamic force. The HV is considered as a plane-symmetric lifting-body shape with one pair of air rudders, which can provide high L/D up to 3.5 [28]. Hence, the HV is required to employ bank-to-turn control. The active defense vehicle is a companion vehicle launched by HV whose aerodynamic performance is slightly worse than HV. Conversely, the interceptor is designed as an axisymmetric structure, with two pairs of air rudders, and employed skid-to-turn control for high agility. The desired roll rate of the interceptor can be expected to be much smaller than the HV since the interceptor can reorient the aerodynamic acceleration by changing the ratio of the AOA to the angel of sideslip [29]. This means that the interceptor sacrifices the aerodynamic performance in exchange for mobility and control stability. To compensate for the shortcoming of aerodynamic maneuverability, the interceptor was equipped with a rocket-based reaction-jet system (RCS), to obtain instant lateral acceleration. The RCS can only be turned on for a short time around the collision, due to limited fuel cost. The instantaneous overload of the interceptor is expected to be 9 g when exhausting the RCS. For these practical factors, the AOAs are assumed to be bounded as ${\alpha }_I^{\max }={\alpha }_D^{\max }={\alpha }_M^{\max }=35°$ with bounded changing rates as ${\dot{\alpha }}_I=6 \mathrm{deg}/\mathrm{s}$, ${\dot{\alpha }}_D=5 \mathrm{deg}/\mathrm{s}$ and ${\dot{\alpha }}_M=3 \mathrm{deg}/\mathrm{s}$, respectively. Time constant of each player is ${\tau }_I=5 \mathrm{ms}$ and ${\tau }_D={\tau }_M=10 \mathrm{ms}$, respectively. The simulation parameters of all adversaries are listed in

Table 1. Simulation parameters.

Parameters	Adversary
	Interceptor	Defender	Target
Latitude	0	0	0
Longitude	0 deg	0.0031 deg	0.023 deg
Altitude	40.5 km	40.1 km	40 km
Horizonal velocity	−2000 m/s	3000 m/s	3000 m/s
Vertical velocity	0	0	0
Maximum AOA	35 deg	35 deg	35 deg
Rate of AOA change	6 deg/s	5 deg/s	3 deg/s
Time constant	0.005 s	0.01 s	0.01 s
Killing radius	0.3 m	0.5 m	0.5 m

.

4.2. Numerical Examples

In this subsection, the effectiveness of the proposed LQOGL in Equation (45) is validated through the following three cases:

• The interceptor adopts PN guidance law, the defender adopts PN guidance law, and the target adopts LQOGL (PNvPNvLQOGL);
• The interceptor adopts PN guidance law, the defender adopts LQOGL, and the target adopts LQOGL (PNvLQOGLvLQOGL);
• The interceptor adopts LQOGL, the defender adopts LQOGL, and the target adopts LQOGL (LQOGLvLQOGLvLQOGL).

Generally, high L/D HVs are required for the capabilities of large downrange and cross-range [30]. Thus, unnecessary maneuvers of HV should be avoided since it will increase drag forces and cause kinetic energy cost and range loss. Thus, the LQOGL for the target can be expressed as follows:

$$\{\begin{array}{ll}{u}_M{}^{*}={\left[-R_E^{-1}{G}_E^T\left(t\right)K\left(t\right)\overline{x}\left(t\right)\right]}_{11}\hfill & for \Vert Z_{IM}\Vert \le \rho \hfill \\ u_M^{*}=0\hfill & for \Vert Z_{IM}\Vert >\rho \hfill \end{array}$$

(53)

Since the defender is designed to sacrifice itself to protect the HV, there is no need to take into account the range loss and control saturation for the defender. Unlike the HV and the defender, endoatmosphereic interceptor missile is generally steered by dual control systems of aerodynamic fins and reaction jets [31]. Thus, fuel cost should be taken into account for the interceptor, and the switching time of the jet engines should be strictly controlled. Inspired by Ref. [31], the maximum duration of the engine is assumed up to 1 s, and the engine is turned on when $0.5 \mathrm{s}<t_{go1}<1.5 \mathrm{s}$ and $0 \mathrm{s}<t_{go2}<1 \mathrm{s}$. Additionally, the dynamic of the divert thrust generated by jet engines can be assumed to be presented by a first-order equation,

$${\dot{L}}_J=-\frac{L_J-u_J}{{\tau }_I}$$

(54)

The $L_J$ herein is considered as a complement to lift force in Equation (53). The optimal guidance laws based on Pontryagin minimum principle for a reaction-jet system can be given by

$$u_J^{*}=u_J^{\max }\mathrm{sgn}\left[Z_2\right], 0 \mathrm{s}<t_{go2}<1 \mathrm{s}$$

(55)

$$u_J^{*}=-u_J^{\max }\mathrm{sgn}\left[Z_1\right], 0.5 \mathrm{s}<t_{go1}<1.5 \mathrm{s}$$

(56)

where $u_J^{\max }$ is the maximum thrust that the engines can provide, which is assumed to be seven times of gravitational force.

The simulation results are concluded in

Table 2. Simulation results.

Engagements	Case 1	Case 2	Case 3
	PNvPNvLQOGL	PNvLQOGLvLQOGL	LQOGLvLQOGLvLQOGL
$\mathbf{ZE}{\mathbf{M}}_{\mathbf{ID}}$	15.01 m	−15.03 m	15.01 m
$\mathbf{ZE}{\mathbf{M}}_{\mathbf{IM}}$	−0.17 m	−28.07 m	−0.17 m
Result	Target is intercepted by interceptor	Interceptor is expelled by the defender.	Target is intercepted by the interceptor.

.

Case 1.

The performance of the guidance strategies in multiplayer engagement, PNvPNvLQOGL, is investigated in this case. Based on the linear-quadratic differential game, LQOGL for the target is calculated by Equation (53), corresponding to weight parameters.

$$\left\{p_{11}=1\times {10}^8,p_{22}=1\times {10}^8,r_{11}=1.95,r_{22}=12.54,r_{33}=10.13\right\}, \mathrm{for} \mathrm{target}$$

(57)

Simulation results of case 1 are shown in Figure 2, which include the trajectory, ZEM, and AOA of each adversary. It can be seen from Figure 2a,b that the interceptor is pursued tightly by the defender but evades the defender depending on thrust during $0 \mathrm{s}<t_{go2}<1 \mathrm{s}$ and then turns its head to the target. The maximum acceleration of the interceptor reaches 10 g when the exhaust pipe is burning. Although the target has a higher L/D, the interceptor is able to intercept the target. The ZEM between the target and the interceptor and the ZEM between the defender and the interceptor both converge to zero swiftly. An interesting situation is observed in which even the target is pursued tightly by the interceptor, and it hardly maneuvers in the middle of the endgame. This situation can be understood as the scenario in which the interceptor is also tightly caught by the defender, which is considered safe by the target according to LQOGL. However, the key is that the jet engines on the interceptor can help it evade the defender when $0 \mathrm{s}<t_{go2}<1 \mathrm{s}$. The ZEM between the interceptor and the defender rises suddenly to 15.01 m when $t_{go2}=0 \mathrm{s}$, which is larger than the killing radius of the defender obviously. After that, the engagement is transformed into a one-on-one game, and the target has no time to enlarge the missing distance. Referring to Figure 2c–e, both the target and the interceptor face to control saturation. However, the target is not able to evade the interceptor since the rate of AOA change of the target is limited, while the interceptor can respond quickly by switching on the jet engines when $0.5 \mathrm{s}<t_{go1}<1.5 \mathrm{s}$. As can be seen in Figure 2b, the ZEM between the target and the interceptor converges to −0.17 m when $t_{go1}=0 \mathrm{s}$, which is smaller than the killing radius of the interceptor.

These results demonstrate that the interceptor is able to evade the defender using PN guidance law and successfully intercept the target. Additionally, the accelerations of the three adversaries do not reach the limit in the middle process of this engagement.

Remark 2.

Inspired by Ref. [19], guidance laws for target based on norm differential strategy can be derived from the cost function

$$\left\{p_{11}=1\times {10}^8,p_{22}=1\times {10}^8,r_{11}=1.95,r_{22}=12.54,r_{33}=10.13\right\}, \mathrm{for} \mathrm{target}$$

(58)

The guidance scheme for the target is calculated by

$$u_M^{*}=\mathrm{sgn}\left[{\alpha }_2{Z}_1\left(t_{fIM}\right){\Lambda }_{M1}\right]$$

(59)

It can be seen from Equation (59) that the guidance laws based on norm differential strategy are calculated by a signum function.

A controlled experiment is performed by replacing the target’s LQOGL with the norm optimal guidance law proposed in Equation (59). The results are shown in Figure 3, showing the trajectory, ZEM, and AOA of the target. It can be seen in Figure 3b that the ZEM between the target and the interceptor is kept around a safe distance. However, as shown in Figure 3c, the target meets with the control saturation problem and severe chattering phenomenon. This is consistent with the expression of norm optimal guidance law. Under the bank-to-turn control mode, the desired roll rate of the target will be very large. Additionally, large-amplitude maneuvering will lead to kinetic energy loss as far as range loss.

Case 2.

The performance of guidance strategies in engagement, PNvLQOGLvLQOGL, is investigated in this case. LQOGLs for target pair are calculated through Equations (45) and (53), corresponding to parameters chosen as follows:

$$\left\{p_{11}=1\times {10}^8,p_{22}=1\times {10}^8,r_{11}=1.95,r_{22}=12.54,r_{33}=10.13\right\}, \mathrm{for} \mathrm{target} \mathrm{pair}$$

(60)

Simulation results of case 2 are shown in Figure 4, which include trajectory, ZEM, and AOA of the target pair. The effectiveness of the proposed guidance laws is confirmed by this case. Figure 4a,b show that the interceptor evades from being intercepted by the defender but fails to catch the target. The ZEM between the interceptor and the defender is −15.03 m when $t_{go2}=0$, and the ZEM between the interceptor and the target vehicle is −28.07 m when $t_{go1}=0$, which are larger than the killing radius of the defender and the interceptor, respectively. It can be seen from Figure 4a,b that the interceptor is driven away from the target by the defender when $0 \mathrm{s}<t_{go2}<1 \mathrm{s}$ and cannot catch up with the target even the interceptor can accelerate by jet engines. Compared with the simulation result of case 1, smooth dynamics of the target and the defender are evident in Figure 4c,d. Moreover, the guidance laws of target pairs are not saturated, and the kinetic energy cost is considerably saved. The chattering phenomenon is also alleviated, which is beneficial to control stability.

Case 3.

The performance of guidance strategies in engagement, LQOGLvLQOGLvLQOGL, is investigated in this case. LQOGLs for three adversaries are calculated through Equations (45), (53), (55), and (56). Weight parameters are chosen as follows:

$$\left\{p_{11}=1\times {10}^8,p_{22}=1\times {10}^8,r_{11}=1.95,r_{22}=12.54,r_{33}=10.13\right\}, \mathrm{for} \mathrm{target} \mathrm{pair}$$

(61)

$$\{p_{11}=1\times {10}^8,p_{22}=1\times {10}^8,r_{11}=25,r_{22}=2.22,r_{33}=17.03\}, \mathrm{for} \mathrm{interceptor}$$

(62)

Figure 5 presents the simulation results of case 3 in order similar to the above cases. As Figure 5a shows, the guidance laws proposed for the target pair are effective. Instead of evading the interceptor persistently, the target acts similar to a bat and lures the interceptor close to the defender. It allows the defender to catch the interceptor more easily. However, contrary to the result of case 2, the interceptor skims the defender without changing heading violently and turns its head to the target. Obviously, it can be seen from Figure 5b that the interceptor is able to attack the target vehicle without being intercepted by the defender. The ZEM between the interceptor and the defender is 15.01 m when $t_{go2}=0$, and the ZEM between the interceptor and the target vehicle is −0.17 m when $t_{go1}=0$.

5. Conclusions

• In this research, a set of guidance laws for a hypersonic multiplayer pursuit-evasion game is derived based on linear-quadratic differential strategy. The energy cost, control saturation, chattering phenomenon, and aerodynamics were considered simultaneously. The satisfaction of saddle-point condition in a differential game was also proven theoretically.
• Nonlinear numerical examples of the multiplayer game were presented to validate the analysis. The advantage and efficiency of the proposed guidance were verified by the results. The LQOGLs exactly reduce the maneuverability requirement of the target in the pursuit-evasion game. Compared with the norm differential strategy, the proposed guidance strategy reduces the energy cost, alleviates the saturation problem, and avoids the chattering phenomenon, which guarantees task accomplishment and increases guidance phase stability.
• The performance of the interceptor showed that the proposed optimal guidance approach is able to complete the intercept mission if the interceptor possesses superior maneuverability. It is important to note that the saturation problem cannot be avoided completely when all the adversaries employ the LQOGL, since maneuverability is the most important factor in determining whether they will win or lose in the game. The interceptor or the target pair should make their best effort to attack or defend by exhaustedly performing maneuvers.