A Class of Pursuit Problems in 3D Space via Noncooperative Stochastic Differential Games

Abstract

This paper investigates three-dimensional pursuit problems in noncooperative stochastic differential games. By introducing a novel polynomial value function capable of addressing high-dimensional dynamic systems, the forward–backward stochastic differential equations (FBSDEs) for optimal strategies are derived. The uniqueness of the value function under bounded control inputs is rigorously established as a theoretical foundation. The proposed methodology constructs optimal closed-loop feedback strategies for both pursuers and evaders, ensuring state convergence and solution uniqueness. Furthermore, the Lebesgue measure of the barrier surface is computed, enabling the design of strategies for scenarios involving multiple pursuers and evaders. To validate its applicability, the method is applied to missile interception games. Simulations confirm that the optimal strategies enable pursuers to consistently intercept evaders under stochastic dynamics, demonstrating the robustness and practical relevance of the approach in pursuit–evasion problems.

1. Introduction

Motivation: The study of differential game theory has a long and rich history, beginning with the foundational work of Isaacs [1] in the mid-1950s. Over the decades, this field has found applications in diverse domains, such as aerospace, robotics, and control systems [2,3,4,5,6]. A central focus in this area has been the classical non-cooperative linear-quadratic differential game, characterized by linear dynamics and quadratic cost functions. Key advances include necessary and sufficient conditions for the existence of saddle points in deterministic two-player zero-sum differential games over finite time intervals [7]. In stochastic settings, the complexity increases due to uncertainties in system dynamics and decision-making processes. Recent studies have tackled challenges such as unknown system matrices for regulators and participants, leading to adaptive stability solutions for linear-quadratic stochastic differential games [8]. Despite these advances, multi-agent scenarios involving many pursuers and many evaders remain underexplored, particularly in stochastic differential games. Critical challenges include addressing solution uniqueness and quantifying the Lebesgue measure of barrier surfaces, both of which are fundamental to designing optimal strategies.

This paper aims to address these gaps by introducing a novel framework for stochastic differential games in three-dimensional spaces. Specifically, we focus on multi-agent interactions where pursuers and evaders operate under stochastic dynamics. The contributions of this paper are threefold. First, we model these interactions using stochastic differential equations, explicitly accounting for uncertainties to analyze their impact on game strategies. Second, we establish the uniqueness of solutions, ensuring the solvability of the proposed game model. Finally, we compute the Lebesgue measure of the barrier surface using precise mathematical tools, demonstrating its practical relevance in real-world applications such as missile interception.

Brief Summary of Prior Literature: Linear-quadratic differential games have long provided a theoretical foundation for understanding game dynamics under deterministic and stochastic conditions. For games over infinite time horizons, feedback Nash equilibria are well-established, with their existence linked to solutions of algebraic Riccati equations [9]. Extensions to stochastic systems on finite horizons have emphasized feedback information structures, with verification theorems for feedback Stackelberg Nash equilibria derived using fully nonlinear parabolic partial differential equations [10]. Additionally, coupled Riccati equations have been employed to establish local existence, uniqueness, and sufficient conditions for equilibrium solutions. Significant progress has been made in two-player stochastic differential games. For example, Stackelberg solutions under open-loop information structures have been investigated, with applications to mixed linear-quadratic games incorporating input constraints [11]. Similarly, studies on Stackelberg games with Markov jump-diffusion stochastic differential equations have utilized stochastic maximum principles to derive optimal solutions for both leaders and followers [12]. A notable development is the exploration of linear-quadratic Stackelberg differential games for Markov jump-diffusion systems. By formulating a general stochastic maximum principle, open-loop optimal strategies for leaders and followers are derived. The existence of an open-loop saddle point guarantees optimality for both players, as neither can unilaterally improve their outcomes. However, such studies often focus on scenarios involving a single controller in the optimization process. Extensions to multi-agent systems with Markov jump-diffusion dynamics have been investigated by Lv [13], Moon [14], Sun [15], and Zhang and Li [16]. These works analyze interactions between decision-makers under stochastic dynamics but primarily address two-player settings or assume specific structural simplifications.

Recent advancements in multi-agent stochastic differential games have introduced methods such as mean-field games [17] and mixed mean-field analysis [18] to address scenarios with numerous agents. While these approaches offer valuable insights, they often rely on open-loop solutions or simplifying assumptions, such as convex control sets. Feedback strategies, though explored in linear-quadratic frameworks [19], remain underutilized for multi-agent stochastic games involving bounded controls. Studies like [20] have applied the stochastic maximum principle to sequentially solve leader–follower problems, yielding open-loop Stackelberg equilibria. Adaptive strategies for mean-field stochastic differential game problems have been developed using techniques such as weighted least squares estimation, random regularization, and decreasing incentive methods [21]. Additionally, the state-feedback form of Nash equilibrium strategies has been constructed using coupled Riccati equations [22]. Sun and Yong [23] further investigated the properties of open-loop and closed-loop saddle points, while Yu [24] and Miller and Pham [25] extended these approaches to broader scenarios, including McKean–Vlasov stochastic differential equations and Markov jump-diffusion models [26]. Despite these contributions, the challenges of many-to-many stochastic interactions, solution uniqueness, and the impact of bounded controls remain open. This paper extends existing frameworks by employing forward–backward stochastic differential equations (FBSDEs) to model multi-agent interactions. Building on methods from [27,28], we incorporate bounded controls and multi-agent dynamics, addressing both the uniqueness of solutions and the computation of barrier surface measures.

In game theory, Nash equilibrium refers to a situation where all players make decisions simultaneously, and no player can improve their outcome by unilaterally changing their strategy, assuming others keep their strategies unchanged. This concept is widely used in non-hierarchical settings where players have equal decision-making power. On the other hand, Stackelberg equilibrium involves a hierarchical structure. In this scenario, one player (the leader) makes their decision first, while the other players (followers) observe this decision and respond optimally. This sequential decision-making process reflects many real-world scenarios, such as competition between a dominant firm and smaller firms in an industry. The comparison of Nash and Stackelberg Equilibria is shown in Figure 1.

Our work differs from studies by Laszlo and Lions, as well as Carmona, Delarue, and Rachepel, who explored $ϵ$-mean field games [29]. These works typically assume solvable Riccati equations, simplifying the construction of Stackelberg equilibria. While recent studies have proposed strategies for two-player zero-sum Stackelberg and Nash stochastic linear-quadratic (LQ) games, the relationship between these two types of games remains unclear [30]. Additionally, feedback information structures have been analyzed for Stackelberg games on finite horizons [31], with verification theorems derived from Hamiltonian functions. However, the solvability of Riccati equations remains an open question. Other studies incorporating linear-quadratic partial observation models [32] address correlated states and observation noise but primarily focus on two-player scenarios. In contrast, our study provides a comprehensive framework for addressing multi-agent stochastic differential games. By proving solution uniqueness and computing the Lebesgue measure of barrier surfaces, we address fundamental challenges in the field and broaden its applicability to systems with bounded controls and uncertain dynamics.

Contribution of this paper: Inspired by the above literature, we consider the pursuit problems via the non-cooperative stochastic differential game. The main contributions of this article focus on the following points:

1.

This work extends previous studies (e.g., Qi et al. (2024) in [10]) by addressing pursuit problems in many-to-one and many-to-many scenarios, with initial applications in missile interception. Building on the system dynamics introduced in Section 2.2, we derive the optimal strategies for high-dimensional systems through a system of partial differential equations.

2.

In Section 2 presents a rigorous analysis proving the uniqueness of the value function under bounded control inputs. A novel polynomial value function is introduced, which plays a critical role in ensuring the stability and scalability of the proposed framework.

3.

Leveraging the uniqueness of the value function, we further establish in Section 5 the uniqueness of state trajectories within the pursuit problem. Additionally, we analyze the barrier surface separating the pursuit region and the termination set, demonstrating that its Lebesgue measure is zero. This result is crucial for ensuring the feasibility of optimal strategies in practical scenarios.

The value function chosen in this paper is a polynomial, which differs significantly from the one-dimensional value function discussed in references [10]. This gap leads to distinct solutions and results in the formulation of forward–backward stochastic differential equations (FBSDEs) that are also different from those in [20] in prior works. Specifically, the polynomial nature of the value function necessitates a different approach to solving the system, involving matrix calculations and a more complex formulation of the backward differential equations. Additionally, the method proposed in this paper is applied to high-dimensional systems and, for the first time, is applied to missile interception problems, showcasing its versatility and effectiveness in real-world scenarios.

Through this novel approach, we establish the convergence and uniqueness of the state trajectory when both the pursuer and evader follow the optimal state feedback strategy. Furthermore, we demonstrate that the Lebesgue measure of the barrier surface, which separates the pursuit region from the termination set, is zero. These results, stemming from the novel polynomial form of the value function, contribute new insights to the field of non-cooperative linear-quadratic stochastic differential games.

Organization: The organization of this paper is as follows: In Section 2.1, we introduce a three-dimensional model of the pursuit problem. Based on this model, Section 2.2 formulates a high-dimensional stochastic differential game that incorporates the players’ control variables and the diffusion term of the state equation. In Section 3, we provide proof of the uniqueness of the value function and derive the expression for the optimal closed-loop state feedback strategy. In Section 4, based on the uniqueness of the state trajectory within the pursuit region, the Lebesgue measure of the barrier surface is zero, and the barrier surface consists of points where the rate of distance change is zero. In Section 5, we present optimal strategies for many-to-many pursuit problems. Finally, in Section 6, we validate the feasibility of the optimal strategies designed for many-to-one and many-to-many pursuit problems through numerical simulations. The mind map of this paper is shown in Figure 2.

2. Many-to-One Pursuits Problem in Stochastic Differential Games

In this section, we will provide a fundamental definition of the many-to-one pursuit problem and present a mathematical model for stochastic differential games. Some assumptions are given.

2.1. Notation

The pursuers $P_i$ are collaboratively capturing an evader $E_i$, and the evader is trying to evade the capture of the pursuers. The game is played in a three-dimensional space. Assume that both the pursuers and the evader are mass points with normal acceleration constraints. The direction of the velocity is adjusted by the normal acceleration. The direction of the acceleration is perpendicular to the direction of the velocity. The chase model between a pursuer and an evader is shown in Figure 3. From the above references, the non-linear differential equations are obtained based on the relationship between the pursuer $P_i$ and the evader $E_i$ in three dimensions [33], i.e.,

$${\dot{R}}_{Pi}=v_{Ei}cos{\theta }_{Ei}cos{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}cos{\phi }_{Pi}$$

(1)

$$R_{Pi}{\dot{\theta }}_{Li}=v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Pi}$$

(2)

$$R_{Pi}{\dot{\phi }}_{Li}cos{\theta }_{Li}=v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}$$

(3)

$$\begin{array}{cc}\hfill {\dot{\theta }}_{Pi}=& {\displaystyle \frac{A_{yPi}}{v_{Pi}}}+tan{\theta }_{Li}sin{\phi }_{Pi}\times {\displaystyle \frac{(v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei})}{R_{Pi}}}\hfill \\ \hfill & +cos{\phi }_{Pi}{\displaystyle \frac{(v_{Pi}sin{\theta }_{Pi}-v_{Ei}sin{\theta }_{Ei})}{R_{Pi}}}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\dot{\phi }}_{Pi}=& {\displaystyle \frac{A_{zPi}}{v_{Pi}cos{\theta }_{Pi}}}+sin{\theta }_{Pi}cos{\phi }_{Pi}tan{\theta }_{Li}+{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}cos{\theta }_{Pi}}}\hfill \\ \hfill & -sin{\theta }_{Pi}sin{\phi }_{Pi}{\displaystyle \frac{v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Ei}}{R_{Pi}cos{\theta }_{Pi}}}-{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}}}\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\dot{\theta }}_{Ei}=& {\displaystyle \frac{A_{yEi}}{v_{Ei}}}+tan{\theta }_{Li}sin{\phi }_{Ei}\times {\displaystyle \frac{\left(v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}\right)}{R_{Pi}}}\hfill \\ \hfill & +cos{\phi }_{Ei}{\displaystyle \frac{\left(v_{Pi}sin{\theta }_{Pi}-v_{Ei}sin{\theta }_{Ei}\right)}{R_{Pi}}}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\dot{\phi }}_{Ei}=& {\displaystyle \frac{A_{zEi}}{v_{Ei}cos{\theta }_{Ei}}}+sin{\theta }_{Ei}cos{\phi }_{Ei}tan{\theta }_{Li}+{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}cos{\theta }_{Ei}}}\hfill \\ \hfill & -sin{\theta }_{Ei}sin{\phi }_{Ei}{\displaystyle \frac{v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Ei}}{R_{Pi}cos{\theta }_{Ei}}}-{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}}}\hfill \end{array}$$

(7)

The pursuit–evasion problem in three-dimensional space is modeled with multiple pursuers attempting to capture an evader. The notation used to describe the system dynamics is as follows in

Table 1. List of symbols and their descriptions.

Symbol	Description	Coordinate System
$(X_I,Y_I,Z_I)$	Inertial reference coordinate system	Inertial
$(X_L,Y_L,Z_L)$	Line-of-sight (LOS) coordinate system	LOS
$(X_E,Y_E,Z_E)$	Velocity coordinate system of the i-th pursuer	Pursuer
$v_{Ei}$	Velocity of the i-th evader	Evader
$v_{Pi}$	Velocity of the i-th pursuer	Pursuer
$A_{Pi}$	Acceleration of the i-th pursuer	Pursuer
$A_{Ei}$	Acceleration of the i-th evader	Evader
${\gamma }_{Pi}$	Angle between the acceleration of the i-th pursuer and axis $Y_{Pi}$	Pursuer
${\gamma }_{Ei}$	Angle between the acceleration of the i-th evader and axis $Y_E$	Evader
$R_{Pi}$	Distance between the i-th pursuer and the evader	Spatial
${\theta }_{Li},{\phi }_{Li}$	LOS angles between the evader and the i-th pursuer relative to the inertial reference coordinate system	LOS
${\theta }_{Pi},{\phi }_{Pi}$	Elevation and azimuth angles of $v_{Pi}$ relative to the LOS coordinate system from pointing toward E	Pursuer
${\theta }_{Ei},{\phi }_{Ei}$	Elevation and azimuth angles of $v_{Ei}$ relative to the LOS coordinate system from $E_i$ pointing toward $P_i$	Evader
$A_{zPi},A_{yPi}$	Projections of the pursuer’s normal acceleration on the $Z_{Pi}$ and $Y_{Pi}$ axes in the velocity coordinate system	Pursuer
$A_{zEi},A_{yEi}$	Projections of the evader’s normal acceleration on the $Z_{Ei}$ and $Y_{Ei}$ axes in the velocity coordinate system	Evader

.

2.2. Problem Formulation

In this paper, $\left(\Omega ,\mathcal{F},{\left({\mathcal{F}}_t\right)}_{t>0},\mathcal{P}\right)$ is a complete probability space, and $W\left(s\right)$ is m-dimensional Brownian motion. For any initial time $t>0$ , terminal time $T>t$, and the initial state $x_0\in {\mathbb{R}}^n$, the filtration ${\left({\mathcal{F}}_t\right)}_{t>0}$ is the natural filtration generated by the Brownian motion $W\left(s\right)$ for $t\le s\le T$, augmented by all the $\mathcal{P}$-null sets of $\mathcal{F}$. According to Equations (1)–(7), we consider a stochastic differential game in a pursuit problem with control input constraints:

$$\left\{\begin{array}{c}dx\left(s\right)=\left(b\left(s,x\left(s\right)\right)+b_{u1}\left(x\left(s\right)\right)u_1\left(s\right)+b_{u2}\left(x\left(s\right)\right)u_2\left(s\right)\right.+b_{v1}\left(x\left(s\right)\right)v_1\left(s\right)+b_{v2}\left(x\left(s\right)\right)v_2\left(s\right))ds\hfill \\ +\sigma \left(s,x\left(s\right),u_1\left(s\right),u_2\left(s\right),v_1\left(s\right),v_2\left(s\right)\right)dW\left(s\right),\hfill \\ s\in \left(t,T\right),\hfill \\ x\left(t\right)=x_t.\hfill \end{array}\right.$$

(8)

where $x\left(s\right)\in {\mathbb{R}}^{m\times 1}$ is the system state with the initial condition $x_t$, $m=5n+2k$, n is the number of pursuers, and k is the number of evaders. $x={\left[\begin{array}{c}{R}_{P1},R_{P2},\dots ,{\theta }_{L1},{\phi }_{L1},{\theta }_{L2},{\phi }_{L2},\\ {\theta }_{P1},{\phi }_{P1},{\theta }_{P2},{\phi }_{P2},{\theta }_{E1},{\phi }_{E1},\dots \end{array}\right]}^T$. $u_1\left(s\right)\in {\mathbb{R}}^{n\times 1}$ is the acceleration of pursuers in the velocity coordinate system of the y-axis. $u_2\left(s\right)\in {\mathbb{R}}^{n\times 1}$ is the acceleration of pursuers in the velocity coordinate system of the z-axis. $v_1\left(s\right)\in {\mathbb{R}}^{k\times 1}$ is the acceleration of evader in the velocity coordinate system of the y-axis. $v_2\left(s\right)\in {\mathbb{R}}^{k\times 1}$ is the acceleration of evader in the velocity coordinate system of the z-axis. $\left\{W\left(s\right)\in {\mathbb{R}}^{m\times 1},{\left({\mathcal{F}}_t\right)}_{t>0},t\le s\le T\right\}$ is the standard Wiener process, $b(s,x\left(s\right))\in {\mathbb{R}}^{m\times 1}$, $b_{u1}(s,x\left(s\right))\in {\mathbb{R}}^{m\times n}$, $b_{u2}(s,x\left(s\right))\in {\mathbb{R}}^{m\times n}$, $b_{v1}(s,x\left(s\right))\in {\mathbb{R}}^{m\times 1}$, $b_{v2}(s,x\left(s\right))\in {\mathbb{R}}^{m\times 1}$ are real matrices. $\sigma \left(s,x\left(s\right),u_1\left(s\right),u_2\left(s\right),v_1\left(s\right),v_2\left(s\right)\right)\in {\mathbb{R}}^{m\times m}$ is the diffusion term of the system equation. The relevant parameters are illustrated using an example involving three pursuers and two evaders:

1. Matrix $b_{u1}$:

$$b_{u1}=\left[\begin{array}{ccc}0& 0& 0\\ ⋮& ⋮& ⋮\\ {\displaystyle \frac{1}{v_{P1}}}& 0& 0\\ 0& 0& 0\\ 0& {\displaystyle \frac{1}{v_{P2}}}& 0\\ 0& 0& 0\\ 0& 0& {\displaystyle \frac{1}{v_{P3}}}\\ ⋮& ⋮& ⋮\\ 0& 0& 0\end{array}\right]\begin{array}{c}\\ \\ \mathrm{Start}\mathrm{of}\mathrm{numbers}(10\mathrm{th}\mathrm{row})\\ \\ \\ \end{array}$$

2. Matrix $b_{u2}$:

$$b_{u2}=\left[\begin{array}{ccc}0& 0& 0\\ ⋮& ⋮& ⋮\\ {\displaystyle \frac{1}{v_{P1}cos\left({\theta }_{P_1}\right)}}& 0& 0\\ 0& 0& 0\\ 0& {\displaystyle \frac{1}{v_{P2}cos\left({\theta }_{P_2}\right)}}& 0\\ 0& 0& 0\\ 0& 0& {\displaystyle \frac{1}{v_{P3}cos\left({\theta }_{P_3}\right)}}\\ ⋮& ⋮& ⋮\\ 0& 0& 0\end{array}\right]\begin{array}{c}\\ \\ \mathrm{Start}\mathrm{of}\mathrm{numbers}(11\mathrm{th}\mathrm{row})\\ \\ \\ \end{array}$$

3. Matrix $b_{v1}$:

$$b_{v1}=\left[\begin{array}{cc}0& 0\\ ⋮& ⋮\\ {\displaystyle \frac{1}{v_{E1}}}& 0\\ 0& 0\\ 0& {\displaystyle \frac{1}{v_{E2}}}\\ ⋮& ⋮\\ 0& 0\end{array}\right]\begin{array}{c}\\ \\ \mathrm{Start}\mathrm{of}\mathrm{numbers}(16\mathrm{th}\mathrm{row})\\ \\ \\ \end{array}$$

4. Matrix $b_{v2}$:

$$b_{v2}=\left[\begin{array}{cc}0& 0\\ ⋮& ⋮\\ {\displaystyle \frac{1}{v_{E1}cos\left({\theta }_{E1}\right)}}& 0\\ 0& 0\\ 0& {\displaystyle \frac{1}{v_{E2}cos\left({\theta }_{E2}\right)}}\\ ⋮& ⋮\\ 0& 0\end{array}\right]\begin{array}{c}\\ \\ \mathrm{Start}\mathrm{of}\mathrm{numbers}(17\mathrm{th}\mathrm{row})\\ \\ \\ \end{array}$$

5. Vector b:

$$b=\left[\begin{array}{c}{v}_{E1}cos{\theta }_{E1}cos{\phi }_{E1}-v_{P1}cos{\theta }_{P1}cos{\phi }_{P1},\\ v_{E1}cos{\theta }_{E1}cos{\phi }_{E1}-v_{P2}cos{\theta }_{P2}cos{\phi }_{P2},\\ v_{E2}cos{\theta }_{E2}cos{\phi }_{E2}-v_{P3}cos{\theta }_{P3}cos{\phi }_{P3},\\ {\displaystyle \frac{v_{E1}sin{\theta }_{E1}-v_{P1}sin{\theta }_{P1}}{R_{P1}}},\\ {\displaystyle \frac{v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}-v_{E1}cos{\theta }_{E1}}{sin{\phi }_{E1}/R_{P1}cos{\theta }_{L1}}},\\ {\displaystyle \frac{v_{E1}sin{\theta }_{E1}-v_{P2}sin{\theta }_{P2}}{R_{P2}}},\\ {\displaystyle \frac{v_{P2}cos{\theta }_{P2}sin{\phi }_{P2}-v_{E1}cos{\theta }_{E1}}{sin{\phi }_{E1}/R_{P2}cos{\theta }_{L2}}},\\ {\displaystyle \frac{v_{E2}sin{\theta }_{E2}-v_{P3}sin{\theta }_{P3}}{R_{P3}}},\\ {\displaystyle \frac{v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}-v_{E2}cos{\theta }_{E2}}{sin{\phi }_{E2}/R_{P3}cos{\theta }_{L3}}},\\ tan{\theta }_{L1}sin{\phi }_{P1}{\displaystyle \frac{(v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}-v_{E1}cos{\theta }_{E1}sin{\phi }_{E1})}{R_{P1}}}+cos{\phi }_{P1}{\displaystyle \frac{(v_{P1}sin{\theta }_{P1}-v_{E1}sin{\theta }_{E1})}{R_{P1}}},\\ sin{\theta }_{P1}cos{\phi }_{P1}tan{\theta }_{L1}+{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}}{R_{P1}cos{\theta }_{P1}}}\\ -sin{\theta }_{P1}sin{\phi }_{P1}{\displaystyle \frac{v_{E1}sin{\theta }_{E1}-v_{P1}sin{\theta }_{E1}}{R_{P1}cos{\theta }_{P1}}}-{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}}{R_{P1}}},\\ tan{\theta }_{L2}sin{\phi }_{P2}{\displaystyle \frac{(v_{P2}cos{\theta }_{P2}sin{\phi }_{P2}-v_{E1}cos{\theta }_{E1}sin{\phi }_{E1})}{R_{P2}}}+cos{\phi }_{P2}{\displaystyle \frac{(v_{P2}sin{\theta }_{P2}-v_{E1}sin{\theta }_{E1})}{R_{P2}}},\\ sin{\theta }_{P2}cos{\phi }_{P2}tan{\theta }_{L2}+{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P2}cos{\theta }_{P2}sin{\phi }_{P2}}{R_{P2}cos{\theta }_{P2}}}\\ -sin{\theta }_{P2}sin{\phi }_{P2}{\displaystyle \frac{v_{E1}sin{\theta }_{E1}-v_{P2}sin{\theta }_{E1}}{R_{P2}cos{\theta }_{P2}}}-{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P2}cos{\theta }_{P2}sin{\phi }_{P2}}{R_{P2}}},\\ tan{\theta }_{L3}sin{\phi }_{P3}{\displaystyle \frac{(v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}-v_{E2}cos{\theta }_{E2}sin{\phi }_{E2})}{R_{P3}}}+cos{\phi }_{P3}{\displaystyle \frac{(v_{P3}sin{\theta }_{P3}-v_{E2}sin{\theta }_{E2})}{R_{P3}}},\\ sin{\theta }_{P3}cos{\phi }_{P3}tan{\theta }_{L3}+{\displaystyle \frac{v_{E2}cos{\theta }_{E2}sin{\phi }_{E2}-v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}}{R_{P3}cos{\theta }_{P3}}}\\ -sin{\theta }_{P3}sin{\phi }_{P3}{\displaystyle \frac{v_{E2}sin{\theta }_{E2}-v_{P3}sin{\theta }_{E2}}{R_{P3}cos{\theta }_{P3}}}-{\displaystyle \frac{v_{E2}cos{\theta }_{E2}sin{\phi }_{E2}-v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}}{R_{P3}}},\\ tan{\theta }_{L1}sin{\phi }_{E1}{\displaystyle \frac{(v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}-v_{E1}cos{\theta }_{E1}sin{\phi }_{E1})}{R_{P1}}}+cos{\phi }_{E1}{\displaystyle \frac{(v_{P1}sin{\theta }_{P1}-v_{E1}sin{\theta }_{E1})}{R_{P1}}},\\ sin{\theta }_{E1}cos{\phi }_{E1}tan{\theta }_{L1}+{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}}{R_{P1}cos{\theta }_{E1}}}\\ -sin{\theta }_{E1}sin{\phi }_{E1}{\displaystyle \frac{v_{E1}sin{\theta }_{E1}-v_{P1}sin{\theta }_{E1}}{R_{P1}cos{\theta }_{E1}}}-{\displaystyle \frac{v_{E1}cos{\theta }_{E1}sin{\phi }_{E1}-v_{P1}cos{\theta }_{P1}sin{\phi }_{P1}}{R_{P1}}},\\ tan{\theta }_{L3}sin{\phi }_{E2}{\displaystyle \frac{(v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}-v_{E2}cos{\theta }_{E3}sin{\phi }_{E3})}{R_{P3}}}+cos{\phi }_{E2}{\displaystyle \frac{(v_{P3}sin{\theta }_{P3}-v_{E2}sin{\theta }_{E2})}{R_{P3}}},\\ sin{\theta }_{E2}cos{\phi }_{E2}tan{\theta }_{L3}+{\displaystyle \frac{v_{E2}cos{\theta }_{E2}sin{\phi }_{E2}-v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}}{R_{P3}cos{\theta }_{E2}}}\\ -sin{\theta }_{E2}sin{\phi }_{E2}{\displaystyle \frac{v_{E2}sin{\theta }_{E2}-v_{P3}sin{\theta }_{E2}}{R_{P3}cos{\theta }_{E2}}}-{\displaystyle \frac{v_{E2}cos{\theta }_{E2}sin{\phi }_{E2}-v_{P3}cos{\theta }_{P3}sin{\phi }_{P3}}{R_{P3}}},\end{array}\right]$$

6. Pursuer Inputs $u_1$ and $u_2$:

$$u_1=\left[\begin{array}{c}{A}_{yP1}\\ A_{yP2}\\ A_{yP3}\end{array}\right],u_2=\left[\begin{array}{c}{A}_{zP1}\\ A_{zP2}\\ A_{zP3}\end{array}\right]$$

7. Evader Inputs $v_1$ and $v_2$:

$$v_1=\left[\begin{array}{c}{A}_{yE1}\\ A_{yE2}\end{array}\right],v_2=\left[\begin{array}{c}{A}_{zE1}\\ A_{zE2}\end{array}\right]$$

The cost function of the pursuers and the evader is as follows:

$$\begin{array}{cc}\hfill J(u,v)& =\mathbb{E}\left\{{\int }_t^T{e}^{-{\int }_t^{s}c\left(r\right)dr}\left(x^{T}Cx+u^{T}Du-v^T\Delta v\right)ds\right.+\left.e^{-{\int }_t^{T}c\left(r\right)dr}{x}^T\left(T\right)R_{T}x\left(T\right)\right\}\hfill \end{array}$$

(9)

where $e^{-{\int }_t^{s}c\left(r\right)dr}$ is the discounting function, and $c\left(r\right)$ is the function of time. $C\in {\mathbb{R}}^{m\times m}$, $D\in {\mathbb{R}}^{2n\times 2n}$, $\Delta \in {\mathbb{R}}^{2\times 2}$, $R_T\in {\mathbb{R}}^{m\times m}$, $u={[u_1,u_2]}^T$, $v={[v_1,v_2]}^T$, $D=\left[\begin{array}{cc}{D}_1& \\ & D_2\end{array}\right]$, $\Delta =\left[\begin{array}{cc}{\Delta }_1& \\ & {\Delta }_2\end{array}\right]$, $D,\Delta$ is the orthogonal matrix.

Assumption 1. 

For the convenience of subsequent calculations of the cost function $J\left(u,v\right)$, it is assumed that [9]:

• $L\left(s,x,u_1,u_2,v_1,v_2\right)=x^{T}Cx+u^{T}Du-v^T\Delta v$, $g\left(x\right)=x{\left(s\right)}^T{R}_{T}x\left(s\right)$. There exist positive constants $C_L$, $C_g$, and $p\ge 2$ such that:

$$\begin{array}{cc}\hfill & \left|e^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)\right|\le C_L\left(1+{∥x∥}^p+{\left|u_1\right|}^p+{\left|u_2\right|}^p+{\left|v_1\right|}^p+{\left|v_2\right|}^p\right)\hfill \end{array}$$

(10)

$$\left|e^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}g\left(x\right)\right|\le C_g\left(1+{∥x∥}^p\right)$$

(11)

Assumption 2. 

It could be assumed that $B=b+b_{u1}{u}_1+b_{u2}{u}_2+b_{v1}{v}_1+b_{v2}{v}_2$, and σ is continuous and bounded. There exist positive constants $C_b,C_u,C_{\sigma },C_{b\sigma }$ such that [18]:

$$\begin{array}{cc}\hfill & ∥B\left(s,x,u_1,u_2,v_1,v_2\right)-B\left(s,y,u_1,u_2,v_1,v_2\right)∥\le C_b∥x-y∥\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill & ∥B\left(s,x,u_1,u_2,v_1,v_2\right)-B\left(s,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)∥\hfill \\ \hfill & \begin{array}{cc}& \end{array}\le C_u\left({\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\right)\hfill \end{array}$$

(13)

$$∥\sigma \left(s,x,u_1,u_2,v_1,v_2\right)-\sigma \left(s,y,u_1,u_2,v_1,v_2\right)∥\le C_{\sigma }∥x-y∥$$

(14)

$$\begin{array}{cc}\hfill & ∥B\left(s,x,u_1,u_2,v_1,v_2\right)+∥\sigma \left(s,x,u_1,u_2,v_1,v_2\right)∥∥\le C_{b\sigma }\left(1+∥x∥\right)\hfill \end{array}$$

(15)

where $s\in [t,T].x,y\in {\mathbb{R}}^n$. The admissible control sets for pursuers and evader could be defined as follows:

$$U=\left\{\begin{array}{c}{u}_i\mid u_i\in \left[t,T\right]\times {\mathbb{R}}^{n\times 1},u_i\mathit{is}\mathit{uniformly}\mathit{locally}\mathit{Lipschitz}\mathit{continuous},\hfill \\ \left|u_i(s,x)\right|\le u_i^{max},i=1,2.\hfill \end{array}\right\}$$

(16)

$$\nu =\left\{\begin{array}{c}{v}_i\mid v_i\in \left[t,T\right]\times {\mathbb{R}}^n,v_i\mathit{is}\mathit{uniformly}\mathit{locally}\mathit{Lipschitz}\mathit{continuous},\hfill \\ \left|v_i(s,x)\right|\le v_i^{max},i=1,2.\hfill \end{array}\right\}$$

(17)

for all $s\in \left[t,T\right]$, $L\left(s,x,u_1,u_2,v_1,v_2\right)$ and $g\left(x\right)$ is continuously differentiable. In the missile interception model presented in this paper, we assume that the control inputs are bounded. Specifically, the trajectory control engine, which governs the missile’s movement, has an upper limit on its output, which is related to the normal acceleration. This assumption is crucial for modeling practical missile systems, where limitations on engine power and physical constraints on acceleration are common.

This assumption of bounded control inputs is supported by previous works in the field. For instance, references [34,35] provide theoretical justification for such constraints in control systems, while references [36,37] discuss their applicability to missile interception problems. Moreover, reference [38] highlights the impact of input constraints on system stability and performance, further validating the choice of this assumption in our model.

3. The Optimal Feedback Strategies

Definition 1. 

The triple strategy $\left(u_i^{*},v_i^{*}\right)\in \left(U,\nu \right),i=1,2$ is called to constitute a Nash equilibrium if the triple strategy is within the admissible control set, and if the following hold:

$${\displaystyle \underset{u_i^{*}\in U}{min}}J\left(u_1^{*},u_2^{*},v_1,v_2\right)\le {\displaystyle \underset{u_i^{*}\in U}{min}}{\displaystyle \underset{v_i^{*}\in \nu }{max}}J\left(u_1^{*},u_2^{*},v_1^{*},v_2^{*}\right)\le {\displaystyle \underset{v_i^{*}\in \nu }{max}}J\left(u_1,u_2,v_1^{*},v_2^{*}\right)$$

(18)

for any $\left[s,x\right]\in \left(t,T\right)\times {\mathbb{R}}^m$. In this context, the pursuer aims to minimize the profit function, whereas the evader seeks to maximize it.

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u\in U}{min}}{\displaystyle \underset{v\in \nu }{max}}J=\mathbb{E}\left\{{\int }_t^T{e}^{-{\int }_t^{s}c\left(r\right)dr}\left(x^T{C}_{s}x+u^{T}Du-v^T\Delta v\right)ds\right.\hfill \\ \hfill & \left.+e^{-{\int }_t^{T}c\left(r\right)dr}x{\left(T\right)}^T{R}_{T}x\left(T\right)\right\}\hfill \end{array}$$

(19)

For the second term of the cost function, A matrix $p\in {\mathbb{R}}^{m\times m}$ is a solution for the terminal term,

$$dp=\alpha ds+\beta dW=\left(-A^{T}p+\zeta \left(x-\mathbb{E}\left(x\right)\right)\right)ds+\beta dW$$

(20)

where $p\left(T\right)=R_T$, $\alpha =-A^{T}p+\zeta \left(x-\mathbb{E}\left(x\right)\right)$, $\zeta \left(x\right)$ is a function that depends on the state, $\zeta \left(0\right)=0.$ $\beta$ is a stochastic diffusion term. Applying Itô’s Lemma to compute $\langle p\left(T\right)x\left(T\right),x\left(T\right)\rangle$

$$\begin{array}{cc}\hfill & x{\left(T\right)}^T{R}_{T}x\left(T\right)=〈p\left(T\right)x\left(T\right),x\left(T\right)〉=〈p\left(t\right)x\left(t\right),x\left(t\right)〉\hfill \\ \hfill & +{\int }_t^T{x}^T\alpha x+x^{T}pB+B^{T}px+2{\sigma }^T\beta x+tr\left(p\sigma {\sigma }^T\right)ds+{\int }_t^T{x}^T\beta x+2{\sigma }^{T}pxdW.\hfill \end{array}$$

(21)

Combining (19) and (21), the following could be obtained:

$$\begin{array}{cc}\hfill & J=\mathbb{E}\left\{\begin{array}{c}{e}^{-{\int }_t^{s}c\left(r\right)dr}{\int }_t^T\left\{x^T{C}_{s}x+u^T{D}_{s}u-v^T{D}_{s}v\right\}ds+e^{-{\int }_t^{s}c\left(r\right)dr}{x}^{T}Rx\hfill \end{array}\right\}\hfill \\ \hfill & =\mathbb{E}\left\{\begin{array}{c}{e}^{-{\int }_t^{s}c\left(r\right)dr}{\int }_t^T\left\{\begin{array}{c}{x}^T{C}_{s}x+u^T{D}_{s}u-v^T{D}_{s}v+x^T\alpha x+2B^{T}px+2{\sigma }^T\beta x+tr\left(p\sigma {\sigma }^T\right)\hfill \end{array}\right\}ds\hfill \\ +e^{-{\int }_t^{s}c\left(r\right)dr}{\int }_t^T\beta x^{T}x+2p{\sigma }^{T}xdW+e^{-{\int }_t^{s}c\left(r\right)dr}〈p\left(t\right)x\left(t\right),x\left(t\right)〉\hfill \end{array}\right\}.\hfill \end{array}$$

(22)

The second term of the cost function (22) is equal to zero, and the third term is relative to the initial state, such that

$$\begin{array}{cc}\hfill & J=\mathbb{E}\left\{\begin{array}{c}{e}^{-{\int }_t^{s}c\left(r\right)dr}{\int }_t^T\left\{x^T{C}_{s}x+u^T{D}_{s}u-v^T{D}_{s}v\right\}ds+e^{-{\int }_t^{s}c\left(r\right)dr}{x}^{T}Rx\hfill \end{array}\right\}\hfill \\ \hfill & =\mathbb{E}\left\{{\int }_t^T{e}^{-{\int }_t^{s}c\left(r\right)dr}\left\{\begin{array}{c}{x}^T{C}_{s}x+u^T{D}_{s}u-v^T{D}_{s}v+x^T\alpha x+2B^{T}px+2{\sigma }^T\beta x+tr\left(p\sigma {\sigma }^T\right)\hfill \end{array}\right\}ds\right\}.\hfill \end{array}$$

(23)

In this paper, the $HJB$-equation for $\Psi \left(s,x\right)={\displaystyle \underset{u_i\in U}{inf}}{\displaystyle \underset{v_i\in \nu }{sup}}J\left(s,x\right)$ becomes:

$$\begin{array}{c}\hfill {\partial }_s\Psi +{\displaystyle \underset{u_i\in U}{inf}}{\displaystyle \underset{v_i\in \nu }{sup}}\left\{\begin{array}{c}{\displaystyle \frac{1}{2}}tr\left({\sigma }_s{{\sigma }^{*}}_s\right){\partial }_{xx}^2\Psi +〈b_s,{\partial }_x\Psi 〉+c\Psi +L\hfill \end{array}\right\}=0,s\in \left[t,T\right].\end{array}$$

(24)

Theorem 1. 

the value function $\Psi \left(t,x\right)\in C^{1,2}\left(\left[t,T\right]\times {\mathbb{R}}^m\right)$ is the classical solution of HJB, $\left(u_i^{*},v_i^{*}\right)\in \left(U,\nu \right)$, $\left|u_i\left(s,x\right)\right|\le u_i^{max},$$\left|v_i\left(s,x\right)\right|\le v_i^{max},i=1,2.$ such that $\left|\Psi \right|,\left|{\partial }_s\Psi \right|,\left|{\partial }_x\Psi \right|,\left|{\partial }_{xx}^2\Psi \right|\le C_{\Psi }\left(1+{∥x∥}^N\right)$, for some $N\ge 0$. If $(x^{*},u_1^{*},u_2^{*},v_1^{*},v_2^{*})$ is an admissible pair at $\left(t,x\right)$, $F_{cv}={\displaystyle \frac{1}{2}}tr\left({\sigma }_s{{\sigma }^{*}}_s\right){\partial }_{xx}^2\Psi +〈b_s,{\partial }_x\Psi 〉+c\Psi +L$, such that $\left(u_1^{*},u_2^{*},v_1^{*},v_2^{*}\right)\in arg{\displaystyle \underset{u_i\in U}{inf}}{\displaystyle \underset{v_i\in nu}{sup}}{F}_{cv}$. For almost everywhere, $s\in \left[t,T\right]$. Then, the pair $(x^{*},u_1^{*},u_2^{*},v_1^{*},v_2^{*})$ is optimal at $\left(t,x\right)$.

Proof. 

Based on Itô’s Lemma:

$$\begin{array}{cc}\hfill & \Psi \left(s,x\right)=\mathbb{E}\left\{\begin{array}{c}\Psi \left(T,x\right)-{\int }_t^T\left[{\partial }_s\Psi +〈b,{\partial }_x\Psi 〉+{\displaystyle \frac{1}{2}}tr\left({\sigma }_s{{\sigma }^{*}}_s\right){\partial }_{xx}^2\Psi \right]ds\hfill \end{array}\right\}\hfill \end{array}$$

(25)

Then, the following could be obtained:

$$\begin{array}{cc}\hfill & J\left(t,x,u_1^{*},u_2^{*},v_1,v_2\right)\le \mathbb{E}\left\{g\left(x\left(T\right)\right)-{\int }_t^T\left[{\partial }_s\Psi -L+F_{cv}-\Psi +\Psi \right]ds\right\}\le J\left(t,x,u_1,u_2,v_1^{*},v_2^{*}\right).\hfill \end{array}$$

(26)

If and only if $F_{cv}=\Psi$, $\Psi \left(t,x\right)=J\left(t,x,u_1,u_2,v_1,v_2\right)$. If $\left(u_1,u_2,v_1,v_2\right)=\left(u_1^{*},u_2^{*},v_1^{*},v_2^{*}\right)$, then the equation above gives: $\Psi \left(t,x\right)=V\left(t,x\right)=J\left(t,x,u_1^{*},u_2^{*},v_1^{*},v_2^{*}\right)$, where value function $V\left(t,x\right)$ is the viscosity solution of the HJB equation. It may be the case that a continuous first and second derivative $C^{1,2}$ does not exist. □

Step 1: the value function $V\left(t,x\right)$ is the Lipschitz continuous function.

Proof. 

We introduce three Lemmas 1–3, to complete the proof. To prove that the value function is Lipschitz continuous, it is necessary to demonstrate that each input parameter of the value function satisfies the Lipschitz continuity condition. This step ensures the mathematical rigor of the subsequent solution process. While the numerical simulations in this paper are conducted within the ${\mathbb{R}}^3$ space, the uniqueness of the value function as a solution holds true in higher-dimensional spaces as well. We provide a simplified overview of the lemmas used:

• Lemma 1: Proves that time is Lipschitz continuous, ensuring stability with respect to temporal variations.
• Lemma 2: Shows that the state is Lipschitz continuous, accounting for the relationship between states and dynamics.
• Lemma 3: Demonstrates that the control inputs are Lipschitz continuous under bounded constraints, ensuring consistency of input–output relationships.

These lemmas collectively establish the Lipschitz continuity of the value function, providing a foundation for the uniqueness proof in both low- and high-dimensional spaces.

Lemma 1. 

For any $s\in [t,T]$, $x_1,x_2\in {\mathbb{R}}^m$, there exists $C_{\lambda 1},C_{\lambda 2}$, such that:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{C_{\lambda 1}{∥x_1-x_2∥}^p\right\}\le \left|V\left(t,x_1,u_1,u_2,v_1,v_2\right)-V\left(t,x_2,u_1,u_2,v_1,v_2\right)\right|\le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_{\lambda 2}{∥x_1-x_2∥}^p\right\}.\hfill \end{array}$$

(27)

$V\left(t\right)$ is Lipschitz continuous.

Proof. 

For any $s\in [t,T]$, we can obtain the following:

$$\begin{array}{cc}\hfill & x_1\left(s\right)=x_1\left(t\right)+{\int }_t^{s}B\left(s,x_1\right)ds+{\int }_t^s\sigma \left(s,x_1\right)dW.x_2\left(s\right)=x_2\left(t\right)+{\int }_t^{s}B\left(s,x_2\right)ds+{\int }_t^s\sigma \left(s,x_2\right)dW.\hfill \end{array}$$

(28)

Then,

$$\begin{array}{cc}\hfill & x_1\left(s\right)-x_2\left(s\right)=x_1\left(t\right)-x_2\left(t\right)+{\int }_t^{s}B\left(s,x_1\right)-B\left(s,x_2\right)ds+{\int }_t^s\sigma \left(s,x_1\right)-\sigma \left(s,x_2\right)dW.\hfill \end{array}$$

(29)

Taking expectations on both sides of the equation:

$$\begin{array}{cc}\hfill \mathbb{E}\left\{∥x_1\left(s\right)-x_2\left(s\right)∥\right\}& \le \mathbb{E}\left\{∥x_1\left(t\right)-x_2\left(t\right)∥\right.\left.+{\int }_t^s{C}_b∥x_1-x_2∥ds\right\}\hfill \end{array}$$

(30)

According to the Gronwall inequality, there exists a positive constant $C_b$ such that:

$$\mathbb{E}\left\{∥x_1\left(s\right)-x_2\left(s\right)∥\right\}\le \mathbb{E}\left\{e^{C_b\left(s-t\right)}∥x_1\left(t\right)-x_2\left(t\right)∥\right\}$$

(31)

for any $s-t\le T$, such that:

$$\mathbb{E}\left\{∥x_1\left(s\right)-x_2\left(s\right)∥\right\}\le \mathbb{E}\left\{e^{C_{b}T}∥x_1\left(t\right)-x_2\left(t\right)∥\right\}$$

(32)

$\forall \epsilon >0$, $x_1,x_2\in {\mathbb{R}}^n$, such that:

$$\begin{array}{cc}\hfill & V\left(t,x_2,u_1,u_2,v_1,v_2\right)+\epsilon \ge {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{\begin{array}{c}{\int }_t^T{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x_1,u_1,u_2,v_1,v_2\right)ds+e^{-{\int }_t^{s}c\left(r\right)dr}g\left(x\left(T\right)\right)\hfill \end{array}\right\}\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill & V\left(t,x_1,u_1,u_2,v_1,v_2\right)-V\left(t,x_2,u_1,u_2,v_1,v_2\right)-\epsilon \hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{\begin{array}{c}{\int }_t^T{e}^{-{\int }_t^{s}c\left(r\right)dr}\left(\begin{array}{c}L\left(s,x_1,u_1,u_2,v_1,v_2\right)-L\left(s,x_2,u_1,u_2,v_1,v_2\right)\hfill \end{array}\right)ds\hfill \\ +e^{-{\int }_t^{s}c\left(r\right)dr}g\left(x_1\left(T\right)\right)-e^{-{\int }_t^{s}c\left(r\right)dr}g\left(x_2\left(T\right)\right)\hfill \end{array}\right\}\hfill \end{array}$$

(34)

There exist constants $C_L$ and $C_g$, such that:

$$\begin{array}{cc}\hfill & V\left(t,x_1,u_1,u_2,v_1,v_2\right)-V\left(t,x_2,u_1,u_2,v_1,v_2\right)-\epsilon \hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_t^T{C}_L\left({∥x_1-x_2∥}^p\right)ds+C_g{∥x_1\left(T\right)-x_2\left(T\right)∥}^p\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_L{e}^{C_{b}T}{∥x_1-x_2∥}^{p}T+C_g{e}^{C_{b}T}{∥x_1-x_2∥}^p\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_{\lambda }{∥x_1-x_2∥}^p\right\}.\hfill \end{array}$$

(35)

Similarly, we could obtain the following:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{C_{\lambda 1}{∥x_1-x_2∥}^p\right\}\le \left|V\left(t,x_1,u_1,u_2,v_1,v_2\right)-V\left(t,x_2,u_1,u_2,v_1,v_2\right)\right|\le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_{\lambda 2}{∥x_1-x_2∥}^p\right\}\hfill \end{array}$$

(36)

where $C_{\lambda }$ is a constant, which depends on $T,C_L,C_b,C_g$. This completes the proof of Lemma 1. □

Lemma 2. 

For any $x\in {\mathbb{R}}^m$, $V\left(x\right)$ is Lipschitz continuous. There exist constants $k_1,k_2$, such that:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{k_2{\left(1+∥x∥\right)}^{P}T\right\}\le V\left(t_1,x\left(t_1\right)\right)-V\left(t_2,x\left(t_1\right)\right)\le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{k_1{\left(1+∥x∥\right)}^{P}T\right\}.\hfill \end{array}$$

(37)

Proof. 

$\forall t_1\le t_2,t_1,t_2\in \left[0,T\right]$.

$$x\left(t_2\right)-x\left(t_1\right)={\int }_{t_1}^{t_2}B\left(s,x\right)ds+{\int }_{t_1}^{t_2}\sigma \left(s,x\right)dW$$

(38)

Then,

$$\mathbb{E}\left\{∥x\left(t_2\right)-x\left(t_1\right)∥\right\}=\mathbb{E}\left\{∥{\int }_{t_1}^{t_2}B\left(s,x\right)ds∥\right\}.$$

(39)

We could obtain the following:

$$\begin{array}{cc}\hfill & \mathbb{E}\left\{∥B\left(t,x,u_1,u_2,v_1,v_2\right)-B\left(t,0,u_1,u_2,v_1,v_2\right)∥\right\}\le \mathbb{E}\left\{C_b∥x∥\right\},\hfill \\ \hfill & \mathbb{E}\left\{∥B\left(t,x,u_1,u_2,v_1,v_2\right)∥\right\}\le \mathbb{E}\left\{∥B\left(t,0,u_1,u_2,v_1,v_2\right)∥+C_b∥x∥\right\}\hfill \\ \hfill & \le \mathbb{E}\left\{{\displaystyle \underset{v_i\in \nu }{max}}∥B\left(t,0,u_1,u_2,v_1,v_2\right)∥+C_b∥x∥\right\}\hfill \\ \hfill & \le \mathbb{E}\left\{M_{\mathcal{F}}+C_b∥x∥\right\},\hfill \end{array}$$

(40)

where $M_{\mathcal{F}}:={\displaystyle {\displaystyle \underset{v_i\in \nu }{max}}}∥B\left(0,u_1,u_2,v_1,v_2\right)∥.$

$$\begin{array}{cc}\hfill & \mathbb{E}\left\{{\displaystyle \underset{u_i\in U}{min}}∥B\left(0,u_1,u_2,v_1,v_2\right)∥-C_b∥x∥\right\}\le \mathbb{E}\left\{∥B\left(x,u_1,u_2,v_1,v_2\right)∥\right\}\hfill \end{array}$$

(41)

where $m_{\mathcal{F}}:={\displaystyle {\displaystyle \underset{u_i\in U}{min}}}∥B\left(0,u_1,u_2,v_1,v_2\right)∥$. Then, $\forall x\in {\mathbb{R}}^n,t_1,t_2\in \left[0,T\right]$.

$$\begin{array}{cc}\hfill & V\left(t_1,x\right)={\displaystyle \underset{u_i\in U}{inf}}{\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{\begin{array}{c}{\int }_{t_1}^{t_1+h}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds+V\left(t_2,x\right)\hfill \end{array}\right\}\hfill \end{array}$$

(42)

where $h=t_2-t_1$. The following could be obtained:

$$\begin{array}{cc}\hfill & V\left(t_1,x\left(t_1\right)\right)-V\left(t_2,x\left(t_1\right)\right)\hfill \\ \hfill & \begin{array}{ccc}& & \end{array}\le V\left(t_2,x\left(t_2\right)\right)-V\left(t_2,x\left(t_1\right)\right)+{\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_{t_1}^{t_2}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & \begin{array}{ccc}& & \end{array}\le \mathbb{E}\left\{C_{\lambda }{∥x\left(t_2\right)-x\left(t_1\right)∥}^p\right\}+{\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_{t_1}^{t_2}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{\begin{array}{c}{C}_{\lambda }{\left(M_{\mathcal{F}}+C_b∥x∥\right)}^{P}T+{\left(M_L+C_M∥x∥\right)}^{P}T\hfill \end{array}\right\}\le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{k_1{\left(1+∥x∥\right)}^{P}T\right\}.\hfill \end{array}$$

(43)

On the other hand, for any $\delta >0$, such that:

$$\begin{array}{cc}\hfill & V\left(t_1,x\left(t_1\right)\right)-V\left(t_2,x\left(t_1\right)\right)+\delta \hfill \\ \hfill & \begin{array}{ccc}& & \end{array}\ge V\left(t_2,x\left(t_2\right)\right)-V\left(t_2,x\left(t_1\right)\right)+{\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{{\int }_{t_1}^{t_2}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & \begin{array}{ccc}& & \end{array}\ge {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{k_2{\left(1+∥x∥\right)}^{P}T\right\}.\hfill \end{array}$$

(44)

Then, let $\delta \to 0$. We can obtain the following:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{k{\left(1+∥x∥\right)}^{P}T\right\}\le V\left(t_1,x\left(t_1\right)\right)-V\left(t_2,x\left(t_1\right)\right)\le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{k{\left(1+∥x∥\right)}^{P}T\right\}.\hfill \end{array}$$

(45)

This completes the proof of Lemma 2. □

Then, we need to add a proof regarding the impact of bounded inputs on value functions.

Lemma 3. 

$\forall \left(u_1,u_2,{\overline{u}}_1,{\overline{u}}_2\right)\in U,{\overline{u}}_i=u^{max},\left(v_1,v_2,{\overline{v}}_1,{\overline{v}}_2\right)\in \nu ,{\overline{v}}_i=v^{max}$, there exist constants $C_{L1},C_{L2},$ such that:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{C_{L1}{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\right\}\hfill \\ \hfill & \begin{array}{ccc}& & \end{array}\le V\left(t,x,u_1,u_2,v_1,v_2\right)-V\left(t,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_{L2}{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\right\}.\hfill \end{array}$$

(46)

Proof. 

$\exists s\in \left[t,T\right],x\in {\mathbb{R}}^N,$ there exists a constant $C_{L1}$, such that:

$$\begin{array}{cc}\hfill & V\left(t,x,u_1,u_2,v_1,v_2\right)-V\left(t,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_t^T{e}^{-C_{e}s}\left(\begin{array}{c}L\left(s,x,u_1,u_2,v_1,v_2\right)-L\left(s,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\hfill \end{array}\right)ds\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_t^T{C}_{L1}∥\begin{array}{c}1+{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}∥ds\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_{L1}{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\right\}.\hfill \end{array}$$

(47)

Then, there exists a constant $C_{L2}$, such that:

$$\begin{array}{cc}\hfill & V\left(t,x,u_1,u_2,v_1,v_2\right)-V\left(t,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_t^T{e}^{-C_{e}s}\left(\begin{array}{c}L\left(s,x,u_1,u_2,v_1,v_2\right)-L\left(s,x,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\hfill \end{array}\right)ds\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{{\int }_t^T{C}_L∥\begin{array}{c}1+{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}∥ds\right\}\hfill \\ \hfill & \le {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{C_L{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\right\}.\hfill \end{array}$$

(48)

This completes the proof of Lemma 3. □

According to Lemmas 1–3, we can obtain the following:

$$\begin{array}{cc}\hfill & \left|V\left(t_1,x_1,u_1,u_2,v_1,v_2\right)-V\left(t_2,x_2,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\right|\hfill \\ \hfill & \le \left|V\left(t_1,x_1,u_1,u_2,v_1,v_2\right)-V\left(t_1,x_2,u_1,u_2,v_1,v_2\right)\right|\hfill \\ \hfill & +\left|V\left(t_1,x_2,u_1,u_2,v_1,v_2\right)-V\left(t_2,x_2,u_1,u_2,v_1,v_2\right)\right|\hfill \\ \hfill & +\left|V\left(t_2,x_2,u_1,u_2,v_1,v_2\right)-V\left(t_2,x_2,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\right|.\hfill \end{array}$$

(49)

Then, we could obtain the following:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{\begin{array}{c}{C}_{L1}{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\hfill \\ +C_{\lambda 1}{∥x_1-x_2∥}^p+k_1{\left(1+∥x∥\right)}^{P}T\hfill \end{array}\right\}\hfill \\ \hfill & \le \left|V\left(t_1,x_1,u_1,u_2,v_1,v_2\right)-V\left(t_2,x_2,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)\right|\le \hfill \\ \hfill & {\displaystyle \underset{v_i\in \nu }{sup}}\mathbb{E}\left\{\begin{array}{c}{C}_L{e}^{C_{u}T}T\left(\begin{array}{c}{\left|u_1-{\overline{u}}_1\right|}^p+{\left|u_2-{\overline{u}}_2\right|}^p+{\left|v_1-{\overline{v}}_1\right|}^p+{\left|v_2-{\overline{v}}_2\right|}^p\hfill \end{array}\right)\hfill \\ +C_{\lambda }{∥x_1-x_2∥}^p+k{\left(1+∥x∥\right)}^{P}T\hfill \end{array}\right\}.\hfill \end{array}$$

(50)

This completes the proof of step 1. □

Step 2: the value function $V\left(t,x\right)$ is the unique viscosity solution of the $HJB$ equation.

Proof. 

We assume that $V_1$ is a sub-solution. Let $\mu$ be the standard complete probability space: $\mu =\left({\Omega }_1,\mathcal{F},{\mathcal{F}}_s^t,\mathcal{P}\right),$ where $W_{\eta }$ is a Wiener process in the filtration probability space $\left(\Omega ,\mathcal{F},{\mathcal{F}}_s^t,\mathcal{P}\right).$ Here, $\mathcal{F}$ is the augmentation by $\mathcal{P}$-sets, and ${\mathcal{F}}_s^t$ is the $\sigma$-algebra generated by $W_{\eta }$.

$$\begin{array}{cc}\hfill & J\left(t,x,u_1,u_2,v_1,v_2\right)=\hfill \\ \hfill & \mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & +\mathbb{E}\left\{\begin{array}{c}{\int }_{t+h}^T{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds+e^{-{\int }_t^{s}c\left(r\right)dr}g\left(x\left(T\right)\right)|{\mathcal{F}}_{t+h}^T\hfill \end{array}\right\}\hfill \\ \hfill & =\mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_t^{s}c\left(s\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & +\mathbb{E}\left\{\mathbb{E}\left\{\begin{array}{c}{\int }_{t+h}^T{e}^{-{\int }_{t+h}^{s}c\left(s\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds+e^{-{\int }_t^{s}c\left(s\right)dr}g\left(x\left(T\right)\right)|{\mathcal{F}}_{t+h}^T\hfill \end{array}\right\}\right\}\hfill \\ \hfill & =\mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_t^{s}c\left(s\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\hfill \\ \hfill & +\mathbb{E}\left\{e^{-{\int }_{t+h}^{s}c\left(r\right)dr}{J}^u\left(t+h,x,u_1,u_2,v_1,v_2\right)+\varsigma \right\}\hfill \\ \hfill & \ge \mathbb{E}\left\{{\int }_t^{t+h}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}+{\displaystyle \underset{u_i\in U}{inf}}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}{V}_1\left(t+h,x,u_1,u_2,v_1,v_2\right).\hfill \end{array}$$

(51)

Then, we can obtain the following:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{u_i\in U}{inf}}{V}_1\left(t,x,u_1,u_2,v_1,v_2\right)\ge \hfill \\ \hfill & \mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_t^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}+{\displaystyle \underset{u_i\in U}{inf}}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}V\left(t+h,x,u_1,u_2,v_1,v_2\right).\hfill \\ \hfill & {\displaystyle \underset{u_i\in U}{inf}}{V}_1\left(t,x,u_1,u_2,v_1,v_2\right)-{\displaystyle \underset{u_i\in U}{inf}}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}{V}_1\left(t+h,x,u_1,u_2,v_1,v_2\right)\hfill \\ \hfill & \ge \mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}.\hfill \\ \hfill & \mathbb{E}\left\{{\int }_t^{t+h}{e}^{-C_{e}s}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}+{\displaystyle \underset{u_i\in U}{inf}}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}{V}_1\left(t+h,x,u_1,u_2,v_1,v_2\right)\hfill \\ \hfill & -{\displaystyle \underset{u_i\in U}{inf}}{V}_1\left(t,x,u_1,u_2,v_1,v_2\right)\le 0.\hfill \\ \hfill & {\displaystyle \underset{u_i\in U}{inf}}{V}_1\left(t,x,u_1,u_2,v_1,v_2\right)-{\displaystyle \underset{u_i\in U}{inf}}{e}^{-{\int }_{t+h}^{T}c\left(r\right)dr}{V}_1\left(t+h,x,u_1,u_2,v_1,v_2\right)\hfill \\ \hfill & -\mathbb{E}\left\{{\int }_t^{t+h}{e}^{-{\int }_{t+h}^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\ge 0.\hfill \end{array}$$

(52)

For any $\left(t_0,x_0\right)\in \left(t,T\right)\times {\mathbb{R}}^n$, and there exists ${\Psi }_1\in C^{1,2}(\left(t,T\right)\times {\mathbb{R}}^n)$, with $V_1-{\Psi }_1$ attaining its maximum at $\left(t_0,x_0\right)$, we have:

$$\begin{array}{cc}& e^{-{\int }_{t+h}^{T}c\left(r\right)dr}{V}_1\left(t,x\right)-e^{-{\int }_{t+h}^{T}c\left(r\right)dr}{\Psi }_1\left(t,x\right)\le V_1\left(t_0,x_0\right)-{\Psi }_1\left(t_0,x_0\right)\hfill \end{array}$$

where $\forall \left(t_0,x_0\right)\in \left(t,T\right)\times {\mathbb{R}}^n$.

$$\begin{array}{cc}& e^{-{\int }_{t_0+h}^{T}c\left(r\right)dr}{V}_1\left(t_0+h,x_0\right)-e^{-{\int }_{t_0+h}^{T}c\left(r\right)dr}{\Psi }_1\left(t_0+h,x_0+h\right)\le V_1\left(t_0,x_0\right)-{\Psi }_1\left(t_0,x_0\right).\hfill \end{array}$$

$$\begin{array}{cc}& {\Psi }_1\left(t_0,x_0\right)-e^{-{\int }_{t_0+h}^{T}c\left(r\right)dr}{\Psi }_1\left(t_0+h,x_0+h\right)\le V\left(t_0,x_0\right)-e^{-{\int }_{t_0+h}^{T}c\left(r\right)dr}V\left(t_0+h,x_0\right).\hfill \end{array}$$

$$\begin{array}{cc}& {\Psi }_1\left(t_0,x_0\right)-e^{-{\int }_{t_0+h}^{T}c\left(r\right)dr}{\Psi }_1\left(t_0+h,x_0+h\right)-\mathbb{E}\left\{{\int }_{t_0}^{t_0+h}{e}^{-{\int }_{t_0}^{s}c\left(r\right)dr}L\left(s,x,u_1,u_2,v_1,v_2\right)ds\right\}\ge 0.\hfill \end{array}$$

Then, we can obtain the following:

$${\displaystyle \underset{u_i\in U}{inf}}\mathbb{E}\left\{\begin{array}{cc}& {\displaystyle \frac{1}{h}}{\int }_{t_0}^{t_0+h}-e^{-{\int }_{t_0}^{s}c\left(r\right)dr}{\partial }_s{\Psi }_1-e^{-{\int }_{t_0}^{s}c\left(r\right)dr}{B}^T{\partial }_x{\Psi }_1-e^{-{\int }_{t_0}^{s}c\left(r\right)dr}{\displaystyle \frac{1}{2}}\left({\sigma }_s^T{\sigma }_s\right){\partial }_{xx}^2{\Psi }_1\hfill \\ & -\left(1-e^{-{\int }_{t_0}^{s}c\left(r\right)dr}\right){\Psi }_1-e^{-{\int }_{t_0}^{s}c\left(r\right)dr}Lds\hfill \end{array}\right\}\ge 0.$$

$$\begin{array}{cc}\hfill & {\displaystyle \underset{h\to 0}{lim}}{\partial }_s{\Psi }_1+{\displaystyle \underset{u_i\in U}{inf}}\left(\begin{array}{c}{B}^T{\partial }_x{\Psi }_1+{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T{\partial }_{xx}^2{\Psi }_1+C_e{\Psi }_1+L\hfill \end{array}\right)\le 0.\hfill \end{array}$$

(53)

where $C_e={\displaystyle \underset{h\to 0}{lim}}{\displaystyle \frac{e^{-{\int }_{t_0}^{t_0+h}c\left(r\right)dr}-1}{h}}$. Similarly, $V_2$ is a supersolution. For any $\left(t_0,x_0\right)\in \left(t,T\right)\times {\mathbb{R}}^m$, there exists ${\Psi }_2\in C^{1,2}(\left(t,T\right)\times {\mathbb{R}}^n)$, such that $V_2-{\Psi }_2$ attains a minimum at $\left(t_0,x_0\right)$, and we have $V_2\left(t,x\right)-{\Psi }_2\left(t,x\right)\le V_2\left(t_0,x_0\right)-{\Psi }_2\left(t_0,x_0\right)$. Then, we have the following: ${\partial }_s{\Psi }_2+{\displaystyle \underset{v_i\in \nu }{sup}}(B{\partial }_x{\Psi }_2+{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T{\partial }_{xx}^2{\Psi }_2+C_e{\Psi }_2+L)\ge 0$. Next, we prove the existence of a unique solution V to ensure that the HJB equation has a unique solution. Let $M:={\displaystyle \underset{t\in \left[0,T\right]}{sup}}\{V_1-V_2\}$. Assuming by contradiction that $M>0$, we construct the following function using the doubling variable technique: for $\epsilon ,\beta ,\chi >0$ and $0<m1<1$.

$$\begin{array}{cc}\hfill & \Phi \left(t_1,x_1,u_1,u_2,v_1,v_2,t_2,x_2,u_1^{\prime },u_2^{\prime },v_1^{\prime },v_2^{\prime }\right)\hfill \\ \hfill & =V_1\left(t_1,x_1,u_1,u_2,v_1,v_2\right)-V_2\left(t_2,x_2,u_1^{\prime },u_2^{\prime },v_1^{\prime },v_2^{\prime }\right)-{\displaystyle \frac{{\left|t_1-t_2\right|}^p+{∥x_1-x_2∥}^p}{p\epsilon }}\hfill \\ \hfill & -\beta {\left(\sqrt{1+{∥x_1∥}^p}+\sqrt{1+{∥x_2∥}^p}\right)}^{m1}-\chi \left(∥u_1-u_1^{\prime }∥+∥u_2-u_2^{\prime }∥+∥v_1-v_1^{\prime }∥+∥v_2-v_2^{\prime }∥\right)\hfill \end{array}$$

(54)

where $\Phi$ is continuous and the last three terms of equation (54) are used to estimate the discontinuities. When

$$\Phi (t_1,x_1,u_1,u_2,v_1,v_2,t_2,x_2,u_1^{\prime },u_2^{\prime },v_1^{\prime },v_2^{\prime })\to -\infty ,$$

then $max\left\{∥x_1∥,∥x_2∥\right\}\to \infty$. Then, the following exists:

$$\begin{array}{cc}\hfill & sup\Phi \left(t_1,x_1,u_1,u_2,v_1,v_2,t_2,x_2,u_1^{\prime },u_2^{\prime },v_1^{\prime },v_2^{\prime }\right)=\Phi \left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2,{\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right).\hfill \end{array}$$

(55)

According to the definition of M, there exists $(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2)$, such that:

$$V_1\left(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2\right)-V_2\left(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2\right)\ge {\displaystyle \frac{M}{2}}$$

(56)

Then:

$$\begin{array}{cc}\hfill & \Phi \left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2,{\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right)\hfill \\ \hfill & \ge \Phi \left(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2,\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2\right)\hfill \\ \hfill & \ge V_1\left(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2\right)-V_2\left(\tilde{t},\tilde{x},{\tilde{u}}_1,{\tilde{u}}_2,{\tilde{v}}_1,{\tilde{v}}_2\right)-2\beta {\left(\sqrt{1+{∥\tilde{x}∥}^p}\right)}^{m1}\hfill \\ \hfill & -\chi \left(∥{\tilde{u}}_1-{\tilde{u}}_1∥+∥{\tilde{u}}_2-{\tilde{u}}_2∥+∥{\tilde{v}}_1-{\tilde{v}}_1∥+∥{\tilde{v}}_2-{\tilde{v}}_2∥\right)\hfill \\ \hfill & \ge {\displaystyle \frac{M}{2}}-2\beta {\left(\sqrt{1+{∥\tilde{x}∥}^p}\right)}^{m1}.\hfill \end{array}$$

(57)

There exist $\beta \in \mathbb{R}$, such that $\beta {\left(\sqrt{1+{∥\tilde{x}∥}^p}\right)}^{m1}={\displaystyle \frac{M}{8}}$. Then:

$$\Phi \left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2,{\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right)\ge {\displaystyle \frac{M}{4}}.$$

(58)

We can obtain the following:

$$\begin{array}{cc}\hfill & V_1\left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)-V_2\left({\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right)-{\displaystyle \frac{M}{4}}\ge \beta {\left(\sqrt{1+{∥{\overline{x}}_1∥}^p}+\sqrt{1+{∥{\overline{x}}_2∥}^p}\right)}^{m1}\hfill \end{array}$$

(59)

$V_1$ and $V_2$ are bound, then there exists a constant $C_x$ , such that:

$$\beta {\left(\sqrt{1+{∥{\overline{x}}_1∥}^p}+\sqrt{1+{∥{\overline{x}}_2∥}^p}\right)}^{m1}\le C_x$$

(60)

According to this assumption, we can obtain the following:

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{\left|{\overline{t}}_1-{\overline{t}}_2\right|}^p+{∥{\overline{x}}_1-{\overline{x}}_2∥}^p}{p\epsilon }}\le \hfill \\ \hfill & V_1\left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)-V_1\left({\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right)+V_2\left({\overline{t}}_1,{\overline{x}}_1,{\overline{u}}_1,{\overline{u}}_2,{\overline{v}}_1,{\overline{v}}_2\right)-V_2\left({\overline{t}}_2,{\overline{x}}_2,{\overline{u}}_1^{\prime },{\overline{u}}_2^{\prime },{\overline{v}}_1^{\prime },{\overline{v}}_2^{\prime }\right)\hfill \end{array}$$

(61)

We assume the existence of a continuity modulus $\omega$:

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{\left|{\overline{t}}_1-{\overline{t}}_2\right|}^p+{∥{\overline{x}}_1-{\overline{x}}_2∥}^p}{p\epsilon }}\hfill \\ \hfill & \le 2\omega \left(\begin{array}{c}\left|{\overline{t}}_1-{\overline{t}}_2\right|+∥{\overline{x}}_1-{\overline{x}}_2∥+∥{\overline{u}}_1-{\overline{u}}_1^{\prime }∥+∥{\overline{u}}_2-{\overline{u}}_2^{\prime }∥+∥{\overline{v}}_1-{\overline{v}}_1^{\prime }∥+∥{\overline{v}}_2-{\overline{v}}_2^{\prime }∥\hfill \end{array}\right)\hfill \end{array}$$

(62)

According to the state bounded, control inputs are strictly bounded, $\left|{\overline{t}}_1-{\overline{t}}_2\right|\le \sqrt{C\epsilon }$, $∥{\overline{x}}_1-{\overline{x}}_2∥\le \sqrt{C\epsilon }$. $∥{\overline{u}}_1-{\overline{u}}_1^{\prime }∥+∥{\overline{u}}_2-{\overline{u}}_2^{\prime }∥+∥{\overline{v}}_1-{\overline{v}}_1^{\prime }∥+∥{\overline{v}}_2-{\overline{v}}_2^{\prime }∥\le 4\sqrt{C\epsilon }$, we can obtain the following:

$${\displaystyle \frac{{\left|{\overline{t}}_1-{\overline{t}}_2\right|}^p+{∥{\overline{x}}_1-{\overline{x}}_2∥}^p}{p\epsilon }}\le 2\omega \left(6\sqrt{p\epsilon }\right),$$

(63)

$$\begin{array}{cc}\hfill & \chi \left(∥u_1-u_1^{\prime }∥+∥u_2-u_2^{\prime }∥+∥v_1-v_1^{\prime }∥+∥v_2-v_2^{\prime }∥\right)\le 4\chi \sqrt{C\epsilon }.\hfill \end{array}$$

(64)

Next, we assume:

$$\begin{array}{cc}\hfill & {\varphi }_1=V_2\left(t_2,x_2,u_1^{\prime },u_2^{\prime },v_1^{\prime },v_2^{\prime }\right)+{\displaystyle \frac{{\left|t_1-t_2\right|}^p+{∥x_1-x_2∥}^p}{p\epsilon }}+\beta {\left(\sqrt{1+{∥x_1∥}^p}+\sqrt{1+{∥x_2∥}^p}\right)}^{m1}\hfill \\ \hfill & +\chi \left(∥u_1-u_1^{\prime }∥+∥u_2-u_2^{\prime }∥+∥v_1-v_1^{\prime }∥+∥v_2-v_2^{\prime }∥\right).\hfill \end{array}$$

(65)

$$\begin{array}{cc}\hfill & {\varphi }_2=V_1\left(t_1,x_1,u_1,u_2,v_1,v_2\right)-{\displaystyle \frac{{\left|t_1-t_2\right|}^p+{∥x_1-x_2∥}^p}{p\epsilon }}\hfill \\ \hfill & -\beta {\left(\sqrt{1+{∥x_1∥}^p}+\sqrt{1+{∥x_2∥}^p}\right)}^{m1}-\chi \left(∥u_1-u_1^{\prime }∥+∥u_2-u_2^{\prime }∥+∥v_1-v_1^{\prime }∥+∥v_2-v_2^{\prime }∥\right).\hfill \end{array}$$

(66)

Incorporateing the above Equations (62)–(64) into the Equations (65) and (66), the following could be obtained:

$$\begin{array}{cc}\hfill & {\partial }_s{\varphi }_1+{\displaystyle \underset{u_i\in U}{inf}}\left(B^T{\partial }_x{\varphi }_1+{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T{\partial }_{xx}^2{\varphi }_1+C_e{V}_1+L\right)\hfill \\ \hfill & ={\displaystyle \frac{|t_1-t_2{|}^{p-1}}{\epsilon }}+C_e{V}_1(\overline{t_1},\overline{x_1})\hfill \\ \hfill & +B^T\left({\displaystyle \frac{|x_1-x_2{|}^{p-1}}{\epsilon }}+\beta m1{\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-1}{\displaystyle \frac{p|x_1{|}^{p-1}}{2\sqrt{1+|x_1{|}^p}}}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left({\displaystyle \frac{|x_1-x_2{|}^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-2}{\displaystyle \frac{p|x_1{|}^{p-1}}{2\sqrt{1+|x_1{|}^p}}}\right)\hfill \\ \hfill & +\beta m1{\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-1}{\displaystyle \frac{p(p-1)|x_1{|}^{p-2}2\sqrt{1+|x_1{|}^p}-p{\left|x_1\right|}^{p-1}}{{\left(2\sqrt{1+|x_1{|}^p}\right)}^2}}+L\le 0.\hfill \end{array}$$

(67)

We could obtain the following:

$$\begin{array}{cc}\hfill & {\partial }_s{\varphi }_2+{\displaystyle \underset{v_i\in \nu }{sup}}\left(B{\partial }_x{\varphi }_2+{\displaystyle \frac{1}{2}}\left({\sigma }_s^T{\sigma }_s\right){\partial }_{xx}^2\varphi +C_e{V}_2+L\right)\hfill \\ \hfill & ={\displaystyle \frac{|t_1-t_2{|}^{p-1}}{\epsilon }}+C_e{V}_2(\overline{t_2},\overline{x_2})\hfill \\ \hfill & +B^T\left({\displaystyle \frac{|x_1-x_2{|}^{p-1}}{\epsilon }}+\beta m1{\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-1}{\displaystyle \frac{p|x_2{|}^{p-1}}{2\sqrt{1+|x_2{|}^p}}}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left({\displaystyle \frac{|x_1-x_2{|}^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-2}{\displaystyle \frac{p|x_2{|}^{p-1}}{2\sqrt{1+|x_2{|}^p}}}\right)\hfill \\ \hfill & +\beta m1{\left(\sqrt{1+|x_1{|}^p}+\sqrt{1+|x_2{|}^p}\right)}^{m1-1}{\displaystyle \frac{p(p-1)|x_2{|}^{p-2}2\sqrt{1+|x_2{|}^p}-p{\left|x_2\right|}^{p-1}}{{\left(2\sqrt{1+|x_2{|}^p}\right)}^2}}\hfill \\ \hfill & +L\ge 0.\hfill \end{array}$$

(68)

Subtracting the above two Equations (67) and (68) yields the following:

$$\begin{array}{cc}\hfill & C_e\left(V_1(\overline{t_1},\overline{x_1})-V_2(\overline{t_2},\overline{x_2})\right)\hfill \\ \hfill & +B^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-1}}{\epsilon }}+\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}{\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}{\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\hfill \\ \hfill & +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}{\displaystyle \frac{p(p-1)\parallel x_1{\parallel }^{p-2}2\sqrt{1+\parallel x_1{\parallel }^p}-p{\parallel x_1\parallel }^{p-1}\left(\frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}\right)}{{\left(2\sqrt{1+\parallel x_1{\parallel }^p}\right)}^2}}\hfill \\ \hfill & -B^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-1}}{\epsilon }}+\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}{\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}\right)\hfill \\ \hfill & -{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{1-2}{\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}\right)\hfill \\ \hfill & +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}{\displaystyle \frac{p(p-1)\parallel x_2{\parallel }^{p-2}2\sqrt{1+\parallel x_2{\parallel }^p}-p{\parallel x_2\parallel }^{p-1}\left(\frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}\right)}{{\left(2\sqrt{1+\parallel x_2{\parallel }^p}\right)}^2}}\hfill \\ \hfill & \le 0.\hfill \end{array}$$

(69)

For the second term of Equation (69):

$$\begin{array}{cc}& B^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-1}}{\epsilon }}\right)+\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\left({\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}\right)\hfill \\ & -B^T\left({\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-1}}{\epsilon }}\right)-\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\left({\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\hfill \end{array}$$

$$=B^T\left(\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\left({\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}-{\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\right)$$

Then:

$$\begin{array}{cc}\hfill & =B^T\left(\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\left({\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}-{\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\right)\hfill \\ \hfill & =B^T\left(\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\left(C_p\parallel x_2-x_1\parallel \right)\right)\hfill \end{array}$$

(70)

for the third term of Equations (69) and (70).

$$\begin{array}{cc}\hfill & {\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left(\begin{array}{c}{\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}\times \left({\displaystyle \frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}}\right)\hfill \\ +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\hfill \\ \times {\displaystyle \frac{p(p-1)\parallel x_2{\parallel }^{p-2}2\sqrt{1+\parallel x_2{\parallel }^p}-p{\parallel x_2\parallel }^{p-1}\left(\frac{p\parallel x_2{\parallel }^{p-1}}{2\sqrt{1+\parallel x_2{\parallel }^p}}\right)}{{\left(2\sqrt{1+\parallel x_2{\parallel }^p}\right)}^2}}\hfill \end{array}\right)\hfill \\ \hfill & -{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left(\begin{array}{c}{\displaystyle \frac{\parallel x_1-x_2{\parallel }^{p-2}}{(p-1)\epsilon }}+\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}\times \left({\displaystyle \frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}}\right)\hfill \\ +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\hfill \\ \times {\displaystyle \frac{p(p-1)\parallel x_1{\parallel }^{p-2}2\sqrt{1+\parallel x_1{\parallel }^p}-p{\parallel x_1\parallel }^{p-1}\left(\frac{p\parallel x_1{\parallel }^{p-1}}{2\sqrt{1+\parallel x_1{\parallel }^p}}\right)}{{\left(2\sqrt{1+\parallel x_1{\parallel }^p}\right)}^2}}\hfill \end{array}\right)\hfill \end{array}$$

(71)

There exists constant $C_p$ and $C_{p2}$ , such that:

$${\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left(\begin{array}{c}\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}\times C_p\parallel x_2-x_1\parallel \hfill \\ +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\times C_{p2}\parallel x_2-x_1\parallel \hfill \end{array}\right)$$

(72)

Therefore, we can obtain the following:

$$\begin{array}{cc}\hfill & C_e\left(V_1\left({\overline{t}}_1,{\overline{x}}_1\right)-V_2\left({\overline{t}}_2,{\overline{x}}_2\right)\right)\hfill \\ \hfill & \le B^T\left(\begin{array}{c}\beta m{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\times C_p\parallel x_2-x_1\parallel \hfill \end{array}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left(\begin{array}{c}\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}\times C_p\parallel x_2-x_1\parallel \hfill \\ +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\times C_{p2}\parallel x_2-x_1\parallel \hfill \end{array}\right)\hfill \end{array}$$

(73)

$$\begin{array}{cc}\hfill & C_e{\displaystyle \frac{M}{4}}\le B^T\left(\begin{array}{c}\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\times C_p\parallel x_2-x_1\parallel \hfill \end{array}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\left({\sigma }_s^T{\sigma }_s\right)}^T\left(\begin{array}{c}\beta m1(m1-1){\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-2}\times C_p\parallel x_2-x_1\parallel \hfill \\ +\beta m1{\left(\sqrt{1+\parallel x_1{\parallel }^p}+\sqrt{1+\parallel x_2{\parallel }^p}\right)}^{m1-1}\times C_{p2}\parallel x_2-x_1\parallel \hfill \end{array}\right)\hfill \end{array}$$

(74)

Let $\beta \to 0$, and assume that $C_e>0$. Under this assumption, we obtain $M\le 0$, which contradicts the initial assumption that $M>0$. Therefore, we conclude that: $M={\displaystyle \underset{t\in \left[0,T\right]}{sup}}\left\{V_1-V_2\right\}\le 0.$ This implies: $V_1\le V_2.$ Similarly, if $V_1$ is a viscosity supersolution and $V_2$ is a viscosity subsolution, we can deduce that: $V_1\ge V_2.$ Thus, it can be concluded that: $V_1=V_2.$ This shows that when the control input is strictly bounded, the value function is the unique viscosity solution of the Hamilton–Jacobi–Bellman (HJB) equation. □

Viscosity solutions may not have first- or second-order continuity. Given that the problem involves stochastic elements, we introduce functions ${\Psi }_1$ and ${\Psi }_2$ to approximate $V_1$ and $V_2$, respectively, which allows us to represent the Hamilton–Jacobi–Bellman (HJB) equation. By using ${\Psi }_1-{\Psi }_2$ to approximate $V_1-V_2$, we avoid dealing with the discontinuities of $V_1$ and $V_2$ directly. Specifically, Formulas (65) and (66) allow us to remove the non-differentiable points of $V_1$ and $V_2$ through the introduction of boundedness conditions. Our ultimate goal is to prove that the upper bound of $V_1-V_2$ is less than zero, thereby confirming that $V_1\le V_2$. A similar proof can be used to show $V_1\ge V_2$.

We provide a detailed explanation of the derivation process for the HJB equation as a solution. To demonstrate the existence of a unique solution, two aspects must be addressed: Existence of a solution: The existence of a solution is established by proving that the value function is Lipschitz continuous. Uniqueness of the solution: The uniqueness of the solution is proven using a contradiction method. Specifically, by showing that the upper and lower solutions converge to the same value, we confirm the uniqueness of the solution. To support these points, we use Lemmas 1–3: Lemma 1 demonstrates that time is Lipschitz continuous. Lemma 2 establishes that the state is Lipschitz continuous. Lemma 3 confirms that under bounded control inputs, the control inputs are Lipschitz continuous. The derivation process for the HJB equation is shown in the Figure 4.

Next, we proceed to solve the $HJB$ equation. Let $\Psi (t,x)$ be the value function that solves the $HJB$ equation. The solution takes the following form:

$$\Psi \left(s,x\right)=x^T{S}_{1}x+S_2^{T}x+x^T{S}_3+S_4$$

(75)

where $S_1\in {\mathbb{R}}^{m\times m}$ with $S_1\left(T\right)=0$, $S_2\in {\mathbb{R}}^{m\times 1}$ with $S_2\left(T\right)=0$, $S_3\in {\mathbb{R}}^{m\times 1}$ with $S_3\left(T\right)=0$, and $S_4\in {\mathbb{R}}^{1\times 1}$ with $S_4\left(T\right)=0$. Using Equation (75), we obtain the following:

$$\begin{array}{cc}\hfill & f(s,x)=x^T{S}_1^{\prime }x+{\left(S_2^{\prime }\right)}^{T}x+x^T{S}_3^{\prime }+S_4^{\prime }\hfill \\ \hfill & +e^{-{\int }_t^{s}c\left(r\right)dr}\left\{\begin{array}{c}{x}^{T}Cx+u^{T}Du-v^T\Delta v+x^T\alpha x+x^{T}pB+B^{T}px+{\beta }^T\sigma x+x^T\sigma \beta +\mathrm{tr}\left(p{\sigma }^T\sigma \right)\hfill \end{array}\right\}\hfill \\ \hfill & +C_e\left(x^T{S}_{1}x+S_2^{T}x+x^T{S}_3+S_4\right)+B^T\left(S_{1}x+S_1^{T}x+S_2+S_3\right)+{\displaystyle \sum _{i,j}}\left({\left({\sigma }_s{\sigma }_s^T\right)}_{i,j}{S}_{1,i,j}\right)\hfill \end{array}$$

(76)

where $S_1^{\prime }={\displaystyle \frac{d}{ds}}{S}_1,S_2^{\prime }={\displaystyle \frac{d}{ds}}{S}_2,S_3^{\prime }={\displaystyle \frac{d}{ds}}{S}_3,S_4^{\prime }={\displaystyle \frac{d}{ds}}{S}_4$. According to the first-order condition optimality, the optimal feedback strategy could be expressed as follows:

$$\begin{array}{cc}\hfill & u_1^{*}=-D_1^{-1}\left({\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{2}}{b}_{u1}^T\left(S_{1}x+S_1^{T}x+S_2+S_3\right)+b_{u1}^{T}px\right),\hfill \\ \hfill & u_2^{*}=-D_2^{-1}\left({\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{2}}{b}_{u2}^T\left(S_{1}x+S_1^{T}x+S_2+S_3\right)+b_{u2}^{T}px\right),\hfill \\ \hfill & v_1^{*}={\Delta }_1^{-1}\left({\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{2}}{b}_{v1}^T\left(S_{1}x+S_1^{T}x+S_2+S_3\right)-b_{v1}^{T}px\right),\hfill \\ \hfill & v_2^{*}={\Delta }_2^{-1}\left({\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{2}}{b}_{v2}^T\left(S_{1}x+S_1^{T}x+S_2+S_3\right)-b_{v2}^{T}px\right).\hfill \end{array}$$

(77)

Next, by incorporating the optimal strategies from Equation (77) into the $HJB$ Equation (24), we can compare the quadratic, linear, and constant terms. This leads to the following coupled Riccati equations:

$$\begin{array}{cc}\hfill & S_1^{\prime }+e^{-{\int }_t^{s}c\left(r\right)dr}(\alpha +C)+C_e{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u1}{D}_1{b}_{u1}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u1}{D}_1{S}_1^T\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{S}_1^T{b}_{u1}{D}_1{b}_{u1}^{T}p+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{D}_1{b}_{u1}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{D}_1{S}_1^T\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{S}_1{D}_1{b}_{u1}^{T}p+{\displaystyle \frac{1}{2}}{p}^T{b}_{u1}{D}_1{b}_{u1}^T{S}_1+{\displaystyle \frac{1}{2}}{p}^T{b}_{u1}{D}_1{S}_1^T+e^{-{\int }_t^{s}c\left(r\right)dr}{p}^T{b}_{u1}{D}_1{b}_{u1}^{T}p\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u2}{D}_2{b}_{u2}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u2}{D}_2{S}_1^T+{\displaystyle \frac{1}{2}}{S}_1^T{b}_{u2}{D}_2{b}_{u2}^{T}p+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{D}_2{b}_{u2}^T{S}_1\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{D}_2{S}_1^T+{\displaystyle \frac{1}{2}}{S}_1{D}_2{b}_{u2}^{T}p+{\displaystyle \frac{1}{2}}{p}^T{b}_{u2}{D}_2{b}_{u2}^T{S}_1+{\displaystyle \frac{1}{2}}{p}^T{b}_{u2}{D}_2{S}_1^T+e^{-{\int }_t^{s}c\left(r\right)dr}{p}^T{b}_{u2}{D}_2{b}_{u2}^{T}p\hfill \\ \hfill & -{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v1}{\Delta }_1{b}_{v1}^T{S}_1-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v1}{\Delta }_1{S}_1^T+{\displaystyle \frac{1}{2}}{S}_1^T{b}_{v1}{D}_1{b}_{v1}^{T}p-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{\Delta }_1{b}_{v1}^T{S}_1\hfill \\ \hfill & -{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{\Delta }_1{S}_1^T+{\displaystyle \frac{1}{2}}{S}_1{\Delta }_1{b}_{v1}^{T}p+{\displaystyle \frac{1}{2}}{p}^T{b}_{v1}{\Delta }_1{b}_{v1}^T{S}_1+{\displaystyle \frac{1}{2}}{p}^T{b}_{v1}{\Delta }_1{S}_1^T\hfill \\ \hfill & -e^{-{\int }_t^{s}c\left(r\right)dr}{x}^T{p}^T{b}_{v1}{\Delta }_1{b}_{v1}^{T}px-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v2}{\Delta }_2{b}_{v2}^T{S}_1-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v2}{\Delta }_2{S}_1^T\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{S}_1^T{b}_{v2}{D}_2{b}_{v2}^{T}p-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{\Delta }_2{b}_{v2}^T{S}_1-{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{\Delta }_2{S}_1^T+{\displaystyle \frac{1}{2}}{S}_1{\Delta }_2{b}_{v2}^{T}p\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{p}^T{b}_{v2}{\Delta }_1{b}_{v2}^T{S}_1+{\displaystyle \frac{1}{2}}{p}^T{b}_{v2}{\Delta }_2{S}_1^T-e^{-{\int }_t^{s}c\left(r\right)dr}{x}^T{p}^T{b}_{v2}{\Delta }_2{b}_{v2}^{T}px=0.\hfill \end{array}$$

(78)

where $S_1\left(T\right)=0$.

$$\begin{array}{cc}\hfill & {\left(S_2^{\prime }\right)}^T+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{D}_1{b}_{u1}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{D}_1{b}_{u1}^T{S}_1^T+{\displaystyle \frac{1}{2}}{\left(S_2+S_3\right)}^T{D}_1{b}_{u1}^{T}p\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{D}_2{b}_{u2}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{D}_2{b}_{u2}^T{S}_1^T+{\displaystyle \frac{1}{2}}{\left(S_2+S_3\right)}^T{D}_2{b}_{u2}^{T}p\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{\Delta }_1{b}_{v1}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{\Delta }_1{b}_{v1}^T{S}_1^T+{\displaystyle \frac{1}{2}}{\left(S_2+S_3\right)}^T{\Delta }_1{b}_{v1}^{T}p\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{\Delta }_2{b}_{v2}^T{S}_1+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{\left(S_2+S_3\right)}^T{\Delta }_2{b}_{v2}^T{S}_1^T+{\displaystyle \frac{1}{2}}{\left(S_2+S_3\right)}^T{\Delta }_2{b}_{v2}^{T}p\hfill \\ \hfill & +e^{-{\int }_t^{s}c\left(r\right)dr}{B}^{T}p+e^{-{\int }_t^{s}c\left(r\right)dr}{\beta }^T\sigma +C_e{S}_2^T+B^T\left(S_1+S_1^T\right)=0.\hfill \end{array}$$

(79)

where $S_2\left(T\right)=0$.

$$\begin{array}{cc}\hfill & S_3^{\prime }+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u1}{D}_1(S_2+S_3)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{b}_{u1}{D}_1(S_2+S_3)+{\displaystyle \frac{1}{2}}{p}^T{b}_{u1}{D}_1(S_2+S_3)\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{u2}{D}_2(S_2+S_3)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{b}_{u2}{D}_2(S_2+S_3)+{\displaystyle \frac{1}{2}}{p}^T{b}_{u2}{D}_2(S_2+S_3)\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v1}{\Delta }_1(S_2+S_3)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{b}_{v1}{\Delta }_1(S_2+S_3)+{\displaystyle \frac{1}{2}}{p}^T{b}_{v1}{\Delta }_1(S_2+S_3)\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1^T{b}_{v2}{\Delta }_2(S_2+S_3)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}{S}_1{b}_{v2}{\Delta }_2(S_2+S_3)+{\displaystyle \frac{1}{2}}{p}^T{b}_{v2}{\Delta }_2(S_2+S_3)\hfill \\ \hfill & +e^{-{\int }_t^{s}c\left(r\right)dr}{B}^{T}p+e^{-{\int }_t^{s}c\left(r\right)dr}{\beta }^T\sigma +C_e{S}_3+B^T{S}_3.\hfill \end{array}$$

(80)

where $S_3\left(T\right)=0$.

$$\begin{array}{cc}\hfill & S_4^{\prime }+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}\left({(S_2+S_3)}^T{b}_{u1}{D}_1{b}_{u1}^T(S_2+S_3)\right)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}\left({(S_2+S_3)}^T{b}_{u2}{D}_2{b}_{u2}^T(S_2+S_3)\right)\hfill \\ \hfill & +{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}\left({(S_2+S_3)}^T{b}_{v1}{\Delta }_1{b}_{v1}^T(S_2+S_3)\right)+{\displaystyle \frac{e^{{\int }_t^{s}c\left(r\right)dr}}{4}}\left({(S_2+S_3)}^T{b}_{v2}{\Delta }_2{b}_{v2}^T(S_2+S_3)\right)\hfill \\ \hfill & +e^{-{\int }_t^{s}c\left(r\right)dr}tr\left(p\sigma {\sigma }^T\right)+C_e{S}_4+{\displaystyle \sum _{i,j}}\left({\left({\sigma }_s{\sigma }_s^T\right)}_{i,j}{S}_{1,i,j}\right)=0.\hfill \end{array}$$

(81)

where $S_4\left(T\right)=0$. Equations (8), (20) and (78)–(81) represent the forward–backward stochastic differential equations (FBSDEs). Next, we will demonstrate that the state is globally stable for any initial state $x_0$.

4. The Barrier Surface in the Stochastic Differential Game

Theorem 2. 

The solution $\left\{x\left(s\right),t\le s\le T\right\}$ of the system (8) under the optimal strategies (77) of the pursuers and evader is globally stable in the sense that for any initial state $x_0$, the following holds:

$${\displaystyle \underset{T\to \infty }{limsup}}{\displaystyle \frac{1}{T}}\mathbb{E}\left({\int }_t^T{∥x\left(s\right)∥}^{2}ds\right)<\infty$$

(82)

Proof. 

It can be assumed that $b\left(s,x\left(s\right)\right)\le \Phi \left(s\right)x\left(s\right)+{\gamma }_{\left[t\right]}\left(v\left(t\right)-v\left(\left[t\right]\right)\right),$ where $\Phi \left(s\right)$ is uniformly stable and converging to a random matrix $\Phi \left(\infty \right)$, and ${\Phi }^T\left(s\right)K\left(s\right)+K\left(s\right)\Phi \left(s\right)=-I.$ To facilitate estimation, we employ reduced excitation or exploration signals. We use a random sequence where ${\displaystyle \underset{k\to \infty }{lim}}{\gamma }_k=0$, with $min{\gamma }_k\ge k^{-{\displaystyle \frac{1}{5}}}$ and ${\gamma }_0=0$. An independent standard Wiener process sequence is selected to counteract the randomness of the states. It satisfies the following:

$${\displaystyle \underset{N\to \infty }{limsup}}{\displaystyle \frac{1}{N}}\sum _{k=1}^N{\int }_k^{k+1}{\gamma }_k^2{\left|v\left(t\right)-v\left(k\right)\right|}^{2}dt=0.$$

System (8) can now be rewritten as follows:

$$\begin{array}{cc}\hfill & dx\left(s\right)\le \left(\begin{array}{c}\Phi \left(s\right)x\left(s\right)+{\gamma }_{\left[t\right]}(v\left(t\right)-v\left(\left[t\right]\right))+b_{u1}\left(x\left(s\right)\right)u_1\left(s\right)+b_{u2}\left(x\left(s\right)\right)u_2\left(s\right)\hfill \\ +b_{v1}\left(x\left(s\right)\right)v_1\left(s\right)+b_{v2}\left(x\left(s\right)\right)v_2\left(s\right)\hfill \end{array}\right)ds\hfill \\ \hfill & +\sigma (s,x\left(s\right),u_1\left(s\right),u_2\left(s\right),v_1\left(s\right),v_2\left(s\right))dW\left(s\right)\hfill \end{array}$$

(83)

Let $\Omega ={\left[\begin{array}{ccc}{b}_{u1}& b_{u2}& \begin{array}{cc}{b}_{v1}& b_{v2}\end{array}\end{array}\right]}^T$, $\Sigma ={\left[\begin{array}{cccc}{u}_1& u_2& v_1& v_2\end{array}\right]}^T$. The following could be obtained:

$$\begin{array}{cc}\hfill & dx\left(s\right)\le \left(\Phi \left(s\right)x\left(s\right)+{\gamma }_{\left[t\right]}\left(v\left(t\right)-v\left(\left[t\right]\right)\right)+{\Omega }^T\Sigma \right)ds+\sigma dW\left(s\right)\hfill \end{array}$$

(84)

According to the closed optimal backward strategy (77), we could obtain the following:

$$\begin{array}{cc}\hfill & u_1=L_{u1}x+c_{u1},u_2=L_{u2}x+c_{u2},v_1=L_{v1}x+c_{v1},v_2=L_{v2}x+c_{v2},\Sigma =L_{uv}x+c_{uv}.\hfill \end{array}$$

(85)

By applying the $It\widehat{o}$ Lemma, the following can be obtained:

$$\begin{array}{cc}\hfill & d〈K\left(s\right)x\left(s\right),x\left(s\right)〉=\hfill \\ \hfill & 2〈\begin{array}{c}K\left(s\right)x\left(s\right),\Phi \left(s\right)x\left(s\right)+{\Omega }^T(L_{uv}x+c_{uv})+{\gamma }_{\left[t\right]}(v\left(t\right)-v\left(\left[t\right]\right))\hfill \end{array}〉dt\hfill \\ \hfill & +\mathrm{tr}\left(K\left(s\right)\sigma {\sigma }^T\right)dt+2〈K\left(s\right)x\left(s\right),\sigma dW\left(s\right)〉+\left(1-m_{uv}\right){\left|x\left(s\right)\right|}^{2}dt\hfill \\ \hfill & =2〈K\left(s\right)x\left(s\right),{\Omega }^T{c}_{uv}+{\gamma }_{\left[t\right]}(v\left(t\right)-v\left(\left[t\right]\right))〉dt+\mathrm{tr}\left(K\left(s\right)\sigma {\sigma }^T\right)dt+2〈K\left(s\right)x\left(s\right),\sigma dW\left(s\right)〉\hfill \end{array}$$

(86)

where $\left|{\int }_t^T〈K\left(s\right)x\left(s\right),\sigma dW\left(s\right)〉\right|=O\left({\left({\int }_t^T{\left|x\left(s\right)\right|}^{2}dt\right)}^{{\displaystyle \frac{1}{2}}+\epsilon }\right)$, $\epsilon \in \left(0,1/2\right)$. By integrating Equation (86) and applying the Cauchy–Schwarz inequality, we obtain the following:

$$\begin{array}{cc}\hfill & \sum _{i=\left[t\right]}^{\left[T\right]}\left(〈K(i-1)x\left(i\right),x\left(i\right)〉-〈K(i-1)x(i-1),x(i-1)〉\right)+〈K\left(\right[T\left]\right)x\left(T\right),x\left(T\right)〉-〈K\left(\right[T\left]\right)x\left(\right[T\left]\right),x\left(\right[T\left]\right)〉\hfill \\ \hfill & +〈K\left(\right[t\left]\right)x\left(t\right),x\left(t\right)〉-〈K\left(\right[t\left]\right)x\left(\right[t\left]\right),x\left(\right[t\left]\right)〉+{\int }_t^T\left(1-m_{uv}\right){\left|x\left(t\right)\right|}^{2}dt\hfill \\ \hfill & ={\int }_t^{T}2〈K\left(s\right)x\left(s\right),{\Omega }^T{c}_{uv}+{\gamma }_{\left[t\right]}\left(v\left(t\right)-v\left(\right[t\left]\right)\right)〉dt+{\int }_t^T\mathrm{tr}\left(K\left(s\right)\sigma {\sigma }^T\right)dt+{\int }_t^{T}2〈K\left(s\right)x\left(s\right),\sigma dW\left(s\right)〉\hfill \\ \hfill & =O\left({\left({\int }_t^T{\left|x\left(s\right)\right|}^{2}dt\right)}^{{\displaystyle \frac{1}{2}}}\right)+O\left({\left(\left({\int }_t^T{\left|x\left(s\right)\right|}^{2}dt\right){\int }_t^T{\gamma }_{\left[t\right]}^2{\left(v\left(t\right)-v\left(\right[t\left]\right)\right)}^{2}dt\right)}^{{\displaystyle \frac{1}{2}}}\right)+O\left(T\right)\hfill \\ \hfill & +O\left({\left({\int }_t^T{\left|x\left(s\right)\right|}^{2}dt\right)}^{{\displaystyle \frac{1}{2}}+\epsilon }\right)\hfill \end{array}$$

(87)

Then, according to Equation (87), and integrating the system Equation (83), the integral interval is $\left(k,k+1\right]$.

$$\begin{array}{cc}\hfill & x\left(k+1\right)\le e^{\Phi \left(k\right)}x\left(k\right)+{\int }_k^{k+1}{e}^{\left(k+1-s\right)\Phi \left(k\right)}\left({\gamma }_{\left[t\right]}\left(v\left(s\right)-v\left(\left[k\right]\right)\right)+{\Omega }^T{c}_{uv}\right)ds\hfill \\ \hfill & +{\int }_k^{k+1}{e}^{\left(k+1-s\right)\Phi \left(k\right)}\sigma dW\left(s\right)\hfill \end{array}$$

(88)

According to the Cauchy–Schwarz inequality, the following can be concluded:

$$\begin{array}{cc}\hfill & {\left|x\left(k+1\right)\right|}^2\le m{\left|x\left(k\right)\right|}^2+m_2{\left|{\int }_k^{k+1}\left({\Omega }^T{c}_{uv}\right)ds\right|}^2\hfill \\ \hfill & \begin{array}{cccc}& & & \end{array}+m_3{\int }_k^{k+1}\left({\gamma }_{\left[t\right]}{\left|v\left(s\right)-v\left(\left[k\right]\right)\right|}^2\right)ds+m_1{\left|{\int }_k^{k+1}{e}^{\left(k+1-s\right)\Phi \left(k\right)}\sigma dW\left(s\right)\right|}^2\hfill \end{array}$$

(89)

where $0<m<1,m_1,$$m_2,m_3>0$. Some fixed constants are relative to the maximum of $\left\{e^{\Phi \left(k\right)},k\in N\right\}$.

$$\begin{array}{cc}\hfill & \sum _{k=1}^N{\left|x\left(k\right)\right|}^2\le m\sum _{k=0}^{N-1}{\left|x\left(k\right)\right|}^2+m_2{\int }_0^{N-1}{\left|{\Omega }^T{c}_{uv}\right|}^{2}ds+m_3\sum _{i=0}^{N-1}{\int }_k^{k+1}\left({\gamma }_k^2{\left|v\left(s\right)-v\left(\left[k\right]\right)\right|}^2\right)ds\hfill \\ \hfill & \begin{array}{cccc}& & & \end{array}+m_1\sum _{k=0}^{N-1}{\left|{\int }_k^{k+1}{e}^{\left(k+1-s\right)\Phi \left(k\right)}\sigma dW\left(s\right)\right|}^2.\hfill \end{array}$$

(90)

The following could be obtained:

$$\begin{array}{cc}\hfill & {\displaystyle \underset{N\to \infty }{lim}}{\displaystyle \frac{\left(1-m\right)}{N}}\sum _{k=1}^N{\left|x\left(k\right)\right|}^2\le {\displaystyle \frac{m}{N}}{\left|x\left(0\right)\right|}^2+{\displaystyle \frac{m_2}{N}}{\int }_0^{N-1}{\left|{\Omega }^T{c}_{uv}\right|}^{2}ds\hfill \\ \hfill & \begin{array}{cccc}& & & \end{array}+{\displaystyle \frac{m_3}{N}}\sum _{i=0}^{N-1}{\int }_k^{k+1}\left({\gamma }_k^2{\left|v\left(s\right)-v\left(\left[k\right]\right)\right|}^2\right)ds+{\displaystyle \frac{m_1}{N}}\sum _{k=0}^{N-1}{\left|{\int }_k^{k+1}{e}^{\left(k+1-s\right)\Phi \left(k\right)}\sigma dW\left(s\right)\right|}^2.\hfill \end{array}$$

(91)

Since the second term is bounded and the expectation of the third and fourth terms are equal to zero, we obtain the following:

$${\displaystyle \underset{T\to \infty }{limsup}}\mathbb{E}\left({\displaystyle \frac{1}{T}}{\int }_t^T{\left|x\left(s\right)\right|}^{2}ds\right)<\infty .$$

This shows that for the state variables, as time approaches infinity, the norm of the state is a linear function of time. Hence, the proof is completed. □

Let $x^i\left(s\right):\left({\Omega }_i,{\mathcal{F}}_i,{\mathcal{P}}_i\right)\to \left(\Omega ,\mathcal{F}\right),s\in \left[t,T\right]$ be two stochastic processes. The processes $x^1\left(s\right)$ and $x^2\left(s\right)$ are said to have the same finite-dimensional distribution on $\left[t,T\right]$ if there exists a set D of full measures on $\left[t,T\right]$, such that for all $t\le t_1\le t_2\le \cdots \le t_n\le T$, where $t_i\in D$ and $A\in {\mathcal{F}}^{\otimes n}$ (with ⊗ denoting the tensor product). We have the following:

$$\begin{array}{cc}& {\mathcal{P}}_1\left(W_1,\left(x^1\left(t_1\right),x^1\left(t_2\right),\dots ,x^1\left(t_n\right)\right)\in A\right)={\mathcal{P}}_2\left(W_2,\left(x^2\left(t_1\right),x^2\left(t_2\right),\dots ,x^2\left(t_n\right)\right)\in A\right)\hfill \end{array}.$$

This can be written as ${\mathbb{L}}_{{\mathcal{P}}_1}\left(x^1\right)={\mathbb{L}}_{{\mathcal{P}}_2}\left(x^2\right)$, where $L_{{\mathcal{P}}_i}$ denotes the law of the process $x^i$ under probability measure $P_i$. Now, let ${\mu }_1=\left({\Omega }_1,{\mathcal{F}}_1,{\mathcal{F}}_s^{1,t},{\mathcal{P}}_1,W_1\right)$ and ${\mu }_2=\left({\Omega }_2,{\mathcal{F}}_2,{\mathcal{F}}_s^{2,t},{\mathcal{P}}_2,W_2\right)$ be two filtered probability spaces.

Theorem 3. 

Let ${\eta }_i\in {\mathbb{L}}^2\left({\Omega }_i,{\mathcal{F}}_s^{i,t},{\mathcal{P}}_i\right)$, and let $x^i\left(s\right)$ be the unique solution of the state equation given the control inputs $u_1^i,u_2^i\in U$, $v_1^i,v_2^i\in \nu$ with initial state $x^i\left(t\right)={\eta }_t$. If the following equality holds:

$${\mathbb{L}}_{{\mathcal{P}}_1}\left(u_1^1,u_2^1,v_1^1,v_2^1,W_1,{\eta }_1\right)={\mathbb{L}}_{{\mathcal{P}}_2}\left(u_1^2,u_2^2,v_1^2,v_2^2,W_2,{\eta }_2\right)$$

then we have:

$${\mathbb{L}}_{{\mathcal{P}}_1}\left(x^1,u_1^1,u_2^1,v_1^1,v_2^1\right)={\mathbb{L}}_{{\mathcal{P}}_2}\left(x^2,u_1^2,u_2^2,v_1^2,v_2^2\right)$$

Proof. 

$x^i\left(s\right)$ are the obtained limits of iterations of the maps. This implies the following:

$$\begin{array}{cc}\hfill & K_i\left[z^i\right]\left(s\right)={\eta }_t+{\int }_t^{s}B\left(r,z^i\left(r\right),u_1^i,u_2^i,v_1^i,v_2^i\right)dr+{\int }_t^s\sigma \left(r,z^i\left(r\right),u_1^i,u_2^i,v_1^i,v_2^i\right)dW\hfill \end{array}$$

(92)

where $z_1^i\left(s\right)={\eta }_i$, $z_{k+1}^i\left(s\right)=K_i\left[z_k^i\right]\left(s\right)$, $x^i\left(s\right)={\displaystyle \underset{k\to \infty }{lim}}{z}_{k+1}^i\left(s\right)$.

$$\begin{array}{cc}\hfill & z_k^1\left(s\right)-z_k^2\left(s\right)={\int }_t^{s}B\left(r,z_{k-1}^1\left(r\right),u_1^1,u_2^1,v_1^1,v_2^1\right)-B\left(r,z_{k-1}^2\left(r\right),,u_1^2,u_2^2,v_1^2,v_2^2\right)dr\hfill \\ \hfill & +{\int }_t^s\sigma \left(r,z_{k-1}^1\left(r\right),u_1^1,u_2^1,v_1^1,v_2^1\right)-\sigma \left(r,z_{k-1}^2\left(r\right),u_1^2,u_2^2,v_1^2,v_2^2\right)dW\hfill \end{array}$$

(93)

Hence, we use the Burkholder–Davis–Gundy inequalities in the first term and writing $C_z$ for the constant in the inequalities.

$$\begin{array}{cc}\hfill & \mathbb{E}\left({\left|z_k^1\left(s\right)-z_k^2\left(s\right)\right|}^2\right)\le 2\mathbb{E}\left(\begin{array}{c}{\left|{\int }_t^s\begin{array}{c}B\left(r,z_{k-1}^1\left(r\right),u_1^1,u_2^1,v_1^1,v_2^1\right)-B\left(r,z_{k-1}^1\left(r\right),,u_1^1,u_2^1,v_1^1,v_2^1\right)dr\hfill \end{array}\right|}^2\hfill \\ +{\left|{\int }_t^s\begin{array}{c}\sigma \left(r,z_{k-1}^2\left(r\right),u_1^2,u_2^2,v_1^2,v_2^2\right)-\sigma \left(r,z_{k-1}^2\left(r\right),u_1^2,u_2^2,v_1^2,v_2^2\right)dW\hfill \end{array}\right|}^2\hfill \end{array}\right)\hfill \\ \hfill & \le 2\left(\begin{array}{c}{C}_z\mathbb{E}\left({\int }_t^s{\left(\begin{array}{c}\sigma \left(r,z_{k-1}^1\left(r\right),u_1^1,u_2^1,v_1^1,v_2^1\right)-\sigma \left(r,z_{k-1}^1\left(r\right),u_1^1,u_2^1,v_1^1,v_2^1\right)\hfill \end{array}\right)}^{2}dW\right)\hfill \\ +T\mathbb{E}\left({\int }_t^s{\left(\begin{array}{c}B\left(r,z_{k-1}^2\left(r\right),u_1^2,u_2^2,v_1^2,v_2^2\right)-B\left(r,z_{k-1}^2\left(r\right),,u_1^2,u_2^2,v_1^2,v_2^2\right)\hfill \end{array}\right)}^{2}dr\right)\hfill \end{array}\right)\hfill \\ \hfill & \le 2\left(C_z+T\right)C_k^2\mathbb{E}{\int }_t^s{\left|z_{k-1}^1-z_{k-1}^2\right|}^{2}dr\le C_v\mathbb{E}{\int }_t^s{\left|z_{k-1}^1-z_{k-1}^2\right|}^{2}dr\hfill \end{array}$$

(94)

where $C_v=2\left(C_z+T\right)C_k^2$. According to this definition, we could have the following:

$${\mathbb{L}}_{{\mathcal{P}}_1}\left(z_k^1,w_1,u_1^1,u_2^1,v_1^1,v_2^1\right)={\mathbb{L}}_{{\mathcal{P}}_2}\left(z_k^2,w_2,u_1^2,u_2^2,v_1^2,v_2^2\right).$$

(95)

This implies the following:

$$\begin{array}{cc}\hfill & \mathbb{E}\left({\left|x^1\left(s\right)-x^2\left(s\right)\right|}^2\right)={\displaystyle \underset{k\to \infty }{lim}}\mathbb{E}\left({\left|z_k^1\left(s\right)-z_k^2\left(s\right)\right|}^2\right)\le {\displaystyle \underset{k\to \infty }{lim}}{C}_{v1}\mathbb{E}{\int }_t^s{\left|z_{k-1}^1-z_{k-1}^2\right|}^{2}dr\hfill \\ \hfill & ...\hfill \\ \hfill & \le {\displaystyle \underset{k\to \infty }{lim}}{C}_{vn}\mathbb{E}{\int }_t^s{\left|z_1^1-z_1^2\right|}^{2}dr\le C_v\mathbb{E}{\int }_t^s{\left|{\eta }_t-{\eta }_t\right|}^{2}dr\le 0.\hfill \end{array}$$

(96)

Therefore, passing to the limit as $k\to \infty$ gives the following result: $\mathbb{E}\left(x^1\left(s\right)\right)=\mathbb{E}\left(x^2\left(s\right)\right)$. This completes the proof. □

Theorem 4. 

According to Assumptions 1 and 2, there exists a unique solution for $(s,x,\alpha ,u_1^{*},u_2^{*},v_1^{*},v_2^{*})$$\in [t,T]\times {\mathbb{R}}^{m\times 1}\times {\mathbb{R}}^{m\times m}\times {\mathbb{R}}^{n\times 1}\times {\mathbb{R}}^{n\times 1}\times {\mathbb{R}}^{1\times 1}\times {\mathbb{R}}^{1\times 1}$ to the forward–backward stochastic differential equations (FBSDEs).

Proof. 

(The proof by contradiction): Suppose that there exist two solutions, $\left(x^1,{\alpha }^1,u_1^1,u_2^1,v_1^1,v_2^1\right)$, $\left(x^2,{\alpha }^2,u_1^2,u_2^2,v_1^2,v_2^2\right)$, and denote $\widehat{x}=x^1-x^2,\widehat{\alpha }={\alpha }^1-{\alpha }^2,{\widehat{u}}_1=u_1^1-u_1^2,{\widehat{u}}_2=u_2^1-u_2^2,$${\widehat{v}}_1=v_1^1-v_1^2.$ Then, we can obtain the following:

$$\left\{\begin{array}{c}d\widehat{x}\left(s\right)=\left(b+b_{u1}{\widehat{u}}_1\left(s\right)+b_{u2}{\widehat{u}}_2\left(s\right)+b_{v1}{\widehat{v}}_1\left(s\right)+b_{v2}{\widehat{v}}_2\left(s\right)\right)ds,+\widehat{\sigma }dW\left(s\right),\hfill \\ d\widehat{p}=\widehat{\alpha }ds+\widehat{\beta }dW=\left(-A^T\widehat{p}+\zeta \left(\widehat{x}-E\left(\widehat{x}\right)\right)\right)ds+\widehat{\beta }dW,\hfill \\ {\widehat{x}}_0=0,\hfill \\ {\widehat{p}}_T=-R_T\left({\widehat{x}}_T-E\left({\widehat{x}}_T\right)\right).\hfill \end{array}\right.$$

(97)

Applying the $It\widehat{o}$ Lemma to $〈\widehat{p},\widehat{x}〉$ and taking the expectation on both sides:

$$\begin{array}{cc}\hfill 0& =\mathbb{E}〈-R_T\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right),\widehat{x}〉+\mathbb{E}〈\widehat{p},d\widehat{x}〉+\mathbb{E}〈d\widehat{p},\widehat{x}〉+\mathbb{E}〈d\widehat{p},d\widehat{x}〉\hfill \\ \hfill & =\mathbb{E}〈-R_T\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right),\widehat{x}〉\hfill \\ \hfill & +\mathbb{E}〈\widehat{p},\left(b+b_{u1}{\widehat{u}}_1\left(s\right)+b_{u2}{\widehat{u}}_2\left(s\right)+b_{v1}{\widehat{v}}_1\left(s\right)+b_{v2}{\widehat{v}}_2\left(s\right)\right)ds+\widehat{\sigma }dW〉\hfill \\ \hfill & +\mathbb{E}〈\left(-A^T\widehat{p}+\zeta \left(\widehat{x}-\mathbb{E}\left(\widehat{x}\right)\right)\right)ds+\widehat{\beta }dW,\widehat{x}〉+\mathbb{E}\left({\beta }^T\sigma ds\right)\hfill \\ \hfill & =\mathbb{E}〈R_T\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right),{\widehat{x}}_T〉\hfill \\ \hfill & +\mathbb{E}\left({\int }_t^T\right({\widehat{p}}^T\left(b+b_{u1}{\widehat{u}}_1\left(s\right)+b_{u2}{\widehat{u}}_2\left(s\right)+b_{v1}{\widehat{v}}_1\left(s\right)+b_{v2}{\widehat{v}}_2\left(s\right)\right)\hfill \\ \hfill & +{\widehat{x}}^T\left(-A^T\widehat{p}+\zeta \left(\widehat{x}-\mathbb{E}\left(\widehat{x}\right)\right)\right)+{\beta }^T\sigma \left)ds\right)\hfill \\ \hfill & \ge \mathbb{E}〈R_T^{{\displaystyle \frac{1}{2}}}\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right),R_T^{{\displaystyle \frac{1}{2}}}\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right)〉+\mathbb{E}{\int }_t^T〈{\zeta }^{{\displaystyle \frac{1}{2}}}\left(\widehat{x}-\mathbb{E}\left(\widehat{x}\right)\right),{\zeta }^{{\displaystyle \frac{1}{2}}}\left(\widehat{x}-\mathbb{E}\left(\widehat{x}\right)\right)〉ds.\hfill \end{array}$$

(98)

Thus, we obtain $\mathbb{E}\left({\widehat{x}}_T-\mathbb{E}\left({\widehat{x}}_T\right)\right)=0,$ and $\mathbb{E}\left(\zeta \left(\widehat{x}-\mathbb{E}\left(\widehat{x}\right)\right)\right)=0,$ which implies that $\widehat{p}\equiv 0$. Therefore, the value function is the unique viscosity solution of the HJB equation, indicating that $S_1,S_2,S_3,S_4$ are unique. Consequently, we have $\mathbb{E}\left({\widehat{u}}_1\left(s\right)\right)=\mathbb{E}\left({\widehat{u}}_2\left(s\right)\right)=\mathbb{E}\left({\widehat{v}}_1\left(s\right)\right)=\mathbb{E}\left({\widehat{v}}_2\left(s\right)\right).$ This leads to $\mathbb{E}\left(\widehat{x}\right)=0.$ This completes the proof. □

Based on the global stability and uniqueness of the state, we can derive a comprehensive mapping of state information relative to the initial conditions. Next, we will explore the Lebesgue measure problem associated with barrier surfaces in the context of Nash equilibrium.

Theorem 5. 

The Lebesgue measure of a set composed of state points where the distance change rate between pursuers and evaders (Nash equilibrium) is zero is shown in Figure 5:

$$m^{*}\left\{x\left(s\right)|\mathbb{E}\left({\stackrel{•}{R}}_{Pi}\left(s\right)\right)=0,\forall i=1,2,3,...\right\}=0$$

(99)

Proof. 

The barrier surface can be defined as the set of points where the relative dynamics between the evader and pursuer reach a critical state such that the expected relative velocity projection between the two becomes zero. Mathematically, this surface is expressed as follows:

$$I_i=\left\{x\left(s\right)|\mathbb{E}\left({\stackrel{\cdot }{R}}_{Pi}\left(s\right)\right)=0,\forall i=1,2,3,\cdots \right\},$$

where

$${\stackrel{\cdot }{R}}_{Pi}=v_{E}cos{\theta }_{E}cos{\phi }_E-v_{Pi}cos{\theta }_{Pi}cos{\phi }_{Pi},$$

with $I_i=({\theta }_E,{\phi }_E,{\theta }_{Pi},{\phi }_{Pi})$ representing the points on the barrier surface such that

$$v_{E}cos{\theta }_{E}cos{\phi }_E=v_{Pi}cos{\theta }_{Pi}cos{\phi }_{Pi}.$$

The barrier surface describes the critical configurations where the pursuer’s control actions cannot effectively reduce the distance to the evader. This implies that the states on the barrier surface act as a dynamic boundary between regions of successful pursuit and regions where the evader can potentially avoid capture. The barrier surface can be further understood as a countable set of points:

$$I_n=\left\{I_1,I_2,\cdots ,I_N,\cdots \right\},$$

with a small neighborhood around each point:

$$Q_{\xi }=\left\{Q_1,Q_2,Q_3,\cdots ,Q_N,\cdots \right\},Q_i=B\left(I_i,{\displaystyle \frac{\epsilon }{N}}\right)\supset \left\{I_i\right\},$$

where $I_i\in I_n$. According to the nested open interval theorem, there exists $\epsilon >0$ such that the neighborhoods $Q_{\xi }$ form a cover for the points on the barrier surface. The measure of these points on the barrier surface can be expressed as follows:

$$\begin{array}{cc}\hfill & m^{*}\left(I_n\right)=\left(\sum _{i=1}^{\infty }{m}^{*}\left(I_i\right)\right)=inf\left(m^{*}\left(Q_{\xi }\right)\right)=inf\left(m^{*}\left(\bigcup _{i=1}^{\infty }{Q}_i\right)\right)\hfill \end{array}$$

(100)

Thus:

$$\begin{array}{cc}\hfill & inf\left(m^{*}\left(\bigcup _{i=1}^{\infty }{Q}_i\right)\right)\le inf\left(\sum _{i=1}^{N}m\ast \left(Q_i\right)\right)\le {\displaystyle \frac{4}{3}}{\displaystyle \frac{\pi {\epsilon }^3}{N^3}}\ast {\displaystyle \frac{N(1+N)}{2}}\approx {\displaystyle \frac{2}{3}}{\displaystyle \frac{\pi {\epsilon }^3}{N}}\hfill \end{array}$$

(101)

where $m^{*}\left(Q_i\right)={\displaystyle \frac{4}{3}}\pi {\left({\displaystyle \frac{\epsilon }{N}}\right)}^3$ represents the volume of the ball. This implies the following:

$$m^{*}\left(I_n\right)\le {\displaystyle \frac{2}{3}}{\displaystyle \frac{\pi {\epsilon }^3}{N}}.$$

Taking the limit as $\epsilon \to 0$, we obtain the following:

$${\displaystyle \underset{\epsilon \to 0}{lim}}{m}^{*}\left(I_n\right)\le {\displaystyle \underset{\epsilon \to 0}{lim}}{\displaystyle \frac{2}{3}}{\displaystyle \frac{\pi {\epsilon }^3}{N}}\le 0$$

(102)

where N is a real number with an upper bound. Thus, $m^{*}\left(I_n\right)=0$, completing the proof. □

Note: In the context of stochastic differential games, the Lebesgue measure of the barrier surface is zero, which divides the state space of the pursuit region into two distinct parts: the pursuit region and the terminal region. The state within the pursuit region has two ranges, and there is no equilibrium state between the pursuer and the evader. When discussing the equilibrium problem, it is important to clarify that equilibrium is not the desired outcome. We are primarily concerned with two results: capture and escape. Once an equilibrium state is reached, it indicates that the current strategies are ineffective—neither allowing the pursuer to capture the evader nor enabling the evader to escape from the pursuit region. Even if a termination time is established, the final state cannot be regarded as the definitive outcome of the game. Given any initial conditions, the state trajectory over a certain time frame is unique. Notably, when the state transitions through the barrier surface from the pursuit region to the terminal region, the time required for this transition is effectively zero.

In practical pursuit scenarios, it is often assumed that both sides maintain a balanced state when evenly matched. However, in reality, this balance is not static but fluctuates around the equilibrium. Using a missile interception model as our simulation framework, we illustrate that the condition $\dot{R}=0$, representing a constant relative distance, does not persist in real-world scenarios. Instead, interception success corresponds to $\dot{R}<0$, while $\dot{R}>0$ indicates that the evader escapes, resulting in the two participants diverging. This highlights an important insight: the condition $\dot{R}=0$ is a transient state, consisting of a set of discrete points forming a surface with a Lebesgue measure of zero. In practical terms, this means that the system’s state crosses the barrier surface in zero time. For missile interception problems, this translates to a binary outcome—either interception is successful, or it fails entirely.

5. The Multiple Pursuers and Evaders in a Stochastic Differential Game

When multiple pursuers are chasing evaders, the pursuers should be grouped, then each group should be incorporated into the state equation. As shown in Figure 6, the pursuit process is divided into the following five steps:

• Step 1: Dividing the n pursuers into groups, with each group corresponding to a respective number of evaders denoted by y.
• Step 2: Substituting the information of the pursuers into the system Equation (8), where $v_1\left(s\right)\in {\mathbb{R}}^{y\times 1}$ and $v_2\left(s\right)\in {\mathbb{R}}^{y\times 1}$. The status includes the following: $\mathrm{Status}:\{R_{P1},R_{P2},\dots ,{\theta }_{L1},{\phi }_{L1},{\theta }_{L2},{\phi }_{L2},\dots ,{\theta }_{P1},{\phi }_{P1},{\theta }_{P2},{\phi }_{P2},\dots ,{\theta }_{E1},{\phi }_{E1},{\theta }_{E2},{\phi }_{E2},\dots \}.$
• Step 3: Calculating the parameters $S_1,S_2,S_3,S_4$ of the value function based on the $HJB$ equation.
• Step 4: Solving for the optimal closed-loop feedback strategies $u_1^{*},u_2^{*},v_1^{*},v_2^{*}$ based on FBSDEs.
• Step 5: Implementing the optimal strategies into the motion equations for the pursuers to capture the evaders.

6. Numerical Analysis

Example 1. 

The initial conditions for this simulation are as follows:

• Initial positions: The initial position of pursuer $P_1$ is [0, 15,000, 5000] m, while that of $P_2$ is [0, 13,000, −5000] m. The initial position of the evader E is [100,000, 14,000, 0] m.
• Initial velocities: The pursuers’ velocities are set to 3000 m/s and 3500 m/s, respectively, while the evader’s velocity is 2000 m/s.
• Acceleration limits: The maximum normal accelerations are 30 g for the pursuers and 20 g for the evader (g = 9.81 m/s²).
• Flight path angles: The initial flight path angles for the pursuers are set to −4° (elevation) and 4° (azimuth). The evader’s initial angles are 10° (elevation) and 170° (azimuth).
• Time step: A time interval of 0.1s was used for numerical integration.

During the simulation, both the pursuers and the evader employ the optimal strategy. The optimal trajectory is depicted in Figure 7. Two-dimensional projections in the X–Y and X–Z planes are shown in Figure 8 and Figure 9, respectively. These figures demonstrate the feasibility of the optimal state feedback strategy algorithm proposed in this paper. The distance between the evader and the pursuers is illustrated in Figure 10, where the final miss distance is $0.83\mathrm{m}$. In missile interception problems, it is commonly assumed that interception occurs when the distance is less than one meter. Furthermore, in this paper, the interception termination set is defined by $\dot{R}>0$ or $R<1$. The evader is captured by the pursuers after $31.7\mathrm{s}$. Generally, the terminal guidance phase of missile interception lasts between 20 s and 40 s, and the result from Experiment 1 is within this range. Acceleration variations are presented in Figure 11 and Figure 12. The state norm is shown in Figure 13. During the interception process, the state continuously converges, and we have proven this in Theorem 2. The experimental results confirm the validity of Theorem 2, with the terminal state approaching zero. The value of the cost function is shown in Figure 14. It is evident that the cost function converges to a constant value, indicating the convergence of state and input over a finite time. The value function is shown in Figure 15. The value function satisfies the termination condition of the forward–backward stochastic differential equation, with the terminal value being zero. The simulation environment is summarized in

Table 2. Experiment environment.

Item	Environment
Development language	Python
Library	Numpy
Disk capacity	2 T
RAM	32 G
CPU	i7 2.2 GHZ
OS	Ubantu 16.04

. During the simulation process, the Runge–Kutta method is utilized for each iteration.

The mean value of the state is 503,775.07. The state graph demonstrates that the state norm gradually converges as time increases, indicating that as the time approaches the termination point, the state norm behaves like a linear function. Concurrently, the cost function stabilizes at a constant value, showing a decreasing growth rate over time. Furthermore, the value function satisfies the final convergence criterion, with $V\left(T\right)=0$.

Example 2. 

In this case, compared to Example 1, the evader employs a sinusoidal maneuver while the pursuer utilizes an optimal feedback strategy, with all other initial conditions remaining unchanged. After 32 s, the pursuer successfully captures the evader, achieving a mission distance of 0.38 m. The mean state value is 504,990.71. The three-dimensional pursuit diagram is displayed in Figure 16, while two-dimensional plane projections in the X–Y and X–Z planes are shown in Figure 17 and Figure 18, respectively. Since the evader’s maneuver is different from that in Example 1, the optimal trajectory is also different. However, in both cases, the pursuer successfully intercepts the evader. The primary objective of Example 2 is to demonstrate that, even when the evader adopts different evasion maneuvers, the pursuer is still able to intercept the evader. The distance between the evader and the pursuer is illustrated in Figure 19, confirming that the evader is captured. Acceleration variations are depicted in Figure 20 and Figure 21. The state norm is shown in Figure 22, the cost function in Figure 23, and the value function in Figure 24. The final experimental results confirm the feasibility of the proposed algorithm.

Example 3. 

In this scenario, the evader employs a constant maneuver while the pursuer implements an optimal feedback strategy, with all other initial conditions remaining unchanged from Example 1. The pursuer successfully captures the evader, resulting in a final mission distance of 0.62 m. The mean state converges to 460,599.78, and the value function reaches zero at the moment of capture. The three-dimensional pursuit diagram is illustrated in Figure 25. Two-dimensional plane projections in the X–Y and X–Z planes are shown in Figure 26 and Figure 27, respectively. In Experiment 3, since the evader employs a constant-value maneuver, the trajectory of the target in three-dimensional space forms an arc. The distance between the evader and the pursuer is depicted in Figure 28, confirming that the evader has been captured. Additionally, the acceleration variations are presented in Figure 29 and Figure 30. In this experiment, the normal acceleration of the pursuer reached its maximum value, but the pursuer successfully captured the evader. The state norm is shown in Figure 31, the cost function in Figure 32, and the value function in Figure 33. The final experimental results confirm the feasibility of the algorithm.

Based on Examples 1–3, we observe the following: In Example 1, both the pursuers and the evader utilize the optimal feedback strategy, allowing the pursuer to successfully capture the evader. The changes in state throughout the pursuit process align with the proof presented in the theorem, and the cost function converges continuously. Ultimately, the value function approaches the designated terminal value. In Example 2, the evader does not employ the optimal feedback strategy; nevertheless, the pursuer successfully captures the evader in both cases. Notably, the cost function at termination is lower than that observed in Example 1. Furthermore, the data for the value function at the terminal time confirms that $V\left(T\right)=0$.

Example 4. 

We simulated a scenario involving three pursuers and two evaders. Pursuers 1 and 2 successfully captured Evader 1, while Pursuer 3 captured Evader 2. The final mission distances were recorded as follows: $R_2=0.91m$ and $R_3=0.75m$. The state ultimately converged to 878,031.05. At the moment of capture, the value function converged to zero. The three-dimensional pursuit diagram is illustrated in Figure 34, while the two-dimensional plane projections in the X–Y and X–Z planes are shown in Figure 35 and Figure 36, respectively. In this experiment, both the pursuers and the evaders employed optimal game strategies. As a result, the pursuers successfully intercepted the evaders. However, due to the change in initial positions, the optimal trajectories differ from those in the previous experiments, demonstrating the adaptability of the strategy to varying initial conditions. The three-dimensional pursuit diagram in Figure 34 clearly shows the distinct paths taken by each pursuer and evader during the pursuit. Figure 37 depicts the distance between the evaders and the pursuers at the moment of capture. The distance between each pursuer and their respective evader decreased progressively over time, confirming the effectiveness of the optimal feedback strategy in achieving interception. The mission distances at the moment of capture were recorded as $R_2=0.91m$ and $R_3=0.75m$, indicating the close proximity of the pursuers to the evaders when interception occurred. The acceleration variations are presented in Figure 38 and Figure 39. These plots reveal how the pursuers’ accelerations changed throughout the pursuit, with notable spikes occurring during the final stages of interception. These spikes indicate the maximum efforts exerted by the pursuers as they close the gap with the evaders. The state norm is illustrated in Figure 40, showing the progression of the system state during the pursuit. As expected, the state norm decreases steadily, reflecting the convergence of the system towards the interception point. This convergence is a strong indicator of the effectiveness of the optimal strategy. The cost function is shown in Figure 41, and the value function is depicted in Figure 42. The cost function reaches a constant value as the simulation progresses, demonstrating the stability of the optimal strategy. Notably, the value function converges to zero at the moment of capture, indicating that the pursuers successfully achieved their objective of intercepting the evaders. These results confirm the robustness and effectiveness of the optimal game strategy in various pursuit scenarios, as well as the adaptability of the strategy to different initial conditions.

Based on the many-to-many strategy presented in this article, we conducted a simulation for Example 4. The experimental results demonstrated that the pursuer successfully captured the evader, with the state converging to zero and the mean value of the state being finite. At the termination time, the value $V\left(T\right)$ of the value function equals zero. Both Example 1 and Example 4 illustrate that the proposed many-to-one and many-to-many pursuit algorithms effectively enable the interception of multiple targets. We conducted 100 separate experiments with different initial conditions to test the robustness of the model. The initial positions of the pursuer and evader were randomly selected, with the constraint that the initial distance between them was always greater than 100,000 m. The initial angle was chosen within a predefined permissible range to ensure consistency in all trials. The experimental parameters and results are presented in

Table 3. Experimental setup and parameters.

Parameter	Value
Initial distance	100,000
Evader initial elevation angle	$[-{20}^{\circ },{20}^{\circ }]$ (elevation) and $[{160}^{\circ },{200}^{\circ }]$ (azimuth)
Evader initial azimuth angle	$[-{20}^{\circ },{20}^{\circ }]$ (elevation) and $[-{20}^{\circ },{20}^{\circ }]$ (azimuth)
Iteration time	0.1–0.5 s
Maximum normal acceleration	20–40 g
100 experiments	98 successful, 2 failed

. The simulation results indicate a high success rate for interception. Out of the 100 experiments, 98 resulted in successful interception, while only 2 cases saw the evader escaping. This demonstrates that the proposed model and algorithm are highly robust and effective, even under varied initial conditions. Numerical stability: To further assess the stability of the solution, we varied key system parameters such as control input constraints and system dynamics and observed the performance. The results showed that, despite variations in these parameters, the system consistently performed well, with the pursuer successfully intercepting the evader in the majority of the trials. This suggests that the solution to the FBSDEs remains stable and robust, even under small perturbations in the system parameters.

7. Conclusions

In this paper, we investigated optimal backward strategies in pursuit problems through stochastic differential games. We assumed that players’ strategies are bounded, which implies that there are constraints on the actions they can take, ensuring that strategies remain realistic within the context of the game. Our approach emphasizes the uniqueness of the value function as a viscosity solution, providing a robust theoretical foundation for our findings. We demonstrated that the parameters of the optimal backward strategies are intrinsically linked to the solutions of forward–backward stochastic differential equations (FBSDEs) and the terminal conditions of the cost function. This relationship represents a significant extension of the work done in references [10,20], where we offer a matrix formulation of the polynomial growth value function. This formulation not only simplifies the computational aspect of deriving optimal strategies but also enhances the applicability of our findings to real-world scenarios.

Furthermore, we provided the expressions for the optimal strategies through rigorous calculations and established the convergence of the state trajectory within the pursuit region. Our results, illustrated in Figure 6, indicate that when the state transitions from the pursuit region to the termination set, it encounters a barrier surface with a Lebesgue measure of zero. This phenomenon illustrates that the state cannot achieve Nash equilibrium during the pursuit process, reflecting the dynamics observed in actual pursuit scenarios. Finally, we validated our proposed optimal feedback strategy through numerical simulations, which demonstrated effective state convergence and successful evader capture. These findings not only reinforce the theoretical contributions of our work but also open avenues for further research in multi-target pursuit strategies.