Optimal Pursuit Strategies in Missile Interception: Mean Field Game Approach

Abstract

This paper investigates Mean Field Game methods to solve missile interception strategies in three-dimensional space, with a focus on analyzing the pursuit–evasion problem in many-to-many scenarios. By extending traditional missile interception models, an efficient solution is proposed to avoid dimensional explosion and communication burdens, particularly for large-scale, multi-missile systems. The paper presents a system of stochastic differential equations with control constraints, describing the motion dynamics between the missile (pursuer) and the target (evader), and defines the associated cost function, considering proximity group distributions with other missiles and targets. Next, Hamilton–Jacobi–Bellman equations for the pursuers and evaders are derived, and the uniqueness of the distributional solution is proved. Furthermore, using the $ϵ$-Nash equilibrium framework, it is demonstrated that, under the MFG model, participants can deviate from the optimal strategy within a certain tolerance, while still minimizing the cost. Finally, the paper summarizes the derivation process of the optimal strategy and proves that, under reasonable assumptions, the system can achieve a uniquely stable equilibrium, ensuring the stability of the strategies and distributions of both the pursuers and evaders. The research provides a scalable solution to high-risk, multi-agent control problems, with significant practical applications, particularly in fields such as missile defense systems.

1. Introduction

Motivation: With the increasingly complex global security situation, missile interception technology has become an indispensable part of modern air defense and missile defense operations. Traditional missile interception problems often focus on the interactions between a few missiles and targets. However, as the number of missiles increases and the battlefield environment becomes more complex, the limitations of traditional methods have become more apparent. To address this challenge, the Mean Field Game method provides a new solution approach. The introduction of MFG theory allows for the effective resolution of control and optimization problems in large-scale, multi-target environments, avoiding the computational explosion caused by high-dimensional problems found in traditional methods [1,2]. Previous studies on the missile multi-to-multi interception problem mostly expanded from one-to-one or many-to-one situations to many-to-many. However, this process often leads to dimensional explosion, and as the number of individuals increases, the communication burden intensifies, often causing communication blockage. Therefore, this paper employs the MFG method to solve the missile multi-to-multi interception problem. As $N\to \infty$, the coupling effects between individuals are minimized. By deriving the optimal solution of the mean field for both parties in the game, this paper proves the consistency between the Nash equilibrium of finite individuals and the mean field Nash equilibrium.

The application of MFG has covered multiple fields, especially in military decision-making, resource allocation, and multi-agent systems, showing broad potential. In missile interception problems, the MFG method models the behavior of each missile, assuming that the behavior of participants is influenced solely by the statistical distribution of the group. This avoids complex mutual computations between individuals [3]. This makes MFG an effective tool for handling large-scale missile interception systems, providing near-optimal control strategies, without relying on the state of each missile [4,5]. In recent years, researchers have conducted extensive theoretical studies on MFG methods, covering topics from the derivation of the Hamilton–Jacobi–Bellman (HJB) equations to Nash equilibrium analysis in mean field games [6]. The HJB equations provide a mathematical framework for solving optimal control strategies, considering the control constraints, state variables, and optimality conditions of dynamic systems, thus leading to the optimal solution [7]. However, the difficulty of solving the HJB equations increases significantly as the number of participants grows [8]. Therefore, the introduction of MFG effectively simplifies this problem, allowing for reasonable solutions in large-scale systems [9,10]. In the specific application of missile interception problems, the MFG framework not only focuses on the interaction between missiles and targets but also considers the cooperation and competition among multiple missiles [11]. This is especially important because modern missile defense systems typically contain multiple interceptors. When facing multiple enemy targets, optimizing the strategy of each missile to achieve optimal performance for the overall system is a critical issue. By using MFG-based modeling, researchers are able to derive the optimal interception strategy for each missile, while ensuring computational efficiency [12]. With the continuous development of MFG theory, more and more research has begun to focus on how to effectively solve HJB equations in high-dimensional spaces and handle complex boundary conditions. For example, Bensoussan et al. [4] proposed a numerical solution method based on MFG theory that can handle complex problems involving large numbers of participants, while Fathi et al. [5] explored the optimization of control strategies in multi-agent collaboration and competition. These studies provide theoretical support for the design of multi-missile interception control strategies. Furthermore, the advantages of the MFG method lie in its ability to simplify computational complexity using the statistical distribution of group behavior, rather than directly calculating the interactions between individuals [13]. This characteristic makes the MFG method highly applicable in real-time defense systems, especially in complex battlefield environments with multiple targets and interceptors. Researchers have successfully solved many challenges that traditional methods find difficult to address by continuously improving the MFG framework [9].

In this study, we assume that the number of targets is sufficiently large to apply the Mean Field Game (MFG) approach. However, in practical applications, the number of targets is typically finite, which may affect the applicability of this method. While the MFG approach effectively handles large-scale systems and provides equilibrium solutions, its validity might be limited when the number of targets is small. We will discuss how a limited number of targets can influence the applicability and effectiveness of the MFG method.

The goal of this paper is to explore the optimal control strategy in missile interception problems based on the MFG theory, and to analyze the existence and uniqueness of the $ϵ$-Nash equilibrium [14] in mean field games. Through an in-depth study of this problem, this paper not only promotes the application of multi-agent systems in missile defense but also provides theoretical support for optimal control problems in large-scale complex systems [15]. The main innovations of this paper are as follows:

• Application of MFG to missile interception problems in three-dimensional space: This paper is the first to apply the Mean Field Game method to multi-missile interception target problems in three-dimensional space, particularly focusing on nonlinear interception systems. This innovation supplants traditional one-to-one and many-to-one interception models, extending them to a many-to-many interception problem. By fully utilizing the distributed nature of the MFG method, missile strategies are optimized in complex environments, avoiding the issue of dimensional explosion, while not requiring individuals to perceive the behavior of all other participants. Compared to existing studies that mainly considered two-dimensional cases, this paper starts from three-dimensional space, where both state and control information are constrained, making it more aligned with practical application scenarios.
• Consideration of complex interactions between missiles and targets: When multiple missiles simultaneously attack adjacent and closely located targets, interference between missiles may occur. To address this challenge, this paper proposes a dynamic adjustment mechanism based on the distribution of adjacent groups in three-dimensional space. Each missile adjusts its strategy according to the distribution of nearby groups, both its own and the target’s, thus minimizing interference. Furthermore, when approaching a target group, the missile updates its strategy based on the target group’s distribution to increase the interception probability. To reduce the risk of being intercepted, the target adopts a decentralized evasion strategy by perceiving changes in the missile distribution and the nearby target distribution, thus determining the optimal escape path. The optimal strategies for missiles and targets are inherently adversarial.
• $ϵ$-Nash equilibrium study in a two-group MFG model: This paper investigates a two-group MFG problem, where individuals within the same group cooperate with each other, and cooperation is realized through distribution, while individuals from different groups compete. Each player does not need to perceive the behavior of all individuals, but only needs to understand the distribution of both groups and the distribution of the adjacent group. As the number of players increases, the $ϵ$-Nash equilibrium can be approximated as a global Nash equilibrium [16], greatly simplifying the problem-solving process and improving the computational efficiency of the model. This approach is inspired by the multi-population MFG model described in Section 7.1.1 of Probabilistic Theory of Mean Field Games with Application [17].

This paper proposes a method for calculating the distribution of nearby groups in three-dimensional space. By considering the distribution of nearby groups around the missile, it reduces the interference from surrounding missiles, as well as from groups nearby the target. Compared to traditional methods, which require calculating interactions between all individuals, this paper only considers the effects brought by nearby groups, making the computation more efficient and practically applicable. These innovations highlight the unique contribution of this paper in the existing literature.

In comparison with the approach proposed by Toumi et al. (2024) [18] in their study on large-scale multi-agent systems, which also considered interactions between agents in a distributed manner, our model extends the concept into a three-dimensional space, where both missile and target distributions become more complex, due to input constraints rather than spatial constraints. While Toumi et al. focused on congestion avoidance in crowd scenarios with agent rewards based on their distance from other agents, our model specifically incorporates missile–target dynamics in a three-dimensional combat scenario, optimizing strategies to minimize interference and improve interception success, despite the input limitations in such a constrained control environment.

Compared to the study by [17], in the field of missile interception, when the number of missiles is sufficiently large, the difference between the individual state and the group mean under the $ϵ$-equilibrium will gradually decrease, indicating that the individual state will progressively follow the changes in the group state.

The rest of the paper is organized as follows: In Section 2, we first present the missile interception model, based on which we derive the spatial equations and the cost functions for both the pursuers and the evaders. We also describe the basic form of the $ϵ$-Nash equilibrium. In Section 3, we derive the distribution functions for both the pursuers and the evaders. By manipulating the terminal function, we introduce a new cost function. In Section 4, we apply the principles of dynamic programming to obtain the Hamilton–Jacobi–Bellman (HJB) equations for both the pursuers and the evaders, from which we derive the optimal strategies. In Section 5, we present the unique solution form of the forward–backward stochastic differential equations. We then prove the boundedness of the states, and finally show the difference between individuals and the mean field in the $ϵ$-Nash equilibrium. In Section 6, we provide the experimental results. Finally, in Section 7, we present the concluding remarks.

2. Problem Setting

Here, we first present the mathematical model of the missile interception problem, then derive the state-space equations for the pursuit problem based on this model. Following that, we outline the related basic assumptions and, finally, we define the dual-population $ϵ$-Nash equilibrium.

2.1. Notation

From the field of missile interception, we will discuss the problem where the missile is represented as the pursuer and the target is represented as the evader. The pursuers, $P_i$, are collaboratively capturing the evaders, $E_i$, who are attempting to evade the capture. The game takes place in a three-dimensional space. It is assumed that both the pursuers and the evaders are treated as point masses with normal acceleration constraints. The direction of velocity is adjusted by the normal acceleration, with the direction of acceleration being perpendicular to the velocity direction. The pursuer model between a pursuer and an evader is shown in Figure 1. Based on the relationships between the pursuer $P_i$ and the evader $E_i$ in three-dimensional space, the non-linear differential equations are derived, as presented in the work by Song et al. [19]. Using this model helps analyze and solve interference issues in the terminal guidance phase, and it reflects the actual situation more accurately. While other models may lead to different impacts on missile interference and target selection strategies, this model demonstrates strong applicability and advantages for the current research.

$${\dot{R}}_{Pi}=v_{Ei}cos{\theta }_{Ei}cos{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}cos{\phi }_{Pi}$$

(1)

$$R_{Pi}{\dot{\theta }}_{Li}=v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Pi}$$

(2)

$$R_{Pi}{\dot{\phi }}_{Li}cos{\theta }_{Li}=v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}$$

(3)

$$\begin{array}{cc}\hfill {\dot{\theta }}_{Pi}=& {\displaystyle \frac{A_{yPi}}{v_{Pi}}}+tan{\theta }_{Li}sin{\phi }_{Pi}\times {\displaystyle \frac{(v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei})}{R_{Pi}}}\hfill \\ \hfill & +cos{\phi }_{Pi}{\displaystyle \frac{(v_{Pi}sin{\theta }_{Pi}-v_{Ei}sin{\theta }_{Ei})}{R_{Pi}}}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\dot{\phi }}_{Pi}=& {\displaystyle \frac{A_{zPi}}{v_{Pi}cos{\theta }_{Pi}}}+sin{\theta }_{Pi}cos{\phi }_{Pi}tan{\theta }_{Li}+{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}cos{\theta }_{Pi}}}\hfill \\ \hfill & -sin{\theta }_{Pi}sin{\phi }_{Pi}{\displaystyle \frac{v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Ei}}{R_{Pi}cos{\theta }_{Pi}}}-{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}}}\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\dot{\theta }}_{Ei}=& {\displaystyle \frac{A_{yEi}}{v_{Ei}}}+tan{\theta }_{Li}sin{\phi }_{Ei}\ast {\displaystyle \frac{\left(v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}-v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}\right)}{R_{Pi}}}\hfill \\ \hfill & +cos{\phi }_{Ei}{\displaystyle \frac{\left(v_{Pi}sin{\theta }_{Pi}-v_{Ei}sin{\theta }_{Ei}\right)}{R_{Pi}}}\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\dot{\phi }}_{Ei}=& {\displaystyle \frac{A_{zEi}}{v_{Ei}cos{\theta }_{Ei}}}+sin{\theta }_{Ei}cos{\phi }_{Ei}tan{\theta }_{Li}+{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}cos{\theta }_{Ei}}}\hfill \\ \hfill & -sin{\theta }_{Ei}sin{\phi }_{Ei}{\displaystyle \frac{v_{Ei}sin{\theta }_{Ei}-v_{Pi}sin{\theta }_{Ei}}{R_{Pi}cos{\theta }_{Ei}}}-{\displaystyle \frac{v_{Ei}cos{\theta }_{Ei}sin{\phi }_{Ei}-v_{Pi}cos{\theta }_{Pi}sin{\phi }_{Pi}}{R_{Pi}}}\hfill \end{array}$$

(7)

The pursuit–evasion problem in three-dimensional space is modeled with multiple pursuers attempting to capture evaders. The notation used to describe the system dynamics is as given in

Table 1. Table of symbols and coordinate systems.

Symbol	Description	Coordinate System
$(X_I,Y_I,Z_I)$	Inertial reference coordinate system	Inertial
$(X_L,Y_L,Z_L)$	Line-of-sight (LOS) coordinate system	LOS
$(X_E,Y_E,Z_E)$	Velocity coordinate system of the i-th pursuer	Pursuer
$v_{Ei}$	Velocity of the i-th evader	Evader
$v_{Pi}$	Velocity of the i-th pursuer	Pursuer
$A_{Pi}$	Acceleration of the i-th pursuer	Pursuer
$A_{Ei}$	Acceleration of the i-th evader	Evader
${\gamma }_{Pi}$	Angle between the acceleration of the i-th pursuer and axis $Y_{Pi}$	Pursuer
${\gamma }_{Ei}$	Angle between the acceleration of the i-th evader and axis $Y_E$	Evader
$R_{Pi}$	Distance between the i-th pursuer and the evader	Spatial
${\theta }_{Li},{\phi }_{Li}$	LOS angles between the evader and the i-th pursuer relative to the inertial reference coordinate system	LOS
${\theta }_{Pi},{\phi }_{Pi}$	Elevation and azimuth angles of $v_{Pi}$ relative to the LOS coordinate system from pointing toward E	Pursuer
${\theta }_{Ei},{\phi }_{Ei}$	Elevation and azimuth angles of $v_{Ei}$ relative to the LOS coordinate system from $E_i$ pointing toward $P_i$	Evader
$A_{zPi},A_{yPi}$	Projections of the pursuer’s normal acceleration on the $Z_{Pi}$ and $Y_{Pi}$ axes in the velocity coordinate system	Pursuer
$A_{zEi},A_{yEi}$	Projections of the evader’s normal acceleration on the $Z_{Ei}$ and $Y_{Ei}$ axes in the velocity coordinate system	Evader

:

In this paper, to clearly distinguish the parameters of pursuers and evaders, we use superscripts and subscripts. Specifically, the parameters of pursuers are denoted by the letter “P”, with corresponding superscripts and subscripts, while the parameters of the evaders are denoted by the letter “E”, with their respective superscripts and subscripts. Additionally, the letter “i” is used to represent each player, which helps clearly differentiate the state and behavior of each individual, especially in scenarios involving multiple pursuers and evaders. This notation ensures the clarity of formulas, figures, and descriptions, making it easier for readers to understand the components of the model and the relationships between players, while avoiding potential confusion.

2.2. Problem Formulation

In this paper, $\left(\overline{\mathrm{\Omega }},\mathcal{F},{\left({\mathcal{F}}_t\right)}_{t>0},\mathcal{P}\right)$ is a complete probability space, $W^i\left(s\right)$ is m-dimensional Brownian motion. For any initial time $t\ge 0$, terminal time $T>t$, and initial state $x_0^i\in {\mathbb{R}}^n$, the filtration ${\left({\mathcal{F}}_t\right)}_{t>0}$ is the natural filtration generated by the Brownian motion $W^i\left(s\right)$ for $t\le s\le T$, augmented by all the $\mathcal{P}$-null sets of $\mathcal{F}$. According to Equations (1)–(7), we consider a stochastic differential equation in a mean field game with control input constraints [20]:

$$\left\{\begin{array}{cc}\hfill & d{x}^i\left(s\right)=\left(b^i\left(s,x_i\left(s\right)\right)+b_{u1}^i\left(x_i\left(s\right)\right)u_1^i\left(s\right)+b_{u2}^i\left(x_i\left(s\right)\right)u_2^i\left(s\right)\right.\hfill \\ \hfill & \left.+b_{v1}^i\left(x_i\left(s\right)\right)v_1^i\left(s\right)+b_{v2}^i\left(x_i\left(s\right)\right)v_2^i\left(s\right)\right)ds\hfill \\ \hfill & +{\sigma }^i\left(s,x_i\left(s\right),u_1\left(s\right),u_2\left(s\right),v_1\left(s\right),v_2\left(s\right)\right)d{W}^i\left(s\right),\hfill \\ \hfill & s\in \left(t,T\right),\hfill \\ \hfill & x^i\left(0\right)=x_0.\hfill \end{array}\right.$$

(8)

where $i\in N$ is the number of players in each population. $x^i\left(s\right)\in {\mathbb{R}}^{m\times 1}$ is the system state, with the initial condition $x_0,$ $x^i={\left[\begin{array}{c}{R}_{Pi},{\theta }_{Li},{\phi }_{Li},{\phi }_{Pi},{\theta }_{Pi},{\theta }_{Ei},{\phi }_{Ei}\end{array}\right]}^T$ Let the accelerations of the pursuers and evader be defined as follows: $u_1^i\left(s\right)\in {\mathbb{R}}^{n\times 1}$ is the acceleration of the i-th pursuer in the velocity coordinate system of the y-axis. $u_2^i\left(s\right)\in {\mathbb{R}}^{n\times 1}$ is the acceleration of the i-th pursuer in the velocity coordinate system of the z-axis. $v_1^i\left(s\right)\in {\mathbb{R}}^{k\times 1}$ is the acceleration of the i-th evader in the velocity coordinate system of the y-axis. $v_2^i\left(s\right)\in {\mathbb{R}}^{k\times 1}$ is the acceleration of the i-th evader in the velocity coordinate system of thez-axis. $\left\{W^i\left(s\right)\in {\mathbb{R}}^{m\times 1},{\left({\mathcal{F}}_t\right)}_{t>0},t\le s\le T\right\}$ is the standard Wiener process for the i-th process.

$$b^i(s,x^i\left(s\right))\in {\mathbb{R}}^{m\times 1},b_{u1}^i(s,x^i\left(s\right))\in {\mathbb{R}}^{m\times 1},b_{u2}^i(s,x^i\left(s\right))\in {\mathbb{R}}^{m\times 1}$$

$$b_{v1}^i(s,x^i\left(s\right))\in {\mathbb{R}}^{m\times 1},b_{v2}^i(s,x^i\left(s\right))\in {\mathbb{R}}^{m\times 1}$$

are real matrices for the i-th process. ${\sigma }^i\left(s,x^i\left(s\right),u_1^i\left(s\right),u_2^i\left(s\right),v_1^i\left(s\right),v_2^i\left(s\right)\right)\in {\mathbb{R}}^{m\times m}$ is the diffusion term of the system equation for the i-th process.

Assumption 1. 

It can be assumed that $B^i=b^i+{b^i}_{u1}{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v2}^i{v}_2^i$, and σ is continuous and bounded. There exist positive constants $C_b,C_u,C_{\sigma },C_{b\sigma }$such that

$$\begin{array}{cc}\hfill & ∥B^i\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)-B^i\left(s,y^i,u_1^i,u_2^i,v_1^i,v_2^i\right)∥\le C_b∥x^i-y^i∥\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & ∥B^i\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)-B^i\left(s,x^i,{\overline{u}}_1^i,{\overline{u}}_2^i,{\overline{v}}_1^i,{\overline{v}}_2^i\right)∥\hfill \\ \hfill & \begin{array}{cc}& \end{array}\le C_u\left({\left|u_1^i-{\overline{u}}_1^i\right|}^p+{\left|u_2^i-{\overline{u}}_2^i\right|}^p+{\left|v_1^i-{\overline{v}}_1^i\right|}^p+{\left|v_2^i-{\overline{v}}_2^i\right|}^p\right)\hfill \end{array}$$

(10)

$$∥{\sigma }^i\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)-{\sigma }^i\left(s,y^i,u_1^i,u_2^i,v_1^i,v_2^i\right)∥\le C_{\sigma }∥x^i-y^i∥$$

(11)

$$\begin{array}{cc}\hfill & ∥B^i\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)+∥{\sigma }^i\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)∥∥\le C_{b\sigma }\left(1+∥x^i∥\right)\hfill \end{array}$$

(12)

where $s\in [t,T].x^i,y^i\in {\mathbb{R}}^n$. The admissible control sets for pursuers and evaders can be defined as follows:

$${\pi }_u=\left\{\begin{array}{c}{u}_j^i\mid u_j^i\in \left[t,T\right]\times {\mathbb{R}}^{n\times 1},u_j^i\mathrm{is}\mathrm{uniformly}\mathrm{locally}\mathrm{Lipschitz}\mathrm{continuous},\hfill \\ \left|u_j^i\right|\le u_j^{i,max},j=1,2,i=1,\dots N.\hfill \end{array}\right\}$$

(13)

$${\pi }_v=\left\{\begin{array}{c}{v}_j^i\mid v_j^i\in \left[t,T\right]\times {\mathbb{R}}^n,v_j^i\mathrm{is}\mathrm{uniformly}\mathrm{locally}\mathrm{Lipschitz}\mathrm{continuous},\hfill \\ \left|v_j^i\right|\le v_j^{i,max},j=1,2,i=1,\dots ,N.\hfill \end{array}\right\}$$

(14)

for all $s\in \left[t,T\right]$, $L\left(s,x^i,u_1^i,u_2^i,v_1^i,v_2^i\right)$, and $g\left(x^i\right)$ is continuously differentiable. In the missile interception model presented in this paper, we assume that the control inputs are constrained. Specifically, the trajectory control engine, which governs the missile’s movement, has an upper bound on its output, which is directly related to the normal acceleration. This assumption is essential for accurately modeling real-world missile systems, where limitations on engine power and physical constraints on acceleration are commonly encountered. The assumption of bounded control inputs is well-supported by prior research in the field. For example, references [21] offered theoretical foundations for such constraints in control systems, while references [22,23] explored their relevance to missile interception problems. Furthermore, reference [24,25] discussed the effect of input constraints on system stability and performance, further reinforcing the validity of this assumption in our model [26].

In missile interception models, the pursuer’s cost function is crucial for optimizing the missile behavior to effectively intercept the target, as shown in Equation (15). The cost function can be expressed as a function of several key distributions within the missile–target environment. These distributions represent various interactions between the missile and surrounding entities, each of which contributes to the overall cost. The missile operates within a dynamic environment that requires careful consideration of four primary distribution factors:

• Self-Proximity Distribution: This distribution aims to minimize the impact of other missiles in the same class on the missile’s behavior, ensuring that the missile’s movement is not adversely affected by its peers in close proximity.
• Total Missile Distribution: The purpose of this distribution is to guide all missiles along a predetermined trajectory that aligns with the target distributions, optimizing the collective missile effort toward target interception.
• Target-Proximity Distribution: This distribution is designed to avoid interference from nearby targets, ensuring that the missile’s approach is not obstructed by the presence of other targets in the vicinity.
• Total Target Distribution: This distribution guides all missiles to converge toward the collective distribution of all targets, optimizing the strategy for interception of multiple targets simultaneously.

$$\begin{array}{cc}\hfill & J_{Pi}\left(s,u_1^i,u_2^i\right)\hfill \\ \hfill & =\underset{X_{Ei}\in C_E}{inf}\mathbb{E}\left[∥X_{Pi}\left(t\right)-X_{Ei}\left(t\right)∥\right]\hfill \\ \hfill & +\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[{\int }_t^{T}exp\left\{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr\right\}\left(x^i{\left(s\right)}^T{Q}_{P1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right.\right.\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·d{x}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P1}{u}_2^i\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E1}{u}_2^i\right)\left)ds\right]\hfill \\ \hfill & +\mathbb{E}\left[exp\left\{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr\right\}\left({\left(x^i\right)}^T\left(T\right)R_{P1}{x}^i\left(T\right)+{\mu }^P\left(T\right)-{\mu }^E\left(T\right)\right)\right]\hfill \end{array}$$

(15)

where ${\mu }^P$ refers to the probability measure of the pursuer group, and ${\mu }^E$ refers to the probability measure of the evader group; ${\overline{X}}_{Pi}$ is the mean of the pursuer’s neighboring group, and ${\overline{X}}_{Ei}$ is the mean of the evader’s neighboring group; $\mathrm{\Omega }$ is used only for computing the distribution of the neighboring group, not for the entire set of individuals; $Q_{P1}\in {\mathbb{R}}^{n\times n},Q_{P3}\in {\mathbb{R}}^{M\times M},Q_{P4}\in {\mathbb{R}}^{M\times M},R_{P1}\in {\mathbb{R}}^{n\times n}$ are all positive definite matrices; ${\psi }_P$ is a function with continuous first derivatives; $X_{Pi}\in {\mathbb{R}}^{5\times 5}$ is the vector of the pursuer’s distribution state, which includes the position and the line-of-sight angle in three-dimensional space; ${\overline{X}}_{Pi}$ is the mean vector of the distribution state; ${\gamma }_{P1},{\gamma }_{P2}$ are constants; and $c_P$ is a state-dependent function. The cost function associated with the pursuer is composed of three distinct parts, each contributing to the optimal decision-making process for the missile:

• Initial Cost: At the initial moment, the missile selects an appropriate target, with the aim of minimizing the cost. Once the target is chosen, the missile will not alter its choice, ensuring that the cost is minimized from the outset.
• Running Cost: The running cost continues to accumulate over time, incorporating the dynamic effects of distribution on the missile’s trajectory and behavior, adjusting for the proximity and interactions with both other missiles and targets.
• Terminal Cost: At the terminal time, the cost is defined by the final states of both the missile and its target, as well as the terminal values of the various distributions. This cost function seeks to minimize the final discrepancy between the missile’s state and the target’s state, considering both the interception success and the distributions at the end of the engagement.

Next, we present the cost function for the evader, as shown in Equation (16). This cost function primarily depends on three distributions:

• Self-Proximity Distribution: This distribution aims to reduce the influence of other evaders of the same type on the evader’s behavior, avoiding interference between groups.
• Total Evader Distribution: The purpose of this distribution is to guide all evaders along a predetermined trajectory to effectively move away from the pursuer’s area.
• Total Pursuer Distribution: The evader adjusts its own distribution to ensure that its behavior in the group avoids getting too close to the distribution of the pursuer.

$$\begin{array}{cc}\hfill & J_{Ei}(s,v_1^i,v_2^i)\hfill \\ \hfill & =\underset{v_i\in {\pi }_v}{sup}\mathbb{E}\left[{\int }_t^{T}exp\left\{-{\int }_t^s{c}_E\left(x\left(r\right)\right)dr\right\}\left(x^i{\left(s\right)}^T{Q}_{E1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(v_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(v_2^i\right)}^2}{2}}\right.\right.\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_E\left({\partial }_t{\psi }_E(s,x)+{\partial }_x{\psi }_E(s,x)·d{x}^i\right)d({\mu }^E\left(s\right)-{\mu }^P\left(s\right))\hfill \\ \hfill & +exp{(X_{Ei}-{\overline{X}}_{Pi})}^T{Q}_{E3}(X_{Ei}-{\overline{X}}_{Ei})\left({\vartheta }_{E1}{v}_1^i+{\vartheta }_{E1}{v}_2^i\right)\hfill \\ \hfill & +\left.\left.{(X_{Ei}-{\overline{X}}_{Pi})}^T{Q}_{E4}(X_{Ei}-{\overline{X}}_{Pi})\left({\vartheta }_{E1}{v}_1^i+{\vartheta }_{E1}{v}_2^i\right)\right)ds\right]\hfill \\ \hfill & +\mathbb{E}\left[exp\left\{-{\int }_t^s{c}_E\left(x\left(r\right)\right)dr\right\}\left({\left(x^i\right)}^T\left(T\right)R_{E1}{x}^i\left(T\right)+m^P\left(T\right)-m^E\left(T\right)\right)\right]\hfill \end{array}$$

(16)

where $Q_{E1}\in {\mathbb{R}}^{n\times n},Q_{E3}\in {\mathbb{R}}^{M\times M},Q_{E4}\in {\mathbb{R}}^{M\times M},R_{E1}\in {\mathbb{R}}^{n\times n}$ are positive definite matrices; ${\psi }_E$ is a function with continuous first derivatives; $X_{Ei}\in {\mathbb{R}}^{5\times 5}$ is the vector representing the evader’s distribution state, which includes position and line-of-sight angles in three-dimensional space; ${\overline{X}}_{Ei}$ is the mean vector of the evader’s distribution state; ${\gamma }_{E1},{\gamma }_{E2}$ are constants; and $c_E$ is a state-dependent function. The cost function consists of two main parts:

• Running Cost: This part is similar to the previous ones but requires additional consideration of the influence of the evader group distribution on the cost.
• Terminal Cost: This part concerns the difference between the evader’s state and the target state at the terminal time, as well as the final value for the influence of each distribution.

2.3. The $ϵ$ Nash Equilibrium of Two-Group Games

In a two-group game, assume there are two groups: the pursuer group P and the evader group E. Each group has N participants. Each participant makes decisions based on their strategy, and their strategy depends not only on their own choices but also on the strategies of other members in the same group and the strategies of members in the other group. For the pursuer group P, the cost function of each pursuer $P_i$ is defined as

$$J_{P_i}(u_1^i,u_2^i,u_1^{-i},u_2^{-i}),$$

where $u_1^i,u_2^i$ are the strategies of the pursuer $P_i$, and $u_1^{-i},u_2^{-i}$ are the strategies of other members in the pursuer group $P_{-i}$. Under the $ϵ$-Nash equilibrium, assume the evader group’s strategy E remains unchanged, and the strategies of the other pursuers $P_{-i}$ remain fixed at $u_1^{-i},u_2^{-i}$. If pursuer $P_i$ deviates from the optimal strategy $u_1^{i,\ast },u_2^{i,\ast }$, the cost function of pursuer $P_i$ will increase by at most $ϵ$, i.e.

$$J_{P_i}(u_1^{i,\ast },u_2^{i,\ast },u_1^{-i},u_2^{-i})-ϵ\le J_{P_i}(u_1^i,u_2^i,u_1^{-i},u_2^{-i})\le J_{P_i}(u_1^{i,\ast },u_2^{i,\ast },u_1^{-i},u_2^{-i}).$$

(17)

where $ϵ$ is a positive constant that represents the tolerance for deviations from the optimal strategy. For the evader group E, the cost function of each evader $E_i$ is defined as

$$J_{E_i}(v_1^i,v_2^i,v_1^{-i},v_2^{-i}),$$

where $v_1^i,v_2^i$ are the strategies of the evader $E_i$, and $v_1^{-i},v_2^{-i}$ are the strategies of the other members in the evader group $E_{-i}$. Under the $ϵ$-Nash equilibrium, assume the pursuer group’s strategy P remains unchanged, and the strategies of the other evaders $E_{-i}$ remain fixed at $v_1^{-i},v_2^{-i}$. If evader $E_i$ deviates from the current optimal strategy $v_1^{i,\ast },v_2^{i,\ast }$, the cost function of evader $E_i$ will increase by at most $ϵ$, i.e.,

$$J_{E_i}(v_1^{i,\ast },v_2^{i,\ast },v_1^{-i},v_2^{-i})+ϵ\ge J_{E_i}(v_1^i,v_2^i,v_1^{-i},v_2^{-i})\ge J_{E_i}(v_1^{i,\ast },v_2^{i,\ast },v_1^{-i},v_2^{-i}).$$

(18)

Following the method of Huang et al. (2007) [2], the results can be generalized to the given system (8). Additionally, the system described by Equations (8), (17), and (18) can be viewed as a special case of the major–minor LQG mean field game (LQG MFG) systems analyzed by Firoozi, Jaimungal, and Caines (2020) [27]. For a comparison of various approaches to major–minor LQG mean field games, refer to Huang (2020) [28]. Furthermore, the analysis in [29] offered valuable insights into entropy regularization techniques applied to these models. In two-group games, the $ϵ$-Nash equilibrium is a relaxed equilibrium concept. This allows each participant to deviate from the optimal strategy within a certain range, without causing a significant loss in utility. Specifically, both the pursuers and the evaders can choose strategies that are not completely optimal, as long as the gain from deviation does not exceed the tolerance $ϵ$. In this context, the $ϵ$-Nash equilibrium provides an effective description that better captures the stability of the game and the behavior patterns of the participants. The concept of $ϵ$-Nash equilibrium in two-group games offers a relaxed equilibrium condition, taking into account real-world factors such as limited rationality and incomplete information. By allowing participants to deviate from their strategies to some extent, the $ϵ$-Nash equilibrium more accurately reflects the behavior and decision-making in real-world games.

3. The Distribution Functions for the Pursuers and Evaders

For convenience, in the subsequent derivations, we first introduce the distribution functions for the pursuer and evaders. Let $h\left(x\right):{\mathbb{R}}^d\to \mathbb{R}$ be a smooth function that is compactly supported and possesses at least second-order continuous derivatives. According to Itô’s lemma,

$$d\mathbb{E}\left[h\left(x\right)\right]=\mathbb{E}\left[\left({\displaystyle \frac{1}{2}}\mathrm{\Delta }h\left(x\right)+\alpha (s,x,m)·\nabla h\left(x\right)\right)\right]ds,$$

(19)

where $\alpha (s,x,m)$ is a function of s, x, and m, representing the dynamics of the system. We assume the difference in distribution of the pursuers and evaders is given by

$$m_s=m_s^P-m_s^E$$

(20)

where $m_s^P$ and $m_s^E$ are the distribution functions for the pursuer and the evaders, respectively. Substituting this into the equation, we obtain The following equations describe the evolution of the distribution of the system. We first start by writing down the change in the function $h\left(x\right)$ over the domain $\overline{\mathrm{\Omega }}$:

$$d{\int }_{\overline{\mathrm{\Omega }}}h\left(x\right)d{m}_s={\int }_{\overline{\mathrm{\Omega }}}\left[{\displaystyle \frac{1}{2}}\mathrm{\Delta }h\left(x\right)+\alpha (t,x,m_s)·\nabla h\left(x\right)\right]d{m}_{s}ds.$$

(21)

This equation shows the rate of change in the integral of $h\left(x\right)$ with respect to $m_s$, involving both the Laplacian of $h\left(x\right)$ and the interaction term $\alpha (t,x,m_s)·\nabla h\left(x\right)$, which represents the influence of the pursuers on the system’s evolution.

Next, we use the duality product to express the change in the system in terms of the inner product:

$$d\langle h,m_s\rangle =〈{\displaystyle \frac{1}{2}}\mathrm{\Delta }h+\alpha (s,·,m_s)·\nabla h,m_s〉ds,$$

(22)

where $\langle ·,·\rangle$ denotes the inner product. This equation expresses the rate of change in the inner product of h and $m_s$, which corresponds to the evolution of the system in the dual space.

Further, we can write the next expression for the system dynamics:

$$〈h,{\displaystyle \frac{d}{ds}}{m}_s〉ds=〈{\displaystyle \frac{1}{2}}\mathrm{\Delta }{m}_s-\mathrm{div}\left(\alpha (s,x_s,m_s)m_s\right),h〉ds.$$

(23)

This equation shows the rate of change in the inner product of h with the distribution $m_s$, where the evolution of $m_s$ is described in terms of its Laplacian and the divergence of the interaction term.

Finally, based on the above derivation, we obtain the following partial differential equation for $m_s$:

$${\partial }_s{m}_s={\displaystyle \frac{1}{2}}\mathrm{\Delta }{m}_s-\mathrm{div}\left(\alpha (s,x_s,m_s)m_s\right).$$

(24)

This equation is the key partial differential equation that governs the dynamics of the pursuer and evader system. It describes how the distribution $m_s$ evolves over time.

The term $\alpha (s,x_s,m_s)$ represents the relative information between the pursuer and the evaders, given by the following equation:

$$\alpha (s,x_s,m_s)=\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}.$$

This expression measures the relative information between the pursuer’s and evader’s distributions.

Finally, we can express the time evolution of the difference between the pursuer’s and evader’s distributions as

$${\displaystyle \frac{\partial (m_s^P-m_s^E)}{\partial s}}={\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_s.$$

(25)

This final equation describes the evolution of the difference between the pursuer’s and evader’s distributions. The term ${\eta }_s$ represents any external influence on the system.

The continuity equation for the pursuer is defined as

$${\displaystyle \frac{\partial \left(m_s^P\right)}{\partial s}}={\displaystyle \frac{\partial \left(m_s^E\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_P$$

(26)

The continuity equation for the evaders is defined similarly, but with the opposite direction of change:

$${\displaystyle \frac{\partial \left(m_s^E\right)}{\partial s}}=-{\displaystyle \frac{\partial \left(m_s^P\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_E$$

(27)

where ${\eta }_P\left(s\right)$ and ${\eta }_E\left(s\right)$ are random processes with zero mean and finite variance. The following definitions are used throughout the derivations: $m_s^P$ is the distribution function of the pursuer at time s. $m_s^E$ is the distribution function of the evaders at time s, which may be a Gaussian distribution or other distribution that describes the dynamics of the target. $\lambda$ is a positive constant that controls the speed at which the missile distribution $m_s^P$ converges to the target distribution $m_s^E$. ${\nabla }^2{m}_s^P$ is the Laplace operator, which represents the spatial diffusion of the missile distribution. ${\eta }_s$ is the perturbation term, which represents random noise in the system, typically modeled as white noise or a stochastic process. Finally, we have the normalization condition:

$${\int }_{\overline{\mathrm{\Omega }}}d\mu \left(X\right)={\int }_{{\mathbb{R}}^n}m\left(X\right)dX=1,$$

where $\overline{\mathrm{\Omega }}$ represents the support of the distribution, and the integral ensures that the total probability is normalized to 1.

4. The Optimal Feedback Strategies

For convenience, in the subsequent calculations, we simplify the cost function. The pursuer’s cost function is redefined as

$$\begin{array}{cc}\hfill J_{Pi}(x^i,s;{\mu }^P,{\mu }^E)& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^T{c}_P\left(x\left(r\right)\right)dr}{g}_{Pi}(x\left(T\right),{\mu }^P\left(T\right))\right.\hfill \\ \hfill & +{\int }_t^T{e}^{-{\int }_t^s{c}_P\left(x\left(r\right)\right)dr}{h}_{Pi}(x,s,{\mu }^P\left(s\right))ds]\hfill \end{array}$$

(28)

where This equation is a simplification of Equation (15), designed to facilitate subsequent calculations and proofs. We have divided Equation (15) into two parts: the first part represents the running cost, and the second part corresponds to the terminal cost.

The running cost includes the state during motion, the sum of the probability measures of the neighboring group distributions, and the functional changes between the neighboring groups of the pursuers and evaders. The terminal cost function incorporates the state at the terminal time and the distribution difference between pursuers and evaders at the terminal time.

$g_{Pi}(x\left(T\right),{\mu }^P\left(T\right))$ is the terminal cost, which depends on the state $x\left(T\right)$ and distribution ${\mu }^P\left(T\right)$ at the final time T, $h_{Pi}(x,s,{\mu }^P\left(s\right))$ is the running cost at each time s, with control strategies $u_1^{Pi},u_2^{Pi}$ and distribution $m_s$. The terminal cost $g_{Pi}(x\left(T\right),{\mu }^P\left(T\right))$ is defined as

$$g_{Pi}(x\left(T\right),{\mu }^P\left(T\right))={\left(x^i\left(T\right)\right)}^T{R}_{P1}{x}^i\left(T\right)+{\mu }^P\left(T\right)-{\mu }^E\left(T\right)$$

The running cost $h_{Pi}(x,s,{\mu }^P\left(s\right))$ is given by

$$\begin{array}{cc}\hfill h_{Pi}(x,s,{\mu }^P\left(s\right))& =\left[\left(x^i{\left(s\right)}^T{Q}_{P1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right)\right.\hfill \\ & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·d{x}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P1}{u}_2^i\right)\hfill \\ & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E1}{u}_2^i\right)]\hfill \end{array}$$

Assumption 2. 

For the convenience of the subsequent calculations of the cost function $J(u,v)$, it is assumed that [30]: There exist positive constants $C_L$, $C_g$, and $p\ge 2$ such that

$$\begin{array}{cc}\hfill & \left|e^{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr}{h}_{Pi}(x^i,s,{\mu }^P\left(s\right))\right|\le C_L\left(1+\parallel x^i{\parallel }^p+|u_1^i{|}^p+{\left|u_2^i\right|}^p\right)\hfill \end{array}$$

(29)

$$\left|e^{-{\int }_t^s{c}_p\left(x^i\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right))\right|\le C_g\left(1+\parallel x^i{\parallel }^p\right)$$

(30)

To process the terminal value $g_{Pi}(x_T,m_T)$ using the Itô–Wentzell formula, we consider the auxiliary process $p_{Pi}\left(s\right)$, which represents the terminal condition. The dynamics of $p_{Pi}\left(t\right)$ are given by

$$d{p}_{Pi}\left(s\right)=a_{Pi}(s,x^i,m^P\left(t\right))dt+b_{Pi}(s,x^i,m^P\left(t\right))d{W}^i\left(s\right),$$

(31)

with the terminal condition as follows:

$$p_{Pi}\left(T\right)=g_{Pi}(x_T,m_T).$$

(32)

By substituting the above formula and applying the Itô–Wentzell formula, we have

$$\begin{array}{cc}\hfill d{p}_{Pi}\left(s\right)=& \left({\displaystyle \frac{\partial g_{Pi}(s,x^i\left(t\right),m^P\left(t\right))}{\partial s}}+a_{Pi}(s,x^i\left(t\right),m^P\left(t\right))·{\displaystyle \frac{\partial g_{Pi}(s,x^i\left(s\right),m^P\left(s\right))}{\partial x}}\right.\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left(b_{Pi}(t,x\left(s\right),m\left(t\right))b_{Pi}{(s,x\left(s\right),m\left(s\right))}^T{\displaystyle \frac{{\partial }^2{g}_{Pi}(s,x^i)}{\partial x^2}}\right.\hfill \\ \hfill & \left.+{\displaystyle \frac{\delta g_{Pi}(s,x^i)}{\delta m}}·\dot{m}\left(s\right)\right)ds\hfill \\ \hfill & +{\displaystyle \frac{\partial g_{Pi}(s,x^i)}{\partial x}}·{\sigma }^i(s,x\left(s\right),m^P)d{W}^i\left(s\right)\hfill \end{array}$$

(33)

Derivative with respect to time t. Using the chain rule,

$${\displaystyle \frac{\partial g(t,x^i,m^P)}{\partial s}}={\displaystyle \frac{\partial g(t,x^i,m^P)}{\partial x}}{\displaystyle \frac{\partial x^i\left(s\right)}{\partial s}}+{\displaystyle \frac{\partial g(t,x^i,m^P)}{\partial m_s^P\left(X\right)}}{\displaystyle \frac{\partial m_s^P\left(X\right)}{\partial s}}+{\displaystyle \frac{\partial g(t,x^i,m^P)}{\partial m_s^E\left(X\right)}}{\displaystyle \frac{\partial m_s^E\left(X\right)}{\partial s}}$$

(34)

Substituting the evolution equations for the pursuer and evaders,

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial m_s^P\left(X\right)}{\partial s}}& ={\displaystyle \frac{\partial m_s^E\left(X\right)}{\partial s}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\nabla }^2\left(m_s^P\left(X\right)-m_s^E\left(X\right)\right)\hfill \\ \hfill & -\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)\hfill \\ \hfill & +{\eta }_P\hfill \end{array}$$

(35)

According to the Equation (34), the following is the distribution change of the evader:

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial m_s^E\left(X\right)}{\partial s}}& =-{\displaystyle \frac{\partial m_s^P\left(X\right)}{\partial s}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}{\nabla }^2\left(m_s^P\left(X\right)-m_s^E\left(X\right)\right)\hfill \\ \hfill & -\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)\hfill \\ \hfill & +{\eta }_E\hfill \end{array}$$

(36)

Substituting the above equations, we obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial g}{\partial s}}& ={\displaystyle \frac{\partial g}{\partial x}}{\displaystyle \frac{\partial x}{\partial s}}\hfill \\ \hfill & +{\displaystyle \frac{\partial g}{\partial m_s^P\left(X\right)}}\left({\displaystyle \frac{\partial m_s^E\left(X\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P\left(X\right)-m_s^E\left(X\right))\right.\hfill \\ \hfill & \left.-\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)+{\eta }_P\right)\hfill \\ \hfill & +{\displaystyle \frac{\partial g}{\partial m_s^E\left(X\right)}}\left(-{\displaystyle \frac{\partial m_s^P\left(X\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P\left(X\right)-m_s^E\left(X\right))\right.\hfill \\ \hfill & \left.-\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)+{\eta }_E\right)\hfill \end{array}$$

(37)

Therefore, we can obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial g}{\partial s}}& ={\displaystyle \frac{\partial g}{\partial x}}{\displaystyle \frac{\partial x}{\partial s}}\hfill \\ \hfill & +\left({\displaystyle \frac{\partial m_s^E\left(X\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P\left(X\right)-m_s^E\left(X\right))\right.\hfill \\ \hfill & \left.-\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)+{\eta }_P\right)\hfill \\ \hfill & -\left(-{\displaystyle \frac{\partial m_s^P\left(X\right)}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P\left(X\right)-m_s^E\left(X\right))\right.\hfill \\ \hfill & -\left.\mathrm{div}\left(\int m_s^P\left(X\right)ln{\displaystyle \frac{m_s^P\left(X\right)}{m_s^E\left(X\right)}}ds·(m_s^P\left(X\right)-m_s^E\left(X\right))\right)+{\eta }_E\right)\hfill \end{array}$$

(38)

Taking the Derivative with Respect to x: ${\displaystyle \frac{\partial g}{\partial x}}=2R_{P1}{x}^i\left(T\right)$ This is the derivative of $x^T\left(T\right)R_{P1}x\left(T\right)$. Taking the Second Derivative with Respect to x: ${\displaystyle \frac{{\partial }^{2}g}{\partial x^2}}=2R_{P1}$ This is the second derivative of $x^T\left(T\right)R_{P1}x\left(T\right)$. Variational Derivative with Respect to m: ${\displaystyle \frac{\delta g}{\delta m}}={\displaystyle \frac{\partial g}{\partial m}}$ Substituting Derivative Terms into the Formula. Substituting all the terms, we obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial g}{\partial s}}& =2R_{P1}x\left(T\right)\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2+b_{v1}^i{v}_1+b_{v2}^i{v}_2\right)\hfill \\ \hfill & +\left({\displaystyle \frac{\partial m_s^E}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_P\right)\hfill \\ \hfill & -\left(-{\displaystyle \frac{\partial m_s^P}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_E\right)\hfill \end{array}$$

(39)

Pursuer’s Dynamic Equation:

$$\begin{array}{cc}\hfill d{p}_{Pi}\left(s\right)& =\left({\displaystyle \frac{\partial g_{Pi}}{\partial s}}+a_{Pi}(s,x^i\left(s\right),m\left(s\right))·\left(2R_{P1}{x}^i\left(T\right)\right)\right.\hfill \\ \hfill & +\left.{\displaystyle \frac{1}{2}}\mathrm{Tr}\left(b_{Pi}{b}_{Pi}^T·2R_{P1}\right)+{\partial }_s{m}^P\right)ds\hfill \\ \hfill & +\left(2R_{P1}{x}^i\left(T\right)\right)·\sigma (s,x^i\left(s\right),m^P\left(s\right))d{W}^i\left(s\right)\hfill \\ \hfill & =\left(2R_{P1}{x}^i\left(T\right)\left(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v2}^i{v}_2^i\right)\right.\hfill \\ \hfill & +{\displaystyle \frac{\partial m_s^E}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)-\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)+{\eta }_P\hfill \\ \hfill & +{\displaystyle \frac{\partial m_s^P}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^P-m_s^E)+\mathrm{div}\left(\int m_s^{P}ln{\displaystyle \frac{m_s^P}{m_s^E}}ds·(m_s^P-m_s^E)\right)-{\eta }_E\hfill \\ \hfill & +\left.{\displaystyle \frac{1}{2}}\mathrm{Tr}\left(b{b}^T·2R_{P1}\right)+{\partial }_s{m}^P\right)ds\hfill \\ \hfill & +\left(2R_{P1}{x}^i\left(T\right)\right)·{\sigma }^i(s,x^i\left(s\right),m^P)d{W}^i\left(s\right)\hfill \end{array}$$

(40)

Evader’s Dynamic Equation:

$$\begin{array}{cc}\hfill d{p}_{Ei}\left(s\right)& =\left({\displaystyle \frac{\partial g_{Ei}}{\partial s}}+a_{Ei}(s,x^i\left(s\right),m\left(s\right))·\left(2R_{P1}{x}^i\left(T\right)\right)\right.\hfill \\ \hfill & +\left.{\displaystyle \frac{1}{2}}\mathrm{Tr}\left(b_{Ei}{b}_{Ei}^T·2R_{P1}\right)+{\partial }_s{m}^E\right)ds\hfill \\ \hfill & +\left(2R_{P1}{x}^i\left(T\right)\right)·\sigma (s,x^i\left(s\right),m^E\left(s\right))d{W}^i\left(s\right)\hfill \\ \hfill & =\left(2R_{P1}{x}^i\left(T\right)\left(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v2}^i{v}_2^i\right)\right.\hfill \\ \hfill & +{\displaystyle \frac{\partial m_s^P}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^E-m_s^P)-\mathrm{div}\left(\int m_s^{E}ln{\displaystyle \frac{m_s^E}{m_s^P}}ds·(m_s^E-m_s^P)\right)+{\eta }_E\hfill \\ \hfill & +{\displaystyle \frac{\partial m_s^E}{\partial s}}+{\displaystyle \frac{1}{2}}{\nabla }^2(m_s^E-m_s^P)+\mathrm{div}\left(\int m_s^{E}ln{\displaystyle \frac{m_s^E}{m_s^P}}ds·(m_s^E-m_s^P)\right)-{\eta }_P\hfill \\ \hfill & +\left.{\displaystyle \frac{1}{2}}\mathrm{Tr}\left(b{b}^T·2R_{P1}\right)+{\partial }_s{m}^E\right)ds\hfill \\ \hfill & +\left(2R_{E1}{x}^i\left(T\right)\right)·{\sigma }^i(s,x^i\left(s\right),m^E)d{W}^i\left(s\right)\hfill \end{array}$$

(41)

To incorporate the terminal condition into the optimization process, we introduce an auxiliary process $p\left(t\right)$, with the dynamics as shown above. The terminal condition is now processed dynamically using the formula:

$$V_{Pi}(x^i,s)=\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^s{c}_{Pi}\left(x^i\left(r\right)\right)dr}{h}_{pi}\left(x^i\right)ds+e^{-{\int }_t^{s}c\left(x^i\left(r\right)\right)dr}{\left(x^i\right)}^T{p}_{Pi}\left(s\right)x^{i}ds+V_{Pi}(x_{s+ds}^i,s+ds)\right]$$

(42)

where $e^{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr}$ represents the discount factor, accounting for the accumulated cost over time [31]. Next, we apply Ito’s Lemma to expand $V_{Pi}(x_{s+ds}^i,s+ds)$:

$$V_{Pi}(x_{s+ds}^i,s+ds)=V_{Pi}(x^i,s)+{\displaystyle \frac{\partial V_{Pi}}{\partial s}}ds+{\displaystyle \frac{\partial V_{Pi}}{\partial x}}d{x}^i+{\displaystyle \frac{1}{2}}{\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}{\left(d{x}^i\right)}^2$$

(43)

We then compute the expectation:

$$\begin{array}{cc}\hfill \mathbb{E}\left[V_{Pi}(x_{s+ds}^i,s+ds)\right]& =V_{Pi}(x^i,s)\hfill \\ \hfill & +\mathbb{E}\left[{\displaystyle \frac{\partial V_{Pi}}{\partial s}}+{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{\displaystyle \frac{\partial V_{Pi}}{\partial x}}\right.\hfill \\ \hfill & \left.+{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}\right]ds\hfill \end{array}$$

(44)

Substituting this into the dynamic programming equation,

$$\begin{array}{cc}\hfill V_{Pi}(x^i,t)& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr}\left(h_{Pi}(x^i,s)ds\right.\right.\hfill \\ \hfill & \left.+{\left(x^i\right)}^T{p}_{Pi}\left(T\right)x^{i}ds\right)+V_{Pi}(x^i,t)\hfill \\ \hfill & +{\displaystyle \frac{\partial V_{Pi}}{\partial s}}ds+{\displaystyle \frac{\partial V_{Pi}}{\partial x}}d{x}^i\hfill \\ \hfill & \left.+{\displaystyle \frac{1}{2}}{\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}{\left(d{x}^i\right)}^2\right]\hfill \end{array}$$

(45)

Simplifying yields the Hamilton–Jacobi–Bellman (HJB) equation:

$$\begin{array}{cc}\hfill V_{Pi}(x^i,t)& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}[e^{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr}\left(h_{Pi}(x^i,{\mu }^P)ds+p_{Pi}\left(s\right)ds\right)+V_{Pi}(x^i,t)+{\displaystyle \frac{\partial V_{Pi}}{\partial s}}ds\hfill \\ \hfill & +{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{\displaystyle \frac{\partial V_{Pi}}{\partial x}}ds\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}ds]\hfill \end{array}$$

(46)

Substituting $h_{Pi}(s,x^i)$ into the equation,

$$\begin{array}{cc}\hfill V_{Pi}(x^i,s)& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}[e^{-{\int }_t^s{c}_P\left(x\left(r\right)\right)dr}\left(x^i{\left(s\right)}^T{Q}_{P1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right)\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·d{x}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P1}{u}_2^i\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E1}{u}_2^i\right)+p_{Pi}\left(T\right))ds\hfill \\ \hfill & +V_{Pi}(x^i,s)+{\displaystyle \frac{\partial V_{Pi}}{\partial s}}ds\hfill \\ \hfill & +{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{\displaystyle \frac{\partial V_{Pi}}{\partial x}}ds\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}ds]\hfill \end{array}$$

(47)

where $e^{-{\int }_t^s{c}_P\left(x^i\left(r\right)\right)dr}$ is the discount factor affecting all terms. $h_{Pi}(s,x^i)$ represents the instantaneous cost, while $p_{Pi}\left(s\right)$ handles the terminal condition. $V_{Pi}(x^i,t)$ is the value function of the pursuer’s strategy at time t and state $x^i$. The term ${\displaystyle \frac{\partial V_{Pi}}{\partial s}}$ represents the time derivative of the value function, indicating the rate of change in $V_{Pi}$ with respect to time. ${\displaystyle \frac{\partial V_{Pi}}{\partial x}}$ is the spatial derivative, showing how $V_{Pi}$ changes with respect to the state variable $x^i$. ${\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}$ is the second-order spatial derivative, describing the curvature of $V_{Pi}$ in space. The term ${(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v_2}^i{v}_2^i)}^T{\displaystyle \frac{\partial V_{Pi}}{\partial x}}$ captures the interaction between the state process dynamics and the spatial gradient of the value function. ${\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}$ represents the diffusion term, accounting for uncertainty or noise in the process. We first simplify and obtain the following HJB equation for the pursuer:

$$\begin{array}{cc}\hfill 0& ={\displaystyle \frac{\partial V_{Pi}}{\partial s}}\hfill \\ \hfill & +\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^T{c}_P\left(x\left(r\right)\right)dr}\right(\left(x^i{\left(s\right)}^T{Q}_{P1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right)\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·d{x}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P1}{u}_2^i\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E1}{u}_2^i\right)+p_{Pi}\left(t\right)\hfill \\ \hfill & +{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{\displaystyle \frac{\partial V_{Pi}}{\partial x}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}]\hfill \end{array}$$

(48)

Similarly, the HJB equation for the evaders is obtained as

$$\begin{array}{cc}\hfill 0& ={\displaystyle \frac{\partial V_{Ei}}{\partial s}}\hfill \\ \hfill & +\underset{u_i\in {\pi }_v}{sup}\mathbb{E}\left[e^{-{\int }_t^T{c}_E\left(x\left(r\right)\right)dr}\right(\left(x^i{\left(s\right)}^T{Q}_{E1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(v_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(v_2^i\right)}^2}{2}}\right)\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_E\left({\partial }_t{\psi }_E(s,x^i)+{\partial }_x{\psi }_E(s,x^i)·d{x}^i\right)d({\mu }^E\left(s\right)-{\mu }^P\left(s\right))\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Ei})}^T{Q}_{E3}(X_{Ei}-{\overline{X}}_{Ei})\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Pi})}^T{Q}_{E4}(X_{Ei}-{\overline{X}}_{Pi})+p_{Ei}\left(t\right))\hfill \\ \hfill & +{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{\displaystyle \frac{\partial V_{Ei}}{\partial x}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){\displaystyle \frac{{\partial }^2{V}_{Ei}}{\partial x^2}}]\hfill \end{array}$$

(49)

Assumption 3. 

Initial Conditions: We assume that there exists a constant K such that, for all $N\in \mathbb{N}$ (the number of players), the initial state $X_0^i$ of each player i lies within a closed ball ${\mathbb{B}}_K\left(0\right)$ centered on the initial mean $\overline{X}$ with radius K. In other words, the initial states of all players are constrained within a finite region.

This assumption ensures that, when analyzing and solving the game, the initial states of the players are not excessively large, thereby avoiding issues of instability or intractability due to extreme initial conditions. This is crucial for ensuring the existence and solvability of the solution. Based on Theorem 4.12 in [32], we assume that the population distribution $\mu$ is absolutely continuous and satisfies the following partial differential equation as its distributional solution:

$${\int }_t^T{\int }_{\overline{\mathrm{\Omega }}}\left({\partial }_t\psi (t,x)+\nabla \psi (t,x)·v(t,x)\right)d{\mu }_t\left(X\right)dt=0\forall \psi \in C_c^1((t,T)\times \overline{\mathrm{\Omega }}),$$

where ${\mu }_t$ is the population distribution at time t, and $v(t,x)$ is the Borel vector field associated with individual decisions. This condition ensures that the evolution of the population follows the distributional solution of the above equation, which directly influences the strategies of individuals in the game. In the game model, we assume that a player’s behavior is not only driven by their own state but also influenced by the distribution of the entire population. This influence is reflected in the second term of the HJB equation, through the distributional solution [33]. In this case, individuals respond to the distribution of the population in their decision-making process, ensuring that each player’s behavior is consistent with the distribution of the overall population. We assume that the value functions for the pursuer and the evaders are given by the following expressions:

$$V_{Pi}={\left(x^i\right)}^T{S}_1^{Pi}{x}^i+S_2^{Pi}{x}^i+S_3^{Pi}$$

(50)

Equation (50) represents the value function for the i-th pursuer. The terms $S_1^{Pi}$, $S_2^{Pi}$, and $S_3^{Pi}$ are parameters that determine the quadratic, linear, and constant contributions to the value function, respectively, while $x^i$ is the position of the i-th pursuer.

$$V_{Ei}={\left(x^i\right)}^T{S}_1^{Ei}{x}^i+S_2^{Ei}{x}^i+S_3^{Ei}$$

(51)

Equation (51) represents the value function for the i-th evader. Similarly, $S_1^{Ei}$, $S_2^{Ei}$, and $S_3^{Ei}$ are parameters that define the quadratic, linear, and constant terms, while $x^i$ is the position of the i-th evader.

Next, we compute the partial derivatives of the value functions with respect to s and x:

$${\displaystyle \frac{\partial V_{Pi}}{\partial s}}={\left(x^i\right)}^T{\displaystyle \frac{\partial S_1^{Pi}}{\partial s}}{x}^i+{\displaystyle \frac{\partial S_2^{Pi}}{\partial s}}{x}^i+{\displaystyle \frac{\partial S_3^{Pi}}{\partial s}},{\displaystyle \frac{\partial V_{Pi}}{\partial x}}=S_1^{Pi}{x}^i+S_2^{Pi},{\displaystyle \frac{{\partial }^2{V}_{Pi}}{\partial x^2}}=S_1^{Pi}$$

Substituting these derivatives into the equation, we obtain the following HJB equation for the pursuer:

$$\begin{array}{cc}\hfill 0& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^T{c}_P\left(x\left(r\right)\right)dr}\right(\left(x^i{\left(s\right)}^T{Q}_{P1}{x}^i\left(s\right)+{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right)\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·\left(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v_2}^i{v}_2^i\right)\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P1}{u}_2^i\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E1}{u}_2^i\right)+p_{Pi}\left(T\right))\hfill \\ \hfill & +\left({\left(x^i\right)}^T{\displaystyle \frac{\partial S_1^{Pi}}{\partial s}}{x}^i+{\displaystyle \frac{\partial S_2^{Pi}}{\partial s}}{x}^i+{\displaystyle \frac{\partial S_3^{Pi}}{\partial s}}\right)\hfill \\ \hfill & +{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T\left(S_1^{Pi}{x}^i+S_2^{Pi}\right)\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)S_1^{Pi}]\hfill \end{array}$$

(52)

Next, we compute the time derivatives of the parameters $S_1^{Pi}$, $S_2^{Pi}$, and $S_3^{Pi}$:

$${\displaystyle \frac{\partial S_1^{Pi}}{\partial s}}=-e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)ds}\left(Q_{P1}\right)$$

(53)

$${\displaystyle \frac{\partial S_2^{Pi}}{\partial s}}=-{\left(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v_2}^i{v}_2^i\right)}^T\left(S_1^{Pi}\right)$$

(54)

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial S_3^{Pi}}{\partial s}}& =-e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)ds}\left[{\displaystyle \frac{{\left(u_1^i\right)}^2}{2}}+{\displaystyle \frac{{\left(u_2^i\right)}^2}{2}}\right.\hfill \\ \hfill & +{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_t{\psi }_P(s,x^i)+{\partial }_x{\psi }_P(s,x^i)·\left(b^i+b_{u1}^i{u}_1^i+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1^i+b_{v_2}^i{v}_2^i\right)\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}{u}_1^i+{\gamma }_{P2}{u}_2^i\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}{u}_1^i+{\gamma }_{E2}{u}_2^i\right)+p_{Pi}\left(s\right)\hfill \\ \hfill & +\left.{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T\left(S_2^{Pi}\right)\right]\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)S_1^{Pi}\hfill \end{array}$$

(55)

We assume the boundary conditions at time T are

$$S_1^{Pi}\left(T\right)=0,S_2^{Pi}\left(T\right)=0,S_3^{Pi}\left(T\right)=0.$$

The optimal strategy for the pursuer can be derived as follows:

$$\begin{array}{cc}\hfill u_1^{i,\ast }& =-\left[{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_x{\psi }_P(s,x^i)·b_{u1}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\right.\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P1}\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E1}\right)\hfill \\ \hfill & +\left.{\left(b_{u1}^i\right)}^T\left(S_1^{Pi}{x}^i+S_2^{Pi}\right)\right]\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill u_2^{i,\ast }& =-\left[{\int }_{\mathrm{\Omega }}{\lambda }_P\left({\partial }_x{\psi }_P(s,x^i)·b_{u2}^i\right)d({\mu }^P\left(s\right)-{\mu }^E\left(s\right))\right.\hfill \\ \hfill & +exp\left\{{\displaystyle \frac{1}{{(X_{Pi}-{\overline{X}}_{Pi})}^T{Q}_{P3}(X_{Pi}-{\overline{X}}_{Pi})}}\right\}\left({\gamma }_{P2}\right)\hfill \\ \hfill & +{(X_{Pi}-{\overline{X}}_{Ei})}^T{Q}_{P4}(X_{Pi}-{\overline{X}}_{Ei})\left({\gamma }_{E2}\right)\hfill \\ \hfill & +\left.{\left(b_{u2}^i\right)}^T\left(S_1^{Pi}{x}^i+S_2^{Pi}\right)\right]\hfill \end{array}$$

(57)

The optimal strategy for the evaders is derived as follows:

$$\begin{array}{cc}\hfill v_1^{i,\ast }& =\left[{\int }_{\mathrm{\Omega }}{\lambda }_E\left({\partial }_x{\psi }_E(s,x^i)·b_{v1}^i\right)d({\mu }^E\left(s\right)-{\mu }^P\left(s\right))\right.\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Ei})}^T{Q}_{E3}(X_{Ei}-{\overline{X}}_{Ei})\left({\vartheta }_{v1}\right)\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Pi})}^T{Q}_{P4}(X_{Ei}-{\overline{X}}_{Pi})\left({\vartheta }_{v1}\right)\hfill \\ \hfill & +\left.{\left(b_{v1}^i\right)}^T\left(S_1^{Ei}{x}^i+S_2^{Ei}\right)\right]\hfill \end{array}$$

(58)

$$\begin{array}{cc}\hfill v_2^{i,\ast }& =-\left[{\int }_{\mathrm{\Omega }}{\lambda }_E\left({\partial }_x{\psi }_E(s,x^i)·b_{v2}^i\right)d({\mu }^E\left(s\right)-{\mu }^P\left(s\right))\right.\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Ei})}^T{Q}_{E3}(X_{Ei}-{\overline{X}}_{Ei})\left({\vartheta }_{v2}\right)\hfill \\ \hfill & +{(X_{Ei}-{\overline{X}}_{Pi})}^T{Q}_{P4}(X_{Ei}-{\overline{X}}_{Pi})\left({\vartheta }_{v2}\right)\hfill \\ \hfill & +\left.{\left(b_{v2}^i\right)}^T\left(S_1^E{x}^i+S_2^E\right)\right]\hfill \end{array}$$

(59)

5. $ϵ$ Nash Equilibrium

In this section, we first prove the uniqueness of the solution to the forward-backward stochastic differential equations using monotone operators, as well as the uniqueness of the distribution function. Then, we demonstrate the boundedness of the state and, based on the uniqueness of the solution and the boundedness of the state, we establish the $ϵ$ Nash equilibrium.

Theorem 1. 

Under the conditions of Assumptions 1 and 3, the solution to the forward-backward stochastic differential equations is unique, and for each player, the value function is unique.

Proof. 

Suppose the state satisfies the uniqueness condition. We assume that the time-independent condition holds. Then, the backward stochastic differential equations for $S_1^{Pi}$, $S_2^{Pi}$, and $S_3^{Pi}$ have a unique solution: Define the linear operator $\mathbb{A}$ as follows:

$$\mathbb{A}\left(\begin{array}{c}{S}_1^{Pi}\\ S_2^{Pi}\\ S_3^{Pi}\end{array}\right)=\left(\begin{array}{c}-e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)ds}\left(Q_{P1}\right)\\ -{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{S}_1^P\\ \begin{array}{cc}\hfill & -{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{S}_2^P-{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)S_1^P\hfill \end{array}\end{array}\right)$$

(60)

To obtain the unique solution for Equation (33), we only need to verify whether the monotonicity condition in [34] is satisfied. If there is no ambiguity, $\langle ·,·\rangle$ represents the usual inner product in Euclidean space. Then, the following can be obtained:

$$\begin{array}{cc}\hfill 〈\mathbb{A}\left(\begin{array}{c}{S}_1^{Pi}\\ S_2^{Pi}\\ S_3^{Pi}\end{array}\right),\left(\begin{array}{c}{S}_1^{Pi}\\ S_2^{Pi}\\ S_3^{Pi}\end{array}\right)〉& ={\left(\begin{array}{ccc}{S}_1^{Pi}& S_2^{Pi}& S_3^{Pi}\end{array}\right)}^T\hfill \\ \hfill & \left(\begin{array}{c}-e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)ds}\left(Q_{P1}\right)\\ -{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{S}_1^P\\ -{\left(b^i+b_{u1}^i{u}_1+b_{u2}^i{u}_2^i+b_{v1}^i{v}_1+b_{v_2}^i{v}_2\right)}^T{S}_2^P\\ -{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)S_1^P\end{array}\right)\hfill \end{array}$$

(61)

Applying Young’s inequality, we obtain

$$\le -e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)}∥Q_{P1}∥{∥S^{Pi}∥}^2-2∥B^i∥{∥S^{Pi}∥}^2-{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right){∥S^{Pi}∥}^2$$

$$\le \left(-e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)}∥Q_{P1}∥-2∥B^i∥-{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)\right){∥S^{Pi}∥}^2<0$$

where: $S^{Pi}=min∥S_i^{Pi}∥,i=1,2,3.$ Since $\left(e^{-{\int }_t^T{c}_P\left(x\left(s\right)\right)}∥Q_{P1}∥+2∥B^i∥+{\displaystyle \frac{1}{2}}\mathrm{Tr}\left({\sigma }^i{\left({\sigma }^i\right)}^T\right)\right)>0$ is satisfied, the monotonicity condition in this theorem holds, and therefore the value function is unique, which implies that the corresponding optimal strategy is also unique. The proof for the evaders follows a similar process and is omitted here. □

Theorem 2. 

Under the conditions set by Assumptions 1 and 2, the distribution processes $m^P$ and $m^E$ satisfy the uniqueness property, meaning that they each have a unique solution.

Proof. 

We will prove the uniqueness of the distribution process by assuming the contrary; that is, there are two different solutions $m^P\left(s\right),m^P\left(T\right)$ and $m^P{\left(s\right)}^{\prime },m^P{\left(T\right)}^{\prime }$ for the same initial conditions. Let $m^P\left(s\right)$ and $m^P{\left(s\right)}^{\prime }$ represent two different solutions. We consider the following equations:

$$m^P\left(s\right)=m_s^P\left(X_s\right)\mathrm{and}{m}^P{\left(s\right)}^{\prime }=m_s^P\left(X_s^{\prime }\right)$$

(62)

where $X,x$ and $X^{\prime },x^{\prime }$ are the state processes under the distributions $m^P$ and $m^{P^{\prime }}$, respectively. The optimal control strategies $u_1^{i,\ast },u_2^{i,\ast }$ correspond to the strategies under $m^P$, while the optimal control strategies ${\left(u_1^{i,\ast }\right)}^{\prime },{\left(u_2^{i,\ast }\right)}^{\prime }$ correspond to the strategies under $m^{P^{\prime }}$. Let ${\mathcal{J}}_{Pi}(u_1^i,u_2^i,m^P)$ be the cost functional associated with the control problem:

$${\mathcal{J}}_{Pi}(u_1^i,u_2^i,m^P)=\underset{u_i\in {\pi }_u}{inf}\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),m^P\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P\left(s\right))ds\right]$$

(63)

where $g_{Pi}(x^i\left(T\right),m^P)$ is the terminal cost function, and $h_{Pi}(x^i,X_{Pi},m^P\left(s\right))$ is the running cost function. We now compare the cost functionals for both distributions m and $m^{\prime }$. Since we assume $m\ne m^{\prime }$, we have

$${\mathcal{J}}_{Pi}(u_1^i,u_2^i,{\mu }^P)<{\mathcal{J}}_{Pi}({\left(u_1^i\right)}^{\prime },{\left(u_2^i\right)}^{\prime },{\mu }^P)$$

and

$${\mathcal{J}}_{Pi}({\left(u_1^i\right)}^{\prime },{\left(u_2^i\right)}^{\prime },{\mu }^{P^{\prime }})<\mathcal{J}(u_1^i,u_2^i,{\mu }^{P^{\prime }})$$

This means that the control strategies corresponding to ${\left(u_1^i\right)}^{\prime },{\left(u_2^i\right)}^{\prime }$ and $u_1^i,u_2^i$ give different cost values. By adding the two inequalities, we obtain

$$\begin{array}{cc}\hfill & \mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P)ds\right]\hfill \\ \hfill & <\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^{i^{\prime }}\left(T\right),{\mu }^P\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^{i^{\prime }},X_{Pi}^{\prime },{\mu }^P)ds\right],\hfill \\ \hfill & \mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^{i^{\prime }}\left(T\right),{\mu }^{P^{\prime }}\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^{i^{\prime }},X_{Pi}^{\prime },{\mu }^{P^{\prime }})ds\right]\hfill \\ \hfill & <\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),{\mu }^{P^{\prime }}\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^{P^{\prime }})ds\right].\hfill \end{array}$$

(64)

By subtracting the two inequalities, we obtain

$$\begin{array}{cc}\hfill & \mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P)ds\right]\hfill \\ \hfill & -\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^i\left(T\right),{\mu }^{P^{\prime }}\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^{P^{\prime }})ds\right]\hfill \\ \hfill & -\left(\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^{i^{\prime }}\left(T\right),{\mu }^P\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^{i^{\prime }},X_{Pi}^{\prime },{\mu }^P)ds\right]\right.\hfill \\ \hfill & -\left.\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x^{i^{\prime }}\left(T\right),{\mu }^{P^{\prime }}\left(T\right))+{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^{i^{\prime }},X_{Pi}^{\prime },{\mu }^{P^{\prime }})ds\right]\right)\hfill \\ \hfill & <0\hfill \end{array}$$

(65)

This implies that the difference in terminal costs must be negative. Thus, we have

$${\int }_{\mathrm{\Omega }}\left[g_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right))-g_{Pi}(x^i\left(T\right),{\mu }^{P^{\prime }}\left(T\right))\right]d{\mu }_T^P-{\int }_{\mathrm{\Omega }}{g}_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right))-g_{Pi}(x^i\left(T\right),{\mu }^{P^{\prime }}\left(T\right))d{\mu }_T^{P^{\prime }}$$

Then, it can be obtained that

$$\begin{array}{cc}& {\int }_{\mathrm{\Omega }}\left[{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P)ds-{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^{P^{\prime }})ds\right]d{\mu }_T^P\hfill \\ & -{\int }_{\mathrm{\Omega }}\left[{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P)ds-{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^{P^{\prime }})ds\right]d{\mu }_T^{P^{\prime }}\hfill \end{array}$$

This leads to the following contradiction:

$$0\le {\int }_{\mathrm{\Omega }}\left[g_{Pi}(x^i,{\mu }^P\left(T\right))-g_{Pi}(x^i,{\mu }^{P^{\prime }})\right]d({\mu }_T^P-{\mu }_T^{P^{\prime }})<0$$

(66)

$$0\le {\int }_{\mathrm{\Omega }}\left[{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^P)ds-{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x^i,X_{Pi},{\mu }^{P^{\prime }})ds\right]d({\mu }_T^P-{\mu }_T^{P^{\prime }})<0$$

(67)

The inequality above cannot hold, because the integral over a non-negative measure must be non-negative. Therefore, our assumption that ${\mu }^P\ne {\mu }^{P^{\prime }}$ is false. The distribution process is unique. □

Lemma 1. 

Suppose the initial assumption about the distribution state holds, then there exists a constant $C>0$ such that for all $r\in [t,T]$ and $i\in \{1,\dots ,N\}$, the following holds:

$$\mathbb{E}∥x^i\left(r\right)∥\le C.$$

(68)

Proof. 

Since $x^i\left(r\right)$ is absolutely continuous in the state Equation (8), we can first express it as

$$\mathbb{E}∥x^i\left(r\right)∥=\mathbb{E}∥x_t^i+{\int }_t^r{\dot{x}}^i\left(s\right)ds∥,$$

(69)

where ${\dot{x}}^i\left(s\right)$ is the derivative of the state evolution equation. Next, applying the triangle inequality, we obtain

$$\mathbb{E}∥x^i\left(r\right)∥\le \mathbb{E}∥x_t^i∥+\mathbb{E}∥{\int }_t^r{B}^i\left(s\right)ds∥.$$

(70)

We now expand $B^i\left(s\right)$ according to the known system model:

$$B^i\left(s\right)=b^i+b_{u1}^i{u}_1^i\left(s\right)+b_{u2}^i{u}_2^i\left(s\right)+b_{v1}^i{v}_1^i\left(s\right)+b_{v2}^i{v}_2^i\left(s\right),$$

where $u_1^i$, $u_2^i$, $v_1^i$, and $v_2^i$ are the control inputs. Thus, we have

$$\mathbb{E}∥x^i\left(r\right)∥\le \mathbb{E}∥x_t^i∥+\mathbb{E}∥{\int }_t^r\left(b^i+b_{u1}^i{u}_1^i\left(s\right)+b_{u2}^i{u}_2^i\left(s\right)+b_{v1}^i{v}_1^i\left(s\right)+b_{v2}^i{v}_2^i\left(s\right)\right)ds∥.$$

(71)

Using the linearity of expectations and the subadditivity of the norm, we estimate each term as the maximum constant value:

$$\mathbb{E}∥x^i\left(r\right)∥\le C_{xt}+\mathbb{E}{\int }_t^r\left(\parallel b^i\parallel +\parallel b_{u1}^i\parallel \parallel u_1^i\left(s\right)\parallel +\parallel b_{u2}^i\parallel \parallel u_2^i\left(s\right)\parallel +\parallel b_{v1}^i\parallel \parallel v_1^i\left(s\right)\parallel +\parallel b_{v2}^i\parallel \parallel v_2^i\left(s\right)\parallel \right)ds.$$

(72)

In the above estimate, we introduce a constant $C_{\pi }$, which represents the upper bound on the control inputs:

$$C_{\pi }=max\left\{\parallel b_{u1}^i\parallel ,\parallel b_{u2}^i\parallel ,\parallel b_{v1}^i\parallel ,\parallel b_{v2}^i\parallel \right\}.$$

And we let $\pi =max\{\parallel u_1^i\parallel ,\parallel u_2^i\parallel ,\parallel v_1^i\parallel ,\parallel v_2^i\parallel \}$. We also assume that $\parallel b^i\parallel <A\parallel x^i\parallel$, where A is a positive definite matrix, and thus we obtain

$$\mathbb{E}∥x^i\left(r\right)∥\le C_{xt}+\mathbb{E}{\int }_t^r\left(A\parallel x^i\parallel +4C_{\pi }\pi \right)ds.$$

(73)

By applying Gronwall’s inequality, we obtain the upper bound:

$$\begin{array}{cc}\hfill \mathbb{E}∥x^i\left(r\right)∥& \le C_{xt}+4(T-t)C_{\pi }\pi +\mathbb{E}{\int }_t^r\left(C_{xt}+4C_{\pi }\pi s\right)\parallel A\parallel e^{T\parallel A\parallel }ds.\hfill \\ \hfill & \le C_{xt}+4T{C}_{\pi }\pi +T\left(C_{xt}+4C_{\pi }\pi T\right)\parallel A\parallel e^{T\parallel A\parallel }.\hfill \end{array}$$

(74)

Finally, we can conclude that there exists a constant C, such that for all $r\in [t,T]$, the following holds:

$$\mathbb{E}∥x^i\left(r\right)∥\le C.$$

□

Theorem 3. 

Consider a scenario where all players adopt optimal distributions. Although individual distributions may deviate from the optimal distribution, as the number of players increases, the difference between individual states and cost functions becomes smaller and approaches zero. Specifically, we have

$$\underset{n\to \infty }{lim}\mathbb{E}{∥{\displaystyle \frac{1}{n}}\sum _{i=1}^n{x}^i\left(s\right)-\overline{x}\left(s\right)∥}^2\le C_a$$

(75)

$$\mathbb{E}∥J^{Pi}-{\overline{J}}^P∥\le C_J$$

(76)

where $C_J$ and $C_a$ are constants that do not depend on n.

Proof. 

We begin by considering the mean state equation:

$$d\overline{x}=\left(b^i+b_{u1}{\overline{u}}_1+b_{u2}{\overline{u}}_2+b_{v1}{\overline{v}}_1+b_{v2}{\overline{v}}_2\right)ds+\overline{\sigma }dW$$

(77)

Next, we need to prove the following inequality for the state difference:

$$\begin{array}{cc}\hfill ∥x^i\left(s\right)-\overline{x}\left(s\right)∥& \le e^{At}\left(x_0^i-{\overline{x}}_0\right)\hfill \\ \hfill & +{\int }_t^T{e}^{A(T-s)}\left(b_{u1}\left(u_1^i-{\overline{u}}_1\right)+b_{u2}\left(u_2^i-{\overline{u}}_2\right)+b_{v1}\left(v_1^i-{\overline{v}}_1\right)+b_{v2}\left(v_2^i-{\overline{v}}_2\right)\right)ds\hfill \end{array}$$

(78)

Using the inequality,

$${∥\sum _{i=1}^n{a}_i∥}^2=n\sum _{i=1}^n{∥a_i∥}^2$$

(79)

Now, we can expand this into several terms for each input:

$$\mathbb{E}{∥x^i\left(s\right)-\overline{x}\left(s\right)∥}^2\le 5\mathbb{E}{∥e^{At}(x_0^i-{\overline{x}}_0)∥}^2$$

$$+5\mathbb{E}{∥{\int }_t^T{e}^{A(T-s)}{b}_{u1}(u_1^i-{\overline{u}}_1)ds∥}^2$$

$$+5\mathbb{E}{∥{\int }_t^T{e}^{A(T-s)}{b}_{u2}(u_2^i-{\overline{u}}_2)ds∥}^2$$

$$+5\mathbb{E}{∥{\int }_t^T{e}^{A(T-s)}{b}_{v1}(v_1^i-{\overline{v}}_1)ds∥}^2$$

$$+5\mathbb{E}{∥{\int }_t^T{e}^{A(T-s)}{b}_{v2}(v_2^i-{\overline{v}}_2)ds∥}^2$$

Using the Cauchy–Schwarz and Jensen inequalities, we can now bound each term:

$$\mathbb{E}{∥x^i\left(s\right)-\overline{x}\left(s\right)∥}^2\le 5{∥e^{At}∥}^2\mathbb{E}{∥(x_0^i-{\overline{x}}_0)∥}^2$$

$$+5{\int }_t^T{∥e^{A(T-s)}∥}^2\mathbb{E}{∥b_{u1}(u_1^i-{\overline{u}}_1)∥}^{2}ds$$

$$+5{\int }_t^T{∥e^{A(T-s)}∥}^2\mathbb{E}{∥b_{u2}(u_2^i-{\overline{u}}_2)∥}^{2}ds$$

$$+5{\int }_t^T{∥e^{A(T-s)}∥}^2\mathbb{E}{∥b_{v1}(v_1^i-{\overline{v}}_1)∥}^{2}ds$$

$$+5{\int }_t^T{∥e^{A(T-s)}∥}^2\mathbb{E}{∥b_{v2}(v_2^i-{\overline{v}}_2)∥}^{2}ds$$

Next, we calculate the convergence of each term. For the first term, due to the boundedness of the state, we can assume

$$\mathbb{E}{∥(x_0^i-{\overline{x}}_0)∥}^2\le C_x$$

where $C_x$ is a bound derived from the above formulas. The remaining terms are related to the control inputs. We assume that there exists an upper bound for the control inputs. Therefore, we can conclude

$$\mathbb{E}{∥x^i\left(s\right)-\overline{x}\left(s\right)∥}^2\le C\left(\mathbb{O}\right)$$

As $N\to \infty$, we can obtain

$$\underset{N\to \infty }{lim}\mathbb{E}{∥{\displaystyle \frac{1}{N}}\sum _{i=1}^N{x}^i\left(s\right)-\overline{x}\left(s\right)∥}^2\le {\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathbb{E}{∥{\displaystyle \frac{x^i\left(s\right)-\overline{x}\left(s\right)}{\sqrt{N}}}∥}^2\le C_a$$

$$\begin{array}{cc}\hfill J_{Pi}(x^i,s;{\mu }^P,{\mu }^E)& =\underset{u_i\in {\pi }_u}{inf}\mathbb{E}[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}{g}_{Pi}(x\left(T\right),{\mu }^P\left(T\right),{\mu }^E\left(T\right))\hfill \\ \hfill & +{\int }_t^T{e}^{-{\int }_t^{s}c\left(x\left(r\right)\right)dr}{h}_{Pi}(x,s,{\mu }^P\left(s\right),{\mu }^E\left(s\right))ds]\hfill \end{array}$$

(80)

Next, we prove the second inequality. By using the Lipschitz assumption, we can compute

$$\begin{array}{cc}\hfill \mathbb{E}\left|J^{Pi}-{\overline{J}}^P\right|& \le \mathbb{E}\left[{\int }_t^T{e}^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}\left|h_{Pi}(s,x^i,X^i,u_1^i,u_2^i,{\mu }^P\left(s\right),{\mu }^E\left(s\right))-h_{Pi}(s,\overline{x},\overline{X},{\overline{u}}_1,{\overline{u}}_2,{\mu }^P\left(s\right),{\mu }^E\left(s\right))\right|ds\right]\hfill \\ & +\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}\left|g_{Pi}(x^i\left(T\right),{\mu }^P\left(T\right),{\mu }^E\left(T\right))-g_{Pi}(\overline{x}\left(T\right),{\mu }^P\left(T\right),{\mu }^E\left(T\right))\right|\right]\hfill \end{array}$$

$$\begin{array}{cc}\hfill \mathbb{E}\left|J^{Pi}-{\overline{J}}^P\right|& \le \mathbb{E}\left[{\int }_t^T{e}^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}\left|C_x(x^i-\overline{x})+C_X(X^i-\overline{X})+C_{u1}(u_1^i-{\overline{u}}_1)+C_{u2}(u_2^i-{\overline{u}}_2)\right.\right.\hfill \\ & +\left.\left.C_W{d}_W({\mu }^P,{\mu }^E)\right|ds\right]\hfill \\ & +\mathbb{E}\left[e^{-{\int }_t^{T}c\left(x\left(r\right)\right)dr}\left|C_{xT}(x^i\left(T\right)-\overline{x}\left(T\right))+C_{WT}{d}_W({\mu }^P\left(T\right)-{\mu }^E\left(T\right))\right|\right]\hfill \end{array}$$

For the state and control input bounds, the choice of upper bounds is the same as for the state bounds. Next, we need to establish the upper bound for the distribution. Considering the distribution differences between the pursuer group ${\mu }^P\left(t\right)$ and the evaders group ${\mu }^E\left(t\right)$, we can use the Wasserstein distance to measure the difference between the two groups [35]. Specifically, we have

$$\mathbb{E}\left[d_W({\mu }^P\left(t\right),{\mu }^E\left(t\right))\right]={\left(\underset{\gamma \in \Gamma ({\mu }^P,{\mu }^E)}{inf}{\int }_{{\mathbb{R}}^n\times {\mathbb{R}}^n}{\parallel X_{Pi}-X_{Ei}\parallel }^{2}d\gamma (X_{Pi},X_{Ei})\right)}^{1/2}\le C_l$$

where $X_{Pi}$ and $X_{Ei}$ represent the sample points from the pursuer group ${\mu }^P\left(t\right)$ and the evader group ${\mu }^E\left(t\right)$. $\gamma \in \mathrm{\Gamma }({\mu }^P,{\mu }^E)$ is the set of joint distributions that have ${\mu }^P\left(t\right)$ and ${\mu }^E\left(t\right)$ as marginal distributions. $\parallel X_{Pi}-X_{Ei}{\parallel }^2$ is the distance metric (usually Euclidean) between the sample points of the pursuer and evaders. This formula represents the expected Wasserstein distance, which measures the distributional difference between the two groups. By minimizing all possible joint distributions $\gamma$, we obtain the minimal distance metric. Therefore, for the mean, the difference in cost functions can be expressed as

$$\mathbb{E}\left|J^{Pi}-{\overline{J}}^P\right|\le C_J$$

Similarly, the $ϵ$ Nash equilibrium for the evaders is consistent with the Nash equilibrium for a finite number of individuals. The proof follows a similar approach and is omitted. □

6. Numerical Analysis

In this study, we assume that there are differences in the flight dynamics between the pursuers and the evaders. Specifically, the pursuers typically have higher speeds and a greater normal acceleration, which enables them to more quickly approach the target and perform the interception task. In contrast, the evaders have lower speeds and a smaller normal acceleration, which results in weaker maneuverability, thus affecting their evasion ability.

The differences in flight dynamics between the pursuers and evaders directly influence the success rate of the interception task. Under otherwise similar conditions, the higher speed and greater maneuverability of the pursuers allow them to track the evaders more effectively and reduce the interception time. On the other hand, the lower maneuverability of the evaders makes them more likely to be caught, unless specific strategies are employed to increase the probability of successful evasion. We will further analyze the specific impact of differences in flight dynamics on interception strategies and explore the effects of optimization of these differences on system performance in the subsequent sections.

In this study, the control strategies for both the pursuers and evaders are calculated using the optimal state feedback strategy. Despite the differences in flight dynamics, the strategies for both parties are designed to maximize their individual performance. However, the differences in flight dynamics lead to adjustments in the strategies, so that the pursuers can approach the target more quickly, while the evaders must employ strategies to delay capture, despite their limited maneuverability.

Example 1. 

The initial conditions for this simulation are as follows:

• Initial positions: The initial positions of both the pursuer and the evaders follow a normal distribution. The mean position of the pursuer is $[015,0005000]$, with the covariance matrix:

  $$\left[\begin{array}{ccc}10,000,000& 1& 2\\ 1& 10,000,000& 3\\ 2& 3& 10,000,000\end{array}\right]$$
  The mean position of the evaders are $[100,00014,0000]$, with the covariance matrix:

  $$\left[\begin{array}{ccc}10,000,000& 1& 2\\ 1& 10,000,000& 3\\ 2& 3& 10,000,000\end{array}\right]$$
• Initial velocities: The pursuers’ velocities are set to 4000 m/s, respectively, while the evaders’ velocity is 3500 m/s.
• Acceleration limits: The maximum normal accelerations are 30 g for the pursuers and 20 g for the evaders (g = 9.81 m/s²).
• Flight path angles: The initial flight path angles for both the pursuers are set to $[0^{\circ },0^{\circ }]$ (elevation and azimuth). The initial flight path angles for both the evaders are set to $[0^{\circ },{180}^{\circ }]$ (elevation and azimuth). The covariance matrix for these angles is given by:

  $$\left[\begin{array}{cc}10& 1\\ 1& 10\end{array}\right]$$
• Time step: A time interval of 0.1 s was used for numerical integration.

In the first experiment, five pursuers and five evaders each employed the optimal strategy during the simulation. The simulation environment is summarized in

Table 2. Experiment environment.

Item	Environment
Development language	Python
Library	Numpy
Disk capacity	2 T
RAM	32 G
CPU	i7 2.2 GHZ
OS	Ubuntu 16.04

. During the simulation, the Runge–-Kutta method was applied in each iteration.

The optimal trajectory is depicted in Figure 2, which illustrates the efficiency and effectiveness of the optimal feedback strategy. The strategy ensured that both the pursuers and the evaders followed paths that minimized or maximized their respective objectives, demonstrating the capability of the algorithm to handle dynamic environments.

To further analyze the performance, two-dimensional projections in the X–Y and X–Z planes are shown in Figure 3 and Figure 4, respectively. These projections offer a clearer view of the trajectories in different planes, revealing how the optimal state feedback strategy successfully adapted to the spatial constraints. The figures demonstrate the feasibility of the optimal state feedback strategy algorithm proposed in this paper, highlighting its ability to maintain precise control over the motion of the pursuers and evaders, even under varying conditions.

These results show that the strategy not only guided the system toward the desired outcomes, but also did so in a manner that was robust to changes in the initial conditions and perturbations. The efficiency of the strategy is reflected in the smooth and predictable nature of the trajectories, making it an effective solution for real-time applications in pursuit–evasion problems.

The 3D plot illustrates the overall trajectory of the pursuers and evaders. The paths taken by both entities followed smooth, well-defined curves, which highlights the efficacy of the optimal feedback strategy in controlling their movement across three-dimensional space. The optimal feedback strategy is clearly visible, as both the pursuers and the evaders followed well-defined paths. This demonstrates the robustness of the strategy in guiding the system to its desired outcome.

The XY projection highlights the movement of both pursuers and evaders along the horizontal plane. It visually demonstrates how the optimal strategy governed their trajectories, ensuring effective pursuit and evasion on the two-dimensional surface. This projection shows the horizontal movement of the pursuers and evaders, illustrating the optimality of the feedback strategy in maintaining a controlled pursuit−evasion scenario.

In the XZ projection, we can observe the vertical motion of both the pursuers and the evaders. This visualization confirms the optimal feedback strategy’s ability to manage the 3D movement dynamics and maintain the desired behavior in the vertical direction. This view highlights the vertical movement of both the pursuers and evaders, further validating the strategy’s ability to effectively handle 3D motion dynamics.

The acceleration variations of the pursuers are shown in Figure 5 and Figure 6, while the acceleration variations of the evaders are shown in Figure 7 and Figure 8. Since the simulation model was based on a missile interception scenario, we introduced a constraint on the normal acceleration to better reflect real-world conditions. The sign of the normal acceleration only indicates direction, with the acceleration starting from zero and eventually returning to zero. This represents the convergence of the line-of-sight angle, achieving a head-on interception. Here, the normal acceleration refers to the acceleration in the velocity frame, where the acceleration directions of the pursuers and evaders are opposite. However, due to the nature of the game theory problem, the optimal strategies of both parties evolve simultaneously.

The distance between the evaders and the pursuers is shown in Figure 9, where the final miss distances were $0.53\mathrm{m}$, $0.83\mathrm{m}$, $0.62\mathrm{m}$, $0.94\mathrm{m}$, and $0.78\mathrm{m}$. In missile interception problems, an interception is typically considered successful when the distance is less than one meter. In this paper, interception is defined by the condition $\dot{R}>0$ or $R<1$, where R represents the distance between the pursuers and the evaders. The evaders is captured by the pursuers between $30\mathrm{s}$ and $37\mathrm{s}$. Generally, the terminal guidance phase in missile interception lasts between 20 and 40 s, and the results from Experiment 1 fell within this range. The convergence of the final distance and acceleration led to the convergence of the system state. Given that the initial state was bounded and ultimately converged, this demonstrates the bounded nature of the system state. The successful interception of five evaders by the pursuers proves the feasibility of the proposed approach.

Example 2. 

To verify the feasibility of the algorithm, we increased the number of pursuers and evaders. In Experiment 2, we used 10 pursuers and 10 evaders. Figure 10 shows a 3D interception diagram for the 10VS10 scenario. From this, it can be observed that the double-population game strategy based on MFG is effective in practical situations. This experiment demonstrated the feasibility of the proposed strategy in addressing the evasion problem of nearby populations.

Figure 11 shows the projection in the vertical direction, clearly presenting the positional relationships between the groups. Figure 12 shows the projection in the horizontal direction, further enhancing the understanding of the interactions between the two groups.

To further analyze the movement patterns of the pursuers and evaders, we present the acceleration changes of both the pursuers and evaders along the y-axis and z-axis. Figure 13 shows the acceleration changes of the pursuers along the y-axis, illustrating how the pursuers adjusted their vertical acceleration to respond to the movement of the evaders. Figure 14 displays the acceleration changes of the pursuers along the z-axis, providing further insight into their vertical adjustments.

Similarly, Figure 15 shows the acceleration changes of the evaders along the y-axis, revealing how the evaders modified their vertical acceleration to increase the likelihood of a successful escape. Figure 16 demonstrates the acceleration changes of the evaders along the z-axis, helping us understand how they adjusted their movement strategy to avoid capture by the pursuers. These figures provide important information regarding the acceleration changes of both the pursuers and evaders in three-dimensional space, serving as a basis for subsequent behavioral analysis and strategy optimization.

Figure 17 shows the variation in distance between 10 pursuers and 10 evaders, where the final miss distances were $0.78\mathrm{m}$, $0.92\mathrm{m}$, $0.94\mathrm{m}$, $0.65\mathrm{m}$, $0.56\mathrm{m}$, $0.79\mathrm{m}$, $0.84\mathrm{m}$, $0.62\mathrm{m}$, $0.98\mathrm{m}$, and $0.73\mathrm{m}$. As the experiment progressed, the relative positions of the pursuers and evaders continuously changed in three-dimensional space, resulting in fluctuations in the distance between them. By observing this figure, we can clearly see how the pursuers gradually shortened the distance to the evaders, while at certain moments, the evaders successfully increased the distance through effective evasion strategies. These data provided important insights for further analyzing the behavior patterns of chasing and evading, helping to optimize the dynamic interaction strategies between pursuers and evaders.

Based on the many-to-many strategy presented in this article, the experimental results demonstrate that the pursuers successfully captured the evaders, with the system’s state converging to zero and the mean value of the state being finite. At the termination time, the values of the value functions equaled zero. The proposed many-to-one and many-to-many pursuit algorithms effectively enabled the interception of multiple targets.

To test the robustness of the model, we conducted 100 independent experiments with varying initial conditions. The initial positions of the pursuers and evaders were randomly selected, with the constraint that the initial distance between them was always greater than 100,000 m. The initial angles were chosen within a predefined permissible range, to ensure consistency across all trials. The experimental parameters and results are presented in

Table 3. Experimental setup and parameters.

Parameter	Value
Initial distance	100,000
Initial number of evaders	50
Initial number of pursuers	50
Evader initial elevation angle	$[-{20}^{\circ },{20}^{\circ }]$ (elevation), $[{160}^{\circ },{200}^{\circ }]$ (azimuth)
pursuer initial azimuth angle	$[-{20}^{\circ },{20}^{\circ }]$ (elevation), $[-{20}^{\circ },{20}^{\circ }]$ (azimuth)
Iteration time	0.1 s–0.5 s
Maximum normal acceleration	20–40 g
Number of experiments	100 experiments
Exponential distribution	98 successful, 2 failed
Chi-square distribution	95 successful, 5 failed
Weibull distribution	97 successful, 3 failed
Normal distribution	99 successful, 1 failed

.

Numerical Stability: To further evaluate the stability of the solution, we varied key system parameters, including control input constraints, system dynamics, and the distribution functions of both the pursuers and evaders, using different distributions—such as exponential, chi-square, Weibull, and normal distributions—for comparison. The results showed that, despite variations in these parameters, the system consistently performed well, with the pursuers successfully intercepting the evaders in most trials. This indicates that the solution to the FBSDEs remained stable and robust, even when small perturbations were introduced into the system parameters.

To provide a more comprehensive view of the advantages and limitations of the MFG approach, we will include a detailed comparison with traditional methods in this paper. Specifically, we compare the performance of the MFG approach and traditional methods in solving the same missile interception problem, particularly in the case of a limited number of targets. Traditional methods typically use proportional guidance laws to intercept targets and assume a smaller number of targets, calculating the behavior of each target individually. In contrast, the MFG approach handles the behavior of large-scale target populations using the mean field assumption. This comparison will help us better demonstrate the advantages of the MFG approach in various scenarios, as well as its potential limitations when the number of targets is limited.

Traditional methods use proportional guidance laws to intercept targets. In comparison, the MFG approach proposed in this paper was tested with 100 targets in 100 experiments, and the results were compared in terms of average miss distance and interception success rate. The experimental results are shown in

Table 4. Experimental setup and results.

Metric	Proportional Guidance Law	Mean Field Game
Initial Number of Evaders	100	100
Initial Number of Pursuers	100	100
Number of Experiments	100	100
Miss Distance	0.93 m	0.82 m
Interception Success Rate	86 successful	97 successful

.

The primary reason for the failure of traditional methods is that, as the number of targets increases, there is mutual interference between the individuals, making the system dynamics complex and difficult to control. Additionally, the proximity of target groups can significantly affect the interception results, as interactions between targets may lead to misjudgments or deviations in the pursuers’ actions, thereby reducing the success rate of interception. In this case, the density and relative positions of the target group become key factors influencing interception performance.

To overcome these issues, this paper adopts the Mean Field Game (MFG) strategy, which effectively simplifies the behavior model of large-scale target populations using the mean field assumption. Through this approach, we are able to reduce interference between individuals, particularly mitigating the negative impact of groups in proximity on the interception results. The experimental results show that the MFG strategy significantly improved the interception success rate, while reducing the miss distance, demonstrating the effectiveness and advantages of this method in complex environments.

7. Conclusions

This paper used the Mean Field Game approach to deeply explore an optimal control strategy for solving a multi-missile interception problem in three-dimensional space. The study showed that the mean field game model not only effectively overcame the computational explosion problems caused by high dimensions and large-scale systems in traditional methods, but also provided an optimized solution suitable for large-scale, multi-target environments. By introducing the Hamilton–Jacobi–Bellman equation and proving the uniqueness of its solution, this paper provides strong theoretical support for solving complex multi-agent system control problems.

Firstly, this paper derived the motion models of the pursuers and evaders, and described the game behaviors in the multi-missile interception problem through control constraints and cost function design. Using the framework of mean field games, it was shown that the participants could find approximately optimal strategies based on the given population statistics, without being aware of the state of each individual. Furthermore, by leveraging the concept of $ϵ$-Nash equilibrium, this paper further demonstrated that, within the tolerance range, the participants’ strategies could still minimize costs, ensuring the stability and feasibility of the system.

Finally, the research findings in this paper have broad application prospects, especially in high-risk multi-agent control problems such as missile defense systems. Through the effective application of mean field games, the study not only provides a theoretical basis for the design of practical missile interception systems, but also offers scalable solutions for similar multi-agent systems. The successful application of this theoretical framework demonstrates the immense potential of mean field games in complex multi-target environments. Future research could further explore more efficient solution methods and optimization strategies based on this foundation.