Relationship between Nash Equilibrium Strategies and H 2 / H ∞ Control of Mean-Field Stochastic Differential Equations with Multiplicative Noise

: The relationship between ﬁnite-horizon mean-ﬁeld stochastic H 2 / H ∞ control and Nash equilibrium strategies is investigated in this technical note. First, the ﬁnite-horizon mean-ﬁeld stochastic bounded real lemma (SBRL) is established, which is key to developing the H ∞ theory. Second, for mean-ﬁeld stochastic differential equations (MF-SDEs) with control-and state-dependent noises, it is revealed that the existence of Nash equilibrium strategies is equivalent to the solvability of generalized differential Riccati equations (GDREs). Furthermore, the existence of Nash equilibrium strategies is equivalent to the solvability of H 2 / H ∞ control for MF-SDEs with control-and state-dependent noises. However, for mean-ﬁeld stochastic systems with disturbance-dependent noises, these two problems are not equivalent. Finally, a sufﬁcient and necessary condition is presented via coupled matrix-valued equations for the ﬁnite-horizon H 2 / H ∞ control of mean-ﬁeld stochastic differential equations with disturbance-dependent


Introduction
Mixed H 2 /H ∞ control is one of the most popular control design approaches.It requires one to search for a controller, which can not only restrain the effect of external disturbance to some extent but also minimize an additional H 2 cost function when the worst-case disturbance is applied.Thus, mixed H 2 /H ∞ is more appealing in engineering practice than H 2 or H ∞ control alone.Deterministic H 2 /H ∞ control and its practical applications have been extensively investigated [1,2].Compared with the convex optimization approach [1], the Nash game approach is a more powerful method to deal with mixed H 2 /H ∞ control.It has been proven that the solvability of mixed H 2 /H ∞ control is equivalent to the existence of Nash equilibrium strategies in the deterministic case [2].The H ∞ controls for linear It ô-type stochastic differential and difference equations were discussed in 1998 [3] and 1999 [4], respectively.Based on SBRL, the authors in [5] studied the finiteand infinite-horizon stochastic mixed H 2 /H ∞ control of continuous-time systems with multiplicative noise.Since then, stochastic H 2 /H ∞ control has become a hot research topic and attracted the attention of many scholars [6][7][8][9][10][11][12][13][14][15][16].In [6], the finite-horizon stochastic H 2 /H ∞ control was investigated for linear discrete time-varying systems with multiplicative noise.The corresponding infinite-horizon stochastic H 2 /H ∞ control, which is more complex, was also discussed.Furthermore, the results of the finite-and infinite-horizon H 2 /H ∞ control were also generalized to discrete-time time-varying Markovian jumps systems with multiplicative noise.As mentioned above, the solvability of mixed H 2 /H ∞ control is equivalent to the existence of Nash equilibrium strategies in the deterministic case [2], which is also true for stochastic systems with state-dependent noise.However, when the disturbance enters into the diffusion term, these two problems are not equivalent [6].In [7], the results of mixed H 2 /H ∞ control and Nash equilibrium strategies were generalized to continuous-time stochastic Markovian jumps systems.Currently, there are still many authors studying mixed H 2 /H ∞ control and achieving much progress.The authors in [8] studied the H 2 /H ∞ state feedback control laws for Markov jump linear systems with state and mode-observation delays.In [9], the stochastic H 2 /H ∞ control problem for Poisson jump diffusion system driven by Brownian motion and Poisson process was investigated.Attention was also focused on the mixed H 2 /H ∞ control of nonlinear stochastic chaotic systems and partially unknown nonlinear stochastic systems [10,11].The mixed H 2 /H ∞ state feedback and the mixed H 2 /H ∞ dynamic output feedback model reference control systems were designed using a parameter-dependent Lyapunov function in [12].In the past few years, mixed H 2 /H ∞ control has been applied to unmanned aerial vehicles [13], the path tracking of autonomous vehicles [14], automated highway driving [15], and interconnected microgrids [16].
In this paper, the relationship between Nash equilibrium strategies and H 2 /H ∞ control of mean-field stochastic systems is considered.The results are the natural generalization of those in [6].In addition to the state x(s), input u(s) and disturbance ν(s), the system dynamics of mean-field stochastic systems also involves E x(s), E u(s) and E ν(s), which is different from the classical stochastic control problems.The mean-field theory has been successfully applied in economics, engineering, finance and games in the past few years.The theory of MF-SDEs can be traced back to [17], which presented a stochastic toy model for Vlasov's kinetic equation of plasma.Since then, many important conclusions have been presented in characterizing MF-SDEs and relevant contents by many scholars [18][19][20][21][22][23].Particularly, as applications, the authors in [22,23] investigated the Markowitz mean-variance portfolio selection and a class of mean-field LQ problems by using the stochastic maximum principle.The reader can see [24][25][26][27][28][29][30][31] for recent developments.In particular, the mean-field linear quadratic (MF-LQ) optimal control problem of MF-SDEs was first investigated in [24], where a sufficient and necessary condition was given to characterize the finite-horizon MF-LQ control problem of continuous-time stochastic systems.Furthermore, the infinite-horizon MF-LQ control problem for linear continuous-time stochastic systems was studied in [25], where the equivalence of several definitions of stability for MF-SDEs was presented.For MF-SDEs, the open-loop and closed-loop solvability of the problem were discussed in [25,26].In particular, an LQ stochastic two-person nonzero-sum differential game was considered in [26].The existence of open-loop Nash equilibria was described by the solvability of forward-backward SDEs, and that of closed-loop Nash equilibria was presented via the solvability of coupled symmetric Riccati differential equations.Corresponding finite-and infinite-horizon MF-LQ controls were studied in [27,28].For discrete-time MF-SDEs, the authors in [29] presented a solvability condition for its finite-horizon LQ control problem in terms of operator-type conditions.In addition, using the matrix dynamical optimization method, the explicit optimal controller was derived.Furthermore, for the corresponding infinite-horizon case, the equivalence of some stabilizability notions for MF-SDEs was investigated, and the solution to the associated algebra Riccati equation was presented in [30].For continuous-time MF-SDEs with state-and disturbance-dependent noise, finite-horizon mixed H 2 /H ∞ control was studied in [31].In the last few years, the mean-field formulation has been applied in various fields of finance, economics, games and engineering.
To the best of our knowledge, there have been no other results on the relationship between Nash equilibrium strategies and mean-field stochastic mixed H 2 /H ∞ control.In this paper, we shall investigate the relationship between Nash equilibrium strategies and mixed H 2 /H ∞ control of mean-field stochastic differential equations with multiplicative noise, and provide a deeper characterization than that in [31].We describe the main ideas as follows.First, we study the pure stochastic H ∞ control problem, and obtain that the effect of external disturbance on the regulated output less than a given positive number is equivalent to the solvability of the generalized differential Riccati equation by It ô's formula and the technique of completing squares.Then, based on It ô's formula and the well-known results on the mean-field stochastic linear quadratic (LQ) optimal control problem, the existence of the Nash equilibrium point is equivalent to the solvability of the generalized differential Riccati equation.Furthermore, for continuous-time mean-field stochastic systems with control-and state-dependent noises, we conclude that the existence of Nash equilibrium strategies is equivalent to the solvability of mixed H 2 /H ∞ control.However, when the diffusion term involves disturbance, the above equivalence relationship does not hold.In this case, we present a sufficient and necessary condition to characterize the finite-horizon H 2 /H ∞ control in terms of coupled matrix-valued equations.Finally, we conclude that the equivalence relationship depends upon whether the diffusion term involves disturbance or not.
The remainder of this paper is organized as follows: In Section 2, we present some preliminaries.A finite-horizon SBRL for MF-SDEs is investigated in Section 3, and a sufficient and necessary condition is shown for MF-SDEs with disturbance-and state-dependent noise.In Section 4, we present some results on the standard and indefinite linear quadratic optimal control problem.Based on Sections 3 and 4, the relationship between mean-field stochastic H 2 /H ∞ control and Nash equilibrium strategies is established in Section 5.It is shown that their equivalence relies on the relationship between the disturbance and the diffusion term.Section 6 proceeds with two examples to verify the correctness of the conclusions.
Notations: Z is the transpose of a vector Z or matrix Z; Z > 0 (Z ≥ 0); Z is a positive definite (positive semi-definite) symmetric matrix; R n is an n-dimensional real Euclidean space; A ∈ S n means that A is an n-dimensional real symmetric matrix; L 2 F ([0, S], R n ) is the space of nonanticipative stochastic processes x(s) ∈ R n with respect to an increasing ; and I is the identity matrix.

Preliminaries
Based on H ∞ and H 2 performance functions, we introduce the definitions of the finite horizon mean-field stochastic H 2 /H ∞ control and the Nash equilibrium point of a twoperson non-zero sum LQ game in this section.Furthermore, the relationship between mixed H 2 /H ∞ control and pure H ∞ or H 2 control also is presented.
Consider the following MF-SDEs with (x, ν)-dependent noises: where and z 1 (s) ∈ R m 3 are the system state, external disturbance, control input and regulated output, respectively.E is the expectation operator.Without loss of generality, we assume B(s) to be one-dimensional standard Brown motion defined on the filtered probability space (Ω, F , {F s } s≥0 , P ) with F s := σ{B(t) : 0 ≤ t ≤ s}, where F is a σ-field consisting of subsets of Ω, and P denotes a probability measure.ζ 0 ∈ R n is the initial state, which is assumed to be deterministic.
To better characterize mixed H 2 /H ∞ control and the Nash equilibrium point, for the given disturbance attenuation level λ > 0, we define two associated performance indices of system (1) as follows: Similar to the definition of mixed H 2 /H ∞ control of classic stochastic systems [5], mean-field stochastic H 2 /H ∞ control can be defined as follows.
Definition 1.Consider system (1); for a given λ > 0, 0 < S < ∞, search for a state feedback control u * S (s) , where L S is an operator associated with the realization which is defined as (2) When the worst case disturbance ν * S (s) where ) only satisfies Definition 1-(1) but does not necessarily satisfy Definitions 1-(2), then such a u * S is called an H ∞ control for system (1).J 1,S is the H ∞ performance function.According to Definition 1, J 1,S > 0 can attenuate the effect of external disturbance ν on regulated output z to the given positive number λ, and J 2,S means the output energy, which is the corresponding H 2 performance function.
) is called the Nash equilibrium point of the two-person non-zero sum LQ game corresponding to the cost functions J 1,S (u, ν) and Remark 2. From the above Definitions 1 and 2, mixed H 2 /H ∞ control has close relation to the Nash equilibrium point.If (u * S , ν * S ) is the Nash equilibrium point, ν * S is the worst-case disturbance; in this case, u * S can not only minimize J 2,S (u, ν * S ) but also guarantee In what follows, our main results are the relationships between the solvability of the mean-field stochastic H 2 /H ∞ control with the existence of Nash equilibrium point (u * S , ν * S ) of (3).In particular, the pure stochastic H ∞ control problem only needs to solve while the pure stochastic H 2 control only needs to solve

Finite Horizon Mean-Field SBRL
Mixed H 2 /H ∞ control is the combination of pure H ∞ control and pure H 2 control.To study the mixed H 2 /H ∞ control problem, we first need to establish the finite time mean-field SBRL, which is the key to developing the pure H ∞ control theory.
Consider the following MF-SDEs with state-and disturbance-dependent noises: where and D 2 (s) are continuous matrixvalued functions of appropriate dimensions.E is the mathematical expectation, and B(s), x(s), ν(s), z 2 (s) are as in (1).Associated with system (4), the perturbation operator L S is defined as follows: and .
Remark 3. Lemma 1 presents a sufficient and necessary condition of the effect of external disturbance ν(s) on regulated output z 2 (s) less than the given positive number λ, which is the key procedure in the characterization of mixed H 2 /H ∞ control (Definition 1).

Finite Horizon Mean-Field LQ Control
Some well-known results on standard and indefinite mean-field stochastic LQ controls are stated in this section, which play a central role in the study of mean-field stochastic H 2 /H ∞ control.We start with the definition of the pseudo matrix inverse.

Definition 3 ([32]
).Given a matrix W ∈ R m×n , the pseudoinverse (Moore-Penrose inverse) of W is the unique matrix W + ∈ R n×m such that In particular, when W ∈ S n , the Moore-Penrose inverse has the following properties: (i) Consider the following stochastic system of the mean-field type: Under the constraint of ( 26), the linear quadratic performance is given as follows: with Q(s) and R(s) being suitable symmetric matrix-valued functions.The finite horizon mean-field stochastic LQ problem can be described as follows.28) is called the optimal control, and the state x * (s) corresponding to u * S (s) is called the optimal trajectory.When Q(s) and R(s) are indefinite matrices, the above is indefinite LQ control, which is associated with the following GDRE:

Problem (MF-LQ):
where For the coefficients of system (26), corresponding assumptions are stated as follows: (A1) For the cost function (27), corresponding weighting coefficients assumptions are stated as follows: and for some δ > 0 and t ∈ [0, S], Q(s) ≥ 0 and R(s 29) is said to be regular if Next, the well-known results on standard and indefinite mean-field stochastic LQ controls are stated as follows.
Lemma 2 ([24]).Let (A1) and (A2) hold, then (30) admits unique solutions ( P * S , Q * S ).The unique optimal control law and the optimal cost value are respectively given by and min Lemma 3 ([26]).Let (A1) -(A2) hold, and there is an optimal linear feedback u * S (s 29) is regularly solvable.In this case, and min where Remark 4. Lemmas 2 and 3 present the conditions under which the optimal linear feedback control exists, as well as the form of the optimal cost value and the optimal feedback control.These two lemmas are the key to developing the H 2 control.

Mean-Field Stochastic H 2 /H ∞ Control and Nash Equilibrium Strategy
We first consider the following MF-SDE with only the x-dependent noise: Taking the mathematical expectations in (33), we have and For system (33), a necessary and sufficient condition for the existence of two-person non-zero sum Nash equilibrium strategies is established, which is an extension of [6].
Theorem 1.For system (33), there exists linear memoryless state feedback Nash equilibrium strategies if and only if the following coupled GDREs . If the solution of ( 38) and (39) exists, then and Proof.Sufficiency: Applying It ô's formula, under the constraint of (34) and (35), we have (with the time argument s suppressed) In consideration of by a series of simple computations and considering (38), it can be obtained that where ν * S (s) and u * S (s) are given by ( 40) and (41).The above equation easily yields that which finishes the proof of (36).Furthermore, when Under the constraint of (42), to minimize J 2,S (u, ν * S ) is a standard mean-field stochastic LQ optimization problem.By Lemma 2 and GDRE (39), we can obtain min So, Nash inequality (37) holds, which ends the proof of sufficiency.
has a unique solution (P * S (s), Q * S (s)).Moreover, is as in (41), and Under the constraint of (47), to minimize J 2,S (u, ν * S ) is a standard mean-field stochastic LQ optimal problem.By Lemma 2, we can obtain that has a unique solution ( P * S (s), Q * S (s)) with Q * S (s) ≥ 0, which is the same as (39).Furthermore, 38) is obtained.The rest is to show Q S (s) ≤ 0, which can be seen from ( 46) and (36) that for any The proof is complete.
Remark 5.By Theorem 1, the existence of Nash equilibrium strategies is equivalent to the solvability of GDREs.Combined with Theorem 3.1 given in [31], we can get the following Theorem 2, which states the relationship between the solvability of mixed H 2 /H ∞ control of system (33) and the existence of Nash equilibrium strategies: Theorem 2. The following are equivalent: (i) There exists a Nash equilibrium strategy pair (u According to Theorem 3.1 in [31], L S [0,S] ≤ λ can be replaced by L S [0,S] < λ.So (i)=⇒(ii).(ii)=⇒(i): Keeping the definition of ν * in mind, by Definition 1, (ii)=⇒(i).To summarize , the proof is complete.Remark 6.According to Theorem 2, the existence of the Nash equilibrium strategy (36) and ( 37) is equivalent to the solvability of H 2 /H ∞ control of system (33) or the existence of solutions to coupled GDREs (38) and (39).There exist the same results for the following MF-SDEs with x-and u-dependent noises: If the solution of GDREs (50) and (51) exists, then Proof.Sufficiency: For u * S = K 0,u (s)x(s) + (K 0,u (s) − K 0,u (s))E x(s), one needs to note that GDRE (56) corresponds to indefinite LQ control problem min By Lemma 3, for ∀ζ 0 ∈ R n , Similarly, GDRE (57) corresponds to standard LQ control problem min For the LQ optimization (63) and (64), by Lemma 2, we have Remark 8.According to Theorem 4, for system (1), the solvability of GDREs (56) and ( 57) is equivalent to the solvability of H 2 /H ∞ control.However, it is not equivalent to the existence of Nash equilibrium strategies, which is stated by the following Theorem 5.

Examples
In this section, a numerical experiment is given to demonstrate the effectiveness of the presented calculation approach of GDREs (38) and (39) in Theorem 2. Here, the obtained GDREs are viewed as a set of backward differential equations with known terminal conditions, which can be solved by the Runge-Kutta method.

Conclusions
In this paper, we discussed the relationship between the solvability of H 2 /H ∞ control and the existence of a Nash equilibrium strategy pair for continuous-time mean-field stochastic systems.For mean-field stochastic systems with disturbance-dependent noises, the SBRL was presented to characterize the necessary and sufficient condition of the solvability of L [0,S] < λ in the form of GDREs.Based on It ô's formula and mean-field stochastic LQ problem, it was revealed that the existence of Nash equilibrium strategies is equivalent to the solvability of GDREs for MF-SDEs with control-and state-dependent noises.It was also shown that their equivalence depends upon whether the MF-SDEs is a stochastic system with disturbance-dependent noises or not.For the finite-horizon H 2 /H ∞ control of more general mean-field stochastic systems with disturbance-dependent noises, based on mean-field SBRL and mean-field LQ control, some sufficient and necessary conditions were also presented in the form of GDREs, which can be solved with the Runge-Kutta algorithm.

Figure 1 .Figure 2 .
Figure 1.The trajectories of P * S and P * S .