Next Article in Journal
The Stability of the Aggregate Loss Distribution
Next Article in Special Issue
CoRisk: Credit Risk Contagion with Correlation Network Models
Previous Article in Journal
The Determinants of CDS Spreads in Multiple Industry Sectors: A Comparison between the US and Europe

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

# Mean Field Game with Delay: A Toy Model

Department of Statistics & Applied Probability, University of California, Santa Barbara, CA 93106-3110, USA
*
Author to whom correspondence should be addressed.
Work supported by NSF grants DMS-1409434 and DMS-1814091.
Risks 2018, 6(3), 90; https://doi.org/10.3390/risks6030090
Submission received: 12 July 2018 / Revised: 17 August 2018 / Accepted: 20 August 2018 / Published: 1 September 2018
(This article belongs to the Special Issue Systemic Risk in Finance and Insurance)

## Abstract

:
We study a toy model of linear-quadratic mean field game with delay. We “lift” the delayed dynamic into an infinite dimensional space, and recast the mean field game system which is made of a forward Kolmogorov equation and a backward Hamilton-Jacobi-Bellman equation. We identify the corresponding master equation. A solution to this master equation is computed, and we show that it provides an approximation to a Nash equilibrium of the finite player game.
MSC:
91A15; 91G80; 60G99

## 1. Introduction

A linear quadratic stochastic game model of inter-bank borrowing and lending was proposed in (Carmona et al. 2015). In this model, each individual bank tries to minimize its costs by controlling its rate of borrowing or lending to a central bank with no obligation to pay back its loan. The finding is that, in equilibrium, the central bank acts as a clearing house providing liquidity, and hence stability is enhanced. This model was extended in (Carmona et al. 2018), where a delay in the controls was introduced. The financial motivation is that banks are responsible for the past borrowing or lending, and need to make a repayment after a fixed time (the delay). In this model, the dynamics of the log-monetary reserves of the banks are described by stochastic delayed differential equations (SDDE). A closed-loop Nash equilibrium is identified by formulating the original SDDE in an infinite dimensional space formed by the state and the past of the control, and by solving the corresponding infinite dimensional Hamilton-Jacobi-Bellman (HJB) equation. For general stochastic equations and control theory in infinite dimension, we refer to (Bensoussan et al. 2007; Fabbri et al. 2017; Da Prato and Zabczyk 2008).
In this paper, we study the mean field game (MFG) corresponding to the model proposed in (Carmona et al. 2018) as the number of banks goes to infinity. We identify the mean field game system, which is a system of coupled partial differential equations (PDEs). The forward Kolmogorov equation describes the dynamics of the joint law of current state and past control, and the backward HJB equation describes the evolution of the value function. Recently, J.-M. Lasry and P.-L. Lions introduced the concept of “master equation” which contains all the information about the MFG. The well-posedness of this master equation in presence of a common noise and convergence of the N-player system is analyzed in (Cardaliaguet et al. 2015) by a PDE approach. A probabilistic approach is proposed in (Carmona and Delarue 2014; Chassagneux et al. 2014). See also the two-volume book (Carmona and Delarue 2018) for a complete account of this approach.
In this paper, the master equation for our delayed mean field game is derived, a solution is given explicitly, and we show that it is the limit of the closed-loop Nash equilibrium of the N-player game system as $N → ∞$.
The paper is organized as follows. In Section 2, we briefly review the stochastic game model with delay presented in (Carmona et al. 2018). Then, in Section 3, we construct the corresponding mean field game system. In Section 4, we define derivatives with respect to probability measures in the space $P ( H )$ where $H$ is the Hilbert space defined at the beginning of Section 2.2. In addition, we derive the master equation, and exhibit an explicit solution. Furthermore, in Section 5, we show that this solution of the master equation is an approximation of order $1 / N$ to the solution of the finite-player Nash system. Lastly, in Section 6, we compare the solution of the Nash system, the solution of the mean field game system, and the solution to the master equation.

## 2. A Differential Game with Delay

#### 2.1. The Model

Let represents the log-monetary reserves of the N banks at time t. At each time t, bank i controls its rate of borrowing or lending $α t i$, and it also needs to make a repayment after a fixed time $τ$ such that $0 ≤ τ ≤ T$, at a rate denoted by $α t − τ i$. The dynamic of log-monetary reserves for each bank is given by
$d X t i = ( α t i − α t − τ i ) d t + σ d W t i ,$
with deterministic initial conditions
where are independent standard Brownian motions, and banks have the same volatility $σ > 0$.
Bank i interacts with other banks by choosing its own strategy in order to minimize its cost functional $J i ( α i , α − i )$, which involves the average of log-monetary reserves of all the other banks. The notation $α − i$ is a $( N − 1 )$ tuple of the $α j$ with $j ≠ i$ and $j ∈ { 1 , ⋯ , N }$, which represents all other banks’ control except bank i. The cost functional for bank $i ∈ { 1 , ⋯ , N }$ is given by:
where the running and terminal cost functions f and g are:

#### 2.2. Construction of a Nash Equilibrium

In order to apply the dynamic programming principle to identify a closed-loop Nash equilibrium, we have to enlarge the state space by including the path of past controls, which lie in $H : = L 2 ( [ − τ , 0 ] ; R )$, the Hilbert space of square integrable real functions defined on $[ − τ , 0 ]$, and write an infinite dimensional representation for our system. This evolution equation approach was initiated in (Vinter and Kwong 1981) under a deterministic control setting, and later was generalized in (Gozzi and Marinelli 2006) to a stochastic control problem.
Given $z ∈ R × H$, $z 0 ∈ R$, and $z 1 ∈ H$ will denote the two components of the product space $R × H$. The inner product on $R × H$ will be denoted by $〈 · , · 〉$, and it is defined by
$〈 z , z ˜ 〉 = z 0 z ˜ 0 + ∫ − τ 0 z 1 ( s ) z ˜ 1 ( s ) d s .$
Therefore, the new state is denoted by , which corresponds to $( X t i , α t − τ − s i )$ in the notation of the original system (1).
Bank i tries to minimize its cost functional $J i ( α i , α − i )$ defined by
After all other players $j ≠ i$ have chosen their optimal strategies which minimize their cost functionals, player i’s value function $V i ( t , z )$ is defined by
$V i ( t , z ) = inf α i J i ( t , z , α i , α − i ) .$
By dynamic programming principle, the value function $V i ( t , z )$ must satisfy the following infinite dimensional HJB equation (see Fabbri et al. 2017 Chapter 2 for details):
with terminal condition $V i ( T , z ) = c 2 ( z ¯ 0 − z 0 i ) 2$, where the operator $A : D ( A ) ⊂ R × H → R × H$ is defined as
and its domain is .
The adjoint of A is $A * : D ( A * ) ⊂ R × H → R × H$ and is defined by
with domain .
The operator $B : R → R × H$ is defined by
where $δ − τ ( · )$ is the Dirac measure at $− τ$.
The adjoint of B is $B * : R × H → R$ given by
$B * : ( z 0 , z 1 ( s ) ) → z 0 − z 1 ( − τ ) .$
The operator $G : R N → R N × H N$ is defined by
$G : z 0 → ( σ z 0 , 0 ) .$
The infinite dimensional representation of the original system (1) is given by
By minimizing the Hamiltonian in (7), the infimum can be computed, so that the optimal control is attained at
Assuming that each player follows its own optimal strategy $( α ^ i ) 1 ≤ i ≤ N$, which forms a Nash equilibrium, the corresponding value function follows the HJB equation
After applying the definitions of the operators $A , B$ and Q, the HJB equation for player i becomes:
As shown in (Carmona et al. 2018), a solution of the system (11) can be found in the form
$V i ( t , z ) = E 0 ( t ) ( z ¯ 0 − z 0 i ) 2 − 2 ( z ¯ 0 − z 0 i ) ∫ − τ 0 E 1 ( t , − τ − s ) ( z ¯ 1 − z 1 i ) d s + ∫ − τ 0 ∫ − τ 0 E 2 ( t , − τ − s , − τ − r ) ( z ¯ 1 − z 1 i ) ( z ¯ 1 − z 1 i ) d s d r + E 3 ( t ) ,$
for some deterministic functions $E 0 ( t )$, $E 1 ( t , s )$, $E 2 ( t , s , r )$, and $E 3 ( t )$ satisfying the following PDEs
with boundary conditions: $∀ t ∈ [ 0 , T ]$ and $∀ s , r ∈ [ − τ , 0 ]$,
$E 0 ( T ) = c 2 , E 1 ( T , s ) = 0 , E 2 ( T , s , r ) = 0 , E 2 ( t , s , r ) = E 2 ( t , r , s ) , E 1 ( t , − τ ) = − E 0 ( t ) , E 2 ( t , s , − τ ) = − E 1 ( t , s ) , E 3 ( T ) = 0 .$
This set of PDEs (13) with boundary conditions (14) admits a unique solution as shown in (Vinter and Kwong 1981), and the optimal strategies take the integral form

## 3. The Mean Field Game System

The mean field game theory describes the structure of a game with infinite many indistinguishable players. All players are rational, i.e., each player tries to minimize their cost against the mass of other players. This assumption implies that the running cost and terminal cost in (4) only depend on i-th player’s state $z 0 i$ and the empirical distribution of $( z 0 j ) j ≠ i$. Denoting this empirical distribution by
$μ 0 i = 1 N − 1 ∑ j ≠ i δ z 0 j ,$
these costs, as in (4), can be re-written as
As the number N of players goes to , the joint empirical distribution of the states and past controls $Z t j = ( Z 0 , t j , Z 1 , t j )$
$ν t i : = 1 N − 1 ∑ j ≠ i δ ( Z 0 , t j , Z 1 , t j ) ,$
with marginals
$μ 0 , t i = 1 N − 1 ∑ j ≠ i δ Z 0 , t j , μ 1 , t i = 1 N − 1 ∑ j ≠ i δ Z 1 , t j ,$
converges to a deterministic limit denoted by $ν ( t )$ (with marginals denoted by $μ 0 ( t )$ and $μ 1 ( t )$). Here, we assume that, at time 0, $ν 0 i$ satisfies the LLN (for instance with i.i.d. $Z 0 j$), and that the propagation of chaos property holds. A full justification of this property would involve generalizing the result in Section 2.1 of (Carmona and Delarue 2014) to an infinite dimensional setting in order to take into account the past of the controls. This is highly technical but intuitively sound. A complete proof is beyond the scope of this paper.
In the limit, a single representative player tries to minimize his cost functional, and, dropping the index i, his value function is defined as
subject to
$d Z t = ( A Z t + B α t ) d t + G d W t .$
The HJB equation for the value function $V ( t , z )$ reads
with terminal condition $V ( T , z ) = c 2 ( ∫ R y 0 d μ 0 ( y 0 ) − z 0 ) 2 .$ Then, we minimize in $α$ to get
After plugging it into (19), our backward HJB equation reads:
Next, since we “lift” the original non-Markovian optimization problem into a infinite dimensional Markovian control problem, we are able to characterize the corresponding generator for (18), which is denoted by $L t$,
$L t φ ( z ) = 〈 ( A Z + B α ^ t ) , ∂ z φ 〉 + 1 2 T r ( G * G ∂ z z φ ) ,$
where $φ$ is a smooth function and the time dependency comes from $α ^ t$ given by (20). The derivation of the adjoint $L t *$ of $L t$ is given in Appendix A. Consequently, the forward Kolmogorov equation for the distribution $ν ( t )$ reads
Combining (21) with (23), we obtain the mean field game system. To solve this, We make the following ansatz for the value function
$V ( t , z ) = E 0 ( t ) ( m 0 − z 0 ) 2 − 2 ( m 0 − z 0 ) ∫ − τ 0 E 1 ( t , − τ − s ) ( m 1 − z 1 ) d s + ∫ − τ 0 ∫ − τ 0 E 2 ( t , − τ − s , − τ − r ) ( m 1 − z 1 ) ( m 1 − z 1 ) d s d r + E 3 ( t ) .$
where we denote the mean of state $m 0 : = ∫ R z 0 d μ 0 ( z 0 )$, and the mean of past control $m 1 : = ∫ H z 1 d μ 1 ( z 1 )$. Plugging (24) into (23), multiplying both sides of (23) by $z 0$, and integrating over $R × H$, we have
After integration by parts, we obtain
as can be seen directly using (24).
Similarly, plugging (24) to (23), multiplying both sides of (23) by $z 1$, and integrating over $R × H$, we get
By integration by parts, we deduce
Now we are ready to verify the ansatz (24). We first compute the derivative of the ansatz,
Then, we plug the ansatz (24) into (7), and by collecting $( m 0 − z 0 ) 2$ terms, $( m 0 − z 0 ) ( m 1 − z 1 )$ terms, $( m 1 − z 1 ) 2$ terms, and constant terms, we obtain the following system of PDEs:
$d E 0 ( t ) d t − 2 ( E 0 ( t ) + E 1 ( t , 0 ) ) 2 + ϵ 2 = 0 , ∂ E 1 ( t , s ) ∂ t − ∂ E 1 ( t , s ) ∂ s − 2 ( E 0 ( t ) + E 1 ( t , 0 ) ) ( E 1 ( t , s ) + E 2 ( t , s , 0 ) ) = 0 , ∂ E 2 ( t , s , r ) ∂ t − ∂ E 2 ( t , s , r ) s − ∂ E 2 ( t , s , r ) r − 2 ( E 1 ( t , s ) + E 2 ( t , s , 0 ) ) ( E 1 ( t , r ) + E 2 ( t , r , 0 ) ) = 0 , d E 3 ( t ) d t + σ 2 E 0 ( t ) = 0 ,$
with boundary conditions
As for (13)–(14), the system (30)–(31) admits a unique solution.

## 4. The Master Equation

#### 4.1. Derivatives

The master equation for this delayed game lies in an infinite dimensional space, and it requires a notion of derivatives in the space of measures in $P ( H )$.
The set $P ( H )$ of probability measure on $H$ is endowed with Monge-Kantorovich distance
where $L i p ( H )$ is the collection of real-valued Lipschitz functions on $H$ with Lipschitz constant 1.
Definition 1.
We say that $F : P ( H ) → H$ is $C 1$ if there exists an operator $δ F δ ν : P ( H ) × H → H$ such that for any $μ 1$ and $μ 1 ′ ∈ P ( H )$
$lim ϵ → 0 + F ( μ 1 + ϵ ( μ 1 ′ − μ 1 ) ) − F ( μ 1 ) ϵ = ∫ H δ F δ μ 1 ( μ 1 , y 1 ) d ( μ 1 ′ − μ 1 ) ( y 1 ) .$
Definition 2.
If $δ F δ μ 1 ( μ 1 , y 1 )$ is of class $C 1$ with respect to $y 1$, the marginal derivative $D μ 1 F : P ( H ) × H → H$ is defined in the sense of Fréchet derivative:
$D μ 1 F ( μ 1 , y 1 ) : = D y 1 δ F δ μ 1 ( μ 1 , y 1 ) .$
Remark 1.
Usually we will encounter a map $U : P ( H ) → R$. In this case, U can be expressed in a form of composition $U ˜ ∘ F$, where $U ˜ : H → R$, and $F : P ( H ) → H$, i.e., $U = ( U ˜ ∘ F ) ( μ 1 )$.
If $δ F δ μ 1$ is $C 1$ with respect to $y 1$, and $U ˜$ is Fréchet differentiable, then $δ U δ μ 1 : P ( H ) × H → H$, and $D μ 1 U : P ( H ) × H → H$ are defined by
Example 1.
Suppose $U ( μ 1 ) = ∫ − τ 0 ∫ H g ( x 1 ( s ) ) d μ 1 ( x 1 ) d s$, where $g : H → H$ is Fréchet differentiable. Then $U ( μ 1 )$ can be written as $U ˜ [ F ( μ 1 ) ] ( s )$, where $U ˜ [ F ] = ∫ − τ 0 F ( s ) d s$, and $F ( μ 1 ) = ∫ H g ( x 1 ( s ) ) d μ 1 ( x 1 )$. Then
$F ( μ 1 + ϵ ( μ 1 ′ − μ 1 ) ) = ∫ H g ( x 1 ( s ) ) d ( μ 1 + ϵ ( μ 1 ′ − μ 1 ) ) .$
So
$F ( μ 1 + ϵ ( μ 1 ′ − μ 1 ) ) − F ( μ 1 ) ϵ = ∫ H g ( x 1 ( s ) ) d ( μ 1 ′ − μ 1 ) .$
Then
Since $D F U ˜ [ F ] = 1$, we have

#### 4.2. The Master Equation

Theorem 1.
For any $( t 0 , ν 0 ) ∈ [ 0 , T ] × P ( R × H )$, we define
$U ( t 0 , · , ν 0 ) : = V ( t 0 , · ) ,$
where $( V , ν )$ is a classical solution to the system of forward-backward Equations (21) and (23), with initial condition $ν ( t 0 ) = ν 0$, and terminal condition $V ( T , z ) = c 2 ( ∫ R y 0 d μ 0 ( y 0 ) − z 0 ) 2$, respectively. Then U must satisfy the following master equation
where $μ 0$ and $μ 1$ are the marginal law for $Z 0$ and $Z 1$ respectively.
Proof.
For any $h ∈ [ 0 , T − t 0 ]$, $V ( t 0 + h , · ) = U ( t 0 + h , · , ν ( t 0 + h ) )$. Then
On the other hand, V satisfies the HJB Equation (7).
Therefore, subtracting (39) from (38), we have shown that U satisfies the master Equation (37). □

#### 4.3. Explicit Solution of the Master Equation

It turns out that this master Equation (37) can be solved explicitly by making the following ansatz, and we also define $m 0 : = ∫ R y 0 d μ 0 ( y 0 )$ and $m 1 : = ∫ H y 1 d μ 1 ( y 1 )$ for convenience.
$U ( t , z 0 , z 1 , ν ) = E 0 ( t ) ( m 0 − z 0 ) 2 − 2 ( m 0 − z 0 ) ∫ − τ 0 E 1 ( t , − τ − s ) ( m 1 − z 1 ) d s + ∫ − τ 0 ∫ − τ 0 E 2 ( t , − τ − s , − τ − r ) ( m 1 − z 1 ) ( m 1 − z 1 ) d s d r + E 3 ( t ) .$
Then, we compute the partial derivatives needed in (37) explicitly, we have
and plug those into our master Equation (37). We have
Collecting $( m 0 − z 0 ) 2$ terms, $( m 0 − z 0 ) ( m 1 − z 1 )$ terms, $( m 1 − z 1 ) 2$ terms, and constant terms, we obtain that the function $E i , i = 0 , ⋯ , 3$, satisfy the system of PDEs (30) with boundary conditions (31).

## 5. Convergence of the Nash System

From the previous section, we have seen that our master equation is well posed, and we obtained an explicit solution. Furthermore, it also describes the limit of Nash equilibria of the N-player games as $N → ∞$. In this section, generalizing to the case with delay the results of (Cardaliaguet et al. 2015) (see also Kolokoltsov et al. 2014), we show that the solution of the Nash system (11) converges to the solution of the master Equation (37) as number of players $N → + ∞$, with a $1 / N$ Cesaro convergence rate.
In Section 4, we find that (40) is a solution to the master Equation (37). We set $u i ( t , z 0 , z 1 ) : = U ( t , z 0 i , z 1 i , ν i )$, where $ν i = 1 N − 1 ∑ k ≠ i δ ( z 0 k , z 1 k )$, denotes the joint empirical measure of $z 0$ and $z 1$. The empirical measure of $z 0$ is given by $μ 0 i = 1 N − 1 ∑ k ≠ i δ z 0 k$, and the empirical measure of $z 1$ is given by $μ 1 i = 1 N − 1 ∑ k ≠ i δ z 1 k$. Note that, by direct computation, for $k ≠ i$, and any $N ≥ 2$,
$∂ z 0 k u i ( t , z 0 , z 1 ) = 1 N − 1 D μ 0 i U ( t , z 0 i , z 1 i , ν i , z 0 k ) , ∂ z 1 k u i ( t , z 0 , z 1 ) = 1 N − 1 D μ 1 i U ( t , z 0 i , z 1 i , ν i , z 1 k ) , ∂ z 0 k z 0 k u i ( t , z 0 , z 1 ) = 1 N − 1 ∂ z 0 k [ D μ 0 i U ] ( t , z 0 i , z 1 i , ν i , z 0 k ) + 1 ( N − 1 ) 2 D μ 0 i μ 0 i U ( t , z 0 i , z 1 i , ν i , z 0 k , z 0 k ) .$
Proposition 1.
For any $i ∈ { 1 , ⋯ , N }$, $u i ( t , z 0 , z 1 )$ satisfies
where $∥ e i ( t , z ) ∥ < C N$, with terminal condition $u i ( T , z ) = c 2 ( z ¯ 0 − z 0 i ) 2$.
This shows that $( u i ) i ∈ { 1 , ⋯ , N }$ is “almost” a solution to the Nash system (11).
Proof.
We compute each term in the above equation in terms of U using the relationship (42), and we use the fact that U is a solution to the master equation.
• $∑ k = 1 N 1 2 σ 2 ∂ z 0 k z 0 k u i ( t , z 0 , z 1 ) = 1 2 σ 2 ∂ z 0 i z 0 i u i ( t , z 0 , z 1 ) + ∑ k ≠ i 1 2 σ 2 ∂ z 0 k z 0 k u i ( t , z 0 , z 1 ) = 1 2 σ 2 ∂ z 0 i z 0 i U ( t , z 0 i , z 1 i , ν i ) + 1 2 σ 2 ∑ k ≠ i 1 N − 1 ∂ z 0 k [ D μ 0 i U ] ( t , z 0 i , z 1 i , ν i , z 0 k ) + 1 2 σ 2 ∑ k ≠ i 1 ( N − 1 ) 2 D μ 0 i μ 0 i U ( t , z 0 i , z 1 i , ν i , z 0 k , z 0 k ) = 1 2 σ 2 ∂ z 0 i z 0 i U ( t , z 0 i , z 1 i , ν i ) + 1 2 σ 2 ∫ R ∂ y 0 [ D μ 0 i U ] ( t , z 0 i , z 1 i , ν i , y 0 ) d μ 0 i ( y 0 ) + 1 2 σ 2 1 N − 1 ∫ R D μ 0 i μ 0 i U ( t , z 0 i , z 1 i , ν i , y 0 , y 0 ) d μ 0 i ( y 0 ) .$
• From the solution (40) of the master equation, $∂ z U$ is Lipschitz with respect to the measures. Namely,
$| ∂ z 0 U ( t , z k , ν i ) − ∂ z 0 U ( t , z k , ν k ) | ≤ C 1 ( d M K ( μ 0 i , μ 0 k ) + d M K ( μ 1 i , μ 1 k ) ) ≤ C 1 N − 1 , ∥ ∂ z 1 U ( t , z k , ν i ) − ∂ z 1 U ( t , z k , ν k ) ∥ H ≤ C 2 ( d M K ( μ 0 i , μ 0 k ) + d M K ( μ 1 i , μ 1 k ) ) ≤ C 2 N − 1 .$
Thus,
Then,
Theorem 2.
Let $V i$ be the solution to the HJB Equation (11) of the N-player system, where $N ≥ 1$ fixed, and U be the solution to the master Equation (37). Fix any $( t 0 , ν 0 ) ∈ [ 0 , T ] × P ( R × H )$. Then for any $z ∈ R N$, let $ν i = 1 N − 1 ∑ j ≠ i N δ ( z 0 j , z 1 j )$, we have
$1 N ∑ i = 1 N | V i ( t 0 , z ) − U ( t 0 , z i , ν i ) | ≤ C N − 1 .$
Proof.
We first apply Ito’s formula to $( V i ) i ∈ { 1 , ⋯ , N }$, and use the fact that $V i$ satisfies the HJB Equation (11) for the Nash system.
Then, we apply Ito’s formula to $u i ( t , Z t )$, and use the fact that u satisfies (43)
Substracting (46) from (47), taking the square and applying Ito’s formula again, we obtain
Recall that $∂ z 0 k u i ( t , z 0 , z 1 ) = 1 N − 1 D μ 0 i U ( t , z 0 i , z 1 i , ν i , z 0 k )$ is bounded by $C N$ for $k ≠ i$, and $e i$ is bounded by $C N$. Let $( Ξ i ) i ∈ { 1 , ⋯ , N }$ be a family of independent random variable with common law $ν 0$. By integrating (48) from t to T, and taking expectation conditional on Ξ, we have
By the fact that $u T i = V T i$, and using Young’s inequality, we have
Taking average on both sides, we have
By Gronwall’s inequality and taking supremum over $[ 0 , T ]$, we have
which implies
$1 N ∑ i = 1 N | u i ( t 0 , Ξ ) − V i ( t 0 , Ξ ) | ≤ C N .$
Choosing Ξ uniformly distributed in $( R × H ) N$, then by continuity of $u i$ and $V i$, and the fact that $u i ( t , Z )$ is defined by $U ( t , Z 0 i , Z 1 i , ν i )$, we have, for any $z ∈ ( R × H ) N$,
$1 N ∑ i = 1 N | U ( t 0 , z i , ν 0 i ) − V i ( t 0 , z ) | ≤ C N .$

## 6. Conclusions

The mean field game system acts as a characteristic of the master equation. The master equation contains all the information in the mean field game system, and it turns the forward-backward PDE into a single equation. The solution to the mean field game system is a pair $( V , ν )$, that is the value function and the joint law of current state and past law. The solution to the master equation is a function of $( t , z , ν )$.
Since our model is linear quadratic, we are able to solve both the mean field game system and the master equation as shown in Section 3 and Section 4, however, the techniques are not the same. The technique for solving the mean field game is that we first make an ansatz for the solution of the HJB equation. Then plugging this ansatz into the Fokker-Planck Equation (23), we find that the means of state and past control are constant. Hence, the ansatz (24) can be verified. On the other hand, a notion of derivative with respect to measure is needed in order to solve the master equation. Again, we make an ansatz (40), which has a similar form as (24) but is a function of $( t , z , ν )$, and we verify that it satisfies the master equation.
The sets of PDEs (30) with boundary conditions (31) are the same for the two problems. This is due to the fact that our model is linear-quadratic and the means of states and past controls are constants.
Last but not the least, the Nash equilibrium of the corresponding N-player game is presented in Section 2. The value function (12) looks similar to the value function (24) in the mean field game system and the solution (40) to the master equation. As $N → ∞$, the set of PDEs (13) becomes the same as (30). This implies that the solution to the mean filed game appears to be the limit of the Nash system, but generally, the convergence has been known in very few specific situations. Additionally, the solution to the master equation is also a limit to the Nash system, as shown in Section 5.
To summarize, we have extended the notion of master equation in the context of our toy model with delay, and we have shown that, as in the case without delay, this master equation provides an approximation to the corresponding finite-player game with delay. A general form of such a result, not necessarily for linear-quadratic games, is part of our ongoing research.

## Author Contributions

The two authors contributed equally to all aspects of the content of this article.

## Funding

This research was funded by NSF grants DMS-1409434 and DMS-1814091.

## Conflicts of Interest

The authors declare no conflict of interest.

## Abbreviations

The following abbreviations are used in this manuscript:
 SDDE Stochastic Delayed Differential Equation HJB Hamilton-Jacobi-Bellman MFG Mean Field Game PDE Partial Differential Equation LLN Law of Large Numbers NSF National Science Foundation

Let φ be a smooth test function defined on $R × H$. In the following computation, we use the notation
$〈 φ , ν ( t ) 〉 = ∫ R × H φ ( z ) d ν ( t , z ) .$
If the test function φ is of the form $φ ( z ) = ∫ − τ 0 ψ ( z 0 , z 1 ( s ) ) d s$ for a smooth function ψ defined on $R 2$, then
$〈 φ , ν ( t ) 〉 = ∫ − τ 0 ∫ R × R ψ ( z 0 , z 1 ( s ) ) ν ( t , z 0 , z 1 ( s ) ) d z 0 d z 1 ( s ) d s ,$
where $ν ( t , z 0 , z 1 ( s ) )$ is understood as a two-dimensional density. By abuse of notation, we also use
$〈 φ , ν ( t ) 〉 = ∫ R × H φ ( z ) ν ( t , z ) d z = ∫ − τ 0 ∫ R × R ψ ( z ) ν ( t , z ) d z d s .$
Then, we have

## References

1. Bensoussan, Alain, Giuseppe Da Prato, Michel C. Delfour, and Sanjoy Mitter. 2007. Representation and Control of Infinite Dimensional Systems. Basel: Birkhauser. [Google Scholar]
2. Cardaliaguet, Pierre, Francois Delarue, Jean-Michel Lasry, and Pierre-Louis Lions. 2015. The master equation and the convergence problem in mean field games. arXiv, arXiv:1509.02505. [Google Scholar]
3. Carmona, Rene, and Francois Delarue. 2014. The Master Equation for Large Population Equilibriums. arXiv, arXiv:1404.4694. [Google Scholar]
4. Carmona, Rene, and Francois Delarue. 2018. Probabilistic Theory of Mean Field Games with Applications I & II. Berlin/Heidelberg: Springer International Publishing. [Google Scholar]
5. Carmona, Rene, Jean-Pierre Fouque, Seyyed Mostafa Mousavi, and Li-Hsien Sun. 2018. Systemic risk and stochastic games with delay. Journal of Optimization and Applications (JOTA), 1–34. [Google Scholar] [CrossRef]
6. Carmona, Rene, Jean-Pierre Fouque, and Li-Hsien Sun. 2015. Mean field games and systemic risk. Communications in Mathematical Sciences 13: 911–33. [Google Scholar] [CrossRef]
7. Chassagneux, Jean-Francois, Dan Crisan, and Francois Delarue. 2014. A Probabilistic approach to classical solutions of the master equation for large population equilibria. arXiv, arXiv:1411.3009. [Google Scholar]
8. Da Prato, Guiseppe, and Jerzy Zabczyk. 2008. Stochastic Equations in Infinite Dimensions. Cambridge: Cambridge University Press. [Google Scholar]
9. Fabbri, Giorgio, Fausto Gozzi, and Andrzej Swiech. 2017. Stochastic Optimal Control in Infinite Dimension. Berlin/Heidelberg: Springer International Publishing. [Google Scholar]
10. Gozzi, Fausto, and Carlo Marinelli. 2006. Stochastic optimal control of delay equations arising in advertising models. Stochastic PDEs and Applications VII 245: 133–48. [Google Scholar]
11. Kolokoltsov, Vassili, Marianna Troeva, and Wei Yang. 2014. On the rate of convergence for the mean-field approximation of controlled diffusions with large number of players. Dynamic Games and Applications 4: 208–30. [Google Scholar] [CrossRef]
12. Vinter, Richard B., and Raymond H. Kwong. 1981. The infinite time quadratic control problem for linear systems with state and control delays: An evolution equation approach. SIAM Journal on Control and Optimization 19: 139–53. [Google Scholar] [CrossRef]

## Share and Cite

MDPI and ACS Style

Fouque, J.-P.; Zhang, Z. Mean Field Game with Delay: A Toy Model. Risks 2018, 6, 90. https://doi.org/10.3390/risks6030090

AMA Style

Fouque J-P, Zhang Z. Mean Field Game with Delay: A Toy Model. Risks. 2018; 6(3):90. https://doi.org/10.3390/risks6030090

Chicago/Turabian Style

Fouque, Jean-Pierre, and Zhaoyu Zhang. 2018. "Mean Field Game with Delay: A Toy Model" Risks 6, no. 3: 90. https://doi.org/10.3390/risks6030090

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.