The Stability of Two-Community Replicator Dynamics with Discrete Multi-Delays

: This article investigates the stability of evolutionarily stable strategy in replicator dynamics of two-community with multi-delays. In the real environment, players interact simultaneously while the return of their choices may not be observed immediately, which implies one or more time-delays exists. In addition to using the method of classic characteristic equations, we also apply linear matrix inequality (i.e., LMI) to discuss the stability of the mixed evolutionarily stable strategy in replicator dynamics of two-community with multi-delays. We derive a delay-dependent stability and a delay-independent stability sufﬁcient conditions of the evolutionarily stable strategy in the two-community replicator dynamics with two delays, and manage to extend the sufﬁcient condition to n time delays. Lastly, numerical trials of the Hawk–Dove game are given to verify the effectiveness of the theoretical consequences.


Introduction
Evolutionary Game Theory, introduced by [1] to simulate competitions among animals, is one of the latest game theory developments, and aimed at predicting the population dynamics caused by many local interactions between individuals. The evolutionary game is applied to describe the population dynamics in interaction defined by strategy, benefit, and adaptation mechanism, which has found lots of applications in economics, computer sciences, ecologies as well as social sciences [2,3]. Replicator dynamics, which combines evolutionary game theory with differential equations, provides an important theoretical basis to investigate how the spread of advantageous strategies is more probable to happen through imitation than inheritance [3,4].
In classic replicator dynamics, it is supposed that the growth proportion of the strategy adjusts according to the current rate of the strategy in the population. However, in the real environment such as complex social networks, especially financial investments and biology, this assumption is often inaccurate in many situations. Indeed, there is much difficulty in imitating the optimal strategy in a moment in terms of the comparison between the current payoff of one's strategy and its competitor's strategy, for it does not have an immediate effect but appears with some delays. For example, the produced revenues from economic investments cannot be realized immediately until an uncertain time in the future. In biological models, time delays have been used to represent the time of resource regeneration, the growth periods of organisms and the time from infection to onset of disease, etc. [5].
Due to the critical effect of time delays in replicator dynamics, many researchers have investigated the delayed replicator dynamic equations, and have obtained a series of new evolutionary outcomes [6].
A classic method is to study the time-delay replicator dynamics equation by introducing a fixed time delay in the fitness function. In the replicator dynamics system, large values of delays may make the evolutionarily stable strategy(i.e., ESS, see definition 1 for details) lose the stability and persistent periodic oscillations surrounding the solution of the system appear [5]. In the case of discrete time delay, the authors in [7] discussed two models of replicator dynamics with time delay: the mixed ESS of the social-type model would be instability at large delays, but the stability of the ESS would not be unaffected by any value of delay in the biological-type model. The authors in [8,9] studied symmetric delay or asymmetric delay over the strategies in replicator dynamics, and both of their conclusions have shown that time delays make the instability possible. The authors in [10,11] investigated two types of delays in two-community or groups, and proved that the stability of the mixed ESS is affected by strategic delays, but is not affected by the spatial delay. The authors in [12] studied the two-phenotype game model and obtained some associated conclusions. The authors in [5] proposed replicator dynamics in two-community with two-strategy delays. The authors in [13] investigated the stability of mixed ESS in imitation dynamics with discrete multi-delays. In the case of continuous-time delay, the authors in [8] studied the effect of time delays to the ESS in the convergence of evolutionary dynamics. The authors in [5] introduced uniform distribution delay, erlang distribution delay, and exponential distribution delay in the replicator dynamics. The authors in [14] considered bounded continuously distributed time delay in replicator dynamics. Besides, many papers such as [15][16][17] investigated the stability of the Hopf bifurcation and its period solution in replicator dynamics with delays.
Based on the previous research, we have found that replicator dynamics of two-community with multi-delays are rarely studied. Under the efforts of our team, this paper attempts to investigate the replicator dynamics in two-community and manage to extend two delays to n delays. We aim to discuss whether the stability of the fully mixed Nash equilibrium is affected by discrete multi-delays, and obtain some sufficient conditions that can make the equilibrium asymptotical stability in the replicator dynamics.
The paper is arranged as follows. In Section 2, we introduce some basic concepts of evolutionary game theory and compute Nash equilibriums, and then construct the replicator dynamics with delays and obtain some sufficient conditions to make the fully mixed Nash equilibrium asymptotical stable. Then, we will give examples of Hawk-Dove game to verify our results by numerical experiments in Section 3. At last, the conclusions are listed in the paper in Section 4, and all the proofs can be found in the Appendix A.

Establishment and Analysis of the Model
In this section, we take into account the two-community replicator dynamics with asymmetric payoff matrix. We review the replicator dynamics with time delays, and consider the case of time-dependent stability as well as the case of time-independent stability with multi-delays.

Two-Community Replicator Dynamics
For clarity, we focus only on two communities in which interaction is influenced by population size. All results obtained in two communities can be generalized to more than two communities.
We assume that all individuals come from community i (i = 1, 2). The number of community i is denoted as M i . In practical situations, it is difficult for us to calculate the interaction probability between different communities. So we consider the simplest case and suppose the community members live freely in a limited area where the interaction of individuals is random and related to the ratio of population. The interaction rate of the individual from community 1 interacting with the same community members is recorded as φ = M 1 /(M 1 + M 2 ), and with an individual from the other community recorded as 1 − φ. The interaction probability of the individual from community it means strategy S i is better than strategy S j . Similar to the f (S i ) + f (S j ) > 0 and 2 f (S i )/ f (S i ) + f (S j ) < 1, it shows that the benefit of strategy S i is lower than strategy S j . The Equation (5) is used to express the possibility of switching between strategies. We will consider the equation in the differential equations which are used to study the replicator dynamics in the latter part of the paper.
According to the definition above, when one member of community i using strategy E i (i = 1, 2.) plays against an individual of community 1 and community 2, the probability that the E i -strategist switches to P i is expressed by F P i E i . Similar to the F E i P i , their meanings can be acquired easily. In the former research, it is a normal hypothesis that the imitation rate F E i P i depends upon the current expected payoffs f E i (x, y), f P i (x, y). Combining Equation (5), F E i P i (x, y) − F P i E i (x, y) can be represented as follows:

Replicator Dynamics without Delay
The replicator dynamics is assumed that the growth rate of a strategy changes according to the disparity between the expected return of that strategy and the expected payoffs of other strategies in the population [18]. Then the replicator dynamics without delay are obtained below: In addition, the parameters involved in the above formula are defined in Table 1. The parameters L 1 , L 2 , L 12 , L 21 , L 3 , L 4 , L 5 , L 6 and ∆ depend on the payoffs. The parameters R 1 , R 2 , R 3 , R 4 and R * depend on the payoffs and the interaction frequencies. Table 1. This is a table caption. Tables should be placed in the main text near to the first time they are cited.

Parameters
Value Whenẋ = 0 andẏ = 0, we can obtain nine stationary points which are: , 1) and the interior point s * defined in Theorem 1. In replicator dynamics, the stationary points are Nash equilibriums, and the strict Nash equilibrium must have asymptotic stability in the local area [5].

Fully Mixed Nash Equilibrium
An evolutionary stability strategy (i.e., ESS) is a strategy that cannot be invaded by some rare alternative strategies when it is adopted by all individuals in the population [19,20]. In other words, an ESS is an strategy which resists invasion from some rare alternative strategies. Definition 1 ([5]). Assume that the mixed strategy q is adopted by the whole population, and that a fraction of mutants deviates to mixed strategy p. Strategy q is an ESS if ∀ p = q, there exists some p > 0 such that ∀ ∈ (0, p ), the expected fitness J satisfies the following inequality: Namely, ESS implies that the inequality is strictly established, which means that any small mutations(relative to ) of the population profile will be defeated by ESS. In that situation, the equilibrium concept of ESS is a refinement of Nash Equilibrium. Then the former is more robust, because any small deviation of a fraction of population can be defeated by ESS.
We obtain the following result on s * : (i) s * is known as an interior mixed Nash equilibrium [21], i.e., 0 < and ∆ > 0, the interior mixed Nash equilibrium s * is asymptotically stable in the replicator dynamics [21] (i.e., s * is a mixed evolutionarily stable strategy).
Definition 2 ( [13,22]). If any small deviation from initial state is eliminated by the dynamics in the process of approaching a stationary point as t → ∞, the stationary point of the replicator dynamics is said to be asymptotically stable.

Replicator Dynamics with Multi-Delays
Assuming that when an individual makes a choice at time t, he (or she) may obtain his (or her) reward after an uncertain time delay τ. In particular, suppose there are two time-delays (τ 1 = 0, τ 2 = 0) in the replicator dynamic model. An individual of community 1 chooses a strategy and his rival of the same community (or the other community) would take a delay τ 1 with probability p 1 (or q 1 ), a delay τ 2 (τ 2 = 0) with probability p 2 (or q 2 ). Assuming that the information of the two communities is symmetric, we can get the situation of community 2. In addition, we consider the individual of community 1 (or community 2) may take no delay (τ 0 = 0) with probability p 0 (or q 0 ). Obviously p 0 + p 1 + p 2 = 1 and q 0 + q 1 + q 2 = 1. Under these circumstances, we can get the expected payoffs of strategy E 1 and E 2 : Similarly, the expected payoffs of strategy P 1 and P 2 are: The replicator dynamics are given as follows: On the basis of Equations (16) and (17) and the model parameters which are given in Table 1 above, we can calculate the interior mixed Nash equilibrium of replicator dynamics, which is given by: For the situation with (x * , y * ), Equations (16) and (17) can be expressed as follows: To verify the asymptotic stability of the equilibrium point (x * , y * ), suppose that u(t) = x(t) − x * and u(t − τ i ) = x(t − τ i ) − x * , v(t) = y(t) − y * and v(t − τ i ) = y(t − τ i ) − y * . Therefore, one obtains: According to the method in [13] , we perform the Taylor expansion of (20), (21) around (u, v) = 0, then: We can easily obtain the linearized approximation of (22), (23) around z = (u, v) T = 0, which is: where: As a summary of the above discussion, we hypothesize that a strategy used by both sides of the game would take a delay τ i with probability p i ,q i respectively, where i = 0, ..., n, τ 0 = 0, τ i > 0(i = 0) and ∑ n i=0 p i = 1, ∑ n i=0 q i = 1. Similarly, the linearized replicator dynamics can be expressed as the following: Therefore, the above formulas can be expressed as: where: Next, we will discuss the effect of discrete distributed delays on the stability of the replicator dynamics, and derive a delay-dependent asymptotic stability and a delay-independent asymptotic stability sufficient condition for the interior equilibrium. Firstly, we investigate the situation of one or two delays, and then we discuss the multi-delays situation on this basis.

Theorem 2.
In the two-community game model where the strategy has two delays, if: the fully mixed Nash equilibrium is asymptotically stable under the replication equation, where: Proof of Theorem 2. See "Appendix A.2 Proof of Theorem 2".

Corollary 1.
In the two-community game model where the strategy has n delays, if: the fully mixed Nash equilibrium is asymptotically stable under the replication equation.

Theorem 3 ([23-25])
. Let the delay τ ≥ 0. Suppose that z(t) ∈ R 2 is a system state variable, and N 0 , N 1 ∈ R 2×2 are parameter matrices. Then the systemż(t) = N 0 z(t) + N 1 z(t − τ) is stable independently of delay if there is a symmetric positive definite matrix P, such that: Then fully mixed Nash equilibrium is asymptotically stable under the replication equation.

Corollary 2.
Let τ i ≥ 0, (i = 1, 2, · · · , m) be independent, incommensurate delays. Suppose that z(t) ∈ R 2 is a system state variable, and N 0 , N 1 , · · · , N m ∈ R 2×2 are the parameter matrix. Then the systemż(t) = N 0 z(t)+ ∑ m i=1 N i z(t − τ i ) is stable independently of delay if there is a symmetric positive definite matrix P,such that: Then the fully mixed Nash equilibrium is asymptotically stable under the replication equation.
Theorem 4. The conditions in Theorem 3 can be expressed in a more concise form: Proof of Theorem 4. "Appendix A.4 Proof of Theorem 4".

The Two-Community Hawk-Dove Game Model
In this section, we investigate the impact of multiple delays and the asymmetric payoff parameters on the asymptotic stability of the mixed evolutionarily stable strategy in replicator dynamics of two-community by numerical examples. We consider the Hawk-Dove Game [21,28], and apply our theoretical results to two communities which are composed of hawks and doves with different levels of aggressiveness. The interactions within the community 1 (or community 2) are represented by the matrices J 11 (or J 22 ): The interactions between two communities can be described by matrices J 12 and J 21 as follows: The parameters of payoff matrices: V, C, C AW , C WA and α are used to describe the difference of the two communities with different levels aggressiveness. The definition of the parameters are given below [21]: (i) V describes the value of the resource in the survival of hawks and doves. To obtain more survival conditions, they have to compete with each other; (ii) C represents the loss caused by fighting between two hawks from the same community; (iii) C AW is the loss caused by a hawk with more aggressiveness when fighting against a hawk with less aggressiveness; (iv) C WA is the loss caused by a hawk with less aggressiveness when fighting against a hawk with more aggressiveness; (v) α is the resource allocation ratio that takes a more aggressive dove when competing with a less aggressive dove.
Firstly, we draw the evolution diagram of the replicator dynamics system under the first set of parameters in Figure 1.
Secondly, we draw the evolution diagram of the replicator dynamics system under the second set of parameters in Figure 2.
At last, we discuss the information shown in the four pictures below. In Figure 1a, the replication system is obviously asymptotically stable, and parameters meet the requirements of Theorem 2. In Figure 1b, the solutions of the replication system oscillate periodically around the equilibrium point, and parameters which contain large delays do not meet the requirements of Theorem 2. However, when we control the values of p 0 and q 0 in the interval [0.5, 1), the solution of the replication system changes from periodic oscillation to asymptotically stable by observing Figures 1b and 2a. When p 0 ≥ 0 and q 0 ≥ 0, even if we set larger time delays, the system is still asymptotically stable by observing Figure 2b.
Firstly, we draw the evolution diagram of the replicator dynamics system under the first set of parameters in Figure 3.
Secondly, we draw the evolution diagram of the replicator dynamics system under the second set of parameters in Figure 4.
Finally, we discuss the four time delays with reference to the situation of two time delays. By observing the information in the above four pictures, we find that the system is asymptotically stable, even if the number of delays is more than two, as long as the conditions of Theorem 2 or Theorem 5 are satisfied. The contribution of this article can be expressed by comparing two articles. The authors in [5] proposed replicator dynamics in two-community with two-strategy delays, but the author did not consider the case of more than two delays. The authors in [13] investigated the stability of mixed ESS in imitation dynamics with more than two delays, but the author only considered one group.

Conclusions
We applied the method of classic characteristic equations and linear matrix inequality to discuss the stability of the mixed evolutionarily stable strategy in replicator dynamics of two-community with multi-delays. We derived a delay-dependent stability and a delay-independent stability sufficient condition of the evolutionarily stable strategy in the two-community replicator dynamics with two delays, and managed to extend the sufficient condition to n time delays. Finally, we verified the correctness of the conclusions through numerical trials of the Hawk-Dove game. This paper also has some deficiencies. For example, for Theorem 2, we only discussed the sufficiency, but did not give the necessary conditions. Moreover, the model only considered discrete distributed time delays. However, in reality, the time delay may be uncertain and has a continuous distribution. As an extension of this work, we plan to study the stability for replicator dynamics with continuously distributed time delay. system is given by s * = (x * , y * ), with to the case of multiple time delays. For the systemż(t) = N 0 z(t) + N 1 z(t − τ), we suppose there is a symmetric positive definite matrix P and defined as the following Lyapunov function: Along any trajectory of the system, the derivative of V with respect to time is: If there is a symmetric positive definite matrix P, such that: Therefore, Q is a negative definite matrix. According to the stability theory of linear differential equations, it is not difficult to judge that the systemż(t) = N 0 z(t) + N 1 z(t − τ) is stable. The purpose of Theorem 4 is to decompose the positive definite matrix P to reduce the complexity of the calculation in Theorem 3. If the positive definite matrix P can be expressed as W T W, this process is called the square decomposition of P. Then the derivative of V(z(t)) can be expressed as: And for the appropriate dimension matrix A, B, the following matrix inequality holds: Therefore, if N T 0 P + PN 0 + N T 1 PN 1 + P < 0, then according to Lyapunov theorem, the system is asymptotically stable. From Lemma 1 (Schur Complement), we can get then the following conditions are equivalent. Then: If any of them holds, it will makeV(z(t)) ≤ 0.
Appendix A.5. Proof of Theorem 5 According to Theorem 4, systemż(t) = N 0 z(t) + N 1 z(t − τ) is asymptotically stable if there is a symmetric positive definite matrix P, such that N T 0 P + PN 0 + N T 1 PN 1 + P < 0. Suppose that when P = I (Identity matrix), the requirement of matrix inequality is satisfied. Therefore, the condition of asymptotically stable matrix inequality of the system becomes N T 0 + N 0 + N T 1 N 1 + I < 0. Generally speaking N 1 + N T 1 ≤ N T 1 N 1 + I, where "| · |"represents the absolute value of matrix elements. From the previous conclusion, N i < 0(i = 0, 1, . . . m) is obvious. Then N T 0 + N 0 − N T 1 − N 1 < 0 is established. In the case of only one time delay, the sum of probabilities p 0 + p 1 = 1 and q 0 + q 1 = 1. Therefore, to make N T 0 + N 0 − N T 1 − N 1 < 0, only p 0 ≥ 1 2 and q 0 ≥ 1 2 . When there are multiple time delays, the condition of asymptotic stability of the system is p 0 ≥ 1 2 and q 0 ≥ 1 2 . If we want to prove the necessity of the theorem, we only need to verify that N 0 and N 0 + N 1 are stable in asymptotically stable system, which has been proven in reference [27]. The method of proof is similar to the case of multiple time delay.