Effects of Relatedness on the Evolution of Cooperation in Nonlinear Public Goods Games

Evolution of cooperation has traditionally been studied by assuming that individuals 1 adopt either of two pure strategies, to cooperate or defect. Recent work have considered continuous 2 cooperative investments, turning full cooperation and full defection into two opposing ends of a 3 spectrum and sometimes allowing for the emergence of the traditionally-studied pure strategies 4 through evolutionary diversification. These studies have typically assumed a well-mixed population 5 in which individuals are encountered with equal probability, Here, we allow for the possibility of 6 assortative interactions by assuming that, with specified probabilities, an individual interacts with 7 one or more other individuals of the same strategy. A closely related assumption has previously been 8 made in evolutionary game theory and has been interpreted in terms of relatedness. We systematically 9 study the effect of relatedness and find, among other conclusions, that the scope for evolutionary 10 branching is reduced by either higher average degree of, or higher uncertainty in, relatedness with 11 interaction partners. We also determine how different types of non-linear dependencies of benefits 12 and costs constrain the types of evolutionary outcomes that can occur. While our results overall 13 corroborate the conclusions of earlier studies, that higher relatedness promotes the evolution of 14 cooperation, our investigation gives a comprehensive picture of how relatedness affects the evolution 15 of cooperation with continuous investments. 16


Introduction
Cooperation, and in particular the willingness to cooperate with relatives, is regularly observed in many species.Observed forms of cooperation include egg trading, cooperative foraging, and predator inspection among fishes; defensive coalitions, cooperative hunting, food sharing, and alarm calls among birds; grooming behaviour, alarm signals, coalitions, alloparenting, and cooperative hunting among various mammals; bloodsharing among vampire bats; care-giving behaviour in dolphins; foraging, anti-predator behaviour, and hive thermoregulation in honeybee colonies; and "social contracts" among paper wasps [1].Social insects such as ants and bees also cooperate frequently (e.g., [2,3]) and cooperation is observed even among microbes (e.g.[4]).Finally, human societies are striking examples of cooperation.
Since cooperation often involves a cost to the individual without conveying a commensurate immediate benefit, one would naively suspect that cooperation cannot evolve to any higher degree.This, however, is not the case, and several mechanisms capable of promoting and supporting cooperation has been proposed and explored [5,6].An influential explanation of cooperation between related individuals is kin selection, which states that there can be positive selection for a gene conveying cooperative behavior if the beneficiaries of that behavior are likely to share the same gene.This is typically the case when interactions occurs between relatives, but the principle applies more generally and assortative interaction between individuals can arise for a range of other reasons such as spatial structure or social dynamics.In certain settings, the principle of kin selection can be formulated quantitatively in a form that is now known as Hamilton's rule: a cooperative gene is selected for if and only if rb > c, where r is the average degree of relatedness, b is the average benefit to the recipients of the cooperative act, and c is the average cost to the individual actor [7].
Cooperation has often been studied in the setting of evolutionary game theory based on the prisoner's dilemma or its many-player generalization, the public-goods game (see e.g, [8,9]).Although the latter game in principle allows for continuous investment, it is often assumed that individuals adopt either of two pure strategies corresponding to cooperation and defection.The evolutionary dynamics are studied with the so-called replicator equations in which strategies with a higher payoff than average in the population increase in frequency.To allow for the possibility of assortative interactions in this framework, Grafen [10] introduced the assumption that a fraction of an individual's interaction are with individuals of the same type with the remaining interactions occurring with partners drawn at random from the population.This approach and related approaches has since been used to address a range of evolutionary questions [11][12][13][14][15][16][17].
Recently, several studies have moved beyond the traditional cooperative games by considering continuous cooperative investments as well as non-linearity of benefits and costs [4,[18][19][20][21][22][23].With this setting, additional evolutionary outcomes become possible, including evolutionary branching in the cooperative trait under consideration and the emergence of two or more different coexisting strategies.
In a landmark study, Doebeli et al. [18] considered a setting in which individuals interact in pairs and each make a cooperative investment for their common good.Their joint benefit is given by a non-linear function of their joint investment and their costs are given by a non-linear function of their respective investments.Finally, their payoffs are determined as the difference of the benefit and their respective costs.By assuming that the benefit and cost function were given by quadrative polynomials, they classified all possible evolutionary outcomes.Building on this framework, Cornforth et al. [4] showed how interaction assortment, interpreted as relatedness, can be incorporated.Though developed independently, the idea is similar to Grafen [10].Cornforth et al. investigated how assortment affects the evolution of cooperation for three different benefit functions and under the assumption of proportional costs.
Here, we extend the previous results by Cornforth et al. and systematically study the effects of relatedness on the evolution of cooperation in non-linear public-goods games with continuous investment.In Section 2, we present the general model.We consider both quadratic cost and benefit functions and general, increasing, cost and benefit functions.In Section 3, we analyze the dynamics of quadratic cost and benefit functions, and then derive results for general cost and benefit functions.
Finally, in Section 4, we recapitulate and discuss our main results and identify important challenges for future research.

Demographical dynamics
We consider a model based upon [18] and [4].We assume an infinite population in which the ith individual has trait value x i representing cooperative investment, with 0 ≤ x i ≤ 1.At each generation, N random individuals with respective trait values x 1 , . . ., x N are chosen for an interaction, in which the payoff to the ith individual, with 1 ≤ i ≤ N, is given by where We assume that the population dynamics follows the replicator equation [8], i.e., that successful strategies increase in frequency at a rate proportional to the difference with the average population-level payoff.
We first carry out a comprehensive analysis of quadratic benefit and cost functions x that typically look like in Figure 1.We restrict the range of possibilities by requiring that B(x 1 + . . .+ x N ) and C(x i ) should be increasing, i.e., that b 1 > 0, and c 1 ≥ −2c 2 .The signs of b 2 and c 2 will depend on whether the functions are accelerating or decelerating.As a second step, we derive results for general increasing benefit and cost functions.
Readers not famililar with these methods may want to refer to an introductory text such as Brännström et al. [27].In brief, we consider the initial growth rate of a rare strategy m in an environment dominated by a resident strategy r.This growth rate is called the invasion fitness and written S r (m).The sign of the invasion fitness determines whether the rare strategy can grow in numbers and invade.From the invasion fitness we obtain the selectiong gradient S r (r).Strategies at which selection ceases, S r (r) = 0 are called evolutionary singular.They can be evolutionary attractors, in which case they are called convergence stable.Once established, the subsequent evolutionary dynamics depends on whether the singular strategy is evolutionarily stable or an evolutionary branching point.In the former case, evolution comes to a halt while in the latter case, the population diversifies and two coexisting strategies emerges.
From the replicator equation (see Appendix A.1) we obtain the following expression for the invasion fitness Differentiating with respect to the mutant trait value and evalutating at m = r yields the selection gradient where µ is the average whole-group relatedness, defined as the expected value of fraction of the group that is i.b.d. to the focal individual (including the focal individual itself), i.e.
Given this selection gradient, we obtain the condition for convergence stability of an evolutionarily singular strategy and the condition for evolutionary stability

NB (Nr
where σ 2 = Var [K/N], with K being the random variable with probability density function Pr(k).For a derivation of these conditions, see the appendix of Cornforth et al. [4], as well as Appendix A.2.

Results
We first carry out a comprehensive analysis of the effect of assortment on the evolution of cooperation with quadratic benefit and cost functions and then derive results for general cost and benefit functions.Our key findings are illustrated by Figure 2 and 3.These show the effects of increased average relatedness and increased variance in relatedness on the direction of selection and evolutionary stability of the cooperative investment.Note, in particular, that a singular strategy increases with increased relatedness if and only if it is convergence stable, and that increased relatedness as well as how increased uncertainty in relatedness can change the evolutionary stability of singular strategies.
Finally we corroborate selected results using numerical simulations.

Quadratic benefit and cost functions
Recall that we assume benefit and cost functions, B and C, that are increasing, satisfy B(0) = C(0) = 0, and are given by quadratic polynomials, as shown in Figure 1.For these functions, we completely classify the evolutionary dynamics and its dependence on on the mean and variance of relatedness.Our conclusions are corroborated by numerical investigations (Fig. 4).
Recalling from Sect.2.1 that that we must have b 1 ≥ −2Nb 2 for B to be increasing, we have 2Nb 2 r + b 1 > 0 and it follows as expected that increased relatedness always alters selection pressures in the direction of higher cooperative investments.
From the selection gradient, Eq. 8, we solve for the singular strategies, i.e., the values of r at which the selection gradient vanishes, and find that for each µ, there is at most one interior evolutionarily singular strategy in the allowed range of cooperative investments.It is given by whenever this value lies between 0 and 1.
Momentarily leaving aside the constraints on µ and r * imposed by their biological interpretation, we note that the graph of r * (µ) crosses zero at and a horizontal asymptote at r H = −b 1 /(2Nb 2 ).The graph consists of two curves that are located above and below the horizontal asymptote, respectively.Recalling from Sect.2.1 that we must have b 1 ≥ −2Nb 2 for B to be increasing, we see that the horizontal asymptote is located outside of the range 0 ≤ r < 1.Specifically, for accelerating benefits we have r H > 1 and for decelerating benefits we have It follows that only one of the curves can be biologically relevant and this is the curve that crosses zero at The evolutionary dynamics now depends on whether the horizontal asymptote is located above or below the allowed trait range and on whether the graph of r * crosses zero before or after the vertical asymptote, i.e. on whether µ H > µ 0 .As we have already noted, the answer to the former question depends on whether benefits are accelerating or decelerating.We have not found a similar straightforward interpretation of the latter condition, but we note that µ 0 > µ V whenever benefits and costs are not simultaneously accelerating or simultaneously decelerating.For each of the four possible combinations, we can characterize the evolutionary dynamics.
Theorem 1.Let µ 0 , µ V , and r H be as defined above.Let r * (µ) be the interior strategy given by Eq. 3.1 whenever Otherwise, let r * (µ) be the boundary strategy that results from directional selection.The following conclusions then hold.
• Assume B is accelerating and that µ 0 > µ V , as is the case if C is decelerating.For µ < µ V , we have r * (µ) = 0 and selection is thus towards no cooperation.For µ V < µ < µ 0 , r * (µ) is decreasing and the evolutionary dynamics is bistable.For µ > µ 0 we have r * (µ) = 1 and selection is thus towards full cooperation.
• Assume B is decelerating and µ 0 > µ V , as is the case if C is accelerating.For µ < µ 0 we have r * (µ) = 0 and selection is thus towards no cooperation.For µ > µ V , r * (µ) is increasing and convergence stable, i.e., an evolutionary attractor.
Proof.The conclusions follow from the geometric observations that precede the theorem and the fact, also discussed above, that the selection gradient is increasing function of µ.
We next investigate the effects of relatedness on evolutionary stability.The condition for evolutionary stability, Eq. 7, is  Green thin line represents convergence stability without evolutionary stability, blue thick line represents convergence stability with evolutionary stability, black dotted line represents neither convergence stability nor evolutionary stability, and black thick line represents evolutionary stability without convergence stability.We see that the four cases are qualitatively different.We also see that when B and C are both decelerating, a certain range of relatedness leads to evolutionary branching.The parameters used were in all cases N = 10, and b 2 , b 1 , c 2 , c 1 as in Figure 1.
We thus see that both increased average relatedness (µ) and increased variance in relatedness (σ 2 ) can affect the evolutionary stability of a singular strategy.We consider four cases depending on the signs of b 2 and c 2 .When benefits are accelerating (b 2 > 0) and costs decelerating (c 2 < 0), the singular strategy is never evolutionarily stable.As Theorem 1 shows, any interior singular strategy will lack convergence stability and there will thus not be any evolutionary branching points.Diversification may still be possible, but would have to be established through a large change in strategy.When benefits are decelerating (b 2 < 0) and costs accelerating (c 2 > 0), any interior singular strategy is always evolutionarily stable.If benefits and costs are both accelerating (b 2 > 0, c 2 > 0), either is possible and any change with increased relatedness µ or variance σ 2 is a loss of evolutionary stability.
Finally, if benefits and costs are both decelerating (b 2 < 0, c 2 < 0), either is possible and any change with increased relatedness µ or variance σ 2 is a gain of evolutionary stability.
Figure 2 shows typical outcomes for four combinations of accelerating and decelerating costs and benefits.The outcomes are typical and consistent with our conclusions, though it should be noted that Theorem 1 alows for alternative outcomes of directional selection when benefits and costs are both accelerating or both decelerating.Finally, we investigate the potential for evolutionary branching when benefits and costs are both accelerating or both decelerating.Solving Eq. 6 for µ, the boundary between convergence stability and not convergence stability becomes assuming b 2 = 0.If B is decelerating, then µ > µ c implies convergence stability, and if B is accelerating, then µ < µ c implies convergence stability.
Solving Eq. 10 for σ 2 , we get the boundary between evolutionary stability and instability In other words, evolutionary stability will switch as soon as or after convergence stability switches when increasing the average relatedness, depending on whether the variance is at its maximum or not.
There is a region in the b 2 − c 2 -plane where µ e is undefined.This is when b 2 < 0, c 2 > Nb 2 σ 2 ; or When the first one of these is the case, the condition for evolutionary stability will always hold, and hence the strategy will always be evolutionarily stable.Conversely, when the second is the case, the strategy will never be evolutionarily stable.
Using (12), we can plot σ 2 against µ to find out the effects of increased variance in relatedness on the evolutionary stability, as in Figure 3.We see that increased variance has a noticeable effect: The higher the variance, the sooner the switch to evolutionary stability.We also see that increased relatedness can make the evolutionary dynamics bistable.
We have carried out numerical simulations to corroborate our findings results, using the method described in Appendix B. Our analytical investigation predicts evolutionary branching at µ = 0.4 and evolutionary stability at µ = 0.8 for B and C decelerating, and bistability when µ = 0.5 for B and C accelerating.This is consistent with the results from the simulations, see Figure 4.

General cost and benefit functions
Having analyzed the case of quadratic cost and benefit functions, we now turn to the case of general strictly increasing non-linear functions.As our first result, we note that the position of an interior singular strategy, if it exists, depends only on the average relatedness and not on any higher moment of the probability distribution.To see this, we recall that an interior singular strategy r * is by definition a point at which the selection gradient vanishes, µB (Nr * ) − C (r * ) = 0.
To see how increased relatedness affects an interior singular strategy, we differentiate implicitly with respect to µ which gives, .
Figure 3. Plots of variance in relatedness against average relatedness.The grey region represents combinations of mean and variance that are not logically possible (see Appendix A.3).We see that for decreasing cost and benefit, the region with evolutionary branching shrinks with increased variance in relatedness.We also see that for increasing cost and benefit, the region with neither convergence stability nor evolutionary stability shrinks with increased variance in relatedness.In the latter case, with increased average relatedness or variance in relatedness, the singular strategy moves from evolutionary stability without convergence stability to neither evolutionary stability nor convergence stability.The parameters used are in both cases N = 10, and b 2 , b 1 , c 2 , c 1 as in Figure 1 From Eq. 6 we see that the denominator is positive if and only if the intertior singular strategy is convergence stable.Since the benefit function is strictly increasing, we conclude that increased relatedness has opposite effect on interior singular strategies depending on whether they are evolutionary attractors or repellors.As illustrated in Fig. 2, evolutionary attractors increase with relatedness while evolutionary repellors decrease with relatedness.
Next, we restrict attention to benefit and cost functions that are either accelerating or decelerating for the full range of strategies considered, giving a total of four combinations to be explored.For two cases, we give a full classification of the evolutionary dynamics, allowing for the possibility of large mutational steps.
Theorem 2. Assume that benefits are decelerating and costs are accelerating, more specifically that B (x) < 0 and C (x) > 0 for all x ∈ [0, 1].Then there is exactly one continuously stable strategy r * .Furthermore, we The continuously stable strategy is always evolutionarily stable and increases with the degree of relatedness.
In short, the theorem asserts that the evolutionary dynamics will lead to a monomorphic population that, once established, cannot be invaded by any other strategy.
Proof.We have already shown in the analysis of quadratic benefit and cost functions that all three outcomes can in fact occur, hence we only need to assert that no additional outcomes are possible.To first see that there can never be more than one interior singular strategy, we show that the selection gradient is a strictly declining function of the resident strategy.Differentiating the selection gradient with respect to the resident trait value gives, Peer-reviewed version available at Games 2018, 9, 87; doi:10.3390/g9040087 Thus, we see that there can be maximally one interior singular strategy which can be either a boundary strategy, r * = 0 (no investment), r * = 1 (maximal investment) or an interior strategy, 0 < r * < 1 (intermediate investment).As the selection gradient is a declining function of the resident strategy, we also conclude that directional selection will small evolutionary steps will eventually lead to the vicinity of this strategy, i.e., the singular strategy is convergence stable.To see that large mutational steps will also leads towards the strategy and that, once established, it cannot be invaded by any other strategy, we write the invasion fitness as It is easily seen that S r (m) < 0 and hence the double integral in the second term is always negative since x is intermediate between r and m.It follows that S r (m) < 0 whenever (r − m)S r (r) < 0, which is precisely when m < r ≤ r * or m > r ≥ r * .Thus, the strategy is continuously stable as asserted.
The other case which can similarly be classified is that of accelerating costs and decelerating benefits.In this case, coexistence of cooperators and defectors is possible whenever the evolutionary dynamics is bistable.
Theorem 3. Assume that benefits are accelerating and costs are decelerating, more specifically that B (x) < 0 and C (x) > 0 for all x ∈ [0, 1].Then selection is either towards full cooperation, towards no cooperation, or there is exactly one interior singular stable strategy r * in the sense that S r (m) > 0 if m < r ≤ r * or m > r ≥ r * .In the case of an interior singular strategy, we have that S 0 (1) > 0 and S 1 (0) > 0, enabling a protected dimorphism or cooperators and defectors.The interior singular strategy is never evolutionarily stable and decreases with the degree of relatedness.
Proof.The first part of the proof is nearly identical to the previous case, but with opposite signs.
Specifically, we show by differentiation that the selection gradient is increasing which implies that there can be at most one interior singular strategy, r * .Noting that S r (m) > 0, we conclude from Eq. 15 with r = r * that S r (m) > 0 if m < r ≤ r * or m > r ≥ r * .
It remains to show that a protected dimorphism of cooperators and defectors is possible whenever we have an interior singular strategy, 0 < r * < 1.Note that in this case, S 1 (1) > 0 and S 0 (0) < 0 due to the bistable evolutionary dynamics.Furthermore noting that the double integral in Eq. 15 is positive independent of whether m > r or r < m, we have that and showing that a protected dimorphism of cooperators and defectors is possible.

Discussion
In this paper, we have carried out a systematic investigation of the effects of relatedness on the evolution of cooperation in non-linear public goods game with continuous investments.As expected, we found that relatedness is beneficial for cooperation.When the evolutionary outcome is a single intermediate level of cooperation, higher relatedness increases that level.If, on the other hand, the evolutionary outcome is bistable such that an initially uncooperative population evolves to lower levels of cooperation and an initially cooperative society evolves to higher levels of cooperation, higher relatedness decreases the threshold level of cooperation that separates the two outcomes.The degree of relatedness also affects the potential for evolutionary branching, i.e., the emergence of two or more  coexisting strategies.We find that relatedness reduces the scope for evolutionary branching, making it more likely that all individual evolve the same strategy.
Our framework allow us to also investigate the consequences of uncertainty in the degree to which an individual is related with his or her interaction partners, and we find that such uncertainty also reduces the scope for evolutionary branching.This can be compared with an earlier result showing that uncertainty in the size of interaction groups reduces the scope for evolutionary branching when the payoff structure has an additive form but not when it has a multiplicative form [20].In this study, we have considered only payoff functions with an additive form and by analogy it appears plausible that uncertainty in relatedness might increase the scope for evolutionary branching under multiplicative payoff structures.
In a related study, Molina and Earn [23] rigorously analyzed a similar public goods game with non-linear benefits and linear costs.While they did not consider relatedness or assortment per se, one of their results, Theorem 4.4 in which they give conditions for the existence of a continuously stable strategy, allow for finite proportions of mutants and can be recast as a result on assortment.
Our results differ by asserting global convergence stability and global evolutionary stability as well as in allowing for non-linear cost functions.The latter is important, as one cannot reduce to the case of linear costs by measuring investments in units of fitness costs, i.e., by redefining trait values as x := C(x), with C being the cost function.Although this change of unit would make the cost function linear, the benefit function B would no longer depend on the sum of individual investments.Hence, the results by Molina and Earn [23] do not extend to public goods games with non-linear costs.
There are several directions in which the work presented here could be extended and we particularly wish to highlight two.First, our assumptions that the public goods are formed by adding individual contributions and that the payoff can be expressed as a difference of benefit and cost function are rather limiting and are unlikely to adequately describe all relevant settings involving public goods.Recently, Ito et al. [21] introduced and considered a large class of possible interaction structures in cooperative games.We think it would be interesting to extend our work to this larger framework and, in particular, systematically explore the effects of relatedness under the different options for aggregating rewards.Second, we base our results on a simplified representation of assortment.While simple representations have advantages, we think it would be good to complement our approach with more mechanistically-grounded studies of how assortment arises and influences the evolution of cooperation.
Our conclusion that increased assortment facilitates the evolution of cooperation is expected, in-line with empirical studies (e.g., [28]), and only a few exceptions to this rule has been reported in the literature (see for example [29] which shows how increased competition may prevent selection for altruistic genes).The value of our study thus do not lie as much in its overall conclusion as in revealing in detail how increased assortment helps to promote cooperation, for example by showing how boundaries in bistable evolutionary regimes shift and revealing that the scope for evolutionary diversification is reduced by both increased assortment and increased uncertainty in assortment.As such, our findings should prove valuable both to theoreticians and empiricists striving to interpret results on the evolution of cooperation from laboratory experiments and field observations.

Figure 2 .
Figure 2.Bifurcation diagrams showing how the evolutionary dynamics depend on average relatedness for all four possible combinations of the accelerating and decelerating benefit and cost functions in Fig.1.Green thin line represents convergence stability without evolutionary stability, blue thick line represents convergence stability with evolutionary stability, black dotted line represents neither convergence stability nor evolutionary stability, and black thick line represents evolutionary stability without convergence stability.We see that the four cases are qualitatively different.We also see that when B and C are both decelerating, a certain range of relatedness leads to evolutionary branching.The parameters used were in all cases N = 10, and b 2 , b 1 , c 2 , c 1 as in Figure1.

Figure 4 .
Figure 4. Simulations using group size N = 10, and B and C both decelerating or both accelerating.As predicted by the analytical investigation, when B and C are decelerating, the population undergoes evolutionary branching when µ = 0.4, and it reaches evolutionary stability when µ = 0.8.When B and C are accelerating, bistability occurs when µ = 0.5.In all three cases, σ 2 = 0.

(www.preprints.org) | NOT PEER-REVIEWED | Posted: 29 October 2018 doi:10.20944/preprints201810.0645.v1
) is the collective benefit of the cooperative investment of the entire group, and C(x i ) is the cost of the individual's own investment.The collective benefit is assumed to be divided equally among the members of the group, yielding a per capita benefit of B(x 1 + . . .+ x N )/N.