This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

We study evolutionary dynamics in a population of individuals engaged in pairwise social interactions, encoded as iterated games. We consider evolution within the space of memory-1strategies, and we characterize all evolutionary robust outcomes, as well as their tendency to evolve under the evolutionary dynamics of the system. When mutations are restricted to be local, as opposed to non-local, then a wider range of evolutionary robust outcomes tend to emerge, but mutual cooperation is more difficult to evolve. When we further allow heritable mutations to the player’s investment level in each cooperative interaction, then co-evolution leads to changes in the payoff structure of the game itself and to specific pairings of robust games and strategies in the population. We discuss the implications of these results in the context of the genetic architectures that encode how an individual expresses its strategy or investment.

Cooperation is a puzzle. Although it is intuitively obvious that pairs or groups of individuals can benefit from mutual cooperation, pinning down precisely when and how and for how long cooperation will occur is surprisingly difficult. This is especially true in evolution [

Evolvability is the capacity of a system to generate adaptive mutations [

Evolving social interactions. We study repeated pairwise social interactions using

These results illustrate that the evolvability of cooperation depends critically on both the type of mutations underlying the evolution of social behavior and the size of their effects. Different model choices lead to qualitatively different expectations for when and how cooperation will evolve, the types of strategies and even the types of games we might expect to find in nature. Thus, understanding the genetic architecture that encodes the range of strategies and investment schemes allowable to an organism by genetic mutations is a prerequisite for predicting the patterns of behavior that will arise in an evolving population.

We study the evolution of cooperation in a finite population of

In this study, we focus on the effects of local mutations, which alter a player’s strategy or investment by a small amount, on evolutionary dynamics. We assume that each local mutation is introduced into a monomorphic population, as is typical when studying adaptive dynamics. We look for strategies that form stable fixed points of these evolutionary dynamics. In general, a strategy that is a fixed point must have a selection gradient equal to zero, except for the special case when a fixed-point strategy lies on the boundary of strategy space. For an interior fixed point, stability is determined by the curvature of the fitness landscape around the fixed point. In this study, we start by showing that, with a particular exception that has been classified elsewhere, interior fixed points with a zero-selection gradient never occur in the space of memory-1 strategies for two-player iterated games. Thus, we need not perform any analysis of the curvature of the fitness landscape around such fixed points in our study. Instead, our analysis is focused on the selection gradient at the boundaries, where the only fixed points in our system can occur. Stable strategies at the boundary of the strategy space, which have a non-zero selection gradient facing perpendicular to the boundary, are necessarily neighbored by other strategies of the same character. Mutations that move strategies parallel to the boundary are neutral, and as a result, there exist regions of stable strategies, all of which are evolutionary robust, rather than strict evolutionary stable strategies.

We consider populations in which each player

This coordinate system allows us to obtain the following identity for the payoff of player

Furthermore, when

As in previous studies of iterated games [

In the framework of adaptive dynamics, a homogenous population is assumed monomorphic at all times and assumed to evolve towards the mutants with the highest invasion fitness [

A zero selection gradient, which ensures a fixed point of the adaptive dynamics, can only be produced in the following ways (see Materials and Methods):

The first case requires a population of size

The second case corresponds to an equalizer strategy [

The third case contains a number of possible sub-cases. However, we find (see Materials and Methods) that there are no viable strategies that produce a zero selection gradient, except at the boundaries of strategy space. Thus, the only locally-robust strategies under local mutations lie at the boundary of strategy space. The same result also holds under non-local mutations [

A strategy with the selection gradient perpendicular to and pointing towards the boundary of strategy space can only be produced in the following ways (see Materials and Methods):

The first case corresponds to a zero-determinant (ZD) strategy with

The second case corresponds to

The third case contains four sub-cases (see Materials and Methods). The first sub-case corresponds to

The second sub-case corresponds to

Thus, there are only three possible classes of locally-robust strategies: self-cooperators, self-defectors and self-alternators. These three classes of strategies are summarized and compared to the case of non-local mutations in

We have identified three classes of strategy that can be locally robust, which correspond to the same qualitative classes of globally-robust strategies identified previously [

Robust volumes under local and non-local mutations. We considered populations playing an iterated public goods game, with payoffs

Quite surprisingly, local mutations make self-alternate much easier to evolve for some choices of payoff, and the probability of reaching these strategies varies non-monotonically with the benefits for cooperation (

The probability of reaching different strategy classes under local, non-local and global mutations. We simulated populations playing an iterated public goods game, with payoffs

The probability of reaching a strategy class is not the whole story. Once a population arrives at a locally-robust strategy, which sits on the boundary of strategy space and has a selection gradient perpendicular to the boundary, mutations occurring parallel to the boundary are neutral. For example, for a robust self-cooperator strategy with

Thus, neutral drift can allow a population to eventually move away from a robust strategy type. This phenomenon is illustrated in

In summary, our results on the probability of reaching a strategy class and the dynamics of neutral drift show that mutual cooperation is harder to evolve, but persists for longer once it arises, under local

So far, we have offered little discussion of payoffs, besides constraining

We now consider the co-evolution of payoffs and strategies under local mutations. We assume each player

Neutral drift under local and non-local mutations. (Top) We simulated evolution under weak mutation, as described in

Because the equilibrium plays

and for self-alternators, the stable equilibrium satisfies:

Depending on the function

To illustrates this phenomenon, we consider a sigmoid benefit function:

We can use these results to determine which pairs of games and strategies are robust against local mutations. These are shown in

The evolution of different games. Under a sigmoidal benefit function (Equation (13)), different pairs of strategies and investment levels are robust (Equations (14) and (15)). Depending on the choice of threshold in the benefit function,

The problem of understanding the evolution of cooperation has two components. The first is to understand whether cooperation can be evolutionary stable [

An important conclusion from our results is that local mutations can substantially alter the types of robust strategies that are likely to evolve. In particular, local as opposed to non-local mutations can substantially increase the chance for self-alternating strategies to evolve (

Our results all highlight the need to understand the genetic architecture that underlies social behaviors if we are to understand behavioral evolution. While this goal may be a long way off when it comes to human behavior, it is likely more accessible when studying the evolution of cooperation in simple micro-organisms [

Here, we determine the equilibrium solutions to Equations (5) and (11) for the adaptive dynamics of memory-1 strategies and the levels of investment in cooperation.

An equilibrium solution can occur in two ways in this system. When the selection gradient is zero, the resident strategy of the population

Because viable strategies must lie in the range

We will first consider the case in which the four variables

From Equation (16), we can immediately deduce the following:

It will also be necessary to calculate the selection gradient in the original

and we see immediately that the selection gradient Equation (19) is zero if the selection gradient Equation (5) is zero.

As noted above, in order for Equation (5) to be zero, we must have either

At the boundaries, since

Extremal values of

Extremal values of

Similarly to the previous case, the sum

The difference

Finally, the difference

Strategies may lie in more than one of the class of locally-robust strategies described above. In particular, the strategy tit-for-tat lies at the intersection of all three strategy classes. However, as discussed in [

The solutions to Equations (11) and (12) for a sigmoidal benefit function as given by Equation (13) are

We thank Charles Mullon and Todd Parsons for helpful discussions. J.B.P. acknowledges funding from the Burroughs Wellcome Fund, the David and Lucile Packard Foundation, US Department of the Interior Grant D12AP00025, and Foundational Questions in Evolutionary Biology Fund Grant RFP-12-16.

Alexander J. Stewart and Joshua B. Plotkin designed and conducted the research and wrote the manuscript.

The authors declare no conflict of interest.