# Evolution of Decisions in Population Games with Sequentially Searching Individuals

^{*}

Next Article in Journal / Special Issue

Previous Article in Journal

Previous Article in Special Issue

Previous Article in Special Issue

Institute of Science and Technology Austria (IST Austria), Am Campus 1, A-3400 Klosterneuburg, Austria

Author to whom correspondence should be addressed.

Academic Editors: Martin Nowak and Christian Hilbe

Received: 12 June 2015
/
Revised: 28 August 2015
/
Accepted: 10 September 2015
/
Published: 29 September 2015

(This article belongs to the Special Issue Cooperation, Trust, and Reciprocity)

In many social situations, individuals endeavor to find the single best possible partner, but are constrained to evaluate the candidates in sequence. Examples include the search for mates, economic partnerships, or any other long-term ties where the choice to interact involves two parties. Surprisingly, however, previous theoretical work on mutual choice problems focuses on finding equilibrium solutions, while ignoring the evolutionary dynamics of decisions. Empirically, this may be of high importance, as some equilibrium solutions can never be reached unless the population undergoes radical changes and a sufficient number of individuals change their decisions simultaneously. To address this question, we apply a mutual choice sequential search problem in an evolutionary game-theoretical model that allows one to find solutions that are favored by evolution. As an example, we study the influence of sequential search on the evolutionary dynamics of cooperation. For this, we focus on the classic snowdrift game and the prisoner’s dilemma game.

The problem of mutual choice was first introduced in the economic literature by [1], who studied how to pair students with colleges so that the preference of both sides would be satisfied. Their framework, further developed in [2,3,4,5,6,7], assumes that there exists a sort of central processor that carries out the pairing. In natural populations, and often in human societies, however, the pairing of individuals happens in a more decentralized and random fashion [8]. For example, the composition of the employee-employer pair who end up concluding a contract depends on who happens to read the job advertisement and in which order the applicants come for the interview. To account for this, theoretical work on sequential and random encounters has been developed and applied to mutual choice problems, such as mate choice [9,10,11,12,13,14] and labor markets [15,16,17,18]. The main focus in sequential search problems, either with mutual or one-sided choice (e.g., female choice only) is to find decision rules that determine the acceptance or rejection of possible partners in such a way that no individual is better off by changing its rule. Thus, if the search season is finite, individuals must find a balance between waiting for a beneficial interaction by rejecting unfavorable ones, while not waiting too long so as not to remain without a partner and no payoff [19,20,21,22,23,24]. However, if the choice is mutual, the model is not a mere optimization problem, and so, there may be multiple solutions [12]. The drawback here is that there is no way of knowing which solution populations will actually reach, if any. That is, we may be able to calculate what individuals should do, but we do not know the solution followed by evolution.

On the other end of the spectrum, we have evolutionary game theory, which has been specifically developed to address evolutionary properties of decisions (e.g., whether to cooperate or defect). However, even though previous work has explored various forms of optional interactions and partner choice, particularly in the theory of the evolution of cooperation [25,26,27,28,29,30,31,32], trade-offs arising from sequential search problems have not been studied. In this paper, we address this issue by embedding a mutual choice sequential search problem to an evolutionary game-theoretical framework, and look for decisions that are favored by evolution. Moreover, we investigate how such social situations affect the evolution of cooperation.

We assume that all individuals are one of two types, cooperators or defectors, where each individual is equipped with a decision rule that dictates throughout the search season whether an encountered individual will be accepted or rejected for an interaction. The search season is assumed to consist of discrete time steps, or rounds, such that at each round, all individuals are being paired. Once an interaction is mutually accepted the interacting individuals drop out of the game pool and will not be available for future rounds. We look for decisions that are favored by evolution and study, on the one hand, how costs are associated with the search process and, on the other hand, how mistakes in evaluating the type of the opponent influence the results. For analytical results, we work out an example where we consider a search season with two rounds and give conditions for which we find unique, multiple or no stable dynamic equilibria, in which case, an evolutionary cycling occurs. Furthermore, similarly to the previous work on optional interactions and partner choice, we find that the option to refuse an interaction always favors cooperators. Interestingly, if the search for new opponents is considered to be costly, high levels of cooperation may evolve even in the prisoner’s dilemma, where for obligatory interactions, defection yields the highest payoff. This is because discriminated defectors have to on average search longer for an accepted interaction than cooperators and, thus, pay higher costs associated with search. However, mistakes in evaluating the type of opponent favor defection, because even if defectors are being discriminated against, they may be accepted by mistake.

Consider an infinitely large and well-mixed population where each individual is either a cooperator (C) or a defector (D). In every generation, each individual is about to enter a maximum of $M\in {N}_{+}$ rounds, such that at each round, individuals are pitted against each other and given a choice to interact. Encountered players are said to interact if they both decide to accept an offer from each other, in which case their search for new opponents is terminated, and they receive a strictly positive payoff according to a payoff matrix (see the section below). If, however, at least one of the two individuals decides not to interact, i.e., she declines the offer from the opponent, both will move to the next round and will be randomly assigned a new opponent. If an individual declines all of the offers or if she will be declined, she will receive a zero payoff. In every generation, all individuals thus participate in at most one interaction (i.e., play at most one one-shot game). The frequency distribution of the next generation is updated based on the payoffs received in the parent generation.

Individuals that engage in an interaction, i.e., accept an offer from each other) receive a payoff according to their types: if at least one of the two interacting individuals is a cooperator, a benefit $2b$ is produced and shared equally (b for each player). A cooperator pays a cost c for his cooperation, which is divided if both individuals are cooperators, in which case each pays $\frac{c}{2}$. Defectors do not pay costs, but if both individuals are defectors, then no benefit is produced. In addition, we assume that playing a game, even if against an unfavorable opponent, is more beneficial than remaining without an opponent. We implement this by considering an additional payoff u (see, e.g., [31]) so that all individuals that play a game receive a strictly positive payoff. For example, we may think of interactions that themselves have a value, such as offspring production. Clearly, individuals that produce offspring, even with a defector, are better off than when not reproducing at all.

By setting $c=kb$, where k describes how many times the cost of cooperation is greater than the benefit, we recover the snowdrift game (SD) for $0<k<1$ (i.e., $0<c<b$) and the prisoner’s dilemma (PD) for $1<k<2$ (i.e., $b<c<2b$). The payoff matrix for an accepted game is thus:
where $T=b,R=b(1-\frac{1}{2}k),S=b(1-k)$, $P=0$, with the additional payoff u, so that all the payoffs are strictly positive. Finally, entering each new round may be costly (e.g., the cost of search), and therefore, a will be subtracted per each entered (not necessarily accepted) round. Throughout the paper, we will use k as our main model parameter.

For both games, SD and PD, the ideal opponent is a cooperator (i.e., $R>S$ and $T>P$), and so the best decision, for both types C and D, is to always accept an opponent of type C. The question is therefore in which rounds an encountered type D should be accepted for the payoff to be maximized. This is not trivial for two reasons. First, there is a trade-off between interacting with an encountered type D in the current round and the costs associated with waiting to encounter type C in future rounds. Second, since interacting individuals drop out of the game pool, the frequency distribution and hence the expected payoff may change throughout the search season. Therefore, an individual may want to accept (decline) type D in one round, but decline (accept) him in the next.

A decision may be represented as a vector, where each entry is either zero or one, depending on whether in that round an opponent of type D is rejected or accepted, respectively. If M is the maximum number of rounds, then the maximum number of decisions in the population is ${N}_{s}={2}^{M}$. We may therefore write the complete set of decisions as:
where ${q}_{j}=({q}_{{j}_{1}},\cdots ,{q}_{{j}_{M}})\in Q$, $1\le j\le {N}_{s}$, is a decision vector and where each element ${q}_{{j}_{i}}$ is either a zero or one (i.e., say no or yes, respectively, to an encountered D) at round $1\le i<M$. Each individual possesses a decision vector, and we will denote with ${C}_{i}={C}_{{q}_{i}}$ and ${D}_{j}={D}_{{q}_{j}}$ types C and D, respectively, who use decisions ${q}_{i},{q}_{j}\in Q$. We call ${C}_{i},{D}_{j}$ strategies that individuals are identified with, and the set of all strategies we denote with S.

$$Q=\{{q}_{1},\cdots ,{q}_{{N}_{s}}\},$$

Consider maximum M rounds per generation. The expected payoff per generation for strategies ${C}_{j},{D}_{j}\in S$ may be written as:
where ${n}_{{C}_{j}},{n}_{{D}_{j}}$ are the expected number of rounds that strategies ${C}_{j},{D}_{j}$, respectively, have entered and ${\pi}_{{C}_{j}{C}_{i}}$ is the probability that strategy ${C}_{j}$ will play against ${C}_{i}$. The expected number of rounds may be expressed as:
where ${r}_{A,i}$ is the probability that in round i, strategy $A\in S$ does not play a game, that is strategy A encounters an opponent who is either not going to be accepted by A and/or who would not accept strategy A. We may write this as:
where ${p}_{B,i}$ is the frequency of $B\in S$ in round i and ${v}_{AB,i}$ is the probability that in round i, upon an encounter between A and B, either strategy A does not accept strategy B and/or B does not accept strategy A (notice that ${v}_{AB,i}={v}_{BA,i}$). This depends on the decision vector, but also on how accurately an individual evaluates the type of opponent. This is model dependent and may, for example, be a function of the reputation of the individuals type or the quality of the signal. In the next section, we analyze a model where in each round, there is a fixed probability ε that a mistake is made in correctly evaluating the type of the opponent (C vs. D). If there is a strictly positive probability $\epsilon >0$ that a mistake is made, we say that the signal of the opponent is imperfect. We assume that the opponent’s decision ${q}_{i}\in Q$ is unknown to the individual.

$$\begin{array}{cc}& {E}_{{C}_{j}}=R\sum _{{q}_{i}\in Q}{\pi}_{{C}_{j}{C}_{i}}+S\sum _{{q}_{i}\in Q}{\pi}_{{C}_{j}{D}_{i}}-a{n}_{{C}_{j}},\phantom{\rule{2.em}{0ex}}1\le j\le {N}_{s}\hfill \end{array}$$

$$\begin{array}{cc}& {E}_{{D}_{j}}=T\sum _{{q}_{i}\in Q}{\pi}_{{D}_{j}{C}_{i}}+P\sum _{{q}_{i}\in Q}{\pi}_{{D}_{j}{D}_{i}}-a{n}_{{D}_{j}},\phantom{\rule{2.em}{0ex}}1\le j\le {N}_{s},\hfill \end{array}$$

$${n}_{A}=1\xb7(1-{r}_{A,1})+2\xb7{r}_{A,1}(1-{r}_{A,2})+\cdots =\sum _{i=1}^{M}\left(i(1-{r}_{{A}_{i}})\prod _{j=1}^{i-1}{r}_{A,j}\right),\phantom{\rule{2.em}{0ex}}A\in S,$$

$${r}_{A,i}=\sum _{B\in S}{p}_{B,i}{v}_{AB,i}$$

As indicated in Equation (4), frequencies of C and D change from round to round. This is because in each round, some players may accept each other for an interaction and, thus, drop out of the game pool. Since ${r}_{A,i}$ in Equation (4) can also be interpreted as the fraction of A in round i who enter round $i+1$, we can write the frequency of $A\in S$ at round $i+1$, where $1\le i<M$, as:
where $\overline{p}={\sum}_{B\in S}{p}_{B,i}{r}_{B,i}$ is the mean absolute frequency. We may now write out the probability that a game is played between strategies $A\in S$ and $B\in S$:

$${p}_{A,i+1}={p}_{A,i}\frac{{r}_{A,i}}{\overline{p}}$$

$$\begin{array}{cc}\hfill {\pi}_{AB}& ={p}_{B,1}(1-{v}_{AB,1})+{r}_{A,1}{p}_{B,2}(1-{v}_{AB,2})+{r}_{A,1}{r}_{A,2}{p}_{B,3}(1-{v}_{AB,3})+\cdots \hfill \\ & =\sum _{i=1}^{M}\left({p}_{B,i}(1-{v}_{AB,i})\prod _{j=1}^{i-1}{r}_{A,j}\right).\hfill \end{array}$$

We study the evolution of strategies $A\in S$ by applying replicator equations:
where the dot represents a time derivative, $\overline{E}={\sum}_{B\in S}{p}_{B}{E}_{B}$ is the mean expected payoff and ${\sum}_{A}{p}_{A}=1$. Notice, that the dynamical system Equation (7) is fully characterized by the functions ${v}_{AB,i}$. After ${v}_{AB,i}$ has been found, we can simply substitute it into Equations (2)–Equation (6), and the system Equation (7) is obtained. Furthermore, note that the frequencies at the start of each generation are the frequencies in the first round of the game, i.e., ${p}_{A}={p}_{A,1}$.

$${\dot{p}}_{A}={p}_{A}\left({E}_{A}-\overline{E}\right),\phantom{\rule{2.em}{0ex}}A\in S,$$

In this section, we look for (pure) strategies ${C}_{i},{D}_{j}$, as well as their frequencies ${p}_{{C}_{i}},{p}_{{D}_{j}}$ that are favored by evolution. We analyze the existence of strategies that are uninvadable (evolutionarily-stable strategies or ESS) or that can only be invaded by neutral drift. Since, in the latter case, no other strategy has a selective advantage, i.e., strictly positive growth rate, we call it a best selective strategy (BSS). Jointly with ESS, we call them best strategies. We define a best strategy solution as the frequency distribution over a set of strategies, such that no deviation from this state increases individuals’ payoff. In other words, when the population is at the best strategy solution, no individual is (strictly) better off by changing its strategy. Note that this concept satisfies the conditions of a Nash equilibrium. Moreover, we are interested whether choosy decisions (i.e., decisions where individuals decline interactions) evolve, which best strategy solution, if any, is reached and whether evolution increases the level of cooperation.

To facilitate the analytical treatment, we focus on generations with two rounds ($M=2$). In the final Section 4, we discuss what happens when this assumption is relaxed. Because every mutually-accepted interaction yields a strictly positive payoff (see the payoff matrix in Section 2.1) and declined interaction yields a zero payoff, then at least in the final round, all individuals should accept their opponent. This reduces the number of decisions to only two decisions: either accept or decline a D at Round 1 and accept everyone at Round 2, i.e., ${q}_{1}=(1,1)$ and ${q}_{2}=(0,1)$. There are thus only four strategies ${C}_{(1,1)},{D}_{(1,1)},{C}_{(0,1)}$ and ${D}_{(0,1)}$, and we write for short ${C}_{1},{D}_{1},{C}_{0}$ and ${D}_{0}$, respectively, so that $S=\{{C}_{1},{D}_{1},{C}_{0},{D}_{0}\}.$ The dynamical model we will study is:
with ${\sum}_{A}{p}_{A}=1$ and where the expected payoffs ${E}_{A}$ are obtained from Equation (2) by first calculating ${v}_{AB,i}$ and then substituting this into Equations (3)–(6). Since, in the last round, everyone is accepted, we have ${v}_{AB,2}=0$, and for ${v}_{AB,1}$, we find:
where ${v}_{AB}={v}_{AB,1}$.

$$\left\{\begin{array}{cc}& {\dot{p}}_{{C}_{1}}={p}_{{C}_{1}}\left({E}_{{C}_{1}}-\overline{E}\right)\hfill \\ & {\dot{p}}_{{D}_{1}}={p}_{{D}_{1}}\left({E}_{{D}_{1}}-\overline{E}\right)\hfill \\ & {\dot{p}}_{{C}_{0}}={p}_{{C}_{0}}\left({E}_{{C}_{0}}-\overline{E}\right)\hfill \\ & {\dot{p}}_{{D}_{0}}={p}_{{D}_{0}}\left({E}_{{D}_{0}}-\overline{E}\right)\hfill \end{array}\right.$$

$$\begin{array}{cc}& {v}_{{C}_{1}{C}_{1}}=0\hfill \\ & {v}_{{C}_{1}{C}_{0}}=\epsilon \hfill \\ & {v}_{{C}_{1}{D}_{1}}=0\hfill \\ & {v}_{{C}_{1}{D}_{0}}=\epsilon \hfill \end{array}\phantom{\rule{2.em}{0ex}}\begin{array}{cc}& {v}_{{D}_{1}{C}_{1}}=0\hfill \\ & {v}_{{D}_{1}{C}_{0}}=1-\epsilon \hfill \\ & {v}_{{D}_{1}{D}_{0}}=1-\epsilon \hfill \\ & {v}_{{D}_{1}{D}_{1}}=0\hfill \end{array}\phantom{\rule{2.em}{0ex}}\begin{array}{cc}& {v}_{{C}_{0}{C}_{1}}=\epsilon \hfill \\ & {v}_{{C}_{0}{C}_{0}}=1-{(1-\epsilon )}^{2}\hfill \\ & {v}_{{C}_{0}{D}_{0}}=1-\epsilon (1-\epsilon )\hfill \\ & {v}_{{C}_{0}{D}_{1}}=1-\epsilon \hfill \end{array}\phantom{\rule{2.em}{0ex}}\begin{array}{cc}& {v}_{{D}_{0}{C}_{1}}=\epsilon \hfill \\ & {v}_{{D}_{0}{C}_{0}}=1-\epsilon (1-\epsilon )\hfill \\ & {v}_{{D}_{0}{D}_{0}}=1-{\epsilon}^{2}\hfill \\ & {v}_{{D}_{0}{D}_{1}}=1-\epsilon \hfill \end{array}$$

The best strategies are found by calculating the (dynamic) equilibria of system Equation (8) and analyzing their stability. Asymptotically-stable equilibria are the uninvadable strategies (ESS), and stable, but not asymptotically-stable equilibria are the best selective strategies (BSS). In our model, all BSS are stable, but not asymptotically stable, because they form a (continuous) line of equilibria, so that the dynamics is neutral in one direction of the phase plane, but asymptotically stable in all other directions. We will denote equilibrium frequencies with ${\widehat{p}}_{(\xb7)}$, such that, for example, ${\widehat{p}}_{{C}_{1}{D}_{0}}=(0,{p}_{{C}_{1}},{p}_{{D}_{0}},0)$ denotes an equilibrium where only strategies ${C}_{1}$ and ${D}_{0}$ have strictly positive frequencies. If there is more than one equilibrium, we enumerate them.

To study whether an evolutionary trajectory converges to a particular best strategy solution, we assume that a new strategy appearing in the population is a much rarer event than other demographic processes, such as strategy inheritance. We thus assume that the population settles at an attractor before a new strategy is introduced into the population. An evolutionary trajectory is thus a sequence of successful invasion events, which we simply call an invasion-substitution sequence. We assume that each invasion-substitution sequence starts from a population where everyone uses non-choosy decisions ${q}_{1}=(1,1)$. Since all individuals accept the first encountered interaction, we obtain as the initial state the solutions of the one-shot snowdrift game and the prisoner’s dilemma: for the snowdrift game $0<k<1$, the initial frequency distribution is:
and for the prisoner’s dilemma $1<k<2$, it is:

$${\widehat{p}}_{{C}_{1}{D}_{1}}=\left(2\frac{1-k}{2-k},\frac{k}{2-k}\right)$$

$${\widehat{p}}_{{D}_{1}}=1.$$

In Section 3.1, we analyze a basic model, which assumes no search costs and where the type of opponent is evaluated correctly ($a=0,\epsilon =0$). In Section 3.2, we relax the assumptions by considering the effect of imperfect signals ($\epsilon >0$, with $a=0$), and in Section 3.3, we consider the effect of search costs ($a>0$, with $\epsilon =0$). Lastly, in Section 3.4, we discuss the full model ($\epsilon >0,a>0$). For all models, we first find analytical conditions for choosy strategies ${C}_{0}$ and ${D}_{0}$ to invade a population that uses only non-choosy decisions (see Equation (10)). If choosy strategies are not able to invade, the initial state given in Equation (10) is the best strategy solution. Next, we find all of the (remaining) best strategy solutions, and investigate to which best strategy solution invasion-substitution sequence converges. The results are summarized in Figure 1, Figure 2, Figure 3 and Figure 4, where each model has its own figure and where each panel gives a simplex spanned by strategies in S with arrows that indicate the direction of the evolutionary dynamics. In addition, we draw grey dashed arrows for each qualitatively different invasion-substitution sequence that our model contains. See the captions, Section 3.1, Section 3.2, Section 3.3 and Section 3.4 and the Appendix for further explanations and discussions.

In the basic model, we suppose that entering a new round is not costly, $a=0$, and that the type of the opponent (C vs. D) is evaluated correctly, $\epsilon =0$. This model can be fully analyzed analytically. Figure 1 summarizes the results (see Appendix for the full analysis).

Our general finding is that if the benefit from mutual cooperation is high enough (parameter k is not too large), optional interactions favor choosy decisions. This is because by rejecting encountered defectors in the first round, choosy individuals have a chance to interact with a cooperator in the second round. For the snowdrift game, the payoff matrix favors choosy cooperators over choosy (and non-choosy) defectors, and so, cooperation is the winning strategy. In the prisoner’s dilemma, choosy defectors do better than choosy cooperators, and so, defection takes over, even if initially choosy cooperators have an advantage over non-choosy defectors.

Choosy decisions ${q}_{2}=(0,1)$ can evolve if rare choosy strategies ${C}_{0}$ and/or ${D}_{0}$ can invade a non-choosy population Equation (10). We find that for all $0<k<2$, choosy defectors ${D}_{0}$ have a zero growth rate and can thus increase in frequency only via neutral drift (see the Appendix and Figure 1). This is simply because the only pairs who go to the second round are defectors ${D}_{0}$ and ${D}_{1}$, and thus, rejecting a defector in the first round yields no payoff advantage. However, when choosy cooperators ${C}_{0}$ are introduced to the population, then in the second round, half of the individuals are choosy cooperators (the other half are defectors), and so, rejecting a defector in the first round gives cooperators an equal chance to eventually interact with a cooperator. We get that ${C}_{0}$ can invade the initial population whenever $0<k<\frac{4}{3}$. For $\frac{4}{3}<k<2$, the initial state given in Equation (10) is the best strategy solution. Within this parameter range, the benefit from mutual cooperation is too low to outweigh the fact that with probability one half, a ${C}_{0}$ ends up interacting with a D in the second round.

In this section, we look for best strategy solutions. Each enumeration corresponds to the panels of Figure 1:

(A) $0<k<1$: The best selective strategy is a strategy pair (${C}_{0},{C}_{1}$) that lies on the line of equilibria ${\widehat{p}}_{{C}_{1}{C}_{0}}$ with $k<{p}_{{C}_{0}}\le 1$. This line of equilibria exists because ${C}_{0}$ has the same payoff against ${C}_{1}$ and vice versa. Interestingly, defectors can invade the strategy pair (${C}_{0},{C}_{1}$) if population drifts past the point ${p}_{{C}_{0}{C}_{1}}^{*}=k$ (see Figure 1A). Eventually, however, the dynamics will lead defectors to extinction, and a mix of cooperative strategies $({C}_{0},{C}_{1})$ is again established. This evolutionary cycle, which relies on the absence of selection on the effect of neutral drift, can be written symbolically as $({C}_{0},{C}_{1})\stackrel{\text{drift}}{\to}({C}_{0},{C}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}/{D}_{1}}{\to}({C}_{0},{C}_{1})$. We thus have that a strategy pair $({C}_{0},{C}_{1})$ can drift from one fully-cooperative equilibrium to another, but after passing a certain threshold value, it can be temporarily invaded by defectors.

(B) $1<k<\frac{4}{3}$: A strategy pair of defectors (${D}_{0},{D}_{1}$) that lies on the line of equilibria ${\widehat{p}}_{{D}_{1}{D}_{0}}$, with $0<{p}_{{D}_{0}}\le 1$, is the best selective strategy. This solution can only be escaped if strategy ${D}_{0}$ drifts to complete extinction (${p}_{{D}_{0}}=0$), after which strategy ${C}_{0}$ can invade, and the coexistence of $({D}_{1},{C}_{0})$ is established at ${\widehat{p}}_{{D}_{1}{C}_{0}}=({p}_{{D}_{1}},{p}_{{C}_{0}})=(\frac{2(k-1)}{2-k},\frac{4-3k}{2-k})$. However, this state can be invaded by ${D}_{0}$, and the strategy pair (${D}_{0},{D}_{1}$) is established again as a BSS, i.e., we have $({D}_{0},{D}_{1})\stackrel{\text{drift}}{\to}{D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({C}_{0},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({D}_{0},{D}_{1})$.

(C) $\frac{4}{3}<k<2$: Any strategy pair (${D}_{0},{D}_{1}$) that lies on the line of equilibria ${\widehat{p}}_{{D}_{1}{D}_{0}}$ for all $0\le {p}_{{D}_{0}}\le 1$ is a BSS.

In this section, we describe all of the invasion-substitution sequences (see the Appendix for the full analysis). As in the previous section, each enumeration corresponds to the panels of Figure 1:

(A) $0<k<1$: The initial population $({C}_{1},{D}_{1})$, indicated by a grey dashed circle, can be (selectively) invaded by a choosy cooperator ${C}_{0}$, but not by a choosy defector ${D}_{0}$ (see Section 3.1.1 and Figure 1A). If ${C}_{0}$ succeeds at invading, a strategy pair (${C}_{0}$, ${C}_{1}$) at ${\widehat{p}}_{{C}_{1}{C}_{0}}$, with $k<{p}_{{C}_{0}}\le 1$, is established as a BSS. The invasion-substitution sequence is thus $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({C}_{0},{C}_{1})$ (see the dashed grey trajectory drawn in Figure 1A). Recall that this solution can be escaped by neutral drift (see Section 3.1.2).

(B) $1<k<\frac{4}{3}$: The initial population consist of defectors ${D}_{1}$ only. If choosy strategies are introduced, ${C}_{0}$ may invade and can be established at an equilibrium together with ${D}_{1}$, at least, until ${D}_{0}$ is introduced, and a strategy pair of defectors $({D}_{0},{D}_{1})$ at ${\widehat{p}}_{{D}_{1}{D}_{0}}$ with $0<{p}_{{D}_{0}}\le 1$, will be established as a BSS. Interestingly, pure defection takes over the coexistence of (${C}_{0}$, ${D}_{1}$) at ${\widehat{p}}_{{D}_{1}{C}_{0}}$ only if defectors start to be choosy against themselves. We have ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({D}_{1},{D}_{0})$. This solution may be escaped by drift (see Section 3.1.2).

(C) $\frac{4}{3}<k<2$: No strategy can selectively invade the initial state ${\widehat{p}}_{{D}_{1}}=1$. Due to drift, any best strategy solution on the line of equilibria ${\widehat{p}}_{{D}_{1},{D}_{0}}$ with $0\le {p}_{{D}_{1}}\le 1$ may be reached.

In this model, we suppose that individuals make a mistake with probability $\epsilon >0$ in evaluating whether the encountered opponent is a cooperator or a defector. We suppose there are no search costs, $a=0$.

The basic model unfolds in three ways when $\epsilon >0$. Firstly, for $0<k<1$, the equilibrium ${\widehat{p}}_{{C}_{0}}=1$ becomes unstable. This is because, when most of the individuals are of type ${C}_{0}$, i.e., ${p}_{{C}_{0}}$ is close to one, and mistakes are being made, then most of the declined individuals are ${C}_{0}$’s, but only mistaken as D’s. Therefore, most of the individuals in the first and second round are ${C}_{0}$, and so, defectors, being rare, will end up playing against the common C. Since the payoff for D against C is always greater than C against C, i.e., $R<T$ for all $0<k<2$, defectors can always invade ${C}_{0}$ when rare. The reason why this is not true in the basic model for $0<k<1$, is that since no mistakes are made and D’s are rare, only the defector-cooperator pairs go to the second round. Since D’s play in the second round against D and C with equal probability, then for $0<k<1$ we have $\frac{1}{2}(T+P)<R$, and so, defectors cannot invade ${C}_{0}$.

The second unfolding happens at ${p}_{{C}_{1}}=1$. In the model with imperfect signals, the equilibrium ${\widehat{p}}_{{C}_{1}}=1$ is stable in the direction of ${p}_{{D}_{0}}=1$, for $0<k<\epsilon $, whereas in the basic model, this is always unstable. Because near ${p}_{{C}_{1}}=1$, the choosy defectors ${D}_{0}$ encounter mainly ${C}_{1}$’s, then fraction $1-\epsilon $ is being accepted, and the game is played. However, fraction ε is declined, and so, in the second round with probability half, the game will be played against another defector. In the basic model, all ${D}_{0}$’s play against ${C}_{1}$. Thus, in the imperfect signal model, for $0<k<\epsilon $, the cost of playing against another D is greater than the benefit of playing against a C, and so, a rare ${D}_{0}$ is not able to invade ${C}_{1}$.

Finally, numerical analysis shows that an unstable trimorphic equilibrium ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}$ exists for $0<k<1$ (see the Appendix). As it is not the best strategy solution and it does not affect the invasion-substitution dynamics, it is not drawn in Figure 2.

We find that for $0<k<1$, the initial non-choosy population given in Equation (10) can never be invaded by ${D}_{0}$ because its growth rate is negative. For $1<k<2$, the strategy ${D}_{0}$ has a zero growth rate and can thus increase in frequency only by neutral drift (Appendix). Strategy ${C}_{0}$ can invade ${\widehat{p}}_{{C}_{1}{D}_{1}}$ for all $0<k<1$, and it can invade ${\widehat{p}}_{{D}_{1}}$ whenever $1<k<\frac{4}{3+\epsilon}$.

All of the best strategy solutions (besides the solution ${\widehat{p}}_{{D}_{1}}$; see above) are found numerically (see the Appendix for the numerical analysis). Below, the enumeration corresponds to the panels of Figure 2.

(A)–(B) $0<k<1$: Strategy pair (${C}_{0},{D}_{0}$) at the equilibrium ${\widehat{p}}_{{C}_{0}{D}_{0}}$ is an ESS. No other best strategy solutions are found.

(C) $1<k<\frac{4}{3+\epsilon}$: A strategy pair (${D}_{0},{D}_{1}$) on the line of equilibria ${\widehat{p}}_{{D}_{0}{D}_{1}}$, for $0<{p}_{{D}_{0}}\le 1$, is a BSS and the only best strategy solution. A population may move between different values of ${\widehat{p}}_{{D}_{0},{D}_{1}}$ due to drift. If ${D}_{0}$ becomes extinct, a ${C}_{0}$ may invade, and the coexistence of ${D}_{1}$ and ${C}_{0}$ at ${\widehat{p}}_{{D}_{1}{C}_{0}}$ is established. However, ${D}_{0}$ can invade this state, and the best selective strategy solution (${D}_{0},{D}_{1}$) is again established. Neutral drift may thus cause the following cycle: $({D}_{0},{D}_{1})\stackrel{\text{drift}}{\to}{D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({C}_{0},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({D}_{0},{D}_{1})$.

(D) $\frac{4}{3+\epsilon}<k<2$: A strategy pair (${D}_{0},{D}_{1}$) at any point on the line of equilibria ${\widehat{p}}_{{D}_{0}{D}_{1}}$, $0\le {p}_{{D}_{0}}\le 1$ is a BSS and the only best strategy solution.

All of the invasion-substitution sequences were found numerically (see the Appendix). The enumeration corresponds to the panels of Figure 2:

(A)–(B) $0<k<1$: Of the choosy strategies, only ${C}_{0}$ is able to invade the initial resident population ${\widehat{p}}_{{C}_{1}{D}_{1}}$, after which a strategy pair $({D}_{1},{C}_{0})$ is established at ${\widehat{p}}_{{D}_{1}{C}_{0}}$. This state can be invaded by ${D}_{0}$, and a stable coexistence of choosy strategies (${C}_{0},{D}_{0}$) is established at ${\widehat{p}}_{{C}_{0},{D}_{0}}$. As this is an ESS, it is an evolutionary end point. We have: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({C}_{0},{D}_{0})$.

(C) $1<k<\frac{4}{3+\epsilon}$: Only ${C}_{0}$ can selectively invade the initial population ${\widehat{p}}_{{D}_{1}}=1$, after which ${D}_{0}$ can invade, and a BSS solution ${\widehat{p}}_{{D}_{0},{D}_{1}}$, with $0<{p}_{{D}_{0}}\le 1$, is reached. We have ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({D}_{1},{D}_{0})$ (the invasion-substitution sequence is qualitatively similar to the one in Figure 1B). This state can only be escaped by neutral drift (see Section 3.2.2).

(D) $\frac{4}{3+\epsilon}<k<2$: No strategy can selectively invade the initial state ${\widehat{p}}_{{D}_{1}}=1$. Due to drift, any BSS at equilibrium ${\widehat{p}}_{{D}_{1},{D}_{0}}$, with $0\le {p}_{{D}_{1}}\le 1$, may be reached (this case is similar to Figure 1C).

In this section, we suppose that searching for a new opponent is costly $a>0$, but that no mistakes are made $\epsilon =0$. We analyze in detail a case $0<a<\frac{1}{10}b$ where costs are relatively small compared to the benefit from cooperation. The effect of higher costs is briefly discussed in the paragraph below. When analytical treatment was not possible, we resorted to numerical investigations (see the Appendix).

Search costs bring out a richer bifurcation structure than the basic model (see Figure 3). First of all, the line of equilibria $({p}_{{D}_{0}},{p}_{{D}_{1}})$ unfolds to a one-directional flow towards ${p}_{{D}_{1}}=1$. This is simply because rounds are costly, and so, accepting a D in the first round is better than accepting D in the second round. This does not happen to the other line of equilibria with only cooperators, because a population of cooperators plays only one round (since $\epsilon =0$, no cooperators are mistaken for defectors). Secondly, the non-choosy population ${\widehat{p}}_{{C}_{1},{D}_{1}}$ for $k<4\frac{a}{b}$ cannot be invaded by ${C}_{0}$, because the costs of going to the second round are too high compared to the benefit gain from cooperation. Furthermore, numerical analysis shows that there are several trimorphic equilibria, but since they are all unstable and they do not affect the invasion substitution sequence, they are not drawn in Figure 3. We found no four-morphic equilibria.

We observe from Figure 3 (and the Appendix) that increasing search costs affects the order in which monomorphic equilibria change stability. In this paper, we present only the case $0<a<\frac{1}{10}b$, but all of the other cases can be derived in a straightforward fashion by swapping the order in which these bifurcations occur.

Rare defectors ${D}_{0}$ have for all $0<k<2$ a negative growth rate and, thus, are not able to invade the initial population given in Equation (10) (see the Appendix). However, rare cooperators ${C}_{0}$ can invade the initial state ${\widehat{p}}_{{C}_{1}{D}_{1}}$ whenever:
and the initial state ${\widehat{p}}_{{D}_{1}}$ whenever:

$$4\frac{a}{b}<k<1,$$

$$1<k<\frac{4}{3}\left(1-\frac{a}{b}\right).$$

We therefore have that for $0<k<4\frac{a}{b}$, the state ${\widehat{p}}_{{C}_{1}{D}_{1}}$ is the best strategy solution and for $\frac{4}{3}\left(1-\frac{a}{b}\right)<k<2$, the state ${\widehat{p}}_{{D}_{1}}$ is the best strategy solution.

The enumeration corresponds to the panels of Figure 3. We only describe the intervals of k with qualitatively different best strategies.

(A) $0<k<4\frac{a}{b}$: There exist two best strategies: the equilibrium ${\widehat{p}}_{{C}_{1},{D}_{1}}$ is an ESS, and every equilibrium ${\widehat{p}}_{{C}_{1},{C}_{0}}$ where $\frac{b}{b+2a}k<{p}_{{C}_{0}}\le 1$ is a BSS.

(B)–(D) $4\frac{a}{b}<k<1+2\frac{a}{b}$: In Figure 3B–D, the only best strategy is a BSS pair $({C}_{1},{C}_{0})$ at ${\widehat{p}}_{{C}_{1}{C}_{0}}$ with $\frac{b}{b+2a}k<{p}_{{C}_{0}}\le 1$. Interestingly, this cooperative state is the best selective strategy also in the prisoner’s dilemma when $1<k<1+2\frac{a}{b}$.

(E) $1+2\frac{a}{b}<k<\frac{4}{3}(1-\frac{a}{b})$: There are no best strategies.

(F) $\frac{4}{3}(1-\frac{a}{b})<k<2$: The only best strategy is ${D}_{1}$, and it is an ESS.

Below, the enumeration corresponds to the panels of Figure 3. We only describe the intervals of k with qualitatively different invasion-substitution sequences.

(A) $0<k<4\frac{a}{b}$: The initial state $({C}_{1},{D}_{1})$ at ${\widehat{p}}_{{C}_{1}{D}_{1}}$ is also the evolutionary endpoint.

(B) $4\frac{a}{b}<k<1$: In Figure 3B–C, a strategy ${C}_{0}$ can invade the pair $({C}_{1},{D}_{1})$, after which a BSS pair $({C}_{1},{C}_{0})$ at ${\widehat{p}}_{{C}_{1}{C}_{0}}$ with $\frac{b}{b+2a}k<{p}_{{C}_{0}}\le 1$ gets established. We get $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({C}_{1},{C}_{0})$ (this is similar to Figure 1A).

(C)–(D) $1<k<1+2\frac{a}{b}$: An initial population ${D}_{1}$ can be invaded and substituted by ${C}_{0}$, which is at ${\widehat{p}}_{{C}_{0}}=1$ a BSS. Neutral drift may establish a pair $({C}_{1},{C}_{0})$ at ${\widehat{p}}_{{C}_{1}{C}_{0}}$ with any $\frac{b}{b+2a}k<{p}_{{C}_{0}}\le 1$ as a BSS.

(E) $1+2\frac{a}{b}<k<\frac{4}{3}(1-\frac{a}{b})$: We get an evolutionary cycle: ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}{D}_{0}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{1}}{\to}{D}_{1}$, etc.

In this section, we discuss the full model for $a<\frac{1}{7}b$ and for mistakes that are sufficiently smaller than the ratio between search costs and benefits, i.e., $\epsilon <<\frac{a}{b}$. If this assumption is relaxed, it affects the order in which the stability of monomorphic equilibria changes (of course, other changes in the interior of the space are possible, as well). To give a complete analysis of this full model is out of the scope of this paper, and so, in cases where analytical results could not be obtained, we use $b=15,a=1,\epsilon =0.01$ and where $0<k<2$.

We find that choosy defectors ${D}_{0}$ can never invade the initial population given in Equation (10), but that choosy cooperators ${C}_{0}$ can invade ${\widehat{p}}_{{C}_{1}{D}_{1}}$ whenever:
and ${\widehat{p}}_{{D}_{1}}$ whenever:

$$\frac{2}{b(1-\epsilon )}\left[a(1-3\epsilon )+\sqrt{9{a}^{2}{\epsilon}^{2}-2ab{\epsilon}^{2}-6{a}^{2}\epsilon +2ab\epsilon +{a}^{2})}\right]<k<1$$

$$1<k<\frac{4}{3+\epsilon}\left[1-\frac{a}{b}(1-\epsilon )\right].$$

The opposite conditions say for which parameter values the corresponding initial states are best strategy solutions.

In this section, we deal simultaneously with the best solutions and invasion-substitutions sequences. The enumeration corresponds to the panels of Figure 4:

(A)–(D) $0<k<\epsilon (1+2\frac{a}{b})$: The strategy pair $({C}_{1},{D}_{1})$ at ${\widehat{p}}_{{C}_{1}{D}_{1}}=(2\frac{1-k}{2-k},\frac{k}{2-k})$ is the only ESS and the end point of evolution.

(E) ${k}_{{\left({C}_{1}{D}_{1}{C}_{0}\right)}^{\text{in}}}<k<{k}_{{\left({C}_{1}{D}_{1}{C}_{0}{D}_{0}\right)}^{\text{in}}}$: A trimorphic equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}}$ enters the interior of the phase space by passing through ${\widehat{p}}_{{C}_{1}{D}_{1}}$, which becomes unstable in the ${C}_{0}$-direction. The strategy pair $({C}_{1},{D}_{1})$ can thus be invaded by ${C}_{0}$, and a trimorphism $({C}_{1},{D}_{1},{C}_{0})$ is established at ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}}$. Interestingly, ${D}_{0}$ can invade this state, but the dynamics will lead the population back to the initial state ${\widehat{p}}_{{C}_{1}{D}_{1}}$. After ${D}_{1}$ becomes completely extinct ${p}_{{D}_{1}}=0$, choosy cooperators ${C}_{0}$ can invade again, and an evolutionary cycle occurs: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({C}_{1}{D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({C}_{1},{D}_{1})$, etc.

(F) ${k}_{{\left({C}_{1}{D}_{1}{C}_{0}{D}_{0}\right)}^{\text{in}}}<k<{k}_{{\left({C}_{1}{D}_{1}{C}_{0}{D}_{0}\right)}^{{\text{out}}_{s}}}$: A pair of four-morphic equilibria ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}{D}_{0}}^{\text{s}}$ and ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}{D}_{0}}^{\text{u}}$ appear via a saddle-node bifurcation. We get that a trimorphic population $({C}_{1},{D}_{1},{C}_{0})$ at ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}}$ that is invaded by ${D}_{0}$ is in the basin of attraction of the stable four-morphic equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}{D}_{0}}^{\text{s}}$, which is an ESS and, thus, also the end point of evolution. We have: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({C}_{1}{D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({C}_{1},{D}_{1},{C}_{0},{D}_{0})$.

Within this parameter range, a pair of trimorphic equilibria ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}^{\text{s}}$ and ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}^{\text{u}}$ appear via a saddle bifurcation. Both equilibria are unstable in the ${D}_{1}$-direction, and thus, neither of them is an ESS (and hence, not drawn).

(G) ${k}_{{\left({C}_{1}{D}_{1}{C}_{0}{D}_{0}\right)}^{{\text{out}}_{s}}}<k<{k}_{{\left({C}_{1}{C}_{0}{D}_{0}\right)}^{{\text{out}}_{s}}}$: The stable equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}{D}_{0}}^{\text{s}}$ exits the interior state space by passing through ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}^{\text{s}}$, which becomes stable also in the ${D}_{1}$-direction and, thus, becomes an ESS. Because the neighborhood of ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}}$ lies in the basin of attractions of ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}^{\text{s}}$, we have: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({C}_{1}{D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({C}_{1},{C}_{0},{D}_{0})$.

(H) ${k}_{{\left({C}_{1}{C}_{0}{D}_{0}\right)}^{{\text{out}}_{s}}}<k<{k}_{{\left({C}_{1}{D}_{1}{C}_{0}\right)}^{\text{out}}}$: the equilibrium ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}$ exits the interior state space by passing through ${\widehat{p}}_{{D}_{0}{C}_{0}}$, which becomes stable. We get: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({C}_{1}{D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({C}_{0},{D}_{0})$.

(I) ${k}_{{\left({C}_{1}{D}_{1}{C}_{0}\right)}^{\text{out}}}<k<1$: The trimorphic equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}{C}_{0}}$ exits the interior state space by passing through ${\widehat{p}}_{{D}_{1}{C}_{0}}$, which becomes stable in the ${C}_{1}$-direction, but is unstable in the ${D}_{0}$-direction, and we get: $({C}_{1},{D}_{1})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}({C}_{0},{D}_{0})$.

(J) $1<k<1+\epsilon \frac{a}{b}\left(1-2\epsilon \right)$: The strategy ${D}_{1}$ at ${\widehat{p}}_{{D}_{1}}=1$ is the initial non-choosy population. Otherwise, the invasion-substitution sequence is not altered: ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({C}_{0},{D}_{0})$.

(K) $1+\epsilon \frac{a}{b}\left(1-2\epsilon \right)<k<{k}_{{\left({C}_{0}{D}_{0}\right)}^{\text{SN}}}$: An unstable equilibrium ${\widehat{p}}_{{C}_{0}{D}_{0}}^{\text{u}}$ enters the interior of the phase space by passing through ${\widehat{p}}_{{D}_{1}}=1$, which becomes stable in the ${C}_{0}$-direction. The dynamics is the same as above: ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}\phantom{\rule{4.pt}{0ex}}{D}_{0}}{\to}({C}_{0},{D}_{0})$.

(L) ${k}_{{\left({C}_{0}{D}_{0}\right)}^{\text{SN}}}<k<1+\frac{a}{b}\left(1-\epsilon -{\epsilon}^{2}\right)$: The equilibria ${\widehat{p}}_{{C}_{0}{D}_{0}}^{\text{u}}$ and ${\widehat{p}}_{{C}_{0}{D}_{0}}^{\text{s}}$ disappear via a saddle-node bifurcation. This results in an evolutionary cycle: ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}{D}_{1}$, etc.

(M) $1+\frac{a}{b}\left(1-\epsilon -{\epsilon}^{2}\right)<k<\frac{4}{3+\epsilon}\left(1-\frac{a}{b}(1-\epsilon )\right)$: The equilibrium ${\widehat{p}}_{{C}_{1}{D}_{0}}$ exits the interior state space, but the invasion-substitution sequence is not affected: ${D}_{1}\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{C}_{0}}{\to}({D}_{1},{C}_{0})\stackrel{\text{invasion}\phantom{\rule{4.pt}{0ex}}\text{by}{D}_{0}}{\to}{D}_{1}$ etc.

(N) $\frac{4}{3+\epsilon}\left(1-\frac{a}{b}(1-\epsilon )\right)<k<2$: The equilibrium ${\widehat{p}}_{{D}_{1}{C}_{0}}$ exits the interior state space by passing through ${\widehat{p}}_{{D}_{1}}$, which becomes stable in all directions and, thus, becomes an ESS and an evolutionary endpoint.

In this paper, we have presented an evolutionary game-theoretical model, where individuals search for a single long-term interaction. At each encounter an opponent may be rejected, which opens up the opportunity for individuals to find a more beneficial interaction in the future. The search time being finite, the decision is conditioned not only by the payoff matrix and interacting individuals, but also by the point in time of the search season. We embedded our model in the classic games of cooperation and studied the coevolution of cooperative behavior and decisions to accept/decline an interaction. We then analyzed the model’s consequences when individuals face costs associated with the search and when mistakes occur in interpreting the type of encountered individual.

In Table 1, we have summarized the results presented in Section 3.1, Section 3.2, Section 3.3 and Section 3.4 and in Figure 1, Figure 2, Figure 3 and Figure 4. We list for each model the set of strategies that are reached by evolution when initially all individuals are assumed to be non-choosy, and to simplify, we group the results according to the game that is played (snowdrift vs. prisoner’s dilemma). Note that for each parameter value, there is only one evolutionary endpoint, but within a range of values, there may be many. Our general finding is that optional interactions and search costs facilitate cooperation, while mistakes have a counteracting effect. The reasoning is as follows. When the benefit from mutual cooperation is high enough (parameter k is not too large), optional interactions favor choosy decisions. This is due to the fact that cooperators and even defectors benefit from rejecting a defector in the first round in order to find a cooperator in the second round. Notice a subtlety here: choosy defectors do not have a payoff advantage over non-choosy defectors unless there exist (choosy) cooperators in the population to be taken advantage of in the second round.

Best Strategy Solutions (reached by evolution) | ||
---|---|---|

Snowdrift game ($0<k<1$) | Prisoner’s dilemma ($1<k<2$) | |

Non-choosy Population | ESS: $({C}_{1},{D}_{1})$ | ESS: ${D}_{1}$ |

Basic Model ($a=0,\epsilon =0$) | BSS: $({C}_{0},{C}_{1})$ | BSS: $({D}_{0},{D}_{1})$ |

Imperfect Signals ($a=0,\epsilon >0$) | ESS: $({C}_{0},{D}_{0})$ | BSS: $({D}_{0},{D}_{1})$ |

Search Costs ($a>0,\epsilon =0$) | ESS: $({C}_{1},{D}_{1})$ BSS: $({C}_{0},{C}_{1})$ | BSS: $({C}_{0},{C}_{1})$BSS: $({D}_{0},{D}_{1})$Evolutionary Cycle |

Full Model ($a>0,\epsilon >0$) | ESS: $({C}_{1},{D}_{1})$ESS: $({C}_{0},{D}_{0})$ESS: $({C}_{1},{C}_{0},{D}_{0})$ESS: $({C}_{1},{D}_{1},{C}_{0},{D}_{0})$Evolutionary Cycle | ESS: $({C}_{0},{D}_{0})$ESS: ${D}_{1}$Evolutionary Cycle |

In the snowdrift game, the payoff matrix is such that choosy cooperators do better than choosy defectors, and so, evolution favors cooperation. In the prisoner’s dilemma, this is the other way round, and so, even if initially choosy cooperators have an advantage over non-choosy defectors, choosy defectors take over and pure defection is established. However, this is not necessarily true when the search is costly, because cooperators play on average less rounds than defectors and, thus, may outcompete defectors by paying less costs. Therefore, in the prisoner’s dilemma, optional interactions together with search costs may lead to high levels of cooperation. However, if mistakes are made in evaluating the type of opponent, then defectors may be accepted, while they would normally be rejected. Thus, with a sufficiently high probability of making mistakes, the advantage of cooperation is lost, and evolution may lead to defection (compare Figure 3 and Figure 4).

To facilitate the analytical treatment, we made several simplifying assumptions. Firstly, we considered each search season to be only two rounds long. Since the number of decisions grows exponentially as a function of the number of rounds, our assumption keeps the number of equations manageable. However, dealing with a greater number of rounds is a straightforward task and is readily integrated in our general model. Moreover, since it is the cooperators that benefit from optional interactions (see above), we expect that the more rounds there are, the less stringent are the conditions for the evolution of cooperation [31]. Secondly, we only deal with two types of individuals. It would be again straightforward to extend this model to any number of types. Complexity arises when introducing continuous distributions, because not only does it lead to infinite dimensional dynamical systems (e.g., partial differential equations), but also to function-valued strategies where the inheritance mechanism remains largely an open question. Another direction for future work is to consider the number of rounds as random, thereby introducing the uncertainty of knowing whether the encountered opponent is the last one before the end of a search season. We expect this to benefit defectors, as it becomes increasingly risky to reject an interaction. Finally, we note that another way of carrying out errors would be to introduce the possibility of mistakes in implementing the strategies of individuals [33]. However, as we study games with long social ties, we view strategies as an inherent property of an individual and, thus, static. Therefore, we find it unlikely that a strategy would be executed by a mistake, and hence, such stochasticity is not considered here.

Our results are in line with previous models that have incorporated optional interactions or partner choice allowing cooperative individuals to refuse non-beneficial interactions [27,28,29,30,31]. The most notable work is by [31], where, similarly to our study, the coevolution of choosiness and cooperation was examined. However, they investigated the effect of mortality, which influences the length of the game (corresponding to the parameter M in our model; see above), but did not consider mistakes nor the effect of search costs (this parameter was fixed). Importantly, since in their work, non-beneficial interactions could be terminated and partner switching was allowed, the nature of social ties under consideration was different than in the present paper.

To conclude, we feel that the class of games that considers trade-offs arising from the problems of sequential search, a commonly-assumed context in mate choice and economics, has been largely overlooked in the theoretical literature of cooperation. We believe, however, that this direction of research not only broadens the work on games with partner choice and optional interactions, but also keeps the models simple enough to reach general conclusions as to why in many species, individuals work together for a common purpose or benefit.

We thank Sebastian Novak, Tiago Paix$\tilde{\mathrm{a}}$o, Gabby Přiklopilová, Sheena McHattie Rieder, Barbora Trubenová and two anonymous reviewers whose comments on an earlier manuscript greatly improved this paper. The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme (FP7/2007–2013) under REAGrant Agreement No. 291734, the Austrian Science Fund (FWF) S11407-N23 (RiSE/SHiNE) and the ERCStart Grant (279307: Graph Games).

Authors contributed equally to this work.

The author declares no conflict of interest.

We find all equilibria $\widehat{p}=({p}_{{C}_{1}},{p}_{{D}_{1}},{p}_{{C}_{0}},{p}_{{D}_{0}})$ by solving the zero states of Equation (8). We denote equilibrium frequencies with ${\widehat{p}}_{(\xb7)}$, such that, for example, ${\widehat{p}}_{{C}_{1}{D}_{0}}=({p}_{{C}_{1}},0,0,{p}_{{D}_{0}})$ denotes an equilibrium where only strategies ${C}_{1}$ and ${D}_{0}$ have strictly positive frequencies. If there is more than one equilibrium, we enumerate them. The stability is found by calculating the eigenvalues associated with each of the three eigenvectors that are connected to the equilibrium. Eigenvalues are denoted such that, for example, ${\lambda}_{AB,ABC}$ is an eigenvalue associated with equilibrium ${\widehat{p}}_{AB}$ with an eigenvector lying in the space spanned by strategies $A,B$ and C.

By construction, model Equation (7) (or Equation (8) contains always the monomorphic equilibria ${\widehat{p}}_{{C}_{1}}=1,{\widehat{p}}_{{D}_{1}}=1,{\widehat{p}}_{{C}_{0}}=1,$ and ${\widehat{p}}_{{D}_{0}}=1$. The eigenvalues connected to these equilibria are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{1}}=\frac{bk}{2},\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{C}_{1}}=0,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{0}}=\frac{bk}{2},\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{0}}=b(1-\frac{3}{4}k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{D}_{1}}=\frac{1}{2}b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{D}_{1}}=0,\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{C}_{0}}=0,\hfill \\ & {\lambda}_{{D}_{1},{D}_{0}}=0\hfill \\ & {\lambda}_{{C}_{0},{D}_{0}}=\frac{1}{2}b(k-1)\hfill \\ & {\lambda}_{{D}_{0},{C}_{0}}=b(1-k).\hfill \end{array}$$

The basic model has two lines of dimorphic equilibria ${\widehat{p}}_{{C}_{1},{C}_{0}}=({p}_{{C}_{1}},1-{p}_{{C}_{1}})$ and ${\widehat{p}}_{{D}_{1},{D}_{0}}=({p}_{{D}_{1}},1-{p}_{{D}_{1}})$, where each point on the line, for all $0\le {p}_{{C}_{1}},{p}_{{D}_{1}}\le 1$, is an equilibrium. In addition, there are three isolated (in the space of dimorphic populations) dimorphic equilibria ${\widehat{p}}_{{C}_{1},{D}_{1}}=(\frac{2(1-k)}{2-k},\frac{k}{2-k})$, ${\widehat{p}}_{{C}_{1},{D}_{0}}=(\frac{2(1-k)}{2-k},\frac{k}{2-k})$ and ${\widehat{p}}_{{D}_{1},{C}_{0}}=(\frac{2(k-1)}{2-k},\frac{4-3k}{2-k})$. The eigenvalues connected to these equilibria are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1}{C}_{0},{C}_{1}{C}_{0}}=0,\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1}{D}_{0},{D}_{1}{D}_{0}}=0,\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{0},{C}_{1}{D}_{0}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1}{C}_{0},{D}_{1}{C}_{0}}=\frac{1}{2}\frac{b(3{k}^{2}-7k+4)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1}{C}_{0},{C}_{1}{C}_{0}{D}_{1}}=\frac{1}{2}b(k-{p}_{{C}_{0}}),\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1}{D}_{0},{D}_{1}{D}_{0}{C}_{1}}=b(1-k),\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{D}_{0}}=0,\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{0},{C}_{1}{D}_{0}{D}_{1}}=0,\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1}{C}_{0},{D}_{1}{C}_{0}{C}_{1}}=\frac{1}{2}\frac{bk(1-k)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1}{C}_{0},{C}_{1}{C}_{0}{D}_{0}}=\frac{1}{2}b(k-{p}_{{C}_{0}})\hfill \\ & {\lambda}_{{D}_{1}{D}_{0},{D}_{1}{D}_{0}{C}_{0}}=b(1-k)\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{C}_{0}}=\frac{1}{4}\frac{b{k}^{2}}{(2-k)}\hfill \\ & {\lambda}_{{C}_{1}{D}_{0},{C}_{1}{D}_{0}{C}_{0}}=0\hfill \\ & {\lambda}_{{D}_{1}{C}_{0},{D}_{1}{C}_{0}{D}_{0}}=\frac{b(k-1)}{2-k}\hfill \end{array}$$

The only trimorphic equilibrium is the line of equilibria ${\widehat{p}}_{{C}_{1}{D}_{1}{D}_{0}}=(\frac{2(1-k)}{2-k},1-{p}_{{D}_{0}}-\frac{2(1-k)}{2-k},0,{p}_{{D}_{0}})$, for $0<{p}_{{D}_{0}}<1$. The eigenvalues are:
where the first eigenvalue is associated with the eigenvector along the line of equilibria while the second one is transversal to the line of equilibria, both in the same plane spanned by strategies $({C}_{1},{D}_{1},{D}_{0})$.

$${\lambda}_{{C}_{1}{D}_{1}{D}_{0},{C}_{1}{D}_{1}{D}_{0}}=0,\phantom{\rule{1.em}{0ex}}{\lambda}_{{C}_{1}{D}_{1}{D}_{0},{C}_{1}{D}_{1}{D}_{0}^{T}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}{\lambda}_{{C}_{1}{D}_{1}{D}_{0},{C}_{1}{D}_{1}{D}_{0}{C}_{0}}=0,$$

There are no four-morphic equilibria.

Above, we find that for $0<k<\frac{4}{3}$, the strategy ${C}_{0}$ can invade the initial non-choosy population given in Equation (10) (${D}_{0}$ can never selectively invade).

For $0<k<1$, the state space spanned by strategies ${C}_{1},{D}_{1},{C}_{0}$ contains no interior equilibria (i.e., equilibria where all strategies have strictly positive frequencies). The Poincare–Bendixon theorem thus ensures that no steady states (limit cycles nor any other attractors) exist in the interior state space, and thus, after an invasion of a rare strategy ${C}_{0}$, the dynamics must approach one of the equilibria on the boundary of the state space. For $0<k<1$, the only stable equilibria lie on the line ${\widehat{p}}_{{C}_{1},{C}_{0}}=({p}_{{C}_{1}},1-{p}_{{C}_{1}})$ for $0<{p}_{{C}_{0}}<k$, and so, after an invasion, strategy pair $({C}_{1},{C}_{0})$ substitutes $({C}_{1},{D}_{1})$ as the resident population. This state cannot be selectively invaded by any other strategy and is thus an evolutionary end-point (but, see the main text for the effect of neutral drift).

For $1<k<\frac{4}{3}$, the initial population is invaded and substituted by a pair $({D}_{1},{C}_{0})$ at ${\widehat{p}}_{{D}_{1},{C}_{0}}=(\frac{2(k-1)}{2-k},\frac{4-3k}{2-k})$. This state can always be invaded by ${D}_{0}$, after which, using the same argumentation as for $0<k<1$, the dynamics must approach the only stable boundary equilibria ${\widehat{p}}_{{D}_{1},{D}_{0}}=({p}_{{D}_{1}},1-{p}_{{D}_{1}})$ for $0<{p}_{{D}_{0}}\le 1$. This state cannot be selectively invaded by any other strategy and is, thus, an evolutionary end-point (see the main text for the effect of neutral drift).

For $\frac{4}{3}<k<2$, no rare strategy can selectively invade the initial state, and so, ${\widehat{p}}_{{D}_{1}}$ is the evolutionary end-point (see the main text for the effect of neutral drift).

The eigenvalues connected to the monomorphic equilibria are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{1}}=\frac{bk}{2},\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{C}_{1}}=0,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{0}}=\frac{1}{2}b(k-\epsilon ),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{0}}=b(1-\frac{3}{4}k-\frac{3}{4}\epsilon k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{D}_{1}}=\frac{1}{2}bk,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{D}_{1}}=0,\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{C}_{0}}=0,\hfill \\ & {\lambda}_{{D}_{1},{D}_{0}}=0\hfill \\ & {\lambda}_{{C}_{0},{D}_{0}}=\frac{1}{2}bk\hfill \\ & {\lambda}_{{D}_{0},{C}_{0}}=b(1-k).\hfill \end{array}$$

There are dimorphic equilibria ${\widehat{p}}_{{C}_{1},{C}_{0}}=({p}_{{C}_{1}},0,1-{p}_{{C}_{0}},0)$ and ${\widehat{p}}_{{D}_{1},{D}_{0}}=(0,{p}_{{D}_{1}},0,1-{p}_{{D}_{1}})$, where each point on the line, for all $0<{p}_{{C}_{1}},{p}_{{D}_{1}}<1$, is an equilibrium. In addition, there are four (isolated) dimorphic equilibria, ${\widehat{p}}_{{C}_{1},{D}_{1}}=(\frac{2(1-k)}{2-k},\frac{k}{2-k},0,0)$, ${\widehat{p}}_{{C}_{1},{D}_{0}},{\widehat{p}}_{{D}_{1},{C}_{0}}$ and ${\widehat{p}}_{{C}_{0},{D}_{0}}$, which have a too long expression to be written here. The eigenvalues connected to ${\widehat{p}}_{{C}_{1},{D}_{1}}$ are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{D}_{0}}=\frac{b\epsilon (k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{C}_{0}}=\frac{b{k}^{2}(1-\epsilon )}{4(2-k)},\phantom{\rule{1.em}{0ex}}\hfill \end{array}$$

The eigenvalues connected to the remaining equilibria ${\widehat{p}}_{{C}_{1},{D}_{0}},{\widehat{p}}_{{D}_{1},{C}_{0}}$ and ${\widehat{p}}_{{C}_{0},{D}_{0}}$ were checked numerically with $b=15,u=20,\epsilon =0.01,0.05,0.1$ and for a range of $0<k<2$. We checked that a perturbation away from these parameter values did not qualitatively alter the dynamics. The stability of the equilibria is indicated in Figure 2.

Trimorphic and four-morphic equilibria were investigated numerically for $b=15,u=20,\epsilon =0.01,0.05,0.1$ and for a range of $0<k<2$. We found only one unstable trimorphic equilibrium ${\widehat{p}}_{{C}_{1}{C}_{0}{D}_{0}}$ and no four-morphic equilibria.

Above, we find that the initial population given in Equation (10) can be invaded by ${C}_{0}$ for $0<k<\frac{4}{3+\epsilon}$, but it can never be invaded by ${D}_{0}$. For $\frac{4}{3+\epsilon}<k<2$, the population of ${D}_{1}$ is the end-point of evolution.

For $0<k<1$, the state space spanned by strategies ${C}_{1},{D}_{1},{C}_{0}$ contains no interior equilibria (analytical result). Using similar arguments as in the basic model (see above), we find that a strategy pair $({D}_{1},{C}_{0})$ at ${\widehat{p}}_{{D}_{1}{C}_{0}}$ substitutes the initial state as the resident population. Since ${\widehat{p}}_{{D}_{1}{C}_{0}}$ is the only strictly positive equilibrium on that boundary of the state-space, strategy pair $({D}_{1},{C}_{0})$ substitutes the initial population for $1<k<\frac{4}{3+\epsilon}$, as well. Numerically, using $b=15,u=20,\epsilon =0.01,0.05,0.1$, we find that for $0<k<1$, the population at ${\widehat{p}}_{{D}_{1}{C}_{0}}$ may be invaded by ${D}_{0}$ and substituted by $({C}_{0},{D}_{0})$ at ${\widehat{p}}_{{C}_{0}{D}_{0}}$. This is an ESS and, thus, the end point of evolution. For $1<k<\frac{4}{3+\epsilon}$, we find that ${D}_{0}$ can invade ${\widehat{p}}_{{D}_{1}{C}_{0}}$, after which a mix of defectors $({D}_{0},{D}_{1})$, for $0<{p}_{{D}_{0}}\le 1$, becomes the end-point of evolution (see the main text for the effects of neutral drift).

The eigenvalues connected to the monomorphic equilibria are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{1}}=\frac{1}{2}bk,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{C}_{1}}=0,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{C}_{1}}=b(1-k)+a,\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{0}}=\frac{1}{2}bk,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{0}}=b(1-\frac{3}{4}k)-a,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{D}_{1}}=\frac{1}{2}b(k-1)-a,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{D}_{1}}=0,\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{C}_{0}}=0,\hfill \\ & {\lambda}_{{D}_{1},{D}_{0}}=-a\hfill \\ & {\lambda}_{{C}_{0},{D}_{0}}=\frac{1}{2}b(k-1)-a\hfill \\ & {\lambda}_{{D}_{0},{C}_{0}}=b(1-k).\hfill \end{array}$$

The eigenvalues connected to the dimorphic equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}}$ are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{D}_{0}}=\frac{ak}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{C}_{0}}=\frac{k}{4(2-k)}\left(bk-4a\right)\phantom{\rule{1.em}{0ex}}\hfill \end{array}$$

The existence and stability of all of the other dimorphic equilibria were studied numerically using $b=15,u=20,a=1$ and a range $0<k<2$. The results are summarized in Figure 3. We do not draw the arrows if the equilibrium is not a best strategy solution or if it does not affect the invasion-substitution sequence.

We studied the existence and stability of trimorphic and four-morphic equilibria numerically, using values $b=15,u=20,a=1$ and a range $0<k<2$. We found only unstable trimorphic equilibria and no four-morphic equilibria. Since trimorphic equilibria are not the best strategy solutions or they do not affect the invasion-substitution sequence, they are not drawn in Figure 3.

Using the eigenvalues calculated above, we find that the initial population given in Equation (10) can be invaded by ${C}_{0}$ for $4\frac{a}{b}<k<\frac{4}{3}(1-\frac{a}{b})$, but it can never be invaded by ${D}_{0}$. For $0<k<4\frac{a}{b}$ and for $\frac{4}{3}(1-\frac{a}{b})<k<2$, the initial population given in Equation (10) is the end-point of evolution.

Further invasion and substitutions were found numerically using values $b=15,u=20,a=1$ and a range $0<k<2$. Figure 3 and the main text present a summary of all of the qualitatively different best strategy solution and invasion-substitution sequences.

The eigenvalues connected to the monomorphic equilibria are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{1}}=\frac{bk}{2},\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{1}}=b(1-k),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{C}_{1}}=a\epsilon (1-\epsilon ),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{C}_{1}}=b(1-k)+a(1-\epsilon -{\epsilon}^{2}),\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{D}_{0}}=\frac{1}{2}b(k-\epsilon )-a\epsilon ,\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{1},{C}_{0}}=b(1-\frac{3}{4}k-\frac{3}{4}\epsilon k)-a(1-\epsilon ),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{0},{D}_{1}}=\frac{1}{2}bk-a(1-3\epsilon +{\epsilon}^{2}),\phantom{\rule{2.em}{0ex}}\hfill \\ & {\lambda}_{{D}_{0},{D}_{1}}=a\epsilon (1-\epsilon ),\phantom{\rule{2.em}{0ex}}\hfill \end{array}\begin{array}{cc}& {\lambda}_{{C}_{1},{C}_{0}}=-a\epsilon ,\hfill \\ & {\lambda}_{{D}_{1},{D}_{0}}=-a(1-\epsilon )\hfill \\ & {\lambda}_{{C}_{0},{D}_{0}}=\frac{1}{2}bk-a(1-3\epsilon +2{\epsilon}^{2})\hfill \\ & {\lambda}_{{D}_{0},{C}_{0}}=b(1-k)+a\epsilon (1-2\epsilon ).\hfill \end{array}$$

The eigenvalues connected to the dimorphic equilibrium ${\widehat{p}}_{{C}_{1}{D}_{1}}$ are:

$$\begin{array}{cc}& {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}}=\frac{bk(k-1)}{2-k},\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{D}_{0}}=\frac{1}{2-k}\left[b\epsilon (k-1)-a(k+2\epsilon -3\epsilon k)\right],\phantom{\rule{1.em}{0ex}}\hfill \\ & {\lambda}_{{C}_{1}{D}_{1},{C}_{1}{D}_{1}{C}_{0}}=\frac{1}{4(2-k)}\left[b{k}^{2}(1-\epsilon )-4a(k+2\epsilon +3\epsilon k)\right]\phantom{\rule{1.em}{0ex}}\hfill \end{array}$$

The existence and stability of all other dimorphic, trimorphic and four-morphic equilibria were investigated numerically. We used the parameter values $b=15,a=1,u=20,\epsilon =0.01$ and $0<k<2$.

- Gale, D.; Shapley, L.S. College admissions and the stability of marriage. Am. Math. Mon.
**1962**, 69, 9–15. [Google Scholar] - Knuth, D.E. Mariages Stables et Leurs Relations Avec D’autres Problèmes Combinatoires; Les Presses de l’Université de Montréal: Montreal, QC, Canada, 1976. [Google Scholar]
- Roth, A.E. Incentive compatibility in a market with indivisible goods. Econ. Lett.
**1982**, 9, 127–132. [Google Scholar] [CrossRef] - Roth, A.E. The evolution of the labor market for medical interns and residents: A case study in game theory. J. Political Econ.
**1984**, 92, 991–1016. [Google Scholar] [CrossRef] - Roth, E.A.; Sotomayor, M.A.O. Two-Sided Matching: A Study in Game-Theoretic Modeling and Analysis; Cambridge University Press: Cambridge, UK, 1990. [Google Scholar]
- Kojima, F.; Pathak, P.A. Incentives and stability in large two-sided matching markets. Am. Econ. Rev.
**2009**, 99, 608–627. [Google Scholar] [CrossRef] - Ackermann, H.; Goldberg, P.W.; Mirrokni, V.S.; Röglin, H.; Vöcking, B. Uncoordinated two-sided matching markets. Siam J. Comput.
**2011**, 40, 92–106. [Google Scholar] [CrossRef] - Eriksson, K.; Häggström, O. Instability of matchings in decentralized markets with various preference structures. Int. J. Game Theory
**2008**, 36, 409–442. [Google Scholar] [CrossRef] - Kalick, S.M.; Hamilton, T.E. The matching hypothesis reexamined. J. Personal. Soc. Psychol.
**1986**, 51, 673–682. [Google Scholar] [CrossRef] - Johnstone, R.A. The tactics of mutual mate choice and competitive search. Behav. Ecol. Sociobiol
**1997**, 40, 51–59. [Google Scholar] [CrossRef] - Alpern, S.; Reyniers, D. Strategic mating with common preferences. J. Theor. Biol.
**2005**, 237, 337–354. [Google Scholar] [CrossRef] [PubMed] - Alpern, S.; Katranzi, I. Equilibria of two-sided matching games with common preferences. Eur. J. Oper. Res.
**2009**, 196, 1214–1222. [Google Scholar] [CrossRef] - Ramsey, D.M. Partnership formation based on multiple traits. Eur. J. Oper. Res.
**2012**, 216, 624–637. [Google Scholar] [CrossRef] - Alpern, S.; Katranzi, I.; Ramsey, D. Equilibrium population dynamics when mating is by mutual choice based on age. Theor. Poput. Biol.
**2014**, 94, 63–72. [Google Scholar] [CrossRef] [PubMed] - McNamara, J.M.; Collins, E.J. The job search problem as an employer-candidate game. J. Appl. Prob.
**1990**, 27, 815–827. [Google Scholar] [CrossRef] - Bloch, F.; Ryder, H. Two-sided search, marriages, and matchmakers. Internat. Econ. Rev.
**2000**, 41, 93–114. [Google Scholar] [CrossRef] - Shimer, R.; Smith, L. Assortative matching and search. Econometrica
**2000**, 68, 343–369. [Google Scholar] [CrossRef] - Eriksson, K.; Sjöstrand, J.; Strimling, P. Optimal expected rank in a two-sided secretary problem. Oper. Res.
**2007**, 55, 921–931. [Google Scholar] [CrossRef] - Ward, S.A. Optimal habitat selection in time-limited dispersers. Am. Nat.
**1987**, 129, 568–579. [Google Scholar] [CrossRef] - Real, L.A. Search theory and mate choice. I. Models of single-sex discrimination. Am. Nat.
**1990**, 136, 376–404. [Google Scholar] [CrossRef] - Backwell, P.R.Y.; Passmore, N.I. Time constraints and multiple choice criteria in the sampling behavior and mate choice of the fiddler crab, Uca annulipes. Behav. Ecol. Soc.
**1996**, 38, 407–416. [Google Scholar] [CrossRef] - Gray, D. Intrinsic factors affecting female choice in house crickets: time of first reproduction, female age, nutritional condition, body size, and size-relative reproductive investment. J. Insect Behav.
**1999**, 12, 691–700. [Google Scholar] - Tinghitella, R.M.; Weigel, E.G.; Head, M.; Boughman, J.W. Flexible mate choice when mates are rare and time is short. Ecol. Evol.
**2013**, 3, 2820–2831. [Google Scholar] [CrossRef] - Priklopil, T.; Kisdi, E.; Gyllenberg, M. Evolutionarily stable mating decisions for sequentially searching females and the stability of reproductive isolation by assortative mating. Evolution
**2015**, 69, 1015–1026. [Google Scholar] [CrossRef] [PubMed] - Nöe, R.; Hammerstein, P. Biological markets: Supply and demand determine the effect of partner choice in cooperation, mutualism and mating. Behav. Ecol. Sociobiol.
**1994**, 35, 1–11. [Google Scholar] [CrossRef] - Batali, J.; Kitcher, P. Evolution of altruism in optional and compulsory games. J. Theor. Biol.
**1995**, 175, 161–171. [Google Scholar] [CrossRef] [PubMed] - Sherratt, T.; Roberts, G. The Evolution of generosity and choosiness in cooperative exchanges. J. Theor. Biol.
**1998**, 193, 167–177. [Google Scholar] [CrossRef] - Hauert, C.; De Monte, S.; Hofbauer, J.; Sigmund, K. Volunteering as red queen mechanism for cooperation in public goods games. Science
**2002**, 296, 1129–1132. [Google Scholar] [CrossRef] [PubMed] - Hruschka, D.J.; Henrich, J. Friendship, cliquishness, and the emergence of cooperation. J. Theor. Biol.
**2006**, 239, 1–15. [Google Scholar] [CrossRef] [PubMed] - Fu, F.; Hauert, C.; Nowak, M.A.; Wang, L. Reputation-based partner choice promotes cooperation in social networks. Phys. Rev.
**2008**, E78, 026117. [Google Scholar] [CrossRef] - McNamara, J.M.; Barta, Z.; Fromhage, L.; Houston, A.I. The coevolution of choosiness and cooperation. Nature
**2008**, 451, 189–192. [Google Scholar] [CrossRef] [PubMed] - Ghang, W.; Nowak, M.A. Indirect reciprocity with optional interactions. J. Theor. Biol.
**2015**, 21, 1–11. [Google Scholar] [CrossRef] [PubMed] - Sigmund, K. Calculus of Selfishness; Princeton University Press: Princeton, NJ, USA, 2010. [Google Scholar]

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).