Evolution of “Pay-It-Forward” in the Presence of the Temptation to Free-Ride

Abstract

“Paying it forward” is a behavior in which people help someone else because they were helped in the past. Although experimental evidence exists that indicates that real human beings often “pay-it-forward” even in the face of free-rider risks, the theoretical basis for the evolution of this behavior remains unclear. In this paper, we propose a game-theoretical model that explains how pay-it-forward behavior can evolve despite the temptation to free-ride. By assuming that human beings exhibit cognitive distortions, as predicted by prospect theory, and that free-riding is punished with a tiny probability, we demonstrate that pay-it-forward, alongside unconditional altruistic behavior, can evolve and effectively deter free-riding behavior.

1. Introduction

People often “pay-it-forward” by passing on benefits they have received from someone in the past to others in the future. Experimental and observational studies have provided evidence that people indeed engage in decisions reflective of pay-it-forward behavior [1,2]. For instance, in [1], game experiments conducted in a laboratory setting revealed that the altruistic behavior of one game player spread to other players within the social network. This propagation of altruistic behaviors from one player to another encourages cooperation among players.

The presence of pay-it-forward behaviors in the real world presents a significant challenge to game theory, especially within the context of evolutionary game theory. Essentially, when a player chooses to pass on benefits received from someone in the past to others, they do not gain any immediate personal profits. Consequently, individuals who opt not to pay-it-forward (i.e., those who engage in “free-riding”) often find themselves at a greater advantage compared to those who participate in pay-it-forward activities (“pay-it-forwarders”). Without introducing an additional mechanism, natural selection or social learning processes are expected to favor non-participants [3]. Thus, identifying a mechanism that discourages free-riding behavior has become a crucial theoretical challenge.

Unconditional altruistic behavior, in which individuals help others without expecting anything in return, often leads to what is known as the social dilemma, or free-rider problem, as documented in the literature [4,5,6,7,8,9,10]. Pay-it-forward practices, however, represent a form of conditional altruistic behavior where individuals help others only if they themselves have received help in the past. This fact highlights that conditional altruistic behavior shares the same underlying structure as the social dilemma, characterized by the following: (1) individuals in a society have the choice to either engage in pay-it-forward practices or to free-ride, (2) a society comprised mainly of pay-it-forwarders is generally more beneficial than one filled with free-riders, and (3), on an individual level, free-riders tend to fare better than pay-it-forwarders. This is because pay-it-forwarders distribute the benefits they have received to others, whereas free-riders keep any advantages for themselves. In situations presenting a social dilemma, opting for free-riding becomes more appealing for individuals irrespective of the choices made by others.

In this paper, we show that pay-it-forward can be sustainably maintained if individuals adhere to what is known as prospect theory (PT) and if punishment for not helping others is applied, albeit infrequently. This stands in contrast to scenarios where individuals are so rational that they follow the (linear) expected utility theory (EUT), under which pay-it-forward practices do not persist. In our previous research, we established that prospect theory facilitates cooperation through peer punishment [11], whereas EUT does not lead to the same outcome when free-riders are only occasionally punished. We hypothesize that a similar rationale applies to the evolution of cooperation through pay-it-forward mechanisms. Investigating this hypothesis is the main focus of the current paper.

Prospect theory is widely recognized as one of the most successful frameworks for describing the cognitive biases inherent in human decision-making [12,13,14]. Within this theory, distortions in the perception of probabilities and the evaluation of outcomes are represented through a weighting probability function and a value function, respectively. Both functions are characterized by their non-linearity and asymmetry, capturing the essence of human irrationality. This theoretical approach highlights how individuals often deviate from rational decision-making models, particularly in the context of risk and uncertainty, by assigning disproportionate weights to certain outcomes and probabilities.

Most studies on the evolution of cooperation in evolutionary game theory have traditionally been conducted under the assumption that individuals adhere to EUT [5]. EUT posits that individuals are perfectly rational, making their calculations regarding potential benefits, or so-called expected payoffs, with precision. Contrary to this assumption, experimental research on both behavioral economics and experimental economics has consistently demonstrated that real humans do not conform to EUT [15,16]. In reality, humans are incapable of estimating probabilities and values without bias. The capacity to accurately calculate probabilities and values, devoid of any distortions, underpins the concept of linearly calculated expected payoffs [17,18,19,20,21,22]. However, the payoffs, as perceived by real humans, are subjective, and this subjectivity is a critical factor in the actual decision-making process. This discrepancy between the theoretical assumptions of EUT and the practical observations under PT highlights the complexities and nuances in understanding human behavior in evolutionary game-theoretic contexts.

In the subsequent sections of this paper, we demonstrate that, when individuals exhibit the level of irrationality encapsulated by PT, pay-it-forward behaviors emerge, and cooperation persists even within parameter regions where conditional altruistic actions cannot be sustained under the rationality assumptions of EUT. Specifically, the risk associated with free-riding behaviors is mitigated by the cognitive biases and decision-making patterns described by PT. Furthermore, the strategy of pay-it-forward behaviors has the potential to overturn unconditional altruistic behavior, highlighting a critical distinction between paying-it-forward and unconditional cooperation, despite their superficial similarities.

Following a brief review of the relevant literature in the next section, we will outline our model before presenting our findings. The paper concludes with a section dedicated to our conclusions, summarizing the implications of our research on the understanding of cooperative behaviors and the influence of cognitive biases on the interactions among individuals.

2. Brief Literature Review

There are several recent papers that have combined evolutionary game theory with prospect theory, aiming to derive practical implications for addressing social issues. For instance, the study referenced as [23] considers the irrationality of players, as described by prospect theory, and constructs an evolutionary game-theoretical model to explore the decision-making process of enterprises when choosing between innovation and imitation. A paper by Sun, Gao, and Li published in 2022 [24] analyzes interactions between enterprises and governments, particularly in the context of establishing carbon emission regulations. Meanwhile, the paper referenced as [25] focuses on the decision-making processes of polluting enterprises and their interactions with government bodies. A more recent study [26] investigates the recycling of power batteries within the electric vehicle industry, proposing a game model that involves three distinct types of players (governments, manufacturers, and recycling companies). Each of these papers develops an evolutionary game model that incorporates aspects of prospect theory.

The evolution of cooperation has been extensively studied within the field of evolutionary game theory, as highlighted in [27]. A recent theoretical study [28] attempts to elucidate the evolution of cooperative (and coordinated) behavior by incorporating the effects of prospect theory into a model. This is achieved through the lens of a coordination game known as the stag-hunt game.

In this paper, we aim to show a path to the evolution of cooperation through pay-it-forward behaviors using a prisoner’s-dilemma-type game called the “donation game”. The mechanism to sustain cooperation via pay-it-forward is a type of so-called indirect reciprocity mechanism. Indirect reciprocity is one of the five typical mechanisms to sustain cooperation [29]. In indirect reciprocity, beneficial acts are returned not by the recipient, as in the direct reciprocity mechanism, but by third parties. In the literature [6], indirect reciprocity is classified into the following two categories: downstream reciprocity and upstream reciprocity. Pay-it-forward is categorized as upstream reciprocity, which represents the ”I-help-you-because-someone-helped-me” attitude. Downstream reciprocity represents the “I-help-you-because-you-helped-someone” attitude, which is the mainstream in the study of indirect reciprocity.

While the literature on the evolution of cooperation via downstream reciprocity is ample [30,31,32,33,34,35,36,37,38], evolutionary theory has yet to explain the effectiveness of upstream reciprocity in regard to the evolution of cooperation. A well-cited paper on upstream reciprocity in the context of evolutionary game theory was published in 2007 [3], but it concludes that upstream reciprocity can evolve as a “by-product of direct reciprocity”, i.e., upstream reciprocity works if it is combined with another effective mechanism, direct reciprocity. In [39], the evolution of upstream reciprocity was explained in a complex network, and in [40], it was shown that cooperation can evolve by upstream reciprocity only in small interaction groups. According to a thorough review on the theoretical research of indirect reciprocity published recently [41], although there is some experimentation on upstream reciprocity, explaining it via theoretical viewpoints is still challenging.

3. Model

3.1. Game

To maintain simplicity and facilitate analysis of the model, in line with several studies in the field of the evolution of cooperation [11,42,43,44], we consider a large, well-mixed population of individuals (or players). Two players are chosen at random from the population from time to time. Let us call the chosen players A and B. Players A and B engage in the following “donation game” (or “giving game”) [6]: each player decides whether to help the opponent by paying a cost $c$. If Player A (B) chooses to help B (A), the opponent receives a benefit $b>c$; otherwise A (or B) does not pay anything and B (A) obtains nothing. Every player in the population is selected as a game player (infinitely) many times and experiences decision-making processes. From here on, we denote the action “cooperate with its opponent (or just cooperate)” by “C” and “defect its opponent (or just defect)” by “D”.

Table 1. Payoffs player A obtains in donation game.

Player B’s Option Player A’s Option	Cooperate (C)	Defect (D)
Cooperate (C)	b − c	−c
Defect (D)	b	0

shows the payoffs Player A obtains when playing the donation game with Player B. It consists of four possible outcomes, as both players have two options each. If both players choose C, Player A gains a benefit of $b-c$. If Player A chooses C while Player B opts for D, Player A incurs a loss of $c$. Conversely, if Player A selects D and Player B chooses C, then Player A receives a benefit of $b$. If both players select D, Player A receives no benefit (and neither does Player B).

3.2. Game, Strategies and Payoff Matrix

We consider the following three strategies: AC, AD, and S [42]. Strategy AC prescribes to help others with probability $1-\epsilon$. Due to the shortage of resources, AC-players do not give help with probability $\epsilon$, even if they would like to. We call $\epsilon$ an implementation error and set its value to 0.05. Players using strategy AD are free-riders and never help others. We call those players AD-players. Strategy S players correspond to pay-it-forward, which prescribes to choosing C (with probability $1-\epsilon$) if they were helped in the last game. If they were not helped in the last game, S-players choose D in the current game. We also refer to the strategies by numbers: AC = strategy 1, AD = strategy 2, and S = strategy 3.

3.3. Payoff Matrix

We examine a type of punishment known as pool punishment [45,46,47,48,49,50,51]. In this framework, a punishment institution exists in the population and executes punishment to those who chose D in the donation game. Players in the population pay a cost $d$, which will be used to maintain the punishment institution.

In this paper, we assume that the institution executes punishment with only a small probability to account for situations where the institution functions insufficiently. From the viewpoints of players, after the two selected players have played the donation game, they have a small chance of being punished by the external institution if they chose D in the donation game. The probability of being punished is $\delta$, which is set to be 0.2 in the following numerical simulations.

We assume that AD-players, who never give help to others, do not pay $d$ (See Section 3.5 for more details). In this sense, AD-players are not only first-order free-riders but also second-order free-riders. Here, first-order free-riders mean that they do not engage in cooperative activities, while second-order free-riders are defined as those who do not contribute to punishment activities. AD-players are assumed to be both first-order and second-order free-riders at the same time. This is a harsher assumption for the evolution of cooperation than the assumption that AD-players also pay the cost to maintain the punishment institution, as other types of players are at more of a disadvantage if AD-players do not incur the punishment cost.

The strength of punishment is denoted by $s$. Thus, if a player chooses D in the donation game, and if punishment is executed by the institution (which occurs with probability $\delta$), the player suffers a fine, $s$. If punishment is not executed, nothing is imposed on the player. As a result, we obtain

Table 2. Payoff matrix $M$ (payoffs Player A obtains in total).

Outcome Player A’s Option	Cooperate Not-Punish (CN)	Cooperate Punish (CP)	Defect Not-Punish (DN)	Defect Punish (DP)
Cooperate (C)	$b-c$	0	$-c$	0
Defect (D)	$b$	$b-s$	0	$-s$

as the payoff matrix studied in this paper. This matrix comprises eight elements, as Player A has two options (C or D), and, for each option, there are four possibilities depending on whether the opponent, Player B, chooses C or D and whether the institution executes punishment or not.

We call this payoff matrix $M$. In Table 2, for example, “CP” means that Player B helps Player A and the institution executes punishment. In this case, if A does not help B, A obtains benefit $b$ but suffers $s$ at the same time. The total payoff in this case is $b-s(=M_{22})$.

It is worth noting that if Player A helps Player B, A will never be punished. Consequently, the element of the payoff matrix corresponding to this scenario ($M_{12}$) is undefined. For the same reason, $M_{14}$ is also undefined, as Player A’s action of choosing C will never result in punishment. We formally put value “0” for the elements $M_{12}$ and $M_{14}$ in the table. However, these values are never used, as these situations never occur. The cost of implementing punishment is not included in the payoff matrix, as it does not result from interactions with other players. The cost for punishment is introduced after estimating the expected payoffs on the basis of interactions with other players (See Section 3.5).

3.4. Strategy Switching

Players occasionally switch their strategies by social learning. As a result of the switching process, the time evolution of the proportion of strategy $i$ ($i\in \{\mathrm{1,2},3\}$), denoted by $x_i$, is governed by the so-called replicator equation

$${\dot{x}}_i=x_i\left(P_i-\overline{P}\right),$$

where $P_i$ is the expected payoff of strategy $i$ and $\overline{P}={\sum }_{i=1}^3{x}_i{P}_i$ is the expected payoff of the whole population. Since $x_i$ is the proportion of strategy $i$, ${\sum }_{i=1}^3{x}_i=1$ holds.

By numerically integrating the replicator equation mentioned above, we can analyze the stability of each strategy. In the subsequent subsection, we will describe how the expected payoff $P_i$ is calculated.

3.5. Probability Distribution on $M$ and Expected Payoff $P_i$

3.5.1. Linear Expected Utility Theory

Expected payoff $P_i$ reflects the long-term payoff of players using strategy $i$. It can be determined by calculating the probability that each outcome for payoff matrix $M_{mn}$ occurs. The probabilities are dependent on strategies. By $B_i$, we denote the probability distribution on $M$ of players using strategy $i$.

To determine probability distributions, it is crucial to first know the probability of whether players following a certain strategy will offer help to others. For those adhering to the AC strategy, this probability is $1-\mathsf{\epsilon }$, while for AD, it is 0. To ascertain the probability that S-players will offer help, we define $p$ as the frequency at which players receive help from a random player within the population. This frequency, $p$, can also be interpreted as the probability of a player being given help by a random participant in the game.

It is important to note that the act of receiving help in the game does not rely on the strategy used by the recipient. Therefore, $p$ is strategy-independent and represents the chance that any given player is helped in a game. Consequently, the probability that S-players will choose action C is represented by $(1-\epsilon )p$.

The frequency $p$ evolves over time as individuals engage in the game. We assume that the dynamics of $p$ occur at a much faster pace than changes in strategy. In essence, players experience various interactions an infinite number of times before altering their strategies. This approach keeps the variables $x_i$ constant while allowing $p$ to vary, a premise similarly assumed in Sections 4.2 and 4.3 of [6].

As demonstrated in Appendix A, $p$ reaches an equilibrium defined by the following equation:

$$p=\left(1-\epsilon \right)x_1+\left(1-\epsilon \right)p{x}_3.$$

Here, the first term on the right-hand side represents the probability of encountering an AC-player ($x_1$), who then opts for cooperation (C) with probability $1-\epsilon$. The second term accounts for the likelihood of meeting an S-player ($x_3$) who, having previously been given help, chooses C with a probability of $(1-\epsilon )p$.

By solving this equilibrium equation, we find $p$ to be:

$$p=\frac{(1-\epsilon )x_1}{1-(1-\epsilon )x_3}.$$

Since probability distributions on $M$ depend on strategies, the expected payoff is dependent on strategies.

Table 3. Probability distributions for strategy 1 (a), strategy 2 (b), and strategy 3 (c).

(a) ${\mathit{B}}_{\mathbf{1}}$
Outcome Player A’s option	Cooperate Not-Punish (CN)	Cooperate Punish (CP)	Defect Not-Punish (DN)	Defect Punish (DP)
Cooperate (C)	$\overline{\epsilon }p$	$0$	$\overline{\epsilon }\overline{p}$	$0$
Defect (D)	$\epsilon \overline{\delta }p$	$\epsilon \delta p$	$\epsilon \overline{\delta }\overline{p}$	$\epsilon \delta \overline{p}$
(b) ${\mathit{B}}_{\mathbf{2}}$
Outcome Player A’s option	Cooperate Not-Punish (CN)	Cooperate Punish (CP)	Defect Not-Punish (DN)	Defect Punish (DP)
Cooperate (C)	0	0	0	0
Defect (D)	$\overline{\delta }p$	$\delta p$	$\overline{\delta }\overline{p}$	$\delta \overline{p}$
(c) ${\mathit{B}}_{\mathbf{3}}$
Outcome Player A’s option	Cooperate Not-Punish (CN)	Cooperate Punish (CP)	Defect Not-Punish (DN)	Defect Punish (DP)
Cooperate (C)	$\overline{\epsilon }{p}^2$	$0$	$\overline{\epsilon }\overline{p}p$	$0$
Defect (D)	$(1-\overline{\epsilon }p)\overline{\delta }p$	$(1-\overline{\epsilon }p)\delta p$	$(1-\overline{\epsilon }p)\overline{\delta }\overline{p}$	$(1-\overline{\epsilon }p)\delta \overline{p}$

(a)–(c) show probability distributions $B_i$ with parameters ($\epsilon ,\delta ,p$). In the tables, $\overline{\epsilon }=1-\epsilon$, $\overline{\delta }=1-\delta$, $\overline{p}=1-p$.

For example, ${{(B}_1)}_{22}$ is the product of the following three probabilities: the probability that an AC-player is given help by the opponent player of a game ($p$), the probability that the AC-player chooses D ($\epsilon$), and the probability that this behavior (i.e., choosing D) is punished by the external institution ($\delta$). Note that ${\left(B_i\right)}_{12}={\left(B_i\right)}_{14}=0$ for any $i$.

With these probability distributions on $M$, the expected payoff of players using strategy $i$ through interactions with other players is given by

$$Q_i=\sum _{m=1}^2\sum _{n=1}^4{\left(M\right)}_{mn}{\left(B_i\right)}_{mn}.$$

For example, $Q_1=\left(b-c\right)\overline{\epsilon }p-c\overline{\epsilon }\overline{p}+b\epsilon \overline{\delta }p+\left(b-s\right)\epsilon \delta p-s\epsilon \delta \overline{p}$.

Considering that AD-players do not pay any cost to maintain the external punishment institution, we find the final form of the expected payoff of players following strategy $i$:

$$P_i=Q_i-d,$$

for $i=\mathrm{1,3}$ and

$$P_i=Q_i,$$

for $i=2$.

Here, $d$ is the cost to maintain the punishment institution. AD-players are at more of an advantage than AC-players and S-players by the value of $d$.

3.5.2. Prospect Theory

The way we include prospect theory in our evolutionary game model is essentially the same as in our previous paper [11]. Thus, we apply a value function and weighting function to each element of the payoff matrix $M$ and to the probability distribution $B_i$, respectively:

$$\begin{array}{c}{\left(V\right)}_{mn}=v\left({\left(M\right)}_{mn}\right),\\ {\left({\mathrm{W}}_i\right)}_{mn}=w\left({\left(B_i\right)}_{mn}\right),\end{array}$$

where $v$ and $w$ are the value function and the weighting function, respectively. In Appendix B, we describe the appearance of the functions.

The subjectively distorted payoff matrix $V$ is independent of strategies. It is well-known that the nonlinearity of the value function means that the baseline of the payoff matrix, or the reference point, significantly influences the results, differing from EUT. Specifically, within the EUT framework, adding the same constant to all elements of the payoff matrix $M$ does not alter the payoff structure. In other words, the relative superiority or inferiority of strategies remains unchanged, even if each player consistently receives the same payoff. However, under PT, adding the same constant (for instance, by assuming each player obtains a constant payoff in playing a game) can affect the outcomes. We assume the payoff obtained from the outcome (2,3) in $M$, which is zero, serves as the so-called reference point, with no additional constants added.

With these subjective probabilities and payoffs, the expected payoff is calculated as

$$P_i=\sum _{m=1}^2\sum _{n=1}^4{\left(V\right)}_{mn}{\left(W_i\right)}_{mn}+v\left(-d\right),$$

for $i=\mathrm{1,3}$ and

$$P_i=\sum _{m=1}^2\sum _{n=1}^4{\left(V\right)}_{mn}{\left(W_i\right)}_{mn},$$

for $i=2$.

Note that we apply only the value function, not the weighting function, to the last term ($-d$: the loss of payoff by paying a cost for punishment) for the following reason: it does not represent a loss suffered through interaction in a game and is therefore not a stochastic event (hence, the weighting function plays no role). By applying the value function to $d$, the perceived impact of the cost for maintaining the punishment institution is regarded as more significant than its actual impact. We also note that the strategy change is described by the ordinary replicator dynamics mentioned above.

4. Results

4.1. Analysis in Case of EUT

In the context of expected utility theory (EUT), the replicator dynamics can be thoroughly analyzed due to the simplicity of the expected payoffs. The expected payoffs for AC-players, AD-players, and S-players are, respectively represented as

$$P_1=pb-\left(1-\epsilon \right)c-\epsilon \delta s-d,$$

$$P_2=pb-\delta s,$$

$$P_3=pb-\left(1-\epsilon \right)pc-\left[1-\left(1-\epsilon \right)p\right]\delta s-d.$$

From these equations, it can be demonstrated that AD-players become extinct through the replicator dynamics, while AC-players evolve and are sustained, provided the punishment is effective or substantial enough (that is, if the parameters $s$ or $\delta$ surpass certain threshold values).

Specifically, $P_1>P_2$ suggests that $\delta s>c+d/(1-\epsilon )$, while $P_1>P_3$ implies $\delta s>c$. If these conditions are met, the AC strategy emerges as the unique evolutionarily stable strategy (ESS), leading all players to eventually adopt it, resulting in $x_1=1$. Conversely, when $P_1<P_2$, the AD strategy becomes the unique ESS.

The S strategy can never be an ESS because a population consisting solely of S-players is vulnerable to invasion by an AD-player. Indeed, at the point where $x_3=1$, which implies $p=0$, we find that $P_3=-\delta s-d<-\delta s=P_2$. This indicates the inability of the S strategy to sustain itself as a stable strategy in the population if players follow EUT.

In the following numerical investigations, we fix parameter values as $b=5.0,c=1.0,\epsilon =0.05,d=0.05.$ With these values, the conditions $P_1>P_2$ and $P_1>P_3$ hold if $\delta s>20/19\approx 1.05$. Figure 1a–c display trajectories in the state space obtained by numerically integrating the replicator dynamics with $\delta =0.2$ and with $s=5.6,5.1,4.6$, respectively.

The replicator dynamics analyzed to produce the figure are given by the following coupled equation system:

$$\frac{d{x}_1}{dt}=x_1\left(P_1-\sum _{i=1}^3{x}_i{P}_i\right),$$

$$\frac{d{x}_2}{dt}=x_2\left(P_2-\sum _{i=1}^3{x}_i{P}_i\right),$$

$$\frac{d{x}_3}{dt}=x_3\left(P_3-\sum _{i=1}^3{x}_i{P}_i\right).$$

The initial conditions are set randomly. The state space is represented as the simplex $\left\{\left(x_1,x_2,x_3\right)\right|0\le x_1\le 1,0\le x_2\le 1,0\le x_3\le 1,x_1+x_2+x_3=1\}$, visualized as an equilateral triangle. The vertex AC corresponds to $\left(x_1,x_2,x_3\right)=(\mathrm{1,0},0)$, AD to $(0,\mathrm{1,0})$ and S to $(0,\mathrm{0,1})$. The state of the population $(x_1,x_2,x_3)$ evolves within this space according to the replicator equation. The figures also illustrate vector fields generated by the replicator dynamics.

If $s$ is sufficiently large, for example, 5.6, all trajectories converge to the vertex AC, as predicted by the above analysis (see Figure 1a). Consequently, all players ultimately adopt strategy AC, consistently aiming to help others.

With lower values of $s$, as depicted in Figure 1b,c, all trajectories converge towards the AD strategy. At these parameter values, rational individuals opt for the AD strategy because the punishment executed by the institution is insufficiently severe (i.e., $s$ is small) and the probability of being punished is too low (i.e., $\delta$ is small). Consequently, the punitive institution fails to function effectively in this scenario.

A notable difference between Figure 1b ($s=5.1$) and Figure 1c ($s=4.6$) is observed in the vector directions on the edge between AC and S. In the former scenario, AC is able to invade S, whereas, in the latter, S invades AC. However, it is important to note that S can never achieve evolutionary stability as it is susceptible to invasion by AD. Consequently, we can deduce that the pay-it-forward strategy, S, does not significantly contribute to the evolution of cooperation with the institutional punishment within the EUT framework.

4.2. Numerical Results in Case of PT

Figure 2a–c illustrate the trajectories and vector fields when prospect theory is applied, using the same parameter values as in Figure 1. We highlight the following two observations: (1) Figure 2b demonstrates that, even with parameter values for which AD is the unique evolutionarily stable strategy under EUT, cooperation evolves when players adhere to PT; that is, AC is the ESS. (2) In Figure 2c, a new stable fixed point, called T, emerges on the edge between S and AC. This fixed point represents a mixed state mainly comprising S-players, with AC-players also being present. The system exhibits bi-stability, as some trajectories converge towards the vertex AD, depending on their initial conditions.

In the case of Figure 1 and Figure 2c in particular, the AD strategy has several advantages. First, the condition $b>s$ indicates that the punishment is not severe enough to offset the benefits gained from free-riding. In addition, with $\delta$ being relatively small, choosing the D strategy entails an 80% chance of avoiding punishment. As such, AD players face a low probability (0.2) of being penalized despite consistently choosing D. Moreover, AD players receive an additional payoff from $d$, as they incur no costs to support the institution.

It should be noted that, if all players were to adopt strategy S, cooperation would fail to materialize. This is due to the spread of action D across the population as a result of implementation errors. Consequently, at vertex S, the rate of cooperation is zero. This phenomenon can be understood by examining the equation for the variable $p$, as discussed earlier. Importantly, $p$ represents the cooperation rate of the entire population, which equals zero when $x_1=0$ (i.e., in the absence of AC-players).

Plotting the value of $p$ (the ”cooperation rate”) as a function of $x_1$ on the edge between S and AC reveals that $p$ increases sharply with even small increments in $x_1$ (see Figure 3). Since $x_1$ and $x_3$ are related by $x_3=1-x_1$ along this edge, $p$ effectively becomes a function of a single variable. At the stable fixed-point T, where $x_1$ is approximately 0.1, the value of $p$ is around 0.6, indicating that more than half of the population engages in helping others.

Figure 4 demonstrates that, when the punishment strength $s$ becomes smaller ($s=3$ is chosen for this figure), all trajectories converge to the vertex AD. In this case, qualitative differences between EUT and PT are eliminated.

5. Conclusions

We have seen that there is a set of parameter values with which prospect theory leads to the evolution of pay-it-forward behaviors and with which linear expected utility theory does not. Ironically, punishment is more effective when people are irrational. One reason why even minimal punishment can be highly effective lies in the fact that players disproportionately fear the small possibility of punishment, perceiving it as more likely to occur than it actually does. Cognitive distortions described by PT significantly contribute to mitigating the risk of catastrophic population collapse due to free-riding.

As mentioned in the Introduction, in [11] it was demonstrated that combining prospect theory with punishment enables it to evolve and sustain the unconditional strategy AC, thus preventing free-riders from infiltrating a population of cooperators. In this paper, we applied a similar logic to pay-it-forward and have shown that prospect theory, coupled with slight punishment, facilitates the evolution of pay-it-forward. Given that the fixed-point T is located near vertex S, we can assert that the pay-it-forward strategy is ”risk-dominant” over the unconditional strategy AC, making it more advantageous than unconditional altruistic behaviors.

We noted that the probability of an S-player helping another player X does not rely on X’s strategy, indicating that S-players do not discriminate among other players. However, an S-player’s decision to help another is contingent upon whether the S-player received help in the previous game. Thus, while pay-it-forwarders do not discriminate, their actions are conditional. Taking these facts into account, we suggest that strategy S serves as an intermediate strategy between unconditional altruistic strategies, such as AC, and conditional cooperative strategies like “stern-judging”, “simple-standing”, or “image-scoring” in downstream indirect reciprocity.

Indeed, the bi-stable structure depicted in Figure 2c resembles the structure commonly observed in studies on downstream indirect reciprocity, where it has been found that, if players adopt strategies that consider so called “second-order information”, such as stern-judging or simple-standing in downstream reciprocity, the system exhibits bi-stability. Conversely, we have demonstrated that, when players adhere to prospect theory, upstream reciprocity exhibits a similar structure.

Image-scoring, a strategy in downstream reciprocity that relies solely on first-order information, is insufficient for the evolution of cooperation within the framework of EUT. Consequently, image-scoring fails to sustain stable cooperation in traditional settings [6], even though experimental evidence suggests that real humans only consider first-order information. Investigations into how the cognitive distortions described by prospect theory influence the stability of conditional strategies that use only first-order information in downstream indirect reciprocity present an intriguing research avenue.