# The Evolution of Cooperation and Diversity under Integrated Indirect Reciprocity

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Results

#### 2.1. The Setup

#### 2.2. Modeling Integrated Reciprocators

#### 2.3. Model I: Stable Coexistence of the Good and the Bad

#### 2.4. Model II: Robustness against the Invasion of Cooperators

#### 2.5. Cooperator, Defector, Upstream Reciprocator, and Downstream Reciprocator

## 3. Discussion

## 4. Materials and Methods

#### 4.1. Evolutionary Dynamics and Image Dynamics

#### 4.2. Model I

#### 4.3. Model II

#### 4.4. Stability of the Attractor P against Invasion of Pure Downstream Reciprocators in Model II

#### 4.5. Cooperator, Defector, Upstream Reciprocator, and Downstream Reciprocator

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Trivers, R.L. The evolution of reciprocal altruism. Q. Rev. Biol.
**1971**, 46, 35–57. [Google Scholar] [CrossRef] - Axelrod, R.; Hamilton, W.D. The evolution of cooperation. Science
**1981**, 211, 1390–1396. [Google Scholar] [CrossRef] [PubMed] - Press, W.H.; Dyson, F.D. Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent. Proc. Natl. Acad. Sci. USA
**2012**, 109, 10409–10413. [Google Scholar] [CrossRef] [PubMed] - Yamagishi, T.; Cook, K.S. Generalized exchange and social dilemmas. Soc. Psychol. Q.
**1993**, 56, 235–248. [Google Scholar] [CrossRef] - Sugden, R. The Economics of Rights, Co-Operation and Welfare; Blackwell: Chichester, UK, 1986. [Google Scholar]
- Alexander, R. The Biology of Moral Systems; Aldine de Gruyter: Berlin, Germany, 1987. [Google Scholar]
- Boyd, R.; Richerson, P.J. The evolution of indirect reciprocity. Soc. Netw.
**1989**, 11, 213–236. [Google Scholar] [CrossRef] - Kandori, M. Social norms and community enforcement. Rev. Econ. Stud.
**1992**, 59, 63–80. [Google Scholar] [CrossRef] - Nowak, M.A.; Sigmund, K. Evolution of indirect reciprocity. Nature
**2005**, 437, 1291–1298. [Google Scholar] [CrossRef] - Milinski, M.; Semmann, D.; Krambeck, H.J. Reputation helps solve the ‘tragedy of the commons’. Nature
**2002**, 415, 424–426. [Google Scholar] [CrossRef] [PubMed] - Simpson, B.; Willer, R. Altruism and indirect reciprocity: The interaction of person and situation in prosocial behavior. Soc. Psychol. Q.
**2008**, 71, 37–52. [Google Scholar] [CrossRef] - Yoeli, E.; Hoffman, M.; Rand, D.G.; Nowak, M.A. Powering up with indirect reciprocity in a large-scale field experiment. Proc. Natl. Acad. Sci. USA
**2013**, 110, 10424–10429. [Google Scholar] [CrossRef] - Baker, W.E.; Bulkley, N. Paying it forward vs. rewarding reputation: Mechanisms of generalized reciprocity. Organ. Sci.
**2014**, 25, 1493–1510. [Google Scholar] [CrossRef] - Melamed, D.; Simpson, B.; Abernathy, J. The robustness of reciprocity: Experimental evidence that each form of reciprocity is robust to the presence of other forms of reciprocity. Sci. Adv.
**2020**, 6, eaba0504. [Google Scholar] [CrossRef] [PubMed] - Fowler, J.H.; Christakis, N.A. Cooperative behavior cascades in human social networks. Proc. Natl. Acad. Sci. USA
**2010**, 107, 5334–5338. [Google Scholar] [CrossRef] [PubMed] - Bearman, P. Generalized exchange. Am. J. Sociol.
**1997**, 102, 1383–1415. [Google Scholar] [CrossRef] - Malinowski, B. Argonauts of the Western Pacific: An Account of Native Enterprise and Adventure in the Archipelagoes of Melanesian New Guinea; Routledge: London, UK, 1922. [Google Scholar]
- Uehara, E. Dual exchange theory, social networks, and informal social support. Am. J. Sociol.
**1990**, 96, 521–557. [Google Scholar] [CrossRef] - Bartlett, M.Y.; DeSteno, D. Gratitude and prosocial behavior: Helping when it costs you. Psychol. Sci.
**2006**, 17, 319–325. [Google Scholar] [CrossRef] [PubMed] - DeSteno, D.; Bartlett, M.Y.; Baumann, J.; Williams, L.A.; Dickens, L. Gratitude as moral sentiment: Emotion-guided cooperation in economic exchange. Emotion
**2010**, 10, 289–293. [Google Scholar] [CrossRef] - Tsvetkova, M.; Macy, M.W. The social contagion of generosity. PLoS ONE
**2014**, 9, e87275. [Google Scholar] [CrossRef] - Simpson, B.; Harrell, A.; Melamed, D.; Heiserman, N.; Negraia, D.V. The roots of reciprocity: Gratitude and reputation in generalized exchange systems. Am. Sociol. Rev.
**2018**, 83, 88–110. [Google Scholar] [CrossRef] - Rutte, C.; Taborsky, M. Generalized reciprocity in rats. PLoS Biol.
**2007**, 5, e196. [Google Scholar] [CrossRef] - Stanca, L. Measuring indirect reciprocity: Whose back do we scratch? J. Econ. Psychol.
**2009**, 30, 190–202. [Google Scholar] [CrossRef] - Yoshikawa, K.; Wu, H.C.; Lee, H.J. Generalized exchange orientation: Conceptualization and scale development. J. Appl. Psychol.
**2020**, 105, 294. [Google Scholar] [CrossRef] - Eriksson, T.; Ferreira, C. Who pays it forward the most? Examining organizational citizenship behavior in the workplace. J. Theor. Soc. Psychol.
**2021**, 5, 215–228. [Google Scholar] [CrossRef] - Romano, A.; Saral, A.S.; Wu, J. Direct and indirect reciprocity among individuals and groups. Curr. Opin. Psychol.
**2022**, 43, 254–259. [Google Scholar] [CrossRef] [PubMed] - Bolton, G.E.; Katok, E.; Ockenfels, A. Cooperation among strangers with limited information about reputation. J. Public Econ.
**2005**, 89, 1457–1468. [Google Scholar] [CrossRef] - Swakman, V.; Molleman, L.; Ule, A.; Egas, M. Reputation-based cooperation: Empirical evidence for behavioral strategies. Evol. Hum. Behav.
**2016**, 37, 230–235. [Google Scholar] [CrossRef] - Gaudeul, A.; Keser, C.; Müller, S. The evolution of morals under indirect reciprocity. Games Econ. Behav.
**2022**, 126, 251–277. [Google Scholar] [CrossRef] - Rand, D.G.; Nowak, M.A. Human cooperation. Trends Cogn. Sci.
**2013**, 17, 413–425. [Google Scholar] [CrossRef] [PubMed] - Allen, B.; Lippner, G.; Chen, Y.T.; Fotouhi, B.; Momeni, N.; Yau, S.T.; Nowak, M.A. Evolutionary dynamics on any population structure. Nature
**2017**, 544, 227–230. [Google Scholar] [CrossRef] - Su, Q.; McAvoy, A.; Mori, Y.; Plotkin, J.B. Evolution of prosocial behaviours in multilayer populations. Nat. Hum. Behav.
**2022**, 6, 338–348. [Google Scholar] [CrossRef] - Su, Q.; Allen, B.; Plotkin, J.B. Evolution of cooperation with asymmetric social interactions. Proc. Natl. Acad. Sci. USA
**2022**, 119, e2113468118. [Google Scholar] [CrossRef] [PubMed] - Huang, Y.; Wan, S.; Zheng, J.; Liu, W. Evolution of cooperation in spatial public goods game with interactive diversity. Phys. A
**2023**, 621, 128794. [Google Scholar] [CrossRef] - Pfeiffer, T.; Rutte, C.; Killingback, T.; Taborsky, M.; Bonhoeffer, S. Evolution of cooperation by generalized reciprocity. Proc. R. Soc. B
**2005**, 272, 1115–1120. [Google Scholar] [CrossRef] [PubMed] - Nowak, M.A.; Roch, S. Upstream reciprocity and the evolution of gratitude. Proc. R. Soc. B
**2007**, 274, 605–610. [Google Scholar] [CrossRef] [PubMed] - Rankin, D.J.; Taborsky, M. Assortment and the evolution of generalized reciprocity. Evolution
**2009**, 63, 1913–1922. [Google Scholar] [CrossRef] [PubMed] - Barta, Z.; McNamara, J.M.; Huszar, D.B.; Taborsky, M. Cooperation among non-relatives evolves by state-dependent generalized reciprocity. Proc. R. Soc. B
**2011**, 278, 843–848. [Google Scholar] [CrossRef] [PubMed] - Sigmund, K. The Calculus of Selfishness; Princeton Univ. Press: Princeton, NJ, USA, 2010. [Google Scholar]
- Hofbauer, J.; Sigmund, K. Evolutionary Games and Population Dynamics; Cambridge University Press: Cambridge, UK, 1998. [Google Scholar]
- Brandt, H.; Sigmund, K. The good, the bad and the discriminator—Errors in direct and indirect reciprocity. J. Theor. Biol.
**2006**, 239, 183–194. [Google Scholar] [CrossRef] [PubMed] - Brandt, H.; Sigmund, K. The logic of reprobation: Assessment and action rules for indirect reciprocation. J. Theor. Biol.
**2004**, 231, 475–486. [Google Scholar] [CrossRef] - Ohtsuki, H.; Iwasa, Y. How should we define goodness?—Reputation dynamics in indirect reciprocity. J. Theor. Biol.
**2004**, 231, 107–120. [Google Scholar] [CrossRef] - Ohtsuki, H.; Iwasa, Y. The leading eight: Social norms that can maintain cooperation by indirect reciprocity. J. Theor. Biol.
**2006**, 239, 435–444. [Google Scholar] [CrossRef] - Nowak, M.A.; Sigmund, K. Evolution of indirect reciprocity by image scoring. Nature
**1998**, 393, 573–577. [Google Scholar] [CrossRef] [PubMed] - Nowak, M.A.; Sigmund, K. The dynamics of indirect reciprocity. J. Theor. Biol.
**1998**, 194, 561–574. [Google Scholar] [CrossRef] [PubMed] - Berger, U. Learning to cooperate via indirect reciprocity. Games Econ. Behav.
**2011**, 72, 30–37. [Google Scholar] [CrossRef] - Leimar, O.; Hammerstein, P. Evolution of cooperation through indirect reciprocity. Proc. R. Soc. B
**2001**, 268, 745–753. [Google Scholar] [CrossRef] [PubMed] - Panchanathan, K.; Boyd, R. A tale of two defectors: The importance of standing for evolution of indirect reciprocity. J. Theor. Biol.
**2003**, 224, 115–126. [Google Scholar] [CrossRef] [PubMed] - Okada, I. A review of theoretical studies on indirect reciprocity. Games
**2020**, 11, 27. [Google Scholar] [CrossRef] - Yamamoto, H.; Suzuki, T.; Umetani, R. Justified defection is neither justified nor unjustified in indirect reciprocity. PLoS ONE
**2020**, 15, e0235137. [Google Scholar] [CrossRef] [PubMed] - Sasaki, T.; Okada, I.; Nakai, Y. The evolution of conditional moral assessment in indirect reciprocity. Sci. Rep.
**2017**, 7, 41870. [Google Scholar] [CrossRef] [PubMed] - Okada, I.; Yamamoto, H.; Sato, Y.; Uchida, S.; Sasaki, T. Experimental evidence of selective inattention in reputation-based cooperation. Sci. Rep.
**2018**, 8, 14813. [Google Scholar] [CrossRef] - Fishman, M.A. Indirect reciprocity among imperfect individuals. J. Theor. Biol.
**2003**, 225, 285–292. [Google Scholar] [CrossRef] - Mohtashemi, M.; Mui, L. Evolution of indirect reciprocity by social information: The role of trust and reputation in evolution of altruism. J. Theor. Biol.
**2003**, 223, 523–531. [Google Scholar] [CrossRef] [PubMed] - Brandt, H.; Sigmund, K. Indirect reciprocity, image scoring, and moral hazard. Proc. Natl. Acad. Sci. USA
**2005**, 102, 2666–2670. [Google Scholar] [CrossRef] - Ohtsuki, H.; Iwasa, Y. Global analyses of evolutionary dynamics and exhaustive search for social norms that maintain cooperation by reputation. J. Theor. Biol.
**2007**, 244, 518–531. [Google Scholar] [CrossRef] [PubMed] - Okada, I.; Sasaki, T.; Nakai, Y. Tolerant indirect reciprocity can boost social welfare through solidarity with unconditional cooperators in private monitoring. Sci. Rep.
**2017**, 7, 9737. [Google Scholar] [CrossRef] [PubMed] - Lotem, A.; Fishman, M.A.; Stone, L. Evolution of cooperation between individuals. Nature
**1999**, 400, 226–227. [Google Scholar] [CrossRef] [PubMed] - Sherratt, T.N.; Roberts, G. The role of phenotypic defectors in stabilizing reciprocal altruism. Behav. Ecol.
**2001**, 12, 313–317. [Google Scholar] [CrossRef] - Takahashi, N.; Mashima, R. The importance of subjectivity in perceptual errors on the emergence of indirect reciprocity. J. Theor. Biol.
**2006**, 243, 418–436. [Google Scholar] [CrossRef] [PubMed] - Gray, K.; Ward, A.F.; Norton, M.I. Paying it forward: Generalized reciprocity and the limits of generosity. J. Exp. Psychol.
**2012**, 143, 247. [Google Scholar] [CrossRef] [PubMed] - Kim, J.E.; Tsvetkova, M. Cheating in online gaming spreads through observation and victimization. Netw. Sci.
**2021**, 9, 425–442. [Google Scholar] [CrossRef] - Martinez-Vaquero, L.A.; Cuesta, J.A. Evolutionary stability and resistance to cheating in an indirect reciprocity model based on reputation. Phys. Rev. E
**2013**, 87, 052810. [Google Scholar] [CrossRef] - Santos, F.P.; Santos, F.C.; Pacheco, J.M. Social norm complexity and past reputations in the evolution of cooperation. Nature
**2018**, 555, 242–245. [Google Scholar] [CrossRef] [PubMed] - Uchida, S. Effect of private information on indirect reciprocity. Phys. Rev. E
**2010**, 82, 036111. [Google Scholar] [CrossRef] - Uchida, S.; Sigmund, K. The competition of assessment rules for indirect reciprocity. J. Theor. Biol.
**2010**, 263, 13–19. [Google Scholar] [CrossRef] [PubMed] - Okada, I.; Sasaki, T.; Nakai, Y. A solution for private assessment in indirect reciprocity using solitary observation. J. Theor. Biol.
**2018**, 455, 7–15. [Google Scholar] [CrossRef] - Yamamoto, H.; Okada, I.; Uchida, S.; Sasaki, T. A norm knockout method on indirect reciprocity to reveal indispensable norms. Sci. Rep.
**2017**, 7, 44146. [Google Scholar] [CrossRef] - Uchida, S.; Yamamoto, H.; Okada, I.; Sasaki, T. A theoretical approach to norm ecosystems: Two adaptive architectures of indirect reciprocity show different paths to the evolution of cooperation. Front. Phys.
**2018**, 6, 14. [Google Scholar] [CrossRef] - Hilbe, C.; Schmid, L.; Tkadlec, J.; Chatterjee, K.; Nowak, M.A. Indirect reciprocity with private, noisy, and incomplete information. Proc. Natl. Acad. Sci. USA
**2018**, 115, 12241–12246. [Google Scholar] [CrossRef] - Krellner, M.; Han, T.A. Pleasing enhances indirect reciprocity under private assessment. Artif. Life
**2021**, 27, 246–276. [Google Scholar] [CrossRef] - Reiter, J.G.; Hilbe, C.; Rand, D.G.; Chatterjee, K.; Nowak, M.A. Crosstalk in concurrent repeated games impedes direct reciprocity and requires stronger levels of forgiveness. Nat. Commun.
**2018**, 9, 555. [Google Scholar] [CrossRef] [PubMed] - Schmid, L.; Chatterjee, K.; Hilbe, C.; Nowak, M.A. A unified framework of direct and indirect reciprocity. Nat. Hum. Behav.
**2021**, 5, 1292–1302. [Google Scholar] [CrossRef] - Ohtsuki, H.; Iwasa, Y.; Nowak, M.A. Indirect reciprocity provides only a narrow margin of efficiency for costly punishment. Nature
**2009**, 457, 79–82. [Google Scholar] [CrossRef] [PubMed] - Podder, S.; Righi, S.; Pancotto, F. Reputation and punishment sustain cooperation in the optional public goods game. Phil. Trans. R. Soc. B
**2021**, 376, 20200293. [Google Scholar] [CrossRef] [PubMed] - Doebeli, M.; Hauert, C.; Killingback, T. The evolutionary origin of cooperators and defectors. Science
**2004**, 306, 859–862. [Google Scholar] [CrossRef] [PubMed] - Archetti, M.; Scheuring, I. Coexistence of cooperation and defection in public goods games. Evolution
**2011**, 65, 1140–1148. [Google Scholar] [CrossRef] - Hauert, C.; Doebeli, M. Spatial social dilemmas promote diversity. Proc. Natl. Acad. Sci. USA
**2021**, 118, e2105252118. [Google Scholar] [CrossRef]

**Figure 1.**Three types of indirect reciprocity. Each panel illustrates a different reciprocity mechanism. The subscript $i$ in the label ${t}_{i}$ of each arrow represents the order in which the help actions occur in that direction. (

**a**) Upstream reciprocity: first A helps B, and then B (upstream reciprocator) forwards the help received to C. (

**b**) Downstream reciprocity: first B helps C, and then A (downstream reciprocator) rewards B by helping. (

**c**) Integrated reciprocity: first D helps E, then B (integrated reciprocator) rewards D by helping; next, given that the other integrated reciprocator, A, rewards B by helping them, B forwards the help received to C. Another integrated reciprocator may also subsequently reward B, who is moved by this and again forwards the help received to someone else.

**Figure 2.**Evolution of integrated reciprocity for Model I. Panels (

**a**,

**b**) depict phase portraits of the replicator dynamics for the unconditional cooperator X, unconditional defector Y, and integrated reciprocator Z without and with errors, respectively. The triangles describe a simplex of the state space $\mathsf{\Delta}=\left\{\right(x,y,z):x+y+z=1\}$. Each node (X, Y, or Z) of the triangle corresponds to the homogeneous state of each strategy ($x$, $y$, or $z=1$, respectively). Moreover, filled and empty circles denote stable and unstable fixed points, respectively. (

**a**), Without errors, the simplex has three continua of fixed points: XZ, RY, and PQ, among which only PQ remains when assuming errors in (

**b**). Whether errors are present or absent, considering a random shock on the population composition, the presence of the continuum of interior fixed points, PQ, prevents the population from staying at the boundary attractor P. (

**b**), In particular, the population will eventually converge to the node Y (100% defector state). Parameters: $c=1$, $b=5$, (

**a**) $\u03f5=0$, and (

**b**) $\u03f5=0.05$.

**Figure 3.**Evolution of integrated reciprocity for Model II. Panels (

**a**) and (

**b**) depict phase portraits of the replicator dynamics for the unconditional cooperator X, unconditional defector Y, and integrated reciprocator Z, as in Figure 2. (

**a**) Without errors, the dynamics show the global attractor P along the edge YZ. At P, integrated reciprocators and defectors stably coexist. (

**b**) The global dynamics become bistable: the population will eventually converge to either the local attractors P or Y (100% defector state). Parameters: $c=1$, $b=5$, (

**a**) $\u03f5=0$, and (

**b**) $\u03f5=0.1$.

**Figure 4.**Evolution of four traditional strategies. Panels (

**a**) and (

**b**) depict phase portraits of the replicator dynamics for the unconditional cooperator X, unconditional defector Y, upstream reciprocator V, and downstream reciprocator W, on the surfaces and in the interior space, respectively. The tetrahedral simplex in (

**b**) denotes the state space ${\mathsf{\Delta}}_{4}=\left\{\left(x,y,v,w\right):x+y+v+w=1\right\}$. Each node (X, Y, V, or W) corresponds to the homogeneous state for each strategy. The tetrahedral simplex intersects a planar set that consists of stable and unstable fixed points. The red solid lines in (

**a**) denote a set of continuums of boundary fixed points, among which PQ, QR, and RP are the intersections between each triangular surface and the planar set. In (

**b**), the global dynamics show that, on one hand, blue interior orbits are converging to points on the plane while, on the other hand, the red interior orbits are tending to the node Y, which indicates the local stability of the node Y. Due to the planar set, while considering random fluctuations, the evolution can end up in the 100% defector state at the node Y. Parameters: $c=1$, $b=5$, and $\u03f5=0.1$.

**Table 1.**Action and assessment rules for integrated reciprocity. Integrated reciprocators (as donors) act following the action rule and then are evaluated based on the assessment rule. In the assessment rules, G and B denote the donor’s image changing to good and bad, respectively; and K means that no change occurs in the donor’s image.

a. Action rule for integrated reciprocity | Image of recipient | ||

G | B | ||

In the previous round for the focal donor | received C | C | C |

received D | C | D | |

b. Assessment rule for Model I | In the current round | ||

give C | give D | ||

In the previous round for the focal donor | received C | G | B |

received D | G | B | |

c. Assessment rule for Model II | In the current round | ||

give C | give D | ||

In the previous round for the focal donor | received C | G | B |

received D | K | K |

**Table 2.**Action rules for upstream and downstream reciprocity. In the model, upstream and downstream reciprocators as a donor act following the corresponding action rule as below.

a. Action rule for upstream reciprocity | Image of recipient | ||

G | B | ||

In the previous round for the focal donor | received C | C | C |

received D | D | D | |

b. Action rule for downstream reciprocity | Image of recipient | ||

G | B | ||

In the previous round for the focal donor | received C | C | D |

received D | C | D |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sasaki, T.; Uchida, S.; Okada, I.; Yamamoto, H.
The Evolution of Cooperation and Diversity under Integrated Indirect Reciprocity. *Games* **2024**, *15*, 15.
https://doi.org/10.3390/g15020015

**AMA Style**

Sasaki T, Uchida S, Okada I, Yamamoto H.
The Evolution of Cooperation and Diversity under Integrated Indirect Reciprocity. *Games*. 2024; 15(2):15.
https://doi.org/10.3390/g15020015

**Chicago/Turabian Style**

Sasaki, Tatsuya, Satoshi Uchida, Isamu Okada, and Hitoshi Yamamoto.
2024. "The Evolution of Cooperation and Diversity under Integrated Indirect Reciprocity" *Games* 15, no. 2: 15.
https://doi.org/10.3390/g15020015