Hybrid Assessment Scheme Based on the Stern-Judging Rule for Maintaining Cooperation under Indirect Reciprocity

: Intensive studies on indirect reciprocity have explored rational assessment rules for maintaining cooperation and several have demonstrated the effects of the stern-judging rule. Uchida and Sasaki demonstrated that the stern-judging rule is not suitable for maintaining cooperative regimes in private assessment conditions while a public assessment system has been assumed in most studies. Although both assessment systems are oversimpliﬁed and society is most accurately represented by a mixture of these systems, little analysis has been reported on their mixture. Here, we investigated how much weight on the use of information originating from a public source is needed to maintain cooperative regimes for players adopting the stern-judging rule when players get information from both public and private sources. We did this by considering a hybrid-assessment scheme in which players use both assessment systems and by using evolutionary game theory. We calculated replicator equations using the expected payoffs of three strategies: unconditional cooperation, unconditional defection, and stern-judging rule adoption. Our analysis shows that the use of the rule helps to maintain cooperation if reputation information from a unique public notice board is used with more than a threshold probability. This hybrid-assessment scheme can be applied to other rules, including the simple-standing rule and the staying rule.


Introduction
Indirect reciprocity [1][2][3][4][5] is a well known mechanism for maintaining cooperation among unrelated players, in which cooperators request information about potential recipients. This reciprocity-based cooperation mechanism prevents free-riders, i.e., players who do not cooperate but receive benefits from naive cooperators, from invading the population of players. It does this by imposing a discrimination that allows players to cooperate only with players with a good reputation, studies, cooperation is defined here as the action of a player giving another player something of value that has a certain cost for the player, such as time or effort. So, every cooperator loses a cost while its recipient receives a benefit. This is why many theoretical studies on the evolution of cooperation have used this game.
We focused on an adaptive process for this rule and used a replicator dynamics analysis, a widely accepted method, to analyze the dynamics of several strategies. We focused on three types of strategies: stern judgers (who apply the stern-judging rule), perfect cooperators, and perfect non-cooperators. The standard theory on the evolution of cooperation teaches us that naive cooperators are defeated by naive non-cooperators. Therefore, preventing free-riders from invading cooperative regimes is a core objective of this study. Using a typical theoretical approach, we considered the effect of the stern-judging strategy for a population including both naive strategies.

Model of Hybrid-Assessment Scheme
We built a model of a hybrid-assessment scheme based on the stern-judging rule in the framework of evolutionary game theory.

Game and Strategy
We considered an infinite number of well mixed players who play a giving game multiple times. Time is discrete, and only one game is played each period. In this game, two players, a donor D and a recipient R, are randomly chosen, and D decides whether to contribute to R or not. If D contributes, R gains benefit b, while D pays cost c. Otherwise, nothing happens. We assume that b > c. Each player adopts one of three strategies: perfect cooperation X, perfect defection Y, or stern-juding rule adoption Z. Set S = {X, Y, Z}. An X player always contributes when it is a donor while a Y player never contributes. Contributions by a Z player depend on the recipient. If the recipient is assessed as Good by a Z player, the player contributes to R; if the recipient is assessed as Bad, the player does not contribute to the recipient. Each stern judger thus labels the images of the other players as either Good or Bad. The contribution patterns under the rule are shown in Table 1. For example, if D contributes to R, and R has a Bad image, D will also be labeled as Bad, as shown by the third pattern. To generalize the game, we introduced two types of errors. One is an implementation error, in which there is a probability e of not contributing when a player intends to contribute. The other is a cognitive error in which there is a probability e that a donor's image is reversed when updating. We assume that both probabilities are constant. Let = (1 − e )(1 − e) + e e.

Hybrid Assessment
We consider a hybrid assessment using two types of assessment schemes: a public scheme and a private scheme. In the public scheme, when a Z player is chosen as D, the player accesses a public notice board and obtains reputation information on R, i.e., R's image, and decides on the basis of this information whether to contribute to R. In the private scheme, when a Z player is chosen as D, the player decides whether to contribute in accordance with the player's private image of R. We combined the two schemes by introducing a constant probability, p. When a Z player is chosen as D, the player adopts either the public scheme with probability p or the private scheme with probability 1 − p. That is, stern judgers use the public assessment scheme with probability p and use the private one with probability 1 − p. Let g(p) be a rate at which a Z player assesses players as Good.
The updating rule for images in the public scheme is simple. A donor's action in every game is immediately reflected on the public notice board. If a player i is the donor and j is the recipient, i's image updated at the end of the game in accordance with the patterns in Table 1. For example, if i did not make a contribution and j's image is Good on the public notice board, i's image on the board is switched to Bad if it was Good before the game.
The updating rule for images in the private scheme is not so simple. First, there are three possible assessments for each game: it is observed and assessed as Good, it is observed and assessed as Bad, or it is not observed, in which case the donor player's image is not updated. If a Z player observes a game, the player updates its image of the donor in that game on the basis of the donor's action in accordance with the assessment pattern in Table 1. Note that if the number of observers goes to infinity, a theoretical analysis is virtually impossible because the number of equation systems also goes to infinity [32]. Therefore, we assume that the number of observers of each game is limited. Under this assumption, the image of a player in the eyes of a stern judger is rarely updated. This is why infinite time is required to saturate the value of g(p) [29].

Results
We analyzed our model theoretically to explore the effects of hybrid assessment under the stern-judging rule on indirect reciprocity using replicator dynamics. First, g(p) is solved to obtain the expected payoff for each strategy. Then the replicator dynamics are solved. Finally, the evolutionary stable points are calculated, and their conditions are solved.

Image Dynamics
We first consider the public assessment scheme. Let x, y, and z be the fractions (population ratios) of players using X, Y, and Z strategies, respectively, where x + y + z = 1. Like Sasaki [24], we use g Pub s to denote the fraction of players with a Good image on a unique public notice board adopting each strategy, where s ∈ S. We use g Pub to denote the average fraction of players with Good images over the population. Thus, g Pub = xg Pub X + yg Pub Y + zg Pub Z . In addition, we use g S,I to denote the probability that a Good image is assigned to a potential donor who adopts strategy S and also faces a potential recipient with an image score I = Good [G] or Bad [B]. The population size is infinite, so we assume that the composition of the population does not change between consecutive one-round giving games. Thus, the population frequencies of Good players satisfy Using the patterns in Table 1, Equation (1) yields We consider g X,G as an example. This example considers the case where a donor is X while a recipient is assessed as Good. In this case, the donor contributes to the recipient with the probability 1 − e , and if so, this donor is assessed as Good. However, with the probability e, this assessment is changed to Bad. On the other hand, the donor does not contribute to the recipient with the probability e , and this donor is assessed as Bad. However, with the probability e, this assessment is changed to Good. Summing up, for the probability assessed as Good, g X,G yields (1 − e )(1 − e) + e e.
For simplicity, we assume that 0 < e < 1/2 < < 1. Therefore, is satisfied. Note that g Pub ≥ 1/2 is satisfied regardless of the values of (x, y, z). We next consider the private assessment scheme. In this scheme, all stern judgers (Z) have images of all players privately. Therefore, there is an image matrix, I = (i uv ) where i uv is an image of player v in the eyes of player u, and u adopts Z strategy [26]. In each game, if a player adopting the Z strategy observes a game, the player updates a donorfs private image following their private image of a recipient and the donorfs action based on the assessment rule shown in Table 1.
We used g Pri s to denote the fraction of players with a privately labeled Good image adopting each strategy, where s ∈ S. Players using the same strategy are homogeneous, so the value of g Pri s is unique, as discussed in detail by Okada et al. [32]. The definition of g Pri Z differs from that in the public scheme only because the actions of perfect cooperators and perfect defectors are pre-determined regardless of the images of the potential recipients. Let there be a case in which a stern judger is chosen as a potential donor. In the public scheme, both the potential donor and an observer who also adopts the stern-judging rule use the same public reputation to assess the potential recipient. In the private scheme, however, the two stern judgers (the donor and observer) use their private images of the potential recipient. Thus, we introduce a new parameter, g Pri 2 , which is the probability that the two stern judgers share the same image of the potential recipient.
Thus, the population frequencies of Good players satisfy g Pri .
If the observers are limited to a finite number as assumed, g Pri 2 = (g Pri ) 2 is satisfied. Substituting g Pri = xg Pri X + yg Pri X + zg Pri X into Equation (4) yields Thus, g Pri = 1/2 is satisfied regardless of the values of (x, y, z). Finally, we consider the hybrid assessment scheme. A player who adopts the stern-judging rule mixes information on the public notice board and the player's own private image. They use the reputation information provided by the public assessment system with the probability p, and use the private impression provided by the private assessment system with the probability 1 − p. Thus, the average population frequencies of Good players satisfy where s ∈ S.

Replicator Dynamics
We use evolutionary game theory for the updating rule to investigate whether cooperative regimes are maintained with the stern-judging rule. We calculate replicator equations using the expected payoffs of the strategies. The replicator dynamics are described aṡ where U s is the expected payoff for players whose strategy is s ∈ S, andŪ = xU X + yU Y + zU Z is the average payoff over the population. These values can be calculated using g(p). The expected payoffs of the three strategists are where we omit the factor 1 − e .
Our results show that a population consisting of only stern judgers (the top vertices of the triangles in Figure 1) cannot be invaded by either perfect cooperators or perfect defectors if p exceeds a certain threshold, as shown in Figure 1.
There is a phase transition on the threshold, as shown in Figure 2. To prove that no other stable points exist, we first consider the non-existence of inner fixed points (that is, that all x > 0, y > 0, and z > 0 are satisfied). This is because if an inner fixed point exists, U X = U Y = U Z is satisfied at that point. If so, U X = U Y yields pbz( − e)(1 − 2g) = c and U Y = U Z yields pbz( − e) = c, so g Pub must be 1/2. However, g Pub > 1/2 when z > 0, which is a contradiction. Next, we consider the three edges of the triangle. When y = 0, U Z > U X is satisfied. When z = 0, This positive condition means that the point (x, y, z) = (1 − z * , 0, z * ) is unstable. Therefore, there are no other stable points except for (x, y, z) = (0, 1, 0) and (0, 0, 1).
We calculate the border of b (in the left panel of Figure 2). Equation (9)

Discussion
We analyzed a hybrid scheme of public and private assessments, that is, based on the stern-judging rule using an evolutionary game. This rule maintains cooperation in a public system but not in a private one, so whether it maintains cooperation in a hybrid system is worth consideration. Our theoretical analysis shows that the stern-judging rule maintains cooperation if holders of the rule can access reputation information on a unique public notice board with more than a threshold probability, as shown in Figure 1. Theoretically, actions based on a public reputation are more successful than ones based on a private image because g Pub ≥ g Pri = 1/2, as shown in the Results Section.
As Figure 2 shows, the hybrid system can maintain cooperation if the cost-benefit ratio of a cooperative action exceeds a threshold (depending on the errors). The greater the cost-benefit ratio, the lower the limit of the probability for using the public assessment scheme. Moreover, the smaller the implementation and cognition errors, the lower the lower limit of the probability for using the public scheme. A cognitive error affects the lower limit more than an implementation error.
The stern-judging rule is too strict to maintain cooperative regimes in a private system. In part, this is because the rule is vulnerable to errors and unintended non-contributions. This vulnerability is emphasized more in a private assessment system. This is a problem related to "justified defection" [12,31,32]. For example, if a potential donor faces a potential recipient labeled as having a Bad image, a donor adopting the stern-judging rule does not contribute in accordance with the patterns in Table 1. However, this justified defection is seen as justified only by those whose image of the recipient is also Bad. There is thus no stable cooperative regime in a perfectly private system. This justification dilemma does not seem like it would be eliminated in the hybrid scheme because players use their private images with a certain probability. Our theoretical results suggest, however, that this assumption is incorrect. Even if a justified defection sometimes does not appear justified to others, the payoff to the stern-judging rule adopters can exceed that of unconditional defectors because their other justified defections can be seen as justified due to common reputations in the public assessments.
This paper is a first step in examining a hybrid assessment using a mix of public and private assessment schemes in the context of indirect reciprocity and thus, we analyzed the case of the stern-judging rule only. However, hybrid assessment can be applied to other rules. For example, the simple-standing rule [36,37] and the staying rule [10,24,31,32] should be considered in future work. This is because a private assessment scheme under both the simple-standing rule and the staying rule brings a higher cooperation rate than a public one while the stern-judging rule is the opposite, as shown by Okada et al. [32].
We next compared our hybrid assessment scheme with the "imperfect monitoring scheme" [38], which is often referenced in game theory literature. In general, the imperfect monitoring scheme considers the case of a player playing a game against an opponent without observing the opponent's actions. Therefore, in the "imperfect monitoring" scheme, players themselves cannot observe their opponents' actions, and thus, this is a situation between the parties. Here, the term "hybrid assessment" means that it is the third persons, not the players playing games, that are observing games. Therefore, our analysis of such a hybrid assessment has originality.
In this paper, we used the probability of choosing one of the two systems; thus, in our model, the players mix the two systems. This assumption reflects actual society because people generally use public and private schemes effectively. When they have to assess another person, they sometimes access the reputation information generated by a public assessment scheme and sometimes consider their own impression generated by a private assessment scheme.
Despite the above-mentioned extension of conventional indirect reciprocity models, we must note that our model still has limitations. It is true that players of our model use both public and private information sources in the long run, but they judge a single game based on either public or private information. The probability that public information is chosen is p. However, it is likely that real human beings even make a single decision based on mixture of information from various sources. This type of information mixing is not included in our present model and incorporating it into our model remains a future task.
We can extend our scheme for mixing public and private assessments. For example, there could be two types of players (those who use a public system and those who use a private system) and there could be homogeneous players who adaptively change their usage of the two systems. Another extension is to make the probability of choosing an assessment scheme endogenous. That is, the players' strategies include the probability. Adaptive dynamics [39] may be able to solve such a system.
Other extensions can also be considered. In our paper, the system provides information on a unique public notice board in the public assessment scheme. However, people could use a majority voting system as public information instead [29]. Future work will explore such extensions. Exploring a norm ecosystem [40,41] consisting of many norms would be another good extension. Introducing these aspects would bring the system closer to actual society, as well as considering hybrid assessment.
Author Contributions: All authors designed and discussed the research; I.O. performed the research and wrote the paper. All authors have read and agreed to the published version of the manuscript.

Conflicts of Interest:
The authors declare no conflicts of interest.