On Playing with Emotion: A Spatial Evolutionary Variation of the Ultimatum Game

The Ultimatum Game is a simplistic representation of bargaining processes occurring in social networks. In the standard version of this game, the first player, called the proposer, makes an offer on how to split a certain amount of money. If the second player, called the responder, accepts the offer, the money is divided according to the proposal; if the responder declines the offer, both players receive no money. In this article, an agent-based model is employed to evaluate the performance of five distinct strategies of playing a modified version of this game. A strategy corresponds to instructions on how a player must act as the proposer and as the responder. Here, the strategies are inspired by the following basic emotions: anger, fear, joy, sadness, and surprise. Thus, in the game, each interacting agent is a player endowed with one of these five basic emotions. In the modified version explored in this article, the spatial dimension is taken into account and the survival of the players depends on successful negotiations. Numerical simulations are performed in order to determine which basic emotion dominates the population in terms of prevalence and accumulated money. Information entropy is also computed to assess the time evolution of population diversity and money distribution. From the obtained results, a conjecture on the emergence of the sense of fairness is formulated.


Introduction
The systematic study of emotion on an evolutionary perspective begins with the book The Expression of the Emotions in Man and Animals written by Charles Darwin [1].In this book, first published in 1872, Darwin discusses the biological aspects and the social value of emotional expression.To Darwin, an emotion reflects an underlying mental state generated by the brain of human beings and other animals, which can be associated with typical facial muscular pattern and body posture [1,2].Also, to Darwin, emotions have evolved just as anatomical structures have evolved [1,3,4].Thus, the process of natural selection has shaped not only phenotypic traits, but also psychological responses, in order to enhance the evolutionary fitness of the species.Therefore, our emotional behavior is, at least in part, genetically inherited [1,3,4].
Economic behavior, a particular form of decision-making, is influenced by emotions [7][8][9][10][11].This statement can be validated by analyzing, for instance, the attitudes of individuals playing the Ultimatum Game.In the original version of this game [12][13][14][15][16], a given amount of money must be divided between two players.The first player, referred to as the proposer, makes an offer on how to split this amount.If the second player, referred to as the responder, accepts the deal, both players divide the money as suggested by the proposer; however, if the responder rejects the deal, neither player receives anything [12][13][14][15][16].According to classic game theory [17,18], a purely rational responder, seeking to maximize resources for self-interest, should take any offered share, as receiving something is preferable to receiving nothing.As a consequence, a purely rational proposer should offer the minimum possible amount.However, experimental studies revealed that the mutually expected rationality is barely observed, because the average offer is around 40% of the total [13].Also, around half of the offers below 20% are rejected [14].Thus, some fifty percent of the responders punish proposals perceived as unfair, despite exiting the game without any gain.
The Ultimatum Game can represent the final stage of a real-world bargaining process, wherein the ultimate offer can either be accepted or rejected [16,17].During this process, emotions can arise and impact the final proposer's offer and the final responder's reply.Functional magnetic resonance imaging has been employed to investigate the neural activity elicited by this game [19,20].Such neuroimaging studies have shown, for instance, that unfair offers (about 20% or less) increase the activation of brain regions related to cognition (dorsolateral prefrontal cortex), emotion (bilateral anterior insula), and cognitive conflict (anterior cingulate cortex) [19,20].Thus, unfair offers provoke a struggle in the responder between logic reasons (to take any money) and emotional reasons (to reject low proposals) [19,20].The interplay between an emotional system and a rational system shapes the responder's decision [21].
Despite being paradigmatic in studies of cooperation and fairness [12][13][14][15][16][17][18][19][20][21][22], the original version of this game presents two unrealistic features: the resource to be split appears out of nowhere and it disappears in the case of disagreement between the players.In a variation of this game [23], the resource to be divided belongs to the proposer; thus, it returns to the proposer if the negotiation fails.However, in the case of uncompleted negotiation, the survival chances of both players is reduced.On the other hand, a completed negotiation increases the resources of the responder and the survival chances of both.This variation can be suitable to represent a negotiation between employer (the proposer who possesses the money) and employee (the responder who provides essential services to the proposer after receiving the offered amount) [23].
A basic emotion can be simplistically viewed as an information-processing program implemented in the brain to generate specific behaviors in response to specific adaptive problems [1,3,4,24].Basic emotions modulate the actions in the Ultimatum Game [19,20,[25][26][27][28][29][30].For instance, studies have shown that induced sadness (by previously presenting short movie clips to responders) decreases the acceptance rate of unfair offers [26] and increases the offered share [27].Joy promotes a more flexible attitude towards making concessions [28].Fear of failed negotiation increases the offers and decreases the rejection rate [29].Anger from the responder can be a signal that only elevated amounts will be accepted; anger from the proposer can be a signal of low proposals [30].
Here, five distinct strategies of playing the Ultimatum Game are associated with five basic emotions.This association intends to reflect results found in the literature [13,14,19,20,[25][26][27][28][29][30].The aim is to determine the performance of these strategies.The question to be answered here is the following: which strategies will prevail?In other words, which emotions will have the greatest impact on enhancing survival and reproduction in a competitive environment?This study is computationally performed by considering a spatial evolutionary variation of the Ultimatum Game.The standard version of this game was already investigated from the perspective of evolutionary game theory [31][32][33][34][35][36][37].In the variation investigated here, a single emotion is assigned to each agent and this emotion reflects a game strategy (as the proposer and as the responder).Also, the money does not vanish after the rejection of an offer, but the survival of the agents depends on closed deals.This variation has not been explored in the literature.
The remainder of this text is organized as follows.In Section 2, the agent-based model used to simulate this evolutionary game is introduced.In Section 3, the metrics based on information entropy used to analyze the population diversity and the money distribution are defined.In Section 4, the results obtained from numerical simulations are presented.In Section 5, these results are discussed from a Darwinian standpoint.

The Model
Agent-based models have been developed to study complex systems of diverse nature [38][39][40][41][42].For instance, agents with emotional states have been used to simulate the emotion contagion [43] and the emergence of polarization opinion [44] in online social media and to evaluate a candidate in a selection process [45].The model proposed in this manuscript is formulated in terms of interacting emotion-endowed agents.
The agents inhabit a two-dimensional lattice represented by an n × n matrix with periodic boundary conditions (that is, the upper and lower edges are joined and the left and right edges are also joined, in order to avoid edge effects).Each one of N = n 2 agents lives in a cell of this matrix and interacts with the four nearest agents (the north, east, south, and west neighbors), as illustrated by Figure 1.In cellular automata literature, this connectivity pattern is known as the von Neumann neighborhood of unit radius [46] and it has been employed in theoretical investigations on the Ultimatum Game [15,23,[32][33][34]36].Let d i (t) and ℓ i (t) be, respectively, the amount of money and the number of lives of the i-th agent at the time step t.At t = 0, each agent receives D dollars and L lives.Then, at each time step t, each agent acts as the proposer if it has an accumulated amount of money equal to or exceeding the minimum value m (that is, if As the proposer, the i-th agent at the time step t allocates the fraction f of its accumulated amount to play the Ultimatum Game with its neighbors.Thus, as the proposer, the i-th agent sets aside the amount f d i (t).Since it has four neighbors, the amount available to play with each neighbor is f d i (t)/4.The effective offered amount depends on its strategy (its emotion), which will affect the responder's decision.If the deal is closed, both players gain a life each and the amount is split as determined by the proposer.If the deal is not closed, the share f d i (t)/4 stays with the proposer and both players lose a life.Observe that the number of lives L represents a predetermined quantity of attempts available to a player to continue playing the game without closing transactions.This limited quantity serves as a buffer against failed transactions and is a parameter that critically influences the player's survival.
If an agent dies at the time step t (that is, if ℓ i (t) = 0), the predominant emotion in its neighborhood is assigned to the newborn agent that will occupy that empty cell at the time step t + 1 (an alternative would be assigning the emotion of the richest neighbor to this newborn agent).If a tie occurs, a random selection is made.This replacement mechanism is called copycat and has been used in other studies on spatial games [47][48][49].The newborn agent receives L lives and inherits the money from the deceased agent.Notice that, at each time step, the i-th agent acts as the proposer four times (if d i (t) ≥ m, with i = 1, 2, . . ., N) and as the responder four times.In a simulation with T time steps (that is, t = 1, 2, . . ., T), up to 8NT negotiations can be performed.
A game strategy consists of specifying how to act as the proposer and how to act as the responder.Assume that, as the proposer, the i-th agent endowed with the emotion e offers the percentage x i,e of f d i (t)/4 to each neighbor; as the responder, it accepts offers equal to or greater than the percentage y j,ϵ of f d j (t)/4 of the j-th neighbor imbued with the emotion ϵ.Therefore, if x i,e ≥ y j,ϵ , a negotiation between the i-th proposer agent and its j-th responder neighbor is carried out; otherwise, it fails.
Consider the basic emotions e = {anger, fear, joy, sadness, surprise}.Here, these emotions correspond to game strategies, which are characterized by the values of x e and y e .Inspired by results reported in the literature, the following values of x e and y e were chosen: • Anger: x anger = 20% and y anger = 50% (an angry agent offers little and only accepts high shares [30]); • Fear: x f ear = 50% and y f ear = 0 (a fearful agent makes generous proposals and accepts any offer [29]); • Joy: x joy is a random number and y joy = 0. (Here, a joyful agent makes unpredictable offers averaging around 50% and rationally takes any amount.As about 50% of the unfair offers are rejected [14] and sad individuals tend to decline unfair offers [26], it is assumed that the remaining 50% of accepted unfair offers are accepted by non-sad individuals.In fact, joyful negotiators tend to be more cooperative [28]); • Sadness: x sadness = 50% and y sadness = 20% (a sad agent makes generous proposals [27] and rejects too-low offers [26]); • Surprise: x surprise is a random number and any offer is either accepted or rejected with a 50%/50% chance (a surprising agent makes unpredictable proposals and the acceptance/rejection is arbitrary and independent of the offered percentage).
Evidently, other values for x e and y e could have been chosen.Notice that anger, fear, and sadness follow deterministic rules; joy and surprise follow probabilistic rules.Notice also that the total number of agents N and the total amount of money DN remain constant throughout a numerical simulation (because the number of cells is fixed and the money is only redistributed among the agents as time progresses).Table 1 shows the payoff matrix of this game.
Table 1.In this variation of the Ultimatum Game, the i-th agent expressing the emotion e (the proposer) makes an offer to the j-th agent expressing the emotion ϵ (the responder) at the time step t only if d i,e (t) ≥ m; that is, if the accumulated resources d i,e (t) of the i-th agent is greater than or equal to the minimum amount m.For ϵ = {anger, fear, joy, sadness}, a successful negotiation occurs if x i,e ≥ y j,ϵ ; otherwise, the negotiation fails.Here, x i,e is the percentage offered by i-th agent with the emotion e and y j,ϵ is the percentage aspired by the j-th agent with the emotion ϵ.The payoffs and the lives gained or lost by both players are shown in this table.As in the standard payoff matrices employed in game theory, the first number within parentheses corresponds to the payoff/life gained/lost by the i-th agent playing as the proposer and the second number to the payoff/life gained/lost by the j-th agent playing as the responder.For ϵ = {surprise}, there is a 50% chance of the negotiation being completed.
• for ϵ = {anger, fear, joy, sadness} In the next section, two metrics based on information entropy are defined.These metrics are employed to analyze how the basic emotions and the accumulated money vary among the agents.

Metrics
Let N e (t) and M e (t), respectively, be the number of agents expressing the emotion e and the total amount of money accumulated by these N e agents at the time step t, with e = {anger, fear, joy, sadness, surprise}.Obviously, ∑ e N e (t) = N and ∑ e M e (t) = DN.Also, let p e (t) and q e (t) be defined as: Thus, at the time step t, p e (t) is the percentage of agents endowed with the emotion e and q e (t) is the percentage of accumulated money by such agents.Obviously, ∑ e p e (t) = 1 and ∑ e q e (t) = 1.
Let ϕ e (t) be the accumulated money per agent with the emotion e at the time step t.This metric is obtained from: The population diversity at t can be assessed by computing the normalized information entropy H p (t) defined as: in which the information entropy h p (t) is written in terms of p e (t) [50].The maximum value h max p occurs when all the emotions appear in equal proportion [50].Since five distinct emotions are considered, h max p is obtained from p e = 1/5 for any e; hence, h max p = log 5. Similarly, the money distribution at t can be evaluated by calculating the normalized information entropy H q (t) defined as: in which the information entropy h q (t) is expressed as a function of q e (t) [50].Since this maximum value h max q occurs for a uniform distribution [50], then h max q = log 5. Here, the normalized inhomogeneity in population diversity I p and the normalized inhomogeneity in money distribution I q are calculated by [51]: Similar metrics based on information entropy have been used to analyze the dynamics of complex systems [52][53][54][55][56][57].Plots of I p (t) and I q (t) reveal how the heterogeneities in agent variability and resource allocation evolve as time passes.

Numerical Results
Computer simulations were performed in order the examine the influence of D (the initial amount of money per agent) and L (the initial number of lives of each agent) on the asymptotic values of p e (t), q e (t), ϕ e (t), I p (t), and I q (t).In addition, the influences of x e (the offered percentage) and y e (the aspired percentage) for anger, fear, and sadness were also investigated.
In the simulations, one of five emotions is randomly assigned to each agent at t = 0. Thus, each emotion accounts for approximately 20% of the population at t = 0; consequently, p e (0) ≃ 1/5, H p (0) ≃ 1, and I p (0) ≃ 0. In addition, q e (0) ≃ 1/5, because each agent starts with D dollars.Hence, H q (0) ≃ 1 and I q (0) ≃ 0.Then, the N agents interact with their four neighbors by playing the variation of the Ultimatum Game described in Section 2 for T time steps.
and T = 100 from p e (0) ≃ 20% and q e (0) ≃ 20% (consequently, I p (0) ≃ 0 and I q (0) ≃ 0).The emotion-endowed agents interact as described in Section 2. In this plot, anger, fear, joy, sadness, and surprise are, respectively, represented by cyan, green, red, blue, and magenta.(b) Time evolution of I p (t). (c) Time evolution of q e (t).(d) Time evolution of I q (t).Notice that these variables converge to nearly constant values.
Here, a competitive scenario is specified by a particular combination of D, L, x e , and y e for e = {anger, fear, sadness}, in which the agents compete for monetary resources.As the proposer, the minimum amount to play and the allocated percentage remain fixed in m = 1 and f = 1/2, respectively.In addition, the number of agents N also remains fixed in N = 2500.Fifteen scenarios were examined.Each scenario was numerically simulated 10 times.Then, the average values and the standard deviations of p e (t), q e (t), ϕ e (t), I p (t), and I q (t) at t = 100 were calculated (at t = 100, these variables already reached their approximate asymptotic values, as shown in Figure 2).These average values are, respectively, denoted by ⟨p e ⟩, ⟨q e ⟩, ⟨ϕ e ⟩, ⟨I p ⟩, and ⟨I q ⟩.For instance, Table 2 exhibits the results for D = 100 and L = 1.Table 3 presents the results for n = 100 (that is, N = 10000).The other parameter values are the same as those shown in Table 2. Notice that the lattice size does not significantly affect the average results; however, the standard deviations usually become smaller when n is increased.Evidently, as n increases, the computer processing time to finish the simulation also increases.Figures 3-10 show how ⟨p e ⟩, ⟨q e ⟩, ⟨ϕ e ⟩, ⟨I p ⟩, and ⟨I q ⟩ vary with D and L, for L = 1, D = {20, 60, 100, 140} and for D = 100, L = {1, 2, 3, 4, 5, 6}.These figures reveal that fear, joy, and sadness tend to dominate the population in terms of prevalence and wealth.Notice that agents endowed with fear or joy are purely rational responders (because they take any offered share).As the proposer, agents endowed with fear or sadness are generous proposers, because they always offer 50%.Agents endowed with joy or surprise makes offers averaging around 50%; thus, sometimes they propose unfair offers.Angry agents always present unfair proposals.Figures 3-6 show that the parameter D has no evident impact on the asymptotic solution of this game.Since the agent's actions depend only on the offered/aspired percentages of the negotiable amount, the actual value of this amount does not influence their decisions; hence, D does not significantly affect the results.However, Figures 7-10 indicate that the population inhomogeneity and the economic inhomogeneity are reduced by increasing L; that is, ⟨I p ⟩ and ⟨I q ⟩ decrease with L. Also, increasing L benefits anger and surprise, the lowest prevalent emotions for L = 1.Figures 5 and 9 show that, by following the rules presented in Section 2, angry agents accumulate about 150 dollars per agent; fearful agents, joyful agents, and sad agents about 100 dollars per agent; and surprising agents about 50 dollars per agent.These numbers are not strongly influenced by D or L (or N, as shown in Tables 2 and 3).
. The plots of ⟨I p ⟩ (thin line) and ⟨I q ⟩ (thick line) in function of L. The parameter values are the same as those used in Figure 7.
For e = {anger, fear, sadness}, two variations on x e and y e were simulated for D = 100 and L = 1.As defined in Section 2, an angry agent is characterized by x anger = 20% and y anger = 50%.Therefore, a more angry agent corresponds to x anger = 20% − ∆ with ∆ > 0 (it decreases its offer) and y anger = 50% + ∆ (it increases its aspired percentage).Conversely, a less angry agent can be described by these same expressions by taking ∆ < 0. A fearful agent is defined by x f ear = 50% and y f ear = 0. Hence, for a more fearful agent, x f ear = 50% + ∆ with ∆ > 0 (it increases its offer); for a less fearful agent, ∆ < 0. A sad agent is identified by x sadness = 50% and y sadness = 20%.For a more sad agent, x sadness = 50% + ∆ with ∆ > 0 (it increases its offer) and y sadness = 20% + ∆ (it increases its aspired percentage); for a less sad agent, ∆ < 0. Figures 12, 14, and 16, respectively, indicate that playing with less anger, less fear, or less sadness increases the accumulated resources by angry agents, fearful agents, and sad agents, respectively.This conclusion can be drawn by observing the cyan bars in Figure 12, the green bars in Figure 14, and the blue bars in Figure 16.According to Figure 11, the percentage of angry agents decreases with ∆; according to Figures 13 and 15, the percentages of fearful agents and sad agents are not noticeably affected by ∆.In Figures 11 and 12, the average inhomogeneities ⟨I p ⟩ and ⟨I q ⟩ increase with ∆; in Figures 13 and 14, these inhomogeneities decrease with ∆; in Figures 15 and 16, these relations are not monotonous.The parameter values are the same as those used in Figure 13.For ∆ = −0.1,⟨I q ⟩ = 0.183 ± 0.010; for ∆ = 0, ⟨I q ⟩ = 0.164 ± 0.009; and for ∆ = 0.1, ⟨I q ⟩ = 0.064 ± 0.005.15.For ∆ = −0.1,⟨I q ⟩ = 0.156 ± 0.009; for ∆ = 0, ⟨I q ⟩ = 0.164 ± 0.009; and for ∆ = 0.1, ⟨I q ⟩ = 0.114 ± 0.013.

Discussion
In human beings, basic emotions are characterized by particular facial expressions [1,2] and distinct patterns of cardiorespiratory activity [58,59] and neural dynamics [60,61], reflecting differentiated brain states and affecting decision-making.In the Ultimatum Game, the proposer's decision is about resource allocation and the responder's decision depends on the personal aversion to inequity [12][13][14][15][16][17][18][19][20][21].Here, a spatial evolutionary variation of this game was played by interacting agents.Their strategies intended to emulate the attitudes associated with five basic emotions.In the proposed model, agents manifesting anger, fear, or sadness were guided by deterministic rules; agents expressing joy or surprise exhibited probabilistic behavior.
In the original version of this game, the players must cooperate to obtain money.In the simulations conducted here, lives are lost if negotiations are not successfully completed.Thus, the agents must cooperate to survive, since lack of cooperation leads to death.Greater cooperation is provided by agents with "high" x e and "low" y e ; that is, by agents who make generous proposals and even accept unfair offers.Fearful agents fit perfectly into this description.Hence, fear (the green color) has a tendency to prevail in Figures 3 and 7.In contrast, angry agents are characterized by "low" x e and "high" y e ; hence, anger (the cyan color) is the less prevalent emotion.However, as can be viewed in Figures 5 and 9, angry agents are those that have more money per agent.These two figures also show that surprised agents are the poorest.This result is aligned with the intuitive notion that using a random approach in bargaining processes does not appear to be the most effective strategy for achieving good results.In Figures 11-16, the rules guiding the agents imbued with anger, fear, or sadness were modified.These figures suggest that modifications towards rationality increases p e and/or q e of the emotion e with altered rules.
Figure 6 indicates that natural selection is barely impacted by varying D; however, according to Figure 10, natural selection is clearly affected by varying L. In fact, Figure 10 reveals that, by increasing L, the inhomogeneities in agent variability and in resource allocation decrease.Increasing L implies refraining from immediately punishing non-cooperative agents with death, which enhances the prevalence of angry agents and surprising agents.
Basic emotions have been associated with responses to opportunities and threats related to survival-relevant situations [1][2][3][4][5][6].Fear is a response to menacing stimuli, which may have been ubiquitous in nature since the very beginning.It is considered to be a (or the most) primitive basic emotion, because it may have emerged at earlier stages in the history of animal evolution [1,3,6,62].In fact, even invertebrates can feel fear [63].In this study, the fear of unsuccessful negotiation is equivalent to the fear of dying.
In the model, joyful agents differ from fearful agents by offering an averaging of 50% (instead of the fixed percentage of 50%); sad agents differ from fearful agents by accepting proposals of at least 20% (instead of accepting any share).These slight differences imply similar but distinct performances, as illustrated by Figures 2-10.Notice that these three basic emotions are relevant in bargaining processes: fear aims at self-preservation; joy can strengthen social connections by making very generous offers; sadness intensifies aversion to unfairness.On the other hand, expressing anger may not be suitable in negotiations, because it denotes a non-cooperative attitude.Anger, however, can be a driving force for gathering more resources, as indicated by Figures 5 and 9.
Future research projects can consider that each agent can probabilistically exhibit different emotions.Such emotions may be triggered by the neighbors' behavior or employed as a strategy to improve performance.For instance, a responder can adopt the persona of an angry individual in order to up the proposer's offer.
Fairness is a moral and social concept.In the context of this study, fairness means equitable distribution of resources.Conjectures on the origins of the sense of fairness based on reputation [31,37], mutation and natural selection [35], food-sharing practices [64], and reciprocity [65] have been proposed.The results found in the computer simulations reported here suggest the following alternative conjecture.In primitive communities, business transactions might have involved the exchange of labor-power for clothes and food.In such ancient times, natural selection may have benefited (fair) individuals who deal with "high" x e (who are generous proposers) and "low" y e (who are rational responders), as fear, joy, and sadness are represented in the model.Thus, the bargaining processes may have been shaped by economic considerations ("low" y e ) and by affective impulses ("high" x e ).In this scenario, the selective pressure exerted by L (the number of chances of staying alive without completing negotiations) may have been more pronounced than that imposed by D (which determines the actual amount to be dealt).
Cooperation means acting together to achieve mutual advantages.Here, cooperation between two fair individuals implies successful negotiation, which contributes to form social bonds and improves the biological fitness of both.Therefore, the development of the sense of fairness may have been supported by basic emotions and it may have been a consequence of creating reliable social connections in order to increase the odds of reproducing and staying alive.

Figure 1 .
Figure 1.A block 5 × 5 of a lattice showing the von Neumann neighborhood of unit radius of the black central cell.This neighborhood is formed by the four gray cells.The white cells are not neighbors of the black central cell.

Figure 4 .
Figure 4.The histograms of ⟨q e ⟩ in function of D, for e = {anger, fear, joy, sadness, surprise}.The parameter values are the same as those used in Figure 3.

Figure 5 .
Figure 5.The plots of ⟨ϕ e ⟩ in function of D, for e = {anger, fear, joy, sadness, surprise}.The parameter values are the same as those used in Figure 3.

Figure 6 .
Figure 6.The plots of ⟨I p ⟩ (thin line) and ⟨I q ⟩ (thick line) in function of D. The parameter values are the same as those used in Figure 3.

Figure 8 .
Figure 8.The histograms of ⟨q e ⟩ in function of L, for e = {anger, fear, joy, sadness, surprise}.The parameter values are the same as those used in Figure

Figure 9 .
Figure 9.The plots of ⟨ϕ e ⟩ in function of L, for e = {anger, fear, joy, sadness, surprise}.The parameter values are the same as those used in Figure 7.
exhibit ⟨p e ⟩ and ⟨q e ⟩ in the function of ∆.Only a single emotion is altered in each of these six figures (the other four emotions behave as presented in Section 2).Figures 11 and 12 are for anger, Figures 13 and 14 for fear, and Figures 15 and 16 for sadness.
anger; center: anger; right: more anger