Inefficient Learning Leads to Efficient Coordination: A Repeated Stag-Hunt Game with Learning Agents

Abstract

In coordination games, the distinctive presence of multiple equilibria poses a challenge to coordination, and this has led over time to the proposal of numerous determinants of the latter. In this paper, we study coordination in a repeated Stag-hunt game played by two agents who are able to learn from the outcomes of each game, but do not know the payoff matrix. By modulating their learning capabilities, we show that agents are able to coordinate on the social optimum when they learn slowly from experience and are prone to making mistakes. A comparison with the results from a recent laboratory experiment is also provided.

1. Introduction

The emergence of cooperation among humans has become a significant area of research since the late XX Century (Axelrod, 1981; Challet & Zhang, 1997). Much of this body of work examines the evolutionary dynamics that favor altruistic behavior in natural environments, framing the issue as a problem of population dynamics (Hofbauer & Sigmund, 1998; Roca et al., 2009b). However, cooperation is pervasive and can also be observed in pair interactions within competitive contexts. In this type of interaction, cooperation between both individuals is usually desirable, as it allows everyone to obtain the maximum possible benefit, the so-called social optimum. But from an individual point of view, the choice to cooperate is not always so straightforward. In some scenarios, cooperating or defecting may be the best and safest choice for the individual, regardless of the actions of the other, making it easier to make a decision (or predict actions). However, in other situations, this is not always the case: there is no universal best choice, and predicting the implications of one’s decisions becomes more difficult, as they are conditioned by the actions of the other.

In game theory, this situation of strategic uncertainty is formally described by the existence of multiple Nash equilibria, outlining a fundamental coordination problem. To study this phenomenon, numerous frameworks, under the name of coordination games, have been proposed. In these settings, unlike social dilemmas characterized by an implicit conflict of interest, players achieve better results if they make the same choices, with the consequent absence of a unique dominant strategy (Devetag & Ortmann, 2014).

From a theoretical point of view, the presence of multiple equilibria represents a deadlock: standard equilibrium analysis does not allow behavioral predictions to be made, forcing theorists to be agnostic about the actual equilibrium selection. As highlighted by Cooper and Weber (2020), this gap has stimulated experimental exploration of coordination, revealing a general tendency for people to coordinate, albeit not always on the most efficient outcome (Cooper & Weber, 2020; Devetag & Ortmann, 2014). In this broad and diverse strand of literature, scholars have investigated different aspects and possible determinants of coordination. Sociocultural factors (e.g., Chen & Chen, 2011; De Kwaadsteniet & Van Dijk, 2010; Engelmann & Normann, 2010; Fallucchi & Nosenzo, 2022; Krupka et al., 2022; Krupka & Weber, 2013), such as the existence of shared social representations and cultural standards, may facilitate successful coordination thanks to their effect on players’ expectations (implicit or otherwise). Studies on cognitive aspects, on the other hand, have instead investigated internal reasoning processes, exploring decision-making styles (intuitive vs. deliberative) and players’ levels of sophistication (e.g., Arad & Rubinstein, 2012; Bardsley et al., 2010; Belloc et al., 2019; Camerer et al., 2004; Nagel, 1995; Rubinstein, 2016; Stahl & Wilson, 1995). The question of equilibrium selection has also been extensively addressed by more theoretical strands of the literature, such as evolutionary game theory. These numerical studies have investigated coordination games in both well-mixed populations and network structures, testing different update rules, such as myopic best response or imitative mechanisms (Raducha & San Miguel, 2022). These studies have shown how topological features can, in some cases, facilitate coordination (Tomassini & Pestelacci, 2010; Vilone et al., 2021), although this kind of effect is not generally universal (Roca et al., 2009a). Finally, a minority of studies have explored coordination by moving away from the evolutionary approach in favor of cognitive and learning mechanisms (Flache & Macy, 2002; Macy & Flache, 2002; Roth & Erev, 1995; Skyrms, 2004). A common feature of these works is the use of relatively simple learning processes and a deliberately unsophisticated representation of the agents’ cognitive mechanisms. By eschewing the traditional assumption of perfect rationality, this perspective reveals that coordination on the social optimum can be successfully achieved even through minimal adaptive heuristics.

Following this latter perspective, in this work, we studied the coordination dynamics between two agents who participate in a repeated coordination game. In particular, we explored how fundamental learning processes can shape cooperation and strategic coordination in an exclusively pair-wise interaction, where the dynamics of the game and the strategic outcomes are initially unknown and more than one Nash equilibrium is possible. The objective of this work is twofold, as it aims to (i) model some recent experimental results and (ii) to show how simple and imperfect cognitive mechanisms are sufficient to navigate strategic uncertainty and resolve coordination problems in dyadic settings, without the need to employ complex deductive reasoning.

The paper is organized as follows: Section 2 provides the general configuration and dynamics of the model; the results obtained from the numerical simulations and a comparison with some recent experimental results are reported in Section 3. Finally, Section 4 discusses the results, limitations, and future prospects.

2. Materials and Methods

A classic game-theory framework that investigates coordination between two agents is the Stag-hunt (SH) game (Skyrms, 2001). The game represents a situation in which two hunters can cooperate to capture a larger prey, the Stag, or decide not to do so and settle for a smaller one (Hare). In the first case, the payoff is greater, but there is a risk of getting nothing if the other player decides not to cooperate. In the other case, the hare does not require coordination between the two players and represents a guaranteed payoff, albeit smaller than that of the Stag. This scenario inevitably generates tension between two strategic criteria, payoff and risk dominance, resulting in two possible equilibria. In the first case, both agents coordinate by choosing Stag. This is the Pareto-optimal and payoff-dominant equilibrium, as both players will obtain the maximum expected payoff. On the other hand, agents both choose Hare, obtaining a secure but significantly lower payoff. This risk-dominant equilibrium is the result of the strategic uncertainty inherent in the SH game: choosing Hare is in fact an optimal strategy given the high risk of miscoordination. The SH game therefore exhibits two pure Nash equilibria, and once one of these is reached, the agents have no incentive to change their strategy.

The following paragraphs in this section describe the general configuration adopted for the SH game, the update rule that governs the strategic choices of agents, and the different scenarios tested.

2.1. General Configuration

Let us consider two players,“player 1” and “player 2,” engaged in a repeated SH game. Each agent i is characterized by a probability $p_i^S$ of choosing Stag (S), while $p_i^H=1-p_i^S$ is the complementary probability of choosing Hare (H). Depending on their chosen strategy, each player receives a reward determined by the payoff matrix $\widehat{P}$ given in

Table 1. Payoff matrix $\widehat{P}$ for the player i. The parameter r indicates the payoff obtained by choosing Hare, while $ϵ$ is the additional payoff relative to Hare, obtained by both agents when they coordinate on Stag.

	Stag	Hare
Stag	$r+\epsilon$	0
Hare	r	r

, before updating the probability of the adopted strategy according to the overall reward obtained up to that round of the game.

When both agents choose Stag, the payoff obtained by each player is given by $r+\epsilon$, where r is the gain for (Hare, Hare) and the parameter $\epsilon$ represents the coordination prize for both moving on Stag. It is important to note that in this version of the game, a player adopting the Hare strategy gains the reward r independently of the opponent’s choice. The introduction of this variation allowed us to test the ability of the players to learn the best option even though the alternative guaranties the same reward in any case. This makes such SH closer to a Prisoner’s Dilemma game, with the difference being that the best option here is also a Nash equilibrium. At the end of each round, the agents’ cumulative payoff (that is, the sum of all the rewards received up to that round) is updated based on the strategies adopted.

Finally, it is important to emphasize that in the experiments we refer to in this work, players can check the other agent’s behavior, while in this theoretical framework, they choose their strategy without knowing the opponent’s decision (see Section 3.3 and Appendix A). It is noteworthy that, despite this strong simplification, the results of our simulations are nevertheless in good agreement with the empirical findings, as we are going to see in the following sections.

2.2. Dynamics

2.2.1. Initial Conditions

Regarding the initial strategies, at the time step $t=0$, the probability that each agent will choose Stag, $p_i^S$, is equal to $p_1^S=p_2^S=0.5=p_1^H=p_2^H$. This implies that, at the beginning of each simulation, agents do not know the rules of the game and the payoffs associated with the strategies.

As regards the cumulative payoff, in the experimental setup of this game (see again Section 3.3 and Appendix A), players lose a small amount of their payoff at each round, because it is assumed that just playing costs some energy, and if one player reaches zero energy, the game ends. Therefore, in order to be as coherent as possible with the experiments, we assign each agent an initial endowment of 10 units of payoff at the beginning of each realization. Every time they play a game stage, they lose 0.5 units, with this quantity representing the effort players put into playing the game, and if one of the agents continues to have no energy (that is, zero payoff), the simulation stops. Regardless, in all the simulations performed in every scenario tested, the agents managed to never reach zero (which happened in the experiments shown in Section 3.3 and Appendix A), so we are no longer going to mention it in the following sections.

2.2.2. Update Rule

At the end of each round of the game, players update the probabilities associated with the two strategies based on the payoff obtained. More precisely, if the player i has chosen the strategy K at the N-th stage, the probability of choosing K will be updated according to the following learning rule:

$$p_i^K\left(new\right)=p_i^K+{\alpha }_i(1-p_i^K){\displaystyle \frac{{\Pi }_K^i}{(r+\epsilon )N}},$$

(1)

where ${\alpha }_i$ is the learning rate of the player i, ${\Pi }_K^i$ is the total amount of reward gained by the player until the N-th stage when adopting the strategy K, and the normalization factor $1/\left[\right(r+\epsilon \left)N\right]$ guarantees that $p_i^K\in [0,1]$ at each time. Of course, since $p_i^S+p_i^H\equiv 1$, the probability to adopt the other strategy will change accordingly as well.

The learning rate ${\alpha }_i\in (0,1]$ of each agent i is assigned during initialization and does not vary over time. This parameter modulates the impact of the result obtained by the strategy K adopted at the time step t, dictating the extent to which the agent reacts to the experiences. Specifically, at the beginning of each single run of the model, agents 1 and 2 are assigned ${\alpha }_1$ and ${\alpha }_2$, respectively, whose values remain constant for the entire duration of the simulation and are chosen from the interval $(0,1]$. Through this initial assignment, we can test different scenarios in which agents exhibit the same, similar, or different learning rates, enabling us to study how coordination evolves in pairs of agents with homogeneous or heterogeneous learning capabilities. Overall, the introduction of this parameter allows us to increase the realism of the model, representing the cognitive abilities of the agent, as well as allowing us to study the interaction of heterogeneous agents, characterized by different values of $\alpha$.

2.2.3. Error Probability

At the beginning of each round, each agent i adopts a strategy K based on the corresponding probability $p_i^K$. This implies that a successful strategy, in terms of payoff, will have a higher probability of being adopted. This mechanism, although not deterministic, does not take into account the possibility that the agent makes mistakes. For this reason, among some of the scenarios, we will include an error probability ${\mu }_i$, assigned to each agent and constant over time. In these cases, agent i initially determines the strategy K to adopt based on $p_i^K$; once the strategy has been selected, agent i can choose the opposite strategy with probability ${\mu }_i$.

2.3. Simulations

The learning rates ${\alpha }_i$ and the error probabilities ${\mu }_i$ allowed us to explore different scenarios. The first case (baseline model) that we tested refers to the representation of perfectly rational agents, where players exhibit the highest learning rate (${\alpha }_1={\alpha }_2=1$) and do not make mistakes (${\mu }_1={\mu }_2=0$). Next, we evaluated the impact of the learning rate alone by testing different combinations of the values of ${\alpha }_1$ and ${\alpha }_2$ (keeping ${\mu }_1={\mu }_2=0$). Similarly, we repeated the simulations to evaluate the influence of the probabilities of error $\mu$ alone on agents’ strategies, for different combinations of ${\mu }_1$ and ${\mu }_2$, while keeping ${\alpha }_1={\alpha }_2=1$. Finally, we investigated the combined effect of both learning rate and error probability.

For each scenario, we studied the impact of $r\in [1,10]$ on the evolution of the probabilities associated with the two strategies: Stag and Hare. After assigning the agents their respective values for the parameters $\alpha$ and $\mu$, we performed $T={10}^3$ independent simulations for each scenario and value of r and computed the final average probabilities. For each run, agents continued to interact until convergence or the maximum number of iterations ($t={10}^5$) was reached. The convergence criterion was satisfied if the change in the probability of each agent, $p_1^S$ and $p_2^S$, was less than $ϵ<0.01$ over a time window of ${10}^2$ consecutive iterations.

3. Results

3.1. Baseline Model

In the baseline version of the model, we set ${\alpha }_i=1$ and ${\mu }_i=0$, $\forall i$. Therefore, agents learn at the fastest pace possible and never make mistakes. Considering perfectly rational learning agents is of course a huge approximation, we nevertheless start with this toy version for two main reasons: firstly, if we are able to capture some of the experimental results with such simplified model, this would indicate that the selected features are sufficiently powerful to provide a reasonable description of reality; secondly, it can work as a baseline to compare the effect of further refinements of the model itself.

Figure 1 reports the average probability of the two agents choosing Stag (S), which is equivalent to the probability that a single simulated experiment ends up with players always choosing S. As is easy to see, for $r\to +\infty$ we have $p^S\to 1^{-}$: an expected outcome, since if the Stag option is infinitely more profitable than Hare, there is no chance that the Hare choice can be selected.

When $r=4$, the average probability that both agents choose S is $p^S=0.5$. This indicates that players learn with a probability larger than $50\%$ that $(S,S)$ is actually the best option when the associated reward is at least four times more profitable than the Hare’s payoff.

This outcome can theoretically be justified as follows. From Equation (1), we can write down the average elementary variation $\Delta P_t$ of the probability $p^S$ after the $t-$th interaction (of course, the same can be done for $p^H=1-p^S$). Assuming that ${\alpha }_i=1,\forall i$, and that, by symmetry, the probability of choosing the strategy K evolves equally for both players, we obtain the following after a few passages:

$$\Delta P_t^S=P_t^S(1-P_t^S){\displaystyle \frac{P_t^S{\Pi }_S-{\Pi }_H}{(r+\epsilon )t}},$$

(2)

where

$${\Pi }_s={\displaystyle \sum _{j=0}^N}{\left(P_j^S\right)}^2·(r+\epsilon );{\Pi }_h={\displaystyle \sum _{j=0}^N}(1-P_j^S)·r,$$

(3)

that are the average rewards gained up to the N-th round when a player has adopted the Stag/Hare strategy, respectively. From Equation (2), it is easy to see that $P_N^S=P^{*}=0$ and $P_N^S=P^{+}=1$ are two fixed points of the dynamics, which is confirmed in simulations since, for every independent run, both players end up with the probability of choosing Stag equal to 0 or 1 for both, i.e., they always coordinate. There is also a third equilibrium, $\overline{P}$, that is,

$$\overline{P}={\displaystyle \frac{{\Pi }_H}{{\Pi }_S}}.$$

Now, if $\overline{P}$ is a physical equilibrium, that is, if $\overline{P}\in (0,1)$, then it is unstable. Indeed, let us assume that both players have reached $P^S=\overline{P}$. If they go again to Stag, for the rule given in Equation (1), at the subsequent stage, the probability will tend to increase again and go away from $\overline{P}$.

Therefore, we have established in a single run that players coordinate, with the final configuration always being one where both players always go for the Stag or always go for the Hare. We can evaluate the probability that a single run ends up with both players choosing the Stag, i.e., we can explain the results shown in Figure 1. To this end, let us consider the interactions in the very early stages of the game, when we can pose $p^S=p^H=0.5$ for both players. Focusing on a given player, its probability to choose Stag at the first interaction will change as follows:

$$\Delta p_0^S=\left\{\begin{array}{c}{\displaystyle \frac{1}{2}}\mathrm{with}\mathrm{probability}{\displaystyle \frac{1}{4}}\hfill \\ \hfill \\ -{\displaystyle \frac{r}{2(r+\epsilon )}}\mathrm{with}\mathrm{probability}{\displaystyle \frac{3}{4}}\hfill \end{array}\right.$$

(4)

Of course, on average, the double Stag is always the best configuration for the interacting players, but since they do not know the rewards at the beginning, they have to discover that it is the most convenient option just by trying. Now, in order to understand that mutual Stag is the best option, both players have to go for both on S to collect the payoff: on average, this happens once every four runs. More precisely, in the early stages of the game, both players select the strategy at random so that with a probability of $1/4$ only, they go for the Stag at the same time: therefore, for $r\ge 4$ only, it becomes convenient for the Stag strategy, as it rightly appears in the figure.

3.2. Learning Rate and Error Probability

In the baseline model, agents update the probability of choosing Stag based on the payoffs obtained. The impact of the latter is maximal and, in the first interactions, it is able to significantly modify the behavior of agents. Furthermore, decisions are not subject to errors and agents possess the same characteristics. However, this scenario may not represent reality. Humans do not learn at the same speed and are prone to making mistakes. In order to better capture these characteristics, we will therefore evaluate the impact of two factors on the behavior of agents: (i) the learning rate ($\alpha$) and (ii) the possibility of making mistakes ($\mu$). These two characteristics will be studied separately and in combination. The analysis of the following results will focus on the case where the payoff of stag $S=4$, i.e., the payoff where agents show a probability of cooperating equal to $p^s=0.5$ in the baseline model.

3.2.1. Learning Rate Effect

The pair of agents 1 and 2 are assigned ${\alpha }_1$ and ${\alpha }_2$, respectively, where ${\alpha }_1,{\alpha }_2\in (0,1]$. The learning rate controls how quickly the agent learns from its experiences: a high value of $\alpha$, especially in early interactions, can cause the agent to drastically change its strategy, while low values make the decision-making process more stable and slower. In the baseline model, both agents had a value of $\alpha =1$. In this first extension, we will consider not only the impact of lower learning rates, but also the interaction between heterogeneous agents characterized by different values of $\alpha$.

Figure 2 displays the evolution of $p^S$ as the Stag’s payoff ${\pi }_S$ increases for four different scenarios that reflect the interaction between agents with different values of the learning rate $\alpha$, including the baseline case (${\alpha }_1={\alpha }_2=1$). The scenarios were not selected based on any specific criteria but are intended simply to explore different combinations of $\alpha$, where the agents exhibit increasing values of the learning rate ($0.2,0.5,1$), in addition to a mixed case ($0.8,0.2$) where agents learn at different speeds. As we have already seen in Section 3.1, as the Stag’s payoff increases, we observe higher values of $p^S$ for any combination of ${\alpha }_1$ and ${\alpha }_2$. However, the highest values are observed as the learning rate decreases, particularly when both agents exhibit a very low learning rate $\alpha$, as we can see for ${\alpha }_1={\alpha }_2=0.2$. Also in the case of heterogeneous agents, such as in the scenario ${\alpha }_1=0.8$ and ${\alpha }_2=0.2$, where only one of the agents has a low learning rate, we observe higher average values of $p^S$ compared to the baseline.

To better understand the underlying dynamics, we can examine Figure 3, which shows the probability of choosing Stag for each combination of the values ${\alpha }_1$ and ${\alpha }_2$ when ${\pi }_S=4$. The dark red areas indicate higher values of $p^S$, while the dark blue areas indicate a low probability of coordinating on Stag. As can be clearly seen, the probability $p^S$ varies widely depending on the pair of values ${\alpha }_1$ and ${\alpha }_2$, ranging from $p^S\sim 0.5$ in the baseline model ($\alpha =1$) to $p^S\sim 0.9$ for ${\alpha }_1={\alpha }_2=0.1$. As the values of $\alpha$ decrease for both agents, the probability of cooperation increases monotonically.

Table 2. Simulation parameters. The table lists the learning rates (${\alpha }_1,{\alpha }_2$) and error probabilities (${\mu }_1,{\mu }_2$) of agents 1 and 2, the Stag’s payoff (${\pi }_S$), the maximum number of iterations (t) if convergence is not achieved, and the total number of generations (T).

${\mathit{\alpha }}_{\mathbf{1}}$	${\mathit{\alpha }}_{\mathbf{2}}$	${\mathit{\mu }}_{\mathbf{1}}$	${\mathit{\mu }}_{\mathbf{2}}$	${\mathit{\pi }}_{\mathit{S}}$	t	T
$(0,1]$	$(0,1]$	0	0	4	${10}^5$	${10}^3$

lists the simulation parameters employed for the results shown in Figure 3.

The highest value of $p^S$ is therefore reached when both agents exhibit the lowest learning rate tested ($\alpha =0.1$). In heterogeneous pairs of agents with different $\alpha$, the presence of a “slow learner” is generally beneficial for cooperation, increasing the overall probability of choosing Stag. For example, in the case where ${\alpha }_1=1$, $p^S$ tends to increase as the value of ${\alpha }_2$ decreases. The results therefore seem to suggest that slower learning benefits cooperation between agents. Gradually updating one’s behavioral rule provides sufficient space to test different strategies and avoid becoming stuck in a strategic stalemate.

3.2.2. Error Probability Effect

In the previous section, we examined the effect of the learning rate on the probability of coordination in Stag, but without considering the possibility of making mistakes. To analyze this different scenario, let us again consider two agents, each endowed with its own probability of making mistakes, $\mu$. For now, let us assume a constant and maximum learning rate ${\alpha }_1={\alpha }_2=1$. It is important to emphasize that the results presented do not report the actual choices made by the agents (Stag or Hare), but only the internal probabilities of each agent choosing Stag. For a comparison of actual behavior and internal probabilities, see Appendix B.

As we can see in Figure 4, the introduction of $\mu$ can either increase or decrease the probability of coordinating on Stag $p^S$. Compared to the baseline model (${\mu }_1={\mu }_2=0$), a small probability of making mistakes can be beneficial, as can be observed when $\mu =0.3$ for both agents. Conversely, high values of $\mu$ tend to result in values of $p^S$ that are lower than the baseline as the payoff of Stag ${\pi }_S$ increases. In the case of heterogeneous agents (${\mu }_1=0.3,{\mu }_2=0.8$), the presence of a single agent with a low probability of making errors is sufficient to offset the negative effect of a high $\mu$, resulting in an overall increase in $p^S$ compared to the baseline.

If we examine the values of $p^S$ for different combinations of ${\mu }_1$ and ${\mu }_2$ (${\mu }_1,{\mu }_2\in [0,1]$) when ${\pi }_S=4$ (Figure 5), we can see how the probability of choosing Stag initially tends to rise as $\mu$ increases (see

Table 3. Simulation parameters. The table lists the learning rates (${\alpha }_1,{\alpha }_2$) and error probabilities (${\mu }_1,{\mu }_2$) of agents 1 and 2, the Stag’s payoff (${\pi }_S$), the maximum number of iterations (t) if convergence is not achieved, and the total number of generations (T).

${\mathit{\alpha }}_{\mathbf{1}}$	${\mathit{\alpha }}_{\mathbf{2}}$	${\mathit{\mu }}_{\mathbf{1}}$	${\mathit{\mu }}_{\mathbf{2}}$	${\mathit{\pi }}_{\mathit{S}}$	t	T
1	1	$[0,0.5]$	$[0,0.5]$	4	${10}^5$	${10}^3$

for the list of simulation parameters employed). The maximum point is reached when both agents have a value of $\mu$ in the range 0.2–0.3. Moving towards higher values, the probability of cooperating gradually decreases. When the maximum value $\mu =1$ is reached, the value of $p^S$ is close to the case $\mu =0$. In this region, high behavioral instability due to frequent mistakes makes any form of learning difficult, if not impossible. Keeping the error probability ${\mu }_1$ of one of the two agents in the optimal region 0.2–0.3 fixed, we can observe that for increasing values of ${\mu }_2$, the probability $p^S$ is still relatively high ($p_{{\mu }_2=1}^S$∼0.75), compared to the baseline model.

In this scenario, despite rapid learning, the possibility of making mistakes represents another way to experiment with different strategies, with the same positive effects seen in the case of a low learning rate. In fact, even a small chance of making mistakes is enough to allow agents to break through possible strategic deadlocks. In the case of an agent with a high probability of error, this can still be beneficial if the other party exhibits a relatively low probability of errors.

3.2.3. Combined Effect of Learning Rate and Error Probability

Let’s now consider the joint impact of the learning rate $\alpha$ and the error probability $\mu$ on the probability of cooperating $p^S$. Figure 6 shows three different scenarios: for each combination of ${\mu }_1$ and ${\mu }_2$, we assign the agents the maximum (${\alpha }_1={\alpha }_2=1$) and minimum (${\alpha }_1={\alpha }_2=0.1$) learning rate values, as well as a mixed case (${\alpha }_1=0.8,{\alpha }_2=0.2$).

As seen in the previous section, when the learning rate of both agents is maximum, we observe the smallest probability of choosing Stag; conversely, when $\alpha =0.1$, the value of $p^S$ is the highest. On the other hand, the effect of the error $\mu$ is non-linear: an error probability between 0.2 and 0.3, even for a single agent, is usually beneficial.

Combining the two factors, we observe how their interplay contributes to widening the region of high coordination (red areas). In particular, as the learning rate decreases, the optimal interval for coordination given by $\mu$ tends to widen. From a narrow range of approximately $0.2\le \mu \le 0.3$ with ${\alpha }_1={\alpha }_2=1$, the interval widens significantly, up to approximately $0\le \mu \le 0.5$ when ${\alpha }_1={\alpha }_2=0.1$. We can observe this effect even in the case of heterogeneous agents (${\alpha }_1=0.8,{\alpha }_2=0.2$), where the presence of a single slow learner is capable of extending the positive effect of $\mu$.

These results therefore seem to suggest a specific synergy between the learning rate and the error probability that can promote coordination. While decreasing learning rates tend to increase $p^S$, the noise introduced by $\mu$ is beneficial only if limited. However, the detrimental effect of overly frequent errors can be counterbalanced by a gradual learning process, promoting cooperation even in situations where behavior is more erratic.

3.3. Comparison with Experimental Results

In this section, we will compare the results of our simulations with a recent experiment (see Appendix A), where pairs of participants played a computerized version of the SH game. Unlike other experimental designs, players were not provided with any information about the structure and rules of the game, in order to stimulate learning and exploration. During the game, participants were able to observe their partner’s moves, but no communication was possible. In this version of the SH, each player was presented with a horizontal grid, whose two ends represented the Stag and the Hare, and could move to the right or the left to reach one of the two ends. Each move involved a cost in “energy” to be subtracted from their initial endowment, similarly to the mechanism employed in our computational model. Each session consisted of at least 100 rounds. In total, 45 dyads completed the experiment, resulting in 5186 rounds. The payoff matrix used is reported in

Table 4. Payoff matrix employed in the experimental setting. Players were not informed about the potential payouts associated with the two actions, i.e., left and right movements.

	Stag	Hare
Stag	$(4a,4a)$	$(0,a)$
Hare	$(a,0)$	$(a,a)$

.

To evaluate the dynamics of cooperation, we analyzed when and how the dyads reached a stable joint strategy within each round. For the purpose of empirical comparison, cooperation is intended as convergence on a strategy. Furthermore, convergence was defined as the first moment within a round when both players simultaneously selected a defined joint state: $(S,S)$ (BothStag), $(H,H)$ (BothHare), $(S,H)$ (Split). For each dyad and round, we identified the earliest move corresponding to a valid joint state and computed the convergence time as the maximum elapsed time between the two participants to account for slower players. Players quickly learned that the $(S,S)$ strategy was the most profitable, adopting it most frequently (2915 rounds), followed by the $(H,H)$ strategy (1408 rounds) and the least-profitable $(S,H)$ strategy (863 rounds). The behavioral results also highlighted a rapid conversion within each round, with a mean convergence time of 334 ms (Figure 7). Most rounds reached a stable configuration within 834 ms, implying early mutual adaptation.

Early convergence suggests that participants rapidly coordinated their strategies, requiring only a few movement corrections. The speed of the players’ actions would seem to suggest a tendency toward a more intuitive and less deliberate decision-making approach, as assumed in our model. Moreover, the predominance of the $(S,S)$ state (BothStag) indicates that cooperative balance was not only achievable, but also more stable once reached (Figure 8). Finally, similarly to our computational results, in more than half of the cases, the players were able to coordinate by both choosing Stag. Furthermore, the possibility of observing the opponent’s actions during the game seems to have a very limited effect, justifying a posteriori the simplification made to the simulation model by not allowing the agents to observe each other during the game round.

4. Discussion

In this work, we simulated the learning dynamics of a couple of agents interacting in a Stag-hunt game, where one strategy (Hare) provides a reward independently from the opponent’s choice, while the other one (Stag) guarantees a larger payoff but only if both players adopt it. Therefore, to achieve the best outcome, agents have to learn not only which is the best behavior, but also to coordinate and choose it simultaneously. In order to describe and analyze the dynamics at stake, we considered a very simple model where the agents update their inner probability to choose a given strategy on the basis of the rewards accumulated during the entire interaction when such strategy has been adopted. In a refined version of this model, we tested the effect of agents’ (bounded) cognitive abilities on the fate of the dynamics, i.e., the possibility to learn at a slower rate and to make mistakes, that is, to adopt the opposite strategy selected by applying the learning rule.

Despite the oversimplification of the models, we have been able to reproduce some important features observed in real experiments. In particular, we see that the players always coordinate, that is, they learn that adopting the same behavior is always better (or, at least, not worse) than choosing different strategies. This is in good accordance with the empirical findings described in Section 3.3, where only a small minority of pairs of players were unable to coordinate. These results also appear to be consistent with other experimental findings in the literature: for example, in Van Huyck et al. (1990), dyads formed by the same players over time are able to coordinate to achieve the most efficient outcome.

As shown by the experimental results described in the previous section, in our model, the probability to find the best option (coordinating on Stag) increases with the reward provided by the corresponding strategy. This finding also emerges in some experimental studies on the optimization premium (Battalio et al., 2001) or the salience of the payoff (Silva, 2024), although the literature remains mixed on the role played by payoff differences in the strategies adopted.

Counterintuitively, but not surprisingly, adding a small but not negligible amount of deficiencies to the agents’ behavior helps them to find the best option more efficiently. In fact, for fast-learning and flawless agents, selecting Hare always increases the likelihood of this strategy, regardless of the partner’s behavior, but the same cannot be said for choosing Stag. Therefore, high-performing agents can understand that coordinating on Stag is the most profitable behavior only if both players opt for Stag at the beginning of the interaction, which on average happens with a probability of 1/4 (indeed, the final average probability to choose Stag becomes larger than 1/2 when the reward for mutual Stag is larger than Hare’s reward). The asymmetry inherent in the SH game and its payoff structure can therefore penalize the best-performing agents, who may remain stuck in a less remunerative position. Therefore, letting the players “make mistakes” allows them to explore more strategic options and understand if there are better configurations, given that this probability to make such mistakes is not too high in order to avoid random behavior. This result appears to be consistent with previous studies in the literature, in which agents equipped with simpler strategic mechanisms, such as imitation (Weidenholzer, 2010), random walks (Macy & Flache, 2002), or the use of heuristics (Belloc et al., 2019), are able to coordinate their actions to achieve the most efficient outcome.

The main limit of the models presented is that they are classical game-theoretical models where players have to decide instantaneously their strategy with no information about the opponent’s intentions. In the experimental setting described in Section 3.3, each player decided whether to choose Stag or Hare while being able to observe the direction taken by their partner. Information about the behavior of other players can certainly have an impact on decision-making processes, in addition to the learning and evaluation abilities of each individual player that we investigated in this work. For this reason, we believe that future extensions of this model should integrate the role of such information and different game mechanics, such as the absence of simultaneous choices, in the decision-making processes of agents. Furthermore, other extensions could consider multi-strategy frameworks, where agents are not limited to two strategies but have the option, for example, to not play. Finally, in this study, we assumed a constant and uniform error probability and a learning mechanism separate from the former, finding that a combination of low learning rates and low error probabilities promotes coordination toward the social optimum. However, as suggested by some empirical studies on coordination (Bilancini et al., 2021; Mäs & Nax, 2016), human error does not appear to be constant but rather sensitive to changes in payoffs, expected costs, and past successes, outlining a more complex and structured dynamic. Future extensions could therefore integrate the error probability and learning into a single adaptive mechanism, allowing agents to change their propensity to make errors through experience.