On the Nash Equilibria of a Duel with Terminal Payoffs

Abstract

We formulate and study a two-player duel game as a terminal payoffs stochastic game. Players $P_1,P_2$ are standing in place and, in every turn, each may shoot at the other (in other words, abstention is allowed). If $P_n$ shoots $P_m$ ($m\ne n$), either they hit and kill them (with probability $p_n$) or they miss and $P_m$ is unaffected (with probability $1-p_n$). The process continues until at least one player dies; if no player ever dies, the game lasts an infinite number of turns. Each player receives a positive payoff upon killing their opponent and a negative payoff upon being killed. We show that the unique stationary equilibrium is for both players to always shoot at each other. In addition, we show that the game also possesses “cooperative” (i.e., non-shooting) non-stationary equilibria. We also discuss a certain similarity that the duel has to the iterated Prisoner’s Dilemma.

1. Introduction

In this paper, we study a two-player duel game played in turns. Players $P_1,P_2$ are standing in place and, in each turn, each player may shoots at the other; in other words, abstention is allowed. If $P_n$ shoots at $P_m$ ($m\ne n$), either they hit and kill them or they miss and $P_m$ is unaffected; the respective probabilities are $p_n$ ($P_n$’s marksmanship) and $1-p_n$. The process continues until at least one player dies; if no player ever dies then the game lasts an infinite number of turns. We formulate the above as a stochastic game with terminal payoffs. The precise game rules and players’ payoffs will be presented in Section 2.

Little work has been done on the duel. In fact, to the best of our knowledge, it has only been studied as a preliminary step in the study of the “truel”, in which three stationary players shoot at each other. In early works on the truel [1,2,3,4], the postulated game rules guarantee the existence of exactly one survivor (“winner”). In an important early paper [5], the somewhat paradoxical result of “survival of the weakest” is established; namely for certain marksmanship combinations, the player with lowest marksmanship has the highest probability of survival. A more general analysis appears in a further study [6], which considers the possibility of “cooperation” between the players, in the sense that each player has the option of abstaining, i.e., not shooting at their opponent in one or more turns of the game. This idea is further studied by Kilgour (for the simultaneous truel) [7] and (the sequential truel) [8,9]. These papers are, to the best of our knowledge, the first to address the truel problem using a rigorous game theoretic analysis. Kilgour formulates both the simultaneous and sequential truel as stochastic games with terminal payoffs (i.e., the players receive a single payoff at the end of the game) and obtains Nash equilibria, under appropriate conditions. A similar analysis appears in a further study [10], where, however, the truel is formulated as a discounted stochastic game. Recent papers on the truel include: Refs. [11,12,13,14] where, among other innovations, the truel is formulated as an extensive form game; Refs. [15,16,17,18], where a Markov chain formulation of several truel variants is presented; and Refs. [19,20,21], in which truels among N players are studied, with each player being represented by a node in a scale-free network.1

Several applications of the duel and, more frequently, of the truel have been proposed in the above literature. The truel has been used to model behavior in confrontation situations [25] and in political conflicts [26]. A truel variant has been used as a model of opinion dissemination [17]. Business applications have been presented in a further study [27], in which it is shown that, under certain conditions, weaker companies can grow stronger and stronger companies can grow weaker with all the parties eventually converging. In legal studies, the truel has been used to explore equality issues [28]. Last but not least, the nuel (an N-person generalization of the duel and truel) has been used in biology to explain the maintenance of variation in natural populations [29] and study marriage and reproduction mechanisms [30]. Furthermore, the truel is relevant to the existence of “suicidal strategies” employed by cells and bacteria [31,32].

A common characteristic of all the above-mentioned works is that they limit themselves to the study of stationary strategies. As we will show in the current paper, the duel also possesses Nash equilibria in non-stationary strategies and it is safe to assume that the same is true of the truel and the nuel (the N-player generalization of the duel and truel).

While the above papers focus on various forms of the truel, we believe that the duel is interesting in its own right and has not received the attention it deserves. In particular we will show that, under our formulation, the duel has a certain similarity to the iterated Prisoner’s Dilemma (IPD) and possesses “cooperative” Nash equilibria in non-stationary strategies.

In this paper, we study two versions of the duel with terminal payoffs. The rest of the paper is structured as follows. In Section 2, we define the game rigorously. In Section 3 we establish the existence of equilibria in stationary strategies. In Section 4, we discuss some similarities between the game and the IPD. In Section 5, we prove that the duel also has equilibria in non-stationary strategies (namely grim cooperation and Tit-for-Tat). In Section 6, we summarize our results and propose some future research directions.

2. Game Description

Our duel game involves players $P_1,P_2$ and evolves in discrete time steps (turns) $t\in \left\{1,2,\dots \right\}$. The state at time t is

$$\mathbf{s}\left(t\right)=s_1\left(t\right)s_2\left(t\right)\in S=\left\{11,10,01,00\right\}.$$

For $n\in \left\{1,2\right\}$, $s_n\left(t\right)$ is $P_n$’s state at $t\in \left\{0,1,2,\dots \right\}$ and can be

$$\begin{array}{cc}{s}_n\left(t\right)=1:\hfill & \mathrm{when}{P}_n\mathrm{is}\mathrm{alive}\mathrm{at}\mathrm{the}t\mathrm{-th}\mathrm{turn};\hfill \\ s_n\left(t\right)=0:\hfill & \mathrm{when}{P}_n\mathrm{is}\mathrm{dead}\mathrm{at}\mathrm{the}t\mathrm{-th}\mathrm{turn}.\hfill \end{array}$$

$P_n$’s action at $t\in \left\{1,2,\dots \right\}$ is $f_n\left(t\right)$, which can be $\mathbf{F}$ ($P_n$ is shooting) or $\mathbf{A}$ ($P_n$ is not shooting). If $f_n\left(t\right)=\mathbf{F}$ then: (a) we have $s_{-n}\left(t\right)=0$ (i.e., $P_{-n}$ dies)2 with probability $p_n\in \left(0,1\right)$ and (b)$s_{-n}\left(t\right)=1$ with probability $1-p_n$. We set $\mathbf{f}\left(t\right)=f_1\left(t\right)f_2\left(t\right)$ and $\mathbf{p}=\left(p_1,p_2\right)$. We assume throughout the paper that for $n\in \left\{1,2\right\}$, $p_n\in \left(0,1\right)$, i.e., it is strictly between zero and one.

The game starts at an initial state $\mathbf{s}\left(0\right)$; obviously, the case of interest is $\mathbf{s}\left(0\right)=11$. At times $t\in \left\{1,2,\dots \right\}$, the players simultaneously choose their actions $f_1\left(t\right)$, $f_2\left(t\right)$ and the game moves to state $\mathbf{s}\left(t\right)$ according to the conditional state transition probability $Pr\left(\mathbf{s}\left(t\right)|\mathbf{s}\left(t-1\right),\mathbf{f}\left(t\right)\right)$. If we number the states as follows

$$00\to 1,01\to 2,10\to 3,11\to 4,$$

then we get a “controlled” transition probability matrix $\Pi \left(\mathit{\varphi }\right)$ where

$${\Pi }_{ij}\left(\mathit{\varphi }\right)=Pr\left(\mathbf{s}\left(t\right)=j|\mathbf{s}\left(t-1\right)=i,\mathbf{f}=\mathit{\varphi }\right).$$

For every action vector, a terminal state $\mathbf{s}\in \left\{1,2,3\right\}$ (or, equivalently, $\mathbf{s}\in \left\{00,01,10\right\}$) transits to itself with probability one; i.e., for all $i\in \left\{1,2,3\right\}$:

$$\begin{array}{cc}\hfill {\Pi }_{ii}\left(\mathbf{AA}\right)& =Pr\left(\mathbf{s}\left(t\right)=i|\mathbf{s}\left(t-1\right)=i,\mathbf{f}=\mathbf{AA}\right)=1;\hfill \\ \hfill {\Pi }_{ii}\left(\mathbf{AF}\right)& =Pr\left(\mathbf{s}\left(t\right)=i|\mathbf{s}\left(t-1\right)=i,\mathbf{f}=\mathbf{AF}\right)=1;\hfill \\ \hfill {\Pi }_{ii}\left(\mathbf{FA}\right)& =Pr\left(\mathbf{s}\left(t\right)=i|\mathbf{s}\left(t-1\right)=i,\mathbf{f}=\mathbf{FA}\right)=1;\hfill \\ \hfill {\Pi }_{ii}\left(\mathbf{FF}\right)& =Pr\left(\mathbf{s}\left(t\right)=i|\mathbf{s}\left(t-1\right)=i,\mathbf{f}=\mathbf{FF}\right)=1.\hfill \end{array}$$

Transitions from the state $\mathbf{s}=4$ (or $\mathbf{s}=11$) are a little more complicated. Consider, for example, the case $\mathbf{f}=\mathbf{FF}$, i.e., when both players fire. Then, letting ${\overline{p}}_n=1-p_n$ for $n\in \left\{1,2\right\}$, we have:

$$\begin{array}{cc}\hfill {\Pi }_{41}\left(\mathbf{FF}\right)& =Pr\left(\mathbf{s}\left(t\right)=00|\mathbf{s}\left(t-1\right)=11,\mathbf{f}=\mathbf{FF}\right)=Pr\left(“P_1\mathrm{hits}{P}_2,P_2\mathrm{hits}{P}_1”\right)=p_1{p}_2;\hfill \\ \hfill {\Pi }_{42}\left(\mathbf{FF}\right)& =Pr\left(\mathbf{s}\left(t\right)=01|\mathbf{s}\left(t-1\right)=11,\mathbf{f}=\mathbf{FF}\right)=Pr\left(“P_1\mathrm{misses}{P}_2,P_2\mathrm{hits}{P}_1”\right)={\overline{p}}_1{p}_2;\hfill \\ \hfill {\Pi }_{43}\left(\mathbf{FF}\right)& =Pr\left(\mathbf{s}\left(t\right)=10|\mathbf{s}\left(t-1\right)=11,\mathbf{f}=\mathbf{FF}\right)=Pr\left(“P_1\mathrm{hits}{P}_2,P_2\mathrm{misses}{P}_1”\right)=p_1{\overline{p}}_2;\hfill \\ \hfill {\Pi }_{44}\left(\mathbf{FF}\right)& =Pr\left(\mathbf{s}\left(t\right)=11|\mathbf{s}\left(t-1\right)=11,\mathbf{f}=\mathbf{FF}\right)=Pr\left(“P_1\mathrm{misses}{P}_2,P_2\mathrm{misses}{P}_1”\right)={\overline{p}}_1{\overline{p}}_2.\hfill \end{array}$$

The elements of the other matrices are computed similarly, yielding

$$\begin{array}{ccc}\Pi \left(\mathbf{AA}\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\hfill & & \Pi \left(\mathbf{AF}\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& p_2& 0& {\overline{p}}_2\end{array}\right]\hfill \\ & & \\ \Pi \left(\mathbf{FA}\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& p_1& {\overline{p}}_1\end{array}\right]\hfill & & \Pi \left(\mathbf{FF}\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ p_1{p}_2& {\overline{p}}_1{p}_2& p_1{\overline{p}}_2& {\overline{p}}_1{\overline{p}}_2\end{array}\right]\hfill \end{array}$$

From the above matrices (or from game rules), we see that there exist two possibilities.

• The game stays in state 11 ad infinitum (no player is ever killed);
• At some $t^{\prime }$ the game moves to a state $\mathbf{s}\left(t^{\prime }\right)\in \left\{10,01,00\right\}$ (one or both players are killed). These are terminal states, i.e., as soon as they are reached, the game terminates.

When the game reaches a terminal state $\mathbf{s}$, $P_n$ ($n\in \left\{1,2\right\}$) receives payoff $q_n\left(\mathbf{s}\right)$ as follows:

$$\begin{array}{ccc}{q}_1\left(10\right)=a_1\hfill & & q_2\left(01\right)=-b_2\hfill \\ q_1\left(01\right)=-b_1\hfill & & q_2\left(01\right)=a_2\hfill \\ q_1\left(00\right)=a_1-b_1\hfill & & q_2\left(00\right)=a_2-b_2\hfill \end{array}$$

where we assume that for $n\in \left\{1,2\right\}$, $a_n>0$ and $b_n>0$. We set $\mathbf{a}=\left(a_1,a_2\right)$ and $\mathbf{b}=\left(b_1,b_2\right)$.

A finite history is a sequence $h=\mathbf{s}\left(0\right)\mathbf{f}\left(1\right)\mathbf{s}\left(1\right)\dots \mathbf{f}\left(T\right)\mathbf{s}\left(T\right)$, a non-terminal finite history is an $h=\mathbf{s}\left(0\right)\mathbf{f}\left(1\right)\mathbf{s}\left(1\right)\dots \mathbf{f}\left(T\right)\mathbf{s}\left(T\right)$ where $\mathbf{s}\left(T\right)=11$ and an infinite history is an $h=\mathbf{s}\left(0\right)\mathbf{f}\left(1\right)\mathbf{s}\left(1\right)\dots$. An admissible history is one which conforms to the game rules; the set of all admissible finite (resp. infinite) histories is denoted by $H^{*}$ (resp. $H^{\infty }$); ${\overline{H}}^{*}$ denotes the set of all non-terminal finite histories. The set of all histories is $H=H^{*}\cup H^{\infty }$. It will be useful to define payoff as a function ${\mathrm{Q}}_n:H\to \mathbb{R}$ as follows

$${\mathrm{Q}}_n\left(h\right)=\left\{\begin{array}{cc}{q}_n\left(\mathbf{s}\left(T\right)\right)\hfill & \mathrm{if}h=\mathbf{s}\left(0\right)\mathbf{f}\left(1\right)\mathbf{s}\left(1\right)\dots \mathbf{f}\left(T\right)\mathbf{s}\left(T\right)\in H^{*},\mathbf{s}\left(T\right)\mathrm{is}\mathrm{terminal},\hfill \\ 0\hfill & \mathrm{if}h=\mathbf{s}\left(0\right)\mathbf{f}\left(1\right)\mathbf{s}\left(1\right)\dots \mathbf{f}\left(T\right)\mathbf{s}\left(T\right)\in H^{*},\mathbf{s}\left(T\right)\mathrm{is}\mathrm{non-terminal},\hfill \\ 0\hfill & \mathrm{if}h\in H^{\infty }\hfill \end{array}\right.$$

Note that if the game never terminates, both players receive zero payoff.

A strategy for $P_n$ is a function ${\sigma }_n:{\overline{H}}^{*}\to \left[0,1\right]$; it corresponds to, for every non-terminal finite history h, the probability that, given that the current history is h, $P_n$ will shoot $P_{-n}$:

$${\sigma }_n\left(h\right)=Pr\left("P_n\mathrm{shoots}{P}_{-n}"\right).$$

A stationary strategy is a ${\sigma }_n$ depending only on the current state $\mathbf{s}$, hence we simply write ${\sigma }_n\left(\mathbf{s}\right)$. Since a stationary strategy ${\sigma }_n$ depends only on the current state, it is fully determined by the values ${\sigma }_n\left(\mathbf{s}\right)$ for $\mathbf{s}\in \left\{00,01,10,11\right\}$, i.e., from

$${\sigma }_n\left(00\right),{\sigma }_n\left(01\right),{\sigma }_n\left(10\right),{\sigma }_n\left(11\right).$$

But any admissible strategy (i.e., compatible with the game rules) must assign

$${\sigma }_n\left(00\right)={\sigma }_n\left(01\right)={\sigma }_n\left(10\right)=0.$$

Consequently, a stationary strategy is determined by a single number $x_n={\sigma }_n\left(11\right)$.

A strategy profile is a vector $\sigma =\left({\sigma }_1,{\sigma }_2\right)$. We denote the set of all admissible strategies by $\Sigma$ and the set of all admissible stationary strategies by $\overline{\Sigma }$.

An initial state $\mathbf{s}\left(0\right)$ and two strategies ${\sigma }_1$ and ${\sigma }_2$ (used, respectively, by $P_1$ and $P_2$) determine a probability measure on the set of all histories; hence we can define the expected payoffs

$$\forall n\in \left\{1,2\right\}{:}_{}{Q}_n\left(\mathbf{s}\left(0\right),{\sigma }_1,{\sigma }_2\right)={\mathbb{E}}_{\mathbf{s}\left(0\right),{\sigma }_1,{\sigma }_2}\left({\mathsf{Q}}_n\left(h\right)\right).$$

We have, thus, formulated the terminal payoffs duel as a game. We are interested in the game that starts at $\mathbf{s}\left(0\right)=11$, which we will denote by $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$. We assume that $P_1$ and $P_2$ are looking for a Nash equilibrium (NE), i.e., a strategy profile $\left({\widehat{\sigma }}_1,{\widehat{\sigma }}_2\right)$ such that

$$\forall n\in \left\{1,2\right\},\forall {\sigma }_n\in \Sigma :Q_n\left(\left(1,1\right),{\widehat{\sigma }}_n,{\widehat{\sigma }}_{-n}\right)\ge Q_n\left(\left(1,1\right),{\sigma }_n,{\widehat{\sigma }}_{-n}\right).$$

3. Stationary Equilibria

As already noted, an admissible stationary strategy ${\sigma }_1$ for $P_1$ is fully determined by $x_1={\sigma }_1\left(11\right)=Pr\left(P_1\mathrm{shoots}{P}_2\right)$; i.e., ${\sigma }_1$ is determined by a single variable $x_1\in \left[0,1\right]$. Similarly, every admissible stationary strategy ${\sigma }_2$ for $P_2$ is fully determined by a single variable $x_2\in \left[0,1\right]$. Hence, we will often speak of the strategy $x_n$ (rather than ${\sigma }_n$) and the strategy profile $\left(x_1,x_2\right)$ (rather than $\left({\sigma }_1,{\sigma }_2\right)$). When $P_1$ and $P_2$ use strategies $x_1$ and $x_2$, the state sequence is a Markov chain; using the previous numbering of states we have the transition probability matrix

$$\Pi \left(x_1,x_2\right)=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ x_1{p}_1{x}_2{p}_2& \left(x_1{\overline{p}}_1+{\overline{x}}_1\right)x_2{p}_2& x_1{p}_1\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)& \left(x_1{\overline{p}}_1+{\overline{x}}_1\right)\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)\end{array}\right]$$

Also, we can define

$$V_n\left(x_1,x_2\right)=Q_n\left(11,\left(x_1,x_2\right)\right).$$

If $\left(x_1,x_2\right)=\left(0,0\right)$ we obviously get $V_1\left(0,0\right)=0$. Conversely, if $\left(x_1,x_2\right)\ne \left(0,0\right)$ then we have the following equation for $V_1$ (temporarily omitting the $x_1,x_2$ arguments for brevity of notation):

$$\begin{array}{c}{V}_1=x_1{p}_1\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)a_1-\left(x_1{\overline{p}}_1+{\overline{x}}_1\right)x_2{p}_2{b}_1\\ +x_1{p}_1{x}_2{p}_2\left(a_1-b_1\right)+\left({\overline{x}}_1+x_1{\overline{p}}_1\right)\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)V_1.\end{array}$$

The equation is obtained as follows: the expected payoff from state 11 is the sum of four terms:

• The transition to state 10 gives payoff $a_1$ and takes place with probability $x_1{p}_1$ ($P_1$ shot and hit $P_2$) multiplied by $\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)$ ($P_2$ either shot and missed or did not shoot);
• The transition to state 01 gives payoff $-b_1$ and takes place with probability $x_2{p}_2$ ($P_2$ shot and hit $P_1$) multiplied by $\left(x_1{\overline{p}}_1+{\overline{x}}_1\right)$ ($P_1$ either shot and missed or did not shoot);
• The transition to state 00 gives payoff $a_1-b_1$ and takes place with probability $x_1{p}_1$ ($P_1$ shot and hit $P_2$) multiplied by $x_2{p}_2$ ($P_2$ shot and hit $P_1$);
• The transition to state 11 gives payoff $V_1$ (it is as if the game starts from the beginning) and takes place with probability $\left({\overline{x}}_1+x_1{\overline{p}}_1\right)$ ($P_1$ either shot and missed or did not shoot) multiplied by $\left(x_2{\overline{p}}_2+{\overline{x}}_2\right)$ ($P_2$ either shot and missed or did not shoot).

After some algebra, the $V_1$ equation is simplified to

$$V_1=x_1{p}_1{a}_1-x_2{p}_2{b}_1+V_1{x}_1{p}_1{x}_2{p}_2-V_1{x}_1{p}_1-V_1{x}_2{p}_2+V_1$$

and has the following solution3:

$$V_1\left(x_1,x_2\right)=\frac{x_1{p}_1{a}_1-x_2{p}_2{b}_1}{1-\left(1-x_1{p}_1\right)\left(1-x_2{p}_2\right)}.$$

By the same analysis for $V_2\left(x_1,x_2\right)$, we finally get the expressions

$$\begin{array}{cc}\hfill V_1\left(x_1,x_2\right)& =\left\{\begin{array}{cc}0\hfill & \mathrm{if}{x}_1=x_2=0\hfill \\ \frac{x_1{p}_1{a}_1-x_2{p}_2{b}_1}{1-\left(1-x_1{p}_1\right)\left(1-x_2{p}_2\right)}\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill V_2\left(x_1,x_2\right)& =\left\{\begin{array}{cc}0\hfill & \mathrm{if}{x}_1=x_2=0\hfill \\ \frac{x_2{p}_2{a}_2-x_1{p}_1{b}_2}{1-\left(1-x_1{p}_1\right)\left(1-x_2{p}_2\right)}\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(2)

Proposition 1.

The only stationary NE of $\Gamma (\mathbf{a},\mathbf{b},\mathbf{p})$ is $\left(x_1,x_2\right)=\left(1,1\right)$.

Proof. 

Suppose that $P_1$ and $P_2$ use the profile $\left(x_1,x_2\right)$. To determine whether this is an NE, from $P_1$’s point of view we have to check whether they have anything to gain by unilaterally deviating to some other strategy ${\sigma }_1$. A crucial fact is that we only have to check whether $P_1$ gains by switching to another stationary strategy. This is true because, if $P_2$ uses the stationary strategy $x_2$, then $P_1$ must solve an Markov Decision Process problem; it is well known that in this case he gains nothing by using non-stationary strategies [33].

Let us first check whether $\left(0,0\right)$ is a Nash equilibrium. If $P_1$ deviates to another stationary strategy $x_1$, we will have

$$V_1\left(0,0\right)-V_1\left(x_1,0\right)=0-\frac{x_1{p}_1{a}_1-0p_2{b}_1}{1-\left(1-x_1{p}_1\right)\left(1-0p_2\right)}=-a_1<0.$$

Hence, $\left(0,0\right)$ cannot be an NE. Next, take any $\left(x_1,x_2\right)\ne \left(0,0\right)$ and suppose $P_1$ deviates to $y_1$. Then

$$\begin{array}{cc}\hfill & V_1\left(x_1,x_2\right)-V_1\left(y_1,x_2\right)\hfill \\ \hfill & =\frac{x_1{p}_1{a}_1-x_2{p}_2{b}_1}{1-\left(1-x_2{p}_2\right)\left(1-x_1{p}_1\right)}-\frac{y_1{p}_1{a}_1-x_2{p}_2{b}_1}{1-\left(1-x_2{p}_2\right)\left(1-y_1{p}_1\right)}\hfill \\ \hfill & =\frac{p_1{x}_2{p}_2\left(a_1+b_1\left(1-x_2{p}_2\right)\right)\left(x_1-y_1\right)}{\left(1-\left(1-x_1{p}_1\right)\left(1-x_2{p}_2\right)\right)\left(1-\left(1-y_1{p}_1\right)\left(1-x_2{p}_2\right)\right)}\hfill \end{array}$$

The denominator is positive. The numerator has the sign of $x_1-y_1$. Hence, the sign of $V_1\left(x_1,x_2\right)-V_1\left(y_1,x_2\right)$ is the same as that of $x_1-y_1$ and consequently, $P_1$ never (resp. always) has an incentive to deviate from $x_1$ to a smaller (resp. greater)$y_1$. The same arguments can be applied to $P_2$ and their strategy $x_2$. It follows that the only stationary NE is $\left(x_1,x_2\right)=\left(1,1\right)$ and this completes the proof. □

4. Connection to the Iterated Prisoner’s Dilemma

Applying Formulas (1) and (2) to $\left(x_1,x_2\right)\in \left\{\left(0,0\right),\left(0,1\right),\left(1,0\right),\left(1,1\right)\right\}$, we get

$$\begin{array}{ccc}{V}_1\left(0,0\right)=0\hfill & & V_2\left(0,0\right)=0\hfill \\ V_1\left(0,1\right)=-b_1\hfill & & V_2\left(0,1\right)=a_2\hfill \\ V_1\left(1,0\right)=a_1\hfill & & V_2\left(1,0\right)=-b_2\hfill \\ V_1\left(1,1\right)=\frac{p_1{a}_1-p_2{b}_1}{1-\left(1-p_2\right)\left(1-p_1\right)}\hfill & & V_2\left(1,1\right)=\frac{p_2{a}_2-p_1{b}_2}{1-\left(1-p_2\right)\left(1-p_1\right)}\hfill \end{array}$$

It can immediately be seen that

$$-b_1=V_1\left(0,1\right)<0=V_1\left(0,0\right)<a_1=V_1\left(1,0\right)$$

and if we identify the strategy $x_n=0$ (never shooting at the opponent) with “cooperation” and the strategy $x_n=1$ (always shooting at the opponent) with “defection”, the above inequalities remind us of the Prisoner’s Dilemma (PD). The similarity would be complete if the additional inequalities

$$V_1\left(0,1\right)<V_1\left(1,1\right)<V_1\left(0,0\right)$$

also held; because in this case we would have

$$V_1\left(0,1\right)<V_1\left(1,1\right)<V_1\left(0,0\right)<V_1\left(1,0\right)$$

(3)

which corresponds exactly to the well known sequence of PD inequalities [22]:

$$S<P<R<T.$$

Now, (3) is equivalent to

$$-b_1<\frac{p_1{a}_1-p_2{b}_1}{1-\left(1-p_2\right)\left(1-p_1\right)}<0<a_1.$$

The first inequality is equivalent to

$$0<\frac{p_1{a}_1-p_2{b}_1}{1-\left(1-p_2\right)\left(1-p_1\right)}+b_1=p_1\frac{a_1+b_1\left(1-p_2\right)}{p_1+p_2\left(1-p_1\right)}$$

which is always satisfied. The second inequality is

$$\frac{p_1{a}_1-p_2{b}_1}{1-\left(1-p_2\right)\left(1-p_1\right)}<0,$$

which will be satisfied iff

$$p_1{a}_1<p_2{b}_1$$

The third inequality is always satisfied. Similarly, the inequalities

$$V_2\left(0,1\right)<V_2\left(1,1\right)<V_2\left(0,0\right)<V_2\left(1,0\right)$$

(4)

will be satisfied iff

$$p_2{a}_2<p_1{b}_2$$

Combining the above, we get the following “PD-like condition”

$$\frac{a_2}{b_2}<\frac{p_1}{p_2}<\frac{b_1}{a_1}$$

(5)

which is necessary and sufficient to have the following ordering of the payoffs

$$\begin{array}{cc}\hfill V_1\left(0,1\right)& <V_1\left(1,1\right)<V_1\left(0,0\right)<V_1\left(1,0\right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill V_2\left(0,1\right)& <V_2\left(1,1\right)<V_2\left(0,0\right)<V_2\left(1,0\right)\hfill \end{array}$$

(7)

In light of (6) and (7), we will call the never-shooting strategy $x_n=0$ (which henceforth will also be denoted by ${\sigma }^C$) the cooperating strategy, and the always-shooting strategy $x_n=1$ (which henceforth will also be denoted by ${\sigma }^D$) the defecting strategy. The terminology is inspired by the analogy to the PD. Namely, in both the PD and the duel, both players would have a higher payoff if they adhered to $\left({\sigma }^C,{\sigma }^C\right)$; but this is not a NE and each player has incentive to switch to ${\sigma }^D$. Consequently, rational players will follow the strategy profile $\left({\sigma }^D,{\sigma }^D\right)$, which, while being an NE, yields lower payoff to both players.4

As is well known, cooperative NE do exist for the iterated PD, and these involve the use of non-stationary strategies, such as grim-cooperation and Tit-for-Tat (TfT). Hence, in the next section, we will show that there exist corresponding non-stationary cooperative strategies which are NE of $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$.

Before concluding this section, it is worth discussing in what ways our duel game $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$ differs from the IPD. Three obvious differences are:

• The IPD is a deterministic game, while $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$ involves randomness;
• In the IPD, each player receives a payoff in every turn and the total payoff is the discounted (by a discount factor $\gamma$) sum of turn payoffs, while in $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$, payoff is obtained only at the final turn and is undiscounted;
• The IPD will last an infinite number of turns, while $\Gamma \left(\mathbf{p},\mathbf{a},\mathbf{b}\right)$ may (depending on the p values and the strategy used) terminate in a finite number of turns (in fact, it may be the case that it will terminate in a finite number of terms with probability one).

However, there is an formulation of the IPD in which the payoffs are not discounted but the game may terminate in every turn with a positive probability $p=1-\gamma >0$. In this formulation, the IPD is also a random game and will terminate in a finite number of turns with probability one; the total expected payoff of each player equals the discounted payoff of the deterministic IPD version.

5. Non-Stationary Equilibria

Drawing upon similar results for the IPD, we will now show that the duel has cooperative NE in non-stationary strategies. The first such strategy we introduce is the grim cooperation strategy ${\sigma }^G$, which is defined as follows for $P_n$ ($n\in \left\{1,2\right\}$):

$${\sigma }^G:\begin{array}{c}\mathrm{As}\mathrm{long}\mathrm{as}{P}_{-n}\mathrm{does}\mathrm{not}\mathrm{shoot}{P}_n,P_n\mathrm{never}\mathrm{shoots}{P}_{-n};\hfill \\ \mathrm{if}{P}_{-n}\mathrm{shoots}{P}_n\mathrm{at}\mathrm{round}t,\mathrm{then}{P}_n\mathrm{shoots}{P}_{-n}\mathrm{at}\mathrm{all}\mathrm{rounds}{t}^{\prime }>t.\hfill \end{array}$$

This strategy was originally used in the analysis of the IPD.

Proposition 2.

$\left({\sigma }^G,{\sigma }^G\right)$ is an NE of $\Gamma \left(\mathbf{a},\mathbf{b},\mathbf{p}\right)$ iff

$$\frac{b_1}{a_1}>\frac{1+p_2\left(1-p_1\right)}{1-p_1}·\frac{p_1}{p_2}\mathit{and}\frac{b_2}{a_2}>\frac{1+p_1\left(1-p_2\right)}{1-p_2}·\frac{p_2}{p_1}.$$

(8)

Proof. 

We have

$$V_1^G=Q_1\left(11,\left({\sigma }^G,{\sigma }^G\right)\right)=0$$

since, if both players adhere to ${\sigma }^G$, nobody will ever get killed. Next, let us consider possible $P_1$ strategies ${\sigma }_1$ deviating from ${\sigma }^G$. It is easy to see that it suffices to consider the strategy ${\sigma }^D$, because, as soon as $P_1$ deviates from ${\sigma }^G$, $P_2$ will shoot at $P_1$ on every turn and hence, $P_1$ has no incentive to not shoot; furthermore, if $P_1$ deviates from ${\sigma }^G$, they might as well deviate on the first turn. Now, let us compute

$$V_1^R=Q_1\left(11,\left({\sigma }^D,{\sigma }^G\right)\right).$$

If $P_1$ uses ${\sigma }^D$ at $t=1$, then $P_2$ will also revert to ${\sigma }^D$ at times $t\in \left\{2,3,\dots \right\}$. Hence, $P_1$’s expected payoff will be

$$\begin{array}{cc}\hfill V_1^R& =p_1{a}_1+\left(1-p_1\right)\left(0+V_1^D\right)\hfill \\ \hfill & =p_1{a}_1+\left(1-p_1\right)\frac{p_1{a}_1-p_2{b}_1}{1-\left(1-p_2\right)\left(1-p_1\right)}\hfill \\ \hfill & =\frac{p_2\left(1-p_1\right)\left(p_1{a}_1-b_1\right)+p_1{a}_1}{1-\left(1-p_1\right)\left(1-p_2\right)}.\hfill \end{array}$$

For $\left({\sigma }^G,{\sigma }^G\right)$ to be an NE, we must have $V_1^R$$<V_1^G$, which is equivalent to

$$p_2\left(1-p_1\right)\left(p_1{a}_1-b_1\right)+p_1{a}_1<0.$$

(9)

By assumption

$$\begin{array}{cc}\hfill \frac{1+p_2\left(1-p_1\right)}{p_2\left(1-p_1\right)}{p}_1{a}_1-b_1& <0\iff \hfill \\ \hfill \frac{-b_1{p}_2+b_1{p}_2{p}_1+p_1{a}_1+p_1{a}_1{p}_2-p_1^2{a}_1{p}_2}{p_2\left(1-p_1\right)}& <0\iff \hfill \\ \hfill -b_1{p}_2+b_1{p}_2{p}_1+p_1{a}_1+p_1{a}_1{p}_2-p_1^2{a}_1{p}_2& <0\iff \hfill \\ \hfill p_2\left(1-p_1\right)\left(p_1{a}_1-b_1\right)+p_1{a}_1& <0.\hfill \end{array}$$

Hence, (9) holds and $P_1$ has no incentive to deviate from ${\sigma }^G$. By a similar analysis, we can also show that $P_2$ has no incentive to deviate from ${\sigma }^G$. This completes the proof. □

Remark 1.

The duel NE conditions (8) imply

$$\begin{array}{cc}\hfill \frac{b_1}{a_1}& >\frac{1+p_2\left(1-p_1\right)}{1-p_1}·\frac{p_1}{p_2}\Rightarrow \frac{p_1}{p_2}<\frac{1-p_1}{1+p_2\left(1-p_1\right)}·\frac{b_1}{a_1}<\frac{b_1}{a_1},\hfill \\ \hfill \frac{b_2}{a_2}& >\frac{1+p_1\left(1-p_2\right)}{1-p_2}·\frac{p_2}{p_1}\Rightarrow \frac{p_2}{p_1}<\frac{1-p_2}{1+p_1\left(1-p_2\right)}·\frac{b_2}{a_2}<\frac{b_2}{a_2}.\hfill \end{array}$$

Hence, the conditions (8) are stronger than the originally postulated condition (5) for the existence of a “PD-like” ordering in the duel.

Now, we will define another non-stationary cooperative strategy, which will turn out to be an NE of the duel. This is the Tit-for-Tat strategy ${\sigma }^{TfT}$, defined for $P_n$ ($n\in \left\{1,2\right\}$) as follows:

$${\sigma }^{TfT}:\begin{array}{c}\mathrm{In}\mathrm{the}\mathrm{first}\mathrm{turn}{P}_n\mathrm{does}\mathrm{not}\mathrm{shoot}{P}_{-n};\hfill \\ \mathrm{at}\mathrm{every}\mathrm{other}\mathrm{turn}{P}_n\mathrm{performs}\mathrm{the}\mathrm{same}\mathrm{action}(\mathrm{shooting}\mathrm{or}\mathrm{not}\mathrm{shooting})\hfill \\ \mathrm{that}{P}_{-n}\mathrm{performed}\mathrm{in}\mathrm{the}\mathrm{previous}\mathrm{round}.\hfill \end{array}$$

This strategy was also originally used in the analysis of the iterated PD.

Proposition 3.

$\left({\sigma }^{TfT},{\sigma }^{TfT}\right)$ is an NE of $\Gamma \left(\mathbf{a},\mathbf{b},\mathbf{p}\right)$ iff

$$\frac{b_1}{a_1}>\frac{1+p_2\left(1-p_1\right)}{1-p_1}·\frac{p_1}{p_2}\mathit{and}\frac{b_2}{a_2}>\frac{1+p_1\left(1-p_2\right)}{1-p_2}·\frac{p_2}{p_1}.$$

(10)

Proof. 

If both players play the strategy ${\sigma }^{TfT}$, then they never shoot at each other and their payoffs are

$$\forall n\in \left\{1,2\right\}:V_n^{TfT}=Q_n\left(11,{\sigma }^{TfT},{\sigma }^{TfT}\right)=Q_n\left(11,{\sigma }^C,{\sigma }^C\right)=0.$$

Now, suppose that $P_2$ adheres to ${\sigma }^{TfT}$ but $P_1$ deviates. If $P_1$ gains by deviating from ${\sigma }^{TfT}$ at some turn, then they must also gain by shooting at $P_2$ in the first turn. If they do so, then $P_2$ shoots at $P_1$ for all subsequent turns, until $P_1$ reverts to not firing. Thus, $P_1$ has two options after their first deviation.

• They can continue shooting in all subsequent turns, in which case, so will $P_2$;
• They can revert to not shooting, in which case, in the next turn, they are in the same situation as at the start of the game.

Consequently, if $P_1$ can increase their payoff by deviating, then they can do so, either (a) by shooting in every turn, or (b) by alternating between shooting and not shooting. If we find conditions under which $P_1$ cannot increase their payoff by either of the above strategies, then, under the same conditions, $P_1$ cannot increase their payoff by deviating, which implies that $\left({\sigma }^{TfT},{\sigma }^{TfT}\right)$ is an NE.

• Consider first the case in which $P_1$ adopts the strategy ${\sigma }^D$ of shooting in each turn. Then we have

  $$Q_n\left(11,{\sigma }^D,{\sigma }^{TfT}\right)=Q_1\left(11,\left({\sigma }^D,{\sigma }^G\right)\right)=V_1^R$$

  and, by the same analysis as in the proof of Proposition 2, we know that $V_1^C-V_1^R>0$ iff

  $$p_2\left(1-p_1\right)\left(p_1{a}_1-b_1\right)+p_1{a}_1<0$$

  which is equivalent to our assumption

  $$\frac{b_1}{a_1}>\frac{1+p_2\left(1-p_1\right)}{1-p_1}·\frac{p_1}{p_2}.$$
• Next consider the case in which $P_1$ alternates between shooting and not shooting. Then their payoff will be

  $$V_1^S=p_1{a}_1+\left(1-p_1\right)\left(0+p_2\left(-b_1\right)+\left(1-p_2\right)V_1^S\right).$$
  The above equation holds because the expected payoff $V_1^S$ is computed by summing the following possibilities. $P_1$ will certainly shoot and then:

  (a)

  With probability $p_1$, $P_2$ will kill $P_2$ and hence, receive payoff $a_1$;

  (b)

  With probability $1-p_1$, $P_2$ will miss (and receive zero payoff) and in the next turn $P_2$ will shoot and kill $P_1$; this combination has probability $\left(1-p_1\right)p_2$ and gives to $P_1$ payoff $-b_1$;

  (c)

  With probability $1-p_1$, $P_1$ will miss and in the next turn $P_2$ will shoot and miss $P_1$; this combination has probability $\left(1-p_1\right)\left(1-p_2\right)$ and returns the game to the original state, in which $P_1$ receives payoff $V_1^S$.

  Simplifying the above equation and solving we obtain

  $$V_1^S=\frac{p_1{a}_1-p_2{b}_1+p_2{b}_1{p}_1}{p_1+p_2-p_2{p}_1}.$$
  For an NE we must have $V_1^C-V_1^S>0$ and this will hold when

  $$0>p_1{a}_1-p_2{b}_1+p_2{b}_1{p}_1\iff p_2{b}_1\left(1-p_1\right)>p_1{a}_1\iff \frac{b_1}{a_1}>\frac{1}{1-p_1}·\frac{p_1}{p_2}.$$
  However, from our assumption (10), we have

  $$\frac{b_1}{a_1}>\frac{1+p_2\left(1-p_1\right)}{1-p_1}·\frac{p_1}{p_2}>\frac{1}{1-p_1}·\frac{p_1}{p_2}.$$
  Hence $V_1^C-V_1^S>0$.

Combining 1 and 2, we see that $P_1$ has no advantage in deviating from ${\sigma }^{TfT}$; by a similar analysis, the same holds for $P_2$ and hence, the proof is completed. □

Corollary 1.

The duel NE conditions (8) and (10) are the same. In other words, $\left({\sigma }^D,{\sigma }^D\right)$ is an NE of $\Gamma \left(\mathbf{a},\mathbf{b},\mathbf{p}\right)$ iff $\left({\sigma }^{TfT},{\sigma }^{TfT}\right)$ is an NE of $\Gamma \left(\mathbf{a},\mathbf{b},\mathbf{p}\right)$.

Let us compare the stationary and non-stationary NE. Initially, we made no assumption regarding the relative size of $a_n$ and $b_n$ (although we did assume they are both positive). In other words, $a_n-b_n$ may be positive ($P_n$ sets more value in surviving), negative ($P_n$ hates their opponent so much that they value killing them more than surviving) or zero. However, even when $a_n>b_n$ for both n, if the players limit themselves to using stationary strategies, then the only Nash equilibrium consists of both players shooting at each other with probability one (by Proposition 1); for the more desirable outcome of both players surviving to be (another) Nash equilibrium, they must use non-stationary strategies.

6. Conclusions

We have defined a turn-based duel game with terminal payoffs and shown that it has both stationary and non-stationary Nash equilibria. The non-stationary equilibria that we have established are the grim cooperation and Tit-for-Tat pairs. These are of the same form as the synonymous strategies used in the iterated Prisoner’s Dilemma; we were motivated to use these in the duel by the previously explained similarity between the payoff structure of our duel game and that of the IPD.

In addition to their independent interest, the above results have potential application to the truel and nuel problems. As we have pointed out, to the best of our knowledge, the literature on truel and nuel is limited to the study of stationary strategies. In the case of the duel, in addition to stationary NE, we also have non-stationary NE. We reported here two such non-stationary NE ($\left({\sigma }^G,{\sigma }^G\right)$ and $\left({\sigma }^{TfT},{\sigma }^{TfT}\right)$), and it is not hard to construct additional ones, using an approach similar to that used in the study of repeated games [35]. We conjecture that, using the methods of the current paper, it is also possible to establish a plethora of non-stationary NE for the general, N-player nuel; we intend to pursue this research direction in the future.

Several variants of the duel can be formulated and are worth exploring. In addition to the variant described in this paper, we have explored a variant in which each player receives some discounted payoff for every turn in which they stay alive. Including those results (and the techniques required for their proof) would increase the size of the current paper inordinately; hence, they will be reported in a separate publication. Further variants to be explored in the future include:

• sequential play, in which a single player is allowed to shoot in each turn;
• random play, in which the player allowed to shoot in each turn is chosen randomly and equi-probably.

In addition, in the future we intend to study the use of non-stationary strategies in truels and nuels.