Applications of Variational Analysis to Strong Nash Equilibria ^†

Abstract

Game theory problems have a wide array of applications and intricate structures. Because of this, there are different types of solution concepts available in the literature. In this work, strong Nash equilibria (a type of solution to game theoretic problems that extends Nash equilibria) are explored. Some new sufficient conditions for the existence of said solutions are put forth. An algorithm is also provided that, when convergent, will lead to a strong Nash equilibrium. Some tests to determine the practical efficiency of the algorithm are included.

1. Introduction

Game theory, pioneered in [1], has many suitable applications with many different frameworks and goals. For example, [2] developed a suitable type of strategy choice for players of non-cooperative games that came to be known as a Nash equilibrium, whereby no player can unilaterally deviate from the strategy to achieve a strictly better result for themself. Nash equilibria are known to exist under certain conditions: one such early example was from [3], where compact Hausdorff pure strategy spaces and continuous cost functions for a game imply the existence of a Nash equilibrium by the extension of the Kakutani fixed-point theorem. A natural extension for cooperative games was made in [4], where the strategy profile is one for which no coalition of players can collectively change their strategies to achieve strictly better results for all members of the coalition. This is known as a strong Nash equilibrium (SNE), and while there are no general results on the existence of an SNE, some work characterizing them for specific enviroments has been carried out. Recently, in [5], some specific game properties enabled the existence of an SNE. which provided some good results. Additionally, algorithms have been created for finding an SNE, as can be found in sources like [6,7]. These algorithms in [6,7] are considerably different to the one found in this paper. A few places where Nash equilibria are explored in relation to real-world problems by utilizing variational analysis are [8,9]. Additionally, another work for characterizing Nash equilibria using the variational inequality was carried out in [10].

The structure of this work proceeds as follows: First, a section on some necessary background information is given. Strong Nash equilibria are shown to be connected to efficiency in a particular case. Then, using the variational inequality, as defined in [10], it is shown that a strategy profile is an SNE when, for every coalition, a form of the variational inequality holds. A simple characterization of an SNE is given for two-player games. Finally, an algorithm is given that produces a sequence of strategies that, when convergent, converge to a strong Nash equilibrium. A simple example of the algorithm in use is then given. Tests of the algorithm are also performed and a summary is provided. A portion of the results were part of the PhD thesis of the first author while studying at Northern Illinois University [11].

2. Preliminaries and Notations

This work uses only real Euclidean vector spaces. The notations and definitions that follow in this section will be used throughout the paper:

Notation 1. 

Given the general vectors $x=(x_1,\dots ,x_n)\in {\mathbb{R}}^n$ and $y=(y_1,\dots ,y_n)\in {\mathbb{R}}^n$, $x=0$ refers to the vector $x=(0,0,\dots ,0)$, $x=y$ implies $x_i=y_i$ for all i values, $x<y$ implies $x_i<y_i$ for all i values, and $x\le y$ implies $x_i\le y_i$ for all i values.

Notation 2. 

For a function $f\left(x\right)$ with input vector $x=(x_1,\dots ,x_n)$ of length n and some set $S\subseteq \{1,\dots ,n\}$, the gradient ${\nabla }_{x_i}f\left(x\right)$ is understood to be the gradient of f restricted to the $x_i$ component of x (more than one component may be listed) and ${\nabla }_{S}f\left(x\right)$ is the gradient restricted to the component $x_j$ where $j\in S$. This rules is used when the gradient of f is to be investigated but certain entries of x are held fixed.

The structure of a problem and the solutions to that problem are as described by the following definition:

Definition 1. 

Let $F\left(x\right)=(f_1\left(x\right),\dots ,f_m\left(x\right))$ be a vector-valued function and $X\subseteq {\mathbb{R}}^n$. A problem is described by

$$\begin{array}{cccc}\hfill \left(P\right)& \mathit{Minimize}:\hfill & \hfill & F\left(x\right)\hfill \\ \hfill & \mathit{Subject}\mathit{to}:\hfill & \hfill & x\in X.\hfill \end{array}$$

This is a multi-objective optimization problem (MOOP). A feasible solution for (P) is any point $x\in X$. An efficient solution (sometimes called the Pareto efficient) for (P) is a feasible solution x with the property stating that there is no other feasible solution $y\in X$ such that $F\left(y\right)\le F\left(x\right)$ and $F\left(y\right)\ne F\left(x\right)$. Identifying all the efficient points of a problem (P) is what is meant by “solving (P)”. Lastly, a feasible solution x is said to be a weakly efficient solution to (P) if there is no other $y\in X$ such that $F\left(y\right)<F\left(x\right)$.

Efficiency first made its début in [12] and then was modernized in [13]. Further formalization that resulted in the form described here was provided in [14]. A good reference on variational analysis is [15].

With regard to game theory, a complete explanation of how a player will react or behave in a game given all the potential possible situations that the player may encounter is called a pure strategy. A mixed strategy is a vector from the simplex space of a dimension equal to the number of pure strategies available to a player, and that vector determines the probability that any of the potential pure strategies will be employed in a game for that player [16]. Mixed strategies are continuous in nature, and so they are the type of strategies under consideration in this work.

Definition 2. 

A game (G) is described by the triplet (G)$=(I,X,F)$ with $I=\{1,\dots ,n\}$ representing the n different players, while the space $X={\prod }_{i=1}^n{X}_i$ is referred to as the strategy space where each $X_i$ is the closed convex simplex ${\Delta }^{p^i-1}\subset {\mathbb{R}}^{p_i}$, which describes the $i^{th}$ player’s collection of mixed strategies to choose from based on the $p_i$ potential pure strategies available to the player. Here, $x_i\in X_i$ is called the $i^{th}$ player’s strategy and $x=(x_1,\dots ,x_n)\in X$ is known as a strategy profile for the game. The function $F=(f_1,\dots ,f_n)$ is a multi-objective function where every $i^{th}$ player is assigned $f_i:X\to \mathbb{R}$ as a cost function that they wish to minimize. These types of games are multi-objective games in the sense that all the players have their own individual objective, not in the sense that each player has multiple objectives that they are tying to minimize for themselves simultaneously.

For the index set $S\subseteq \{1,\dots ,n\}$, which will be thought of as a coalition of players, the function $f_i(y_S,x_{-S})$ is the same as $f_i$ with input x but with the specific values $x_j$ changed to $y_j$ for $j\in S$, where $y\in X_S={\Pi }_{k\in S}{X}_k$. A coalition can comprise any combination of players (including the coalition of all players), and all the members of the coalition will be working together to reduce the cost for all coalition members. For this reason, within a coalition, the multi-objective nature of the problems becomes a group of players working together to minimize all the objectives simultaneously. For clarity, replacement cases where the vectors are denoted with a subscript may have the subscript migrated to a superscript, such as $\nabla f_i({y_k}_S,{x_k}_{-S})=\nabla f_i(y_S^k,x_{-S}^k)$. Also, when only the $i^{th}$ term is to be replaced, we write $f_i(x_i^{*},x_{-i}):=f_i(x_1,\dots ,x_{i-1},x_i^{*},x_{i+1},\dots ,x_n)$.

A Nash equilibrium (NE) is a point $x=(x_1,\dots ,x_n)$, where for all i, there is no $x_i^{*}\in \mathbb{R}$ for which $f_i(x_i^{*},x_{-i})<f_i\left(x\right)$. That is to say, a unilateral strategy change by a single player will not result in a better outcome for that player. A strong Nash equilibrium (SNE) is a point $x=(x_1,\dots ,x_n)$ where, given any coalition $S\subseteq \{1,\dots ,n\}$, there is no other strategy profile $x_S^{*}\in X_S$ for which $f_i(x_S^{*},x_{-S})<f_i\left(x\right)$ for all $i\in S$. So, an SNE is a strategy profile where no coalition can make a group change to their strategies, which would result in a strictly better result for everyone in the coalition, including the coalition of all players.

It is a well-known fact that every SNE is automatically an NE and weakly efficient (consider the coalition of all players).

3. New Results for Strong Nash Equilibria

To begin with, a connection can be made between strong Nash equilibria and the concept of efficiency in a specific case where every player shares a function component underlying their specific composite cost function.

Proposition 1. 

If $x=(x_1,\dots ,x_n)$ is an SNE for the game (G) with $f_i\left(x\right)=g_i\left(h\left(x\right)\right)$ for all i and the functions $g_i$ are linear and non-constant (i.e., $g_i\left(x\right)=m_i\ast x+b_i$) and $h\left(x\right):X\to \mathbb{R}$, then it is also efficient for the MOOP of minimizing $f=(f_1,\dots ,f_n)$.

Proof. 

If x is not efficient, then there is a $y=(y_1,\dots ,y_n)$ with at least one i where $f_i\left(y\right)<f_i\left(x\right)$ along with $f_j\left(y\right)\le f_j\left(x\right)$ for all other $j\ne i$. Since $f_i\left(y\right)<f_i\left(x\right)$, then $h\left(x\right)\ne h\left(y\right)$ and $x\ne y$. Since all $g_i$ are linear and non-constant, for any given j, $f_j\left(y\right)\ne f_j\left(x\right)$. So, for all j, $f_j\left(y\right)<f_j\left(x\right)$. Thus, x is not weakly efficient and, thus, it cannot be an SNE. So, being an SNE for the game (G) implies efficiency for the given MOOP by way of contraposition.    □

Some definitions for characterizing strategy profiles that are Nash equilibria are the concepts of a normal cone and the variational inequality, as described in [10]. A vector v is a normal vector to X at x if and only if $v·(x^{\prime }-x)\le 0$ for all $x^{\prime }\in X$. This is written as $v\in N_X\left(x\right)$, so $N_X$ is a set-valued mapping described by $\mathrm{gph}{N}_X=\{(x,v):x\in X$ and $v\in N_X\left(x\right)\}$. In general, for a map f, the variational inequality for f and X is

$$\begin{array}{c}\hfill -f\left(x\right)\in N_X\left(x\right)\mathrm{that}\mathrm{is}\mathrm{to}\mathrm{say}x\in X\mathrm{and}f\left(x\right)·(x^{\prime }-x)\ge 0\mathrm{for}\mathrm{all}{x}^{\prime }\in X.\end{array}$$

In [10], it was determined that a strategy profile x is an NE for a game with convex cost functions if and only if $-{\nabla }_{x_i}{f}_i(x_i,x_{-i})\in N_{X_i}\left(x_i\right)$ for all i. This concept can be extended to an SNE.

Theorem 1. 

Let (G) be a game with strategy space X and cost functions $F=(f_1,\dots ,f_n)$. If $f_i$ is convex for all $i\in \{1,\dots ,n\}$ and $\overline{x}=({\overline{x}}_1,\dots ,{\overline{x}}_n)\in X={\prod }_{i\in \{1,\dots ,n\}}{X}_i$ has the property that for all $S\subseteq \{1,\dots ,n\}$ there is a $j\in S$ such that

$$\begin{array}{c}\hfill -{\nabla }_S{f}_j({\overline{x}}_S,{\overline{x}}_{-S})\in N_{X_S}\left({\overline{x}}_S\right),\end{array}$$

then $\overline{x}$ is an SNE.

Proof. 

If we assume $\overline{x}$ is not an SNE, then there is a coalition $S^{*}$ and strategy profile $x_{S^{*}}^{*}\in X_{S^{*}}$ such that $f_i(x_{S^{*}}^{*},{\overline{x}}_{-S^{*}})<f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})$ for all $i\in S^{*}$. But since $f_i$ is convex for all $i\in S^{*}$, then

$$\begin{array}{c}\hfill \nabla f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})·(x_{S^{*}}^{*}-{\overline{x}}_{S^{*}})\le f_i(x_{S^{*}}^{*},{\overline{x}}_{-S^{*}})-f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})<0.\end{array}$$

This means that $-\nabla f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})\notin N_{X_{S^{*}}}$ by definition. Therefore, $\overline{x}$ must be an SNE.    □

Example 1. 

Let

$$\begin{array}{cc}\hfill f_1\left(x\right)& =x^T\left[\begin{array}{ccccc}2& & 1& & 0\\ 1& & 1& & 0\\ 0& & 0& & 1\end{array}\right]x+\left[\begin{array}{c}{\displaystyle \frac{1}{2}}\\ 0\\ {\displaystyle \frac{1}{2}}\end{array}\right]x,\hfill \\ \hfill f_2\left(x\right)& =x^T\left[\begin{array}{ccccc}1& & 1& & 0\\ 1& & 2& & 1\\ 0& & 1& & 1\end{array}\right]x,\hfill \\ \hfill \mathrm{and}{f}_3\left(x\right)& =x^T\left[\begin{array}{ccccc}1& & 0& & 0\\ 0& & 1& & 1\\ 0& & 1& & 2\end{array}\right]x\hfill \end{array}$$

be the cost functions for a game (G) with strategy space $X=\{(x_1,x_2,x_3)\in {\mathbb{R}}^3:-1\le x_1,x_2,x_3\le 1\}$. Now, instead of an in-depth consideration of a player’s strategy, coalitions, and oppositions, we can simply observe all the coalitions’ relevant gradients and reach a conclusion about what points must be an SNE. Observe the restricted gradients

   

${\nabla }_{x_1}{f}_1\left(x\right)=\left(4x_1+2x_2+{\displaystyle \frac{1}{2}},0,0\right)$

${\nabla }_{x_2}{f}_2\left(x\right)=(0,2x_1+4x_2+2x_3,0)$

${\nabla }_{x_3}{f}_3\left(x\right)=(0,0,2x_2+4x_3)$

${\nabla }_{x_1,x_2}{f}_1\left(x\right)=\left(4x_1+2x_2+{\displaystyle \frac{1}{2}},2x_1+4x_2,0\right)$

${\nabla }_{x_1,x_3}{f}_1\left(x\right)=\left(4x_1+2x_2+{\displaystyle \frac{1}{2}},0,2x_3+{\displaystyle \frac{1}{2}}\right)$

${\nabla }_{x_1,x_2}{f}_2\left(x\right)=(2x_1+2x_2,2x_1+4x_2+2x_3,0)$

${\nabla }_{x_2,x_3}{f}_2\left(x\right)=(0,2x_1+4x_2+2x_2,2x_2+2x_3)$

${\nabla }_{x_1,x_3}{f}_3\left(x\right)=(2x_1,0,2x_2+4x_3)$

${\nabla }_{x_2,x_3}{f}_3\left(x\right)=(0,2x_2+2x_3,2x_2+4x_3)$

${\nabla }_{x_1,x_2,x_3}{f}_1\left(x\right)=\left(4x_1+2x_2+{\displaystyle \frac{1}{2}},2x_1+4x_2,2x_3+{\displaystyle \frac{1}{2}}\right)$

${\nabla }_{x_1,x_2,x_3}{f}_2\left(x\right)=(2x_1+2x_2,2x_1+4x_2+2x_3,2x_2+2x_3)$

${\nabla }_{x_1,x_2,x_3}{f}_3\left(x\right)=(2x_1,2x_2+2x_3,2x_2+4x_3)$

   

From this we can work out what an SNE might be by going through all the coalitions S and making sure that $-{\nabla }_S{f}_j({\overline{x}}_S,{\overline{x}}_{-S})\in N_{X_S}\left({\overline{x}}_S\right)$ for at least some $j\in S$. After looking at the single-player coalitions $S=1$, 2, or 3, it can be seen that ${\overline{x}}_S=(-1,1,-1)$ is a suitable vector for which $-{\nabla }_S{f}_j({\overline{x}}_S,{\overline{x}}_{-S})\in N_{X_S}\left({\overline{x}}_S\right)$. Indeed, when $S=\left\{1\right\}$ for all $x^{\prime }=(x_1^{\prime },x_2^{\prime },x_3^{\prime })\in X={[-1,1]}^3$,

$$\begin{array}{c}\hfill -{\nabla }_{x_1}{f}_1\left((-1,1,-1)\right)·(x^{\prime }-(-1,1,-1))=1.5(x_1^{\prime }+1)\ge 0\end{array}$$

so $-{\nabla }_{x_1}{f}_1\left((-1,1,-1)\right)\in N_{X_1}\left((-1,1,-1)\right)$.

When $S=\left\{2\right\}$, since ${\nabla }_{x_2}{f}_2\left((-1,1,-1)\right)=0$, then it must be true that $-{\nabla }_{x_2}{f}_2\left((-1,1,-1)\right)\in N_{X_2}\left((-1,1,-1)\right)$. Lastly, similar to the $S=\left\{1\right\}$ case, for $S=\left\{3\right\}$,

$$\begin{array}{c}\hfill -{\nabla }_{x_3}{f}_3\left((-1,1,-1)\right)·(x^{\prime }-(-1,1,-1))=2(x_3^{\prime }+1)\ge 0.\end{array}$$

When more players are involved, we only need to find one such gradient per coalition in the normal cone. So, for $S=\{1,2\}$, since

$$\begin{array}{cc}\hfill & -{\nabla }_{x_1,x_2}{f}_2\left((-1,1,-1)\right)·(x^{\prime }-(-1,1,-1))\hfill \\ \hfill & =(0,0,0)·(x_1^{\prime }+1,x_2^{\prime }-1,x_3^{\prime }+1)\ge 0\hfill \end{array}$$

then $-{\nabla }_{x_1,x_2}{f}_2\left((-1,1,-1)\right)\in N_{X_{\{1,2\}}}\left((-1,1,-1)\right)$.

For $S=\{1,3\}$, notice that

$$\begin{array}{cc}\hfill -{\nabla }_{x_1,x_2}{f}_3\left((-1,1,-1)\right)·(x^{\prime }-(-1,1,-1))& =(2,0,2)·(x_1^{\prime }+1,x_2^{\prime }-1,x_3^{\prime }+1)\hfill \\ \hfill & =2(x_1^{\prime }+1)+2(x_3^{\prime }+1)\hfill \end{array}$$

and $2(x_1^{\prime }+1)+2(x_3^{\prime }+1)\ge 0$ for all $x^{\prime }\in X={[-1,1]}^3$, so $-{\nabla }_{x_1,x_2}{f}_3\left((-1,1,-1)\right)$ is in the normal cone.

For $S=\{2,3\}$, $-{\nabla }_{x_2,x_3}{f}_2\left((-1,1,-1)\right)=(0,0,0)$, so again it will be in the normal cone. Lastly, for $S=\{1,2,3\}$, notice that $-{\nabla }_{x_1,x_2,x_3}{f}_2\left((-1,1,-1)\right)=(0,0,0)$, so it will also be in the normal cone.

Since, at some point, every coalition has some player’s cost function’s negative gradient in the normal cone of the coalition, Theorem 1 says that that point is an SNE. So, $(-1,1,-1)$ is an SNE.

The converse of Theorem 1 is not necessarily true. If $\overline{x}$ is an SNE, then $-\nabla f_j({\overline{x}}_S,{\overline{x}}_{-S})$ does not need to be in $N_{X_S}\left({\overline{x}}_S\right)$ for every $S\subseteq \{1,\dots ,n\}$. This can be checked in as restrictive a case as a two-player game with cost functions $f_1\left(x\right)=(1,-1)·x$ and $f_2\left(x\right)=(-1,1)·x$ and a strategy space that is the box $X=\{(x_1,x_2)\in {\mathbb{R}}^2:0\le x_1,x_2\le 1\}$.

Another proposition is provided that generalizes Theorem 1 in the context of two-player games. In fact, it gives a characterization of an SNE, and an example is also provided for describing such an SNE.

Proposition 2. 

Let (G) be a game with the convex strategy space $X=X_1\times X_2$ and cost functions $F=(f_1,f_2)$, where each $f_i$ is convex and differentiable. Then, the following statement holds:

A pair $({\overline{x}}_1,{\overline{x}}_2)\in X_1\times X_2$ is an SNE if and only if $\lambda =({\lambda }_1,{\lambda }_2)\in R_{+}^2\setminus \left\{(0,0)\right\}$ such that

$$-\nabla f_{\lambda }({\overline{x}}_1,{\overline{x}}_2)\in N_{X_1}\left({\overline{x}}_1\right)\times N_{X_2}\left({\overline{x}}_2\right)$$

(1)

where $f_{\lambda }={\lambda }_1{f}_1+{\lambda }_2{f}_2$, and

$$-{\nabla }_{x_i}{f}_i({\overline{x}}_1,{\overline{x}}_2)\in N_{X_i}\left({\overline{x}}_i\right)fori=1,2.$$

(2)

Proof. 

As each $f_i$ is convex, $({\overline{x}}_1,{\overline{x}}_2)$ is an SNE if and only if it is an NE and a weakly efficient solution for ${min}_{x\in X}F$. But $({\overline{x}}_1,{\overline{x}}_2)$ being an NE and a weakly efficient solution is equivalent to having (2) and (1) hold, respectively.    □

Example 2. 

Let $f_1(x_1,x_2)=x_1^2+x_2^2+x_1-x_2$, $f_2(x_1,x_2)=x_1^2+x_2^2+x_2-x_1$, and $X_1=X_2=[0,1]$. For $({\overline{x}}_1,{\overline{x}}_2)=(0,0)$, it is easy to see that (2) holds, and $-\nabla f_{\lambda }(0,0)\in N_{[0,1]}\left(0\right)\times N_{[0,1]}\left(0\right)$ with $\lambda =(1,1)$, i.e., (1) holds. Based on Proposition 2, $(0,0)$ is an SNE. But $(0,0)$ does not satisfy the condition in Theorem 1.

4. Algorithm for Finding SNE

The algorithm in this work is of the same nature as other projected gradient methods like the ones found in [17,18] for multi-objective optimization. This algorithm differs by using the arg min function over all coalitions and those coalitions’ potential choices of strategies. So, the direction of descent that the algorithm seeks at any step is one heading towards the weakly Pareto efficient solution of a problem restricted to cost functions from a coalition (see Algorithm 1).

Algorithm 1 Locating a strong Nash equilibrium for convex games

Require:  Strategy space $X={\Pi }_{i\in \{1,\dots ,n\}}{X}_i$ with all $X_i$ convex, closed, and bounded, cost function F with convex components, initial guess $x_0\in X$ $k=0$, $\beta \in (0,1)$ 1: while $x_k$ is not an SNE do 2:      $$(S_k,d_k):=\underset{S\subseteq \{1,\dots ,n\},d\in X_S-x_k}{\mathrm{a}rgmin}\left\{{\displaystyle \frac{{∥d∥}^2}{2}}+\beta \underset{j\in S}{max}\{\nabla f_j(x_S^k,x_{-S}^k)·d\}\right\}$$ (3) (If there exists more than one argmin, choose the $(S_k,d_k)$ that corresponds to the lexicographical minimum of the first component, the index sets $S_k$.) 3:      $$j_k:=\underset{j\in S_k}{\mathrm{a}rgmax}\left\{{\displaystyle \frac{{∥d_k∥}^2}{2}}+\beta \nabla f_j(x_{S_k}^k,x_{-S_k}^k)·d_k\right\}$$ (4) (choose $j_k$ to be the smallest index if $\arg \max$ contains more than one element.) 4:     ${\alpha }_k:=\underset{\alpha \in {\mathbb{R}}_{+}}{\mathrm{a}rgmin}\{f_{j_k}(x_k+\alpha d_k)$ such that $x_k+\alpha d_k\in X$ and $f_i(x_k+\alpha d_k)-f_i\left(x_k\right)\le 0$ for all $i\in S_k\}$ 5:     $x_{k+1}:=x_k+{\alpha }_k{d}_k$ 6:     $k=k+1$ 7: end while 8: if $x_k$ is an SNE then 9:     Set $\overline{x}=x_k$ 10: else[a sequence ${\left(x_k\right)}_{k\in \mathbb{N}}$ was created] 11:     if ${\left(x_k\right)}_{k\in \mathbb{N}}$ converges then 12:         Let $\overline{x}$ be such that ${\left(x_k\right)}_{k\in \mathbb{N}}\to \overline{x}$ 13:     else[${\left(x_k\right)}_{k\in \mathbb{N}}$ does not converge] 14:         return “Algorithm Failed” 15:     end if 16: end if 17: return $\overline{x}$

When F is a one-dimensional function, the algorithm works similar to a gradient descent method with a direct calculation of the step size. The additional ${\displaystyle \frac{{∥d∥}^2}{2}}$ term is included to ensure that the minimum exists by making the argument strongly convex.

The first thing to be proven is that such step sizes exist.

Proposition 3. 

The step size ${\alpha }_k$ calculated in Algorithm 1 exists for all $k\in \mathbb{N}$.

Proof. 

Since $d=0$ is a candidate for (3) from Algorithm 1, the minimum will have to be less than or equal to 0. If $d=0$, then any $\alpha$, for example, $\alpha =0$, would be sufficient as a value that minimizes the function.

Now, for $d\ne 0$, if

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{∥d∥}^2}{2}}+\beta \underset{j\in S}{max}\{\nabla f_j(x_S^k,x_{-S}^k)·d\}=0\hfill \end{array}$$

then consider ${\displaystyle \frac{d}{2}}$ in place of d,

$$\begin{array}{cc}& {\displaystyle \frac{{∥\frac{d}{2}∥}^2}{2}}+\beta \underset{j\in S}{max}\{\nabla f_j(x_S^k,x_{-S}^k)·{\displaystyle \frac{d}{2}}\}\hfill \\ \hfill & ={\displaystyle \frac{-\frac{{∥d∥}^2}{4}+\frac{{∥d∥}^2}{2}+\beta {max}_{j\in S}\{\nabla f_j(x_S^k,x_{-S}^k)·d\}}{2}}\hfill \\ \hfill & ={\displaystyle \frac{-\frac{{∥d∥}^2}{4}+0}{2}}=-{\displaystyle \frac{{∥d∥}^2}{8}}<0.\hfill \end{array}$$

Now, because $d_k$ is a minimizer, it must be true that

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{∥d_k∥}^2}{2}}+\beta \underset{j\in S_k}{max}\{\nabla f_j(x_S^k,x_{-S}^k)·d_k\}<0,\hfill \end{array}$$

and so $\nabla f_j(x_S^k,x_{-S}^k)·d_k<0$ for all $j\in S_k$. So, $d_k$ acts as a direction of descent heading away from $x_k$ for $f_j$ for all $j\in S_k$.

Observing the constraints of the minimization problem defining $\alpha$, since X is convex, closed, and bounded, there is a maximum $\alpha$ such that $x_k+\alpha d_k\in X$. Going further, because all the functions $f_i$ are convex and $d_k$ is a direction of descent away from $x_k$ for all $f_i$ with $i\in S_k$, there is some interval $[0,a]$ in which $f_i(x_k+\alpha d_k)-f_i\left(x_k\right)\le 0$ for all $i\in S_k$ and $\alpha \in [0,a]$.

Now, because $f_{j_k}$ is convex, it will be continuous on any line segment in its domain. And because $x_k+\alpha d_k$ for $\alpha \in [0,a]$ defines a line segment, a minimum can be obtained for $f_{j_k}$ on that segment using the extreme-value theorem. Note that $\alpha$ cannot be 0 because $f_{j_k}\left(x_k\right)>f_{j_k}(x_k+\alpha d_k)$ for at least one $\alpha$ in $(0,a]$. □

When $x_k$ is an SNE, the algorithm will calculate $d_k$ to be 0 over and over again, so it should be noted that the check statement at the beginning of the “while” loop may not be necessary.

In order to show that the convergence of the algorithm does actually imply that an SNE has been located, more lemmas will be required. The next lemma shows that there is an implication in convergence between the directions of descent and the indices giving the least descent from the coalitions that provide that direction.

Lemma 1. 

If $d_k$ converges to d and $x_k$ converges to $\overline{x}$, then $j_k$, as described in (4), converges.

Proof. 

Since

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}{\displaystyle \frac{{∥d_k∥}^2}{2}}+\beta \nabla f_j(x_{S_k}^k,x_{-S_k}^k)·d_k={\displaystyle \frac{{∥d∥}^2}{2}}+\beta f_j\left(\overline{x}\right)·d\end{array}$$

for all j, then

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}{j}_k=min\left\{j\in \arg {\max }_{j\in S}\left\{{\displaystyle \frac{{∥d∥}^2}{2}}+\beta f_j\left(\overline{x}\right)·d\right\}\right\},\end{array}$$

where S is the collection of indices of non-zero components of d. This is because if all the limits exist, then the limit of the maximums is the maximum of the limits. Here, the minimum is only being used since in the calculation of $j_k$, if there is more than one element given in the $\arg \max$ from (4), the smallest index of all the $j\in S_k$ that give the maximum is chosen. □

For the next lemma, it will be shown that if the directions converge to 0, then if the algorithm is convergent, it actually finds a strong Nash equilibrium.

Lemma 2. 

Assuming that $d_k\to 0$ and $x_k$ converges from Algorithm 1, then the algorithm converges to a strong Nash equilibrium.

Proof. 

Instead consider when $x_k\to \overline{x}$ but that $\overline{x}$ is not an SNE. Since every component $f_i$ of F is convex from the requirements of Algorithm (1), there are an $S^{*}$ and $x^{*}\in X_{S^{*}}$ such that for each $i\in S^{*}$

$$\begin{array}{c}\hfill \nabla f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})·(x^{*}-{\overline{x}}_{S^{*}})<0.\end{array}$$

Convexity is necessary because without that condition, it could end up in a situation where $\nabla f_i({\overline{x}}_{S^{*}},{\overline{x}}_{-S^{*}})=0$. But this is the same as

$$\begin{array}{c}\hfill \nabla f_i\left(\overline{x}\right)·v<0\end{array}$$

for all $i\in S^{*}$ where $v=x-{\overline{x}}_{S^{*}}$. Now, take a small enough $\alpha >0$ so that

$$\begin{array}{c}\hfill {\displaystyle \frac{{∥\alpha v∥}^2}{2}}+\beta \alpha \nabla f_i\left(\overline{x}\right)·v<0\end{array}$$

for all $i\in S^{*}$ and pass $\alpha v$ into v.

Take a $\delta >0$ so that

$$\begin{array}{c}\hfill {\displaystyle \frac{{∥v∥}^2}{2}}+\beta \nabla f_i\left(\overline{x}\right)·v<-\delta <0\end{array}$$

for all $i\in S^{*}$, and then take $ϵ={\displaystyle \frac{\delta }{4}}$ and a large enough $K>0$ so that $|\nabla f_i\left(\overline{x}\right)·v-\nabla f_i\left(x_k\right)·v|<ϵ$ for all $k>K$. In that case,

$$\begin{array}{c}\hfill {\displaystyle \frac{{∥v∥}^2}{2}}+\beta \nabla f_i\left(x_k\right)·v\le {\displaystyle \frac{{∥v∥}^2}{2}}+\beta \nabla f_i\left(\overline{x}\right)·v+ϵ<ϵ-\delta <{\displaystyle \frac{\delta }{2}}<0\end{array}$$

for all $k>K$ and $i\in S^{*}$. But since $d_k\in X_{S_k}$ is a minimizer,

$$\begin{array}{c}\hfill {\displaystyle \frac{{∥d_k∥}^2}{2}}+\beta \underset{i\in S_k}{max}\{\nabla f_i\left(x_k\right)·d_k\}\le {\displaystyle \frac{{∥v∥}^2}{2}}+\beta \underset{i\in S^{*}}{max}\{\nabla f_i\left(x_k\right)·v\}\le {\displaystyle \frac{\delta }{2}}<0.\end{array}$$

But because $\delta$ is fixed, $d_k\cancel{\to }0$. So, if $d_k\to 0$ and $x_k\to \overline{x}$, then it must be true that $\overline{x}$ is a strong Nash equilibrium. □

Moving on, we next discuss another lemma that will be helpful, as it shows that under certain conditions $\left\{d_k\right\}$ cannot converge to a non-zero vector.

Lemma 3. 

If all the cost functions $f_j$ are continuously differentiable and ${\alpha }_k\to 0$ and $x_k$ converges to $\overline{x}$ from Algorithm 1, then either $d_k$ converges to 0 or it does not converge at all.

Proof. 

Assume $d_k\to d\ne 0$. Note that

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\nabla f_{j_k}\left(x_k\right)·d_k\ngeqq 0.\end{array}$$

Indeed, otherwise, the $\arg \min$ statement that defines $d_k$ would give a positive value as the minimum because $∥d_k∥\to ∥d∥>0$. But that is not possible because the zero vector is always a possible choice for $d_k$, and that in turn would return 0 as the minimum value. Also, note that as ${\alpha }_k$ is defined as a minimizer for a continuously differentiable function (that is, the parameterized function $f_{j_k}(x_k+\alpha d_k)$ with parameter $\alpha$), $\nabla f_{j_k}(x_k+{\alpha }_k{d}_k)·d_k=0$ for all k.

Using Lemma 1, let $j^{*}={lim}_{k\to \infty }{j}_k$. Now, take b so that

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}\nabla f_{j_k}\left(x_k\right)·d_k=\nabla f_{j^{*}}\left(\overline{x}\right)·d=-b<0.\end{array}$$

This must be non-zero or else $d_k$ would have to converge to 0.

Since $x_k$ converges, it must be Cauchy, and so taking $ϵ={\displaystyle \frac{b}{2∥d∥}}$, a $K>0$ can be chosen that is large enough so that

$$\begin{array}{c}\hfill ∥\nabla f_{j_k}(x_k+{\alpha }_k{d}_k)-\nabla f_{j_k}\left(x_k\right)∥<ϵ={\displaystyle \frac{b}{2∥d∥}}.\end{array}$$

So, for all $k>K$, this gives

$$\begin{array}{cc}\hfill ∥\nabla f_{j_k}\left(x_k\right)·d_k∥& =∥\left(\nabla f_{j_k}(x_k+{\alpha }_k{d}_k)-\nabla f_{j_k}\left(x_k\right)\right)·d_k∥\hfill \\ \hfill & =|cos\left(w_k\right)|∥\left(\nabla f_{j_k}(x_k+{\alpha }_k{d}_k)-\nabla f_{j_k}\left(x_k\right)\right)∥∥d_k∥\hfill \\ \hfill & <{\displaystyle \frac{b}{2∥d∥}}∥d_k∥,\hfill \end{array}$$

where $w_k$ is the angle between $\nabla f_{j_k}(x_k+{\alpha }_k{d}_k)-\nabla f_{j_k}\left(x_k\right)$ and $d_k$. The limit is taken on both sides of the inequality, which yields

$$\begin{array}{c}\hfill \left|b\right|=\underset{k\to \infty }{lim}∥\nabla f_{j_k}\left(x_k\right)·d_k∥\le \underset{k\to \infty }{lim}{\displaystyle \frac{b}{2∥d∥}}∥d_k∥={\displaystyle \frac{b}{2}},\end{array}$$

which is a contradiction as a non-zero b cannot be less than or equal to half of itself. So, simply put, $d_k\cancel{\to }d\ne 0$. □

All of these lemmas finally lead to some conditions for which the convergence of the algorithm will lead to an SNE.

Theorem 2. 

With regard to Algorithm 1, when X is compact and all $f_i$ are convex and continuously differentiable, then $x_k$ converging to $\overline{x}$ implies that $\overline{x}$ must be an SNE.

Proof. 

The space X being compact implies it is closed and bounded. Now, because $x_k\to \overline{x}$ and $x_{k+1}=x_k+{\alpha }_k{d}_k$, either ${\alpha }_k\to 0$ or $d_k\to 0$. However, Lemma 2 shows that $d_k\to 0$ implies that $x_k$ will converge to an SNE. For the case when ${\alpha }_k\to 0$, Lemma 3 shows that either $d_k\to 0$ or $d_k$ does not converge. Again, for the first case, $d_k\to 0$ will imply that $x_k$ converges to an SNE. So, it only remains to consider if $d_k$ does not converge. But because X is compact, ${\left\{d_k\right\}}_{k\in \mathbb{N}}$ has a convergent sub-sequence, known as ${\left\{d_{k_i}\right\}}_{i\in \mathbb{N}}$. Since $d_{k_i}$ is convergent in X, it will either converge to 0 or some other point d within the bounds of X.

Case 1: $d_{k_i}\to 0$.

Assuming that $\overline{x}$ is not an SNE (otherwise, the theorem would be proven), there must be a coalition $S^{*}$ and at least one direction $d^{*}$ for which

$$\begin{array}{c}\hfill {\displaystyle \frac{{∥d^{*}∥}^2}{2}}+\beta \arg {\max }_{j\in S^{*}}\nabla f_j\left(\overline{x}\right)·d^{*}=b<0.\end{array}$$

(5)

The direction $d^{*}$ can indeed be found because there is some d where $\beta \nabla f_j\left(\overline{x}\right)·d\le c<0$ for $c:=\beta \arg {\max }_{j\in S^{*}}\nabla f_j\left(\overline{x}\right)·d$ for all $j\in S^{*}$. That d can then be scaled down to $d^{*}=ad$ with $0<a<{\displaystyle \frac{-c}{{∥d∥}^2}}$ and $a\le 1$, which ensures

$$\begin{array}{cc}\hfill {\displaystyle \frac{{∥d^{*}∥}^2}{2}}+\beta \arg {\max }_{j\in S^{*}}\nabla f_j\left(\overline{x}\right)·d^{*}& ={\displaystyle \frac{a^2{∥d∥}^2}{2}}+a\beta \arg {\max }_{j\in S^{*}}\nabla f_j\left(\overline{x}\right)·d\hfill \\ \hfill & <{\displaystyle \frac{a\left(\frac{-c}{{∥d∥}^2}\right){∥d∥}^2}{2}}+ac={\displaystyle \frac{ac}{2}}<0.\hfill \end{array}$$

We then take that $d^{*}$ and also $b:=ac$ as defined to make certain that the inequality on line (5) holds.

Let $ϵ={\displaystyle \frac{-b}{2∥d^{*}∥}}$ and choose a large enough $K>0$ so that for all $j\in \{1,\dots ,n\}$, $∥\nabla f_j\left(\overline{x}\right)-\nabla f_j\left(x_{k_i}\right)∥<ϵ$ for all $i>K$. Using this $ϵ$ inequality, along with the fact that $d_{k_i}$ is defined as a minimizer from line (3) in Algorithm 1, gives

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{∥d_{k_i}∥}^2}{2}}+\beta \arg {\max }_{j\in S_{k_i}}\nabla f_j(x_{S_{k_i}}^{k_i},x_{-S_{k_i}}^{k_i})·d_{k_i}\hfill \\ \hfill & <{\displaystyle \frac{{∥d^{*}∥}^2}{2}}+\beta \arg {\max }_{j\in S^{*}}\nabla f_j(x_{S^{*}}^{k_i},x_{-S^{*}}^{k_i})·d^{*}\hfill \\ \hfill & <{\displaystyle \frac{{∥d^{*}∥}^2}{2}}+\beta \arg {\max }_{j\in S^{*}}\nabla f_j\left(\overline{x}\right)·d^{*}+\beta ϵ∥d^{*}∥\hfill \\ \hfill & =b+ϵ∥d^{*}∥\hfill \\ \hfill & <b+{\displaystyle \frac{-b}{2∥d^{*}∥}}∥d^{*}∥={\displaystyle \frac{b}{2}}<0.\hfill \end{array}$$

Since $d_{k_i}\to 0$, assuming that the limit as $k_i$ goes to infinity on this whole inequality reveals the contradiction that $0\le {\displaystyle \frac{b}{2}}<0$, which is impossible. So, $\overline{x}$ would have to be an SNE.

Case 2: $d_{k_i}\to d\ne 0$.

This argument’s approach is the same as that defined in Lemma 3 but on the sub-sequence $d_{k_i}$ instead of $d_k$. Since $d_{k_i}\to d\ne 0$, it must be true that ${lim}_{i\to \infty }\nabla f_{j_{k_i}}\left(x_{k_i}\right)·d_{k_i}=\nabla f_{j^{*}}\left(\overline{x}\right)·d=b<0$, where $j^{*}={lim}_{i\to \infty }{j}_{k_i}$ from Lemma 1 using the sub-sequence $d_{k_i}$ in place of $d_k$.

The original assumption that $x_k$ converges implies that $x_{k_i}$ is Cauchy, so assuming that $ϵ={\displaystyle \frac{\left|b\right|}{2∥d∥}}$, there will be a large enough $K>0$ so that for all $i>K$,

$$\begin{array}{cc}\hfill ∥\nabla f_j\left(x_{k_i+1}\right)-\nabla f_j\left(x_{k_i}\right)∥& <ϵ.\hfill \end{array}$$

It must also be noted by the definition of $d_{k_i}$ that $\nabla f_{j_{k_i}}(x_{k_i}+{\alpha }_{k_i}{d}_{k_i})·d_{k_i}=0$. By using the Cauchy–Schwarz inequality,

$$\begin{array}{cc}\hfill ∥\nabla f_{j_{k_i}}\left(x_{k_i}\right)·d_{k_i}∥& =∥\left(\nabla f_{j_{k_i}}(x_{k_i}+{\alpha }_{k_i}{d}_{k_i})-\nabla f_{j_{k_i}}\left(x_{k_i}\right)\right)·d_{k_i}∥\hfill \\ \hfill & \le ∥\nabla f_{j_{k_i}}(x_{k_i}+{\alpha }_{k_i}{d}_{k_i})-\nabla f_j\left(x_{k_i}\right)∥∥d_{k_i}∥\hfill \\ \hfill & <ϵ∥d_{k_i}∥.\hfill \end{array}$$

Again, a limit is taken on this inequality, which gives

$$\begin{array}{c}\hfill \left|b\right|=∥\nabla f_{j^{*}}\left(\overline{x}\right)·d∥=\underset{i\to \infty }{lim}∥f_{j_{k_i}}\left(x_{k_i}\right)·d_{k_i}∥\le \underset{i\to \infty }{lim}ϵ∥d_{k_i}∥=ϵ∥d∥={\displaystyle \frac{\left|b\right|}{2∥d∥}}∥d∥={\displaystyle \frac{\left|b\right|}{2}},\end{array}$$

which is a contradiction as $b\ne 0$, so $d_{k_i}\cancel{\to }d$. □

So, whenever the algorithm converges, it will be to a strong Nash equilibrium. It can be seen when the algorithm is applied to an example game without an SNE that multiple cluster points will be produced [11]. To show the algorithm in action, an example is given.

Example 3. 

Consider the following problem:

$$\begin{array}{cc}\hfill X& ={[0,1]}^3,\hfill \\ \hfill F:X\to {\mathbb{R}}^3\mathit{with}F& =(f_1,f_2,f_3),\hfill \\ \hfill f_1(y_1,y_2,y_3)& =y_1(y_2-y_3)+y_1,\hfill \\ \hfill f_2(y_1,y_2,y_3)& =y_2(y_3-y_1)+y_2,\hfill \\ \hfill \mathit{and}{f}_3(y_1,y_2,y_3)& =y_3(y_1-y_2)+y_3.\hfill \end{array}$$

Note that the cost functions are convex and there is an SNE at $(0,0,0)$.

Consider Algorithm 1 using $\beta ={\displaystyle \frac{1}{2}}$ and apply it to the initial guess $x_0=\left({\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right)$; then,

$$\begin{array}{c}\hfill \nabla f_1\left(x_0\right)=\left(1,{\displaystyle \frac{1}{2}},-{\displaystyle \frac{1}{2}}\right),\nabla f_2\left(x_0\right)=\left(-{\displaystyle \frac{1}{2}},1,{\displaystyle \frac{1}{2}}\right)\\ \hfill \mathit{and}\nabla f_3\left(x_0\right)=\left({\displaystyle \frac{1}{2}},-{\displaystyle \frac{1}{2}},1\right).\end{array}$$

All of the gradients have a similarity and so, when generalizing, there are only three types of cases that need to be checked. These cases are similar to when $S_0$ is $\left\{1\right\}$, $\{1,2\}$, or $\{1,2,3\}$. The easiest case to check is when player 1 is acting on their own, which is when $S_0=\left\{1\right\}$. After simplifying, this is carried out by finding $d_1$ in

$$\begin{array}{c}\hfill \mathit{arg}{\mathit{min}}_{d_1\in \left[-{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right]}{\displaystyle \frac{d^2}{2}}+{\displaystyle \frac{1}{2}}d,\end{array}$$

which gives $d_1=-{\displaystyle \frac{1}{2}}$, that is to say, the direction of descent is $d=\left(-{\displaystyle \frac{1}{2}},0,0\right)$, which gives the corresponding minimum of $-{\displaystyle \frac{1}{8}}$.

When $S_0=\{1,2\}$, then $d_1$ and $d_2$ need to be found in

$$\begin{array}{c}\hfill \mathit{arg}{\mathit{min}}_{d_1,d_2\in \left[-{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right]}{\displaystyle \frac{d_1^2+d_2^2}{2}}+{\displaystyle \frac{1}{2}}max\left\{d_1+{\displaystyle \frac{d_2}{2}},-{\displaystyle \frac{d_1}{2}}+d_2\right\},\end{array}$$

which ends up giving the direction $d=\left(-{\displaystyle \frac{1}{8}},-{\displaystyle \frac{3}{8}},0\right)$ and a corresponding minimum of $-{\displaystyle \frac{5}{64}}$.

Lastly, when $S_0=\{1,2,3\}$, $d_1$, $d_2$, and $d_3$ all need to be found from

$$\begin{array}{cc}\hfill \mathit{arg}{\mathit{min}}_{d_1,d_2,d_3\in \left[-{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right]}& {\displaystyle \frac{d_1^2+d_2^2+d_3^2}{2}}\hfill \\ \hfill & +{\displaystyle \frac{1}{2}}max\left\{d_1+{\displaystyle \frac{d_2}{2}}-{\displaystyle \frac{d_3}{2}},-{\displaystyle \frac{d_1}{2}}+d_2+{\displaystyle \frac{d_3}{2}},{\displaystyle \frac{d_1}{2}}-{\displaystyle \frac{d_2}{2}}+d_3\right\},\hfill \end{array}$$

which gives the direction of descent as $d=\left(-{\displaystyle \frac{1}{6}},-{\displaystyle \frac{1}{6}},-{\displaystyle \frac{1}{6}}\right)$ and a corresponding minimum of $-{\displaystyle \frac{1}{24}}$.

From these three cases, the direction that actually minimizes the step in the algorithm is $\left(-{\displaystyle \frac{1}{2}},0,0\right)$ as $-{\displaystyle \frac{1}{8}}$ is the smallest corresponding minimum, and ${\alpha }_0$ is found to be 1, so $x_1=\left(0,{\displaystyle \frac{1}{2}},{\displaystyle \frac{1}{2}}\right)$. The coalition of the first player is the one that the step uses but only because it is lexicographically first compared to the similar single-player coalitions $\left\{2\right\}$ and $\left\{3\right\}$ that we generalized.

For brevity, after that, the next direction is $\left(0,-{\displaystyle \frac{1}{2}},0\right)$ and $x_2=\left(0,0,{\displaystyle \frac{1}{2}}\right)$. Then, the last descent direction is $\left(0,0,-{\displaystyle \frac{1}{2}}\right)$, which gives the SNE $x_3=(0,0,0)$.

While this may seem like a drawn out process, each step is relatively easy to compute for many convex problems using computer programming. However easy it may be to calculate, it can be shown that Algorithm 1 may still become stuck in an endless loop even for convex functions with an SNE present [11].

5. Testing

Tests were programmed using Mathematica to determine some metrics on the algorithm performance. The problem types designed for testing were games with convex quadratic cost functions where the quadratic coefficient was generated by taking a random matrix with entries from $-10$ to 10 and multiplying it by its transpose to obtain a positive semi-definite matrix, and the linear coefficient was taken as a random vector with entries from $-10$ to 10. The strategy space was determined by giving each player a random interval from $-10$ to 10 with integer endpoints. The stopping criteria was to run beyond the maximum number of iterations (200 unless noted otherwise), step to a solution that has already been found that will result in a loop, or have the last two consecutive solutions within a convergence tolerance (0.00001 unless otherwise noted).

Table 1. Algorithm 1 results.

200 Iterations Max Unless Indicated Otherwise
$\mathit{n}$	# Prob	# ${\mathit{x}}_{\mathbf{0}}$’s	${\mathit{\alpha }}_{\mathit{k}}$	Conv	Conv It	Loops	Loop It	Fails
2-6	50	1	Calc	14	10	23	59	13
2-6	50	1	1	5	4.6	45	9.6$\overline{2}$	0
2	50	1	Calc	46	7.22	4	27.5	0
2	50	1	1	27	4.19	23	5.17	0
4	50	1	1	7	5.43	43	7.86	0
6	50	1	1	2	5	48	12.60	0
All Subsequent Pairs Tallied with same Problems and Start Points
3	1	100	Calc	29	7.93	71	87.03	0
3	1	100	1	0	N/A	100	21.26	0
Subsequent Pair used a Convergence Tolerance of 0.0000001
2	50	200	Calc	6397	6.80	3374	2.05	229
2	50	200	1	9096	3.14	331	19.78	573
Subsequent Pair used 300 Iterations Max
4	1	200	Calc	0	N/A	11	205.01	189
4	1	200	1	0	N/A	200	4.78	0
Subsequent Pair used 300 Iterations Max and a Conv. Tol. of 0.0000001
3	20	20	Calc	225	13.81	49	2.04	126
3	20	20	1	237	4.57	56	39.77	107

describes the performance of the algorithm in different contexts. Column “n” gives the number of players possible for the random games, “$\#Prob$” gives the number of random games the algorithm was run on, and “$\#x_0$’s” gives the number of random start points for every random game. The “${\alpha }_k$” column indicates if it is calculated as described in the algorithm or if it is fixed at 1. “Conv” is the count of convergent runs. “Conv It” is the average number of iterations that the algorithm took to converge. “Loops” gives the number of runs for which the algorithm ended up in a loop, with “Loop It” being the average number of iterations that it took a run to fall into a loop. “Fails” is the number of times the algorithm did not converge or ended up in a loop by the maximum number of iterations.

As the table shows, whether ${\alpha }_k$ is calculated or not, the number of iterations required to find convergence is low when the algorithm converges, often taking only five to ten steps. It should also be noted that the algorithm may still fall into loops even when an SNE strategy is available, and it take more iterations to end up in those loops.

In this test, ${\alpha }_k=1$ was considered for the practical purpose that it significantly reduced the calculation time by removing a minimization algorithm that was built into the Mathematica tool set. The pairs tallied with the same problems and start points were used to further investigate the difference found between ${\alpha }_k=1$ and when ${\alpha }_k$ was calculated. The pairs that may cause some confusion are in the algorithm tests with increased convergence tolerance. The increased tolerance caused some issues whereby the program recorded convergences as loops due to the fact that ${\alpha }_k$ was calculated and coming out as practically 0 (values like $3.1235\times {10}^9$). The record keeping system that was programmed could not differentiate convergence from loop due to the small values and lumped many convergences in with the loops. This is also evident by the extremely low values in the loop iteration column. After reviewing the data, there were loops but not nearly as often as indicated on the table. So, this data needs to be carefully considered.

This data revealed some good practical techniques in execution. When using this algorithm, it should first be run using a fixed ${\alpha }_k=1$ to initially obtain a quick check for convergence. If convergence failure occurs when ${\alpha }_k=1$, the algorithm could then be run again using the line search method laid out in the original algorithm, which will have longer a computing time and require more iterations, but it can more often succeed in finding an SNE and keep the algorithm from ending up in loops. A few random start positions can be used if convergence initially fails. Care needs to be taken with near-0 calculations. And lastly, the maximum number of iterations can be lowered significantly.

6. Conclusions

Often times, players in game theoretic problems are allowed to collaborate, and in games where there is not any particular resentment among players, there may be a desire to find outcomes that befit as many players as possible. In such games, a strong Nash equilibrium may be desirable, as this is the sort of solution where no amount of scheming can yield better results for any players. While some readers may be initially dismissive of strong Nash equilibria as being too restrictive, rare, or hard to identify, the results in this paper indicate that they can potentially be identified with relatively low effort and so seeking an SNE should not be overlooked. Considering that an SNE could be a unanimously favored solution for a problem if it exists, investing a small amount of time in seeking one would not be time wasted. What was carried out in this paper was a further exploration of the concept of strong Nash equilibria using modern methods from variational analysis. Also, a new algorithm was developed and tested for aiding in findings SNEs.

First, the concept of strong Nash equilibria was connected to the concept of efficiency under certain circumstances. Next, the concepts of the normal cone and variational inequality were used to find a sufficient condition for showing a solution to be an SNE. A characterization of two-player games using the variational inequality was also provided. The algorithm for finding an SNE was developed using a projected gradient method by calling the $\arg \min$ function over all coalitions of players possible. It was shown that when the algorithm converges, it does so to an SNE. Some tests were performed to check the practical efficiency of the algorithm.

Further research can be carried out to explore the convergence rate of this sort of projected gradient algorithm. The relatively quick convergence of the test results, while interesting, is no proof, but it does make one hopeful that the convergence is not too slow. Additionally, the computational complexity of this algorithm and some further test results and analyses of the algorithm could be of interest. It would also be prudent to compare the algorithm results with other algorithms for calculating strong Nash equilibria like those in [6,7].

Another direction for advacning this research could be creating an SNE algorithm based off the weighted best reply, as mentioned in [5]. That algorithm may be as simple as just finding the best reply to $x_k$ and moving there. Then, the fact that a fixed best reply is in fact an SNE for convex problems may make it likely that the algorithm converging results in an SNE. Hopefully, that type of algorithm would always converge for problems that have the coalition consistency principle.