An Enhanced Gradient Algorithm for Computing Generalized Nash Equilibrium Applied to Electricity Market Games

Abstract

This paper introduces an enhanced algorithm for computing generalized Nash equilibria for multiple player nonlinear games, which degenerates in a gradient algorithm for single player games (i.e., optimization problems) or potential games (i.e., equivalent to minimizing the respective potential function), based on the Rosen gradient algorithm. Analytical examples show that it has similar theoretical guarantees of finding a generalized Nash equilibrium when compared to the relaxation algorithm, while numerical examples show that it is faster. Furthermore, the proposed algorithm is as fast as, but more stable than, the Rosen gradient algorithm, especially when dealing with constraints and non-convex games. The algorithm is applied to an electricity market game representing the current electricity market model in Brazil.

1. Introduction

The energy sector in many countries around the world has been reformed since the end of the 1980s [1,2]. It changed from a centralized, monopolistic sector to one that stimulates competition, with an open market that gencos (and often consumers) can make bids for energy prices and negotiate future energy contracts. The energy spot price is then defined by the market operator (and/or system operator) using the marginal unit dispatched price, considering the bids and transmission constraints [3].

The change in the energy markets also created challenges for healthy market operation. While the move from regulated energy amounts and prices gave gencos the freedom to define better business strategies, it also gave larger companies the possibility to manipulate prices and exert market power [1,4,5,6]. Furthermore, a more open market tends to require shorter time bids [7].

The Brazilian electric sector reforms started in 1996 [8], and now it works in a mix of regulated and open markets. The open market allows gencos and consumers to negotiate contracts directly, and has spot prices calculated from audited costs and future prices of water (as hydroelectric power plants are the main source of energy), using a complex chain of models. There are proposals for moving the market to a bid-based model that would allow generators to make bids, declaring the price and quantity curves they are offering for each time slot [8,9].

The use of game theory as a tool to study competitive energy markets is well established [1,10]. As the electricity markets usually derived from an integrated, monopolistic and state-owned company, the analyses usually focus on oligopolist competitive game models [10], particularly looking for Nash equilibrium strategies and their relationships to marginal cost curves. In this sense, Nash equilibrium analysis can be used by market participants to define best bid strategies and by regulatory agencies to evaluate market power.

A Nash equilibrium is defined by

$$\begin{array}{cc}\hfill x_1^{★}& =arg\underset{x_1}{min}{f}_1(x_1,x_{-1}^{★})\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill x_2^{★}& =arg\underset{x_2}{min}{f}_2(x_2,x_{-2}^{★})\hfill \\ \hfill \dots \end{array}$$

(2)

$$\begin{array}{cc}\hfill x_p^{★}& =arg\underset{x_p}{min}{f}_p(x_p,x_{-p}^{★})\hfill \end{array}$$

(3)

where $x=(x_1,x_2,\dots ,x_p)\in {\mathbb{R}}^n$ are the player variables (i.e., strategy), $f_v:{\mathbb{R}}^n\to \mathbb{R}$ is the objective function of player v, $x_v\in {\mathbb{R}}^{n_v}$ are the variables of player v and $x_{-v}$ denotes all variables except the ones of player v. The Nash equilibrium concept can be extended to concurrent constrained optimization problems, as defined further in this paper.

The first known use of the Nash equilibrium concept dates back to 1838. However, it was named after John Nash in 1950 who proved its existence for continuous convex objectives relative to player own variables and nonempty convex compact feasible sets [11]. Rosen has proved extra conditions on continuously differentiable convex games in order to guarantee uniqueness of a Nash equilibrium [12]. Facchinei et al., have analyzed the existence of a Nash equilibrium for pseudo-convex objective functions relatively to player own variables in terms of variational inequalities on compact convex feasible sets [13].

The Best Response Dynamics (BRD) is a straightforward algorithm to find a Nash equilibrium where at each iteration a different player optimizes its own strategy looking only to its own objective, while fixing other player strategies. It enjoys theoretical guarantees of convergence to a Nash equilibrium only in special cases, including potential games [14], aggregative games [15], acyclic games [16] and quasi-acyclic games [17,18].

Rosen has proposed a gradient algorithm with guaranteed convergence and optimality for games satisfying certain convex conditions [12], which introduces the concept of normalized Nash equilibrium for constrained games and also the fundamental idea for the search direction of this paper.

The relaxation algorithm considers the optimization of Nikaido–Isoda [19] function at each iteration, which is built from player objective functions, so that the next player strategies lies in the segment connecting the previous iteration point and the Nikaido–Isoda function solution. It enjoys theoretical guarantees of convergence to a Nash equilibrium for games with weakly convex–concave Nikaido–Isoda functions [20,21] respecting a few extra conditions, which is a broader class of games than BRD and Rosen gradient algorithm can cope with with theoretical guarantees.

This paper proposes and analyzes an enhanced algorithm based on a line search along a composition of objective function gradients of players as search direction at each iteration, without the need of an optimization on player variables at each iteration as in the relaxation method. The result is a faster and more stable algorithm, especially in games of many variables. This algorithm has already been used, but not analyzed in depth [22].

2. The Gradient Algorithm

2.1. Fundamental Idea

The fundamental idea of the proposed algorithm is to follow the vector field composed of a negatively weighted gradient of each player objective

$$F\left(x\right)=-({\lambda }_1{\nabla }_{x_1}{f}_1\left(x\right),{\lambda }_2{\nabla }_{x_2}{f}_2\left(x\right),\dots ,{\lambda }_p{\nabla }_{x_p}{f}_p\left(x\right)),\lambda >0$$

(4)

until it eventually stops at a Nash equilibrium. Let $\widehat{F}\left(x\right)$ denote the unit vector towards a non-null $F\left(x\right)$. Then, the algorithm can be defined by the iterative update

$$x_{k+1}=x_k+{\alpha }^{★}\widehat{F}\left(x_k\right)$$

(5)

where the step length is

$${\alpha }^{★}=\underset{\alpha }{max}\alpha :\widehat{F}{\left(x_k\right)}^T\widehat{F}\left(x_{k+1}\right)\ge 1-\eta ,\eta \in (0,2)$$

(6)

where $\eta$ controls the change direction threshold to follow the vector field ($\eta =0$ for no direction change).

The search direction (4) is the same as proposed by Rosen [12]. However, the step length (6) is different (i.e., it is not the one that minimizes $∥F(x+\alpha F(x\left)\right)∥$) and behaves better in practice).

Notice that for single player games (i.e., optimization problems) and $\eta =1$, this algorithm degenerates into the classical gradient descent algorithm [23]. Furthermore, if $\eta =1$, $\lambda =1$ and the vector field (4) is conservative, i.e., the game is potential [14], then the algorithm also degenerates into the classical gradient descent algorithm for minimizing the associated potential function.

2.2. Algorithm Analysis

The convergence of the algorithm depends on the following concept of attractor and basin of attraction.

Definition 1

(Attractor and basin of attraction). Let $0\le t\in \mathbb{R}$ represent time and $s(t,x)$ be a function which specifies the dynamics of the system, so that $s(0,x)=x\in {\mathbb{R}}^n$ is the initial state of the system and $s(t,x)$ is the result of the evolution of this state after t units of time. An attractor $\mathbb{A}\subset {\mathbb{R}}^n$ of a dynamic system $s(t,x)$ is characterized by the following three conditions:

1. 

If $s(t_a,x)\in \mathbb{A}$ then $s(t,x)\in \mathbb{A}$, $\forall t>t_a$;

2. 

There exists a neighborhood of $\mathbb{A}$, called the basin of attraction $\mathbb{B}$ of $\mathbb{A}$, such that for any open neighborhood $\mathbb{V}$ of $\mathbb{A}$ there is a positive constant T such that $f(t,b)\in \mathbb{V}$, $\forall t>T$ and $\forall b\in \mathbb{B}$;

3. 

There is no non-empty subset of $\mathbb{A}$ having the first two properties.

Indeed, the iterative update (5) with a step length (6) can be seen as a dynamic system $s(t,x)$, where t is the iteration k and x define the player variables. When $\eta \to 0$, the dynamic system can be visualized by the vector field given by (4). Convergence and optimality proofs for the proposed algorithm are given next.

Theorem 1

(Convergence). The iterative update (5) with step length (6) converges to an attractor of vector field (4) for $\eta \to 0$ and a enough large k if at least one attractor exists and the starting point $x_0$ belongs to its basin of attraction.

Proof. 

The iterative update (5) with given $\eta \to 0$ for step length (6) defines a dynamic system given by the vector field (4) and the convergence to an attractor follows by construction. □

Theorem 2

(Optimality). Every fixed point attractor of vector field (4) has necessary conditions to be also a Nash equilibrium of game (1)–(3) $\forall \lambda >0$ for functions $f_v$ continuously differentiable in x.

Proof. 

If the attractor is a single fixed point, then $F\left(x\right)=0$ because otherwise the respective dynamic system would continue moving. This $F\left(x\right)=0$ defines a necessary optimality condition for each player, i.e., a Nash equilibrium. □

For pseudo-convex (i.e., objective functions are pseudo-convex to respective player own variables) unconstrained games, the condition $F\left(x\right)=0$ also becomes sufficient for Nash equilibria. Indeed, if pseudo-convexity is proven for a game and the proposed algorithm converges to a single point, then it is a Nash equilibrium. Furthermore, for finite pseudo-convex potential games, the proposed algorithm converges to a Nash equilibrium just like a classical gradient algorithm converges to the minimum of finite pseudo-convex optimization problems. For convex games, the proposed enhanced gradient algorithm converges to the Nash equilibrium just like Rosen gradient algorithm because $∥F\left(x_k\right)∥$ is reduced after each iteration k for a small enough $\eta$.

2.3. Examples for Convergence Analysis

Consider a 2-player game where one player controls the price that it will sell a product and the other player controls the amount of the product it will buy. Both players are penalized for rising their control variables squared by a factor $c/2$. The game of maximum gain of player 1 and minimum expenses of player 2 can be written as

$$\begin{array}{cc}\hfill x_1^{★}& =arg\underset{x_1}{min}{f}_1(x_1,x_2^{★})={\displaystyle \frac{1}{2}}c{x}_1^2-x_1{x}_2^{★}\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill x_2^{★}& =arg\underset{x_2}{min}{f}_2(x_1^{★},x_2)={\displaystyle \frac{1}{2}}c{x}_2^2+x_1^{★}{x}_2\hfill \end{array}$$

(8)

for $c>0$, so that each objective function is convex relative to respective player variable.

The best response curves are given by

$$\begin{array}{cc}\hfill X_1^{★}\left(x_2\right)& ={\displaystyle \frac{x_2}{c}}\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill X_2^{★}\left(x_1\right)& =-{\displaystyle \frac{x_1}{c}}\hfill \end{array}$$

(10)

so that it has a unique Nash equilibrium at

$$x^{★}=(0,0)$$

(11)

The iterative update (5) with step length (6) converges to the Nash equilibrium of the game (7) and (8) whenever

$$\eta <\overline{\eta }={\displaystyle \frac{2c^2}{1+c^2}},\forall \lambda$$

(12)

given by $\eta =1-\widehat{F}{\left(x_k\right)}^T\widehat{F}\left(x_{k+1}\right)$ where $∥x_k∥>∥x_{k+1}∥$ and $x_{k+1}=x_k+\alpha F\left(x_k\right)$. Furthermore, it converges at maximum rate for

$${\eta }^{★}=1-{\displaystyle \frac{1}{\sqrt{1+c^2}}},\forall \lambda$$

(13)

given by $\eta =1-\widehat{F}{\left(x_k\right)}^T\widehat{F}\left(x_{k+1}\right)$, where $x_{k+1}^{T}F\left(x_k\right)=0$ and $x_{k+1}=x_k+\alpha F\left(x_k\right)$. Notice that ${\eta }^{★}<\overline{\eta }$, $\forall c,\lambda$.

Hence, even for convex games with a unique Nash equilibrium, the iterative update (5) may not converge if $\eta \in (0,2)$ is large enough. Furthermore, a large convergent step length may lead to a slower convergence, just like a small step length. Indeed, even the relaxation algorithm [20] only converges to the Nash equilibrium of the game (7) and (8) for specific fixed step lengths. For $c=0$, the game becomes linear (i.e., objective functions are linear to respective player own variables), and there is still a Nash equilibrium at $x=(0,0)$, but both algorithms cannot find it starting from somewhere else.

An interesting example of a non-convex game is

$$\begin{array}{cc}\hfill x_1^{★}& =arg\underset{x_1}{min}{f}_1(x_1,x_2^{★})=-x_1{x}_2^{★}+c{x}_1^2\left({\displaystyle \frac{1}{4}}{x}_1^2+{\displaystyle \frac{1}{2}}{x}_2^{★2}-r^2\right)\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill x_2^{★}& =arg\underset{x_2}{min}{f}_2(x_1^{★},x_2)=x_1^{★}{x}_2+c{x}_2^2\left({\displaystyle \frac{1}{4}}{x}_2^2+{\displaystyle \frac{1}{2}}{x}_1^{★2}-r^2\right)\hfill \end{array}$$

(15)

whose attractor is the circle

$$x_1^2+x_2^2=r^2$$

(16)

for all ${\mathbb{R}}^2$, except at the origin $x=(0,0)$, where $F\left(x\right)=0$, but it is not a Nash equilibrium. Indeed, this game has no Nash equilibrium. The proposed algorithm will never stop for this game, being trapped in the circular attractor, unless the starting point is the origin, where it stops immediately because of a necessary optimality condition that is not sufficient.

3. Generalized Nash Equilibrium

A generalized Nash equilibrium is defined by

$$\begin{array}{cc}\hfill x_1^{★}& =arg\underset{x_1}{min}{f}_1(x_1,x_{-1}^{★}):x_1\in {\mathbb{X}}_1\left(x_{-1}^{★}\right)\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill x_2^{★}& =arg\underset{x_2}{min}{f}_2(x_2,x_{-2}^{★}):x_2\in {\mathbb{X}}_2\left(x_{-2}^{★}\right)\hfill \\ \hfill \dots \end{array}$$

(18)

$$\begin{array}{cc}\hfill x_p^{★}& =arg\underset{x_p}{min}{f}_p(x_p,x_{-p}^{★}):x_p\in {\mathbb{X}}_p\left(x_{-p}^{★}\right)\hfill \end{array}$$

(19)

where ${\mathbb{X}}_v\left(x_{-v}\right)\subseteq {\mathbb{R}}^{n_v}$ is the feasible set of player v as a function of other player variables $x_{-v}$.

Consider the joint constraints

$${\mathbb{X}}_v\left(x_{-v}\right)=\left\{x_v\right|(x_v,x_{-v})\in \mathbb{X}\}$$

(20)

where $\mathbb{X}=\left\{x\right|g\left(x\right)\le 0\}\subseteq {\mathbb{R}}^n$ is the feasible set for joint constraint functions $g:{\mathbb{R}}^n\to {\mathbb{R}}^m$.

3.1. Generalized Gradient Algorithm

The generalized algorithm can be defined by the iterative update

$$x_{k+1}=x_k+{\alpha }^{★}{\widehat{d}}_k$$

(21)

where the step length is

$${\alpha }^{★}=\underset{\alpha }{max}\alpha :{\widehat{d}}_k^T\widehat{F}\left(x_{k+1}\right)\ge 1-\eta ,g\left(x_{k+1}\right)\le 0,\eta \in (0,2)$$

(22)

starting from a feasible point $x_0\in \mathbb{X}$.

Define the directions

$$D=[F\left(x\right)-\nabla g_{\mathbb{J}}\left(x\right)]$$

(23)

where

$$\mathbb{J}=\left\{j\right|g_j\left(x\right)=0\}$$

(24)

is the active set of constraints. Then, the unit search direction $\widehat{d}$ must lie inside the cone $D^{T}d>0$ in order to walk at least locally in the direction of the vector field while reducing the constraint function values. A good choice is a direction d so that not only $D^{T}d>0$, but also $d=Dw$, $w\ge 0$ and $w\ne 0$ [24]. A direction in this intersection can be found by the linear optimization problem,

$$\begin{array}{cc}\hfill \mathrm{minimize}& -s\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill \mathrm{subject} \mathrm{to}& -D^{T}Dw+\sigma s\le 0\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill & \sum _{j=1}^{n_d}{w}_j=1\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill & w\in {[0,1]}^{n_d}\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill & s\in \mathbb{R}\hfill \end{array}$$

(29)

where $\sigma \in {\mathbb{R}}^{n_d}$ defines a weight for each director. These weights $\sigma$ allow us to obtain directions $\widehat{d}$ closer to $F\left(x_k\right)$, which improves the convergence rate and also reduces divergent behaviors of the algorithm (as seen for Problems (7) and (8)). A weight w relative to $F\left(x\right)$ four times the ones for activating nonlinear constraints and 4000 times the ones for activating linear constraints is typical for a good performance in practice. The solution of this linear problem $(w^{★},s^{★})$ defines the search direction,

$$d_k=D{w}^{★}$$

(30)

if $s^{★}>0$, otherwise the algorithm stops.

3.2. Generalized Gradient Algorithm Analysis

The primal cone of directions $d=Dw$, $w\ge 0$, $w\ne 0$, is introduced in the formulation (25)–(29) only to lead to a better (i.e., more centralized) search direction. The following lemma proves that its intersection with a dual nonempty cone $D^d>0$ is also always nonempty.

Lemma 1

(cone intersection). If $D^{T}d>0$ then $\exists w\ge 0,w\ne 0$ so that $Dw=d$.

Proof. 

Suppose the direction d can be written as $Dw$. If it belongs to the cone $D^{T}d>0$, then must exist a $w\ge 0$, $w\ne 0$ such that

$$\begin{array}{cc}\hfill D^{T}d=D^{T}Dw& >0\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill \Rightarrow c^T{D}^{T}Dw& >0,\forall c\ge 0,c\ne 0\hfill \end{array}$$

(32)

which can be interpreted as if exists a strictly supporting hyperplane at the origin to the cone $\mathbb{C}=\left\{d\right|d=Dc,c\ge 0c\ne 0\}$ whose normal $Dw$ belongs to $\mathbb{C}$. Because $\mathbb{C}$ is a convex set contained in an open halfspace intersection (i.e., $D^{T}d>0$), the origin lies in the boundary of the closure of $\mathbb{C}$, the existence of w is guaranteed by the supporting hyperplane theorem for convex sets. □

The proofs of convergence and optimality of the proposed algorithm to find a generalized Nash equilibrium are given next.

Theorem 3

(generalized convergence). The iterative update (21) with step length (22) converges to an attractor of vector field (4) constrained to $x\in \mathbb{X}$ for $\eta \to 0$ and a large enough k if at least one attractor exists and the starting point $x_0$ belongs to its basin of attraction.

Proof. 

The linear programming provides a direction $d=Dw$, $w\ge 0$, $w\ne 0$, such that $D^{T}d>0$ if $s^{★}>0$. Because of Lemma 1, it is sufficient to consider a direction d such that $D^{T}d>0$, since in this case d can always be written as $Dw$, $w\ge 0$, $w\ne 0$. When there is a direction d that satisfies $D^{T}d>0$, then there exists an infinitesimal step towards d that does not violate any constraint and still has a component on vector field $F\left(x\right)$. Hence, the algorithm converges to an attractor of vector field (4) constrained to $x\in \mathbb{X}$ for $\eta \to 0$ by construction. □

Theorem 4

(Generalized optimality). Every fixed point attractor $x^{★}$ of the vector field (4) constrained to $x\in \mathbb{X}$ satisfies the variational constraint $F{\left(x^{★}\right)}^T(y-x^{★})\le 0$, $\forall y\in \mathbb{X}\cap \mathbb{B}$, where $\mathbb{B}$ is a ball of infinitesimal radius around $x^{★}$. Furthermore, every solution $x^{★}$ to the variational inequality $F{\left(x^{★}\right)}^T(y-x^{★})\le 0$, $\forall y\in \mathbb{X}$, for a closed convex set $\mathbb{X}$ and the vector field $F\left(x\right)$ in (4) is also a generalized Nash equilibrium for all $\lambda >0$ of the game (17)–(19), with constraints respecting (20) and objective functions $f_v$ continuously differentiable in x and pseudo-convex in $x_v$.

Proof. 

Every fixed point attractor $x^{★}$ of $F\left(x\right)$ constrained to $x\in \mathbb{X}$ is characterized when there is no feasible direction to step towards $F\left(x\right)$ by definition, so that mathematically leads to $F{\left(x^{★}\right)}^{T}d\le 0$, $\forall d:\nabla g{\left(x^{★}\right)}^{T}d\le 0$, which is equivalent to $F{\left(x^{★}\right)}^T(y-x^{★})\le 0$, $\forall y\in \mathbb{X}\cap \mathbb{B}$.

Let $x_v$ be any point in ${\mathbb{X}}_v\left(x_{-v}^{★}\right)$. From (20), it is clear that $y=(x_v,x_{-v}^{★})\in \mathbb{X}$. Since $x^{★}$ is a solution to the variational inequality and $y\in \mathbb{X}$, the definition of vector field (4) based on diffentiability of $f_v$ leads to

$$0\le -F{\left(x\right)}^T(y-x^{★})={\lambda }_v{\nabla }_{x_v}{f}_v{(x_y,x_{-v}^{★})}^T(x_v-x_v^{★}),\forall {\lambda }_v>0,\forall x_v\in {\mathbb{X}}_v\left(x_{-v}^{★}\right),$$

(33)

so that $x^{★}$ is a solution of game (17)–(19) by the minimum principle under the assumed pseudo-convexity of $f_v$. □

Every point that satisfies the variational inequality $F{\left(x^{★}\right)}^T(y-x^{★})\le 0$ is called normalized Nash equilibrium [12] or variational equilibrium [25], which are the type of equilibrium that the proposed gradient algorithm is able to find. Every normalized Nash equilibrium is a generalized Nash equilibrium, but the converse is not true in general [13]. The following example illustrates this fact.

3.3. An Example with Infinite Generalized Nash Equilibria

Consider the two-player game [13],

$$\begin{array}{cc}\hfill x_1^{★}& =arg\underset{x_1}{min}{f}_1(x_1,x_2^{★})={(x_1-1)}^2:x_1+x_2^{★}\le 1\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill x_2^{★}& =arg\underset{x_2}{min}{f}_2(x_1^{★},x_2)={\left(x_2-{\displaystyle \frac{1}{2}}\right)}^2:x_1^{★}+x_2\le 1\hfill \end{array}$$

(35)

whose best response curves are given by

$$\begin{array}{cc}\hfill X_1^{★}\left(x_2\right)& =\left\{\begin{array}{cc}1,\hfill & x_2\le 0\hfill \\ 1-x_2,\hfill & x_2>0\hfill \end{array}\right.\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill X_2^{★}\left(x_1\right)& =\left\{\begin{array}{cc}{\displaystyle \frac{1}{2}},\hfill & x_1\le {\displaystyle \frac{1}{2}}\hfill \\ 1-x_1,\hfill & x_1>{\displaystyle \frac{1}{2}}\hfill \end{array}\right.\hfill \end{array}$$

(37)

so that the generalized Nash equilibria are

$$x^{★}=\left({\displaystyle \frac{1}{2}}(1+a),{\displaystyle \frac{1}{2}}(1-a)\right),a\in [0,1]$$

(38)

Considering that $x=(1,1/2)$ is infeasible so that $F\left(x\right)\ne 0$ for all feasible points, the variational inequality $F{\left(x\right)}^T(y-x)\le 0$, $\forall y\in \mathbb{X}$, holds true for $\mathbb{X}=\{x=(x_1,x_2)|x_1+x_2\le 1\}$ only when $F\left(x\right)=(b,b)$, $b\ge 0$ for $x_1+x_2=1$, so that

$$\begin{array}{cc}\hfill F\left(x\right)& =-({\lambda }_1(2x_1-2),(1-{\lambda }_1)(2x_2-1)),{\lambda }_1\in (0,1)\hfill \\ \hfill & =-({\lambda }_1(2x_1-2),(1-{\lambda }_1)(1-2x_1)),{\lambda }_1\in (0,1)\hfill \\ \hfill & =(b,b)\hfill \end{array}$$

(39)

so that, because $\lambda \ne 0$ and $\lambda \ne 1$ by definition of the vector field (4), only the Nash equilibria

$$x^{★}=\left({\displaystyle \frac{1}{2}}(1+{\lambda }_1),{\displaystyle \frac{1}{2}}(1-{\lambda }_1)\right),{\lambda }_1\in (0,1)$$

(40)

can be found by the proposed algorithm, leaving two generalized Nash equilibria out of reach: $x^{★}=(1,0)$ and $x^{★}=(1/2,1/2)$. However, if no positive weights $\lambda$ were associated with $F\left(x\right)$, only a single generalized Nash equilibria could be found: $x^{★}=(3/4,1/4)$, the one where ${\lambda }_1={\lambda }_2=1/2$.

3.4. A Numerical Example: Internet Game

Consider the extended internet switching p-player symmetrical game [26]

$$x_v^{★}=arg\underset{x_v}{min}{\displaystyle \frac{{\sum }_{i=1}^n{x}_{v,i}}{B}}-{\displaystyle \frac{{\sum }_{i=1}^n{x}_{v,i}}{{\sum }_{u=1}^p{\sum }_{i=1}^n{x}_{u,i}}}:\sum _{u=1}^p\sum _{i=1}^n{x}_{u,i}\le B,x_v\ge ϵ,v=1,\dots ,p$$

(41)

which has Nash equilibria at points satisfying

$$\sum _{i=1}^n{x}_{v,i}^{★}={\displaystyle \frac{B(p-1)}{p^2}},v=1,\dots ,p$$

(42)

which is unique when $n=1$.

Figure 1 shows the typical convergence of the proposed enhanced gradient algorithm (EGNE) for $\eta =1$ compared to the relaxation algorithm (RNE) for a fixed step of $1/2$, Rosen gradient algorithm (GNE), and BRD algorithm (BRD), measured by number of function evaluations. Notice that EGNE, GNE, and BRD are typically one order of magnitude faster than RNE for $p=5$. As the number of players increases, or the number of variables increases, this difference becomes even greater as shown in Figure 2 and Figure 3, respectively. This is because relaxation algorithm solves an optimization problem every iteration, while the proposed algorithm and GNE only performs a line search, and BRD solves an optimization problem with less variables. Notice also that the relaxation algorithm (RNE) becomes unstable for $n\ge 4$.

Figure 1. Convergence of proposed Enhanced Gradient Algorithm (EGNE) for $\eta =1$ and ${\lambda }_v=1/5,$ $v=1,\dots ,5$ (EGNE), relaxation algorithm for a fixed step of $1/2$ (RNE), Rosen gradient algorithm for ${\lambda }_v=1/5,v=1,\dots ,5$ (GNE), and the BRD algorithm (BRD), for game (41) with $p=5$, $n=1$, $B=1$ and $ϵ={10}^{-2}$.

Figure 2. Average computational cost of 30 solutions of game (41) with proposed enhanced gradient algorithm for $\eta =1$ and ${\lambda }_v=1/p,v=1,\dots ,p$ (EGNE), relaxation algorithm for a fixed step of $1/2$ (RNE), Rosen gradient algorithm for ${\lambda }_v=1/p,v=1,\dots ,p$ (GNE), and BRD algorithm (BRD), for an increasing number of players p starting from a random feasible point $x_{0,v}\in [ϵ,B/p]$, $v=1,\dots ,p$, with $n=1$, $B=1$ and $ϵ={10}^{-2}$. The stop criterion is to move closer than a Euclidean distance of ${10}^{-6}$ to the know Nash equilibrium.

Figure 3. Average computational cost of 30 solutions of game (41) with proposed enhanced gradient algorithm for $\eta =1$ and ${\lambda }_v=1/p,v=1,\dots ,p$ (EGNE), relaxation algorithm for a fixed step of $1/2$ (RNE), Rosen gradient algorithm for ${\lambda }_v=1/p,v=1,\dots ,p$ (GNE), and BRD algorithm (BRD), for an increasing number of variables n starting from a random feasible point $x_{0,v}\in [ϵ,B/\left(np\right)]$, $v=1,\dots ,p$, with $p=4$, $B=1$ and $ϵ={10}^{-6}$. The stop criterion is to move closer than a Euclidean distance of ${10}^{-6}$ to the known Nash equilibrium.

4. Energy Market Application

The current Brazilian electric sector is mainly powered by hydroelectric plants, that are dispatched centrally by the system operator, that calculates the water future cost of each to estimate the energy spot prices. As the generation of hydroelectric power plants are strongly affected by water inflow, which depends on the rain regime and upstream hydroelectric plant generation, there is a compulsory financial hedge mechanism to share the hydrological risk of all hydroelectric power plants. In this mechanism, called energy reallocation mechanism, the generation of all hydroelectric power plants is bundled together and each plant payment is proportional to its physical guarantee (PG) that is prefixed for commercialization and does not depend on its actual generation [22].

In this context, each genco v can seasonalize their hydroelectric plants PG $x_{v,t}$ along months $t=1,\dots ,T$, where T is the number of periods ($T=12$ in this paper for real seasonalization game), as long as its yearly average PG $X_v$ is unchanged. This opens possibility to each genco to seasonalize strategically in order to maximize its own total income $R_{v,t}$ along periods t, which will be modulated by a generation scaling factor, i.e., the ratio between the actual generation $H_t$ and the amount of PG allocated by all players in each period t. The Nash equilibrium of the seasonalization non-cooperative game with p-player can be formulated by [22]:

$$x_v^{★}=arg\underset{x_v}{max}\sum _{t=1}^T{R}_{v,t}\left(x\right):\sum _{t=1}^T{x}_{v,t}\le T{X}_v,\underline{\rho }{X}_v\le x_v\le \overline{\rho }{X}_v,v=1,\dots ,p$$

(43)

where the income of each player v in each period t is given by

$$R_{v,t}\left(x\right)=h_t{P}_{m_v,t}\left[{\displaystyle \frac{x_v{H}_t}{{\sum }_{u=1}^4{x}_{u,t}}}-(1-s)X_v{\eta }_t\right]$$

(44)

where $h_t$ is the duration of period t, $P_{m,t}$ is the spot price of period t in sub-market m, $m_v$ is the sub-market where player v is located, s is the reduction ratio, and ${\eta }_t$ is the generation scaling factor for a flat seasonalization so that a negative income means a worse income than a reference flat seasonalization. Considering $\overline{\rho }>\underline{\rho }>0$ and $X_v>0$, $\forall v$, the objective function of each player is smooth and pseudo-convex over the feasible domain. The linear constraint functions tend to be activated as the players maximize their income, which is especially difficult to handle, and makes them a challenging test for the constraint handling proposed in this paper.

Figure 4 shows a typical convergence for the seasonalization 4-player game considering one player per sub-market (i.e., $m_v=v$) and year 2020 input data given by $X_1=\mathrm{32,926}$, $X_2=7186.4$, $X_3=5962.1$, $X_4=9948.7$, $s=0.1$, $\underline{\rho }=0.5$, $\overline{\rho }=1.6$, and the parameters given in Table 1. Table 2 shows the respective Nash equilibrium obtained by the proposed algorithm. The maximum variable variation for the proposed enhanced gradient algorithm in 5 runs starting from random points was about ${10}^{-6}$, while for the relaxation algorithm was about ${10}^{-1}$ and the Rosen gradient algorithm was ${10}^1$. The proposed enhanced gradient algorithm convergence is about six times faster than the relaxation algorithm, and a bit faster than the Rosen gradient algorithm.

Figure 4. Typical convergence for the seasonalization game using the proposed gradient algorithm for $\eta =1$ and ${\lambda }_v=1/4,i=1,\dots ,4$ (EGNE) the relaxation method for a fixed step of $1/2$ (RNE), and Rosen gradient algorithm for ${\lambda }_v=1/4,i=1,\dots ,4$ (GNE), compared to EGNE solution.

Table 1. Seasonalization game parameters.

t	${\mathit{h}}_{\mathit{t}}$	${\mathit{H}}_{\mathit{t}}$	${\mathit{\eta }}_{\mathit{t}}$	${\mathit{P}}_{1,\mathit{t}}$	${\mathit{P}}_{2,\mathit{t}}$	${\mathit{P}}_{3,\mathit{t}}$	${\mathit{P}}_{4,\mathit{t}}$
1	744	54,169	0.99670380	13,767	13,757	13,735	13,701
2	672	56,765	1.03818298	12,148	12,148	11,182	11,117
3	744	58,476	1.00380676	11,781	11,782	10,617	9953
4	720	52,581	0.83536170	10,342	10,334	9652	8529
5	744	48,936	0.86609332	10,067	10,042	9505	8848
6	720	46,301	0.91454965	10,048	9937	9742	9772
7	744	45,573	0.98870119	10,234	10,080	9672	9782
8	744	45,880	1.01770868	10,161	9991	9736	9879
9	720	46,472	1.06667054	10,369	10,167	9984	10,094
10	744	48,262	1.09996905	10,293	10,115	10,021	10,086
11	720	48,630	1.09499348	10,126	10,021	9802	9830
12	744	51,927	1.07686986	8659	8645	8419	8401

Table 2. Nash equilibrium for seasonalization game.

t	${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{x}}_3$	${\mathit{x}}_4$
1	46,805	8982.91	9539	15,918
2	38,630	9569	5685	11,247
3	42,278	11,498	6779	9215
4	32,363	9014	6590	5655
5	30,513	7621	5851	7303
6	28,156	5434	5240	9628
7	29,064	5962	4,816	9687
8	29,121	5574	5102	10,117
9	29,136	5415	5285	10,141
10	31,089	5635	5916	10,903
11	29,806	5811	5500	10,138
12	28,151	5719	5240	9433

Figure 5 shows how the EGNE algorithm scales with an increasing number of players for $\underline{\rho }=1/2$, $\overline{\rho }=3/2$, $T=12$, $M=4$, $P_{m,t}\in [10,600]$, $H_t\in [0,4\times {10}^3]$, $X_v\in [{10}^3,3\times {10}^3]$ and $m_v\in \{1,\dots ,M\}$.

Figure 5. Average computational cost of 30 solutions of seasonalization game with proposed enhanced gradient algorithm for $\eta =1$ and ${\lambda }_v=1/p,v=1,\dots ,p$ (EGNE), for an increasing number of players p starting from a random feasible point $x_{0,v}\in [\underline{\rho }{X}_v,X_v]$, $v=1,\dots ,p$.

5. Conclusions

This paper proposes an enhanced gradient algorithm to find a generalized Nash equilibrium with joint constraints that seems to have similar convergence and optimality behavior when compared to the state-of-the-art relaxation method, but it is typically about one order of magnitude faster in practice. A promising future application is to open or semi-open electricity market game formulations, where the players make price offers and the consumers or system-market operators define which player will actually generate energy. Furthermore, further details can be added to the electricity market game formulation, e.g., control mechanisms to assert proper operation or prevent spurious behavior under any scenario.

The convergence and optimality guarantees of the proposed algorithm are based on the concept of attractor. Future developments in vector field conditions for existence and uniqueness of fixed point attractors can improve the ability to give theoretical guarantees to find a Nash equilibrium for specific game instances.