Co-Operative Optimization Framework for Energy Management Considering CVaR Assessment and Game Theory

Abstract

In this paper, a bi-level energy management framework based on Conditional Value at Risk (CVaR) and game theory is presented in the context of different ownership of multiple microgrid systems (MMGS) and microgrid aggregators (MAs). The energy interaction between MMGS and MAs can be regarded as a master–slave game, where microgrid aggregators as the leaders set the differentiated tariff for each MG to maximize its benefits, and MMGS as the follower responds to the tariff decision specified by the leader through peer-to-peer (P2P) energy sharing. The P2P energy sharing of MMGS can be regarded as a co-operative game, employing asymmetric Nash bargaining theory to allocate the co-operative surplus. The Conditional Value at Risk model was used to characterize the expected losses by microgrid aggregators due to the uncertainties of renewable energy resources. The Karush–Kuhn–Tucker conditions, Big-M method, and strong duality theory were employed to transform the bi-level nonlinear model of energy management into a single-level mixed integer linear programming model. The simulation results show that when MGs adopt the P2P energy-sharing operation mode, the total operating cost of MMGS can be reduced by 7.82%. The simulation results show that the proposed co-operative optimization framework can make the multiple microgrid systems obtain extra benefits and improve the risk resistance of microgrid aggregators.

1. Introduction

The mainstream direction of energy development in the world today is efficient, clean and low-carbon [1]. With the maturity of distributed power generation technology in recent years, distributed renewable energy generation has been developing rapidly [2]. A microgrid is an effective way to consume renewable energy, but the volatility and intermittency of distributed power generation hinder its consumption [3].

The identity of microgrids has gradually changed from traditional consumers to producers and sellers, and microgrids can not only trade electricity with distribution networks but also between MGs [4]. Microgrids enhance the autonomy and consumption capacity of MMGS and realize economic benefits through energy mutualization and co-operative optimization control [5]. In recent years, many scholars have achieved fruitful results on the economic operation of multi-microgrids and the benefit allocation problem. In [6], the author establishes a Nash bargaining model for the economic dispatch of multi-operator MGs by day based on a co-operative game. The model not only enables each operator to obtain the Pareto optimal cost but also minimizes the cost of MGs. In [7], the author proposes a co-operative MMGS coordination optimization method based on an opportunity-constrained planning method. This method builds a model to minimize the total cost of the system and considers the co-operative scheduling method among MGs, but does not conduct an in-depth study of the co-operative conditions. In [8], the author proposes a mechanism of MMGS formation co-operation and builds a MMGS coalition-type game model considering the risk of power outage. In [9], the author establishes a co-operative game-theory-based market buying and selling model for MMGS, and formulates real-time transaction settlement rules based on the profits gained by MGs. Vehicle electrification is an irreversible trend nowadays. After the cluster optimization management of EVs, EV clusters can provide high-quality energy storage resources with larger capacity for the power system. However, EVs differ significantly from traditional energy storage devices in terms of the uncertainty of EV users’ vehicle behavior and the need to consider users’ charging needs. The paper [10] proposes a machine learning approach for energy management considering the advanced support vector machine for modeling and estimating the charging demand of EVs. This approach can improve power quality and operation in renewable microgrids. The P2P trading mechanism is a decentralized trading model for the distribution network side of the market. The paper [11] proposes a secured management framework in the P2P energy trading process. This method can effectively prevent systematic risks such as malicious attacks in the P2P trading process. Social, technical, economic, environmental and political factors need to be taken into account in the design of microgrid energy management. In combination with energy storage technology, this “energy buffer” has been used to find the best solution for energy management [12].

The interest game between MGs and the distribution network as different interest holders brings a great challenge to the economic dispatch of MMGS. To reasonably allocate the interests, formulating the game strategy of conflict and co-operation between multi-interest holders is one of the key problems to be solved in economic dispatching, and one of the important ways for decision-makers to win–win. The literature [13] refines the operational constraints and interest game of MGs and a distribution network based on the objective cascade analysis method, and realizes the decoupling and parallel solution of the operation of the two. The paper [14] constructs a multi-buyer–multi-seller pattern and builds a Stackelberg game demand–response model with operators and customers as dominant and followers, respectively, which effectively improves the operator’s revenue and customers’ electricity efficiency while enhancing the level of PV power sharing among customer groups. In order to encourage each MG operator to participate in the power trading of MMGS, the paper [15] proposed an internal tariff optimization model based on the master–slave game and a two-stage optimization model of MMGS, which not only reduces the net load-to-peak ratio of MMGS but also achieves the win–win economic benefits for both MMGS operators and MG operators. The paper [16] equates the bargaining transaction problem among multiple MG operators to a co-operative game optimization model and uses the Nash bargaining solution to solve the MMGS power trading problem.

In [17], a P2P energy trading approach based on a dual-auction market is proposed, in which each producer and consumer initiates trade requests among themselves by interacting with limited information, taking into account historical trade decision information and aiming at a maximum individual benefit. Unlike the auction-based P2P trading mechanism, the multi-master–multi-slave game trading model is a “dynamic” game in which producers and consumers can interact with each other in terms of decisions and behaviors. The authors of [18] analyze the problem of seller price competition and buyer choice competition in P2P transactions by using non-co-operative game theory and evolutionary game theory. The authors of [19] propose a user-level autonomous scheduling model for distributed market participants and a P2P trading mechanism that considers a multi-subject non-co-operative game. However, a market trading model with flexible adjustment of purchase and sale roles based on real-time prices remains to be studied in depth. In [20], the author proposed an energy management strategy for a multi-microgrid, verified the interconnection and mutual benefit strategy through the multi-microgrid, and used Shapley value for benefit allocation. However, in [14,16,17], the degree of contribution in the MG co-operation process was not considered, and excessive preoccupation with the fairness of distribution tends to weaken the motivation of participants to cooperate. In [13,14,15,18,19], using a non-co-operative game to solve the MG benefit allocation problem, the non-co-operative game emphasizes individual rationality, resulting in the Nash equilibrium potentially not existing.

The incorporation of the microgrid aggregator into the distribution network investment model has become an emerging business model to realize the win–win situation of different interest subjects through the coupled decision-making role of the microgrid aggregator and the distribution network operator. Therefore, in this paper, MAs were used as an intermediary between the distribution network and MMGS, so that it can participate in the decision of MMGS optimal dispatch. MAs can not only maintain the order of the MG trading platform by setting the electricity tariff, but also earn profits through the price difference.

The main innovation points of this paper can be summarized as:

• The master–slave game model can take into account the interest demands of each participant, while the direct P2P energy-sharing behavior among MGs can reduce the market power of leaders and improve social welfare. In terms of coalition cost sharing, this paper proposes an allocation method that considers the contribution of cooperation and adopts asymmetric Nash bargaining theory to allocate the co-operation surplus to make the allocation result fairer and more reasonable.
• In the proposed energy management model, MAs integrate social welfare and risk levels in order to determine price discrimination behavior for the whole process of coordinated electricity trading.

The remaining pages of this article are organized as follows: Section 2 gives the problem to be solved in this paper; the mathematical model of the system is given in Section 3; Section 4 gives the bi-level optimization model based on the Stackelberg game and its solution methodology; Section 5 and Section 6 give the case studies and conclusion.

2. Problem Description

The energy management framework schematic of the multi-microgrids is shown in Figure 1. The energy management problem of Figure 1 can be formulated into a master–slave game, where the MA, as the leader of the Stackelberg game, can interact with the distribution grid and MMGS for electric energy, and aims to maximize its own economic benefit $F_{\mathrm{MA}}$, setting the trading tariff $({\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t},{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t})$ for the MMGS. Each MG, as a follower, aims to minimize the cost through the co-operation between surplus and shortage MGs, and it feeds back the calculated optimal set of power purchase/sale plans $(P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t},P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t})$ to the upper layer based on the internal power prices issued by the upper layer while satisfying its own constraints. The P2P energy sharing problem can be formulated into a Nash bargaining problem, and the P2P trading tariffs ${\pi }_{\mathrm{P}2\mathrm{P}}^{ij,t}$ are set based on the contribution of each MG in the energy sharing process. Moreover, the CVaR is to be taken into account when developing the optimal energy management scheme to resist the uncertainty of renewable energy generation.

3. System Modelling

In this section, we will model the various aspects of Figure 1, which are divided into three main parts; namely, the calculation of CVaR, the modeling of MA and the modeling of MMGS.

3.1. Caculation of the CVaR Value

The uncertainty of renewable energy sources and loads can lead to risky losses in microgrid aggregators operations. The VaR and CVaR are commonly used risk management methods, where CVaR can compensate for the shortcomings of VaR with tail risk [21,22].

$f\left(\mathit{G},\mathit{y}\right)$ is a loss function that is represented by a decision variable $\mathit{G}$ and a random variable $\mathit{y}$, with a probability function $p\left(\mathit{y}\right)$. The probability of $f\left(G,y\right)$ does not exceed a given limit of $\alpha$, and it can be expressed in the following form:

$$\psi \left(\mathit{G},\alpha \right)={\displaystyle \underset{f\left(\mathit{G},\mathit{y}\right)\le \alpha }{\int }p\left(\mathit{y}\right)\mathrm{d}\mathit{y}}$$

(1)

where $\mathit{y}:=\left[p^{\mathrm{WT}},p^{\mathrm{PV}},p^{\mathrm{EL}}\right]$ denotes the set of random vectors, including the wind power output $p^{\mathrm{WT}}$, photovoltaic power output $p^{\mathrm{PV}}$, and electric load $p^{\mathrm{EL}}$. $p\left(\mathit{y}\right)$ denotes the joint probability density function. $\mathit{G}=\left[g_1^{\mathrm{MG}},g_2^{\mathrm{MG}},\cdots ,g_M^{\mathrm{MG}}\right]$ denotes the set of decision vectors. $\psi \left(G,\alpha \right)$ is a right continuous and non-decreasing function.

For a given confidence level $\beta$, VaR and CVaR can be calculated as:

$$Va{R}_{\beta }\left(\mathit{G}\right)=\min \left\{\alpha \in \mathit{R}:\psi \left(\mathit{G},\alpha \right)\ge \beta \right\}$$

(2)

$$CVa{R}_{\beta }\left(\mathit{G}\right)=\frac{1}{1-\beta }{\displaystyle \underset{f\left(\mathit{G},\mathit{y}\right)\ge V_{\beta }^{\mathrm{VaR}}\left(\mathit{G}\right)}{\int }f\left(\mathit{G},y\right)p\left(\mathit{y}\right)\mathrm{d}\mathit{y}}$$

(3)

where $\beta$ represents the confidence level, and $Va{R}_{\beta }\left(\mathit{G}\right)$ and $CVa{R}_{\beta }\left(\mathit{G}\right)$ are the value at risk and the conditional value at risk in confidence level $\beta$.

However, the parsing expression of $Va{R}_{\beta }\left(\mathit{G}\right)$ is difficult to obtain directly. Therefore, it is more difficult to directly calculate the value of CVaR. Generally speaking, engineering applications use the transformation function shown in (4) to estimate the value of CVaR:

$$F_{\beta }\left(\mathit{G},\alpha \right)=\alpha +\frac{1}{1-\beta }{\displaystyle \int {\left[f\left(\mathit{G},\mathit{y}\right)-\alpha \right]}^{+}p\left(\mathit{y}\right)\mathrm{d}\mathit{y}}$$

(4)

where ${\left[f\left(\mathit{G},\mathit{y}\right)-\alpha \right]}^{+}$ denotes $\max \left\{\hspace{0.17em}\left[f\left(G,\mathit{y}\right)-\alpha \right],0\right\}$.

Equation (4) is difficult to calculate directly. Similarly, the integration term of Equation (4) is usually estimated by using the historical sample data of the random vectors $\mathit{y}$, or by simulating the sample data through Latin hypercube sampling. Assuming that ${\mathit{y}}_1$, ${\mathit{y}}_2$, $\cdots$, ${\mathit{y}}_N$ is a sample value of the random vector, the expression (4) can become the following expression [23].

$${\widehat{F}}_{\beta }\left(\mathit{G},\alpha \right)=\alpha +\frac{1}{N\left(1-\beta \right)}{\displaystyle \sum _{o=1}^N{\left[f\left(\mathit{G},{\mathit{y}}_n\right)-\alpha \right]}^{+}}$$

(5)

where ${\widehat{F}}_{\beta }\left(\mathit{G},\alpha \right)$ is the estimated value of $F_{\beta }\left(\mathit{G},\alpha \right)$. There are a total of N samples and ${\mathit{y}}_n$ denotes the n-th sample.

3.2. Microgrid Aggregator

In the emerging business model, the microgrid aggregator usually acts as an investment company between the grid company and MGs. A microgrid aggregator earns revenue by providing quality services to multiple customers, with a greater focus on revenue and market competitiveness. The microgrid aggregator is the leader of the Stackelberg game and can interact with the grid and multiple microgrid systems. The microgrid aggregator operates with the goal of maximizing its own economic benefits and the objective function comprises the expected operating profit and the risk of loss. The MA problem is presented in (6)–(11).

$$\max {\displaystyle \sum _{w=1}^W{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{\tau }^w}}}\left[{\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{s}}^t{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}-{\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{b}}^t{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}+{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}-{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}\right]\mathsf{\Delta }t+L{\displaystyle \sum _{i=1}^N\left({\alpha }^i-\frac{1}{1-\beta }{\displaystyle \sum _{w=1}^W{\tau }^w{\eta }^{w,i}}\right)}$$

(6)

s.t.

$${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{\min }\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{\max },\forall i\in N,\forall t\in T$$

(7)

$${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{\min }\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{\max },\forall i\in N,\forall t\in T$$

(8)

$$\frac{{\displaystyle {\sum }_{t=1}^T{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}}}{T}\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,\mathrm{ave}},\forall i\in N$$

(9)

$$\frac{{\displaystyle {\sum }_{t=1}^T{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}}}{T}\le {\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,\mathrm{ave}},\forall i\in N$$

(10)

$${\alpha }^i-{\displaystyle \sum _{t=1}^T\left[{\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{s}}^t{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}-{\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{b}}^t{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}+{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}-{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}\right]\le {\eta }^{w,i}},\forall w\in W,\forall i\in N$$

(11)

In the objective function (6), ${\tau }^w$ denotes the probability of occurrence of the scenario $w$. ${\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{s}}^t$ and ${\pi }_{\mathrm{MA}2\mathrm{G},\mathrm{b}}^t$ denote the tariff that the MA sells electricity to the grid and purchases electricity from the grid, respectively. ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}$ and ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}$ denote the tariff that MG i purchases electricity from the MA and sells electricity to the MA at time slot t, respectively. $P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}$ and $P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}$ denote MG i purchasing electricity from the MA and selling electricity to the MA of the scenario $w$ at time slot t. ${\eta }^{w,i}$ denotes the VaR loss of MG i of the scenario $w$. $L$ is the preference factor of the MA.

In constraints (7) and (8), ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{\max }$ and ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{\min }$ denote the maximal limit and the minimum limit of the electricity selling tariffs, respectively. ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{\max }$ and ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{\min }$ denote the maximal limit and the minimum limit of the electricity purchasing tariff, respectively. In constraints (9) and (10), ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,\mathrm{ave}}$ and ${\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,\mathrm{ave}}$ denote the limit of the average daily electricity purchasing tariff and the average daily electricity selling tariff, respectively.

3.3. Multi-Microgrids

In response to MA tariff decisions, MG, as a prosumer, can participate in the peer-to-peer energy sharing directly with neighborhoods, thereby reducing the amount of electricity trading with MAs. The optimization goal of MMGS is to minimize its total operating costs:

$$\min C_{\mathrm{MMGS}}^{}=\min {\displaystyle \sum _{w=1}^W{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{\tau }_{}^w\left[{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}-{\pi }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}+{\kappa }_{\mathrm{MT}}^i{P}_{\mathrm{MT}}^{w,i,t}+{\kappa }_{\mathrm{BESS}}^i\left(P_{\mathrm{BESS},\mathrm{c}}^{w,i,t}+P_{\mathrm{BESS},\mathrm{d}}^{w,i,t}\right)+{\kappa }_{\mathrm{IL}}^i{P}_{\mathrm{IL}}^{w,i,t}\right]}}}\mathsf{\Delta }t$$

(12)

where ${\kappa }_{\mathrm{MT}}^i$ and ${\kappa }_{\mathrm{BESS}}^i$ represent the cost factor for the operation and maintenance of micro gas turbine and battery electrical storage system. ${\kappa }_{\mathrm{IL}}^i$ represent the coefficient of compensation cost for interruptible loads. $P_{\mathrm{BESS},\mathrm{c}}^{w,i,t}$ and $P_{\mathrm{BESS},\mathrm{d}}^{w,i,t}$ represent the charging power and discharging power of the electric storage system, respectively.

The optimization model for followers has the following constraints:

3.3.1. Energy Balance Constraints

The electrical power balance constraint for each MG is shown below:

$$P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}+P_{\mathrm{MT}}^{w,i,t}+P_{\mathrm{BESS},\mathrm{d}}^{w,i,t}+P_{\mathrm{WT}}^{w,i,t}+P_{\mathrm{P}2\mathrm{P}}^{w,i,t}=P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}+P_{\mathrm{BESS},\mathrm{c}}^{w,i,t}+P_{\mathrm{EL}}^{w,i,t}+{\displaystyle \sum _{j\in \mathrm{N}\backslash \{i\}}{P}_{\mathrm{P}2\mathrm{P}}^{w,ij,t}}:{\lambda }_{\mathrm{MMGS}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(13)

where ${\lambda }_{\mathrm{MMGS}}^{w,i,t}$ is the Lagrange multiplier of the electrical power balance constraint.

3.3.2. Transmission Power Constraints

Transmission power constraints include power constraints on power purchase and sale from the microgrid to the MA and power constraints on energy sharing with neighboring microgrids:

$${\displaystyle \sum _{i\in N,j\in N\backslash \{i\}}{P}_{\mathrm{P}2\mathrm{P}}^{w,ij,t}}=0:{\lambda }_{\mathrm{P}2\mathrm{P}}^{w,i,t},\forall w\in W,\forall t\in T$$

(14)

$$0\le P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}\le u_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,\max }:{\lambda }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(15)

$$0\le P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}\le u_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}{P}_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,\max }:{\lambda }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(16)

$$u_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}+u_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}\le 1,\forall w\in W,\forall i\in N,\forall t\in T$$

(17)

where $P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,\max }$ and $P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,\max }$ represent the upper limit of the power purchased and sold by the MG i to the MA, respectively. $u_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}$ and $u_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}$ are binary variables indicating that MG i under the scenario $w$ is in the state of power purchase or power sale at time t, respectively, with 1 if yes and 0 otherwise. ${\lambda }_{\mathrm{P}2\mathrm{P}}^{w,i,t}$, ${\lambda }_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{w,i,t}$, and ${\lambda }_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{w,i,t}$ are the Lagrange multipliers of the transmission power constraints.

3.3.3. Battery Energy Storage System Operating Constraints

The battery energy storage system operating constraints are shown below:

$$0\le P_{\mathrm{BESS},\mathrm{c}}^{w,i,t}\le u_{\mathrm{BESS},\mathrm{c}}^{w,i,t}{P}_{\mathrm{BESS},\mathrm{c}}^{i,\max }:{\lambda }_{\mathrm{BESS},\mathrm{c}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(18)

$$0\le P_{\mathrm{BESS},\mathrm{d}}^{w,i,t}\le u_{\mathrm{BESS},\mathrm{d}}^{w,i,t}{P}_{\mathrm{BESS},\mathrm{d}}^{i,\max }:{\lambda }_{\mathrm{BESS},\mathrm{d}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(19)

$$u_{\mathrm{BESS},\mathrm{c}}^{w,i,t}+u_{\mathrm{BESS},\mathrm{d}}^{w,i,t}\le 1,\forall w\in W,\forall i\in N,\forall t\in T$$

(20)

$$E_{\mathrm{BESS}}^{i,\min }\le E_{\mathrm{BESS}}^{w,i,t}\le E_{\mathrm{BESS}}^{i,\max }:{\lambda }_{{\mathrm{SOC}}_{\min }}^{w,i,t},{\lambda }_{{\mathrm{SOC}}_{\max }}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(21)

$$E_{\mathrm{BESS}}^{w,i,t}=E_{\mathrm{BESS}}^{w,i,t-1}+{\eta }_{\mathrm{BESS},\mathrm{c}}^i{P}_{\mathrm{BESS},\mathrm{c}}^{w,i,t}-\frac{P_{\mathrm{BESS},\mathrm{d}}^{w,i,t}}{{\eta }_{\mathrm{BESS},\mathrm{d}}^i}:{\lambda }_{\mathrm{SOC}1}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(22)

$$E_{\mathrm{BESS}}^{w,i,t=0}=E_{\mathrm{BESS}}^{w,i,t=24}:{\lambda }_{\mathrm{SOC}2}^{w,i},\forall w\in W,\forall i\in N$$

(23)

where $P_{\mathrm{BESS},\mathrm{c}}^{i,\max }$ and $P_{\mathrm{BESS},\mathrm{d}}^{i,\max }$ represent the maximum charging power and maximum discharging power of BESS in the MG i, respectively. $u_{\mathrm{BESS},\mathrm{c}}^{w,i,t}$ and $u_{\mathrm{BESS},\mathrm{d}}^{w,i,t}$ are binary variables used to restrict the charging and discharging behavior of the BESS from occurring within the same time period. $E_{\mathrm{BESS}}^{i,\max }$ and $E_{\mathrm{BESS}}^{i,\min }$ represent the upper and lower limits of the stored electrical energy of BESS, respectively. ${\eta }_{\mathrm{BESS},\mathrm{c}}^i$ and ${\eta }_{\mathrm{BESS},\mathrm{d}}^i$ denote the BESS charging efficiency and discharging efficiency, respectively. ${\lambda }_{\mathrm{BESS},\mathrm{c}}^{w,i,t}$, ${\lambda }_{\mathrm{BESS},\mathrm{d}}^{w,i,t}$, ${\lambda }_{{\mathrm{SOC}}_{\max }}^{w,i,t}$, ${\lambda }_{\mathrm{SOC}1}^{w,i,t}$ and ${\lambda }_{\mathrm{SOC}2}^{w,i}$ are the Lagrange multipliers of the operating constraints of battery energy storage system.

3.3.4. Micro Gas Turbines Operating Constraints

The micro gas tuirbines operating constraints are shown below:

$$0\le P_{\mathrm{MT}}^{w,i,t}\le P_{\mathrm{MT}}^{i,\max }:{\lambda }_{\mathrm{MT}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(24)

$$P_{\mathrm{MT}}^{w,i,t}=P_{\mathrm{MT}}^{w,i,t-1}+P_{\mathrm{ramping}}^{w,i,t}:{\lambda }_{\mathrm{ramping}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(25)

where $P_{\mathrm{MT}}^{i,\max }$ and $P_{\mathrm{ramping}}^{w,i,t}$ denote the maximum value of gas turbine output power and gas turbine ramping power in the i-th microgrid. ${\lambda }_{\mathrm{MT}}^{w,i,t}$ and ${\lambda }_{\mathrm{ramping}}^{w,i,t}$ are the Lagrange multipliers of MT’s operating constraints.

3.3.5. Constraints on Interruptible Loads

The constraints of interruptible loads are shown below

$$0\le P_{\mathrm{IL}}^{w,i,t}\le {\mu }_{\mathrm{IL}}{P}_{\mathrm{EL}}^{w,i,t}:{\lambda }_{\mathrm{IL}}^{w,i,t},\forall w\in W,\forall i\in N,\forall t\in T$$

(26)

where $P_{\mathrm{IL}}^{w,i,t}$ denotes the interruptible load power. ${\lambda }_{\mathrm{IL}}^{w,i,t}$ is the Lagrange multipliers of interruptible loads constraint.

4. Bi-Level Optimization Model Based on Stackelberg Game and Solution Methodology

4.1. Stackelberg Game

The leader MA and the followers MMGS belong to different interest parties, each of which will show self-interest and optimize their internal operation by formulating electric energy trading strategies according to their own status. The master–slave game problem is a bi-level optimization problem.

For the leader MA, which seeks to maximize daily operating revenue:

$$\max F_{\mathrm{MA}}$$

(27)

For the followers MMGS, which seek to minimize daily operating costs:

$$\min C_{\mathrm{MMGS}}^{}$$

(28)

The expression (27) refers to problems (6)–(11), and expression (28) to problems (12)–(26). The upper layer optimizes the problems (6)–(11) to set the tariff for trading with MMGS with the goal of MA maximizing its own revenue. The lower-level MMGS respond to the tariffs set by the MA as the followers of the master–slave game. The tariffs set by the MA for the slave side affect the tariffs of MMGS, while the tariffs of the follower MMGS also affect the operating revenue of MA. The upper-level optimization problem and the lower-level optimization problem influence and constrain each other until a Nash equilibrium is reached.

4.2. Benefit Allocation Based on Nash Bargaining Theory

On the slave side, the objective function of multiple micro-grids is the lowest total operating cost, and the payment cost/revenue of each MG also needs to be settled. A reasonable benefit allocation mechanism determines the motivation of MG works to collaborate. Nash bargaining solution, as a branch of co-operative game, can effectively deal with the problem of settling the operating costs of each MG. According to the Nash bargaining game theory [6,13,14], the multi-MGs collaborative operation model is as follows:

$$\max {\displaystyle \prod _{i=1}^N{\left[C_0^{w,i}\left({\mathit{x}}_0^{w,i}\right)-C_1^{w,i}\left({\mathit{x}}_1^{w,i}\right)-{\displaystyle \sum _{t\in T}{\displaystyle \sum _{\begin{array}{c}i\in N\\ j\in N\backslash \{i\}\end{array}}{P}_{\mathrm{P}2\mathrm{P}}^{w,ij,t}\cdot {\pi }_{\mathrm{P}2\mathrm{P}}^{ij,t}}}\right]}^{{\epsilon }^{w,i}}}$$

(29)

s.t.

$$C_0^{w,i}\left({\mathit{x}}_0^{w,i}\right)-C_1^{w,i}\left({\mathit{x}}_1^{w,i}\right)-{\displaystyle \sum _{t\in T}{\displaystyle \sum _{\begin{array}{c}i\in N\\ j\in N\backslash \{i\}\end{array}}{P}_{\mathrm{P}2\mathrm{P}}^{w,ij,t}\cdot {\pi }_{\mathrm{P}2\mathrm{P}}^{ij,t}}}\ge 0$$

(30)

$${\epsilon }_{}^{w,i}={\mathrm{e}}^{\frac{E_{\mathrm{supply}}^{w,i}}{E_{\mathrm{supply},\max }^{w,i}}}-{\mathrm{e}}^{-\frac{E_{\mathrm{recieve}}^{w,i}}{E_{\mathrm{recieve},\max }^{w,i}}}$$

(31)

$${\displaystyle \sum _{i=1}^N{\epsilon }_{}^{w,i}}=1,\forall w\in W$$

(32)

$$E_{\mathrm{supply}}^{w,i}={\displaystyle \sum _{t\in T}{\displaystyle \sum _{\begin{array}{c}i\in N\\ j\in N\backslash \{i\}\end{array}}\max \left(0,P_{\mathrm{P}2\mathrm{P}}^{w,ij,t}\cdot \mathsf{\Delta }t\right)}},\forall w\in W$$

(33)

$$E_{\mathrm{recieve}}^{w,i}=-{\displaystyle \sum _{t\in T}{\displaystyle \sum _{\begin{array}{c}i\in N\\ j\in N\backslash \{i\}\end{array}}\min \left(0,P_{\mathrm{P}2\mathrm{P}}^{w,ij,t}\cdot \mathsf{\Delta }t\right)}},\forall w\in W$$

(34)

In (29), $C_0^{w,i}\left({\mathit{x}}_0^{w,i}\right)$ and $C_1^{w,i}\left({\mathit{x}}_1^{w,i}\right)$ denote the optimal operating cost of MMGS in the non-co-operative mode and in co-operative mode, respectively. ${\mathit{x}}_0^{w,i}$ and ${\mathit{x}}_1^{w,i}$ denote the decision variable vector of the non-co-operative mode and in co-operative mode, respectively. In (30), ${\pi }_{\mathrm{P}2\mathrm{P}}^{ij,t}$ represents the trading tariff. ${\epsilon }_{}^{w,i}$ denotes the contribution factor of the P2P energy sharing. The expression (30) is a constraint of Nash bargaining theory, the significance of which indicates that the benefits of all participants in Nash bargaining are enhanced. The expression (31) and (32) are the expressions for calculating the contribution of co-operation and the constraints on the contribution. In (33) and (34), $E_{\mathrm{supply}}^{w,i}$ and $E_{\mathrm{recieve}}^{w,i}$ denote the total electrical energy provided and the total electrical energy received when each MG plays in P2P energy sharing mode, respectively.

The expressions (29)–(34) describe a nonconvex, nonlinear optimization problem. To facilitate the solution, by taking a logarithmic transformation, expression (29) can be equivalently converted to expression (35).

$$-\min {\epsilon }_{}^{w,i}\cdot \ln \left[C_1^{w,i}\left({\mathit{x}}_1^{w,i}\right)+{\displaystyle \sum _{t\in T}{\displaystyle \sum _{\begin{array}{c}i\in N\\ j\in N\backslash \{i\}\end{array}}{P}_{\mathrm{P}2\mathrm{P}}^{w,ij,t}\cdot {\pi }_{\mathrm{P}2\mathrm{P}}^{ij,t}-}}{C}_0^{w,i}\left({\mathit{x}}_0^{w,i}\right)\right]$$

(35)

4.3. Solution Methodology

In the bi-level optimization model, there are bilinear terms in the upper-level model, and the benefit allocation problem based on Nash bargaining theory in the lower-level model belongs to the mixed integer nonlinear programming problem. There is a coupling relationship between the upper-layer and the lower-layer model which is difficult to solve directly. Using a heuristic algorithm to solve the above problem, the global optimal solution usually cannot be found. Therefore, in this paper, by constructing the Lagrangian function of the lower-level model and converting the lower-level model to the constraints of the upper-level model based on the KKT complementary relaxation conditions of the lower-level model, the converted single-level nonlinear model was achieved [12]. The Big-M method was then used to linearize the nonlinear terms in the transformed single-layer nonlinear model to form a single-layer equivalent mixed-integer linear programming problem, as shown in Appendix A.

5. Case Study

In this section, the co-operative operation framework proposed in this paper will be evaluated by comparing simulation experiments. In this paper, the Mosek, a commercial solver, was used to solve the optimization problem in a Matlab environment on a PC with an Intel i5 CPU and with 16 GB RAM.

5.1. Basic Parameters

The energy system studied in this paper consists of an MA and three MGs, where the MA acts as an intermediate service provider and coordinates the energy transactions between the grid and the MGs. Supported by the main grid, each MG acts as a prosumer, consisting of WT, BESS and loads. Under the coordination of the MA, direct energy sharing was available between all MGs to maintain the energy supply and demand balance.

Each MG consisted of a gas turbine, battery energy storage system, wind turbine and other equipment. The technical parameters of the above equipment are shown in

Table 1. Technical parameters of MG.

Item	Parameter	Value
		MG 1	MG 2	MG 3
MT	$P_{\mathrm{MT}}^{i,\max }$	2 MW	1 MW	4 MW
	$P_{\mathrm{ramping}}^{i,\max }$	1 MW	0.5 MW	2 MW
BESS	$E_{\mathrm{BESS}}^{i,\max }$	8 MWh	6 MWh	10 MWh
	$SO{C}_{\mathrm{ini}}$	0.33	0.33	0.33
	$SO{C}_{\min }$	0.2	0.2	0.2
	$SO{C}_{\max }$	0.85	0.85	0.85
	$P_{\mathrm{BESS},\mathrm{c}}^{i,\max }$	2.4 MW	1.8 MW	3 MW
	$P_{\mathrm{BESS},\mathrm{d}}^{i,\max }$	2.4 MW	1.8 MW	3 MW
	${\eta }_{\mathrm{BESS},\mathrm{c}}^i$	0.95	0.95	0.95
	${\eta }_{\mathrm{BESS},\mathrm{d}}^i$	0.95	0.95	0.95
Interruptible load	${\mu }_{\mathrm{IL}}$	0.1	0.15	0.1
Power Interaction	$P_{\mathrm{MG}2\mathrm{MA},\mathrm{b}}^{i,\max }$	10 MW	10 MW	10 MW
	$P_{\mathrm{MG}2\mathrm{MA},\mathrm{s}}^{i,\max }$	10 MW	10 MW	10 MW

, and the electrical load of each microgrid is shown in Figure 2. The Latin hypercube sampling method was used to randomly generate 10,000 random wind speed scenarios, and then the backward reduction technique was used to reduce the generated large number of scenarios, and finally reduce the representative 10 scenarios. In this paper, the above method was used to simulate the uncertainty of wind power generation, and 10 typical scenarios of wind power generation in each microgrid are shown in Figure 3. The price of electricity purchased by MA to the main grid and the price of electricity sold are shown in

Table 2. Time-of-use tariff for microgrid aggregators.

Time Period	Selling Tariff	Purchasing Tariff
Valley (1:00–7:00, 23:00–24:00)	350 CNY/MW	400 CNY/MW
Peak (7:00–10:00, 15:00–18:00, 21:00–23:00)	680 CNY/MW	790 CNY/MW
Flat (1:00–7:00, 23:00–24:00)	1120 CNY/MW	1200 CNY/MW

.

5.2. Impact of Parameter L on Electricity Sales Retail Price Decisions

The trading price between MA and MMGS depends on the MA’s attitude toward the risk of loss. Consequently, it is necessary to study the effect of parameter L on the decision-making behavior of electricity sales retail prices of the MA. In the experimental simulation, the confidence level was set to 0.95 and the value of parameter L varied between 0.01 and 10.

Under this simulation condition, when the MGs did not participate in direct energy P2P sharing, the results of the trading tariff between MA and MGs are shown in Figure 4. When the MGs participated in energy P2P direct sharing, the results of the trading tariff between the MA and MG are shown in Figure 5.

From Figure 4 and Figure 5, it can be seen that when the parameter L increases, MA, in order to reduce the risk loss, will achieve the purpose of suppressing the electricity trading between MGs and MAs by widening the price difference between the purchase and sale of electricity, so as to achieve the effect of reducing the risk loss. Specifically, in Figure 4, MA offers a higher purchase price for MG3 to reduce MG3’s demand for electricity during peak periods (18:00–20:00). As the L value increased from 0.01 to 10, the purchase price from MA increased from 1252.93 CNY/MW to 1336.45 CNY/MW, and from 1252.93 CNY/MW to 1500 CNY/MW for MG1, MG2, and MG3, respectively. Similarly, in order to increase the electricity consumption of MG1 and MG2 during the low electricity consumption hours (21:00–24:00), MG1 and MG2’s electricity selling price to MA gradually reduced. As the L value increased from 0.01 to 10, the selling price of electricity from MG1 and MG2 to MA increased from 380 CNY/MW to 481.32 CNY/MW, and from 380 CNY/MW to 608.15 CNY/MW. In Figure 5, since MGs participating in energy P2P sharing may negotiate with each other to respond to the trading tariff specified by the MA, the trading price difference specified by MA for MGs varied insignificantly with L when MGs participated in P2P energy sharing relative to Figure 4.

It can be seen in Figure 4 and Figure 5 that in some periods the tariffs for MGs can be higher than the tariffs of the grid for the MA. The increase or decrease of the microgrid operator’s cost in different time periods is essentially a game behavior with MA. Although the cost of purchasing electricity from microgrids is not optimal in some time periods, the advantage of the proposed model can be reflected by evaluating the set of tariffs through the total transaction cost of the day. At the same time, the regulator can over-penalize the consumers and favor the emergence of aggregator behavior by setting an average tariff constraint as shown in expressions (5) and (6).

5.3. Impact of Parameter L on the Operating State of MMGS

The sum of power purchased and the sum of power sold by MMGS and MA when the value of L varied from 0.01 to 10 is shown in

Table 3. Electric energy trading quantity of MGs and MA with different risk aversion factors.

Risk Aversion Factor	P2P Energy Sharing Mode between MGs	Total Power Purchased by MGs from MA (MW)	Total Power Sold by MGs from MA (MW)
L = 0.01	Yes	54.01	369.76
	NO	139.64	406.09
L = 0.1	Yes	54.19	380.14
	NO	132.26	405.77
L = 1	Yes	53.43	370.63
	NO	127.71	404.86
L = 10	Yes	50.23	350.32
	NO	120.24	398.21

.

Analyzing Table 3, it can be seen that when MGs participate in energy sharing, MGs will meet their daily load demand by interacting with neighboring MGs for power, which in turn reduces energy transactions with retailers. In other words, the energy demand of MGs that do not participate in energy sharing with retailers is greater than the energy demand of participating MGs, because MGs participating in energy sharing can obtain energy from neighboring MGs. As L increased from 0.01 to 10, in response to risk aversion, the MG operator arranged battery storage system, gas turbine work, interruptible load work, and power interaction with the adjacent MGs to maintain power supply and demand balance, instead of power purchase and sale processing with the MA.

By further analysis of Table 3, it is obvious that MGs try to minimize the total operating cost by interacting with neighboring MGs to meet the daily load demand, thus reducing the energy trade with MA, and the P2P energy sharing in which MGs participate can promote the local consumption of distributed energy, reduce the amount of electricity purchased during peak periods and the amount of electricity sold during low periods, thus reducing the peak and filling the valley. This will reduce the pressure on the power supply of the grid.

Table 4. Microgrid operating costs with different risk aversion factors (negative sign means benefits).

Risk Aversion Factor	Items	MG1	MG2	MG3	Total Cost
L = 0.01	Total cost not in P2P mode (CYN)	−18,157.02	−14,762.25	70,578.95	37,659.68
	O&M cost in P2P mode (CYN)	−10,218.26	−10,878.88	41,092.57	19,995.43
	Transfer payments of Nash bargaining (CYN)	−13,708.70	−7143.23	20,851.92	0
	Total cost in P2P mode (CYN)	−23,926.96	−18,022.10	61,944.50	19,995.43
	Benefit (CYN)	−5769.94	−3259.85	−8634.45	−17,664.25
	Shared Power (MWh)	−10.84	−7.69	18.52	0
L = 0.1	Total cost not in P2P mode (CYN)	−18,132.40	−14,747.67	69,875.03	36,994.96
	O&M cost in P2P mode (CYN)	−10,634.68	−10,247.81	40,714.18	19,831.69
	Transfer payments of Nash bargaining (CYN)	−12,629.04	−8187.94	20,816.98	0
	Total cost in P2P mode (CYN)	−23,263.73	−18,435.74	61,531.16	19,831.69
	Benefit (CYN)	−5131.33	−3688.07	−8343.87	−17,163.27
	Shared Power (MWh)	−10.63	−8.80	19.43	0
L = 1	Total cost not in P2P mode (CYN)	−18,151.99	−14,773.03	69,746.12	36,821.10
	O&M cost in P2P mode (CYN)	−10,955.28	−9523.04	40,162.93	19,684.61
	Transfer payments of Nash bargaining (CYN)	−11,621.84	−10,471.39	22,093.23	0
	Total cost in P2P mode (CYN)	−22,577.12	−19,994.42	62,256.16	19,684.61
	Benefit (CYN)	−4425.13	−5221.39	−7489.96	−17,136.49
	Shared Power (MWh)	−10.17	−8.59	18.76	0
L = 10	Total cost not in P2P mode (CYN)	−18,145.22	−16,360.33	69,843.54	35,337.99
	O&M cost in P2P mode (CYN)	−11,259.75	−9620.55	40,005.04	19,124.74
	Transfer payments of Nash bargaining (CYN)	−11,740.75	−9334.74	21,075.49	0
	Total cost in P2P mode (CYN)	−23,000.50	−18,955.29	61,080.53	19,124.74
	Benefit (CYN)	−4855.28	−2594.96	−8763.01	−16,213.25
	Shared Power (MWh)	−11.75	−6.32	18.06	0

shows the operating cost of MMGS and its payment when MGs do not participate in P2P energy sharing, and when MGs participate in P2P energy sharing. It can be observed that after participating in P2P energy sharing, the operating costs of MG1 and MG2 increase, but the operating costs of MG3 decrease. The reason for this is that MG1 and MG2 preferentially transmit more idle power to MGs instead of selling it to MA. MG3 also gives some financial compensation to MG1 and MG2, and the compensation cost is proportional to the energy it receives. As can be seen from Table 3, the comparison of the operation costs of MGs in P2P mode (L = 0.01) reduced the total cost from CNY 37,659.68 to CNY 19,995.43, and achieved 46.9% cost saving. With the increase of L from 0.01 to 10, the total costs not in or in P2P mode reduced from CNY 35,337.99 (not in P2P mode) to CNY 19,124.74 (in P2P mode) when L = 10. Indeed, the “Transfer payment” is not enough to cope with the increasing costs for MG1 and MG2. However, the MG3’s running costs were significantly reduced, and the total running costs of the MMGS were also reduced. This shows the advantage of MMGS using P2P energy sharing operation mode.

By considering the contribution of each MG in the energy-sharing process, the fairness of benefit distribution can be ensured, and the distributed energy can be better consumed locally by MGs through P2P energy sharing, which can significantly improve energy utilization efficiency.

Another noteworthy point is that with the increase of the parameter L, MAs will suppress the electric trading between MGs and MAs by raising the purchase price and lowering the sale price in order to reduce the risk loss. MGs will inevitably increase the behavior of P2P energy sharing in order to achieve the purpose of reducing the operation cost. The distribution of fairness written in the manuscript does not mean absolute fairness; here it means that the higher contribution of co-operation receives a higher benefit. The co-operative contribution of each MG in the P2P energy sharing process can be obtained by the calculation of expression (31).

6. Conclusions

In this paper, risk assessment and game theory were applied to a bi-level optimal dispatch model of the microgrid aggregators and multi-microgrids system. The Karush–Kuhn–Tucker conditions, Big-M method, and strong duality theory were employed to transform the bi-level nonlinear model of energy management into a single-level mixed integer linear programming model. The effectiveness of the proposed model and solution method was verified by combining specific simulation cases and comparative analysis. The following conclusions can be drawn:

• The energy management framework based on the master–slave game proposed in this paper can take into account the mutual influence of MAs and each MG’s decision, which is in accordance with the actual interests of multiple subjects. The proposed bi-level optimization model in this paper can effectively improve the ability of the energy management system to cope with risks and improve the ability of the expected benefits of MAs under different risk levels.
• In the energy management framework proposed in this paper, the P2P energy sharing mode among MGs has the advantage of reducing the cost of electricity. In the model example of this paper, the cost of electricity consumption of MMGS was about 23% lower than the cost in the non-co-operative mode with complete competition among communities, thus achieving win–win co-operation.
• In the energy management framework proposed in this paper, the MA can enhance its ability to cope with risks to improve the expected revenue under different risk aversion levels.

The limitations of the model in this paper are that the uncertainty on the load side was not considered and the energy management strategy obtained in this paper does not take into account more resources on the demand side. In future work, factors such as multiple time scales and more demand-side flexibility resources will also be taken into account, and case studies will be conducted on the IEEE 34 system.