Energy Trading and Pricing in Microgrids with Uncertain Energy Supply : A Three-Stage Hierarchical Game Approach

This paper studies an energy trading and pricing problem for microgrids with uncertain energy supply. The energy provider with the renewable energy (RE) generation (wind power) determines the energy purchase from the electricity markets and the pricing strategy for consumers to maximize its profit, and then the consumers determine their energy demands to maximize their payoffs. The hierarchical game is established between the energy provider and the consumers. The energy provider is the leader and the consumers are the followers in the hierarchical game. We consider two types of consumers according to their response to the price, i.e., the price-taking consumers and the price-anticipating consumers. We derive the equilibrium point of the hierarchical game through the backward induction method. Comparing the two types of consumers, we study the influence of the types of consumers on the equilibrium point. In particular, the uncertainty of the energy supply from the energy provider is considered. Simulation results show that the energy provider can obtain more profit using the proposed decision-making scheme.


Introduction
In the microgrid, energy trading is an important segment [1,2].The energy provider determines the energy purchase to meet the consumer demands.Meanwhile, in order to increase its profit, the energy provider faces a problem of pricing decision.With the development of renewable energy (RE), it is reasonable for the energy provider to use the renewable energy as supply [3,4].Due to the introduction of the renewable energy, the energy provider has to predict the generating capacity of the renewable energy system, and then decides how much energy it needs to purchase from the electricity markets.The energy provider's prediction can have a deviation from the actual generation, which leads to the uncertainty (UC) of the energy supply [5][6][7].
There are some works in literature related to the interactions among the consumers.A non-cooperative game (NCG) was formulated among the consumers in [8][9][10][11][12][13][14].In [9], the price-taking consumers and the price-anticipating consumers were considered.The price-taking consumers assume that their energy consumption cannot affect the electricity price, whereas the price-anticipating consumers believe that their energy consumption can change the electricity price.Recently, the Stackelberg game (SG) is formulated between the energy provider and the consumers in [15][16][17][18][19][20].In addition, the authors in [20,21] took into account a two-stage Stackelberg game between the power station and the consumers.In order to reduce the cost of the energy purchased from the electricity markets, the renewable energy was taken into account in [22].Most of these works mainly focus on the price-taking consumers, and they seldom take into account the renewable energy generation, thereby the uncertainty of the energy supply is not involved.The differences of the proposed work with the above literature are shown in Table 1.
Table 1.Differences of the proposed work with the literature.
The rest of the paper is organized as follows.Section 2 introduces the problem formulation.Section 3 shows the backward induction method of the three-stage hierarchical game for the energy provider and the price-taking consumers.In Section 4, the backward induction method of the three-stage game for the energy provider and the price-anticipating consumers is described.Section 5 gives the simulation and comparison results.Finally, the conclusions are summarized in Section 6.

Problem Formulation
We consider the energy trading and pricing problem in the microgrid consisting of one energy provider and a set N = {1, ..., N} of consumers.The energy provider and the consumers are integrated into a microgrid with renewable energy generation.The energy provider purchases energy from the electricity markets when the renewable energy supply is not enough.In that case, the energy supply includes the energy generated from the renewable energy source and the energy purchased from the electricity markets.In the microgrid, the system structure of the energy trading is given in Figure 1.Because of the uncertainty of the energy generated from renewable energy sources, the energy purchased from the electricity markets is uncertain.According to the interaction between the energy provider and the consumers, we establish a hierarchical game as below.
• Leader: the energy provider determines the energy purchase and the pricing strategy to maximize its profit.• Followers: the consumers determine the energy demands to maximize their payoffs.
According to the types of the consumers, we consider two scenarios in this paper.In scenario A, there is no competition among the price-taking consumers, i.e., the consumers' energy consumption cannot affect the price announced by the energy provider.In scenario B, the interactions among the price-anticipating consumers are formulated into a non-cooperative game, i.e., the consumers' energy consumption can change the price announced by the energy provider [9].

Wind Power Generation Model
There exists a large body of literature on wind power forecasting, and the day-ahead wind forecast based on numerical weather prediction (NWP) models can enable relatively accurate wind forecasts [25,26].Because the operating time moves closer to the near term, the computation complexity often renders NWP models intractable at a high spatial resolution [26].An adaptive wavelet neural network was proposed for mapping the NWP's wind speed and wind direction forecasts to wind power forecasts in [27].The authors in [28] proposed a novel statistical wind power forecast framework, which leverages the spatio-temporal correlation in wind speed and direction data among geographically dispersed wind farms.In [29], the author developed a feed-forward neural network approach for wind power generation forecasting to improve the wind forecasting accuracy.However, the wind power forecast is relatively complex, and the forecast errors cannot be avoided.Generally, the wind speed can be approximated as the Gamma distribution [30], inverse Gaussian [31], log-normal [32], and Weibull [33][34][35][36].Alternatively, copula theory has recently been applied to wind speed and wind power as a way of modeling nonlinear dependence structures [37].According to the wind speed, we establish the wind power generation model adopting the mixed copula function in this paper.Firstly, we introduce the copula theory.
The copula theory was proposed by Sklar in 1959 [38].Supposing that , then there is a copula function C satisfies [39]: Defining that , respectively.Therefore, the copula function C can be obtained as the following: where the marginal distributions of When N = 2, the copula function C is a binary function.H(x 1 , x 2 ) is a joint distribution function whose marginal distributions are F(x 1 ) and W(x 2 ), and F −1 (u) and W −1 (v) are the pseudo inverse function of F(x 1 ) and W(x 2 ), respectively.Therefore, the copula function C is expressed as the following: The mixed copula function was further proposed in [41,42]: where , and C 3 (u, v, γ 3 ), respectively, and satisfy , and γ 3 are correlation coefficients and can measure the correlation degree of variables.The results of [41] showed that the relevant structures of the mixed copula function are more flexible than a single copula function.The wind power generation model is established by the mixed copula function (see details in [38]).

Scenario A: The Three-Stage Game for Price-Taking Consumers
In this section, we prove the existence and uniqueness of the hierarchical equilibrium by using the backward induction method.First, we analyze the consumers' demands given the energy provider's pricing strategy and energy purchase.Then, we study the energy provider's pricing strategy given the consumers' energy demands and the energy purchase.Finally, we analyze the energy purchased from the electricity markets in the case of uncertain renewable generation, and then obtain the maximum profit of the energy provider.

Consumer's Energy Demands in Stage III
In Stage I, the energy provider needs to determine the energy purchased from the electricity markets.In Stage II, the energy provider announces the price to the consumers.In Stage III, the consumers determine their energy demands given the unit price p announced by the energy provider in Stage II.The payoff of consumer i is defined as the difference between the satisfaction level and the payment for energy purchase, i.e., where h is the consumers' cost coefficient, c i is the actual energy demand of the consumer i, and c i is the energy demand of the consumer i to maintain normal operation of appliances.
The consumers determine their demands to maximize their payoffs.p is a fixed price announced by the energy provider.The first derivative of u i (p, c i ) with respect to c i is: Letting ∂u i (p, c i )/∂c i = 0, we obtain: From Equation (7), we can obtain the energy demands of the consumer i: Adding Equation ( 8) from 1 to N, we obtain the total energy consumption of all consumers: We assume that Q = ∑ i∈N c i for convenience.When all consumers' total demands are ∑ i∈N c * i (p), the payoffs of the consumers are at a maximum.Next, we consider how the energy provider makes the purchase strategy and pricing strategy in Stages I and II based on the total energy demands, respectively.In particular, we show that the energy provider can determine a price in Stage II such that the total energy demands (as a function of price) cannot exceed the total energy supply.

Optimal Pricing Strategy in Stage II
In Stage II, the energy provider determines the pricing strategy according to the consumers' energy demands, given the energy purchase in Stage I.The profit of the energy provider is denoted by: which is the difference between the revenue and the total cost.We assume that p s is the summation of the energy generated from the renewable energy source and the energy purchased from the electricity markets.β 0 p s indicates the energy purchase, β is the uncertainty factor of the renewable energy source, βp s is the wind power generation, and µ is the energy provider's cost coefficient.In this paper, the energy provider's cost comes from the energy purchase and the wind power generation.p w is the cost coefficient and p w βp s is the wind power generating cost.Equation ( 10) denotes the fact that the revenue of the energy provider is determined by the consumers' demands subject to its available supply.In Stage II, the objective is to find the optimal price p that maximizes the energy provider's profit, i.e., where W I I (p s , β) denotes the maximum profit of the energy provider in Stage II.Since the energy supply p s from the energy provider is given in this stage, the total cost µβ 0 p s is already fixed.Therefore, the maximum revenue can be achieved by optimizing the price: Let us define the consumers' energy demands D(p) = p ∑ i∈N c * i and the energy supply S(p) = pβp s .Then, we have: From the above equations, we observe that D(p) is a quadratic function, and S(p) is a linear function.Thus, we can obtain the maximum point of D(p) at p d = Qh/N.The relationships between S(p) and D(p) are described in Figure 2. Letting D(p) = S(p), we obtain a quadratic function with respect to p and make it equal to zero: Solving the above Equation (15), we obtain the intersection point of D(p) and S(p): In the excessive supply regime, the maximum profit of the energy provider is at p = p d : In the conservative supply regime, the maximum profit of the energy provider is at p = p h : The optimal pricing decision and the corresponding optimal profit at Stage II are given in Table 2

Energy Supply Strategy in Stage I
In Stage I, the energy provider determines the energy purchase to maximize its profit by taking into account the uncertainty factor of the energy supply β [15].The profit of the energy provider in the Stage I is given as follows: where W I I (p s , β) is the energy provider's profit functions with respect to p s and the uncertain factor β obtained in Stage II.We assume that the wind power generation P = βp s , and the minimum power and maximum power of the wind power generation are P min and P max , respectively.The probability density function of the wind power f WP (P) can be obtained in [38].From Figure 2, we can obtain that the maximum consumers' demands ∑ i∈N c * i (p) is Q/2 when the price p is Qh/N.Thus, we consider the following two intervals: (1) Interval I: p s ∈ [0, Q  2 ].In this interval, the energy provider's profit function is: (2) Interval II: The energy provider's profit function is: By comparing Interval I with Interval II, we can obtain the maximum profit of the energy provider and the optimal energy purchase in scenario A.

Scenario B: The Three-Stage Game for Price-Anticipating Consumers
Since the price is set by the energy provider based on the total energy consumption, the consumers are interactive with each other.Thus, we formulate a non-cooperative game among the consumers.The non-cooperative game has a unique Nash equilibrium if p(c) is a linear rotational symmetric function, and p(c) is formulated as follows [8]: where ω is a positive parameter to implement the elastic pricing, c i is the actual energy demands of the consumer i, and p 0 is a basic price.

Consumer's Energy Demands in Stage III
In Stage I and Stage II, the energy provider determines the energy purchased from the electricity markets and the pricing strategy for the consumers, respectively.In Stage III, similar to Equation (5), we formulate the payoff of price-anticipating consumer i given the unit price p(c) announced by the energy provider as follows: where h and c i were defined in Equation ( 5), and b i is a base value of the satisfaction level of consumer i and is different for each consumer, which reflects the flexibility of the consumers.The first derivative of u i (c) with respect to c i is: Letting ∂u i (c)/∂c i = 0, we obtain: Adding Equation ( 25) from 1 to N, we have: from which we obtain the total energy consumption of all consumers: To simplify the calculation process, we make: and then we have:

Optimal Pricing Strategy in Stage II
In Stage II, the energy provider determines the pricing strategy according to the consumers' energy demands, given the energy purchase in Stage I.The profit of the energy provider is: and the maximum profit of the energy provider is: where W I I (p s , β) denotes the maximum profit of the energy provider in Stage II.We can maximize the revenue of the energy provider by optimizing the price: Let us define the consumers' total energy demands D(ω) = p(c) ∑ i∈N c * i and the energy supply S(ω) = p(c)(β + β 0 )p s .Then, and the intersection point of D(ω), and the y-axis is p 0 G/2h.
Letting ∂D(ω)/∂ω = 0, we obtain: and and the intersection point of S(ω), and the y-axis is p 0 (β + β 0 )p s .The first derivative of S(ω) with respect to ω is: (39) Since ∑ i∈N c * i > 0 and ∂S(ω)/∂ω > 0, S(ω) is an increasing function.The relationships between S(ω) and D(ω) are described in Figure 3. (a) When G − p 0 (N + 1) < 0, we can obtain the following conclusions from Figure 3a: where ω p is the intersection point of D(ω) and S(ω), and ω * is the optimal parameter of the elastic price.
Because p(c) is a linear rotational symmetric function, the case with ω * = 0 is neglected.
Letting D(ω) = S(ω), we obtain a quadratic function with respect to ω and make it equal to zero: For convenience, we define: Solving the above Equation ( 40), we obtain the intersection point of D(ω) and S(ω): In the excessive supply regime, the maximum profit of the energy provider is at ω = ω 0 : In the conservative supply regime, the maximum profit of the energy provider is at ω = ω p : The optimal pricing decision and the corresponding optimal profit in Stage II are given in Table 3.

Energy Supply Strategy in Stage I
In Stage I, the energy provider also determines the energy purchase to maximize its profit by taking into account the uncertainty of the energy supply.The profit of the energy provider in the Stage I is given by: where W I I (p s , β) is the energy provider's profit function with respect to p s and the uncertain factor β obtained in Stage II.We assume that the wind power generation P = βp s , and the minimum power and maximum power of the wind power generation are P min and P max , respectively.The probability density function of the wind power f WP (P) can be obtained in [38].From Figure 3, we can obtain that the maximum consumers' demands ∑ i∈N c * i (ω) is A/4h(N + 1) when ω = ω 0 .For convenience, we assume that L = A/4h(N + 1) and consider the following two intervals: (1) Interval I: In this interval, the energy provider's profit function is: Similar to scenario A, we can obtain the maximum profit of the energy provider and the optimal amount of energy purchased from the electricity markets.

Simulation Results
This section presents simulation studies of the proposed scheme using MATLAB 7.11.0(MathWorks, Natick, MA, USA).In the simulations, we assume that the wind power generation follows a uniform distribution in [P min , P max ], and we select that h = 0.04, p 0 = 0.1, N = 100, and µ = 10.For the parameter c i , we select a set of stochastic values.Then, we can obtain the profit of the energy provider under different β 0 for two scenarios as shown in Figures 4 and 5, respectively.
From Figures 4 and 5, we observe that the optimal energy supply decreases with the increase of the β 0 and the maximum profit is changed from Interval II to Interval I.In general, the wind power generation cost is less than the cost of purchasing energy.From the profit function of the energy provider, when β 0 increases, only by decreasing p s can the profit of the energy provider be maximized.Thus, it is verified that the proposed method is effective, and the simulation values of Figures 4 and 5 are shown in Table 4.  Taking β 0 = 0.1 as an example, the comparisons between the two scenarios are shown in Figure 6.From Figure 6 and Table 4, we observe that the energy provider can obtain more profit in scenario A. To explain the effect of the uncertainty, taking β 0 = 0.6 under scenario A as an example, we show the profit of the energy provider under the certain and uncertain energy supply in Figure 7.It is observed that the energy provider can achieve the higher profit under the certain energy supply.In reality, the uncertainty of the energy supply is necessary because the energy generated from the renewable energy sources is uncertain.

Conclusions
In this paper, we establish a model for energy trading and pricing in the microgrid.We formulate a hierarchical game between the energy provider with the renewable energy generation and the consumers, e.g., the price-taking consumers and the price-anticipating consumers.In the hierarchical game, the energy provider acts as the leader and the consumers act as the followers.The equilibrium point of the hierarchical game is obtained through the backward induction method.Furthermore, we also consider the uncertainty of the energy supply in the problem formulation.The simulation results show that the optimal energy supply can be obtained based on the reasonable pricing strategy and purchase strategy.Comparing the price-taking consumers with the price-anticipating consumers, we can obtain that the energy provider obtains more profit from the price-taking consumers.From the simulation results, we also can obtain that the energy provider's profit reduces because of the uncertainty of the energy supply.
However, we do not consider that the consumers can sell the energy to the energy provider when the consumers have photovoltaic (PV) panels and a storage system.In that case, the energy demands of the consumers will be uncertain, and the payoff of the consumer includes two additional parts: one part is the PV generation cost, and the other part is the uncertainty of the energy demands.In order to compute the payoff of the consumer, we need to know the distribution that the PV generation follows.Then, we can get the average payoff of the consumer by expectation, and the optimal energy demands are obtained by the derivation method.Simultaneously, the profit of the energy provider needs to introduce two additional parts that denote buying the energy from the consumers and selling the energy to the electricity markets, which will be considered in the future.

Figure 1 .
Figure 1.The system structure of energy trading.

Figure 2 .
Figure 2. The relationships between S(p) and D(p).

Figure 3 .
Figure 3.The relationships between S(ω) and D(ω) under the different conditions.

Figure 4 .Figure 5 .
Figure 4. Scenario A: the profit of the energy provider under different β 0 .

Figure 6 .
Figure 6.Comparisons between Scenario A with Scenario B.

Figure 7 .
Figure 7.The effect of the uncertainty of the energy supply.

Table 2 .
Optimal pricing decision and profit in Stage II in scenario A.
* (p s ,β) W II (p s ,β) Excessive Supply Regime: p s ≥ Q 2 p ES = p d W ES I I (p s , β) in Equation (17) Conservative Supply Regime: p s < Q 2 p CS = p h W CS I I (p s , β) in Equation (18)

Table 3 .
Optimal pricing decision and profit in Stage II in scenario B.

Table 4 .
Simulation values of the two scenarios.