Hierarchical Optimization Method for Energy Scheduling of Multiple Microgrids

This paper proposes a hierarchical optimization method for the energy scheduling of multiple microgrids (MMGs) in the distribution network of power grids. An energy market operator (EMO) is constructed to regulate energy storage systems (ESSs) and load demands in MMGs. The optimization process is divided into two stages. In the first stage, each MG optimizes the scheduling of its own ESS within a rolling horizon control framework based on a long-term forecast of the local photovoltaic (PV) output, the local load demand and the price sent by the EMO. In the second stage, the EMO establishes an internal price incentive mechanism to maximize its own profits based on the load demand of each MG. The optimization problems in these two stages are solved using mixed integer programming (MIP) and Stackelberg game theory, respectively. Simulation results verified the effectiveness of the proposed method in terms of the promotion of energy trading and improvement of economic benefits of MMGs.


Introduction
With the increasing penetration of renewable energy resources (RESs) in the distribution networks of regional power grids, intermittent RESs, which are connected to distribution networks in a distributed way with a small capacity and high density, have great influence on the stability of regional power grids.The energy management and control of distributed generation, load demand and energy storage systems (ESSs) using microgrid (MG) technology can effectively improve the stability of regional power grids [1][2][3].MGs with different characteristics coexist in distribution networks of regional power grids and form multiple microgrids (MMGs).MGs with different characteristics can achieve energy interaction through energy management systems (EMSs), which not only can enhance the reliability of the regional power supply [4], but also promote the utilization of renewable energy, and improve the economic benefits of MGs [5].
In order to manage the energy scheduling between different MGs, centralized coordination control is commonly adopted in MMG architecture [6][7][8].In [9], the energy market operator (EMO) acts as a centralized control system to coordinate the energy iteration between a cluster of selling MGs and a cluster of buying MGs.A two-stage robust optimization for energy transactions in MMGs is proposed in [10], which could minimize system cost under the worst realization of uncertain PV output.In [11], a multiple agent system (MAS)-based hierarchical energy management strategy for MMGs is proposed with easy implementation and low computation cost.A practical model is proposed for distribution companies to minimize the total operation cost of the system including distribution networks and MGs through coordinated operation [12].However, the above optimization methods are all aimed at a situation in which there is no direct conflict of interest between MGs and upper manager.And these methods are not very suitable for competitive hierarchical power market structures [13].
To stimulate the energy transaction potential of MGs, a multi-market participation framework is proposed for distribution network operators (DNOs) [14].Since market participants belong to different stakeholders, DNOs and MGs can trade energy not only through cooperation, but also through competition.The relationship among participants is suitable to be solved by game theory.In [15], an incentive mechanism based on cooperative game theory is proposed to reduce the peak ramp of distribution networks and improve the benefits of MGs.In [16], a two-level day-ahead scheduling structure based on Stackelberg game is proposed to stimulate MGs participating in power sale bidding, which could reduce the total cost of the DNO.A bi-level programming based on Stackelberg is adopted by the DNO to improve the benefits of MGs while the cost of the DNO reaches the minimum [17].However, these methods are based on day-head energy market, and there are still some shortcomings in dealing with the uncertainties of renewable energy generation and load demand.
In this paper, a two-level optimization method is proposed for hour-ahead MMG energy scheduling in distribution network energy markets, where the EMO is the upper manager of the whole MMG system, and the MGO is lower manager of the local MG.This method can be implemented in two stages.According to the short-term forecast information, the local MGO optimizes energy scheduling of storage units by adopting rolling optimization in the first stage.The energy transactions between the EMO and MGs are optimized in the second stage, which is modeled as a Stackelberg game.The EMO is the leader of the game who determines the prices of the next hour to maximize its own utility, while the MGs are followers who respond to prices by adjusting local load demands.Through two stages of optimization, the economic utilities of the EMO and MGOs are both improved.The other significant feature of this proposed method is that energy storage and load demands are dispatched hourly, which makes it more reasonable to manage energy trading in MMGs under the uncertainties of PV output and load demands.

Structure of an MMG
The structure of an MMG in this paper is shown in Figure 1.The parties involved in energy trading in the MMG are MGs, the EMO and a power grid.Each MG mainly consists of a PV system, an ESS, loads, a smart meter and a local EMS.For a rational MG, the first choice of PV generation is to supply its own loads and charge its own ESS.The MG can then act as a seller when there is an energy surplus or as a buyer when there is an energy deficit.The net energy profile of the MG is optimized by the local EMS according to the benefits generated by energy consumption and the internal prices from the EMO.The EMO is responsible for stimulating energy trading in MMGs by establishing a reasonable internal buying price (p h cb ) and selling price (p h cs ) each hour.At the occurrence of an internal energy mismatch, the EMO trades with power grids to balance supply and demand.In order to guarantee that the profits of the MG obtained from energy sharing among MGs are better than those of the MG obtained from energy trading with power grids directly, the internal prices produced by the EMO should be between the selling price (p h gs ) and the buying price (p h gb ) of power girds, allowing the EMO to maximize its own profits under the constraint as follows:

Operation Strategy
Figure 2 shows the details of the operation strategy of an MMG, which can be described by two stages of optimization.

Forecast information of PV output and energy demand
Forecast prices

EMO MGO
The collection of local forecast information

Role of energy trading
Broadcast of internal prices

Operation Strategy
Figure 2 shows the details of the operation strategy of an MMG, which can be described by two stages of optimization.

Operation Strategy
Figure 2 shows the details of the operation strategy of an MMG, which can be described by two stages of optimization.

Forecast information of PV output and energy demand
Forecast prices

EMO MGO
The collection of local forecast information

Role of energy trading
Broadcast of internal prices

First Stage
Considering the time-coupling characteristics of a local ESS, a local EMS adopts a rolling optimization method to determine the scheduling plan of the ESS for next period, which can minimize the cost for the MG to purchase energy from the EMO and determine the role of the MG to participate in energy trading in the next period.The inputs of the rolling optimization are the long-term forecast information of local PV output, local energy demand and the price obtained from the EMO.

Second Stage
According to the short term forecast information collected from MGs and the prices of power grids, the EMO establishes the internal price optimization model for next period based on Stackelberg game theory.The optimization target of the EMO is to maximize its own profits while considering the demand responses of MGs.Furthermore, the EMO broadcasts the results of the internal prices to the MGs, which respond to the prices by adjusting load demands.

Utility Model of MGs
Each MG equipped with a PV system receives a government subsidy for clean energy plus the revenue from selling surplus energy to MMGs.The MG prefers the PV generation to satisfy its own load demands and charge its own ESS.If the PV generation is insufficient, the MG will buy energy form the MMG.The energy consumption in the MG can create revenue, especially for industrial and commercial users [18,19].Therefore, the utility model of the MG mainly considers the government subsidy for PV generation, the benefits of energy consumption, the benefits from selling surplus energy to the EMO and the costs of purchasing energy from the EMO.As the MG may be either a buyer or a seller in different periods, the utility function in time slot h can be expressed as follows: where θ is the subsidy for each kWh generated by the PV system, pv h i is the PV generation in the time slot h.k h i ln(1 + l h i ) is the benefit that the MG i consumes energy l h i .p h cb and p h cs are the buying price and selling price from the MG to the MMG, respectively.nl h s,i and nl h b,i respectively represent the selling energy and buying energy of the MG i in time slot h.c i is the degradation cost coefficient of the ESS in the MG i, and e h i is the charging energy to the ESS in time slot h.The other maintenance and scheduling costs are neglected in this paper.Since each MG in the MMG can only act as a buyer or seller at a special time slot, and the net energy should satisfy the following constraints: Equation ( 3) is the constraint of energy balance in the MG i. nl max i is the maximum energy transaction of the MG with the power grid in the connection line, D h b,i and D h s,i are the binary variables indicating the state of buying and selling energy of the MG i.

Profit Model of EMOs
The MG may be either a seller or a buyer depending on its requirement of net energy.The MMG energy importing from the MGs or exporting to the MGs in time slot h can be expressed by Usually, the mismatch between imported energy and exported energy always exists, and the EMO should trade with the power grid to maintain an internal energy balance.Therefore, the profit function of the EMO can be written as

Rolling Optimization for Local MGs
The scheduling of the ESS is optimized based on the long term forecast information on PV generation, load demands and internal prices, which can be expressed as pv cb and p h, f cs , respectively.The primary target of a local MG is to maximize the operation utility.According to (2), the target of the rolling optimization is equivalent to minimize the cost of energy trading with the EMO, which can be expressed as where k is the length of rolling optimization, and ∆h is the rolling step.In addition, the objective function in (10) needs to satisfy not only the constraints in (3)-( 6) but also the constraints of battery charging and discharging, which can be expressed as follows.
0 ≤ e h ch,i ≤ D h ch,i e max ch,i , 0 ≤ e h dis,i ≤ D h dis,i e max dis,i , where e h ch,i and e h dis,i are the charging and discharging energy of the ESS, respectively.D h ch,i and D h dis,i indicate the charging and discharging state of the ESS, e max ch,i and e max dis,i are the maximum charging and discharging energy in time slot h, respectively.Equation (15) shows the state of charge (SOC) of the ESS during its charging and discharging at the end of the time slot h, η ch,i and η dis,i are the charging and discharging efficiency of the ESS and SOC min i and SOC max i are the lower and upper limits of the SOC.Therefore, the rolling optimal scheduling in (10) can be modeled as a mixed integer programming (MIP) problem.

Stackelberg Game for EMOs
The optimal scheduling of the ESS in the first stage is regarded as one of the inputs of the EMO in second stage.The other inputs include the short term forecast information on PV generation, schedulable and unscheduled loads and the trading roles of the MG in the following hour.After receiving these inputs, the EMO will determine the internal prices and send to local EMSs for energy consumption optimization in the following hour.

Formulation of a Stackelberg Game
The hour-ahead energy sharing within the MMG is formulated as a Stackelberg game.The EMO is the leader of the game, and stimulates energy sharing by setting the internal prices with the goal of maximizing profits, while the MGs act as followers that optimize their utility through properly responding to internal prices.The game between the EMO and the MGs can be defined by its strategic form as where is the set of load strategies adopted by each MG i in the time slot h constrained by where . When all the players in (MGO ∪ {EMO}) reach the SE, the EMO cannot improve its profit by adjusting the internal prices from the SE prices P h cb * and P h cs * .Likewise, no MGs can enhance their utilities by selecting different strategies from L h i * .

Achievement of Game Equilibrium
It is known from (2) and (3) that the utility of MGs can be modified as For the given internal prices p h cb and p h cs , the optimal energy consumption l ho i can be easily obtained by making ∂U h i /∂l h i = 0, which leads to Equation ( 22) can be substituted into ( 7)-( 9) to solve the optimal internal price of the EMO based on the role of energy sharing that each MG wants to play.The following section provides a discussion of the relationship between the optimal prices and the roles of MGs.
For each MG acting as a buyer, the net energy nl h i satisfies l h i + e h i − pv h i ≥ 0. The range of energy consumption can be redefined as Substituting the optimal value in (22) into the energy consumption in (23), we can obtain where p h cs,i is the flexible selling price that the MG i as a buyer expects to trade with the EMO.If p h cs < k h i /(l max i + 1), the optimal energy consumption will be l max , the optimal energy consumption will be max(l h,min i , pv h i − e h i ).Similarly, the flexible buying price p h cb,i of a seller is constrained by , the optimal energy consumption will be min(pv , the optimal energy consumption will be max(l h,min i , pv h i − e h i ).As a result, we can simplify (24) and (25) into Equation ( 26) expresses the feasible region of the optimal price for a seller and equation ( 27) expresses the feasible region of the optimal price for a buyer.Therefore, the profit functions of the EMO in ( 7)-( 9) can be updated with the optimal energy consumption as In order to optimize the objective function in (30), we define a two-dimensional coordinate system for the selling price and buying prices, as shown in Figure 3.The critical values in (26) of the sellers and the power grid prices are located in a transverse axis, such as [p h gb , p h,min cs,i , p h,min cs,j , ..., p h,max cs,k , p h,max cs,l , p h gs ], and the critical values in (27) of the buyers and the power grid prices are located in a longitudinal axis, such as [p h gb , p h,min cb,q , p h,min cb,p , ..., p h,max cs,m , p h,max cs,n , p h gs ].The feasible region S of the profit function ( 30) is divided into a certain number of sub-regions, such as sub-region ∀s ∈ S. For each sub-region s ∈ S, it is easily known that the profit function in (30) is concave (the proof can be seen in Appendix A).The optimal buying and selling prices in sub-region s can be found as [p h * cb,s , p h * cs,s ] by using CPLEX which is the commercial solver for MIP problem.Thus, the optimal prices in S can be obtained as Furthermore, the optimal prices [p h * cb,s , p h * cs,s ] will be sent to MGOs.The MGOs can calculate the optimal load demands L h * i according to formulas (22)-( 27).Thus, the SE of the proposed game is reached.The sensitivity analysis of MGs' utilities over the internal prices adopted by the market operator are shown in Appendix B. The sensitivity analysis of MGs' utilities over the internal prices adopted by the market operator are shown in Appendix B.

Basic Data
We employed MATLAB software to program the proposed model and analyze the simulation results.The MIP problem and convex optimization problem were solved by CPLEX.The model was applied to an MMG consisting of 3 MGs.All of the MGs had a PV system and an ESS installed, and the maximum schedulable loads were set to nearly 20% of the maximum load demand.The capacity of the ESS in each MG was 100kWh at the maximum charging/discharging rate of 0.5 C, and the range of SOC is from 0.2 to 1.The degradation cost coefficient i c is 0.005.
The PV generation and load demands of the MGs in a typical day are shown in Figure 4, which were collected from the operation data of different MGs located in Jinzhai, Anhui Province, China.The time-of-use tariffs of the distribution network are shown in Table 1.The subsidy for PV energy was CNY 0.42 per kWh.The length of the rolling optimization was 8 hours at a step of 1 hour.The forecast information, which included load demands and PV generation, was generated from the collected historical data by using the prediction algorithms [20,21]

Basic Data
We employed MATLAB software to program the proposed model and analyze the simulation results.The MIP problem and convex optimization problem were solved by CPLEX.The model was applied to an MMG consisting of 3 MGs.All of the MGs had a PV system and an ESS installed, and the maximum schedulable loads were set to nearly 20% of the maximum load demand.The capacity of the ESS in each MG was 100kWh at the maximum charging/discharging rate of 0.5 C, and the range of SOC is from 0.2 to 1.The degradation cost coefficient c i is 0.005.
The PV generation and load demands of the MGs in a typical day are shown in Figure 4, which were collected from the operation data of different MGs located in Jinzhai, Anhui Province, China.The time-of-use tariffs of the distribution network are shown in Table 1.The subsidy for PV energy was CNY 0.42 per kWh.The length of the rolling optimization was 8 hours at a step of 1 hour.The forecast information, which included load demands and PV generation, was generated from the collected historical data by using the prediction algorithms [20,21].The sensitivity analysis of MGs' utilities over the internal prices adopted by the market operator are shown in Appendix B.

Basic Data
We employed MATLAB software to program the proposed model and analyze the simulation results.The MIP problem and convex optimization problem were solved by CPLEX.The model was applied to an MMG consisting of 3 MGs.All of the MGs had a PV system and an ESS installed, and the maximum schedulable loads were set to nearly 20% of the maximum load demand.The capacity of the ESS in each MG was 100kWh at the maximum charging/discharging rate of 0.5 C, and the range of SOC is from 0.2 to 1.The degradation cost coefficient i c is 0.005.
The PV generation and load demands of the MGs in a typical day are shown in Figure 4, which were collected from the operation data of different MGs located in Jinzhai, Anhui Province, China.The time-of-use tariffs of the distribution network are shown in Table 1.The subsidy for PV energy was CNY 0.42 per kWh.The length of the rolling optimization was 8 hours at a step of 1 hour.The forecast information, which included load demands and PV generation, was generated from the collected historical data by using the prediction algorithms [20,21].

Internal Prices of the MMG
For the convenience of the following analysis, we provide the optimal prices in the MMG under the condition of market equilibrium as shown in Figure 5.It can be seen that that during the periods of 0:00-8:00 and 18:00-24:00 the internal buying and selling prices were equal to those of the power grid because the PV generation of each MG was very small, or even 0, and the internal load demand was relatively large.While all MGs are in the purchasing state, the EMO cannot improve its own benefits by adjusting the internal prices.During the period of 8:00-18:00, the internal buying price was always higher than that of the power grid, and the MG could sell more electricity by adjusting load demand to maximize operation utility.During the periods of 11:00-12:00 and 13:00-16:00, the internal sale price was lower than the selling price of the power grid, and the MG could increase energy consumption to reduce the purchasing cost of the MG and create more economic benefits.

Internal Prices of the MMG
For the convenience of the following analysis, we provide the optimal prices in the MMG under the condition of market equilibrium as shown in Figure 5.It can be seen that that during the periods of 0:00-8:00 and 18:00-24:00 the internal buying and selling prices were equal to those of the power grid because the PV generation of each MG was very small, or even 0, and the internal load demand was relatively large.While all MGs are in the purchasing state, the EMO cannot improve its own benefits by adjusting the internal prices.During the period of 8:00-18:00, the internal buying price was always higher than that of the power grid, and the MG could sell more electricity by adjusting load demand to maximize operation utility.During the periods of 11:00-12:00 and 13:00-16:00, the internal sale price was lower than the selling price of the power grid, and the MG could increase energy consumption to reduce the purchasing cost of the MG and create more economic benefits.The above results show that the EMO could only adjust the internal electricity price during the period of 8:00-18:00, when PV generation is relatively strong.During the daytime, it could promote energy trading in the MMG, and realize the improvement of the operation efficiency of each MG.Next, we analyze the benefits of the MGs and the EMO in combination with the internal prices proposed in this paper.

Rolling Optimization of ESSs
In the proposed method, the local ESS is charged in the case of a PV generation surplus or low electricity price, and discharged in the case of a PV energy deficit or high electricity price to reduce the cost at which the MG purchases the electricity from the power grid.Figure 6 shows the dispatching results of the ESSs in the local MGs.It can be seen that the ESS charged when the internal selling price was equal to the valley price of the power grid in the period of 0:00-6:00.In the period of 7:00-9:00, as load demands increased, the local PV energy was insufficient, the internal selling price was high and discharge of the ESS occurred to reduce the cost of buying energy from the power grids in that period.During the period of 15:00-18:00, the PV generation surplus was The above results show that the EMO could only adjust the internal electricity price during the period of 8:00-18:00, when PV generation is relatively strong.During the daytime, it could promote energy trading in the MMG, and realize the improvement of the operation efficiency of each MG.Next, we analyze the benefits of the MGs and the EMO in combination with the internal prices proposed in this paper.

Rolling Optimization of ESSs
In the proposed method, the local ESS is charged in the case of a PV generation surplus or low electricity price, and discharged in the case of a PV energy deficit or high electricity price to reduce the cost at which the MG purchases the electricity from the power grid.Figure 6 shows the dispatching results of the ESSs in the local MGs.It can be seen that the ESS charged when the internal selling price was equal to the valley price of the power grid in the period of 0:00-6:00.In the period of 7:00-9:00, as load demands increased, the local PV energy was insufficient, the internal selling price was high and discharge of the ESS occurred to reduce the cost of buying energy from the power grids in that period.During the period of 15:00-18:00, the PV generation surplus was absorbed by the ESS.During the period of 18:00-22:00, when the electricity price was high, the absorbed energy was released to reduce the cost of power consumption in the MG.
absorbed by the ESS.During the period of 18:00-22:00, when the electricity price was high, the absorbed energy was released to reduce the cost of power consumption in the MG.In order to further analyze the benefits of ESS scheduling, the benefits of ESS charging and discharging in each period are shown in Figure 7.These benefits were calculated on the basis of the internal prices.The results show that the positive benefits of each ESS in a day were higher than the total negative benefits.The increased benefits of three MGs were CNY 130.08, CNY 133.66 and CNY 143.96, respectively.This validates the necessity and rationality of ESS scheduling in the first stage.In order to further analyze the benefits of ESS scheduling, the benefits of ESS charging and discharging in each period are shown in Figure 7.These benefits were calculated on the basis of the internal prices.The results show that the positive benefits of each ESS in a day were higher than the total negative benefits.The increased benefits of three MGs were CNY 130.08, CNY 133.66 and CNY 143.96, respectively.This validates the necessity and rationality of ESS scheduling in the first stage.absorbed by the ESS.During the period of 18:00-22:00, when the electricity price was high, the absorbed energy was released to reduce the cost of power consumption in the MG.In order to further analyze the benefits of ESS scheduling, the benefits of ESS charging and discharging in each period are shown in Figure 7.These benefits were calculated on the basis of the internal prices.The results show that the positive benefits of each ESS in a day were higher than the total negative benefits.The increased benefits of three MGs were CNY 130.08, CNY 133.66 and CNY 143.96, respectively.This validates the necessity and rationality of ESS scheduling in the first stage.

Demand Response of Local MGs
On the basis of optimal ESS scheduling, each MG will determine its own trading role (see Appendix C) to participate in energy trading in the MMGs by transferring the local information to the EMO.Under the incentive of internal prices, the MGs in the state of selling are encouraged to sell more energy to the EMO, and the MGs in the state of buying are encouraged to buy more energy from the EMO through the load adjustment as shown in Figure 8.During the periods of 11:00-12:00 and 13:00-16:00, MG1 and MG3 were stimulated by internal selling prices in the corresponding hours, which improved the operation utility by increasing energy consumption.On the contrary, the MG2 in the state of selling during the period of 7:00-18:00 could improve benefits by reducing load demand and selling more energy.Compared with the direct transaction with the power grid, the proposed method improves the utility of MGs, and the increased utility in each hour is shown in Figure 9.

Demand Response of Local MGs
On the basis of optimal ESS scheduling, each MG will determine its own trading role (see Appendix C) to participate in energy trading in the MMGs by transferring the local information to the EMO.Under the incentive of internal prices, the MGs in the state of selling are encouraged to sell more energy to the EMO, and the MGs in the state of buying are encouraged to buy more energy from the EMO through the load adjustment as shown in Figure 8.During the periods of 11:00-12:00 and 13:00-16:00, MG1 and MG3 were stimulated by internal selling prices in the corresponding hours, which improved the operation utility by increasing energy consumption.On the contrary, the MG2 in the state of selling during the period of 7:00-18:00 could improve benefits by reducing load demand and selling more energy.Compared with the direct transaction with the power grid, the proposed method improves the utility of MGs, and the increased utility in each hour is shown in Figure 9.

Results of the EMO
The adjustment of energy consumption in local MGs directly affects the total energy sharing and total energy demand in the energy market, as shown in Figure 10.It can be seen that as the internal selling prices were lower than the grid selling prices during the periods of 11:00-12:00 and 13:00-16:00, the total energy demand in the energy market increased significantly.The total energy sharing in the energy market also increased as internal buying prices rose higher than the grid

Demand Response of Local MGs
On the basis of optimal ESS scheduling, each MG will determine its own trading role (see Appendix C) to participate in energy trading in the MMGs by transferring the local information to the EMO.Under the incentive of internal prices, the MGs in the state of selling are encouraged to sell more energy to the EMO, and the MGs in the state of buying are encouraged to buy more energy from the EMO through the load adjustment as shown in Figure 8.During the periods of 11:00-12:00 and 13:00-16:00, MG1 and MG3 were stimulated by internal selling prices in the corresponding hours, which improved the operation utility by increasing energy consumption.On the contrary, the MG2 in the state of selling during the period of 7:00-18:00 could improve benefits by reducing load demand and selling more energy.Compared with the direct transaction with the power grid, the proposed method improves the utility of MGs, and the increased utility in each hour is shown in Figure 9.

Results of the EMO
The adjustment of energy consumption in local MGs directly affects the total energy sharing and total energy demand in the energy market, as shown in Figure 10.It can be seen that as the internal selling prices were lower than the grid selling prices during the periods of 11:00-12:00 and 13:00-16:00, the total energy demand in the energy market increased significantly.The total energy sharing in the energy market also increased as internal buying prices rose higher than the grid

Results of the EMO
The adjustment of energy consumption in local MGs directly affects the total energy sharing and total energy demand in the energy market, as shown in Figure 10.It can be seen that as the internal selling prices were lower than the grid selling prices during the periods of 11:00-12:00 and 13:00-16:00, the total energy demand in the energy market increased significantly.The total energy sharing in the energy market also increased as internal buying prices rose higher than the grid buying prices during the period of 7:00-18:00.The results show that MGs will actively participate in energy market transactions under the incentive of internal electricity prices.
Appl.Sci.2018, 8, x FOR PEER REVIEW 12 of 17 buying prices during the period of 7:00-18:00.The results show that MGs will actively participate in energy market transactions under the incentive of internal electricity prices.Energy demand and sharing in the energy market is the premise for the EMO to make profits.In the periods of 0:00-7:00 and 18:00-24:00, MGs were in the buying state thus the EMO could only purchase energy from the power grids to meet the demands of MGs and could not obtain profits.Figure 11 compares the profits of the EMO under the internal price strategy and the grid price strategy during the period of 7:00 and 18:00.Under the incentive of the internal price strategy, the profit of the EMO increased significantly in these periods.The daily profit of the EMO increased from CNY 171.6 to CNY 277.7 and the rate of increase is 61.82%.To further illustrate the advantages of the proposed method, the net energy and PV utilization of the MMG were analyzed.The comparison of net energy curves is shown in Figure 12.During the period of 0:00-6:00, the charging of the ESS led to an increase in net energy.During the period of 18:00-24:00, the discharging of the ESS led to a decline in net energy.During the period of 6:00-18:00, the change in net energy was affected by ESS scheduling and the load demand response under internal prices.Obviously, the proposed method produced smaller fluctuations of the net load than the original method, and the PV energy reversal in the MMG during the period of 10:00-15:00 was significantly suppressed.Energy demand and sharing in the energy market is the premise for the EMO to make profits.In the periods of 0:00-7:00 and 18:00-24:00, MGs were in the buying state thus the EMO could only purchase energy from the power grids to meet the demands of MGs and could not obtain profits.Figure 11 compares the profits of the EMO under the internal price strategy and the grid price strategy during the period of 7:00 and 18:00.Under the incentive of the internal price strategy, the profit of the EMO increased significantly in these periods.The daily profit of the EMO increased from CNY 171.6 to CNY 277.7 and the rate of increase is 61.82%.buying prices during the period of 7:00-18:00.The results show that MGs will actively participate in energy market transactions under the incentive of internal electricity prices.Energy demand and sharing in the energy market is the premise for the EMO to make profits.In the periods of 0:00-7:00 and 18:00-24:00, MGs were in the buying state thus the EMO could only purchase energy from the power grids to meet the demands of MGs and could not obtain profits.Figure 11 compares the profits of the EMO under the internal price strategy and the grid price strategy during the period of 7:00 and 18:00.Under the incentive of the internal price strategy, the profit of the EMO increased significantly in these periods.The daily profit of the EMO increased from CNY 171.6 to CNY 277.7 and the rate of increase is 61.82%.To further illustrate the advantages of the proposed method, the net energy and PV utilization of the MMG were analyzed.The comparison of net energy curves is shown in Figure 12.During the period of 0:00-6:00, the charging of the ESS led to an increase in net energy.During the period of 18:00-24:00, the discharging of the ESS led to a decline in net energy.During the period of 6:00-18:00, the change in net energy was affected by ESS scheduling and the load demand response under internal prices.Obviously, the proposed method produced smaller fluctuations of the net load than the original method, and the PV energy reversal in the MMG during the period of 10:00-15:00 was significantly suppressed.To further illustrate the advantages of the proposed method, the net energy and PV utilization of the MMG were analyzed.The comparison of net energy curves is shown in Figure 12.During the period of 0:00-6:00, the charging of the ESS led to an increase in net energy.During the period of 18:00-24:00, the discharging of the ESS led to a decline in net energy.During the period of 6:00-18:00, the change in net energy was affected by ESS scheduling and the load demand response under internal prices.Obviously, the proposed method produced smaller fluctuations of the net load than the original method, and the PV energy reversal in the MMG during the period of 10:00-15:00 was significantly suppressed.Moreover, the comparison of peak-to-average ratio (PAR) and PV utilization ratio are shown in Table 2. Through comparisons, it is found that the proposed method can effectively reduce the peak-to-average ratio of the net load and improve the utilization ratio of PV energy.

Utility Comparisons with Other Methods
To further illustrate the advantages of the proposed hour-ahead optimization over day-ahead optimization, the methods in references [16] and [17] were applied to the proposed case, and the cost optimization problem of references [16] and [17] were transformed into the utility optimization problem to make a comparative analysis of the results.In addition, the day-ahead stochastic prediction errors of PV and the load demand were set to 10% and 12%, while the hour-ahead stochastic prediction errors were set to 5% and 6%, respectively.The results are shown in Table 3.The results show that EMO and MGO can achieve more benefits by using the optimization method proposed in this paper.

Conclusions
In this paper, a hierarchical optimization method is proposed to optimize the energy transaction of a MMG in two stages.Firstly, the EMS of each MG determines the scheduling of ESS in the next hour by adopting rolling optimization, and decides its participating role in energy trading market.Secondly, according to forecast information and the energy trading roles collected from MGs, the EMO optimizes the internal prices of next hour based on Stackelberg game theory.The simulation results show that the utility of both the EMO and MGO are increased by using the Moreover, the comparison of peak-to-average ratio (PAR) and PV utilization ratio are shown in Table 2. Through comparisons, it is found that the proposed method can effectively reduce the peak-to-average ratio of the net load and improve the utilization ratio of PV energy.

Utility Comparisons with Other Methods
To further illustrate the advantages of the proposed hour-ahead optimization over day-ahead optimization, the methods in references [16] and [17] were applied to the proposed case, and the cost optimization problem of references [16] and [17] were transformed into the utility optimization problem to make a comparative analysis of the results.In addition, the day-ahead stochastic prediction errors of PV and the load demand were set to 10% and 12%, while the hour-ahead stochastic prediction errors were set to 5% and 6%, respectively.The results are shown in Table 3.The results show that EMO and MGO can achieve more benefits by using the optimization method proposed in this paper.

Conclusions
In this paper, a hierarchical optimization method is proposed to optimize the energy transaction of a MMG in two stages.Firstly, the EMS of each MG determines the scheduling of ESS in the next hour by adopting rolling optimization, and decides its participating role in energy trading market.Secondly, according to forecast information and the energy trading roles collected from MGs, the EMO optimizes the internal prices of next hour based on Stackelberg game theory.The simulation results show that the utility of both the EMO and MGO are increased by using the proposed method.In addition, the net load curve and utilization ratio of PV energy in the whole MMG system are both improved.
where (p h cb , p h cs ) ∈ s, and p h cb < p h cs .Therefore, the Hessian matrix of Pro h EMO is For MG i, who is a seller, if p h cb,i ∈ [p h,min cb,i , p h,max cb,i ], then the optimal load demand can be expressed as The corresponding utility is rewritten as To further demonstrate the utility of MG response to the internal electricity price, the utilities of MG1 and MG3 in time slot 14 were chosen to show the relationship between utility and changing prices, which are shown in Figures A1 and A2 To further demonstrate the utility of MG response to the internal electricity price, the utilities of MG1 and MG3 in time slot 14 were chosen to show the relationship between utility and changing prices, which are shown in Figures 13 and 14, respectively.).

Figure 2 .
Figure 2. Operation strategy of an MMG (EMO: Energy Market Operator, MGO: Microgrid Operator, PV: photovoltaic).2.2.1.First Stage Considering the time-coupling characteristics of a local ESS, a local EMS adopts a rolling optimization method to determine the scheduling plan of the ESS for next period, which can minimize the cost for the MG to purchase energy from the EMO and determine the role of the MG to participate in energy trading in the next period.The inputs of the rolling optimization are the long-term forecast information of local PV output, local energy demand and the price obtained from the EMO.

Figure 3 .
Figure 3. Feasible region of internal prices.

Figure 3 .
Figure 3. Feasible region of internal prices.

Figure 4 .
Figure 4. Basic data in hour: (a) PV energy output for 3 MGs, and (b) load demands for 3 MGs.

Figure 4 .
Figure 4. Basic data in hour: (a) PV energy output for 3 MGs, and (b) load demands for 3 MGs.

Figure 5 .
Figure 5.Comparison of internal prices and grid prices.

Figure 5 .
Figure 5.Comparison of internal prices and grid prices.

Figure 7 .
Figure 7. Impact of ESS scheduling on the benefits of MGs.

17 Figure 7 .
Figure 7. Impact of ESS scheduling on the benefits of MGs.

Figure 9 .
Figure 9. Increased utility of MGs based on internal prices.

Figure 9 .
Figure 9. Increased utility of MGs based on internal prices.

Figure 9 .
Figure 9. Increased utility of MGs based on internal prices.

Figure 10 .
Figure 10.Comparison of energy trading in the energy market.

Figure 11 .
Figure 11.Comparison of the utility of the EMO.

Figure 10 .
Figure 10.Comparison of energy trading in the energy market.

Figure 10 .
Figure 10.Comparison of energy trading in the energy market.

Figure 11 .
Figure 11.Comparison of the utility of the EMO.

Figure 11 .
Figure 11.Comparison of the utility of the EMO.

Figure 12 .
Figure 12.Comparison of net energy of MMGs.

Figure 12 .
Figure 12.Comparison of net energy of MMGs.

D
determine the roles of MGs participating in energy trading, which are all listed in the follow Table EMO are the utility of the MG and the profit of the EMO which are expressed by (2) and (9), respectively.
Definition: Consider the game G defined in (17) as a set of strategies (L h i * , P h cb * , P h cs * ) constituting a Stackelberg equilibrium (SE) if (and only if) the following set of inequalities are satisfied: .

Table 2 .
Comparison of pea-to-average ratio (PAR) and PV utilization ratio.

Table 3 .
Comparisons with other methods.

Table 2 .
Comparison of pea-to-average ratio (PAR) and PV utilization ratio.

Table 3 .
Comparisons with other methods.
 Consider the fact that k h i > 0, p h gb > 0, p h gs > 0, p h cs > 0, p h cb > 0 ∀i ∈ N, h ∈ H, H is thus negative definite and Pro h MGC is strictly concave with respect to p h cs and p h cb .Formulas (26) and (27) show that if p h cs,i / ∈ [p h,min cs,i , p h,max cs,i ], then ∂U h i /∂p h cs,i = 0, and if p h cb,i / ∈ [p h,min cb,i , p h,max cb,i ], then ∂U h i /∂p h cb,i = 0. Therefore, we mainly discuss the case of p h cs,i ∈ [p h,min cs,i , p h,max cs,i ] and p h cb,i ∈ [p h,min cb,i , p h,max cb,i ].For MG i, who is a buyer, if p h cs,i ∈ [p h,min cs,i , p h,max cs,i ], then the optimal load demand can be expressed as , respectively.

Table 4 .
Roles of Energy Trading.