Energy Management of Combined Cooling, Heating and Power Micro Energy Grid Based on Leader-Follower Game Theory

: In this paper, we consider a general model and solution algorithm for the energy management of combined cooling, heating, and power micro energy grid (MEG) under the game theory framework. An innovative dynamic leader-follower game strategy is proposed in this paper to balance the interactions between MEG and user. We show that such game between MEG and user has a unique Nash equilibrium (NE), and in order to quantify the user’s expenditure and dissatisfaction, we model them and adopt the fuzzy bi-objective algorithm. For more details in the proposed game model, the MEG leads the game by deciding energy sales prices and optimizing the power, cooling and heating outputs according to the user’s load plan to maximize its own proﬁt. With the prices being released by MEG, user’s adjustment of energy consumption follows and is again fed to MEG. In practice, we initialize simulations with daily loads of a typical community. As the numerical results demonstrate, MEG is proﬁcient in consumption capacity of renewable energy and energy optimization. It also shows that the user achieves his economic optimum with experience of energy usage taken into account.


Introduction
As an economic and environment-friendly energy utilization system, the concept of Energy Internet (EI) is first introduced by American scholar Jeremy Rifkin [1]. EI is designed to transport clean energy, connect different energy sources and provide users with cooling, heating, and electrical energy services in various forms. Moreover, it is closely related to and highly compatible with information technology. A Micro Energy Grid (MEG) [2] is a group of multiple distributed generation devices and loads operating as an integrated energy system, which is connected to the power grid and is able to function in parallel with the grid or in isolated island mode. Being the kernel of EI, MEG is thus researched to boost energy management of the whole network.
The technology of combined cooling, heating and power (CCHP) [3,4] has greatly developed MEG and brought benefits to economy and eco-system. With the aid of CCHP, MEG is not only able to organize and manage the distribution of energies, but can also undertake the task of supplying cooling, heating, and electrical needs [5]. MEG can now link diverse energy conversion devices such as internal combustion engine, ice-storage air-conditioner, photovoltaic (PV), and wind turbines (WT), and possess different operation conditions (isolated island/grid). In March 2017, China National Energy Administration published "Notice on Conducting Pilot Project of Market-oriented Distributed Power

•
An innovative dynamic leader-follower game is proposed in this paper to balance the interactions between MEG and user; moreover, different from Pareto front obtained by the traditional multi-objective optimization, the Nash equilibrium (NE) here that provides optimal solutions is proven to exist uniquely.

•
The proposed analytical method accommodates both the individual rationality and activity of participants in the decision-making process, and resolves the conflict between different stakeholders. • In the process of game optimization, the multi-energy needed on the user side are fully considered and used as optimization variables.

•
The user dissatisfaction function is proposed, which is characterized as the inherent relationship between the actual load and the demand load.

•
The user's expenditure and dissatisfaction are quantified in the fuzzy bi-objective algorithm, which makes the user objective function more reasonable.
The rest of the paper proceeds as follows. First, we describe the leader-follower game model in Section 2; we then propose a complete set of analytical methods to solve game model in Section 3; through simulations in different scenarios, we next obtain the equilibrium strategies and analyze the results in Section 4; and, the conclusions and future works are provided in Section 5.

The Modeling of MEG
In this paper, we establish the CCHP integrated MEG model based on energy bus [28]. According to the type of energy conversion, energy buses are divided into three types: electrical energy bus, heating energy bus, and cooling energy bus. The devices connecting to the bus are divided into four types: source, converter, energy storage, and load. Energy bus-based structure of CCHP integrated MEG is shown in Figure 1. The figure shows not only the coupling relationships among different devices in detail, but also the conversion relationships between devices and energies. Internal combustion engine, ice-storage air conditioner, energy storage, PV, WT, and other main devices modeling reference to [6,8,9]. Independent modeling of different sources (wind/light/electricity/gas), storages (electrical/thermal/cold), and conversion devices in MEG must satisfy the cooling, heating, and electrical balance equations, and their operation constraints in each t period. The followings are the energy balance equations in MEG.  (1), Pgrid,t is the exchange power between the electrical bus and the power grid. Defining that Pgrid,t is positive indicating purchasing power from the grid, while negative means selling power to the grid. Ppv,t, Pwt,t, Pce,t are power outputs of PV, WT, and internal combustion engine, respectively. Pch,t, Pice,t, Pmelt,t are, respectively, electricity consumptions while chiller working, ice-storage air-conditioner making, and melting ice. Icc,t, Icd,t represent status values of ice-storage air-conditioner when making and melting ice (0/1, 0 is off and 1 is on). Sbad,t, Sbac,t represent discharging and charging power of battery storage. Ibad,t, Ibac,t represent status values of battery storage when discharging and charging. Le,t is the actual electrical load consumed by the user.
In Equation (2), Hce,t, Hdf,t, are heating outputs of heat exchanger and direct-fired absorption chiller. Shd,t, Shc,t represent heating discharging and charging power of thermal storage. Ihd,t, Ihc,t represent status values of thermal storage when discharging and charging. Lh,t is the actual heating load consumed by the user.
In Equation (3), Cdf,t, Cch,t, Cice,t, are cooling outputs of direct-fired absorption chiller, chiller, and ice-storage air-conditioner. Lc,t is the actual cooling load that is consumed by the user.
In addition to the above energy balances in energy buses, the output constraints and operating conditions of each device need to be taken into account, including the output upper and lower bounds of each device and the status constraints of the energy storage device. Device constraints are as follows:

•
Electrical balance P grid,t + P pv,t + P wt,t + P ce,t + I bad,t S bad,t = L e,t + P ch,t + I cc,t P ice,t + I cd,t P melt,t + I bac,t S bac,t (1) Equations (1)-(3) are the energy balances in electrical energy bus, heating energy bus, and cooling energy bus, respectively. t represents the time period.
In Equation (1), P grid,t is the exchange power between the electrical bus and the power grid. Defining that P grid,t is positive indicating purchasing power from the grid, while negative means selling power to the grid. P pv,t , P wt,t , P ce,t are power outputs of PV, WT, and internal combustion engine, respectively. P ch,t , P ice,t , P melt,t are, respectively, electricity consumptions while chiller working, ice-storage air-conditioner making, and melting ice. I cc,t , I cd,t represent status values of ice-storage air-conditioner when making and melting ice (0/1, 0 is off and 1 is on). S bad,t , S bac,t represent discharging and charging power of battery storage. I bad,t , I bac,t represent status values of battery storage when discharging and charging. L e,t is the actual electrical load consumed by the user.
In Equation (2), H ce,t , H df,t , are heating outputs of heat exchanger and direct-fired absorption chiller. S hd,t , S hc,t represent heating discharging and charging power of thermal storage. I hd,t , I hc,t represent status values of thermal storage when discharging and charging. L h,t is the actual heating load consumed by the user.
In Equation (3), C df,t , C ch,t , C ice,t , are cooling outputs of direct-fired absorption chiller, chiller, and ice-storage air-conditioner. L c,t is the actual cooling load that is consumed by the user.
In addition to the above energy balances in energy buses, the output constraints and operating conditions of each device need to be taken into account, including the output upper and lower bounds of each device and the status constraints of the energy storage device. Device constraints are as follows.

Device Output Constraints
Each device must work within upper and lower bounds. In Equations (4)-(7), P ce,max , P m,max , H n,max , C k,max are, respectively, the maximum power output bounds for internal combustion engine, m-type device, n-type device, and k-type device. P ce,min is the minimum power output bound for internal combustion engine. P ce,t , P m,t , H n,t , C k,t represent the power output of internal combustion engine, m-type device, n-type device, and k-type device, respectively. Here, the m types of devices include direct-fired absorption chiller, chiller, ice-storage air-conditioner, PV, and WT. The n types of devices include heat exchanger and direct-fired absorption chiller. The k types of devices include direct-fired absorption chiller, chiller, and ice-storage air-conditioner.

Constraints for Energy Storage Devices
Energy storage devices (battery storage, thermal storage, ice tank) cannot store more energy than their own capacity and release too much energy per unit time. In addition, the devices need to retain some redundancy in the case of power failure accident. In Equations (8)- (11), E in,t , E out,t are the energy storage and release rates in period t, respectively. E in,max , E out,max are the maximum charge and discharge rates per unit time. SOC t is the state of charging of the energy storage device. It is worth mentioning that charging mode and discharging mode cannot operate simultaneously. Besides, the initial SOC value equals the final SOC value.

The Basic Idea of the Game
Due to the randomness of renewable energy (PV, WT) and the adjustability of user's energy usage, the main focal points of MEG are: optimizing device outputs rationally to consume renewable energy, and deciding the energy sales prices to increase the economic benefits. From the user's point of view, how to adjust consumed energy loads according to different energy sales prices, and reduce the expenditure within the acceptable experience of energy usage are also the key issues. Therefore, the MEG and the user belong to different stakeholders. Both of them have their goals, needs, and adjustment strategies. These two are closely related and have some interest relations. However, the objective benefits of both are affected by each other's strategies. Obviously, it is a typical game problem.
Game theory is an effective tool to solve the conflicts among different stakeholders [29]. Among the different game theories, the leader-follower game is the dynamic game of decision-making existing in order with complete information. There are at least one leader and one follower existing in the leader-follower game. The process of the game is that the leader first determines his own decision, and the follower makes the decision that maximizes his own interests according to the leader's decision. The interactive process of MEG energy sales prices and user's energy utilization strategies can be regarded as the leader-follower game. As the leader, MEG decides the best energy sales prices to maximize its profit. As a follower, the user adjusts energy utilization strategies to minimize the expenditure and dissatisfaction, according to the leader's decision. It is worth mentioning that, Energies 2018, 11, 647 6 of 21 the user energy strategy in the paper is similar to the demand response. In recent years, with the development of smart grid technology, users can actively change their own electricity consumption behavior, according to released information, thus making the demand response possible. With the gradual improvement of relevant policies and technology, more users participate in the electricity market as the demand response side. For residential users, the load is controlled by adjusting the time and period of the usage of energy-consuming equipment, such as refrigerators, air conditioners, heaters, electric vehicles, etc., or by using a home energy management system. In addition, the residential users' loads can be aggregated by a load aggregator. For the prices that were released by MEG, the load aggregator responds by adjusting users' loads. Because the load aggregator has signed a contract with the user, and the users' loads can be adjusted within a certain range, that is, the user can adopt the energy strategy to some extent. Commercial and industrial users have a large base of energy consumption and a regular pattern of energy usage, which can participate well in demand response and make a certain degree of load adjustment. Therefore, it is possible for the user to adopt an energy strategy. The leader-follower game model of MEG and user constructed in this paper is as follow [30]: In Equations (12)- (14), there are three basic elements of a general leader-follower game: participants G, strategies Ω, profit/payment I. In this leader-follower game, participants are MEG and user. During the game, the MEG decides the energy sales prices pr, and the user uses the load l t as strategies to adjust his own profit/payment.
In the leader-follower game, when the follower makes the optimal response accepted by the leader, the game reaches Nash equilibrium. At this moment, the utility function of the participants reaches the optimum, namely: where pr * , l t * are the participants' equilibrium strategies. When all of the game participants' strategies are NE strategies, no player can gain more by adjusting their strategies independently.

Profit Function of the MEG
Profit/payment is the goal pursued by each participant in the game. It is a function of strategy. Reasonably constructing the profit/payment function has a great influence on the solution of the game model. The leader-follower game model established in this paper divides one day into N = 24 periods, and takes each period as the basic unit to make scheduling and optimization.
The profit of MEG is related to the distributed energy capacity, output characteristics, real-time electricity price (RTP), heating price, and unit emissions. The expression is as follow: where pr t l t represents the energy sales revenue in period t, pr t represent the energy sales prices (including the RTP pr e t , heating, and cooling price pr h/c t ) of the MEG in period t, l t represents the actual loads consumed by the user in period t.
Energies 2018, 11, 647 7 of 21 C t is the production cost of MEG, including the operation cost and maintenance cost of each unit, the expression is as follow: where J g is the price of natural gas, J e is electricity price of the grid, V g,t is the total gas volume consumed by MEG in period t. K i is the maintenance cost factor for i-type device, P i,t is the output power of the i-type device in period t, I is the total number of devices in MEG. The operation costs of renewable energy devices (PV and WT) are negligible. Emi t is the emission cost of MEG. The emissions include emissions from the purchase of electricity through the power grid and from the gas combustion. The expression is as follow: where p emi is the emission price of gas per unit volume, and w gasC , w gasN are the CO 2 and NO x emission volume corresponding to the kWh unit of gas consumed, respectively. w elcC , w elcN are the CO 2 and NO x emission volume corresponding to the kWh unit of electricity that is purchased from the grid, respectively.
ρ is the punishment fee that represents the deviation between the generation of renewable energy and the actual electrical load in MEG: where g re is the total generating capacity of renewable energy in period t, and δ is the penalty price per unit of electricity.

Payment Function of the User
The payment function of the user includes the energy-purchasing expenditure P and the user dissatisfaction function R towards the actual loads.
The expenditure P is expressed as: As for the user dissatisfaction function, the following requirements should be satisfied: • Before reaching the user demand load, the larger the actual load reduction ratio is, the higher the dissatisfaction is. • When reaching the demand load, the dissatisfaction becomes zero. • When the actual load exceeds the demand load, the dissatisfaction becomes negative, which means that it becomes satisfaction. • However, the user satisfaction cannot increase indefinitely with the increase of the actual load. When the actual load increases to a certain extent, the change of user satisfaction tends to be slow and close to a preset value.
Based on the above requirements, this paper takes the deviation between the user's actual load and the demand load as variables [31], and constructs the following function to quantify the user dissatisfaction with the actual load. The user dissatisfaction function R is expressed as: where α > 1, β > 0, λ > 1. Parameter β is the preset value that the function close to when the actual load increases to a certain extent. Parameter λ is related to the proportional elasticity of load adjustment, which makes the adjustment more sensitive. Parameter α is related to the magnitude of the function. d t is the user's demand load. When taking into account the economy, the user experience of energy usage should also be considered. Therefore, we take the user dissatisfaction as one of the user's goals.
In other words, the user's actual load is adjusted within a reasonable range, so as not to affect the experience of energy usage. Under the determined demand load d t , with the adjustment of l t , the user dissatisfaction function curves with different parameters are shown in Figure 2. where, α > 1, β > 0, λ > 1. Parameter β is the preset value that the function close to when the actual load increases to a certain extent. Parameter λ is related to the proportional elasticity of load adjustment, which makes the adjustment more sensitive. Parameter α is related to the magnitude of the function. dt is the user's demand load. When taking into account the economy, the user experience of energy usage should also be considered. Therefore, we take the user dissatisfaction as one of the user's goals. In other words, the user's actual load is adjusted within a reasonable range, so as not to affect the experience of energy usage. Under the determined demand load dt, with the adjustment of lt, the user dissatisfaction function curves with different parameters are shown in Figure 2.  Figure 2 shows the changing trend of the user dissatisfaction function R. Analyzing the trend of the three curves, we can see that it meets the requirements of the specific meaning of the user dissatisfaction. In addition, with the different values of α, β, and λ, it can be seen that the values of α, β affect the upper and lower bounds of the function R, and the value of λ affects the sensitivity of load adjustment.
The user payment function needs to take into account the user expenditure P and the user dissatisfaction R, which is a bi-objective optimization problem. When considering the different dimensions and ranges of the functions, it may not be sensible to do the simple linear addition, even if a factor is multiplied. We uses a fuzzy solution to deal with the bi-objective optimization problem based on [32]. The key of fuzzy bi-objective optimization approach is to establish two mappings of objective function values to the optimal solution membership degree, respectively. Here, the membership degree is as the degree of satisfaction of each objective function value. The main steps of fuzzy bi-objective optimization of the user payment function are as follows:

•
Step 1: Solve the optimization problem with only expenditure objective. Get the optimal expenditure Pm, and calculate the user dissatisfaction in this condition, noted as RM.

•
Step 2: Solve the optimization problem with only dissatisfaction objective. Get the optimal dissatisfaction Rm, and calculate the expenditure in this condition, noted as PM.

•
Step 3: Fuzz the objective function value and establish the mapping from single objective function value to membership degree. The membership degree function in this paper is a semi-trapezoidal distribution: the membership degree equals 1 when P < Pm, and it equals 0 when P > PM (the same as R). When Pm < P < PM and Rm < R < RM, the membership degree is a linear distribution. The two membership degree functions are shown in Equations (23) and (24).   The user payment function needs to take into account the user expenditure P and the user dissatisfaction R, which is a bi-objective optimization problem. When considering the different dimensions and ranges of the functions, it may not be sensible to do the simple linear addition, even if a factor is multiplied. We uses a fuzzy solution to deal with the bi-objective optimization problem based on [32]. The key of fuzzy bi-objective optimization approach is to establish two mappings of objective function values to the optimal solution membership degree, respectively. Here, the membership degree is as the degree of satisfaction of each objective function value. The main steps of fuzzy bi-objective optimization of the user payment function are as follows:

•
Step 1: Solve the optimization problem with only expenditure objective. Get the optimal expenditure P m , and calculate the user dissatisfaction in this condition, noted as R M .

•
Step 2: Solve the optimization problem with only dissatisfaction objective. Get the optimal dissatisfaction R m , and calculate the expenditure in this condition, noted as P M .

•
Step 3: Fuzz the objective function value and establish the mapping from single objective function value to membership degree. The membership degree function in this paper is a semi-trapezoidal distribution: the membership degree equals 1 when P < P m , and it equals 0 when P > P M (the same as R). When P m < P < P M and R m < R < R M , the membership degree is a linear distribution. The two membership degree functions are shown in Equations (23) and (24). • Step 4: Add the weighted membership degree functions linearly, and then the fuzzy bi-objective function can be obtained. According to the principle of maximizing membership degree, the fuzzy bi-objective function is taken as the payment function of the user to be optimized in the game.
In summary, the payment function of the user is shown in Equation (25): where a, b are the weight coefficients of objective 1 and objective 2, respectively.

Nash Equilibrium
According to the definition of NE [33], when assuming that (pr t * , l t * ) N t=1 is the NE strategy of the game model when the leader-follower game has a unique NE. It indicates that the MEG (leader) decides the energy sales prices (pr t * ) N t=1 as its strategy, and the user (follower) adjusts his own energy load strategy (l t * ) N t=1 on the basis of the released MEG strategy, in which case the benefits of both participants can reach the equilibrium.
The strategies determined by the MEG and the user in this paper belong to pure strategies. If and only if the following theorem is satisfied, the NE of the leader-follower game exists uniquely. Theorem 1. In a multi-participants game, if (1) the pure strategy set S of each participant is a non-empty, closed, bounded convex set in the Euclid space; (2) for the leader's optimal strategy, the follower has the only optimal strategy; (3) for the follower's optimal strategy, the leader has the only optimal strategy.

Proof of Theorem 1.
(1) Since the strategy space of game between MEG and user is non-empty, tight, convex set in Euclid space, the condition (1) of the Theorem 1 is satisfied.
(2) For the leader's optimal strategy, we solve the follower's optimal strategy. By taking the derivative of the user payment function I user with respect to l t , we can obtain: where P 0 = P M − P m , R 0 = R M − R m . By letting the left of Equation (26) equal to 0, we can obtain the user's optimal strategy: Then, by taking the second-order derivatives of the user payment function I user with respect to l t and l i , we can obtain: From Equation (28), we can see that Hessian matrix of I user is a negative definite matrix, which means that the user's payment function is a concave function, and (l t * ) N t=1 is the user's optimal strategy for the given energy sales prices. So, the condition (2) of the Theorem 1 is satisfied.
(3) Substituting user's optimal strategy (27) into MEG's profit function (17), we can obtain: In Equation (29), Emi t and C t are, respectively, linear and non-linear functions of P grid,t , V g,t , which would not change with the strategy (pr t * ) N t=1 . Therefore, they can be considered as constants. By taking the second-order derivatives of the MEG's profit function I MEG , with respect to pr t and pr i , we can obtain: Similarly, we can see from Equation (30) that the Hessian matrix of I MEG is a negative definite matrix, which means that the profit function of the MEG is a concave function. Moreover, the MEG's strategy set is a convex set, so there is a unique solution to the MEG's profit optimization problem. That is, (pr t * ) N t=1 is the optimal price strategy for the user's optimal load strategy. So, the condition (3) of the Theorem 1 is satisfied.
The above proofs show that there is a unique NE existing in the leader-follower game proposed in this paper, and the equilibrium strategy can be obtained through the following solution.

Game Solving Process
The game in this paper is a dynamic game of complete information. The decision of each round of MEG or user is a function of the information obtained before the decision-making. After a series of rounds, the game results in equilibrium. According to the user's historical data, MEG sets the initial price strategy; the user adjusts his energy strategy according to the energy sales prices and feedbacks to the MEG. The constraints of user's energy strategy are as follows: where l t,un is the uninterruptible load, that is, the user's minimum load requirement must be satisfied in period t. ν is the maximum interruptible load ratio. The user-adjusted load cannot be too much to affect the normal demand. According to the user's energy strategy, MEG adjusts the energy sales prices to optimize the profit I MEG . From Equation (26), the function of pr t about l t can be obtained: The constraints of MEG's strategy are as follows: where pr mc,t is the marginal cost of MEG generating energy. MEG updates price strategy and then feedbacks to the user. This process alternates iteratively until it reaches equilibrium. Flow chart of solving the leader-follower game is shown in Figure 3.

Case Study
In the simulation, we take a CCHP integrated MEG as an example. The PV, WT, power grid, and gas provide power and fuel, and the conversion devices in MEG meet the user's cooling, heating, and electrical loads by converting the energy. In this paper, the leader-follower game between MEG and user is simulated in three scenarios: Scenario 1: MEG is not connected to the power grid and under isolated island operation. The MEG only use electricity price as the strategy to game with the user. Scenario 2: MEG is connecting to the power grid, which electricity can be purchased from or sold to the power grid. The MEG only use electricity price as the strategy to game with the user.
Scenario 3: MEG is connecting to the power grid. The MEG uses electricity price and heating price as strategies to game with the user.
The models proposed in the paper are solved in Matlab platform. Since the problems of this paper are mixed integer nonlinear programming, fmincon is selected as the solution engine. Each model is run for one typical day, with three types of loads, electrical load, heating load, and cooling load.

Simulation Settings
We take the representative typical day, with three types of loads, electrical load, heating load, and cooling load of a certain community in summer as an example to run the simulation. In the simulation, the maximum interruptible load ratio ν is 0.1. In dissatisfaction function, α is 8, β is 12, λ is 5; the weight coefficients a, b are, respectively, 0.85 and 0.15. In the Scenario 1 and 2, the heating

Case Study
In the simulation, we take a CCHP integrated MEG as an example. The PV, WT, power grid, and gas provide power and fuel, and the conversion devices in MEG meet the user's cooling, heating, and electrical loads by converting the energy. In this paper, the leader-follower game between MEG and user is simulated in three scenarios: Scenario 1: MEG is not connected to the power grid and under isolated island operation. The MEG only use electricity price as the strategy to game with the user. Scenario 2: MEG is connecting to the power grid, which electricity can be purchased from or sold to the power grid. The MEG only use electricity price as the strategy to game with the user.
Scenario 3: MEG is connecting to the power grid. The MEG uses electricity price and heating price as strategies to game with the user.
The models proposed in the paper are solved in Matlab platform. Since the problems of this paper are mixed integer nonlinear programming, fmincon is selected as the solution engine. Each model is run for one typical day, with three types of loads, electrical load, heating load, and cooling load.

Simulation Settings
We take the representative typical day, with three types of loads, electrical load, heating load, and cooling load of a certain community in summer as an example to run the simulation. In the simulation, the maximum interruptible load ratio ν is 0.1. In dissatisfaction function, α is 8, β is 12, λ is 5; the weight coefficients a, b are, respectively, 0.85 and 0.15. In the Scenario 1 and 2, the heating price is 0.233 Yuan/kWh. The rated capacity of each device in MEG is shown in Table 1. The cold, thermal,  Table 2. The predicted renewable energy outputs of the typical day are shown in Figure 4. The detailed time of use (TOU) price and the set MEG electricity-selling (to the grid) price are illustrated in Figure 5.

Optimal Results Analysis in Different Scenarios
According to the established game model, the game optimal simulation results of the typical day in different scenarios are shown in Figures 6-11. Figures 6, 8, and 10 are the final strategies of both participants in the game at the NE point. Figures 7, 9, and 11 present the optimal results of the electrical, heating, and cooling outputs of all devices in MEG after game optimization. For Figures 7,9 and 11, the upper part of the abscissa axis represents the power injected into the energy bus from distributed energy sources, while the lower part represents the power flowed out from the energy bus. Table 3 shows MEG production costs, profits, user payment, and user expenditure in different scenarios. This paper takes RMB as the currency unit, and $1 USD is about ¥6.30 RMB.

Optimal Results Analysis in Different Scenarios
According to the established game model, the game optimal simulation results of the typical day in different scenarios are shown in Figures 6-11. Figures 6, 8, and 10 are the final strategies of both participants in the game at the NE point. Figures 7, 9, and 11 present the optimal results of the electrical, heating, and cooling outputs of all devices in MEG after game optimization. For Figures 7,9 and 11, the upper part of the abscissa axis represents the power injected into the energy bus from distributed energy sources, while the lower part represents the power flowed out from the energy bus. Table 3 shows MEG production costs, profits, user payment, and user expenditure in different scenarios. This paper takes RMB as the currency unit, and $1 USD is about ¥6.30 RMB.

Optimal Results Analysis in Different Scenarios
According to the established game model, the game optimal simulation results of the typical day in different scenarios are shown in Figures 6-11. Figures 6, 8 and 10 are the final strategies of both participants in the game at the NE point. Figures 7, 9 and 11 present the optimal results of the electrical, heating, and cooling outputs of all devices in MEG after game optimization. For Figures 7,9 and 11, the upper part of the abscissa axis represents the power injected into the energy bus from distributed energy sources, while the lower part represents the power flowed out from the energy bus. Table 3 shows MEG production costs, profits, user payment, and user expenditure in different scenarios. This paper takes RMB as the currency unit, and $1 USD is about ¥6.30 RMB.

•
The optimal results in Scenario 1 Energies 2018, 11, x FOR PEER REVIEW 13 of 21

•
The optimal results in Scenario 1        From Figures 6-11, the following conclusions can be drawn by analyzing the optimal results of the leader-follower game between MEG and user: (1) In the game mode, the user has high sensitivity to electricity prices, and he can independently and flexibly responds to different prices released by MEG to adjust the load. In the lower electricity price periods (1)(2)(3)(4)(5)(6)(7)(20)(21)(22)(23)(24), the user gives priority to experience of energy usage, so the dissatisfaction drops to lowest. Because the expenditure is within acceptable range, no load needs to be adjusted.
In the higher electricity price periods (9)(10)(11)(12)(13)(14)(15)(16)(17), the electricity consumption is large, so the user gives priority to the expenditure. The user maximizes the reduction of non-essential load for reducing the expenditure, according to his own needs. In other periods (8,(18)(19), the electricity prices are at a medium level, the load need is lower, and the expenditure is within the acceptable range. Therefore, no adjustment is made. (2) The initial electricity price strategy of MEG is based on the historical electricity peak and valley periods data of the user. We can see that the electricity price strategy is closely related to the user load strategy at the game equilibrium point. In the lower load level periods (1)(2)(3)(4)(5)(6)(7)(20)(21)(22)(23)(24), MEG adjusts the price in order to prevent user to reduce the load, so that it could keep the lowest dissatisfaction and ensure the profit. In the higher load level periods (9)(10)(11)(12)(13)(14)(15)(16)(17), MEG raises the price within the user acceptable range in order to maximize the profit. In other periods (8,(18)(19), user loads are at a medium level, MEG chooses the highest price under the condition of ensuring the lowest user dissatisfaction to maximize the profit. (3) When comparing the electrical optimal outputs and production costs of MEG in Scenario 1 and Scenario 2, we can see that the grid mode has better economy than the isolated island mode. MEG stores electricity through the energy storage devices when the electricity price is low, discharges and sells electricity to the power grid during peak period. Therefore, MEG can flexibly adjusts the electrical outputs and increases its profit. (4) When comparing the game optimal results in Scenario 2 and Scenario 3, we can see that the production cost is reduced and the profit is increased, when MEG adopts the electricity price and heating price as the game strategies simultaneously. The user also adjusts his heating and cooling loads according to the heating price. As the heating load is relatively small, the user does not make adjustments and the experience of energy usage would not be affected. As the cooling load is relatively large, the user adjusts the cooling load within the acceptable range to reduce the expenditure. (5) As the user's actual loads in the game are adjusted to the RTP and heating price, the total amount of the user's loads are decreased from the original loads, while the adjusted loads are still in the acceptable range of user experience of energy usage. For MEG, the reduction in actual loads means lower production costs, less emissions, and lower penalty costs.
When compared with the method in this paper, the non-inferior solution front in the Pareto obtained by the traditional multi-objective optimization method is a set of multiple optimal solutions. The final decisions, without doubt, are heavily influenced by the personal biases of the decision-maker. However, game theory is an effective tool to solve the conflict between different stakeholders and reflects the independent decision-making process of the MEG side and the user side. It can not only obtain the unique NE solution, but also can avoid the subjective bias of the decision-maker, which can be well reflected the subjective initiative and individual rationality of participants. From this perspective, game theory has the advantages in information diversity and practical significance that multi-objective optimization does not have.

Conclusions
To facilitate the development of MEG, a general model and solution algorithm for the energy management of MEG under the leader-follower game theory framework are presented, to help the MEG obtain optimal economic profits and operational strategies, with considering user's expenditure and dissatisfaction. Under the proposed framework, it becomes feasible to enable system-level dispatch of multi-type energy resources, and the value of MEG can be appropriately reflected in the energy market. In order to balance the interaction between MEG and user in multi-type energy trading, a leader-follower game model is established in this paper, and the uniqueness of the Nash Equilibrium of the constructed game model has been proven. The user dissatisfaction function is constructed by leveraging the actual load and the demand load, and the fuzzy bi-objective algorithm is adopted to quantify the expenditure and user dissatisfaction. When compared with the mono-price energy market trading model, the leader-follower game model we proposed here not only considers the operational strategy in the MEG, but also takes into account the user's initiative and experience of energy usage in the multiple rounds of dynamic game optimization process. Case studies are taken in three scenarios, comprehensive simulations have been done, and the following conclusions are achieved: (1) Under the proposed leader-follower game framework, MEG has better consumption capacity of renewable energy, and achieves the best energy management in individual rational situations and obtains better economic profits. The MEG can improve the penetration of renewable energy resource, which will alleviate the power supply tension during peak hours and reduce the carbon emission. (2) In the leader-follower game mode, the user can well balance the energy-purchasing expenditure and the experience of energy usage, and then reach an equilibrium point. Therefore, the user can achieve economic optimum, while taking into account the experience of energy usage after the game. (3) When compared with Pareto front that was obtained by traditional multi-objective optimization methods, the proposed approach can not only obtain the unique NE solution, but also avoid the subjective bias of the decision-maker, reflect the subjective initiative and individual rationality of participants well.
In the near future, the proposed leader-follower game model will be extended to incorporate full competitions among MEGs, which will allow for the users to freely choose their energy providers. Moreover, deep analyses will be conducted on how the MEG's profit and user's payment will be affected by the intervention of other participants. Relevant researches, including cooperative game and leader-follower game involving more participants will become the focus in the future work.

Nomenclature
Indices t period m m-type device n n-type device k k-type device i i-type device Sets G set of participants Ω set of strategies Ω pr set of energy sales prices Ω l set of loads Parameters E in,max maximum charge rate per unit time E out,max maximum discharge rate per unit time J g natural gas price, ¥/m 3 J e electricity price of the grid, ¥/kWh K i maintenance cost factor for i-type device, ¥/kW p emi emission price of gas, ¥/m 3 w gasC CO 2 emission volume of gas consumed, m 3 /kWh w gasN NO x emission volume of gas consumed, m 3 /kWh w elcC CO 2 emission volume of electricity purchased from the grid, m 3 /kWh w elcN NO x emission volume of electricity purchased from the grid, m 3 /kWh δ penalty price, ¥/kWh α related to the magnitude of function R β preset value that the function R close to when the actual load increases to a certain extent λ related to the proportional elasticity of load adjustment d t user's demand load, kW a weight coefficient of objective P b weight coefficient of objective R ν maximum interruptible load ratio l t,un uninterruptible load, kW Functions I MEG profit of MEG, ¥ I user payment function of the user C t production cost of MEG, ¥ Emi t emission cost of MEG, ¥ ρ punishment fee, ¥ P energy-purchasing expenditure, ¥ R user dissatisfaction function Variables P grid,t exchange power between the electrical bus and the power grid in period t, kW P pv,t power output of PV in period t, kW P wt,t power output of WT in period t, kW P ce,t power output of internal combustion engine in period t, kW P ch,t electricity consumption while chiller working in period t, kW P ice,t electricity consumption while ice-storage air-conditioner making ice in period t, kW P melt,t electricity consumption while ice-storage air-conditioner melting ice in period t, kW I cc,t status value of ice-storage air-conditioner making ice in period t I cd,t status value of ice-storage air-conditioner melting ice in period t S bad,t discharging power of battery storage in period t, kW S bac,t charging power of battery storage in period t, kW I bad,t status value of battery storage discharging in period t I bac,t status value of battery storage charging in period t L e,t actual electrical load consumed by the user in period t, kW H ce,t heating output of heat exchanger in period t, kW H df,t heating output of direct-fired absorption chiller in period t, kW S hd,t heating discharging power of thermal storage in period t, kW S hc,t heating charging power of thermal storage in period t, kW I hd,t status value of thermal storage discharging in period t I hc,t status value of thermal storage charging in period t L h,t actual heating load consumed by the user in period t, kW C df,t cooling output of direct-fired absorption chiller in period t, kW C ch,t cooling output of chiller in period t, kW C ice,t cooling output of ice-storage air-conditioner in period t, kW L c,t actual cooling load consumed by the user in period t, kW E in,t energy storage rate in period t E out,t energy release rate in period t SOC t state of charging of the energy storage device in period t pr t energy sales prices of MEG in period t, ¥/kWh l t actual loads consumed by the user in period t, kW V g,t total gas volume consumed by MEG in period t, m 3 P i,t output power of the i-type device in period t, kW g re total generating capacity of renewable energy in period t,