Proﬁt Allocation of Hybrid Power System Planning in Energy Internet: A Cooperative Game Study

: The rapid development of Energy Internet (EI) has prompted numbers of generators to participate, leading to a hybrid power system. Hence, how to plan the hybrid power system and allocate its proﬁt becomes necessary. In this paper, the cooperative game theory is introduced to discuss this problem. We ﬁrst design the basic structure of EI, and point out the object of this study—coal power plant-wind farm-photovoltaic power station-energy storage provider (CWPE) alliance. Subsequently, average allocation strategy (AAS), capacity-based allocation strategy (CAS) and Shapley value allocation strategy (SAS) are proposed, and then the modiﬁed disruption propensity (MDP) index is constructed to judge the advantages and disadvantages of the three schemes. Thirdly, taking a certain area of A Province as an example, the proﬁts of CWPE under three strategies are calculated respectively. Finally, by analyzing individual rationality and collective rationality of cooperative game and the MDP index of the three proﬁt allocation schemes, we ﬁnd that SAS is the most stable.


Background of EI
With the depletion of fossil energy and the deterioration of the environment, the traditional energy system is facing severe challenges, such as the unsustainability of energy production and inefficient use of energy [1][2][3][4]. Therefore, building an energy supply and demand system featuring new energy such as wind energy and solar energy, and guiding energy industry restructuring and upgrading have become an important goal of energy revolution [5].
In the process of achieving the above energy revolution, EI came into being. The concept of EI was first proposed by Jeremy Rifkin in his book The Third Industrial Revolution in 2011 [6]. EI is an energy system covering the energy network and the transportation network. It is a high integration of information flow (IF) and energy flow (EF) [7][8][9][10][11]. There are three characteristics of EI [5].
(1) Integration EI can make the interconnection of a variety of energy networks, such as traditional gas network, power grid, heating network, oil network and transportation network, come true.

Basic Research on EI
The development of EI is in the initial stage, and its applications and pilots are mostly concentrated in developed countries such as the United States, Germany and Japan. China mostly focuses on the smart grid, micro-grid, distributed power and other related areas of EI. At present, the basic research on EI at home and abroad mainly focuses on the following two aspects [17]: (1) Empirical Study Developed countries such as the United States, Germany and Japan have already carried out pilot applications of EI. North Carolina State University of America proposed the future renewable electric energy delivery and management (FREEDM), which can guarantee the safety and stability of system operation when the capacity of distributed renewable and alternative energy generation is over 50% [18]. Germany launched the Energy Internet Technology Innovation Program in 2008. The information interconnection of the entire power system was emphasized from the perspective of integration of information communication technology (ICT) and the whole power system [19]. "Digital grid" was introduced to Japan in 2011, which can divide large synchronous grids into smaller segmented grids [20]. China's Energy Internet is still in its infancy. The first batch of "'Internet+' smart energy (Energy Internet) demonstration project" was approved in June 2017, which indicates that China's EI has entered a practical phase from the theoretical stage. (2) Design of EI Operation Mechanism Research on the EI operation mechanism design mainly focuses on concept and framework, technology, and business model. Dong et al. regarded EI as a network in which power systems, transportation systems, natural gas networks and information networks were closely coupled [21]. Sun et al. pointed out that the core concept of EI was openness [9]. Ma et al. believed that EI was the integration of Internet system and energy system so as to enhance the proportion of renewable energy and realize the effective interconnection and efficient utilization of diversified energy sources [22]. Yang et al. proposed that EI was a change of value creation method of energy system caused by the deep integration of ICT and energy system [23]. As for EI technology, Tian et al. pointed out that the key technologies of EI were new energy generation technology, energy storage, information technology, microgrid and demand response technology etc. [24]. Huang and Baliga developed an energy router device which combined EF with IF [25]. Cao et al. explored the EI implementation model with energy routers as the core switching devices [26]. In order to evaluate EI routers' transfer stability, Sun et al. proposed a novel energy function [27]. As for the business model of EI, based on the principle of information economics, Chen et al. [28], Liu et al. [29] and Zhou et al. [30] analyzed the development trend of the business model and market mechanism of EI.

Energy Resource Allocation in EI
Research on energy resource allocation in EI mostly concentrates on power generation and energy storage. Xu et al. established the game model of fossil energy and clean energy producers in EI, and evaluated the game results through the fuzzy comprehensive evaluation method [31]. Zhu et al. studied the coordination and allocation of energy storages according to different energy storages and various demands in EI [32]. Li et al. pointed out two application modes of energy storage in EI, namely, the wide area energy network and the local energy network [33]. Wang et al. proposed a liquid hydrogen with superconducting magnetic energy storage applied to future EI, and simulated the benchmark micro energy grid with distributed generators, electrical vehicle stations, smart loads and a liquid hydrogen with superconducting magnetic energy storage unit in the Matlab/Simulink environment [34]. A double-layer optimal scheduling model was raised in order to take the heat-to-electric ratio of a combined cooling, heating and power unit, energy storage life and real-time electricity price into consideration in a regional EI system [1]. Besides, Dong et al. proposed state estimation model for combined electricity and gas networks in EI [35].

Application of Game Theory in Hybrid Power System Planning
Domestic and foreign scholars used game theory extensively in hybrid power system planning. A hybrid modified game theory was presented in [36,37] to determine the generation expansion planning in a Pool market. Mei et al. discussed five different game scenarios, one non-cooperative game and four cooperative games with investors of wind power, photovoltaic generation and storage battery as game players [16]. After that, Mei et al. introduced game theory in the planning of a grid-connected hybrid power system which consists of wind turbines, photovoltaic panels, and storage batteries [15]. Wang et al. studied the cooperative game with transferable pay off for hybrid power system consisting of wind generations, photovoltaic generations and storage batteries [38]. Li et al. applied game optimization theory, which integrates costs, losses, and voltage index, for the distribution systems planning, including distributed generation [39]. Based on cooperative game theory, He et al. constructed a generation right trade model and analyzed the profits of all players before and after the cooperative game [40]. Yang et al. established a non-cooperative game model of regional power system with wind-thermal-pumped storage power [41]. Khare et al. applied HOMER, BIG BANG CRUNCH and game theory in a renewable energy system. They found that replacing conventional energy sources by solar-wind-fuel cell-hydrogen tank energy system was an effective method for distributing electric power [42]. In order to develop a reliable decision support system, a two-side multi-agent based modeling framework using a hybrid simulation approach of game theory and Particle Swarm Optimization was put forward in [43]. In [44], a conventional power system which is comprised of the utility company, the energy storage company, the microgrid, and electricity users was considered to solve the problem of coordinated management of renewable and traditional energy, and a three-stage Stackelberg game was proposed to formulate the energy management problem.
In general, the above literature solved some problems in EI, and dealt with some problems applying game theory in hybrid power system planning. However, these studies did not combine EI and the planning of hybrid power system. From this point of view, this paper discussed the hybrid power system of CWPE in EI. In addition, the CWPE were regarded as an alliance in this work, since cooperation was the core of EI. However, how to make a binding agreement that can reasonably allocate profits of the alliance so that all generation companies are willing to participate, becomes the focus of this research.

Structure of EI System
The integrated EF and IF is one of the basic components of EI. According to the flow direction of EF and IF, the basic structure of EI system is shown in Figure 1. game theory and Particle Swarm Optimization was put forward in [43]. In [44], a conventional power system which is comprised of the utility company, the energy storage company, the microgrid, and electricity users was considered to solve the problem of coordinated management of renewable and traditional energy, and a three-stage Stackelberg game was proposed to formulate the energy management problem. In general, the above literature solved some problems in EI, and dealt with some problems applying game theory in hybrid power system planning. However, these studies did not combine EI and the planning of hybrid power system. From this point of view, this paper discussed the hybrid power system of CWPE in EI. In addition, the CWPE were regarded as an alliance in this work, since cooperation was the core of EI. However, how to make a binding agreement that can reasonably allocate profits of the alliance so that all generation companies are willing to participate, becomes the focus of this research.

Structure of EI System
The integrated EF and IF is one of the basic components of EI. According to the flow direction of EF and IF, the basic structure of EI system is shown in Figure 1.
The system consists of four modules: centralized energy supply module (CESM), distributed energy supply module (DESM), information processing module (IPM), and multiple intelligent transport network module (MITNM). Each module interacts with each other through IF and EF [5,45,46]. This paper applies cooperative game for the planning of hybrid power system consisting of CWPE in CESM, that is, the module marked in red in Figure 1.

Cooperative Game Theory
If there are multiple decision makers, who are assumed to be rational, in a decision-making situation, and they pursue their respective goals, it is called a game. Game theory is divided into cooperative game theory and non-cooperative game theory [47]. This paper deals with the problem of hybrid power system planning in EI, which can reach an alliance in advance, so it belongs to the category of cooperative game and emphasizes on collective rationality. Moreover, cooperative game The system consists of four modules: centralized energy supply module (CESM), distributed energy supply module (DESM), information processing module (IPM), and multiple intelligent transport network module (MITNM). Each module interacts with each other through IF and EF [5,45,46]. This paper applies cooperative game for the planning of hybrid power system consisting of CWPE in CESM, that is, the module marked in red in Figure 1.

Cooperative Game Theory
If there are multiple decision makers, who are assumed to be rational, in a decision-making situation, and they pursue their respective goals, it is called a game. Game theory is divided into cooperative game theory and non-cooperative game theory [47]. This paper deals with the problem of hybrid power system planning in EI, which can reach an alliance in advance, so it belongs to the category of cooperative game and emphasizes on collective rationality. Moreover, cooperative game can be further divided into transferable payoff and non-transferable payoff. This paper studies the cooperative game of transferable pay off [48][49][50].
The basic elements of cooperative game are players and characteristic function, namely (N, v), where N = {1, 2, . . . , n} is the set of players; the numbers in set N are positive integers and n represents the number of players; v is the characteristic function of this alliance. S represents the alliance of players and S ⊆ N; and s is defined as the number of members in S. v s shows the payoffs of the cooperation among players in alliance S. Some definitions of cooperative game are as follows.
(1) Characteristic Function Suppose there is an alliance A, which can be made up of any number of players in set N, and A ⊆ N. Players in alliance A work together to maximize their mutual worth or payoff which corresponds to a real number. And obviously, this number is associated with A. So, a function v can be used to connect the two together. Therefore, the function v that reflects the mutual payoff or worth is called characteristic function, which indicates that the players in alliance A earn the total payoff or worth without the help of members in N\A (a set that doesn't include A) [51]. If A = S, v is equal to v s mentioned above. Namely, v s is the characteristic function of S.

(2) Marginal Contribution
The marginal contribution of participant i to alliance S is defined as: where v S/i shows the payoff of alliance with (s − 1) players in which player i is not included. (

3) Profit Allocation
For a transferable payoff (N, v), its allocation strategy (x 1 , x 2 , x 3 , . . . , x i , . . . , x n ), which shows profit allocation scheme of players' alliance, is a payoff vector, where x i represents the profit of player i.

(4) Individual Rationality
For a transferable payoff (N, v), when one player is assigned a higher profit than its own payoff when it works alone, which means that this player does not participant in cooperation, the allocation (x 1 , x 2 , x 3 , . . . , x i , . . . , x n ) is consistent with individual rationality, namely: where v i indicates the payoff or worth of player i itself when it does not take part in cooperation, namely, when it works alone.

(5) Collective Rationality
For a transferable payoff (N, v), when the sum of profit distributed by all players equals to the payoff or worth of alliance N, the allocation (x 1 , x 2 , x 3 , . . . , x i , . . . , x n ) is consistent with collective rationality, namely: where v N represents the whole payoff or worth of N.
When the allocation (x 1 , x 2 , x 3 , . . . , x i , . . . , x n ) of transferable payoff is in line with individual rationality and collective rationality, the allocation is an effective one.

Allocation Strategies
In this part, three allocation strategies are given for profit allocation of hybrid power system planning in EI, as described in detail below.
(1) AAS AAS means that the players distribute profits of the whole alliance on average. It can be calculated by Equation (4).
where x c , x w , x p , x e represent the profits of coal power plant (CPP), wind farm (WF), photovoltaic power station (PPS) and energy storage provider (ESP) respectively; v c,w,p,e is the payoff of the entire alliance.
(2) CAS CAS means allocating profits of the entire alliance based on the proportion of each player's capacity to the total capacity and it is computed by Equations (5)-(8). x x where C c , C w , C p , C e represent the capacity of CPP, WF, PPS and ESP separately.
(3) SAS Suppose N is the set of players participating in the cooperative game; S represents a cooperative alliance in N and s shows the number of players in alliance S. Assume that an alliance consists of players according to random order, then the probability of each order is p. p is calculated by Equation (9).
where n is the number of players in alliance N. Player i and the other (s − 1) players form alliance S, and there are (s − 1)!(n − s)! orders. Hence, the probability ∅(i) of alliance S can be calculated by Equation (10).
And the contribution r i of player i to alliance S can be calculated by Equation (11).
where v S is the payoff of alliance S; v s/i represents the payoff of alliance with (s − 1) players which doesn't include player i. Shapley value, which is regarded as the mean of players' contribution in one cooperative game, can be computed by Equation (12).  (N, v) is the profit distributed of player i.

MDP Index
Dermot Gately proposed disruption propensity (DP) index in 1974 [52]. DP is used to quantitatively analyze the attractiveness of allocation strategy to each player. It shows the ratio of other players' loss to player i s loss when player i disobeys cooperation. And the DP index d(i) of player i is defined as: where x i is the allocated profit of player i when it takes part in cooperation; v i is the profit of player i when it refuses to participate in cooperation; ∑ j∈{N\i} x j represents the allocated profit of other players when player i takes part in cooperation and v N/i shows the profit of other players when player i refuses to participate in cooperation. MDP index is a modification of DP index [38]. It is expressed as: D(i) means the proportion of per capita loss of other players to the loss of player i when player i refuses to participate in cooperation. The cooperation tendency analysis of D(i) is shown in Table 1. Table 1. Cooperation tendency analysis of modified disruption propensity (MDP).

D(i) Meaning Results
When D(i) ≥ 1 The non-cooperative behavior of participant i makes the average loss of other participants equal to or more than the loss of participant i itself.
Player i tends to reject the allocation strategy.
When D(i) < 1 The non-cooperative behavior of participant i leads to the average loss of other participants less than the loss of participant i itself.
Player i tends to accept the allocation strategy.

Case Study
Based on the existing cooperative game theory and related allocation strategies, the profit allocation of CWPE in EI is discussed below.

Profit Analysis
Take an area of A Province as an example. It is known that there is a CPP, one WF, one PPS and an ESP. A province lies in northwestern China, hence wind power has a better development trend than photovoltaic power. Therefore, wind power is more attractive to users than solar energy.
The raw data in this work are collected from the Energy Internet Research Institute (EIRI) of this place and they are divided into four parts. The details are as follows.
First of all, the installed capacity, electricity and profit etc. of the four players are obtained directly from CPP, WF, PPS and ESP according to their historical operating data.
Secondly, as one of the EI Demonstration Projects, the leaders in EIRI held the First Coordination Seminar on Energy Internet Generators on 15 July 2017, to promote cooperation among generators. And the CPP, WF and PPS were invited. After consultation, the three reached an agreement on testing power supply from 1 August to 9 September. The test subject was divided into four groups, namely a (CPP, WF), b (CPP, PPS), c (WF, PPS) and d (CPP, WF, PPS). The test time of each group was 10 days. When any two of them cooperated, the quantity of electricity that could not meet the demand of users was temporarily provided by DESM in EI. Besides, in order to meet the demand of EI [7], A province vigorously develops new energy generation, and requires CPP to transfer some generation rights to WF and PPS. Therefore, in groups a and b, the electricity of CPP is lower than before, but WF and PPS are higher.
Thirdly, during the test time from 1 August to 9 September, it was found that while DESM supplemented the insufficient electricity it was less economical and steady. On the contrary, the technology of energy storage was increasingly mature, the cost was decreasing and the development was rapid in China in recent years. As a result, energy storage was used as a way to enhance the stability of power grid and to supplement inadequate electricity. Subsequently, during the period from 21 September to 30 September, a new group e (CPP, WF, PPS, ESP) was tested and the data was obtained. From the data acquired, it is found that the electricity of each generator has increased. This could be the reason that energy storage improves the stability of power grid, leading to the increment of consumer consumption.
Last but not the least, in order to study the problem of profit distribution among the four, we invited 12 experts, of whom 2 were cost engineers from power grid companies, 2 were professors of economic and management from universities and 8 were technical managers from CPP, WF, PPS, ESP, to estimate the profits of other alliances. According to the operating data above and related data such as load demand, electricity price, natural wind speed, solar irradiant intensity, cost of energy storage, etc. and other EI demonstration projects [53], they got the profits. At the same time, the amount of electricity was calculated on the basis of the estimated profit.
In addition, the day-based data are converted into year-based ones so as to compare with pre-collaborative data.
The profit of each participant before and after cooperation is described below.
(1) Profit of each participant before cooperation Since energy storage provider itself cannot supply electricity for users, its profit is zero when it does not participate in cooperation. The specific profit of CWPE (expressed in 1, 2, 3, 4 respectively) before cooperation is shown in Table 2.
(2) Profit of each player after cooperation According to the collected data above, the amount of electricity consumption here is basically unchanged. So, the generating capacity remains unchanged when CPP, WF and PPS cooperate. It is expressed in Equations (15)- (18). However, the quantity of electricity of WF and PPS is increased and CPP's is decreased for CPP transfers some generation rights to WF and PPS.
where W c , W w and W p represent the generating capacity of CPP, WF and PPS respectively; W c,w , W c,p and W w,p are the generating capacity of alliance CPP and WF, CPP and PPS, WF and PPS separately; W w,p,c indicates the generating capacity of CPP, WF and PPS when the three cooperate. Energy storage has the characteristics of absorbing and releasing energy dynamically. The rational distribution of energy storage in EI can effectively compensate for the intermittency and volatility of renewable energy, enhance system stability and improve power quality. Although DESM may also solve the problem, the cost of its operation is much higher. Following are the detailed explanations.
(1) Improve stability of EI system The high-power throughput within a short time of energy storage provides technical support for restraining shocks caused by power imbalances during the working of EI system, guaranteeing the stable operation of the main power generation unit [54,55].
(2) Strengthen power quality Voltage sags are prevented, and power quality indexes such as voltage deviation and voltage fluctuation etc. are improved. Therefore, energy storage can be used as an uninterruptible power supply to provide continuous power for sensitive loads, so as to improve the power quality [56].
(3) Perfect running economy Using energy storage instead of coal power as the reserve capacity of renewable energy will reduce pollutant emissions and generating costs, and perfect the economy of power system operation [57].
Because energy storage can enhance grid stability, increase penetration of renewable energy resources, and improve the efficiency of energy systems [58][59][60][61], electricity consumption increases when there is an ESP. Therefore, the generating capacity of diverse generators rises with a different extent as well, and it can be shown in Equations (19)- (25).  Tables 3-13, of which data of Tables 3, 4, 6, 9 and 13 are real, and profits of Tables 5, 7, 8 and 10-12 are estimated by experts. For the convenience of calculation, the changes of profit of per kW·h generated by ESP are not taken into account. As can be seen from the tables, this cooperative game satisfies the following three points.
(3) Formula (28) is satisfied when any four participants constitute alliance. Table 3. Profits of coal power plant (CPP) and wind farm (WF).  Table 9. Profits of CPP, WF and PPS.  Table 11. Profits of CPP, PPS and ESP.

Allocation Results Analysis
Based on the three allocation schemes of AAS, CAS and SAS, and the real data and estimated data, we simulated the distributed profits of CWPE in EI. The MDP index value was calculated so as to analyze cooperative tendencies under different allocation strategies. The detailed calculation process is as follows.
(1) AAS According to Equation (4), the profit of each player in the alliance under AAS is obtained, and then the MDP index value is calculated according to Equation (14). The results are shown in Table 14. (2) CAS Based on Equations (5)-(8), the profit and MDP index value are computed of each player, as shown in Table 15. (3) SAS The alliance of CWPE is recorded as U = {1, 2, 3, 4}.
As can be seen in Table 2, when the players operate independently, their profits are v 1 = 22, 075.20 ten thousand Yuan, v 2 = 25, 000.00 ten thousand Yuan, v 3 = 16, 000.00 ten thousand Yuan, v 4 = 0. Based on Equations (9)-(12), the allocated profit of CWPS is obtained under SAS, and the computing process are as shown in Tables 16-19. Then, the MDP index value under SAS is calculated by Equation (20), as shown in Table 20.

Discussion
This part is discussed from two aspects to evaluate the effectiveness of the three allocation schemes. One is individual rationality and collective rationality, and the other one is MDP index.
(1) Are the individual rationality and collective rationality satisfied?
As can be seen from Table 20, the following conditions are met under SAS allocation strategy.
That is, the SAS allocation strategy meets the individual rationality and collective rationality of cooperative game.
Obviously, comparing values of profits in Tables 2, 14 and 15, it can be seen that Equation (30) is completely met under AAS strategy and CAS strategy. However, Equation (29) is partially met under these two situations. We can see from Tables 2 and 14, that the profit of WF before cooperation is more than the distributed profit under AAS, leading to WF's disobeying. Moreover, as can be seen in Tables 2 and 15, under the case of CAS, the profit of PPS before cooperation is more than the distributed profit as well, resulting in PPS's refusing. Therefore, the two schemes meet the collective rationality but don't meet individual rationality. So AAS and CAS are unstable from the perspective of individual rationality.
If AAS is adopted, the MDP of CPP will be 6.78, which is far more than 1. That means CPP tends to reject the cooperative game. As for WF, its MDP will be −3.55, which is a negative value, that means less than zero. The MDP of WF is computed in Equation (31) based on Equation (14). As can be seen in Equation (31), the reason why the value is negative is that the denominator (n − 1) × (x WF − v WF ) is negative and the numerator (x CPP + x PPS + x ESP − v CPP,PPS,ESP ) is positive. In the actual situation, this may be due to the fact that the profit of WF is much less than the lower limit under a stable allocation strategy, resulting the profits of CPP, PPS and ESP more than the upper limit, in which case, WF will not agree to accept the allocation. Combining the above two situations, in order to gain much larger number of profits, it is likely PPS and ESP may combine together. Therefore, AAS looks apparently fair. However, in fact, it ignores the status of participants, which leads to its unacceptability and instability.
When CAS is applied, the MDP of WF will be 4.50, more than 1, which means WF is inclined to reject the cooperation. Moreover, the MDP of PPS will be −2.68, which is a negative value as well. It is calculated in Equation (32). Similar to AAS, the denominator (n − 1) × (x PPS − v PPS ) is negative and the numerator (x CPP + x WF + x ESP − v CPP,WF,ESP ) is positive as we can see from Equation (32).
The reason for this case maybe the distributed profit of PPS is much less than the lower limit when it is in a stable allocation strategy, and the allocated profits of CPP, WF, ESP are more than the upper limit, causing the phenomenon that PPS will not accept the cooperation. So, the cooperation of CPP and ESP will be formed so as to get much more profits. Therefore, this strategy is unstable as well. While SAS is used, the MDP values of CPP, WF, PPS and ESP are 0.66, 0.56, 0.75 and 0.56 respectively, all of which is between 0 and 1. That means all of the players are willing to take part in the alliance under SAS. At the same time, it is also shown that SAS is the most stable strategy among these three ones.
In conclusion, from the perspective of individual rationality and collective rationality and MDP, comparing these three strategies, SAS is the most effective and stable one.

Conclusions
The rapid development of EI attracts so many clean energy and fossil energy producers that the planning and profit allocation of hybrid power system has become necessary for the further development of EI. First of all, the basic structure of EI was designed and then the object of this study was pointed out. Secondly, based on cooperative game theory, three profit allocation schemes for the alliance of CWPE in EI were established in this work, and the stability of the strategies was judged through individual rationality and collective rationality of cooperative game and MDP index. The main conclusions of this paper are as follows.
(1) The total profit of alliance will be increased if cooperative game is adopted. The allocation profit applying SAS not only satisfies individual rationality but also satisfies collective rationality. However, AAS and CAS just meet collective rationality and the individual rationality is not met. (2) MDP index confirms that SAS makes all stakeholders tend to participate in the cooperative game, and it is a much more stable strategy. This study presents some limitations. First, the data here were collected from EIRI or estimated by experts. So, this work did not take the capacity allocation of CWPE into consideration when designing the profit allocation, which is an important part in hybrid power system planning. Secondly, for the convenience of calculation, we did not consider the profit changes of per kW·h generated by ESP. Thirdly, Shapley itself exists some problems, such as excluding the cooperation risk of each player. So, future studies can be conducted to deal with the above-mentioned issues of hybrid power system planning in EI.